Adversarial Assignment Perturbation: Effects on Help-Seeking Behaviors in Student Generative AI Chatbot Use
Published conference paper
Sam Gilson; Benyamin Tabarsi; Tiffany Barnes
Artificial Intelligence in Education (AIED 2026), LNAI 16585, pp. 440–449
Citation year: 2027.
First published online: 2026-06-27.
Summary
How do students respond when a chatbot gives them an incorrect solution? We studied the chat logs of eight students who used an optional chatbot in an undergraduate discrete mathematics course. The analysis identified patterns of cognitive scaffolding and GenAI dependence, which were associated with assignment grades within this small sample.
What this study can tell us
This exploratory case study describes interaction patterns in one course. The association with assignment grades does not establish that either pattern caused learning gains.
Abstract
The widespread adoption of generative AI (GenAI) tools in educational settings necessitates methods to differentiate student-AI interactions indicative of productive help-seeking from shallow cognitive outsourcing. This work conducts a mixed-methods analysis of GenAI queries and learning in an undergraduate discrete mathematics course. Participants were provided with an optional, GenAI-powered chatbot to aid them while completing a One-Shot GenAI-Resistant assignment, generated via an iterative perturbation process. We conducted a case study of 8 students who used the chatbot, and found two disparate engagement patterns: cognitive scaffolding, recognizing GenAI chatbot had hallucinated and provided additional context to leverage the chatbot to discover the correct answer, and GenAI dependence behaviors, which failed to critically evaluate hallucinated GenAI chatbot answers. Within this small sample, we found a significant correlation between cognitive scaffolding and GenAI dependent engagement patterns and the perturbed assignment grades. These findings demonstrate that assignment perturbation, combined with chat log analysis, can be used to differentiate deep and shallow GenAI engagement strategies.
The abstract from the accepted manuscript, with one published typo corrected below.
- abstract: recongizing -> recognizing
Concepts used in this work
Iterative perturbation process · One-Shot GenAI Resistance · Cognitive scaffolding · GenAI dependence · DRIVE framework
Citation
@inproceedings{gilson2026perturbation,
title = {{Adversarial Assignment Perturbation: Effects on Help-Seeking Behaviors in Student Generative AI Chatbot Use}},
author = {Sam Gilson and Benyamin Tabarsi and Tiffany Barnes},
year = {2027},
booktitle = {Artificial Intelligence in Education. AIED 2026},
pages = {440--449},
series = {Lecture Notes in Computer Science},
volume = {16585},
note = {First online 27 June 2026; publisher citation year 2027},
doi = {10.1007/978-3-032-29770-9_48},
url = {https://sgilson7.github.io/papers/gilson2026perturbation/}
}