Sam Gilson
I’m a PhD student in computer science in the Game2Learn lab at North Carolina State University, advised by Dr. Tiffany Barnes. I study how undergraduates use generative AI while learning, with a particular interest in discrete mathematics and proof writing.
When a student asks a chatbot for help, I want to understand what happens next. Do they check the reasoning, ask for a justification, or accept an answer that looks plausible? A submitted solution may leave that process unclear. My research uses assignment design and student-chatbot conversations to examine it more closely.
Below are my teaching and the 2026 teaching award, the research that comes out of those courses, and the work I do with collaborators on logic proof tutoring and on how software professionals use LLMs.
I am looking for a teaching professor position. What I am looking for · Teaching statement
Teaching and software
I teach CSC 151, Java Programming, at Central Carolina Community College. At NC State, I have taught CSC 226, Discrete Mathematics, and CSC 216, Software Development Fundamentals. In 2026, I received Central Carolina Community College’s Excellence in Teaching Award for STEM.
Some research tasks also led me to build software. The Perturbation Workbench supports assignment testing, and PDF Redactor helps prepare PDFs for review and sharing. Both can be used in a browser.
Research
The work below follows three connected questions: how students seek help, how assignments make that interaction visible, and how we can teach students to evaluate the help they receive.
Adversarial Assignment Perturbation: Effects on Help-Seeking Behaviors in Student Generative AI Chatbot Use
Sam Gilson; Benyamin Tabarsi; Tiffany Barnes
Published conference paper · Artificial Intelligence in Education (AIED 2026), LNAI 16585, pp. 440–449 · Citation year 2027 · First online 2026-06-27
How do students respond when a chatbot gives them an incorrect solution? We studied the chat logs of eight students who used an optional chatbot in an undergraduate discrete mathematics course. The analysis identified patterns of cognitive scaffolding and GenAI dependence, which were associated with assignment grades within this small sample.
DOI 10.1007/978-3-032-29770-9_48
Designing AI-Resistant Assignments via Iterative Perturbation to Promote Interactive Learning
Sam Gilson
Published poster · Proceedings of the 57th ACM Technical Symposium on Computer Science Education V.2 (SIGCSE TS 2026), p. 1745 · Citation year 2026
This poster presents iterative perturbation as an approach to assignment design: test a question with a target chatbot, revise the question, and evaluate the new response against an explicit rubric. The design challenge is to make the chatbot’s limitations observable while keeping the mathematical task appropriate for students.
How Do Undergraduates Interested in Computer Science Use Online, On-Demand CS Coursework?
Sam Gilson; Tiffany Barnes; Mahzabin Tamanna; Saminur Islam
Published conference paper · Proceedings of the Twelfth ACM Conference on Learning @ Scale (L@S ’25), pp. 271–275 · Citation year 2025
Why do undergraduates take computing courses outside their degree programs, and why do they stop? We surveyed 51 students enrolled in or interested in computer science and interviewed four of them. Participants described using online, on-demand coursework for interview preparation and resume skills. Reported reasons for stopping included losing trial access and turning to LLMs such as ChatGPT.
Perturbed Assignments and Subgoal-Labeled AI Literacy Instruction in Undergraduate Discrete Mathematics
Sam Gilson
Preprint; submitted to the SIGCSE TS 2027 Student Research Competition · Preprint · Citation year 2026
This ongoing classroom study places an AI literacy session between two perturbed assignments in undergraduate discrete mathematics. Eight students consented to participate in a section of 21. The study collected chat logs, surveys, and two cued office-hours sessions. The preprint describes the assignment design and three teaching procedures; analysis of the chat logs was still in progress when it was written.
Free-tier Large Language Models on the MATH Benchmark: Implications for AI Literacy Instruction
Sam Gilson
Manuscript under review; submitted to the AERA 2027 annual meeting · Manuscript under review
This manuscript examines the mathematical performance of models available through free access tiers and considers what capability evidence could contribute to AI literacy instruction. The instructional question is how students might use evidence about a tool’s strengths and limitations when deciding what to check.
Work with collaborators
I also work with colleagues on feedback in logic proof tutoring and on how software professionals use LLMs.
LLMs' reshaping of people, processes, products, and society in software development: a qualitative exploration with early adopters
Benyamin Tabarsi; Heidi Reichert; Sam Gilson; Ally Limke; Sandeep Kuttal; Tiffany Barnes
Published journal article · Empirical Software Engineering, volume 32, article 21 · Citation year 2027 · First online 2026-08-07
What changed when software professionals began using LLMs in their daily work? This qualitative study analyzes interviews with 16 early adopters who used these tools in early to mid-2023. Their accounts describe changes in people, processes, products, and society, including the effort involved in evaluating and integrating generated code. Participants described productivity benefits alongside the work of checking generated output.
DOI 10.1007/s10664-026-10917-0
Confirming Correct, Missing the Rest: LLM Tutoring Agents Struggle Where Feedback Matters Most
Tahreem Yasir; Wenbo Li; Sam Gilson; Sutapa Tithi; Xiaoyi Tian; Tiffany Barnes
Published workshop paper · Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026), pp. 819–840 · Citation year 2026
Can LLM tutoring agents recognize when students need feedback? This study evaluates seven LLM feedback agents on 10,836 solution–feedback pairs in propositional logic. The agents handled optimal steps more reliably than incorrect or valid but suboptimal steps, and accurate diagnosis did not consistently produce actionable feedback.
When Verification Hurts: Asymmetric Effects of Multi-Agent Feedback in Logic Proof Tutoring
Tahreem Yasir; Sutapa Dey Tithi; Benyamin Tabarsi; Dmitri Droujkov; Sam Gilson; Yasitha Rajapaksha; Xiaoyi Tian; Arun Ramesh; DongKuan Xu; Tiffany Barnes
arXiv preprint · arXiv:2603.27076 · Citation year 2026 · First online 2026-03-28
Does adding a verification agent improve feedback in logic proof tutoring? This preprint compares feedback configurations on 516 tutoring states. It examines whether verification affects feedback on correct and incorrect student steps differently. In the evaluated pipelines, verification helped when upstream feedback was error-prone but reduced performance when that feedback was already reliable.
Find this work elsewhere
ORCID: 0009-0002-2394-8649 · Google Scholar · GitHub · Game2Learn lab · Semantic Scholar · OpenAlex