Asking a chatbot to confirm a proof

14 October 2026

When a student submits a proof, I can read the argument they handed in. I cannot see where the reasoning came from. In an induction problem, for example, a student might have written the base case, chosen the inductive hypothesis, and worked through the inductive step. They might also have asked a chatbot to supply that structure and copied the result. The finished page may look much the same.

Two routes to a proof. One student works it out; another exchanges messages with a chatbot. Both routes sit inside a dashed box marked not visible in the submission, and both lead to the same submitted proof.

That distinction has been bothering me in my discrete mathematics courses. A chatbot conversation gives me more to examine than a submission alone, but it raises another question: what was the student doing before the conversation began? I presented this work to the SPLICE LLM Working Group on 9 October 2026; the [talk materials](/talks/gilson-splice2026/) give the slide-by-slide account. Here I want to work through the question that has become harder for me to answer: when a student asks a chatbot to confirm a proof, what are they asking it to do?

Giving students something to check

The iterative perturbation process as a cycle: change the question, ask the model, grade the answer with the rubric, and change the question again. A question is kept once the model scores below 60% on three attempts, drawn as three failed attempts.

I taught NC State's CSC 226, Discrete Mathematics, in summers 2025 and 2026. I was the instructor of record for both sections. Both studies took place during the final third of the course, in the computer science applications unit. They involved different groups of students, and the second study added instruction that the first did not have.

I understand the instructor’s case for restricting chatbot use on assignments where students need to construct proofs themselves. I also want to understand the use that already happens. A blanket prohibition would leave much of that behavior unobserved. I have been designing assignments that permit chatbot use while giving students specific claims to inspect. The method is assignment perturbation: I start with a legitimate problem, test a model's answer against a rubric, change part of the problem, and test again ¹. The purpose is to make a plausible wrong answer possible and, importantly, checkable.

In the 2025 pilot, I used three kinds of perturbation. For spatial injection, I took a counting problem about ten salad-bar items and placed them in a 4–4–2 arrangement. The later questions impose conditions on rows, columns, and which items can be adjacent. A proposed answer now has positions the student can inspect. For axiomatic replacement, I changed the successor rule in a system modeled on the Peano axioms to S(n) = 2n + 3. I called the resulting system the Squemp numbers. The invented name gives the model no familiar mathematical label to lean on. A proof has to follow the axioms in the question, and an answer that silently borrows a rule from ordinary arithmetic is making an error we can locate. We call that failure axiom reversion. Contextual embedding introduces information specific to the task or course that a model must preserve rather than guess.

I tested each revised problem repeatedly against the target model and rubric. In the pilot, a question was kept once the model scored below 60% on three attempts. That threshold described the questions as tested with that model. It was never a claim that the questions would defeat every model.

What the 2025 conversations showed

The final-exam review was assigned to eighteen students. Eight used MerryQuery, a student-facing chatbot built by Benyamin Tabarsi in our lab, backed by Gemini 2.0 Flash. We collected 182 query–response pairs from those eight students and coded the requests using the DRIVE framework ². The codes describe what a student visibly asks the model to do. They do not establish how much reasoning happened before the prompt.

Consider three requests about the salad-bar problem. A student could paste the instructions and ask the model to arrange the items. That is a request for the task to be completed. They could ask why a proposed arrangement meets the row-ordering condition. That is a request for justification. Or they could point out that item 9 was placed above item 1 even though the given condition requires its row to be lower, and ask for a correction. These are illustrative prompts built from the assignment and coding categories, rather than quotations from student logs. The third request makes the student's checking visible in a way the first does not.

Independent component analysis of the coded conversations helped us examine two patterns of help-seeking. In cognitive scaffolding, students challenged an error and supplied constraints or reasoning that could move the conversation toward a valid solution. In GenAI dependence, students relied on generated answers without adequately examining unsupported steps. The underlying analysis also distinguished attention to contextual constraints from attention to formal reasoning, and requests for scaffolding from supplying it. Those are descriptions of the dialogue, not fixed kinds of student.

In this small sample, the patterns were associated with scores on the perturbed assignment. They did not divide students neatly into high-grade and low-grade groups, and the analysis did not predict final-exam scores. The pilot made several kinds of checking easier to see. It did not establish that encountering an incorrect chatbot answer taught students to check the next one, or improved their later exam performance ¹.

That last limitation changed the next study I wanted to run. I could design a question that exposed an error. I still needed to teach students what to do when they found one.

Teaching the checking

An exchange with a chatbot, left to right: the prompt, three steps, and the answer. The strong prompt, B1 to B4, attaches to the prompt. Collaborative prompting, C1 to C6, attaches to each step as a check. The hallucination autopsy, A0 to A6, attaches to the finished answer. In every case, the student writes the final proof.

In summer 2026, I taught a different section of CSC 226. Twenty-one students were enrolled and eight consented to contribute research data. During the same final-third unit, students completed a perturbed assignment, attended a class session on checking chatbot reasoning, and completed a second perturbed assignment. We also collected pre- and post-instruction AI-literacy surveys and invited students to discuss their interactions with me ³. The chatbot available in this course was newer than the one used in 2025, another reason not to treat differences between the two summers as an instructional effect.

I organized the session around three procedures. Each step had a subgoal label, a short description of what that step is supposed to accomplish. The idea comes from computing education work on making the jobs of individual steps explicit ⁴. I drew on the kinds of checking visible in the earlier conversations when deciding what students should practice.

Hallucination Autopsy (A0–A6) starts with a completed answer. Restate the claim, identify which rule each step uses, and locate the first step that no longer follows. Then try a small case to see exactly where the argument breaks. In class I used a deliberately flawed induction proof. The target sum was 2 + 5 + ··· + (3n − 1) = n(3n + 1)/2. The incorrect step substituted k(k + 1)/2 for the induction hypothesis at k. Testing n = 2 makes the error plain: the sum is 7, while the substituted formula gives 3. Students had to identify the faulty substitution, rather than simply announce that the proof was wrong.

Strong Prompting (B1–B4) asks the model to identify its assumptions and justify its steps against the conditions of the problem. The student then has an argument they can examine. I do not assume that a detailed prompt makes the answer correct. It may make an unsupported step easier to find, provided someone actually checks it.

Collaborative Prompting (C1–C6) puts that checking between the model's steps. Tell the model not to give the entire answer. Ask for one proposed step and its justification, examine it, challenge it if necessary, and only then continue. The student writes the final proof. A student can still follow that script without thinking, so the presence of a collaborative prompt is not enough to demonstrate learning.

Part of this is making the student comfortable correcting the system's interpretation. I used the word diversity as an example. In human-subjects research, I might be asking who is represented in a sample. In another computing context, I might be asking how different the data points are from one another. The model's answer can be coherent and still address the wrong meaning. The person using it has to notice that mismatch and clarify the question.

When the answer comes before the prompt

A line marks where the chat log begins. To its left, inside a dashed box, is the student's written work from before the chat, seen only in the written work and interviews. To its right is the chat log, opening with the prompt "Is this proof correct?" and the chatbot's reply.

The 2026 interaction logs are still being coded. I cannot yet say whether the lesson changed how students checked the model. The two interviews I conducted after the final exam have, however, changed what I want to look for.

Both students described trying problems on their own before opening a chatbot. P2 said, “I'll try and solve it myself, and then kind of use it as a confirmation.” P1 put it this way: “I think I've got it … Let me just compare it to the AI so I can be 100% certain.”

A prompt that says Is this proof correct? could follow an hour of handwritten work. The same prompt could come from a student who has not tried the problem. Those conversations might start with identical words. I was initially looking for evidence of dependence or scaffolding in what the student asked. Now I want to know when they asked it, and what they had already written.

The written submissions, interviews, and whatever timing information the files preserve may help. I want to compare a student's first chatbot request with the state of their proof at that point, where that can be reconstructed. Pencil marks, revisions, and PDF edit times are possibilities, not reliable measures I have established. A chat log begins when the chat begins. It cannot supply the missing part of the work by itself.

The interviews gave me a second measurement problem. P1 recalled the procedures without necessarily recalling or deliberately applying their labels: “Did I apply them explicitly? No, but they were in my head … Is it hallucinating, or is my prompt strong enough?” P2 described correcting the chatbot as “stuff that I haven't necessarily thought as much about before.” Remembering a label and doing the check it names are different things. I need evidence of the checking action, not just a student's memory of the lesson.

P2 complicated my interpretation again. They had been “in my head a lot about whether or not I'm using AI correctly,” and said the session “felt like confirmation” that they were. Perhaps some students have already developed habits of questioning a chatbot through other online exchanges, and the lesson gives those habits names. I do not have evidence for where those habits came from. In this interview, the student's account was as much about reassurance as it was about learning a new procedure.

The part of a proof I worry about losing

P1 reached a different conclusion. They said relying on the chatbot for certainty “hurt me in other areas,” and that “it's hard to learn things if you can't make mistakes.” They reported deciding to use it less. That is one student's reflection, not evidence that my lesson caused a change in behavior.

What interests me is the decision to trust an answer. When I teach, I sometimes have to answer a question at the podium that is not settled by the textbook in front of me. I have to work through it and decide whether I can stand behind the reasoning. I learned some of that judgment by teaching and by being wrong in front of other people. I suspect students need opportunities to develop it within a subject, too.

I used a saw as an analogy in the talk. You can explain how it works to someone in a classroom. Their judgment changes when they are standing in front of the blade and have to decide where their hands belong. With a proof, the corresponding moment is deciding that a step follows because you have checked it, without waiting for a confident second voice to agree. Asking a chatbot for reassurance might sometimes help a student find a mistake. It might also let them avoid that decision. The interviews do not tell me how often either happens.

There are limits to what I can infer from those interviews. Only two students participated. They spoke to me after the final exam, and I was both their instructor and the researcher. I cannot generalize their accounts to the whole section or claim that the procedures caused the choices they described.

What I still need to see

I left the SPLICE talk with two questions. When a student asks a chatbot to confirm a proof, what evidence would distinguish checking an answer they already built from handing the reasoning over to the model? And when I teach a labeled checking procedure, should I measure whether students remember the labels, whether they perform the checks, or both?

The 2025 pilot gave me a way to observe some of the reasoning students bring into a chatbot conversation. The two interviews from 2026 exposed a boundary of that observation. I can see a student challenge an incorrect step once the conversation starts. I often cannot see the work that led them to ask the question, or whether they would have trusted their answer without the chatbot. I need the coded conversations alongside the submitted proofs to make a narrower claim about what students are actually doing.

I built the Perturbation Workbench after the 2026 class, so other instructors can test and revise their own problems. It was not part of either classroom intervention.

References

Tagged ai-literacy · discrete-mathematics · generative-ai

I am looking for a teaching professor position. What I am looking for

Back to the blog