Blog
Asking a chatbot to confirm a proof
When a student submits a proof, I can read the argument they handed in. I cannot see where the reasoning came from. In an induction problem, for example, a student might have written the base case, chosen the inductive hypothesis, and worked through the inductive step. They might also have asked a chatbot to supply that structure and copied the result. The finished page may look much the same.
Giving students something to check
I taught NC State's CSC 226, Discrete Mathematics, in summers 2025 and 2026. I was the instructor of record for both sections. Both studies took place during the final third of the course, in the computer science applications unit. They involved different groups of students, and the second study added instruction that the first did not have.
I understand the instructor’s case for restricting chatbot use on assignments where students need to construct proofs themselves. I also want to understand the use that already happens. A blanket prohibition would leave much of that behavior unobserved. I have been designing assignments that permit chatbot use while giving students specific claims to inspect. The method is assignment perturbation: I start with a legitimate problem, test a model's answer against a rubric, change part of the problem, and test again 1. The purpose is to make a plausible wrong answer possible and, importantly, checkable.
In the 2025 pilot, I used three kinds of perturbation. For spatial injection, I took a counting problem about ten salad-bar items and placed them in a 4–4–2 arrangement. The later questions impose conditions on rows, columns, and which items can be adjacent. A proposed answer now has positions the student can inspect. For axiomatic replacement, I changed the successor rule in a system modeled on the Peano axioms to S(n) = 2n + 3. I called the resulting system the Squemp numbers. The invented name gives the model no familiar mathematical label to lean on. A proof has to follow the axioms in the question, and an answer that silently borrows a rule from ordinary arithmetic is making an error we can locate. We call that failure axiom reversion. Contextual embedding introduces information specific to the task or course that a model must preserve rather than guess.
I tested each revised problem repeatedly against the target model and rubric. In the pilot, a question was kept once the model scored below 60% on three attempts. That threshold described the questions as tested with that model. It was never a claim that the questions would defeat every model.
What the 2025 conversations showed
The final-exam review was assigned to eighteen students. Eight used MerryQuery, a student-facing chatbot built by Benyamin Tabarsi in our lab, backed by Gemini 2.0 Flash. We collected 182 query–response pairs from those eight students and coded the requests using the DRIVE framework 2. The codes describe what a student visibly asks the model to do. They do not establish how much reasoning happened before the prompt.
Consider three requests about the salad-bar problem. A student could paste the instructions and ask the model to arrange the items. That is a request for the task to be completed. They could ask why a proposed arrangement meets the row-ordering condition. That is a request for justification. Or they could point out that item 9 was placed above item 1 even though the given condition requires its row to be lower, and ask for a correction. These are illustrative prompts built from the assignment and coding categories, rather than quotations from student logs. The third request makes the student's checking visible in a way the first does not.
Independent component analysis of the coded conversations helped us examine two patterns of help-seeking. In cognitive scaffolding, students challenged an error and supplied constraints or reasoning that could move the conversation toward a valid solution. In GenAI dependence, students relied on generated answers without adequately examining unsupported steps. The underlying analysis also distinguished attention to contextual constraints from attention to formal reasoning, and requests for scaffolding from supplying it. Those are descriptions of the dialogue, not fixed kinds of student.
In this small sample, the patterns were associated with scores on the perturbed assignment. They did not divide students neatly into high-grade and low-grade groups, and the analysis did not predict final-exam scores. The pilot made several kinds of checking easier to see. It did not establish that encountering an incorrect chatbot answer taught students to check the next one, or improved their later exam performance 1.
That last limitation changed the next study I wanted to run. I could design a question that exposed an error. I still needed to teach students what to do when they found one.
Teaching the checking
In summer 2026, I taught a different section of CSC 226. Twenty-one students were enrolled and eight consented to contribute research data. During the same final-third unit, students completed a perturbed assignment, attended a class session on checking chatbot reasoning, and completed a second perturbed assignment. We also collected pre- and post-instruction AI-literacy surveys and invited students to discuss their interactions with me 3. The chatbot available in this course was newer than the one used in 2025, another reason not to treat differences between the two summers as an instructional effect.
I organized the session around three procedures. Each step had a subgoal label, a short description of what that step is supposed to accomplish. The idea comes from computing education work on making the jobs of individual steps explicit 4. I drew on the kinds of checking visible in the earlier conversations when deciding what students should practice.
Hallucination Autopsy (A0–A6) starts with a completed answer. Restate the claim, identify which rule each step uses, and locate the first step that no longer follows. Then try a small case to see exactly where the argument breaks. In class I used a deliberately flawed induction proof. The target sum was 2 + 5 + ··· + (3n − 1) = n(3n + 1)/2. The incorrect step substituted k(k + 1)/2 for the induction hypothesis at k. Testing n = 2 makes the error plain: the sum is 7, while the substituted formula gives 3. Students had to identify the faulty substitution, rather than simply announce that the proof was wrong.
Strong Prompting (B1–B4) asks the model to identify its assumptions and justify its steps against the conditions of the problem. The student then has an argument they can examine. I do not assume that a detailed prompt makes the answer correct. It may make an unsupported step easier to find, provided someone actually checks it.
Collaborative Prompting (C1–C6) puts that checking between the model's steps. Tell the model not to give the entire answer. Ask for one proposed step and its justification, examine it, challenge it if necessary, and only then continue. The student writes the final proof. A student can still follow that script without thinking, so the presence of a collaborative prompt is not enough to demonstrate learning.
Part of this is making the student comfortable correcting the system's interpretation. I used the word diversity as an example. In human-subjects research, I might be asking who is represented in a sample. In another computing context, I might be asking how different the data points are from one another. The model's answer can be coherent and still address the wrong meaning. The person using it has to notice that mismatch and clarify the question.
When the answer comes before the prompt
The 2026 interaction logs are still being coded. I cannot yet say whether the lesson changed how students checked the model. The two interviews I conducted after the final exam have, however, changed what I want to look for.
Both students described trying problems on their own before opening a chatbot. P2 said, “I'll try and solve it myself, and then kind of use it as a confirmation.” P1 put it this way: “I think I've got it … Let me just compare it to the AI so I can be 100% certain.”
A prompt that says Is this proof correct? could follow an hour of handwritten work. The same prompt could come from a student who has not tried the problem. Those conversations might start with identical words. I was initially looking for evidence of dependence or scaffolding in what the student asked. Now I want to know when they asked it, and what they had already written.
The written submissions, interviews, and whatever timing information the files preserve may help. I want to compare a student's first chatbot request with the state of their proof at that point, where that can be reconstructed. Pencil marks, revisions, and PDF edit times are possibilities, not reliable measures I have established. A chat log begins when the chat begins. It cannot supply the missing part of the work by itself.
The interviews gave me a second measurement problem. P1 recalled the procedures without necessarily recalling or deliberately applying their labels: “Did I apply them explicitly? No, but they were in my head … Is it hallucinating, or is my prompt strong enough?” P2 described correcting the chatbot as “stuff that I haven't necessarily thought as much about before.” Remembering a label and doing the check it names are different things. I need evidence of the checking action, not just a student's memory of the lesson.
P2 complicated my interpretation again. They had been “in my head a lot about whether or not I'm using AI correctly,” and said the session “felt like confirmation” that they were. Perhaps some students have already developed habits of questioning a chatbot through other online exchanges, and the lesson gives those habits names. I do not have evidence for where those habits came from. In this interview, the student's account was as much about reassurance as it was about learning a new procedure.
The part of a proof I worry about losing
P1 reached a different conclusion. They said relying on the chatbot for certainty “hurt me in other areas,” and that “it's hard to learn things if you can't make mistakes.” They reported deciding to use it less. That is one student's reflection, not evidence that my lesson caused a change in behavior.
What interests me is the decision to trust an answer. When I teach, I sometimes have to answer a question at the podium that is not settled by the textbook in front of me. I have to work through it and decide whether I can stand behind the reasoning. I learned some of that judgment by teaching and by being wrong in front of other people. I suspect students need opportunities to develop it within a subject, too.
I used a saw as an analogy in the talk. You can explain how it works to someone in a classroom. Their judgment changes when they are standing in front of the blade and have to decide where their hands belong. With a proof, the corresponding moment is deciding that a step follows because you have checked it, without waiting for a confident second voice to agree. Asking a chatbot for reassurance might sometimes help a student find a mistake. It might also let them avoid that decision. The interviews do not tell me how often either happens.
There are limits to what I can infer from those interviews. Only two students participated. They spoke to me after the final exam, and I was both their instructor and the researcher. I cannot generalize their accounts to the whole section or claim that the procedures caused the choices they described.
What I still need to see
I left the SPLICE talk with two questions. When a student asks a chatbot to confirm a proof, what evidence would distinguish checking an answer they already built from handing the reasoning over to the model? And when I teach a labeled checking procedure, should I measure whether students remember the labels, whether they perform the checks, or both?
The 2025 pilot gave me a way to observe some of the reasoning students bring into a chatbot conversation. The two interviews from 2026 exposed a boundary of that observation. I can see a student challenge an incorrect step once the conversation starts. I often cannot see the work that led them to ask the question, or whether they would have trusted their answer without the chatbot. I need the coded conversations alongside the submitted proofs to make a narrower claim about what students are actually doing.
I built the Perturbation Workbench after the 2026 class, so other instructors can test and revise their own problems. It was not part of either classroom intervention.
Related concepts: Iterative perturbation process · Three-Attempt Consistency Protocol · Spatial injection · Axiomatic replacement · Axiom reversion · Contextual embedding · DRIVE framework · Cognitive scaffolding · GenAI dependence · Subgoal label · Hallucination autopsy procedure · Strong prompt structure · Collaborative prompting.
References
- Gilson, Tabarsi, and Barnes (2027; first online 2026). Adversarial Assignment Perturbation: Effects on Help-Seeking Behaviors in Student Generative AI Chatbot Use.
- Oliveira, Zednik, Bombaerts, Sadowski, and Conijn (2025). Assessing Students’ DRIVE: A Framework to Evaluate Learning through Interactions with Generative AI. Computers and Education: Artificial Intelligence 9, 100497.
- Gilson (2026). Perturbed Assignments and Subgoal-Labeled AI Literacy Instruction in Undergraduate Discrete Mathematics. Preprint; submitted to the SIGCSE TS 2027 Student Research Competition.
- Morrison, Margulieux, and Guzdial (2015). Subgoals, Context, and Worked Examples in Learning Computing Problem Solving. ICER 2015, pp. 21–29.
Tagged ai-literacy · discrete-mathematics · generative-ai
Learning to learn, and what games are good for
Riot removed the ability to download replays from arbitrary League of Legends accounts. That matters to third-party match-history and analysis sites that depend on those files. As I understand it, a replay records the actions taken in a match as simulation data, rather than saving a video. That makes it possible to examine what happened in the game without trying to extract the same information from footage. I do not know how many of these services depended on replay downloads, but losing access creates an obvious problem for the ones that did.
That got me thinking about the arguments around AI, changing work, and the ability to learn a new skill. The part I keep coming back to is learning how to learn: understanding how you build knowledge, how you recognize progress, and how you might use that process again. What follows is a hypothesis I am trying to work out. The examples are mostly my own, and they cannot establish a general theory of learning.
The skill you need before the meta-skill
I suspect most people have some subject they take to more readily than others. I do not know why that happens. It might have a great deal to do with what someone encountered while growing up, what interested them, or what they had a reason to practice. I have not tested any of those explanations.
My stronger hypothesis is that learning how to learn is extraordinarily difficult until you have reached some level of mastery in one thing. To examine a learning process, you need a process you can look back on. You need to have experienced the difference between understanding something poorly and understanding it well enough to recognize a mistake, make a judgment, or work out something new from what you already know.
The difficult part is recognizing that change in yourself. We can study metacognition in structured settings, but ordinary life does not necessarily prompt someone to ask how they learned a skill. I suspect that part of the appeal of self-help books is that they offer a framework for noticing and evaluating progress. That is an interpretation of why people use those books, not a finding I have established.
Once you are good enough at something to see when a piece of work looks wrong, you have a way to examine your own improvement. You can look back and ask what helped, where you got stuck, and whether any part of that process would help you learn something else.
This reminds me of how I understood my parents when I was a child. I could barely tie my shoes, and either of them could tie a knot that looked perfect to me. Then my mother could turn around and tie a different kind of knot to hold up a parasol. I remember wondering how adults seemed to know how to do everything. I now associate part of that experience with wisdom: having enough examples of learning and problem-solving to talk about the process itself. Advice can sound obvious or even a little ridiculous when you have not yet had the experience that lets you evaluate it.
What games are actually good at
Games have given me a lot of those smaller experiences. Part of why I play is the feeling of getting better at something. Another part is social. Friends I grew up with have moved away, and getting into Discord together while we play a game can feel surprisingly close to being in the same room. In one tank game, we are all represented by these little vehicles, moving around and talking to each other. The game gives us something to do together while we talk.
For learning, the property that interests me is the size of the experience. A game gives you a set of mechanics to understand and practice. You can become meaningfully better over tens or hundreds of hours, then look back at how you got there. That is a much shorter arc than many skills people spend years building.
I have played about fifty hours of Factorio. At the beginning, I barely thought about ratios or belt speeds. I would build a machine producing blue science and wait for it to make enough to unlock the next research level. Now I think about throughput before laying things out. If I am building engine units, I start with the batch I want to produce, work backward through steel and gears, and plan the belts around the later science production that will need those engines. I described twelve engine units as the smallest useful batch for one layout I was considering. The point is how differently I approach the problem now, not whether twelve is a universal production ratio.
Fifty hours is very little time compared with how long I have spent learning to program. I have spent roughly ten years of my adult life coding, including time working at Microsoft, and I still compare myself with people who could take a complicated request and produce an elegant design almost as they typed. I remember seeing colleagues build structures that passed function pointers around and called them at the right moment, without holding on to all the intermediate data. Watching how quickly they reached those designs made me aware of how much I still had to learn. After fifty hours of Factorio, by comparison, I could already look back at a complete shift in how I built a factory.
That contrast is what makes games interesting to me. You can take a bounded experience of learning, examine it, and ask what you would do differently the next time. I would like to know whether that kind of reflection helps people learn another game or another skill more efficiently. My own experience gives me a reason to ask the question, though it does not answer it.
I also have a story that initially felt like evidence of a specific skill transferring between games and the physical world. I played a great deal of Rocket League in high school. By my senior year, the FIRST Robotics team held tryouts for the person who would drive the robot. The adults running the team had set up an obstacle course. I remember other students taking around five or six minutes, with some runs lasting longer. My first attempt took about a minute and ten seconds. They asked me to drive it again in reverse, and I completed the attempts quickly enough to be chosen as the driver. We went to Worlds that year. Our robot was very well built, which mattered at least as much as who was driving it.
The experience felt familiar because Rocket League had given me practice steering a vehicle from camera angles that do not match its direction of travel. You might be flying backward toward your own goal, adjusting an input while the ball and your car are moving in different directions. In robotics, the robot could be a long way away, partly out of view, and moving around a corner. I did not need to see it from behind to work out the controls. At the time, I took that as a transfer from playing Rocket League to driving the robot.
The paper that complicates the story I just told
Yasitha Rajapaksha, a colleague of mine at NC State, published a study at CHI 2026 with John Thomas Bacher and Tiffany Barnes on how game features and spatial skills interact with performance and perceived difficulty in a block-based 3D programming puzzle game 1. Players program a bot to move a block through an environment while trying to minimize its movements. The activity was designed around an idea from programming: you first work through a repeated action, then identify the part that stays the same and the part a loop can generalize.
As I recall the result, performance in the game was strongly associated with self-reported prior video game experience and performance on a spatial reasoning test, and how much the game itself taught was difficult to establish. That sentence is my memory of the findings rather than a reading of the paper. The citation is there so it can be checked against what the paper actually reports.
This is the difficulty with claiming that a game teaches a specific skill. People arrive with different experience and abilities, and those differences can explain some of what looks like learning during play. A carefully designed game can still help someone learn. The challenge is separating that effect from what the person already brought to it.
That same problem applies to my robotics story. I had played Rocket League for a long time, but I may also have had strong spatial reasoning before I played it. I cannot go back and test what I would have done at the tryout without those hours in the game. The similarity between the two tasks tells me why the robot felt familiar. It does not establish which parts of my performance Rocket League caused.
I still suspect games are useful for learning how to learn. They give you bounded experiences in which you can notice yourself improving, then ask what changed. What I am less comfortable claiming now is that a particular game taught a skill simply because I later used something that felt similar. That is the distinction I would want a study to test.
References
Tagged learning-sciences · games · metacognition
Why people walk looking at their phones
I started wondering about this while walking from the parking deck to a lecture hall last summer, when I was instructor of record for discrete mathematics. I was reading class email and reviewing my notes as I walked. I was also an hour early, with an empty lecture hall waiting and nothing that needed doing in the next ten minutes. I caught myself at it and could not explain it. I was almost certainly doing a worse job of reading than I would have done sitting down, because part of my attention was on where my feet were going.
What follows is a line of thinking, not a result. I have not checked it against the literature, and parts of it may already be answered or already contradicted there.
Where ICAP stops being the tool I need
Chi and Wylie's ICAP framework sorts engagement into passive, active, constructive and interactive, and says the learning gains rise along that order 1. Scrolling an app is close to pure passive. You are taking things in without putting anything back.
By that account, scrolling while walking is the engagement least likely to teach you anything, and that is a fine prediction. The reason it does not settle my question is that ICAP is about how much you learn. It says nothing about how engaging something feels while you are doing it, or about how much you want to keep going. Passive engagement looks like the easiest kind to start and the easiest to stay in, and that is a different axis from how much you learn.
So I want a second thing alongside ICAP, which is why a person reaches for the easy kind at all.
Context is what makes a familiar walk interesting
Walking the same route repeatedly builds an expectation of how the walk goes. Once that expectation exists, you notice what departs from it. There is a man I pass most days; on the day he is not there, I notice he is not there and wonder where he went.
On that same walk in the summer I passed a police car with its lights on, which on a near-empty campus is strange enough to hold my attention. On the way back I saw a man in handcuffs against a wall with three or four cars around him. Whatever else that was, it was as engaging as anything an app has shown me. I am wary of using it as an example, because people carry real history with the police and I do not want the illustration to do work the argument has not earned. But it is the clearest case I have of a familiar route producing something I could not look away from.
Scrolling does not accumulate that way. Each item differs enough from the last that no expectation forms, so there is nothing for a departure to depart from. The context gets replaced before it can build. You can argue that an app supplies more unfamiliar input per minute than my walk does, and I think that is true. It supplies less of the thing that makes a small anomaly worth noticing.
This gives me a reading of "chronically online" that I like better than the usual one. Someone who is in a forum constantly has built exactly this kind of context, and gets the same noticing from it. They can tell when a regular poster is behaving oddly and wonder what happened to their account. That is the walk feeling, built in a digital space.
A guess about self-generated engagement, and the reason to distrust it
Walking without a phone leaves you as the only source of anything to think about. You start from yourself. Walking with one, the phone supplies the starting point and you work from what you were shown.
My guess is that starting from yourself is a skill, and an unevenly distributed one. I have a bias here that I should state plainly: I backpack, so I have spent a lot of hours walking with no input but the trail, and I am comfortable keeping myself occupied with whatever is in front of me. That makes me a poor judge of how hard it is for someone who has not had that practice.
The analogy I keep reaching for is locus of control 2. Mine used to be firmly external. When a senior engineer at Microsoft told me a task was the priority, that instruction was the motivation, and I would drop whatever I was doing. Now I am more likely to decide the task matters and then do it, which often ends in the same action from a different place. Walking with a phone looks like an external locus for engagement: the interest has to be handed to you. Generating it yourself looks internal, and I think I prefer that, though preferring it is not evidence.
Here is where my own argument gives me trouble. Humans are extraordinarily social, and most of what any of us does is built from what other people are doing, varied by our own experience. If that is right, then a person constructing their entertainment alone on a walk is doing something less social than the person reading what their friends posted, and I have been quietly treating the solitary version as the healthier one without having argued for it.
I am leaving that unresolved. The next step is a literature review that looks for work supporting this and work contradicting it, and I would expect attention research and media studies to have said something about the context point already.
Driving is a different problem and I have left it alone. My guess there is that people reach for the phone to resolve uncertainty about where they are going, and that the relief of knowing outweighs a danger they are not feeling at the time. I do not have the insight to take that further.
References
- Chi and Wylie (2014). The ICAP Framework: Linking Cognitive Engagement to Active Learning Outcomes. Educational Psychologist 49(4), 219-243.
- Rotter (1966). Generalized Expectancies for Internal Versus External Control of Reinforcement. Psychological Monographs: General and Applied 80(1), 1-28.
Tagged attention · learning-sciences · media
Thinking about curriculum as a knowledge graph
I want to try designing a curriculum using what I have been learning about knowledge components and knowledge graphs from Ken Koedinger’s and Tiffany Barnes’s work. The knowledge-component idea I am working from is the one in the Knowledge-Learning-Instruction framework 1. I have a picture in my head of how this might work, and I want to get it into a form I can examine. This is a proposal that I think could become part of my dissertation. I still need a literature review to support or deny the points I am making, and to find out how much of this work already exists.
I am starting with a knowledge component as a piece of information that a person can apply in a meaningful way. In the graph I am imagining, each node represents one of those components. An edge represents a relation that the person uses to move from one component to another while thinking. Whether someone can actually apply the knowledge they think they have is another can of worms.
My example is cats. One piece of information I have about cats is that their fur patterns are determined by genetics and can sometimes be related to their sex. That gives me a connection between cats and fur patterns. Another piece of information is that blue lynx point is a type of fur pattern. My cat, Chumbles, is a blue lynx point Siberian, so I have a very concrete connection between those pieces of information. Chumbles is, essentially, my proof that this pattern exists.
I can use one piece of information to get to another. The edge is how I remember that the two are related. That is the graph structure I have in mind when I start thinking about curriculum.
Helping a learner draw the edges
When I design a curriculum, I am trying to help a learner build an understanding with some structure to it. I need to know which knowledge components I am presenting and how I expect them to connect. Those connections could help me work out the flow of the class.
In discrete mathematics, I might start with sets. Once a student has an understanding of a set in terms of whether an element is included or excluded, I can introduce multisets. I then need to help the student draw a connection between the two.
The relation I am trying to explain involves repeated elements and whether elements are distinguishable in the problem I am considering. From there, I start connecting to counting problems and stars and bars. I also start thinking about binary strings, because zeros and ones give me a way to represent problems involving repeated elements. This is the path through the material that I am trying to describe.
For a learner with few knowledge components to draw on in this area, I think the curriculum has a job to do at each connection. I am saying, “You have learned about sets. Here is a multiset, and here is how it relates back.” I am introducing a node and helping the learner draw an edge to it.
That makes me think about what happens to an edge after the class has moved on. The last time I used a relation seems relevant to whether I can use it now. I might remember that sets and multisets both exist while having trouble remembering exactly how they connect.
I picture memory as a source with connections to the components I can recall directly. I was reaching for a max-flow/min-cut picture when I first thought this through: if a connection from that source is cut, I can no longer get directly to the component.
I might still have another route. If I cannot directly remember a multiset, perhaps I can begin with what I remember about sets. If the relation between sets and multisets is still available, I can follow it and recover the idea. In my graph picture, I have searched from another entry point and then redrawn the direct connection. I am imagining something like a breadth-first search through what I know.
I can also use this picture to describe rote memorization. I imagine a student who has memorized much of a textbook and can recall individual pieces of information, while having few connections between them. The source has many outgoing edges, but the rest of the graph is sparse. That student might hope for a multiple-choice test because the individual items are familiar. A problem that asks the student to reason across those items could be harder.
This is part of what I would want a curriculum graph to represent: the connections I hope a student can use by the end of the course.
Concrete problems inside an abstract component
In class, I can present a knowledge component and explain how it connects to another. In homework, I ask students to work with concrete versions of those ideas.
For multisets, I was thinking about a donut store. I have six unflavored donuts and two flavors, vanilla and chocolate. How many different ways can I flavor the six donuts? In the version I am imagining, the donuts are indistinguishable from one another. There can be several items of the same kind.
I start wondering whether the abstract knowledge component itself contains a smaller graph. I have an abstract understanding of multisets, but inside that understanding are the concrete problems I have worked through and the connections I have made between them. A knowledge component could be a boundary around that smaller graph.
My theory is that the internal structure matters when I try to connect the abstract component to something else. If I have worked through several concrete versions and understand how they relate, perhaps I am more able to draw a new edge from that abstract knowledge to another abstract component. I am especially interested in the density of the connections between the examples.
An experience from teaching discrete mathematics is part of why I think about it this way.
I took a numerical analysis course in which I worked on interpolation. Given a set of points, how could I construct a function that passed through them? One approach I remember involved polynomial terms arranged so that, at a particular point, the other terms became zero. I also worked through problems about the error between an interpolation and a function that generated the points.
The idea I took away was something I was thinking of as orthogonality. I pictured different terms as separate axes, with each one representing the effect of a particular point on the interpolation.
Then, while teaching discrete mathematics, I was working with linear homogeneous recurrence relations. When the characteristic equation has a repeated root, I remembered the step of multiplying the second corresponding term in the closed form by n. While working through a problem in class, I suddenly connected that step to what I had been doing with interpolation. I thought I was seeing the same idea of making the terms orthogonal.
What I am trying to preserve here is how the connection occurred to me. I had done many concrete interpolation problems in numerical analysis. I had also practiced many recurrence problems because I was preparing to teach them. During an application, I drew what felt like a long edge between two parts of my understanding.
I do not think I would have made that connection without the numerical analysis class. In my graph picture, the concrete problems I had worked through in both areas helped me notice a structure that I thought they shared.
Searching through what I know
That experience leads me toward a broader account of problem solving. When I encounter a new problem, I may have some relevant knowledge components and some structure connecting them. I can look elsewhere in what I know for a similar structure and try to apply it.
That is how I was thinking about the interpolation and recurrence connection. I drew on an idea from one part of my understanding to make sense of a problem in another. In this picture, a search through the graph can help me connect to a component I did not previously know.
I imagine my own graph as sparse compared with the knowledge I could potentially encounter. Learning would include discovering a component that I have no connection to yet and working out how to connect it to something I already understand.
Research makes this picture more complicated. If I take the available components to represent what people currently know, research could generate new components and then make them available to others. I build an understanding of a new area, and writing a paper gives me a way to help someone else build an understanding of it.
That makes paper writing look a little like curriculum design to me. As I write, I introduce an idea and explain a connection. I use that connection to introduce something else, eventually helping the reader reach the new result. I am trying to arrange a path that someone can follow as they read.
I think this could be an interesting framework for describing how I think and solve problems. There are also many factors that would affect whether I can follow a particular path at a particular time.
I can have multiple ways of remembering how two things are related. I also imagine something like a multiplier on an edge that depends on my motivation or interest in the problem. Chemistry is my example. I have been pretty good at chemistry, but I am just not very interested in it. When I stop using it, my understanding feels as though it decays almost immediately.
I still remember some connections to quantum physics and orbitals. Ask me to do stoichiometry, though, and I no longer know how to do it. I think my interests matter to which connections I maintain and which new ones I am able to draw. I would want to understand how that fits with research on memory and problem solving.
Thinking while teaching
Teaching brings another complication. As an instructor, I have to learn how to answer a question I have never seen before while standing in front of potentially a hundred people. I need to give an explanation that they can follow.
Sometimes I end up giving an approximate answer based on what I think is happening. Then I go home and read about it. If the question was far enough outside what I knew, I might email the student or return to it the next day.
That makes direct recall important to teaching, too. I need to have a lot available off the top of my head. I also wonder how students interpret the time I spend thinking. Some might take a pause as a sign that I do not know the material. Others might see working through the question as evidence that I do. I do not have one account that covers those different reactions.
I think there could be useful research in tracing how someone moves through these connections in real time. Answering an unfamiliar question in class is one setting where I would want to understand that process better. My graph picture gives me a way to start describing it, but the factors affecting those connections still need to be worked out.
A combinatorics curriculum I want to try
I want to bring this back to something I can design. Before I do that, I need to find out how these ideas relate to existing work on knowledge components and knowledge graphs. Ken Koedinger’s work and Tiffany Barnes’s work are starting points for me. I also want to look at how psychology describes human problem solving.
I want that review to test the claims I have made here. I want to know whether someone has already developed the kind of account I am imagining. Since I am in Dr. Barnes’s lab, I also expect to talk with her about it.
The curriculum I want to draft is a discrete mathematics unit on combinatorics. I am thinking about sets and multisets, then counting and an introduction to probability through events and event spaces. I also have in mind the Pyrenees work from Dr. Min Chi at NC State 2.
I want to construct a graph whose knowledge components and edges represent the curriculum. Then I want to produce the teaching artifacts that would make that graph concrete. I would need material that introduces a component and helps students connect it to something they have already encountered. I would also need the tasks through which students practice applying it.
My hope is that, by the end of the unit, a student’s internal understanding would have the structure I was trying to teach. I have been picturing that as the student’s graph coming to resemble the curriculum graph. Whether that is a useful goal is one of the things I want the review to tell me.
The visual image I keep coming back to is a knowledge component with a rim that gets brighter as I understand it better. When I first connect to it, it starts to turn on. As I work through concrete versions, it becomes more available to me, and I imagine myself becoming more capable of drawing a new edge from it. I want to use the combinatorics unit to work out what that picture would mean for the materials I actually ask students to use.
References
- Koedinger, Corbett, and Perfetti (2012). The Knowledge-Learning-Instruction Framework: Bridging the Science-Practice Chasm to Enhance Robust Student Learning. Cognitive Science 36(5), 757-798.
- Chi, VanLehn, Litman, and Jordan (2011). An Evaluation of Pedagogical Tutorial Tactics for a Natural Language Tutoring System: A Reinforcement Learning Approach. International Journal of Artificial Intelligence in Education 21(1-2).
Tagged curriculum-design · knowledge-components · discrete-mathematics
I am looking for a teaching professor position. What I am looking for