Three-Attempt Consistency Protocol
Definition
The Three-Attempt Consistency Protocol repeats the same question in three separate chatbot attempts before applying the study’s resistance criterion.
Explanation
One failed response may be an isolated outcome. The procedure therefore repeats the exact question and grades each response against the stated rubric. In the AIED work, all three scores had to be below 60% for the question to meet the criterion.
Use a fresh conversation for each attempt and record the model, access route, date, prompt version, and rubric. A prompt edit creates a new question version; its results should not be combined with the earlier wording as though they tested the same input.
The resulting record supports a claim about those three attempts. More trials and a broader set of conditions would be needed to estimate reliability beyond them.
Scope and evidence
This is a procedural consistency check, not a statistical guarantee or an independent validation of student learning.
Related concepts
One-Shot GenAI Resistance · Iterative perturbation process · Free-tier accessible model