When Verification Hurts: Asymmetric Effects of Multi-Agent Feedback in Logic Proof Tutoring
arXiv preprint
Tahreem Yasir; Sutapa Dey Tithi; Benyamin Tabarsi; Dmitri Droujkov; Sam Gilson; Yasitha Rajapaksha; Xiaoyi Tian; Arun Ramesh; DongKuan Xu; Tiffany Barnes
arXiv:2603.27076
Citation year: 2026.
Preprint posted: 2026-03-28.
Summary
Does adding a verification agent improve feedback in logic proof tutoring? This preprint compares feedback configurations on 516 tutoring states. It examines whether verification affects feedback on correct and incorrect student steps differently. In the evaluated pipelines, verification helped when upstream feedback was error-prone but reduced performance when that feedback was already reliable.
What this study can tell us
The results concern the evaluated models, prompts, and tutoring states. An additional agent should not be assumed to improve feedback in every setting. This record identifies a preprint rather than an accepted conference paper.
Abstract
Large language models (LLMs) are increasingly used for automated tutoring, but their reliability in structured symbolic domains remains unclear. We study step-level feedback for propositional logic proofs, which require precise symbolic reasoning aligned with a learner’s current proof state. We introduce a knowledge-graph-grounded benchmark of 516 unique proof states with step-level annotations and difficulty metrics. Unlike prior tutoring evaluations that rely on model self-assessment or binary correctness, our framework enables fine-grained analysis of feedback quality against verified solution paths. We evaluate three role-specialized pipelines with varying solution access: Tutor (partial solution access), Teacher (full derivation access), and Judge (verification of Tutor feedback). Our results reveal a striking asymmetry: verification improves outcomes when upstream feedback is error-prone (<70% accuracy), but degrades performance by 4-6 percentage points through over-specification when feedback is already reliable (>85%). Critically, we identify a shared complexity ceiling; no model or pipeline reliably succeeds on proof states exceeding complexity 4-5. These findings challenge the assumption that adding verifiers or richer context universally improves tutoring, motivating adaptive, difficulty-aware architectures that route problems by estimated complexity and upstream reliability.
The abstract as posted on arXiv.
Citation
@misc{yasir2026verification,
title = {{When Verification Hurts: Asymmetric Effects of Multi-Agent Feedback in Logic Proof Tutoring}},
author = {Tahreem Yasir and Sutapa Dey Tithi and Benyamin Tabarsi and Dmitri Droujkov and Sam Gilson and Yasitha Rajapaksha and Xiaoyi Tian and Arun Ramesh and DongKuan Xu and Tiffany Barnes},
year = {2026},
eprint = {2603.27076},
archivePrefix = {arXiv},
primaryClass = {cs.AI},
note = {Preprint, version 1, 28 March 2026},
doi = {10.48550/arXiv.2603.27076},
url = {https://sgilson7.github.io/papers/yasir2026verification/}
}