📄 Paper: https://arxiv.org/abs/2603.29142
Access Online appendix: Appendix.pdf
Formative feedback is central to effective learning, yet providing timely, individualized feedback at scale remains a persistent challenge. While recent work has explored the use of large language models (LLMs) to automate feedback, most existing systems still conceptualize feedback as a static, one-way artifact, offering limited support for interpretation, clarification, or follow-up. In this work, we introduce REFINE, a locally deployable, multi-agent feedback system built on small, open-source LLMs that treats feedback as an interactive process. REFINE combines a pedagogically grounded feedback generation agent with an LLM-as-a-judge–guided regeneration loop using a human-aligned judge, and a self-reflective tool-calling interactive agent that supports student follow-up questions with context-aware, actionable responses. We evaluate REFINE through controlled experiments and an authentic classroom deployment in an undergraduate computer science course. Automatic evaluations show that judge-guided regeneration significantly improves feedback quality, and that the interactive agent produces efficient, high-quality responses comparable to a state-of-the-art closed-source model. Analysis of real student interactions further reveals distinct engagement patterns and indicates that system-generated feedback systematically steers subsequent student inquiry. Our findings demonstrate the feasibility and effectiveness of multi-agent, tool-augmented feedback systems for scalable, interactive feedback.
web/: Next.js (React) frontend. Dependencies and scripts live inweb/package.json.core/: Python backend and agents. Python dependencies live incore/requirements.txt.
REFINE includes a web application (web/) backed by a Python service (core/).
Frontend (Node/Next.js):
cd web
npm installBackend (Python):
python3 -m venv .venv
source .venv/bin/activate
pip install -r core/requirements.txt