Feedback with reasoning

EduEval is a comprehensive hierarchical benchmark for evaluating LLMs in Chinese K-12 education, comprising 24 task types with over 11,000 questions organized across six cognitive dimensions (Memorization, Understanding, Application, Reasoning, Creativity, and Ethics) based on Bloom's Taxonomy and Webb's Depth of Knowledge. The benchmark uses authentic educational materials including real exam questions, classroom dialogues, student essays, and expert-designed prompts spanning primary through high school levels.

FEANEL is a benchmark for evaluating LLMs' ability to provide fine-grained error analysis and pedagogical feedback on K-12 English writing, comprising 1,000 student essays with expert-annotated errors categorized by type, severity, and explanations. The benchmark specifically assesses whether AI systems can identify writing errors and provide educationally meaningful, interpretable feedback to support student learning.

This paper presents LearnLM-Tutor, a fine-tuned Gemini model for educational use, and introduces a comprehensive evaluation framework spanning seven diverse benchmarks (quantitative, qualitative, automatic, and human evaluations) grounded in learning science principles to assess pedagogical quality in K-12 AI tutoring systems. The work includes real-world deployment at Arizona State University and systematic evaluation of pedagogical dimensions including Socratic dialogue, adaptive scaffolding, and learning-centered interactions.

This paper proposes a unified evaluation taxonomy with eight pedagogical dimensions to assess LLM-powered AI tutors' abilities in remediating student mistakes in mathematics, and releases MRBench - a benchmark containing 192 conversations and 1,596 responses from seven tutors with human annotations across all dimensions. The taxonomy evaluates pedagogical interactions including mistake identification, guidance provision, actionability, and tutor tone, directly measuring whether AI tutors demonstrate effective pedagogical abilities rather than simply revealing answers.

TutorBench is a purpose-built benchmark with 1,490 expert-curated samples evaluating LLMs on three core tutoring skills: generating adaptive explanations, providing actionable feedback, and creating effective hints for high-school and AP-level content. The benchmark uses sample-specific rubrics and LLM-judge evaluation to assess 16 frontier models, finding none exceed 56% overall performance and all achieve less than 60% pass rate on criteria related to guiding, diagnosing, and supporting students.

This paper evaluates LLM-based and statistical NLP methods for automated scoring of nationwide high school graduation essay exams in Estonia, comparing machine-generated scores against human raters across multiple rubric dimensions including content, argumentation, and language quality. The study demonstrates that automated scoring achieves reliability comparable to human raters while also examining bias, prompt injection risks, and providing personalized feedback capabilities.

MathDial presents a dataset of 3,000 one-to-one math tutoring dialogues where human teachers guide LLM-simulated students through multi-step reasoning problems using scaffolding questions and pedagogical moves, with extensive annotations for training and evaluating AI tutoring systems. The paper demonstrates that current LLMs fail at effective tutoring by revealing solutions too early or providing incorrect feedback, and shows how models finetuned on MathDial improve interactive tutoring performance.

This paper develops Bridge, a method that translates expert tutors' decision-making processes into a framework for LLMs to remediate elementary math mistakes, using a dataset of 700 real tutoring conversations with 1st-5th grade students. The work evaluates GPT-4 and Llama-2-70b on their ability to provide pedagogically sound responses to student errors when guided by expert decision models.

This paper presents MWPTutor, an LLM-based intelligent tutoring system for math word problems that combines structured pedagogical strategies (finite state transducers) with LLM flexibility, and evaluates it against GPT-4 through human evaluation studies. The system implements guardrails to prevent common tutoring pitfalls like answer-leaking while maintaining pedagogical control through predefined teaching strategies.

This paper presents Physics-STAR, an LLM-powered tutoring system for high school physics education, and evaluates it through a controlled experiment with 12 high school sophomores against traditional lectures and generic LLM tutoring. The system provides step-by-step guidance, reflective learning prompts, and personalized scaffolding to improve conceptual understanding and problem-solving skills in physics.

Mentigo is an AI mentor agent system designed to guide middle school students through creative problem solving (CPS) processes, providing scaffolding, personalized feedback, and Socratic questioning based on real classroom mentor-student interactions. The system was evaluated through comparative experiments with 12 students and reviewed by expert teachers, demonstrating improvements in student engagement and creative outcomes.

This paper develops and evaluates a pedagogical framework for LLM-driven feedback generation in secondary school Python programming education, aligning automated feedback with established pedagogical principles like mastery adaptation and progress-based scaffolding. Through mixed-method evaluation with eight secondary school computer science teachers, the study assesses how well LLM-generated feedback adheres to pedagogical standards compared to human teacher feedback.

This paper presents RockStartIT Tutor, a GPT-4-powered AI tutoring system designed for secondary school students learning programming and computational thinking, using a curriculum-constrained knowledge base and pedagogically controlled prompting. The system was pilot-evaluated with 13 students and teachers using the Technology Acceptance Model to assess its effectiveness in providing scaffolded, personalized support.

This paper develops a chain-of-thought prompting approach using GPT-4 to automatically score and generate explanations for middle school Earth Science formative assessment responses, employing human-in-the-loop few-shot and active learning methods. The system evaluates open-ended short-answer responses and provides meaningful feedback to support student learning.

EducationQ is a multi-agent dialogue framework that evaluates LLMs' teaching capabilities through simulated teacher-student interactions, testing 14 models across 1,498 questions spanning 13 disciplines and 10 difficulty levels. The framework incorporates formative assessment principles to measure pedagogical effectiveness including questioning strategies, adaptive feedback, and scaffolding behaviors.

This qualitative action research study evaluates AI chatbots as auxiliary tools to help Brazilian high school students improve their argumentative ecology essays through iterative feedback and revision cycles. The research directly measures learning outcomes and engagement when students use chatbots for writing improvement in a classroom setting.

This paper evaluates ChatGPT's ability to solve decimal math problems, assess correctness of student self-explanations, and generate feedback within the Decimal Point learning game for middle school students, using over 5,000 real student responses. The study assesses ChatGPT's content knowledge in decimals, automated grading accuracy (75%), and pedagogical quality of generated feedback using a structured rubric.

This paper trains an open-source LLM (Llama 3.1 8B) to generate tutor utterances that maximize student learning outcomes in math tutoring dialogues by optimizing for both student response correctness and pedagogical quality using direct preference optimization. The approach uses a student model to predict correctness and GPT-4o to evaluate adherence to pedagogical principles, directly measuring the impact on student learning through dialogue interactions.

This paper develops and evaluates a dialogue-based intelligent tutoring system (ITS) for teaching sixth-grade students multiplication and division of fractions, using diagnostic teaching methodology with real-time error identification and adaptive instructional strategies. The system was tested through a quasi-experimental study with 66 sixth graders, showing significant learning gains compared to conventional classroom instruction.

This paper presents findings from the BEA 2025 Shared Task that evaluates pedagogical abilities of AI tutors powered by LLMs, specifically focusing on assessing tutor responses for mistake identification, guidance provision, and feedback actionability in mathematics education dialogues. The task established pedagogically-motivated evaluation tracks grounded in learning science principles to measure how effectively AI tutors remediate student mistakes through dialogue.

This paper evaluates the feasibility of using LLMs (GPT-4, Gemini, LearnLM) to automatically identify and assess two specific tutoring moves in real-life math tutoring dialogues: delivering effective praise and responding to student errors. The study analyzes 50 transcripts of college tutors working with middle school students, demonstrating that LLMs can reliably detect tutoring situations (94-98% accuracy for praise detection, 82-88% for error detection) and evaluate adherence to best practices (83-89% and 73-77% alignment with human judgment).

This paper presents an MPNet-based ensemble system for automatically evaluating AI tutor responses in educational dialogues, specifically assessing whether tutors correctly identify and locate student mistakes across two classification tasks at the BEA 2025 Shared Task.

This paper develops an automated system using GPT-3.5 to evaluate five key tutoring strategies (praise, error reaction, knowledge assessment, managing inequity, responding to negative self-talk) in one-on-one tutoring dialogues, classifying whether each strategy is employed effectively or ineffectively. The system analyzes tutoring transcripts to provide color-coded feedback on pedagogical quality of tutor-student interactions.

This paper introduces PedagogicalRL-Thinking, a reinforcement learning framework that trains LLM tutors to generate pedagogically appropriate reasoning traces (not just responses) by rewarding thinking processes grounded in Polya's problem-solving framework, with evaluation on mathematics tutoring dialogues measuring solution correctness, answer leakage prevention, and helpfulness.

This paper introduces Hierarchical Pedagogical Oversight (HPO), a multi-agent adversarial framework that evaluates AI tutor responses in K-12 mathematics dialogues by using specialist agents and structured debate to assess mistake identification and guidance quality. The system is validated on MRBench, a dataset of 1,214 middle-school mathematics tutoring dialogues, achieving superior performance in detecting sycophancy and inappropriate scaffolding.

This paper develops and evaluates a reinforcement learning framework for automatically generating pedagogically valid feedback for incorrect student answers in math education, using GPT-4 to evaluate feedback quality according to a rubric measuring both correctness and alignment with educational goals. The work demonstrates that fine-tuning Llama 2 with direct preference optimization significantly improves feedback quality across correctness and pedagogical alignment dimensions.

This paper evaluates CyberScholar, a GenAI writing assistant that provides formative feedback aligned with teacher rubrics to K-12 students (grades 7-11) across multiple subject areas. The study examines student and teacher perceptions through surveys and interviews in four U.S. schools, focusing on how AI feedback supports writing improvement, metacognition, and self-regulation.

This paper develops and evaluates a multi-agent AI system (AutoFeedback) to automatically generate feedback on student constructed responses in science assessments, specifically addressing issues of over-praise and over-inference found in single-agent LLM feedback. The system uses two agents—one to generate feedback and another to validate and refine it—and demonstrates significant improvements in feedback quality when tested on 240 student responses.

This paper presents Inquizzitor, an LLM-based formative assessment agent for middle school STEM+C education that provides adaptive scaffolding through theory-driven dialogue. The framework integrates Evidence-Centered Design with Social Cognitive Theory to deliver pedagogically sound feedback and assessment grounded in cognitive science principles.

This paper presents a co-design pilot study with 19 K-12 teachers examining the classroom implementation of an AI-powered grading platform (Colleague AI) that generates rubrics and provides automated feedback on student work. Through usage logs, surveys, and interviews, the study evaluates teacher and student perspectives on AI-generated scoring versus narrative feedback, revealing trust issues with automated grading but appreciation for rapid formative feedback.

This paper presents a randomized controlled trial evaluating GPT-4 as an interactive homework tutor for high-school ESL students, measuring impacts on learning outcomes (grammar gains) and student engagement compared to traditional static homework.

This paper reports on aiPlato, an AI-enabled homework platform for introductory college physics that provides stepwise feedback and iterative guidance through tools like 'Evaluate My Work' and 'AI Tutor Chat'. The exploratory classroom study analyzes student engagement patterns, correlations with final exam performance, and student perceptions of the platform's learning value while preserving productive struggle.

This is a three-day lesson plan for 9th-12th grade English Language Arts that uses AI writing assistants to help students analyze their writing through AI-enhanced rubrics, receive automated feedback, and engage in AI-informed peer review to revise their work. The lesson directly evaluates how AI tools impact student writing revision processes and self-assessment skills in an actual classroom setting.

This paper evaluates state-of-the-art VLMs and LLMs on grading over 14,000 handwritten answers from Indonesian Grade 4 students in Mathematics and English, and generating personalized feedback. The study introduces a multimodal pipeline integrating vision models for handwriting recognition and language models for rubric-based assessment in real-world, underrepresented classroom settings.

This paper presents VATE (Virtual AI Teacher for Error Analysis), an LLM-based system deployed on the Squirrel AI platform that autonomously analyzes student mathematical errors from their work drafts, provides targeted error explanations, and engages in multi-round Socratic dialogue to guide correction. The system achieves 78.3% accuracy in error analysis for elementary mathematics and demonstrates improved student learning efficiency in real classroom deployment.

EduAlign proposes a reinforcement learning framework to align large language models with pedagogical principles of helpfulness, personalization, and creativity for AI tutoring. The framework includes HPC-RM, a multi-dimensional reward model trained on 8k educational interactions, and uses GRPO to fine-tune LLMs to become more pedagogically effective AI tutors.

AgentTutor is an LLM-powered multi-turn interactive tutoring system that uses generative agents to deliver personalized instruction by decomposing curricula, assessing learner cognitive levels based on Bloom's Taxonomy, dynamically adjusting teaching strategies, and providing reflective feedback through multi-turn dialogue. The system evaluates its effectiveness on benchmark datasets by measuring learning outcomes and teaching quality in interactive educational contexts.

This paper develops and evaluates a generative AI-based system using ChatGPT to automatically assess and provide feedback on student argumentation writing by extracting argument components (claims, data, warrants, etc.). The system compares AI performance against human expert evaluation across scientific and non-scientific contexts for K-12 argumentation training.

This paper presents an AI-powered Python programming chatbot designed for students that combines static code analysis, dynamic execution tracing, and LLMs (CodeLlama, GPT-4) to provide debugging assistance and educational support. The system was evaluated with 1,500 student submissions showing 85% error resolution success, 59.3% reduction in debugging time, and 34% improvement in coding proficiency through pre/post-test assessments.

This paper evaluates the quality and usefulness of educational questions generated by large language models (specifically GPT-3/InstructGPT) using controllable text generation with Bloom's taxonomy and difficulty levels, validated through teacher assessments across two domains. The study demonstrates that LLM-generated questions are judged by teachers as high quality and sufficiently useful for classroom use.

ChatTutor is a full-fledged LLM-based intelligent tutoring system that provides course planning, instruction, and adaptive quizzes through three interconnected processes (interaction, reflection, reaction) with dynamic memory management. The system is evaluated through user studies and statistical analysis comparing it with real-world online learning platforms, measuring consistency, stability, and pedagogical effectiveness.

This paper evaluates the multilingual performance of large language models (GPT-4o, Gemini, Claude, Llama, Mistral) on four K-12 educational tasks: identifying student misconceptions, providing targeted feedback, interactive tutoring, and grading translations across nine languages including English, Mandarin, Hindi, Arabic, German, Farsi, Telugu, Ukrainian, and Czech.

This paper compares three models (fine-tuned Mistral/GOAT, SBERT-Canberra, and GPT-4) for automatically scoring and providing qualitative feedback on middle-school students' open-ended math responses, evaluating both scoring accuracy and feedback quality using teacher judgments.

EduBench introduces a comprehensive benchmark dataset with 18,821 data points across 9 educational scenarios and 4,000+ educational contexts (covering K-12 and higher education subjects, different difficulty levels, and multiple question types), evaluated using 12 multi-dimensional metrics covering scenario adaptation, factual/reasoning accuracy, and pedagogical application. The benchmark assesses LLM capabilities in diverse educational tasks including assignment grading, study planning, tutoring, and psychological counseling, with both human and automated evaluation.

This paper presents Ruffle&Riley, an LLM-based conversational tutoring system using a learning-by-teaching format with two AI agents (student and professor), and evaluates it through two online studies (N=200) measuring learning outcomes, engagement, and user experience in biology lessons compared to simpler chatbots and reading activities.

This paper develops and evaluates stepwise verification methods for detecting student reasoning errors in math problem-solving, showing that grounding tutor responses in explicit error detection improves feedback quality and reduces hallucinations in LLM-based dialog tutoring systems. The work collects a dataset of 1K annotated student solution chains and demonstrates that verifier-guided generation produces more targeted, correct responses compared to direct generation baselines.

This paper introduces PBLBench, a benchmark for evaluating multimodal large language models (MLLMs) on their ability to assess STEM Project-Based Learning (PBL) outcomes, including evaluating research reports, code, experimental data, and videos using expert-validated criteria derived through the Analytic Hierarchy Process (AHP). The benchmark tests 15 MLLMs on grading and ranking PBL projects across multiple STEM disciplines with long-context multimodal inputs.

This paper evaluates GPT-3.5-Turbo and GPT-4's ability to assess human tutors' performance in responding to K-12 students' math errors, specifically measuring whether tutors use indirect guidance strategies versus direct error correction. The study analyzes 50 real tutoring dialogues to determine if LLMs can provide automated feedback on tutoring quality.

This paper presents DIRECT, a dialogue-based intelligent tutoring system for reading comprehension that asks questions, assesses student answers, provides hints, and engages in encouraging chat. The system is evaluated on a dataset constructed from RACE-M, which contains reading comprehension exercises from English exams for middle school students.

This paper evaluates LLMs (GPT-4o, Llama-3-8B) as teachers generating hints for simulated LLM students solving high-school mathematics problems, measuring hint quality and students' ability to self-correct errors. The study compares different prompting strategies and model configurations to assess pedagogical effectiveness of AI-generated hints.