Pedagogical interactions

EduEval is a comprehensive hierarchical benchmark for evaluating LLMs in Chinese K-12 education, comprising 24 task types with over 11,000 questions organized across six cognitive dimensions (Memorization, Understanding, Application, Reasoning, Creativity, and Ethics) based on Bloom's Taxonomy and Webb's Depth of Knowledge. The benchmark uses authentic educational materials including real exam questions, classroom dialogues, student essays, and expert-designed prompts spanning primary through high school levels.

FoundationalASSIST introduces a 1.7-million interaction K-12 educational dataset with full question text, student responses, and Common Core alignment, specifically designed to evaluate whether LLMs can perform knowledge tracing (predicting student performance) and pedagogical grounding (understanding assessment item properties). The paper evaluates four frontier LLMs on these tasks, revealing significant gaps in their ability to predict student performance and understand item discrimination.

This paper presents LearnLM-Tutor, a fine-tuned Gemini model for educational use, and introduces a comprehensive evaluation framework spanning seven diverse benchmarks (quantitative, qualitative, automatic, and human evaluations) grounded in learning science principles to assess pedagogical quality in K-12 AI tutoring systems. The work includes real-world deployment at Arizona State University and systematic evaluation of pedagogical dimensions including Socratic dialogue, adaptive scaffolding, and learning-centered interactions.

EduGuardBench is a dual-component benchmark designed to evaluate LLMs acting as simulated teachers, measuring both pedagogical fidelity (role-playing accuracy, teaching competence) and adversarial safety (resistance to jailbreaking, handling of academic misconduct requests). The benchmark identifies harmful teaching behaviors (incompetence, indolence, offensiveness) and uses persona-based adversarial prompts to test ethical boundaries specific to educational contexts.

This paper proposes a unified evaluation taxonomy with eight pedagogical dimensions to assess LLM-powered AI tutors' abilities in remediating student mistakes in mathematics, and releases MRBench - a benchmark containing 192 conversations and 1,596 responses from seven tutors with human annotations across all dimensions. The taxonomy evaluates pedagogical interactions including mistake identification, guidance provision, actionability, and tutor tone, directly measuring whether AI tutors demonstrate effective pedagogical abilities rather than simply revealing answers.

This paper introduces GuideEval, a benchmark that evaluates LLMs' ability to provide adaptive Socratic tutoring by assessing three pedagogical phases: perceiving learner states (confusion, comprehension, errors), orchestrating appropriate instructional strategies, and eliciting productive reflections. The benchmark is grounded in authentic K-12 educational dialogues and specifically measures whether LLMs can dynamically adjust their guidance based on student cognitive states rather than just generate generic responses.

TutorBench is a purpose-built benchmark with 1,490 expert-curated samples evaluating LLMs on three core tutoring skills: generating adaptive explanations, providing actionable feedback, and creating effective hints for high-school and AP-level content. The benchmark uses sample-specific rubrics and LLM-judge evaluation to assess 16 frontier models, finding none exceed 56% overall performance and all achieve less than 60% pass rate on criteria related to guiding, diagnosing, and supporting students.

This paper presents PhysicsAssistant, a multimodal robot combining YOLOv8 object detection with GPT-3.5-turbo to provide real-time interactive assistance to 8th-grade students during physics lab experiments. The system is empirically evaluated through a user study with 10 students, where expert ratings based on Bloom's taxonomy assess the quality of responses compared to GPT-4.

This paper presents an LLM-based instructional module to teach prompting literacy to K-12 students through scenario-based practice with AI chatbots, deployed across 11 secondary education classrooms. The study evaluates an AI auto-grader's capability to assess student prompts, measures changes in students' prompting performance and confidence in using AI for learning, and analyzes the quality of assessment materials.

This paper empirically evaluates LLM-based tutoring systems against traditional deep knowledge tracing (DKT) models for learner modelling in K-12 education, demonstrating that LLMs fall short in accurately tracking student knowledge over time even after fine-tuning. The study directly measures prediction accuracy, temporal coherence, and multi-skill mastery estimation using a large-scale K-12 dataset to assess whether LLMs can responsibly support adaptive instruction.

MathDial presents a dataset of 3,000 one-to-one math tutoring dialogues where human teachers guide LLM-simulated students through multi-step reasoning problems using scaffolding questions and pedagogical moves, with extensive annotations for training and evaluating AI tutoring systems. The paper demonstrates that current LLMs fail at effective tutoring by revealing solutions too early or providing incorrect feedback, and shows how models finetuned on MathDial improve interactive tutoring performance.

This paper develops Bridge, a method that translates expert tutors' decision-making processes into a framework for LLMs to remediate elementary math mistakes, using a dataset of 700 real tutoring conversations with 1st-5th grade students. The work evaluates GPT-4 and Llama-2-70b on their ability to provide pedagogically sound responses to student errors when guided by expert decision models.

This paper presents MWPTutor, an LLM-based intelligent tutoring system for math word problems that combines structured pedagogical strategies (finite state transducers) with LLM flexibility, and evaluates it against GPT-4 through human evaluation studies. The system implements guardrails to prevent common tutoring pitfalls like answer-leaking while maintaining pedagogical control through predefined teaching strategies.

MathVC is an LLM-simulated multi-persona virtual classroom that creates AI-powered peer agents to facilitate collaborative mathematical problem-solving for middle school students. The system was evaluated with 14 U.S. middle-schoolers to assess engagement, motivation, and collaborative learning through simulated peer interactions with intentionally injected misconceptions.

This paper presents a Socratic chatbot fine-tuned on open-source LLMs (Llama2 7B/13B) designed to foster critical thinking in students through structured questioning rather than providing direct answers. The system is evaluated through simulated student-chatbot interactions to assess its effectiveness in promoting reflection and critical thinking compared to standard chatbots.

This paper presents Physics-STAR, an LLM-powered tutoring system for high school physics education, and evaluates it through a controlled experiment with 12 high school sophomores against traditional lectures and generic LLM tutoring. The system provides step-by-step guidance, reflective learning prompts, and personalized scaffolding to improve conceptual understanding and problem-solving skills in physics.

Mentigo is an AI mentor agent system designed to guide middle school students through creative problem solving (CPS) processes, providing scaffolding, personalized feedback, and Socratic questioning based on real classroom mentor-student interactions. The system was evaluated through comparative experiments with 12 students and reviewed by expert teachers, demonstrating improvements in student engagement and creative outcomes.

This paper evaluates the effectiveness of LLM-generated analogies for teaching scientific concepts (biology and physics) through controlled in-class tests with high school students and classroom field studies with teachers. The study measures student understanding, learning outcomes, potential over-reliance, and teacher satisfaction with LLM-generated analogies, culminating in the development of a practical system for teachers to generate and refine teaching analogies.

This paper presents a classroom study evaluating an LLM-augmented spherical visualization platform used with Swedish primary school children (ages 9-10) to explore Earth-related datasets through spoken natural language queries and coordinated visual-verbal responses. The study examines children's expectations, engagement patterns, and evaluations of the system in a formal educational context.

This paper develops and evaluates a pedagogical framework for LLM-driven feedback generation in secondary school Python programming education, aligning automated feedback with established pedagogical principles like mastery adaptation and progress-based scaffolding. Through mixed-method evaluation with eight secondary school computer science teachers, the study assesses how well LLM-generated feedback adheres to pedagogical standards compared to human teacher feedback.

This paper reports results from an exploratory RCT (N=165) evaluating LearnLM, a pedagogically fine-tuned AI tutor, in UK secondary school mathematics classrooms, where expert tutors supervised all AI-generated messages. Students receiving supervised AI tutoring performed at least as well as those with human tutors alone, with significantly better knowledge transfer to novel problems (66.2% vs 60.7% success rate).

This mixed-methods study directly evaluates ChatGPT-5 use with 109 secondary (age 16) students in Greek high schools, measuring attitudes (SATAI), anxiety (AIAS), and student responses to deliberately triggered hallucinations across multiple task modalities. The research identifies pedagogical affordances, constraints, and documents how students develop 'epistemic safeguarding' strategies after encountering incorrect AI outputs.

This paper presents RockStartIT Tutor, a GPT-4-powered AI tutoring system designed for secondary school students learning programming and computational thinking, using a curriculum-constrained knowledge base and pedagogically controlled prompting. The system was pilot-evaluated with 13 students and teachers using the Technology Acceptance Model to assess its effectiveness in providing scaffolded, personalized support.

This paper compares LLM-based tutors with human tutors on grade-school math word problems by having educators annotate and compare tutoring dialog snippets on engagement, empathy, scaffolding, and conciseness in a blind text-only setting. The study finds that educators with teaching experience perceive LLM tutors as performing better than human tutors across all four pedagogical dimensions.

This case study evaluates ChatGPT as a substitute teacher for 11th-grade chemistry students in the UAE, comparing student engagement and learning outcomes across cognitive domains (knowledge, application, reasoning) between ChatGPT-taught and traditionally-taught control groups using Bloom's taxonomy. The study found that while ChatGPT showed some promise in knowledge recall and reasoning, the control group significantly outperformed the experimental group, with double the percentage of students achieving good/outstanding results.

EducationQ is a multi-agent dialogue framework that evaluates LLMs' teaching capabilities through simulated teacher-student interactions, testing 14 models across 1,498 questions spanning 13 disciplines and 10 difficulty levels. The framework incorporates formative assessment principles to measure pedagogical effectiveness including questioning strategies, adaptive feedback, and scaffolding behaviors.

This qualitative action research study evaluates AI chatbots as auxiliary tools to help Brazilian high school students improve their argumentative ecology essays through iterative feedback and revision cycles. The research directly measures learning outcomes and engagement when students use chatbots for writing improvement in a classroom setting.

This paper evaluates three multi-agent LLM systems that embed the Knowledge-Learning-Instruction (KLI) framework to generate secondary Math and Science learning activities, comparing them against a baseline single-agent system through teacher evaluations and LLM-as-a-judge assessments using Quality Matters K-12 standards. The study demonstrates that collaborative multi-agent systems can produce more pedagogically sound, creative, and classroom-ready activities by encoding learning science principles directly into the AI architecture.

ConvoLearn introduces a dataset of 1,250 constructivist tutor-student dialogues in middle school Earth Science, grounded in knowledge-building theory across six pedagogical dimensions (cognitive engagement, formative assessment, accountability, cultural responsiveness, metacognition, and power dynamics). The paper demonstrates that fine-tuning LLMs on this dataset shifts model behavior toward constructivist teaching strategies, with the fine-tuned Mistral-7B significantly outperforming base models and Claude Sonnet in teacher evaluations.

This paper trains an open-source LLM (Llama 3.1 8B) to generate tutor utterances that maximize student learning outcomes in math tutoring dialogues by optimizing for both student response correctness and pedagogical quality using direct preference optimization. The approach uses a student model to predict correctness and GPT-4o to evaluate adherence to pedagogical principles, directly measuring the impact on student learning through dialogue interactions.

This paper develops and evaluates a dialogue-based intelligent tutoring system (ITS) for teaching sixth-grade students multiplication and division of fractions, using diagnostic teaching methodology with real-time error identification and adaptive instructional strategies. The system was tested through a quasi-experimental study with 66 sixth graders, showing significant learning gains compared to conventional classroom instruction.

This paper presents findings from the BEA 2025 Shared Task that evaluates pedagogical abilities of AI tutors powered by LLMs, specifically focusing on assessing tutor responses for mistake identification, guidance provision, and feedback actionability in mathematics education dialogues. The task established pedagogically-motivated evaluation tracks grounded in learning science principles to measure how effectively AI tutors remediate student mistakes through dialogue.

CODAE is a framework that fine-tunes open-source LLMs for AI tutoring by augmenting real student-tutor dialogues with Chain-of-Thought prompting to improve pedagogical quality. The paper addresses three key limitations (over-compliance, low response adaptivity, and threat vulnerability) and evaluates models on their ability to provide step-by-step guidance without prematurely revealing answers.

This paper evaluates the feasibility of using LLMs (GPT-4, Gemini, LearnLM) to automatically identify and assess two specific tutoring moves in real-life math tutoring dialogues: delivering effective praise and responding to student errors. The study analyzes 50 transcripts of college tutors working with middle school students, demonstrating that LLMs can reliably detect tutoring situations (94-98% accuracy for praise detection, 82-88% for error detection) and evaluate adherence to best practices (83-89% and 73-77% alignment with human judgment).

This paper presents an MPNet-based ensemble system for automatically evaluating AI tutor responses in educational dialogues, specifically assessing whether tutors correctly identify and locate student mistakes across two classification tasks at the BEA 2025 Shared Task.

This paper provides a comprehensive review of evaluation practices for LLM-powered Intelligent Tutoring Systems (ITSs), critically analyzing existing benchmarks and proposing three pedagogy-driven research directions for establishing unified, scalable evaluation methodologies grounded in learning science principles. It emphasizes cognitive offloading concerns, citing empirical studies showing students' over-reliance on AI tutors leading to reduced independent problem-solving skills.

This paper develops an automated system using GPT-3.5 to evaluate five key tutoring strategies (praise, error reaction, knowledge assessment, managing inequity, responding to negative self-talk) in one-on-one tutoring dialogues, classifying whether each strategy is employed effectively or ineffectively. The system analyzes tutoring transcripts to provide color-coded feedback on pedagogical quality of tutor-student interactions.

This paper develops a method using preference optimization to learn steering vectors that capture diverse tutoring personas from human tutor-student math dialogues, enabling LLMs to exhibit different instructional styles (scaffolding levels, affective support, directiveness) rather than a single tutoring approach. The method is evaluated on real K-12 math tutoring dialogues, showing improved alignment with ground-truth tutor behaviors while preserving pedagogical quality.

This paper introduces PedagogicalRL-Thinking, a reinforcement learning framework that trains LLM tutors to generate pedagogically appropriate reasoning traces (not just responses) by rewarding thinking processes grounded in Polya's problem-solving framework, with evaluation on mathematics tutoring dialogues measuring solution correctness, answer leakage prevention, and helpfulness.

This paper systematically compares AI-simulated tutoring dialogues with authentic human teacher-student dialogues using IRF coding and Epistemic Network Analysis, finding that human dialogues show superior pedagogical questioning, feedback, and cognitively-guided interaction patterns, while AI dialogues exhibit structural simplification and behavioral convergence. The work directly evaluates the pedagogical quality and instructional effectiveness of LLM-generated one-on-one tutoring interactions.

MetaCLASS introduces a framework and benchmark for evaluating LLM-based metacognitive tutoring that explicitly targets self-regulated learning processes (planning, monitoring, debugging, evaluation) through 11 interpretable coach moves, including productive restraint. The paper generates 1,015 annotated tutoring conversations and benchmarks nine LLMs on predicting appropriate metacognitive coaching actions, revealing systematic compulsive intervention bias where models fail to recognize when silence is pedagogically optimal.

This paper introduces Hierarchical Pedagogical Oversight (HPO), a multi-agent adversarial framework that evaluates AI tutor responses in K-12 mathematics dialogues by using specialist agents and structured debate to assess mistake identification and guidance quality. The system is validated on MRBench, a dataset of 1,214 middle-school mathematics tutoring dialogues, achieving superior performance in detecting sycophancy and inappropriate scaffolding.

This paper proposes ERL4SIIP, an evolutionary reinforcement learning framework for AI tutoring that uses Socratic questioning to teach STEM interdisciplinary concepts, specifically designed to avoid cognitive offloading by promoting active knowledge construction rather than direct answer delivery. The system explicitly addresses the 'spoon-feeding trap' and evaluates pedagogical quality through measures of knowledge integration, transfer, and strategic diversity.

This paper evaluates LLMs' mathematical reasoning by testing their ability to simulate novice learners making misconception-based errors and expert tutors diagnosing those misconceptions in grade-school math problems. The evaluation uses a dataset from Eedi's platform with multiple-choice questions tagged with specific mathematical misconceptions.

This paper benchmarks whether large language models (LLMs) can replicate the adaptivity of intelligent tutoring systems by systematically removing context components from 75 real-world tutoring scenarios and evaluating how three LLMs respond to student errors, knowledge states, and pedagogical requirements. The study finds that even the best-performing LLM only marginally mimics ITS adaptivity, with concerning patterns like GPT-4o providing overly direct feedback instead of effective Socratic questioning.

This paper introduces PACE, a personalized conversational tutoring agent for mathematics instruction that adapts teaching strategies based on students' learning styles using the Felder-Silverman model and employs Socratic teaching methods. The system is evaluated on its ability to identify individual learning styles and deliver personalized mathematics instruction through adaptive dialogue.

This paper evaluates ArtiBos, a gamified adaptive intelligent tutoring system designed for 9th-grade mathematics, examining its impact on students' problem-solving skills through system logs, student interviews, and teacher interviews in a Turkish high school setting.

This paper explores using GPT-4 to automatically contextualize math problems in CTAT (an intelligent tutoring system) to align with individual student interests at scale, aiming to increase engagement and learning outcomes. The authors use iterative prompt engineering to personalize problem contexts while preserving difficulty and pedagogical intent.

This study examines how conversational AI (LLM-based) supports K-12 mathematics educators during lesson preparation by analyzing educator-AI interactions across different educational contexts. The research evaluates AI's ability to adapt responses to educators' instructional needs, assessing accuracy, relevance, and usefulness of AI-generated pedagogical content and guidance.

This paper evaluates CyberScholar, a GenAI writing assistant that provides formative feedback aligned with teacher rubrics to K-12 students (grades 7-11) across multiple subject areas. The study examines student and teacher perceptions through surveys and interviews in four U.S. schools, focusing on how AI feedback supports writing improvement, metacognition, and self-regulation.