Education Benchmarks and Evals Mapping

E-EVAL is a comprehensive Chinese K-12 education evaluation benchmark containing 4,351 multiple-choice questions across primary, middle, and high school levels in subjects including Chinese, English, Mathematics, Physics, Chemistry, and more, designed to assess large language models' knowledge and reasoning capabilities in the Chinese K-12 education domain. The benchmark is publicly available with dataset and code, and evaluates both English-dominant and Chinese-dominant LLMs across different grade levels and subject areas.

EduEval is a comprehensive hierarchical benchmark for evaluating LLMs in Chinese K-12 education, comprising 24 task types with over 11,000 questions organized across six cognitive dimensions (Memorization, Understanding, Application, Reasoning, Creativity, and Ethics) based on Bloom's Taxonomy and Webb's Depth of Knowledge. The benchmark incorporates authentic educational materials including real exam questions, classroom dialogues, student essays, and expert-designed prompts spanning primary through high school levels.

EduAdapt is a benchmark dataset of nearly 48k grade-labeled question-answer pairs spanning Grades 1-12 across nine science subjects, designed to evaluate whether LLMs can adapt their responses to different K-12 grade levels with age-appropriate vocabulary and explanations. The benchmark evaluates both multiple-choice and open-ended questions aligned to Next Generation Science Standards and includes publicly available code and datasets.

FEANEL is a benchmark dataset of 1,000 K-12 student essays (elementary and secondary) with fine-grained error annotations by language education experts, evaluating LLMs' ability to identify error types, assess severity, and provide pedagogical explanations for English writing errors. The benchmark uses a part-of-speech-based error taxonomy and evaluates state-of-the-art LLMs on their error analysis and feedback quality capabilities.

This paper presents LearnLM-Tutor, a fine-tuned Gemini model for education, and introduces a comprehensive evaluation framework consisting of seven diverse benchmarks (quantitative, qualitative, automatic, and human evaluations) grounded in learning science principles to assess pedagogical capabilities of AI tutoring systems. The work includes real-world deployment at Arizona State University's Study Hall and demonstrates that LearnLM-Tutor is consistently preferred by educators and learners over prompt-tuned Gemini across multiple pedagogical dimensions.

MDK12-Bench is a large-scale multimodal benchmark comprising 141K instances from real-world K-12 exams across six disciplines (math, physics, chemistry, biology, geography, information science), with 6,225 knowledge points organized in a taxonomy. It evaluates MLLMs across difficulty levels, temporal shifts, contextual shifts, and knowledge-driven reasoning, including a dynamic evaluation framework and knowledge-point reference-augmented generation (KP-RAG).

This paper reports an exploratory RCT with 165 UK secondary school students evaluating LearnLM, a pedagogically fine-tuned AI tutor supervised by human experts, comparing its effectiveness to human-only tutoring for mathematics learning on the Eedi platform. Students receiving LearnLM support demonstrated equal or better learning outcomes, including superior knowledge transfer to novel problems.

MinorBench is a hand-built, open-source benchmark that evaluates LLMs' ability to refuse unsafe or age-inappropriate content requests from children, using a taxonomy of content-based risks specific to minors derived from real middle-school chatbot deployment. The benchmark tests six prominent LLMs under different system prompts to assess child-safety compliance.

TutorBench is a benchmark dataset of 1,490 expert-curated samples evaluating LLMs' tutoring capabilities across three core tasks: generating adaptive explanations, providing actionable feedback, and creating effective hints for high-school and AP-level curricula. The benchmark uses sample-specific rubrics and LLM-judge evaluation to assess 16 frontier models, finding none achieve above 56% overall performance.

This paper evaluates how well Large Language Models (specifically GPT-4) can automatically grade short answer questions across different K-12 subjects (Science and History) and grade levels (ages 5-16) using a novel dataset from Carousel Learning, finding performance close to human-level marking (Kappa 0.70 vs 0.75). The study tests various prompt engineering strategies to assess LLM capabilities for formative assessment tasks.

This paper presents PhysicsAssistant, a multimodal LLM-powered robot designed to assist 8th-grade students during physics lab experiments by combining YOLOv8 object detection with GPT-3.5-turbo for natural language interaction. The study evaluates the system through a user study with 10 students, comparing its performance against GPT-4 and human expert ratings using Bloom's taxonomy.

FACET is a teacher-facing LLM-based multi-agent system that generates individualized mathematics worksheets for grade 8 students by incorporating both cognitive proficiency and intrinsic motivation profiles. The framework uses three specialized agents (learner, teacher, evaluator) and was validated through automated assessments and exploratory feedback from K-12 teachers.

This paper introduces AMMORE, a dataset of 53,000 middle-school math open-response question-answer pairs from an African WhatsApp-based tutoring platform, and evaluates LLM-based approaches (including chain-of-thought prompting) for automated grading of challenging student answers. The study demonstrates that LLM grading improves overall accuracy from 98.7% to 99.9% and significantly reduces misclassification of student mastery status in a Bayesian Knowledge Tracing model.

This paper describes the design and classroom evaluation of an LLM-based instructional module that teaches K-12 students prompting literacy through scenario-based practice with AI chatbots, including an AI auto-grader for evaluating student-written prompts. The study deployed the module across 11 secondary education classrooms in two iterations, measuring students' prompting performance improvements, confidence changes, and the effectiveness of different assessment question types.

This paper empirically evaluates the limitations of LLM-based tutoring systems for K-12 education by comparing a deep knowledge tracing (DKT) model against a widely-used LLM (with and without fine-tuning) on learner knowledge assessment accuracy, reliability, and temporal coherence using large-scale student interaction data. The study demonstrates that specialized learner modelling approaches substantially outperform LLMs in predicting student performance and maintaining consistent knowledge state estimates over time.

FoundationalASSIST is a K-12 educational dataset of 1.7M student interactions with full question text, actual student responses, and Common Core alignment, designed to evaluate LLMs on knowledge tracing (predicting student performance and exact answers) and pedagogical grounding (understanding properties that make assessment items effective). The paper demonstrates that current frontier LLMs struggle significantly on both task families, performing barely above trivial baselines on knowledge tracing and below random chance on item discrimination.

This paper evaluates LLM and statistical NLP methods for automated grading of nationwide school-leaving essay exams in Estonia, comparing their performance against human raters using curriculum-based rubrics. The study examines two full national cohorts of trial exams, assessing reliability, validity, bias, and the viability of human-in-the-loop automated scoring for high-stakes K-12 assessments.

This paper evaluates ChatGPT's ability to generate personalized reading comprehension exercises (passages and multiple-choice questions) for middle school English learners in China, comparing AI-generated materials against human-written textbook exercises through both automatic and manual evaluation by students, teachers, and native speakers.

This paper presents Bridge, a method that translates expert math educators' decision-making processes into a model for LLMs to remediate student math mistakes. The authors construct and publish a dataset of 700 real K-12 tutoring conversations (grades 1-5) annotated with expert decisions and conduct human evaluations comparing expert, novice, and LLM responses to student errors.

This paper presents MWPTutor, an LLM-based tutoring system that combines GPT-4 with finite state transducers to provide pedagogically-structured tutoring for math word problems, with guardrails to prevent answer leakage and maintain desired teaching strategies. The system is evaluated through human assessment comparing it to unstructured GPT-4 tutoring on two math word problem datasets.

MathVC is an LLM-simulated multi-persona virtual classroom platform designed to facilitate collaborative problem-solving in middle school mathematics through simulated peer interactions with diverse mathematical thinking and intentional misconceptions. The paper evaluates the system with 14 U.S. middle-schoolers, measuring engagement, motivation, confidence, and collaborative learning outcomes.

This paper presents a Socratic chatbot built by fine-tuning and prompt-tuning small open-source LLMs (Llama2 7B/13B) to foster critical thinking through questioning rather than direct answers, validated through simulated student-chatbot interactions that measure reflection and critical thinking development.

This paper investigates how pedagogical instructions facilitate scaffolding in LLM-based intelligent tutoring systems through a case study of guiding children to describe images for language learning. The authors construct different tutoring systems grounded in four learning theories and develop a seven-dimension rubric to evaluate the scaffolding process, including both manual and automated evaluation approaches.

This paper presents Physics-STAR, an LLM-powered tutoring system for high school physics education, and evaluates it through a controlled experiment with 12 sophomore students, comparing it against traditional teacher-led lectures and generic LLM tutoring. The study measures learning outcomes (conceptual, computational, and informational problem scores) and efficiency, finding significant improvements especially on complex information problems (100% score increase, 5.95% efficiency gain).

Mentigo is an AI agent system designed to mentor middle school students through the Creative Problem Solving (CPS) process, providing structured guidance, adaptive feedback, and Socratic questioning. The system was evaluated through a comparative experiment with 12 students and reviewed by five expert teachers, demonstrating improvements in student engagement and creative outcomes.

This paper evaluates LLM-generated analogies for teaching scientific concepts through a two-stage study with high school students and teachers, including controlled in-class tests and classroom field studies in biology and physics. The research finds that LLM-generated analogies can enhance student understanding particularly in biology, but require teacher guidance to prevent over-reliance and overconfidence, leading to the development of a practical system for teachers to generate and refine teaching analogies.

This paper presents a classroom study with 9-10 year old Swedish children evaluating an LLM-enabled spherical visualization platform that allows natural language questions about Earth-related datasets with coordinated verbal and visual responses. The study examines children's expectations, engagement patterns, and post-interaction evaluations through structured observation and group discussions in a formal educational setting.

This paper develops a pedagogical framework for LLM-driven feedback generation in programming education and evaluates it through a mixed-methods study with eight secondary-school computer science teachers using a web-based Python programming application. The study assesses whether LLM-generated feedback aligned with pedagogical principles (mastery adaptation, progress adaptation) can match or exceed human teacher feedback quality.

This quasi-experimental study with 110 secondary school students compares financial education outcomes between students using generative AI tools (ChatGPT) versus traditional teaching methods, finding significant improvements in financial literacy dimensions for the AI group.

This mixed-methods study examines 109 Greek secondary students' (age 16) attitudes, anxiety, and responses to hallucinated outputs when using ChatGPT-5 in classroom settings across multiple tasks. The research measures cognitive and behavioral attitudes toward AI, AI-related anxiety, and documents students' 'epistemic safeguarding' strategy of restricting AI use to domains where they can verify outputs after encountering hallucinations.

This paper describes the design and pilot evaluation of RockStartIT Tutor, a GPT-4-powered AI tutoring assistant for a secondary school programming course that uses structured prompts and a modular knowledge base to provide curriculum-constrained, pedagogically controlled support. The system was evaluated using the Technology Acceptance Model with 13 students and teachers, focusing on usability, learning support quality, and pedagogical appropriateness.

This paper develops a human-in-the-loop chain-of-thought prompting approach using GPT-4 to automatically score and generate explanations for middle school Earth Science formative assessment responses. The method combines few-shot learning, active learning, and chain-of-thought reasoning to evaluate open-ended short-answer student responses.

This case study evaluates ChatGPT as a substitute teacher for 11th-grade chemistry students in the UAE, comparing learning outcomes between experimental (ChatGPT) and control (traditional teacher) groups across Bloom's taxonomy cognitive domains. The study found that students in the control group performed significantly better (15% vs 7.5% good/outstanding ratings) and that ChatGPT faced challenges with student engagement and completion rates.

EducationQ is a multi-agent dialogue framework that evaluates LLMs' teaching capabilities through simulated teacher-student interactions, testing 14 LLMs across 1,498 questions spanning 13 disciplines and 10 difficulty levels. The framework incorporates formative assessment principles to measure pedagogical effectiveness including questioning strategies, adaptive feedback, and scaffolding behaviors.

EduGuardBench is a dual-component benchmark that evaluates LLMs as simulated teachers by measuring both professional fidelity (using Role-playing Fidelity Score to detect pedagogical harms like incompetence, indolence, and offensiveness) and adversarial safety (using persona-based jailbreak prompts to test vulnerability to harmful requests including academic misconduct). The benchmark tests 14 leading models and is publicly available with automated evaluation code.

ConvoLearn is a dataset of 1,250 semi-synthetic tutor-student dialogues in middle school Earth Science, grounded in constructivist knowledge-building theory and operationalizing six pedagogical dimensions (cognitive engagement, formative assessment, accountability, cultural responsiveness, metacognition, and power dynamics). The authors demonstrate that fine-tuning LLMs on this dataset shifts their behavior toward knowledge-building strategies, with their Mistral-7B model outperforming base models and Claude Sonnet 4.5 in teacher evaluations.

This paper evaluates ChatGPT's ability to solve decimal problems, grade student self-explanation responses, and generate instructional feedback within the Decimal Point learning game, using over 5,000 real student responses from middle school students. The study finds ChatGPT can accurately assess 75% of student answers and generate high-quality feedback, though it struggles with specific decimal place value problems.

This paper presents a framework for finetuning LLMs to generate grade-appropriate educational content across six grade levels (lower elementary to adult), evaluated through readability metrics and human studies with 208 participants. The work addresses the challenge of making AI tutoring systems produce content matched to students' comprehension levels across K-12 grades.

This paper presents MRBench, a benchmark for evaluating LLM-powered AI tutors' pedagogical abilities in mathematical dialogues, featuring 192 conversations across eight pedagogical dimensions (mistake identification, guidance provision, answer revealing, etc.) with gold human annotations. The taxonomy and dataset enable systematic assessment of whether AI tutors demonstrate sound pedagogical practices when remediating student mistakes.

This paper trains an open-source LLM (Llama 3.1 8B) to generate tutor utterances that maximize student learning outcomes in math tutoring dialogues by optimizing for both student response correctness and pedagogical quality using direct preference optimization. The approach uses a student model to predict correctness and GPT-4o to evaluate pedagogical principles, demonstrating that the trained tutor increases correct student responses while maintaining high pedagogical quality.

This paper develops and evaluates a dialogue-based mathematics intelligent tutoring system (ITS) for teaching multiplication and division of fractions to sixth graders, using diagnostic teaching methods to identify student misconceptions in real-time and provide adaptive instruction. A quasi-experimental study with 66 students showed the ITS significantly outperformed conventional classroom instruction, particularly for lower-performing students.

This paper presents the BEA 2025 Shared Task which evaluates pedagogical abilities of AI tutors (powered by LLMs) in educational dialogues, specifically assessing mistake identification, guidance provision, and feedback actionability in math tutoring contexts. The benchmark includes five evaluation tracks with publicly released datasets and annotation guidelines, achieving macro F1 scores ranging from 58.34 to 71.81 across pedagogical dimensions.

This paper presents an MPNet ensemble system for the BEA 2025 Shared Task that automatically classifies AI tutor responses in educational dialogues across two tracks: whether tutors correctly identify student mistakes (Track 1) and whether they locate the mistakes (Track 2). The system uses fine-tuned Transformer models with grouped cross-validation and hard-voting ensemble to achieve macro-F1 scores of 0.7110 and 0.5543 on the respective tracks.

This paper critically reviews evaluation practices for GenAI-powered Intelligent Tutoring Systems (ITSs), highlighting the lack of reliable, pedagogy-driven evaluation frameworks and benchmarks. It analyzes existing challenges through case studies and proposes three research directions for developing fair, unified, and scalable ITS evaluation methodologies grounded in learning science principles.

This paper introduces GuideEval, a benchmark for evaluating LLMs' ability to provide adaptive Socratic tutoring by assessing their capacity to perceive learner states, orchestrate instructional strategies, and elicit appropriate reflections based on authentic educational dialogues. The benchmark specifically measures whether LLMs can dynamically adjust pedagogical guidance in response to learners' cognitive states (confusion, comprehension, errors) rather than merely generating questions.

This paper introduces PedagogicalRL-Thinking, a reinforcement learning framework that trains LLM tutors by applying pedagogical rewards to both visible responses and internal reasoning traces, using domain-specific educational theory (Polya's problem-solving framework) to guide the model's thinking process. The approach is evaluated on mathematics tutoring dialogues with metrics measuring solution correctness, answer leakage prevention, and pedagogical helpfulness.

This paper compares AI-simulated tutoring dialogues with authentic human teacher-student dialogues in one-on-one instruction using IRF coding and Epistemic Network Analysis, finding that human dialogues exhibit more cognitively guided and diverse interactional patterns while AI dialogues show structural simplification and behavioral convergence. The study provides empirical evidence for limitations in current LLM-generated tutoring interactions and offers guidance for designing more pedagogically effective AI dialogue systems.

MetaCLASS introduces a benchmark for evaluating LLM-based metacognitive tutoring systems that support self-regulated learning through 11 interpretable coach moves (planning, monitoring, debugging, evaluation), including strategic restraint. The framework generates 1,015 annotated tutoring conversations and tests 9 LLMs on predicting pedagogically appropriate metacognitive moves, revealing systematic over-intervention bias where models fail to recognize when silence promotes productive struggle.

This paper introduces Hierarchical Pedagogical Oversight (HPO), a multi-agent adversarial framework that uses structured debate between specialist agents to evaluate AI tutor responses for pedagogical quality, specifically detecting sycophancy and overly direct answers. The framework is evaluated on the MRBench dataset of 1,214 middle-school mathematics dialogues, achieving superior performance in classifying mistake identification and guidance quality compared to GPT-4o.

This paper proposes ERL4SIIP, an Evolutionary Reinforcement Learning framework for AI tutors that implements Socratic teaching methods in interdisciplinary STEM education, specifically designed to foster higher-order cognitive abilities like knowledge integration and critical thinking while avoiding 'spoon-feeding' behaviors. The system uses dynamic student modeling, hierarchical rewards, and combines evolutionary algorithms with PPO to generate diverse teaching strategies that promote active learning rather than passive answer delivery.