Multimodal capabilities

MDK12-Bench is a large-scale benchmark built from 141K real-world K-12 exam questions across six disciplines (Math, Physics, Chemistry, Biology, Geography, Information Science) with 6,225 structured knowledge points, designed to evaluate multimodal large language models on problem-solving capabilities across difficulty levels, temporal shifts, contextual shifts, and knowledge-driven reasoning. The benchmark includes a dynamic evaluation framework and knowledge-point reference-augmented generation (KP-RAG) to assess model generalization and the role of knowledge in problem-solving.

This paper presents PhysicsAssistant, a multimodal robot combining YOLOv8 object detection with GPT-3.5-turbo to provide real-time interactive assistance to 8th-grade students during physics lab experiments. The system is empirically evaluated through a user study with 10 students, where expert ratings based on Bloom's taxonomy assess the quality of responses compared to GPT-4.

MathVC is an LLM-simulated multi-persona virtual classroom that creates AI-powered peer agents to facilitate collaborative mathematical problem-solving for middle school students. The system was evaluated with 14 U.S. middle-schoolers to assess engagement, motivation, and collaborative learning through simulated peer interactions with intentionally injected misconceptions.

This paper presents a classroom study evaluating an LLM-augmented spherical visualization platform used with Swedish primary school children (ages 9-10) to explore Earth-related datasets through spoken natural language queries and coordinated visual-verbal responses. The study examines children's expectations, engagement patterns, and evaluations of the system in a formal educational context.

This paper reports results from an exploratory RCT (N=165) evaluating LearnLM, a pedagogically fine-tuned AI tutor, in UK secondary school mathematics classrooms, where expert tutors supervised all AI-generated messages. Students receiving supervised AI tutoring performed at least as well as those with human tutors alone, with significantly better knowledge transfer to novel problems (66.2% vs 60.7% success rate).

This paper compares LLM-based tutors with human tutors on grade-school math word problems by having educators annotate and compare tutoring dialog snippets on engagement, empathy, scaffolding, and conciseness in a blind text-only setting. The study finds that educators with teaching experience perceive LLM tutors as performing better than human tutors across all four pedagogical dimensions.

This paper develops and evaluates a dialogue-based intelligent tutoring system (ITS) for teaching sixth-grade students multiplication and division of fractions, using diagnostic teaching methodology with real-time error identification and adaptive instructional strategies. The system was tested through a quasi-experimental study with 66 sixth graders, showing significant learning gains compared to conventional classroom instruction.

This paper develops a method using preference optimization to learn steering vectors that capture diverse tutoring personas from human tutor-student math dialogues, enabling LLMs to exhibit different instructional styles (scaffolding levels, affective support, directiveness) rather than a single tutoring approach. The method is evaluated on real K-12 math tutoring dialogues, showing improved alignment with ground-truth tutor behaviors while preserving pedagogical quality.

This paper evaluates ArtiBos, a gamified adaptive intelligent tutoring system designed for 9th-grade mathematics, examining its impact on students' problem-solving skills through system logs, student interviews, and teacher interviews in a Turkish high school setting.

This paper reports on aiPlato, an AI-enabled homework platform for introductory college physics that provides stepwise feedback and iterative guidance through tools like 'Evaluate My Work' and 'AI Tutor Chat'. The exploratory classroom study analyzes student engagement patterns, correlations with final exam performance, and student perceptions of the platform's learning value while preserving productive struggle.

This paper evaluates 63 middle school students (ages 14-15) using ChatGPT for science investigation tasks, analyzing their question-asking skills, answer-evaluation abilities, and learning outcomes. The study finds students over-relied on ChatGPT, struggled with effective questioning and critical evaluation, and achieved only moderate learning gains despite unrestricted AI access.

MusicScaffold is a generative AI framework designed for adolescent music education (ages 12-14) that positions AI as a guide, coach, and partner to support creative expression while preventing cognitive offloading. The four-week study evaluates how the system scaffolds cognitive growth (vague emotions to structured strategies), behavioral growth (passive use to active self-regulation), and affective growth (frustration to confident authorship).

This paper presents ZPDES, an AI-driven intelligent tutoring system that personalizes exercise sequencing based on learning progress and multi-armed bandit algorithms, and evaluates its impact on both learning outcomes and intrinsic motivation in a large-scale randomized controlled trial with 265 children aged 7-8. The study specifically examines how incorporating learner choice affects learning performance and motivation when combined with adaptive personalization versus fixed curricula.

This paper presents Logic-Muse, an Intelligent Tutoring System (ITS) that teaches logical reasoning skills through adaptive activities, using a hybrid student modeling approach combining Bayesian Knowledge Tracing and Deep Knowledge Tracing evaluated with ~300 students across 48 reasoning tasks. The system tracks student acquisition of reasoning skills (inhibition, counterexample generation, model management) across familiar, counterfactual, and abstract contexts using expert-validated Bayesian networks and neural architectures.

This paper evaluates state-of-the-art VLMs and LLMs on grading over 14,000 handwritten answers from Indonesian Grade 4 students in Mathematics and English, and generating personalized feedback. The study introduces a multimodal pipeline integrating vision models for handwriting recognition and language models for rubric-based assessment in real-world, underrepresented classroom settings.

AgentTutor is an LLM-powered multi-turn interactive tutoring system that uses generative agents to deliver personalized instruction by decomposing curricula, assessing learner cognitive levels based on Bloom's Taxonomy, dynamically adjusting teaching strategies, and providing reflective feedback through multi-turn dialogue. The system evaluates its effectiveness on benchmark datasets by measuring learning outcomes and teaching quality in interactive educational contexts.

This paper presents an AI-powered Python programming chatbot designed for students that combines static code analysis, dynamic execution tracing, and LLMs (CodeLlama, GPT-4) to provide debugging assistance and educational support. The system was evaluated with 1,500 student submissions showing 85% error resolution success, 59.3% reduction in debugging time, and 34% improvement in coding proficiency through pre/post-test assessments.

HypoCompass is an LLM-powered intelligent tutoring system where students play teaching assistants who help debug LLM-generated buggy code, focusing on hypothesis construction skills in CS1 programming courses. The system was evaluated with 19 novices showing 12% improvement in debugging performance and 14% reduction in completion time.

This paper develops and evaluates scaffolding techniques for multi-modal intelligent tutoring systems that guide children in describing images for language learning, using GPT-4V with pedagogical instructions grounded in four learning theories. The work constructs a seven-dimension rubric to evaluate the scaffolding process both manually and automatically.

ChatTutor is a full-fledged LLM-based intelligent tutoring system that provides course planning, instruction, and adaptive quizzes through three interconnected processes (interaction, reflection, reaction) with dynamic memory management. The system is evaluated through user studies and statistical analysis comparing it with real-world online learning platforms, measuring consistency, stability, and pedagogical effectiveness.

MalAlgoQA introduces a dataset of K-12 mathematics and reading comprehension questions (grades 3-11) with rationales for both correct answers and incorrect answer choices ('malgorithms'), designed to evaluate LLMs' counterfactual reasoning abilities by testing whether they can identify the flawed reasoning that leads to wrong answers. The paper evaluates multiple state-of-the-art LLMs and finds significant performance drops when identifying malgorithms versus correct algorithms, with important implications for AI tutoring systems that need to diagnose student misconceptions.

This paper presents Ruffle&Riley, an LLM-based conversational tutoring system using a learning-by-teaching format with two AI agents (student and professor), and evaluates it through two online studies (N=200) measuring learning outcomes, engagement, and user experience in biology lessons compared to simpler chatbots and reading activities.

This paper introduces PBLBench, a benchmark for evaluating multimodal large language models (MLLMs) on their ability to assess STEM Project-Based Learning (PBL) outcomes, including evaluating research reports, code, experimental data, and videos using expert-validated criteria derived through the Analytic Hierarchy Process (AHP). The benchmark tests 15 MLLMs on grading and ranking PBL projects across multiple STEM disciplines with long-context multimodal inputs.

This study examines how AI-directed speech recognition technology (generating real-time word clouds from oral discussions) scaffolds knowledge co-construction among pre-service teachers in a science education context, analyzing the process through social system self-organization theory. The research investigates how visual artifacts from speech recognition trigger self-organization characteristics like increased information exchange, far-from-equilibrium states, and crystallization of group thinking.

This paper proposes UCO, a multi-turn interactive reinforcement learning method that trains large language models to become adaptive AI tutors by tracking students' cognitive states through the Zone of Proximal Development framework and rewarding teaching strategies that promote genuine understanding rather than answer repetition. The method is evaluated on mathematics tutoring benchmarks (BigMath and MathTutorBench) and demonstrates performance comparable to closed-source models.

IntelliChain is a framework that integrates LLMs with knowledge graphs to enable Socratic method dialogue for mathematics education, using a multi-agent system with chain-of-thought reasoning to provide personalized tutoring interactions. The system was empirically tested in K-12 mathematics teaching contexts to enhance educational accuracy and learning outcomes.

This paper demonstrates how to customize Google's Gemini LLM into a Socratic physics tutor through 'role engineering' (providing detailed instructions and persona scripts), and evaluates its ability to engage in pedagogical dialogue rather than directly providing solutions. The work includes two case studies showing the tutor's multimodal capabilities (analyzing hand-drawn diagrams) and its ability to guide conceptual reasoning in physics problem-solving through Socratic questioning.

This paper presents and empirically validates the Odychess approach, a dialectic and constructivist methodology for teaching chess using generative AI (specifically a fine-tuned Llama 3.3 model acting as a Socratic tutor). A quasi-experimental study with 60 students demonstrates significant improvements in chess knowledge, strategic understanding, and metacognitive skills compared to a control group.

This paper proposes a framework for simulating students with different personality traits and language abilities to train and evaluate conversational Intelligent Tutoring Systems, with a case study on image description for language learning in primary school students. The work includes multi-aspect validation of how LLM-based tutors adapt scaffolding strategies to different student personalities.

MDK12-Bench is a large-scale multimodal reasoning benchmark comprising 141K questions across six K-12 disciplines (math, physics, chemistry, biology, geography, information science) spanning grades 1-12, designed to evaluate MLLMs' reasoning capabilities using real-world K-12 examination questions with fine-grained knowledge annotations and difficulty labels.

This paper introduces STEM, a large-scale multimodal vision-language benchmark with 1,073,146 questions spanning 448 skills across science, technology, engineering, and math subjects, specifically designed based on K-12 curriculum (Pre-K through grade 8). The benchmark evaluates state-of-the-art AI models (GPT-3.5-Turbo, CLIP) on fundamental STEM skills, finding they perform well below elementary student levels (averaging 54.7%).

This paper develops and evaluates an LLM-powered pipeline (using GPT-4o) that transcreates English reading comprehension passages and questions to align with individual EFL students' interests, maintaining linguistic complexity while adapting semantic content. A controlled experiment with South Korean EFL learners demonstrates that personalized, interest-aligned reading materials improve both comprehension and motivation retention compared to non-personalized materials.

ViScratch is an automated feedback system for Scratch programming that uses vision-language models to diagnose and repair bugs by analyzing both block code and gameplay videos, demonstrating substantial improvements over prior LLM-based debugging tools in identifying and fixing semantic errors in student projects.

KidsArtBench introduces a benchmark of 1,046 children's artworks (ages 5-15) annotated by expert educators across 9 rubric-aligned dimensions, designed to evaluate MLLMs' ability to provide multi-dimensional assessment and formative feedback on student artwork. The paper proposes an attribute-aware multi-LoRA approach with regression-aware fine-tuning to align AI predictions with ordinal evaluation scales used in art education.

This paper presents a system that uses LLMs to transform multimodal data (eye-tracking, assessment scores, and teaching standards) into actionable reading assessment reports for K-12 teachers. The system employs unsupervised clustering to identify reading behavior patterns and LLMs to generate teacher-friendly reports that are evaluated by both LLM experts and human educators.

This qualitative study examines how undergraduate students interact with AI tools (ChatGPT, Grammarly, Khan Academy) during learning tasks, connecting observed interaction patterns (directive, assistive, dialogic, empathetic) to established educational learning theories (behaviorism, cognitivism, constructivism, humanism). The research identifies five themes including cognitive scaffolding, dialogic engagement, and learning agency through semi-structured interviews.

This paper proposes Language model-based Knowledge Tracing (LKT), a novel framework that integrates pre-trained language models with knowledge tracing methods to predict student knowledge states and learning outcomes in online learning platforms. LKT leverages semantic information from question text to outperform traditional numerical-sequence-based knowledge tracing models and addresses the cold-start problem.

ES-KT-24 is a multimodal Knowledge Tracing benchmark dataset derived from educational game-playing videos covering K-12 Mathematics, English, Indonesian, and Malaysian subjects, containing 15,032 users and 7.7M interactions with synthetically generated question texts and game logs. The paper benchmarks deep learning-based KT models against language model-based approaches to predict student performance in game-based learning environments.

This paper introduces KT-PSP, a knowledge tracing formulation that incorporates students' problem-solving processes in mathematics, and proposes StatusKT, an LLM-based framework that extracts mathematical proficiency indicators from students' written solution processes to predict future performance and provide interpretable assessments of student understanding.

This mixed-methods study evaluates ChatGPT's effectiveness in improving writing skills among grade XI vocational high school students in Indonesia over two months, measuring engagement, interactivity, and learning outcomes through pre/post assessments, surveys, and interviews. The research examines both quantitative writing performance improvements and qualitative aspects of student engagement with AI-assisted writing instruction.

This paper evaluates a virtual learning companion (VLC) called 'Courage' that helps high school students develop critical thinking skills for identifying manipulated images and fake news on social media through Socratic questioning and reverse image search. An empirical classroom study with 22 German high school students assessed how the VLC's guidance improved students' ability to judge real versus manipulated social media content.

This paper presents SCB-Dataset, a comprehensive computer vision dataset of 13,330 images with 122,977 labels for detecting 19 classes of student and teacher classroom behaviors using object detection and image classification methods. The dataset enables automatic analysis of classroom performance by detecting behaviors like hand-raising, listening, reading, teaching, and student-teacher interactions.

This paper is a systematic review of large language models (LLMs) used as virtual tutors in education, examining applications including automatic question generation, student assessment, feedback provision, and personalized learning across K-12 and higher education contexts. The review analyzes how LLMs like GPT-3 and BERT are deployed for educational tasks such as generating explanations, grading answers, and providing tutoring support.

This mini review examines automatic item generation (AIG) methods for creating educational assessment questions, covering traditional rule-based approaches and machine learning/NLP techniques. The paper discusses item model development, quality evaluation frameworks, and challenges in scaling AIG for K-12 assessments across subject areas including math, science, and language arts.

This paper presents Tailor-Mind, an LLM-powered interactive visualization system designed to support self-regulated learning for AI beginners through personalized tutoring, adaptive question recommendations, and knowledge mapping. The system fine-tunes LLMs for educational dialogue and scaffolding, with evaluation showing improved self-regulated learning outcomes.

This paper introduces the VNHSGE dataset, comprising over 19,000 multiple-choice questions and 300 essays across nine subjects from Vietnamese high school graduation examinations, designed to benchmark large language models' performance against student-level competencies. The authors evaluate ChatGPT and BingChat on this dataset, comparing their performance to Vietnamese students across subjects including mathematics, literature, sciences, and humanities.

CMMaTH is a Chinese multimodal mathematics benchmark containing 23,000 K-12 math problems (elementary to high school) designed to evaluate foundation models' ability to solve visual-mathematical problems across different grade levels and knowledge points. The paper introduces GradeGPT, an open-source evaluation tool for assessing model performance on this dataset.

This paper evaluates GPT-4o's multimodal and multilingual performance on physics concept inventories spanning multiple subjects and languages, comparing AI performance to undergraduate student benchmarks. The study assesses general reasoning, content knowledge mastery, and multimodal/multilingual capabilities using standardized physics assessments.

VisScience is a benchmark dataset of 3,000 K-12 multi-modal questions spanning mathematics, physics, and chemistry (elementary through high school), designed to evaluate multi-modal large language models' scientific reasoning capabilities across 21 subjects and 5 difficulty levels. The paper evaluates 25 MLLMs on this benchmark, finding closed-source models generally outperform open-source ones, with best accuracies of 53.4% (math), 38.2% (physics), and 47.0% (chemistry).

This R&D study develops and evaluates 'Zeno', an AI chatbot designed to teach Solar System content and English vocabulary to grade 6 elementary students in Indonesia, using multimodal content (text, images, videos). The study reports that 90% of 30 students felt motivated and assisted in learning through the chatbot based on questionnaire responses.