A LLM-Powered Automatic Grading Framework with Human-Level Guidelines Optimization

GradeOpt is an LLM-powered automatic short-answer grading framework that uses multi-agent systems to grade open-text responses and automatically optimize grading guidelines through self-reflection. The system is evaluated on datasets measuring teachers' pedagogical knowledge and students' learning progress in mathematics and physical science.

Open-ended short-answer questions (SAGs) have been widely recognized as a powerful tool for providing deeper insights into learners' responses in the context of learning analytics (LA). However, SAGs often present challenges in practice due to the high grading workload and concerns about inconsistent assessments. With recent advancements in natural language processing (NLP), automatic short-answer grading (ASAG) offers a promising solution to these challenges. Despite this, current ASAG algorith

Source

Framework Categories

Tool Types

Tags