Rethinking AI Evaluation through TEACH-AI: A Human-Centered Benchmark and Toolkit for Evaluating AI Assistants in Education

As generative artificial intelligence (AI) continues to transform education, most existing AI benchmarks focus primarily on technical metrics (e.g., speed, accuracy) while overlooking human identity, agency, and ethical considerations. In this paper, we present TEACH-AI (Trustworthy and Effective AI Classroom Heuristics)—a domain-independent, pedagogically grounded, and stakeholder-aligned benchmark framework with measurable indicators and a practical toolkit to guide the design, development, and evaluation of generative AI systems in educational contexts. Built on an extensive literature review and synthesis, the ten-component assessment framework and toolkit checklist provide a foundation for scalable and value-aligned AI evaluation in education. The framework rethinks “evaluation” through sociotechnical, educational, theoretical, and applied lenses, engaging designers, developers, researchers, and policymakers across AI and education. Our work invites the community to reconsider what constitutes “effective” AI in education and to design benchmarks that promote co-creation, inclusivity, and long-term human, social, and educational impact.