- Source
- arXiv
- Published
- Runtime
- 0:00
- Snippets
- 5
A conversation between
K12-KGraph: A Curriculum-Aligned Knowledge Graph for Benchmarking and Training Educational LLMs
§02
Snippets
-
Curriculum cognition—understanding prerequisite chains, concept taxonomies, and pedagogical sequencing—is largely unmeasured by existing educational benchmarks.
This gap means we can't assess whether educational LLMs truly support learning versus just pattern-matching test answers.
-
K12-KGraph extracts nine node types and fourteen relation types from official textbooks across math, physics, chemistry, and biology, capturing both conceptual structure and visual grounding.
A structured, curriculum-aligned graph enables benchmarks and training data that reflect how teachers actually organize knowledge.
-
Gemini-3-Flash achieves only 57% exact match and Gemma-4-31B-IT reaches 46% on K12-Bench; prerequisite and neighbor tasks are the hardest.
Even frontier models struggle with curriculum cognition, revealing a significant capability gap in educational reasoning.
-
Domain-specific supervision from K12-Train consistently outperforms equally sized subsets of eight mainstream instruction-tuning corpora on educational benchmarks.
Targeted, curriculum-aligned training data is more efficient than scaling generic corpora for educational tasks.
-
Multimodal supervision from K12-Train-Full (combining text QA and visual VQA) achieves best results on vision-language benchmarks and outperforms text-only or visual-only variants.
Textual and visual signals in education are genuinely complementary, justifying the effort to collect both modalities.
§03
Synthesis
The Problem: Testing What Educational LLMs Actually Need to Know
Most benchmarks for educational AI focus on a narrow skill—answering exam questions correctly. But teaching involves much deeper structural knowledge: knowing that concepts build on prerequisites, understanding how ideas connect visually in textbooks, and grasping why concepts are sequenced a particular way. Existing large language models (Gemini-3-Flash, Gemma-4-31B) fail badly at these tasks, achieving only 46–57% accuracy. The authors call this missing capability "curriculum cognition" and argue it's essential for LLMs deployed in K-12 education.
What They Built
The authors extracted a knowledge graph directly from official Chinese K-12 textbooks (People's Education Press) across mathematics, physics, chemistry, and biology from primary through high school. This graph captures nine types of nodes (concepts, skills, experiments, visual elements, etc.) and fourteen relation types (prerequisites, taxonomic groupings, experiment-to-concept links, pedagogical ordering, and visual positioning).
From this graph, they created two resources:
K12-Bench: A 23,640-question multiple-select benchmark organized into five task families. "Ground" tests whether the model can locate concepts in visual diagrams. "Prereq" asks which concepts must come first. "Neighbor" tests immediate conceptual relationships. "Evidence" requires finding supporting information within the graph. "Locate" pins concepts to specific textbook sections. The hardest tasks are Prereq and Neighbor, where models struggle most.
K12-Train: A supervised fine-tuning corpus of 7,335 samples—2,267 text-only question-answer pairs plus 5,068 multimodal (image + text) pairs. These are generated systematically from the graph to teach models curriculum structure.
Why This Matters
The experiments reveal two critical findings. First, domain-specific training works: when comparing equal-sized training sets (2,300 samples), K12-Train's text-only version consistently beats eight mainstream instruction-tuning corpora on established educational benchmarks (GaokaoBench, EduEval). This shows that curriculum-aligned data is more valuable per sample than generic supervised learning data.
Second, for vision-language models, combining text and visual supervision is essential. K12-Train-Full achieved best overall results on three multimodal benchmarks (Gaokao-MM, MDK12-medium, K12Vista) using fewer samples than full DataFlow and WizardLM baselines. Textual and visual understanding are complementary—neither alone captures curriculum cognition fully.
The 46–57% baseline accuracy on K12-Bench, with specific tasks like Prereq reaching near-random performance, establishes a meaningful gap to close. The authors release the knowledge graph, benchmark, training data, and construction pipeline, creating infrastructure for future work on curriculum-aware educational AI.
This work reframes what "understanding" means for educational models: not just answering questions, but grasping how knowledge is structured, visually presented, and pedagogically ordered in actual curricula.
Mine your own.
Lode is a workbench, not a feed. Paste a YouTube URL. The model proposes a transcript, a set of quote-grounded snippets, a synthesis essay, and the fan-out. You decide what stays.