Learning Transferable Cognitive Structures for Process-Oriented Creativity Assessment in STEM Student-AI Dialogue
Yan Sha
Abstract
Large language models are increasingly used as creativity-augmenting tools in education, yet most evaluation methods still score final products rather than the cognitive processes humans exhibit during collaboration with AI. We study whether LLMs can be trained as process-oriented creativity assessors that learn transferable cognitive structures instead of source-domain surface cues. We formalize creativity assessment in human-AI scientific inquiry dialogues as a four-dimensional ordinal scoring task grounded in cognitive psychology, covering problem reframing, interdisciplinary integration, risk-driven innovation, and resource integration. We train CREDO on 10{,}200 expert-annotated dialogues with multi-task scoring and explanation generation, using Dimension-Aware Curriculum, Plasticity-Aware LoRA, and Creativity Contrastive Learning. CREDO reaches a Quadratic Weighted Kappa (QWK) of 0.730 on the source-domain test set, approaching the human-expert consistency ceiling of 0.810. Under $k \in \{0, 5, 10, 20, 50, 100\}$ in-context learning, the best frontier prompting baseline reaches only QWK 0.582, comparable to trained student raters. CREDO further transfers zero-shot to unseen disciplinary domains, achieving QWK 0.65 in mathematics and 0.63 in computer science, while its inter-dimension correlation pattern closely matches human experts with cosine similarity 0.96. These results suggest that LLMs can learn expert-aligned, transferable structures for assessing creative processes in human-AI dialogue.
Chat is not available.
Successful Page Load