Synchronizing Verbal Responses and Board Writing for Multimodal Math Instruction with LLMs

The advancement of large language models (LLMs) has greatly facilitated math instruction, with the generated textual content serving as verbal responses to address student inquiries. However, in instructional settings, teachers often provide both verbal responses and board writing (BW) simultaneously to enhance students' knowledge construction. To address this, we introduce MathBoard, a multimodal large language model (MLLM) designed for elementary mathematics education, which progressively generates BW. Our study focuses on the provision of BW to learners, aiming to reduce their cognitive load effectively. Furthermore, MathBoard can be integrated with other approaches that enhance mathematical reasoning capabilities. An empirical study involving 34 pre-service teachers demonstrated that the multimodal interactions facilitated by MathBoard were more highly accepted and impactful across various dimensions compared to text-only interactions, significantly promoting learners' social construction of knowledge.