Mouse Total Capture: A multi-view Dataset for 3D Motion and Expression Capture of Freely Moving Mouse
Abstract
3D motion and expression capture of laboratory mice is a critical task in behavioral neuroscience, yet it remains underexplored in the computer vision community. Existing studies have focused narrowly on either body pose estimation of freely moving mice or facial expression capture under head-fixed conditions, leaving a significant gap: no dataset supports simultaneous capture of whole-body motion and expression in a freely moving mouse. To fill this gap, we present PanoMouse, a multiview system and dataset for Mouse Total Capture (MTC). The PanoMouse system deploys 24 cameras across three horizontal layers, achieving 360-degree photographic coverage of the mouse. Built upon this system, the PanoMouse dataset collects over 3.4 million frames, with 7,293 frames annotated with 92 whole-body keypoints, totaling approximately 670,000 keypoint annotations. To our knowledge, PanoMouse is the first dataset to provide fine-grained annotations of a mouse in natural behavior. We further propose 4D Triangulation (Triang4D), a test-time optimization method for 3D MTC that jointly enforces multi-view consistency, temporal smoothness, and bone length constraints. Applying Triang4D to the entire dataset produces 3D whole-body keypoint sequences across all recording sessions, enabling downstream skeleton-based behavior analysis. We benchmark baseline methods on PanoMouse across three tasks: 2D pose estimation, 3D pose estimation, and behavior classification, opening a new paradigm for fine-grained behavior analysis and bridging computer vision with behavioral neuroscience.