Multi-modal 3D Human Pose Estimation using mmWave, RGB-D, and Inertial Sensors

The ability to estimate 3D human body pose and movement, also known as human pose estimation~(HPE), enables many applications for home-based health monitoring, such as remote rehabilitation training. Several possible solutions have emerged using sensors ranging from RGB cameras, depth sensors, millimeter-Wave (mmWave) radars, and wearable inertial sensors. Despite previous efforts on datasets and benchmarks for HPE, few dataset exploits multiple modalities and focuses on home-based health monitoring. To bridge this gap, we present human pose estimation using multiple modalities with an in-house dataset. We perform extensive experiments and delineate the strength of each modality.