SeMi-Depth: Semantic-Guided Monocular Metric Depth Estimation Across Diverse Scenes

Monocular depth estimation (MDE) recovers dense depth from a single RGB image and is central to applications such as autonomous driving, robotics, and augmented reality. Yet building a single model that predicts metric depth across heterogeneous environments remains difficult: indoor and outdoor scenes differ systematically in depth range, geometry, and illumination, so priors learned in one domain transfer poorly to the other. Most existing methods either specialize to one domain or rely on test-time median scaling, which is unavailable in real deployment. We propose SeMi-Depth, a unified multi-task framework that combines dense semantic guidance with explicit domain conditioning. A shared Swin-T backbone feeds three branches: a semantic segmentation branch, a domain head that predicts the scene type (indoor or outdoor) along with a domain embedding, and a depth branch. A cross-attention module lets depth features query semantic features, so that depth discontinuities align with object boundaries, and Feature-wise Linear Modulation (FiLM) conditions the fused features on the domain embedding. A DPT decoder with a domain-adaptive inverse-depth bins head then outputs metric depth directly, with no test-time scale calibration. On Cityscapes (0–80m), SeMi-Depth achieves an unscaled RMSE of 5.39m, outperforming median-scaled baselines. On ScanNet, it reaches an RMSE of 0.311m, surpassing Swin-L models such as PixelFormer and NeWCRFs while being ≈5× smaller (44.9M parameters) and running in real time at 41 FPS. It also attains 99.7% domain-classification accuracy and transfers zero-shot to KITTI and NYU Depth V2, demonstrating strong generalization to unseen datasets.