Seeing Inside Buildings: Leveraging Generative AI and Multimodal Data to Automate Building Material Audits

Building retrofits and deconstruction projects are expected to surpass new construction jobs globally, putting pressure on architects and developers to sustainably reuse buildings and their materials (World Economic Forum). However, building reuse and material upcycling is hindered by the complexity of building material audits, which require expensive tests, site visits, and detailed data that is often missing for old structures. We present a Generative AI approach to predict the structural and material make-up of existing buildings from multimodal geospatial, technical, and cadaster data. Leveraging a dataset of 100 buildings across the United States with corresponding building 3D scans, geolocation, and construction data, we demonstrate the capability of a stable diffusion model to reliably predict structural diagrams for subsequent estimation of material contents. This process also offers designers actionable potential material reuse data to streamline and accelerate circularity for existing building design.