Characterizing LLM Artificial Societies using Standard Cross-Cultural Sample

LLM-based social simulations can generate complex, multifaceted societies. This generative capacity creates a distinctive evaluation challenge: prespecified behavioral or collective measures capture only selected aspects of the resulting society and provide no common basis for characterizing societies across different simulations. We address this challenge with a society-level evaluation framework based on the Standard Cross-Cultural Sample (SCCS), which represents simulated and real societies using shared cultural and social variables. The framework uses an LLM annotator to construct SCCS profiles, enabling comparisons among societies. We apply the framework to 24 artificial societies generated under eight simulation conditions. The resulting profiles reveal distinct social configurations that are often unusual relative to real societies, including characteristics not directly specified by the simulation. Matched-pair comparisons further show that different design choices affect distinct societal characteristics. These results demonstrate how SCCS-based profiling can complement phenomenon-specific validation by characterizing artificial societies as multidimensional social configurations and grounding their comparison in the same empirical space as real-world societies.