Adjusting the Gender Wage Gap with a Low-Dimensional Representation of Job History

To estimate adjusted gender wage gaps, economists build models that predict wage from observable data. Although an individual's complete job history may be predictive of wage, economists typically summarize experience with hand-constructed summary statistics about the past. In this work, we estimate the adjusted gender wage gap for an individual's entire job history by learning a low-dimensional representation of their career. We develop a transformer-based representation model that is pretrained on massive, passively-collected resume data that is then fine-tuned to predict wages on the small, nationally representative survey data that economists use for wage gap estimation. This dimension-reduction approach produces unbiased estimates of the adjusted wage gap as long as each representation corresponds to the same full job history for males and females. We discuss how this condition relates to the sufficiency fairness criterion; although the adjusted wage gap is not a causal quantity, we take inspiration from the high-dimensional confounding literature to assess and mitigate sufficiency. We validate our approach with experiments on semi-synthetic and real-world data. Our method makes more accurate wage predictions than economic baselines. When applied to wage survey data in the United States, our method finds that a substantial portion of the gender wage gap can be attributed to differences in job history, although this proportion varies by year and across sub-populations.