Timezone: »
Addressing observational biases in algorithmic fairness assessments
Chirag Nagpal · Olawale Salaudeen · Sanmi Koyejo · Stephen Pfohl
The objective of this early work is to characterize the implications for model development and evaluation of observational biases with subgroup-dependent structure (including selection bias, measurement error, label bias, or censoring that differs across subgroups), extending work that aims to characterize and resolve conflicts among statistical fairness criteria in the absence of such biases. These biases pose challenges because naive approaches to model fitting produce statistically biased results, with potential fairness harms induced by systematic, differential misestimation across subgroups, and it is challenging, or impossible in some contexts, to detect such biases without additional data or domain knowledge. As an example, we present an illustrative case study in the setting with differential censoring across subgroups.
Author Information
Chirag Nagpal (Carnegie Mellon University)
Olawale Salaudeen (University of Illinois, Urbana-Champaign)
Sanmi Koyejo (UIUC)
Stephen Pfohl (Google)
More from the Same Authors
-
2021 : Maintaining fairness across distribution shifts: do we have viable solutions for real-world applications? »
Jessica Schrouff · Natalie Harris · Sanmi Koyejo · Ibrahim Alabdulmohsin · Eva Schnider · Diana Mincu · Christina Chen · Awa Dieng · Yuan Liu · Vivek Natarajan · Katherine Heller · Alexander D'Amour -
2021 : Exploiting Causal Chains for Domain Generalization »
Olawale Salaudeen · Sanmi Koyejo -
2022 : Participatory Systems for Personalized Prediction »
Hailey Joren · Chirag Nagpal · Katherine Heller · Berk Ustun -
2022 : Participatory Systems for Personalized Prediction »
Hailey James · Berk Ustun · Chirag Nagpal · Katherine Heller -
2023 : A Causal Perspective on Label Bias »
Vishwali Mhasawade · Alexander D'Amour · Stephen Pfohl -
2023 : Reward Model Aggregation »
Zihao Wang · Chirag Nagpal · Alexander D'Amour · Victor Veitch · Sanmi Koyejo -
2023 : Reward Model Underspecification in Language Model Alignment »
Jacob Eisenstein · Jonathan Berant · Chirag Nagpal · Alekh Agarwal · Ahmad Beirami · Alexander D'Amour · Krishnamurthy Dvijotham · Katherine Heller · Stephen Pfohl · Deepak Ramachandran -
2023 : Understanding subgroup performance differences of fair predictors using causal models »
Stephen Pfohl · Natalie Harris · Chirag Nagpal · David Madras · Vishwali Mhasawade · Olawale Salaudeen · Katherine Heller · Sanmi Koyejo · Alexander D'Amour -
2023 : A Causal Perspective on Label Bias »
Vishwali Mhasawade · Alexander D'Amour · Stephen Pfohl -
2023 Poster: Participatory Personalization in Classification »
Hailey Joren · Chirag Nagpal · Katherine Heller · Berk Ustun -
2023 Mentorship: Education Outreach »
Tristan Naumann · Sanmi Koyejo · Marzyeh Ghassemi · Saadia Gabriel -
2022 : Participatory Systems for Personalized Prediction »
Hailey Joren · Chirag Nagpal · Katherine Heller · Berk Ustun -
2022 : Participatory Systems for Personalized Prediction »
Hailey Joren · Chirag Nagpal · Katherine Heller · Berk Ustun -
2019 : Poster Session »
Nathalie Baracaldo · Seth Neel · Tuyen Le · Dan Philps · Suheng Tao · Sotirios Chatzis · Toyo Suzumura · Wei Wang · WENHANG BAO · Solon Barocas · Manish Raghavan · Samuel Maina · Reginald Bryant · Kush Varshney · Skyler D. Speakman · Navdeep Gill · Nicholas Schmidt · Kevin Compher · Naveen Sundar Govindarajulu · Vivek Sharma · Praneeth Vepakomma · Tristan Swedish · Jayashree Kalpathy-Cramer · Ramesh Raskar · Shihao Zheng · Mykola Pechenizkiy · Marco Schreyer · Li Ling · Chirag Nagpal · Robert Tillman · Manuela Veloso · Hanjie Chen · Xintong Wang · Michael Wellman · Matthew van Adelsberg · Ben Wood · Hans Buehler · Mahmoud Mahfouz · Antonios Alexos · Megan Shearer · Antigoni Polychroniadou · Lucia Larise Stavarache · Dmitry Efimov · Johnston P Hall · Yukun Zhang · Emily Diana · Sumitra Ganesh · Vineeth Ravi · · Swetasudha Panda · Xavier Renard · Matthew Jagielski · Yonadav Shavit · Joshua Williams · Haoran Wei · Shuang (Sophie) Zhai · Xinyi Li · Hongda Shen · Daiki Matsunaga · Jaesik Choi · Alexis Laignelet · Batuhan Guler · Jacobo Roa Vicens · Ajit Desai · Jonathan Aigrain · Robert Samoilescu -
2019 : Poster Session I »
Shuangjia Zheng · Arnav Kapur · Umar Asif · Eyal Rozenberg · Cyprien Gilet · Oleksii Sidorov · Yogesh Kumar · Tom Van Steenkiste · William Boag · David Ouyang · Paul Jaeger · Sheng Liu · Aparna Balagopalan · Deepta Rajan · Marta Skreta · Nikhil Pattisapu · Jann Goschenhofer · Viraj Prabhu · Di Jin · Laura-Jayne Gardiner · Irene Li · sriram kumar · Qiyuan Hu · Mehul Motani · Justin Lovelace · Usman Roshan · Lucy Lu Wang · Ilya Valmianski · Hyeonwoo Lee · Sunil Mallya · Elias Chaibub Neto · Jonas Kemp · Marie Charpignon · Amber Nigam · Wei-Hung Weng · Sabri Boughorbel · Alexis Bellot · Lovedeep Gondara · Haoran Zhang · Taha Bahadori · John Zech · Rulin Shao · Edward Choi · Laleh Seyyed-Kalantari · Emily Aiken · Ioana Bica · Yiqiu Shen · Kieran Chin-Cheong · Subhrajit Roy · Ioana Baldini · So Yeon Min · Dirk Deschrijver · Pekka Marttinen · Damian Pascual Ortiz · Supriya Nagesh · Niklas Rindtorff · Andriy Mulyar · Katharina Hoebel · Martha Shaka · Pierre Machart · Leon Gatys · Nathan Ng · Matthias Hüser · Devin Taylor · Dennis Barbour · Natalia Martinez · Clara McCreery · Benjamin Eyre · Vivek Natarajan · Ren Yi · Ruibin Ma · Chirag Nagpal · Nan Du · Chufan Gao · Anup Tuladhar · Sam Shleifer · Jason Ren · Pouria Mashouri · Ming Yang Lu · Farideh Bagherzadeh-Khiabani · Olivia Choudhury · Maithra Raghu · Scott Fleming · Mika Jain · GUO YANG · Alena Harley · Stephen Pfohl · Elisabeth Rumetshofer · Alex Fedorov · Saloni Dash · Jacob Pfau · Sabina Tomkins · Colin Targonski · Michael Brudno · Xinyu Li · Yiyang Yu · Nisarg Patel