Timezone: »

 
Q-Pain: A Question Answering Dataset to Measure Social Bias in Pain Management
Cécile Logé · Emily Ross · David Dadey · Saahil Jain · Adriel Saporta · Andrew Ng · Pranav Rajpurkar

Recent advances in Natural Language Processing (NLP), and specifically automated Question Answering (QA) systems, have demonstrated both impressive linguistic fluency and a pernicious tendency to reflect social biases. In this study, we introduce Q-Pain, a dataset for assessing bias in medical QA in the context of pain management, one of the most challenging forms of clinical decision-making. Along with the dataset, we propose a new, rigorous framework, including a sample experimental design, to measure the potential biases present when making treatment decisions. We demonstrate its use by assessing two reference Question-Answering systems, GPT-2 and GPT-3, and find statistically significant differences in treatment between intersectional race-gender subgroups, thus reaffirming the risks posed by AI in medical settings, and the need for datasets like ours to ensure safety before medical AI applications are deployed.

Author Information

Cécile Logé (Stanford University)
Emily Ross (Stanford University)
David Dadey
Saahil Jain (Stanford University)
Adriel Saporta (Apple)
Andrew Ng (Stanford University)

Andrew Ng, Chief Scientist at Baidu, Chairman & Co-Founder of Coursera, Adjunct Professor, Stanford Dr. Andrew Ng joined Baidu in May 2014 as chief scientist. He is responsible for driving the company's global AI strategy and infrastructure. He leads Baidu Research in Beijing and Silicon Valley as well as technical teams in the areas of speech, big data and image search. In addition to his role at Baidu, Dr. Ng is an adjunct professor in the computer science department at Stanford University. In 2011 he led the development of Stanford's Massive Open Online Course (MOOC) platform and taught an online machine learning class that was offered to over 100,000 students. This led to the co-founding of Coursera, where he continues to serve as chairman. Previously, Dr. Ng was the founding lead of the Google Brain deep learning project. Dr. Ng has authored or co-authored over 100 research papers in machine learning, robotics and related fields. In 2013 he was named to the Time 100 list of the most influential persons in the world. He holds degrees from Carnegie Mellon University, MIT and the University of California, Berkeley.

Pranav Rajpurkar (Harvard University)

More from the Same Authors

  • 2021 : RadGraph: Extracting Clinical Entities and Relations from Radiology Reports »
    Saahil Jain · Ashwin Agrawal · Adriel Saporta · Steven Truong · Du Nguyen Duong · Tan Bui · Pierre Chambon · Yuhao Zhang · Matthew Lungren · Andrew Ng · Curtis Langlotz · Pranav Rajpurkar
  • 2021 : RadGraph: Extracting Clinical Entities and Relations from Radiology Reports »
    Saahil Jain · Ashwin Agrawal · Adriel Saporta · Steven Truong · Du Nguyen Duong · Tan Bui · Pierre Chambon · Yuhao Zhang · Matthew Lungren · Andrew Ng · Curtis Langlotz · Pranav Rajpurkar
  • 2021 : Q-Pain: A Question Answering Dataset to Measure Social Bias in Pain Management »
    Cécile Logé · Emily Ross · David Dadey · Saahil Jain · Adriel Saporta · Andrew Ng · Pranav Rajpurkar
  • 2020 : Andrew Ng: Practical limitations of today's deep learning in healthcare »
    Andrew Ng
  • 2019 : Climate Change: A Grand Challenge for ML »
    Yoshua Bengio · Carla Gomes · Andrew Ng · Jeff Dean · Lester Mackey
  • 2019 Workshop: Tackling Climate Change with ML »
    David Rolnick · Priya Donti · Lynn Kaack · Alexandre Lacoste · Tegan Maharaj · Andrew Ng · John Platt · Jennifer Chayes · Yoshua Bengio
  • 2017 : Coffee break and Poster Session I »
    Nishith Khandwala · Steve Gallant · Gregory Way · Aniruddh Raghu · Li Shen · Aydan Gasimova · Alican Bozkurt · William Boag · Daniel Lopez-Martinez · Ulrich Bodenhofer · Samaneh Nasiri GhoshehBolagh · Michelle Guo · Christoph Kurz · Kirubin Pillay · Kimis Perros · George H Chen · Alexandre Yahi · Madhumita Sushil · Sanjay Purushotham · Elena Tutubalina · Tejpal Virdi · Marc-Andre Schulz · Samuel Weisenthal · Bharat Srikishan · Petar Veličković · Kartik Ahuja · Andrew Miller · Erin Craig · Disi Ji · Filip Dabek · Chloé Pou-Prom · Hejia Zhang · Janani Kalyanam · Wei-Hung Weng · Harish Bhat · Hugh Chen · Simon Kohl · Mingwu Gao · Tingting Zhu · Ming-Zher Poh · Iñigo Urteaga · Antoine Honoré · Alessandro De Palma · Maruan Al-Shedivat · Pranav Rajpurkar · Matthew McDermott · Vincent Chen · Yanan Sui · Yun-Geun Lee · Li-Fang Cheng · Chen Fang · Sibt ul Hussain · Cesare Furlanello · Zeev Waks · Hiba Chougrad · Hedvig Kjellstrom · Finale Doshi-Velez · Wolfgang Fruehwirt · Yanqing Zhang · Lily Hu · Junfang Chen · Sunho Park · Gatis Mikelsons · Jumana Dakka · Stephanie Hyland · yann chevaleyre · Hyunwoo Lee · Xavier Giro-i-Nieto · David Kale · Michael Hughes · Gabriel Erion · Rishab Mehra · William Zame · Stojan Trajanovski · Prithwish Chakraborty · Kelly Peterson · Muktabh Mayank Srivastava · Amy Jin · Heliodoro Tejeda Lemus · Priyadip Ray · Tamas Madl · Joseph Futoma · Enhao Gong · Syed Rameel Ahmad · Eric Lei · Ferdinand Legros
  • 2016 Tutorial: Nuts and Bolts of Building Applications using Deep Learning »
    Andrew Ng