In this webinar, Maria De-Arteaga discusses how machine learning (ML) is increasingly being used to support decision-making in many organizational settings. However, the problem occurs in the fact that there is a gap between ML algorithms’ design, evaluation, and the functional role of these algorithms as tools for decision support. In simpler terms, it’s essentially the gap between the predictions humans make versus ML-informed decisions. View the recording!

The first part of the presentation focuses on the role of humans-in-the-loop, and the importance of evaluating decisions instead of predictions, through a study of the adoption of a risk assessment tool in child maltreatment hotline screenings.

The second part of the talk focuses on the gap between the construct of interest and the proxy that the algorithm optimizes for. Using a proposed machine learning methodology that extracts knowledge from experts’ historical decisions, we show the following that in the context of child maltreatment hotline screenings:

(1) there are high-risk cases whose risk is considered by the experts but not wholly captured in the target labels used to train a deployed model

(2) we can bridge this gap if we purposefully design with this goal in mind.

Maria De-Arteaga is an assistant professor at the Information, Risk and Operation Management (IROM) Department at the University of Texas at Austin, where she is also a core faculty member in the Machine Learning Laboratory. She holds a joint PhD in Machine Learning and Public Policy and a M.Sc. in Machine Learning, both from Carnegie Mellon University, and a B.Sc. in Mathematics from Universidad Nacional de Colombia. Her research focuses on the risks and opportunities of using machine learning to support experts’ decisions in high-stakes settings. Her work has been featured by UN Women and Global Pulse, and has received best paper awards at NAACL’19 and Data for Policy’16, and research awards from Google and Microsoft Research.