Many Good Models Leads To…

Cynthia Rudin, Duke University

Abstract: As it turns out, many good models leads to amazing things! The Rashomon Effect, coined by Leo Breiman, describes the phenomenon that there exist many equally good predictive models for the same dataset. This phenomenon happens for many real datasets and when it does, it sparks both magic and consternation, but mostly magic. In light of the Rashomon Effect, my collaborators and I propose to reshape the way we think about machine learning, particularly for tabular data problems in the nondeterministic (noisy) setting. I’ll address how the Rashomon Effect impacts (1) the existence of simple-yet-accurate models, (2) flexibility to address user preferences, such as fairness and monotonicity, without losing performance, (3) uncertainty in predictions, fairness, and explanations, (4) reliable variable importance, (5) algorithm choice, specifically, providing advanced knowledge of which algorithms might be suitable for a given problem, and (6) public policy. I’ll also discuss a theory of when the Rashomon Effect occurs and why: interestingly, noise in data leads to a large Rashomon Effect. My goal is to illustrate how the Rashomon Effect can have a massive impact on the use of machine learning for complex problems in society.

I’ll be discussing the paper “Amazing Things Come From Having Many Good Models” (ICML spotlight, 2024) which is joint work with Chudi Zhong, Lesia Semenova, Margo Seltzer, Ronald Parr, Jiachang Liu, Srikar Katta, Jon Donnelly, Harry Chen, and Zachery Boner. https://arxiv.org/abs/2407.04846