How the Pandemic Made Algorithms Go Haywire

machine-learningdistribution-shiftalgorithmic-biascovid-19healthcare-ai

Abstraction: COVID-19 distribution shift caused healthcare and finance ML algorithms to fail

Key points:

University of Michigan's sepsis early-warning algorithm flagged more than 2x as many patients during COVID while hospital capacity was 35% lower, leading to alert overload and eventual shutdown
A cancer end-of-life care algorithm became 30% less accurate at identifying sick patients during the pandemic, causing missed timely hospice conversations
American Express delayed rollout of a fraud detection algorithm by nearly a year after pre-launch testing revealed pandemic spending shifts (online migration, large purchases) would trigger excessive false alerts
Over one-third of banks reported predictive algorithms became less accurate during COVID per a Bank of England survey, decelerating AI investment
Fixes proposed: human oversight of pre-COVID algorithms, retraining on pandemic-era data, and designing inherently robust models; a reinforcement learning border-screening algorithm in Greece adapted successfully across pandemic phases with 4x better accuracy than random testing

Connections: American Express · University Of Michigan · Distribution Shift · Algorithmic Bias · Machine Learning · Reinforcement Learning