Wednesday, November 8th, 2023
2:00 pm – 3:00 pm PST
Title: Medical AI After Deployment: Data-driven analyses and methods for clinically viable AI
Abstract: Medical AI algorithms have undergone significant development and regulatory approval, with over 600 FDA-approved medical AI devices currently. However, their actual clinical safety and impact remain unclear. First, we analyze FDA submission documents and find that the majority of FDA approvals do not report multi-site evaluation, and nearly none have prospective analyses. Second, we track the occurrences of newly released AI billing codes in a nationwide insurance claims database and find that only a handful of products have meaningful clinical adoption. Finally, we systematically track device updating in FDA submissions and find that the majority of devices have not had updates to model weights since initial approval. Given these limitations, we propose several methods to address common issues with algorithmic deployment. First, we present a framework for understanding the marginal contribution of distribution shifts to overall model degradation. Second, we present a method for efficient missing data collection in the context of fixed models. Finally, we present ways to improve the robustness of evaluating medical LLMs.