The tool could help scientists accelerate discoveries in areas ranging from basic cell biology to disease diagnosis and treatment.
By Jenna Somers
Serena Yeung-Levy, assistant professor of biomedical data science and director of MARVL (Medical AI and ComputeR Vision Lab), has been awarded a four-year, $1.39 million grant from the National Library of Medicine to develop an artificial intelligence tool that could help researchers and clinicians interpret microscopy images. The agentic AI system, trained on a vast collection of scientific knowledge, could respond to researchers’ queries, accelerate data-driven discovery in microscopy and advance the broader capabilities of multimodal AI for biological data analysis.
Microscopy images are essential to biological research and medicine. They can reveal cell behavior, disease progression, and treatment responses. However, analyzing these images is more complex than what meets the eye looking under the microscope. To interpret an image, researchers must connect what they see to relevant images, descriptions and findings scattered across thousands of scientific publications. Conventional AI systems, like the chatbots people use daily, might hold vast amounts of these types of data and recognize broad similarities between images and text; however, they are not as effective at interpreting subtle, fine-grained visual details essential for meaningful biological interpretation, a limitation Yeung-Levy describes as “blurry vision.”
Developing an AI system to accelerate discovery
To better support researchers, Yeung-Levy’s team aims to develop an AI system based on a new vision-language model that can jointly analyze microscopy images and scientific text. “Visual reasoning remains especially hard for today’s frontier models, and researchers experimenting with them to help interpret microscopy images have often found that while the responses aren’t necessarily incorrect, they lack the expert-level interpretation and meaningful insight needed to truly accelerate scientific discovery. That’s the problem we’re trying to solve,” Yeung-Levy said.
Alejandro Lozano, a fourth-year doctoral candidate in Yeung-Levy’s lab, added, “Biomedical microscopy is undergoing an unprecedented transformation. Advances in imaging technologies now enable scientists to visualize biological systems at extraordinary scale and resolution, generating millions to billions of images and terabytes to petabytes of data. Yet our ability to generate microscopy data increasingly outpaces our ability to interpret it. This creates a fundamental bottleneck: we can observe biology at unprecedented scale, but we lack systems capable of connecting what we see to what we know. Our research aims to bridge this gap by developing multimodal AI systems that integrate microscopy images with scientific knowledge to reason, generate hypotheses and accelerate biological discovery.”
…we can observe biology at unprecedented scale, but we lack systems capable of connecting what we see to what we know. Our research aims to bridge this gap. –Alejandro Lozano, fourth-year doctoral candidate
Agentic AI that responds to researchers’ questions
The research team will curate a large, diverse dataset pairing fine-grained microscopy images with descriptive text drawn from scientific literature and public databases. Rather than manually labeling each item, the researchers will use a technique called weak supervision, allowing the AI model to learn from abundant, loosely aligned image-text pairs. It will also learn to simultaneously recognize broad patterns and fine visual details, using an approach called multi-scale contrastive learning.
This new AI model will power an agentic AI system that responds to researchers’ queries about microscopy images, with the goal of improving their ability to connect what they observe under the microscope to broader biological context, such as related experiments, disease traits and prior studies.
The team will test the system on real research problems, including analyses of diseased tissue samples and studies of how cells respond to treatments. Yeung-Levy and her collaborators plan to make their datasets, methods and trained models available to the scientific community to advance the broader capabilities of multimodal AI for biological data analysis.
