AI Enhances Spatial Multi-Omic and Histopathology Analysis
Publication Title: Leveraging multi-modal foundation models for analysing spatial multi-omic and histopathology data
Summary
- Question
- This study introduced spEMO, a computational framework designed to integrate pathology foundation models (PFMs) and large language models (LLMs) for analyzing spatial multi-omic data. The researchers aimed to improve tasks such as identifying spatial domains, classifying spot types, predicting disease states, and generating medical reports by unifying data from multiple sources, including histopathology images and spatial gene expression data.
- Why it Matters
- Advances in pathology and multi-omic technologies have enabled detailed analysis of diseases, but existing tools often fail to integrate data from multiple sources effectively. spEMO addresses this gap by combining information from histopathology images and spatial gene and protein expression data, offering a holistic approach to understanding tissue biology. This has implications for advancing biological discoveries, improving diagnostic accuracy, and informing personalized treatments for diseases such as cancer.
- Methods
- The researchers developed spEMO to combine embeddings (mathematical representations of data) from PFMs and LLMs. The system operates in two modes: zero-shot embedding, which integrates pre-trained models without additional training, and fine-tuning, which adapts models to specific tasks. They evaluated spEMO on tasks like spatial domain identification and disease prediction using datasets of histopathology images and spatial transcriptomic profiles from human tissues.
- Key Findings
- spEMO outperformed single-modality models in multiple tasks. For example, it improved spatial domain identification by leveraging embeddings with spatial correlations, leading to more accurate clustering of tissue regions. It also generated medical reports with higher completeness and accuracy compared to human pathologists. Additionally, spEMO demonstrated the ability to predict disease states and infer novel intercellular interactions, providing insights into tumor microenvironments.
- Implications
- The integration of histopathology and spatial multi-omic data using spEMO could enhance diagnostic precision and facilitate the discovery of new biomarkers and therapeutic targets. By unifying multi-modal data, spEMO offers a reproducible and generalizable approach for analyzing complex biological systems, which could be applied to cancer research, drug response prediction, and personalized medicine.
- Next Steps
- The researchers suggest further development of foundation models trained jointly on pathology and multi-omic data to enhance multi-modal integration. They also highlight the need for larger and more diverse datasets to improve model performance, particularly for rare diseases. Future work may explore additional applications of spEMO in clinical settings, such as real-time diagnostic support and expanded multi-omic analyses.
- Funding Information
- This research was supported by NIH grants U24HG012108 and U01HG013840. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Full Citation
Liu T, Huang T, Ding T, Wu H, Humphrey P, Perincheri S, Schalper K, Ying R, Xu H, Zou J, Mahmood F, Zhao H. Leveraging multi-modal foundation models for analysing spatial multi-omic and histopathology data. Nature Biomedical Engineering 2026, 1-18. PMID: 41644824, DOI: 10.1038/s41551-025-01602-6.
This AI-assisted summary has been reviewed and approved by at least one of the study's authors to ensure it accurately reflects the research.
Authors
Tianyu Liu
First AuthorHongyu Zhao
Last Author