Fine-tuning AI Foundation Models to Quantify Spatially-Resolved Cell-Type Fractions from Breast Cancer Digital Pathology Images

General data

Time schedule of the internship

Requirements and prior knowledge

Bachelor's degree in Bioinformatics. Python and / or R skills. Prior skills in AI modeling (e.g. pytorch) and model tuning are a plus.

Description

AI foundation models (e.g., Virchow, MUSK, OmniCLIP) offer tremendous potential for digital pathology and biomedicine by learning generalizable representations from large, heterogeneous datasets. These representations can be efficiently adapted to a wide range of tasks, enabling more accurate, scalable, and data-efficient analyses across molecular, cellular, and clinical domains. Critically, such models can be specialized for specific biomedical applications through fine-tuning on relatively small, task-specific datasets, which are far smaller than the massive datasets required for their original training.

This project aims to fine-tune AI foundation models to directly quantify spatially resolved cell-type abundances in breast cancer tumor tissues from standard, inexpensive H&E-stained histology images. By doing so, we can reduce dependence on complex and costly spatial transcriptomics experiments. Fine-tuning will leverage thousands of breast cancer H&E images paired with cell-type abundance estimates obtained from the deconvolution of matched spatial and/or bulk RNA-seq data, enabling the models to learn a direct mapping from histological images to the underlying cellular composition of the tumor microenvironment.

References:

• Virchow: https://www.nature.com/articles/s41591-024-03141-0 

• MUSK: https://www.nature.com/articles/s41586-024-08378-w

• OmniCLIP: https://www.nature.com/articles/s41592-025-02707-1

• Cell-type deconvolution: https://linkinghub.elsevier.com/retrieve/pii/S1937-6448(23)00064-3