Resources
Foundation Models in Pharma and Biotech: Unlocking Intelligence from Biological Data
AI Foundation models like LLMs have taken the world by writing code, answering questions, and even generating images and video. These powerful models can quickly adapt to new tasks with minimal data and are fast becoming the backbone of modern AI.
Healthcare, however, has struggled to keep pace. Despite thousands of AI models being built, very few are actually used in clinical care. High development costs fragmented and siloed data, complex clinical workflows, lack of trust in AI models, and stringent privacy regulations all contribute to the challenge.
Concerns about data security and patient confidentiality often limit data sharing and access, making it difficult to train and deploy AI systems at scale. As a result, many promising models remain stuck in research, never reaching the patients who could benefit from them.
Foundation models may finally change that! By learning general patterns from biological or clinical data and reusing that knowledge across many tasks, they promise a smarter, faster, and more affordable way to build healthcare AI.
This article explores how foundation models are being applied in healthcare and bioinformatics and the biological insights they can reveal through powerful representations of data.
What Is a Foundation Model?
Foundation models are large AI systems trained on massive, unlabeled datasets using self-supervised learning. This means they can learn patterns and relationships in data without needing labels for driving the process of learning Unlike many traditional machine learning models, which often require task-specific feature engineering and labeled datasets for each new problem, foundation models are pretrained on large-scale data and can be adapted to a variety of downstream tasks with minimal additional training. Originally developed for natural language and image generation, foundation models are now being applied in healthcare. Because they encode broad knowledge into their structure, they offer a more scalable and cost-effective way to build AI tools—especially where labeled medical data is scarce or difficult to obtain.Foundation Models in Bioinformatics: Bridging Omics and Intelligence
In healthcare, foundation models are advancing most rapidly in bioinformatics. These models, often called Bioinformatics Foundation Models (BFMs), are built to work with complex omics data: DNA, RNA, proteins, and single-cell sequencing. BFMs have become possible thanks to high-throughput sequencing technologies and AI architectures inspired by language models like BERT and GPT. Just like language models learn grammar and meaning from text, BFMs learn biological “rules” from massive genomic, transcriptomic, and proteomic datasets. Some even include associated clinical texts like disease links or treatment responses. This allows BFMs to go beyond simple pattern recognition. They are beginning to model underlying biological logic by learning how genes, cells, and proteins relate in a way that mimics biological function. Many now achieve state-of-the-art performance on bioinformatics benchmarks.Learning from Biology: Pre-training Strategies in BFMs
To learn from complex biological data, Bioinformatics Foundation Models (BFMs) often rely on generative learning (GL) strategies. Adapted from natural language processing, these methods are particularly effective for biological systems, where the context, order, and structure of data play a crucial role. Generative learning helps the model predict missing or next elements in a sequence. The two most common techniques are:- Masked Omics Modeling (MOM): The model hides parts of biological sequences like gene expression values or DNA segments and learns to reconstruct them. This is how language models learn by guessing missing words.
- Next Token Prediction (NTP): The model learns to predict what comes next in a sequence, such as the next nucleotide. This helps it understand the natural order and structure of biological information.
Understanding Biology Through Embeddings
Unlike language models, which communicate with users through text, Bioinformatics Foundation Models (BFMs) represent their knowledge using embeddings. Embedding is a mathematical way of expressing a biological entity like a gene, cell, or tissue sample, as a point in a multi-dimensional space. Think of it as a unique “address” that reflects the biological properties and relationships of that entity. These embeddings help uncover patterns that might not be obvious from raw data. For instance:- Similar genes often appear close together in this space
- Cells of the same type tend to cluster together
- Cells transitioning from healthy to diseased states form gradual, traceable paths.
Applications of Foundation Models in Pharma and Biotech
Foundation Models (FMs) and Bioinformatics Foundation Models (BFMs) enable a wide range of applications across the R&D pipeline in life sciences. Key areas include: Disease Modeling FMs can simulate disease progression by learning from multi-omics data, patient-derived cell lines, or animal models. This helps in understanding mechanisms and stratifying patient populations. Drug Target Identification By integrating transcriptomic, proteomic, and genetic data, BFMs can identify novel druggable targets and prioritize them based on disease relevance and functional insights. Drug Discovery & Repurposing Foundation models can generate molecular structures, predict compound–target interactions, or identify new indications for existing drugs through embedding similarities. Biomarker Discovery Embeddings generated by BFMs can help detect patterns and associations that serve as diagnostic, prognostic, or predictive biomarkers. Functional Genomics Models can predict gene functions, regulatory elements, or protein interactions by training on large-scale gene expression and chromatin accessibility datasets. Clinical Trial Optimization Though still emerging, FMs can assist in patient stratification, eligibility screening, and matching patients to appropriate trials using genomic and phenotypic embeddings.Adapting BFMs for Real-World Applications
Once a BFM has been pre-trained on large biological datasets, it needs to be adapted or fine-tuned for specific tasks. This could include identifying cell types, predict gene functions, or classify disease states. Fine-tuning (FT) adjusts the model’s ipretrained learning to focus on a particular application. However, this can require significant computing power, especially for large models. To address this, researchers are increasingly adopting parameter-efficient fine-tuning methods. One widely used technique is Low-Rank Adaptation (LoRA). Instead of modifying the model’s existing parameters directly, LoRA adds small trainable adapter modules. These low-rank updates interact with the frozen model weights, allowing effective adaptation with significantly reduced computational overhead. Other promising strategies include:- Adapter Tuning (AT): This method adds small plug-in modules to the pre-trained model. These modules can be trained quickly and swapped out for different tasks without changing the whole model.
- Prompt Engineering (PE): Still in its early stages for biology, this technique guides the model using tailored input prompts.For example, tools like GenePT explore how language model embeddings, such as those generated by ChatGPT, can help predict gene functions without additional training data. By varying the input prompts, different embeddings are produced, allowing GenePT to tailor its predictions to the specific biological context or question embedded in the prompt.