Available Theses

Systematic Ablations, New Pre-training Objectives, and Scaling Behaviour for a Single-Cell Foundation Model

Investigate which design choices actually drive the quality of DECODE-RNA’s learned cell representations (the expression-encoding scheme, protein-language-model priors, the dual-attention structure, and the multi-task curriculum) through controlled ablations measured on both in-distribution accuracy and cross-tissue transfer. Introduce new self-supervised objectives that teach the model to distinguish genuine biological silence from technical dropout, and characterise how representation quality scales with data and model size. First validated on blood and PBMC cells, then extended to whole-body Census data and multi-GPU training to derive a scale-aware pre-training recipe.

Required: solid Python and PyTorch; understanding of deep learning fundamentals (transformers, attention, self-supervised/masked pre-training, optimisation, and learning-rate schedules); ability to design controlled experiments and rigorously interpret training curves.

A plus: PyTorch Lightning; experience running jobs on a GPU/Slurm cluster and with multi-GPU (DDP) training; basic familiarity with single-cell RNA-seq and Scanpy/AnnData (can be learned on the job).

Multi-Teacher Knowledge Distillation from Single-Cell Foundation Models

Build a framework that distils the complementary strengths of several existing single-cell foundation models (e.g., scGPT, Geneformer, UCE, scVI) into DECODE-RNA’s unified gene vocabulary and continuous-expression representation. Because the teacher models live in incompatible embedding spaces, the central challenge is cross-space alignment: preserving each teacher’s cell–cell similarity structure and learning per-teacher weighting so the student inherits batch correction, biological resolution, and gene-level knowledge from the most reliable source. First tested on a foetal reference dataset using standard data-integration metrics, then evaluated on cross-tissue transfer against each individual teacher.

Required: strong Python and PyTorch; grasp of representation learning and the core idea of knowledge distillation; comfort working with embeddings and similarity/contrastive objectives.

A plus: familiarity with pretrained model ecosystems (running inference with scGPT, Geneformer, UCE, scVI, and managing their environments); notions of representation alignment (e.g., CKA, Procrustes); single-cell data integration metrics (scib) and Scanpy/AnnData.

Distilling REnformer into a Compact Chromatin-Accessibility Model

REnformer (Riva et al., IEEE CIBCB 2025; preprint) predicts cell-type-specific open chromatin directly from DNA sequence, but it inherits Enformer’s architecture and therefore its scale: a large model with a very long input window, which makes genome-wide scans and in-silico variant screens slow and costly. This project builds a small student model that reproduces REnformer’s predictions at a fraction of that cost.

Distillation suits the problem well, because the teacher can label as much data as you want: running REnformer along the genome yields effectively unlimited soft targets, with no new ATAC experiments needed. The work is to choose the student architecture (how many layers, how wide, how much receptive field can be given up), decide what to match (output tracks alone, or intermediate representations too), and map the accuracy-versus-cost trade-off (parameters, memory and inference throughput against agreement with both the teacher and held-out real single-cell ATAC data).

The most interesting question is where compression starts to hurt. Aggregate accuracy usually survives shrinking; sensitivity to single-nucleotide changes often does not. Re-running the α-thalassemia variant REnformer was validated on, together with other known regulatory variants, tests whether the small model keeps the fine-grained behaviour that makes these models worth having.

Required: solid Python and PyTorch; deep learning fundamentals (CNNs, transformers, training loops, learning-rate schedules); willingness to run GPU training jobs. No genomics background needed (the biology can be picked up on the job).

A plus: knowledge distillation or model compression (pruning, quantisation); familiarity with sequence-to-function models (Enformer, Borzoi, scBasset); handling genomic data formats (FASTA/BED/bigWig); profiling and benchmarking inference.

Validating Attention-Derived Gene Regulatory Networks Against Experimental Ground Truth

Test whether the gene–gene relationships DECODE-RNA extracts from its cross-attention weights reflect real biological regulation rather than mere co-expression. Build an evaluation harness that benchmarks the model’s zero-shot networks against curated regulons and experimental evidence (DoRothEA/CollecTRI, ChIP-seq, and perturbation data), and compares them with established network inference methods such as GENIE3, SCENIC, and CellOracle. First applied to blood cell types, then extended to further tissues to assess how cell-type-specific and transferable the inferred regulation is. This project is more biology-leaning and less deep-learning-heavy.

Required: Python and data analysis; a computational-biology/bioinformatics background with an understanding of gene regulation (transcription factors and their targets); sound evaluation methodology (AUROC/AUPRC, precision–recall, benchmarking design).

A plus: familiarity with regulatory resources (DoRothEA/CollecTRI, ChIP-seq/ChIP-Atlas, perturbation datasets) and network-inference tools (GENIE3, SCENIC, CellOracle); Scanpy/AnnData for single-cell handling.