Multiomics-Analytics-Group/course_protein_language_modeling
Multiomics-Analytics-Group/course_protein_language_modeling has 52 GitHub stars and 12 forks. GitHot measured 0 stars in 24 hours, 0 in 7 days, and 0 in 15 days. Last checked 2026-11-15 21:50 UTC. Last push 2026-08-20.
Crecimiento de estrellas medido
- in 24h
- 0
- in 7 days
- 0
- in 15 days
- 0
- in 1 month
- Actualización retrasada
Cambios netos en períodos que se solapan, no totales diarios. Los datos no disponibles se dejan en blanco. Las estrellas indican interés, no calidad.
Acerca de
To stop run: docker compose stop To stop and remove containers: docker compose down
Aprendizaje automáticoREADME.md
OriginalProtein Language Modeling Course

Course Structure
| Theory | Link | Data | Models | |
|---|---|---|---|---|
| Topics | slides | |||
| Hands-on | ||||
| Sequence Analysis | Seq. Analysis Notebook |
Exploring protein sequence and structure data | ||
| Fine-tuning a model | Model Training Notebook |
Taking an existing model and tuning it for other prediction/classification tasks | Evolutionary Scale Modeling (ESM) | |
| Working with Embeddings | Embeddings Notebook |
Accessing Protein representations (embeddings) generated by existing LMs | ProTrans | |
| Predictions | pML Predictions Notebook |
Using embeddings for predicting features or classifying sequences | ProTrans | |
| Protein Design | Protein Design Notebook |
De novo protein design and engineering using a LLM | ProtGPT2, ESM | |
| Mechanistic Interpretability | Interpretability Notebook |
Opening the black box: training a Sparse Autoencoder on ESM2's internal activations to find human-interpretable features and checking whether they track known $\beta$-lactamase biology | ESM2 |
Working Locally
- Install Docker
- From the root of the repository in the terminal run:
docker compose up --build -d - Open JupyterLab in any browser via
0.0.0.0:8888
To stop run: docker compose stop
To stop and remove containers: docker compose down
Working VM
- Once you have a VM, download docker in it
- (e.g., apt install via ssh)
- e.g. in VM shell run below
# Add Docker's official GPG key: sudo apt-get update sudo apt-get install ca-certificates curl sudo install -m 0755 -d /etc/apt/keyrings sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc sudo chmod a+r /etc/apt/keyrings/docker.asc # Add the repository to Apt sources: echo \ "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \ $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \ sudo tee /etc/apt/sources.list.d/docker.list > /dev/null sudo apt-get update sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
- Clone this repo into the VM or to keep things slim you can move only the below needed files onto VM (e.g., via
scp)- notebooks and data folders
- Dockerfile, docker-compose.yml
- requirements.txt
- Then run
docker compose up --build -d- if you encounter permission denied you may need to run the above command with elevated permissions e.g.
sudo docker compose up --build -d - Note: for troubleshooting you can use
docker logs <container-name>e.g.,docker logs jupyter
- if you encounter permission denied you may need to run the above command with elevated permissions e.g.
- The URL to the JupyterLab will be:
<VM's IP address>:8888e.g.12.26.38.408:8888- Note: may have to expose port 8888 via the VM's network security group
- On your local device, open a browser of your choosing. In the address bar input the URL to JupyterLab. And you should be good to go :)
- Remember to stop the VM container once you are done
References
Neural Machine Translation by Jointly Learning to Align and Translate Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio
Attention Is All You Need Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin
MSA Transformer Roshan Rao, Jason Liu, Robert Verkuil, Joshua Meier, John F. Canny, Pieter Abbeel, Tom Sercu, Alexander Rives
Transformer-based deep learning for predicting protein properties in the life sciences Abel Chandra, Laura Tünnermann, Tommy Löfstedt, Regina Gratz
BERTology Meets Biology: Interpreting Attention in Protein Language Models Jesse Vig, Ali Madani, Lav R. Varshney, Caiming Xiong, Richard Socher, Nazneen Fatema Rajani
Broadly applicable and accurate protein design by integrating structure prediction networks and diffusion generative models Joseph L. Watson, David Juergens, Nathaniel R. Bennett, Brian L. Trippe, Jason Yim, Helen E. Eisenach, Woody Ahern, Andrew J. Borst, Robert J. Ragotte, Lukas F. Milles, Basile I. M. Wicky, Nikita Hanikel, Samuel J. Pellock, Alexis Courbet, William Sheffler, Jue Wang, Preetham Venkatesh, Isaac Sappington, Susana Vázquez Torres, Anna Lauko, Valentin De Bortoli, Emile Mathieu, Regina Barzilay, Tommi S. Jaakkola, Frank DiMaio, Minkyung Baek, David Baker
Large language models generate functional protein sequences across diverse families Ali Madani, Ben Krause, Eric R. Greene, Subu Subramanian, Benjamin P. Mohr, James M. Holton, Jose Luis Olmos Jr., Caiming Xiong, Zachary Z. Sun, Richard Socher, James S. Fraser & Nikhil Naik
Learning functional properties of proteins with language modelsSerbulent Unsal, Heval Atas, Muammer Albayrak, Kemal Turhan, Aybar C. Acar & Tunca Doğan
Learning the protein language: Evolution, structure, and function Tristan Bepler, Bonnie Berger
The language of proteins: NLP, machine learning & protein sequences Dan Ofer, Nadav Brandes, Michal Linial
Evolutionary-scale prediction of atomic-level protein structure with a language model ZEMING LIN, HALIL AKIN, ROSHAN RAO, BRIAN HIE, ZHONGKAI ZHU, WENTING LU, NIKITA SMETANIN, ROBERT VERKUIL, ORI KABELI, ..., ALEXANDER RIVES
Generative power of a protein language model trained on multiple sequence alignments Damiano Sgarbossa, Umberto Lupo, Anne-Florence Bitbol
How Huge Protein Language Models Could Disrupt Structural Biology
Embeddings from protein language models predict conservation and variant effects Céline Marquet, Michael Heinzinger, Tobias Olenyi, Christian Dallago, Kyra Erckert, Michael Bernhofer, Dmitrii Nechaev & Burkhard Rost
Collectively encoding protein properties enriches protein language models Jingmin An & Xiaogang Weng
ProGen: Language Modeling for Protein Generation Ali Madani, Bryan McCann, Nikhil Naik, Nitish Shirish Keskar, Namrata Anand, Raphael R. Eguchi, Po-Ssu Huang, Richard Socher
Transformer protein language models are unsupervised structure learners Roshan Rao, Joshua Meier, Tom Sercu, Sergey Ovchinnikov, Alexander Rives
Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences Alexander Rives, Joshua Meier, Tom Sercu and Rob Fergus
NetSurfP-3.0: accurate and fast prediction of protein structural features by protein language models and deep learning Magnus Haraldson Høie, Erik Nicolas Kiehl, Bent Petersen, Morten Nielsen, Ole Winther, Henrik Nielsen, Jeppe Hallgren, Paolo Marcatili
Modeling Protein Using Large-scale Pretrain Language Model Yijia Xiao, Jiezhong Qiu, Ziang Li, Chang-Yu Hsieh, Jie Tang github
Deciphering antibody affinity maturation with language models and weakly supervised learning Jeffrey A. Ruffolo, Jeffrey J. Gray, Jeremias Sulam github
Protein embeddings improve phage-host interaction prediction Mark Edward M. Gonzales, Jennifer C. Ureta, View ORCID ProfileAnish M.S. Shrestha github
ProteinBERT: a universal deep-learning model of protein sequence and function Nadav Brandes, Dan Ofer, Yam Peleg, Nadav Rappoport, Michal Linial github
ProtGPT2 is a deep unsupervised language model for protein design Noelia Ferruz, Steffen Schmidt & Birte Höcker Hugging Face
Protein-Protein Interaction Prediction is Achievable with Large Language Models Logan Hallee, Jason P. Gleghorn
Accurate prediction of virus-host protein-protein interactions via a Siamese neural network using deep protein sequence embeddings Sumit Madan, Victoria Demina, Marcus Stapf, Oliver Ernst, Holger Fröhlich
Structure-informed Language Models Are Protein Designers Zaixiang Zheng, Yifan Deng, Dongyu Xue, Yi Zhou, Fei YE, Quanquan Gu
Graph-BERT and language model-based framework for protein–protein interaction identification Kanchan Jha, Sourav Karmakar & Sriparna Saha
Ankh: Optimized Protein Language Model Unlocks General-Purpose Modelling Ahmed Elnaggar, Hazem Essam, Wafaa Salah-Eldin, Walid Moustafa, Mohamed Elkerdawy, Charlotte Rochereau, Burkhard Rost
Contrastive learning in protein language space predicts interactions between drugs and protein targets Rohit Singh, Samuel Sledzieski, Bryan Bryson, Bonnie Berger
De novo design of protein structure and function with RFdiffusion Joseph L. Watson, David Juergens, Nathaniel R. Bennett, Brian L. Trippe, Jason Yim, Helen E. Eisenach, Woody Ahern, Andrew J. Borst, Robert J. Ragotte, Lukas F. Milles, Basile I. M. Wicky, Nikita Hanikel, Samuel J. Pellock, Alexis Courbet, William Sheffler, Jue Wang, Preetham Venkatesh, Isaac Sappington, Susana Vázquez Torres, Anna Lauko, Valentin De Bortoli, Emile Mathieu, Sergey Ovchinnikov, Regina Barzilay, Tommi S. Jaakkola, Frank DiMaio, Minkyung Baek & David Baker
Single-sequence protein structure prediction using a language model and deep learning Ratul Chowdhury, Nazim Bouatta, Surojit Biswas, Christina Floristean, Anant Kharkar, Koushik Roy, Charlotte Rochereau, Gustaf Ahdritz, Joanna Zhang, George M. Church, Peter K. Sorger & Mohammed AlQuraishi
Before and after AlphaFold2: An overview of protein structure prediction Letícia M. F. Bertoline, Angélica N. Lima, Jose E. Krieger, and Samantha K. Teixeira
Evaluating Protein Transfer Learning with TAPE Roshan Rao, Nicholas Bhattacharya, Neil Thomas, Yan Duan, Xi Chen, John Canny, Pieter Abbeel, Yun S. Song
FLIP: Benchmark tasks in fitness landscape inference for proteins Christian Dallago, Jody Mou, Kadina E. Johnston, Bruce J. Wittmann, Nicholas Bhattacharya, Samuel Goldman, Ali Madani, Kevin K. Yang
Standards, tooling and benchmarks to probe representation learning on proteins Joaquin Gomez Sanchez, Sebastian Franz, Michael Heinzinger, Burkhard Rost, Christian Dallago
Learning meaningful representations of protein sequences Nicki Skafte Detlefsen, Søren Hauberg & Wouter Boomsma
Language modelling for biological sequences – curated datasets and baselines Jose Juan Almagro Armenteros, Alexander Rosenberg Johansen, Ole Winther, Henrik Nielsen
Language models generalize beyond natural proteins Robert Verkuil, Ori Kabeli, Yilun Du, Basile I. M. Wicky, Lukas F. Milles, Justas Dauparas, David Baker, Sergey Ovchinnikov, Tom Sercu, Alexander Rives
ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Prediction Seyone Chithrananda, Gabriel Grand, Bharath Ramsundar
SETH predicts nuances of residue disorder from protein embeddings Dagmar Ilzhöfer, Michael Heinzinger, Burkhard Rost
Exploiting pretrained biochemical language models for targeted drug design Gökçe Uludoğan, Elif Ozkirimli, Kutlu O Ulgen, Nilgün Karalı, Arzucan Özgür
xTrimoPGLM: unified 100-billion-parameter pretrained transformer for deciphering the language of proteins Bo Chen, Xingyi Cheng, Pan Li, Yangli-ao Geng, Jing Gong, Shen Li, Zhilei Bei, Xu Tan, Boyan Wang, Xin Zeng, Chiming Liu, Aohan Zeng, Yuxiao Dong, Jie Tang, Le Song
PTM-Mamba: a PTM-aware protein language model with bidirectional gated Mamba blocks Fred Zhangzhi Peng, Chentong Wang, Tong Chen, Benjamin Schussheim, Sophia Vincoff, Pranam Chatterjee
Towards Monosemanticity: Decomposing Language Models With Dictionary Learning Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nicholas L Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Hatfield-Dodds, Alex Tamkin, Karina Nguyen, Brayden McLean, Josiah E Burke, Tristan Hume, Shan Carter, Tom Henighan, Chris Olah
InterPLM: Discovering Interpretable Features in Protein Language Models via Sparse Autoencoders Elana Simon, James Zou