BioML Seminar 4.1 - Jeff Ruffolo on Designing proteins with language models

Value of the Information & Strength of the Argument

The presentation provides valuable insights into the scaling behavior of protein language models, with concrete experimental evidence. Ruffolo argues that while predictive tasks like mutation effect prediction may saturate, generative capabilities continue to improve with scale, leading to more diverse and functional proteins. He supports this with in-silico metrics and experimental validation, including expression assays and CRISPR activity measurements. The argumentation is solid, with clear reasoning and acknowledgment of limitations, such as the imperfect correlation between model likelihood and experimental fitness.

The title accurately reflects the content, which focuses on designing proteins with language models, including scaling and applications.

8/10

The talk is given by an expert in the field, presenting recent work from Profluent Bio. The content is technical and appears scientifically sound, but it is a seminar presentation rather than a peer-reviewed publication. The speaker acknowledges limitations and discusses experimental validation, which adds credibility.

Markers derived by PSI from the transcript: the creator did not define chapters.

• 02:00Introduction to proteins as molecular machines and the goal of protein engineering.
• 05:00Explanation of protein language models and their training on sequence databases.
• 10:00Discussion of scaling protein language models, including data collection and rebalancing.
• 15:00Evaluation of generative capabilities across model scales, including in-silico metrics.
• 20:00Experimental validation of generated proteins, including expression assays.
• 25:00Case study on designing CRISPR-Cas proteins with language models.
• 30:00Discussion of retrieval augmentation for protein representation learning.

• ProGen3 paper — Mentioned as the model developed by Profluent, but no specific URL provided.
• OpenCRISPR initiative — Mentioned as a project at Profluent, but no specific URL provided.
• UniRef database — Referenced as a source of protein sequences for training.
• BFD database — Referenced as a metagenomic database used in early models.
• ESM2 — Mentioned as a protein language model from Meta AI.
• ESM3 — Mentioned as a larger protein language model.

• Scaling laws for neural language models — Relevant to the scaling experiments discussed in the talk.
• Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences — ESM1b paper, relevant to protein language models.

• Language models of protein sequences and structures — Some studies suggest that scaling protein language models may not improve mutation effect prediction, which is discussed in the talk.