Latent Space: The AI Engineer Podcast

🔬 Lab-in-the-Loop at Compute Speed — Ian Quigley, Leash Bio

· Summarized

Share this episode
Anyone with this link can open it.
Share Link
Start at 0:00

Key takeaways

  • DNA-encoded libraries let Leash test millions of pooled small molecules by attaching DNA barcodes, pulling out protein binders, and reading the surviving barcodes through sequencing.
  • Leash’s strategy is to screen many protein constructs against large libraries, because repeated measurements across targets reveal assay artifacts, protein-production failures, and transferable chemical patterns that a one-target DEL screen would miss.
  • The Kaggle competition showed that current protein-ligand models perform well when test molecules share building blocks with training data but no better than chance on a genuinely different chemical library.
  • Public chemistry datasets can encode chemist identity and project-selection habits so strongly that an authorship-probability vector predicts activity about as well as chemical representations, indicating that models may learn sociology rather than binding physics.
  • Leash uses activity cliffs, competitive screens, and engineered protein mutations to create causal evidence about which molecule features interact with which protein residues and to expose model shortcuts.
  • Hermes initially focused on distinctive protein sequences away from binding pockets—effectively memorizing protein nicknames—but adding pocket mutations encouraged it to use more relevant information.
  • Language-model agents can query Leash’s experimentally measured database in rapid reinforcement loops, generating thousands of structure-activity reasoning campaigns instead of relying only on slow physical lab-in-the-loop experiments.
  • Leash believes its long-term value comes from combining a large, physically grounded measurement engine with models that generalize across chemical and protein space, rather than merely finding binders for one target.

Summary

Leash Bio’s premise: measure small-molecule binding at enormous scale

Ian Quigley explains that Leash Bio was founded after he, co-founder Andrew Blevins, and colleagues became dissatisfied with the available data for predicting whether small molecules bind protein targets. Leash combines large-scale experimental data collection, machine learning, and open-science releases. Its core technology is the DNA-encoded library (DEL): instead of testing one chemical per tube or well, millions or even billions of chemically distinct molecules are pooled together, with each molecule physically attached to a DNA barcode. The target protein is attached to a magnetic bead, exposed to the pool, washed, and the barcodes on molecules that remain bound are amplified by PCR and read through DNA sequencing. Quigley says conventional libraries reach the low single-digit millions of molecules, while the largest DEL ever produced reached 40 trillion, although very large libraries create their own problems. DELs are made combinatorially from a smaller set of building blocks, which is especially useful for machine learning because many molecules differ by only one or two building blocks. Those near-neighbor comparisons can reveal which part of a molecule is driving binding. Sequencing is attractive because it is cheap, established, and highly scalable; Quigley calls pictures and DNA sequencing the only truly scalable readouts for many life-science problems. He also notes that DEL experiments need carefully designed negative controls, because DNA-binding proteins can create false positives and PCR can amplify some barcodes more efficiently than others.

Why DELs are difficult—and why Leash screens hundreds of targets

The hosts challenge Quigley on the standard objections to DELs: the DNA tag can alter how a molecule behaves, proteins may bind DNA directly, pooled chemistry can mislabel molecules, and hits found with the tag may fail when synthesized without it. Quigley agrees that these are real issues. Leash gives each chemical molecule 32 or 48 independent barcodes, making it possible to distinguish a genuine molecular signal from a barcode-specific artifact. When a hit is advanced as a drug candidate, Leash synthesizes the molecule without DNA and tests it in functional assays; Quigley reports roughly 90% success for easy cases and about 50% for difficult ones. For machine-learning purposes, however, he argues that the DNA-tagged measurement is still real data. The model is explicitly given a placeholder for the DNA attachment point rather than pretending the tag is absent. Repeated screens using fresh protein and library batches usually recover the same hits in approximately the same order. Leash’s strategy differs from traditional DEL groups that spend months preparing a large protein batch and then use a huge library against one difficult target. Leash instead screens enormous numbers of protein constructs, often hundreds of targets per library, allowing it to identify chemistry-specific artifacts, manufacturing differences, and protein-production failures. Quigley attributes this approach to his team’s genomics and high-dimensional-biology background, including lessons from Recursion about detecting batch effects and preventing models from exploiting irrelevant experimental metadata.

Small molecules, safety, and Leash’s two-dimensional data strategy

Quigley describes Leash as a machine-learning small-molecule company: it collects large numbers of measurements of small molecules binding protein targets, trains models on those data, and moves selected chemical matter toward patients or partners. Small molecules matter because they are usually pill-compatible, can enter cells, and can cross barriers such as the blood-brain barrier. Antibodies and other protein therapeutics are larger, often require injection, and have difficulty reaching intracellular targets. Small molecules are more broadly distributed in the body, however, which creates risks including liver toxicity, heart arrhythmias, and other off-target effects. Leash’s intended solution is to measure much more than one target. If the company screens a candidate against many human proteins, it can predict which off-target interactions might cause toxicity. Quigley says the longer-term aspiration is to screen every protein, including proteins from other species, because a compound may behave differently in mice, rats, and humans. He distinguishes these direct protein-binding measurements from more emergent properties—cell penetration, complex cellular effects, or cardiac behavior—which require cellular assays. The company’s data strategy is therefore deliberately two-dimensional: vary the number and diversity of chemicals as well as the number and diversity of proteins. Rather than testing three proteins against a billion molecules, Leash may prefer thousands of proteins against single-digit millions of molecules, with both broad chemical diversity and fine-grained near-neighbor changes.

The Kaggle competition showed that protein-ligand prediction fails out of distribution

Leash organized a Kaggle competition to test whether models could predict small-molecule binding under realistic distribution shifts. The team used a roughly 130-million-molecule DEL from Shaji Liu’s company Alphamab Shanghai. Each molecule shared a triazine hub and had three variable spokes. The hidden evaluation sets were designed at three difficulty levels: molecules sharing a building block with training examples; molecules sharing only the central hub; and molecules from a completely different library with a different hub, attachment chemistry, and building blocks. After about 2,000 teams competed, models performed best when building blocks were shared, worse when only the hub was shared, and no better than chance on the fully different library. The winner, Viktor Shlapov, used a somewhat different approach, but three of the top five entries were essentially noisy tutorial notebooks. Leash had expected a possibility that new structure-prediction advances could make its business unnecessary, but that did not happen. The competition instead showed that current models often exploit memorization of building blocks or library-specific structure rather than learning transferable binding physics. The team also compared molecular representations. Participants tried SMILES strings, three-dimensional graphs, and other more elaborate encodings, but basic SMILES performed at least as well as the alternatives on this problem. Leash released the entire dataset—including raw read counts, not merely processed labels—through Polaris so that others could inspect the evaluation and attempt to improve on it.

Hermes uses cross-attention, but Leash keeps testing for memorization

After the competition, Leash concluded that it needed more protein targets, better characterization of a smaller chemical library, and evaluations that prevented cheap memorization. The resulting Hermes model was intentionally simple rather than state of the art. It combines an ESM protein embedding with a ChemBERTa-style chemistry embedding that predicts SMILES tokens, joins them with cross-attention, and predicts a binary binding label. Cross-attention was chosen partly because the team’s earlier graph-model experiments suggested that attention could help the model examine relevant parts of a protein and molecule rather than relying only on local motifs. Leash evaluated Hermes using increasingly difficult splits. Models received an advantage when chemical building blocks appeared in both training and test sets, so the team clustered building blocks by similarity and kept entire clusters out of the test set. XGBoost performance fell steadily as the split moved farther out of distribution, while Hermes was comparatively stable. Quigley says this result was encouraging but not proof that Hermes had learned the underlying physics. The team also tested Hermes against external datasets rather than training on public data. Across roughly 13 million tests and hundreds of protein targets, Hermes performed about as well as the contemporary structural model from the Boltz team on the binding task. A revealing case involved BRD4: Boltz performed perfectly on a molecule that was apparently already represented in the Protein Data Bank training data, while Hermes performed poorly on that case but better on other chemical libraries. Leash interpreted this as evidence that the models’ apparent capabilities depend strongly on what distributions they have memorized.

The “chemist fingerprint” revealed that public chemistry data contains sociology

Quigley recounts the incident that motivated Leash’s authorship experiment. At Recursion, a medicinal chemist glanced at a molecule on a screen and immediately said it looked like a Stuart Trevor molecule. Quigley and Blevins were alarmed that someone could identify the chemist from the structure alone. If a model can do the same, they reasoned, it may predict a chemist’s habits and project choices rather than chemical ground truth. Leash downloaded molecules and activity data from ChEMBL, selecting roughly 1,815 chemists who had published at least about 30 papers and 600 molecules. A classifier asked to identify the chemist from the molecule alone achieved 60% top-five accuracy. The model was most confused between chemists who worked in the same institution or on the same publications, suggesting that individual chemical styles—and styles transmitted from supervisors to trainees—are highly distinctive. Leash then represented each molecule not by a chemical structure but by an approximately 1,800-element vector containing the probabilities that each chemist created it. A model trained on that authorship vector predicted activity about as well as a model using chemical formulas. Combining the authorship representation with chemical information did not improve results meaningfully. Quigley’s conclusion is that public medicinal-chemistry datasets encode “the sociology of what gets made”: chemists choose which compounds to synthesize, which to abandon, and which project-specific motifs to favor. Even simple features such as molecular weight can appear highly predictive because successful compounds tend to receive more rounds of optimization and acquire extra functional groups—not because heavier molecules inherently bind better. He compares this to Clever Hans, the horse that seemed to do arithmetic but was actually reading its owner’s posture and microexpressions.

Activity cliffs, engineered mutations, and physically grounded model correction

Leash tries to force its models to learn causal relationships by collecting activity cliffs: pairs of proteins or molecules that are nearly identical but behave very differently after a small change. On the chemical side, DELs naturally provide repeated series in which one building block changes. On the protein side, Leash deliberately mutates amino acids in binding pockets and reruns screens. If a mutation removes only the chemical series associated with a known pocket while leaving unrelated binders intact, the team has evidence that it disrupted a specific interaction rather than simply misfolding the whole protein. Quigley describes screening a known competitor, mutating the pocket, and looking for the same families of molecules to disappear. Some mutations can even improve binding, while others eliminate whole clusters. The approach is especially useful for protein families such as kinases, where many proteins differ by only one residue in an important binding region. The resulting maps connect changes in protein sequence to changes in populations of related molecules. These experiments also exposed a weakness in Hermes. Attention maps initially showed the model focusing on short, distinctive amino-acid sequences away from the functional binding pocket and conserved protein domains. Quigley interpreted those sequences as “nicknames” the model could memorize to identify a protein, rather than evidence that it understood the binding mechanism. Leash added engineered pocket mutations to training, then retrained Hermes. Afterward, attention was distributed more evenly across the protein, suggesting that the model was less able to rely on the nickname shortcut. Quigley emphasizes that interpretability was useful not merely for explaining a prediction but for finding data and training interventions that changed the model’s behavior.

Language-model agents turn Leash’s measurements into rapid lab-in-the-loop reasoning

More recently, Leash began representing molecules, proteins, and binding behavior in readable English rather than relying only on embeddings. The team fine-tuned open models, including Qwen- and GLM-family models, to describe molecular structures, explain the differences between molecules, and propose why some molecules bind while nearby examples do not. The agents receive binders, non-binders, and hard negatives, then can query a database containing millions of experimentally measured molecules. The agent proposes a hypothesis, selects additional molecules to test, receives an immediate database result, and updates its explanation. Quigley describes this as a reinforcement-learning loop running at compute speed rather than requiring a robot or a new physical experiment for every iteration. Given ten binders, ten non-binders, and a budget of 100 database queries, the system can search a six-and-a-half-million-molecule space and identify strong binders. The team recently generated more than 10,000 such structure-activity reasoning campaigns, whereas an individual medicinal chemist might encounter only a few dozen campaigns in a career. Quigley argues that natural language may regularize the task better than simply adding parameters to Hermes, because larger conventional models could memorize even the engineered mutants. The agents also need counterintuitive examples and human-unmade molecules; otherwise they may learn chemists’ selection habits instead of chemistry. Leash expects this approach to generalize beyond drug discovery if the chemical reasoning becomes sufficiently reliable, potentially reaching agriculture, veterinary science, and materials science without screening every new domain from scratch.

From DEL hits to medicines: current constraints, business logic, and the next bottleneck

The hosts ask whether DELs are too constrained to support real medicines. Quigley acknowledges that library design limits which chemical geometries and building blocks Leash can explore, and that many useful substitutions—such as moving a nitrogen around a ring—may require custom synthesis. He is less concerned that DNA-compatible chemistry necessarily produces unstable compounds: newer DNA-friendly reactions can create carbon-carbon and carbon-nitrogen bonds, not only easily cleaved amide or peptide bonds. Leash designs libraries with drug-like properties in mind, using optimization functions that incorporate binding, avoidance of unwanted targets, and predicted ADMET properties. Quigley says Leash has moved from library hits to promising PK/PD results in mice within six months and with relatively few medicinal-chemistry cycles for some programs. In one inflammation program, a compound produced the expected changes in mouse blood biomarkers. The company is discussing selective material for hundreds of protein targets with potential partners and may spin out programs rather than advance every opportunity itself. He argues that small-molecule alternatives to antibody therapies could be valuable where pills, room-temperature distribution, intracellular access, or reduced infusion requirements matter. On the business-model question, Quigley says data collection is not sufficient if the goal is to generalize across unmeasured chemistry. Leash has screened about 2,000 protein constructs in roughly 5,000 screens, usually against 6.5 million molecules per screen, and believes its internally generated data can support predictions on external chemical spaces. However, it still plans to expand into more chemical-library geometries and both coarse and fine activity cliffs. The most useful infrastructure improvement would be scalable purification of pooled chemical libraries, enabling cleaner orthogonal validation. He closes by saying Leash wants more molecules, more measurements, machine-learning collaborators, and partners.

Show notes

A few years ago I stumbled upon Leash Bio’s Substack, and was hooked. Be it their Kaggle competition that showed that no, we really haven’t solved protein-ligand binding, or their crazy discovery that knowing a molecule’s author is perhaps all you need...

Follow this show on AudioBrevity and new episodes get summarized for you. Create a free account

Couldn't load the audio. A blocker may be stopping the show's tracking links. Open the file