Large Discovery Models: Empirically-grounded Model-Based Open-Ended Search

The Core Claim

Standard large language models are bad at scientific discovery because they can't reliably estimate how good their own ideas are. This paper introduces the Large Discovery Model (LDM), which pairs an LLM with a separate uncertainty-aware prediction system. The result: LDM discovers better solutions across molecular design, protein engineering, and neural network training—achieving 2.4× better performance on one benchmark and 18% better binding energies for antibodies.

How It Works

The key insight is that LLMs excel at generating plausible candidates from a vast space (molecules, protein sequences, code) but fail at two critical tasks: accurately predicting how well those candidates will perform, and knowing when they're outside their training distribution and thus unreliable.

LDM solves this with a two-part system. First, the LLM stays in its lane: it generates and refines candidate designs based on prompts and feedback. Second, a Bayesian non-parametric surrogate model—essentially a statistical learner that makes fewer assumptions than traditional models—predicts actual performance and quantifies uncertainty around those predictions. This uncertainty measure matters enormously: instead of the LLM's vague "I think this is good," the surrogate says "I predict a score of X, but I'm quite uncertain about it."

The system runs as a loop. The LLM proposes candidates. The surrogate evaluates them and flags uncertainty. That uncertainty-aware feedback steers the LLM toward promising regions of the search space while encouraging exploration of uncertain areas where breakthroughs might hide. As experiments are run and real results come in, both the discovery memory (what the LLM learns over time) and the surrogate model update continuously, improving future proposals.

Why This Matters

Scientific discovery is expensive. Running experiments on molecules or proteins costs time and money. You can't afford to randomly sample the space or even to naively trust an LLM that sounds confident but is actually hallucinating. Traditional search methods (like Bayesian optimization or reinforcement learning) work well in small, structured spaces but break down when the design space is truly open-ended—billions of possible molecules or protein sequences.

LDM bridges this gap. By grounding the LLM's generation with empirical feedback and genuine uncertainty quantification, it navigates massive spaces more efficiently. The empirical results are concrete: on neural network training, it reduces validation loss 2.4× better than LLM-only approaches. For antibodies, it achieves 18.2% better binding energy relative to baselines. On molecular design, it shows 60% relative gains in multi-objective performance.

The broader implication is that discovery engines don't need to choose between expressiveness (what LLMs offer) and accuracy (what traditional statistical methods provide). Coupling them is more powerful than either alone. This suggests a practical tool for wet-lab teams and computational researchers working on drug discovery, materials science, or protein design—anywhere the hypothesis space is huge, evaluations are expensive, and researchers need smarter navigation.