A Practical Guide to Transformer-Based Retrosynthesis

Retrosynthesis, the art and science of working backward from a target molecule to identify viable synthesis routes, has been a cornerstone of organic chemistry since E.J. Corey formalized it in the 1960s. For decades, it remained a purely human endeavor, relying on chemists' deep knowledge of reaction mechanisms, reagent compatibility, and practical laboratory considerations.

Computer-aided retrosynthesis has been attempted since the 1970s, but early systems were rule-based, requiring expert chemists to manually encode reaction templates. These systems were brittle: they could only propose disconnections for reaction types someone had thought to encode, and they struggled with novel chemistry.

The transformer revolution in NLP opened a new path. By treating retrosynthesis as a sequence-to-sequence translation problem (translating a product SMILES string into a reactant SMILES string), we can learn reaction patterns directly from data, without manual template encoding.

At Gimerny, GimernySynth is built on this foundation. Here is what we have learned from training on 15 million reactions and deploying the system with medicinal chemistry teams.

Framing Retrosynthesis as Translation

The key insight is simple: a chemical reaction can be written as a string transformation. The product molecule has a SMILES representation, and so do the reactants. Single-step retrosynthesis becomes: given the product SMILES, predict the reactant SMILES.

We use a standard encoder-decoder transformer architecture with 12 encoder layers, 12 decoder layers, 512 hidden dimensions, and 8 attention heads. The tokenizer operates at the character level on SMILES strings, with special tokens for reaction roles (reactant separator, reagent indicator) and stereochemistry.

Training data comes from three sources: the USPTO reaction dataset (1.8M reactions, cleaned and canonicalized), the Reaxys database (12M reactions, licensed), and proprietary reaction data from partner institutions (1.5M reactions). We train with teacher forcing, using cross-entropy loss on the predicted reactant tokens.

Beyond Single-Step: Multi-Step Route Planning

Single-step retrosynthesis is a building block, not a solution. A real synthesis route requires multiple steps, often 3-8, each of which must use available starting materials and be compatible with the functional groups present.

We embed our single-step model inside a Monte Carlo tree search (MCTS) framework. The search tree starts with the target molecule at the root. At each node, the single-step model proposes the top-K disconnections (we use K=50). Each proposed set of reactants becomes a child node. The search continues recursively until all leaf nodes are commercially available starting materials (checked against our supplier database of 4.2M purchasable compounds) or the maximum depth is reached.

The MCTS scoring function balances multiple objectives: total step count (shorter is better), predicted yield at each step, cost of starting materials, and a novelty bonus that encourages exploration of non-obvious routes.

What Makes Routes "Good"

A common failure mode of automated retrosynthesis is producing routes that are technically correct but practically useless. A route requiring a palladium catalyst at minus 78 degrees Celsius in a sealed tube is valid chemistry but impractical for large-scale synthesis. A route using a starting material that costs $10,000 per gram is theoretically feasible but commercially unviable.

We address this through a multi-factor route scoring function trained on feedback from medicinal chemists. Factors include step count and linear sequence length, estimated cost of goods based on real supplier pricing, scalability (preference for reactions that work at multi-gram scale), safety and environmental considerations (penalty for highly toxic or explosive intermediates), and precedent (bonus for well-established reaction types with high reported reproducibility).

This scoring function is trained on 2,400 route evaluations provided by our chemistry advisors, where expert chemists rated proposed routes on a 1-5 scale for practical utility.

Stereochemistry: The Hard Problem

Most published retrosynthesis models struggle with stereochemistry. Biological activity is often stereospecific: one enantiomer of a drug may be therapeutic while the other is inactive or toxic. Yet SMILES strings encode stereochemistry with subtle notation (@ and @@ symbols for tetrahedral centers, / and \ for double bond geometry) that models can learn to copy but often fail to reason about.

Our approach uses an augmented representation that explicitly encodes stereocenters as separate tokens with spatial context. We also apply a post-processing step that verifies stereo-consistency: checking that the proposed retrosynthetic step preserves or correctly inverts each stereocenter according to the reaction mechanism.

On the USPTO stereo benchmark, this approach achieves 84% top-1 accuracy for stereo-correct predictions, compared to 61% for a standard SMILES transformer.

Deployment Lessons

Three lessons stand out from deploying GimernySynth with medicinal chemistry teams. First, chemists want options, not answers. Presenting a single "optimal" route is less useful than presenting 5-10 diverse routes with different trade-offs (shorter but more expensive, longer but using robust chemistry, etc.). Second, explanations matter. For each proposed disconnection, we highlight the bonds being broken and map the transformation to known reaction classes (Suzuki coupling, amide formation, etc.), allowing chemists to quickly assess plausibility. Third, integration with inventory is essential. A route that uses a starting material the lab already has on the shelf is far more valuable than one requiring a new order. We integrate with electronic lab inventory systems to prioritize routes using in-stock reagents.

The future of retrosynthesis is not replacing chemists but augmenting them: generating the long list of possibilities that a human expert then curates based on practical experience and project-specific constraints.