Skip to collection list Skip to video grid
Skip to collection list Skip to video grid

SC20

Scaling out Transformer “Attention” Model for Retrosynthesis for Chemical Reaction Prediction on Intel® Xeon HPC Supercomputers

Retrosynthesis is the task of building a molecule from smaller precursor molecules. There is a strong analogy between Retrosynthesis and Machine Translation (MT), i.e. each target molecule can have predecessors from which it can be synthesized, similar to source to target language translation. The reactions can be represented using SMILES string-based notation for correctly predicting molecule, e.g. “COC(=O)c1cccc(-c2nc3cccnc3[nH]2)c1  Nc1cccnc1N + COC(=O)c1cccc(C(=O)O)c1”. By treating the problems as a MT problem, the reactants are generated token by token, or in this context, atom by atom. Previous work has shown that good results can be achieved on this task with deep learning techniques, such as the multiheaded “Attention” Transformer Model. In retrosynthesis, the task can be treated as a MT problem where the Transformer model predicts the precursor molecules given a string representation of the target molecule. In this talk, we investigate the effect of distributed training of the Transformer model for the retrosynthesis on supercomputers using 50,000 chemical reactions from USPTO database. We investigate the issues that arise when scaling Transformers to a distributed system like learning rate scheduling and choice of optimizer, after which we present strategies that improve upon results from previous research. Our experiments are conducted on 128 node 2-CPU Intel® Xeon® processor HPC cluster on which we achieve a computational speedup of 70% with over 97.5% training accuracy. In addition, we show that we are able to achieve 2.5% better than previous state-of-the-art accuracy. Learn how a Language Translation AI model is used for Chemical Reaction translation using Intel Xeon® HPC Infrastructure for distributed training. Joris Mollinga, SURF

Read More
Read Less

categories

View more in
SC20

Currently loaded videos are 1 through 15 of 40 total videos.

First page loaded, no previous page available
Load Next Page