Abstract

Molecular representation pretraining is critical in various applications for drug and material discovery. Along this research line, most existing work focuses on pretraining on 2D molecular graphs. Meanwhile, the power of pretraining on 3D geometric structures has been recently explored. In this tutorial, I would like to start the introduction to molecule geometric representation methods (group invariant and equivariant representation) and self-supervised learning for pretraining. After this, I will combine these two topics and comprehensively introduce geometric pretraining for molecule representation, discussing the most recent works in detail.

Keywords: Artificial intelligence, machine learning, deep learning, drug discovery, graph representation learning, geometric representation learning, geometric pretraining, self-supervised pretraining, group symmetry, invariance, E(3)-equivariance, SE(3)-equivariance.

Slides: link

Outline

1. Physics-inspired Geometric Modeling
2. Pretraining
3. Density Estimation & Generative Modeling
4. Physics-inspired Geometric Pretraining
• Physics-inspired Unsupervised Pretraining
• Molecule 2D-3D
• Molecule 3D
• Protein 3D
• Material 3D
• Physics-inspired Supervised Pretraining
• Positive Effect
• Negative Effect
5. Physics-inspired Geometric Pretraining and LLM
• Molecule
• Protein
• Material
6. Conclusion