Teacher-student training improves accuracy and efficiency of machine learning interatomic potentials
Machine learning interatomic potentials (MLIPs) are revolutionizing the field of molecular dynamics (MD) simulations. Recent MLIPs have tended towards more complex architectures trained on larger datasets. The resulting increase in computational and memory costs may prohibit the application of these MLIPs to perform large-scale MD simulations. Here, we present a teacher-student training framework in which the latent knowledge from the teacher (atomic energies) is used to augment the students' training. We show that the light-weight student MLIPs have faster MD speeds at a fraction of the memory footprint compared to the teacher models. Remarkably, the student models can even surpass the accuracy of the teachers, even though both are trained on the same quantum chemistry dataset. Our work highlights a practical method for MLIPs to reduce the resources required for large-scale MD simulations.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Noisy student-teacher training for robust keyword spotting
We propose self-training with noisy student-teacher approach for streaming keyword spotting, that can utilize large-scale unlabeled data and aggressive data augmentation. The proposed method applies aggressive data augme…
Data AugmentationKeyword SpottingWeakly Semi-Supervised Detection in Lung Ultrasound Videos
Frame-by-frame annotation of bounding boxes by clinical experts is often required to train fully supervised object detection models on medical video data. We propose a method for improving object detection in medical vid…
object-detectionObject DetectionTransfer LearningDistilling Knowledge from CNN-Transformer Models for Enhanced Human Action Recognition
This paper presents a study on improving human action recognition through the utilization of knowledge distillation, and the combination of CNN and ViT models. The research aims to enhance the performance and efficiency …
Action RecognitionKnowledge DistillationTemporal Action LocalizationPrefix-Guided On-Policy Distillation: Mining Golden Trajectories from Rollouts
On-policy distillation (OPD) improves reasoning models by applying dense teacher supervision on student-sampled trajectories. However, scaling OPD to long-horizon mathematical reasoning exposes a reliability and efficien…
Mathematical ReasoningSelf-training with Noisy Student improves ImageNet classification
We present Noisy Student Training, a semi-supervised learning approach that works well even when labeled data is abundant. Noisy Student Training achieves 88.4% top-1 accuracy on ImageNet, which is 2.0% better than the s…
Data AugmentationGeneral ClassificationImage Classification