The dark side of the forces: assessing non-conservative force models for atomistic machine learning

The dark side of the forces: assessing non-conservative force models for atomistic machine learning

Speakers: Filippo Bigi, Marcel Langer, Michele Ceriotti

Conference: ICML 2025

YouTube: https://slideslive.com/39044075

Overview

This talk delves into the critical challenges and innovative solutions for integrating machine learning models into classical mechanics simulations, particularly in the realm of atomistic molecular dynamics. Presented by Filippo Bigi, Marcel Langer, and Michele Ceriotti, the research addresses a fundamental dilemma in ML-driven physical simulations: the trade-off between computational speed and physical correctness. While directly predicting forces with neural networks (non-conservative models) offers significant speed advantages, it often leads to simulations that violate fundamental physical laws, rendering their results unreliable.

The core of the presentation explores the inherent limitations of non-conservative force fields and introduces a two-pronged solution designed to harness their speed without compromising the integrity of the simulated dynamics. By proposing a hybrid training methodology and a novel inference-time technique called Multiple Time Stepping, the speakers demonstrate how to achieve substantial speedups—2 to 3 times faster training and simulations—while meticulously preserving physical correctness. This work is highly significant for fields ranging from materials science and drug discovery to environmental chemistry, where accurate and efficient atomistic simulations are paramount for designing new materials, understanding complex biological processes, and optimizing industrial reactions like the Haber process.

The implications of this research are far-reaching. By enabling faster, yet physically sound, simulations, the methods presented pave the way for tackling larger systems, exploring longer timescales, and conducting more extensive parameter searches than previously feasible. This not only accelerates scientific discovery but also enhances the reliability of computational predictions, ensuring that ML models serve as truly powerful and trustworthy tools for scientific exploration rather than just fast approximators.

Background

▶ Watch: Introduction to force models for ML simulations (0:00)

Classical mechanics simulations form the bedrock of much scientific and engineering inquiry, enabling researchers to predict the behavior of systems from planetary orbits to molecular interactions. At their heart, these simulations require two primary inputs: the initial conditions of all objects (their positions x₀ and velocities v₀) and the equations of motion, famously encapsulated by Newton's second law, F = MA. Here, F represents the forces acting on the objects, which are typically represented as a vector field dependent on the objects' positions F(x).

In recent years, machine learning, particularly neural networks, has emerged as a powerful tool for approximating this complex force function F(x). The motivation is clear: traditional force fields, often derived from ab initio quantum mechanical calculations or empirical potentials, can be computationally expensive or lack the generalizability required for diverse chemical environments. ML models, trained on vast datasets of atomic configurations and corresponding forces, promise to bridge this gap by offering highly accurate and computationally efficient force predictions.

However, the integration of ML into physical simulations introduces a critical architectural choice with profound consequences. Broadly, there are two main ways to represent forces through neural networks:

1. Conservative Force Models: In this approach, the neural network is trained to predict a scalar potential energy function V(x). The forces F are then derived from this potential by taking its negative gradient: F = -∇V. This method inherently imposes a mathematical structure on the force field, specifically ensuring that its curl is zero. Force fields derived from a potential energy function are called conservative because they naturally lead to systems where total energy is conserved over time. This structure directly reflects the underlying physics, making these models generally more accurate and reliable, albeit at a higher computational cost due to the required backpropagation step for gradient calculation during inference.

1. Non-conservative Force Models: Alternatively, the neural network can be trained to directly predict the force vector field F(x). This approach treats F(x) as an unconstrained vector field, meaning it does not necessarily derive from a scalar potential energy function. While this direct prediction avoids the computationally intensive backpropagation step for gradient calculation during inference, making it 2 to 3 times faster, it comes at a severe cost: the energy of the simulated system is not necessarily conserved.

The problem, as highlighted by the speakers, is that this lack of energy conservation in non-conservative models is not a minor artifact but a fundamental flaw that renders simulations physically unreliable. For instance, a simulation of liquid water using a non-conservative force model would show the system's temperature rising steeply by hundreds of degrees within seconds, an effect that, when scaled to human perception, implies billions of degrees of heating per second. This dramatic heating is not just an inconvenience; it signifies a complete breakdown of the underlying physics. Without a defined potential energy function, the Hamiltonian of the system cannot be established, which is a cornerstone of classical dynamics. This absence leads to a lack of symplecticity (a property crucial for long-term stability in Hamiltonian systems) and violates the equipartition of energy, where different degrees of freedom should, on average, possess the same amount of energy. Attempts to artificially conserve energy using techniques like velocity rescaling or thermostats also fail, as they merely hide the symptom without addressing the root cause, leading to physically erroneous outcomes such as different atomic species being simulated at vastly different temperatures. This profound unreliability has limited the adoption of faster non-conservative ML force fields in high-fidelity scientific simulations.

Key Findings

▶ Watch: The problem: non-conservative forces cause heating (3:00)

The central discovery of this work is the confirmation that while directly learning force fields (non-conservative models) offers significant speed advantages (2-3x faster), it fundamentally undermines the physical correctness of simulations, leading to severe energy conservation issues and unreliable dynamics. The talk meticulously illustrates this with examples, such as the uncontrolled heating of liquid water, demonstrating that these models violate core principles like equipartition of energy and the existence of a Hamiltonian.

To address this critical trade-off between speed and physical fidelity, the researchers propose a two-part solution:

1. Hybrid Training Strategy: They introduce a novel training methodology that combines the speed of non-conservative learning with the accuracy of conservative models. The process begins with non-conservative pre-training, where the neural network directly learns the force field F(x). This fast initial phase allows the model to quickly capture the general force landscape. Subsequently, the model undergoes conservative fine-tuning, where a different head of the network is used to predict the potential energy V(x), and forces are then derived via backpropagation. This fine-tuning stage, while more computationally intensive, is performed only at the end of the training process. This hybrid approach successfully recovers the full accuracy of purely conservative training while reducing the overall GPU time by approximately two-thirds (i.e., using only one-third of the GPU time).

1. Multiple Time Stepping (MTS) for Inference: For the simulation (inference) phase, the team proposes adapting an established method from classical Hamiltonian dynamics called Multiple Time Stepping. In their application, this translates to a scheme where the computationally cheap non-conservative forces are evaluated at every simulation step. Critically, the more expensive conservative forces are evaluated only every M steps, acting as a periodic correction. The research demonstrates that by pushing M up to approximately 10, the method maintains correct dynamics and completely conserves energy, effectively achieving the speed benefits of non-conservative evaluations with the physical reliability of conservative ones. This allows for simulations that are as fast as non-conservative ones but with a perfectly conserved energy profile.

These combined innovations yield practical and significant benefits: practitioners can achieve 2 to 3 times faster training and simulations without compromising physical correctness. The research further showcases the power of these ideas through concrete applications:

• PetMAd: A universal force field for chemical systems, which achieves a 10x speedup in inference times. This speedup is attributed to a 3x increase from using rotationally unconstrained models and another 3x increase from leveraging the proposed non-conservative force techniques.
• FlashMD: An even more advanced method for molecular dynamics, achieving an astounding 30x speedup over non-conservative forces. FlashMD operates by directly learning the time evolution of the simulation from one step to a future step, entirely bypassing the need to integrate equations of motion at each intermediate step.
• Macroscopic Physics Simulations: The principles are shown to extend beyond atomistic scales, demonstrating the ability to simulate macroscopic physics, such as liquid behavior, with the developed techniques.

In essence, the key findings demonstrate a robust pathway to developing ML force fields that are both highly efficient and rigorously physically sound, overcoming a major hurdle in the application of AI to scientific discovery.

Technical Deep Dive

▶ Watch: Why non-conservative models lead to broken dynamics (4:00)

The technical core of this work lies in understanding the inherent differences between conservative and non-conservative force models, and then ingeniously combining their strengths while mitigating their weaknesses.

Force Model Architectures

At the heart of any classical simulation is the force F acting on particles, which is a function of their positions x. Machine learning models aim to approximate this function F(x).

• Conservative Models: These models learn a scalar potential energy function V(x) using a neural network. The forces F are then calculated by taking the negative gradient of this potential: F = -∇V.
• Mathematical Structure: This approach imposes a crucial mathematical constraint: the force field must have zero curl (∇ × F = 0). This property ensures the existence of a potential energy function and, consequently, a well-defined Hamiltonian for the system. The presence of a Hamiltonian is fundamental for classical dynamics, guaranteeing properties like symplecticity (which ensures long-term stability and preservation of phase space volume) and the equipartition of energy (where, in thermal equilibrium, each degree of freedom holds an average energy of kT/2).
• Computational Cost: During inference (simulation runtime), calculating the forces requires a backpropagation step through the neural network to compute the gradient of V(x). This operation is computationally more expensive, typically making conservative models 2 to 3 times slower than direct force prediction.

• Non-conservative Models: These models directly train a neural network to output the force vector F(x).
• Mathematical Structure: This approach treats F(x) as an unconstrained vector field. It does not guarantee zero curl, meaning a potential energy function V(x) cannot generally be defined. Without a V(x), there is no Hamiltonian, leading to a breakdown of fundamental physical principles. Simulations with non-conservative forces exhibit pathological behaviors such as uncontrolled heating, violation of equipartition, and a lack of symplecticity.
• Computational Cost: During inference, the neural network directly predicts F(x), avoiding the backpropagation step. This makes non-conservative models significantly faster (2 to 3 times faster) than conservative ones.

Hybrid Training Methodology

To overcome the speed-accuracy dilemma, the researchers propose a two-stage training process:

1. Non-conservative Pre-training: In the initial phase, the neural network is configured to directly predict the force vector F(x). This phase leverages the computational efficiency of non-conservative models to quickly learn the general features of the force field from the training data. The model rapidly converges to a reasonable (though not perfectly accurate) representation of the forces. As shown in the training profile, this red curve quickly achieves a low Mean Absolute Error (MAE) but then saturates at a higher value compared to conservative training, as it cannot capture the inherent mathematical structure of the data.

1. Conservative Fine-tuning: After the initial pre-training, the model undergoes a fine-tuning stage. Here, a different "head" or output layer of the neural network is activated, designed to predict the scalar potential energy V(x). The forces are then computed by taking the negative gradient (-∇V) of this predicted potential. This phase is more computationally intensive due to backpropagation, but it is applied only to refine the model. This fine-tuning allows the network to learn the crucial mathematical structure (zero curl) inherent in physically correct force fields. The blue curve in the training profile demonstrates that this fine-tuning step effectively "recovers" the full accuracy of a purely conservative model (the black curve) but achieves it in approximately one-third of the total GPU time that a full conservative training run would require.

Multiple Time Stepping (MTS) for Inference

Even with an accurately trained conservative model, the inference step (running the simulation) remains slower. To address this, the team adapts Multiple Time Stepping, a well-known technique in classical Hamiltonian dynamics. The core idea is to differentiate between "fast" and "slow" components of the forces, evaluating them at different frequencies. In this context:

• Fast (Cheap) Forces: The non-conservative force predictions (F_NC(x)) are considered the cheap component. These are evaluated at every simulation time step due to their low computational cost.
• Slow (Expensive) Forces: The conservative force predictions (F_C(x) = -∇V(x)) are the expensive component. These are evaluated less frequently, specifically only every M simulation steps, serving as a periodic "correction" to the dynamics driven by the fast forces.

The key insight is that by carefully choosing M, the simulation can maintain the energy conservation and correct dynamics characteristic of conservative forces, while benefiting from the speed of non-conservative evaluations most of the time. The researchers found that correct dynamics and perfect energy conservation could be achieved up to M = 10. Since conservative force evaluations are at most 3 times slower than non-conservative ones, evaluating them only once every 10 steps effectively amortizes their cost, leading to an overall simulation speed approaching that of purely non-conservative models, but with guaranteed physical correctness. The energy profile of liquid water using MTS is shown to be as stable as purely conservative simulations.

FlashMD: Learning Time Evolution

Pushing the boundaries further, the talk introduces FlashMD, a highly accelerated molecular dynamics method. Instead of predicting forces at each step and numerically integrating the equations of motion, FlashMD attempts to learn the time evolution of the system directly. The neural network is trained to predict the system's state (positions and velocities) at a future time step, skipping N intermediate steps entirely. This direct trajectory prediction avoids the cumulative errors and computational overhead of step-by-step integration.

• Mechanism: The model learns a mapping from (x_t, v_t) to (x_{t+NΔt}, v_{t+NΔt}).
• Speedup: This approach yields a remarkable 30x speedup over even non-conservative force evaluations. The researchers were able to skip about 30 steps before encountering limitations.
• Limitations: Beyond approximately 30 steps, molecular dynamics simulations exhibit chaotic effects. The target function (the future state) becomes highly oscillatory and effectively "unlearnable" for current neural network architectures. While the underlying dynamics remain deterministic, the complexity makes direct prediction over very long horizons challenging. FlashMD has been successfully applied to reproduce delicate phenomena like superionic transitions and is being considered for complex tasks like protein folding, where such speedups could be transformative.

These technical advancements represent a significant leap in the capability of ML-driven simulations, offering a principled way to achieve both high performance and physical accuracy.

Experimental Setup & Results

▶ Watch: Solution 2: Faster, stable simulations with multiple time stepping (6:00)

The experimental validation of the proposed methods primarily centered on atomistic simulations, with a focus on molecular dynamics, a widely used technique in chemistry and materials science. The core demonstration involved the simulation of liquid water, a crucial system for biology and environmental science, due to its well-understood properties and sensitivity to physical perturbations.

Baseline Comparisons

The initial experiments established a stark contrast between conservative and non-conservative force models:

• Conservative Force Models: When simulating liquid water with a conservative force field (derived from a potential energy function), the system's temperature profile remained remarkably stable and constant over the simulation duration. Minor fluctuations were observed, attributed to finite-sized effects inherent in any simulation, but the overall temperature was conserved, demonstrating physical fidelity.
• Non-conservative Force Models: In stark contrast, running the same liquid water simulation with a non-conservative force model resulted in a dramatic and steep rise in temperature, increasing by "a few hundred degrees" over the simulation period. This uncontrolled heating was highlighted as a severe physical unreliability, equivalent to "several billion degrees of heating per second" on a human scale, which would lead to immediate evaporation. The underlying issues were identified as a violation of the equipartition of energy (e.g., oxygens simulated at 200 degrees hotter than hydrogens) and the absence of a well-defined Hamiltonian, leading to broken symplecticity and physically meaningless dynamics.
• Artificial Energy Conservation: Attempts to mitigate the non-conservative heating using artificial methods like velocity rescaling or thermostats were also tested. While these methods could prevent the macroscopic temperature rise, they failed to restore physical correctness. The equipartition of energy remained violated, demonstrating that these ad-hoc solutions merely masked the problem without addressing the fundamental breakdown of the physics.

Proposed Solutions: Performance & Accuracy

The efficacy of the proposed hybrid training and inference-time Multiple Time Stepping (MTS) solutions was rigorously demonstrated:

• Hybrid Training Profile: A training profile (loss vs. training steps) illustrated the benefits:
• The black curve represented purely conservative training, which was slow but achieved very high accuracy (low MAE).
• The red curve represented non-conservative training, which was fast but saturated at a higher MAE, indicating its inability to fully capture the mathematical structure of the data.
• The blue curve depicted the hybrid approach: non-conservative pre-training followed by conservative fine-tuning. This curve demonstrated that the hybrid method successfully recovered the full accuracy of the conservative training (matching the black curve's final MAE) but achieved this in approximately one-third of the GPU time required for a purely conservative training run. This represents a significant 2-3x speedup in training for physically correct models.

• Multiple Time Stepping (MTS) for Inference: The MTS method was applied to the liquid water simulation.
• Energy Conservation: The results showed that MTS perfectly conserved the system's energy profile, matching the stability of purely conservative simulations. The temperature remained constant, and the physical integrity of the dynamics was restored.
• Speed: Crucially, MTS achieved this energy conservation while delivering speeds "basically the same speed as a non-conservative simulation." This was possible by evaluating the cheap non-conservative forces at every step and the more expensive conservative forces only every M steps. The research found that M = 10 was a practical limit where correct dynamics and energy conservation were maintained, effectively achieving a near 3x speedup in physically correct simulations.

Advanced Applications & Benchmarks

The developed methodologies were not just theoretical but applied to real-world challenges, yielding impressive benchmarks:

• PetMAd (Universal Force Field): The principles were incorporated into PetMAd, a universal force field designed for chemical systems. This model achieved an overall 10x increase in inference speed compared to earlier attempts at universal force fields. This speedup was broken down into two components: a 3x speedup from using rotationally unconstrained models and another 3x speedup attributable to the efficient use of non-conservative forces through the proposed hybrid training and MTS techniques.
• FlashMD (Molecular Dynamics): The talk introduced FlashMD, a method for molecular dynamics that directly learns the time evolution of the system. FlashMD demonstrated an even more dramatic speedup, being 30 times faster than non-conservative force evaluations. This was achieved by learning to skip approximately 30 simulation steps. FlashMD was shown to be capable of reproducing "pretty delicate physical phenomena like superionic transitions" and is being explored for applications in protein folding, where its immense speed could be transformative.
• Macroscopic Physics: The scale-invariance arguments extended the applicability of these techniques beyond atomistic details. A simulation of a macroscopic liquid was presented, showcasing that the methods could be used to model larger-scale physical phenomena, broadening their utility significantly.

These experimental results unequivocally demonstrate that the proposed solutions effectively bridge the gap between computational efficiency and physical reliability, offering powerful new tools for scientific discovery.

Practical Implications

▶ Watch: Conclusion: Faster, physically reliable ML-driven simulations (7:00)

The research presented on reconciling the speed of non-conservative force models with the physical correctness of conservative ones carries profound practical implications across various domains of scientific computing and machine learning.

For practitioners and researchers in fields such as materials science, chemistry, and biology, this work means a significant acceleration of simulation workflows. The ability to run physically correct atomistic simulations 2 to 3 times faster directly translates to:

• Expanded Scope: Simulating larger systems, exploring longer timescales, or conducting more extensive high-throughput screening of materials and drug candidates.
• Faster Iteration: Rapidly testing hypotheses and exploring chemical reaction pathways, accelerating the design-build-test-learn cycle.
• Enhanced Reliability: Trusting that the accelerated simulations adhere to fundamental physical laws, reducing the risk of making decisions based on unphysical dynamics.

Infrastructure teams supporting ML-driven simulations will find benefits in terms of resource utilization. The hybrid training approach, which achieves conservative accuracy in one-third of the GPU time, implies substantial savings in computational resources and energy costs during model development. This can lead to more sustainable and cost-effective ML model training pipelines.

For model builders and developers creating ML force fields, this research provides a critical architectural blueprint. It highlights that simply chasing speed by directly predicting forces is insufficient for scientific applications. Instead, it offers a principled way to design models that inherently enforce physical constraints without being prohibitively slow. The hybrid training strategy and the Multiple Time Stepping (MTS) inference method become essential components in the toolkit for building next-generation, high-fidelity ML potentials.

Deployers of ML models in production environments, such as those running simulations for industrial processes or drug discovery platforms, can now confidently integrate these accelerated models. The PetMAd universal force field, with its 10x speedup, and the even faster FlashMD (30x speedup) represent tangible advancements that can be directly leveraged to enhance the throughput and capabilities of existing simulation platforms. FlashMD's potential for protein folding, for example, could revolutionize drug discovery by enabling previously intractable simulations of complex biological processes.

Tradeoffs and Limitations

While the advancements are substantial, certain tradeoffs and limitations are inherent:

• Multiple Time Stepping (MTS) Parameter M: The efficiency and accuracy of MTS depend critically on the choice of M (the frequency of conservative force corrections). While M=10 was identified as a sweet spot for liquid water, optimal M values might vary for different systems, force fields, or desired levels of precision. Careful tuning and validation would be required for new applications. Pushing M too high risks reintroducing physical inaccuracies.
• FlashMD's Horizon Limitation: FlashMD achieves remarkable speed by directly predicting trajectories, but it is currently limited to skipping approximately 30 steps due to the onset of chaotic effects in molecular dynamics. Beyond this horizon, the target function becomes too oscillatory and complex for current architectures to learn effectively. This means FlashMD is not a "one-shot" solution for arbitrarily long simulations but rather a powerful tool for accelerating segments of trajectories, which then need to be periodically re-initialized or corrected. Further research is needed to overcome this "chaotic barrier" for even longer-range predictions.
• Generality of Solutions: While the principles are broadly applicable, the specific implementations and optimal parameters might require adaptation for different types of atoms, chemical bonds, or physical environments. The "universal force field" goal is ambitious and requires continuous validation across diverse chemical spaces.
• Computational Overhead for Gradient Calculation: Even with fine-tuning, the conservative force calculation still involves backpropagation, which is inherently more complex than a simple forward pass. While MTS mitigates this by reducing frequency, the fundamental cost per evaluation remains.

Despite these considerations, the work significantly pushes the boundary of what is possible with ML-driven physical simulations, offering concrete, validated strategies to achieve both speed and scientific rigor.

Key Takeaways

• Non-conservative machine learning force fields are 2-3 times faster but fundamentally break physical laws like energy conservation, leading to unreliable and unphysical simulation dynamics.
• A novel hybrid training strategy, combining fast non-conservative pre-training with conservative fine-tuning, achieves the full accuracy of purely conservative models in approximately one-third of the GPU training time.
• The Multiple Time Stepping (MTS) method at inference time allows for simulations that maintain perfect energy conservation and physical correctness while achieving speeds comparable to purely non-conservative models (up to M=10 for corrections).
• These combined innovations enable practitioners to perform 2-3 times faster atomistic simulations without sacrificing physical fidelity, critically important for applications in materials science, chemistry, and biology.
• Advanced techniques like PetMAd leverage these principles to achieve 10x speedups for universal force fields, while FlashMD pushes the boundaries further with a 30x speedup by learning direct time evolution, opening new avenues for complex systems like protein folding.
• The research provides a robust framework for bridging the gap between computational efficiency and the rigorous requirements of physical simulations, making ML a more trustworthy tool for scientific discovery.

About the Speaker(s)

The talk was presented by Filippo Bigi, Marcel Langer, and Michele Ceriotti. Their work focuses on the intersection of machine learning and physical simulations, particularly in developing advanced force models for atomistic and molecular dynamics. Their research aims to enhance the speed and accuracy of these simulations, contributing to areas such as materials design, drug discovery, and fundamental understanding of chemical processes. They are key contributors to the development of methods like PetMAd and FlashMD, which represent significant advancements in creating physically correct and highly efficient machine learning potentials for scientific computing.