Zero-Shot Transfer of Neural ODEs

Abstract

Autonomous systems often encounter environments and scenarios beyond the scope of their training data, which underscores a critical challenge: the need to generalize and adapt to unseen scenarios in real time. This challenge necessitates new mathematical and algorithmic tools that enable adaptation and zero-shot transfer. To this end, we leverage the theory of function encoders, which enables zero-shot transfer by combining the flexibility of neural networks with the mathematical principles of Hilbert spaces. Using this theory, we first present a method for learning a space of dynamics spanned by a set of neural ODE basis functions. After training, the proposed approach can rapidly identify dynamics in the learned space using an efficient inner product calculation. Critically, this calculation requires no gradient calculations or retraining during the online phase. This method enables zero-shot transfer for autonomous systems at runtime and opens the door for a new class of adaptable control algorithms. We demonstrate state-of-the-art system modeling accuracy for two MuJoCo robot environments and show that the learned models can be used for more efficient MPC control of a quadrotor.

Our algorithm consists of an offline learning phase and an online execution phase. During the offline phase, we learn a set of basis functions that span the space of dynamics. This space of dynamics can arise due to unknown system parameters, such as a quadrotor with an unknown mass. The basis functions are learned via the function encoder algorithm, and each basis function is a neural ODE. During the online phase, we use the learned basis functions and a small online dataset to identify the dynamics of the system. This online adaptation requires zero gradient updates, and can be computed in real-time. Then, this model can be used for downstream tasks, such as MPC.

Accurate Long Horizon Predictions

We demonstrate that our learned models can accurately predict the dynamics on the MuJoCo HalfCheetah and Ant environments. We vary numerous environment parameters, such as lengths of the limbs and the control authority. Then, given a small online dataset, we attempt to predict the dynamics k steps into the future. Our results indicate that we achieve both the accurate long horizon predictions of neural ODEs and the online adaptability of function encoders.

Online Adaptation for Model Predictive Control

We apply this algorithm to the MPC control of a quadrotor. The quadrotor's mass is varied every episode, and the goal is to reach prespecified coordinates. Our results demonstrate that our approach makes more accurate long horizon predictions in the presence of hidden parameters than neural ODEs alone. When measuring the slew rate of the controller, i.e. its efficiency, we observe that this improved prediction accuracy leads to more efficient control of the quadrotor. An example trajectory

Qualitatively, this difference is visible in the trajectories of the respective controllers. Here we are plotting one trajectory from each controller for the same conditions. We observe that the neural ODE controller alone makes repeated corrections to its trajectory. In contrast, our approach more smoothly approaches the target coordinates.

BibTeX

@misc{functionEncoderNeuralODEs2024,
      title={Zero-Shot Transfer of Neural ODEs},
      author={Tyler Ingebrand and Adam J. Thorpe and Ufuk Topcu},
      year={2024},
      journal={NeurIPS}.
      }