On May 11th, our Head Enrique Zuazua talked about “Neural Differential Equations, Control and Machine Learning” at StudOn AG “Mathematics and Deep Learning”.
Abstract. We discuss Neural Ordinary Differential Equations (NODEs) from a control theoretical perspective to address some of the main challenges in Machine Learning and, in particular, data classification and Universal Approximation. More precisely, we adopt the perspective of the simultaneous control of systems of NODEs. For instance, in the context of classification, each item to be classified corresponds to a different initial datum for the Cauchy problem of the NODE. And all the solutions corresponding the data under consideration need to be driven to the corresponding target by means of the same control. We present a genuinely nonlinear and constructive method, allowing to estimate the complexity of the control strategies we develop. The very nonlinear nature of the activation functions governing the nonlinear dynamics of NODEs under consideration plays a key role. It allows deforming half of the phase space while the other half remains invariant, a property that classical models in mechanics do not fulfill. This very property allows to build elementary controls inducing specific dynamics and transformations whose concatenation, along with properly chosen hyperplanes, allows achieving our goals in finitely many steps. We also present the counterparts in the context of the control of neural transport equations, establishing a link between optimal transport and deep neural networks.
This is a joint work Domènec Ruiz-Balet.