Jon Asier Bárcena-Petisco. Optimal control for neural ODE in a long time horizon and applications to the classification (2021)
Abstract. We study the optimal control in a long time horizon of neural ordinary differential equations which are affine or whose activation function is homogeneous. When considering the classical regularized empirical risk minimization problem we show that, in long time and under suitable assumptions, the final state of the optimal trajectories has zero training error. We assume that the data can be interpolated and that the error can be taken to zero with a cost proportional to the error. These hypotheses are fulfilled in the classification and simultaneous controllability problems for some relevant activation and loss functions. Our proofs are mainly constructive combined with reductio ad absurdum: We find that in long time horizon if the final error is not zero, we can construct a less expensive control which takes the error to zero. Moreover, we prove that the norm of the optimal control is constant. Finally, we show the sharpness of our hypotheses by giving an example for which the error of the optimal state, even if it decays to 0, is strictly positive for any time.