E. Zuazua (2024) Control and Machine Learning Proceedings of the 9th. International Conference on Control and Optimization with Industrial Applications, August 27-29, 2024, Istanbul, Turkey
Abstract. In this talk, we discuss recent results that explore the relationship between control theory and machine learning, specifically applied to supervised learning. First, we study the classification and approximation properties of residual neural networks. Interpreting these problems as simultaneous or ensemble control ones, we build genuinely nonlinear and constructive algorithms, estimating the complexity of controls. Then, we analyze the multilayer perceptron architecture, characterizing the necessary depth and minimum width required to achieve simultaneous controllability. In the domain of large language models, residual networks are combined in an alternating manner with self-attention layers, whose role is to capture the “context”. We view these layers as a dynamical system acting on a collection of points and characterize their asymptotic dynamics and convergence towards special points called leaders. We use our theoretical results to design an interpretable model to solve the task of sentiment analysis of movie reviews. Finally, we investigate federated learning, which enables multiple clients to collaboratively train models without sharing private data, thereby addressing data collection and privacy challenges. Within this framework, we address issues related to training efficiency, incentive mechanisms, and privacy concerns