CoDeFeL. Control for Deep and Federated Learning
- ERC Advanced Grant 2022. Control for Deep and Federated Learning (CoDeFeL)
- Principal Investigator (PI): Prof. Enrique Zuazua
- Host Institution: Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU, Germany) and DeustoCCM (Deusto University, Spain)
- Duration: 5 years (2024 – 2029)
Machine Learning (ML) is forging a new era in Applied Mathematics (AM), leading to innovative and powerful methods. But the need for theoretical guarantees generates challenging, fundamental, deep mathematical questions.
This great challenge can be addressed from the perspective of other, more mature areas of AM. CoDeFeL seeks to do so from the rich interface between Control Theory (CT) and ML, contributing to the analytical foundations of ML methods, significantly enlarging, and updating the range of applications of CT.
As our recent results show, classification, regression, and prediction problems in Supervised Learning (SL) and the Universal Approximation Theorem can be successfully recast as the simultaneous or ensemble controllability property of Residual Neural Networks (ResNets). Following this path, we will develop ResNets of minimal complexity and cost, addressing the deep, intricate issue of linking the structure of the data set to be classified with the dynamics of the networks trained.
Taking the turnpike principle as our inspiration, we will build new simplified ResNet architectures. This, however, raises major challenges for the genuinely nonlinear dynamics that ResNets represent.
Adjoint methods will also be developed and applied, to understand the sensitivity of ResNets, and proposing techniques for Adversarial Training and computing Saliency Maps, applicable in Unsupervised Learning.
The project is strongly inspired on the challenges arising in relevant applications in digital medicine and internet recommendation systems, among other areas. Accordingly, we will also develop a body of rich, hybrid, cutting-edge methods for data-aware modelling, combining ResNet surrogate models and those inspired on Mechanics, with the aid of Model Predictive Control strategies. New Federated Learning methodologies with privacy preservation guarantees will also be developed.
The computational counterparts will be brought together in a new CoDeFeL GitHub repository.
Principal Investigator
Working Packages
WP1 – Asymptotics and Turnpike for Deep Neural Networks
Turnpike theory, originally an economic concept, describes how optimal solutions follow a stable, efficient trajectory (the “turnpike”) for most of the process before diverging toward specific final configurations. In machine learning, this principle applies to optimization dynamics, particularly in training deep neural networks, offering insights with both theoretical and practical significance. By leveraging turnpike principles, we can design more efficient training strategies, enabling machine learning models to achieve optimal performance faster and with fewer computational resources.
Publications
• Trélat, E. & Zuazua, E. (2025). Turnpike in Optimal Control and Beyond: A Survey. Modeling and Optimization in Space Engineering, Challenges of the Near Future. arXiv: 2503.20342
• Zuazua, E. (2026). Machine Learning and Control: Foundations, Advances, and Perspectives. arXiv preprint. arXiv: 2510.03303 , FAU CRIS
WP2 – Complexity of ResNet Dynamics
Residual Neural Networks (ResNets) incorporate shortcut connections that facilitate gradient flow, leading to faster convergence and greater robustness to noise compared to standard deep networks. These residual connections make ResNets behave like time-discrete ordinary differential equations, introducing unique dynamical properties that must be thoroughly understood to unlock their full potential. Building on the foundational pillars of simultaneous control theory and universal approximation principles, we aim to design ResNet architectures with improved explainability and performance.
Publications
• Alcalde, A., Fantuzzi, G., & Zuazua, E. (2025). Clustering in Pure-Attention Hardmax Transformers and its Role in Sentiment Analysis. SIAM J. Math. Data Sci., 7(3), 1367–1393. https://doi.org/10.1137/24M167086X arXiv: 2407.01602
• Alcalde, A., Fantuzzi, G., & Zuazua, E. (2025). Exact sequence interpolation with transformers. (Unpublished, Submitted). arXiv: 2502.02270 , FAU CRIS
• Álvarez-López, A., Orive-Illera, R., & Zuazua, E. (2025). Cluster-based classification with neural odes via control. J. Mach. Learn., 4(2), 128–156. https://doi.org/10.4208/jml.241114
• Biccari, U. (2025). Spiking Neural Networks: a theoretical framework for Universal Approximation and training. (Unpublished, Submitted). arXiv: 2509.21920
• Hernández, M. & Zuazua, E. (2025). Constructive Universal Approximation and Finite Sample Memorization by Narrow Deep ReLU Networks. arXiv preprint.
• Li, Z., Liu, K., Liverani, L., & Zuazua, E. (2024). Universal Approximation of Dynamical Systems by Semi-Autonomous Neural ODEs and Applications. (Unpublished, Submitted). arXiv: 2407.17092
• Liu, K. & Zuazua, E. (2025). Representation and regression problems in neural networks: Relaxation, generalization, and numerics. Math. Models Methods Appl. Sci., 35(6), 1471–1521. https://doi.org/10.1142/S0218202525500228 , FAU CRIS
• Morales, R. (2025). SHAP values through general Fourier representations: theory and applications. (Unpublished, Submitted). arXiv: 2511.00185
WP3 – Federated Learning
Federated Learning (FL) is a decentralized machine learning approach that allows multiple devices to collaboratively train a model without exchanging raw data. This method is especially valuable in privacy-sensitive domains such as healthcare and finance, where strict regulations limit data sharing. As a relatively new paradigm, FL still presents a significant gap between practical algorithmic performance and theoretical understanding. By integrating a control theory perspective, we aim to develop novel FL methodologies with strong mathematical rigor and solid statistical foundations.
Publications
• Liu, K., Wang, Z., & Zuazua, E. (2026). A Potential Game Perspective in Federated Learning. (Unpublished, Submitted). , FAU CRIS
• Morales, R. & Biccari, U. (2025). A Multi-Objective Optimization framework for Decentralized Learning with coordination constraints. (Unpublished, Submitted). arXiv: 2507.13983
• Song, Y., Wang, Z., & Zuazua, E. (2025). FedADMM-InSa: An inexact and self-adaptive ADMM for federated learning. Neural Networks, 181, 106–772. https://doi.org/10.1016/j.neunet.2024.106772 , FAU CRIS
WP4 – Modelling through Control and Machine Learning
Control theory focuses on regulating dynamic systems to achieve desired outcomes, while machine learning excels at identifying patterns and making predictions from data. By integrating these two fields, we aim to develop efficient, robust, and adaptive systems that not only learn from data but also dynamically adjust their behavior based on real-time feedback. This synergy ensures optimal performance in dynamic and uncertain environments.
Publications
• Hernández, M. & Zuazua, E. (2025). Random Batch Methods for Discretized PDEs on Graphs. arXiv preprint. arXiv: 2506.11809
• Li, Z., Liu, K., Song, Y., Yue, H., & Zuazua, E. (2025). Deep Neural ODE Operator Networks for PDEs. arXiv preprint.
• Liu, K. & Zuazua, E. (2025). A PDE perspective on generative diffusion models. (Unpublished, Submitted). , FAU CRIS
• HYCO: Hybrid-Cooperative Learning for Data-Driven PDE Modeling. arXiv preprint. arXiv: 2509.14123
• Lyu, K., Biccari, U., & Wang, J. (2025). Robust stabilization of hyperbolic PDE-ODE systems via Neural Operator-approximated gain kernels. (Unpublished, Submitted). arXiv: 2508.03242
CoDeFeL Toolbox
• Decentralized Learning with coordination constraints
Author: Roberto Morales and Umberto Biccari
Language: Python
A source code to implement Decentralized Learning as a Multi-Objective Optimization problem.
• Semi-autonomous Neural ODEs to approximate dynamical systems
Author: Ziqian Li, Kang Liu, Lorenzo Liverani and Enrique Zuazua
Language: Python
A source code to approximate the behavior of dynamical systems using neural networks.
• Fourier SHAP values
Author: Roberto Morales
Language: Python
Python implementation of Fourier-based SHAP values for explaining neural network predictions in a biomedical classification task.
• Sentiment analysis with transformers
Author: Albert Alcalde
Language: Python
An app for sentiment analysis using transformers.
• HYCO: The hybrid-collapse strategy for time-independent PDEs
Author: Matthys Jacobus Steynberg
Language: Python
Hybrid modelling of Poisson and Helmholtz equations combining a physics-based model, a synthetic (neural network) model, and a PINN.
• Federated Learning: protect your data and privacy
Author: Ziqi Wang
Language: Python
A basic PyTorch implementation of the Federated Learning FedAvg algorithm.
• A potential game perspective in Federated Learning
Author: Kang Liu, Ziqi Wang and Enrique Zuazua
Language: Python
Implementation of a potential game strategy for Federated Learning
• Learning with Neural ODEs Toolbox
Author: Borjan Geshkovski
Language: Python
A toolbox for learning with neural ODEs.
• The ADMM-PINNs Algorithmic Framework for Nonsmooth PDE-Constrained Optimization: A Deep Learning Approach
Author: Yongcun Song
Language: Python / MATLAB
A source code to study the combination of the alternating direction method of multipliers (ADMM) with physics-informed neural networks (PINNs) for a general class of nonsmooth partial differential equation (PDE)-constrained optimization problems, where additional regularization can be employed for constraints on the control or design variables.
• PINNS Wave Equation
Author: Daniël Veldman
Language: Python & MATLAB
An implementation of Physics-Informed Neural Networks (PINNs) to solve various forward and inverse problems for the 1 dimensional wave equation.
• Model Predictive Control
Author: Daniël Veldman
Language: MATLAB
Model Predictive Control for course discretizations of the heat and wave equation.
• RBM-MPC
Author: Daniël Veldman
Language: MATLAB
This is the code we used for the numerical experiments with RBM-MPC, a combination of Model Predictive Control (MPC) and Random Batch Methods (RBMs).
Data management plan
The CoDeFeL project (ERC-2022-ADG) has received funding from the European Union’s Horizon ERC Grants programme under grant agreement No. 101096251.
The views and opinions expressed are solely those of the author(s) and do not necessarily reflect those of the European Research Council Executive Agency (ERCEA), the European Union or the granting authority who cannot be held responsible for them