Skip to content
  • Publications
  • Jobs
  • enzuazua
  • Seminars
  • Events Calendar
cmc.deusto.eus
  • Home
  • About us
    • About the Chair
    • Head of the Chair
    • Team
    • Past Members
  • Research
    • Projects
    • ERC CoDeFeL
    • Research Group in Computational Mathematics
    • DyCon Blog
    • DyCon Toolbox
    • Industrial & Social TransferenceContents related to the industrial and social transference aspects of the work in the Chair of Computational Mathematics.
  • Publications
    • Publications (All)
    • Publications by year
      • Publications 2025
      • Publications 2024
      • Publications 2023
      • Publications 2022
      • Publications 2021
      • Publications 2020
      • Publications 2019
      • Publications 2018
      • Publications 2017
      • Publications 2016
    • AcceptedAccepted to be released
    • SubmittedSubmitted publications
  • Activities
    • Events calendar
    • Past Events
    • News
    • Seminars
    • Courses
    • enzuazua
    • Gallery
  • Jobs
  • Contact

Sparse approximation in learning via neural ODEs

Esteve C., Geshkovski B. Sparse approximation in learning via neural ODEs (2021)

Abstract. We consider the continuous-time, neural ordinary differential equation (neural ODE) perspective of deep supervised learning, and study the impact of the final time horizon T in training. We focus on a cost consisting of an integral of the empirical risk over the time interval, and L1–parameter regularization. Under homogeneity assumptions on the dynamics (typical for ReLU activations), we prove that any global minimizer is sparse, in the sense that there exists a positive stopping time T∗ beyond which the optimal parameters vanish. Moreover, under appropriate interpolation assumptions on the neural ODE, we provide quantitative estimates of the stopping time T∗, and of the training error of the trajectories at the stopping time. The latter stipulates a quantitative approximation property of neural ODE flows with sparse parameters. In practical terms, a shorter time-horizon in the training problem can be interpreted as considering a shallower residual neural network (ResNet), and since the optimal parameters are concentrated over a shorter time horizon, such a consideration may lower the computational cost of training without discarding relevant information.

Read Full Paper

Tags:
deep learningNeural ODEsNonlinear systemsoptimal controlSparsitySupervised Learning

Post navigation

Previous Post
Deep generative models and their applications Deep generative models and their applications
Next Post
Decod2021: Heat equations with memory: flow decomposition, constraints and control by E. Zuazua Decod2021: Heat equations with memory: flow decomposition, constraints and control by E. Zuazua

Last Publications

Regional and Partial Observability and Control of Waves

Cluster-based classification with neural ODEs via control

Optimal convergence rates for the finite element approximation of the Sobolev constant

Boundary observation and control for fractional heat and wave equations

Almost periodic turnpike phenomenon for time-dependent systems

  • DeustoCCM seminar: Controllability on some PDEs with dynamic boundary conditions
  • Regional and Partial Observability and Control of Waves
  • Cluster-based classification with neural ODEs via control
  • Optimal convergence rates for the finite element approximation of the Sobolev constant
  • Boundary observation and control for fractional heat and wave equations
Copyright 2016 - 2025 — cmc.deusto.eus. All rights reserved. Chair of Computational Mathematics, Deusto Foundation - University of Deusto
Scroll to Top
  • Aviso Legal
  • Política de Privacidad
  • Política de Cookies
  • Configuración de Cookies
WE USE COOKIES ON THIS SITE TO ENHANCE USER EXPERIENCE. We also use analytics. By navigating any page you are giving your consent for us to set cookies.    more information
Privacidad