A. Alvarez-Lopez, R. Orive-Illera, E. Zuazua (2024)Optimized classification with neural odes via separability Neural Networks, Vol. 180, 106640, arXiv:2312.13807
Abstract. Classification of N points becomes a simultaneous control problem when viewed through the lens of neural ordinary differential equations (neural ODEs), which represent the time-continuous limit of residual networks. For the narrow model, with one neuron per hidden layer, it has been shown that the task can be achieved using O(N) neurons. In this study, we focus on estimating the number of neurons required for efficient cluster-based classification, particularly in the worst-case scenario where points are independently and uniformly distributed in [0,1]d. Our analysis provides a novel method for quantifying the probability of requiring fewer than O(N) neurons, emphasizing the asymptotic behavior as both d and N increase. Additionally, under the sole assumption that the data are in general position, we propose a new constructive algorithm that simultaneously classifies clusters of d points from any initial configuration, effectively reducing the maximal complexity to O(N/d) neurons.
arxiv: 2312.13807