Speaker
Description
We study the universal approximation property (UAP) of shallow neural networks whose activation function is defined as the flow of a neural ODE. We prove the UAP for the space of such shallow neural networks in the space of continuous functions. In particular, we also prove the UAP with the weight matrices constrained to have unit norm.
Furthermore, in [1] we are able to bound from above the Lipschitz constant of the flow of the neural ODE, that tells us how much a perturbation in input is amplified or shrunk in output. If the upper bound is large, then so it may be the Lipschitz constant, leading to the undesirable situation where certain small perturbations in input cause large changes in output. Therefore, in [2] we compute a perturbation to the weight matrix of the neural ODE such that the flow of the perturbed neural ODE has Lipschitz constant bounded from above as we desire. This leads to a stable flow and so to a stable shallow neural network.
However, the stabilized shallow neural network with unit norm weight matrices does not satisfy the universal approximation property anymore. Nevertheless, we are able to prove approximation bounds that tell us how poorly and how accurately a continuous target function can be approximated by the stabilized shallow neural network.
The results presented during this talk are being collected in [3].
- N. Guglielmi, A. De Marinis, A. Savostianov, and F. Tudisco, Contractivity of neural ODEs: an eigenvalue optimization problem, arXiv preprint arXiv:2402.13092, 2024.
- A. De Marinis, N. Guglielmi, S. Sicilia, and F. Tudisco, Stability of neural ODEs by a control over the expansivity of their flows, work in progress.
- A. De Marinis, D. Murari, E. Celledoni, N. Guglielmi, B. Owren, and F. Tudisco, Approximation properties of neural ODEs, work in progress.
