The goal of this course is twofold: • To present the principles of modern deep neural networks, as well as the technical ways to implement them for solving classification and regression problems. • To provide a detailed overview of the mathematical foundations of modern learning techniques based on deep neural networks. Ce cours est enseigné par L. Garivier (ENSL) à l'ENS Lyon..
Starting with the universal approximation property of neural networks, we will then see why depth improves the capacity of networks to provide accurate function approximations for a given computational budget. Tools to address the optimization problems appearing when training networks on large collections will then be covered, and their convergence properties will be reviewed. Finally, statistical results on the generalization guarantees of deep neural networks will be presented, both in the classical underfitting scenario and in the overfitting scenario leading to the so-called “double descent” phenomenon.