Institutional Repository
Technical University of Crete
Technical University of Crete
EN
|
EL
| URI: | http://purl.tuc.gr/dl/dias/71452342-B134-4FF8-B2B5-06849C9E9FB4 | ||
| Year | 2023 | ||
| Type of Item | Diploma Work | ||
| License |
|
||
| Bibliographic Citation | Christos Polyzos, "Overparametrized deep neural networks: Convergence and generalization properties", Diploma Work, School of Electrical and Computer Engineering, Technical University of Crete, Chania, Greece, 2023 https://doi.org/10.26233/heallink.tuc.97293 | ||
| Appears in Collections |
In this thesis, we consider deep neural networks for Machine Learning. We depict neural networks as weighted directed graphs and we represent them as parametric functions that receive an input and compute an output, or prediction, given some fixed parameters, the weights and the biases. The quintessence of a neural network is the feed-forward model, in which the underlying graph does not contain cycles (acyclic graph) and the parametric function is defined in a compositional, or hierarchical, way.Throughout our presentation, we focus on a supervised learning setting, where our neural network model, or learner, has access to a training set that contains examples of how pairs of input-output data are related. In other words, supervised learning amounts to learning from examples. Given a training set, depending whether the outputs have real or categorical values, we consider regression and logistic regression. For each setting, we provide the basic statistical framework and construct a loss function known as the empirical risk. We train our neural network by minimizing the empirical risk w.r.t. its parameters by using gradient-based optimization methods. The gradient of the loss function is computed via the back-propagation algorithm.We showcase the convergence and generalization properties of different algorithms (deep neural network models and optimization methods) using real-world data.
Copyright © DIAS 2013
