Deep Learning Algorithms and Regularization with Dropout

Postgraduate Thesis uoadl:2698969 955 Read counter

Unit:
Κατεύθυνση / ειδίκευση Επεξεργασία-Μάθηση Σήματος και Πληροφορίας (ΕΜΠ)
Πληροφορική
Deposit date:
2018-03-13
Year:
2018
Author:
Papaioannou Charilaos
Supervisors info:
Σέργιος Θεοδωρίδης, Καθηγητής, Τμήμα Πληροφορικής & Τηλεπικοινωνιών, Εθνικό και Καποδιστριακό Πανεπιστήμιο Αθηνών
Original Title:
Αλγόριθμοι Βαθιάς Μηχανικής Μάθησης και Εξομάλυνση με χρήση της μεθόδου Dropout
Languages:
Greek
Translated title:
Deep Learning Algorithms and Regularization with Dropout
Summary:
The goal of this thesis is the study of a variety of deep learning algorithms and the evaluation of their regularization with Dropout technique. The methodology followed, begins with the presentation of the key theoretical points for each algorithm, followed by a software program for each one of them and concludes with the presentation of the experimental results for the aspects we are interested in.
The whole thesis lies in the scientific field of Artificial Neural Networks. This is the reason we come across an analytical presentation of the main theory of Neural Networks in the first chapters, before we move forward to the modern techniques.
In the third chapter, the regularization theory is presented along with a detailed presentation of the Dropout method. In the next chapters, until the thesis' conclusion, the methodology stated above is followed. After the theoretical analysis for each algorithm, its development is implemented with the usage of a deep learning library, TensorFlow, and in the end the experimental results get investigated.
We see various different architectures of fully-connected Feed Forward Neural Networks for the handwritten digits recognition problem as well as a Convolutional Neural Network for the same task. We, also, study a Recurrent Neural Network for language modeling.
For all algorithms used, the practical significance of their regularization with Dropout has been investigated, and we ascertained experimentally that it boosted their performance and it improved the ability for each model to generalize.
Main subject category:
Science
Other subject categories:
Technology - Computer science
Keywords:
deep-learning, neural-networks, regularization, dropout, tensorflow
Index:
Yes
Number of index pages:
6
Contains images:
Yes
Number of references:
38
Number of pages:
90
thesis_papaioannou.pdf (3 MB) Open in new window