Lecturers
Slides
| Introduction to neural networks – Lars Kai Hansen | [wpdm_file id=6] |
| Markov Random Fields and Restricted Boltzmann Machines – Christian Igel | [wpdm_file id=7] |
| Unsupervised feature learning – Hugo Larochelle | [wpdm_file id=5] |
| Modern deep architectures – Aaron Courville | [wpdm_file id=10] |
| Dan Claudiu Cireșan – Convolutional neural networks | [wpdm_file id=11] |
| Deep learning in breast cancer screening – Michiel Kallenberg | [wpdm_file id=9l] |
| Deep learning lessons from image, text and bioinformatics applications – Ole Winther | [wpdm_file id=8l] |
Practical sessions
| Restricted Boltzmann Machines in Shark | [UPDATE 15/08] Installation instructions (pdf) | [UPDATE 15/08] Code Shark RBM (zip) | |
| Autoencoders for unsupervised pre-training | Installation instructions (pdf) | Linux/Mac: [wpdm_file id=2] | Windows: [wpdm_file id=4] |
| Convolutional neural networks for digit recognition | Linux/Mac: [wpdm_file id=12] | Windows: [wpdm_file id=13] |
Suggested reading
Introduction to neural networks
- Chapter 11 in The Elements of Statistical Learning, Hastie et al., 2013
- Chapter 4 & 5 in Pattern Recognition and Machine Learning.
- Neural networks: a pattern recognition perspective, Bishop, 1996.
- Lecture 1 & 2 in Hugo Larochelle’s course on neural networks.
Markov Random Fields and Restricted Boltzmann Machines
- Training Restricted Boltzmann Machines: An Introduction, Fischer and Igel, 2014
- Structured Learning and Prediction in Computer Vision, Nowozin and Lampert, 2011
Unsupervised feature learning
- Reducing the dimensionality of data with neural networks, Hinton and Salakahutdinov, 2006
- Sparse autoencoder tutorial, Andrew Ng
- Extracting and Composing Robust Features with Denoising
Autoencoders, Vincent et al., 2008
Modern deep architectures
- Deep Boltzmann Machines, Salakhutdinov and Hinton, 2009
- Improving neural networks by preventing co-adaptation of feature detectors, Hinton et al, 2012
- Maxout networks, Goodfellow et al, 2013
- Why Does Unsupervised Pre-training Help Deep Learning?, Erhan et al., 2010
- Auto-Encoding Variational Bayes, Kingma et al., 2014
Convolutional neural networks
- What is the Best Multi-Stage Architecture for Object Recognition?, Jarrett et al., 2009
- ImageNet Classification with Deep Convolutional Neural Networks, Krizhevsky et al., 2012
- Convolutional Deep Belief Networksfor Scalable Unsupervised Learning of Hierarchical Representations, Lee et al. 2009
- Multi-column Deep Neural Networks for Image Classification, Cireșan et al., 2012
Further reading
- Learning Deep Architectures for AI, Bengio, 2009
- Hugo Larochelle’s class on neural networks.
- Geoffrey Hinton’s Coursera class on neural networks
- deeplearning.net: reading list and tutorial.