Deep Learning - Theory and Practice 

offered by IISc Bangalore

Introduction to Deep Learning Course. Examples. Roadmap of the course. slides

Matrix Derivatives. Maximum Likelihood estimation and Gaussian Example. Linear Models for Classification

Logistic regression two class motivation. Posterior probability, sigmoid function, properties

Logistic regression for K classes, softmax function. Non-convex optimization (local and global minima), Gradient Descent - motivation and algorithm.

Code for Logistic Regression

Multi-layer perceptrons, type of hidden layer and output layer activations - sigmoid, tanh, relu, softmax functions. Error functions in MLPs. Backpropagation learning in MLP. MLP for logistic regression in Keras

Backpropagation in multi-layer deep neural networks. Universal approximation properties of single hidden layer networks. Need for depth. The trade-off between depth and width of networks. Representation learning in DNNs. Hierachical data abstractions. Example in Images

Convolutional neural networks. Kernels and convolutional operations. Maxpooling and subsampling. Backpropagation in CNN layers

Recurrent neural networks, back propagation in recurrent neural networks. Different recurrent architectures - teacher forcing networks, encoder/decoder networks, bidirectional networks

Vanishing gradient problem in RNNs. Long short term memory networks. Unsupervised representation learning - Restricted Boltzmann machines, Autoencoders. Discussion of mid-term exam