DL2022

[Home Page]

Deep Learning- Fall 2022

Instructor: Sadegh Eskandari

Lectures: Sunday 12:00-14:00 and Tuesday 14:00-16:00

Textbooks: Deep Learning by I. Goodfellow, Y. Bengio, and A. Courville, and Deep Learning with PyTorch by E. Stevens, L. Antiga, and T. Viehmann

Description: This course aims to teach the fundamentals of deep learning. We will take an in-depth look at artifical neural networks (ANNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), and deep generative models. We will also provide hands-on lectures using Pytorch.

Detailed syllabus

Week 1

• A high Level Introduction to Machine Learning and Deep Learning (Slides)
• Reading assignment for next week: Chapters 1-5 of the deep learning book

Week 2

• An Introduction to MLP, Activation Functions and Backpropagation (Slides)
• Colab notebook on building, training and evaluating a basic MLP for the Iris dataset using Pytorch (Codes)
• Reading assignment for next week: Chapters 6-8 of the deep learning book

Week 3

• A Review to General Optimization Algorithms, Gradient Decsent, Hessian Matrices, Convex Optimization and Deep Learning Optimzation Methods including Stochastic Gradient Desccent, Momentum, AdaGrad, RMSProp, and Adam (Slides)
• Reading assignment for next week: Chapter 9 of the deep learning book

Week 4

• An Introduction to Convolutional Neural Nets, Pooling Layers, Object Detection, and State-of-The-Art CNN Networks (Slides)
• Reading assignment for the next week:
  1. He, K., Zhang, X., Ren, S. and Sun, J., 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778).
  2. Szegedy, C., Ioffe, S., Vanhoucke, V. and Alemi, A.A., 2017, February. Inception-v4, inception-resnet and the impact of residual connections on learning. In Thirty-first AAAI conference on artificial intelligence.

Week 5

• An Introduction to Recurrent Neural Nets, Unfolding, and Long-Short Term Memory (LSTM) Networks (Slides)
• Reading assignment for the next week:
  1. Hochreiter, S. and Schmidhuber, J., 1997. Long short-term memory. Neural computation, 9(8), pp.1735-1780.