Applied Machine Learning

Fall 2020 (COMP551-002)


Class: Tuesdays and Thursdays 10:05 am-11:25 pm, Online
Instructor: Siamak Ravanbakhsh
Office hours: Thursdays 11:30 pm-12:30 pm, Online
TA team
Head TA Arnab Kumar Mondal
Amy Zhang (amy.x.zhang) Wednesdays 5pm
Arna Ghosh (arna.ghosh)
Arushi Jain (arushi.jain)
Tianyu Li (
Haque Ishfaq (haque.ishfaq), Mondays 12pm
Howard Huang (howard.huang)
Samin Yeasar Arnob (samin.arnob)

Course Description

This course covers a selected set of topics in machine learning and data mining, with an emphasis on good methods and practices for deployment of real systems. The majority of sections are related to commonly used supervised learning techniques, and to a lesser degree unsupervised methods. This includes fundamentals of algorithms on linear and logistic regression, decision trees, support vector machines, clustering, neural networks, as well as key techniques for feature selection and dimensionality reduction, error estimation and empirical validation.


This course requires programming skills (Python) and basic knowledge of probabilities, calculus and linear algebra provided by courses similar to MATH-323 or ECSE-305. For more information see the course prerequisites and restrictions at McGill’s webpage.

Course Material:

Assignments, announcements, slides, project descriptions and other course materials are posted on myCourses.


There are no required textbook but the topics are covered by the following books:
[Bishop] Pattern Recognition and Machine Learning by Christopher Bishop (2007)
[Goodfellow]    Deep Learning by Ian Goodfellow, Yoshua Bengio, and Aaron Courville (2016)
[Murphy] Machine Learning: A Probabilistic Perspective by Kevin Murphy (2012)

Other Related References
The Elements of Statistical Learning: Data Mining, Inference, and Prediction by Trevor Hastie, Robert Tibshirani and Jerome Friedman (2009)
Information Theory, Inference, and Learning Algorithms, by David MacKay (2003)
Bayesian Reasoning and Machine Learning , by David Barber (2012).
Understanding Machine Learning: From Theory to Algorithms, by Shai Shalev-Shwartz and Shai Ben-David (2014)
Foundations of Machine Learning, by Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar (2018)
Dive into Deep Learning , by Aston Zhang, Zachary Lipton, Mu Li, and Alexander J. Smola (2019)
Mathematics for Machine Learning , by Marc Peter Deisenroth, A Aldo Faisal, and Cheng Soon Ong (2019)
A Course in Machine Learning, by Hal Daumé III (2017)
Hands-on Machine Learning with Scikit-Learn and TensorFlow, by Aurélien Géron (2017)


ML implementation tutorials
- Wednesdays and Fridays at 2 PM (starting 25/09/2020)
Probability and Linear Algebra
- 12 PM - 1:30 PM on Friday 18/09/2020
- linear algebra (sections 1-3), and probability theory
Python / NumPy
- 4:30 PM - 6 PM EST on Thursday 10/09/2020
- Numpy quickstart guide
tutorials on Pytorch website


Topic / Slides Code Reference Pointers
Syllabus and Introduction
Nearest Neighbours notebook ( Colab) chapter 1 [Murphy]
Classification and regression trees notebook ( Colab) 16.1-16.2.6 [Murphy], 14.4 [Bishop] a visual guide
Some ML concepts notebook for model selection (Colab), notebook for curse of dimensionality (Colab) 16.1-16.2.6 [Murphy], 14.4 [Bishop]
Maximum likelihood and Bayesian Reasoning notebook (Colab) 2-2.3 [Bishop], 3-3.5 [Murphy]
Multivariate Gaussian notebook (Colab) 2.3-2.3.4 [Bishop]
Naive Bayes notebook (Colab) 3.5-3.5.4 [Murphy]
Clustering notebook (Colab) 25.5 [Murphy] and 9.1 [Bishop] demos for K-Means and DB-SCAN
Expectation Maximization notebook (Colab) 9.2-9.3.3 [Bishop], 11-11.2.4 [Murphy]
Dimensionality reduction using PCA notebook (Colab) 12.2 [Murphy], 12.1 [Bishop] demo
Linear regression notebook (Colab) 7-7.3.3 [Murphy], 3-3.1.2 [Bishop]
Logistic and softmax regression notebook (Colab) 8.1-8.3.3 [Murphy], 4.1-4.1.3 + 4.3-4.3.3 [Bishop]
Gradient descent methods notebook (Colab) 8.3.2 [Murphy] optional reading
Regularization notebook (Colab) 3.1.4-3.3 [Bishop]
Bias-variance decomposition, bagging & random forests 3.2 [Bishop] demo, optional reading
Perceptrons and linear support vector machines notebook (Colab) 4.11-4.13 + 4.1.7 + 7.1-7.14 excluding kernels [Bishop]
Multilayer Perceptrons 6-6.5 + parts of 7 [Goodfellow] demo
Gradient computation and automatic differentiation notebook (Colab) 6.5 + 8.2 [Goodfellow] blog post, visualization
Convolutional neural networks 9 [Goodfellow] blog post, optional reading


Evaluation will be based on the following components:
Weekly quizzes (20%) online in myCourses
Mini-projects (60%) group assignments
Late midterm exam (20%) November 11th (online)

Late Submission

All due dates are 11:59 pm in Montreal unless stated otherwise. No make-up quizzes will be given. For mini-projects, late work will be automatically subject to a 20% penalty and can be submitted up to 5 days after the deadline. If you experience barriers to learning in this course, submitting the projects, etc., please do not hesitate to discuss them with me. As a point of reference, you can reach the Office for Students with Disabilities at 514-398-6009.

Academic Integrity

“McGill University values academic integrity. Therefore, all students must understand the meaning and consequences of cheating, plagiarism and other academic offences under the Code of Student Conduct and Disciplinary Procedures” (see for more information). (Approved by Senate on 29 January 2003)