Applied Machine Learning

Fall 2021 (COMP551-001)

Administrative

Class: Tuesdays and Thursdays 10:05 am-11:25 pm, Online
Instructor: Siamak Ravanbakhsh
Office hours: Thursdays 11:30 am-12:30 pm, Online
TA Office hours: Tuesdays 11:30 am-12:30 pm, Thursdays 3pm, Online / in-person
Communication plan:
MyCourses: course material, assignments, quizzes
Zoom: lectures and office hours
Slack: discussions and all other communications (as a substitute for email)
TA team (id):
Head TA: Devin Kreuzer (devin.kreuzer)
Safa Alver (safa.alver)
Nishanth Anand (nishanth.anand)
Lucas Caccia (lucas.page-caccia)
Raymond Chua (raymond.chua)
Wesley Chung (wesley.chung)
Haque Ishfaq (haque.ishfaq)
Harley Wiltzer (harley.wiltzer)


Course Description

This course covers a selected set of topics in machine learning and data mining, with an emphasis on good methods and practices for deployment of real systems. The majority of sections are related to commonly used supervised learning techniques, and to a lesser degree unsupervised methods. This includes fundamentals of algorithms on linear and logistic regression, decision trees, support vector machines, clustering, neural networks, as well as key techniques for feature selection and dimensionality reduction, error estimation and empirical validation.


Prerequisites:

This course requires programming skills (Python and NumPy) and knowledge of probability (e.g., MATH 323 or ECSE 305), calculus (e.g., MATH 222), linear algebra (e.g., MATH 223), and algorithms (e.g., COMP 251). For more information on official requirements see the course prerequisites and restrictions here.

Course Material:

Assignments, slides, project descriptions and other course materials are posted on myCourses.

Textbooks:

There are no required textbook but the topics are covered by the following books:
[Bishop] Pattern Recognition and Machine Learning by Christopher Bishop (2007)
[Goodfellow]    Deep Learning by Ian Goodfellow, Yoshua Bengio, and Aaron Courville (2016)
[Murphy] Machine Learning: A Probabilistic Perspective by Kevin Murphy (2012)

Other Related References
The Elements of Statistical Learning: Data Mining, Inference, and Prediction by Trevor Hastie, Robert Tibshirani and Jerome Friedman (2009)
Information Theory, Inference, and Learning Algorithms, by David MacKay (2003)
Bayesian Reasoning and Machine Learning , by David Barber (2012).
Understanding Machine Learning: From Theory to Algorithms, by Shai Shalev-Shwartz and Shai Ben-David (2014)
Foundations of Machine Learning, by Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar (2018)
Dive into Deep Learning , by Aston Zhang, Zachary Lipton, Mu Li, and Alexander J. Smola (2019)
Mathematics for Machine Learning , by Marc Peter Deisenroth, A Aldo Faisal, and Cheng Soon Ong (2019)
A Course in Machine Learning, by Hal Daumé III (2017)
Hands-on Machine Learning with Scikit-Learn and TensorFlow, by Aurélien Géron (2017)

Outline (subject to change)

Syllabus and Introduction
Nearest Neighbours
Decision Trees
Some ML Concepts
Linear Regression
Logistic Regression
Maximum Likelihood
Gradient Descent
Regularization
Linear Support Vector Machine
Multilayer Perceptron
Automatic Differentiation
Convolutional Neural Networks
Graph Neural Networks (?)
Clustering
Dimensionality Reduction
Intro to Bayesian Inference
Multivariate Gaussian
Naive Bayes
Expectation Maximization
Bagging and Random Forests
Boosting (?)
Frontiers

Evaluation

Evaluation will be based on the following components:
Weekly quizzes (15%) online in myCourses
Mini-projects (60%) group assignments
Late midterm exam (25%) Nov 18, 6-9pm, STBIO S1/4, ADAMS AUD (in person)

Late Submission

All due dates are 11:59 pm in Montreal unless stated otherwise. No make-up quizzes will be given. For mini-projects, late work will be automatically subject to a 20% penalty and can be submitted up to 5 days after the deadline. If you experience barriers to learning in this course, submitting the projects, etc., please do not hesitate to discuss them with me. As a point of reference, you can reach the Office for Students with Disabilities at 514-398-6009.

Academic Integrity

“McGill University values academic integrity. Therefore, all students must understand the meaning and consequences of cheating, plagiarism and other academic offences under the Code of Student Conduct and Disciplinary Procedures” (see www.mcgill.ca/students/srr/honest/ for more information). (Approved by Senate on 29 January 2003)

Language of Submission

“In accord with McGill University’s Charter of Student Rights, students in this course have the right to submit in English or in French any written work that is to be graded. This does not apply to courses in which acquiring proficiency in a language is one of the objectives.” (Approved by Senate on 21 January 2009)