EE 514 (CS 535) Machine Learning

Spring 2021

Syed Babar Ali School of Science and Engineering
Lahore University of Management Sciences

Course Overview

Machine learning (ML) studies the design and development of algorithms that learn from the data and improve their performance through experience. ML refers to a set of methods and that help computers to learn, optimize and adapt on their own. ML has been employed to devise algorithms for diverse applications including object detection or identification in computer vision, sentiment analysis of speaker or writer, detection of disease and planning of therapy in healthcare, product recommendation in e-commerce, learning strategies for playing games, recommending movies to customers, speech recognition systems, fraudulent transaction detection or loan application approval in banking sector, to name a few.

This course provides a thorough introduction to the theoretical foundations and practical applications of ML. We will learn fundamental algorithms in supervised learning and unsupervised learning. We will not only learn how to use ML methods and algorithms but will also try to explain the underlying theory building on mathematical foundations. While reviewing the several problems and algorithms to carry out classification, regression, clustering, dimensionality reduction, we will focus on the core fundamentals which unify all the algorithms. The theory discussed in class will be tested in assignments, quizzes and exams.

Announcements

• (Jan 09) Welcome to EE 514 (CS 535). Course outline has been posted.
  
  

Administrative Details

• Course Outline (Click to download)
  
  

• Suggested Books:
  

  • (CB) Pattern Recognition and Machine Learning, Christopher M. Bishop

  • (KM) Machine Learning: a Probabilistic Perspective, Kevin Murphy

  • (TM) Machine Learning, Tom Mitchell

  • (HTF) The Elements of Statistical Learning: Data mining, Inference, and Prediction, by Hastie, Tibshirani, Friedman

  • (DM) Information Theory, Inference, and Learning Algorithms, David Mackay

• Office Hours and Contact Information
  

  • Instructor: Zubair Khalid (zubair.khalid@lums.edu.pk), Office hours: Wednesday 3:30-5 pm
    

  • Teaching Assistant: Rabeeya Hamid (21100105@lums.edu.pk), Office hours: Wednesday, 11 am-12 pm
    

  • Teaching Assistant: Muhammad Haseeb Chaudhry (21100084@lums.edu.pk), Office hours: Tuesday, 4-5 pm
    

  • Teaching Assistant: Taimoor Arif (21100121@lums.edu.pk), Office hours: Friday, 2-3 pm
    

  • Teaching Assistant: Alishba Rauf (20030026@lums.edu.pk), Office hours: Tuesday, 2-3 pm
    

  • Teaching Assistant: Omer Iqbal (21100220@lums.edu.pk), Office hours: Monday, 1-2 pm
    

  • Teaching Assistant: Mariyam Tanveer (21100315@lums.edu.pk), Office hours: Monday, 3-4 pm
    

  • Teaching Assistant: Qasim Rafi (21100043@lums.edu.pk), Office hours: Thursday, 1-2 pm
    

  • Teaching Assistant: Huzaifa Khan Suri (21100028@lums.edu.pk), Office hours: Thursday, 3-4 pm
    

Grading Distribution

• Programming Assignments and Homeworks, 30 %

• Project, 10 %

• Quizzes (1-2 per week), 20 %

• Mid-Exam and/or Mid-Viva, 15 %

• Final Exam, 25 %
  

Project Description and Exams

• Project Description (Click here to download)
  
  

• Mid Exam (Click here to download)

• Final Exam Part 2 (Click here to download)
  
  

Programming Assignments

• Assignment 01 (Click here to download)

  • Due: February 02, Tuesday, 23:55
    

• Assignment 02 (Click here to download)

  • Due: February 18, Thursday, 23:55

• Assignment 03 (Click here to download)

  • Due: March 08, Monday, 23:55

• Assignment 04 (Click here to download)

  • Due: April 13, Tuesday, 23:55

• Assignment 05 (Click here to download)

  • Due: April 25, Sunday, 23:55
    

Homeworks

Schedule

• Week 01

  • Course Introduction

  • Machine Learning Overview (Notes 01)

  • Supervised Learning: Formulation, Setup, Train-test split, Generalization (Notes 02)

• Week 02

  • k-Nearest Neighbor (kNN) Algorithm, Algorithm Formulation

  • Distance Metrics, Choice of k, Algorithm Convergence, Storage, Time Complexity Analysis, Fast kNN (Notes 03)

• Week 03

  • The Curse of Dimensionality and Connection with kNN

  • Dimensionality Reduction: Feature Selection and Extraction, Principal Component Analysis (Notes 03b)

• Week 04

  • Classifer Performance Evaluation: Confusion Matrix Sensitivity, Specificity, Precision Trade-offs, ROC, AUC, F1-Score and Matthew’s Correlation Coefficient (Notes 04)

• Week 05

  • Multi-class Classification, Evaluation, Micro, Macro Averaging

  • Regression: Linear Regression, Polynomial Regression, Overfitting (Notes 05)

• Week 06

  • Gradient Descent Algorithm

  • Regularization (see Week 05 Notes)

• Week 07

  • Probability Review (Notes 06)

  • Bayesian Learning Framework, MAP and ML Hypothesis

  • Linear Regression as ML estimation

  • Naive Bayes Classifier (Notes 07)

• Week 08

  • Naïve Bayes Classifier for Text Classification (Week 08-01)

  • Bayesian Networks Introduction (see Week 05 Notes)

  • Mid-exam

• Week 09

  • Logistic Regression: Mathematical Model, Decision Boundaries, Loss/Cost Function, Gradient Descent

  • Multi-class Logistic Regression (Notes 09)

• Week 10

  • Perceptron and Perceptron Classifier, Perceptron Learning Algorithm and its Geometric Intuition

  • Perceptron Learning Algorithm Convergence (Notes 10)

  • SVM Overview

• Week 11

  • Hard SVM, Soft SVM, Kernel Trick (Notes 11)

• Week 12

  • Neural Networks Introduction, Model, Forward Pass (Notes 12)

• Week 13

  • Neural Networks: Back Propagation (See Notes 12)

• Week 14

  • Unsupervised Learning, Clustering Overview

  • K-means Clustering

  • Agglomerative Clustering (Notes 13)