EE212 Mathematical Foundations for Machine Learning and Data Science

Summer 2020

Department of Electrical Engineering
Syed Babar Ali School of Science and Engineering
Lahore University of Management Sciences.

Course Overview

Machine Learning and Data Science are being used these days in a variety of applications including, but not limited to, forecasting in economics and finance, predicting anomalies or signal analysis in engineering, identification of speaker in acoustics, detection of cosmic bubbles in astrophysics and diagnosis in medical imaging.

While machine learning and data science have enabled many success stories, and tools are readily available to analyse data or design machine learning systems, the strong mathematical foundations in these areas are of significant importance to understand, review, analyse and evaluate the technical details of the machine learning systems and data science algorithms that are usually abstracted away from the user. This course focuses on the mathematical foundations that are essential to build an intuitive understanding of the concepts related to Machine Learning and Data Science.

Topics covered are

• Linear Algebra: vectors and matrices, vector spaces, system of linear equations, eigen-value decomposition, singular value decomposition, regression, least-squares, regularization

• Calculus: Multivariate calculus and differentials for optimization, gradient descent

• Probability: probability axioms, Bayes rule, random variable, probability distributions

• Statistics: descriptive stats, inferential stats, statistical tests

• Introduction to Neural Networks: single and multi-layer perceptron(s), feedforward and feedback networks

• Application to machine learning and data science: principal component analysis (PCA), time series forecasting, clustering etc

• Hands-on exercises: Implementation of the exercises will be carried out in Python
  

Course Overview Video

Announcements

• (Jul 19) Solutions for assignment 1, quizzes 1 and 2 and lab 01 have been posted.

• (Jul. 07) Course activities calendar added.
  

• (Jun. 19) Course overview video available here.
  

• (May. 13) Welcome to EE212. Course outline has been posted.
  
  

Course Activities Calendar

Administrative Details

• Course Outline (Click to download)
  
  

• Suggested Books:
  

  • Reference 1 (click to download pdf): S.Boyd and L. Vandenberghe. Introduction to Applied Linear Algebra - Vectors, Matrices, and Least Squares. Cambridge University Press, 2019

  • Reference 2 (click to download pdf): M. P. Deisenroth, A. A. Faisal and Cheng Soon Ong. Mathematics for Machine Learning. Cambridge University Press, 2019

  • Reference 3: G. Strang. Introduction to Linear Algebra. 2016

  • Reference 4: J. A. Gubner, Probability and Random Processes for Electrical and Computer Engineers, Cambridge University Press, 2006.

  • Reference 5: S. L. Miller and D. Childers, Probability and Random Processes: With Applications to Signal Processing and Communications.

  • Reference 6: A. Papoulis and S.U. Pillai, Probability, Random Variables, and Stochastic Processes.
    
    

• Office Hours and Contact Information
  

  • Instructor: Zubair Khalid (zubair.khalid@lums.edu.pk), Office hours: Tuesday 5-6 pm
    

  • Lead Teaching Assistant: Muhammad Salaar Arif Khan (muhammad.salaar@lums.edu.pk), Office hours: Friday 7-8 pm
    

  • Teaching Assistant: Hamza Athar (20100003@lums.edu.pk), Office hours: Thursday 6-7 pm
    

  • Teaching Assistant: Rabeeya Hamid (21100105@lums.edu.pk), Office hours: Wednesday 4-5 pm
    

  • Teaching Assistant: Muhammad Huzaifa Khan Suri (21100028@lums.edu.pk), Office hours: Monday 4-5 pm
    

Grading Distribution

• Assignments, 20 %

• Programming Assignments, 10 %

• Quizzes, 10 %

• Mid-Exam, 20 %

• Mid-Viva, 10 %

• Final Exam, 20 %
  

• Final Viva, 10 %
  

Lecture Notes and Exams

• Mid-Exam
  
  

• Final-Exam
  
  

Assignments

Quizzes

Lab Handouts

• Lab Handout 0 (Pre-Lab, must be completed before Lab 01)
  

• Lab Handout 01, Lab 01 Solution
  

• Lab Handout 02, Image for lab
  

• Lab Handout 03
  

• Lab Handout 04
  

• Lab Handout 05 (Data-sets available on LMS)
  
  
  

Course Modules

• Module 01: Vectors – Notation, Applications and Basic Operations
  

• Module 02: Operations on Vectors – Norm
  

• Module 03: Operations on Vectors – Distance, Angle and Standard Deviation
  

• Module 04: Operations on Vectors – Linear Independence, Basis and Orthonormal Vectors
  

• Module 05: Operations on Vectors – Gram-Schmidt Orthogonalization Algorithm
  

• Module 06: Vector Spaces and Subspaces – Linear Algebra
  

• Module 07: Matrices – Notation, Application Examples and Basic Operations
  

• Module 08: Matrix-Vector Product – Interpretation and Application Examples
  

• Pre-Lab Tutorial: Lab 02
  

• Pre-Lab Tutorial: Lab 03