Machine Learning (course 395)

WARNING: This section contains links to pdf files that may be covered by copyright. You may browse them at your convenience in the same spirit as you may read a journal or a proceeding article in a public library. Retrieving, copying, or distributing these files, however, may violate the copyright protection law. We recommend that the user abides international law in accessing this directory.

Machine Learning (course 395) is envisioned to be an introductory course for several groups of students including MSc Advanced Computing students, fourth-year Information Systems Engineering students, and third/fourth-year Mathematics and Computer Science students.

Course aim:

  1. students should be familiar with some of the foundations of the Machine Learning (ML),
  2. students should have an understanding of the basic ML concepts and techniques:
    • Concept Learning,
    • Decision Trees,
    • Neural Networks,
    • Instance Based Learning,
    • Genetic Algorithms,
    • Hypothesis Evaluation,
  3. students should gain programming skills using Matlab with an emphasis on ML and they should learn how to design, implement and test ML systems,
  4. students should enhance their skills in project planning, working with dead-lines, and reflecting on their own involvement in the teamwork.

Course material:

Course schedule:

The curriculum schedules 14 class meetings of one hour each. The CBC for this course will mainly be devoted to course work (+/- 80 hours per group of 4 students).

The CBC accounts for 32% of the final grade for the Machine Learning Course.

In other words, final grade = 0.667*exam_grade + 0.32*CBC_grade + 0.013*questionnaire.

 

To prepare the exam, attend the CBC and complete the exercises provided during the lectures and those provided at the end of chapters 1, 2, 3, 4, 5, 8, and 9 of Tom Mitchell's book "Machine Learning".

 

Computer Based Coursework (CBC)

CBC contents:

The CBC is designed to build on lectures by teaching students how to apply ML techniques about which they have been lectured to real-world problems.

The CBC will consist of two assignments. All assignments will focus on emotion recognition from data on displayed facial expression using decision trees and neural networks. 

CBC assessment:

Assessment of the CBC work will be conducted based upon the following:

  • the quality of the delivered code as measured by the clarity, effectiveness and efficacy of the delivered code when tested on real (previously unseen) data,
  • the quality of the delivered reports for each of the CBC assignments as measured by the correctness, depth and breadth of the provided discussion on the evaluation of the performance of the developed ML systems for emotion recognition,

CBC data and tools:

Decision Tree Coursework
Data
Pruning Example
See manual above for details.

Neural Networks Coursework Advanced (MSc and BEng/MEng students): Manual  Assignment2

Neural Networks Coursework  (BEng/MEng students): Assignment Description, Data, Code

CBC contact:

All Teaching Helpers can be contacted via one email address. If you wish to contact a specific TH, specify the TH's name in the subject of your email.

Group formation:

Please email us this form through this e-mail address to enrol in CBC. MAX group size is 4.

Further Reading: Lecture Slides

  • Lecture 1-2 : Concept Learning (pdf file)
  • Lecture 3-4 : Decision Trees & CBC Intro (pdf file 1) (pdf file 2)
  • Lecture 5-6 : Evaluating Hypotheses (pdf file)
  • Lecture 7-8 : Neural Networks (1)  (pdf file)
  • Lecture 9-10 : Neural Networks (2) (pdf file)
  • Lecture 11-12: Neural Networks (3) (pdf file)
  • Lecture 13-14 :  Instance Based Learning & Genetic Algorithms (pdf file)

Further Reading: Tutorials

These are three modules from www.coursera.com that we recommend you to have a look at for specific revision or knowledge enhancement (detailed below):
1) Machine Learning by Andrew Ng (ML)
2) Introduction to Computational Finance and Financial Econometrics (CF)
3) Probabilistic Graphical Models (PGM)
Matrix Review:
1) section Linear Algebra Review from ML (all video lectures)
2) section Matrix Algebra from CF  (all video lectures)
Octave Tutorial (similar to MATLAB)
1) section Octave Tutorial from ML  (all video lectures)
2) section ML-class Octave Tutorial from PGM  (all video lectures)
Overfitting
1) section Regularization from ML (only video The problem of overfitting)
Precision, Recall rates and F1-measure
1) section Machine Learning Design System from ML (all video lectures) 
Neural Networks
1) section Neural Networks: Representation from ML  (all video lectures)
2) section Neural Networks: Learning from ML  (all video lectures)
Probability Review
1) section Probability Review from CF (all video lectures except for Portfolio example)
You are strongly advised to look at them

These are three modules from www.coursera.com that we recommend you to have a look at for specific revision or knowledge enhancement (detailed below):

  • Machine Learning by Andrew Ng (ML)
  • Introduction to Computational Finance and Financial Econometrics (CF)
  • Probabilistic Graphical Models (PGM)

 

Matrix Review:

  • section Linear Algebra Review from ML (all video lectures)
  • section Matrix Algebra from CF  (all video lectures)

 

Octave Tutorial (similar to MATLAB):

  • section Octave Tutorial from ML  (all video lectures)
  • section ML-class Octave Tutorial from PGM  (all video lectures)

 

Overfitting:

  • section Regularization from ML (only video The problem of overfitting)

 

Precision, Recall rates and F1-measure

  • section Machine Learning Design System from ML (all video lectures)

 

Neural Networks

  • section Neural Networks: Representation from ML  (all video lectures)
  • section Neural Networks: Learning from ML  (all video lectures)

 

Probability Review

  • section Probability Review from CF (all video lectures except for Portfolio example)

You are strongly advised to look at them.

 

Subspace Learning

Dimensionality Reduction (paper, toolkit, survey), PCA (paper), LDA (paper 1, tutorial 2), KPCA and KDA (paper 1, paper 2), Manifold Learning (Local Linear Embeddings paper and Laplacian Eigenmaps paper).  On Probabilistic Component Analysis, Chapter 12 of "Pattern Recognition and Machine Learning" by Christopher M. Bishop.

 

Neural Networks

Neural Networks (tutorial), Reccurent and Long Short Term Memory Neural Networks (thesis)

 

Deep Learning

Practical tutorials with TensorFlow and PyTorch, Convolutional Neural Networks for Visual Recognition course from Stanford

 

The Menpo Project

ibug talk, Mr. James Booth (Tutorial)

 

Active Appearance Models (AAMs)

ibug talk, Mr. Joan Alabort-i-Medina (Tutorial)

 

Multiple Kernel Learning (MKL)

ibug talk, Mr. Sebastian Kaltwang (Tutorial)

 

Prediction-based Audiovisual Fusion

ibug talk, Dr. Stavros Petridis (Tutorial)

 

Robust Recovery of Low-Rank Subspaces

ibug talk, Dr. Yannis Panagakis (Tutorial)

 

Shared Space Component Analysis

ibug talk, Dr. Mihalis A. Nicolaou (Tutorial)

 

Statistical Machine Learning 

Support Vector Machines (Tutorial 1, Turorial 2)


Bayesian Learning and Graphical Models

Download tutorials on Dynamic Bayesian Networks (paper), Hidden Markov Models (tutorial), Relevance Vector Machines (tutorial), Gaussian Processes (book, videolecture 1, videolecture 2), Conditional Random Fields (tutorial, videolecture)

 

Genetic Algorithms

Genetic Algorithms (tutorial) and Genetic Programming (tutorial).

 

Excellent Books

Pattern Classification by R.O. Duda, P.E. Hart, and D.G. Stork, John Wiley Press, 2005.

Pattern Recognition and Machine Learning by Christopher Bishop, Springer, 2006

Deep Learning by Ian Goodfellow and Yoshua Bengio and Aaron Courville, MIT, 2016