Machine Learning (course 395)

WARNING: This section contains links to pdf files that may be covered by copyright. You may browse them at your convenience in the same spirit as you may read a journal or a proceeding article in a public library. Retrieving, copying, or distributing these files, however, may violate the copyright protection law. We recommend that the user abides international law in accessing this directory.

Machine Learning (course 395) is envisioned to be an introductory course for several groups of students including MSc Advanced Computing students, fourth-year Information Systems Engineering students, and third/fourth-year Mathematics and Computer Science students.

Course aim:

  1. students should be familiar with some of the foundations of the Machine Learning (ML),
  2. students should have an understanding of the basic ML concepts and techniques:
    • Concept Learning,
    • Decision Trees,
    • Artificial Neural Networks,
    • Instance Based Learning,
    • Genetic Algorithms,
    • Hypothesis Evaluation,
  3. students should gain programming skills using Matlab with an emphasis on ML and they should learn how to design, implement and test ML systems,
  4. students should enhance their skills in project planning, working with dead-lines, and reflecting on their own involvement in the teamwork.

Course material:

Course schedule:

The curriculum schedules 14 class meetings of one hour each. The CBC for this course will mainly be devoted to course work (+/- 80 hours per group of 4 students).

The CBC accounts for 33% of the final grade for the Machine Learning Course.

In other words, final grade = 0.66*exam_grade + 0.33*CBC_grade. For example, if the exam_grade = 33/100 and the CBC_grade = 80/100, then final_grade = 48/100.

The content of the Machine Learning course 395 that is taught in the academic year 2014/2015 is the same as that taught in the academic year 2013/2014. Hence, the final exam will be of a similar format.

To prepare the exam, attend the CBC and complete the exercises provided during the lectures and those provided at the end of chapters 1, 2, 3, 4, 5, 8, and 9 of Tom Mitchell's book "Machine Learning".

 

Computer Based Coursework (CBC)

CBC contents:

The CBC is designed to build on lectures by teaching students how to apply ML techniques about which they have been lectured to real-world problems.

The CBC will consist of three assignments. All three assignments will focus on emotion recognition from data on displayed facial expression using decision trees, neural networks, and case-based reasoning. The last assignment will also focus on evaluating (by means of paired t-tests) which of these ML techniques is more suitable for the problem in question in the case of clean data and in the case of noisy data.

CBC assessment:

Assessment of the CBC work will be conducted based upon the following:

  • the quality of the delivered code as measured by the clarity, effectiveness and efficacy of the delivered code when tested on real (previously unseen) data,
  • the quality of the delivered reports for each of the CBC assignments as measured by the correctness, depth and breadth of the provided discussion on the evaluation of the performance of the developed ML systems for emotion recognition,
  • individual involvement and contribution to the group’s results (to be judged based upon a final interview with each of the groups).

CBC data and tools:

You can download all the required datasets and the software tools that you need to use in one zip file here.

CBC contact:

All Teaching Helpers can be contacted via one email address. If you wish to contact a specific TH, specify the TH's name in the subject of your email.

Group formation:

Please email us this form through this e-mail address to enrol in CBC.

Further Reading: Lecture Slides

  • Lecture 1-2 : Concept Learning (pdf file)
  • Lecture 3-4 : Decision Trees & CBC Intro (pdf file)
  • Lecture 5-6 : Evaluating Hypotheses (pdf file)
  • Lecture 7-8 : Artificial Neural Networks (1)  (pdf file)
  • Lecture 9-10 : Artificial Neural Networks (2) (pdf file)
  • Lecture 11-12 :  Instance Based Learning (pdf file)
  • Lecture 13-14 :  Genetic Algorithms (pdf file)

Further Reading: Tutorials

These are three modules from www.coursera.com that we recommend you to have a look at for specific revision or knowledge enhancement (detailed below):
1) Machine Learning by Andrew Ng (ML)
2) Introduction to Computational Finance and Financial Econometrics (CF)
3) Probabilistic Graphical Models (PGM)
Matrix Review:
1) section Linear Algebra Review from ML (all video lectures)
2) section Matrix Algebra from CF  (all video lectures)
Octave Tutorial (similar to MATLAB)
1) section Octave Tutorial from ML  (all video lectures)
2) section ML-class Octave Tutorial from PGM  (all video lectures)
Overfitting
1) section Regularization from ML (only video The problem of overfitting)
Precision, Recall rates and F1-measure
1) section Machine Learning Design System from ML (all video lectures) 
Neural Networks
1) section Neural Networks: Representation from ML  (all video lectures)
2) section Neural Networks: Learning from ML  (all video lectures)
Probability Review
1) section Probability Review from CF (all video lectures except for Portfolio example)
You are strongly advised to look at them

These are three modules from www.coursera.com that we recommend you to have a look at for specific revision or knowledge enhancement (detailed below):

  • Machine Learning by Andrew Ng (ML)
  • Introduction to Computational Finance and Financial Econometrics (CF)
  • Probabilistic Graphical Models (PGM)

 

Matrix Review:

  • section Linear Algebra Review from ML (all video lectures)
  • section Matrix Algebra from CF  (all video lectures)

 

Octave Tutorial (similar to MATLAB):

  • section Octave Tutorial from ML  (all video lectures)
  • section ML-class Octave Tutorial from PGM  (all video lectures)

 

Overfitting:

  • section Regularization from ML (only video The problem of overfitting)

 

Precision, Recall rates and F1-measure

  • section Machine Learning Design System from ML (all video lectures)

 

Neural Networks

  • section Neural Networks: Representation from ML  (all video lectures)
  • section Neural Networks: Learning from ML  (all video lectures)

 

Probability Review

  • section Probability Review from CF (all video lectures except for Portfolio example)

You are strongly advised to look at them.

 

Subspace Learning

Dimensionality Reduction (papertoolkit, survey), PCA (paper), LDA (paper 1, tutorial 2), KPCA and KDA (paper 1, paper 2), Manifold Learning (Local Linear Embeddings paper and Laplacian Eigenmaps paper).  On Probabilistic Component Analysis, Chapter 12 of "Pattern Recognition and Machine Learning" by Christopher M. Bishop.

 

Neural Networks

Neural Networks (tutorial), Reccurent and Long Short Term Memory Neural Networks (thesis)

 

Statistical Machine Learning 

Support Vector Machines (Tutorial 1Turorial 2)


Bayesian Learning and Graphical Models

Download tutorials on Dynamic Bayesian Networks (paper), Hidden Markov Models (tutorial), Relevance Vector Machines (tutorial), Gaussian Processes (bookvideolecture 1videolecture 2), Conditional Random Fields (tutorialvideolecture)

 

Genetic Algorithms

Genetic Algorithms (tutorial) and Genetic Programming (tutorial).

 

Further Reading: Machine Learning Applications

 

Decision Trees

Criminisi, A, J. Shotton and E. Konukoglu, "Decision forests for classification, regression, density estimation, manifold learning and semi-supervised learning." ,Microsoft Research Cambridge, Tech. Rep. MSRTR-2011-114 5.6 (2011): 12.

N. Sebe, M.S. Lew, I. Cohen, Y. Sun, T. Gevers and T.S. Huang, 'Authentic Facial Expression analysis (using various Machine Learning techniques)(pdf file)', in Proc. IEEE Int’l Conf. on Automatic Face and Gesture Recognition (FG ’04), pp. 517-522, Seoul, Korea, May 2004 (Copyright 2004 IEEE Press).

A.C. Tan and D. Gilbert, 'An empirical comparison of supervised machine learning techniques in bioinformatics (pdf file)', in Proc. Asia Pacific Conf. on Bioinformatics (APBC’03).

 

Neural Networks

S. Petridis, M. Pantic., Audiovisual Discrimination Between Speech and Laughter: Why and When Visual Information Might Help, In  IEEE Transactions on Multimedia. 13(2): pp. 216 - 234, April 2011.

D. Chen and P. Burrell, 'Case-based reasoning system and artificial neural networks: A Review (pdf file)', in Neural Computing & Applications, vol. 10, no. 3, pp. 264-276, 2001 (Copyright 2001 Springer).

M.F. Valstar and M. Pantic, 'Biologically vs. logic inspired encoding of facial actions and emotions in video (pdf file)', in Proc. IEEE Int'l Conf. on Multimedia and Expo (ICME '06), Toronto, Canada, July 2006 (Copyright 2006 IEEE Press).

S. Petridis, M. Pantic, J. Cohn.,  'Prediction-Based Classification For Audiovisual Discrimination Between Laughter And Speech'Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition (FG'11). Santa Barbara, CA, USA, pp. 619 - 626, March 2011.

M. A. Nicolaou, H. Gunes and M. Pantic, 'Continuous Prediction of Spontaneous Affect from Multiple Cues and Modalities in Valence-Arousal Space (pdf file)',IEEE Transactions on Affective Computing., pp. pp. 92 - 105, 2011. (Copyright © 2011 IEEE Press).

 

Case-based Reasoning

A. Aamodt and E. Plaza, 'CBR: foundational issues, methodological variations and system approaches (pdf file)', in AI Communications, vol. 7, no. 1, pp. 39-59, 1994 (Copyright 1994 IOS Press).

D.W. Aha, 'The omnipresence of case-based reasoning in science and application (pdf file)', in Knowledge-Based Systems, vol. 11, no. 5-6, pp. 261-273, 1998 (Copyright 1998 Elsevier).

M. Pantic and L.J.M. Rothkrantz, 'Case-based reasoning for user-profiled recognition of emotions from face images (pdf file)', in Proc. IEEE Int'l Conf. on Multimedia and Expo (ICME '04), vol. 1, pp. 391-394, Taipei, Taiwan, June 2004 (Copyright 2004 IEEE Press).

 

Bayesian Learning

A. Dearden and Y. Demiris, 'Learning forward models for robots', in Proc. Int’l Joint Conf. Artificial Intelligence (IJCAI’05), pp. 1440-1445, Edinburgh, UK, 2005 (Copyright 2005 IJCAI Press).

Y. Demiris and B. Khadhouri, 'Hierarchical attentive multiple models for execution and recognition of actions', in Robotics and Autonomous Systems', vol. 54, pp. 361-369, 2006 (Copyright 2006 Elsevier).

 

Dynamic Bayesian Networks and Hidden Markov Models

S. Koelstra, M. Pantic and I. Patras, 'A dynamic texture based approach to recognition of facial actions and their temporal models', in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, no. 11, pp. 1940-1954, November 2010

J. Jiao and M. Pantic, 'Implicit Image Tagging via Facial Information', Proceedings ACM Int’l Workshop on Social Signal Processing (SSPW’10), Firenze, Italy, October 2010

M. Nicolaou, H. Gunes and M. Pantic, 'Audio-visual Classification and Fusion of Spontaneous Affect Data in Likelihood Space (pdf file)', Proc. Int’l Conf. Pattern Recognition (ICPR’10), pp. 3695-3699, Istanbul, Turkey, August 2010.

Y. Tong, J. Chen, and Q. Ji, 'A Unified Probabilistic Framework for Spontaneous Facial Action Modeling and Understanding', IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), Vol. 32, No. 2, pp. 258-273, February 2010

 

Excellent Books

Pattern Classification by R.O. Duda, P.E. Hart, and D.G. Stork, John Wiley Press, 2005.

Pattern Recognition and Machine Learning by Christopher Bishop, Springer, 2006