From Basic Machine Learning models to Advanced Kernel Learning

Statistical learning is about the construction and study of systems that can automatically learn from data. With the emergence of massive datasets commonly encountered today, the need for powerful machine learning is of acute importance. Examples of successful applications include effective web search, anti-spam software, computer vision, robotics, practical speech recognition, and a deeper understanding of the human genome.

This course gives an introduction to this exciting field. In the first part, we will introduce basic techniques such as linear regression, logistic regression, multilayer perceptrons, nearest neighbor approaches, both from a theoretical and methodological point of views. In the second part, we will focus on more advanced techniques such as kernel methods, which is a versatile tool to represent data, in combination with (un)supervised learning techniques that are agnostic to the type of data that is learned from. The learning techniques that will be covered include regression, classification, clustering and dimension reduction. We will cover both the theoretical underpinnings of kernels, as well as a series of kernels that are important in practical applications. Finally we will touch upon topics of active research, such as large-scale kernel methods and the use of kernel methods to develop theoretical foundations of deep learning models.

Evaluation

The grading of the class will be done with one final exam.

Reading material

• Francis Bach, Learning Theory from First Principles, 2023.
• Vapnik, The nature of statistical learning theory. Springer.
• Hastie, Tibshirani, Friedman, The elements of statistical learning. (free online).
• Shawe-Taylor, Cristianini. Kernel methods for pattern analysis. 2004.

Optional exercises (ungraded)

Addtional exercises (on kernel learning) can be found in this booklet.

Final exam

The final exam will take place some day in January 2027. It will last 2 hours (the location and precise date is to be confirmed). You are allowed to bring one double-sided single A4 sheet of handwritten notes.

Contact: firstname.lastname@inria.fr