1st Semester 2014/15: Introduction to Statistical Learning Theory

Instructors

Mathias Madsen
If you are interested in this project, please contact the instructor by email.

ECTS

6

Description

At the core of any theory of inference or prediction is the assumption that the world is "simple" or "uniform" in a certain way that makes induction possible. Statistical learning theory is a mathematical theory that makes these concepts precise and thus spells out exactly what we can hope to learn about the future based on the past.

The centerpiece of this theory is the uniform law of large numbers, a theorem that provides the necessary and sufficient conditions under which a theory's ability to fit past data is a reliable indicator of its ability to accurately predict future data. Amazingly, this theorem can be proven without making any assumptions about the underlying mechanism that generates the data.

This course will introduce you to some of the core concepts and methods of statistical learning theory, and you will come to a deep understanding of how to prove the uniform law of large numbers.

Organisation

The course will contain a mixture of lectures, problem-solving sessions, and independent work, and you will be required to write an essay on a topic of your own choice.

The five-day crash course will take place in the week of Monday 12 January 2015. It will consist of a mixture of lectures and problem-solving sessions and will take two hours per day. After class, you will be required to solve one homework exercise and hand it in before midnight.

The writing period in principle begins as soon as the crash course ends, that is, after Friday 16 January 2015. It is divided into two parts, one for writing a first draft of the essay, and one for revising this draft and producing a final version.

The guest lectures will take place in the week of Monday 19 January 2015, that is, during the writing period. More information will be announced as the lecturers are confirmed.

Prerequisites

The course has minimal prerequisites:

• It requires a basic knowledge of combinatorics, particularly of binomial coefficients and the inclusion/exclusion principle. If you lack this background, I advise you to study chapters 1 and 4 of Victor Bryant's Aspects of Combinatorics (1992) or a similar text before the course.
• The course further requires a rudimentary knowledge of probability theory, including the concepts of a probability distribution, conditional probability, and statistical independence. If you have never had a course on probability theory, you may want to study chapters 1 and 5 of William Feller's Introduction to Probability Theory and Its Applications (3rd ed., 1968) or another introductory textbook on probability.
• Lastly, the course also requires familiarity with logarithms and exponential functions, and sound intuitions about growth rates (e.g., the difference between exponential and polynomial growth). If you know high school calculus, this should not be a problem. If you want to brush up on this knowledge, you can skim my fact sheet on logarithms or any introductory text on calculus.

Assessment

You need to participate in the entire five-day crash course in order to pass the course. Each day of the crash course, one small homework exercise is given, and you are required to hand it in (by email) by the end of the day. After the crash course, you will be required to write a 5 to 20 page paper on a topic related to the course. You can write these essays in groups of one or two students. A first draft of this essay is due Sunday 25 January 2015, and a final version, incorporating feedback and revisions, is due Friday 30 January 2015.

The course will be graded as pass/fail.

References

Website: Further information is available at the project website.