icml-tutorial
85
pages
English
Documents
Le téléchargement nécessite un accès à la bibliothèque YouScribe Tout savoir sur nos offres
85
pages
English
Documents
Le téléchargement nécessite un accès à la bibliothèque YouScribe Tout savoir sur nos offres
Support Vector and Kernel MachinesNello CristianiniBIOwulf Technologiesnello@support-vector.nethttp://www.support-vector.net/tutorial.htmlICML 2001A Little Historyz SVMs introduced in COLT-92 by Boser, Guyon, Vapnik. Greatly developed ever since.z Initially popularized in the NIPS community, now an important and active field of all Machine Learning research. z Special issues of Machine Learning Journal, and Journal of Machine Learning Research.z Kernel Machines: large class of learning algorithms, SVMs a particular instance.www.support-vector.netA Little Historyz Annual workshop at NIPSz Centralized website: www.kernel-machines.orgz Textbook (2000): see www.support-vector.netz Now: a large and diverse community: from machine learning, optimization, statistics, neural networks, functional analysis, etc. etcz Successful applications in many fields (bioinformatics, text, handwriting recognition, etc)z Fast expanding field, EVERYBODY WELCOME ! -www.support-vector.netPreliminariesz Task of this class of algorithms: detect and exploit complex patterns in data (eg: by clustering, classifying, ranking, cleaning, etc. the data)z Typical problems: how to represent complex patterns; and how to exclude spurious (unstable) patterns (= overfitting)z The first is a computational problem; the second a statistical problem.www.support-vector.netVery Informal Reasoningz The class of kernel methods implicitly defines the class of possible patterns ...
Voir
Publié par
Langue
English
Support Vector and Kernel Machines
Nello Cristianini BIOwulf Technologies nello@support-vector.net http://mtlotirlah.wpoup.swwotcev-trut/ten.r
ICML 2001
A Little History
z
z
z
z
SVMs introduced in COLT-92 by Boser, Guyon, Vapnik. Greatly developed ever since. Initially popularized in the NIPS community, now an important and active field of all Machine Learning research. Special issues of Machine Learning Journal, and Journal of Machine Learning Research. Kernel Machines: large class of learning algorithms, SVMs a particular instance.
www.support-vector.net
A Little History
z
z
z
z
z
z
Annual workshop at NIPS Centralized website: www.kernel-machines.org Textbook (2000): see www.support-vector.net Now: a large and diverse community: from machine learning, optimization, statistics, neural networks, functional analysis, etc. etc Successful applications in many fields (bioinformatics, text, handwriting recognition, etc) Fast expanding field, EVERYBODY WELCOME !-
www.support-vector.net
Preliminaries
zTask of this class of algorithms: detect and exploit complex patterns in data (eg: by clustering, classifying, ranking, cleaning, etc. the data) zTypical problems: how to represent complex patterns; and how to exclude spurious (unstable) patterns (= overfitting) zThe first is a computational problem; the second a statistical problem.
www.support-vector.net
Very Informal Reasoning
zThe class of kernel methods implicitly defines the class of possible patterns by introducing a notion of similarity between data zExample: similarity between documents zBy length zBy topic zBy language
zChoice of similarityÎChoice of relevant features
www.support-vector.net
More formal reasoning
z
z
z
z
Kernel methods exploit information about the inner products between data items Many standard algorithms can be rewritten so that they only require inner products between data (inputs) Kernel functions = inner products in some feature space (potentially very complex) If kernel given, no need to specify what features of the data are being used
www.support-vector.net
Just in case
zInner product between vectors x,z1åxizi zHyperplane:i x
w,#b10x
x w
x
o
b
o
www.support-vector.net
o
x
o
o
x
x
x
o
Overview of the Tutorial
zIntroduce basic concepts with extended example of Kernel Perceptron zDerive Support Vector Machines zOther kernel based algorithms zProperties and Limitations of Kernels zOn Kernel Alignment zOn Optimizing Kernel Alignment
www.support-vector.net
Parts I and II: overview
zLinear Learning Machines (LLM) zKernel Induced Feature Spaces zGeneralization Theory zOptimization Theory zSupport Vector Machines (SVM)
www.support-vector.net
Modularity
z
z
z
z
IMPORTANT CONCEPT
Any kernel-based learning algorithm composed of two modules: A general purpose learning machine A problem specific kernel function Any K-B algorithm can be fitted with any kernel Kernels themselves can be constructed in a modular way Great for software engineering (and for analysis)
www.support-vector.net
1-Linear Learning Machines
zSimplest case: classification. Decision function is a hyperplane in input space zThe Perceptron Algorithm (Rosenblatt, 57)
zUseful to analyze the Perceptron algorithm, before looking at SVMs and Kernel Methods in general
www.support-vector.net
