Sparse methods for machine learning

pages

English

Documents

Écrit par
Francis Bach

Publié par
pefav

Lire un extrait

Obtenez un accès à la bibliothèque pour le consulter en ligne En savoir plus

Découvre YouScribe en t'inscrivant gratuitement

Je m'inscris

Découvre YouScribe en t'inscrivant gratuitement

Je m'inscris

pages

English

Ebook

Lire un extrait

Obtenez un accès à la bibliothèque pour le consulter en ligne En savoir plus

Publié par

pefav

Nombre de lectures

Langue

English

Sparse methods for machine learning Francis Bach Willow project, INRIA - Ecole Normale Superieure CVPR Tutorial - June 2010 Special thanks to R. Jenatton, G. Obozinski

norms

supervised learning

multi-task learning

sparsity- inducing norms

structured sparse

?1 -norm ?w?1

sparse methods

norm

Voir

Publié par

pefav

Nombre de lectures

Langue

English

Supervised learning Multi-task learning

Sparse methods for machine learning

Francis Bach

Willowproject,INRIA-EcoleNormaleSup´erieure

CVPR Tutorial - June 2010 Special thanks to R. Jenatton, G. Obozinski

Sparse methods for machine learning Outline

•Sparse linear estimation with theℓ1-norm

–Lasso –Important theoretical results

•Structured sparse methods on vectors

–Groups of features / Multiple kernel learning

•Sparse methods on matrices

–Multi-task learning –Matrix factorization (low-rank, sparse PCA, dictionary learning)

Supervised learning and regularization

•Data:xi∈ X,yi∈ Y,i= 1     n

•Minimize with respect to functionf:X → Y:

n Xℓ(yi f(xi)) i=1 Error on data

Loss & function space ?

•Two theoretical/algorithmic issues:

1. Loss 2.Function space / norm

+ +

λ2kfk2 Regularization

Norm ?

Regularizations

• overﬁtting avoidMain goal:

•Two main lines of work:

Regularizations

•Main goal: avoid overﬁtting

•Two main lines of work:

EuclideanandHilbertiannorms (i.e.,ℓ2-norms) –Possibility of non linear predictors –Non parametric supervised learning and kernel methods –Well developped theory and algorithms (see, e.g., Wahba, 1990; Scho¨lkopfandSmola,2001;Shawe-TaylorandCristianini,2004) 2.ngciti-yniudSapsrnorms –restricted to linear predictors on vectorsUsually f(x) =w⊤x –Main example:ℓ1-normkwk1=Ppi=1|wi| –Perform model selection as well as regularization –Theory and algorithms “in the making”

Voir