Sparse methods for machine learning

pages

English

Documents

Écrit par
Francis Bach

Publié par
pefav

Lire un extrait

Obtenez un accès à la bibliothèque pour le consulter en ligne En savoir plus

Découvre YouScribe et accède à tout notre catalogue !

Je m'inscris

Découvre YouScribe et accède à tout notre catalogue !

Je m'inscris

pages

English

Documents

Lire un extrait

Obtenez un accès à la bibliothèque pour le consulter en ligne En savoir plus

Publié par

pefav

Nombre de lectures

Langue

English

Sparse methods for machine learning Francis Bach Willow project, INRIA - Ecole Normale Superieure CVPR Tutorial - June 2010 Special thanks to R. Jenatton, G. Obozinski

norms

supervised learning

multi-task learning

sparsity- inducing norms

structured sparse

?1 -norm ?w?1

sparse methods

norm

Voir

Publié par

pefav

Langue

English

Supervised learning Multi-task learning

Sparse methods for machine learning

Francis Bach

Willowproject,INRIA-EcoleNormaleSup´erieure

CVPR Tutorial - June 2010 Special thanks to R. Jenatton, G. Obozinski

Sparse methods for machine learning Outline

•Sparse linear estimation with theℓ1-norm

–Lasso –Important theoretical results

•Structured sparse methods on vectors

–Groups of features / Multiple kernel learning

•Sparse methods on matrices

–Multi-task learning –Matrix factorization (low-rank, sparse PCA, dictionary learning)

Supervised learning and regularization

•Data:xi∈ X,yi∈ Y,i= 1     n

•Minimize with respect to functionf:X → Y:

n Xℓ(yi f(xi)) i=1 Error on data

Loss & function space ?

•Two theoretical/algorithmic issues:

1. Loss 2.Function space / norm

+ +

λ2kfk2 Regularization

Norm ?

Regularizations

• overﬁtting avoidMain goal:

•Two main lines of work:

Regularizations

•Main goal: avoid overﬁtting

•Two main lines of work:

EuclideanandHilbertiannorms (i.e.,ℓ2-norms) –Possibility of non linear predictors –Non parametric supervised learning and kernel methods –Well developped theory and algorithms (see, e.g., Wahba, 1990; Scho¨lkopfandSmola,2001;Shawe-TaylorandCristianini,2004) 2.ngciti-yniudSapsrnorms –restricted to linear predictors on vectorsUsually f(x) =w⊤x –Main example:ℓ1-normkwk1=Ppi=1|wi| –Perform model selection as well as regularization –Theory and algorithms “in the making”

Voir