February 18th 2008 04:59 pm
Dimensionality reduction: Locally Linear Embedding
One of the most cited algorithm in nonlinear manifold learning, with Isomap, is LLE. Contrary to Isomap, LLE tries to retain the local data structure of the sampled manifold. Whereas Isomap preserves absolute distances, LLE preserves local relative distances (it preserves barycenter weights).
This means that LLE is not suitable for every dimensionality reductions. For visualization purposes, it can lead to very different solutions if the manifold is noisy.
When compressing the SCurve manifold that was shown in my other tickets, LLE gives this result :
It is obvious that the local structure is preserved (the color gradients), but the global structure is not : the result should be some kind of rectangle, not a triangle. Other local techniques that preserve local distances (and not relative ones) can lead to better results, so it really is a “relative” problem.
Although the results are not good for visualization purposes, LLE is very fast to compute, and classification on this subspace is efficient, because of the preservation of the local structure.
2 Comments »
2 Responses to “Dimensionality reduction: Locally Linear Embedding”
Leave a Reply
You must be logged in to post a comment.
Sidneybd on 25 Mar 2008 at 8:33 pm #
thanks much, guy
Matt’s blog » Dimensionality reduction: similarities graph and its use on 04 Apr 2008 at 9:12 am #
[...] the widely used method are based on a similarity graph made with the local structure. For instance LLE uses the relative distances, which is related to similarities. Using similarities allows the use of [...]