Matthieu Brucher's blog

It’s been a while since I last blogged about manifold learning. I don’t think I’ll add much in terms of algorithms to the scikit, but now that a clear API is being defined (http://sourceforge.net/apps/trac/scikit-learn/wiki/ApiDiscussion), it’s time for the manifold module to comply to it. Also, documentation will be enhanced and some dependencies will be removed.

I’ve started a branch available on github.com, and I will some examples in the scikit as well. I may explain them here, but I won’t rewrite what is already published. A future post will explain the changes, and I hope that interested people will understand the modifications and apply them to my former posts. It’s just that I don’t have much time to change everything…

(No Ratings Yet)

 Loading ...

No Comments yet »

In my last post about optimization, I’ve derived my function analytically. Sometimes, it’s not as easy. Sometimes also, a simple gradient optimization is not enough.

scikits.optimization has a special class for handling numerical differentiation, and several tools for conjugate gradients.
Continue Reading »

(No Ratings Yet)

 Loading ...

No Comments yet »

Last time, I’ve made a simple example of a gradient-free optimization. Now, I’d like to use the gradient of my function (analytical gradient I’ve computed) to be able to get the global minimum in less iterations.
Continue Reading »

(No Ratings Yet)

 Loading ...

No Comments yet »

Some weeks ago, the first release of the optimization scikit was done. I’d like to expose here the internal structure and the way the implementation was thought.
Continue Reading »

(No Ratings Yet)

 Loading ...

No Comments yet »

I’m pleased to announce the first release of one of my projects. This scikits is based on a generic framework that can support unconstrained cost function minimization. It is based on a separation principle and is also completely object oriented.

Several optimizers are available:

• Nelder-Mead or simplex minimization
• Unconstrained gradient-based minimization

The usual criterias can be used:

• Iteration limit
• Parameter change (relative and absolute)
• Cost function changer (relative and absolute)
• Composite criterion generation (AND/OR)

Different direction searches are available:

• Gradient
• Several conjugate-gradient (Fletcher-Reeves, …)
• Decorators for selecting part of the gradient
• Marquardt step

Finally several line searches (1D minimization) were coded:

• Fibonacci and gold number methods (exact line searches)
• Wolfe-Powell soft and strong rules
• Goldstein line search
• Cubic interpolation

Additional helper classes can be used:

• Finite difference differentation (central and forward)
• Quadratic cost (for least square estimation)
• Levenberg-Marquardt approximation for least square estimation

Although it is the 0.1 version, the code is quite stable and is used in the learn scikit.

The package can be easy-installed or can be found on PyPI.

Several tutorials are available or will be available on the future at the following locations:

• http://matt.eifelle.com/tag/optimization/
• http://projects.scipy.org/scikits/wiki/Optimization/tutorial

(2 votes, average: 4.50 out of 5)

 Loading ...

2 Comments »

Some months ago, I’ve finished my manifold learning posts serie. As support for the manifold learning toolkit, I’ve also developed an optimization framework, which I’ll be blogging about, starting now.
Continue Reading »

(1 votes, average: 3.00 out of 5)

 Loading ...

No Comments yet »

My manifold learning code was for some time a Technology Preview in the scikit learn. Now I can say that it is available (BSD license) and there should not be any obvious bug left..

I’ve written a small tutorial. It is not an usual tutorial (there is a user tutorial and then what developers should know to enhance it), and some results of the techniques are exposed in my blog. It provides the basic commands to start using the scikit yourself (reducing some data, projecting new points, …) as well as the expoed interface to enhance the scikit.

If you have any question, feel free to ask me, I will add the answers to the tutorial page so that everyone can benefit from it.

Be free to contribute new techniques and additional tools as well, I cannot write them all ! For instance, the scikit lacks some robust neighbors selection to avoid short-cuts in the manifold…

Tutorial and the learn scikit mainpage.

(No Ratings Yet)

 Loading ...

3 Comments »

I got the word today that my paper was accepted, so I can now focus on delivering the code.

I’m in the process of refactoring it so that it depends less on some of our libraries here. In two weeks, there is a nipy sprint in Paris I will attend, and machine learning is one of the topic we will discuss, so this may indicate where and how I’ll contribute the code I will keep going on showing some results next week.

(No Ratings Yet)

 Loading ...

2 Comments »