Wednesday, June 27, 2007

Machine Learning patents

I found a large number of machine learning related patent applications by doing a few queries on

Here are a couple that caught my eye:

  • Logistic regression (A machine implemented system that facilitates maximizing probabilities)
  • Boosting (A computer-implemented process for using feature selection to obtain a strong classifier from a combination of weak classifiers)
  • Decision tree? (Machine learning by construction of a decision function)
  • Bayesian Conditional Random Fields -- notably, Tom Minka is missing from the list of inventors

The interesting question is what the companies will do with those patents. The best thing they can do is to do nothing with them, which seems to be the case for most patents. The worst thing is they can go after regular users that make implementations of those methods freely available. For instance Xerox forced someone I know to remove a visualization applet from their webpage because the applet used hyperbolic space to visualize graphs, and they have patented this method of visualization. Here are some more worst-case scenarios

Does anyone have any other examples of notable machine learning patents/applications?


Markus said...

There is more. Machine Learning patents are about to be used by Microsoft to fight out some turf war. For example, Microsoft vs. Apples iPod with their Playlist patent...

Yaroslav said...

Big companies suing each other over patents doesn't seem so bad. In that case both companies have enough legal power for a court to give a balanced assessment of the patent validity. It's different when companies go after regular users, as what happened with Xerox. In such cases the company can enforce invalid patents simply by sending a cease-and-desist letter

Igor said...

I always thought one could not patent an algorithm, how are these "methods" anything else than algorithms ?


Yaroslav said...

Given that patents for things like Relevance Vector Machine, Lempel-Ziv compression and RSA public key encryption have been granted, you obviously *can* patent mathematical algorithms. From a brief google search it seems like there's a lot of controversy on how the current patent law should be interpreted.

Yaroslav said...

A bit more searching, and it seems like there's some artificial distinction between "mathematical algorithms" and "computer algorithms" -- latter is patentable, former is not

"Although not binding precedent on the Federal Circuit, the district court in Paine, Webber, Jackson & Curtis, Inc. v. Merrill Lynch, Pierce, Fenner & Smith, 564 F. Supp. 1358, 1367, 218 USPQ 212, 218 (D. Del. 1983) stated:

The CCPA [has] . . . held that a computer algorithm, as opposed to a mathematical algorithm is patentable subject matter."

Igor said...

ahah... so this is the reason. I knew there was a catch as our IP people kept on telling me I could not patent an algorithm, yet all these algorithm or methods kept on being patented.

For the bits you cropped, it is interesting to see that the judgement does not seem binding.

I need to check the difference in definition between this "computer algorithm" and "mathematical algorithm".

Thanks for looking it up.


Anonymous said...
This comment has been removed by a blog administrator.
Sumod said...

Viola Jones in Computer vision..

First and Second