Friday, December 16, 2005

Trends in Machine Learning according to Google Scholar

In a previous post I brought up the 1983 Machine Learning workshop which featured "33 papers", and it was the follow up to the 1980 Machine Learning workshop. By contrast, NIPS 2005 had 28 workshops and is just one of several international annual Machine Learning Conferences. You can see how the field grew by looking at the distribution of publication dates for articles containing phrase "machine learning" indexed by Google Scholar (normalized by total Scholar content for each year)
You can see there's a blip at 1983 when the workshop was held.

Yann LeCun quipped at NIPS closing banquet that people who joined the field in the last 5 years probably never heard of the word "Neural Network". Similar search (normalized by results for "machine learning") reveals a recent downward trend.

You can see a major upward trend starting around 1985 (that's when Yann LeCun and several others independently rediscovered backpropagation algorithm), peaking in 1992, and going downwards from then.

An even greater downward trend is seen when searching for "Expert System",

"Genetic algorithms" seem to have taken off in the 90's, and leveled off somewhat in recent years

On other hand, search for "support vector machine" shows no sign of slowing down

(1995 is when Vapnik and Cortez proposed the algorithm)

Also, "Naive Bayes" seems to be growing without bound

If I were to trust this, I would say that Naive Bayes research the hottest machine learning area right now

"HMM"'s seem to have been losing in share since 1981

(or perhaps people are becoming less likely to write things like "hmm, this result was unexpected"?)

What was the catastrophic even of 1981 that forced such a rapid extinction of HMM's (or hmm's) in scientific literature?

Finally a worrying trend is seen in the search for "artificial stupidity" divided by corresponding hits for "artificial intelligence". The 2000 through 2004 graph shows a definite updward direction.


Anonymous said...

Do the results for HMMs include both "HMM" and "Hidden Markov Model"?

Yaroslav Bulatov said...

No, just HMM. I didn't spend much time thinking about the queries, but here are the scripts I used to get the graphs (the two .py files)

Anonymous said...

You'll get slightly different results if you search for conditional frequencies, that is, for example, F("HMM", "machine learning")/F("machine learning").

In this case you'll get an increasing trend for HMM-s for example.

A slightly cleaned-up version of your script is available here:

Anonymous said...

Very interesting diagrams! (: I enjoyed them.

Yaroslav Bulatov said...

Thanks! As you can see from Konstantin's post, you probably shouldn't read too much into them though. Except for the very first graph. It's clear machine learning is growing exponentially. But if machine learning algorithms can ever design machine learning algorithms, then the rate of growth will also be growing exponentially. Therefore the graph will hit a singularity

Anonymous said...

I don't understand how Google allowed you to do this. From their terms of use, they don't allow any automated queries which I find very dissapointing.

I have a "friend" who wrote a script to download every post from the Python newsgroup and after about 10 pages, it blocks you with a captcha.

Anonymous said...

Very entertaining. Especially the last graph.

Anonymous said...

ohoho, Maybe should I stop my research about genetic algorithms for expert systems based HMMs ;-).... Nice job !

Matt Demon said...

The employment on the grounds that in summer somebody sits tight for the occasions, somebody sits tight for go swimming and somebody hold up to appreciate this and mid year after the exhausting and dull days of the winter and snow.

akhilapriya404 said...

This concept is a good way to enhance the knowledge.thanks for sharing. please keep it up machine learning online training

Charles Kee said...

Those who always keep an eye on the japanease magazine,they know the Crayon Shin-chan and the character all about. It is good to find a biography about him too. the website you'll be find some helpful information about the academic writing service.

Charles Kee said...

New writers are always interested in learn something new from people. If there has some tips for writing , they will gather that place to know more on it, like here too. look here for your help and hope you'll love it.

bollings said...

In recent time machine learning is really going as a trend as everyone tends to do this course and they want to know specify things. on this page to check the recent guide and tips about the academic writing.

naina said...

Hi I am here to know more about Google Scholar. I see your are tried lot of info. today I read 4 article site but you are the best on in my study. do you have any video presentation?

visit the website said...

Finally, I got this. Oh, wow! So happy I'm. You are the hero for me. my role model. From now I follow you regularly. What a topic you share for us. just awesome.

draj said...

Excellent machine learning blog,thanks for sharing...
Seo Internship in Bangalore
Smo Internship in Bangalore
Digital Marketing Internship Program in Bangalore

Kanye Co Jamila said...

Great Article
IEEE final year projects on machine learning

JavaScript Training in Chennai

Final Year Project Centers in Chennai

JavaScript Training in Chennai

brayne said...

buy adderall online

buy adderall

adderall for sale

buy vyvanse online

xanax use

buy xanax

buy vyvanse pill

buy adderall xr online

buy oxycodone online

order oxycodone online

buy oxycodone

buy oxycodone online

dihydrocodeine dosage

buy dihydrodeine online

tramadol doses

buy oxycontin

buy oxycontin online

oxycontin for sale