Friday, December 16, 2005

Trends in Machine Learning according to Google Scholar

In a previous post I brought up the 1983 Machine Learning workshop which featured "33 papers", and it was the follow up to the 1980 Machine Learning workshop. By contrast, NIPS 2005 had 28 workshops and is just one of several international annual Machine Learning Conferences. You can see how the field grew by looking at the distribution of publication dates for articles containing phrase "machine learning" indexed by Google Scholar (normalized by total Scholar content for each year)
You can see there's a blip at 1983 when the workshop was held.

Yann LeCun quipped at NIPS closing banquet that people who joined the field in the last 5 years probably never heard of the word "Neural Network". Similar search (normalized by results for "machine learning") reveals a recent downward trend.

You can see a major upward trend starting around 1985 (that's when Yann LeCun and several others independently rediscovered backpropagation algorithm), peaking in 1992, and going downwards from then.

An even greater downward trend is seen when searching for "Expert System",

"Genetic algorithms" seem to have taken off in the 90's, and leveled off somewhat in recent years

On other hand, search for "support vector machine" shows no sign of slowing down

(1995 is when Vapnik and Cortez proposed the algorithm)

Also, "Naive Bayes" seems to be growing without bound

If I were to trust this, I would say that Naive Bayes research the hottest machine learning area right now

"HMM"'s seem to have been losing in share since 1981

(or perhaps people are becoming less likely to write things like "hmm, this result was unexpected"?)

What was the catastrophic even of 1981 that forced such a rapid extinction of HMM's (or hmm's) in scientific literature?

Finally a worrying trend is seen in the search for "artificial stupidity" divided by corresponding hits for "artificial intelligence". The 2000 through 2004 graph shows a definite updward direction.


Anonymous said...

Do the results for HMMs include both "HMM" and "Hidden Markov Model"?

Yaroslav Bulatov said...

No, just HMM. I didn't spend much time thinking about the queries, but here are the scripts I used to get the graphs (the two .py files)

Anonymous said...

You'll get slightly different results if you search for conditional frequencies, that is, for example, F("HMM", "machine learning")/F("machine learning").

In this case you'll get an increasing trend for HMM-s for example.

A slightly cleaned-up version of your script is available here:

Anonymous said...

Very interesting diagrams! (: I enjoyed them.

Yaroslav Bulatov said...

Thanks! As you can see from Konstantin's post, you probably shouldn't read too much into them though. Except for the very first graph. It's clear machine learning is growing exponentially. But if machine learning algorithms can ever design machine learning algorithms, then the rate of growth will also be growing exponentially. Therefore the graph will hit a singularity

Anonymous said...

I don't understand how Google allowed you to do this. From their terms of use, they don't allow any automated queries which I find very dissapointing.

I have a "friend" who wrote a script to download every post from the Python newsgroup and after about 10 pages, it blocks you with a captcha.

Anonymous said...

Very entertaining. Especially the last graph.

Anonymous said...

ohoho, Maybe should I stop my research about genetic algorithms for expert systems based HMMs ;-).... Nice job !

Matt Demon said...

The employment on the grounds that in summer somebody sits tight for the occasions, somebody sits tight for go swimming and somebody hold up to appreciate this and mid year after the exhausting and dull days of the winter and snow.

akhilapriya404 said...

This concept is a good way to enhance the knowledge.thanks for sharing. please keep it up machine learning online training

Charles Kee said...

Those who always keep an eye on the japanease magazine,they know the Crayon Shin-chan and the character all about. It is good to find a biography about him too. the website you'll be find some helpful information about the academic writing service.

Charles Kee said...

New writers are always interested in learn something new from people. If there has some tips for writing , they will gather that place to know more on it, like here too. look here for your help and hope you'll love it.

bollings said...

In recent time machine learning is really going as a trend as everyone tends to do this course and they want to know specify things. on this page to check the recent guide and tips about the academic writing.

naina said...

Hi I am here to know more about Google Scholar. I see your are tried lot of info. today I read 4 article site but you are the best on in my study. do you have any video presentation?

visit the website said...

Finally, I got this. Oh, wow! So happy I'm. You are the hero for me. my role model. From now I follow you regularly. What a topic you share for us. just awesome.

draj said...

Excellent machine learning blog,thanks for sharing...
Seo Internship in Bangalore
Smo Internship in Bangalore
Digital Marketing Internship Program in Bangalore

Kanye Co Jamila said...

Great Article
IEEE final year projects on machine learning

JavaScript Training in Chennai

Final Year Project Centers in Chennai

JavaScript Training in Chennai

brayne said...

buy adderall online

buy adderall

adderall for sale

buy vyvanse online

xanax use

buy xanax

buy vyvanse pill

buy adderall xr online

buy oxycodone online

order oxycodone online

buy oxycodone

buy oxycodone online

dihydrocodeine dosage

buy dihydrodeine online

tramadol doses

buy oxycontin

buy oxycontin online

oxycontin for sale

Nisha said...

4S IT Solutions offers a full range of IT services to support your business infrastructure outside of your data center, too. From networking to hardware to applications, our team of certified experts will monitor, manage and maintain your IT environment.
This is our site please visit for further details...

Brukkevin said...

It was not first article by this author as I always found him as a talented author. Reverse Flash Jacket

Brukkevin said...

Fantastic blog i have never ever read this type of amazing information. Cassian Andor Jacket

Jobi said...

I read this article. I think You put a lot of effort to create this article. I appreciate your work.
freestyle love supreme hoodie

pranay said...

Very interesting to read this article.I would like to thank you for the efforts you had made for writing this awesome article. This article inspired me to read more. keep it up.
best machine learning course online!

andrewjackson said...

Our the purpose is to share the reviews about the latest Jackets,Coats and Vests also share the related Movies,Gaming, Casual,Faux Leather and Leather materials available Baywatch Jacket

Pankaj Singh said...

Very descriptive post, I really like this blog very much. For a professional website designing services visit OGEN Infosystem at the best price.
Web Designing Company in Delhi

Unknown said...

Water bodies are the main source of transportation for international freight forwarding. Due to this, sea freight company

in Delhi,
Freight Forwarder in Vietnam
Sea Freight Company in Delhi
Best freight forwarders in india

Unknown said...

visit here
iso certification in delhi

Unknown said...

Thanks for sharing this blog.

Bis Consultant in delhi

Sofia Jone said...

Thank You For Sharing Such type Of Nice Blog joseph seed

mrbobystone said...

Nice Blog !
Here We are Specialist in Manufacturing of Movies, Gaming, Casual, Faux Leather Jackets, Coats And Vests See Dutton Ranch Vest

Jobi Johnson said...

The great website and information shared are also very appreciable. Usmle Step 1 2020

Hollywood Fashion said...

Excellent article. The writing style which you have used in this article is very good and it made the article of better quality. organization 13 coat

jhonny said...

Thanking you for your bloging. The Equalizer 2021 Outfits

SEO Personal said...

Exactly, the very information that I was looking for. I was looking for such kind of article on the Internet and finally ended up here. Thanks for the help!
Wedding hall in Meerut
Top 10 CBSE School in Meerut
SEO Expert in Meerut
Software Development Company In Delhi NCR
Digital Marketing Company in Hapur
Web Development Meerut