Inductio Ex Machina

MetaOptimize Q+A

2010-07-03T00:00:00+10:00

This is just a quick post to point any readers who have not heard of it already to the newly deployed MetaOptimize Q+A—a question and answer site for machine learning, NLP, data mining, AI and related topics that describes itself as:

A community of data geeks interested in machine learning, natural language processing, artificial intelligence, text analysis, information retrieval, search, data mining, statistical modeling, and data visualization, as well as adjacent topics.

I’ve had good experiences with related sites such as StackOverflow and particularly MathOverflow. So far I’ve only had a chance to sign up and quickly look around MetaOptimize but from what I’ve seen the users seem sharp, the questions on-topic, and the answers informative.

This is a great addition to the burgeoning online machine learning community and Joseph Turian (@turian) has done a great job at setting the tone of the site as well as encouraging a number of solid researchers and practitioners to join before publicising it too widely.

I’m really looking forward to participating more once I’m back from holidays.

ICML 2010 Discussion Site

2010-06-22T00:00:00+10:00

ICML 2010 is already underway so I thought I would quickly mention the new discussion site I built for it. I’d like for the new version to improve on the earlier ones in terms of participation so, if you haven’t done so already, head over to ML Discuss and start discussing!

This is the third year I have set up such a discussion site for ICML. The previous two years were built upon a wiki engine with a plugin that allows comments to be left on wiki pages. This worked fairly well but had one major drawback: it was not easy to be notified when someone commented on your paper or added to a discussion you had participated in. John and I talked about this after the last ICML and agreed that better notifications should be a priority for future discussion sites like this.

To address this, I built the new site from scratch and “outsourced” the comment system to Disqus. The Disqus service allows anyone to put a comment thread on any webpage by adding a small piece of Javascript. Once this is done, all the comment moderation, user management, notifications, and RSS feeds are all handled through the Disqus interface. While, in a sense, this sort of stuff is “trivial” it requires a lot of care to set up the infrastructure correctly and keep it running so I was glad not to have to do that myself.

The rest of the site – parsing, managing and serving the ICML paper and author information – is a few simple python scripts running on top of Google AppEngine. Along with Bosco, I’ve used this combination of AppEngine and Disqus before for a similar service called annotatr for commenting on articles in CiteULike. The development environment for AppEngine is great, allowing you to test everything locally, and deploy with literally the push of a button. I would recommend this approach to anyone wishing to set up a discussion site.

As an aside, while building ML Dicsuss I realised this quick and dirty style of development epitomises what Noble and Biddle call scrap-heap programming in Section 13 of their Notes on Postmodern Programming¹: exploiting the “large number of existing programs that the postmodern programmer can scavenge through and reuse”. Now that more and more programs exist as services on the web I think we will see more of this style of development in the future.

Happy discussing!

Just for fun, see if you can guess what part of this paper that I quoted in my wedding speech.
↩

Prediction Services

2010-05-21T00:00:00+10:00

I have been thinking about learning and prediction as services for some time now. Like all good ideas, they tend to be thought of independently by several people when their time is ripe. Therefore I was not completely surprised when I heard the news yesterday that Google has released a new RESTful prediction API.

As a couple of other bloggers (John, Panos) have already noted, this is very exciting as it has the potential of making statistical inference a commodity and putting machine learning tools in the hands of everyday developers.

Using the API

The details are a little scant as the API is not yet open to the public at the moment but, as the FAQ and sample code explain, it appears to work as follows:

A data set in CSV format is uploaded to Google storage. This can contain up to 100 million rows of text or numeric features. Each row can be associated with one of up to several hundred classes.
The URL obtained after uploading the data set is POSTed to a second URL, /prediction/v1/train/DATA_ID, for Google’s learning algorithm (all URLs are relative to https://googleapis.com). It is not clear what algorithms are being used behind the scenes for this step but the home page says the API will automatically choose from a variety of techniques.
The training occurs asynchronously and its progress can be queried by issuing a GET to /prediction/v1/query/DATA_ID. Once training is completed, this query will return a cross-validated estimate of the learned model’s accuracy.
To make a new prediction with the trained model, a POST request containing the data to classify is sent to the /prediction/v1/query/DATA_ID URL and a label prediction is returned.

Although this is a relatively simple API and, at present, only deals with classification, I believe it has the potential to cover a large proportion of most web developers’ prediction needs (e.g., text classification, sentiment analysis, click-through analysis) as well as several scientific applications.

Related Projects

The Google Prediction API is not the first to offers inferential services over the web but I do think they are the first to focus on building reusable predictors and to do it with a clean API design.

Some other projects offering prediction services include:

uClassify — This is probably the closest existing service to Google’s. It also provides an API for training and predicting but, upon a cursory examination, appears a bit more complicated than the Google prediction API. I believe the main algorithm used by uClassify is a variant of naïve Bayes.
MLcomp — This recently announced service is more aimed at machine learning researchers and provides a convenient way to compare several algorithms on a selection of data sets using a variety of metrics. Unlike Google’s offering, MLcomp does not make the trained predictors available via an API and focuses more on providing easily repeatable experiments. One nice thing about the MLcomp service is that anyone is free to upload learning algorithms provided they implement a simple calling pattern.
predict — A simpler MLcomp-like service built by Joshua Reich that lets users upload CSV files to learn from and/or snippets of R code to run. Once again, the aim is to evaluate algorithms rather than train predictors for subsequent use.
TunedIT — Similar to MLcomp and i2pi’s predict, this service aims to make comparing learning algorithms across data sets easier. As far as I can tell, it does not offer an API for running learners and predictors over the web but rather offers users the ability to create data-mining challenges that other users can compete within.
ExpDB — This is not so much a service as a growing database of experimental results but I thought I’d include it here as it has a similar focus to the last three projects. The main innovation here is the creation of a language – ExpML – for describing and querying the parameters, algorithms, data sets and results of machine learning experiments.
RL-Glue — While not about prediction per se, this somewhat older project is related as it offers an API for defining reinforcement learning problems that can be solved in a programming language-independent way.

Of course, there are also many machine learning toolkits such as Weka, Orange, Elefant, Rattle and more that provide implementations of algorithms, but these do not offer them as services.

Future Predictions

Over the last few years we’ve seen a dramatic increase in the amount of data being generated and made available over the web (e.g., Freebase, DBpedia, Data.gov, Netflix, and protein databases). Also, thanks to services by Google, Amazon and others, there has also been a large-scale commodification of computational power and storage.

There are a handful of companies at present — Flightcaster, for example — who have realised that there is immense opportunity at the intersection of these developments to start applying large-scale machine learning. Hopefully, what the Google Prediction API and other services will provide is the spark for an explosion of new and creative approaches to distilling knowledge from raw information.

I will be watching how this all unfolds with great interest.

Attending AISTATS

2010-05-09T00:00:00+10:00

Provided there is no further volcanic disruptions coming from Iceland, I will be in Sardinia, Italy on Wednesday for AISTATS 2010. As well as being able to indulge in what I imagine to be great food and coffee, I’m also looking forward to engaging with a very interesting looking program.

Some of the posters and talks I’m looking forward to include:

Learning Exponential Families in High-Dimensions: Strong Convexity and Sparsity by Kakade, Shamir, Sindharan and Tewari
Exploiting Feature Covariance in High-Dimensional Online Learning by Ma, Kulesza, Dredze, Crammer, Saul, and Pereira
Empirical Bernstein Boosting by Shivaswamy and Jebara
Guarantees for Approximate Incremental SVMs by Usunier, Bordes, and Bottou
Structured Prediction Cascades by Weiss and Taskar

I’ll be standing by a poster on Friday explaining Bob Williamson and my work on some characterisations of the convexity of a natural class of losses for binary class probability estimation.

Please drop by and say hello if you are also going to be at AISTATS.

Upcoming ALT and DS Conferences

2010-04-13T00:00:00+10:00

Two traditionally co-located international conferences—Algorithmic Learning Theory and Discovery Science—will be hosted in Canberra, Australia in October this year. One of the reasons (amongst many) that this blog has languished recently is that I am one of the local organisers. Eric McCreath is the other.

One of my jobs as local organiser is to make sure every potential attendee knows about these conferences. I thought that a bit of shameless promotion would be as good a way as any to bring Inductio Ex Machina back from the dead. So here goes…

Important Dates

Before I even tell you what the conferences are about I need you to do the following.

Take out your calendars and with your most indelible pen inscribe the these dates:

May 12, 2010 — ALT (full paper) and DS (abstract) deadline
June 17, 2010 — Notifications of acceptance sent out
October 6–8, 2010 — Conferences are run
Prepare some amazingly ground-breaking paper for submission.
Book your flights to Canberra, Australia for October right now.

The Conferences

Together, the ALT and DS conferences cover a broad range of research areas within machine learning.

The ALT conference has been running for 20 years and, as its name suggests, its focus is on foundational issues in the theory of machine learning. A full list of topics can be found in the ALT Call for Papers.

The DS conference has been running for 12 years and emphasises the development, analysis and application of algorithmic methods of knowledge discovery in scientific domains. The DS Call for Papers has more information.

Invited Speakers

We are very lucky to have enticed some great speakers, including:

Alexander Clark from Royal Holloway, University of London
Manfred Warmuth from UC Santa Cruz
Peter Bartlett from UC Berkeley
Ivan Bratko from the University of Ljubljana, and
Ramamohanarao (Rao) Kotagiri from the University of Melbourne.

Personally, I’m very excited to hear Manfred and Peter talk as they are both fantastic learning theorists and I’ve been reading a lot of their work recently.

Bonus Summer School!

As an added bonus, both of these conferences will be held just after a co-located Machine Learning Summer School running from the 27th of September until the 6th of October.

All in all, it is shaping up to be a great combination so if you’ve ever thought you’d like to visit Australia and wanted a good, machine-learning excuse, here it is.

Hope to see you here!