Skip to main content

A Search Engine with Ears

Imagine a music search engine that asks you to type in words like "acoustic guitars" or "electronica with female vocals" and then returns songs that fit. Electrical engineers and computer scientists at the Jacobs School have created this kind of system. They think of it as "Google for music."

Lift up the "hood" of the new search engine and you find algorithms for labeling music that are based on audio signal processing and statistical modeling. You also find a video game. In fact, you find a whole series of short Web-based video games built primarily by Jacobs School students. The games lead players to label songs with words and will soon make their debut on Facebook.

When lots of players label a particular song with the same word, this word-song combination gets used to train and improve the automated song-labeling system that is the backbone of the new search engine and the brainchild of the Jacobs school team.

"While the games are crucial for our research and our search engine, they are also a lot of fun and socially intense. You can even IM with other players," says Gert Lanckriet, leader of the project and a professor in the Department of Electrical and Computer Engineering.

Doug Turnbull, a Ph.D. candidate in the Department of Computer Science and Engineering and one of the masterminds behind the music search engine, hopes the project "will provide exposure for the countless talented musicians that almost no-one has ever heard of."

This hope is not farfetched: their automated song labeler, called a "computer audition system," is capable of annotating an unlimited number of songs with words that describe how the songs actually sound. And once the songs are labeled, famous and obscure "blues songs with tambourines" alike will come back when that's what you type into the new search.

Of course, the songs have to be annotated before they can be indexed by the search engine. To help get lesser known music into the system, the team is setting up a Web site where bands can upload their own songs for annotation.

One of the bands that will surely upload songs is The Audition Laboratory. The guitar player - Luke Barrington - is one of the key members of the music search engine team and an electrical engineering Ph.D. candidate at the Jacobs School.

Jacobs grad student Luke Barrington plays a video game created to capture the song-word information the team needs to train its new music search engine.
Jacobs grad student Luke Barrington plays a video game created to capture the song-word information the team needs to train its new music search engine.