O'Reilly    
 Published on O'Reilly (http://oreilly.com/)
 See this if you're having trouble printing code examples


The magic of Pandora derives from a simple principle: a song listeners enjoy should lead to other songs they'll enjoy. Pandora is an Internet music service with an unusual twist: you merely select a song or artist you like and the system builds a playlist of additional songs based on those musical characteristics.

After Pandora builds the playlist (which it calls a station), the music starts streaming to your Web browser--for free. You can then refine the playlist by clicking a thumbs-up or thumbs-down button as subsequent songs play. Other buttons call up more information on the song or the artist or whisk you to Amazon or the iTunes Music Store to buy the CD or file. (Pandora makes money by collecting referral fees for those clicks and by displaying ads in the Web interface.)

Using Pandora is simple, and the results can be surprisingly good. But as you'd imagine, there's a lot of innovative technology behind the scenes to make it work. I met with chief technical officer Tom Conrad to find out more. Before reading further, why not visit Pandora yourself and set up a little background music?

Pandora Thumbs Up

Pandora's main window shows the stations you've created at left, with the current station highlighted. (Stations with a curving arrow in the name were recommended by another Pandora listener.) The main area shows the currently playing song and the two previous songs, which scroll to the left as new songs enter the queue or as you skip ahead. Here we have rated two songs thumbs-up and one song thumbs-down.

Decoding the Music Genome

Pandora (which is also the name of the company) grew out of the Music Genome Project, which company founder Tim Westergren began six years ago. Westergren had studied music at Stanford and toured the country in a band for many years, eventually spending time discussing music with film directors. He became fascinated with the way directors described the music they were looking for, which led to his wondering what made people enjoy certain types of music. He asked himself, "If people haven't found any music that they love since college, and artists are struggling to find an audience, is there a role for technology to help bridge the gap?"

Westergren started the Genome Project from the idea of creating a platform for connecting people with music that they'll love based on music they already enjoy. The project uses experts called "music analysts" to deconstruct music into its fundamental parts and capture the results into a database. Pandora has 40 professional musicians who come to the office every day and listen to one song at a time, analyzing each in anywhere from 200 to 400 dimensions. (The dimensions are somewhat different for each genre of music.)

Pandora chose the dimensions because they are quantitative. For instance, how breathy are the vocals? Is the music diatonic or chromatic? The music analysts are trained to be able to score songs consistently. In fact, one of the test cases is, "Could a group of 10 musicologists listen to a song and agree on one score for a particular element?"

Pandora Analysts
Pandora analysts plow through CDs in search of musical match-ups.

Once a song is captured, the system loads the analysis into an n-dimensional vector space. To find matches, Pandora uses "nearest-neighbor" computation. For example, let's say you like "Hips Don't Lie" by Shakira. Pandora will take that song and find its nearest neighbors in 200-vector space (the number of data points used for this genre). The resulting songs form a playlist of songs quantitatively similar to "Hips Don't Lie."

As a lover of jazz, I found this system worked remarkably well. The vast majority of the time, my stations stay true to the original profile. But I've heard from others that, although very good, stations built on a particular pop tune are not as accurate. Some listeners have complained that Pandora's database over-represents American music.

Tom Conrad, Pandora CTO

Tom Conrad, Pandora CTO

There is a bit of magic here. For instance, Pandora doesn't weight all attributes equally. The vocal-performance attributes might be more important than how the rhythm guitar is played.

"One of the things we can do for playlist generation is play with that notion of weighting," Conrad says. "We do that to help give form to the listening experience--beyond just a random collection of similar songs. As you listen to Pandora, you'll find there are musical themes that are shared across songs and weave in and out of the listening experience. So, you might hear a sequence of songs that feature similar percussion, and then a sequence of songs where the emphasis is on vocal similarities, and then a sequence of songs that have similar brass sections--which is unique to us because we understand, on a song-by-song basis, what the music is.

"What is exciting about the Music Genome Project, with respect to Pandora the radio-listening experience, is that by understanding the music on a song-by-song basis we can put together a playlist that has a much more natural ebb and flow than you might be able to do with collaborative filtering data," Conrad says.

I asked him to compare the Music Genome Project with Gracenote's new machine-listening approach that takes a song and decomposes it into elements using computers. "When we started the Music Genome Project, the state of the art in machine listening could decompose about six to eight elements," he replied. "Something about the tempo, the key signature, the song form, etc. Over the last 5 years there has been some dramatic progress, but the state of the art today is about 20 elements--a tenth of what we collect for a pop song.

"Since we use a human analyst to analyze song by song, we've experimented with using a smaller number of elements," he continued. "We've determined that you can't create interesting playlists with only 20 attributes. But we do keep an eye on machine listening as it might provide a way to augment the manual analysis."

The Curator for What Ails You

Conrad then brought up an interesting philosophy that suggests how Pandora can help promote new music and artists: being a curator of songs. "There is a piece to Pandora where the songs are 'curated,'" he said. "We are very careful not to play into the popular mechanisms of what constitutes 'good music'--what's on the Top 40 this week. But we do have staff whose full-time job is to go out and look for great music. We receive suggestions from our listeners all the time, including people sending their own music." For tips on submitting your own music, see the sidebar "Submitting Your Music to Pandora."

Submitting Your Music
to Pandora

Would you like the world to hear your original music on Pandora? Simply send your CD with information about yourself, your band, and your music along with the official submission form (PDF) to the company. Be sure to follow the detailed information on their FAQ. The Pandora music analysts review every CD submitted, although they won't share those criticisms with you.

Pandora compensates all artists through licensing fees paid to ASCAP, BMI, SESAC, and SoundExchange. Learn more on this FAQ.

More than 50 percent of the music Pandora receives makes it to the Music Genome Project after passing through the human quality filter. Conrad continues, "We have people whose full-time job is to review every magazine and blog about interesting music, to plumb the Creative Commons depths and find great new music, and to put the very best on the air. So, while we are careful not to tell people what 'good music' is, there is an element where we are finding the very best music out there, music that's finding an audience, music that's finding critical success, and giving it a place to find a bigger audience."

I asked what Pandora was doing to avoid being influenced by big record labels, which have been widely accused of corrupting traditional radio through payola schemes.

"I think curator is the right word," Conrad replied. "Of all the financial models that could be leveraged to make Pandora a successful business, the 'play for pay' model runs completely spiritually opposite to the founding of the company.

"Let me give you a concrete example," he continued. "There are 30 Chicago albums. At 10 songs a piece, that's 300 possible songs by Chicago. iTunes probably has every single one of them, but we don't. Some of the musical judgments we exercise are about which of those 300 songs have found an audience that wants to hear them. There's some art to that, I admit. We make a judgment: at what point have we analyzed a sufficient quantity of Chicago music to be able to create the best possible listening experience when people tell us they like music that is similar to Chicago?"

Intricate Rhythms

Pandora's reasoning is sometimes odd: this classic ambient track has neither cymbals nor intricate rhythms.

I then asked about something that has irritated me about the Pandora service: I've just finished listening to a station that I labeled "Soft Jazz Guitar"--Joe Pass, Kenny Burrell, Herb Ellis--that type of style. Now, I switch to my Bill Evans station and a solo guitar song by Jim Hall comes on. And I ask myself, "What's this song doing in my Bill Evans station? This song should be in my 'Soft Jazz Guitar' station. Why can't I tell Pandora to place this tune in the appropriate station?"

"It's fascinating to me that you raise that particular example," Conrad said. "Because the scenario that you just described is--after we evolved the product over five months and took a lot of low-hanging fruit off the table--probably the number-one listener request.

"It's fascinating because I wouldn't have guessed this would have risen to the top of consumer feedback. It was an area I was blind to. I guess my own usage and my observations of others in usability settings didn't have that perspective. But there is no question that that is the number-one listener request right now."

Shortly after our interview, Pandora added another top customer request--Backstage. Backstage is a feature that provides information, derived from the All Music Guide, about the current artist and song. As an example, searching on "Blind Faith" brought up a comprehensive story by famed record critic Bruce Eder about the trials and tribulations of recording the album. From there you can create a Pandora station built from your search word. It's a wonderful addition and added to my enjoyment of the album.

Pandora Menu

Right-click a song tile and a pop-up menu lefts you rate a song (so you'll hear more or fewer songs like it), see more information, or buy a copy from a partner store.

The Tech Inside the Box

With the exception of the final user interface, which runs in Flash, Pandora is based primarily on open source software. The foundation is the PostgresSQL database running in Debian Linux, with the client tier developed in OpenLaszlo. The server-side infrastructure is mostly Java running in a J2SE servlet container. Database access, the playlist generator, and station and feedback management all live in this Java tier.

Pandora creates playlists with a "matching engine," written in C and Python, for each listener station. This engine builds the low-level linkage to the "source" music (the music that listeners indicate they like) and the music that actually gets played (a mixture of what the listener explicitly indicated, mixed with music that the Pandora service believes listeners will like). The replication system is Slony.

The listener runs the Pandora service in a web browser, seeing a mixture of HTML and Ajax techniques for ad delivery and Flash for music delivery. The player, which Pandora calls the "tuner," runs in Flash; the user interface was written with OpenLaszlo. Because Laszlo has been around for six years, it has a long track record in Internet time--certainly longer than its closest competitor, Adobe Flex.

OpenLaszlo has been open source for about a year, inspiring a vibrant developer community to rise up and extend the platform beyond Flash. This is a very interesting development in comparison to the closed-source universe of Flash. OpenLaszlo allows developers to create in the OpenLaszlo environment and deploy to either the Flash runtime or to DHTML.

Because web browsers don't necessarily provide native mechanisms for playing audio, Pandora required some type of plug-in. Windows Media Player, QuickTime, and Flash were the obvious choices. However, Flash appealed to the developers because it didn't come squarely from the two main camps, Apple and Microsoft. Flash does a nice job of providing audio as well as graphics across the three main platforms--Windows, Macintosh, and Linux.

I asked Conrad if there were any special magic about the way Pandora delivers web audio. He said the most interesting item is that the company uses a highly tuned implementation of the Apache web server. "If you were starting an Internet radio service as little as three years ago, it wouldn't be a workable choice," he explained. "Apache wasn't able to deal well with large numbers of connections. The Internet transport, especially with people trying to access with dial-up connections, was such that a connection-oriented protocol like HTTP wouldn't have been a workable choice. So you'd have to go with something like RTSP."

Skip too much

The twisted terms of online radio licensing occasionally intrude on the listening experience.

Scaling Challenges

Of course, it's one thing to develop a slick web app and another to ensure it continues delivering fast, dependable results as the number of users increases. Conrad said there are two kinds of scaling challenges with Pandora. "The first is the need to scale audio delivery to a really large number of simultaneous listeners," he says. "We tried to create an architecture that would allow us to scale our audio capacity horizontally. We have a system that lets us decouple the audio delivery from the computation of playlists. The basic architecture is that our player asks the playlist-generation servers for a series of songs. The player then stitches those songs into a playlist stream. That allows our player to make audio requests from any audio server on the Internet."

Conrad then walked me through the sequence from the point when listeners log on to the point when they start hearing a song:

  1. The client initializes the web page.
  2. The Pandora Tuner (the music player) authenticates with the Playlist Server, passing along listener credentials.
  3. The Station List for that particular listener is returned to the Tuner and rendered on the screen by the Tuner.
  4. The listener selects a particular station for playback.
  5. The Tuner makes a request to the Playlist Server for a sequence of songs to play.
  6. The Tuner receives a list of playlist fragment descriptors and begins sending audio requests based on the information received from the Playlist Server.
  7. The Tuner stitches those audio resources together into a stream of music.
  8. The Tuner plays the first song.

Also interesting is how the Pandora service balances requests. When a listener makes a request, the request is sent to Pandora's data center, a load balancer distributes that load across an arbitrary number of Playlist Servers, and those Playlist Servers interact with an arbitrary number of replicated music databases that hold all of the music logic information. The playlist information is then returned to the listener.

"Similarly," Conrad said, "When an account-oriented request comes through (e.g., 'Hey, give me my station,' or 'I want to give feedback for the definition of this station,' and so on) that work can also be distributed across an arbitrary number of back-end database servers."

Overload

We experienced occasional outages, but they didn't last long.

The second challenge is scaling the computer power required to deliver a truly personal set of playlists for each listener. "From the beginning, many of our architecture decisions were driven by the desire to scale the service truly horizontally," says Conrad. "Much of the work we've done was filtered through the lens of: How do we achieve the amount of horizontal scaling flexibility that's required so that adding capacity means only racking up more boxes and turning them on?"

"Pandora has grown extremely quickly; in just 10 months more than 2.5 million listeners have signed up. That growth rate makes accurate forecasting crucial," Conrad said. "If it takes three weeks to get a server built, burned in, and deployed, and it takes four weeks to add a gigabit bandwidth capacity, being able to make accurate predictions about demands so that those lead times are accommodated is one of the biggest challenges. The business challenge is accurately forecasting and the technical challenge is about the logistics management of the hardware provider, the bandwidth provider, the co-location provider.... It's a technical operations challenge as much as a software challenge." (For more on the scaling challenge, see the "Rack It Up" sidebar.)

Rack It Up

"Another interesting challenge around services that are growing really fast is the new generation of servers that pack a lot of storage or CPU capacity into a small package," notes Pandora CTO Tom Conrad. "That means you can spend your co-lo power budget in half the number of racks that they planned for. While you can easily get the space, you can easily run your co-lo provider out of electrical power, which is a massive infrastructure challenge for them.

"High-density deployments also have thermal characteristics that challenge the cooling capabilities at data centers," Conrad continues. "I've talked to people who run some of the largest photo-sharing sites in the world and I know they struggle with this problem: lots and lots of storage, lots and lots of power needed, lots and lots of heat generated. They have sufficient physical space, but they don't have sufficient cooling and sufficient power at their data center. A lot of these data centers were built in the bubble and they were built on a set a assumptions that are no longer true.

"One of the reasons that server people are excited by the dual-core chips is that it lets you have more capacity per watt so you can scale up inside a box without dramatically changing the thermal and power picture."

Shelves of CDs

Sampling CDs in the Pandora library.

Outside the Box

Pandora hears a recurring theme from its listeners every day: You're introducing music to me that I might never have heard before.

"When we talk to our listenership about what we could do to enhance the product," says Conrad, "the thing that stands head and shoulders above everything else is making the playlist better. I'm proud of the playlist today, but there is no question that the number-one indicator that consumers will fall in love with the service is how great they think the music is. It isn't, 'Gosh, if you just had tagging, or discussion forums, or artist bios, or digital downloads.' People are asking, 'What percentage of the time do the stations I create play the music I love?'

"We really have a deep belief in building the Genome Project one song at a time, and our listener base one person at a time," Conrad continues. "In a lot of other companies, that would be calculated from a listener acquisition goal. It's really not like that here. It's about a deep understanding that the right thing to do is to get out there and find the people who are making the great music, and meet the people who love the service--or hate it--and bring that back to the company to make the product and music selection better.

"We think of the Music Genome as a collection, something that we are building for prosperity," he concludes, "and we want all great music, where 'great' is defined as 'there's an audience of any size out there that's hungry for music like this.'"

Brad Fuller explores creative technology at www.bradfuller.com.


Return to digitalmedia.oreilly.com

Copyright © 2009 O'Reilly Media, Inc.