Penguins.mooh.org

Music geekery

Picture of music section

It's been a little while since I last posted an update about what's been happening here in Heidelberg. It's not for lack of effort though, as I've been busy developing up a whole new section for the site. You might not have noticed it when you navigated your way here (or read this via your news-reader), but the music link now points to a new page.

Let's get the disclaimers out of the way first. I'm doing this for fun, and as such, I'm not too concerned that the page doesn't work on Internet Explorer. I don't even have access to a machine on which I can try to hack it to work with that godawful browser. The page does work with any Mozilla family browser, or with Safari. It might work with Opera too.

This whole idea for this sub-section was borne from a question which I kept getting asked, and I had no suitable answer for - "What do you listen to?". It's a deceptively hard question to answer. A few years ago, I'd just trot out some general genre names, and maybe some artists, but that only really corresponded to what I was listening to at that particular point of time. The time and mood nature of my listening habits were really the key to making things hard for me to answer the question. What I really needed was some way to keep track of what I was listening to.

A small digression here while I talk about my favourite company - Google. Google really know how to keep track of stuff. They have got the whole passive monitoring thing down to a fine art. They're collecting masses and masses of data from their users, and actually drawing some useful conclusions from their data (look at their Zeitgeists). It's a philosophy which I'd like to be able to extend a bit more to the rest of my life. I'd like to keep track of everything I do, and then at some undertermined point in the future, look back on my data and make some optimisations. Maybe realise that my bus timetables are somehow correlated with my eating habits, or some other result from data mining. It seems a little unfair that the companies you rely on for service (Telephone, power, internet) can collect this data about you - but you have no way of taking advantage of the data yourself.

Last.fm filled the needs that I had for music tracking quite nicely, but the stats on the charts were a bit lacking. Although it did provide some nice overall information, it quickly became obvious I needed to really do some analysis on the data myself. I kept waiting for more data, and by the start of March, I had 20,000 tracks played, and roughly a year of listening data ready to be analysed.

The basic technique for analysing the data was to take weekly artist data (the number of times an artist is listened to in a week), find the tags which best describe the artist, and use those tags to add to a weekly tally of tags. In the end I should have found out whether I had a clear preference for a genre during the week. To implement this, I used a whole bunch of Ruby (I did mention this was a hobby project right?) scripts combined with the Audioscrobbler web services provided by Last.fm. Tag data is notoriously messy, so all tag data goes through a normalisation process by looking up synonyms in a dictionary I created.

The front-end is a bit of XHTML (although it currently doesn't validate), combined with some nice JavaScript libraries from MochiKit, CanvasGraph (I think I shared a few uni courses with this guy!) and the WebFx Slider. The actual content is then generated through an XSLT transformation on the backend, integrating all the data together, and transforming things like RSS feeds into something which is a bit easier to handle.

The current plan is to add a bit more functionality to this, and then I'll add a sub-section of my Geek sub-section with source code and everything you'll need to get this up and running yourself. I'm hoping to eventually mutate the visualisation javascript into something which can be used to apply interactive visualisations (complete with controls) to any set of tabular data.

Did I actually manage to answer the question? Well, it turns out I listen to a bit of electronic, indie-rock, underground hip-hop and trip-hop. There's a more detailed analysis of it all in the music section.

4 comments

Bhautik Joshi *

I sat there wiggling that widget for ages ;) awesome stuff...

Anonymous *

Wow. That is a great use the graphing stuff. Neat.

Hiren Joshi *

Cheers :)

There's still a fair bit I need to do with the page - the UI isn't quite right, and the backend JS probably doesn't degrade well enough for the poor people stuck on ancient browsers. Once I find more time!

Post a Comment