How to get iTunes to take over your music files

iTunes has two different ways to deal with files on disk – adding a file can either create only a reference to it in the library, or iTunes can copy the it into a central folder where it’ll be managed for you. Both methods have their fans and their detractors, but I’m in the process of switching from referenced to managed, because iTunes 9 got a new, much prettier, folder structure and I’m tired of micromanaging this stuff.

The approved way to convert your library is to select “Keep iTunes Media Folder Organised” and “Copy files to iTunes media folder when adding to library” in the ‘advanced’ section of iTunes’ preferences. Then you select the ‘File.. Library.. Organize Library’ menu item and tell it to copy all your media files into the new library.

I don’t want to to this, I don’t want 2 copies of everything, and I don’t have the disk space anyway. To do this the easy way (on the Mac, anyway),

  • Move all your media files into the new library folder. You have to keep them on the same volume or the iTunes references will get lost. If you want the new media folder on a different volume, you’re stuffed, but you’d have to copy them anyway so why are you here? How they’re laid out on disk inside this folder doesn’t matter.
  • Deselect “Keep iTunes Media Folder Organised” in advanced preferences.
  • Restart iTunes.
  • Reselect “Keep iTunes Media Folder Organised”
  • Click the ‘Change’ button next to the Media Folder location in advanced preferences, and click ‘Choose’ to re-select the current folder. We’re not actually changing it, we just want to pretend we did.
  • Click Ok.
  • Watch iTunes happily re-organise all your files.

licenses, daap and md5

Another day, another python-daap release. Version 0.6 allows you to connect to shares requiring a password, and fixes some problems with the source tarball. However, there are still outstanding licensing issues that may entail an annoying change to the library in the future.

First a brief bit of history. Since version 4.2 iTunes has required incoming DAAP client requests to be signed with a hash of their parameters. In iTunes 4.2, this was signed using md5. In iTunes 4.5, this was changed to a slightly modified version of md5 with some internal numbers jiggered, presumably just to annoy me. In iTunes 7, the hash was changed again, and no-one has since reverse-engineered the new system, so no open-source DAAP clients can currently talk to the most recent version of iTunes. This is clearly the intent of the hash – it serves no other useful purpose, it exists only to prevent clients other than iTunes talking to music shares, presumably to enforce the restrictions on shared music.

PythonDaap can talk to all versions of iTunes <7, because I include in the source a modified version of the md5 digest algorithm that was reverse-engineered by David Hammerton. For the most recent version of iTunes, however, you’re just out of luck. There are various people working on reverse-engineering this one but it’s clearly a hopeless battle (it’s far easier for Apple to come up with another hash than it is for anyone else to reverse-engineer it), and not one I have any interest in being involved with.

To complicate this more, the MD5 implementation I based my version on is the ‘RSA Data Security, Inc. MD5 Message-Digest Algorithm’, which is licensed under an annoyingly non-LGPL-compatible license (I’m required to refer to it as the ‘RSA Data Security, Inc. MD5 Message-Digest Algorithm’, for instance). Thus my ability to legally distribute the source of PythonDaap is in question. I see only 2 reasonable ways out for me:

  • I could port the code to use a library with a more friendly license – OpenSSL for instance. But that’s a bunch of work (I didn’t even do most of the work for the current hasher – I just ported the changes mode in Digest::MD5::M4p) and doesn’t help the iTunes 7 problem.

  • I could just rip out the code completely, and only support iTunes servers before version 4 (and open source servers). This approach appeals to me more – I assume that most people are using recent versions of iTunes, so all the users of PythonDaap must be using open source servers. Many of these are far better servers than iTunes itself – they allow any number of connections, for instance, unlike iTunes and its ridiculous restrictions.

I consider it a real shame that the open source community has embraced a protocol that neither works with the most recent version of the software that it was reverse engineered from, nor is actually particularly well-designed or easy to use. There are plenty of open source server implementations of DAAP, and iTunes can talk to all of them just fine, but no open source clients can talk to iTunes. That developers are essentially supporting Apple in this by producing servers for a protocol that they’re not able to write clients for is annoying.

More annoying is that, at the same time as being restricted by the closed protocol, I’m having to deal with problems caused by the fact that not all of my code is open enough. Yay.

blech has been on at me recently to finish my half of our evil ‘Wire and string’ project – it’s a cunning, cunning plan which we really should put more effort into. Anyway, here’s a heap of self-congratulation that I’m done, and he should get off his butt and implement the other half.

Essentially, the problem is as follows: We have a computer in the lounge that can play music, hooked into the network and the stereo system. But there’s a complete lack of decent software to control the thing. We searched and searched, and all the interfaces we could find sucked.

Now, the best music control interface we know of is iTunes. We want iTunes to control what is being played on the lounge machine. But it’s not a mac. Hence the goal – have iTunes on a mac locally, pick up what song is selected/playing, and send that to the lounge computer, which will then play it through the amp. blech took the iTunes half (the Wire), I got the server half (the string).

Now, there are no non-awful MP3 playing interfaces for perl. None. Almost everything that I could get to compile at all (and didn’t need non-free libraries you can’t download any more) wrapped either mpg123 (now that’s a pretty web page) or xmms. The mpg123 wrappers seemed to be the best bet, and for the most part they work, but there are… issues. I didn’t get proper events for songs ending or stopping, it was quite easy to wedge the thing to the point where you couldn’t control it any more, etc, etc. | believe these are limitations of the command line mpg123 interface, and I can work round them, but I don’t want to have to patch the source code just to get a working system, and it makes distributing things a complete pain. I also need the server to be absolutely robust, it’s no good having to SSH in every 5 mins to fix the server – the whole point is to avoid having to think about it.

Wrapping xmms isn’t much of an improvement. It’s got a much nicer interface to control it, but, of course, it’s controlling xmms, which is a seperate app that needs to be running, and it needs to be running on a machine with an X server. Hard, given this is a headless server.

But, in the end, that’s what I used. I was saved by Xvfb – I can run an X server on the machine without having to worry about graphics cards, and launching xmms from the cgi is possible, so it will run as the right user and everything. After a lot of messing with annoying permissions, I have a CGI that will make noise.

The other problem is that the Mac only knows where the music is on the machine locally. Rather than having the iTunes end convert the local path into something the server knows about, I take anything that might possibly be a path to some music, and trim things off the left of it till the right hand side matches something in the ‘locate’ database. Disadvantages – you need an up-to-date locate database. Advantages – you can pass just about anything that is music, and the server will have a shot at playing it. In practice it works really quite well.

So, yeah, code . w00t.