More playing with JSON and Spidermonkey has revealed yet another incredibly annoying fact (I hate those guys). Spidermonkey provides a lovely uneval() function, that does the exact opposite of eval() - turns JS objects into strings. It works on almost everything, and make life very very nice. There's also Object.toSource() which does something similar (but not the same - try uneval("") vs "".toSource()).

But the strings that uneval produce are not valid JSON, as I have been assuming. I've been getting steadily more worked up at all the JSON parsers in the world, refusing to parse things that are clearly valid JavaScript, and eventually I go look at the spec, which fails to list ' as a valid string delimiter. And guess what delimiter uneval produces? Yay. So all the parsers are fine, and it's just SpiderMonkey that's broken.

Fortunately, Mochikit provides a nice serializeJSON() function.

Here's what I want to do. DateTime objects conveniently stringify to their ISO8601 date representation. But last week the clocks changed, and all my code broke, because this representation doesn't include the time zone. I want a DateTime object that will stringify to an ISO8601 date including a timezone. Shouldn't be too hard.

Let's look at the perldoc for DateTime::Format::ISO8601. There's an awful lot of things that can be called an ISO8601 date string. 20060101T120000+0100 is a valid date. 2006-01-01T12:00:00+01:00 is also a valid date string. Personally, I prefer the latter. And fortunately, the W3C have looked at this mess and decided 'It's too complicated'. And they agree with me. So this is the string that I want to produce.

DateTime.pm provides a formatter method - you can define a formatter class that will stringify the object. Horay. But using DateTime::Formatter::ISO8601 doesn't produce a string with a time zone. My next plan is to use DateTime::Format::Strptime and provide a strptime spec that will produce the right string. To the DateTime / strftime docs! Apparently, %FT%T produces a nice 2006-01-01T12:00:00, so all we need is the time zone. And again, here is where we fail. There's %z, but that produces +0100, and I need +01:00.

At this point, I write a hack and get back to work. DateTime lets you use the magic format specifier %{ method_name } to call a method of the DateTime object. So I push a method into the DateTime namespace that produces the string that I want.

sub DateTime::iso8601_with_tz {
  my $self = shift;
  my $val = $self->strftime('%FT%T%z');
  $val =~ s/(dd)$/:$1/;
  return $val;
}

Then I can create a formatter that will format DateTime objects the way I want them.

use DateTime::Format::Strptime;
my $formatter = DateTime::Format::Strptime->new( pattern => "%{iso8601_with_tz}" )

Finally, I can use this as my formatter.

$my_datetime->formatter( $formatter );

Nasty.

Yay, I've released Yet Another Templating Language. This one is a perl implementation of TAL, which is a nice XML-based templating language I've been playing with recently. There are other (sort of) implementations, but I consider Template::TAL (a) closer to the spec, and (b) more abstracted - I can load templates from anywhere, not just disk. It's weaker in other respects, of course, but that's life..

Off I go, making random unsubstantiated claims about the danger of using JSON with non-ASCII characters. This called for a Test. So I wrote one. Visit my JavaScript unicode test page and see how your browser interprets external JavaScript files - I serve an 'é' using JavaScript to the page via 3 methods and 2 character set encodings, and try to render them all.

My conclusions from some limited testing? Owch. You can't include a JavaScript file and expect the client to interpret it properly, unless you control both the server serving the JavaScript and the HTML page requesting it, and can make sure that they're both in the same character set. Alternatively, you can escape all non-ascii characters in your JavaScript files using the xXX or uXXXX notations, which seems to work everywhere I've tried, but also seems like a pathetic work-around. Anyway, needing a work-around for only the non-obvious case means that no-one will actually do it, because no-one ever seems to bother testing with non-ASCII (see any on the JSON examples page, for instance?).

However, requesting JSON using XMLHTTPRequest seems to do the Right Thing in every browser I've tested, including those that include JavaScript wrongly. So if you're using JSON as an RPC transport, instead of XML, for instance, it looks safe. From a character-set point of view.

In the bad old days of web 1.7ish, the cool thing to do for dynamic web applications was to generate HTML snippets on the server-side, pull them into your app using XMLHTTPRequest, and shove them bodily into a DIV on the page somewhere. "generating DOMs is hard" was the excuse, "the server already has a nice templating language", etc, etc. And this was Good.

Nowadays, of course, this is Evil. Pulling HTML across the wire? It's inefficient! Even worse, we want to send XHTML to the client, because it's cool, but then you can't treat your HTML like a string, you really do have to mess with DOMs. Fortunately, cute tools like MochiKit make it really easy to create DOM nodes, and provide really nice tools for making ajax requests. We see the rise of JSON - serializing your raw data on the server into a JavaScript-evaluatable string and sending that across the network, then building the DOM on the client side based on that data. And this is Good.

Soon, I expect, people will run up against annoyances in JSON. For instance, I bet there are lurking character set issues. It's also not very portable - if I go to all the trouble of writing and exposing interesting functions of my web application in machine-readable ways, I'd like to be able to access this data using things other than javascript. Sure, there are modules like JSON that will both create and parse these things, but throwing actual XML around seems much neater to me.

We have a strange mix of XML and JSON APIs at the moment, some toolkits even making it easy to ask for either. But I consider XML far superior for this sort of thing, if only because the character set issues (my personal bugbear) are properly solved with XML. It's disadvantage is that the JavaScript tools for dealing with the DOM are very annoying, but when we have something like XML::Simple or xmltramp for JS, and reading incoming XML is almost as easy as reading JSON, we can get rid of it and use nice sensible RPC mechanisms. JSB is a lovely example of this, for instance. And this will be Good.

Wow, core data is shiny. James re-wrote (the core of) Blotter in 10 minutes using it, so after a bit of polish, I have a 0.9 release, written in pure ObjC/Core Data. Alas, you need 10.4 to run it, and XCode 2.1 to build it (an 800 meg download! ow) but that's the cutting-edge future for you. Sigh.

Today's shiny toy is BluePhoneElite, which sits on the lappy and talks to the phone via bluetooth. It does the usual proximity stuff - lock the screen when the phone goes out of range, pause iTunes and display bezels when there's an incoming call (which is great when the phone is set to silent mode and I don't notice it ringing..), and random other stuff (yay applescript). It'll talk to your address book locally, and let you add unknown numbers to in from your incoming call list. I can use it to send SMSes using my nice big keyboard instead of fumbling around on a tiny phone keyboard. All this stuff is boring - Romeo will do the proximity stuff nicely, for instance, and is free, no less, and the SMS-writing stuff is handled perfectly well by the Apple Address Book. For me, the killer feature of BPE is that I can read past SMSes on the phone using it. This isn't new technology - by far my favourite program in my ancient Psion was an app that would sync my SMS inbox/outbox with the phone, so I could read and write them on a device with a real keyboard. But that was 6 years ago! I've been looking for this app for ages.

Of course, there's a downside. Given the amount of time I spend within bluetooth range of this machine, being bluetooth paired to it at all times is killing the battery life of the phone - it's gone from easily >1 week to about 2 days. That's quite a drop. I still haven't decided if it's worth it to me. Hopefully I'll decide before the software trial period runs out and I have to pay for it..

Props to Brad Choate, who has released Text::Textile 2.03, freeing me from the burden of maintaining it myself. Yay.