ajax

Recently my wife and I were trying to work out why she couldn't submit her address details to a website, even though I could. As we watched her behaviour in filling out the form, we encountered error after error: or rather, exceptional circumstance after exceptional circumstance. And it was clear that very few of the circumstances had been considered, that error handling was the absolute bare minimum, that the form was set up to be almost a trial to use. The postcode lookup part of the process was probably the source of the most unhandled exceptions: difficult if not impossible for the power user to flow through; unwieldy for the standard user; of almost no benefit at all to the web newbie.

People still think of their workflows on the web like they're workflows. Over here there's the start; over there is the goal; somewhere in between there might be some intermediate stages, but ultimately you go from over here to more or less over there eventually. It comes as something as a shock to most people that their beautiful webform does not encompass a workflow: the web has holes all over it; the user is a ball bearing and your application is a pinboard:

Above we see a slightly clumsy metaphor for your web application. The end point of your own particular "workflow" isn't even something visually obvious like the shark's head. It's, oh, let's say that small metallic gate in the bottom right with the red doors (luckily for you) open. The best you can hope for is that the user caroms through your website hitting as few pins as they have to and ends up in one of a trillion end points, that they don't close the browser, or reach a form error, or silently lose their submission, or navigate elsewhere in irritation.

In fact, it dawned on my wife first, and then gradually on me, that postcode lookups are not intended to directly benefit the user filling them in. Instead, they're meant to force the user---remember that phrase---to provide a canonical address, and not the address. That is, the user comes to your site with an opinion about where they live and limited good will about your "product", and the postcode lookup is a mechanism for forcing them to discard the former, while the application as a whole is trying hard to get them to keep hold of the former.

Good luck with that.

Richard Rutter figured out the dirty secret behind postcode lookups---that they're not for the user---long before my wife and I. In order to mitigate this natural tension between forcing the user and keeping them happy, he's done a sizeable chunk of work to condense the postcode lookup pattern here. Along with a quite lively and informed conversation in the comments, this post nails much of the core of the pattern that lookup needs. Much of what's frankly miserable about using a postcode lookup is indeed tackled there, but there's an important omissions that I think needs dealing with. Roughly speaking, that is: never force your users to pretend to be someone they're not.

As an example, consider Spotify. Since inheriting a slightly ropey eMac, I've been able to listen to Spotify, and I like it. I think Spotify is a gamechanger in the field of streaming music. I've heard albums on Spotify that I would never have bought. And yet: I would never consider purchasing Spotify Premium. The obvious barrier for me is that I use Linux, and there are no native Linux binaries for the Spotify desktop client.

People keep telling me that Spotify runs really well on the Windows emulator WINE. I'm sure it does. But that misses a more fundamental point: if something wants me to enter into a relationship with it, commercial or otherwise, I should not have to pretend to be a demographic I'm not in order that the relationship can be properly fulfilling. I'm not a Windows user, and it's an affront to a paying customer to expect them to make out that they're a type of user that they're not if they want to buy your stuff. More consisely: I don't take offence at the interface requirements; I take offence at what they imply about the respect for my needs as a user.

With that in mind, consider the first decision block in Richard's workflow:

does the user know their postcode?

As this is the first pin the user's pinball hits, then this is the one that alters their final resting place the most. This is the critical pin. And what does it tell me, a user who knows his postcode but also knows his address, and doesn't want to bother with lookup? It tells me that I have to pretend to not know my postcode if I want to be in a situation where I don't have to put it in. I have to play games with the application, and mask my true intentions. No, the postcode lookup system must allow for users who simply do not want to fill out their postcode. Postcode lookups should therefore begin with a simple choice of two buttons by a traditional form label. The buttons should not make any assumption about the user's reasons for choosing either route:

Address:  [LOOK UP POSTCODE]
               [JUST LET ME TYPE MY ADDRESS]

When you press the first button, both buttons should still remain on the page: the user might decide they wanted to press the second one after all. In fact, as we're probably using AJAX here, the minimum necessary modification to the form is just to add a postcode box (and to move and maybe change that button) like this:

Address:  [ postcode ]   [LOOK IT UP]
               [JUST LET ME TYPE MY ADDRESS]

You should still present the user with the ability to change their address. The "speed bump" of having to press the button is what works in your favour, is what gets you access to canonical data; anything more than a bump will run the risk of walloping the user right off your pinboard.

If at any stage the user clicks on the second button, the webform should then change to this view:

Address:  [ Address, either as a set of textfields or as a textarea ]
               [ postcode ]   [LOOK IT UP INSTEAD]

For simplicity, I'm almost tempted to drop lookup button altogether at this point: the user has made their choice, after all. But you should never make it difficult for users to go back in a workflow, especially when (almost certainly) the browser back button will have been disabled by these shenanigans.

In comment 10 on his blogpost, Richard agrees with the idea of one big textarea for the address: possibly even including the postcode in that textarea! Again, the simplicity is appealing. You could do all sorts with this to make the user's experience easier: regex matching behind the scenes would retrieve the postcode; the address could even be automatically split into lines rather than setting real estate aside for the user to split them up themselves. It sounds great, but it's not an established pattern, and I think a lot of users---especially power users---would mistrust it. Better to go with Address 1, Address 2 etc. even though from a data perspective they're a horror, slightly improved---but, again, made more complex for the user---by labelling the last address line as "town". But this last detail is up to you. Do some A/B testing. See how it goes.

Richard's workflow, with the addition of the basic prototypes above, permits us to move towards as usable a system as postcode lookup will ever be. Usability means the least number of pins on your pinboard, and exactly the right pins, the ones that nudge and tilt the user just enough. And so we end up with a system that still satisfies your original remit---to nudge the user towards using your shiny, expensive, time-consuming, postcode lookup service, with all its concomitant costs in development and maintenance---while catering for the users who simply do not want to, who will never want to, and who will actively object to your site giving them short shrift if they try not to use it.

I'll make a prediction here, that the users who try not to submit a postcode will tend to be the users you want: the digital natives, the users in flow, the people who will buy ten things from a polished user interface without even stopping to think about it. When they reach that very first pin on the board, they briefly want to be your application's friends on the web. Your application should consider itself honoured. The least it could do in return is be polite.

The past couple of evenings I've been away from a computer and so not free to consolidate my copious notes all the interesting stuff that happened at Google Developer Day on Tuesday. I should start by saying that all the organizers, especially Liz Ericson, should be proud for planning and running such a fun event, and so smoothly. I think everyone had all the free food we could manage (with great veggie options, which is a mark of something), and sweeties and fresh coffee were distributed around the conference area. Lots of beer in the evening too, and a cute little Lego-like USB stick-man to keep me company on the bus home.

The venue was also great---if a bit too big for getting from room to room in a hurry---and the security and catering staff were some of the nicest I've ever encountered. Apparently we weren't allowed to mention the name of the enormous sports stadium west of London that we all attended, but if I say it rhymes with Bembley and looked like it still had the builders in and scaffolding up then you'll know where we were.

As at any conference the talks were a mixed bag, although all the ones I went to were definitely worth listening to. Mano Marks covered developer optimizations for Google App Engine with aplomb and a cheeky grin; Dion Almaer provided what might have been a "will-this-do" overview of the "state of AJAX", which actually became a more interesting discussion of the state of browsers in general, and how they might move towards full support of RIAs; Nimrod Talmon's discussion of Google Visualization, on the other hand, suffered from being immediately after lunch. I could've managed that level of tech details after a nap to sleep off my sugar slump, but I think he should really have been in the pre-lunch slot. We were all jamming on simple carbohydrates anyway, so it's not as though we would have been itching for butties. The content of his talk was still intriguing, though, and it's good to know you can at least consider handing this sort of thing over to a third-party engine if the client wants it but the poor webserver doesn't.

The lightning talks at the end of the day were as shambolic as these things tend to be. Liz said that a lot of people had expressed an interest before the event, but if that was the case then they were all being very optimistic about what they'd actually be able to accomplish before the day, as we only had maybe half a dozen in what sounded from Nick like the most popular slot of the day. Anyway, it was fun to break out a bit, but my laptop wouldn't detect the projector, and then the gist---the actual, pivotal slide---was lost in conversion to whatever it was it was converted to on a Mac. I may do it at a later Geek Night, if I can stomach abusing my position as organiser.

Google very much held off on any go-team antics, hoping that their products would speak for themselves. That was slightly marred by them making it quite clear to everyone, that they did indeed hope that their products would speak for themselves, but at least there were no overt recruitment drives, and rubbishing of the competition was limited to the pretty much warranted chastisement of over-prompting your user and making them numb to modal dialogues, behaviour of which XP is particularly guilty.

Within the reasonably frank and open exploration of their services, however, were occasional notes of discord. Whenever any kind of business case was mentioned things would go quiet, and quite clearly those things weren't up for discussion, which I think is a shame. Google doesn't need to pretend that it's everyone's best friend to have our respect: it's public knowledge that it has shareholders, and I think developers the world over can admire it in that context if we know where we are with it.

Transparency engenders trust, and I'd take comfort from someone replying to a query about the bottom line by saying "well, I asked my boss, and he said that we give this stuff for free because it positions us as an influential brand when it comes to advertising revenue, trusted partnerships, GSAs, Google Minis...." Hell, if I'd been fed on those ciabatta rolls and caramel shortbread slices beforehand then I'd probably be asleep by that point, so it wouldn't matter what they admitted after the fold.

Those occasional moments aside I had a grand day out. I'm itching to start working on App Engine now, and putting into place some of the weirder code patterns Mano mentioned. I've also already had a play with Gears following Dion's talk, and it seems to turn Google Docs into something close to OpenOffice. But before any serious coding, I have to wait for the blood blister on my finger to go downfirst, which came from playing Guitar Hero at the end of the evening. How rock and roll, eh? The young Googler I played against trounced, me: as you'd expect: deep down I think Google's just as cool as it wants to appear to be on the surface.

Following my recent success with putting a Flickr feed on my website’s front page is the conversion of this to an all-purpose feed reporter, where RSS/Atom flavour and feed specifics are dealt with by Javascript associative arrays of functions, keyed on both variables respectively.

If you’re lucky then the feed should wait a bit while it loads the XML via Javscript’s XMLHTTPResponse() code (querying proxies all on my website for foreign feeds like del.icio.us). Then it will report the first feed it finds, while still loading other feeds in the background. Every 15s or so it swaps to a new feed.

After writing it easily in Firefox, I found that a number of drastic changes were required for IE, as follows:

1. IE doesn’t seem to have great support for the IMG element’s width or height properties, at least before the image is rendered on the page. For the Flickr feed, if imgElem.width == 0, I’ve had to employ some horrible regular expressions on imgElem.outerHTML (which isn’t supported by non-IE browsers, so it’s all a bit of a hack).
2. IE doesn’t support assigning arbitrary properties to elements. I was using these to smuggle a collection of titles and descriptions (keyed by date) into a function and that of course failed. You can use setAttribute() for string-like properties, but not objects, which I’ve had to pass as parameters or in some reserved scopes.
3. IE is very unforgiving about character encodings, behaviour which I haven’t been able to work around yet. Some of the programs that create my feeds (specifically Blosxom) aren’t very Unicode-wise and hence will happily produce invalid content. Firefox is very forgiving of this (which may or may not be the right XML way to do it, but is certainly a very patient way of dealing with web content generally). I’m currently working on my proxies to sort this out somehow, even if I have to filter everything down to the ASCII character set.

The Javascript is a single, monolithic file which relies on a beta-release version of Mochikit, and is available at http://www.jpstacey.info/index/javascript/index.js. Feel free to scrape it and re-use anything you find interesting there.

A number of posts are currently in stasis waiting for me to actually finish them, but I thought it was worth mentioning the selection of my most recent Flickr photos that now graces my homepage. While the rest of that page awaits serious styling and content work, I’ve dilly-dallied by creating this bit of eye-candy.

The loveliness of this slightly hackish system is that it uses AJAX (with help from MochiKit) to fetch an RSS 2.0 feed, so it doesn’t require an understanding of Flickr’s huge quasi-REST API. Also, because one can guarantee the RSS feed is good XML, the responseXML property of the XMLHTTPRequest object can be trusted. This means you don’t need to do any string-based munging of the returned data to transform it into the HTML seen on the page: there’s an XML DOM already set up for you. It also means that the method could be adapted to other RSS feeds, in principle.

The major difficulty was getting the Javascript to request a URL from a different server from the one that the current page was on (quick summary: you can’t, because of Javascript’s security rules). But it took two minutes, one and a half of which were googling, to set up a proxy of this feed in Perl using LWP::Agent.

There’s other minor problems to be overcome—the first paragraph of the RSS description feed is a rather trite “username posted a photo:” so naturally I got rid of that, and the images are actually shown half-size because it’s easier to change the <img /> tag’s width and height than it is to try and work out the dimensions and locations of the real thumbnails—but it works. And all in under an hour’s programming.