Christopher Hinze has written a great Firefox addon that exports bookmarks to OPML 1.0. Unfortunately, OPML is a bit of an anything-goes specification. So although Hinze’s plugin produces valid OPML, it isn’t the same sort of valid OPML that Google Reader expects. Google Reader, in fact, gags and chokes on Hinze’s OPML, and refuses to import it.

The main problem is that the <outline/> element, the basic hierarchical building block for OPML, will take any attributes. What does that mean in practice? Well, here’s what Hinze’s export produces:

<outline text=”Coding”>
  <outline type=”link” text=”Joel on Software” url=”http://www.joelonsoftware.com/rss.xml” />
</outline>

and here’s the result of Google Reader exporting its own store of RSS bookmarks:

<outline title=”Coding” text=”Coding”>
  <outline text=”drupal.org - Community plumbing”
    title=”drupal.org - Community plumbing” type=”rss”
    xmlUrl=”http://drupal.org/node/feed” htmlUrl=”http://drupal.org”/>
</outline>

To a computer, these are fundamentally two different data formats: the URLs are stored in different attributes, and there are attributes on each that either have different values or are not present on the other. Someone did a DTD for OPML: looking at those two apparently analogous fragments above you have to ask yourself why they bothered.

Help is at hand, though. This sort of problem is bread and butter to XSLT, and here’s an XSL transform for converting Firefox OPML to Google Reader OPML. If you have xsltproc installed on your system, you would type:

xsltproc http://www.jpstacey.info/applications/google/ff2gr_opml.xsl bookmarks.opml > fixed_bookmarks.opml

Or download the XSLT—it’s released under GPL2—and run it locally, changing that URL there to a local file location.

One thing to note: the XSLT will remove an outline wrapped around your bookmarks with title “Sage Feeds” (case-sensitive). So you can export that branch of your bookmarks, and the XSLT will strip the wrapper off and you won’t import a load of bookmarks tagged “Sage Feeds”. If you don’t like this behaviour then either rename your Sage bookmark container, or learn XSLT: it won’t kill you.

There’s a brief README.txt in the zipped archive linked above, but the theme’s main features are:

• XHTML compatible (in core theme files)
• Minimal, semantic markup
• No sidebar
• Excerpts on archive and category pages
• Implicit RSS feeds: the only orange icon is in your browser chrome
• Adaptive top navigation
• Separate pages for archives, categorisation and blogrolls
• Next/previous rel links in header
• Support for special pages e.g. blogroll, tag cloud

The todo list includes:

• Implement my Blogthis! plugin, while trying to keep minimalist
• Unobtrusive hiding of elements, using jQuery
• Improve styling

The theme is in a fairly alpha state. The PHP is fairly straightforward, apart from some neat theme functions, but don’t blame me if everything goes bang.

Alsa seems stable and graphics support is present from installation onwards. Enabling fancier 3D compiz effects requires the nvidia-glx-new package; compiz spots this, however and prompts for installation. All very smooth. Wireless works; my VoIP headset works; but I haven’t yet tested Bluetooth.

The only problem was in upgrading from Gutsy: my previous peregrinations had rendered my hybrid distribution shafted and incapable of upgrade. This isn’t a problem, though, if one has installed the /home directory (and in my case the /music one too) on a separate partition: the Ubuntu Live CD will blat the root partition with Heron, but leave the other partitions alone if you so require. Don’t resize any of your partitions during installation, though, or you’ll lose everything. Everything!

It’s called Straight Edge, and I’ve switched to it today. Once I’ve finished alpha-testing it I’ll write more about it, and offer it for download if anyone’s interested.

That works just fine for most purposes, but we may have to bring in content from not just outside the core Drupal tables but outside the core database, and even on a remote server through webservices. To this end we’ve decomposed the core user module’s user_load() and user_save() functions. This helps us understand better both the workflow and at what points in it our own code can motor into life, query all those extra resources (or set those queries in motion), assemble the rest of the user++ object, and then hand control back over to Drupal.

For those who don’t know much about Drupal, its core has a hook-based API structure. At certain points in its workflow, it checks all the modules for functions following certain naming conventions (typically the module name followed by the hook name e.g. mymodule_init on response startup, or mymodule_block to return details about the module’s support for Drupal page furniture). Any matching hook functions are called in the order defined by module weightings, and then page processing will generally continue: you can crowbar a grind-to-a-halt exit() in your hook, but it’d be unwise, as you can never be sure what tidying up Drupal might need to do after your hook. Outside these hooks, your code has little control over Drupal’s core functioning, unless you stub out entirely the bits of core you need yourself, and get your request to use those bits instead.z

Because of the way they let your code tag along with Drupal’s powerful core, hooks are essential to developing modules in the most Drupalish way. With that in mind, here are flow diagrams of the three basic aspects of user functionality—create new, load, save existing—lifted straight from examination of the code:

• Loading an existing user with user_load
• Creating a user: saving a new user with user_save
• Updating a user: saving an existing user with user_save

Although you have to save a user before you can load them, I’ve put this functionality first in the above (admittedly unordered) list. There are two main reasons for this:

1. user_save actually calls user_load a number of times, once or twice, to “refresh” the user object
2. user_load is a more primitive function and so bears examination first

Stripped down, user_load consists of: querying the database for a core user record matching the search criteria; returning this and the extended profile data; unserializing a free-data field and inserting it into the user object; discovering user roles; triggering hook_user('load') and returning the object (or boolean false, if no user found).

What this reveals (which I didn’t realise before) is that the anonymous user is in the Drupal users table, with ID=0. Otherwise, searching for this user would return no records, and the anonymous user object could not be instantiated. You could therefore attach rich data to the anonymous user, if you were in a hacky mood.

The two user_save workflows are fairly similar. Creating a user means obtaining an ID from the database: because some MySQL providers have poorer feature sets than others, referential integrity is ensured at the application level rather than the database level. In place of obtaining an ID, user update calls hook_user('update') to pre-process the user. Both workflows then set aside special fields, such as the user’s password, user roles and any profile fields managed by that module (determined from user_fields()). Then they save this data into the database in slightly different orders, with user creation calling hook_user('insert') early on, and the update procedure calling hook_user('after_update') much later in the process, just before determining the external authentication mappings (e.g. OpenID) and returning the user object.

What does this mean for us? Well, we’ll want varying amounts of data to piggyback on the core user object, so we have somewhere to cache it. Ideally this data won’t be summoned—brought out of the distributed data ‘cloud’—on every request/response cycle, so we’ll need to do some local cacheing, but not so much that we’ll get out of synch with the cloud (or that we’ll duplicate sensitive data). We think that, given the pair of hooks in user_save for existing users, we’ll have just enough leverage to do this: the first hook will effectively “tear down” our extra data, so we can do what we want with it, and store it somewhere temporarily; the second hook will “set up” the user for the rest of the request, by putting all that data back in. The existence of user_load within user_save complicates things somewhat, but at the same time it gives us some more wiggle room, because each call to that function fires another hook.

A Drupal hook is worth a thousand lines of module code, but they’re still a bit few and far between for some workflows. Hopefully the accompanying images will help anyone reading to find them, and ditch those thousand lines before they’re even written.

First, a bit of background. Postcode Anywhere provides a per-lookup paid webservice, converting UK postcodes into valid house names/numbers, streets, towns etc. It allows you to accept a user’s submission of a postcode to e.g. a signup form, then present them with a list of sample addresses, rather than them having to fill in all of their address details.

There’s only really one method exposed for the PA functionality, and it takes a text postcode and returns an array of results:

$data = pamb_pa_get_bypostcode($postcode);

MailBuild and CreateSend, on the other hand, provide a way to store email lists and send newsletter campaigns to subscribers. However, our client also wanted to use it to store snail-mail addresses, so they could use it for both requests for printed matter and also to keep subscribers up to date via email. It was possible to implement this using custom fields, of which the service permits around a dozen: address line 1, address line 2 etc.

There’s an existing Mailbuild module, which technically does the trick, but it’s overkill compared to what the client wanted. pamb’s integration is essentially just “push”: it passes the results of a form submission to CreateSend, by using a template (the one you’ve moved to your theme directory) to “theme” the data as an XML packet. If there are no problems then the system returns nothing; otherwise, it returns a structure of data including the HTTP response and the SOAP packet from the request.

Here’s some sample code:

/* Send SOAP packet */
$no_probs = pamb_mb_send_soap($form_id, $form_values);

/* If OK, set a generic message and redirect: the redirect will pick up
on the generic message and know all has gone well */
if(!isset($no_probs)) {
  drupal_set_message(t(AUTO_SUCCESS));
}

/* If not OK, $no_probs contains the diagnostics */
else {
  /* do something else */
  drupal_set_message(t(AUTO_ERROR));
}

For now, for want of a better name, the module is called “pamb.” And [edit] the code is now available on Sourceforge under the GPL: that should make it compatible with Drupal’s core licence.

1. Download pamb: currently only available in subversion, but clients exist for Linux, Windows and OSX.
2. Unpack it into your modules directory
3. Edit the defines at the top of the file for your own Postcode Anywhere or MailBuild/CreateSend accounts.
4. Stick the pamb_soap.tpl.php in your theme directory
5. Switch on the module in the admin interface

You’re now ready to start developing with its handful of functions. Feedback more than welcome, as given Drupal’s minimal support for mailing-list functionality in core it’d be good to make this into a fully-functional module for similar third-party contact services, and even port it to 6.x.

[edit 2008-06-17: now hosted on Sourceforge]

Matthew Somerville is well known for his accessible takes on rubbish websites: most useful is Traintimes, a layer on top of one of the equally poor commercial British rail sites; most notorious is Accessible Odeon, a fixing of the Odeon Cinema’s website that put their own substandard development to shame so much that in 2004 they had Matthew’s version taken down with legal threats. Remember: just because you’re successful doesn’t mean you’re not stupid.

Recently, Matthew gave an OGN talk on split tickets. These are train journeys which one cross-country train company sells at exhorbitant rates, whereas the components of the journey can be bought separately from the local companies for considerably less. This is what we infer; the rail websites try to keep it all quiet, in the hope that you might lose interest and stump up. The whole headshakeworthy situation is either a stinging indictment of the stupidity of privatizing a rail network and making passengers jump through ridiculous hoops to glean even the tiniest advantage, or a perfect demonstration of how the choice-enriched consumer can leverage capitalism in action: take your pick.

To come to the point: Matthew has set up a wiki where people have been adding the split tickets they’ve worked out in an ad-hoc fashion. Last Sunday I added K’s astonishing 47% saving for Oxford–Cardiff to the end of it, and there’s been little activity since. The page hasn’t got fantastic Google juice—”split tickets” means too many other things—and is just non-dynamic HTML, editable through the wiki but otherwise searchable only by eye.

At the moment, of course, there’s no reason for Matthew to expend any more effort on such a small and barely popular data set. But last Sunday I had a sudden instinctive jolt, to the effect that: more people would be likely to take advantage of split tickets if there was an easier way of looking up other people’s discoveries (my colleagues were recently trying to find split tickets and the wiki page was harder to use than Traintimes, for that very purpose).

With this in mind, here’s my take on accessificating the wiki page: Spliticket. It accesses a cached version of the page, and then using Python’s HTMLParser to hack away at the HTML it returns either a HTML or XML representation of any journey it finds. The idea is that it’s a bit easier to use on your mobile device, and lets you pin down journeys better. I’ve also included an option for searching strictly for the same journey: I’m not sure if split tickets—even returns—always work when the journey is reversed.

Spliticket also supports friendly URLs (inspired by Traintimes itself), so York to Edinburgh in HTML becomes <http://www.jpstacey.info/applications/spliticket/html/york/edinburgh/> and the aforementioned Oxford to Cardiff route in XML becomes: <http://www.jpstacey.info/applications/spliticket/xml/oxford/cardiff/> .

Mostly this was just something to fill idle hours, and also to convince myself of some ideas I’ve had recently about loose coupling, data reuse and open data. Hopefully at the same time (a) Matthew won’t take it as a dig, (b) others might find some casual use for it see it, and (c) a select few might see it as a testament to the ease of freeing information from solid if unsemantic markup. And maybe in six months’ time we’ll all be booking split tickets as a matter of course; by then, of course, Spliticket will probably be gratefully obsolete, replaced by the fully-fledged application that poor, embittered rail passengers deserve. I’m sure Matthew will build it if so.

Last.fm’s web interface isn’t optimized for the sort of “media station” use we put it to, and their downloadable widget won’t work on the antiquated version of OSX that I’ve just about managed to squeeze onto dear old Purpurea. So today I’ve spent some time writing a bookmarklet to allow me (and you, dear reader) to hook into Last.fm and its Flash player’s behaviour, to update a larger-view popup window with the necessary information.

Here’s the bookmarklet—which with attention to catchy names I’ve called LastJS—with spaces added for legibility:

javascript:void(function(){
  var s1=document.createElement(’script’);
  var s2=document.createElement(’script’);
  s1.src=’http://www.jpstacey.info/applications/lastjs/jquery.js’;
  s2.src=’http://www.jpstacey.info/applications/lastjs/last.js’;

  var h=document.getElementsByTagName(’head’)[0];
  h.appendChild(s1);
  h.appendChild(s2);
}())

(As the above suggests, the bulk of the code is at http://www.jpstacey.info/applications/lastjs/last.js.)

And here’s what you need to do to use it:

1. Add this link to your browser favourites (IE7) or bookmarks (everyone else): LastJS bookmarklet.
2. While on a Last.FM “now playing” page e.g. for The Hold Steady, click the bookmarklet.
3. This will pull an instance of jQuery and LastJS off my own server, and add a “click for big” link under the title.
4. Click on the link. A new window will open, containing the artist information (track information is a bit obfuscated until the Flash player loads the next track).
5. Wait. As the track changes, the new window will update with the new track information.
6. The text is purposefully large yet not too large. If you’ve got a big monitor you can make it even bigger by increasing the base font size in your browser. To do this in Firefox press CTRL-+ (or Apple-+); I think it’s something similar in IE7.

Feel free to poke around at the code, and distribute it (under GPL, for what it’s worth: I tried looking into precisely which variant I wanted but my knees started to ache). Here are some of the problems it has to solve:

1. Last.fm uses Prototype, and so needs jQuery to restore the $ variable using jQuery.noConflict().
2. Events in the Flash player trigger functions in the window scope: by putting a new method in for a given event handler you can hook into Flash events. Care has to be taken, though, not to wrap and re-wrap too often—the bookmarklet might after all be re-clicked—or you might provoke a recursion warning.
3. Popup blockers mean that you can’t always guarantee the window will exist, and the code has to check accordingly.
4. Firefox seems to blank the contents of the popup window shortly after it’s opened, so any writing of text to it has to be delayed, lest it be lost by this clear-out.

All probably needs work, but feel free to have a play. It doesn’t circumvent any DRM or anything, of course, but I’ve no idea whether it invalidates terms of use, so caveat fructor.

(With any luck D6 still contains my three characters of contribution fixing OpenID autodiscovery.)

Tom Dyson is starting a CRAG for the Wychwood area: that is, more or less anywhere within around fifteen miles of Charlbury in West Oxfordshire. If you’re interested, and you live in Witney, Eynsham, Burford, Chipping Norton, Kidlington, Woodstock, Stow-on-the-Wold, Hook Norton… then come along. It might change your way of life.