Christopher Hinze has written a great Firefox addon that exports bookmarks to OPML 1.0. Unfortunately, OPML is a bit of an anything-goes specification. So although Hinze’s plugin produces valid OPML, it isn’t the same sort of valid OPML that Google Reader expects. Google Reader, in fact, gags and chokes on Hinze’s OPML, and refuses to import it.

The main problem is that the <outline/> element, the basic hierarchical building block for OPML, will take any attributes. What does that mean in practice? Well, here’s what Hinze’s export produces:

<outline text=”Coding”>
  <outline type=”link” text=”Joel on Software” url=”http://www.joelonsoftware.com/rss.xml” />
</outline>

and here’s the result of Google Reader exporting its own store of RSS bookmarks:

<outline title=”Coding” text=”Coding”>
  <outline text=”drupal.org - Community plumbing”
    title=”drupal.org - Community plumbing” type=”rss”
    xmlUrl=”http://drupal.org/node/feed” htmlUrl=”http://drupal.org”/>
</outline>

To a computer, these are fundamentally two different data formats: the URLs are stored in different attributes, and there are attributes on each that either have different values or are not present on the other. Someone did a DTD for OPML: looking at those two apparently analogous fragments above you have to ask yourself why they bothered.

Help is at hand, though. This sort of problem is bread and butter to XSLT, and here’s an XSL transform for converting Firefox OPML to Google Reader OPML. If you have xsltproc installed on your system, you would type:

xsltproc http://www.jpstacey.info/applications/google/ff2gr_opml.xsl bookmarks.opml > fixed_bookmarks.opml

Or download the XSLT—it’s released under GPL2—and run it locally, changing that URL there to a local file location.

One thing to note: the XSLT will remove an outline wrapped around your bookmarks with title “Sage Feeds” (case-sensitive). So you can export that branch of your bookmarks, and the XSLT will strip the wrapper off and you won’t import a load of bookmarks tagged “Sage Feeds”. If you don’t like this behaviour then either rename your Sage bookmark container, or learn XSLT: it won’t kill you.

Matthew Somerville is well known for his accessible takes on rubbish websites: most useful is Traintimes, a layer on top of one of the equally poor commercial British rail sites; most notorious is Accessible Odeon, a fixing of the Odeon Cinema’s website that put their own substandard development to shame so much that in 2004 they had Matthew’s version taken down with legal threats. Remember: just because you’re successful doesn’t mean you’re not stupid.

Recently, Matthew gave an OGN talk on split tickets. These are train journeys which one cross-country train company sells at exhorbitant rates, whereas the components of the journey can be bought separately from the local companies for considerably less. This is what we infer; the rail websites try to keep it all quiet, in the hope that you might lose interest and stump up. The whole headshakeworthy situation is either a stinging indictment of the stupidity of privatizing a rail network and making passengers jump through ridiculous hoops to glean even the tiniest advantage, or a perfect demonstration of how the choice-enriched consumer can leverage capitalism in action: take your pick.

To come to the point: Matthew has set up a wiki where people have been adding the split tickets they’ve worked out in an ad-hoc fashion. Last Sunday I added K’s astonishing 47% saving for Oxford–Cardiff to the end of it, and there’s been little activity since. The page hasn’t got fantastic Google juice—”split tickets” means too many other things—and is just non-dynamic HTML, editable through the wiki but otherwise searchable only by eye.

At the moment, of course, there’s no reason for Matthew to expend any more effort on such a small and barely popular data set. But last Sunday I had a sudden instinctive jolt, to the effect that: more people would be likely to take advantage of split tickets if there was an easier way of looking up other people’s discoveries (my colleagues were recently trying to find split tickets and the wiki page was harder to use than Traintimes, for that very purpose).

With this in mind, here’s my take on accessificating the wiki page: Spliticket. It accesses a cached version of the page, and then using Python’s HTMLParser to hack away at the HTML it returns either a HTML or XML representation of any journey it finds. The idea is that it’s a bit easier to use on your mobile device, and lets you pin down journeys better. I’ve also included an option for searching strictly for the same journey: I’m not sure if split tickets—even returns—always work when the journey is reversed.

Spliticket also supports friendly URLs (inspired by Traintimes itself), so York to Edinburgh in HTML becomes <http://www.jpstacey.info/applications/spliticket/html/york/edinburgh/> and the aforementioned Oxford to Cardiff route in XML becomes: <http://www.jpstacey.info/applications/spliticket/xml/oxford/cardiff/> .

Mostly this was just something to fill idle hours, and also to convince myself of some ideas I’ve had recently about loose coupling, data reuse and open data. Hopefully at the same time (a) Matthew won’t take it as a dig, (b) others might find some casual use for it see it, and (c) a select few might see it as a testament to the ease of freeing information from solid if unsemantic markup. And maybe in six months’ time we’ll all be booking split tickets as a matter of course; by then, of course, Spliticket will probably be gratefully obsolete, replaced by the fully-fledged application that poor, embittered rail passengers deserve. I’m sure Matthew will build it if so.

As I spend the scraps and offcuts of January and February evenings ripping and filing my 2007’s CDs—some of which I won’t listen to very often once they’re fossilized in the collection—I’m aware of a tremendous weight of madeness and invested time and energy on the part of the manufacturers, and of a sort of casual luxuriating in my first-world lifestyle on my own part. You prepare a playlist before me in the presence of my enemies. You anoint my tapeheads with crude oil; my CD tray overflows. So in 2008 I hope to buy as few CDs as possible (none is the target) while also avoiding DRM-crippled music and staying legal.

To this end I’ve been seeking free and semi-free online music—free as in beer, semi-free as in of limited choice—since the new year. So far, outside of bittorrenting (which is obviously of variable legality, depending on what you’re downloading), I’m having some success with Last.fm. Until recently they offered a sort of customized “radio station”, where your input into the of the next track was limited to an intelligent deduction by Last.fm based on what you told it you enjoyed in the past. Now, alongside this potluck service, they’ve just started offering three free streamings of any explicitly chosen track before requiring you to buy the track from a commercial partner.

I’ve yet to try the former service (I think you might have to subscribe to be on the beta wagon: I’ll look into that later), but the latter has so far provided our house with unlimited, free access to a radio station for our very own target market. While such slightly sinister profiling might make it harder for me to discover truly new music, it does at least permit me to expand the boundaries of my comfort zone slowly, and cast a critical eye over my friends’ music preferences, while at the same time giving artists their due and most importantly avoiding physical recordings unless I really want them.

Most commercial support for Linux distributions still consists of monolithic installations, wrapped up with checksums to prevent you tampering with them, and installing themselves on your computer in whatever location and potentially harmful fashion they fancy. Until upgrading to Gutsy this was largely my experience (painfully and often repeated) with such packages as nVidia and wireless drivers, and interesting software that barely gave a second thought to existing Feisty users.

After a spot of Googling I was expecting to have to go through the same palaver with Last.fm’s client, and crossed my fingers that nothing would go horribly wrong. But I needn’t have worried: the Linux client for Last.fm is

1. free of cost, as in beer
2. free of restrictions, as in open source
3. free of hard work, as in a no-sweat installation utilizing the Debian packages and apt package management that’s core to Ubuntu

To install it on Gutsy, you first want to add the GPG key for the repository for security reasons. At a command line, type:

wget -q http://apt.last.fm/last.fm.repo.gpg -O- | sudo apt-key add -

You’ll be asked by sudo for your password. Then, open Synaptic Package Manager (under “System > Administration” in the GNOME menus); then, via “Settings > Repositories”, add the following new third-party repository:

deb http://apt.last.fm/ debian stable

You can then search for the Last.fm widget’s package in the manager (hint: it’s called lastfm) and install it. When you first run it after installation it’ll ask for your Last.fm account, so best have one of those in advance. And that’s it: you’ve now got Last.fm’s widget on your Ubuntu PC.

All of the above is explained briefly on the very URL of the apt repository. Not only that, but they have a free bonus photo of a very cute bunny in case all the apt stuff bores you rigid. Like a TV licence for the Flash version of the BBC iPlayer, all of this is practically worth a subscription alone. As I type, my mouse sits over the very location of the link to do so in a separate tab. I just need to know first: how many more rabbits do I get when I join?

This is wonderful news, especially because the collection is being released through a slightly adapted version of Flickr’s existing website. This means, apart from it being an established interface that millions of people already know vaguely how to use, that you can do all the Flickry things with the photos—dedicated Flickr-heads will hopefully give a more qualified response in due course—and that third-party tools should already be set up to work with the content. The meta information storage won’t particularly excite any Dublin-Core enthusiasts—a block of unstructured HTML in the standard Flickr notes field, plus of course Flickr tagging—but the whole project is still a fascinating experiment, and interesting for even the casual observer of American history. How exciting does it get? More exciting than the World of Mirth Shows?

Thinking offline for a moment, this hopefully presages more leaps forward in MLA culture. One of the first would be to remove the “NO PHOTOGRAPHS” signs from all museums. At the very least such signs could be more honest, and instead read “NO PHOTOGRAPHS; unless our security guards don’t catch you at it, in which case we’ll be blissful in our ignorance. Anyway, in five years time it’ll all be online so we don’t know why we’re bothering, to be honest….” On reflection, I suppose they would need bigger signs.

Seriously: who among us is just itchy and fidgety, waiting for analogue TV signals to be switched off? Who wakes up of a morning thinking, ooh, Whitehaven have to buy Freeview boxes now! with a frisson of glee? Radio switchover would be even more disastrous, of course: whereas TVs normally have a gap between themselves and an aerial into which to insert a Freeview box and convert digital to analogue, most radios are monolithic: receiver, “decoder” and audio equipment all together. That means that the average of five radios per household would be simply landfill material, useless fizzing boxes capable of picking up nothing but static. Future generations will already find a sliver in the geological strata that they can classify as “analogue to digital”; FM switchoff would add a shiny, plasticky laminate to that layer.

Here’s a thought, though: how about a little digital-to-analogue converter for radio? It could be the size and construction of, say, a Fon, and configurable over USB. It would transmit FM in the style of an iTrip to fill perhaps the room it’s kept in, plus neighbouring rooms. It could transmit multiple FM stations at the same time, and even mimic the frequencies of the analogue stations (if they were ever, say, switched off without anyone wanting them to be: God forbid!)

But what if your converter’s chosen frequencies conflict with next door? FM transmission depends on phase coherence, so even if your two boxes were within an electronic ace of each other frequency-wise, they’d still interfere with each other and cause all sorts of beating, flangeing and other dirtily-named auditory confusion. At least when someone passes you on the motorway and their phat BMW iPod transmitter briefly swamps yours then they’re gone quickly, hopefully to come to some sort of messy grief on the next bend. But if you’re stuck near someone who permanently foxes your radio listening—and it would only take maybe three or four such converters for a conflict to be inevitable in the narrow FM band—then what do you do?

Answer: let the converter boxes know how to talk to each other. Let them negotiate their frequency spreads. Let them, moreover, report back to you over what sort of understandings you and your neighbour’s radios can come to. Let them synchronize, if possible, on stations you both want to listen to, strengthening the signal and meaning both of your houses are entirely covered by the FM cloud. Exploit, don’t squash, the wisdom of the group.

OOo arrow libraries are suffixed .soe, and just contain plain-text XML for a full library (so multiple arrows are defined in one file, and only one library can be loaded at once). These libraries sit in your user-specific config directory: to find out where that is, right-click on a line and go to “Line…”, then click on the “Arrow Styles” tab and find the little folder icon for loading new libraries. Click on that, and see where your file explorer puts you.

Arguably it’s easier to either add to or replicate an existing .soe file: the structure is simple enough but there are a lot of namespaces to get right. So open a copy of one of the existing files and have a look inside. It should be structured something like the following:

<?xml version=”1.0″ ?>
<office:marker-table … xmlns declarations… >
  <draw:marker
    draw:name=”Arrow name”
    svg:viewBox=”x1 y1 x2 y2”
    svg:d=”mess of semi-numerical data” />
  <draw:marker … />
  <draw:marker … />
</office:marker-table>

The mess of data up there is the kicker: it looks a bit like SVG path data, but not quite. The reason is that the spacing is often all over the place, so whilst SVG path commands are usually of the form XN, where X is a letter and N a cursor position or a length, OOo frequently seems to prefix commands with numbers. This is actually just an artifact of the format: put spaces in after any number that doesn’t have them, and it reads as a sane SVG path.

So the structure of the arrow can be constructed using the basic, etch-a-sketchy SVG commands, with the following provisos:

• Whitespace is irrelevant (see above)
• OpenOffice always deals with closed paths. If your path isn’t closed with a terminating z SVG command, OpenOffice will do so for you. It will then fill in this closed path with the line colour! The tutorial above gives you ways of avoiding this, but I think it’s a consequence of SVG not having a native understanding of line width.
• OpenOffice has a creative understanding of the standard SVG directions “V” and “H”
• The SVG viewbox parameter is probably best left alone: I think it’s similar to the bounding box on postscript files, creating a window for the vector graphic to sit in.

An example is worth a thousand words, even if that example is made up of a thousand hard-to-read SVG commands, so here we go. The image below is a schematic of a three-pronged many-to-one arrow I want to use. The arrow line continues up the page from the junction of the three prongs.

As you can see, when the parent line is running horizontally (as it appears in the OOo configuration, H actually describes movement along the vertical. With this in mind, the entry for this arrow would read:

<draw:marker
  draw:name=”Arrow name”
  svg:viewBox=”x1 y1 x2 y2”
  svg:d=”m1000 1162
    l 0 -125 -125 0 -859 859
    v-859
    h-176
    v859
    l -859-859 -125 0 0 125 859 859 213 213
    z” />

1. [M]ove the cursor to an x/y position to start (this makes the arrow start at the end of the line)
2. [L]ine: draw towards -v, -h, then along a diagonal
3. [V]ertical line towards -v, [h]orizontal towards -h, [v]ertical towards +v
4. [L]ine: draw a diagonal, then towards -h and +v, then two diagonals in the same direction
5. Clo[z]e the drawing

And that’s it. Fairly simple, as long as you break down your design into SVG primitives first, and realise how OOo is working. There’s some refinements that, as I’ve reverse-engineered much of this, I’ve not been able to go into: the starting position seems to only partly affect the arrow, as it gets centred on the line regardless; the viewbox seems to get ignored. But if anyone can take this any further, than do let me know.

To improve Coldfusion for dedicated XSL programmers, I’ve turned Mark’s one-off function into a more granular component for cacheable, Java-based XSL transformations, called CFJavaXML. You can cache this component from request to request in a persisting scope. You can also compile an XSL transformation once, then store that in a persisting scope too and re-use it without having to keep accessing the XSL file (and compiling it, which can take time). It’ll bring in all its xsl:import references at compilation too, so you needn’t worry about having to keep track of your XSL directory from transformation to transformation.

The component needs no initialization, so create it as follows:

<cfset comp_cfjx = createObject(”component”, “#PATH_TO_COMPONENTS#.cfjavaxml”)>

You can cache this in a persisting scope at this point.

Using CFJavaXML is always a two-stage process, so that you get a compiled transformation object you can store and re-use. In the following example, XSL and XML can be either local file://(/) references, http:// URLs or even strings of valid XML:

<script>
    t = comp_cfjx.XslTextToTransformer( XSL );
    xmltext = comp_cfjx.XmlTransformFromTextAndJava( XML, t [, params]);
<script>

(The transform t is the compiled transform that you can cache from request to request.) Note also where any optional parameters can be inserted, using a params struct.

Depending on the transformation (and how many times you use it) the speed increase of using CFJavaXML has been quite striking: up to ten times with certain transformations. Benchmarking is quite difficult because it depends so much on the intricacy of the XSL: your mileage may vary.

(Thanks to my employers, Torchbox, who’ve given me permission to make this code available, and thanks to Mark for showing how to do it in the first place!)

Similar solutions abound on the web: Tom Taylor has a method involving mencoder; other methods can be found all over the place. However, these all involve a bit of ad hoc command-line intervention, or scripts which aren’t terribly configurable. There are GUI and proprietary commands, but they tend not to offer great support for command-line and therefore scheduled operation.

I’ve knocked together a Python application called rmrip: it’s available in a tar file from http://www.jpstacey.info/blog/files/code/rmrip.tgz. If you unzip this to a directory you’ll find a number of .py files and a config.conf configuration file. Edit config.conf to match your system requirements and stream preferences, make sure rmrip.py is executable, then run it. mp3s should eventually appear in a subdirectory called YYYYMMDD unless you configure the system otherwise.

The application can in principle be run from a cronjob, so it could tick over late at night when everyone’s internet is otherwise nice and quiet. In addition, conversion works via a named pipe, which is a funky way of piping the intermediary, enormous .wav audio file straight into lame, rather than saving it to disk. This does unfortunately restrict the application to non-Windows machines, but it’s a great help for audiophiles with limited disk space: .ra and .mp3 files can be in the hundreds of megabytes for many-hour programmes, but the associated .wav would take up gigabytes.

Current requirements include (please give any feedback on this!):

• mplayer and lame: their locations are configurable
• The subprocess module in Python

Current file types supported:

• Direct rtsp://….ra Real Audio stream links
• http://….ram references to Real Audio streams
• http://….rpm Real Audio playlists (BBC so far only format tested)

To get you started, dave.org.uk has provided information on how to get stream information using a standalone Python program, and also has potentially out-of-date static pages detailing the current BBC streams.

It worked very well with practically no configuration or edits, but there were a few issues with the out-of-the-box behaviour of the import script:

1. Unicode character entities were being escaped in titles, leading to the exposure of the alphanumeric code e.g. “Z&amp;#252;rich” instead of “Zürich”.
2. Whitespace in post bodies is converted to hard newlines by WordPress, and so must be excised to avoid tags being broken e.g. ‘<a [newline] href=”…”>’ becoming ‘<a <br/> href=”…”>’.
3. Multiple hierarchical categories are not supported (a known problem).
4. Although categories are created and posts are linked to them, the number of posts that a category is used in is not incremented and hence the list of categories on the front-end has zero posts for each category(possibly owing to a change between WordPress versions of how this has been handled).

I’ve come up with a number of fixes that I’ve mentioned both to Davis and on the WordPress support forums. As they’ve been greeted with an eerie silence that I’ve found typical of such forums, I’ll put them up here instead.

To fix the first three problems I created rss_to_wp, a Blosxom plugin that, along with the standard interpolate_fancy package, you can use to wrap your title and category processing bits in the RSS2.0 flavour templates. Respectively, this plugin tackles the above problems by:

1. Providing an interpolate_fancy method to unescape entities
2. Normalizing any whitespace in the body of your Blosxom posts to single spaces
3. Providing an interpolate_fancy method to convert a Blosxom-style category path into a set of category tags

You’ll need to change the Davis-recommended story.rss20 template to implement the two interpolation methods. I’ve made a sample available.

The final issue was a more knotty problem, as it was a bug in the script (possibly caused by WordPress’ handling of categories changing over time). It’s easily fixed by adding a few lines to the category-handling part of import-blosxom.php as follows:

294    if (!$exists)
295    {
296        $wpdb->query("INSERT INTO $wpdb->post2cat (post_id, category_id)
297                      VALUES ($post_id, $cat_id)");
298    }
299
300    // JPS' addition - increment count if cat ID exists
301    if ($cat_id) {
302        $wpdb->query("UPDATE $wpdb->categories SET category_count = category_count + 1 WHERE cat_ID = $cat_id");
303    }
304    // End JPS' addition

Exit gracefully: exporting and then importing—transporting?—works well if the two tasks are separable. That way the integrity of the exported data can be checked in its transitory state and any bugs worked out, before it’s imported into the new system. It’s certainly worthwhile backing up the target database for the import, as this lets you preserve any quirks of your target database if you have to dump all the imported data and start again. The standard WordPress install includes a plugin for doing this, but the command-line tool mysqldump is arguably more powerful.

Most Javascript libraries these days are written in a similar way, so it seems to be de facto recognised best practice, but it’s worth showing the anatomy of the simple case so you can build on it rather than having to work out what’s going on from an enormous, somewhat crufty sprawl.

/*
    @description Javascript template
    @createdBy JPS
    @createdOn 2006-10-03
    @notes Standard template
*/
    
MyObjectWithHandyName = {
    
    // Properties
    p: {
        // HTML IDs
        i: { },
    
        // HTML classes
        c: { },
    
        // Something else we might need to reference
        sthg: {}
    
        // Be wary of accidental trailing commas here, as it’s the end of
        // the array and IE doesn’t like a comma at that position
    },
    
    // Methods here - use hierarchy if large object
    DOM: {},
    eyeCandy: { dropdowns: {}, errors: {} },
    httpReq: {},
    
    // Window onload method - instantiates everything
    go: function() {
        alert(’OK!’);
    }
    
};
    
// Now add onload handler to do anything your object needs to do when page loads
    
// Prototype
// Event.observe(window, ‘onload’, MyObjectWithHandyName.go, false);
    
// Mochikit & others
// addLoadEvent(MyObjectWithHandyName.go);
    
// No library?
window.onload = MyObjectWithHandyName.go;

What’re the advantages of the above? Well, first of all, it just formalizes what you’ve already decided to do: that is, to encapsulate all the functionality to do with a certain something in one file. This just puts it all in one object, which you could call DHTML, or iFoo, or GoogleHack, or MyApp. It prevents collisions with standard Javascript functions, library functions you might include etc. Also, if in future you want to know if a function has been defined on a page, but from a different Javascript file, it’s sufficient for a smallish project to check the top-level object exists.

Secondly, the system is very extensible, and tidy with it. If configuration variables go in the hierarchical p(roperties) block at the top of the file, then you can re-use your code by, say, including a second Javascript file on certain pages, that rewrites this configuration. You can even change methods like this, if you know where they’re going to be, in a safe, extensible way. The hierarchy of the whole object means you can nest methods as far down as you want: then, if you find yourself repeating much of the hierarchy, you can use the with(object) control structure to tidy your code:

foo: { bar: { quux: { a: function() {…}, b: function() {…} } } },
    
blort: {
    with(foo.bar.quux) {
        a();
        b();
        a(b(a()));
    }
    foo.bar.quux.a();
}

Thirdly, it’s easy to maintain. Encapsulation and a certain predictability, and the encouragement to make methods small and put them somewhere that makes sense rather than build e.g. sprawling validation methods that, oh, do a bit of browser sniffing as well, and a bit of alert() calling… this definitely forces me to be careful in what I write, and that puts me in a good position to fix things later on.

I can’t say I’ve done any serious testing, but this way of building functionality seems far more robust, and exits more gracefully (on most decent browsers), than other paradigms for Javascript design. It’s possible this is how the whole Javascript community is now coding and I’m teaching my grannies to suck eggs: certainly it’s not how you’d code given ten minutes on Google, so it probably bears repeating.

A few caveats, of course, because there is no silver bullet:

• All function definitions go in MyObjectWithHandyName: nothing outside that apart from the onload to do any actual function calls.
• Any text used more than once (URLs, HTML classes, alert text, repalcement text etc.) goes in p at the top of the script.
• Any event handlers should be wary of what they get as their first parameter, and what the this object refers to: depending on how they get called, that might change from event to element to their hierarchical container in MyObjectWithHandyName.
• Trailing commas: at the end of an associative array, don’t leave a trailing comma as Firefox will quietly ignore it but IE will give one of its typically opaque syntax errors.
• Similarly, don’t omit commas between elements—{ a: {} b: {} } is wrong—as all browsers will die. Easily done, if you’re writing a new method and you forget the go() already exists.

Anyway, give it a go and see what you think.