What if you want the same output for several different data structures? Let’s say we have three people, and we’d like to run the following include, saved as name.html, on all three:

<p>{{ person.firstname }} {{ person.surname }}</p>

Ideally you’d have the three people in a structure called people, and be able to run:

{% for person in people %}{% include “name.html” %}{% endfor %}

But what if the order matters? What if you need account_director to do different things in the template from project_manager? You’d want to send them through as separate data structures, and this:

{% include “name.html” %}
{% include “name.html” %}
{% include “name.html” %}

would just do nothing, because it references person and not account_director. An alternative is to turn name.html into a custom tag, taking arguments; but if the content of name.html were any more HTML-heavy then you might want to abstract it into the include nonetheless, and then you’d just be running a custom tag method, to pass variables to a template fragment, to render the fragment, to pass it back up to the template. It’d be nicer if we could do this with core functionality instead.

Using the latest development version of Django, we can implement basic variable assignment—getting person to reference whatever we want in the template layer—using the with template tag. This means that we can wrap each include tag to produce the required output:

{% with account_director as person %}
  {% include “name.html” %}
{% endwith %}
{% with project_manager as person %}
  {% include “name.html” %}
{% endwith %}
{% with lead_developer as person %}
  {% include “name.html” %}
{% endwith %}

It seems like overkill for our name.html example, but there are plenty of situations—I just found one, hence this post—where it makes sense to factor out code into an include: being able to fiddle with the template context before including it is a lightweight alternative to template tags and a tidy alternative to dumping the HTML right there in the primary template.

[Edit: run Python using Javascript too.]

… This brings us to the second obviously-bad thing that can go wrong with code bases: copy and paste. It doesn’t take very long for programmers to learn this lesson the hard way. It’s not so much a rule you have to memorize as a scar you’re going to get whether you like it or not. Computers make copy-and-paste really easy, so every programmer falls into the trap once in a while. The lesson you eventually learn is that code always changes, always always always, and as soon as you have to change the same thing in N places, where N is more than 1, you’ll have earned your scar….

Code reuse is great, although some languages punish you for it: Coldfusion takes a performance hit on behalf of debug logging whenever it crosses a component boundary, even if debugging is turned off. And a lot of includes scattered across the place can make it difficult to track what’s going on (unless you use those very IDEs that Steve suggests treat code like heaps of dirt to be rearranged). I suppose ultimately it’s about developing an instinct for balancing short-term performance, ongoing maintainability and long-term scalability, and you only learn that by dint of many, many scars.

Now I’d like to start exploiting that behaviour, and I’m going to use the hCard microformat to illuminate its use. Briefly: a microformat is a set of agreed HTML classes used to invisibly encode structured semantic data in HTML; hCard is the implementation in HTML of the vCard specification for “virtual business card” files, using microformat classes. If you mark up people’s addresses using the hCard classes, then it’s possible to automate the conversion from hCard-enabled HTML to vCards, meaning you can click on buttons on webpages and have a vCard served up to you containing the contact information present in the webpage, verbatim, in a format you can put into your address book of choice.

One of the most-used conversion methods—Brian Suda’s X2V, a web service which converts XHTML with hCard markup into vCards and then presents them to the site visitor—uses XSL. In fact, that was what got me thinking about this whole system. Brian’s work is neat, although his own server takes a hit every time someone uses the web service (and it only works on XHTML, not non-XML HTML. What if, I thought, we could get the browser to do it instead; if we could implement template-like functional Javascript?

Anyway, below we find a couple of hCards, culled more or less directly from the Microformats examples page.

Frank Dawson

Lotus Development Corporation

work address (mail and packages):

6544 Battleford Drive

Raleigh NC 27613-3502

U.S.A.

+1-919-676-9515 (w, vm)

+1-919-676-9564 (wf)

Frank_Dawson@Lotus.com alternative email fdawson@earthlink.net

Tim Howes

Netscape Communications Corp.

work address:

501 E. Middlefield Rd.

Mountain View, CA 94043

U.S.A.

+1-415-937-3419 (w, vm)

+1-415-528-4164 (wf)

They look like slightly unstructured HTML, don’t they? That’s sort of the point. But hidden in the HTML are vCard classes. How do we tease them out with Javascript?

Well, there’s a question to be asked before that, I suppose, which is: why would we follow your method, and not someone else’s? What’s so good about functional Javascript? Good question. Well, if every hCard looked like the above, then you could write some completely procedural Javascript to turn it into a vCard. No problem.

But what if the order of the content was changed? The hCard—indeed, microformats in general—has quite a malleable structure, with some classes sometimes appearing on elements inside other elements, and sometimes not. What if there were more telephone numbers and email addresses, and what if they turned up in all sorts of different orders? These are just HTML classes, after all. With procedural Javascript you could start writing switch/case statements to cover every opportunity, and essentially come up with one big unavoidably recursive function. It’ll be hard to structure, hard to maintain and completely unmodular. A document-driven method of extracting the vCard, on the other hand, doesn’t need to worry about all the various different combinations of nested elements: it would just keep one eye on context and process whatever it found. Also, the development cycle could be faster, because templates could be overridden without breaking existing behaviour: just use the template() command to override existing behaviour.

Let’s instead assume you’re following my every word. For this next bit, you’ll need Firefox and Firebug, or to stuff all these instructions into a single file. Otherwise, you’ll have to take my word for it. Firstly, I’ve included jQuery on every page of my blog, so if you’ve got the ‘bug then you don’t have to resort to my insert-JS bookmarklet to squirt it in.

So: first, create the treewalker() and template() functions from part 2. Next, assign treewalker() to body and everything below it:

template("body, body *", treewalker);
template("body, body *", treewalker, "default");

You could restrict this assignment to everything within the .vcard elements, by giving the relevant CSS specifier instead, if there were a lot of content outside the hCards. It would speed up the initial setup phase, but it does complicate the demonstration so I’ve left that refinement out.

Remember we ran the treewalking before? Do that now:

var result = document.body.treewalk();

All being well, you should get a blank string back. Now it’s time to start adding some alternative rules with template(). Try this:

template(".vcard", function() { return "BEGIN:VCARDn" + this.default() + "END:VCARDn" });

Now run the treewalker again. Oh, each hCard has just given you a vCard! An… empty vCard. Isn’t that great? Um. We can add to that, though:

template(".vcard .fn", function() { return "FN:" + $(this).text() + "n" + this.default(); });
template(".vcard .org", function() { return "ORG:" + $(this).text() + "n" + this.default(); });

Now document.body.treewalk() doesn’t just return a vCard for every hCard, but it knows about names and organisations. Also, because we keep including the call to this.default() in our overrides, we still treewalk into any element inside the FN or ORG containers.

What about emails? Well, in the source we can spot an a.email element up there, so let’s give the following a whirl:

template(".vcard .email", function() { return "EMAIL;TYPE=internet:" + this.href.replace(/mailto:/, "") + "n" + this.default(); });

Try running document.body.treewalk() again. Hm. I don’t know about you, but I’m getting an error from that. Ah, wait: sometimes we have span.email rather than a.email. Spans don’t have @href attributes. Well, we could change the above rule and immediately reapply it using template() with no ill effects. But instead let’s keep it in place, and use a more specific specifier to override it just on spans:

template(".vcard span.email", function() { return "EMAIL;TYPE=internet:" + $(this).find(".value").text() + "n" + this.default(); });

Re-run the treewalker. It now finds all email hCard elements and brings them out into the vCards!

I’ll leave you with one more demonstration, for the slightly more complex TELephone field. As you can see above, there are lots of “types” for this field (Work, VoiceMail, etc.) and these sit in child elements of the telephone element. So we need to assign overrides to both the telephone element and its children.

Here’s a rule for the TELephone container:

template(".vcard .tel", function() {
  var t = "TEL";
  // Run defaults to get types where appropriate
  t += this.default().replace(/,/, ";") + ":";
  // See if we’ve got a “value” child
  var val = $(this).find(".value");
  return t + (val.length ? val.text() : $(this).text()) + "n";
});

This method is a bit more complex because we need default() to just get the .type children, and then we reach down to get. Maybe if we could give specifier argument to the default behaviour e.g. default('.type') first, then default('.value')… But that’s a project for another day, I think. Right now, let’s assign a rule to the types children and then run our treewalker:

template(".vcard .tel .type", function() {
  var jQ = $(this);
  return "," + (jQ.attr("title") ? jQ.attr("title") : jQ.text());
});

Result? You should now have Javascript which can produce vCards (currently without geographical address support, as I don’t have time and you might get bored) from the hCard microformat. It’s easy to extend, easy to maintain and, in my opinion, fairly concise. Here’s the whole shebang, less the two framework functions from my previous posts:

// Start with body
template("body, body *", treewalker);
template("body, body *", treewalker, "default");
// vCard wrapper
template(".vcard", function() { return "BEGIN:VCARDn" + this.default() + "END:VCARDn" });
// FN and ORG
template(".vcard .fn", function() { return "FN:" + $(this).text() + "n" + this.default(); });
template(".vcard .org", function() { return "ORG:" + $(this).text() + "n" + this.default(); });
// Email - A and SPAN tags
template(".vcard .email", function() { return "EMAIL;TYPE=internet:" + this.href.replace(/mailto:/, "") + "n" + this.default(); });
template(".vcard span.email", function() { return "EMAIL;TYPE=internet:" + $(this).find(".value").text() + "n" + this.default(); });
// TEL
template(".vcard .tel", function() {
  var t = "TEL";
  // Run defaults to get types where appropriate
  t += this.default().replace(/,/, ";") + ":";
  // See if we’ve got a “value” child
  var val = $(this).find(".value");
  return t + (val.length ? val.text() : $(this).text()) + "n";
});
// TEL types
template(".vcard .tel .type", function() {
  var jQ = $(this);
  return "," + (jQ.attr("title") ? jQ.attr("title") : jQ.text());
});

And that’s it. I hope the approach comes in useful. By next year, you’ll have hCard-enabled pages, with vCard conversion in the browser. Happy Christmas!

So: functional Javascript. Not just functional, but with all the automation of XSL transformations and CSS applications, where you can set it all running and it’ll produce something and hopefully throw no errors. Let’s start with jQuery.

jQuery provides Javascript with a functional framework. Here’s the equivalent of the examples in XSL and CSS, supported by the inclusion of jquery.js:

jQuery("p.intro").each(
  function() { this.style.color = green; }
);

I hope the similarities are clear, and not too strained. Now all three languages do implicit looping over sets of element nodes, and no longer require checks for missing elements; that’s evidence that it’s starting to behave functionally. There’s still a few pieces missing, though. We’d like to be able to iterate over the tree with a set of default rules, and also replace the default rules with our own where necessary.

What would the default rule look like? Well, we can pass around all sorts of objects—this being an object-oriented language—but for now let’s play it safe and follow XSL’s lead, and have each node return the concatenated text returned by all its child nodes. That means that, by default, the whole HTML document would return an empty string. It might be nice to return an array of equivalent objects, or even some transformed tree, but let’s remain old-skool. Anyway, we can always serialize any HTML elements we want to include as text, and then stick them back into the DOM later. There’s probably a way of doing some of these tasks with core jQuery, but as we’re also passing result data around as well as input data, I’m going to step outside the framework (its extension model typically takes a jQuery object in, and returns a modified jQuery object, which isn’t quite what we’re after).

Here’s a default rule: it says “call the default rule (i.e. me) on all my children”. We’ll call this rule treewalker, because that’s what it does. We’ll also assume that we’re going to assign this function as the .treewalk method on each element:

var treewalker = function(i) {
  var t = "";
  $(this).children().each(function(i) { t += this.treewalk(); } );
  return t;
}

And here’s a way of assigning rules to elements. It’ll assign the rule as the .treewalk method unless we specify otherwise.

var template = function(specifier, fn, property) {
  if (typeof property == "undefined") property = "treewalk";
  $(specifier).each(function(i) { this[property] = fn; });
}

It looks a bit clunky, because falling back on the default property means we have to have an if-exists check. That’s to be avoided where possible in functional programming, but bear in mind that we’re still looking under the bonnet (or “hood” if you like), not at the actual functional code. We’ll get to the fully-functional bit shortly.

We’ve got one last bit and we’re done. We want to put the default rule on every element within a certain scope: we’ll assume for now that the whole HTML document body is to be treated; that might be computationally heavy for big documents, but we could change that. We’ve already defined a way of putting rules onto things, so let’s use that to put the treewalker function in as both .treewalk and .default. That way, we have a copy of the method hanging around, that we can fall back on if we overwrite it:

template("body, body *", treewalker);
template("body, body *", treewalker, "default");

That’s it. We’re now ready to pretend our Javascript is XSL. Here’s how we run it:

var result = document.body.treewalk();

Try it. “But that’s just an empty string!” you might, if you were feeling ungrateful, complain. Are you never satisfied? More on this later.

p.intro { color: green; }

And here’s a sort-of XSL equivalent:

<xsl:template match="p[@class='intro']">
  <p color="green"><xsl:apply-templates /></p>
</xsl:template>

They both take place in the context of some generic processor, which rattles through the document executing default rules (XSL: strip out all but text nodes; CSS: apply the plain styles of your browser) unless your program—a list of disconnected rules, really—tells it differently. The combination of (XSL/CSS)+(X)HTML+defaults is thus turned into an explicit script for the browser to run.

So far, so reasonable. But what about the third technology, Javascript? Well, plain Javascript is an object-oriented procedural language. It orders the browser around for a bit, and then when you want to do something to the current page, Javascript manipulates the (X)HTML tree by grasping hold of it with both hands and giving it a tug, using DOM methods like .getElementById(id) and attributes like .parentNode. This procedural approach expects the tree to have a certain structure, or at the very least has to keep checking if the structure has changed and coping with that. This means that the programmer generally has to construct a lot of loops over, say, child elements, and also check for existence a lot. There’s a slight anomaly, in that you could think of the event-driven aspect of Javascript as being functional—it turns the user’s input through clicks, mouse movement and keypresses into browser behaviour, remaining otherwise dormant—but by and large Javascript’s meat is procedural.

There’s two routes you can take at this point. You can either say that, because Javascript is meant to be object-oriented, then the best way to work with it is to augment its functionality and simplify object construction, but ultimately leave it as that: if it weren’t functional, then it wouldn’t be Javascript. Or you can say that, given the advantages that XSL and CSS gain by being functional—a kind of “safety”, some scaleability, and document-driven processing—Javascript might want to have a piece of that too, while sacrificing some of its object orientation.

The first route is entirely laudable, because some problems are object-shaped and some are function-shaped. But, in the spirit of adventure, let’s investigate the second route for a while: pack some sandwiches and get some stout shoes on, and I’ll meet you in my next blog post.

Certainly marketing genius and total orphan Lily Allen, and internationally renowned cuttinge-edge futurologist Gene Simmonds are pulling the sort of pouts you’d expect from them both, and Guy Hands has a look on him like they just cancelled Christmas. But even in these hilariously gurning faces of criticism, and amid the wafting and intermittent atmospheres of genial misunderstanding of how content works these days from the TV and radio monoliths, Radiohead are keeping chipper. Far more so than I’ve ever seen them before, in fact. And when everyone’s on YouTube for free, letting rip with their Thumbs Down webcast, and accepting its reappearance—syndication, if you like—all over the shop very shortly afterwards, was a refreshing change from everywhere else exercising rigid control at the loss of an audience.

But for those of you (like me) who were thinking of taking part in Radiohead’s distribution revolution, yet weren’t keeping an eye on the time:

• The download-only area of “In Rainbows” closed this morning. I just managed to get a copy of the tracks yesterday: I’m sure if you’ve missed out then you’ll all know someone who’s got a copy they can loan you, right? Loan you until the plain old CD comes out at the start of 2008, right?
• Discboxes (40-quid monstrosities that I was secretly waiting till next year to buy) are actually already out and limited stock. I thought from various reportings of the event that they too weren’t going to be on sale till the new year. Get yours while it’s hot.

If there’s demand I bet there’ll be more discboxes, but frankly if Radiohead don’t stamp “SECOND IMPRESSION” over the next lot then I might sue. Actually, if my discbox doesn’t have “A TOTAL W.A.S.T.E. OF CARBON” scrawled over it then I’ll be terribly disappointed.

What I like most about progressive enhancement is that it gives me a way of tackling the age-old problem of turning a design for a client into a workable, accessible page without going completely mental. Before, if I saw a dropdown on the designs without a submit button or some such, I’d think, well, the HTML won’t have a submit button either, so I need to build the Javascript at the same time and get it all to work at once. This led me into all sorts of tangles—each one on its own easy to extricate myself from—and at the end of it I would have a form that would work or look right without every layer of technology in place.

But if you’re thinking progressively at the back of your mind all the time, you can start with an unstyled, unscripted form that works, and then use subtle styling to get exactly where you want to be. Some onloaded Javascript can give you a body.js class with which to hide unwanted submit buttons. At the end, if nothing else, the form still submits when Javascript is turns off, and it looks OK because you checked at the outset.

Progressive enhancement, ironically for the amount of work it implies, actually places the least stress (and maintenance workload) on the developer, and makes them learn a thing or two about layering and modularity at the same time. And as Steve points out, in this model there is no degradation: just the near-hotswapping of a range of technologies on top of a sturdy markup language that has been around in various forms for over a decade.

REST plays the same role as open source and open APIs: It eliminates tooling and vendoring as artificial barriers to adoption.

Interestingly, a corollary to this was brought up at Barcamp Brighton this weekend. During Gareth Rushgrove’s talk about REST and Nabaztag, a chap whose name I’ve again forgotten (although I’m sure someone like Fatty will enlighten me) pointed out that much of the push of SOAP is coming from the vendors, because the vendors make their money from selling tools, and REST development needs very few tools, most of which are free.

Undoubtedly there’s a set of problems that REST finds hard, but this truism is extended by SOAP vendors to the hard-to-prove (but also hard-to-contradict) claim that it’s a larger set, or a set more pertinent to enterprise solutions, than the set which SOAP finds hard. It convinces the consumers, because intelligent data mining and storage has always been a difficult problem, and a simple solution like REST feels like underkill for the job in hand. They let you confuse libre and gratis, the vendors point out (I see them sitting on the consumer’s shoulders with tridents at this point): so where’s the hidden cost of this free lunch?

(hat tip to Simon Willison)

You wouldn’t happen to have a video you could link to of that person who showed you Twitter being valuable. Why? Because I (still) don’t get it…

I know how he feels. I still don’t get MySpace, and according to Rick Rubin it’s already on its way out. No wonder I’m starting to feel old.

It took life events to really make me understand Twitter; it took actually fiddling around with it and trying to use it. A more committed Luddite would have bowed out long before I did, and I could hardly blame them. During the last Glastonbury festival, I twittered the events happening around me at a much smaller festival near the Ascot racing ground. My friends at Glastonbury twittered back. Later on, Janet McKnight was to exclaim: “oh! I thought you were making your festival up to be part of the gang.” That aside, as we were all telling each other what we were up to at these large, disparate, staged events, it dawned on me that we’d basically built an add-on to the Glastonbury Festival, without having to deal with the odious Michael Eavis, and I thought: I get Twitter.

That’s a specific instance, of course, but in a more general sense Twitter is providing a narrative version of where you’re at, a real-time way of positioning yourself—and reading off the positions of others—in terms of what you see, hear and smell rather than in terms of a longitude and latitude. It takes all those narratives and binds them together: or not, if you don’t want.

So maybe they should market it as the alternative to an address book: “if you liked knowing where your friends are currently living, you might also want to know what they’re currently up to.” Or how about: “Twitter: the RFID tag for the soul!” Actually, that sounds a lot more sinister than chicken soup.