standards

Recently a blogpost discussed Jen Lampton's superb conference session: "Wordpress is better than Drupal: developers take note." The author said that they really liked the session, but

• Wordpress isn't actually better than Drupal
• Because you can't compare Wordpress and Drupal
• So the usability issues arise from a false comparison
• And here's what's going to solve our problems instead

In the first paragraph or two it managed to simultaneously both praise Jen's talk and completely deny the core message of it. I think that highlights a number of trends in the way that the Drupal community as an aggregate (though by no means as a whole) tends to think about usability versus technology. Actually, I think the communities behind most large open-source projects behave this way, and it's ones like Wordpress or Ubuntu that are actually unusual. But I think it's worth talking about this phenomenon in the context of Drupal.

Apples and oranges

Anyone who argues a point of view can be sure they've ruffled some feathers, when someone else complains that they're "comparing apples and oranges." It's a definite sign that the complainer doesn't like the problem that's been raised, and instead wants to treat an informal analogy as if it were a keystone in a tense, logical structure; then attack the analogy instead. It's like hurling around cries of "ad hominem". It doesn't really prove or solve anything. Apart from anything else, what if my point is to compare apples with oranges?

Whatever you compare Drupal to, whatever your motives, whatever important (often fundamental) issues you want to highlight about the way Drupal does things... eventually you'll have to compare Drupal to something that isn't Drupal. You will have to compare the apple to some other fruit. That's the only comparison worth making. Are we just going to forever compare Drupal 5 to Drupal 4.7, Drupal 6 to Drupal 5, and gradually Drupal 7 to Drupal 6, purely as a teleological exercise in patting ourselves on the back? Of course not! And however close the something else might be to being Drupal, there'll be some slight difference, which will inevitably provide an opportunity for hairsplitting. Maybe we could call it fruitsplitting instead, because that difference in variety of fruit can be used as an excuse, if not a reason, to dismiss your broader motivations in making the comparison.

"X isn't a CMS." "Y isn't in PHP." "Z isn't aspect-oriented programming." "P is aspect-oriented programming, but it's intended to be purer than Drupal, with pointcuts." "Q doesn't have a genial tousle-headed giant as its benevolent dictator."

Apples aren't oranges, but then some apples aren't the same as other apples, either. Apples come in all sorts of varieties. So you can even compare apples with apples, if you like, and someone will come along and protest. Except they won't so often these days. Want to know why? Well, here's a story about apples, from the ever-readable George Monbiot.

Briefly: there used to be a huge market in different apple varieties. Hundreds and hundreds of varieties: Belle de Boskoop, Michaelmas Red, Brown Cockle. And then the supermarkets came along, and they decided that they could essentially compel their customers to standardize on a handful of apple types: Royal Gala, Jazz, Cox's. They did it, because the customer didn't really mind what type of fruit he was eating.

And that was essentially the end of the line for most types of apples. "Extinct, extinct, extinct." Most of them just gone. The free market effectively wiped out those varieties, ruthlessly and ignorantly. It reduced the number of breeds to a tiny fraction of what it was. People compared apples with other, different apples, and they just picked... apples. Nobody will fruitsplit between apple varieties now, because they're just all apples.

Drupal already has plenty of greengrocers, fruiterers of the programming world, people who rightfully and meticulously care about, and tend with love to, our apples and our oranges. But the prize, the broad user base, the real success story... that all lies in attracting non-greengrocers. And they'll just think: "Apples. Oranges. Other apples. Whatever." They probably won't care about different types of fruit; they've been told that their website has to have its vitamin C for the day, so they'll get the apple that's easier to get into, and to hell with your awkward-to-peel oranges. If your business model depends on selling individually wrapped fruit to other greengrocers, then good luck with that. I'm off to take some coals to Newcastle.

Who's going to tell the customers?

The blogpost suggests that:

Instead of comparing a toolset and a product, there are other, appropriate comparisons that businesses and developers should be making...

Well, it's wonderful that I know that now. But who's going to tell everyone else to change their ways? Is there going to be some sort of government-funded outreach programme? Will we all be making some sort of world tour? I try to picture a random drupal.org user appearing, like a superhero in a flash, in offices around the globe. "Hark! I hear someone in Mumbai making comparisons between Wordpress and Drupal that appear reasonably valid to them based on their expectations derived from: Drupal websites, Wordpress websites, drupal.org and wordpress.org. Yet I must intervene, swiftly! To the nearest train station!"

Anyone who decides that the solution is to tell businesses and developers what comparisons are appropriate will end up like Wowbagger the Infinitely Prolonged, travelling in time so that they can tackle simultaneous disparagements of Drupal. We can't rely on that: Drupal ultimately has to be able to say this for itself: it should be obvious to these businesses and developers what Drupal should be compared to. Otherwise these businesses and developers will continue to make comparisons based on the (admittedly incomplete and sometimes wrong) information they have available to them. 

Saying what comparisons end users ought to be making is a lot like that other canard, "educating the user." Unless you're talking about a site's administrative user base, people at a client who you could count on one hand, that's a pretty mammoth task. Who gets to ensure that such universal education about the vagaries of, say, module weights, or the vagaries of CCK field display, eradicates doubt in the same way that we once got rid of smallpox? Similarly, if Drupal's success against Wordpress depends on people only ever making the comparisons we want them to make, we're in trouble.

There's also tendency to call Drupal a CMS, right up to the point where someone complains it can't do what a CMS does out of the box. Then it's called something else. A toolset. A content management framework. A management framework toolkit. A content toolkit framework management system tool. It might deflect debate, but ultimately the jargon just puts people off. You might have won the battle with potential new users, but the war goes to whichever CMS with limitations that accepts from the outset that it's a CMS with limitations, and tackles those limitations---in Drupal's case, most notabbly user experience and getting started with it if you've never heard of Drupal before---head on.

The documentation on Drupal.org is pretty impenetrable to the average newbie, but if you can figure it out then you get the impression that Drupal is basically a CMS, a bit like Wordpress only with other (waves hands) stuff. So it must have a WYSIWYG editor, only somehow more so, right...? And the user experience of Drupal 7 is a great improvement, much like improvements that have already happened in Wordpress; but, despite the fantastic news that we're moving to beta releases, only greengrocers are using that right now. If we want non-Drupal users to make the right comparisons about Drupal, today, will we really fix it by arguing about distributions; products; toolsets; downloads; projects; modules; features...?

The technology will save us, of course

Here's a handy tip. Technology alone will probably never save you. You only have to have sat through Jen's talk, and to have read about Hagen's similar Wordpress-Drupal-Joomla walkthroughs in the unconference on DrupalCon Monday (I blogged about Hagen's marvellously excruciating walkthroughs on the g.d.o usability group) to know that the technology won't save us. Or rather, specific bits won't save us. Aegir won't save us. Products won't save us. Distributions won't save us. The drupal.org redesign probably won't save us either, because that's nearly finished and yet we're still denying the very usability and expectation problems that are still in plain sight.

Don't get me wrong. All of these things are great tech--really fantastic, mindblowing tech---and they solve specific problems that developers come across. I think drush make is one of the most exciting bits of Drupal I've ever used. But I'm a developer! Of course I will think that! Not only do I know how to set a Drupal site up in 15 minutes, but I also know from experience how to avoid the many, many ways in which it will take you two days. These tools  don't fix the fact that Wordpress is easy to use. Wordpress just works. Drupal needs tools to help you install it, and command-line fu before you can even make those damn upgrade errors go away, go away, please just go away.

You can't wait for technology to solve these things. Distributions won't change the fact that you can click around the drupal.org website for ten minutes and still not be completely sure what it is, or how to use it. The learning curve is still too large; and I'm just talking about how you use drupal.org, not how you use Drupal. Until everyone in the world is re-educated to know, and care, about what a distribution is, and how it's subtly different from just getting the blasted thing to work, then distributions won't save us.

What will save us is usability. User experience reviews. Jen Lampton's talk. Hagen Graf's talk. Watching users try to get to grips with this thing we love, when they just don't love it so much, won't excuse it as much. Feeling the cringe when a new user falls into the same trap over and over again, and knowing you won't be able to appear on their shoulder to help them. Then, though: this is key. You can't just say, "well, we'll let the themers fix that," or "maybe the forums can discuss it," or even: "when users ask for the wrong thing it is rarely, if ever, the right answer to humor them." Instead, you have to accept the application has a usability problem; and work out a plan fix it, right there and then. Developers, deciding to fixing usability problems above functionality problems. Again and again and again. Test, observe, cringe, fix, repeat.

It's a slog. But it could work where technology won't, because ultimately end-user functionality with serious usability problems will end up massively underused, and as far as the greater course of Drupal is concerned it might as well not be written.

Does this matter?

I feel like I've gone on a bit, and if you feel like that too then I apologise. After all, the usability community in Drupal is healthy and growing. People are starting to take usability seriously. But I feel like there's still no UX standards to point to, with the same weight and depth as the coding standards. Talk is silver; code is gold; UX is presumably 24-carat meh. There are still no easy ways of sharing usable interface patterns as easily as one might share patches. Usability is still a teenager, and it's easy to freeze it out of the conversation it in favour of adults talking about established technical preferences. Usability needs cultivating, and the user's behaviour needs to be king, whatever comparisons that user might make. We're right to worry about Wordpress, but that's a good sign! It means that Drupal is doing something right. It's worrying about the user.

Maybe, returning to the original blogpost, "the time is right for Drupal products." I'm happy to agree with that. Featurization and ease of deployment certainly suggest that productization will swiftly follow. But that has nothing whatsoever to do with the usability talks. It really won't solve the fundamental problems that Jen's talk tried to address.

One thing she said really stuck with me; as a developer, I took note. Well, I took lots of notes, but here's what stood out for me:

"Wordpress is behind us, but they can move fast, and they're looking at us."

With version 3.0, Wordpress quietly turned itself into a CMS. Maybe they saw fields were working well for Drupal, and thought: we'll have that. "It's simple but it'll do: let's ship!" Bang. Instant easy CMS. Usable too.

That's the point, in the end. Usability is not a nice-to-have. It's the canary in the cage, the indicator species: when usability suffers, it's telling you about lots of other attitudes that lead to the user being made unhappy. Whereas Wordpress emphasizes usability; it's friendly to the user out of the box; with things rich-text editing that the user wants to have without any issues; with straightforward media management; and with upgrade methods so simple they're frightening.

Drupal? Drupal's a brilliant, smart, well-oiled, massively functional framework (it's far more fun to develop with than Wordpress.) It's a CMS, except when it's not (but when it's not, it's still a rich seam for developers to mine.) It's still tricky for newbies to install (but developers learn the tricks.) Image handling is odd, and if you pick the wrong method early on you're in for a lot of pain later (but developers know which one to pick.) Upgrading modules is not a one-click experience (unless you're a developer and you've played with drush.)

Yet with all that in mind, here's a question. If you'd never heard of Drupal or Wordpress before, and you were 90% of the internet, rather than a developer; if you were the same sort of user as a client's marketing guy, who's never written a line of PHP in his life; based on the experience of trying to get each of them up and running yourself, which would you choose? Apple or orange?

The UK government’s Central Office of Information (COI) has produced a draft report on governmental departments' adherence to browser standards and asked for feedback. Depressingly, the report is not available in a web friendly format even though there's no real reason for it to be only released as Word and PDF.

You can read a breakdown of the worst bits on The Web Standards Project, but here's the feedback I just emailed webguidelines@coi.gsi.gov.uk:

Hi,

I appreciate the COI's desire to provide roadmaps for cross-browser support, but I'm otherwise dismayed at the recent publication's back-to-front emphasis regarding browser standards support. It feels like the publication has been put together by people who are remote from the current thinking on web standards and how best to both promote them and take advantage of them.

As a web developer I've had a reasonable amount of experience in developing sites cross-browser, and generally the quickest and cheapest site creation (and the least patronising or inconveniencing to the eventual end users) is effected by developing first on the most standards-compliant browser available, regardless of its user base. Then, once the site is stable and well-built according to the standards (which are there for reasons other than semantic fussiness), one can use *freely* *available* frameworks to compensate for browsers which do not follow the standards (which may be more popular). It involves more effort at the start in exchange for a much smoother site construction later on.

Let me repeat that: building sites according to web standards, then accommodating bad browser behaviour, is CHEAPER, QUICKER and MORE ROBUST than what you implicitly propose i.e. building with bad-but-popular browsers in mind, and then looking at shoe-horning in browser-specific fixes for the others.

If you inconvenience the users of good browsers because there aren't many of them, you are taking us back to the bad old days of 1990s browsing. I haven't seen the phrase "We advise you to upgrade your browser version" (which I take verbatim from p4 of your report) on any site I respect and trust for almost ten years. Ask any decent web developer: it's no longer necessary, and frankly it reveals far more about the mindset of the site designers than about the state of web technology.

Like the proposal's guidelines, its consultation is happening the wrong way round, and highlights a top-down mentality behind composing such documents. Sharp, savvy developers recruited from any of the vibrant UK web conferences---dConstruct, OpenTech, Barcamps---would have been glad to have helped the COI out before this fundamental missing of the point was published, were they to have been asked. It would certainly have saved both COI and the web community at large a lot of heat, friction and wasted effort.

Yours faithfully,

J-P Stacey

PS: I was originally going to send this via your website's feedback form. After having read your proposals for building good sites I decided to email it instead, just in case.

You've got a month to comment yourself, if you're a UK or European developer. Feel free to use the above template, only I'd recommend changing the name at the bottom.

The next OGN next Tuesday has developed broad themes without us really trying. Generally it’s best to have a mixture of talks, because of the eclectic mix of interests represented by our audience, but two clear topics have arisen, with some overlap in between.

Gobion Rowlands leads our e-learning procession, with a keynote on “turning hard data into fun.” Red Redemption, Gobion’s company, has recently won an award for its online Flash game about climate change. His keynote neatly segues into microslot (5-minute) talks on e-Learning and technology in archaeology.

Meanwhile, Charlton Barreto will be discussing in his keynote what we can learn from Web 2.0. Charlton has worked with OASIS, W3C and the IEEE on related topics. But his keynote sets the ground for talks about mashups, custom CRM systems, and a possible standard for portable social-network information.

And neatly bridging the two camps of educational outreach and web-2.0 hackery is the Oxford Barcamp microslot. And although I’ll be a bag of nerves on the day, I can’t wait.

What happens when nobody will take responsibility for a standard that the web relies on?

RSS, the standard millions of us use to syndicate content, and view other people’s syndicated content, was originally invented by Ramanathan Guha at Netscape, for use on its my.netscape.com portal. Soon afterwards, Netscape lost interest in the format, leaving it ownerless and later on picked up by a development community spearheaded by UserLand Software.RSS 0.91 became 1.0 and 2.0, yet despite the deprecation of the grandaddy of them all 0.91 is still around and in use, arguably because of the vast overcomplications in its immediate successor and the divisions that it caused in the community.

The problem with that is as follows. Every time someone views an RSS 0.91 syndication feed with certain types of syndication software, their computer attempts to get the DTD from this location on the my.netscape portal—it’s hardcoded into the way that the software understands what XML format it’s dealing with. So this URL gets plenty of hits:

http://my.netscape.com/publish/formats/rss-0.9.dtd

Which is great, until Netscape decide—legitimately, one might argue—to update the my.netscape portal and get rid of the DTD. Which they did, at the start of the year. At that point, a good portion of the syndication lights go out across the world. And although we now have a moratorium until July 2007, nothing has really been solved in the long run.

Anyway, Netscape shouldn’t have to support the bandwidth of millions of DTD downloads for a standard they declared defunct—when did they sign the don’t-be-evil contract?—and maybe people should “just” move to a newer version of RSS, or Atom. But this whole episode is an Ozymandian warning of what is to come. We’ve reached the point where the URLs of industry (one-time) giants are simply no longer to be trusted as the location of standards.

One day Microsoft, and Sun, and IBM, will cease to exist, and their websites become the 22nd century equivalent of Google-adsensed search engines (Google, of course, will be around forever, more’s the pity). Sooner or later something really horrible will happen for the open communities, say Purl disappearing for good, taking things like the Dublin Core XML specification with it. We need to know how to deal with the loss of their specifications and standards now: the unreliability of the URL as a locator for DTDs and schemata. Or is the only lesson we can draw from history, that we’re destined to wander from standard to standard as the specifications drop off the radar, leading the nomadic life of those standardized today, obsolete tomorrow?

Elliotte Rusty Harold roundly disses microformats, comparing the practice of utilising them to homeopathy, of all “disciplines.” A bit of cheeky banter, so it’s probably churlish to point out that the comparison itself turns out to be unsound within his own argument: whereas homeopathy might arguably be no solution to any problem, Elliotte’s beef with microformats seems to be that they solve a problem—expressing non-XHTML structure within XHTML—for which he believes there are more efficient solutions.

Not being immediately aware of any such alternative (microformats having evolved from a number of web practitioners’ frustrated wishes to add extra semantics to their XHTML), I was a bit surprised to read:

The only reason I can imagine you might choose a microformat over a macroformat is because macroformats are invalid XHTML, but so what? XML doesn’t have to be valid! That’s a deliberate design decision in XML. Some say invalidity is the real revolution in XML. It’s what XML brings to the table that SGML never had.

Well. This is true, in a sense, but not really pertinent to the actual problem that microformats are intended to solve. SGML didn’t “have” to be valid, if we’re talking pragmata (you say prag-may-ta). Millions of webpages out there had the most godawful pseudo-HTML on them, and browsers muddled along reasonably well. But that wasn’t enough. Our data didn’t soar. Our browsers didn’t leap, nor did they bound.

There were a number of motivations behind establishing DTDs for HTML, let alone for XHTML, and one important one was being able to hand a webpage to someone who wants or needs to be able to maintain it in good HTML, and give them an editor that could enforce the standards, either by beeping at them (Sean McGrath might turn in his blog at that one) or by quietly fudging good HTML in the background without telling them. If you start putting arbitrary tags in, you break the silent checks that mean non-technical people can actually write webpages.

Many of the clients I work with have government or charitable funding, and a proviso of this funding is that their pages be accessible to web users with special needs, and to be strongly future-proofed in the light of past mistakes. In part for the reasons above, but generally because it’s safest to enforce as strongly as possible without affecting the primary goal of the medium, accessibility standards and funding bodies’ definitions of “future-proofing” tend to require pages to be valid XHTML1.0 . It’s fine for someone technical to say to themselves “this is valid XHTML1.0, except for the bit I’m putting in now”, but there’s no guarantee that the next person to touch that page will understand or care. Or the next person. Or the next person. Didn’t we… didn’t we tackle the problem of markup rot once already?

Adding your own tags to XHTML is something that Elliotte Rusty Harold can do with aplomb, and probably ought to because he knows what he’s doing. And if he’s got the spare time to maintain every site on the web on their owners’ behalf, to ensure we don’t return to the tag soup of HTML that one couldn’t even begin to validate, then he’s welcome to give it a go. In the mean time you might want to campaign for an exception to current UK disability law, or at any rate almost every organisation’s interpretation of it, in the case of sites maintained by Elliotte Rusty Harold—I’ll even sign the petition if there’s one passed round—but I can’t see it gaining much traction.

There’s a pragmatic solution to the validation problem, though. We could reprogram our XHTML validation engines to ignore specific blocks of markup based on particular criteria, say a reserved attribute on particular elements that means the content is checked for well-formedness—CDATA elements won’t do, because they can contain absolutely anything, and you lose the power of XML—but ignored by DTD and schema validation. Might I suggest <div class=”xhtml-ignore”/>…? It’s funny, but that particular method rings a bell.