Why this might be a plug, although it probably isn’t

I’ve also been waiting to categorically declare a very minor conflict of interest, which I can now do: yesterday a lovely paper copy of the Google Chrome comic arrived in my letterbox. So if you think I’m blogging because of that, you can navigate elsewhere now.

Certainly a product’s incompatibility with X/Linux would normally make me avoid it, so perhaps I have being persuaded by the marketing. But a large part of Chrome is marketing, and what makes it most interesting is what it reveals about Google’s marketing internals and about whether or not they matter; but more about that later.

How Google Chrome feels and acts

The ever-lucid Jon Hicks has already posted some thoughts, so my delay in writing has saved me time on that score. I don’t have a great eye for design, but I feel like Chrome’s appearance is interesting if less revolutionary than the promotional material might suggest. Tabs sit in a blue surrounding background, making them look like a half-hearted IE7 reskin. Menus have, as in IE7, been relegated to two weird buttons on the right of the address bar. Full-screen mode is nice, though, as the tabs sit over the top window bar, combining Windows chrome and Chrome chrome—does that make sense?—to increase the window size.

Browsers are browsers and, as with word processors and spreadsheet software, they should really be free and open-source by now, leaving proper software companies with time to develop the next generation of applications. So in a sense most of what Google Chrome does, it does well: unobtrusively and unremarkably, and that’s how it should be; but quite hard to comment on. What’s most noticeable is the speed: it’s faster than any browser I’ve used on XP, ever. Opening new tabs and windows—although in Chrome there’s very little difference under the hood, as they’re all processes—is nippy, despite the extra overhead that Google have decided is essential to Chrome’s distributed stability. Pulling a tab out of one window; letting it drop as a new window, or dropping it into a different window; maximizing windows and general rendering of content: these are all sharp and impressive. But again, they shouldn’t be as obvious as they are, and it merely reflects on other browsers that Chrome feels so fast.

There’s a kind of Firebug-like inspector in Chrome, and it’s nice enough, although it ought to be extracted as a plugin before long, otherwise my guess is it will end up neglected. And popup blocking—represented in the promotional bumf as the window swallowing the popup, and you the user pulling it out in a reactive way—is just a case of a popup window appearing in the main tab so that only its bar is visible. There’s no obvious feeling of resistance as you pull the hidden window up to be visible. Maybe that’s the point, but after the reaction of tab dragging and dropping you feel like you’re moving popups around using an entirely different UI. Also, as I’ve just discovered, text search doesn’t look in form textareas, which makes proofreading your blogposts difficult.

Generally, though, Chrome has at least run rings around anything that Microsoft can produce in the browser market, and then Google managed to completely open-source the code which, like some old John McCain company, Microsoft can neither do nor understand why it should. It’s astonishing to see one huge company outmanoeuvre another like this, and suggests interesting times still ahead

What packaging Chrome has been wrapped in

As regards the marketing, Google has also managed to completely confound the other big player Microsoft is intent on gradually rebranding itself as something in between the silver-and-blueblack chunky mens’ toiletries packaging that make it acceptable to possess both moisturisers and Y chromosomes, and transparent, flashy interfaces of the sort that IE6 always fucks up and means web developers have to work around. Google, on the other hand, has essentially presented itself through Chrome as a kind of retro-yet-futuristic 1950s take on a science-fiction OSX, all meals-in-a-tablet and egg-shaped seats.

Much of this has been down to the artistic skills of Scott McCloud. I never know what to think about him. On the one hand he’s got this unique, flowing, clean style that’s something like a scrubbed Daniel Clowes; on the other, his drawings can sometimes feel washed out, pretentious and affected. On the one hand, he’s tried to revolutionize the way that people think about comics, often by exposing what good comics have been doing for years; on the other, very few people have got rich on the micropayment model he espouses, and if it isn’t working for musicians it’s unlikely to work for graphic novelists.

Where that backlash came from

By portraying itself as different from other industry behemoths—which, to be fair, it is in some ways—Google has left itself in a bind. It still has shareholders, and on one level legally has to conduct itself as a responsible company, however much it wants to be treated like, or possibly with, Ubuntu. Fronting what’s essentially a business exercise with a divisive figure like McCloud leaves you ripe to parody, and The Register has tried to step in with Google Chrome comic funnies. They fly in the face of the no-alteration Creative Commons licence that Google/McCloud released the work under, but that’s fine because the uniquely American concept of fair use lets them do that if it’s satire.

Except they don’t work as satire, because they’re not funny. And yet at the same time the font they’ve used makes it look eerily reminiscent of the shockingly explicit Jerk City, which hints at a far better way of parodying the style: the 4chan/yayhooray parodies are in a way more honest and hence funnier: probably because they’re more anarchic and less interested in squeezing out another humourless Googlebashing.

No product launch is smooth, and there’ve been bumps in the otherwise smooth journey that Chrome has made to mass testing (if not mass acceptance). The end-user licence turned out to contain mad MyPlace-like terms of use which was sort of an accident, although it’s drawn attention to the fact that they still exist in other Google products. Whoops. The original beta—or, given that everything from Google is a beta, maybe we should just call it an alpha and be done—was susceptible to Safari-like carpetbombing, and The Register criticized the rather flaky bugfix and its rollout procedure.

I love el Reg and its journalistic instincts. It’s more than happy to puncture someone’s silly bubble, and it displays a dogged tenacity in pursuing the “real” story: although they’re basically wrong about climate change, in the way that Private Eye turned out to be wrong about MMR; and their grammar and sub-editing is atrocious for an outlet that considers itself to be conducting serious journalism.

But I think they’re being unfair on Google: what other open-source product would launch to such scrutiny? What other product has had seamless security procedure baked into it from its alpha, and why should that matter? Google are big, but they can only cover so many bases: there’s so much infrastructure glossed over by McCloud’s comic, and maybe a FOSS-like boring list of features and a changeset would have led people to underestimate less the sheer amount of stress testing, and the sheer amount of work that can only now be stress-tested, now there’s a user base and the animosity of the press to contend with.

Google did after all still manage a big reveal—only two weeks before London’s own Google Developer Day—in their usual manner. But I do wonder about the timing. Was there a danger in letting crowds into Google UK while Chrome was still secret? Did someone want Chrome to be out—prematurely, if need be—for the coup of having people drool over it at a GDD? Was the news about to leak anyway, and did damage limitation dictate the software’s release? If that’s the case, though… does it matter? Google gets its theatre; the world gets an interesting FOSS project; early adopters get an unstable pre-release: everyone’s happy. Ish.

Which hand is on Google’s tiller

Pessimistic journalists—and in my honest-to-goodness opinion there’s no better sort—always point to Google’s lack of revenue from its non-core offerings and suggest that it’d be far better for Google to concentrate on the products that directly earn it money. But they forget that Google’s profitless products exist as a perpetual rebranding and repositioning of Google: indirectly, they maintain Google’s status as a company that other companies, developers and end users actively want to be associated with, and actively trust. While they’ll never entirely remove the patina and dust collecting on their “don’t be evil” statement, they can at least act like a company that trusts open source, and whom open-source communities trust in turn.

Google can keep on pushing things like Chrome out, and its launch cycle can be dictated by something other than the developers’ whims, because: its main rival has nothing like Chrome, or GMail, or Google Docs; and successful FOSS projects like Django or Ubuntu have hardly suffered from bleeding-edge alphas or crotchety betas, as long as community, or honesty, or image, has been there to prop them up.

Who the hell am I to be telling you this anyway

I quite like Chrome. But I completely accept that receiving personalized communication from the company taints my status as a reliable blogger. So don’t take my opinion on Chrome at face value.

I certainly won’t: I’m about to reboot into a proper operating system, and I’ll lose Chrome as I do so. It’s a nice addition to the existing ranks of browsers, but not that nice.

Web architects must understand that resources are just consistent mappings from an identifier to some set of views on server-side state. If one view doesn’t suit your needs, then feel free to create a different resource that provides a better view (for any definition of “better”). These views need not have anything to do with how the information is stored on the server, or even what kind of state it ultimately reflects. It just needs to be understandable (and actionable) by the recipient.

— Roy Fielding on creating new resources for REST architectures.

What if you want the same output for several different data structures? Let’s say we have three people, and we’d like to run the following include, saved as name.html, on all three:

<p>{{ person.firstname }} {{ person.surname }}</p>

Ideally you’d have the three people in a structure called people, and be able to run:

{% for person in people %}{% include “name.html” %}{% endfor %}

But what if the order matters? What if you need account_director to do different things in the template from project_manager? You’d want to send them through as separate data structures, and this:

{% include “name.html” %}
{% include “name.html” %}
{% include “name.html” %}

would just do nothing, because it references person and not account_director. An alternative is to turn name.html into a custom tag, taking arguments; but if the content of name.html were any more HTML-heavy then you might want to abstract it into the include nonetheless, and then you’d just be running a custom tag method, to pass variables to a template fragment, to render the fragment, to pass it back up to the template. It’d be nicer if we could do this with core functionality instead.

Using the latest development version of Django, we can implement basic variable assignment—getting person to reference whatever we want in the template layer—using the with template tag. This means that we can wrap each include tag to produce the required output:

{% with account_director as person %}
  {% include “name.html” %}
{% endwith %}
{% with project_manager as person %}
  {% include “name.html” %}
{% endwith %}
{% with lead_developer as person %}
  {% include “name.html” %}
{% endwith %}

It seems like overkill for our name.html example, but there are plenty of situations—I just found one, hence this post—where it makes sense to factor out code into an include: being able to fiddle with the template context before including it is a lightweight alternative to template tags and a tidy alternative to dumping the HTML right there in the primary template.

Django has its own library for building forms, currently called newforms (to distinguish it from the old library, deprecated but left around for compatibility reasons). The ModelForm class in this library interrogates a particular class from the model layer and builds an HTML form—complete with post-submit validation, field widgets, the lot. The beauty of this is that you can run:

import django.newforms as forms
from myproject.models import MyObject

class MyForm(forms.ModelForm):
    class Meta:
        model = MyObject

and that’s it: you can then use an instance of MyForm anywhere in your view layer to add or edit MyObject instances in the database.

Problems arise, of course, when the automatically generated form doesn’t provide exactly what you want. It might, for example produce select dropdowns for you with thousands of items, whereas with better knowledge of the page your user is on then you might be able to whittle that list down to a handful of items.

Let’s imagine your objects can belong to Django users i.e. in your models.py you have:

class MyObject(models.Model):
    owner = models.ForeignKey(User)

This means that when you try to create an object, the HTML from newforms will give you a select dropdown of all the users in the database. But what if you have loads of inactive users, cluttering up the dropdown? How do you get rid of them with the minimal amount of coding i.e. without having to build your own form if possible?

Luckily, newforms sticks the queryset it creates somewhere that you can fiddle with it. With lazy evaluation, you get a chance to change the queryset before it hits the database; with chaining, you don’t have to worry about rebuilding the queryset yourself, but can just add to it instead.

In the MyForm class, you can override the inherited __init__ method as follows:

def __init__(self, *a, **kw):
    forms.ModelForm.__init__(self, *a, **kw)
    self.fields['owner'].queryset = self.fields['owner'].queryset.filter(is_active=True)

This calls the existing superclass constructor first, passing along any values. Then, after the queryset has been created for the select dropdown but before it’s been evaluated, the queryset is modified to whittle down the resultant set of users to only those where the “is_active” flag is set to True.

Laziness and chaining make Django’s ORM layer very effective; forms and newforms help to greatly reduce the amount of coding required by the view and control layers; together they can really simplify a codebase.

But the eyes of even the most hardened LAMPer would widen, if he were to dip into a conversation to find someone saying the following about something they’ve built over PostgreSQL:

Yeah, I could add the current date/time to the end, so as long as you don’t add more than one [entry] with the same name within 1 second, it’ll be fine.

It might take you a while to work out (a) what that’s referring to (b) what kind of error in thinking it demonstrates and (c) what it implies about the quality of the underlying code. By that time—if you’ve worked at all with that sort of programming—your head may well be in your hands.

The comparison in the Guardian website is with Facebook (which is turning rapidly more socially evil by the minute anyway). I think it would be more illuminating to compare mySociety sites with such large, government-commissioned (and consultant-propelled) projects like the NHS Spine.

The Spine is a vast, overarching superproject; it’s a messy amalgam of large, penny-pinching businesses; separation of responsibilities—bizarrely—is in part geographical, so despite purporting to provide a single system there’s no guarantee (outside of the weird steampunk astrolabe brains of consultants) that Yorkshire and Humber will be able to chat freely with the West Midlands. It’s been put together without any real thought about whether the problems tackled are the ones that the problem-sufferers (NHS staff and patients) encounter most often. In a nutshell, i’s overblown, overpriced and over the top.

But if mySociety had been brought in? You would have around a dozen projects, all quite tiny, and all solving fairly simple, easily-scoped problems. Most importantly, they would all stem from consulting the staff: each would have a valid reason for existing before even the planning began. The basic version of each site would be ready within six weeks, and a more polished version in three months. There’d be ongoing updates and improvements as user requests made them appear worth while ((the Spine, on the other hand, would shudder at the thought that the hoi polloi of front-line employees might dictate project direction, rather than a consultancy firm). And from the point of view of an end result, almost everything would be ready by now, in comparison to a couple of fringe benefits: it would be improving staff and patient experience today, across the country, in those specifically targeted areas, by a hundred times the effect the Spine will have, and for a hundredth of the cost.

mySociety produces perfect examples of the open-source maxim: find an itch, and scratch it. But what distinguishes them from almost every other FOSS project (apart from perhaps Ubuntu) is that they’re happy to treat other people’s itches.

… This brings us to the second obviously-bad thing that can go wrong with code bases: copy and paste. It doesn’t take very long for programmers to learn this lesson the hard way. It’s not so much a rule you have to memorize as a scar you’re going to get whether you like it or not. Computers make copy-and-paste really easy, so every programmer falls into the trap once in a while. The lesson you eventually learn is that code always changes, always always always, and as soon as you have to change the same thing in N places, where N is more than 1, you’ll have earned your scar….

Code reuse is great, although some languages punish you for it: Coldfusion takes a performance hit on behalf of debug logging whenever it crosses a component boundary, even if debugging is turned off. And a lot of includes scattered across the place can make it difficult to track what’s going on (unless you use those very IDEs that Steve suggests treat code like heaps of dirt to be rearranged). I suppose ultimately it’s about developing an instinct for balancing short-term performance, ongoing maintainability and long-term scalability, and you only learn that by dint of many, many scars.

There’s already a tutorial with a CCK submodule available. The premise of the sample module is to provide two fairly generic numerical formats. In principle that simplifies the submodule, but in practice it also means it’s difficult to work out precisely why particular programmatic choices were made and what you can tweak. Also, at work we wanted a much richer content type. Here’s what we wanted to display:

1. Build an “Available in” field for e.g. all the different formats that a book might come in.
2. Record the dimensions DxWxH
3. And some dscriptive information
4. And let the user pick a thumbnail from a dropdown
5. And let all of this be done multiple times

Here’s a mockup of what we wanted to appear on the admin page (excuse styling—Wordpress isn’t the best for this):

And here’s sort of what we wanted on the front end (again, sorry for the HTML: it’s table-heavy as it’s a one off):

Available in
Hardback			Paperback			Trade p/bk
	ins	cm		ins	cm		ins	cm
W	5.1	13	W	5.1	13	W	5.1	13
H	7.6	19.2	H	7.6	19.2	H	7.6	19.2
D	0.79	2.0	D	0.79	2.0	D	0.79	2.0

It was difficult, from the simplistic submodule available, to work out exactly how you leverage complex editing widgets, and how you get CCK to save all the database information in the five columns we needed: description, thumbnail image reference and three dimensions in inches. What’s worse is that a CCK submodule with an incomplete set of hook functions seems to do a fandango on certain areas of the database (possibly the cacheing of certain bits of the module’s behaviour): certainly developing a CCK submodule was quite painful, as the process of trial and error frequently required a database restore so as to try again. That tutorial does warn that “all hooks are required. CCK expects them to all be present and will not function correctly if some are missing.” But there’s a world of developer pain in those brief phrases.

Since I originally mentioned the problems we were having on the development mailing lists, a couple of people have asked me how to build a CCK submodule to support rich fields. With this in mind my employers Torchbox have very gracefully let me make the module we built public, with some minor edits (the client’s site isn’t live yet, so names and details changed to protect the innocent).

The module is available here. It consists of 4 1/2 pairs of functions, as follows:

1. Declare:
   1. field types that the module supports
   2. widgets that the module supports, for editing fields on the node-edit page
2. Settings for:
   1. field (so settings inherent to the way that particular field is stored e.g. decimal accuracy for numeric values)
   2. widgets (so e.g. the image gallery that populates the dropdown for the wireframes)
3. Handlers for:
   1. field, so what happens to it when a node is loaded
   2. widget, so pre-processing of the field to fit a HTML form, or producing a chunk of Form API, or post-processing the HTML form into a database-saveable format.
4. BONUS UNNECESSARY FUNCTION: cck_book_complex_input just produces the Form API for function 3b. Ignore this one!
5. Formatters, which give you output options for fields on the “Display Fields” page.
   1. Declaring formatters
   2. Formatters

A few extra pointers:

1. CCK does most stuff for you, so you don’t need to actually load/save field contents to the database; declaring the columns for widgets and fields is enough.
2. If you’re in the developing phase with your CCK submodule, back up the database first. See above.
3. We sidestepped formatters altogether as they don’t cope with e.g. multiple field values that need a header value. Instead, we just overrode theme('field'). One thing, though: you do need some formatters present, or the admin configuration page for “Display Fields” defaults every dropdown to “<Hidden>”, and you can’t get anything to appear.
4. To reiterate, CCK needs all hooks present in its submodules from the start. The further you get from Drupal core, the more bleeding-edge APIs will get.

Note that as I say we had to peer into the node (use var_dump($node) and search for the right array) and then override the theme, because we wanted a fancy header to our field. But that was pretty painless—mostly a matter of knowing one’s PHP rather than one’s Drupal—and it all worked smoothly otherwise. And we ended up with by and large exactly what we wanted. It works and, in retrospect, Drupal has helped us to write readable, maintainable code.

The hallmark of a good frameworked site is that it’s not easy to guess which framework was used, and TeamSugar manages that admirably. While I can’t really comment on the design—I find most networking sites pretty busy, and this is no exception—it certainly feels like a lot of hard work is distancing the user experience from the limitations of the framework, and that’s admirable from a usability perspective. The fewer visitors jumping through hoops to indulge the whims of your CMS the better.

There are a couple of interesting observations in that thread, mostly because they’re guarded criticism coming from an otherwise happy user of the framework:

… We have four app servers now, I believe, and a dedicated database server. We have also extensively edited the code. The architecture of drupal, unfortunately, lends itself to doing things like issuing one query per module per node on a page. For our 8 content sites this isn’t too bad because they’re easily cached and don’t make use of many modules. For TeamSugar it’s much worse because each feature generally requires at least one module. We cache where we can and refactor code elsewhere…. [link to 1971521]

… When you’re creating web applications you’re typically going for one of three things: speed/scalability, flexibility, or maintainability. Drupal is high on flexibility, average on maintainability, and poor on speed, in my opinion. If your priority is maintainability then I’d go for one of the RAD environments like Rails or CakePHP. If speed is your priority then I’d write your own lightweight framework, or whatever, and do it yourself…. [link to 198497]

If you haven’t ever had cause to use it, there’s a standard called ETag out there which you can probably implement using existing technology that can boost the efficiency of your content delivery a hundredfold. Along with the longer-standing HTTP header Last-Modified it can be used with compliant browser/aggregator software to drastically lower your overheads, while scarcely impacting on your non-compliant users. And although the two standards see most use in the blogosphere, they can be used for anything else from a company’s record in a directory to enormous high-resolution image feeds from astronomy laboratories.

The idea is that you embed in every outgoing page request a couple of HTTP header lines. That’s easier for the total n00b than it sounds: you can do it in one line each with, say, the PHP function header() or the Coldfusion tag <cfheader>, or even in the HTML if you don’t have that level of access, using <meta http-equiv=.../>. The point is that you set the following two flags on the “envelope” that surrounds the page you send to the browser:

Last-Modified: Wed, 15 Nov 2006 18:20:54 +0000
ETag: “78c4d3d8-1834-11dc-8314-0800200c9a66″

Typically for the first field you’ll want e.g. the latest date from your RSS feed, or the date on which a semi-static page was last edited. The second field is really up to you: if you never go back and edit posts without changing the published date then (a) well done you—there’s a space in heaven already reserved—and (b) you can just calculate ETag from the published date. It can actually be the published date if you’re a bit slack, although you might want to hash it instead, in the way that you might with passwords, to avoid any sloppy client software depending on ETags being dates.

What happens next? Well, you won’t see anything at first. But compliant software that’s visited your site before will start sending you two headers that correspond to your original submissions:

If-Modified-Since: Wed, 15 Nov 2006 18:20:54 +0000
If-None-Match: “78c4d3d8-1834-11dc-8314-0800200c9a66″

The idea is that you make sure that you can compare these to the values you’re about to send out, quite early on in your workflow. That way, if they match, you can immediately terminate all further work and just send a “304: Not Modified” HTTP header. The result? Well, with quite complex pages, involving the computation of tag clouds and term hierarchies and archive structures, you can work out quite early on whether it’s worth bothering, or whether the remote client will know exactly what to do if you just tell it, concisely, that nothing has changed since it last looked.

Word to the wise, though: if you are going to send a 304, you should also send your two headers as you always would. If you don’t then you only win out every other time, because the remote client will see the absence of ETag and Last-Modified headers and duly forget the ones it had in its cache.