In the mean time I’ve finally followed Tom’s advice (which I didn’t take when he volunteered it at the time) and upgraded Wordpress to a subversion checkout of 2.6+ . It was no more painful than the previous upgrade, and looks like being a much simpler procedure in future owing to subversion’s interface for switching versions.

In a parallel with the acronym LAMP, the overall stack is probably best described as “LAPD”:

• Linux
• Apache
• PostgreSQL
• Django

although there are refinements at almost every level, which I’ll go into below.

I18n with Django was a joy to implement. We had to have the site translated in six languages, but this was practically a doddle with Django’s core internationalization behaviour. Before we’d put the translations in, I clicked on “POLSKI” and was sure the l10n wasn’t working, until I spotted that the “ENGLISH” link had magically changed to “ANGIELSKI”.

The only issue we’ve had—which we’re hoping to work around—was deciding on an initial translation when the visitor arrived for the first time. Some parts of the system seemed to need l10n—the plumping for a specific version of the site—before Django’s LocaleMiddleware had done that for us, so we’ve had to force English until the user states otherwise. I’m hoping that a version 2 rewrite will fix that.

PostgreSQL might seem an odd choice of database to the LAMP community, who are used to MySQL’s minuscule overhead and often actively work around its deficiencies to keep that low overhead. But PostgreSQL’s stability and maturity outweigh the performance issues which—with a little care and attention—it’s possible to at least partly mitigate.

Compared to MySQL or Oracle, PostgreSQL also has the accolade of being supported by Django but having no outstanding database issues. Whether that’s because nobody else uses it, or because the integration is tight and relatively bug-free, I daren’t comment, but we’ve had nothing but seamless, transparent behaviour thus far.

Our Django-oriented hosting is with Webfaction, who are geared up for one-click mod_python Django deployments. The Django app sits in its own Apache process, while static files are served by the Linux server’s main Apache process: differential serving of content lets us take advantage of differential hosting fees. The hosting is a pretty good package, although TimeToLead.eu (despite being a pretty small application) is already finding the maximum memory package a bit restrictive, so we’ll need to keep an eye on that.

On top of this, Django has to a greater or lesser extent enabled us to use a whole host of other neat little technologies to integrate the site and improve both the user and the developer’s experience:

• The main Flash widget is written in Flex: I’m no particular fan of Flash myself, but it fulfills the remit admirably here. Because of text flow issues, there are actually six versions of the Flash file, one for each language, but the majority of the explanatory text is then picked up from an XML feed provided by Django, so can be retranslated by the client without recompiling Flex.
• Swfobject serves up our main widget and the YouTube file. It degrades well, but bear in mind that the API to it has changed considerably in the most recent major version.
• Unit tests in Django cover a number of stress points in the code (although not all, owing to the timescale). Towards the end, we were testing any major functionality changes by writing the tests first, which I was really pleased with.
• BeautifulSoup, which I mentioned a few days ago, is employed in the unit tests to check that certain content is coming through on the front end. It also managed to help us proof against a browser-dependent FOUC we encountered during development.
• Translations are managed by the Django application Rosetta: the most recent stable version is for Django 0.96 but the bleeding-edge repository copy seems to work OK on 1.0b1

Django has been the saviour here, living up to its promise as a framework for rapid application development. No framework is ever perfect, and there’ll always be arguments over the right way to proceed, reconciling what the framework developers want you to do with what the end developer wants to do, but this particular brief sojourn alongside Holovaty, Kaplan-Moss, Willison et al has been tremendous fun.

One big problem is vacation programs, aspects of your email provision that permit automated emails on your behalf to say “I’m not here right now.” They’re horrible, all of them. Outlook Express’ system only runs when your desktop is switched on: imagine the cheery post-seasonal look on your face as you return to work after two weeks away, power up your computer and then watch everyone who’s sent you an email for the past two weeks suddenly get an email each, per email they sent, before you can hit “cancel, for the sake of the newborn baby Jesus!” Outlook’s and Exchange’s equivalents merrily reply to any old bulk mailing list, leading to incredibly annoyed readers of the same address until someone takes matters into their own hands and either unsubscribes the offender or hunts them down, ties them to a post and shoots them in the head. Even the *nix .vacation is pretty grim, and you have to be careful not to set up some sort of nightmarish positive-feedback loop and bring down a server.

It occurred to me that it would be nice for subscription management systems to let you unsubscribe for a bit. The idea is that you’d let them tick a box at the point of unsubscription, and it would set a scheduled task to run in, say, two or three weeks’ time and resubscribe them. That would in a sense trump all the current user experiences when it comes to making sure you’re absent from all your mailing lists for the vacation duration. It would be fairly easy to do, because after a prompt from cron on the right date it could pretty much run off all the existing subscription code by calling a URL internally (and hence not requiring CAPTCHA or validation).

And then I realised: the people who would best benefit from the functionality wouldn’t even think to look for it. They’d just keep on telling all their fellow readers that they want to unsubscibe.

Seriously: who among us is just itchy and fidgety, waiting for analogue TV signals to be switched off? Who wakes up of a morning thinking, ooh, Whitehaven have to buy Freeview boxes now! with a frisson of glee? Radio switchover would be even more disastrous, of course: whereas TVs normally have a gap between themselves and an aerial into which to insert a Freeview box and convert digital to analogue, most radios are monolithic: receiver, “decoder” and audio equipment all together. That means that the average of five radios per household would be simply landfill material, useless fizzing boxes capable of picking up nothing but static. Future generations will already find a sliver in the geological strata that they can classify as “analogue to digital”; FM switchoff would add a shiny, plasticky laminate to that layer.

Here’s a thought, though: how about a little digital-to-analogue converter for radio? It could be the size and construction of, say, a Fon, and configurable over USB. It would transmit FM in the style of an iTrip to fill perhaps the room it’s kept in, plus neighbouring rooms. It could transmit multiple FM stations at the same time, and even mimic the frequencies of the analogue stations (if they were ever, say, switched off without anyone wanting them to be: God forbid!)

But what if your converter’s chosen frequencies conflict with next door? FM transmission depends on phase coherence, so even if your two boxes were within an electronic ace of each other frequency-wise, they’d still interfere with each other and cause all sorts of beating, flangeing and other dirtily-named auditory confusion. At least when someone passes you on the motorway and their phat BMW iPod transmitter briefly swamps yours then they’re gone quickly, hopefully to come to some sort of messy grief on the next bend. But if you’re stuck near someone who permanently foxes your radio listening—and it would only take maybe three or four such converters for a conflict to be inevitable in the narrow FM band—then what do you do?

Answer: let the converter boxes know how to talk to each other. Let them negotiate their frequency spreads. Let them, moreover, report back to you over what sort of understandings you and your neighbour’s radios can come to. Let them synchronize, if possible, on stations you both want to listen to, strengthening the signal and meaning both of your houses are entirely covered by the FM cloud. Exploit, don’t squash, the wisdom of the group.

Until very recently all seemed to be going well with these two jobs, until I suddenly found a rash of OpenSSH popups appearing as Ubuntu struggled in the face of them to start. Often enough of them would appear that GNOME would panic and warn me that my keyboard might have been taken over by, I don’t know, aliens, or lolcats.

It turned out that my machine was presenting itself as IPv6-addressed to one server, on which the authorised key was only set up with the domain name and IPv4 expected values. This was quick to fix: I added the IPv6 address to the entry in authorized_keys and hey! presto, the automated tasks are now running with nary a whisper.

What’s more worrying is that, from the looks of my mail backups, they haven’t run since March. Presumably it’s only been a recent update to ssh on Ubuntu that has directed these password requests to the screen: until now they were probably being lost to /dev/null . So a change that I initially thought was a pain and had reconfigured what ssh was declaring was my machine’s IP address, could actually just have been a tightening-up of the error reporting; in the end I was very glad ssh had annoyed me so much, as the annoyance has potentially saved my email backups.

./rmrip.py foo.conf

As the user himself implied, this has the added bonus of making it more crontab-friendly e.g. you can have a crontab entry saying:

0 2 * * 2 $HOME/rmrip/rmrip.py tuesday.conf

You can download rmrip as before from http://www.jpstacey.info/blog/files/code/rmrip.tgz. Thanks to Matt for the idea.

I’m currently taking over a maintenance project from a co-worker. The architecture in question integrates our CMS with Raiser’s Edge, the charity-oriented system from Blackbaud for tracking people, their donations, gifts, bequests etc. It’s a comprehensive but sprawling system, and because of licence limitations all data changes have to go through their equally sprawling API.

A prerequisite of development is that I install the RE client libraries on my machine: actually inside a Windows virtual machine sitting on my Linux box. The first time I tried this, it was over our between-offices ADSL line. That wasn’t terribly successful, owing to the enormous .cab archives included in the distribution. In fact, even stopping it half-way was unsuccessful because of the network delay, so the process had to be killed.

Unfortunately this meant that a later attempt at installation off a CD failed. The error report was, unhelpfully:

Error
Error Number: 0×8004041B
Description: Unknown Error
Setup will now terminate

There’s no clue as to what spoor the previous installation has left behind, nor is there any suggestion of what I might do to fix the problem. Although I’ve since found a solution for the error via Google, I thought it might be good to post it here, along with a possible reason for the error in the first place. Some of the reports of the solution are behind Blackbaud’s login system; some of them are on web feeds that might expire.

The registry key stores something to do with the date that you’re making the installation. To resolve the error, set your computer’s—or your virtual machine’s—clock to January 31, 2006. It probably needs to just be set in the past, but that suggestion worked. Once you’ve changed the clock, perform the install, which ought to go without a hitch. Afterwards, reset the clock to whatever the current date might be.

Why the error popup couldn’t just say that, I don’t know.

Similar solutions abound on the web: Tom Taylor has a method involving mencoder; other methods can be found all over the place. However, these all involve a bit of ad hoc command-line intervention, or scripts which aren’t terribly configurable. There are GUI and proprietary commands, but they tend not to offer great support for command-line and therefore scheduled operation.

I’ve knocked together a Python application called rmrip: it’s available in a tar file from http://www.jpstacey.info/blog/files/code/rmrip.tgz. If you unzip this to a directory you’ll find a number of .py files and a config.conf configuration file. Edit config.conf to match your system requirements and stream preferences, make sure rmrip.py is executable, then run it. mp3s should eventually appear in a subdirectory called YYYYMMDD unless you configure the system otherwise.

The application can in principle be run from a cronjob, so it could tick over late at night when everyone’s internet is otherwise nice and quiet. In addition, conversion works via a named pipe, which is a funky way of piping the intermediary, enormous .wav audio file straight into lame, rather than saving it to disk. This does unfortunately restrict the application to non-Windows machines, but it’s a great help for audiophiles with limited disk space: .ra and .mp3 files can be in the hundreds of megabytes for many-hour programmes, but the associated .wav would take up gigabytes.

Current requirements include (please give any feedback on this!):

• mplayer and lame: their locations are configurable
• The subprocess module in Python

Current file types supported:

• Direct rtsp://….ra Real Audio stream links
• http://….ram references to Real Audio streams
• http://….rpm Real Audio playlists (BBC so far only format tested)

To get you started, dave.org.uk has provided information on how to get stream information using a standalone Python program, and also has potentially out-of-date static pages detailing the current BBC streams.

Could I get this to work? Could I hell. xorg.conf configuration after configuration threw the error

(WW) MGA: No matching Device section for instance (BusID PCI:5:0:0) found

Once in a while there was a flicker from X. A couple of times my machine returned to its text-only state with text on both monitors. So something was happening, but not even the same thing every time.

Eventually, I began to grow suspicious. I finally did what I probably ought to have done earlier and read the man page for xorg.conf. Yeah, I realise: RTFM; but when the X system seems to be both spread over so many different programs and also quite different from the old pre-Xorg layout that I remember, it’s difficult to second-guess which files will have man pages named after them and which won’t.

After some hunting, the following caught my eye:

When the Xorg server is started by the “root” user, the config file search locations are as follows:

…
$HOME/xorg.conf

That’s it. I’d left an old xorg.conf hanging around in /root/, and being called—depending on how I instantiated my X session, and hence apparently not repeatably—in preference to the “standard” one in /etc/X11. So if I used the init.d script to start gdm it would ignore the dud xorg.conf; if I ran X or startx it wouldn’t; but if I then specified -config /etc/X11/xorg.conf on the command line I’d get flickers and pops from my monitors again. Sigh.

So, my rather ludicrous, meandering experiences aside, these are my humble recommendations for anyone trying to get similar hardware to mine working. I’m running Dapper Drake Ubuntu including Xorg 7.0.0 on a dual-core 2×3.0GHz Dell Optiplex GX520 with 1GB RAM and oh, so many GB HD. The monitor card is a Matrox G450 (VGA + DVI sockets), and with malice aforethought I’d probably install a similar-looking system as follows:

1. Start X, wherever possible, using /etc/init.d/gdm (re)start. This prevents loading any stray xorg.conf files. Also, just don’t leave them lying around with the name xorg.conf!
2. sudo dpkg-reconfigure -phigh xserver-xorg can be handy to version /etc/X11/xorg.conf, as it re-parses it to make sense for dpkg-reconfigure and saves the old version. Be aware that reparsing might modify any weird configuration that dpkg-reconfigure doesn’t understand.
3. Avoid older Matrox cards, such as the G200 or G250, for purely physical reasons: they don’t fit in some of the smaller Dell boxes.
4. Matrox drivers come with Xorg 7.0.0, although I used Matrox’s own drivers for reasons of not knowing beforehand. Given most people recommend the stability of former set of drivers, it looks like either might do.
5. Run lspci to see which location on the PCI bus is occupied by the Matrox card. The output lines begin with numbers like “0000:05:00.0″: these correspond to PCI addresses e.g. a BusID setting in xorg.conf of “PCI:5:0:0″ in this case. I’ve read in some places that xorg.conf sometimes wants extra zeroes e.g. “PCI:05:0:0″, but I’ve not noticed that behaviour.
6. In xorg.conf, you’ll need two device entries, two screen entries and two monitor entries. You can get away with one monitor entry if both your monitors are the same, but two will future-proof you against any modifications to your hardware.
7. The G450, unlike some cards, only occupies one PCI address. This means you’ll need the “Screen 0″ and “Screen 1″ attributes to differentiate your two Device entries in xorg.conf
8. There are other options like Xinerama (standard multiple-head management drivers) that you ought to switch on, but without these X will at least recognise your monitors are there.
9. Screen sizes, sync rates etc. should be set conservatively at first: with most modern monitors you can probably leave sync rates out, but caveat lector: it’s possible to explode old monitors using funny sync settings.
10. For reference, my xorg.conf is available at http://www.jpstacey.info/blog/files/code/xorg.conf. Obviously check the sync and resolution settings with the above caveat, but you should only need to modify this slightly for your own setup, if the hardware is at all similar.

The system works fine; really well, in fact. My old machine was dual-head, but it’s easy to get used to slowness caused (in that case) by the processing power of the machine and not realise those shackled were present. The graphics on this machine have a kind of OSX slickness to them that I’m still a little dazzled by. Now, if only I can get it to swap to kubuntu; gnome’s big buttons and stultified interface always remind me of primary school.