Why you should be using Drush make

Drush make is an unassuming bit of Drupal-associated technology that sits at the heart of practically every distribution hosted on drupal.org. It's the ideal way to build a codebase, really: a clear shorthand format for all the manual effort of downloading the right versions of this and that, and putting it together exactly how you want it.

You might have heard of Drush make; you might even have dabbled with it yourself; in fact, you might even be using it in your projects. But do you know the many, many ways in which it's so very, very good? Here are a few.

Get amazing "compression" on your codebase

Drush make turns megabytes of contributed site code into kilobytes of makefile entries. The codebase for the Views project can be downloaded as a 1.56MB gzipped archive at the time of writing; an equivalent makefile entry -'projects[views][version] = 3.7' - is 31 bytes, including the newline.

You've probably worked out where I'm going with this: the difference between those two file sizes can be said to define a kind of pseudo-"compression factor"; in this case, nearly 53,000. It's cheating, of course: gunzipping is not the same as automating a dynamic download. But there's a clear case for wanting to pass around - and maintain - a multi-kilobyte makefile in your development team, rather than a multi-megabyte gzip or its rsync equivalent.

Utilize drupal.org's version control

When work is done on a contributed module, d.o tracks those changes for you, in its own git repositories. If you're storing your own copies of those modules, in your own version control, then unless you're doing something very clever with git submodule (which I don't recommend) then you're basically repeating drupal.org's version control yourself, with no clear advantage. Not only can you start off with quite a large codebase, but after a year or two of incremental upgrades, your repository can be much bigger still.

The difference between two releases of a big module can sometimes be made up of many atomic changesets. Especially when something goes wrong, it can be useful to step through those changesets and see what might have caused the problem: regression testing, effectively. If you've flattened them all into one re-commit into your own repository, then you're out of luck.

Avoid forking code

When you take code from drupal.org, remove its original VCS history, and then add it to your own VCS repository, you have effectively severed any formal ties between your copy and the original code. This is a worse consequence them the previously discussed problem of mere history duplication or flattening: the new copy is effectively forked, in principle if not in practice.

That means that, for the rest of the lifetime of the website, you and your team and any freelancers should all resist hacking that module. Because if you do, and if there's no obvious agreed sign that you've done so, then the next developer to take on the project has to know what you've hacked; or you need to use a special subfolder; or that next developer needs to have the presence of mind to use something like md5checkto see what's changed. Even then, unless you leave perfectly crafted and unambiguous log messages, they will still have to work out why you hacked it.

Even if your only "hacks" are patches applied from drupal.org issue queues, you've no guarantee they'll be approved by the time the module needs upgrading to a new d.o release. A simple security sweep of out-of-date modules could roll back your hack, which was so important that you just had to fork the code, and - often subtly - cripple your site.

Drush make avoids all of this in the first instance, by simply preventing you hacking and then re-committing contributed code. Whenever the site is rebuilt, it happens on top of a scorched earth: your hacks are gone. Although that makes rapid, indiscriminate changes harder, it also means you can rely on your codebase to always consist of the d.o standard contributions.

Track, apply and contribute patches

However, just because Drush make is preventing you from in situ forking and hacking of contributed modules, as part of your own version control, that doesn't mean you can't still do it. But rather than trusting your luck to a code fork, Drush make lets you apply patches to any given d.o project, every time the site is built, as a list of URLs to patch files. If you've found a bug, fire up a d.o issue and submit a patch file. Then reference that URL in your makefile, and every time you redeploy your site, the project will be re-patched.

You could even host the patches on your own server(s), but ask yourself: why? However foolish your reasons for patching might seem to you, they're no less foolish than ending up in a situation where some proprietary fileserver has fallen over, but d.o is chugging along and if only you had submitted your patch file there, with a suitably sheepish explanation on the issue....

Look at it this way: for years at Torchbox, we tried to work out how we might contribute to the community, especially through code. It now turns out that, by moving to a make-powered workflow, we have effectively forced ourselves to contribute more via patches than we've contributed by all other methods combined. It's simply easier to post your fix on d.o than not. Drush make encourages giving back.

And, for the really adventurous: you can even consider hacking I mean patching core; safe in the knowledge that, one by one, each patch can be rolled back. 

Reference non-d.o modules and themes

The syntax of a makefile has been understandably made as straightforward as possible for the most likely use case, that of a module or theme being downloaded from drupal.org. That's why a project like Views can be "compressed" into a makefile reference totalling so few bytes. However, that doesn't mean that you're restricted to projects - modules, themes or profiles - hosted on d.o alone.

The syntax is easily extensible to cover projects hosted anywhere, as long as they're either (a) maintained using a popular VCS (CVS, subversion, git, bazaar) or (b) accessible at a persisting URL in a popular archive format (tar, gzip, zip). So you can host projects on github, Sourceforge or your own hosting provision, and Drush make is able to use them to build your codebase.

Control 3rd-party library versions

Through the libraries API, Drush make can generate and later interpret entries for the third-party libraries you tend to use: TinyMCE, jQuery UI, PHPMailer; if it can be sourced as a URL or VCS repository, Drush make can find it.

(The one disadvantage of this is when e.g. Sourceforge changed all of their download URLs. Although that hasn't happened very often, unlike the URL of e.g. Colorbox. All of this can be improved by local mirrors, although see the discussion on patches above: if you're going to mirror libraries, then ensure you're moving to a more reliable source when you do so.)

Deploy everwhere

Well, not quite everywhere. But any server you can install Drush on, you can build a make-powered website on. Drush doesn't need sudo: if you've got access to where the codebase is kept, then if nothing else you can build your site in a temporary location and swap things round (more on that in a later post....)

So why isn't everyone using Drush make?

I'd claim that Drush make is absolutely the perfect tool for your initial site build. And if you combine it with e.g. an install profile, then you've got a way of building an entire codebase, ready for a potentially complex installation.

That said, there are a few reasons why Drush make isn't a panacea for the requirements of ongoing (re-)deployment. Depending on your existing deployment workflow, it might be difficult to fit makefiles into your way of doing things. The cleverer your methods - rsyncs, git hooks, anything like that - then the harder it will be to incorporate a build phase. Makefiles literally build the contributed code, plus libraries, and nothing else. And an existing site doesn't always lend itself to conversion into a makefile at a later date (not least because by then you might have already forked contrib, which has to be unravelled by hand!)

But I hope to talk about those a bit more in a future post. Right now, even if not everyone is using Drush make, you can still give it a go. Go on: if you don't have Drush, install it now (regardless of whether or not you use Drush make, Drush alone will change your life). Drush make comes with it, so next thing is to start playing!

On your next site build, try using a makefile; keep it in sites/default with the site's custom code. Or maybe generate a makefile for your production site, and then try redeploying it yourself on a different server. It might take a few goes, but when it works then not only do you get all the advantages above, but also a make-powered site build is a wonderful thing to watch.