Trying out Drupal Aegir using Vagrant virtual machine automation

Saturday, September 17, 2011 - 20:08

Motivations

After attending the fascinating talk on Aegir by Antoine and Steven at DrupalCon London (my notes here) I really wanted to try out both the Aegir framework - it's clearly come on leaps and bounds since I first saw it two years ago in Paris - and also their apt-get based installation process. But Antoine's initial problems with his live demo (soon sorted) made me slightly trepidatious about trying it out on my work laptop.

Two months ago, I was able to take enough time out from there was a great talk at Oxford Geek Night 22 by Branden Faulls about Vagrant. This is a framework for managing Virtualbox VMs, and provisioning them using a number of frameworks, including Chef and Puppet. With Vagrant you can;

  • Download a generic-installation VM image (a sort of whitelabel machine) from vagrantbox.es (this is the slow bit)
  • Use that VM as a starting point for a number of Vagrant configuration instances
  • Turn the whitelabel VM into e.g. a fully configured reverse proxy, mail server or web server, in a repeatable way
  • Start up an instance, ssh into it, work with it, reconfigure it and tear it down when you're done

The idea is that if you do want to work with VMs, Vagrant avoids you having to have e.g. three VM images if you really want to replicate your client's live load balancer setup on your local machine: you just need one image, and three version-controlled sets of provisioning instructions. You can also use Vagrant to create your production VMs, if you're happy using Virtualbox as your live platform.

Below I provide a route to getting a fully installed Aegir server VM working on your local machine. I'm not going to go into the installation of Vagrant on your own system, as there's already pretty good documentation on how to do just that. Instead, I just want to take you through downloading and provisioning the VM, and some of the pitfalls involved.

Method

Getting a bootable VM

Once you've installed Vagrant, you need a VM image to start configuring. Images can be many hundreds of megabytes in size, so this can take quite a while to download. Using the Ubuntu 11.04 i386 server image as an example, run the following anywhere on your filesystem:

jp@chaplain:~/Documents/programming/ruby/vagrant$ vagrant box add ubuntu-1104-server-i386 \ 
http://dl.dropbox.com/u/7490647/talifun-ubuntu-11.04-server-i386.box
[vagrant] Downloading with Vagrant::Downloaders::HTTP...
[vagrant] Downloading box: http://dl.dropbox.com/u/7490647/talifun-ubuntu-11.04-server-i386.box
Progress: N% (123456 / 123456789)

The progress indicator at least gives you an idea of how long you have to wait. It will download the image to ~/.vagrant.d/boxes, so it doesn't matter where you run it from.

Then, create a new configuration directory - why not call it aegir - and cd into it. In this directory you can run vagrant init ubuntu-1104-server-i386 to create a new VagrantFile configuration file, but here's one I rolled earlier:

Vagrant::Config.run do |config|
  # The base image we just downloaded
  config.vm.box = "ubuntu-1104-server-i386"
 
  # Forward the VM "guest" port 80 to my laptop's "host" port 4567
  config.vm.forward_port("web", 80, 4567)
 
  # Enable provisioning with chef solo, specifying a cookbooks path (relative
  # to this Vagrantfile), and adding some recipes and/or roles.
  config.vm.provision :chef_solo do |chef|
    # Store both Vagrant's default cookbooks, plus our own, locally
    chef.cookbooks_path = "cookbooks"
 
    # Tell chef what recipes to run.
    chef.add_recipe("vagrant_main") # Vagrant's own main recipe
    # chef.add_recipe("aegir")        # Our custom Aegir recipe
  end
end

You can see that we've got two Chef-based recipes there, which will provision our box. I've commented out the custom Aegir recipe for now so we can get our box working. In the aegir directory, save this to VagrantFile.

Next, get the Vagrant cookbooks from http://files.vagrantup.com/getting_started/cookbooks.tar.gz. Extract them so that you now have:

  • a subfolder in aegir of cookbooks
  • and subfolders in there called cookbooks/vagrant_main, .../apache and .../apt

Now let's try out your new box! Run vagrant up to watch it configure itself, then vagrant ssh to log into it!

jp@chaplain:~/Documents/programming/ruby/vagrant/aegir$ vagrant up
[default] Importing base box 'ubuntu-1104-server-i386'...
Progress: 0%
...
jp@chaplain:~/Documents/programming/ruby/vagrant/aegir$ vagrant ssh
Welcome to Ubuntu 11.04 (GNU/Linux 2.6.38-8-generic-pae i686)
 
 * Documentation:  https://help.ubuntu.com/
Last login: Sun May  8 16:01:18 2011
vagrant@natty:~$

Once you've logged out, at this early stage you really ought to run vagrant destroy to get rid of it all (see later), but if you don't read this bit then there's probably no harm done....

Installing Aegir

Aegir's aptitude-based installation is a great trick - installing aegir with sudo apt-get install aegir - but it does require some user input. So to get round this we're going to write a new Chef cookbook, with a single default.rb recipe, that does the following:

  • Add the Koumbit package repository, install Koumbit's secure key and then update apt-get
  • Implement Debian preseeding with debconf-set-selections to avoid the Aegir installation wanting to ask any awkward questions.
  • "Silently" install Aegir and its MySQL server dependencies, with the MySQL root password left as an empty string (which, as detailed here, is a minor security risk but fine for our purposes.)

Chef recipes are easy enough to write once you know what you're doing, but the Chef documentation is atrocious. Here's how to get a cookbook for Aegir going. First, create the following directories:

  • aegir/cookbooks/aegir
    • .../recipes
    • .../templates
      • .../default

Then, save the following into a file aegir/cookbooks/aegir/recipes/default.rb:

# Add the koumbit package repository
e = execute "echo deb http://debian.koumbit.net/debian stable main > /etc/apt/sources.list.d/aegir.list" do
    action :run
end
# Add Koumbit's secure key and update
e = execute "wget http://debian.koumbit.net/debian/key.asc && apt-key add key.asc && apt-get update" do
    action :run
end
 
# Configure debian default selections for Postfix and Aegir
package "debconf-utils" do
    action [:install]
end
# Put debconf.selections.conf in /tmp, then debconf-set-selections it
template "/tmp/debconf.selections.conf" do
    source "debconf.selections.erb"
    mode 0755
    owner "root"
    group "root"
end
e = execute "debconf-set-selections /tmp/debconf.selections.conf" do
    action :run
end
 
# Now see if we can install Aegir with no user input required
package "aegir" do
    action [:install]
end

This will do all the tasks you want, but it also needs some preconfigurations in the file aegir/cookbooks/aegir/templates/default/debconf.selections.erb (don't forget that stray "/default/".) Here's some examples of what you'll need in this file:

# Postfix might not need all of these, but it's best to provide them to keep it quiet
postfix postfix/root_address    string  
postfix postfix/rfc1035_violation       boolean false
postfix postfix/mydomain_warning        boolean 
postfix postfix/mynetworks      string  127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
postfix postfix/mailname        string  MYMAILHOST.com
postfix postfix/tlsmgr_upgrade_warning  boolean 
postfix postfix/recipient_delim string  +
postfix postfix/main_mailer_type        select  Satellite system
postfix postfix/destinations    string  MYMAILHOST.com, localhost
postfix postfix/retry_upgrade_warning   boolean 
postfix postfix/kernel_version_warning  boolean 
postfix postfix/not_configured  error   
postfix postfix/mailbox_limit   string  0
postfix postfix/relayhost       string  MYLOCALMAILRELAY.com
postfix postfix/procmail        boolean false
postfix postfix/bad_recipient_delimiter error   
postfix postfix/protocols       select  all
postfix postfix/chattr  boolean false
 
# Aegir doesn't store the root password, which is good, but makes life hard
aegir-hostmaster        aegir/db_user   string  root
aegir-hostmaster        aegir/email     string  MYEMAILADDRESS@MYMAILHOST.com
aegir-hostmaster        aegir/db_host   string  localhost
aegir-hostmaster        aegir/makefile  string  
aegir-hostmaster        aegir/site      string  aegir.local

You'll need to replace anything above in capital letters with your own configuration. As you can see, we've only preconfigured two software packages:

  1. Postfix - at the time of writing, Aegir's packages will not install without a working email server on the machine. I think the only motivation for this is to be able to send the root user a one-time login email; after that, it's completely unnecessary to Aegir's functioning. This is frankly bizarre, but there you go.
  2. aegir-hostmaster - a few defaults, including the URL you're going to use to access Aegir in a browser window (note you'll actually access it at http://aegir.local:4567/ , as per the port mapping in your VagrantFile above.
  3. This leaves e.g. MySQL. As we said earlier, we leave MySQL with the default root password: an empty string. Luckily, Aegir's installation also assumes an empty string and it all mostly works out.

    4. If you get as far as a successful Aegir installation, you can actually grab these configuration options yourself with debconf-get-selections | grep postfix etc, but I've given you them above so you don't have a bootstrapping problem of wanting to work out the configuration for something you don't already have configured!

    It should just work

    Now's the moment of truth: bring up your Vagrant box with these new configurations. If you've already brought the box up with vagrant up, you might be able to get away with vagrant reload. But for sanity's sake, I recommend tearing down the box and starting it again:

    jp@chaplain:~/Documents/programming/ruby/vagrant/aegir$ vagrant destroy
[default] Forcing shutdown of VM...
[default] Destroying VM and associated drives...
jp@chaplain:~/Documents/programming/ruby/vagrant/aegir$ vagrant up
[default] Importing base box 'ubuntu-1104-server-i386'...
Progress: 0%
...

    It will take some time, but you should see lines saying e.g:

    ...
    [default] [Sat, 17 Sep 2011 14:32:35 +0000] INFO: execute[wget http://debian.koumbit.net/debian/key.asc && apt-key add key.asc && apt-get update] sh(wget http://debian.koumbit.net/debian/key.asc && apt-key add key.asc && apt-get update)
    : stdout
    ...
    [default] [Sat, 17 Sep 2011 14:32:45 +0000] INFO: execute[debconf-set-selections /tmp/debconf.selections.conf] sh(debconf-set-selections /tmp/debconf.selections.conf)
    : stdout
    ...
    [default] [Sat, 17 Sep 2011 14:32:45 +0000] INFO: Processing package[aegir] action install (aegir::default line 19)
    : stdout
    ...

    After that last line, it might take ages on a slow host machine for Aegir to install. Be patient (although at this point you can always ssh in and check with ps that everything's working - slowly!) Eventually it should all finish with no errors as before.

    Now, in your own computer's (i.e. the "host") /etc/hosts file, add a line saying:

    127.0.0.1    aegir.local

    and you should then be able to navigate to http://aegir.local:4567/ in a web browser, and see the Aegir access denied error.

    Nearly there! Finally, to get into the admin interface, log into your Vagrant box and look for the email containing your onetime login:

    jp@chaplain:~/Documents/programming/ruby/vagrant/aegir$ vagrant ssh
Welcome to Ubuntu 11.04 (GNU/Linux 2.6.38-8-generic-pae i686)
 
 * Documentation:  https://help.ubuntu.com/
Last login: Sun May  8 16:01:18 2011
vagrant@natty:~$ sudo cat /var/mail/aegir | grep user/reset | sed -e "s/local/local:4567/"
http://aegir.local:4567/user/reset/1/1316285605/59081fae44d5aeda3c767995e48d165f
vagrant@natty:~$

    That shell one-liner gives you a onetime login link, with the right port mapping (remember, HTTP port 80 is exposed on your host machine as 4567 in the Vagrant file) which you should be able to click on and get into the Aegir admin interface.

    And you're done! Have fun playing with your new Aegir installation.

    (If Chef or Vagrant throws any major wobblies on the way, you're welcome to ask me about them in the comments. Just don't be surprised if I run away screaming.)