puppetverse

January 12th, 2013  |  Published in puppet

In the spirit of getting things out in front of people and maybe attracting useful feedback, it would please me very much to offer to you a little Puppet ecosystem I call puppetverse.

I’m putting it out there less to encourage you to use it, specifically, but to note the existence of this thing called Vagrant that I find very promising and am now fiddling with quite a bit. Before leaving the matter of puppetverse, I’ll just offer that if you choose to visit that link and follow the README, you’ll end up with a small virtual laboratory that will allow you to explore the open source release of Puppet with relative ease.

Some background:

Have you ever managed a website with a few outside web developers, a testing server, a staging server and a production server? How did you do it?

I had to for a while. Part of that time was spent dealing with a staging server and production server using different Linux distributions, different versions of PHP, and different versions of MySQL. Worse, I was locked out of the staging server (it belonged to and was managed by a small support shop) so there was no way to harmonize it with production: You just did an svn commit, waited 10 minutes for the staging server to run a cron job to svn up the codebase, and checked your work.

At the development work level, it was even worse. When I arrived, the official development setup involved a malconfigured plug-n-play desktop LAMP bundle that wasn’t even up to importing our application’s database reliably. Outside developers were invited to use that method or cobble together whatever they could on their own. Sometimes, things just didn’t work on delivery and introduction to our environment. Other times they worked in the staging environment but failed in production.

Eventually I set up a virtual environment for myself that I could keep in tight harmony with production systems. We moved from our little third-party support contractor to hosting with Acquia, which I would recommend to just about anybody running Drupal who’s feeling maybe in a little over her head. Acquia’s cloud hosting stuff is very slick. That handled our testing, staging, and production discontinuities overnight.

It still left us with how to help outside developers, and I never had a satisfactory answer beyond “here’s a cloud server we keep spun up that you can bang on, we’ll periodically update the codebase there so you can tell if your code will blow up on production, probably.”

So, Vagrant

Vagrant would have solved a lot of problems for me.

Vagrant provides a way to create a minimal virtual machine that makes just enough assumptions about its own configuration to be useful in a variety of contexts that involve many divergent assumptions.

Prior to learning about Vagrant, I dealt with VMs this way:

I’d set up a minimal virtual machine and configure it to a certain point, then I’d save it somewhere. Each time I needed the base configuration, I’d copy the VM to a new file and start using it. So far, so good.

Then I’d screw something up. Maybe I’d mistakenly use my clean base VM, or I’d explore in a certain direction and discover it wasn’t a good direction, and I’d be caught with a VM that was pretty messed up and probably not worth the time to get it back to baseline.

Vagrant offers a way to create a base VM just once, then freeze it in that state, then reuse it over and over with custom configurations each time. If you take a VM based on one of those custom configurations in a direction you don’t like, that’s o.k.: You just tear it down and it goes back to baseline.

For a Puppet or Chef user, Vagrant is pretty nice because it uses either of those tools to provision freshly powered up VMs.

While we were preparing for the Puppet 3.1.0-rc1 release, I used puppetverse to help test the Ubuntu packages:

I edited the base Vagrantfile and pointed it to a different Ubuntu version. Then I’d have Vagrant power up the two VMs I’d configured (one puppet master, one puppet agent), watch them provision themselves with a simple Puppet manifest, then confirm that Puppet was working properly.

All I had to change for each Ubuntu release was the name of the base virtual machine Vagrant was to load. Once I’d done my testing, I powered down the virtual machine with Vagrant’s destroy command, which also deleted the VM’s files.

Meanwhile, the base VM images were in use on a few other projects: One for my documentation work on Hiera and one for a side project to help stand up a quick development environment. Each had its own configuration saved in a few text files, and as I stopped needing to work on each, I could discard them without worrying about saving my work.

Vagrant offers a few other nice features:

ssh into the VM “just works,” meaning there’s no cumbersome ssh key setup or remembering passwords. Just issue the command vagrant ssh and you’re logged into the VM.

Creating mount points in the guest operating system from the host operating system is pretty simple, and part of Vagrant’s big appeal to Web developers: You check out your website’s codebase into a directory on the host machine and mount it in the guest machine, so a web developer doesn’t need a Linux toolchain to work on code running on a Linux VM.

Port forwarding from guest to host is also simple to set up, so the web server running inside the guest VM is readily available to a browser running out in the host system.

All of that combines to make it incredibly simple to set up a non-admin developer with a virtual environment that closely maps to production conditions, and to allow admins to keep that environment up-to-date as conditions in production change. If the web developer can type git pull and vagrant provision on the command line, she can keep her testing environment up to date.

So, puppetverse

Which brings me back around to puppetverse, which leverages Vagrant to help me with my technical writing at Puppet Labs.

I have to be able to do a few things as I work on Puppet documentation:

  1. Test and verify any assertions I make about how Puppet works.

  2. Provide working example code that I’ve tested and verified in a current Puppet environment.

Puppetverse allows me to bring up a puppet master and a pair of agents in a virtual environment so I can test what I’m saying in my documentation or write example configuration code in virtual machines running Ubuntu (or Debian). By configuring the basics for such an ecosystem in my Vagrantfile and Puppet manifests, I don’t have to step through the tedious part of getting Puppet up and running on three machines: I did it once and I can reuse it as many times as I need. If I mess up something inside one of my virtual machines, or need to know that all the systems are back to baseline, I can use the vagrant destroy command to wipe them all out and bring them back up to baseline: No need to manually uninstall packages, edit files or reconfigure the agents.

Thanks to the ability to easily mount directories in the host filesystem inside the Vagrant virtual machines, I can store my example Puppet configuration in my puppetverse repository. That allows me to test the same configuration across multiple versions of Puppet or different operating systems, depending on the combination of base provisioning and virtual machine puppetverse happens to be running. I’ve started storing different tasks in different branches to make Puppetverse more reusable: Each branch is checked out into its own Vagrant directory with a different set of VMs running in it. Switching from work on Hiera to testing release packages is as simple as changing directories in the shell or opening a different directory as a BBEdit project.

Leave a Response

© Michael Hall, licensed under a Creative Commons Attribution-ShareAlike 3.0 United States license.