ServerSpec testing in Chef without the cruft

Where I currently work we use AWS, primarily EC2, we build our servers on demand from AWS's generic Amazon Linux AMI's. We don't bake AMI's and all our servers use the same generic launch configuration. The launch config reads the tags attached to the instance with the AWS CLI and adds then to a JSON file. The tags contain things like the service the instance is supposed to run, it's cookbook name, the cookbook version to execute and so on.

On instantiation our instances are bootstrapped against a chef server, the run list only executes a base cookbook and the chef-users cookbook, for the application install we download the cookbook noted in the tags and execute it once using chef solo, using the JSON file as inpute for attributes. We scrap our machines every day as we scale up and down as well as during deployments so there's no need to execute it more than once.

I'll detail more about this process in a later post....

We got to the point where we wanted to use ServerSpec to unit test our instances, e.g Is this server configured how we think it is, is it doing what we think it is? If not, shoot it in the head so we can fire up another one. The typical way people would achieve this is with test-kichen, berkshelf and so on, however - we didn't want to do that as it doesn't fit our workflow. We just want to build the damn machine and test it from the cookbook.

In essence, setup ServerSpec in the base cookbook, then run some tests that we expected every machine to pass, then in each application deployment cookbook we run the tests specific to that application.

We have a recipe that's run inside our base cookbook, it sets up the basic ruby environment, installs server spec and ioconsole, then runs the spec test:

default['packages'] = [ 'ruby20-devel', 'gcc-c++', 'ruby-devel' ]

# Install common utilities
package node['packages'] do
  action :install

  gem_package 'serverspec' do
    action :install

  # I don't know why, but this won't install with a gem_package function
  execute "install_ioconsole" do
    command "gem install io-console"

  execute "serverspec_run" do
    command "/usr/bin/ruby -S rspec spec/localhost/server_spec.rb"
    # For Chef Solo
    #cwd "/opt/chef/cookbooks/#{node['cookbook']}/files/default/test"
    # For Base
    cwd "/var/chef/cache/cookbooks/base/files/default/test/"

Now, inside the files directory of the cookbook there's a standard serverspec setup, created with severspec-init, that's pretty much it, if the spec test fails, it's exit's with a code !=0 and the chef run halts, also with an exit code that !=0 - our launch config detects this and sends us a notification :)

We do something similar in each of the application configuration cookbooks, install the app, unit test, move on.