Slaughter is a tool which is used to automate the configuration, maintenance, and management of a large number of servers.

Slaughter is implemented in perl, a language which has traditionally been suited to system-administration tasks, and thanks to CPAN there exists a wide range of utilities and libraries which are available to you.

Although Slaughter is a modern tool, written from scratch, it was significantly inspired by CFEngine. (The supplied primitives are very similar, but the client-server model was replaced by the more flexible system of transports.)

Slaughter is very small both in terms of the number of lines of code, and the concepts which are need to understand and use it. (These are important considerations when you're evaluating a new configuration-management tool, as you might need to experiment with several systems before picking the most suitable.)

However despite being simple to use slaughter is flexible enough to allow you to carry out a wide range of maintenance upon a large number of systems.

Slaughter is a client-pull application, which means that each machine which has slaughter installed upon it is expected to schedule itself. There is no single-server which is in charge of scheduling, control, or mediation. This makes the deployment very straight-forward. (It is true that a central "thing" is required, such that clients can pull their policies/files/modules from it, however this might not be a traditional server, it could just be a remote github repository, or a HTTP-server)

In terms of requirements the dependencies of slaughter are very minimal; you merely require a working Perl environment, and a small number of very commonly available modules.

When the slaughter command is invoked several things will happen in quick succession:

• The parsing of the global configuration file, if present. (See Appendix A)

• The parsing of the command line arguments, if any. (See Appendix B)

• The discovery of information about the current host. (See Appendix D)

• The downloading of the policy file(s) from the central server.

• The creation of an executable script, using the policies obtained in the previous step.

• The execution of the generated script.

• Cleanup of temporary downloads, and the generated script.

The end result is that policies stored on a single central host are executed upon the local machine.

It is assumed you'll schedule slaughter via cron, to ensure it runs regularly.

With slaughter you fetch policies from a central location, and these are executed on each system you manage. The policies allow you to carry out configuration changes, etc.

The following example installs and configures molly-guard upon managed systems. molly-guard is a simple package which prompts you to enter the current hostname prior to allowing the commands "shutdown", "reboot" to complete - it can save you from rebooting the wrong server.

#
# Ensure we have don't reboot the wrong server(s).
#
#
# Install the package
#
if ( !PackageInstalled( Package => "molly-guard" ) )
{
 InstallPackage( Package => "molly-guard" );
}
#
# Enable the package.
#
if ( -e "/etc/molly-guard/rc" )
{
 AppendIfMissing( File => "/etc/molly-guard/rc",
 Line => "ALWAYS_QUERY_HOSTNAME=true" );
}

The remainder of this documentation will explain how this works, and how to configure Slaughter on your systems. This example was merely designed to express concisely how you might control systems with Slaughter.

Assuming you're running the Squeeze release of Debian GNU/Linux you can install by running the following commands:

echo "deb http://packages.steve.org.uk/slaughter/squeeze/ ./" >> /etc/apt/sources.list.d/slaughter.list
apt-get update
apt-get install slaughter2-client

At the time of this documents creation the most recent release was 2.6. Installing this can be carried out via:

$ wget http://www.steve.org.uk/Software/slaughter/slaughter-2.6.tar.gz
$ tar zxf slaughter-2.6.tar.gz
$ cd slaughter-2.6
$ sudo make install

The whole point of a server-automation tool is that you'll write your list of configuration steps, recipes, or actions in one place and then they will be applied over a wide range of hosts.

In slaughter the list of configuration steps is known as a policy, and a policy is comprised of distinct actions that are applied to each host. These policies are written in 100% pure perl code - with a range of appropriate primitives supplied to allow you to carry out your tasks.

When the slaughter client runs it gathers information about the current host before it begins to fetch and execute the remote policies.

The meta-information includes such things as details of the IP addresses the host has, the hostname, the free RAM, & etc. The information obtained is made available to your policies, and is documented in Appendix D

Because the policies you write are going to be pure perl it might be useful to abstract out some of the nuts and bolts, the actual individual steps, into a simple module. In slaughter this is permissible, and there is special functionality to allow you to fetch these modules from the central server.

For example you might find you wish to configure Apache and that might mean enabling some modules, disabling others, and creating a new "site" for it to serve. You could easily write the code to do this in your policies, but it would make slightly more sense to create an "Apache Module" to cover the common cases, and then invoke that in your policies.

The Slaughter distribution does contain a simple Apache module, and you can see a couple more in the live repository I host.

As has been previously mentioned slaughter is a client-pull system. This means that when slaughter runs it will attempt to connect to a central server to download the policies which should be applied to the local system.

Because there is no "slaughter-server" we've had to create a system by which these downloads can be carried out. We came up with a flexible approach which means you can configure the slaughter client to download the policies over one of several available mechanisms:

• Via HTTP-fetches, from a web-server.

• Via a rsync fetch from a remote rsync server.

• Via the cloning/checkout of a remote revision control system such as git, subversion, or mercurial.

The transport is nothing more than the means by which the remote policies are fetched. So if you're fetching them via HTTP-fetches then you're using the HTTP-transport, etc.

As briefly discussed in Section 3.4 there are many options when it comes to fetching the policies from the central server.

To keep things simple, and to demonstrate more than one approach our server setup will consist of configuring two transports: HTTP & git.

With these two quick examples we'll have discussed the server-setup enough for the moment, although we'll come back it later. For the moment it is sufficient to realise what we're doing:

• We'll configure two different ways of hosting the file policies/default.policy.

• We'll show that we don't need to be running anything complex on the server-side. (Specifically we don't need any custom daemons, and nor do we need to have slaughter itself installed upon the master-host.)

• We'll implicitly hint that you can store your policies in a Github repository if you wish, which gives you high-availability and revision control.

# cd /var/www/
# mkdir -p slaughter/policies/
# echo 'print "I am alive\n";' > slaughter/policies/default.policy

git clone https://github.com/skx/slaughter-example.git

Once you've got an appropriate server configured (Section 4.1) we can actually begin to use the slaughter client itself. (See Chapter 2 if you've not yet installed the client.)

Because it is anticipated that you'll be making system-wide changes with your slaughter policies it is worth noting explicitly that it will refuse to run, except in minimal ways, unless you're root.

To launch the slaughter client we merely need to invoke slaughter with two pieces of information:

• The transport to use, which in our case will first be "git" and then be "http".

• The location from which to fetch the initial policy, policies/default.policy.

In the previous section we briefly discussed the setup of a HTTP-server, and we also documented the location of a git repository which contains a sample policy. We'll use the git transport first because it doesn't require you to perform any action - just install the slaughter software and the git command.

We'll launch the client like this:

Example 4-3. Running slaughter against a git repository.

# slaughter --transport=git --prefix=https://github.com/skx/slaughter-example.git
This is perl code and I am alive!
#

We've specified the location of the remote tree (which could contain top-level directories; files/, modules/ and policies/, but in this case we've only got the latter). We've also specified the transport to use, that has lead to a temporary directory being created and the remote repository cloned into it. Once that happened the script was executed which lead to the output of the message "This is perl code and I am alive!".

Using the HTTP-location will be almost the same, except in that case the prefix will be specified as the URL of the location where the policies live, and the transport will be updated to match:

Example 4-4. Running slaughter against a HTTP server.

# slaughter --transport=http --prefix=https://example.com/slaughter/
..
I am alive!
#

In the HTTP-example the same series of steps occurred; the remote policy was downloaded into a temporary directory, wrapped up to become an executable and then executed.

The behaviour of the client can be altered by the addition of command-line flags (documented in Appendix B). Adding the two flags "--no-delete" and "--no-execute" will prevent the generated script from being executed and from being removed allowing you to examine it: Example 4-5. Keeping the generated script around. # slaughter --transport=http --prefix=https://example.com/slaughter/ \ --no-execute --no-delete --verbose .. Policy written to: /tmp/I6mY_AgUxv Not launching script due to --no-execute. #

Example 5-1. Simple Primitive Example.

#
# Append an administrative IP address to /etc/hosts.allow.
#
# This will override any blacklisting denyhosts tries to perform.
#
AppendIfMissing( File => "/etc/hosts.allow",
 Line => "All: 1.2.3.4" );

We've demonstrated previously (Chapter 4) that when the slaughter client launches it will fetch and execute the contents of the file policies/default.policy. It is expected that you'll not actually write any of your code in there, although you certainly can if you wish, instead we assume the initial file will just pull in your real policies.

The following example should make that clear:

#
# This is policies/default.policy
#
#
# Ensure slaughter runs on each node once an hour.
#
FetchPolicy "slaughter.policy";
#
# If we have MySQL then we should have backups of the contents.
#
FetchPolicy "mysql.policy";
#
# ... more
#
#
# Fetch the host-specific policy, if present.
#
FetchPolicy "hosts/$fqdn.policy";

This policy references several distinct files, each of which will be fetched via the transport. In the example above we would expect our master-server to look like this:

$ cd policies/
$ ls
default.policy hosts/ mysql.policy slaughter.policy

You'll notice that the final inclusion uses the variable $fqdn to refer to the current host-name. If the fetching of this policy fails it will be silently ignored. (See Appendix D for more on defined variables).

The FetchPolicy directive will only work in the default.policy file. Policies are not recursively processed for inclusion.

Example 6-1. Fetching a file from the central server

#
# Fetch the master denyhosts.conf file from the server
#
if ( FetchFile( Source => "/etc/denyhosts.conf",
 Dest => "/etc/denyhosts.conf",
 Owner => "root",
 Group => "root",
 Mode => "644",
 Expand => "false") )
{
 RunCommand( Cmd => "/etc/init.d/denyhosts restart" );
}

When you invoke the FetchFile primitive you can set "Expand => True", which will cause the file being retrieved to be template-expanded.

The template expansion happens via the Perl Text::Template module, and this allows you to set the contents of the file with ease.

This facility is demonstrated in the FileCopy example, located in Section 9.2

Slaughter has built-in support for fetching per-host files, this allows you to easily serve different files to different clients, without having to make that conditional work part of your policy.

If a FetchFile primitive is invoked to fetch /files/foo it is actually expanded into request for each of the following files (with the first successful fetch terminating the search):

• foo.$fqdn - Where "fqdn" is the fully-qualified domain name of the client.

• foo.$hostname - Where "hostname" is the unqualified hostname.

• foo.$role - Where "role" is role assigned to this client.

• foo.$os - Where "os" is the operating system code-name.

• foo.$arch - Where "arch" is the architecture of the client, e.g. i386.

• foo

These variables, with the exception of $role, are all set automatically at run-time, and are documented in Appendix D.

$role is special because it will only exist if you set it for a host, via either the command-line, or the configuration file.

Example 7-1. Configuring Apache with slaughter

#
# If apache isn't installed, then get it
#
if ( ! -x "/etc/init.d/apache2" )
{
 InstallPackage( Package => "apache2-mpm-worker" );
}
#
# Restart apache?
#
my $restart = 0;
#
# If mod_rewrite isn't installed, enable it.
#
if ( ! -e "/etc/apache2/mods-enabled/rewrite.load" )
{
 RunCommand( Cmd => "a2enmod rewrite" );
 $restart = 1;
}
#
# ...
#
if ( $restart )
{
 RunCommand( Cmd => "/etc/init.d/apache2 restart" );
}

The previous section demonstrated how you might manage Apache "manually", and although this is possible there will be a lot of Apache-specific code in your policies if you do that.

The alternative is to write a simple Perl module, which can be used to contain the Apache-specific code. If you do that you can simplify your policies by hiding the individual steps, as in this example:

#
# We have the Apache module loaded, so:
#
my $a = Apache->new();
$a->enable_module( "rewrite" );
$a->enable_module( "rpaf" );
#
# ..
#

If you wish to use a module like this you need to do three things:

• Write the module. (There is an Apache module included in the slaughter distribution.)

• Save it in your slaughter repository, beneath the top-level directory modules/

• Load the module in your default.policy file, using the FetchModule directive.

There is a sample Github repository which demonstrates this, and you can see it tested like so:

# slaughter --transport=git --prefix=https://github.com/skx/slaughter-module-example
Hello, world

The FetchModule directive will only work in the default.policy file.

Ignoring the transports, and the meta-information gathered at run-time, the core of the code is the implementation of the primitives.

Each of the primitives is written as a standard Perl subroutine, exported to the main name-space via some module trickery. The routines are exported whenever you load the Slaughter module, which means you can use them outside of a policy script if you so-wished:

#!/usr/bin/perl -w
use strict;
use warnings;
use Slaughter;
RunCommand( Cmd => "/usr/bin/uptime" );

Internally the Slaughter.pm module does very little, it merely loads the real implementation of the primitives from the platform-specific libraries. For example on a Debian GNU/Linux host the code "use Slaughter;" actually becomes:

use Slaughter::API::generic;
use Slaughter::API::linux;
use Slaughter::API::Local::linux;

The Slaughter::API::Local namespace won't exist by default, so errors in loading it will be ignored, however it does provide a great starting point for the addition of local-primitives you might wish to implement.

Adding new primitives is documented in the HACKING file, distributed with Slaughter, but in brief you have two choices:

• Create the Slaughter::API::Local::linux module, to make your primitives available in-house.

• Update the Slaughter::API::generic module and submit it for inclusion in future releases of Slaughter.

The transports are implemented using a plugin-like system. Each transport is implemented via a module beneath the Slaughter::Transport namespace.

Transport modules will be dynamically discovered, created, and invoked at run-time based upon their module names. Thanks to a standard API adding a new one is very straight-forward. You merely need to give your module the appropriate name, and implement the following API methods:

If your transport is based upon executing an external program, such as "git", or "rsync", there is a helper available to simplify the code considerably, see the existing git.pm, hg.pm files.

The information gathered at run-time also comes from a series of modules. There is a module prefix of Slaughter::Info beneath which platform-specific modules exist.

For example the following snippet will gather all the information available for a GNU/Linux host:

use Slaughter::Info::linux;
my $obj = Slaughter::Info::linux->new();
my $data = $obj->getInformation();
# use data here ..

You can augment this information by creating your own local module. When a platform-specific module is loaded an additional module is looked for too. For example if your system is a GNU/Linux host the following two modules will be loaded:

use Slaughter::Info::linux;
use Slaughter::Info::Local::linux;

Create the local module, and implement the getInformation method, and your extra data will be available for use in all your policy scripts.

There are several existing configuration management tools, most notably CFEngine and Puppet.

Because CFengine and Puppet are pretty heavyweight there are also some simpler alternatives, like slaughter:

• Ansible, which uses pure SSH to manage hosts.

• Bcfg2 another mature and stable project.

• Salt, configuration via YAML.

• Fabric, a python-based solution for writing adhoc scripts which manipulate hosts over SSH.

Example A-1. Slaughter configuration file.

#
# The transport to use, in this case 'rsync'.
#
transport = rsync
#
# The location from which to fetch our policies/files/modules
#
prefix = rsync://www.steve.org.uk/slaughter
#
# The (optional) role for this client. You could setup "webserver"
# for some hosts, and "database" for others.
#
role = example
#
# Sleep up to a minute before launching.
#
delay = 60
#
# Don't be verbose.
#
verbose = 0

Table B-1. Command line flags

Example D-1. Accessing meta-information in a policy script.

print $fqdn . "\n";
print $template->{'fqdn'} . "\n";

The complete list of meta-information available on the current host may be dumped by executing "slaughter --dump":

root@da-web4 ~ # slaughter --dump
 $VAR1 = {
 'load_average_5' => '0.04',
 'verbose' => 0,
 'arch' => 'amd64',
 'memfree' => '3736436',
 'hostname' => 'da-web4',
 'cpumodel' => 'QEMU Virtual CPU version 0.15.0',
 ..
 };

Dumping the variables is a good way to get a feel for what has been discovered upon your system(s), because the variables really will be host-specific.

The following is a list of the variables upon my current system, with a brief explanation of their meanings.

Note that some variables might not necessarily be displayed, for example softwareraid will only be defined if there is software-RAID detected upon the current host.