Master of Puppets

Sunday, June 29th, 2008 12:09AM UTC

PuppetNo this post isn’t about the Metallica album, it’s about the provisioning system we use named Puppet.  It allows us to automate many of the system administration tasks, to a level that was previously very costly or hard to do.  It’s a declarative programming language that, at a very high level, describes the state you want to “Make it so“.  This includes applications installed/removed, files configured and dependencies with other applications.  To put simply, Puppet is the glue between an operating system’s package manager and the configuration needed to make your specific setup work.  Puppet binds these two together and makes the task a consistent and repeatable process. The Puppet scripts (known as recipes) are operating system independent, and can easily apply to other operating systems with little or no changes.  Since we are primarily a CentOS/RHEL shop, this blog discuss our specific setup.  I believe an example recipe will speak volumes:

package { "openssh-server":
      ensure  => latest,
      notify  => Service["sshd"],
file { "sshd_config":
       name     => "/etc/ssh/sshd_config",
       checksum => md5,
       ensure   => present,
       owner    => 'root',
       group    => 'root',
       mode     => '0600',
       require  => Package["openssh-server"],
       notify   => Service["sshd"],
service { "sshd":
       name       => "sshd",
       ensure     => running,
       enable     => true,
       hasrestart => true,
       hasstatus  => true,
       require => Package["openssh-server"],

This 22 line recipe does all of the following:

  1. Installs the openssh-server RPM via ‘yum’.
  2. Automatically upgrades the openssh-server RPM, if a newer version is available.
  3. Makes sure the sshd_config configuration file exists and has the proper permissions.
  4. Ensure the sshd server starts at boot time.
  5. Ensure the sshd server is currently running.
  6. If either the RPM is upgraded OR the sshd_config file changes restart the sshd service.
  7. If during any time puppet runs again and the server doesn’t match the recipe it will change it back to this state.
  8. Perform this task on every server you specify.

While the above recipe hasn’t been tested on other Unix platforms, only minor changes would be required.  Previously to do this you needed to create custom shell scripts, use Cfengine, purchase an expensive software automation tool, or manually perform this on each server installation.  Most options are hacks and not as graceful as Puppet. Package managers, while moved Unix administration into the 21st century (instead of the medieval times of compiling software), still have some warts.  Specifically package managers lack:

  1. a good updating procedure.  Installations are well covered.
  2. passing your own configuration files specific to your needs/wants.
  3. performing the tasks in a specific order, or making sure specific actions occur before an application is installed
  4. a service is running and will run at boot time

Things like ‘yum’ on CentOS/RHEL addressed #1 and #3 somewhat, but didn’t address configuration files, and the state of the service.  Before Puppet, it required creating custom RPMs.  With custom RPMs, the issue then became when updates occurred from the distro provider.

Puppet makes system administration a programming task, rather then manual labor process. It’s still very common to see administrators use a SSH prompt to manage each server. Manually performing administration is a time consuming and error prone process.  Puppet allows us a transfer of our best practices, apply our administration experience to the server’s configuration, and allows us to make network-wide installations with ease.

I agree with the notion; “Operations: The New Secret Sauce” (article #1 , article #2).  Puppet makes deployment of new VPS instances quick and exact.   System administrators are skilled individuals, great with keeping operations running smoothly, but usually not good with automation.  Puppet allows to transfer an administrator’s knowledge into a repeatable process.  Automating system administration is the next advancement with Web 2.0, Cloud Computing, SaaS, or whatever the latest trend may be called.  When using our HostCube service, Puppet puts the power of a large operations center like Google in the hands of much smaller companies, Cloud computing, while may address the quick provisioning of hardware, it doesn’t address operations. The ability to automate the install, configure, patch, monitor and backup are important aspects and HostCube does for you automatically and seamlessly.  The bigger and much more complex problem is system administration, not hardware provisioning.

Puppet allows us to Malkovich a setup, over and over and over again.

Create A Comment

Comments must be relevant to the posting.

(not dsplayed publicly)