Learning Puppet: Idempotence

I am learning Puppet so I wanted to try out the manifests on a local VM. So, I created a Vagrant based Arch linux VM.

Vagrant.configure("2") do |config|
 config.vm.box = "archlinux/archlinux"
 config.vm.hostname = "learn.box"
 config.vm.synced_folder "puppet/", "/home/vagrant/puppet"
end

When I tried to bring the VM up, I got the following error.

Vagrant was unable to mount VirtualBox shared folders. This is usually
because the filesystem "vboxsf" is not available. This filesystem is
made available via the VirtualBox Guest Additions and kernel module.
Please verify that these guest additions are properly installed in the
guest. This is not a bug in Vagrant and is usually caused by a faulty
Vagrant box. For context, the command attempted was:

mount -t vboxsf -o uid=1000,gid=1000 vagrant /vagrant

The error output from the command was:

: Invalid argument

It did not stop the VM from coming up so I was able to test. I created the following manifest to start with and it worked fine.

file {
 '/tmp/motd': content => 'hello puppet'
}

I moved on to actually installing a package.

class logstash{
  package{
    "logstash":
      name => "logstash",
      alias => "logstash",
      ensure => "7.0"
  }
}

I tried to apply it using the following:

puppet apply logstash.pp -v

The package was not installed. I soon figured out it was because, I was not applying the class. So, I modified the manifest to the following:

  package{
    "logstash":
      name => "logstash",
      alias => "logstash",
      ensure => "7.0"
  }

Now Puppet tried to install the package but it failed with the following error.

Error: Parameter ensure failed on Package[logstash]: Provider pacman must have features 'versionable' to set 'ensure' to '7.0' (file: /home/vagrant/logstash.pp, line: 2)

Arch linux uses pacman as its package manager and Puppet was trying to install a specific version of the package using that. However, pacman does not support different versions of the same package. So, Puppet can't ensure a specific version. So, I changed the manifest to the following:

  package{
    "logstash":
      name => "logstash",
      alias => "logstash",
      ensure => "installed"
  }
Now, Puppet attempted to install the package again and hit the following error.

Error: Execution of '/usr/bin/pacman --noconfirm --needed --noprogressbar -Sy logstash' returned 1: error: you cannot perform this operation unless you are root.
Error: /Stage[main]/Main/Package[logstash]/ensure: change from 'absent' to 'present' failed: Execution of '/usr/bin/pacman --noconfirm --needed --noprogressbar -Sy logstash' returned 1: error: you cannot perform this operation unless you are root.

So, I attempted using sudo. The process was taking time. I waited for some time then interrupted it and tried to see the logs. The logs seemed fine so I started again and the this time I got the following error.

Error: Execution of '/usr/bin/pacman --noconfirm --needed --noprogressbar -Sy logstash' returned 1: :: Synchronizing package databases...
error: failed to update core (unable to lock database)
error: failed to update extra (unable to lock database)
error: failed to update community (unable to lock database)
error: failed to synchronize all databases
Error: /Stage[main]/Main/Package[logstash]/ensure: change from 'absent' to 'present' failed: Execution of '/usr/bin/pacman --noconfirm --needed --noprogressbar -Sy logstash' returned 1: :: Synchronizing package databases...
error: failed to update core (unable to lock database)
error: failed to update extra (unable to lock database)
error: failed to update community (unable to lock database)
error: failed to synchronize all databases

This is specific to pacman. As I had interrupted the execution, the lock file was not deleted, i.e. pacman had not cleaned up properly. I manually removed the lock file from /var/lib/pacman/db.lck and tried to apply the manifest again. It completed successfully and logstash was installed on the VM. Now, it is established that the script works but it might need manual intervention. In other words, the manifest needs to be idempotent so that manual intervention is minimal.

To achieve that, I modified the manifest as follows.

  package{
    "logstash":
      name => "logstash",
      alias => "logstash",
      ensure => "installed"
  }
  file{
    "/var/lib/pacman/db.lck":
      path => "/var/lib/pacman/db.lck",
      name => "/var/lib/pacman/db.lck",
      ensure => "absent"
  }
To test that the manifest is idempotent, I removed logstash and applied the manifest. Midway, I interrupted the execution and re-applied the manifest. This time re-application succeeded. The logs show that the lock file was created when the manifest was applied the previous time and interrupted. It was deleted when the manifest was applied again.

Info: Applying configuration version '1555432879'
Info: Computing checksum on file /var/lib/pacman/db.lck
Info: /Stage[main]/Main/File[/var/lib/pacman/db.lck]: Filebucketed /var/lib/pacman/db.lck to puppet with sum d41d8cd98f00b204e9800998ecf8427e
Notice: /Stage[main]/Main/File[/var/lib/pacman/db.lck]/ensure: removed
Notice: Applied catalog in 0.25 seconds

Key take-aways:

1. Puppet is a framework that depends on providers to install packages and its capabilities are as good as the providers.
2. Without idempotent behaviour, Puppet manifests will not be achieve much of automation.