Ansible Testing Using Molecule with Ansible as Verifier

0
16350

In this tutorial, we would be learning how to test infrastructure code written in Ansible using a testing framework known as Molecule. Inside Molecule we would be making use of Ansible as our verifier which is something I wasn’t able to find anywhere yet. Let’s do this!

Table of Contents

1. Introduction

Ansible is an Open-source IT automation tool used in Configuration Management, Application Deployment , Infrastructure Service Orchestration, Cloud Provisioning and many more. It is an easy-to-use tool yet makes very complex repetitive IT automation tasks look easy. It can be used for multi-tier IT application deployment.

Just like in any other IT sphere, testing is inevitable. An untested infrastructure can be easily written off as an already broken infrastructure. By testing infrastructure code, we ensure the development of production-grade infrastructure code that is void of errors and bugs which can be very costly if not discovered before production.

Molecule is a framework designed to aid the development and testing of roles in Ansible. As of September 26, Ansible announced its adoption of Molecule and Ansible-lint as an official Red Hat Ansible projects. This shows the confidence the Red Hat community has in this tool and the amount of work they are putting into making it even better and better.

Molecule makes it possible to test roles across different instances, operating systems and distributions, virtualisation providers, test frameworks and testing scenarios.
Molecule supports a TDD-like form of testing infrastructure code. In this tutorial, we would look at the lifecycle that molecule testing should follow, according to my opinion.

2. Installing Molecule

It is assumed that the reader already has some experience with package management on UNIX systems.

Molecule requires the following packages to function:

  • Python 2.7 or Python 3.5 or greater (For this tutorial, we would be using Python 3.7)
  • Ansible 2.5 or greater (For this tutorial, we would be using Ansible 2.9.6)
  • Docker (latest version)

Pip is the only officially supported package manager for the installation of Molecule. If you’re using Python 2.7.9 (or greater) or Python 3.4 (or greater), then PIP comes installed with Python by default.

To install Molecule using pip:

See molecule installation for more installation tips. To check if molecule was properly installed, run $ molecule --version.

When testing Ansible Playbooks, it is important to understand that we do not test that Ansible works, i.e that Ansible has done its job of, for example, creating a file, a user or a group, rather what we test is that our intent, as expressed in plain English, corresponds to Ansible’s declarative language, i.e what has been created is exactly what we wished to create and there was no human errors (e.g, typographical errors or omissions).

3. Initialising Molecule in Ansible Roles

There are two ways of initialising Molecule for testing Ansible roles:

a. Initiating Molecule with a new Ansible Role
Molecules makes use of Ansible Galaxy to generate the standard Ansible role layout. To create a new role with Molecule:

b. Initiating Molecule for an already existing Ansible role
Molecule can also be used to test already existing roles, simply enter the following command in the root directory where the role is located or inside the role directory making sure that the role names match:

Regardless of how you initialise Molecule, a new Molecule folder is added to the root folder of the project. The resulting folder layout is as follows:

Below we would discuss the contents of the Molecule folder and their usage:

molecule.yml

In the molecule.yml, we specify all the molecule configuration needed to test the roles.

dependency:
This is the dependency manager which is responsible for resolving all role dependencies. Ansible Galaxy is the default dependency used by Molecule. Other dependency managers are Shell and Gilt. By default, dependency  is set to true, but can be disabled by setting enabled to false.

driver:
Driver tells Molecule where we want our test instances to come from. Molecule’s default driver is Docker but also has other options such as: AWS, Azure, Google Cloud, Vagrant, Hetzner Cloud and many more. See molecule drivers for more on this.

platforms:
The platforms key indicate what type of instances we want to launch to test our roles. This should correspond to the driver, for example, in the above snippet, it says what type of docker image we want to launch.

provisioner:
The provisioner is the tool that runs the converge.yml file against all launched instances (specified in platforms). The only supported provisioner is Ansible.

verifier:
The verifier is the tool that validates our roles. This verifier runs the verify.yml file to assert that our instance’s actual state (converge state) matches the desired state (verify state). The default verifier is Ansible but there are also other verifiers, such as: testinfra, goss and inspec. Earlier, testinfra was the default verifier but because of the need for a unified testing UX and to avoid the need to learn another language, Python, in the case of testinfra, the community has decided that Ansible becomes the default verifier and I support this decision. See git issue here.

Additional keys that are not generated by default are lint and scenario. These keys can be added to the molecule.yml file at will.

lint:
Lint represents what tool Molecule must use to ensure that declarative errors, bugs, stylistic errors, and suspicious constructs are spotted and flagged. Popular lints are yamllint, ansible-lint, flake8, etc.

scenario:
Scenario describes the lifecycle of the Molecule test. The test scenario is customisable as the steps in the sequence can be interchanged or commented out to suit whatever scenario needed. Every role should have a default scenario which is called default.
Unless otherwise stated, the scenario name is usually the name of the directory where the Molecule files are located. Below is the default scenario run when we run the corresponding command sequence:

From they above snippet, we can tell what happens when we run a Molecule command, for example, $ molecule create would run then create_sequence while $ molecule check would run the check_sequence and so on.

In general, we only add the scenario key when we want to customise our scenario else it is unnecessary as it is the default scenario and hence, implicit.

converge.yml

The converge.yml file, just as the name implies, is used to convert the state of the instances to the real state declared in the actual roles to be tested. It runs the single converge play on the launched instances. This file is run when we run the $ molecule converge  command.

verify.yml

The verify.yml file runs the play that calls the test roles. These roles are used to validate that the already converged instance state matches the desired state. This file is run when we run the $ molecule verify command.

INSTALL.rst

This file contains instructions for additional dependencies needed for a successful interaction between Molecule and the driver.

4. Writing Ansible Tests with Ansible Verifier

In this section, we would practice what in my opinion should be the workflow for testing Ansible roles using the Ansible verifier in Molecule.

Running $ molecule test runs the entire test_sequence but always destroys the created instance(s) at the end and this can consume a lot of time considering we have to recreate the instances everytime we make changes to our actual or test roles.
Therefore, the workflow to follow which suits the Given-When-Then approach of BDD is:

In the above snippet, the given phase doesn’t change often, so we just create the instance(s) once. After that, we iterate between when and then phases until our tests are all verified and error free.

In this tutorial, our goal is to implement TDD while testing our infrastructure. We would be writing unit tests. So say we wanted to implement a role called  alpha-services, that accomplished the following tasks:

  • Task 1: Installs Java-1.8 on the host machine
  • Task 2: Creates a dir at path /var/log/tomcat belonging to owner ‘tomcat’, group ‘tomcat’ and of mode ‘0755’
  • Task 3: Installs, starts and enables httpd
  • Task 4: Copy a template file from template/tomcat/context.xml to /etc/tomcat/context.xml

First, we create the role using the molecule init command:

This creates a similar folder layout as shown earlier. Next, we create alpha-services/molecule/default/roles/test_alpha-services  path:

This is where our test roles would be contained. Inside test_alpha-services directory, we
create our test roles using the standard Ansible role layout (we create only the folders that we require for testing, in this case, defaults, tasks and vars). Each created folder should have its main.yml file. For the individual task, we would create separate yml files to differentiate them for each other, prefixing test_ to the task name. For example, the task to install java would be called test_java.yml .

We would then be left with the following folder layout:

We configure the molecule.yml file:

We leave the converge.yml file as is:

We edit the verify.yml file to include our test_provisioner role:

GIVEN PHASE: We run $ molecule create  to create the instances.

WHEN PHASE: We run $ molecule converge to run the actual roles which are yet to be implemented. This doesn’t effect any change on the created instance.

Now we go ahead to develop the roles.
Following TDD approach, we first create the tests and check that they fail before implementing the roles that they are testing.

TASK 1: Install Java-1.8.0 on the host machine

Check Java package status task tries to install java-1.8.0 in check mode and registers the result of that operation in pkg_status. In actual sense, if java-1.8.0  is already installed, the assertion not pkg_status.changed  would return true  because the state would not have changed. Thanks to Juan Antonio for this tip.

We include the test_java.yml tasks in alpha-services/molecule/default/roles/test_alpha-services/tasks/main.yml file like so:

THEN PHASE: We run $ molecule verify. As expected, it should fail with the following error:

Now we implement Task 1. We first create an alpha-services/tasks/java.yml file and populate it with the following:

We then include the java.yml tasks in alpha-services/tasks/main.yml file like so:

WHEN PHASE: Now we run $ molecule converge to effect changes on the instance.

THEN PHASE: Here we run $ molecule verify which should pass the test if the converge phase was successful.

TASK 2: Create a dir at path /var/log/tomcat belonging to owner ‘tomcat’, group ‘tomcat’ and of mode ‘0755’

We define the variables in Molecule’s test defaults yml file:

The first task uses Ansible’s stat module to get file system status while the second one checks that the statuses are match.

Next, we include the test_java.yml task in alpha-services/molecule/default/roles/test_alpha-services/tasks/main.yml file like so:

After this, we go ahead to run $ molecule verify which rightfully fails just like in the previous test. We therefore implement the actual task:

We define the variables in the actual role’s defaults yml file:

We include the tomcat.yml tasks in alpha-services/tasks/main.yml file like so:

WHEN PHASE: Now we run $ molecule converge to effect changes on the instance.
THEN PHASE: We then run $ molecule verify which should pass the test if the converge phase was successful.

TASK 3: Install, start and enable httpd

We will not test this task because Ansible does that for us. As stated in the Ansible documentation, «Ansible resources are models of desired-state. As such, it should not be necessary to test that services are started, packages are installed, or other such things. Ansible is the system that will ensure these things are declaratively true».
As such, if the service doesn’t exit and we try to start it, the task will fail with the error shown below:

Therefore we will only implement the task:

It is worth noting that running httpd on a linux systems requires systemd which is not present by default in docker containers. To be able to start a service on the docker container, we add the following edited platforms key in the molecule.yml file:

For more information on running systemd, see link.
We now run $ molecule create and $ molecule converge. If they both run successfully then the httpd service is up and running. To manually check for the httpd service, we run:

TASK 4: Copy a template file from template/tomcat/context.xml to /etc/tomcat/context.xml

Just like before, what we will be testing here is the existence of the exact file that we want to copy from the controller to the host system. We want to test that the file name is what we expect and probably some specific contents of the file are present as expected. We could have made an error while naming the file or creating the file contents and these are what we need to check. By default, if the file is not copied, Ansible would let us know.

We add the following to their respective files:

After this, we run $ molecule verify to see that it fails. After that, we implement the actual tasks:

We then run $ molecule converge and $ molecule verify subsequently. The tests should pass if everything was done right.

Finally, just to be sure, we run the $ molecule test to execute the entire Molecule test_sequence. Everything should run smoothly without any errors.

5. Conclusions

In conclusion, in my opinion, this is the right approach to developing Molecule tests for ansible roles. Infrastructure code should be tested before being deployed in production to avoid unpleasant surprises. This tutorial has been a simple demonstration of how Ansible testing can be done with Molecule using Ansible verifier. This way there is no need to learn another programming language such as Python, Ruby or Go.

Dejar respuesta

Please enter your comment!
Please enter your name here