testing-puppet-code-part-5

facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Hello there and welcome back.

Testing puppet Code:
Part 1
Part 2
Part 3
Part 4
Part 5 This post.

In this installment we will cover our s first Puppet-Rspec test defining a  package and a service into our puppet manifest. To keep things simple we will define Apache2 as the resource to be installed and  ensuring the service enabled and starting at boot.

Before delving into the howto we should set a dedicated Jenkins job to run our new test. If you been following the whole installment you should already have a Jenkins setup, if you don’t, please follow the post Nr.1 and Nr.2 of the series.

Now, I’m going to assume you have a Jenkins instance ready to go. In this particular case the easiest way to start a new Jenkins job is to clone the existing  “puppet_rspec_hello job” into a new job that we could call “puppet_rspec_hello_adv1″.

Go to Jenkins main web-page, Click on “New Item”, add “puppet_rspec_hello_adv1″ into Item name field and tick “Copy existing item”, add “puppet_rspec_hello_adv1″ into it. See the screenshot.
jenkins clone

Once the puppet_rspec_hello_adv1 has been created, go into its configuration and change theGitHub repository from “git@github.com:<youuseranme>/puppet_rspec_hello.git” to “git@github.com:<yourusername>/puppet_rspec_hello_adv1.git”, this step is required in order to pull and push from the current git example repository.

jenkinsclone

Now click on Save and you should be good to go.

Workflow.

It is definitely a good idea to code locally on our computer and afterwards commit and push the code to a remote git repository and let Jenkins fetch and test the code for us. Coding  locally on a laptop or a desktop gives all the advantages of using powerful tools such “Atom” editor and “gitg” git client. Furthermore this routine will introduce you to remote collaboration and code review practice.

 Laptop (dev branch)=> Git Repository => Jenkins Server => (master branch) Git Repository

Our workflow embrace a simple two branches approach, we’ll write the code on our computer using the dev branch, we’ll push the code to dev branch on the Git Repository, Jenkins will keep polling the Git Repository and would kick a build job after a commit has been pushed. In our case the build job would invoke previously defined Rspec test, see post “testing-puppet-code-Part-3″. If the Rspec test succeeds, the code from dev branch will be pushed into the master branch  upon the remote git repository. Whether the test fails, no code will be pushed and you’ll get a red warning on Jenkins main dashboard.

 Let’s begin pulling the puppet code from our Git remote repository:

 

Let’s open the file spec/classes/init_spec.rb

Line 5   – The manifest should contain a class named: “puppet_rspec_hello_adv1″
Line 8   – The basic test, the manifest should compile.
Line 13 – Class “puppet_rspec_hello_adv1″ should contain a package named: “apache2″
Line 14 – Class “puppet_rspec_hello_adv1″ should contain a service named: “apache2″
Line 16 – Service apache2 should be running.
Line 17 – Service apache2 should be enabled to start at boot.

vim manifests/init.pp

Line 5   – Class “puppet_rspec_hello_adv1″ declared.
Line 6   – Package “apache” ensured installed.
Line 10 – Service apache
Line 11 – Service apache2 ensured running.
Line 12 – Service apache2 enabled to start at boot.

Because now we have a puppet manifest along with its rspec test we would like to check whether the test works through our Jenkins  continuous integration system.

 Once we’ve pushed the code to the Git remote repository it’s worth to kick off the Jenkins job to check our test. Let’s go to Jenkins main dashboard and run manually “puppet_rspec_hello_adv1″ job.

buildnow

Once the  Jenkins job has completed, as well you should see on the job history panel the jobs marked as blue or green (AKA success).

Console output.

console_output

Breaking the test.

Now let’s break few Rspec statements  to make sure the Rspec tests are working. In our case we wrote a r-spec test to ensure the  puppet manifest must contain a class named “puppet_rspec_hello_adv1″ along with apache2 server installed and configured to run at boot time.

Now let’s open manifests/init.pp and comment the two Rspec statements that would break the test.

Should be changed as follow.

Commit and push back to the Git remote repository.

Now go back to Jenkins dashboard and kick manually the test, wait until the job has completed. It should now have a job reporting the test failure.

test_failure

Looking at the console log we can spot two failures caused by the apache service not enabled and not  started in the puppet manifest. In order fix the test we should undo the comments previously made in the puppet manifest, and of course rerun the test.

I hope you have enjoyed this installment.

Stay tuned!

testing-puppet-code-Part-4

facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Hello there!

This is the fourth blog post of testing puppet code series, in this installment I am going to walk you through  additional Jenkins tips and tricks.

Testing puppet Code:
Part 1
Part 2
Part 3
Part 4 This post.
Part 5

 

To start using Jenkins in an effective way we should automate the build tasks. Even though triggering Jenkins from its crontab is quite inefficient, with the current setup which consist a single tenancy server and runs a small codebase we shouldn’t face any particular performances issue.

Let’s go to our Rspec project page => Configure  and scroll till Build triggers.

Tick “Poll Scm” and add ” */5 * * * *  ” , for those familiar with Unix/Linux the value is very self explanatory, for whom is not familiar with *NIX systems the above value means that Jenkins will check every five minutes for any changes (commits) into the Git repository if there are, it will run the Lint and Rspec tests. As I said earlier there are better ways to trigger Jenkins, but this option is slightly better than  building periodically, which creates a build even if there are no changes in given branch.

jenkinscron

At this point we can start to edit some puppet code and see Puppet-Lint in action.

If you have forked my example code:

git clone git@github.com:<your github username>/puppet_rspec_hello.git

cd puppet_rspec_hello/

git checkout dev

vim manifests/init.pp

Go to the end of the file and add double quotes instead of single quotes:

Example code:
————————
class hello {
file {“/tmp/test”:
ensure  => “present”,
}
}
————————

Save, then run:

git add manifests/init.pp
git commit -m “double quotes instead of single”
git push origin dev

Go back to Jenkins web interface and wait for five minutes and you should see the job kick-off.

Once the job is completed you should get a blue ball along with a report.

Changes: shows the commit message.
Revision: shows the commit hash.
Puppet-Lint Warnings: shows puppet style errors.

puppetlint1

By clicking on Puppet-Lint Warning we can see in detail which style errors we have in our code, in our case we have “double_quoted_strings” into the init.pp file.

puppetlint2

If we want to fix the issuer we need to go back to our code and put back the single quotes.

Example code:
————————
class hello {
file {‘/tmp/test':
ensure  => ‘present’,
}
}
————————

Save, then run:

git add manifests/init.pp
git commit -m “fix: reverted  single quotes instead of double”
git push origin dev

Syntax fix.

Go back to Jenkins web interface and wait another five minutes and you should see the job kick-off.

Once the job is completed you should get a blue ball along with a new report. In this case you can notice about “Puppet-Lint Warnings: Warnings” *2 fixed warnings.

puppetlint3

If you click on * 2 fixed warning you will see the Puppet-Lint report finally fixed.

puppetlint4

Jenkins and Puppet-Lint are amazing, aren’t’ they? Next time we’ll break some puppet code and test it with Rspec-Puppet.

Stay tuned!

testing-puppet-code-Part-3

facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Hello once again!

This is the third part of Testing Puppet Code series and I hope you are enjoying it so far.

Testing puppet Code:
Part 1
Part 2
Part 3 This post.
Part 4
Part 5

 

Is part of my job  writing, changing and testing puppet code and I want to do that in the same of way as if i was a developer writing a web app or any other kind of software. Hence I need to maintain a high level of standard, checking that the code  meets the Puppet Labs Style Guidelines and test the behaviour of Puppet when it compiles the manifests into a catalogue of Puppet resources.

In this post we will cover Puppet-Lint, which is a tool to check Puppet code against the Puppet Labs Style Guide and return any errors or warnings and a basic rspec-puppet test.

Now, before we get started with puppet-lint and rspec-puppet we need to install two fundamental Jenkins plugins.

Log into Jenkins web interface (http://<ip address>:8080) and go to:

Manage Jenkins => Manage Plugins => available

Install these plugins:

Rake plugin
Warnings Plugin

Once the plugins are installed let’s go back GitHubRspec project  page.

projectpage1

Puppet-lint setup.

Scroll down till “Build” and click on “Add build step” => “Execute shell”.

In the command field paste:

lint

Add another Build step, “Add build step” => “Invoke Rake”.

In the rake version field you should seed (Default), in the task field just add “spec”.
rake

Now we are good to add the post build steps.

Go to “Post-build Actions” and select “Scan for compiler warnings” Add => Parser => “Puppet-Lint”

parserlint

Now we need to add a second post-build Action which will push the code to another git branch e.g master if the build succeed.

“Post-build Actions” => Git publisher.

publisher

It’s time to run our test, go to Project GitHubRspec page and click on “Build now”, as soon you hit the button things should start to kick-off.

build2

Congratulations, you have just created your first Puppet test job!

Next time we’ll dig a bit deeper into Jenkins setup, stay tuned!

 

 

Testing Puppet Code – Part 2

facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Welcome back! This is the follow up of Testing Puppet Code – Part 1. In this post, we’ll enable our local Jenkins to authenticate with GitHub and perform a git pull from a remote repository along with a basic build.

Testing puppet Code:
Part 1
Part 2 This post.
Part 3
Part 4
Part5

Puppet-Rspec boilerplate.

To make things easier , I’ve created a puppet-rspec boilerplate with a “Hello world” puppet class, the best way to take advantage of the example is to fork it from my GitHub page at:

https://github.com/p0bailey/puppet_rspec_hello

Once forker you would be able to push and pull from GithHub using Jenkins, please note that  the forked version of puppet_rspec_hello  will have your user-name!!!

https://github.com/<your username>/puppet_rspec_hello

Jenkins GitHub authentication.

To enable Jenkins to perform the Git Push/Pull stuff we need to install a Git plugin.

Go to : Manage Jenkins => Manage Plugins => available , and look for “Git Plugin”  and install it, make sure you tick the box “Restart Jenkins when installation is complete and no jobs are running”.

Once Jenkins has restarted go back to its main page http://<ip address>:8080,
and click on Credentials => Add domain => In the “Domain Name” field add: GitHub and press OK.

domain

Adding SSH keys.

In my previous post I’ve explained how to create a ssh key pair, just in case you haven’t done it yet… here’s a snippet of the post.

#sudo su
#su – jenkins

To generate the SSH keys type:

jenkins@jenkins:~$ ssh-keygen

Press ENTER and skip the passphrase.

At this point you should have a  SSH key pair that allow to autenticate
your local Jenkins server with Github, you can check the keys running:

#Private key stays on Jenkins server.
cat /var/lib/jenkins/.ssh/id_rsa
#Public Key must be placed into GitHub repository => deploy keys.
cat /var/lib/jenkins/.ssh/id_rsa.pub

Once the SSH key pair has been created you can set the details for GitHub authentication.

Add Credential:
Choose: SSH Username with private key.
Username: Add your github username.
Select “Private key from a file on Jenkins master.”  add the private key full path:
/var/lib/jenkins/.ssh/id_rsa

Press OK

credentialJenkins

 

GitHub Setup.

Go to your previously forked  puppet_rspec_hello ( https://github.com/<your username>/puppet_rspec_hello) click on Settings => Deploy keys => Add deploy key.

Paste the public key (not the private), just double check with cat /var/lib/jenkins/.ssh/id_rsa.pub

Press Add key.

deploykey
First Jenkins job.

At this point you should be good to go to create the first Jenkins job.

Go to Jenkins main page http://<jenkins ip address:8080/ and click on “New Item”, choose “Freestyle project” and a name that is suitable for you, then press OK.

jenkins create job 1

Source Code Management:

Select Git => Repository URL => git@github.com:<your github usrname>/puppet_rspec_hello.git

Tick the Advanced tab and add a name to the repository, will come in handy later on.

Branches to build => add:  ‘*/dev’

Press Save.

The screenshot below will help you get your bearings.

githubadvanced

Now it’s time to run our first Jenkins job, for the moment we will run the build manually but there are many different ways to trigger Jenkins.

By now you should be into the Rspec Jenkins project page, to trigger the first job just press “Build Now”

build

Once the Jenkins job has finished as well you should have a Jenkins jobs marked as blue (AKA success). If you jet the job marked as red you should go to the job console output and check the log.

build2

If you wonder where the files pulled from GitHub are stored, you can find them in the project workspace.

workspace2
And that’s all for today! We now have a Jenkins instance able to authenticate to GitHub, pull from a remote and execute a basic build. Next time we’ll add a bit of action using Puppet-Rspec tests, code style checking, and pushing successful build to a different git branch.

Hope you have enjoyed so far, stay tuned!

 

Testing Puppet Code – Part 1

facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

With the proliferation of infrastructure automation tools our codebase evolve on daily basis, today is very likely to have multiple teams working across different environments and data-centers. With the adoption of cloud architecture,  servers and clusters are provisioned  at the speed of light, testing, staging and production environments are deployed within minutes. This tremendous shift has brought new challenges along with the need for operational consistency. Application and infrastructure software has to be reliable, consistent and delivered with a testable and repeatable work-flow. In the era of  modern infrastructure we cannot rely on “ssh loops” and script to perform our tests, we should therefore enable as much as possible  automated tests to catch fat-fingered errors in your code commits. We are human beings,we tend to make mistakes, especially typos. So, why not to take advantage of tools like Jenkins, rspec-puppet, puppet-lint, when writing puppet code?

Testing puppet Code:
Part 1  This post.
Part 2
Part 3
Part 4
Part 5

Jenkins CI provides continuous integration services such builds and tests.

Rspec-puppet tests are there to test the behaviour of Puppet when it compiles your manifests into a catalogue of Puppet resources.

Puppet-lint check that your Puppet manifest conform to the style guide.

In this series of tutorials I’m going to show how to integrate Jenkins CI with rspec-puppet and puppet-lint, for this how to I’ll be using Vagrant, Ubuntu 14.04 trusty and a Github account. Setting up Vagrant  is out of scope of this tutorial, if you need to install Vagrant please check the excellent tutorial on getting stated with Vagrant.

1) Jenkins setup.

From the Vagrant box run:

Once Jenkins  setup is over, point the web browser to http://<vagrant box ip>:8080 and you should get into Jenkins web interface, as we have checked  the installation went through successfully we can move to the next step.

Screenshot - 111014 - 17:25:20

2) SSH Keys setup for GitHub authentication.

You need to set password less SSH keys in order to be able to pull and push from Github.

Within Jenkins Vagrant box run:

#sudo su
#su – jenkins

To generate the SSH keys type:

jenkins@jenkins:~$ ssh-keygen

Press ENTER and skip the passphrase.

At this point you should have a public SSH key that allow to autenticate your local Jenkins server with Github, you can check the key running:

cat /var/lib/jenkins/.ssh/id_rsa.pub

In the next post I’ll be covering Jenkins and Github setup and the basics of git pull/push.

Stay tuned.