testing-puppet-code-Part-4

facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Hello there!

This is the fourth blog post of testing puppet code series, in this installment I am going to walk you through adding additional Jenkins tips and configurations.

Testing puppet Code:
Part 1
Part 2
Part 3
Part 4 This post.

 

To start using Jenkins in an effective way we should automate the build tasks, even though triggering Jenkins from its crontab is quite inefficient, with the current setup which a single tenancy server and runs a small codebase we shouldn’t face any particular performance issue.

Let’s go to our Rspec project page => Configure  and scroll till Build triggers.

Tick “Poll Scm” and add ” */5 * * * *  ” , for those familiar with Unix/Linux the value is very self explanatory, for whom is not familiar with *NIX systems the above value means that Jenkins will check every five minutes for any changes (commits) into the Git repository if there are, it will run the Lint and Rspec tests. As I said earlier there are better ways to trigger Jenkins, but this option is slightly better than  building periodically, which creates a build even if there are no changes in given branch.

jenkinscron

At this point we can start to edit some puppet code and see Puppet-Lint in action.

If you have forked my example code:

git clone git@github.com:<your github username>/puppet_rspec_hello.git

cd puppet_rspec_hello/

git checkout dev

vim manifests/init.pp

Go to the end of the file and add double quotes instead of single quotes:

Example code:
————————
class hello {
file {“/tmp/test”:
ensure  => “present”,
}
}
————————

Save, then run:

git add manifests/init.pp
git commit -m “double quotes instead of single”
git push origin dev

Go back to Jenkins web interface and wait for five minutes and you should see the job kick-off.

Once the job is completed you should get a blue ball along with a report.

Changes: shows the commit message.
Revision: shows the commit hash.
Puppet-Lint Warnings: shows puppet style errors.

puppetlint1

By clicking on Puppet-Lint Warning we can see in detail which style errors we have in our code, in our case we have “double_quoted_strings” into the init.pp file.

puppetlint2

If we want to fix the issuer we need to go back to our code and put back the single quotes.

Example code:
————————
class hello {
file {‘/tmp/test':
ensure  => ‘present’,
}
}
————————

Save, then run:

git add manifests/init.pp
git commit -m “fix: reverted  single quotes instead of double”
git push origin dev

Syntax fix.

Go back to Jenkins web interface and wait another five minutes and you should see the job kick-off.

Once the job is completed you should get a blue ball along with a new report. In this case you can notice about “Puppet-Lint Warnings: Warnings” *2 fixed warnings.

puppetlint3

If you click on * 2 fixed warning you will see the Puppet-Lint report finally fixed.

puppetlint4

Jenkins and Puppet-Lint are amazing, aren’t’ they? Next time we’ll break some puppet code and test it with Rspec-Puppet.

Stay tuned!

testing-puppet-code-Part-3

facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Hello once again!

This is the third part of Testing Puppet Code series and I hope you are enjoying it so far.

Testing puppet Code:
Part 1
Part 2
Part 3 This post.
Part 4

 

Is part of my job  writing, changing and testing puppet code and I want to do that in the same of way as if i was a developer writing a web app or any other kind of software. Hence I need to maintain a high level of standard, checking that the code  meets the Puppet Labs Style Guidelines and test the behaviour of Puppet when it compiles the manifests into a catalogue of Puppet resources.

In this post we will cover Puppet-Lint, which is a tool to check Puppet code against the Puppet Labs Style Guide and return any errors or warnings and a basic rspec-puppet test.

Now, before we get started with puppet-lint and rspec-puppet we need to install two fundamental Jenkins plugins.

Log into Jenkins web interface (http://<ip address>:8080) and go to:

Manage Jenkins => Manage Plugins => available

Install these plugins:

Rake plugin
Warnings Plugin

Once the plugins are installed let’s go back GitHubRspec project  page.

projectpage1

Puppet-lint setup.

Scroll down till “Build” and click on “Add build step” => “Execute shell”.

In the command field paste:

lint

Add another Build step, “Add build step” => “Invoke Rake”.

In the rake version field you should seed (Default), in the task field just add “spec”.
rake

Now we are good to add the post build steps.

Go to “Post-build Actions” and select “Scan for compiler warnings” Add => Parser => “Puppet-Lint”

parserlint

Now we need to add a second post-build Action which will push the code to another git branch e.g master if the build succeed.

“Post-build Actions” => Git publisher.

publisher

It’s time to run our test, go to Project GitHubRspec page and click on “Build now”, as soon you hit the button things should start to kick-off.

build2

Congratulations, you have just created your first Puppet test job!

Next time we’ll dig a bit deeper into Jenkins setup, stay tuned!

 

 

Testing Puppet Code – Part 2

facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Welcome back! This is the follow up of Testing Puppet Code – Part 1. In this post, we’ll enable our local Jenkins to authenticate with GitHub and perform a git pull from a remote repository along with a basic build.

Testing puppet Code:
Part 1
Part 2 This post.
Part 3
Part 4

Puppet-Rspec boilerplate.

To make things easier , I’ve created a puppet-rspec boilerplate with a “Hello world” puppet class, the best way to take advantage of the example is to fork it from my GitHub page at:

https://github.com/p0bailey/puppet_rspec_hello

Once forker you would be able to push and pull from GithHub using Jenkins, please note that  the forked version of puppet_rspec_hello  will have your user-name!!!

https://github.com/<your username>/puppet_rspec_hello

Jenkins GitHub authentication.

To enable Jenkins to perform the Git Push/Pull stuff we need to install a Git plugin.

Go to : Manage Jenkins => Manage Plugins => available , and look for “Git Plugin”  and install it, make sure you tick the box “Restart Jenkins when installation is complete and no jobs are running”.

Once Jenkins has restarted go back to its main page http://<ip address>:8080,
and click on Credentials => Add domain => In the “Domain Name” field add: GitHub and press OK.

domain

Adding SSH keys.

In my previous post I’ve explained how to create a ssh key pair, just in case you haven’t done it yet… here’s a snippet of the post.

#sudo su
#su – jenkins

To generate the SSH keys type:

jenkins@jenkins:~$ ssh-keygen

Press ENTER and skip the passphrase.

At this point you should have a  SSH key pair that allow to autenticate
your local Jenkins server with Github, you can check the keys running:

#Private key stays on Jenkins server.
cat /var/lib/jenkins/.ssh/id_rsa
#Public Key must be placed into GitHub repository => deploy keys.
cat /var/lib/jenkins/.ssh/id_rsa.pub

Once the SSH key pair has been created you can set the details for GitHub authentication.

Add Credential:
Choose: SSH Username with private key.
Username: Add your github username.
Select “Private key from a file on Jenkins master.”  add the private key full path:
/var/lib/jenkins/.ssh/id_rsa

Press OK

credentialJenkins

 

GitHub Setup.

Go to your previously forked  puppet_rspec_hello ( https://github.com/<your username>/puppet_rspec_hello) click on Settings => Deploy keys => Add deploy key.

Paste the public key (not the private), just double check with cat /var/lib/jenkins/.ssh/id_rsa.pub

Press Add key.

deploykey
First Jenkins job.

At this point you should be good to go to create the first Jenkins job.

Go to Jenkins main page http://<jenkins ip address:8080/ and click on “New Item”, choose “Freestyle project” and a name that is suitable for you, then press OK.

jenkins create job 1

Source Code Management:

Select Git => Repository URL => git@github.com:<your github usrname>/puppet_rspec_hello.git

Tick the Advanced tab and add a name to the repository, will come in handy later on.

Branches to build => add:  ‘*/dev’

Press Save.

The screenshot below will help you get your bearings.

githubadvanced

Now it’s time to run our first Jenkins job, for the moment we will run the build manually but there are many different ways to trigger Jenkins.

By now you should be into the Rspec Jenkins project page, to trigger the first job just press “Build Now”

build

Once the Jenkins job has finished as well you should have a Jenkins jobs marked as blue (AKA success). If you jet the job marked as red you should go to the job console output and check the log.

build2

If you wonder where the files pulled from GitHub are stored, you can find them in the project workspace.

workspace2
And that’s all for today! We now have a Jenkins instance able to authenticate to GitHub, pull from a remote and execute a basic build. Next time we’ll add a bit of action using Puppet-Rspec tests, code style checking, and pushing successful build to a different git branch.

Hope you have enjoyed so far, stay tuned!

Testing Puppet Code – Part 1

Testing Puppet Code – Part 1

facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

With the proliferation of infrastructure automation tools our codebase evolve on daily basis, today is very likely to have multiple teams working across different environments and data-centers. With the adoption of cloud architecture,  servers and clusters are provisioned  at the speed of light, testing, staging and production environments are deployed within minutes. This tremendous shift has brought new challenges along with the need for operational consistency. Application and infrastructure software has to be reliable, consistent and delivered with a testable and repeatable work-flow. In the era of  modern infrastructure we cannot rely on “ssh loops” and script to perform our tests, we should therefore enable as much as possible  automated tests to catch fat-fingered errors in your code commits. We are human beings,we tend to make mistakes, especially typos. So, why not to take advantage of tools like Jenkins, rspec-puppet, puppet-lint, when writing puppet code?

Testing puppet Code:
Part 1  This post.
Part 2
Part 3
Part 4

Jenkins CI provides continuous integration services such builds and tests.

Rspec-puppet tests are there to test the behaviour of Puppet when it compiles your manifests into a catalogue of Puppet resources.

Puppet-lint check that your Puppet manifest conform to the style guide.

In this series of tutorials I’m going to show how to integrate Jenkins CI with rspec-puppet and puppet-lint, for this how to I’ll be using Vagrant, Ubuntu 14.04 trusty and a Github account. Setting up Vagrant  is out of scope of this tutorial, if you need to install Vagrant please check the excellent tutorial on getting stated with Vagrant.

1) Jenkins setup.

From the Vagrant box run:

Once Jenkins  setup is over, point the web browser to http://<vagrant box ip>:8080 and you should get into Jenkins web interface, as we have checked  the installation went through successfully we can move to the next step.

Screenshot - 111014 - 17:25:20

2) SSH Keys setup for GitHub authentication.

You need to set password less SSH keys in order to be able to pull and push from Github.

Within Jenkins Vagrant box run:

#sudo su
#su – jenkins

To generate the SSH keys type:

jenkins@jenkins:~$ ssh-keygen

Press ENTER and skip the passphrase.

At this point you should have a public SSH key that allow to autenticate your local Jenkins server with Github, you can check the key running:

cat /var/lib/jenkins/.ssh/id_rsa.pub

In the next post I’ll be covering Jenkins and Github setup and the basics of git pull/push.

Stay tuned.

Testing Puppet Code – Part 2

How to compile and install Sysbench for MariaDB under Centos 6.4

facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

In the past few months I’ve been experimenting with many MariaDB instances, so after the initial setup I had to measure the DB performances and spot eventual bottlenecks using Sysbench. The natural choice for the operating system was Centos 6.4 x86_64, it was very odd for me to discover that the Sysbench package was not available from the yum repositories,  even though few RPMs were available around the net, none of them were compiled with MariaDB support. Without wasting more time, I realized had to build it by myself from the sourcecode.

Here’s how:

1) Make sure you have MariaDB already installed.

2) Install the dev tools and the dependencies to compile Sysbench.

yum -y install MariaDB-devel  gcc gcc-c++ autoconf automake make libtool \
zlib zlib-devel openssl-devel

3) Grab Sysbench latest version (0.4.12).

http://sourceforge.net/projects/sysbench/

3) Decompress the archive.

tar xvfz sysbench-0.4.12.tar.gz

4) LIBTOOL error free.

cd sysbench-0.4.12
vim configure.ac

Comment AC_PROG_LIBTOOL as follow:

#AC_PROG_LIBTOOL

add the following line:

AC_PROG_RANLIB

5) Compile and install Sysbench

./autogen.sh

./configure –bindir=/usr/local/bin

make && make install

6) Test Sysbench

$which sysbench

$sysbench

sysbench
Missing required command argument.
Usage:
sysbench [general-options]… –test=<test-name> [test-options]… command

General options:
–num-threads=N            number of threads to use [1]
–max-requests=N           limit for total number of requests [10000]
–max-time=N               limit for total execution time in seconds [0]
–forced-shutdown=STRING   amount of time to wait after –max-time before forcing shutdown [off]
–thread-stack-size=SIZE   size of stack per thread [32K]
–init-rng=[on|off]        initialize random number generator [off]
–test=STRING              test to run
–debug=[on|off]           print more debugging info [off]
–validate=[on|off]        perform validation checks where possible [off]
–help=[on|off]            print help and exit
–version=[on|off]         print version and exit

Compiled-in tests:
fileio – File I/O test
cpu – CPU performance test
memory – Memory functions speed test
threads – Threads subsystem performance test
mutex – Mutex performance test
oltp – OLTP test

Commands: prepare run cleanup help version

See ‘sysbench –test=<name> help’ for a list of options for each test.