Beginning Docker video course

Blog updates have been scarce recently as I have been busy working on a couple of publications… the first of which has just been released…

https://www.linuxjournal.com/node/1338951

This is a hands-on video course packed with practical examples to get you started with Docker.

Here is the course overview video:

And here is a free sample video from Section 2, “Docker Basics” where we take a look at running containers and the 3 different types of “containerized” commands:

and this final sample video is taken from Section 5 – “Running a Web Application with Docker“.

In this clip we build our own web application using Python, pip and Redis, which we will then “dockerize” and ship to “production”:

About This Video

  • Master Docker commands by creating and publishing a sample web application
  • Build and manage your own custom Docker Containers to set up data sources, filesystems, and networking
  • Build your own personal Heroku PaaS with Dokku

Who This Video Is For

If you’re a developer who wants to learn about Docker, a powerful tool to manage your applications effectively on various platforms, this course is perfect for you! It assumes basic knowledge of Linux but supplies everything you need to know to get your own Docker environment up and running.

What You Will Learn

  • Build new Docker containers and find and manage existing ones
  • Use the Docker Index, and create your own private one by using containers
  • Discover ways to automate Docker, and harness the power of containers!
  • Build your own Docker powered mini-Heroku Paas with Dokku
  • Set up Docker on your environment based on your application’s custom requirements
  • Master Docker patterns and enhancements using the Ambassador and Minimal containers

In Detail

One of the major challenges while creating an application is adapting your application to run smoothly on all of the plethora of operating systems available. Docker is an extremely efficient technology that allows you to wrap all your code along with its supporting files into a single bundle; it also guarantees that your application will behave in the same way on any host powered by Docker. You can also easily reuse existing Docker containers or create and publish your own. Unlike Virtual Machines, Docker containers are lightweight and more efficient.

Beginning Docker starts with the fundamentals of Docker—explaining how it works, how to set it up, and how to get started on leveraging the benefits of this technology. The course goes on to cover more advanced features and shows you how to create and share your own Docker images.

You will learn how to install Docker on your own machine, then how to manage it effectively, and then progress to creating and publishing your very own application. You will then learn a bit more about Docker Containers; built-in features and commands such as volumes, mounts, ports, and linking and constraining containers; before diving into running a web application.

Docker has functionality such as the Docker web API to handle complex automation processes which will be explained in detail. You will also learn how to use the Docker Hub to fetch and share containers, before running through the creation of your own Docker powered mini-Heroku

Beginning Docker covers everything required to get you up and running with Docker, with detailed real-world examples and helpful tips to make sure you get the most from it.

Style and Approach

An easy-to-follow and structured video tutorial with practical examples of Docker to help you get to grips with each and every aspect.

The course will take you on a journey from the basics to the advanced application of Docker containers, and includes several real-world scenarios to learn from.

Cheers,

Don

Jenkins and Docker – Part One

This post lists the steps taken to get started with Docker – the basic install and getting a Docker container up and running on Ubuntu.

There’s nothing new or unusual in this section, but it forms the background of the next post(s) where I plan to go in to detail on different approaches to using Docker with Jenkins (and vice versa).

If you’re unfamiliar with Docker here is a good introduction:

and you can try Docker out easily on the docker site: https://www.docker.com/tryit/

 

 

In my case the Ubuntu is a pretty ordinary “ubuntu 14.04.1 LTS (GNU/Linux 3.13.0-35-generic x86_64)” running as a guest VM on my Development VM Ware ESXi Server.

Installing docker on Ubuntu is trivial – here are the commands I found via a quick google, there were no issues…

apt-get -y install docker.io

(FYI it’s called docker.io as there’s a previous/existing package with the name docker)

fix path issues:

ln -sf /usr/bin/docker.io /usr/local/bin/docker
sed -i '$acomplete -F _docker docker' /etc/bash_completion.d/docker.io

have docker start up at boot:

update-rc.d docker.io defaults

and that’s that done – all very simple, quick and straightforward, now it’s time to pull down an Ubuntu image…

docker pull ubuntu

once that’s done, you are ready to run a command (bash in the case of this quick test I found suggested elsewhere) in a docker container like so:

docker run -i -t ubuntu /bin/bash

(-i attaches stdin & stdout and -t allocates a terminal)

That’s it for this post – the next will look at dynamic Jenkins Slave provisioning using Docker Containers, Jenkins plugins for Docker and ways to use Docker for a variety of Jenkins build, deployment and test tasks for Continuous Integration, Continuous Build and DevOps purposes.

Cheers,

Don

 

 

 

 

Extracting data from Jenkins

In Part I,  Information Radiators, I covered what they are, what the main benefits are, and the approach I usually use to set them up. This post goes in to more technical detail on how I extract this data from Jenkins.

My usual setup/architecture for Jenkins Information Radiators goes something along these lines:

  • TV screens running Mozilla Firefox or Google Chrome in Kiosk Mode, and Tab Mix Plus set up to rotate tabs (if required)
  • JSP Pages served via Tomcat on Linux server (which also runs the data extracting script described below)
  • MySQL database on Linux server – contains tables with data pulled from Jenkins and other sources, and the config data too (which URL’s to monitor)

And you’ll need some Jenkins instances/jobs to monitor too, obviously 🙂

The Jenkins XML API is very useful for automating tasks like this – if you simply append “/api/xml” to a
Jenkins job URL, it will serve up an XML version – note there is also a JSON API and a CLI and plenty of other options, but I’m using what suits me.

The Jenkins XML API

For example, if you go to one of your Jenkins jobs and add /api/xml like this:

“http://yourjenkinsserver:8080/job/yourjobname/api/xml

you should get back some XML, possibly roughly like this example:

<?xml version="1.0"?>
<freeStyleBuild>
 <action>
 <parameter>
 <name>LOWER_ENV</name>
 <value>dev</value>
 </parameter>
 </action>
 <action>
 <cause>
 <shortDescription>Started by timer</shortDescription>
 </cause>
 </action>
 <building>false</building>
 <duration>61886</duration>
 <fullDisplayName>MyJob #580</fullDisplayName>
 <id>2014-04-01_10-01-50</id>
 <keepLog>false</keepLog>
 <number>580</number>
 <result>SUCCESS</result>
 <timestamp>1396342910088</timestamp>
 <url>http://jenkinsserver:8080/view/MyView/job/MyJob/580/</url>
 <builtOn/>
 <changeSet/>
</freeStyleBuild>

That XML contains loads of very useful information inside handy XML tag descriptions – you just need a way to get at that data and then you can present it as you like…

XPAth queries and the Jenkins XML API

so to automate that, I used to extend that approach a to query Jenkins via the XML API using XPAth queries to bring back just the data I actually wanted, quite like querying a database.

For example, wget’ing this URL would return just the current value of the <building> tag in the above XML:

http://yourjenkinsserver:8080/job/yourjobname/api/xml?xpath=//building/text()

e.g. “true” or “false” – this was very useful and easy to do, but the functionality was removed/disabled in recent versions of Jenkins for security reasons, meaning that my processes that used it needed rewritten 🙁

Extracting the data – Plan B…

So, here’s the new solution I went for – the real scripts/methods do some error handling and cleaning up etc but I’m just highlighting the main functions and the high level logic behind each of them here;

get_url’s method:

query a table in MySQL that contains a list of the job names and URL’s to monitor
for each $JOB_NAME found, it calls the get_file method, passing that the URL as a parameter.

get_file method:

this takes a URL param, and uses curl to fetch and save the XML data from that URL to a temporary file (“xmlfile”):

curl -sL "$1" | xmllint --format - > xmlfile

Note I’m using “xmllint –format” there to nicely format the XML data, which makes processing it later much easier.

get_data method:

this first calls “get_if_building” (see below) to see if the job is currently running or not, then it does:

TRUE_VAR="true"
 if [[ "$IS_BUILDING" == "$TRUE_VAR" ]]; then
 RESULT_TEXT="building..."
 else
 RESULT_TEXT=`grep "result>" xmlfile | awk -F\> '{print $2}' | awk -F\< '{print $1}'`
 fi

get_if_building method:

this simply checks and sets the IS_BUILDING var like so:

IS_BUILDING=`grep building xmlfile | awk -F\> '{print $2}' | awk -F\< '{print $1}'`

Putting it all together

My script then updates the MySQL database with the results from each check: success/failure, date, build number, user, change details etc

I then have JSP pages that read data from that table, and translate things like true/false in to HTML that sets the background colours (Red, Amber, Green), and shows the appropriate blocks and details per job.

If you have a few browsers/TV’s or Monitors showing these strategically placed around the office, developers get rapid feedback on the result of their code changes which speeds up development, increases quality and reduces development time and costs – and they can be fun to watch and set up too 🙂

Cheers,

Don

DIY Information Radiator

 

 

Information Radiators are used to provide people with feedback on the current status of code builds and automated tests in Continuous Integration and Agile development environments.

The basic idea is that when developers commit a code change, they can easily and quickly see that it has been picked up by the automated build process, and then (ideally within 10 minutes) see the result of their change; did the build succeeded and did the automated tests pass?

Martin Fowler’s description goes in to more detail on the ideas behind this approach and the function that Information Radiators serve.

The normal convention for these is to use colour coded blocks per build, using:

  • Green for good/passing jobs
  • Amber for either currently building/running jobs (or sometimes for unstable ones)
  • Red for failed jobs

Generally you want to keep things as clean, simple and uncluttered as possible, but sometimes it is helpful to add in a bit more info.

Details I have occasionally found worth adding include things like;

  • name and/or picture of the user who triggered (or broke!) the build
  • commit message from the code change that triggered the build
  • build number
  • history – number of recent fails or passes
  • date/time last failed and last checked

if you use amber for “unstable” rather than “build in progress”, you may want to add text to say if the job is currently building – I often use the “spinning wheel” icon thingy from jenkins itself :

spinner

Why build your own?

There are tons of readily available plugins that allow you to quickly and easily produce a Radiator or Wall panel from Jenkins, so why go to the bother of making your own?

Plugins are usually linked to one Jenkins instance (the one they are running on) and I have often found that alone to be too restrictive – having too many different radiators all over the place makes things too cluttered and uncertain, and people can easily start to “switch off” from them all – having one screen that people can understand at a glance usually works best.

Changing requirements – developers are constantly wanting/looking to improve processes and often come up with ideas and requests for things to try that may help them do their jobs – adding a bit of information from another source, for example, or changing the colours used to a different shade, or adding curved borders etc etc…

What I have found often works best, is to get all of the data I am interested in inside a database then write my own simple but flexible presentation layer from scratch – this gives me all the flexibility I could want (or may find myself wanting or needing later…) and importantly, it also allows me to leverage additional benefits by combining data from Jenkins with data from elsewhere – this means I can easily produce reports, charts, metrics etc that present a view comprised from multiple data sources throughout an organisation – for example, you can then easily create reports that combine:

  • jenkins – live information on the current state of builds and tests
  • defects – data on bugs and changes pulled from bug tracking apps (usually via jenkins jobs)
  • version control – the actual code change can be extracted from version control and linked to both the developer and bug details – the “svn log” command is useful for this
  • environment monitors – state and health of environments; database and app server health, deployed patch and code levels etc (again, these are usually other Jenkins jobs!)

and you can add in anything else you can get your hands on 🙂

This allows you to track the flow of a change right through the development life cycle – from the initial change/requirement/defect to the change itself at the code/file level, then the testing and building of it and the eventual release. This is far more than you need for a simple Information Radiator, but using this approach means that you can easily reuse much of the work in different ways.

Part II covers the technical approach I use for Extracting data from Jenkins

Jenkins Agent Nodes – using the Swarm Plugin

NOTE: This post and approach is quite old now; a better alternative for dynamically provisioning and scaling Jenkins Agents and running CI/CD Pipelines is demonstrated here.


I’ve been trying out a new (to me at least) way to add a Jenkins Agent Node – using UDP auto discovery via the Jenkins Swarm Plugin

This is a very easy and nice way to do it, with minimal configuration/hassle required so you can quickly and easily add new Jenkins Agent Nodes to your Master Jenkins instance as and when they are required.

Here are my notes from setting this up – it’s pretty simple to do and worked well for me out of the box…

Set up a new instance of Jenkins:

  • Make & cd to a working directory

mkdir jenkinsswarm; cd jenkinsswarm

  • Fetch jenkins.war

curl -O http://mirrors.karan.org/jenkins/war/1.506/jenkins.war

  • Start Jenkins

{/path/to/java/bin/}java -jar jenkins.war

After that, you should get console output along these lines…

Running from: /root/jenkinsswarm/jenkins.war
webroot: $user.home/.jenkins
18-Mar-2013 15:19:26 winstone.Logger logInternal
INFO: Beginning extraction from war file
Jenkins home directory: /root/.jenkins found at: $user.home/.jenkins
18-Mar-2013 15:19:33 winstone.Logger logInternal
INFO: HTTP Listener started: port=8080
18-Mar-2013 15:19:33 winstone.Logger logInternal
INFO: Winstone Servlet Engine v0.9.10 running: controlPort=disabled
18-Mar-2013 15:19:34 jenkins.InitReactorRunner$1 onAttained
INFO: Started initialization
18-Mar-2013 15:19:35 jenkins.InitReactorRunner$1 onAttained
INFO: Listed all plugins
18-Mar-2013 15:19:35 jenkins.InitReactorRunner$1 onAttained
INFO: Prepared all plugins
18-Mar-2013 15:19:35 jenkins.InitReactorRunner$1 onAttained
INFO: Started all plugins
18-Mar-2013 15:19:41 jenkins.InitReactorRunner$1 onAttained
INFO: Augmented all extensions
18-Mar-2013 15:19:41 jenkins.InitReactorRunner$1 onAttained
INFO: Loaded all jobs
18-Mar-2013 15:19:44 org.jenkinsci.main.modules.sshd.SSHD start
INFO: Started SSHD at port 25133
18-Mar-2013 15:19:44 jenkins.InitReactorRunner$1 onAttained
INFO: Completed initialization
18-Mar-2013 15:19:44 hudson.TcpAgentListener <init>
INFO: JNLP agent listener started on TCP port 41790
18-Mar-2013 15:19:44 hudson.WebAppMain$2 run
INFO: Jenkins is fully up and running

– that looks happy enough, and as you can see from the line “HTTP Listener started: port=8080” it’s running on the default port, so connect to http://yourhost:8080 and you should see something like this…

the next step is to install the Swarm Plugin (https://wiki.jenkins-ci.org/display/JENKINS/Swarm+Plugin) on this Jenkins Master instance so that Swarm Clients can connect to it.

Do this by going to “Manage Jenkins > Manage Plugins > Available” then selecting to install the “Swarm Plugin“.

Once that’s done you should see that the plugin has been installed…


Now that your new Jenkins server is set up and ready, hop over to your other Jenkins Agent/Client host and do the following…

mkdir for the swarm client

mkdir swarmclient; cd swarmclient/

Get the Swarm Client jar file from the ‘net

curl -O http://maven.jenkins-ci.org/content/repositories/releases/org/jenkins-ci/plugins/swarm-client/1.8/swarm-client-1.8-jar-with-dependencies.jar

Start up the Client

java -jar swarm-client-1.8-jar-with-dependencies.jar
Found 1 eligible Jenkins.
Connecting to http://mydomain.com:8080/
Attempting to connect to http://mydomain.com:8080/ a2721b16-04e4-0d962
18-Mar-2013 15:33:22 org.apache.commons.httpclient.HttpMethodDirector authenticateHost
WARNING: Required credentials not available for BASIC <any realm>@mydomain.com:8080
18-Mar-2013 15:33:22 org.apache.commons.httpclient.HttpMethodDirector authenticateHost
WARNING: Preemptive authentication requested but no default credentials available
18-Mar-2013 15:33:23 hudson.remoting.jnlp.Main$CuiListener <init>
INFO: Hudson agent is running in headless mode.
18-Mar-2013 15:33:23 hudson.remoting.jnlp.Main$CuiListener status
INFO: Locating server among [http://mydomain.com:8080/]
18-Mar-2013 15:33:23 hudson.remoting.jnlp.Main$CuiListener status
INFO: Connecting to myhost.mydomain.com:43932
18-Mar-2013 15:33:23 hudson.remoting.jnlp.Main$CuiListener status
INFO: Handshaking
18-Mar-2013 15:33:23 hudson.remoting.jnlp.Main$CuiListener status
INFO: Connected

Now take a look at your browser and you should see a new Node automatically added to the Master Jenkins instance…

A very handy and flexible approach to adding/managing Nodes and workload – many thanks to the developers behind this!

Cheers,

Don

Managing Jenkins as a service and starting at boot time

Linux Services

I have an Ubuntu Linux VM that runs Jenkins, and to make life simpler I wanted to set up Jenkins to run as a service. I also wanted it to start up automatically at boot time, as that’s the sole function of the VM it’s running on.

So, here are my notes on setting up Jenkins as a service on Ubuntu Linux, which includes a script to manage (start, stop, check, restart) the Jenkins process too.

On Ubuntu (and most othe rLinux versions) you can check the current services with the command “service –status-all” – this should give you a list of all services, and their current status. You can (as root) also do “service <name> start” (or stop or restart) for each one.

Jenkins as a Service

To create a new service for Jenkins, take a look at the existing scripts in /etc/init.d/ for some examples.

By convention there are three main methods a basic service implements, they are; start, stop and restart. About the most basic structure of a service script is therfore something like this:

#!/bin/bash
start() {
echo “Starting Service”
# Do start things here.
}stop() {
echo “Stopping Service”
# Do stop things here.
}restart() {
echo “Restarting Service…”
stop
start
}

case “$1″ in
start)
start
;;
stop)
stop
;;
restart)
restart
;;
*)

obviously you still need to implement the start and stop functions though 🙂

To see a far more detailed example, take a look at the file /etc/init.d/skeleton, which you could copy and update to suit your needs if you prefer.

I decided on something halfway between the very basic example above and the far more complex example in “skeleton”, and did something like this (saved as “/etc/init.d/jenkins”):

# Example Jenkins auto-start service script
#
# description: manages Jenkins as a service
# processname: jenkins
# pidfile: /var/run/jenkins.pid
# author: www.DonaldSimpson.co.uk# The user and the home dir that Jenkins runs under
jenkins=/usr/local/jenkins
# Your startup and stop scripts (see below)
startup=$jenkins/bin/startup.sh
shutdwn=$jenkins/bin/shutdown.shstart_jenkins() {
echo “Starting Jenkins services…”
su – jenkins -c “sh $startup”
}

stop_jenkins() {
echo “Stopping Jenkins services…”
su – jenkins -c “sh $shutdwn”
}

status_jenkins() {
# Check for any other process containing jenkins.war
# This could be improved upon (see script below)
numproc=`ps -ef | grep [j]enkins.war | wc -l`
if [ $numproc -ne 0 ]; then
echo “Jenkins is running…”
else
echo “Jenkins is NOT running…”
fi
}

case “$1” in
start)
start_jenkins
;;
stop)
stop_jenkins
;;
status)
status_jenkins
;;
restart)
stop_jenkins
start_jenkins
;;
*)
echo “Usage: $0 {start|stop|status|restart}”
exit 1
esac
exit 0

Update that to suit then save and change the permissions to make it executable:

chmod +x /etc/init.d/jenkins

then you can check (as root) that you can call the methods in the script:

service jenkins status

service jenkins stop

service jenkins start

These should all now run as the Jenkins user via sudo and say something when called, even if it is just “I can’t find the scripts you told me to call” 🙂

Jenkins Startup and Stop scripts

So, you now need to create the missing startup and shutdown scripts, in my example they were:

/usr/local/jenkins/bin/startup.sh

and

/usr/local/jenkins/bin/shutdown.sh

An example of the Jenkins start-up and management script I normally use is at the end of this post – the main idea though, is that it sets what you want to set then starts Jenkins via nohup and outputs to a log file

I have also added and included some basic tests to my scripts and some (very) rudimentary error handling/checking, but you shoudl get the idea and all you really need is this line (with the variables set correctly):

${NOHUP} ${JAVA} -jar ${JENKINS_WAR} -D${MARKER} –httpListenAddress=0.0.0.0 –httpPort=${HTTP_PORT} > ${LOG_FILE} &

Note: I often run multiple instances of Jenkins, so I explicitly specify the HTTP_PORT, and I use the -D${MARKER} to allow me to safely and easily find the correct PID for each project.
The httpListenAddress can normally go unset, but it’s something I’ve had to set before on multi-home’d hosts.

The stop part of my Jenkins management script finds the correct PID like this (and you could use a filter for the correct -D${MARKER} if you want that too):

PID=`${LSOF} -w -n -i tcp:${HTTP_PORT} | ${GREP} -v COMMAND | ${AWK} {‘print $2’}`

and simply kills it.

For some of my scripts I also filter for process ID’s that were started in the current directory by checking “pwdx” against the PID, but only where I’m sure the corresponding start-up process is correct/reliable.

So, tweak to your taste and that should be the Jenkins service created and working too now; for start|stop|restart you can create individual scripts or wrappers that call something like the one script below with parameters, or break the script in to separate files if you prefer.

Setting Jenkins to start at boot time

If you want to start Jenkins at boot time/startup automatically then you still need to do one more small step.  There are many different ways to do this depending on personal preference, your requirements and your version of Linux, but on Ubuntu it can be done easily with:

update-rc.d jenkins defaults

“update-rc.d” simply installs and removes System-V style init script links – read the man page for full details, but the same idea applies to most versions of Linux.

update-rc.d -f  jenkins remove

will undo this if you no longer want it.

An example Jenkins server/process management script

As an example, here is my script to manage Jenkins processes – it could use some improvements but the basic start, stop, restart and check should give you enough to sort out something that works and suits your needs.

Better formatted version available here: https://github.com/DonaldSimpson/scripts/blob/master/JenkinsProcessManager.sh

#! /bin/bash -p
# www.donaldsimpson.co.uk
# Script to start|stop|restart|check an instance of Jenkins
# For each new instance, the PROJECT and HTTP_PORT need to be updated:
export PROJECT=jenkins
export HTTP_PORT=9000
EXIT_STRING=””
JENKINS_ROOT=/opt/apps/jenkins
export JENKINS_HOME=${JENKINS_ROOT}/${PROJECT}
JENKINS_WAR=${JENKINS_HOME}/jenkins.warWAIT_TIME=5
START_WAIT_TIME=15
JAVA_HOME=/usr
PATH=$JAVA_HOME/bin:$PATH
JAVA=${JAVA_HOME}/bin/java
NOHUP=/usr/bin/nohup
LOG_FILE=${JENKINS_HOME}/debug_${PROJECT}.log
MARKER=”JenkinsProcFor_${PROJECT}”NC=/bin/nc
WGET=/usr/bin/wget
LSOF=/usr/bin/lsof
AWK=/usr/bin/awk
GREP=/bin/grep
FUSER=/bin/fuser
#################################################################################
### Functions Start #############################################################
#################################################################################

cleanup(){
# Perform any clean-up activities here…
[ “${EXIT_STRING}” != “0” ] && echo `date “+%d/%m/%y %H:%M:%S”` ${EXIT_STRING}
exit 0
}

trap ” QUIT

_error() {
EXIT_STRING=”${1}”
[ “${1}” != “0” ] && EXIT_STRING=”ERROR: ${1}, please investigate, terminating.”
cleanup  # Never returns from this call…
}

say(){
echo `date “+%d/%m/%y %H:%M:%S”` “${1}”
}

saybold(){
tput bold 2>/dev/null
echo `date “+%d/%m/%y %H:%M:%S”` “${1}”
tput sgr0 2>/dev/null
}

check_folders(){
say “”
saybold “Checking all required folders exist…”
for REQUIRED_DIR in ${JENKINS_ROOT} ${JENKINS_HOME} ${JAVA_HOME}
do
[ ! -d “${REQUIRED_DIR}” ] && _error “Necessary directory: ${REQUIRED_DIR} does not exist”
say “Found required directory: ${REQUIRED_DIR}”
done
saybold “Done.”
say “”
}

check_files(){
say “”
saybold “Checking all required files exist…”
for REQUIRED_FILE in ${JENKINS_WAR} ${NC} ${WGET} ${JAVA} ${LSOF} ${FUSER} ${GREP} ${AWK}
do
[ ! -f “${REQUIRED_FILE}” ] && _error “Necessary file: ${REQUIRED_file} does not exist”
say “Found required file ${REQUIRED_FILE}”
done
saybold “Done.”
say “”
}

check_port_closed(){
say “Checking that port ${HTTP_PORT} is closed…”
${NC} -w 1 localhost ${HTTP_PORT}
if [ $? -eq 0 ]; then
_error “Required Jenkins port ${HTTP_PORT} is already in use”
else
say “Ok, required port ${HTTP_PORT} is available, continuing…”
fi
}

check_port_open(){
say “Checking that port ${HTTP_PORT} is open…”
${NC} -w 1 localhost ${HTTP_PORT}
if [ $? -eq 0 ]; then
say “Ok, a process is listening on port ${HTTP_PORT}, continuing…”
else
_error “Required Jenkins port ${HTTP_PORT} has not been opened.”
fi
}

start_process(){
cd ${JENKINS_HOME}
saybold “Starting Process now…”
${NOHUP} ${JAVA} -jar ${JENKINS_WAR} -D${MARKER} –httpListenAddress=0.0.0.0 –httpPort=${HTTP_PORT} > ${LOG_FILE} &
say “Process initiated.”
}

check_log(){
say “Checking the log file ${LOG_FILE} for an HTTP listener…”
STARTED=`${GREP} -c “HTTP Listener started” ${LOG_FILE}`
if [ ${STARTED} -eq “0” ]; then
_error “An HTTP Listener has not reported as started in the log file ${LOG_FILE}”
else
saybold “An HTTP Listener is reported as started in the log file ${LOG_FILE}”
fi
}

check_html(){
# These checks need error handling, but you get the general idea.
say “Checking that localhost:${HTTP_PORT} is serving Jenkins pages…”
TEMP_WGETDIR=tempwgetdir$$
mkdir ${TEMP_WGETDIR}
cd ${TEMP_WGETDIR}
${WGET} -q http://localhost:${HTTP_PORT}
GOT_HTML=`${GREP} -c Dashboard index.html`
cd ${JENKINS_HOME}
rm -rf ${TEMP_WGETDIR}
if [ ${GOT_HTML} -eq “0” ]; then
_error “Unable to get an HTML page from the Server.”
fi
saybold “Recieved valid HTML from the server, all looks ok.”
}

check_process(){
check_port_open
check_log
check_html
say “A Jenkins instance is listening on port ${HTTP_PORT} for project ${PROJECT}.”
say “The process is logging debug info to the log file: ${LOG_FILE}”
}

stop_proc(){
check_port_open
saybold “Looking for the Process ID attached to port ${HTTP_PORT}”
PID=`${LSOF} -w -n -i tcp:${HTTP_PORT} | ${GREP} -v COMMAND | ${AWK} {‘print $2’}`
if [ “${PID}” == “” ]; then
saybold “Unable to detect the Process ID that is listening on port ${HTTP_PORT}!”
PID=`${FUSER} ${LOG_FILE}`
if [ “${PID}” == “” ]; then
_error “Unable to find the PID that has the log file open too!”
else
say “Ok, found PID ${PID}”
fi
else
saybold “Found a PID of $PID, killing it now…”
fi
kill -9 ${PID}
say “Waiting ${WAIT_TIME} seconds for the process to die…”
sleep ${WAIT_TIME}
saybold “Done, checking port is closed…”
check_port_closed
saybold “All done.”
}

start(){
check_folders
check_files
check_port_closed
start_process
saybold “Waiting ${START_WAIT_TIME} seconds for the process to start up…”
sleep ${START_WAIT_TIME}
check_process
}

restart(){
stop_proc
start
}

#################################################################################
### Script Start ################################################################
#################################################################################

case “$1” in
start)
start
;;
stop)
stop_proc
;;
restart)
restart
;;
check)
check_process
;;
*)
echo “Usage: $0 {start|stop|restart|check}”
esac
# Exit cleanly
_error “0”

Hope that helps! Any constructive comments, requests or suggestions for improvement are very welcome 🙂

Cheers, and sorry about the indenting,

Don

Jenkins Agent Nodes

This Jenkins Agent Nodes post covers:

  • What are they?
  • Why may I want one?
  • How do you create one?
    • tasks on On the Master/Server
    • tasks on On the Agent/Client
  • Other ways of creating Agent Nodes
  • Related posts and links

What are they?

Jenkins Agents  are small Java “Client” processes that connect back to a “Master” Jenkins instance over the Java Network Launch Protocol (JNLP).

Why may I want one?

Once it’s up and running, an Agent instance can be used to run tasks from a Master Jenkins instance on one or more remote machines providing an easy to use and flexible distributed system which can lend itself to a wide variety of tasks.

As these are Java processes, you are not restricted by architecture and can mix and match the OS’s of your agent nodes as required – Windows, Linux, UNIX, iSeries, OVMS etc – anything capable of running a modern version of Java (I think JNLP was introduced in 1.5?) and you can also group and categorize subsets of different types (both logical and physical) of Agents; intended use, availability, location, available resources, Cloud or VM versus Physical tin – anything that helps you decide when you want to use which host.

There are many different ways you can choose to utilize these nodes – they can be used to spread the load of an intensive build process over many machines when they are available, you can delegate specific tasks to specific machines only, or you can use labels to group different classes or types of Nodes that are available for certain tasks, making the most use of your available resources. You can also have Jenkins create Cloud server instances – Amazon EC2 for example – when certain thresholds are reached, and stop them when they are no longer required.

This post focuses on a pretty manual approach to the creation of Jenkins Agent Nodes with the intention of explaining them well enough to allow you to create them on any platform that can run a modern version of Java – there are probably simpler solutions depending on your needs and setup. A later post will touch on a few of the many possible uses for these nodes.

So, how do you create one?

There are several different ways to go about setting up an Agent, and the “best” approach depends on your situation, needs and environment(s) – for a simple Linux setup letting Jenkins do all the work for you makes life really easy, you can just select that option in your new Jenkins Agent Node and complete this screen to have Jenkins set it up for you:

Where the Username and Password are the credentials you want Jenkins to use to connect and start the Agent process on the remote server. This simple approach also allows the Master instance to initiate the JNLP connection and bring your agents online when required, avoiding any need to manually visit each agent node.

This keeps things nice and simple and reduces the admin overhead too,  but sometimes this type of approach can’t be used (on different OS’s like OVMS, iSeries, Windows etc) and I’m going to go on to outline what I think is the most “versatile” method – defining the Node on the Master instance, and manually setting up and starting the corresponding Agent/Client Node on the remote host – going through these steps should provide enough detail on how Agent Nodes work and connect to get one up and running on anything that can run a JVM.

1. On the Master/Server

Define the host: navigate to Jenkins > Manage Jenkins > Manage Nodes > New Node
Enter a suitable Node Name (I’d recommend something descriptive, and usually including the host name or part of it) then either select to create a “Dumb Agent” or copy an existing Node if you have one, then complete the configuration page similar to this:


where you specify the requested properties – path, labels, usage, executors etc. These are explained in more detail in the “?” for each item if required.

Here you can also state if you want to keep your Jenkins Agent for tied jobs only, or if it is to be utilized as much as possible – this obviously depends on your requirements. You can also specify the Launch method that best applies to your needs & requirements.

2. On the Agent/Client host

You don’t need to do very much to create a new agent node – typically if I’m setting up a few *NIX and Windows hosts I would archive a simple shell/DOS script that starts and manages the process along with the slave.jar file from the Master Jenkins instance. There are alternative methods that may suit your needs – you can start agents via SSH from the Master server for example and there’s a comparable method for Windows – but this simple approach should help you understand the underlying idea that applies to them all.

You can “wget” (or use a Browser on Windows) the slave.jar file directly from the Master Jenkins instance using the URL

http://[your jenkins host]:[port number]/jnlpJars/slave.jar

If you let JNLP initiate the process, the slave.jar will be downloaded from Jenkins automatically.

Note that Jenkins will inherit the effective permissions of the user that starts the process – this is to be expected, but it’s often worth having a think about the security aspects of this, along with the access requirements for the types of things you want your agent to be able to do… or not do.

On Windows hosts, you can use the jenkins-agent.exe to easily install Jenkins as a Windows Service, which can then be started at boot time and run under whatever user/permissions you wish set via the Services panel.

My *NIX “startagent.sh” script does a few environment/sanity checks, then kicks of the agent process something like so:

${NOHUP} ${JAVA} -jar slave.jar -jnlpUrl http://SERVERNAME:PORT/computer/USER__NODENAME/slave-agent.jnlp &

The HTTP URL there should match the one provided by the Jenkins page when you were defining the Node. If all goes well you should see the node state changed to Connected on the Master Hudson instance, and if not, then nohup.out should provide some pretty obvious pointers on the problem.

Some common causes are:

Jenkins host, port or node name wrong
Java version not found/wrong
Lack of write permissions to the file system
Lack of space (check /tmp too)
Port already in use
Errors in the jenkins-slave.xml file if you’ve tweaked it
Firewalls…

Jenkins also provides some health monitoring of the connected Node which you can see in the Jenkins > Nodes page:
Disk Space, Free Temp Space, Clock time/sync, Response Time and Free Swap are monitored
and you can have your Node taken off line if any of these passes a set threshold.

This should hopefully be enough info to provide an overview of what Jenkins Agents are, and enough to get one up and running on your chosen platform. Where possible it’s best to keep things simple – use SSH and let the Master instance manage things for you if you can – but when that’s not possible there are alternatives.

When I get the chance I will add some information on the next steps – creating and delegating jobs on Jenkins Agent Nodes, and some thoughts and suggestions for just a few of the many uses for this sort of distributed Build and Deployment system.

Related Posts and Links:

Monitoring Jenkins Slave Nodes with Groovy
– how to keep an  eye on your Jenkins Slaves

Jenkins Slave Nodes – using the Swarm Plugin
– automatically connect new Slave Nodes to create a “Swarm”

Getting the current user in Jenkins
– several approaches

Managing Jenkins as a service and starting at boot time
– on Linux & Windows

Jenkins plugins
– details on some of my most frequently used plugins

Jenkins DIY Information Radiators
– what they are for, and how to make your own

The Jenkins Wiki has more details information on Distributed Builds and different slave-launching strategies.

Feedback, questions and constructive comments are very welcome!

Pin It on Pinterest

%d bloggers like this: