This is a hands-on video course packed with practical examples to get you started with Docker.
Here is the course overview video:
And here is a free sample video from Section 2, “Docker Basics” where we take a look at running containers and the 3 different types of “containerized” commands:
and this final sample video is taken from Section 5 – “Running a Web Application with Docker“.
In this clip we build our own web application using Python, pip and Redis, which we will then “dockerize” and ship to “production”:
About This Video
Master Docker commands by creating and publishing a sample web application
Build and manage your own custom Docker Containers to set up data sources, filesystems, and networking
Build your own personal Heroku PaaS with Dokku
Who This Video Is For
If you’re a developer who wants to learn about Docker, a powerful tool to manage your applications effectively on various platforms, this course is perfect for you! It assumes basic knowledge of Linux but supplies everything you need to know to get your own Docker environment up and running.
What You Will Learn
Build new Docker containers and find and manage existing ones
Use the Docker Index, and create your own private one by using containers
Discover ways to automate Docker, and harness the power of containers!
Build your own Docker powered mini-Heroku Paas with Dokku
Set up Docker on your environment based on your application’s custom requirements
Master Docker patterns and enhancements using the Ambassador and Minimal containers
In Detail
One of the major challenges while creating an application is adapting your application to run smoothly on all of the plethora of operating systems available. Docker is an extremely efficient technology that allows you to wrap all your code along with its supporting files into a single bundle; it also guarantees that your application will behave in the same way on any host powered by Docker. You can also easily reuse existing Docker containers or create and publish your own. Unlike Virtual Machines, Docker containers are lightweight and more efficient.
Beginning Docker starts with the fundamentals of Docker—explaining how it works, how to set it up, and how to get started on leveraging the benefits of this technology. The course goes on to cover more advanced features and shows you how to create and share your own Docker images.
You will learn how to install Docker on your own machine, then how to manage it effectively, and then progress to creating and publishing your very own application. You will then learn a bit more about Docker Containers; built-in features and commands such as volumes, mounts, ports, and linking and constraining containers; before diving into running a web application.
Docker has functionality such as the Docker web API to handle complex automation processes which will be explained in detail. You will also learn how to use the Docker Hub to fetch and share containers, before running through the creation of your own Docker powered mini-Heroku
Beginning Docker covers everything required to get you up and running with Docker, with detailed real-world examples and helpful tips to make sure you get the most from it.
Style and Approach
An easy-to-follow and structured video tutorial with practical examples of Docker to help you get to grips with each and every aspect.
The course will take you on a journey from the basics to the advanced application of Docker containers, and includes several real-world scenarios to learn from.
This post lists the steps taken to get started with Docker – the basic install and getting a Docker container up and running on Ubuntu.
There’s nothing new or unusual in this section, but it forms the background of the next post(s) where I plan to go in to detail on different approaches to using Docker with Jenkins (and vice versa).
If you’re unfamiliar with Docker here is a good introduction:
In my case the Ubuntu is a pretty ordinary “ubuntu 14.04.1 LTS (GNU/Linux 3.13.0-35-generic x86_64)” running as a guest VM on my Development VM Ware ESXi Server.
Installing docker on Ubuntu is trivial – here are the commands I found via a quick google, there were no issues…
apt-get -y install docker.io
(FYI it’s called docker.io as there’s a previous/existing package with the name docker)
and that’s that done – all very simple, quick and straightforward, now it’s time to pull down an Ubuntu image…
docker pull ubuntu
once that’s done, you are ready to run a command (bash in the case of this quick test I found suggested elsewhere) in a docker container like so:
docker run -i -t ubuntu /bin/bash
(-i attaches stdin & stdout and -t allocates a terminal)
That’s it for this post – the next will look at dynamic Jenkins Slave provisioning using Docker Containers, Jenkins plugins for Docker and ways to use Docker for a variety of Jenkins build, deployment and test tasks for Continuous Integration, Continuous Build and DevOps purposes.
In Part I, Information Radiators, I covered what they are, what the main benefits are, and the approach I usually use to set them up. This post goes in to more technical detail on how I extract this data from Jenkins.
My usual setup/architecture for Jenkins Information Radiators goes something along these lines:
TV screens running Mozilla Firefox or Google Chrome in Kiosk Mode, and Tab Mix Plus set up to rotate tabs (if required)
JSP Pages served via Tomcat on Linux server (which also runs the data extracting script described below)
MySQL database on Linux server – contains tables with data pulled from Jenkins and other sources, and the config data too (which URL’s to monitor)
And you’ll need some Jenkins instances/jobs to monitor too, obviously 🙂
The Jenkins XML API is very useful for automating tasks like this – if you simply append “/api/xml” to a
Jenkins job URL, it will serve up an XML version – note there is also a JSON API and a CLI and plenty of other options, but I’m using what suits me.
The Jenkins XML API
For example, if you go to one of your Jenkins jobs and add /api/xml like this:
That XML contains loads of very useful information inside handy XML tag descriptions – you just need a way to get at that data and then you can present it as you like…
XPAth queries and the Jenkins XML API
so to automate that, I used to extend that approach a to query Jenkins via the XML API using XPAth queries to bring back just the data I actually wanted, quite like querying a database.
For example, wget’ing this URL would return just the current value of the <building> tag in the above XML:
e.g. “true” or “false” – this was very useful and easy to do, but the functionality was removed/disabled in recent versions of Jenkins for security reasons, meaning that my processes that used it needed rewritten 🙁
Extracting the data – Plan B…
So, here’s the new solution I went for – the real scripts/methods do some error handling and cleaning up etc but I’m just highlighting the main functions and the high level logic behind each of them here;
get_url’s method:
query a table in MySQL that contains a list of the job names and URL’s to monitor
for each $JOB_NAME found, it calls the get_file method, passing that the URL as a parameter.
get_file method:
this takes a URL param, and uses curl to fetch and save the XML data from that URL to a temporary file (“xmlfile”):
curl -sL "$1" | xmllint --format - > xmlfile
Note I’m using “xmllint –format” there to nicely format the XML data, which makes processing it later much easier.
get_data method:
this first calls “get_if_building” (see below) to see if the job is currently running or not, then it does:
TRUE_VAR="true"
if [[ "$IS_BUILDING" == "$TRUE_VAR" ]]; then
RESULT_TEXT="building..."
else
RESULT_TEXT=`grep "result>" xmlfile | awk -F\> '{print $2}' | awk -F\< '{print $1}'`
fi
get_if_building method:
this simply checks and sets the IS_BUILDING var like so:
My script then updates the MySQL database with the results from each check: success/failure, date, build number, user, change details etc
I then have JSP pages that read data from that table, and translate things like true/false in to HTML that sets the background colours (Red, Amber, Green), and shows the appropriate blocks and details per job.
If you have a few browsers/TV’s or Monitors showing these strategically placed around the office, developers get rapid feedback on the result of their code changes which speeds up development, increases quality and reduces development time and costs – and they can be fun to watch and set up too 🙂
Information Radiators are used to provide people with feedback on the current status of code builds and automated tests in Continuous Integration and Agile development environments.
The basic idea is that when developers commit a code change, they can easily and quickly see that it has been picked up by the automated build process, and then (ideally within 10 minutes) see the result of their change; did the build succeeded and did the automated tests pass?
Martin Fowler’s description goes in to more detail on the ideas behind this approach and the function that Information Radiators serve.
The normal convention for these is to use colour coded blocks per build, using:
Green for good/passing jobs
Amber for either currently building/running jobs (or sometimes for unstable ones)
Red for failed jobs
Generally you want to keep things as clean, simple and uncluttered as possible, but sometimes it is helpful to add in a bit more info.
Details I have occasionally found worth adding include things like;
name and/or picture of the user who triggered (or broke!) the build
commit message from the code change that triggered the build
build number
history – number of recent fails or passes
date/time last failed and last checked
if you use amber for “unstable” rather than “build in progress”, you may want to add text to say if the job is currently building – I often use the “spinning wheel” icon thingy from jenkins itself :
Why build your own?
There are tons of readily available plugins that allow you to quickly and easily produce a Radiator or Wall panel from Jenkins, so why go to the bother of making your own?
Plugins are usually linked to one Jenkins instance (the one they are running on) and I have often found that alone to be too restrictive – having too many different radiators all over the place makes things too cluttered and uncertain, and people can easily start to “switch off” from them all – having one screen that people can understand at a glance usually works best.
Changing requirements – developers are constantly wanting/looking to improve processes and often come up with ideas and requests for things to try that may help them do their jobs – adding a bit of information from another source, for example, or changing the colours used to a different shade, or adding curved borders etc etc…
What I have found often works best, is to get all of the data I am interested in inside a database then write my own simple but flexible presentation layer from scratch – this gives me all the flexibility I could want (or may find myself wanting or needing later…) and importantly, it also allows me to leverage additional benefits by combining data from Jenkins with data from elsewhere – this means I can easily produce reports, charts, metrics etc that present a view comprised from multiple data sources throughout an organisation – for example, you can then easily create reports that combine:
jenkins – live information on the current state of builds and tests
defects – data on bugs and changes pulled from bug tracking apps (usually via jenkins jobs)
version control – the actual code change can be extracted from version control and linked to both the developer and bug details – the “svn log” command is useful for this
environment monitors – state and health of environments; database and app server health, deployed patch and code levels etc (again, these are usually other Jenkins jobs!)
and you can add in anything else you can get your hands on 🙂
This allows you to track the flow of a change right through the development life cycle – from the initial change/requirement/defect to the change itself at the code/file level, then the testing and building of it and the eventual release. This is far more than you need for a simple Information Radiator, but using this approach means that you can easily reuse much of the work in different ways.
I’ve been trying out a new (to me at least) way to add a Jenkins Agent Node – using UDP auto discovery via the Jenkins Swarm Plugin
This is a very easy and nice way to do it, with minimal configuration/hassle required so you can quickly and easily add new Jenkins Agent Nodes to your Master Jenkins instance as and when they are required.
Here are my notes from setting this up – it’s pretty simple to do and worked well for me out of the box…
After that, you should get console output along these lines…
Running from: /root/jenkinsswarm/jenkins.war
webroot: $user.home/.jenkins
18-Mar-2013 15:19:26 winstone.Logger logInternal
INFO: Beginning extraction from war file
Jenkins home directory: /root/.jenkins found at: $user.home/.jenkins
18-Mar-2013 15:19:33 winstone.Logger logInternal INFO: HTTP Listener started: port=8080
18-Mar-2013 15:19:33 winstone.Logger logInternal
INFO: Winstone Servlet Engine v0.9.10 running: controlPort=disabled
18-Mar-2013 15:19:34 jenkins.InitReactorRunner$1 onAttained
INFO: Started initialization
18-Mar-2013 15:19:35 jenkins.InitReactorRunner$1 onAttained
INFO: Listed all plugins
18-Mar-2013 15:19:35 jenkins.InitReactorRunner$1 onAttained
INFO: Prepared all plugins
18-Mar-2013 15:19:35 jenkins.InitReactorRunner$1 onAttained
INFO: Started all plugins
18-Mar-2013 15:19:41 jenkins.InitReactorRunner$1 onAttained
INFO: Augmented all extensions
18-Mar-2013 15:19:41 jenkins.InitReactorRunner$1 onAttained
INFO: Loaded all jobs
18-Mar-2013 15:19:44 org.jenkinsci.main.modules.sshd.SSHD start
INFO: Started SSHD at port 25133
18-Mar-2013 15:19:44 jenkins.InitReactorRunner$1 onAttained
INFO: Completed initialization
18-Mar-2013 15:19:44 hudson.TcpAgentListener <init>
INFO: JNLP agent listener started on TCP port 41790
18-Mar-2013 15:19:44 hudson.WebAppMain$2 run INFO: Jenkins is fully up and running
– that looks happy enough, and as you can see from the line “HTTP Listener started: port=8080” it’s running on the default port, so connect to http://yourhost:8080 and you should see something like this…
java -jar swarm-client-1.8-jar-with-dependencies.jar
Found 1 eligible Jenkins.
Connecting to http://mydomain.com:8080/
Attempting to connect to http://mydomain.com:8080/ a2721b16-04e4-0d962
18-Mar-2013 15:33:22 org.apache.commons.httpclient.HttpMethodDirector authenticateHost
WARNING: Required credentials not available for BASIC <any realm>@mydomain.com:8080
18-Mar-2013 15:33:22 org.apache.commons.httpclient.HttpMethodDirector authenticateHost
WARNING: Preemptive authentication requested but no default credentials available
18-Mar-2013 15:33:23 hudson.remoting.jnlp.Main$CuiListener <init>
INFO: Hudson agent is running in headless mode.
18-Mar-2013 15:33:23 hudson.remoting.jnlp.Main$CuiListener status
INFO: Locating server among [http://mydomain.com:8080/]
18-Mar-2013 15:33:23 hudson.remoting.jnlp.Main$CuiListener status
INFO: Connecting to myhost.mydomain.com:43932
18-Mar-2013 15:33:23 hudson.remoting.jnlp.Main$CuiListener status
INFO: Handshaking
18-Mar-2013 15:33:23 hudson.remoting.jnlp.Main$CuiListener status
INFO: Connected
Now take a look at your browser and you should see a new Node automatically added to the Master Jenkins instance…
A very handy and flexible approach to adding/managing Nodes and workload – many thanks to the developers behind this!
I have an Ubuntu Linux VM that runs Jenkins, and to make life simpler I wanted to set up Jenkins to run as a service. I also wanted it to start up automatically at boot time, as that’s the sole function of the VM it’s running on.
So, here are my notes on setting up Jenkins as a service on Ubuntu Linux, which includes a script to manage (start, stop, check, restart) the Jenkins process too.
On Ubuntu (and most othe rLinux versions) you can check the current services with the command “service –status-all” – this should give you a list of all services, and their current status. You can (as root) also do “service <name> start” (or stop or restart) for each one.
Jenkins as a Service
To create a new service for Jenkins, take a look at the existing scripts in /etc/init.d/ for some examples.
By convention there are three main methods a basic service implements, they are; start, stop and restart. About the most basic structure of a service script is therfore something like this:
#!/bin/bash
start() {
echo “Starting Service”
# Do start things here.
}stop() {
echo “Stopping Service”
# Do stop things here.
}restart() {
echo “Restarting Service…”
stop
start
}
case “$1″ in
start)
start
;;
stop)
stop
;;
restart)
restart
;;
*)
obviously you still need to implement the start and stop functions though 🙂
To see a far more detailed example, take a look at the file /etc/init.d/skeleton, which you could copy and update to suit your needs if you prefer.
I decided on something halfway between the very basic example above and the far more complex example in “skeleton”, and did something like this (saved as “/etc/init.d/jenkins”):
# Example Jenkins auto-start service script
#
# description: manages Jenkins as a service
# processname: jenkins
# pidfile: /var/run/jenkins.pid
# author: www.DonaldSimpson.co.uk# The user and the home dir that Jenkins runs under
jenkins=/usr/local/jenkins
# Your startup and stop scripts (see below)
startup=$jenkins/bin/startup.sh
shutdwn=$jenkins/bin/shutdown.shstart_jenkins() {
echo “Starting Jenkins services…”
su – jenkins -c “sh $startup”
}
status_jenkins() {
# Check for any other process containing jenkins.war
# This could be improved upon (see script below)
numproc=`ps -ef | grep [j]enkins.war | wc -l`
if [ $numproc -ne 0 ]; then
echo “Jenkins is running…”
else
echo “Jenkins is NOT running…”
fi
}
Update that to suit then save and change the permissions to make it executable:
chmod +x /etc/init.d/jenkins
then you can check (as root) that you can call the methods in the script:
service jenkins status
service jenkins stop
service jenkins start
These should all now run as the Jenkins user via sudo and say something when called, even if it is just “I can’t find the scripts you told me to call” 🙂
Jenkins Startup and Stop scripts
So, you now need to create the missing startup and shutdown scripts, in my example they were:
/usr/local/jenkins/bin/startup.sh
and
/usr/local/jenkins/bin/shutdown.sh
An example of the Jenkins start-up and management script I normally use is at the end of this post – the main idea though, is that it sets what you want to set then starts Jenkins via nohup and outputs to a log file
I have also added and included some basic tests to my scripts and some (very) rudimentary error handling/checking, but you shoudl get the idea and all you really need is this line (with the variables set correctly):
Note: I often run multiple instances of Jenkins, so I explicitly specify the HTTP_PORT, and I use the -D${MARKER} to allow me to safely and easily find the correct PID for each project.
The httpListenAddress can normally go unset, but it’s something I’ve had to set before on multi-home’d hosts.
The stop part of my Jenkins management script finds the correct PID like this (and you could use a filter for the correct -D${MARKER} if you want that too):
For some of my scripts I also filter for process ID’s that were started in the current directory by checking “pwdx” against the PID, but only where I’m sure the corresponding start-up process is correct/reliable.
So, tweak to your taste and that should be the Jenkins service created and working too now; for start|stop|restart you can create individual scripts or wrappers that call something like the one script below with parameters, or break the script in to separate files if you prefer.
Setting Jenkins to start at boot time
If you want to start Jenkins at boot time/startup automatically then you still need to do one more small step. There are many different ways to do this depending on personal preference, your requirements and your version of Linux, but on Ubuntu it can be done easily with:
update-rc.d jenkins defaults
“update-rc.d” simply installs and removes System-V style init script links – read the man page for full details, but the same idea applies to most versions of Linux.
update-rc.d -f jenkins remove
will undo this if you no longer want it.
An example Jenkins server/process management script
As an example, here is my script to manage Jenkins processes – it could use some improvements but the basic start, stop, restart and check should give you enough to sort out something that works and suits your needs.
#! /bin/bash -p
# www.donaldsimpson.co.uk
# Script to start|stop|restart|check an instance of Jenkins
# For each new instance, the PROJECT and HTTP_PORT need to be updated:
export PROJECT=jenkins
export HTTP_PORT=9000
EXIT_STRING=””
JENKINS_ROOT=/opt/apps/jenkins
export JENKINS_HOME=${JENKINS_ROOT}/${PROJECT}
JENKINS_WAR=${JENKINS_HOME}/jenkins.warWAIT_TIME=5
START_WAIT_TIME=15
JAVA_HOME=/usr
PATH=$JAVA_HOME/bin:$PATH
JAVA=${JAVA_HOME}/bin/java
NOHUP=/usr/bin/nohup
LOG_FILE=${JENKINS_HOME}/debug_${PROJECT}.log
MARKER=”JenkinsProcFor_${PROJECT}”NC=/bin/nc
WGET=/usr/bin/wget
LSOF=/usr/bin/lsof
AWK=/usr/bin/awk
GREP=/bin/grep
FUSER=/bin/fuser
#################################################################################
### Functions Start #############################################################
#################################################################################
check_folders(){
say “”
saybold “Checking all required folders exist…”
for REQUIRED_DIR in ${JENKINS_ROOT} ${JENKINS_HOME} ${JAVA_HOME}
do
[ ! -d “${REQUIRED_DIR}” ] && _error “Necessary directory: ${REQUIRED_DIR} does not exist”
say “Found required directory: ${REQUIRED_DIR}”
done
saybold “Done.”
say “”
}
check_files(){
say “”
saybold “Checking all required files exist…”
for REQUIRED_FILE in ${JENKINS_WAR} ${NC} ${WGET} ${JAVA} ${LSOF} ${FUSER} ${GREP} ${AWK}
do
[ ! -f “${REQUIRED_FILE}” ] && _error “Necessary file: ${REQUIRED_file} does not exist”
say “Found required file ${REQUIRED_FILE}”
done
saybold “Done.”
say “”
}
check_port_closed(){
say “Checking that port ${HTTP_PORT} is closed…”
${NC} -w 1 localhost ${HTTP_PORT}
if [ $? -eq 0 ]; then
_error “Required Jenkins port ${HTTP_PORT} is already in use”
else
say “Ok, required port ${HTTP_PORT} is available, continuing…”
fi
}
check_port_open(){
say “Checking that port ${HTTP_PORT} is open…”
${NC} -w 1 localhost ${HTTP_PORT}
if [ $? -eq 0 ]; then
say “Ok, a process is listening on port ${HTTP_PORT}, continuing…”
else
_error “Required Jenkins port ${HTTP_PORT} has not been opened.”
fi
}
start_process(){
cd ${JENKINS_HOME}
saybold “Starting Process now…”
${NOHUP} ${JAVA} -jar ${JENKINS_WAR} -D${MARKER} –httpListenAddress=0.0.0.0 –httpPort=${HTTP_PORT} > ${LOG_FILE} &
say “Process initiated.”
}
check_log(){
say “Checking the log file ${LOG_FILE} for an HTTP listener…”
STARTED=`${GREP} -c “HTTP Listener started” ${LOG_FILE}`
if [ ${STARTED} -eq “0” ]; then
_error “An HTTP Listener has not reported as started in the log file ${LOG_FILE}”
else
saybold “An HTTP Listener is reported as started in the log file ${LOG_FILE}”
fi
}
check_html(){
# These checks need error handling, but you get the general idea.
say “Checking that localhost:${HTTP_PORT} is serving Jenkins pages…”
TEMP_WGETDIR=tempwgetdir$$
mkdir ${TEMP_WGETDIR}
cd ${TEMP_WGETDIR}
${WGET} -q http://localhost:${HTTP_PORT}
GOT_HTML=`${GREP} -c Dashboard index.html`
cd ${JENKINS_HOME}
rm -rf ${TEMP_WGETDIR}
if [ ${GOT_HTML} -eq “0” ]; then
_error “Unable to get an HTML page from the Server.”
fi
saybold “Recieved valid HTML from the server, all looks ok.”
}
check_process(){
check_port_open
check_log
check_html
say “A Jenkins instance is listening on port ${HTTP_PORT} for project ${PROJECT}.”
say “The process is logging debug info to the log file: ${LOG_FILE}”
}
stop_proc(){
check_port_open
saybold “Looking for the Process ID attached to port ${HTTP_PORT}”
PID=`${LSOF} -w -n -i tcp:${HTTP_PORT} | ${GREP} -v COMMAND | ${AWK} {‘print $2’}`
if [ “${PID}” == “” ]; then
saybold “Unable to detect the Process ID that is listening on port ${HTTP_PORT}!”
PID=`${FUSER} ${LOG_FILE}`
if [ “${PID}” == “” ]; then
_error “Unable to find the PID that has the log file open too!”
else
say “Ok, found PID ${PID}”
fi
else
saybold “Found a PID of $PID, killing it now…”
fi
kill -9 ${PID}
say “Waiting ${WAIT_TIME} seconds for the process to die…”
sleep ${WAIT_TIME}
saybold “Done, checking port is closed…”
check_port_closed
saybold “All done.”
}
start(){
check_folders
check_files
check_port_closed
start_process
saybold “Waiting ${START_WAIT_TIME} seconds for the process to start up…”
sleep ${START_WAIT_TIME}
check_process
}
Jenkins Agents are small Java “Client” processes that connect back to a “Master” Jenkins instance over the Java Network Launch Protocol (JNLP).
Why may I want one?
Once it’s up and running, an Agent instance can be used to run tasks from a Master Jenkins instance on one or more remote machines providing an easy to use and flexible distributed system which can lend itself to a wide variety of tasks.
As these are Java processes, you are not restricted by architecture and can mix and match the OS’s of your agent nodes as required – Windows, Linux, UNIX, iSeries, OVMS etc – anything capable of running a modern version of Java (I think JNLP was introduced in 1.5?) and you can also group and categorize subsets of different types (both logical and physical) of Agents; intended use, availability, location, available resources, Cloud or VM versus Physical tin – anything that helps you decide when you want to use which host.
There are many different ways you can choose to utilize these nodes – they can be used to spread the load of an intensive build process over many machines when they are available, you can delegate specific tasks to specific machines only, or you can use labels to group different classes or types of Nodes that are available for certain tasks, making the most use of your available resources. You can also have Jenkins create Cloud server instances – Amazon EC2 for example – when certain thresholds are reached, and stop them when they are no longer required.
This post focuses on a pretty manual approach to the creation of Jenkins Agent Nodes with the intention of explaining them well enough to allow you to create them on any platform that can run a modern version of Java – there are probably simpler solutions depending on your needs and setup. A later post will touch on a few of the many possible uses for these nodes.
So, how do you create one?
There are several different ways to go about setting up an Agent, and the “best” approach depends on your situation, needs and environment(s) – for a simple Linux setup letting Jenkins do all the work for you makes life really easy, you can just select that option in your new Jenkins Agent Node and complete this screen to have Jenkins set it up for you:
Where the Username and Password are the credentials you want Jenkins to use to connect and start the Agent process on the remote server. This simple approach also allows the Master instance to initiate the JNLP connection and bring your agents online when required, avoiding any need to manually visit each agent node.
This keeps things nice and simple and reduces the admin overhead too, but sometimes this type of approach can’t be used (on different OS’s like OVMS, iSeries, Windows etc) and I’m going to go on to outline what I think is the most “versatile” method – defining the Node on the Master instance, and manually setting up and starting the corresponding Agent/Client Node on the remote host – going through these steps should provide enough detail on how Agent Nodes work and connect to get one up and running on anything that can run a JVM.
1. On the Master/Server
Define the host: navigate to Jenkins > Manage Jenkins > Manage Nodes > New Node
Enter a suitable Node Name (I’d recommend something descriptive, and usually including the host name or part of it) then either select to create a “Dumb Agent” or copy an existing Node if you have one, then complete the configuration page similar to this:
where you specify the requested properties – path, labels, usage, executors etc. These are explained in more detail in the “?” for each item if required.
Here you can also state if you want to keep your Jenkins Agent for tied jobs only, or if it is to be utilized as much as possible – this obviously depends on your requirements. You can also specify the Launch method that best applies to your needs & requirements.
2. On the Agent/Client host
You don’t need to do very much to create a new agent node – typically if I’m setting up a few *NIX and Windows hosts I would archive a simple shell/DOS script that starts and manages the process along with the slave.jar file from the Master Jenkins instance. There are alternative methods that may suit your needs – you can start agents via SSH from the Master server for example and there’s a comparable method for Windows – but this simple approach should help you understand the underlying idea that applies to them all.
You can “wget” (or use a Browser on Windows) the slave.jar file directly from the Master Jenkins instance using the URL
If you let JNLP initiate the process, the slave.jar will be downloaded from Jenkins automatically.
Note that Jenkins will inherit the effective permissions of the user that starts the process – this is to be expected, but it’s often worth having a think about the security aspects of this, along with the access requirements for the types of things you want your agent to be able to do… or not do.
On Windows hosts, you can use the jenkins-agent.exe to easily install Jenkins as a Windows Service, which can then be started at boot time and run under whatever user/permissions you wish set via the Services panel.
My *NIX “startagent.sh” script does a few environment/sanity checks, then kicks of the agent process something like so:
The HTTP URL there should match the one provided by the Jenkins page when you were defining the Node. If all goes well you should see the node state changed to Connected on the Master Hudson instance, and if not, then nohup.out should provide some pretty obvious pointers on the problem.
Some common causes are:
Jenkins host, port or node name wrong
Java version not found/wrong
Lack of write permissions to the file system
Lack of space (check /tmp too)
Port already in use
Errors in the jenkins-slave.xml file if you’ve tweaked it
Firewalls…
Jenkins also provides some health monitoring of the connected Node which you can see in the Jenkins > Nodes page:
Disk Space, Free Temp Space, Clock time/sync, Response Time and Free Swap are monitored
and you can have your Node taken off line if any of these passes a set threshold.
This should hopefully be enough info to provide an overview of what Jenkins Agents are, and enough to get one up and running on your chosen platform. Where possible it’s best to keep things simple – use SSH and let the Master instance manage things for you if you can – but when that’s not possible there are alternatives.
When I get the chance I will add some information on the next steps – creating and delegating jobs on Jenkins Agent Nodes, and some thoughts and suggestions for just a few of the many uses for this sort of distributed Build and Deployment system.