Jenkins Slave Nodes

 

 

This Jenkins Slave Nodes post covers:

  • What are they?
  • Why may I want one?
  • How do you create one?
    • tasks on On the Master/Server
    • tasks on On the Slave/Client
  • Other ways of creating Slave Nodes
  • Related posts and links

What are they?

Jenkins Slave Nodes are small Java “Client” processes that connect back to a “Master” Jenkins instance over the Java Network Launch Protocol (JNLP).

Why may I want one?

Once it’s up and running, a Slave instance can be used to run tasks from a Master Jenkins instance on one or more remote machines providing an easy to use and flexible distributed system which can lend itself to a wide variety of tasks.

As these are Java processes, you are not restricted by architecture and can mix and match the OS’s of your slave nodes as required – Windows, Linux, UNIX, iSeries, OVMS etc – anything capable of running a modern version of Java (I think JNLP was introduced in 1.5?) and you can also group and categorize subsets of different types (both logical and physical) of Slaves; intended use, availability, location, available resources, Cloud or VM versus Physical tin – anything that helps you decide when you want to use which host.

There are many different ways you can choose to utilize these nodes – they can be used to spread the load of an intensive build process over many machines when they are available, you can delegate specific tasks to specific machines only, or you can use labels to group different classes or types of Nodes that are available for certain tasks, making the most use of your available resources. You can also have Jenkins create Cloud server instances – Amazon EC2 for example – when certain thresholds are reached, and stop them when they are no longer required.

This post focuses on a pretty manual approach to the creation of Jenkins Slave Nodes with the intention of explaining them well enough to allow you to create them on any platform that can run a modern version of Java – there are probably simpler solutions depending on your needs and setup. A later post will touch on a few of the many possible uses for these nodes.

So, how do you create one?

There are several different ways to go about setting up a Slave, and the “best” approach depends on your situation, needs and environment(s) – for a simple Linux setup letting Jenkins do all the work for you makes life really easy, you can just select that option in your new Jenkins Slave Node and complete this screen to have Jenkins set it up for you:

Where the Username and Password are the credentials you want Jenkins to use to connect and start the Slave process on the remote server. This simple approach also allows the Master instance to initiate the JNLP connection and bring your slaves online when required, avoiding any need to manually visit each slave node.

This keeps things nice and simple and reduces teh admin overhead too,  but sometimes this type of approach can’t be used (on different OS’s like OVMS, iSeries, Windows etc) and I’m going to go on to outline what I think is the most “versatile” method – defining the Node on the Master instance, and manually setting up and starting the corresponding Slave/Client Node on the remote host – going through these steps should provide enough detail on how Slave Nodes work and connect to get one up and running on anything that can run a JVM.

1. On the Master/Server

Define the host: navigate to Jenkins > Manage Jenkins > Manage Nodes > New Node
Enter a suitable Node Name (I’d recommend something descriptive, and usually including the host name or part of it) then either select to create a “Dumb Slave” or copy an existing Node if you have one, then complete the configuration page similar to this:


where you specify the requested properties – path, labels, usage, executors etc. These are explained in more detail in the “?” for each item if required.

Here you can also state if you want to keep your Jenkins Slave for tied jobs only, or if it is to be utilized as much as possible – this obviously depends on your requirements. You can also specify the Launch method that best applies to your needs & requirements.

2. On the Slave/Client

You don’t need to do very much to create a new slave node – typically if I’m setting up a few *NIX and Windows hosts I would archive a simple shell/DOS script that starts and manages the process along with the slave.jar file from the Master Jenkins instance. There are alternative methods that may suit your needs – you can start slaves via SSH from the Master server for example and there’s a comparable method for Windows – but this simple approach should help you understand the underlying idea that applies to them all.

You can “wget” (or use a Browser on Windows) the slave.jar file directly from the Master Jenkins instance using the URL

http://[your jenkins host]:[port number]/jnlpJars/slave.jar

If you let JNLP initiate the process, the slave.jar will be downloaded from Jenkins automatically.

Note that Jenkins will inherit the effective permissions of the user that starts the process – this is to be expected, but it’s often worth having a think about the security aspects of this, along with the access requirements for the types of things you want your slave to be able to do… or not do.

On Windows hosts, you can use the jenkins-slave.exe to easily install Jenkins as a Windows Service, which can then be started at boot time and run under whatever user/permissions you wish set via the Services panel.

My *NIX “startslave.sh” script does a few environment/sanity checks, then kicks of the slave process something like so:

${NOHUP} ${JAVA} -jar slave.jar -jnlpUrl http://SERVERNAME:PORT/computer/USER__NODENAME/slave-agent.jnlp &

The HTTP URL there should match the one provided by the Jenkins page when you were defining the Node. If all goes well you should see the node state changed to Connected on the Master Hudson instance, and if not, then nohup.out should provide some pretty obvious pointers on the problem.

Some common causes are:

Jenkins host, port or node name wrong
Java version not found/wrong
Lack of write permissions to the file system
Lack of space (check /tmp too)
Port already in use
Errors in the jenkins-slave.xml file if you’ve tweaked it
Firewalls…

Jenkins also provides some health monitoring of the connected Node which you can see in the Jenkins > Nodes page:
Disk Space, Free Temp Space, Clock time/sync, Response Time and Free Swap are monitored
and you can have your Node taken off line if any of these passes a set threshold.

This should hopefully be enough info to provide an overview of what Jenkins Slaves are, and enough to get one up and running on your chosen platform. Where possible it’s best to keep things simple – use SSH and let the Master instance manage things for you if you can – but when that’s not possible there are alternatives.

When I get the chance I will add some information on the next steps – creating and delegating jobs on Jenkins Slave Nodes, and some thoughts and suggestions for just a few of the many uses for this sort of distributed Build and Deployment system.

Related Posts and Links:

Monitoring Jenkins Slave Nodes with Groovy
– how to keep an  eye on your Jenkins Slaves

Jenkins Slave Nodes – using the Swarm Plugin
– automatically connect new Slave Nodes to create a “Swarm”

Getting the current user in Jenkins
– several approaches

Managing Jenkins as a service and starting at boot time
– on Linux & Windows

Jenkins plugins
– details on some of my most frequently used plugins

Jenkins DIY Information Radiators
– what they are for, and how to make your own

The Jenkins Wiki has more details information on Distributed Builds and different slave-launching strategies.

Feedback, questions and constructive comments are very welcome!

 

25 thoughts on “Jenkins Slave Nodes”

  1. very very informative, if needed the information to create and configure a slave node from scratch…..
    !but some need to be corrected like ‘urls’……
    …..any ways thanks a lot for this rare information…..!! i really appreciate your work, keep posting…..

  2. Please post the second part to this tutorial as I’ve been able to do everything listed here fine. It’s the next steps I can’t find any information about on what do you configure on the slave machine to make the jobs build there.

    1. I’m also waiting for the second part. I’ve read this tutorial a hundred times. No one posted the second part..about how to make the slaves to do the build..

  3. How i can change slave of one type to others means suppose i configured as a dumb slave how i ll change the same node to Slave virtual computer running under vSphere Cloud.?

  4. Hi,
    I am facing one issue while adding new linux node to jenkins master(linux machine)
    Here is the error.
    [10/24/14 09:47:30] [SSH] Opening SSH connection to hostname:22.
    [10/24/14 09:47:30] [SSH] Authentication successful.
    SSH connection reports a garbage before a command execution.
    Check your .bashrc, .profile, and so on to make sure it is quiet.
    The received junk text is as follows:
    stty: standard input: Invalid argument
    stty: standard input: Invalid argument

    hudson.AbortException
    at hudson.plugins.sshslaves.SSHLauncher.verifyNoHeaderJunk(SSHLauncher.java:854)
    at hudson.plugins.sshslaves.SSHLauncher.access$100(SSHLauncher.java:134)
    at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:698)
    at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:691)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
    at java.util.concurrent.FutureTask.run(FutureTask.java:166)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:701)
    [10/24/14 09:47:30] Launch failed – cleaning up connection
    [10/24/14 09:47:30] [SSH] Connection closed.

    Not able to understand the above error.

    Could you please help here?

  5. Hi,

    It looks like there’s something in one of your shell/profile settings causing unexpected output during the SSH connection – as the message says, have you had a good look through your .cshrc, .basrc, .profile etc to see if there’s anything recently changed or unusual there? Anything doing an echo or print would be a likely suspect I think.

    If that doesn’t do it hopefully someone else will see this and have an alternative suggestion…

    Cheers,

    Don

    1. Thanks Doc for you reply.

      This issue has been fixed. I have made changes into .bashrc to comment out stty statements and it worked.

      Thanks,
      Ojasvi

  6. i have a master Jenkins on windows 2008 R2 and now i am trying to setup slave Jenkins on windows 2008 R2. When i open jenkins website on slave server and go to manage nodes page, i get following error message “Ping response time is too long or timed out.”

    I added an entry to host file and also routing table entry for master jenkins IP but issue is still there.

    Please help me.

    Thanks in advance.

    Thanks..
    -Anuj

    1. Hi Anuj,

      This looks like a current Windows issue: https://issues.jenkins-ci.org/browse/JENKINS-22932

      You could maybe try changing your version of Jenkins and see if that helps? Or you may be able get more detail on the problem by trying to initiate the connection on the slave host manually, doing something like this:

      java -jar slave.jar -jnlpUrl http://YOURJENKINSMASTER:8080/computer/YOURSLAVEDEFINITION/slave-agent.jnlp

      (after downloading the slave.jar from the master) and see if that helps at all or at least gives you a more informative error message?

      Would it be possible to disable your firewall(s) temporarily to rule out issues there?

      Please update if you find a solution, hope that helps!

      Cheers,

      Don

  7. Hello Sir,

    I have getting an error in Jenkins slave as “java.io.IOException: An existing connection was forcibly closed by the remote host” …..Could you please suggest the possible causes and the solution to fix it???

    1. Here is the error message i got :

      INFO: Ping failed. Terminating the channel.
      java.net.SocketException: Software caused connection abort: socket write error
      at java.net.SocketOutputStream.socketWrite0(Native Method)
      at java.net.SocketOutputStream.socketWrite(Unknown Source)
      at java.net.SocketOutputStream.write(Unknown Source)
      at java.io.BufferedOutputStream.flushBuffer(Unknown Source)
      at java.io.BufferedOutputStream.flush(Unknown Source)
      at java.io.ObjectOutputStream$BlockDataOutputStream.flush(Unknown Source
      )
      at java.io.ObjectOutputStream.flush(Unknown Source)
      at hudson.remoting.ClassicCommandTransport.write(ClassicCommandTransport
      .java:53)
      at hudson.remoting.Channel.send(Channel.java:528)
      at hudson.remoting.Request.callAsync(Request.java:208)
      at hudson.remoting.Channel.callAsync(Channel.java:749)
      at hudson.remoting.PingThread.ping(PingThread.java:99)
      at hudson.remoting.PingThread.run(PingThread.java:81)

      Sep 11, 2015 12:20:43 PM hudson.slaves.ChannelPinger$1 onDead
      SEVERE: Failed to terminate the channel:
      java.net.SocketException: Software caused connection abort: socket write error
      at java.net.SocketOutputStream.socketWrite0(Native Method)
      at java.net.SocketOutputStream.socketWrite(Unknown Source)
      at java.net.SocketOutputStream.write(Unknown Source)
      at java.io.BufferedOutputStream.flushBuffer(Unknown Source)
      at java.io.BufferedOutputStream.flush(Unknown Source)
      at java.io.ObjectOutputStream$BlockDataOutputStream.flush(Unknown Source
      )
      at java.io.ObjectOutputStream.flush(Unknown Source)
      at hudson.remoting.ClassicCommandTransport.write(ClassicCommandTransport
      .java:53)
      at hudson.remoting.Channel.send(Channel.java:528)
      at hudson.remoting.Channel.close(Channel.java:1006)
      at hudson.slaves.ChannelPinger$1.onDead(ChannelPinger.java:110)
      at hudson.remoting.PingThread.run(PingThread.java:91)

  8. Hello sir,

    i have an existing Jenkins on a server and i needed to move it to different server by adding the slaves from old to new server.

    Suppose i have Jenkins in 192.168.100.1 and 192.168.100.2 is my new server.

    I wanted to move content of old server to new server.

    10.0.1.1,10.0.1.2,10.0.1.3 are the slaves of old server wanted to move to new server.

    Can u please explain me in points.

  9. I have created 2 slave nodes and am attempting to execute the build on both. I have the following in my project.

    Ristrict where this project can be run
    Label = (“XpressionGPDEV279”) || (“XpressionGPDEV280”) if I use the OR condition it works
    Slaves in label:2

    Label = (“XpressionGPDEV279”) && (“XpressionGPDEV280”) If I use the AND condition it give the following warning “Theres no slave/cloud that matches this assignment”

      1. Don,

        No I am not using any plugins to manage. Using the Build Executor to create (slave) connections to the Unix (AIX) system. Creating a build package that uses the “Restrict when this project can be run” which allow you to enter a “Label Expression” to the slave(s). In this case I want to do an AND (&&) to include 2 slaves. It works if I use the OR (||) condition.

        Thanks,
        Tom

  10. Hi,

    Thank you for your blog!

    I have a question: I have my master Jenkins on a Linux machine (it’s a VM) and I want to add slaves on Windows machines (also VMs).

    How do I proceed?

    Thanks!!

    1. Hi,

      Thanks Rebecca – VM or physical shouldn’t make any difference: as long as your hosts can connect to each other ok you should be able to set them up.

      Connect to your master Jenkins via a browser then go to “Manage Jenkins” then “Manage Nodes”, click “New Node”, then give it a name (e.g. ‘WindowsSlave’). That should get you to a page where you then specify the settings for your slave host – one of the main options is “Launch method”, where you can select whatever connection method suits your needs best – normally, use ssh for Linux hosts, Windows Service for Windows, or Java Web Start for pretty much anything. Selecting any of these Launch Methods should show you what’s needed for each case, and clicking the blue “?” should give you a more detailed explanation of each approach.

      Hope that helps, thanks for visiting!

      Don

  11. Hi Donald,

    I am searching for a groovy script which can launch the slave(windows).

    Do you know any other way to achieve so.

    Thanks,
    Shreedhar

Leave a Reply