Adrián Deccico

How to organize and avoid concurrency problems in big projects, splitted in multiple jobs and chained to each other in Hudson?

If you have a lengthy process, it will be a good practice to split it in multiple tasks, having each calling the next one, like in the (somewhat simplified) image on the left.

Problem number one: How to avoid Clean and Build (for instance) running at the same time?

As (almost) always, “you have a plugin for that” in Hudson. This time is Locks and Latches, which will let you create a lock, shared by all the jobs you want. Only the job that has the lock, will be executed.

So far, so good, but, where is the problem?

As you may guess, Clean could be called multiple times, while Build, Publish, or the same Clean are being executed. How could this happen? If you trigger Clean with a SCM (like SVN) this could be an expected situation, as you have no way to prioritize the jobs in Hudson. This is explained in bug 833.

The following Groovy script, checks if there is any other job waiting for an executor or being executed, preventing Clean to destroy the others jobs data. (Note: you will need the Groovy plugin and then create a “system Groovy script”.)

//Jobs currently building or in the queue
jobs = hudson.model.Hudson.instance.items.findAll{job -> job.isBuilding()} + hudson.model.Hudson.instance.items.findAll{job -> job.isInQueue()}

//Keep only the names from the collected job objects
jobs = jobs.collect{x -> x.name}

//get current job name
currentJob = Thread.currentThread().executable.toString().split()[0]

//cut current job from the jobs list
jobs -= currentJob

//decide if there is any running (or waiting) job
res = true
if (jobs.size()  > 0)
{
    res = false
    println "The following jobs are being executed or waiting for an executor"
    jobs.each{println it}
}
else
{
    println "Concurrency OK"
}

return res

In case you have another jobs in your Hudson instance, that have nothing to do with our “Clean -> Build -> Publish” project, you can restrict to our project, the concurrency checking. You just have to do a slight modification to our Groovy script:

//Jobs currently building or in the queue
jobs = hudson.model.Hudson.instance.items.findAll{job -> job.isBuilding()} + hudson.model.Hudson.instance.items.findAll{job -> job.isInQueue()}

//Keep only the names from the collected job objects
jobs = jobs.collect{x -> x.name}

//list of jobs that belong to the same project
project_jobs = ["Compile", "Publish"]

//cut current job from the jobs list
jobs = project_jobs.intersect(jobs)

//decide if there is any running (or waiting) job
res = true
if (jobs.size()  > 0)
{
    res = false
    println "The following jobs are being executed or waiting for an executor"
    jobs.each{println it}
}
else
{
    println "Concurrency OK"
}

return res