CoreOS Jenkins build system
Requirements
Storage Requirements
CoreOS manifests are managed using git and repo, so a git server must be available to host these. An additional git repo is used by the os-manifest job to store a temporary manifest commit that is passed through to downstream jobs.
The jobs also require google storage buckets for storing build artifacts such as binary packages and CoreOS images.
Jenkins Requirements
The jobs use a number of Jenkins plugins during jobs. These are:
- Git
- Rebuilder
- Parameterized Trigger
- Copy Artifact
- SSH Agent
- Job Restrictions
- Credentials Binding
- TAP - for collecting test results from kola
- Matrix Project
Slave Requirements
The Jenkins jobs assume that each build slave is running CoreOS. The scripts that execute as part of the jobs use a number of tools present in CoreOS. Different host operating systems are untested.
All jobs assume that the Jenkins user on the slaves have sudo access, which is used by the cork tool.
The Jenkins slave used to execute the os-kola-qemu job must be run on a host that has KVM, so this slave cannot be in a VM unless it is using nested KVM, which is untested.
Most jobs use the slave label coreos to execute on a CoreOS system, so at least one slave (or the master) is required with this label. The os-kola-qemu job requires a slave with the coreos and kvm label.
Secret Requirements
Some secrets are required for the various things done during the build:
- Slave SSH keys (if Jenkins slaves are used)
- git ssh key for manifest-build pushes from
os-manifest - google storage api key for artifact uploads and downloads
Setup
Running Jenkins
If you have no Jenkins instance, you can run one in a Docker container on CoreOS:
docker run -p 8080:8080 -p 50000:50000 jenkins
To create a slave, you will need to run CoreOS and either give Jenkins Oracle.com credentials to automatically install the JDK, or copy a JDK tarball to the CoreOS system and set $JAVA_HOME in the slave settings.
Install plugins and jobs
Jenkins jobs in XML format are available in the jobs directory. A script called install.sh is provided to copy jobs and install Jenkins plugins. The script will restart Jenkins as required by some plugins.
For example, if the Jenkins instance is at http://127.0.0.1:8080:
./install.sh http://127.0.0.1:8080
Configuring CoreOS jobs
Some jobs will require modification to work correctly in any setup outside CoreOS.
os-manifestwill need the git url for themanifest-buildsrepo. You will also need to configure the correct SSH secret for git pushes.- Any job using google storage will need
GOOGLE_APPLICATION_CREDENTIALSconfigured, which points to a JSON Service Account key. Additionally, these jobs will need to point to your specific google storage buckets. - Any job signing artifacts will need
GPG_SECRET_KEY_FILE, which is a GPG private key used to sign built artifacts.