- Tighten the patterns used for nightly tags detection.
- Compare hashes instead of names to figure out if we are on top of a
branch (fixes the issue of no nightly tags reachable from the
release branches). Jenkins is doing `git fetch origin "${branch}";
git checkout FETCH_HEAD` and this was confusing the `git rev-parse
--abbrev-ref HEAD` code (it returned `HEAD` instead of `${branch}`).
- Account for possible multiple tags in a single commit.
- Made the tagging fail in dubious situations.
- Reindent the code, modernize a bit.
We push a commit with the nightly SDK tag to the main branch if the
SDK was built from the main branch. Which is what happens when we
build the nightly intermediate SDK. The final nightly SDK is not built
from the main branch, but rather from the nightly intermediate SDK
tag. Both of them point to the exactly same commit, but the difference
is in what `git rev-parse --abbrev-ref HEAD` returns for each of
those. When the main branch is checked out, the command will return
"main". When the nightly intermediate SDK tag is checked out, the
command will return "HEAD". So when nightly final SDK is being built,
the command returns a string different than "main" and thus decides
not to push the commit with the final nightly SDK tag to the main
branch. Rework it to assume that if `git rev-parse HEAD` and `git
rev-parse origin/main` return the same commit hash (and it's the
nightly build and all that) then the commit should be pushed.
We use "origin/main" instead of just "main" just in case the main
branch was not checked out before, for some reason (may come up in
testing with different names for the main branch when testing).
The two-phase nightly builds create an intermediate tag first, which
didn't match the main nightly tag regexp before. Because of that, the
commit was not pushed to the main branch. The following final SDK
build had a version that matched the regexp, but the last commit (with
the intermediate tag on it) wasn't in main, and thus was also not
pushed.
For embargoed releases it is useful to apply patches locally to build
with them before they are public. This allows to push the same patches
to the repo during the Flatcar release at the embargo lift. The result
is the same (as long as the scripts patches did not change parts of the
setup logic that was running before they got applied), we can just build
earlier and thus do the Flatcar release directly on the embargo lift
instead of having to wait with the build because it would require the
patches to be in the repos.
FLATCAR_VERSION and FLATCAR_SDK_VERSION are defined in the version
file, so it should be sourced before trying to use those. Here we try
to do it in a limited scope.
Also, SDK container link should use the dockerized version in a
directory name.
Currently we skip the nightly build if there are no changes. This
didn't work well because a new run doesn't fix any failure because the
rerun became a no-op.
Check if the main artifacts we expect from a step are found, as simple
heuristic on whether a rerun is needed.
It uses the SIGNER environment variable to decide whether the
signatures should be created or not. It expect the key of the SIGNER
to exist in GPGHOME, and that's what gpg_setup.sh is already doing.
In some places we need to recursively change the owner of the
directory that contains artifacts to be signed, otherwise we won't be
able to create new files with signatures there. This is because some
of the artifacts are either created inside the SDK container (so the
created files belong to root outside the container) or are created
with `sudo`.
The functions are sourcing other files that define global variables,
so they will spill into the callers shell unnecessarily. We will also
add some functionality that uses traps in follow-up commits, so it's
good to limit the scope of traps too.
When a nightly build is started that pushes the version file to the
branch it was doing so only at the end of the build, causing the push
to fail if something else got merged in between.
Push the version file early by generating it the same way it would be
generated by the run_sdk_container/bootstrap_sdk_container scripts.
In the case of the SDK the version file gets the same version for the
OS and the SDK. Add some explanations about the version formats. Note
that the scripts will still rewrite the file but it should be a no-op.
The coreos/portage refs were allowed to be empty strings but the way
the function was run from Groovy the lack of quoting caused the empty
strings to be missing parameters.
Since the two parameters are meant to be optional, support omitting
them.
The pipeline created two tags if an SDK was built, one for the SDK and
one for the OS build (which was a free-standing tag or a local state
that was equivalent to the existing tag of the same name). The
nightlies created update commits on the main branch, even if no change
was done, and on the release branches we lacked these commits.
Create the release tag in the nightly SDK bootstrap already and reuse
it for the nightly OS build. Instead of local state, checkout the
existing tags explicitly. Extend the nightly update commit logic to
cover release branches and detect if we can skip building because no
changes were done.
This change has sdk_bootstrap update the origin branch when run from the
main branch, updating the SDK and OS version in 'main' for each SDK
bootstrap build.
Release / maintenance branches have the SDK version set in the
versionfile at release time. But main is never updated.
Updating the versionfile in main when a new SDK is built ensures that
dev branches based on main will also use the correct SDK version (e.g.
in subsequent CI builds).
ci-automation builds on the SDK container and simplifies CI automation
build tasks (SDK bootstrap, SDK container, packages, image, VMs).
See ci-automation/README.md for a brief introduction.
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
