Commit Graph

43 Commits

Author SHA1 Message Date
Kai Lueke
db7220eced ci-automation: set the channel from the git tag
For now we had only "developer" images in the new pipeline.
Based on the git tag like "alpha-1234.0.0" set the channel (group) for
the image and also use this logic when finding the channel in the QEMU
update test.
2022-03-04 13:49:18 +01:00
Thilo Fromm
2b2bfad5e1 ci-automation/tapfile_helper_lib.sh: use subshell, break lines after 200
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-03-02 15:35:57 +01:00
Thilo Fromm
22af2876e6 ci-automation/tapfile_helper_lib.sh: read test output from file
This change updates the tapfile helper to read test output from a file
instead of passig it inline in the SQL INSERT statement.
This fixes an issue with large error output from tests, which breaks
tap_ingest_tapfile():
    ci-automation/tapfile_helper_lib.sh: line 31: /usr/bin/sqlite3: Argument list too long
Error observed with the cl.toolbox.dnf-install test, which can generate
8000 lines of output. Fix tested with the same output.

Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-03-02 13:02:52 +01:00
Kai Lüke
ef9914e06e
Merge pull request #235 from flatcar-linux/kai/add-update-test
ci-automation: add update test
2022-03-02 11:27:43 +01:00
Thilo Fromm
b72586c4de
Merge pull request #236 from flatcar-linux/t-lo/tool-for-fetching-build-stage-images
ci-automation/util/fetch_image.sh: fetch CI build stage image
2022-03-02 10:10:17 +01:00
Thilo Fromm
637e5d52ec
Apply suggestions of my favourite proofreader
Krzesimir continues to save me from embarrassing spelling mistakes 💙

Co-authored-by: Krzesimir Nowak <knowak@microsoft.com>
2022-03-02 10:09:59 +01:00
Kai Lueke
a9c3a31efb ci-automation: add missing update test
The kola update test was missing. It is performed as update from the
old image to the newly built payload to ensure that the new image is
compatible for old clients.
2022-02-28 15:59:17 +01:00
Thilo Fromm
88a4df98b1
Merge pull request #239 from flatcar-linux/t-lo/ci-automation-add-qemu-uefi-test
ci-automation/vendor-testing: add qemu_uefi
2022-02-25 12:15:01 +01:00
Thilo Fromm
1b4022d237
Merge pull request #237 from flatcar-linux/t-lo/container-builds-update-version-in-main-branch
ci-automation: SDK build updates version.txt in main branch
2022-02-25 12:14:24 +01:00
Thilo Fromm
9afab3aac4 ci-automation/vendor-testing: add qemu_uefi
This change adds the qemu_uefi.sh vendor test. It reuses most of the
implementation in qemu.sh (qemu_uefi.sh is a soft-link to qemu.sh).

This also enables qemu testing for ARM64.

Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-24 13:54:58 +01:00
Thilo Fromm
308a2a2ab6 ci-automation/sdk_bootstrap: check submodules for nightly branch push
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-24 12:06:28 +01:00
Thilo Fromm
82da911c27 ci-automation/sdk_bootstrap: Only push to main in nightlies
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-24 11:02:34 +01:00
Thilo Fromm
39b65765b4 ci-automation: fix test_update_reruns typo
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-24 10:11:38 +01:00
Thilo Fromm
7cadae957a ci-automation: SDK build updates version.txt in main branch
This change has sdk_bootstrap update the origin branch when run from the
main branch, updating the SDK and OS version in 'main' for each SDK
bootstrap build.

Release / maintenance branches have the SDK version set in the
versionfile at release time. But main is never updated.

Updating the versionfile in main when a new SDK is built ensures that
dev branches based on main will also use the correct SDK version (e.g.
in subsequent CI builds).
2022-02-23 20:41:57 +01:00
Thilo Fromm
248ffcef03 ci-automation/util/fetch_image.sh: fetch CI build stage image
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-23 19:25:12 +01:00
Thilo Fromm
e92521c166 ci-automation/test*: per-image summary, honor parallel
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-23 11:30:54 +01:00
Thilo Fromm
3253435d6c ci-automation/test.sh: publish to _kola_tmp, not debug/_kola_tmp
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-22 18:27:36 +01:00
Thilo Fromm
3397167b5e
Update ci-automation/test.sh: fix typo.
Co-authored-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2022-02-22 18:13:15 +01:00
Thilo Fromm
8157bf0302 ci-automation: publish test results, add to GC
This change adds copying test results to the build cache server, and
adds respective deletion to the garbage collector.

Also, the patch fixes an issue with torcx publishing (manifest
publishing had arch hard-coded).

Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-22 16:22:57 +01:00
Thilo Fromm
081df6cd2c ci-automtion/packages.sh: fix torcx URL, add manifest
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-22 15:44:04 +01:00
Thilo Fromm
4f39e0112f ci-automation/tests.sh: use http in torcx manifest
Use HTTP instead of https because Ignition does not recognise
letsencrypt certificates, leading to test breakage in
docker.torcx-manifest-pkgs.

Add a note in settings.env to explicitly call out HTTP requirement of
build cache server.

Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-21 17:23:53 +01:00
Thilo Fromm
0fa985b872 ci-automation/test.sh: stage torcx manifest
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-21 16:30:30 +01:00
Thilo Fromm
1045fd5ac8 ci-automation/README.md: add docs for qemu test 2022-02-21 13:57:11 +01:00
Thilo Fromm
bee5ac7f74 ci-automation/tapfile: enforce foreign keys, simplify 2022-02-21 12:56:45 +01:00
Thilo Fromm
cafa385164 ci-automation: publish torcx json and use in tests
This change updates the package build script to publish the torcx
manifest file to the build cache so it can be used by tests.
It also updates the generic test script to use the SDK container instead
of the packages container image, and to download and use the torcx
manifest from the build cache.

Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-18 15:52:47 +01:00
Thilo Fromm
a5b958fd07 ci-automation/test.sh: fix reruns, set retry to 20
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-18 14:40:18 +01:00
Thilo Fromm
95ef0b7322 ci-automation: git author and curl verboseness
- Git author configuration moves to tagging function and put under a
  condition so as to not pollute peoples' workspaces.
- curl now less verbose since it was spamming logs with TLS debug
  information.

Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-17 12:31:07 +01:00
Thilo Fromm
3a416fbf32 ci-automation testing: address PR review comments
- add cleanup script to test.sh
- remove wrapper function from qemu test

Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-17 12:30:36 +01:00
Thilo Fromm
6c76bfa1cd
ci-automation/tapfile_helper_lib.sh: add @pothos' retcode fix
Co-authored-by: Kai Lüke <pothos@users.noreply.github.com>
2022-02-17 10:45:10 +01:00
Thilo Fromm
5bfe2f395c
Apply @pothos' suggestions from code review
Co-authored-by: Kai Lüke <pothos@users.noreply.github.com>
2022-02-17 10:29:05 +01:00
Thilo Fromm
f6f44e2ca8 ci-automation: first stab at adding testing
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-16 19:59:45 +01:00
flatcar-ci
0bbae51a5e settings / ci-automation: remove "binpkg" prefix
The original intention of the "binpkg" prefix in the CI binary package
cache URL was to separate packages from other build artifacts like
containers, images, and SDK tarballs. Motivation was to separate
developer content (binary packages) from CI automation artifacts
(everything else); since binary packages are not used by the CI.

This broke assumptions in scripts which use the binary host URL for
other things than packages - e.g. SDK tarballs or images. These
scripts would get a bincache URL with "binpkg/" prepended, while CI
automation would *not* use that prefix.

This change removes the use of "binpkg/" altogether since it would not
work as intended without more significant changes to build scripts.
2022-01-11 09:56:21 +01:00
Thilo Fromm
e076931c7d ci-automation/garbage_collect: fix binpkg url
garbage_collect.sh was using 'docker_vernum' where it should have been
using 'vernum' (as push_pkgs.sh does).

Also, make sure release directories are removed, not just packages.

Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-01-10 14:07:33 +01:00
Thilo Fromm
a6ddcda88e ci-automation: Apply suggestions from PR review
Co-authored-by: Krzesimir Nowak <knowak@microsoft.com>
2022-01-10 11:41:03 +01:00
Thilo Fromm
0ecd0be77a ci-automation/README.md: pkg publish, bin cache added
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-01-07 17:47:50 +01:00
flatcar-ci
7d76cfedf7 ci-automation + setup_board: publish and use binpkgs
This change adds a job for publishing binary packages to the build cache
server to the ci automation.

Also, setup_board is updated to use the buildcache package cache if a
nightly build version is detected.

Signed-off-by: flatcar-ci <infra+ci@flatcar-linux.org>
2022-01-07 17:16:44 +01:00
Kai Lueke
e85a16fe1c ci-automation: allow to optionally push and sign the commit
For test builds the commit that updates the submodules can be free-
standing but for releases we need to push it to the branch and also
sign the tag.
Add optional arguments that are used by the tag-release script in
flatcar-build-scripts.
2022-01-05 15:25:31 +01:00
flatcar-ci
b780fd4b19 ci-automation/garbage_collect: remove spurious line number 2021-11-26 17:54:43 +01:00
Thilo Fromm
1d1c6048d4 sdk-container: add @krnowak's suggestions from code review
Co-authored-by: Krzesimir Nowak <knowak@microsoft.com>
2021-11-26 17:54:43 +01:00
Thilo Fromm
90af105e9f ci/garbage_collect: pick version, fix images + vms
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2021-11-26 17:54:43 +01:00
Thilo Fromm
5ed28497ba ci-autmation: support main- alongside alpha-, beta-...
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2021-11-26 17:54:43 +01:00
Thilo Fromm
337e0d7c3c SDK container: spelling + cp optimisation from code review
Add suggestions by @pothos from code review
- use `cp --reflink=auto`
- spelling error fixes

Co-authored-by: Kai Lüke <pothos@users.noreply.github.com>
2021-11-26 17:54:43 +01:00
Thilo Fromm
9fba5789f9 ci-automation: simplified CI automation unsing containers
ci-automation builds on the SDK container and simplifies CI automation
build tasks (SDK bootstrap, SDK container, packages, image, VMs).

See ci-automation/README.md for a brief introduction.

Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2021-11-26 17:54:43 +01:00