4618 Commits

Author SHA1 Message Date
Kai Lueke
f694f17c75 Merge pull request #244 from flatcar-linux/scripts
Fix and improve channel handling
2022-03-04 14:27:17 +01:00
Kai Lueke
206db24a04 ci-automation: set the channel from the git tag
For now we had only "developer" images in the new pipeline.
Based on the git tag like "alpha-1234.0.0" set the channel (group) for
the image and also use this logic when finding the channel in the QEMU
update test.
2022-03-04 14:27:17 +01:00
Kai Lueke
639c131b83 jenkins/images: find channel from base channel variable
Using the tags on the branch is not enough to find the channel we want
to be the dev build be related to.
Use the base channel variable which was introduced for this.
2022-03-04 14:27:17 +01:00
Thilo Fromm
c7614ff30d Merge pull request #242 from flatcar-linux/scripts
ci-automation/tapfile_helper_lib.sh: read test output from file
2022-03-03 09:29:55 +01:00
Thilo Fromm
a58233a27a ci-automation/tapfile_helper_lib.sh: use subshell, break lines after 200
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-03-03 09:29:55 +01:00
Thilo Fromm
6af6fdb351 ci-automation/tapfile_helper_lib.sh: read test output from file
This change updates the tapfile helper to read test output from a file
instead of passig it inline in the SQL INSERT statement.
This fixes an issue with large error output from tests, which breaks
tap_ingest_tapfile():
    ci-automation/tapfile_helper_lib.sh: line 31: /usr/bin/sqlite3: Argument list too long
Error observed with the cl.toolbox.dnf-install test, which can generate
8000 lines of output. Fix tested with the same output.

Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-03-03 09:29:55 +01:00
Kai Lueke
ee311e5814 Merge pull request #235 from flatcar-linux/scripts
ci-automation: add update test
2022-03-02 11:30:14 +01:00
Kai Lueke
cf309c8a4a ci-automation: add missing update test
The kola update test was missing. It is performed as update from the
old image to the newly built payload to ensure that the new image is
compatible for old clients.
2022-03-02 11:30:14 +01:00
Thilo Fromm
fa1fbe6510 ci-automation/vendor-testing: add qemu_uefi
This change adds the qemu_uefi.sh vendor test. It reuses most of the
implementation in qemu.sh (qemu_uefi.sh is a soft-link to qemu.sh).

This also enables qemu testing for ARM64.

Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-25 14:37:00 +01:00
Thilo Fromm
351af55ccc Merge pull request #238 from flatcar-linux/scripts
ci-automation: fix test_update_reruns typo
2022-02-24 12:38:51 +01:00
Thilo Fromm
2641a8b68b ci-automation: fix test_update_reruns typo
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-24 12:38:51 +01:00
Thilo Fromm
7fb2142af0 Merge pull request #234 from flatcar-linux/scripts
ci-automation/test*: per-image summary report, honour pre-set PARALLEL_TESTS
2022-02-24 11:13:07 +01:00
Thilo Fromm
e32d23aadb ci-automation/test*: per-image summary, honor parallel
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-24 11:13:07 +01:00
Thilo Fromm
9c19a1df43 Merge pull request #232 from flatcar-linux/scripts
ci-automation: publish test results, add to GC
2022-02-23 10:25:22 +01:00
Thilo Fromm
24f1a1e468 ci-automation/test.sh: publish to _kola_tmp, not debug/_kola_tmp
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-23 10:25:22 +01:00
Thilo Fromm
cd14a1bac1 Update ci-automation/test.sh: fix typo.
Co-authored-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2022-02-23 10:25:22 +01:00
Thilo Fromm
1ec3e57319 ci-automation: publish test results, add to GC
This change adds copying test results to the build cache server, and
adds respective deletion to the garbage collector.

Also, the patch fixes an issue with torcx publishing (manifest
publishing had arch hard-coded).

Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-23 10:25:22 +01:00
Thilo Fromm
25b233ba08 Merge pull request #230 from flatcar-linux/scripts
update_sdk_container_image: New SDK container from existing image (minor / patch updates)
2022-02-22 18:37:53 +01:00
Thilo Fromm
1ceb2f768f update_sdk_container_image: update SDK container image
This change introduces update_sdk_container_image, a script to generate
a new SDK container image based on an existing SDK container. The
script is meant to be used for minor / patch level SDK changes (like
test suite updates).

Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-22 18:37:53 +01:00
Thilo Fromm
c4dceee5a0 Merge pull request #228 from flatcar-linux/scripts
ci-automation: Add foundational testing primitives
2022-02-22 18:36:31 +01:00
Thilo Fromm
03297fba41 ci-automtion/packages.sh: fix torcx URL, add manifest
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-22 18:36:31 +01:00
Thilo Fromm
5e76288261 ci-automation/tests.sh: use http in torcx manifest
Use HTTP instead of https because Ignition does not recognise
letsencrypt certificates, leading to test breakage in
docker.torcx-manifest-pkgs.

Add a note in settings.env to explicitly call out HTTP requirement of
build cache server.

Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-22 18:36:31 +01:00
Thilo Fromm
72ff9691ef ci-automation/test.sh: stage torcx manifest
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-22 18:36:31 +01:00
Thilo Fromm
7f8d125505 ci-automation/README.md: add docs for qemu test 2022-02-22 18:36:31 +01:00
Thilo Fromm
96ec2fb4a2 ci-automation/tapfile: enforce foreign keys, simplify 2022-02-22 18:36:31 +01:00
Thilo Fromm
1b529d287a ci-automation: publish torcx json and use in tests
This change updates the package build script to publish the torcx
manifest file to the build cache so it can be used by tests.
It also updates the generic test script to use the SDK container instead
of the packages container image, and to download and use the torcx
manifest from the build cache.

Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-22 18:36:31 +01:00
Thilo Fromm
310eff9aa3 ci-automation/test.sh: fix reruns, set retry to 20
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-22 18:36:31 +01:00
Thilo Fromm
438e6aca11 ci-automation: git author and curl verboseness
- Git author configuration moves to tagging function and put under a
  condition so as to not pollute peoples' workspaces.
- curl now less verbose since it was spamming logs with TLS debug
  information.

Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-22 18:36:31 +01:00
Thilo Fromm
3532916e7c ci-automation testing: address PR review comments
- add cleanup script to test.sh
- remove wrapper function from qemu test

Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-22 18:36:31 +01:00
Thilo Fromm
d54322a449 ci-automation/tapfile_helper_lib.sh: add @pothos' retcode fix
Co-authored-by: Kai Lüke <pothos@users.noreply.github.com>
2022-02-22 18:36:31 +01:00
Thilo Fromm
08191b99a8 Apply @pothos' suggestions from code review
Co-authored-by: Kai Lüke <pothos@users.noreply.github.com>
2022-02-22 18:36:31 +01:00
Thilo Fromm
e5991bb6a6 ci-automation: first stab at adding testing
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-22 18:36:31 +01:00
Kai Lueke
63cb4a1e11 Merge pull request #226 from flatcar-linux/scripts
jenkins/images: print out image diff report
2022-02-16 12:12:01 +01:00
Kai Lueke
41436cabc3 jenkins/images: print out image diff report
Often a change results in unexpected effects on the image, e.g., when
a wrong package version gets chosen or the package installs files under
/etc, or binaries of library dependencies get pulled in. Besides
inspecting the image manually, the package-diff tool also gives
valuable insights.
Run the package-diff tool in a comparison to the last release and print
the image URL alongside for convenience.
2022-02-16 12:12:01 +01:00
flatcar-ci
55913189eb
New version: beta-3066.1.2 beta-3066.1.2 2022-02-03 22:46:57 +05:30
Mathieu Tortuyaux
0847b6696e
jenkins/kola/packet: use metro instead of facility
Follow up of: https://github.com/flatcar-linux/mantle/pull/281

Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
2022-01-26 16:31:12 +01:00
flatcar-ci
b6225ded25
New version: beta-3066.1.1 beta-3066.1.1 2022-01-25 00:15:59 +05:30
flatcar-ci
ff1efade8b
New version: beta-3066.1.1 2022-01-24 23:04:49 +05:30
Thilo Fromm
99f6a6c5ae Merge pull request #215 from flatcar-linux/scripts
SDK container: enable binpkg cache for nightly builds
2022-01-12 16:12:28 +01:00
Thilo Fromm
04f126e8e2 SDK container: enable binpkg cache for nightly builds
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-01-12 16:12:28 +01:00
Thilo Fromm
197cde5ab5 Merge pull request #213 from flatcar-linux/scripts
settings / ci-automation: remove &quot;binpkg&quot; prefix
2022-01-11 11:27:10 +01:00
flatcar-ci
ea2e8b1a45 settings / ci-automation: remove "binpkg" prefix
The original intention of the "binpkg" prefix in the CI binary package
cache URL was to separate packages from other build artifacts like
containers, images, and SDK tarballs. Motivation was to separate
developer content (binary packages) from CI automation artifacts
(everything else); since binary packages are not used by the CI.

This broke assumptions in scripts which use the binary host URL for
other things than packages - e.g. SDK tarballs or images. These
scripts would get a bincache URL with "binpkg/" prepended, while CI
automation would *not* use that prefix.

This change removes the use of "binpkg/" altogether since it would not
work as intended without more significant changes to build scripts.
2022-01-11 11:27:10 +01:00
Thilo Fromm
4453d733ff Merge pull request #211 from flatcar-linux/scripts
checkout: omit checking for tags in submodules
2022-01-10 15:47:37 +01:00
flatcar-ci
19341b958c checkout: omit checking for tags in submodules
Signed-off-by: flatcar-ci <infra+ci@flatcar-linux.org>
2022-01-10 15:47:37 +01:00
Thilo Fromm
c715d3cc4e Merge pull request #210 from flatcar-linux/scripts
ci-automation/garbage_collect: fix binpkg url
2022-01-10 14:44:53 +01:00
Thilo Fromm
4de4f20d7e ci-automation/garbage_collect: fix binpkg url
garbage_collect.sh was using 'docker_vernum' where it should have been
using 'vernum' (as push_pkgs.sh does).

Also, make sure release directories are removed, not just packages.

Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-01-10 14:44:53 +01:00
Thilo Fromm
90ae5cb6ef Merge pull request #209 from flatcar-linux/scripts
ci-automation + setup_board: publish and use binpkgs
2022-01-10 12:12:21 +01:00
Thilo Fromm
f7a5b585fd ci-automation: Apply suggestions from PR review
Co-authored-by: Krzesimir Nowak <knowak@microsoft.com>
2022-01-10 12:12:21 +01:00
Thilo Fromm
265cf0f341 ci-automation/README.md: pkg publish, bin cache added
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-01-10 12:12:21 +01:00
flatcar-ci
bd25c8f23f ci-automation + setup_board: publish and use binpkgs
This change adds a job for publishing binary packages to the build cache
server to the ci automation.

Also, setup_board is updated to use the buildcache package cache if a
nightly build version is detected.

Signed-off-by: flatcar-ci <infra+ci@flatcar-linux.org>
2022-01-10 12:12:21 +01:00