4612 Commits

Author SHA1 Message Date
Thilo Fromm
16632a681b ci-automation/tapfile_helper_lib.sh: fix CI TAP parse errors
This change fixes and adds more string chars escaping in the test error
debug output ("\" are removed and a bug in removing '"' is fixed),
addressing a parser errof the CI encountered when ingesting TAP output.

Furthermore, line numbering is shortened, and test names have a spurious
"-" prefix removed.

Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-03-08 16:48:52 +01:00
Kai Lueke
435d53e216 Merge pull request #244 from flatcar-linux/scripts
Fix and improve channel handling
2022-03-04 14:27:27 +01:00
Kai Lueke
672722d73c ci-automation: set the channel from the git tag
For now we had only "developer" images in the new pipeline.
Based on the git tag like "alpha-1234.0.0" set the channel (group) for
the image and also use this logic when finding the channel in the QEMU
update test.
2022-03-04 14:27:27 +01:00
Kai Lueke
250be82cd8 jenkins/images: find channel from base channel variable
Using the tags on the branch is not enough to find the channel we want
to be the dev build be related to.
Use the base channel variable which was introduced for this.
2022-03-04 14:27:27 +01:00
Thilo Fromm
9eefa2682f Merge pull request #242 from flatcar-linux/scripts
ci-automation/tapfile_helper_lib.sh: read test output from file
2022-03-03 09:30:03 +01:00
Thilo Fromm
d9fe3333d1 ci-automation/tapfile_helper_lib.sh: use subshell, break lines after 200
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-03-03 09:30:03 +01:00
Thilo Fromm
9c8e39b6fe ci-automation/tapfile_helper_lib.sh: read test output from file
This change updates the tapfile helper to read test output from a file
instead of passig it inline in the SQL INSERT statement.
This fixes an issue with large error output from tests, which breaks
tap_ingest_tapfile():
    ci-automation/tapfile_helper_lib.sh: line 31: /usr/bin/sqlite3: Argument list too long
Error observed with the cl.toolbox.dnf-install test, which can generate
8000 lines of output. Fix tested with the same output.

Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-03-03 09:30:03 +01:00
Dongsu Park
9d49c657d7 New version: stable-3033.2.3 stable-3033.2.3 2022-03-02 18:15:36 +01:00
Kai Lueke
a104e1041e Merge pull request #235 from flatcar-linux/scripts
ci-automation: add update test
2022-03-02 11:30:22 +01:00
Kai Lueke
0adf376148 ci-automation: add missing update test
The kola update test was missing. It is performed as update from the
old image to the newly built payload to ensure that the new image is
compatible for old clients.
2022-03-02 11:30:22 +01:00
Dongsu Park
148421a5ea New version: stable-3033.2.3 2022-03-01 17:53:24 +01:00
Thilo Fromm
44456e1042 ci-automation/vendor-testing: add qemu_uefi
This change adds the qemu_uefi.sh vendor test. It reuses most of the
implementation in qemu.sh (qemu_uefi.sh is a soft-link to qemu.sh).

This also enables qemu testing for ARM64.

Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-25 14:36:52 +01:00
Thilo Fromm
cd0c4703d6 Merge pull request #238 from flatcar-linux/scripts
ci-automation: fix test_update_reruns typo
2022-02-24 12:38:58 +01:00
Thilo Fromm
a5885d1d3e ci-automation: fix test_update_reruns typo
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-24 12:38:58 +01:00
Thilo Fromm
d38d43ad76 Merge pull request #234 from flatcar-linux/scripts
ci-automation/test*: per-image summary report, honour pre-set PARALLEL_TESTS
2022-02-24 11:12:58 +01:00
Thilo Fromm
c6c077a14e ci-automation/test*: per-image summary, honor parallel
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-24 11:12:58 +01:00
Thilo Fromm
fd13368989 Merge pull request #232 from flatcar-linux/scripts
ci-automation: publish test results, add to GC
2022-02-23 10:25:13 +01:00
Thilo Fromm
92dd6b086c ci-automation/test.sh: publish to _kola_tmp, not debug/_kola_tmp
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-23 10:25:13 +01:00
Thilo Fromm
2e9f5c79e1 Update ci-automation/test.sh: fix typo.
Co-authored-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2022-02-23 10:25:13 +01:00
Thilo Fromm
c0a9b616ef ci-automation: publish test results, add to GC
This change adds copying test results to the build cache server, and
adds respective deletion to the garbage collector.

Also, the patch fixes an issue with torcx publishing (manifest
publishing had arch hard-coded).

Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-23 10:25:13 +01:00
Thilo Fromm
9f883f9313 Merge pull request #230 from flatcar-linux/scripts
update_sdk_container_image: New SDK container from existing image (minor / patch updates)
2022-02-22 18:37:40 +01:00
Thilo Fromm
368bacafc3 update_sdk_container_image: update SDK container image
This change introduces update_sdk_container_image, a script to generate
a new SDK container image based on an existing SDK container. The
script is meant to be used for minor / patch level SDK changes (like
test suite updates).

Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-22 18:37:40 +01:00
Thilo Fromm
da5ed87a21 Merge pull request #228 from flatcar-linux/scripts
ci-automation: Add foundational testing primitives
2022-02-22 18:36:22 +01:00
Thilo Fromm
70af6a9a19 ci-automtion/packages.sh: fix torcx URL, add manifest
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-22 18:36:21 +01:00
Thilo Fromm
65e39c2868 ci-automation/tests.sh: use http in torcx manifest
Use HTTP instead of https because Ignition does not recognise
letsencrypt certificates, leading to test breakage in
docker.torcx-manifest-pkgs.

Add a note in settings.env to explicitly call out HTTP requirement of
build cache server.

Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-22 18:36:21 +01:00
Thilo Fromm
bd000b68b1 ci-automation/test.sh: stage torcx manifest
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-22 18:36:21 +01:00
Thilo Fromm
c264fd4b51 ci-automation/README.md: add docs for qemu test 2022-02-22 18:36:21 +01:00
Thilo Fromm
bbc7a3c542 ci-automation/tapfile: enforce foreign keys, simplify 2022-02-22 18:36:21 +01:00
Thilo Fromm
b7b224eb43 ci-automation: publish torcx json and use in tests
This change updates the package build script to publish the torcx
manifest file to the build cache so it can be used by tests.
It also updates the generic test script to use the SDK container instead
of the packages container image, and to download and use the torcx
manifest from the build cache.

Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-22 18:36:21 +01:00
Thilo Fromm
c4afa20435 ci-automation/test.sh: fix reruns, set retry to 20
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-22 18:36:21 +01:00
Thilo Fromm
b66f42abfd ci-automation: git author and curl verboseness
- Git author configuration moves to tagging function and put under a
  condition so as to not pollute peoples' workspaces.
- curl now less verbose since it was spamming logs with TLS debug
  information.

Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-22 18:36:21 +01:00
Thilo Fromm
1fa12b8d0c ci-automation testing: address PR review comments
- add cleanup script to test.sh
- remove wrapper function from qemu test

Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-22 18:36:21 +01:00
Thilo Fromm
5ce1413e12 ci-automation/tapfile_helper_lib.sh: add @pothos' retcode fix
Co-authored-by: Kai Lüke <pothos@users.noreply.github.com>
2022-02-22 18:36:21 +01:00
Thilo Fromm
dd735792af Apply @pothos' suggestions from code review
Co-authored-by: Kai Lüke <pothos@users.noreply.github.com>
2022-02-22 18:36:21 +01:00
Thilo Fromm
457d8972da ci-automation: first stab at adding testing
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-22 18:36:21 +01:00
Kai Lueke
9c8579b109 Merge pull request #226 from flatcar-linux/scripts
jenkins/images: print out image diff report
2022-02-16 12:11:24 +01:00
Kai Lueke
b45f0db9c1 jenkins/images: print out image diff report
Often a change results in unexpected effects on the image, e.g., when
a wrong package version gets chosen or the package installs files under
/etc, or binaries of library dependencies get pulled in. Besides
inspecting the image manually, the package-diff tool also gives
valuable insights.
Run the package-diff tool in a comparison to the last release and print
the image URL alongside for convenience.
2022-02-16 12:11:24 +01:00
flatcar-ci
f389cd8de1
New version: stable-3033.2.2 stable-3033.2.2 2022-02-03 22:54:41 +05:30
Mathieu Tortuyaux
84f1838b26
jenkins/kola/packet: use metro instead of facility
Follow up of: https://github.com/flatcar-linux/mantle/pull/281

Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
2022-01-26 16:30:03 +01:00
flatcar-ci
e8a77b8bda
New version: stable-3033.2.1 stable-3033.2.1 2022-01-25 00:21:40 +05:30
flatcar-ci
abaf3c1adb
New version: stable-3033.2.1 2022-01-24 19:48:18 +05:30
Thilo Fromm
64611a67ee Merge pull request #215 from flatcar-linux/scripts
SDK container: enable binpkg cache for nightly builds
2022-01-12 16:12:42 +01:00
Thilo Fromm
0035711cd4 SDK container: enable binpkg cache for nightly builds
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-01-12 16:12:42 +01:00
Thilo Fromm
8bdcabd389 Merge pull request #213 from flatcar-linux/scripts
settings / ci-automation: remove &quot;binpkg&quot; prefix
2022-01-11 11:26:58 +01:00
flatcar-ci
024b655dbc settings / ci-automation: remove "binpkg" prefix
The original intention of the "binpkg" prefix in the CI binary package
cache URL was to separate packages from other build artifacts like
containers, images, and SDK tarballs. Motivation was to separate
developer content (binary packages) from CI automation artifacts
(everything else); since binary packages are not used by the CI.

This broke assumptions in scripts which use the binary host URL for
other things than packages - e.g. SDK tarballs or images. These
scripts would get a bincache URL with "binpkg/" prepended, while CI
automation would *not* use that prefix.

This change removes the use of "binpkg/" altogether since it would not
work as intended without more significant changes to build scripts.
2022-01-11 11:26:58 +01:00
Thilo Fromm
a1196d0895 Merge pull request #211 from flatcar-linux/scripts
checkout: omit checking for tags in submodules
2022-01-10 15:47:28 +01:00
flatcar-ci
a360206686 checkout: omit checking for tags in submodules
Signed-off-by: flatcar-ci <infra+ci@flatcar-linux.org>
2022-01-10 15:47:28 +01:00
Thilo Fromm
f906d6cb0a Merge pull request #210 from flatcar-linux/scripts
ci-automation/garbage_collect: fix binpkg url
2022-01-10 14:45:04 +01:00
Thilo Fromm
701c25a230 ci-automation/garbage_collect: fix binpkg url
garbage_collect.sh was using 'docker_vernum' where it should have been
using 'vernum' (as push_pkgs.sh does).

Also, make sure release directories are removed, not just packages.

Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-01-10 14:45:04 +01:00
flatcar-ci
0504098af2 ci-automation + setup_board: publish and use binpkgs
It is a back-port of the 3 commits in
https://github.com/flatcar-linux/scripts/pull/209
to flatcar-3033. The backport was required because of a conflict
in set_version.

This change adds a job for publishing binary packages to the build cache
server to the ci automation.

Also, setup_board is updated to use the buildcache package cache if a
nightly build version is detected.

Lastly, ci-automation/README.md is updated to reflect the changes. THe
readme now also elaborates on the binary package cache server (build
infrastructure).

Co-authored-by: Krzesimir Nowak <knowak@microsoft.com>
2022-01-10 12:18:14 +01:00