Commit Graph

5244 Commits

Author SHA1 Message Date
Kai Lueke
cc9d43b01e run_sdk_container: add remove flag
At the moment one must remove the leftover containers manually.
Add a flag to do so when stopping the container.
2022-03-16 15:55:14 +01:00
Kai Lueke
a0574bcf37 run_sdk_container: recognize --help flag
When the user passes --help we should not start a container and throw a
late error message afterwards but show the help directly.
2022-03-16 15:36:32 +01:00
Kai Lueke
c149d24ced run_sdk_container: Fall back to tar ball download for SDK image
The nightly SDK image is not pushed to a registry but has to be
downloaded from the build server as tar ball.
Fall back to the tar ball import for a better user experience.
To reuse the ci logic it had to support the "docker" env variable.
The use of the pigz container is not always needed if the user has
pigz available.
2022-03-16 15:31:03 +01:00
Thilo Fromm
6286b0a442 ci-automation/test.sh: use the latest kola from coreos-overlay 2022-03-16 14:14:46 +01:00
Thilo Fromm
08b620b5de
Merge pull request #257 from flatcar-linux/t-lo/ci-automation-tap-remove-nonprint-ascii
ci-automation/tapfile_helper_lib.sh: remove non-printable ASCII
2022-03-16 10:59:08 +01:00
Thilo Fromm
a9700e16fb ci-automation/tapfile_helper_lib.sh: remove non-printable ASCII
Jenkins TAP file parser does not process non-printable ASCII characters
but bails out. This change removes all ASCII < 0x1F, so non-printable
characters are not included in the TAP report.

Fixes
    Caused by: unacceptable character '' (0x1B) special characters are not allowed
2022-03-16 09:37:48 +01:00
flatcar-ci
24a74477a8 New version: sdk-3180.0.0-nightly-20220316-0139 2022-03-16 04:46:27 +00:00
Thilo Fromm
aaf17af27d
Merge pull request #256 from flatcar-linux/t-lo/ci-automation-fix-update-test
ci-automation/vendor-testing/qemu_update.sh: fix unbound variable
2022-03-15 17:53:58 +01:00
Thilo Fromm
53c90388c0 ci-automation/vendor-testing/qemu_update.sh: fix unbound
One-line fix to resolve
    ci-automation/vendor-testing/qemu_update.sh: line 64: testscript: unbound variable
error.

Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-03-15 17:39:28 +01:00
flatcar-ci
1ff70fb888 New version: sdk-3179.0.0-nightly-20220315-0754 2022-03-15 11:06:55 +00:00
Jeremi Piotrowski
d28ceb03b7
Merge pull request #254 from flatcar-linux/jepio/azure-private-networking
jenkins/kola/azure: forward azure private networking parameters to kola
2022-03-14 20:33:31 +01:00
Jeremi Piotrowski
b353d8cf6b jenkins/kola/azure: forward azure private networking parameters to kola
These allow the configuration of virtual network for the created
instances to join, and tell kola to use the private instance IP for
connectivity.
2022-03-14 15:35:24 +01:00
flatcar-ci
722e8fc3bf New version: sdk-3178.0.0-nightly-20220314-0139 2022-03-14 04:45:46 +00:00
flatcar-ci
e784d11a67 New version: sdk-3176.0.0-nightly-20220312-0139 2022-03-12 04:53:20 +00:00
flatcar-ci
534d055f61 New version: sdk-3175.0.0-nightly-20220311-0139 2022-03-11 04:48:39 +00:00
Kai Lüke
c3142dd901
Merge pull request #250 from flatcar-linux/kai/test-aws
jenkins/kola/aws: run tests on additional instance types
2022-03-10 18:22:08 +01:00
Kai Lüke
e7e9c7abbf
Merge pull request #252 from flatcar-linux/kai/define-sysext-level
set_lsb_release: define Flatcar sysext level
2022-03-10 18:16:48 +01:00
Kai Lueke
7fafef2084 set_lsb_release: define Flatcar sysext level
Sysext images have a compatibility matching mechanism that searches for
the matching OS version or custom sysext level setting. On Flatcar
there is just the OS version set in /etc/os-release until now which
means that sysext images can't easily be used together with autoupdates
that change the OS version.

Define a sysext level for Flatcar so that users can refer to it instead
of the OS version when they have images that don't rely on a particular
Flatcar version.
Here an example of the now possible metadata:
/etc/extensions/NAME/usr/lib64/extension-release.d/extension-release.NAME
  ID=flatcar
  SYSEXT_LEVEL=1.0
and a symlink /etc/extensions/NAME/usr/lib → /etc/extensions/NAME/usr/lib64
to work around the problem that using lib/ as path destroys Flatcar's
lib → lib64 symlink.
In the future the matching logic hopefully gets more flexible because
now it is just a string comparison. Also, the architecture is not
matched either for now - we should work with upstream to improve this.

Closes: https://github.com/flatcar-linux/Flatcar/issues/643
2022-03-10 18:15:04 +01:00
Kai Lueke
d26530d4cb jenkins/kola/packet|aws: silence debug output of concurrent test
The concurrent tests' debug output is not helping and causes confusion.
2022-03-10 15:42:45 +01:00
Kai Lueke
f2c7bcb78d jenkins/kola/aws: run tests on additional instance types
There was a regression for an instance type which could have been
prevented by testing.
Add the same extended test logic used for Equinix Metal to AWS.
2022-03-10 15:10:02 +01:00
Kai Lüke
5ebe64ba2f
Merge pull request #251 from flatcar-linux/kai/kconfig-diff
jenkins/images: print kernel config changes
2022-03-10 12:44:43 +01:00
Kai Lüke
d3aa4f1331
Merge pull request #247 from flatcar-linux/kai/set-official
ci-automation: set images as official based on version
2022-03-10 12:41:10 +01:00
Kai Lueke
0ce2b0bef8 jenkins/images: print kernel config changes
Sometimes the Linux build system changes and results in unexpected
kernel config results.
Print changes in the kernel config as part of the image diff report.
2022-03-10 12:25:21 +01:00
Thilo Fromm
b6caa4163d
Merge pull request #246 from flatcar-linux/t-lo/ci-automation-tests-pass-parallel-env-to-container
automation/test.sh: pass PARALLEL_TESTS to container
2022-03-10 12:24:55 +01:00
Kai Lüke
fff05f00c5
Merge pull request #245 from flatcar-linux/kai/sdk-tests
ci-automation: print kola command line
2022-03-10 12:06:07 +01:00
Thilo Fromm
194c503b56
Merge pull request #249 from flatcar-linux/t-lo/ci-automation-tapfile-ascii
ci-automation/tapfile_helper_lib.sh: only ASCII chars
2022-03-10 11:34:04 +01:00
Thilo Fromm
aa6a742efa ci-automation/tapfile_helper_lib.sh: only ASCII chars
This change removes all non-ASCII characters from test debug / error
output when ingesting the output for inclusion in the TAP report.
Jenkins TAP parser does not handle some unicode chars, leading to tap
parser errors with e.g. Cilium output (which uses unicode).

Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-03-10 10:20:11 +01:00
Jeremi Piotrowski
957981740a
Merge pull request #248 from flatcar-linux/jepio/fix-tap-ingest
ci-automation/tapfile_helper_lib: fix commiting last transaction
2022-03-10 10:18:46 +01:00
Jeremi Piotrowski
25cf7c4fc5 ci-automation/tapfile_helper_lib: fix commiting last transaction
Move the final db commit to inside the subshell. Since the while loop
runs inside a subshell, the SQL variable outside of the subshell is not
modified, and so the last contents of the SQL variable are dropped. This
shows up when the last couple test cases don't have an error message,
and simply append the transaction to 'SQL'. They are never written to
the db.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2022-03-10 09:14:42 +01:00
Thilo Fromm
d266229434
ci-automation/test.sh: handle unset PARALLEL_TESTS
Co-authored-by: Kai Lüke <pothos@users.noreply.github.com>
2022-03-10 08:24:44 +01:00
Kai Lueke
c3ae1ce3b0 ci-automation: set images as official based on version
The image needs to be set into official mode through a helper script
(see jenkins/images.sh) and the COREOS_OFFICIAL variable needs to be
set for prod_image_util.sh/build_image_util.sh/grub_install.sh.
2022-03-09 18:19:52 +01:00
Thilo Fromm
8ca2393eb8 automation/test.sh: pass PARALLEL_TESTS to container
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-03-09 17:38:15 +01:00
Kai Lueke
79e07bca44 ci-automation: print kola command line
For running kola manually and knowing which parameters where set, it
helps to print the kola command line in the job.
2022-03-09 16:44:06 +01:00
Kai Lüke
0c8d96edaa
Merge pull request #233 from gabriel-samfira/add-configurable-compression
Make image compression format configurable
2022-03-09 16:30:31 +01:00
Gabriel
b7d51af5dc
Update changelog/changes/2022-02-22-configurable-image-compression.md
Co-authored-by: Kai Lüke <pothos@users.noreply.github.com>
2022-03-09 17:26:05 +02:00
Gabriel
99801d24cd
Update jenkins/vms.sh
Co-authored-by: Kai Lüke <pothos@users.noreply.github.com>
2022-03-09 17:19:07 +02:00
flatcar-ci
4e96a2d3b5 New version: sdk-3173.0.0-nightly-20220309-0139 2022-03-09 04:51:02 +00:00
Gabriel Adrian Samfira
e88f12c69c Rename zstd to zst
Rename sztd to zst and amend the changelog. The zstd binary generates a
compressed file with the .zst extension by default.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2022-03-08 14:02:37 +00:00
Gabriel Adrian Samfira
71e4f1c660 Remove commented code 2022-03-08 08:07:47 +00:00
Gabriel Adrian Samfira
f126debb37 Allow specifying multiple compression formats
Add the ability to specify a comma separated list of compression formats.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2022-03-07 12:01:21 +00:00
Gabriel Adrian Samfira
f0a535271c Use gzip compression for OpenStack images
This change makes the Jenkins job output openstack images using gzip
compression format. This allows OpenStack users to directly consume images
by simply specifying the URL to the image. Glance will then download the
image, unarchive it and add it to it's catalogue.

Fixes #575

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2022-03-07 12:01:21 +00:00
Gabriel Adrian Samfira
571de4064b Make image compression format configurable
This change adds a new flag called --image_compression_format which
allows us to output the final VM image as one of the supported formats:

bz2 (default), gz, zip or none

if the compression format is "none" or "", the image will not be compressed.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2022-03-07 12:01:21 +00:00
flatcar-ci
05ae47afd7 New version: sdk-3170.0.0-nightly-20220306-0139 2022-03-06 04:46:43 +00:00
Kai Lüke
0cc95e3b3e
Merge pull request #244 from flatcar-linux/kai/set-group
Fix and improve channel handling
2022-03-04 14:22:10 +01:00
Kai Lueke
db7220eced ci-automation: set the channel from the git tag
For now we had only "developer" images in the new pipeline.
Based on the git tag like "alpha-1234.0.0" set the channel (group) for
the image and also use this logic when finding the channel in the QEMU
update test.
2022-03-04 13:49:18 +01:00
Thilo Fromm
68305d60f0
Merge pull request #243 from flatcar-linux/t-lo/fix-tap-parser-errorr
ci-automation: fix tap parser errors
2022-03-04 13:43:07 +01:00
flatcar-ci
44f1275126 New version: sdk-3168.0.0-nightly-20220304-0139 2022-03-04 04:53:45 +00:00
Kai Lueke
b12c5b8783 jenkins/images: find channel from base channel variable
Using the tags on the branch is not enough to find the channel we want
to be the dev build be related to.
Use the base channel variable which was introduced for this.
2022-03-03 16:31:53 +01:00
Thilo Fromm
eba1cdb4c2 ci-automation/tapfile_helper_lib.sh: fix CI TAP parse errors
This change fixes and adds more string chars escaping in the test error
debug output ("\" are removed and a bug in removing '"' is fixed),
addressing a parser errof the CI encountered when ingesting TAP output.

Furthermore, line numbering is shortened, and test names have a spurious
"-" prefix removed.

Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-03-03 16:29:38 +01:00
flatcar-ci
7755567f31 New version: sdk-3167.0.0-nightly-20220303-0139 2022-03-03 05:00:50 +00:00