Commit Graph

102 Commits

Author SHA1 Message Date
Krzesimir Nowak
cec96aeec5 ci-automation/vendor-testing/azure.sh: Small fixes
Fix some comments, quote some variables.
2022-06-02 18:40:06 +02:00
Kai Lueke
6c0fb8959d ci-automation/vendor-testing/azure.sh: Align timeout with GC duration
The kola test run time shouldn't be longer than the GC duration to
prevent failing tests caused by GC interference.
Align the Azure kola timeout with the GC duration.
2022-06-02 14:08:28 +09:00
Sayan Chowdhury
42608d3c67
ci-automation/azure: Add initial container tests infra for Azure (#274)
The Azure tests use a similar logic as the GCE tests where an the
instance type parameter normally used in AWS/Equinix Metal tests is
here used to specify whether the VM gets started in Gen V1 or V2 mode.

Signed-off-by: Sayan Chowdhury <schowdhury@microsoft.com>
Co-authored-by: Kai Lüke <pothos@users.noreply.github.com>
2022-05-27 08:01:59 +02:00
Kai Lueke
cee8a6aadf ci-automation: Push version file early
When a nightly build is started that pushes the version file to the
branch it was doing so only at the end of the build, causing the push
to fail if something else got merged in between.
Push the version file early by generating it the same way it would be
generated by the run_sdk_container/bootstrap_sdk_container scripts.
In the case of the SDK the version file gets the same version for the
OS and the SDK. Add some explanations about the version formats. Note
that the scripts will still rewrite the file but it should be a no-op.
2022-05-23 22:40:02 +09:00
Kai Lueke
95367851fa ci-automation/sdk_bootstrap.sh: Allow omitting the optional parameters
The coreos/portage refs were allowed to be empty strings but the way
the function was run from Groovy the lack of quoting caused the empty
strings to be missing parameters.
Since the two parameters are meant to be optional, support omitting
them.
2022-05-23 19:29:39 +09:00
Krzesimir Nowak
8c3d7b977b ci-automation: Fix potential use of unbound variable error
`local -a stuff` does not make `stuff` a bound array variable, so
checking length of the array will trigger an error about unbound
variable. Fortunately, `local stuff=()` does the trick.
2022-05-11 12:43:08 +02:00
Krzesimir Nowak
bf1bc21498 ci-automation: Do not rerun tests on unrelated instances
We forgot to clear the array with instance tests to rerun, so the list
grew from one iteration to another when going over all the instance
types. I did not spot it before, because I tested it with only one
extra instance.
2022-05-11 12:15:52 +02:00
Kai Lüke
42fd4919c4
Merge pull request #331 from flatcar-linux/kai/equinix-metal-refactor
ci-automation/vendor-testing/equinix_metal.sh: Use test framework
2022-05-11 19:12:32 +09:00
Kai Lueke
3fd7825310 ci-automation/vendor-testing/gce.sh: Test GVNIC and break retest cycle
The logic we had in some tests for covering different instance types
now got more easy to reuse for testing the GVNIC mode in GCE.
Align the GCE test with AWS and DigitalOcean to test an additional
"instance type" (here just changing the NIC) and break the retest spin
case it gets called for arm64.
2022-05-11 12:07:58 +09:00
Kai Lueke
9fe14ffe34 ci-automation/vendor-testing/equinix_metal.sh: Use test framework
The test framework from the AWS PR allows us to align the logic which
also addresses some bugs we had here.
Port the Equinix Metal test over to the new framework (and also use
different test basenames per architecture while at it which could
otherwise result in clashes).
2022-05-11 11:39:30 +09:00
Krzesimir Nowak
d60d514482 ci-automation: Make AWS test script to work 2022-05-10 12:46:33 +02:00
Krzesimir Nowak
6278762fa8 ci-automation: Add helper functions for running tests on multiple instances 2022-05-10 12:46:33 +02:00
Krzesimir Nowak
e1d9beaeee ci-automation: Fix passing multiple test names to vendor scripts 2022-05-10 12:46:02 +02:00
Krzesimir Nowak
f0765e22c3 ci-automation: Let vendor scripts know if this is their first run
I will need it to correctly handle test reruns as we will need to
handle passed test names differently on first runs than on reruns.
2022-05-10 12:46:02 +02:00
Dongsu Park
76abe0d9cb ci-automation: Add WIP AWS test script for CI automation 2022-05-10 12:45:43 +02:00
Krzesimir Nowak
a8ac124d53 ci-automation: Add new vendor test for VMware 2022-05-06 12:57:20 +02:00
Krzesimir Nowak
3b3cffabc8 ci-automation: Fix credentials handling in digital ocean 2022-05-06 09:16:23 +02:00
Krzesimir Nowak
3c119f14b2 ci-automation: Fix secret file handling
It can't be done in a subshell, because the file will be gone after
subshell quits.
2022-05-06 09:16:23 +02:00
Krzesimir Nowak
413689c779 ci-automation: Rename some variables and make them overridable 2022-05-06 09:16:23 +02:00
Krzesimir Nowak
197e9a334f ci-automation: Add secrets handling 2022-05-06 09:16:23 +02:00
Krzesimir Nowak
cd2f3f0d6c ci-automation: Drop boilerplate code from digital ocean test 2022-05-06 09:16:23 +02:00
Krzesimir Nowak
2fe896b558 ci-automation: Add retest cycle breaking functionality 2022-05-06 09:16:23 +02:00
Krzesimir Nowak
9650650e4b ci-automation: Add URL template handling 2022-05-06 09:16:23 +02:00
Krzesimir Nowak
b6bb07acdc ci-automation: Initial test script for Digital Ocean 2022-05-06 09:16:23 +02:00
Krzesimir Nowak
d26f2b3b39 ci-automation: Use vendor_test.sh in equinix_metal and gce tests 2022-05-05 13:07:52 +02:00
Krzesimir Nowak
1d6f38a72e ci-automation: Reduce boilerplate in vendor tests
Move the common setup to the vendor_test.sh script, which will be
sourced by the vendor scripts.
2022-05-05 12:57:14 +02:00
Kai Lueke
f7edd4e061 ci-automation: add GCE image test
The GCE image test runs on a single instance type for now. In the
future it would be good to test the new NIC type with the cl.internet
test.
2022-05-05 16:52:42 +09:00
Mathieu Tortuyaux
550e702f90 ci-container/test: add equinix-metal test script
Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
Co-authored-by: Kai Lüke <pothos@users.noreply.github.com>
2022-05-04 16:34:37 +09:00
Kai Lüke
28ee2a3256
Merge pull request #298 from flatcar-linux/kai/test-lts
CI: Support comparing to current LTS and updating from it
2022-04-29 16:34:47 +09:00
Mathieu Tortuyaux
4bd316ac74
Merge pull request #272 from flatcar-linux/tormath1/pxe
ci-automation/vm: build PXE if Equinix Metal is built
2022-04-28 11:52:52 +02:00
Kai Lueke
9a98cc2917 ci-automation/vms: handle platform names and generate the image formats
The kola test scripts are named by the platforms. The image naming is
also quite difficult to know and remember, e.g., whether "ami" or
"ami_vmdk" is needed for AWS tests and whether it's "vmware" or
"vmware_ova".

To address these problems the vms build stage now accepts the platform
names as format input, and for each platform it will automatically
generate the needed image types to run the tests.
2022-04-28 17:15:02 +09:00
Kai Lueke
c4af762e26 ci-automation/garbage_collect: clean up kola cloud resources
The garbage collect job should also clean up kola resources if a test
job failed to do so due to forced terminator or misbehavior. The
cleanup is done by "ore" which needs credentials like kola.

Run ore from the mantle container image. Unfortunately Docker does not
support Podman's --env-host option and the env vars had to be passed
explicitly. While --env-file=<(env) would work it contains a lot of
variables that cause the container to behave a bit weird.
2022-04-28 16:27:14 +09:00
Kai Lueke
856929d357 CI: Support comparing to current LTS and updating from it
When the restriction that the CI can't access the LTS release is gone
we can support to run the image comparison and the kola update test.
2022-04-26 15:00:31 +09:00
Krzesimir Nowak
1916936e34 ci-automation: Update test.sh script docs
We are not using SDK container for running the tests any more - it was
replaced with mantle container. Update the docs accordingly.
2022-04-20 16:34:07 +02:00
Mathieu Tortuyaux
19ca42b3dd
ci-automation/vm: build PXE if Equinix Metal is built
Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
2022-04-20 16:33:00 +02:00
Kai Lueke
98e9947a06 ci-automation: silence rsync
The rsync copy logs made it hard to navigate the job output.
Remove the --progress and -v flags.
2022-04-20 19:13:02 +09:00
Kai Lueke
da0380c7e8 Run CI container pipeline kola tests with the new mantle image
The SDK container does not exist for arm64 and is quite heavy. We
currently also resort to a unconditional rebuilding of mantle inside
the SDK.
Use the new mantle container image to run kola tests, and pin its
version through a text file that gets updated by GitHub Actions.
2022-04-20 19:13:02 +09:00
Mathieu Tortuyaux
de7e05403b
ci-automation/vms: rename equinix_metal to packet
this is required to keep "packet" in the SDK linguo while the user can
use "equinix_metal" term.

Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
Co-authored-by: Krzesimir Nowak <knowak@microsoft.com>
2022-04-13 13:09:51 +02:00
Kai Lüke
7376494ef2
Merge pull request #266 from flatcar-linux/kai/sdk-from-release-tag
ci-automation: use a single git tag and skip nightlies with no changes
2022-04-04 17:12:36 +02:00
Kai Lueke
bd970357c8 ci-automation: use a single git tag and skip nightlies with no changes
The pipeline created two tags if an SDK was built, one for the SDK and
one for the OS build (which was a free-standing tag or a local state
that was equivalent to the existing tag of the same name). The
nightlies created update commits on the main branch, even if no change
was done, and on the release branches we lacked these commits.

Create the release tag in the nightly SDK bootstrap already and reuse
it for the nightly OS build. Instead of local state, checkout the
existing tags explicitly. Extend the nightly update commit logic to
cover release branches and detect if we can skip building because no
changes were done.
2022-04-01 17:18:51 +02:00
Thilo Fromm
6dcfd9aeb6 ci-automation/test.sh: remove PARALLEL_TEST passing (move to CI)
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-04-01 13:59:47 +02:00
Thilo Fromm
1e0dc777fe ci-automation/test.sh: export PARALLEL_TESTS
Export PARALLEL_TESTS in the container's .env file to ensure it is
passed to the vendor script.
2022-03-23 12:11:12 +01:00
Thilo Fromm
9b83d3e80b
Merge pull request #258 from flatcar-linux/t-lo/ci-automation-tests-use-latest-kola
ci-automation/test.sh: use the latest kola from coreos-overlay
2022-03-16 17:04:16 +01:00
Kai Lueke
c149d24ced run_sdk_container: Fall back to tar ball download for SDK image
The nightly SDK image is not pushed to a registry but has to be
downloaded from the build server as tar ball.
Fall back to the tar ball import for a better user experience.
To reuse the ci logic it had to support the "docker" env variable.
The use of the pigz container is not always needed if the user has
pigz available.
2022-03-16 15:31:03 +01:00
Thilo Fromm
6286b0a442 ci-automation/test.sh: use the latest kola from coreos-overlay 2022-03-16 14:14:46 +01:00
Thilo Fromm
a9700e16fb ci-automation/tapfile_helper_lib.sh: remove non-printable ASCII
Jenkins TAP file parser does not process non-printable ASCII characters
but bails out. This change removes all ASCII < 0x1F, so non-printable
characters are not included in the TAP report.

Fixes
    Caused by: unacceptable character '' (0x1B) special characters are not allowed
2022-03-16 09:37:48 +01:00
Thilo Fromm
53c90388c0 ci-automation/vendor-testing/qemu_update.sh: fix unbound
One-line fix to resolve
    ci-automation/vendor-testing/qemu_update.sh: line 64: testscript: unbound variable
error.

Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-03-15 17:39:28 +01:00
Kai Lüke
d3aa4f1331
Merge pull request #247 from flatcar-linux/kai/set-official
ci-automation: set images as official based on version
2022-03-10 12:41:10 +01:00
Thilo Fromm
b6caa4163d
Merge pull request #246 from flatcar-linux/t-lo/ci-automation-tests-pass-parallel-env-to-container
automation/test.sh: pass PARALLEL_TESTS to container
2022-03-10 12:24:55 +01:00
Kai Lüke
fff05f00c5
Merge pull request #245 from flatcar-linux/kai/sdk-tests
ci-automation: print kola command line
2022-03-10 12:06:07 +01:00