diff --git a/ci-automation/README.md b/ci-automation/README.md index 7db3f97654..87beaf09e0 100644 --- a/ci-automation/README.md +++ b/ci-automation/README.md @@ -119,13 +119,14 @@ Testing is implemented in two layers: 2. `ci-automation/vendor-testing/` contains low-level vendor-specific test wrappers around [`kola`](https://github.com/flatcar-linux/mantle/tree/flatcar-master/kola/), our test scenario orchestrator. Testing relies on the SDK container and will use tools / test suites from the SDK. -The low-level vendor / image specific script (layer 2. in the list above) is run inside the SDK. -Testing will use the vendor image published by `vms.sh` from buildcache, and the torcx manifest published by `packages`. +The low-level vendor / image specific script (layer 2. in the list above) runs inside the SDK. +Testing will use the vendor image published by `vms.sh` from buildcache, and the torcx manifest published by `packages.sh`. Additionally, a script library is provided (at `ci-automation/tapfile_helper_lib.sh`) to help handling `.tap` test result files produced by test runs. Library functions may be used to merge the result of multiple test runs (e.g. for multiple image types / vendors) into a single test result report. The test runs are considered successful only if all tests succeeded for all vendors / images at least once. + **Usage** ``` ./checkout @@ -140,3 +141,20 @@ source ci-automation/test.sh test_run amd64 qemu ``` +### QEmu test + +`ci-automation/vendor-testing/qemu.sh` implements a `kola` wrapper for testing the `qemu` image. +The wrapper is a straightforward call to `kola` and does not have any additional requirements. + +**NOTE** that the generic image (`flatcar_production_image.bin`) is used for the test instead of the QEmu vendor image. + +**NOTE on host firewalling** The test automation uses bridged networking and will handle forwarding and NAT. +However, we experienced test failures from lack of internet access with several firewall implementations. +It is recommended to stop firewalling on the host the tests are run on (for example, use `systemctl stop firewalld` if the host used `firewalld`). + +**Settings** + +* `QEMU_IMAGE_NAME` - file name of the QEmu image to fetch from bincache. +* `QEMU_PARALLEL` - Number of parallel test cases to run. + Note that test cases may involve launching mutliple QEmu VMs (network testing etc.). + Tests are memory bound, not CPU bound; e.g. `20` is a sensible value for a 6 core / 12 threads systwem w/ 32 GB RAM.