4596 Commits

Author SHA1 Message Date
Thilo Fromm
72ff9691ef ci-automation/test.sh: stage torcx manifest
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-22 18:36:31 +01:00
Thilo Fromm
7f8d125505 ci-automation/README.md: add docs for qemu test 2022-02-22 18:36:31 +01:00
Thilo Fromm
96ec2fb4a2 ci-automation/tapfile: enforce foreign keys, simplify 2022-02-22 18:36:31 +01:00
Thilo Fromm
1b529d287a ci-automation: publish torcx json and use in tests
This change updates the package build script to publish the torcx
manifest file to the build cache so it can be used by tests.
It also updates the generic test script to use the SDK container instead
of the packages container image, and to download and use the torcx
manifest from the build cache.

Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-22 18:36:31 +01:00
Thilo Fromm
310eff9aa3 ci-automation/test.sh: fix reruns, set retry to 20
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-22 18:36:31 +01:00
Thilo Fromm
438e6aca11 ci-automation: git author and curl verboseness
- Git author configuration moves to tagging function and put under a
  condition so as to not pollute peoples' workspaces.
- curl now less verbose since it was spamming logs with TLS debug
  information.

Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-22 18:36:31 +01:00
Thilo Fromm
3532916e7c ci-automation testing: address PR review comments
- add cleanup script to test.sh
- remove wrapper function from qemu test

Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-22 18:36:31 +01:00
Thilo Fromm
d54322a449 ci-automation/tapfile_helper_lib.sh: add @pothos' retcode fix
Co-authored-by: Kai Lüke <pothos@users.noreply.github.com>
2022-02-22 18:36:31 +01:00
Thilo Fromm
08191b99a8 Apply @pothos' suggestions from code review
Co-authored-by: Kai Lüke <pothos@users.noreply.github.com>
2022-02-22 18:36:31 +01:00
Thilo Fromm
e5991bb6a6 ci-automation: first stab at adding testing
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-02-22 18:36:31 +01:00
Kai Lueke
63cb4a1e11 Merge pull request #226 from flatcar-linux/scripts
jenkins/images: print out image diff report
2022-02-16 12:12:01 +01:00
Kai Lueke
41436cabc3 jenkins/images: print out image diff report
Often a change results in unexpected effects on the image, e.g., when
a wrong package version gets chosen or the package installs files under
/etc, or binaries of library dependencies get pulled in. Besides
inspecting the image manually, the package-diff tool also gives
valuable insights.
Run the package-diff tool in a comparison to the last release and print
the image URL alongside for convenience.
2022-02-16 12:12:01 +01:00
flatcar-ci
55913189eb
New version: beta-3066.1.2 beta-3066.1.2 2022-02-03 22:46:57 +05:30
Mathieu Tortuyaux
0847b6696e
jenkins/kola/packet: use metro instead of facility
Follow up of: https://github.com/flatcar-linux/mantle/pull/281

Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
2022-01-26 16:31:12 +01:00
flatcar-ci
b6225ded25
New version: beta-3066.1.1 beta-3066.1.1 2022-01-25 00:15:59 +05:30
flatcar-ci
ff1efade8b
New version: beta-3066.1.1 2022-01-24 23:04:49 +05:30
Thilo Fromm
99f6a6c5ae Merge pull request #215 from flatcar-linux/scripts
SDK container: enable binpkg cache for nightly builds
2022-01-12 16:12:28 +01:00
Thilo Fromm
04f126e8e2 SDK container: enable binpkg cache for nightly builds
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-01-12 16:12:28 +01:00
Thilo Fromm
197cde5ab5 Merge pull request #213 from flatcar-linux/scripts
settings / ci-automation: remove &quot;binpkg&quot; prefix
2022-01-11 11:27:10 +01:00
flatcar-ci
ea2e8b1a45 settings / ci-automation: remove "binpkg" prefix
The original intention of the "binpkg" prefix in the CI binary package
cache URL was to separate packages from other build artifacts like
containers, images, and SDK tarballs. Motivation was to separate
developer content (binary packages) from CI automation artifacts
(everything else); since binary packages are not used by the CI.

This broke assumptions in scripts which use the binary host URL for
other things than packages - e.g. SDK tarballs or images. These
scripts would get a bincache URL with "binpkg/" prepended, while CI
automation would *not* use that prefix.

This change removes the use of "binpkg/" altogether since it would not
work as intended without more significant changes to build scripts.
2022-01-11 11:27:10 +01:00
Thilo Fromm
4453d733ff Merge pull request #211 from flatcar-linux/scripts
checkout: omit checking for tags in submodules
2022-01-10 15:47:37 +01:00
flatcar-ci
19341b958c checkout: omit checking for tags in submodules
Signed-off-by: flatcar-ci <infra+ci@flatcar-linux.org>
2022-01-10 15:47:37 +01:00
Thilo Fromm
c715d3cc4e Merge pull request #210 from flatcar-linux/scripts
ci-automation/garbage_collect: fix binpkg url
2022-01-10 14:44:53 +01:00
Thilo Fromm
4de4f20d7e ci-automation/garbage_collect: fix binpkg url
garbage_collect.sh was using 'docker_vernum' where it should have been
using 'vernum' (as push_pkgs.sh does).

Also, make sure release directories are removed, not just packages.

Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-01-10 14:44:53 +01:00
Thilo Fromm
90ae5cb6ef Merge pull request #209 from flatcar-linux/scripts
ci-automation + setup_board: publish and use binpkgs
2022-01-10 12:12:21 +01:00
Thilo Fromm
f7a5b585fd ci-automation: Apply suggestions from PR review
Co-authored-by: Krzesimir Nowak <knowak@microsoft.com>
2022-01-10 12:12:21 +01:00
Thilo Fromm
265cf0f341 ci-automation/README.md: pkg publish, bin cache added
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-01-10 12:12:21 +01:00
flatcar-ci
bd25c8f23f ci-automation + setup_board: publish and use binpkgs
This change adds a job for publishing binary packages to the build cache
server to the ci automation.

Also, setup_board is updated to use the buildcache package cache if a
nightly build version is detected.

Signed-off-by: flatcar-ci <infra+ci@flatcar-linux.org>
2022-01-10 12:12:21 +01:00
Thilo Fromm
8a26bfec8f Merge pull request #202 from flatcar-linux/scripts
checkout: helper script for checking out branches and tags
2022-01-07 16:10:44 +01:00
Thilo Fromm
012001ec3c checkout: fix typo in update strategy detection
Co-authored-by: Kai Lüke <pothos@users.noreply.github.com>
2022-01-07 16:10:44 +01:00
Thilo Fromm
d60a003702 checkout: helper script for checking out branches and tags
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-01-07 16:10:44 +01:00
Thilo Fromm
0d361563bf Merge pull request #205 from flatcar-linux/scripts
fix sdk_entry.sh setup_board fallout
2022-01-07 13:27:36 +01:00
Thilo Fromm
b9e9b8f050 sdk_lib/sdk_entry.sh: only call setup_board if /build/... exists
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-01-07 13:27:36 +01:00
Thilo Fromm
6aa5174bec sdk_lib/Dockerfile.sdk-import: use root user by default
sdk_entry.sh is expected to be called by the root user, so we set USER
root:root. Also we add a "root" entry to passwd and group since it does
not exist in the SDK tarball.

Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-01-07 13:27:36 +01:00
Thilo Fromm
f3f45d6b11 Merge pull request #207 from flatcar-linux/scripts
sdk_container: better docker / podman detection
2022-01-07 12:20:38 +01:00
Thilo Fromm
0ace755cb8 sdk_container: better docker / podman detection
In bce3bd9031fa78c79173f29bcac5bc49c81a03b4, we added support for podman
for building and running the SDK container. The presence of podman is
auto-detected in sdk_container_common.sh. However, podman is preverred
over docker, requiring users to use *sudo* (which podman requires and
docker does not).

This change uses docker when present, podman otherwise. It also improves
podman detection - 'podman' uses argv[0] in its version string, so if
'docker' is a symlink to 'podman', 'podman --version' output uses
'docker'. This broke the SDK container on hosts which have a 'docker'
symlink to 'podman' since 'podman' is then run w/o 'sudo'.

Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-01-07 12:20:38 +01:00
Kai Lueke
026a5274a8 Merge pull request #206 from flatcar-linux/scripts
build_sdk_container_image: force removal of running container
2022-01-07 11:51:39 +01:00
Kai Lueke
b0e098a464 build_sdk_container_image: force removal of running container
At least with Podman it's not possible to call "container rm" on a
running container without the force flag.
Add the force flag which is also used elsewhere already.
2022-01-07 11:51:39 +01:00
Kai Lueke
e7dfd931eb Merge pull request #199 from flatcar-linux/scripts
run/build_sdk_container: support Podman
2022-01-07 11:51:24 +01:00
Kai Lueke
a0ddca9e2f run/build_sdk_container: support Podman
When the docker wrapper script for Podman is used, we need to
explicitly create a root user container with "sudo podman".
Podman also has its own bridge for root user containers which we need
to detect, and it requires to explicitly say to use the Docker Hub
Caddy image.
Add a "$docker" variable that uses sudo podman as needed, and also
check which bridge interface to use. The filter had to be changed
because it didn't work with Podman. Use the Docker Hub Caddy image
explicitly.
2022-01-07 11:51:24 +01:00
Kai Lueke
c8b1909e8e Merge pull request #203 from flatcar-linux/scripts
sdk_lib/sdk_entry: handle permission error for target version file
2022-01-06 19:01:49 +01:00
Kai Lueke
f8d5685389 sdk_lib/sdk_entry: handle permission error for target version file
The creation of the target version file failed:
/home/sdk/sdk_entry.sh: line 32: /build/amd64-usr/etc/target-version.txt: Permission denied
Use root permissions to create the file.
2022-01-06 19:01:49 +01:00
Thilo Fromm
a501865e3b Merge pull request #201 from flatcar-linux/scripts
sdk_entry.sh: ensure currect binpkg host
2022-01-06 17:13:57 +01:00
Thilo Fromm
56e5fc8a29 sdk_entry.sh: ensure currect binpkg host
This change ensures the binpkg host is updated if the board (OS) version
differs from the SDK version.

This is to ensure /build/[arch] uses the correct binary package cache.

Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-01-06 17:13:57 +01:00
flatcar-ci
291bc51a2b Merge pull request #200 from flatcar-linux/scripts
sdk_lib/sdk_entry.sh: use a login shell to source /etc/profile
2022-01-05 18:15:09 +01:00
flatcar-ci
867c162ad1 sdk_lib/sdk_entry.sh: use a login shell to source /etc/profile
For execution of the compiled binaries in /build/arm64-usr we rely on
qemu-user binfmt emulation and have to tell it where the root is with
QEMU_LD_PREFIX because build systems don't chroot into /build/arm64-usr
themselves (which also works just by chance on amd64 because we have
similar glibc versions and so on). The env var setup was done in
/etc/profile.d/qemu-aarch64.sh but is now not read anymore since the
container runs the shell not as login shell.

Add the login options to the bash and su calls when starting the
container.
2022-01-05 18:15:09 +01:00
flatcar-ci
f28cecc3d8 Merge pull request #198 from flatcar-linux/scripts
ci-automation: allow to optionally push and sign the commit
2022-01-05 18:14:31 +01:00
Kai Lueke
f91119a9c7 ci-automation: allow to optionally push and sign the commit
For test builds the commit that updates the submodules can be free-
standing but for releases we need to push it to the branch and also
sign the tag.
Add optional arguments that are used by the tag-release script in
flatcar-build-scripts.
2022-01-05 18:14:31 +01:00
Thilo Fromm
f0feddcbc4 Merge pull request #196 from flatcar-linux/scripts
run_sdk_container: use exact match for existing containers
2022-01-05 15:00:20 +01:00
Thilo Fromm
54f2e647be run_sdk_container: use exact match for existing containers
run_sdk_container uses the sourcetree version to decide whether to
re-use existing containers or create new ones. However, containers were
not matched by exact name - instead, plain --filter name="..." was used,
leading to prefix matching. This change updates name="..." to use
regular expressions for exact matching.

Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2022-01-05 15:00:20 +01:00