4432 Commits

Author SHA1 Message Date
Sayan Chowdhury
66e56bd8b3
Merge pull request #337 from flatcar-linux/sayan/no-upload-to-private-iii
jenkins: don't upload to private LTS/Pro bucket any longer
lts-2605.31.1 lts-2605.30.1 lts-2605.29.1 lts-2605.28.1
2022-05-30 19:36:34 +05:30
Sayan Chowdhury
a11f7ff3a7
jenkins: don't upload to private LTS/Pro bucket any longer
With the LTS/Pro features public, we can use the public bucket to upload
the artifacts instead of the private one

Signed-off-by: Sayan Chowdhury <schowdhury@microsoft.com>
2022-05-30 15:41:29 +05:30
Kai Lueke
e3bb9751fd jenkins/kola/qemu_uefi.sh: create missing folders
This ports a similar change from "main" to create the directory for
amd64 if it doesn't exist because mantle is not installed in the SDK.
2022-04-28 00:20:01 +09:00
Kai Lüke
3031b60759
Merge pull request #299 from flatcar-linux/kai/kola-2605-test
jenkins/kola/qemu.sh: create missing folders
2022-04-26 16:30:32 +09:00
Kai Lueke
28596bb422 jenkins/kola/qemu.sh: create missing folders
This ports a similar change from "main" to create the directory for
amd64 if it doesn't exist because mantle is not installed in the SDK.
2022-04-26 15:25:59 +09:00
Kai Lueke
ea2f8fc6cd Merge pull request #280 from flatcar-linux/scripts
Use gangue compiled from Jenkins
2022-04-21 17:19:32 +09:00
Kai Lueke
65793be44c Use gangue compiled from Jenkins
These scripts happened to use the copy of gangue in the SDK which isn't
expected because it should use the binaries complied by Jenkins.
2022-04-21 17:19:32 +09:00
Mathieu Tortuyaux
389c030bf0
kola/gce: shrink hostname to be lower than 63 char
GCP Pro is failing because hostname is > 63 char:
```
Apr  5 19:52:27.522820 kubelet[1762]: E0405 19:52:27.522513    1762 kubelet_node_status.go:93] "Unable to register node with API server" err="Node \"jenkins-gce-pro-5-91a967ef5450cb932bc5.c.flatcar-212911.internal\" is invalid: metadata.labels: Invalid value: \"jenkins-gce-pro-5-91a967ef5450cb932bc5.c.flatcar-212911.internal\": must be no more than 63 characters" node="jenkins-gce-pro-5-91a967ef5450cb932bc5.c.flatcar-212911.internal"
```

Let's remove `jenkins` and `gce` from the hostname, these
information are not critical for debugging purposes.

Hostname should now looks like
"basic-5-91a967ef5450cb932bc5.c.flatcar-212911.internal" or
"pro-5-91a967ef5450cb932bc5.c.flatcar-212911.internal"

Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
2022-04-06 16:48:08 +02:00
Jeremi Piotrowski
829fba0d0b Merge pull request #254 from flatcar-linux/scripts
jenkins/kola/azure: forward azure private networking parameters to kola
lts-2605.27.1
2022-03-15 08:12:50 +01:00
Jeremi Piotrowski
345ddb43ed jenkins/kola/azure: forward azure private networking parameters to kola
These allow the configuration of virtual network for the created
instances to join, and tell kola to use the private instance IP for
connectivity.
2022-03-15 08:12:47 +01:00
Kai Lueke
4f21679f76 Merge pull request #250 from flatcar-linux/scripts
jenkins/kola/aws: run tests on additional instance types
2022-03-10 18:23:11 +01:00
Kai Lueke
69b90bb00a jenkins/kola/packet|aws: silence debug output of concurrent test
The concurrent tests' debug output is not helping and causes confusion.
2022-03-10 18:23:11 +01:00
Kai Lueke
10b1e6abbd jenkins/kola/aws: run tests on additional instance types
There was a regression for an instance type which could have been
prevented by testing.
Add the same extended test logic used for Equinix Metal to AWS.
2022-03-10 18:23:11 +01:00
Kai Lueke
a860616ee8 Merge pull request #251 from flatcar-linux/scripts
jenkins/images: print kernel config changes
2022-03-10 12:45:53 +01:00
Kai Lueke
f385e71d3b jenkins/images: print kernel config changes
Sometimes the Linux build system changes and results in unexpected
kernel config results.
Print changes in the kernel config as part of the image diff report.
2022-03-10 12:45:53 +01:00
Kai Lueke
0f2d0e4e97 jenkins/images: find channel from base channel variable
Using the tags on the branch is not enough to find the channel we want
to be the dev build be related to.
Use the base channel variable which was introduced for this.
2022-03-04 14:28:42 +01:00
Kai Lueke
23824661c4 Merge pull request #226 from flatcar-linux/scripts
jenkins/images: print out image diff report
lts-2605.26.1
2022-02-16 12:12:40 +01:00
Kai Lueke
01b90639c6 jenkins/images: print out image diff report
Often a change results in unexpected effects on the image, e.g., when
a wrong package version gets chosen or the package installs files under
/etc, or binaries of library dependencies get pulled in. Besides
inspecting the image manually, the package-diff tool also gives
valuable insights.
Run the package-diff tool in a comparison to the last release and print
the image URL alongside for convenience.
2022-02-16 12:12:40 +01:00
Mathieu Tortuyaux
4837be9c85
jenkins/kola/packet: use metro instead of facility
Follow up of: https://github.com/flatcar-linux/mantle/pull/281

Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
2022-01-26 16:32:47 +01:00
Kai Lueke
acebe6d32c Merge pull request #176 from flatcar-linux/scripts
jenkins/kola/packet: Remove c3.small.x86 to try running cl.internet
lts-2605.25.1 lts-2605.24.1 lts-2605.23.1 lts-2605.22.1
2021-10-22 18:25:30 +02:00
Sayan Chowdhury
f68e5e9a46 jenkins/kola/packet: Remove c3.small.x86 to try running cl.internet
Signed-off-by: Sayan Chowdhury <schowdhury@microsoft.com>
2021-10-22 18:25:30 +02:00
Kai Lueke
c1f673a3e2 Merge pull request #175 from flatcar-linux/scripts
jenkins/kola/aws.sh: use larger arm64 instance type for kubeadm
2021-10-21 14:13:22 +02:00
Kai Lueke
e50a261aa6 jenkins/kola/aws.sh: use larger arm64 instance type for kubeadm
Currently the kubeadm tests fail on arm64 because the instance type
only offers 1 vCPU:
cluster.go:117: error execution phase preflight: [preflight] Some fatal errors occurred:
cluster.go:117: [ERROR NumCPU]: the number of available CPUs 1 is less than the required 2

Switch to the next larger instance type which has 2 vCPUS.
2021-10-21 14:13:22 +02:00
Sayan Chowdhury
288e6a8b1a
Merge pull request #174 from flatcar-linux/scripts
packet: Update the base URL to point to bucket.release.f-ln
2021-10-21 17:37:09 +05:30
Sayan Chowdhury
2281cb957c
packet: Update the base URL to point to bucket.release.f-ln
Signed-off-by: Sayan Chowdhury <schowdhury@microsoft.com>
2021-10-21 17:37:08 +05:30
Kai Lueke
5667d83d12 Merge pull request #173 from flatcar-linux/scripts
do: Update the base URL to point to bucket.release.f-ln
2021-10-21 12:17:18 +02:00
Sayan Chowdhury
827bea0e4e do: Update the base URL to point to bucket.release.f-ln
Signed-off-by: Sayan Chowdhury <schowdhury@microsoft.com>
2021-10-21 12:17:18 +02:00
Kai Lueke
5e06bb71b1 jenkins/kola/qemu_uefi: port changes from qemu script
In LTS we don't have the common qemu script and porting the changes
from "main" needs the additional copy to the uefi script.
2021-10-20 11:15:27 +02:00
Kai Lueke
e8e2382e4e Merge pull request #170 from flatcar-linux/scripts
jenkins: use the SDK_URL_PATH path for DOWNLOAD_ROOT_SDK
2021-10-15 16:09:36 +02:00
Kai Lueke
e6512e2e1e jenkins: use the SDK_URL_PATH path for DOWNLOAD_ROOT_SDK
The SDK can either be a release SDK or a dev build SDK which are stored
in different paths. DOWNLOAD_ROOT_SDK should be based on the
SDK_URL_PATH value which indicates whether it's a release or dev build
path.
2021-10-15 16:09:18 +02:00
Kai Lueke
df7119d228 Merge pull request #169 from flatcar-linux/scripts
jenkins/sdk: download SDK Digest to prevent twice downloading
2021-10-15 15:38:39 +02:00
Mathieu Tortuyaux
235828c99f jenkins/sdk: download SDK Digest to prevent twice downloading
bootstrap_sdk runs catalyst.sh which will try to download the SDK if the
verify digest fails.
Importing the DIGEST allows to skip this step and to continue with the
previously downloaded SDK.

Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
2021-10-15 15:38:39 +02:00
Mathieu Tortuyaux
84af532d42
jenkins/toolchain: do not use the mantle embedded key to verify
Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
2021-10-14 09:59:36 +02:00
Mathieu Tortuyaux
45baa6ffdd jenkins/kola/container: pass PORTAGE_BINHOST to container
Otherwise, it was failing since we check for unbound variable:
```
/bin/bash: line 1: PORTAGE_BINHOST: unbound variable
```

Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
2021-10-13 15:17:36 +02:00
Mathieu Tortuyaux
7cf7b48ec1 jenkins/sdk: add default DOWNLOAD_ROOT
Otherwise, the variable is empty and it creates errors later. Default
value is `gs://flatcar-jenkins`. Not `GS_DEVEL_ROOT` because if we check
the previous behavior, `DOWNLOAD_ROOT` was hardcoded with:
```shell
DOWNLOAD_ROOT_SDK=https://storage.googleapis.com/flatcar-jenkins/sdk
```

Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
2021-10-13 15:17:30 +02:00
Mathieu Tortuyaux
0b0b9ec2ac jenkins/kola/container: fix verify-key sharing in systemd container
`$verify_key` actually holds `--verify-key=verify.asc` so of course
`systemd-nspawn` fails since it does not expect `--verify-key` value.

Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
2021-10-13 15:17:25 +02:00
Jeremi Piotrowski
5430d8c1e5 jenkins/toolchains: fetch DIGESTS file to allow reusing downloaded SDK as seed
The catalyst build uses the same SDK version as seed as the current SDK, but
will only reuse the cached tarball if a DIGESTS file exists and is correct.
Prefetch this file to prevent the build from trying to access google storage
anonymously.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2021-10-13 15:17:19 +02:00
Mathieu Tortuyaux
a5f6a44ef3 jenkins/vm: remove --sdk-url
`download-image` already holds the remote host with the --root flag.

Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
2021-10-13 15:17:10 +02:00
Jeremi Piotrowski
d232135512 jenkins: use 'cork create' instead of 'cork update'
because we need to pass google credentials to update_chroot, and 'cork update'
doesn't support that.

Add --sdk-url-path to sdk.sh for new cork default.
2021-10-13 15:16:07 +02:00
Jeremi Piotrowski
1de5baed6f jenkins: use gs:// uri for DOWNLOAD_ROOT_SDK so that we can apply authentication 2021-10-13 15:05:55 +02:00
Mathieu Tortuyaux
6667783efd kola/dev-container: override binhost to use private GCS
in this commit we make sure to use GCS bucket for dev container tests by
providing the required credentials and the associated fetch command.

Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
2021-10-13 15:05:51 +02:00
Mathieu Tortuyaux
f16ebdf021 jenkins: add --json-key to cork update commands
it pulls https://github.com/flatcar-linux/mantle/pull/239 to be able to
use `--json-key` in order to access private GCS bucket

Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
2021-10-13 15:05:26 +02:00
Thilo Fromm
a51f9accc4 set_version: safely iterate over argv
Co-authored-by: Kai Lüke <pothos@users.noreply.github.com>
2021-10-13 14:54:22 +02:00
Thilo Fromm
f55495fbc5 build_image: add binhost option for dev container
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2021-10-13 14:54:18 +02:00
Thilo Fromm
fa80c77a2a setup_board, update_chroot, dev container: use new bincache mirror
This updates the default settings in build scripts to use
https://mirror.release.flatcar-linux.net/
instead of the google storage bucket if no binhost or FLATCAR_DEV_BUILDS
is specified.

Defaults are updated for
* update_chroot (runs at SDK initialisation time)
* setup_board (creates /boards/[ARCH]/) chroots
* the development container
* set_version
2021-10-13 14:54:04 +02:00
Thilo Fromm
9670ad1dfe build_library/release_util.sh: improve ssh opts for rsync
Co-authored-by: Kai Lüke <pothos@users.noreply.github.com>
2021-10-13 14:53:56 +02:00
Thilo Fromm
f764f145a6 release_util.sh: script improvements
Co-authored-by: Krzesimir Nowak <knowak@microsoft.com>
2021-10-13 14:53:52 +02:00
Thilo Fromm
a428f3ddfd release_util.sh: add rsync:// upload 2021-10-13 14:53:40 +02:00
Mathieu Tortuyaux
8e698f16a1 jenkins/manifest: use gsutil to download SDK version
Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
2021-10-13 14:53:34 +02:00
Kai Lueke
a173afe61d Merge pull request #156 from flatcar-linux/scripts
jenkins/kola/packet.sh: fix check for skipping machine type tests
2021-09-30 11:54:55 +02:00