Commit Graph

4540 Commits

Author SHA1 Message Date
Dongsu Park
7a0c0028b5
Merge pull request #183 from flatcar-linux/dongsu/em-arm-parallel-tests
jenkins: override PARALLEL_TESTS for ARM servers in da11
2021-11-26 15:49:51 +01:00
Dongsu Park
16efbc99c0 changelog: use date format of YYYY-MM-DD
If we use date format of DD-MM-YYYY in changelog file names, the files
will not sorted by date. e.g. 01-12-2021 will come before 25-11-2021.
Use date format of YYYY-MM-DD to make the files sorted by date.
2021-11-26 15:29:11 +01:00
Dongsu Park
5c391e9008 jenkins: override PARALLEL_TESTS for ARM servers in da11
We override `PARALLEL_TESTS`, because kola run with PARALLEL_TESTS >= 4
causes the tests to provision >= 12 ARM servers at the same time. As the
da11 region does not have that many free ARM servers, the whole tests
will fail. With PARALLEL_TESTS=2 the total number of servers stays < 10.
In addition, we override `timeout` to 10 hours, because it takes more
than 8 hours to run all tests only with 2 tests in parallel.
2021-11-25 16:55:10 +01:00
Sayan Chowdhury
38b1b51b46
Merge pull request #180 from flatcar-linux/sayan/mandate-changelog-entry
changelog: Add placeholder directory to add the changelogs
2021-11-24 22:53:26 +05:30
Sayan Chowdhury
663fee4c68
changelog: Add placeholder directory to add the changelogs
Signed-off-by: Sayan Chowdhury <schowdhury@microsoft.com>
2021-11-24 22:50:02 +05:30
Vincent Batts
5c21980ff3
Merge pull request #182 from flatcar-linux/tormath1/cpe
build_library/set_release: add CPE_NAME - v2.3
2021-11-24 09:51:44 -05:00
Mathieu Tortuyaux
dc0f93015b
build_library/set_release: add CPE_NAME - v2.3
Flatcar is in the NIST CPE dictionary. Let's programmatically build the
`CPE_NAME` in the build process in order to be scanned.

`CPE_NAME` is part of `/etc/os-release` with the following manual entry:

```
CPE_NAME=
  A CPE name for the operating system, in URI binding syntax, following the Common Platform Enumeration Specification[2] as proposed by the NIST.
  This field is optional. Example: "CPE_NAME="cpe:/o:fedoraproject:fedora:17""
...
[^2]: Common Platform Enumeration Specification
           http://scap.nist.gov/specifications/cpe/
```

Which indicates that the current version of CPE is 2.3.

Closes: https://github.com/flatcar-linux/Flatcar/issues/536

Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
2021-11-23 18:22:16 +01:00
Mathieu Tortuyaux
e86eadcdeb
Merge pull request #181 from flatcar-linux/tormath1/arm-em-server
kola/packet: override EM region for ARM64 server
2021-11-22 19:59:30 +01:00
Mathieu Tortuyaux
c746ab2333
kola/packet: override EM region for ARM64 server
Equinix Metal ARM server are not yet hourly available in the default `sv15` region
so we override the `PACKET_REGION` to `Dallas` since it's available in this region.
We do not override `PACKET_REGION` for both board on top level because we need to keep proximity
for PXE booting.

Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
2021-11-22 19:43:27 +01:00
Kai Lüke
e5d3c03b00
Merge pull request #171 from flatcar-linux/kai/bucket-packages
set_version: Document the bucket cache for packages
2021-10-28 18:07:19 +02:00
Kai Lueke
af8daf7dc8 set_version: Document the bucket cache for packages
Dev builds need to use the bucket cache instead of the release
binary package mirror.
Document how the user can select the bucket cache.
2021-10-28 17:45:48 +02:00
Jeremi Piotrowski
50ed7f3007
Merge pull request #178 from flatcar-linux/jepio/update-chroot-setuponly
update_chroot: introduce '--setuponly' flag
2021-10-26 13:53:41 +02:00
Jeremi Piotrowski
385f5779b7
Merge pull request #177 from flatcar-linux/jepio/github-status
jenkins: publish github status 'pending' at beginning of manifest job
2021-10-25 16:46:55 +02:00
Jeremi Piotrowski
2213e9beea update_chroot: introduce '--setuponly' flag
Currently the os/sdk and os/toolchains job perform a chroot update whose
results are immediately discarded because the rest of the build uses a fresh
chroot and catalyst. Towards the end of a release period this can extend the
build time by about an hour (longer if rust is involved).

Introduce a `--setuponly` flag that bails after the chroot configuration, and
the skips chroot update.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2021-10-25 14:14:26 +02:00
Jeremi Piotrowski
abf849ce57 jenkins: publish github status 'pending' at beginning of manifest job
and add script used for that purpose. This requires access to a github PAT
with 'repo.status' permissions.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2021-10-25 13:44:15 +02:00
Sayan Chowdhury
9b5918f70f
Merge pull request #176 from flatcar-linux/sayan/remove-c3-small-packet
jenkins/kola/packet: Remove c3.small.x86 to try running cl.internet
2021-10-22 21:25:08 +05:30
Sayan Chowdhury
2fe1e979c0
jenkins/kola/packet: Remove c3.small.x86 to try running cl.internet
Signed-off-by: Sayan Chowdhury <schowdhury@microsoft.com>
2021-10-22 21:21:39 +05:30
Kai Lüke
bf4e013ac7
Merge pull request #175 from flatcar-linux/kai/aws-arm
jenkins/kola/aws.sh: use larger arm64 instance type for kubeadm
2021-10-21 14:11:19 +02:00
Kai Lueke
0bd6d1aae5 jenkins/kola/aws.sh: use larger arm64 instance type for kubeadm
Currently the kubeadm tests fail on arm64 because the instance type
only offers 1 vCPU:
cluster.go:117: error execution phase preflight: [preflight] Some fatal errors occurred:
cluster.go:117: [ERROR NumCPU]: the number of available CPUs 1 is less than the required 2

Switch to the next larger instance type which has 2 vCPUS.
2021-10-21 14:03:39 +02:00
Sayan Chowdhury
f426fe1996
Merge pull request #174 from flatcar-linux/sayan/update-packet-base-url
packet: Update the base URL to point to bucket.release.f-ln
2021-10-21 17:25:59 +05:30
Sayan Chowdhury
0028f95a26
packet: Update the base URL to point to bucket.release.f-ln
Signed-off-by: Sayan Chowdhury <schowdhury@microsoft.com>
2021-10-21 14:01:09 +05:30
Sayan Chowdhury
d8762c831f
Merge pull request #173 from flatcar-linux/sayan/update-do-base-url
do: Update the base URL to point to bucket.release.f-ln
2021-10-21 12:15:43 +05:30
Sayan Chowdhury
e04af554fa
do: Update the base URL to point to bucket.release.f-ln
Signed-off-by: Sayan Chowdhury <schowdhury@microsoft.com>
2021-10-21 12:03:54 +05:30
Mathieu Tortuyaux
2a44ca59f3
Merge pull request #172 from flatcar-linux/tormath1/fix-qemu-common-arm
jenkins/kola: fix qemu test execution
2021-10-19 12:08:40 +02:00
Mathieu Tortuyaux
5c304ffac9
jenkins/kola/qemu: run update_chroot only for amd
if the test is ran for ARM64, there is no need to run `update_chroot`
since there is no SDK.

Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
2021-10-19 11:29:56 +02:00
Kai Lüke
df63e60b94
Merge pull request #170 from flatcar-linux/kai/use-supplied-sdk-path
jenkins: use the SDK_URL_PATH path for DOWNLOAD_ROOT_SDK
2021-10-15 16:06:24 +02:00
Kai Lueke
a668d961a9 jenkins: use the SDK_URL_PATH path for DOWNLOAD_ROOT_SDK
The SDK can either be a release SDK or a dev build SDK which are stored
in different paths. DOWNLOAD_ROOT_SDK should be based on the
SDK_URL_PATH value which indicates whether it's a release or dev build
path.
2021-10-15 14:35:08 +02:00
Mathieu Tortuyaux
97de2c4301
Merge pull request #169 from flatcar-linux/tormath1/fix-sdk-download
jenkins/sdk: download SDK Digest to prevent twice downloading
2021-10-14 17:40:45 +02:00
Mathieu Tortuyaux
46a03d3940
jenkins/sdk: download SDK Digest to prevent twice downloading
bootstrap_sdk runs catalyst.sh which will try to download the SDK if the
verify digest fails.
Importing the DIGEST allows to skip this step and to continue with the
previously downloaded SDK.

Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
2021-10-14 17:39:59 +02:00
Jeremi Piotrowski
aae8bdf74b
Merge pull request #167 from flatcar-linux/jepio/simplify-pr-builds
jenkins/manifest: support PR numbers in PORTAGE_REF/OVERLAY_REF variables
2021-10-14 17:36:14 +02:00
Jeremi Piotrowski
824199efea jenkins/manifest: accept PR numbers in PORTAGE_REF/OVERLAY_REF
When PORTAGE_REF or OVERLAY_REF are numbers, we can change the way the refspec
is constructed to allow fetching a PR instead instead of a branch. Checking for
equality using '[' works to detect numbers, bash's '[[' doesn't.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2021-10-14 11:16:11 +02:00
Mathieu Tortuyaux
d8b36d5415
Merge pull request #168 from flatcar-linux/tormath1/fix-verify
jenkins/toolchain: do not use the mantle embedded key to verify
2021-10-13 18:42:23 +02:00
Mathieu Tortuyaux
dcfe4cfa09
jenkins/toolchain: do not use the mantle embedded key to verify
Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
2021-10-13 18:23:54 +02:00
Mathieu Tortuyaux
af647340c6
Merge pull request #166 from flatcar-linux/tormath1/fix-dev-container
jenkins/kola/container: pass PORTAGE_BINHOST to container
2021-10-13 13:42:13 +02:00
Mathieu Tortuyaux
4e1e707628
jenkins/kola/container: pass PORTAGE_BINHOST to container
Otherwise, it was failing since we check for unbound variable:
```
/bin/bash: line 1: PORTAGE_BINHOST: unbound variable
```

Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
2021-10-13 10:57:35 +02:00
Dongsu Park
62707ca204
Merge pull request #157 from flatcar-linux/krnowak/break-new-util-linux-cycle
build_packages: Add another cycle to break
2021-10-13 08:15:10 +02:00
Mathieu Tortuyaux
b88177f1b0
Merge pull request #165 from flatcar-linux/tormath1/fix-sdk
jenkins/sdk: add default `DOWNLOAD_ROOT`
2021-10-12 19:00:42 +02:00
Mathieu Tortuyaux
982a26c835
jenkins/sdk: add default DOWNLOAD_ROOT
Otherwise, the variable is empty and it creates errors later. Default
value is `gs://flatcar-jenkins`. Not `GS_DEVEL_ROOT` because if we check
the previous behavior, `DOWNLOAD_ROOT` was hardcoded with:
```shell
DOWNLOAD_ROOT_SDK=https://storage.googleapis.com/flatcar-jenkins/sdk
```

Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
2021-10-12 17:17:01 +02:00
Mathieu Tortuyaux
b0dd9324fc
Merge pull request #164 from flatcar-linux/tormath1/fix-dev-container
jenkins/kola/container: fix verify-key sharing in systemd container
2021-10-12 16:38:31 +02:00
Mathieu Tortuyaux
47f5feff68
jenkins/kola/container: fix verify-key sharing in systemd container
`$verify_key` actually holds `--verify-key=verify.asc` so of course
`systemd-nspawn` fails since it does not expect `--verify-key` value.

Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
2021-10-12 16:27:29 +02:00
Krzesimir Nowak
8e8cedcbf7 build_packages: Add another cycle to break
With the update of the sys-apps/util-linux package, another cycle has
reared its ugly head. Looks like the package got an optional
dependency on sys-fs/cryptsetup, so break it.
2021-10-12 14:39:07 +02:00
Jeremi Piotrowski
24128594e0
Merge pull request #163 from flatcar-linux/tormath1-jepio/ci-authenticated
jenkins: use private bucket with authentication for fetching binaries.
2021-10-12 13:33:22 +02:00
Jeremi Piotrowski
d9542cec7d jenkins/toolchains: fetch DIGESTS file to allow reusing downloaded SDK as seed
The catalyst build uses the same SDK version as seed as the current SDK, but
will only reuse the cached tarball if a DIGESTS file exists and is correct.
Prefetch this file to prevent the build from trying to access google storage
anonymously.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2021-10-12 13:32:10 +02:00
Mathieu Tortuyaux
46ac63fdee jenkins/vm: remove --sdk-url
`download-image` already holds the remote host with the --root flag.

Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
2021-10-12 13:32:10 +02:00
Jeremi Piotrowski
cbf003e617 jenkins: use 'cork create' instead of 'cork update'
because we need to pass google credentials to update_chroot, and 'cork update'
doesn't support that.

Add --sdk-url-path to sdk.sh for new cork default.
2021-10-12 13:32:07 +02:00
Jeremi Piotrowski
93cf5aa5c3 jenkins: use gs:// uri for DOWNLOAD_ROOT_SDK so that we can apply authentication 2021-10-12 13:31:39 +02:00
Mathieu Tortuyaux
dcccdb8815
Merge pull request #162 from flatcar-linux/tormath1/container-private
kola/dev-container: override binhost to use private GCS
2021-10-11 14:47:40 +02:00
Mathieu Tortuyaux
7ef55eb15d
kola/dev-container: override binhost to use private GCS
in this commit we make sure to use GCS bucket for dev container tests by
providing the required credentials and the associated fetch command.

Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
2021-10-11 14:42:09 +02:00
Mathieu Tortuyaux
dc4e4a8701
Merge pull request #158 from flatcar-linux/tormath1/cork-private
jenkins: add `--json-key` and `--sdk-url` to `cork update` commands
2021-10-07 14:42:35 +02:00
Mathieu Tortuyaux
998d2f4fc6
jenkins: add --json-key to cork update commands
it pulls https://github.com/flatcar-linux/mantle/pull/239 to be able to
use `--json-key` in order to access private GCS bucket

Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
2021-10-07 14:41:16 +02:00