Commit Graph

1091 Commits

Author SHA1 Message Date
Mathieu Tortuyaux
dc0f93015b
build_library/set_release: add CPE_NAME - v2.3
Flatcar is in the NIST CPE dictionary. Let's programmatically build the
`CPE_NAME` in the build process in order to be scanned.

`CPE_NAME` is part of `/etc/os-release` with the following manual entry:

```
CPE_NAME=
  A CPE name for the operating system, in URI binding syntax, following the Common Platform Enumeration Specification[2] as proposed by the NIST.
  This field is optional. Example: "CPE_NAME="cpe:/o:fedoraproject:fedora:17""
...
[^2]: Common Platform Enumeration Specification
           http://scap.nist.gov/specifications/cpe/
```

Which indicates that the current version of CPE is 2.3.

Closes: https://github.com/flatcar-linux/Flatcar/issues/536

Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
2021-11-23 18:22:16 +01:00
Thilo Fromm
7f80cb138b
Merge pull request #159 from flatcar-linux/t-lo/use-new-bincache-mirror
setup_board, update_chroot, dev container: use new bincache mirror
2021-10-07 11:57:30 +02:00
Thilo Fromm
cc469d2f47
build_library/release_util.sh: improve ssh opts for rsync
Co-authored-by: Kai Lüke <pothos@users.noreply.github.com>
2021-10-07 11:17:33 +02:00
Thilo Fromm
0aad4b9dd1
release_util.sh: script improvements
Co-authored-by: Krzesimir Nowak <knowak@microsoft.com>
2021-10-07 10:48:26 +02:00
Thilo Fromm
9c3ba12936 release_util.sh: add rsync:// upload 2021-10-07 09:27:49 +02:00
Thilo Fromm
eaff2d47eb build_image: add binhost option for dev container
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2021-10-05 16:38:43 +02:00
Thilo Fromm
51aac23dc8 setup_board, update_chroot, dev container: use new bincache mirror
This updates the default settings in build scripts to use
https://mirror.release.flatcar-linux.net/
instead of the google storage bucket if no binhost or FLATCAR_DEV_BUILDS
is specified.

Defaults are updated for
* update_chroot (runs at SDK initialisation time)
* setup_board (creates /boards/[ARCH]/) chroots
* the development container
* set_version
2021-10-05 15:03:00 +02:00
Sayan Chowdhury
ccdc52cc15
Merge pull request #155 from flatcar-linux/krnowak/trap-fixes
Fix the traps to align with bash 5.1 upgrade
2021-10-01 21:49:27 +05:30
Krzesimir Nowak
ba29a29717 build_library: Ignore more categories in write_licenses
acct-user and acct-group categories aren't licensed too.
2021-09-28 17:55:17 +02:00
Krzesimir Nowak
d801ecccdf build_library/board_options: Fix pkg_version function
The documentation says it always returns zero, which is not true -
portageq could return a non-zero return value and that would be the
return value of the function. Fix the function to actually follow the
documentation - apparently the function should just return an empty
string in case of failure (like package not found).
2021-09-28 16:32:30 +02:00
Krzesimir Nowak
eee6b50aa5 *: Do not use RETURN traps
It has some weird semantics that seem to trip us up after updating
bash to 5.1. We tried to use it inside functions to clean up some
stuff after function returns. This can be emulated with an EXIT trap
within a subshell. Fortunately all the users of the RETURN trap were
not setting any global variables - modifications of such variables are
local to the subshell and are lost when the subshell exits.
2021-09-27 20:18:22 +02:00
Dongsu Park
ec19fd5aa2 build_library: add entries to the GLSA allow list
Now that GLSA metadata was updated as of 2021-09-03, we need to
add the following entries to the GLSA allow list, to avoid build
failures caused by `glsa-check -t all`.

202006-03: perl 5.26.2, only SDK, allowlist
202008-01: python 2.7.15 & 3.6.5, only SDK, allowlist
202101-18: python 2.7.15 & 3.6.5, only SDK, allowlist
202104-04: python 2.7.15 & 3.6.5, only SDK, allowlist
202105-22: samba 4.12.9, not affected, samba has no ldap flag, no smbd.
202105-34: bash 4.3, non-trivial to update
202107-31: polkit 0.113, in-progress
202107-48: systemd 247.9, backported the fixes to v247.9.
2021-09-06 16:03:13 +02:00
Dongsu Park
dfe49e7bcf build_library: remove unnecessary entries from GLSA allow list
201904-13: git 2.26.3, so not affected
201909-08: dbus 1.12.20, so not affected
201911-01: openssh 8.6, so not affected
202003-12: sudo 1.9.5, so not affected
202003-20: systemd 246+, so not affected
202003-24: file 5.39, so not affected
202003-30: git 2.26.3, so not affected
202003-31: gdb 9.2, so not affected
202003-52: samba 4.12.9, so not affected
202004-10: openssl 1.1.1l, so not affected
202004-13: git 2.26.3, so not affected
202005-02: qemu 5.2, so not affected
2021-09-03 17:43:49 +02:00
Krzesimir Nowak
4ca7027365
Merge pull request #145 from kinvolk/krnowak/drop-unused-stuff
Remove unused scripts
2021-08-30 17:53:05 +02:00
Thilo Fromm
e4c22ef6c7 build_library/disk_util: use byte array for conversion
disk_util sometimes bails out during build with an ASCII conversion
error:

Traceback (most recent call last):
  File "/mnt/host/source/src/scripts/build_library/disk_util", line 1114, in <module>
    main(sys.argv)
  File "/mnt/host/source/src/scripts/build_library/disk_util", line 1110, in main
    options.func(options)
  File "/mnt/host/source/src/scripts/build_library/disk_util", line 779, in Verity
    Tune2fsReadWrite(options, part, disable_rw=True)
  File "/mnt/host/source/src/scripts/build_library/disk_util", line 716, in Tune2fsReadWrite
    image.write(chr(flag_value))
UnicodeEncodeError: 'ascii' codec can't encode character '\xff' in position 0: ordinal not in range(128)

Curiously, the error does not reproduce every time (though the code
leading to the error is straightforward).

This change converts the integer to be written to a byte array (of size
1) instead of using chr(). Also, the file to be written is explicitly
opened in binary mode.

Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2021-08-27 17:25:34 +02:00
Krzesimir Nowak
ef0dfe5c5c Move cros_vm_constants to build_library
This file used to be imported by scripts coreos-base/cros-devutils,
which we have dropped. Now it is imported only from some other script
in build library so move it there. This leaves lib as a directory
where we keep shflags library.
2021-08-27 14:20:54 +02:00
Krzesimir Nowak
f143645215 generate_au_zip: Remove useless line 2021-08-24 16:08:15 +02:00
Krzesimir Nowak
3f9c073b4b build_library: Fix some typos 2021-08-18 19:03:11 +02:00
Krzesimir Nowak
a314348a38 build_library: Fix getting integer from string
`string.atoi` was deprecated since at least the first release of
python2, and looks like it was dropped in python3.
2021-08-18 19:01:51 +02:00
Krzesimir Nowak
ea0f478cfa build_library: Fix some string vs bytes issues
This is some fallout from converting scripts from python2 to
python3. Output received from the functions in subprocess module now
return bytearrays, but we operate on them as if they were a text. So
decode the bytearrays to strings. Otherwise we are either getting some
junk values passed to the command line utilities (for example:
`b'/dev/loop2'` instead of `/dev/loop2`), or exceptions are thrown,
because a function expected a string.
2021-08-18 18:58:16 +02:00
Krzesimir Nowak
b6c6a05404 build_library: Fix python script extension
It's not a shell script. It never was a shell script. Must have been a
mistake to name it as such.
2021-08-18 18:57:04 +02:00
Krzesimir Nowak
336a967941 build_library: Convert python2 scripts to python3
This is just a conversion done by 2to3 with a manual updates of
shebangs to mention python3 explicitly. The fixups for bytearray vs
string issues will follow up.
2021-08-18 18:56:48 +02:00
Dongsu Park
04d9fea0c1 build_library: fix docstring issues
Fix several docstring issues, so that the docstrings could match with
the actual code.
2021-08-16 10:39:26 +02:00
Dongsu Park
34cb6d305a build_library: support multi-arch in generate_au_zip
To be able to support arm64 native SDK without cross builds, we should
make generate_au_zip support both architectures, amd64 and arm64.
Without doing that, `build_image` fails with `ERROR : Required
WHITE_LIST items ld-linux-x86-64.so.2 not found!!!`, because the
script recognizes only amd64 libs in WHITE_LIST.

We should first determine the architecture in build_image, before
running generate_au_zip, and pass the architecture, either amd64 or
arm64. Also add allow_list and ld_linux parameters to necessary
functions.
2021-08-13 10:03:33 +02:00
Dongsu Park
a7f251da44 build_library: use proper vocabulary in generate_au_zip
Replace black list with deny list, white list with allow list.
2021-08-13 09:30:53 +02:00
Jeremi Piotrowski
a8a57c733b
Merge pull request #134 from kinvolk/jepio/arm64-sdk-support
bootstrap_sdk: enable arm64 sdk bootstrap
2021-08-03 12:18:01 +02:00
Kai Lüke
4321b2ab83 Revert "Revert "disk_layout: use btrfs for the OEM partition""
This reverts commit bb9ddfb08a,
meaning that the planned change is now done and we switch the OEM
partition to btrfs. The reason for the revert is resolved in
https://github.com/kinvolk/ignition/pull/22
2021-08-03 10:50:06 +02:00
Kai Lüke
bb9ddfb08a Revert "disk_layout: use btrfs for the OEM partition"
This reverts commit bc97e15c3c.
2021-07-30 20:44:36 +02:00
Kai Lüke
c1b2d88aa2 build_image_util.sh: delete vmlinuz file from /usr partition
The vmlinuz kernel image gets installed to /usr/boot/ but isn't usable
for dm-verity until it gets copied over to /boot/flatcar/ and the hash
gets embedded at a particular offset. The file in /usr/boot/ uses space
while it's not having a real purpose as long as dm-verity is used.

Delete the vmlinuz file under /usr/boot/ to free up space. When
generating the ISO image we use the vmlinuz file from /boot/flatcar/
which also has the advantage that we only distribute a single vmlinuz
file with one particular checksum.
2021-07-28 20:59:16 +02:00
Jeremi Piotrowski
6ce5d3165f
build_library/toolchain_util: check CBUILD+CHOST for rust warning
For consistency with code further down in the file: aarch64 cross compilation only applies when CBUILD is x86,
for native aarch64 builds rust is guaranteed to have aarch64 rustlibs.
2021-07-28 16:47:13 +02:00
Kai Lüke
e4f811dd0d disk_layout: optimize btrfs filesystem overhead
The defaults already give more space than the ext4 defaults but it's
recommended to use the mixed mode for filesystems smaller than 1-5 GB.
Another aspect is the duplication of metadata and while it currently is
off it's actually related to the underlying block device and could
change as soon as the block device type changes.

Select the mixed mode that uses a merged area for data and metadata
blocks. Also ensure that no metadata duplication gets enabled
automatically.
2021-07-27 14:23:47 +02:00
Kai Lüke
bc97e15c3c disk_layout: use btrfs for the OEM partition
The compression feature of btrfs allows us to store more in the
size-limited /usr and OEM partitions. The size should of course still
be monitored to not bloat the image but more headroom helps to try
things out quickly without hitting the hard limit which fails the
build.
Use btrfs for the OEM partition but with zlib compression because
the outdated GRUB version doesn't support zstd yet.
New subvolumes currently can't be used for the OEM partition as default
subvolumes because GRUB tries to read the grub.cfg from the top
subvolume (at least with our old version). (We could however use
subvolumes for the /usr partition when switching to btrfs if that
makes any sense.)
2021-07-27 14:08:42 +02:00
Kai Lüke
d0cf1a4d19 disk_util: support compressed btrfs filesystems
The limited /usr and OEM partiton size is a challenge when adding new
packages or updating a package. Since the disk layout can't be changed
for compatibility reasons when updating an existing instance, we can't
simply try out something without ensuring first that enough space is
there by removing something else. This situation can be relaxed by
leveraging btrfs compression. There was some support for btrfs but it
was a bit outdated and didn't allow to configure compression or setting
read-only flags.
Fix the btrfs support, allow to mark the default subvolume as read only
and add a compression variable that allows to select a compression
algorithm. Instead of enabling compression by setting the mount option,
we can set the filesystem attribute which has the benefit that
compression is still used with the default mount options for this (top)
directory and its contents. While for the ext2 /usr partition a hack
existed to force read-only mode by modifying some bytes and checking
these bytes could also be used to know if read-only should be used to
prevent corruption of dm-verity data, we rather check directly whether
dm-verity is active for this partition and mount it read-only (and
with the norecovery option to really prevent any write attempt).
2021-07-27 14:08:42 +02:00
Jeremi Piotrowski
8e44a167d5
Merge pull request #133 from kinvolk/jepio/fix-arm64-devcontainer
disk_layout: devcontainer: use architecture agnostic part uuid
2021-07-22 22:10:29 +02:00
Jeremi Piotrowski
4ba3218d6a boostrap_sdk: only build aarch64 cross-rust on x86 hosts
The rust ebuild has some magic to detect cross-toolchains present on the
system and enable building additional cross targets. The code to trigger
the rebuild of rust is part of install_cross_rust, and checks whether
the cross directories exist in the rust installation. If they don't,
then rust is removed and rebuilt to allow for the auto-detection to
happen.

Right now there are two issues with the code. Firstly, the path that is
checked is wrong, which leads to rust always being removed and rebuilt.
The path checked is /usr/lib/rust-*/rustlib but /usr/lib/rustlib is
where the files are installed.

The second issue is that it checks for aarch64 dirs when CHOST is
aarch64-cros-linux-gnu. However, on an aarch64 host the aarch64 dirs
will already exist from building the sdk itself. The rust ebuild is not
ready to handle aarch64 hosts yet and blows up. The correct behavior is
to combine the check for CHOST with a check for the right CBUILD.

On an aarch64 host we should presumably check for the x86 CHOST and rust
dirs, but that can be added later, because it needs more work.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2021-07-19 15:11:12 +00:00
Jeremi Piotrowski
e4f57bea82 bootstrap_sdk: fix issues around lib->lib64 symlink
The arm64 profiles don't specify SYMLINK_LIB=yes, which makes sense
since arm64 systems don't support multilib in the way that we are used
to from x86. What this means is that build artifacts are installed into
separate lib and lib64 directories. The root overlay installed in stage4
needs to check for SYMLINK_LIB before trying to create a symlink,
otherwise it fails to be applied because it collides with the directory
in the rootfs.

This uncovered a second minor issues - the rust toolchain bootstrap
scripts checked for /usr/lib64/rust*, but the ebuild installs to
/usr/lib/rust.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2021-07-19 15:11:09 +00:00
Jeremi Piotrowski
911da26072 disk_layout: devcontainer: use architecture agnostic part uuid
The previously used uuid 4f68bce3-e8cd-4db1-96e7-fbcaf984b709 is valid
for x86_64 root partitions, which resulted in the dev container not
working with systemd-nspawn on aarch64. systemd-nspawn fails with:

  No suitable root partition found in image

Change the partition uuid to the architecture agnostic one documented
in the man page:

  A GUID partition table (GPT) with a single partition of type 0fc63daf-8483-4772-8e79-3d69d8477de4.

This makes systemd-nspawn happy on aarch64.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2021-07-16 15:35:18 +00:00
Mathieu Tortuyaux
35387cfdff
build_lib/qemu_tmpl: fix short-form warning
starting from qemu-6.0.0, options with short-form are deprecated.

```
$ ./flatcar_production_qemu.sh
qemu-system-x86_64: -fsdev local,id=conf,security_model=none,readonly,path=/tmp/flatcar-configdrive.gyqmdrQqfA: warning: short-form boolean option 'readonly' deprecated
Please use readonly=on instead
```

See this commit: ccd3b3b811

Signed-off-by: Mathieu Tortuyaux <mathieu@kinvolk.io>
2021-07-16 16:04:50 +02:00
Kai Lüke
177bea4a74 Generate test update payload and run the kola update test
The kola update tests need a dev-key-signed update payload. This was
lacking and caused the update tests to be skipped.
Generate the test update payload for both dev builds and release builds
and run the kola tests for both. The test update payload has a special
name to not confuse it with the real update payload for releases, and
we keep the previous behavior to sign releases. Therefore, the
generate_update function wasn't used but the extract_update function
extended with generating the additional test payload.
2021-07-12 18:49:54 +02:00
Thilo Fromm
d1e2f6d1f2 catalyst_toolchains: don't touch the SDK
This change removes 8 years old code from the toolchains build which
tries to update SDK libraries for unknown reasons, breaking the
toolchains build in the glibc-2.33 update.

Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2021-06-25 10:37:21 +02:00
Krzesimir Nowak
3cf5d83056
Merge pull request #121 from kinvolk/krnowak/jobs-be-gone
Drop `--jobs` parameter
2021-05-05 19:25:20 +02:00
Thilo Fromm
12d59f88e0 build_library/catalyst_sdk.sh: save snapshot of coreos-overlay
This change to stage 4 of the SDK bootstrap process will keep a
snapshot of coreos-overlay in the SDK tarball. This snapshot can be
used in future SDK bootstraps' stage1 to ensure a clean stage 1 output
without any package updates.

Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2021-03-04 18:59:13 +01:00
Thilo Fromm
84d7bf63b1 bootstrap_sdk: no package updates in stage 1
This change updates the stage1 SDK bootstrap build to use local
("known good") package ebuilds only, preventing updated package ebuilds
to apply in stage 1. This fixes SDK build breakage we observed when
upgrading core libraries like readline.

The change also removes the seed update from stage 1 as it should not
be needed anymore now that we postpone any package updates to stage 2.

The following package ebuild repos are used for stage 1:
- for portage-stable, we simply copy /var/gentoo/repos/gentoo
   from the SDK root.
- coreos-overlay is more complicated since ebuilds are missing from
   the SDK. So we grok the version the SDK was built with from
     /mnt/host/source/.repo/manifests/default.xml
   and then we create a local stage 1 clone of
     https://github.com/kinvolk/coreos-overlay.git
   in which we then check out the revision noted in the default mnifest.

Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2021-03-03 17:32:38 +01:00
Krzesimir Nowak
6ed7cd66d5 *: Drop jobs parameter
The `--jobs` parameter that some scripts defined was not used anywhere
in jenkins or mantle. So the value of the parameter always ended up
being equal to `${NUM_JOBS}` set by `common.sh`. Also, even if the
`--jobs` parameter was used for some script, that script usually
didn't forward the jobs value to other scripts, so the other scripts
ended up using `${NUM_JOBS}` again. Also, the `${FLAGS_jobs}` variable
was used by some functions in the build library, and those functions
were sometimes invoked by scripts that didn't define the
`${FLAGS_jobs}` variable. It is tedious to track which script should
actually define the parameter, and where it should be forwarded.

Just get rid of this half-working pretense. If you want to affect how
many jobs `emerge` uses, export the `NUM_JOBS` environment variable
before calling any script.

For `EMERGE_FLAGS` and `REBUILD_FLAGS` we unconditionally specify the
`--jobs` flag's value to `${NUM_JOBS}` because they are passed to
`emerge`. On the other hand we drop the `--jobs` parameter from the
`UPDATE_ARGS` variable, because this variable passed to `setup_board`
or `update_chroot`, which don't have this flag any more.
2021-02-17 13:26:36 +01:00
Krzesimir Nowak
9cbf3049ea vm_image-util: Forward jobs parameter to build_oem_aci 2021-02-16 08:36:57 +01:00
Krzesimir Nowak
647690e264 build_library/check_root: Port to python3
The script needs to be ported, because it is importing portage code
which became python3 only.

The porting I did is likely a lousy job, but at least it stopped
failing with some p(yt)hony errors.
2021-02-12 10:03:16 +01:00
Krzesimir Nowak
0e41ffc336 build_library/catalyst.sh: Fix paths to ebuild repos
I have no idea how this thing worked before - the repos never were in
/usr/portage nor in /usr/local/portage… But the newer version of
portage seems to be pretty picky about the validity of repos location,
so fix them.
2021-02-03 16:47:59 +01:00
Margarita Manterola
ad8204af82 vm_image_util: Use new aws_pro flag for ec2-compat 2021-01-22 11:57:52 +01:00
Kai Lüke
bd34d059f3 Fall back to source repository license files if not in portage
When a license file is newly added, portage may not yet have it in the
shared folder and the license inclusion step fails.
Fall back to the source repositories and look for the license file
there, too. Print a warning if not found instead of failing to build.
2021-01-20 17:03:05 +01:00
Krzesimir Nowak
45d3a0d9a9 build_library: Fix git URLs
The repos were moved to kinvolk. Redirects still work, but it's better
not to rely on them.
2021-01-19 17:31:17 +01:00
Margarita Manterola
cf7f4bd4a4 Add GCE Pro OEM type 2020-12-22 13:52:52 +01:00
Vincent Batts
d46b95ba29
Merge pull request #91 from kinvolk/vbatts/arm
qemu_template: use more cpus for ARM if available
2020-12-15 08:29:10 -05:00
Kai Lüke
ef631cd7af
Merge pull request #106 from kinvolk/kai/aws-pro
Add AWS Pro OEM type
2020-12-15 11:14:25 +01:00
Sayan Chowdhury
95f05a9ad6
dev_container_util.sh: Use the shflags group instead of the default flag
The build_image script invokes the create_dev_container function, and
passes the `FLAGS_group` as param. Use the param, to generate the
binhost URL instead of using the DEFAULT_GROUP which stays as developer
always.

Fixes: kinvolk/Flatcar#298
Signed-off-by: Sayan Chowdhury <sayan@kinvolk.io>
2020-12-14 19:30:08 +05:30
Kai Lüke
f84ca169e5 Add AWS Pro OEM type
This is the initial creation of an AWS Pro image coming with support,
and features added on top of it.
2020-12-09 16:38:28 +01:00
Sayan Chowdhury
9e2098bfff
Update the azure pro package reference to the oem-azure-pro
This commit also fixes the BINHOST URL for the for the developer
container portage

Signed-off-by: Sayan Chowdhury <sayan@kinvolk.io>
2020-11-25 14:42:56 +05:30
Thilo Fromm
81d4ed8940
Merge pull request #102 from kinvolk/t-lo/build_packages-fix-symlink-run-gcc-config
build_packages, build_image_util.sh: fix up liblto symlink
2020-11-18 14:42:21 +01:00
Thilo Fromm
22b08b0ae4 build_packages, build_image_util.sh: fix up liblto symlink
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
2020-11-16 19:06:59 +01:00
Dongsu Park
6eae505f16 build_library: Ignore broken symlink in Kernel source tree
Kernel source tree started to have a broken link
`tools/testing/selftests/powerpc/copyloops/memcpy_mcsafe_64.S`.
Especially in case of Kernel 5.8.18, like:

```
broken link: /usr/src/linux-5.8.18-coreos/tools/testing/selftests/powerpc/copyloops/memcpy_mcsafe_64.S
ERROR   build_packages: test_image_content: Failed symlink check
```

Ignore the symlink when checking broken symlinks.
2020-11-13 17:27:49 +01:00
Dongsu Park
829cec45e8 jenkins: do not configure ccache variables
Setting the invalid CCACHE_ variables resulted in strange failure
in projects depending on meson, newer version like 0.55.3. For example
systemd build fails like the following errors:

```
 * ACCESS DENIED:  utimes:       /mnt/host/source/ccache
 * ACCESS DENIED:  utimes:       /mnt/host/source/ccache

F: utimes
S: deny
P: /mnt/host/source/ccache
A: /mnt/host/source/ccache
R: /mnt/host/source/ccache
C: ccache cc /build/amd64-usr/var/tmp/portage/sys-apps/systemd-246/work/systemd-246-abi_x86_64.amd64/meson-private/sanitycheckc.c -o /build/amd64-usr/var/tmp/portage/sys-apps/systemd-246/work/systemd-246-abi_x86_64.amd64/meson-private/sanitycheckc.exe -O1 -pipe -pipe -D_FILE_OFFSET_BITS=64
```

We should not set up ccache at all, as it has been already disabled in
coreos-overlay repo.
2020-10-26 15:38:57 +01:00
Sayan Chowdhury
8799028007
Add OEM package for QEMU
Signed-off-by: Sayan Chowdhury <sayan@kinvolk.io>
2020-10-08 11:51:24 +05:30
Sayan Chowdhury
9953cc8c8f
build_library: Rename the images to use pro instead of premium
Signed-off-by: Sayan Chowdhury <sayan@kinvolk.io>
2020-09-21 20:31:41 +05:30
Sayan Chowdhury
e66c89ef31
Merge pull request #93 from flatcar-linux/sayan/azure-seperate-build
Add Azure OEM and selective private upload
2020-09-21 19:55:23 +05:30
Sayan Chowdhury
cc147a1255
Add the new OEM type, and selective upload azure as private
Signed-off-by: Sayan Chowdhury <sayan@kinvolk.io>
2020-09-19 18:58:12 +05:30
Vincent Batts
ed7de96c1f
qemu_template: shell lint and update
Signed-off-by: Vincent Batts <vbatts@kinvolk.io>
2020-09-16 10:30:07 -04:00
Vincent Batts
a97df50696
qemu_template: use more cpus for ARM if available
But don't hog all `VM_NCPUS`, as we are still emulating them

Signed-off-by: Vincent Batts <vbatts@kinvolk.io>
2020-09-15 17:39:35 -04:00
Marga Manterola
c832af024e grub.cfg: Stop probing for UUID 0x01 2020-08-04 11:45:29 +02:00
Kai Lüke
cfdb7b0ab7
build_library/toolchain_util.sh: Remove old download code for Rust
The SDK now includes a Rust version with the aarch64 cross-compilation
libraries and the toolchain job doesn't build it anymore. Yet it was
still recompiled because the path had changed.
Remove the adjustment of the download URL and any automatic building
of Rust. Just issue a warning so that any problem can be spotted easily.
This change does not affect the SDK bootstrapping (full or just stage4)
but affects ./build_packages and the toolchains job. For the toolchains
job the crossdev setup is missing anyway and rebuilding wouldn't help
but only downloading, yet since in stage4 there are no binary package
URLs at all, it's best to remove this step and if it is needed later,
the warning will help.
2020-07-31 22:54:53 +02:00
Kai Lüke
c016b581b4
build_library/catalyst.sh: Specify which gcc package to rebuild
The default update seed command does only specify gcc which leads to
an error because »The short ebuild name "gcc" is ambiguous«.
Choose the standard package name instead of the cross compiler packages
which are only known to emerge because we build them as part of an SDK
release now.
2020-07-27 20:31:17 +02:00
Kai Lüke
6754d2d979
build_library/template_vmware.ovf: Newer OS type and hardware version
The VM hardware and OS type versions were outdated and resulted in
features not being available by default.
Choose a newer ESXi host version (requires 6.5) and set the guest
OS type to Linux 3.x 64 bit.
2020-07-09 12:19:23 +02:00
Marga Manterola
e04156a0f1
bootstrap_sdk: build and upload SDK toolchain pkgs
Before, we were relying on the toolchains job to build and upload
packages that were part of the SDK. With this change, all packages that
should be part of the SDK are built and uploaded by the SDK job. The
toolchains job only builds toolchain packages specific for the release.

This change includes several adjustments done to both the SDK and the
toolchains jobs to make this work:
 * Make the SDK job build all cross toolchains, including Rust
 * Stop building Rust in the toolchains job and use the one in the SDK
   instead.
 * In toolchain_util.sh: detect when the symlink folder for crossdev
   packages is missing and run crossdev to create it during
   update_chroot setup.
 * Make it possible to build the SDK starting from stage 4 instead of
   stage 1, to make the SDK building faster for PR branches / nightlies
   (full build should still be done for releases / weeklies).
2020-06-29 15:18:55 +05:30
Dongsu Park
4e109a49cf
Merge pull request #72 from flatcar-linux/dongsu/catalyst-rust-alpha
build_library/catalyst: Update virtual/rust during catalyst stage1 for alpha
2020-06-18 12:06:50 +02:00
Kai Lüke
3dadf125c8
Filter out unicode characters in package content list
For some unicode characters in ca-certificates file names "rev" complains
about an "invalid or incomplete multibyte or wide character"
and gives no output.
Filter out any unexpected characters for "rev" and replace them with "?"
so that "ls some?name" will still resolve the original name.
2020-06-11 18:00:12 +02:00
Dongsu Park
b13976285e build_library: deal with both virtual/rust and dev-lang/rust
Toolchain utils have installed only `dev-lang/rust`. It could result
in version mismatch between `virtual/rust` and `dev-lang/rust`, because
`dev-lang/rust` does not automatically pull in `virtual/rust`.

So install `virtual/rust` instead of `dev-lang/rust`.
2020-06-10 20:30:22 +02:00
Dongsu Park
8047522874 build_library/catalyst: Update virtual/rust during catalyst stage1
Install `virtual/rust` to avoid version conflicts that happen in case of
rust versions in the SDK being different from those in the new ebuilds.

`/usr/share/catalyst/targets/stage1/stage1-chroot.sh` installs gcc and
its dependencies, including `dev-lang/rust`, while `virtual/rust` does
not get updated. That results in version conflicts between
`virtual/rust` and `dev-lang/rust`. To avoid such an issue, we should
update also `virtual/rust` when building stage1. Since `virtual/rust`
automatically pulls in `dev-lang/rust`, we do not need to explicitly
specify `dev-lang/rust` here.
2020-06-10 20:28:40 +02:00
Kai Lüke
c7ad59e28c
build_library: Enhance license info and store it on the image
The license JSON file did only include the package names but not
any other metadata. Also since the file was not on the image itself,
it had to be downloaded.
Add more metadata to the license JSON and store it on the image.
2020-06-09 21:09:44 +02:00
Benjamin Gilbert
1c0da3c910
test_image_content: whitelist OpenSSL GLSA
We updated to 1.0.2u instead of 1.1.1g.
2020-05-22 15:30:54 +02:00
Benjamin Gilbert
42a45ac727
test_image_content: whitelist some GLSAs
Git was fixed by updating to 2.23.3, not 2.26.2.  Python and QEMU are
only in the SDK.
2020-05-22 15:30:48 +02:00
Benjamin Gilbert
df718abc57
test_image_content: whitelist some GLSAs
systemd and sudo are already fixed.  Git was fixed by updating to 2.23.2,
not 2.24.1.  Samba is 2 years old and customized, thus difficult to update.
file, Python, and gdb are only in the SDK.
2020-05-22 15:30:30 +02:00
Kai Lüke
15681e1d77
Support the /boot/coreos/first_boot flag file
If a user or old software creates the flag file on the old CoreOS location,
nothing would happen.
Check the old location, too, so that Ignition is rerun.
2020-05-22 15:22:46 +02:00
Kai Lüke
76abe2e8c5
set_lsb_release: update codename 2020-05-18 12:22:51 +02:00
Thilo Fromm
56341ea725 qemu_template.sh: update naming CoreOS -> Flatcar 2020-05-15 15:25:27 +02:00
Kai Lüke
06c4894f71
SDK: Take environment variable to specify SDK location
The dev build SDKs are not in $FLATCAR_DEV_BUILDS/sdk but published under
$FLATCAR_DEV_BUILDS/developer/sdk.
Add an environment variable to specify where the SDK is to be found
but default to $FLATCAR_DEV_BUILDS/sdk if it is not specified.
From Jenkins this variable is exported as DOWNLOAD_ROOT_SDK.
2020-05-14 16:03:15 +02:00
Kai Lüke
74f04c59f7
Reuse correct binary packages for a Flatcar version
Two Flatcar versions were used in /etc/portage/make.conf both in the SDK
and in the boards.
Use only a single version by default to get the expected results and not
something else when using binary packages.

The Rust crossdev package was never uploaded to /sdk/ and always
had to be compiled again.
Upload it in a separate toolchain-arm64 directory because /Packages in /crossdev/
doesn't refer to the Rust package and its use flags.
2020-05-11 15:45:21 +02:00
Kai Lüke
d1fb202fe2
build_library/dev_container_util.sh: Use correct BINHOST URLs
The BINHOST was still configured to be the CoreOS CL upstream location
which does not work for independent Flatcar CL releases. This broke
binary package installation in the development container.
Use the correct BINHOST to fix installation of binary packages in the
development container.
2020-05-04 13:17:59 +02:00
Kai Lüke
aebba12996
build_library/template_vmware.ovf: Document and add cloud-init OVF vars
The configuration variables for the Ignition configuration also serve as
data source for coreos-cloudinit config data (which includes plain scripts).
Document them properly and also call out that the networking variables only
work if coreos-cloudinit data is used.
For some use cases, too few networking variables were available. Add secondary
routing variables for the main network interface and add a second interface.
2020-04-27 13:48:10 +02:00
Kai Lüke
42cf3423d4
build_library/template_vmware.ovf: Remove old CoreOS OVF variables
There was a logical mistake in Ignition that caused ignition.config.*
only to work when it was part of the ovfenv. Thus they were added but
the old CoreOS variables marked deprecated and kept. With both as OVF
variables each of them worked but directly specifying ignition.config.*
as guest variable still didn't because of the logical mistake.
Now there is a fix and both work well when specified directly as guest
variable (https://github.com/flatcar-linux/ignition/pull/11).
Delete the old CoreOS OVF variables because they just clutter the UI
and only the Ignition variables should be used in the UI.
2020-04-17 17:05:02 +02:00
Kai Lüke
687bbefad8
build_library/vm_image_util.sh: Generate iPXE script
Write out an iPXE script file for Packet.
The script uses relative URLs to refer to
the other PXE files and thus can be copied
along with the files to any server.
This is useful because it saves the creation
of an iPXE script for a release/channel on a
third-party service. For CI testing it is
also helpful because the script does not only
end up on the release server but also already
on the Google buckets, refering to unpublished
PXE payloads.
2020-03-18 16:09:40 +01:00
Kai Lüke
f9c8feb09c
build_library/template_vmware.ovf: Use Ignition OVF vars
For the Ignition variables to be usable they need to be
specified in the OVF.
Call out that the CoreOS variables are deprecated to
reduce confusion when both are displayed besides each other.
2020-03-10 13:18:36 +01:00
Kai Lüke
1bf2bba4bf
build_library/dev_container_util.sh: Correct repo URLs
Same content as in flatcar-build-2387.
2020-02-19 14:12:02 +01:00
Kai Lüke
42068f1cbc
build_image: Add prodtar command to build a tar ball
Create a tar ball with the contents of the / and /usr partitions
to be used as follows with systemd-nspawn (via machinectl):
  machinectl import-tar flatcar-container.tar.gz flatcar-container
  machinectl start flatcar-container
  machinectl shell flatcar-container
or with docker by converting it to an OCI image:
  docker import -c "CMD /bin/bash" flatcar-container.tar.gz flatcar-container

Since the new "prodtar" command relies on the results of the "prod" command,
it bundles it so that "prod prodtar" and "prodtar" is the same.
2020-02-07 17:33:37 +01:00
Kai Lüke
1a1fcadf0a
Use same APPID for arm64 and amd64
Nebraska supports an additional arch
variable to distinguish both boards.
2020-01-22 10:50:25 +01:00
Andrew Jeddeloh
15745d520e
disk_util: retry loopback mounts if they fail
This fails frequently but should succeed if retried. This should reduce
failed builds.
2020-01-21 13:29:45 +01:00
Andrew Jeddeloh
00d77d199a
build_library/disk_util: work around losetup bug
Retry losetup if it fails, up to 5 times with 5 seconds between retries.
2020-01-21 13:29:30 +01:00
Flatcar Buildbot
8ac3182bc3 2345.0.1 2019-12-05 09:01:56 +01:00
Flatcar Buildbot
c7bbb2b1e2 2345.0.0 2019-12-04 14:59:11 +01:00
Flatcar Buildbot
7ee9c9c033 2331.0.0 2019-11-22 14:34:26 +01:00
Flatcar Buildbot
1dad511f69 2317.0.1 2019-11-07 19:40:01 +01:00
Benjamin Gilbert
bd79eaf9d1 test_image_content: whitelist Perl and dbus GLSAs
Backported the dbus GLSA.  Ignoring the Perl one.
2019-09-10 21:47:55 -04:00
Benjamin Gilbert
97e61a164b test_image_content: whitelist polkit GLSA
Both CVE fixes were backported.
2019-08-24 01:28:29 -04:00
David Michael
9b863fa7ae Revert "build_library: Add temporary workaround for binutils update"
This reverts commit faf07f1b8f.
2019-08-08 15:53:06 +00:00
David Michael
faf07f1b8f build_library: Add temporary workaround for binutils update
Revert this after the new binutils is built into the SDK.
2019-08-03 16:24:27 +00:00
Andrew Jeddeloh
006fe8765d disk_layout: bump dev container from 3 to 4GB
Builds are beginning to run out of space with 3GB. Bump to 4GB.
2019-06-11 21:39:02 +00:00
David Michael
2791e33eb5 Revert "catalyst: Temporarily disable update_seed again"
This reverts commit a85cb08443.
2019-05-02 03:23:07 +00:00
David Michael
a85cb08443 catalyst: Temporarily disable update_seed again
The glib security update blocks itself.  Disable this until a new
SDK is generated, then it can be reverted again.
2019-04-23 03:29:09 +00:00
David Michael
3254c63690 build_library: Drop whitelisted systemd GLSA
This no longer matches since upgrading to 241.
2019-03-12 16:29:48 +00:00
David Michael
4bd4cd5a8e disk_util: Add -I for mkfs.vfat
When loop device partition nodes aren't cleaned up, building images
will fail with:

mkfs.vfat: Partitions or virtual mappings on device '/dev/loop0', not making filesystem (use -I to override)

Just add the flag unconditionally to work around it.
2019-02-10 02:44:01 +00:00
David Michael
cea88d9c52 Revert "catalyst: Temporarily disable update_seed again"
This reverts commit 7d99b4dbdd.
2019-01-28 15:58:47 +00:00
David Michael
06cbc02e0c check_root: Silence libperl warnings 2019-01-25 21:52:27 +00:00
David Michael
7d99b4dbdd catalyst: Temporarily disable update_seed again
The Perl update will break SDK bootstrapping during seed update, so
disable it again.  This can be reverted after bumping the SDK to a
version that includes the new Perl.
2019-01-25 18:12:55 +00:00
David Michael
7b7c78b592 build_library: Drop old workarounds for toolchain binpkgs 2019-01-24 23:25:51 +00:00
David Michael
45c9ef7867 build_library: Drop whitelisted Go GLSAs
Since Go is no longer cross-compiled, it won't hit the boards' GLSA
tests anymore.
2018-12-21 16:27:55 +00:00
David Michael
e1761bee54 Revert "catalyst: Temporarily disable update_seed again"
This reverts commit 373d5a814b.
2018-11-16 14:11:00 +00:00
David Michael
103c7425fe build_library: Whitelist the systemd GLSA until we have 239 2018-11-01 14:45:21 +00:00
David Michael
373d5a814b catalyst: Temporarily disable update_seed again
This works around the edk2 rename breaking SDK bootstrapping.  It
can be reverted when an SDK has the new version built into it.
2018-10-25 17:08:02 -04:00
David Michael
f9432f6702 vm_image_util: Switch back to edk2-ovmf paths
The custom sys-firmware/edk2 package has been replaced by Gentoo's
sys-firmware/edk2-ovmf package now that only amd64 is supported.

This partially reverts 1761d9d071 .
2018-10-25 17:15:03 +00:00
David Michael
2467923d56 Remove arm64 from supported board operations 2018-10-25 16:00:09 +00:00
David Michael
a598864413 catalyst: Update sharedir for catalyst 3 2018-10-19 12:12:46 -04:00
David Michael
23c5b018c6 prod_image_util: Drop /usr/include
This should be covered by INSTALL_MASK, but it apparently gets
confused by a symlink loop installed by json-c.  Remove it here.
2018-10-18 16:32:32 +00:00
David Michael
9adb659106 prod_image_util: Drop the portage cache database 2018-10-18 15:58:27 +00:00
David Michael
729575c22e update_chroot: Allow upgrading glibc and binutils 2018-10-12 17:58:09 +00:00
David Michael
c887cfdce6 bootstrap_sdk: Explicitly set the Python version to 2.7
It has recently started picking Python 3 for /usr/bin/python.
2018-09-28 13:53:13 +00:00
David Michael
5ffb938934 image_to_vm: Fix OEM install roots for new portage 2018-08-30 00:53:17 +00:00
David Michael
afff45366a build_toolchains: Update sysroot settings for newer portage
Since EAPI=7 was supported, portage can no longer use different
ROOT and SYSROOT values.  This adjusts the paths so that the first
phase builds cross-toolchains under /usr/${CHOST}, then the native
toolchains are built under /build/${BOARD} (as was being done
previously).  Now that the cross-toolchain development files can't
be used when building the native toolchain, the headers and libs
are stupidly copied into the board root to be used used and then
overwritten by the board packages as they are built.  Since this is
all done in a chroot, these changes shouldn't affect the SDK host.
2018-08-29 21:40:21 +00:00
David Michael
ecc6d6bd3e build_toolchains: Stop building arm64 toolchains 2018-08-29 18:04:55 +00:00
Andrew Jeddeloh
16d5f05275 build_image: rename dev_image to dev_container 2018-07-25 13:11:17 -07:00
Andrew Jeddeloh
1718adb22a *: remove dev image
The dev image has been broken for some time. Some bits are left because
the dev container (not broken) still uses them. They should be renamed.
2018-07-25 13:11:17 -07:00
David Michael
be1344f12d Revert "catalyst: Temporarily disable update_seed"
This reverts commit efbc542c59.
2018-06-29 13:38:08 +00:00
David Michael
efbc542c59 catalyst: Temporarily disable update_seed
This can be reverted after switching to an SDK with Python 3.6
built into it.
2018-06-25 09:28:37 -04:00
Benjamin Gilbert
25fffe839d build_library: update VMware virtual hardware version to 11
This will run on ESXi 6.0 and above, and all non-EOL versions of Fusion
and Workstation.

Also enable a few useful VMX features (HPET; CPU and memory hotplug) that
are added by VMware Workstation 14.1.1's Change Hardware Compatibility
wizard. Correspondingly, enable CPU/memory hotplug in the OVF; omit
HPET because there's no obvious way to enable it.
2018-05-18 16:23:32 -04:00
Benjamin Gilbert
e9d1fd1d10
Merge pull request #815 from bgilbert/unrestricted
build_library: add --unrestricted to default GRUB menuentry
2018-05-17 17:37:21 -04:00
Benjamin Gilbert
819c0803dd build_library: add --unrestricted to default GRUB menuentry
This allows booting the default entry even if GRUB authentication is
configured in /usr/share/oem/grub.cfg.
2018-05-17 16:57:22 -04:00
Benjamin Gilbert
1717709dbf build_library: avoid obsolete QEMU -net syntax
It's been deprecated since QEMU 0.12. Fixes warning on QEMU startup:

    qemu-system-x86_64: -net nic,vlan=0,model=virtio: 'vlan' is deprecated. Please use 'netdev' instead.
2018-05-15 17:33:27 -04:00
Benjamin Gilbert
72bd0b175b build_library: attach virtio-rng-pci device to QEMU VMs
Pass /dev/urandom through to QEMU VMs. (Not /dev/random, to avoid
draining the host's entropy.) This speeds initialization of VM entropy.
2018-05-15 17:22:24 -04:00
Benjamin Gilbert
0e83301d8c build_library: drop oracle_oci_qcow image format
Images have been published to the release buckets, but never to OCI.
2018-04-25 22:40:29 -04:00
David Michael
a9f4311f86 build_library: Whitelist the new Go 1.10 GLSA
This is the same story as the others: our images will fail the GLSA
checks as long as we build old Go versions.  However, this one will
fail for any version less than 1.10.1 now.
2018-04-16 13:21:21 -04:00
Benjamin Gilbert
fdf8cc2cda Revert "Revert "disk_util: use FAT32 on ESP""
FAT32 seemed to aggravate https://github.com/coreos/bugs/issues/2284, but
now that that issue has been addressed, we can return to the correct
filesystem type.

This reverts commit 299f8fb3d1.
2018-04-12 17:00:28 -04:00
David Michael
d2f2e11225 update_chroot: Allow upgrading glibc 2018-04-06 18:30:38 -04:00
Benjamin Gilbert
9665a5e6fb
Merge pull request #799 from dongsupark/dongsu/fix-qemu-template
Fix a bug when setting VM_CDROM in qemu template
2018-03-12 17:01:05 -04:00
Dongsu Park
52214f533b build_library: fix a bug when setting a CDROM image
Setting `$VM_CDROM` in the qemu script does not work as expected when
installing Container Linux from the given bootable CDROM image. That's
probably because qemu-system-x86_64 expects another boot option `-boot
order=d` to be able to boot from the given CDROM drive. Let's specify
specify a `-boot` as well as `-drive` option for the given CDROM drive.
2018-03-12 21:38:42 +01:00
David Michael
3383790d28 build_library: Whitelist the new Go 1.9 GLSA
This is the same case as the previous one.  Our Go 1.8 package has
the fix, but none of the older unsupported versions do.  Since we
have multiple installed versions and this says anything less than
Go 1.9 is vulnerable, we have to whitelist it until all older
versions of Go are removed from the OS.
2018-03-07 18:54:15 -05:00
David Michael
09e3c49811 catalyst: Disable ccache
It provides no value when it works, and it's randomly causing
failures to build toolchains due to permissions problems after
certain releases.  This also requires taking it out of FEATURES in
the portage profile (which is the SDK profile by default).

Test Jenkins runs of SDK and toolchains jobs both ran in the same
time as with ccache enabled.
2018-03-05 17:58:25 -05:00
Benjamin Gilbert
086b708c2f set_lsb_release: update codename 2018-01-31 09:37:02 -08:00
David Michael
e9a61e9333
Merge pull request #783 from dm0-/gcc
Upgrade to GCC 6.4.0
2018-01-08 19:35:37 -05:00
Andrew Jeddeloh
5dfe3897b9
Merge pull request #778 from ajeddeloh/prod-default
*: make prod images default
2018-01-05 12:50:39 -08:00
David Michael
73dc0c971c build_library: Drop the binutils GLSA from the whitelist 2018-01-04 17:41:13 -05:00
David Michael
ac8402c1aa update_chroot: Allow upgrading binutils and GCC
Normally toolchains packages are prevented from upgrading.  This
drops that restriction and explicitly removes old versions so that
conflicting tool profiles are not accidentally used.

This reverts commit 20975049b3.
2018-01-04 14:27:47 -05:00
Andrew Jeddeloh
be16e7ce3f *: remove --prod_image in other scripts 2017-12-21 14:44:23 -08:00
Andrew Jeddeloh
299f8fb3d1 Revert "disk_util: use FAT32 on ESP"
This reverts commit 7f058d61a1.

Reverting because of bug 2284 [1] where grub will sometimes fail due to
memory corruption. This is _not_ the cause of the bug, and the bug can
even be reproduced with this reversion, but it seems to occur less when
not using fat32.

[1] https://github.com/coreos/bugs/issues/2284
2017-12-18 13:36:56 -08:00
Benjamin Gilbert
7f058d61a1 disk_util: use FAT32 on ESP
mkfs.vfat was defaulting to FAT16 based on the size of the partition.
The UEFI spec (2.7 errata A, section 13.3) implies that only FAT32 is
necessarily supported on the ESP, and we've received a report of
hardware that doesn't recognize FAT16.
2017-11-13 11:27:31 -08:00
Benjamin Gilbert
b091c2d0c2 disk_util: drop support for syslinux partition feature
It hasn't been used since 5bfa0c8d20.
2017-11-11 14:39:02 -08:00
Benjamin Gilbert
b1beff7462 build_library: upload kernel config as release artifact
It's useful to have this information available without starting an
image or digging up a binpkg.
2017-11-09 01:28:03 -08:00
Euan Kemp
706e4d588d Merge pull request #766 from euank/board-binhost
dev_image_util: hardcode correct portage binhost
2017-10-25 18:17:34 -07:00
David Michael
e0f6100de8 build_library: Whitelist the Go 1.9 GLSA
We handle Go differently than Gentoo, so our 1.8.4 package includes
the same security fixes.  When all packages are built with Go 1.9,
the older Go packages shouldn't be installed anymore, so this line
can be dropped.
2017-10-23 14:35:52 -07:00
Euan Kemp
dcbc6a8686 dev_image_util: hardcode correct portage binhost
We already hardcode similar urls a bit below (the sync-uri).

Not hardcoding the binhost results in an incorrect value during
embargoed build uploads.
2017-10-23 14:14:58 -07:00
David Michael
20975049b3 Revert "update_chroot: Avoid a portage crash while upgrading binutils"
The binutils update prevents Linux from uncompressing during boot,
so this is being put off until it's fixed.

This reverts commit 5e659964d0.
2017-10-19 08:43:41 -07:00
David Michael
059a21701b build_library: Whitelist the binutils GLSA 2017-10-19 08:42:57 -07:00
David Michael
5e659964d0 update_chroot: Avoid a portage crash while upgrading binutils
This omits the toolchain packages' version-pinning flag for the
binutils package while it is being upgraded.  It also removes older
versions installed in parallel that cause unwanted rebuilds.

When stable has the upgraded version, this can be reverted.
2017-09-29 14:10:58 -07:00
David Michael
5e6604c85b Merge pull request #754 from dm0-/rust
check_root: Ignore missing Rust libraries
2017-09-22 16:51:55 -07:00
Benjamin Gilbert
fcbc7c3219 vm_image_util: bump PXE and ISO scripts to 2 GiB RAM
Now that we're shipping multiple docker packages in the image, 1 GiB
is not enough to boot successfully.
2017-09-22 15:52:19 -07:00
David Michael
abc99cc29e check_root: Ignore missing Rust libraries 2017-09-21 14:37:09 -07:00
David Michael
1742e8ce18 build_library: Filter out null paths in torcx manifests
This avoids odd "null" lines appearing in torcx image names.
2017-09-15 12:17:52 -07:00
David Michael
8fde256e45 build_library: Add torcx source packages to the package list
This includes the source package of all torcx packages that are
installed on disk, including cases where multiple versions of the
same package are available.
2017-09-11 15:32:11 -07:00
Euan Kemp
83a16990d1 Merge pull request #737 from euank/torcx-cas
rework torcx uploading to include a manifest
2017-09-08 17:09:54 -07:00
Euan Kemp
d3867403ed build_image: populate torcx store from manifest
This moves the default symlinking logic into build image as well.

This assumes that a torcx store is available locally with all images
referenced in the torcx manifest.

This is accomplished with a highly-indented double-for-loop, but I think
it's still decently readable.
2017-09-08 16:58:51 -07:00
Euan Kemp
31d7b40af3 build_library: add more torcx_manifest helpers
These are useful for the build_image_util code soon to come.
2017-09-08 16:58:51 -07:00
Euan Kemp
8f712b2af1 build_library: add torcx upload/download roots
Torcx is special in that it wishes to be uploaded under a prefixed
directory (torcx), typically wishes to be downloaded from there, but
ultimately wants to be downloaded from a location without that prefix.

In fact, I expect during a normal release process, it will be uploaded
with that prefix to the build bucket, copied without that prefix to the
final bucket (during pre-release), and then finally downloaded without
the prefix.

I think this set of variables ends up being the cleanest way to
represent this complexity.
2017-09-08 16:42:47 -07:00
Euan Kemp
0604b5c40a build_library: add torcx_manifest helper
This implements some basic functionality around working with torcx
manifests.
2017-09-07 12:21:36 -07:00
Benjamin Gilbert
2024326789 build_library: rename oracle_oci image to oracle_oci_qcow
Reserve the unsuffixed name for a raw image, in case we ever want to
support installing to disk using "coreos-install -o" on a PXE-booted
system.
2017-09-06 17:46:07 -07:00
Benjamin Gilbert
51c5345c0d build_library: Add Oracle OCI OEM and format 2017-08-30 03:10:33 -07:00
Geoff Levand
e76030e0e9 build_library: Update qemu script to run on arm64 machines
Signed-off-by: Geoff Levand <geoff@infradead.org>
2017-07-28 14:39:53 -07:00
Geoff Levand
5e94a55069 build_library: Add missing die function
Signed-off-by: Geoff Levand <geoff@infradead.org>
2017-07-28 14:39:53 -07:00
David Michael
71ebc66065 build_library: avoid a SIGPIPE while building arm64 UEFI firmware
This just sets the code file size to the var file size, so it gets
zero-padding without having to pipe commands together.

From: David Michael <david.michael@coreos.com>
[Rebased]
Signed-off-by: Geoff Levand <geoff@infradead.org>
2017-07-27 15:03:58 -07:00
Geoff Levand
1761d9d071 vm_image_util: Update to use edk2 package
Signed-off-by: Geoff Levand <geoff@infradead.org>
2017-07-27 14:51:32 -07:00
David Michael
963fc72f0e check_root: ignore cgi.py in any Python version 2017-07-26 13:04:55 -07:00
David Michael
336ac28026 Merge pull request #695 from dm0-/perl
Add workarounds for upgrading to Perl 5.24
2017-06-26 16:50:23 -07:00
Benjamin Gilbert
e98efe074d build_library: Add vagrant_virtualbox image type 2017-06-22 13:41:15 -07:00
Alex Crawford
65986e02bc build_library: split vagrant and vagrant vbox
This reverts the vagrant image back to using oem-vagrant because we
don't want to break the existing images. It moves the new,
Ignition-powered virtualbox flavor of vagrant into a new image.
2017-06-22 13:23:50 -07:00
Benjamin Gilbert
ab4705f9dc Merge pull request #697 from bgilbert/generic
Drop ISO/PXE support for amd64-generic, and an unused variable
2017-06-21 13:28:55 -07:00
Benjamin Gilbert
1ef4609ef2 scripts: Drop unused BOARD_USE assignments 2017-06-16 19:22:51 -07:00
Benjamin Gilbert
cb8b8f34ca build_library: Drop ISO/PXE support for amd64-generic 2017-06-16 19:05:07 -07:00
Euan Kemp
849af767a6 Merge pull request #692 from AlexNPavel/vagrant-virtualbox
Use new oem packages for virtualbox and vagrant
2017-06-16 17:00:39 -07:00
David Michael
18d89accde Merge pull request #685 from dm0-/torcx
Populate the torcx vendor store in OS images
2017-06-16 15:57:04 -07:00
David Michael
5a50abc4ac check_root: silence the incorrect libperl warnings 2017-06-14 16:29:14 -07:00
Alexander Pavel
7298b0bf2a build_library: Change oem package for vagrant to oem-vagrant-virtualbox
This changes the oem-package for vagrant to vagrant-virtualbox,
which uses ignition instead of cloud-clonfig and sets the oem id
to "virtualbox" so that ignition can handle the machine correctly
2017-06-14 15:58:17 -07:00
Alexander Pavel
cab8e3ed37 build_library: Add virtualbox oem to vm_image_util
This commit adds support for the oem-virtualbox package for
virtualbox builds
2017-06-14 13:21:21 -07:00
David Michael
a707c1b597 Revert "catalyst: temporarily disable update_seed"
This reverts commit 43807d6763.
2017-06-14 13:04:09 -07:00
Benjamin Gilbert
40978865d2 Revert "A pile of goo to support a minimal secure boot demo"
This reverts commit 0c6adece3d.
2017-06-08 14:51:15 -07:00
Benjamin Gilbert
8b322c6556 test_oem_pkgs: Drop reference to oem-xendom0 2017-06-08 14:48:38 -07:00
Benjamin Gilbert
6878bd8f5f Revert "vm_image_util: add qemu_xen image type for testing"
This reverts commit fe9db4157b.
2017-06-08 14:47:23 -07:00
David Michael
a0bebcc38b build_image: copy torcx images into the vendor store
This adds the option --torcx_store to specify the path to a
directory containing torcx images to be baked into the OS image.  A
blank string can be given instead of a path to restore the previous
behavior and leave an empty vendor store.

The default value is the default path created by build_torcx_store,
which is used when build_packages updates torcx images.  This means
that the current pattern "./build_packages && ./build_image prod"
should result in a fully updated OS image with all torcx images
available in the vendor store.
2017-06-02 14:23:03 -07:00
Benjamin Gilbert
6cfb286748 grub: Update bug report URL 2017-05-25 11:38:14 -07:00
Benjamin Gilbert
5177c10191 Merge pull request #679 from bgilbert/format
image_to_vm: Default to qemu_uefi on ARM
2017-05-08 14:09:52 -07:00
Pan Luo
234ae23443
Change default value of dhcp for vmware template to no
When OVA template is not being used, the default dhcp value yes will
trigger cloud-init to generate a 00-.network file, which will break
network connectivity Intermittently. Please see the details here:
https://github.com/coreos/bugs/issues/1802#issuecomment-297847614
2017-05-04 09:30:55 -07:00
Benjamin Gilbert
c98788befc image_to_vm: Default to qemu_uefi on ARM
qemu makes no sense on that platform.
2017-05-02 15:02:09 -07:00
David Michael
d9bfa7e1ac build_image: do not use gs:// URLs in dev images
When building dev images, the PORTAGE_BINHOST value during build
time is written to the image's make.conf.  This breaks the default
binary package setup, since Jenkins is using gs:// URLs for signed
package verification and authenticated downloads, and the make.conf
doesn't inherit the GS_* variables to handle those schemes.

This should be reverted when signed packages are properly supported
by default in the dev images.
2017-04-25 11:29:44 -07:00
Benjamin Gilbert
329f4bd113 build_library: Run dd with status=none
Avoid dumping dd statistics to stderr.
2017-04-21 14:06:00 -07:00
David Michael
fdbb2d1bd6 check_root: silence the incorrect libperl warnings 2017-04-19 13:00:10 -07:00
Euan Kemp
3dec02768d Merge pull request #669 from euank/bug-url
lsb_release: update BUG_REPORT_URL
2017-04-14 12:56:12 -07:00
Euan Kemp
539723169b lsb_release: update BUG_REPORT_URL
See https://github.com/coreos/coreos-overlay/pull/2463 for the
reasoning.

This was missed in that sweep of changes.
2017-04-14 12:39:07 -07:00