4061 Commits

Author SHA1 Message Date
Euan Kemp
18b6fc9343 dev_image_util: hardcode correct portage binhost
We already hardcode similar urls a bit below (the sync-uri).

Not hardcoding the binhost results in an incorrect value during
embargoed build uploads.
2017-10-25 13:03:00 -07:00
David Michael
4883c0377f Merge pull request #761 from dm0-/build-1520
Update Docker to 17.09 and add 17.03 in beta
2017-10-03 11:22:21 -07:00
David Michael
27856d043d build_torcx_store: Build a non-local Docker 17.03 image 2017-10-02 15:14:33 -07:00
David Michael
1cb7c18acb build_torcx_store: Update the latest Docker to 17.09 2017-10-02 15:14:24 -07:00
Benjamin Gilbert
2c08a15c3a Merge pull request #758 from bgilbert/docker-flag-1520
Add Docker 17.06 to the image for 1520
2017-09-26 16:15:37 -07:00
Benjamin Gilbert
4d115a4218 vm_image_util: bump PXE and ISO scripts to 2 GiB RAM
Now that we're shipping multiple docker packages in the image, 1 GiB
is not enough to boot successfully.
2017-09-26 15:07:00 -07:00
Benjamin Gilbert
017f028548 build_torcx_store: add docker 17.06 to the image 2017-09-26 15:06:47 -07:00
David Michael
e5b81e9065 Merge pull request #750 from dm0-/build-1520
build_library: Filter out null paths in torcx manifests
2017-09-15 12:16:49 -07:00
David Michael
fea4b4624e build_library: Filter out null paths in torcx manifests
This avoids odd "null" lines appearing in torcx image names.
2017-09-15 12:13:09 -07:00
David Michael
7995d7dde9 Merge pull request #744 from dm0-/build-1520
Include on-disk torcx source packages in the package list
2017-09-15 12:11:36 -07:00
Benjamin Gilbert
e3a5995074 Merge pull request #745 from bgilbert/docker-1520
build_torcx_store: default to docker 1.12 for 1520
2017-09-11 18:50:45 -07:00
Benjamin Gilbert
fbcc06b07b build_torcx_store: default to docker 1.12 2017-09-11 18:48:28 -07:00
David Michael
d06c6a1956 build_library: Add torcx source packages to the package list
This includes the source package of all torcx packages that are
installed on disk, including cases where multiple versions of the
same package are available.
2017-09-11 15:42:10 -07:00
Euan Kemp
9e9ea26c4e Merge pull request #742 from euank/build-1520-torcx-cas
Backport build_torcx_store manifest changes
2017-09-08 17:36:15 -07:00
Euan Kemp
7e5906ceeb build_torcx_store: back-out reproducible digests
Per the comment there, they were implemented in a broken fashion.

This leaves the door open for using them in the future, but in the
meanwhile simply uses the sha512sum as the digest (which solves the
immediate issue).
2017-09-08 17:20:22 -07:00
Euan Kemp
a9fcf77818 build_torcx_store: upload manifests to upload_root
Rather than to a location special to them.
The comment in the code explains the reasoning for this.
2017-09-08 17:20:22 -07:00
Euan Kemp
c5594a39db jenkins/images: use torcx packages from manifest
This accompanies the other changes around moving to a torcx manifest.
2017-09-08 17:20:22 -07:00
Euan Kemp
6e567733d0 jenkins/packages: configure download root
Pending jenkins also including this variable.
2017-09-08 17:20:22 -07:00
Euan Kemp
61c2cc7015 build_torcx_store: upload to 'pkgs'
This is consistent with the local directory hierarchy
2017-09-08 17:20:22 -07:00
Euan Kemp
7e0324b069 build_image: populate torcx store from manifest
This moves the default symlinking logic into build image as well.

This assumes that a torcx store is available locally with all images
referenced in the torcx manifest.

This is accomplished with a highly-indented double-for-loop, but I think
it's still decently readable.
2017-09-08 17:20:22 -07:00
Euan Kemp
7a72d707d2 build_library: add more torcx_manifest helpers
These are useful for the build_image_util code soon to come.
2017-09-08 17:20:22 -07:00
Euan Kemp
537f2900ad build_torcx_store: add package 'url' location
This also moves the 'torcx' prefixing logic over to the torcx upload
root introduced in the release util library.

It also corrects a bug in how the source package was being determined.
2017-09-08 17:20:22 -07:00
Euan Kemp
e14cd0f68b build_library: add torcx upload/download roots
Torcx is special in that it wishes to be uploaded under a prefixed
directory (torcx), typically wishes to be downloaded from there, but
ultimately wants to be downloaded from a location without that prefix.

In fact, I expect during a normal release process, it will be uploaded
with that prefix to the build bucket, copied without that prefix to the
final bucket (during pre-release), and then finally downloaded without
the prefix.

I think this set of variables ends up being the cleanest way to
represent this complexity.
2017-09-08 17:20:22 -07:00
Euan Kemp
a94f5ab328 build_torcx_store: generate a torcx_manifest
This modifies the `build_torcx_store` script to produce a manifest and
cas-like structure of packages referenced by that manifest.

It also removes the symlink creation logic (which will be re-added in
build_image in coming commits).

The concept of "extra packages", which are referenced in the manifest,
but aren't installed in the rootfs, is also introduced.

Since the logic of what to include in the rootfs is also extracted into
build_image, supporting these "extra packages" isn't very complicated
for this file.
2017-09-08 17:20:22 -07:00
Euan Kemp
3ae894783b build_library: add torcx_manifest helper
This implements some basic functionality around working with torcx
manifests.
2017-09-08 17:20:22 -07:00
David Michael
8f85212687 Merge pull request #732 from dm0-/docker
build_torcx_store: Bump the Docker image to 17.06
2017-08-24 14:41:55 -07:00
David Michael
219d7558b1 build_torcx_store: Bump the Docker image to 17.06 2017-08-22 16:40:25 -07:00
Andrew Jeddeloh
8b0106a6f3 Merge pull request #731 from ajeddeloh/quiet-torcx
build_torcx_store: emerge quietly
2017-08-18 16:19:59 -07:00
Andrew Jeddeloh
860f878634 build_torcx_store: emerge quietly
Add --quiet so build_torcx_store doesn't flood the terminal.
2017-08-16 15:36:05 -07:00
David Michael
6dd9a75d63 Merge pull request #725 from dm0-/perl
Revert "update_chroot: fix SDK updates during the Perl 5.24 upgrade"
2017-08-14 14:46:58 -07:00
David Michael
3c24b28e35 Revert "update_chroot: fix SDK updates during the Perl 5.24 upgrade"
This reverts commit 6508cf3faaef0643e6ece9124048591c7435de4f.

All update channels are now using Perl 5.24, so workarounds are no
longer required.
2017-08-14 14:07:28 -07:00
Benjamin Gilbert
91f9af2f9e Merge pull request #722 from bgilbert/oracle
oem/oracle: Add some scripts to manage images/instances
2017-08-14 14:04:05 -07:00
Benjamin Gilbert
68a163ac52 oem/oracle: Update for oraclebmc-cli 2.4.6
bmcs compute image create --image-source-details is deprecated.
2017-08-10 23:22:53 -07:00
David Michael
7f99054c9d Merge pull request #687 from dm0-/workon-mantle
enter_chroot: initialize with mantle in cros_workon
2017-08-10 17:39:45 -07:00
Benjamin Gilbert
1ea637794c oem/oracle: Add option to add kernel arguments to iPXE script 2017-08-08 15:00:32 -07:00
Benjamin Gilbert
41aa818f0c oem/oracle: Add some scripts to manage images/instances 2017-08-08 13:59:21 -07:00
David Michael
2ebd6c0ec4 Merge pull request #721 from glevand/for-merge-qemu
build_library: Update qemu script to run on arm64 machines
2017-07-28 15:02:48 -07:00
Geoff Levand
e76030e0e9 build_library: Update qemu script to run on arm64 machines
Signed-off-by: Geoff Levand <geoff@infradead.org>
2017-07-28 14:39:53 -07:00
Geoff Levand
5e94a55069 build_library: Add missing die function
Signed-off-by: Geoff Levand <geoff@infradead.org>
2017-07-28 14:39:53 -07:00
David Michael
8817fdb02a Merge pull request #720 from glevand/for-merge-edk2
vm_image_util: Update to use edk2 package
2017-07-27 16:16:20 -07:00
David Michael
71ebc66065 build_library: avoid a SIGPIPE while building arm64 UEFI firmware
This just sets the code file size to the var file size, so it gets
zero-padding without having to pipe commands together.

From: David Michael <david.michael@coreos.com>
[Rebased]
Signed-off-by: Geoff Levand <geoff@infradead.org>
2017-07-27 15:03:58 -07:00
Geoff Levand
1761d9d071 vm_image_util: Update to use edk2 package
Signed-off-by: Geoff Levand <geoff@infradead.org>
2017-07-27 14:51:32 -07:00
Euan Kemp
8a84488da9 Merge pull request #719 from euank/update_ebuilds-git
update_ebuilds: replace cvs with git
2017-07-26 17:15:11 -07:00
Euan Kemp
ac65e78e0f update_ebuilds: replace cvs with git
And default to git instead of rsync.

git has no rate limiting and will generally be quicker after the first
run.

This does leave a bit of extra data in your local portage-stable `.git`
directory, but it doesn't seem unreasonable to me.

Note: this means we lose the "ChangeLog" file. In the rsync
repositories, that file has been generated by egencache, but the git
repository never has it checked in.
2017-07-26 17:05:43 -07:00
David Michael
1ff1a8cf7e Merge pull request #718 from dm0-/python3
Silence a Python 3 script warning
2017-07-26 13:13:52 -07:00
David Michael
963fc72f0e check_root: ignore cgi.py in any Python version 2017-07-26 13:04:55 -07:00
Euan Kemp
50ef678e25 Merge pull request #712 from euank/fixup-core-sign-update
core_sign_update: fix flag parsing for keys_separator
2017-07-18 18:30:48 -07:00
Euan Kemp
e174703778 core_sign_update: fix flag parsing for keys_separator
Introduced in #710, whoops.
2017-07-18 18:24:38 -07:00
Euan Kemp
cdf275cd3e Merge pull request #710 from euank/coreos-card_signing
coreos_sign_update: Use smartcards for signing (updated)
2017-07-18 11:35:35 -07:00
Euan Kemp
61be39c999 core_sign_update: remain compatible with older sign.sh
The motivation behind retaining this backwards compatibility, at least
now, is that it's actually non-trivial to revert these code changes for
a given release.

The `sign.sh` changes can easily be changed, but the `core_sign_update`
code is included in the update-specific "au_zip" file. Replacing that is
a little more fiddly.

Since it's possible we'll still want to revert to the previous signing
behavior, make it so the update payload (namely core_sign_update) should
work both under the previous `sign.sh` script, and when using the new
one.
2017-07-11 13:55:59 -07:00