4047 Commits

Author SHA1 Message Date
Euan Kemp
7e5906ceeb build_torcx_store: back-out reproducible digests
Per the comment there, they were implemented in a broken fashion.

This leaves the door open for using them in the future, but in the
meanwhile simply uses the sha512sum as the digest (which solves the
immediate issue).
2017-09-08 17:20:22 -07:00
Euan Kemp
a9fcf77818 build_torcx_store: upload manifests to upload_root
Rather than to a location special to them.
The comment in the code explains the reasoning for this.
2017-09-08 17:20:22 -07:00
Euan Kemp
c5594a39db jenkins/images: use torcx packages from manifest
This accompanies the other changes around moving to a torcx manifest.
2017-09-08 17:20:22 -07:00
Euan Kemp
6e567733d0 jenkins/packages: configure download root
Pending jenkins also including this variable.
2017-09-08 17:20:22 -07:00
Euan Kemp
61c2cc7015 build_torcx_store: upload to 'pkgs'
This is consistent with the local directory hierarchy
2017-09-08 17:20:22 -07:00
Euan Kemp
7e0324b069 build_image: populate torcx store from manifest
This moves the default symlinking logic into build image as well.

This assumes that a torcx store is available locally with all images
referenced in the torcx manifest.

This is accomplished with a highly-indented double-for-loop, but I think
it's still decently readable.
2017-09-08 17:20:22 -07:00
Euan Kemp
7a72d707d2 build_library: add more torcx_manifest helpers
These are useful for the build_image_util code soon to come.
2017-09-08 17:20:22 -07:00
Euan Kemp
537f2900ad build_torcx_store: add package 'url' location
This also moves the 'torcx' prefixing logic over to the torcx upload
root introduced in the release util library.

It also corrects a bug in how the source package was being determined.
2017-09-08 17:20:22 -07:00
Euan Kemp
e14cd0f68b build_library: add torcx upload/download roots
Torcx is special in that it wishes to be uploaded under a prefixed
directory (torcx), typically wishes to be downloaded from there, but
ultimately wants to be downloaded from a location without that prefix.

In fact, I expect during a normal release process, it will be uploaded
with that prefix to the build bucket, copied without that prefix to the
final bucket (during pre-release), and then finally downloaded without
the prefix.

I think this set of variables ends up being the cleanest way to
represent this complexity.
2017-09-08 17:20:22 -07:00
Euan Kemp
a94f5ab328 build_torcx_store: generate a torcx_manifest
This modifies the `build_torcx_store` script to produce a manifest and
cas-like structure of packages referenced by that manifest.

It also removes the symlink creation logic (which will be re-added in
build_image in coming commits).

The concept of "extra packages", which are referenced in the manifest,
but aren't installed in the rootfs, is also introduced.

Since the logic of what to include in the rootfs is also extracted into
build_image, supporting these "extra packages" isn't very complicated
for this file.
2017-09-08 17:20:22 -07:00
Euan Kemp
3ae894783b build_library: add torcx_manifest helper
This implements some basic functionality around working with torcx
manifests.
2017-09-08 17:20:22 -07:00
David Michael
8f85212687 Merge pull request #732 from dm0-/docker
build_torcx_store: Bump the Docker image to 17.06
2017-08-24 14:41:55 -07:00
David Michael
219d7558b1 build_torcx_store: Bump the Docker image to 17.06 2017-08-22 16:40:25 -07:00
Andrew Jeddeloh
8b0106a6f3 Merge pull request #731 from ajeddeloh/quiet-torcx
build_torcx_store: emerge quietly
2017-08-18 16:19:59 -07:00
Andrew Jeddeloh
860f878634 build_torcx_store: emerge quietly
Add --quiet so build_torcx_store doesn't flood the terminal.
2017-08-16 15:36:05 -07:00
David Michael
6dd9a75d63 Merge pull request #725 from dm0-/perl
Revert "update_chroot: fix SDK updates during the Perl 5.24 upgrade"
2017-08-14 14:46:58 -07:00
David Michael
3c24b28e35 Revert "update_chroot: fix SDK updates during the Perl 5.24 upgrade"
This reverts commit 6508cf3faaef0643e6ece9124048591c7435de4f.

All update channels are now using Perl 5.24, so workarounds are no
longer required.
2017-08-14 14:07:28 -07:00
Benjamin Gilbert
91f9af2f9e Merge pull request #722 from bgilbert/oracle
oem/oracle: Add some scripts to manage images/instances
2017-08-14 14:04:05 -07:00
Benjamin Gilbert
68a163ac52 oem/oracle: Update for oraclebmc-cli 2.4.6
bmcs compute image create --image-source-details is deprecated.
2017-08-10 23:22:53 -07:00
David Michael
7f99054c9d Merge pull request #687 from dm0-/workon-mantle
enter_chroot: initialize with mantle in cros_workon
2017-08-10 17:39:45 -07:00
Benjamin Gilbert
1ea637794c oem/oracle: Add option to add kernel arguments to iPXE script 2017-08-08 15:00:32 -07:00
Benjamin Gilbert
41aa818f0c oem/oracle: Add some scripts to manage images/instances 2017-08-08 13:59:21 -07:00
David Michael
2ebd6c0ec4 Merge pull request #721 from glevand/for-merge-qemu
build_library: Update qemu script to run on arm64 machines
2017-07-28 15:02:48 -07:00
Geoff Levand
e76030e0e9 build_library: Update qemu script to run on arm64 machines
Signed-off-by: Geoff Levand <geoff@infradead.org>
2017-07-28 14:39:53 -07:00
Geoff Levand
5e94a55069 build_library: Add missing die function
Signed-off-by: Geoff Levand <geoff@infradead.org>
2017-07-28 14:39:53 -07:00
David Michael
8817fdb02a Merge pull request #720 from glevand/for-merge-edk2
vm_image_util: Update to use edk2 package
2017-07-27 16:16:20 -07:00
David Michael
71ebc66065 build_library: avoid a SIGPIPE while building arm64 UEFI firmware
This just sets the code file size to the var file size, so it gets
zero-padding without having to pipe commands together.

From: David Michael <david.michael@coreos.com>
[Rebased]
Signed-off-by: Geoff Levand <geoff@infradead.org>
2017-07-27 15:03:58 -07:00
Geoff Levand
1761d9d071 vm_image_util: Update to use edk2 package
Signed-off-by: Geoff Levand <geoff@infradead.org>
2017-07-27 14:51:32 -07:00
Euan Kemp
8a84488da9 Merge pull request #719 from euank/update_ebuilds-git
update_ebuilds: replace cvs with git
2017-07-26 17:15:11 -07:00
Euan Kemp
ac65e78e0f update_ebuilds: replace cvs with git
And default to git instead of rsync.

git has no rate limiting and will generally be quicker after the first
run.

This does leave a bit of extra data in your local portage-stable `.git`
directory, but it doesn't seem unreasonable to me.

Note: this means we lose the "ChangeLog" file. In the rsync
repositories, that file has been generated by egencache, but the git
repository never has it checked in.
2017-07-26 17:05:43 -07:00
David Michael
1ff1a8cf7e Merge pull request #718 from dm0-/python3
Silence a Python 3 script warning
2017-07-26 13:13:52 -07:00
David Michael
963fc72f0e check_root: ignore cgi.py in any Python version 2017-07-26 13:04:55 -07:00
Euan Kemp
50ef678e25 Merge pull request #712 from euank/fixup-core-sign-update
core_sign_update: fix flag parsing for keys_separator
2017-07-18 18:30:48 -07:00
Euan Kemp
e174703778 core_sign_update: fix flag parsing for keys_separator
Introduced in #710, whoops.
2017-07-18 18:24:38 -07:00
Euan Kemp
cdf275cd3e Merge pull request #710 from euank/coreos-card_signing
coreos_sign_update: Use smartcards for signing (updated)
2017-07-18 11:35:35 -07:00
Euan Kemp
61be39c999 core_sign_update: remain compatible with older sign.sh
The motivation behind retaining this backwards compatibility, at least
now, is that it's actually non-trivial to revert these code changes for
a given release.

The `sign.sh` changes can easily be changed, but the `core_sign_update`
code is included in the update-specific "au_zip" file. Replacing that is
a little more fiddly.

Since it's possible we'll still want to revert to the previous signing
behavior, make it so the update payload (namely core_sign_update) should
work both under the previous `sign.sh` script, and when using the new
one.
2017-07-11 13:55:59 -07:00
Euan Kemp
5cbc755abc offline_signing: use a smartcard URI 2017-07-06 13:50:27 -07:00
Euan Kemp
2146975588 coreos_sign_update: return 'legacy' signing support
We currently sign with both a devel key and a prod key. The devel key is
insecure and need not be included on a smartcard, so it makes sense to
leave it be on disk.

However, the previous commit's padding changes removed this legacy
method of signing.
For simplicity, simply re-introduce the old logic conditionally based on
whether it's a smartcard or not.

Alternate options could be using `-pkcs` instead of `-raw` for both
keys, but that is a more intricate change I'd be less confident in
making.
2017-07-06 13:50:27 -07:00
Matthew Garrett
54048fbb00 coreos_sign_update: Use smartcards for signing
Sign updates using private keys on smartcards. This involves changing the
padding approach - rather than including the padding in the hash, ask the
card to generate the padding itself, since the card will refuse to sign
pre-padded material. Use + as a key separator rather than : as the PKCS#11
URI includes colons.
2017-07-06 13:50:27 -07:00
David Michael
2447debece Merge pull request #705 from dm0-/jenkins
Add Jenkins scripts
2017-06-30 17:04:46 -07:00
Benjamin Gilbert
c47ef878ef Merge pull request #706 from bgilbert/sdk_version
tag_release: Verify SDK version exists
2017-06-29 17:41:04 -07:00
Benjamin Gilbert
7e93698c9b tag_release: Verify SDK version exists
Some releases may omit the SDK for expediency.
2017-06-29 17:20:29 -07:00
David Michael
9256893e83 jenkins: Drop conditions not affecting this branch 2017-06-29 15:34:29 -07:00
David Michael
9cfed47be0 jenkins: Re-add Jenkins scripts to this repository
We are going to restore the split-script setup from the old Jenkins
server.  This ensures that the each version's release process is
actually running with scripts in the correct release branch.  It
also allows branching the VM format lists.

Note that the scripts added here only cover the currently active
jobs in the main build pipeline.  There is no reason to add other
jobs, since they are mostly just running a single command using a
mantle binary from its master branch.

The scripts in this repository pick up after Jenkins has set up an
environment with all parameters and credentials defined, and an SDK
was prepared and validated.
2017-06-29 15:16:54 -07:00
David Michael
336ac28026 Merge pull request #695 from dm0-/perl
Add workarounds for upgrading to Perl 5.24
2017-06-26 16:50:23 -07:00
Benjamin Gilbert
84db4c6f99 Merge pull request #700 from bgilbert/add-vagrant-vbox
build_library: Add vagrant_virtualbox image type
2017-06-22 13:45:28 -07:00
Benjamin Gilbert
e98efe074d build_library: Add vagrant_virtualbox image type 2017-06-22 13:41:15 -07:00
Benjamin Gilbert
7db2e77b0b Merge pull request #699 from crawford/vagrant
build_library: split vagrant and vagrant vbox
2017-06-22 13:38:33 -07:00
Alex Crawford
65986e02bc build_library: split vagrant and vagrant vbox
This reverts the vagrant image back to using oem-vagrant because we
don't want to break the existing images. It moves the new,
Ignition-powered virtualbox flavor of vagrant into a new image.
2017-06-22 13:23:50 -07:00
Benjamin Gilbert
ab4705f9dc Merge pull request #697 from bgilbert/generic
Drop ISO/PXE support for amd64-generic, and an unused variable
2017-06-21 13:28:55 -07:00