Commit Graph

291 Commits

Author SHA1 Message Date
Michael Marineau
4d3c198161 tools: remove support for parallel_emerge
Using parallel_emerge has been disabled by default for all commands
except build_image for quite a while now, build_image kept it just
because it was still a bit faster than normal emerge. Keeping
parallel_emerge complicates future changes to build_image so it needs to
drop it entirely. Since that means nothing uses it by default we might
as well just rip out support for it entirely.
2014-07-19 16:38:17 -07:00
Michael Marineau
51c78a4685 build_image: generate update tools zip by default
Missed this in 7231b95a, the update zip should still be built when the
usr partition is extracted for generating updates but build_image itself
is not generating and signing the update.
2014-06-24 14:01:09 -07:00
Michael Marineau
7231b95af1 updates: extract usr partition when building images
The current generate_update function is now less useful, the important
part that we need is just the partition image now. Also by defaulting to
extracting the partition the old cors_generate_update which is still in
use by devserver can be removed entirely, devserver will just expect the
extracted partition image instead.
2014-06-23 12:26:17 -07:00
Michael Marineau
388220cfad fix(build_image): move upload calls to build_library
This will let the image building code upload extra data without the
wrapper script having to know about it.
2014-06-08 16:37:48 -04:00
Michael Marineau
2e9911b978 fix(*): Migrate image storage to new buckets
- Automated builds drop SDK and binary packages into
  gs://builds.developer.core-os.net/ and the new download URL is
  http://builds.developer.core-os.net/ (COREOS_DEV_BUILDS)
- Change default upload path to gs://users.developer.core-os.net/ for
  misc developer builds. Official builds go elsewhere and will just be
  configured in buildbot/jenkins so some COREOS_OFFICIAL stuff is gone.
- Automated builds of images go to a private bucket,
  gs://builds.release.core-os.net which later gets copied to
  gs://alpha.release.core-os.net and friends by core_promote.
2014-05-21 13:21:07 -07:00
Michael Marineau
7372f9f6e6 Merge pull request #254 from marineam/dev-config
Build-time dev cloud configs
2014-05-18 20:46:07 -07:00
Michael Marineau
822644fc54 feat(build_image): Add support for injecting a cloud config
The new --developer_data option can be used to specify a path to a cloud
config to bundle into the image. If none is provided but a shared user
password (for core) is set then generate a config to set that password.
This lets us use the same mechanism for setting the default password for
both disk and PXE images.
2014-05-17 16:23:18 -07:00
Michael Marineau
b24df04465 feat(build_image): Add 'container' image type.
This image type is the same as the developer image except that it is a
single root filesystem and is bootable via systemd-nspawn. This may
become obsolete eventually when it becomes possible to boot the normal
disk images under nspawn but it is useful for testing until then.

The partition type is defined by the Discoverable Partitions Spec.
http://www.freedesktop.org/wiki/Specifications/DiscoverablePartitionsSpec/
2014-05-16 21:11:01 -07:00
Michael Marineau
56b550dc21 cleanup(build_image_util): Delete unused code 2014-05-09 11:18:19 -07:00
Michael Marineau
1c5393b78e refactor(build_image_util): Merge with base_image_util
There isn't a base image now, and less file clutter is nice.
2014-05-09 11:16:48 -07:00
Michael Marineau
1836d5998d refactor(base_image_util): Split into two functions.
Use what was the base image build function as setup/finalize steps in
the dev and prod build functions. This eliminates duplicate code
that mounted and unmounted the filesystem images.
2014-05-09 11:13:54 -07:00
Michael Marineau
39086358bf refactor(build_image): Build dev and prod images independently
We need some more control over exactly what lands in dev vs prod images
which will require letting them diverge in what is currently the common
base image step. There isn't any real need for the base image in the
first place other than to speed up building both dev and prod images at
the same time but that isn't common enough to worry about.

As part of this cleanup also remove references to CHROMEOS_* variables
and the recovery image that never actually existed in CoreOS.
2014-05-09 10:55:09 -07:00
Michael Marineau
9dd0a9d87f Merge pull request #245 from marineam/set-group
add(image_set_group): New script to set the update group in prod images.
2014-05-07 16:58:24 -07:00
Michael Marineau
a7e1e5a177 add(image_set_group): New script to set the update group in prod images.
For generating images for groups other than the one given to build_image
run this script along with the usual image_to_vm.sh commands. To avoid
ambiguity with the 'latest' symlink, this script creates $group-latest
symlinks instead. build_image creates the new symlink too.
2014-05-07 15:07:14 -07:00
Michael Marineau
b4d6fb9597 fix(set_lsb_release): Set default GROUP= from arg again.
This makes sure developer builds stick to the developer group. After
commit c3d07e94 developer PXE/ISO images would report as 'alpha'.
2014-05-07 12:52:05 -07:00
Michael Marineau
4815c38194 fix(tag_release): Fix compatibility with portage's config parser
Another fix for 10d98e7. Empty values must be quoted, otherwise the
tokenizer gets confused and everything falls apart. :(
2014-05-04 22:52:53 -07:00
Michael Marineau
fccefb278e Merge pull request #236 from marineam/vagrant
Vagrant related script updates
2014-05-03 14:18:39 -07:00
Michael Marineau
4e85b172df feat(build_image): Optionally generate update payload with prod images. 2014-05-01 09:59:58 -07:00
Michael Marineau
7a7494e999 fix(prod_image_util): Do not install update test key anymore.
Leave this to the ebuild, no reason for it to be done here.
2014-04-30 17:12:59 -07:00
Michael Marineau
10d98e7b32 fix(common): Switch to os-release style names in version.txt
The existing version.txt is kinda annoying. The common case of referring
to the current version requires joining three values and the names of
those values only make sense in ChromeOS. Instead just use version as a
string, using VERSION, VERSION_ID, and BUILD_ID just as they appear in
os-release. It is up to the few scripts that need the individual parts
to break the version apart.

The old values remain for the sake of compatibility.
2014-04-29 01:44:47 -04:00
Michael Marineau
7f5b5baf22 maint(build_library): Remove special case code for -usr disk layouts
More cleanup now that amd64-generic is gone.
2014-04-26 14:45:28 -07:00
Michael Marineau
2f8f0159f8 maint(*): Delete special case code for amd64-generic 2014-04-23 16:14:50 -07:00
Michael Marineau
add4519021 fix(build_image): Only use 'alpha' group in official builds. 2014-04-09 18:46:07 -07:00
Michael Marineau
934675d19b fix(build_image): Add auto-enter-chroot call
This makes build_image and image_to_vm behave like build_packages, where
if you call the script outside of the SDK's chroot it will automatically
reexecute itself inside of the chroot.
2014-03-23 20:17:05 -04:00
Michael Marineau
8244c7a260 cleanup(set_lsb_release): Rework release config setup
- Remove custom COREOS_* attributes from /etc/lsb-release
- Move dev image logic to dev_image_util
  For extra fun fix detection of local host URL for devserver.
- Remove weirdly verbose "DESCRIPTION" format.
- Add COREOS_RELEASE_BOARD back to /usr/share/coreos/release
  This is mostly just so update_engine and gmerge report the correct
  board name to devserver, informative-only on prod images.
- Remove version info from /etc/gentoo-release
- Switch from 'track' to 'group' terminology.
2014-03-19 15:56:21 -07:00
Michael Marineau
10025571d9 fix(common): Disable parallel_emerge by default for most commands.
I would like to phase out parallel_emerge so disable it for all commands
other than build_image which is the only one that shows a noticeable
benefit from it (~2 min with --fast, ~3 min with --nofast).
2014-02-17 12:42:22 -08:00
Michael Marineau
feb59db9f5 fix(release_util): Add argument to specify the name to use for .DIGESTS
For multi-file uploads we should explicitly declare what the name of the
.DIGESTS file should be instead of using the first file name. Relying on
the ordering was subtle and easy to break.
2014-01-29 16:30:03 -08:00
Michael Marineau
bfe47817e4 fix(build_image): Add support for COREOS_DISK_LAYOUT_SUFFIX
Append COREOS_DISK_LAYOUT_SUFFIX from the portage profile to disk layout
names. Allows switching from "base" to "base-usr" and similar.
2014-01-05 18:39:47 -08:00
Michael Marineau
e84f922ba5 fix(build_library): remove references to chromeos-common.sh
Nothing from chromeos-common.sh is needed for image building now. Also
kill off build_common.sh which was just a weird way of sourcing
common.sh. The two piddly functions it provided fit better in
build_image_util.sh
2014-01-05 18:39:47 -08:00
Michael Marineau
15eead682b cleanup(build_library): Remove old disk_layout_util.sh
All of this code can go away now.
2014-01-04 21:17:06 -08:00
Michael Marineau
213472652c add(prod_image_util): Rename what remains of cros_make_image_bootable
cros_make_image_bootable now only is relevant for prod images, so move
the remaining code to prod_image_util in a similar scheme that base and
dev images use.
2013-12-30 16:12:44 -08:00
Michael Marineau
306a2f6cbc fix(cros_make_image_bootable): Switch to disk_util mount/umount 2013-12-30 16:12:44 -08:00
Michael Marineau
52e1dba3ec fix(build_image): Migrate to new disk_util commands 2013-12-30 16:12:43 -08:00
Michael Marineau
9bc36ee344 fix(base_image_util): Remove dead code, leave rootfs writable.
The verification flag was being passed through to the bootloader
template script but no longer had any effect.
Force the base image to always remain writable, its only purpose is to
be modified in a later build step anyway.
2013-12-30 16:12:41 -08:00
Michael Marineau
576c8996f4 fix(disk_util): Remove support for --adjust_part
This isn't a feature we've been using as far as I know and if someone
needs a custom partition layout it's probably better to just add it to
the json file. Removing this avoids some complexity.
2013-12-30 16:12:02 -08:00
Michael Marineau
6d414d275e fix(*): remove lingering broken support for board overlays.
This is a feature from the chromeos sdk that we haven't used and doesn't
work any more. Clean it up...
2013-12-08 16:02:00 -08:00
Michael Marineau
6452857e82 fix(*): Change default for --jobs to $NUM_JOBS
This makes parallelism behavior between parallel_emerge and emerge
consistent. NUM_JOBS is defined as the number of CPUs by common.sh
2013-12-01 16:04:10 -08:00
Michael Marineau
c9c1efd276 fix(build_image): Add --fast to control use of parallel_emerge
This makes it possible to toggle parallel_emerge just as other scripts
do. In other scripts update the help string to be more specific, the
--jobs option can be used to control parallelism.
2013-12-01 16:04:10 -08:00
Michael Marineau
241d135126 fix(test_build_root): Switch to using test_image_content
Start using the more complete test_image_content test for the build root
and add the test to the end of build_packages.
2013-11-26 17:14:50 -08:00
Michael Marineau
41099eef6f fix(board_options.sh): Move CHOST->ARCH mapping code.
Use the complete chost to portage arch mapping list from crossdev and
move the logic to toolchain_util.sh where it can be used by other
scripts.
2013-11-07 13:28:13 -08:00
Michael Marineau
207cc2f6a3 cleanup(*): Remove unused/broken test and factory images 2013-09-19 19:04:54 -04:00
Michael Marineau
c86eb16098 fix(build_image): Remove defunct bootcache option.
From ChromeOS, not applicable to our systems.
2013-09-19 18:21:32 -04:00
Michael Marineau
325755e923 cleanup(build_image): Remove some unused flags.
Remove --verity_*: Unused, we don't support verity
Remove --usb_disk: Unused, we use PARTUUID now.
Remove --enable_serial: Unused, and serial is enabled for syslinux
2013-08-19 18:09:11 -04:00
Michael Marineau
2edd22024e fix(build_image): Remove noinitrd from default boot_args 2013-08-14 13:15:20 -04:00
Michael Marineau
7c48115da6 fix(image_to_vm): Use saved version info instead of parsing directory.
Trying to include version info by adding the directory name to VM image
names didn't work and a better solution is for build_image to write out
a version.txt file. This should also fix an issue where uploading from
image_to_vm.sh didn't always go to the same location as the images
uploaded from build_image did.
2013-07-31 21:22:52 -04:00
Michael Marineau
f46ee8c332 fix(build_image): Fix building dev and prod without base
Ever since adding prod it hasn't been possible to build just dev and
prod without explicitly building base. Base is always built but usually
there is no point to keeping it around. Add some logic to make dev not
conflict with prod and make sure base is deleted and not uploaded if it
wasn't explicitly requested.
2013-07-26 22:56:27 -04:00
Michael Marineau
cc0873bbd8 fix(set_lsb_release): Add option for production updates, misc updates.
Add --production_track argument to set_lsb_release and
cros_make_image_bootable to support using the production update service
on developer builds of the 'prod' image. This replaces the previous hack
of setting COREOS_OFFICIAL=1 in the middle of the build.

Since lsb-release doesn't exist prior to the first call to
set_lsb_release switch to sudo_clobber instead of append. That way if it
is called a second time later the contents aren't duplicated.

Write the info to gentoo-release and os-release as well so everything
gets the same information.
2013-07-25 18:14:51 -04:00
Brandon Philips
95b2a4b8cd chore(*): use coreos_ everywhere 2013-07-21 23:20:50 -07:00
Brandon Philips
4c637bbe8c feat(build_image): official lsb-release on prod images
The /etc/lsb-release should have official values on the prod image. That
way someone can ping the official api.core-os.net with this image.
2013-07-21 22:10:02 -07:00
Brandon Philips
f73cbac8db feat(build_image): install the au-key
Install the developer au key on production images. This will need to
change when we have the production key.
2013-07-20 16:24:13 -07:00