Commit Graph

150 Commits

Author SHA1 Message Date
Brandon Philips
69803d5ad6 feat(create_legacy_bootloader): add hvc0 on Xen
add console=hvc0 for systems that use pygrub/pvgrub by adding it to the
menu.lst files.
2013-07-22 17:37:51 -07:00
Brandon Philips
a923365449 chore(build_library/create_legacy_bootloader): remove old stuff
We don't need all of these args. Remove them.
2013-07-22 17:36:17 -07:00
Brandon Philips
95b2a4b8cd chore(*): use coreos_ everywhere 2013-07-21 23:20:50 -07:00
Brandon Philips
836ffc9327 feat(build_library): reintroduce A/B menu.lst
reintroduce unique A/B menu.lsts to work around the kexec problems that
we have. Essentially instead of always using boot_kernel on pvgrub
systems use the A/B kernels installed at update time to the boot
partition.
2013-07-21 22:30:46 -07:00
Michael Marineau
99089076af fix(build_image): Don't unmount when the rootfs isn't mounted
This function is never called when rootfs is mounted but leaving in a
check for it as a just in case sort of thing.
2013-07-19 02:41:52 -04:00
Michael Marineau
7564371200 hack(build_image): Make check for /usr/share/locale a warning.
Switching the toolchain to upstream Gentoo brought this directory back
and based on the Chromium OS history keeping this directory out of the
builds is a bit tedious. Keeping image sizes down isn't *that* important
right now so just let it be.
2013-07-18 10:38:16 -07:00
Brandon Philips
7ea2b4924f feat(boot_kernel): add support for the boot kernel
This adds the boot_kernel to the build boot partiton and updates the
relevant config files. Mission accomplished.

TODO: Update the installer to not worry about moving files around
anymore
2013-07-17 18:02:27 -07:00
Brandon Philips
ed037ea05c chore(build_library): remove efi image
we don't really care about booting on pure efi systems right now and I don't
think this all works so remove it for now.
2013-07-17 18:02:02 -07:00
Brandon Philips
06444a4ffc chore(build_library): strip out verity logic
remove some unused verity variables.
2013-07-17 18:00:00 -07:00
Michael Marineau
6b336e17b2 fix(bootstrap_sdk): Move gsutil code to release_util.sh
Sync up bootstrap_sdk with other tools by using the common upload
functions. As part of this refactor release_util a bit to provide a
truly generic upload function.
2013-07-17 19:25:46 -04:00
Brandon Philips
7cc09c349b feat(disk_layout): increase OEM size
increase the size of the OEM partition to 128M
2013-07-17 16:19:30 -07:00
Brandon Philips
938e0132d1 feat(legacy_disk_layout): increase boot metadata
increase the boot metadata size to 64M
2013-07-17 16:18:41 -07:00
Michael Marineau
ddb92a0887 feat(build_library): Add --upload_path option to override default
This will be used to upload the latest images built from master, we
don't need every build so we just want to upload to a 'master'
directory, not one named for the current version.
2013-07-10 19:44:40 -04:00
Michael Marineau
ef7aea86a9 fix(generate_au_zip): Add core-admin to au-generator.zip
Forgot to add this in previous commits, so here it is!
2013-07-08 21:54:57 -07:00
Michael Marineau
c9a6956f6f fix(core_upload_update): Add to au-generator.zip, move some operations.
Add core_upload_update to au-generator.zip which requires some extra
logic to make it runnable anywhere it may be. To organize the code a
little better all the delta_generator calls have been moved to
cros_generate_update_payload. core_upload_update is now just a wrapper
around cros_generate_update_payload and core-admin.
2013-07-08 21:39:05 -07:00
Michael Marineau
12935743b6 fix(common.sh): Fallback to loading version.txt from current directory.
This makes version handling continue to work normally with
au-generator.zip
2013-07-08 21:39:04 -07:00
Michael Marineau
12149d8421 fix(generate_au_zip): Remove unused scripts from au-generator.zip
We have no need to ship convert_recovery_to_ssd.sh with builds.
2013-07-08 21:39:04 -07:00
Don Garrett
048b3cecdf Update generate_au_zip.py location for cgpt.
Cgpt was moved and a symlink based wrapper was added. That wrapper will
be improved soon, when when that's true we'll need to change this back.

A specific note... cgpt is currently statically linked. If that wrapper does
not remain statically linked, then a simple revert won't be enough.

BUG=chromium-os:39814
TEST=Manual au-generate.zip creation.

Change-Id: I2705b1eddd8ef28c7eb099512513daf80f586218
Reviewed-on: https://gerrit.chromium.org/gerrit/45128
Reviewed-by: Chris Sosa <sosa@chromium.org>
Commit-Queue: Don Garrett <dgarrett@chromium.org>
Tested-by: Don Garrett <dgarrett@chromium.org>
2013-07-08 21:35:36 -07:00
Michael Marineau
3fa29024e1 fix(build_image): Configure legacy bootloaders to set root by UUID.
Make use of the new partition UUIDs for ROOT-A and ROOT-B in the root=
kernel parameters provided by the legacy (non-kexec) bootloaders. This
makes all of our images bootable as-is without having to pass them
through image_to_vm.sh. :-D
2013-07-08 16:28:12 -04:00
Michael Marineau
21b9f711a4 feat(cgpt): Statically allocate UUIDs for ROOT partitions.
Before we can switch from using device names in root= to partition table
UUIDs we need some values that will remain consistent across upgrades
since the partition table is not updated when filesystems are.
2013-07-08 15:49:39 -04:00
Brandon Philips
31cd4c47a9 fix(build_library/legacy_disk_layout): Use coreos- prefixes
vboot_reference now recognizes coreos-reserved and coreos-rootfs. Use
these prefixes so we stop using the chromeos GUIDs.

Test-plan: Tested on a VM and it boots and updates.
2013-07-07 21:18:49 -07:00
Michael Marineau
b4252985f2 feat(build_image): Add upload support for official builds 2013-07-05 23:55:50 -04:00
Michael Marineau
34bbdc1996 fix(build_packages): Move gsutil code from build_packages to a library.
To avoid making the same gsutil changes in more scripts move the code to
a library file where some of the basic parts can be shared.
2013-07-05 23:54:23 -04:00
Brandon Philips
a45c529549 feat(build*): add CoreOS production image building
This will create a CoreOS production image and support it with the tools
like image_to_vm.sh and build_image.
2013-07-05 12:06:40 -07:00
Brandon Philips
195d052495 fix(*): drop FACTORY images
we don't need factory image functionality. Drop it to simplify our
scripts and prepare for our "production" feature and flags.
2013-07-05 09:37:38 -07:00
Michael Marineau
2482291e7d fix(scripts): Cleanup build version handling.
A few things here:
 - Source manifests/version.txt directly instead of coreos-version.sh
 - Remove Chrome branch from target image directory names.
 - Use proper version instead of timestap for catalyst builds.
 - Move lsb_release script from coreos-overlay to build_library.
2013-07-02 16:12:02 -04:00
Michael Marineau
31dbe34f10 feat(build_image): Record directories installed in stateful_partition
During builds var_overlay is always mounted over /var. We want to do the
same at run time but we also want to ensure everything expected to be
there always does. After emerge completes gen_tmpfiles.py will scan /var
for any .keep files that were installed and records their parent
directories' permissions and ownership to /usr/lib/tmpfiles.d. On each
boot systemd will automatically recreate anything that goes missing.

This also means that going forward any ebuild that needs a directory in
/var (or anywhere else the stateful partition is bound) can simply rely
on the 'keepdir' ebuild function instead of adding things to
coreos_startup.
2013-06-06 14:18:52 -04:00
Brandon Philips
4057d5590d feat(disk_layout): follow new CoreOS partition layout
As outlined here we need a new partition layout, this patch makes the
necessary changes:
https://groups.google.com/forum/#!topic/coreos-dev/bA7gwGGoTng

The first big change is making all of the scripts obey partition numbers
based on labels in the disk_layout.json. This makes it much easier to
change later on.

The second big change is in the layout itself. The json file was updated
to reflect the document above.

And finally the grub boot configuration needed for pv-grub and pygrub
were added to the create_legacy_bootloader_templates.sh library utlity.

Everything seems to work and boot now.
2013-05-12 12:31:03 -07:00
Brandon Philips
edf40e1863 fix(build_library/legacy_bootloader): add menu.lst entries
these entries are needed for pvgrub, pygrub. Add them!
2013-05-03 18:17:21 -07:00
Brandon Philips
386ac79bd6 feat(build_library/container): more fixes
this sets up the container to a reasonable level. I stopped using this
container tool after I the dev server got up and running so YMMV.
2013-04-30 18:39:09 -07:00
Brandon Philips
6bb635d51c hack(build_container): add a script to build containers
this is a bit of a hack but I wanted to see if it had any utility during
development before making it all pretty. Essentially this is a copy of
build_image but instead of building up an entire image it simply puts
the files into directories on disk to be ran with systemd-nspawn/lxc/etc
2013-03-11 18:55:27 -07:00
Brandon Philips
7d9acd0de1 Revert "fix(dev_util): restore removed code from dev_util"
so it is a bit complicated but essentially gtest pulls in python which
pulls in pyton-updater which wants portage so portage gets installed in
teh real root not the dev one. Just leave it for now.
2013-03-07 11:32:19 -08:00
Brandon Philips
8a60480ff2 fix(dev_util): restore removed code from dev_util
93ed764 accidently removed some code, restore it.
2013-02-27 18:50:29 -08:00
Brandon Philips
93ed76460b feat(build_library): let python into root image
python allows systemd-analyze to work and gives a full portage stack.
Just let it in.
2013-02-26 08:00:40 -08:00
Brandon Philips
9f544e77a2 fix(build_library): cleanup kernel cmdline
don't be quiet, talk on tty0, and remove unneeded args

Change-Id: I6d49d0b119528ecb8082d38e77066691039092cb
2013-02-20 09:32:07 -08:00
Brandon Philips
f40133ceac hack(build_library/base_image_util.sh): don't zero fs
the math is wrong here as a build will often fail with "disk full". Fix
this.

Change-Id: Ib812991b584816862f0c72aeec79bebf37e214ba
2013-02-16 11:29:40 -08:00
Brandon Philips
a85c3f9167 fix(build_library/test_image_content.sh): Remove Xorg requirement
Remove Xorg and chrome, we don't ship those in coreos

Change-Id: Id3ffadda31c31e0d6afdc18738d2ed46fc41b537
2013-02-15 06:59:58 -08:00
Brandon Philips
78e2afafea fix(build_library/disk_layout_util.sh): unmount rootfs
Sometimes during the buildprocess we get here and the rootfs is still mounted?!

Change-Id: I6dc044d9d1318be606c1645f925421fa84c1f513
2013-02-14 16:26:48 -08:00
Brandon Philips
005de74765 fix(build_library/dev_image_util): use coreos
remove dev-init stuff for now and use coreos

Change-Id: I3f55550d197e3045eb26ea46fb9792d7c35dd5ab
2013-02-14 16:18:57 -08:00
Brandon Philips
9191f32d77 fix(test_build_root): fix tests for coreos
remove chrome and Xorg test

Change-Id: I6fe575954a8110c6bd94cbe7874b2cbea90def45
2013-02-14 16:18:57 -08:00
Brandon Philips
11472e5166 common: make safe_umount retry a few times
on Fedora 18 on Gnome 3.0 something is making the first attempt at
unmounting return busy. Unfortunatly, the return code is 32 everytime
so we have to parse the output of umount :( :( :(

Change-Id: I7f94bf6c2059c7e7cb4fb173d9ffbabd59f2b24f
2013-02-14 16:18:56 -08:00
Denis Glotov
94b00acd95 Revert "Revert "Extend STATEfull partition to 3Gb in 2gb-rootfs-updatable disk layout.""
This reverts commit 285b5338b8

Change-Id: I19f8238cb55f88b358244c09c32841ad9f79dd3f
Reviewed-on: https://gerrit.chromium.org/gerrit/42457
Tested-by: Denis Glotov <glotov@chromium.org>
Reviewed-by: Vincent Palatin <vpalatin@chromium.org>
Commit-Queue: Denis Glotov <glotov@chromium.org>
2013-02-01 12:00:17 -08:00
Don Garrett
7b69fb2525 Reorder rootfs partitions to be spinning disk friendly.
Change the order of partitions to given better performance for spinning disks.
However, this change affects all devices, not just those with spinning disks.

We will be able to remove the per-board overrides from several devices if/
when this change is accepted.

BUG=chromium-os:38122
TEST=image_to_live, (Will run trybots, and perform a recovery).

Change-Id: Ibe60e384e0392936f9f8ceb326618951367974df
Reviewed-on: https://gerrit.chromium.org/gerrit/41792
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: Don Garrett <dgarrett@chromium.org>
Commit-Queue: Don Garrett <dgarrett@chromium.org>
2013-02-01 11:39:24 -08:00
Simran Basi
285b5338b8 Revert "Extend STATEfull partition to 3Gb in 2gb-rootfs-updatable disk layout."
Suspecting this CL is killing devices in the lab: crosbug.com/38483

This reverts commit 38fe561a92

Change-Id: I6a6f0c170d5410de79ad0d2b96c7b9c95046cae4
Reviewed-on: https://gerrit.chromium.org/gerrit/42452
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Tested-by: Simran Basi <sbasi@chromium.org>
2013-02-01 09:03:45 -08:00
Denis Glotov
38fe561a92 Extend STATEfull partition to 3Gb in 2gb-rootfs-updatable disk layout.
This is needed for vmtests to have enough space, because default size,
although expandable, is too small in vm images.

BUG=chromium-os:38088
TEST=cbuildbot x86-generic-asan amd-64-generic-asan --remote -g I07d31d8b -gI18fab01e -gI9f55fcd3

Change-Id: I9f55fcd3707859dca8304b8e0c78ebd39afea566
Reviewed-on: https://gerrit.chromium.org/gerrit/42390
Tested-by: Denis Glotov <glotov@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>
Commit-Queue: Denis Glotov <glotov@chromium.org>
2013-02-01 02:33:54 -08:00
Denis Glotov
61772d46aa Add 2gb-rootfs-updateble disk layout.
This is to be used with build_image and image_to_vm by cros buildbots
that need more rootfs size.

Unlike 2gb-rootfs, it has both ROOT-A and ROOT-B enlarged to 2gb, so
that image update test (1_update in SimpleTestsUpdateAndVerify) is possible.

BUG=chromium-os:38088
TEST=cbuildbot x86-generic-asan amd-64-generic-asan --remote -g I6f497214 -g I07d31d8b

Change-Id: I6f497214f54f4457cf2b40610beb9272622d1a47
Reviewed-on: https://gerrit.chromium.org/gerrit/41814
Tested-by: Denis Glotov <glotov@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>
Commit-Queue: Denis Glotov <glotov@chromium.org>
2013-01-28 08:16:28 -08:00
Mike Frysinger
79ae5b4a06 use new status=none with dd
The latest coreutils introduces a status=none option which suppresses
all information which is more than the current status=noxfer.

BUG=None
TEST=`./build_image` no longer spews any dd info

Change-Id: I3cfefed1d38e7e5ff52342c98e6a306d9a48950a
Reviewed-on: https://gerrit.chromium.org/gerrit/41930
Reviewed-by: David James <davidjames@chromium.org>
Commit-Queue: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
2013-01-24 13:58:27 -08:00
Paul Taysom
5e39bb6cd1 Configure the serial port for debugging
Enable the serial port from the build_image command line.
Currenly, you have to edit build_kernel_image.sh to enable
the serial port for kernel debugging. Now:

./build_image --board=${BOARD} --enable_serial=ttyS0

Will enable sending printks to the specified serial port.

BUG=chromium-os:38026
TEST=built images with and without serial enabled

Change-Id: I9ef4f2a20f0d451e132371339c4eba1faf4c94de
Reviewed-on: https://gerrit.chromium.org/gerrit/41638
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Queue: Paul Taysom <taysom@chromium.org>
Tested-by: Paul Taysom <taysom@chromium.org>
2013-01-23 17:44:43 -08:00
Mike Frysinger
baae8eb628 common.sh: convert to bash
We don't need to support POSIX shell, so convert to bash.

BUG=None
TEST=`cbuildbot {amd64,x86,arm}-generic-full chromiumos-sdk daisy-release` worked

Change-Id: I33ad25d2310c593f3e346d955e3aa27da41091fc
Reviewed-on: https://gerrit.chromium.org/gerrit/41271
Reviewed-by: David James <davidjames@chromium.org>
Commit-Queue: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
2013-01-16 13:59:21 -08:00
Gilad Arnold
8f861c05b6 common.sh: limit pv progress bar initial width to 80 characters
By default, pv automatically infers and uses the full width of the
terminal. This generally makes sense for a console application, but in
the case of pv it just causes the progress bar to be ridiculously wide
when run on wide terminals, to the point it's hard to read. This CL is
setting it to 80 characters, the widely accepted standard width for
a terminal, in cases where the terminal appears to be larger than
80 columns. Note that:

* Even with -w, pv appears to be resizing the progress bar as the
  terminal width changes midway through the run. This means that if
  a user widens the window, then the progress bar will go wide again and
  there's nothing to be done about it.

* Theoretically, in very rare cases this may lead to a progress bar the
  exceeds the width of the terminal (i.e. set to 80 columns on
  a terminal that has just shrunk to fewer columns). The odds for such
  timing are close to nil and even then the damage is minimal.

* This will work for non-terminal runs, or otherwise runs where stty
  does not produce any output.

* To avoid the initialization overhead for all common.sh inclusion,
  replacing the variable with a function that prints the pv/cat command.

BUG=None
TEST=Ran ./image_to_usb on wide and narrow terminal windows, it works.

Change-Id: I549df1dd29e93909ea646ae9b9e09d9a588ad382
Reviewed-on: https://gerrit.chromium.org/gerrit/40937
Commit-Queue: Gilad Arnold <garnold@chromium.org>
Reviewed-by: Gilad Arnold <garnold@chromium.org>
Tested-by: Gilad Arnold <garnold@chromium.org>
2013-01-12 23:06:31 -08:00