Commit Graph

53 Commits

Author SHA1 Message Date
Andrew Jeddeloh
15745d520e
disk_util: retry loopback mounts if they fail
This fails frequently but should succeed if retried. This should reduce
failed builds.
2020-01-21 13:29:45 +01:00
Andrew Jeddeloh
00d77d199a
build_library/disk_util: work around losetup bug
Retry losetup if it fails, up to 5 times with 5 seconds between retries.
2020-01-21 13:29:30 +01:00
Flatcar Buildbot
c7bbb2b1e2 2345.0.0 2019-12-04 14:59:11 +01:00
David Michael
4bd4cd5a8e disk_util: Add -I for mkfs.vfat
When loop device partition nodes aren't cleaned up, building images
will fail with:

mkfs.vfat: Partitions or virtual mappings on device '/dev/loop0', not making filesystem (use -I to override)

Just add the flag unconditionally to work around it.
2019-02-10 02:44:01 +00:00
Benjamin Gilbert
fdf8cc2cda Revert "Revert "disk_util: use FAT32 on ESP""
FAT32 seemed to aggravate https://github.com/coreos/bugs/issues/2284, but
now that that issue has been addressed, we can return to the correct
filesystem type.

This reverts commit 299f8fb3d1.
2018-04-12 17:00:28 -04:00
Andrew Jeddeloh
299f8fb3d1 Revert "disk_util: use FAT32 on ESP"
This reverts commit 7f058d61a1.

Reverting because of bug 2284 [1] where grub will sometimes fail due to
memory corruption. This is _not_ the cause of the bug, and the bug can
even be reproduced with this reversion, but it seems to occur less when
not using fat32.

[1] https://github.com/coreos/bugs/issues/2284
2017-12-18 13:36:56 -08:00
Benjamin Gilbert
7f058d61a1 disk_util: use FAT32 on ESP
mkfs.vfat was defaulting to FAT16 based on the size of the partition.
The UEFI spec (2.7 errata A, section 13.3) implies that only FAT32 is
necessarily supported on the ESP, and we've received a report of
hardware that doesn't recognize FAT16.
2017-11-13 11:27:31 -08:00
Benjamin Gilbert
b091c2d0c2 disk_util: drop support for syslinux partition feature
It hasn't been used since 5bfa0c8d20.
2017-11-11 14:39:02 -08:00
David Michael
08ed31d70d disk_util: differentiate between partition and FS labels 2017-01-18 16:21:19 -08:00
Alex Crawford
7232e74bcc disk_util: use the dummy disk UUID
This dummy UUID (00000000-0000-0000-0000-000000000001) is used to
signify to Ignition that the disk is uninitialized.
2015-06-30 20:04:41 -07:00
Alex Crawford
cf4b282482 disk_util: convert some tabs to spaces 2015-06-30 19:58:10 -07:00
George Tankersley
93f033c59e verity: add support for root hash output to disk_util 2015-06-11 18:37:51 -07:00
Michael Marineau
07fc135a12 disk_util: include fsck output during resize operation
Previously fsck output was suppressed to reduce the amount of noise in
build logs on the assumption that fsck really shouldn't have a reason to
fail. The filesystem is freshly created after all. However some users
have reported that fsck is failing but without error messages we don't
know why.
2015-03-25 14:16:15 -07:00
Michael Marineau
d09aeb368c disk_util: allocate the maximum number of usable inodes
This change changes the default 'bytes-per-inode' ration from 16K to 4K,
the block size. To prevent this from wasting too much space change the
inode size from the default 256 to the minimum size, 128. Larger inodes
are used to store extended attributes more efficiently but since we do
not use SELinux the majority of files do not have security attributes.

These defaults may be modified via the new `bytes_per_inode` and
`inode_size` options.
2015-02-11 17:12:18 -08:00
Michael Marineau
3587784bc4 disk_util: Add support for computing verity hashes 2014-11-15 18:58:10 -08:00
Michael Marineau
931610d5bb disk_util: Set priority and successful attributes in new images
Mark the initial copy of CoreOS as 'successful' and with a non-zero
priority. Required to boot with a stricter interpretation of the
partition selection scheme which ignores partitions that have a priority
of zero. The new grub implementation follows this rule and is what the
original ChromeOS spec used too.

For the sake of completeness if multiple partitions are configured in
the json file with this feature they will be prioritized in disk-order.
2014-11-15 16:01:25 -08:00
Michael Marineau
2f081e91c8 disk_util: ensure partitions are layed out on disk in order
So far the default iteration order of python dicts has mostly matched
the order that we want the partitions on disk but this is not always the
case. I caught the BIOS-BOOT partition being ordered on disk after the
USR-A partition. Nothing bad came of this but consistancy is good.
2014-11-03 13:07:29 -08:00
Michael Marineau
e77e4e5499 disk_layout: Align disk sizes to both 1MB and cylinder boundaries.
The VHD disk format internally includes CHS addressing and qemu-img
respectfully aligns disk images to the common 16 heads 63 sectors
geometry when possible. This is unfortunate since images uploaded to
Azure must also be aligned to 1MB we normally do.

Since qemu-img doesn't have a way to handle this well right now adjust
our existing alignment logic to create disk images aligned to both.
2014-10-08 20:40:45 -07:00
Michael Marineau
25b20b420c disk_util: support exposing a hybrid partition without syslinux
We don't need to do anything like manually install the MBR boot code
for grub but we do need to continue to expose the ESP partition as a
hybrid partition to support pvgrub.
2014-08-30 16:26:26 -07:00
Michael Marineau
fd8618336d disk_util: do not zero MBR and GPT when resizing disk
Calling cgpt create when resizing zeros the MBR boot code. This worked
with the syslinux setup because the boot code was re-written. When not
using syslinux it is easier to just preserve the existing MBR instead.
2014-08-30 16:00:07 -07:00
Michael Marineau
919ba5a3b9 fix(disk_util): Install syslinux with extlinux
Attempting to work around an apparent race in mtools, the command
'extlinux' these days is just the install tool for mounted partitions
while 'syslinux' is for unmounted devices.
2014-06-18 14:20:29 -07:00
Michael Marineau
02bd7351dd fix(disk_util): don't run blkid on empty partitions 2014-06-06 16:12:37 -07:00
Alex Crawford
17835eeb79 Merge pull request #263 from crawford/teeth
feat(teeth): Add new OEM and disk layout for teeth
2014-05-29 17:06:37 -07:00
Alex Crawford
ed50f5fe0b feat(disk_util): Replace Resize() with Update()
The new Update() performs the same tasks as the old Resize()
in addition to formatting previously-unformatted partitions. This
allows children disk-layouts to repartition the base layout in
addition to resizing.
2014-05-29 16:46:27 -07:00
Michael Marineau
d11815a68d fix(disk_layout): Bump up default ROOT volume sizes for btrfs.
btrfs isn't designed for small volumes and can run out of space sooner
than one would expect in our current setup, particularly with docker.
To try to improve the situation always create the filesystem initially
as 2GB instead of 512MB using the default settings: metadata is
duplicated, data is single, not mixed. The mixed setting may have been
partly why our performance can be so poor. For the default vm layout
use 6GB instead of 3GB, about what we use for EC2.
2014-05-23 18:22:02 -07:00
Michael Marineau
ee6b9ee1f9 fix(disk_util): Set btrfs default subvolume
This avoids needing to always pass the subvol=root mount option.
2014-05-16 19:09:23 -07:00
Michael Marineau
acb9800259 fix(disk_util): Don't try to format blank partitions. 2014-05-16 17:40:35 -07:00
Michael Marineau
e1d7b29436 feat(disk_util): Add support for extracting partitions.
This will simplify the process of generating update payloads.
2014-04-30 19:04:42 -07:00
Michael Marineau
7f5b5baf22 maint(build_library): Remove special case code for -usr disk layouts
More cleanup now that amd64-generic is gone.
2014-04-26 14:45:28 -07:00
Michael Marineau
476bae747e fix(disk_util): Switch back to gptmbr.bin for our MBR boot code.
Using the classic mbr.bin was only needed during the transition from
syslinux 3 to 6 because the behavior of gptmbr.bin changed after 3.
Now that the transition is done and cgpt supports the new scheme now it
is time we switched back. This avoids depending on using a hybrid MBR.
2014-04-25 11:15:17 -07:00
Michael Marineau
0c1198c3bb feat(disk_util): Switch from SYSLINUX's gptmbr.bin to mbr.bin
cgpt now supports generating hybrid MBRs and the classic style mbr.bin
from any version of SYSLINUX should work the same with the hybrid MBR.
The other code, gptmbr.bin, changes after SYSLINUX 3. Switching lets me
play with different versions of SYSLINUX without breaking everything.

With this change all images feature a hybrid MBR so the special case for
some VM platforms has been removed.
2014-03-12 15:38:34 -07:00
Michael Marineau
87c0814006 fix(disk_util): Fix undefined variable introduce in 4dbdb391 2014-03-06 12:41:07 -08:00
Michael Marineau
4dbdb391a4 feat(disk_util): Switch to btrfs for root filesystem in /usr images
This is all that is required to swap out ext4 for btrfs.
No btrfs user space tools or auto-resize support yet.
2014-03-05 18:52:23 -08:00
Michael Marineau
a022642e54 rename(disk_layout): Remove the legacy use of "legacy" 2014-03-05 15:57:22 -08:00
Michael Marineau
453c784779 fix(build_image): Only clear UUID on rootfs/usr filesystems.
The funky UUID and other special settings should only be applied to
coreos-rootfs and coreos-usr partitions which will never be fscked. When
STATE becomes ROOT in -usr images it gets fsked while mounted read-only
and fsck updates the filesystem's UUID if it is blank. Turns out this
causes disagreement between the kernel and the disk leading to bad
things. A related issue was fixed in a newer version of tune2fs but
unless I missed it the same bugfix didn't make it into e2fsck so
updating wouldn't resolve the issue.

http://e2fsprogs.sourceforge.net/e2fsprogs-release.html#1.42.9
2014-01-17 21:41:18 -08:00
Michael Marineau
19ecd4572f fix(disk_util): Sort mount order instead of special casing /
Now that /usr is a mount point we need to make sure it gets mounted
before /usr/share/oem. Simply sorting by path length works well enough.
2014-01-05 18:39:47 -08:00
Michael Marineau
ba532952b1 fix(disk_util): Fix layout inheritance in config verification. 2014-01-05 13:55:52 -08:00
Michael Marineau
e066f91ca3 feat(disk_util): Add tune command, controlling fs read-only hack.
Now disk_util is aware of the weird ext2 read-only hack, both by
providing a command to manipulate it and support in the mount command to
automatically set the 'ro' mount option for filesystems with it.

Making mount aware of the hack makes it much easier to mount prod
images with a mix read-only and read-write filesystems.
2014-01-04 21:00:45 -08:00
Michael Marineau
89b61ac7b0 fix(disk_util): Rework how code handles existing images.
Read partition size and offset into the normal partition config with the
prefix 'image_' for each key.
2014-01-04 21:00:45 -08:00
Michael Marineau
1ee1840a89 fix(disk_util): Fix mount --read_only option. 2014-01-04 16:55:03 -08:00
Michael Marineau
c992d45dd3 feat(disk_util): Add resize command.
Based on the new gpt update code add the ability to resize filesystems
in existing images. Usually just the last (STATE) partition is resized.
2014-01-03 18:14:12 -08:00
Michael Marineau
ebc83599b3 fix(disk_util): Move loop device setup code to generic function 2014-01-03 17:59:15 -08:00
Michael Marineau
805cc65ffd fix(disk_util): Add option to safely rewrite partition table.
write_gpt --update <img> will read an existing image and make sure all
existing partitions will not get moved or truncated in the new layout.
This is mostly useful for resizing the final partition or just rewriting
metadata like partition types and labels.
2014-01-03 16:40:54 -08:00
Michael Marineau
5afd720956 fix(disk_util): Move existing image reading code to new function.
This will allow the code to be shared, making it possible to safely
resize existing images with disk_util.
2014-01-03 16:40:21 -08:00
Michael Marineau
359505490d fix(disk_util): Install syslinux as part of filesystem formatting.
This means scripts later don't need to deal with the ESP block device
directly, they can just copy the config files into place.
2013-12-30 16:12:43 -08:00
Michael Marineau
8ca62121d9 feat(disk_util): Add mount and unmount commands.
This will replace the assorted mix of mount/umount scripts.
2013-12-30 16:12:04 -08:00
Michael Marineau
ba45152b4b feat(disk_util): Add 'format' command to create filesystems in an image.
This will replace the less flexible code in base_image_util.sh
2013-12-30 16:12:04 -08:00
Michael Marineau
ab23353448 fix(disk_util): Rework config parsing.
Merge GetPartitionTable and partition alignment from WritePartitionTable
into LoadPartitionConfig so that all this config manipulation code is in
one place and inheritance from the 'base' layout is more predictable.
2013-12-30 16:12:04 -08:00
Michael Marineau
0f4b52134a fix(disk_util): Convert partition list to an object.
Hey, 'num' is a primary key! Lets use it and avoid pointless looping.
2013-12-30 16:12:04 -08:00
Michael Marineau
576c8996f4 fix(disk_util): Remove support for --adjust_part
This isn't a feature we've been using as far as I know and if someone
needs a custom partition layout it's probably better to just add it to
the json file. Removing this avoids some complexity.
2013-12-30 16:12:02 -08:00