Commit Graph

220 Commits

Author SHA1 Message Date
Michael Marineau
a78c5fb665 fix(vm_image_util): Keep most partitions mounted read-only.
This ensures that the only partition being modified is OEM.
2014-03-27 21:52:40 -07:00
Michael Marineau
a6328f7c2b feat(vm_image_util): Add /usr/.noupdate flag file in PXE images.
This can be used by update_engine as a quick test to determine if it is
running on a system that it can handle. This avoids needing something
like the 'coreos.diskless' kernel command line flag.
2014-03-27 15:41:41 -07:00
Michael Marineau
11aabd7bd0 fix(vm_image_util): Do not generate a machine UUID.
If QEMU is given a uuid systemd will detect that and in turn use it for
the machine-id. This made the bug causing the machine-id to be always
re-generated on boot harder to notice since it didn't happen on QEMU.
2014-03-25 14:23:23 -07:00
Michael Marineau
c489d2f7c8 Merge pull request #196 from marineam/no_kexec
feat(vm_image_util): Add qemu_no_kexec image type
2014-03-24 14:48:55 -07:00
Michael Marineau
7c0e821953 Merge pull request #202 from marineam/pxe
PXE images
2014-03-23 20:19:41 -07:00
Michael Marineau
0a8f178002 feat(vm_image_util): Support other disk formats with Xen, such as vhd. 2014-03-23 17:15:36 -07:00
Michael Marineau
d4d06cfe57 feat(vm_image_util): Add new Rackspace target with VHD disk images. 2014-03-23 16:31:09 -07:00
Michael Marineau
5ab320d317 fix(vm_image_util): Fix typo in README 2014-03-23 11:43:36 -07:00
Michael Marineau
066bd23df8 feat(vm_image_util): Rework the qemu wrapper script to work for PXE
Now the script can be used with a disk image or a kernel/initrd.
Using a disk with the PXE kernel should work too but haven't tried it.
2014-03-22 22:28:10 -07:00
Michael Marineau
e151a33cee fix(vm_image_util): Fix building squashfs for /usr
Taking a bit of a new approach to booting PXE images here for both
amd64-generic and amd64-usr. Instead of requiring the user to specify
squashfs and tmpfs on the kernel command line we can simply provide
defaults in the initrd's fstab.
2014-03-22 22:23:40 -07:00
Michael Marineau
441a982a39 fix(vm_image_util): Disable installing oem-pxe
It only installed /usr/share/oem/oem-release which we no longer use.
2014-03-22 14:27:48 -07:00
Michael Marineau
81367903bf fix(vm_image_util): Package up /usr for pxe images. 2014-03-22 14:27:48 -07:00
Michael Marineau
e2536ad76f feat(vm_image_util): Add qemu_no_kexec image type
This makes it easy to test booting without kexec on qemu/kvm similar to
how images are booted on Xen.
2014-03-18 15:07:09 -07:00
Michael Marineau
0c1198c3bb feat(disk_util): Switch from SYSLINUX's gptmbr.bin to mbr.bin
cgpt now supports generating hybrid MBRs and the classic style mbr.bin
from any version of SYSLINUX should work the same with the hybrid MBR.
The other code, gptmbr.bin, changes after SYSLINUX 3. Switching lets me
play with different versions of SYSLINUX without breaking everything.

With this change all images feature a hybrid MBR so the special case for
some VM platforms has been removed.
2014-03-12 15:38:34 -07:00
Michael Marineau
c90ea5f504 fix(vm_image_util): Skip kexec when booting on Xen HVM
This is buggy, instead we will need to handle Xen HVM similar to PV
except this requires SYSLINUX config tweaking instead of grub's menu.lst
2014-03-11 11:38:46 -07:00
Michael Marineau
a022642e54 rename(disk_layout): Remove the legacy use of "legacy" 2014-03-05 15:57:22 -08:00
Michael Marineau
1177d198da fix(release_util): Fix DIGESTS regression from feb59db9f
While attempting to fix the easy to mix up DIGESTS names in feb59db9f I
stumbled across yet another way that the DIGESTS names were a bit
unpredictable: previously a .bz2 got stuck into the file name when
upload_images automatically compressed some file types. The new code
missed this and never added the .bz2. Correct this now, both for
image_to_vm which has a pile of glue to keep the legacy names in place
for now and build_image which I never intended to change.
2014-02-19 17:50:40 -08:00
Michael Marineau
821d718a74 fix(vm_image_util): assert qemu wrapper is sh friendly 2014-02-09 18:22:34 -08:00
Michael Marineau
bf6ee33244 fix(vm_image_util): If DIGESTS wasn't generated don't copy it.
Failed to test my last change with uploads disabled :(

Also, it needs to be return 0, not return.
2014-01-30 13:31:36 -08:00
Michael Marineau
8a15840d6a fix(vm_image_util): Use more consistent names for vm .DIGESTS
Switch from naming DIGESTS based in disk image name to a common prefix.

old: coreos_production_qemu_image.img.DIGESTS ->
new: coreos_production_qemu.DIGESTS

The old behavior wasn't very consistent since plain disk images aren't
used by all types and the code implementing that was easy to brake,
namely by mistake coreos_production_pxe_image.cpio.gz.DIGESTS became
coreos_production_pxe.vmlinuz.DIGESTS a couple releases ago.

The old names will continue to exist as well for the time being to avoid
breaking existing install/download scripts and the original pxe DIGESTS
name is back.
2014-01-29 17:23:10 -08:00
Michael Marineau
feb59db9f5 fix(release_util): Add argument to specify the name to use for .DIGESTS
For multi-file uploads we should explicitly declare what the name of the
.DIGESTS file should be instead of using the first file name. Relying on
the ordering was subtle and easy to break.
2014-01-29 16:30:03 -08:00
Michael Marineau
b3e81f22e4 fix(image_to_vm): Wrap upload_image in vm_upload.
This keeps all references to VM_GENERATED_FILES inside of
build_library/vm_image_util.sh so changes that impact it more obvious.
2014-01-29 14:01:33 -08:00
Michael Marineau
bfe47817e4 fix(build_image): Add support for COREOS_DISK_LAYOUT_SUFFIX
Append COREOS_DISK_LAYOUT_SUFFIX from the portage profile to disk layout
names. Allows switching from "base" to "base-usr" and similar.
2014-01-05 18:39:47 -08:00
Michael Marineau
3546256597 fix(vm_image_util): Unmount STATE before making squashfs 2014-01-04 21:39:31 -08:00
Michael Marineau
bfcbb3f57f fix(image_to_vm): Move to disk_util, modify images in-place.
Avoid unpacking/repacking partitions by using disk_util's resize and
mount commands. Also simplify assorted bits along the way.
2014-01-04 21:00:45 -08:00
Michael Marineau
8e0d630699 fix(image_to_vm): Remove support for alternate state fs image.
We haven't really needed this, remove the option to simplify things.
2013-12-30 16:32:37 -08:00
Michael Marineau
de9ff4fc8e fix(disk_util): Rework how argument parsing/passing works.
Move from optparse to argparse. Move layout file and layout type to
global options with reasonable default values so every command doesn't
need to them. Adjust calling scripts to match.

For now layout type is being passed via an environment variable
DISK_LAYOUT_TYPE but this is a temporary situation.
2013-12-30 16:10:46 -08:00
Michael Marineau
95388caab1 fix(image_to_vm): Add GCE oem package 2013-11-30 17:33:33 -08:00
Michael Marineau
0cb0cbf71e feat(image_to_vm): Add minimal GCE support.
Simply rolls our disk image into a tarball that can be passed to gcutil.
To actually be useful we need an OEM package to handle SSH keys.
2013-11-30 16:39:31 -08:00
unclejack
655c9de59e fix(build_library): set tz to UTC in vmx template
VMware is using the local time zone by default for the clock.
This changes the vmx template in vm_image_util.sh to set it to UTC
instead.
2013-10-15 23:14:11 +03:00
Alex Polvi
dcfcb00f5e feat(oem-rackspace): add rackspace as valid format option 2013-10-13 19:56:39 -07:00
Brandon Philips
178783f1c8 fix(build_library): fix the pxe image instructions
the args to the kernel have changed to state= and root=squashfs: also
make the boot be 1024Mb of RAM otherwise qemu falls over with initrds.
2013-09-30 17:05:55 -07:00
Brandon Philips
05a4974836 fix(vm_image_util): remove /usr/share/oem prefix
remove this prefix from the PARTIIONED=0 builds
2013-09-09 18:06:40 -07:00
Brandon Philips
2605c6ca95 feat(vm_image_util): use bind mounts for pxe oem
solid cleanup idea from marineam to use bind mounts instead of copying
to generate the desired root layout.
2013-09-09 17:47:10 -07:00
Brandon Philips
f3413b473d feat(build_library): generate squashfs in cpio
squashfs is more efficient and plays well with namespaces. Switch to
generating a cpio with a single squashfs files.
2013-09-09 16:12:44 -07:00
Michael Marineau
b223b45800 fix(image_to_vm): Fix upload pxe, VM_GENERATED_FILES must be an abspath 2013-09-05 16:14:02 -07:00
Michael Marineau
bbacf2f0bd fix(image_to_vm): Mount source rootfs as ro when creating cpio.
When building with a write-protected root mounting as rw, the default,
fails.
2013-09-05 16:24:45 -04:00
Michael Marineau
c8d986d408 fix(image_to_vm): Cleanup must run as root.
Failures mid-build leave root-owned files behind.
2013-09-05 16:17:14 -04:00
Michael Marineau
2d5a0a29e4 fix(image_to_vm): Use SCSI for VMware disks. 2013-09-01 17:23:59 -07:00
Michael Marineau
1539ff334e Merge pull request #98 from marineam/image-size
Image size
2013-08-28 12:25:56 -07:00
Brandon Philips
043dfeb868 Merge pull request #94 from philips/pxe-initial
Pxe initial
2013-08-28 12:09:41 -07:00
Brandon Philips
ce445512fd fix(build_library): fixup naming for the pxe image
based on @marineam's comments fixup the naming and ensure the output
files get picked up by the build system.
2013-08-28 12:04:55 -07:00
Michael Marineau
8a488f890b fix(image_to_vm): Increase STATE in vagrant images to 16GB
Vagrant users are accustomed to much larger disk sizes so lets give it
to them. I'm leaving the others as-is since it is easier to grow than
shrink disks if anyone has a particular size they need.
2013-08-28 11:52:19 -07:00
Michael Marineau
9734ac017f feat(image_to_vm): Make disk layout configurable by VM type.
Use the smaller base format for 'raw' disk images since these will
usually be dd'd to a block device to create AMIs and what not. For
images using qcow2 and vmdk stick with the larger vm size.
2013-08-28 11:35:06 -07:00
Michael Marineau
c6bbadd6e0 feat(image_to_vm): Add 'vmware_insecure' image type.
This bundles the vagrant ssh key, vmdk, and vmx into a zip file for easy
downloading and login access via ssh.
2013-08-27 21:18:12 -07:00
Brandon Philips
a840ae036c feat(build_library): pxe support
This adds support for creating a PXE image and kernel using build_image
2013-08-27 17:33:44 -07:00
Brandon Philips
047e3e0df1 feat(build_library): put all of the partition stuff behind a flag
this lays the groundwork for supporting a cpio img which doesn't have
partitions.
2013-08-22 10:51:11 -07:00
Michael Marineau
34cfe6e07e churn(build_library): Change lots of stateful references to state.
For consistency we are using 'state' these days, not 'stateful'. While
I'm mucking around in this code it seems like a good time to switch.
2013-08-20 22:36:53 -04:00
Michael Marineau
34c324cc3a fix(build_library): Remove GPT writer scripts, call cgpt directly.
This change removes /usr/sbin/write_gpt.sh from images which we have no
use for. This allows us to drop the indirection of writing partition
tables by first writing out a script to call. Now cgpt.py can call cgpt
directly to initialize the partition layout. This opens the way for
further improvements to how disk images are created.
2013-08-20 21:30:22 -04:00
Michael Marineau
0f84e3b05f feat(image_to_vm): Add new and improved qemu wrapper script.
This one is more automagical and sets up ssh keys from ssh-agent and the
user's home directory by default. Also adds an option for setting the
ssh port so it can be something other than 2222. Script should be
sufficiently portable, tested in bash, dash, and ash.
2013-08-18 19:43:07 -04:00
Michael Marineau
b0d2775e0a fix(image_to_vm): Fix backslash in qemu README.
Go me fixing this issue in the script while adding it in the README.
2013-08-11 23:41:05 -04:00
Michael Marineau
bef5728f6e fix(image_to_vm): Cleanup qemu script, document -virtfs 2013-08-11 22:17:20 -04:00
polvi
3a1d4dbe1b Merge pull request #80 from polvi/vagrant-vmware
feat(vagrant): vmware fusion support
2013-08-06 14:01:00 -07:00
Alex Polvi
324f4d86f2 call it --format=vagrant_vmware_fusion 2013-08-06 11:14:03 -07:00
Alex Polvi
55062ec902 feat(vagrant): vmware fusion support 2013-08-06 10:58:18 -07:00
Michael Marineau
f8c0033de2 feat(image_to_vm): Add openstack image, uses oem-ami and qcow2 format. 2013-08-02 15:07:12 -04:00
Michael Marineau
c7ba8cd4a4 fix(image_to_vm): Replace qemu's config file with a script.
The recommended command using the config file was triggering a massive
memory leak in qemu because it was adding both the default virtual
hardware nic as well as the virtio nic. This could be worked around by
adding something like -net none or moving all the -net commands from the
file to the command line but eh. Clearly qemu config files are used and
tested by nobody else so lets just use a trusty script instead.
2013-08-02 14:53:08 -04:00
Michael Marineau
99982182b9 feat(image_to_vm): Switch to qcow2 as default qemu disk format.
We only really need raw images for dd'ing to block devices and this
format has been requested for some cases so lets just use it!
2013-08-02 14:53:08 -04:00
Alex Polvi
9ddb254ac1 feat(vagrant) add version requirement to the vagrantfile 2013-08-01 20:29:38 -07:00
Michael Marineau
9c88c24131 fix(image_to_vm): gzip vagrant's .box files 2013-08-01 22:42:37 -04:00
Michael Marineau
ae3202da32 fix(image_to_vm): Add VM MAC address to Vagrantfile, create box image.
Vagrant should now be good to go!
2013-08-01 19:39:12 -04:00
Michael Marineau
a2046afc40 fix(image_to_vm): Use vmdk disks for vagrant and write configs.
Vagrant will need the virtualbox ovf plus its own Vagrantfile config.
After this we will need an optional "package" step to this script to
take these files and bundle them into a .box tarball. This could also be
used to switch from .ovf+vmdk for plain virtualbox images to a bundled
.ova archive which combines the two.
2013-08-01 16:35:28 -04:00
Michael Marineau
ce01e133db fix(image_to_vm): Use vmdk+ovf for virtual box images.
Now we have a config file that can be imported into VirtualBox.
2013-08-01 14:42:04 -04:00
Michael Marineau
5e1337307a feat(image_to_vm): Basic vagrant disk image support
This just generates a vdi that includes the vagrant ssh key.
2013-08-01 00:02:33 -04:00
Michael Marineau
7c48115da6 fix(image_to_vm): Use saved version info instead of parsing directory.
Trying to include version info by adding the directory name to VM image
names didn't work and a better solution is for build_image to write out
a version.txt file. This should also fix an issue where uploading from
image_to_vm.sh didn't always go to the same location as the images
uploaded from build_image did.
2013-07-31 21:22:52 -04:00
Michael Marineau
a1a1ed830c fix(build_library): Use sparse files for disk images, no useless sudo
Enable sparse files for all dd and cp commands and replace some dd
commands that are really better off being truncate commands.

While in the neighborhood there were a number of useless sudo commands
for things that just happen to be in sbin. Call them directly instead.
2013-07-26 23:45:10 -04:00
Michael Marineau
c5cd245603 feat(image_to_vm): Add support for uploading vm images
After this I can make production AMI images available for download! :-D
2013-07-26 16:31:11 -04:00
Michael Marineau
9379ee19f0 fix(vm_image_util): Remove space before # comment in qemu config.
Last minute bug slipped in because of a line I commented out since the
current coreos kernel doesn't support virtio block devices (that change
coming soon). Qemu doesn't tolerate any spaces before # in comments.
2013-07-25 13:54:50 -04:00
Michael Marineau
df886c91e7 fix(vm_image_util): Remove tab characters. 2013-07-25 13:16:04 -04:00
Michael Marineau
033cf224f1 feat(image_to_vm): Refactor to make adding/changing vm formats easier.
The old script was heading towards spaghetti code realm. This breaks up
all the image variations such as hybrid MBR, OEM packages, etc into
configuration options and small functions that actually do the work.

All this is in the new vm_image_util.sh library but the command line
parsing and overall procedure remains in image_to_vm.sh

As part of this we gain support for putting some qemu options in a
config file as well as Xen virtual machines using pygrub and pvgrub.

Lots of generally unused options have been removed to simplify things
and keep output file names consistent.
2013-07-24 23:11:50 -04:00