mirrors/flatcar-scripts
1
0
Fork 0
mirror of https://github.com/flatcar/scripts.git synced 2025-08-11 15:06:58 +02:00
Code Issues Packages Projects Releases Wiki Activity

README: add documentation on submodules and on tags

Browse Source 
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
This commit is contained in:
Thilo Fromm 2021-10-28 16:34:15 +02:00 committed by flatcar-ci
parent 877120d96c
commit 10dc85b099
1 changed files with 59 additions and 18 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

77
README.md
View File

@ -11,17 +11,53 @@ The SDK can be used to
[flatcar-docs]: https://docs.flatcar-linux.org/os/sdk-modifying-flatcar/ [flatcar-docs]: https://docs.flatcar-linux.org/os/sdk-modifying-flatcar/
# Using the scripts repository: submodules and tags
The repository is meant to be the entry point for Flatcar builds and development.
For building pakages, there are 2 addigional repositories, [coreos-overlay](https://github.com/flatcar-linux/) and [portage-stable](https://github.com/flatcar-linux/portage-stable), which contain all packages' `ebuild` (build configuration) files.
These repositories are included in `scripts` via git submodules and are used by the SDK container wrapper scripts detailed on further below.
The submodules reside in:
```
scripts
+--sdk_container
+---------src
+--third_party
+------coreos-overlay
+------portage-stable
```
When working with the scripts repo always make sure to initialise and to update these submodules; otherwise builds will break because build configuration is missing:
```bash
$ git clone https://github.com/flatcar-linux/scripts.git
$ cd scripts
$ git submodule init
$ git submodule update
```
The `scripts` repository makes ample use of tags to mark releases.
Sometimes, local and origin tags can diverge (e.g. when re-tagging something locally to test a build).
Also, `git pull` and `git fetch` do not automatically pull new tags, so long-standing local sourcetrees may lack newer versions.
To fetch and update all tags and to remove tags locally which have been deleted upstream, do
```
$ git pull --all --tags --prune --prune-tags
```
If upstream retagged (of if a tag was changed locally) the corresponding upstream tag will not be pulled so the local tag remains.
In order to override local tags with upstream, run
```
$ git pull --all --tags --prune --prune-tags --force
```
# Using the SDK container # Using the SDK container
We provide a containerised SDK via https://github.com/orgs/flatcar-linux/packages. The container comes in 3 flavours: We provide a containerised SDK via https://github.com/orgs/flatcar-linux/packages. The container comes in 3 flavours:
* Full SDK initialised with both architectures supported by Flatcar (amd64 and arm64). This is the largest container, it's about 7GB in size. * Full SDK initialised with both architectures supported by Flatcar (amd64 and arm64). This is the largest container, it's about 8GB in size (~3 GB compressed).
* AMD64 SDK initialised for building AMD64 OS images. About 5.5GB in size. * AMD64 SDK initialised for building AMD64 OS images. About 6GB in size (2GB compressed).
* ARM64 SDK initialised for building ARM64 OS images on AMD64 hosts. Also about 5.5GB in size. (While work on a ARM64 native SDK is ongoing, it's unfortunately not ready yet). * ARM64 SDK initialised for building ARM64 OS images on AMD64 hosts. Also about 6GB in size.
While work on a native ARM64 native SDK is ongoing, it's unfortunately not ready yet. If you want to help, patches are welcome!
The container can be run in one of two ways - "standalone", or integrated with the [Scripts](https://github.com/flatcar-linux/scripts) repo: The container can be run in one of two ways - "standalone", or integrated with the [Scripts](https://github.com/flatcar-linux/scripts) repo:
* Standalone mode will use no host volumes and will allow you to play with the SDK in a sandboxed throw-away environment. In standalone mode, you interface with Docker directly to use the SDK container. * Standalone mode will use no host volumes and will allow you to play with the SDK in a sandboxed throw-away environment. In standalone mode, you interface with Docker directly to use the SDK container.
* Integrated mode will closely integrate with the Scripts directory and bind-mount it as well as the portage-stable and coreos-overlay gitmodules into the container. Integrated mode uses wrapper scripts to interact with the SDK container. * Integrated mode will closely integrate with the Scripts directory and bind-mount it as well as the portage-stable and coreos-overlay gitmodules into the container. Integrated mode uses wrapper scripts to interact with the SDK container. This is the recommended way for developing patches for Flatcar.
## Standalone mode ## Standalone mode
@ -29,10 +65,10 @@ In standalone mode, the SDK is just another Docker container. Interaction with t
* Check the list of available versions and pick a version to use. The SDK Major versions correspond to Flatcar Major release versions. * Check the list of available versions and pick a version to use. The SDK Major versions correspond to Flatcar Major release versions.
List of images: `https://github.com/orgs/flatcar-linux/packages/container/package/flatcar-sdk-all` List of images: `https://github.com/orgs/flatcar-linux/packages/container/package/flatcar-sdk-all`
For the purpose of this example we'll use version `3005.0.0`. For the purpose of this example we'll use version `3033.0.0`.
* Fetch the container image: `docker pull ghcr.io/flatcar-linux/flatcar-sdk-all:3005.0.0` * Fetch the container image: `docker pull ghcr.io/flatcar-linux/flatcar-sdk-all:3033.0.0`
* Start the image in interactive (tty) mode: `docker run -ti ghcr.io/flatcar-linux/flatcar-sdk-all:3005.0.0` * Start the image in interactive (tty) mode: `docker run -ti ghcr.io/flatcar-linux/flatcar-sdk-all:3033.0.0`
You are now inside the SDK container: You are now inside the SDK container (the hostname will likely differ):
`sdk@f236fda982a4 ~/trunk/src/scripts $` `sdk@f236fda982a4 ~/trunk/src/scripts $`
* Initialise the SDK in self-contained mode. This needs to be done once per container and will check out the scripts, coreos-overlay, and portage-stable repositories into the container. * Initialise the SDK in self-contained mode. This needs to be done once per container and will check out the scripts, coreos-overlay, and portage-stable repositories into the container.
`sdk@f236fda982a4 ../sdk_init_selfcontained.sh` `sdk@f236fda982a4 ../sdk_init_selfcontained.sh`
@ -45,7 +81,7 @@ In order to build OS images (via `./build_image` and `./image_to_vm`) the SDK to
This is necessary because the SDK currently employs loop devices to create and to partition OS images. This is necessary because the SDK currently employs loop devices to create and to partition OS images.
To start a container in privileged mode with `/dev` available use: To start a container in privileged mode with `/dev` available use:
* `docker run -ti --privileged -v /dev:/dev ghcr.io/flatcar-linux/flatcar-sdk-all:3005.0.0` * `docker run -ti --privileged -v /dev:/dev ghcr.io/flatcar-linux/flatcar-sdk-all:3033.0.0`
## Integrated mode ## Integrated mode
@ -56,10 +92,11 @@ The wrapper scripts will re-use existing containers instead of creating new ones
To clone the scripts repo and pick a version: To clone the scripts repo and pick a version:
* Clone the scripts repo: `git clone https://github.com/flatcar-linux/scripts.git` * Clone the scripts repo: `git clone https://github.com/flatcar-linux/scripts.git`
* Make sure to initialise and fetch git submodules - Flatcar's ebuilds are in 2 separate repositories, connected to `scripts` via submodules.
* `git submodule init; git submodule update`
* Optionally, check out a release tag to base your work on * Optionally, check out a release tag to base your work on
* list releases (e.g. all Alpha releases): `git tag -l alpha-*` * list releases (e.g. all Alpha releases): `git tag -l alpha-*`
* check out the release version, e.g. `3005.0.0`: `git checkout 3005.0.0` * check out the release version, e.g. `3033.0.0`: `git checkout 3033.0.0`
* Update the overlay submodules: `git submodules update`
To use the SDK container: To use the SDK container:
* Fetch image and start the SDK container: `./run_sdk_container -t` * Fetch image and start the SDK container: `./run_sdk_container -t`
@ -73,10 +110,10 @@ Subsequent calls to `./run_sdk_container` will re-use the container (as long as
Check out `docker container ls --all` and you'll see something like Check out `docker container ls --all` and you'll see something like
``` ```
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
19ea3b6d00ad ghcr.io/flatcar-linux/flatcar-sdk-all:3005.0.0 "/bin/sh -c /home/sd…" 4 hours ago Exited (0) About an hour ago flatcar-sdk-all-3005.0.0_os-3005.0.0 19ea3b6d00ad ghcr.io/flatcar-linux/flatcar-sdk-all:3033.0.0 "/bin/sh -c /home/sd…" 4 hours ago Exited (0) About an hour ago flatcar-sdk-all-3033.0.0_os-3033.0.0
``` ```
Re-use of containers happens on a per-name basis. The above example's container name `flatcar-sdk-all-3005.0.0_os-3005.0.0` is generated automatically. Using `docker container rm` the container can be discarded - a subsequent call to `./run_sdk_container` will create a new one. Custom containers can be created by use of the `-n <name>` command line option; these will be re-used in subsequent calls to `./run_sdk_container` when using the same `<name>`. Re-use of containers happens on a per-name basis. The above example's container name `flatcar-sdk-all-3033.0.0_os-3033.0.0` is generated automatically. Using `docker container rm` the container can be discarded - a subsequent call to `./run_sdk_container` will create a new one. Custom containers can be created by use of the `-n <name>` command line option; these will be re-used in subsequent calls to `./run_sdk_container` when using the same `<name>`.
The local sourcetree can also be used with an entirely custom SDK container image. Users must ensure that the image is either fetch-able or present locally. The custom image can be specified using `-C <custom-image>`. This option is useful e.g. for building the local sourcetree with different SDK versions. The local sourcetree can also be used with an entirely custom SDK container image. Users must ensure that the image is either fetch-able or present locally. The custom image can be specified using `-C <custom-image>`. This option is useful e.g. for building the local sourcetree with different SDK versions.
@ -84,11 +121,15 @@ Check out `./run_sdk_container -h` for more information on command line options.
# Building a new SDK container # Building a new SDK container
Building an SDK container is done using `./build_sdk_container_image`. Building an SDK container is done using `./build_sdk_container_image <tarball>`.
The SDK container is based on an SDK tarball which the script will fetch. The tarball input is the result of an SDK bootstrap (see below). Version information for both OS as well as for the SDK will be extracted from the tarball name.
By default, the current git tree's release version will be built; this can be changed with the `-v` flag. The version file will be updated accordingly before the SDK container is built.
When using `-v`, the corresponding release version of the Scripts repository is checked out (unless suppressed by `-c`) before the container is generated. During the build, toolchain packages will be built and installed into the SDK container image. Both supported boards (`amd64-usr` and `arm64-usr`) will be initialisedin the container image.
# Bootstrapping a new SDK tarball using the SDK container # Bootstrapping a new SDK tarball using the SDK container
The script `./bootstrap_sdk_container` bootstraps a new SDK tarball using an existing SDK container and seed tarball. Specifying the seed version is required for this script. The script `./bootstrap_sdk_container` bootstraps a new SDK tarball using an existing SDK container and seed tarball. Specifying the seed version as well as the designated new SDK version is required for this script.
# Automation stubs for continuous integration
Script stubs for various build stages can be found in the [ci-automation](ci-automation) folder. THese are helpful for glueing Flatcar Container Linux builds to a continuous integration system.