25395 Commits

Author SHA1 Message Date
Jeremi Piotrowski
f9c4e2d42a WIP nvidia sysext 2024-03-11 10:17:32 +00:00
Jeremi Piotrowski
6ebadf4420 build_library: Add manglefs script for podman sysext
Debug symbols and development files get removed. /etc content is
temporarily pushed to /usr/share/flatcar/etc, which doesn't work.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2024-03-11 10:17:32 +00:00
Jeremi Piotrowski
042f30ea40 coreos/config: Export PKG_CONFIG for podman
Podman build uses pkg-config support in Go to link against some library.
This needs the correct sysroot version of PKG_CONFIG to be exported.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2024-03-11 10:17:32 +00:00
Jeremi Piotrowski
5405224aea coreos-overlay: Add podman to board packages
and add keywords to profile. Podman 4.9.2 is the stable version
according to the website but it's keyworded ~ARCH in Gentoo. One of the
dependencies (containers-common) also needs to be keyworded or else
there are file collosions with the containers-storage package.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2024-03-11 10:17:32 +00:00
Jeremi Piotrowski
922e4f4cb0 portage-stable: Import podman (& deps) ebuilds
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2024-03-11 10:17:32 +00:00
Jeremi Piotrowski
6a05cda498 github: Import podman (& deps) to auto-update package list.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2024-03-11 10:17:32 +00:00
Jeremi Piotrowski
307d8378ae build_library: Add mangle script for zfs sysext
There are two challenges with the sysext: it needs config files in /etc
and it needs udev rules for mounting during boot to work. The etc files
are placed in the standard flatcar etc overlay path but the overlay is
mounted from the initrd. So instead, we create a tmpfiles.d rule that
symlinks the best important files over. For the udev issue, we create a
drop-in in /etc that ensures udev runs after systemd-sysext.

We also can't rely on systemd presets to work, so instead parse the
preset file and statically create the service dependencies. For the
primary zfs.target we rely on an Upholds entry. Users can still disabled
unwanted services if they want.

We also removed unnecessary files:
- development files
- initramfs related scripts

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2024-03-11 10:17:32 +00:00
Jeremi Piotrowski
733618cf53 coreos/config: Add hooks for zfs ebuilds
The zfs-kmod ebuild needs KERNEL_DIR to point to the correct
/lib/modules directory.

The zfs ebuild installs two systemd unit masks to /usr/lib/systemd which
result in "dangling symlink" errors during the image build. These
systemd unit masks are only necessary for old Ubuntu systems that have
sysv-init-systemd compat wrappers.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2024-03-11 10:17:32 +00:00
Jeremi Piotrowski
e3407ce4ef .github: Add zfs ebuilds and related eclasses to sync list
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2024-03-11 10:17:32 +00:00
Jeremi Piotrowski
94826f066d portage-stable: Add zfs & zfs-kmod ebuilds
From Gentoo commit df182f2891606e757be3e8406a69f4a0e53ee324. Also import
dist-kernel-utils.eclass.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2024-03-11 10:17:32 +00:00
Jeremi Piotrowski
03a50427da profiles/coreos: Add zfs useflags
and a package.provided entry for a dependency that we don't need.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2024-03-11 10:17:32 +00:00
Jeremi Piotrowski
a8bbe47ca2 coreos-base/coreos: Switch to app-alternatives/awk
We explicitly install gawk but our profile explicitly forces the gawk
useflag for app-alternatives/awk.  Some packages, like zfs, depend on
app-alternatives/awk and it also installs the awk -> gawk that the gawk
ebuild creates through pkg_postinst. So switch to app-alternatives/awk
to make the implementation cleaner.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2024-03-11 10:17:32 +00:00
Jeremi Piotrowski
276746fd55 coreos-kernel.eclass: Adapt install_build_source for compatibility
build/source, which is accidentally an empty directory, needs to be a
symlink so that Gentoo kmod ebuilds can build when setting
KERNEL_DIR=/lib/modules/.../build. They detect the proper layout with
the symlink.

The other issue is building the Nvidia driver with
SYSSRC=/lib/modules/.../build. This works on Ubuntu but fails on
Flatcar. Ubuntus build directory contains symlinks to includes from the
source tree so recreate the same layout.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2024-03-11 10:17:32 +00:00
Jeremi Piotrowski
822d0a1478 update_ebuilds: Fix support for rsync of eclass
For eclasses we need to skip the mkdir and trailing slash.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2024-03-11 10:17:32 +00:00
Jeremi Piotrowski
30c509e26f build_image: Allow building sysexts for an existing image
by skipping the check for an existing image directory if we're not
building an image. This makes './build_image sysext' work.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2024-03-11 10:17:32 +00:00
Jeremi Piotrowski
fb83a191f4 build_image: Build sysext store after image when requested
This function is meant to prebuild certain sysexts to be released along
with each release. These will not be built into the image, but instead
can be fetched by the user on demand.

The command to build sysexts would be:

  ./build_image prod sysext

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2024-03-11 10:17:32 +00:00
Jeremi Piotrowski
2e491fb22e build_sysext: Set SOURCE_DATE_EPOCH for reproducibility
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2024-03-11 10:17:32 +00:00
Jeremi Piotrowski
68afa4a67d build_sysext: Use parallel jobs
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2024-03-11 10:14:21 +00:00
Jeremi Piotrowski
2d930c7748 build_library: Ignore btrfs xattrs
To prevent mksquashfs from spamming the console about btrfs.compression.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2024-02-27 07:48:23 +01:00
Jeremi Piotrowski
baebe124ce fixup! app-containers/nvidia-container-toolkit: Add default config file 2024-02-26 16:44:55 +00:00
Jeremi Piotrowski
f76769f88c x11-drivers/nvidia-drivers: Generate CDI definitions during loading
Nvidia CDI definitions are used by Kubernetes to represent the topology
of GPUs available to a node and allow workload placement on suitable
GPUs.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2024-02-26 15:22:20 +00:00
Jeremi Piotrowski
38024d4b14 docker: Add nvidia runtime definition
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2024-02-26 15:18:41 +00:00
Jeremi Piotrowski
edc7bc903f containerd: Add nvidia runtime definition to default config.toml
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2024-02-26 15:18:24 +00:00
Jeremi Piotrowski
9aa59f9adb app-containers/nvidia-container-toolkit: Add default config file
Created using `nvidia-ctk config generate-default`.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2024-02-26 15:14:15 +00:00
Jeremi Piotrowski
c687413bb8 x11-drivers/nvidia-drivers: Enable persistence mode by default
nvidia-persistenced keeps a reference to the GPU open at all times,
which allows applications/containerd accessing gpus to start faster.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2024-02-26 15:12:57 +00:00
Jeremi Piotrowski
cb1ddeb5a5 Add nvidia-container-toolkit to coreos meta package
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2024-02-26 13:08:35 +00:00
Jeremi Piotrowski
2a41f14325 Add nvidia-container-toolkit ebuild
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2024-02-26 11:53:03 +00:00
Jeremi Piotrowski
2051993ecf Add libnvidia-container ebuild
From scratch because the guru one patches too match and is going to break too
much. This package is primarily supposed to provide nvidia-container-cli.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2024-02-26 11:51:25 +00:00
Krzesimir Nowak
b51e3cbd94 Revert "New version: main-3889.0.0-nightly-20240223-2100-INTERMEDIATE"
This reverts commit 1fcad52f3a0c09458326089b454dd1ab03a1cb15.

Last nightly failed to build, possibly some intermittent error.
2024-02-26 09:13:24 +01:00
flatcar-ci
1fcad52f3a New version: main-3889.0.0-nightly-20240223-2100-INTERMEDIATE 2024-02-23 21:00:26 +00:00
Jeremi Piotrowski
57f2ead111
Merge pull request #1688 from flatcar/jepio/improve-gpu-match
nvidia-drivers: Improve match for NVIDIA GPUs
2024-02-23 16:38:07 +01:00
Mathieu Tortuyaux
c3c046529c
Merge pull request #1689 from flatcar/tormath1/keyutils
sys-apps/keyutils: sync with upstream
2024-02-23 14:54:09 +01:00
Mathieu Tortuyaux
e5970bb3cb
changelog: add entry
Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
2024-02-23 14:53:10 +01:00
Mathieu Tortuyaux
f84eedf688
sys-apps/keyutils: Apply Flatcar modifications
Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
Co-authored-by: Krzesimir Nowak <knowak@microsoft.com>
2024-02-23 14:53:10 +01:00
Mathieu Tortuyaux
c57607f3dd
Merge pull request #1687 from flatcar/tormath1/cloudinit
coreos-base/coreos-cloudinit: bump commit ID
2024-02-23 14:34:15 +01:00
Mathieu Tortuyaux
ea3d0cf1c4
changelog: add entry
Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
2024-02-23 14:33:29 +01:00
Mathieu Tortuyaux
af249419bc
coreos-base/coreos-cloudinit: bump commit ID
Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
2024-02-23 14:28:01 +01:00
Jeremi Piotrowski
2cbd78091e nvidia-drivers: Improve match for NVIDIA GPUs
The logic for checking if we need to probe depends on the device showing up
with the product type in lspci output, which doesn't hold for all sorts of
GPUs. The NVIDIA_PRODUCT_TYPE is used for fetching the drivers and is "tesla"
across datacenter GPUs.

Switch to matching on vendor id and device class. The values are the same
ones that the nvidia driver binds to.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2024-02-23 10:05:15 +01:00
Mathieu Tortuyaux
7b720fb337
sys-apps/keyutils: sync with upstream
Commit-Ref: 7456de7c55

Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
2024-02-23 10:05:02 +01:00
flatcar-ci
b2f4560d8e New version: main-3888.0.0-nightly-20240222-2100 2024-02-22 21:00:32 +00:00
Krzesimir Nowak
bf4b088cb8
Merge pull request #1672 from flatcar/buildbot/weekly-portage-stable-package-updates-2024-02-19
Weekly portage-stable package updates 2024-02-19
2024-02-22 09:01:41 +01:00
flatcar-ci
4bd0d342ac New version: main-3887.0.0-nightly-20240221-2100 2024-02-21 21:00:26 +00:00
Dongsu Park
051232e560
Merge pull request #1681 from flatcar/dongsu/gnupg-2.2.42
app-crypt/gnupg: update to 2.2.42-r2
2024-02-21 16:04:41 +01:00
Mathieu Tortuyaux
5a4c7d993a
Merge pull request #1680 from flatcar/tormath1/ignition
sys-apps/ignition: bump to 2.17.0
2024-02-21 15:47:59 +01:00
Mathieu Tortuyaux
2c2b20dd7d
changelog: add entry
Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
2024-02-21 15:47:14 +01:00
Mathieu Tortuyaux
0c3c14e9c3
sys-apps/ignition: bump to 2.17.0
Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
2024-02-21 15:47:14 +01:00
Krzesimir Nowak
1020bc7dcf changelog: Add entries 2024-02-21 11:05:42 +01:00
Krzesimir Nowak
c272472a40 overlay profiles: Add accept keywords for sys-apps/coreutils 2024-02-21 10:54:28 +01:00
Krzesimir Nowak
14ac663023 overlay profiles: Update accept keywords for net-misc/curl 2024-02-21 10:54:28 +01:00
Krzesimir Nowak
1bd3db3f19 overlay profiles: Update accept keywords for dev-libs/opensc 2024-02-21 10:54:28 +01:00