diff --git a/.github/workflows/portage-stable-packages-list b/.github/workflows/portage-stable-packages-list index 5ba23e9d6c..cb68fa1aab 100644 --- a/.github/workflows/portage-stable-packages-list +++ b/.github/workflows/portage-stable-packages-list @@ -378,6 +378,7 @@ sec-policy/selinux-unconfined sys-apps/acl sys-apps/attr +sys-apps/bubblewrap sys-apps/checkpolicy sys-apps/config-site sys-apps/coreutils @@ -508,6 +509,7 @@ virtual/perl-IO virtual/pkgconfig virtual/service-manager virtual/ssh +virtual/tmpfiles x11-base/xorg-proto diff --git a/PREFIX.md b/PREFIX.md new file mode 100644 index 0000000000..e353a0d5b6 --- /dev/null +++ b/PREFIX.md @@ -0,0 +1,98 @@ +# Prefix - build portable, distro-independent apps + +**!!! NOTE: Prefix support in the Flatcar SDK is EXPERIMENTAL at this time !!!** + +## Path to stabilisation TODO list + +Before prefix build support are considered stable, the below must be implemented: +1. Integrate `cb-bootstrap` with the Flatcar SDK. + Currently, `setup_prefix` uses cross-boss' `cb-bootstrap` to set up the prefix environment. + Bootstrapping must be fully integrated with the Flatcar SDK before prefix builds are considered stable. +2. Integrate prefix builds with `/build/` environment and use board cross toolchain. + Prefix builds currently use the SDK cross toolchains (`/usr/-gnu/`) instead of board toolchains in `/build/`. + Prefix builds must be integrated with the board toolchains and stop using `cb-emerge` before considered stable. +3. Add prefix wrappers for all portage tools (similar to board wrappers), not just `emerge`. +4. Add test cases for prefix builds to [mantle/kola](https://github.com/flatcar/mantle/tree/flatcar-master/kola). + +## About + +Prefix builds let you build and ship applications and all their dependencies in a custom directory. +This custom directory is self-contained, all dependencies are included, and binaries are only linked against libraries in the custom directory. +The applications' root will be `/` - i.e. there's no need to `chroot` into the custom directory. + +For example, applications built with the prefix `/usr/local/my-app` will ship +* binaries in `/usr/local/my-app/bin`, `/usr/local/my-app/usr/bin` +* libraries in `/usr/local/my-app/lib[64]`, `/usr/local/my-app/usr/lib[64]` + +These binaries can be called directly, e.g. `/usr/local/my-app/usr/bin/myprog`. +`myprog` will only use libraries from `/usr/local/my-app/lib` etc., not from `/`. + +A good use case example for prefix builds is to create distro independent, portable [system extensions](https://www.flatcar.org/docs/latest/provisioning/sysext/). + +## How does it do that? + +Prefix uses a _staging environment_ to build binary packages, then installs these to a _final environment_. +The _staging environment_ contains toolchains and all build tools required to create binary packages (a full `@system`). +The _final environment_ only contains run-time dependencies. + +Packages are built from ebuilds in coreos-overlay, portage-stable, and prefix-overlay. + +A QoL `emerge` wrapper is included to install packages to the prefix. + +## Prerequisites + +Prefix utilises the [cross-boss](https://github.com/chewi/cross-boss) project to bootstrap prefixes and to build packages. +For the time being the user is expected to provide cross-boss manually. +By default, a `cross-boss` sub-directory is expected in the scripts repository root. +Cross-boss location can be customised via the `--cross_boss_root` option to `setup_prefix`. + +* Run `git clone https://github.com/chewi/cross-boss` in the scripts directory. + +## Quick-start guide + +For working with a prefix, you will need to agree on: +1. A name for the prefix. Should be a single word and is used for generating protage wrappers. +2. A prefix directory where applications and libraries will live on the target system. + For use with systemd-sysext this should be a path below `/usr` or `/opt`. + +For the purpose of the example below we'll use +* `my-prefix` as the prefix name, and +* `/usr/local/my-stuff` as prefix directory. + +**TL;DR** +* `./setup_prefix my-prefix /usr/local/my-stuff` +* `emerge-prefix-my-stuff-amd64-usr python` +will create a portable python installation in `__prefix__/amd64-usr/my-stuff/root`. + + +**Step by step** + +First we'll create the prefix. +This will create "staging" and "final" roots and cross-compile a staging environment into "staging". +* In the SDK container, run `./setup_prefix my-prefix /usr/local/my-stuff` +* Go fetch a coffee, bootstrapping may take some 20-ish minutes to complete. + +`setup_prefix` will default to `amd64-usr` architecture and will use +* `/build/prefix-/my-stuff` for the staging environment +* `__prefix__//my-stuff` in the scripts directory as install root (aka "final") +* It will also create an emerge wrapper `emerge-prefix-my-stuff-` to install packages. + +Time to use the wrapper! Let's build a portable python sysext. +* `emerge-prefix-my-stuff-amd64-usr python` + +Now we'll use [bake.sh](https://raw.githubusercontent.com/flatcar/sysext-bakery/main/bake.sh) from Flatcar's [sysext-bakery](https://github.com/flatcar/sysext-bakery) to create a python sysext. +```shell +wget https://raw.githubusercontent.com/flatcar/sysext-bakery/main/bake.sh +chmod 755 bake.sh +cd __prefix__/amd64-usr/my-stuff +sudo cp -R root python +sudo ../../../bake.sh python +``` + +On a Flatcar instance, we now copy the resulting `python.raw` to `/etc/extensions`. +We merge with `systemd-sysext refresh`. +Then we can run: +* `/usr/local/my-stuff/usr/bin/python` + +Note that this sysext can be used on any Linux distro that ships `systemd-sysext`. +It is self-contained, there are no user space dependencies. diff --git a/build_library/prefix_util.sh b/build_library/prefix_util.sh new file mode 100644 index 0000000000..a8a7734e3c --- /dev/null +++ b/build_library/prefix_util.sh @@ -0,0 +1,220 @@ +# Copyright (c) 2023 The Flatcar Maintainers. All rights reserved. +# Use of this source code is governed by the Apache 2.0 license. + +DEFAULT_STAGING_ROOT="/build/" + +function lineprepend() { + awk -v msg="$*" '{ print msg ": " $0}' +} +# -- + +function set_prefix_vars() { + local name="${1}" + local prefix="${2}" + + EPREFIX="${prefix}" + PREFIXNAME="${name}" + STAGINGDIR="${FLAGS_staging_dir}" + STAGINGROOT="${STAGINGDIR}/root" + FINALDIR="${FLAGS_final_dir}" + FINALROOT="${FINALDIR}/root" + + CB_ROOT="${FLAGS_cross_boss_root}" + + # the prefix profile enables unstable via MAKE_DEFAULTS; we don't want those. + PREFIX_BOARD="${FLAGS_board}" + case "${PREFIX_BOARD}" in + amd64-usr) + PREFIX_CHOST="x86_64-cros-linux-gnu" + PREFIX_KEYWORDS="amd64 -~amd64" + ;; + arm64-usr) + PREFIX_CHOST="aarch64-cros-linux-gnu" + PREFIX_KEYWORDS="arm64 -~arm64" + ;; + esac + + export EPREFIX PREFIXNAME STAGINGDIR STAGINGROOT FINALDIR FINALROOT CB_ROOT \ + PREFIX_CHOST PREFIX_KEYWORDS PREFIX_BOARD +} +# -- + +function install_prereqs() { + # Make sure cross-boss prerequisites are installed in the SDK + local prefix_repo="${1}" + + sudo emerge --newuse sys-apps/bubblewrap + sudo emerge --newuse -1 ">=dev-python/gpep517-15" + + # HACK ALERT: needed for cb-bootstrap to build the initial toolchain in staging. + # cb-bootstrap should be ported to use the prefix repos.conf instead. + sudo cp -r "${prefix_repo}/skel/etc/portage/repos.conf" /usr/x86_64-cros-linux-gnu/etc/portage/ + sudo cp -r "${prefix_repo}/skel/etc/portage/repos.conf" /usr/aarch64-cros-linux-gnu/etc/portage/ +} +# -- + +function setup_prefix_dirs() { + local prefix_repo="${1}" + sudo mkdir -v -p \ + "${STAGINGDIR}/logs" \ + "${STAGINGDIR}/pkgs" \ + "${STAGINGDIR}/tmp" \ + "${STAGINGROOT}${EPREFIX}/etc" \ + "${FINALDIR}/logs" \ + "${FINALDIR}/tmp" \ + "${FINALROOT}${EPREFIX}/etc" + + sudo cp -vR "${prefix_repo}/skel/etc/portage" "${STAGINGROOT}${EPREFIX}/etc/" + sudo cp -vR "${prefix_repo}/skel/etc/portage" "${FINALROOT}${EPREFIX}/etc/" + + local profile="/mnt/host/source/src/third_party/portage-stable/profiles/default/linux" + case "${PREFIX_BOARD}" in + amd64-usr) profile="${profile}/amd64/17.1/no-multilib/prefix/kernel-3.2+";; + arm64-usr) profile="${profile}/arm64/17.0/prefix/kernel-3.2+";; + esac + + sudo ln -s "${profile}" "${STAGINGROOT}${EPREFIX}/etc/portage/make.profile" + sudo ln -s "${profile}" "${FINALROOT}${EPREFIX}/etc/portage/make.profile" +} +# -- + +function extract_gcc_libs() { + # GCC libs aren't available in a separate package but a full GCC install would make final too big + # TODO: the below is effectively a copy of build_library/prod_image_util.sh::extract_prod_gcc() + # and should eventually be reconciled. + gcc_ver="$(sudo -E PORTAGE_CONFIGROOT="${STAGINGROOT}${EPREFIX}" \ + portageq best_visible "${STAGINGROOT}${EPREFIX}" installed sys-devel/gcc)" + pkgdir="$(sudo -E PORTAGE_CONFIGROOT="${STAGINGROOT}${EPREFIX}" portageq pkgdir)" + qtbz2 -O -t "$pkgdir/$gcc_ver".tbz2 \ + | sudo tar -v -C "${FINALROOT}" -xj \ + --transform "s#.${EPREFIX}/usr/lib/.*/#.${EPREFIX}/usr/lib64/#" \ + --wildcards ".${EPREFIX}/usr/lib/gcc/*.so*" +} +# -- + +function create_make_conf() { + local which="${1}" \ + filepath \ + dir \ + portage_profile \ + emerge_opts + + case "${which}" in + staging) + filepath="${STAGINGROOT}${EPREFIX}/etc/portage/make.conf" + dir="${STAGINGDIR}" + emerge_opts="--buildpkg" + ;; + final) + filepath="${FINALROOT}${EPREFIX}/etc/portage/make.conf" + dir="${FINALDIR}" + emerge_opts="--root-deps=rdeps --usepkgonly" + ;; + esac + +sudo_clobber "${filepath}" <" + echo " Builds packages in prefix' staging and installs w/ runtime dependencies" + echo " to prefix' final root." + echo " --stage Build binpkg in staging but don't install." + echo " --install Skip build, just install. Binpkg must exist in staging." + echo + echo " Prefix configuration:" + echo " PREFIXNAME=${PREFIXNAME@Q}" + echo " EPREFIX=${EPREFIX@Q}" + echo " STAGINGROOT=${STAGINGROOT@Q}" + echo " FINALROOT=${FINALROOT@Q}" + echo " CB_ROOT=${CB_ROOT@Q}" + exit +fi + +skip_build="false" +skip_install="false" + +case "${1}" in + --install) skip_build="true"; shift;; + --stage) skip_install="true"; shift;; +esac + +if [ "${skip_build}" = "true" ] ; then + echo "Skipping build into staging as requested." + echo "NOTE that install into final will fail if binpkgs are missing." +else + echo "Building in staging..." + sudo -E EPREFIX="${EPREFIX}" "${CB_ROOT}/bin/cb-emerge" "${STAGINGROOT}" "$@" +fi + +if [ "${skip_install}" = "true" ] ; then + echo "Skipping install into final as requested." +else + echo "Installing..." + sudo -E EPREFIX="${EPREFIX}" \ + ROOT="${FINALROOT}" \ + PORTAGE_CONFIGROOT="${FINALROOT}${EPREFIX}" emerge "$@" +fi +EOF + + sudo chmod 755 "${filename}" +} +# -- diff --git a/changelog/changes/2023-09-20-experimental-prefix-builds.md b/changelog/changes/2023-09-20-experimental-prefix-builds.md new file mode 100644 index 0000000000..c9403fc037 --- /dev/null +++ b/changelog/changes/2023-09-20-experimental-prefix-builds.md @@ -0,0 +1 @@ +- SDK: Experimental support for [prefix builds](https://github.com/flatcar/scripts/blob/main/PREFIX.md) to create distro independent, portable, self-contained applications w/ all dependencies included. With contributions from [chewi](https://github.com/chewi) and [HappyTobi](https://github.com/HappyTobi). diff --git a/sdk_container/src/third_party/coreos-overlay/coreos-devel/sdk-depends/sdk-depends-0.0.1-r45.ebuild b/sdk_container/src/third_party/coreos-overlay/coreos-devel/sdk-depends/sdk-depends-0.0.1-r46.ebuild similarity index 100% rename from sdk_container/src/third_party/coreos-overlay/coreos-devel/sdk-depends/sdk-depends-0.0.1-r45.ebuild rename to sdk_container/src/third_party/coreos-overlay/coreos-devel/sdk-depends/sdk-depends-0.0.1-r46.ebuild diff --git a/sdk_container/src/third_party/coreos-overlay/coreos-devel/sdk-depends/sdk-depends-0.0.1.ebuild b/sdk_container/src/third_party/coreos-overlay/coreos-devel/sdk-depends/sdk-depends-0.0.1.ebuild index 4f6d96b08f..c326d07ff7 100644 --- a/sdk_container/src/third_party/coreos-overlay/coreos-devel/sdk-depends/sdk-depends-0.0.1.ebuild +++ b/sdk_container/src/third_party/coreos-overlay/coreos-devel/sdk-depends/sdk-depends-0.0.1.ebuild @@ -61,4 +61,9 @@ DEPEND="${DEPEND} ) sys-devel/m4" +DEPEND="${DEPEND} + sys-apps/bubblewrap + >=dev-python/gpep517-15 +" + RDEPEND="${DEPEND}" diff --git a/sdk_container/src/third_party/coreos-overlay/sys-libs/glibc/glibc-2.37-r5.ebuild b/sdk_container/src/third_party/coreos-overlay/sys-libs/glibc/glibc-2.37-r5.ebuild index a4bdd4a317..f071bac3b4 100644 --- a/sdk_container/src/third_party/coreos-overlay/sys-libs/glibc/glibc-2.37-r5.ebuild +++ b/sdk_container/src/third_party/coreos-overlay/sys-libs/glibc/glibc-2.37-r5.ebuild @@ -447,6 +447,10 @@ setup_flags() { # https://sourceware.org/PR27837 filter-ldflags '-Wl,--relax' + # Flag added for cross-prefix, but causes ldconfig to segfault. Not needed + # anyway because glibc already handles this by itself. + filter-ldflags '-Wl,--dynamic-linker=*' + # some weird software relies on sysv hashes in glibc, bug 863863, bug 864100 # we have to do that here already so mips can filter it out again :P if use hash-sysv-compat ; then diff --git a/sdk_container/src/third_party/portage-stable/dev-python/gpep517/gpep517-15.ebuild b/sdk_container/src/third_party/portage-stable/dev-python/gpep517/gpep517-15.ebuild index 75da542d13..b59c5342de 100644 --- a/sdk_container/src/third_party/portage-stable/dev-python/gpep517/gpep517-15.ebuild +++ b/sdk_container/src/third_party/portage-stable/dev-python/gpep517/gpep517-15.ebuild @@ -21,7 +21,7 @@ SRC_URI=" LICENSE="MIT" SLOT="0" -KEYWORDS="~alpha ~amd64 ~arm ~arm64 ~hppa ~ia64 ~loong ~m68k ~mips ~ppc ~ppc64 ~riscv ~s390 ~sparc ~x86 ~amd64-linux ~x86-linux ~arm64-macos ~ppc-macos ~x64-macos ~x64-solaris" +KEYWORDS="~alpha amd64 ~arm arm64 ~hppa ~ia64 ~loong ~m68k ~mips ~ppc ~ppc64 ~riscv ~s390 ~sparc ~x86 ~amd64-linux ~x86-linux ~arm64-macos ~ppc-macos ~x64-macos ~x64-solaris" RDEPEND=" >=dev-python/installer-0.5.0[${PYTHON_USEDEP}] diff --git a/sdk_container/src/third_party/portage-stable/sys-apps/bubblewrap/Manifest b/sdk_container/src/third_party/portage-stable/sys-apps/bubblewrap/Manifest new file mode 100644 index 0000000000..43cda60169 --- /dev/null +++ b/sdk_container/src/third_party/portage-stable/sys-apps/bubblewrap/Manifest @@ -0,0 +1 @@ +DIST bubblewrap-0.8.0.tar.xz 149088 BLAKE2B 5853cf42a7ab653540ec5134866c6f2459aa101e9eea724a4f283405cbcae2beb3551b7c1a7aa93d82016d4eb0d12f9c97c47df53a6d9b589db40483696253de SHA512 1cbc33f3c834ff83f4c1808d3ec2555921277d495f903ad152cbd5065a6e100c5420b4b5c62386bb2d303eb1734e074b09625013e55e3bd8631cfb3582d70e1c diff --git a/sdk_container/src/third_party/portage-stable/sys-apps/bubblewrap/bubblewrap-0.8.0.ebuild b/sdk_container/src/third_party/portage-stable/sys-apps/bubblewrap/bubblewrap-0.8.0.ebuild new file mode 100644 index 0000000000..756847be4a --- /dev/null +++ b/sdk_container/src/third_party/portage-stable/sys-apps/bubblewrap/bubblewrap-0.8.0.ebuild @@ -0,0 +1,58 @@ +# Copyright 1999-2023 Gentoo Authors +# Distributed under the terms of the GNU General Public License v2 + +EAPI=8 + +inherit linux-info meson + +DESCRIPTION="Unprivileged sandboxing tool, namespaces-powered chroot-like solution" +HOMEPAGE="https://github.com/containers/bubblewrap/" +SRC_URI="https://github.com/containers/${PN}/releases/download/v${PV}/${P}.tar.xz" + +LICENSE="LGPL-2+" +SLOT="0" +KEYWORDS="amd64 arm arm64 ~loong ppc ppc64 ~riscv x86" +IUSE="selinux suid" + +RDEPEND=" + sys-libs/libseccomp + sys-libs/libcap + selinux? ( >=sys-libs/libselinux-2.1.9 ) +" +DEPEND="${RDEPEND}" +BDEPEND=" + app-text/docbook-xml-dtd:4.3 + app-text/docbook-xsl-stylesheets + dev-libs/libxslt + virtual/pkgconfig +" + +# tests require root privileges +RESTRICT="test" + +pkg_setup() { + if [[ ${MERGE_TYPE} != buildonly ]]; then + CONFIG_CHECK="~UTS_NS ~IPC_NS ~USER_NS ~PID_NS ~NET_NS" + linux-info_pkg_setup + fi +} + +src_configure() { + local emesonargs=( + -Dbash_completion=enabled + -Dman=enabled + -Dtests=false + -Dzsh_completion=enabled + $(meson_feature selinux) + ) + + meson_src_configure +} + +src_install() { + meson_src_install + + if use suid; then + chmod u+s "${ED}"/usr/bin/bwrap + fi +} diff --git a/sdk_container/src/third_party/portage-stable/sys-apps/bubblewrap/metadata.xml b/sdk_container/src/third_party/portage-stable/sys-apps/bubblewrap/metadata.xml new file mode 100644 index 0000000000..5c14b5b490 --- /dev/null +++ b/sdk_container/src/third_party/portage-stable/sys-apps/bubblewrap/metadata.xml @@ -0,0 +1,11 @@ + + + + + gnome@gentoo.org + Gentoo GNOME Desktop + + + containers/bubblewrap + + diff --git a/sdk_container/src/third_party/portage-stable/virtual/tmpfiles/tmpfiles-0-r1.ebuild b/sdk_container/src/third_party/portage-stable/virtual/tmpfiles/tmpfiles-0-r1.ebuild deleted file mode 100644 index ab126e516e..0000000000 --- a/sdk_container/src/third_party/portage-stable/virtual/tmpfiles/tmpfiles-0-r1.ebuild +++ /dev/null @@ -1,17 +0,0 @@ -# Copyright 1999-2021 Gentoo Authors -# Distributed under the terms of the GNU General Public License v2 - -EAPI=7 - -DESCRIPTION="Virtual to select between different tmpfiles.d handlers" -SLOT="0" -KEYWORDS="~alpha amd64 arm arm64 hppa ~ia64 ~m68k ~mips ppc ppc64 ~riscv ~s390 sparc x86 ~x64-cygwin ~amd64-linux ~x86-linux ~ppc-macos ~x64-macos ~sparc-solaris ~sparc64-solaris ~x64-solaris ~x86-solaris" - -RDEPEND=" - !prefix-guest? ( - || ( - sys-apps/systemd-tmpfiles - sys-apps/opentmpfiles - sys-apps/systemd - ) - )" diff --git a/sdk_container/src/third_party/portage-stable/virtual/tmpfiles/tmpfiles-0-r5.ebuild b/sdk_container/src/third_party/portage-stable/virtual/tmpfiles/tmpfiles-0-r5.ebuild new file mode 100644 index 0000000000..2d31b47c69 --- /dev/null +++ b/sdk_container/src/third_party/portage-stable/virtual/tmpfiles/tmpfiles-0-r5.ebuild @@ -0,0 +1,16 @@ +# Copyright 1999-2023 Gentoo Authors +# Distributed under the terms of the GNU General Public License v2 + +EAPI=8 + +DESCRIPTION="Virtual to select between different tmpfiles.d handlers" +SLOT="0" +KEYWORDS="~alpha amd64 arm arm64 hppa ~ia64 ~loong ~m68k ~mips ppc ppc64 ~riscv ~s390 sparc x86 ~amd64-linux ~x86-linux ~arm64-macos ~ppc-macos ~x64-macos ~x64-solaris" +IUSE="systemd" + +RDEPEND=" + !prefix-guest? ( + systemd? ( sys-apps/systemd ) + !systemd? ( sys-apps/systemd-utils[tmpfiles] ) + ) +" diff --git a/sdk_container/src/third_party/prefix-overlay/app-misc/ca-certificates/Manifest b/sdk_container/src/third_party/prefix-overlay/app-misc/ca-certificates/Manifest new file mode 100644 index 0000000000..205eb90064 --- /dev/null +++ b/sdk_container/src/third_party/prefix-overlay/app-misc/ca-certificates/Manifest @@ -0,0 +1,3 @@ +DIST ca-certificates_20230311.tar.xz 257772 BLAKE2B b807a6415126afdc11896efea8e6509d7ad58b26bc8562b276e93176e80bb8b467a5bd2ba948d3dbbeaf0e4477d93f3ea2b99d3186e856fb47d1033cb779d560 SHA512 00571bdc87897813fd7dbe024f3a186cfc9f0d4f55e92545a90888c9e5282f99cb8d75b5932c034731b911bf27a9b38fd7d062dd511eb1152acf8b2811490fa7 +DIST nss-3.93.tar.gz 72281331 BLAKE2B 99e50f450a451f2b0bc0aad9b0fba405c987d88546d4aad6c490cb43dc274f23eb99d03d5fa8cf7ef16585abebfdae942fe1092d3f1c86816ba35e16ed3d490f SHA512 d96f13a70e825b39efadfe7c973c24c1e5ad43319bd813599010383e2b8434181f53489672f68fe79e2cb0c4d4ea0088499e588c3524eccf9298aafc57b94951 +DIST nss-cacert-class1-class3-r2.patch 21925 BLAKE2B 7627ff9a09f084c19d72d0490676865e3cab3ca7c920ae1ce4bea2db664f37fd0aa84fcda919809a516891ab2a62e2e7a43a9d6ada4c231adfe4c216525fac7d SHA512 1ce6ff9ab310aaca9005eafb461338b291df8523cc7044e096cd75774ce746c26eed19ec6bb2643c6c67f94650f2f309463492d80a90568f38ce2557f8ada2f4 diff --git a/sdk_container/src/third_party/prefix-overlay/app-misc/ca-certificates/ca-certificates-20230311.3.93.ebuild b/sdk_container/src/third_party/prefix-overlay/app-misc/ca-certificates/ca-certificates-20230311.3.93.ebuild new file mode 100644 index 0000000000..cd5ede09c0 --- /dev/null +++ b/sdk_container/src/third_party/prefix-overlay/app-misc/ca-certificates/ca-certificates-20230311.3.93.ebuild @@ -0,0 +1,206 @@ +# Copyright 1999-2023 Gentoo Authors +# Distributed under the terms of the GNU General Public License v2 + +# The Debian ca-certificates package merely takes the CA database as it exists +# in the nss package and repackages it for use by openssl. +# +# The issue with using the compiled debs directly is two fold: +# - they do not update frequently enough for us to rely on them +# - they pull the CA database from nss tip of tree rather than the release +# +# So we take the Debian source tools and combine them with the latest nss +# release to produce (largely) the same end result. The difference is that +# now we know our cert database is kept in sync with nss and, if need be, +# can be sync with nss tip of tree more frequently to respond to bugs. + +# Where possible, bump to stable/LTS releases of NSS for the last part +# of the version (when not using a pure Debian release). + +# When triaging user reports, refer to our wiki for tips: +# https://wiki.gentoo.org/wiki/Certificates#Debugging_certificate_issues + +EAPI=8 + +PYTHON_COMPAT=( python3_{10..12} ) + +inherit python-any-r1 + +if [[ ${PV} == *.* ]] ; then + # Compile from source ourselves. + PRECOMPILED=false + + DEB_VER=$(ver_cut 1) + NSS_VER=$(ver_cut 2-) + RTM_NAME="NSS_${NSS_VER//./_}_RTM" +else + # Debian precompiled version. + PRECOMPILED=true + inherit unpacker +fi + +DESCRIPTION="Common CA Certificates PEM files" +HOMEPAGE="https://packages.debian.org/sid/ca-certificates" +NMU_PR="" +if ${PRECOMPILED} ; then + SRC_URI="mirror://debian/pool/main/c/${PN}/${PN}_${PV}${NMU_PR:++nmu}${NMU_PR}_all.deb" +else + SRC_URI=" + mirror://debian/pool/main/c/${PN}/${PN}_${DEB_VER}${NMU_PR:++nmu}${NMU_PR}.tar.xz + https://archive.mozilla.org/pub/security/nss/releases/${RTM_NAME}/src/nss-${NSS_VER}.tar.gz + cacert? ( + https://dev.gentoo.org/~whissi/dist/ca-certificates/nss-cacert-class1-class3-r2.patch + ) + " +fi + +LICENSE="MPL-1.1" +SLOT="0" +KEYWORDS="~alpha ~amd64 ~arm ~arm64 ~hppa ~ia64 ~loong ~m68k ~mips ~ppc ~ppc64 ~riscv ~s390 ~sparc ~x86 ~amd64-linux ~x86-linux ~arm64-macos ~ppc-macos ~x64-macos ~x64-solaris" +IUSE="" +${PRECOMPILED} || IUSE+=" cacert" + +# c_rehash: we run `c_rehash` +# debianutils: we run `run-parts` +CDEPEND=" + sys-apps/debianutils" + +BDEPEND="${CDEPEND}" +if ! ${PRECOMPILED} ; then + BDEPEND+=" ${PYTHON_DEPS}" +fi + +DEPEND="" +if ${PRECOMPILED} ; then + DEPEND+=" !/dev/null || die + eapply "${DISTDIR}"/nss-cacert-class1-class3-r2.patch + popd >/dev/null || die + fi + fi + + default + eapply -p2 "${FILESDIR}"/${PN}-20150426-root.patch + + pushd "${S}/${PN}" >/dev/null || die + # We patch out the dep on cryptography as it's not particularly useful + # for us. Please see the discussion in bug #821706. Not to be removed lightly! + eapply "${FILESDIR}"/${PN}-20230311.3.89-no-cryptography.patch + popd >/dev/null || die + + local relp=$(echo "${EPREFIX}" | sed -e 's:[^/]\+:..:g') + sed -i \ + -e '/="$ROOT/s:ROOT:ROOT'"${EPREFIX}"':' \ + -e '/RELPATH="\.\./s:"$:'"${relp}"'":' \ + usr/sbin/update-ca-certificates || die +} + +src_compile() { + cd "image/${EPREFIX}" || die + + if ! ${PRECOMPILED} ; then + local d="${S}/${PN}/mozilla" c="usr/share/${PN}" + + # Grab the database from the nss sources. + cp "${S}"/nss-${NSS_VER}/nss/lib/ckfw/builtins/{certdata.txt,nssckbi.h} "${d}" || die + emake -C "${d}" + + # Now move the files to the same places that the precompiled would. + mkdir -p etc/ssl/certs \ + etc/ca-certificates/update.d \ + "${c}"/mozilla \ + || die + if use cacert ; then + mkdir -p "${c}"/cacert.org || die + mv "${d}"/CA_Cert_Signing_Authority.crt \ + "${c}"/cacert.org/cacert.org_class1.crt || die + mv "${d}"/CAcert_Class_3_Root.crt \ + "${c}"/cacert.org/cacert.org_class3.crt || die + fi + mv "${d}"/*.crt "${c}"/mozilla/ || die + else + mv usr/share/doc/{ca-certificates,${PF}} || die + fi + + ( + echo "# Automatically generated by ${CATEGORY}/${PF}" + echo "# $(date -u)" + echo "# Do not edit." + cd "${c}" || die + find * -name '*.crt' | LC_ALL=C sort + ) > etc/ca-certificates.conf + + sh usr/sbin/update-ca-certificates --root "${S}/image" || die +} + +src_install() { + cp -pPR image/* "${D}"/ || die + if ! ${PRECOMPILED} ; then + cd ${PN} || die + doman sbin/*.8 + dodoc debian/README.* examples/ca-certificates-local/README + fi + + echo 'CONFIG_PROTECT_MASK="/etc/ca-certificates.conf"' > 98ca-certificates || die + doenvd 98ca-certificates +} + +pkg_postinst() { + if [[ -d "${EROOT}/usr/local/share/ca-certificates" ]] ; then + # If the user has local certs, we need to rebuild again + # to include their stuff in the db. + # However it's too overzealous when the user has custom certs in place. + # --fresh is to clean up dangling symlinks + "${EROOT}"/usr/sbin/update-ca-certificates --root "${ROOT}" + fi + + if [[ -n "$(find -L "${EROOT}"/etc/ssl/certs/ -type l)" ]] ; then + ewarn "Removing the following broken symlinks:" + ewarn "$(find -L "${EROOT}"/etc/ssl/certs/ -type l -printf '%p -> %l\n' -delete)" + fi +} diff --git a/sdk_container/src/third_party/prefix-overlay/app-misc/ca-certificates/files/ca-certificates-20150426-root.patch b/sdk_container/src/third_party/prefix-overlay/app-misc/ca-certificates/files/ca-certificates-20150426-root.patch new file mode 100644 index 0000000000..6e41ac39e8 --- /dev/null +++ b/sdk_container/src/third_party/prefix-overlay/app-misc/ca-certificates/files/ca-certificates-20150426-root.patch @@ -0,0 +1,49 @@ +add a --root option so we can generate with DESTDIR installs + +--- a/image/usr/sbin/update-ca-certificates ++++ b/image/usr/sbin/update-ca-certificates +@@ -30,6 +30,8 @@ LOCALCERTSDIR=/usr/local/share/ca-certificates + CERTBUNDLE=ca-certificates.crt + ETCCERTSDIR=/etc/ssl/certs + HOOKSDIR=/etc/ca-certificates/update.d ++ROOT="" ++RELPATH="" + + while [ $# -gt 0 ]; + do +@@ -59,13 +61,25 @@ do + --hooksdir) + shift + HOOKSDIR="$1";; ++ --root|-r) ++ shift ++ # Needed as c_rehash wants to read the files directly. ++ # This gets us from $CERTSCONF to $CERTSDIR. ++ RELPATH="../../.." ++ ROOT=$(readlink -f "$1");; + --help|-h|*) +- echo "$0: [--verbose] [--fresh]" ++ echo "$0: [--verbose] [--fresh] [--root ]" + exit;; + esac + shift + done + ++CERTSCONF="$ROOT$CERTSCONF" ++CERTSDIR="$ROOT$CERTSDIR" ++LOCALCERTSDIR="$ROOT$LOCALCERTSDIR" ++ETCCERTSDIR="$ROOT$ETCCERTSDIR" ++HOOKSDIR="$ROOT$HOOKSDIR" ++ + if [ ! -s "$CERTSCONF" ] + then + fresh=1 +@@ -94,7 +107,7 @@ add() { + -e 's/,/_/g').pem" + if ! test -e "$PEM" || [ "$(readlink "$PEM")" != "$CERT" ] + then +- ln -sf "$CERT" "$PEM" ++ ln -sf "${RELPATH}${CERT#$ROOT}" "$PEM" + echo "+$PEM" >> "$ADDED" + fi + # Add trailing newline to certificate, if it is missing (#635570) diff --git a/sdk_container/src/third_party/prefix-overlay/app-misc/ca-certificates/files/ca-certificates-20230311.3.89-no-cryptography.patch b/sdk_container/src/third_party/prefix-overlay/app-misc/ca-certificates/files/ca-certificates-20230311.3.89-no-cryptography.patch new file mode 100644 index 0000000000..1f27603e8f --- /dev/null +++ b/sdk_container/src/third_party/prefix-overlay/app-misc/ca-certificates/files/ca-certificates-20230311.3.89-no-cryptography.patch @@ -0,0 +1,25 @@ +Remove the dependency on non-portable dev-python/cryptography. +https://bugs.gentoo.org/821706#c4 by Alex Xu +--- a/mozilla/certdata2pem.py ++++ b/mozilla/certdata2pem.py +@@ -28,7 +28,6 @@ import sys + import textwrap + import io + +-from cryptography import x509 + + + objects = [] +@@ -122,12 +121,6 @@ for obj in objects: + if not obj['CKA_LABEL'] in trust or not trust[obj['CKA_LABEL']]: + continue + +- cert = x509.load_der_x509_certificate(bytes(obj['CKA_VALUE'])) +- if cert.not_valid_after < datetime.datetime.utcnow(): +- print('!'*74) +- print('Trusted but expired certificate found: %s' % obj['CKA_LABEL']) +- print('!'*74) +- + bname = obj['CKA_LABEL'][1:-1].replace('/', '_')\ + .replace(' ', '_')\ + .replace('(', '=')\ diff --git a/sdk_container/src/third_party/prefix-overlay/app-misc/ca-certificates/metadata.xml b/sdk_container/src/third_party/prefix-overlay/app-misc/ca-certificates/metadata.xml new file mode 100644 index 0000000000..250e30768a --- /dev/null +++ b/sdk_container/src/third_party/prefix-overlay/app-misc/ca-certificates/metadata.xml @@ -0,0 +1,13 @@ + + + + + base-system@gentoo.org + Gentoo Base System + + + + Include root/class3 certs from CAcert (https://www.cacert.org/) + + + diff --git a/sdk_container/src/third_party/prefix-overlay/dev-libs/libgpg-error/Manifest b/sdk_container/src/third_party/prefix-overlay/dev-libs/libgpg-error/Manifest new file mode 100644 index 0000000000..d4f01bcd4f --- /dev/null +++ b/sdk_container/src/third_party/prefix-overlay/dev-libs/libgpg-error/Manifest @@ -0,0 +1,2 @@ +DIST libgpg-error-1.47.tar.bz2 1020862 BLAKE2B bc04efa0686b1b7d7cdce045fc080c090c1abec60349b673c2e1ce27900483aea090eb6ebcb3fb49a4eed36f18156a12413d5446f739475632f4ed2a2481ff27 SHA512 bbb4b15dae75856ee5b1253568674b56ad155524ae29a075cb5b0a7e74c4af685131775c3ea2226fff2f84ef80855e77aa661645d002b490a795c7ae57b66a30 +DIST libgpg-error-1.47.tar.bz2.sig 119 BLAKE2B d23ea6c38621407c8f9f0c6bde71abd0e50c136d2e5de9a6cef64627f5d398c344a3438995a2405c4ef148ad8638ef7125f34670819957acd7d597370f1630e5 SHA512 09343016eaf7fcc455f8ce533847153a8a9b7c36f375a8ebe71ef5fc2923edf7b70842f834f52c51874e427869487b74a2286ea0112cffad0d72f79cb6d4eceb diff --git a/sdk_container/src/third_party/prefix-overlay/dev-libs/libgpg-error/files/libgpg-error-1.44-remove_broken_check.patch b/sdk_container/src/third_party/prefix-overlay/dev-libs/libgpg-error/files/libgpg-error-1.44-remove_broken_check.patch new file mode 100644 index 0000000000..043099c097 --- /dev/null +++ b/sdk_container/src/third_party/prefix-overlay/dev-libs/libgpg-error/files/libgpg-error-1.44-remove_broken_check.patch @@ -0,0 +1,22 @@ +This breaks our multilib builds: + + Confirm gpg-error-config works... no + *** Please report to with gpg-error-config-test.log + +--- libgpg-error-1.44/src/Makefile.am ++++ libgpg-error-1.44/src/Makefile.am +@@ -347,14 +347,6 @@ + cp gpg-error.h gpgrt.h + + gpg-error-config: gpgrt-config gpg-error-config-old gpg-error-config-test.sh +- @echo $(ECHO_N) "Confirm gpg-error-config works... $(ECHO_C)" +- @if ./gpg-error-config-test.sh --old-new; then \ +- echo "good"; \ +- else \ +- echo "no"; \ +- echo "*** Please report to with gpg-error-config-test.log"; \ +- exit 1; \ +- fi + cp gpg-error-config-old $@ + + install-data-local: diff --git a/sdk_container/src/third_party/prefix-overlay/dev-libs/libgpg-error/libgpg-error-1.47.ebuild b/sdk_container/src/third_party/prefix-overlay/dev-libs/libgpg-error/libgpg-error-1.47.ebuild new file mode 100644 index 0000000000..69878152eb --- /dev/null +++ b/sdk_container/src/third_party/prefix-overlay/dev-libs/libgpg-error/libgpg-error-1.47.ebuild @@ -0,0 +1,88 @@ +# Copyright 1999-2023 Gentoo Authors +# Distributed under the terms of the GNU General Public License v2 + +EAPI=8 + +# Maintainers should: +# 1. Join the "Gentoo" project at https://dev.gnupg.org/project/view/27/ +# 2. Subscribe to release tasks like https://dev.gnupg.org/T6159 +# (find the one for the current release then subscribe to it + +# any subsequent ones linked within so you're covered for a while.) + +VERIFY_SIG_OPENPGP_KEY_PATH="${BROOT}"/usr/share/openpgp-keys/gnupg.asc +inherit autotools multilib-minimal toolchain-funcs prefix verify-sig + +DESCRIPTION="Contains error handling functions used by GnuPG software" +HOMEPAGE="https://www.gnupg.org/related_software/libgpg-error" +SRC_URI="mirror://gnupg/${PN}/${P}.tar.bz2" +SRC_URI+=" verify-sig? ( mirror://gnupg/${PN}/${P}.tar.bz2.sig )" + +LICENSE="GPL-2 LGPL-2.1" +SLOT="0" +KEYWORDS="~alpha amd64 arm arm64 hppa ~ia64 ~loong ~m68k ~mips ppc ppc64 ~riscv ~s390 sparc x86 ~amd64-linux ~x86-linux ~arm64-macos ~ppc-macos ~x64-macos ~x64-solaris" +IUSE="common-lisp nls static-libs test" +RESTRICT="!test? ( test )" + +RDEPEND="nls? ( >=virtual/libintl-0-r1[${MULTILIB_USEDEP}] )" +DEPEND="${RDEPEND}" +BDEPEND=" + nls? ( sys-devel/gettext ) + verify-sig? ( sec-keys/openpgp-keys-gnupg ) +" + +MULTILIB_WRAPPED_HEADERS=( + /usr/include/gpg-error.h + /usr/include/gpgrt.h +) + +MULTILIB_CHOST_TOOLS=( + /usr/bin/gpg-error-config + /usr/bin/gpgrt-config +) + +PATCHES=( + "${FILESDIR}/${PN}-1.44-remove_broken_check.patch" +) + +src_prepare() { + default + + if use prefix ; then + # don't hardcode /usr/xpg4/bin/sh as shell on Solaris + sed -i -e 's/solaris\*/disabled/' configure.ac || die + fi + + # only necessary for as long as we run eautoreconf, configure.ac + # uses ./autogen.sh to generate PACKAGE_VERSION, but autogen.sh is + # not a pure /bin/sh script, so it fails on some hosts + # Flatcar / t-lo 2023-09-27: pull in upstream fix https://github.com/gentoo/gentoo/pull/33010 + # for prefix builds. + sed -i -e "1s:.*:#\!${BASH}:" autogen.sh || die + eautoreconf +} + +multilib_src_configure() { + local myeconfargs=( + $(multilib_is_native_abi || echo --disable-languages) + $(use_enable common-lisp languages) + $(use_enable nls) + # required for sys-power/suspend[crypt], bug 751568 + $(use_enable static-libs static) + $(use_enable test tests) + + # See bug #699206 and its duplicates wrt gpgme-config + # Upstream no longer install this by default and we should + # seek to disable it at some point. + --enable-install-gpg-error-config + + --enable-threads + CC_FOR_BUILD="$(tc-getBUILD_CC)" + $("${S}/configure" --help | grep -o -- '--without-.*-prefix') + ) + ECONF_SOURCE="${S}" econf "${myeconfargs[@]}" +} + +multilib_src_install_all() { + einstalldocs + find "${ED}" -type f -name '*.la' -delete || die +} diff --git a/sdk_container/src/third_party/prefix-overlay/dev-libs/libgpg-error/metadata.xml b/sdk_container/src/third_party/prefix-overlay/dev-libs/libgpg-error/metadata.xml new file mode 100644 index 0000000000..0c03a7f547 --- /dev/null +++ b/sdk_container/src/third_party/prefix-overlay/dev-libs/libgpg-error/metadata.xml @@ -0,0 +1,14 @@ + + + + + base-system@gentoo.org + Gentoo Base System + + + Install common-lisp files + + + cpe:/a:gnupg:libgpg-error + + diff --git a/sdk_container/src/third_party/prefix-overlay/eclass/distutils-r1.eclass b/sdk_container/src/third_party/prefix-overlay/eclass/distutils-r1.eclass new file mode 100644 index 0000000000..5a1bf4ff13 --- /dev/null +++ b/sdk_container/src/third_party/prefix-overlay/eclass/distutils-r1.eclass @@ -0,0 +1,2202 @@ +# Copyright 1999-2023 Gentoo Authors +# Distributed under the terms of the GNU General Public License v2 + +# @ECLASS: distutils-r1.eclass +# @MAINTAINER: +# Python team +# @AUTHOR: +# Author: Michał Górny +# Based on the work of: Krzysztof Pawlik +# @SUPPORTED_EAPIS: 7 8 +# @PROVIDES: python-r1 python-single-r1 +# @BLURB: A simple eclass to build Python packages using distutils. +# @DESCRIPTION: +# A simple eclass providing functions to build Python packages using +# the distutils build system. It exports phase functions for all +# the src_* phases. Each of the phases runs two pseudo-phases: +# python_..._all() (e.g. python_prepare_all()) once in ${S}, then +# python_...() (e.g. python_prepare()) for each implementation +# (see: python_foreach_impl() in python-r1). +# +# In distutils-r1_src_prepare(), the 'all' function is run before +# per-implementation ones (because it creates the implementations), +# per-implementation functions are run in a random order. +# +# In remaining phase functions, the per-implementation functions are run +# before the 'all' one, and they are ordered from the least to the most +# preferred implementation (so that 'better' files overwrite 'worse' +# ones). +# +# If the ebuild doesn't specify a particular pseudo-phase function, +# the default one will be used (distutils-r1_...). Defaults are provided +# for all per-implementation pseudo-phases, python_prepare_all() +# and python_install_all(); whenever writing your own pseudo-phase +# functions, you should consider calling the defaults (and especially +# distutils-r1_python_prepare_all). +# +# Please note that distutils-r1 sets RDEPEND and BDEPEND (or DEPEND +# in earlier EAPIs) unconditionally for you. +# +# Also, please note that distutils-r1 will always inherit python-r1 +# as well. Thus, all the variables defined and documented there are +# relevant to the packages using distutils-r1. +# +# For more information, please see the Python Guide: +# https://projects.gentoo.org/python/guide/ + +case ${EAPI} in + 7|8) ;; + *) die "${ECLASS}: EAPI ${EAPI:-0} not supported" ;; +esac + +# @ECLASS_VARIABLE: DISTUTILS_EXT +# @DEFAULT_UNSET +# @DESCRIPTION: +# Set this variable to a non-null value if the package (possibly +# optionally) builds Python extensions (loadable modules written in C, +# Cython, Rust, etc.). +# +# When enabled, the eclass: +# +# - adds PYTHON_DEPS to DEPEND (for cross-compilation support), unless +# DISTUTILS_OPTIONAL is used +# +# - adds `debug` flag to IUSE that controls assertions (i.e. -DNDEBUG) +# +# - calls `build_ext` command if setuptools build backend is used +# and there is potential benefit from parallel builds + +# @ECLASS_VARIABLE: DISTUTILS_OPTIONAL +# @DEFAULT_UNSET +# @DESCRIPTION: +# If set to a non-null value, distutils part in the ebuild will +# be considered optional. No dependencies will be added and no phase +# functions will be exported. +# +# If you enable DISTUTILS_OPTIONAL, you have to set proper dependencies +# for your package (using ${PYTHON_DEPS}) and to either call +# distutils-r1 default phase functions or call the build system +# manually. + +# @ECLASS_VARIABLE: DISTUTILS_SINGLE_IMPL +# @DEFAULT_UNSET +# @DESCRIPTION: +# If set to a non-null value, the ebuild will support setting a single +# Python implementation only. It will effectively replace the python-r1 +# eclass inherit with python-single-r1. +# +# Note that inheriting python-single-r1 will cause pkg_setup() +# to be exported. It must be run in order for the eclass functions +# to function properly. + +# @ECLASS_VARIABLE: DISTUTILS_USE_PEP517 +# @PRE_INHERIT +# @DEFAULT_UNSET +# @DESCRIPTION: +# Enable the PEP 517 mode for the specified build system. In this mode, +# the complete build and install is done in python_compile(), +# a venv-style install tree is provided to python_test(), +# and python_install() just merges the temporary install tree +# into the real fs. +# +# This mode is recommended for Python packages. However, some packages +# using custom hacks on top of distutils/setuptools may not install +# correctly in this mode. Please verify the list of installed files +# when using it. +# +# The variable specifies the build system used. Currently, +# the following values are supported: +# +# - flit - flit-core backend +# +# - flit_scm - flit_scm backend +# +# - hatchling - hatchling backend (from hatch) +# +# - jupyter - jupyter_packaging backend +# +# - maturin - maturin backend +# +# - meson-python - meson-python (mesonpy) backend +# +# - no - no PEP517 build system (see below) +# +# - pbr - pbr backend +# +# - pdm - pdm.pep517 backend +# +# - pdm-backend - pdm.backend backend +# +# - poetry - poetry-core backend +# +# - setuptools - distutils or setuptools (incl. legacy mode) +# +# - sip - sipbuild backend +# +# - standalone - standalone build systems without external deps +# (used for bootstrapping). +# +# The variable needs to be set before the inherit line. The eclass +# adds appropriate build-time dependencies and verifies the value. +# +# The special value "no" indicates that the package has no build system. +# This is not equivalent to unset DISTUTILS_USE_PEP517 (legacy mode). +# It causes the eclass not to include any build system dependencies +# and to disable default python_compile() and python_install() +# implementations. Baseline Python deps and phase functions will still +# be set (depending on the value of DISTUTILS_OPTIONAL). Most of +# the other eclass functions will work. Testing venv will be provided +# in ${BUILD_DIR}/install after python_compile(), and if any (other) +# files are found in ${BUILD_DIR}/install after python_install(), they +# will be merged into ${D}. + +# @ECLASS_VARIABLE: DISTUTILS_USE_SETUPTOOLS +# @DEFAULT_UNSET +# @PRE_INHERIT +# @DESCRIPTION: +# Controls adding dev-python/setuptools dependency. The allowed values +# are: +# +# - no -- do not add the dependency (pure distutils package) +# +# - bdepend -- add it to BDEPEND (the default) +# +# - rdepend -- add it to BDEPEND+RDEPEND (e.g. when using pkg_resources) +# +# - manual -- do not add the dependency and suppress the checks +# (assumes you will take care of doing it correctly) +# +# This variable is effective only if DISTUTILS_OPTIONAL is disabled. +# It is available only in non-PEP517 mode. It needs to be set before +# the inherit line. + +# @ECLASS_VARIABLE: DISTUTILS_DEPS +# @OUTPUT_VARIABLE +# @DESCRIPTION: +# This is an eclass-generated build-time dependency string for the build +# system packages. This string is automatically appended to BDEPEND +# unless DISTUTILS_OPTIONAL is used. This variable is available only +# in PEP 517 mode. +# +# Example use: +# @CODE +# DISTUTILS_OPTIONAL=1 +# # ... +# RDEPEND="${PYTHON_DEPS}" +# BDEPEND=" +# ${PYTHON_DEPS} +# ${DISTUTILS_DEPS}" +# @CODE + +if [[ -z ${_DISTUTILS_R1_ECLASS} ]]; then +_DISTUTILS_R1_ECLASS=1 + +inherit flag-o-matic +inherit multibuild multilib multiprocessing ninja-utils toolchain-funcs + +if [[ ! ${DISTUTILS_SINGLE_IMPL} ]]; then + inherit python-r1 +else + inherit python-single-r1 +fi + +_distutils_set_globals() { + local rdep bdep + if [[ ${DISTUTILS_USE_PEP517} ]]; then + if [[ ${DISTUTILS_USE_SETUPTOOLS} ]]; then + die "DISTUTILS_USE_SETUPTOOLS is not used in PEP517 mode" + fi + + bdep=' + >=dev-python/gpep517-13[${PYTHON_USEDEP}] + ' + case ${DISTUTILS_USE_PEP517} in + flit) + bdep+=' + >=dev-python/flit-core-3.9.0[${PYTHON_USEDEP}] + ' + ;; + flit_scm) + bdep+=' + >=dev-python/flit_scm-1.7.0[${PYTHON_USEDEP}] + ' + ;; + hatchling) + bdep+=' + >=dev-python/hatchling-1.17.0[${PYTHON_USEDEP}] + ' + ;; + jupyter) + bdep+=' + >=dev-python/jupyter-packaging-0.12.3[${PYTHON_USEDEP}] + ' + ;; + maturin) + bdep+=' + >=dev-util/maturin-1.0.1[${PYTHON_USEDEP}] + ' + ;; + no) + # undo the generic deps added above + bdep= + ;; + meson-python) + bdep+=' + >=dev-python/meson-python-0.13.1[${PYTHON_USEDEP}] + ' + ;; + pbr) + bdep+=' + >=dev-python/pbr-5.11.1[${PYTHON_USEDEP}] + ' + ;; + pdm) + bdep+=' + >=dev-python/pdm-pep517-1.1.4[${PYTHON_USEDEP}] + ' + ;; + pdm-backend) + bdep+=' + >=dev-python/pdm-backend-2.1.0[${PYTHON_USEDEP}] + ' + ;; + poetry) + bdep+=' + >=dev-python/poetry-core-1.6.1[${PYTHON_USEDEP}] + ' + ;; + scikit-build-core) + bdep+=' + >=dev-python/scikit-build-core-0.4.6[${PYTHON_USEDEP}] + ' + ;; + setuptools) + bdep+=' + >=dev-python/setuptools-67.8.0-r1[${PYTHON_USEDEP}] + ' + ;; + sip) + bdep+=' + >=dev-python/sip-6.7.9[${PYTHON_USEDEP}] + ' + ;; + standalone) + ;; + *) + die "Unknown DISTUTILS_USE_PEP517=${DISTUTILS_USE_PEP517}" + ;; + esac + elif [[ ${DISTUTILS_OPTIONAL} ]]; then + if [[ ${DISTUTILS_USE_SETUPTOOLS} ]]; then + eqawarn "QA Notice: DISTUTILS_USE_SETUPTOOLS is not used when DISTUTILS_OPTIONAL" + eqawarn "is enabled." + fi + else + local setuptools_dep='>=dev-python/setuptools-67.8.0-r1[${PYTHON_USEDEP}]' + + case ${DISTUTILS_USE_SETUPTOOLS:-bdepend} in + no|manual) + ;; + bdepend) + bdep+=" ${setuptools_dep}" + ;; + rdepend) + bdep+=" ${setuptools_dep}" + rdep+=" ${setuptools_dep}" + ;; + pyproject.toml) + die "DISTUTILS_USE_SETUPTOOLS=pyproject.toml is no longer supported, use DISTUTILS_USE_PEP517" + ;; + *) + die "Invalid DISTUTILS_USE_SETUPTOOLS=${DISTUTILS_USE_SETUPTOOLS}" + ;; + esac + fi + + if [[ ! ${DISTUTILS_SINGLE_IMPL} ]]; then + bdep=${bdep//\$\{PYTHON_USEDEP\}/${PYTHON_USEDEP}} + rdep=${rdep//\$\{PYTHON_USEDEP\}/${PYTHON_USEDEP}} + else + [[ -n ${bdep} ]] && bdep="$(python_gen_cond_dep "${bdep}")" + [[ -n ${rdep} ]] && rdep="$(python_gen_cond_dep "${rdep}")" + fi + + if [[ ${DISTUTILS_USE_PEP517} ]]; then + if [[ ${DISTUTILS_DEPS+1} ]]; then + if [[ ${DISTUTILS_DEPS} != "${bdep}" ]]; then + eerror "DISTUTILS_DEPS have changed between inherits!" + eerror "Before: ${DISTUTILS_DEPS}" + eerror "Now : ${bdep}" + die "DISTUTILS_DEPS integrity check failed" + fi + else + DISTUTILS_DEPS=${bdep} + readonly DISTUTILS_DEPS + fi + fi + + if [[ ! ${DISTUTILS_OPTIONAL} ]]; then + RDEPEND="${PYTHON_DEPS} ${rdep}" + BDEPEND="${PYTHON_DEPS} ${bdep}" + REQUIRED_USE=${PYTHON_REQUIRED_USE} + + if [[ ${DISTUTILS_EXT} ]]; then + DEPEND="${PYTHON_DEPS}" + fi + fi + + if [[ ${DISTUTILS_EXT} ]]; then + IUSE="debug" + fi +} +_distutils_set_globals +unset -f _distutils_set_globals + +# @ECLASS_VARIABLE: PATCHES +# @DEFAULT_UNSET +# @DESCRIPTION: +# An array containing patches to be applied to the sources before +# copying them. +# +# If unset, no custom patches will be applied. +# +# Please note, however, that at some point the eclass may apply +# additional distutils patches/quirks independently of this variable. +# +# Example: +# @CODE +# PATCHES=( "${FILESDIR}"/${P}-make-gentoo-happy.patch ) +# @CODE + +# @ECLASS_VARIABLE: DOCS +# @DEFAULT_UNSET +# @DESCRIPTION: +# An array containing documents installed using dodoc. The files listed +# there must exist in the directory from which +# distutils-r1_python_install_all() is run (${S} by default). +# +# If unset, the function will instead look up files matching default +# filename pattern list (from the Package Manager Specification), +# and install those found. +# +# Example: +# @CODE +# DOCS=( NEWS README ) +# @CODE + +# @ECLASS_VARIABLE: HTML_DOCS +# @DEFAULT_UNSET +# @DESCRIPTION: +# An array containing documents installed using dohtml. The files +# and directories listed there must exist in the directory from which +# distutils-r1_python_install_all() is run (${S} by default). +# +# If unset, no HTML docs will be installed. +# +# Example: +# @CODE +# HTML_DOCS=( doc/html/. ) +# @CODE + +# @ECLASS_VARIABLE: DISTUTILS_IN_SOURCE_BUILD +# @DEFAULT_UNSET +# @DESCRIPTION: +# If set to a non-null value, in-source builds will be enabled. +# If unset, the default is to use in-source builds when python_prepare() +# is declared, and out-of-source builds otherwise. +# +# If in-source builds are used, the eclass will create a copy of package +# sources for each Python implementation in python_prepare_all(), +# and work on that copy afterwards. +# +# If out-of-source builds are used, the eclass will instead work +# on the sources directly, prepending setup.py arguments with +# 'build --build-base ${BUILD_DIR}' to enforce keeping & using built +# files in the specific root. + +# @ECLASS_VARIABLE: DISTUTILS_ALL_SUBPHASE_IMPLS +# @DEFAULT_UNSET +# @DESCRIPTION: +# An array of patterns specifying which implementations can be used +# for *_all() sub-phase functions. If undefined, defaults to '*' +# (allowing any implementation). If multiple values are specified, +# implementations matching any of the patterns will be accepted. +# +# For the pattern syntax, please see _python_impl_matches +# in python-utils-r1.eclass. +# +# If the restriction needs to apply conditionally to a USE flag, +# the variable should be set conditionally as well (e.g. in an early +# phase function or other convenient location). +# +# Please remember to add a matching || block to REQUIRED_USE, +# to ensure that at least one implementation matching the patterns will +# be enabled. +# +# Example: +# @CODE +# REQUIRED_USE="doc? ( || ( $(python_gen_useflags 'python2*') ) )" +# +# pkg_setup() { +# use doc && DISTUTILS_ALL_SUBPHASE_IMPLS=( 'python2*' ) +# } +# @CODE + +# @ECLASS_VARIABLE: DISTUTILS_ARGS +# @DEFAULT_UNSET +# @DESCRIPTION: +# An array containing options to be passed to the build system. +# Supported by a subset of build systems used by the eclass. +# +# For setuptools, the arguments will be passed as first parameters +# to setup.py invocations (via esetup.py), as well as to the PEP517 +# backend. For future compatibility, only global options should be used +# and specifying commands should be avoided. +# +# For sip, the options are passed to the PEP517 backend in a form +# resembling sip-build calls. Options taking arguments need to +# be specified in the "--key=value" form, while flag options as "--key". +# If an option takes multiple arguments, it can be specified multiple +# times, same as for sip-build. +# +# Example: +# @CODE +# python_configure_all() { +# DISTUTILS_ARGS=( --enable-my-hidden-option ) +# } +# @CODE + +# @FUNCTION: distutils_enable_sphinx +# @USAGE: [--no-autodoc | ...] +# @DESCRIPTION: +# Set up IUSE, BDEPEND, python_check_deps() and python_compile_all() for +# building HTML docs via dev-python/sphinx. python_compile_all() will +# append to HTML_DOCS if docs are enabled. +# +# This helper is meant for the most common case, that is a single Sphinx +# subdirectory with standard layout, building and installing HTML docs +# behind USE=doc. It assumes it's the only consumer of the three +# aforementioned functions. If you need to use a custom implementation, +# you can't use it. +# +# If your package uses additional Sphinx plugins, they should be passed +# (without PYTHON_USEDEP) as . The function will take care +# of setting appropriate any-of dep and python_check_deps(). +# +# If no plugin packages are specified, the eclass will still utilize +# any-r1 API to support autodoc (documenting source code). +# If the package uses neither autodoc nor additional plugins, you should +# pass --no-autodoc to disable this API and simplify the resulting code. +# +# This function must be called in global scope. Take care not to +# overwrite the variables set by it. If you need to extend +# python_compile_all(), you can call the original implementation +# as sphinx_compile_all. +distutils_enable_sphinx() { + debug-print-function ${FUNCNAME} "${@}" + [[ ${#} -ge 1 ]] || die "${FUNCNAME} takes at least one arg: " + + _DISTUTILS_SPHINX_SUBDIR=${1} + shift + _DISTUTILS_SPHINX_PLUGINS=( "${@}" ) + + local deps autodoc=1 d + deps=">=dev-python/sphinx-5.3.0[\${PYTHON_USEDEP}]" + for d; do + if [[ ${d} == --no-autodoc ]]; then + autodoc= + else + deps+=" + ${d}[\${PYTHON_USEDEP}]" + if [[ ! ${autodoc} ]]; then + die "${FUNCNAME}: do not pass --no-autodoc if external plugins are used" + fi + fi + done + + if [[ ${autodoc} ]]; then + if [[ ${DISTUTILS_SINGLE_IMPL} ]]; then + deps="$(python_gen_cond_dep "${deps}")" + else + deps="$(python_gen_any_dep "${deps}")" + fi + + python_check_deps() { + use doc || return 0 + + local p + for p in ">=dev-python/sphinx-5.3.0" \ + "${_DISTUTILS_SPHINX_PLUGINS[@]}" + do + python_has_version "${p}[${PYTHON_USEDEP}]" || + return 1 + done + } + else + deps=">=dev-python/sphinx-5.3.0" + fi + + sphinx_compile_all() { + use doc || return + + local confpy=${_DISTUTILS_SPHINX_SUBDIR}/conf.py + [[ -f ${confpy} ]] || + die "${confpy} not found, distutils_enable_sphinx call wrong" + + if [[ ${_DISTUTILS_SPHINX_PLUGINS[0]} == --no-autodoc ]]; then + if grep -F -q 'sphinx.ext.autodoc' "${confpy}"; then + die "distutils_enable_sphinx: --no-autodoc passed but sphinx.ext.autodoc found in ${confpy}" + fi + elif [[ -z ${_DISTUTILS_SPHINX_PLUGINS[@]} ]]; then + if ! grep -F -q 'sphinx.ext.autodoc' "${confpy}"; then + die "distutils_enable_sphinx: sphinx.ext.autodoc not found in ${confpy}, pass --no-autodoc" + fi + fi + + build_sphinx "${_DISTUTILS_SPHINX_SUBDIR}" + } + python_compile_all() { sphinx_compile_all; } + + IUSE+=" doc" + BDEPEND+=" doc? ( ${deps} )" + + # we need to ensure successful return in case we're called last, + # otherwise Portage may wrongly assume sourcing failed + return 0 +} + +# @FUNCTION: distutils_enable_tests +# @USAGE: [--install] +# @DESCRIPTION: +# Set up IUSE, RESTRICT, BDEPEND and python_test() for running tests +# with the specified test runner. Also copies the current value +# of RDEPEND to test?-BDEPEND. The test-runner argument must be one of: +# +# - nose: nosetests (dev-python/nose) +# +# - pytest: dev-python/pytest +# +# - setup.py: setup.py test (no deps included) +# +# - unittest: for built-in Python unittest module +# +# Additionally, if --install is passed as the first parameter, +# 'distutils_install_for_testing --via-root' is called before running +# the test suite. +# +# This function is meant as a helper for common use cases, and it only +# takes care of basic setup. You still need to list additional test +# dependencies manually. If you have uncommon use case, you should +# not use it and instead enable tests manually. +# +# This function must be called in global scope, after RDEPEND has been +# declared. Take care not to overwrite the variables set by it. +distutils_enable_tests() { + debug-print-function ${FUNCNAME} "${@}" + + _DISTUTILS_TEST_INSTALL= + case ${1} in + --install) + if [[ ${DISTUTILS_USE_PEP517} ]]; then + die "${FUNCNAME} --install is not implemented in PEP517 mode" + fi + _DISTUTILS_TEST_INSTALL=1 + shift + ;; + esac + + [[ ${#} -eq 1 ]] || die "${FUNCNAME} takes exactly one argument: test-runner" + local test_pkg + case ${1} in + nose) + test_pkg=">=dev-python/nose-1.3.7_p20221026" + ;; + pytest) + test_pkg=">=dev-python/pytest-7.3.1" + ;; + setup.py) + ;; + unittest) + # dep handled below + ;; + *) + die "${FUNCNAME}: unsupported argument: ${1}" + esac + + _DISTUTILS_TEST_RUNNER=${1} + python_test() { distutils-r1_python_test; } + + local test_deps=${RDEPEND} + if [[ -n ${test_pkg} ]]; then + if [[ ! ${DISTUTILS_SINGLE_IMPL} ]]; then + test_deps+=" ${test_pkg}[${PYTHON_USEDEP}]" + else + test_deps+=" $(python_gen_cond_dep " + ${test_pkg}[\${PYTHON_USEDEP}] + ")" + fi + elif [[ ${1} == unittest ]]; then + # unittest-or-fail is needed in py<3.12 + test_deps+=" + $(python_gen_cond_dep ' + dev-python/unittest-or-fail[${PYTHON_USEDEP}] + ' 3.{9..11}) + " + fi + if [[ -n ${test_deps} ]]; then + IUSE+=" test" + RESTRICT+=" !test? ( test )" + BDEPEND+=" test? ( ${test_deps} )" + fi + + # we need to ensure successful return in case we're called last, + # otherwise Portage may wrongly assume sourcing failed + return 0 +} + +# @FUNCTION: esetup.py +# @USAGE: [...] +# @DESCRIPTION: +# Run setup.py using currently selected Python interpreter +# (if ${EPYTHON} is set; fallback 'python' otherwise). +# +# setup.py will be passed the following, in order: +# +# 1. ${DISTUTILS_ARGS[@]} +# +# 2. ${mydistutilsargs[@]} (deprecated) +# +# 3. additional arguments passed to the esetup.py function. +# +# Please note that setup.py will respect defaults (unless overridden +# via command-line options) from setup.cfg that is created +# in distutils-r1_python_compile and in distutils-r1_python_install. +# +# This command dies on failure. +esetup.py() { + debug-print-function ${FUNCNAME} "${@}" + + _python_check_EPYTHON + + if [[ ${BUILD_DIR} && ! ${DISTUTILS_USE_PEP517} ]]; then + _distutils-r1_create_setup_cfg + fi + + local setup_py=( setup.py ) + if [[ ! -f setup.py ]]; then + # The following call can succeed even if the package does not + # feature any setuptools configuration. In non-PEP517 mode this + # could lead to installing an "empty" package. In PEP517 mode, + # we verify the build system when invoking the backend, + # rendering this check redundant (and broken for projects using + # pyproject.toml configuration). + if [[ ! ${DISTUTILS_USE_PEP517} && ! -f setup.cfg ]]; then + die "${FUNCNAME}: setup.py nor setup.cfg not found" + fi + setup_py=( -c "from setuptools import setup; setup()" ) + fi + + if [[ ${EAPI} != 7 && ${mydistutilsargs[@]} ]]; then + die "mydistutilsargs is banned in EAPI ${EAPI} (use DISTUTILS_ARGS)" + fi + + set -- "${EPYTHON}" "${setup_py[@]}" "${DISTUTILS_ARGS[@]}" \ + "${mydistutilsargs[@]}" "${@}" + + echo "${@}" >&2 + "${@}" || die -n + local ret=${?} + + if [[ ${BUILD_DIR} && ! ${DISTUTILS_USE_PEP517} ]]; then + rm "${HOME}"/.pydistutils.cfg || die -n + fi + + return ${ret} +} + +# @FUNCTION: distutils_install_for_testing +# @USAGE: [--via-root|--via-home|--via-venv] [...] +# @DESCRIPTION: +# Install the package into a temporary location for running tests. +# Update PYTHONPATH appropriately and set TEST_DIR to the test +# installation root. The Python packages will be installed in 'lib' +# subdir, and scripts in 'scripts' subdir (like in BUILD_DIR). +# +# Please note that this function should be only used if package uses +# namespaces (and therefore proper install needs to be done to enforce +# PYTHONPATH) or tests rely on the results of install command. +# For most of the packages, tests built in BUILD_DIR are good enough. +# +# The function supports three install modes. These are: +# +# --via-root (the default) that uses 'setup.py install --root=...' +# combined with PYTHONPATH and is recommended for the majority +# of packages. +# +# --via-venv that creates a (non-isolated) venv and installs the package +# into it via 'setup.py install'. This mode does not use PYTHONPATH +# but requires python to be called via PATH. It may solve a few corner +# cases that --via-root do not support. +# +# --via-home that uses 'setup.py install --home=...'. This is +# a historical mode that was mostly broken by setuptools 50.3.0+. +# If your package does not work with the other two modes but works with +# this one, please report a bug. +# +# Please note that in order to test the solution properly you need +# to unmerge the package first. +# +# This function is not available in PEP517 mode. The eclass provides +# a venv-style install unconditionally and therefore it should no longer +# be necessary. +distutils_install_for_testing() { + debug-print-function ${FUNCNAME} "${@}" + + if [[ ${DISTUTILS_USE_PEP517} ]]; then + die "${FUNCNAME} is not implemented in PEP517 mode" + fi + + # A few notes about --via-home mode: + # 1) 'install --home' is terribly broken on pypy, so we need + # to override --install-lib and --install-scripts, + # 2) non-root 'install' complains about PYTHONPATH and missing dirs, + # so we need to set it properly and mkdir them, + # 3) it runs a bunch of commands which write random files to cwd, + # in order to avoid that, we add the necessary path overrides + # in _distutils-r1_create_setup_cfg. + + local install_method=root + case ${1} in + --via-home) + [[ ${EAPI} == 7 ]] || die "${*} is banned in EAPI ${EAPI}" + install_method=home + shift + ;; + --via-root) + install_method=root + shift + ;; + --via-venv) + install_method=venv + shift + ;; + esac + + TEST_DIR=${BUILD_DIR}/test + local add_args=() + + if [[ ${install_method} == venv ]]; then + # create a quasi-venv + mkdir -p "${TEST_DIR}"/bin || die + ln -s "${PYTHON}" "${TEST_DIR}/bin/${EPYTHON}" || die + ln -s "${EPYTHON}" "${TEST_DIR}/bin/python3" || die + ln -s "${EPYTHON}" "${TEST_DIR}/bin/python" || die + cat > "${TEST_DIR}"/pyvenv.cfg <<-EOF || die + include-system-site-packages = true + EOF + + # we only do the minimal necessary subset of activate script + PATH=${TEST_DIR}/bin:${PATH} + # unset PYTHONPATH in order to prevent BUILD_DIR from overriding + # venv packages + unset PYTHONPATH + + # force root-style install (note: venv adds TEST_DIR to prefixes, + # so we need to pass --root=/) + add_args=( + --root=/ + ) + else + local bindir=${TEST_DIR}/scripts + local libdir=${TEST_DIR}/lib + PATH=${bindir}:${PATH} + PYTHONPATH=${libdir}:${PYTHONPATH} + + case ${install_method} in + home) + add_args=( + --home="${TEST_DIR}" + --install-lib="${libdir}" + --install-scripts="${bindir}" + ) + mkdir -p "${libdir}" || die + ;; + root) + add_args=( + --root="${TEST_DIR}" + --install-lib=lib + --install-scripts=scripts + ) + ;; + esac + fi + + esetup.py install "${add_args[@]}" "${@}" +} + +# @FUNCTION: distutils_write_namespace +# @USAGE: ... +# @DESCRIPTION: +# Write the __init__.py file for the requested namespace into PEP517 +# install tree, in order to fix running tests when legacy namespace +# packages are installed (dev-python/namespace-*). +# +# This function must only be used in python_test(). The created file +# will automatically be removed upon leaving the test phase. +distutils_write_namespace() { + debug-print-function ${FUNCNAME} "${@}" + + if [[ ! ${DISTUTILS_USE_PEP517:-no} != no ]]; then + die "${FUNCNAME} is available only in PEP517 mode" + fi + if [[ ${EBUILD_PHASE} != test || ! ${BUILD_DIR} ]]; then + die "${FUNCNAME} should only be used in python_test" + fi + + local namespace + for namespace; do + if [[ ${namespace} == *[./]* ]]; then + die "${FUNCNAME} does not support nested namespaces at the moment" + fi + + local path=${BUILD_DIR}/install$(python_get_sitedir)/${namespace}/__init__.py + if [[ -f ${path} ]]; then + die "Requested namespace ${path} exists already!" + fi + cat > "${path}" <<-EOF || die + __path__ = __import__('pkgutil').extend_path(__path__, __name__) + EOF + _DISTUTILS_POST_PHASE_RM+=( "${path}" ) + done +} + +# @FUNCTION: _distutils-r1_disable_ez_setup +# @INTERNAL +# @DESCRIPTION: +# Stub out ez_setup.py and distribute_setup.py to prevent packages +# from trying to download a local copy of setuptools. +_distutils-r1_disable_ez_setup() { + if [[ ${DISTUTILS_USE_PEP517} ]]; then + die "${FUNCNAME} is not implemented in PEP517 mode" + fi + + local stub="def use_setuptools(*args, **kwargs): pass" + if [[ -f ez_setup.py ]]; then + echo "${stub}" > ez_setup.py || die + fi + if [[ -f distribute_setup.py ]]; then + echo "${stub}" > distribute_setup.py || die + fi +} + +# @FUNCTION: _distutils-r1_handle_pyproject_toml +# @INTERNAL +# @DESCRIPTION: +# Verify whether DISTUTILS_USE_SETUPTOOLS is set correctly +# for pyproject.toml build systems (in non-PEP517 mode). +_distutils-r1_handle_pyproject_toml() { + if [[ ${DISTUTILS_USE_PEP517} ]]; then + die "${FUNCNAME} is not implemented in PEP517 mode" + fi + + [[ ${DISTUTILS_USE_SETUPTOOLS} == manual ]] && return + + if [[ ! -f setup.py && -f pyproject.toml ]]; then + eerror "No setup.py found but pyproject.toml is present. Please migrate" + eerror "the package to use DISTUTILS_USE_PEP517. See:" + eerror " https://projects.gentoo.org/python/guide/distutils.html" + die "No setup.py found and PEP517 mode not enabled" + fi +} + +# @FUNCTION: _distutils-r1_check_all_phase_mismatch +# @INTERNAL +# @DESCRIPTION: +# Verify whether *_all phase impls is not called from from non-*_all +# subphase. +_distutils-r1_check_all_phase_mismatch() { + if has "python_${EBUILD_PHASE}" "${FUNCNAME[@]}"; then + eqawarn "QA Notice: distutils-r1_python_${EBUILD_PHASE}_all called" + eqawarn "from python_${EBUILD_PHASE}. Did you mean to use" + eqawarn "python_${EBUILD_PHASE}_all()?" + [[ ${EAPI} != 7 ]] && + die "distutils-r1_python_${EBUILD_PHASE}_all called from python_${EBUILD_PHASE}." + fi +} + +# @FUNCTION: _distutils-r1_print_package_versions +# @INTERNAL +# @DESCRIPTION: +# Print the version of the relevant build system packages to aid +# debugging. +_distutils-r1_print_package_versions() { + local packages=() + + if [[ ${DISTUTILS_USE_PEP517} ]]; then + packages+=( + dev-python/gpep517 + dev-python/installer + ) + if [[ ${DISTUTILS_EXT} ]]; then + packages+=( + dev-python/cython + ) + fi + case ${DISTUTILS_USE_PEP517} in + flit) + packages+=( + dev-python/flit-core + ) + ;; + flit_scm) + packages+=( + dev-python/flit-core + dev-python/flit_scm + dev-python/setuptools-scm + ) + ;; + hatchling) + packages+=( + dev-python/hatchling + dev-python/hatch-fancy-pypi-readme + dev-python/hatch-vcs + ) + ;; + jupyter) + packages+=( + dev-python/jupyter-packaging + dev-python/setuptools + dev-python/setuptools-scm + dev-python/wheel + ) + ;; + maturin) + packages+=( + dev-util/maturin + ) + ;; + no) + return + ;; + meson-python) + packages+=( + dev-python/meson-python + ) + ;; + pbr) + packages+=( + dev-python/pbr + dev-python/setuptools + dev-python/wheel + ) + ;; + pdm) + packages+=( + dev-python/pdm-pep517 + dev-python/setuptools + ) + ;; + pdm-backend) + packages+=( + dev-python/pdm-backend + dev-python/setuptools + ) + ;; + poetry) + packages+=( + dev-python/poetry-core + ) + ;; + scikit-build-core) + packages+=( + dev-python/scikit-build-core + ) + ;; + setuptools) + packages+=( + dev-python/setuptools + dev-python/setuptools-rust + dev-python/setuptools-scm + dev-python/wheel + ) + ;; + sip) + packages+=( + dev-python/sip + ) + ;; + esac + else + case ${DISTUTILS_USE_SETUPTOOLS} in + manual|no) + return + ;; + *) + packages+=( + dev-python/setuptools + ) + ;; + esac + fi + + local pkg + einfo "Build system packages:" + for pkg in "${packages[@]}"; do + local installed=$(best_version -b "${pkg}") + einfo " $(printf '%-30s' "${pkg}"): ${installed#${pkg}-}" + done +} + +# @FUNCTION: distutils-r1_python_prepare_all +# @DESCRIPTION: +# The default python_prepare_all(). It applies the patches from PATCHES +# array, then user patches and finally calls python_copy_sources to +# create copies of resulting sources for each Python implementation. +# +# At some point in the future, it may also apply eclass-specific +# distutils patches and/or quirks. +distutils-r1_python_prepare_all() { + debug-print-function ${FUNCNAME} "${@}" + _distutils-r1_check_all_phase_mismatch + + if [[ ! ${DISTUTILS_OPTIONAL} ]]; then + default + fi + + # by default, use in-source build if python_prepare() is used + if [[ ! ${DISTUTILS_IN_SOURCE_BUILD+1} ]]; then + if declare -f python_prepare >/dev/null; then + DISTUTILS_IN_SOURCE_BUILD=1 + fi + fi + + if [[ ! ${DISTUTILS_USE_PEP517} ]]; then + _distutils-r1_disable_ez_setup + _distutils-r1_handle_pyproject_toml + + case ${DISTUTILS_USE_SETUPTOOLS} in + no) + eqawarn "Non-PEP517 builds are deprecated for ebuilds using plain distutils." + eqawarn "Please migrate to DISTUTILS_USE_PEP517=setuptools." + eqawarn "Please see Python Guide for more details:" + eqawarn " https://projects.gentoo.org/python/guide/distutils.html" + ;; + esac + fi + + if [[ ${DISTUTILS_IN_SOURCE_BUILD} && ! ${DISTUTILS_SINGLE_IMPL} ]] + then + # create source copies for each implementation + python_copy_sources + fi + + python_export_utf8_locale + _distutils-r1_print_package_versions + + _DISTUTILS_DEFAULT_CALLED=1 +} + +# @FUNCTION: _distutils-r1_create_setup_cfg +# @INTERNAL +# @DESCRIPTION: +# Create implementation-specific configuration file for distutils, +# setting proper build-dir (and install-dir) paths. +_distutils-r1_create_setup_cfg() { + if [[ ${DISTUTILS_USE_PEP517} ]]; then + die "${FUNCNAME} is not implemented in PEP517 mode" + fi + + cat > "${HOME}"/.pydistutils.cfg <<-_EOF_ || die + [build] + build_base = ${BUILD_DIR} + + # using a single directory for them helps us export + # ${PYTHONPATH} and ebuilds find the sources independently + # of whether the package installs extensions or not + # + # note: due to some packages (wxpython) relying on separate + # platlib & purelib dirs, we do not set --build-lib (which + # can not be overridden with --build-*lib) + build_platlib = %(build_base)s/lib + build_purelib = %(build_base)s/lib + + # make the ebuild writer lives easier + build_scripts = %(build_base)s/scripts + + # this is needed by distutils_install_for_testing since + # setuptools like to create .egg files for install --home. + [bdist_egg] + dist_dir = ${BUILD_DIR}/dist + + # avoid packing up eggs in a zip as it often breaks test suites + [options] + zip_safe = False + _EOF_ + + if [[ ${EBUILD_PHASE} == install ]]; then + # we can't refer to ${D} before src_install() + cat >> "${HOME}"/.pydistutils.cfg <<-_EOF_ || die + + # installation paths -- allow calling extra install targets + # without the default 'install' + [install] + compile = True + optimize = 2 + root = ${D%/} + _EOF_ + + if [[ ! ${DISTUTILS_SINGLE_IMPL} ]]; then + # this gets appended to [install] + cat >> "${HOME}"/.pydistutils.cfg <<-_EOF_ || die + install_scripts = $(python_get_scriptdir) + _EOF_ + fi + fi +} + +# @FUNCTION: _distutils-r1_copy_egg_info +# @INTERNAL +# @DESCRIPTION: +# Copy egg-info files to the ${BUILD_DIR} (that's going to become +# egg-base in esetup.py). This way, we respect whatever's in upstream +# egg-info. +_distutils-r1_copy_egg_info() { + if [[ ${DISTUTILS_USE_PEP517} ]]; then + die "${FUNCNAME} is not implemented in PEP517 mode" + fi + + mkdir -p "${BUILD_DIR}" || die + # stupid freebsd can't do 'cp -t ${BUILD_DIR} {} +' + find -name '*.egg-info' -type d -exec cp -R -p {} "${BUILD_DIR}"/ ';' || die +} + +# @FUNCTION: _distutils-r1_backend_to_key +# @USAGE: +# @INTERNAL +# @DESCRIPTION: +# Print the DISTUTILS_USE_PEP517 value corresponding to the backend +# passed as the only argument. +_distutils-r1_backend_to_key() { + debug-print-function ${FUNCNAME} "${@}" + + local backend=${1} + case ${backend} in + flit_core.buildapi|flit.buildapi) + echo flit + ;; + flit_scm:buildapi) + echo flit_scm + ;; + hatchling.build) + echo hatchling + ;; + jupyter_packaging.build_api) + echo jupyter + ;; + maturin) + echo maturin + ;; + mesonpy) + echo meson-python + ;; + pbr.build) + echo pbr + ;; + pdm.backend) + echo pdm-backend + ;; + pdm.pep517.api) + echo pdm + ;; + poetry.core.masonry.api|poetry.masonry.api) + echo poetry + ;; + scikit_build_core.build) + echo scikit-build-core + ;; + setuptools.build_meta|setuptools.build_meta:__legacy__) + echo setuptools + ;; + sipbuild.api) + echo sip + ;; + *) + die "Unknown backend: ${backend}" + ;; + esac +} + +# @FUNCTION: _distutils-r1_get_backend +# @INTERNAL +# @DESCRIPTION: +# Read (or guess, in case of setuptools) the build-backend +# for the package in the current directory. +_distutils-r1_get_backend() { + debug-print-function ${FUNCNAME} "${@}" + + local build_backend legacy_fallback + if [[ -f pyproject.toml ]]; then + # if pyproject.toml exists, try getting the backend from it + # NB: this could fail if pyproject.toml doesn't list one + build_backend=$(gpep517 get-backend) + fi + if [[ -z ${build_backend} && ${DISTUTILS_USE_PEP517} == setuptools && + -f setup.py ]] + then + # use the legacy setuptools backend as a fallback + build_backend=setuptools.build_meta:__legacy__ + legacy_fallback=1 + fi + if [[ -z ${build_backend} ]]; then + die "Unable to obtain build-backend from pyproject.toml" + fi + + if [[ ${DISTUTILS_USE_PEP517} != standalone ]]; then + # verify whether DISTUTILS_USE_PEP517 was set correctly + local expected_value=$(_distutils-r1_backend_to_key "${build_backend}") + if [[ ${DISTUTILS_USE_PEP517} != ${expected_value} ]]; then + eerror "DISTUTILS_USE_PEP517 does not match pyproject.toml!" + eerror " have: DISTUTILS_USE_PEP517=${DISTUTILS_USE_PEP517}" + eerror "expected: DISTUTILS_USE_PEP517=${expected_value}" + eerror "(backend: ${build_backend})" + die "DISTUTILS_USE_PEP517 value incorrect" + fi + + # fix deprecated backends up + local new_backend= + case ${build_backend} in + flit.buildapi) + new_backend=flit_core.buildapi + ;; + poetry.masonry.api) + new_backend=poetry.core.masonry.api + ;; + setuptools.build_meta:__legacy__) + # this backend should only be used as implicit fallback + [[ ! ${legacy_fallback} ]] && + new_backend=setuptools.build_meta + ;; + esac + + if [[ -n ${new_backend} ]]; then + if [[ ! -f ${T}/.distutils_deprecated_backend_warned ]]; then + eqawarn "${build_backend} backend is deprecated. Please see:" + eqawarn "https://projects.gentoo.org/python/guide/qawarn.html#deprecated-pep-517-backends" + eqawarn "The eclass will be using ${new_backend} instead." + > "${T}"/.distutils_deprecated_backend_warned || die + fi + build_backend=${new_backend} + fi + fi + + echo "${build_backend}" +} + +# @FUNCTION: distutils_wheel_install +# @USAGE: +# @DESCRIPTION: +# Install the specified wheel into . +# +# This function is intended for expert use only. +distutils_wheel_install() { + debug-print-function ${FUNCNAME} "${@}" + if [[ ${#} -ne 2 ]]; then + die "${FUNCNAME} takes exactly two arguments: " + fi + if [[ -z ${PYTHON} ]]; then + die "PYTHON unset, invalid call context" + fi + + local root=${1} + local wheel=${2} + + einfo " Installing ${wheel##*/} to ${root}" + local cmd=( + gpep517 install-wheel + --destdir="${root}" + --interpreter="${PYTHON}" + --prefix="${EPREFIX}/usr" + --optimize=all + "${wheel}" + ) + printf '%s\n' "${cmd[*]}" + "${cmd[@]}" || die "Wheel install failed" + + # remove installed licenses + find "${root}$(python_get_sitedir)" -depth \ + \( -path '*.dist-info/COPYING*' \ + -o -path '*.dist-info/LICENSE*' \ + -o -path '*.dist-info/license_files/*' \ + -o -path '*.dist-info/license_files' \ + -o -path '*.dist-info/licenses/*' \ + -o -path '*.dist-info/licenses' \ + \) -delete || die +} + +# @FUNCTION: distutils_pep517_install +# @USAGE: +# @DESCRIPTION: +# Build the wheel for the package in the current directory using PEP 517 +# backend and install it into . +# +# This function is intended for expert use only. It does not handle +# wrapping executables. +distutils_pep517_install() { + debug-print-function ${FUNCNAME} "${@}" + [[ ${#} -eq 1 ]] || die "${FUNCNAME} takes exactly one argument: root" + + if [[ ! ${DISTUTILS_USE_PEP517:-no} != no ]]; then + die "${FUNCNAME} is available only in PEP517 mode" + fi + + local root=${1} + export BUILD_DIR + local -x WHEEL_BUILD_DIR=${BUILD_DIR}/wheel + mkdir -p "${WHEEL_BUILD_DIR}" || die + + if [[ -n ${mydistutilsargs[@]} ]]; then + die "mydistutilsargs are banned in PEP517 mode (use DISTUTILS_ARGS)" + fi + + local config_settings= + case ${DISTUTILS_USE_PEP517} in + maturin) + # `maturin pep517 build-wheel --help` for options + local maturin_args=( + "${DISTUTILS_ARGS[@]}" + --jobs="$(makeopts_jobs)" + --skip-auditwheel # see bug #831171 + $(in_iuse debug && usex debug '--profile=dev' '') + ) + + config_settings=$( + "${EPYTHON}" - "${maturin_args[@]}" <<-EOF || die + import json + import sys + print(json.dumps({"build-args": sys.argv[1:]})) + EOF + ) + ;; + meson-python) + local -x NINJAOPTS=$(get_NINJAOPTS) + config_settings=$( + "${EPYTHON}" - "${DISTUTILS_ARGS[@]}" <<-EOF || die + import json + import os + import shlex + import sys + + ninjaopts = shlex.split(os.environ["NINJAOPTS"]) + print(json.dumps({ + "builddir": "${BUILD_DIR}", + "setup-args": sys.argv[1:], + "compile-args": ["-v"] + ninjaopts, + })) + EOF + ) + ;; + setuptools) + if [[ -n ${DISTUTILS_ARGS[@]} ]]; then + config_settings=$( + "${EPYTHON}" - "${DISTUTILS_ARGS[@]}" <<-EOF || die + import json + import sys + print(json.dumps({"--build-option": sys.argv[1:]})) + EOF + ) + fi + ;; + sip) + if [[ -n ${DISTUTILS_ARGS[@]} ]]; then + # NB: for practical reasons, we support only --foo=bar, + # not --foo bar + local arg + for arg in "${DISTUTILS_ARGS[@]}"; do + [[ ${arg} != -* ]] && + die "Bare arguments in DISTUTILS_ARGS unsupported: ${arg}" + done + + config_settings=$( + "${EPYTHON}" - "${DISTUTILS_ARGS[@]}" <<-EOF || die + import collections + import json + import sys + + args = collections.defaultdict(list) + for arg in (x.split("=", 1) for x in sys.argv[1:]): \ + args[arg[0]].extend( + [arg[1]] if len(arg) > 1 else []) + + print(json.dumps(args)) + EOF + ) + fi + ;; + *) + [[ -n ${DISTUTILS_ARGS[@]} ]] && + die "DISTUTILS_ARGS are not supported by ${DISTUTILS_USE_PEP517}" + ;; + esac + + local build_backend=$(_distutils-r1_get_backend) + einfo " Building the wheel for ${PWD#${WORKDIR}/} via ${build_backend}" + local cmd=( + gpep517 build-wheel + --prefix="${EPREFIX}/usr" + --backend "${build_backend}" + --output-fd 3 + --wheel-dir "${WHEEL_BUILD_DIR}" + ) + if [[ -n ${config_settings} ]]; then + cmd+=( --config-json "${config_settings}" ) + fi + if [[ -n ${SYSROOT} ]]; then + cmd+=( --sysroot "${SYSROOT}" ) + fi + printf '%s\n' "${cmd[*]}" + local wheel=$( + "${cmd[@]}" 3>&1 >&2 || die "Wheel build failed" + ) + [[ -n ${wheel} ]] || die "No wheel name returned" + + distutils_wheel_install "${root}" "${WHEEL_BUILD_DIR}/${wheel}" + + # clean the build tree; otherwise we may end up with PyPy3 + # extensions duplicated into CPython dists + if [[ ${DISTUTILS_USE_PEP517:-setuptools} == setuptools ]]; then + rm -rf build || die + fi +} + +# @FUNCTION: distutils-r1_python_compile +# @USAGE: [additional-args...] +# @DESCRIPTION: +# The default python_compile(). +# +# If DISTUTILS_USE_PEP517 is set to "no", a no-op. +# +# If DISTUTILS_USE_PEP517 is set to any other value, builds a wheel +# using the PEP517 backend and installs it into ${BUILD_DIR}/install. +# May additionally call build_ext prior to that when using setuptools +# and the eclass detects a potential benefit from parallel extension +# builds. +# +# In legacy mode, runs 'esetup.py build'. Any parameters passed to this +# function will be appended to setup.py invocation, i.e. passed +# as options to the 'build' command. +distutils-r1_python_compile() { + debug-print-function ${FUNCNAME} "${@}" + + _python_check_EPYTHON + + case ${DISTUTILS_USE_PEP517:-setuptools} in + setuptools) + # call setup.py build when using setuptools (either via PEP517 + # or in legacy mode) + + if [[ ${DISTUTILS_USE_PEP517} ]]; then + if [[ -d build ]]; then + eqawarn "A 'build' directory exists already. Artifacts from this directory may" + eqawarn "be picked up by setuptools when building for another interpreter." + eqawarn "Please remove this directory prior to building." + fi + else + _distutils-r1_copy_egg_info + fi + + # distutils is parallel-capable since py3.5 + local jobs=$(makeopts_jobs "${MAKEOPTS} ${*}") + + if [[ ${DISTUTILS_USE_PEP517} ]]; then + # issue build_ext only if it looks like we have at least + # two source files to build; setuptools is expensive + # to start and parallel builds can only benefit us if we're + # compiling at least two files + # + # see extension.py for list of suffixes + # .pyx is added for Cython + # + # esetup.py does not respect SYSROOT, so skip it there + if [[ -z ${SYSROOT} && ${DISTUTILS_EXT} && 1 -ne ${jobs} + && 2 -eq $( + find '(' -name '*.c' -o -name '*.cc' -o -name '*.cpp' \ + -o -name '*.cxx' -o -name '*.c++' -o -name '*.m' \ + -o -name '*.mm' -o -name '*.pyx' ')' -printf '\n' | + head -n 2 | wc -l + ) + ]]; then + esetup.py build_ext -j "${jobs}" "${@}" + fi + else + esetup.py build -j "${jobs}" "${@}" + fi + ;; + no) + return + ;; + esac + + if [[ ${DISTUTILS_USE_PEP517} ]]; then + distutils_pep517_install "${BUILD_DIR}/install" + fi +} + +# @FUNCTION: _distutils-r1_wrap_scripts +# @USAGE: +# @INTERNAL +# @DESCRIPTION: +# Moves and wraps all installed scripts/executables as necessary. +_distutils-r1_wrap_scripts() { + debug-print-function ${FUNCNAME} "${@}" + + [[ ${#} -eq 1 ]] || die "usage: ${FUNCNAME} " + local bindir=${1} + + local scriptdir=$(python_get_scriptdir) + local f python_files=() non_python_files=() + + if [[ -d ${D%/}${scriptdir} ]]; then + for f in "${D%/}${scriptdir}"/*; do + [[ -d ${f} ]] && die "Unexpected directory: ${f}" + debug-print "${FUNCNAME}: found executable at ${f#${D%/}/}" + + local shebang + read -r shebang < "${f}" + if [[ ${shebang} == '#!'*${EPYTHON}* ]]; then + debug-print "${FUNCNAME}: matching shebang: ${shebang}" + python_files+=( "${f}" ) + else + debug-print "${FUNCNAME}: non-matching shebang: ${shebang}" + non_python_files+=( "${f}" ) + fi + + mkdir -p "${D%/}${bindir}" || die + done + + for f in "${python_files[@]}"; do + local basename=${f##*/} + + debug-print "${FUNCNAME}: installing wrapper at ${bindir}/${basename}" + local dosym=dosym + [[ ${EAPI} == 7 ]] && dosym=dosym8 + "${dosym}" -r /usr/lib/python-exec/python-exec2 \ + "${bindir#${EPREFIX}}/${basename}" + done + + for f in "${non_python_files[@]}"; do + local basename=${f##*/} + + debug-print "${FUNCNAME}: moving ${f#${D%/}/} to ${bindir}/${basename}" + mv "${f}" "${D%/}${bindir}/${basename}" || die + done + fi +} + +# @FUNCTION: distutils-r1_python_test +# @USAGE: [additional-args...] +# @DESCRIPTION: +# The python_test() implementation used by distutils_enable_tests. +# Runs tests using the specified test runner, possibly installing them +# first. +# +# This function is used only if distutils_enable_tests is called. +distutils-r1_python_test() { + debug-print-function ${FUNCNAME} "${@}" + + if [[ -z ${_DISTUTILS_TEST_RUNNER} ]]; then + die "${FUNCNAME} can be only used after calling distutils_enable_tests" + fi + + _python_check_EPYTHON + + if [[ ${_DISTUTILS_TEST_INSTALL} ]]; then + distutils_install_for_testing + fi + + case ${_DISTUTILS_TEST_RUNNER} in + nose) + "${EPYTHON}" -m nose -v "${@}" + ;; + pytest) + epytest + ;; + setup.py) + nonfatal esetup.py test --verbose + ;; + unittest) + eunittest + ;; + *) + die "Mis-synced test runner between ${FUNCNAME} and distutils_enable_testing" + ;; + esac + + if [[ ${?} -ne 0 ]]; then + die -n "Tests failed with ${EPYTHON}" + fi +} + +# @FUNCTION: distutils-r1_python_install +# @USAGE: [additional-args...] +# @DESCRIPTION: +# The default python_install(). +# +# In PEP517 mode, merges the files from ${BUILD_DIR}/install +# (if present) to the image directory. +# +# In the legacy mode, calls `esetup.py install` to install the package. +# Any parameters passed to this function will be appended +# to the setup.py invocation (i.e. as options to the 'install' command). +distutils-r1_python_install() { + debug-print-function ${FUNCNAME} "${@}" + + _python_check_EPYTHON + + local scriptdir=${EPREFIX}/usr/bin + local merge_root= + if [[ ${DISTUTILS_USE_PEP517} ]]; then + local root=${BUILD_DIR}/install + local reg_scriptdir=${root}/${scriptdir} + local wrapped_scriptdir=${root}$(python_get_scriptdir) + + # we are assuming that _distutils-r1_post_python_compile + # has been called and ${root} has not been altered since + # let's explicitly verify these assumptions + + # remove files that we've created explicitly + rm "${reg_scriptdir}"/{"${EPYTHON}",python3,python,pyvenv.cfg} || die + + # Automagically do the QA check to avoid issues when bootstrapping + # prefix. + if type diff &>/dev/null ; then + # verify that scriptdir & wrapped_scriptdir both contain + # the same files + ( + cd "${reg_scriptdir}" && find . -mindepth 1 + ) | sort > "${T}"/.distutils-files-bin + assert "listing ${reg_scriptdir} failed" + ( + if [[ -d ${wrapped_scriptdir} ]]; then + cd "${wrapped_scriptdir}" && find . -mindepth 1 + fi + ) | sort > "${T}"/.distutils-files-wrapped + assert "listing ${wrapped_scriptdir} failed" + if ! diff -U 0 "${T}"/.distutils-files-{bin,wrapped}; then + die "File lists for ${reg_scriptdir} and ${wrapped_scriptdir} differ (see diff above)" + fi + fi + + # remove the altered bindir, executables from the package + # are already in scriptdir + rm -r "${reg_scriptdir}" || die + if [[ ${DISTUTILS_SINGLE_IMPL} ]]; then + if [[ -d ${wrapped_scriptdir} ]]; then + mv "${wrapped_scriptdir}" "${reg_scriptdir}" || die + fi + fi + # prune empty directories to see if ${root} contains anything + # to merge + find "${BUILD_DIR}"/install -type d -empty -delete || die + [[ -d ${BUILD_DIR}/install ]] && merge_root=1 + else + local root=${D%/}/_${EPYTHON} + [[ ${DISTUTILS_SINGLE_IMPL} ]] && root=${D%/} + + # inline DISTUTILS_ARGS logic from esetup.py in order to make + # argv overwriting easier + local args=( + "${DISTUTILS_ARGS[@]}" + "${mydistutilsargs[@]}" + install --skip-build --root="${root}" "${args[@]}" + "${@}" + ) + local DISTUTILS_ARGS=() + local mydistutilsargs=() + + # enable compilation for the install phase. + local -x PYTHONDONTWRITEBYTECODE= + + # python likes to compile any module it sees, which triggers sandbox + # failures if some packages haven't compiled their modules yet. + addpredict "${EPREFIX}/usr/lib/${EPYTHON}" + addpredict "${EPREFIX}/usr/lib/pypy3.10" + addpredict "${EPREFIX}/usr/local" # bug 498232 + + if [[ ! ${DISTUTILS_SINGLE_IMPL} ]]; then + merge_root=1 + + # user may override --install-scripts + # note: this is poor but distutils argv parsing is dumb + + # rewrite all the arguments + set -- "${args[@]}" + args=() + while [[ ${@} ]]; do + local a=${1} + shift + + case ${a} in + --install-scripts=*) + scriptdir=${a#--install-scripts=} + ;; + --install-scripts) + scriptdir=${1} + shift + ;; + *) + args+=( "${a}" ) + ;; + esac + done + fi + + esetup.py "${args[@]}" + fi + + if [[ ${merge_root} ]]; then + multibuild_merge_root "${root}" "${D%/}" + fi + if [[ ! ${DISTUTILS_SINGLE_IMPL} ]]; then + _distutils-r1_wrap_scripts "${scriptdir}" + fi +} + +# @FUNCTION: distutils-r1_python_install_all +# @DESCRIPTION: +# The default python_install_all(). It installs the documentation. +distutils-r1_python_install_all() { + debug-print-function ${FUNCNAME} "${@}" + _distutils-r1_check_all_phase_mismatch + + einstalldocs +} + +# @FUNCTION: distutils-r1_run_phase +# @USAGE: [...] +# @INTERNAL +# @DESCRIPTION: +# Run the given command. +# +# If out-of-source builds are used, the phase function is run in source +# directory, with BUILD_DIR pointing at the build directory +# and PYTHONPATH having an entry for the module build directory. +# +# If in-source builds are used, the command is executed in the directory +# holding the per-implementation copy of sources. BUILD_DIR points +# to the 'build' subdirectory. +distutils-r1_run_phase() { + debug-print-function ${FUNCNAME} "${@}" + + if [[ ${DISTUTILS_IN_SOURCE_BUILD} ]]; then + [[ ${DISTUTILS_USE_PEP517} ]] && + die "DISTUTILS_IN_SOURCE_BUILD is not supported in PEP517 mode" + # only force BUILD_DIR if implementation is explicitly enabled + # for building; any-r1 API may select one that is not + # https://bugs.gentoo.org/701506 + if [[ ! ${DISTUTILS_SINGLE_IMPL} ]] && + has "${EPYTHON/./_}" ${PYTHON_TARGETS}; then + cd "${BUILD_DIR}" || die + fi + local BUILD_DIR=${BUILD_DIR}/build + fi + + if [[ ${DISTUTILS_USE_PEP517} ]]; then + local -x PATH=${BUILD_DIR}/install${EPREFIX}/usr/bin:${PATH} + else + local -x PYTHONPATH="${BUILD_DIR}/lib:${PYTHONPATH}" + + # make PATH local for distutils_install_for_testing calls + # it makes little sense to let user modify PATH in per-impl phases + # and _all() already localizes it + local -x PATH=${PATH} + + if _python_impl_matches "${EPYTHON}" 3.{9..11}; then + # Undo the default switch in setuptools-60+ for the time being, + # to avoid replacing .egg-info file with directory in-place. + local -x SETUPTOOLS_USE_DISTUTILS="${SETUPTOOLS_USE_DISTUTILS:-stdlib}" + fi + + # Bug 559644 + # using PYTHONPATH when the ${BUILD_DIR}/lib is not created yet might lead to + # problems in setup.py scripts that try to import modules/packages from that path + # during the build process (Python at startup evaluates PYTHONPATH, if the dir is + # not valid then associates a NullImporter object to ${BUILD_DIR}/lib storing it + # in the sys.path_importer_cache) + mkdir -p "${BUILD_DIR}/lib" || die + fi + + # Set up build environment, bug #513664. + local -x AR=${AR} CC=${CC} CPP=${CPP} CXX=${CXX} + tc-export AR CC CPP CXX + + if [[ ${DISTUTILS_EXT} ]]; then + local -x CPPFLAGS="${CPPFLAGS} $(usex debug '-UNDEBUG' '-DNDEBUG')" + # always generate .c files from .pyx files to ensure we get latest + # bug fixes from Cython (this works only when setup.py is using + # cythonize() but it's better than nothing) + local -x CYTHON_FORCE_REGEN=1 + fi + + # Rust extensions are incompatible with C/C++ LTO compiler + # see e.g. https://bugs.gentoo.org/910220 + if has cargo ${INHERITED}; then + filter-lto + fi + + # How to build Python modules in different worlds... + local ldopts + case "${CHOST}" in + # provided by grobian, 2014-06-22, bug #513664 c7 + *-darwin*) ldopts='-bundle -undefined dynamic_lookup';; + *) ldopts='-shared';; + esac + + local -x LDSHARED="${CC} ${ldopts}" LDCXXSHARED="${CXX} ${ldopts}" + local _DISTUTILS_POST_PHASE_RM=() + + "${@}" + local ret=${?} + + if [[ -n ${_DISTUTILS_POST_PHASE_RM} ]]; then + rm "${_DISTUTILS_POST_PHASE_RM[@]}" || die + fi + + cd "${_DISTUTILS_INITIAL_CWD}" || die + if [[ ! ${_DISTUTILS_IN_COMMON_IMPL} ]] && + declare -f "_distutils-r1_post_python_${EBUILD_PHASE}" >/dev/null + then + "_distutils-r1_post_python_${EBUILD_PHASE}" + fi + return "${ret}" +} + +# @FUNCTION: _distutils-r1_run_common_phase +# @USAGE: [...] +# @INTERNAL +# @DESCRIPTION: +# Run the given command, restoring the state for a most preferred Python +# implementation matching DISTUTILS_ALL_SUBPHASE_IMPLS. +# +# If in-source build is used, the command will be run in the copy +# of sources made for the selected Python interpreter. +_distutils-r1_run_common_phase() { + local DISTUTILS_ORIG_BUILD_DIR=${BUILD_DIR} + local _DISTUTILS_IN_COMMON_IMPL=1 + + if [[ ${DISTUTILS_SINGLE_IMPL} ]]; then + # reuse the dedicated code branch + _distutils-r1_run_foreach_impl "${@}" + else + local -x EPYTHON PYTHON + local -x PATH=${PATH} PKG_CONFIG_PATH=${PKG_CONFIG_PATH} + python_setup "${DISTUTILS_ALL_SUBPHASE_IMPLS[@]}" + + local MULTIBUILD_VARIANTS=( "${EPYTHON/./_}" ) + # store for restoring after distutils-r1_run_phase. + local _DISTUTILS_INITIAL_CWD=${PWD} + multibuild_foreach_variant \ + distutils-r1_run_phase "${@}" + fi +} + +# @FUNCTION: _distutils-r1_run_foreach_impl +# @INTERNAL +# @DESCRIPTION: +# Run the given phase for each implementation if multiple implementations +# are enabled, once otherwise. +_distutils-r1_run_foreach_impl() { + debug-print-function ${FUNCNAME} "${@}" + + # store for restoring after distutils-r1_run_phase. + local _DISTUTILS_INITIAL_CWD=${PWD} + set -- distutils-r1_run_phase "${@}" + + if [[ ! ${DISTUTILS_SINGLE_IMPL} ]]; then + local _DISTUTILS_CALLING_FOREACH_IMPL=1 + python_foreach_impl "${@}" + else + if [[ ! ${EPYTHON} ]]; then + die "EPYTHON unset, python-single-r1_pkg_setup not called?!" + fi + local BUILD_DIR=${BUILD_DIR:-${S}} + BUILD_DIR=${BUILD_DIR%%/}_${EPYTHON} + + "${@}" + fi +} + +distutils-r1_src_prepare() { + debug-print-function ${FUNCNAME} "${@}" + local ret=0 + local _DISTUTILS_DEFAULT_CALLED + + # common preparations + if declare -f python_prepare_all >/dev/null; then + python_prepare_all || ret=${?} + else + distutils-r1_python_prepare_all || ret=${?} + fi + + if [[ ! ${_DISTUTILS_DEFAULT_CALLED} ]]; then + die "QA: python_prepare_all() didn't call distutils-r1_python_prepare_all" + fi + + if declare -f python_prepare >/dev/null; then + _distutils-r1_run_foreach_impl python_prepare || ret=${?} + fi + + return ${ret} +} + +distutils-r1_src_configure() { + debug-print-function ${FUNCNAME} "${@}" + local ret=0 + + if declare -f python_configure >/dev/null; then + _distutils-r1_run_foreach_impl python_configure || ret=${?} + fi + + if declare -f python_configure_all >/dev/null; then + _distutils-r1_run_common_phase python_configure_all || ret=${?} + fi + + return ${ret} +} + +# @FUNCTION: _distutils-r1_post_python_compile +# @INTERNAL +# @DESCRIPTION: +# Post-phase function called after python_compile. In PEP517 mode, +# it adjusts the install tree for venv-style usage. +_distutils-r1_post_python_compile() { + debug-print-function ${FUNCNAME} "${@}" + + local root=${BUILD_DIR}/install + if [[ ${DISTUTILS_USE_PEP517} && -d ${root} ]]; then + # copy executables to python-exec directory + # we do it early so that we can alter bindir recklessly + local bindir=${root}${EPREFIX}/usr/bin + local rscriptdir=${root}$(python_get_scriptdir) + [[ -d ${rscriptdir} ]] && + die "${rscriptdir} should not exist!" + if [[ -d ${bindir} ]]; then + mkdir -p "${rscriptdir}" || die + cp -a "${bindir}"/. "${rscriptdir}"/ || die + fi + + # enable venv magic inside the install tree + mkdir -p "${bindir}" || die + ln -s "${PYTHON}" "${bindir}/${EPYTHON}" || die + ln -s "${EPYTHON}" "${bindir}/python3" || die + ln -s "${EPYTHON}" "${bindir}/python" || die + cat > "${bindir}"/pyvenv.cfg <<-EOF || die + include-system-site-packages = true + EOF + + # we need to change shebangs to point to the venv-python + find "${bindir}" -type f -exec sed -i \ + -e "1s@^#!\(${EPREFIX}/usr/bin/\(python\|pypy\)\)@#!${root}\1@" \ + {} + || die + fi +} + +distutils-r1_src_compile() { + debug-print-function ${FUNCNAME} "${@}" + local ret=0 + + if declare -f python_compile >/dev/null; then + _distutils-r1_run_foreach_impl python_compile || ret=${?} + else + _distutils-r1_run_foreach_impl distutils-r1_python_compile || ret=${?} + fi + + if declare -f python_compile_all >/dev/null; then + _distutils-r1_run_common_phase python_compile_all || ret=${?} + fi + + return ${ret} +} + +# @FUNCTION: _distutils-r1_clean_egg_info +# @INTERNAL +# @DESCRIPTION: +# Clean up potential stray egg-info files left by setuptools test phase. +# Those files ended up being unversioned, and caused issues: +# https://bugs.gentoo.org/534058 +_distutils-r1_clean_egg_info() { + if [[ ${DISTUTILS_USE_PEP517} ]]; then + die "${FUNCNAME} is not implemented in PEP517 mode" + fi + + rm -rf "${BUILD_DIR}"/lib/*.egg-info || die +} + +# @FUNCTION: _distutils-r1_post_python_test +# @INTERNAL +# @DESCRIPTION: +# Post-phase function called after python_test. +_distutils-r1_post_python_test() { + debug-print-function ${FUNCNAME} "${@}" + + if [[ ! ${DISTUTILS_USE_PEP517} ]]; then + _distutils-r1_clean_egg_info + fi +} + +distutils-r1_src_test() { + debug-print-function ${FUNCNAME} "${@}" + local ret=0 + + if declare -f python_test >/dev/null; then + _distutils-r1_run_foreach_impl python_test || ret=${?} + fi + + if declare -f python_test_all >/dev/null; then + _distutils-r1_run_common_phase python_test_all || ret=${?} + fi + + return ${ret} +} + +# @FUNCTION: _distutils-r1_strip_namespace_packages +# @USAGE: +# @INTERNAL +# @DESCRIPTION: +# Find and remove setuptools-style namespaces in the specified +# directory. +_distutils-r1_strip_namespace_packages() { + debug-print-function ${FUNCNAME} "${@}" + + local sitedir=${1} + local f ns had_any= + while IFS= read -r -d '' f; do + while read -r ns; do + einfo "Stripping pkg_resources-style namespace ${ns}" + had_any=1 + done < "${f}" + + rm "${f}" || die + done < <( + # NB: this deliberately does not include .egg-info, in order + # to limit this to PEP517 mode. + find "${sitedir}" -path '*.dist-info/namespace_packages.txt' -print0 + ) + + # If we had any namespace packages, remove .pth files as well. + if [[ ${had_any} ]]; then + find "${sitedir}" -name '*-nspkg.pth' -delete || die + fi +} + +# @FUNCTION: _distutils-r1_post_python_install +# @INTERNAL +# @DESCRIPTION: +# Post-phase function called after python_install. Performs QA checks. +# In PEP517 mode, additionally optimizes installed Python modules. +_distutils-r1_post_python_install() { + debug-print-function ${FUNCNAME} "${@}" + + local sitedir=${D%/}$(python_get_sitedir) + if [[ -d ${sitedir} ]]; then + _distutils-r1_strip_namespace_packages "${sitedir}" + + local forbidden_package_names=( + examples test tests + .pytest_cache .hypothesis _trial_temp + ) + local strays=() + local p + mapfile -d $'\0' -t strays < <( + find "${sitedir}" -maxdepth 1 -type f '!' '(' \ + -name '*.egg-info' -o \ + -name '*.pth' -o \ + -name '*.py' -o \ + -name '*.pyi' -o \ + -name "*$(get_modname)" \ + ')' -print0 + ) + for p in "${forbidden_package_names[@]}"; do + [[ -d ${sitedir}/${p} ]] && strays+=( "${sitedir}/${p}" ) + done + + if [[ -n ${strays[@]} ]]; then + eerror "The following unexpected files/directories were found top-level" + eerror "in the site-packages directory:" + eerror + for p in "${strays[@]}"; do + eerror " ${p#${ED}}" + done + eerror + eerror "This is most likely a bug in the build system. More information" + eerror "can be found in the Python Guide:" + eerror "https://projects.gentoo.org/python/guide/qawarn.html#stray-top-level-files-in-site-packages" + die "Failing install because of stray top-level files in site-packages" + fi + + if [[ ! ${DISTUTILS_EXT} && ! ${_DISTUTILS_EXT_WARNED} ]]; then + if [[ $(find "${sitedir}" -name "*$(get_modname)" | head -n 1) ]] + then + eqawarn "Python extension modules (*$(get_modname)) found installed. Please set:" + eqawarn " DISTUTILS_EXT=1" + eqawarn "in the ebuild." + _DISTUTILS_EXT_WARNED=1 + fi + fi + fi +} + +# @FUNCTION: _distutils-r1_check_namespace_pth +# @INTERNAL +# @DESCRIPTION: +# Check if any *-nspkg.pth files were installed (by setuptools) +# and warn about the policy non-conformance if they were. +_distutils-r1_check_namespace_pth() { + local f pth=() + + while IFS= read -r -d '' f; do + pth+=( "${f}" ) + done < <(find "${ED%/}" -name '*-nspkg.pth' -print0) + + if [[ ${pth[@]} ]]; then + eerror "The following *-nspkg.pth files were found installed:" + eerror + for f in "${pth[@]}"; do + eerror " ${f#${ED%/}}" + done + eerror + eerror "The presence of those files may break namespaces in Python 3.5+. Please" + eerror "read our documentation on reliable handling of namespaces and update" + eerror "the ebuild accordingly:" + eerror + eerror " https://projects.gentoo.org/python/guide/concept.html#namespace-packages" + + die "Installing *-nspkg.pth files is banned" + fi +} + +distutils-r1_src_install() { + debug-print-function ${FUNCNAME} "${@}" + local ret=0 + + if declare -f python_install >/dev/null; then + _distutils-r1_run_foreach_impl python_install || ret=${?} + else + _distutils-r1_run_foreach_impl distutils-r1_python_install || ret=${?} + fi + + if declare -f python_install_all >/dev/null; then + _distutils-r1_run_common_phase python_install_all || ret=${?} + else + _distutils-r1_run_common_phase distutils-r1_python_install_all || ret=${?} + fi + + _distutils-r1_check_namespace_pth + + return ${ret} +} + +fi + +if [[ ! ${DISTUTILS_OPTIONAL} ]]; then + EXPORT_FUNCTIONS src_prepare src_configure src_compile src_test src_install +fi diff --git a/sdk_container/src/third_party/prefix-overlay/eclass/python-utils-r1.eclass b/sdk_container/src/third_party/prefix-overlay/eclass/python-utils-r1.eclass new file mode 100644 index 0000000000..bd30c12031 --- /dev/null +++ b/sdk_container/src/third_party/prefix-overlay/eclass/python-utils-r1.eclass @@ -0,0 +1,1485 @@ +# Copyright 1999-2023 Gentoo Authors +# Distributed under the terms of the GNU General Public License v2 + +# @ECLASS: python-utils-r1.eclass +# @MAINTAINER: +# Python team +# @AUTHOR: +# Author: Michał Górny +# Based on work of: Krzysztof Pawlik +# @SUPPORTED_EAPIS: 7 8 +# @BLURB: Utility functions for packages with Python parts. +# @DESCRIPTION: +# A utility eclass providing functions to query Python implementations, +# install Python modules and scripts. +# +# This eclass does not set any metadata variables nor export any phase +# functions. It can be inherited safely. +# +# For more information, please see the Python Guide: +# https://projects.gentoo.org/python/guide/ + +# NOTE: When dropping support for EAPIs here, we need to update +# metadata/install-qa-check.d/60python-pyc +# See bug #704286, bug #781878 + +case ${EAPI} in + 7|8) ;; + *) die "${ECLASS}: EAPI ${EAPI:-0} not supported" ;; +esac + +if [[ ! ${_PYTHON_UTILS_R1_ECLASS} ]]; then +_PYTHON_UTILS_R1_ECLASS=1 + +[[ ${EAPI} == 7 ]] && inherit eapi8-dosym +inherit multiprocessing toolchain-funcs + +# @ECLASS_VARIABLE: _PYTHON_ALL_IMPLS +# @INTERNAL +# @DESCRIPTION: +# All supported Python implementations, most preferred last. +_PYTHON_ALL_IMPLS=( + pypy3 + python3_{10..12} +) +readonly _PYTHON_ALL_IMPLS + +# @ECLASS_VARIABLE: _PYTHON_HISTORICAL_IMPLS +# @INTERNAL +# @DESCRIPTION: +# All historical Python implementations that are no longer supported. +_PYTHON_HISTORICAL_IMPLS=( + jython2_7 + pypy pypy1_{8,9} pypy2_0 + python2_{5..7} + python3_{1..9} +) +readonly _PYTHON_HISTORICAL_IMPLS + +# @ECLASS_VARIABLE: PYTHON_COMPAT_NO_STRICT +# @INTERNAL +# @DESCRIPTION: +# Set to a non-empty value in order to make eclass tolerate (ignore) +# unknown implementations in PYTHON_COMPAT. +# +# This is intended to be set by the user when using ebuilds that may +# have unknown (newer) implementations in PYTHON_COMPAT. The assumption +# is that the ebuilds are intended to be used within multiple contexts +# which can involve revisions of this eclass that support a different +# set of Python implementations. + +# @FUNCTION: _python_verify_patterns +# @USAGE: ... +# @INTERNAL +# @DESCRIPTION: +# Verify whether the patterns passed to the eclass function are correct +# (i.e. can match any valid implementation). Dies on wrong pattern. +_python_verify_patterns() { + debug-print-function ${FUNCNAME} "${@}" + + local impl pattern + for pattern; do + case ${pattern} in + -[23]|3.[89]|3.1[012]) + continue + ;; + esac + + for impl in "${_PYTHON_ALL_IMPLS[@]}" "${_PYTHON_HISTORICAL_IMPLS[@]}" + do + [[ ${impl} == ${pattern/./_} ]] && continue 2 + done + + die "Invalid implementation pattern: ${pattern}" + done +} + +# @FUNCTION: _python_set_impls +# @INTERNAL +# @DESCRIPTION: +# Check PYTHON_COMPAT for well-formedness and validity, then set +# two global variables: +# +# - _PYTHON_SUPPORTED_IMPLS containing valid implementations supported +# by the ebuild (PYTHON_COMPAT - dead implementations), +# +# - and _PYTHON_UNSUPPORTED_IMPLS containing valid implementations that +# are not supported by the ebuild. +# +# Implementations in both variables are ordered using the pre-defined +# eclass implementation ordering. +# +# This function must be called once in global scope by an eclass +# utilizing PYTHON_COMPAT. +_python_set_impls() { + local i + + # TODO: drop BASH_VERSINFO check when we require EAPI 8 + if [[ ${BASH_VERSINFO[0]} -ge 5 ]]; then + [[ ${PYTHON_COMPAT@a} == *a* ]] + else + [[ $(declare -p PYTHON_COMPAT) == "declare -a"* ]] + fi + if [[ ${?} -ne 0 ]]; then + if ! declare -p PYTHON_COMPAT &>/dev/null; then + die 'PYTHON_COMPAT not declared.' + else + die 'PYTHON_COMPAT must be an array.' + fi + fi + + local obsolete=() + if [[ ! ${PYTHON_COMPAT_NO_STRICT} ]]; then + for i in "${PYTHON_COMPAT[@]}"; do + # check for incorrect implementations + # we're using pattern matching as an optimization + # please keep them in sync with _PYTHON_ALL_IMPLS + # and _PYTHON_HISTORICAL_IMPLS + case ${i} in + pypy3|python3_9|python3_1[0-2]) + ;; + jython2_7|pypy|pypy1_[89]|pypy2_0|python2_[5-7]|python3_[1-9]) + obsolete+=( "${i}" ) + ;; + *) + if has "${i}" "${_PYTHON_ALL_IMPLS[@]}" \ + "${_PYTHON_HISTORICAL_IMPLS[@]}" + then + die "Mis-synced patterns in _python_set_impls: missing ${i}" + else + die "Invalid implementation in PYTHON_COMPAT: ${i}" + fi + esac + done + fi + + if [[ -n ${obsolete[@]} && ${EBUILD_PHASE} == setup ]]; then + # complain if people don't clean up old impls while touching + # the ebuilds recently. use the copyright year to infer last + # modification + # NB: this check doesn't have to work reliably + if [[ $(head -n 1 "${EBUILD}" 2>/dev/null) == *2022* ]]; then + eqawarn "Please clean PYTHON_COMPAT of obsolete implementations:" + eqawarn " ${obsolete[*]}" + fi + fi + + local supp=() unsupp=() + + for i in "${_PYTHON_ALL_IMPLS[@]}"; do + if has "${i}" "${PYTHON_COMPAT[@]}"; then + supp+=( "${i}" ) + else + unsupp+=( "${i}" ) + fi + done + + if [[ ! ${supp[@]} ]]; then + die "No supported implementation in PYTHON_COMPAT." + fi + + if [[ ${_PYTHON_SUPPORTED_IMPLS[@]} ]]; then + # set once already, verify integrity + if [[ ${_PYTHON_SUPPORTED_IMPLS[@]} != ${supp[@]} ]]; then + eerror "Supported impls (PYTHON_COMPAT) changed between inherits!" + eerror "Before: ${_PYTHON_SUPPORTED_IMPLS[*]}" + eerror "Now : ${supp[*]}" + die "_PYTHON_SUPPORTED_IMPLS integrity check failed" + fi + if [[ ${_PYTHON_UNSUPPORTED_IMPLS[@]} != ${unsupp[@]} ]]; then + eerror "Unsupported impls changed between inherits!" + eerror "Before: ${_PYTHON_UNSUPPORTED_IMPLS[*]}" + eerror "Now : ${unsupp[*]}" + die "_PYTHON_UNSUPPORTED_IMPLS integrity check failed" + fi + else + _PYTHON_SUPPORTED_IMPLS=( "${supp[@]}" ) + _PYTHON_UNSUPPORTED_IMPLS=( "${unsupp[@]}" ) + readonly _PYTHON_SUPPORTED_IMPLS _PYTHON_UNSUPPORTED_IMPLS + fi +} + +# @FUNCTION: _python_impl_matches +# @USAGE: [...] +# @INTERNAL +# @DESCRIPTION: +# Check whether the specified matches at least one +# of the patterns following it. Return 0 if it does, 1 otherwise. +# Matches if no patterns are provided. +# +# can be in PYTHON_COMPAT or EPYTHON form. The patterns +# can either be fnmatch-style or stdlib versions, e.g. "3.8", "3.9". +# In the latter case, pypy3 will match if there is at least one pypy3 +# version matching the stdlib version. +_python_impl_matches() { + [[ ${#} -ge 1 ]] || die "${FUNCNAME}: takes at least 1 parameter" + [[ ${#} -eq 1 ]] && return 0 + + local impl=${1/./_} pattern + shift + + for pattern; do + case ${pattern} in + -2|python2*|pypy) + if [[ ${EAPI} != 7 ]]; then + eerror + eerror "Python 2 is no longer supported in Gentoo, please remove Python 2" + eerror "${FUNCNAME[1]} calls." + die "Passing ${pattern} to ${FUNCNAME[1]} is banned in EAPI ${EAPI}" + fi + ;; + -3) + # NB: "python3*" is fine, as "not pypy3" + if [[ ${EAPI} != 7 ]]; then + eerror + eerror "Python 2 is no longer supported in Gentoo, please remove Python 2" + eerror "${FUNCNAME[1]} calls." + die "Passing ${pattern} to ${FUNCNAME[1]} is banned in EAPI ${EAPI}" + fi + return 0 + ;; + 3.10) + [[ ${impl} == python${pattern/./_} || ${impl} == pypy3 ]] && + return 0 + ;; + 3.8|3.9|3.1[1-2]) + [[ ${impl} == python${pattern/./_} ]] && return 0 + ;; + *) + # unify value style to allow lax matching + [[ ${impl} == ${pattern/./_} ]] && return 0 + ;; + esac + done + + return 1 +} + +# @ECLASS_VARIABLE: PYTHON +# @DEFAULT_UNSET +# @DESCRIPTION: +# The absolute path to the current Python interpreter. +# +# This variable is set automatically in the following contexts: +# +# python-r1: Set in functions called by python_foreach_impl() or after +# calling python_setup(). +# +# python-single-r1: Set after calling python-single-r1_pkg_setup(). +# +# distutils-r1: Set within any of the python sub-phase functions. +# +# Example value: +# @CODE +# /usr/bin/python2.7 +# @CODE + +# @ECLASS_VARIABLE: EPYTHON +# @DEFAULT_UNSET +# @DESCRIPTION: +# The executable name of the current Python interpreter. +# +# This variable is set automatically in the following contexts: +# +# python-r1: Set in functions called by python_foreach_impl() or after +# calling python_setup(). +# +# python-single-r1: Set after calling python-single-r1_pkg_setup(). +# +# distutils-r1: Set within any of the python sub-phase functions. +# +# Example value: +# @CODE +# python2.7 +# @CODE + +# @FUNCTION: _python_export +# @USAGE: [] ... +# @INTERNAL +# @DESCRIPTION: +# Set and export the Python implementation-relevant variables passed +# as parameters. +# +# The optional first parameter may specify the requested Python +# implementation (either as PYTHON_TARGETS value, e.g. python2_7, +# or an EPYTHON one, e.g. python2.7). If no implementation passed, +# the current one will be obtained from ${EPYTHON}. +# +# The variables which can be exported are: PYTHON, EPYTHON, +# PYTHON_SITEDIR. They are described more completely in the eclass +# variable documentation. +_python_export() { + debug-print-function ${FUNCNAME} "${@}" + + local impl var + + case "${1}" in + python*|jython*) + impl=${1/_/.} + shift + ;; + pypy|pypy3) + impl=${1} + shift + ;; + *) + impl=${EPYTHON} + if [[ -z ${impl} ]]; then + die "_python_export called without a python implementation and EPYTHON is unset" + fi + ;; + esac + debug-print "${FUNCNAME}: implementation: ${impl}" + + for var; do + case "${var}" in + EPYTHON) + export EPYTHON=${impl} + debug-print "${FUNCNAME}: EPYTHON = ${EPYTHON}" + ;; + PYTHON) + # Under EAPI 7+, this should just use ${BROOT}, but Portage + # <3.0.50 was buggy, and prefix users need this to update. + export PYTHON=${BROOT-${EPREFIX}}/usr/bin/${impl} + debug-print "${FUNCNAME}: PYTHON = ${PYTHON}" + ;; + PYTHON_SITEDIR) + [[ -n ${PYTHON} ]] || die "PYTHON needs to be set for ${var} to be exported, or requested before it" + PYTHON_SITEDIR=$( + "${PYTHON}" - "${EPREFIX}/usr" <<-EOF || die + import sys, sysconfig + print(sysconfig.get_path("purelib", vars={"base": sys.argv[1]})) + EOF + ) + export PYTHON_SITEDIR + debug-print "${FUNCNAME}: PYTHON_SITEDIR = ${PYTHON_SITEDIR}" + ;; + PYTHON_INCLUDEDIR) + [[ -n ${PYTHON} ]] || die "PYTHON needs to be set for ${var} to be exported, or requested before it" + PYTHON_INCLUDEDIR=$( + "${PYTHON}" - "${ESYSROOT}/usr" <<-EOF || die + import sys, sysconfig + print(sysconfig.get_path("platinclude", vars={"installed_platbase": sys.argv[1]})) + EOF + ) + export PYTHON_INCLUDEDIR + debug-print "${FUNCNAME}: PYTHON_INCLUDEDIR = ${PYTHON_INCLUDEDIR}" + + # Jython gives a non-existing directory + if [[ ! -d ${PYTHON_INCLUDEDIR} ]]; then + die "${impl} does not install any header files!" + fi + ;; + PYTHON_LIBPATH) + [[ -n ${PYTHON} ]] || die "PYTHON needs to be set for ${var} to be exported, or requested before it" + PYTHON_LIBPATH=$( + "${PYTHON}" - <<-EOF || die + import os.path, sysconfig + print( + os.path.join( + sysconfig.get_config_var("LIBDIR"), + sysconfig.get_config_var("LDLIBRARY")) + if sysconfig.get_config_var("LDLIBRARY") + else "") + EOF + ) + export PYTHON_LIBPATH + debug-print "${FUNCNAME}: PYTHON_LIBPATH = ${PYTHON_LIBPATH}" + + if [[ ! ${PYTHON_LIBPATH} ]]; then + die "${impl} lacks a (usable) dynamic library" + fi + ;; + PYTHON_CFLAGS) + local val + + case "${impl}" in + python*) + # python-2.7, python-3.2, etc. + val=$($(tc-getPKG_CONFIG) --cflags ${impl/n/n-}) || die + ;; + *) + die "${impl}: obtaining ${var} not supported" + ;; + esac + + export PYTHON_CFLAGS=${val} + debug-print "${FUNCNAME}: PYTHON_CFLAGS = ${PYTHON_CFLAGS}" + ;; + PYTHON_LIBS) + local val + + case "${impl}" in + python*) + # python3.8+ + val=$($(tc-getPKG_CONFIG) --libs ${impl/n/n-}-embed) || die + ;; + *) + die "${impl}: obtaining ${var} not supported" + ;; + esac + + export PYTHON_LIBS=${val} + debug-print "${FUNCNAME}: PYTHON_LIBS = ${PYTHON_LIBS}" + ;; + PYTHON_CONFIG) + local flags val + + case "${impl}" in + python*) + [[ -n ${PYTHON} ]] || die "PYTHON needs to be set for ${var} to be exported, or requested before it" + flags=$( + "${PYTHON}" - <<-EOF || die + import sysconfig + print(sysconfig.get_config_var("ABIFLAGS") + or "") + EOF + ) + val=${PYTHON}${flags}-config + ;; + *) + die "${impl}: obtaining ${var} not supported" + ;; + esac + + export PYTHON_CONFIG=${val} + debug-print "${FUNCNAME}: PYTHON_CONFIG = ${PYTHON_CONFIG}" + ;; + PYTHON_PKG_DEP) + local d + case ${impl} in + python*) + PYTHON_PKG_DEP="dev-lang/python:${impl#python}" + ;; + pypy3) + PYTHON_PKG_DEP="dev-python/${impl}:=" + ;; + *) + die "Invalid implementation: ${impl}" + esac + + # use-dep + if [[ ${PYTHON_REQ_USE} ]]; then + PYTHON_PKG_DEP+=[${PYTHON_REQ_USE}] + fi + + export PYTHON_PKG_DEP + debug-print "${FUNCNAME}: PYTHON_PKG_DEP = ${PYTHON_PKG_DEP}" + ;; + PYTHON_SCRIPTDIR) + local dir + export PYTHON_SCRIPTDIR=${EPREFIX}/usr/lib/python-exec/${impl} + debug-print "${FUNCNAME}: PYTHON_SCRIPTDIR = ${PYTHON_SCRIPTDIR}" + ;; + *) + die "_python_export: unknown variable ${var}" + esac + done +} + +# @FUNCTION: python_get_sitedir +# @USAGE: [] +# @DESCRIPTION: +# Obtain and print the 'site-packages' path for the given +# implementation. If no implementation is provided, ${EPYTHON} will +# be used. +python_get_sitedir() { + debug-print-function ${FUNCNAME} "${@}" + + _python_export "${@}" PYTHON_SITEDIR + echo "${PYTHON_SITEDIR}" +} + +# @FUNCTION: python_get_includedir +# @USAGE: [] +# @DESCRIPTION: +# Obtain and print the include path for the given implementation. If no +# implementation is provided, ${EPYTHON} will be used. +python_get_includedir() { + debug-print-function ${FUNCNAME} "${@}" + + _python_export "${@}" PYTHON_INCLUDEDIR + echo "${PYTHON_INCLUDEDIR}" +} + +# @FUNCTION: python_get_library_path +# @USAGE: [] +# @DESCRIPTION: +# Obtain and print the Python library path for the given implementation. +# If no implementation is provided, ${EPYTHON} will be used. +# +# Please note that this function can be used with CPython only. Use +# in another implementation will result in a fatal failure. +python_get_library_path() { + debug-print-function ${FUNCNAME} "${@}" + + _python_export "${@}" PYTHON_LIBPATH + echo "${PYTHON_LIBPATH}" +} + +# @FUNCTION: python_get_CFLAGS +# @USAGE: [] +# @DESCRIPTION: +# Obtain and print the compiler flags for building against Python, +# for the given implementation. If no implementation is provided, +# ${EPYTHON} will be used. +# +# Please note that this function can be used with CPython only. +# It requires Python and pkg-config installed, and therefore proper +# build-time dependencies need be added to the ebuild. +python_get_CFLAGS() { + debug-print-function ${FUNCNAME} "${@}" + + _python_export "${@}" PYTHON_CFLAGS + echo "${PYTHON_CFLAGS}" +} + +# @FUNCTION: python_get_LIBS +# @USAGE: [] +# @DESCRIPTION: +# Obtain and print the compiler flags for linking against Python, +# for the given implementation. If no implementation is provided, +# ${EPYTHON} will be used. +# +# Please note that this function can be used with CPython only. +# It requires Python and pkg-config installed, and therefore proper +# build-time dependencies need be added to the ebuild. +python_get_LIBS() { + debug-print-function ${FUNCNAME} "${@}" + + _python_export "${@}" PYTHON_LIBS + echo "${PYTHON_LIBS}" +} + +# @FUNCTION: python_get_PYTHON_CONFIG +# @USAGE: [] +# @DESCRIPTION: +# Obtain and print the PYTHON_CONFIG location for the given +# implementation. If no implementation is provided, ${EPYTHON} will be +# used. +# +# Please note that this function can be used with CPython only. +# It requires Python installed, and therefore proper build-time +# dependencies need be added to the ebuild. +python_get_PYTHON_CONFIG() { + debug-print-function ${FUNCNAME} "${@}" + + _python_export "${@}" PYTHON_CONFIG + echo "${PYTHON_CONFIG}" +} + +# @FUNCTION: python_get_scriptdir +# @USAGE: [] +# @DESCRIPTION: +# Obtain and print the script install path for the given +# implementation. If no implementation is provided, ${EPYTHON} will +# be used. +python_get_scriptdir() { + debug-print-function ${FUNCNAME} "${@}" + + _python_export "${@}" PYTHON_SCRIPTDIR + echo "${PYTHON_SCRIPTDIR}" +} + +# @FUNCTION: python_optimize +# @USAGE: [...] +# @DESCRIPTION: +# Compile and optimize Python modules in specified directories (absolute +# paths). If no directories are provided, the default system paths +# are used (prepended with ${D}). +python_optimize() { + debug-print-function ${FUNCNAME} "${@}" + + [[ ${EPYTHON} ]] || die 'No Python implementation set (EPYTHON is null).' + + local PYTHON=${PYTHON} + [[ ${PYTHON} ]] || _python_export PYTHON + [[ -x ${PYTHON} ]] || die "PYTHON (${PYTHON}) is not executable" + + # default to sys.path + if [[ ${#} -eq 0 ]]; then + local f + while IFS= read -r -d '' f; do + # 1) accept only absolute paths + # (i.e. skip '', '.' or anything like that) + # 2) skip paths which do not exist + # (python2.6 complains about them verbosely) + + if [[ ${f} == /* && -d ${D%/}${f} ]]; then + set -- "${D%/}${f}" "${@}" + fi + done < <( + "${PYTHON}" - <<-EOF || die + import sys + print("".join(x + "\0" for x in sys.path)) + EOF + ) + + debug-print "${FUNCNAME}: using sys.path: ${*/%/;}" + fi + + local jobs=$(makeopts_jobs) + local d + for d; do + # make sure to get a nice path without // + local instpath=${d#${D%/}} + instpath=/${instpath##/} + + einfo "Optimize Python modules for ${instpath}" + case "${EPYTHON}" in + python3.8) + # both levels of optimization are separate since 3.5 + "${PYTHON}" -m compileall -j "${jobs}" -q -f -d "${instpath}" "${d}" + "${PYTHON}" -O -m compileall -j "${jobs}" -q -f -d "${instpath}" "${d}" + "${PYTHON}" -OO -m compileall -j "${jobs}" -q -f -d "${instpath}" "${d}" + ;; + python*|pypy3) + # Python 3.9+ + "${PYTHON}" -m compileall -j "${jobs}" -o 0 -o 1 -o 2 --hardlink-dupes -q -f -d "${instpath}" "${d}" + ;; + pypy|jython2.7) + "${PYTHON}" -m compileall -q -f -d "${instpath}" "${d}" + ;; + *) + die "${FUNCNAME}: unexpected EPYTHON=${EPYTHON}" + ;; + esac + done +} + +# @FUNCTION: python_scriptinto +# @USAGE: +# @DESCRIPTION: +# Set the directory to which files passed to python_doexe(), +# python_doscript(), python_newexe() and python_newscript() +# are going to be installed. The new value needs to be relative +# to the installation root (${ED}). +# +# If not set explicitly, the directory defaults to /usr/bin. +# +# Example: +# @CODE +# src_install() { +# python_scriptinto /usr/sbin +# python_foreach_impl python_doscript foo +# } +# @CODE +python_scriptinto() { + debug-print-function ${FUNCNAME} "${@}" + + _PYTHON_SCRIPTROOT=${1} +} + +# @FUNCTION: python_doexe +# @USAGE: ... +# @DESCRIPTION: +# Install the given executables into the executable install directory, +# for the current Python implementation (${EPYTHON}). +# +# The executable will be wrapped properly for the Python implementation, +# though no shebang mangling will be performed. +python_doexe() { + debug-print-function ${FUNCNAME} "${@}" + + [[ ${EBUILD_PHASE} != install ]] && + die "${FUNCNAME} can only be used in src_install" + + local f + for f; do + python_newexe "${f}" "${f##*/}" + done +} + +# @FUNCTION: python_newexe +# @USAGE: +# @DESCRIPTION: +# Install the given executable into the executable install directory, +# for the current Python implementation (${EPYTHON}). +# +# The executable will be wrapped properly for the Python implementation, +# though no shebang mangling will be performed. It will be renamed +# to . +python_newexe() { + debug-print-function ${FUNCNAME} "${@}" + + [[ ${EBUILD_PHASE} != install ]] && + die "${FUNCNAME} can only be used in src_install" + [[ ${EPYTHON} ]] || die 'No Python implementation set (EPYTHON is null).' + [[ ${#} -eq 2 ]] || die "Usage: ${FUNCNAME} " + + local wrapd=${_PYTHON_SCRIPTROOT:-/usr/bin} + + local f=${1} + local newfn=${2} + + local scriptdir=$(python_get_scriptdir) + local d=${scriptdir#${EPREFIX}} + + ( + dodir "${wrapd}" + exeopts -m 0755 + exeinto "${d}" + newexe "${f}" "${newfn}" || return ${?} + ) + + # install the wrapper + local dosym=dosym + [[ ${EAPI} == 7 ]] && dosym=dosym8 + "${dosym}" -r /usr/lib/python-exec/python-exec2 "${wrapd}/${newfn}" + + # don't use this at home, just call python_doscript() instead + if [[ ${_PYTHON_REWRITE_SHEBANG} ]]; then + python_fix_shebang -q "${ED%/}/${d}/${newfn}" + fi +} + +# @FUNCTION: python_doscript +# @USAGE: ... +# @DESCRIPTION: +# Install the given scripts into the executable install directory, +# for the current Python implementation (${EPYTHON}). +# +# All specified files must start with a 'python' shebang. The shebang +# will be converted, and the files will be wrapped properly +# for the Python implementation. +# +# Example: +# @CODE +# src_install() { +# python_foreach_impl python_doscript ${PN} +# } +# @CODE +python_doscript() { + debug-print-function ${FUNCNAME} "${@}" + + [[ ${EBUILD_PHASE} != install ]] && + die "${FUNCNAME} can only be used in src_install" + + local _PYTHON_REWRITE_SHEBANG=1 + python_doexe "${@}" +} + +# @FUNCTION: python_newscript +# @USAGE: +# @DESCRIPTION: +# Install the given script into the executable install directory +# for the current Python implementation (${EPYTHON}), and name it +# . +# +# The file must start with a 'python' shebang. The shebang will be +# converted, and the file will be wrapped properly for the Python +# implementation. It will be renamed to . +# +# Example: +# @CODE +# src_install() { +# python_foreach_impl python_newscript foo.py foo +# } +# @CODE +python_newscript() { + debug-print-function ${FUNCNAME} "${@}" + + [[ ${EBUILD_PHASE} != install ]] && + die "${FUNCNAME} can only be used in src_install" + + local _PYTHON_REWRITE_SHEBANG=1 + python_newexe "${@}" +} + +# @FUNCTION: python_moduleinto +# @USAGE: +# @DESCRIPTION: +# Set the Python module install directory for python_domodule(). +# The can either be an absolute target system path (in which +# case it needs to start with a slash, and ${ED} will be prepended to +# it) or relative to the implementation's site-packages directory +# (then it must not start with a slash). The relative path can be +# specified either using the Python package notation (separated by dots) +# or the directory notation (using slashes). +# +# When not set explicitly, the modules are installed to the top +# site-packages directory. +# +# In the relative case, the exact path is determined directly +# by each python_domodule invocation. Therefore, python_moduleinto +# can be safely called before establishing the Python interpreter and/or +# a single call can be used to set the path correctly for multiple +# implementations, as can be seen in the following example. +# +# Example: +# @CODE +# src_install() { +# python_moduleinto bar +# # installs ${PYTHON_SITEDIR}/bar/baz.py +# python_foreach_impl python_domodule baz.py +# } +# @CODE +python_moduleinto() { + debug-print-function ${FUNCNAME} "${@}" + + _PYTHON_MODULEROOT=${1} +} + +# @FUNCTION: python_domodule +# @USAGE: ... +# @DESCRIPTION: +# Install the given modules (or packages) into the current Python module +# installation directory. The list can mention both modules (files) +# and packages (directories). All listed files will be installed +# for all enabled implementations, and compiled afterwards. +# +# The files are installed into ${D} when run in src_install() phase. +# Otherwise, they are installed into ${BUILD_DIR}/install location +# that is suitable for picking up by distutils-r1 in PEP517 mode. +# +# Example: +# @CODE +# src_install() { +# # (${PN} being a directory) +# python_foreach_impl python_domodule ${PN} +# } +# @CODE +python_domodule() { + debug-print-function ${FUNCNAME} "${@}" + + [[ ${EPYTHON} ]] || die 'No Python implementation set (EPYTHON is null).' + + local d + if [[ ${_PYTHON_MODULEROOT} == /* ]]; then + # absolute path + d=${_PYTHON_MODULEROOT} + else + # relative to site-packages + local sitedir=$(python_get_sitedir) + d=${sitedir#${EPREFIX}}/${_PYTHON_MODULEROOT//.//} + fi + + if [[ ${EBUILD_PHASE} == install ]]; then + ( + insopts -m 0644 + insinto "${d}" + doins -r "${@}" || return ${?} + ) + python_optimize "${ED%/}/${d}" + elif [[ -n ${BUILD_DIR} ]]; then + local dest=${BUILD_DIR}/install${EPREFIX}/${d} + mkdir -p "${dest}" || die + cp -pR "${@}" "${dest}/" || die + ( + cd "${dest}" && + chmod -R a+rX "${@##*/}" + ) || die + else + die "${FUNCNAME} can only be used in src_install or with BUILD_DIR set" + fi +} + +# @FUNCTION: python_doheader +# @USAGE: ... +# @DESCRIPTION: +# Install the given headers into the implementation-specific include +# directory. This function is unconditionally recursive, i.e. you can +# pass directories instead of files. +# +# Example: +# @CODE +# src_install() { +# python_foreach_impl python_doheader foo.h bar.h +# } +# @CODE +python_doheader() { + debug-print-function ${FUNCNAME} "${@}" + + [[ ${EBUILD_PHASE} != install ]] && + die "${FUNCNAME} can only be used in src_install" + [[ ${EPYTHON} ]] || die 'No Python implementation set (EPYTHON is null).' + + local includedir=$(python_get_includedir) + local d=${includedir#${EPREFIX}} + + ( + insopts -m 0644 + insinto "${d}" + doins -r "${@}" || return ${?} + ) +} + +# @FUNCTION: _python_wrapper_setup +# @USAGE: [ []] +# @INTERNAL +# @DESCRIPTION: +# Create proper 'python' executable and pkg-config wrappers +# (if available) in the directory named by . Set up PATH +# and PKG_CONFIG_PATH appropriately. defaults to ${T}/${EPYTHON}. +# +# The wrappers will be created for implementation named by , +# or for one named by ${EPYTHON} if no passed. +# +# If the named directory contains a python symlink already, it will +# be assumed to contain proper wrappers already and only environment +# setup will be done. If wrapper update is requested, the directory +# shall be removed first. +_python_wrapper_setup() { + debug-print-function ${FUNCNAME} "${@}" + + local workdir=${1:-${T}/${EPYTHON}} + local impl=${2:-${EPYTHON}} + + [[ ${workdir} ]] || die "${FUNCNAME}: no workdir specified." + [[ ${impl} ]] || die "${FUNCNAME}: no impl nor EPYTHON specified." + + if [[ ! -x ${workdir}/bin/python ]]; then + mkdir -p "${workdir}"/{bin,pkgconfig} || die + + # Clean up, in case we were supposed to do a cheap update. + rm -f "${workdir}"/bin/python{,2,3}{,-config} || die + rm -f "${workdir}"/bin/2to3 || die + rm -f "${workdir}"/pkgconfig/python{2,3}{,-embed}.pc || die + + local EPYTHON PYTHON + _python_export "${impl}" EPYTHON PYTHON + + # Python interpreter + # note: we don't use symlinks because python likes to do some + # symlink reading magic that breaks stuff + # https://bugs.gentoo.org/show_bug.cgi?id=555752 + cat > "${workdir}/bin/python" <<-_EOF_ || die + #!/bin/sh + exec "${PYTHON}" "\${@}" + _EOF_ + cp "${workdir}/bin/python" "${workdir}/bin/python3" || die + chmod +x "${workdir}/bin/python" "${workdir}/bin/python3" || die + + local nonsupp=( python2 python2-config ) + + # CPython-specific + if [[ ${EPYTHON} == python* ]]; then + cat > "${workdir}/bin/python-config" <<-_EOF_ || die + #!/bin/sh + exec "${PYTHON}-config" "\${@}" + _EOF_ + cp "${workdir}/bin/python-config" \ + "${workdir}/bin/python3-config" || die + chmod +x "${workdir}/bin/python-config" \ + "${workdir}/bin/python3-config" || die + + # Python 2.6+. + ln -s "${PYTHON/python/2to3-}" "${workdir}"/bin/2to3 || die + + # Python 2.7+. + ln -s "${EPREFIX}"/usr/$(get_libdir)/pkgconfig/${EPYTHON/n/n-}.pc \ + "${workdir}"/pkgconfig/python3.pc || die + + # Python 3.8+. + ln -s "${EPREFIX}"/usr/$(get_libdir)/pkgconfig/${EPYTHON/n/n-}-embed.pc \ + "${workdir}"/pkgconfig/python3-embed.pc || die + else + nonsupp+=( 2to3 python-config python3-config ) + fi + + local x + for x in "${nonsupp[@]}"; do + cat >"${workdir}"/bin/${x} <<-_EOF_ || die + #!/bin/sh + echo "${ECLASS}: ${FUNCNAME}: ${x} is not supported by ${EPYTHON} (PYTHON_COMPAT)" >&2 + exit 127 + _EOF_ + chmod +x "${workdir}"/bin/${x} || die + done + fi + + # Now, set the environment. + # But note that ${workdir} may be shared with something else, + # and thus already on top of PATH. + if [[ ${PATH##:*} != ${workdir}/bin ]]; then + PATH=${workdir}/bin${PATH:+:${PATH}} + fi + if [[ ${PKG_CONFIG_PATH##:*} != ${workdir}/pkgconfig ]]; then + PKG_CONFIG_PATH=${workdir}/pkgconfig${PKG_CONFIG_PATH:+:${PKG_CONFIG_PATH}} + fi + export PATH PKG_CONFIG_PATH +} + +# @FUNCTION: python_fix_shebang +# @USAGE: [-f|--force] [-q|--quiet] ... +# @DESCRIPTION: +# Replace the shebang in Python scripts with the full path +# to the current Python implementation (PYTHON, including EPREFIX). +# If a directory is passed, works recursively on all Python scripts +# found inside the directory tree. +# +# Only files having a Python shebang (a path to any known Python +# interpreter, optionally preceded by env(1) invocation) will +# be processed. Files with any other shebang will either be skipped +# silently when a directory was passed, or an error will be reported +# for any files without Python shebangs specified explicitly. +# +# Shebangs that are compatible with the current Python version will be +# mangled unconditionally. Incompatible shebangs will cause a fatal +# error, unless --force is specified. +# +# --force causes the function to replace shebangs with incompatible +# Python version (but not non-Python shebangs). --quiet causes +# the function not to list modified files verbosely. +python_fix_shebang() { + debug-print-function ${FUNCNAME} "${@}" + + [[ ${EPYTHON} ]] || die "${FUNCNAME}: EPYTHON unset (pkg_setup not called?)" + + local force quiet + while [[ ${@} ]]; do + case "${1}" in + -f|--force) force=1; shift;; + -q|--quiet) quiet=1; shift;; + --) shift; break;; + *) break;; + esac + done + + [[ ${1} ]] || die "${FUNCNAME}: no paths given" + + local path f + for path; do + local any_fixed is_recursive + + [[ -d ${path} ]] && is_recursive=1 + + while IFS= read -r -d '' f; do + local shebang i + local error= match= + + # note: we can't ||die here since read will fail if file + # has no newline characters + IFS= read -r shebang <"${f}" + + # First, check if it's shebang at all... + if [[ ${shebang} == '#!'* ]]; then + local split_shebang=() + read -r -a split_shebang <<<${shebang#"#!"} || die + + local in_path=${split_shebang[0]} + local from='^#! *[^ ]*' + # if the first component is env(1), skip it + if [[ ${in_path} == */env ]]; then + in_path=${split_shebang[1]} + from+=' *[^ ]*' + fi + + case ${in_path##*/} in + "${EPYTHON}") + match=1 + ;; + python|python3) + match=1 + ;; + python2|python[23].[0-9]|python3.[1-9][0-9]|pypy|pypy3|jython[23].[0-9]) + # Explicit mismatch. + match=1 + error=1 + ;; + esac + fi + + # disregard mismatches in force mode + [[ ${force} ]] && error= + + if [[ ! ${match} ]]; then + # Non-Python shebang. Allowed in recursive mode, + # disallowed when specifying file explicitly. + [[ ${is_recursive} ]] && continue + error=1 + fi + + if [[ ! ${quiet} ]]; then + einfo "Fixing shebang in ${f#${D%/}}." + fi + + if [[ ! ${error} ]]; then + debug-print "${FUNCNAME}: in file ${f#${D%/}}" + debug-print "${FUNCNAME}: rewriting shebang: ${shebang}" + sed -i -e "1s@${from}@#!${EPREFIX}/usr/bin/${EPYTHON}@" "${f}" || die + any_fixed=1 + else + eerror "The file has incompatible shebang:" + eerror " file: ${f#${D%/}}" + eerror " current shebang: ${shebang}" + eerror " requested impl: ${EPYTHON}" + die "${FUNCNAME}: conversion of incompatible shebang requested" + fi + done < <(find -H "${path}" -type f -print0 || die) + + if [[ ! ${any_fixed} ]]; then + eerror "QA error: ${FUNCNAME}, ${path#${D%/}} did not match any fixable files." + eerror "There are no Python files in specified directory." + die "${FUNCNAME} did not match any fixable files" + fi + done +} + +# @FUNCTION: _python_check_locale_sanity +# @USAGE: +# @RETURN: 0 if sane, 1 otherwise +# @INTERNAL +# @DESCRIPTION: +# Check whether the specified locale sanely maps between lowercase +# and uppercase ASCII characters. +_python_check_locale_sanity() { + local -x LC_ALL=${1} + local IFS= + + local lc=( {a..z} ) + local uc=( {A..Z} ) + local input="${lc[*]}${uc[*]}" + + local output=$(tr '[:lower:][:upper:]' '[:upper:][:lower:]' <<<"${input}") + [[ ${output} == "${uc[*]}${lc[*]}" ]] +} + +# @FUNCTION: python_export_utf8_locale +# @RETURN: 0 on success, 1 on failure. +# @DESCRIPTION: +# Attempts to export a usable UTF-8 locale in the LC_CTYPE variable. Does +# nothing if LC_ALL is defined, or if the current locale uses a UTF-8 charmap. +# This may be used to work around the quirky open() behavior of python3. +python_export_utf8_locale() { + debug-print-function ${FUNCNAME} "${@}" + + # If the locale program isn't available, just return. + type locale &>/dev/null || return 0 + + if [[ $(locale charmap) != UTF-8 ]]; then + # Try English first, then everything else. + local lang locales="C.UTF-8 en_US.UTF-8 en_GB.UTF-8 $(locale -a)" + + for lang in ${locales}; do + if [[ $(LC_ALL=${lang} locale charmap 2>/dev/null) == UTF-8 ]]; then + if _python_check_locale_sanity "${lang}"; then + export LC_CTYPE=${lang} + if [[ -n ${LC_ALL} ]]; then + export LC_NUMERIC=${LC_ALL} + export LC_TIME=${LC_ALL} + export LC_COLLATE=${LC_ALL} + export LC_MONETARY=${LC_ALL} + export LC_MESSAGES=${LC_ALL} + export LC_PAPER=${LC_ALL} + export LC_NAME=${LC_ALL} + export LC_ADDRESS=${LC_ALL} + export LC_TELEPHONE=${LC_ALL} + export LC_MEASUREMENT=${LC_ALL} + export LC_IDENTIFICATION=${LC_ALL} + export LC_ALL= + fi + return 0 + fi + fi + done + + ewarn "Could not find a UTF-8 locale. This may trigger build failures in" + ewarn "some python packages. Please ensure that a UTF-8 locale is listed in" + ewarn "/etc/locale.gen and run locale-gen." + return 1 + fi + + return 0 +} + +# @FUNCTION: build_sphinx +# @USAGE: +# @DESCRIPTION: +# Build HTML documentation using dev-python/sphinx in the specified +# . Takes care of disabling Intersphinx and appending +# to HTML_DOCS. +# +# If is relative to the current directory, care needs +# to be taken to run einstalldocs from the same directory +# (usually ${S}). +build_sphinx() { + debug-print-function ${FUNCNAME} "${@}" + [[ ${#} -eq 1 ]] || die "${FUNCNAME} takes 1 arg: " + + local dir=${1} + + sed -i -e 's:^intersphinx_mapping:disabled_&:' \ + "${dir}"/conf.py || die + # 1. not all packages include the Makefile in pypi tarball, + # so we call sphinx-build directly + # 2. if autodoc is used, we need to call sphinx via EPYTHON, + # to ensure that PEP 517 venv is respected + # 3. if autodoc is not used, then sphinx might not be installed + # for the current impl, so we need a fallback to sphinx-build + local command=( "${EPYTHON}" -m sphinx.cmd.build ) + if ! "${EPYTHON}" -c "import sphinx.cmd.build" 2>/dev/null; then + command=( sphinx-build ) + fi + command+=( + -b html + -d "${dir}"/_build/doctrees + "${dir}" + "${dir}"/_build/html + ) + echo "${command[@]}" >&2 + "${command[@]}" || die + + HTML_DOCS+=( "${dir}/_build/html/." ) +} + +# @FUNCTION: _python_check_EPYTHON +# @INTERNAL +# @DESCRIPTION: +# Check if EPYTHON is set, die if not. +_python_check_EPYTHON() { + if [[ -z ${EPYTHON} ]]; then + die "EPYTHON unset, invalid call context" + fi +} + +# @FUNCTION: _python_check_occluded_packages +# @INTERNAL +# @DESCRIPTION: +# Check if the current directory does not contain any incomplete +# package sources that would block installed packages from being used +# (and effectively e.g. make it impossible to load compiled extensions). +_python_check_occluded_packages() { + debug-print-function ${FUNCNAME} "${@}" + + # DO NOT ENABLE THIS unless you're going to check for false + # positives before filing bugs. + [[ ! ${PYTHON_EXPERIMENTAL_QA} ]] && return + + [[ -z ${BUILD_DIR} || ! -d ${BUILD_DIR}/install ]] && return + + local sitedir="${BUILD_DIR}/install$(python_get_sitedir)" + # avoid unnecessarily checking if we are inside install dir + [[ ${sitedir} -ef . ]] && return + + local f fn diff l + for f in "${sitedir}"/*/; do + f=${f%/} + fn=${f##*/} + + # skip metadata directories + [[ ${fn} == *.dist-info || ${fn} == *.egg-info ]] && continue + + if [[ -d ${fn} ]]; then + diff=$( + comm -1 -3 <( + find "${fn}" -type f -not -path '*/__pycache__/*' | + sort + assert + ) <( + cd "${sitedir}" && + find "${fn}" -type f -not -path '*/__pycache__/*' | + sort + assert + ) + ) + + if [[ -n ${diff} ]]; then + eqawarn "The directory ${fn} occludes package installed for ${EPYTHON}." + eqawarn "The installed package includes additional files:" + eqawarn + while IFS= read -r l; do + eqawarn " ${l}" + done <<<"${diff}" + eqawarn + + if [[ ! ${_PYTHON_WARNED_OCCLUDED_PACKAGES} ]]; then + eqawarn "For more information on occluded packages, please see:" + eqawarn "https://projects.gentoo.org/python/guide/test.html#importerrors-for-c-extensions" + _PYTHON_WARNED_OCCLUDED_PACKAGES=1 + fi + fi + fi + done +} + +# @VARIABLE: EPYTEST_DESELECT +# @DEFAULT_UNSET +# @DESCRIPTION: +# Specifies an array of tests to be deselected via pytest's --deselect +# parameter, when calling epytest. The list can include file paths, +# specific test functions or parametrized test invocations. +# +# Note that the listed files will still be subject to collection, +# i.e. modules imported in global scope will need to be available. +# If this is undesirable, EPYTEST_IGNORE can be used instead. + +# @VARIABLE: EPYTEST_IGNORE +# @DEFAULT_UNSET +# @DESCRIPTION: +# Specifies an array of paths to be ignored via pytest's --ignore +# parameter, when calling epytest. The listed files will be entirely +# skipped from test collection. + +# @FUNCTION: epytest +# @USAGE: [...] +# @DESCRIPTION: +# Run pytest, passing the standard set of pytest options, then +# --deselect and --ignore options based on EPYTEST_DESELECT +# and EPYTEST_IGNORE, then user-specified options. +# +# This command dies on failure and respects nonfatal. +epytest() { + debug-print-function ${FUNCNAME} "${@}" + + _python_check_EPYTHON + _python_check_occluded_packages + + local color + case ${NOCOLOR} in + true|yes) + color=no + ;; + *) + color=yes + ;; + esac + + local args=( + # verbose progress reporting and tracebacks + -vv + # list all non-passed tests in the summary for convenience + # (includes failures, skips, xfails...) + -ra + # print local variables in tracebacks, useful for debugging + -l + # override filterwarnings=error, we do not really want -Werror + # for end users, as it tends to fail on new warnings from deps + -Wdefault + # override color output + "--color=${color}" + # count is more precise when we're dealing with a large number + # of tests + -o console_output_style=count + # disable the undesirable-dependency plugins by default to + # trigger missing argument strips. strip options that require + # them from config files. enable them explicitly via "-p ..." + # if you *really* need them. + -p no:cov + -p no:flake8 + -p no:flakes + -p no:pylint + # sterilize pytest-markdown as it runs code snippets from all + # *.md files found without any warning + -p no:markdown + # pytest-sugar undoes everything that's good about pytest output + # and makes it hard to read logs + -p no:sugar + # pytest-xvfb automatically spawns Xvfb for every test suite, + # effectively forcing it even when we'd prefer the tests + # not to have DISPLAY at all, causing crashes sometimes + # and causing us to miss missing virtualx usage + -p no:xvfb + # tavern is intrusive and breaks test suites of various packages + -p no:tavern + ) + local x + for x in "${EPYTEST_DESELECT[@]}"; do + args+=( --deselect "${x}" ) + done + for x in "${EPYTEST_IGNORE[@]}"; do + args+=( --ignore "${x}" ) + done + set -- "${EPYTHON}" -m pytest "${args[@]}" "${@}" + + echo "${@}" >&2 + "${@}" || die -n "pytest failed with ${EPYTHON}" + local ret=${?} + + # remove common temporary directories left over by pytest plugins + rm -rf .hypothesis .pytest_cache || die + # pytest plugins create additional .pyc files while testing + # see e.g. https://bugs.gentoo.org/847235 + if [[ -n ${BUILD_DIR} && -d ${BUILD_DIR} ]]; then + find "${BUILD_DIR}" -name '*-pytest-*.pyc' -delete || die + fi + + return ${ret} +} + +# @FUNCTION: eunittest +# @USAGE: [...] +# @DESCRIPTION: +# Run unit tests using dev-python/unittest-or-fail, passing the standard +# set of options, followed by user-specified options. +# +# This command dies on failure and respects nonfatal. +eunittest() { + debug-print-function ${FUNCNAME} "${@}" + + _python_check_EPYTHON + _python_check_occluded_packages + + # unittest fails with "no tests" correctly since Python 3.12 + local runner=unittest + if _python_impl_matches "${EPYTHON}" 3.{9..11}; then + runner=unittest_or_fail + fi + set -- "${EPYTHON}" -m "${runner}" discover -v "${@}" + + echo "${@}" >&2 + "${@}" || die -n "Tests failed with ${EPYTHON}" + return ${?} +} + +# @FUNCTION: _python_run_check_deps +# @INTERNAL +# @USAGE: +# @DESCRIPTION: +# Verify whether is an acceptable choice to run any-r1 style +# code. Checks whether the interpreter is installed, runs +# python_check_deps() if declared. +_python_run_check_deps() { + debug-print-function ${FUNCNAME} "${@}" + + local impl=${1} + + einfo "Checking whether ${impl} is suitable ..." + + local PYTHON_PKG_DEP + _python_export "${impl}" PYTHON_PKG_DEP + ebegin " ${PYTHON_PKG_DEP}" + has_version -b "${PYTHON_PKG_DEP}" + eend ${?} || return 1 + declare -f python_check_deps >/dev/null || return 0 + + local PYTHON_USEDEP="python_targets_${impl}(-)" + local PYTHON_SINGLE_USEDEP="python_single_target_${impl}(-)" + ebegin " python_check_deps" + python_check_deps + eend ${?} +} + +# @FUNCTION: python_has_version +# @USAGE: [-b|-d|-r] ... +# @DESCRIPTION: +# A convenience wrapper for has_version() with verbose output and better +# defaults for use in python_check_deps(). +# +# The wrapper accepts -b/-d/-r options to indicate the root to perform +# the lookup on. Unlike has_version, the default is -b. +# +# The wrapper accepts multiple package specifications. For the check +# to succeed, *all* specified atoms must match. +python_has_version() { + debug-print-function ${FUNCNAME} "${@}" + + local root_arg=( -b ) + case ${1} in + -b|-d|-r) + root_arg=( "${1}" ) + shift + ;; + esac + + local pkg + for pkg; do + ebegin " ${pkg}" + has_version "${root_arg[@]}" "${pkg}" + eend ${?} || return + done + + return 0 +} + +fi diff --git a/sdk_container/src/third_party/prefix-overlay/metadata/layout.conf b/sdk_container/src/third_party/prefix-overlay/metadata/layout.conf new file mode 100644 index 0000000000..a83f5b1222 --- /dev/null +++ b/sdk_container/src/third_party/prefix-overlay/metadata/layout.conf @@ -0,0 +1,5 @@ +masters = portage-stable +thin-manifests = true +use-manifests = strict +cache-format = md5-dict +profile-formats = portage-2 diff --git a/sdk_container/src/third_party/prefix-overlay/prefix/prefix-final/metadata.xml b/sdk_container/src/third_party/prefix-overlay/prefix/prefix-final/metadata.xml new file mode 100644 index 0000000000..221192c916 --- /dev/null +++ b/sdk_container/src/third_party/prefix-overlay/prefix/prefix-final/metadata.xml @@ -0,0 +1,8 @@ + + + +flatcar +Flatcar distro-independent prefix "final" packages. + + + diff --git a/sdk_container/src/third_party/prefix-overlay/prefix/prefix-final/prefix-final-0.0.1-r1.ebuild b/sdk_container/src/third_party/prefix-overlay/prefix/prefix-final/prefix-final-0.0.1-r1.ebuild new file mode 120000 index 0000000000..3bff3013e9 --- /dev/null +++ b/sdk_container/src/third_party/prefix-overlay/prefix/prefix-final/prefix-final-0.0.1-r1.ebuild @@ -0,0 +1 @@ +prefix-final-0.0.1.ebuild \ No newline at end of file diff --git a/sdk_container/src/third_party/prefix-overlay/prefix/prefix-final/prefix-final-0.0.1.ebuild b/sdk_container/src/third_party/prefix-overlay/prefix/prefix-final/prefix-final-0.0.1.ebuild new file mode 100644 index 0000000000..e3cd428c25 --- /dev/null +++ b/sdk_container/src/third_party/prefix-overlay/prefix/prefix-final/prefix-final-0.0.1.ebuild @@ -0,0 +1,16 @@ +# Copyright (c) 2032 the Flatcar maintainers. +# Distributed under the terms of the Apache 2.0 license. + +EAPI=7 + +DESCRIPTION="Prefix final layer base dependencies" + +LICENSE="Apache-2.0" +SLOT="0" +KEYWORDS="amd64 arm64" + +# These should be the absolute minimum runtime dependencies of the "final" prefix. +# "Staging" has @system so it is pretty heavyweight. +RDEPEND=" + virtual/libc +" diff --git a/sdk_container/src/third_party/prefix-overlay/profiles/categories b/sdk_container/src/third_party/prefix-overlay/profiles/categories new file mode 100644 index 0000000000..9c5a215fe7 --- /dev/null +++ b/sdk_container/src/third_party/prefix-overlay/profiles/categories @@ -0,0 +1 @@ +prefix diff --git a/sdk_container/src/third_party/prefix-overlay/profiles/repo_name b/sdk_container/src/third_party/prefix-overlay/profiles/repo_name new file mode 100644 index 0000000000..9c5a215fe7 --- /dev/null +++ b/sdk_container/src/third_party/prefix-overlay/profiles/repo_name @@ -0,0 +1 @@ +prefix diff --git a/sdk_container/src/third_party/prefix-overlay/skel/etc/group b/sdk_container/src/third_party/prefix-overlay/skel/etc/group new file mode 120000 index 0000000000..3075d99a60 --- /dev/null +++ b/sdk_container/src/third_party/prefix-overlay/skel/etc/group @@ -0,0 +1 @@ +/etc/group \ No newline at end of file diff --git a/sdk_container/src/third_party/prefix-overlay/skel/etc/passwd b/sdk_container/src/third_party/prefix-overlay/skel/etc/passwd new file mode 120000 index 0000000000..3594e94c04 --- /dev/null +++ b/sdk_container/src/third_party/prefix-overlay/skel/etc/passwd @@ -0,0 +1 @@ +/etc/passwd \ No newline at end of file diff --git a/sdk_container/src/third_party/prefix-overlay/skel/etc/portage/package.accept_keywords/portage b/sdk_container/src/third_party/prefix-overlay/skel/etc/portage/package.accept_keywords/portage new file mode 100644 index 0000000000..7b038e1a64 --- /dev/null +++ b/sdk_container/src/third_party/prefix-overlay/skel/etc/portage/package.accept_keywords/portage @@ -0,0 +1 @@ +sys-apps/portage ~amd64 diff --git a/sdk_container/src/third_party/prefix-overlay/skel/etc/portage/package.accept_keywords/prefix b/sdk_container/src/third_party/prefix-overlay/skel/etc/portage/package.accept_keywords/prefix new file mode 100644 index 0000000000..e8fefc40da --- /dev/null +++ b/sdk_container/src/third_party/prefix-overlay/skel/etc/portage/package.accept_keywords/prefix @@ -0,0 +1,2 @@ +=app-misc/ca-certificates-20230311.3.93 ~* +=sys-apps/systemd-utils-254.3 ~* diff --git a/sdk_container/src/third_party/prefix-overlay/skel/etc/portage/package.mask/coreos b/sdk_container/src/third_party/prefix-overlay/skel/etc/portage/package.mask/coreos new file mode 100644 index 0000000000..c619723b27 --- /dev/null +++ b/sdk_container/src/third_party/prefix-overlay/skel/etc/portage/package.mask/coreos @@ -0,0 +1,2 @@ +app-misc/ca-certificates::coreos # coreos ca-certificates doesn't support prefix +sys-apps/baselayout::coreos # coreos baselayout depends on systemd diff --git a/sdk_container/src/third_party/prefix-overlay/skel/etc/portage/package.mask/cross b/sdk_container/src/third_party/prefix-overlay/skel/etc/portage/package.mask/cross new file mode 100644 index 0000000000..f6bb7302ef --- /dev/null +++ b/sdk_container/src/third_party/prefix-overlay/skel/etc/portage/package.mask/cross @@ -0,0 +1 @@ +>=sys-devel/gcc-13 # FIXME: Cannot cross-compile gcc 13+ using older version diff --git a/sdk_container/src/third_party/prefix-overlay/skel/etc/portage/repos.conf/coreos.conf b/sdk_container/src/third_party/prefix-overlay/skel/etc/portage/repos.conf/coreos.conf new file mode 100644 index 0000000000..d73d143487 --- /dev/null +++ b/sdk_container/src/third_party/prefix-overlay/skel/etc/portage/repos.conf/coreos.conf @@ -0,0 +1,8 @@ +[DEFAULT] +main-repo = portage-stable + +[coreos] +location = /mnt/host/source/src/third_party/coreos-overlay + +[portage-stable] +location = /mnt/host/source/src/third_party/portage-stable diff --git a/sdk_container/src/third_party/prefix-overlay/skel/etc/portage/repos.conf/crossdev.conf b/sdk_container/src/third_party/prefix-overlay/skel/etc/portage/repos.conf/crossdev.conf new file mode 100644 index 0000000000..ef2bcc750a --- /dev/null +++ b/sdk_container/src/third_party/prefix-overlay/skel/etc/portage/repos.conf/crossdev.conf @@ -0,0 +1,2 @@ +[x-crossdev] +location = /usr/local/portage/crossdev diff --git a/sdk_container/src/third_party/prefix-overlay/skel/etc/portage/repos.conf/prefix.conf b/sdk_container/src/third_party/prefix-overlay/skel/etc/portage/repos.conf/prefix.conf new file mode 100644 index 0000000000..af80e47e24 --- /dev/null +++ b/sdk_container/src/third_party/prefix-overlay/skel/etc/portage/repos.conf/prefix.conf @@ -0,0 +1,3 @@ +[prefix] +location = /mnt/host/source/src/third_party/prefix-overlay +priority = 1000 diff --git a/sdk_container/src/third_party/prefix-overlay/sys-apps/baselayout/Manifest b/sdk_container/src/third_party/prefix-overlay/sys-apps/baselayout/Manifest new file mode 100644 index 0000000000..91204109c0 --- /dev/null +++ b/sdk_container/src/third_party/prefix-overlay/sys-apps/baselayout/Manifest @@ -0,0 +1 @@ +DIST baselayout-2.14.tar.bz2 30182 BLAKE2B c5f67795233e565c2c75c97a55c000aec98e901bb0a25f1aeb52b01b44d7c09bfc6e67813234629ca71ff32d603e82ada8e66e5ab6007fa0664b95367256320d SHA512 bffd118f5e92975b9247d854fc5683a311dbcd03efa37a13dfd05d04e92a6e784858d3a55aa689f782229afc5985e829eb332c08a79eed081bf0a47720ca7e8a diff --git a/sdk_container/src/third_party/prefix-overlay/sys-apps/baselayout/baselayout-2.14.ebuild b/sdk_container/src/third_party/prefix-overlay/sys-apps/baselayout/baselayout-2.14.ebuild new file mode 100644 index 0000000000..98121cf026 --- /dev/null +++ b/sdk_container/src/third_party/prefix-overlay/sys-apps/baselayout/baselayout-2.14.ebuild @@ -0,0 +1,350 @@ +# Copyright 1999-2023 Gentoo Authors +# Distributed under the terms of the GNU General Public License v2 + +EAPI=7 + +inherit multilib prefix + +DESCRIPTION="Filesystem baselayout and init scripts" +HOMEPAGE="https://wiki.gentoo.org/wiki/No_homepage" +if [[ ${PV} = 9999 ]]; then + EGIT_REPO_URI="https://anongit.gentoo.org/git/proj/${PN}.git" + inherit git-r3 +else + SRC_URI="https://gitweb.gentoo.org/proj/${PN}.git/snapshot/${P}.tar.bz2" + KEYWORDS="~alpha amd64 arm arm64 hppa ~ia64 ~loong ~m68k ~mips ppc ppc64 ~riscv ~s390 sparc x86 ~amd64-linux ~x86-linux ~arm64-macos ~ppc-macos ~x64-macos ~x64-solaris" +fi + +LICENSE="GPL-2" +SLOT="0" +IUSE="build +split-usr" + +RDEPEND="!sys-apps/baselayout-prefix" + +riscv_compat_symlink() { + # Here we apply some special sauce for riscv. + # Two multilib layouts exist for now: + # 1) one level libdirs, (32bit) "lib" and (64bit) "lib64" + # these are chosen by us to closely resemble other arches + # 2) two level libdirs, "lib64/lp64d" "lib64/lp64" "lib32/ilp32d" ... + # this is the glibc/gcc default + # Unfortunately, the default has only one fallback, which is "lib" + # for both 32bit and 64bit. So things do not break in 1), we need + # to provide compatibility symlinks... + + # This function has exactly two parameters: + # - the default libdir, to determine if 1) or 2) applies + # - the location of the symlink (which points to ".") + + # Note: we call this only in the ${SYMLINK_LIB} = no codepath, since + # there never was a ${SYMLINK_LIB} = yes riscv profile. + + case ${CHOST} in + riscv*) + # are we on a one level libdir profile? is there no symlink yet? + if [[ ${1} != */* && ! -L ${2} ]] ; then + ln -s . $2 || die "Unable to make $2 riscv compatibility symlink" + fi + ;; + esac +} + +# Create our multilib dirs - the Makefile has no knowledge of this +multilib_layout() { + local dir def_libdir libdir libdirs + local prefix prefix_lst + def_libdir=$(get_abi_LIBDIR $DEFAULT_ABI) + libdirs=$(get_all_libdirs) + + if [[ -z "${SYMLINK_LIB}" || ${SYMLINK_LIB} = no ]] ; then + prefix_lst=( "${EROOT}"/{,usr/,usr/local/} ) + for prefix in "${prefix_lst[@]}"; do + for libdir in ${libdirs}; do + dir="${prefix}${libdir}" + if [[ -e "${dir}" ]]; then + [[ ! -d "${dir}" ]] && + die "${dir} exists but is not a directory" + continue + fi + if ! use split-usr && [[ ${prefix} = ${EROOT}/ ]]; then + libdir="${libdir%%/*}" + dir="${prefix}${libdir}" + einfo "symlinking ${dir} to usr/${libdir}" + ln -s usr/${libdir} ${dir} || + die "Unable to make ${dir} symlink" + else + einfo "creating directory ${dir}" + mkdir -p "${dir}" || + die "Unable to create ${dir} directory" + fi + done + [[ -d "${prefix}${def_libdir}" ]] && riscv_compat_symlink "${def_libdir}" "${prefix}${def_libdir}/${DEFAULT_ABI}" + done + return 0 + fi + + [ -z "${def_libdir}" ] && + die "your DEFAULT_ABI=$DEFAULT_ABI appears to be invalid" + + # figure out which paths should be symlinks and which should be directories + local dirs syms exp d + for libdir in ${libdirs} ; do + if use split-usr ; then + exp=( {,usr/,usr/local/}${libdir} ) + else + exp=( {usr/,usr/local/}${libdir} ) + fi + for d in "${exp[@]}" ; do + # most things should be dirs + if [ "${SYMLINK_LIB}" = "yes" ] && [ "${libdir}" = "lib" ] ; then + [ ! -h "${d}" ] && [ -e "${d}" ] && dirs+=" ${d}" + else + [ -h "${d}" ] && syms+=" ${d}" + fi + done + done + if [ -n "${syms}${dirs}" ] ; then + ewarn "Your system profile has SYMLINK_LIB=${SYMLINK_LIB:-no}, so that means you need to" + ewarn "have these paths configured as follows:" + [ -n "${dirs}" ] && ewarn "symlinks to '${def_libdir}':${dirs}" + [ -n "${syms}" ] && ewarn "directories:${syms}" + ewarn "The ebuild will attempt to fix these, but only for trivial conversions." + ewarn "If things fail, you will need to manually create/move the directories." + echo + fi + + # setup symlinks and dirs where we expect them to be; do not migrate + # data ... just fall over in that case. + if use split-usr ; then + prefix_lst=( "${EROOT}"/{,usr/,usr/local/} ) + else + prefix_lst=( "${EROOT}"/{usr/,usr/local/} ) + fi + for prefix in "${prefix_lst[@]}"; do + if [ "${SYMLINK_LIB}" = yes ] ; then + # we need to make sure "lib" points to the native libdir + if [ -h "${prefix}lib" ] ; then + # it's already a symlink! assume it's pointing to right place ... + continue + elif [ -d "${prefix}lib" ] ; then + # "lib" is a dir, so need to convert to a symlink + ewarn "Converting ${prefix}lib from a dir to a symlink" + rm -f "${prefix}lib"/.keep || die + if rmdir "${prefix}lib" 2>/dev/null ; then + ln -s ${def_libdir} "${prefix}lib" || die + else + die "non-empty dir found where we needed a symlink: ${prefix}lib" + fi + else + # nothing exists, so just set it up sanely + ewarn "Initializing ${prefix}lib as a symlink" + mkdir -p "${prefix}" || die + rm -f "${prefix}lib" || die + ln -s ${def_libdir} "${prefix}lib" || die + mkdir -p "${prefix}${def_libdir}" || die #423571 + fi + else + # we need to make sure "lib" is a dir + if [ -h "${prefix}lib" ] ; then + # "lib" is a symlink, so need to convert to a dir + ewarn "Converting ${prefix}lib from a symlink to a dir" + rm -f "${prefix}lib" || die + if [ -d "${prefix}lib32" ] ; then + ewarn "Migrating ${prefix}lib32 to ${prefix}lib" + mv "${prefix}lib32" "${prefix}lib" || die + else + mkdir -p "${prefix}lib" || die + fi + elif [ -d "${prefix}lib" ] && ! has lib32 ${libdirs} ; then + # make sure the old "lib" ABI location does not exist; we + # only symlinked the lib dir on systems where we moved it + # to "lib32" ... + case ${CHOST} in + i?86*|x86_64*|powerpc*|sparc*|s390*) + if [[ -d ${prefix}lib32 && ! -h ${prefix}lib32 ]] ; then + rm -f "${prefix}lib32"/.keep || die + if ! rmdir "${prefix}lib32" 2>/dev/null ; then + ewarn "You need to merge ${prefix}lib32 into ${prefix}lib" + die "non-empty dir found where there should be none: ${prefix}lib32" + fi + fi + ;; + esac + else + # nothing exists, so just set it up sanely + ewarn "Initializing ${prefix}lib as a dir" + mkdir -p "${prefix}lib" || die + fi + fi + done + if ! use split-usr ; then + for libdir in ${libdirs}; do + if [[ ! -e "${EROOT}${libdir}" ]]; then + ln -s usr/"${libdir}" "${EROOT}${libdir}" || + die "Unable to make ${EROOT}${libdir} symlink" + fi + done + fi +} + +pkg_setup() { + multilib_layout +} + +src_prepare() { + default + + # don't want symlinked directories in PATH on systems with usr-merge + if ! use split-usr && ! use prefix-guest; then + sed \ + -e 's|:/usr/sbin:|:|g' \ + -e 's|:/sbin:|:|g' \ + -e 's|:/bin:|:|g' \ + -i etc/env.d/50baselayout || die + fi + + if use prefix; then + hprefixify -e "/EUID/s,0,${EUID}," -q '"' etc/profile + hprefixify etc/shells share/passwd + hprefixify -w '/PATH=/' etc/env.d/50baselayout + hprefixify -w 1 etc/env.d/50baselayout + echo PATH=/usr/sbin:/sbin:/usr/bin:/bin >> etc/env.d/99host + + # change branding + sed -i \ + -e '/gentoo-release/s/Gentoo Base/Gentoo Prefix Base/' \ + -e '/make_os_release/s/${OS}/Prefix/' \ + Makefile || die + fi + + # handle multilib paths. do it here because we want this behavior + # regardless of the C library that you're using. we do explicitly + # list paths which the native ldconfig searches, but this isn't + # problematic as it doesn't change the resulting ld.so.cache or + # take longer to generate. similarly, listing both the native + # path and the symlinked path doesn't change the resulting cache. + local libdir ldpaths + for libdir in $(get_all_libdirs) ; do + if use split-usr || use prefix-guest; then + ldpaths+=":${EPREFIX}/${libdir}" + fi + ldpaths+=":${EPREFIX}/usr/${libdir}" + ldpaths+=":${EPREFIX}/usr/local/${libdir}" + done + echo "LDPATH='${ldpaths#:}'" >> etc/env.d/50baselayout +} + +src_install() { + emake \ + DESTDIR="${ED}" \ + install + + if [[ ${CHOST} == *-darwin* ]] ; then + # add SDK path which contains development manpages + echo "MANPATH=${EPREFIX}/MacOSX.sdk/usr/share/man" \ + > "${ED}"/etc/env.d/98macos-sdk + fi + + # need the makefile in pkg_preinst + insinto /usr/share/${PN} + doins Makefile + + dodoc ChangeLog + + # bug 858596 + if use prefix-guest ; then + dodir sbin + cat > "${ED}"/sbin/runscript <<- EOF + #!/usr/bin/env sh + source "${EPREFIX}/lib/gentoo/functions.sh" + + eerror "runscript/openrc-run not supported by Gentoo Prefix Base System release ${PV}" 1>&2 + exit 1 + EOF + chmod 755 "${ED}"/sbin/runscript || die + cp "${ED}"/sbin/{runscript,openrc-run} || die + fi +} + +pkg_preinst() { + # We need to install directories and maybe some dev nodes when building + # stages, but they cannot be in CONTENTS. + # Also, we cannot reference $S as binpkg will break so we do this. + multilib_layout + if use build ; then + if use split-usr ; then + emake -C "${ED}/usr/share/${PN}" DESTDIR="${EROOT}" layout + else + emake -C "${ED}/usr/share/${PN}" DESTDIR="${EROOT}" layout-usrmerge + fi + fi + rm -f "${ED}"/usr/share/${PN}/Makefile || die + + # Create symlinks in pkg_preinst to avoid Portage collision check. + # Create the symlinks in ${ED} via dosym so that we own it. + # Only create the symlinks if it wont cause a conflict in ${EROOT}. + if [[ -L ${EROOT}/var/lock || ! -e ${EROOT}/var/lock ]]; then + dosym ../run/lock /var/lock + fi + if [[ -L ${EROOT}/var/run || ! -e ${EROOT}/var/run ]]; then + dosym ../run /var/run + fi +} + +pkg_postinst() { + local x + + # We installed some files to /usr/share/baselayout instead of /etc to stop + # (1) overwriting the user's settings + # (2) screwing things up when attempting to merge files + # (3) accidentally packaging up personal files with quickpkg + # If they don't exist then we install them + for x in master.passwd passwd shadow group fstab ; do + [ -e "${EROOT}/etc/${x}" ] && continue + [ -e "${EROOT}/usr/share/baselayout/${x}" ] || continue + cp -p "${EROOT}/usr/share/baselayout/${x}" "${EROOT}"/etc || die + done + + # Force shadow permissions to not be world-readable #260993 + for x in shadow ; do + if [ -e "${EROOT}/etc/${x}" ] ; then + chmod o-rwx "${EROOT}/etc/${x}" || die + fi + done + # whine about users that lack passwords #193541 + if [[ -e "${EROOT}"/etc/shadow ]] ; then + local bad_users=$(sed -n '/^[^:]*::/s|^\([^:]*\)::.*|\1|p' "${EROOT}"/etc/shadow) + if [[ -n ${bad_users} ]] ; then + echo + ewarn "The following users lack passwords!" + ewarn ${bad_users} + fi + fi + + # whine about users with invalid shells #215698 + if [[ -e "${EROOT}"/etc/passwd ]] ; then + local bad_shells=$(awk -F: 'system("test -e ${ROOT}" $7) { print $1 " - " $7}' "${EROOT}"/etc/passwd | sort) + if [[ -n ${bad_shells} ]] ; then + echo + ewarn "The following users have non-existent shells!" + ewarn "${bad_shells}" + fi + fi + + # https://bugs.gentoo.org/361349 + if use kernel_linux; then + mkdir -p "${EROOT}"/run || die + + local found fstype mountpoint + while read -r _ mountpoint fstype _; do + [[ ${mountpoint} = /run ]] && [[ ${fstype} = tmpfs ]] && found=1 + done < "${ROOT}"/proc/mounts + [[ -z ${found} ]] && + ewarn "You should reboot now to get /run mounted with tmpfs!" + fi + + if [[ -e "${EROOT}"/etc/env.d/00basic ]]; then + ewarn "${EROOT}/etc/env.d/00basic is now ${EROOT}/etc/env.d/50baselayout" + ewarn "Please migrate your changes." + fi +} diff --git a/sdk_container/src/third_party/prefix-overlay/sys-apps/baselayout/baselayout-9999.ebuild b/sdk_container/src/third_party/prefix-overlay/sys-apps/baselayout/baselayout-9999.ebuild new file mode 100644 index 0000000000..9e5d229d2e --- /dev/null +++ b/sdk_container/src/third_party/prefix-overlay/sys-apps/baselayout/baselayout-9999.ebuild @@ -0,0 +1,350 @@ +# Copyright 1999-2023 Gentoo Authors +# Distributed under the terms of the GNU General Public License v2 + +EAPI=7 + +inherit multilib prefix + +DESCRIPTION="Filesystem baselayout and init scripts" +HOMEPAGE="https://wiki.gentoo.org/wiki/No_homepage" +if [[ ${PV} = 9999 ]]; then + EGIT_REPO_URI="https://anongit.gentoo.org/git/proj/${PN}.git" + inherit git-r3 +else + SRC_URI="https://gitweb.gentoo.org/proj/${PN}.git/snapshot/${P}.tar.bz2" + KEYWORDS="~alpha ~amd64 ~arm ~arm64 ~hppa ~ia64 ~loong ~m68k ~mips ~ppc ~ppc64 ~riscv ~s390 ~sparc ~x86 ~amd64-linux ~x86-linux ~arm64-macos ~ppc-macos ~x64-macos ~x64-solaris" +fi + +LICENSE="GPL-2" +SLOT="0" +IUSE="build +split-usr" + +RDEPEND="!sys-apps/baselayout-prefix" + +riscv_compat_symlink() { + # Here we apply some special sauce for riscv. + # Two multilib layouts exist for now: + # 1) one level libdirs, (32bit) "lib" and (64bit) "lib64" + # these are chosen by us to closely resemble other arches + # 2) two level libdirs, "lib64/lp64d" "lib64/lp64" "lib32/ilp32d" ... + # this is the glibc/gcc default + # Unfortunately, the default has only one fallback, which is "lib" + # for both 32bit and 64bit. So things do not break in 1), we need + # to provide compatibility symlinks... + + # This function has exactly two parameters: + # - the default libdir, to determine if 1) or 2) applies + # - the location of the symlink (which points to ".") + + # Note: we call this only in the ${SYMLINK_LIB} = no codepath, since + # there never was a ${SYMLINK_LIB} = yes riscv profile. + + case ${CHOST} in + riscv*) + # are we on a one level libdir profile? is there no symlink yet? + if [[ ${1} != */* && ! -L ${2} ]] ; then + ln -s . $2 || die "Unable to make $2 riscv compatibility symlink" + fi + ;; + esac +} + +# Create our multilib dirs - the Makefile has no knowledge of this +multilib_layout() { + local dir def_libdir libdir libdirs + local prefix prefix_lst + def_libdir=$(get_abi_LIBDIR $DEFAULT_ABI) + libdirs=$(get_all_libdirs) + + if [[ -z "${SYMLINK_LIB}" || ${SYMLINK_LIB} = no ]] ; then + prefix_lst=( "${EROOT}"/{,usr/,usr/local/} ) + for prefix in "${prefix_lst[@]}"; do + for libdir in ${libdirs}; do + dir="${prefix}${libdir}" + if [[ -e "${dir}" ]]; then + [[ ! -d "${dir}" ]] && + die "${dir} exists but is not a directory" + continue + fi + if ! use split-usr && [[ ${prefix} = ${EROOT}/ ]]; then + libdir="${libdir%%/*}" + dir="${prefix}${libdir}" + einfo "symlinking ${dir} to usr/${libdir}" + ln -s usr/${libdir} ${dir} || + die "Unable to make ${dir} symlink" + else + einfo "creating directory ${dir}" + mkdir -p "${dir}" || + die "Unable to create ${dir} directory" + fi + done + [[ -d "${prefix}${def_libdir}" ]] && riscv_compat_symlink "${def_libdir}" "${prefix}${def_libdir}/${DEFAULT_ABI}" + done + return 0 + fi + + [ -z "${def_libdir}" ] && + die "your DEFAULT_ABI=$DEFAULT_ABI appears to be invalid" + + # figure out which paths should be symlinks and which should be directories + local dirs syms exp d + for libdir in ${libdirs} ; do + if use split-usr ; then + exp=( {,usr/,usr/local/}${libdir} ) + else + exp=( {usr/,usr/local/}${libdir} ) + fi + for d in "${exp[@]}" ; do + # most things should be dirs + if [ "${SYMLINK_LIB}" = "yes" ] && [ "${libdir}" = "lib" ] ; then + [ ! -h "${d}" ] && [ -e "${d}" ] && dirs+=" ${d}" + else + [ -h "${d}" ] && syms+=" ${d}" + fi + done + done + if [ -n "${syms}${dirs}" ] ; then + ewarn "Your system profile has SYMLINK_LIB=${SYMLINK_LIB:-no}, so that means you need to" + ewarn "have these paths configured as follows:" + [ -n "${dirs}" ] && ewarn "symlinks to '${def_libdir}':${dirs}" + [ -n "${syms}" ] && ewarn "directories:${syms}" + ewarn "The ebuild will attempt to fix these, but only for trivial conversions." + ewarn "If things fail, you will need to manually create/move the directories." + echo + fi + + # setup symlinks and dirs where we expect them to be; do not migrate + # data ... just fall over in that case. + if use split-usr ; then + prefix_lst=( "${EROOT}"/{,usr/,usr/local/} ) + else + prefix_lst=( "${EROOT}"/{usr/,usr/local/} ) + fi + for prefix in "${prefix_lst[@]}"; do + if [ "${SYMLINK_LIB}" = yes ] ; then + # we need to make sure "lib" points to the native libdir + if [ -h "${prefix}lib" ] ; then + # it's already a symlink! assume it's pointing to right place ... + continue + elif [ -d "${prefix}lib" ] ; then + # "lib" is a dir, so need to convert to a symlink + ewarn "Converting ${prefix}lib from a dir to a symlink" + rm -f "${prefix}lib"/.keep || die + if rmdir "${prefix}lib" 2>/dev/null ; then + ln -s ${def_libdir} "${prefix}lib" || die + else + die "non-empty dir found where we needed a symlink: ${prefix}lib" + fi + else + # nothing exists, so just set it up sanely + ewarn "Initializing ${prefix}lib as a symlink" + mkdir -p "${prefix}" || die + rm -f "${prefix}lib" || die + ln -s ${def_libdir} "${prefix}lib" || die + mkdir -p "${prefix}${def_libdir}" || die #423571 + fi + else + # we need to make sure "lib" is a dir + if [ -h "${prefix}lib" ] ; then + # "lib" is a symlink, so need to convert to a dir + ewarn "Converting ${prefix}lib from a symlink to a dir" + rm -f "${prefix}lib" || die + if [ -d "${prefix}lib32" ] ; then + ewarn "Migrating ${prefix}lib32 to ${prefix}lib" + mv "${prefix}lib32" "${prefix}lib" || die + else + mkdir -p "${prefix}lib" || die + fi + elif [ -d "${prefix}lib" ] && ! has lib32 ${libdirs} ; then + # make sure the old "lib" ABI location does not exist; we + # only symlinked the lib dir on systems where we moved it + # to "lib32" ... + case ${CHOST} in + i?86*|x86_64*|powerpc*|sparc*|s390*) + if [[ -d ${prefix}lib32 && ! -h ${prefix}lib32 ]] ; then + rm -f "${prefix}lib32"/.keep || die + if ! rmdir "${prefix}lib32" 2>/dev/null ; then + ewarn "You need to merge ${prefix}lib32 into ${prefix}lib" + die "non-empty dir found where there should be none: ${prefix}lib32" + fi + fi + ;; + esac + else + # nothing exists, so just set it up sanely + ewarn "Initializing ${prefix}lib as a dir" + mkdir -p "${prefix}lib" || die + fi + fi + done + if ! use split-usr ; then + for libdir in ${libdirs}; do + if [[ ! -e "${EROOT}${libdir}" ]]; then + ln -s usr/"${libdir}" "${EROOT}${libdir}" || + die "Unable to make ${EROOT}${libdir} symlink" + fi + done + fi +} + +pkg_setup() { + multilib_layout +} + +src_prepare() { + default + + # don't want symlinked directories in PATH on systems with usr-merge + if ! use split-usr && ! use prefix-guest; then + sed \ + -e 's|:/usr/sbin:|:|g' \ + -e 's|:/sbin:|:|g' \ + -e 's|:/bin:|:|g' \ + -i etc/env.d/50baselayout || die + fi + + if use prefix; then + hprefixify -e "/EUID/s,0,${EUID}," -q '"' etc/profile + hprefixify etc/shells share/passwd + hprefixify -w '/PATH=/' etc/env.d/50baselayout + hprefixify -w 1 etc/env.d/50baselayout + echo PATH=/usr/sbin:/sbin:/usr/bin:/bin >> etc/env.d/99host + + # change branding + sed -i \ + -e '/gentoo-release/s/Gentoo Base/Gentoo Prefix Base/' \ + -e '/make_os_release/s/${OS}/Prefix/' \ + Makefile || die + fi + + # handle multilib paths. do it here because we want this behavior + # regardless of the C library that you're using. we do explicitly + # list paths which the native ldconfig searches, but this isn't + # problematic as it doesn't change the resulting ld.so.cache or + # take longer to generate. similarly, listing both the native + # path and the symlinked path doesn't change the resulting cache. + local libdir ldpaths + for libdir in $(get_all_libdirs) ; do + if use split-usr || use prefix-guest; then + ldpaths+=":${EPREFIX}/${libdir}" + fi + ldpaths+=":${EPREFIX}/usr/${libdir}" + ldpaths+=":${EPREFIX}/usr/local/${libdir}" + done + echo "LDPATH='${ldpaths#:}'" >> etc/env.d/50baselayout +} + +src_install() { + emake \ + DESTDIR="${ED}" \ + install + + if [[ ${CHOST} == *-darwin* ]] ; then + # add SDK path which contains development manpages + echo "MANPATH=${EPREFIX}/MacOSX.sdk/usr/share/man" \ + > "${ED}"/etc/env.d/98macos-sdk + fi + + # need the makefile in pkg_preinst + insinto /usr/share/${PN} + doins Makefile + + dodoc ChangeLog + + # bug 858596 + if use prefix-guest ; then + dodir sbin + cat > "${ED}"/sbin/runscript <<- EOF + #!/usr/bin/env sh + source "${EPREFIX}/lib/gentoo/functions.sh" + + eerror "runscript/openrc-run not supported by Gentoo Prefix Base System release ${PV}" 1>&2 + exit 1 + EOF + chmod 755 "${ED}"/sbin/runscript || die + cp "${ED}"/sbin/{runscript,openrc-run} || die + fi +} + +pkg_preinst() { + # We need to install directories and maybe some dev nodes when building + # stages, but they cannot be in CONTENTS. + # Also, we cannot reference $S as binpkg will break so we do this. + multilib_layout + if use build ; then + if use split-usr ; then + emake -C "${ED}/usr/share/${PN}" DESTDIR="${EROOT}" layout + else + emake -C "${ED}/usr/share/${PN}" DESTDIR="${EROOT}" layout-usrmerge + fi + fi + rm -f "${ED}"/usr/share/${PN}/Makefile || die + + # Create symlinks in pkg_preinst to avoid Portage collision check. + # Create the symlinks in ${ED} via dosym so that we own it. + # Only create the symlinks if it wont cause a conflict in ${EROOT}. + if [[ -L ${EROOT}/var/lock || ! -e ${EROOT}/var/lock ]]; then + dosym ../run/lock /var/lock + fi + if [[ -L ${EROOT}/var/run || ! -e ${EROOT}/var/run ]]; then + dosym ../run /var/run + fi +} + +pkg_postinst() { + local x + + # We installed some files to /usr/share/baselayout instead of /etc to stop + # (1) overwriting the user's settings + # (2) screwing things up when attempting to merge files + # (3) accidentally packaging up personal files with quickpkg + # If they don't exist then we install them + for x in master.passwd passwd shadow group fstab ; do + [ -e "${EROOT}/etc/${x}" ] && continue + [ -e "${EROOT}/usr/share/baselayout/${x}" ] || continue + cp -p "${EROOT}/usr/share/baselayout/${x}" "${EROOT}"/etc || die + done + + # Force shadow permissions to not be world-readable #260993 + for x in shadow ; do + if [ -e "${EROOT}/etc/${x}" ] ; then + chmod o-rwx "${EROOT}/etc/${x}" || die + fi + done + # whine about users that lack passwords #193541 + if [[ -e "${EROOT}"/etc/shadow ]] ; then + local bad_users=$(sed -n '/^[^:]*::/s|^\([^:]*\)::.*|\1|p' "${EROOT}"/etc/shadow) + if [[ -n ${bad_users} ]] ; then + echo + ewarn "The following users lack passwords!" + ewarn ${bad_users} + fi + fi + + # whine about users with invalid shells #215698 + if [[ -e "${EROOT}"/etc/passwd ]] ; then + local bad_shells=$(awk -F: 'system("test -e ${ROOT}" $7) { print $1 " - " $7}' "${EROOT}"/etc/passwd | sort) + if [[ -n ${bad_shells} ]] ; then + echo + ewarn "The following users have non-existent shells!" + ewarn "${bad_shells}" + fi + fi + + # https://bugs.gentoo.org/361349 + if use kernel_linux; then + mkdir -p "${EROOT}"/run || die + + local found fstype mountpoint + while read -r _ mountpoint fstype _; do + [[ ${mountpoint} = /run ]] && [[ ${fstype} = tmpfs ]] && found=1 + done < "${ROOT}"/proc/mounts + [[ -z ${found} ]] && + ewarn "You should reboot now to get /run mounted with tmpfs!" + fi + + if [[ -e "${EROOT}"/etc/env.d/00basic ]]; then + ewarn "${EROOT}/etc/env.d/00basic is now ${EROOT}/etc/env.d/50baselayout" + ewarn "Please migrate your changes." + fi +} diff --git a/sdk_container/src/third_party/prefix-overlay/sys-apps/baselayout/metadata.xml b/sdk_container/src/third_party/prefix-overlay/sys-apps/baselayout/metadata.xml new file mode 100644 index 0000000000..6a97f0c772 --- /dev/null +++ b/sdk_container/src/third_party/prefix-overlay/sys-apps/baselayout/metadata.xml @@ -0,0 +1,17 @@ + + + + + williamh@gentoo.org + William Hubbs + + + base-system@gentoo.org + Gentoo Base System + + + + proj/baselayout + gentoo/baselayout + + diff --git a/sdk_container/src/third_party/prefix-overlay/sys-apps/systemd-utils/Manifest b/sdk_container/src/third_party/prefix-overlay/sys-apps/systemd-utils/Manifest new file mode 100644 index 0000000000..f2f3cf4247 --- /dev/null +++ b/sdk_container/src/third_party/prefix-overlay/sys-apps/systemd-utils/Manifest @@ -0,0 +1,2 @@ +DIST systemd-musl-patches-254.3.tar.gz 28640 BLAKE2B 54837f49cdb8cf025e367ad13bab0d0509c2e11ad84d29724bb6baa226c54e0ab97a91035361f66009dd9b1a22f7b3e82f90b1c14adf4aa20d576b9410589d38 SHA512 07d028a57025b2626471d6f48507f2dfc50658db24efaac93bafae9a1d4cdc3ec82e80da426d2a6280c32af2d813565609dab7df5538260ba809b63309a0ffed +DIST systemd-stable-254.3.tar.gz 14329148 BLAKE2B 10b947e04a4ef9ccaeb7adaa67ac0f391927fb172c0750ffb93d4df69d970fd91f26b052f8bfdfb4f81ae69566d0a3459cbc87cc86b624014cfb8781a2914121 SHA512 a0c361c993ac9a121823bdd58e29ef7bd25ccfd206ae0c3e1eed9833b3ddf24f53afe6f669eb9fbff5078977403236b0e4ef5a5f6fde56c504caed1d411e71fe diff --git a/sdk_container/src/third_party/prefix-overlay/sys-apps/systemd-utils/files/40-gentoo.rules b/sdk_container/src/third_party/prefix-overlay/sys-apps/systemd-utils/files/40-gentoo.rules new file mode 100644 index 0000000000..6b96bd0705 --- /dev/null +++ b/sdk_container/src/third_party/prefix-overlay/sys-apps/systemd-utils/files/40-gentoo.rules @@ -0,0 +1,3 @@ +# Gentoo specific groups +ACTION=="add", SUBSYSTEM=="block", KERNEL=="fd[0-9]", GROUP="floppy" +ACTION=="add", SUBSYSTEM=="usb", ENV{DEVTYPE}=="usb_device", GROUP="usb" diff --git a/sdk_container/src/third_party/prefix-overlay/sys-apps/systemd-utils/files/legacy.conf b/sdk_container/src/third_party/prefix-overlay/sys-apps/systemd-utils/files/legacy.conf new file mode 100644 index 0000000000..2d322e8869 --- /dev/null +++ b/sdk_container/src/third_party/prefix-overlay/sys-apps/systemd-utils/files/legacy.conf @@ -0,0 +1,3 @@ +# Based on legacy.conf from systemd +d /run/lock +L /var/lock - - - - ../run/lock diff --git a/sdk_container/src/third_party/prefix-overlay/sys-apps/systemd-utils/files/systemd-tmpfiles-clean b/sdk_container/src/third_party/prefix-overlay/sys-apps/systemd-utils/files/systemd-tmpfiles-clean new file mode 100644 index 0000000000..e474f0e616 --- /dev/null +++ b/sdk_container/src/third_party/prefix-overlay/sys-apps/systemd-utils/files/systemd-tmpfiles-clean @@ -0,0 +1,2 @@ +#!/bin/sh +exec ionice -c idle -t systemd-tmpfiles --clean diff --git a/sdk_container/src/third_party/prefix-overlay/sys-apps/systemd-utils/files/systemd-tmpfiles-setup b/sdk_container/src/third_party/prefix-overlay/sys-apps/systemd-utils/files/systemd-tmpfiles-setup new file mode 100644 index 0000000000..a36c1020e0 --- /dev/null +++ b/sdk_container/src/third_party/prefix-overlay/sys-apps/systemd-utils/files/systemd-tmpfiles-setup @@ -0,0 +1,18 @@ +#!/sbin/openrc-run +# Copyright 2022 Gentoo Authors +# Released under the 2-clause BSD license. + +description="Create Volatile Files and Directories" + +depend() +{ + provide tmpfiles-setup tmpfiles.setup + need localmount +} + +start() +{ + ebegin "${description}" + systemd-tmpfiles --create --remove --boot --exclude-prefix=/dev + eend $? +} diff --git a/sdk_container/src/third_party/prefix-overlay/sys-apps/systemd-utils/files/systemd-tmpfiles-setup-dev b/sdk_container/src/third_party/prefix-overlay/sys-apps/systemd-utils/files/systemd-tmpfiles-setup-dev new file mode 100644 index 0000000000..496de522d0 --- /dev/null +++ b/sdk_container/src/third_party/prefix-overlay/sys-apps/systemd-utils/files/systemd-tmpfiles-setup-dev @@ -0,0 +1,20 @@ +#!/sbin/openrc-run +# Copyright 2022 Gentoo Authors +# Released under the 2-clause BSD license. + +description="Create Static Devices Nodes in /dev" + +depend() +{ + provide tmpfiles-dev tmpfiles.dev + use dev-mount + before dev + keyword -prefix -vserver +} + +start() +{ + ebegin "${description}" + systemd-tmpfiles --prefix=/dev --create --boot + eend $? +} diff --git a/sdk_container/src/third_party/prefix-overlay/sys-apps/systemd-utils/files/systemd-utils-254.3-add-link-kernel-install-shared-option.patch b/sdk_container/src/third_party/prefix-overlay/sys-apps/systemd-utils/files/systemd-utils-254.3-add-link-kernel-install-shared-option.patch new file mode 100644 index 0000000000..17e4ea4832 --- /dev/null +++ b/sdk_container/src/third_party/prefix-overlay/sys-apps/systemd-utils/files/systemd-utils-254.3-add-link-kernel-install-shared-option.patch @@ -0,0 +1,58 @@ +From 5973e4b237e7b50dca1c9f3157db459ef1ee6da5 Mon Sep 17 00:00:00 2001 +From: Violet Purcell +Date: Sat, 9 Sep 2023 13:22:54 -0400 +Subject: [PATCH] meson: add link-kernel-install-shared option + +Signed-off-by: Violet Purcell +--- + meson.build | 9 ++++++++- + meson_options.txt | 2 ++ + 2 files changed, 10 insertions(+), 1 deletion(-) + +diff --git a/meson.build b/meson.build +index 053e772567..003a34574a 100644 +--- a/meson.build ++++ b/meson.build +@@ -4420,11 +4420,17 @@ executable( + install : true, + install_dir : rootlibexecdir) + ++if get_option('link-kernel-install-shared') ++ kernel_install_link_with = [libshared] ++else ++ kernel_install_link_with = [libsystemd_static, libshared_static] ++endif ++ + kernel_install = executable( + 'kernel-install', + 'src/kernel-install/kernel-install.c', + include_directories : includes, +- link_with : [libshared], ++ link_with : kernel_install_link_with, + dependencies : [userspace, + versiondep], + install_rpath : rootpkglibdir, +@@ -5059,6 +5065,7 @@ foreach tuple : [ + ['link-timesyncd-shared', get_option('link-timesyncd-shared')], + ['link-journalctl-shared', get_option('link-journalctl-shared')], + ['link-boot-shared', get_option('link-boot-shared')], ++ ['link-kernel-install-shared', get_option('link-kernel-install-shared')], + ['link-portabled-shared', get_option('link-portabled-shared')], + ['first-boot-full-preset'], + ['fexecve'], +diff --git a/meson_options.txt b/meson_options.txt +index 1909323850..36794e6d98 100644 +--- a/meson_options.txt ++++ b/meson_options.txt +@@ -29,6 +29,8 @@ option('link-journalctl-shared', type: 'boolean', + description : 'link journalctl against libsystemd-shared.so') + option('link-boot-shared', type: 'boolean', + description : 'link bootctl and systemd-bless-boot against libsystemd-shared.so') ++option('link-kernel-install-shared', type: 'boolean', ++ description : 'link kernel-install against libsystemd-shared.so') + option('link-portabled-shared', type: 'boolean', + description : 'link systemd-portabled and its helpers to libsystemd-shared.so') + option('first-boot-full-preset', type: 'boolean', value: false, +-- +2.42.0 + diff --git a/sdk_container/src/third_party/prefix-overlay/sys-apps/systemd-utils/metadata.xml b/sdk_container/src/third_party/prefix-overlay/sys-apps/systemd-utils/metadata.xml new file mode 100644 index 0000000000..a5585a7ca0 --- /dev/null +++ b/sdk_container/src/third_party/prefix-overlay/sys-apps/systemd-utils/metadata.xml @@ -0,0 +1,18 @@ + + + + + systemd@gentoo.org + + + Enable systemd-boot (UEFI boot manager) + Enable kernel module loading via sys-apps/kmod + Enable systemd-sysusers + Enable systemd-tmpfiles + Enable systemd-udev (userspace device manager) + + + systemd/systemd + systemd/systemd-stable + + diff --git a/sdk_container/src/third_party/prefix-overlay/sys-apps/systemd-utils/systemd-utils-254.3.ebuild b/sdk_container/src/third_party/prefix-overlay/sys-apps/systemd-utils/systemd-utils-254.3.ebuild new file mode 100644 index 0000000000..849af65cc7 --- /dev/null +++ b/sdk_container/src/third_party/prefix-overlay/sys-apps/systemd-utils/systemd-utils-254.3.ebuild @@ -0,0 +1,537 @@ +# Copyright 2022-2023 Gentoo Authors +# Distributed under the terms of the GNU General Public License v2 + +EAPI=8 +PYTHON_COMPAT=( python3_{10..11} ) + +QA_PKGCONFIG_VERSION=$(ver_cut 1) + +inherit bash-completion-r1 flag-o-matic linux-info meson-multilib python-single-r1 +inherit secureboot toolchain-funcs udev usr-ldscript + +DESCRIPTION="Utilities split out from systemd for OpenRC users" +HOMEPAGE="https://systemd.io/" + +if [[ ${PV} == *.* ]]; then + MY_P="systemd-stable-${PV}" + S="${WORKDIR}/${MY_P}" + SRC_URI="https://github.com/systemd/systemd-stable/archive/refs/tags/v${PV}.tar.gz -> ${MY_P}.tar.gz" +else + MY_P="systemd-${PV}" + S="${WORKDIR}/${MY_P}" + SRC_URI="https://github.com/systemd/systemd/archive/refs/tags/v${PV}.tar.gz -> ${MY_P}.tar.gz" +fi + +MUSL_PATCHSET="systemd-musl-patches-254.3" +SRC_URI+=" elibc_musl? ( https://dev.gentoo.org/~floppym/dist/${MUSL_PATCHSET}.tar.gz )" + +LICENSE="GPL-2 LGPL-2.1 MIT public-domain" +SLOT="0" +KEYWORDS="~alpha ~amd64 ~arm ~arm64 ~hppa ~ia64 ~loong ~m68k ~mips ~ppc ~ppc64 ~riscv ~s390 ~sparc ~x86" +IUSE="+acl boot +kmod selinux split-usr sysusers +tmpfiles test +udev" +REQUIRED_USE=" + || ( boot tmpfiles sysusers udev ) + ${PYTHON_REQUIRED_USE} +" +RESTRICT="!test? ( test )" + +COMMON_DEPEND=" + elibc_musl? ( >=sys-libs/musl-1.2.3 ) + selinux? ( sys-libs/libselinux:0= ) + tmpfiles? ( + acl? ( sys-apps/acl:0= ) + ) + udev? ( + >=sys-apps/util-linux-2.30:0=[${MULTILIB_USEDEP}] + sys-libs/libcap:0=[${MULTILIB_USEDEP}] + virtual/libcrypt:=[${MULTILIB_USEDEP}] + acl? ( sys-apps/acl:0= ) + kmod? ( >=sys-apps/kmod-15:0= ) + ) + !udev? ( + >=sys-apps/util-linux-2.30:0= + sys-libs/libcap:0= + virtual/libcrypt:= + ) +" +DEPEND="${COMMON_DEPEND} + >=sys-kernel/linux-headers-3.11 +" + +PEFILE_DEPEND='dev-python/pefile[${PYTHON_USEDEP}]' + +RDEPEND="${COMMON_DEPEND} + boot? ( + !=dev-python/pyelftools-0.30[\${PYTHON_USEDEP}] + test? ( ${PEFILE_DEPEND} ) + ) + ") +" + +TMPFILES_OPTIONAL=1 +UDEV_OPTIONAL=1 + +QA_EXECSTACK="usr/lib/systemd/boot/efi/*" +QA_FLAGS_IGNORED="usr/lib/systemd/boot/efi/.*" + +CONFIG_CHECK="~BLK_DEV_BSG ~DEVTMPFS ~!IDE ~INOTIFY_USER ~!SYSFS_DEPRECATED + ~!SYSFS_DEPRECATED_V2 ~SIGNALFD ~EPOLL ~FHANDLE ~NET ~UNIX" + +pkg_setup() { + if [[ ${MERGE_TYPE} != buildonly ]] && use udev; then + linux-info_pkg_setup + fi + use boot && secureboot_pkg_setup +} + +src_prepare() { + local PATCHES=( + "${FILESDIR}/${PN}-254.3-add-link-kernel-install-shared-option.patch" + ) + + if use elibc_musl; then + PATCHES+=( + "${WORKDIR}/${MUSL_PATCHSET}" + ) + fi + default + + # Remove install_rpath; we link statically + local rpath_pattern="install_rpath : rootpkglibdir," + grep -q -e "${rpath_pattern}" meson.build || die + sed -i -e "/${rpath_pattern}/d" meson.build || die +} + +src_configure() { + python_setup + meson-multilib_src_configure +} + +multilib_src_configure() { + local emesonargs=( + $(meson_use split-usr) + $(meson_use split-usr split-bin) + -Drootprefix="$(usex split-usr "${EPREFIX:-/}" "${EPREFIX}/usr")" + -Drootlibdir="${EPREFIX}/usr/$(get_libdir)" + -Dsysvinit-path= + $(meson_native_use_bool boot bootloader) + $(meson_native_use_bool selinux) + $(meson_native_use_bool sysusers) + $(meson_use test tests) + $(meson_native_use_bool tmpfiles) + $(meson_use udev hwdb) + + # Link staticly with libsystemd-shared + -Dlink-boot-shared=false + -Dlink-kernel-install-shared=false + -Dlink-udev-shared=false + + # systemd-tmpfiles has a separate "systemd-tmpfiles.standalone" target + -Dstandalone-binaries=true + + # Disable all optional features + -Dadm-group=false + -Danalyze=false + -Dapparmor=false + -Daudit=false + -Dbacklight=false + -Dbinfmt=false + -Dbpf-framework=false + -Dbzip2=false + -Dcoredump=false + -Ddbus=false + -Delfutils=false + -Denvironment-d=false + -Dfdisk=false + -Dgcrypt=false + -Dglib=false + -Dgshadow=false + -Dgnutls=false + -Dhibernate=false + -Dhostnamed=false + -Didn=false + -Dima=false + -Dinitrd=false + -Dfirstboot=false + -Dldconfig=false + -Dlibcryptsetup=false + -Dlibcurl=false + -Dlibfido2=false + -Dlibidn=false + -Dlibidn2=false + -Dlibiptc=false + -Dlocaled=false + -Dlogind=false + -Dlz4=false + -Dmachined=false + -Dmicrohttpd=false + -Dnetworkd=false + -Dnscd=false + -Dnss-myhostname=false + -Dnss-resolve=false + -Dnss-systemd=false + -Doomd=false + -Dopenssl=false + -Dp11kit=false + -Dpam=false + -Dpcre2=false + -Dpolkit=false + -Dportabled=false + -Dpstore=false + -Dpwquality=false + -Drandomseed=false + -Dresolve=false + -Drfkill=false + -Dseccomp=false + -Dsmack=false + -Dsysext=false + -Dtimedated=false + -Dtimesyncd=false + -Dtpm=false + -Dqrencode=false + -Dquotacheck=false + -Duserdb=false + -Dutmp=false + -Dvconsole=false + -Dwheel-group=false + -Dxdg-autostart=false + -Dxkbcommon=false + -Dxz=false + -Dzlib=false + -Dzstd=false + ) + + if use tmpfiles || use udev; then + emesonargs+=( $(meson_native_use_bool acl) ) + else + emesonargs+=( -Dacl=false ) + fi + + if use udev; then + emesonargs+=( $(meson_native_use_bool kmod) ) + else + emesonargs+=( -Dkmod=false ) + fi + + if use elibc_musl; then + # Avoid redefinition of struct ethhdr. + append-cppflags -D__UAPI_DEF_ETHHDR=0 + fi + + if multilib_is_native_abi || use udev; then + meson_src_configure + fi +} + +efi_arch() { + case "$(tc-arch)" in + amd64) echo x64 ;; + arm) echo arm ;; + arm64) echo aa64 ;; + x86) echo x86 ;; + esac +} + +multilib_src_compile() { + local targets=() + if multilib_is_native_abi; then + if use boot; then + targets+=( + bootctl + kernel-install + man/bootctl.1 + man/kernel-install.8 + 90-loaderentry.install + src/boot/efi/linux$(efi_arch).efi.stub + src/boot/efi/systemd-boot$(efi_arch).efi + ) + fi + if use sysusers; then + targets+=( + systemd-sysusers.standalone + man/sysusers.d.5 + man/systemd-sysusers.8 + ) + if use test; then + targets+=( + systemd-runtest.env + ) + fi + fi + if use tmpfiles; then + targets+=( + systemd-tmpfiles.standalone + man/tmpfiles.d.5 + man/systemd-tmpfiles.8 + tmpfiles.d/{etc,static-nodes-permissions,var}.conf + ) + if use test; then + targets+=( test-tmpfile-util ) + fi + fi + if use udev; then + targets+=( + udevadm + systemd-hwdb + src/udev/ata_id + src/udev/cdrom_id + src/udev/fido_id + src/udev/mtd_probe + src/udev/scsi_id + src/udev/udev.pc + src/udev/v4l_id + man/udev.conf.5 + man/systemd.link.5 + man/hwdb.7 + man/udev.7 + man/systemd-hwdb.8 + man/systemd-udevd.service.8 + man/udevadm.8 + hwdb.d/60-autosuspend-chromiumos.hwdb + rules.d/50-udev-default.rules + rules.d/60-persistent-storage.rules + rules.d/64-btrfs.rules + ) + if use test; then + targets+=( + test-fido-id-desc + test-udev-builtin + test-udev-event + test-udev-node + test-udev-util + udev-rule-runner + ) + fi + fi + fi + if use udev; then + targets+=( + udev:shared_library + src/libudev/libudev.pc + ) + if use test; then + targets+=( + test-libudev + test-libudev-sym + test-udev-device-thread + ) + fi + fi + if multilib_is_native_abi || use udev; then + meson_src_compile "${targets[@]}" + fi +} + +multilib_src_test() { + local tests=() + if multilib_is_native_abi; then + if use sysusers; then + tests+=( + test-sysusers.standalone + ) + fi + if use tmpfiles; then + tests+=( + test-systemd-tmpfiles.standalone + test-tmpfile-util + ) + fi + if use udev; then + tests+=( + rule-syntax-check + test-fido-id-desc + test-udev + test-udev-builtin + test-udev-event + test-udev-node + test-udev-util + ) + fi + fi + if use udev; then + tests+=( + test-libudev + test-libudev-sym + test-udev-device-thread + ) + fi + if [[ ${#tests[@]} -ne 0 ]]; then + meson_src_test "${tests[@]}" + fi +} + +src_install() { + local rootprefix="$(usex split-usr '' /usr)" + meson-multilib_src_install +} + +multilib_src_install() { + if multilib_is_native_abi; then + if use boot; then + into /usr + dobin bootctl kernel-install + doman man/{bootctl.1,kernel-install.8} + # 90-loaderentry.install is generated from 90-loaderentry.install.in + exeinto usr/lib/kernel/install.d + doexe src/kernel-install/*.install + insinto usr/lib/systemd/boot/efi + doins src/boot/efi/{linux$(efi_arch).{efi,elf}.stub,systemd-boot$(efi_arch).efi} + fi + if use sysusers; then + into "${rootprefix:-/}" + newbin systemd-sysusers{.standalone,} + doman man/{systemd-sysusers.8,sysusers.d.5} + fi + if use tmpfiles; then + into "${rootprefix:-/}" + newbin systemd-tmpfiles{.standalone,} + doman man/{systemd-tmpfiles.8,tmpfiles.d.5} + insinto /usr/lib/tmpfiles.d + doins tmpfiles.d/{etc,static-nodes-permissions,var}.conf + fi + if use udev; then + into "${rootprefix:-/}" + dobin udevadm systemd-hwdb + dosym ../../bin/udevadm "${rootprefix}"/lib/systemd/systemd-udevd + + exeinto "${rootprefix}"/lib/udev + doexe src/udev/{ata_id,cdrom_id,fido_id,mtd_probe,scsi_id,v4l_id} + + rm -f rules.d/99-systemd.rules + insinto "${rootprefix}"/lib/udev/rules.d + doins rules.d/*.rules + + insinto "${rootprefix}"/lib/udev/hwdb.d + doins hwdb.d/*.hwdb + + insinto /usr/share/pkgconfig + doins src/udev/udev.pc + + doman man/{udev.conf.5,systemd.link.5,hwdb.7,systemd-hwdb.8,udev.7,udevadm.8} + newman man/systemd-udevd.service.8 systemd-udevd.8 + fi + fi + if use udev; then + meson_install --no-rebuild --tags libudev + gen_usr_ldscript -a udev + insinto "/usr/$(get_libdir)/pkgconfig" + doins src/libudev/libudev.pc + fi +} + +multilib_src_install_all() { + einstalldocs + if use boot; then + into /usr + exeinto usr/lib/kernel/install.d + doexe src/kernel-install/*.install + dobashcomp shell-completion/bash/bootctl + insinto /usr/share/zsh/site-functions + doins shell-completion/zsh/{_bootctl,_kernel-install} + fi + if use tmpfiles; then + doinitd "${FILESDIR}"/systemd-tmpfiles-setup + doinitd "${FILESDIR}"/systemd-tmpfiles-setup-dev + exeinto /etc/cron.daily + doexe "${FILESDIR}"/systemd-tmpfiles-clean + insinto /usr/share/zsh/site-functions + doins shell-completion/zsh/_systemd-tmpfiles + insinto /usr/lib/tmpfiles.d + doins tmpfiles.d/{tmp,x11}.conf + doins "${FILESDIR}"/legacy.conf + fi + if use udev; then + doheader src/libudev/libudev.h + + insinto /etc/udev + doins src/udev/udev.conf + keepdir /etc/udev/{hwdb.d,rules.d} + + insinto "${rootprefix}"/lib/systemd/network + doins network/99-default.link + + # Remove to avoid conflict with elogind + # https://bugs.gentoo.org/856433 + rm rules.d/70-power-switch.rules || die + insinto "${rootprefix}"/lib/udev/rules.d + doins rules.d/*.rules + doins "${FILESDIR}"/40-gentoo.rules + + insinto "${rootprefix}"/lib/udev/hwdb.d + doins hwdb.d/*.hwdb + + dobashcomp shell-completion/bash/udevadm + + insinto /usr/share/zsh/site-functions + doins shell-completion/zsh/_udevadm + fi + + use boot && secureboot_auto_sign +} + +add_service() { + local initd=$1 + local runlevel=$2 + + ebegin "Adding '${initd}' service to the '${runlevel}' runlevel" + mkdir -p "${EROOT}/etc/runlevels/${runlevel}" && + ln -snf "${EPREFIX}/etc/init.d/${initd}" "${EROOT}/etc/runlevels/${runlevel}/${initd}" + eend $? +} + +pkg_postinst() { + if [[ -z ${REPLACING_VERSIONS} ]]; then + add_service systemd-tmpfiles-setup-dev sysinit + add_service systemd-tmpfiles-setup boot + fi + if use udev; then + ebegin "Updating hwdb" + systemd-hwdb --root="${ROOT}" update + eend $? + udev_reload + fi +} diff --git a/sdk_container/src/third_party/prefix-overlay/sys-libs/libxcrypt/Manifest b/sdk_container/src/third_party/prefix-overlay/sys-libs/libxcrypt/Manifest new file mode 100644 index 0000000000..416fefe5b9 --- /dev/null +++ b/sdk_container/src/third_party/prefix-overlay/sys-libs/libxcrypt/Manifest @@ -0,0 +1 @@ +DIST libxcrypt-4.4.36-autotools.tar.xz 624660 BLAKE2B 8dc3d0f354baf8c64dc011e95e7df10d48b0dfe428503936ffd55edf2745de04003c7efe231ed5d9a14cea7f682ba377b7e00f0463b4060c50c9c29f555b790f SHA512 fb8391ecb89622eb0d74d13c5fc1369718e83c47671449044ca0c2f78a236d7b06177a60bf8cda47694caa840c68eaaf0b23690e8975fa5d64b734c8eb246d10 diff --git a/sdk_container/src/third_party/prefix-overlay/sys-libs/libxcrypt/files/libxcrypt-4.4.19-multibuild.patch b/sdk_container/src/third_party/prefix-overlay/sys-libs/libxcrypt/files/libxcrypt-4.4.19-multibuild.patch new file mode 100644 index 0000000000..5b3958e091 --- /dev/null +++ b/sdk_container/src/third_party/prefix-overlay/sys-libs/libxcrypt/files/libxcrypt-4.4.19-multibuild.patch @@ -0,0 +1,14 @@ +diff --git a/Makefile.am b/Makefile.am +index d0cca1d..4a5d4a1 100644 +--- a/Makefile.am ++++ b/Makefile.am +@@ -86,9 +86,7 @@ noinst_HEADERS = \ + test/des-cases.h \ + test/ka-table.inc + +-if ENABLE_XCRYPT_COMPAT_FILES + nodist_include_HEADERS += xcrypt.h +-endif + + noinst_PROGRAMS = \ + lib/gen-des-tables diff --git a/sdk_container/src/third_party/prefix-overlay/sys-libs/libxcrypt/libxcrypt-4.4.36.ebuild b/sdk_container/src/third_party/prefix-overlay/sys-libs/libxcrypt/libxcrypt-4.4.36.ebuild new file mode 100644 index 0000000000..0943c85e2c --- /dev/null +++ b/sdk_container/src/third_party/prefix-overlay/sys-libs/libxcrypt/libxcrypt-4.4.36.ebuild @@ -0,0 +1,340 @@ +# Copyright 2004-2023 Gentoo Authors +# Distributed under the terms of the GNU General Public License v2 + +EAPI=8 + +PYTHON_COMPAT=( python3_{10..11} ) +# NEED_BOOTSTRAP is for developers to quickly generate a tarball +# for publishing to the tree. +NEED_BOOTSTRAP="no" +inherit multibuild multilib python-any-r1 flag-o-matic toolchain-funcs multilib-minimal + +DESCRIPTION="Extended crypt library for descrypt, md5crypt, bcrypt, and others" +HOMEPAGE="https://github.com/besser82/libxcrypt" +if [[ ${NEED_BOOTSTRAP} == "yes" ]] ; then + inherit autotools + SRC_URI="https://github.com/besser82/libxcrypt/releases/download/v${PV}/${P}.tar.xz" +else + SRC_URI="https://dev.gentoo.org/~sam/distfiles/${CATEGORY}/${PN}/${P}-autotools.tar.xz" +fi + +LICENSE="LGPL-2.1+ public-domain BSD BSD-2" +SLOT="0/1" +KEYWORDS="~alpha amd64 arm arm64 hppa ~ia64 ~loong ~m68k ~mips ~ppc ppc64 ~riscv ~s390 ~sparc x86" +IUSE="+compat split-usr static-libs +system test headers-only" +REQUIRED_USE="split-usr? ( system )" +RESTRICT="!test? ( test )" + +export CTARGET=${CTARGET:-${CHOST}} +if [[ ${CTARGET} == ${CHOST} ]] ; then + if [[ ${CATEGORY/cross-} != ${CATEGORY} ]] ; then + export CTARGET=${CATEGORY/cross-} + fi +fi + +is_cross() { + local enabled_abis=( $(multilib_get_enabled_abis) ) + [[ "${#enabled_abis[@]}" -le 1 ]] && [[ ${CHOST} != ${CTARGET} ]] +} + +DEPEND=" + system? ( + elibc_glibc? ( + ${CATEGORY}/glibc[-crypt(+)] + !${CATEGORY}/glibc[crypt(+)] + ) + elibc_musl? ( + ${CATEGORY}/musl[-crypt(+)] + !${CATEGORY}/musl[crypt(+)] + ) + ) +" +RDEPEND="${DEPEND}" +BDEPEND=" + dev-lang/perl + test? ( $(python_gen_any_dep 'dev-python/passlib[${PYTHON_USEDEP}]') ) +" + +python_check_deps() { + python_has_version "dev-python/passlib[${PYTHON_USEDEP}]" +} + +pkg_pretend() { + if has "distcc" ${FEATURES} ; then + ewarn "Please verify all distcc nodes are using the same versions of GCC (>= 10) and Binutils!" + ewarn "Older/mismatched versions of GCC may lead to a misbehaving library: bug #823179." + + if [[ ${BUILD_TYPE} != "binary" ]] && tc-is-gcc && [[ $(gcc-major-version) -lt 10 ]] ; then + die "libxcrypt is known to fail to build or be broken at runtime with < GCC 10 (bug #823179)!" + fi + fi +} + +pkg_setup() { + MULTIBUILD_VARIANTS=( + $(usev compat 'xcrypt_compat') + xcrypt_nocompat + ) + + use test && python-any-r1_pkg_setup +} + +src_prepare() { + default + + # WARNING: Please read on bumping or applying patches! + # + # There are two circular dependencies to be aware of: + # 1) + # if we're bootstrapping configure and makefiles: + # libxcrypt -> automake -> perl -> libxcrypt + # + # mitigation: + # toolchain@ manually runs `make dist` after running autoconf + `./configure` + # and the ebuild uses that. + # (Don't include the pre-generated Perl artefacts.) + # + # solution for future: + # Upstream are working on producing `make dist` tarballs. + # https://github.com/besser82/libxcrypt/issues/134#issuecomment-871833573 + # + # 2) + # configure *unconditionally* needs Perl at build time to generate + # a list of enabled algorithms based on the set passed to `configure`: + # libxcrypt -> perl -> libxcrypt + # + # mitigation: + # None at the moment. + # + # solution for future: + # Not possible right now. Upstream intend on depending on Perl for further + # configuration options. + # https://github.com/besser82/libxcrypt/issues/134#issuecomment-871833573 + # + # Therefore, on changes (inc. bumps): + # * You must check whether upstream have started providing tarballs with bootstrapped + # auto{conf,make}; + # + # * diff the build system changes! + # + if [[ ${NEED_BOOTSTRAP} == "yes" ]] ; then + # Facilitate our split variant build for compat + non-compat + eapply "${FILESDIR}"/${PN}-4.4.19-multibuild.patch + eautoreconf + fi +} + +src_configure() { + # Avoid possible "illegal instruction" errors with gold + # bug #821496 + tc-ld-disable-gold + + # Doesn't work with LTO: bug #852917. + # https://github.com/besser82/libxcrypt/issues/24 + filter-lto + + # ideally we want !tc-ld-is-bfd for best future-proofing, but it needs + # https://github.com/gentoo/gentoo/pull/28355 + # mold needs this too but right now tc-ld-is-mold is also not available + if tc-ld-is-lld; then + append-ldflags -Wl,--undefined-version + fi + + multibuild_foreach_variant multilib-minimal_src_configure +} + +get_xcprefix() { + if is_cross; then + echo "${EPREFIX}/usr/${CTARGET}" + else + echo "${EPREFIX}" + fi +} + +get_xclibdir() { + printf -- "%s/%s/%s/%s\n" \ + "$(get_xcprefix)" \ + "$(usev !split-usr '/usr')" \ + "$(get_libdir)" \ + "$(usev !system 'xcrypt')" +} + +get_xcincludedir() { + printf -- "%s/usr/include/%s\n" \ + "$(get_xcprefix)" \ + "$(usev !system 'xcrypt')" +} + +get_xcmandir() { + printf -- "%s/usr/share/man\n" \ + "$(get_xcprefix)" +} + +get_xcpkgconfigdir() { + printf -- "%s/usr/%s/pkgconfig\n" \ + "$(get_xcprefix)" \ + "$(get_libdir)" +} + +multilib_src_configure() { + local -a myconf=( + --host=${CTARGET} + --disable-werror + --libdir=$(get_xclibdir) + --with-pkgconfigdir=$(get_xcpkgconfigdir) + --includedir=$(get_xcincludedir) + --mandir="$(get_xcmandir)" + ) + + tc-export PKG_CONFIG + + if is_cross; then + if tc-is-clang; then + export CC="${CTARGET}-clang" + else + export CC="${CTARGET}-gcc" + fi + fi + + case "${MULTIBUILD_ID}" in + xcrypt_compat-*) + myconf+=( + --disable-static + --disable-xcrypt-compat-files + --enable-obsolete-api=yes + ) + ;; + xcrypt_nocompat-*) + myconf+=( + --enable-obsolete-api=no + $(use_enable static-libs static) + ) + ;; + *) die "Unexpected MULTIBUILD_ID: ${MULTIBUILD_ID}";; + esac + + if use headers-only; then + # Nothing is compiled here which would affect the headers for the target. + # So forcing CC is sane. + headers_only_flags="CC=$(tc-getBUILD_CC)" + fi + + ECONF_SOURCE="${S}" econf "${myconf[@]}" "${headers_only_flags}" +} + +src_compile() { + use headers-only && return + + multibuild_foreach_variant multilib-minimal_src_compile +} + +multilib_src_test() { + emake check +} + +src_test() { + multibuild_foreach_variant multilib-minimal_src_test +} + +src_install() { + multibuild_foreach_variant multilib-minimal_src_install + + use headers-only || \ + ( + shopt -s failglob || die "failglob failed" + + # Make sure our man pages do not collide with glibc or man-pages. + for manpage in "${D}$(get_xcmandir)"/man3/crypt{,_r}.?*; do + mv -n "${manpage}" "$(dirname "${manpage}")/xcrypt_$(basename "${manpage}")" \ + || die "mv failed" + done + ) || die "failglob error" + + # Remove useless stuff from installation + find "${ED}"/usr/share/doc/${PF} -type l -delete || die + find "${ED}" -name '*.la' -delete || die + + # workaround broken upstream cross-* --docdir by installing files in proper locations + if is_cross; then + insinto "$(get_xcprefix)"/usr/share + doins -r "${ED}"/usr/share/doc + rm -r "${ED}"/usr/share/doc || die + fi +} + +multilib_src_install() { + if use headers-only; then + emake DESTDIR="${D}" install-nodist_includeHEADERS + return + fi + + emake DESTDIR="${D}" install + + # Don't install the libcrypt.so symlink for the "compat" version + case "${MULTIBUILD_ID}" in + xcrypt_compat-*) + rm "${D}"$(get_xclibdir)/libcrypt$(get_libname) \ + || die "failed to remove extra compat libraries" + ;; + xcrypt_nocompat-*) + if use split-usr; then + ( + if use static-libs; then + # .a files are installed to /$(get_libdir) by default + # Move static libraries to /usr prefix or portage will abort + shopt -s nullglob || die "failglob failed" + static_libs=( "${D}"/$(get_xclibdir)/*.a ) + + if [[ -n ${static_libs[*]} ]]; then + dodir "/usr/$(get_xclibdir)" + mv "${static_libs[@]}" "${ED}/usr/$(get_xclibdir)" \ + || die "Moving static libs failed" + fi + fi + + if use system; then + # Move versionless .so symlinks from /$(get_libdir) to /usr/$(get_libdir) + # to allow linker to correctly find shared libraries. + shopt -s failglob || die "failglob failed" + + for lib_file in "${D}"$(get_xclibdir)/*$(get_libname); do + lib_file_basename="$(basename "${lib_file}")" + lib_file_target="$(basename "$(readlink -f "${lib_file}")")" + + # We already know we're in split-usr (checked above) + # See bug #843209 (also worth keeping in mind bug #802222 too) + local libdir_no_prefix=$(get_xclibdir) + libdir_no_prefix=${libdir_no_prefix#${EPREFIX}} + libdir_no_prefix=${libdir_no_prefix%/usr} + dosym -r "/$(get_libdir)/${lib_file_target}" "/usr/${libdir_no_prefix}/${lib_file_basename}" + done + + rm "${D}"$(get_xclibdir)/*$(get_libname) || die "Removing symlinks in incorrect location failed" + fi + ) + fi + ;; + *) die "Unexpected MULTIBUILD_ID: ${MULTIBUILD_ID}";; + esac +} + +pkg_preinst() { + # Verify we're not in a bad case like bug #843209 with broken symlinks. + # This can be dropped when, if ever, the split-usr && system && compat case + # is cleaned up in *_src_install. + local broken_symlinks=() + mapfile -d '' broken_symlinks < <( + find "${ED}" -xtype l -print0 + ) + + if [[ ${#broken_symlinks[@]} -gt 0 ]]; then + eerror "Broken symlinks found before merging!" + local symlink target resolved + for symlink in "${broken_symlinks[@]}" ; do + target="$(readlink "${symlink}")" + resolved="$(readlink -f "${symlink}")" + eerror " '${symlink}' -> '${target}' (${resolved})" + done + die "Broken symlinks found! Aborting to avoid damaging system. Please report a bug." + fi +} diff --git a/sdk_container/src/third_party/prefix-overlay/sys-libs/libxcrypt/metadata.xml b/sdk_container/src/third_party/prefix-overlay/sys-libs/libxcrypt/metadata.xml new file mode 100644 index 0000000000..cef5e501f6 --- /dev/null +++ b/sdk_container/src/third_party/prefix-overlay/sys-libs/libxcrypt/metadata.xml @@ -0,0 +1,21 @@ + + + + + toolchain@gentoo.org + Gentoo Toolchain Project + + + Crypt library for DES, MD5, and blowfish. Libxcrypt is a replacement for + libcrypt, which comes with the GNU C Library. It supports DES crypt, + MD5, and passwords with blowfish encryption. + + + Build with compatibility interfaces for other crypt implementations + Install as system libcrypt.so rather than to an alternate directory (will collide with sys-libs/glibc's version) + Build and install only the headers. + + + besser82/libxcrypt + + diff --git a/setup_prefix b/setup_prefix new file mode 100755 index 0000000000..dd0c8ade67 --- /dev/null +++ b/setup_prefix @@ -0,0 +1,143 @@ +#!/bin/bash + + +. "$(dirname "$0")/common.sh" || exit 1 +. "${BUILD_LIBRARY_DIR}/prefix_util.sh" || exit 1 + +assert_inside_chroot +assert_not_root_user + +staging_dir_opt_placeholder="${DEFAULT_STAGING_ROOT}prefix-/" +final_dir_opt_placeholder="${SCRIPTS_DIR}/__prefix__//" + +DEFINE_string board "${DEFAULT_BOARD}" \ + "Board (architecture) to build for in prefix." +DEFINE_string staging_dir "${staging_dir_opt_placeholder}" \ + "Staging (build) directory for this prefix." +DEFINE_string final_dir "${final_dir_opt_placeholder}" \ + "Local directory to install the final prefixed binaries and runtime dependencies to. + '/root' will contain the FS root to e.g. create a sysext from." +DEFINE_boolean force "${FLAGS_FALSE}" \ + "Force re-creating a prefix that already exists. THIS WILL REMOVE THE OLD PREFIX ENTIRELY." +DEFINE_boolean uninstall "${FLAGS_FALSE}" \ + "Uninstall an existing prefix, removing all directories and wrapper scripts associated with it. + If set, can be omitted." +DEFINE_string cross_boss_root "${SCRIPTS_DIR}/cross-boss" \ + "Custom cross-boss scripts root." + +# TODO: implement +#DEFINE_string custom_ebuild_overlays "" \ +# "Comma-separated list of additional ebuild overlays to add to the prefix." + +FLAGS_HELP="usage: setup_prefix [flags] + +setup_prefix creates a new prefix as well as an emerge wrapper. + + - Common name of the prefix. Will be used for naming portage wrappers. + + - Absolute library / executables path to use for this prefix, e.g. '/usr/local/mystuff'. + Binaries and libraries will live below ; binaries will be + linked against /lib etc. Should start with /usr or /opt if you + want to use the installation directory to create a sysext. + +Please refer to PREFIX.md for general information on prefixes. +" + +show_help_if_requested "$@" +FLAGS "$@" || exit 1 +eval set -- "${FLAGS_ARGV}" + +switch_to_strict_mode -uo pipefail + +name="${1:-}" +prefix="${2:-}" + +if [ "${FLAGS_uninstall}" = "${FLAGS_TRUE}" ] ; then + # We don't really care about prefix when uninstalling. + # Make sure it is set so we don't need to set it on the uninstall command line. + prefix="ignored" +fi + +if [[ ! ${name} || ! ${prefix} ]] ; then + error "Missing mandatory positional parameter." + flags_help + exit 1 +fi + +if [ "${FLAGS_staging_dir}" = "${staging_dir_opt_placeholder}" ] ; then + FLAGS_staging_dir="${DEFAULT_STAGING_ROOT}prefix-${FLAGS_board}/${name}" +fi +if [ "${FLAGS_final_dir}" = "${final_dir_opt_placeholder}" ] ; then + FLAGS_final_dir="${SCRIPTS_DIR}/__prefix__/${FLAGS_board}/${name}" +fi + +# +# Helper functions +# + +function check_force_dirs() { + local what="${1}" + local dir="${2}" + + if [ -e "${dir}" ] ; then + if [ "${FLAGS_force}" = "${FLAGS_FALSE}" ] ; then + error "${what} directory '${dir}' already exists! Use --force to remove and to re-create prefix." + exit 1 + else + warn "Removing ${what} directory '${dir}' as requested ('--force' option)." + sudo rm -rf "${dir}" + fi + fi +} +# -- + +# +# Main +# + +set_prefix_vars "${name}" "${prefix}" +prefix_repo="$(dirname "$(EPREFIX="" portageq get_repo_path / portage-stable)")/prefix-overlay" + +if [ "${FLAGS_uninstall}" = "${FLAGS_TRUE}" ] ; then + warn "Removing prefix '${name}' and all associated direcroties and wrappers." + sudo rm -vrf "${STAGINGDIR}" "${FINALDIR}" + # TODO: cover all portage tools, not just emerge + sudo rm -vf "$(emerge_name with-path)" + exit +fi + +if [ ! -e "${CB_ROOT}/bin/cb-bootstrap" ] ; then + error "Cross-boss not found at '${CB_ROOT}'" + error "Please make sure cross-boss is available (i.e. git clone https://github.com/chewi/cross-boss)." + error "See PREFIX.md for more information." + exit 1 +fi + +info "Installing SDK prerequisites and creating prefix directories" + +check_force_dirs "staging" "${STAGINGDIR}" +check_force_dirs "installation" "${FINALDIR}" + +setup_prefix_dirs "${prefix_repo}" 2>&1 | lineprepend "Prefix directories" + +create_make_conf "staging" +create_make_conf "final" + +install_prereqs "${prefix_repo}" 2>&1 | lineprepend "SDK prereqs" + +# -- + +info "Bootstrapping staging environment in '${STAGINGROOT}'". +sudo env EPREFIX="${EPREFIX}" "${CB_ROOT}"/bin/cb-bootstrap "${STAGINGROOT}" 2>&1 | lineprepend "cb-bootstrap" + +info "Extracting GCC libraries to installation root / final." +extract_gcc_libs 2>&1 | lineprepend "GCC libs for final" + +# TODO: cover all portage tools, not just emerge +info "Creating wrappers" +create_emerge_wrapper + +info "Emerging installation root foundational dependencies." +$(emerge_name) prefix/prefix-final | lineprepend "final init" + +info "Done. Use '$(emerge_name)' to emerge packages into the prefix."