hashed-OCI-revisioned-debs: introduce "reversioning" of .deb packages

> tl-dr:
> - maximize OCI cache hit ratio across nightlies/releases/PRs/etc;
> - publish simple `Version:`'s that don't include a crazy hash in repo and images
> - introduce `output/packages-hashed` directory
> - radically change the `output/debs` directory structure

- simplify artifact's `prepare_version()` method for `deb` and `deb-tar` artifacts:
  - `artifact_base_dir` and `artifact_final_file` will now be auto-calculated; thus removed from each artifact (except `rootfs`)
  - `artifact_deb_repo` ("global", "jammy", "bookworm") is now required; "global" means common across all RELEASES
  - `artifact_deb_arch` is now required, "all" is arch-independent, otherwise use `${ARCH}`
  - `artifact_map_debs` is now auto-calculated based on the above, and shouldn't be specified manually
  - `artifact_final_version_reversioned` is optional, and can force the final version of the artifact (specific for the `base-files` case)
  - artifacts that need special handling for reversioning can add function names to `artifact_debs_reversion_functions` array (`base-files` and `bsp-cli` cases)
  - artifacts `prepare_version()` should set `artifact_version`, but _never_ include it in other variables; `artifact_version` is now changed by framework after `prepare_version()` returns
- no longer use/refer/mention `${REVISION}` when building packages. All packages should be `${REVISION}`-agnostic.
- `${REVISION}` (actually, `artifact_final_version_reversioned`) will be automatically swapped in the `control` file during reversioning
- `fakeroot_dpkg_deb_build()` now takes exactly two arguments: the directory to pack, and the deb ID (key of `artifact_map_packages` dict); add this change in all the artifact's code for this
- `obtain_complete_artifact()`:
  - automatically adds `-Rxxxx` "revisioning-hash" to `artifact_version`, by hashing the revisioning functions and any `artifact_debs_reversion_functions` set
  - calculates more complex subdirectory paths for both the `output/packages-hashed` and `output/debs`/`output/debs-beta` directories
    - with the new subdirectories we can be sure a re-version is already done correctly and can skip it (eg, for partial `download-debs` re-runs)
    - in the future we can automatically clean/remove old versions that are no longer relevant based on the dir structure
    - exports a lot more information to JSON, including the new subdirectory paths
  - comment-out code that implemented `skip_unpack_if_found_in_caches`, I'm very unsure why we had this in the first place
- `obtain_artifact_from_remote_cache()`
  - for `deb` type artifacts, OCI won't preserve the subdirectory structure, so move downloaded files to the correct subdirectory manually
  - this is not needed for `deb-tar`, since that can preserve the dir structure itself
- introduce `artifacts-reversion.sh` and its main function `artifact_reversion_for_deployment()`
  - this has the logic for reversioning .deb's, by `ar`-unpacking them, changing `control.tar` (and possibly `data.tar`), handling `.xz` compression, etc.
  - also handles hashing those functions, for consistency. Any changes in reversioning code actually change the artifact itself so we're not caught by surprise
  - by default, it changes `control` file only:
    - replace `Version:` (which is the hash-version originally) with `artifact_final_version_reversioned` (which is mostly just `${REVISION}`)
    - add a custom field `Armbian-Original-Hash:` with the original hash-version
  - `artifact_reversion_for_deployment()` is called by
    - new CLI wrapper `cli_obtain_complete_artifact()`, used for CLI building of specific artifact, but also for `download-artifact`
    - `build_artifact_for_image()` used during image build
- `armbian-bsp-cli-deb.sh`: move `${REVISION}` related stuff from the main package build to new reversioning functions.
- `artifact-armbian-base-files.sh`: move `${REVISION}` related stuff from the main package build to new reversioning functions.
- `kernel`:
  - add some custom fields to `DEBIAN/control`:
    - `Armbian-Kernel-Version:` / `Armbian-Kernel-Version-Family:` (for future use: cleanup of usage of `Source: ` field which should be removed)
  - declutter the `Description:` field, moving long description out of the first line
  - obtain `IMAGE_INSTALLED_KERNEL_VERSION` from the reversioned deb (this is still a hack and has not been fixed)
- `uboot`:
  - declutter the `Description:` field, moving long description out of the first line
  - use the reversioned .deb when deploying u-boot to the image
- `main_default_build_packages()` now stores reversioned values and complete paths to reversioned .deb's
- `list_installed_packages()` now compares custom field `Armbian-Original-Hash: `, and not the `Version:` to make sure debs in the image are the ones we want
- `install_artifact_deb_chroot()` is a new wrapper around `install_deb_chroot()` for easy handling of reversioned debs
  - use it everywhere `install_deb_chroot()` was used in `distro-agnostic.sh` and `distro-specific.sh`
This commit is contained in:
Ricardo Pardini 2023-08-02 16:45:16 +02:00 committed by Igor
parent a876d0f882
commit 2142f73b97
33 changed files with 528 additions and 240 deletions

View File

@ -8,8 +8,7 @@
# We find what is the latest version of the original distro's base-files.
# Then we download it, and we modify it to suit our needs.
# The artifact is named "armbian-base-files".
# But the package is still named "base-files"; its version is bumped to ${REVISION}
# This is similar to what Linux Mint does for the same purpose.
# But the package is still named "base-files", this is similar to what Linux Mint does for the same purpose.
function artifact_armbian-base-files_config_dump() {
artifact_input_variables[RELEASE]="${RELEASE}"
@ -51,14 +50,17 @@ function artifact_armbian-base-files_prepare_version() {
artifact_version_reason="${reasons[*]}" # outer scope
artifact_name="armbian-base-files-${RELEASE}-${ARCH}"
declare deb_name="base-files" # the artifact_name is only Armbian's reference; the deb_name is still base_files
artifact_type="deb"
artifact_base_dir="${PACKAGES_HASHED_STORAGE}/${RELEASE}"
artifact_final_file="${PACKAGES_HASHED_STORAGE}/${RELEASE}/${deb_name}_${artifact_version}_${ARCH}.deb"
artifact_deb_repo="${RELEASE}" # release-specific repo (jammy etc)
artifact_deb_arch="${ARCH}" # arch-specific packages (arm64 etc)
artifact_map_packages=(["armbian-base-files"]="base-files")
artifact_map_packages=(["armbian-base-files"]="${deb_name}")
# Important. Force the final reversioned version to contain the release name.
# Otherwise, when publishing to a repo, pool/main/b/base-files/base-files_${REVISION}.deb will be the same across releases.
artifact_final_version_reversioned="${REVISION}-${RELEASE}"
artifact_map_debs=(["armbian-base-files"]="${RELEASE}/${deb_name}_${artifact_version}_${ARCH}.deb")
# Register the function used to re-version the _contents_ of the base-files deb file.
artifact_debs_reversion_functions+=("reversion_armbian-base-files_deb_contents")
return 0
}
@ -123,18 +125,14 @@ function compile_armbian-base-files() {
cp "${destination}"/etc/issue.net "${destination}"/etc/issue.net.orig
cp "${destination}"/DEBIAN/conffiles "${destination}"/DEBIAN/conffiles.orig
# Change the PRETTY_NAME and add ARMBIAN_PRETTY_NAME in os-release, and change issue, issue.net
declare orig_distro_release="${RELEASE}"
# Attention: this is just a few base changes that don't involve "$REVISION".
# More are done in reversion_armbian-base-files_deb_contents()
cat <<- EOD >> "${destination}/etc/dpkg/origins/armbian"
Vendor: ${VENDOR}
Vendor-URL: ${VENDORURL}
Bugs: ${VENDORBUGS}
Parent: ${DISTRIBUTION}
EOD
echo "ARMBIAN_PRETTY_NAME=\"${VENDOR} ${REVISION} ${orig_distro_release}\"" >> "${destination}"/etc/os-release
echo -e "${VENDOR} ${REVISION} ${orig_distro_release} \\l \n" > "${destination}"/etc/issue
echo -e "${VENDOR} ${REVISION} ${orig_distro_release}" > "${destination}"/etc/issue.net
sed -i "s/^PRETTY_NAME=.*/PRETTY_NAME=\"${VENDOR} $REVISION ${orig_distro_release}\"/" "${destination}"/etc/os-release
sed -i "s|^HOME_URL=.*|HOME_URL=\"${VENDORURL}\"|" "${destination}"/etc/os-release
sed -i "s|^SUPPORT_URL=.*|SUPPORT_URL=\"${VENDORSUPPORT}\"|" "${destination}"/etc/os-release
sed -i "s|^BUG_REPORT_URL=.*|BUG_REPORT_URL=\"${VENDORBUGS}\"|" "${destination}"/etc/os-release
@ -181,11 +179,38 @@ function compile_armbian-base-files() {
rm -f "${destination}"/etc/os-release.orig "${destination}"/etc/issue.orig "${destination}"/etc/issue.net.orig "${destination}"/DEBIAN/conffiles.orig
# Done, pack it.
fakeroot_dpkg_deb_build "${destination}"
fakeroot_dpkg_deb_build "${destination}" "armbian-base-files"
done_with_temp_dir "${cleanup_id}" # changes cwd to "${SRC}" and fires the cleanup function early
}
# Used to reversion the artifact contents.
function reversion_armbian-base-files_deb_contents() {
display_alert "Reversioning" "reversioning base-files CONTENTS: '$*'" "debug"
declare orig_distro_release="${RELEASE}"
artifact_deb_reversion_unpack_data_deb
: "${data_dir:?data_dir is not set}"
# Change the PRETTY_NAME and add ARMBIAN_PRETTY_NAME in os-release, and change issue, issue.net
echo "ARMBIAN_PRETTY_NAME=\"${VENDOR} ${REVISION} ${orig_distro_release}\"" >> "${data_dir}"/etc/os-release
echo -e "${VENDOR} ${REVISION} ${orig_distro_release} \\l \n" > "${data_dir}"/etc/issue
echo -e "${VENDOR} ${REVISION} ${orig_distro_release}" > "${data_dir}"/etc/issue.net
sed -i "s/^PRETTY_NAME=.*/PRETTY_NAME=\"${VENDOR} $REVISION ${orig_distro_release}\"/" "${data_dir}"/etc/os-release
# Show results if debugging
if [[ "${SHOW_DEBUG}" == "yes" ]]; then
run_tool_batcat --file-name "/etc/os-release.sh" "${data_dir}"/etc/os-release
run_tool_batcat --file-name "/etc/issue" "${data_dir}"/etc/issue
run_tool_batcat --file-name "/etc/issue.net" "${data_dir}"/etc/issue.net
fi
artifact_deb_reversion_repack_data_deb
return 0
}
function artifact_armbian-base-files_cli_adapter_pre_run() {
declare -g ARMBIAN_COMMAND_REQUIRE_BASIC_DEPS="yes" # Require prepare_host_basic to run before the command.

View File

@ -101,22 +101,20 @@ function artifact_armbian-bsp-cli_prepare_version() {
artifact_version_reason="${reasons[*]}" # outer scope
artifact_deb_repo="global" # "global" meaning: release-independent repo. could be '${RELEASE}' for a release-specific package.
artifact_deb_arch="${ARCH}" # arch-specific package, or 'all' for arch-independent package.
artifact_name="armbian-bsp-cli-${BOARD}-${BRANCH}${EXTRA_BSP_NAME}"
artifact_type="deb-tar"
artifact_base_dir="${PACKAGES_HASHED_STORAGE}"
artifact_final_file="${PACKAGES_HASHED_STORAGE}/${artifact_name}_${artifact_version}_${ARCH}.tar"
artifact_map_packages=(
["armbian-bsp-cli"]="${artifact_name}"
)
artifact_map_packages=(["armbian-bsp-cli"]="${artifact_name}")
artifact_map_debs=(
["armbian-bsp-cli"]="${artifact_name}_${artifact_version}_${ARCH}.deb"
)
# Register the function used to re-version the _contents_ of the bsp-cli deb file (non-transitional)
artifact_debs_reversion_functions+=("reversion_armbian-bsp-cli_deb_contents")
if artifact_armbian-bsp-cli_needs_transitional_package ; then
if artifact_armbian-bsp-cli_needs_transitional_package; then
artifact_map_packages+=(["armbian-bsp-cli-transitional"]="armbian-bsp-cli-${BOARD}${EXTRA_BSP_NAME}")
artifact_map_debs+=(["armbian-bsp-cli-transitional"]="armbian-bsp-cli-${BOARD}${EXTRA_BSP_NAME}_${artifact_version}_${ARCH}.deb")
# Register the function used to re-version the _contents_ of the bsp-cli deb file (transitional)
artifact_debs_reversion_functions+=("reversion_armbian-bsp-cli-transitional_deb_contents")
fi
return 0
@ -124,7 +122,7 @@ function artifact_armbian-bsp-cli_prepare_version() {
function artifact_armbian-bsp-cli_build_from_sources() {
# Generate transitional package when needed.
if artifact_armbian-bsp-cli_needs_transitional_package ; then
if artifact_armbian-bsp-cli_needs_transitional_package; then
LOG_SECTION="compile_armbian-bsp-cli" do_with_logging compile_armbian-bsp-cli-transitional
fi
@ -164,11 +162,11 @@ function artifact_armbian-bsp-cli_deploy_to_remote_cache() {
}
function artifact_armbian-bsp-cli_needs_transitional_package() {
if [[ "${KERNEL_TARGET}" == "${BRANCH}" ]] ; then
if [[ "${KERNEL_TARGET}" == "${BRANCH}" ]]; then
return 0
elif [[ "${BRANCH}" == "current" ]] ; then
elif [[ "${BRANCH}" == "current" ]]; then
return 0
elif [[ "${KERNEL_TARGET}" != *current* && "${BRANCH}" == "legacy" ]] ; then
elif [[ "${KERNEL_TARGET}" != *current* && "${BRANCH}" == "legacy" ]]; then
return 0
else
return 1

View File

@ -53,16 +53,10 @@ function artifact_armbian-bsp-desktop_prepare_version() {
artifact_name="armbian-bsp-desktop-${BOARD}-${BRANCH}"
artifact_type="deb"
artifact_base_dir="${PACKAGES_HASHED_STORAGE}/${RELEASE}"
artifact_final_file="${PACKAGES_HASHED_STORAGE}/${RELEASE}/${artifact_name}_${artifact_version}_${ARCH}.deb"
artifact_deb_repo="${RELEASE}"
artifact_deb_arch="${ARCH}"
artifact_map_packages=(
["armbian-bsp-desktop"]="${artifact_name}"
)
artifact_map_debs=(
["armbian-bsp-desktop"]="${RELEASE}/${artifact_name}_${artifact_version}_${ARCH}.deb"
)
artifact_map_packages=(["armbian-bsp-desktop"]="${artifact_name}")
return 0
}

View File

@ -50,18 +50,12 @@ function artifact_armbian-config_prepare_version() {
artifact_version_reason="${reasons[*]}" # outer scope
artifact_map_packages=(
["armbian-config"]="armbian-config"
)
artifact_map_debs=(
["armbian-config"]="armbian-config_${artifact_version}_all.deb"
)
artifact_map_packages=(["armbian-config"]="armbian-config")
artifact_name="armbian-config"
artifact_type="deb"
artifact_base_dir="${PACKAGES_HASHED_STORAGE}"
artifact_final_file="${PACKAGES_HASHED_STORAGE}/armbian-config_${artifact_version}_all.deb"
artifact_deb_repo="global"
artifact_deb_arch="all"
return 0
}

View File

@ -59,16 +59,10 @@ function artifact_armbian-desktop_prepare_version() {
artifact_name="armbian-${RELEASE}-desktop-${DESKTOP_ENVIRONMENT}"
artifact_type="deb"
artifact_base_dir="${PACKAGES_HASHED_STORAGE}/${RELEASE}"
artifact_final_file="${PACKAGES_HASHED_STORAGE}/${RELEASE}/${artifact_name}_${artifact_version}_all.deb"
artifact_deb_repo="${RELEASE}"
artifact_deb_arch="all"
artifact_map_packages=(
["armbian-desktop"]="${artifact_name}"
)
artifact_map_debs=(
["armbian-desktop"]="${RELEASE}/${artifact_name}_${artifact_version}_all.deb"
)
artifact_map_packages=(["armbian-desktop"]="${artifact_name}")
return 0
}

View File

@ -36,18 +36,12 @@ function artifact_armbian-plymouth-theme_prepare_version() {
artifact_version_reason="${reasons[*]}" # outer scope
artifact_map_packages=(
["armbian-plymouth-theme"]="armbian-plymouth-theme"
)
artifact_map_debs=(
["armbian-plymouth-theme"]="armbian-plymouth-theme_${artifact_version}_all.deb"
)
artifact_map_packages=(["armbian-plymouth-theme"]="armbian-plymouth-theme")
artifact_name="armbian-plymouth-theme"
artifact_type="deb"
artifact_base_dir="${PACKAGES_HASHED_STORAGE}"
artifact_final_file="${PACKAGES_HASHED_STORAGE}/armbian-plymouth-theme_${artifact_version}_all.deb"
artifact_deb_repo="global"
artifact_deb_arch="all"
return 0
}

View File

@ -51,18 +51,12 @@ function artifact_armbian-zsh_prepare_version() {
artifact_version_reason="${reasons[*]}" # outer scope
artifact_map_packages=(
["armbian-zsh"]="armbian-zsh"
)
artifact_map_debs=(
["armbian-zsh"]="armbian-zsh_${artifact_version}_all.deb"
)
artifact_map_packages=(["armbian-zsh"]="armbian-zsh")
artifact_name="armbian-zsh"
artifact_type="deb"
artifact_base_dir="${PACKAGES_HASHED_STORAGE}"
artifact_final_file="${PACKAGES_HASHED_STORAGE}/armbian-zsh_${artifact_version}_all.deb"
artifact_deb_repo="global"
artifact_deb_arch="all"
return 0
}

View File

@ -36,18 +36,12 @@ function artifact_fake_ubuntu_advantage_tools_prepare_version() {
artifact_version_reason="${reasons[*]}" # outer scope
artifact_map_packages=(
["fake-ubuntu-advantage-tools"]="fake-ubuntu-advantage-tools"
)
artifact_map_debs=(
["fake-ubuntu-advantage-tools"]="fake-ubuntu-advantage-tools_${artifact_version}_all.deb"
)
artifact_map_packages=(["fake-ubuntu-advantage-tools"]="fake-ubuntu-advantage-tools")
artifact_name="fake-ubuntu-advantage-tools"
artifact_type="deb"
artifact_base_dir="${PACKAGES_HASHED_STORAGE}"
artifact_final_file="${PACKAGES_HASHED_STORAGE}/fake-ubuntu-advantage-tools_${artifact_version}_all.deb"
artifact_deb_repo="global"
artifact_deb_arch="all"
return 0
}

View File

@ -51,18 +51,12 @@ function artifact_firmware_prepare_version() {
artifact_version_reason="${reasons[*]}" # outer scope
artifact_map_packages=(
["armbian-firmware"]="armbian-firmware"
)
artifact_map_debs=(
["armbian-firmware"]="armbian-firmware_${artifact_version}_all.deb"
)
artifact_map_packages=(["armbian-firmware"]="armbian-firmware")
artifact_name="armbian-firmware"
artifact_type="deb"
artifact_base_dir="${PACKAGES_HASHED_STORAGE}"
artifact_final_file="${PACKAGES_HASHED_STORAGE}/armbian-firmware_${artifact_version}_all.deb"
artifact_deb_repo="global"
artifact_deb_arch="all"
return 0
}

View File

@ -61,18 +61,12 @@ function artifact_full_firmware_prepare_version() {
artifact_version_reason="${reasons[*]}" # outer scope
artifact_map_packages=(
["armbian-firmware-full"]="armbian-firmware-full"
)
artifact_map_debs=(
["armbian-firmware-full"]="armbian-firmware-full_${artifact_version}_all.deb"
)
artifact_map_packages=(["armbian-firmware-full"]="armbian-firmware-full")
artifact_name="armbian-firmware-full"
artifact_type="deb"
artifact_base_dir="${PACKAGES_HASHED_STORAGE}"
artifact_final_file="${PACKAGES_HASHED_STORAGE}/armbian-firmware-full_${artifact_version}_all.deb"
artifact_deb_repo="global"
artifact_deb_arch="all"
return 0
}

View File

@ -196,24 +196,21 @@ function artifact_kernel_prepare_version() {
# linux-image is always produced...
artifact_map_packages=(["linux-image"]="linux-image-${BRANCH}-${LINUXFAMILY}")
artifact_map_debs=(["linux-image"]="linux-image-${BRANCH}-${LINUXFAMILY}_${artifact_version}_${ARCH}.deb")
# some/most kernels have also working headers...
if [[ "${KERNEL_HAS_WORKING_HEADERS:-"no"}" == "yes" ]]; then
artifact_map_packages+=(["linux-headers"]="linux-headers-${BRANCH}-${LINUXFAMILY}")
artifact_map_debs+=(["linux-headers"]="linux-headers-${BRANCH}-${LINUXFAMILY}_${artifact_version}_${ARCH}.deb")
fi
# x86, specially, does not have working dtbs...
if [[ "${KERNEL_BUILD_DTBS:-"yes"}" == "yes" ]]; then
artifact_map_packages+=(["linux-dtb"]="linux-dtb-${BRANCH}-${LINUXFAMILY}")
artifact_map_debs+=(["linux-dtb"]="linux-dtb-${BRANCH}-${LINUXFAMILY}_${artifact_version}_${ARCH}.deb")
fi
artifact_name="kernel-${LINUXFAMILY}-${BRANCH}"
artifact_type="deb-tar" # this triggers processing of .deb files in the maps to produce a tarball
artifact_base_dir="${PACKAGES_HASHED_STORAGE}"
artifact_final_file="${PACKAGES_HASHED_STORAGE}/kernel-${LINUXFAMILY}-${BRANCH}_${artifact_version}.tar"
artifact_deb_repo="global"
artifact_deb_arch="${ARCH}"
return 0
}
@ -222,7 +219,7 @@ function artifact_kernel_build_from_sources() {
compile_kernel
if [[ "${ARTIFACT_WILL_NOT_BUILD}" != "yes" ]]; then # true if kernel-patch, kernel-config, etc.
display_alert "Kernel build finished" "${artifact_version_reason}" "info"
display_alert "Kernel build finished" "${artifact_version}" "info"
fi
}

View File

@ -55,16 +55,16 @@ function artifact_uboot_prepare_version() {
declare patches_hash="undetermined"
declare hash_files="undetermined"
declare -a uboot_patch_dirs=()
for patch_dir in ${BOOTPATCHDIR} ; do
uboot_patch_dirs+=( "${SRC}/patch/u-boot/${patch_dir}" "${USERPATCHES_PATH}/u-boot/${patch_dir}" )
for patch_dir in ${BOOTPATCHDIR}; do
uboot_patch_dirs+=("${SRC}/patch/u-boot/${patch_dir}" "${USERPATCHES_PATH}/u-boot/${patch_dir}")
done
if [[ -n "${ATFSOURCE}" && "${ATFSOURCE}" != "none" ]]; then
uboot_patch_dirs+=( "${SRC}/patch/atf/${ATFPATCHDIR}" "${USERPATCHES_PATH}/atf/${ATFPATCHDIR}" )
uboot_patch_dirs+=("${SRC}/patch/atf/${ATFPATCHDIR}" "${USERPATCHES_PATH}/atf/${ATFPATCHDIR}")
fi
if [[ -n "${CRUSTCONFIG}" ]]; then
uboot_patch_dirs+=( "${SRC}/patch/crust/${CRUSTPATCHDIR}" "${USERPATCHES_PATH}/crust/${CRUSTPATCHDIR}" )
uboot_patch_dirs+=("${SRC}/patch/crust/${CRUSTPATCHDIR}" "${USERPATCHES_PATH}/crust/${CRUSTPATCHDIR}")
fi
calculate_hash_for_all_files_in_dirs "${uboot_patch_dirs[@]}"
@ -122,20 +122,12 @@ function artifact_uboot_prepare_version() {
"framework bash hash \"${bash_hash}\""
)
artifact_deb_repo="global"
artifact_deb_arch="${ARCH}"
artifact_version_reason="${reasons[*]}" # outer scope
artifact_map_packages=(
["uboot"]="linux-u-boot-${BOARD}-${BRANCH}"
)
artifact_map_debs=(
["uboot"]="linux-u-boot-${BOARD}-${BRANCH}_${artifact_version}_${ARCH}.deb"
)
artifact_map_packages=(["uboot"]="linux-u-boot-${BOARD}-${BRANCH}")
artifact_name="uboot-${BOARD}-${BRANCH}"
artifact_type="deb"
artifact_base_dir="${PACKAGES_HASHED_STORAGE}"
artifact_final_file="${PACKAGES_HASHED_STORAGE}/linux-u-boot-${BOARD}-${BRANCH}_${artifact_version}_${ARCH}.deb"
return 0
}

View File

@ -72,12 +72,17 @@ function obtain_complete_artifact() {
declare -g artifact_type="undetermined"
declare -g artifact_version="undetermined"
declare -g artifact_version_reason="undetermined"
declare -g artifact_final_version_reversioned="${REVISION}" # by default
declare -g artifact_base_dir="undetermined"
declare -g artifact_final_file="undetermined"
declare -g artifact_final_file_basename="undetermined"
declare -g artifact_full_oci_target="undetermined"
declare -g artifact_deb_repo="undetermined"
declare -g artifact_deb_arch="undetermined"
declare -A -g artifact_map_packages=()
declare -A -g artifact_map_debs=()
declare -A -g artifact_map_debs_reversioned=()
declare -a -g artifact_debs_reversion_functions=()
# Contentious; it might be that prepare_version is complex enough to warrant more than 1 logging section.
LOG_SECTION="artifact_prepare_version" do_with_logging artifact_prepare_version
@ -86,47 +91,98 @@ function obtain_complete_artifact() {
debug_var artifact_type
debug_var artifact_version
debug_var artifact_version_reason
debug_var artifact_base_dir
debug_var artifact_final_file
debug_dict artifact_map_packages
debug_dict artifact_map_debs
# sanity checks. artifact_version/artifact_version_reason/artifact_final_file *must* be set
[[ "x${artifact_name}x" == "xx" || "${artifact_name}" == "undetermined" ]] && exit_with_error "artifact_name is not set after artifact_prepare_version"
[[ "x${artifact_type}x" == "xx" || "${artifact_type}" == "undetermined" ]] && exit_with_error "artifact_type is not set after artifact_prepare_version"
[[ "x${artifact_version}x" == "xx" || "${artifact_version}" == "undetermined" ]] && exit_with_error "artifact_version is not set after artifact_prepare_version"
[[ "x${artifact_version_reason}x" == "xx" || "${artifact_version_reason}" == "undetermined" ]] && exit_with_error "artifact_version_reason is not set after artifact_prepare_version"
[[ "x${artifact_base_dir}x" == "xx" || "${artifact_base_dir}" == "undetermined" ]] && exit_with_error "artifact_base_dir is not set after artifact_prepare_version"
[[ "x${artifact_final_file}x" == "xx" || "${artifact_final_file}" == "undetermined" ]] && exit_with_error "artifact_final_file is not set after artifact_prepare_version"
# validate artifact_version begins with a digit when building deb packages (or deb-tar); dpkg requires it
if [[ "${artifact_type}" != "tar.zst" ]]; then
[[ "${artifact_version}" =~ ^[0-9] ]] || exit_with_error "${artifact_type}: artifact_version '${artifact_version}' does not begin with a digit"
fi
declare -a artifact_map_debs_values=() artifact_map_packages_values=() artifact_map_debs_keys=() artifact_map_packages_keys=()
declare -a artifact_map_debs_values=()
declare -a artifact_map_packages_values=()
declare -a artifact_map_debs_keys=()
declare -a artifact_map_packages_keys=()
declare -a artifact_map_debs_reversioned_keys=()
declare -a artifact_map_debs_reversioned_values=()
# validate artifact_type... it must be one of the supported types
case "${artifact_type}" in
deb | deb-tar)
# validate artifact_version begins with a digit
# check artifact_base_dir and artifact_base_dir are 'undetermined', or bomb; deb/deb-tar shouldn't set those anymore
[[ "${artifact_base_dir}" != "undetermined" ]] && exit_with_error "artifact ${artifact_name} is setting artifact_base_dir, legacy code, remove."
[[ "${artifact_final_file}" != "undetermined" ]] && exit_with_error "artifact ${artifact_name} is setting artifact_final_file, legacy code, remove."
# validate artifact_version begins with a digit when building deb packages; dpkg requires it
[[ "${artifact_version}" =~ ^[0-9] ]] || exit_with_error "${artifact_type}: artifact_version '${artifact_version}' does not begin with a digit"
# since it's a deb or deb-tar, validate deb-specific variables
[[ "x${artifact_deb_repo}x" == "xx" || "${artifact_deb_repo}" == "undetermined" ]] && exit_with_error "artifact_deb_repo is not set after artifact_prepare_version"
[[ "x${artifact_deb_arch}x" == "xx" || "${artifact_deb_arch}" == "undetermined" ]] && exit_with_error "artifact_deb_arch is not set after artifact_prepare_version"
# validate there's at least one item in artifact_map_packages
[[ "${#artifact_map_packages[@]}" -eq 0 ]] && exit_with_error "artifact_map_packages is empty after artifact_prepare_version"
# Add the reversioning hash to the artifact_version
declare artifact_reversioning_hash="undetermined"
artifact_calculate_reversioning_hash
declare artifact_reversioning_hash_short="${artifact_reversioning_hash:0:4}"
artifact_version="${artifact_version}-R${artifact_reversioning_hash_short}"
display_alert "Final artifact_version with reversioning hash" "${artifact_version}" "debug"
debug_dict artifact_map_packages
debug_dict artifact_map_debs
debug_dict artifact_map_debs_reversioned
# produce the mapped/reversioned deb info given the debs.
declare one_artifact_deb_id one_artifact_deb_package
declare -i debs_counter=0
declare single_deb_hashed_rel_path
for one_artifact_deb_id in "${!artifact_map_packages[@]}"; do
one_artifact_deb_package="${artifact_map_packages["${one_artifact_deb_id}"]}"
# @TODO: might be "${artifact_name}/${artifact_version}/" in the middle can be beneficial for cleaning, later?
single_deb_hashed_rel_path="${artifact_deb_repo}/${one_artifact_deb_package}_${artifact_version}_${artifact_deb_arch}.deb"
artifact_map_debs+=(["${one_artifact_deb_id}"]="${single_deb_hashed_rel_path}")
artifact_map_debs_reversioned+=(["${one_artifact_deb_id}"]="${REVISION}/${artifact_deb_repo}/${artifact_name}/${artifact_version}/${one_artifact_deb_package}_${artifact_final_version_reversioned}_${artifact_deb_arch}.deb")
debs_counter+=1
done
# moved from each artifact:
# deb-tar:
if [[ "${artifact_type}" == "deb-tar" ]]; then
artifact_base_dir="${PACKAGES_HASHED_STORAGE}" # deb-tar's always at the root. they're temporary anyway
artifact_final_file="${artifact_base_dir}/${artifact_name}_${artifact_version}_${artifact_deb_arch}.tar"
else # deb, single-deb
# bomb if we have more than one...
[[ "${debs_counter}" -gt 1 ]] && exit_with_error "artifact_type '${artifact_type}' has more than one deb file. This is not supported."
# just use the single deb rel path
artifact_base_dir="${PACKAGES_HASHED_STORAGE}"
artifact_final_file="${artifact_base_dir}/${single_deb_hashed_rel_path}"
fi
debug_dict artifact_map_packages
debug_dict artifact_map_debs
debug_dict artifact_map_debs_reversioned
# grab the the deb maps, and add them to plain arrays.
artifact_map_debs_keys=("${!artifact_map_debs[@]}")
artifact_map_debs_values=("${artifact_map_debs[@]}")
artifact_map_packages_keys=("${!artifact_map_packages[@]}")
artifact_map_packages_values=("${artifact_map_packages[@]}")
artifact_map_debs_reversioned_keys=("${!artifact_map_debs_reversioned[@]}")
artifact_map_debs_reversioned_values=("${artifact_map_debs_reversioned[@]}")
;;
tar.zst)
: # valid, no restrictions on tar.zst versioning
# tar.zst (rootfs) must specify the directories directly, since we can't determine from deb info.
[[ "x${artifact_base_dir}x" == "xx" || "${artifact_base_dir}" == "undetermined" ]] && exit_with_error "artifact_base_dir is not set after artifact_prepare_version"
[[ "x${artifact_final_file}x" == "xx" || "${artifact_final_file}" == "undetermined" ]] && exit_with_error "artifact_final_file is not set after artifact_prepare_version"
;;
*)
exit_with_error "artifact_type '${artifact_type}' is not supported"
;;
esac
debug_var artifact_base_dir
debug_var artifact_final_file
# set those as outputs for GHA
github_actions_add_output artifact_name "${artifact_name}"
github_actions_add_output artifact_type "${artifact_type}"
@ -168,8 +224,11 @@ function obtain_complete_artifact() {
declare -a wanted_vars=(
artifact_name
artifact_type
artifact_deb_repo
artifact_deb_arch
artifact_version
artifact_version_reason
artifact_final_version_reversioned
artifact_base_dir
artifact_final_file
artifact_final_file_basename
@ -181,6 +240,8 @@ function obtain_complete_artifact() {
artifact_map_debs_values
artifact_map_packages_keys
artifact_map_packages_values
artifact_map_debs_reversioned_keys
artifact_map_debs_reversioned_values
)
declare -A ARTIFACTS_VAR_DICT=()
@ -232,10 +293,11 @@ function obtain_complete_artifact() {
if [[ "${artifact_exists_in_remote_cache}" == "yes" ]]; then
display_alert "artifact" "exists in remote cache: ${artifact_name} ${artifact_version}" "debug"
if [[ "${skip_unpack_if_found_in_caches:-"no"}" == "yes" ]]; then
display_alert "artifact" "skipping obtain from remote & unpacking as requested" "info"
return 0
fi
# @TODO: rpardini: WHY THE HELL WE HAD THIS? there's no point in this at all?
#if [[ "${skip_unpack_if_found_in_caches:-"no"}" == "yes" ]]; then
# display_alert "artifact" "skipping obtain from remote & unpacking as requested" "info"
# return 0
#fi
LOG_SECTION="artifact_obtain_from_remote_cache" do_with_logging artifact_obtain_from_remote_cache
LOG_SECTION="unpack_artifact_from_local_cache" do_with_logging unpack_artifact_from_local_cache
display_alert "artifact" "obtained from remote cache: ${artifact_name} ${artifact_version}" "cachehit"
@ -306,13 +368,16 @@ function build_artifact_for_image() {
obtain_complete_artifact
fi
artifact_reversion_for_deployment
debug_dict artifact_map_debs_reversioned
return 0
}
function pack_artifact_to_local_cache() {
if [[ "${artifact_type}" == "deb-tar" ]]; then
declare -a files_to_tar=()
run_host_command_logged tar -C "${artifact_base_dir}" -cvf "${artifact_final_file}" "${artifact_map_debs[@]}"
run_host_command_logged tar -C "${artifact_base_dir}" -cf "${artifact_final_file}" "${artifact_map_debs[@]}"
display_alert "Created deb-tar artifact" "deb-tar: ${artifact_final_file}" "info"
fi
}
@ -362,7 +427,7 @@ function upload_artifact_to_oci() {
# If this is a deb-tar, delete the .tar after the upload. We won't ever need it again.
if [[ "${artifact_type}" == "deb-tar" ]]; then
display_alert "Deleting deb-tar after OCI deploy" "deb-tar: ${artifact_final_file_basename}" "warn" # @TODO
display_alert "Deleting deb-tar after OCI deploy" "deb-tar: ${artifact_final_file_basename}" "debug"
run_host_command_logged rm -fv "${artifact_final_file}"
fi
}
@ -423,6 +488,15 @@ function obtain_artifact_from_remote_cache() {
display_alert "Obtaining artifact from remote cache" "${artifact_full_oci_target} into ${artifact_final_file_basename}" "info"
oras_pull_artifact_file "${artifact_full_oci_target}" "${artifact_base_dir}" "${artifact_final_file_basename}"
# if this is a 'deb', (not deb-tar, not tar.zst), OCI hasn't kept the directory structure, so move it into place.
if [[ "${artifact_type}" == "deb" ]]; then
declare final_file_dirname
final_file_dirname="$(dirname "${artifact_final_file}")"
mkdir -p "${final_file_dirname}"
display_alert "Moving deb into place" "deb: ${artifact_final_file_basename}" "debug"
run_host_command_logged mv "${artifact_base_dir}/${artifact_final_file_basename}" "${artifact_final_file}"
fi
# sanity check: after obtaining remotely, is it available locally? it should, otherwise there's some inconsistency.
declare artifact_exists_in_local_cache="not-yet-after-obtaining-remotely"
is_artifact_available_in_local_cache

View File

@ -0,0 +1,156 @@
#
# SPDX-License-Identifier: GPL-2.0
# Copyright (c) 2023 Ricardo Pardini <ricardo@pardini.net>
# This file is a part of the Armbian Build Framework https://github.com/armbian/build/
#
function artifact_reversion_for_deployment() {
standard_artifact_reversion_for_deployment "${artifact_debs_reversion_functions[@]}"
}
function artifact_calculate_reversioning_hash() {
declare hash_functions="undetermined"
declare -a all_functions=("standard_artifact_reversion_for_deployment" "standard_artifact_reversion_for_deployment_one_deb")
all_functions+=("artifact_deb_reversion_unpack_data_deb" "artifact_deb_reversion_repack_data_deb")
all_functions+=("${artifact_debs_reversion_functions[@]}")
calculate_hash_for_function_bodies "${all_functions[@]}" # sets hash_functions
artifact_reversioning_hash="${hash_functions}" # outer scope
return 0
}
function standard_artifact_reversion_for_deployment() {
display_alert "Reversioning package" "re-version '${artifact_name}(${artifact_type})::${artifact_version}' to '${artifact_final_version_reversioned}'" "info"
declare artifact_mapped_deb one_artifact_deb_package
for one_artifact_deb_package in "${!artifact_map_packages[@]}"; do
declare artifact_mapped_deb="${artifact_map_debs["${one_artifact_deb_package}"]}"
declare hashed_storage_deb_full_path="${PACKAGES_HASHED_STORAGE}/${artifact_mapped_deb}"
if [[ ! -f "${hashed_storage_deb_full_path}" ]]; then
exit_with_error "hashed storage does not have ${hashed_storage_deb_full_path}"
fi
display_alert "Found hashed storage file" "'${artifact_mapped_deb}': ${hashed_storage_deb_full_path}" "debug"
# find the target dir and full path to the reversioned file
declare deb_versioned_rel_path="${artifact_map_debs_reversioned["${one_artifact_deb_package}"]}"
declare deb_versioned_full_path="${DEB_STORAGE}/${deb_versioned_rel_path}"
declare deb_versioned_dirname
deb_versioned_dirname="$(dirname "${deb_versioned_full_path}")"
run_host_command_logged mkdir -p "${deb_versioned_dirname}"
# since the full versioned path includes the original hash, if the file already exists, we can trust
# it's the correct one, and skip reversioning.
if [[ -f "${deb_versioned_full_path}" ]]; then
display_alert "Skipping reversioning" "deb: ${deb_versioned_full_path} already exists" "debug"
continue
fi
# call function for each deb, pass parameters
standard_artifact_reversion_for_deployment_one_deb "${@}"
# make sure reversioning produced the expected file
if [[ ! -f "${deb_versioned_full_path}" ]]; then
exit_with_error "reversioning did not produce the expected file: ${deb_versioned_full_path}"
fi
done
return 0
}
function standard_artifact_reversion_for_deployment_one_deb() {
display_alert "Will repack" "one_artifact_deb_package: ${one_artifact_deb_package}" "debug"
display_alert "Will repack" "hashed_storage_deb_full_path: ${hashed_storage_deb_full_path}" "debug"
display_alert "Will repack" "deb_versioned_full_path: ${deb_versioned_full_path}" "debug"
display_alert "Will repack" "artifact_version: ${artifact_version}" "debug"
declare cleanup_id="" unpack_dir=""
prepare_temp_dir_in_workdir_and_schedule_cleanup "reversion-${artifact_name}" cleanup_id unpack_dir # namerefs
declare deb_contents_dir="${unpack_dir}/deb-contents"
mkdir -p "${deb_contents_dir}"
# unpack the hashed_storage_deb_full_path .deb, which is just an "ar" file, to the deb_contents_dir
run_host_command_logged ar x "${hashed_storage_deb_full_path}" --output="${deb_contents_dir}"
# find out if compressed or not, and store for future recompressing
control_compressed=""
if [[ -f "${deb_contents_dir}/control.tar.xz" ]]; then
control_compressed=".xz"
run_host_command_logged xz -d "${deb_contents_dir}/control.tar.xz" # decompress
fi
# untar the control into its own specific dir
declare control_dir="${unpack_dir}/control"
mkdir -p "${control_dir}"
run_host_command_logged tar -xf "${deb_contents_dir}/control.tar" --directory="${control_dir}"
# prepare for unpacking the data tarball as well
declare data_dir="${unpack_dir}/data"
mkdir -p "${data_dir}"
declare data_compressed=""
if [[ -f "${deb_contents_dir}/data.tar.xz" ]]; then
data_compressed=".xz"
fi
# Hack at the control file...
declare control_file="${control_dir}/control"
declare control_file_new="${control_dir}/control.new"
# Replace "Version: " field with our own
sed -e "s/^Version: .*/Version: ${artifact_final_version_reversioned}/" "${control_file}" > "${control_file_new}"
echo "Armbian-Original-Hash: ${artifact_version}" >> "${control_file_new}" # non-standard field.
for one_reversion_function_name in "${@}"; do
display_alert "reversioning" "call custom function: '${one_reversion_function_name}'" "debug"
"${one_reversion_function_name}" "${one_artifact_deb_package}"
done
# Show a nice diff using batcat if debugging
if [[ "${SHOW_DEBUG}" == "yes" ]]; then
diff -u "${control_file}" "${control_file_new}" > "${unpack_dir}/control.diff" || true
run_tool_batcat "${unpack_dir}/control.diff"
fi
# Move new control on top of old
run_host_command_logged mv "${control_file_new}" "${control_file}"
run_host_command_logged rm "${deb_contents_dir}/control.tar"
cd "${control_dir}" || exit_with_error "cray-cray about control_dir ${control_dir}"
run_host_command_logged tar cf "${deb_contents_dir}/control.tar" .
# if it was compressed to begin with, recompress...
if [[ "${control_compressed}" == ".xz" ]]; then
run_host_command_logged xz "${deb_contents_dir}/control.tar"
fi
# re-ar the whole .deb back in place, using the new version for filename.
run_host_command_logged ar rcs "${deb_versioned_full_path}" \
"${deb_contents_dir}/debian-binary" \
"${deb_contents_dir}/control.tar${control_compressed}" \
"${deb_contents_dir}/data.tar${data_compressed}"
done_with_temp_dir "${cleanup_id}" # changes cwd to "${SRC}" and fires the cleanup function early
return 0
}
function artifact_deb_reversion_unpack_data_deb() {
if [[ "${data_compressed}" == ".xz" ]]; then
run_host_command_logged xz -d "${deb_contents_dir}/data.tar.xz" # decompress
fi
run_host_command_logged tar -xf "${deb_contents_dir}/data.tar" --directory="${data_dir}"
}
function artifact_deb_reversion_repack_data_deb() {
run_host_command_logged rm "${deb_contents_dir}/data.tar"
cd "${data_dir}" || exit_with_error "cray-cray about data_dir ${data_dir}"
run_host_command_logged tar cf "${deb_contents_dir}/data.tar" .
# if it was compressed to begin with, recompress...
if [[ "${data_compressed}" == ".xz" ]]; then
run_host_command_logged xz "${deb_contents_dir}/data.tar"
fi
}

View File

@ -30,18 +30,30 @@ function compile_armbian-bsp-cli-transitional() {
Maintainer: $MAINTAINER <$MAINTAINERMAIL>
Section: oldlibs
Priority: optional
Depends: ${artifact_name} (= ${artifact_version})
Description: Armbian CLI BSP for board '${BOARD}' - transitional package
EOF
# Build / close the package. This will run shellcheck / show the generated files if debugging
fakeroot_dpkg_deb_build "${destination}"
fakeroot_dpkg_deb_build "${destination}" "armbian-bsp-cli-transitional"
done_with_temp_dir "${cleanup_id}" # changes cwd to "${SRC}" and fires the cleanup function early
display_alert "Done building BSP CLI transitional package" "${destination}" "debug"
}
function reversion_armbian-bsp-cli-transitional_deb_contents() {
if [[ "${1}" != "armbian-bsp-cli-transitional" ]]; then
return 0 # Not our deb, nothing to do.
fi
display_alert "Reversion" "reversion_armbian-bsp-cli-transitional_deb_contents: '$*'" "debug"
# Depends on the new package
cat <<- EOF >> "${control_file_new}"
Depends: ${artifact_name} (= ${REVISION})
EOF
}
function compile_armbian-bsp-cli() {
: "${artifact_version:?artifact_version is not set}"
: "${artifact_name:?artifact_name is not set}"
@ -63,14 +75,6 @@ function compile_armbian-bsp-cli() {
declare -a extra_description=()
[[ "${EXTRA_BSP_NAME}" != "" ]] && extra_description+=("(variant '${EXTRA_BSP_NAME}')")
# Replaces: base-files is needed to replace /etc/update-motd.d/ files on Xenial
# Depends: linux-base is needed for "linux-version" command in initrd cleanup script
# Depends: fping is needed for armbianmonitor to upload armbian-hardware-monitor.log
# Depends: base-files (>= ${REVISION}) is to force usage of our base-files package (not the original Distro's).
declare depends_base_files=", base-files (>= ${REVISION})"
if [[ "${KEEP_ORIGINAL_OS_RELEASE:-"no"}" == "yes" ]]; then
depends_base_files=""
fi
cat <<- EOF > "${destination}"/DEBIAN/control
Package: ${artifact_name}
Version: ${artifact_version}
@ -78,10 +82,7 @@ function compile_armbian-bsp-cli() {
Maintainer: $MAINTAINER <$MAINTAINERMAIL>
Section: kernel
Priority: optional
Depends: bash, linux-base, u-boot-tools, initramfs-tools, lsb-release, fping${depends_base_files}
Suggests: armbian-config
Replaces: zram-config, armbian-bsp-cli-${BOARD}${EXTRA_BSP_NAME} (<< ${artifact_version})
Breaks: armbian-bsp-cli-${BOARD}${EXTRA_BSP_NAME} (<< ${artifact_version})
Recommends: bsdutils, parted, util-linux, toilet
Description: Armbian CLI BSP for board '${BOARD}' branch '${BRANCH}' ${extra_description[@]}
EOF
@ -95,7 +96,6 @@ function compile_armbian-bsp-cli() {
BOARDFAMILY=${BOARDFAMILY}
BUILD_REPOSITORY_URL=${BUILD_REPOSITORY_URL}
BUILD_REPOSITORY_COMMIT=${BUILD_REPOSITORY_COMMIT}
VERSION=${REVISION}
LINUXFAMILY=$LINUXFAMILY
ARCH=$ARCHITECTURE
IMAGE_TYPE=$IMAGE_TYPE
@ -104,7 +104,6 @@ function compile_armbian-bsp-cli() {
KERNEL_IMAGE_TYPE=$KERNEL_IMAGE_TYPE
FORCE_BOOTSCRIPT_UPDATE=$FORCE_BOOTSCRIPT_UPDATE
VENDOR=$VENDOR
REVISION=$REVISION
EOF
# copy general overlay from packages/bsp-cli
@ -210,13 +209,53 @@ function compile_armbian-bsp-cli() {
fi
# Build / close the package. This will run shellcheck / show the generated files if debugging
fakeroot_dpkg_deb_build "${destination}"
fakeroot_dpkg_deb_build "${destination}" "armbian-bsp-cli"
done_with_temp_dir "${cleanup_id}" # changes cwd to "${SRC}" and fires the cleanup function early
display_alert "Done building BSP CLI package" "${destination}" "debug"
}
# Reversion function is called with the following parameters:
# ${1} == deb_id
function reversion_armbian-bsp-cli_deb_contents() {
if [[ "${1}" != "armbian-bsp-cli" ]]; then
return 0 # Not our deb, nothing to do.
fi
display_alert "Reversion" "reversion_armbian-bsp-cli_deb_contents: '$*'" "debug"
# Replaces: base-files is needed to replace /etc/update-motd.d/ files on Xenial
# Depends: linux-base is needed for "linux-version" command in initrd cleanup script
# Depends: fping is needed for armbianmonitor to upload armbian-hardware-monitor.log
# Depends: base-files (>= ${REVISION}) is to force usage of our base-files package (not the original Distro's).
declare depends_base_files=", base-files (>= ${REVISION})"
if [[ "${KEEP_ORIGINAL_OS_RELEASE:-"no"}" == "yes" ]]; then
depends_base_files=""
fi
cat <<- EOF >> "${control_file_new}"
Depends: bash, linux-base, u-boot-tools, initramfs-tools, lsb-release, fping${depends_base_files}
Replaces: zram-config, armbian-bsp-cli-${BOARD}${EXTRA_BSP_NAME} (<< ${REVISION})
Breaks: armbian-bsp-cli-${BOARD}${EXTRA_BSP_NAME} (<< ${REVISION})
EOF
artifact_deb_reversion_unpack_data_deb
: "${data_dir:?data_dir is not set}"
cat <<- EOF >> "${data_dir}"/etc/armbian-release
VERSION=${REVISION}
REVISION=$REVISION
EOF
# Show results if debugging
if [[ "${SHOW_DEBUG}" == "yes" ]]; then
run_tool_batcat --file-name "armbian-release.sh" "${data_dir}"/etc/armbian-release
fi
artifact_deb_reversion_repack_data_deb
return 0
}
function get_bootscript_info() {
bootscript_info[has_bootscript]="no"
bootscript_info[has_extlinux]="no"

View File

@ -59,7 +59,7 @@ function compile_armbian-bsp-desktop() {
eval "${AGGREGATED_DESKTOP_BSP_PREPARE}"
display_alert "Done with bsp-desktop -specific aggregated prepare script" "AGGREGATED_DESKTOP_BSP_PREPARE" "debug"
fakeroot_dpkg_deb_build "${destination}"
fakeroot_dpkg_deb_build "${destination}" "armbian-bsp-desktop"
done_with_temp_dir "${cleanup_id}" # changes cwd to "${SRC}" and fires the cleanup function early
}

View File

@ -84,7 +84,7 @@ function cli_artifact_run() {
esac
# Force artifacts download we need to populate repository
if [[ "${FORCE_ARTIFACTS_DOWNLOAD}" == "yes" ]]; then
if [[ "${FORCE_ARTIFACTS_DOWNLOAD}" == "yes" ]]; then # @TODO should we remove this, Igor?
skip_unpack_if_found_in_caches="no"
fi
@ -93,8 +93,13 @@ function cli_artifact_run() {
if [[ "${ARTIFACT_BUILD_INTERACTIVE}" == "yes" ]]; then # Set by `kernel-config`, `kernel-patch`, `uboot-config`, `uboot-patch`, etc.
display_alert "Running artifact build in interactive mode" "log file will be incomplete" "info"
do_with_default_build obtain_complete_artifact
do_with_default_build cli_obtain_complete_artifact
else
do_with_default_build obtain_complete_artifact < /dev/null
do_with_default_build cli_obtain_complete_artifact < /dev/null
fi
}
function cli_obtain_complete_artifact() {
obtain_complete_artifact
artifact_reversion_for_deployment
}

View File

@ -68,17 +68,17 @@ function prepare_kernel_packaging_debs() {
# package the linux-image (image, modules, dtbs (if present))
display_alert "Packaging linux-image" "${LINUXFAMILY} ${LINUXCONFIG}" "info"
create_kernel_deb "linux-image-${BRANCH}-${LINUXFAMILY}" "${debs_target_dir}" kernel_package_callback_linux_image
create_kernel_deb "linux-image-${BRANCH}-${LINUXFAMILY}" "${debs_target_dir}" kernel_package_callback_linux_image "linux-image"
# if dtbs present, package those too separately, for u-boot usage.
if [[ -d "${tmp_kernel_install_dirs[INSTALL_DTBS_PATH]}" ]]; then
display_alert "Packaging linux-dtb" "${LINUXFAMILY} ${LINUXCONFIG}" "info"
create_kernel_deb "linux-dtb-${BRANCH}-${LINUXFAMILY}" "${debs_target_dir}" kernel_package_callback_linux_dtb
create_kernel_deb "linux-dtb-${BRANCH}-${LINUXFAMILY}" "${debs_target_dir}" kernel_package_callback_linux_dtb "linux-dtb"
fi
if [[ "${KERNEL_HAS_WORKING_HEADERS}" == "yes" ]]; then
display_alert "Packaging linux-headers" "${LINUXFAMILY} ${LINUXCONFIG}" "info"
create_kernel_deb "linux-headers-${BRANCH}-${LINUXFAMILY}" "${debs_target_dir}" kernel_package_callback_linux_headers
create_kernel_deb "linux-headers-${BRANCH}-${LINUXFAMILY}" "${debs_target_dir}" kernel_package_callback_linux_headers "linux-headers"
else
display_alert "Skipping linux-headers package" "for ${KERNEL_MAJOR_MINOR} kernel version" "info"
fi
@ -88,6 +88,7 @@ function create_kernel_deb() {
declare package_name="${1}"
declare deb_output_dir="${2}"
declare callback_function="${3}"
declare artifact_deb_id="${4}"
declare cleanup_id="" package_directory=""
prepare_temp_dir_in_workdir_and_schedule_cleanup "deb-k-${package_name}" cleanup_id package_directory # namerefs
@ -138,7 +139,7 @@ function create_kernel_deb() {
#display_alert "Package dir" "for package ${package_name}" "debug"
#run_host_command_logged tree -C -h -d --du "${package_directory}"
fakeroot_dpkg_deb_build "${package_directory}"
fakeroot_dpkg_deb_build "${package_directory}" "${artifact_deb_id}"
done_with_temp_dir "${cleanup_id}" # changes cwd to "${SRC}" and fires the cleanup function early
}
@ -228,8 +229,8 @@ function kernel_package_callback_linux_image() {
# Clean up symlinks in lib/modules/${kernel_version_family}/build and lib/modules/${kernel_version_family}/source; will be in the headers package
run_host_command_logged rm -v -f "${package_directory}/lib/modules/${kernel_version_family}/build" "${package_directory}/lib/modules/${kernel_version_family}/source"
display_alert "Showing contents of Kbuild produced modules" "linux-image" "debug"
if [[ -d "${package_directory}/lib/modules/${kernel_version_family}/kernel" ]]; then
display_alert "Showing contents of Kbuild produced modules" "linux-image" "debug"
run_host_command_logged tree -C --du -h -d -L 1 "${package_directory}/lib/modules/${kernel_version_family}/kernel" "|| true" # do not fail
fi
@ -246,14 +247,16 @@ function kernel_package_callback_linux_image() {
Package: ${package_name}
Version: ${artifact_version}
Source: linux-${kernel_version}
Armbian-Kernel-Version: ${kernel_version}
Armbian-Kernel-Version-Family: ${kernel_version_family}
Architecture: ${ARCH}
Maintainer: ${MAINTAINER} <${MAINTAINERMAIL}>
Section: kernel
Priority: optional
Provides: linux-image, linux-image-armbian, armbian-$BRANCH
Description: Armbian Linux $BRANCH kernel image ${artifact_version_reason:-"${kernel_version_family}"}
This package contains the Linux kernel, modules and corresponding other
files, kernel_version_family: $kernel_version_family.
Description: Armbian Linux $BRANCH kernel image $kernel_version_family
This package contains the Linux kernel, modules and corresponding other files.
${artifact_version_reason:-"${kernel_version_family}"}
CONTROL_FILE
# Install the maintainer scripts
@ -328,8 +331,9 @@ function kernel_package_callback_linux_dtb() {
Architecture: ${ARCH}
Priority: optional
Provides: linux-dtb, linux-dtb-armbian, armbian-$BRANCH
Description: Armbian Linux $BRANCH DTBs ${artifact_version_reason:-"${kernel_version_family}"} in /boot/dtb-${kernel_version_family}
This package contains device blobs from the Linux kernel, version ${kernel_version_family}
Description: Armbian Linux $BRANCH DTBs in /boot/dtb-${kernel_version_family}
This package contains device tree blobs from the Linux kernel, version ${kernel_version_family}
${artifact_version_reason:-"${kernel_version_family}"}
CONTROL_FILE
kernel_package_hook_helper "preinst" <(
@ -464,10 +468,11 @@ function kernel_package_callback_linux_headers() {
Priority: optional
Provides: linux-headers, linux-headers-armbian, armbian-$BRANCH
Depends: make, gcc, libc6-dev, bison, flex, libssl-dev, libelf-dev
Description: Armbian Linux $BRANCH headers ${artifact_version_reason:-"${kernel_version_family}"}
Description: Armbian Linux $BRANCH headers ${kernel_version_family}
This package provides kernel header files for ${kernel_version_family}
.
This is useful for DKMS and building of external modules.
${artifact_version_reason:-"${kernel_version_family}"}
CONTROL_FILE
# Make sure the target dir is clean/not-existing before installing.

View File

@ -68,7 +68,7 @@ compile_armbian-config() {
ln -sf /usr/sbin/armbian-config "${tmp_dir}/${armbian_config_dir}"/usr/bin/armbian-config
ln -sf /usr/sbin/softy "${tmp_dir}/${armbian_config_dir}"/usr/bin/softy
fakeroot_dpkg_deb_build "${tmp_dir}/${armbian_config_dir}"
fakeroot_dpkg_deb_build "${tmp_dir}/${armbian_config_dir}" "armbian-config"
done_with_temp_dir "${cleanup_id}" # changes cwd to "${SRC}" and fires the cleanup function early

View File

@ -58,7 +58,7 @@ function compile_armbian-desktop() {
eval "${AGGREGATED_DESKTOP_CREATE_DESKTOP_PACKAGE}"
display_alert "Running desktop-specific aggregated prepare script" "AGGREGATED_DESKTOP_CREATE_DESKTOP_PACKAGE" "debug"
fakeroot_dpkg_deb_build "${destination}"
fakeroot_dpkg_deb_build "${destination}" "armbian-desktop"
done_with_temp_dir "${cleanup_id}" # changes cwd to "${SRC}" and fires the cleanup function early
}

View File

@ -55,7 +55,7 @@ compile_armbian-plymouth-theme() {
run_host_command_logged cp "${SRC}"/packages/plymouth-theme-armbian/armbian.plymouth \
"${tmp_dir}/${plymouth_theme_armbian_dir}"/usr/share/plymouth/themes/armbian/
fakeroot_dpkg_deb_build "${tmp_dir}/${plymouth_theme_armbian_dir}"
fakeroot_dpkg_deb_build "${tmp_dir}/${plymouth_theme_armbian_dir}" "armbian-plymouth-theme"
done_with_temp_dir "${cleanup_id}" # changes cwd to "${SRC}" and fires the cleanup function early
}

View File

@ -80,7 +80,7 @@ compile_armbian-zsh() {
chmod 755 "${tmp_dir}/${armbian_zsh_dir}"/DEBIAN/postinst
fakeroot_dpkg_deb_build "${tmp_dir}/${armbian_zsh_dir}"
fakeroot_dpkg_deb_build "${tmp_dir}/${armbian_zsh_dir}" "armbian-zsh"
done_with_temp_dir "${cleanup_id}" # changes cwd to "${SRC}" and fires the cleanup function early
}

View File

@ -43,9 +43,7 @@ function compile_fake_ubuntu_advantage_tools() {
END
chmod 755 DEBIAN/postinst
cd "${fw_temp_dir}" || exit_with_error "can't change directory"
fakeroot_dpkg_deb_build "fake_ubuntu_advantage_tools"
fakeroot_dpkg_deb_build "${fw_temp_dir}/${fw_dir}" "fake-ubuntu-advantage-tools"
done_with_temp_dir "${cleanup_id}" # changes cwd to "${SRC}" and fires the cleanup function early
}

View File

@ -71,7 +71,7 @@ function compile_firmware() {
cd "${fw_temp_dir}" || exit_with_error "can't change directory"
fakeroot_dpkg_deb_build "armbian-firmware${FULL}"
fakeroot_dpkg_deb_build "armbian-firmware${FULL}" "armbian-firmware${FULL}"
done_with_temp_dir "${cleanup_id}" # changes cwd to "${SRC}" and fires the cleanup function early
}

View File

@ -7,49 +7,67 @@
# This file is a part of the Armbian Build Framework
# https://github.com/armbian/build/
# for RAW deb building. does a bunch of magic to "DEBIAN" directory. Argument is the open package directory. Target is always artifact_base_dir
# for RAW deb building. does a bunch of magic to "DEBIAN" directory. Arguments are the open package directory and the artifact_deb_id
function fakeroot_dpkg_deb_build() {
# check artifact_name and artifact_version is set otherwise exit_with_error
[[ -z "${artifact_name}" ]] && exit_with_error "fakeroot_dpkg_deb_build: artifact_name is not set"
[[ -z "${artifact_version}" ]] && exit_with_error "fakeroot_dpkg_deb_build: artifact_version is not set"
[[ -z "${artifact_base_dir}" ]] && exit_with_error "fakeroot_dpkg_deb_build: artifact_base_dir is not set"
[[ -z "${artifact_deb_repo}" ]] && exit_with_error "fakeroot_dpkg_deb_build: artifact_deb_repo is not set"
display_alert "Building .deb package" "${artifact_name}: $*" "debug"
declare first_arg="${1}"
declare package_directory="${1}"
declare artifact_deb_id="${2}"
if [[ ! -d "${first_arg}" ]]; then
exit_with_error "fakeroot_dpkg_deb_build: can't find source package directory: ${first_arg}"
if [[ ! -d "${package_directory}" ]]; then
exit_with_error "fakeroot_dpkg_deb_build: can't find source package directory: ${package_directory}"
fi
# Get the basename of the dir
declare pkg_name
pkg_name=$(basename "${first_arg}")
# Check artifact_deb_id is set and not empty
if [[ -z "${artifact_deb_id}" ]]; then
exit_with_error "fakeroot_dpkg_deb_build: artifact_deb_id (2nd parameter) is not set, called with package_directory: '${package_directory}'"
fi
# Obtain from the globals
declare -A -g artifact_map_packages
declare -A -g artifact_map_debs
debug_dict artifact_map_packages
debug_dict artifact_map_debs
declare artifact_deb_package="${artifact_map_packages[${artifact_deb_id}]}"
declare artifact_deb_rel_path="${artifact_map_debs[${artifact_deb_id}]}"
# If either is empty, bomb
if [[ -z "${artifact_deb_package}" ]]; then
exit_with_error "fakeroot_dpkg_deb_build: artifact_deb_package (artifact_map_packages) is not set or found for '${artifact_deb_id}'"
fi
if [[ -z "${artifact_deb_rel_path}" ]]; then
exit_with_error "fakeroot_dpkg_deb_build: artifact_deb_rel_path (artifact_map_debs) is not set or found for '${artifact_deb_id}'"
fi
# Show the total human size of the source package directory.
display_alert "Source package size" "${first_arg}: $(du -sh "${first_arg}" | cut -f1)" "debug"
display_alert "Source package size" "${package_directory}: $(du -sh "${package_directory}" | cut -f1)" "debug"
# Lets fix all packages with Installed-Size:
# get the size of the package in bytes
declare -i pkg_size_bytes
pkg_size_bytes=$(du -s -b "${first_arg}" | cut -f1)
pkg_size_bytes=$(du -s -b "${package_directory}" | cut -f1)
# edit DEBIAN/control, removed any Installed-Size: line
sed -i '/^Installed-Size:/d' "${first_arg}/DEBIAN/control"
sed -i '/^Installed-Size:/d' "${package_directory}/DEBIAN/control"
# add the new Installed-Size: line. The disk space is given as the integer value of the estimated installed size in bytes, divided by 1024 and rounded up.
declare -i installed_size
installed_size=$(((pkg_size_bytes + 1023) / 1024))
echo "Installed-Size: ${installed_size}" >> "${first_arg}/DEBIAN/control"
echo "Installed-Size: ${installed_size}" >> "${package_directory}/DEBIAN/control"
# Lets create DEBIAN/md5sums, for all the files in ${first_arg}. Do not include the paths in the md5sums file. Don't include the DEBIAN/* files.
find "${first_arg}" -type f -not -path "${first_arg}/DEBIAN/*" -print0 | xargs -0 md5sum | sed "s|${first_arg}/||g" > "${first_arg}/DEBIAN/md5sums"
# Lets create DEBIAN/md5sums, for all the files in ${package_directory}. Do not include the paths in the md5sums file. Don't include the DEBIAN/* files.
find "${package_directory}" -type f -not -path "${package_directory}/DEBIAN/*" -print0 | xargs -0 md5sum | sed "s|${package_directory}/||g" > "${package_directory}/DEBIAN/md5sums"
# Parse the DEBIAN/control and get the real package name...
declare control_package_name
control_package_name=$(grep -E "^Package:" "${first_arg}/DEBIAN/control" | cut -d' ' -f2)
control_package_name=$(grep -E "^Package:" "${package_directory}/DEBIAN/control" | cut -d' ' -f2)
# generate minimal DEBIAN/changelog
cat <<- EOF > "${first_arg}"/DEBIAN/changelog
${control_package_name} (${artifact_version}) armbian-repo-name; urgency=low
cat <<- EOF > "${package_directory}"/DEBIAN/changelog
${control_package_name} (${artifact_version}) ${artifact_deb_repo}; urgency=low
* Initial changelog entry for ${control_package_name} package hash ${artifact_version}
@ -57,38 +75,42 @@ function fakeroot_dpkg_deb_build() {
EOF
# Also a usr/share/doc/${control_package_name}/changelog.gz
mkdir -p "${first_arg}/usr/share/doc/${control_package_name}"
gzip -9 -c "${first_arg}/DEBIAN/changelog" > "${first_arg}/usr/share/doc/${control_package_name}/changelog.gz"
mkdir -p "${package_directory}/usr/share/doc/${control_package_name}"
gzip -9 -c "${package_directory}/DEBIAN/changelog" > "${package_directory}/usr/share/doc/${control_package_name}/changelog.gz"
# find the DEBIAN scripts (postinst, prerm, etc) and run shellcheck on them.
dpkg_deb_run_shellcheck_on_scripts "${first_arg}"
dpkg_deb_run_shellcheck_on_scripts "${package_directory}"
# Debug, dump the generated postrm/preinst/postinst
if [[ "${SHOW_DEBUG}" == "yes" || "${SHOW_DEBIAN}" == "yes" ]]; then
# Dump the CONTROL file to the log
run_tool_batcat --file-name "${artifact_name}/DEBIAN/control" "${first_arg}/DEBIAN/control"
run_tool_batcat --file-name "${artifact_name}/DEBIAN/control" "${package_directory}/DEBIAN/control"
if [[ -f "${first_arg}/DEBIAN/changelog" ]]; then
run_tool_batcat --file-name "${artifact_name}/DEBIAN/changelog" "${first_arg}/DEBIAN/changelog"
if [[ -f "${package_directory}/DEBIAN/changelog" ]]; then
run_tool_batcat --file-name "${artifact_name}/DEBIAN/changelog" "${package_directory}/DEBIAN/changelog"
fi
if [[ -f "${first_arg}/DEBIAN/postrm" ]]; then
run_tool_batcat --file-name "${artifact_name}/DEBIAN/postrm.sh" "${first_arg}/DEBIAN/postrm"
if [[ -f "${package_directory}/DEBIAN/postrm" ]]; then
run_tool_batcat --file-name "${artifact_name}/DEBIAN/postrm.sh" "${package_directory}/DEBIAN/postrm"
fi
if [[ -f "${first_arg}/DEBIAN/preinst" ]]; then
run_tool_batcat --file-name "${artifact_name}/DEBIAN/preinst.sh" "${first_arg}/DEBIAN/preinst"
if [[ -f "${package_directory}/DEBIAN/preinst" ]]; then
run_tool_batcat --file-name "${artifact_name}/DEBIAN/preinst.sh" "${package_directory}/DEBIAN/preinst"
fi
if [[ -f "${first_arg}/DEBIAN/postinst" ]]; then
run_tool_batcat --file-name "${artifact_name}/DEBIAN/postinst.sh" "${first_arg}/DEBIAN/postinst"
if [[ -f "${package_directory}/DEBIAN/postinst" ]]; then
run_tool_batcat --file-name "${artifact_name}/DEBIAN/postinst.sh" "${package_directory}/DEBIAN/postinst"
fi
run_tool_batcat --file-name "${artifact_name}/DEBIAN/md5sums" "${first_arg}/DEBIAN/md5sums"
run_tool_batcat --file-name "${artifact_name}/DEBIAN/md5sums" "${package_directory}/DEBIAN/md5sums"
fi
mkdir -p "${artifact_base_dir}"
run_host_command_logged_raw fakeroot dpkg-deb -b "-Z${DEB_COMPRESS}" "${first_arg}" "${artifact_base_dir}/"
declare deb_final_filename="${PACKAGES_HASHED_STORAGE}/${artifact_deb_rel_path}"
declare deb_final_dir
deb_final_dir=$(dirname "${deb_final_filename}")
mkdir -p "${deb_final_dir}"
run_host_command_logged_raw fakeroot dpkg-deb -b "-Z${DEB_COMPRESS}" "${package_directory}" "${deb_final_filename}"
}
function dpkg_deb_run_shellcheck_on_scripts() {

View File

@ -414,7 +414,8 @@ function compile_uboot() {
Provides: armbian-u-boot
Replaces: armbian-u-boot
Conflicts: armbian-u-boot, u-boot-sunxi
Description: Das U-Boot for ${BOARD} ${artifact_version_reason:-"${version}"}
Description: Das U-Boot for ${BOARD}
${artifact_version_reason:-"${version}"}
EOF
# copy license files, config, etc.
@ -424,7 +425,7 @@ function compile_uboot() {
[[ -n $atftempdir && -f $atftempdir/license.md ]] && run_host_command_logged cp "${atftempdir}/license.md" "$uboottempdir/usr/lib/u-boot/LICENSE.atf"
display_alert "Building u-boot deb" "(version: ${artifact_version})"
fakeroot_dpkg_deb_build "$uboottempdir"
fakeroot_dpkg_deb_build "$uboottempdir" "uboot"
[[ -n $atftempdir ]] && rm -rf "${atftempdir:?}" # @TODO: intricate cleanup; u-boot's pkg uses ATF's tempdir...

View File

@ -93,9 +93,9 @@ function create_image_from_sdcard_rootfs() {
display_alert "Mount point" "$(echo -e "$freespace" | awk -v mp="${MOUNT}" '$6==mp {print $5}')" "info"
# stage: write u-boot, unless BOOTCONFIG=none
declare -g -A image_artifacts_debs
declare -g -A image_artifacts_debs_reversioned
if [[ "${BOOTCONFIG}" != "none" ]]; then
write_uboot_to_loop_image "${LOOP}" "${DEB_STORAGE}/${image_artifacts_debs["uboot"]}"
write_uboot_to_loop_image "${LOOP}" "${DEB_STORAGE}/${image_artifacts_debs_reversioned["uboot"]}"
fi
# fix wrong / permissions

View File

@ -93,11 +93,14 @@ function main_default_build_packages() {
declare -g -a image_artifacts_all=()
declare -g -A image_artifacts_packages=()
declare -g -A image_artifacts_packages_version=()
declare -g -A image_artifacts_packages_version_reversioned=()
declare -g -A image_artifacts_debs=()
declare -g -A image_artifacts_debs_reversioned=()
declare one_artifact one_artifact_package
for one_artifact in "${artifacts_to_build[@]}"; do
declare -A artifact_map_packages=()
declare -A artifact_map_debs=()
declare -A artifact_map_debs_reversioned=()
declare artifact_version
WHAT="${one_artifact}" build_artifact_for_image
@ -107,7 +110,9 @@ function main_default_build_packages() {
image_artifacts_all+=("${one_artifact_package}")
image_artifacts_packages["${one_artifact_package}"]="${artifact_map_packages[${one_artifact_package}]}"
image_artifacts_debs["${one_artifact_package}"]="${artifact_map_debs[${one_artifact_package}]}"
image_artifacts_debs_reversioned["${one_artifact_package}"]="${artifact_map_debs_reversioned[${one_artifact_package}]}"
image_artifacts_packages_version["${artifact_map_packages[${one_artifact_package}]}"]="${artifact_version}"
image_artifacts_packages_version_reversioned["${artifact_map_packages[${one_artifact_package}]}"]="${artifact_final_version_reversioned}"
done
done

View File

@ -102,15 +102,15 @@ function list_installed_packages() {
declare -g -A image_artifacts_packages_version # global scope, set in main_default_build_packages()
declare pkg_name pkg_wanted_version
for pkg_name in "${!image_artifacts_packages_version[@]}"; do
[[ "${pkg_name}" =~ ^linux-headers ]] && continue # linux-headers is a special case, since its always built with kernel, but not always installed (deb-tar)
pkg_wanted_version="${image_artifacts_packages_version[${pkg_name}]}"
[[ "${pkg_name}" =~ ^linux-headers ]] && continue # linux-headers is a special case, since its always built with kernel, but not always installed (deb-tar)
pkg_wanted_version="${image_artifacts_packages_version[${pkg_name}]}" # this is the hash-version
display_alert "Checking installed version of package" "${pkg_name}=${pkg_wanted_version}" "debug"
declare actual_version
actual_version=$(chroot "${SDCARD}" dpkg-query -W -f='${Status} ${Package} ${Version}\n' "${pkg_name}" | grep "install ok installed" | cut -d " " -f 5)
actual_version=$(chroot "${SDCARD}" dpkg-query -W -f='${Status} ${Package} ${Armbian-Original-Hash}\n' "${pkg_name}" | grep "install ok installed" | cut -d " " -f 5)
if [[ "${actual_version}" != "${pkg_wanted_version}" ]]; then
display_alert "Installed version of package does not match wanted version. Check for inconsistent repo, customize.sh/hooks, extensions, or upgrades installing wrong version" "${pkg_name} :: actual:'${actual_version}' wanted:'${pkg_wanted_version}'" "warn"
display_alert "Installed hash of package does not match wanted hash. Check for inconsistent repo, customize.sh/hooks, extensions, or upgrades installing wrong version" "${pkg_name} :: actual:'${actual_version}' wanted:'${pkg_wanted_version}'" "warn"
else
display_alert "Image installed package version" "${pkg_name} = ${actual_version}" "info"
display_alert "Image installed package hash" "${pkg_name} = ${actual_version}" "info"
fi
done
}

View File

@ -109,3 +109,14 @@ function install_deb_chroot() {
# IMPORTANT! Do not use short-circuit above as last statement in a function, since it determines the result of the function.
return 0
}
function install_artifact_deb_chroot() {
declare deb_name="$1"
declare -A -g image_artifacts_debs_reversioned # global associative array
declare revisioned_deb_rel_path="${image_artifacts_debs_reversioned["${deb_name}"]}"
if [[ -z "${revisioned_deb_rel_path}" ]]; then
exit_with_error "No revisioned deb path found for '${deb_name}'"
fi
display_alert "Installing artifact deb" "${deb_name} :: ${revisioned_deb_rel_path}" "debug"
install_deb_chroot "${DEB_STORAGE}/${revisioned_deb_rel_path}"
}

View File

@ -265,11 +265,11 @@ function install_distribution_agnostic() {
# install u-boot
# @TODO: add install_bootloader() extension method, refactor into u-boot extension
declare -g image_artifacts_packages image_artifacts_debs
declare -g image_artifacts_packages image_artifacts_debs_reversioned
debug_dict image_artifacts_packages
debug_dict image_artifacts_debs
debug_dict image_artifacts_debs_reversioned
if [[ "${BOOTCONFIG}" != "none" ]]; then
install_deb_chroot "${DEB_STORAGE}/${image_artifacts_debs["uboot"]}"
install_artifact_deb_chroot "uboot"
fi
call_extension_method "pre_install_kernel_debs" <<- 'PRE_INSTALL_KERNEL_DEBS'
@ -282,20 +282,20 @@ function install_distribution_agnostic() {
# install kernel: image/dtb/headers
if [[ -n $KERNELSOURCE ]]; then
install_deb_chroot "${DEB_STORAGE}/${image_artifacts_debs["linux-image"]}"
install_artifact_deb_chroot "linux-image"
if [[ "${KERNEL_BUILD_DTBS:-"yes"}" == "yes" ]]; then
install_deb_chroot "${DEB_STORAGE}/${image_artifacts_debs["linux-dtb"]}"
install_artifact_deb_chroot "linux-dtb"
fi
if [[ "${KERNEL_HAS_WORKING_HEADERS:-"no"}" == "yes" ]]; then
if [[ $INSTALL_HEADERS == yes ]]; then # @TODO remove? might be a good idea to always install headers.
install_deb_chroot "${DEB_STORAGE}/${image_artifacts_debs["linux-headers"]}"
install_artifact_deb_chroot "linux-headers"
fi
fi
# Determine "IMAGE_INSTALLED_KERNEL_VERSION" for compatiblity with legacy update-initramfs code. @TODO get rid of this one day
IMAGE_INSTALLED_KERNEL_VERSION=$(dpkg --info "${DEB_STORAGE}/${image_artifacts_debs["linux-image"]}" | grep "^ Source:" | sed -e 's/ Source: linux-//')
IMAGE_INSTALLED_KERNEL_VERSION=$(dpkg --info "${DEB_STORAGE}/${image_artifacts_debs_reversioned["linux-image"]}" | grep "^ Source:" | sed -e 's/ Source: linux-//')
display_alert "Parsed kernel version from local package" "${IMAGE_INSTALLED_KERNEL_VERSION}" "debug"
fi
@ -308,19 +308,19 @@ function install_distribution_agnostic() {
# install armbian-firmware by default. Set BOARD_FIRMWARE_INSTALL="-full" to install full firmware variant
if [[ "${INSTALL_ARMBIAN_FIRMWARE:-yes}" == "yes" ]]; then
if [[ ${BOARD_FIRMWARE_INSTALL:-""} == "-full" ]]; then
install_deb_chroot "${DEB_STORAGE}/${image_artifacts_debs["armbian-firmware-full"]}"
install_artifact_deb_chroot "armbian-firmware-full"
else
install_deb_chroot "${DEB_STORAGE}/${image_artifacts_debs["armbian-firmware"]}"
install_artifact_deb_chroot "armbian-firmware"
fi
fi
# install board support packages
install_deb_chroot "${DEB_STORAGE}/${image_artifacts_debs["armbian-bsp-cli"]}"
install_artifact_deb_chroot "armbian-bsp-cli"
# install armbian-desktop
if [[ $BUILD_DESKTOP == yes ]]; then
install_deb_chroot "${DEB_STORAGE}/${image_artifacts_debs["armbian-desktop"]}"
install_deb_chroot "${DEB_STORAGE}/${image_artifacts_debs["armbian-bsp-desktop"]}"
install_artifact_deb_chroot "armbian-desktop"
install_artifact_deb_chroot "armbian-bsp-desktop"
# install display manager and PACKAGE_LIST_DESKTOP_FULL packages if enabled per board
desktop_postinstall
fi
@ -328,27 +328,27 @@ function install_distribution_agnostic() {
# install armbian-config
if [[ "${PACKAGE_LIST_RM}" != *armbian-config* ]]; then
if [[ $BUILD_MINIMAL != yes ]]; then
install_deb_chroot "${DEB_STORAGE}/${image_artifacts_debs["armbian-config"]}"
install_artifact_deb_chroot "armbian-config"
fi
fi
# install armbian-zsh
if [[ "${PACKAGE_LIST_RM}" != *armbian-zsh* ]]; then
if [[ $BUILD_MINIMAL != yes ]]; then
install_deb_chroot "${DEB_STORAGE}/${image_artifacts_debs["armbian-zsh"]}"
install_artifact_deb_chroot "armbian-zsh"
fi
fi
# install armbian-plymouth-theme
if [[ $PLYMOUTH == yes ]]; then
install_deb_chroot "${DEB_STORAGE}/${image_artifacts_debs["armbian-plymouth-theme"]}"
install_artifact_deb_chroot "armbian-plymouth-theme"
else
chroot_sdcard_apt_get_remove --auto-remove plymouth
fi
# install wireguard tools
if [[ $WIREGUARD == yes ]]; then
install_deb_chroot "wireguard-tools" "remote"
install_deb_chroot "wireguard-tools" "remote" # @TODO: move this to some image pkg list in config
fi
# freeze armbian packages

View File

@ -51,8 +51,7 @@ function install_distribution_specific() {
disable_systemd_service_sdcard ondemand.service
# Remove Ubuntu APT spamming
declare -g -A image_artifacts_debs
install_deb_chroot "${DEB_STORAGE}/${image_artifacts_debs["fake-ubuntu-advantage-tools"]}"
install_artifact_deb_chroot "fake-ubuntu-advantage-tools"
truncate --size=0 "${SDCARD}"/etc/apt/apt.conf.d/20apt-esm-hook.conf
;;
@ -60,7 +59,7 @@ function install_distribution_specific() {
# install our base-files package (this replaces the original from Debian/Ubuntu)
if [[ "${KEEP_ORIGINAL_OS_RELEASE:-"no"}" != "yes" ]]; then
install_deb_chroot "${DEB_STORAGE}/${image_artifacts_debs["armbian-base-files"]}"
install_artifact_deb_chroot "armbian-base-files"
fi
# Basic Netplan config. Let NetworkManager/networkd manage all devices on this system

View File

@ -136,6 +136,15 @@ set -o errexit ## set -e : exit the script if any statement returns a non-true
# shellcheck source=lib/functions/artifacts/artifacts-registry.sh
source "${SRC}"/lib/functions/artifacts/artifacts-registry.sh
# no errors tolerated. invoked before each sourced file to make sure.
#set -o pipefail # trace ERR through pipes - will be enabled "soon"
#set -o nounset ## set -u : exit the script if you try to use an uninitialised variable - one day will be enabled
set -o errtrace # trace ERR through - enabled
set -o errexit ## set -e : exit the script if any statement returns a non-true return value - enabled
### lib/functions/artifacts/artifacts-reversion.sh
# shellcheck source=lib/functions/artifacts/artifacts-reversion.sh
source "${SRC}"/lib/functions/artifacts/artifacts-reversion.sh
# no errors tolerated. invoked before each sourced file to make sure.
#set -o pipefail # trace ERR through pipes - will be enabled "soon"
#set -o nounset ## set -u : exit the script if you try to use an uninitialised variable - one day will be enabled