Commit Graph

73 Commits

Author SHA1 Message Date
Ricardo Pardini
03dd0d623b
armbian-next: debs: introduce shellcheck for DEBIAN/postinst and others, before dpkg build 2023-02-18 07:45:50 -03:00
Ricardo Pardini
f0d3ea7961
armbian-next: tooling: introduce helper/downloader for shellcheck; include in Dockerfile 2023-02-18 07:45:48 -03:00
Ricardo Pardini
871f3cfcee
armbian-next: tooling: include ORAS tools in Dockerfile; use them if available instead of downloading 2023-02-18 07:45:47 -03:00
Ricardo Pardini
da02fde9cc
armbian-next: git: backport safe.directory method from master (just add *, once) -- thanks Igor 2023-02-18 07:45:44 -03:00
Ricardo Pardini
bf53bd2988
armbian-next: tmpfs: use size=99% for all tmpfs's; (rootfs/WORKDIR/chroot); thanks Igor for suggestion 2023-02-18 07:45:42 -03:00
Ricardo Pardini
3c79aaa79f
armbian-next: hash/cache: remove some cruft, some better logging 2023-02-18 07:45:39 -03:00
Ricardo Pardini
555a972442
armbian-next: git ref2info: dereference annotated tags to get "real" commit SHA1; fail if curl failed; validate parse_makefile_version() 2023-02-18 07:45:38 -03:00
Ricardo Pardini
68d76d6f3a
armbian-next: squash some shellcheck warnings; don't local and assign subshell value in same statement, that masks errors (declare/local almost-always works) 2023-02-18 07:45:31 -03:00
Ricardo Pardini
6258ea566a
armbian-next: shellcheck: squash a bunch of 'critical' warnings 2023-02-18 07:45:29 -03:00
Ricardo Pardini
bef7f8fee4
armbian-next: artifacts: introduce kernel/u-boot artifacts; git ref2info; "memoizer"; some hashing
- artifacts: u-boot/kernel - pt7 - adapt legacy/artifact versions; use common `capture_rename_legacy_debs_into_artifacts()`
- artifacts: u-boot - pt6: add artifact for u-boot
  - use artifact version / reason in actual u-boot .deb if present
- artifacts: kernel - pt5: tune kernel version, refactor
- artifacts: kernel - pt4: squash unrelated bugs that show up; move `prepare_compilation_vars()` to default build
- artifacts: kernel - pt3: drivers+patches+.config hashing
  - split file hashing function from drivers-harness; fix it so filenames are relative and sorted; sort from ${SRC}, always
  - aplit prepare_kernel_config_core_or_userpatches() from `kernel_config_initialize()`
- artifacts: kernel - pt2:  memoizing git ref2info
- artifacts: kernel - pt1:  versioning
2023-02-18 07:45:20 -03:00
Ricardo Pardini
19aa7e95cf
armbian-next: introduce assert_prepared_host() and assert before using any Python, to un-mask any further fallacies 2023-02-18 07:45:19 -03:00
Ricardo Pardini
44733e1afc
armbian-next: git: refactor git_parse_ref() out of fetch_from_repo() for reuse 2023-02-18 07:45:12 -03:00
Ricardo Pardini
8574b87f8d
armbian-next: curb some logging for git non-checkouts 2023-02-18 07:45:06 -03:00
Ricardo Pardini
5fa7bf735b
armbian-next: git: fetch_from_repo(): add fetched_revision(_ts) as well as checked_out_revision(_ts); introduce param do_checkout=no to only fetch, not checkout 2023-02-18 07:45:01 -03:00
Ricardo Pardini
9849990031
armbian-next: all chroots: mount /var/tmp as tmpfs; create dir /run/user/0 and mount as tmpfs @TODO cleanups 2023-02-18 07:44:55 -03:00
Ricardo Pardini
1cec42392e
armbian-next: GHA & rootfs adventures, pt 1
- rootfs: create readonly global `rootfs_cache_id` (type+hash+date/version)
- add "oras-upload" CLI command (dumb, no retries, single target/single source)
- docker: mount-bind for & re-pass envs `GITHUB_OUTPUT` and `GITHUB_STEP_SUMMARY` down to Docker
- introduce `github-actions.sh::github_actions_add_output()`
- during logs cleanup, dump the Markdown log into GITHUB_STEP_SUMMARY if it is available
2023-02-18 07:44:41 -03:00
Ricardo Pardini
8a63a0ae20
armbian-next: downloads.sh: some cleanup and add debugging to get_urls() 2023-02-18 07:44:17 -03:00
Ricardo Pardini
18c9946a4d
armbian-next: countdown: if to test -t 0 (0: stdin, 1:stdout) for interactive-ness; also test in if-not-aborted version 2023-02-18 07:44:11 -03:00
Ricardo Pardini
8d2be2c694
armbian-next: python/pip: solve crazy with /usr/bin/pip3 by not using it (instead, python3 -m pip); don't mask errors
- "| cut" masked errors in multiple places
- include real pip3 version in python deps hash, all will be invalidated
- be less happy about hitting hash/cache, should happen often
2023-02-18 07:44:09 -03:00
Ricardo Pardini
eb30aac310
armbian-next: extensions: move 'extensions.sh' into library; replace manual source with init function
- wraps globals in `extension_manager_declare_globals()` -- and make them actually global
- whoever wrote this original code is insane, add copyright
2023-02-18 07:44:05 -03:00
Ricardo Pardini
da77aad1dd
armbian-next: git: detect, and stop, when git bare trees vs worktrees have been shuffled around and can't be recovered
- hopefully this will never happen ;-)
2023-02-18 07:43:58 -03:00
Ricardo Pardini
88ff66ae11
armbian-next: adaptive shallow/full Kernel git bare git (3gb -> 300mb download)
- allows using pre-shallow-ed bare trees, specific to the KERNEL_MAJOR_MINOR
- uses shallow by default if storage is "slow" (mmc), or free disk space < 32g (@TODO tweak this)
- if full tree already there keeps using it
- allow forcing decision with `KERNEL_GIT=full` or `KERNEL_GIT=shallow`
- countdown to confirm if decision based on storage type/free space and user is interactive
  - this has some of the worst copy ever written
- @TODO refactor the device type / free space code, reuse across codebase (3-4 other spots detect free space)
- @TODO GHA self-hosted (full) vs hosted (shallow) runners decision
- sprinkle with `wait_for_disk_sync()` around clean/download/extract
- sprinkle git with debugs, and an info between fetch and checkout (otherwise it seems "fetch" takes a long time)
2023-02-18 07:43:54 -03:00
Ricardo Pardini
72c61f2f68
armbian-next: replace bare sync invocations by introducing wait_for_disk_sync(); sync early at start of build
- wait 10s for sync call; otherwise warn user to be patient
- wait_for_sync "reason for sync" everywhere
- add @TODO for /run/user/0 etc and XDG_RUNTIME_DIR in chroot
2023-02-18 07:43:53 -03:00
Ricardo Pardini
4d43c05f7b
armbian-next: kernel: git: remove dead code, GIT_PRE_FETCH_HOOK / checked_out_revision_mtime 2023-02-18 07:43:52 -03:00
Ricardo Pardini
3eacae68c7
armbian-next: kernel: git: split via-ORAS and via-Bundle tangle 2023-02-18 07:43:51 -03:00
Ricardo Pardini
8af14d4f8c
armbian-next: countdown: split off file for countdown functions (countdown-to-abort and countdown-to-continue)
- introduce `countdown.sh`, move `traps.sh::exit_if_countdown_not_aborted()` there, introduce `exit_if_countdown_not_aborted()`
2023-02-18 07:43:50 -03:00
Ricardo Pardini
549ae08061
armbian-next: git: show "fetch completed" info, lest user thinks fetch is hanging while checking out 2023-02-18 07:43:46 -03:00
Ricardo Pardini
be49433b41
armbian-next: use retries for downloading ORAS tooling
- `do_with_retries()`: add `IS_A_RETRY` and `RETRY_FMT_MORE_THAN_ONCE` for convenient logging in retried functions
2023-02-18 07:43:35 -03:00
Ricardo Pardini
1636a2afea
armbian-next: tune logging: unmounting, u-boot prep/source/install; traps; extensions 2023-02-18 07:43:25 -03:00
Ricardo Pardini
bf891d54d9
armbian-next: split prepare_host(), fix .tmp reset trap
- `prepare_host()`: split; do checks earlier and allow them to be interactive
- introduce `exit_if_countdown_not_aborted()` for "Low Disk Space" and other critical conditions
- split `prepare_host()` into interactive & non-interactive parts
- split off `clean_deprecated_mountpoints()` from prepare into `cleaning.sh`
- introduce and use `reset_uid_owner_non_recursive()` for `.tmp` reset in trap, to avoid disasters
- add more logging sections to default-build.sh, avoid unlogged parts
2023-02-18 07:43:10 -03:00
Ricardo Pardini
7beeae6219
armbian-next: Python tooling: use consolidated+hashed+cached pip base/pycache; don't pip during Dockerfile build, nor cli-requirements
- consolidate at `prepare_python_and_pip()`
- sanity check for Python version 3.9+ regardless of HOSTRELEASE
- TODO: pip vs sudo/root: need pip 22.2+ to curb warning, not doing it
2023-02-18 07:42:44 -03:00
Ricardo Pardini
53266feff8
armbian-next: kernel: deterministic .config mtime handling; kernel build logging sections reorg
- .config is now compared with previous one byte-wise, diff shown, and mtime preserved directly via `cp -p`
- this does away with the previous mtime-based BS I had written during the fasthash era (and thus remove file-mtime.sh, regen lib)
- split some functions, to better control the interactive piece in the _middle_ of kernel config process
- no more `kernel_config_maybe_interactive()`, now `kernel_config()` that controls its own logging sections
- extract `kernel_determine_toolchain()` out of config into make
- introduce internal hook `armbian_kernel_config()` that runs before custom version (for .config defaults, coming soon)
2023-02-18 07:42:39 -03:00
Ricardo Pardini
df058ea3f1
armbian-next: fix quoting of retry var (cosmetic)
- this commit is proof of OCD
2023-02-18 07:42:21 -03:00
Ricardo Pardini
7493ce1b09
armbian-next: long overdue split of logging.sh 2023-02-18 07:42:19 -03:00
Ricardo Pardini
1ed9dada4d
armbian-next: curb aria2c's complaining a bit 2023-02-18 07:42:17 -03:00
Ricardo Pardini
3563a5f1e2
armbian-next: curb down git fetch logging; be verbose if user on terminal & not logging 2023-02-18 07:42:16 -03:00
Ricardo Pardini
2a22cee345
armbian-next: curb down wget logging for ORAS tooling download 2023-02-18 07:42:15 -03:00
Ricardo Pardini
47af820f5d
armbian-oleg: junk: drastically reduce host-side dependencies / "remove junk"
- remove a lot of hopefully / hardly confirmed, unused dependencies
- don't use crossbuild-essential-xxx; avoid the c++ compiler that comes with it, install gcc-only instead
- hostdeps: use `libc6-dev make dpkg-dev gcc` (without `g++`) instead of `build-essential`
- drop `btrfs-progs` and `f2fs-tools`  (@TODO add in extension when/whereused)
- more: drop `cryptsetup` (@TODO add in extension when/whereused)
- don't be too quiet when doing apt-update for hostdeps
2023-02-18 07:42:14 -03:00
Ricardo Pardini
d4d3c1346c
armbian-oleg: split hostdeps again, full of ifs, and reduce the minimum set of pkgs for Oleg-conditions while trying to keep Docker full
- probably breaks a lot of stuff; c++ compilers, distcc, possibly others
- split into late_prepare_host_dependencies / early_prepare_host_dependencies calling adaptative_prepare_host_dependencies() full of if's

also:
- calc deps out of configuration
- reorg checks into obtain_and_check_host_release_and_arch()
2023-02-18 07:42:11 -03:00
Ricardo Pardini
8d55a3258b
armbian-next: [focal-host] don't use git's --initial-branch which is not supported on focal's git 2023-02-18 07:41:52 -03:00
Ricardo Pardini
2b54793899
armbian-next: [focal-host] run pip3 under the python3_binary_path (so coerce it to use 3.9 under focal) 2023-02-18 07:41:50 -03:00
Ricardo Pardini
6dda660fb6
armbian-next: [focal-host] try to support focal-like host OS's again, by avoiding git's --no-auto-maintenance 2023-02-18 07:41:48 -03:00
Ricardo Pardini
6cb12a3397
armbian-next: [focal-host] try to support focal-like host OS's again, via python3.9 (and drop buster completely)
- completely remove support for building under `buster` -- that's way too old, sorry.
- de-hardcode `python3` invocations, instead use `python3_binary_path` set by `prepare_python3_binary_for_python_tools()`
- juggle `$HOSTRELEASE`: read from actual host, or determined from Docker image name (during Dockerfile build)
- TL;DR: include and use `python3.9` for focal-like host OS's
2023-02-18 07:41:47 -03:00
Ricardo Pardini
8f3fd355eb
armbian-next: tune logging in multiple places, there was a lot of redundant/un-needed info's 2023-02-18 07:41:22 -03:00
Ricardo Pardini
12b31afa51
armbian-next: git: mark submodule's dirs as safe before recursive fetch_from_repo() 2023-02-18 07:41:15 -03:00
Ricardo Pardini
f679b81b10
armbian-next: git: use --recurse-submodules=no for fetch; fetch_from_repo() handles submodules separately
- also remove `du` working copy size debugging
2023-02-18 07:41:13 -03:00
Ricardo Pardini
9370c19a44
armbian-next: drop old code for patching & fasthash
- split the modification time stuff to general
2023-02-18 07:41:05 -03:00
Ricardo Pardini
143ba996ca
armbian-next: git: setup the worktree first before trying to ensure safe 2023-02-18 07:41:04 -03:00
Ricardo Pardini
85009a6431
armbian-next: git worktree: forcefully fix the worktree's reference to the bare repo, and vice-versa 2023-02-18 07:41:02 -03:00
Ricardo Pardini
e607e9e54e
armbian-next: remove spurious output.log redirections in downloads.sh; warn that download_and_verify() is unhandled in armbian-next 2023-02-18 07:41:01 -03:00