Commit Graph

109 Commits

Author SHA1 Message Date
Ricardo Pardini
bf53bd2988
armbian-next: tmpfs: use size=99% for all tmpfs's; (rootfs/WORKDIR/chroot); thanks Igor for suggestion 2023-02-18 07:45:42 -03:00
Ricardo Pardini
0e480e685e
armbian-next: docker: pass down git info via env var, since Docker doesn't get ${SRC}/.git, yet we need that info in the Docker logs 2023-02-18 07:45:40 -03:00
Ricardo Pardini
7431984025
armbian-next: docker: pass CARD_DEVICE to Docker via --device 2023-02-18 07:45:35 -03:00
Ricardo Pardini
6258ea566a
armbian-next: shellcheck: squash a bunch of 'critical' warnings 2023-02-18 07:45:29 -03:00
Ricardo Pardini
19aa7e95cf
armbian-next: introduce assert_prepared_host() and assert before using any Python, to un-mask any further fallacies 2023-02-18 07:45:19 -03:00
Ricardo Pardini
956ef9ac7d
armbian-next: fix: for the 1000th time had broken KERNEL_ONLY=yes; curb some warnings 2023-02-18 07:45:18 -03:00
Ricardo Pardini
281e3f0828
armbian-next: cli: try (and fail, probably...) to be helpful with half-working Docker installs 2023-02-18 07:45:16 -03:00
Ricardo Pardini
8329b2e970
armbian-next: docker: split docker_cli_build_dockerfile() into docker_cli_prepare_dockerfile(); introduce FAST_DOCKER
- armbian-next: docker: fix hostrelease crazy
2023-02-18 07:45:11 -03:00
Ricardo Pardini
fa666aa3d4
armbian-next: interactive: review -t 0 instead of -t 1; move non-interactive check_basic_host() from prepare-host to config-prepare, for consistency 2023-02-18 07:45:09 -03:00
Ricardo Pardini
41d7d0155e
armbian-next: docker: fix: not all find's are made equal, back to -mtime +1 (full 24-hour period needed for match) 2023-02-18 07:45:08 -03:00
Ricardo Pardini
b91d752aa6
armbian-next: prepare-host: don't manage ACNG in prepare if not asked to 2023-02-18 07:45:07 -03:00
Ricardo Pardini
941bc81868
armbian-next: docker: pass ARMBIAN_ENABLE_CALL_TRACING=yes via env, and only if DOCKER_ARMBIAN_ENABLE_CALL_TRACING=yes 2023-02-18 07:44:56 -03:00
Ricardo Pardini
962ae65a45
armbian-next: docker: include ${SRC}/.git static via Dockerfile, for kernel-patches-to-git / archeology; use fast archeology
- include FAST_ARCHEOLOGY in call to Python, default yes; sync kernel & u-boot call
2023-02-18 07:44:54 -03:00
Ricardo Pardini
6e267f5cd1
armbian-next: docker: passthrough of the SSH agent, via Docker magic on Darwin; enable for docker-shell and kernel-patches-to-git
- don't check/verify/update SSH-related hostkey etc for git tree push
2023-02-18 07:44:53 -03:00
Ricardo Pardini
bd0a7936da
armbian-next: docker/gha: pass down a bunch of GHA specific vars (list unconfirmed)
- docker: curb logging about stdin
2023-02-18 07:44:46 -03:00
Ricardo Pardini
6177b94268
armbian-next: logging: pure ASCII .log; included in Markdown for GHA 2023-02-18 07:44:45 -03:00
Ricardo Pardini
1cec42392e
armbian-next: GHA & rootfs adventures, pt 1
- rootfs: create readonly global `rootfs_cache_id` (type+hash+date/version)
- add "oras-upload" CLI command (dumb, no retries, single target/single source)
- docker: mount-bind for & re-pass envs `GITHUB_OUTPUT` and `GITHUB_STEP_SUMMARY` down to Docker
- introduce `github-actions.sh::github_actions_add_output()`
- during logs cleanup, dump the Markdown log into GITHUB_STEP_SUMMARY if it is available
2023-02-18 07:44:41 -03:00
Ricardo Pardini
7f74b2e83f
armbian-next: docker: don't spew errors about sharing cache if cache doesn't even exist 2023-02-18 07:44:40 -03:00
Ricardo Pardini
041297c563
armbian-next: binfmt: make sure we have all binfmt's setup & enabled for rootfs build (v2) 2023-02-18 07:44:39 -03:00
Ricardo Pardini
b82badd214
armbian-next: wait_for_disk_sync(): warn only if more than 30s fsync 2023-02-18 07:44:38 -03:00
Ricardo Pardini
78dcf01c92
armbian-next: cleanup, kill and add to-do's, remove dead code, turn down logging, squash future shortcircuit bugs
- killed `[[ $ROOTFS_TYPE != ext4 ]] && display_alert "Assuming ${BOARD} ${BRANCH} kernel supports ${ROOTFS_TYPE}" "" "wrn"`
  - which definitely didn't belong in rootfs
- disable usage of run_host_command_logged_long_running for kernel-make
- stop lying about long_running stuff being any different from non-long_running versions
- stop lying about 'set -e' when there's still a bunch of pipes for "pv" and stuff all around
2023-02-18 07:44:33 -03:00
Ricardo Pardini
3cac8f44a4
armbian-next: we don't really need target RELEASE defined to install host deps (for now), don't freak out about it 2023-02-18 07:44:31 -03:00
Ricardo Pardini
44ae761927
armbian-next: apt: expand local caching to include package lists, as well as .deb's
- `/var/cache/apt` in chroot was already cached;
- add `/var/lib/apt/lists`
- remove usage of namerefs, replace with global dict
2023-02-18 07:44:19 -03:00
Ricardo Pardini
1a7877bb2e
armbian-next: docker: warn & countdown if we can't write to cache/docker; continue by using .tmp/docker if not aborted. 2023-02-18 07:44:10 -03:00
Ricardo Pardini
8d2be2c694
armbian-next: python/pip: solve crazy with /usr/bin/pip3 by not using it (instead, python3 -m pip); don't mask errors
- "| cut" masked errors in multiple places
- include real pip3 version in python deps hash, all will be invalidated
- be less happy about hitting hash/cache, should happen often
2023-02-18 07:44:09 -03:00
Ricardo Pardini
2453722962
armbian-next: docker: pass CI env down to Docker (for GHA etc) 2023-02-18 07:44:08 -03:00
Ricardo Pardini
d4f969c67d
armbian-next: docker: don't bomb if stdin is not a terminal (eg, in CI) 2023-02-18 07:44:07 -03:00
Ricardo Pardini
25d5fe5926
armbian-next: darwin: try to get rid of .DS_Store sneaky bastards before launching Docker 2023-02-18 07:43:56 -03:00
Ricardo Pardini
84e0bc1423
armbian-next: fix, don't try to get pid descendants on non-Linux (fixes Darwin cleanups) 2023-02-18 07:43:55 -03:00
Ricardo Pardini
72c61f2f68
armbian-next: replace bare sync invocations by introducing wait_for_disk_sync(); sync early at start of build
- wait 10s for sync call; otherwise warn user to be patient
- wait_for_sync "reason for sync" everywhere
- add @TODO for /run/user/0 etc and XDG_RUNTIME_DIR in chroot
2023-02-18 07:43:53 -03:00
Ricardo Pardini
2c9582f9ba
armbian-next: add Debian Ports keyring hostdep to strap Debian riscv64 from ports
- rpardini: actually use ports keyring for Debian Risc-V
- `risc-v`: correctly include `debian-ports-archive-keyring` in Risc-V CLI packages
  - include in debootstrap list, via `config/optional/architectures/riscv64/_config/cli/_all_distributions/main/packages`
  - can't have two `--include` in debootstrap call
- riscv64: move `debian-ports-archive-keyring` from main to debootstrap; also only for `sid`
2023-02-18 07:43:49 -03:00
Ricardo Pardini
da1cf2b475
armbian-next: rename too-long main function to prep_conf_main_build_single()
- this commit is proof of OCD (old name didn't fit stacktrace space)
2023-02-18 07:43:42 -03:00
Ricardo Pardini
242a5dad1f
armbian-next: somewhat-better handling/logging when using Docker (don't show two logfiles to inspect)
- do not show path to Docker run-host's logfile, unless the Docker run itself didn't produce logs
- introduce `global_final_exit_code` to allow to maintain/preserve Docker's exit code without triggering an error host-side
2023-02-18 07:43:41 -03:00
Ricardo Pardini
bf3f346bee
armbian-next: still fighting tee leaking: now I think I won, again
- avoid using the "last resort, use lazy umount" by killing tee_pid _and descendants_ before hand. Works!
- introduce `get_descendants_of_pid_array()` using recursive trick to get all descendants
- do not `unset CURRENT_LOGFILE` -- during `logging_error_show_log()` (from SHOW_LOG=no days)
  - do `unset CURRENT_LOGFILE` immediately before processing logs -- although it magically survives. mysteries of traps in bash...
- `declare -g` instead of `export` for `start_logging_section()`
- don't `check_and_close_fd_13()` in `run_cleanup_handlers()`
- do run `check_and_close_fd_13()`, but only after archiving the old logs (in logging trap!)
- add a default error for unhandled TRAP's -- of which should be done
- fix reset of `global_tee_pid=0` in section-logging
2023-02-18 07:43:40 -03:00
Ricardo Pardini
c1100fa461
armbian-next: still fighting tee leaking under duress, turns out I _hadn't_ really won
- last resort, use lazy umount.
- include `lsof` in hostdeps, useful to debug these situations
2023-02-18 07:43:39 -03:00
Ricardo Pardini
0ea0fe3528
armbian-next: mktemp and tmpfs related utility functions with automatic cleanup handlers 2023-02-18 07:43:27 -03:00
Ricardo Pardini
4d59ad7ac4
armbian-next: try harder to get en_US.UTF-8 logging 2023-02-18 07:43:22 -03:00
Ricardo Pardini
99a7a7980b
armbian-next: docker: remove old code 2023-02-18 07:43:21 -03:00
Ricardo Pardini
8444e7e308
armbian-next: bring back swig hostdep -- needed for some u-boot's 2023-02-18 07:43:15 -03:00
Ricardo Pardini
c4ca062709
armbian-next: tmpfs-utils: last-resort debugging, to stderr, of failed tmpfs unmount 2023-02-18 07:43:13 -03:00
Ricardo Pardini
ff94b79e60
armbian-next: do basic host checks before config if interactive, or during prepare-host if not
- group basic checks at `check_basic_host()`
2023-02-18 07:43:12 -03:00
Tony
f549ddb273
armbian-next: officially support WSL2; pester user for UTF-8 terminal
- new `wsl2.sh` file with WSL2-related functions; checking/validating/pestering user; allow WSL2 (not WSL1)

Co-authored-by: Ricardo Pardini <ricardo@pardini.net>
2023-02-18 07:43:11 -03:00
Ricardo Pardini
bf891d54d9
armbian-next: split prepare_host(), fix .tmp reset trap
- `prepare_host()`: split; do checks earlier and allow them to be interactive
- introduce `exit_if_countdown_not_aborted()` for "Low Disk Space" and other critical conditions
- split `prepare_host()` into interactive & non-interactive parts
- split off `clean_deprecated_mountpoints()` from prepare into `cleaning.sh`
- introduce and use `reset_uid_owner_non_recursive()` for `.tmp` reset in trap, to avoid disasters
- add more logging sections to default-build.sh, avoid unlogged parts
2023-02-18 07:43:10 -03:00
Ricardo Pardini
2d9f9216eb
armbian-next: introduce tmpfs-utils.sh; put LOGDIR and WORKDIR under tmpfs; set CCACHE_TEMPDIR under WORKDIR
- introduce generic `prepare_tmpfs_for()`, which manages it's own cleanup/dir removal
- use it for `WORKDIR` (which is `TMPDIR`) and `LOGDIR`
  - adapt previous cleanup handlers for those, so they delete their contents but not the dir itself (which might be mounted)
- also: make `ARMBIAN_LOG_CLI_ID` readonly together with other superglobals
- set `XDG_RUNTIME_DIR` & `XDG_RUNTIME_DIR` together with `TMPDIR`
- kernel-make.sh: pass `CCACHE_TEMPDIR` down to Kernel make (thanks @the-Going)
2023-02-18 07:43:09 -03:00
Ricardo Pardini
029076d0a0
armbian-next: docker: use dash-lines before/after launching docker; set ARMBIAN_INSIDE_DOCKERFILE_BUILD=yes when it is so 2023-02-18 07:43:05 -03:00
Ricardo Pardini
d8254d1e6e
armbian-next: docker: fix Dockerfile output indentation & warning msgs about generation 2023-02-18 07:43:01 -03:00
Ricardo Pardini
b94074ec0a
armbian-next: Docker: abstract away and only run "docker info" once, it's very expensive 2023-02-18 07:42:47 -03:00
Ricardo Pardini
7879895049
armbian-next: core extensions for xfs / f2fs / brtfs / LUKS/cryptroot hostdeps; enable in main-config 2023-02-18 07:42:46 -03:00
Ricardo Pardini
7beeae6219
armbian-next: Python tooling: use consolidated+hashed+cached pip base/pycache; don't pip during Dockerfile build, nor cli-requirements
- consolidate at `prepare_python_and_pip()`
- sanity check for Python version 3.9+ regardless of HOSTRELEASE
- TODO: pip vs sudo/root: need pip 22.2+ to curb warning, not doing it
2023-02-18 07:42:44 -03:00
Ricardo Pardini
461f82f95a
armbian-next: completely remove acl (which provided getfacl) and it's usages (basic deps) 2023-02-18 07:42:42 -03:00