Commit Graph

83 Commits

Author SHA1 Message Date
Ricardo Pardini
fc14d62c52 capture: set globals CAPTURED_VARS_NAMES and CAPTURED_VARS_ARRAY (instead of CAPTURED_VARS_ARRAY that was space-delimited); filter and sort last-minute 2023-03-31 09:12:44 +02:00
Ricardo Pardini
9a74b838ce runners: introduce internal skip_error_info=yes for run_host_command_logged()
- doesn't show nor clear vars `if_error_find_files_sdcard` and `if_error_detail_message`
- used by `chroot_sdcard_apt_get()` which uses `run_host_command_logged()` multiple times, and the first would clear the important/last one
2023-03-31 09:12:44 +02:00
Igor
d4a7a1c816 Revert "Update export-logs.sh to use primary Armbian paste server (#4936)"
This reverts commit bc62c8cd42.
2023-03-18 21:39:02 +01:00
Ricardo Pardini
736fe7ecf4 don't use "bright black" ANSI for logging unless we're running under a known-good terminal (eg, iTerm2)
- giving up on the "bright black" == "gray" thing. There's enough bad default themes out there, and not the first time people (rightfully) complained...
2023-03-17 08:32:11 +01:00
Ricardo Pardini
ee11f5dbd9 chroot_sdcard_apt_get: don't use -qq for all apt operations on chroot if SHOW_DEBUG=yes (or DEBUG=yes)
- helps debugging crazy upgrades happening due to @rpardini's stupid `Version: ` insanity
2023-03-12 18:40:20 -03:00
TheLinuxBug
bc62c8cd42
Update export-logs.sh to use primary Armbian paste server (#4936)
Changing export-logs.sh script from posting to armbian.next.armbian.com to posting to the primary paste server which has now been upgraded at paste.armbian.com.
2023-03-11 22:03:40 +01:00
Igor Pečovnik
b7b8eb7b72
Add / modify (c) in bash scripts (#4922)
* Add / modify (c) in bash scripts

Signed-off-by: Igor <igor@armbian.com>

* Add (c) to the source config files

---------

Signed-off-by: Igor <igor@armbian.com>
2023-03-09 18:30:40 +01:00
Ricardo Pardini
25606ef3c8 logs: remove git diff from GIT_INFO_ANSI, it's somewhat useless and potentially huge leading to argument list too long 2023-02-24 13:46:28 -03:00
Ricardo Pardini
2f7423d0e4
shellfmt/.editorconfig: enable trim whitespaces, run shellfmt, no actual code changes (#4849)
- shellfmt: bump to 3.6.0 (run with `bash lib/tools/shellfmt.sh`)
- .editorconfig: rationalize, remove duplicated logic
2023-02-23 09:24:41 +01:00
Ricardo Pardini
8f23c9d2e4
armbian-next: debug_dict()/debug_var() only if debugging 2023-02-18 07:46:08 -03:00
Ricardo Pardini
ecbf09491d
armbian-next: logging: introduce RAW_LOG=yes to export .tar with raw unprocessed logs (for development) 2023-02-18 07:45:46 -03:00
Ricardo Pardini
3bcf89d6c6
armbian-next: logging: introduce SHARE_LOG=yes to auto-curl the logs to paste 2023-02-18 07:45:45 -03:00
Ricardo Pardini
0e480e685e
armbian-next: docker: pass down git info via env var, since Docker doesn't get ${SRC}/.git, yet we need that info in the Docker logs 2023-02-18 07:45:40 -03:00
Ricardo Pardini
388c76c91b
armbian-next: shellcheck: squash more shellcheck warnings; some long-lost variables being used; some unset's that are better reset's, etc 2023-02-18 07:45:33 -03:00
Ricardo Pardini
ba7487b1a1
armbian-next: docker: fix Dockerfile generation; don't output logs info during Dockerfile build 2023-02-18 07:45:23 -03:00
Ricardo Pardini
c5de5e0f1f
armbian-next: logging: ansi-haste sharing beta 2 (now with URL) 2023-02-18 07:45:22 -03:00
Ricardo Pardini
bef7f8fee4
armbian-next: artifacts: introduce kernel/u-boot artifacts; git ref2info; "memoizer"; some hashing
- artifacts: u-boot/kernel - pt7 - adapt legacy/artifact versions; use common `capture_rename_legacy_debs_into_artifacts()`
- artifacts: u-boot - pt6: add artifact for u-boot
  - use artifact version / reason in actual u-boot .deb if present
- artifacts: kernel - pt5: tune kernel version, refactor
- artifacts: kernel - pt4: squash unrelated bugs that show up; move `prepare_compilation_vars()` to default build
- artifacts: kernel - pt3: drivers+patches+.config hashing
  - split file hashing function from drivers-harness; fix it so filenames are relative and sorted; sort from ${SRC}, always
  - aplit prepare_kernel_config_core_or_userpatches() from `kernel_config_initialize()`
- artifacts: kernel - pt2:  memoizing git ref2info
- artifacts: kernel - pt1:  versioning
2023-02-18 07:45:20 -03:00
Ricardo Pardini
b4df4b83c3
armbian-next: logging: detect TERM=alacritty and use normal instead of gray 2023-02-18 07:45:17 -03:00
Ricardo Pardini
41d7d0155e
armbian-next: docker: fix: not all find's are made equal, back to -mtime +1 (full 24-hour period needed for match) 2023-02-18 07:45:08 -03:00
Ricardo Pardini
3c5488b002
armbian-next: bump logging archive greet to 14 files 2023-02-18 07:44:59 -03:00
Ricardo Pardini
89d6c686c6
armbian-next: chroot_sdcard_apt_get_install_dry_run(): run again if first try fails, with more logging 2023-02-18 07:44:52 -03:00
Ricardo Pardini
6177b94268
armbian-next: logging: pure ASCII .log; included in Markdown for GHA 2023-02-18 07:44:45 -03:00
Ricardo Pardini
1cec42392e
armbian-next: GHA & rootfs adventures, pt 1
- rootfs: create readonly global `rootfs_cache_id` (type+hash+date/version)
- add "oras-upload" CLI command (dumb, no retries, single target/single source)
- docker: mount-bind for & re-pass envs `GITHUB_OUTPUT` and `GITHUB_STEP_SUMMARY` down to Docker
- introduce `github-actions.sh::github_actions_add_output()`
- during logs cleanup, dump the Markdown log into GITHUB_STEP_SUMMARY if it is available
2023-02-18 07:44:41 -03:00
Ricardo Pardini
78dcf01c92
armbian-next: cleanup, kill and add to-do's, remove dead code, turn down logging, squash future shortcircuit bugs
- killed `[[ $ROOTFS_TYPE != ext4 ]] && display_alert "Assuming ${BOARD} ${BRANCH} kernel supports ${ROOTFS_TYPE}" "" "wrn"`
  - which definitely didn't belong in rootfs
- disable usage of run_host_command_logged_long_running for kernel-make
- stop lying about long_running stuff being any different from non-long_running versions
- stop lying about 'set -e' when there's still a bunch of pipes for "pv" and stuff all around
2023-02-18 07:44:33 -03:00
Ricardo Pardini
44ae761927
armbian-next: apt: expand local caching to include package lists, as well as .deb's
- `/var/cache/apt` in chroot was already cached;
- add `/var/lib/apt/lists`
- remove usage of namerefs, replace with global dict
2023-02-18 07:44:19 -03:00
Ricardo Pardini
805085ff45
armbian-next: logging: validate where not already in a logging section when trying to start a new one
- with a small exception for 'entrypoint' ;-)
2023-02-18 07:44:18 -03:00
Ricardo Pardini
3f0c480e9d
armbian-next: mark remove_all_trap_handlers() deprecated, it never did anything 2023-02-18 07:44:16 -03:00
Ricardo Pardini
780fe26ea4
armbian-next: logging: (very, very early beta) for uploading/sharing of ANSI log file online 2023-02-18 07:44:13 -03:00
Ricardo Pardini
d95478281b
armbian-next: fix ANSI logs, use reset at strategic places 2023-02-18 07:44:06 -03:00
Ricardo Pardini
eb30aac310
armbian-next: extensions: move 'extensions.sh' into library; replace manual source with init function
- wraps globals in `extension_manager_declare_globals()` -- and make them actually global
- whoever wrote this original code is insane, add copyright
2023-02-18 07:44:05 -03:00
Ricardo Pardini
22074a0cde
armbian-next: logging: new log file / summary nomenclature 2023-02-18 07:44:01 -03:00
Ricardo Pardini
a6182bbd89
armbian-next: docker: archive logfiles again; warn & greet testers when archiving many old logfiles 2023-02-18 07:44:00 -03:00
Ricardo Pardini
72c61f2f68
armbian-next: replace bare sync invocations by introducing wait_for_disk_sync(); sync early at start of build
- wait 10s for sync call; otherwise warn user to be patient
- wait_for_sync "reason for sync" everywhere
- add @TODO for /run/user/0 etc and XDG_RUNTIME_DIR in chroot
2023-02-18 07:43:53 -03:00
Ricardo Pardini
8af14d4f8c
armbian-next: countdown: split off file for countdown functions (countdown-to-abort and countdown-to-continue)
- introduce `countdown.sh`, move `traps.sh::exit_if_countdown_not_aborted()` there, introduce `exit_if_countdown_not_aborted()`
2023-02-18 07:43:50 -03:00
Ricardo Pardini
8cedc82ec2
armbian-next: hopefully improved error logging; avoid logging errors up to 3x; subdued red stacktraces
- reorg `display_alert()` so more frequent levels are handled first
- improve some ANSI color usage, still a mess
- don't write failed commands stackstrace as warnings to screen -- just to file (yes it's redundant, except if in test)
- more useful / subdued stacktraces, to make actual error more obvious...
2023-02-18 07:43:43 -03:00
Ricardo Pardini
242a5dad1f
armbian-next: somewhat-better handling/logging when using Docker (don't show two logfiles to inspect)
- do not show path to Docker run-host's logfile, unless the Docker run itself didn't produce logs
- introduce `global_final_exit_code` to allow to maintain/preserve Docker's exit code without triggering an error host-side
2023-02-18 07:43:41 -03:00
Ricardo Pardini
bf3f346bee
armbian-next: still fighting tee leaking: now I think I won, again
- avoid using the "last resort, use lazy umount" by killing tee_pid _and descendants_ before hand. Works!
- introduce `get_descendants_of_pid_array()` using recursive trick to get all descendants
- do not `unset CURRENT_LOGFILE` -- during `logging_error_show_log()` (from SHOW_LOG=no days)
  - do `unset CURRENT_LOGFILE` immediately before processing logs -- although it magically survives. mysteries of traps in bash...
- `declare -g` instead of `export` for `start_logging_section()`
- don't `check_and_close_fd_13()` in `run_cleanup_handlers()`
- do run `check_and_close_fd_13()`, but only after archiving the old logs (in logging trap!)
- add a default error for unhandled TRAP's -- of which should be done
- fix reset of `global_tee_pid=0` in section-logging
2023-02-18 07:43:40 -03:00
Ricardo Pardini
ec7639e518
armbian-next: severe bug, there was no prefix-emoji for Windows 2023-02-18 07:43:33 -03:00
Ricardo Pardini
62d948b1b4
armbian-next: still fighting tee leaking under duress, and I think I won
- call `check_and_close_fd_13()` directly when `exit_with_error()` called, don't wait for trap
- store PID spawned by `do_with_logging()` under `$global_tee_pid`
- `check_and_close_fd_13()`:
  - sprinkle with `sync` to give a chance for stuff to die gracefully
  - check if `$global_tee_pid` is set and actually a running PID, if so, kill it
  - do not blindly "wait" for stuff, instead, kill it and wait for it to die -- 100% more likely to not hang ;-)
2023-02-18 07:43:31 -03:00
Ricardo Pardini
35fbb7ed6c
armbian-next: small cleanups: squash typos / add function keyword / add types / mark dead code 2023-02-18 07:43:28 -03:00
Ricardo Pardini
a6a5152695
armbian-next: chroot_sdcard_apt_get_update() to replace and better log chroot_sdcard_apt_get update
- also add retries where it was missing
- @TODO: desktop_postinstall() is a mess, needs aggregation work
2023-02-18 07:43:19 -03:00
Ricardo Pardini
2455c55dac
armbian-next: firmware: don't build -full firmware if not on CI/noninteractive and board's not going to use it
- move `fakeroot_dpkg_deb_build()` from runners.sh to new utils-dpkgdeb.sh
  - and make it debug show the size of the source
- clean off `.git` directory earlier for full build
- @TODO: get rid of the "hardlinks" funky biz in there and not-copy `.git` to begin with
2023-02-18 07:43:18 -03:00
Ricardo Pardini
48c0e9acf4
armbian-next: severe bug, countdown counted up 2023-02-18 07:43:16 -03:00
Ricardo Pardini
9d26ee1373
armbian-next: logging: squash nasty leaking file descriptor bug due to "tee"
- check and make sure the fd is closed before processing and/or deleting logs.
- change from fd 3 to 13, for extra "this is evil". is it? is 13 an evil number? you decide.
2023-02-18 07:43:14 -03:00
Ricardo Pardini
bf891d54d9
armbian-next: split prepare_host(), fix .tmp reset trap
- `prepare_host()`: split; do checks earlier and allow them to be interactive
- introduce `exit_if_countdown_not_aborted()` for "Low Disk Space" and other critical conditions
- split `prepare_host()` into interactive & non-interactive parts
- split off `clean_deprecated_mountpoints()` from prepare into `cleaning.sh`
- introduce and use `reset_uid_owner_non_recursive()` for `.tmp` reset in trap, to avoid disasters
- add more logging sections to default-build.sh, avoid unlogged parts
2023-02-18 07:43:10 -03:00
Ricardo Pardini
2d9f9216eb
armbian-next: introduce tmpfs-utils.sh; put LOGDIR and WORKDIR under tmpfs; set CCACHE_TEMPDIR under WORKDIR
- introduce generic `prepare_tmpfs_for()`, which manages it's own cleanup/dir removal
- use it for `WORKDIR` (which is `TMPDIR`) and `LOGDIR`
  - adapt previous cleanup handlers for those, so they delete their contents but not the dir itself (which might be mounted)
- also: make `ARMBIAN_LOG_CLI_ID` readonly together with other superglobals
- set `XDG_RUNTIME_DIR` & `XDG_RUNTIME_DIR` together with `TMPDIR`
- kernel-make.sh: pass `CCACHE_TEMPDIR` down to Kernel make (thanks @the-Going)
2023-02-18 07:43:09 -03:00
Ricardo Pardini
231e179703
armbian-next: traps: allow for cleanup handlers with arguments; unify around run_one_cleanup_handler()
- show error if trap handler fails
2023-02-18 07:43:06 -03:00
Ricardo Pardini
43b4f45018
armbian-next: introduce POOR_MAN_PROFILER=yes (under ANSI_COLOR=none)
- turns out EPOCHREALTIME can be comma separated, and have multiple leading zeroes
2023-02-18 07:43:00 -03:00
Ricardo Pardini
2a68cc6e9e
armbian-next: optimize for CONFIG_DEFS_ONLY=yes, skipping stacktraces/git info 2023-02-18 07:42:59 -03:00
Ricardo Pardini
ef36016ee4
armbian-next: logging: reinit logging after processing cmdline parameters
- so `DEBUG=yes` actually works
2023-02-18 07:42:34 -03:00