Commit Graph

73 Commits

Author SHA1 Message Date
Ricardo Pardini
ecbf09491d
armbian-next: logging: introduce RAW_LOG=yes to export .tar with raw unprocessed logs (for development) 2023-02-18 07:45:46 -03:00
Ricardo Pardini
3bcf89d6c6
armbian-next: logging: introduce SHARE_LOG=yes to auto-curl the logs to paste 2023-02-18 07:45:45 -03:00
Ricardo Pardini
0e480e685e
armbian-next: docker: pass down git info via env var, since Docker doesn't get ${SRC}/.git, yet we need that info in the Docker logs 2023-02-18 07:45:40 -03:00
Ricardo Pardini
388c76c91b
armbian-next: shellcheck: squash more shellcheck warnings; some long-lost variables being used; some unset's that are better reset's, etc 2023-02-18 07:45:33 -03:00
Ricardo Pardini
ba7487b1a1
armbian-next: docker: fix Dockerfile generation; don't output logs info during Dockerfile build 2023-02-18 07:45:23 -03:00
Ricardo Pardini
c5de5e0f1f
armbian-next: logging: ansi-haste sharing beta 2 (now with URL) 2023-02-18 07:45:22 -03:00
Ricardo Pardini
bef7f8fee4
armbian-next: artifacts: introduce kernel/u-boot artifacts; git ref2info; "memoizer"; some hashing
- artifacts: u-boot/kernel - pt7 - adapt legacy/artifact versions; use common `capture_rename_legacy_debs_into_artifacts()`
- artifacts: u-boot - pt6: add artifact for u-boot
  - use artifact version / reason in actual u-boot .deb if present
- artifacts: kernel - pt5: tune kernel version, refactor
- artifacts: kernel - pt4: squash unrelated bugs that show up; move `prepare_compilation_vars()` to default build
- artifacts: kernel - pt3: drivers+patches+.config hashing
  - split file hashing function from drivers-harness; fix it so filenames are relative and sorted; sort from ${SRC}, always
  - aplit prepare_kernel_config_core_or_userpatches() from `kernel_config_initialize()`
- artifacts: kernel - pt2:  memoizing git ref2info
- artifacts: kernel - pt1:  versioning
2023-02-18 07:45:20 -03:00
Ricardo Pardini
b4df4b83c3
armbian-next: logging: detect TERM=alacritty and use normal instead of gray 2023-02-18 07:45:17 -03:00
Ricardo Pardini
41d7d0155e
armbian-next: docker: fix: not all find's are made equal, back to -mtime +1 (full 24-hour period needed for match) 2023-02-18 07:45:08 -03:00
Ricardo Pardini
3c5488b002
armbian-next: bump logging archive greet to 14 files 2023-02-18 07:44:59 -03:00
Ricardo Pardini
89d6c686c6
armbian-next: chroot_sdcard_apt_get_install_dry_run(): run again if first try fails, with more logging 2023-02-18 07:44:52 -03:00
Ricardo Pardini
6177b94268
armbian-next: logging: pure ASCII .log; included in Markdown for GHA 2023-02-18 07:44:45 -03:00
Ricardo Pardini
1cec42392e
armbian-next: GHA & rootfs adventures, pt 1
- rootfs: create readonly global `rootfs_cache_id` (type+hash+date/version)
- add "oras-upload" CLI command (dumb, no retries, single target/single source)
- docker: mount-bind for & re-pass envs `GITHUB_OUTPUT` and `GITHUB_STEP_SUMMARY` down to Docker
- introduce `github-actions.sh::github_actions_add_output()`
- during logs cleanup, dump the Markdown log into GITHUB_STEP_SUMMARY if it is available
2023-02-18 07:44:41 -03:00
Ricardo Pardini
78dcf01c92
armbian-next: cleanup, kill and add to-do's, remove dead code, turn down logging, squash future shortcircuit bugs
- killed `[[ $ROOTFS_TYPE != ext4 ]] && display_alert "Assuming ${BOARD} ${BRANCH} kernel supports ${ROOTFS_TYPE}" "" "wrn"`
  - which definitely didn't belong in rootfs
- disable usage of run_host_command_logged_long_running for kernel-make
- stop lying about long_running stuff being any different from non-long_running versions
- stop lying about 'set -e' when there's still a bunch of pipes for "pv" and stuff all around
2023-02-18 07:44:33 -03:00
Ricardo Pardini
44ae761927
armbian-next: apt: expand local caching to include package lists, as well as .deb's
- `/var/cache/apt` in chroot was already cached;
- add `/var/lib/apt/lists`
- remove usage of namerefs, replace with global dict
2023-02-18 07:44:19 -03:00
Ricardo Pardini
805085ff45
armbian-next: logging: validate where not already in a logging section when trying to start a new one
- with a small exception for 'entrypoint' ;-)
2023-02-18 07:44:18 -03:00
Ricardo Pardini
3f0c480e9d
armbian-next: mark remove_all_trap_handlers() deprecated, it never did anything 2023-02-18 07:44:16 -03:00
Ricardo Pardini
780fe26ea4
armbian-next: logging: (very, very early beta) for uploading/sharing of ANSI log file online 2023-02-18 07:44:13 -03:00
Ricardo Pardini
d95478281b
armbian-next: fix ANSI logs, use reset at strategic places 2023-02-18 07:44:06 -03:00
Ricardo Pardini
eb30aac310
armbian-next: extensions: move 'extensions.sh' into library; replace manual source with init function
- wraps globals in `extension_manager_declare_globals()` -- and make them actually global
- whoever wrote this original code is insane, add copyright
2023-02-18 07:44:05 -03:00
Ricardo Pardini
22074a0cde
armbian-next: logging: new log file / summary nomenclature 2023-02-18 07:44:01 -03:00
Ricardo Pardini
a6182bbd89
armbian-next: docker: archive logfiles again; warn & greet testers when archiving many old logfiles 2023-02-18 07:44:00 -03:00
Ricardo Pardini
72c61f2f68
armbian-next: replace bare sync invocations by introducing wait_for_disk_sync(); sync early at start of build
- wait 10s for sync call; otherwise warn user to be patient
- wait_for_sync "reason for sync" everywhere
- add @TODO for /run/user/0 etc and XDG_RUNTIME_DIR in chroot
2023-02-18 07:43:53 -03:00
Ricardo Pardini
8af14d4f8c
armbian-next: countdown: split off file for countdown functions (countdown-to-abort and countdown-to-continue)
- introduce `countdown.sh`, move `traps.sh::exit_if_countdown_not_aborted()` there, introduce `exit_if_countdown_not_aborted()`
2023-02-18 07:43:50 -03:00
Ricardo Pardini
8cedc82ec2
armbian-next: hopefully improved error logging; avoid logging errors up to 3x; subdued red stacktraces
- reorg `display_alert()` so more frequent levels are handled first
- improve some ANSI color usage, still a mess
- don't write failed commands stackstrace as warnings to screen -- just to file (yes it's redundant, except if in test)
- more useful / subdued stacktraces, to make actual error more obvious...
2023-02-18 07:43:43 -03:00
Ricardo Pardini
242a5dad1f
armbian-next: somewhat-better handling/logging when using Docker (don't show two logfiles to inspect)
- do not show path to Docker run-host's logfile, unless the Docker run itself didn't produce logs
- introduce `global_final_exit_code` to allow to maintain/preserve Docker's exit code without triggering an error host-side
2023-02-18 07:43:41 -03:00
Ricardo Pardini
bf3f346bee
armbian-next: still fighting tee leaking: now I think I won, again
- avoid using the "last resort, use lazy umount" by killing tee_pid _and descendants_ before hand. Works!
- introduce `get_descendants_of_pid_array()` using recursive trick to get all descendants
- do not `unset CURRENT_LOGFILE` -- during `logging_error_show_log()` (from SHOW_LOG=no days)
  - do `unset CURRENT_LOGFILE` immediately before processing logs -- although it magically survives. mysteries of traps in bash...
- `declare -g` instead of `export` for `start_logging_section()`
- don't `check_and_close_fd_13()` in `run_cleanup_handlers()`
- do run `check_and_close_fd_13()`, but only after archiving the old logs (in logging trap!)
- add a default error for unhandled TRAP's -- of which should be done
- fix reset of `global_tee_pid=0` in section-logging
2023-02-18 07:43:40 -03:00
Ricardo Pardini
ec7639e518
armbian-next: severe bug, there was no prefix-emoji for Windows 2023-02-18 07:43:33 -03:00
Ricardo Pardini
62d948b1b4
armbian-next: still fighting tee leaking under duress, and I think I won
- call `check_and_close_fd_13()` directly when `exit_with_error()` called, don't wait for trap
- store PID spawned by `do_with_logging()` under `$global_tee_pid`
- `check_and_close_fd_13()`:
  - sprinkle with `sync` to give a chance for stuff to die gracefully
  - check if `$global_tee_pid` is set and actually a running PID, if so, kill it
  - do not blindly "wait" for stuff, instead, kill it and wait for it to die -- 100% more likely to not hang ;-)
2023-02-18 07:43:31 -03:00
Ricardo Pardini
35fbb7ed6c
armbian-next: small cleanups: squash typos / add function keyword / add types / mark dead code 2023-02-18 07:43:28 -03:00
Ricardo Pardini
a6a5152695
armbian-next: chroot_sdcard_apt_get_update() to replace and better log chroot_sdcard_apt_get update
- also add retries where it was missing
- @TODO: desktop_postinstall() is a mess, needs aggregation work
2023-02-18 07:43:19 -03:00
Ricardo Pardini
2455c55dac
armbian-next: firmware: don't build -full firmware if not on CI/noninteractive and board's not going to use it
- move `fakeroot_dpkg_deb_build()` from runners.sh to new utils-dpkgdeb.sh
  - and make it debug show the size of the source
- clean off `.git` directory earlier for full build
- @TODO: get rid of the "hardlinks" funky biz in there and not-copy `.git` to begin with
2023-02-18 07:43:18 -03:00
Ricardo Pardini
48c0e9acf4
armbian-next: severe bug, countdown counted up 2023-02-18 07:43:16 -03:00
Ricardo Pardini
9d26ee1373
armbian-next: logging: squash nasty leaking file descriptor bug due to "tee"
- check and make sure the fd is closed before processing and/or deleting logs.
- change from fd 3 to 13, for extra "this is evil". is it? is 13 an evil number? you decide.
2023-02-18 07:43:14 -03:00
Ricardo Pardini
bf891d54d9
armbian-next: split prepare_host(), fix .tmp reset trap
- `prepare_host()`: split; do checks earlier and allow them to be interactive
- introduce `exit_if_countdown_not_aborted()` for "Low Disk Space" and other critical conditions
- split `prepare_host()` into interactive & non-interactive parts
- split off `clean_deprecated_mountpoints()` from prepare into `cleaning.sh`
- introduce and use `reset_uid_owner_non_recursive()` for `.tmp` reset in trap, to avoid disasters
- add more logging sections to default-build.sh, avoid unlogged parts
2023-02-18 07:43:10 -03:00
Ricardo Pardini
2d9f9216eb
armbian-next: introduce tmpfs-utils.sh; put LOGDIR and WORKDIR under tmpfs; set CCACHE_TEMPDIR under WORKDIR
- introduce generic `prepare_tmpfs_for()`, which manages it's own cleanup/dir removal
- use it for `WORKDIR` (which is `TMPDIR`) and `LOGDIR`
  - adapt previous cleanup handlers for those, so they delete their contents but not the dir itself (which might be mounted)
- also: make `ARMBIAN_LOG_CLI_ID` readonly together with other superglobals
- set `XDG_RUNTIME_DIR` & `XDG_RUNTIME_DIR` together with `TMPDIR`
- kernel-make.sh: pass `CCACHE_TEMPDIR` down to Kernel make (thanks @the-Going)
2023-02-18 07:43:09 -03:00
Ricardo Pardini
231e179703
armbian-next: traps: allow for cleanup handlers with arguments; unify around run_one_cleanup_handler()
- show error if trap handler fails
2023-02-18 07:43:06 -03:00
Ricardo Pardini
43b4f45018
armbian-next: introduce POOR_MAN_PROFILER=yes (under ANSI_COLOR=none)
- turns out EPOCHREALTIME can be comma separated, and have multiple leading zeroes
2023-02-18 07:43:00 -03:00
Ricardo Pardini
2a68cc6e9e
armbian-next: optimize for CONFIG_DEFS_ONLY=yes, skipping stacktraces/git info 2023-02-18 07:42:59 -03:00
Ricardo Pardini
ef36016ee4
armbian-next: logging: reinit logging after processing cmdline parameters
- so `DEBUG=yes` actually works
2023-02-18 07:42:34 -03:00
Ricardo Pardini
898d567ae0
armbian-oleg: if commands fail, write to log in bright red 2023-02-18 07:42:30 -03:00
Ricardo Pardini
68d8f10666
armbian-oleg: make chroot_sdcard_apt_get_install_dry_run() logging more useful by filtering the noise out 2023-02-18 07:42:29 -03:00
Ricardo Pardini
c7c7d2e0d6
armbian-oleg: drastically curb apt-get logging when running in chroot 2023-02-18 07:42:28 -03:00
Ricardo Pardini
2c0c968645
armbian-oleg: curb Python launcher logs 2023-02-18 07:42:24 -03:00
Ricardo Pardini
95ffd76b3b
armbian-oleg: curb apt-get's logging with -q / -qq 2023-02-18 07:42:23 -03:00
Ricardo Pardini
b03530dacd
armbian-oleg: better logs; only include non-empty .log files 2023-02-18 07:42:22 -03:00
Ricardo Pardini
d24f363187
armbian-oleg: logging: new logfile format; SHOW_LOG=yes default; introduce DEBUG=yes
- default `SHOW_LOG=yes` if user on terminal
- shortcut: `DEBUG=yes` defaults both `SHOW_LOG=yes` & `SHOW_DEBUG=yes`
- much simpler logging format for log file
- skip all display_alert()'s more verbose than DEBUG from logfile
  - set `DEBUG=yes` to log everything again
- skip Markdown assets from being included in ANSI log
- use of `ccze`
- introduce internal var for controlling COMMAND-level log
- remove unused `run_on_sdcard()` that somehow was left there
- drop `export_html_logs()` and `EXPORT_HTML_LOG` -- are no more
2023-02-18 07:42:20 -03:00
Ricardo Pardini
7493ce1b09
armbian-next: long overdue split of logging.sh 2023-02-18 07:42:19 -03:00
Ricardo Pardini
93c8ce99af
armbian-next: apt-cacher-ng is now optional and activated via MANAGE_ACNG=yes; drop NO_APT_CACHER
- acng is constant source of headaches
- we don't need it anymore, since we have local caching now
- You can also have acng running on a different machine/container and set `APT_PROXY_ADDR=xxx:3142` if you're running a build farm
2023-02-18 07:42:12 -03:00
Ricardo Pardini
f167864f2c
armbian-next: split of config-prepare into multiple functions, allowing for logging of all non-interactive sections
- introduce `do_with_conditional_logging()` which only starts logging sections if `do_logging=no`
- with this we should get complete logs (ofc except for the interactive sections)
2023-02-18 07:41:46 -03:00