Commit Graph

57 Commits

Author SHA1 Message Date
Pander
0d72163868 Fixed execution rights docker.sh 2023-04-22 20:08:26 +02:00
Ricardo Pardini
d0716b65c0 docker/sudo: better control/info over CLI relaunch args; introduce CLI relaunch envs
- allow adding re-launch environments by populating `ARMBIAN_CLI_RELAUNCH_PARAMS` dict
- similar to `ARMBIAN_CLI_RELAUNCH_PARAMS`
- rename/consolidate `()`'s output globals:
  - `ARMBIAN_CLI_FINAL_RELAUNCH_ARGS` (rename)
  - `ARMBIAN_CLI_FINAL_RELAUNCH_ENVS` (new)
    - includes `ARMBIAN_HIDE_REPEAT_PARAMS` automatically
- `sudo`: include envs when relaunching, use bash explicitly
- `docker`: better logging
- `docker`: include envs when relaunching via `--env` args
2023-04-21 14:26:33 +02:00
Ricardo Pardini
ed0b406265 lib: fix: replace undue export statements with declare -g 2023-03-31 09:12:44 +02:00
Ricardo Pardini
fe7a24885b docker: introduce DOCKER_SKIP_UPDATE=yes to skip updating prebuilt Docker image, only when using 2023-03-31 09:12:44 +02:00
Ricardo Pardini
40c5df55ab docker: introduce ${DOCKER_ARMBIAN_BASE_COORDINATE_PREFIX:-"ghcr.io/armbian/docker-armbian-build:armbian-"} for easy override 2023-03-31 09:12:44 +02:00
Ricardo Pardini
9931350483 docker: fix: pass down OCI_TARGET_BASE to Docker instance automatically 2023-03-17 08:32:11 +01:00
Michael Beck
1223b22ffd
WSL2: Ensure the Windows Terminal is detected under WSL2 docker build. (#4915)
- by passing the WT_SESSION environment variable into the docker container  if it is set
2023-03-09 19:19:30 +01:00
Igor Pečovnik
b7b8eb7b72
Add / modify (c) in bash scripts (#4922)
* Add / modify (c) in bash scripts

Signed-off-by: Igor <igor@armbian.com>

* Add (c) to the source config files

---------

Signed-off-by: Igor <igor@armbian.com>
2023-03-09 18:30:40 +01:00
Ricardo Pardini
aa70ad570b
armbian-next: use new/definitive pkg/image coordinates for Docker image, on Armbian org 2023-02-18 12:13:42 -03:00
Ricardo Pardini
20de5dd2c6
armbian-next: switch to using Armbian-org GH hosted git shallow trees and Docker images 2023-02-18 07:46:46 -03:00
Ricardo Pardini
e697f2e67c
armbian-next: docker: curb logging about OCI_TARGET_BASE 2023-02-18 07:46:10 -03:00
Ricardo Pardini
9bffa5e749
armbian-next: artifacts: firmware/rootfs and actual usage of artifacts during image build
- artifacts: introduce `ARTIFACT_IGNORE_CACHE=yes`
- artifacts: introduce `DONT_BUILD_ARTIFACTS`, list of artifacts that if not found cached, fail the build
- kernel_package_source() is no more
- a long dissertation about kernels, families, and the universe
- artifacts: actually use rootfs artifact for image build
- artifacts: detangle via artifact_base_dir
- artifacts: rootfs: use folders in artifact_name; include cache_type
- artifacts: some cleanups / validations
- rootfs artifact; drop old rootfs cli
- artifacts: new CLI shortcuts; remove old firmware CLI
- artifacts: full firmware & usage
- use firmware artifacts in image build and install its debs
- kernel artifact sans legacy; use tmpdir for .deb target for all packages
- legacy artifact versions is no more; pack/unpack now in common obtain;
- artifacts: uboot: cleanup legacy renaming, use artifact version directly
- artifacts: add firmware (small) artifact
- deploy uboot to loop from artifact; allow tty to artifact; todos for cleaning
- fixes, kernel dtb/headers conditional; remove `.git` from Makefile url; use mapfile for finding files to hash
  - completely remove KERNEL_HAS_WORKING_HEADERS_FULL_SOURCE and `kernel_package_callback_linux_headers_full_source()`
  - don't use realpath for artifact_file_relative
  - curb some warnings
  - fix: only install headers & dtbs if such exist
- kernel .config hook modification hash versioning
- OCI_TARGET_BASE vs per-artifact defaults; only deploy to remote from CLI with OTB
- artifact revolver & installing into image
  - add artifact_map_packages and artifact_map_debs dicts
  - revolver accumulates all info
  - REPOSITORY_INSTALL is no more (for uboot/kernel, later others)
  - rename `VER` to `IMAGE_INSTALLED_KERNEL_VERSION`
2023-02-18 07:46:03 -03:00
Ricardo Pardini
a75e58c320
armbian-next: docker: show $SECONDS in Docker dash logs 2023-02-18 07:46:01 -03:00
Ricardo Pardini
ca6bf9046b
armbian-next: remove all traces of Vagrant and FEL boot 2023-02-18 07:46:00 -03:00
Ricardo Pardini
399d9964dd
armbian-next: python/pip: include pip pkgs in Dockerfile; use them if available instead of downloading 2023-02-18 07:45:51 -03:00
Ricardo Pardini
0e480e685e
armbian-next: docker: pass down git info via env var, since Docker doesn't get ${SRC}/.git, yet we need that info in the Docker logs 2023-02-18 07:45:40 -03:00
Ricardo Pardini
7431984025
armbian-next: docker: pass CARD_DEVICE to Docker via --device 2023-02-18 07:45:35 -03:00
Ricardo Pardini
956ef9ac7d
armbian-next: fix: for the 1000th time had broken KERNEL_ONLY=yes; curb some warnings 2023-02-18 07:45:18 -03:00
Ricardo Pardini
281e3f0828
armbian-next: cli: try (and fail, probably...) to be helpful with half-working Docker installs 2023-02-18 07:45:16 -03:00
Ricardo Pardini
8329b2e970
armbian-next: docker: split docker_cli_build_dockerfile() into docker_cli_prepare_dockerfile(); introduce FAST_DOCKER
- armbian-next: docker: fix hostrelease crazy
2023-02-18 07:45:11 -03:00
Ricardo Pardini
41d7d0155e
armbian-next: docker: fix: not all find's are made equal, back to -mtime +1 (full 24-hour period needed for match) 2023-02-18 07:45:08 -03:00
Ricardo Pardini
941bc81868
armbian-next: docker: pass ARMBIAN_ENABLE_CALL_TRACING=yes via env, and only if DOCKER_ARMBIAN_ENABLE_CALL_TRACING=yes 2023-02-18 07:44:56 -03:00
Ricardo Pardini
962ae65a45
armbian-next: docker: include ${SRC}/.git static via Dockerfile, for kernel-patches-to-git / archeology; use fast archeology
- include FAST_ARCHEOLOGY in call to Python, default yes; sync kernel & u-boot call
2023-02-18 07:44:54 -03:00
Ricardo Pardini
6e267f5cd1
armbian-next: docker: passthrough of the SSH agent, via Docker magic on Darwin; enable for docker-shell and kernel-patches-to-git
- don't check/verify/update SSH-related hostkey etc for git tree push
2023-02-18 07:44:53 -03:00
Ricardo Pardini
bd0a7936da
armbian-next: docker/gha: pass down a bunch of GHA specific vars (list unconfirmed)
- docker: curb logging about stdin
2023-02-18 07:44:46 -03:00
Ricardo Pardini
1cec42392e
armbian-next: GHA & rootfs adventures, pt 1
- rootfs: create readonly global `rootfs_cache_id` (type+hash+date/version)
- add "oras-upload" CLI command (dumb, no retries, single target/single source)
- docker: mount-bind for & re-pass envs `GITHUB_OUTPUT` and `GITHUB_STEP_SUMMARY` down to Docker
- introduce `github-actions.sh::github_actions_add_output()`
- during logs cleanup, dump the Markdown log into GITHUB_STEP_SUMMARY if it is available
2023-02-18 07:44:41 -03:00
Ricardo Pardini
7f74b2e83f
armbian-next: docker: don't spew errors about sharing cache if cache doesn't even exist 2023-02-18 07:44:40 -03:00
Ricardo Pardini
1a7877bb2e
armbian-next: docker: warn & countdown if we can't write to cache/docker; continue by using .tmp/docker if not aborted. 2023-02-18 07:44:10 -03:00
Ricardo Pardini
8d2be2c694
armbian-next: python/pip: solve crazy with /usr/bin/pip3 by not using it (instead, python3 -m pip); don't mask errors
- "| cut" masked errors in multiple places
- include real pip3 version in python deps hash, all will be invalidated
- be less happy about hitting hash/cache, should happen often
2023-02-18 07:44:09 -03:00
Ricardo Pardini
2453722962
armbian-next: docker: pass CI env down to Docker (for GHA etc) 2023-02-18 07:44:08 -03:00
Ricardo Pardini
d4f969c67d
armbian-next: docker: don't bomb if stdin is not a terminal (eg, in CI) 2023-02-18 07:44:07 -03:00
Ricardo Pardini
25d5fe5926
armbian-next: darwin: try to get rid of .DS_Store sneaky bastards before launching Docker 2023-02-18 07:43:56 -03:00
Ricardo Pardini
242a5dad1f
armbian-next: somewhat-better handling/logging when using Docker (don't show two logfiles to inspect)
- do not show path to Docker run-host's logfile, unless the Docker run itself didn't produce logs
- introduce `global_final_exit_code` to allow to maintain/preserve Docker's exit code without triggering an error host-side
2023-02-18 07:43:41 -03:00
Ricardo Pardini
99a7a7980b
armbian-next: docker: remove old code 2023-02-18 07:43:21 -03:00
Ricardo Pardini
029076d0a0
armbian-next: docker: use dash-lines before/after launching docker; set ARMBIAN_INSIDE_DOCKERFILE_BUILD=yes when it is so 2023-02-18 07:43:05 -03:00
Ricardo Pardini
d8254d1e6e
armbian-next: docker: fix Dockerfile output indentation & warning msgs about generation 2023-02-18 07:43:01 -03:00
Ricardo Pardini
b94074ec0a
armbian-next: Docker: abstract away and only run "docker info" once, it's very expensive 2023-02-18 07:42:47 -03:00
Ricardo Pardini
7beeae6219
armbian-next: Python tooling: use consolidated+hashed+cached pip base/pycache; don't pip during Dockerfile build, nor cli-requirements
- consolidate at `prepare_python_and_pip()`
- sanity check for Python version 3.9+ regardless of HOSTRELEASE
- TODO: pip vs sudo/root: need pip 22.2+ to curb warning, not doing it
2023-02-18 07:42:44 -03:00
Ricardo Pardini
d4d3c1346c
armbian-oleg: split hostdeps again, full of ifs, and reduce the minimum set of pkgs for Oleg-conditions while trying to keep Docker full
- probably breaks a lot of stuff; c++ compilers, distcc, possibly others
- split into late_prepare_host_dependencies / early_prepare_host_dependencies calling adaptative_prepare_host_dependencies() full of if's

also:
- calc deps out of configuration
- reorg checks into obtain_and_check_host_release_and_arch()
2023-02-18 07:42:11 -03:00
Ricardo Pardini
225e878743
armbian-oleg: introduce DOCKER_SIMULATE_CLEAN=yes so I can pretend to be Oleg, but using Docker
- if set, the Docker launcher will not install any dependencies
- so every Docker run will reinstall everything from scratch
- do NOT use, **even if you're Oleg**
2023-02-18 07:42:10 -03:00
Ricardo Pardini
6cb12a3397
armbian-next: [focal-host] try to support focal-like host OS's again, via python3.9 (and drop buster completely)
- completely remove support for building under `buster` -- that's way too old, sorry.
- de-hardcode `python3` invocations, instead use `python3_binary_path` set by `prepare_python3_binary_for_python_tools()`
- juggle `$HOSTRELEASE`: read from actual host, or determined from Docker image name (during Dockerfile build)
- TL;DR: include and use `python3.9` for focal-like host OS's
2023-02-18 07:41:47 -03:00
Ricardo Pardini
39ac9e00b1
armbian-next: docker: don't try to double init when recovering from failed pull 2023-02-18 07:40:57 -03:00
Ricardo Pardini
76e276c6a9
armbian-next: Python patching delusion, pt1 & pt2 & pt3
- WiP: Python patching delusion, pt 1: finding & parsing patches; apply & git commit with pygit2; Markdown summaries (also for aggregation); git-to-patches tool
  - Python: Markdown aggregation and patching summaries; collapsible; SummarizedMarkdownWriter
  - Python: Markdown aggregation and patching summaries
  - Python: reorg a bit into common/armbian_utils; define the `ASSET_LOG_BASE` in preparation for Markdown delusion
  - Python patching: initial apply patches & initial commit patches to git (using pygit2)
  - Python patching: add basic `series.conf` support
  - Python patching: force use of utf-8; better error handling; use realpath of dirs
  - Python patching: `git-to-patches` initial hack. not proud. half-reused some of the patches-to-git
  - Python patching: "tag" the git commits with info for extracting later; introduce REWRITE_PATCHES/rewrite_patches_in_place
  - Python patching: commented-out, recover-bad-patches hacks
  - Python patching: shorten the signature
  - Python patching: allow BASE_GIT_TAG as well as BASE_GIT_REVISION
  - Python patching: git-archeology for patches missing descriptions; avoid UTF-8 in header/desc (not diff)
  - Python patching: use modern-er email.utils.parsedate_to_datetime to parse commit date
  - Python patching: unify PatchInPatchFile; better git-commiting; re-exporting patches from Git (directly)
  - Python patching: switch to GitPython
    - GitPython is like 100x slower than pygit2, but actually allows for date & committer
    - also allows to remove untracked files before starting
  - Python aggregation: fix missing `AGGREGATED_APT_SOURCES_DICT`
  - Python patching: add `unidecode` dependency to pip3 install
  - Python patching: don't try archeology if SRC is not a Git Repo (eg, in Docker)
  - Python patching: don't try archeology if not applying patches to git
- WiP: Python patching delusion, pt2: actually use for u-boot & kernel patching
  - Python patching: much better problem handling/logging; lenient with recreations (kernel)
  - Python patching: don't force SHOW_LOG for u-boot patching
  - Python patching: don't bomb for no reason when there are no patches to apply
  - Python patching: fully (?) switch kernel patching to Python
  - Python patching: more logging fixups
  - Python patching: capture `kernel_git_revision` from `fetch_from_repo()`'s `checked_out_revision`
  - Python patching: fully switch u-boot patching to Python
  - Python aggregation/patching: colored logging; patching: always reset to git revision
  - Python aggregation/patching: better logging; introduce u-boot Python patching
- Python patching pt3: recovers and better Markdown
  - Python patching: detect, and rescue, `wrong_strip_level` problem; don't try to export patches that didn't apply, bitch instead
  - Python patching: Markdown patching summary table, complete with emoji
  - Python patching: include the problem breakdown in Markdown summary
  - Python patching: sanity check against half-bare, half-mbox patches
  - Python patching: try to recover from 1) bad utf-8 encoded patches; 2) bad unidiff patches; add a few sanity checks
  - Python patching: try, and fail, to apply badly utf-8 encoded patches directly as bytes [reverted]
  - Python patching: try to recover from patch *parse* failures; show summary; better logging
      - set `GIT_ARCHEOLOGY=yes` to do archeology, default not

- armbian-next: Python `pip` dependencies handling, similar to `hostdeps`
  - same scheme for Dockerfile caching
  - @TODO: still using global/shared environment; should move to a dir under `cache` or some kinda venv
- WiP: add `python3-pip` to hostdeps; remove `python-setuptools`
  - remove `python-setuptools` (Python2, no longer exists in Sid) from hostdeps
  - add `python3-pip` to hostdeps; part of virtualenv saga
- WiP: split `kernel.sh` a bit, into `kernel-patching.sh`, `kernel-config.sh` and `kernel-make.sh`
  - `advanced_patch()`: rename vars for clarity; no real changes
- Python patching: introduce FAST_ARCHEOLOGY; still trying for Markdown links
2023-02-18 07:40:52 -03:00
Ricardo Pardini
b606ae9715
armbian-next: docker: fix when using buildx, always pass --load so produced image is actually available to Docker proper 2023-02-18 07:40:48 -03:00
Ricardo Pardini
9657ceac9b
armbian-next: docker: implement docker-purge and docker-shell cli commands; deprecated mountpoints 2023-02-18 07:40:35 -03:00
Ricardo Pardini
578b21922e
armbian-next: call trace analyzer and call-graph generator 2023-02-18 07:40:20 -03:00
Ricardo Pardini
b2d02071bd
armbian-next: better ccache logs/stats/Docker-support/use in u-boot/kernel
- introduce `SHOW_CCACHE=yes` for detailed ccache statistics and logging
- fix: kernel build is done under "env -i", so pass CCACHE_DIR down to Make if CCACHE_DIR is set...
- split from kernel.sh, show stats also for u-boot targets; show compile time
- add volume definition (under `${SRC}/cache/ccache`); auto-use that in Docker builds via `CCACHE_DIR`
- better logging for `do_with_ccache_statistics()`
- there's some CCACHE_DIR code from before; unify @TODO
2023-02-18 07:40:19 -03:00
Ricardo Pardini
9fbbe897cd
armbian-next: docker: disable showing Docker volume usage; fix syntax in test for ARMBIAN_RUNNING_IN_CONTAINER 2023-02-18 07:40:12 -03:00
Ricardo Pardini
b703eb6da3
armbian-next: docker: locale-gen + enable all existing extensions before calculating docker hostdeps
- that way all extensions' hostdeps are included in Dockerfile
- also run locale-gen in Dockerfile
2023-02-18 07:40:11 -03:00
Ricardo Pardini
75de1c2d82
armbian-next: remove usage of REQUIREMENTS_DEFS_ONLY; initialize extensions for requirements cli command & docker pre-game deps 2023-02-18 07:40:07 -03:00