625 Commits

Author SHA1 Message Date
Ricardo Pardini
e49d91ec40 popular demand: bring back KERNEL_CONFIGURE=yes *during image build*
- partially revert 6ef394d95de6a113099caf3d31b668c04dda3862 (thus bring back the TUI/dialog for selecting KERNEL_CONFIGURE=yes/no)
- partially revert d890b418c7107af183b5f1f2a9304923537766c4 (thus bring back the capacity to config & build image in one go)
- stop after configuring kernel, but only if command is `kernel-config`, not regular image-build KERNEL_CONFIGURE=yes
2023-05-09 10:05:09 +02:00
Ricardo Pardini
83e35b1473 binfmts: don't raise warning if qemu-arm failed to load under aarch64 (some are native, some are emulated, all are good)
- reported by @igorpecovnik on Discord, during massive CI build, when armhf builds landed on arm64 builders
2023-05-09 10:04:57 +02:00
Ricardo Pardini
8f14f33f78 logging: squash undue warnings rootfs/aptcache
- rootfs is built with BOARD= so just live with it (idea about this warn was that families might hook rootfs build process)
- count, instead of du, apt caches. seems some empty dirs are reported as "1Mb" by du?
2023-05-09 10:04:57 +02:00
Ricardo Pardini
5316c6685b host-utils: more logging for when junk is found in image apt caches 2023-05-09 10:04:57 +02:00
Ricardo Pardini
b18cd8aaf4 logging: curb some warnings into info's 2023-05-09 10:04:57 +02:00
Ricardo Pardini
ab696d0776 rootfs: bump non-desktop tmpfs_estimated_size from 2000mb to 2300mb (determined empirically, after building all images) 2023-05-09 10:04:57 +02:00
Ricardo Pardini
b7c659c0a9 pipeline: gha: fix IMAGES_ONLY_OUTDATED_ARTIFACTS and SKIP_IMAGES by exporting them so Python can see; tune logging 2023-05-09 10:04:57 +02:00
Ricardo Pardini
637c4fec01 include (clean/real) apt lists in built images; update xapian index during image build, not rootfs; don't leave junk in the rootfs cache
- revert #5139 Revert "Apt lists is needed for synpatic index generation"
  This reverts commit 63e9bd3baad4f6faa8e6e6f6c82e46716b89acf7
  - this actually only left junk in rootfs cache for the future to find
2023-05-09 10:04:50 +02:00
Ricardo Pardini
a872fe8773 config-prepare: config_pre_main(): try to be more helpful when BRANCH is unset or wrong instead of just "bad KERNEL_MAJOR_MINOR" 2023-05-08 11:21:47 +02:00
amazingfate
0b5dc0c921 fix uwe5622 for kernel v6.3 2023-05-06 12:55:10 +02:00
Igor
63e9bd3baa Apt lists is needed for synpatic index generation 2023-05-05 19:37:19 +02:00
Ricardo Pardini
1ad9f7cbd2 artifact-rootfs: add desktop info (environment, config_name, appgrous) to artifact_version_reason (so we can debug when I mess up later) 2023-05-04 16:08:11 +02:00
Ricardo Pardini
7c0fb70a2b configdump: alias config-dump, config-dump-json and new config-dump-no-json (bash declare format) 2023-05-04 16:08:11 +02:00
Ricardo Pardini
0ff694cfbd produce_repeat_args_array: @mhoffrog was right, we need to quote repeat params; (here I'm being stubborn and only quoting the ones that have spaces in them) 2023-05-04 16:08:11 +02:00
Ricardo Pardini
73e9c89807 docker: fix: always mount .docker/config.json if it exists (in GHA) 2023-05-03 15:09:56 +02:00
Ricardo Pardini
6ef394d95d kernel: get better rid of KERNEL_CONFIGURE, remove UI asking for it, it's now a separate command kernel-config and wouldn't work anyway 2023-05-03 15:09:56 +02:00
Ricardo Pardini
724a67d3e5 pipeline: cli: add proper gha- prefix to CLI commands that have something to do with GitHub Actions (thus gha-matrix now, and the new gha-template)
- stop at the correct places for `inventory` and `targets` non-GHA-specific commands
- handle `gha-template` first (and return), since it does not really depend on any info/targets/etc
  - batcat the output for show-off
2023-05-03 07:49:53 +02:00
Ricardo Pardini
23a2b34847 pipeline: add output-gha-workflow-template.py utility, for rendering GHA workflow YAML templates with chunks
- python-tools: add Jinja2. Incredible how we made it this far without it.
- output-gha-workflow-template: a double-templater, first runs jinja with a custom syntax, then "moar magic" to be useful for GHA
2023-05-03 07:49:53 +02:00
Ricardo Pardini
4bf98ebfab artifact kernel/uboot: add ARCH to artifact_input_variables; used for (maybe) mapping to runner tags in JSON pipeline 2023-05-03 07:49:53 +02:00
igorpecovnik
ba1a84a5e8
Adjusting ORAS cache location, runners (#5129)
* Move packages to new location
* Adjust build runners
* Switch to OS

Signed-off-by: Igor <igor@armbian.com>
---------

Signed-off-by: Igor <igor@armbian.com>
Co-authored-by: Ricardo Pardini <ricardo@pardini.net>
2023-05-02 11:08:34 +02:00
Vyacheslav Bocharov
64f434ab33 Revert "fix rtl88x2cs compilation for kernel 6.3" as fixed in rtl88x2cs repository.
This reverts commit dc9f751012b43e690a9341d63bd6950709a4dd44.
2023-05-02 12:04:51 +03:00
Ricardo Pardini
3767729dc5 rootfs: add artifact name prefix and EXTRA_ROOTFS_NAME suffix (for extensions), yyyymm one-per-month auto-version in place of REVISION for artifact version
- rootfs is not a deb package, and thus not subject to the same strict ordering rules of `artifact_prefix_version`
  - replace with `yyyymm` (`202305`) in version so we get one-per-month automatically
2023-05-01 22:47:12 +02:00
Ricardo Pardini
f992dcd466 git-ref2info: retries for getting Makefile body (https://raw.githubusercontent.com has been failing a lot) 2023-05-01 22:47:03 +02:00
Ricardo Pardini
d890b418c7 kernel/uboot/atf: introduce kernel-patch, uboot-patch, atf-patch, uboot-config, kernel-config CLI commands; enhanced manual patching; block deprecated ways
- all interactive commands now **don't build the artifact** anymore; just patches/.configs are produced and then build ends
  - user is required to put the produced patches in the right place and build again, for full consistency
- split ATF and U-BOOT manual patching process; use CLI command `atf-patch` to patch ATF, and `uboot-patch` to patch u-boot
- non-interactive artifact builds are now 100% sans-stdin
- introduce `uboot-config` CLI command; still experimental, only produces a defconfig and not a patch
- reworked `userpatch_create()` to be (hopefully) more useful:
  - detects a previous patch and offers to apply it before continuing
  - enters a loop showing the diff, and only proceeds when user indicates he's happy with the patch
  - produces `mbox`-formatted patches via `format-patch` and standard Armbian parameters
  - uses MAINTAINER and MAINTAINERMAIL instead of git configuration (so it works in containers)
- don't allow image builds with any patching or configuring _at all_ (it has been deprecated with a warning for months already, and results are inconsistent)
2023-05-01 22:46:55 +02:00
Ricardo Pardini
f77edd0591 rtl8822bs: fix: remove binaries (firmware?) being added to kernel driver source and breaking patching; related to #5117 2023-05-01 22:46:43 +02:00
Ricardo Pardini
e74ab59f48 kernel-make: introduce KERNEL_EXTRA_CFLAGS; fill it to add to what's passed to KCFLAGS; related to #5117 2023-05-01 22:46:43 +02:00
Ricardo Pardini
369eb80f9c driver_uwe5622_allwinner: only apply "pre-6.1" park-link patch if kernel >= 5 (eg: not for rockchip64-legacy); possibly fixes #5120 2023-05-01 22:46:43 +02:00
Ricardo Pardini
91bb00bfa2 artifact-uboot: fix: include UBOOT_TARGET_MAP in variables hash of version 2023-05-01 22:46:43 +02:00
Ricardo Pardini
c692676da7 kernel-debs: postinst/etc: fix: unify is_boot_dev_vfat() function; make work during image build too via env var
- determinining if `/boot` is vfat is more complex than it looks;
  - use a common function to unify across all scripts
- during image build, the kernel install is always done with a non-fat, non-mounted /boot
  - use a special variable passed through the env so unified function can know ahead of time
  - introduce `extra_apt_envs` param to `chroot_sdcard_apt_get()`; array with extra environment vars
- if vfat /boot:
  - `linux-update-symlinks` is not called (Debian would just choke on this)
  - don't even try to symlink Armbian stuff, move instead
- more/better logging board-side (unfortunately mixed in all the set -x debugs)
2023-05-01 22:46:43 +02:00
Ricardo Pardini
f8ddf7f9e2 🔥 JSON info pipeline: v18
- pipeline: add `pipeline` context object to targets; use it to filter artifacts and images to build; warn about oci-name with multiple oci-tags
- pipeline: better error messages when info's fail; show some (out-of-order) progress messages during parallel info gathering
- pipeline: targets-compositor: add `not-eos` inventory
- TARGETS_FILENAME, log all OCI lookups
- SKIP_IMAGES
- IMAGES_ONLY_OUTDATED_ARTIFACTS
- no dash in chunk id in JSON
- pipeline: very initial chunking, using the same outputs
- pipeline: template targets, `items-from-inventory:` inventory expansion, CHECK_OCI=yes, CLEAN_MATRIX=yes, CLEAN_INFO=yes, many fixes
- cli: `inventory` / `targets` / `matrix` / `workflow`
- pipeline: workflow beginnings
- pipeline: general log cleanup + OCI stats / better miss handling
- pipeline: fixes/reorg
- pipeline: catch & log JSON parsing errors
- pipeline: gha matrix: use IMAGE_FILE_ID as job description
- pipeline (delusion): gha workflow output, based on old matrix code
- pipeline: better parsing and reporting of stderr log lines (under `ANSI_COLOR=none`)
- pipeline: mapper-oci-uptodate: use separate positive/negative cache dirs (GHA will only cache positives); cache negs for 5 minutes locally
- pipeline: output-gha-matrix artifacts + images
  - pipeline: output-gha-matrix artifacts + images: "really" and fake 1-item matrix if empty
- pipeline: move files into subdir; update copyright & cleanup
- pipeline: refactor bash jsoninfo driver a bit
- pipeline: outdated-artifact-image-reducer
- pipeline: introduce `target_id` at the compositor, aggregate it at the reducer, carry it over in the artifact info mapper
- pipeline: mapper-oci-uptodate
- pipeline: info-gatherer-artifact, with PRE_PREPARED_HOST
- pipeline: refactor/rename info-gatherer-image.py
- pipeline: beginnings
2023-05-01 22:46:25 +02:00
Ricardo Pardini
3143db5df4 kernel-debs: call linux-update-symlinks in postinst; expected of Debian-compatible kernels
- this handles /vmlinuz etc symlinks (doesn't know about uInitrd, etc)
2023-04-28 21:32:52 +02:00
Ricardo Pardini
59d6690a11 git-ref2info: many fixes for nonstandard repos; try a few different dereference and refs/ combinations in a certain order; remove codeaurora repo handling
- git-ref2info: when resolving SHA1's for 'xxx' tag or branch, try first `refs/heads/xxx`, then just 'xxx'; validate SHA1 with 40-char regexp
  - this works-around Gerrit-style repositories that have a 'refs/for/xxx' ref that gets in the way
  - case in point: https://github.com/hardkernel/u-boot/tree/odroidc-v2011.03 (has both refs/heads/odroidc-v2011.03 and refs/for/odroidc-v2011.03)
- git-ref2info: when processing tags for a SHA1, try both dereference and non-dereferenced, in that order, to get a SHA1
  - this solves old problem reported in #4916: GitHub incorrectly processes dereference requests for tags that are not actually annotated
  - case in point: https://github.com/ayufan-rock64/linux-kernel/releases/tag/4.4.202-1237-rockchip-ayufan
- `codeaurora` is no more, https://bye.codeaurora.org/
2023-04-28 21:32:52 +02:00
Ricardo Pardini
dc9f751012 fix rtl88x2cs compilation for kernel 6.3 2023-04-28 21:32:29 +02:00
Paolo Sabatino
0d5fe62403 fix rtl8723cs compilation for kernel 6.3 2023-04-27 21:29:42 +02:00
Paolo Sabatino
8c4a35bfb0 fix rtl88x2bu compilation on kernel 6.3 2023-04-27 21:29:22 +02:00
Paolo Sabatino
f2d9756bb0 fix rtl8812au compilation for kernel 6.3 2023-04-27 21:29:02 +02:00
Paolo Sabatino
be0ecb1f16 fix rtl8723du compilation on kernel 6.3 2023-04-27 21:28:10 +02:00
Paolo Sabatino
3e28c86a05 fix rtl8811cu compilation for kernel 6.3 2023-04-27 21:27:49 +02:00
Pander
0d72163868 Fixed execution rights docker.sh 2023-04-22 20:08:26 +02:00
Ricardo Pardini
3d2575639d logging: include Repeat Build Args in markdown/ANSI logs 2023-04-21 14:26:33 +02:00
Ricardo Pardini
27d9fdf4bf interactive: sort the params included in Repeat Build Options; don't quote values 2023-04-21 14:26:33 +02:00
Ricardo Pardini
3ffbdfc97c interactive: hide auto-relaunched params from Repeat Build Options 2023-04-21 14:26:33 +02:00
Ricardo Pardini
d0716b65c0 docker/sudo: better control/info over CLI relaunch args; introduce CLI relaunch envs
- allow adding re-launch environments by populating `ARMBIAN_CLI_RELAUNCH_PARAMS` dict
- similar to `ARMBIAN_CLI_RELAUNCH_PARAMS`
- rename/consolidate `()`'s output globals:
  - `ARMBIAN_CLI_FINAL_RELAUNCH_ARGS` (rename)
  - `ARMBIAN_CLI_FINAL_RELAUNCH_ENVS` (new)
    - includes `ARMBIAN_HIDE_REPEAT_PARAMS` automatically
- `sudo`: include envs when relaunching, use bash explicitly
- `docker`: better logging
- `docker`: include envs when relaunching via `--env` args
2023-04-21 14:26:33 +02:00
Ricardo Pardini
9a073d06fb interactive: desktop: use set_interactive_config_value() so user-selected desktop options appear in repeat build args 2023-04-21 14:26:33 +02:00
Markus Hoffrogge
d1252fc0ee interactive: better Repeat Build Options [by @mhoffrog, squashed, rebased]
Add interactive configurations to Repeat Build Options

Simplify function produce_repeat_args_array()

- make use of $ARMBIAN_NON_PARAM_ARGS and $ARMBIAN_PARSED_CMDLINE_PARAMS (associated array)
- quote parameter values to be on the safe side

Move "Repeat Build Options" logs to start-end.sh

- cli-build.sh, start-end.sh:
  - move function function produce_repeat_args_array() to start-end.sh
  - add $WHAT as first arg to produce_repeat_args_array() if
    $WHAT is not empty
  - move early display log of "Repeat Build Options" to
    function main_default_start_build()
  - move last log of "Repeat Build Options" to
    function main_default_end_build()
  - add last log of "Repeat Build Options" to
    LOG_SECTION="repeat_build_options"
2023-04-21 14:26:33 +02:00
amazingfate
97a2e41f62 use common sed for GITHUB_SOURCE replace 2023-04-21 14:21:28 +02:00
Igor
0590a70597 Change custom Ubuntu mirror logic
Signed-off-by: Igor <igor@armbian.com>
2023-04-19 20:25:03 +02:00
Ricardo Pardini
2b28f29ebe rootfs/image: avoid apt junk leftover by debootstrap; add warnings and debugs over target cache and general rootfs sizes
- rootfs: rootfs-create: show a summary of the 20 biggest dirs, right before tarring the rootfs (for debugging)
- rootfs: rootfs-create: show usage of caches between first and second stages
- rootfs: rootfs-create: cleanup junk left by `debootstrap` after second stage
- rootfs: rootfs-create: _always_ clean apt stuff at the end
- rename `apt_purge_unneeded_packages()` to `apt_purge_unneeded_packages_and_clean_apt_caches()` for clarity
- image: `apt_purge_unneeded_packages_and_clean_apt_caches()`: warn if apt caches not empty; clean them off, always.
- host-utils: `local_apt_deb_cache_prepare()`: also test the target, warn if not empty
- extension: cleanup-space-final-image: do NOT clean apt stuff. done in core now
- the metric shit-ton of debugs added should help the next person who faces this in the future
2023-04-18 16:27:06 +02:00
Werner
0b90ebc54a Use native xz 2023-04-18 09:29:17 +02:00
Werner
945bb6dff9 Use native xz 2023-04-18 09:29:17 +02:00