`Validate changed board configs` (PR #9747 CI run 25258999953) failed
on the three Ayn-Odin2 derivative boards with:
ERROR: config/boards/ayn-odin2mini.csc: BOARDFAMILY: required, ...
ERROR: config/boards/ayn-odin2mini.csc: KERNEL_TARGET: required, ...
(same pair on ayn-odin2portal.csc and ayn-thor.csc)
The validator's header explicitly says it does NOT source the file
("boards have side-effecty function bodies"), and the regex
`parse_assignments` only sees top-level `KEY=value` lines. Boards
that consist of one `source "${SRC}/config/boards/<parent>.csc"` line
plus a handful of overrides (BOARD_NAME, BOARD_VENDOR, BOARD_MAINTAINER,
ARCH) inherit BOARDFAMILY / KERNEL_TARGET / KERNEL_TEST_TARGET from
the parent, so the validator saw them as missing and erred out.
The errors are pre-existing — they would fire on any future change
to those .csc files. PR #9747 just made them visible by being the
first PR to touch an inheriting board since the validator landed.
Add `collect_inherited_assignments`: when a top-level `source` line
points at another file under config/boards/ (matched by a regex
anchored to start-of-line so `source` calls inside function bodies —
which are always indented — aren't followed), parse that file's
top-level assignments and lay them behind the child's own. Child's
explicit values still win via dict.setdefault.
Recursion is guarded by a visited set keyed on resolved paths, so
a self-source or a parent-source-child cycle terminates instead of
looping. Missing source targets are silently skipped — the main
required-field checks will still flag any field that ends up unset
after the merge, so a typo'd source path doesn't mask a real gap.
Verified against the five PR-changed files: 6 errors → 0 errors,
4 warnings → 1 warning (the unmaintained-board warning on
aml-t95z-plus.tvb, which is unrelated to inheritance and was
already there). Self-contained boards (musepipro.conf etc.)
produce identical output to before this change.
Adds a PR-time validator that checks board configs for missing
required / recommended fields. Helps catch issues before merge
rather than at first build attempt by the unlucky maintainer who
picks up the orphan board.
tools/validate-board-config.py:
- Standalone Python validator, runnable locally:
tools/validate-board-config.py config/boards/orangepi5.conf
- Extracts top-level KEY=VALUE assignments via regex (does NOT
source the bash file; boards have side-effecty function bodies)
- Emits plain text findings; with --github also emits ::error
/ ::warning workflow commands that annotate the PR diff
- Exit 1 if any error, else 0
Rule table:
ERRORS:
BOARD_NAME required, non-empty
BOARD_VENDOR required, non-empty
BOARDFAMILY required, must match config/sources/families/
KERNEL_TARGET must contain at least one of:
legacy / current / edge / vendor / mainline
BOOTCONFIG required (except .wip)
WARNINGS:
BOARD_MAINTAINER recommended; orphan boards rot
INTRODUCED recommended; year first shipped
BOOT_FDT_FILE recommended (skip on .tvb)
Per-extension behavior:
.conf / .csc full rule set
.tvb full rule set, skip BOOT_FDT_FILE warning
.wip only BOARD_NAME is error; rest demoted to warning
.eos skipped (dead boards aren't validated)
.github/workflows/maintenance-validate-board-configs.yml:
- Triggers on pull_request touching config/boards/** (or the
validator/workflow itself)
- Detects changed files via git diff against PR base — only
validates what the PR actually touched, not all 500+ boards
- Runs validator with --github so findings show on the PR diff
- Skips cleanly when no board configs changed (other PRs unaffected)
Smoke-tested against orangepi5.conf, odroidn2.conf, aml-t95z-plus.tvb
(real boards, no errors) and synthetic broken .conf / .wip files
(errors and warnings as expected).
This commit implements a complete parallel repository management system
that allows building and publishing Debian repositories in parallel,
significantly reducing build time for multiple distributions.
- `update-main`: Builds common/main component once for all releases
- `update -R <release>`: Builds release-specific components in isolated DBs
- `merge`: Combines common + release-specific components into final repos
- Isolated databases (aptly-isolated-<release>) avoid locking during parallel builds
- Common component built once, not duplicated per release
- Release-specific components (utils, desktop) built independently
- Final merge combines all components with proper GPG signing
- Fixed GPG signing to target top-level Release files (dists/{release}/Release)
- Pool cleanup before publishing avoids "file already exists" errors
- Smart package import skips duplicates during merge
- Proper handling of empty repositories and missing components
- Improved error handling and logging throughout
1. update-main: Build common component (once)
2. update -R <release>: Parallel workers build release-specific components
3. merge: Combine all components and publish with GPG signatures
This enables GitHub Actions to run multiple release builders in parallel,
reducing total repository build time from hours to minutes.
Signed-off-by: Igor Pecovnik <igor@armbian.com>
* tools/repository/extract-repo.sh: simplify extraction by copying directly from pool
Remove dependency on Packages index files. Instead of parsing package metadata
to find file locations, directly scan the pool/ directory structure and copy all
.deb files found in each component subdirectory.
This simplifies the code and makes it more robust since it doesn't rely on
index files being present or correctly formatted.
Signed-off-by: Igor Pecovnik <igor@armbian.com>
* Add helper script: recursively clean Armbian Debian package artifacts
* Update tools/repository/cleanup-debs.sh
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
---------
Signed-off-by: Igor Pecovnik <igor@armbian.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
- Remove redundant run_cmd function, use run_aptly consistently
- Include EOS releases in repository processing (remove -ve 'eos' filter)
- Fix showall command to auto-discover repos from isolated databases
- Fix source file preservation in parallel mode (keep sources for workers)
- Fix snapshot management to handle published snapshots correctly
- Fix drop_unsupported_releases to only drop unsupported, not all repos
- Remove isolated DB cleanup that caused data loss on each run
- Remove inappropriate sudo usage from html and date commands
- Enable FORCE_PUBLISH by default for better workflow
- Improve common snapshot creation in isolated mode
- Add isolated database support to showall function
Signed-off-by: Igor Pecovnik <igor@armbian.com>
**Repository tools: enable parallel repo generation and refactor repo.sh**
Add support for parallel repository generation using GitHub Actions, allowing
multiple workers to build different releases concurrently without database
locking conflicts.
### Highlights
- Add `-R/--single-release`, `update-main`, and `merge` for CI-level parallelism
- Use isolated per-release databases; remove local `-j/--parallel-jobs`
- Sign all Release files; optimize signing order
- Add `extract-repo.sh` for extracting and reorganizing packages from repositories
- Replace file-based logging with syslog (`logger`), removing sudo requirement
- Improve error handling, signing logic, and robustness
- Refactor repo.sh for better readability and documentation
- Apply CodeRabbit-recommended fixes
Signed-off-by: Igor Pecovnik <igor@armbian.com>
* merge patches from rockchip and rk322x families for current and edge
kernels
* adjust patches for tinkerboard to remove some cruft and overclocking
* rework kernel configs
* fix rk322x dmc to avoid lockup on rk3288
* migrate rockchip-6.6 into patch series, rename all patches with
more understandable names
* add gen-series.sh script in tools directory (it is a naive tool to
create patch series when you don't want to rebase everything)
> How to use:
>
> `./compile.sh inventory` - does just the board inventory; look for output in `output/info`
>
> `./compile.sh targets-dashboard` - does inventory, targets compositing, and images info; look for output in `output/info`, read the instructions output by the command if you want to load the OpenSearch dashboards.
>
> `./compile.sh targets` - does the full targets compositing and artifacts, look for output in `output/info`
>
> If you don't have a `userpatches/targets.yaml`, _one will be provided for you_ defaulting to Jammy minimal CLI
> and Jammy xfce desktop, for all boards in all branches. You can pass filters via `TARGETS_FILTER_INCLUDE=...` to narrow.
>
- board JSON inventory:
- more generic regex parsing of variables from board files:
- all top-level (non-indented) variables are parsed and included in the JSON board inventory
- this allows us to add new variables to the board files without having to update the parser
- variables can be bare, `export` or `declare -g`, but **_must_ be quoted** (single or double) and UPPER_CASE
- some special treatment for certain variables:
- `KERNEL_TARGET` is parsed as a _comma-separated_ list of valid BRANCH'es
- `BOARD_MAINTAINER` is parsed as _space-separated_ list of valid maintainer GH usernames as `BOARD_MAINTAINERS: [...]` in the JSON
- script complains if `BOARD_MAINTAINER` is not set in core boards. Empty is still allowed.
- `HAS_VIDEO_OUTPUT="no"` causes `BOARD_HAS_VIDEO: false` in the JSON (for desktop-only inventorying, see below)
- introduce `not-eos-with-video` in `items-from-inventory` at the targets compositor
- the same as `not-eos`, but with added `BOARD_HAS_VIDEO: true` filter, see above
- introduce `TARGETS_FILTER_INCLUDE` for targets compositor
- this filters the targets _after_ compositing (but before getting image info), based on the board inventory data
- it's a comma-separated list of `key:value` pairs, which are OR-ed together
- new virtual info `BOARD_SLASH_BRANCH` post-compositing inventory for filtering of a specific BOARD/BRANCH combo (e.g. `odroidhc4/edge`)
- some interesting possible filters:
- `TARGETS_FILTER_INCLUDE="BOARD:odroidhc4"`: _only_ build a single board, all branches. JIRA [AR-1806]
- `TARGETS_FILTER_INCLUDE="BOARD_SLASH_BRANCH:odroidhc4/current"`: _only_ build a single board/branch combo
- `TARGETS_FILTER_INCLUDE="BOARD:odroidhc4,BOARD:odroidn2"`: _only_ build _two_ boards, all branches.
- `TARGETS_FILTER_INCLUDE="BOARD_MAINTAINERS:rpardini"`: build all boards and branches where rpardini is a maintainer
- `TARGETS_FILTER_INCLUDE="BOARDFAMILY:rockchip64"`: build all boards and branches in the rockchip64 family
- image-info-only variables like `LINUXFAMILY` is **not** available for filtering at this stage
- rename `config/templates` `targets-all-cli.yaml` to `targets-default.yaml`
- this is used when no `userpatches/targets.yaml` is found
- new default includes all boards vs branches for non-EOS boards
- also desktop for all boards that _don't_ have `HAS_VIDEO_OUTPUT='no``
- introduce simplified `targets-dashboard` CLI:
- does only inventory, compositing, and image info, but not artifact reducing, etc.
- ignore desktop builds in the OpenSearch indexer
- update the OpenSearch Dashboards, including new information now available
- invert the logic used for `CLEAN_INFO` and `CLEAN_MATRIX`
- defaults to `yes` now, so new users/CI don't get hit by stale caches by default
- repo pipeline CLI stuff is usually run on saved/restored artifacts for `output/info`, so don't clean by default via the CLI
* Added Debian 12 Bookworm.
* Link identical
* Make minimal truly minimal again
Move netplan.io to CLI image only
* Add netplan.io to Bullseye standard CLI
Co-authored-by: Konstantin Litvinov <klitvinov@piesoft.us>
Co-authored-by: Igor <igor@armbian.com>
* Moving repository handling from main script
- cleaning
- working
TBD: handling of parameters
* Not needed anymore
* Paramter handling, edit config in mktemp
* Typo
* Improve string handling
* Cleanining
* Compacting
* Change to backslash compatible sed replacement
* Publish all, update selected to avoid having erros at update
* Add support for html index generation
* Bugfix
* More arch
* Don't run publish if there is nothing in the incoming
* Add repo sync control file creation
* Fix hardcoded value
* sunxi-5.18: add megous patches: tag: orange-pi-5.18-20220627-1924
* tools: mk_format_patch: ignore-matching-lines git version
* sunxi-5.18: re-extracted armbian patches after being applied to 5.18.8
* sunxi-5.18: switch to version tag=v5.18.8
* sunxi-5.18: add new megous patches
* switch to v5.18.5: exclude a previously applied patch
* fix: tools/mk_format_patch: numbered=false by default
* sunxi-5.18: rebasing and extraction using the mk_format_patch script
* add a hand command line tool
* Rename folder
* Add empty readme
* Add tool for adjustin configs, set executable bit
* Add folder to labeller
Co-authored-by: Igor Pecovnik <igor.pecovnik@gmail.com>
