diff --git a/extensions/ccache-remote/README.server-setup.md b/extensions/ccache-remote/README.server-setup.md new file mode 100644 index 0000000000..7c01ed8300 --- /dev/null +++ b/extensions/ccache-remote/README.server-setup.md @@ -0,0 +1,142 @@ +# ccache-remote: Server Setup Guide + +## Redis Server + +1. Install packages: + ```bash + apt install redis-server avahi-daemon avahi-utils + ``` + +2. Configure Redis — merge the settings from `misc/redis/redis-ccache.conf` into + `/etc/redis/redis.conf`, or add an `include` directive at the end of your + existing config (`include /etc/redis/redis-ccache.conf`), then restart: + ```bash + sudo systemctl restart redis-server + ``` + + **Authentication (recommended).** Set a password in the config file — either via + `requirepass` (Redis < 6) or ACL user entry (Redis 6+). See comments in + `misc/redis/redis-ccache.conf` for both methods. Generate a URL-safe password: + ```bash + openssl rand -hex 24 + ``` + **Important:** Do not use `openssl rand -base64` — base64 passwords contain + `/`, `+`, and `=` which break URL parsing in `redis://` connection strings. + On the build host, pass the password in the Redis URL: + ```bash + ./compile.sh ENABLE_EXTENSIONS=ccache-remote \ + CCACHE_REMOTE_STORAGE="redis://default:YOUR_PASSWORD@192.168.1.65:6379" BOARD=... + ``` + + **No authentication (trusted network only).** If all machines are on a fully + isolated private network and access control is not needed, remove `requirepass`, + set `nopass` in the ACL user entry, and set `protected-mode no`. See comments + in `misc/redis/redis-ccache.conf`. No password is needed in the URL: + ```bash + ./compile.sh ENABLE_EXTENSIONS=ccache-remote \ + CCACHE_REMOTE_STORAGE="redis://192.168.1.65:6379" BOARD=... + ``` + + For advanced security (TLS, ACL, rename-command), see: + https://redis.io/docs/latest/operate/oss_and_stack/management/security/ + +3. Publish DNS-SD service — copy `misc/avahi/ccache-redis.service` to `/etc/avahi/services/`: + ```bash + cp misc/avahi/ccache-redis.service /etc/avahi/services/ + ``` + Avahi will pick it up automatically. Clients running `avahi-browse -rpt _ccache._tcp` + will discover the Redis service. + + Or use a systemd unit that ties the announcement to `redis-server` lifetime + (stops advertising when Redis is down): + ```bash + cp misc/systemd/ccache-avahi-redis.service /etc/systemd/system/ + systemctl enable --now ccache-avahi-redis + ``` + + Alternatively, publish legacy mDNS hostname: + ```bash + avahi-publish-address -R ccache.local + ``` + Or as a systemd service (`/etc/systemd/system/ccache-hostname.service`): + ```ini + [Unit] + Description=Publish ccache.local hostname via Avahi + After=avahi-daemon.service redis-server.service + BindsTo=redis-server.service + [Service] + Type=simple + ExecStart=/usr/bin/avahi-publish-address -R ccache.local + Restart=on-failure + [Install] + WantedBy=redis-server.service + ``` + +## HTTP/WebDAV Server (nginx) + +1. Install nginx with WebDAV support: + ```bash + apt install nginx-extras avahi-daemon avahi-utils + ``` + +2. Copy `misc/nginx/ccache-webdav.conf` to `/etc/nginx/sites-available/ccache-webdav`, + then enable and prepare storage: + ```bash + cp misc/nginx/ccache-webdav.conf /etc/nginx/sites-available/ccache-webdav + ln -s /etc/nginx/sites-available/ccache-webdav /etc/nginx/sites-enabled/ + mkdir -p /var/cache/ccache-webdav/ccache + chown -R www-data:www-data /var/cache/ccache-webdav + systemctl reload nginx + ``` + +3. Verify: + ```bash + curl -X PUT -d "test" http://localhost:8088/ccache/test.txt + curl http://localhost:8088/ccache/test.txt + ``` + + **WARNING:** No authentication configured. + Use ONLY in a fully trusted private network. + +4. Publish DNS-SD service — copy `misc/avahi/ccache-webdav.service` to `/etc/avahi/services/`: + ```bash + cp misc/avahi/ccache-webdav.service /etc/avahi/services/ + ``` + + Or use a systemd unit that ties the announcement to `nginx` lifetime: + ```bash + cp misc/systemd/ccache-avahi-webdav.service /etc/systemd/system/ + systemctl enable --now ccache-avahi-webdav + ``` + +## DNS SRV Records (for remote/hosted servers) + +Set `CCACHE_REMOTE_DOMAIN` on the client, then create DNS records. + +Redis backend: +```text +_ccache._tcp.example.com. SRV 0 0 6379 ccache.example.com. +_ccache._tcp.example.com. TXT "type=redis" +``` + +HTTP/WebDAV backend: +```text +_ccache._tcp.example.com. SRV 0 0 8088 ccache.example.com. +_ccache._tcp.example.com. TXT "type=http" "path=/ccache/" +``` + +## Client Requirements for mDNS + +Install one of the following for `.local` hostname resolution: + +- **libnss-resolve** (systemd-resolved): + ```bash + apt install libnss-resolve + ``` + `/etc/nsswitch.conf`: `hosts: files resolve [!UNAVAIL=return] dns myhostname` + +- **libnss-mdns** (standalone): + ```bash + apt install libnss-mdns + ``` + `/etc/nsswitch.conf`: `hosts: files mdns4_minimal [NOTFOUND=return] dns myhostname` diff --git a/extensions/ccache-remote/ccache-remote.sh b/extensions/ccache-remote/ccache-remote.sh new file mode 100644 index 0000000000..67dce10169 --- /dev/null +++ b/extensions/ccache-remote/ccache-remote.sh @@ -0,0 +1,492 @@ +# Extension: ccache-remote +# Enables ccache with remote storage for sharing compilation cache across build hosts. +# Supports Redis and HTTP/WebDAV backends (ccache 4.4+). +# +# Documentation: +# Redis: https://ccache.dev/howto/redis-storage.html +# HTTP: https://ccache.dev/howto/http-storage.html +# General: https://ccache.dev/manual/4.10.html#config_remote_storage +# +# Usage: +# # With explicit Redis server: +# ./compile.sh ENABLE_EXTENSIONS=ccache-remote CCACHE_REMOTE_STORAGE="redis://192.168.1.65:6379" BOARD=... +# +# # With HTTP/WebDAV server: +# ./compile.sh ENABLE_EXTENSIONS=ccache-remote CCACHE_REMOTE_STORAGE="http://192.168.1.65:8088/ccache/" BOARD=... +# +# # Auto-discovery via DNS-SD (no URL needed, discovers type/host/port): +# ./compile.sh ENABLE_EXTENSIONS=ccache-remote BOARD=... +# +# # DNS SRV discovery for remote build servers: +# ./compile.sh ENABLE_EXTENSIONS=ccache-remote CCACHE_REMOTE_DOMAIN="example.com" BOARD=... +# +# # Disable local cache, use remote only (saves local disk space): +# ./compile.sh ENABLE_EXTENSIONS=ccache-remote CCACHE_REMOTE_ONLY=yes BOARD=... +# +# Automatically sets USE_CCACHE=yes +# +# Supported ccache environment variables (passed through to builds): +# See: https://ccache.dev/manual/latest.html#_configuration_options +# CCACHE_BASEDIR - base directory for path normalization (enables cache sharing) +# CCACHE_REMOTE_STORAGE - remote storage URL (redis://... or http://...) +# CCACHE_REMOTE_DOMAIN - domain for DNS SRV discovery (e.g., "example.com") +# CCACHE_REMOTE_ONLY - use only remote storage, disable local cache +# CCACHE_READONLY - read-only mode, don't update cache +# CCACHE_RECACHE - don't use cached results, but update cache +# CCACHE_RESHARE - rewrite cache entries to remote storage +# CCACHE_DISABLE - disable ccache completely +# CCACHE_MAXSIZE - maximum cache size (e.g., "10G") +# CCACHE_MAXFILES - maximum number of files in cache +# CCACHE_NAMESPACE - cache namespace for isolation +# CCACHE_SLOPPINESS - comma-separated list of sloppiness options +# CCACHE_UMASK - umask for cache files +# CCACHE_LOGFILE - path to log file +# CCACHE_DEBUGLEVEL - debug level (1-2) +# CCACHE_STATSLOG - path to stats log file +# CCACHE_PCH_EXTSUM - include PCH extension in hash +# +# CCACHE_REMOTE_STORAGE format (ccache 4.4+): +# Redis: redis://[[USERNAME:]PASSWORD@]HOST[:PORT][|attribute=value...] +# HTTP: http://HOST[:PORT]/PATH/[|attribute=value...] +# Common attributes: +# connect-timeout=N - connection timeout in milliseconds (default: 100) +# operation-timeout=N - operation timeout in milliseconds (default: 10000) +# Examples: +# "redis://default:secretpass@192.168.1.65:6379|connect-timeout=500" +# "redis://192.168.1.65:6379|connect-timeout=500" +# "http://192.168.1.65:8088/ccache/" +# +# Auto-discovery (priority order): +# 1. Explicit CCACHE_REMOTE_STORAGE - used as-is, no discovery +# 2. DNS-SD browse for _ccache._tcp on local network (avahi-browse) +# 3. DNS SRV record _ccache._tcp.DOMAIN (when CCACHE_REMOTE_DOMAIN is set) +# 4. Legacy mDNS: resolve 'ccache.local' hostname (fallback) +# +# When multiple services are found, Redis is preferred over HTTP. +# +# DNS-SD service publication (on cache server): +# # For HTTP/WebDAV: +# avahi-publish-service "ccache-webdav" _ccache._tcp 8088 type=http path=/ccache/ +# # For Redis: +# avahi-publish-service "ccache-redis" _ccache._tcp 6379 type=redis +# +# DNS SRV record (for remote/hosted build servers): +# Set CCACHE_REMOTE_DOMAIN to your domain, then create DNS records: +# _ccache._tcp.example.com. SRV 0 0 8088 ccache.example.com. +# _ccache._tcp.example.com. TXT "type=http" "path=/ccache/" +# The cache server must be reachable from the build host (e.g., via port forwarding). +# +# Legacy mDNS (backward compatible): +# Publish 'ccache.local' hostname via Avahi: +# avahi-publish-address -R ccache.local +# Or create a systemd service (see below). +# +# Server setup: see README.server-setup.md and config files in misc/ +# - misc/redis/redis-ccache.conf — Redis configuration example +# - misc/nginx/ccache-webdav.conf — nginx WebDAV configuration example +# - misc/avahi/ccache-*.service — Avahi DNS-SD service files (static, always announce) +# - misc/systemd/ccache-avahi-*.service — systemd units (announce only while service runs) +# +# Fallback behavior: +# If CCACHE_REMOTE_STORAGE is not set and ccache.local is not resolvable, +# extension silently falls back to local ccache only. +# +# Cache sharing requirements: +# For cache to be shared across multiple build hosts, the Armbian project +# path must be identical on all machines (e.g., /home/build/armbian). +# This is because ccache includes the working directory in the cache key. +# Docker builds automatically use consistent paths (/armbian/...). + +# Default Redis connection timeout in milliseconds (can be overridden by user) +# Note: Must be set before extension loads (e.g., via environment or command line) +declare -g -r CCACHE_REDIS_CONNECT_TIMEOUT="${CCACHE_REDIS_CONNECT_TIMEOUT:-500}" + +# List of ccache environment variables to pass through to builds +declare -g -a CCACHE_PASSTHROUGH_VARS=( + CCACHE_REDIS_CONNECT_TIMEOUT + CCACHE_REMOTE_DOMAIN + CCACHE_BASEDIR + CCACHE_REMOTE_STORAGE + CCACHE_REMOTE_ONLY + CCACHE_READONLY + CCACHE_RECACHE + CCACHE_RESHARE + CCACHE_DISABLE + CCACHE_MAXSIZE + CCACHE_MAXFILES + CCACHE_NAMESPACE + CCACHE_SLOPPINESS + CCACHE_UMASK + CCACHE_LOGFILE + CCACHE_DEBUGLEVEL + CCACHE_STATSLOG + CCACHE_PCH_EXTSUM +) + +# Format host:port, wrapping IPv6 addresses in brackets for URL compatibility (RFC 2732) +function ccache_format_host_port() { + local host="$1" port="$2" + if [[ "${host}" == *:* ]]; then + echo "[${host}]:${port}" + else + echo "${host}:${port}" + fi +} + +# Extract hostname from CCACHE_REMOTE_STORAGE URL (strips scheme, userinfo, port, path) +function ccache_extract_url_host() { + local url="$1" + local after_scheme="${url#*://}" + # Strip userinfo if present + if [[ "${after_scheme}" == *@* ]]; then + after_scheme="${after_scheme##*@}" + fi + local host + # Handle bracketed IPv6: [addr]:port + if [[ "${after_scheme}" == \[* ]]; then + host="${after_scheme#\[}" + host="${host%%\]*}" + else + # Strip port, path, and ccache attributes + host="${after_scheme%%[:\/|]*}" + fi + echo "${host}" +} + +# Discover ccache remote storage via DNS-SD (mDNS/Avahi) or DNS SRV records. +# Looks for _ccache._tcp services with TXT records: type=http|redis, path=/... +# Prefers Redis over HTTP when multiple services are found. +# Sets CCACHE_REMOTE_STORAGE on success, returns 1 if nothing found. +function ccache_discover_remote_storage() { + # Method 1: DNS-SD browse on local network (requires avahi-browse) + if command -v avahi-browse &>/dev/null; then + local browse_output + browse_output=$(timeout 5 avahi-browse -rpt _ccache._tcp 2>/dev/null || true) + if [[ -n "${browse_output}" ]]; then + # Parse resolved lines: =;IFACE;PROTO;NAME;TYPE;DOMAIN;HOSTNAME;ADDRESS;PORT;"txt"... + # Prefer IPv4 (proto=IPv4), prefer type=redis over type=http + local redis_url="" http_url="" + local redis_host="" redis_host_ip="" http_host="" http_host_ip="" + while IFS=';' read -r status iface proto name stype domain hostname address port txt_rest; do + [[ "${status}" == "=" && "${proto}" == "IPv4" ]] || continue + local svc_type="" svc_path="" + # Parse TXT records from remaining fields + if [[ "${txt_rest}" =~ \"type=([a-z]+)\" ]]; then + svc_type="${BASH_REMATCH[1]}" + fi + if [[ "${txt_rest}" =~ \"path=([^\"]+)\" ]]; then + svc_path="${BASH_REMATCH[1]}" + fi + # Use hostname for URL (Docker --add-host resolves it), fall back to address + local svc_host="${hostname%.local}" + svc_host="${svc_host%.}" + [[ -z "${svc_host}" ]] && svc_host="${address}" + if [[ "${svc_type}" == "redis" ]]; then + redis_url="redis://${svc_host}:${port}|connect-timeout=${CCACHE_REDIS_CONNECT_TIMEOUT}" + redis_host="${svc_host}" + redis_host_ip="${address}" + elif [[ "${svc_type}" == "http" ]]; then + http_url="http://${svc_host}:${port}${svc_path}" + http_host="${svc_host}" + http_host_ip="${address}" + fi + done <<< "${browse_output}" + # Redis preferred over HTTP; set hostname->IP mapping for Docker --add-host + if [[ -n "${redis_url}" ]]; then + export CCACHE_REMOTE_STORAGE="${redis_url}" + declare -g CCACHE_REMOTE_HOST="${redis_host}" + declare -g CCACHE_REMOTE_HOST_IP="${redis_host_ip}" + display_alert "DNS-SD: discovered Redis ccache" "$(ccache_mask_storage_url "${CCACHE_REMOTE_STORAGE}")" "info" + return 0 + elif [[ -n "${http_url}" ]]; then + export CCACHE_REMOTE_STORAGE="${http_url}" + declare -g CCACHE_REMOTE_HOST="${http_host}" + declare -g CCACHE_REMOTE_HOST_IP="${http_host_ip}" + display_alert "DNS-SD: discovered HTTP ccache" "$(ccache_mask_storage_url "${CCACHE_REMOTE_STORAGE}")" "info" + return 0 + fi + fi + fi + + # Method 2: DNS SRV record for remote setups (CCACHE_REMOTE_DOMAIN must be set) + if [[ -n "${CCACHE_REMOTE_DOMAIN}" ]] && command -v dig &>/dev/null; then + local srv_output + srv_output=$(dig +short SRV "_ccache._tcp.${CCACHE_REMOTE_DOMAIN}" 2>/dev/null || true) + if [[ -n "${srv_output}" ]]; then + local srv_port srv_host + # SRV format: priority weight port target + read -r _ _ srv_port srv_host <<< "${srv_output}" + srv_host="${srv_host%.}" # strip trailing dot + if [[ -n "${srv_host}" && -n "${srv_port}" ]]; then + # Check TXT record for service type and path + local txt_output svc_type="redis" svc_path="" + txt_output=$(dig +short TXT "_ccache._tcp.${CCACHE_REMOTE_DOMAIN}" 2>/dev/null || true) + if [[ "${txt_output}" =~ type=([a-z]+) ]]; then + svc_type="${BASH_REMATCH[1]}" + fi + if [[ "${txt_output}" =~ path=([^\"[:space:]]+) ]]; then + svc_path="${BASH_REMATCH[1]}" + fi + local host_port + host_port=$(ccache_format_host_port "${srv_host}" "${srv_port}") + if [[ "${svc_type}" == "http" ]]; then + export CCACHE_REMOTE_STORAGE="http://${host_port}${svc_path}" + else + export CCACHE_REMOTE_STORAGE="redis://${host_port}|connect-timeout=${CCACHE_REDIS_CONNECT_TIMEOUT}" + fi + display_alert "DNS SRV: discovered ccache" "$(ccache_mask_storage_url "${CCACHE_REMOTE_STORAGE}")" "info" + return 0 + fi + fi + fi + + # Method 3: Legacy fallback - resolve ccache.local hostname + local ccache_ip + ccache_ip=$(getent hosts ccache.local 2>/dev/null | awk '{print $1; exit}' || true) + if [[ -n "${ccache_ip}" ]]; then + local host_port + host_port=$(ccache_format_host_port "${ccache_ip}" "6379") + export CCACHE_REMOTE_STORAGE="redis://${host_port}|connect-timeout=${CCACHE_REDIS_CONNECT_TIMEOUT}" + display_alert "mDNS: discovered ccache" "$(ccache_mask_storage_url "${CCACHE_REMOTE_STORAGE}")" "info" + return 0 + fi + + return 1 +} + +# Query Redis stats (keys count and memory usage) +function ccache_get_redis_stats() { + local ip="$1" + local port="${2:-6379}" + local password="$3" + local stats="" + + if command -v redis-cli &>/dev/null; then + local auth_args=() + [[ -n "${password}" ]] && auth_args+=(-a "${password}" --no-auth-warning) + local keys mem + keys=$(timeout 2 redis-cli -h "$ip" -p "$port" "${auth_args[@]}" DBSIZE 2>/dev/null | grep -oE '[0-9]+' || true) + mem=$(timeout 2 redis-cli -h "$ip" -p "$port" "${auth_args[@]}" INFO memory 2>/dev/null | grep "used_memory_human" | cut -d: -f2 | tr -d '[:space:]' || true) + if [[ -n "$keys" ]]; then + stats="keys=${keys:-0}, mem=${mem:-?}" + fi + else + # Fallback: try netcat for basic connectivity check + if nc -z -w 2 "$ip" "$port" 2>/dev/null; then + stats="reachable (redis-cli not installed for detailed stats)" + fi + fi + echo "$stats" +} + +# Check HTTP/WebDAV storage reachability via HEAD request +function ccache_get_http_stats() { + local url="$1" + local stats="" + local http_code + http_code=$(timeout 3 curl -s -o /dev/null -w "%{http_code}" -X HEAD "${url}" 2>/dev/null || true) + if [[ -n "${http_code}" && "${http_code}" != "000" ]]; then + stats="reachable (HTTP ${http_code})" + fi + echo "$stats" +} + +# Query remote storage stats based on URL scheme (redis:// or http://) +# Parses userinfo (user:pass@) from Redis URLs to pass credentials to redis-cli +function ccache_get_remote_stats() { + local url="$1" + if [[ "${url}" =~ ^redis:// ]]; then + local password="" host="" port="6379" + # Strip scheme and attributes + local authority="${url#redis://}" + authority="${authority%%|*}" + # Extract password from userinfo (before last @) + if [[ "${authority}" =~ ^(.+)@(.+)$ ]]; then + local userinfo="${BASH_REMATCH[1]}" + authority="${BASH_REMATCH[2]}" + # password is after : in userinfo (user:pass or just :pass) + [[ "${userinfo}" == *:* ]] && password="${userinfo#*:}" + fi + # Parse host:port (IPv6 in brackets or plain) + if [[ "${authority}" =~ ^\[([^]]+)\]:?([0-9]*) ]]; then + host="${BASH_REMATCH[1]}" + [[ -n "${BASH_REMATCH[2]}" ]] && port="${BASH_REMATCH[2]}" + elif [[ "${authority}" =~ ^([^:]+):?([0-9]*) ]]; then + host="${BASH_REMATCH[1]}" + [[ -n "${BASH_REMATCH[2]}" ]] && port="${BASH_REMATCH[2]}" + fi + [[ -n "${host}" ]] && ccache_get_redis_stats "${host}" "${port}" "${password}" + elif [[ "${url}" =~ ^https?:// ]]; then + # Strip ccache attributes after | for the URL + ccache_get_http_stats "${url%%|*}" + fi +} + +# Mask credentials in storage URLs to avoid leaking secrets into build logs +# Handles any URI scheme with userinfo component (e.g., redis://user:pass@host) +# Uses last @ as delimiter since userinfo may contain special characters +function ccache_mask_storage_url() { + local url="$1" + if [[ "${url}" =~ ^([a-zA-Z][a-zA-Z0-9+.-]*://)(.+)@([^@]+)$ ]]; then + echo "${BASH_REMATCH[1]}****@${BASH_REMATCH[3]}" + else + echo "${url}" + fi +} + +# Validate that credentials in storage URL do not contain characters unsafe for URL parsing. +# Passwords with / + = or spaces break URL parsing in ccache and in our mask function. +# Returns 1 and displays error if invalid characters are found. +function ccache_validate_storage_url() { + local url="$1" + # Extract userinfo (part between :// and last @) + if [[ "${url}" =~ ^[a-zA-Z][a-zA-Z0-9+.-]*://(.+)@[^@]+$ ]]; then + local userinfo="${BASH_REMATCH[1]}" + if [[ "${userinfo}" =~ [/+=[:space:]] ]]; then + display_alert "Password contains URL-unsafe characters (/ + = or spaces)" \ + "Generate a safe password: openssl rand -hex 24" "err" + return 1 + fi + fi + return 0 +} + +# This runs on the HOST just before Docker container is launched. +# Resolves 'ccache.local' via mDNS (requires Avahi on server publishing this hostname +# Docker hook: resolve hostnames and handle loopback for container access. +# mDNS/local DNS may not work inside Docker, so we resolve on host and +# pass the mapping via --add-host. Loopback addresses are rewritten to +# host.docker.internal. +function host_pre_docker_launch__setup_remote_ccache() { + if [[ -n "${CCACHE_REMOTE_STORAGE}" ]]; then + ccache_validate_storage_url "${CCACHE_REMOTE_STORAGE}" || return 1 + display_alert "Remote ccache pre-configured" "$(ccache_mask_storage_url "${CCACHE_REMOTE_STORAGE}")" "info" + elif ! ccache_discover_remote_storage; then + display_alert "Remote ccache not found on host" "no service discovered" "debug" + fi + + # Show backend stats if we have a remote storage URL + if [[ -n "${CCACHE_REMOTE_STORAGE}" ]]; then + local stats + stats=$(ccache_get_remote_stats "${CCACHE_REMOTE_STORAGE}") + if [[ -n "$stats" ]]; then + display_alert "Remote ccache stats" "${stats}" "info" + fi + fi + + # Ensure hostname in CCACHE_REMOTE_STORAGE is resolvable inside Docker. + # Docker containers may not have access to host mDNS/local DNS. + if [[ -n "${CCACHE_REMOTE_STORAGE}" ]]; then + local _host + _host=$(ccache_extract_url_host "${CCACHE_REMOTE_STORAGE}") + if [[ -n "${_host}" ]]; then + # Loopback addresses: rewrite to host.docker.internal + if [[ "${_host}" == "localhost" || "${_host}" == "127.0.0.1" || "${_host}" == "::1" ]]; then + CCACHE_REMOTE_STORAGE="${CCACHE_REMOTE_STORAGE//localhost/host.docker.internal}" + CCACHE_REMOTE_STORAGE="${CCACHE_REMOTE_STORAGE//127.0.0.1/host.docker.internal}" + CCACHE_REMOTE_STORAGE="${CCACHE_REMOTE_STORAGE//\[::1\]/host.docker.internal}" + DOCKER_EXTRA_ARGS+=("--add-host=host.docker.internal:host-gateway") + display_alert "Rewriting loopback URL for Docker" "$(ccache_mask_storage_url "${CCACHE_REMOTE_STORAGE}")" "info" + # Hostname (not IP): resolve on host and pass via --add-host + elif [[ "${_host}" =~ [a-zA-Z] ]]; then + local _resolved_ip="${CCACHE_REMOTE_HOST_IP:-}" + # If not from discovery, resolve now; prefer IPv4 (Docker bridge often lacks IPv6) + if [[ -z "${_resolved_ip}" || "${CCACHE_REMOTE_HOST}" != "${_host}" ]]; then + _resolved_ip=$(getent ahostsv4 "${_host}" 2>/dev/null | awk '{print $1; exit}' || true) + [[ -z "${_resolved_ip}" ]] && _resolved_ip=$(getent hosts "${_host}" 2>/dev/null | awk '{print $1; exit}' || true) + fi + if [[ -n "${_resolved_ip}" ]]; then + DOCKER_EXTRA_ARGS+=("--add-host=${_host}:${_resolved_ip}") + display_alert "Docker --add-host" "${_host}:${_resolved_ip}" "info" + else + display_alert "Cannot resolve hostname for Docker" "${_host}" "wrn" + fi + fi + fi + fi + + # Pass all set CCACHE_* variables to Docker + local var val + for var in "${CCACHE_PASSTHROUGH_VARS[@]}"; do + val="${!var}" + if [[ -n "${val}" ]]; then + DOCKER_EXTRA_ARGS+=("--env" "${var}=${val}") + local log_val="${val}" + [[ "${var}" == "CCACHE_REMOTE_STORAGE" ]] && log_val="$(ccache_mask_storage_url "${val}")" + display_alert "Docker env" "${var}=${log_val}" "debug" + fi + done +} + +# Hook: Show ccache remote storage statistics after each compilation (kernel, uboot) +function ccache_post_compilation__show_remote_stats() { + if [[ -n "${CCACHE_REMOTE_STORAGE}" ]]; then + local stats_output pct + local read_hit read_miss write error + stats_output=$(ccache --print-stats 2>&1 || true) + read_hit=$(ccache_get_stat "$stats_output" "remote_storage_read_hit") + read_miss=$(ccache_get_stat "$stats_output" "remote_storage_read_miss") + write=$(ccache_get_stat "$stats_output" "remote_storage_write") + error=$(ccache_get_stat "$stats_output" "remote_storage_error") + pct=$(ccache_hit_pct "$read_hit" "$read_miss") + display_alert "Remote ccache result" "hit=${read_hit} miss=${read_miss} write=${write} err=${error} (${pct}%)" "info" + fi +} + +# This runs inside Docker (or native build) during configuration +function extension_prepare_config__setup_remote_ccache() { + # Enable ccache with a consistent cache directory ($SRC/cache/ccache). + # PRIVATE_CCACHE ensures the same CCACHE_DIR is used in native and Docker builds, + # avoiding fragmented caches in /root/.cache/ccache vs $SRC/cache/ccache. + declare -g USE_CCACHE=yes + declare -g PRIVATE_CCACHE=yes + + # If CCACHE_REMOTE_STORAGE was passed from host (via Docker env), it's already set + if [[ -n "${CCACHE_REMOTE_STORAGE}" ]]; then + ccache_validate_storage_url "${CCACHE_REMOTE_STORAGE}" || return 1 + display_alert "Remote ccache configured" "$(ccache_mask_storage_url "${CCACHE_REMOTE_STORAGE}")" "info" + return 0 + fi + + # For native (non-Docker) builds, try to discover + if ccache_discover_remote_storage; then + return 0 + fi + + if [[ "${CCACHE_REMOTE_ONLY}" == "yes" ]]; then + display_alert "Remote ccache not available" "CCACHE_REMOTE_ONLY=yes but no remote found, ccache will be ineffective" "wrn" + else + display_alert "Remote ccache not available" "using local cache only" "debug" + fi + + return 0 +} + +# Inject all set CCACHE_PASSTHROUGH_VARS into the given make environment array +# Uses bash nameref to write into the caller's array variable +function ccache_inject_envs() { + local -n target_array="$1" + local label="$2" + local var val + for var in "${CCACHE_PASSTHROUGH_VARS[@]}"; do + val="${!var}" + if [[ -n "${val}" ]]; then + target_array+=("${var}=${val@Q}") + local log_val="${val}" + [[ "${var}" == "CCACHE_REMOTE_STORAGE" ]] && log_val="$(ccache_mask_storage_url "${val}")" + display_alert "${label}: ${var}" "${log_val}" "debug" + fi + done +} + +# This hook runs right before kernel make - add ccache env vars to make environment. +# Required because kernel build uses 'env -i' which clears all environment variables. +function kernel_make_config__add_ccache_remote_storage() { + ccache_inject_envs common_make_envs "Kernel make" +} + +# This hook runs right before u-boot make - add ccache env vars to make environment. +# Required because u-boot build uses 'env -i' which clears all environment variables. +function uboot_make_config__add_ccache_remote_storage() { + ccache_inject_envs uboot_make_envs "U-boot make" +} diff --git a/extensions/ccache-remote/misc/avahi/ccache-redis.service b/extensions/ccache-remote/misc/avahi/ccache-redis.service new file mode 100644 index 0000000000..f3df80354a --- /dev/null +++ b/extensions/ccache-remote/misc/avahi/ccache-redis.service @@ -0,0 +1,10 @@ + + + + ccache-redis + + _ccache._tcp + 6379 + type=redis + + diff --git a/extensions/ccache-remote/misc/avahi/ccache-webdav.service b/extensions/ccache-remote/misc/avahi/ccache-webdav.service new file mode 100644 index 0000000000..68b9806cb8 --- /dev/null +++ b/extensions/ccache-remote/misc/avahi/ccache-webdav.service @@ -0,0 +1,11 @@ + + + + ccache-webdav + + _ccache._tcp + 8088 + type=http + path=/ccache/ + + diff --git a/extensions/ccache-remote/misc/nginx/ccache-webdav.conf b/extensions/ccache-remote/misc/nginx/ccache-webdav.conf new file mode 100644 index 0000000000..eac57506fb --- /dev/null +++ b/extensions/ccache-remote/misc/nginx/ccache-webdav.conf @@ -0,0 +1,27 @@ +# nginx WebDAV configuration for ccache remote storage +# Copy to /etc/nginx/sites-available/ccache-webdav +# Then: ln -s /etc/nginx/sites-available/ccache-webdav /etc/nginx/sites-enabled/ +# +# Prepare storage directory: +# mkdir -p /var/cache/ccache-webdav/ccache +# chown -R www-data:www-data /var/cache/ccache-webdav +# +# WARNING: No authentication configured. +# Use ONLY in a fully trusted private network. +# For auth, add auth_basic directives. See nginx WebDAV documentation. +# Note: ccache does not support HTTPS directly. Use a reverse proxy for TLS. +# Requires: nginx with http_dav_module (e.g. nginx-full or nginx-extras on Debian/Ubuntu) + +server { + listen 8088; + server_name _; + root /var/cache/ccache-webdav; + + location /ccache/ { + dav_methods PUT DELETE; + create_full_put_path on; + dav_access user:rw group:rw all:r; + client_max_body_size 100M; + autoindex on; + } +} diff --git a/extensions/ccache-remote/misc/redis/redis-ccache-insecure.conf b/extensions/ccache-remote/misc/redis/redis-ccache-insecure.conf new file mode 100644 index 0000000000..145fb4fb90 --- /dev/null +++ b/extensions/ccache-remote/misc/redis/redis-ccache-insecure.conf @@ -0,0 +1,13 @@ +# Redis configuration for ccache remote storage +# Copy to /etc/redis/redis-ccache.conf or merge into /etc/redis/redis.conf +# +# WARNING: This configuration is INSECURE - Redis is open without authentication. +# Use ONLY in a fully trusted private network with no internet access. +# For secure setup (password, TLS, ACL), see: https://redis.io/docs/management/security/ + +bind 0.0.0.0 :: +protected-mode no + +# Limit memory and use LRU eviction so old cache entries are purged automatically +maxmemory 4G +maxmemory-policy allkeys-lru diff --git a/extensions/ccache-remote/misc/redis/redis-ccache.conf b/extensions/ccache-remote/misc/redis/redis-ccache.conf new file mode 100644 index 0000000000..eadc467b51 --- /dev/null +++ b/extensions/ccache-remote/misc/redis/redis-ccache.conf @@ -0,0 +1,35 @@ +# Redis configuration for ccache remote storage +# Copy to /etc/redis/redis-ccache.conf or merge into /etc/redis/redis.conf +# +# For advanced security (TLS, ACL, rename-command), see: +# https://redis.io/docs/management/security/ + +bind 0.0.0.0 :: + +# --- Authentication (recommended) --- +# Clients connect with: CCACHE_REMOTE_STORAGE="redis://default:YOUR_PASSWORD@host:6379" +# Generate a URL-safe password (hex only, no special characters): +# openssl rand -hex 24 +# WARNING: Do not use base64 passwords (openssl rand -base64) — they contain +# / + = characters that break redis:// URL parsing. +# +# Redis 6+: use ACL user entry (preferred, overrides requirepass): +# user default on >CHANGE_ME sanitize-payload ~* &* +@all +# +# Redis < 6: use requirepass instead: +# requirepass CHANGE_ME +# +# With authentication set, protected-mode can stay enabled (default). +# It blocks unauthenticated access from non-loopback interfaces. +protected-mode yes + +# --- No authentication (trusted network only) --- +# If all build hosts and the Redis server are on a fully isolated private +# network and you do not need access control, you can skip auth: +# user default on nopass sanitize-payload ~* &* +@all +# protected-mode no +# WARNING: Anyone on the network can read/write/flush the cache. + +# Limit memory and use LRU eviction so old cache entries are purged automatically +maxmemory 4G +maxmemory-policy allkeys-lru diff --git a/extensions/ccache-remote/misc/systemd/ccache-avahi-redis.service b/extensions/ccache-remote/misc/systemd/ccache-avahi-redis.service new file mode 100644 index 0000000000..d6e097718e --- /dev/null +++ b/extensions/ccache-remote/misc/systemd/ccache-avahi-redis.service @@ -0,0 +1,13 @@ +[Unit] +Description=Publish ccache Redis service via Avahi DNS-SD +After=avahi-daemon.service redis-server.service +BindsTo=redis-server.service + +[Service] +Type=simple +ExecStart=/usr/bin/avahi-publish-service "ccache-redis" _ccache._tcp 6379 type=redis +Restart=on-failure +RestartSec=5 + +[Install] +WantedBy=redis-server.service diff --git a/extensions/ccache-remote/misc/systemd/ccache-avahi-webdav.service b/extensions/ccache-remote/misc/systemd/ccache-avahi-webdav.service new file mode 100644 index 0000000000..d048f42fe0 --- /dev/null +++ b/extensions/ccache-remote/misc/systemd/ccache-avahi-webdav.service @@ -0,0 +1,13 @@ +[Unit] +Description=Publish ccache WebDAV service via Avahi DNS-SD +After=avahi-daemon.service nginx.service +BindsTo=nginx.service + +[Service] +Type=simple +ExecStart=/usr/bin/avahi-publish-service "ccache-webdav" _ccache._tcp 8088 type=http path=/ccache/ +Restart=on-failure +RestartSec=5 + +[Install] +WantedBy=nginx.service diff --git a/lib/functions/compilation/ccache.sh b/lib/functions/compilation/ccache.sh index 43f854542a..a3a953adbf 100644 --- a/lib/functions/compilation/ccache.sh +++ b/lib/functions/compilation/ccache.sh @@ -7,6 +7,43 @@ # This file is a part of the Armbian Build Framework # https://github.com/armbian/build/ +# Parse a single numeric field from "ccache --print-stats" tab-separated output +# Returns 0 if field not found or not numeric +function ccache_get_stat() { + local stats_output="$1" field="$2" + local val + val=$(echo "$stats_output" | grep "^${field}" | cut -f2 || true) + [[ "${val}" =~ ^[0-9]+$ ]] || val=0 + echo "$val" +} + +# Calculate hit percentage from hit and miss counts +function ccache_hit_pct() { + local hit="$1" miss="$2" + local total=$(( hit + miss )) + if [[ $total -gt 0 ]]; then + echo $(( hit * 100 / total )) + else + echo 0 + fi +} + +# Helper function to show ccache stats - used as cleanup handler for interruption case +function ccache_show_compilation_stats() { + local stats_output direct_hit direct_miss pct + stats_output=$(ccache --print-stats 2>&1 || true) + direct_hit=$(ccache_get_stat "$stats_output" "direct_cache_hit") + direct_miss=$(ccache_get_stat "$stats_output" "direct_cache_miss") + pct=$(ccache_hit_pct "$direct_hit" "$direct_miss") + display_alert "Ccache result" "hit=${direct_hit} miss=${direct_miss} (${pct}%)" "info" + + # Hook for extensions to show additional stats (e.g., remote storage) + call_extension_method "ccache_post_compilation" <<- 'CCACHE_POST_COMPILATION' + *called after ccache-wrapped compilation completes (success or failure)* + Useful for displaying remote cache statistics or other post-build info. + CCACHE_POST_COMPILATION +} + function do_with_ccache_statistics() { display_alert "Clearing ccache statistics" "ccache" "ccache" @@ -35,8 +72,20 @@ function do_with_ccache_statistics() { run_host_command_logged ccache --show-config "&&" sync fi + # Register cleanup handler to show stats even if build is interrupted + add_cleanup_handler ccache_show_compilation_stats + display_alert "Running ccache'd build..." "ccache" "ccache" - "$@" + local build_exit_code=0 + "$@" || build_exit_code=$? + + # Show stats and remove from cleanup handlers (so it doesn't run twice on exit) + execute_and_remove_cleanup_handler ccache_show_compilation_stats + + # Re-raise the error if the build failed + if [[ ${build_exit_code} -ne 0 ]]; then + return ${build_exit_code} + fi if [[ "${SHOW_CCACHE}" == "yes" ]]; then display_alert "Display ccache statistics" "ccache" "ccache" diff --git a/lib/functions/compilation/kernel-make.sh b/lib/functions/compilation/kernel-make.sh index 5f0ca10d2b..8142225a0d 100644 --- a/lib/functions/compilation/kernel-make.sh +++ b/lib/functions/compilation/kernel-make.sh @@ -31,7 +31,7 @@ function run_kernel_make_internal() { # If CCACHE_DIR is set, pass it to the kernel build; Pass the ccache dir explicitly, since we'll run under "env -i" if [[ -n "${CCACHE_DIR}" ]]; then - common_make_envs+=("CCACHE_DIR='${CCACHE_DIR}'") + common_make_envs+=("CCACHE_DIR=${CCACHE_DIR@Q}") fi # Add the distcc envs, if any. @@ -74,7 +74,19 @@ function run_kernel_make_internal() { common_make_params_quoted+=("${llvm_flag}") fi - # Allow extensions to modify make parameters and environment variables + # Hook order: kernel_make_config runs first (generic extension config), + # then custom_kernel_make_params (user/board overrides can take precedence). + call_extension_method "kernel_make_config" <<- 'KERNEL_MAKE_CONFIG' + *Hook to customize kernel make environment and parameters* + Called right before invoking make for kernel compilation. + Available arrays to modify: + - common_make_envs[@]: environment variables passed via "env -i" (e.g., CCACHE_REMOTE_STORAGE) + - common_make_params_quoted[@]: make command parameters (e.g., custom flags) + Available read-only variables: + - KERNEL_COMPILER, ARCHITECTURE, BRANCH, LINUXFAMILY + KERNEL_MAKE_CONFIG + + # Runs after kernel_make_config — allows user/board overrides to take precedence call_extension_method "custom_kernel_make_params" <<- 'CUSTOM_KERNEL_MAKE_PARAMS' *Customize kernel make parameters before compilation* Called after all standard make parameters are set but before invoking make. diff --git a/lib/functions/compilation/uboot.sh b/lib/functions/compilation/uboot.sh index a6be864605..23c3728eb5 100644 --- a/lib/functions/compilation/uboot.sh +++ b/lib/functions/compilation/uboot.sh @@ -253,6 +253,27 @@ function compile_uboot_target() { "PYTHONPATH=\"${PYTHON3_INFO[MODULES_PATH]}:${PYTHONPATH}\"" # Insert the pip modules downloaded by Armbian into PYTHONPATH (needed e.g. for pyelftools) ) + # Pass the ccache directories explicitly, since we'll run under "env -i" + if [[ -n "${CCACHE_DIR}" ]]; then + uboot_make_envs+=("CCACHE_DIR=${CCACHE_DIR@Q}") + fi + if [[ -n "${CCACHE_TEMPDIR}" ]]; then + uboot_make_envs+=("CCACHE_TEMPDIR=${CCACHE_TEMPDIR@Q}") + fi + + # workaround when two compilers are needed + cross_compile="CROSS_COMPILE=\"${CCACHE:+$CCACHE }$UBOOT_COMPILER\"" + # When UBOOT_TOOLCHAIN2 is set, the board's uboot_custom_postprocess handles compilers; + # pass a harmless dummy env var since empty make parameters cause errors + [[ -n $UBOOT_TOOLCHAIN2 ]] && cross_compile="ARMBIAN=foe" + + call_extension_method "uboot_make_config" <<- 'UBOOT_MAKE_CONFIG' + *Hook to customize u-boot make environment* + Called right before invoking make for u-boot compilation. + Available array to modify: + - uboot_make_envs[@]: environment variables passed via "env -i" (e.g., CCACHE_REMOTE_STORAGE) + UBOOT_MAKE_CONFIG + display_alert "${uboot_prefix}Compiling u-boot" "${version} ${target_make} with gcc '${gcc_version_main}'" "info" declare -g if_error_detail_message="${uboot_prefix}Failed to build u-boot ${version} ${target_make}" do_with_ccache_statistics run_host_command_logged_long_running \ diff --git a/lib/functions/configuration/compilation-config.sh b/lib/functions/configuration/compilation-config.sh index f917689d97..dfdaec04ae 100644 --- a/lib/functions/configuration/compilation-config.sh +++ b/lib/functions/configuration/compilation-config.sh @@ -17,6 +17,11 @@ function prepare_compilation_vars() { # private ccache directory to avoid permission issues when using build script with "sudo" # see https://ccache.samba.org/manual.html#_sharing_a_cache for alternative solution [[ $PRIVATE_CCACHE == yes ]] && export CCACHE_DIR=$SRC/cache/ccache # actual export + + # Set default umask for ccache to allow write access for all users (enables cache sharing) + # CCACHE_UMASK=000 creates files with permissions 666 (rw-rw-rw-) and dirs with 777 (rwxrwxrwx) + # Only set this for shared cache, not for private cache + [[ -z "${CCACHE_UMASK}" && "${PRIVATE_CCACHE}" != "yes" ]] && export CCACHE_UMASK=000 else CCACHE="" fi diff --git a/lib/functions/host/docker.sh b/lib/functions/host/docker.sh index 85f8220c0d..0fabf92b87 100644 --- a/lib/functions/host/docker.sh +++ b/lib/functions/host/docker.sh @@ -572,8 +572,23 @@ function docker_cli_prepare_launch() { display_alert "Not running in a terminal" "not passing through stdin to Docker" "debug" fi - # if DOCKER_EXTRA_ARGS is an array and has more than zero elements, add its contents to the DOCKER_ARGS array - if [[ "${DOCKER_EXTRA_ARGS[*]+isset}" == "isset" && "${#DOCKER_EXTRA_ARGS[@]}" -gt 0 ]]; then + # Preserve any pre-existing DOCKER_EXTRA_ARGS (e.g., from user environment) and let extensions append + declare -g -a DOCKER_EXTRA_ARGS=("${DOCKER_EXTRA_ARGS[@]+"${DOCKER_EXTRA_ARGS[@]}"}") + + # Hook for extensions to add Docker arguments before launch + call_extension_method "host_pre_docker_launch" <<- 'HOST_PRE_DOCKER_LAUNCH' + *run on host just before Docker container is launched* + Extensions can add Docker arguments by appending to DOCKER_EXTRA_ARGS array. + Each array element should be a complete argument (e.g., "--env", "MY_VAR=value" as separate elements). + Example: DOCKER_EXTRA_ARGS+=("--env" "MY_VAR=value" "--mount" "type=bind,src=/a,dst=/b") + Available variables: + - DOCKER_ARGS[@]: current Docker arguments (do not modify directly) + - DOCKER_EXTRA_ARGS[@]: array to append extra arguments for docker run + - DOCKER_ARMBIAN_TARGET_PATH: path inside container (/armbian) + HOST_PRE_DOCKER_LAUNCH + + # Add DOCKER_EXTRA_ARGS to DOCKER_ARGS if any were added by extensions + if [[ "${#DOCKER_EXTRA_ARGS[@]}" -gt 0 ]]; then display_alert "Adding extra Docker arguments" "${DOCKER_EXTRA_ARGS[*]}" "debug" DOCKER_ARGS+=("${DOCKER_EXTRA_ARGS[@]}") fi