garage/doc/book/development/miscellaneous-notes.md

+++ title = "Miscellaneous notes" weight = 20 +++

Nix

Nix has no armv7 + musl toolchains but armv7l is backward compatible with armv6l.

cat > $HOME/.awsrc <<EOF
export AWS_ACCESS_KEY_ID="xxx"
export AWS_SECRET_ACCESS_KEY="xxx"
EOF

# source each time you want to send on the cache
source ~/.awsrc

# copy garage build dependencies (and not only the output)
nix-build
nix-store -qR --include-outputs $(nix-instantiate default.nix) 
  | xargs nix copy --to 's3://nix?endpoint=garage.deuxfleurs.fr&region=garage'
  
# copy shell dependencies
nix-build shell.nix -A inputDerivation
nix copy $(nix-store -qR result/) --to 's3://nix?endpoint=garage.deuxfleurs.fr&region=garage' 

More example of nix-copy

# nix-build produces a result/ symlink
nix copy result/ --to 's3://nix?endpoint=garage.deuxfleurs.fr&region=garage'

# alternative ways to use nix copy
nix copy nixpkgs.garage --to ...
nix copy /nix/store/3rbb9qsc2w6xl5xccz5ncfhy33nzv3dp-crate-garage-0.3.0 --to ...

Clear the cache:

mc rm --recursive --force garage/nix/

---

A desirable nix.conf for a consumer:

substituters = https://cache.nixos.org https://nix.web.deuxfleurs.fr
trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= nix.web.deuxfleurs.fr:eTGL6kvaQn6cDR/F9lDYUIP9nCVR/kkshYfLDJf1yKs=

And now, whenever you run a command like:

nix-shell
nix-build

Our cache will be checked.

Some references about Nix

• https://doc.rust-lang.org/nightly/rustc/platform-support.html
• https://nix.dev/tutorials/cross-compilation
• https://nixos.org/manual/nix/unstable/package-management/s3-substituter.html
• https://fzakaria.com/2020/09/28/nix-copy-closure-your-nix-shell.html
• http://www.lpenz.org/articles/nixchannel/index.html

Woodpecker

Woodpecker can do parallelism both at the step and the pipeline level. At the step level, parallelism is restricted to the same runner.

Building Docker containers

We were:

• Unable to use the official Docker plugin because
  • it requires to mount docker socket in the container but it is not recommended
  • you cant set the platform when building
• Unable to use buildah because it needs CLONE_USERNS capability
• Unable to use the kaniko plugin for Drone as we can't set the target platform
• Unable to use the kaniko container provided by Google as we can't run arbitrary logic: we need to put our secret in .docker/config.json.

Finally we chose to build kaniko through nix and use it in a nix-shell. We then switched to using kaniko from nixpkgs when it was packaged.