The container registry is always going to rewrite parts of the image in an uncontrollable way. As far as we know, it's not possible to download a 1:1 copy of a build output from the container registry (until someone figures this out).
As far as I understand it, it also explains why `diffoci --semantic` is a thing and why it's generally considered "good enough" (give current constraints).
This is more informative than anything, we're primarily looking at providing a bit for bit reproducible image. The reproducibility of the userspace is not fully guaranteed at the moment
this commit takes the relevant repro steps from the wsl image, and wraps
breaking changes to only affect the :repro image
testing reproducability is not yet included, so we can discuss the
approach first
Currently the release stage does 3-4 fairly distinct things:
- uploads the rootfs tarballs to the Gitlab Generic Packages repo
- creates and commits Dockerfile* to the release branch, for the
Dockerhub/Github publishing
- creates a Gitlab release, referencing the Packages artefacts and the
release branch
- updates the Dockerhub webpage
Currently these are intertwined, where the docker website is updated
first, the packages are uploaded, then the Dockerfiles are created and
committed and finally the release is made.
As result we intermix the tools required and permissions.
Split out the Gitlab release into it's own stage, using the yaml driven
format. This should save us some escaping/quoting mayhem.
In addition, move the Dockerhub webpage update as part of the github
publishing, since they require the same permissions.
Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
The bug that mandated the get-public-download-for-generic-package.sh
use has been fixed a while ago.
Directly use the URL variable and drop the no longer needed script.
Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
Currently things are a bit mixed up - just rename all the files to
include the BUILD_VERSION and consistently use the rootfs_file variable.
Throw in some quotes while in there.
Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
The image template, has a before_script stage which is overridden by all
instances that use it. Just drop it - it's rather misleading.
Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
So it can be useful for the wider community. I've intentionally left it
out as separate commit, since:
- it is untested
- there is lots of duplication that can/should be fixed
- if publishing seems like an overkill, people can locally still use it
Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
Can be used to build 32bit (aka multilib) packages. With all the
simplification and de-duplication done with the previous commits, this
turns out to be a fairly simple job ;-)
Notes:
- there is currently no test stage for this container
- the release bits will come with next commit
v2:
- preserve extra.conf for base/base-devel
Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
Bonus point, we actually error out when the git command fails.
v2:
- fix curl quoting
- sed match-complete-line-and-remove
- inline update make-dockerfile.sh variables
Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
The field is an inline comment, which by default is not present in the
podman log. Plus we do have the exact URL list a couple of lines further
down.
Haven't seen any other Dockerfile have one either, so let's nuke it.
Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
Currently the TEMPLATE_ROOTFS_DOWNLOAD handling is overly complicated.
For the local builds, we set a ROOTFS=$GROUP.tar.zst. While for remote
builds, we:
- invoke curl to fetch the remote tarball - ok
- do curl and shell escaping contortions to prints the filename - ehhh
- that we already now
- and rely upon to not change, otherwise sha256sum will fail
Just use a dummy "true", for the local builds and a normal curl
command otherwise.
v2:
- don't call curl in a sub-shell - no longer needed
Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
The option was moved the to template and an earlier commit
reintroduced/copied them back seemingly by mistake.
Cc: nl6720 <nl6720@gmail.com>
Fixes: ca9957f ("Do not use secure runners on forks")
Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
We use git in the makefile to create the container revision label.
This did not have any affect outside the local containers, since the
official ones use the CI_COMMIT_TAG variable from the CI.
Fixes: 5ec09f5 ("gitlab-ci: install devtools without its dependencies")
Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
The file may be useful locally, but for the other stages we don't need
it. Explicitly remove it since it tends to be 3-4x the size of
everything else.
v2
- switch from rm to artefacts:exclude:
Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
