build_image_util: Store path list for recreating tmpfiles rules

The removal of files in the overlay present in the lowerdir creates
whiteout entries that mask the lowerdir entries. For those files that
have a tmpfile rule for creation, a reboot would cause the file to be
created in the upperdir, meaning this file is not updated from the
lowerdir when it changes. In addition we have filtered out some tmpfile
rules that caused upcopies (symlinks and directories) which meant that
removing the /etc/resolv.conf symlink didn't bring it back after reboot.
To make files from the lowerdir show up if they have a tmpfile rule that
normally would recreate them we keep a list of whiteout entries that we
clean up on boot. This also prevents freezing files because
systemd-tmpfiles does not need to recreate them in the upperdir.

build_library/build_image_util.sh

@@ -821,9 +821,16 @@ EOF
   done
   sudo "${root_fs_dir}"/usr/sbin/flatcar-tmpfiles "${root_fs_dir}"
   # Now that we used the tmpfiles for creating /etc we delete them because
-  # the L, d, and C entries cause upcopies. Also filter out rules with ! or - but no other modifiers
+  # the L, d, D, and C entries cause upcopies. Also filter out rules with ! or - but no other modifiers
   # like + or = which explicitly recreate files.
-  sudo sed -i '/^[CLd]-*!*-*[ \t]*\/etc\//d' "${root_fs_dir}"/usr/lib/tmpfiles.d/*
+  # But before filtering, first store rules that would recreate missing files
+  # to /usr/share/flatcar/etc-no-whiteouts so that we can ensure that
+  # no overlayfs whiteouts exist for these files (example: /etc/resolv.conf).
+  # These rules are combined with the + modifier in addition.
+  # Other rules like w, e, x, do not create files that don't exist.
+  # Note: '-' must come first in the modifier pattern.
+  grep -Ph '^[fcCdDLvqQpb][-=~^!+]*[ \t]*/etc' "${root_fs_dir}"/usr/lib/tmpfiles.d/* | grep -oP '/etc[^ \t]*' | sudo_clobber "${root_fs_dir}"/usr/share/flatcar/etc-no-whiteouts
+  sudo sed -i '/^[CdDL][-=~^!]*[ \t]*\/etc\//d' "${root_fs_dir}"/usr/lib/tmpfiles.d/*
 
   # SELinux: Label the root filesystem for using 'file_contexts'.
   # The labeling has to be done before moving /etc to /usr/share/flatcar/etc to prevent wrong labels for these files and as

changelog/bugfixes/2023-11-29-recreate-etc-files.md

@@ -0,0 +1 @@
+- Deleted files in `/etc` that have a tmpfiles rule that normally would recreate them will now show up again through the `/etc` lowerdir ([Flatcar#1265](https://github.com/flatcar/Flatcar/issues/1265), [bootengine#79](https://github.com/flatcar/bootengine/pull/79))

sdk_container/src/third_party/coreos-overlay/sys-kernel/bootengine/bootengine-9999.ebuild

@@ -10,7 +10,7 @@ CROS_WORKON_REPO="https://github.com"
 if [[ "${PV}" == 9999 ]]; then
 	KEYWORDS="~amd64 ~arm ~arm64 ~x86"
 else
-	CROS_WORKON_COMMIT="5db3a7eaf06e0d01937689b8bac619db8634b5a4" # flatcar-3602-backport
+	CROS_WORKON_COMMIT="ed7837a59df5775939e655e9e172d71a3c70f18c" # flatcar-3602-backport
 	KEYWORDS="amd64 arm arm64 x86"
 fi