| Message ID | 20250217100024.42995-1-alexander.heinisch@siemens.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | Added support for rootfs-overlay (for development). | expand |
On 17.02.25 11:00, alexander.heinisch@siemens.com wrote: > From: Alexander Heinisch <alexander.heinisch@siemens.com> > > Updateable images based on isar-cip-core come with immutable rootfs > out of the box. During development this oftentimes comes with the penalty > of having to either rebuild images with additional files, packages aso. > to accomplish development and debugging tasks or derive from the production > image in a way that some fundamental features like verity, squashfs, swupdate, aso. > are not part of such. > > Both cases are far from optimal. With this patch series we want to share > our attempt to streamline development by staying as close as possible to > our production image by overlaying the squashfs or erofs based ro filesystem > with an overlay. To make changes redundant accross reboots, we decided > to use a persistent storage option in favour of a tmpfs based approach. > (Although, technically there is no limitation on doing so) > > Keep in mind, support for the root overlay is thought as a development > feature! Thus, it is not thought for production, as it invalidates many > of the properties we have from ro-filesystems. Also, keep in mind that > changes on the overlay may partially hide updates or lead to "unexpected" > results after applying updates. > > Alexander Heinisch (3): > Removed "ro" option from read-only-rootfs's fstab. > Fix return value handling on filesystem check > Added support for rootfs-overlay. > > classes/read-only-rootfs.bbclass | 2 +- > .../files/local-bottom.tmpl | 97 ++++++++++++++++--- > .../initramfs-overlay-hook_0.3.bb | 2 +- > 3 files changed, 87 insertions(+), 14 deletions(-) > Thanks, already applied patches 1 and 2. 3 is still under review and will need at least some rewording. Jan
> On 17.02.25 11:00, alexander.heinisch@siemens.com wrote: > > From: Alexander Heinisch <alexander.heinisch@siemens.com> > > > > Updateable images based on isar-cip-core come with immutable rootfs > > out of the box. During development this oftentimes comes with the > > penalty of having to either rebuild images with additional files, packages aso. > > to accomplish development and debugging tasks or derive from the > > production image in a way that some fundamental features like verity, squashfs, swupdate, aso. > > are not part of such. > > > > Both cases are far from optimal. With this patch series we want to > > share our attempt to streamline development by staying as close as > > possible to our production image by overlaying the squashfs or erofs > > based ro filesystem with an overlay. To make changes redundant accross > > reboots, we decided to use a persistent storage option in favour of a tmpfs based approach. > > (Although, technically there is no limitation on doing so) > > > > Keep in mind, support for the root overlay is thought as a development > > feature! Thus, it is not thought for production, as it invalidates > > many of the properties we have from ro-filesystems. Also, keep in mind > > that changes on the overlay may partially hide updates or lead to "unexpected" > > results after applying updates. > > > > Alexander Heinisch (3): > > Removed "ro" option from read-only-rootfs's fstab. > > Fix return value handling on filesystem check > > Added support for rootfs-overlay. > > > > classes/read-only-rootfs.bbclass | 2 +- > > .../files/local-bottom.tmpl | 97 ++++++++++++++++--- > > .../initramfs-overlay-hook_0.3.bb | 2 +- > > 3 files changed, 87 insertions(+), 14 deletions(-) > > > > Thanks, already applied patches 1 and 2. 3 is still under review and will need at least some rewording. Unfortunately, we identified some issues with >> Removed "ro" option from read-only-rootfs's fstab. << in a downstream layer! - will share our observations in the other thread. Sry > > Jan > > -- > Siemens AG, Foundational Technologies > Linux Expert Center
> From: cip-dev@lists.cip-project.org <cip-dev@lists.cip-project.org> On Behalf Of Heinisch, Alexander via lists.cip-project.org > Sent: Montag, 17. Februar 2025 11:00 > To: cip-dev@lists.cip-project.org > Cc: Kiszka, Jan (FT RPD CED) <jan.kiszka@siemens.com>; Gylstorff, Quirin (FT RPD CED OES-DE) <quirin.gylstorff@siemens.com>; Heinisch, Alexander (FT RPD CED SES-AT) <alexander.heinisch@siemens.com> > Subject: [cip-dev] [isar-cip-core][PATCH 1/3] Removed "ro" option from read-only-rootfs's fstab. > > From: Alexander Heinisch <alexander.heinisch@siemens.com> > > Since we only support erofs and squashfs this option is not needed any more. Further, it causes potential overlay rootfs variants to be remounted read-only. > > Signed-off-by: Alexander Heinisch <alexander.heinisch@siemens.com> > --- > classes/read-only-rootfs.bbclass | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/classes/read-only-rootfs.bbclass b/classes/read-only-rootfs.bbclass > index 35a3ab3..83ddc33 100644 > --- a/classes/read-only-rootfs.bbclass > +++ b/classes/read-only-rootfs.bbclass > @@ -41,7 +41,7 @@ SQUASHFS_EXCLUDE_DIRS = "${RO_ROOTFS_EXCLUDE_DIRS}" > image_configure_fstab() { > sudo tee '${IMAGE_ROOTFS}/etc/fstab' << EOF # Begin /etc/fstab > -/dev/root / auto defaults,ro 0 0 > +/dev/root / auto defaults 0 0 > LABEL=var /var auto defaults 0 0 > proc /proc proc nosuid,noexec,nodev 0 0 > sysfs /sys sysfs nosuid,noexec,nodev 0 0 > -- > 2.39.5 We observed `systemd-remount-fs` failing in a downstream layer. (using a ro squashfs and apply full disk encryption on top) Although, the service is executed in isar-cip-core as well, upstream, the service can be executed successfully. In essence what happens is that, the `systemd-remount-fs` remounts various mountpoints based on their settings in `/etc/fstab` ``` root@device-mgmt:~# cat /etc/fstab # Begin /etc/fstab /dev/root / auto defaults 0 0 LABEL=var /var auto defaults 0 0 proc /proc proc nosuid,noexec,nodev 0 0 sysfs /sys sysfs nosuid,noexec,nodev 0 0 devpts /dev/pts devpts gid=5,mode=620 0 0 tmpfs /run tmpfs nodev,nosuid,size=500M,mode=755 0 0 devtmpfs /dev devtmpfs mode=0755,nosuid 0 0 # End /etc/fstab ``` Since `defaults` in fstab renders to: `rw,suid,dev,exec,auto,nouser,async` the service fails trying to remount the `squashfs` as `rw` as shown below: ``` root@device-mgmt:~# systemctl status systemd-remount-fs × systemd-remount-fs.service - Remount Root and Kernel File Systems Loaded: loaded (/lib/systemd/system/systemd-remount-fs.service; enabled-ru> Active: failed (Result: exit-code) since Fri 2025-02-21 08:55:39 UTC; 5min> Docs: man:systemd-remount-fs.service(8) https://www.freedesktop.org/wiki/Software/systemd/APIFileSystems Process: 7530 ExecStart=/lib/systemd/systemd-remount-fs (code=exited, statu> Main PID: 7530 (code=exited, status=1/FAILURE) CPU: 15ms ``` ``` root@device-mgmt:~# journalctl --no-pager -u systemd-remount-fs Feb 21 08:55:39 device-mgmt.local systemd[1]: Starting systemd-remount-fs.service - Remount Root and Kernel File Systems... Feb 21 08:55:39 device-mgmt.local systemd-remount-fs[7531]: mount: /: cannot remount /dev/root read-write, is write-protected. Feb 21 08:55:39 device-mgmt.local systemd-remount-fs[7531]: dmesg(1) may have more information after failed mount system call. Feb 21 08:55:39 device-mgmt.local systemd-remount-fs[7530]: /bin/mount for / exited with exit status 32. Feb 21 08:55:39 device-mgmt.local systemd[1]: systemd-remount-fs.service: Main process exited, code=exited, status=1/FAILURE Feb 21 08:55:39 device-mgmt.local systemd[1]: systemd-remount-fs.service: Failed with result 'exit-code'. Feb 21 08:55:39 device-mgmt.local systemd[1]: Failed to start systemd-remount-fs.service - Remount Root and Kernel File Systems. ``` The rootfs is mounted as follows: ``` root@device-mgmt:~# mount | grep " / " /dev/mapper/verityroot on / type squashfs (ro,relatime,errors=continue) ``` I understand why this happens on our system, but could not clarify why it does not happen upstream! So any idea, welcome! Nevertheless, after rethinking this patch. Maybe it was advisible to not use `default` here and either use a options like `suid,dev,exec,auto,nouser,async` or drop the line for `/dev/root` at all, keeping the mounts as they were setup in the initramfs. Any opinions, recommendations, hints, ...? Thx in advance, BR Alexander
From: Alexander Heinisch <alexander.heinisch@siemens.com> Updateable images based on isar-cip-core come with immutable rootfs out of the box. During development this oftentimes comes with the penalty of having to either rebuild images with additional files, packages aso. to accomplish development and debugging tasks or derive from the production image in a way that some fundamental features like verity, squashfs, swupdate, aso. are not part of such. Both cases are far from optimal. With this patch series we want to share our attempt to streamline development by staying as close as possible to our production image by overlaying the squashfs or erofs based ro filesystem with an overlay. To make changes redundant accross reboots, we decided to use a persistent storage option in favour of a tmpfs based approach. (Although, technically there is no limitation on doing so) Keep in mind, support for the root overlay is thought as a development feature! Thus, it is not thought for production, as it invalidates many of the properties we have from ro-filesystems. Also, keep in mind that changes on the overlay may partially hide updates or lead to "unexpected" results after applying updates. Alexander Heinisch (3): Removed "ro" option from read-only-rootfs's fstab. Fix return value handling on filesystem check Added support for rootfs-overlay. classes/read-only-rootfs.bbclass | 2 +- .../files/local-bottom.tmpl | 97 ++++++++++++++++--- .../initramfs-overlay-hook_0.3.bb | 2 +- 3 files changed, 87 insertions(+), 14 deletions(-)