From patchwork Thu Mar 7 07:46:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adithya Balakumar X-Patchwork-Id: 13585146 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3DEDFC54E55 for ; Thu, 7 Mar 2024 07:45:58 +0000 (UTC) Received: from mo-csw.securemx.jp (mo-csw.securemx.jp [210.130.202.135]) by mx.groups.io with SMTP id smtpd.web11.18217.1709797555596097576 for ; Wed, 06 Mar 2024 23:45:56 -0800 Authentication-Results: mx.groups.io; dkim=none (message not signed); spf=pass (domain: toshiba-tsip.com, ip: 210.130.202.135, mailfrom: adithya.balakumar@toshiba-tsip.com) Received: by mo-csw.securemx.jp (mx-mo-csw1801) id 4277jrnJ2670744; Thu, 7 Mar 2024 16:45:53 +0900 X-Iguazu-Qid: 2yAawbgn0wgfftke5F X-Iguazu-QSIG: v=2; s=0; t=1709797552; q=2yAawbgn0wgfftke5F; m=v1aFdulxRbpHqMWFKfRBERZwGrZOmTOaOxMh9F+ezyw= Received: from imx2-a.toshiba.co.jp (imx2-a.toshiba.co.jp [106.186.93.35]) by relay.securemx.jp (mx-mr1800) id 4277jpXR1313310 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Thu, 7 Mar 2024 16:45:52 +0900 From: Adithya Balakumar To: cip-dev@lists.cip-project.org, jan.kiszka@siemens.com Cc: shivanand.kunijadar@toshiba-tsip.com, sai.sathujoda@toshiba-tsip.com, dinesh.kumar@toshiba-tsip.com, kazuhiro3.hayashi@toshiba.co.jp Subject: [isar-cip-core][RFC v1 1/6] conf: Use bookworm backports for enabling delta handler in swupdate Date: Thu, 7 Mar 2024 13:16:07 +0530 X-TSB-HOP2: ON Message-Id: <20240307074612.1996609-2-Adithya.Balakumar@toshiba-tsip.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240307074612.1996609-1-Adithya.Balakumar@toshiba-tsip.com> References: <20240307074612.1996609-1-Adithya.Balakumar@toshiba-tsip.com> MIME-Version: 1.0 X-OriginalArrivalTime: 07 Mar 2024 07:45:48.0462 (UTC) FILETIME=[78621CE0:01DA7063] List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 07 Mar 2024 07:45:58 -0000 X-Groupsio-URL: https://lists.cip-project.org/g/cip-dev/message/15252 Delta handler requires zchunk > 1.3, but the required version is only available in bookworm backports Signed-off-by: Adithya Balakumar --- conf/distro/cip-core-bookworm.conf | 3 +++ conf/distro/debian-bookworm-backports.list | 1 + .../preferences.swupdate-bookworm-backports.conf | 12 ++++++++++++ 3 files changed, 16 insertions(+) create mode 100644 conf/distro/debian-bookworm-backports.list create mode 100644 conf/distro/preferences.swupdate-bookworm-backports.conf diff --git a/conf/distro/cip-core-bookworm.conf b/conf/distro/cip-core-bookworm.conf index 9bcf559..e0a4f7f 100644 --- a/conf/distro/cip-core-bookworm.conf +++ b/conf/distro/cip-core-bookworm.conf @@ -12,6 +12,9 @@ require conf/distro/debian-bookworm.conf require cip-core-common.inc +DISTRO_APT_SOURCES:append:swupdate = " conf/distro/debian-bookworm-backports.list" +DISTRO_APT_PREFERENCES:append:swupdate = " conf/distro/preferences.swupdate-bookworm-backports.conf" + PREFERRED_VERSION_swupdate ?= "2023.05%" PREFERRED_VERSION_linux-cip ?= "6.1.%" PREFERRED_VERSION_linux-cip-rt ?= "6.1.%" diff --git a/conf/distro/debian-bookworm-backports.list b/conf/distro/debian-bookworm-backports.list new file mode 100644 index 0000000..c4873b4 --- /dev/null +++ b/conf/distro/debian-bookworm-backports.list @@ -0,0 +1 @@ +deb http://deb.debian.org/debian bookworm-backports main contrib non-free diff --git a/conf/distro/preferences.swupdate-bookworm-backports.conf b/conf/distro/preferences.swupdate-bookworm-backports.conf new file mode 100644 index 0000000..b6efffc --- /dev/null +++ b/conf/distro/preferences.swupdate-bookworm-backports.conf @@ -0,0 +1,12 @@ +Package: libzck-dev +Pin: release n=bookworm-backports +Pin-Priority: 801 + +Package: libzck1 +Pin: release n=bookworm-backports +Pin-Priority: 801 + +Package: zchunk +Pin: release n=bookworm-backports +Pin-Priority: 801 + From patchwork Thu Mar 7 07:46:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adithya Balakumar X-Patchwork-Id: 13585148 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4E097C48BF6 for ; Thu, 7 Mar 2024 07:45:58 +0000 (UTC) Received: from mo-csw.securemx.jp (mo-csw.securemx.jp [210.130.202.132]) by mx.groups.io with SMTP id smtpd.web11.18218.1709797556184302639 for ; Wed, 06 Mar 2024 23:45:56 -0800 Authentication-Results: mx.groups.io; dkim=none (message not signed); spf=pass (domain: toshiba-tsip.com, ip: 210.130.202.132, mailfrom: adithya.balakumar@toshiba-tsip.com) Received: by mo-csw.securemx.jp (mx-mo-csw1121) id 4277jrEG829344; Thu, 7 Mar 2024 16:45:54 +0900 X-Iguazu-Qid: 2rWglQcuM2IOr0AF2w X-Iguazu-QSIG: v=2; s=0; t=1709797553; q=2rWglQcuM2IOr0AF2w; m=DLYvheW5XbgV7EgNVvXjxady6RfG2v/ZgwB/GDzvU/I= Received: from imx2-a.toshiba.co.jp (imx2-a.toshiba.co.jp [106.186.93.35]) by relay.securemx.jp (mx-mr1121) id 4277jq6l996455 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Thu, 7 Mar 2024 16:45:53 +0900 From: Adithya Balakumar To: cip-dev@lists.cip-project.org, jan.kiszka@siemens.com Cc: shivanand.kunijadar@toshiba-tsip.com, sai.sathujoda@toshiba-tsip.com, dinesh.kumar@toshiba-tsip.com, kazuhiro3.hayashi@toshiba.co.jp Subject: [isar-cip-core][RFC v1 2/6] swupdate-handler-roundrobin: Increase revision for Delta Handler support Date: Thu, 7 Mar 2024 13:16:08 +0530 X-TSB-HOP2: ON Message-Id: <20240307074612.1996609-3-Adithya.Balakumar@toshiba-tsip.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240307074612.1996609-1-Adithya.Balakumar@toshiba-tsip.com> References: <20240307074612.1996609-1-Adithya.Balakumar@toshiba-tsip.com> MIME-Version: 1.0 X-OriginalArrivalTime: 07 Mar 2024 07:45:48.0634 (UTC) FILETIME=[787C5BA0:01DA7063] List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 07 Mar 2024 07:45:58 -0000 X-Groupsio-URL: https://lists.cip-project.org/g/cip-dev/message/15254 This revision allows the use of delta handler in swupdate for zchunk based delta updates Signed-off-by: Adithya Balakumar --- .../swupdate-handler-roundrobin_0.1.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/recipes-core/swupdate-handler-roundrobin/swupdate-handler-roundrobin_0.1.bb b/recipes-core/swupdate-handler-roundrobin/swupdate-handler-roundrobin_0.1.bb index b39278e..41eb06c 100644 --- a/recipes-core/swupdate-handler-roundrobin/swupdate-handler-roundrobin_0.1.bb +++ b/recipes-core/swupdate-handler-roundrobin/swupdate-handler-roundrobin_0.1.bb @@ -13,7 +13,7 @@ inherit dpkg-raw PROVIDES = "swupdate-handlers" SRC_URI += "git://gitlab.com/cip-project/cip-sw-updates/swupdate-handler-roundrobin.git;protocol=https;destsuffix=swupdate-handler-roundrobin;name=swupdate-handler-roundrobin;nobranch=1" -SRCREV_swupdate-handler-roundrobin ?= "bb35127231ec08a67f79a7584ccfc0cada88cc4e" +SRCREV_swupdate-handler-roundrobin ?= "e6936b8c29a20cb6a6893faae57e12499ebbc342" SWUPDATE_LUASCRIPT = "swupdate-handler-roundrobin/swupdate_handlers_roundrobin.lua" From patchwork Thu Mar 7 07:46:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adithya Balakumar X-Patchwork-Id: 13585150 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56BAFC54E55 for ; Thu, 7 Mar 2024 07:46:08 +0000 (UTC) Received: from mo-csw.securemx.jp (mo-csw.securemx.jp [210.130.202.134]) by mx.groups.io with SMTP id smtpd.web10.18098.1709797555629100521 for ; Wed, 06 Mar 2024 23:45:56 -0800 Authentication-Results: mx.groups.io; dkim=none (message not signed); spf=pass (domain: toshiba-tsip.com, ip: 210.130.202.134, mailfrom: adithya.balakumar@toshiba-tsip.com) Received: by mo-csw.securemx.jp (mx-mo-csw1800) id 4277jrvh385348; Thu, 7 Mar 2024 16:45:54 +0900 X-Iguazu-Qid: 2yAaO2s1xlLVusC6mK X-Iguazu-QSIG: v=2; s=0; t=1709797553; q=2yAaO2s1xlLVusC6mK; m=Ouzuz8Ru1yAcU0nLsnEU9ntpJ6s4tXvomgOmAkVvyD4= Received: from imx2-a.toshiba.co.jp (imx2-a.toshiba.co.jp [106.186.93.35]) by relay.securemx.jp (mx-mr1802) id 4277jqQe1046023 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Thu, 7 Mar 2024 16:45:53 +0900 From: Adithya Balakumar To: cip-dev@lists.cip-project.org, jan.kiszka@siemens.com Cc: shivanand.kunijadar@toshiba-tsip.com, sai.sathujoda@toshiba-tsip.com, dinesh.kumar@toshiba-tsip.com, kazuhiro3.hayashi@toshiba.co.jp Subject: [isar-cip-core][RFC v1 3/6] swupdate: Enable Delta handler in swupdate Date: Thu, 7 Mar 2024 13:16:09 +0530 X-TSB-HOP2: ON Message-Id: <20240307074612.1996609-4-Adithya.Balakumar@toshiba-tsip.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240307074612.1996609-1-Adithya.Balakumar@toshiba-tsip.com> References: <20240307074612.1996609-1-Adithya.Balakumar@toshiba-tsip.com> MIME-Version: 1.0 X-OriginalArrivalTime: 07 Mar 2024 07:45:48.0712 (UTC) FILETIME=[78884280:01DA7063] List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 07 Mar 2024 07:46:08 -0000 X-Groupsio-URL: https://lists.cip-project.org/g/cip-dev/message/15253 Enable delta handler in swupdate for bookworm and disable cross compile for arm and arm64 on bookworm to avoid build issues Signed-off-by: Adithya Balakumar --- recipes-core/swupdate/swupdate_2023.05.bb | 3 +++ 1 file changed, 3 insertions(+) diff --git a/recipes-core/swupdate/swupdate_2023.05.bb b/recipes-core/swupdate/swupdate_2023.05.bb index d1fe1fa..7a22947 100644 --- a/recipes-core/swupdate/swupdate_2023.05.bb +++ b/recipes-core/swupdate/swupdate_2023.05.bb @@ -36,10 +36,13 @@ SRC_URI:append:bullseye = " file://0004-d-swupdate-www.install-Fix-path-for-debi DEB_BUILD_PROFILES += "pkg.swupdate.suricattalua" # add suricatta wfx DEB_BUILD_PROFILES += "pkg.swupdate.suricattawfx" +# add delta update build profile for bookworm +DEB_BUILD_PROFILES:append:bookworm = " pkg.swupdate.delta" # Disable cross for arm and arm64 on bullseye # with cross compile we have a unsat-dependency to dh-nodejs on arm/arm64 ISAR_CROSS_COMPILE:bullseye = "0" +ISAR_CROSS_COMPILE:bookworm = "0" # use backport build profile for bullseye DEB_BUILD_PROFILES:append:bullseye = " pkg.swupdate.bpo" From patchwork Thu Mar 7 07:46:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adithya Balakumar X-Patchwork-Id: 13585147 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3CFF2C54E49 for ; Thu, 7 Mar 2024 07:45:58 +0000 (UTC) Received: from mo-csw.securemx.jp (mo-csw.securemx.jp [210.130.202.132]) by mx.groups.io with SMTP id smtpd.web11.18220.1709797557494404160 for ; Wed, 06 Mar 2024 23:45:57 -0800 Authentication-Results: mx.groups.io; dkim=none (message not signed); spf=pass (domain: toshiba-tsip.com, ip: 210.130.202.132, mailfrom: adithya.balakumar@toshiba-tsip.com) Received: by mo-csw.securemx.jp (mx-mo-csw1121) id 4277jtnD829439; Thu, 7 Mar 2024 16:45:55 +0900 X-Iguazu-Qid: 2rWg9o4uiHTCjdeege X-Iguazu-QSIG: v=2; s=0; t=1709797555; q=2rWg9o4uiHTCjdeege; m=cTHPZuynB6uKoZaXElZXHPuiv56N7DqOPoSyPKOGQw4= Received: from imx2-a.toshiba.co.jp (imx2-a.toshiba.co.jp [106.186.93.35]) by relay.securemx.jp (mx-mr1121) id 4277jrIj996508 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Thu, 7 Mar 2024 16:45:55 +0900 From: Adithya Balakumar To: cip-dev@lists.cip-project.org, jan.kiszka@siemens.com Cc: shivanand.kunijadar@toshiba-tsip.com, sai.sathujoda@toshiba-tsip.com, dinesh.kumar@toshiba-tsip.com, kazuhiro3.hayashi@toshiba.co.jp Subject: [isar-cip-core][RFC v1 4/6] Add Delta update support with rdiff_image and delta handler Date: Thu, 7 Mar 2024 13:16:10 +0530 X-TSB-HOP2: ON Message-Id: <20240307074612.1996609-5-Adithya.Balakumar@toshiba-tsip.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240307074612.1996609-1-Adithya.Balakumar@toshiba-tsip.com> References: <20240307074612.1996609-1-Adithya.Balakumar@toshiba-tsip.com> MIME-Version: 1.0 X-OriginalArrivalTime: 07 Mar 2024 07:45:48.0728 (UTC) FILETIME=[788AB380:01DA7063] List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 07 Mar 2024 07:45:58 -0000 X-Groupsio-URL: https://lists.cip-project.org/g/cip-dev/message/15256 swupdate supports delta updates with rdiff_image and delta(zchunk) handler. This change adds support to use either of the handler for creating delta update artifacts. The type of handler can be set in the delta-update.yml file along with additional variables needed for the chosen handler. Signed-off-by: Adithya Balakumar --- classes/delta_update.bbclass | 90 ++++++++++++++++++++++++++++++++++++ kas/opt/delta-update.yml | 21 +++++++++ 2 files changed, 111 insertions(+) create mode 100644 classes/delta_update.bbclass create mode 100644 kas/opt/delta-update.yml diff --git a/classes/delta_update.bbclass b/classes/delta_update.bbclass new file mode 100644 index 0000000..8d47081 --- /dev/null +++ b/classes/delta_update.bbclass @@ -0,0 +1,90 @@ +# +# CIP Core, generic profile +# +# Copyright (c) Toshiba Corporation 2024 +# +# Authors: +# Adithya Balakumar +# +# SPDX-License-Identifier: MIT +# + +IMAGER_INSTALL:delta_update += "fdisk zchunk rdiff" + +FILESEXTRAPATHS:append = ":${LAYERDIR_cip-core}" +IMAGE_SRC_URI:delta_update += " ${@ "file://"+d.getVar('DELTA_RDIFF_REF_IMAGE') if d.getVar('DELTA_UPDATE_TYPE') == 'rdiff' and d.getVar('DELTA_RDIFF_REF_IMAGE') else '' }" + +do_image_delta_update[cleandirs] += "${WORKDIR}/delta_interim_artifacts" + +extract_root_fs() { + INPUT_ARTIFACT=$1 + OUTPUT_ARTIFACT=$2 + SECTOR_SIZE=512 + ROOTFS_LABEL="primary" + ROOTFS_START_SECTOR=$(${SUDO_CHROOT} /sbin/fdisk -l -o Name,Start "${INPUT_ARTIFACT}" | awk -v name="${ROOTFS_LABEL}" '$0 ~ name {print $2}' | head -n 1) + SKIP_VALUE="$(expr $(expr $ROOTFS_START_SECTOR \* $SECTOR_SIZE) / $(expr 1024 \* 1024))" + ${SUDO_CHROOT} /bin/dd if=${INPUT_ARTIFACT} bs=1M skip=$SKIP_VALUE count=1024 of=${OUTPUT_ARTIFACT} status=none +} + +do_delta_update_sw_description_template() { + # Since the sw-description file for delta update would not be very different from that of a + # normal update, copy sw-description.tmpl as sw-description-delta.tmpl and add delta update + # specific properties + SWU_DESCRIPTION_DELTA_UPDATE="sw-description-delta" + cp "${WORKDIR}/${SWU_DESCRIPTION_FILE}".tmpl "${WORKDIR}/${SWU_DESCRIPTION_DELTA_UPDATE}".tmpl + sed -i 's/\SWU_ROOTFS_PARTITION_NAME/SWU_DELTA_UPDATE_ARTIFACT/g' ${WORKDIR}/${SWU_DESCRIPTION_DELTA_UPDATE}.tmpl + sed -i '/configfilecheck =/a \ \'$'{SWU_DELTA_UPDATE_ADDITIONAL_PROPERTIES}' ${WORKDIR}/${SWU_DESCRIPTION_DELTA_UPDATE}.tmpl +} +addtask do_delta_update_sw_description_template before do_transform_template after do_generate_image_uuid + +IMAGE_CMD:delta_update() { + if [ "${DELTA_UPDATE_TYPE}" != "zchunk" ] && [ "${DELTA_UPDATE_TYPE}" != "rdiff" ]; then + bbfatal "You must set a valid DELTA_UPDATE_TYPE (rdiff/zchunk)" + fi + + DELTA_ARTIFACT_SWU="" + + if [ "${DELTA_UPDATE_TYPE}" = "zchunk" ]; then + extract_root_fs "${PP_DEPLOY}/${IMAGE_FULLNAME}.wic" "${WORKDIR}/delta_interim_artifacts/${IMAGE_FULLNAME}.raw" + + # Create .zck file + ${SUDO_CHROOT} /bin/zck \ + --output ${PP_DEPLOY}/${IMAGE_FULLNAME}.zck \ + -u --chunk-hash-type sha256 \ + ${WORKDIR}/delta_interim_artifacts/${IMAGE_FULLNAME}.raw + + # Calculate size of zck header + HSIZE="$(${SUDO_CHROOT} /bin/zck_read_header -v ${PP_DEPLOY}/${IMAGE_FULLNAME}.zck | grep "Header size" | cut -d ':' -f 2)" + + # Extract the zck header + ${SUDO_CHROOT} /bin/dd if="${PP_DEPLOY}/${IMAGE_FULLNAME}".zck of="${PP_DEPLOY}/${IMAGE_FULLNAME}".zck.header bs=1 count="$HSIZE" status=none + + DELTA_ARTIFACT_SWU=${IMAGE_FULLNAME}.zck.header + + elif [ "${DELTA_UPDATE_TYPE}" = "rdiff" ]; then + if [ -z "${DELTA_RDIFF_REF_IMAGE}" ]; then + bbfatal "You must set DELTA_RDIFF_REF_IMAGE and provide the required files as artifacts to this recipe" + fi + + # Extracting root partition from image1 + extract_root_fs "${WORKDIR}/${DELTA_RDIFF_REF_IMAGE}" "${WORKDIR}/delta_interim_artifacts/old-image-rootfs.raw" + + # Extracting root partition from image2 + extract_root_fs "${PP_DEPLOY}/${IMAGE_FULLNAME}.wic" "${WORKDIR}/delta_interim_artifacts/new-image-rootfs.raw" + + rm -f ${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.delta + # create signature file with rdiff + ${SUDO_CHROOT} /usr/bin/rdiff signature ${WORKDIR}/delta_interim_artifacts/old-image-rootfs.raw \ + ${WORKDIR}/delta_interim_artifacts/old-image-rootfs.sig + + # create delta file with the signature file + ${SUDO_CHROOT} /usr/bin/rdiff delta ${WORKDIR}/delta_interim_artifacts/old-image-rootfs.sig \ + ${WORKDIR}/delta_interim_artifacts/new-image-rootfs.raw ${PP_DEPLOY}/${IMAGE_FULLNAME}.delta + + DELTA_ARTIFACT_SWU=${IMAGE_FULLNAME}.delta + fi + # create a symbolic link as IMAGE_CMD expects a *.delta_update file in deploy image directory + ln -sf ${DELTA_ARTIFACT_SWU} ${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.delta_update +} + +addtask do_image_delta_update before do_image_swu after do_image_wic diff --git a/kas/opt/delta-update.yml b/kas/opt/delta-update.yml new file mode 100644 index 0000000..c7a3893 --- /dev/null +++ b/kas/opt/delta-update.yml @@ -0,0 +1,21 @@ +# +# CIP Core, generic profile +# +# Copyright (c) Toshiba Corporation 2024 +# +# Authors: +# Adithya Balakumar +# +# SPDX-License-Identifier: MIT +# + +header: + version: 14 + +local_conf_header: + delta-update: | + IMAGE_CLASSES:append = " delta_update" + IMAGE_FSTYPES:append = " delta_update" + DELTA_UPDATE_TYPE = "" + DELTA_ZCK_URL = "" + DELTA_RDIFF_REF_IMAGE = "" From patchwork Thu Mar 7 07:46:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adithya Balakumar X-Patchwork-Id: 13585151 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 553DAC54E49 for ; Thu, 7 Mar 2024 07:46:08 +0000 (UTC) Received: from mo-csw.securemx.jp (mo-csw.securemx.jp [210.130.202.132]) by mx.groups.io with SMTP id smtpd.web10.18099.1709797557578798239 for ; Wed, 06 Mar 2024 23:45:57 -0800 Authentication-Results: mx.groups.io; dkim=none (message not signed); spf=pass (domain: toshiba-tsip.com, ip: 210.130.202.132, mailfrom: adithya.balakumar@toshiba-tsip.com) Received: by mo-csw.securemx.jp (mx-mo-csw1121) id 4277juGx829453; Thu, 7 Mar 2024 16:45:56 +0900 X-Iguazu-Qid: 2rWg6j0N7RLe8F3pah X-Iguazu-QSIG: v=2; s=0; t=1709797555; q=2rWg6j0N7RLe8F3pah; m=PAYRmkbvZc3K6iwzgfdhlyLYm3e58hFuPN6QzykMZ4M= Received: from imx2-a.toshiba.co.jp (imx2-a.toshiba.co.jp [106.186.93.35]) by relay.securemx.jp (mx-mr1121) id 4277js5I996572 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Thu, 7 Mar 2024 16:45:55 +0900 From: Adithya Balakumar To: cip-dev@lists.cip-project.org, jan.kiszka@siemens.com Cc: shivanand.kunijadar@toshiba-tsip.com, sai.sathujoda@toshiba-tsip.com, dinesh.kumar@toshiba-tsip.com, kazuhiro3.hayashi@toshiba.co.jp Subject: [isar-cip-core][RFC v1 5/6] swupdate.bbclass: Generate swu for delta updates Date: Thu, 7 Mar 2024 13:16:11 +0530 X-TSB-HOP2: ON Message-Id: <20240307074612.1996609-6-Adithya.Balakumar@toshiba-tsip.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240307074612.1996609-1-Adithya.Balakumar@toshiba-tsip.com> References: <20240307074612.1996609-1-Adithya.Balakumar@toshiba-tsip.com> MIME-Version: 1.0 X-OriginalArrivalTime: 07 Mar 2024 07:45:48.0806 (UTC) FILETIME=[78969A60:01DA7063] List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 07 Mar 2024 07:46:08 -0000 X-Groupsio-URL: https://lists.cip-project.org/g/cip-dev/message/15257 This allows the creation of a swu file for delta update. The swu file for delta update is only created when delta-update.yml file is included at build time. Signed-off-by: Adithya Balakumar --- classes/swupdate.bbclass | 35 +++++++++++++++++++++++++++++++++-- 1 file changed, 33 insertions(+), 2 deletions(-) diff --git a/classes/swupdate.bbclass b/classes/swupdate.bbclass index 2c69892..2964024 100644 --- a/classes/swupdate.bbclass +++ b/classes/swupdate.bbclass @@ -26,10 +26,13 @@ SWU_EBG_UPDATE ?= "" SWU_EFI_BOOT_DEVICE ?= "/dev/disk/by-uuid/4321-DCBA" SWU_BOOTLOADER ??= "ebg" SWU_DESCRIPITION_FILE_BOOTLOADER ??= "${SWU_DESCRIPTION_FILE}-${SWU_BOOTLOADER}" +SWU_DESCRIPITION_FILE_DELTA_UPDATE ??= "${SWU_DESCRIPTION_FILE}-delta" +SWU_DELTA_UPDATE_ARTIFACT = "${SWU_ROOTFS_NAME}.delta_update${@get_swu_compression_type(d)}" +SWU_DELTA_UPDATE_ARTIFACT_TYPE = "delta_update${@get_swu_compression_type(d)}" SWU_IMAGE_FILE ?= "${IMAGE_FULLNAME}" SWU_DESCRIPTION_FILE ?= "sw-description" -SWU_ADDITIONAL_FILES ?= "linux.efi ${SWU_ROOTFS_PARTITION_NAME}" +SWU_ADDITIONAL_FILES ?= "linux.efi ${SWU_ROOTFS_PARTITION_NAME} ${SWU_DELTA_UPDATE_ARTIFACT}" SWU_SIGNED ??= "" SWU_SIGNATURE_EXT ?= "sig" SWU_SIGNATURE_TYPE ?= "cms" @@ -37,6 +40,7 @@ SWU_SIGNATURE_TYPE ?= "cms" SWU_BUILDCHROOT_IMAGE_FILE ?= "${@os.path.basename(d.getVar('SWU_IMAGE_FILE'))}" IMAGE_TYPEDEP:swu = "${SWU_ROOTFS_TYPE}${@get_swu_compression_type(d)}" +IMAGE_TYPEDEP:swu += "${@ '${SWU_DELTA_UPDATE_ARTIFACT_TYPE}' if d.getVar('DELTA_UPDATE_TYPE') else ''}" IMAGER_BUILD_DEPS:swu += "${@'swupdate-certificates-key' if bb.utils.to_boolean(d.getVar('SWU_SIGNED')) else ''}" IMAGER_INSTALL:swu += "cpio ${@'openssl swupdate-certificates-key' if bb.utils.to_boolean(d.getVar('SWU_SIGNED')) else ''}" IMAGE_INSTALL += "${@'swupdate-certificates' if bb.utils.to_boolean(d.getVar('SWU_SIGNED')) else ''}" @@ -46,6 +50,7 @@ IMAGE_SRC_URI:swu = "file://${SWU_DESCRIPTION_FILE}.tmpl" IMAGE_SRC_URI:swu += "file://${SWU_DESCRIPITION_FILE_BOOTLOADER}.tmpl" IMAGE_TEMPLATE_FILES:swu = "${SWU_DESCRIPTION_FILE}.tmpl" IMAGE_TEMPLATE_FILES:swu += "${SWU_DESCRIPITION_FILE_BOOTLOADER}.tmpl" +IMAGE_TEMPLATE_FILES:delta_update += "${SWU_DESCRIPITION_FILE_DELTA_UPDATE}.tmpl" IMAGE_TEMPLATE_VARS:swu = " \ SWU_ROOTFS_PARTITION_NAME \ TARGET_IMAGE_UUID \ @@ -58,6 +63,8 @@ IMAGE_TEMPLATE_VARS:swu = " \ SWU_FILE_NODES \ SWU_BOOTLOADER_FILE_NODE \ SWU_SCRIPTS_NODE \ + SWU_DELTA_UPDATE_ARTIFACT \ + SWU_DELTA_UPDATE_ADDITIONAL_PROPERTIES \ " # TARGET_IMAGE_UUID needs to be generated before completing the template @@ -136,6 +143,22 @@ python add_scripts(){ d.appendVar('SWU_SCRIPTS_NODE', swu_scripts_node) } +SWU_EXTEND_SW_DESCRIPTION += "add_swu_delta_update_properties" +python add_swu_delta_update_properties() { + delta_type = d.getVar('DELTA_UPDATE_TYPE') + swu_delta_update_properties = "" + if delta_type == "rdiff": + swu_delta_update_properties = 'chainhandler = "rdiff_image";' + elif delta_type == "zchunk": + zck_url = d.getVar('DELTA_ZCK_URL') + swu_delta_update_properties = f""" + chainhandler = "delta"; + url = "{zck_url}"; + zckloglevel = "error"; + """ + d.setVar('SWU_DELTA_UPDATE_ADDITIONAL_PROPERTIES', swu_delta_update_properties) +} + # convert between swupdate compressor name and imagetype extension def get_swu_compression_type(d): swu_ct = d.getVar('SWU_COMPRESSION_TYPE') @@ -154,13 +177,21 @@ FILESEXTRAPATHS:append = ":${LAYERDIR_cip-core}/recipes-core/images/swu" do_image_swu[depends] += "${PN}:do_transform_template" do_image_swu[stamp-extra-info] = "${DISTRO}-${MACHINE}" -do_image_swu[cleandirs] += "${WORKDIR}/swu ${WORKDIR}/swu-${SWU_BOOTLOADER}" +do_image_swu[cleandirs] += "${WORKDIR}/swu ${WORKDIR}/swu-${SWU_BOOTLOADER} ${WORKDIR}/swu-delta" IMAGE_CMD:swu() { rm -f '${DEPLOY_DIR_IMAGE}/${SWU_IMAGE_FILE}'*.swu + if [ -z "${DELTA_UPDATE_TYPE}" ]; then + rm -rf ${WORKDIR}/swu-delta + fi + cp '${WORKDIR}/${SWU_DESCRIPTION_FILE}' '${WORKDIR}/swu/${SWU_DESCRIPTION_FILE}' if [ -f '${WORKDIR}/${SWU_DESCRIPITION_FILE_BOOTLOADER}' ]; then cp '${WORKDIR}/${SWU_DESCRIPITION_FILE_BOOTLOADER}' '${WORKDIR}/swu-${SWU_BOOTLOADER}/${SWU_DESCRIPTION_FILE}' fi + if [ -f '${WORKDIR}/${SWU_DESCRIPITION_FILE_DELTA_UPDATE}' ]; then + cp '${WORKDIR}/${SWU_DESCRIPITION_FILE_DELTA_UPDATE}' '${WORKDIR}/swu-delta/${SWU_DESCRIPTION_FILE}' + fi + for swu_file in "${WORKDIR}"/swu*; do swu_file_base=$(basename $swu_file) From patchwork Thu Mar 7 07:46:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adithya Balakumar X-Patchwork-Id: 13585152 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 61647C48BF6 for ; Thu, 7 Mar 2024 07:46:08 +0000 (UTC) Received: from mo-csw.securemx.jp (mo-csw.securemx.jp [210.130.202.132]) by mx.groups.io with SMTP id smtpd.web10.18100.1709797557826109324 for ; Wed, 06 Mar 2024 23:45:58 -0800 Authentication-Results: mx.groups.io; dkim=none (message not signed); spf=pass (domain: toshiba-tsip.com, ip: 210.130.202.132, mailfrom: adithya.balakumar@toshiba-tsip.com) Received: by mo-csw.securemx.jp (mx-mo-csw1121) id 4277ju2a829478; Thu, 7 Mar 2024 16:45:56 +0900 X-Iguazu-Qid: 2rWgQ8FBVOzxzEuOOG X-Iguazu-QSIG: v=2; s=0; t=1709797556; q=2rWgQ8FBVOzxzEuOOG; m=YjPJFeuQBsRfmA/oU26LR3ZMAawmO0Uqnvw+1nhDWHk= Received: from imx12-a.toshiba.co.jp ([38.106.60.135]) by relay.securemx.jp (mx-mr1123) id 4277jsYU1092844 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Thu, 7 Mar 2024 16:45:55 +0900 From: Adithya Balakumar To: cip-dev@lists.cip-project.org, jan.kiszka@siemens.com Cc: shivanand.kunijadar@toshiba-tsip.com, sai.sathujoda@toshiba-tsip.com, dinesh.kumar@toshiba-tsip.com, kazuhiro3.hayashi@toshiba.co.jp Subject: [isar-cip-core][RFC v1 6/6] doc/README.swupdate.md: Update steps to test Delta software Update Date: Thu, 7 Mar 2024 13:16:12 +0530 X-TSB-HOP2: ON Message-Id: <20240307074612.1996609-7-Adithya.Balakumar@toshiba-tsip.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240307074612.1996609-1-Adithya.Balakumar@toshiba-tsip.com> References: <20240307074612.1996609-1-Adithya.Balakumar@toshiba-tsip.com> MIME-Version: 1.0 X-OriginalArrivalTime: 07 Mar 2024 07:45:48.0884 (UTC) FILETIME=[78A28140:01DA7063] List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 07 Mar 2024 07:46:08 -0000 X-Groupsio-URL: https://lists.cip-project.org/g/cip-dev/message/15258 This change includes steps to verify Delta Software Update with rdiff_image and delta handler Signed-off-by: Adithya Balakumar --- doc/README.swupdate.md | 95 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 95 insertions(+) diff --git a/doc/README.swupdate.md b/doc/README.swupdate.md index cf1bcfb..4003243 100644 --- a/doc/README.swupdate.md +++ b/doc/README.swupdate.md @@ -324,6 +324,101 @@ user variables: ``` +# Building and testing the CIP Core image for Delta Software Update + +Set up `kas-container` as described in the [top-level README](../README.md), and then proceed with the following steps. + +First build an image using the following command: +``` +host$ ./kas-container build kas-cip.yml:kas/board/qemu-amd64.yml:kas/opt/ebg-swu.yml:kas/opt/rt.yml +``` +Currently, Delta Software Update is supported only for root file system. +Now, to verify a delta update, build an image with some modifications to the root file system (ex: Add additional packages). +For this example, add the packages vim and nano (since the packages are not already included in the base image) to the image. This can be done by modifying the `recipes-core/images/cip-core-image.bb` file as shown below. +``` +diff --git a/recipes-core/images/cip-core-image.bb b/recipes-core/images/cip-core-image.bb +index 0ec7220..f61ce23 100644 +--- a/recipes-core/images/cip-core-image.bb ++++ b/recipes-core/images/cip-core-image.bb +@@ -14,6 +14,7 @@ inherit image + ISAR_RELEASE_CMD = "git -C ${LAYERDIR_cip-core} describe --tags --dirty --always --match 'v[0-9].[0-9]*'" + DESCRIPTION = "CIP Core image" + ++IMAGE_PREINSTALL += "vim nano" + IMAGE_INSTALL += "customizations" + + CIP_IMAGE_OPTIONS ?= "" +``` +## Delta Software Update using rdiff_image handler + +Creating a delta update file for rdiff_image handler requires a reference image (against which the delta is computed). In this case, the image built in the previous section can be used as the reference image. +Copy the reference image to the project directory. And set the variable `DELTA_RDIFF_REF_IMAGE` with the path to the reference image and set the variable `DELTA_UPDATE_TYPE` to `rdiff`. + +``` +header: + version: 14 + +local_conf_header: + delta-update: | + IMAGE_CLASSES:append = " delta_update" + DELTA_UPDATE_TYPE = "rdiff" + DELTA_ZCK_URL = "" + DELTA_RDIFF_REF_IMAGE = "path/to/image/from/project/root/directory.wic" + +``` +Run the below command to build the image with the modification as shown above and with the root file system modification as described in previous section. +``` +KAS_BUILD_DIR=image2 ./kas-container build kas-cip.yml:kas/board/qemu-amd64.yml:kas/opt/ebg-swu.yml:kas/opt/delta-update.yml +``` +Run the below command to start first image +``` +host$ DISTRO_RELEASE=bookworm SWUPDATE_BOOT=y ./start-qemu.sh amd64 +``` +Copy `cip-core-image-cip-core-bookworm-qemu-amd64-delta.swu` file from `image2/tmp/deploy/images/qemu-amd64/` folder into the running system: +``` +host$ cd image2/tmp/deploy/images/qemu-amd64/ +host$ scp -P 22222 ./cip-core-image-cip-core-bookworm-qemu-amd64-delta.swu root@localhost: +``` + +### Delta Software Update Verification + +Follow the steps mentioned in the section [SWUpdate verification](#swupdate-verification) for verification. + +Since our target was to update the root file system by adding a few packages (vim and nano in this example), Check if the packages are available after the update by running either the `vim` command or `nano` command. + +## Delta Software Update using delta handler + +For Delta update with zchunk, set the variable `DELTA_ZCK_URL` with the URL of the zck file that is hosted in a http server and set the variable `DELTA_UPDATE_TYPE` to `zchunk`. +``` +header: + version: 14 + +local_conf_header: + delta-update: | + IMAGE_CLASSES:append = " delta_update" + DELTA_UPDATE_TYPE = "zchunk" + DELTA_ZCK_URL = "" + DELTA_RDIFF_REF_IMAGE = "" + +``` +Run the below command to build the image with the modification as shown above and with the root file system modification as described in previous section. +``` +KAS_BUILD_DIR=image2 ./kas-container build kas-cip.yml:kas/board/qemu-amd64.yml:kas/opt/ebg-swu.yml:kas/opt/delta-update.yml +``` +Run the below command to start first image +``` +host$ DISTRO_RELEASE=bookworm SWUPDATE_BOOT=y ./start-qemu.sh amd64 +``` +Copy `cip-core-image-cip-core-bookworm-qemu-amd64-delta.swu` file from `image2/tmp/deploy/images/qemu-amd64/` folder into the running system: +``` +host$ cd image2/tmp/deploy/images/qemu-amd64/ +host$ scp -P 22222 ./cip-core-image-cip-core-bookworm-qemu-amd64-delta.swu root@localhost: +``` +The `cip-core-image-cip-core-bookworm-qemu-amd64.zck` file must be hosted in a server. + +### Delta Software Update Verification + +Follow the steps mentioned in the section [Delta Software Update Verification](#delta-software-update-verification) for verification. # Building and testing the CIP Core image for BBB