From patchwork Tue Jul 13 18:05:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12374685 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EDFECC07E96 for ; Tue, 13 Jul 2021 18:06:30 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A221061374 for ; Tue, 13 Jul 2021 18:06:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A221061374 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.155537.287112 (Exim 4.92) (envelope-from ) id 1m3MnT-0002lQ-Ac; Tue, 13 Jul 2021 18:06:19 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 155537.287112; Tue, 13 Jul 2021 18:06:19 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3MnT-0002lJ-7b; Tue, 13 Jul 2021 18:06:19 +0000 Received: by outflank-mailman (input) for mailman id 155537; Tue, 13 Jul 2021 18:06:18 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3MnR-0002kr-UR for xen-devel@lists.xenproject.org; Tue, 13 Jul 2021 18:06:17 +0000 Received: from mo4-p01-ob.smtp.rzone.de (unknown [81.169.146.167]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 04584618-e405-11eb-8793-12813bfff9fa; Tue, 13 Jul 2021 18:06:17 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.28.1 AUTH) with ESMTPSA id 30791cx6DI6BtAi (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Tue, 13 Jul 2021 20:06:11 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 04584618-e405-11eb-8793-12813bfff9fa DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1626199571; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=oJEcT6v2n/O5Zj5MH8UEB3htWfj1QTk7O+DK+AM3TXA=; b=PciP9VORIaGFuJ6qCss2A4mnbbOe3K7r8nCJEEeY0df5QeLCvNsAWdxdTi7yOZgbtZ vYBPx1JZAGcofS+BtlzPu5X2ur1W1HxgQZ0cFo3a+AuADZ1TQJigZT8yQhXMSRsiCmr6 71Qo0bws7mPRd0nXPxmuYE4I+V6+GZeV0X1E/9t9AxVqxLZUOI8r0JlNMjJJkZ3lTahF 2Fd0FU4e5Aq8xoyfH81c+YW10a4Jfw+bPYE8JFpKCqvlkc1CRGSriq1eqECbv6gcn5cF qhfiNK08nJuri8KlYheJWB9LZl66rKRxhnfRPKCFMoJUnYXp7Xk34LjWUvpClDR2ZgsO Nepg== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAgthqlagRsstpH9XYEN5v9xf7dypFU9ge5p0Oi0V" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu Subject: [PATCH v20210713 01/31] tools: fix make rpmball Date: Tue, 13 Jul 2021 20:05:35 +0200 Message-Id: <20210713180605.12096-2-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210713180605.12096-1-olaf@aepfle.de> References: <20210713180605.12096-1-olaf@aepfle.de> MIME-Version: 1.0 Commit 438c5ffa44e99cceb574c0f9946aacacdedd2952 ("rpmball: Adjust to new rpm, do not require --force") attempted to handle stricter directory permissions in newer distributions. This introduced a few issues: - /boot used to be a constant prior commit 6475d700055fa952f7671cee982a23de2f5e4a7c ("use BOOT_DIR as xen.gz install location"), since this commit the location has to be referenced via ${BOOT_DIR} - it assumed the prefix and the various configurable paths match the glob pattern /*/*/* Adjust the code to build a filelist on demand and filter directories from an installed filesystem.rpm. This works on a SUSE system, and will likely work on a RedHat based system as well. Take the opportunity to replace the usage of $RPM_BUILD_ROOT with %buildroot, and use pushd/popd pairs. Signed-off-by: Olaf Hering --- tools/misc/mkrpm | 33 ++++++++++++++++++++++++--------- 1 file changed, 24 insertions(+), 9 deletions(-) diff --git a/tools/misc/mkrpm b/tools/misc/mkrpm index 68819b2d73..b31054444d 100644 --- a/tools/misc/mkrpm +++ b/tools/misc/mkrpm @@ -50,20 +50,35 @@ uninstall. %build %install -rm -rf \$RPM_BUILD_ROOT -mkdir -p \$RPM_BUILD_ROOT -cd %{_xenroot} -dist/install.sh \$RPM_BUILD_ROOT/ +rm -rf %{buildroot} +mkdir -p %{buildroot} +pushd %{_xenroot} +dist/install.sh %{buildroot} + +pushd %{buildroot} +popd +rm -f dist/filesystem.txt +rm -f dist/directories.txt +rm -f dist/files.txt +find %{buildroot} -type d | sed 's|^%{buildroot}||' | sort > dist/directories.txt +find %{buildroot} -type f | sed 's|^%{buildroot}||' | sort > dist/files.txt +find %{buildroot} -type l | sed 's|^%{buildroot}||' | sort >> dist/files.txt +if rpm -ql filesystem > dist/filesystem.txt +then + while read + do + sed -i "s|^\${REPLY}$||" dist/directories.txt + done < dist/filesystem.txt +fi +sed 's@^@%%dir @' dist/directories.txt >> dist/files.txt -cd \$RPM_BUILD_ROOT +popd %clean -rm -rf \$RPM_BUILD_ROOT +rm -rf %{buildroot} -%files +%files -f %{_xenroot}/dist/files.txt %defattr(-,root,root,-) -/*/*/* -/boot/* %post EOF From patchwork Tue Jul 13 18:05:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12374683 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 49896C11F68 for ; Tue, 13 Jul 2021 18:06:31 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1056561369 for ; Tue, 13 Jul 2021 18:06:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1056561369 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.155539.287135 (Exim 4.92) (envelope-from ) id 1m3MnX-0003ML-SU; Tue, 13 Jul 2021 18:06:23 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 155539.287135; Tue, 13 Jul 2021 18:06:23 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3MnX-0003MB-OV; Tue, 13 Jul 2021 18:06:23 +0000 Received: by outflank-mailman (input) for mailman id 155539; Tue, 13 Jul 2021 18:06:22 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3MnW-0002kr-T2 for xen-devel@lists.xenproject.org; Tue, 13 Jul 2021 18:06:22 +0000 Received: from mo4-p01-ob.smtp.rzone.de (unknown [81.169.146.167]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 04ccd15e-e405-11eb-8793-12813bfff9fa; Tue, 13 Jul 2021 18:06:17 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.28.1 AUTH) with ESMTPSA id 30791cx6DI6BtAj (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Tue, 13 Jul 2021 20:06:11 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 04ccd15e-e405-11eb-8793-12813bfff9fa DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1626199572; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=G9MXmcB02gC5L6Rf3KqCQCmQRU/JibmaTKTy7tmcVlI=; b=pQ5/nQNGQg0JTqc78E/qLUshNNM7s61SiAI126cpuRIVtR+uUzFWR5MvQNfV2RaYZF dwFUiVK3cSBvF8Gme1wOEyS3sfwPkY+13sZ6QTreEt8L4IrPnjkdBa5f/Fw9P3rMhgwX kHV3mZp2Yc5uMON8VXYpFG48xKUbjtY4slTIDA+NYS1fFTyOTbztevO7MDjBvbkB1n+A O+gfYWLuHrG01HwpaneYYrwgXbvok+PQ92UZbWdThNu5zM5hdesO6bP4B6bQ/0PxCZRA Q/5Q9TuE3TCTxFshFxXuPNjpeKLrD3ZUJDJ//M7TcFop+79jZqpq7qBqjho2v7qOLvXE rn/g== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAgthqlagRsstpH9XYEN5v9xf7dypFU9ge5p0Oi0V" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu Subject: [PATCH v20210713 02/31] hotplug/Linux: fix starting of xenstored with restarting systemd Date: Tue, 13 Jul 2021 20:05:36 +0200 Message-Id: <20210713180605.12096-3-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210713180605.12096-1-olaf@aepfle.de> References: <20210713180605.12096-1-olaf@aepfle.de> MIME-Version: 1.0 A hard to trigger race with another unrelated systemd service and xenstored.service unveiled a bug in the way how xenstored is launched with systemd. launch-xenstore may start either a daemon or a domain. In case a domain is used, systemd-notify was called. If another service triggered a restart of systemd while xenstored.service was executed, systemd may temporary lose track of services with Type=notify. As a result, xenstored.service would be marked as failed and units that depend on it will not be started. This breaks the enire Xen toolstack. The chain of events is basically: xenstored.service sends the notification to systemd, this is a one-way event. Then systemd may be restarted by the other unit. During this time, xenstored.service is done and exits. Once systemd is done with its restart, it collects the pending notifications and childs. If it does not find the unit which sent the notification it will declare it as failed. A workaround for this scenario is to leave the child processes running for a short time after sending the "READY=1" notification. If systemd happens to restart it will still find the unit it launched. Adjust the callers of launch-xenstore to specifiy the init system: Do not fork xenstored with systemd, preserve pid. This wil also avoid the need for a sleep because the process which sent the "READY=1" (the previously forked child) is still alive. Remove the --pid-file in the systemd case because the pid of the child is known, and the file had probably little effect anyway due to lack of PidFile= and Type=forking in the unit file. Be verbose about xenstored startup only with sysv to avoid interleaved output in systemd journal. Do the same also for domain case, even if is not strictly needed because init-xenstore-domain has no output. The fix for upstream systemd which is supposed to fix it: 575b300b795b6 ("pid1: rework how we dispatch SIGCHLD and other signals") Signed-off-by: Olaf Hering --- v04: - do mkdir unconditionally because init-xenstore-domain writes the domid to xenstored.pid v03: - remove run_xenstored function, follow style of shell built-in test function v02: - preserve Type=notify --- tools/hotplug/Linux/init.d/xencommons.in | 2 +- tools/hotplug/Linux/launch-xenstore.in | 40 ++++++++++++++----- .../Linux/systemd/xenstored.service.in | 2 +- 3 files changed, 31 insertions(+), 13 deletions(-) diff --git a/tools/hotplug/Linux/init.d/xencommons.in b/tools/hotplug/Linux/init.d/xencommons.in index 7fd6903b98..dcb0ce4b73 100644 --- a/tools/hotplug/Linux/init.d/xencommons.in +++ b/tools/hotplug/Linux/init.d/xencommons.in @@ -60,7 +60,7 @@ do_start () { mkdir -m700 -p ${XEN_LOCK_DIR} mkdir -p ${XEN_LOG_DIR} - @XEN_SCRIPT_DIR@/launch-xenstore || exit 1 + @XEN_SCRIPT_DIR@/launch-xenstore 'sysv' || exit 1 echo Setting domain 0 name, domid and JSON config... ${LIBEXEC_BIN}/xen-init-dom0 ${XEN_DOM0_UUID} diff --git a/tools/hotplug/Linux/launch-xenstore.in b/tools/hotplug/Linux/launch-xenstore.in index 019f9d6f4d..d40c66482a 100644 --- a/tools/hotplug/Linux/launch-xenstore.in +++ b/tools/hotplug/Linux/launch-xenstore.in @@ -15,6 +15,17 @@ # License along with this library; If not, see . # +initd=$1 + +case "$initd" in + sysv) nonl='-n' ;; + systemd) nonl= ;; + *) + echo "first argument must be 'sysv' or 'systemd'" + exit 1 + ;; +esac + XENSTORED=@XENSTORED@ . @XEN_SCRIPT_DIR@/hotplugpath.sh @@ -44,14 +55,16 @@ timeout_xenstore () { return 0 } -test_xenstore && exit 0 +mkdir -p @XEN_RUN_DIR@ + +if test "$initd" = 'sysv' ; then + test_xenstore && exit 0 +fi test -f @CONFIG_DIR@/@CONFIG_LEAF_DIR@/xencommons && . @CONFIG_DIR@/@CONFIG_LEAF_DIR@/xencommons [ "$XENSTORETYPE" = "" ] && XENSTORETYPE=daemon -/bin/mkdir -p @XEN_RUN_DIR@ - [ "$XENSTORETYPE" = "daemon" ] && { [ -z "$XENSTORED_TRACE" ] || XENSTORED_ARGS="$XENSTORED_ARGS -T @XEN_LOG_DIR@/xenstored-trace.log" [ -z "$XENSTORED" ] && XENSTORED=@XENSTORED@ @@ -59,13 +72,15 @@ test -f @CONFIG_DIR@/@CONFIG_LEAF_DIR@/xencommons && . @CONFIG_DIR@/@CONFIG_LEAF echo "No xenstored found" exit 1 } + [ "$initd" = 'sysv' ] && { + echo $nonl Starting $XENSTORED... + $XENSTORED --pid-file @XEN_RUN_DIR@/xenstored.pid $XENSTORED_ARGS + timeout_xenstore $XENSTORED || exit 1 + exit 0 + } - echo -n Starting $XENSTORED... - $XENSTORED --pid-file @XEN_RUN_DIR@/xenstored.pid $XENSTORED_ARGS - - systemd-notify --booted 2>/dev/null || timeout_xenstore $XENSTORED || exit 1 - - exit 0 + exec $XENSTORED -N $XENSTORED_ARGS + exit 1 } [ "$XENSTORETYPE" = "domain" ] && { @@ -75,9 +90,12 @@ test -f @CONFIG_DIR@/@CONFIG_LEAF_DIR@/xencommons && . @CONFIG_DIR@/@CONFIG_LEAF XENSTORE_DOMAIN_ARGS="$XENSTORE_DOMAIN_ARGS --memory $XENSTORE_DOMAIN_SIZE" [ -z "$XENSTORE_MAX_DOMAIN_SIZE" ] || XENSTORE_DOMAIN_ARGS="$XENSTORE_DOMAIN_ARGS --maxmem $XENSTORE_MAX_DOMAIN_SIZE" - echo -n Starting $XENSTORE_DOMAIN_KERNEL... + echo $nonl Starting $XENSTORE_DOMAIN_KERNEL... ${LIBEXEC_BIN}/init-xenstore-domain $XENSTORE_DOMAIN_ARGS || exit 1 - systemd-notify --ready 2>/dev/null + [ "$initd" = 'systemd' ] && { + systemd-notify --ready + sleep 9 + } exit 0 } diff --git a/tools/hotplug/Linux/systemd/xenstored.service.in b/tools/hotplug/Linux/systemd/xenstored.service.in index 80c1d408a5..c226eb3635 100644 --- a/tools/hotplug/Linux/systemd/xenstored.service.in +++ b/tools/hotplug/Linux/systemd/xenstored.service.in @@ -11,7 +11,7 @@ Type=notify NotifyAccess=all RemainAfterExit=true ExecStartPre=/bin/grep -q control_d /proc/xen/capabilities -ExecStart=@XEN_SCRIPT_DIR@/launch-xenstore +ExecStart=@XEN_SCRIPT_DIR@/launch-xenstore 'systemd' [Install] WantedBy=multi-user.target From patchwork Tue Jul 13 18:05:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12374679 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 05E56C11F66 for ; Tue, 13 Jul 2021 18:06:30 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 808B661374 for ; Tue, 13 Jul 2021 18:06:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 808B661374 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.155538.287124 (Exim 4.92) (envelope-from ) id 1m3MnV-000332-Hz; Tue, 13 Jul 2021 18:06:21 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 155538.287124; Tue, 13 Jul 2021 18:06:21 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3MnV-00032r-Et; Tue, 13 Jul 2021 18:06:21 +0000 Received: by outflank-mailman (input) for mailman id 155538; Tue, 13 Jul 2021 18:06:19 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3MnT-0002UL-Fo for xen-devel@lists.xenproject.org; Tue, 13 Jul 2021 18:06:19 +0000 Received: from mo4-p01-ob.smtp.rzone.de (unknown [85.215.255.50]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id b883459d-473c-4d3c-a22e-a01fa3628ede; Tue, 13 Jul 2021 18:06:17 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.28.1 AUTH) with ESMTPSA id 30791cx6DI6CtAk (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Tue, 13 Jul 2021 20:06:12 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: b883459d-473c-4d3c-a22e-a01fa3628ede DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1626199572; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=4WYZHRcGpgtt0AApxzyOy4HkByS6ppN/3uviqFvGNT0=; b=gtSK6ABErb1YI8s+5/58wut9xJISpnDjs25U6sS+QGQDeyPNXkZ2MM7VY3ekmJt/uC 22VYTQ7goGuzoXZTubM2lITtegcQrARH5NGJjB4y2OZRCzsvWozHG1MrMkNzEvkhaYZG nAhHQb6CquW7HEh+HiwJ+9u0UaT9AF7pOuOJ34r94KSsnCq8YK2mN7zXNrh8abNFPP/l j1iLhqTykQ0MSmpJRpVgFUzDOC5kgKb2EkgXmb3E9bqg2mQ0LOU0/XygzaaTQj/hnmDz gtG/FG+CGug5iv5YyOxWtbDwXjTFlgsmULXQDowbOx9qUkpuFfLABMmSiqlAguK0hHFM lGdg== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAgthqlagRsstpH9XYEN5v9xf7dypFU9ge5p0Oi0V" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu , Juergen Gross Subject: [PATCH v20210713 03/31] tools: add API to work with sevaral bits at once Date: Tue, 13 Jul 2021 20:05:37 +0200 Message-Id: <20210713180605.12096-4-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210713180605.12096-1-olaf@aepfle.de> References: <20210713180605.12096-1-olaf@aepfle.de> MIME-Version: 1.0 Introduce new API to test if a fixed number of bits is clear or set, and clear or set them all at once. The caller has to make sure the input bitnumber is a multiple of BITS_PER_LONG. This API avoids the loop over each bit in a known range just to see if all of them are either clear or set. Signed-off-by: Olaf Hering --- v02: - change return type from int to bool (jgross) --- tools/libs/ctrl/xc_bitops.h | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/tools/libs/ctrl/xc_bitops.h b/tools/libs/ctrl/xc_bitops.h index f0bac4a071..8e8c6efb45 100644 --- a/tools/libs/ctrl/xc_bitops.h +++ b/tools/libs/ctrl/xc_bitops.h @@ -3,6 +3,7 @@ /* bitmap operations for single threaded access */ +#include #include #include @@ -77,4 +78,31 @@ static inline void bitmap_or(void *_dst, const void *_other, dst[i] |= other[i]; } +static inline bool test_bit_long_set(unsigned long nr_base, const void *_addr) +{ + const unsigned long *addr = _addr; + unsigned long val = addr[nr_base / BITS_PER_LONG]; + + return val == ~0; +} + +static inline bool test_bit_long_clear(unsigned long nr_base, const void *_addr) +{ + const unsigned long *addr = _addr; + unsigned long val = addr[nr_base / BITS_PER_LONG]; + + return val == 0; +} + +static inline void clear_bit_long(unsigned long nr_base, void *_addr) +{ + unsigned long *addr = _addr; + addr[nr_base / BITS_PER_LONG] = 0; +} + +static inline void set_bit_long(unsigned long nr_base, void *_addr) +{ + unsigned long *addr = _addr; + addr[nr_base / BITS_PER_LONG] = ~0; +} #endif /* XC_BITOPS_H */ From patchwork Tue Jul 13 18:05:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12374693 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 044B7C07E95 for ; Tue, 13 Jul 2021 18:06:41 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C6D0E61369 for ; Tue, 13 Jul 2021 18:06:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C6D0E61369 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.155543.287179 (Exim 4.92) (envelope-from ) id 1m3Mni-0004v0-DV; Tue, 13 Jul 2021 18:06:34 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 155543.287179; Tue, 13 Jul 2021 18:06:34 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3Mni-0004up-9A; Tue, 13 Jul 2021 18:06:34 +0000 Received: by outflank-mailman (input) for mailman id 155543; Tue, 13 Jul 2021 18:06:33 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3Mng-0002kr-Tc for xen-devel@lists.xenproject.org; Tue, 13 Jul 2021 18:06:32 +0000 Received: from mo4-p01-ob.smtp.rzone.de (unknown [85.215.255.50]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 05524e76-e405-11eb-8793-12813bfff9fa; Tue, 13 Jul 2021 18:06:20 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.28.1 AUTH) with ESMTPSA id 30791cx6DI6CtAl (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Tue, 13 Jul 2021 20:06:12 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 05524e76-e405-11eb-8793-12813bfff9fa DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1626199573; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=sGK5MVxjAVkaDlLv2beeaP+qQqyTAMbH073gSSJ9pMg=; b=j9zo2skvObV82f3sAdqJlzNQ2egJfGE23XTdygGlcV6tTMcE1vM/Xa4T1jpmnnbF7U v32DmdcLl7kopNAWu03Z/Q3/vV71CdVanXp+hgD0rFOdpebmPKWvPdh5FZWq+mkoVuzD fftBP6GZQLYXwVy3TEa7IwgO7dMTztdTAUb2jUfh0inFhLG1i3IyksokZ+wUQVdTDpEs wOTVucqVf2NxEX07JQoTW9MhLNkz3qz3B1TJr2RNWfjEgtcUxSc4deKX9H1B0kROP2E+ 8j7bj3PvTQfBaLvbVIUtQv+OcRLPKTmNDL6EZX6nfZ/lsQtzTQIuKDDAyfJWm51K31Yo l4yg== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAgthqlagRsstpH9XYEN5v9xf7dypFU9ge5p0Oi0V" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu , Anthony PERARD Subject: [PATCH v20210713 04/31] xl: fix description of migrate --debug Date: Tue, 13 Jul 2021 20:05:38 +0200 Message-Id: <20210713180605.12096-5-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210713180605.12096-1-olaf@aepfle.de> References: <20210713180605.12096-1-olaf@aepfle.de> MIME-Version: 1.0 xl migrate --debug used to track every pfn in every batch of pages. But these times are gone. The code in xc_domain_save is the consumer of this knob, now may enable verification mode. Signed-off-by: Olaf Hering --- v03: - adjust to describe what --debug would do when the code which consumes this knob is fixed. v02: - the option has no effect anymore --- docs/man/xl.1.pod.in | 4 +++- tools/xl/xl_cmdtable.c | 2 +- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/man/xl.1.pod.in b/docs/man/xl.1.pod.in index e2176bd696..bae557b1de 100644 --- a/docs/man/xl.1.pod.in +++ b/docs/man/xl.1.pod.in @@ -481,7 +481,9 @@ domain. =item B<--debug> -Display huge (!) amount of debug information during the migration process. +This enables verification mode, which will transfer the entire domU memory +once more to the receiving host to make sure the content is identical on +both sides. =item B<-p> diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c index 661323d488..e4f27a3248 100644 --- a/tools/xl/xl_cmdtable.c +++ b/tools/xl/xl_cmdtable.c @@ -172,7 +172,7 @@ const struct cmd_spec cmd_table[] = { " migrate-receive [-d -e]\n" "-e Do not wait in the background (on ) for the death\n" " of the domain.\n" - "--debug Print huge (!) amount of debug during the migration process.\n" + "--debug Enable verification mode.\n" "-p Do not unpause domain after migrating it.\n" "-D Preserve the domain id" }, From patchwork Tue Jul 13 18:05:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12374689 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2A063C07E96 for ; Tue, 13 Jul 2021 18:06:36 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id EAB0761370 for ; Tue, 13 Jul 2021 18:06:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EAB0761370 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.155541.287157 (Exim 4.92) (envelope-from ) id 1m3Mnd-00043C-GH; Tue, 13 Jul 2021 18:06:29 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 155541.287157; Tue, 13 Jul 2021 18:06:29 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3Mnd-00042y-CI; Tue, 13 Jul 2021 18:06:29 +0000 Received: by outflank-mailman (input) for mailman id 155541; Tue, 13 Jul 2021 18:06:28 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3Mnb-0002kr-TC for xen-devel@lists.xenproject.org; Tue, 13 Jul 2021 18:06:27 +0000 Received: from mo4-p02-ob.smtp.rzone.de (unknown [81.169.146.171]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 04c70dca-e405-11eb-8793-12813bfff9fa; Tue, 13 Jul 2021 18:06:18 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.28.1 AUTH) with ESMTPSA id 30791cx6DI6DtAm (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Tue, 13 Jul 2021 20:06:13 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 04c70dca-e405-11eb-8793-12813bfff9fa DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1626199573; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=ZZ0YSTyEBhj0g0V+Ydgj4/Yi9sY29bXNxKbK7IWr/j0=; b=or2CNPZebb1mJf1DOfIfMr473GTLnQNrbvFumssG8yG0z+Smm2mRS83ULtrtnP75ZH TgmQuSoPN/BR1OBl7L6WB5GPFXElXHstZ3aK3FFmQzAFqND5XaAN/Vudw7UNcqV0QdNQ BjqonEGOaPVDKgRPoU0rd/Bme3L3sPasN3Z0kSNkraLKqyaBbnyLfnv5Tb0RmAmfUz+i CNDqpIWt8jxdxSEf2Ki3vbMQAp9XQIC5LMQQpTwn1BUYztfAv6n7/uYumOMrf6yzxcgn UfP+eg4KoCg4CeeMLIH2BzV106unOEyMB+IHTnlPsJptLQTf91ZHVfNLCfNOzqqHnW+2 gk2w== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAgthqlagRsstpH9XYEN5v9xf7dypFU9ge5p0Oi0V" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu , Juergen Gross Subject: [PATCH v20210713 05/31] tools: add readv_exact to libxenctrl Date: Tue, 13 Jul 2021 20:05:39 +0200 Message-Id: <20210713180605.12096-6-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210713180605.12096-1-olaf@aepfle.de> References: <20210713180605.12096-1-olaf@aepfle.de> MIME-Version: 1.0 Read a batch of iovec's. Short reads are the common case, finish the trailing iov with read_exact. Signed-off-by: Olaf Hering --- v2: - add comment to short-read handling --- tools/libs/ctrl/xc_private.c | 57 +++++++++++++++++++++++++++++++++++- tools/libs/ctrl/xc_private.h | 1 + 2 files changed, 57 insertions(+), 1 deletion(-) diff --git a/tools/libs/ctrl/xc_private.c b/tools/libs/ctrl/xc_private.c index c0422662f0..bab9a31a70 100644 --- a/tools/libs/ctrl/xc_private.c +++ b/tools/libs/ctrl/xc_private.c @@ -698,8 +698,23 @@ int write_exact(int fd, const void *data, size_t size) #if defined(__MINIOS__) /* - * MiniOS's libc doesn't know about writev(). Implement it as multiple write()s. + * MiniOS's libc doesn't know about readv/writev(). + * Implement it as multiple read/write()s. */ +int readv_exact(int fd, const struct iovec *iov, int iovcnt) +{ + int rc, i; + + for ( i = 0; i < iovcnt; ++i ) + { + rc = read_exact(fd, iov[i].iov_base, iov[i].iov_len); + if ( rc ) + return rc; + } + + return 0; +} + int writev_exact(int fd, const struct iovec *iov, int iovcnt) { int rc, i; @@ -714,6 +729,46 @@ int writev_exact(int fd, const struct iovec *iov, int iovcnt) return 0; } #else +int readv_exact(int fd, const struct iovec *iov, int iovcnt) +{ + int rc = 0, idx = 0; + ssize_t len; + + while ( idx < iovcnt ) + { + len = readv(fd, &iov[idx], min(iovcnt - idx, IOV_MAX)); + if ( len == -1 && errno == EINTR ) + continue; + if ( len <= 0 ) + { + rc = -1; + goto out; + } + + /* Finish a potential short read in the last iov */ + while ( len > 0 && idx < iovcnt ) + { + if ( len >= iov[idx].iov_len ) + { + len -= iov[idx].iov_len; + } + else + { + void *p = iov[idx].iov_base + len; + size_t l = iov[idx].iov_len - len; + + rc = read_exact(fd, p, l); + if ( rc ) + goto out; + len = 0; + } + idx++; + } + } +out: + return rc; +} + int writev_exact(int fd, const struct iovec *iov, int iovcnt) { struct iovec *local_iov = NULL; diff --git a/tools/libs/ctrl/xc_private.h b/tools/libs/ctrl/xc_private.h index 3e299b943f..66086ef19f 100644 --- a/tools/libs/ctrl/xc_private.h +++ b/tools/libs/ctrl/xc_private.h @@ -410,6 +410,7 @@ int xc_flush_mmu_updates(xc_interface *xch, struct xc_mmu *mmu); /* Return 0 on success; -1 on error setting errno. */ int read_exact(int fd, void *data, size_t size); /* EOF => -1, errno=0 */ +int readv_exact(int fd, const struct iovec *iov, int iovcnt); int write_exact(int fd, const void *data, size_t size); int writev_exact(int fd, const struct iovec *iov, int iovcnt); From patchwork Tue Jul 13 18:05:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12374687 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 038C4C07E95 for ; Tue, 13 Jul 2021 18:06:35 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AC81361370 for ; Tue, 13 Jul 2021 18:06:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AC81361370 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.155540.287146 (Exim 4.92) (envelope-from ) id 1m3Mna-0003fq-5m; Tue, 13 Jul 2021 18:06:26 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 155540.287146; Tue, 13 Jul 2021 18:06:26 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3Mna-0003fd-20; Tue, 13 Jul 2021 18:06:26 +0000 Received: by outflank-mailman (input) for mailman id 155540; Tue, 13 Jul 2021 18:06:24 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3MnY-0002UL-GA for xen-devel@lists.xenproject.org; Tue, 13 Jul 2021 18:06:24 +0000 Received: from mo4-p02-ob.smtp.rzone.de (unknown [81.169.146.169]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id b220a611-2cbe-434e-8ba0-601220fb6ed2; Tue, 13 Jul 2021 18:06:19 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.28.1 AUTH) with ESMTPSA id 30791cx6DI6DtAn (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Tue, 13 Jul 2021 20:06:13 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: b220a611-2cbe-434e-8ba0-601220fb6ed2 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1626199573; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=Clrkbg11Ho2H2QvphK9GJKKzvhCVY4UGRclGnlEVBNo=; b=ncMC99+8T5s1sxnqRqUJetKk4Ynh023mspDlqYFfw21vrPc9vPmbwd4iNgTktq39zg 5P2NX4LhClwagTbO9H2zaxMPd1Aj4d1iv1uDj+GUp3bG2nDEpLuO7WXlhZ9HONA+7DdP KARztatEq/Rc251lD7N2vUB6lTwbWxCg+JvcK5k7v6+x3JdfTZDbPWXB9uKLS1hwYkQn AAIHv0u8p34wWirG59r9QG/zhSYSDeSylI5/JgSfG67PqSWGh5YovuKXOWAMhOft0t1/ mpnsFdsKQYHYl9EKkLdKFo50ZmHwk4Wxzs4bQqjd34T/2FRiCemmLCn1I5MelgmGU2FY 5NSg== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAgthqlagRsstpH9XYEN5v9xf7dypFU9ge5p0Oi0V" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu , Juergen Gross Subject: [PATCH v20210713 06/31] tools: show migration transfer rate in send_dirty_pages Date: Tue, 13 Jul 2021 20:05:40 +0200 Message-Id: <20210713180605.12096-7-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210713180605.12096-1-olaf@aepfle.de> References: <20210713180605.12096-1-olaf@aepfle.de> MIME-Version: 1.0 Show how fast domU pages are transferred in each iteration. The relevant data is how fast the pfns travel, not so much how much protocol overhead exists. So the reported MiB/sec is just for pfns. Signed-off-by: Olaf Hering --- v02: - rearrange MiB_sec calculation (jgross) --- tools/libs/guest/xg_sr_common.h | 2 ++ tools/libs/guest/xg_sr_save.c | 46 +++++++++++++++++++++++++++++++++ 2 files changed, 48 insertions(+) diff --git a/tools/libs/guest/xg_sr_common.h b/tools/libs/guest/xg_sr_common.h index 36d45ef56f..2e70cc6269 100644 --- a/tools/libs/guest/xg_sr_common.h +++ b/tools/libs/guest/xg_sr_common.h @@ -238,6 +238,8 @@ struct xc_sr_context bool debug; unsigned long p2m_size; + size_t pages_sent; + size_t overhead_sent; struct precopy_stats stats; diff --git a/tools/libs/guest/xg_sr_save.c b/tools/libs/guest/xg_sr_save.c index 3fce41b6cf..00b81033a9 100644 --- a/tools/libs/guest/xg_sr_save.c +++ b/tools/libs/guest/xg_sr_save.c @@ -1,5 +1,6 @@ #include #include +#include #include "xg_sr_common.h" @@ -238,6 +239,8 @@ static int write_batch(struct xc_sr_context *ctx) iov[3].iov_len = nr_pfns * sizeof(*rec_pfns); iovcnt = 4; + ctx->save.pages_sent += nr_pages; + ctx->save.overhead_sent += sizeof(rec) + sizeof(hdr) + nr_pfns * sizeof(*rec_pfns); if ( nr_pages ) { @@ -357,6 +360,42 @@ static int suspend_domain(struct xc_sr_context *ctx) return 0; } +static void show_transfer_rate(struct xc_sr_context *ctx, struct timespec *start) +{ + xc_interface *xch = ctx->xch; + struct timespec end = {}, diff = {}; + size_t ms, MiB_sec; + + if (!ctx->save.pages_sent) + return; + + if ( clock_gettime(CLOCK_MONOTONIC, &end) ) + PERROR("clock_gettime"); + + if ( (end.tv_nsec - start->tv_nsec) < 0 ) + { + diff.tv_sec = end.tv_sec - start->tv_sec - 1; + diff.tv_nsec = end.tv_nsec - start->tv_nsec + (1000U*1000U*1000U); + } + else + { + diff.tv_sec = end.tv_sec - start->tv_sec; + diff.tv_nsec = end.tv_nsec - start->tv_nsec; + } + + ms = (diff.tv_nsec / (1000U*1000U)); + ms += (diff.tv_sec * 1000U); + if (!ms) + ms = 1; + + MiB_sec = (ctx->save.pages_sent * PAGE_SIZE * 1000U) / ms / (1024U*1024U); + + errno = 0; + IPRINTF("%s: %zu bytes + %zu pages in %ld.%09ld sec, %zu MiB/sec", __func__, + ctx->save.overhead_sent, ctx->save.pages_sent, + diff.tv_sec, diff.tv_nsec, MiB_sec); +} + /* * Send a subset of pages in the guests p2m, according to the dirty bitmap. * Used for each subsequent iteration of the live migration loop. @@ -370,9 +409,15 @@ static int send_dirty_pages(struct xc_sr_context *ctx, xen_pfn_t p; unsigned long written; int rc; + struct timespec start = {}; DECLARE_HYPERCALL_BUFFER_SHADOW(unsigned long, dirty_bitmap, &ctx->save.dirty_bitmap_hbuf); + ctx->save.pages_sent = 0; + ctx->save.overhead_sent = 0; + if ( clock_gettime(CLOCK_MONOTONIC, &start) ) + PERROR("clock_gettime"); + for ( p = 0, written = 0; p < ctx->save.p2m_size; ++p ) { if ( !test_bit(p, dirty_bitmap) ) @@ -396,6 +441,7 @@ static int send_dirty_pages(struct xc_sr_context *ctx, if ( written > entries ) DPRINTF("Bitmap contained more entries than expected..."); + show_transfer_rate(ctx, &start); xc_report_progress_step(xch, entries, entries); return ctx->save.ops.check_vm_state(ctx); From patchwork Tue Jul 13 18:05:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12374691 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A5395C07E96 for ; Tue, 13 Jul 2021 18:06:39 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 689F061369 for ; Tue, 13 Jul 2021 18:06:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 689F061369 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.155542.287167 (Exim 4.92) (envelope-from ) id 1m3Mnf-0004Rl-QV; Tue, 13 Jul 2021 18:06:31 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 155542.287167; Tue, 13 Jul 2021 18:06:31 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3Mnf-0004RT-MM; Tue, 13 Jul 2021 18:06:31 +0000 Received: by outflank-mailman (input) for mailman id 155542; Tue, 13 Jul 2021 18:06:29 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3Mnd-0002UL-GI for xen-devel@lists.xenproject.org; Tue, 13 Jul 2021 18:06:29 +0000 Received: from mo4-p02-ob.smtp.rzone.de (unknown [81.169.146.170]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id d82de438-271f-492e-98e5-d7dd1528d28f; Tue, 13 Jul 2021 18:06:19 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.28.1 AUTH) with ESMTPSA id 30791cx6DI6DtAo (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Tue, 13 Jul 2021 20:06:13 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: d82de438-271f-492e-98e5-d7dd1528d28f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1626199574; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=2FOHALqjtAayiATag1w1tv+uvw3U+jkLCXoGYWrbCds=; b=VO3jOq8wx19i17M6djKvvv/rVPvoCedUnBlNa2YflKQht9tDddIEXbmES7UG+Yr01N OTB/5NSLG+V/an6eErcPVeqqtATRJkX+NgxBqZweHpvzbjcE1ceMWIkqJZvssjYu58j8 6QnNMi2bl8Nc6/WDFXKlh64L8jtDLezKsrE5g5rhNKa/mmaN3BvQzcW9rVisOx8n7BZz GvWh+EnRgbstv5CEyCzn8YgnM0aPaJahvGKajWPvkOeEYWRBEZYeeErf7/0D/te9hPF6 u0tDjCIDx25bBjCPzyYYMF7W8qr0OXxj7pa/umD/GUjl6EILnS3fITrfEKxJ3MBXxFMB OSZw== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAgthqlagRsstpH9XYEN5v9xf7dypFU9ge5p0Oi0V" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu , Juergen Gross Subject: [PATCH v20210713 07/31] tools: save: preallocate mfns array Date: Tue, 13 Jul 2021 20:05:41 +0200 Message-Id: <20210713180605.12096-8-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210713180605.12096-1-olaf@aepfle.de> References: <20210713180605.12096-1-olaf@aepfle.de> MIME-Version: 1.0 Remove repeated allocation from migration loop. There will never be more than MAX_BATCH_SIZE pages to process in a batch, see add_to_batch. Allocate the space once. Signed-off-by: Olaf Hering --- tools/libs/guest/xg_sr_common.h | 1 + tools/libs/guest/xg_sr_save.c | 25 +++++++++++++------------ 2 files changed, 14 insertions(+), 12 deletions(-) diff --git a/tools/libs/guest/xg_sr_common.h b/tools/libs/guest/xg_sr_common.h index 2e70cc6269..fc403adc64 100644 --- a/tools/libs/guest/xg_sr_common.h +++ b/tools/libs/guest/xg_sr_common.h @@ -244,6 +244,7 @@ struct xc_sr_context struct precopy_stats stats; xen_pfn_t *batch_pfns; + xen_pfn_t *mfns; unsigned int nr_batch_pfns; unsigned long *deferred_pages; unsigned long nr_deferred_pages; diff --git a/tools/libs/guest/xg_sr_save.c b/tools/libs/guest/xg_sr_save.c index 00b81033a9..0abe63a377 100644 --- a/tools/libs/guest/xg_sr_save.c +++ b/tools/libs/guest/xg_sr_save.c @@ -88,7 +88,7 @@ static int write_checkpoint_record(struct xc_sr_context *ctx) static int write_batch(struct xc_sr_context *ctx) { xc_interface *xch = ctx->xch; - xen_pfn_t *mfns = NULL, *types = NULL; + xen_pfn_t *types = NULL; void *guest_mapping = NULL; void **guest_data = NULL; void **local_pages = NULL; @@ -105,8 +105,6 @@ static int write_batch(struct xc_sr_context *ctx) assert(nr_pfns != 0); - /* Mfns of the batch pfns. */ - mfns = malloc(nr_pfns * sizeof(*mfns)); /* Types of the batch pfns. */ types = malloc(nr_pfns * sizeof(*types)); /* Errors from attempting to map the gfns. */ @@ -118,7 +116,7 @@ static int write_batch(struct xc_sr_context *ctx) /* iovec[] for writev(). */ iov = malloc((nr_pfns + 4) * sizeof(*iov)); - if ( !mfns || !types || !errors || !guest_data || !local_pages || !iov ) + if ( !types || !errors || !guest_data || !local_pages || !iov ) { ERROR("Unable to allocate arrays for a batch of %u pages", nr_pfns); @@ -127,11 +125,11 @@ static int write_batch(struct xc_sr_context *ctx) for ( i = 0; i < nr_pfns; ++i ) { - types[i] = mfns[i] = ctx->save.ops.pfn_to_gfn(ctx, + types[i] = ctx->save.mfns[i] = ctx->save.ops.pfn_to_gfn(ctx, ctx->save.batch_pfns[i]); /* Likely a ballooned page. */ - if ( mfns[i] == INVALID_MFN ) + if ( ctx->save.mfns[i] == INVALID_MFN ) { set_bit(ctx->save.batch_pfns[i], ctx->save.deferred_pages); ++ctx->save.nr_deferred_pages; @@ -150,20 +148,21 @@ static int write_batch(struct xc_sr_context *ctx) { if ( !is_known_page_type(types[i]) ) { - ERROR("Unknown type %#"PRIpfn" for pfn %#"PRIpfn, types[i], mfns[i]); + ERROR("Unknown type %#"PRIpfn" for pfn %#"PRIpfn, + types[i], ctx->save.mfns[i]); goto err; } if ( !page_type_has_stream_data(types[i]) ) continue; - mfns[nr_pages++] = mfns[i]; + ctx->save.mfns[nr_pages++] = ctx->save.mfns[i]; } if ( nr_pages > 0 ) { guest_mapping = xenforeignmemory_map( - xch->fmem, ctx->domid, PROT_READ, nr_pages, mfns, errors); + xch->fmem, ctx->domid, PROT_READ, nr_pages, ctx->save.mfns, errors); if ( !guest_mapping ) { PERROR("Failed to map guest pages"); @@ -179,7 +178,7 @@ static int write_batch(struct xc_sr_context *ctx) if ( errors[p] ) { ERROR("Mapping of pfn %#"PRIpfn" (mfn %#"PRIpfn") failed %d", - ctx->save.batch_pfns[i], mfns[p], errors[p]); + ctx->save.batch_pfns[i], ctx->save.mfns[p], errors[p]); goto err; } @@ -277,7 +276,6 @@ static int write_batch(struct xc_sr_context *ctx) free(guest_data); free(errors); free(types); - free(mfns); return rc; } @@ -851,9 +849,11 @@ static int setup(struct xc_sr_context *ctx) xch, dirty_bitmap, NRPAGES(bitmap_size(ctx->save.p2m_size))); ctx->save.batch_pfns = malloc(MAX_BATCH_SIZE * sizeof(*ctx->save.batch_pfns)); + ctx->save.mfns = malloc(MAX_BATCH_SIZE * sizeof(*ctx->save.mfns)); ctx->save.deferred_pages = bitmap_alloc(ctx->save.p2m_size); - if ( !ctx->save.batch_pfns || !dirty_bitmap || !ctx->save.deferred_pages ) + if ( !ctx->save.batch_pfns || !ctx->save.mfns || + !dirty_bitmap || !ctx->save.deferred_pages ) { ERROR("Unable to allocate memory for dirty bitmaps, batch pfns and" " deferred pages"); @@ -884,6 +884,7 @@ static void cleanup(struct xc_sr_context *ctx) xc_hypercall_buffer_free_pages(xch, dirty_bitmap, NRPAGES(bitmap_size(ctx->save.p2m_size))); free(ctx->save.deferred_pages); + free(ctx->save.mfns); free(ctx->save.batch_pfns); } From patchwork Tue Jul 13 18:05:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12374697 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C6D26C07E96 for ; Tue, 13 Jul 2021 18:06:46 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8E2DF61370 for ; Tue, 13 Jul 2021 18:06:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8E2DF61370 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.155545.287201 (Exim 4.92) (envelope-from ) id 1m3Mnn-0005kW-6b; Tue, 13 Jul 2021 18:06:39 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 155545.287201; Tue, 13 Jul 2021 18:06:39 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3Mnn-0005jo-0s; Tue, 13 Jul 2021 18:06:39 +0000 Received: by outflank-mailman (input) for mailman id 155545; Tue, 13 Jul 2021 18:06:38 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3Mnl-0002kr-TS for xen-devel@lists.xenproject.org; Tue, 13 Jul 2021 18:06:37 +0000 Received: from mo4-p02-ob.smtp.rzone.de (unknown [85.215.255.80]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 05524e77-e405-11eb-8793-12813bfff9fa; Tue, 13 Jul 2021 18:06:20 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.28.1 AUTH) with ESMTPSA id 30791cx6DI6EtAp (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Tue, 13 Jul 2021 20:06:14 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 05524e77-e405-11eb-8793-12813bfff9fa DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1626199574; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=LtyVn9iWhbeG3QGf2nqJ2jlqNwljeyLy9bqS6muxnCY=; b=TwKmNEFGrByXOUSKlKyVErjP2hkP9l9R4JHcqbeop4rPR+2Ihe1ItAOP8mHGOSz2zw msCAcjiJ8UYk8Srchu1mPulRWdnpVi2SGwA1pKEFiNnnM6FdYn1ZkSz13UqG4RJJfHvl 1yTz1mqfJU69hYPBsujLWN0cCDqCI2bZ4RF9XDQptM84w9+buvB4PuCBfUkSwchaI9Li umqESpwS+me9yWnyMdmQk9qenWTiCWtD8ffiPQuE7ZxE4wsnkgxhG71YVHINCSfT3ES1 v3Xe5Stifm8ZMJFPcFUk1NqWHbCrFc4Sc1yNjkmT79ZiNsgxkM98YoniQMOP7kCscPa5 n8sw== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAgthqlagRsstpH9XYEN5v9xf7dypFU9ge5p0Oi0V" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu , Juergen Gross Subject: [PATCH v20210713 08/31] tools: save: preallocate types array Date: Tue, 13 Jul 2021 20:05:42 +0200 Message-Id: <20210713180605.12096-9-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210713180605.12096-1-olaf@aepfle.de> References: <20210713180605.12096-1-olaf@aepfle.de> MIME-Version: 1.0 Remove repeated allocation from migration loop. There will never be more than MAX_BATCH_SIZE pages to process in a batch. Allocate the space once. Signed-off-by: Olaf Hering --- tools/libs/guest/xg_sr_common.h | 1 + tools/libs/guest/xg_sr_save.c | 28 +++++++++++++--------------- 2 files changed, 14 insertions(+), 15 deletions(-) diff --git a/tools/libs/guest/xg_sr_common.h b/tools/libs/guest/xg_sr_common.h index fc403adc64..d80de6e7da 100644 --- a/tools/libs/guest/xg_sr_common.h +++ b/tools/libs/guest/xg_sr_common.h @@ -245,6 +245,7 @@ struct xc_sr_context xen_pfn_t *batch_pfns; xen_pfn_t *mfns; + xen_pfn_t *types; unsigned int nr_batch_pfns; unsigned long *deferred_pages; unsigned long nr_deferred_pages; diff --git a/tools/libs/guest/xg_sr_save.c b/tools/libs/guest/xg_sr_save.c index 0abe63a377..29a5fe82f0 100644 --- a/tools/libs/guest/xg_sr_save.c +++ b/tools/libs/guest/xg_sr_save.c @@ -88,7 +88,6 @@ static int write_checkpoint_record(struct xc_sr_context *ctx) static int write_batch(struct xc_sr_context *ctx) { xc_interface *xch = ctx->xch; - xen_pfn_t *types = NULL; void *guest_mapping = NULL; void **guest_data = NULL; void **local_pages = NULL; @@ -105,8 +104,6 @@ static int write_batch(struct xc_sr_context *ctx) assert(nr_pfns != 0); - /* Types of the batch pfns. */ - types = malloc(nr_pfns * sizeof(*types)); /* Errors from attempting to map the gfns. */ errors = malloc(nr_pfns * sizeof(*errors)); /* Pointers to page data to send. Mapped gfns or local allocations. */ @@ -116,7 +113,7 @@ static int write_batch(struct xc_sr_context *ctx) /* iovec[] for writev(). */ iov = malloc((nr_pfns + 4) * sizeof(*iov)); - if ( !types || !errors || !guest_data || !local_pages || !iov ) + if ( !errors || !guest_data || !local_pages || !iov ) { ERROR("Unable to allocate arrays for a batch of %u pages", nr_pfns); @@ -125,7 +122,7 @@ static int write_batch(struct xc_sr_context *ctx) for ( i = 0; i < nr_pfns; ++i ) { - types[i] = ctx->save.mfns[i] = ctx->save.ops.pfn_to_gfn(ctx, + ctx->save.types[i] = ctx->save.mfns[i] = ctx->save.ops.pfn_to_gfn(ctx, ctx->save.batch_pfns[i]); /* Likely a ballooned page. */ @@ -136,7 +133,7 @@ static int write_batch(struct xc_sr_context *ctx) } } - rc = xc_get_pfn_type_batch(xch, ctx->domid, nr_pfns, types); + rc = xc_get_pfn_type_batch(xch, ctx->domid, nr_pfns, ctx->save.types); if ( rc ) { PERROR("Failed to get types for pfn batch"); @@ -146,14 +143,14 @@ static int write_batch(struct xc_sr_context *ctx) for ( i = 0; i < nr_pfns; ++i ) { - if ( !is_known_page_type(types[i]) ) + if ( !is_known_page_type(ctx->save.types[i]) ) { ERROR("Unknown type %#"PRIpfn" for pfn %#"PRIpfn, - types[i], ctx->save.mfns[i]); + ctx->save.types[i], ctx->save.mfns[i]); goto err; } - if ( !page_type_has_stream_data(types[i]) ) + if ( !page_type_has_stream_data(ctx->save.types[i]) ) continue; ctx->save.mfns[nr_pages++] = ctx->save.mfns[i]; @@ -172,7 +169,7 @@ static int write_batch(struct xc_sr_context *ctx) for ( i = 0, p = 0; i < nr_pfns; ++i ) { - if ( !page_type_has_stream_data(types[i]) ) + if ( !page_type_has_stream_data(ctx->save.types[i]) ) continue; if ( errors[p] ) @@ -183,7 +180,7 @@ static int write_batch(struct xc_sr_context *ctx) } orig_page = page = guest_mapping + (p * PAGE_SIZE); - rc = ctx->save.ops.normalise_page(ctx, types[i], &page); + rc = ctx->save.ops.normalise_page(ctx, ctx->save.types[i], &page); if ( orig_page != page ) local_pages[i] = page; @@ -194,7 +191,7 @@ static int write_batch(struct xc_sr_context *ctx) { set_bit(ctx->save.batch_pfns[i], ctx->save.deferred_pages); ++ctx->save.nr_deferred_pages; - types[i] = XEN_DOMCTL_PFINFO_XTAB; + ctx->save.types[i] = XEN_DOMCTL_PFINFO_XTAB; --nr_pages; } else @@ -223,7 +220,7 @@ static int write_batch(struct xc_sr_context *ctx) rec.length += nr_pages * PAGE_SIZE; for ( i = 0; i < nr_pfns; ++i ) - rec_pfns[i] = ((uint64_t)(types[i]) << 32) | ctx->save.batch_pfns[i]; + rec_pfns[i] = ((uint64_t)(ctx->save.types[i]) << 32) | ctx->save.batch_pfns[i]; iov[0].iov_base = &rec.type; iov[0].iov_len = sizeof(rec.type); @@ -275,7 +272,6 @@ static int write_batch(struct xc_sr_context *ctx) free(local_pages); free(guest_data); free(errors); - free(types); return rc; } @@ -850,9 +846,10 @@ static int setup(struct xc_sr_context *ctx) ctx->save.batch_pfns = malloc(MAX_BATCH_SIZE * sizeof(*ctx->save.batch_pfns)); ctx->save.mfns = malloc(MAX_BATCH_SIZE * sizeof(*ctx->save.mfns)); + ctx->save.types = malloc(MAX_BATCH_SIZE * sizeof(*ctx->save.types)); ctx->save.deferred_pages = bitmap_alloc(ctx->save.p2m_size); - if ( !ctx->save.batch_pfns || !ctx->save.mfns || + if ( !ctx->save.batch_pfns || !ctx->save.mfns || !ctx->save.types || !dirty_bitmap || !ctx->save.deferred_pages ) { ERROR("Unable to allocate memory for dirty bitmaps, batch pfns and" @@ -884,6 +881,7 @@ static void cleanup(struct xc_sr_context *ctx) xc_hypercall_buffer_free_pages(xch, dirty_bitmap, NRPAGES(bitmap_size(ctx->save.p2m_size))); free(ctx->save.deferred_pages); + free(ctx->save.types); free(ctx->save.mfns); free(ctx->save.batch_pfns); } From patchwork Tue Jul 13 18:05:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12374695 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C5F48C07E95 for ; Tue, 13 Jul 2021 18:06:44 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9753861370 for ; Tue, 13 Jul 2021 18:06:44 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9753861370 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.155544.287190 (Exim 4.92) (envelope-from ) id 1m3Mnk-0005Ja-PS; Tue, 13 Jul 2021 18:06:36 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 155544.287190; Tue, 13 Jul 2021 18:06:36 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3Mnk-0005JJ-K8; Tue, 13 Jul 2021 18:06:36 +0000 Received: by outflank-mailman (input) for mailman id 155544; Tue, 13 Jul 2021 18:06:34 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3Mni-0002UL-GY for xen-devel@lists.xenproject.org; Tue, 13 Jul 2021 18:06:34 +0000 Received: from mo4-p02-ob.smtp.rzone.de (unknown [85.215.255.84]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 7b4da61c-8660-40a3-9938-c1ee21a1e524; Tue, 13 Jul 2021 18:06:21 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.28.1 AUTH) with ESMTPSA id 30791cx6DI6EtAq (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Tue, 13 Jul 2021 20:06:14 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 7b4da61c-8660-40a3-9938-c1ee21a1e524 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1626199575; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=JHY4TWvZbK0Ujx5K1FzKpVZtdjbItuCiEYMs+LDvHbM=; b=ljQs4jTzCjTxtbdo8sA7N9xF0Frzyd3ti0GAU/YGeaD7VRXzfPhSOkiqaYr2A0oUEi 2FmWlhYXYGqpOgyoA+xVhj8VwyraZda50fG1ARqL4PwvAFBcL9o366CrhXst+Wh1fMCA 2KyCI+nIy2UERtzUCcbR83PRD8/Z6hP9HZI3SY5+vYNJG+lXVj3c95LxB+pEdx3rdgEE ZIiN80EuBYQ0UjKaR7E+yP5rGq720osynus+Jzlgp8ZRr0lpaKpJ3UsttsWSkW6XD+JR JAjNy9txaZCln6fLDH1BQ5jDWhBBlC5yY1488B9HlueYEMGAI7MMY0yZhytJpLEfSjty VgEg== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAgthqlagRsstpH9XYEN5v9xf7dypFU9ge5p0Oi0V" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu , Juergen Gross Subject: [PATCH v20210713 09/31] tools: save: preallocate errors array Date: Tue, 13 Jul 2021 20:05:43 +0200 Message-Id: <20210713180605.12096-10-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210713180605.12096-1-olaf@aepfle.de> References: <20210713180605.12096-1-olaf@aepfle.de> MIME-Version: 1.0 Remove repeated allocation from migration loop. There will never be more than MAX_BATCH_SIZE pages to process in a batch. Allocate the space once. Signed-off-by: Olaf Hering --- tools/libs/guest/xg_sr_common.h | 1 + tools/libs/guest/xg_sr_save.c | 20 ++++++++++---------- 2 files changed, 11 insertions(+), 10 deletions(-) diff --git a/tools/libs/guest/xg_sr_common.h b/tools/libs/guest/xg_sr_common.h index d80de6e7da..d2b838ed33 100644 --- a/tools/libs/guest/xg_sr_common.h +++ b/tools/libs/guest/xg_sr_common.h @@ -246,6 +246,7 @@ struct xc_sr_context xen_pfn_t *batch_pfns; xen_pfn_t *mfns; xen_pfn_t *types; + int *errors; unsigned int nr_batch_pfns; unsigned long *deferred_pages; unsigned long nr_deferred_pages; diff --git a/tools/libs/guest/xg_sr_save.c b/tools/libs/guest/xg_sr_save.c index 29a5fe82f0..5b416fda60 100644 --- a/tools/libs/guest/xg_sr_save.c +++ b/tools/libs/guest/xg_sr_save.c @@ -91,7 +91,7 @@ static int write_batch(struct xc_sr_context *ctx) void *guest_mapping = NULL; void **guest_data = NULL; void **local_pages = NULL; - int *errors = NULL, rc = -1; + int rc = -1; unsigned int i, p, nr_pages = 0, nr_pages_mapped = 0; unsigned int nr_pfns = ctx->save.nr_batch_pfns; void *page, *orig_page; @@ -104,8 +104,6 @@ static int write_batch(struct xc_sr_context *ctx) assert(nr_pfns != 0); - /* Errors from attempting to map the gfns. */ - errors = malloc(nr_pfns * sizeof(*errors)); /* Pointers to page data to send. Mapped gfns or local allocations. */ guest_data = calloc(nr_pfns, sizeof(*guest_data)); /* Pointers to locally allocated pages. Need freeing. */ @@ -113,7 +111,7 @@ static int write_batch(struct xc_sr_context *ctx) /* iovec[] for writev(). */ iov = malloc((nr_pfns + 4) * sizeof(*iov)); - if ( !errors || !guest_data || !local_pages || !iov ) + if ( !guest_data || !local_pages || !iov ) { ERROR("Unable to allocate arrays for a batch of %u pages", nr_pfns); @@ -158,8 +156,8 @@ static int write_batch(struct xc_sr_context *ctx) if ( nr_pages > 0 ) { - guest_mapping = xenforeignmemory_map( - xch->fmem, ctx->domid, PROT_READ, nr_pages, ctx->save.mfns, errors); + guest_mapping = xenforeignmemory_map(xch->fmem, ctx->domid, PROT_READ, + nr_pages, ctx->save.mfns, ctx->save.errors); if ( !guest_mapping ) { PERROR("Failed to map guest pages"); @@ -172,10 +170,11 @@ static int write_batch(struct xc_sr_context *ctx) if ( !page_type_has_stream_data(ctx->save.types[i]) ) continue; - if ( errors[p] ) + if ( ctx->save.errors[p] ) { ERROR("Mapping of pfn %#"PRIpfn" (mfn %#"PRIpfn") failed %d", - ctx->save.batch_pfns[i], ctx->save.mfns[p], errors[p]); + ctx->save.batch_pfns[i], ctx->save.mfns[p], + ctx->save.errors[p]); goto err; } @@ -271,7 +270,6 @@ static int write_batch(struct xc_sr_context *ctx) free(iov); free(local_pages); free(guest_data); - free(errors); return rc; } @@ -847,10 +845,11 @@ static int setup(struct xc_sr_context *ctx) sizeof(*ctx->save.batch_pfns)); ctx->save.mfns = malloc(MAX_BATCH_SIZE * sizeof(*ctx->save.mfns)); ctx->save.types = malloc(MAX_BATCH_SIZE * sizeof(*ctx->save.types)); + ctx->save.errors = malloc(MAX_BATCH_SIZE * sizeof(*ctx->save.errors)); ctx->save.deferred_pages = bitmap_alloc(ctx->save.p2m_size); if ( !ctx->save.batch_pfns || !ctx->save.mfns || !ctx->save.types || - !dirty_bitmap || !ctx->save.deferred_pages ) + !ctx->save.errors || !dirty_bitmap || !ctx->save.deferred_pages ) { ERROR("Unable to allocate memory for dirty bitmaps, batch pfns and" " deferred pages"); @@ -881,6 +880,7 @@ static void cleanup(struct xc_sr_context *ctx) xc_hypercall_buffer_free_pages(xch, dirty_bitmap, NRPAGES(bitmap_size(ctx->save.p2m_size))); free(ctx->save.deferred_pages); + free(ctx->save.errors); free(ctx->save.types); free(ctx->save.mfns); free(ctx->save.batch_pfns); From patchwork Tue Jul 13 18:05:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12374705 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 799C3C11F66 for ; Tue, 13 Jul 2021 18:06:56 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4113161369 for ; Tue, 13 Jul 2021 18:06:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4113161369 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.155551.287245 (Exim 4.92) (envelope-from ) id 1m3Mnx-0007zy-EL; Tue, 13 Jul 2021 18:06:49 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 155551.287245; Tue, 13 Jul 2021 18:06:49 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3Mnx-0007z7-6i; Tue, 13 Jul 2021 18:06:49 +0000 Received: by outflank-mailman (input) for mailman id 155551; Tue, 13 Jul 2021 18:06:48 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3Mnv-0002kr-Tv for xen-devel@lists.xenproject.org; Tue, 13 Jul 2021 18:06:47 +0000 Received: from mo4-p02-ob.smtp.rzone.de (unknown [85.215.255.83]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 06e2c55c-e405-11eb-8793-12813bfff9fa; Tue, 13 Jul 2021 18:06:21 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.28.1 AUTH) with ESMTPSA id 30791cx6DI6FtAr (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Tue, 13 Jul 2021 20:06:15 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 06e2c55c-e405-11eb-8793-12813bfff9fa DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1626199575; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=VixOpRBMsoM0fDQKIvQSxrobwoztUo22eEi66PpWlks=; b=AVCvEuKrE0LIt1SG29Y5us1wSfMsF0h+fPKTaGaZ1W8adH59yDqT9H6WSPjLOeG3d6 rpKNTUQpwPwQFnoJZHAe244nYJVrojPKzFNAQ+YM9gIwKf0BLXXS69yePFMivqBapQvD FKUhywPEjBhWuktE5tXwjuXXApsN/7w17X4mYRITmpgVLOnSM4+IB+y4tpauw6n9gSQH vdDl7o/FHRHemtvrfZ+6ihl4scNghZeomLPtMWfgUuwPvRHn+oh29q4gJCMchZh5HDHm tou+a0RAQAobxOmqomf0F9VqqfKznq7KPffA66if6z4aQqCaX55dtnXyg0QR4ub1N6q5 LDRQ== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAgthqlagRsstpH9XYEN5v9xf7dypFU9ge5p0Oi0V" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu , Juergen Gross Subject: [PATCH v20210713 10/31] tools: save: preallocate iov array Date: Tue, 13 Jul 2021 20:05:44 +0200 Message-Id: <20210713180605.12096-11-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210713180605.12096-1-olaf@aepfle.de> References: <20210713180605.12096-1-olaf@aepfle.de> MIME-Version: 1.0 Remove repeated allocation from migration loop. There will never be more than MAX_BATCH_SIZE pages to process in a batch. Allocate the space once. Signed-off-by: Olaf Hering --- tools/libs/guest/xg_sr_common.h | 1 + tools/libs/guest/xg_sr_save.c | 34 ++++++++++++++++----------------- 2 files changed, 18 insertions(+), 17 deletions(-) diff --git a/tools/libs/guest/xg_sr_common.h b/tools/libs/guest/xg_sr_common.h index d2b838ed33..e2a068bb7e 100644 --- a/tools/libs/guest/xg_sr_common.h +++ b/tools/libs/guest/xg_sr_common.h @@ -247,6 +247,7 @@ struct xc_sr_context xen_pfn_t *mfns; xen_pfn_t *types; int *errors; + struct iovec *iov; unsigned int nr_batch_pfns; unsigned long *deferred_pages; unsigned long nr_deferred_pages; diff --git a/tools/libs/guest/xg_sr_save.c b/tools/libs/guest/xg_sr_save.c index 5b416fda60..92eba456e6 100644 --- a/tools/libs/guest/xg_sr_save.c +++ b/tools/libs/guest/xg_sr_save.c @@ -96,7 +96,7 @@ static int write_batch(struct xc_sr_context *ctx) unsigned int nr_pfns = ctx->save.nr_batch_pfns; void *page, *orig_page; uint64_t *rec_pfns = NULL; - struct iovec *iov = NULL; int iovcnt = 0; + int iovcnt = 0; struct xc_sr_rec_page_data_header hdr = { 0 }; struct xc_sr_record rec = { .type = REC_TYPE_PAGE_DATA, @@ -108,10 +108,8 @@ static int write_batch(struct xc_sr_context *ctx) guest_data = calloc(nr_pfns, sizeof(*guest_data)); /* Pointers to locally allocated pages. Need freeing. */ local_pages = calloc(nr_pfns, sizeof(*local_pages)); - /* iovec[] for writev(). */ - iov = malloc((nr_pfns + 4) * sizeof(*iov)); - if ( !guest_data || !local_pages || !iov ) + if ( !guest_data || !local_pages ) { ERROR("Unable to allocate arrays for a batch of %u pages", nr_pfns); @@ -221,17 +219,17 @@ static int write_batch(struct xc_sr_context *ctx) for ( i = 0; i < nr_pfns; ++i ) rec_pfns[i] = ((uint64_t)(ctx->save.types[i]) << 32) | ctx->save.batch_pfns[i]; - iov[0].iov_base = &rec.type; - iov[0].iov_len = sizeof(rec.type); + ctx->save.iov[0].iov_base = &rec.type; + ctx->save.iov[0].iov_len = sizeof(rec.type); - iov[1].iov_base = &rec.length; - iov[1].iov_len = sizeof(rec.length); + ctx->save.iov[1].iov_base = &rec.length; + ctx->save.iov[1].iov_len = sizeof(rec.length); - iov[2].iov_base = &hdr; - iov[2].iov_len = sizeof(hdr); + ctx->save.iov[2].iov_base = &hdr; + ctx->save.iov[2].iov_len = sizeof(hdr); - iov[3].iov_base = rec_pfns; - iov[3].iov_len = nr_pfns * sizeof(*rec_pfns); + ctx->save.iov[3].iov_base = rec_pfns; + ctx->save.iov[3].iov_len = nr_pfns * sizeof(*rec_pfns); iovcnt = 4; ctx->save.pages_sent += nr_pages; @@ -243,15 +241,15 @@ static int write_batch(struct xc_sr_context *ctx) { if ( guest_data[i] ) { - iov[iovcnt].iov_base = guest_data[i]; - iov[iovcnt].iov_len = PAGE_SIZE; + ctx->save.iov[iovcnt].iov_base = guest_data[i]; + ctx->save.iov[iovcnt].iov_len = PAGE_SIZE; iovcnt++; --nr_pages; } } } - if ( writev_exact(ctx->fd, iov, iovcnt) ) + if ( writev_exact(ctx->fd, ctx->save.iov, iovcnt) ) { PERROR("Failed to write page data to stream"); goto err; @@ -267,7 +265,6 @@ static int write_batch(struct xc_sr_context *ctx) xenforeignmemory_unmap(xch->fmem, guest_mapping, nr_pages_mapped); for ( i = 0; local_pages && i < nr_pfns; ++i ) free(local_pages[i]); - free(iov); free(local_pages); free(guest_data); @@ -846,10 +843,12 @@ static int setup(struct xc_sr_context *ctx) ctx->save.mfns = malloc(MAX_BATCH_SIZE * sizeof(*ctx->save.mfns)); ctx->save.types = malloc(MAX_BATCH_SIZE * sizeof(*ctx->save.types)); ctx->save.errors = malloc(MAX_BATCH_SIZE * sizeof(*ctx->save.errors)); + ctx->save.iov = malloc((4 + MAX_BATCH_SIZE) * sizeof(*ctx->save.iov)); ctx->save.deferred_pages = bitmap_alloc(ctx->save.p2m_size); if ( !ctx->save.batch_pfns || !ctx->save.mfns || !ctx->save.types || - !ctx->save.errors || !dirty_bitmap || !ctx->save.deferred_pages ) + !ctx->save.errors || !ctx->save.iov || !dirty_bitmap || + !ctx->save.deferred_pages ) { ERROR("Unable to allocate memory for dirty bitmaps, batch pfns and" " deferred pages"); @@ -880,6 +879,7 @@ static void cleanup(struct xc_sr_context *ctx) xc_hypercall_buffer_free_pages(xch, dirty_bitmap, NRPAGES(bitmap_size(ctx->save.p2m_size))); free(ctx->save.deferred_pages); + free(ctx->save.iov); free(ctx->save.errors); free(ctx->save.types); free(ctx->save.mfns); From patchwork Tue Jul 13 18:05:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12374701 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6DD5FC07E96 for ; Tue, 13 Jul 2021 18:06:51 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 41BDB61370 for ; Tue, 13 Jul 2021 18:06:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 41BDB61370 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.155549.287223 (Exim 4.92) (envelope-from ) id 1m3Mns-0006tY-IY; Tue, 13 Jul 2021 18:06:44 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 155549.287223; Tue, 13 Jul 2021 18:06:44 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3Mns-0006sL-Cs; Tue, 13 Jul 2021 18:06:44 +0000 Received: by outflank-mailman (input) for mailman id 155549; Tue, 13 Jul 2021 18:06:43 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3Mnq-0002kr-To for xen-devel@lists.xenproject.org; Tue, 13 Jul 2021 18:06:42 +0000 Received: from mo4-p02-ob.smtp.rzone.de (unknown [85.215.255.84]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 06d8627e-e405-11eb-8793-12813bfff9fa; Tue, 13 Jul 2021 18:06:21 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.28.1 AUTH) with ESMTPSA id 30791cx6DI6FtAs (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Tue, 13 Jul 2021 20:06:15 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 06d8627e-e405-11eb-8793-12813bfff9fa DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1626199575; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=XZ2E1yVunQhUbaAeVwgeHEncoGw74SQmLvitByCyUhE=; b=RHia578d3804xnF/HNtoZnA2X/2CGarVamyyO7j3GnUaX6TPSHvkUoZK4DBlJjFUZB g0x8gmoFUdYRSH6umMAoyT5JHTyV03egjJudIRQKn7B24wzYCOywDQkgjoP6exjcu+yj ymO6EU2PhPiiAcIdfm7PVZbfvi69FQhGKeK+fPBcdtYd35jy1vl1d9qH6bbPNceObsx9 /0WefHFCiwMB1CCwRafv/C6SDJeoBqXdn4GX7nZCjyiS5ATOFWL9wfiLB3PzCgflbUbx YnmE9XevkbQlXt50QBTHjsaqpCmcliu6AD/G2JtMrhEORfuINwgA6Hi7eNb/1ZGUBxOf RegA== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAgthqlagRsstpH9XYEN5v9xf7dypFU9ge5p0Oi0V" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu , Juergen Gross Subject: [PATCH v20210713 11/31] tools: save: preallocate rec_pfns array Date: Tue, 13 Jul 2021 20:05:45 +0200 Message-Id: <20210713180605.12096-12-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210713180605.12096-1-olaf@aepfle.de> References: <20210713180605.12096-1-olaf@aepfle.de> MIME-Version: 1.0 Remove repeated allocation from migration loop. There will never be more than MAX_BATCH_SIZE pages to process in a batch. Allocate the space once. Signed-off-by: Olaf Hering --- tools/libs/guest/xg_sr_common.h | 1 + tools/libs/guest/xg_sr_save.c | 28 +++++++++++----------------- 2 files changed, 12 insertions(+), 17 deletions(-) diff --git a/tools/libs/guest/xg_sr_common.h b/tools/libs/guest/xg_sr_common.h index e2a068bb7e..b054661897 100644 --- a/tools/libs/guest/xg_sr_common.h +++ b/tools/libs/guest/xg_sr_common.h @@ -248,6 +248,7 @@ struct xc_sr_context xen_pfn_t *types; int *errors; struct iovec *iov; + uint64_t *rec_pfns; unsigned int nr_batch_pfns; unsigned long *deferred_pages; unsigned long nr_deferred_pages; diff --git a/tools/libs/guest/xg_sr_save.c b/tools/libs/guest/xg_sr_save.c index 92eba456e6..7c417f4a2a 100644 --- a/tools/libs/guest/xg_sr_save.c +++ b/tools/libs/guest/xg_sr_save.c @@ -95,7 +95,6 @@ static int write_batch(struct xc_sr_context *ctx) unsigned int i, p, nr_pages = 0, nr_pages_mapped = 0; unsigned int nr_pfns = ctx->save.nr_batch_pfns; void *page, *orig_page; - uint64_t *rec_pfns = NULL; int iovcnt = 0; struct xc_sr_rec_page_data_header hdr = { 0 }; struct xc_sr_record rec = { @@ -202,22 +201,15 @@ static int write_batch(struct xc_sr_context *ctx) } } - rec_pfns = malloc(nr_pfns * sizeof(*rec_pfns)); - if ( !rec_pfns ) - { - ERROR("Unable to allocate %zu bytes of memory for page data pfn list", - nr_pfns * sizeof(*rec_pfns)); - goto err; - } - hdr.count = nr_pfns; rec.length = sizeof(hdr); - rec.length += nr_pfns * sizeof(*rec_pfns); + rec.length += nr_pfns * sizeof(*ctx->save.rec_pfns); rec.length += nr_pages * PAGE_SIZE; for ( i = 0; i < nr_pfns; ++i ) - rec_pfns[i] = ((uint64_t)(ctx->save.types[i]) << 32) | ctx->save.batch_pfns[i]; + ctx->save.rec_pfns[i] = ((uint64_t)(ctx->save.types[i]) << 32) | + ctx->save.batch_pfns[i]; ctx->save.iov[0].iov_base = &rec.type; ctx->save.iov[0].iov_len = sizeof(rec.type); @@ -228,12 +220,13 @@ static int write_batch(struct xc_sr_context *ctx) ctx->save.iov[2].iov_base = &hdr; ctx->save.iov[2].iov_len = sizeof(hdr); - ctx->save.iov[3].iov_base = rec_pfns; - ctx->save.iov[3].iov_len = nr_pfns * sizeof(*rec_pfns); + ctx->save.iov[3].iov_base = ctx->save.rec_pfns; + ctx->save.iov[3].iov_len = nr_pfns * sizeof(*ctx->save.rec_pfns); iovcnt = 4; ctx->save.pages_sent += nr_pages; - ctx->save.overhead_sent += sizeof(rec) + sizeof(hdr) + nr_pfns * sizeof(*rec_pfns); + ctx->save.overhead_sent += sizeof(rec) + sizeof(hdr) + + nr_pfns * sizeof(*ctx->save.rec_pfns); if ( nr_pages ) { @@ -260,7 +253,6 @@ static int write_batch(struct xc_sr_context *ctx) rc = ctx->save.nr_batch_pfns = 0; err: - free(rec_pfns); if ( guest_mapping ) xenforeignmemory_unmap(xch->fmem, guest_mapping, nr_pages_mapped); for ( i = 0; local_pages && i < nr_pfns; ++i ) @@ -844,11 +836,12 @@ static int setup(struct xc_sr_context *ctx) ctx->save.types = malloc(MAX_BATCH_SIZE * sizeof(*ctx->save.types)); ctx->save.errors = malloc(MAX_BATCH_SIZE * sizeof(*ctx->save.errors)); ctx->save.iov = malloc((4 + MAX_BATCH_SIZE) * sizeof(*ctx->save.iov)); + ctx->save.rec_pfns = malloc(MAX_BATCH_SIZE * sizeof(*ctx->save.rec_pfns)); ctx->save.deferred_pages = bitmap_alloc(ctx->save.p2m_size); if ( !ctx->save.batch_pfns || !ctx->save.mfns || !ctx->save.types || - !ctx->save.errors || !ctx->save.iov || !dirty_bitmap || - !ctx->save.deferred_pages ) + !ctx->save.errors || !ctx->save.iov || !ctx->save.rec_pfns || + !dirty_bitmap || !ctx->save.deferred_pages ) { ERROR("Unable to allocate memory for dirty bitmaps, batch pfns and" " deferred pages"); @@ -879,6 +872,7 @@ static void cleanup(struct xc_sr_context *ctx) xc_hypercall_buffer_free_pages(xch, dirty_bitmap, NRPAGES(bitmap_size(ctx->save.p2m_size))); free(ctx->save.deferred_pages); + free(ctx->save.rec_pfns); free(ctx->save.iov); free(ctx->save.errors); free(ctx->save.types); From patchwork Tue Jul 13 18:05:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12374709 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1D020C07E95 for ; Tue, 13 Jul 2021 18:07:01 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DB1EA61374 for ; Tue, 13 Jul 2021 18:07:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DB1EA61374 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.155554.287267 (Exim 4.92) (envelope-from ) id 1m3Mo1-0000eJ-VZ; Tue, 13 Jul 2021 18:06:53 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 155554.287267; Tue, 13 Jul 2021 18:06:53 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3Mo1-0000cb-Ic; Tue, 13 Jul 2021 18:06:53 +0000 Received: by outflank-mailman (input) for mailman id 155554; Tue, 13 Jul 2021 18:06:53 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3Mo0-0002kr-UC for xen-devel@lists.xenproject.org; Tue, 13 Jul 2021 18:06:52 +0000 Received: from mo4-p02-ob.smtp.rzone.de (unknown [85.215.255.84]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 06e2c55e-e405-11eb-8793-12813bfff9fa; Tue, 13 Jul 2021 18:06:21 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.28.1 AUTH) with ESMTPSA id 30791cx6DI6GtAt (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Tue, 13 Jul 2021 20:06:16 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 06e2c55e-e405-11eb-8793-12813bfff9fa DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1626199576; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=OMEAEb7NX4AU8uov75H5dJFOZP1BqztZJlpHGBMjQLg=; b=P2TG24K45DBPU/4N7u49xflxJX+B0lyYYFgOgIBOmGVMdlb0dVr84dZ9mWSph+fTyF mzJEQoShbw4m6PiJ4WwXpmUyL5rDVA4Js8B6BBnBiqGBVoC+eX8h64fQPiFnnzJOJYZD fzIPs+UpX0H1cHBM7qWA9xbB4WQW515+K7T8EGTica5RINupOdTnjkrbGfx4veOtYzmJ uv5BrYrOXfPQbilpjmrWzum/+jXMyyaLXvnjpM0XN26aqnWm3Ep6wpeTE1YvAv5nvgMy FHlIpgYkIUnbvYaA8a0qO8uhP/+2UONtGF6K6HO5cTmhoEax+iaRbrNkSx0R+tLBxy9x 0qAw== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAgthqlagRsstpH9XYEN5v9xf7dypFU9ge5p0Oi0V" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu , Juergen Gross Subject: [PATCH v20210713 12/31] tools: save: preallocate guest_data array Date: Tue, 13 Jul 2021 20:05:46 +0200 Message-Id: <20210713180605.12096-13-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210713180605.12096-1-olaf@aepfle.de> References: <20210713180605.12096-1-olaf@aepfle.de> MIME-Version: 1.0 Remove repeated allocation from migration loop. There will never be more than MAX_BATCH_SIZE pages to process in a batch. Allocate the space once. Because this was allocated with calloc: Adjust the loop to clear unused entries as needed. Signed-off-by: Olaf Hering --- tools/libs/guest/xg_sr_common.h | 1 + tools/libs/guest/xg_sr_save.c | 20 +++++++++++--------- 2 files changed, 12 insertions(+), 9 deletions(-) diff --git a/tools/libs/guest/xg_sr_common.h b/tools/libs/guest/xg_sr_common.h index b054661897..b90b35e564 100644 --- a/tools/libs/guest/xg_sr_common.h +++ b/tools/libs/guest/xg_sr_common.h @@ -249,6 +249,7 @@ struct xc_sr_context int *errors; struct iovec *iov; uint64_t *rec_pfns; + void **guest_data; unsigned int nr_batch_pfns; unsigned long *deferred_pages; unsigned long nr_deferred_pages; diff --git a/tools/libs/guest/xg_sr_save.c b/tools/libs/guest/xg_sr_save.c index 7c417f4a2a..0ab8b7205c 100644 --- a/tools/libs/guest/xg_sr_save.c +++ b/tools/libs/guest/xg_sr_save.c @@ -89,7 +89,6 @@ static int write_batch(struct xc_sr_context *ctx) { xc_interface *xch = ctx->xch; void *guest_mapping = NULL; - void **guest_data = NULL; void **local_pages = NULL; int rc = -1; unsigned int i, p, nr_pages = 0, nr_pages_mapped = 0; @@ -103,12 +102,10 @@ static int write_batch(struct xc_sr_context *ctx) assert(nr_pfns != 0); - /* Pointers to page data to send. Mapped gfns or local allocations. */ - guest_data = calloc(nr_pfns, sizeof(*guest_data)); /* Pointers to locally allocated pages. Need freeing. */ local_pages = calloc(nr_pfns, sizeof(*local_pages)); - if ( !guest_data || !local_pages ) + if ( !local_pages ) { ERROR("Unable to allocate arrays for a batch of %u pages", nr_pfns); @@ -165,7 +162,10 @@ static int write_batch(struct xc_sr_context *ctx) for ( i = 0, p = 0; i < nr_pfns; ++i ) { if ( !page_type_has_stream_data(ctx->save.types[i]) ) + { + ctx->save.guest_data[i] = NULL; continue; + } if ( ctx->save.errors[p] ) { @@ -183,6 +183,7 @@ static int write_batch(struct xc_sr_context *ctx) if ( rc ) { + ctx->save.guest_data[i] = NULL; if ( rc == -1 && errno == EAGAIN ) { set_bit(ctx->save.batch_pfns[i], ctx->save.deferred_pages); @@ -194,7 +195,7 @@ static int write_batch(struct xc_sr_context *ctx) goto err; } else - guest_data[i] = page; + ctx->save.guest_data[i] = page; rc = -1; ++p; @@ -232,9 +233,9 @@ static int write_batch(struct xc_sr_context *ctx) { for ( i = 0; i < nr_pfns; ++i ) { - if ( guest_data[i] ) + if ( ctx->save.guest_data[i] ) { - ctx->save.iov[iovcnt].iov_base = guest_data[i]; + ctx->save.iov[iovcnt].iov_base = ctx->save.guest_data[i]; ctx->save.iov[iovcnt].iov_len = PAGE_SIZE; iovcnt++; --nr_pages; @@ -258,7 +259,6 @@ static int write_batch(struct xc_sr_context *ctx) for ( i = 0; local_pages && i < nr_pfns; ++i ) free(local_pages[i]); free(local_pages); - free(guest_data); return rc; } @@ -837,11 +837,12 @@ static int setup(struct xc_sr_context *ctx) ctx->save.errors = malloc(MAX_BATCH_SIZE * sizeof(*ctx->save.errors)); ctx->save.iov = malloc((4 + MAX_BATCH_SIZE) * sizeof(*ctx->save.iov)); ctx->save.rec_pfns = malloc(MAX_BATCH_SIZE * sizeof(*ctx->save.rec_pfns)); + ctx->save.guest_data = malloc(MAX_BATCH_SIZE * sizeof(*ctx->save.guest_data)); ctx->save.deferred_pages = bitmap_alloc(ctx->save.p2m_size); if ( !ctx->save.batch_pfns || !ctx->save.mfns || !ctx->save.types || !ctx->save.errors || !ctx->save.iov || !ctx->save.rec_pfns || - !dirty_bitmap || !ctx->save.deferred_pages ) + !ctx->save.guest_data ||!dirty_bitmap || !ctx->save.deferred_pages ) { ERROR("Unable to allocate memory for dirty bitmaps, batch pfns and" " deferred pages"); @@ -872,6 +873,7 @@ static void cleanup(struct xc_sr_context *ctx) xc_hypercall_buffer_free_pages(xch, dirty_bitmap, NRPAGES(bitmap_size(ctx->save.p2m_size))); free(ctx->save.deferred_pages); + free(ctx->save.guest_data); free(ctx->save.rec_pfns); free(ctx->save.iov); free(ctx->save.errors); From patchwork Tue Jul 13 18:05:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12374699 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6994AC07E95 for ; Tue, 13 Jul 2021 18:06:50 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3485A61369 for ; Tue, 13 Jul 2021 18:06:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3485A61369 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.155546.287212 (Exim 4.92) (envelope-from ) id 1m3Mnp-0006KQ-O6; Tue, 13 Jul 2021 18:06:41 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 155546.287212; Tue, 13 Jul 2021 18:06:41 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3Mnp-0006JK-Hq; Tue, 13 Jul 2021 18:06:41 +0000 Received: by outflank-mailman (input) for mailman id 155546; Tue, 13 Jul 2021 18:06:39 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3Mnn-0002UL-Gf for xen-devel@lists.xenproject.org; Tue, 13 Jul 2021 18:06:39 +0000 Received: from mo4-p02-ob.smtp.rzone.de (unknown [81.169.146.168]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 2477b7d6-a2ac-427c-997e-ddef9a8e3457; Tue, 13 Jul 2021 18:06:23 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.28.1 AUTH) with ESMTPSA id 30791cx6DI6GtAu (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Tue, 13 Jul 2021 20:06:16 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 2477b7d6-a2ac-427c-997e-ddef9a8e3457 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1626199576; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=PTsQYvP6bBBv2G619w4kDBe3knppN9bIh+m3fSDyz7g=; b=labA6MsB39oPGMVWJ22VGeOWY5y7tTfBKBx4UIM2aUQZlpcy9RdUZznD48oXJr59r0 OyZu/zfPUIW0DdeKGmOZVUNP9eWyUIUtx1ZMgW4iZg4pgWkKeujB1EF3YS2NHRZHJUOM fxA62Cai41o4FMSSEugvI8eB1FHwvozRY5qEatmTqL9Ci3e16J6xuRD/jXClDQpPHDzT QrlPb3/2mUJrVV9fkFepPsMXn+dLRcZ9gnLru61HaTJQhiZsce8jYIOIHHM8LsYzq1nw QpNCFLjOLXFk84j/rkTUM/H0w9L5GehLvB+4H+m6qDK+RC67FCTfXLojplWANNmJtqwf j6bA== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAgthqlagRsstpH9XYEN5v9xf7dypFU9ge5p0Oi0V" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu , Juergen Gross Subject: [PATCH v20210713 13/31] tools: save: preallocate local_pages array Date: Tue, 13 Jul 2021 20:05:47 +0200 Message-Id: <20210713180605.12096-14-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210713180605.12096-1-olaf@aepfle.de> References: <20210713180605.12096-1-olaf@aepfle.de> MIME-Version: 1.0 Remove repeated allocation from migration loop. There will never be more than MAX_BATCH_SIZE pages to process in a batch. Allocate the space once. Adjust the code to use the unmodified src page in case of HVM. In case of PV the page may need to be normalised, use a private memory area for this purpose. Signed-off-by: Olaf Hering --- tools/libs/guest/xg_sr_common.h | 22 ++++++++++--------- tools/libs/guest/xg_sr_save.c | 26 ++++------------------ tools/libs/guest/xg_sr_save_x86_hvm.c | 5 +++-- tools/libs/guest/xg_sr_save_x86_pv.c | 31 ++++++++++++++++++--------- 4 files changed, 40 insertions(+), 44 deletions(-) diff --git a/tools/libs/guest/xg_sr_common.h b/tools/libs/guest/xg_sr_common.h index b90b35e564..34d4c15b22 100644 --- a/tools/libs/guest/xg_sr_common.h +++ b/tools/libs/guest/xg_sr_common.h @@ -33,16 +33,12 @@ struct xc_sr_save_ops * Optionally transform the contents of a page from being specific to the * sending environment, to being generic for the stream. * - * The page of data at the end of 'page' may be a read-only mapping of a - * running guest; it must not be modified. If no transformation is - * required, the callee should leave '*pages' untouched. + * The page of data '*src' may be a read-only mapping of a running guest; + * it must not be modified. If no transformation is required, the callee + * should leave '*src' untouched, and return it via '**ptr'. * - * If a transformation is required, the callee should allocate themselves - * a local page using malloc() and return it via '*page'. - * - * The caller shall free() '*page' in all cases. In the case that the - * callee encounters an error, it should *NOT* free() the memory it - * allocated for '*page'. + * If a transformation is required, the callee should provide the + * transformed page in a private buffer and return it via '**ptr'. * * It is valid to fail with EAGAIN if the transformation is not able to be * completed at this point. The page shall be retried later. @@ -50,7 +46,7 @@ struct xc_sr_save_ops * @returns 0 for success, -1 for failure, with errno appropriately set. */ int (*normalise_page)(struct xc_sr_context *ctx, xen_pfn_t type, - void **page); + void *src, unsigned int idx, void **ptr); /** * Set up local environment to save a domain. (Typically querying @@ -357,6 +353,12 @@ struct xc_sr_context union { + struct + { + /* Used by write_batch for modified pages. */ + void *normalised_pages; + } save; + struct { /* State machine for the order of received records. */ diff --git a/tools/libs/guest/xg_sr_save.c b/tools/libs/guest/xg_sr_save.c index 0ab8b7205c..c74c8d9348 100644 --- a/tools/libs/guest/xg_sr_save.c +++ b/tools/libs/guest/xg_sr_save.c @@ -89,11 +89,10 @@ static int write_batch(struct xc_sr_context *ctx) { xc_interface *xch = ctx->xch; void *guest_mapping = NULL; - void **local_pages = NULL; int rc = -1; unsigned int i, p, nr_pages = 0, nr_pages_mapped = 0; unsigned int nr_pfns = ctx->save.nr_batch_pfns; - void *page, *orig_page; + void *src; int iovcnt = 0; struct xc_sr_rec_page_data_header hdr = { 0 }; struct xc_sr_record rec = { @@ -102,16 +101,6 @@ static int write_batch(struct xc_sr_context *ctx) assert(nr_pfns != 0); - /* Pointers to locally allocated pages. Need freeing. */ - local_pages = calloc(nr_pfns, sizeof(*local_pages)); - - if ( !local_pages ) - { - ERROR("Unable to allocate arrays for a batch of %u pages", - nr_pfns); - goto err; - } - for ( i = 0; i < nr_pfns; ++i ) { ctx->save.types[i] = ctx->save.mfns[i] = ctx->save.ops.pfn_to_gfn(ctx, @@ -175,11 +164,9 @@ static int write_batch(struct xc_sr_context *ctx) goto err; } - orig_page = page = guest_mapping + (p * PAGE_SIZE); - rc = ctx->save.ops.normalise_page(ctx, ctx->save.types[i], &page); - - if ( orig_page != page ) - local_pages[i] = page; + src = guest_mapping + (p * PAGE_SIZE); + rc = ctx->save.ops.normalise_page(ctx, ctx->save.types[i], src, i, + &ctx->save.guest_data[i]); if ( rc ) { @@ -194,8 +181,6 @@ static int write_batch(struct xc_sr_context *ctx) else goto err; } - else - ctx->save.guest_data[i] = page; rc = -1; ++p; @@ -256,9 +241,6 @@ static int write_batch(struct xc_sr_context *ctx) err: if ( guest_mapping ) xenforeignmemory_unmap(xch->fmem, guest_mapping, nr_pages_mapped); - for ( i = 0; local_pages && i < nr_pfns; ++i ) - free(local_pages[i]); - free(local_pages); return rc; } diff --git a/tools/libs/guest/xg_sr_save_x86_hvm.c b/tools/libs/guest/xg_sr_save_x86_hvm.c index 1634a7bc43..11232b9f1d 100644 --- a/tools/libs/guest/xg_sr_save_x86_hvm.c +++ b/tools/libs/guest/xg_sr_save_x86_hvm.c @@ -129,9 +129,10 @@ static xen_pfn_t x86_hvm_pfn_to_gfn(const struct xc_sr_context *ctx, return pfn; } -static int x86_hvm_normalise_page(struct xc_sr_context *ctx, - xen_pfn_t type, void **page) +static int x86_hvm_normalise_page(struct xc_sr_context *ctx, xen_pfn_t type, + void *src, unsigned int idx, void **ptr) { + *ptr = src; return 0; } diff --git a/tools/libs/guest/xg_sr_save_x86_pv.c b/tools/libs/guest/xg_sr_save_x86_pv.c index 4964f1f7b8..defc044001 100644 --- a/tools/libs/guest/xg_sr_save_x86_pv.c +++ b/tools/libs/guest/xg_sr_save_x86_pv.c @@ -999,29 +999,31 @@ static xen_pfn_t x86_pv_pfn_to_gfn(const struct xc_sr_context *ctx, * save_ops function. Performs pagetable normalisation on appropriate pages. */ static int x86_pv_normalise_page(struct xc_sr_context *ctx, xen_pfn_t type, - void **page) + void *src, unsigned int idx, void **ptr) { xc_interface *xch = ctx->xch; - void *local_page; + void *dst; int rc; type &= XEN_DOMCTL_PFINFO_LTABTYPE_MASK; if ( type < XEN_DOMCTL_PFINFO_L1TAB || type > XEN_DOMCTL_PFINFO_L4TAB ) + { + *ptr = src; return 0; + } - local_page = malloc(PAGE_SIZE); - if ( !local_page ) + if ( idx >= MAX_BATCH_SIZE ) { - ERROR("Unable to allocate scratch page"); - rc = -1; - goto out; + ERROR("idx %u out of range", idx); + errno = ERANGE; + return -1; } - rc = normalise_pagetable(ctx, *page, local_page, type); - *page = local_page; + dst = ctx->x86.pv.save.normalised_pages + (idx * PAGE_SIZE); + rc = normalise_pagetable(ctx, src, dst, type); + *ptr = dst; - out: return rc; } @@ -1031,8 +1033,16 @@ static int x86_pv_normalise_page(struct xc_sr_context *ctx, xen_pfn_t type, */ static int x86_pv_setup(struct xc_sr_context *ctx) { + xc_interface *xch = ctx->xch; int rc; + ctx->x86.pv.save.normalised_pages = malloc(MAX_BATCH_SIZE * PAGE_SIZE); + if ( !ctx->x86.pv.save.normalised_pages ) + { + PERROR("Failed to allocate normalised_pages"); + return -1; + } + rc = x86_pv_domain_info(ctx); if ( rc ) return rc; @@ -1118,6 +1128,7 @@ static int x86_pv_check_vm_state(struct xc_sr_context *ctx) static int x86_pv_cleanup(struct xc_sr_context *ctx) { + free(ctx->x86.pv.save.normalised_pages); free(ctx->x86.pv.p2m_pfns); if ( ctx->x86.pv.p2m ) From patchwork Tue Jul 13 18:05:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12374717 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 31638C07E95 for ; Tue, 13 Jul 2021 18:07:12 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id F33A761374 for ; Tue, 13 Jul 2021 18:07:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F33A761374 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.155563.287311 (Exim 4.92) (envelope-from ) id 1m3MoC-00039r-FC; Tue, 13 Jul 2021 18:07:04 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 155563.287311; Tue, 13 Jul 2021 18:07:04 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3MoC-00039O-6v; Tue, 13 Jul 2021 18:07:04 +0000 Received: by outflank-mailman (input) for mailman id 155563; Tue, 13 Jul 2021 18:07:03 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3MoA-0002kr-UQ for xen-devel@lists.xenproject.org; Tue, 13 Jul 2021 18:07:02 +0000 Received: from mo4-p02-ob.smtp.rzone.de (unknown [81.169.146.170]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 08701ffa-e405-11eb-8793-12813bfff9fa; Tue, 13 Jul 2021 18:06:24 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.28.1 AUTH) with ESMTPSA id 30791cx6DI6HtAw (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Tue, 13 Jul 2021 20:06:17 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 08701ffa-e405-11eb-8793-12813bfff9fa DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1626199577; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=SHDAXXjQ4k5wqGqgpZ5ML0uGXKdasaiKGHiNj6axBrA=; b=USdFIKxpv0yDNCsW+ZdmUnwSYK+mlD+t9QhFMzLQMGv38mnlBbQ4pE6ErpP/pUguwf jbniADxY0RwY0LeHmvos3wlJTmz8oTXwpsb/vkNQFSQtnrSrxoQrSpNNL7OG+feIJbwv y3DzpcZajCgmFPnQiCW+SZvOC1aAPeSTO0okqDdgT7iWIlRneiDpGiGuq5V50PDqlaoo dH79WEABvneFg7XnwSn37PMKR3l/i4QS8u02HQXgjBeARX3qzQYVpUl4N6Z7IDRiK3wP sfiDbeOp2Mplfo+MzMNKIRkE+EeYosRGrbOyX3z/iald+672r+x5xRJyGg3rWcXfIXs6 6nig== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAgthqlagRsstpH9XYEN5v9xf7dypFU9ge5p0Oi0V" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu , Juergen Gross Subject: [PATCH v20210713 14/31] tools: restore: preallocate pfns array Date: Tue, 13 Jul 2021 20:05:48 +0200 Message-Id: <20210713180605.12096-15-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210713180605.12096-1-olaf@aepfle.de> References: <20210713180605.12096-1-olaf@aepfle.de> MIME-Version: 1.0 Remove repeated allocation from migration loop. There will never be more than MAX_BATCH_SIZE pages to process in an incoming batch. Allocate the space once. Adjust the verification for page count. It must be at least one page, but not more than MAX_BATCH_SIZE. Signed-off-by: Olaf Hering --- tools/libs/guest/xg_sr_common.h | 1 + tools/libs/guest/xg_sr_restore.c | 23 +++++++++++++++-------- 2 files changed, 16 insertions(+), 8 deletions(-) diff --git a/tools/libs/guest/xg_sr_common.h b/tools/libs/guest/xg_sr_common.h index 34d4c15b22..d32b4c46f3 100644 --- a/tools/libs/guest/xg_sr_common.h +++ b/tools/libs/guest/xg_sr_common.h @@ -256,6 +256,7 @@ struct xc_sr_context { struct xc_sr_restore_ops ops; struct restore_callbacks *callbacks; + xen_pfn_t *pfns; int send_back_fd; unsigned long p2m_size; diff --git a/tools/libs/guest/xg_sr_restore.c b/tools/libs/guest/xg_sr_restore.c index aa4113d7f6..e812f65f99 100644 --- a/tools/libs/guest/xg_sr_restore.c +++ b/tools/libs/guest/xg_sr_restore.c @@ -314,7 +314,7 @@ static int handle_page_data(struct xc_sr_context *ctx, struct xc_sr_record *rec) unsigned int i, pages_of_data = 0; int rc = -1; - xen_pfn_t *pfns = NULL, pfn; + xen_pfn_t pfn; uint32_t *types = NULL, type; /* @@ -349,9 +349,9 @@ static int handle_page_data(struct xc_sr_context *ctx, struct xc_sr_record *rec) goto err; } - if ( pages->count < 1 ) + if ( !pages->count || pages->count > MAX_BATCH_SIZE ) { - ERROR("Expected at least 1 pfn in PAGE_DATA record"); + ERROR("Unexpected pfn count %u in PAGE_DATA record", pages->count); goto err; } @@ -362,9 +362,8 @@ static int handle_page_data(struct xc_sr_context *ctx, struct xc_sr_record *rec) goto err; } - pfns = malloc(pages->count * sizeof(*pfns)); types = malloc(pages->count * sizeof(*types)); - if ( !pfns || !types ) + if ( !types ) { ERROR("Unable to allocate enough memory for %u pfns", pages->count); @@ -393,7 +392,7 @@ static int handle_page_data(struct xc_sr_context *ctx, struct xc_sr_record *rec) * have a page worth of data in the record. */ pages_of_data++; - pfns[i] = pfn; + ctx->restore.pfns[i] = pfn; types[i] = type; } @@ -407,11 +406,10 @@ static int handle_page_data(struct xc_sr_context *ctx, struct xc_sr_record *rec) goto err; } - rc = process_page_data(ctx, pages->count, pfns, types, + rc = process_page_data(ctx, pages->count, ctx->restore.pfns, types, &pages->pfn[pages->count]); err: free(types); - free(pfns); return rc; } @@ -728,6 +726,14 @@ static int setup(struct xc_sr_context *ctx) goto err; } + ctx->restore.pfns = malloc(MAX_BATCH_SIZE * sizeof(*ctx->restore.pfns)); + if ( !ctx->restore.pfns ) + { + ERROR("Unable to allocate memory"); + rc = -1; + goto err; + } + ctx->restore.buffered_records = malloc( DEFAULT_BUF_RECORDS * sizeof(struct xc_sr_record)); if ( !ctx->restore.buffered_records ) @@ -758,6 +764,7 @@ static void cleanup(struct xc_sr_context *ctx) free(ctx->restore.buffered_records); free(ctx->restore.populated_pfns); + free(ctx->restore.pfns); if ( ctx->restore.ops.cleanup(ctx) ) PERROR("Failed to clean up"); From patchwork Tue Jul 13 18:05:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12374713 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 28433C07E95 for ; Tue, 13 Jul 2021 18:07:07 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id EE2E861369 for ; Tue, 13 Jul 2021 18:07:06 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EE2E861369 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.155557.287288 (Exim 4.92) (envelope-from ) id 1m3Mo7-0001v7-Bn; Tue, 13 Jul 2021 18:06:59 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 155557.287288; Tue, 13 Jul 2021 18:06:59 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3Mo7-0001tb-6m; Tue, 13 Jul 2021 18:06:59 +0000 Received: by outflank-mailman (input) for mailman id 155557; Tue, 13 Jul 2021 18:06:58 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3Mo5-0002kr-UL for xen-devel@lists.xenproject.org; Tue, 13 Jul 2021 18:06:57 +0000 Received: from mo4-p02-ob.smtp.rzone.de (unknown [81.169.146.171]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 0840d682-e405-11eb-8793-12813bfff9fa; Tue, 13 Jul 2021 18:06:23 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.28.1 AUTH) with ESMTPSA id 30791cx6DI6HtAx (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Tue, 13 Jul 2021 20:06:17 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 0840d682-e405-11eb-8793-12813bfff9fa DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1626199577; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=EqbkUFGoJ5kAajdvgf66aZTTmFJgr8buS42CRf7IZCM=; b=s3tvuDYTIJBgYEAhcTEYPn+hd/6Lox5NyjwWlrhxIQPcX0AMX321FES4gI/cm16cVH aKEO92boPkt+pmCfzl39cbgZXfD4grPxXSdLlTPlV56wOusEU5XVI73C9BGqjfP70WMJ 9T1+qM3PQif9Hy6kCjBrihoLjyDQs2TWOEk/YSDwvycL+C0e7b7C2lRSel7bAmGdJIZT FkZRzljpZsOLxWGQRrsUcqnl+tQpJOAcgkFFiptA4OkORj7SELNN7yaiRrSE2QCsUghD cgF1wDDi4y3zbKAwV6sEuF9ydc6xMbcceOsOH+lQ2ama6MLT2Z8wAsK7CD3jwb0uZowD I2iQ== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAgthqlagRsstpH9XYEN5v9xf7dypFU9ge5p0Oi0V" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu , Juergen Gross Subject: [PATCH v20210713 15/31] tools: restore: preallocate types array Date: Tue, 13 Jul 2021 20:05:49 +0200 Message-Id: <20210713180605.12096-16-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210713180605.12096-1-olaf@aepfle.de> References: <20210713180605.12096-1-olaf@aepfle.de> MIME-Version: 1.0 Remove repeated allocation from migration loop. There will never be more than MAX_BATCH_SIZE pages to process in an incoming batch. Allocate the space once. Signed-off-by: Olaf Hering --- tools/libs/guest/xg_sr_common.h | 1 + tools/libs/guest/xg_sr_restore.c | 22 +++++++--------------- 2 files changed, 8 insertions(+), 15 deletions(-) diff --git a/tools/libs/guest/xg_sr_common.h b/tools/libs/guest/xg_sr_common.h index d32b4c46f3..bd2027ba5d 100644 --- a/tools/libs/guest/xg_sr_common.h +++ b/tools/libs/guest/xg_sr_common.h @@ -257,6 +257,7 @@ struct xc_sr_context struct xc_sr_restore_ops ops; struct restore_callbacks *callbacks; xen_pfn_t *pfns; + uint32_t *types; int send_back_fd; unsigned long p2m_size; diff --git a/tools/libs/guest/xg_sr_restore.c b/tools/libs/guest/xg_sr_restore.c index e812f65f99..bbb55b6844 100644 --- a/tools/libs/guest/xg_sr_restore.c +++ b/tools/libs/guest/xg_sr_restore.c @@ -315,7 +315,7 @@ static int handle_page_data(struct xc_sr_context *ctx, struct xc_sr_record *rec) int rc = -1; xen_pfn_t pfn; - uint32_t *types = NULL, type; + uint32_t type; /* * v2 compatibility only exists for x86 streams. This is a bit of a @@ -362,14 +362,6 @@ static int handle_page_data(struct xc_sr_context *ctx, struct xc_sr_record *rec) goto err; } - types = malloc(pages->count * sizeof(*types)); - if ( !types ) - { - ERROR("Unable to allocate enough memory for %u pfns", - pages->count); - goto err; - } - for ( i = 0; i < pages->count; ++i ) { pfn = pages->pfn[i] & PAGE_DATA_PFN_MASK; @@ -393,7 +385,7 @@ static int handle_page_data(struct xc_sr_context *ctx, struct xc_sr_record *rec) pages_of_data++; ctx->restore.pfns[i] = pfn; - types[i] = type; + ctx->restore.types[i] = type; } if ( rec->length != (sizeof(*pages) + @@ -406,11 +398,9 @@ static int handle_page_data(struct xc_sr_context *ctx, struct xc_sr_record *rec) goto err; } - rc = process_page_data(ctx, pages->count, ctx->restore.pfns, types, - &pages->pfn[pages->count]); + rc = process_page_data(ctx, pages->count, ctx->restore.pfns, + ctx->restore.types, &pages->pfn[pages->count]); err: - free(types); - return rc; } @@ -727,7 +717,8 @@ static int setup(struct xc_sr_context *ctx) } ctx->restore.pfns = malloc(MAX_BATCH_SIZE * sizeof(*ctx->restore.pfns)); - if ( !ctx->restore.pfns ) + ctx->restore.types = malloc(MAX_BATCH_SIZE * sizeof(*ctx->restore.types)); + if ( !ctx->restore.pfns || !ctx->restore.types ) { ERROR("Unable to allocate memory"); rc = -1; @@ -764,6 +755,7 @@ static void cleanup(struct xc_sr_context *ctx) free(ctx->restore.buffered_records); free(ctx->restore.populated_pfns); + free(ctx->restore.types); free(ctx->restore.pfns); if ( ctx->restore.ops.cleanup(ctx) ) From patchwork Tue Jul 13 18:05:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12374703 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 814A1C07E95 for ; Tue, 13 Jul 2021 18:06:55 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 543D161369 for ; Tue, 13 Jul 2021 18:06:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 543D161369 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.155550.287234 (Exim 4.92) (envelope-from ) id 1m3Mnu-0007QK-VD; Tue, 13 Jul 2021 18:06:46 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 155550.287234; Tue, 13 Jul 2021 18:06:46 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3Mnu-0007PD-MM; Tue, 13 Jul 2021 18:06:46 +0000 Received: by outflank-mailman (input) for mailman id 155550; Tue, 13 Jul 2021 18:06:44 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3Mns-0002UL-Gq for xen-devel@lists.xenproject.org; Tue, 13 Jul 2021 18:06:44 +0000 Received: from mo4-p02-ob.smtp.rzone.de (unknown [81.169.146.170]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id c567cd93-a813-4192-b89f-81bbb939ef08; Tue, 13 Jul 2021 18:06:24 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.28.1 AUTH) with ESMTPSA id 30791cx6DI6HtAy (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Tue, 13 Jul 2021 20:06:17 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: c567cd93-a813-4192-b89f-81bbb939ef08 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1626199578; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=YRVl0CnaOM9g6RFoif4tiboPYUw5yevknXWVbW/ME0k=; b=Zs0Xodk75jN9h8CKY05mz+GWB0Wz5eCjvywwarFzGRPrEFHLwAHucVDm8au4Ux6H66 GtTv3WTWHmD6XQYoLcwZZ+W9ijQz/8VpWCoNBjpKy1l+MOiWvdpHruGmTbOC7UrD697J lXG3wzeply4r3vA6GZWact/xyfIx++IO0LgqWmcpoXUd4QsjjOxMuXS1o2aa+kjwkAm6 G3ZsvgM4Q9zIlbyUtyEpVID6ePweIT0UOaxldvdb9kT9CuYZbKxdiu143sYRMjssmgJe e6OzjVAhuGbh1iZzr8BPPBw4ZRdnU240FHtjHybfjY28FwtgT+rVXZCVAwfJr1w3y331 6M8g== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAgthqlagRsstpH9XYEN5v9xf7dypFU9ge5p0Oi0V" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu , Juergen Gross Subject: [PATCH v20210713 16/31] tools: restore: preallocate mfns array Date: Tue, 13 Jul 2021 20:05:50 +0200 Message-Id: <20210713180605.12096-17-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210713180605.12096-1-olaf@aepfle.de> References: <20210713180605.12096-1-olaf@aepfle.de> MIME-Version: 1.0 Remove repeated allocation from migration loop. There will never be more than MAX_BATCH_SIZE pages to process in an incoming batch. Allocate the space once. Signed-off-by: Olaf Hering --- tools/libs/guest/xg_sr_common.h | 1 + tools/libs/guest/xg_sr_restore.c | 16 ++++++++-------- 2 files changed, 9 insertions(+), 8 deletions(-) diff --git a/tools/libs/guest/xg_sr_common.h b/tools/libs/guest/xg_sr_common.h index bd2027ba5d..11e45dbdc5 100644 --- a/tools/libs/guest/xg_sr_common.h +++ b/tools/libs/guest/xg_sr_common.h @@ -258,6 +258,7 @@ struct xc_sr_context struct restore_callbacks *callbacks; xen_pfn_t *pfns; uint32_t *types; + xen_pfn_t *mfns; int send_back_fd; unsigned long p2m_size; diff --git a/tools/libs/guest/xg_sr_restore.c b/tools/libs/guest/xg_sr_restore.c index bbb55b6844..24cd8c6a07 100644 --- a/tools/libs/guest/xg_sr_restore.c +++ b/tools/libs/guest/xg_sr_restore.c @@ -204,7 +204,6 @@ static int process_page_data(struct xc_sr_context *ctx, unsigned int count, xen_pfn_t *pfns, uint32_t *types, void *page_data) { xc_interface *xch = ctx->xch; - xen_pfn_t *mfns = malloc(count * sizeof(*mfns)); int *map_errs = malloc(count * sizeof(*map_errs)); int rc; void *mapping = NULL, *guest_page = NULL; @@ -212,11 +211,11 @@ static int process_page_data(struct xc_sr_context *ctx, unsigned int count, j, /* j indexes the subset of pfns we decide to map. */ nr_pages = 0; - if ( !mfns || !map_errs ) + if ( !map_errs ) { rc = -1; ERROR("Failed to allocate %zu bytes to process page data", - count * (sizeof(*mfns) + sizeof(*map_errs))); + count * sizeof(*map_errs)); goto err; } @@ -232,7 +231,7 @@ static int process_page_data(struct xc_sr_context *ctx, unsigned int count, ctx->restore.ops.set_page_type(ctx, pfns[i], types[i]); if ( page_type_has_stream_data(types[i]) ) - mfns[nr_pages++] = ctx->restore.ops.pfn_to_gfn(ctx, pfns[i]); + ctx->restore.mfns[nr_pages++] = ctx->restore.ops.pfn_to_gfn(ctx, pfns[i]); } /* Nothing to do? */ @@ -241,7 +240,7 @@ static int process_page_data(struct xc_sr_context *ctx, unsigned int count, mapping = guest_page = xenforeignmemory_map( xch->fmem, ctx->domid, PROT_READ | PROT_WRITE, - nr_pages, mfns, map_errs); + nr_pages, ctx->restore.mfns, map_errs); if ( !mapping ) { rc = -1; @@ -259,7 +258,7 @@ static int process_page_data(struct xc_sr_context *ctx, unsigned int count, { rc = -1; ERROR("Mapping pfn %#"PRIpfn" (mfn %#"PRIpfn", type %#"PRIx32") failed with %d", - pfns[i], mfns[j], types[i], map_errs[j]); + pfns[i], ctx->restore.mfns[j], types[i], map_errs[j]); goto err; } @@ -298,7 +297,6 @@ static int process_page_data(struct xc_sr_context *ctx, unsigned int count, xenforeignmemory_unmap(xch->fmem, mapping, nr_pages); free(map_errs); - free(mfns); return rc; } @@ -718,7 +716,8 @@ static int setup(struct xc_sr_context *ctx) ctx->restore.pfns = malloc(MAX_BATCH_SIZE * sizeof(*ctx->restore.pfns)); ctx->restore.types = malloc(MAX_BATCH_SIZE * sizeof(*ctx->restore.types)); - if ( !ctx->restore.pfns || !ctx->restore.types ) + ctx->restore.mfns = malloc(MAX_BATCH_SIZE * sizeof(*ctx->restore.mfns)); + if ( !ctx->restore.pfns || !ctx->restore.types || !ctx->restore.mfns ) { ERROR("Unable to allocate memory"); rc = -1; @@ -755,6 +754,7 @@ static void cleanup(struct xc_sr_context *ctx) free(ctx->restore.buffered_records); free(ctx->restore.populated_pfns); + free(ctx->restore.mfns); free(ctx->restore.types); free(ctx->restore.pfns); From patchwork Tue Jul 13 18:05:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12374707 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 22BA0C07E96 for ; Tue, 13 Jul 2021 18:06:58 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E8DCE61369 for ; Tue, 13 Jul 2021 18:06:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E8DCE61369 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.155552.287254 (Exim 4.92) (envelope-from ) id 1m3Mnz-0008Nu-6N; Tue, 13 Jul 2021 18:06:51 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 155552.287254; Tue, 13 Jul 2021 18:06:51 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3Mny-0008Ln-QF; Tue, 13 Jul 2021 18:06:50 +0000 Received: by outflank-mailman (input) for mailman id 155552; Tue, 13 Jul 2021 18:06:49 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3Mnx-0002UL-Gz for xen-devel@lists.xenproject.org; Tue, 13 Jul 2021 18:06:49 +0000 Received: from mo4-p02-ob.smtp.rzone.de (unknown [81.169.146.171]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id a7527314-1255-4f65-8912-cbbd84ac5443; Tue, 13 Jul 2021 18:06:24 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.28.1 AUTH) with ESMTPSA id 30791cx6DI6ItAz (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Tue, 13 Jul 2021 20:06:18 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a7527314-1255-4f65-8912-cbbd84ac5443 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1626199578; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=mxNpwWOhZuO/yAfsj1BhbOC7Eeea1lW7O20Bo/SSJmE=; b=HbNTpUkaCGH7smmG0mRQLhqgGG/1A9vgPoaBIZvvS0ju1bTcHhjH8bJgkThGS+5kj8 Vw7H9McJFKbB9GEXnNxveKT5SETAMzjYr49v6Kc514vP/JsCqEs0MwnmQ3pRbopO1w1V 66eiIFywf3sEEsjffgaqezJ8UAfuuOSDF3/FmW/O29dAe0p2e1fbkAUB88eFaQBneVmd O8X3VLh2RlkPObAi+kI7ivGHR0A0Sv08BX/gpSMLv3M0W2/X1VpC7JwfVHu4ohGRlsqX s9obyuUsrbhve2kMKU4nQ8owzFf/SAkXGWv6HdtumjWT4/HVHVnCXS7O3c9R1QMBP5W2 0IBQ== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAgthqlagRsstpH9XYEN5v9xf7dypFU9ge5p0Oi0V" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu , Juergen Gross Subject: [PATCH v20210713 17/31] tools: restore: preallocate map_errs array Date: Tue, 13 Jul 2021 20:05:51 +0200 Message-Id: <20210713180605.12096-18-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210713180605.12096-1-olaf@aepfle.de> References: <20210713180605.12096-1-olaf@aepfle.de> MIME-Version: 1.0 Remove repeated allocation from migration loop. There will never be more than MAX_BATCH_SIZE pages to process in an incoming batch. Allocate the space once. Signed-off-by: Olaf Hering --- tools/libs/guest/xg_sr_common.h | 1 + tools/libs/guest/xg_sr_restore.c | 22 +++++++--------------- 2 files changed, 8 insertions(+), 15 deletions(-) diff --git a/tools/libs/guest/xg_sr_common.h b/tools/libs/guest/xg_sr_common.h index 11e45dbdc5..336be98c93 100644 --- a/tools/libs/guest/xg_sr_common.h +++ b/tools/libs/guest/xg_sr_common.h @@ -259,6 +259,7 @@ struct xc_sr_context xen_pfn_t *pfns; uint32_t *types; xen_pfn_t *mfns; + int *map_errs; int send_back_fd; unsigned long p2m_size; diff --git a/tools/libs/guest/xg_sr_restore.c b/tools/libs/guest/xg_sr_restore.c index 24cd8c6a07..866a7c86cd 100644 --- a/tools/libs/guest/xg_sr_restore.c +++ b/tools/libs/guest/xg_sr_restore.c @@ -204,21 +204,12 @@ static int process_page_data(struct xc_sr_context *ctx, unsigned int count, xen_pfn_t *pfns, uint32_t *types, void *page_data) { xc_interface *xch = ctx->xch; - int *map_errs = malloc(count * sizeof(*map_errs)); int rc; void *mapping = NULL, *guest_page = NULL; unsigned int i, /* i indexes the pfns from the record. */ j, /* j indexes the subset of pfns we decide to map. */ nr_pages = 0; - if ( !map_errs ) - { - rc = -1; - ERROR("Failed to allocate %zu bytes to process page data", - count * sizeof(*map_errs)); - goto err; - } - rc = populate_pfns(ctx, count, pfns, types); if ( rc ) { @@ -240,7 +231,7 @@ static int process_page_data(struct xc_sr_context *ctx, unsigned int count, mapping = guest_page = xenforeignmemory_map( xch->fmem, ctx->domid, PROT_READ | PROT_WRITE, - nr_pages, ctx->restore.mfns, map_errs); + nr_pages, ctx->restore.mfns, ctx->restore.map_errs); if ( !mapping ) { rc = -1; @@ -254,11 +245,11 @@ static int process_page_data(struct xc_sr_context *ctx, unsigned int count, if ( !page_type_has_stream_data(types[i]) ) continue; - if ( map_errs[j] ) + if ( ctx->restore.map_errs[j] ) { rc = -1; ERROR("Mapping pfn %#"PRIpfn" (mfn %#"PRIpfn", type %#"PRIx32") failed with %d", - pfns[i], ctx->restore.mfns[j], types[i], map_errs[j]); + pfns[i], ctx->restore.mfns[j], types[i], ctx->restore.map_errs[j]); goto err; } @@ -296,8 +287,6 @@ static int process_page_data(struct xc_sr_context *ctx, unsigned int count, if ( mapping ) xenforeignmemory_unmap(xch->fmem, mapping, nr_pages); - free(map_errs); - return rc; } @@ -717,7 +706,9 @@ static int setup(struct xc_sr_context *ctx) ctx->restore.pfns = malloc(MAX_BATCH_SIZE * sizeof(*ctx->restore.pfns)); ctx->restore.types = malloc(MAX_BATCH_SIZE * sizeof(*ctx->restore.types)); ctx->restore.mfns = malloc(MAX_BATCH_SIZE * sizeof(*ctx->restore.mfns)); - if ( !ctx->restore.pfns || !ctx->restore.types || !ctx->restore.mfns ) + ctx->restore.map_errs = malloc(MAX_BATCH_SIZE * sizeof(*ctx->restore.map_errs)); + if ( !ctx->restore.pfns || !ctx->restore.types || !ctx->restore.mfns || + !ctx->restore.map_errs ) { ERROR("Unable to allocate memory"); rc = -1; @@ -754,6 +745,7 @@ static void cleanup(struct xc_sr_context *ctx) free(ctx->restore.buffered_records); free(ctx->restore.populated_pfns); + free(ctx->restore.map_errs); free(ctx->restore.mfns); free(ctx->restore.types); free(ctx->restore.pfns); From patchwork Tue Jul 13 18:05:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12374733 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B74ADC07E95 for ; Tue, 13 Jul 2021 18:17:20 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 782A86044F for ; Tue, 13 Jul 2021 18:17:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 782A86044F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.155670.287417 (Exim 4.92) (envelope-from ) id 1m3Mxz-0003kz-Vp; Tue, 13 Jul 2021 18:17:11 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 155670.287417; Tue, 13 Jul 2021 18:17:11 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3Mxz-0003kF-Rf; Tue, 13 Jul 2021 18:17:11 +0000 Received: by outflank-mailman (input) for mailman id 155670; Tue, 13 Jul 2021 18:17:10 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3MoF-0002kr-Ud for xen-devel@lists.xenproject.org; Tue, 13 Jul 2021 18:07:07 +0000 Received: from mo4-p03-ob.smtp.rzone.de (unknown [81.169.146.174]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 08701ffb-e405-11eb-8793-12813bfff9fa; Tue, 13 Jul 2021 18:06:24 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.28.1 AUTH) with ESMTPSA id 30791cx6DI6ItB0 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Tue, 13 Jul 2021 20:06:18 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 08701ffb-e405-11eb-8793-12813bfff9fa DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1626199579; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=9FxM3Bx9wXja4XqBD1sGOusWUB3Xe2+i7IOocJp1ap0=; b=pmlv471FKO6AMjwr8Z6WkjOjfeDTfhS4g1PsNX7OPQtPlshRqiH1D8t1MyyDwoQY28 KvOnByp+tMenERKqJ44CYafWgoXdqW+quP8mocvhZcsZmcM5vo7wj6xa48FdiuBKMD2k svubuN12GRx9hb6T0FYR1RyYj0t9FThxcE8ujtpcF+LMoOL7kyhr22c/25KS2UkmG0C3 ihAdX79HLuAIQ3vrzcEWEtTb/H4lEN4AXnjX7PMk5dRFgimqOzTjo2L5TnDh7verjJkE /3+jRvcwtro77LyVxMS1amPehmuTuM0nRIgltngD/ivuO651cAOq2C8J8fRiNsjmDWOT W+9g== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAgthqlagRsstpH9XYEN5v9xf7dypFU9ge5p0Oi0V" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu , Juergen Gross Subject: [PATCH v20210713 18/31] tools: restore: preallocate populate_pfns pfns array Date: Tue, 13 Jul 2021 20:05:52 +0200 Message-Id: <20210713180605.12096-19-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210713180605.12096-1-olaf@aepfle.de> References: <20210713180605.12096-1-olaf@aepfle.de> MIME-Version: 1.0 Remove repeated allocation from migration loop. There will never be more than MAX_BATCH_SIZE pages to process in an incoming batch. Allocate the space once. Use some prefix to avoid conflict with an array used in handle_page_data. Signed-off-by: Olaf Hering --- tools/libs/guest/xg_sr_common.h | 1 + tools/libs/guest/xg_sr_restore.c | 14 +++++++------- 2 files changed, 8 insertions(+), 7 deletions(-) diff --git a/tools/libs/guest/xg_sr_common.h b/tools/libs/guest/xg_sr_common.h index 336be98c93..ca44e59382 100644 --- a/tools/libs/guest/xg_sr_common.h +++ b/tools/libs/guest/xg_sr_common.h @@ -260,6 +260,7 @@ struct xc_sr_context uint32_t *types; xen_pfn_t *mfns; int *map_errs; + xen_pfn_t *pp_pfns; int send_back_fd; unsigned long p2m_size; diff --git a/tools/libs/guest/xg_sr_restore.c b/tools/libs/guest/xg_sr_restore.c index 866a7c86cd..caac4bb5fa 100644 --- a/tools/libs/guest/xg_sr_restore.c +++ b/tools/libs/guest/xg_sr_restore.c @@ -138,12 +138,11 @@ int populate_pfns(struct xc_sr_context *ctx, unsigned int count, const xen_pfn_t *original_pfns, const uint32_t *types) { xc_interface *xch = ctx->xch; - xen_pfn_t *mfns = malloc(count * sizeof(*mfns)), - *pfns = malloc(count * sizeof(*pfns)); + xen_pfn_t *mfns = malloc(count * sizeof(*mfns)); unsigned int i, nr_pfns = 0; int rc = -1; - if ( !mfns || !pfns ) + if ( !mfns ) { ERROR("Failed to allocate %zu bytes for populating the physmap", 2 * count * sizeof(*mfns)); @@ -158,7 +157,7 @@ int populate_pfns(struct xc_sr_context *ctx, unsigned int count, rc = pfn_set_populated(ctx, original_pfns[i]); if ( rc ) goto err; - pfns[nr_pfns] = mfns[nr_pfns] = original_pfns[i]; + ctx->restore.pp_pfns[nr_pfns] = mfns[nr_pfns] = original_pfns[i]; ++nr_pfns; } } @@ -182,14 +181,13 @@ int populate_pfns(struct xc_sr_context *ctx, unsigned int count, goto err; } - ctx->restore.ops.set_gfn(ctx, pfns[i], mfns[i]); + ctx->restore.ops.set_gfn(ctx, ctx->restore.pp_pfns[i], mfns[i]); } } rc = 0; err: - free(pfns); free(mfns); return rc; @@ -707,8 +705,9 @@ static int setup(struct xc_sr_context *ctx) ctx->restore.types = malloc(MAX_BATCH_SIZE * sizeof(*ctx->restore.types)); ctx->restore.mfns = malloc(MAX_BATCH_SIZE * sizeof(*ctx->restore.mfns)); ctx->restore.map_errs = malloc(MAX_BATCH_SIZE * sizeof(*ctx->restore.map_errs)); + ctx->restore.pp_pfns = malloc(MAX_BATCH_SIZE * sizeof(*ctx->restore.pp_pfns)); if ( !ctx->restore.pfns || !ctx->restore.types || !ctx->restore.mfns || - !ctx->restore.map_errs ) + !ctx->restore.map_errs || !ctx->restore.pp_pfns ) { ERROR("Unable to allocate memory"); rc = -1; @@ -745,6 +744,7 @@ static void cleanup(struct xc_sr_context *ctx) free(ctx->restore.buffered_records); free(ctx->restore.populated_pfns); + free(ctx->restore.pp_pfns); free(ctx->restore.map_errs); free(ctx->restore.mfns); free(ctx->restore.types); From patchwork Tue Jul 13 18:05:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12374725 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 228CBC07E95 for ; Tue, 13 Jul 2021 18:16:28 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E01CA6136D for ; Tue, 13 Jul 2021 18:16:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E01CA6136D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.155619.287368 (Exim 4.92) (envelope-from ) id 1m3Mx9-0000dr-6Z; Tue, 13 Jul 2021 18:16:19 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 155619.287368; Tue, 13 Jul 2021 18:16:19 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3Mx9-0000dN-3F; Tue, 13 Jul 2021 18:16:19 +0000 Received: by outflank-mailman (input) for mailman id 155619; Tue, 13 Jul 2021 18:16:17 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3MoK-0002kr-Um for xen-devel@lists.xenproject.org; Tue, 13 Jul 2021 18:07:12 +0000 Received: from mo4-p03-ob.smtp.rzone.de (unknown [85.215.255.100]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 08fbe437-e405-11eb-8793-12813bfff9fa; Tue, 13 Jul 2021 18:06:25 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.28.1 AUTH) with ESMTPSA id 30791cx6DI6JtB1 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Tue, 13 Jul 2021 20:06:19 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 08fbe437-e405-11eb-8793-12813bfff9fa DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1626199579; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=wFaHjRr5CaNl9QDq6ACoz1L0ZIK5luGgqWXq90MCCnY=; b=iGZkvu+nFDp2Qi9G/qyWxe53jJ5q5rpan1wOShcflsFO4qJkJDt19PiA+mHfcbxnDs 4mKieBHWjIJld2E6m1z3zkX0P1f2F/tai4zZufT5ngAXAf8KCRR/cHZxFmJlHu4++E3n dhiiXQ9bwccdaAdpCH/AW8A8S11ZhevYyoLSiGWNYYy2IiNVzs1XCd1Ss2hSPpzp6ru3 aBlrRnhBGBHTGRGyQZYB6y4lgxvGrHIWeQ8vFB/tbSC/+Z9kHznU2lnKiy7nk5nVueJc 2emK1CkzG69D1aj0YCdSBQ5yNRMH2LZvquqR5o/yQdUmrYekoYi4rgOZvtRGLjVaDbpe 1kMA== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAgthqlagRsstpH9XYEN5v9xf7dypFU9ge5p0Oi0V" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu , Juergen Gross Subject: [PATCH v20210713 19/31] tools: restore: preallocate populate_pfns mfns array Date: Tue, 13 Jul 2021 20:05:53 +0200 Message-Id: <20210713180605.12096-20-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210713180605.12096-1-olaf@aepfle.de> References: <20210713180605.12096-1-olaf@aepfle.de> MIME-Version: 1.0 Remove repeated allocation from migration loop. There will never be more than MAX_BATCH_SIZE pages to process in an incoming batch. Allocate the space once. Use some prefix to avoid conflict with an array used in handle_page_data. Signed-off-by: Olaf Hering --- tools/libs/guest/xg_sr_common.h | 1 + tools/libs/guest/xg_sr_restore.c | 23 ++++++++--------------- 2 files changed, 9 insertions(+), 15 deletions(-) diff --git a/tools/libs/guest/xg_sr_common.h b/tools/libs/guest/xg_sr_common.h index ca44e59382..c6c6a31dc6 100644 --- a/tools/libs/guest/xg_sr_common.h +++ b/tools/libs/guest/xg_sr_common.h @@ -261,6 +261,7 @@ struct xc_sr_context xen_pfn_t *mfns; int *map_errs; xen_pfn_t *pp_pfns; + xen_pfn_t *pp_mfns; int send_back_fd; unsigned long p2m_size; diff --git a/tools/libs/guest/xg_sr_restore.c b/tools/libs/guest/xg_sr_restore.c index caac4bb5fa..2734ef06fd 100644 --- a/tools/libs/guest/xg_sr_restore.c +++ b/tools/libs/guest/xg_sr_restore.c @@ -138,17 +138,9 @@ int populate_pfns(struct xc_sr_context *ctx, unsigned int count, const xen_pfn_t *original_pfns, const uint32_t *types) { xc_interface *xch = ctx->xch; - xen_pfn_t *mfns = malloc(count * sizeof(*mfns)); unsigned int i, nr_pfns = 0; int rc = -1; - if ( !mfns ) - { - ERROR("Failed to allocate %zu bytes for populating the physmap", - 2 * count * sizeof(*mfns)); - goto err; - } - for ( i = 0; i < count; ++i ) { if ( (!types || page_type_to_populate(types[i])) && @@ -157,7 +149,7 @@ int populate_pfns(struct xc_sr_context *ctx, unsigned int count, rc = pfn_set_populated(ctx, original_pfns[i]); if ( rc ) goto err; - ctx->restore.pp_pfns[nr_pfns] = mfns[nr_pfns] = original_pfns[i]; + ctx->restore.pp_pfns[nr_pfns] = ctx->restore.pp_mfns[nr_pfns] = original_pfns[i]; ++nr_pfns; } } @@ -165,7 +157,7 @@ int populate_pfns(struct xc_sr_context *ctx, unsigned int count, if ( nr_pfns ) { rc = xc_domain_populate_physmap_exact( - xch, ctx->domid, nr_pfns, 0, 0, mfns); + xch, ctx->domid, nr_pfns, 0, 0, ctx->restore.pp_mfns); if ( rc ) { PERROR("Failed to populate physmap"); @@ -174,22 +166,20 @@ int populate_pfns(struct xc_sr_context *ctx, unsigned int count, for ( i = 0; i < nr_pfns; ++i ) { - if ( mfns[i] == INVALID_MFN ) + if ( ctx->restore.pp_mfns[i] == INVALID_MFN ) { ERROR("Populate physmap failed for pfn %u", i); rc = -1; goto err; } - ctx->restore.ops.set_gfn(ctx, ctx->restore.pp_pfns[i], mfns[i]); + ctx->restore.ops.set_gfn(ctx, ctx->restore.pp_pfns[i], ctx->restore.pp_mfns[i]); } } rc = 0; err: - free(mfns); - return rc; } @@ -706,8 +696,10 @@ static int setup(struct xc_sr_context *ctx) ctx->restore.mfns = malloc(MAX_BATCH_SIZE * sizeof(*ctx->restore.mfns)); ctx->restore.map_errs = malloc(MAX_BATCH_SIZE * sizeof(*ctx->restore.map_errs)); ctx->restore.pp_pfns = malloc(MAX_BATCH_SIZE * sizeof(*ctx->restore.pp_pfns)); + ctx->restore.pp_mfns = malloc(MAX_BATCH_SIZE * sizeof(*ctx->restore.pp_mfns)); if ( !ctx->restore.pfns || !ctx->restore.types || !ctx->restore.mfns || - !ctx->restore.map_errs || !ctx->restore.pp_pfns ) + !ctx->restore.map_errs || !ctx->restore.pp_pfns || + !ctx->restore.pp_mfns ) { ERROR("Unable to allocate memory"); rc = -1; @@ -744,6 +736,7 @@ static void cleanup(struct xc_sr_context *ctx) free(ctx->restore.buffered_records); free(ctx->restore.populated_pfns); + free(ctx->restore.pp_mfns); free(ctx->restore.pp_pfns); free(ctx->restore.map_errs); free(ctx->restore.mfns); From patchwork Tue Jul 13 18:05:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12374711 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 090E3C07E96 for ; Tue, 13 Jul 2021 18:07:03 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C4021613D1 for ; Tue, 13 Jul 2021 18:07:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C4021613D1 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.155556.287275 (Exim 4.92) (envelope-from ) id 1m3Mo3-00017q-Sk; Tue, 13 Jul 2021 18:06:55 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 155556.287275; Tue, 13 Jul 2021 18:06:55 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3Mo3-000175-Nt; Tue, 13 Jul 2021 18:06:55 +0000 Received: by outflank-mailman (input) for mailman id 155556; Tue, 13 Jul 2021 18:06:54 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3Mo2-0002UL-H5 for xen-devel@lists.xenproject.org; Tue, 13 Jul 2021 18:06:54 +0000 Received: from mo4-p03-ob.smtp.rzone.de (unknown [85.215.255.101]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 21a92166-9a3d-403a-96dc-95dc64418e01; Tue, 13 Jul 2021 18:06:26 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.28.1 AUTH) with ESMTPSA id 30791cx6DI6JtB2 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Tue, 13 Jul 2021 20:06:19 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 21a92166-9a3d-403a-96dc-95dc64418e01 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1626199579; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=Fd4Q1B2DPhDxZ+ElDKm1Z9i8GuAzFG2GaIOlaJriZzY=; b=pGHNg93OpQHnVpiCxnMQ9Pw4QQhRBMPvijxM9tSsveV5FJl2EGBWmbmGF4yYDlirbn tn1FDgsNc1FfLeVXG9sEhvVtG/uH7B4pD+rGSDaGx5l4++29llXNoHdGpwNuZ9ljJTTN 3HP39YcSzBPOBYlBe6KgUm5eRb/RTjImATr/rmrYjYtQdFmTIntAL/5a0/Lz92KnxLA6 MBUL1Mmv5U5F/KVGrIB8Y0tRwGgi8MoU/n+5KvxwTS8PJvX9P5z8H41APDM4grf9dDOy DpdrrAWC9iL4ZEZWhFj6A827uyOvgvhnBZMP4rUF8BHuAi0U2sko5AjJPP+9+6aLdsDn s27w== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAgthqlagRsstpH9XYEN5v9xf7dypFU9ge5p0Oi0V" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Juergen Gross , Ian Jackson , Wei Liu Subject: [PATCH v20210713 20/31] tools: restore: split record processing Date: Tue, 13 Jul 2021 20:05:54 +0200 Message-Id: <20210713180605.12096-21-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210713180605.12096-1-olaf@aepfle.de> References: <20210713180605.12096-1-olaf@aepfle.de> MIME-Version: 1.0 handle_page_data must be able to read directly into mapped guest memory. This will avoid unneccesary memcpy calls for data which can be consumed verbatim. Rearrange the code to allow decisions based on the incoming record. This change is preparation for future changes in handle_page_data, no change in behavior is intended. Signed-off-by: Olaf Hering Reviewed-by: Juergen Gross --- tools/libs/guest/xg_sr_common.c | 33 ++++++++++++--------- tools/libs/guest/xg_sr_common.h | 4 ++- tools/libs/guest/xg_sr_restore.c | 49 ++++++++++++++++++++++---------- tools/libs/guest/xg_sr_save.c | 7 ++++- 4 files changed, 63 insertions(+), 30 deletions(-) diff --git a/tools/libs/guest/xg_sr_common.c b/tools/libs/guest/xg_sr_common.c index 17567ab133..cabde4ef74 100644 --- a/tools/libs/guest/xg_sr_common.c +++ b/tools/libs/guest/xg_sr_common.c @@ -91,26 +91,33 @@ int write_split_record(struct xc_sr_context *ctx, struct xc_sr_record *rec, return -1; } -int read_record(struct xc_sr_context *ctx, int fd, struct xc_sr_record *rec) +int read_record_header(struct xc_sr_context *ctx, int fd, struct xc_sr_rhdr *rhdr) { xc_interface *xch = ctx->xch; - struct xc_sr_rhdr rhdr; - size_t datasz; - if ( read_exact(fd, &rhdr, sizeof(rhdr)) ) + if ( read_exact(fd, rhdr, sizeof(*rhdr)) ) { PERROR("Failed to read Record Header from stream"); return -1; } - if ( rhdr.length > REC_LENGTH_MAX ) + if ( rhdr->length > REC_LENGTH_MAX ) { - ERROR("Record (0x%08x, %s) length %#x exceeds max (%#x)", rhdr.type, - rec_type_to_str(rhdr.type), rhdr.length, REC_LENGTH_MAX); + ERROR("Record (0x%08x, %s) length %#x exceeds max (%#x)", rhdr->type, + rec_type_to_str(rhdr->type), rhdr->length, REC_LENGTH_MAX); return -1; } - datasz = ROUNDUP(rhdr.length, REC_ALIGN_ORDER); + return 0; +} + +int read_record_data(struct xc_sr_context *ctx, int fd, struct xc_sr_rhdr *rhdr, + struct xc_sr_record *rec) +{ + xc_interface *xch = ctx->xch; + size_t datasz; + + datasz = ROUNDUP(rhdr->length, REC_ALIGN_ORDER); if ( datasz ) { @@ -119,7 +126,7 @@ int read_record(struct xc_sr_context *ctx, int fd, struct xc_sr_record *rec) if ( !rec->data ) { ERROR("Unable to allocate %zu bytes for record data (0x%08x, %s)", - datasz, rhdr.type, rec_type_to_str(rhdr.type)); + datasz, rhdr->type, rec_type_to_str(rhdr->type)); return -1; } @@ -128,18 +135,18 @@ int read_record(struct xc_sr_context *ctx, int fd, struct xc_sr_record *rec) free(rec->data); rec->data = NULL; PERROR("Failed to read %zu bytes of data for record (0x%08x, %s)", - datasz, rhdr.type, rec_type_to_str(rhdr.type)); + datasz, rhdr->type, rec_type_to_str(rhdr->type)); return -1; } } else rec->data = NULL; - rec->type = rhdr.type; - rec->length = rhdr.length; + rec->type = rhdr->type; + rec->length = rhdr->length; return 0; -}; +} static void __attribute__((unused)) build_assertions(void) { diff --git a/tools/libs/guest/xg_sr_common.h b/tools/libs/guest/xg_sr_common.h index c6c6a31dc6..077d8f2890 100644 --- a/tools/libs/guest/xg_sr_common.h +++ b/tools/libs/guest/xg_sr_common.h @@ -458,7 +458,9 @@ static inline int write_record(struct xc_sr_context *ctx, * * On failure, the contents of the record structure are undefined. */ -int read_record(struct xc_sr_context *ctx, int fd, struct xc_sr_record *rec); +int read_record_header(struct xc_sr_context *ctx, int fd, struct xc_sr_rhdr *rhdr); +int read_record_data(struct xc_sr_context *ctx, int fd, struct xc_sr_rhdr *rhdr, + struct xc_sr_record *rec); /* * This would ideally be private in restore.c, but is needed by diff --git a/tools/libs/guest/xg_sr_restore.c b/tools/libs/guest/xg_sr_restore.c index 2734ef06fd..8f6da97d09 100644 --- a/tools/libs/guest/xg_sr_restore.c +++ b/tools/libs/guest/xg_sr_restore.c @@ -466,7 +466,7 @@ static int send_checkpoint_dirty_pfn_list(struct xc_sr_context *ctx) return rc; } -static int process_record(struct xc_sr_context *ctx, struct xc_sr_record *rec); +static int process_buffered_record(struct xc_sr_context *ctx, struct xc_sr_record *rec); static int handle_checkpoint(struct xc_sr_context *ctx) { xc_interface *xch = ctx->xch; @@ -505,7 +505,7 @@ static int handle_checkpoint(struct xc_sr_context *ctx) for ( i = 0; i < ctx->restore.buffered_rec_num; i++ ) { - rc = process_record(ctx, &ctx->restore.buffered_records[i]); + rc = process_buffered_record(ctx, &ctx->restore.buffered_records[i]); if ( rc ) goto err; } @@ -566,10 +566,11 @@ static int handle_checkpoint(struct xc_sr_context *ctx) return rc; } -static int buffer_record(struct xc_sr_context *ctx, struct xc_sr_record *rec) +static int buffer_record(struct xc_sr_context *ctx, struct xc_sr_rhdr *rhdr) { xc_interface *xch = ctx->xch; unsigned int new_alloc_num; + struct xc_sr_record rec; struct xc_sr_record *p; if ( ctx->restore.buffered_rec_num >= ctx->restore.allocated_rec_num ) @@ -587,8 +588,13 @@ static int buffer_record(struct xc_sr_context *ctx, struct xc_sr_record *rec) ctx->restore.allocated_rec_num = new_alloc_num; } + if ( read_record_data(ctx, ctx->fd, rhdr, &rec) ) + { + return -1; + } + memcpy(&ctx->restore.buffered_records[ctx->restore.buffered_rec_num++], - rec, sizeof(*rec)); + &rec, sizeof(rec)); return 0; } @@ -619,7 +625,7 @@ int handle_static_data_end(struct xc_sr_context *ctx) return rc; } -static int process_record(struct xc_sr_context *ctx, struct xc_sr_record *rec) +static int process_buffered_record(struct xc_sr_context *ctx, struct xc_sr_record *rec) { xc_interface *xch = ctx->xch; int rc = 0; @@ -657,6 +663,19 @@ static int process_record(struct xc_sr_context *ctx, struct xc_sr_record *rec) return rc; } +static int process_incoming_record_header(struct xc_sr_context *ctx, struct xc_sr_rhdr *rhdr) +{ + struct xc_sr_record rec; + int rc; + + rc = read_record_data(ctx, ctx->fd, rhdr, &rec); + if ( rc ) + return rc; + + return process_buffered_record(ctx, &rec); +} + + static int setup(struct xc_sr_context *ctx) { xc_interface *xch = ctx->xch; @@ -753,7 +772,7 @@ static void cleanup(struct xc_sr_context *ctx) static int restore(struct xc_sr_context *ctx) { xc_interface *xch = ctx->xch; - struct xc_sr_record rec; + struct xc_sr_rhdr rhdr; int rc, saved_rc = 0, saved_errno = 0; IPRINTF("Restoring domain"); @@ -764,7 +783,7 @@ static int restore(struct xc_sr_context *ctx) do { - rc = read_record(ctx, ctx->fd, &rec); + rc = read_record_header(ctx, ctx->fd, &rhdr); if ( rc ) { if ( ctx->restore.buffer_all_records ) @@ -774,25 +793,25 @@ static int restore(struct xc_sr_context *ctx) } if ( ctx->restore.buffer_all_records && - rec.type != REC_TYPE_END && - rec.type != REC_TYPE_CHECKPOINT ) + rhdr.type != REC_TYPE_END && + rhdr.type != REC_TYPE_CHECKPOINT ) { - rc = buffer_record(ctx, &rec); + rc = buffer_record(ctx, &rhdr); if ( rc ) goto err; } else { - rc = process_record(ctx, &rec); + rc = process_incoming_record_header(ctx, &rhdr); if ( rc == RECORD_NOT_PROCESSED ) { - if ( rec.type & REC_TYPE_OPTIONAL ) + if ( rhdr.type & REC_TYPE_OPTIONAL ) DPRINTF("Ignoring optional record %#x (%s)", - rec.type, rec_type_to_str(rec.type)); + rhdr.type, rec_type_to_str(rhdr.type)); else { ERROR("Mandatory record %#x (%s) not handled", - rec.type, rec_type_to_str(rec.type)); + rhdr.type, rec_type_to_str(rhdr.type)); rc = -1; goto err; } @@ -803,7 +822,7 @@ static int restore(struct xc_sr_context *ctx) goto err; } - } while ( rec.type != REC_TYPE_END ); + } while ( rhdr.type != REC_TYPE_END ); remus_failover: if ( ctx->stream_type == XC_STREAM_COLO ) diff --git a/tools/libs/guest/xg_sr_save.c b/tools/libs/guest/xg_sr_save.c index c74c8d9348..ef8c49f23d 100644 --- a/tools/libs/guest/xg_sr_save.c +++ b/tools/libs/guest/xg_sr_save.c @@ -591,6 +591,7 @@ static int send_memory_live(struct xc_sr_context *ctx) static int colo_merge_secondary_dirty_bitmap(struct xc_sr_context *ctx) { xc_interface *xch = ctx->xch; + struct xc_sr_rhdr rhdr; struct xc_sr_record rec; uint64_t *pfns = NULL; uint64_t pfn; @@ -599,7 +600,11 @@ static int colo_merge_secondary_dirty_bitmap(struct xc_sr_context *ctx) DECLARE_HYPERCALL_BUFFER_SHADOW(unsigned long, dirty_bitmap, &ctx->save.dirty_bitmap_hbuf); - rc = read_record(ctx, ctx->save.recv_fd, &rec); + rc = read_record_header(ctx, ctx->save.recv_fd, &rhdr); + if ( rc ) + goto err; + + rc = read_record_data(ctx, ctx->save.recv_fd, &rhdr, &rec); if ( rc ) goto err; From patchwork Tue Jul 13 18:05:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12374719 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1F635C07E95 for ; Tue, 13 Jul 2021 18:16:05 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B2F366044F for ; Tue, 13 Jul 2021 18:16:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B2F366044F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.155598.287322 (Exim 4.92) (envelope-from ) id 1m3Mwj-0007Dz-Lv; Tue, 13 Jul 2021 18:15:53 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 155598.287322; Tue, 13 Jul 2021 18:15:53 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3Mwj-0007Ds-Iu; Tue, 13 Jul 2021 18:15:53 +0000 Received: by outflank-mailman (input) for mailman id 155598; Tue, 13 Jul 2021 18:15:52 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3MoP-0002kr-Uz for xen-devel@lists.xenproject.org; Tue, 13 Jul 2021 18:07:18 +0000 Received: from mo4-p03-ob.smtp.rzone.de (unknown [81.169.146.173]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 0a456ae3-e405-11eb-8793-12813bfff9fa; Tue, 13 Jul 2021 18:06:27 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.28.1 AUTH) with ESMTPSA id 30791cx6DI6KtB3 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Tue, 13 Jul 2021 20:06:20 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 0a456ae3-e405-11eb-8793-12813bfff9fa DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1626199580; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=KXZqlmH3BT1hKTi/gqEWtzOcR1ftjdBkrbLwzUIkCqY=; b=oKqOBiKczsL0VEbHMdRM8eTatQ/c4qBBN8C0orirKp84TgFzaCoDVbKJ0AUObahCKe 95ZFIub0tYsWfzbruWy8YloWi0pQ0nSO2CYou6qBqpvPb8Awad8daxuLnIPsEaajCePl JbNGT7tsFG5VGue0N1+n9BuLOFWacdo+9iYRAd4nWtr7T6AHOrzE+PHOQiNnRDLaroc1 5RDxgwO8K8ebJFuTegEI8OpOmFmgLdv3oihSV9QDCZgFH/OPXdfeWxuyZW42ADWcluKh Fa9c9G0XoI12PgCVI+B27SsKbFQ8jH2iEC180ksQr64zSOorfX1ClajKSrofVkKRzlgu tDAA== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAgthqlagRsstpH9XYEN5v9xf7dypFU9ge5p0Oi0V" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu , Juergen Gross Subject: [PATCH v20210713 21/31] tools: restore: split handle_page_data Date: Tue, 13 Jul 2021 20:05:55 +0200 Message-Id: <20210713180605.12096-22-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210713180605.12096-1-olaf@aepfle.de> References: <20210713180605.12096-1-olaf@aepfle.de> MIME-Version: 1.0 handle_page_data must be able to read directly into mapped guest memory. This will avoid unneccesary memcpy calls for data that can be consumed verbatim. Split the various steps of record processing: - move processing to handle_buffered_page_data - adjust xenforeignmemory_map to set errno in case of failure - adjust verify mode to set errno in case of failure This change is preparation for future changes in handle_page_data, no change in behavior is intended. Signed-off-by: Olaf Hering --- tools/libs/guest/xg_sr_common.h | 4 + tools/libs/guest/xg_sr_restore.c | 320 ++++++++++++++++++++----------- 2 files changed, 207 insertions(+), 117 deletions(-) diff --git a/tools/libs/guest/xg_sr_common.h b/tools/libs/guest/xg_sr_common.h index 077d8f2890..020b41ea8b 100644 --- a/tools/libs/guest/xg_sr_common.h +++ b/tools/libs/guest/xg_sr_common.h @@ -262,6 +262,10 @@ struct xc_sr_context int *map_errs; xen_pfn_t *pp_pfns; xen_pfn_t *pp_mfns; + void **guest_data; + + void *guest_mapping; + uint32_t nr_mapped_pages; int send_back_fd; unsigned long p2m_size; diff --git a/tools/libs/guest/xg_sr_restore.c b/tools/libs/guest/xg_sr_restore.c index 8f6da97d09..777d005fbd 100644 --- a/tools/libs/guest/xg_sr_restore.c +++ b/tools/libs/guest/xg_sr_restore.c @@ -183,121 +183,18 @@ int populate_pfns(struct xc_sr_context *ctx, unsigned int count, return rc; } -/* - * Given a list of pfns, their types, and a block of page data from the - * stream, populate and record their types, map the relevant subset and copy - * the data into the guest. - */ -static int process_page_data(struct xc_sr_context *ctx, unsigned int count, - xen_pfn_t *pfns, uint32_t *types, void *page_data) +static int handle_static_data_end_v2(struct xc_sr_context *ctx) { - xc_interface *xch = ctx->xch; - int rc; - void *mapping = NULL, *guest_page = NULL; - unsigned int i, /* i indexes the pfns from the record. */ - j, /* j indexes the subset of pfns we decide to map. */ - nr_pages = 0; - - rc = populate_pfns(ctx, count, pfns, types); - if ( rc ) - { - ERROR("Failed to populate pfns for batch of %u pages", count); - goto err; - } - - for ( i = 0; i < count; ++i ) - { - ctx->restore.ops.set_page_type(ctx, pfns[i], types[i]); - - if ( page_type_has_stream_data(types[i]) ) - ctx->restore.mfns[nr_pages++] = ctx->restore.ops.pfn_to_gfn(ctx, pfns[i]); - } - - /* Nothing to do? */ - if ( nr_pages == 0 ) - goto done; - - mapping = guest_page = xenforeignmemory_map( - xch->fmem, ctx->domid, PROT_READ | PROT_WRITE, - nr_pages, ctx->restore.mfns, ctx->restore.map_errs); - if ( !mapping ) - { - rc = -1; - PERROR("Unable to map %u mfns for %u pages of data", - nr_pages, count); - goto err; - } - - for ( i = 0, j = 0; i < count; ++i ) - { - if ( !page_type_has_stream_data(types[i]) ) - continue; - - if ( ctx->restore.map_errs[j] ) - { - rc = -1; - ERROR("Mapping pfn %#"PRIpfn" (mfn %#"PRIpfn", type %#"PRIx32") failed with %d", - pfns[i], ctx->restore.mfns[j], types[i], ctx->restore.map_errs[j]); - goto err; - } - - /* Undo page normalisation done by the saver. */ - rc = ctx->restore.ops.localise_page(ctx, types[i], page_data); - if ( rc ) - { - ERROR("Failed to localise pfn %#"PRIpfn" (type %#"PRIx32")", - pfns[i], types[i] >> XEN_DOMCTL_PFINFO_LTAB_SHIFT); - goto err; - } - - if ( ctx->restore.verify ) - { - /* Verify mode - compare incoming data to what we already have. */ - if ( memcmp(guest_page, page_data, PAGE_SIZE) ) - ERROR("verify pfn %#"PRIpfn" failed (type %#"PRIx32")", - pfns[i], types[i] >> XEN_DOMCTL_PFINFO_LTAB_SHIFT); - } - else - { - /* Regular mode - copy incoming data into place. */ - memcpy(guest_page, page_data, PAGE_SIZE); - } - - ++j; - guest_page += PAGE_SIZE; - page_data += PAGE_SIZE; - } - - done: - rc = 0; - - err: - if ( mapping ) - xenforeignmemory_unmap(xch->fmem, mapping, nr_pages); - - return rc; -} + int rc = 0; -/* - * Validate a PAGE_DATA record from the stream, and pass the results to - * process_page_data() to actually perform the legwork. - */ -static int handle_page_data(struct xc_sr_context *ctx, struct xc_sr_record *rec) -{ +#if defined(__i386__) || defined(__x86_64__) xc_interface *xch = ctx->xch; - struct xc_sr_rec_page_data_header *pages = rec->data; - unsigned int i, pages_of_data = 0; - int rc = -1; - - xen_pfn_t pfn; - uint32_t type; - /* * v2 compatibility only exists for x86 streams. This is a bit of a * bodge, but it is less bad than duplicating handle_page_data() between * different architectures. */ -#if defined(__i386__) || defined(__x86_64__) + /* v2 compat. Infer the position of STATIC_DATA_END. */ if ( ctx->restore.format_version < 3 && !ctx->restore.seen_static_data_end ) { @@ -315,12 +212,26 @@ static int handle_page_data(struct xc_sr_context *ctx, struct xc_sr_record *rec) ERROR("No STATIC_DATA_END seen"); goto err; } + + rc = 0; +err: #endif - if ( rec->length < sizeof(*pages) ) + return rc; +} + +static bool verify_rec_page_hdr(struct xc_sr_context *ctx, uint32_t rec_length, + struct xc_sr_rec_page_data_header *pages) +{ + xc_interface *xch = ctx->xch; + bool ret = false; + + errno = EINVAL; + + if ( rec_length < sizeof(*pages) ) { ERROR("PAGE_DATA record truncated: length %u, min %zu", - rec->length, sizeof(*pages)); + rec_length, sizeof(*pages)); goto err; } @@ -330,13 +241,28 @@ static int handle_page_data(struct xc_sr_context *ctx, struct xc_sr_record *rec) goto err; } - if ( rec->length < sizeof(*pages) + (pages->count * sizeof(uint64_t)) ) + if ( rec_length < sizeof(*pages) + (pages->count * sizeof(uint64_t)) ) { ERROR("PAGE_DATA record (length %u) too short to contain %u" - " pfns worth of information", rec->length, pages->count); + " pfns worth of information", rec_length, pages->count); goto err; } + ret = true; + +err: + return ret; +} + +static bool verify_rec_page_pfns(struct xc_sr_context *ctx, uint32_t rec_length, + struct xc_sr_rec_page_data_header *pages) +{ + xc_interface *xch = ctx->xch; + uint32_t i, pages_of_data = 0; + xen_pfn_t pfn; + uint32_t type; + bool ret = false; + for ( i = 0; i < pages->count; ++i ) { pfn = pages->pfn[i] & PAGE_DATA_PFN_MASK; @@ -363,19 +289,177 @@ static int handle_page_data(struct xc_sr_context *ctx, struct xc_sr_record *rec) ctx->restore.types[i] = type; } - if ( rec->length != (sizeof(*pages) + + if ( rec_length != (sizeof(*pages) + (sizeof(uint64_t) * pages->count) + (PAGE_SIZE * pages_of_data)) ) { ERROR("PAGE_DATA record wrong size: length %u, expected " - "%zu + %zu + %lu", rec->length, sizeof(*pages), + "%zu + %zu + %lu", rec_length, sizeof(*pages), (sizeof(uint64_t) * pages->count), (PAGE_SIZE * pages_of_data)); goto err; } - rc = process_page_data(ctx, pages->count, ctx->restore.pfns, - ctx->restore.types, &pages->pfn[pages->count]); + ret = true; + +err: + return ret; +} + +/* + * Populate pfns, if required + * Fill guest_data with either mapped address or NULL + * The caller must unmap guest_mapping + */ +static int map_guest_pages(struct xc_sr_context *ctx, + struct xc_sr_rec_page_data_header *pages) +{ + xc_interface *xch = ctx->xch; + uint32_t i, p; + int rc; + + rc = populate_pfns(ctx, pages->count, ctx->restore.pfns, ctx->restore.types); + if ( rc ) + { + ERROR("Failed to populate pfns for batch of %u pages", pages->count); + goto err; + } + + ctx->restore.nr_mapped_pages = 0; + + for ( i = 0; i < pages->count; i++ ) + { + ctx->restore.ops.set_page_type(ctx, ctx->restore.pfns[i], ctx->restore.types[i]); + + if ( page_type_has_stream_data(ctx->restore.types[i]) == false ) + { + ctx->restore.guest_data[i] = NULL; + continue; + } + + ctx->restore.mfns[ctx->restore.nr_mapped_pages++] = ctx->restore.ops.pfn_to_gfn(ctx, ctx->restore.pfns[i]); + } + + /* Nothing to do? */ + if ( ctx->restore.nr_mapped_pages == 0 ) + goto done; + + ctx->restore.guest_mapping = xenforeignmemory_map(xch->fmem, ctx->domid, + PROT_READ | PROT_WRITE, ctx->restore.nr_mapped_pages, + ctx->restore.mfns, ctx->restore.map_errs); + if ( !ctx->restore.guest_mapping ) + { + rc = -1; + PERROR("Unable to map %u mfns for %u pages of data", + ctx->restore.nr_mapped_pages, pages->count); + goto err; + } + + /* Verify mapping, and assign address to pfn data */ + for ( i = 0, p = 0; i < pages->count; i++ ) + { + if ( !page_type_has_stream_data(ctx->restore.types[i]) ) + continue; + + if ( ctx->restore.map_errs[p] == 0 ) + { + ctx->restore.guest_data[i] = ctx->restore.guest_mapping + (p * PAGE_SIZE); + p++; + continue; + } + + errno = ctx->restore.map_errs[p]; + rc = -1; + PERROR("Mapping pfn %#"PRIpfn" (mfn %#"PRIpfn", type %#"PRIx32") failed", + ctx->restore.pfns[i], ctx->restore.mfns[p], ctx->restore.types[i]); + goto err; + } + +done: + rc = 0; + +err: + return rc; +} + +/* + * Handle PAGE_DATA record from an existing buffer + * Given a list of pfns, their types, and a block of page data from the + * stream, populate and record their types, map the relevant subset and copy + * the data into the guest. + */ +static int handle_buffered_page_data(struct xc_sr_context *ctx, + struct xc_sr_record *rec) +{ + xc_interface *xch = ctx->xch; + struct xc_sr_rec_page_data_header *pages = rec->data; + void *p; + uint32_t i; + int rc = -1, idx; + + rc = handle_static_data_end_v2(ctx); + if ( rc ) + goto err; + + /* First read and verify the header */ + if ( !verify_rec_page_hdr(ctx, rec->length, pages) ) + { + rc = -1; + goto err; + } + + /* Then read and verify the pfn numbers */ + if ( !verify_rec_page_pfns(ctx, rec->length, pages) ) + { + rc = -1; + goto err; + } + + /* Map the target pfn */ + rc = map_guest_pages(ctx, pages); + if ( rc ) + goto err; + + for ( i = 0, idx = 0; i < pages->count; i++ ) + { + if ( !ctx->restore.guest_data[i] ) + continue; + + p = &pages->pfn[pages->count] + (idx * PAGE_SIZE); + rc = ctx->restore.ops.localise_page(ctx, ctx->restore.types[i], p); + if ( rc ) + { + ERROR("Failed to localise pfn %#"PRIpfn" (type %#"PRIx32")", + ctx->restore.pfns[i], ctx->restore.types[i] >> XEN_DOMCTL_PFINFO_LTAB_SHIFT); + goto err; + + } + + if ( ctx->restore.verify ) + { + if ( memcmp(ctx->restore.guest_data[i], p, PAGE_SIZE) ) + { + errno = EIO; + ERROR("verify pfn %#"PRIpfn" failed (type %#"PRIx32")", + ctx->restore.pfns[i], ctx->restore.types[i] >> XEN_DOMCTL_PFINFO_LTAB_SHIFT); + goto err; + } + } + else + { + memcpy(ctx->restore.guest_data[i], p, PAGE_SIZE); + } + + idx++; + } + + rc = 0; + err: + if ( ctx->restore.guest_mapping ) + { + xenforeignmemory_unmap(xch->fmem, ctx->restore.guest_mapping, ctx->restore.nr_mapped_pages); + ctx->restore.guest_mapping = NULL; + } return rc; } @@ -636,7 +720,7 @@ static int process_buffered_record(struct xc_sr_context *ctx, struct xc_sr_recor break; case REC_TYPE_PAGE_DATA: - rc = handle_page_data(ctx, rec); + rc = handle_buffered_page_data(ctx, rec); break; case REC_TYPE_VERIFY: @@ -716,9 +800,10 @@ static int setup(struct xc_sr_context *ctx) ctx->restore.map_errs = malloc(MAX_BATCH_SIZE * sizeof(*ctx->restore.map_errs)); ctx->restore.pp_pfns = malloc(MAX_BATCH_SIZE * sizeof(*ctx->restore.pp_pfns)); ctx->restore.pp_mfns = malloc(MAX_BATCH_SIZE * sizeof(*ctx->restore.pp_mfns)); + ctx->restore.guest_data = malloc(MAX_BATCH_SIZE * sizeof(*ctx->restore.guest_data)); if ( !ctx->restore.pfns || !ctx->restore.types || !ctx->restore.mfns || !ctx->restore.map_errs || !ctx->restore.pp_pfns || - !ctx->restore.pp_mfns ) + !ctx->restore.pp_mfns || !ctx->restore.guest_data ) { ERROR("Unable to allocate memory"); rc = -1; @@ -755,6 +840,7 @@ static void cleanup(struct xc_sr_context *ctx) free(ctx->restore.buffered_records); free(ctx->restore.populated_pfns); + free(ctx->restore.guest_data); free(ctx->restore.pp_mfns); free(ctx->restore.pp_pfns); free(ctx->restore.map_errs); From patchwork Tue Jul 13 18:05:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12374731 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F37FEC07E95 for ; Tue, 13 Jul 2021 18:16:39 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C0F4A6044F for ; Tue, 13 Jul 2021 18:16:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C0F4A6044F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.155639.287402 (Exim 4.92) (envelope-from ) id 1m3MxN-00023T-73; Tue, 13 Jul 2021 18:16:33 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 155639.287402; Tue, 13 Jul 2021 18:16:33 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3MxN-00022K-2U; Tue, 13 Jul 2021 18:16:33 +0000 Received: by outflank-mailman (input) for mailman id 155639; Tue, 13 Jul 2021 18:16:31 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3MoC-0002UL-HN for xen-devel@lists.xenproject.org; Tue, 13 Jul 2021 18:07:04 +0000 Received: from mo4-p03-ob.smtp.rzone.de (unknown [81.169.146.173]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 2ecee7c7-15e0-4e66-8c9d-84abd882dd66; Tue, 13 Jul 2021 18:06:26 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.28.1 AUTH) with ESMTPSA id 30791cx6DI6KtB4 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Tue, 13 Jul 2021 20:06:20 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 2ecee7c7-15e0-4e66-8c9d-84abd882dd66 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1626199580; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=XpbwYZI6TX0rUOJJRsTgz/BUclsrX9blWzzFnveyZbk=; b=VFXG3isBNY/F8rK3W7hVJ+jAumS2jr7OBkfKi/kQyVsjO9MhFxPG8EMZfthuBNd0Ca gbpfcdPvvHMeIQHpGU/CA6KyiBijdWqraaSFTSTv8ilxhYhOW2etctIm7X0rw1bjnYeJ HF7sjJrjjJRZ8yAyP6V7IGOgwGVL225GxXMcRUx9u/o7ebMtgdKiKilfu7ptuaeQzNyw ASF81BzLfIQz43rTpRcVUjR3E1eTo06QJbAI18HBifHDclpJORK7cTmIoMiOq+mlA9QV UEwW+0OEye86Y/oW/Vd1w8Oc/WRKUhE96FpEuQt1THvSdlvDkUfj0+VI24fHNFpIjEct 7LnQ== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAgthqlagRsstpH9XYEN5v9xf7dypFU9ge5p0Oi0V" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu , Juergen Gross Subject: [PATCH v20210713 22/31] tools: restore: write data directly into guest Date: Tue, 13 Jul 2021 20:05:56 +0200 Message-Id: <20210713180605.12096-23-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210713180605.12096-1-olaf@aepfle.de> References: <20210713180605.12096-1-olaf@aepfle.de> MIME-Version: 1.0 Read incoming migration stream directly into the guest memory. This avoids the memory allocation and copying, and the resulting performance penalty. Signed-off-by: Olaf Hering --- tools/libs/guest/xg_sr_common.h | 3 + tools/libs/guest/xg_sr_restore.c | 155 ++++++++++++++++++++++++++++++- 2 files changed, 153 insertions(+), 5 deletions(-) diff --git a/tools/libs/guest/xg_sr_common.h b/tools/libs/guest/xg_sr_common.h index 020b41ea8b..8981c21acd 100644 --- a/tools/libs/guest/xg_sr_common.h +++ b/tools/libs/guest/xg_sr_common.h @@ -263,6 +263,8 @@ struct xc_sr_context xen_pfn_t *pp_pfns; xen_pfn_t *pp_mfns; void **guest_data; + struct iovec *iov; + struct xc_sr_rec_page_data_header *pages; void *guest_mapping; uint32_t nr_mapped_pages; @@ -311,6 +313,7 @@ struct xc_sr_context /* Sender has invoked verify mode on the stream. */ bool verify; + void *verify_buf; } restore; }; diff --git a/tools/libs/guest/xg_sr_restore.c b/tools/libs/guest/xg_sr_restore.c index 777d005fbd..d4012bcffd 100644 --- a/tools/libs/guest/xg_sr_restore.c +++ b/tools/libs/guest/xg_sr_restore.c @@ -381,6 +381,129 @@ err: return rc; } +/* + * Handle PAGE_DATA record from the stream. + * Given a list of pfns, their types, and a block of page data from the + * stream, populate and record their types, map the relevant subset and copy + * the data into the guest. + */ +static int handle_incoming_page_data(struct xc_sr_context *ctx, + struct xc_sr_rhdr *rhdr) +{ + xc_interface *xch = ctx->xch; + struct xc_sr_rec_page_data_header *pages = ctx->restore.pages; + uint64_t *pfn_nums = &pages->pfn[0]; + uint32_t i; + int rc, iov_idx; + + rc = handle_static_data_end_v2(ctx); + if ( rc ) + goto err; + + /* First read and verify the header */ + rc = read_exact(ctx->fd, pages, sizeof(*pages)); + if ( rc ) + { + PERROR("Could not read rec_pfn header"); + goto err; + } + + if ( !verify_rec_page_hdr(ctx, rhdr->length, pages) ) + { + rc = -1; + goto err; + } + + /* Then read and verify the incoming pfn numbers */ + rc = read_exact(ctx->fd, pfn_nums, sizeof(*pfn_nums) * pages->count); + if ( rc ) + { + PERROR("Could not read rec_pfn data"); + goto err; + } + + if ( !verify_rec_page_pfns(ctx, rhdr->length, pages) ) + { + rc = -1; + goto err; + } + + /* Finally read and verify the incoming pfn data */ + rc = map_guest_pages(ctx, pages); + if ( rc ) + goto err; + + /* Prepare read buffers, either guest or throw-away memory */ + for ( i = 0, iov_idx = 0; i < pages->count; i++ ) + { + struct iovec *iov; + + if ( !ctx->restore.guest_data[i] ) + continue; + + iov = &ctx->restore.iov[iov_idx]; + iov->iov_len = PAGE_SIZE; + if ( ctx->restore.verify ) + iov->iov_base = ctx->restore.verify_buf + (i * PAGE_SIZE); + else + iov->iov_base = ctx->restore.guest_data[i]; + iov_idx++; + } + + if ( !iov_idx ) + goto done; + + rc = readv_exact(ctx->fd, ctx->restore.iov, iov_idx); + if ( rc ) + { + PERROR("read of %d pages failed", iov_idx); + goto err; + } + + /* Post-processing of pfn data */ + for ( i = 0, iov_idx = 0; i < pages->count; i++ ) + { + void *addr; + + if ( !ctx->restore.guest_data[i] ) + continue; + + addr = ctx->restore.iov[iov_idx].iov_base; + rc = ctx->restore.ops.localise_page(ctx, ctx->restore.types[i], addr); + if ( rc ) + { + ERROR("Failed to localise pfn %#"PRIpfn" (type %#"PRIx32")", + ctx->restore.pfns[i], + ctx->restore.types[i] >> XEN_DOMCTL_PFINFO_LTAB_SHIFT); + goto err; + + } + + if ( ctx->restore.verify ) + { + if ( memcmp(ctx->restore.guest_data[i], addr, PAGE_SIZE) ) + { + ERROR("verify pfn %#"PRIpfn" failed (type %#"PRIx32")", + ctx->restore.pfns[i], + ctx->restore.types[i] >> XEN_DOMCTL_PFINFO_LTAB_SHIFT); + } + } + + iov_idx++; + } + +done: + rc = 0; + +err: + if ( ctx->restore.guest_mapping ) + { + xenforeignmemory_unmap(xch->fmem, ctx->restore.guest_mapping, ctx->restore.nr_mapped_pages); + ctx->restore.guest_mapping = NULL; + } + return rc; +} + /* * Handle PAGE_DATA record from an existing buffer * Given a list of pfns, their types, and a block of page data from the @@ -726,6 +849,15 @@ static int process_buffered_record(struct xc_sr_context *ctx, struct xc_sr_recor case REC_TYPE_VERIFY: DPRINTF("Verify mode enabled"); ctx->restore.verify = true; + if ( !ctx->restore.verify_buf ) + { + ctx->restore.verify_buf = malloc(MAX_BATCH_SIZE * PAGE_SIZE); + if ( !ctx->restore.verify_buf ) + { + PERROR("Unable to allocate verify_buf"); + rc = -1; + } + } break; case REC_TYPE_CHECKPOINT: @@ -752,11 +884,19 @@ static int process_incoming_record_header(struct xc_sr_context *ctx, struct xc_s struct xc_sr_record rec; int rc; - rc = read_record_data(ctx, ctx->fd, rhdr, &rec); - if ( rc ) - return rc; + switch ( rhdr->type ) + { + case REC_TYPE_PAGE_DATA: + rc = handle_incoming_page_data(ctx, rhdr); + break; + default: + rc = read_record_data(ctx, ctx->fd, rhdr, &rec); + if ( rc == 0 ) + rc = process_buffered_record(ctx, &rec);; + break; + } - return process_buffered_record(ctx, &rec); + return rc; } @@ -801,9 +941,12 @@ static int setup(struct xc_sr_context *ctx) ctx->restore.pp_pfns = malloc(MAX_BATCH_SIZE * sizeof(*ctx->restore.pp_pfns)); ctx->restore.pp_mfns = malloc(MAX_BATCH_SIZE * sizeof(*ctx->restore.pp_mfns)); ctx->restore.guest_data = malloc(MAX_BATCH_SIZE * sizeof(*ctx->restore.guest_data)); + ctx->restore.iov = malloc(MAX_BATCH_SIZE * sizeof(*ctx->restore.iov)); + ctx->restore.pages = malloc(MAX_BATCH_SIZE * sizeof(*ctx->restore.pages->pfn) + sizeof(*ctx->restore.pages)); if ( !ctx->restore.pfns || !ctx->restore.types || !ctx->restore.mfns || !ctx->restore.map_errs || !ctx->restore.pp_pfns || - !ctx->restore.pp_mfns || !ctx->restore.guest_data ) + !ctx->restore.pp_mfns || !ctx->restore.guest_data || + !ctx->restore.iov || !ctx->restore.pages ) { ERROR("Unable to allocate memory"); rc = -1; @@ -840,6 +983,8 @@ static void cleanup(struct xc_sr_context *ctx) free(ctx->restore.buffered_records); free(ctx->restore.populated_pfns); + free(ctx->restore.pages); + free(ctx->restore.iov); free(ctx->restore.guest_data); free(ctx->restore.pp_mfns); free(ctx->restore.pp_pfns); From patchwork Tue Jul 13 18:05:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12374715 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EC651C07E96 for ; Tue, 13 Jul 2021 18:07:08 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BBB1A61369 for ; Tue, 13 Jul 2021 18:07:08 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BBB1A61369 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.155559.287295 (Exim 4.92) (envelope-from ) id 1m3Mo8-0002Le-WD; Tue, 13 Jul 2021 18:07:01 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 155559.287295; Tue, 13 Jul 2021 18:07:00 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3Mo8-0002JA-MT; Tue, 13 Jul 2021 18:07:00 +0000 Received: by outflank-mailman (input) for mailman id 155559; Tue, 13 Jul 2021 18:06:59 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3Mo7-0002UL-H7 for xen-devel@lists.xenproject.org; Tue, 13 Jul 2021 18:06:59 +0000 Received: from mo4-p03-ob.smtp.rzone.de (unknown [85.215.255.100]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 045a64bf-ada9-407e-9b5b-9982c9b9430b; Tue, 13 Jul 2021 18:06:27 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.28.1 AUTH) with ESMTPSA id 30791cx6DI6LtB5 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Tue, 13 Jul 2021 20:06:21 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 045a64bf-ada9-407e-9b5b-9982c9b9430b DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1626199581; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=0LMXyjqr5dhJU6Kt86LbarPPbfCTtmro+NAOq1cH7Ts=; b=NbJSr7qRx5y1Mp3a4m25zYbTv6HLGzbHHCVKMN+Gg8MyFXyxcPyht+QXzJ/+z5T6zU Bb8DfaI6GiJI5/jS/wALXD20j2+0sW1hozlazEnDKC+tJnXSWu60yOx9nAfDgI7v6TDX pdbbtJwg/s1xY8Hoj467EEDmaO+Q73184qCKLKrdRo/FK+axgcjLLQRYuiR0xp8noVzt j0vxOn3RAGxzg9QiTI2F7BLA5TAT5FcNwHoHfiGq2xZjVJ65/8fsuQpEnNS3sVLKPPqi 8br5s2ILbnfAY5Mv1KpQ+D/xW6fDIChi+WE646SoQluHjj6s3yFXDQU3EDsmrWY1bOZ9 M34w== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAgthqlagRsstpH9XYEN5v9xf7dypFU9ge5p0Oi0V" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu , Anthony PERARD Subject: [PATCH v20210713 23/31] tools: recognize LIBXL_API_VERSION for 4.16 Date: Tue, 13 Jul 2021 20:05:57 +0200 Message-Id: <20210713180605.12096-24-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210713180605.12096-1-olaf@aepfle.de> References: <20210713180605.12096-1-olaf@aepfle.de> MIME-Version: 1.0 This is required by upcoming API changes. Signed-off-by: Olaf Hering --- tools/include/libxl.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tools/include/libxl.h b/tools/include/libxl.h index ae7fe27c1f..29931626a2 100644 --- a/tools/include/libxl.h +++ b/tools/include/libxl.h @@ -729,7 +729,8 @@ typedef struct libxl__ctx libxl_ctx; #if LIBXL_API_VERSION != 0x040200 && LIBXL_API_VERSION != 0x040300 && \ LIBXL_API_VERSION != 0x040400 && LIBXL_API_VERSION != 0x040500 && \ LIBXL_API_VERSION != 0x040700 && LIBXL_API_VERSION != 0x040800 && \ - LIBXL_API_VERSION != 0x041300 && LIBXL_API_VERSION != 0x041400 + LIBXL_API_VERSION != 0x041300 && LIBXL_API_VERSION != 0x041400 && \ + LIBXL_API_VERSION != 0x041600 #error Unknown LIBXL_API_VERSION #endif #endif From patchwork Tue Jul 13 18:05:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12374723 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5E402C07E95 for ; Tue, 13 Jul 2021 18:16:18 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 138486044F for ; Tue, 13 Jul 2021 18:16:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 138486044F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.155611.287344 (Exim 4.92) (envelope-from ) id 1m3Mx0-00087U-DJ; Tue, 13 Jul 2021 18:16:10 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 155611.287344; Tue, 13 Jul 2021 18:16:10 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3Mx0-00087J-9Y; Tue, 13 Jul 2021 18:16:10 +0000 Received: by outflank-mailman (input) for mailman id 155611; Tue, 13 Jul 2021 18:16:09 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3MoZ-0002kr-VO for xen-devel@lists.xenproject.org; Tue, 13 Jul 2021 18:07:28 +0000 Received: from mo4-p03-ob.smtp.rzone.de (unknown [85.215.255.101]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 0b687569-e405-11eb-8793-12813bfff9fa; Tue, 13 Jul 2021 18:06:30 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.28.1 AUTH) with ESMTPSA id 30791cx6DI6LtB6 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Tue, 13 Jul 2021 20:06:21 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 0b687569-e405-11eb-8793-12813bfff9fa DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1626199581; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=GjudnVF2IHqXv7qlToU0RRwDe7Jc7WilBXGXYBOzt/s=; b=J/0o1XrmqfrXKI3DihqrrnnSctYBw0s8LXUYPsJXNCmiklD8n8Aj/Tkr0R4D4jSDUV GrrgaaxGbsNqdffLtaKR7ifUph8rxmZYP+Uj8jaa5kbmlkXKnOVvyZn3av5k43Pnt8c8 fXFqeX/OZn3fNgBgHji4nMsNg3eh0vMsaAyOgAjpp+wIUh6lgifGly0dOgVhb6AVhTUZ JP9XuOa5pDmYQ7wMZdxTxj64v5X0i6JZgTLG3hkgM8vQJXLXie8u9ZjET5xgn/DA+8RI H6D9QKPeKLSNfyw0WUGTCkie47tzN1+R/N+rVSFdC7RYPXvMmMf5i/I0oyTrspS0yd/Z m6YQ== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAgthqlagRsstpH9XYEN5v9xf7dypFU9ge5p0Oi0V" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Christian Lindig , Ian Jackson , Wei Liu , Anthony PERARD , Juergen Gross , David Scott Subject: [PATCH v20210713 24/31] tools: adjust libxl_domain_suspend to receive a struct props Date: Tue, 13 Jul 2021 20:05:58 +0200 Message-Id: <20210713180605.12096-25-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210713180605.12096-1-olaf@aepfle.de> References: <20210713180605.12096-1-olaf@aepfle.de> MIME-Version: 1.0 Upcoming changes will pass more knobs down to xc_domain_save. Adjust the libxl_domain_suspend API to allow easy adding of additional knobs. No change in behavior intented. Signed-off-by: Olaf Hering Acked-by: Christian Lindig --- tools/include/libxl.h | 26 +++++++++++++++++++++++--- tools/libs/light/libxl_domain.c | 7 ++++--- tools/ocaml/libs/xl/xenlight_stubs.c | 3 ++- tools/xl/xl_migrate.c | 9 ++++++--- tools/xl/xl_saverestore.c | 3 ++- 5 files changed, 37 insertions(+), 11 deletions(-) diff --git a/tools/include/libxl.h b/tools/include/libxl.h index 29931626a2..9a4d7514ed 100644 --- a/tools/include/libxl.h +++ b/tools/include/libxl.h @@ -1706,12 +1706,32 @@ static inline int libxl_retrieve_domain_configuration_0x041200( libxl_retrieve_domain_configuration_0x041200 #endif +/* + * LIBXL_HAVE_DOMAIN_SUSPEND_PROPS indicates that the + * libxl_domain_suspend_props() function takes a props struct. + */ +#define LIBXL_HAVE_DOMAIN_SUSPEND_PROPS 1 + +typedef struct { + uint32_t flags; /* LIBXL_SUSPEND_* */ +} libxl_domain_suspend_props; +#define LIBXL_SUSPEND_DEBUG 1 +#define LIBXL_SUSPEND_LIVE 2 + int libxl_domain_suspend(libxl_ctx *ctx, uint32_t domid, int fd, - int flags, /* LIBXL_SUSPEND_* */ + libxl_domain_suspend_props *props, const libxl_asyncop_how *ao_how) LIBXL_EXTERNAL_CALLERS_ONLY; -#define LIBXL_SUSPEND_DEBUG 1 -#define LIBXL_SUSPEND_LIVE 2 +#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION < 0x041600 +static inline int libxl_domain_suspend_0x041500(libxl_ctx *ctx, uint32_t domid, + int fd, int flags, /* LIBXL_SUSPEND_* */ + const libxl_asyncop_how *ao_how) +{ + libxl_domain_suspend_props props = { .flags = flags, }; + return libxl_domain_suspend(ctx, domid, fd, &props, ao_how); +} +#define libxl_domain_suspend libxl_domain_suspend_0x041500 +#endif /* * Only suspend domain, do not save its state to file, do not destroy it. diff --git a/tools/libs/light/libxl_domain.c b/tools/libs/light/libxl_domain.c index c00c36c928..5dbd27900f 100644 --- a/tools/libs/light/libxl_domain.c +++ b/tools/libs/light/libxl_domain.c @@ -505,7 +505,8 @@ static void domain_suspend_cb(libxl__egc *egc, } -int libxl_domain_suspend(libxl_ctx *ctx, uint32_t domid, int fd, int flags, +int libxl_domain_suspend(libxl_ctx *ctx, uint32_t domid, int fd, + libxl_domain_suspend_props *props, const libxl_asyncop_how *ao_how) { AO_CREATE(ctx, domid, ao_how); @@ -526,8 +527,8 @@ int libxl_domain_suspend(libxl_ctx *ctx, uint32_t domid, int fd, int flags, dss->domid = domid; dss->fd = fd; dss->type = type; - dss->live = flags & LIBXL_SUSPEND_LIVE; - dss->debug = flags & LIBXL_SUSPEND_DEBUG; + dss->live = props->flags & LIBXL_SUSPEND_LIVE; + dss->debug = props->flags & LIBXL_SUSPEND_DEBUG; dss->checkpointed_stream = LIBXL_CHECKPOINTED_STREAM_NONE; rc = libxl__fd_flags_modify_save(gc, dss->fd, diff --git a/tools/ocaml/libs/xl/xenlight_stubs.c b/tools/ocaml/libs/xl/xenlight_stubs.c index 45b8af61c7..ea5ee33ce9 100644 --- a/tools/ocaml/libs/xl/xenlight_stubs.c +++ b/tools/ocaml/libs/xl/xenlight_stubs.c @@ -618,10 +618,11 @@ value stub_libxl_domain_suspend(value ctx, value domid, value fd, value async, v int ret; uint32_t c_domid = Int_val(domid); int c_fd = Int_val(fd); + libxl_domain_suspend_props props = {}; libxl_asyncop_how *ao_how = aohow_val(async); caml_enter_blocking_section(); - ret = libxl_domain_suspend(CTX, c_domid, c_fd, 0, ao_how); + ret = libxl_domain_suspend(CTX, c_domid, c_fd, &props, ao_how); caml_leave_blocking_section(); free(ao_how); diff --git a/tools/xl/xl_migrate.c b/tools/xl/xl_migrate.c index b8594f44a5..144890924f 100644 --- a/tools/xl/xl_migrate.c +++ b/tools/xl/xl_migrate.c @@ -186,7 +186,10 @@ static void migrate_domain(uint32_t domid, int preserve_domid, char *away_domname; char rc_buf; uint8_t *config_data; - int config_len, flags = LIBXL_SUSPEND_LIVE; + int config_len; + libxl_domain_suspend_props props = { + .flags = LIBXL_SUSPEND_LIVE, + }; save_domain_core_begin(domid, preserve_domid, override_config_file, &config_data, &config_len); @@ -205,8 +208,8 @@ static void migrate_domain(uint32_t domid, int preserve_domid, xtl_stdiostream_adjust_flags(logger, XTL_STDIOSTREAM_HIDE_PROGRESS, 0); if (debug) - flags |= LIBXL_SUSPEND_DEBUG; - rc = libxl_domain_suspend(ctx, domid, send_fd, flags, NULL); + props.flags |= LIBXL_SUSPEND_DEBUG; + rc = libxl_domain_suspend(ctx, domid, send_fd, &props, NULL); if (rc) { fprintf(stderr, "migration sender: libxl_domain_suspend failed" " (rc=%d)\n", rc); diff --git a/tools/xl/xl_saverestore.c b/tools/xl/xl_saverestore.c index 953d791d1a..476d4d9a6a 100644 --- a/tools/xl/xl_saverestore.c +++ b/tools/xl/xl_saverestore.c @@ -130,6 +130,7 @@ static int save_domain(uint32_t domid, int preserve_domid, int fd; uint8_t *config_data; int config_len; + libxl_domain_suspend_props props = {}; save_domain_core_begin(domid, preserve_domid, override_config_file, &config_data, &config_len); @@ -146,7 +147,7 @@ static int save_domain(uint32_t domid, int preserve_domid, save_domain_core_writeconfig(fd, filename, config_data, config_len); - int rc = libxl_domain_suspend(ctx, domid, fd, 0, NULL); + int rc = libxl_domain_suspend(ctx, domid, fd, &props, NULL); close(fd); if (rc < 0) { From patchwork Tue Jul 13 18:05:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12374729 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8A0EBC07E96 for ; Tue, 13 Jul 2021 18:16:37 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 595046136D for ; Tue, 13 Jul 2021 18:16:37 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 595046136D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.155634.287385 (Exim 4.92) (envelope-from ) id 1m3MxI-0001Ld-Vq; Tue, 13 Jul 2021 18:16:28 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 155634.287385; Tue, 13 Jul 2021 18:16:28 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3MxI-0001KS-LR; Tue, 13 Jul 2021 18:16:28 +0000 Received: by outflank-mailman (input) for mailman id 155634; Tue, 13 Jul 2021 18:16:27 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3MoW-0002UL-Ht for xen-devel@lists.xenproject.org; Tue, 13 Jul 2021 18:07:24 +0000 Received: from mo4-p03-ob.smtp.rzone.de (unknown [85.215.255.100]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 2d529767-36b1-4653-86ed-96c892b17c81; Tue, 13 Jul 2021 18:06:30 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.28.1 AUTH) with ESMTPSA id 30791cx6DI6MtB7 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Tue, 13 Jul 2021 20:06:22 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 2d529767-36b1-4653-86ed-96c892b17c81 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1626199582; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=+k3zEXoyDaXQEiwvjOV10rf+liXTt7laH3C3jd8kr68=; b=FOKUhjofFTsweTT9yIXVCx+FyE+wsZGTUdNwhv7ts3CvDvMsT/m/Ih+OZszMb2BqK0 EQY51Er931hHXpHOVSLjw90N6/lzp7qO0VAfzr0sii+XeDjQIMw/KCo8lSPnAyb8xw3S YxBXIrL6EbowucDFcKLrQdjkvvVJCZ7FlIDLTzFOHYeqnsqtgzUj9oA2PT3vKb5ACmcz ou9fBTCDsSsQboq4HVYFioY3KeJo6rta0zyTFF6RJshhXvHry3GKuY9pnJ8Hpe3LWo2m 1sOMeDcZzyE7e2i4jcZrDpEHw+me9xJZW9zqoH5MDLK1fPy6N5z9A4k++C0HwceqHP9A iEAw== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAgthqlagRsstpH9XYEN5v9xf7dypFU9ge5p0Oi0V" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu , Anthony PERARD , Juergen Gross Subject: [PATCH v20210713 25/31] tools: add callback to libxl for precopy_policy and precopy_stats Date: Tue, 13 Jul 2021 20:05:59 +0200 Message-Id: <20210713180605.12096-26-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210713180605.12096-1-olaf@aepfle.de> References: <20210713180605.12096-1-olaf@aepfle.de> MIME-Version: 1.0 This duplicates simple_precopy_policy. To recap its purpose: - do up to 5 iterations of copying dirty domU memory to target, including the initial copying of all domU memory, excluding the final copying while the domU is suspended - do fewer iterations in case the domU dirtied less than 50 pages Take the opportunity to also move xen_pfn_t into qw(). Signed-off-by: Olaf Hering --- v02: - use plain struct precopy_stats instead of inventing a new precopy_stats_t (anthony) --- tools/libs/light/libxl_dom_save.c | 19 +++++++++++++++++++ tools/libs/light/libxl_internal.h | 2 ++ tools/libs/light/libxl_save_msgs_gen.pl | 3 ++- 3 files changed, 23 insertions(+), 1 deletion(-) diff --git a/tools/libs/light/libxl_dom_save.c b/tools/libs/light/libxl_dom_save.c index 32e3cb5a13..839e061881 100644 --- a/tools/libs/light/libxl_dom_save.c +++ b/tools/libs/light/libxl_dom_save.c @@ -373,6 +373,24 @@ int libxl__save_emulator_xenstore_data(libxl__domain_save_state *dss, return rc; } +static int libxl__domain_save_precopy_policy(struct precopy_stats stats, void *user) +{ + libxl__save_helper_state *shs = user; + libxl__domain_save_state *dss = shs->caller_state; + STATE_AO_GC(dss->ao); + + LOGD(DEBUG, shs->domid, "iteration %u dirty_count %ld total_written %lu", + stats.iteration, stats.dirty_count, stats.total_written); + if (stats.dirty_count >= 0 && stats.dirty_count < LIBXL_XGS_POLICY_TARGET_DIRTY_COUNT) + goto stop_copy; + if (stats.iteration >= LIBXL_XGS_POLICY_MAX_ITERATIONS) + goto stop_copy; + return XGS_POLICY_CONTINUE_PRECOPY; + +stop_copy: + return XGS_POLICY_STOP_AND_COPY; +} + /*----- main code for saving, in order of execution -----*/ void libxl__domain_save(libxl__egc *egc, libxl__domain_save_state *dss) @@ -430,6 +448,7 @@ void libxl__domain_save(libxl__egc *egc, libxl__domain_save_state *dss) callbacks->suspend = libxl__domain_suspend_callback; callbacks->switch_qemu_logdirty = libxl__domain_suspend_common_switch_qemu_logdirty; + callbacks->precopy_policy = libxl__domain_save_precopy_policy; dss->sws.ao = dss->ao; dss->sws.dss = dss; diff --git a/tools/libs/light/libxl_internal.h b/tools/libs/light/libxl_internal.h index 0b4671318c..f7df4ab397 100644 --- a/tools/libs/light/libxl_internal.h +++ b/tools/libs/light/libxl_internal.h @@ -124,6 +124,8 @@ #define DOMID_XS_PATH "domid" #define PVSHIM_BASENAME "xen-shim" #define PVSHIM_CMDLINE "pv-shim console=xen,pv" +#define LIBXL_XGS_POLICY_MAX_ITERATIONS 5 +#define LIBXL_XGS_POLICY_TARGET_DIRTY_COUNT 50 /* Size macros. */ #define __AC(X,Y) (X##Y) diff --git a/tools/libs/light/libxl_save_msgs_gen.pl b/tools/libs/light/libxl_save_msgs_gen.pl index 9d425b1dee..b95942ed91 100755 --- a/tools/libs/light/libxl_save_msgs_gen.pl +++ b/tools/libs/light/libxl_save_msgs_gen.pl @@ -23,6 +23,7 @@ our @msgs = ( STRING doing_what), 'unsigned long', 'done', 'unsigned long', 'total'] ], + [ 'scxW', "precopy_policy", ['struct precopy_stats', 'stats'] ], [ 'srcxA', "suspend", [] ], [ 'srcxA', "postcopy", [] ], [ 'srcxA', "checkpoint", [] ], @@ -142,7 +143,7 @@ static void bytes_put(unsigned char *const buf, int *len, END -foreach my $simpletype (qw(int uint16_t uint32_t unsigned), 'unsigned long', 'xen_pfn_t') { +foreach my $simpletype (qw(int uint16_t uint32_t unsigned xen_pfn_t), 'struct precopy_stats', 'unsigned long') { my $typeid = typeid($simpletype); $out_body{'callout'} .= < X-Patchwork-Id: 12374727 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8D894C07E95 for ; Tue, 13 Jul 2021 18:16:35 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 496EE6044F for ; Tue, 13 Jul 2021 18:16:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 496EE6044F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.155633.287380 (Exim 4.92) (envelope-from ) id 1m3MxI-0001IF-H5; Tue, 13 Jul 2021 18:16:28 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 155633.287380; Tue, 13 Jul 2021 18:16:28 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3MxI-0001Hy-Da; Tue, 13 Jul 2021 18:16:28 +0000 Received: by outflank-mailman (input) for mailman id 155633; Tue, 13 Jul 2021 18:16:27 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3Mob-0002UL-I1 for xen-devel@lists.xenproject.org; Tue, 13 Jul 2021 18:07:29 +0000 Received: from mo4-p03-ob.smtp.rzone.de (unknown [81.169.146.173]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 4a8a3ea9-6210-4d4a-a421-7fa78117dac6; Tue, 13 Jul 2021 18:06:31 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.28.1 AUTH) with ESMTPSA id 30791cx6DI6MtB8 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Tue, 13 Jul 2021 20:06:22 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 4a8a3ea9-6210-4d4a-a421-7fa78117dac6 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1626199582; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=qvO8GzSJw/Kk2FcJhvVEgMzp+lMjpREiYaCjuaK8qSw=; b=tRRttb8vYOfgsZYnk/bwRjPm3BeTUdiaQl7IH/iqlKEtzSmz+Bn1+jQkHPc0IY+DPn eUIrgIr9sF1AqdAOP2hzxZIBm11miE8FvBL+/wQ2YSPUXbnndr6uUC8tLbSNBUWvLuqr 3kPGrTt2IzUTajYkDAYHwTo2Awnuqjsib7P6kiYBV3N5wgXeTY92vG9iK/BHM71rKGW1 UQdz/Ovoa87e8vAMjO7Nn+KVjbTtw3v4cj0V3+/tnbSQHspe8vR2g6zNkLFAo9NEdeu1 bip6bV7QhBTTNXnmP71pvmwjNNkmkbas+EMTtddKZpXpxBXKyx4WRc88qPDZc+Prf5l7 IlFg== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAgthqlagRsstpH9XYEN5v9xf7dypFU9ge5p0Oi0V" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu , Anthony PERARD , Juergen Gross Subject: [PATCH v20210713 26/31] tools: add --max_iters to libxl_domain_suspend Date: Tue, 13 Jul 2021 20:06:00 +0200 Message-Id: <20210713180605.12096-27-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210713180605.12096-1-olaf@aepfle.de> References: <20210713180605.12096-1-olaf@aepfle.de> MIME-Version: 1.0 Migrating a large, and potentially busy, domU will take more time than neccessary due to excessive number of copying iterations. Allow to host admin to control the number of iterations which copy cumulated domU dirty pages to the target host. The default remains 5, which means one initial iteration to copy the entire domU memory, and up to 4 additional iterations to copy dirty memory from the still running domU. After the given number of iterations the domU is suspended, remaining dirty memory is copied and the domU is finally moved to the target host. This patch adjusts xl(1) and the libxl API. External users check LIBXL_HAVE_DOMAIN_SUSPEND_PROPS for the availibility of the new .max_iters property. Signed-off-by: Olaf Hering --- docs/man/xl.1.pod.in | 4 ++++ tools/include/libxl.h | 1 + tools/libs/light/libxl_dom_save.c | 2 +- tools/libs/light/libxl_domain.c | 1 + tools/libs/light/libxl_internal.h | 1 + tools/xl/xl_cmdtable.c | 3 ++- tools/xl/xl_migrate.c | 10 +++++++++- 7 files changed, 19 insertions(+), 3 deletions(-) diff --git a/docs/man/xl.1.pod.in b/docs/man/xl.1.pod.in index bae557b1de..9b104720d3 100644 --- a/docs/man/xl.1.pod.in +++ b/docs/man/xl.1.pod.in @@ -496,6 +496,10 @@ such that it will be identical on the destination host, unless that configuration is overridden using the B<-C> option. Note that it is not possible to use this option for a 'localhost' migration. +=item B<--max_iters> I + +Number of copy iterations before final suspend+move (default: 5) + =back =item B [I] I I diff --git a/tools/include/libxl.h b/tools/include/libxl.h index 9a4d7514ed..bf77da0524 100644 --- a/tools/include/libxl.h +++ b/tools/include/libxl.h @@ -1714,6 +1714,7 @@ static inline int libxl_retrieve_domain_configuration_0x041200( typedef struct { uint32_t flags; /* LIBXL_SUSPEND_* */ + uint32_t max_iters; } libxl_domain_suspend_props; #define LIBXL_SUSPEND_DEBUG 1 #define LIBXL_SUSPEND_LIVE 2 diff --git a/tools/libs/light/libxl_dom_save.c b/tools/libs/light/libxl_dom_save.c index 839e061881..106c08a199 100644 --- a/tools/libs/light/libxl_dom_save.c +++ b/tools/libs/light/libxl_dom_save.c @@ -383,7 +383,7 @@ static int libxl__domain_save_precopy_policy(struct precopy_stats stats, void *u stats.iteration, stats.dirty_count, stats.total_written); if (stats.dirty_count >= 0 && stats.dirty_count < LIBXL_XGS_POLICY_TARGET_DIRTY_COUNT) goto stop_copy; - if (stats.iteration >= LIBXL_XGS_POLICY_MAX_ITERATIONS) + if (stats.iteration >= dss->max_iters) goto stop_copy; return XGS_POLICY_CONTINUE_PRECOPY; diff --git a/tools/libs/light/libxl_domain.c b/tools/libs/light/libxl_domain.c index 5dbd27900f..9f98cd7f2b 100644 --- a/tools/libs/light/libxl_domain.c +++ b/tools/libs/light/libxl_domain.c @@ -527,6 +527,7 @@ int libxl_domain_suspend(libxl_ctx *ctx, uint32_t domid, int fd, dss->domid = domid; dss->fd = fd; dss->type = type; + dss->max_iters = props->max_iters ?: LIBXL_XGS_POLICY_MAX_ITERATIONS; dss->live = props->flags & LIBXL_SUSPEND_LIVE; dss->debug = props->flags & LIBXL_SUSPEND_DEBUG; dss->checkpointed_stream = LIBXL_CHECKPOINTED_STREAM_NONE; diff --git a/tools/libs/light/libxl_internal.h b/tools/libs/light/libxl_internal.h index f7df4ab397..cfb83d80d7 100644 --- a/tools/libs/light/libxl_internal.h +++ b/tools/libs/light/libxl_internal.h @@ -3648,6 +3648,7 @@ struct libxl__domain_save_state { int live; int debug; int checkpointed_stream; + uint32_t max_iters; const libxl_domain_remus_info *remus; /* private */ int rc; diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c index e4f27a3248..678b236d76 100644 --- a/tools/xl/xl_cmdtable.c +++ b/tools/xl/xl_cmdtable.c @@ -174,7 +174,8 @@ const struct cmd_spec cmd_table[] = { " of the domain.\n" "--debug Enable verification mode.\n" "-p Do not unpause domain after migrating it.\n" - "-D Preserve the domain id" + "-D Preserve the domain id\n" + "--max_iters N Number of copy iterations before final stop+move" }, { "restore", &main_restore, 0, 1, diff --git a/tools/xl/xl_migrate.c b/tools/xl/xl_migrate.c index 144890924f..af117d4d56 100644 --- a/tools/xl/xl_migrate.c +++ b/tools/xl/xl_migrate.c @@ -178,6 +178,7 @@ static void migrate_do_preamble(int send_fd, int recv_fd, pid_t child, static void migrate_domain(uint32_t domid, int preserve_domid, const char *rune, int debug, + uint32_t max_iters, const char *override_config_file) { pid_t child = -1; @@ -189,6 +190,7 @@ static void migrate_domain(uint32_t domid, int preserve_domid, int config_len; libxl_domain_suspend_props props = { .flags = LIBXL_SUSPEND_LIVE, + .max_iters = max_iters, }; save_domain_core_begin(domid, preserve_domid, override_config_file, @@ -542,8 +544,10 @@ int main_migrate(int argc, char **argv) char *host; int opt, daemonize = 1, monitor = 1, debug = 0, pause_after_migration = 0; int preserve_domid = 0; + uint32_t max_iters = 0; static struct option opts[] = { {"debug", 0, 0, 0x100}, + {"max_iters", 1, 0, 0x101}, {"live", 0, 0, 0x200}, COMMON_LONG_OPTS }; @@ -571,6 +575,9 @@ int main_migrate(int argc, char **argv) case 0x100: /* --debug */ debug = 1; break; + case 0x101: /* --max_iters */ + max_iters = atoi(optarg); + break; case 0x200: /* --live */ /* ignored for compatibility with xm */ break; @@ -605,7 +612,8 @@ int main_migrate(int argc, char **argv) pause_after_migration ? " -p" : ""); } - migrate_domain(domid, preserve_domid, rune, debug, config_filename); + migrate_domain(domid, preserve_domid, rune, debug, + max_iters, config_filename); return EXIT_SUCCESS; } From patchwork Tue Jul 13 18:06:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12374737 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 27C7DC07E95 for ; Tue, 13 Jul 2021 18:17:37 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E32916044F for ; Tue, 13 Jul 2021 18:17:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E32916044F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.155684.287435 (Exim 4.92) (envelope-from ) id 1m3MyI-0004ox-7l; Tue, 13 Jul 2021 18:17:30 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 155684.287435; Tue, 13 Jul 2021 18:17:30 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3MyI-0004oo-4R; Tue, 13 Jul 2021 18:17:30 +0000 Received: by outflank-mailman (input) for mailman id 155684; Tue, 13 Jul 2021 18:17:29 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3Moe-0002kr-Vh for xen-devel@lists.xenproject.org; Tue, 13 Jul 2021 18:07:33 +0000 Received: from mo4-p03-ob.smtp.rzone.de (unknown [85.215.255.101]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 0d8f4cae-e405-11eb-8793-12813bfff9fa; Tue, 13 Jul 2021 18:06:32 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.28.1 AUTH) with ESMTPSA id 30791cx6DI6NtB9 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Tue, 13 Jul 2021 20:06:23 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 0d8f4cae-e405-11eb-8793-12813bfff9fa DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1626199583; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=I/lmWw3IgCqn4YYqBC4pCwVLRg7vIlArFuK/FzOA0bI=; b=P61u/Ci+HPY72HauyHCN+5Avj8F/wP2x1/Y9Fv4FTdxfsI3QCFWkczjPGfTORgy0vO cqvwgUwW5zCnBunpFY8OEhaFhvcnuFuQ/BkGfSMX8v9p2eu3a7bBPSdLOwn1ea1sbLkm eHaTMEdUNZcGCogcGWKiayJP3J35Rs8EebntWXkT0F5KUvL3K6nKFllAzBNxr976x6EX n42cWcEZgGkeycoTdnZqLAlcEnoTKr0wQn03ig97gGEr+xpKxynRJK10FGiFVxS6lSVZ R33gcoNpl+zVc+XNe8/Sd0BEG2KvZhemQgeAS7Fpo0K10mITC0IdVf/rBbGInlcX7V9b B24w== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAgthqlagRsstpH9XYEN5v9xf7dypFU9ge5p0Oi0V" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu , Anthony PERARD , Juergen Gross Subject: [PATCH v20210713 27/31] tools: add --min_remaining to libxl_domain_suspend Date: Tue, 13 Jul 2021 20:06:01 +0200 Message-Id: <20210713180605.12096-28-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210713180605.12096-1-olaf@aepfle.de> References: <20210713180605.12096-1-olaf@aepfle.de> MIME-Version: 1.0 The decision to stop+move a domU to the new host must be based on two factors: - the available network bandwidth for the migration stream - the maximum time a workload within a domU can be savely suspended Both values define how many dirty pages a workload may produce prior the final stop+move. The default value of 50 pages is much too low with todays network bandwidths. On an idle 1GiB link these 200K will be transferred within ~2ms. Give the admin a knob to adjust the point when the final stop+move will be done, so he can base this decision on his own needs. This patch adjusts xl(1) and the libxl API. External users check LIBXL_HAVE_DOMAIN_SUSPEND_PROPS for the availibility of the new .min_remaining property. Signed-off-by: Olaf Hering --- docs/man/xl.1.pod.in | 8 ++++++++ tools/include/libxl.h | 1 + tools/libs/light/libxl_dom_save.c | 2 +- tools/libs/light/libxl_domain.c | 1 + tools/libs/light/libxl_internal.h | 1 + tools/xl/xl_cmdtable.c | 23 ++++++++++++----------- tools/xl/xl_migrate.c | 9 ++++++++- 7 files changed, 32 insertions(+), 13 deletions(-) diff --git a/docs/man/xl.1.pod.in b/docs/man/xl.1.pod.in index 9b104720d3..a40f1d4ba0 100644 --- a/docs/man/xl.1.pod.in +++ b/docs/man/xl.1.pod.in @@ -500,6 +500,14 @@ possible to use this option for a 'localhost' migration. Number of copy iterations before final suspend+move (default: 5) +=item B<--min_remaing> I + +Number of remaining dirty pages. If the number of dirty pages drops that +low, the guest is suspended and the domU will finally be moved to I. + +This allows the host admin to control for how long the domU will likely +be suspended during transit. + =back =item B [I] I I diff --git a/tools/include/libxl.h b/tools/include/libxl.h index bf77da0524..28d70b1078 100644 --- a/tools/include/libxl.h +++ b/tools/include/libxl.h @@ -1715,6 +1715,7 @@ static inline int libxl_retrieve_domain_configuration_0x041200( typedef struct { uint32_t flags; /* LIBXL_SUSPEND_* */ uint32_t max_iters; + uint32_t min_remaining; } libxl_domain_suspend_props; #define LIBXL_SUSPEND_DEBUG 1 #define LIBXL_SUSPEND_LIVE 2 diff --git a/tools/libs/light/libxl_dom_save.c b/tools/libs/light/libxl_dom_save.c index 106c08a199..14e197c581 100644 --- a/tools/libs/light/libxl_dom_save.c +++ b/tools/libs/light/libxl_dom_save.c @@ -381,7 +381,7 @@ static int libxl__domain_save_precopy_policy(struct precopy_stats stats, void *u LOGD(DEBUG, shs->domid, "iteration %u dirty_count %ld total_written %lu", stats.iteration, stats.dirty_count, stats.total_written); - if (stats.dirty_count >= 0 && stats.dirty_count < LIBXL_XGS_POLICY_TARGET_DIRTY_COUNT) + if (stats.dirty_count >= 0 && stats.dirty_count < dss->min_remaining) goto stop_copy; if (stats.iteration >= dss->max_iters) goto stop_copy; diff --git a/tools/libs/light/libxl_domain.c b/tools/libs/light/libxl_domain.c index 9f98cd7f2b..06ca7a7df6 100644 --- a/tools/libs/light/libxl_domain.c +++ b/tools/libs/light/libxl_domain.c @@ -528,6 +528,7 @@ int libxl_domain_suspend(libxl_ctx *ctx, uint32_t domid, int fd, dss->fd = fd; dss->type = type; dss->max_iters = props->max_iters ?: LIBXL_XGS_POLICY_MAX_ITERATIONS; + dss->min_remaining = props->min_remaining ?: LIBXL_XGS_POLICY_TARGET_DIRTY_COUNT; dss->live = props->flags & LIBXL_SUSPEND_LIVE; dss->debug = props->flags & LIBXL_SUSPEND_DEBUG; dss->checkpointed_stream = LIBXL_CHECKPOINTED_STREAM_NONE; diff --git a/tools/libs/light/libxl_internal.h b/tools/libs/light/libxl_internal.h index cfb83d80d7..c503fe156c 100644 --- a/tools/libs/light/libxl_internal.h +++ b/tools/libs/light/libxl_internal.h @@ -3649,6 +3649,7 @@ struct libxl__domain_save_state { int debug; int checkpointed_stream; uint32_t max_iters; + uint32_t min_remaining; const libxl_domain_remus_info *remus; /* private */ int rc; diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c index 678b236d76..e10b580261 100644 --- a/tools/xl/xl_cmdtable.c +++ b/tools/xl/xl_cmdtable.c @@ -165,17 +165,18 @@ const struct cmd_spec cmd_table[] = { &main_migrate, 0, 1, "Migrate a domain to another host", "[options] ", - "-h Print this help.\n" - "-C Send instead of config file from creation.\n" - "-s Use instead of ssh. String will be passed\n" - " to sh. If empty, run instead of ssh xl\n" - " migrate-receive [-d -e]\n" - "-e Do not wait in the background (on ) for the death\n" - " of the domain.\n" - "--debug Enable verification mode.\n" - "-p Do not unpause domain after migrating it.\n" - "-D Preserve the domain id\n" - "--max_iters N Number of copy iterations before final stop+move" + "-h Print this help.\n" + "-C Send instead of config file from creation.\n" + "-s Use instead of ssh. String will be passed\n" + " to sh. If empty, run instead of ssh xl\n" + " migrate-receive [-d -e]\n" + "-e Do not wait in the background (on ) for the death\n" + " of the domain.\n" + "--debug Enable verification mode.\n" + "-p Do not unpause domain after migrating it.\n" + "-D Preserve the domain id\n" + "--max_iters N Number of copy iterations before final stop+move\n" + "--min_remaining N Number of remaining dirty pages before final stop+move" }, { "restore", &main_restore, 0, 1, diff --git a/tools/xl/xl_migrate.c b/tools/xl/xl_migrate.c index af117d4d56..14feb2b7ec 100644 --- a/tools/xl/xl_migrate.c +++ b/tools/xl/xl_migrate.c @@ -179,6 +179,7 @@ static void migrate_do_preamble(int send_fd, int recv_fd, pid_t child, static void migrate_domain(uint32_t domid, int preserve_domid, const char *rune, int debug, uint32_t max_iters, + uint32_t min_remaining, const char *override_config_file) { pid_t child = -1; @@ -191,6 +192,7 @@ static void migrate_domain(uint32_t domid, int preserve_domid, libxl_domain_suspend_props props = { .flags = LIBXL_SUSPEND_LIVE, .max_iters = max_iters, + .min_remaining = min_remaining, }; save_domain_core_begin(domid, preserve_domid, override_config_file, @@ -545,9 +547,11 @@ int main_migrate(int argc, char **argv) int opt, daemonize = 1, monitor = 1, debug = 0, pause_after_migration = 0; int preserve_domid = 0; uint32_t max_iters = 0; + uint32_t min_remaining = 0; static struct option opts[] = { {"debug", 0, 0, 0x100}, {"max_iters", 1, 0, 0x101}, + {"min_remaining", 1, 0, 0x102}, {"live", 0, 0, 0x200}, COMMON_LONG_OPTS }; @@ -578,6 +582,9 @@ int main_migrate(int argc, char **argv) case 0x101: /* --max_iters */ max_iters = atoi(optarg); break; + case 0x102: /* --min_remaining */ + min_remaining = atoi(optarg); + break; case 0x200: /* --live */ /* ignored for compatibility with xm */ break; @@ -613,7 +620,7 @@ int main_migrate(int argc, char **argv) } migrate_domain(domid, preserve_domid, rune, debug, - max_iters, config_filename); + max_iters, min_remaining, config_filename); return EXIT_SUCCESS; } From patchwork Tue Jul 13 18:06:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12374721 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 22434C11F66 for ; Tue, 13 Jul 2021 18:16:05 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B27656136D for ; Tue, 13 Jul 2021 18:16:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B27656136D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.155599.287326 (Exim 4.92) (envelope-from ) id 1m3Mwj-0007Gd-Vj; Tue, 13 Jul 2021 18:15:53 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 155599.287326; Tue, 13 Jul 2021 18:15:53 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3Mwj-0007GO-Rh; Tue, 13 Jul 2021 18:15:53 +0000 Received: by outflank-mailman (input) for mailman id 155599; Tue, 13 Jul 2021 18:15:52 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3Moq-0002UL-Ip for xen-devel@lists.xenproject.org; Tue, 13 Jul 2021 18:07:44 +0000 Received: from mo4-p03-ob.smtp.rzone.de (unknown [81.169.146.173]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 09db5836-ef13-4b8c-9624-2d4ead1701d9; Tue, 13 Jul 2021 18:06:32 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.28.1 AUTH) with ESMTPSA id 30791cx6DI6NtBA (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Tue, 13 Jul 2021 20:06:23 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 09db5836-ef13-4b8c-9624-2d4ead1701d9 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1626199583; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=qKsJRazTHUTD2Ybtcf5xNrQhUiwz4suhpfKf90n01qY=; b=MifQ4DoZPVgmf+LZCCmY1rG3HBSr6oI+WcjM6O3Uh7+rtMobbpLmRz4tgFEtwJy2B8 f1UE4OWSAFcj2GIJJTzKb8WQEQbjKbzpxYtUXrSDL8cy4VEZIHIURUy75wDNsKMEhUFy 9pPHbcqjEaLYEI9xcP8B9FQJGb7zy/p7DGdRtuDThAVaDEqaymCIur+xDRKxXLjOZqCi lNggFMT0jjwOaFYWnQbXlvKYATYv3W6t2Gx2KUQ9Mw4TwpE6keh1reGgdePj6+kzB2M3 uNENFYtOClJZD3ULdjIxoN+gL/nRvf8FgJctLAfntwgmNVjcn8Si/0hwJu6TnTwlNCc9 TNqw== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAgthqlagRsstpH9XYEN5v9xf7dypFU9ge5p0Oi0V" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu , Anthony PERARD , Juergen Gross Subject: [PATCH v20210713 28/31] tools: add --abort_if_busy to libxl_domain_suspend Date: Tue, 13 Jul 2021 20:06:02 +0200 Message-Id: <20210713180605.12096-29-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210713180605.12096-1-olaf@aepfle.de> References: <20210713180605.12096-1-olaf@aepfle.de> MIME-Version: 1.0 Provide a knob to the host admin to abort the live migration of a running domU if the downtime during final transit will be too long for the workload within domU. Adjust error reporting. Add ERROR_MIGRATION_ABORTED to allow callers of libxl_domain_suspend to distinguish between errors and the requested constraint. Adjust precopy_policy to simplify reporting of remaining dirty pages. The loop in send_memory_live populates ->dirty_count in a different place than ->iteration. Let it proceeed one more time to provide the desired information before leaving the loop. This patch adjusts xl(1) and the libxl API. External users check LIBXL_HAVE_DOMAIN_SUSPEND_PROPS for the availibility of the new .abort_if_busy property. Signed-off-by: Olaf Hering --- docs/man/xl.1.pod.in | 8 +++++++ tools/include/libxl.h | 1 + tools/libs/light/libxl_dom_save.c | 7 ++++++- tools/libs/light/libxl_domain.c | 1 + tools/libs/light/libxl_internal.h | 2 ++ tools/libs/light/libxl_stream_write.c | 9 +++++++- tools/libs/light/libxl_types.idl | 1 + tools/xl/xl_cmdtable.c | 6 +++++- tools/xl/xl_migrate.c | 30 ++++++++++++++++++++------- 9 files changed, 55 insertions(+), 10 deletions(-) diff --git a/docs/man/xl.1.pod.in b/docs/man/xl.1.pod.in index a40f1d4ba0..c49d595f5e 100644 --- a/docs/man/xl.1.pod.in +++ b/docs/man/xl.1.pod.in @@ -508,6 +508,14 @@ low, the guest is suspended and the domU will finally be moved to I. This allows the host admin to control for how long the domU will likely be suspended during transit. +=item B<--abort_if_busy> + +Abort migration instead of doing final suspend/move/resume if the +guest produced more than I dirty pages during th number +of I iterations. +This avoids long periods of time where the guest is suspended, which +may confuse the workload within domU. + =back =item B [I] I I diff --git a/tools/include/libxl.h b/tools/include/libxl.h index 28d70b1078..cc056ed627 100644 --- a/tools/include/libxl.h +++ b/tools/include/libxl.h @@ -1719,6 +1719,7 @@ typedef struct { } libxl_domain_suspend_props; #define LIBXL_SUSPEND_DEBUG 1 #define LIBXL_SUSPEND_LIVE 2 +#define LIBXL_SUSPEND_ABORT_IF_BUSY 4 int libxl_domain_suspend(libxl_ctx *ctx, uint32_t domid, int fd, libxl_domain_suspend_props *props, diff --git a/tools/libs/light/libxl_dom_save.c b/tools/libs/light/libxl_dom_save.c index 14e197c581..467bbc44f1 100644 --- a/tools/libs/light/libxl_dom_save.c +++ b/tools/libs/light/libxl_dom_save.c @@ -383,11 +383,16 @@ static int libxl__domain_save_precopy_policy(struct precopy_stats stats, void *u stats.iteration, stats.dirty_count, stats.total_written); if (stats.dirty_count >= 0 && stats.dirty_count < dss->min_remaining) goto stop_copy; - if (stats.iteration >= dss->max_iters) + if (stats.dirty_count >= 0 && stats.iteration >= dss->max_iters) goto stop_copy; return XGS_POLICY_CONTINUE_PRECOPY; stop_copy: + if (dss->abort_if_busy) + { + dss->remaining_dirty_pages = stats.dirty_count; + return XGS_POLICY_ABORT; + } return XGS_POLICY_STOP_AND_COPY; } diff --git a/tools/libs/light/libxl_domain.c b/tools/libs/light/libxl_domain.c index 06ca7a7df6..e4740b063e 100644 --- a/tools/libs/light/libxl_domain.c +++ b/tools/libs/light/libxl_domain.c @@ -529,6 +529,7 @@ int libxl_domain_suspend(libxl_ctx *ctx, uint32_t domid, int fd, dss->type = type; dss->max_iters = props->max_iters ?: LIBXL_XGS_POLICY_MAX_ITERATIONS; dss->min_remaining = props->min_remaining ?: LIBXL_XGS_POLICY_TARGET_DIRTY_COUNT; + dss->abort_if_busy = props->flags & LIBXL_SUSPEND_ABORT_IF_BUSY; dss->live = props->flags & LIBXL_SUSPEND_LIVE; dss->debug = props->flags & LIBXL_SUSPEND_DEBUG; dss->checkpointed_stream = LIBXL_CHECKPOINTED_STREAM_NONE; diff --git a/tools/libs/light/libxl_internal.h b/tools/libs/light/libxl_internal.h index c503fe156c..862976b2b2 100644 --- a/tools/libs/light/libxl_internal.h +++ b/tools/libs/light/libxl_internal.h @@ -3647,9 +3647,11 @@ struct libxl__domain_save_state { libxl_domain_type type; int live; int debug; + int abort_if_busy; int checkpointed_stream; uint32_t max_iters; uint32_t min_remaining; + long remaining_dirty_pages; const libxl_domain_remus_info *remus; /* private */ int rc; diff --git a/tools/libs/light/libxl_stream_write.c b/tools/libs/light/libxl_stream_write.c index 634f3240d1..1ab3943f3e 100644 --- a/tools/libs/light/libxl_stream_write.c +++ b/tools/libs/light/libxl_stream_write.c @@ -344,11 +344,18 @@ void libxl__xc_domain_save_done(libxl__egc *egc, void *dss_void, goto err; if (retval) { + if (dss->remaining_dirty_pages) { + LOGD(NOTICE, dss->domid, "saving domain: aborted," + " %ld remaining dirty pages.", dss->remaining_dirty_pages); + } else { LOGEVD(ERROR, errnoval, dss->domid, "saving domain: %s", dss->dsps.guest_responded ? "domain responded to suspend request" : "domain did not respond to suspend request"); - if (!dss->dsps.guest_responded) + } + if (dss->remaining_dirty_pages) + rc = ERROR_MIGRATION_ABORTED; + else if(!dss->dsps.guest_responded) rc = ERROR_GUEST_TIMEDOUT; else if (dss->rc) rc = dss->rc; diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl index f45adddab0..b91769ee10 100644 --- a/tools/libs/light/libxl_types.idl +++ b/tools/libs/light/libxl_types.idl @@ -76,6 +76,7 @@ libxl_error = Enumeration("error", [ (-30, "QMP_DEVICE_NOT_ACTIVE"), # a device has failed to be become active (-31, "QMP_DEVICE_NOT_FOUND"), # the requested device has not been found (-32, "QEMU_API"), # QEMU's replies don't contains expected members + (-33, "MIGRATION_ABORTED"), ], value_namespace = "") libxl_domain_type = Enumeration("domain_type", [ diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c index e10b580261..b5225a5f1b 100644 --- a/tools/xl/xl_cmdtable.c +++ b/tools/xl/xl_cmdtable.c @@ -176,7 +176,11 @@ const struct cmd_spec cmd_table[] = { "-p Do not unpause domain after migrating it.\n" "-D Preserve the domain id\n" "--max_iters N Number of copy iterations before final stop+move\n" - "--min_remaining N Number of remaining dirty pages before final stop+move" + "--min_remaining N Number of remaining dirty pages before final stop+move\n" + "--abort_if_busy Abort migration instead of doing final stop+move,\n" + " if the number of dirty pages is higher than \n" + " after iterations. Otherwise the amount of memory\n" + " to be transfered would exceed maximum allowed domU downtime." }, { "restore", &main_restore, 0, 1, diff --git a/tools/xl/xl_migrate.c b/tools/xl/xl_migrate.c index 14feb2b7ec..f523746e5b 100644 --- a/tools/xl/xl_migrate.c +++ b/tools/xl/xl_migrate.c @@ -177,7 +177,7 @@ static void migrate_do_preamble(int send_fd, int recv_fd, pid_t child, } static void migrate_domain(uint32_t domid, int preserve_domid, - const char *rune, int debug, + const char *rune, int debug, int abort_if_busy, uint32_t max_iters, uint32_t min_remaining, const char *override_config_file) @@ -213,14 +213,20 @@ static void migrate_domain(uint32_t domid, int preserve_domid, if (debug) props.flags |= LIBXL_SUSPEND_DEBUG; + if (abort_if_busy) + props.flags |= LIBXL_SUSPEND_ABORT_IF_BUSY; rc = libxl_domain_suspend(ctx, domid, send_fd, &props, NULL); if (rc) { fprintf(stderr, "migration sender: libxl_domain_suspend failed" " (rc=%d)\n", rc); - if (rc == ERROR_GUEST_TIMEDOUT) - goto failed_suspend; - else - goto failed_resume; + switch (rc) { + case ERROR_GUEST_TIMEDOUT: + goto failed_suspend; + case ERROR_MIGRATION_ABORTED: + goto failed_busy; + default: + goto failed_resume; + } } //fprintf(stderr, "migration sender: Transfer complete.\n"); @@ -302,6 +308,12 @@ static void migrate_domain(uint32_t domid, int preserve_domid, fprintf(stderr, "Migration failed, failed to suspend at sender.\n"); exit(EXIT_FAILURE); + failed_busy: + close(send_fd); + migration_child_report(recv_fd); + fprintf(stderr, "Migration aborted as requested, domain is too busy.\n"); + exit(EXIT_FAILURE); + failed_resume: close(send_fd); migration_child_report(recv_fd); @@ -545,13 +557,14 @@ int main_migrate(int argc, char **argv) char *rune = NULL; char *host; int opt, daemonize = 1, monitor = 1, debug = 0, pause_after_migration = 0; - int preserve_domid = 0; + int preserve_domid = 0, abort_if_busy = 0; uint32_t max_iters = 0; uint32_t min_remaining = 0; static struct option opts[] = { {"debug", 0, 0, 0x100}, {"max_iters", 1, 0, 0x101}, {"min_remaining", 1, 0, 0x102}, + {"abort_if_busy", 0, 0, 0x103}, {"live", 0, 0, 0x200}, COMMON_LONG_OPTS }; @@ -585,6 +598,9 @@ int main_migrate(int argc, char **argv) case 0x102: /* --min_remaining */ min_remaining = atoi(optarg); break; + case 0x103: /* --abort_if_busy */ + abort_if_busy = 1; + break; case 0x200: /* --live */ /* ignored for compatibility with xm */ break; @@ -619,7 +635,7 @@ int main_migrate(int argc, char **argv) pause_after_migration ? " -p" : ""); } - migrate_domain(domid, preserve_domid, rune, debug, + migrate_domain(domid, preserve_domid, rune, debug, abort_if_busy, max_iters, min_remaining, config_filename); return EXIT_SUCCESS; } From patchwork Tue Jul 13 18:06:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12374735 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C3B50C07E96 for ; Tue, 13 Jul 2021 18:17:20 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 91AA76136D for ; Tue, 13 Jul 2021 18:17:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 91AA76136D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.155669.287413 (Exim 4.92) (envelope-from ) id 1m3Mxz-0003hg-Mu; Tue, 13 Jul 2021 18:17:11 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 155669.287413; Tue, 13 Jul 2021 18:17:11 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3Mxz-0003hX-Jg; Tue, 13 Jul 2021 18:17:11 +0000 Received: by outflank-mailman (input) for mailman id 155669; Tue, 13 Jul 2021 18:17:09 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3MoU-0002kr-V7 for xen-devel@lists.xenproject.org; Tue, 13 Jul 2021 18:07:22 +0000 Received: from mo4-p03-ob.smtp.rzone.de (unknown [81.169.146.175]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 0bea7630-e405-11eb-8793-12813bfff9fa; Tue, 13 Jul 2021 18:06:29 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.28.1 AUTH) with ESMTPSA id 30791cx6DI6NtBB (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Tue, 13 Jul 2021 20:06:23 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 0bea7630-e405-11eb-8793-12813bfff9fa DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1626199584; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=JrGxNpYov5voRDtMV5w4X2CbyE19s8jrygFtsIfZxys=; b=dqjaMd1F8I7hTN9jlHVKhrUl1RimwUM5FkZ/HrMgw2O9/o9nPFeQIRAvTK3NJB2UxS 1iQbMgtVB66RRjrQQ/eopanr+m8VLVVBKcnuKfLOwoSvaTYBp3hlQ+cjS+NETGmKjrDX +DOEexmLpBJHCJSy5nd1pJiDetjG8qx5JM9bVmg8sZGICKgkCh5J0s3YGE7awkNK47jb mZqJ68kCoKtUzPPCrWPzAWRFfxwoA3UF3LeeqbGoILqjThjSSILUxEe2Wco34lFijEj/ RlSNFtmhun3avVAe0NlYr5yoF8PQRNefs+rwHzSMsbnnpBRnCw8sv0n8b5RsCmBz0Hkj o9LQ== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAgthqlagRsstpH9XYEN5v9xf7dypFU9ge5p0Oi0V" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu , Juergen Gross Subject: [PATCH v20210713 29/31] tools: add API for expandable bitmaps Date: Tue, 13 Jul 2021 20:06:03 +0200 Message-Id: <20210713180605.12096-30-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210713180605.12096-1-olaf@aepfle.de> References: <20210713180605.12096-1-olaf@aepfle.de> MIME-Version: 1.0 Since the incoming migration stream lacks info about what the highest pfn will be, some data structures can not be allocated upfront. Add an API for expandable bitmaps, loosely based on pfn_set_populated. Signed-off-by: Olaf Hering --- tools/libs/guest/xg_sr_common.c | 39 +++++++++++++++++++ tools/libs/guest/xg_sr_common.h | 67 +++++++++++++++++++++++++++++++++ 2 files changed, 106 insertions(+) diff --git a/tools/libs/guest/xg_sr_common.c b/tools/libs/guest/xg_sr_common.c index cabde4ef74..6d495ba72d 100644 --- a/tools/libs/guest/xg_sr_common.c +++ b/tools/libs/guest/xg_sr_common.c @@ -163,6 +163,45 @@ static void __attribute__((unused)) build_assertions(void) BUILD_BUG_ON(sizeof(struct xc_sr_rec_hvm_params) != 8); } +/* + * Expand the tracking structures as needed. + * To avoid realloc()ing too excessively, the size increased to the nearest + * power of two large enough to contain the required number of bits. + */ +bool _sr_bitmap_expand(struct sr_bitmap *bm, unsigned long bits) +{ + size_t new_max; + size_t old_sz, new_sz; + void *p; + + if (bits <= bm->bits) + return true; + + /* Round up to the nearest power of two larger than bit, less 1. */ + new_max = bits; + new_max |= new_max >> 1; + new_max |= new_max >> 2; + new_max |= new_max >> 4; + new_max |= new_max >> 8; + new_max |= new_max >> 16; + new_max |= sizeof(unsigned long) > 4 ? new_max >> 32 : 0; + + /* Allocate units of unsigned long */ + new_max = (new_max + BITS_PER_LONG - 1) & ~(BITS_PER_LONG - 1); + + old_sz = bitmap_size(bm->bits); + new_sz = bitmap_size(new_max); + p = realloc(bm->p, new_sz); + if (!p) + return false; + + memset(p + old_sz, 0, new_sz - old_sz); + bm->p = p; + bm->bits = new_max; + + return true; +} + /* * Local variables: * mode: C diff --git a/tools/libs/guest/xg_sr_common.h b/tools/libs/guest/xg_sr_common.h index 8981c21acd..686a9614c2 100644 --- a/tools/libs/guest/xg_sr_common.h +++ b/tools/libs/guest/xg_sr_common.h @@ -18,6 +18,73 @@ const char *rec_type_to_str(uint32_t type); struct xc_sr_context; struct xc_sr_record; +struct sr_bitmap +{ + void *p; + unsigned long bits; +}; + +extern bool _sr_bitmap_expand(struct sr_bitmap *bm, unsigned long bits); + +static inline bool sr_bitmap_expand(struct sr_bitmap *bm, unsigned long bits) +{ + if (bits > bm->bits) + return _sr_bitmap_expand(bm, bits); + return true; +} + +static inline void sr_bitmap_free(struct sr_bitmap *bm) +{ + free(bm->p); + bm->p = NULL; +} + +static inline bool sr_set_bit(unsigned long bit, struct sr_bitmap *bm) +{ + if (sr_bitmap_expand(bm, bit) == false) + return false; + + set_bit(bit, bm->p); + return true; +} + +static inline bool sr_test_bit(unsigned long bit, struct sr_bitmap *bm) +{ + if (bit > bm->bits) + return false; + return !!test_bit(bit, bm->p); +} + +static inline void sr_clear_bit(unsigned long bit, struct sr_bitmap *bm) +{ + if (bit <= bm->bits) + clear_bit(bit, bm->p); +} + +static inline bool sr_test_and_clear_bit(unsigned long bit, struct sr_bitmap *bm) +{ + if (bit > bm->bits) + return false; + return !!test_and_clear_bit(bit, bm->p); +} + +/* No way to report potential allocation error, bitmap must be expanded prior usage */ +static inline bool sr_test_and_set_bit(unsigned long bit, struct sr_bitmap *bm) +{ + if (bit > bm->bits) + return false; + return !!test_and_set_bit(bit, bm->p); +} + +static inline bool sr_set_long_bit(unsigned long base_bit, struct sr_bitmap *bm) +{ + if (sr_bitmap_expand(bm, base_bit + BITS_PER_LONG) == false) + return false; + + set_bit_long(base_bit, bm->p); + return true; +} + /** * Save operations. To be implemented for each type of guest, for use by the * common save algorithm. From patchwork Tue Jul 13 18:06:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12374741 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B864DC07E95 for ; Tue, 13 Jul 2021 18:17:54 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7F7906044F for ; Tue, 13 Jul 2021 18:17:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7F7906044F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.155695.287457 (Exim 4.92) (envelope-from ) id 1m3MyY-0005wY-W5; Tue, 13 Jul 2021 18:17:47 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 155695.287457; Tue, 13 Jul 2021 18:17:46 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3MyY-0005wP-SU; Tue, 13 Jul 2021 18:17:46 +0000 Received: by outflank-mailman (input) for mailman id 155695; Tue, 13 Jul 2021 18:17:45 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3MoR-0002UL-Hr for xen-devel@lists.xenproject.org; Tue, 13 Jul 2021 18:07:19 +0000 Received: from mo4-p03-ob.smtp.rzone.de (unknown [85.215.255.101]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 920c1936-b06e-4238-bb74-3dc166dee162; Tue, 13 Jul 2021 18:06:30 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.28.1 AUTH) with ESMTPSA id 30791cx6DI6OtBC (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Tue, 13 Jul 2021 20:06:24 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 920c1936-b06e-4238-bb74-3dc166dee162 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1626199584; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=QjAnxC7dPLjpKo0oeaxvn6b/YLOTQAxI2SNhM+rtNAo=; b=mVKfBMVDjEa8wL52rmtOvPqoWZYC3ggCeGbWsNwZoho6RBC1RutG+D1KJ0eU3unRqy kKHKGtgIp29GIDTO29z2ZtWTw3ELjdDX2abecJP44XNH+s/3MoMa2+dn+CdEPBygF5TL woxm1gtsLPlrZJQafuW5uvq9t5bqyLawnH0pbcdBg9RMspufeHIPqkA8pwGTMFKezGt5 h346NtLHjwNpVs+Iri59vSIATU0FhxyOEDRFGKxsiM43uy3Pq40NCoxY+cg0hnX5DY/3 zt4FRklFB2pouJKyxlOQumW1ddEcqnhWWI6bfLpqsoknMW9aXmj2EJicmFqDt9nBWA3u oL+Q== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAgthqlagRsstpH9XYEN5v9xf7dypFU9ge5p0Oi0V" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu , Juergen Gross Subject: [PATCH v20210713 30/31] tools: use xg_sr_bitmap for populated_pfns Date: Tue, 13 Jul 2021 20:06:04 +0200 Message-Id: <20210713180605.12096-31-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210713180605.12096-1-olaf@aepfle.de> References: <20210713180605.12096-1-olaf@aepfle.de> MIME-Version: 1.0 Signed-off-by: Olaf Hering --- tools/libs/guest/xg_sr_common.h | 20 ++++++- tools/libs/guest/xg_sr_restore.c | 69 ------------------------ tools/libs/guest/xg_sr_restore_x86_hvm.c | 9 ++++ tools/libs/guest/xg_sr_restore_x86_pv.c | 7 +++ 4 files changed, 34 insertions(+), 71 deletions(-) diff --git a/tools/libs/guest/xg_sr_common.h b/tools/libs/guest/xg_sr_common.h index 686a9614c2..2ae5c74d23 100644 --- a/tools/libs/guest/xg_sr_common.h +++ b/tools/libs/guest/xg_sr_common.h @@ -375,8 +375,7 @@ struct xc_sr_context uint32_t xenstore_domid, console_domid; /* Bitmap of currently populated PFNs during restore. */ - unsigned long *populated_pfns; - xen_pfn_t max_populated_pfn; + struct sr_bitmap populated_pfns; /* Sender has invoked verify mode on the stream. */ bool verify; @@ -632,6 +631,23 @@ static inline bool page_type_has_stream_data(uint32_t type) } } +static inline bool pfn_is_populated(struct xc_sr_context *ctx, xen_pfn_t pfn) +{ + return sr_test_bit(pfn, &ctx->restore.populated_pfns); +} + +static inline int pfn_set_populated(struct xc_sr_context *ctx, xen_pfn_t pfn) +{ + xc_interface *xch = ctx->xch; + + if ( sr_set_bit(pfn, &ctx->restore.populated_pfns) == false ) + { + PERROR("Failed to realloc populated_pfns bitmap"); + errno = ENOMEM; + return -1; + } + return 0; +} #endif /* * Local variables: diff --git a/tools/libs/guest/xg_sr_restore.c b/tools/libs/guest/xg_sr_restore.c index d4012bcffd..a897e4bb89 100644 --- a/tools/libs/guest/xg_sr_restore.c +++ b/tools/libs/guest/xg_sr_restore.c @@ -71,64 +71,6 @@ static int read_headers(struct xc_sr_context *ctx) return 0; } -/* - * Is a pfn populated? - */ -static bool pfn_is_populated(const struct xc_sr_context *ctx, xen_pfn_t pfn) -{ - if ( pfn > ctx->restore.max_populated_pfn ) - return false; - return test_bit(pfn, ctx->restore.populated_pfns); -} - -/* - * Set a pfn as populated, expanding the tracking structures if needed. To - * avoid realloc()ing too excessively, the size increased to the nearest power - * of two large enough to contain the required pfn. - */ -static int pfn_set_populated(struct xc_sr_context *ctx, xen_pfn_t pfn) -{ - xc_interface *xch = ctx->xch; - - if ( pfn > ctx->restore.max_populated_pfn ) - { - xen_pfn_t new_max; - size_t old_sz, new_sz; - unsigned long *p; - - /* Round up to the nearest power of two larger than pfn, less 1. */ - new_max = pfn; - new_max |= new_max >> 1; - new_max |= new_max >> 2; - new_max |= new_max >> 4; - new_max |= new_max >> 8; - new_max |= new_max >> 16; -#ifdef __x86_64__ - new_max |= new_max >> 32; -#endif - - old_sz = bitmap_size(ctx->restore.max_populated_pfn + 1); - new_sz = bitmap_size(new_max + 1); - p = realloc(ctx->restore.populated_pfns, new_sz); - if ( !p ) - { - ERROR("Failed to realloc populated bitmap"); - errno = ENOMEM; - return -1; - } - - memset((uint8_t *)p + old_sz, 0x00, new_sz - old_sz); - - ctx->restore.populated_pfns = p; - ctx->restore.max_populated_pfn = new_max; - } - - assert(!test_bit(pfn, ctx->restore.populated_pfns)); - set_bit(pfn, ctx->restore.populated_pfns); - - return 0; -} - /* * Given a set of pfns, obtain memory from Xen to fill the physmap for the * unpopulated subset. If types is NULL, no page type checking is performed @@ -924,16 +866,6 @@ static int setup(struct xc_sr_context *ctx) if ( rc ) goto err; - ctx->restore.max_populated_pfn = (32 * 1024 / 4) - 1; - ctx->restore.populated_pfns = bitmap_alloc( - ctx->restore.max_populated_pfn + 1); - if ( !ctx->restore.populated_pfns ) - { - ERROR("Unable to allocate memory for populated_pfns bitmap"); - rc = -1; - goto err; - } - ctx->restore.pfns = malloc(MAX_BATCH_SIZE * sizeof(*ctx->restore.pfns)); ctx->restore.types = malloc(MAX_BATCH_SIZE * sizeof(*ctx->restore.types)); ctx->restore.mfns = malloc(MAX_BATCH_SIZE * sizeof(*ctx->restore.mfns)); @@ -982,7 +914,6 @@ static void cleanup(struct xc_sr_context *ctx) xch, dirty_bitmap, NRPAGES(bitmap_size(ctx->restore.p2m_size))); free(ctx->restore.buffered_records); - free(ctx->restore.populated_pfns); free(ctx->restore.pages); free(ctx->restore.iov); free(ctx->restore.guest_data); diff --git a/tools/libs/guest/xg_sr_restore_x86_hvm.c b/tools/libs/guest/xg_sr_restore_x86_hvm.c index d6ea6f3012..ac3a9824bb 100644 --- a/tools/libs/guest/xg_sr_restore_x86_hvm.c +++ b/tools/libs/guest/xg_sr_restore_x86_hvm.c @@ -136,6 +136,7 @@ static int x86_hvm_localise_page(struct xc_sr_context *ctx, static int x86_hvm_setup(struct xc_sr_context *ctx) { xc_interface *xch = ctx->xch; + unsigned long max_pfn; if ( ctx->restore.guest_type != DHDR_TYPE_X86_HVM ) { @@ -161,6 +162,13 @@ static int x86_hvm_setup(struct xc_sr_context *ctx) } #endif + max_pfn = max(ctx->restore.p2m_size, ctx->dominfo.max_memkb >> (PAGE_SHIFT-10)); + if ( !sr_bitmap_expand(&ctx->restore.populated_pfns, max_pfn) ) + { + PERROR("Unable to allocate memory for populated_pfns bitmap"); + return -1; + } + return 0; } @@ -241,6 +249,7 @@ static int x86_hvm_stream_complete(struct xc_sr_context *ctx) static int x86_hvm_cleanup(struct xc_sr_context *ctx) { + sr_bitmap_free(&ctx->restore.populated_pfns); free(ctx->x86.hvm.restore.context.ptr); free(ctx->x86.restore.cpuid.ptr); diff --git a/tools/libs/guest/xg_sr_restore_x86_pv.c b/tools/libs/guest/xg_sr_restore_x86_pv.c index dc50b0f5a8..477b6329f7 100644 --- a/tools/libs/guest/xg_sr_restore_x86_pv.c +++ b/tools/libs/guest/xg_sr_restore_x86_pv.c @@ -1060,6 +1060,12 @@ static int x86_pv_setup(struct xc_sr_context *ctx) if ( rc ) return rc; + if ( !sr_bitmap_expand(&ctx->restore.populated_pfns, 32 * 1024 / 4) ) + { + PERROR("Unable to allocate memory for populated_pfns bitmap"); + return -1; + } + ctx->x86.pv.restore.nr_vcpus = ctx->dominfo.max_vcpu_id + 1; ctx->x86.pv.restore.vcpus = calloc(sizeof(struct xc_sr_x86_pv_restore_vcpu), ctx->x86.pv.restore.nr_vcpus); @@ -1153,6 +1159,7 @@ static int x86_pv_stream_complete(struct xc_sr_context *ctx) */ static int x86_pv_cleanup(struct xc_sr_context *ctx) { + sr_bitmap_free(&ctx->restore.populated_pfns); free(ctx->x86.pv.p2m); free(ctx->x86.pv.p2m_pfns); From patchwork Tue Jul 13 18:06:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 12374739 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3587FC07E95 for ; Tue, 13 Jul 2021 18:17:40 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id EBD826136D for ; Tue, 13 Jul 2021 18:17:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EBD826136D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=aepfle.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.155686.287446 (Exim 4.92) (envelope-from ) id 1m3MyK-00059B-Gm; Tue, 13 Jul 2021 18:17:32 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 155686.287446; Tue, 13 Jul 2021 18:17:32 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3MyK-000594-D1; Tue, 13 Jul 2021 18:17:32 +0000 Received: by outflank-mailman (input) for mailman id 155686; Tue, 13 Jul 2021 18:17:31 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1m3Moo-0002kr-Vy for xen-devel@lists.xenproject.org; Tue, 13 Jul 2021 18:07:43 +0000 Received: from mo4-p03-ob.smtp.rzone.de (unknown [85.215.255.102]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 0e6d9fe0-e405-11eb-8793-12813bfff9fa; Tue, 13 Jul 2021 18:06:34 +0000 (UTC) Received: from sender by smtp.strato.de (RZmta 47.28.1 AUTH) with ESMTPSA id 30791cx6DI6OtBD (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Tue, 13 Jul 2021 20:06:24 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 0e6d9fe0-e405-11eb-8793-12813bfff9fa DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1626199585; s=strato-dkim-0002; d=aepfle.de; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=wGKWX5aTtfp07dJARW3c54kyVscPxuckFQpYQ0Sy/fE=; b=kkAtcH82o5J5C+nsMxvglRQY8df4pcJAAYfigNUqTGPWi211etbsLkrJmkvQdqrFNx cOlTuFNgimK0L/GKSHDUsUAtQ380WkRCr18ulf000qJl8wnZlt2qvJAwGYZ3/Ut8ekD9 QjZcDQdefrvGugv2z0ApcgM3k44kB4WsL/doIMiyVZxqnqczFN0sAR8+fXigLemGqzSu +1X2mieZjfEEt0sQiOGppdrVewRmjaOj1ghvr6rk+7MBuF7CgTgvkndHpN4budP+C6gD lqnyxLKbfjZwMIIn7hgVdSoJmGRZ6Vw+Oqv8gaS2uQh1RgfJApQu3aTHyG3GyVW3F1Fx zpBQ== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QXkBR9MXjAuzpIG0mv9coXAgthqlagRsstpH9XYEN5v9xf7dypFU9ge5p0Oi0V" X-RZG-CLASS-ID: mo00 From: Olaf Hering To: xen-devel@lists.xenproject.org Cc: Olaf Hering , Ian Jackson , Wei Liu , Juergen Gross Subject: [PATCH v20210713 31/31] tools: use superpages during restore of HVM guest Date: Tue, 13 Jul 2021 20:06:05 +0200 Message-Id: <20210713180605.12096-32-olaf@aepfle.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210713180605.12096-1-olaf@aepfle.de> References: <20210713180605.12096-1-olaf@aepfle.de> MIME-Version: 1.0 During creating of a HVM domU meminit_hvm() tries to map superpages. After save/restore or migration this mapping is lost, everything is allocated in single pages. This causes a performance degradation after migration. Add neccessary code to preallocate a superpage for an incoming chunk of pfns. In case a pfn was not populated on the sending side, it must be freed on the receiving side to avoid over-allocation. The existing code for x86_pv is moved unmodified into its own file. Signed-off-by: Olaf Hering --- tools/libs/guest/xg_dom_x86.c | 5 - tools/libs/guest/xg_private.h | 5 + tools/libs/guest/xg_sr_common.h | 28 +- tools/libs/guest/xg_sr_restore.c | 60 +--- tools/libs/guest/xg_sr_restore_x86_hvm.c | 370 ++++++++++++++++++++++- tools/libs/guest/xg_sr_restore_x86_pv.c | 61 +++- 6 files changed, 456 insertions(+), 73 deletions(-) diff --git a/tools/libs/guest/xg_dom_x86.c b/tools/libs/guest/xg_dom_x86.c index d2eb89ce01..ec0d18fd60 100644 --- a/tools/libs/guest/xg_dom_x86.c +++ b/tools/libs/guest/xg_dom_x86.c @@ -44,11 +44,6 @@ #define SUPERPAGE_BATCH_SIZE 512 -#define SUPERPAGE_2MB_SHIFT 9 -#define SUPERPAGE_2MB_NR_PFNS (1UL << SUPERPAGE_2MB_SHIFT) -#define SUPERPAGE_1GB_SHIFT 18 -#define SUPERPAGE_1GB_NR_PFNS (1UL << SUPERPAGE_1GB_SHIFT) - #define X86_CR0_PE 0x01 #define X86_CR0_ET 0x10 diff --git a/tools/libs/guest/xg_private.h b/tools/libs/guest/xg_private.h index 28441ee13f..b7372e6bd5 100644 --- a/tools/libs/guest/xg_private.h +++ b/tools/libs/guest/xg_private.h @@ -179,4 +179,9 @@ struct xc_cpu_policy { }; #endif /* x86 */ +#define SUPERPAGE_2MB_SHIFT 9 +#define SUPERPAGE_2MB_NR_PFNS (1UL << SUPERPAGE_2MB_SHIFT) +#define SUPERPAGE_1GB_SHIFT 18 +#define SUPERPAGE_1GB_NR_PFNS (1UL << SUPERPAGE_1GB_SHIFT) + #endif /* XG_PRIVATE_H */ diff --git a/tools/libs/guest/xg_sr_common.h b/tools/libs/guest/xg_sr_common.h index 2ae5c74d23..5af0468603 100644 --- a/tools/libs/guest/xg_sr_common.h +++ b/tools/libs/guest/xg_sr_common.h @@ -207,6 +207,16 @@ struct xc_sr_restore_ops */ int (*setup)(struct xc_sr_context *ctx); + /** + * Populate PFNs + * + * Given a set of pfns, obtain memory from Xen to fill the physmap for the + * unpopulated subset. + */ + int (*populate_pfns)(struct xc_sr_context *ctx, unsigned count, + const xen_pfn_t *original_pfns, const uint32_t *types); + + /** * Process an individual record from the stream. The caller shall take * care of processing common records (e.g. END, PAGE_DATA). @@ -338,6 +348,8 @@ struct xc_sr_context int send_back_fd; unsigned long p2m_size; + unsigned long max_pages; + unsigned long tot_pages; xc_hypercall_buffer_t dirty_bitmap_hbuf; /* From Image Header. */ @@ -471,6 +483,14 @@ struct xc_sr_context { /* HVM context blob. */ struct xc_sr_blob context; + + /* Bitmap of currently allocated PFNs during restore. */ + struct sr_bitmap attempted_1g; + struct sr_bitmap attempted_2m; + struct sr_bitmap allocated_pfns; + xen_pfn_t prev_populated_pfn; + xen_pfn_t iteration_tracker_pfn; + unsigned long iteration; } restore; }; } hvm; @@ -535,14 +555,6 @@ int read_record_header(struct xc_sr_context *ctx, int fd, struct xc_sr_rhdr *rhd int read_record_data(struct xc_sr_context *ctx, int fd, struct xc_sr_rhdr *rhdr, struct xc_sr_record *rec); -/* - * This would ideally be private in restore.c, but is needed by - * x86_pv_localise_page() if we receive pagetables frames ahead of the - * contents of the frames they point at. - */ -int populate_pfns(struct xc_sr_context *ctx, unsigned int count, - const xen_pfn_t *original_pfns, const uint32_t *types); - /* Handle a STATIC_DATA_END record. */ int handle_static_data_end(struct xc_sr_context *ctx); diff --git a/tools/libs/guest/xg_sr_restore.c b/tools/libs/guest/xg_sr_restore.c index a897e4bb89..8a97747c38 100644 --- a/tools/libs/guest/xg_sr_restore.c +++ b/tools/libs/guest/xg_sr_restore.c @@ -71,60 +71,6 @@ static int read_headers(struct xc_sr_context *ctx) return 0; } -/* - * Given a set of pfns, obtain memory from Xen to fill the physmap for the - * unpopulated subset. If types is NULL, no page type checking is performed - * and all unpopulated pfns are populated. - */ -int populate_pfns(struct xc_sr_context *ctx, unsigned int count, - const xen_pfn_t *original_pfns, const uint32_t *types) -{ - xc_interface *xch = ctx->xch; - unsigned int i, nr_pfns = 0; - int rc = -1; - - for ( i = 0; i < count; ++i ) - { - if ( (!types || page_type_to_populate(types[i])) && - !pfn_is_populated(ctx, original_pfns[i]) ) - { - rc = pfn_set_populated(ctx, original_pfns[i]); - if ( rc ) - goto err; - ctx->restore.pp_pfns[nr_pfns] = ctx->restore.pp_mfns[nr_pfns] = original_pfns[i]; - ++nr_pfns; - } - } - - if ( nr_pfns ) - { - rc = xc_domain_populate_physmap_exact( - xch, ctx->domid, nr_pfns, 0, 0, ctx->restore.pp_mfns); - if ( rc ) - { - PERROR("Failed to populate physmap"); - goto err; - } - - for ( i = 0; i < nr_pfns; ++i ) - { - if ( ctx->restore.pp_mfns[i] == INVALID_MFN ) - { - ERROR("Populate physmap failed for pfn %u", i); - rc = -1; - goto err; - } - - ctx->restore.ops.set_gfn(ctx, ctx->restore.pp_pfns[i], ctx->restore.pp_mfns[i]); - } - } - - rc = 0; - - err: - return rc; -} - static int handle_static_data_end_v2(struct xc_sr_context *ctx) { int rc = 0; @@ -259,7 +205,8 @@ static int map_guest_pages(struct xc_sr_context *ctx, uint32_t i, p; int rc; - rc = populate_pfns(ctx, pages->count, ctx->restore.pfns, ctx->restore.types); + rc = ctx->restore.ops.populate_pfns(ctx, pages->count, ctx->restore.pfns, + ctx->restore.types); if ( rc ) { ERROR("Failed to populate pfns for batch of %u pages", pages->count); @@ -1091,6 +1038,9 @@ int xc_domain_restore(xc_interface *xch, int io_fd, uint32_t dom, return -1; } + /* See xc_domain_getinfo */ + ctx.restore.max_pages = ctx.dominfo.max_memkb >> (PAGE_SHIFT-10); + ctx.restore.tot_pages = ctx.dominfo.nr_pages; ctx.restore.p2m_size = nr_pfns; ctx.restore.ops = ctx.dominfo.hvm ? restore_ops_x86_hvm : restore_ops_x86_pv; diff --git a/tools/libs/guest/xg_sr_restore_x86_hvm.c b/tools/libs/guest/xg_sr_restore_x86_hvm.c index ac3a9824bb..4d0583c4f7 100644 --- a/tools/libs/guest/xg_sr_restore_x86_hvm.c +++ b/tools/libs/guest/xg_sr_restore_x86_hvm.c @@ -130,6 +130,25 @@ static int x86_hvm_localise_page(struct xc_sr_context *ctx, return 0; } +static bool x86_hvm_expand_sp_bitmaps(struct xc_sr_context *ctx, unsigned long max_pfn) +{ + struct sr_bitmap *bm; + + bm = &ctx->x86.hvm.restore.attempted_1g; + if ( !sr_bitmap_expand(bm, max_pfn >> SUPERPAGE_1GB_SHIFT) ) + return false; + + bm = &ctx->x86.hvm.restore.attempted_2m; + if ( !sr_bitmap_expand(bm, max_pfn >> SUPERPAGE_2MB_SHIFT) ) + return false; + + bm = &ctx->x86.hvm.restore.allocated_pfns; + if ( !sr_bitmap_expand(bm, max_pfn) ) + return false; + + return true; +} + /* * restore_ops function. Confirms the stream matches the domain. */ @@ -164,12 +183,21 @@ static int x86_hvm_setup(struct xc_sr_context *ctx) max_pfn = max(ctx->restore.p2m_size, ctx->dominfo.max_memkb >> (PAGE_SHIFT-10)); if ( !sr_bitmap_expand(&ctx->restore.populated_pfns, max_pfn) ) - { - PERROR("Unable to allocate memory for populated_pfns bitmap"); - return -1; - } + goto out; + + if ( !x86_hvm_expand_sp_bitmaps(ctx, max_pfn) ) + goto out; + + /* FIXME: distinguish between PVH and HVM */ + /* No superpage in 1st 2MB due to VGA hole */ + sr_set_bit(0, &ctx->x86.hvm.restore.attempted_1g); + sr_set_bit(0, &ctx->x86.hvm.restore.attempted_2m); return 0; + +out: + PERROR("Unable to allocate memory for pfn bitmaps"); + return -1; } /* @@ -250,6 +278,9 @@ static int x86_hvm_stream_complete(struct xc_sr_context *ctx) static int x86_hvm_cleanup(struct xc_sr_context *ctx) { sr_bitmap_free(&ctx->restore.populated_pfns); + sr_bitmap_free(&ctx->x86.hvm.restore.attempted_1g); + sr_bitmap_free(&ctx->x86.hvm.restore.attempted_2m); + sr_bitmap_free(&ctx->x86.hvm.restore.allocated_pfns); free(ctx->x86.hvm.restore.context.ptr); free(ctx->x86.restore.cpuid.ptr); @@ -258,6 +289,336 @@ static int x86_hvm_cleanup(struct xc_sr_context *ctx) return 0; } +/* + * Set a range of pfns as allocated + */ +static void pfn_set_long_allocated(struct xc_sr_context *ctx, xen_pfn_t base_pfn) +{ + sr_set_long_bit(base_pfn, &ctx->x86.hvm.restore.allocated_pfns); +} + +static void pfn_set_allocated(struct xc_sr_context *ctx, xen_pfn_t pfn) +{ + sr_set_bit(pfn, &ctx->x86.hvm.restore.allocated_pfns); +} + +struct x86_hvm_sp { + xen_pfn_t pfn; + xen_pfn_t base_pfn; + unsigned long index; + unsigned long count; +}; + +/* + * Try to allocate a 1GB page for this pfn, but avoid Over-allocation. + * If this succeeds, mark the range of 2MB pages as busy. + */ +static bool x86_hvm_alloc_1g(struct xc_sr_context *ctx, struct x86_hvm_sp *sp) +{ + xc_interface *xch = ctx->xch; + unsigned int order; + int i, done; + xen_pfn_t extent; + + /* Only one attempt to avoid overlapping allocation */ + if ( sr_test_and_set_bit(sp->index, &ctx->x86.hvm.restore.attempted_1g) ) + return false; + + order = SUPERPAGE_1GB_SHIFT; + sp->count = SUPERPAGE_1GB_NR_PFNS; + + /* Allocate only if there is room for another superpage */ + if ( ctx->restore.tot_pages + sp->count > ctx->restore.max_pages ) + return false; + + extent = sp->base_pfn = (sp->pfn >> order) << order; + done = xc_domain_populate_physmap(xch, ctx->domid, 1, order, 0, &extent); + if ( done < 0 ) { + PERROR("populate_physmap failed."); + return false; + } + if ( done == 0 ) + return false; + + DPRINTF("1G %" PRI_xen_pfn "\n", sp->base_pfn); + + /* Mark all 2MB pages as done to avoid overlapping allocation */ + for ( i = 0; i < (SUPERPAGE_1GB_NR_PFNS/SUPERPAGE_2MB_NR_PFNS); i++ ) + sr_set_bit((sp->base_pfn >> SUPERPAGE_2MB_SHIFT) + i, &ctx->x86.hvm.restore.attempted_2m); + + return true; +} + +/* Allocate a 2MB page if x86_hvm_alloc_1g failed, avoid Over-allocation. */ +static bool x86_hvm_alloc_2m(struct xc_sr_context *ctx, struct x86_hvm_sp *sp) +{ + xc_interface *xch = ctx->xch; + unsigned int order; + int done; + xen_pfn_t extent; + + /* Only one attempt to avoid overlapping allocation */ + if ( sr_test_and_set_bit(sp->index, &ctx->x86.hvm.restore.attempted_2m) ) + return false; + + order = SUPERPAGE_2MB_SHIFT; + sp->count = SUPERPAGE_2MB_NR_PFNS; + + /* Allocate only if there is room for another superpage */ + if ( ctx->restore.tot_pages + sp->count > ctx->restore.max_pages ) + return false; + + extent = sp->base_pfn = (sp->pfn >> order) << order; + done = xc_domain_populate_physmap(xch, ctx->domid, 1, order, 0, &extent); + if ( done < 0 ) { + PERROR("populate_physmap failed."); + return false; + } + if ( done == 0 ) + return false; + + DPRINTF("2M %" PRI_xen_pfn "\n", sp->base_pfn); + return true; +} + +/* Allocate a single page if x86_hvm_alloc_2m failed. */ +static bool x86_hvm_alloc_4k(struct xc_sr_context *ctx, struct x86_hvm_sp *sp) +{ + xc_interface *xch = ctx->xch; + unsigned int order; + int done; + xen_pfn_t extent; + + order = 0; + sp->count = 1UL; + + /* Allocate only if there is room for another page */ + if ( ctx->restore.tot_pages + sp->count > ctx->restore.max_pages ) { + errno = E2BIG; + return false; + } + + extent = sp->base_pfn = (sp->pfn >> order) << order; + done = xc_domain_populate_physmap(xch, ctx->domid, 1, order, 0, &extent); + if ( done < 0 ) { + PERROR("populate_physmap failed."); + return false; + } + if ( done == 0 ) { + errno = ENOMEM; + return false; + } + + DPRINTF("4K %" PRI_xen_pfn "\n", sp->base_pfn); + return true; +} +/* + * Attempt to allocate a superpage where the pfn resides. + */ +static int x86_hvm_allocate_pfn(struct xc_sr_context *ctx, xen_pfn_t pfn) +{ + bool success; + unsigned long idx_1g, idx_2m; + struct x86_hvm_sp sp = { + .pfn = pfn + }; + + if ( sr_test_bit(pfn, &ctx->x86.hvm.restore.allocated_pfns) ) + return 0; + + idx_1g = pfn >> SUPERPAGE_1GB_SHIFT; + idx_2m = pfn >> SUPERPAGE_2MB_SHIFT; + + sp.index = idx_1g; + success = x86_hvm_alloc_1g(ctx, &sp); + + if ( success == false ) { + sp.index = idx_2m; + success = x86_hvm_alloc_2m(ctx, &sp); + } + + if ( success == false ) { + sp.index = 0; + success = x86_hvm_alloc_4k(ctx, &sp); + } + + if ( success == false ) + return -1; + + do { + if ( sp.count >= BITS_PER_LONG ) { + sp.count -= BITS_PER_LONG; + ctx->restore.tot_pages += BITS_PER_LONG; + pfn_set_long_allocated(ctx, sp.base_pfn + sp.count); + } else { + sp.count--; + ctx->restore.tot_pages++; + pfn_set_allocated(ctx, sp.base_pfn + sp.count); + } + } while ( sp.count ); + + return 0; +} + +/* + * Deallocate memory. + * There was likely an optimistic superpage allocation. + * This means more pages may have been allocated past gap_end. + * This range is not freed now. Incoming higher pfns will release it. + */ +static int x86_hvm_punch_hole(struct xc_sr_context *ctx, + xen_pfn_t gap_start, xen_pfn_t gap_end) +{ + xc_interface *xch = ctx->xch; + xen_pfn_t _pfn, pfn; + uint32_t domid, freed = 0; + int rc; + + pfn = gap_start >> SUPERPAGE_1GB_SHIFT; + do + { + sr_set_bit(pfn, &ctx->x86.hvm.restore.attempted_1g); + } while (++pfn <= gap_end >> SUPERPAGE_1GB_SHIFT); + + pfn = gap_start >> SUPERPAGE_2MB_SHIFT; + do + { + sr_set_bit(pfn, &ctx->x86.hvm.restore.attempted_2m); + } while (++pfn <= gap_end >> SUPERPAGE_2MB_SHIFT); + + pfn = gap_start; + + while ( pfn <= gap_end ) + { + if ( sr_test_and_clear_bit(pfn, &ctx->x86.hvm.restore.allocated_pfns) ) + { + domid = ctx->domid; + _pfn = pfn; + rc = xc_domain_decrease_reservation_exact(xch, domid, 1, 0, &_pfn); + if ( rc ) + { + PERROR("Failed to release pfn %" PRI_xen_pfn, pfn); + return -1; + } + ctx->restore.tot_pages--; + freed++; + } + pfn++; + } + if ( freed ) + DPRINTF("freed %u between %" PRI_xen_pfn " %" PRI_xen_pfn "\n", + freed, gap_start, gap_end); + return 0; +} + +static int x86_hvm_unpopulate_page(struct xc_sr_context *ctx, xen_pfn_t pfn) +{ + sr_clear_bit(pfn, &ctx->restore.populated_pfns); + return x86_hvm_punch_hole(ctx, pfn, pfn); +} + +static int x86_hvm_populate_page(struct xc_sr_context *ctx, xen_pfn_t pfn) +{ + xen_pfn_t gap_start, gap_end; + bool has_gap, first_iteration; + int rc; + + /* + * Check for a gap between the previous populated pfn and this pfn. + * In case a gap exists, it is required to punch a hole to release memory, + * starting after the previous pfn and before this pfn. + * + * But: this can be done only during the first iteration, which is the + * only place there superpage allocations are attempted. All following + * iterations lack the info to properly maintain prev_populated_pfn. + */ + has_gap = ctx->x86.hvm.restore.prev_populated_pfn + 1 < pfn; + first_iteration = ctx->x86.hvm.restore.iteration == 0; + if ( has_gap && first_iteration ) + { + gap_start = ctx->x86.hvm.restore.prev_populated_pfn + 1; + gap_end = pfn - 1; + + rc = x86_hvm_punch_hole(ctx, gap_start, gap_end); + if ( rc ) + goto err; + } + + rc = x86_hvm_allocate_pfn(ctx, pfn); + if ( rc ) + goto err; + pfn_set_populated(ctx, pfn); + ctx->x86.hvm.restore.prev_populated_pfn = pfn; + + rc = 0; +err: + return rc; +} + +/* + * Try to allocate superpages. + * This works without memory map because the pfns arrive in incremental order. + * All pfn numbers and their type are submitted. + * Only pfns with data will have also pfn content transmitted. + */ +static int x86_hvm_populate_pfns(struct xc_sr_context *ctx, unsigned count, + const xen_pfn_t *original_pfns, + const uint32_t *types) +{ + xc_interface *xch = ctx->xch; + xen_pfn_t pfn, min_pfn, max_pfn; + bool to_populate, populated; + unsigned i = count; + int rc = 0; + + min_pfn = count ? original_pfns[0] : 0; + max_pfn = count ? original_pfns[count - 1] : 0; + DPRINTF("batch of %u pfns between %" PRI_xen_pfn " %" PRI_xen_pfn "\n", + count, min_pfn, max_pfn); + + if ( !x86_hvm_expand_sp_bitmaps(ctx, max_pfn) ) + { + ERROR("Unable to allocate memory for pfn bitmaps"); + return -1; + } + + /* + * There is no indicator for a new iteration. + * Simulate it by checking if a lower pfn is coming in. + * In the end it matters only to know if this iteration is the first one. + */ + if ( min_pfn < ctx->x86.hvm.restore.iteration_tracker_pfn ) + ctx->x86.hvm.restore.iteration++; + ctx->x86.hvm.restore.iteration_tracker_pfn = min_pfn; + + for ( i = 0; i < count; ++i ) + { + pfn = original_pfns[i]; + + to_populate = page_type_to_populate(types[i]); + populated = pfn_is_populated(ctx, pfn); + + /* + * page has data, pfn populated: nothing to do + * page has data, pfn not populated: likely never seen before + * page has no data, pfn populated: likely ballooned out during migration + * page has no data, pfn not populated: nothing to do + */ + if ( to_populate && !populated ) + { + rc = x86_hvm_populate_page(ctx, pfn); + } else if ( !to_populate && populated ) + { + rc = x86_hvm_unpopulate_page(ctx, pfn); + } + if ( rc ) + break; + } + + return rc; +} + + struct xc_sr_restore_ops restore_ops_x86_hvm = { .pfn_is_valid = x86_hvm_pfn_is_valid, @@ -266,6 +627,7 @@ struct xc_sr_restore_ops restore_ops_x86_hvm = .set_page_type = x86_hvm_set_page_type, .localise_page = x86_hvm_localise_page, .setup = x86_hvm_setup, + .populate_pfns = x86_hvm_populate_pfns, .process_record = x86_hvm_process_record, .static_data_complete = x86_static_data_complete, .stream_complete = x86_hvm_stream_complete, diff --git a/tools/libs/guest/xg_sr_restore_x86_pv.c b/tools/libs/guest/xg_sr_restore_x86_pv.c index 477b6329f7..0e994d4277 100644 --- a/tools/libs/guest/xg_sr_restore_x86_pv.c +++ b/tools/libs/guest/xg_sr_restore_x86_pv.c @@ -959,6 +959,64 @@ static void x86_pv_set_gfn(struct xc_sr_context *ctx, xen_pfn_t pfn, ((uint32_t *)ctx->x86.pv.p2m)[pfn] = mfn; } +/* + * Given a set of pfns, obtain memory from Xen to fill the physmap for the + * unpopulated subset. If types is NULL, no page type checking is performed + * and all unpopulated pfns are populated. + */ +static int x86_pv_populate_pfns(struct xc_sr_context *ctx, unsigned count, + const xen_pfn_t *original_pfns, + const uint32_t *types) +{ + xc_interface *xch = ctx->xch; + xen_pfn_t *mfns = ctx->restore.pp_mfns, + *pfns = ctx->restore.pp_pfns; + unsigned int i, nr_pfns = 0; + int rc = -1; + + for ( i = 0; i < count; ++i ) + { + if ( (!types || + (types && page_type_has_stream_data(types[i]) == true)) && + !pfn_is_populated(ctx, original_pfns[i]) ) + { + rc = pfn_set_populated(ctx, original_pfns[i]); + if ( rc ) + goto err; + pfns[nr_pfns] = mfns[nr_pfns] = original_pfns[i]; + ++nr_pfns; + } + } + + if ( nr_pfns ) + { + rc = xc_domain_populate_physmap_exact( + xch, ctx->domid, nr_pfns, 0, 0, mfns); + if ( rc ) + { + PERROR("Failed to populate physmap"); + goto err; + } + + for ( i = 0; i < nr_pfns; ++i ) + { + if ( mfns[i] == INVALID_MFN ) + { + ERROR("Populate physmap failed for pfn %u", i); + rc = -1; + goto err; + } + + ctx->restore.ops.set_gfn(ctx, pfns[i], mfns[i]); + } + } + + rc = 0; + + err: + return rc; +} + /* * restore_ops function. Convert pfns back to mfns in pagetables. Possibly * needs to populate new frames if a PTE is found referring to a frame which @@ -1003,7 +1061,7 @@ static int x86_pv_localise_page(struct xc_sr_context *ctx, } } - if ( to_populate && populate_pfns(ctx, to_populate, pfns, NULL) ) + if ( to_populate && x86_pv_populate_pfns(ctx, to_populate, pfns, NULL) ) return -1; for ( i = 0; i < (PAGE_SIZE / sizeof(uint64_t)); ++i ) @@ -1200,6 +1258,7 @@ struct xc_sr_restore_ops restore_ops_x86_pv = .set_gfn = x86_pv_set_gfn, .localise_page = x86_pv_localise_page, .setup = x86_pv_setup, + .populate_pfns = x86_pv_populate_pfns, .process_record = x86_pv_process_record, .static_data_complete = x86_static_data_complete, .stream_complete = x86_pv_stream_complete,