From patchwork Fri Feb 21 06:36:05 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Zhijian X-Patchwork-Id: 13984881 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AA0B9C021AA for ; Fri, 21 Feb 2025 06:36:43 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tlMd3-0001R9-RZ; Fri, 21 Feb 2025 01:35:17 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tlMd0-0001P9-Pu for qemu-devel@nongnu.org; Fri, 21 Feb 2025 01:35:14 -0500 Received: from esa5.hc1455-7.c3s2.iphmx.com ([68.232.139.130]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tlMcx-0003k0-Pp for qemu-devel@nongnu.org; Fri, 21 Feb 2025 01:35:14 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=fujitsu.com; i=@fujitsu.com; q=dns/txt; s=fj2; t=1740119712; x=1771655712; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=OGaTqxGXEo1MTajaJFuUlh9b3n9a82sHYYsh9iM782A=; b=HLTtzCdw9WtKr6zWZ2ePM/lRFLjz6SuoA9DqrK/zKjqJrm0SBtBz9poL J8qvmpq0PW4atHt8dG2qPrPqoKjX//V/uPxEciMzKfUukP+czZne1FRwa IzBMlZ9L/pvhPXqLXgpWiH7cI8iE0ObBPLSfE7tCxqLK6R6GBUHUshpbX 75gURFw8Jtfqc3pHLfB5EHGPVSZz0V+ZDvnnsC1dpYtA5EzGdVMi1H0L9 dh0tbCp3amiXa9w8IIy4fBLKm4n57LF5dA8MkBXUYVwdpBIVAQNgydK2x 0K82TcEtJdWnA01jItJmhHHgKlGYOvX72tnCWk889kBTh5RBEV1H4798j A==; X-CSE-ConnectionGUID: W9uhMt+SScef71MLSZCSIg== X-CSE-MsgGUID: jgc6Xo+NRk2YR84MuLdvWg== X-IronPort-AV: E=McAfee;i="6700,10204,11351"; a="189940639" X-IronPort-AV: E=Sophos;i="6.13,303,1732546800"; d="scan'208";a="189940639" Received: from unknown (HELO oym-r2.gw.nic.fujitsu.com) ([210.162.30.90]) by esa5.hc1455-7.c3s2.iphmx.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Feb 2025 15:35:07 +0900 Received: from oym-m3.gw.nic.fujitsu.com (oym-nat-oym-m3.gw.nic.fujitsu.com [192.168.87.60]) by oym-r2.gw.nic.fujitsu.com (Postfix) with ESMTP id 68287E60B6 for ; Fri, 21 Feb 2025 15:35:04 +0900 (JST) Received: from edo.cn.fujitsu.com (edo.cn.fujitsu.com [10.167.33.5]) by oym-m3.gw.nic.fujitsu.com (Postfix) with ESMTP id 2DE87D562E for ; Fri, 21 Feb 2025 15:35:04 +0900 (JST) Received: from iaas-rdma.. (unknown [10.167.135.44]) by edo.cn.fujitsu.com (Postfix) with ESMTP id 909151A006C; Fri, 21 Feb 2025 14:35:03 +0800 (CST) To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , Laurent Vivier , Paolo Bonzini , Li Zhijian Subject: [PATCH v2 1/8] migration: Prioritize RDMA in ram_save_target_page() Date: Fri, 21 Feb 2025 14:36:05 +0800 Message-ID: <20250221063612.695909-2-lizhijian@fujitsu.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20250221063612.695909-1-lizhijian@fujitsu.com> References: <20250221063612.695909-1-lizhijian@fujitsu.com> MIME-Version: 1.0 Received-SPF: pass client-ip=68.232.139.130; envelope-from=lizhijian@fujitsu.com; helo=esa5.hc1455-7.c3s2.iphmx.com X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-to: Li Zhijian X-Patchwork-Original-From: Li Zhijian via From: Li Zhijian Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Address an error in RDMA-based migration by ensuring RDMA is prioritized when saving pages in `ram_save_target_page()`. Previously, the RDMA protocol's page-saving step was placed after other protocols due to a refactoring in commit bc38dc2f5f3. This led to migration failures characterized by unknown control messages and state loading errors destination: (qemu) qemu-system-x86_64: Unknown control message QEMU FILE qemu-system-x86_64: error while loading state section id 1(ram) qemu-system-x86_64: load of migration failed: Operation not permitted source: (qemu) qemu-system-x86_64: RDMA is in an error state waiting migration to abort! qemu-system-x86_64: failed to save SaveStateEntry with id(name): 1(ram): -1 qemu-system-x86_64: rdma migration: recv polling control error! qemu-system-x86_64: warning: Early error. Sending error. qemu-system-x86_64: warning: rdma migration: send polling control error RDMA migration implemented its own protocol/method to send pages to destination side, hand over to RDMA first to prevent pages being saved by other protocol. Fixes: bc38dc2f5f3 ("migration: refactor ram_save_target_page functions") Signed-off-by: Li Zhijian --- migration/ram.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 589b6505eb2..424df6d9f13 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1964,6 +1964,11 @@ static int ram_save_target_page(RAMState *rs, PageSearchStatus *pss) ram_addr_t offset = ((ram_addr_t)pss->page) << TARGET_PAGE_BITS; int res; + /* Hand over to RDMA first */ + if (control_save_page(pss, offset, &res)) { + return res; + } + if (!migrate_multifd() || migrate_zero_page_detection() == ZERO_PAGE_DETECTION_LEGACY) { if (save_zero_page(rs, pss, offset)) { @@ -1976,10 +1981,6 @@ static int ram_save_target_page(RAMState *rs, PageSearchStatus *pss) return ram_save_multifd_page(block, offset); } - if (control_save_page(pss, offset, &res)) { - return res; - } - return ram_save_page(rs, pss); } From patchwork Fri Feb 21 06:36:06 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Zhijian X-Patchwork-Id: 13984877 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 193FDC021AA for ; Fri, 21 Feb 2025 06:36:29 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tlMd3-0001RB-Rs; Fri, 21 Feb 2025 01:35:17 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tlMd0-0001P6-PI for qemu-devel@nongnu.org; Fri, 21 Feb 2025 01:35:14 -0500 Received: from esa12.hc1455-7.c3s2.iphmx.com ([139.138.37.100]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tlMcx-0003jz-Ja for qemu-devel@nongnu.org; Fri, 21 Feb 2025 01:35:14 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=fujitsu.com; i=@fujitsu.com; q=dns/txt; s=fj2; t=1740119711; x=1771655711; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=f+W7xHv1eRI0J7FNacCobhI1uc+xjuUjaE8S1N4URnY=; b=PoqTxgBP6ysEU7pPYN2TDQnMjT8Wr7H9/ElScgZhYHwGfjSH2IPDNIRM W9A12nusTdcNAxSHtbj1A94NgAwjJjA2yRRPpNqDqms2lLDpjH14TxuUw dhNRGe5BWYD7KwhNLJciR5DoXiQUkTWaUdDPJlGGg/TllZlKwqFHNEuyj 2fSxpdGDFQnUSUBSnf3aTB6dNVAwrHfu0uKtLAZ5fJCrkSljStvOeAGHe wi70MFzANJaxTgYsu7rntNZ7bfrSH1N7+Scz5L7PXeJtjIkYumpbzgDlB sXTMoL02jcfQ8igtoEl70CXlc49J3RrRsfQm29ZmLDIP9YSdFDygfbPXe Q==; X-CSE-ConnectionGUID: BaqAVsWIQ8+0yHxUv/m5KQ== X-CSE-MsgGUID: zXdIBZYPRWO4U38i63W4eA== X-IronPort-AV: E=McAfee;i="6700,10204,11351"; a="169689014" X-IronPort-AV: E=Sophos;i="6.13,303,1732546800"; d="scan'208";a="169689014" Received: from unknown (HELO yto-r3.gw.nic.fujitsu.com) ([218.44.52.219]) by esa12.hc1455-7.c3s2.iphmx.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Feb 2025 15:35:07 +0900 Received: from yto-m2.gw.nic.fujitsu.com (yto-nat-yto-m2.gw.nic.fujitsu.com [192.168.83.65]) by yto-r3.gw.nic.fujitsu.com (Postfix) with ESMTP id E9705E8529 for ; Fri, 21 Feb 2025 15:35:04 +0900 (JST) Received: from edo.cn.fujitsu.com (edo.cn.fujitsu.com [10.167.33.5]) by yto-m2.gw.nic.fujitsu.com (Postfix) with ESMTP id A968DD5B26 for ; Fri, 21 Feb 2025 15:35:04 +0900 (JST) Received: from iaas-rdma.. (unknown [10.167.135.44]) by edo.cn.fujitsu.com (Postfix) with ESMTP id 0B0C81A000B; Fri, 21 Feb 2025 14:35:04 +0800 (CST) To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , Laurent Vivier , Paolo Bonzini , Li Zhijian Subject: [PATCH v2 2/8] migration/rdma: Remove redundant RAM_SAVE_CONTROL_NOT_SUPP check Date: Fri, 21 Feb 2025 14:36:06 +0800 Message-ID: <20250221063612.695909-3-lizhijian@fujitsu.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20250221063612.695909-1-lizhijian@fujitsu.com> References: <20250221063612.695909-1-lizhijian@fujitsu.com> MIME-Version: 1.0 Received-SPF: pass client-ip=139.138.37.100; envelope-from=lizhijian@fujitsu.com; helo=esa12.hc1455-7.c3s2.iphmx.com X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-to: Li Zhijian X-Patchwork-Original-From: Li Zhijian via From: Li Zhijian Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org qemu_rdma_save_page() no longer returns RAM_SAVE_CONTROL_NOT_SUPP since commit a4832d299dd ("migration/rdma: Check sooner if we are in postcopy for save_page()") Signed-off-by: Li Zhijian --- migration/rdma.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/migration/rdma.c b/migration/rdma.c index 76fb0349238..af8e6234a9f 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -3290,8 +3290,7 @@ int rdma_control_save_page(QEMUFile *f, ram_addr_t block_offset, int ret = qemu_rdma_save_page(f, block_offset, offset, size); - if (ret != RAM_SAVE_CONTROL_DELAYED && - ret != RAM_SAVE_CONTROL_NOT_SUPP) { + if (ret != RAM_SAVE_CONTROL_DELAYED) { if (ret < 0) { qemu_file_set_error(f, ret); } From patchwork Fri Feb 21 06:36:07 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Zhijian X-Patchwork-Id: 13984880 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 47EB1C021AA for ; Fri, 21 Feb 2025 06:36:40 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tlMd7-0001UG-R3; Fri, 21 Feb 2025 01:35:21 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tlMd3-0001RA-B4 for qemu-devel@nongnu.org; Fri, 21 Feb 2025 01:35:17 -0500 Received: from esa12.hc1455-7.c3s2.iphmx.com ([139.138.37.100]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tlMd1-0003jz-6b for qemu-devel@nongnu.org; Fri, 21 Feb 2025 01:35:17 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=fujitsu.com; i=@fujitsu.com; q=dns/txt; s=fj2; t=1740119715; x=1771655715; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=23C5J9GjiTBVLzzFhi4s9G3Cy0DKVtnCKHfbmfdFK7M=; b=YW3o+ZSC7bKtSrZYxrCHmu8YWI2Ah3LHPuArC37paSfg+6lZcap3eo7s x87x/5RQ659RHOqI7Pu01chIRKueCZhVkUCq0EtGSpbaGC4LpTy3RspqC pwN9YoFQd20ZIVhvSeQcBw4Het0UiGhrYOueudvbic8qp5P4QOnezCfFW EfV9t0VBkJCcYCgL4mjpRZWGDdI20UNvWSiw7CR25bcTWLkniSl/o9Dty DTzgzDtlV6O7a+Bhuq9PMug/FgR1VPgRvRoVJ3Nd3TTRfqB1UN5VZ9ter c0fXC7JMWHrP/gjcgaHxNilGzuvymKUl0nDKm7cwniywY8X8aQ/bxj3MA Q==; X-CSE-ConnectionGUID: syoDI4VORyyAmE44My2plw== X-CSE-MsgGUID: 8REujhPOSuS22j2CIkhA7w== X-IronPort-AV: E=McAfee;i="6700,10204,11351"; a="169689017" X-IronPort-AV: E=Sophos;i="6.13,303,1732546800"; d="scan'208";a="169689017" Received: from unknown (HELO yto-r1.gw.nic.fujitsu.com) ([218.44.52.217]) by esa12.hc1455-7.c3s2.iphmx.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Feb 2025 15:35:07 +0900 Received: from yto-m3.gw.nic.fujitsu.com (yto-nat-yto-m3.gw.nic.fujitsu.com [192.168.83.66]) by yto-r1.gw.nic.fujitsu.com (Postfix) with ESMTP id 5A81DD6EAD for ; Fri, 21 Feb 2025 15:35:05 +0900 (JST) Received: from edo.cn.fujitsu.com (edo.cn.fujitsu.com [10.167.33.5]) by yto-m3.gw.nic.fujitsu.com (Postfix) with ESMTP id 22AFA17D1C for ; Fri, 21 Feb 2025 15:35:05 +0900 (JST) Received: from iaas-rdma.. (unknown [10.167.135.44]) by edo.cn.fujitsu.com (Postfix) with ESMTP id 83EF71A006C; Fri, 21 Feb 2025 14:35:04 +0800 (CST) To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , Laurent Vivier , Paolo Bonzini , Li Zhijian Subject: [PATCH v2 3/8] migration: Kill RAM_SAVE_CONTROL_NOT_SUPP Date: Fri, 21 Feb 2025 14:36:07 +0800 Message-ID: <20250221063612.695909-4-lizhijian@fujitsu.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20250221063612.695909-1-lizhijian@fujitsu.com> References: <20250221063612.695909-1-lizhijian@fujitsu.com> MIME-Version: 1.0 Received-SPF: pass client-ip=139.138.37.100; envelope-from=lizhijian@fujitsu.com; helo=esa12.hc1455-7.c3s2.iphmx.com X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-to: Li Zhijian X-Patchwork-Original-From: Li Zhijian via From: Li Zhijian Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Refactor the migration control logic by eliminating the `RAM_SAVE_CONTROL_NOT_SUPP` return value within the migration codebase. This involves moving the checks for RDMA migration status and postcopy state from rdma_control_save_page() to control_save_page() With this change, control_save_page() now takes responsibility for determining whether RDMA operations can proceed, based on the state of migration. Signed-off-by: Li Zhijian --- migration/ram.c | 19 ++++++++++--------- migration/rdma.c | 4 +--- migration/rdma.h | 3 +-- 3 files changed, 12 insertions(+), 14 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 424df6d9f13..b7157b9b175 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1155,18 +1155,19 @@ static bool control_save_page(PageSearchStatus *pss, { int ret; - ret = rdma_control_save_page(pss->pss_channel, pss->block->offset, offset, - TARGET_PAGE_SIZE); - if (ret == RAM_SAVE_CONTROL_NOT_SUPP) { - return false; - } + if (migrate_rdma() && !migration_in_postcopy()) { + ret = rdma_control_save_page(pss->pss_channel, pss->block->offset, + offset, TARGET_PAGE_SIZE); - if (ret == RAM_SAVE_CONTROL_DELAYED) { - *pages = 1; + if (ret == RAM_SAVE_CONTROL_DELAYED) { + *pages = 1; + } else { + *pages = ret; + } return true; } - *pages = ret; - return true; + + return false; } /* diff --git a/migration/rdma.c b/migration/rdma.c index af8e6234a9f..c6876347e1e 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -3284,9 +3284,7 @@ err: int rdma_control_save_page(QEMUFile *f, ram_addr_t block_offset, ram_addr_t offset, size_t size) { - if (!migrate_rdma() || migration_in_postcopy()) { - return RAM_SAVE_CONTROL_NOT_SUPP; - } + assert(migrate_rdma()); int ret = qemu_rdma_save_page(f, block_offset, offset, size); diff --git a/migration/rdma.h b/migration/rdma.h index f55f28bbed1..8eeb0117b91 100644 --- a/migration/rdma.h +++ b/migration/rdma.h @@ -33,7 +33,6 @@ void rdma_start_incoming_migration(InetSocketAddress *host_port, Error **errp); #define RAM_CONTROL_ROUND 1 #define RAM_CONTROL_FINISH 3 -#define RAM_SAVE_CONTROL_NOT_SUPP -1000 #define RAM_SAVE_CONTROL_DELAYED -2000 #ifdef CONFIG_RDMA @@ -56,7 +55,7 @@ static inline int rdma_control_save_page(QEMUFile *f, ram_addr_t block_offset, ram_addr_t offset, size_t size) { - return RAM_SAVE_CONTROL_NOT_SUPP; + g_assert_not_reached(); } #endif #endif From patchwork Fri Feb 21 06:36:08 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Zhijian X-Patchwork-Id: 13984879 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 14708C021B3 for ; Fri, 21 Feb 2025 06:36:33 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tlMdA-0001Ux-7E; Fri, 21 Feb 2025 01:35:24 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tlMd3-0001RT-Rz for qemu-devel@nongnu.org; Fri, 21 Feb 2025 01:35:17 -0500 Received: from esa9.hc1455-7.c3s2.iphmx.com ([139.138.36.223]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tlMd1-0003l4-MZ for qemu-devel@nongnu.org; Fri, 21 Feb 2025 01:35:17 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=fujitsu.com; i=@fujitsu.com; q=dns/txt; s=fj2; t=1740119715; x=1771655715; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Aq8SNc7rV72DFQv8V4a5SuEOqjhqhl4aZY+tkt+16uI=; b=MoZzumlSdZB2HV/Kk0f7WgZ0liQAKJ0mJBY3tjkZ+JV5i9Be/7qB5wCu TpN5HAYEi2QRfniy38Dk88EL9r0RBZoxgmOC1AFjeIWw603sWWe0zK8hL P1CN/6WKTzHLQhfnI5E9JRFL/l52BCUAeFcABm7PFnhaisvAxHIV22lJz r5B9KfU42iNKIT/xgYnTZ7UKgW+ekzwDv2hv/to0Fn7wz7wOZQCK6/jFg 69MQyuC3OFoTI+qfZg9JKnHJswgujBD/sFLvL78EpZu+X7oTTTe9RwY/s b9tS1Lff0Sm7iWtAVSmT1yPipFpRmSbj6S9s3hJjZ0DLZ9g01fUsHJ9r4 A==; X-CSE-ConnectionGUID: 35W+rgOgRaC9DXFAZqduzw== X-CSE-MsgGUID: hAFxGNyoSWOpCelzXfzbSg== X-IronPort-AV: E=McAfee;i="6700,10204,11351"; a="179148616" X-IronPort-AV: E=Sophos;i="6.13,303,1732546800"; d="scan'208";a="179148616" Received: from unknown (HELO oym-r2.gw.nic.fujitsu.com) ([210.162.30.90]) by esa9.hc1455-7.c3s2.iphmx.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Feb 2025 15:35:07 +0900 Received: from oym-m3.gw.nic.fujitsu.com (oym-nat-oym-m3.gw.nic.fujitsu.com [192.168.87.60]) by oym-r2.gw.nic.fujitsu.com (Postfix) with ESMTP id C6CDEE60BF for ; Fri, 21 Feb 2025 15:35:05 +0900 (JST) Received: from edo.cn.fujitsu.com (edo.cn.fujitsu.com [10.167.33.5]) by oym-m3.gw.nic.fujitsu.com (Postfix) with ESMTP id 8DA02D9A77 for ; Fri, 21 Feb 2025 15:35:05 +0900 (JST) Received: from iaas-rdma.. (unknown [10.167.135.44]) by edo.cn.fujitsu.com (Postfix) with ESMTP id F0E1B1A000B; Fri, 21 Feb 2025 14:35:04 +0800 (CST) To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , Laurent Vivier , Paolo Bonzini , Li Zhijian Subject: [PATCH v2 4/8] migration: Integrate control_save_page() logic into ram_save_target_page() Date: Fri, 21 Feb 2025 14:36:08 +0800 Message-ID: <20250221063612.695909-5-lizhijian@fujitsu.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20250221063612.695909-1-lizhijian@fujitsu.com> References: <20250221063612.695909-1-lizhijian@fujitsu.com> MIME-Version: 1.0 Received-SPF: pass client-ip=139.138.36.223; envelope-from=lizhijian@fujitsu.com; helo=esa9.hc1455-7.c3s2.iphmx.com X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-to: Li Zhijian X-Patchwork-Original-From: Li Zhijian via From: Li Zhijian Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Refactor the page saving logic by integrating the control_save_page() function directly into ram_save_target_page(). This change consolidates the RDMA migration decision-making process into a single function, enhancing clarity and maintainability. Signed-off-by: Li Zhijian --- migration/ram.c | 35 +++++++---------------------------- 1 file changed, 7 insertions(+), 28 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index b7157b9b175..e07651aee8d 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1143,33 +1143,6 @@ static int save_zero_page(RAMState *rs, PageSearchStatus *pss, return len; } -/* - * @pages: the number of pages written by the control path, - * < 0 - error - * > 0 - number of pages written - * - * Return true if the pages has been saved, otherwise false is returned. - */ -static bool control_save_page(PageSearchStatus *pss, - ram_addr_t offset, int *pages) -{ - int ret; - - if (migrate_rdma() && !migration_in_postcopy()) { - ret = rdma_control_save_page(pss->pss_channel, pss->block->offset, - offset, TARGET_PAGE_SIZE); - - if (ret == RAM_SAVE_CONTROL_DELAYED) { - *pages = 1; - } else { - *pages = ret; - } - return true; - } - - return false; -} - /* * directly send the page to the stream * @@ -1966,7 +1939,13 @@ static int ram_save_target_page(RAMState *rs, PageSearchStatus *pss) int res; /* Hand over to RDMA first */ - if (control_save_page(pss, offset, &res)) { + if (migrate_rdma() && !migration_in_postcopy()) { + res = rdma_control_save_page(pss->pss_channel, pss->block->offset, + offset, TARGET_PAGE_SIZE); + + if (res == RAM_SAVE_CONTROL_DELAYED) { + res = 1; + } return res; } From patchwork Fri Feb 21 06:36:09 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Zhijian X-Patchwork-Id: 13984882 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BDFA5C021AA for ; Fri, 21 Feb 2025 06:36:50 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tlMd5-0001TP-8a; Fri, 21 Feb 2025 01:35:19 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tlMd1-0001QE-3t for qemu-devel@nongnu.org; Fri, 21 Feb 2025 01:35:15 -0500 Received: from esa8.hc1455-7.c3s2.iphmx.com ([139.138.61.253]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tlMcy-0003kN-6v for qemu-devel@nongnu.org; Fri, 21 Feb 2025 01:35:14 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=fujitsu.com; i=@fujitsu.com; q=dns/txt; s=fj2; t=1740119712; x=1771655712; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=k/LqOSlmor1UQwuRbxoZIb/A4xPPUncC4VvDl/x5/zc=; b=MtmXkztnShLN/0Eb4cIxIq7Vwde83ErVa8TIxaW+Dv12HCHq5rXp0pS9 TjkxMXxeXxZwFMMLhkmMdoTExIInwb4D9axOKJQXl+eOxjjdQgQurU2De hObzR+N3PViTiqpyoYt1JITxztkujoUDtZjaKtaJ/2QR9g/p+a5K/feaF 1LCMj5X3ZEQLLIA/jMfefgRJpbYTTGsWsiIBF7skxbtsVcWgUhi/o6KdK /wKKOfU8fmxvEB8cei3CI13n2aXhDgBRTghYRSUyriNzKOj11Sd5hziIc 0Njiqdc88W32blOH5TNOmFokSyv+BFH7P85U8V6wxS7Zn4MKvm79HDAUx g==; X-CSE-ConnectionGUID: s81hMSPbRyq+mfbOKvPtkg== X-CSE-MsgGUID: hUt+Ac9BSQ+r88H12oKC+Q== X-IronPort-AV: E=McAfee;i="6700,10204,11351"; a="178755519" X-IronPort-AV: E=Sophos;i="6.13,303,1732546800"; d="scan'208";a="178755519" Received: from unknown (HELO oym-r2.gw.nic.fujitsu.com) ([210.162.30.90]) by esa8.hc1455-7.c3s2.iphmx.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Feb 2025 15:35:08 +0900 Received: from oym-m3.gw.nic.fujitsu.com (oym-nat-oym-m3.gw.nic.fujitsu.com [192.168.87.60]) by oym-r2.gw.nic.fujitsu.com (Postfix) with ESMTP id 7EB6BE60C5 for ; Fri, 21 Feb 2025 15:35:06 +0900 (JST) Received: from edo.cn.fujitsu.com (edo.cn.fujitsu.com [10.167.33.5]) by oym-m3.gw.nic.fujitsu.com (Postfix) with ESMTP id 4480CD9A77 for ; Fri, 21 Feb 2025 15:35:06 +0900 (JST) Received: from iaas-rdma.. (unknown [10.167.135.44]) by edo.cn.fujitsu.com (Postfix) with ESMTP id 69CDB1A006C; Fri, 21 Feb 2025 14:35:05 +0800 (CST) To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , Laurent Vivier , Paolo Bonzini , Li Zhijian Subject: [PATCH v2 5/8] migration: Add migration_capabilities_and_transport_compatible() helper Date: Fri, 21 Feb 2025 14:36:09 +0800 Message-ID: <20250221063612.695909-6-lizhijian@fujitsu.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20250221063612.695909-1-lizhijian@fujitsu.com> References: <20250221063612.695909-1-lizhijian@fujitsu.com> MIME-Version: 1.0 Received-SPF: pass client-ip=139.138.61.253; envelope-from=lizhijian@fujitsu.com; helo=esa8.hc1455-7.c3s2.iphmx.com X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-to: Li Zhijian X-Patchwork-Original-From: Li Zhijian via From: Li Zhijian Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Similar to migration_channels_and_transport_compatible(), introduce a new helper migration_capabilities_and_transport_compatible() to check if the capabilites is compatible with the transport. Currently, only move the capabilities vs RDMA transport to this function. Signed-off-by: Li Zhijian --- migration/migration.c | 36 ++++++++++++++++++++++++++---------- 1 file changed, 26 insertions(+), 10 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index c597aa707e5..2eacae25e0e 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -238,6 +238,30 @@ migration_channels_and_transport_compatible(MigrationAddress *addr, return true; } +static bool +migration_capabilities_and_transport_compatible(MigrationAddress *addr, + Error **errp) +{ + if (addr->transport == MIGRATION_ADDRESS_TYPE_RDMA) { + if (migrate_xbzrle()) { + error_setg(errp, "RDMA and XBZRLE can't be used together"); + return false; + } + if (migrate_multifd()) { + error_setg(errp, "RDMA and multifd can't be used together"); + return false; + } + } + + return true; +} + +static bool migration_transport_compatible(MigrationAddress *addr, Error **errp) +{ + return migration_channels_and_transport_compatible(addr, errp) && + migration_capabilities_and_transport_compatible(addr, errp); +} + static gint page_request_addr_cmp(gconstpointer ap, gconstpointer bp) { uintptr_t a = (uintptr_t) ap, b = (uintptr_t) bp; @@ -716,7 +740,7 @@ static void qemu_start_incoming_migration(const char *uri, bool has_channels, } /* transport mechanism not suitable for migration? */ - if (!migration_channels_and_transport_compatible(addr, errp)) { + if (!migration_transport_compatible(addr, errp)) { return; } @@ -735,14 +759,6 @@ static void qemu_start_incoming_migration(const char *uri, bool has_channels, } #ifdef CONFIG_RDMA } else if (addr->transport == MIGRATION_ADDRESS_TYPE_RDMA) { - if (migrate_xbzrle()) { - error_setg(errp, "RDMA and XBZRLE can't be used together"); - return; - } - if (migrate_multifd()) { - error_setg(errp, "RDMA and multifd can't be used together"); - return; - } rdma_start_incoming_migration(&addr->u.rdma, errp); #endif } else if (addr->transport == MIGRATION_ADDRESS_TYPE_EXEC) { @@ -2159,7 +2175,7 @@ void qmp_migrate(const char *uri, bool has_channels, } /* transport mechanism not suitable for migration? */ - if (!migration_channels_and_transport_compatible(addr, errp)) { + if (!migration_transport_compatible(addr, errp)) { return; } From patchwork Fri Feb 21 06:36:10 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Zhijian X-Patchwork-Id: 13984878 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 25022C021B4 for ; Fri, 21 Feb 2025 06:36:31 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tlMd6-0001Tk-B4; Fri, 21 Feb 2025 01:35:20 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tlMd3-0001R8-9h for qemu-devel@nongnu.org; Fri, 21 Feb 2025 01:35:17 -0500 Received: from esa5.hc1455-7.c3s2.iphmx.com ([68.232.139.130]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tlMd1-0003k0-9e for qemu-devel@nongnu.org; Fri, 21 Feb 2025 01:35:17 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=fujitsu.com; i=@fujitsu.com; q=dns/txt; s=fj2; t=1740119715; x=1771655715; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=EeqxOxnfJGp1OeklxYaU7QutfMay++/S+GaNNIxlkvY=; b=oVdB4Cd7/b0xRyAp3/fbwncNf5aS3a4O6l24C6b4X7quOBnr/9q6MwvT Ml59Gt0LcMSyeIDYoln/Nq/aHaq1yR+wnyBwG9dO93xqJ8hAdCJw4odmf yc1aHc40U6C1foQBhuQMBGZtS6XPXwpXT7wUbSafC/x1eficEprLUIqVJ 90379Jw3licM+0JizNZHvbSqdTWEfrvHUoUkMTqmNGxHmM7KcB2MbrAgm O4sa292abSOifKBg7EuSgbCHuIuaTh8h00B9wBOa6nsuPJ/k1DxHCh2Hm VGc9P0T6vDwUTx9UB24AEwtcxyqJSpPHv7/zc0eHmSvVo3kY5wq60Z4Bm A==; X-CSE-ConnectionGUID: nOdJyc5MQlSkUphRsTGdIg== X-CSE-MsgGUID: 0YLmBaTuSCCHrfzFOtNqMA== X-IronPort-AV: E=McAfee;i="6700,10204,11351"; a="189940651" X-IronPort-AV: E=Sophos;i="6.13,303,1732546800"; d="scan'208";a="189940651" Received: from unknown (HELO oym-r4.gw.nic.fujitsu.com) ([210.162.30.92]) by esa5.hc1455-7.c3s2.iphmx.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Feb 2025 15:35:09 +0900 Received: from oym-m3.gw.nic.fujitsu.com (oym-nat-oym-m3.gw.nic.fujitsu.com [192.168.87.60]) by oym-r4.gw.nic.fujitsu.com (Postfix) with ESMTP id 008DC7C24E for ; Fri, 21 Feb 2025 15:35:07 +0900 (JST) Received: from edo.cn.fujitsu.com (edo.cn.fujitsu.com [10.167.33.5]) by oym-m3.gw.nic.fujitsu.com (Postfix) with ESMTP id BC204D9A7E for ; Fri, 21 Feb 2025 15:35:06 +0900 (JST) Received: from iaas-rdma.. (unknown [10.167.135.44]) by edo.cn.fujitsu.com (Postfix) with ESMTP id 20C321A000B; Fri, 21 Feb 2025 14:35:06 +0800 (CST) To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , Laurent Vivier , Paolo Bonzini , Li Zhijian Subject: [PATCH v2 6/8] migraion: disable RDMA + postcopy-ram Date: Fri, 21 Feb 2025 14:36:10 +0800 Message-ID: <20250221063612.695909-7-lizhijian@fujitsu.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20250221063612.695909-1-lizhijian@fujitsu.com> References: <20250221063612.695909-1-lizhijian@fujitsu.com> MIME-Version: 1.0 Received-SPF: pass client-ip=68.232.139.130; envelope-from=lizhijian@fujitsu.com; helo=esa5.hc1455-7.c3s2.iphmx.com X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-to: Li Zhijian X-Patchwork-Original-From: Li Zhijian via From: Li Zhijian Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org It's believed that RDMA + postcopy-ram has been broken for a while. Rather than spending time re-enabling it, let's simply disable it as a trade-off. Signed-off-by: Li Zhijian --- migration/migration.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/migration/migration.c b/migration/migration.c index 2eacae25e0e..d414a4b1379 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -251,6 +251,10 @@ migration_capabilities_and_transport_compatible(MigrationAddress *addr, error_setg(errp, "RDMA and multifd can't be used together"); return false; } + if (migrate_postcopy_ram()) { + error_setg(errp, "RDMA and postcopy-ram can't be used together"); + return false; + } } return true; From patchwork Fri Feb 21 06:36:11 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Zhijian X-Patchwork-Id: 13984876 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3034BC021B3 for ; Fri, 21 Feb 2025 06:36:26 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tlMdC-0001VJ-8c; Fri, 21 Feb 2025 01:35:26 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tlMd8-0001UH-9G for qemu-devel@nongnu.org; Fri, 21 Feb 2025 01:35:22 -0500 Received: from esa10.hc1455-7.c3s2.iphmx.com ([139.138.36.225]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tlMd6-0003mC-HY for qemu-devel@nongnu.org; Fri, 21 Feb 2025 01:35:22 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=fujitsu.com; i=@fujitsu.com; q=dns/txt; s=fj2; t=1740119720; x=1771655720; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=uziJ79JcjjLUelIQEISwlLOaz+nkR8/k+C5OgBg9cr8=; b=sHLgSr7yXaxGloEGL1gYosGa25QQA2fIWH1PzHsdKxTNiA40zTTlITvQ pYi+98m95os6G10MxFjijl3Vjl4V9CxP5JB714C0Yv+Y2gi5UsACRpvcK V2LYW/L8/cFYoT6YYIc5pz2nyKXqiwOhS0RpYldbDvaUe41osifDJarNl Uhnb3qILS16OSPwyrByWevZVymdAYMGcoCl666yrRa88aOFEwamijCZx+ V8snFJIYxWtSTggpx/IirKlXoy4R/D9CnDTlUvgDFu64/yOs+dAUiNF4P 77lBunj6lqELhRWEZM7d/PHtWG0kVU4EkDsFjFudZFP1Mjt+tRlxMPKYy Q==; X-CSE-ConnectionGUID: RozmevsRSoadCFnflvfhjQ== X-CSE-MsgGUID: Bgxq6NClRwWEI3cJataq4w== X-IronPort-AV: E=McAfee;i="6700,10204,11351"; a="178123240" X-IronPort-AV: E=Sophos;i="6.13,303,1732546800"; d="scan'208";a="178123240" Received: from unknown (HELO oym-r3.gw.nic.fujitsu.com) ([210.162.30.91]) by esa10.hc1455-7.c3s2.iphmx.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Feb 2025 15:35:09 +0900 Received: from oym-m2.gw.nic.fujitsu.com (oym-nat-oym-m2.gw.nic.fujitsu.com [192.168.87.59]) by oym-r3.gw.nic.fujitsu.com (Postfix) with ESMTP id AB80CC2272 for ; Fri, 21 Feb 2025 15:35:07 +0900 (JST) Received: from edo.cn.fujitsu.com (edo.cn.fujitsu.com [10.167.33.5]) by oym-m2.gw.nic.fujitsu.com (Postfix) with ESMTP id 72630BDA5D for ; Fri, 21 Feb 2025 15:35:07 +0900 (JST) Received: from iaas-rdma.. (unknown [10.167.135.44]) by edo.cn.fujitsu.com (Postfix) with ESMTP id D56201A000B; Fri, 21 Feb 2025 14:35:06 +0800 (CST) To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , Laurent Vivier , Paolo Bonzini , Li Zhijian Subject: [PATCH v2 7/8] migration/rdma: Remove redundant migration_in_postcopy checks Date: Fri, 21 Feb 2025 14:36:11 +0800 Message-ID: <20250221063612.695909-8-lizhijian@fujitsu.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20250221063612.695909-1-lizhijian@fujitsu.com> References: <20250221063612.695909-1-lizhijian@fujitsu.com> MIME-Version: 1.0 Received-SPF: pass client-ip=139.138.36.225; envelope-from=lizhijian@fujitsu.com; helo=esa10.hc1455-7.c3s2.iphmx.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-to: Li Zhijian X-Patchwork-Original-From: Li Zhijian via From: Li Zhijian Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Since we have disabled RDMA + postcopy, it's safe to remove the migration_in_postcopy() that follows the migration_rdma(). Signed-off-by: Li Zhijian --- migration/ram.c | 2 +- migration/rdma.c | 5 +++-- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index e07651aee8d..c363034c882 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1939,7 +1939,7 @@ static int ram_save_target_page(RAMState *rs, PageSearchStatus *pss) int res; /* Hand over to RDMA first */ - if (migrate_rdma() && !migration_in_postcopy()) { + if (migrate_rdma()) { res = rdma_control_save_page(pss->pss_channel, pss->block->offset, offset, TARGET_PAGE_SIZE); diff --git a/migration/rdma.c b/migration/rdma.c index c6876347e1e..0349dd4a8b8 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -3826,7 +3826,7 @@ int rdma_block_notification_handle(QEMUFile *f, const char *name) int rdma_registration_start(QEMUFile *f, uint64_t flags) { - if (!migrate_rdma() || migration_in_postcopy()) { + if (!migrate_rdma()) { return 0; } @@ -3858,7 +3858,8 @@ int rdma_registration_stop(QEMUFile *f, uint64_t flags) RDMAControlHeader head = { .len = 0, .repeat = 1 }; int ret; - if (!migrate_rdma() || migration_in_postcopy()) { + /* Hand over to RDMA first */ + if (!migrate_rdma()) { return 0; } From patchwork Fri Feb 21 06:36:12 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Zhijian X-Patchwork-Id: 13984875 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2C26FC021AA for ; Fri, 21 Feb 2025 06:36:26 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tlMd9-0001UQ-8L; Fri, 21 Feb 2025 01:35:23 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tlMd2-0001Qm-Eh for qemu-devel@nongnu.org; Fri, 21 Feb 2025 01:35:16 -0500 Received: from esa1.hc1455-7.c3s2.iphmx.com ([207.54.90.47]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tlMcz-0003ki-UA for qemu-devel@nongnu.org; Fri, 21 Feb 2025 01:35:16 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=fujitsu.com; i=@fujitsu.com; q=dns/txt; s=fj2; t=1740119714; x=1771655714; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=/RHv9Mp88/2oIuKI/eCTbMWqujLouJj70h/jd8CFmRE=; b=cUjJO2xGoHrzWPNxrC7WKXfAoz6e3yl0UP+zw0YcJOl6tnp5TOJjFwku BMarmSyyO3W1yK+oK4UBtBqk7JH/Kh6S10DgjS82WWGTntUF54epTHfbH Jx3PvwT4ukgKExMXcuwn0D+JyYGE+NZalooHnDubVC6cByaIFMtOAJC/E YrUXFhz8ssqb9j+CPKamVjLkFS8/KTm/2rw1ZvrX482Zh0gpwcrY1nmEO b1w2TqxiPlG+z1A8F7ybJOnXaEZ8M454RFqM5asqgsgkYMF+4lfmBx/BJ YKF4/Bq8xK42GCCq1Ss+QnRrGSemrJNbFM+qjegRCaRl+9HWuQPa8co3d g==; X-CSE-ConnectionGUID: ukQWWWX1RRCEJuyQTH8PUg== X-CSE-MsgGUID: nfUNY9H9SQC1q3yCr+MmnA== X-IronPort-AV: E=McAfee;i="6700,10204,11351"; a="190808398" X-IronPort-AV: E=Sophos;i="6.13,303,1732546800"; d="scan'208";a="190808398" Received: from unknown (HELO oym-r1.gw.nic.fujitsu.com) ([210.162.30.89]) by esa1.hc1455-7.c3s2.iphmx.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Feb 2025 15:35:11 +0900 Received: from oym-m3.gw.nic.fujitsu.com (oym-nat-oym-m3.gw.nic.fujitsu.com [192.168.87.60]) by oym-r1.gw.nic.fujitsu.com (Postfix) with ESMTP id 262B6D480A for ; Fri, 21 Feb 2025 15:35:08 +0900 (JST) Received: from edo.cn.fujitsu.com (edo.cn.fujitsu.com [10.167.33.5]) by oym-m3.gw.nic.fujitsu.com (Postfix) with ESMTP id E093AD9A79 for ; Fri, 21 Feb 2025 15:35:07 +0900 (JST) Received: from iaas-rdma.. (unknown [10.167.135.44]) by edo.cn.fujitsu.com (Postfix) with ESMTP id 4E7B51A006C; Fri, 21 Feb 2025 14:35:07 +0800 (CST) To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , Laurent Vivier , Paolo Bonzini , Li Zhijian Subject: [PATCH v2 8/8] migration: Add qtest for migration over RDMA Date: Fri, 21 Feb 2025 14:36:12 +0800 Message-ID: <20250221063612.695909-9-lizhijian@fujitsu.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20250221063612.695909-1-lizhijian@fujitsu.com> References: <20250221063612.695909-1-lizhijian@fujitsu.com> MIME-Version: 1.0 Received-SPF: pass client-ip=207.54.90.47; envelope-from=lizhijian@fujitsu.com; helo=esa1.hc1455-7.c3s2.iphmx.com X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-to: Li Zhijian X-Patchwork-Original-From: Li Zhijian via From: Li Zhijian Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org This qtest requires there is a RDMA(RoCE) link in the host. In order to make the test work smoothly, introduce a scripts/rdma-migration-helper.sh to - setup a new Soft-RoCE(aka RXE) if it's root - detect existing RoCE link Test will be skipped if there is no available RoCE link. # Start of rdma tests # Running /x86_64/migration/precopy/rdma/plain ok 1 /x86_64/migration/precopy/rdma/plain # SKIP There is no available rdma link to run RDMA migration test. To enable the test: (1) Run 'scripts/rdma-migration-helper.sh setup' with root and rerun the test or (2) Run the test with root privilege # End of rdma tests Signed-off-by: Li Zhijian --- MAINTAINERS | 1 + scripts/rdma-migration-helper.sh | 41 +++++++++++++++++ tests/qtest/migration/precopy-tests.c | 64 +++++++++++++++++++++++++++ 3 files changed, 106 insertions(+) create mode 100755 scripts/rdma-migration-helper.sh diff --git a/MAINTAINERS b/MAINTAINERS index 3848d37a38d..15360fcdc4b 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -3480,6 +3480,7 @@ R: Li Zhijian R: Peter Xu S: Odd Fixes F: migration/rdma* +F: scripts/rdma-migration-helper.sh Migration dirty limit and dirty page rate M: Hyman Huang diff --git a/scripts/rdma-migration-helper.sh b/scripts/rdma-migration-helper.sh new file mode 100755 index 00000000000..66557d9e267 --- /dev/null +++ b/scripts/rdma-migration-helper.sh @@ -0,0 +1,41 @@ +#!/bin/bash + +# Copied from blktests +get_ipv4_addr() +{ + ip -4 -o addr show dev "$1" | + sed -n 's/.*[[:blank:]]inet[[:blank:]]*\([^[:blank:]/]*\).*/\1/p' | + tr -d '\n' +} + +has_soft_rdma() +{ + rdma link | grep -q " netdev $1[[:blank:]]*\$" +} + +rdma_rxe_setup_detect() +{ + ( + cd /sys/class/net && + for i in *; do + [ -e "$i" ] || continue + [ "$i" = "lo" ] && continue + [ "$(<"$i/addr_len")" = 6 ] || continue + [ "$(<"$i/carrier")" = 1 ] || continue + + has_soft_rdma "$i" && break + [ "$operation" = "setup" ] && + rdma link add "${i}_rxe" type rxe netdev "$i" && break + done + has_soft_rdma "$i" || return + get_ipv4_addr "$i" + ) +} + +operation=${1:-setup} + +if [ "$operation" == "setup" ] || [ "$operation" == "detect" ]; then + rdma_rxe_setup_detect +else + echo "Usage: $0 [setup | detect]" +fi diff --git a/tests/qtest/migration/precopy-tests.c b/tests/qtest/migration/precopy-tests.c index ba273d10b9a..bf97f4e9325 100644 --- a/tests/qtest/migration/precopy-tests.c +++ b/tests/qtest/migration/precopy-tests.c @@ -99,6 +99,66 @@ static void test_precopy_unix_dirty_ring(void) test_precopy_common(&args); } +#ifdef CONFIG_RDMA + +#define RDMA_MIGRATION_HELPER "scripts/rdma-migration-helper.sh" +static int new_rdma_link(char *buffer) +{ + const char *argument = (geteuid() == 0) ? "setup" : "detect"; + char cmd[1024]; + + snprintf(cmd, sizeof(cmd), "%s %s", RDMA_MIGRATION_HELPER, argument); + + FILE *pipe = popen(cmd, "r"); + if (pipe == NULL) { + perror("Failed to run script"); + return -1; + } + + int idx = 0; + while (fgets(buffer + idx, 128 - idx, pipe) != NULL) { + idx += strlen(buffer); + } + + int status = pclose(pipe); + if (status == -1) { + perror("Error reported by pclose()"); + return -1; + } else if (WIFEXITED(status)) { + return WEXITSTATUS(status); + } + + return -1; +} + +static void test_precopy_rdma_plain(void) +{ + char buffer[128] = {}; + + if (new_rdma_link(buffer)) { + g_test_skip("\nThere is no available rdma link to run RDMA migration test.\n" + "To enable the test:\n" + "(1) Run \'" RDMA_MIGRATION_HELPER " setup\' with root and rerun the test\n" + "or\n" + "(2) Run the test with root privilege\n"); + return; + } + + /* + * TODO: query a free port instead of hard code. + * 29200=('R'+'D'+'M'+'A')*100 + **/ + g_autofree char *uri = g_strdup_printf("rdma:%s:29200", buffer); + + MigrateCommon args = { + .listen_uri = uri, + .connect_uri = uri, + }; + + test_precopy_common(&args); +} +#endif + static void test_precopy_tcp_plain(void) { MigrateCommon args = { @@ -1124,6 +1184,10 @@ static void migration_test_add_precopy_smoke(MigrationTestEnv *env) test_multifd_tcp_uri_none); migration_test_add("/migration/multifd/tcp/plain/cancel", test_multifd_tcp_cancel); +#ifdef CONFIG_RDMA + migration_test_add("/migration/precopy/rdma/plain", + test_precopy_rdma_plain); +#endif } void migration_test_add_precopy(MigrationTestEnv *env)