From patchwork Tue Oct 29 15:09:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Prasad Pandit X-Patchwork-Id: 13855009 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E65EFD3A66B for ; Tue, 29 Oct 2024 15:10:12 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t5nqk-0006WK-Ge; Tue, 29 Oct 2024 11:09:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t5nqi-0006Vl-QH for qemu-devel@nongnu.org; Tue, 29 Oct 2024 11:09:36 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t5nqf-0007Jn-Ev for qemu-devel@nongnu.org; Tue, 29 Oct 2024 11:09:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1730214567; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8mE+VKNyaO+nL5L+9GQV3DwO8jHatnhKEZ6MnmFcS1Y=; b=T9tj3/qn6Dzrekf+LGRmNEN04awA9pIX5LqdbGYM7tj4pFgsTOVh1gCSTJBBURL2vlNfWu +VB3PUAJlxwEg6y4AlKXZDuC/oxy7H9q1K4kPVYl0s9RGfybEwsNVBdK3xQlwRIul+Vuu3 j8JFca38uypyGfoZ+JZH9lTTPbO2Ed8= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-551-5LFhq5_YMGyqXwtJh7EztQ-1; Tue, 29 Oct 2024 11:09:24 -0400 X-MC-Unique: 5LFhq5_YMGyqXwtJh7EztQ-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id E8EEB19560BD; Tue, 29 Oct 2024 15:09:22 +0000 (UTC) Received: from kaapi.redhat.com (unknown [10.74.16.130]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 41AAA1956088; Tue, 29 Oct 2024 15:09:19 +0000 (UTC) From: Prasad Pandit To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , Prasad Pandit Subject: [PATCH 1/5] migration/multifd: move macros to multifd header Date: Tue, 29 Oct 2024 20:39:04 +0530 Message-ID: <20241029150908.1136894-2-ppandit@redhat.com> In-Reply-To: <20241029150908.1136894-1-ppandit@redhat.com> References: <20241029150908.1136894-1-ppandit@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 Received-SPF: pass client-ip=170.10.129.124; envelope-from=ppandit@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -23 X-Spam_score: -2.4 X-Spam_bar: -- X-Spam_report: (-2.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.302, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Prasad Pandit Move MULTIFD_ macros to the header file so that they are accessible from other files. Signed-off-by: Prasad Pandit --- migration/multifd.c | 4 ---- migration/multifd.h | 5 +++++ 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/migration/multifd.c b/migration/multifd.c index 9b200f4ad9..97f6b6242a 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -33,10 +33,6 @@ #include "io/channel-socket.h" #include "yank_functions.h" -/* Multiple fd's */ - -#define MULTIFD_MAGIC 0x11223344U -#define MULTIFD_VERSION 1 typedef struct { uint32_t magic; diff --git a/migration/multifd.h b/migration/multifd.h index 50d58c0c9c..519dffeb9e 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -33,6 +33,11 @@ bool multifd_queue_page(RAMBlock *block, ram_addr_t offset); bool multifd_recv(void); MultiFDRecvData *multifd_get_recv_data(void); +/* Multiple fd's */ + +#define MULTIFD_MAGIC 0x11223344U +#define MULTIFD_VERSION 1 + /* Multifd Compression flags */ #define MULTIFD_FLAG_SYNC (1 << 0) From patchwork Tue Oct 29 15:09:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Prasad Pandit X-Patchwork-Id: 13855008 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A8472D3A66B for ; Tue, 29 Oct 2024 15:09:55 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t5nqj-0006WJ-Oy; Tue, 29 Oct 2024 11:09:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t5nqi-0006Vr-Uf for qemu-devel@nongnu.org; Tue, 29 Oct 2024 11:09:36 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t5nqf-0007K3-Es for qemu-devel@nongnu.org; Tue, 29 Oct 2024 11:09:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1730214572; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EawrAMiqZPHCdfZ9f5osnaUKjQDNMwjBz2dwMHPU2NM=; b=RwzE2gh55IAmilFoiSQiDPtthMRMTbw3w6Kej3+5OUUw6gpgITXl49eNe00KqVssw66cGu 0foiVu7wCXrr5so18nx0RO8icU7nBoiWTYt5NwCGLJ3Ru2uN9A31cJb9ozRGaS2UqZug/b GJWuufp+GzV+SsSX4MaRUH2mRs0u8jo= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-605-Sf4MOfgjPmKRKAhmnb6HKQ-1; Tue, 29 Oct 2024 11:09:29 -0400 X-MC-Unique: Sf4MOfgjPmKRKAhmnb6HKQ-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 2228F195609D; Tue, 29 Oct 2024 15:09:28 +0000 (UTC) Received: from kaapi.redhat.com (unknown [10.74.16.130]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id B9EF51956086; Tue, 29 Oct 2024 15:09:23 +0000 (UTC) From: Prasad Pandit To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , Prasad Pandit Subject: [PATCH 2/5] migration/postcopy: magic value for postcopy channel Date: Tue, 29 Oct 2024 20:39:05 +0530 Message-ID: <20241029150908.1136894-3-ppandit@redhat.com> In-Reply-To: <20241029150908.1136894-1-ppandit@redhat.com> References: <20241029150908.1136894-1-ppandit@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 Received-SPF: pass client-ip=170.10.129.124; envelope-from=ppandit@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -23 X-Spam_score: -2.4 X-Spam_bar: -- X-Spam_report: (-2.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.302, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Prasad Pandit During migration, the precopy and the multifd channels send magic value (4-bytes) to the destination side, for it to identify the channel and properly establish connection. But Postcopy channel did not send such value. Introduce a magic value to be sent on the postcopy channel. It helps to identify channels when both multifd and postcopy migration are enabled together. An explicitly defined magic value makes code easier to follow, because it is consistent with the other channels. Signed-off-by: Prasad Pandit --- migration/postcopy-ram.c | 7 +++++++ migration/postcopy-ram.h | 3 +++ 2 files changed, 10 insertions(+) diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 83f6160a36..5bc19b541c 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -1580,6 +1580,9 @@ void postcopy_unregister_shared_ufd(struct PostCopyFD *pcfd) void postcopy_preempt_new_channel(MigrationIncomingState *mis, QEMUFile *file) { + if (mis->postcopy_qemufile_dst) { + return; + } /* * The new loading channel has its own threads, so it needs to be * blocked too. It's by default true, just be explicit. @@ -1612,6 +1615,10 @@ postcopy_preempt_send_channel_done(MigrationState *s, * postcopy_qemufile_src to know whether it failed or not. */ qemu_sem_post(&s->postcopy_qemufile_src_sem); + + /* Send magic value to identify postcopy channel on the destination */ + uint32_t magic = cpu_to_be32(POSTCOPY_MAGIC); + qio_channel_write_all(ioc, (char *)&magic, sizeof(magic), NULL); } static void diff --git a/migration/postcopy-ram.h b/migration/postcopy-ram.h index a6df1b2811..49e2982558 100644 --- a/migration/postcopy-ram.h +++ b/migration/postcopy-ram.h @@ -15,6 +15,9 @@ #include "qapi/qapi-types-migration.h" +/* Magic value to identify postcopy channel on the destination */ +#define POSTCOPY_MAGIC 0x55667788U + /* Return true if the host supports everything we need to do postcopy-ram */ bool postcopy_ram_supported_by_host(MigrationIncomingState *mis, Error **errp); From patchwork Tue Oct 29 15:09:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Prasad Pandit X-Patchwork-Id: 13855011 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A98D1D3A66B for ; Tue, 29 Oct 2024 15:10:39 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t5nqt-0006Y4-9m; Tue, 29 Oct 2024 11:09:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t5nqr-0006XR-6W for qemu-devel@nongnu.org; Tue, 29 Oct 2024 11:09:45 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t5nql-0007LC-Nn for qemu-devel@nongnu.org; Tue, 29 Oct 2024 11:09:40 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1730214578; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gnKvSAGsXn7wVbU9jG9Lh0cUI9aeI16g1oxT5Kl0Gq8=; b=OogzQWEU5Dw3T2nEvuqPWxdQkkqSgP96lscRcJN5yNVLFXbCsQ8GiGURHv11hJj2hVfUm+ p+enFbX1uN1FF+CbWefKR6p/KBfxLQxXvumlqo4bSPC4h8fVBhNQrf4vEh039qMKhkei8S yrvEJ0XF+r52Gha1v5zlwQTa9sMvnOA= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-307-X9M4qPLePCKOWphul-yhjw-1; Tue, 29 Oct 2024 11:09:33 -0400 X-MC-Unique: X9M4qPLePCKOWphul-yhjw-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id F28201955F42; Tue, 29 Oct 2024 15:09:32 +0000 (UTC) Received: from kaapi.redhat.com (unknown [10.74.16.130]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id D3AB61956086; Tue, 29 Oct 2024 15:09:28 +0000 (UTC) From: Prasad Pandit To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , Prasad Pandit Subject: [PATCH 3/5] migration: remove multifd check with postcopy Date: Tue, 29 Oct 2024 20:39:06 +0530 Message-ID: <20241029150908.1136894-4-ppandit@redhat.com> In-Reply-To: <20241029150908.1136894-1-ppandit@redhat.com> References: <20241029150908.1136894-1-ppandit@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 Received-SPF: pass client-ip=170.10.129.124; envelope-from=ppandit@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -23 X-Spam_score: -2.4 X-Spam_bar: -- X-Spam_report: (-2.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.302, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Prasad Pandit Remove multifd capability check with Postcopy mode. This helps to enable both multifd and postcopy together. Update migrate_multifd() to return false when migration reaches Postcopy phase. In Postcopy phase, source guest is paused, so the migration threads on the source stop sending/pushing data on the channels. The destination guest starts running and Postcopy threads there begin to request/pull data from the source side. Signed-off-by: Prasad Pandit --- migration/options.c | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/migration/options.c b/migration/options.c index ad8d6989a8..47c5137d5f 100644 --- a/migration/options.c +++ b/migration/options.c @@ -266,7 +266,8 @@ bool migrate_multifd(void) { MigrationState *s = migrate_get_current(); - return s->capabilities[MIGRATION_CAPABILITY_MULTIFD]; + return s->capabilities[MIGRATION_CAPABILITY_MULTIFD] + && !migration_in_postcopy(); } bool migrate_pause_before_switchover(void) @@ -479,11 +480,6 @@ bool migrate_caps_check(bool *old_caps, bool *new_caps, Error **errp) error_setg(errp, "Postcopy is not compatible with ignore-shared"); return false; } - - if (new_caps[MIGRATION_CAPABILITY_MULTIFD]) { - error_setg(errp, "Postcopy is not yet compatible with multifd"); - return false; - } } if (new_caps[MIGRATION_CAPABILITY_BACKGROUND_SNAPSHOT]) { From patchwork Tue Oct 29 15:09:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Prasad Pandit X-Patchwork-Id: 13855012 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 471B7D3A661 for ; Tue, 29 Oct 2024 15:10:43 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t5nqu-0006YY-FV; Tue, 29 Oct 2024 11:09:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t5nqs-0006Xk-E8 for qemu-devel@nongnu.org; Tue, 29 Oct 2024 11:09:46 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t5nqq-0007LI-Pm for qemu-devel@nongnu.org; Tue, 29 Oct 2024 11:09:46 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1730214580; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KmZ8F0qsUU0LCTzHXDV9zhG89gZuFvGDJ+PdF5DBmig=; b=Rqi4fhBBPdKvhwTtHkxDXZb2lUiRJC/5ekB7ombA5uyTWyVSlDAsMS2R2o17efsacM62Rv NrFBl7EBvyy24x+dkjLGCUiCrvuprnqCEFXDlqKFzhvUzVi0xQ3LnGNUkrw3kdLS0xNiJm RIKj6yNSUnZmuYiNY/Qf7iHVli27E+A= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-479-L2KPxKaFOU-XQknto7eVlw-1; Tue, 29 Oct 2024 11:09:38 -0400 X-MC-Unique: L2KPxKaFOU-XQknto7eVlw-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 2D89219560A3; Tue, 29 Oct 2024 15:09:37 +0000 (UTC) Received: from kaapi.redhat.com (unknown [10.74.16.130]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 9609C195607C; Tue, 29 Oct 2024 15:09:33 +0000 (UTC) From: Prasad Pandit To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , Prasad Pandit Subject: [PATCH 4/5] migration: refactor ram_save_target_page functions Date: Tue, 29 Oct 2024 20:39:07 +0530 Message-ID: <20241029150908.1136894-5-ppandit@redhat.com> In-Reply-To: <20241029150908.1136894-1-ppandit@redhat.com> References: <20241029150908.1136894-1-ppandit@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 Received-SPF: pass client-ip=170.10.129.124; envelope-from=ppandit@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -23 X-Spam_score: -2.4 X-Spam_bar: -- X-Spam_report: (-2.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.302, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Prasad Pandit Refactor ram_save_target_page legacy and multifd functions into one. Other than simplifying it, it avoids reinitialization of the 'migration_ops' object, when migration moves from multifd to postcopy phase. Signed-off-by: Prasad Pandit --- migration/ram.c | 54 ++++++++++++++++++++----------------------------- 1 file changed, 22 insertions(+), 32 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 326ce7eb79..f9a6395d00 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1985,18 +1985,36 @@ int ram_save_queue_pages(const char *rbname, ram_addr_t start, ram_addr_t len, } /** - * ram_save_target_page_legacy: save one target page + * ram_save_target_page_common: + * send one target page to multifd workers OR save one target page. * - * Returns the number of pages written + * Multifd mode: returns 1 if the page was queued, -1 otherwise. + * + * Non-multifd mode: returns the number of pages written * * @rs: current RAM state * @pss: data about the page we want to send */ -static int ram_save_target_page_legacy(RAMState *rs, PageSearchStatus *pss) +static int ram_save_target_page_common(RAMState *rs, PageSearchStatus *pss) { ram_addr_t offset = ((ram_addr_t)pss->page) << TARGET_PAGE_BITS; int res; + if (migrate_multifd()) { + RAMBlock *block = pss->block; + /* + * While using multifd live migration, we still need to handle zero + * page checking on the migration main thread. + */ + if (migrate_zero_page_detection() == ZERO_PAGE_DETECTION_LEGACY) { + if (save_zero_page(rs, pss, offset)) { + return 1; + } + } + + return ram_save_multifd_page(block, offset); + } + if (control_save_page(pss, offset, &res)) { return res; } @@ -2008,32 +2026,6 @@ static int ram_save_target_page_legacy(RAMState *rs, PageSearchStatus *pss) return ram_save_page(rs, pss); } -/** - * ram_save_target_page_multifd: send one target page to multifd workers - * - * Returns 1 if the page was queued, -1 otherwise. - * - * @rs: current RAM state - * @pss: data about the page we want to send - */ -static int ram_save_target_page_multifd(RAMState *rs, PageSearchStatus *pss) -{ - RAMBlock *block = pss->block; - ram_addr_t offset = ((ram_addr_t)pss->page) << TARGET_PAGE_BITS; - - /* - * While using multifd live migration, we still need to handle zero - * page checking on the migration main thread. - */ - if (migrate_zero_page_detection() == ZERO_PAGE_DETECTION_LEGACY) { - if (save_zero_page(rs, pss, offset)) { - return 1; - } - } - - return ram_save_multifd_page(block, offset); -} - /* Should be called before sending a host page */ static void pss_host_page_prepare(PageSearchStatus *pss) { @@ -3055,12 +3047,10 @@ static int ram_save_setup(QEMUFile *f, void *opaque, Error **errp) } migration_ops = g_malloc0(sizeof(MigrationOps)); + migration_ops->ram_save_target_page = ram_save_target_page_common; if (migrate_multifd()) { multifd_ram_save_setup(); - migration_ops->ram_save_target_page = ram_save_target_page_multifd; - } else { - migration_ops->ram_save_target_page = ram_save_target_page_legacy; } bql_unlock(); From patchwork Tue Oct 29 15:09:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Prasad Pandit X-Patchwork-Id: 13855013 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E999FD3A66B for ; Tue, 29 Oct 2024 15:11:18 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t5nqv-0006Yv-5n; Tue, 29 Oct 2024 11:09:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t5nqt-0006YI-UA for qemu-devel@nongnu.org; Tue, 29 Oct 2024 11:09:48 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t5nqs-0007Lf-2C for qemu-devel@nongnu.org; Tue, 29 Oct 2024 11:09:47 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1730214584; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oRa6uy1hrBS+4vNzMUBzGk1+RdEP5lh5kmw7SH2VOJQ=; b=IJkna4X7U06z9W9EimFE4ESWXeAncP7hg0QIsGJIyKXtvuAo+/OiiQmfAow4/3b3hdBb7q 54jmfEofn5FJYLM5wvSxtaqsLTem5ppHT+Ugj782h8bNGDBsMQBZuCQlXAgHBBy7kzeBEI LIq7U7IyvRLplOmTt2zRGff35npu05A= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-520-1WJIZWvMOVCBXMCNWAwBGg-1; Tue, 29 Oct 2024 11:09:41 -0400 X-MC-Unique: 1WJIZWvMOVCBXMCNWAwBGg-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 720C419560A3; Tue, 29 Oct 2024 15:09:40 +0000 (UTC) Received: from kaapi.redhat.com (unknown [10.74.16.130]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id EF7321956086; Tue, 29 Oct 2024 15:09:37 +0000 (UTC) From: Prasad Pandit To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , Prasad Pandit Subject: [PATCH 5/5] migration: enable multifd and postcopy together Date: Tue, 29 Oct 2024 20:39:08 +0530 Message-ID: <20241029150908.1136894-6-ppandit@redhat.com> In-Reply-To: <20241029150908.1136894-1-ppandit@redhat.com> References: <20241029150908.1136894-1-ppandit@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 Received-SPF: pass client-ip=170.10.133.124; envelope-from=ppandit@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -23 X-Spam_score: -2.4 X-Spam_bar: -- X-Spam_report: (-2.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.302, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Prasad Pandit Enable Multifd and Postcopy migration together. The migration_ioc_process_incoming() routine checks magic value sent on each channel and helps to properly setup multifd and postcopy channels. Idea is to take advantage of the multifd threads to accelerate transfer of large guest RAM to the destination and switch to postcopy mode sooner. The Precopy and Multifd threads work during the initial guest RAM transfer. When migration moves to the Postcopy phase, the source guest is paused, so the Precopy and Multifd threads stop sending data on their channels. Postcopy threads on the destination request/pull data from the source side. Signed-off-by: Prasad Pandit --- migration/migration.c | 73 ++++++++++++++++++++++++++----------------- 1 file changed, 44 insertions(+), 29 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 021faee2f3..11fcc1e012 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -92,6 +92,9 @@ enum mig_rp_message_type { MIG_RP_MSG_MAX }; +/* Migration channel types */ +enum { CH_DEFAULT, CH_MULTIFD, CH_POSTCOPY }; + /* When we add fault tolerance, we could have several migrations at once. For now we don't need to add dynamic creation of migration */ @@ -919,16 +922,15 @@ void migration_fd_process_incoming(QEMUFile *f) * Returns true when we want to start a new incoming migration process, * false otherwise. */ -static bool migration_should_start_incoming(bool main_channel) +static bool migration_should_start_incoming(uint8_t channel) { + if (channel == CH_POSTCOPY) { + return false; + } + /* Multifd doesn't start unless all channels are established */ if (migrate_multifd()) { - return migration_has_all_channels(); - } - - /* Preempt channel only starts when the main channel is created */ - if (migrate_postcopy_preempt()) { - return main_channel; + return multifd_recv_all_channels_created(); } /* @@ -936,7 +938,7 @@ static bool migration_should_start_incoming(bool main_channel) * it's the main channel that's being created, and we should always * proceed with this channel. */ - assert(main_channel); + assert(channel == CH_DEFAULT); return true; } @@ -945,13 +947,11 @@ void migration_ioc_process_incoming(QIOChannel *ioc, Error **errp) MigrationIncomingState *mis = migration_incoming_get_current(); Error *local_err = NULL; QEMUFile *f; - bool default_channel = true; uint32_t channel_magic = 0; + uint8_t channel = CH_DEFAULT; int ret = 0; - if (migrate_multifd() && !migrate_mapped_ram() && - !migrate_postcopy_ram() && - qio_channel_has_feature(ioc, QIO_CHANNEL_FEATURE_READ_MSG_PEEK)) { + if (qio_channel_has_feature(ioc, QIO_CHANNEL_FEATURE_READ_MSG_PEEK)) { /* * With multiple channels, it is possible that we receive channels * out of order on destination side, causing incorrect mapping of @@ -969,35 +969,49 @@ void migration_ioc_process_incoming(QIOChannel *ioc, Error **errp) return; } - default_channel = (channel_magic == cpu_to_be32(QEMU_VM_FILE_MAGIC)); - } else { - default_channel = !mis->from_src_file; + if (channel_magic == cpu_to_be32(QEMU_VM_FILE_MAGIC)) { + channel = CH_DEFAULT; + } else if (channel_magic == cpu_to_be32(MULTIFD_MAGIC)) { + channel = CH_MULTIFD; + } else if (channel_magic == cpu_to_be32(POSTCOPY_MAGIC)) { + if (qio_channel_read_all(ioc, (char *)&channel_magic, + sizeof(channel_magic), &local_err)) { + error_report_err(local_err); + return; + } + channel = CH_POSTCOPY; + } else { + error_report("%s: could not identify channel, unknown magic: %u", + __func__, channel_magic); + return; + } } if (multifd_recv_setup(errp) != 0) { return; } - if (default_channel) { + if (channel == CH_DEFAULT) { f = qemu_file_new_input(ioc); migration_incoming_setup(f); - } else { + } else if (channel == CH_MULTIFD) { /* Multiple connections */ - assert(migration_needs_multiple_sockets()); if (migrate_multifd()) { multifd_recv_new_channel(ioc, &local_err); - } else { + } + if (local_err) { + error_propagate(errp, local_err); + return; + } + } else if (channel == CH_POSTCOPY) { + if (migrate_postcopy()) { assert(migrate_postcopy_preempt()); f = qemu_file_new_input(ioc); postcopy_preempt_new_channel(mis, f); } - if (local_err) { - error_propagate(errp, local_err); - return; - } } - if (migration_should_start_incoming(default_channel)) { + if (migration_should_start_incoming(channel)) { /* If it's a recovery, we're done */ if (postcopy_try_recover()) { return; @@ -1014,21 +1028,22 @@ void migration_ioc_process_incoming(QIOChannel *ioc, Error **errp) */ bool migration_has_all_channels(void) { + bool ret = false; MigrationIncomingState *mis = migration_incoming_get_current(); if (!mis->from_src_file) { - return false; + return ret; } if (migrate_multifd()) { - return multifd_recv_all_channels_created(); + ret = multifd_recv_all_channels_created(); } - if (migrate_postcopy_preempt()) { - return mis->postcopy_qemufile_dst != NULL; + if (ret && migrate_postcopy_preempt()) { + ret = mis->postcopy_qemufile_dst != NULL; } - return true; + return ret; } int migrate_send_rp_switchover_ack(MigrationIncomingState *mis)