From patchwork Tue Nov 28 10:42:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 13470962 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E5304C4167B for ; Tue, 28 Nov 2023 10:44:53 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1r7vYm-0006jz-Oc; Tue, 28 Nov 2023 05:43:20 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r7vYk-0006jW-Vl for qemu-devel@nongnu.org; Tue, 28 Nov 2023 05:43:18 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r7vYj-0005dq-Ig for qemu-devel@nongnu.org; Tue, 28 Nov 2023 05:43:18 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1701168195; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VlbL1HFd0cOmdquKv/xWNmr/CPE5aCshT232rfU4Gmk=; b=O9Qzdct5SqPSwNkGe1qDYjKZHuW5W1badgipI82Fp1zWIlmAYmWWyI00wJ7kMy3D0roRxr O3UQGWvdKShsEDgn0doQYXYjAAQmG4SxYXhuerPAieTTL6GGv5EVFSqkxnzN8T2T0KwCzh zYgs0bzM/4p0YUveAgwDHrg9xkCruRE= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-133-VigptARGM4-l-eOpI6UVpg-1; Tue, 28 Nov 2023 05:43:11 -0500 X-MC-Unique: VigptARGM4-l-eOpI6UVpg-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 5952A811E7B; Tue, 28 Nov 2023 10:43:10 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.193.65]) by smtp.corp.redhat.com (Postfix) with ESMTP id 25EED20268D8; Tue, 28 Nov 2023 10:43:08 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Gautam Dawar , Jason Wang , Zhu Lingshan , yin31149@gmail.com, Shannon Nelson , "Michael S. Tsirkin" , Dragos Tatulea , Yajun Wu , Juan Quintela , Laurent Vivier , Stefano Garzarella , Parav Pandit , Lei Yang , si-wei.liu@oracle.com Subject: [RFC PATCH v2 01/10] vdpa: do not set virtio status bits if unneeded Date: Tue, 28 Nov 2023 11:42:54 +0100 Message-Id: <20231128104303.3314000-2-eperezma@redhat.com> In-Reply-To: <20231128104303.3314000-1-eperezma@redhat.com> References: <20231128104303.3314000-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.4 Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Next commits will set DRIVER and ACKNOWLEDGE flags repeatedly in the case of a migration destination. Let's save ioctls with this. Signed-off-by: Eugenio Pérez --- hw/virtio/vhost-vdpa.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 7500c2fc82..cc252fc2d8 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -510,6 +510,10 @@ static int vhost_vdpa_add_status(struct vhost_dev *dev, uint8_t status) if (ret < 0) { return ret; } + if ((s & status) == status) { + /* Don't set bits already set */ + return 0; + } s |= status; From patchwork Tue Nov 28 10:42:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 13470955 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 46053C4167B for ; Tue, 28 Nov 2023 10:44:03 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1r7vYm-0006k1-Vt; Tue, 28 Nov 2023 05:43:21 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r7vYm-0006jr-4m for qemu-devel@nongnu.org; Tue, 28 Nov 2023 05:43:20 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r7vYj-0005e7-Ja for qemu-devel@nongnu.org; Tue, 28 Nov 2023 05:43:19 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1701168196; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jiCsDaivPOvgkYPv1nwn65nD5kYXp+LU4DCyQMQzLN4=; b=MroIS/Ar0pbF7fPKHx1QOL1hUbvRgdVPOXuEHCg58Xgdqu0M2ISoUx64NlI0lqMiZ1yC7V jeT8A7UQ8fdC6gg5M04zjEDCp+gxL3TYMvTQJbXpcEYKHl/gIFwaqAiza4qNjy59GW7EnN o8Otrb/HHsoKhW8gked0jYoe/Kzs8cw= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-627-hE13P9OkM-KnaogtQDKufQ-1; Tue, 28 Nov 2023 05:43:13 -0500 X-MC-Unique: hE13P9OkM-KnaogtQDKufQ-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D43391C0512B; Tue, 28 Nov 2023 10:43:12 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.193.65]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9483220268D7; Tue, 28 Nov 2023 10:43:10 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Gautam Dawar , Jason Wang , Zhu Lingshan , yin31149@gmail.com, Shannon Nelson , "Michael S. Tsirkin" , Dragos Tatulea , Yajun Wu , Juan Quintela , Laurent Vivier , Stefano Garzarella , Parav Pandit , Lei Yang , si-wei.liu@oracle.com Subject: [RFC PATCH v2 02/10] vdpa: make batch_begin_once early return Date: Tue, 28 Nov 2023 11:42:55 +0100 Message-Id: <20231128104303.3314000-3-eperezma@redhat.com> In-Reply-To: <20231128104303.3314000-1-eperezma@redhat.com> References: <20231128104303.3314000-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.4 Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Prefer early return so it is easier to merge vhost_vdpa_listener_begin_batch here and make iotlb baches begin and end symmetrical. Signed-off-by: Eugenio Pérez --- hw/virtio/vhost-vdpa.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index cc252fc2d8..bf9771870a 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -160,11 +160,12 @@ static void vhost_vdpa_listener_begin_batch(VhostVDPAShared *s) static void vhost_vdpa_iotlb_batch_begin_once(VhostVDPAShared *s) { - if (s->backend_cap & (0x1ULL << VHOST_BACKEND_F_IOTLB_BATCH) && - !s->iotlb_batch_begin_sent) { - vhost_vdpa_listener_begin_batch(s); + if (!(s->backend_cap & (0x1ULL << VHOST_BACKEND_F_IOTLB_BATCH)) || + s->iotlb_batch_begin_sent) { + return; } + vhost_vdpa_listener_begin_batch(s); s->iotlb_batch_begin_sent = true; } From patchwork Tue Nov 28 10:42:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 13470959 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5E6BAC4167B for ; Tue, 28 Nov 2023 10:44:32 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1r7vYs-0006lu-MV; Tue, 28 Nov 2023 05:43:26 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r7vYr-0006l5-8B for qemu-devel@nongnu.org; Tue, 28 Nov 2023 05:43:25 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r7vYo-0005f0-SQ for qemu-devel@nongnu.org; Tue, 28 Nov 2023 05:43:24 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1701168201; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6D5rlfw7G1yLNiSv7CgLqkOdOD7ElVEtb3VUVH2sVwA=; b=WPc2DBjcBiSxkjeqmQYOikD9HP/JXaCm0D9UMdy8HnU76QyH88RJebhQU2eiCjpLB2A88t B57waV/18TUVmf/26Ka4pok08CehDd9E+4/qjJlKZBgzlJwVF7KYNWNZurBRm0r8iEDO3t 6V19Rqe+mJ0yNajDn65eH7b3z6wX/5g= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-179-y85cGbMBN0CnvmCWgQlIUA-1; Tue, 28 Nov 2023 05:43:15 -0500 X-MC-Unique: y85cGbMBN0CnvmCWgQlIUA-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 5F54B1C0512B; Tue, 28 Nov 2023 10:43:15 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.193.65]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1DCD320268D7; Tue, 28 Nov 2023 10:43:12 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Gautam Dawar , Jason Wang , Zhu Lingshan , yin31149@gmail.com, Shannon Nelson , "Michael S. Tsirkin" , Dragos Tatulea , Yajun Wu , Juan Quintela , Laurent Vivier , Stefano Garzarella , Parav Pandit , Lei Yang , si-wei.liu@oracle.com Subject: [RFC PATCH v2 03/10] vdpa: merge _begin_batch into _batch_begin_once Date: Tue, 28 Nov 2023 11:42:56 +0100 Message-Id: <20231128104303.3314000-4-eperezma@redhat.com> In-Reply-To: <20231128104303.3314000-1-eperezma@redhat.com> References: <20231128104303.3314000-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.4 Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org There was only one call. This way we can make the begin and end of the batch symmetrical. Signed-off-by: Eugenio Pérez --- hw/virtio/vhost-vdpa.c | 17 ++++++----------- 1 file changed, 6 insertions(+), 11 deletions(-) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index bf9771870a..a533fc5bc7 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -143,7 +143,7 @@ int vhost_vdpa_dma_unmap(VhostVDPAShared *s, uint32_t asid, hwaddr iova, return ret; } -static void vhost_vdpa_listener_begin_batch(VhostVDPAShared *s) +static void vhost_vdpa_iotlb_batch_begin_once(VhostVDPAShared *s) { int fd = s->device_fd; struct vhost_msg_v2 msg = { @@ -151,21 +151,16 @@ static void vhost_vdpa_listener_begin_batch(VhostVDPAShared *s) .iotlb.type = VHOST_IOTLB_BATCH_BEGIN, }; - trace_vhost_vdpa_listener_begin_batch(s, fd, msg.type, msg.iotlb.type); - if (write(fd, &msg, sizeof(msg)) != sizeof(msg)) { - error_report("failed to write, fd=%d, errno=%d (%s)", - fd, errno, strerror(errno)); - } -} - -static void vhost_vdpa_iotlb_batch_begin_once(VhostVDPAShared *s) -{ if (!(s->backend_cap & (0x1ULL << VHOST_BACKEND_F_IOTLB_BATCH)) || s->iotlb_batch_begin_sent) { return; } - vhost_vdpa_listener_begin_batch(s); + trace_vhost_vdpa_listener_begin_batch(s, fd, msg.type, msg.iotlb.type); + if (write(fd, &msg, sizeof(msg)) != sizeof(msg)) { + error_report("failed to write, fd=%d, errno=%d (%s)", + fd, errno, strerror(errno)); + } s->iotlb_batch_begin_sent = true; } From patchwork Tue Nov 28 10:42:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 13470963 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8374AC4167B for ; Tue, 28 Nov 2023 10:45:17 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1r7vYs-0006lL-5I; Tue, 28 Nov 2023 05:43:26 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r7vYp-0006kr-TN for qemu-devel@nongnu.org; Tue, 28 Nov 2023 05:43:23 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r7vYo-0005f1-GU for qemu-devel@nongnu.org; Tue, 28 Nov 2023 05:43:23 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1701168201; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=czRuS48Nn1k6ylS+U5+ONFWw9urqtIfLEQqVSIBrLjc=; b=i108wtazkOoE/8ccsj9pkJKonrUMmayrxQq1W+IMChPz53ng0LbY/EpM7KUrTnC3IPPwRu 8FHLOv+jflzCcVFuIRDh3XPorxK7VmKKxs7cvJYQzTwTHiYW70sPDetrnB3lNic9IUzkVx 9E5C3RAvSu2YMPmMPbsEvg4D1oQov1A= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-500-xNYNfw_wM3Gbg4qw7e26OQ-1; Tue, 28 Nov 2023 05:43:18 -0500 X-MC-Unique: xNYNfw_wM3Gbg4qw7e26OQ-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D0B1A84AC62; Tue, 28 Nov 2023 10:43:17 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.193.65]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9CF9F20268D7; Tue, 28 Nov 2023 10:43:15 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Gautam Dawar , Jason Wang , Zhu Lingshan , yin31149@gmail.com, Shannon Nelson , "Michael S. Tsirkin" , Dragos Tatulea , Yajun Wu , Juan Quintela , Laurent Vivier , Stefano Garzarella , Parav Pandit , Lei Yang , si-wei.liu@oracle.com Subject: [RFC PATCH v2 04/10] vdpa: extract out _dma_end_batch from _listener_commit Date: Tue, 28 Nov 2023 11:42:57 +0100 Message-Id: <20231128104303.3314000-5-eperezma@redhat.com> In-Reply-To: <20231128104303.3314000-1-eperezma@redhat.com> References: <20231128104303.3314000-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.4 Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org So we can call out vhost_vdpa_dma_end_batch out of the listener callbacks. Signed-off-by: Eugenio Pérez --- hw/virtio/vhost-vdpa.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index a533fc5bc7..57a8043cd4 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -164,9 +164,8 @@ static void vhost_vdpa_iotlb_batch_begin_once(VhostVDPAShared *s) s->iotlb_batch_begin_sent = true; } -static void vhost_vdpa_listener_commit(MemoryListener *listener) +static void vhost_vdpa_dma_end_batch(VhostVDPAShared *s) { - VhostVDPAShared *s = container_of(listener, VhostVDPAShared, listener); struct vhost_msg_v2 msg = {}; int fd = s->device_fd; @@ -190,6 +189,13 @@ static void vhost_vdpa_listener_commit(MemoryListener *listener) s->iotlb_batch_begin_sent = false; } +static void vhost_vdpa_listener_commit(MemoryListener *listener) +{ + VhostVDPAShared *s = container_of(listener, VhostVDPAShared, listener); + + vhost_vdpa_dma_end_batch(s); +} + static void vhost_vdpa_iommu_map_notify(IOMMUNotifier *n, IOMMUTLBEntry *iotlb) { struct vdpa_iommu *iommu = container_of(n, struct vdpa_iommu, n); From patchwork Tue Nov 28 10:42:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 13470958 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6381CC4167B for ; Tue, 28 Nov 2023 10:44:26 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1r7vYw-0006mb-BH; Tue, 28 Nov 2023 05:43:30 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r7vYu-0006m8-P9 for qemu-devel@nongnu.org; Tue, 28 Nov 2023 05:43:28 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r7vYt-0005fx-7r for qemu-devel@nongnu.org; Tue, 28 Nov 2023 05:43:28 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1701168205; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=A8WueJS/E1fA/bzQAcF/yF24VB8sgCLQokVl1j8eJtk=; b=IxH/d0BAymxKWcLqwfuE/Ra4rIPX6oqcYqaPZkVnT8ju7NT75cGAsdLulM3YKSEeesW+iG E3CAvLr201SJ7gwgjiQPMfDI7GuwiTmQL8iRlQuQ2doX4l8DUI+iVT3+P5nKlvM19sn3Mj W8crxXv+3YeQXWH+apMKtM+Mofal9kM= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-674-SCCtnt-qNxyXh-zAz7r-8Q-1; Tue, 28 Nov 2023 05:43:20 -0500 X-MC-Unique: SCCtnt-qNxyXh-zAz7r-8Q-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 4F14481D873; Tue, 28 Nov 2023 10:43:20 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.193.65]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1B35220268D7; Tue, 28 Nov 2023 10:43:17 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Gautam Dawar , Jason Wang , Zhu Lingshan , yin31149@gmail.com, Shannon Nelson , "Michael S. Tsirkin" , Dragos Tatulea , Yajun Wu , Juan Quintela , Laurent Vivier , Stefano Garzarella , Parav Pandit , Lei Yang , si-wei.liu@oracle.com Subject: [RFC PATCH v2 05/10] vdpa: factor out stop path of vhost_vdpa_dev_start Date: Tue, 28 Nov 2023 11:42:58 +0100 Message-Id: <20231128104303.3314000-6-eperezma@redhat.com> In-Reply-To: <20231128104303.3314000-1-eperezma@redhat.com> References: <20231128104303.3314000-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.4 Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org This makes easier to build an error path in next patches. No functional change. Signed-off-by: Eugenio Pérez --- hw/virtio/vhost-vdpa.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 57a8043cd4..449c3794b2 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -1302,7 +1302,7 @@ static void vhost_vdpa_suspend(struct vhost_dev *dev) static int vhost_vdpa_dev_start(struct vhost_dev *dev, bool started) { struct vhost_vdpa *v = dev->opaque; - bool ok; + bool ok = true; trace_vhost_vdpa_dev_start(dev, started); if (started) { @@ -1313,8 +1313,7 @@ static int vhost_vdpa_dev_start(struct vhost_dev *dev, bool started) } } else { vhost_vdpa_suspend(dev); - vhost_vdpa_svqs_stop(dev); - vhost_vdpa_host_notifiers_uninit(dev, dev->nvqs); + goto out_stop; } if (dev->vq_index + dev->nvqs != dev->vq_index_end) { @@ -1333,6 +1332,11 @@ static int vhost_vdpa_dev_start(struct vhost_dev *dev, bool started) } return 0; + +out_stop: + vhost_vdpa_svqs_stop(dev); + vhost_vdpa_host_notifiers_uninit(dev, dev->nvqs); + return ok ? 0 : -1; } static void vhost_vdpa_reset_status(struct vhost_dev *dev) From patchwork Tue Nov 28 10:42:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 13470954 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4609BC46CA0 for ; Tue, 28 Nov 2023 10:44:03 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1r7vZ0-0006mv-Oh; Tue, 28 Nov 2023 05:43:34 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r7vYz-0006md-55 for qemu-devel@nongnu.org; Tue, 28 Nov 2023 05:43:33 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r7vYu-0005gN-K9 for qemu-devel@nongnu.org; Tue, 28 Nov 2023 05:43:32 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1701168207; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9X3lY7EwesmvNy+U/t0ivD5sWmuCNHr3WZysO4tIJ8U=; b=cFUTmXM3mFVIkIMUYisWqmxUWCl2YHWm61CGZpoFxRlJWU9iY6PIXauN5/toRk709ttdR+ tUTC2KpOAFzqo3ceKlBwftQ4/YxRrgMyCz5v6C4oxZQOFydUXN6Aoo7eG69T8Rw0sdyhex RMsKHEVORtXZIrQ6k9kcHxHRw+BvLGY= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-467-fE16MYqpPDuLOzbJx1LwwQ-1; Tue, 28 Nov 2023 05:43:23 -0500 X-MC-Unique: fE16MYqpPDuLOzbJx1LwwQ-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id BFD2B280BC43; Tue, 28 Nov 2023 10:43:22 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.193.65]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8C77720268D7; Tue, 28 Nov 2023 10:43:20 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Gautam Dawar , Jason Wang , Zhu Lingshan , yin31149@gmail.com, Shannon Nelson , "Michael S. Tsirkin" , Dragos Tatulea , Yajun Wu , Juan Quintela , Laurent Vivier , Stefano Garzarella , Parav Pandit , Lei Yang , si-wei.liu@oracle.com Subject: [RFC PATCH v2 06/10] vdpa: check for iova tree initialized at net_client_start Date: Tue, 28 Nov 2023 11:42:59 +0100 Message-Id: <20231128104303.3314000-7-eperezma@redhat.com> In-Reply-To: <20231128104303.3314000-1-eperezma@redhat.com> References: <20231128104303.3314000-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.4 Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -16 X-Spam_score: -1.7 X-Spam_bar: - X-Spam_report: (-1.7 / 5.0 requ) BAYES_00=-1.9, DKIM_INVALID=0.1, DKIM_SIGNED=0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org To map the guest memory while it is migrating we need to create the iova_tree, as long as the destination uses x-svq=on. Checking to not override it. The function vhost_vdpa_net_client_stop clear it if the device is stopped. If the guest starts the device again, the iova tree is recreated by vhost_vdpa_net_data_start_first or vhost_vdpa_net_cvq_start if needed, so old behavior is kept. Signed-off-by: Eugenio Pérez --- net/vhost-vdpa.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index 3fb209cd35..a37de7860e 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -341,7 +341,9 @@ static void vhost_vdpa_net_data_start_first(VhostVDPAState *s) migration_add_notifier(&s->migration_state, vdpa_net_migration_state_notifier); - if (v->shadow_vqs_enabled) { + + /* iova_tree may be initialized by vhost_vdpa_net_load_setup */ + if (v->shadow_vqs_enabled && !v->shared->iova_tree) { v->shared->iova_tree = vhost_iova_tree_new(v->shared->iova_range.first, v->shared->iova_range.last); } From patchwork Tue Nov 28 10:43:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 13470956 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 10768C07E98 for ; Tue, 28 Nov 2023 10:44:09 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1r7vZ9-0006nZ-Pb; Tue, 28 Nov 2023 05:43:43 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r7vZ8-0006nI-Oz for qemu-devel@nongnu.org; Tue, 28 Nov 2023 05:43:42 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r7vYv-0005gb-Tk for qemu-devel@nongnu.org; Tue, 28 Nov 2023 05:43:34 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1701168209; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dEVdqJdWKralGQPtSdv/lMX0iMS7IIhcmIcBa2iN+lk=; b=F2kN+iL0/xsaq8BXayD+PhkO3oUbiLLGqBaxZOQc9WCb7SqdxyomHcWiT4ZgQOv/i0awB7 PfXmtxdIxp83tXZ+wGxym2E7e/foSSRZPPLCL7TnT4SvUcL7xxSXdd5375BbWerH7BaOi8 sMsGrjne4kqPQN3phHOYKebOHgGxwmY= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-86-YHpmpDUBNSOhoCdiePKT2A-1; Tue, 28 Nov 2023 05:43:25 -0500 X-MC-Unique: YHpmpDUBNSOhoCdiePKT2A-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 43D541C294A3; Tue, 28 Nov 2023 10:43:25 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.193.65]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0A7B320268D7; Tue, 28 Nov 2023 10:43:22 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Gautam Dawar , Jason Wang , Zhu Lingshan , yin31149@gmail.com, Shannon Nelson , "Michael S. Tsirkin" , Dragos Tatulea , Yajun Wu , Juan Quintela , Laurent Vivier , Stefano Garzarella , Parav Pandit , Lei Yang , si-wei.liu@oracle.com Subject: [RFC PATCH v2 07/10] vdpa: set backend capabilities at vhost_vdpa_init Date: Tue, 28 Nov 2023 11:43:00 +0100 Message-Id: <20231128104303.3314000-8-eperezma@redhat.com> In-Reply-To: <20231128104303.3314000-1-eperezma@redhat.com> References: <20231128104303.3314000-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.4 Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org The backend does not reset them until the vdpa file descriptor is closed so there is no harm in doing it only once. This allows the destination of a live migration to premap memory in batches, using VHOST_BACKEND_F_IOTLB_BATCH. Signed-off-by: Eugenio Pérez --- hw/virtio/vhost-vdpa.c | 50 ++++++++++++++++-------------------------- 1 file changed, 19 insertions(+), 31 deletions(-) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 449c3794b2..43f7c382b1 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -587,11 +587,25 @@ static int vhost_vdpa_init(struct vhost_dev *dev, void *opaque, Error **errp) struct vhost_vdpa *v = opaque; assert(dev->vhost_ops->backend_type == VHOST_BACKEND_TYPE_VDPA); trace_vhost_vdpa_init(dev, v->shared, opaque); + uint64_t backend_features; + uint64_t qemu_backend_features = 0x1ULL << VHOST_BACKEND_F_IOTLB_MSG_V2 | + 0x1ULL << VHOST_BACKEND_F_IOTLB_BATCH | + 0x1ULL << VHOST_BACKEND_F_IOTLB_ASID | + 0x1ULL << VHOST_BACKEND_F_SUSPEND; int ret; v->dev = dev; dev->opaque = opaque ; v->shared->listener = vhost_vdpa_memory_listener; + + if (vhost_vdpa_call(dev, VHOST_GET_BACKEND_FEATURES, &backend_features)) { + return -EFAULT; + } + + backend_features &= qemu_backend_features; + + dev->backend_cap = backend_features; + v->shared->backend_cap = backend_features; vhost_vdpa_init_svq(dev, v); error_propagate(&dev->migration_blocker, v->migration_blocker); @@ -599,6 +613,11 @@ static int vhost_vdpa_init(struct vhost_dev *dev, void *opaque, Error **errp) return 0; } + ret = vhost_vdpa_call(dev, VHOST_SET_BACKEND_FEATURES, &backend_features); + if (ret) { + return -EFAULT; + } + /* * If dev->shadow_vqs_enabled at initialization that means the device has * been started with x-svq=on, so don't block migration @@ -829,36 +848,6 @@ static int vhost_vdpa_set_features(struct vhost_dev *dev, return vhost_vdpa_add_status(dev, VIRTIO_CONFIG_S_FEATURES_OK); } -static int vhost_vdpa_set_backend_cap(struct vhost_dev *dev) -{ - struct vhost_vdpa *v = dev->opaque; - - uint64_t features; - uint64_t f = 0x1ULL << VHOST_BACKEND_F_IOTLB_MSG_V2 | - 0x1ULL << VHOST_BACKEND_F_IOTLB_BATCH | - 0x1ULL << VHOST_BACKEND_F_IOTLB_ASID | - 0x1ULL << VHOST_BACKEND_F_SUSPEND; - int r; - - if (vhost_vdpa_call(dev, VHOST_GET_BACKEND_FEATURES, &features)) { - return -EFAULT; - } - - features &= f; - - if (vhost_vdpa_first_dev(dev)) { - r = vhost_vdpa_call(dev, VHOST_SET_BACKEND_FEATURES, &features); - if (r) { - return -EFAULT; - } - } - - dev->backend_cap = features; - v->shared->backend_cap = features; - - return 0; -} - static int vhost_vdpa_get_device_id(struct vhost_dev *dev, uint32_t *device_id) { @@ -1512,7 +1501,6 @@ const VhostOps vdpa_ops = { .vhost_set_vring_kick = vhost_vdpa_set_vring_kick, .vhost_set_vring_call = vhost_vdpa_set_vring_call, .vhost_get_features = vhost_vdpa_get_features, - .vhost_set_backend_cap = vhost_vdpa_set_backend_cap, .vhost_set_owner = vhost_vdpa_set_owner, .vhost_set_vring_endian = NULL, .vhost_backend_memslots_limit = vhost_vdpa_memslots_limit, From patchwork Tue Nov 28 10:43:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 13470957 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 197E1C4167B for ; Tue, 28 Nov 2023 10:44:23 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1r7vZ9-0006nY-Kd; Tue, 28 Nov 2023 05:43:43 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r7vZ8-0006nJ-P5 for qemu-devel@nongnu.org; Tue, 28 Nov 2023 05:43:42 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r7vYz-0005gl-15 for qemu-devel@nongnu.org; Tue, 28 Nov 2023 05:43:34 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1701168212; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=IMGBqBxHbDIzPYVYBCJ4kY+zoFlcXenNG6ZZ68h3Q48=; b=cM4i2fA8qZge4cKhf0ZsOoMVBDWHVlpG1dMOcB7XHz9UtAuStLmG6clD8bon4hU5oueL5S H1je9XPxmbmphlnhewBYdmj/VUqK+aiZtqyVYDB078dUans3MJwDVto6P8OMXp0MdH9Y67 nkKDrAXyr6mZWqsW6eRgZA082c6UUro= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-159-k8rMvenXNIS1U4xt27kKgA-1; Tue, 28 Nov 2023 05:43:28 -0500 X-MC-Unique: k8rMvenXNIS1U4xt27kKgA-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B5780185A782; Tue, 28 Nov 2023 10:43:27 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.193.65]) by smtp.corp.redhat.com (Postfix) with ESMTP id 809FF20268D7; Tue, 28 Nov 2023 10:43:25 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Gautam Dawar , Jason Wang , Zhu Lingshan , yin31149@gmail.com, Shannon Nelson , "Michael S. Tsirkin" , Dragos Tatulea , Yajun Wu , Juan Quintela , Laurent Vivier , Stefano Garzarella , Parav Pandit , Lei Yang , si-wei.liu@oracle.com Subject: [RFC PATCH v2 08/10] vdpa: add vhost_vdpa_load_setup Date: Tue, 28 Nov 2023 11:43:01 +0100 Message-Id: <20231128104303.3314000-9-eperezma@redhat.com> In-Reply-To: <20231128104303.3314000-1-eperezma@redhat.com> References: <20231128104303.3314000-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.4 Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Callers can use this function to setup the incoming migration thread. This thread is able to map the guest memory while the migration is ongoing, without blocking QMP or other important tasks. While this allows the destination QEMU not to block, it expands the mapping time during migration instead of making it pre-migration. This thread joins at vdpa backend device start, so it could happen that the guest memory is so large that we still have guest memory to map before this time. This can be improved in later iterations, when the destination device can inform QEMU that it is not ready to complete the migration. If the device is not started, the clean of the mapped memory is done at .load_cleanup. This is far from ideal, as the destination machine has mapped all the guest ram for nothing, and now it needs to unmap it. However, we don't have information about the state of the device so its the best we can do. Once iterative migration is supported, this will be improved as we know the virtio state of the device. TODO RFC: if the VM migrates before finishing all the maps, the source will stop but the destination is still not ready to continue, and it will wait until all guest RAM is mapped. It is still an improvement over doing all the map when the migration finish, but it should be easy to forbid the guest to stop until a condition is met. TODO RFC: The memory unmapping if the device is not started is weird too, as ideally nothing would be mapped. This can be fixed when we migrate the device state iteratively, and we know for sure if the device is started or not. At this moment we don't have such information so there is no better alternative. Other options considered: * Coroutines: Overkill? What thread can I assign them, as vdpa does not have any dedicated iothread for the moment? * QemuEvent or Mutex + Cond: Need to synchronize list access then, complicating the synchronization. As maps ops are heavier enough, it is not worth. Signed-off-by: Eugenio Pérez --- RFC v2: * Use a dedicated thread for map instead of doing all in .load_setup, blocking QMP etc. Signed-off-by: Eugenio Pérez --- include/hw/virtio/vhost-vdpa.h | 25 +++++ hw/virtio/vhost-vdpa.c | 167 ++++++++++++++++++++++++++++++++- 2 files changed, 191 insertions(+), 1 deletion(-) diff --git a/include/hw/virtio/vhost-vdpa.h b/include/hw/virtio/vhost-vdpa.h index 8f54e5edd4..6533ad211c 100644 --- a/include/hw/virtio/vhost-vdpa.h +++ b/include/hw/virtio/vhost-vdpa.h @@ -17,6 +17,7 @@ #include "hw/virtio/vhost-iova-tree.h" #include "hw/virtio/vhost-shadow-virtqueue.h" #include "hw/virtio/virtio.h" +#include "qemu/thread.h" #include "standard-headers/linux/vhost_types.h" /* @@ -43,8 +44,30 @@ typedef struct vhost_vdpa_shared { /* Copy of backend features */ uint64_t backend_cap; + /* + * Thread to map memory in QEMU incoming migration. + * + * Incoming migration calls devices ->load_setup in the main thread, but + * map operations can take a long time. This forbids the main thread to + * serve other requests like QMP. + * + * It works by fetching jobs from map_queue until it receives + * VhostVDPAShared, signalling the end of thread job. From that point, + * thread is joined and maps requests are synchronous again. These new + * maps are not served from main thread, so there is no danger there. + */ + QemuThread map_thread; + GAsyncQueue *map_queue; + bool map_thread_enabled; + bool iotlb_batch_begin_sent; + /* + * The memory listener has been registered, so DMA maps have been sent to + * the device. + */ + bool listener_registered; + /* Vdpa must send shadow addresses as IOTLB key for data queues, not GPA */ bool shadow_data; } VhostVDPAShared; @@ -73,6 +96,8 @@ int vhost_vdpa_dma_map(VhostVDPAShared *s, uint32_t asid, hwaddr iova, hwaddr size, void *vaddr, bool readonly); int vhost_vdpa_dma_unmap(VhostVDPAShared *s, uint32_t asid, hwaddr iova, hwaddr size); +int vhost_vdpa_load_setup(VhostVDPAShared *s, AddressSpace *dma_as); +int vhost_vdpa_load_cleanup(VhostVDPAShared *s, bool vhost_will_start); typedef struct vdpa_iommu { VhostVDPAShared *dev_shared; diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 43f7c382b1..24844b5dfa 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -101,6 +101,15 @@ int vhost_vdpa_dma_map(VhostVDPAShared *s, uint32_t asid, hwaddr iova, msg.iotlb.perm = readonly ? VHOST_ACCESS_RO : VHOST_ACCESS_RW; msg.iotlb.type = VHOST_IOTLB_UPDATE; + if (s->map_thread_enabled && !qemu_thread_is_self(&s->map_thread)) { + struct vhost_msg_v2 *new_msg = g_new(struct vhost_msg_v2, 1); + + *new_msg = msg; + g_async_queue_push(s->map_queue, new_msg); + + return 0; + } + trace_vhost_vdpa_dma_map(s, fd, msg.type, msg.asid, msg.iotlb.iova, msg.iotlb.size, msg.iotlb.uaddr, msg.iotlb.perm, msg.iotlb.type); @@ -131,6 +140,15 @@ int vhost_vdpa_dma_unmap(VhostVDPAShared *s, uint32_t asid, hwaddr iova, msg.iotlb.size = size; msg.iotlb.type = VHOST_IOTLB_INVALIDATE; + if (s->map_thread_enabled && !qemu_thread_is_self(&s->map_thread)) { + struct vhost_msg_v2 *new_msg = g_new(struct vhost_msg_v2, 1); + + *new_msg = msg; + g_async_queue_push(s->map_queue, new_msg); + + return 0; + } + trace_vhost_vdpa_dma_unmap(s, fd, msg.type, msg.asid, msg.iotlb.iova, msg.iotlb.size, msg.iotlb.type); @@ -156,6 +174,15 @@ static void vhost_vdpa_iotlb_batch_begin_once(VhostVDPAShared *s) return; } + if (s->map_thread_enabled && !qemu_thread_is_self(&s->map_thread)) { + struct vhost_msg_v2 *new_msg = g_new(struct vhost_msg_v2, 1); + + *new_msg = msg; + g_async_queue_push(s->map_queue, new_msg); + + return; + } + trace_vhost_vdpa_listener_begin_batch(s, fd, msg.type, msg.iotlb.type); if (write(fd, &msg, sizeof(msg)) != sizeof(msg)) { error_report("failed to write, fd=%d, errno=%d (%s)", @@ -180,6 +207,15 @@ static void vhost_vdpa_dma_end_batch(VhostVDPAShared *s) msg.type = VHOST_IOTLB_MSG_V2; msg.iotlb.type = VHOST_IOTLB_BATCH_END; + if (s->map_thread_enabled && !qemu_thread_is_self(&s->map_thread)) { + struct vhost_msg_v2 *new_msg = g_new(struct vhost_msg_v2, 1); + + *new_msg = msg; + g_async_queue_push(s->map_queue, new_msg); + + return; + } + trace_vhost_vdpa_listener_commit(s, fd, msg.type, msg.iotlb.type); if (write(fd, &msg, sizeof(msg)) != sizeof(msg)) { error_report("failed to write, fd=%d, errno=%d (%s)", @@ -1288,6 +1324,94 @@ static void vhost_vdpa_suspend(struct vhost_dev *dev) vhost_vdpa_reset_device(dev); } +static void *vhost_vdpa_load_map(void *opaque) +{ + VhostVDPAShared *shared = opaque; + GPtrArray *ret = NULL; + + while (true) { + g_autofree struct vhost_msg_v2 *msg = NULL; + void *job = g_async_queue_pop(shared->map_queue); + int r = 0; + + if (job == shared) { + /* exit signal */ + break; + } + + msg = job; + switch (msg->iotlb.type) { + case VHOST_IOTLB_UPDATE: + r = vhost_vdpa_dma_map(shared, msg->asid, msg->iotlb.iova, + msg->iotlb.size, + (void *)(uintptr_t)msg->iotlb.uaddr, + msg->iotlb.perm == VHOST_ACCESS_RO); + break; + case VHOST_IOTLB_INVALIDATE: + r = vhost_vdpa_dma_unmap(shared, msg->asid, msg->iotlb.iova, + msg->iotlb.size); + break; + case VHOST_IOTLB_BATCH_BEGIN: + vhost_vdpa_iotlb_batch_begin_once(shared); + break; + case VHOST_IOTLB_BATCH_END: + vhost_vdpa_dma_end_batch(shared); + break; + default: + error_report("Invalid IOTLB msg type %d", msg->iotlb.type); + break; + }; + + if (unlikely(r != 0)) { + /* Add to return value so we can remove it from iova_tree */ + if (ret == NULL) { + ret = g_ptr_array_new_full(0, g_free); + } + + g_ptr_array_add(ret, g_steal_pointer(&msg)); + } + } + + return ret; +} + +static void vhost_vdpa_spawn_maps_thread(VhostVDPAShared *shared) +{ + shared->map_queue = g_async_queue_new(); + qemu_thread_create(&shared->map_thread, "vdpa map thread", + vhost_vdpa_load_map, shared, QEMU_THREAD_JOINABLE); + shared->map_thread_enabled = true; +} + +static bool vhost_vdpa_join_maps_thread(VhostVDPAShared *shared) +{ + g_autoptr(GPtrArray) failed_iova = NULL; + + /* Signal end of offloading maps */ + g_async_queue_push(shared->map_queue, shared); + failed_iova = qemu_thread_join(&shared->map_thread); + g_async_queue_unref(shared->map_queue); + + shared->map_thread_enabled = false; + + if (likely(!failed_iova)) { + return true; + } + + /* If it is a failed IOVA, abort starting */ + for (size_t i = 0; failed_iova->len; ++i) { + struct vhost_msg_v2 *msg = g_ptr_array_index(failed_iova, i); + DMAMap mem_region = { + .iova = msg->iotlb.iova, + .size = msg->iotlb.size - 1, /* Inclusive */ + }; + + vhost_iova_tree_remove(shared->iova_tree, mem_region); + } + + return false; +} + static int vhost_vdpa_dev_start(struct vhost_dev *dev, bool started) { struct vhost_vdpa *v = dev->opaque; @@ -1315,7 +1439,15 @@ static int vhost_vdpa_dev_start(struct vhost_dev *dev, bool started) "IOMMU and try again"); return -1; } - memory_listener_register(&v->shared->listener, dev->vdev->dma_as); + if (!v->shared->listener_registered) { + memory_listener_register(&v->shared->listener, dev->vdev->dma_as); + v->shared->listener_registered = true; + } else { + ok = vhost_vdpa_join_maps_thread(v->shared); + if (unlikely(!ok)) { + goto out_stop; + } + } return vhost_vdpa_add_status(dev, VIRTIO_CONFIG_S_DRIVER_OK); } @@ -1340,6 +1472,8 @@ static void vhost_vdpa_reset_status(struct vhost_dev *dev) vhost_vdpa_add_status(dev, VIRTIO_CONFIG_S_ACKNOWLEDGE | VIRTIO_CONFIG_S_DRIVER); memory_listener_unregister(&v->shared->listener); + v->shared->listener_registered = false; + } static int vhost_vdpa_set_log_base(struct vhost_dev *dev, uint64_t base, @@ -1522,3 +1656,34 @@ const VhostOps vdpa_ops = { .vhost_set_config_call = vhost_vdpa_set_config_call, .vhost_reset_status = vhost_vdpa_reset_status, }; + +int vhost_vdpa_load_setup(VhostVDPAShared *shared, AddressSpace *dma_as) +{ + uint8_t s = VIRTIO_CONFIG_S_ACKNOWLEDGE | VIRTIO_CONFIG_S_DRIVER; + int r = ioctl(shared->device_fd, VHOST_VDPA_SET_STATUS, &s); + if (unlikely(r < 0)) { + return r; + } + + vhost_vdpa_spawn_maps_thread(shared); + memory_listener_register(&shared->listener, dma_as); + shared->listener_registered = true; + return 0; +} + +int vhost_vdpa_load_cleanup(VhostVDPAShared *shared, bool vhost_will_start) +{ + if (!shared->map_thread_enabled) { + return 0; + } + + if (vhost_will_start) { + /* + * Delegate the join of map thread to vhost_vdpa_dev_start, as it runs + * out of main qemu lock. + */ + return 0; + } + + return vhost_vdpa_join_maps_thread(shared) ? 0 : -1; +} From patchwork Tue Nov 28 10:43:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 13470960 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C45E0C4167B for ; Tue, 28 Nov 2023 10:44:41 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1r7vZC-0006or-PA; Tue, 28 Nov 2023 05:43:46 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r7vZB-0006oP-Fr for qemu-devel@nongnu.org; Tue, 28 Nov 2023 05:43:45 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r7vZ8-0005hE-HN for qemu-devel@nongnu.org; Tue, 28 Nov 2023 05:43:45 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1701168215; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Xzdm5YKmAcApcYJb/8xg+e8LPU0ALlufPyzIkHlXIls=; b=BCafkelNHgcWfgJv5P0uZupH+eHrb3qL5wlSAn55E3NjtOYeBJ1DAEHahNU8NldGy+AXtt 2rZS3iYcXZoiZaEkz/I5F3Patryfk0FCqPjhu61F14UAsHDXw56s16G9DsxxCP+qH+sU+H Dpuh7BoTA+Ucm+CH6Ir03nQPdTBnG8Q= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-542-TWrBiRRdPC6xv7FHK5ke8w-1; Tue, 28 Nov 2023 05:43:30 -0500 X-MC-Unique: TWrBiRRdPC6xv7FHK5ke8w-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 34F1D8309A3; Tue, 28 Nov 2023 10:43:30 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.193.65]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0096420268D8; Tue, 28 Nov 2023 10:43:27 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Gautam Dawar , Jason Wang , Zhu Lingshan , yin31149@gmail.com, Shannon Nelson , "Michael S. Tsirkin" , Dragos Tatulea , Yajun Wu , Juan Quintela , Laurent Vivier , Stefano Garzarella , Parav Pandit , Lei Yang , si-wei.liu@oracle.com Subject: [RFC PATCH v2 09/10] vdpa: add vhost_vdpa_net_load_setup NetClient callback Date: Tue, 28 Nov 2023 11:43:02 +0100 Message-Id: <20231128104303.3314000-10-eperezma@redhat.com> In-Reply-To: <20231128104303.3314000-1-eperezma@redhat.com> References: <20231128104303.3314000-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.4 Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org So the vDPA backend knows when a migration incoming starts. NicState argument is needed so we can get the dma address space. Signed-off-by: Eugenio Pérez --- RFC v2: * Solve git conflict with .set_steering_ebpf * Fix x-svq=on use case which did not allocated iova_tree. --- include/net/net.h | 6 ++++++ net/vhost-vdpa.c | 33 +++++++++++++++++++++++++++++++++ 2 files changed, 39 insertions(+) diff --git a/include/net/net.h b/include/net/net.h index ffbd2c8d56..68282dde31 100644 --- a/include/net/net.h +++ b/include/net/net.h @@ -42,6 +42,7 @@ typedef struct NICConf { /* Net clients */ +struct NICState; typedef void (NetPoll)(NetClientState *, bool enable); typedef bool (NetCanReceive)(NetClientState *); typedef int (NetStart)(NetClientState *); @@ -69,6 +70,9 @@ typedef void (SocketReadStateFinalize)(SocketReadState *rs); typedef void (NetAnnounce)(NetClientState *); typedef bool (SetSteeringEBPF)(NetClientState *, int); typedef bool (NetCheckPeerType)(NetClientState *, ObjectClass *, Error **); +/* This can be called before start & pair, so get also the peer */ +typedef int (NetMigrationLoadSetup)(NetClientState *, struct NICState *); +typedef int (NetMigrationLoadCleanup)(NetClientState *, struct NICState *); typedef struct NetClientInfo { NetClientDriver type; @@ -98,6 +102,8 @@ typedef struct NetClientInfo { NetAnnounce *announce; SetSteeringEBPF *set_steering_ebpf; NetCheckPeerType *check_peer_type; + NetMigrationLoadSetup *load_setup; + NetMigrationLoadCleanup *load_cleanup; } NetClientInfo; struct NetClientState { diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index a37de7860e..90f41280d2 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -407,6 +407,37 @@ static void vhost_vdpa_net_client_stop(NetClientState *nc) } } +static int vhost_vdpa_net_load_setup(NetClientState *nc, NICState *nic) +{ + VhostVDPAState *s = DO_UPCAST(VhostVDPAState, nc, nc); + VirtIONet *n = qemu_get_nic_opaque(&nic->ncs[0]); + VhostVDPAShared *shared = s->vhost_vdpa.shared; + int r; + + if (s->always_svq) { + /* iova tree is needed because of SVQ */ + shared->iova_tree = vhost_iova_tree_new(shared->iova_range.first, + shared->iova_range.last); + } + + r = vhost_vdpa_load_setup(shared, n->parent_obj.dma_as); + if (unlikely(r < 0)) { + g_clear_pointer(&s->vhost_vdpa.shared->iova_tree, + vhost_iova_tree_delete); + } + + return r; +} + +static int vhost_vdpa_net_load_cleanup(NetClientState *nc, NICState *nic) +{ + VhostVDPAState *s = DO_UPCAST(VhostVDPAState, nc, nc); + VirtIONet *n = qemu_get_nic_opaque(&nic->ncs[0]); + + return vhost_vdpa_load_cleanup(s->vhost_vdpa.shared, + n->parent_obj.status & VIRTIO_CONFIG_S_DRIVER_OK); +} + static NetClientInfo net_vhost_vdpa_info = { .type = NET_CLIENT_DRIVER_VHOST_VDPA, .size = sizeof(VhostVDPAState), @@ -419,6 +450,8 @@ static NetClientInfo net_vhost_vdpa_info = { .has_ufo = vhost_vdpa_has_ufo, .check_peer_type = vhost_vdpa_check_peer_type, .set_steering_ebpf = vhost_vdpa_set_steering_ebpf, + .load_setup = vhost_vdpa_net_load_setup, + .load_cleanup = vhost_vdpa_net_load_cleanup, }; static int64_t vhost_vdpa_get_vring_group(int device_fd, unsigned vq_index, From patchwork Tue Nov 28 10:43:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 13470961 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5372BC4167B for ; Tue, 28 Nov 2023 10:44:50 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1r7vZC-0006om-6Y; Tue, 28 Nov 2023 05:43:46 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r7vZA-0006nb-3H for qemu-devel@nongnu.org; Tue, 28 Nov 2023 05:43:44 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r7vZ7-0005hA-En for qemu-devel@nongnu.org; Tue, 28 Nov 2023 05:43:43 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1701168214; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Wj2KOCSd3gkqmbd2ClDeaOLOq55F9jaRkAO+a/H8Y0M=; b=QQQIOYOUtdGr0o7ptCRwlXVzp02reWtPAZnw5VmwsBBTuTsB3LPsy+mNjSK65M5VclWFG1 v1xQ+/ofCwdM/wCaX3h37rtLcbyqc4XW0vdDt7yg7yAlXGicrZqHnpLZCBQ0w3Mygc5K45 drXwq1b4vlxi2b7FSZXgE9fbTucKDLc= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-398-KUyxP7KWMa26lB9DH_zD5w-1; Tue, 28 Nov 2023 05:43:33 -0500 X-MC-Unique: KUyxP7KWMa26lB9DH_zD5w-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id A559481D871; Tue, 28 Nov 2023 10:43:32 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.193.65]) by smtp.corp.redhat.com (Postfix) with ESMTP id 70FFE20268D7; Tue, 28 Nov 2023 10:43:30 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Gautam Dawar , Jason Wang , Zhu Lingshan , yin31149@gmail.com, Shannon Nelson , "Michael S. Tsirkin" , Dragos Tatulea , Yajun Wu , Juan Quintela , Laurent Vivier , Stefano Garzarella , Parav Pandit , Lei Yang , si-wei.liu@oracle.com Subject: [RFC PATCH v2 10/10] virtio_net: register incremental migration handlers Date: Tue, 28 Nov 2023 11:43:03 +0100 Message-Id: <20231128104303.3314000-11-eperezma@redhat.com> In-Reply-To: <20231128104303.3314000-1-eperezma@redhat.com> References: <20231128104303.3314000-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.4 Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org This way VirtIONet can detect when the incoming migration starts. While registering in the backend (nc->peer) seems more logical, we need nic dma address space, and we cannot get it from the backend. Signed-off-by: Eugenio Pérez --- This could be done in vhost_vdpa or VirtIODevice struct, but future series will add state restore through CVQ so it's easier to start in VirtIONet directly. If we need to make this more generic, we can move to VirtIODevice and expose callbacks from VirtIONet class. Also, the pointer may not be the best id, but there are not a lot of things initialized in n. --- hw/net/virtio-net.c | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c index 80c56f0cfc..374d0b4ec8 100644 --- a/hw/net/virtio-net.c +++ b/hw/net/virtio-net.c @@ -38,6 +38,7 @@ #include "qapi/qapi-events-migration.h" #include "hw/virtio/virtio-access.h" #include "migration/misc.h" +#include "migration/register.h" #include "standard-headers/linux/ethtool.h" #include "sysemu/sysemu.h" #include "trace.h" @@ -3810,9 +3811,39 @@ static void virtio_net_device_unrealize(DeviceState *dev) virtio_cleanup(vdev); } +static int virtio_net_load_setup(QEMUFile *f, void *opaque) +{ + VirtIONet *n = opaque; + NetClientState *nc = qemu_get_subqueue(n->nic, 0); + + if (nc->peer->info->load_setup) { + return nc->peer->info->load_setup(nc->peer, n->nic); + } + + return 0; +} + +static int virtio_net_load_cleanup(void *opaque) +{ + VirtIONet *n = opaque; + NetClientState *nc = qemu_get_subqueue(n->nic, 0); + + if (nc->peer->info->load_cleanup) { + return nc->peer->info->load_cleanup(nc->peer, n->nic); + } + + return 0; +} + +static const SaveVMHandlers savevm_virtio_net_handlers = { + .load_setup = virtio_net_load_setup, + .load_cleanup = virtio_net_load_cleanup, +}; + static void virtio_net_instance_init(Object *obj) { VirtIONet *n = VIRTIO_NET(obj); + g_autoptr(GString) id = g_string_new(NULL); /* * The default config_size is sizeof(struct virtio_net_config). @@ -3824,6 +3855,10 @@ static void virtio_net_instance_init(Object *obj) DEVICE(n)); ebpf_rss_init(&n->ebpf_rss); + + g_string_printf(id, "%p", n); + register_savevm_live(id->str, VMSTATE_INSTANCE_ID_ANY, 1, + &savevm_virtio_net_handlers, n); } static int virtio_net_pre_save(void *opaque)