From patchwork Thu Nov 24 15:51:47 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 13055148 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CEDD4C433FE for ; Thu, 24 Nov 2022 15:53:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229805AbiKXPxR (ORCPT ); Thu, 24 Nov 2022 10:53:17 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38854 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229626AbiKXPxP (ORCPT ); Thu, 24 Nov 2022 10:53:15 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7BA4931EC9 for ; Thu, 24 Nov 2022 07:52:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1669305134; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=F+8Ig73/fLUeRTg3onTKDE4DpjNQE7LZwNbUPhE0FNg=; b=SZz5jnRl31S8cVizWh8UX2Cs7YFu4OqGuTejO5v5/98wl+Q0GQq1n9doXXAqgJh0/NP+FF Pm/xFtNfnvSeIPldsp7LlFlgdPUfwSso53wAIlGty6wHj7+3Pv0iq2MC01/Veh+mpgV8ep MjlkTP9h5zfJrNBosIpgNNVH84oXwfg= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-658-eNLmhS8vPZ-5GEI775XNGQ-1; Thu, 24 Nov 2022 10:52:10 -0500 X-MC-Unique: eNLmhS8vPZ-5GEI775XNGQ-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 8C54E800B23; Thu, 24 Nov 2022 15:52:09 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.169]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7A0F2111DCE6; Thu, 24 Nov 2022 15:52:06 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: kvm@vger.kernel.org, Cornelia Huck , Eli Cohen , Jason Wang , Cindy Lu , Parav Pandit , Laurent Vivier , "Michael S. Tsirkin" , Si-Wei Liu , Zhu Lingshan , Paolo Bonzini , Stefan Hajnoczi , Gautam Dawar , Liuxiangdong , "Gonglei (Arei)" , Harpreet Singh Anand , Stefano Garzarella Subject: [PATCH for 8.0 v8 01/12] vdpa: use v->shadow_vqs_enabled in vhost_vdpa_svqs_start & stop Date: Thu, 24 Nov 2022 16:51:47 +0100 Message-Id: <20221124155158.2109884-2-eperezma@redhat.com> In-Reply-To: <20221124155158.2109884-1-eperezma@redhat.com> References: <20221124155158.2109884-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org This function used to trust in v->shadow_vqs != NULL to know if it must start svq or not. This is not going to be valid anymore, as qemu is going to allocate svq array unconditionally (but it will only start them conditionally). Signed-off-by: Eugenio Pérez Acked-by: Jason Wang --- hw/virtio/vhost-vdpa.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 7468e44b87..7f0ff4df5b 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -1029,7 +1029,7 @@ static bool vhost_vdpa_svqs_start(struct vhost_dev *dev) Error *err = NULL; unsigned i; - if (!v->shadow_vqs) { + if (!v->shadow_vqs_enabled) { return true; } @@ -1082,7 +1082,7 @@ static void vhost_vdpa_svqs_stop(struct vhost_dev *dev) { struct vhost_vdpa *v = dev->opaque; - if (!v->shadow_vqs) { + if (!v->shadow_vqs_enabled) { return; } From patchwork Thu Nov 24 15:51:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 13055149 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6DA45C4321E for ; Thu, 24 Nov 2022 15:53:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229808AbiKXPxT (ORCPT ); Thu, 24 Nov 2022 10:53:19 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38866 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229772AbiKXPxP (ORCPT ); Thu, 24 Nov 2022 10:53:15 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A164812E213 for ; Thu, 24 Nov 2022 07:52:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1669305136; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TSjVQAN1hGLhWwbJytaWeNULUe7TeqdaxnpYJGUlvrs=; b=Jnh7Rc4f/h+PY++46THteTq+47z4gbPWNaJEqi1YVK103WpYXoogG3axBZBaw/hYnj1SIp loHVtVK4gSb2j/szGkgC+lusje6C25JIWbYy0Cme1pjpGAa6QQDDqT5JQ/5tXvVF6isNd3 joEsApUm7kX+0V0GDDZwSGsiyGXFc7A= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-65-JM78j4smMnO63DX_IuidHQ-1; Thu, 24 Nov 2022 10:52:13 -0500 X-MC-Unique: JM78j4smMnO63DX_IuidHQ-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id EB533833A06; Thu, 24 Nov 2022 15:52:12 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.169]) by smtp.corp.redhat.com (Postfix) with ESMTP id D3FE7111DCE6; Thu, 24 Nov 2022 15:52:09 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: kvm@vger.kernel.org, Cornelia Huck , Eli Cohen , Jason Wang , Cindy Lu , Parav Pandit , Laurent Vivier , "Michael S. Tsirkin" , Si-Wei Liu , Zhu Lingshan , Paolo Bonzini , Stefan Hajnoczi , Gautam Dawar , Liuxiangdong , "Gonglei (Arei)" , Harpreet Singh Anand , Stefano Garzarella Subject: [PATCH for 8.0 v8 02/12] vhost: set SVQ device call handler at SVQ start Date: Thu, 24 Nov 2022 16:51:48 +0100 Message-Id: <20221124155158.2109884-3-eperezma@redhat.com> In-Reply-To: <20221124155158.2109884-1-eperezma@redhat.com> References: <20221124155158.2109884-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org By the end of this series CVQ is shadowed as long as the features support it. Since we don't know at the beginning of qemu running if this is supported, move the event notifier handler setting to the start of the SVQ, instead of the start of qemu run. This will avoid to create them if the device does not support SVQ. Signed-off-by: Eugenio Pérez Acked-by: Jason Wang --- hw/virtio/vhost-shadow-virtqueue.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-virtqueue.c index 5bd14cad96..264ddc166d 100644 --- a/hw/virtio/vhost-shadow-virtqueue.c +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -648,6 +648,7 @@ void vhost_svq_start(VhostShadowVirtqueue *svq, VirtIODevice *vdev, { size_t desc_size, driver_size, device_size; + event_notifier_set_handler(&svq->hdev_call, vhost_svq_handle_call); svq->next_guest_avail_elem = NULL; svq->shadow_avail_idx = 0; svq->shadow_used_idx = 0; @@ -704,6 +705,7 @@ void vhost_svq_stop(VhostShadowVirtqueue *svq) g_free(svq->desc_state); qemu_vfree(svq->vring.desc); qemu_vfree(svq->vring.used); + event_notifier_set_handler(&svq->hdev_call, NULL); } /** @@ -740,7 +742,6 @@ VhostShadowVirtqueue *vhost_svq_new(VhostIOVATree *iova_tree, } event_notifier_init_fd(&svq->svq_kick, VHOST_FILE_UNBIND); - event_notifier_set_handler(&svq->hdev_call, vhost_svq_handle_call); svq->iova_tree = iova_tree; svq->ops = ops; svq->ops_opaque = ops_opaque; @@ -763,7 +764,6 @@ void vhost_svq_free(gpointer pvq) VhostShadowVirtqueue *vq = pvq; vhost_svq_stop(vq); event_notifier_cleanup(&vq->hdev_kick); - event_notifier_set_handler(&vq->hdev_call, NULL); event_notifier_cleanup(&vq->hdev_call); g_free(vq); } From patchwork Thu Nov 24 15:51:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 13055151 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 49142C43217 for ; Thu, 24 Nov 2022 15:53:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229825AbiKXPx2 (ORCPT ); Thu, 24 Nov 2022 10:53:28 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38884 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229847AbiKXPxZ (ORCPT ); Thu, 24 Nov 2022 10:53:25 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D10F0131833 for ; Thu, 24 Nov 2022 07:52:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1669305143; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wLU39RtuBP+GapQRx/xyrSR9LCcfBeflobv1obIr0ZE=; b=LG9VsmmDGj/CI1qQ5byh/ccnKWJ5/DItylwPuKXgDuLd1IXg3tOZt/okt36aiFA80qYbNj C2VU2NBTwLrdvSDnOwuQuAGoklNnQaoWoHldIWdOWmYi2OuKg1gEhT2+ett274VS/CltSR uv9dr3rWs/cRXyXCiPzQavoOKZB7L2E= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-319-CopEi7jLNsaV5SEWHQEFuA-1; Thu, 24 Nov 2022 10:52:17 -0500 X-MC-Unique: CopEi7jLNsaV5SEWHQEFuA-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 674341C05B0C; Thu, 24 Nov 2022 15:52:16 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.169]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3E2A1111DCE8; Thu, 24 Nov 2022 15:52:13 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: kvm@vger.kernel.org, Cornelia Huck , Eli Cohen , Jason Wang , Cindy Lu , Parav Pandit , Laurent Vivier , "Michael S. Tsirkin" , Si-Wei Liu , Zhu Lingshan , Paolo Bonzini , Stefan Hajnoczi , Gautam Dawar , Liuxiangdong , "Gonglei (Arei)" , Harpreet Singh Anand , Stefano Garzarella Subject: [PATCH for 8.0 v8 03/12] vhost: allocate SVQ device file descriptors at device start Date: Thu, 24 Nov 2022 16:51:49 +0100 Message-Id: <20221124155158.2109884-4-eperezma@redhat.com> In-Reply-To: <20221124155158.2109884-1-eperezma@redhat.com> References: <20221124155158.2109884-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The next patches will start control SVQ if possible. However, we don't know if that will be possible at qemu boot anymore. Delay device file descriptors until we know it at device start. This will avoid to create them if the device does not support SVQ. Signed-off-by: Eugenio Pérez Acked-by: Jason Wang --- hw/virtio/vhost-shadow-virtqueue.c | 31 ++------------------------ hw/virtio/vhost-vdpa.c | 35 ++++++++++++++++++++++++------ 2 files changed, 30 insertions(+), 36 deletions(-) diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-virtqueue.c index 264ddc166d..3b05bab44d 100644 --- a/hw/virtio/vhost-shadow-virtqueue.c +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -715,43 +715,18 @@ void vhost_svq_stop(VhostShadowVirtqueue *svq) * @iova_tree: Tree to perform descriptors translations * @ops: SVQ owner callbacks * @ops_opaque: ops opaque pointer - * - * Returns the new virtqueue or NULL. - * - * In case of error, reason is reported through error_report. */ VhostShadowVirtqueue *vhost_svq_new(VhostIOVATree *iova_tree, const VhostShadowVirtqueueOps *ops, void *ops_opaque) { - g_autofree VhostShadowVirtqueue *svq = g_new0(VhostShadowVirtqueue, 1); - int r; - - r = event_notifier_init(&svq->hdev_kick, 0); - if (r != 0) { - error_report("Couldn't create kick event notifier: %s (%d)", - g_strerror(errno), errno); - goto err_init_hdev_kick; - } - - r = event_notifier_init(&svq->hdev_call, 0); - if (r != 0) { - error_report("Couldn't create call event notifier: %s (%d)", - g_strerror(errno), errno); - goto err_init_hdev_call; - } + VhostShadowVirtqueue *svq = g_new0(VhostShadowVirtqueue, 1); event_notifier_init_fd(&svq->svq_kick, VHOST_FILE_UNBIND); svq->iova_tree = iova_tree; svq->ops = ops; svq->ops_opaque = ops_opaque; - return g_steal_pointer(&svq); - -err_init_hdev_call: - event_notifier_cleanup(&svq->hdev_kick); - -err_init_hdev_kick: - return NULL; + return svq; } /** @@ -763,7 +738,5 @@ void vhost_svq_free(gpointer pvq) { VhostShadowVirtqueue *vq = pvq; vhost_svq_stop(vq); - event_notifier_cleanup(&vq->hdev_kick); - event_notifier_cleanup(&vq->hdev_call); g_free(vq); } diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 7f0ff4df5b..3df2775760 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -428,15 +428,11 @@ static int vhost_vdpa_init_svq(struct vhost_dev *hdev, struct vhost_vdpa *v, shadow_vqs = g_ptr_array_new_full(hdev->nvqs, vhost_svq_free); for (unsigned n = 0; n < hdev->nvqs; ++n) { - g_autoptr(VhostShadowVirtqueue) svq; + VhostShadowVirtqueue *svq; svq = vhost_svq_new(v->iova_tree, v->shadow_vq_ops, v->shadow_vq_ops_opaque); - if (unlikely(!svq)) { - error_setg(errp, "Cannot create svq %u", n); - return -1; - } - g_ptr_array_add(shadow_vqs, g_steal_pointer(&svq)); + g_ptr_array_add(shadow_vqs, svq); } v->shadow_vqs = g_steal_pointer(&shadow_vqs); @@ -864,11 +860,23 @@ static int vhost_vdpa_svq_set_fds(struct vhost_dev *dev, const EventNotifier *event_notifier = &svq->hdev_kick; int r; + r = event_notifier_init(&svq->hdev_kick, 0); + if (r != 0) { + error_setg_errno(errp, -r, "Couldn't create kick event notifier"); + goto err_init_hdev_kick; + } + + r = event_notifier_init(&svq->hdev_call, 0); + if (r != 0) { + error_setg_errno(errp, -r, "Couldn't create call event notifier"); + goto err_init_hdev_call; + } + file.fd = event_notifier_get_fd(event_notifier); r = vhost_vdpa_set_vring_dev_kick(dev, &file); if (unlikely(r != 0)) { error_setg_errno(errp, -r, "Can't set device kick fd"); - return r; + goto err_init_set_dev_fd; } event_notifier = &svq->hdev_call; @@ -876,8 +884,18 @@ static int vhost_vdpa_svq_set_fds(struct vhost_dev *dev, r = vhost_vdpa_set_vring_dev_call(dev, &file); if (unlikely(r != 0)) { error_setg_errno(errp, -r, "Can't set device call fd"); + goto err_init_set_dev_fd; } + return 0; + +err_init_set_dev_fd: + event_notifier_set_handler(&svq->hdev_call, NULL); + +err_init_hdev_call: + event_notifier_cleanup(&svq->hdev_kick); + +err_init_hdev_kick: return r; } @@ -1089,6 +1107,9 @@ static void vhost_vdpa_svqs_stop(struct vhost_dev *dev) for (unsigned i = 0; i < v->shadow_vqs->len; ++i) { VhostShadowVirtqueue *svq = g_ptr_array_index(v->shadow_vqs, i); vhost_vdpa_svq_unmap_rings(dev, svq); + + event_notifier_cleanup(&svq->hdev_kick); + event_notifier_cleanup(&svq->hdev_call); } } From patchwork Thu Nov 24 15:51:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 13055150 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9BECEC43217 for ; Thu, 24 Nov 2022 15:53:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229832AbiKXPxW (ORCPT ); Thu, 24 Nov 2022 10:53:22 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38882 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229453AbiKXPxV (ORCPT ); Thu, 24 Nov 2022 10:53:21 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 568F6132F77 for ; Thu, 24 Nov 2022 07:52:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1669305143; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4FrNEK9BW9vrejue8PJ2vXmvbvlWh0jBFggXf0s4JdU=; b=HCVbRT0Q/maavS8zsccV/9CeEDMMT7RWpf/Az88mQpWNxl93KS1zxko5JbH0lVgLjAyx6f Yw5xg3mrX3Ksfhjhe3zYpdbUCVLfArSFczugE9nwMr/XfJVmo3FmUMvaLh78VryPs/Mz13 FU+qXqv2NdrMZhzXUbk7FqTQCZ8KM2s= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-154-FPglt4DFOp2_Nqm4mCBFmg-1; Thu, 24 Nov 2022 10:52:20 -0500 X-MC-Unique: FPglt4DFOp2_Nqm4mCBFmg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id BEBCE1C05B0F; Thu, 24 Nov 2022 15:52:19 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.169]) by smtp.corp.redhat.com (Postfix) with ESMTP id AE661111DCE6; Thu, 24 Nov 2022 15:52:16 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: kvm@vger.kernel.org, Cornelia Huck , Eli Cohen , Jason Wang , Cindy Lu , Parav Pandit , Laurent Vivier , "Michael S. Tsirkin" , Si-Wei Liu , Zhu Lingshan , Paolo Bonzini , Stefan Hajnoczi , Gautam Dawar , Liuxiangdong , "Gonglei (Arei)" , Harpreet Singh Anand , Stefano Garzarella Subject: [PATCH for 8.0 v8 04/12] vhost: move iova_tree set to vhost_svq_start Date: Thu, 24 Nov 2022 16:51:50 +0100 Message-Id: <20221124155158.2109884-5-eperezma@redhat.com> In-Reply-To: <20221124155158.2109884-1-eperezma@redhat.com> References: <20221124155158.2109884-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Since we don't know if we will use SVQ at qemu initialization, let's allocate iova_tree only if needed. To do so, accept it at SVQ start, not at initialization. This will avoid to create it if the device does not support SVQ. Signed-off-by: Eugenio Pérez Acked-by: Jason Wang --- hw/virtio/vhost-shadow-virtqueue.h | 5 ++--- hw/virtio/vhost-shadow-virtqueue.c | 9 ++++----- hw/virtio/vhost-vdpa.c | 5 ++--- 3 files changed, 8 insertions(+), 11 deletions(-) diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-virtqueue.h index d04c34a589..926a4897b1 100644 --- a/hw/virtio/vhost-shadow-virtqueue.h +++ b/hw/virtio/vhost-shadow-virtqueue.h @@ -126,11 +126,10 @@ size_t vhost_svq_driver_area_size(const VhostShadowVirtqueue *svq); size_t vhost_svq_device_area_size(const VhostShadowVirtqueue *svq); void vhost_svq_start(VhostShadowVirtqueue *svq, VirtIODevice *vdev, - VirtQueue *vq); + VirtQueue *vq, VhostIOVATree *iova_tree); void vhost_svq_stop(VhostShadowVirtqueue *svq); -VhostShadowVirtqueue *vhost_svq_new(VhostIOVATree *iova_tree, - const VhostShadowVirtqueueOps *ops, +VhostShadowVirtqueue *vhost_svq_new(const VhostShadowVirtqueueOps *ops, void *ops_opaque); void vhost_svq_free(gpointer vq); diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-virtqueue.c index 3b05bab44d..4307296358 100644 --- a/hw/virtio/vhost-shadow-virtqueue.c +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -642,9 +642,10 @@ void vhost_svq_set_svq_kick_fd(VhostShadowVirtqueue *svq, int svq_kick_fd) * @svq: Shadow Virtqueue * @vdev: VirtIO device * @vq: Virtqueue to shadow + * @iova_tree: Tree to perform descriptors translations */ void vhost_svq_start(VhostShadowVirtqueue *svq, VirtIODevice *vdev, - VirtQueue *vq) + VirtQueue *vq, VhostIOVATree *iova_tree) { size_t desc_size, driver_size, device_size; @@ -655,6 +656,7 @@ void vhost_svq_start(VhostShadowVirtqueue *svq, VirtIODevice *vdev, svq->last_used_idx = 0; svq->vdev = vdev; svq->vq = vq; + svq->iova_tree = iova_tree; svq->vring.num = virtio_queue_get_num(vdev, virtio_get_queue_index(vq)); driver_size = vhost_svq_driver_area_size(svq); @@ -712,18 +714,15 @@ void vhost_svq_stop(VhostShadowVirtqueue *svq) * Creates vhost shadow virtqueue, and instructs the vhost device to use the * shadow methods and file descriptors. * - * @iova_tree: Tree to perform descriptors translations * @ops: SVQ owner callbacks * @ops_opaque: ops opaque pointer */ -VhostShadowVirtqueue *vhost_svq_new(VhostIOVATree *iova_tree, - const VhostShadowVirtqueueOps *ops, +VhostShadowVirtqueue *vhost_svq_new(const VhostShadowVirtqueueOps *ops, void *ops_opaque) { VhostShadowVirtqueue *svq = g_new0(VhostShadowVirtqueue, 1); event_notifier_init_fd(&svq->svq_kick, VHOST_FILE_UNBIND); - svq->iova_tree = iova_tree; svq->ops = ops; svq->ops_opaque = ops_opaque; return svq; diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 3df2775760..691bcc811a 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -430,8 +430,7 @@ static int vhost_vdpa_init_svq(struct vhost_dev *hdev, struct vhost_vdpa *v, for (unsigned n = 0; n < hdev->nvqs; ++n) { VhostShadowVirtqueue *svq; - svq = vhost_svq_new(v->iova_tree, v->shadow_vq_ops, - v->shadow_vq_ops_opaque); + svq = vhost_svq_new(v->shadow_vq_ops, v->shadow_vq_ops_opaque); g_ptr_array_add(shadow_vqs, svq); } @@ -1063,7 +1062,7 @@ static bool vhost_vdpa_svqs_start(struct vhost_dev *dev) goto err; } - vhost_svq_start(svq, dev->vdev, vq); + vhost_svq_start(svq, dev->vdev, vq, v->iova_tree); ok = vhost_vdpa_svq_map_rings(dev, svq, &addr, &err); if (unlikely(!ok)) { goto err_map; From patchwork Thu Nov 24 15:51:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 13055152 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7F4BCC433FE for ; Thu, 24 Nov 2022 15:53:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229450AbiKXPxc (ORCPT ); Thu, 24 Nov 2022 10:53:32 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39172 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229835AbiKXPxb (ORCPT ); Thu, 24 Nov 2022 10:53:31 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C2828E9C for ; Thu, 24 Nov 2022 07:52:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1669305148; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=t+qR/RbRnzTmnM7TqHoM4ncCb2+5woArLJeALcMnuIs=; b=IIiWumtQO68a0nfxWkaJvBdppHBT9QhcGBgqXFEbDNbOQA/+BfdO8TFreQ3OXfrIi0O+Rx YeS7DgPlE+9FbJdZ1uo05K2VKBE2WUHMso5mP+oRQnb6buF6rL7ErS6xKVpMVag1BZHieO U5Mz5u9dpULpOgeBBOcDmoDi+0lbUMI= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-207-SCsHdSB7Nx-vINcTA543gw-1; Thu, 24 Nov 2022 10:52:23 -0500 X-MC-Unique: SCsHdSB7Nx-vINcTA543gw-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 2FFCB101A528; Thu, 24 Nov 2022 15:52:23 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.169]) by smtp.corp.redhat.com (Postfix) with ESMTP id 12585111E3F8; Thu, 24 Nov 2022 15:52:19 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: kvm@vger.kernel.org, Cornelia Huck , Eli Cohen , Jason Wang , Cindy Lu , Parav Pandit , Laurent Vivier , "Michael S. Tsirkin" , Si-Wei Liu , Zhu Lingshan , Paolo Bonzini , Stefan Hajnoczi , Gautam Dawar , Liuxiangdong , "Gonglei (Arei)" , Harpreet Singh Anand , Stefano Garzarella Subject: [PATCH for 8.0 v8 05/12] vdpa: add vhost_vdpa_net_valid_svq_features Date: Thu, 24 Nov 2022 16:51:51 +0100 Message-Id: <20221124155158.2109884-6-eperezma@redhat.com> In-Reply-To: <20221124155158.2109884-1-eperezma@redhat.com> References: <20221124155158.2109884-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org It will be reused at vdpa device start so let's extract in its own function. Signed-off-by: Eugenio Pérez Acked-by: Jason Wang --- net/vhost-vdpa.c | 26 +++++++++++++++++--------- 1 file changed, 17 insertions(+), 9 deletions(-) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index 2b4b85d8f8..88e0eec5fa 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -107,6 +107,22 @@ VHostNetState *vhost_vdpa_get_vhost_net(NetClientState *nc) return s->vhost_net; } +static bool vhost_vdpa_net_valid_svq_features(uint64_t features, Error **errp) +{ + uint64_t invalid_dev_features = + features & ~vdpa_svq_device_features & + /* Transport are all accepted at this point */ + ~MAKE_64BIT_MASK(VIRTIO_TRANSPORT_F_START, + VIRTIO_TRANSPORT_F_END - VIRTIO_TRANSPORT_F_START); + + if (invalid_dev_features) { + error_setg(errp, "vdpa svq does not work with features 0x%" PRIx64, + invalid_dev_features); + } + + return !invalid_dev_features; +} + static int vhost_vdpa_net_check_device_id(struct vhost_net *net) { uint32_t device_id; @@ -676,15 +692,7 @@ int net_init_vhost_vdpa(const Netdev *netdev, const char *name, if (opts->x_svq) { struct vhost_vdpa_iova_range iova_range; - uint64_t invalid_dev_features = - features & ~vdpa_svq_device_features & - /* Transport are all accepted at this point */ - ~MAKE_64BIT_MASK(VIRTIO_TRANSPORT_F_START, - VIRTIO_TRANSPORT_F_END - VIRTIO_TRANSPORT_F_START); - - if (invalid_dev_features) { - error_setg(errp, "vdpa svq does not work with features 0x%" PRIx64, - invalid_dev_features); + if (!vhost_vdpa_net_valid_svq_features(features, errp)) { goto err_svq; } From patchwork Thu Nov 24 15:51:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 13055153 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 29ED4C433FE for ; Thu, 24 Nov 2022 15:53:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229847AbiKXPxf (ORCPT ); Thu, 24 Nov 2022 10:53:35 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39174 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229580AbiKXPxe (ORCPT ); Thu, 24 Nov 2022 10:53:34 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3F054F25 for ; Thu, 24 Nov 2022 07:52:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1669305148; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WhSLPDhzhFxcNenczXsbaDgqHh/5TPD4sy5gJvmX1zw=; b=VR2ZDb2xynfwkTZ9VXg4tnd0wzINhKZC8KmE7tJt31Zy6imTQI53nRwiJQ5+Vl2+JX99Rg JxLPZHndpE0UXjLjJzt0iviHI2bJm2ktiWfjpF7abGSJN3jBG6L9yfLDz4NJ+f2gGpTIr/ ksRA4TcMilqimCsW7k+3OfONiduuXGE= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-450-EUNe_WFbMRmg49YabOmcBw-1; Thu, 24 Nov 2022 10:52:27 -0500 X-MC-Unique: EUNe_WFbMRmg49YabOmcBw-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 86A9B811E75; Thu, 24 Nov 2022 15:52:26 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.169]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7706F111E3F8; Thu, 24 Nov 2022 15:52:23 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: kvm@vger.kernel.org, Cornelia Huck , Eli Cohen , Jason Wang , Cindy Lu , Parav Pandit , Laurent Vivier , "Michael S. Tsirkin" , Si-Wei Liu , Zhu Lingshan , Paolo Bonzini , Stefan Hajnoczi , Gautam Dawar , Liuxiangdong , "Gonglei (Arei)" , Harpreet Singh Anand , Stefano Garzarella Subject: [PATCH for 8.0 v8 06/12] vdpa: extract vhost_vdpa_svq_allocate_iova_tree Date: Thu, 24 Nov 2022 16:51:52 +0100 Message-Id: <20221124155158.2109884-7-eperezma@redhat.com> In-Reply-To: <20221124155158.2109884-1-eperezma@redhat.com> References: <20221124155158.2109884-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org It can be allocated either if all virtqueues must be shadowed or if vdpa-net detects it can shadow only cvq. Extract in its own function so we can reuse it. Signed-off-by: Eugenio Pérez --- net/vhost-vdpa.c | 29 +++++++++++++++++------------ 1 file changed, 17 insertions(+), 12 deletions(-) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index 88e0eec5fa..9ee3bc4cd3 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -240,6 +240,22 @@ static NetClientInfo net_vhost_vdpa_info = { .check_peer_type = vhost_vdpa_check_peer_type, }; +static int vhost_vdpa_get_iova_range(int fd, + struct vhost_vdpa_iova_range *iova_range) +{ + int ret = ioctl(fd, VHOST_VDPA_GET_IOVA_RANGE, iova_range); + + return ret < 0 ? -errno : 0; +} + +static VhostIOVATree *vhost_vdpa_svq_allocate_iova_tree(int vdpa_device_fd) +{ + struct vhost_vdpa_iova_range iova_range; + + vhost_vdpa_get_iova_range(vdpa_device_fd, &iova_range); + return vhost_iova_tree_new(iova_range.first, iova_range.last); +} + static void vhost_vdpa_cvq_unmap_buf(struct vhost_vdpa *v, void *addr) { VhostIOVATree *tree = v->iova_tree; @@ -587,14 +603,6 @@ static NetClientState *net_vhost_vdpa_init(NetClientState *peer, return nc; } -static int vhost_vdpa_get_iova_range(int fd, - struct vhost_vdpa_iova_range *iova_range) -{ - int ret = ioctl(fd, VHOST_VDPA_GET_IOVA_RANGE, iova_range); - - return ret < 0 ? -errno : 0; -} - static int vhost_vdpa_get_features(int fd, uint64_t *features, Error **errp) { int ret = ioctl(fd, VHOST_GET_FEATURES, features); @@ -690,14 +698,11 @@ int net_init_vhost_vdpa(const Netdev *netdev, const char *name, } if (opts->x_svq) { - struct vhost_vdpa_iova_range iova_range; - if (!vhost_vdpa_net_valid_svq_features(features, errp)) { goto err_svq; } - vhost_vdpa_get_iova_range(vdpa_device_fd, &iova_range); - iova_tree = vhost_iova_tree_new(iova_range.first, iova_range.last); + iova_tree = vhost_vdpa_svq_allocate_iova_tree(vdpa_device_fd); } ncs = g_malloc0(sizeof(*ncs) * queue_pairs); From patchwork Thu Nov 24 15:51:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 13055154 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1D253C43217 for ; Thu, 24 Nov 2022 15:53:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229852AbiKXPxj (ORCPT ); Thu, 24 Nov 2022 10:53:39 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39444 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229870AbiKXPxg (ORCPT ); Thu, 24 Nov 2022 10:53:36 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D3E3EE017 for ; Thu, 24 Nov 2022 07:52:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1669305156; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xfr4y8K+ipo09Wo9rlQQiQ9rlXW0vTMWU5HjYZ33inE=; b=WyBJQS6ypNhUaficbM9X/ECGBl0GsXCnFeDWrnRzh43I2GrGaLIl4CqiMbGUgmLAwZVH3W vwgufqo4jxD9EMYL3Hy7utmQkOS/WI35MD9hOu4It+1jv2FXFwZWvsMmBPMKM7nvhHWof/ D0BkhOPSlJ4rivchjsUhKuMlR/K50fY= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-426-WmSAaO1lMqm6oUEvoR0aAg-1; Thu, 24 Nov 2022 10:52:30 -0500 X-MC-Unique: WmSAaO1lMqm6oUEvoR0aAg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E0F4A381A727; Thu, 24 Nov 2022 15:52:29 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.169]) by smtp.corp.redhat.com (Postfix) with ESMTP id CE615111E3F8; Thu, 24 Nov 2022 15:52:26 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: kvm@vger.kernel.org, Cornelia Huck , Eli Cohen , Jason Wang , Cindy Lu , Parav Pandit , Laurent Vivier , "Michael S. Tsirkin" , Si-Wei Liu , Zhu Lingshan , Paolo Bonzini , Stefan Hajnoczi , Gautam Dawar , Liuxiangdong , "Gonglei (Arei)" , Harpreet Singh Anand , Stefano Garzarella Subject: [PATCH for 8.0 v8 07/12] vdpa: move SVQ vring features check to net/ Date: Thu, 24 Nov 2022 16:51:53 +0100 Message-Id: <20221124155158.2109884-8-eperezma@redhat.com> In-Reply-To: <20221124155158.2109884-1-eperezma@redhat.com> References: <20221124155158.2109884-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The next patches will start control SVQ if possible. However, we don't know if that will be possible at qemu boot anymore. Since the moved checks will be already evaluated at net/ to know if it is ok to shadow CVQ, move them. Signed-off-by: Eugenio Pérez Acked-by: Jason Wang --- hw/virtio/vhost-vdpa.c | 33 ++------------------------------- net/vhost-vdpa.c | 3 ++- 2 files changed, 4 insertions(+), 32 deletions(-) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 691bcc811a..6f2dabd0bf 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -402,29 +402,9 @@ static int vhost_vdpa_get_dev_features(struct vhost_dev *dev, return ret; } -static int vhost_vdpa_init_svq(struct vhost_dev *hdev, struct vhost_vdpa *v, - Error **errp) +static void vhost_vdpa_init_svq(struct vhost_dev *hdev, struct vhost_vdpa *v) { g_autoptr(GPtrArray) shadow_vqs = NULL; - uint64_t dev_features, svq_features; - int r; - bool ok; - - if (!v->shadow_vqs_enabled) { - return 0; - } - - r = vhost_vdpa_get_dev_features(hdev, &dev_features); - if (r != 0) { - error_setg_errno(errp, -r, "Can't get vdpa device features"); - return r; - } - - svq_features = dev_features; - ok = vhost_svq_valid_features(svq_features, errp); - if (unlikely(!ok)) { - return -1; - } shadow_vqs = g_ptr_array_new_full(hdev->nvqs, vhost_svq_free); for (unsigned n = 0; n < hdev->nvqs; ++n) { @@ -435,7 +415,6 @@ static int vhost_vdpa_init_svq(struct vhost_dev *hdev, struct vhost_vdpa *v, } v->shadow_vqs = g_steal_pointer(&shadow_vqs); - return 0; } static int vhost_vdpa_init(struct vhost_dev *dev, void *opaque, Error **errp) @@ -460,11 +439,7 @@ static int vhost_vdpa_init(struct vhost_dev *dev, void *opaque, Error **errp) dev->opaque = opaque ; v->listener = vhost_vdpa_memory_listener; v->msg_type = VHOST_IOTLB_MSG_V2; - ret = vhost_vdpa_init_svq(dev, v, errp); - if (ret) { - goto err; - } - + vhost_vdpa_init_svq(dev, v); vhost_vdpa_get_iova_range(v); if (!vhost_vdpa_first_dev(dev)) { @@ -475,10 +450,6 @@ static int vhost_vdpa_init(struct vhost_dev *dev, void *opaque, Error **errp) VIRTIO_CONFIG_S_DRIVER); return 0; - -err: - ram_block_discard_disable(false); - return ret; } static void vhost_vdpa_host_notifier_uninit(struct vhost_dev *dev, diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index 9ee3bc4cd3..b47acf5a2d 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -118,9 +118,10 @@ static bool vhost_vdpa_net_valid_svq_features(uint64_t features, Error **errp) if (invalid_dev_features) { error_setg(errp, "vdpa svq does not work with features 0x%" PRIx64, invalid_dev_features); + return false; } - return !invalid_dev_features; + return vhost_svq_valid_features(features, errp); } static int vhost_vdpa_net_check_device_id(struct vhost_net *net) From patchwork Thu Nov 24 15:51:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 13055155 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1BA13C4321E for ; Thu, 24 Nov 2022 15:53:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229895AbiKXPxl (ORCPT ); Thu, 24 Nov 2022 10:53:41 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39446 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229888AbiKXPxi (ORCPT ); Thu, 24 Nov 2022 10:53:38 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E1CED10060 for ; Thu, 24 Nov 2022 07:52:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1669305157; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vUhr69+TgZmndftT7btVTmABb13n2gplbiYWMXh4Xjw=; b=Mr0R/6hR0wOwzM4EkccLI3nvCPSzsTIFlBOQif4b9joSTLdaWHs1fU0udND+pxNWX1A3Ni L6kHzu8kRdQOhLnIKzZJD9ds/kl1FUh6P5EM8x6lg9aYxnzmE4OwNKE77KfiK9Sw+XtZUy 6V56zpz5QLp7KnODQIKOiz9uYe3eZfs= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-43-8pIWLS0SPW2xxUKQUZ5Lrg-1; Thu, 24 Nov 2022 10:52:33 -0500 X-MC-Unique: 8pIWLS0SPW2xxUKQUZ5Lrg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 4654C1C05EC0; Thu, 24 Nov 2022 15:52:33 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.169]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3450B111F3B6; Thu, 24 Nov 2022 15:52:30 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: kvm@vger.kernel.org, Cornelia Huck , Eli Cohen , Jason Wang , Cindy Lu , Parav Pandit , Laurent Vivier , "Michael S. Tsirkin" , Si-Wei Liu , Zhu Lingshan , Paolo Bonzini , Stefan Hajnoczi , Gautam Dawar , Liuxiangdong , "Gonglei (Arei)" , Harpreet Singh Anand , Stefano Garzarella Subject: [PATCH for 8.0 v8 08/12] vdpa: allocate SVQ array unconditionally Date: Thu, 24 Nov 2022 16:51:54 +0100 Message-Id: <20221124155158.2109884-9-eperezma@redhat.com> In-Reply-To: <20221124155158.2109884-1-eperezma@redhat.com> References: <20221124155158.2109884-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org SVQ may run or not in a device depending on runtime conditions (for example, if the device can move CVQ to its own group or not). Allocate the SVQ array unconditionally at startup, since its hard to move this allocation elsewhere. Signed-off-by: Eugenio Pérez Acked-by: Jason Wang --- hw/virtio/vhost-vdpa.c | 4 ---- 1 file changed, 4 deletions(-) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 6f2dabd0bf..ed360b7fdf 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -546,10 +546,6 @@ static void vhost_vdpa_svq_cleanup(struct vhost_dev *dev) struct vhost_vdpa *v = dev->opaque; size_t idx; - if (!v->shadow_vqs) { - return; - } - for (idx = 0; idx < v->shadow_vqs->len; ++idx) { vhost_svq_stop(g_ptr_array_index(v->shadow_vqs, idx)); } From patchwork Thu Nov 24 15:51:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 13055156 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 71C2DC43217 for ; Thu, 24 Nov 2022 15:53:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229920AbiKXPxq (ORCPT ); Thu, 24 Nov 2022 10:53:46 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39562 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229890AbiKXPxk (ORCPT ); Thu, 24 Nov 2022 10:53:40 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 92A071BE8C for ; Thu, 24 Nov 2022 07:52:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1669305160; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xwqiEr5SlgEI/MINJPyBvBK8+yHdIggDrwdzZzipAcU=; b=H2Y9r9f0AZNM3jbL7J/UKfGQh2ft8lH5Mla5rGoG3nrMqlppg5pmotdWjCfysRERLWWav7 gd63hGjtEZX118u9Z3jq6JFvVsw9k9vIoGLT++cIRGjnGGofpCdOyc1cyPnBmL0FGqXtFt gFxpzZvejbcSOR33+yN64cDmacZEtS4= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-613-3YYXfx8XOPaoxwBM90dYBQ-1; Thu, 24 Nov 2022 10:52:37 -0500 X-MC-Unique: 3YYXfx8XOPaoxwBM90dYBQ-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 9E98F380671D; Thu, 24 Nov 2022 15:52:36 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.169]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8D735111E403; Thu, 24 Nov 2022 15:52:33 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: kvm@vger.kernel.org, Cornelia Huck , Eli Cohen , Jason Wang , Cindy Lu , Parav Pandit , Laurent Vivier , "Michael S. Tsirkin" , Si-Wei Liu , Zhu Lingshan , Paolo Bonzini , Stefan Hajnoczi , Gautam Dawar , Liuxiangdong , "Gonglei (Arei)" , Harpreet Singh Anand , Stefano Garzarella Subject: [PATCH for 8.0 v8 09/12] vdpa: add asid parameter to vhost_vdpa_dma_map/unmap Date: Thu, 24 Nov 2022 16:51:55 +0100 Message-Id: <20221124155158.2109884-10-eperezma@redhat.com> In-Reply-To: <20221124155158.2109884-1-eperezma@redhat.com> References: <20221124155158.2109884-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org So the caller can choose which ASID is destined. No need to update the batch functions as they will always be called from memory listener updates at the moment. Memory listener updates will always update ASID 0, as it's the passthrough ASID. All vhost devices's ASID are 0 at this moment. Signed-off-by: Eugenio Pérez Acked-by: Jason Wang --- v7: * Move comment on zero initailization of vhost_vdpa_dma_map above the functions. * Add VHOST_VDPA_GUEST_PA_ASID macro. v5: * Solve conflict, now vhost_vdpa_svq_unmap_ring returns void * Change comment on zero initialization. v4: Add comment specifying behavior if device does not support _F_ASID v3: Deleted unneeded space --- include/hw/virtio/vhost-vdpa.h | 14 ++++++++++--- hw/virtio/vhost-vdpa.c | 36 +++++++++++++++++++++++----------- net/vhost-vdpa.c | 6 +++--- hw/virtio/trace-events | 4 ++-- 4 files changed, 41 insertions(+), 19 deletions(-) diff --git a/include/hw/virtio/vhost-vdpa.h b/include/hw/virtio/vhost-vdpa.h index 1111d85643..e57dfa1fd1 100644 --- a/include/hw/virtio/vhost-vdpa.h +++ b/include/hw/virtio/vhost-vdpa.h @@ -19,6 +19,12 @@ #include "hw/virtio/virtio.h" #include "standard-headers/linux/vhost_types.h" +/* + * ASID dedicated to map guest's addresses. If SVQ is disabled it maps GPA to + * qemu's IOVA. If SVQ is enabled it maps also the SVQ vring here + */ +#define VHOST_VDPA_GUEST_PA_ASID 0 + typedef struct VhostVDPAHostNotifier { MemoryRegion mr; void *addr; @@ -29,6 +35,7 @@ typedef struct vhost_vdpa { int index; uint32_t msg_type; bool iotlb_batch_begin_sent; + uint32_t address_space_id; MemoryListener listener; struct vhost_vdpa_iova_range iova_range; uint64_t acked_features; @@ -42,8 +49,9 @@ typedef struct vhost_vdpa { VhostVDPAHostNotifier notifier[VIRTIO_QUEUE_MAX]; } VhostVDPA; -int vhost_vdpa_dma_map(struct vhost_vdpa *v, hwaddr iova, hwaddr size, - void *vaddr, bool readonly); -int vhost_vdpa_dma_unmap(struct vhost_vdpa *v, hwaddr iova, hwaddr size); +int vhost_vdpa_dma_map(struct vhost_vdpa *v, uint32_t asid, hwaddr iova, + hwaddr size, void *vaddr, bool readonly); +int vhost_vdpa_dma_unmap(struct vhost_vdpa *v, uint32_t asid, hwaddr iova, + hwaddr size); #endif diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index ed360b7fdf..1100cde29e 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -72,22 +72,28 @@ static bool vhost_vdpa_listener_skipped_section(MemoryRegionSection *section, return false; } -int vhost_vdpa_dma_map(struct vhost_vdpa *v, hwaddr iova, hwaddr size, - void *vaddr, bool readonly) +/* + * The caller must set asid = 0 if the device does not support asid. + * This is not an ABI break since it is set to 0 by the initializer anyway. + */ +int vhost_vdpa_dma_map(struct vhost_vdpa *v, uint32_t asid, hwaddr iova, + hwaddr size, void *vaddr, bool readonly) { struct vhost_msg_v2 msg = {}; int fd = v->device_fd; int ret = 0; msg.type = v->msg_type; + msg.asid = asid; msg.iotlb.iova = iova; msg.iotlb.size = size; msg.iotlb.uaddr = (uint64_t)(uintptr_t)vaddr; msg.iotlb.perm = readonly ? VHOST_ACCESS_RO : VHOST_ACCESS_RW; msg.iotlb.type = VHOST_IOTLB_UPDATE; - trace_vhost_vdpa_dma_map(v, fd, msg.type, msg.iotlb.iova, msg.iotlb.size, - msg.iotlb.uaddr, msg.iotlb.perm, msg.iotlb.type); + trace_vhost_vdpa_dma_map(v, fd, msg.type, msg.asid, msg.iotlb.iova, + msg.iotlb.size, msg.iotlb.uaddr, msg.iotlb.perm, + msg.iotlb.type); if (write(fd, &msg, sizeof(msg)) != sizeof(msg)) { error_report("failed to write, fd=%d, errno=%d (%s)", @@ -98,18 +104,24 @@ int vhost_vdpa_dma_map(struct vhost_vdpa *v, hwaddr iova, hwaddr size, return ret; } -int vhost_vdpa_dma_unmap(struct vhost_vdpa *v, hwaddr iova, hwaddr size) +/* + * The caller must set asid = 0 if the device does not support asid. + * This is not an ABI break since it is set to 0 by the initializer anyway. + */ +int vhost_vdpa_dma_unmap(struct vhost_vdpa *v, uint32_t asid, hwaddr iova, + hwaddr size) { struct vhost_msg_v2 msg = {}; int fd = v->device_fd; int ret = 0; msg.type = v->msg_type; + msg.asid = asid; msg.iotlb.iova = iova; msg.iotlb.size = size; msg.iotlb.type = VHOST_IOTLB_INVALIDATE; - trace_vhost_vdpa_dma_unmap(v, fd, msg.type, msg.iotlb.iova, + trace_vhost_vdpa_dma_unmap(v, fd, msg.type, msg.asid, msg.iotlb.iova, msg.iotlb.size, msg.iotlb.type); if (write(fd, &msg, sizeof(msg)) != sizeof(msg)) { @@ -229,8 +241,8 @@ static void vhost_vdpa_listener_region_add(MemoryListener *listener, } vhost_vdpa_iotlb_batch_begin_once(v); - ret = vhost_vdpa_dma_map(v, iova, int128_get64(llsize), - vaddr, section->readonly); + ret = vhost_vdpa_dma_map(v, VHOST_VDPA_GUEST_PA_ASID, iova, + int128_get64(llsize), vaddr, section->readonly); if (ret) { error_report("vhost vdpa map fail!"); goto fail_map; @@ -303,7 +315,8 @@ static void vhost_vdpa_listener_region_del(MemoryListener *listener, vhost_iova_tree_remove(v->iova_tree, *result); } vhost_vdpa_iotlb_batch_begin_once(v); - ret = vhost_vdpa_dma_unmap(v, iova, int128_get64(llsize)); + ret = vhost_vdpa_dma_unmap(v, VHOST_VDPA_GUEST_PA_ASID, iova, + int128_get64(llsize)); if (ret) { error_report("vhost_vdpa dma unmap error!"); } @@ -883,7 +896,7 @@ static void vhost_vdpa_svq_unmap_ring(struct vhost_vdpa *v, hwaddr addr) } size = ROUND_UP(result->size, qemu_real_host_page_size()); - r = vhost_vdpa_dma_unmap(v, result->iova, size); + r = vhost_vdpa_dma_unmap(v, v->address_space_id, result->iova, size); if (unlikely(r < 0)) { error_report("Unable to unmap SVQ vring: %s (%d)", g_strerror(-r), -r); return; @@ -923,7 +936,8 @@ static bool vhost_vdpa_svq_map_ring(struct vhost_vdpa *v, DMAMap *needle, return false; } - r = vhost_vdpa_dma_map(v, needle->iova, needle->size + 1, + r = vhost_vdpa_dma_map(v, v->address_space_id, needle->iova, + needle->size + 1, (void *)(uintptr_t)needle->translated_addr, needle->perm == IOMMU_RO); if (unlikely(r != 0)) { diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index b47acf5a2d..b40b1ddf0d 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -275,7 +275,7 @@ static void vhost_vdpa_cvq_unmap_buf(struct vhost_vdpa *v, void *addr) return; } - r = vhost_vdpa_dma_unmap(v, map->iova, map->size + 1); + r = vhost_vdpa_dma_unmap(v, v->address_space_id, map->iova, map->size + 1); if (unlikely(r != 0)) { error_report("Device cannot unmap: %s(%d)", g_strerror(r), r); } @@ -315,8 +315,8 @@ static int vhost_vdpa_cvq_map_buf(struct vhost_vdpa *v, void *buf, size_t size, return r; } - r = vhost_vdpa_dma_map(v, map.iova, vhost_vdpa_net_cvq_cmd_page_len(), buf, - !write); + r = vhost_vdpa_dma_map(v, v->address_space_id, map.iova, + vhost_vdpa_net_cvq_cmd_page_len(), buf, !write); if (unlikely(r < 0)) { goto dma_map_err; } diff --git a/hw/virtio/trace-events b/hw/virtio/trace-events index 820dadc26c..0ad9390307 100644 --- a/hw/virtio/trace-events +++ b/hw/virtio/trace-events @@ -30,8 +30,8 @@ vhost_user_write(uint32_t req, uint32_t flags) "req:%d flags:0x%"PRIx32"" vhost_user_create_notifier(int idx, void *n) "idx:%d n:%p" # vhost-vdpa.c -vhost_vdpa_dma_map(void *vdpa, int fd, uint32_t msg_type, uint64_t iova, uint64_t size, uint64_t uaddr, uint8_t perm, uint8_t type) "vdpa:%p fd: %d msg_type: %"PRIu32" iova: 0x%"PRIx64" size: 0x%"PRIx64" uaddr: 0x%"PRIx64" perm: 0x%"PRIx8" type: %"PRIu8 -vhost_vdpa_dma_unmap(void *vdpa, int fd, uint32_t msg_type, uint64_t iova, uint64_t size, uint8_t type) "vdpa:%p fd: %d msg_type: %"PRIu32" iova: 0x%"PRIx64" size: 0x%"PRIx64" type: %"PRIu8 +vhost_vdpa_dma_map(void *vdpa, int fd, uint32_t msg_type, uint32_t asid, uint64_t iova, uint64_t size, uint64_t uaddr, uint8_t perm, uint8_t type) "vdpa:%p fd: %d msg_type: %"PRIu32" asid: %"PRIu32" iova: 0x%"PRIx64" size: 0x%"PRIx64" uaddr: 0x%"PRIx64" perm: 0x%"PRIx8" type: %"PRIu8 +vhost_vdpa_dma_unmap(void *vdpa, int fd, uint32_t msg_type, uint32_t asid, uint64_t iova, uint64_t size, uint8_t type) "vdpa:%p fd: %d msg_type: %"PRIu32" asid: %"PRIu32" iova: 0x%"PRIx64" size: 0x%"PRIx64" type: %"PRIu8 vhost_vdpa_listener_begin_batch(void *v, int fd, uint32_t msg_type, uint8_t type) "vdpa:%p fd: %d msg_type: %"PRIu32" type: %"PRIu8 vhost_vdpa_listener_commit(void *v, int fd, uint32_t msg_type, uint8_t type) "vdpa:%p fd: %d msg_type: %"PRIu32" type: %"PRIu8 vhost_vdpa_listener_region_add(void *vdpa, uint64_t iova, uint64_t llend, void *vaddr, bool readonly) "vdpa: %p iova 0x%"PRIx64" llend 0x%"PRIx64" vaddr: %p read-only: %d" From patchwork Thu Nov 24 15:51:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 13055157 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2BBFBC433FE for ; Thu, 24 Nov 2022 15:53:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229877AbiKXPxt (ORCPT ); Thu, 24 Nov 2022 10:53:49 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39606 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229902AbiKXPxn (ORCPT ); Thu, 24 Nov 2022 10:53:43 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 52A7132B8F for ; Thu, 24 Nov 2022 07:52:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1669305163; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ySh46XbYk9VIs4uLkfJkTIjcH+z9Qv25PLQCXJV9doc=; b=I5GpWmX8X46LVDYdhiAcn+5tnkA3U9r7JolKCMwn9LyiFH/ibDBKLDGPSC+LhWs8jluX2J 3y+olY+7X4XKzP62r4kDDPDsCdw1y2vwUI8CdimYI1GkKRchIlOzqz6+UjYFDLNof503r0 42HJAkBQ1QkCnjSJ24FQxSABwdqBvjI= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-213-LBxWdrbqMHiyDE2LEqTnng-1; Thu, 24 Nov 2022 10:52:40 -0500 X-MC-Unique: LBxWdrbqMHiyDE2LEqTnng-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 0CDC3800B23; Thu, 24 Nov 2022 15:52:40 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.169]) by smtp.corp.redhat.com (Postfix) with ESMTP id E6E5B111E3F8; Thu, 24 Nov 2022 15:52:36 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: kvm@vger.kernel.org, Cornelia Huck , Eli Cohen , Jason Wang , Cindy Lu , Parav Pandit , Laurent Vivier , "Michael S. Tsirkin" , Si-Wei Liu , Zhu Lingshan , Paolo Bonzini , Stefan Hajnoczi , Gautam Dawar , Liuxiangdong , "Gonglei (Arei)" , Harpreet Singh Anand , Stefano Garzarella Subject: [PATCH for 8.0 v8 10/12] vdpa: store x-svq parameter in VhostVDPAState Date: Thu, 24 Nov 2022 16:51:56 +0100 Message-Id: <20221124155158.2109884-11-eperezma@redhat.com> In-Reply-To: <20221124155158.2109884-1-eperezma@redhat.com> References: <20221124155158.2109884-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org CVQ can be shadowed two ways: - Device has x-svq=on parameter (current way) - The device can isolate CVQ in its own vq group QEMU needs to check for the second condition dynamically, because CVQ index is not known before the driver ack the features. Since this is dynamic, the CVQ isolation could vary with different conditions, making it possible to go from "not isolated group" to "isolated". Saving the cmdline parameter in an extra field so we never disable CVQ SVQ in case the device was started with x-svq cmdline. Signed-off-by: Eugenio Pérez Acked-by: Jason Wang --- net/vhost-vdpa.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index b40b1ddf0d..3ff149dfd9 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -38,6 +38,8 @@ typedef struct VhostVDPAState { void *cvq_cmd_out_buffer; virtio_net_ctrl_ack *status; + /* The device always have SVQ enabled */ + bool always_svq; bool started; } VhostVDPAState; @@ -583,6 +585,7 @@ static NetClientState *net_vhost_vdpa_init(NetClientState *peer, s->vhost_vdpa.device_fd = vdpa_device_fd; s->vhost_vdpa.index = queue_pair_index; + s->always_svq = svq; s->vhost_vdpa.shadow_vqs_enabled = svq; s->vhost_vdpa.iova_tree = iova_tree; if (!is_datapath) { From patchwork Thu Nov 24 15:51:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 13055158 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D5C46C433FE for ; Thu, 24 Nov 2022 15:53:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229921AbiKXPxw (ORCPT ); Thu, 24 Nov 2022 10:53:52 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39668 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229917AbiKXPxq (ORCPT ); Thu, 24 Nov 2022 10:53:46 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 06B28442E0 for ; Thu, 24 Nov 2022 07:52:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1669305167; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0QDAmttFt0/uzyyfXC+uWtyx2ufyevmD2rR2wGnwGck=; b=EJUEiix1mIDh2wkiT2KfnZHtro4nCInHxObM6A3Qqhis9FWcwyLVP9gEZFrWXQwi38wwn0 FUAY3MDrT/b/JPH3+aWS0pXcXY9YBhtV0Iqg2oL7HbTeFmdFLH9yaNUyhxyz1COtQekDZA BDf9CBdgp/2vFX6j1s9VQbnBzBJos4Q= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-629-KOa_z0INP--O0gfHRnMKsA-1; Thu, 24 Nov 2022 10:52:44 -0500 X-MC-Unique: KOa_z0INP--O0gfHRnMKsA-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 69A8485A5A6; Thu, 24 Nov 2022 15:52:43 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.169]) by smtp.corp.redhat.com (Postfix) with ESMTP id 54489111E3F8; Thu, 24 Nov 2022 15:52:40 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: kvm@vger.kernel.org, Cornelia Huck , Eli Cohen , Jason Wang , Cindy Lu , Parav Pandit , Laurent Vivier , "Michael S. Tsirkin" , Si-Wei Liu , Zhu Lingshan , Paolo Bonzini , Stefan Hajnoczi , Gautam Dawar , Liuxiangdong , "Gonglei (Arei)" , Harpreet Singh Anand , Stefano Garzarella Subject: [PATCH for 8.0 v8 11/12] vdpa: add shadow_data to vhost_vdpa Date: Thu, 24 Nov 2022 16:51:57 +0100 Message-Id: <20221124155158.2109884-12-eperezma@redhat.com> In-Reply-To: <20221124155158.2109884-1-eperezma@redhat.com> References: <20221124155158.2109884-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The memory listener that thells the device how to convert GPA to qemu's va is registered against CVQ vhost_vdpa. memory listener translations are always ASID 0, CVQ ones are ASID 1 if supported. Let's tell the listener if it needs to register them on iova tree or not. Signed-off-by: Eugenio Pérez Acked-by: Jason Wang --- v7: Rename listener_shadow_vq to shadow_data v5: Solve conflict about vhost_iova_tree_remove accepting mem_region by value. --- include/hw/virtio/vhost-vdpa.h | 2 ++ hw/virtio/vhost-vdpa.c | 6 +++--- net/vhost-vdpa.c | 1 + 3 files changed, 6 insertions(+), 3 deletions(-) diff --git a/include/hw/virtio/vhost-vdpa.h b/include/hw/virtio/vhost-vdpa.h index e57dfa1fd1..45b969a311 100644 --- a/include/hw/virtio/vhost-vdpa.h +++ b/include/hw/virtio/vhost-vdpa.h @@ -40,6 +40,8 @@ typedef struct vhost_vdpa { struct vhost_vdpa_iova_range iova_range; uint64_t acked_features; bool shadow_vqs_enabled; + /* Vdpa must send shadow addresses as IOTLB key for data queues, not GPA */ + bool shadow_data; /* IOVA mapping used by the Shadow Virtqueue */ VhostIOVATree *iova_tree; GPtrArray *shadow_vqs; diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 1100cde29e..8e54c5c0fc 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -224,7 +224,7 @@ static void vhost_vdpa_listener_region_add(MemoryListener *listener, vaddr, section->readonly); llsize = int128_sub(llend, int128_make64(iova)); - if (v->shadow_vqs_enabled) { + if (v->shadow_data) { int r; mem_region.translated_addr = (hwaddr)(uintptr_t)vaddr, @@ -251,7 +251,7 @@ static void vhost_vdpa_listener_region_add(MemoryListener *listener, return; fail_map: - if (v->shadow_vqs_enabled) { + if (v->shadow_data) { vhost_iova_tree_remove(v->iova_tree, mem_region); } @@ -296,7 +296,7 @@ static void vhost_vdpa_listener_region_del(MemoryListener *listener, llsize = int128_sub(llend, int128_make64(iova)); - if (v->shadow_vqs_enabled) { + if (v->shadow_data) { const DMAMap *result; const void *vaddr = memory_region_get_ram_ptr(section->mr) + section->offset_within_region + diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index 3ff149dfd9..a1f1e29b7c 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -587,6 +587,7 @@ static NetClientState *net_vhost_vdpa_init(NetClientState *peer, s->vhost_vdpa.index = queue_pair_index; s->always_svq = svq; s->vhost_vdpa.shadow_vqs_enabled = svq; + s->vhost_vdpa.shadow_data = svq; s->vhost_vdpa.iova_tree = iova_tree; if (!is_datapath) { s->cvq_cmd_out_buffer = qemu_memalign(qemu_real_host_page_size(), From patchwork Thu Nov 24 15:51:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 13055159 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E3971C43217 for ; Thu, 24 Nov 2022 15:53:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229898AbiKXPx6 (ORCPT ); Thu, 24 Nov 2022 10:53:58 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39714 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229863AbiKXPxv (ORCPT ); Thu, 24 Nov 2022 10:53:51 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4D8A6442EA for ; Thu, 24 Nov 2022 07:52:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1669305169; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2skwpC+eVA6yYC2x/mr1Nxhuj4mB+d6nCE7s2zzx13A=; b=WJI8sNlbcPkvAXKoYsRXoMRyKTVdf5Wi6Qbor7tyuyjTv5X3GE/ekrHUHgMtn6Wn+Xm9nc WAyMs1SXcsrk1p1dr/dQAtr5Yt/LT526lX2/h8+4UfVIcQZnopBp5xd+jVKNTsTvsSH5C5 PWmw5gp4o984jsKezuTWVxfReWn0jJw= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-532-pv3OueikMXSiQWcWzl7bxw-1; Thu, 24 Nov 2022 10:52:47 -0500 X-MC-Unique: pv3OueikMXSiQWcWzl7bxw-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C4C2A800186; Thu, 24 Nov 2022 15:52:46 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.169]) by smtp.corp.redhat.com (Postfix) with ESMTP id B136F111E3F8; Thu, 24 Nov 2022 15:52:43 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: kvm@vger.kernel.org, Cornelia Huck , Eli Cohen , Jason Wang , Cindy Lu , Parav Pandit , Laurent Vivier , "Michael S. Tsirkin" , Si-Wei Liu , Zhu Lingshan , Paolo Bonzini , Stefan Hajnoczi , Gautam Dawar , Liuxiangdong , "Gonglei (Arei)" , Harpreet Singh Anand , Stefano Garzarella Subject: [PATCH for 8.0 v8 12/12] vdpa: always start CVQ in SVQ mode if possible Date: Thu, 24 Nov 2022 16:51:58 +0100 Message-Id: <20221124155158.2109884-13-eperezma@redhat.com> In-Reply-To: <20221124155158.2109884-1-eperezma@redhat.com> References: <20221124155158.2109884-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Isolate control virtqueue in its own group, allowing to intercept control commands but letting dataplane run totally passthrough to the guest. Signed-off-by: Eugenio Pérez --- v8: * Do not allocate iova_tree on net_init_vhost_vdpa if only CVQ is shadowed. Move the iova_tree handling in this case to vhost_vdpa_net_cvq_start and vhost_vdpa_net_cvq_stop. v7: * Never ask for number of address spaces, just react if isolation is not possible. * Return ASID ioctl errors instead of masking them as if the device has no asid. * Simplify net_init_vhost_vdpa logic * Add "if possible" suffix v6: * Disable control SVQ if the device does not support it because of features. v5: * Fixing the not adding cvq buffers when x-svq=on is specified. * Move vring state in vhost_vdpa_get_vring_group instead of using a parameter. * Rename VHOST_VDPA_NET_CVQ_PASSTHROUGH to VHOST_VDPA_NET_DATA_ASID v4: * Squash vhost_vdpa_cvq_group_is_independent. * Rebased on last CVQ start series, that allocated CVQ cmd bufs at load * Do not check for cvq index on vhost_vdpa_net_prepare, we only have one that callback registered in that NetClientInfo. v3: * Make asid related queries print a warning instead of returning an error and stop the start of qemu. --- hw/virtio/vhost-vdpa.c | 3 +- net/vhost-vdpa.c | 106 ++++++++++++++++++++++++++++++++++++++++- 2 files changed, 107 insertions(+), 2 deletions(-) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 8e54c5c0fc..45bb72d359 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -652,7 +652,8 @@ static int vhost_vdpa_set_backend_cap(struct vhost_dev *dev) { uint64_t features; uint64_t f = 0x1ULL << VHOST_BACKEND_F_IOTLB_MSG_V2 | - 0x1ULL << VHOST_BACKEND_F_IOTLB_BATCH; + 0x1ULL << VHOST_BACKEND_F_IOTLB_BATCH | + 0x1ULL << VHOST_BACKEND_F_IOTLB_ASID; int r; if (vhost_vdpa_call(dev, VHOST_GET_BACKEND_FEATURES, &features)) { diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index a1f1e29b7c..bce57fa724 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -102,6 +102,8 @@ static const uint64_t vdpa_svq_device_features = BIT_ULL(VIRTIO_NET_F_RSC_EXT) | BIT_ULL(VIRTIO_NET_F_STANDBY); +#define VHOST_VDPA_NET_CVQ_ASID 1 + VHostNetState *vhost_vdpa_get_vhost_net(NetClientState *nc) { VhostVDPAState *s = DO_UPCAST(VhostVDPAState, nc, nc); @@ -259,6 +261,40 @@ static VhostIOVATree *vhost_vdpa_svq_allocate_iova_tree(int vdpa_device_fd) return vhost_iova_tree_new(iova_range.first, iova_range.last); } +static int64_t vhost_vdpa_get_vring_group(int device_fd, unsigned vq_index) +{ + struct vhost_vring_state state = { + .index = vq_index, + }; + int r = ioctl(device_fd, VHOST_VDPA_GET_VRING_GROUP, &state); + + if (unlikely(r < 0)) { + error_report("Cannot get VQ %u group: %s", vq_index, + g_strerror(errno)); + return r; + } + + return state.num; +} + +static int vhost_vdpa_set_address_space_id(struct vhost_vdpa *v, + unsigned vq_group, + unsigned asid_num) +{ + struct vhost_vring_state asid = { + .index = vq_group, + .num = asid_num, + }; + int r; + + r = ioctl(v->device_fd, VHOST_VDPA_SET_GROUP_ASID, &asid); + if (unlikely(r < 0)) { + error_report("Can't set vq group %u asid %u, errno=%d (%s)", + asid.index, asid.num, errno, g_strerror(errno)); + } + return r; +} + static void vhost_vdpa_cvq_unmap_buf(struct vhost_vdpa *v, void *addr) { VhostIOVATree *tree = v->iova_tree; @@ -333,11 +369,71 @@ dma_map_err: static int vhost_vdpa_net_cvq_start(NetClientState *nc) { VhostVDPAState *s; - int r; + struct vhost_vdpa *v; + uint64_t backend_features; + int64_t cvq_group; + int cvq_index, r; assert(nc->info->type == NET_CLIENT_DRIVER_VHOST_VDPA); s = DO_UPCAST(VhostVDPAState, nc, nc); + v = &s->vhost_vdpa; + + v->shadow_data = s->always_svq; + v->shadow_vqs_enabled = s->always_svq; + s->vhost_vdpa.address_space_id = VHOST_VDPA_GUEST_PA_ASID; + + if (s->always_svq) { + /* SVQ is already configured for all virtqueues */ + goto out; + } + + /* Backend features are not available in v->dev yet. */ + r = ioctl(v->device_fd, VHOST_GET_BACKEND_FEATURES, &backend_features); + if (unlikely(r < 0)) { + error_report("Cannot get vdpa backend_features: %s(%d)", + g_strerror(errno), errno); + return -1; + } + if (!(backend_features & VHOST_BACKEND_F_IOTLB_ASID) || + !vhost_vdpa_net_valid_svq_features(v->dev->features, NULL)) { + return 0; + } + + /** + * Check if all the virtqueues of the virtio device are in a different vq + * than the last vq. VQ group of last group passed in cvq_group. + */ + cvq_index = v->dev->vq_index_end - 1; + cvq_group = vhost_vdpa_get_vring_group(v->device_fd, cvq_index); + if (unlikely(cvq_group < 0)) { + return cvq_group; + } + for (int i = 0; i < cvq_index; ++i) { + int64_t group = vhost_vdpa_get_vring_group(v->device_fd, i); + + if (unlikely(group < 0)) { + return group; + } + + if (unlikely(group == cvq_group)) { + warn_report( + "CVQ %"PRId64" group is the same as VQ %d one (%"PRId64")", + cvq_group, i, group); + return 0; + } + } + + r = vhost_vdpa_set_address_space_id(v, cvq_group, VHOST_VDPA_NET_CVQ_ASID); + if (unlikely(r < 0)) { + return r; + } + + v->iova_tree = vhost_vdpa_svq_allocate_iova_tree(v->device_fd); + v->shadow_vqs_enabled = true; + s->vhost_vdpa.address_space_id = VHOST_VDPA_NET_CVQ_ASID; + +out: if (!s->vhost_vdpa.shadow_vqs_enabled) { return 0; } @@ -366,6 +462,14 @@ static void vhost_vdpa_net_cvq_stop(NetClientState *nc) if (s->vhost_vdpa.shadow_vqs_enabled) { vhost_vdpa_cvq_unmap_buf(&s->vhost_vdpa, s->cvq_cmd_out_buffer); vhost_vdpa_cvq_unmap_buf(&s->vhost_vdpa, s->status); + if (!s->always_svq) { + /* + * If only the CVQ is shadowed we can delete this safely. + * If all the VQs are shadows this will be needed by the time the + * device is started again to register SVQ vrings and similar. + */ + g_clear_pointer(&s->vhost_vdpa.iova_tree, vhost_iova_tree_delete); + } } }