From patchwork Wed Aug 10 17:15:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 12940827 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C5023C00140 for ; Wed, 10 Aug 2022 17:15:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233406AbiHJRPd (ORCPT ); Wed, 10 Aug 2022 13:15:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49524 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233382AbiHJRPa (ORCPT ); Wed, 10 Aug 2022 13:15:30 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 1CBDF61D9B for ; Wed, 10 Aug 2022 10:15:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1660151728; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=i3xMckH4pXPfYRScm/pgwCzvro6+t56bA2voo3F7GcY=; b=K3ASpx9aEOOEJtbNknGtyi1chVLHsXCv4cUuXyk0zG7eGubzV/ymYseE4nlraHkJDpBECj xppw5xfy25Eo0LbFgZWDmcfeNvtRD7bgmVATN6BQWyk2SvViYTaIG7DBGQlFmvT8kNAQsg Of/25ihfg+QJKvqCWwuglBqZUahwyWE= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-649-QocaGs_zPLKxQQ1TirQ3-w-1; Wed, 10 Aug 2022 13:15:25 -0400 X-MC-Unique: QocaGs_zPLKxQQ1TirQ3-w-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 9FF1C1824600; Wed, 10 Aug 2022 17:15:24 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.193.156]) by smtp.corp.redhat.com (Postfix) with ESMTP id C791E1415128; Wed, 10 Aug 2022 17:15:19 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: kvm@vger.kernel.org, "Michael S. Tsirkin" , linux-kernel@vger.kernel.org, Jason Wang , virtualization@lists.linux-foundation.org, netdev@vger.kernel.org Cc: dinang@xilinx.com, martinpo@xilinx.com, Wu Zongyong , Piotr.Uminski@intel.com, gautam.dawar@amd.com, ecree.xilinx@gmail.com, martinh@xilinx.com, Stefano Garzarella , pabloc@xilinx.com, habetsm.xilinx@gmail.com, lvivier@redhat.com, Zhu Lingshan , tanuj.kamde@amd.com, Longpeng , lulu@redhat.com, hanand@xilinx.com, Parav Pandit , Si-Wei Liu , Eli Cohen , Xie Yongji , Zhang Min , Dan Carpenter , Christophe JAILLET Subject: [PATCH v7 1/4] vdpa: Add suspend operation Date: Wed, 10 Aug 2022 19:15:09 +0200 Message-Id: <20220810171512.2343333-2-eperezma@redhat.com> In-Reply-To: <20220810171512.2343333-1-eperezma@redhat.com> References: <20220810171512.2343333-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.85 on 10.11.54.7 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org This operation is optional: It it's not implemented, backend feature bit will not be exposed. Signed-off-by: Eugenio Pérez Message-Id: <20220623160738.632852-2-eperezma@redhat.com> Signed-off-by: Michael S. Tsirkin --- include/linux/vdpa.h | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/include/linux/vdpa.h b/include/linux/vdpa.h index 7b4a13d3bd91..d282f464d2f1 100644 --- a/include/linux/vdpa.h +++ b/include/linux/vdpa.h @@ -218,6 +218,9 @@ struct vdpa_map_file { * @reset: Reset device * @vdev: vdpa device * Returns integer: success (0) or error (< 0) + * @suspend: Suspend or resume the device (optional) + * @vdev: vdpa device + * Returns integer: success (0) or error (< 0) * @get_config_size: Get the size of the configuration space includes * fields that are conditional on feature bits. * @vdev: vdpa device @@ -319,6 +322,7 @@ struct vdpa_config_ops { u8 (*get_status)(struct vdpa_device *vdev); void (*set_status)(struct vdpa_device *vdev, u8 status); int (*reset)(struct vdpa_device *vdev); + int (*suspend)(struct vdpa_device *vdev); size_t (*get_config_size)(struct vdpa_device *vdev); void (*get_config)(struct vdpa_device *vdev, unsigned int offset, void *buf, unsigned int len); From patchwork Wed Aug 10 17:15:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 12940828 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0F4E6C19F2A for ; Wed, 10 Aug 2022 17:15:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233412AbiHJRPp (ORCPT ); Wed, 10 Aug 2022 13:15:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49702 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233426AbiHJRPg (ORCPT ); Wed, 10 Aug 2022 13:15:36 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 64E09606B9 for ; Wed, 10 Aug 2022 10:15:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1660151733; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iMpQUSDWpUJEJo5k/gaQy5pnVja3hB7pQ6/xlm1Oxjw=; b=QV+Xfvd+kGZ1yDipXZuOUDQW32DUZSS/iQy0GsZTt3x7Gz5x0RDxHk3tXsfE9oO0zHfdq/ Av9w4VThFKzmW5AlBPqFGqgMqyVg4HiinraPyWY23nl0cNmJSFyCLveYKM6mtd+vWec1F1 +Nuk8DinIJdEyh/RzpVlUQEf+Tw2TXU= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-341-WG9qIJSPMvadnQW-uALkAw-1; Wed, 10 Aug 2022 13:15:30 -0400 X-MC-Unique: WG9qIJSPMvadnQW-uALkAw-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 613C61019C8D; Wed, 10 Aug 2022 17:15:29 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.193.156]) by smtp.corp.redhat.com (Postfix) with ESMTP id E15A81415128; Wed, 10 Aug 2022 17:15:24 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: kvm@vger.kernel.org, "Michael S. Tsirkin" , linux-kernel@vger.kernel.org, Jason Wang , virtualization@lists.linux-foundation.org, netdev@vger.kernel.org Cc: dinang@xilinx.com, martinpo@xilinx.com, Wu Zongyong , Piotr.Uminski@intel.com, gautam.dawar@amd.com, ecree.xilinx@gmail.com, martinh@xilinx.com, Stefano Garzarella , pabloc@xilinx.com, habetsm.xilinx@gmail.com, lvivier@redhat.com, Zhu Lingshan , tanuj.kamde@amd.com, Longpeng , lulu@redhat.com, hanand@xilinx.com, Parav Pandit , Si-Wei Liu , Eli Cohen , Xie Yongji , Zhang Min , Dan Carpenter , Christophe JAILLET Subject: [PATCH v7 2/4] vhost-vdpa: introduce SUSPEND backend feature bit Date: Wed, 10 Aug 2022 19:15:10 +0200 Message-Id: <20220810171512.2343333-3-eperezma@redhat.com> In-Reply-To: <20220810171512.2343333-1-eperezma@redhat.com> References: <20220810171512.2343333-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.85 on 10.11.54.7 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Userland knows if it can suspend the device or not by checking this feature bit. It's only offered if the vdpa driver backend implements the suspend() operation callback, and to offer it or userland to ack it if the backend does not offer that callback is an error. Signed-off-by: Eugenio Pérez Message-Id: <20220623160738.632852-3-eperezma@redhat.com> Signed-off-by: Michael S. Tsirkin --- drivers/vhost/vdpa.c | 16 +++++++++++++++- include/uapi/linux/vhost_types.h | 2 ++ 2 files changed, 17 insertions(+), 1 deletion(-) diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c index 23dcbfdfa13b..3d636e192061 100644 --- a/drivers/vhost/vdpa.c +++ b/drivers/vhost/vdpa.c @@ -347,6 +347,14 @@ static long vhost_vdpa_set_config(struct vhost_vdpa *v, return 0; } +static bool vhost_vdpa_can_suspend(const struct vhost_vdpa *v) +{ + struct vdpa_device *vdpa = v->vdpa; + const struct vdpa_config_ops *ops = vdpa->config; + + return ops->suspend; +} + static long vhost_vdpa_get_features(struct vhost_vdpa *v, u64 __user *featurep) { struct vdpa_device *vdpa = v->vdpa; @@ -577,7 +585,11 @@ static long vhost_vdpa_unlocked_ioctl(struct file *filep, if (cmd == VHOST_SET_BACKEND_FEATURES) { if (copy_from_user(&features, featurep, sizeof(features))) return -EFAULT; - if (features & ~VHOST_VDPA_BACKEND_FEATURES) + if (features & ~(VHOST_VDPA_BACKEND_FEATURES | + BIT_ULL(VHOST_BACKEND_F_SUSPEND))) + return -EOPNOTSUPP; + if ((features & BIT_ULL(VHOST_BACKEND_F_SUSPEND)) && + !vhost_vdpa_can_suspend(v)) return -EOPNOTSUPP; vhost_set_backend_features(&v->vdev, features); return 0; @@ -628,6 +640,8 @@ static long vhost_vdpa_unlocked_ioctl(struct file *filep, break; case VHOST_GET_BACKEND_FEATURES: features = VHOST_VDPA_BACKEND_FEATURES; + if (vhost_vdpa_can_suspend(v)) + features |= BIT_ULL(VHOST_BACKEND_F_SUSPEND); if (copy_to_user(featurep, &features, sizeof(features))) r = -EFAULT; break; diff --git a/include/uapi/linux/vhost_types.h b/include/uapi/linux/vhost_types.h index 634cee485abb..1bdd6e363f4c 100644 --- a/include/uapi/linux/vhost_types.h +++ b/include/uapi/linux/vhost_types.h @@ -161,5 +161,7 @@ struct vhost_vdpa_iova_range { * message */ #define VHOST_BACKEND_F_IOTLB_ASID 0x3 +/* Device can be suspended */ +#define VHOST_BACKEND_F_SUSPEND 0x4 #endif From patchwork Wed Aug 10 17:15:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 12940829 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D057C25B0C for ; Wed, 10 Aug 2022 17:15:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233424AbiHJRPr (ORCPT ); Wed, 10 Aug 2022 13:15:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50002 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233482AbiHJRPm (ORCPT ); Wed, 10 Aug 2022 13:15:42 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id ABEAA60535 for ; Wed, 10 Aug 2022 10:15:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1660151739; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+9sF3KA2QSMcF5wWOCwmBqvZxHtmEUvtswHt6xdWHZc=; b=TBAkRV9/9CGPwl3fCQsz123TiYobXQtpV88gcIzOYI+QJsrHW3P3uvSnt9QlhmtjnTjk1z Twv9Nsah+5ZalRRGsDMiSwzkiMi6CdJLwclz52oCayxDgOUQESooIgQ2bTeP4D08kC9Ssa 5b1rQZZzfDGh7qss4r9CHWeKcea5e24= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-179-7mK-R0nDNJ2lHUSOMexoYA-1; Wed, 10 Aug 2022 13:15:35 -0400 X-MC-Unique: 7mK-R0nDNJ2lHUSOMexoYA-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 0DA111824603; Wed, 10 Aug 2022 17:15:34 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.193.156]) by smtp.corp.redhat.com (Postfix) with ESMTP id A414B1415125; Wed, 10 Aug 2022 17:15:29 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: kvm@vger.kernel.org, "Michael S. Tsirkin" , linux-kernel@vger.kernel.org, Jason Wang , virtualization@lists.linux-foundation.org, netdev@vger.kernel.org Cc: dinang@xilinx.com, martinpo@xilinx.com, Wu Zongyong , Piotr.Uminski@intel.com, gautam.dawar@amd.com, ecree.xilinx@gmail.com, martinh@xilinx.com, Stefano Garzarella , pabloc@xilinx.com, habetsm.xilinx@gmail.com, lvivier@redhat.com, Zhu Lingshan , tanuj.kamde@amd.com, Longpeng , lulu@redhat.com, hanand@xilinx.com, Parav Pandit , Si-Wei Liu , Eli Cohen , Xie Yongji , Zhang Min , Dan Carpenter , Christophe JAILLET Subject: [PATCH v7 3/4] vhost-vdpa: uAPI to suspend the device Date: Wed, 10 Aug 2022 19:15:11 +0200 Message-Id: <20220810171512.2343333-4-eperezma@redhat.com> In-Reply-To: <20220810171512.2343333-1-eperezma@redhat.com> References: <20220810171512.2343333-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.85 on 10.11.54.7 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The ioctl adds support for suspending the device from userspace. This is a must before getting virtqueue indexes (base) for live migration, since the device could modify them after userland gets them. There are individual ways to perform that action for some devices (VHOST_NET_SET_BACKEND, VHOST_VSOCK_SET_RUNNING, ...) but there was no way to perform it for any vhost device (and, in particular, vhost-vdpa). After a successful return of the ioctl call the device must not process more virtqueue descriptors. The device can answer to read or writes of config fields as if it were not suspended. In particular, writing to "queue_enable" with a value of 1 will not make the device start processing buffers of the virtqueue. Signed-off-by: Eugenio Pérez Message-Id: <20220623160738.632852-4-eperezma@redhat.com> Signed-off-by: Michael S. Tsirkin --- v7: Delete argument to ioctl, unused --- drivers/vhost/vdpa.c | 19 +++++++++++++++++++ include/uapi/linux/vhost.h | 9 +++++++++ 2 files changed, 28 insertions(+) diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c index 3d636e192061..7fa671ac4bdf 100644 --- a/drivers/vhost/vdpa.c +++ b/drivers/vhost/vdpa.c @@ -478,6 +478,22 @@ static long vhost_vdpa_get_vqs_count(struct vhost_vdpa *v, u32 __user *argp) return 0; } +/* After a successful return of ioctl the device must not process more + * virtqueue descriptors. The device can answer to read or writes of config + * fields as if it were not suspended. In particular, writing to "queue_enable" + * with a value of 1 will not make the device start processing buffers. + */ +static long vhost_vdpa_suspend(struct vhost_vdpa *v) +{ + struct vdpa_device *vdpa = v->vdpa; + const struct vdpa_config_ops *ops = vdpa->config; + + if (!ops->suspend) + return -EOPNOTSUPP; + + return ops->suspend(vdpa); +} + static long vhost_vdpa_vring_ioctl(struct vhost_vdpa *v, unsigned int cmd, void __user *argp) { @@ -654,6 +670,9 @@ static long vhost_vdpa_unlocked_ioctl(struct file *filep, case VHOST_VDPA_GET_VQS_COUNT: r = vhost_vdpa_get_vqs_count(v, argp); break; + case VHOST_VDPA_SUSPEND: + r = vhost_vdpa_suspend(v); + break; default: r = vhost_dev_ioctl(&v->vdev, cmd, argp); if (r == -ENOIOCTLCMD) diff --git a/include/uapi/linux/vhost.h b/include/uapi/linux/vhost.h index cab645d4a645..f9f115a7c75b 100644 --- a/include/uapi/linux/vhost.h +++ b/include/uapi/linux/vhost.h @@ -171,4 +171,13 @@ #define VHOST_VDPA_SET_GROUP_ASID _IOW(VHOST_VIRTIO, 0x7C, \ struct vhost_vring_state) +/* Suspend a device so it does not process virtqueue requests anymore + * + * After the return of ioctl the device must preserve all the necessary state + * (the virtqueue vring base plus the possible device specific states) that is + * required for restoring in the future. The device must not change its + * configuration after that point. + */ +#define VHOST_VDPA_SUSPEND _IO(VHOST_VIRTIO, 0x7D) + #endif From patchwork Wed Aug 10 17:15:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 12940830 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 84EA0C00140 for ; Wed, 10 Aug 2022 17:16:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233456AbiHJRQD (ORCPT ); Wed, 10 Aug 2022 13:16:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50168 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233419AbiHJRPr (ORCPT ); Wed, 10 Aug 2022 13:15:47 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id BB7E279689 for ; Wed, 10 Aug 2022 10:15:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1660151745; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Imye1gnnfzcz+CnTHG0VF+fJyp01Qs5zM0UuNG7EPiA=; b=HCom9CbZhs/ec/YkjzzfXiJkS/b35R62VFdIZ25ZKA+cI8Vii8Musq7Qut189d3RPJKDAB QGCjJaeoat99+OgQQ7423/Fnx9tv/yX85Q6LpEi+fmsBCQAz1ki/SuqY4lE3nu0nQvHHYJ enMGkDQEE6ePpSgef3AOVbCMG+QV99s= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-223-JOUjUaghNQ25Zncct0Exdg-1; Wed, 10 Aug 2022 13:15:40 -0400 X-MC-Unique: JOUjUaghNQ25Zncct0Exdg-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id AE1B01C006C9; Wed, 10 Aug 2022 17:15:38 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.193.156]) by smtp.corp.redhat.com (Postfix) with ESMTP id 512A51415125; Wed, 10 Aug 2022 17:15:34 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: kvm@vger.kernel.org, "Michael S. Tsirkin" , linux-kernel@vger.kernel.org, Jason Wang , virtualization@lists.linux-foundation.org, netdev@vger.kernel.org Cc: dinang@xilinx.com, martinpo@xilinx.com, Wu Zongyong , Piotr.Uminski@intel.com, gautam.dawar@amd.com, ecree.xilinx@gmail.com, martinh@xilinx.com, Stefano Garzarella , pabloc@xilinx.com, habetsm.xilinx@gmail.com, lvivier@redhat.com, Zhu Lingshan , tanuj.kamde@amd.com, Longpeng , lulu@redhat.com, hanand@xilinx.com, Parav Pandit , Si-Wei Liu , Eli Cohen , Xie Yongji , Zhang Min , Dan Carpenter , Christophe JAILLET Subject: [PATCH v7 4/4] vdpa_sim: Implement suspend vdpa op Date: Wed, 10 Aug 2022 19:15:12 +0200 Message-Id: <20220810171512.2343333-5-eperezma@redhat.com> In-Reply-To: <20220810171512.2343333-1-eperezma@redhat.com> References: <20220810171512.2343333-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.85 on 10.11.54.7 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Implement suspend operation for vdpa_sim devices, so vhost-vdpa will offer that backend feature and userspace can effectively suspend the device. This is a must before get virtqueue indexes (base) for live migration, since the device could modify them after userland gets them. There are individual ways to perform that action for some devices (VHOST_NET_SET_BACKEND, VHOST_VSOCK_SET_RUNNING, ...) but there was no way to perform it for any vhost device (and, in particular, vhost-vdpa). Reviewed-by: Stefano Garzarella Signed-off-by: Eugenio Pérez Message-Id: <20220623160738.632852-5-eperezma@redhat.com> Signed-off-by: Michael S. Tsirkin --- v7: Delete leftover resume code --- drivers/vdpa/vdpa_sim/vdpa_sim.c | 14 ++++++++++++++ drivers/vdpa/vdpa_sim/vdpa_sim.h | 1 + drivers/vdpa/vdpa_sim/vdpa_sim_blk.c | 3 +++ drivers/vdpa/vdpa_sim/vdpa_sim_net.c | 3 +++ 4 files changed, 21 insertions(+) diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim.c b/drivers/vdpa/vdpa_sim/vdpa_sim.c index 0f2865899647..79a50edf8998 100644 --- a/drivers/vdpa/vdpa_sim/vdpa_sim.c +++ b/drivers/vdpa/vdpa_sim/vdpa_sim.c @@ -107,6 +107,7 @@ static void vdpasim_do_reset(struct vdpasim *vdpasim) for (i = 0; i < vdpasim->dev_attr.nas; i++) vhost_iotlb_reset(&vdpasim->iommu[i]); + vdpasim->running = true; spin_unlock(&vdpasim->iommu_lock); vdpasim->features = 0; @@ -505,6 +506,17 @@ static int vdpasim_reset(struct vdpa_device *vdpa) return 0; } +static int vdpasim_suspend(struct vdpa_device *vdpa) +{ + struct vdpasim *vdpasim = vdpa_to_sim(vdpa); + + spin_lock(&vdpasim->lock); + vdpasim->running = false; + spin_unlock(&vdpasim->lock); + + return 0; +} + static size_t vdpasim_get_config_size(struct vdpa_device *vdpa) { struct vdpasim *vdpasim = vdpa_to_sim(vdpa); @@ -694,6 +706,7 @@ static const struct vdpa_config_ops vdpasim_config_ops = { .get_status = vdpasim_get_status, .set_status = vdpasim_set_status, .reset = vdpasim_reset, + .suspend = vdpasim_suspend, .get_config_size = vdpasim_get_config_size, .get_config = vdpasim_get_config, .set_config = vdpasim_set_config, @@ -726,6 +739,7 @@ static const struct vdpa_config_ops vdpasim_batch_config_ops = { .get_status = vdpasim_get_status, .set_status = vdpasim_set_status, .reset = vdpasim_reset, + .suspend = vdpasim_suspend, .get_config_size = vdpasim_get_config_size, .get_config = vdpasim_get_config, .set_config = vdpasim_set_config, diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim.h b/drivers/vdpa/vdpa_sim/vdpa_sim.h index 622782e92239..061986f30911 100644 --- a/drivers/vdpa/vdpa_sim/vdpa_sim.h +++ b/drivers/vdpa/vdpa_sim/vdpa_sim.h @@ -66,6 +66,7 @@ struct vdpasim { u32 generation; u64 features; u32 groups; + bool running; /* spinlock to synchronize iommu table */ spinlock_t iommu_lock; }; diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim_blk.c b/drivers/vdpa/vdpa_sim/vdpa_sim_blk.c index 42d401d43911..bcdb1982c378 100644 --- a/drivers/vdpa/vdpa_sim/vdpa_sim_blk.c +++ b/drivers/vdpa/vdpa_sim/vdpa_sim_blk.c @@ -204,6 +204,9 @@ static void vdpasim_blk_work(struct work_struct *work) if (!(vdpasim->status & VIRTIO_CONFIG_S_DRIVER_OK)) goto out; + if (!vdpasim->running) + goto out; + for (i = 0; i < VDPASIM_BLK_VQ_NUM; i++) { struct vdpasim_virtqueue *vq = &vdpasim->vqs[i]; diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim_net.c b/drivers/vdpa/vdpa_sim/vdpa_sim_net.c index 5125976a4df8..886449e88502 100644 --- a/drivers/vdpa/vdpa_sim/vdpa_sim_net.c +++ b/drivers/vdpa/vdpa_sim/vdpa_sim_net.c @@ -154,6 +154,9 @@ static void vdpasim_net_work(struct work_struct *work) spin_lock(&vdpasim->lock); + if (!vdpasim->running) + goto out; + if (!(vdpasim->status & VIRTIO_CONFIG_S_DRIVER_OK)) goto out;