From patchwork Wed Jan 26 07:35:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 12724681 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6979EC63686 for ; Wed, 26 Jan 2022 07:35:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237891AbiAZHfl (ORCPT ); Wed, 26 Jan 2022 02:35:41 -0500 Received: from out199-6.us.a.mail.aliyun.com ([47.90.199.6]:34467 "EHLO out199-6.us.a.mail.aliyun.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237874AbiAZHfi (ORCPT ); Wed, 26 Jan 2022 02:35:38 -0500 X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R171e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04400;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=11;SR=0;TI=SMTPD_---0V2uAwQt_1643182534; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0V2uAwQt_1643182534) by smtp.aliyun-inc.com(127.0.0.1); Wed, 26 Jan 2022 15:35:35 +0800 From: Xuan Zhuo To: virtualization@lists.linux-foundation.org, netdev@vger.kernel.org Cc: "Michael S. Tsirkin" , Jason Wang , "David S. Miller" , Jakub Kicinski , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , bpf@vger.kernel.org Subject: [PATCH v3 01/17] virtio_pci: struct virtio_pci_common_cfg add queue_notify_data Date: Wed, 26 Jan 2022 15:35:17 +0800 Message-Id: <20220126073533.44994-2-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20220126073533.44994-1-xuanzhuo@linux.alibaba.com> References: <20220126073533.44994-1-xuanzhuo@linux.alibaba.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Add queue_notify_data in struct virtio_pci_common_cfg, which comes from here https://github.com/oasis-tcs/virtio-spec/issues/89 Since I want to add queue_reset after it, I submitted this patch first. Signed-off-by: Xuan Zhuo --- include/uapi/linux/virtio_pci.h | 1 + 1 file changed, 1 insertion(+) diff --git a/include/uapi/linux/virtio_pci.h b/include/uapi/linux/virtio_pci.h index 3a86f36d7e3d..492c89f56c6a 100644 --- a/include/uapi/linux/virtio_pci.h +++ b/include/uapi/linux/virtio_pci.h @@ -164,6 +164,7 @@ struct virtio_pci_common_cfg { __le32 queue_avail_hi; /* read-write */ __le32 queue_used_lo; /* read-write */ __le32 queue_used_hi; /* read-write */ + __le16 queue_notify_data; /* read-write */ }; /* Fields in VIRTIO_PCI_CAP_PCI_CFG: */ From patchwork Wed Jan 26 07:35:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 12724683 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E6662C636C9 for ; Wed, 26 Jan 2022 07:35:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237897AbiAZHfl (ORCPT ); Wed, 26 Jan 2022 02:35:41 -0500 Received: from out30-54.freemail.mail.aliyun.com ([115.124.30.54]:40462 "EHLO out30-54.freemail.mail.aliyun.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237878AbiAZHfj (ORCPT ); Wed, 26 Jan 2022 02:35:39 -0500 X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R721e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04357;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=11;SR=0;TI=SMTPD_---0V2uVHlA_1643182535; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0V2uVHlA_1643182535) by smtp.aliyun-inc.com(127.0.0.1); Wed, 26 Jan 2022 15:35:36 +0800 From: Xuan Zhuo To: virtualization@lists.linux-foundation.org, netdev@vger.kernel.org Cc: "Michael S. Tsirkin" , Jason Wang , "David S. Miller" , Jakub Kicinski , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , bpf@vger.kernel.org Subject: [PATCH v3 02/17] virtio: queue_reset: add VIRTIO_F_RING_RESET Date: Wed, 26 Jan 2022 15:35:18 +0800 Message-Id: <20220126073533.44994-3-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20220126073533.44994-1-xuanzhuo@linux.alibaba.com> References: <20220126073533.44994-1-xuanzhuo@linux.alibaba.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Added VIRTIO_F_RING_RESET, it came from here https://github.com/oasis-tcs/virtio-spec/issues/124 Signed-off-by: Xuan Zhuo --- include/uapi/linux/virtio_config.h | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/include/uapi/linux/virtio_config.h b/include/uapi/linux/virtio_config.h index b5eda06f0d57..0862be802ff8 100644 --- a/include/uapi/linux/virtio_config.h +++ b/include/uapi/linux/virtio_config.h @@ -52,7 +52,7 @@ * rest are per-device feature bits. */ #define VIRTIO_TRANSPORT_F_START 28 -#define VIRTIO_TRANSPORT_F_END 38 +#define VIRTIO_TRANSPORT_F_END 41 #ifndef VIRTIO_CONFIG_NO_LEGACY /* Do we get callbacks when the ring is completely used, even if we've @@ -92,4 +92,9 @@ * Does the device support Single Root I/O Virtualization? */ #define VIRTIO_F_SR_IOV 37 + +/* + * This feature indicates that the driver can reset a queue individually. + */ +#define VIRTIO_F_RING_RESET 40 #endif /* _UAPI_LINUX_VIRTIO_CONFIG_H */ From patchwork Wed Jan 26 07:35:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 12724685 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 49CDAC6369B for ; Wed, 26 Jan 2022 07:35:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237892AbiAZHfn (ORCPT ); Wed, 26 Jan 2022 02:35:43 -0500 Received: from out30-57.freemail.mail.aliyun.com ([115.124.30.57]:57805 "EHLO out30-57.freemail.mail.aliyun.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237872AbiAZHfk (ORCPT ); Wed, 26 Jan 2022 02:35:40 -0500 X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R171e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04394;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=11;SR=0;TI=SMTPD_---0V2uc6jX_1643182536; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0V2uc6jX_1643182536) by smtp.aliyun-inc.com(127.0.0.1); Wed, 26 Jan 2022 15:35:37 +0800 From: Xuan Zhuo To: virtualization@lists.linux-foundation.org, netdev@vger.kernel.org Cc: "Michael S. Tsirkin" , Jason Wang , "David S. Miller" , Jakub Kicinski , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , bpf@vger.kernel.org Subject: [PATCH v3 03/17] virtio: queue_reset: struct virtio_config_ops add callbacks for queue_reset Date: Wed, 26 Jan 2022 15:35:19 +0800 Message-Id: <20220126073533.44994-4-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20220126073533.44994-1-xuanzhuo@linux.alibaba.com> References: <20220126073533.44994-1-xuanzhuo@linux.alibaba.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Performing reset on a queue is divided into two steps: 1. reset_vq: reset one vq 2. enable_reset_vq: re-enable the reset queue In the first step, these tasks will be completed: 1. notify the hardware queue to reset 2. recycle the buffer from vq 3. release the ring of the vq The second step is similar to find vqs, passing parameters callback and name, etc. Based on the original vq, the ring is re-allocated and configured to the backend. So add two callbacks reset_vq, enable_reset_vq to struct virtio_config_ops. Add a structure for passing parameters. This will facilitate subsequent expansion of the parameters of enable reset vq. There is currently only one default extended parameter ring_num. Signed-off-by: Xuan Zhuo --- include/linux/virtio_config.h | 43 ++++++++++++++++++++++++++++++++++- 1 file changed, 42 insertions(+), 1 deletion(-) diff --git a/include/linux/virtio_config.h b/include/linux/virtio_config.h index 4d107ad31149..51dd8461d1b6 100644 --- a/include/linux/virtio_config.h +++ b/include/linux/virtio_config.h @@ -16,6 +16,44 @@ struct virtio_shm_region { u64 len; }; +typedef void vq_callback_t(struct virtqueue *); + +/* virtio_reset_vq: specify parameters for queue_reset + * + * vdev: the device + * queue_index: the queue index + * + * free_unused_cb: callback to free unused bufs + * data: used by free_unused_cb + * + * callback: callback for the virtqueue, NULL for vq that do not need a + * callback + * name: virtqueue names (mainly for debugging), NULL for vq unused by + * driver + * ctx: ctx + * + * ring_num: specify ring num for the vq to be re-enabled. 0 means use the + * default value. MUST be a power of 2. + */ +struct virtio_reset_vq; +typedef void vq_reset_callback_t(struct virtio_reset_vq *param, void *buf); +struct virtio_reset_vq { + struct virtio_device *vdev; + u16 queue_index; + + /* reset vq param */ + vq_reset_callback_t *free_unused_cb; + void *data; + + /* enable reset vq param */ + vq_callback_t *callback; + const char *name; + const bool *ctx; + + /* ext enable reset vq param */ + u16 ring_num; +}; + /** * virtio_config_ops - operations for configuring a virtio device * Note: Do not assume that a transport implements all of the operations @@ -74,8 +112,9 @@ struct virtio_shm_region { * @set_vq_affinity: set the affinity for a virtqueue (optional). * @get_vq_affinity: get the affinity for a virtqueue (optional). * @get_shm_region: get a shared memory region based on the index. + * @reset_vq: reset a queue individually + * @enable_reset_vq: enable a reset queue */ -typedef void vq_callback_t(struct virtqueue *); struct virtio_config_ops { void (*enable_cbs)(struct virtio_device *vdev); void (*get)(struct virtio_device *vdev, unsigned offset, @@ -100,6 +139,8 @@ struct virtio_config_ops { int index); bool (*get_shm_region)(struct virtio_device *vdev, struct virtio_shm_region *region, u8 id); + int (*reset_vq)(struct virtio_reset_vq *param); + struct virtqueue *(*enable_reset_vq)(struct virtio_reset_vq *param); }; /* If driver didn't advertise the feature, it will never appear. */ From patchwork Wed Jan 26 07:35:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 12724684 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5713FC63682 for ; Wed, 26 Jan 2022 07:35:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237904AbiAZHfm (ORCPT ); Wed, 26 Jan 2022 02:35:42 -0500 Received: from out30-45.freemail.mail.aliyun.com ([115.124.30.45]:34290 "EHLO out30-45.freemail.mail.aliyun.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237881AbiAZHfk (ORCPT ); Wed, 26 Jan 2022 02:35:40 -0500 X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R441e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04423;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=11;SR=0;TI=SMTPD_---0V2ubkjl_1643182537; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0V2ubkjl_1643182537) by smtp.aliyun-inc.com(127.0.0.1); Wed, 26 Jan 2022 15:35:38 +0800 From: Xuan Zhuo To: virtualization@lists.linux-foundation.org, netdev@vger.kernel.org Cc: "Michael S. Tsirkin" , Jason Wang , "David S. Miller" , Jakub Kicinski , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , bpf@vger.kernel.org Subject: [PATCH v3 04/17] virtio: queue_reset: add helper Date: Wed, 26 Jan 2022 15:35:20 +0800 Message-Id: <20220126073533.44994-5-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20220126073533.44994-1-xuanzhuo@linux.alibaba.com> References: <20220126073533.44994-1-xuanzhuo@linux.alibaba.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Add helper for virtio queue reset. * virtio_reset_vq: reset a queue individually * virtio_enable_resetq: enable a reset queue Signed-off-by: Xuan Zhuo --- include/linux/virtio_config.h | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/include/linux/virtio_config.h b/include/linux/virtio_config.h index 51dd8461d1b6..3c971d9a0a59 100644 --- a/include/linux/virtio_config.h +++ b/include/linux/virtio_config.h @@ -260,6 +260,38 @@ int virtio_find_vqs_ctx(struct virtio_device *vdev, unsigned nvqs, desc); } +/** + * virtio_reset_vq - reset a queue individually + * @param: struct virtio_reset_vq + * + * returns 0 on success or error status + * + */ +static inline +int virtio_reset_vq(struct virtio_reset_vq *param) +{ + if (!param->vdev->config->reset_vq) + return -ENOENT; + + return param->vdev->config->reset_vq(param); +} + +/** + * virtio_enable_resetq - enable a reset queue + * @param: struct virtio_reset_vq + * + * returns vq on success or error status + * + */ +static inline +struct virtqueue *virtio_enable_resetq(struct virtio_reset_vq *param) +{ + if (!param->vdev->config->enable_reset_vq) + return ERR_PTR(-ENOENT); + + return param->vdev->config->enable_reset_vq(param); +} + /** * virtio_device_ready - enable vq use in probe function * @vdev: the device From patchwork Wed Jan 26 07:35:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 12724686 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BF3B4C636D9 for ; Wed, 26 Jan 2022 07:35:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237881AbiAZHfn (ORCPT ); Wed, 26 Jan 2022 02:35:43 -0500 Received: from out30-57.freemail.mail.aliyun.com ([115.124.30.57]:56021 "EHLO out30-57.freemail.mail.aliyun.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237870AbiAZHfl (ORCPT ); Wed, 26 Jan 2022 02:35:41 -0500 X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R691e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04394;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=11;SR=0;TI=SMTPD_---0V2ubkjx_1643182538; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0V2ubkjx_1643182538) by smtp.aliyun-inc.com(127.0.0.1); Wed, 26 Jan 2022 15:35:39 +0800 From: Xuan Zhuo To: virtualization@lists.linux-foundation.org, netdev@vger.kernel.org Cc: "Michael S. Tsirkin" , Jason Wang , "David S. Miller" , Jakub Kicinski , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , bpf@vger.kernel.org Subject: [PATCH v3 05/17] vritio_ring: queue_reset: extract the release function of the vq ring Date: Wed, 26 Jan 2022 15:35:21 +0800 Message-Id: <20220126073533.44994-6-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20220126073533.44994-1-xuanzhuo@linux.alibaba.com> References: <20220126073533.44994-1-xuanzhuo@linux.alibaba.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Extract a function __vring_del_virtqueue() from vring_del_virtqueue() to handle releasing vq's ring. Signed-off-by: Xuan Zhuo --- drivers/virtio/virtio_ring.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index 028b05d44546..b6434e36c447 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -2300,12 +2300,10 @@ struct virtqueue *vring_new_virtqueue(unsigned int index, } EXPORT_SYMBOL_GPL(vring_new_virtqueue); -void vring_del_virtqueue(struct virtqueue *_vq) +static void __vring_del_virtqueue(struct vring_virtqueue *vq) { - struct vring_virtqueue *vq = to_vvq(_vq); - spin_lock(&vq->vq.vdev->vqs_list_lock); - list_del(&_vq->list); + list_del(&vq->vq.list); spin_unlock(&vq->vq.vdev->vqs_list_lock); if (vq->we_own_ring) { @@ -2338,6 +2336,13 @@ void vring_del_virtqueue(struct virtqueue *_vq) kfree(vq->split.desc_state); kfree(vq->split.desc_extra); } +} + +void vring_del_virtqueue(struct virtqueue *_vq) +{ + struct vring_virtqueue *vq = to_vvq(_vq); + + __vring_del_virtqueue(vq); kfree(vq); } EXPORT_SYMBOL_GPL(vring_del_virtqueue); From patchwork Wed Jan 26 07:35:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 12724689 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0DDA9C63697 for ; Wed, 26 Jan 2022 07:36:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237968AbiAZHf7 (ORCPT ); Wed, 26 Jan 2022 02:35:59 -0500 Received: from out30-57.freemail.mail.aliyun.com ([115.124.30.57]:42896 "EHLO out30-57.freemail.mail.aliyun.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237895AbiAZHfp (ORCPT ); Wed, 26 Jan 2022 02:35:45 -0500 X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R301e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04423;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=11;SR=0;TI=SMTPD_---0V2uc6kA_1643182539; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0V2uc6kA_1643182539) by smtp.aliyun-inc.com(127.0.0.1); Wed, 26 Jan 2022 15:35:40 +0800 From: Xuan Zhuo To: virtualization@lists.linux-foundation.org, netdev@vger.kernel.org Cc: "Michael S. Tsirkin" , Jason Wang , "David S. Miller" , Jakub Kicinski , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , bpf@vger.kernel.org Subject: [PATCH v3 06/17] virtio_ring: queue_reset: split: add __vring_init_virtqueue() Date: Wed, 26 Jan 2022 15:35:22 +0800 Message-Id: <20220126073533.44994-7-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20220126073533.44994-1-xuanzhuo@linux.alibaba.com> References: <20220126073533.44994-1-xuanzhuo@linux.alibaba.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Extract vq's initialization function __vring_init_virtqueue() from __vring_new_virtqueue() Signed-off-by: Xuan Zhuo --- drivers/virtio/virtio_ring.c | 61 +++++++++++++++++++++++++----------- 1 file changed, 42 insertions(+), 19 deletions(-) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index b6434e36c447..4e1d345b0557 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -2167,23 +2167,17 @@ irqreturn_t vring_interrupt(int irq, void *_vq) EXPORT_SYMBOL_GPL(vring_interrupt); /* Only available for split ring */ -struct virtqueue *__vring_new_virtqueue(unsigned int index, - struct vring vring, - struct virtio_device *vdev, - bool weak_barriers, - bool context, - bool (*notify)(struct virtqueue *), - void (*callback)(struct virtqueue *), - const char *name) +static int __vring_init_virtqueue(struct virtqueue *_vq, + unsigned int index, + struct vring vring, + struct virtio_device *vdev, + bool weak_barriers, + bool context, + bool (*notify)(struct virtqueue *), + void (*callback)(struct virtqueue *), + const char *name) { - struct vring_virtqueue *vq; - - if (virtio_has_feature(vdev, VIRTIO_F_RING_PACKED)) - return NULL; - - vq = kmalloc(sizeof(*vq), GFP_KERNEL); - if (!vq) - return NULL; + struct vring_virtqueue *vq = to_vvq(_vq); vq->packed_ring = false; vq->vq.callback = callback; @@ -2243,13 +2237,42 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index, spin_lock(&vdev->vqs_list_lock); list_add_tail(&vq->vq.list, &vdev->vqs); spin_unlock(&vdev->vqs_list_lock); - return &vq->vq; + return 0; err_extra: kfree(vq->split.desc_state); err_state: - kfree(vq); - return NULL; + return -ENOMEM; +} + +struct virtqueue *__vring_new_virtqueue(unsigned int index, + struct vring vring, + struct virtio_device *vdev, + bool weak_barriers, + bool context, + bool (*notify)(struct virtqueue *), + void (*callback)(struct virtqueue *), + const char *name) +{ + struct vring_virtqueue *vq; + int err; + + if (virtio_has_feature(vdev, VIRTIO_F_RING_PACKED)) + return NULL; + + vq = kmalloc(sizeof(*vq), GFP_KERNEL); + if (!vq) + return NULL; + + err = __vring_init_virtqueue(&vq->vq, index, vring, vdev, weak_barriers, + context, notify, callback, name); + + if (err) { + kfree(vq); + return NULL; + } + + return &vq->vq; } EXPORT_SYMBOL_GPL(__vring_new_virtqueue); From patchwork Wed Jan 26 07:35:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 12724690 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EF140C6369B for ; Wed, 26 Jan 2022 07:36:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237927AbiAZHf7 (ORCPT ); Wed, 26 Jan 2022 02:35:59 -0500 Received: from out30-132.freemail.mail.aliyun.com ([115.124.30.132]:40438 "EHLO out30-132.freemail.mail.aliyun.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237900AbiAZHfp (ORCPT ); Wed, 26 Jan 2022 02:35:45 -0500 X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R171e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e01424;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=11;SR=0;TI=SMTPD_---0V2ubkkL_1643182540; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0V2ubkkL_1643182540) by smtp.aliyun-inc.com(127.0.0.1); Wed, 26 Jan 2022 15:35:41 +0800 From: Xuan Zhuo To: virtualization@lists.linux-foundation.org, netdev@vger.kernel.org Cc: "Michael S. Tsirkin" , Jason Wang , "David S. Miller" , Jakub Kicinski , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , bpf@vger.kernel.org Subject: [PATCH v3 07/17] virtio_ring: queue_reset: split: support enable reset queue Date: Wed, 26 Jan 2022 15:35:23 +0800 Message-Id: <20220126073533.44994-8-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20220126073533.44994-1-xuanzhuo@linux.alibaba.com> References: <20220126073533.44994-1-xuanzhuo@linux.alibaba.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org The purpose of this patch is to make vring split support re-enable reset vq. The core idea is that when calling vring_create_virtqueue_split, use the existing vq instead of re-allocating vq. vq is passed in through function parameters. Modify vring_create_virtqueue() to vring_setup_virtqueue(), and add a new parameter vq. At the same time, add a new inline function, the definition of the function is exactly the same as the original vring_create_virtqueue(). Signed-off-by: Xuan Zhuo --- drivers/virtio/virtio_ring.c | 41 ++++++++++++++++++++++++++---------- include/linux/virtio.h | 1 + include/linux/virtio_ring.h | 37 +++++++++++++++++++++++--------- 3 files changed, 58 insertions(+), 21 deletions(-) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index 4e1d345b0557..a93a677008ba 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -198,6 +198,15 @@ struct vring_virtqueue { #endif }; +static int __vring_init_virtqueue(struct virtqueue *_vq, + unsigned int index, + struct vring vring, + struct virtio_device *vdev, + bool weak_barriers, + bool context, + bool (*notify)(struct virtqueue *), + void (*callback)(struct virtqueue *), + const char *name); /* * Helpers. @@ -925,9 +934,9 @@ static struct virtqueue *vring_create_virtqueue_split( bool context, bool (*notify)(struct virtqueue *), void (*callback)(struct virtqueue *), - const char *name) + const char *name, + struct virtqueue *vq) { - struct virtqueue *vq; void *queue = NULL; dma_addr_t dma_addr; size_t queue_size_in_bytes; @@ -964,12 +973,17 @@ static struct virtqueue *vring_create_virtqueue_split( queue_size_in_bytes = vring_size(num, vring_align); vring_init(&vring, num, queue, vring_align); - vq = __vring_new_virtqueue(index, vring, vdev, weak_barriers, context, - notify, callback, name); if (!vq) { - vring_free_queue(vdev, queue_size_in_bytes, queue, - dma_addr); - return NULL; + vq = __vring_new_virtqueue(index, vring, vdev, weak_barriers, + context, notify, callback, name); + if (!vq) + goto err; + + } else { + if (__vring_init_virtqueue(vq, index, vring, vdev, + weak_barriers, context, notify, + callback, name)) + goto err; } to_vvq(vq)->split.queue_dma_addr = dma_addr; @@ -977,6 +991,9 @@ static struct virtqueue *vring_create_virtqueue_split( to_vvq(vq)->we_own_ring = true; return vq; +err: + vring_free_queue(vdev, queue_size_in_bytes, queue, dma_addr); + return NULL; } @@ -2185,6 +2202,7 @@ static int __vring_init_virtqueue(struct virtqueue *_vq, vq->vq.name = name; vq->vq.num_free = vring.num; vq->vq.index = index; + vq->vq.reset = false; vq->we_own_ring = false; vq->notify = notify; vq->weak_barriers = weak_barriers; @@ -2276,7 +2294,7 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index, } EXPORT_SYMBOL_GPL(__vring_new_virtqueue); -struct virtqueue *vring_create_virtqueue( +struct virtqueue *vring_setup_virtqueue( unsigned int index, unsigned int num, unsigned int vring_align, @@ -2286,7 +2304,8 @@ struct virtqueue *vring_create_virtqueue( bool context, bool (*notify)(struct virtqueue *), void (*callback)(struct virtqueue *), - const char *name) + const char *name, + struct virtqueue *vq) { if (virtio_has_feature(vdev, VIRTIO_F_RING_PACKED)) @@ -2296,9 +2315,9 @@ struct virtqueue *vring_create_virtqueue( return vring_create_virtqueue_split(index, num, vring_align, vdev, weak_barriers, may_reduce_num, - context, notify, callback, name); + context, notify, callback, name, vq); } -EXPORT_SYMBOL_GPL(vring_create_virtqueue); +EXPORT_SYMBOL_GPL(vring_setup_virtqueue); /* Only available for split ring */ struct virtqueue *vring_new_virtqueue(unsigned int index, diff --git a/include/linux/virtio.h b/include/linux/virtio.h index 41edbc01ffa4..98c89cbeafa1 100644 --- a/include/linux/virtio.h +++ b/include/linux/virtio.h @@ -32,6 +32,7 @@ struct virtqueue { unsigned int index; unsigned int num_free; void *priv; + bool reset; }; int virtqueue_add_outbuf(struct virtqueue *vq, diff --git a/include/linux/virtio_ring.h b/include/linux/virtio_ring.h index b485b13fa50b..e90323fce4bf 100644 --- a/include/linux/virtio_ring.h +++ b/include/linux/virtio_ring.h @@ -65,16 +65,33 @@ struct virtqueue; * expected. The caller should query virtqueue_get_vring_size to learn * the actual size of the ring. */ -struct virtqueue *vring_create_virtqueue(unsigned int index, - unsigned int num, - unsigned int vring_align, - struct virtio_device *vdev, - bool weak_barriers, - bool may_reduce_num, - bool ctx, - bool (*notify)(struct virtqueue *vq), - void (*callback)(struct virtqueue *vq), - const char *name); +struct virtqueue *vring_setup_virtqueue(unsigned int index, + unsigned int num, + unsigned int vring_align, + struct virtio_device *vdev, + bool weak_barriers, + bool may_reduce_num, + bool ctx, + bool (*notify)(struct virtqueue *vq), + void (*callback)(struct virtqueue *vq), + const char *name, + struct virtqueue *vq); + +static inline struct virtqueue *vring_create_virtqueue(unsigned int index, + unsigned int num, + unsigned int vring_align, + struct virtio_device *vdev, + bool weak_barriers, + bool may_reduce_num, + bool ctx, + bool (*notify)(struct virtqueue *vq), + void (*callback)(struct virtqueue *vq), + const char *name) +{ + return vring_setup_virtqueue(index, num, vring_align, vdev, + weak_barriers, may_reduce_num, ctx, + notify, callback, name, NULL); +} /* Creates a virtqueue with a custom layout. */ struct virtqueue *__vring_new_virtqueue(unsigned int index, From patchwork Wed Jan 26 07:35:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 12724702 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C37E9C28CF5 for ; Wed, 26 Jan 2022 07:36:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238019AbiAZHgQ (ORCPT ); Wed, 26 Jan 2022 02:36:16 -0500 Received: from out30-56.freemail.mail.aliyun.com ([115.124.30.56]:50685 "EHLO out30-56.freemail.mail.aliyun.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237915AbiAZHfr (ORCPT ); Wed, 26 Jan 2022 02:35:47 -0500 X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R101e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04426;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=11;SR=0;TI=SMTPD_---0V2uPGNw_1643182542; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0V2uPGNw_1643182542) by smtp.aliyun-inc.com(127.0.0.1); Wed, 26 Jan 2022 15:35:42 +0800 From: Xuan Zhuo To: virtualization@lists.linux-foundation.org, netdev@vger.kernel.org Cc: "Michael S. Tsirkin" , Jason Wang , "David S. Miller" , Jakub Kicinski , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , bpf@vger.kernel.org Subject: [PATCH v3 08/17] virtio_ring: queue_reset: packed: support enable reset queue Date: Wed, 26 Jan 2022 15:35:24 +0800 Message-Id: <20220126073533.44994-9-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20220126073533.44994-1-xuanzhuo@linux.alibaba.com> References: <20220126073533.44994-1-xuanzhuo@linux.alibaba.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org The purpose of this patch is to make vring packed support re-enable reset vq. The core idea is that when calling vring_create_virtqueue_packed, use the existing vq instead of re-allocating vq. vq is passed in through function parameters. Signed-off-by: Xuan Zhuo --- drivers/virtio/virtio_ring.c | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index a93a677008ba..4beb7c7127c1 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -1680,7 +1680,8 @@ static struct virtqueue *vring_create_virtqueue_packed( bool context, bool (*notify)(struct virtqueue *), void (*callback)(struct virtqueue *), - const char *name) + const char *name, + struct virtqueue *_vq) { struct vring_virtqueue *vq; struct vring_packed_desc *ring; @@ -1710,15 +1711,20 @@ static struct virtqueue *vring_create_virtqueue_packed( if (!device) goto err_device; - vq = kmalloc(sizeof(*vq), GFP_KERNEL); - if (!vq) - goto err_vq; + if (_vq) { + vq = to_vvq(_vq); + } else { + vq = kmalloc(sizeof(*vq), GFP_KERNEL); + if (!vq) + goto err_vq; + } vq->vq.callback = callback; vq->vq.vdev = vdev; vq->vq.name = name; vq->vq.num_free = num; vq->vq.index = index; + vq->vq.reset = false; vq->we_own_ring = true; vq->notify = notify; vq->weak_barriers = weak_barriers; @@ -1789,7 +1795,8 @@ static struct virtqueue *vring_create_virtqueue_packed( err_desc_extra: kfree(vq->packed.desc_state); err_desc_state: - kfree(vq); + if (!_vq) + kfree(vq); err_vq: vring_free_queue(vdev, event_size_in_bytes, device, device_event_dma_addr); err_device: @@ -2311,7 +2318,7 @@ struct virtqueue *vring_setup_virtqueue( if (virtio_has_feature(vdev, VIRTIO_F_RING_PACKED)) return vring_create_virtqueue_packed(index, num, vring_align, vdev, weak_barriers, may_reduce_num, - context, notify, callback, name); + context, notify, callback, name, vq); return vring_create_virtqueue_split(index, num, vring_align, vdev, weak_barriers, may_reduce_num, From patchwork Wed Jan 26 07:35:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 12724701 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 651E3C2BA4C for ; Wed, 26 Jan 2022 07:36:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238012AbiAZHgP (ORCPT ); Wed, 26 Jan 2022 02:36:15 -0500 Received: from out30-43.freemail.mail.aliyun.com ([115.124.30.43]:33398 "EHLO out30-43.freemail.mail.aliyun.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237917AbiAZHfr (ORCPT ); Wed, 26 Jan 2022 02:35:47 -0500 X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R361e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04423;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=11;SR=0;TI=SMTPD_---0V2ubkkx_1643182543; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0V2ubkkx_1643182543) by smtp.aliyun-inc.com(127.0.0.1); Wed, 26 Jan 2022 15:35:43 +0800 From: Xuan Zhuo To: virtualization@lists.linux-foundation.org, netdev@vger.kernel.org Cc: "Michael S. Tsirkin" , Jason Wang , "David S. Miller" , Jakub Kicinski , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , bpf@vger.kernel.org Subject: [PATCH v3 09/17] virtio_ring: queue_reset: add vring_reset_virtqueue() Date: Wed, 26 Jan 2022 15:35:25 +0800 Message-Id: <20220126073533.44994-10-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20220126073533.44994-1-xuanzhuo@linux.alibaba.com> References: <20220126073533.44994-1-xuanzhuo@linux.alibaba.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Added vring_reset_virtqueue() for reset vring_virtqueue. In this process, vq is removed from the vdev->vqs queue. And the memory of the ring is released Signed-off-by: Xuan Zhuo --- drivers/virtio/virtio_ring.c | 12 +++++++++++- include/linux/virtio_ring.h | 5 +++++ 2 files changed, 16 insertions(+), 1 deletion(-) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index 4beb7c7127c1..bba9f3c67b33 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -2391,11 +2391,21 @@ void vring_del_virtqueue(struct virtqueue *_vq) { struct vring_virtqueue *vq = to_vvq(_vq); - __vring_del_virtqueue(vq); + if (!_vq->reset) + __vring_del_virtqueue(vq); kfree(vq); } EXPORT_SYMBOL_GPL(vring_del_virtqueue); +void vring_reset_virtqueue(struct virtqueue *_vq) +{ + struct vring_virtqueue *vq = to_vvq(_vq); + + __vring_del_virtqueue(vq); + _vq->reset = true; +} +EXPORT_SYMBOL_GPL(vring_reset_virtqueue); + /* Manipulates transport-specific feature bits. */ void vring_transport_features(struct virtio_device *vdev) { diff --git a/include/linux/virtio_ring.h b/include/linux/virtio_ring.h index e90323fce4bf..84b55fb8686d 100644 --- a/include/linux/virtio_ring.h +++ b/include/linux/virtio_ring.h @@ -124,6 +124,11 @@ struct virtqueue *vring_new_virtqueue(unsigned int index, */ void vring_del_virtqueue(struct virtqueue *vq); +/* + * Resets a virtqueue. Just frees the ring, not free vq. + */ +void vring_reset_virtqueue(struct virtqueue *vq); + /* Filter out transport-specific feature bits. */ void vring_transport_features(struct virtio_device *vdev); From patchwork Wed Jan 26 07:35:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 12724691 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E00A8C28CF5 for ; Wed, 26 Jan 2022 07:36:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237935AbiAZHgA (ORCPT ); Wed, 26 Jan 2022 02:36:00 -0500 Received: from out30-133.freemail.mail.aliyun.com ([115.124.30.133]:59721 "EHLO out30-133.freemail.mail.aliyun.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237916AbiAZHfr (ORCPT ); Wed, 26 Jan 2022 02:35:47 -0500 X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R111e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04394;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=11;SR=0;TI=SMTPD_---0V2ubkl4_1643182544; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0V2ubkl4_1643182544) by smtp.aliyun-inc.com(127.0.0.1); Wed, 26 Jan 2022 15:35:44 +0800 From: Xuan Zhuo To: virtualization@lists.linux-foundation.org, netdev@vger.kernel.org Cc: "Michael S. Tsirkin" , Jason Wang , "David S. Miller" , Jakub Kicinski , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , bpf@vger.kernel.org Subject: [PATCH v3 10/17] virtio_pci: queue_reset: update struct virtio_pci_common_cfg and option functions Date: Wed, 26 Jan 2022 15:35:26 +0800 Message-Id: <20220126073533.44994-11-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20220126073533.44994-1-xuanzhuo@linux.alibaba.com> References: <20220126073533.44994-1-xuanzhuo@linux.alibaba.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Add queue_reset in virtio_pci_common_cfg, and add related operation functions. Signed-off-by: Xuan Zhuo --- drivers/virtio/virtio_pci_modern_dev.c | 28 ++++++++++++++++++++++++++ include/linux/virtio_pci_modern.h | 2 ++ include/uapi/linux/virtio_pci.h | 1 + 3 files changed, 31 insertions(+) diff --git a/drivers/virtio/virtio_pci_modern_dev.c b/drivers/virtio/virtio_pci_modern_dev.c index e11ed748e661..be02f812a2f6 100644 --- a/drivers/virtio/virtio_pci_modern_dev.c +++ b/drivers/virtio/virtio_pci_modern_dev.c @@ -463,6 +463,34 @@ void vp_modern_set_status(struct virtio_pci_modern_device *mdev, } EXPORT_SYMBOL_GPL(vp_modern_set_status); +/* + * vp_modern_get_queue_reset - get the queue reset status + * @mdev: the modern virtio-pci device + * @index: queue index + */ +int vp_modern_get_queue_reset(struct virtio_pci_modern_device *mdev, u16 index) +{ + struct virtio_pci_common_cfg __iomem *cfg = mdev->common; + + vp_iowrite16(index, &cfg->queue_select); + return vp_ioread16(&cfg->queue_reset); +} +EXPORT_SYMBOL_GPL(vp_modern_get_queue_reset); + +/* + * vp_modern_set_queue_reset - reset the queue + * @mdev: the modern virtio-pci device + * @index: queue index + */ +void vp_modern_set_queue_reset(struct virtio_pci_modern_device *mdev, u16 index) +{ + struct virtio_pci_common_cfg __iomem *cfg = mdev->common; + + vp_iowrite16(index, &cfg->queue_select); + vp_iowrite16(1, &cfg->queue_reset); +} +EXPORT_SYMBOL_GPL(vp_modern_set_queue_reset); + /* * vp_modern_queue_vector - set the MSIX vector for a specific virtqueue * @mdev: the modern virtio-pci device diff --git a/include/linux/virtio_pci_modern.h b/include/linux/virtio_pci_modern.h index eb2bd9b4077d..cc4154dd7b28 100644 --- a/include/linux/virtio_pci_modern.h +++ b/include/linux/virtio_pci_modern.h @@ -106,4 +106,6 @@ void __iomem * vp_modern_map_vq_notify(struct virtio_pci_modern_device *mdev, u16 index, resource_size_t *pa); int vp_modern_probe(struct virtio_pci_modern_device *mdev); void vp_modern_remove(struct virtio_pci_modern_device *mdev); +int vp_modern_get_queue_reset(struct virtio_pci_modern_device *mdev, u16 index); +void vp_modern_set_queue_reset(struct virtio_pci_modern_device *mdev, u16 index); #endif diff --git a/include/uapi/linux/virtio_pci.h b/include/uapi/linux/virtio_pci.h index 492c89f56c6a..bf10f349087f 100644 --- a/include/uapi/linux/virtio_pci.h +++ b/include/uapi/linux/virtio_pci.h @@ -165,6 +165,7 @@ struct virtio_pci_common_cfg { __le32 queue_used_lo; /* read-write */ __le32 queue_used_hi; /* read-write */ __le16 queue_notify_data; /* read-write */ + __le16 queue_reset; /* read-write */ }; /* Fields in VIRTIO_PCI_CAP_PCI_CFG: */ From patchwork Wed Jan 26 07:35:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 12724692 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2C556C636C9 for ; Wed, 26 Jan 2022 07:36:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237902AbiAZHgB (ORCPT ); Wed, 26 Jan 2022 02:36:01 -0500 Received: from out30-130.freemail.mail.aliyun.com ([115.124.30.130]:34955 "EHLO out30-130.freemail.mail.aliyun.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237920AbiAZHfs (ORCPT ); Wed, 26 Jan 2022 02:35:48 -0500 X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R141e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04395;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=11;SR=0;TI=SMTPD_---0V2uVHmm_1643182545; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0V2uVHmm_1643182545) by smtp.aliyun-inc.com(127.0.0.1); Wed, 26 Jan 2022 15:35:46 +0800 From: Xuan Zhuo To: virtualization@lists.linux-foundation.org, netdev@vger.kernel.org Cc: "Michael S. Tsirkin" , Jason Wang , "David S. Miller" , Jakub Kicinski , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , bpf@vger.kernel.org Subject: [PATCH v3 11/17] virtio_pci: queue_reset: release vq by vp_dev->vqs Date: Wed, 26 Jan 2022 15:35:27 +0800 Message-Id: <20220126073533.44994-12-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20220126073533.44994-1-xuanzhuo@linux.alibaba.com> References: <20220126073533.44994-1-xuanzhuo@linux.alibaba.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org In the process of queue reset, vq leaves vdev->vqs, so the original processing logic may miss some vq. So modify the processing method of releasing vq. Release vq by listing vqs. Signed-off-by: Xuan Zhuo --- drivers/virtio/virtio_pci_common.c | 22 ++++++++++++++++++---- drivers/virtio/virtio_pci_common.h | 2 ++ 2 files changed, 20 insertions(+), 4 deletions(-) diff --git a/drivers/virtio/virtio_pci_common.c b/drivers/virtio/virtio_pci_common.c index fdbde1db5ec5..6b2573ec1ae8 100644 --- a/drivers/virtio/virtio_pci_common.c +++ b/drivers/virtio/virtio_pci_common.c @@ -260,12 +260,20 @@ static void vp_del_vq(struct virtqueue *vq) void vp_del_vqs(struct virtio_device *vdev) { struct virtio_pci_device *vp_dev = to_vp_device(vdev); - struct virtqueue *vq, *n; - int i; + struct virtio_pci_vq_info *info; + struct virtqueue *vq; + int i, v; + + for (i = 0; i < vp_dev->nvqs; ++i) { + + info = vp_dev->vqs[i]; + if (!info) + continue; + + vq = info->vq; - list_for_each_entry_safe(vq, n, &vdev->vqs, list) { if (vp_dev->per_vq_vectors) { - int v = vp_dev->vqs[vq->index]->msix_vector; + v = info->msix_vector; if (v != VIRTIO_MSI_NO_VECTOR) { int irq = pci_irq_vector(vp_dev->pci_dev, v); @@ -275,6 +283,7 @@ void vp_del_vqs(struct virtio_device *vdev) } } vp_del_vq(vq); + vp_dev->vqs[i] = NULL; } vp_dev->per_vq_vectors = false; @@ -308,6 +317,7 @@ void vp_del_vqs(struct virtio_device *vdev) vp_dev->msix_affinity_masks = NULL; kfree(vp_dev->vqs); vp_dev->vqs = NULL; + vp_dev->nvqs = 0; } static int vp_find_vqs_msix(struct virtio_device *vdev, unsigned nvqs, @@ -324,6 +334,8 @@ static int vp_find_vqs_msix(struct virtio_device *vdev, unsigned nvqs, if (!vp_dev->vqs) return -ENOMEM; + vp_dev->nvqs = nvqs; + if (per_vq_vectors) { /* Best option: one for change interrupt, one per vq. */ nvectors = 1; @@ -395,6 +407,8 @@ static int vp_find_vqs_intx(struct virtio_device *vdev, unsigned nvqs, if (!vp_dev->vqs) return -ENOMEM; + vp_dev->nvqs = nvqs; + err = request_irq(vp_dev->pci_dev->irq, vp_interrupt, IRQF_SHARED, dev_name(&vdev->dev), vp_dev); if (err) diff --git a/drivers/virtio/virtio_pci_common.h b/drivers/virtio/virtio_pci_common.h index 23f6c5c678d5..392d990b7c73 100644 --- a/drivers/virtio/virtio_pci_common.h +++ b/drivers/virtio/virtio_pci_common.h @@ -60,6 +60,8 @@ struct virtio_pci_device { /* array of all queues for house-keeping */ struct virtio_pci_vq_info **vqs; + u32 nvqs; + /* MSI-X support */ int msix_enabled; int intx_enabled; From patchwork Wed Jan 26 07:35:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 12724699 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4629BC28CF5 for ; Wed, 26 Jan 2022 07:36:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237983AbiAZHgK (ORCPT ); Wed, 26 Jan 2022 02:36:10 -0500 Received: from out30-132.freemail.mail.aliyun.com ([115.124.30.132]:46590 "EHLO out30-132.freemail.mail.aliyun.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237929AbiAZHfv (ORCPT ); Wed, 26 Jan 2022 02:35:51 -0500 X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R151e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04426;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=11;SR=0;TI=SMTPD_---0V2uPGOq_1643182546; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0V2uPGOq_1643182546) by smtp.aliyun-inc.com(127.0.0.1); Wed, 26 Jan 2022 15:35:47 +0800 From: Xuan Zhuo To: virtualization@lists.linux-foundation.org, netdev@vger.kernel.org Cc: "Michael S. Tsirkin" , Jason Wang , "David S. Miller" , Jakub Kicinski , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , bpf@vger.kernel.org Subject: [PATCH v3 12/17] virtio_pci: queue_reset: setup_vq use vring_setup_virtqueue() Date: Wed, 26 Jan 2022 15:35:28 +0800 Message-Id: <20220126073533.44994-13-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20220126073533.44994-1-xuanzhuo@linux.alibaba.com> References: <20220126073533.44994-1-xuanzhuo@linux.alibaba.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org setup_vq replaces vring_create_virtuque() with vring_setup_virtqueue() to support the need to enable reset vq. Signed-off-by: Xuan Zhuo --- drivers/virtio/virtio_pci_common.c | 8 ++++---- drivers/virtio/virtio_pci_common.h | 4 +++- drivers/virtio/virtio_pci_legacy.c | 5 +++-- drivers/virtio/virtio_pci_modern.c | 20 ++++++++++++++------ 4 files changed, 24 insertions(+), 13 deletions(-) diff --git a/drivers/virtio/virtio_pci_common.c b/drivers/virtio/virtio_pci_common.c index 6b2573ec1ae8..c02936d29a31 100644 --- a/drivers/virtio/virtio_pci_common.c +++ b/drivers/virtio/virtio_pci_common.c @@ -209,7 +209,7 @@ static struct virtqueue *vp_setup_vq(struct virtio_device *vdev, unsigned index, void (*callback)(struct virtqueue *vq), const char *name, bool ctx, - u16 msix_vec) + u16 msix_vec, u16 num) { struct virtio_pci_device *vp_dev = to_vp_device(vdev); struct virtio_pci_vq_info *info = kmalloc(sizeof *info, GFP_KERNEL); @@ -221,7 +221,7 @@ static struct virtqueue *vp_setup_vq(struct virtio_device *vdev, unsigned index, return ERR_PTR(-ENOMEM); vq = vp_dev->setup_vq(vp_dev, info, index, callback, name, ctx, - msix_vec); + msix_vec, NULL, num); if (IS_ERR(vq)) goto out_info; @@ -368,7 +368,7 @@ static int vp_find_vqs_msix(struct virtio_device *vdev, unsigned nvqs, msix_vec = VP_MSIX_VQ_VECTOR; vqs[i] = vp_setup_vq(vdev, queue_idx++, callbacks[i], names[i], ctx ? ctx[i] : false, - msix_vec); + msix_vec, 0); if (IS_ERR(vqs[i])) { err = PTR_ERR(vqs[i]); goto error_find; @@ -423,7 +423,7 @@ static int vp_find_vqs_intx(struct virtio_device *vdev, unsigned nvqs, } vqs[i] = vp_setup_vq(vdev, queue_idx++, callbacks[i], names[i], ctx ? ctx[i] : false, - VIRTIO_MSI_NO_VECTOR); + VIRTIO_MSI_NO_VECTOR, 0); if (IS_ERR(vqs[i])) { err = PTR_ERR(vqs[i]); goto out_del_vqs; diff --git a/drivers/virtio/virtio_pci_common.h b/drivers/virtio/virtio_pci_common.h index 392d990b7c73..65db92245e41 100644 --- a/drivers/virtio/virtio_pci_common.h +++ b/drivers/virtio/virtio_pci_common.h @@ -84,7 +84,9 @@ struct virtio_pci_device { void (*callback)(struct virtqueue *vq), const char *name, bool ctx, - u16 msix_vec); + u16 msix_vec, + struct virtqueue *vq, + u16 num); void (*del_vq)(struct virtio_pci_vq_info *info); u16 (*config_vector)(struct virtio_pci_device *vp_dev, u16 vector); diff --git a/drivers/virtio/virtio_pci_legacy.c b/drivers/virtio/virtio_pci_legacy.c index b3f8128b7983..9bc41b764624 100644 --- a/drivers/virtio/virtio_pci_legacy.c +++ b/drivers/virtio/virtio_pci_legacy.c @@ -113,9 +113,10 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev, void (*callback)(struct virtqueue *vq), const char *name, bool ctx, - u16 msix_vec) + u16 msix_vec, + struct virtqueue *vq, + u16 ring_num) { - struct virtqueue *vq; u16 num; int err; u64 q_pfn; diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c index 5455bc041fb6..2ce58de549de 100644 --- a/drivers/virtio/virtio_pci_modern.c +++ b/drivers/virtio/virtio_pci_modern.c @@ -187,11 +187,12 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev, void (*callback)(struct virtqueue *vq), const char *name, bool ctx, - u16 msix_vec) + u16 msix_vec, + struct virtqueue *vq, + u16 ring_num) { struct virtio_pci_modern_device *mdev = &vp_dev->mdev; - struct virtqueue *vq; u16 num; int err; @@ -203,6 +204,13 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev, if (!num || vp_modern_get_queue_enable(mdev, index)) return ERR_PTR(-ENOENT); + if (ring_num) { + if (ring_num > num) + return ERR_PTR(-ENOENT); + + num = ring_num; + } + if (num & (num - 1)) { dev_warn(&vp_dev->pci_dev->dev, "bad queue size %u", num); return ERR_PTR(-EINVAL); @@ -211,10 +219,10 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev, info->msix_vector = msix_vec; /* create the vring */ - vq = vring_create_virtqueue(index, num, - SMP_CACHE_BYTES, &vp_dev->vdev, - true, true, ctx, - vp_notify, callback, name); + vq = vring_setup_virtqueue(index, num, + SMP_CACHE_BYTES, &vp_dev->vdev, + true, true, ctx, + vp_notify, callback, name, vq); if (!vq) return ERR_PTR(-ENOMEM); From patchwork Wed Jan 26 07:35:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 12724700 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5C89EC28CF5 for ; Wed, 26 Jan 2022 07:36:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237999AbiAZHgM (ORCPT ); Wed, 26 Jan 2022 02:36:12 -0500 Received: from out30-42.freemail.mail.aliyun.com ([115.124.30.42]:39588 "EHLO out30-42.freemail.mail.aliyun.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237925AbiAZHfu (ORCPT ); Wed, 26 Jan 2022 02:35:50 -0500 X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R161e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04423;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=11;SR=0;TI=SMTPD_---0V2uPGP7_1643182547; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0V2uPGP7_1643182547) by smtp.aliyun-inc.com(127.0.0.1); Wed, 26 Jan 2022 15:35:48 +0800 From: Xuan Zhuo To: virtualization@lists.linux-foundation.org, netdev@vger.kernel.org Cc: "Michael S. Tsirkin" , Jason Wang , "David S. Miller" , Jakub Kicinski , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , bpf@vger.kernel.org Subject: [PATCH v3 13/17] virtio_pci: queue_reset: support VIRTIO_F_RING_RESET Date: Wed, 26 Jan 2022 15:35:29 +0800 Message-Id: <20220126073533.44994-14-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20220126073533.44994-1-xuanzhuo@linux.alibaba.com> References: <20220126073533.44994-1-xuanzhuo@linux.alibaba.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org This patch implements virtio pci support for QUEUE RESET. Performing reset on a queue is divided into two steps: 1. reset_vq: reset one vq 2. enable_reset_vq: re-enable the reset queue In the first step, these tasks will be completed: 1. notify the hardware queue to reset 2. recycle the buffer from vq 3. release the ring of the vq The process of enable reset vq: vp_modern_enable_reset_vq() vp_enable_reset_vq() __vp_setup_vq() setup_vq() vring_setup_virtqueue() In this process, we added two parameters, vq and num, and finally passed them to vring_setup_virtqueue(). And reuse the original info and vq. Signed-off-by: Xuan Zhuo --- drivers/virtio/virtio_pci_common.c | 36 +++++++---- drivers/virtio/virtio_pci_common.h | 5 ++ drivers/virtio/virtio_pci_modern.c | 100 +++++++++++++++++++++++++++++ 3 files changed, 128 insertions(+), 13 deletions(-) diff --git a/drivers/virtio/virtio_pci_common.c b/drivers/virtio/virtio_pci_common.c index c02936d29a31..ad21638fbf66 100644 --- a/drivers/virtio/virtio_pci_common.c +++ b/drivers/virtio/virtio_pci_common.c @@ -205,23 +205,28 @@ static int vp_request_msix_vectors(struct virtio_device *vdev, int nvectors, return err; } -static struct virtqueue *vp_setup_vq(struct virtio_device *vdev, unsigned index, - void (*callback)(struct virtqueue *vq), - const char *name, - bool ctx, - u16 msix_vec, u16 num) +struct virtqueue *vp_setup_vq(struct virtio_device *vdev, unsigned index, + void (*callback)(struct virtqueue *vq), + const char *name, + bool ctx, + u16 msix_vec, u16 num) { struct virtio_pci_device *vp_dev = to_vp_device(vdev); - struct virtio_pci_vq_info *info = kmalloc(sizeof *info, GFP_KERNEL); + struct virtio_pci_vq_info *info; struct virtqueue *vq; unsigned long flags; - /* fill out our structure that represents an active queue */ - if (!info) - return ERR_PTR(-ENOMEM); + info = vp_dev->vqs[index]; + if (!info) { + info = kzalloc(sizeof *info, GFP_KERNEL); + + /* fill out our structure that represents an active queue */ + if (!info) + return ERR_PTR(-ENOMEM); + } vq = vp_dev->setup_vq(vp_dev, info, index, callback, name, ctx, - msix_vec, NULL, num); + msix_vec, info->vq, num); if (IS_ERR(vq)) goto out_info; @@ -238,6 +243,9 @@ static struct virtqueue *vp_setup_vq(struct virtio_device *vdev, unsigned index, return vq; out_info: + if (info->vq && info->vq->reset) + return vq; + kfree(info); return vq; } @@ -248,9 +256,11 @@ static void vp_del_vq(struct virtqueue *vq) struct virtio_pci_vq_info *info = vp_dev->vqs[vq->index]; unsigned long flags; - spin_lock_irqsave(&vp_dev->lock, flags); - list_del(&info->node); - spin_unlock_irqrestore(&vp_dev->lock, flags); + if (!vq->reset) { + spin_lock_irqsave(&vp_dev->lock, flags); + list_del(&info->node); + spin_unlock_irqrestore(&vp_dev->lock, flags); + } vp_dev->del_vq(info); kfree(info); diff --git a/drivers/virtio/virtio_pci_common.h b/drivers/virtio/virtio_pci_common.h index 65db92245e41..c1d15f7c0be4 100644 --- a/drivers/virtio/virtio_pci_common.h +++ b/drivers/virtio/virtio_pci_common.h @@ -119,6 +119,11 @@ int vp_find_vqs(struct virtio_device *vdev, unsigned nvqs, struct virtqueue *vqs[], vq_callback_t *callbacks[], const char * const names[], const bool *ctx, struct irq_affinity *desc); +struct virtqueue *vp_setup_vq(struct virtio_device *vdev, unsigned index, + void (*callback)(struct virtqueue *vq), + const char *name, + bool ctx, + u16 msix_vec, u16 num); const char *vp_bus_name(struct virtio_device *vdev); /* Setup the affinity for a virtqueue: diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c index 2ce58de549de..6789411169e4 100644 --- a/drivers/virtio/virtio_pci_modern.c +++ b/drivers/virtio/virtio_pci_modern.c @@ -34,6 +34,9 @@ static void vp_transport_features(struct virtio_device *vdev, u64 features) if ((features & BIT_ULL(VIRTIO_F_SR_IOV)) && pci_find_ext_capability(pci_dev, PCI_EXT_CAP_ID_SRIOV)) __virtio_set_bit(vdev, VIRTIO_F_SR_IOV); + + if (features & BIT_ULL(VIRTIO_F_RING_RESET)) + __virtio_set_bit(vdev, VIRTIO_F_RING_RESET); } /* virtio config->finalize_features() implementation */ @@ -176,6 +179,94 @@ static void vp_reset(struct virtio_device *vdev) vp_disable_cbs(vdev); } +static int vp_modern_reset_vq(struct virtio_reset_vq *param) +{ + struct virtio_pci_device *vp_dev = to_vp_device(param->vdev); + struct virtio_pci_modern_device *mdev = &vp_dev->mdev; + struct virtio_pci_vq_info *info; + u16 msix_vec, queue_index; + unsigned long flags; + void *buf; + + if (!virtio_has_feature(param->vdev, VIRTIO_F_RING_RESET)) + return -ENOENT; + + queue_index = param->queue_index; + + vp_modern_set_queue_reset(mdev, queue_index); + + /* After write 1 to queue reset, the driver MUST wait for a read of + * queue reset to return 1. + */ + while (vp_modern_get_queue_reset(mdev, queue_index) != 1) + msleep(1); + + info = vp_dev->vqs[queue_index]; + msix_vec = info->msix_vector; + + /* Disable VQ callback. */ + if (vp_dev->per_vq_vectors && msix_vec != VIRTIO_MSI_NO_VECTOR) + disable_irq(pci_irq_vector(vp_dev->pci_dev, msix_vec)); + + while ((buf = virtqueue_detach_unused_buf(info->vq)) != NULL) + param->free_unused_cb(param, buf); + + /* delete vq */ + spin_lock_irqsave(&vp_dev->lock, flags); + list_del(&info->node); + spin_unlock_irqrestore(&vp_dev->lock, flags); + + INIT_LIST_HEAD(&info->node); + + if (vp_dev->msix_enabled) + vp_modern_queue_vector(mdev, info->vq->index, + VIRTIO_MSI_NO_VECTOR); + + if (!mdev->notify_base) + pci_iounmap(mdev->pci_dev, + (void __force __iomem *)info->vq->priv); + + vring_reset_virtqueue(info->vq); + + return 0; +} + +static struct virtqueue *vp_modern_enable_reset_vq(struct virtio_reset_vq *param) +{ + struct virtio_pci_device *vp_dev = to_vp_device(param->vdev); + struct virtio_pci_modern_device *mdev = &vp_dev->mdev; + struct virtio_pci_vq_info *info; + u16 msix_vec, queue_index; + struct virtqueue *vq; + + if (!virtio_has_feature(param->vdev, VIRTIO_F_RING_RESET)) + return ERR_PTR(-ENOENT); + + queue_index = param->queue_index; + + info = vp_dev->vqs[queue_index]; + + if (!info->vq->reset) + return ERR_PTR(-EPERM); + + /* check queue reset status */ + if (vp_modern_get_queue_reset(mdev, queue_index) != 1) + return ERR_PTR(-EBUSY); + + vq = vp_setup_vq(param->vdev, queue_index, param->callback, param->name, + param->ctx, info->msix_vector, param->ring_num); + if (IS_ERR(vq)) + return vq; + + vp_modern_set_queue_enable(&vp_dev->mdev, vq->index, true); + + msix_vec = vp_dev->vqs[queue_index]->msix_vector; + if (vp_dev->per_vq_vectors && msix_vec != VIRTIO_MSI_NO_VECTOR) + enable_irq(pci_irq_vector(vp_dev->pci_dev, msix_vec)); + + return vq; +} + static u16 vp_config_vector(struct virtio_pci_device *vp_dev, u16 vector) { return vp_modern_config_vector(&vp_dev->mdev, vector); @@ -284,6 +375,11 @@ static void del_vq(struct virtio_pci_vq_info *info) struct virtio_pci_device *vp_dev = to_vp_device(vq->vdev); struct virtio_pci_modern_device *mdev = &vp_dev->mdev; + if (vq->reset) { + vring_del_virtqueue(vq); + return; + } + if (vp_dev->msix_enabled) vp_modern_queue_vector(mdev, vq->index, VIRTIO_MSI_NO_VECTOR); @@ -403,6 +499,8 @@ static const struct virtio_config_ops virtio_pci_config_nodev_ops = { .set_vq_affinity = vp_set_vq_affinity, .get_vq_affinity = vp_get_vq_affinity, .get_shm_region = vp_get_shm_region, + .reset_vq = vp_modern_reset_vq, + .enable_reset_vq = vp_modern_enable_reset_vq, }; static const struct virtio_config_ops virtio_pci_config_ops = { @@ -421,6 +519,8 @@ static const struct virtio_config_ops virtio_pci_config_ops = { .set_vq_affinity = vp_set_vq_affinity, .get_vq_affinity = vp_get_vq_affinity, .get_shm_region = vp_get_shm_region, + .reset_vq = vp_modern_reset_vq, + .enable_reset_vq = vp_modern_enable_reset_vq, }; /* the PCI probing function */ From patchwork Wed Jan 26 07:35:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 12724693 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9C6BDC636D5 for ; Wed, 26 Jan 2022 07:36:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237912AbiAZHgC (ORCPT ); Wed, 26 Jan 2022 02:36:02 -0500 Received: from out30-45.freemail.mail.aliyun.com ([115.124.30.45]:35563 "EHLO out30-45.freemail.mail.aliyun.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237870AbiAZHfw (ORCPT ); Wed, 26 Jan 2022 02:35:52 -0500 X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R171e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04394;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=11;SR=0;TI=SMTPD_---0V2uAwTM_1643182548; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0V2uAwTM_1643182548) by smtp.aliyun-inc.com(127.0.0.1); Wed, 26 Jan 2022 15:35:49 +0800 From: Xuan Zhuo To: virtualization@lists.linux-foundation.org, netdev@vger.kernel.org Cc: "Michael S. Tsirkin" , Jason Wang , "David S. Miller" , Jakub Kicinski , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , bpf@vger.kernel.org Subject: [PATCH v3 14/17] virtio_net: virtnet_tx_timeout() fix style Date: Wed, 26 Jan 2022 15:35:30 +0800 Message-Id: <20220126073533.44994-15-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20220126073533.44994-1-xuanzhuo@linux.alibaba.com> References: <20220126073533.44994-1-xuanzhuo@linux.alibaba.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Replace priv with vi which is commonly used in the virtio-net module. Signed-off-by: Xuan Zhuo --- drivers/net/virtio_net.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index 569eecfbc2cd..97eb4dddba1f 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -2683,8 +2683,8 @@ static int virtnet_set_features(struct net_device *dev, static void virtnet_tx_timeout(struct net_device *dev, unsigned int txqueue) { - struct virtnet_info *priv = netdev_priv(dev); - struct send_queue *sq = &priv->sq[txqueue]; + struct virtnet_info *vi = netdev_priv(dev); + struct send_queue *sq = &vi->sq[txqueue]; struct netdev_queue *txq = netdev_get_tx_queue(dev, txqueue); u64_stats_update_begin(&sq->stats.syncp); From patchwork Wed Jan 26 07:35:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 12724698 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3EB67C28CF5 for ; Wed, 26 Jan 2022 07:36:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237945AbiAZHgH (ORCPT ); Wed, 26 Jan 2022 02:36:07 -0500 Received: from out30-45.freemail.mail.aliyun.com ([115.124.30.45]:45605 "EHLO out30-45.freemail.mail.aliyun.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229721AbiAZHfy (ORCPT ); Wed, 26 Jan 2022 02:35:54 -0500 X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R171e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04426;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=11;SR=0;TI=SMTPD_---0V2uc6mL_1643182549; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0V2uc6mL_1643182549) by smtp.aliyun-inc.com(127.0.0.1); Wed, 26 Jan 2022 15:35:50 +0800 From: Xuan Zhuo To: virtualization@lists.linux-foundation.org, netdev@vger.kernel.org Cc: "Michael S. Tsirkin" , Jason Wang , "David S. Miller" , Jakub Kicinski , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , bpf@vger.kernel.org Subject: [PATCH v3 15/17] virtio_net: virtnet_tx_timeout() stop ref sq->vq Date: Wed, 26 Jan 2022 15:35:31 +0800 Message-Id: <20220126073533.44994-16-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20220126073533.44994-1-xuanzhuo@linux.alibaba.com> References: <20220126073533.44994-1-xuanzhuo@linux.alibaba.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Make virtnet_tx_timeout() no longer refer to vq directly. Because sq->vq may be equal to NULL after implementing rx/tx queue disable/enable. Signed-off-by: Xuan Zhuo --- drivers/net/virtio_net.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index 97eb4dddba1f..2add7fe749b8 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -2692,7 +2692,7 @@ static void virtnet_tx_timeout(struct net_device *dev, unsigned int txqueue) u64_stats_update_end(&sq->stats.syncp); netdev_err(dev, "TX timeout on queue: %u, sq: %s, vq: 0x%x, name: %s, %u usecs ago\n", - txqueue, sq->name, sq->vq->index, sq->vq->name, + txqueue, sq->name, txq2vq(sq - vi->sq), sq->name, jiffies_to_usecs(jiffies - READ_ONCE(txq->trans_start))); } From patchwork Wed Jan 26 07:35:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 12724694 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 293F7C636F5 for ; Wed, 26 Jan 2022 07:36:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237924AbiAZHgC (ORCPT ); Wed, 26 Jan 2022 02:36:02 -0500 Received: from out30-57.freemail.mail.aliyun.com ([115.124.30.57]:36020 "EHLO out30-57.freemail.mail.aliyun.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237959AbiAZHf5 (ORCPT ); Wed, 26 Jan 2022 02:35:57 -0500 X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R111e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04407;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=11;SR=0;TI=SMTPD_---0V2uPGPw_1643182551; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0V2uPGPw_1643182551) by smtp.aliyun-inc.com(127.0.0.1); Wed, 26 Jan 2022 15:35:51 +0800 From: Xuan Zhuo To: virtualization@lists.linux-foundation.org, netdev@vger.kernel.org Cc: "Michael S. Tsirkin" , Jason Wang , "David S. Miller" , Jakub Kicinski , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , bpf@vger.kernel.org Subject: [PATCH v3 16/17] virtio_net: split free_unused_bufs() Date: Wed, 26 Jan 2022 15:35:32 +0800 Message-Id: <20220126073533.44994-17-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20220126073533.44994-1-xuanzhuo@linux.alibaba.com> References: <20220126073533.44994-1-xuanzhuo@linux.alibaba.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org This patch separates two functions for freeing sq buf and rq buf from free_unused_bufs(). When supporting the enable/disable tx/rq queue in the future, it is necessary to support separate recovery of a sq buf or a rq buf. Signed-off-by: Xuan Zhuo --- drivers/net/virtio_net.c | 46 ++++++++++++++++++++++++---------------- 1 file changed, 28 insertions(+), 18 deletions(-) diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index 2add7fe749b8..ea90a1a57c9e 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -2804,33 +2804,43 @@ static void free_receive_page_frags(struct virtnet_info *vi) put_page(vi->rq[i].alloc_frag.page); } +static void virtnet_sq_free_unused_buf(struct virtnet_info *vi, + struct send_queue *sq, void *buf) +{ + if (!is_xdp_frame(buf)) + dev_kfree_skb(buf); + else + xdp_return_frame(ptr_to_xdp(buf)); +} + +static void virtnet_rq_free_unused_buf(struct virtnet_info *vi, + struct receive_queue *rq, void *buf) +{ + if (vi->mergeable_rx_bufs) + put_page(virt_to_head_page(buf)); + else if (vi->big_packets) + give_pages(rq, buf); + else + put_page(virt_to_head_page(buf)); +} + static void free_unused_bufs(struct virtnet_info *vi) { + struct receive_queue *rq; + struct send_queue *sq; void *buf; int i; for (i = 0; i < vi->max_queue_pairs; i++) { - struct virtqueue *vq = vi->sq[i].vq; - while ((buf = virtqueue_detach_unused_buf(vq)) != NULL) { - if (!is_xdp_frame(buf)) - dev_kfree_skb(buf); - else - xdp_return_frame(ptr_to_xdp(buf)); - } + sq = &vi->sq[i]; + while ((buf = virtqueue_detach_unused_buf(sq->vq)) != NULL) + virtnet_sq_free_unused_buf(vi, sq, buf); } for (i = 0; i < vi->max_queue_pairs; i++) { - struct virtqueue *vq = vi->rq[i].vq; - - while ((buf = virtqueue_detach_unused_buf(vq)) != NULL) { - if (vi->mergeable_rx_bufs) { - put_page(virt_to_head_page(buf)); - } else if (vi->big_packets) { - give_pages(&vi->rq[i], buf); - } else { - put_page(virt_to_head_page(buf)); - } - } + rq = &vi->rq[i]; + while ((buf = virtqueue_detach_unused_buf(rq->vq)) != NULL) + virtnet_rq_free_unused_buf(vi, rq, buf); } } From patchwork Wed Jan 26 07:35:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 12724697 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9B5EC63697 for ; Wed, 26 Jan 2022 07:36:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237949AbiAZHgF (ORCPT ); Wed, 26 Jan 2022 02:36:05 -0500 Received: from out30-56.freemail.mail.aliyun.com ([115.124.30.56]:48342 "EHLO out30-56.freemail.mail.aliyun.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237948AbiAZHfz (ORCPT ); Wed, 26 Jan 2022 02:35:55 -0500 X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R211e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04407;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=11;SR=0;TI=SMTPD_---0V2uPGQG_1643182552; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0V2uPGQG_1643182552) by smtp.aliyun-inc.com(127.0.0.1); Wed, 26 Jan 2022 15:35:52 +0800 From: Xuan Zhuo To: virtualization@lists.linux-foundation.org, netdev@vger.kernel.org Cc: "Michael S. Tsirkin" , Jason Wang , "David S. Miller" , Jakub Kicinski , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , bpf@vger.kernel.org Subject: [PATCH v3 17/17] virtio_net: support pair disable/enable Date: Wed, 26 Jan 2022 15:35:33 +0800 Message-Id: <20220126073533.44994-18-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20220126073533.44994-1-xuanzhuo@linux.alibaba.com> References: <20220126073533.44994-1-xuanzhuo@linux.alibaba.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org This patch implements virtio-net rx/tx pair disable/enable functionality based on virtio queue reset. The purpose of the current implementation is to quickly recycle the buffer submitted to vq. In the process of pair disable, in theory, as long as virtio supports queue reset, there will be no exceptions. However, in the process of pari enable, there may be exceptions due to memory allocation. In this case, vq == NULL, but napi will still be enabled. Because napi_disable is similar to a lock, napi_enable must be called after calling napi_disable. Since enable fails, the driver will not receive an interrupt from the device to wake up napi, so the driver is safe. But we still need to add vq checks in some places to ensure safety, such as refill_work(). Signed-off-by: Xuan Zhuo --- drivers/net/virtio_net.c | 168 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 168 insertions(+) diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index ea90a1a57c9e..cf77ef1bad1c 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -1369,6 +1369,9 @@ static void virtnet_napi_enable(struct virtqueue *vq, struct napi_struct *napi) { napi_enable(napi); + if (!vq) + return; + /* If all buffers were filled by other side before we napi_enabled, we * won't get another interrupt, so process any outstanding packets now. * Call local_bh_enable after to trigger softIRQ processing. @@ -1413,6 +1416,10 @@ static void refill_work(struct work_struct *work) struct receive_queue *rq = &vi->rq[i]; napi_disable(&rq->napi); + if (!rq->vq) { + virtnet_napi_enable(rq->vq, &rq->napi); + continue; + } still_empty = !try_fill_recv(vi, rq, GFP_KERNEL); virtnet_napi_enable(rq->vq, &rq->napi); @@ -2871,6 +2878,167 @@ static unsigned int mergeable_min_buf_len(struct virtnet_info *vi, struct virtqu (unsigned int)GOOD_PACKET_LEN); } +static void virtnet_rq_free_unused_buf_cb(struct virtio_reset_vq *param, + void *buf) +{ + virtnet_rq_free_unused_buf(param->vdev->priv, param->data, buf); +} + +static void virtnet_sq_free_unused_buf_cb(struct virtio_reset_vq *param, + void *buf) +{ + virtnet_rq_free_unused_buf(param->vdev->priv, param->data, buf); +} + +static int __virtnet_rx_vq_disable(struct virtnet_info *vi, + struct receive_queue *rq) +{ + struct virtio_reset_vq param = {0}; + int err, qnum; + + qnum = rxq2vq(rq - vi->rq); + + napi_disable(&rq->napi); + + param.vdev = vi->vdev; + param.queue_index = qnum; + param.free_unused_cb = virtnet_rq_free_unused_buf_cb; + param.data = rq; + + err = virtio_reset_vq(¶m); + if (err) { + virtnet_napi_enable(rq->vq, &rq->napi); + return err; + } + + rq->vq = NULL; + + return err; +} + +static int __virtnet_tx_vq_disable(struct virtnet_info *vi, + struct send_queue *sq) +{ + struct virtio_reset_vq param = {0}; + struct netdev_queue *txq; + int err, qnum; + + qnum = txq2vq(sq - vi->sq); + + netif_stop_subqueue(vi->dev, sq - vi->sq); + virtnet_napi_tx_disable(&sq->napi); + + /* wait xmit done */ + txq = netdev_get_tx_queue(vi->dev, qnum); + __netif_tx_lock(txq, raw_smp_processor_id()); + __netif_tx_unlock(txq); + + param.vdev = vi->vdev; + param.queue_index = qnum; + param.free_unused_cb = virtnet_sq_free_unused_buf_cb; + param.data = sq; + + err = virtio_reset_vq(¶m); + if (err) { + virtnet_napi_tx_enable(vi, sq->vq, &sq->napi); + netif_start_subqueue(vi->dev, sq - vi->sq); + return err; + } + + sq->vq = NULL; + + return err; +} + +static int virtnet_pair_disable(struct virtnet_info *vi, int i) +{ + int err; + + err = __virtnet_rx_vq_disable(vi, vi->rq + i); + if (err) + return err; + + return __virtnet_tx_vq_disable(vi, vi->sq + i); +} + +static int virtnet_enable_resetq(struct virtnet_info *vi, + struct receive_queue *rq, + struct send_queue *sq) +{ + struct virtio_reset_vq param = {0}; + vq_callback_t *callback; + struct virtqueue *vq; + const char *name; + int vq_idx; + bool ctx; + + if (rq) { + vq = rq->vq; + vq_idx = rxq2vq(rq - vi->rq); + callback = skb_recv_done; + name = rq->name; + + } else { + vq = sq->vq; + vq_idx = txq2vq(sq - vi->sq); + callback = skb_xmit_done; + name = sq->name; + } + + if (vq) + return -EBUSY; + + if (!vi->big_packets || vi->mergeable_rx_bufs) + ctx = true; + else + ctx = false; + + param.vdev = vi->vdev; + param.queue_index = vq_idx; + param.callback = callback; + param.name = name; + param.ctx = &ctx; + param.ring_num = 0; + + vq = virtio_enable_resetq(¶m); + if (IS_ERR(vq)) + return PTR_ERR(vq); + + if (rq) + rq->vq = vq; + else + sq->vq = vq; + + return 0; +} + +static int virtnet_pair_enable(struct virtnet_info *vi, int i) +{ + struct receive_queue *rq; + struct send_queue *sq; + int err; + + rq = vi->rq + i; + sq = vi->sq + i; + + /* tx */ + err = virtnet_enable_resetq(vi, NULL, sq); + if (err) + goto err; + else + netif_start_subqueue(vi->dev, sq - vi->sq); + + /* rx */ + err = virtnet_enable_resetq(vi, rq, NULL); + if (err) + return err; + +err: + virtnet_napi_tx_enable(vi, sq->vq, &sq->napi); + virtnet_napi_enable(rq->vq, &rq->napi); + return 0; +} + static int virtnet_find_vqs(struct virtnet_info *vi) { vq_callback_t **callbacks;