From patchwork Sat Jan 28 07:17:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 13119710 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A693FC61DA4 for ; Sat, 28 Jan 2023 07:23:51 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pLfYC-0003rZ-Sa; Sat, 28 Jan 2023 02:23:00 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pLfY9-0003qz-96 for qemu-devel@nongnu.org; Sat, 28 Jan 2023 02:22:57 -0500 Received: from out199-15.us.a.mail.aliyun.com ([47.90.199.15]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pLfY5-0000kw-KD for qemu-devel@nongnu.org; Sat, 28 Jan 2023 02:22:56 -0500 X-Alimail-AntiSpam: AC=PASS; BC=-1|-1; BR=01201311R161e4; CH=green; DM=||false|; DS=||; FP=0|-1|-1|-1|0|-1|-1|-1; HT=ay29a033018045192; MF=xuanzhuo@linux.alibaba.com; NM=1; PH=DS; RN=3; SR=0; TI=SMTPD_---0VaGyhOl_1674890245; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0VaGyhOl_1674890245) by smtp.aliyun-inc.com; Sat, 28 Jan 2023 15:17:26 +0800 From: Xuan Zhuo To: qemu-devel@nongnu.org Cc: "Michael S. Tsirkin" , Jason Wang Subject: [PATCH 1/3] virtio: move struct VirtQueue to include file Date: Sat, 28 Jan 2023 15:17:22 +0800 Message-Id: <20230128071724.33677-2-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.32.0.3.g01195cf9f In-Reply-To: <20230128071724.33677-1-xuanzhuo@linux.alibaba.com> References: <20230128071724.33677-1-xuanzhuo@linux.alibaba.com> MIME-Version: 1.0 X-Git-Hash: 850524d63b Received-SPF: pass client-ip=47.90.199.15; envelope-from=xuanzhuo@linux.alibaba.com; helo=out199-15.us.a.mail.aliyun.com X-Spam_score_int: -98 X-Spam_score: -9.9 X-Spam_bar: --------- X-Spam_report: (-9.9 / 5.0 requ) BAYES_00=-1.9, ENV_AND_HDR_SPF_MATCH=-0.5, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, UNPARSEABLE_RELAY=0.001, USER_IN_DEF_SPF_WL=-7.5 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org This patch move struct VirtQueue into virtio.h. In order to implement Queue Reset, we have to record the queue reset status of in struct VirtQueue and provide it to device. Signed-off-by: Xuan Zhuo --- hw/virtio/virtio.c | 49 ----------------------------------- include/hw/virtio/virtio.h | 52 ++++++++++++++++++++++++++++++++++++-- 2 files changed, 50 insertions(+), 51 deletions(-) diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c index f35178f5fc..03077b2ecf 100644 --- a/hw/virtio/virtio.c +++ b/hw/virtio/virtio.c @@ -101,60 +101,11 @@ typedef struct VRingMemoryRegionCaches { MemoryRegionCache used; } VRingMemoryRegionCaches; -typedef struct VRing -{ - unsigned int num; - unsigned int num_default; - unsigned int align; - hwaddr desc; - hwaddr avail; - hwaddr used; - VRingMemoryRegionCaches *caches; -} VRing; - typedef struct VRingPackedDescEvent { uint16_t off_wrap; uint16_t flags; } VRingPackedDescEvent ; -struct VirtQueue -{ - VRing vring; - VirtQueueElement *used_elems; - - /* Next head to pop */ - uint16_t last_avail_idx; - bool last_avail_wrap_counter; - - /* Last avail_idx read from VQ. */ - uint16_t shadow_avail_idx; - bool shadow_avail_wrap_counter; - - uint16_t used_idx; - bool used_wrap_counter; - - /* Last used index value we have signalled on */ - uint16_t signalled_used; - - /* Last used index value we have signalled on */ - bool signalled_used_valid; - - /* Notification enabled? */ - bool notification; - - uint16_t queue_index; - - unsigned int inuse; - - uint16_t vector; - VirtIOHandleOutput handle_output; - VirtIODevice *vdev; - EventNotifier guest_notifier; - EventNotifier host_notifier; - bool host_notifier_enabled; - QLIST_ENTRY(VirtQueue) node; -}; - const char *virtio_device_names[] = { [VIRTIO_ID_NET] = "virtio-net", [VIRTIO_ID_BLOCK] = "virtio-blk", diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h index 77c6c55929..1c0d77c670 100644 --- a/include/hw/virtio/virtio.h +++ b/include/hw/virtio/virtio.h @@ -214,6 +214,56 @@ struct VirtioDeviceClass { struct vhost_dev *(*get_vhost)(VirtIODevice *vdev); }; +typedef struct VRingMemoryRegionCaches VRingMemoryRegionCaches; +typedef void (*VirtIOHandleOutput)(VirtIODevice *, VirtQueue *); + +typedef struct VRing { + unsigned int num; + unsigned int num_default; + unsigned int align; + hwaddr desc; + hwaddr avail; + hwaddr used; + VRingMemoryRegionCaches *caches; +} VRing; + +struct VirtQueue { + VRing vring; + VirtQueueElement *used_elems; + + /* Next head to pop */ + uint16_t last_avail_idx; + bool last_avail_wrap_counter; + + /* Last avail_idx read from VQ. */ + uint16_t shadow_avail_idx; + bool shadow_avail_wrap_counter; + + uint16_t used_idx; + bool used_wrap_counter; + + /* Last used index value we have signalled on */ + uint16_t signalled_used; + + /* Last used index value we have signalled on */ + bool signalled_used_valid; + + /* Notification enabled? */ + bool notification; + + uint16_t queue_index; + + unsigned int inuse; + + uint16_t vector; + VirtIOHandleOutput handle_output; + VirtIODevice *vdev; + EventNotifier guest_notifier; + EventNotifier host_notifier; + bool host_notifier_enabled; + QLIST_ENTRY(VirtQueue) node; +}; + void virtio_instance_init_common(Object *proxy_obj, void *data, size_t vdev_size, const char *vdev_name); @@ -226,8 +276,6 @@ void virtio_error(VirtIODevice *vdev, const char *fmt, ...) G_GNUC_PRINTF(2, 3); /* Set the child bus name. */ void virtio_device_set_child_bus_name(VirtIODevice *vdev, char *bus_name); -typedef void (*VirtIOHandleOutput)(VirtIODevice *, VirtQueue *); - VirtQueue *virtio_add_queue(VirtIODevice *vdev, int queue_size, VirtIOHandleOutput handle_output); From patchwork Sat Jan 28 07:17:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 13119711 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A22B3C38142 for ; Sat, 28 Jan 2023 07:23:51 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pLfYC-0003rf-TZ; Sat, 28 Jan 2023 02:23:00 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pLfY9-0003qy-8t for qemu-devel@nongnu.org; Sat, 28 Jan 2023 02:22:57 -0500 Received: from out30-45.freemail.mail.aliyun.com ([115.124.30.45]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pLfY5-0000lk-7K for qemu-devel@nongnu.org; Sat, 28 Jan 2023 02:22:56 -0500 X-Alimail-AntiSpam: AC=PASS; BC=-1|-1; BR=01201311R381e4; CH=green; DM=||false|; DS=||; FP=0|-1|-1|-1|0|-1|-1|-1; HT=ay29a033018046059; MF=xuanzhuo@linux.alibaba.com; NM=1; PH=DS; RN=3; SR=0; TI=SMTPD_---0VaGs2Fz_1674890246; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0VaGs2Fz_1674890246) by smtp.aliyun-inc.com; Sat, 28 Jan 2023 15:17:27 +0800 From: Xuan Zhuo To: qemu-devel@nongnu.org Cc: "Michael S. Tsirkin" , Jason Wang Subject: [PATCH 2/3] virtio: struct VirtQueue introduce reset Date: Sat, 28 Jan 2023 15:17:23 +0800 Message-Id: <20230128071724.33677-3-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.32.0.3.g01195cf9f In-Reply-To: <20230128071724.33677-1-xuanzhuo@linux.alibaba.com> References: <20230128071724.33677-1-xuanzhuo@linux.alibaba.com> MIME-Version: 1.0 X-Git-Hash: 850524d63b Received-SPF: pass client-ip=115.124.30.45; envelope-from=xuanzhuo@linux.alibaba.com; helo=out30-45.freemail.mail.aliyun.com X-Spam_score_int: -98 X-Spam_score: -9.9 X-Spam_bar: --------- X-Spam_report: (-9.9 / 5.0 requ) BAYES_00=-1.9, ENV_AND_HDR_SPF_MATCH=-0.5, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, UNPARSEABLE_RELAY=0.001, USER_IN_DEF_SPF_WL=-7.5 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org In the current design, we stop the device from operating on the vring during per-queue reset by resetting the structure VirtQueue. But before the reset operation, when recycling some resources, we should stop referencing new vring resources. For example, when recycling virtio-net's asynchronous sending resources, virtio-net should be able to perceive that the current queue is in the per-queue reset state, and stop sending new packets from the tx queue. Signed-off-by: Xuan Zhuo --- hw/virtio/virtio.c | 8 ++++++++ include/hw/virtio/virtio.h | 3 +++ 2 files changed, 11 insertions(+) diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c index 03077b2ecf..907d5b8bde 100644 --- a/hw/virtio/virtio.c +++ b/hw/virtio/virtio.c @@ -2030,6 +2030,12 @@ void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index) { VirtioDeviceClass *k = VIRTIO_DEVICE_GET_CLASS(vdev); + /* + * Mark this queue is per-queue reset status. The device should release the + * references of the vring, and not refer more new vring item. + */ + vdev->vq[queue_index].reset = true; + if (k->queue_reset) { k->queue_reset(vdev, queue_index); } @@ -2053,6 +2059,8 @@ void virtio_queue_enable(VirtIODevice *vdev, uint32_t queue_index) } */ + vdev->vq[queue_index].reset = false; + if (k->queue_enable) { k->queue_enable(vdev, queue_index); } diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h index 1c0d77c670..b888538d09 100644 --- a/include/hw/virtio/virtio.h +++ b/include/hw/virtio/virtio.h @@ -251,6 +251,9 @@ struct VirtQueue { /* Notification enabled? */ bool notification; + /* Per-Queue Reset status */ + bool reset; + uint16_t queue_index; unsigned int inuse; From patchwork Sat Jan 28 07:17:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 13119713 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AC435C38142 for ; Sat, 28 Jan 2023 07:24:01 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pLfY8-0003qf-Lt; Sat, 28 Jan 2023 02:22:56 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pLfY7-0003os-2f for qemu-devel@nongnu.org; Sat, 28 Jan 2023 02:22:55 -0500 Received: from out30-118.freemail.mail.aliyun.com ([115.124.30.118]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pLfY3-0000ky-Bv for qemu-devel@nongnu.org; Sat, 28 Jan 2023 02:22:54 -0500 X-Alimail-AntiSpam: AC=PASS; BC=-1|-1; BR=01201311R301e4; CH=green; DM=||false|; DS=||; FP=0|-1|-1|-1|0|-1|-1|-1; HT=ay29a033018046056; MF=xuanzhuo@linux.alibaba.com; NM=1; PH=DS; RN=4; SR=0; TI=SMTPD_---0VaGyJFO_1674890247; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0VaGyJFO_1674890247) by smtp.aliyun-inc.com; Sat, 28 Jan 2023 15:17:28 +0800 From: Xuan Zhuo To: qemu-devel@nongnu.org Cc: "Michael S. Tsirkin" , Jason Wang , Alexander Bulekov Subject: [PATCH 3/3] virtio-net: virtio_net_flush_tx() check for per-queue reset Date: Sat, 28 Jan 2023 15:17:24 +0800 Message-Id: <20230128071724.33677-4-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.32.0.3.g01195cf9f In-Reply-To: <20230128071724.33677-1-xuanzhuo@linux.alibaba.com> References: <20230128071724.33677-1-xuanzhuo@linux.alibaba.com> MIME-Version: 1.0 X-Git-Hash: 850524d63b Received-SPF: pass client-ip=115.124.30.118; envelope-from=xuanzhuo@linux.alibaba.com; helo=out30-118.freemail.mail.aliyun.com X-Spam_score_int: -98 X-Spam_score: -9.9 X-Spam_bar: --------- X-Spam_report: (-9.9 / 5.0 requ) BAYES_00=-1.9, ENV_AND_HDR_SPF_MATCH=-0.5, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, UNPARSEABLE_RELAY=0.001, USER_IN_DEF_SPF_WL=-7.5 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Check whether it is per-queue reset state in virtio_net_flush_tx(). Before per-queue reset, we need to recover async tx resources. At this time, virtio_net_flush_tx() is called, but we should not try to send new packets, so virtio_net_flush_tx() should check the current per-queue reset state. Fixes: 7dc6be52 ("virtio-net: support queue reset") Fixes: https://gitlab.com/qemu-project/qemu/-/issues/1451 Reported-by: Alexander Bulekov Signed-off-by: Xuan Zhuo --- hw/net/virtio-net.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c index 3ae909041a..e7e651e915 100644 --- a/hw/net/virtio-net.c +++ b/hw/net/virtio-net.c @@ -2627,7 +2627,7 @@ static int32_t virtio_net_flush_tx(VirtIONetQueue *q) VirtQueueElement *elem; int32_t num_packets = 0; int queue_index = vq2q(virtio_get_queue_index(q->tx_vq)); - if (!(vdev->status & VIRTIO_CONFIG_S_DRIVER_OK)) { + if (!(vdev->status & VIRTIO_CONFIG_S_DRIVER_OK) || q->tx_vq->reset) { return num_packets; }