From patchwork Fri Feb 23 08:27:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 13568645 Received: from out30-133.freemail.mail.aliyun.com (out30-133.freemail.mail.aliyun.com [115.124.30.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4C34517552; Fri, 23 Feb 2024 08:27:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.133 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708676861; cv=none; b=kHOP7IJUklKyl6IIk2atfpMIfO0SQUEPDoXXzIrblyfjmKru7XZQAdQfqJ4P5xTq4zJN/LQZiCETD6YeIvsS7rFTjgoQaCDwaxzqbsS9iR6SB3SwVYi8CgC4yf4bF/mau9nsr+WR1AO/ePTdHyTK/UqGRjuOVWB1dMwqpxFYS/U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708676861; c=relaxed/simple; bh=J5cJ2czOhIvgpSmEC6fnTwEkg5TGNWL6ZotrgsHxZkI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=lGZEYzGJEmZeQ9nXxF90BhOVMP5Ji2wM/Sbg89Kl64n/wgiCFYwP5s0wObtmcIQhvJlLVIBQq7maz5z1X+UkydSMVkZe3rsz4SXtwLKHkHwF0CZcK7sHOTtSirLEze7xYzyGhvi9eGqEti3bEgy7e+P9Q4ge65bh2HfvHnJmXks= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=CHM+CtVh; arc=none smtp.client-ip=115.124.30.133 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="CHM+CtVh" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1708676850; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=saSIU1bHCwAg5v0mYMoHrXba5As9q/PsyzQZAzeMUCk=; b=CHM+CtVhh5Sbg5mUrk2yYrFD/Pe3aQ6laVA3sBeYGj4QTCTHwzuZJK6CrqE3ebinyAvsjZBQTAN+Kg1FZWRaTtcR2rpz9pI0kO64C86L0YhjhutQvTIHBb5bPWQ0sN4r+DQqoYffJI6a6qlSk1bmtQkj56WlwD2h4rfd+VvTaiA= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R911e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046056;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=35;SR=0;TI=SMTPD_---0W13mUvf_1708676847; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0W13mUvf_1708676847) by smtp.aliyun-inc.com; Fri, 23 Feb 2024 16:27:28 +0800 From: Xuan Zhuo To: virtualization@lists.linux.dev Cc: Richard Weinberger , Anton Ivanov , Johannes Berg , "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Hans de Goede , =?utf-8?q?Ilpo_J=C3=A4rvinen?= , Vadim Pasternak , Bjorn Andersson , Mathieu Poirier , Cornelia Huck , Halil Pasic , Eric Farman , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , linux-um@lists.infradead.org, netdev@vger.kernel.org, platform-driver-x86@vger.kernel.org, linux-remoteproc@vger.kernel.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH vhost v2 01/19] virtio_ring: introduce vring_need_unmap_buffer Date: Fri, 23 Feb 2024 16:27:08 +0800 Message-Id: <20240223082726.52915-2-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.32.0.3.g01195cf9f In-Reply-To: <20240223082726.52915-1-xuanzhuo@linux.alibaba.com> References: <20240223082726.52915-1-xuanzhuo@linux.alibaba.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Git-Hash: 510995f33855 To make the code readable, introduce vring_need_unmap_buffer() to replace do_unmap. use_dma_api premapped -> vring_need_unmap_buffer() 1. false false false 2. true false true 3. true true false Signed-off-by: Xuan Zhuo Acked-by: Jason Wang --- drivers/virtio/virtio_ring.c | 27 ++++++++++++--------------- 1 file changed, 12 insertions(+), 15 deletions(-) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index 6f7e5010a673..21c461e2d851 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -175,11 +175,6 @@ struct vring_virtqueue { /* Do DMA mapping by driver */ bool premapped; - /* Do unmap or not for desc. Just when premapped is False and - * use_dma_api is true, this is true. - */ - bool do_unmap; - /* Head of free buffer list. */ unsigned int free_head; /* Number we've added since last sync. */ @@ -297,6 +292,11 @@ static bool vring_use_dma_api(const struct virtio_device *vdev) return false; } +static bool vring_need_unmap_buffer(const struct vring_virtqueue *vring) +{ + return vring->use_dma_api && !vring->premapped; +} + size_t virtio_max_dma_size(const struct virtio_device *vdev) { size_t max_segment_size = SIZE_MAX; @@ -445,7 +445,7 @@ static void vring_unmap_one_split_indirect(const struct vring_virtqueue *vq, { u16 flags; - if (!vq->do_unmap) + if (!vring_need_unmap_buffer(vq)) return; flags = virtio16_to_cpu(vq->vq.vdev, desc->flags); @@ -475,7 +475,7 @@ static unsigned int vring_unmap_one_split(const struct vring_virtqueue *vq, (flags & VRING_DESC_F_WRITE) ? DMA_FROM_DEVICE : DMA_TO_DEVICE); } else { - if (!vq->do_unmap) + if (!vring_need_unmap_buffer(vq)) goto out; dma_unmap_page(vring_dma_dev(vq), @@ -643,7 +643,7 @@ static inline int virtqueue_add_split(struct virtqueue *_vq, } /* Last one doesn't continue. */ desc[prev].flags &= cpu_to_virtio16(_vq->vdev, ~VRING_DESC_F_NEXT); - if (!indirect && vq->do_unmap) + if (!indirect && vring_need_unmap_buffer(vq)) vq->split.desc_extra[prev & (vq->split.vring.num - 1)].flags &= ~VRING_DESC_F_NEXT; @@ -802,7 +802,7 @@ static void detach_buf_split(struct vring_virtqueue *vq, unsigned int head, VRING_DESC_F_INDIRECT)); BUG_ON(len == 0 || len % sizeof(struct vring_desc)); - if (vq->do_unmap) { + if (vring_need_unmap_buffer(vq)) { for (j = 0; j < len / sizeof(struct vring_desc); j++) vring_unmap_one_split_indirect(vq, &indir_desc[j]); } @@ -1236,7 +1236,7 @@ static void vring_unmap_extra_packed(const struct vring_virtqueue *vq, (flags & VRING_DESC_F_WRITE) ? DMA_FROM_DEVICE : DMA_TO_DEVICE); } else { - if (!vq->do_unmap) + if (!vring_need_unmap_buffer(vq)) return; dma_unmap_page(vring_dma_dev(vq), @@ -1251,7 +1251,7 @@ static void vring_unmap_desc_packed(const struct vring_virtqueue *vq, { u16 flags; - if (!vq->do_unmap) + if (!vring_need_unmap_buffer(vq)) return; flags = le16_to_cpu(desc->flags); @@ -1632,7 +1632,7 @@ static void detach_buf_packed(struct vring_virtqueue *vq, if (!desc) return; - if (vq->do_unmap) { + if (vring_need_unmap_buffer(vq)) { len = vq->packed.desc_extra[id].len; for (i = 0; i < len / sizeof(struct vring_packed_desc); i++) @@ -2091,7 +2091,6 @@ static struct virtqueue *vring_create_virtqueue_packed( vq->dma_dev = dma_dev; vq->use_dma_api = vring_use_dma_api(vdev); vq->premapped = false; - vq->do_unmap = vq->use_dma_api; vq->indirect = virtio_has_feature(vdev, VIRTIO_RING_F_INDIRECT_DESC) && !context; @@ -2636,7 +2635,6 @@ static struct virtqueue *__vring_new_virtqueue(unsigned int index, vq->dma_dev = dma_dev; vq->use_dma_api = vring_use_dma_api(vdev); vq->premapped = false; - vq->do_unmap = vq->use_dma_api; vq->indirect = virtio_has_feature(vdev, VIRTIO_RING_F_INDIRECT_DESC) && !context; @@ -2804,7 +2802,6 @@ int virtqueue_set_dma_premapped(struct virtqueue *_vq) } vq->premapped = true; - vq->do_unmap = false; END_USE(vq); From patchwork Fri Feb 23 08:27:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 13568644 Received: from out30-130.freemail.mail.aliyun.com (out30-130.freemail.mail.aliyun.com [115.124.30.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4EAB918E1A; Fri, 23 Feb 2024 08:27:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708676858; cv=none; b=Zoj2SMisK4XFniU4f8DmoJAYPbNDNM2rkg96d9wLLgVSkQHXDM5RrGpXBXlCmK/2V1NSOeXTQUrzmbfjYX2DVOzfutv9dC9XmejaHtSdaEWpQkdlhIprYSO/4Rd51Vp8nWUbAIWvEpziAp392xWLBL+delljrpVOgWK3s9pPvzU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708676858; c=relaxed/simple; bh=QQmZvjdFSHKOk9YfDFitjAeVmh38HMrb0nZV9PaD1s0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=pFRw6G2tRQ44886dZ79GDlHrGDqS1hQkaNI3uEgKj4A3guwoYGGyvilYO3FX2xH6bDdd0pRndmgG+/t9i6UbisAOBkHL/awNJUHvNPI62H5mpPgaY06orYysFTDcDpwhAZE2yPXrKz/qojxzWNYIgCjM4ntmZVTvGQvUKS3YfSU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=wGF+CYcU; arc=none smtp.client-ip=115.124.30.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="wGF+CYcU" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1708676852; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=5/H5nlAswdup+I0SCxn8A5NdgIxcS2d0JfKC/TbQdkU=; b=wGF+CYcUvACEtg5dQY2j3UQNFn6luzYUJ2mpLp5JfBjE5r5WgWTVMlmWdmfXh3fwFunwH9DwA51FcR0W97DtQWQJIV6PS0ey/DnoGlUNUt+TH8oit+vbpAEi6xpJcDdRvYYJs8UfbSelaJFn4gob+73wCMUFH4VO7D8u4f2bW4Y= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R201e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045176;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=35;SR=0;TI=SMTPD_---0W13nF1h_1708676849; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0W13nF1h_1708676849) by smtp.aliyun-inc.com; Fri, 23 Feb 2024 16:27:30 +0800 From: Xuan Zhuo To: virtualization@lists.linux.dev Cc: Richard Weinberger , Anton Ivanov , Johannes Berg , "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Hans de Goede , =?utf-8?q?Ilpo_J=C3=A4rvinen?= , Vadim Pasternak , Bjorn Andersson , Mathieu Poirier , Cornelia Huck , Halil Pasic , Eric Farman , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , linux-um@lists.infradead.org, netdev@vger.kernel.org, platform-driver-x86@vger.kernel.org, linux-remoteproc@vger.kernel.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH vhost v2 02/19] virtio_ring: packed: remove double check of the unmap ops Date: Fri, 23 Feb 2024 16:27:09 +0800 Message-Id: <20240223082726.52915-3-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.32.0.3.g01195cf9f In-Reply-To: <20240223082726.52915-1-xuanzhuo@linux.alibaba.com> References: <20240223082726.52915-1-xuanzhuo@linux.alibaba.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Git-Hash: 510995f33855 In the functions vring_unmap_extra_packed and vring_unmap_desc_packed, multiple checks are made whether unmap is performed and whether it is INDIRECT. These two functions are usually called in a loop, and we should put the check outside the loop. And we unmap the descs with VRING_DESC_F_INDIRECT on the same path with other descs, that make the thing more complex. If we distinguish the descs with VRING_DESC_F_INDIRECT before unmap, thing will be clearer. 1. only one desc of the desc table is used, we do not need the loop 2. the called unmap api is difference from the other desc 3. the vq->premapped is not needed to check 4. the vq->indirect is not needed to check 5. the state->indir_desc must not be null Signed-off-by: Xuan Zhuo --- drivers/virtio/virtio_ring.c | 78 ++++++++++++++++++------------------ 1 file changed, 40 insertions(+), 38 deletions(-) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index 21c461e2d851..98d27dfdcf16 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -1220,6 +1220,7 @@ static u16 packed_last_used(u16 last_used_idx) return last_used_idx & ~(-(1 << VRING_PACKED_EVENT_F_WRAP_CTR)); } +/* caller must check vring_need_unmap_buffer() */ static void vring_unmap_extra_packed(const struct vring_virtqueue *vq, const struct vring_desc_extra *extra) { @@ -1227,33 +1228,18 @@ static void vring_unmap_extra_packed(const struct vring_virtqueue *vq, flags = extra->flags; - if (flags & VRING_DESC_F_INDIRECT) { - if (!vq->use_dma_api) - return; - - dma_unmap_single(vring_dma_dev(vq), - extra->addr, extra->len, - (flags & VRING_DESC_F_WRITE) ? - DMA_FROM_DEVICE : DMA_TO_DEVICE); - } else { - if (!vring_need_unmap_buffer(vq)) - return; - - dma_unmap_page(vring_dma_dev(vq), - extra->addr, extra->len, - (flags & VRING_DESC_F_WRITE) ? - DMA_FROM_DEVICE : DMA_TO_DEVICE); - } + dma_unmap_page(vring_dma_dev(vq), + extra->addr, extra->len, + (flags & VRING_DESC_F_WRITE) ? + DMA_FROM_DEVICE : DMA_TO_DEVICE); } +/* caller must check vring_need_unmap_buffer() */ static void vring_unmap_desc_packed(const struct vring_virtqueue *vq, const struct vring_packed_desc *desc) { u16 flags; - if (!vring_need_unmap_buffer(vq)) - return; - flags = le16_to_cpu(desc->flags); dma_unmap_page(vring_dma_dev(vq), @@ -1329,7 +1315,7 @@ static int virtqueue_add_indirect_packed(struct vring_virtqueue *vq, total_sg * sizeof(struct vring_packed_desc), DMA_TO_DEVICE); if (vring_mapping_error(vq, addr)) { - if (vq->premapped) + if (!vring_need_unmap_buffer(vq)) goto free_desc; goto unmap_release; @@ -1344,10 +1330,11 @@ static int virtqueue_add_indirect_packed(struct vring_virtqueue *vq, vq->packed.desc_extra[id].addr = addr; vq->packed.desc_extra[id].len = total_sg * sizeof(struct vring_packed_desc); - vq->packed.desc_extra[id].flags = VRING_DESC_F_INDIRECT | - vq->packed.avail_used_flags; } + vq->packed.desc_extra[id].flags = VRING_DESC_F_INDIRECT | + vq->packed.avail_used_flags; + /* * A driver MUST NOT make the first descriptor in the list * available before all subsequent descriptors comprising @@ -1388,6 +1375,8 @@ static int virtqueue_add_indirect_packed(struct vring_virtqueue *vq, unmap_release: err_idx = i; + WARN_ON(!vring_need_unmap_buffer(vq)); + for (i = 0; i < err_idx; i++) vring_unmap_desc_packed(vq, &desc[i]); @@ -1481,12 +1470,13 @@ static inline int virtqueue_add_packed(struct virtqueue *_vq, desc[i].len = cpu_to_le32(sg->length); desc[i].id = cpu_to_le16(id); - if (unlikely(vq->use_dma_api)) { + if (vring_need_unmap_buffer(vq)) { vq->packed.desc_extra[curr].addr = addr; vq->packed.desc_extra[curr].len = sg->length; - vq->packed.desc_extra[curr].flags = - le16_to_cpu(flags); } + + vq->packed.desc_extra[curr].flags = le16_to_cpu(flags); + prev = curr; curr = vq->packed.desc_extra[curr].next; @@ -1536,6 +1526,8 @@ static inline int virtqueue_add_packed(struct virtqueue *_vq, vq->packed.avail_used_flags = avail_used_flags; + WARN_ON(!vring_need_unmap_buffer(vq)); + for (n = 0; n < total_sg; n++) { if (i == err_idx) break; @@ -1605,7 +1597,9 @@ static void detach_buf_packed(struct vring_virtqueue *vq, struct vring_desc_state_packed *state = NULL; struct vring_packed_desc *desc; unsigned int i, curr; + u16 flags; + flags = vq->packed.desc_extra[id].flags; state = &vq->packed.desc_state[id]; /* Clear data ptr. */ @@ -1615,22 +1609,32 @@ static void detach_buf_packed(struct vring_virtqueue *vq, vq->free_head = id; vq->vq.num_free += state->num; - if (unlikely(vq->use_dma_api)) { - curr = id; - for (i = 0; i < state->num; i++) { - vring_unmap_extra_packed(vq, - &vq->packed.desc_extra[curr]); - curr = vq->packed.desc_extra[curr].next; + if (!(flags & VRING_DESC_F_INDIRECT)) { + if (vring_need_unmap_buffer(vq)) { + curr = id; + for (i = 0; i < state->num; i++) { + vring_unmap_extra_packed(vq, + &vq->packed.desc_extra[curr]); + curr = vq->packed.desc_extra[curr].next; + } } - } - if (vq->indirect) { + if (ctx) + *ctx = state->indir_desc; + } else { + const struct vring_desc_extra *extra; u32 len; + if (vq->use_dma_api) { + extra = &vq->packed.desc_extra[id]; + dma_unmap_single(vring_dma_dev(vq), + extra->addr, extra->len, + (flags & VRING_DESC_F_WRITE) ? + DMA_FROM_DEVICE : DMA_TO_DEVICE); + } + /* Free the indirect table, if any, now that it's unmapped. */ desc = state->indir_desc; - if (!desc) - return; if (vring_need_unmap_buffer(vq)) { len = vq->packed.desc_extra[id].len; @@ -1640,8 +1644,6 @@ static void detach_buf_packed(struct vring_virtqueue *vq, } kfree(desc); state->indir_desc = NULL; - } else if (ctx) { - *ctx = state->indir_desc; } } From patchwork Fri Feb 23 08:27:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 13568649 Received: from out30-133.freemail.mail.aliyun.com (out30-133.freemail.mail.aliyun.com [115.124.30.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D94E42208A; Fri, 23 Feb 2024 08:27:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.133 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708676866; cv=none; b=qSTrGDPAMvw16+EOEV4Oe4ymS563UKhi4SUS3LIUo0x9skhiAIPlpRBW9ZpWR08d1A2T/0AjG60dxMjIIq2JLkXcEBqKqOK5OBeSf7HhQsFgF5p4rw/E2b8iwIX1UUbx/wwDpWSnUObWplWdFxCETdyONk8sDqhr+6wwuKvP4/I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708676866; c=relaxed/simple; bh=evlgRHodcZps3pWynlLYjgKtTZVMPuPLf69EUhOF3pU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=I+3Uw3IsLYePsw5DSNKj5R8j80V2g2L6NEWqedaDbj49GYL6OvFt3E+2knbwxWFeGwMQhIzK/C29CFdLlm/of+c9LaDEI3mtLCYPQLIwwucEfbxsarvbaZeIzCbsMv0g6yZzSyudWqUqMeuzc6eQ1yJzGhkkyWhoD2+3hI7v7nI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=EhEe0RLk; arc=none smtp.client-ip=115.124.30.133 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="EhEe0RLk" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1708676855; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=MEVtA8vb6lj50l3aO2/I3wlv6226v1gV9tFg60hG/lc=; b=EhEe0RLkOYKkNd2KX9b8DkYxmlIEohT5aGdxz38ppyaE3Qsjq6/7/7nCTXNgzlIG/wf8Pwuz/LxuAuUdtGQPtDXKxh86qZCPbR76aL9rjK6kFAm113Mb7MO+a5qqSygCXqI+zLwC/SXrtbdWwBj2MR3m9gZYw2lpB4tXLp4jBXU= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R611e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046056;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=35;SR=0;TI=SMTPD_---0W13gV2p_1708676851; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0W13gV2p_1708676851) by smtp.aliyun-inc.com; Fri, 23 Feb 2024 16:27:32 +0800 From: Xuan Zhuo To: virtualization@lists.linux.dev Cc: Richard Weinberger , Anton Ivanov , Johannes Berg , "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Hans de Goede , =?utf-8?q?Ilpo_J=C3=A4rvinen?= , Vadim Pasternak , Bjorn Andersson , Mathieu Poirier , Cornelia Huck , Halil Pasic , Eric Farman , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , linux-um@lists.infradead.org, netdev@vger.kernel.org, platform-driver-x86@vger.kernel.org, linux-remoteproc@vger.kernel.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH vhost v2 03/19] virtio_ring: packed: structure the indirect desc table Date: Fri, 23 Feb 2024 16:27:10 +0800 Message-Id: <20240223082726.52915-4-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.32.0.3.g01195cf9f In-Reply-To: <20240223082726.52915-1-xuanzhuo@linux.alibaba.com> References: <20240223082726.52915-1-xuanzhuo@linux.alibaba.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Git-Hash: 510995f33855 This commit structure the indirect desc table. Then we can get the desc num directly when doing unmap. And save the dma info to the struct, then the indirect will not use the dma fields of the desc_extra. The subsequent commits will make the dma fields are optional. But for the indirect case, we must record the dma info. Signed-off-by: Xuan Zhuo --- drivers/virtio/virtio_ring.c | 66 +++++++++++++++++++++--------------- 1 file changed, 38 insertions(+), 28 deletions(-) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index 98d27dfdcf16..2d4d1417a1d6 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -72,9 +72,16 @@ struct vring_desc_state_split { struct vring_desc *indir_desc; /* Indirect descriptor, if any. */ }; +struct vring_packed_desc_indir { + dma_addr_t addr; /* Descriptor Array DMA addr. */ + u32 len; /* Descriptor Array length. */ + u32 num; + struct vring_packed_desc desc[]; +}; + struct vring_desc_state_packed { void *data; /* Data for callback. */ - struct vring_packed_desc *indir_desc; /* Indirect descriptor, if any. */ + struct vring_packed_desc_indir *indir_desc; /* Indirect descriptor, if any. */ u16 num; /* Descriptor list length. */ u16 last; /* The last desc state in a list. */ }; @@ -1249,10 +1256,13 @@ static void vring_unmap_desc_packed(const struct vring_virtqueue *vq, DMA_FROM_DEVICE : DMA_TO_DEVICE); } -static struct vring_packed_desc *alloc_indirect_packed(unsigned int total_sg, +static struct vring_packed_desc_indir *alloc_indirect_packed(unsigned int total_sg, gfp_t gfp) { - struct vring_packed_desc *desc; + struct vring_packed_desc_indir *in_desc; + u32 size; + + size = struct_size(in_desc, desc, total_sg); /* * We require lowmem mappings for the descriptors because @@ -1261,9 +1271,10 @@ static struct vring_packed_desc *alloc_indirect_packed(unsigned int total_sg, */ gfp &= ~__GFP_HIGHMEM; - desc = kmalloc_array(total_sg, sizeof(struct vring_packed_desc), gfp); - return desc; + in_desc = kmalloc(size, gfp); + + return in_desc; } static int virtqueue_add_indirect_packed(struct vring_virtqueue *vq, @@ -1274,6 +1285,7 @@ static int virtqueue_add_indirect_packed(struct vring_virtqueue *vq, void *data, gfp_t gfp) { + struct vring_packed_desc_indir *in_desc; struct vring_packed_desc *desc; struct scatterlist *sg; unsigned int i, n, err_idx; @@ -1281,10 +1293,12 @@ static int virtqueue_add_indirect_packed(struct vring_virtqueue *vq, dma_addr_t addr; head = vq->packed.next_avail_idx; - desc = alloc_indirect_packed(total_sg, gfp); - if (!desc) + in_desc = alloc_indirect_packed(total_sg, gfp); + if (!in_desc) return -ENOMEM; + desc = in_desc->desc; + if (unlikely(vq->vq.num_free < 1)) { pr_debug("Can't add buf len 1 - avail = 0\n"); kfree(desc); @@ -1321,17 +1335,19 @@ static int virtqueue_add_indirect_packed(struct vring_virtqueue *vq, goto unmap_release; } + if (vring_need_unmap_buffer(vq)) + in_desc->num = i; + + if (vq->use_dma_api) { + in_desc->addr = addr; + in_desc->len = total_sg * sizeof(struct vring_packed_desc); + } + vq->packed.vring.desc[head].addr = cpu_to_le64(addr); vq->packed.vring.desc[head].len = cpu_to_le32(total_sg * sizeof(struct vring_packed_desc)); vq->packed.vring.desc[head].id = cpu_to_le16(id); - if (vq->use_dma_api) { - vq->packed.desc_extra[id].addr = addr; - vq->packed.desc_extra[id].len = total_sg * - sizeof(struct vring_packed_desc); - } - vq->packed.desc_extra[id].flags = VRING_DESC_F_INDIRECT | vq->packed.avail_used_flags; @@ -1362,7 +1378,7 @@ static int virtqueue_add_indirect_packed(struct vring_virtqueue *vq, /* Store token and indirect buffer state. */ vq->packed.desc_state[id].num = 1; vq->packed.desc_state[id].data = data; - vq->packed.desc_state[id].indir_desc = desc; + vq->packed.desc_state[id].indir_desc = in_desc; vq->packed.desc_state[id].last = id; vq->num_added += 1; @@ -1381,7 +1397,7 @@ static int virtqueue_add_indirect_packed(struct vring_virtqueue *vq, vring_unmap_desc_packed(vq, &desc[i]); free_desc: - kfree(desc); + kfree(in_desc); END_USE(vq); return -ENOMEM; @@ -1595,7 +1611,6 @@ static void detach_buf_packed(struct vring_virtqueue *vq, unsigned int id, void **ctx) { struct vring_desc_state_packed *state = NULL; - struct vring_packed_desc *desc; unsigned int i, curr; u16 flags; @@ -1622,27 +1637,22 @@ static void detach_buf_packed(struct vring_virtqueue *vq, if (ctx) *ctx = state->indir_desc; } else { - const struct vring_desc_extra *extra; - u32 len; + struct vring_packed_desc_indir *in_desc; + + in_desc = state->indir_desc; if (vq->use_dma_api) { - extra = &vq->packed.desc_extra[id]; dma_unmap_single(vring_dma_dev(vq), - extra->addr, extra->len, + in_desc->addr, in_desc->len, (flags & VRING_DESC_F_WRITE) ? DMA_FROM_DEVICE : DMA_TO_DEVICE); } - /* Free the indirect table, if any, now that it's unmapped. */ - desc = state->indir_desc; - if (vring_need_unmap_buffer(vq)) { - len = vq->packed.desc_extra[id].len; - for (i = 0; i < len / sizeof(struct vring_packed_desc); - i++) - vring_unmap_desc_packed(vq, &desc[i]); + for (i = 0; i < in_desc->num; i++) + vring_unmap_desc_packed(vq, &in_desc->desc[i]); } - kfree(desc); + kfree(in_desc); state->indir_desc = NULL; } } From patchwork Fri Feb 23 08:27:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 13568648 Received: from out30-100.freemail.mail.aliyun.com (out30-100.freemail.mail.aliyun.com [115.124.30.100]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 16BC622611; Fri, 23 Feb 2024 08:27:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.100 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708676866; cv=none; b=H5NC7J9DxJFvmix8EjJZRIfW3rbNLwDjX2Oeck7S8ZZTUL8jq56SnOMvtsrug2sK3Cb/moJiZSfNi2thvgtfx4KPdEAWqRkoTSHE7axc8LUNbfoTc8wN9NgGqvT9XGdpTaSWikcnfLmWXrHNLgsT/SLG98Z9RKQCKbt4CH7uMQ8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708676866; c=relaxed/simple; bh=TRrAKbfwzzkdL2jPh1gz0R+f8MO1U3YSLR9p9bHBPmE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=UHtzECvTizgHXursLX82a/b+i/eMiWvLESlAiwoPwJ5AHVYEuyIE7JqDZNe+GnqkdZGERiAfCwG/2DN+mwI6QQAlmVGxX+oYptLgVIZBUiX8e5i8Etk8QbwLjGPCM6te25gBheozH69o1hNGh+zN0LA8bGPEZq63jSIZw1txUBg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=piHRdFnk; arc=none smtp.client-ip=115.124.30.100 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="piHRdFnk" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1708676855; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=6IMo6cIkJbw27Q1oQFwPBSLp5dDwzhNSd1jF51DsYTA=; b=piHRdFnkq7sVQT1Z1PxWrVnWc//BKftKy9ikgMZa7AlwHMQIv1dh1uw3jXOykrcSuLa3F+/DM8jJUCH2bOlxfPumTymxf0GuX44kKOgbOzg/hRaGLshen6fEwbFTBaRD0nc/R1I2aOj69ftuPUHOm8tvY/Xrl1VUJG4yFnLuP+A= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R121e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045176;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=35;SR=0;TI=SMTPD_---0W13nF3Y_1708676853; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0W13nF3Y_1708676853) by smtp.aliyun-inc.com; Fri, 23 Feb 2024 16:27:33 +0800 From: Xuan Zhuo To: virtualization@lists.linux.dev Cc: Richard Weinberger , Anton Ivanov , Johannes Berg , "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Hans de Goede , =?utf-8?q?Ilpo_J=C3=A4rvinen?= , Vadim Pasternak , Bjorn Andersson , Mathieu Poirier , Cornelia Huck , Halil Pasic , Eric Farman , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , linux-um@lists.infradead.org, netdev@vger.kernel.org, platform-driver-x86@vger.kernel.org, linux-remoteproc@vger.kernel.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH vhost v2 04/19] virtio_ring: split: remove double check of the unmap ops Date: Fri, 23 Feb 2024 16:27:11 +0800 Message-Id: <20240223082726.52915-5-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.32.0.3.g01195cf9f In-Reply-To: <20240223082726.52915-1-xuanzhuo@linux.alibaba.com> References: <20240223082726.52915-1-xuanzhuo@linux.alibaba.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Git-Hash: 510995f33855 In the functions vring_unmap_one_split and vring_unmap_one_split_indirect, multiple checks are made whether unmap is performed and whether it is INDIRECT. These two functions are usually called in a loop, and we should put the check outside the loop. And we unmap the descs with VRING_DESC_F_INDIRECT on the same path with other descs, that make the thing more complex. If we distinguish the descs with VRING_DESC_F_INDIRECT before unmap, thing will be clearer. 1. only one desc of the desc table is used, we do not need the loop 2. the called unmap api is difference from the other desc 3. the vq->premapped is not needed to check 4. the vq->indirect is not needed to check 5. the state->indir_desc must not be null Signed-off-by: Xuan Zhuo --- drivers/virtio/virtio_ring.c | 79 +++++++++++++++++------------------- 1 file changed, 38 insertions(+), 41 deletions(-) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index 2d4d1417a1d6..194bd877ab5a 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -452,9 +452,6 @@ static void vring_unmap_one_split_indirect(const struct vring_virtqueue *vq, { u16 flags; - if (!vring_need_unmap_buffer(vq)) - return; - flags = virtio16_to_cpu(vq->vq.vdev, desc->flags); dma_unmap_page(vring_dma_dev(vq), @@ -472,27 +469,12 @@ static unsigned int vring_unmap_one_split(const struct vring_virtqueue *vq, flags = extra[i].flags; - if (flags & VRING_DESC_F_INDIRECT) { - if (!vq->use_dma_api) - goto out; - - dma_unmap_single(vring_dma_dev(vq), - extra[i].addr, - extra[i].len, - (flags & VRING_DESC_F_WRITE) ? - DMA_FROM_DEVICE : DMA_TO_DEVICE); - } else { - if (!vring_need_unmap_buffer(vq)) - goto out; - - dma_unmap_page(vring_dma_dev(vq), - extra[i].addr, - extra[i].len, - (flags & VRING_DESC_F_WRITE) ? - DMA_FROM_DEVICE : DMA_TO_DEVICE); - } + dma_unmap_page(vring_dma_dev(vq), + extra[i].addr, + extra[i].len, + (flags & VRING_DESC_F_WRITE) ? + DMA_FROM_DEVICE : DMA_TO_DEVICE); -out: return extra[i].next; } @@ -660,7 +642,7 @@ static inline int virtqueue_add_split(struct virtqueue *_vq, vq, desc, total_sg * sizeof(struct vring_desc), DMA_TO_DEVICE); if (vring_mapping_error(vq, addr)) { - if (vq->premapped) + if (!vring_need_unmap_buffer(vq)) goto free_indirect; goto unmap_release; @@ -713,6 +695,9 @@ static inline int virtqueue_add_split(struct virtqueue *_vq, return 0; unmap_release: + + WARN_ON(!vring_need_unmap_buffer(vq)); + err_idx = i; if (indirect) @@ -774,34 +759,42 @@ static void detach_buf_split(struct vring_virtqueue *vq, unsigned int head, { unsigned int i, j; __virtio16 nextflag = cpu_to_virtio16(vq->vq.vdev, VRING_DESC_F_NEXT); + u16 flags; /* Clear data ptr. */ vq->split.desc_state[head].data = NULL; + flags = vq->split.desc_extra[head].flags; /* Put back on free list: unmap first-level descriptors and find end */ i = head; - while (vq->split.vring.desc[i].flags & nextflag) { - vring_unmap_one_split(vq, i); - i = vq->split.desc_extra[i].next; - vq->vq.num_free++; - } - - vring_unmap_one_split(vq, i); - vq->split.desc_extra[i].next = vq->free_head; - vq->free_head = head; + if (!(flags & VRING_DESC_F_INDIRECT)) { + while (vq->split.vring.desc[i].flags & nextflag) { + if (vring_need_unmap_buffer(vq)) + vring_unmap_one_split(vq, i); + i = vq->split.desc_extra[i].next; + vq->vq.num_free++; + } - /* Plus final descriptor */ - vq->vq.num_free++; + if (vring_need_unmap_buffer(vq)) + vring_unmap_one_split(vq, i); - if (vq->indirect) { + if (ctx) + *ctx = vq->split.desc_state[head].indir_desc; + } else { struct vring_desc *indir_desc = vq->split.desc_state[head].indir_desc; u32 len; - /* Free the indirect table, if any, now that it's unmapped. */ - if (!indir_desc) - return; + if (vq->use_dma_api) { + struct vring_desc_extra *extra = vq->split.desc_extra; + + dma_unmap_single(vring_dma_dev(vq), + extra[i].addr, + extra[i].len, + (flags & VRING_DESC_F_WRITE) ? + DMA_FROM_DEVICE : DMA_TO_DEVICE); + } len = vq->split.desc_extra[head].len; @@ -816,9 +809,13 @@ static void detach_buf_split(struct vring_virtqueue *vq, unsigned int head, kfree(indir_desc); vq->split.desc_state[head].indir_desc = NULL; - } else if (ctx) { - *ctx = vq->split.desc_state[head].indir_desc; } + + vq->split.desc_extra[i].next = vq->free_head; + vq->free_head = head; + + /* Plus final descriptor */ + vq->vq.num_free++; } static bool more_used_split(const struct vring_virtqueue *vq) From patchwork Fri Feb 23 08:27:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 13568646 Received: from out30-101.freemail.mail.aliyun.com (out30-101.freemail.mail.aliyun.com [115.124.30.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9A75B1AADE; Fri, 23 Feb 2024 08:27:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.101 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708676863; cv=none; b=IX5KQ6zAvZG4Tj445trz9bygUAitaeEE9Kd3gZFluzkXOZxg5UnSe3Fsu3sQPPJGfdqvnlQbZXwqv+FGLhmUxmVHz52d+O2Sz+ZBm0mrzNaRBmEUlftWAGRnhVJ/9vV+UXZxZwZi+6F2mYRJUo45Yzv0rFpxjS77vUmGg+SdHQE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708676863; c=relaxed/simple; bh=5ypWt7yf7hDToKKXL1Vw4VBylzx3H8eAF2+EmY2vrkA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=rDssudYxjqEi8t/aneP1K0Ttw5IUDhlQowweUYNgAIvmtzLvcnMhE2nFYHJnJlICmufxiAlFzv2p27WAY6aBiW2UZoz2AHe0EzqRyujeW8My65fNuGCKGmQcKkqXdDkjWRfHDfzW61OizYdCkiZm52425K4DadlTo+87HKbQd1Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=T8qriEKm; arc=none smtp.client-ip=115.124.30.101 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="T8qriEKm" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1708676857; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=rZEA3HZfZtKV/P1RBQjdCvHf1tsCYt8NUzyvMGlh/fw=; b=T8qriEKm1PfJ1/SBeJmQGMrUVQAPTrwW/4s3NCWjLHObBHrtl0+tiiH8dIf2CkUsEu19cxs4oSm5dNWvqUm4rfva746fEqb5bxNb3ldNmt5wVoGZkNaM3oprZOTHJzpO6lAPEp87BfisKRZ8uYVXeelGD064KTgg4tlYOiVEg/I= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R381e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045192;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=35;SR=0;TI=SMTPD_---0W13nF4n_1708676854; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0W13nF4n_1708676854) by smtp.aliyun-inc.com; Fri, 23 Feb 2024 16:27:35 +0800 From: Xuan Zhuo To: virtualization@lists.linux.dev Cc: Richard Weinberger , Anton Ivanov , Johannes Berg , "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Hans de Goede , =?utf-8?q?Ilpo_J=C3=A4rvinen?= , Vadim Pasternak , Bjorn Andersson , Mathieu Poirier , Cornelia Huck , Halil Pasic , Eric Farman , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , linux-um@lists.infradead.org, netdev@vger.kernel.org, platform-driver-x86@vger.kernel.org, linux-remoteproc@vger.kernel.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH vhost v2 05/19] virtio_ring: split: structure the indirect desc table Date: Fri, 23 Feb 2024 16:27:12 +0800 Message-Id: <20240223082726.52915-6-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.32.0.3.g01195cf9f In-Reply-To: <20240223082726.52915-1-xuanzhuo@linux.alibaba.com> References: <20240223082726.52915-1-xuanzhuo@linux.alibaba.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Git-Hash: 510995f33855 This commit structure the indirect desc table. Then we can get the desc num directly when doing unmap. And save the dma info to the struct, then the indirect will not use the dma fields of the desc_extra. The subsequent commits will make the dma fields are optional. But for the indirect case, we must record the dma info. Signed-off-by: Xuan Zhuo --- drivers/virtio/virtio_ring.c | 91 ++++++++++++++++++++++-------------- 1 file changed, 55 insertions(+), 36 deletions(-) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index 194bd877ab5a..9680b3827d5b 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -67,9 +67,16 @@ #define LAST_ADD_TIME_INVALID(vq) #endif +struct vring_split_desc_indir { + dma_addr_t addr; /* Descriptor Array DMA addr. */ + u32 len; /* Descriptor Array length. */ + u32 num; + struct vring_desc desc[]; +}; + struct vring_desc_state_split { void *data; /* Data for callback. */ - struct vring_desc *indir_desc; /* Indirect descriptor, if any. */ + struct vring_split_desc_indir *indir_desc; /* Indirect descriptor, if any. */ }; struct vring_packed_desc_indir { @@ -478,12 +485,16 @@ static unsigned int vring_unmap_one_split(const struct vring_virtqueue *vq, return extra[i].next; } -static struct vring_desc *alloc_indirect_split(struct virtqueue *_vq, - unsigned int total_sg, - gfp_t gfp) +static struct vring_split_desc_indir *alloc_indirect_split(struct virtqueue *_vq, + unsigned int total_sg, + gfp_t gfp) { + struct vring_split_desc_indir *in_desc; struct vring_desc *desc; unsigned int i; + u32 size; + + size = struct_size(in_desc, desc, total_sg); /* * We require lowmem mappings for the descriptors because @@ -492,13 +503,16 @@ static struct vring_desc *alloc_indirect_split(struct virtqueue *_vq, */ gfp &= ~__GFP_HIGHMEM; - desc = kmalloc_array(total_sg, sizeof(struct vring_desc), gfp); - if (!desc) + in_desc = kmalloc(size, gfp); + if (!in_desc) return NULL; + desc = in_desc->desc; + for (i = 0; i < total_sg; i++) desc[i].next = cpu_to_virtio16(_vq->vdev, i + 1); - return desc; + + return in_desc; } static inline unsigned int virtqueue_add_desc_split(struct virtqueue *vq, @@ -540,6 +554,7 @@ static inline int virtqueue_add_split(struct virtqueue *_vq, gfp_t gfp) { struct vring_virtqueue *vq = to_vvq(_vq); + struct vring_split_desc_indir *in_desc; struct scatterlist *sg; struct vring_desc *desc; unsigned int i, n, avail, descs_used, prev, err_idx; @@ -562,9 +577,13 @@ static inline int virtqueue_add_split(struct virtqueue *_vq, head = vq->free_head; - if (virtqueue_use_indirect(vq, total_sg)) - desc = alloc_indirect_split(_vq, total_sg, gfp); - else { + if (virtqueue_use_indirect(vq, total_sg)) { + in_desc = alloc_indirect_split(_vq, total_sg, gfp); + if (!in_desc) + desc = NULL; + else + desc = in_desc->desc; + } else { desc = NULL; WARN_ON_ONCE(total_sg > vq->split.vring.num && !vq->indirect); } @@ -637,10 +656,10 @@ static inline int virtqueue_add_split(struct virtqueue *_vq, ~VRING_DESC_F_NEXT; if (indirect) { + u32 size = total_sg * sizeof(struct vring_desc); + /* Now that the indirect table is filled in, map it. */ - dma_addr_t addr = vring_map_single( - vq, desc, total_sg * sizeof(struct vring_desc), - DMA_TO_DEVICE); + dma_addr_t addr = vring_map_single(vq, desc, size, DMA_TO_DEVICE); if (vring_mapping_error(vq, addr)) { if (!vring_need_unmap_buffer(vq)) goto free_indirect; @@ -648,11 +667,20 @@ static inline int virtqueue_add_split(struct virtqueue *_vq, goto unmap_release; } - virtqueue_add_desc_split(_vq, vq->split.vring.desc, - head, addr, - total_sg * sizeof(struct vring_desc), - VRING_DESC_F_INDIRECT, - false); + desc = &vq->split.vring.desc[head]; + + desc->flags = cpu_to_virtio16(_vq->vdev, VRING_DESC_F_INDIRECT); + desc->addr = cpu_to_virtio64(_vq->vdev, addr); + desc->len = cpu_to_virtio32(_vq->vdev, size); + + vq->split.desc_extra[head].flags = VRING_DESC_F_INDIRECT; + + if (vq->use_dma_api) { + in_desc->addr = addr; + in_desc->len = size; + } + + in_desc->num = total_sg; } /* We're using some buffers from the free list. */ @@ -667,7 +695,7 @@ static inline int virtqueue_add_split(struct virtqueue *_vq, /* Store token and indirect buffer state. */ vq->split.desc_state[head].data = data; if (indirect) - vq->split.desc_state[head].indir_desc = desc; + vq->split.desc_state[head].indir_desc = in_desc; else vq->split.desc_state[head].indir_desc = ctx; @@ -717,7 +745,7 @@ static inline int virtqueue_add_split(struct virtqueue *_vq, free_indirect: if (indirect) - kfree(desc); + kfree(in_desc); END_USE(vq); return -ENOMEM; @@ -782,32 +810,23 @@ static void detach_buf_split(struct vring_virtqueue *vq, unsigned int head, if (ctx) *ctx = vq->split.desc_state[head].indir_desc; } else { - struct vring_desc *indir_desc = - vq->split.desc_state[head].indir_desc; - u32 len; + struct vring_split_desc_indir *in_desc; - if (vq->use_dma_api) { - struct vring_desc_extra *extra = vq->split.desc_extra; + in_desc = vq->split.desc_state[head].indir_desc; + if (vq->use_dma_api) { dma_unmap_single(vring_dma_dev(vq), - extra[i].addr, - extra[i].len, + in_desc->addr, in_desc->len, (flags & VRING_DESC_F_WRITE) ? DMA_FROM_DEVICE : DMA_TO_DEVICE); } - len = vq->split.desc_extra[head].len; - - BUG_ON(!(vq->split.desc_extra[head].flags & - VRING_DESC_F_INDIRECT)); - BUG_ON(len == 0 || len % sizeof(struct vring_desc)); - if (vring_need_unmap_buffer(vq)) { - for (j = 0; j < len / sizeof(struct vring_desc); j++) - vring_unmap_one_split_indirect(vq, &indir_desc[j]); + for (j = 0; j < in_desc->num; j++) + vring_unmap_one_split_indirect(vq, &in_desc->desc[j]); } - kfree(indir_desc); + kfree(in_desc); vq->split.desc_state[head].indir_desc = NULL; } From patchwork Fri Feb 23 08:27:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 13568647 Received: from out30-130.freemail.mail.aliyun.com (out30-130.freemail.mail.aliyun.com [115.124.30.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2F7AE1B5B2; Fri, 23 Feb 2024 08:27:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708676864; cv=none; b=GlRkgEdl4zsIw/M02vUUtkiDOhtWIsvwMKtM9a+bbLdqY74Qq5DTN8x60jYaCQBVUNo6eoXxmSkl9NQ9j+4rHmKgPuXozFPMrICpNw3DdtAp68QD5XsXrJI/1597l4WZsX81TjNiaSgmR+Zai+BX0aYB+wE6OnHQoD6DRrRPBfk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708676864; c=relaxed/simple; bh=CniNnIZ9GNdg9mZl/L/CRuDJhJzQveSh/1T8nV5GtDY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=EEn/eE2iUvQ5K1V3kMHvqI9yZ4nIF87RZzGH5doJcrEfIXgc4Rb8WpqrZStercGC+XhFa+5653NpcgazWGUNI7q/F9DRauEYGZMFfzUQMIqGZvyIpG5p+BaBjdUCsoAv5C7oww/QNgleBx9g8sE/pIe1rE4FCh5P1WEj8lDnffM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=DP3pP5EJ; arc=none smtp.client-ip=115.124.30.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="DP3pP5EJ" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1708676859; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=sjGZ6V6Wg7dragljIJeNGw4jGyz/uv3T+cDPwN7GEJ4=; b=DP3pP5EJEnyFKzQpRj0fBpAykhu4UdhlPTipJJbWNKoj3n1yRExVISMyw76mwHq/qgQ9B433wWnl6AuDaTFokRjp9wsd1KoNvcwq/tLNqCRKtKgo2tsp8InTbl29srn/Yp/J/Q4Gum7WINyrOXYpkDgW5Ofzu0E/kCsmZYFamss= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R911e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046059;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=35;SR=0;TI=SMTPD_---0W13oNpL_1708676856; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0W13oNpL_1708676856) by smtp.aliyun-inc.com; Fri, 23 Feb 2024 16:27:37 +0800 From: Xuan Zhuo To: virtualization@lists.linux.dev Cc: Richard Weinberger , Anton Ivanov , Johannes Berg , "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Hans de Goede , =?utf-8?q?Ilpo_J=C3=A4rvinen?= , Vadim Pasternak , Bjorn Andersson , Mathieu Poirier , Cornelia Huck , Halil Pasic , Eric Farman , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , linux-um@lists.infradead.org, netdev@vger.kernel.org, platform-driver-x86@vger.kernel.org, linux-remoteproc@vger.kernel.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH vhost v2 06/19] virtio_ring: no store dma info when unmap is not needed Date: Fri, 23 Feb 2024 16:27:13 +0800 Message-Id: <20240223082726.52915-7-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.32.0.3.g01195cf9f In-Reply-To: <20240223082726.52915-1-xuanzhuo@linux.alibaba.com> References: <20240223082726.52915-1-xuanzhuo@linux.alibaba.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Git-Hash: 510995f33855 As discussed: http://lore.kernel.org/all/CACGkMEug-=C+VQhkMYSgUKMC==04m7-uem_yC21bgGkKZh845w@mail.gmail.com When the vq is premapped mode, the driver manages the dma info is a good way. So this commit make the virtio core not to store the dma info and release the memory which is used to store the dma info. If the use_dma_api is false, the memory is also not allocated. Signed-off-by: Xuan Zhuo --- drivers/virtio/virtio_ring.c | 89 ++++++++++++++++++++++++++++-------- 1 file changed, 70 insertions(+), 19 deletions(-) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index 9680b3827d5b..c1e4edaeb446 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -94,12 +94,15 @@ struct vring_desc_state_packed { }; struct vring_desc_extra { - dma_addr_t addr; /* Descriptor DMA addr. */ - u32 len; /* Descriptor length. */ u16 flags; /* Descriptor flags. */ u16 next; /* The next desc state in a list. */ }; +struct vring_desc_dma { + dma_addr_t addr; /* Descriptor DMA addr. */ + u32 len; /* Descriptor length. */ +}; + struct vring_virtqueue_split { /* Actual memory layout for this queue. */ struct vring vring; @@ -116,6 +119,7 @@ struct vring_virtqueue_split { /* Per-descriptor state. */ struct vring_desc_state_split *desc_state; struct vring_desc_extra *desc_extra; + struct vring_desc_dma *desc_dma; /* DMA address and size information */ dma_addr_t queue_dma_addr; @@ -156,6 +160,7 @@ struct vring_virtqueue_packed { /* Per-descriptor state. */ struct vring_desc_state_packed *desc_state; struct vring_desc_extra *desc_extra; + struct vring_desc_dma *desc_dma; /* DMA address and size information */ dma_addr_t ring_dma_addr; @@ -472,13 +477,14 @@ static unsigned int vring_unmap_one_split(const struct vring_virtqueue *vq, unsigned int i) { struct vring_desc_extra *extra = vq->split.desc_extra; + struct vring_desc_dma *dma = vq->split.desc_dma; u16 flags; flags = extra[i].flags; dma_unmap_page(vring_dma_dev(vq), - extra[i].addr, - extra[i].len, + dma[i].addr, + dma[i].len, (flags & VRING_DESC_F_WRITE) ? DMA_FROM_DEVICE : DMA_TO_DEVICE); @@ -535,8 +541,11 @@ static inline unsigned int virtqueue_add_desc_split(struct virtqueue *vq, next = extra[i].next; desc[i].next = cpu_to_virtio16(vq->vdev, next); - extra[i].addr = addr; - extra[i].len = len; + if (vring->split.desc_dma) { + vring->split.desc_dma[i].addr = addr; + vring->split.desc_dma[i].len = len; + } + extra[i].flags = flags; } else next = virtio16_to_cpu(vq->vdev, desc[i].next); @@ -1072,16 +1081,26 @@ static void virtqueue_vring_attach_split(struct vring_virtqueue *vq, vq->free_head = 0; } -static int vring_alloc_state_extra_split(struct vring_virtqueue_split *vring_split) +static int vring_alloc_state_extra_split(struct vring_virtqueue_split *vring_split, + bool need_unmap) { struct vring_desc_state_split *state; struct vring_desc_extra *extra; + struct vring_desc_dma *dma; u32 num = vring_split->vring.num; state = kmalloc_array(num, sizeof(struct vring_desc_state_split), GFP_KERNEL); if (!state) goto err_state; + if (need_unmap) { + dma = kmalloc_array(num, sizeof(struct vring_desc_dma), GFP_KERNEL); + if (!dma) + goto err_dma; + } else { + dma = NULL; + } + extra = vring_alloc_desc_extra(num); if (!extra) goto err_extra; @@ -1090,9 +1109,12 @@ static int vring_alloc_state_extra_split(struct vring_virtqueue_split *vring_spl vring_split->desc_state = state; vring_split->desc_extra = extra; + vring_split->desc_dma = dma; return 0; err_extra: + kfree(dma); +err_dma: kfree(state); err_state: return -ENOMEM; @@ -1108,6 +1130,7 @@ static void vring_free_split(struct vring_virtqueue_split *vring_split, kfree(vring_split->desc_state); kfree(vring_split->desc_extra); + kfree(vring_split->desc_dma); } static int vring_alloc_queue_split(struct vring_virtqueue_split *vring_split, @@ -1209,7 +1232,8 @@ static int virtqueue_resize_split(struct virtqueue *_vq, u32 num) if (err) goto err; - err = vring_alloc_state_extra_split(&vring_split); + err = vring_alloc_state_extra_split(&vring_split, + vring_need_unmap_buffer(vq)); if (err) goto err_state_extra; @@ -1245,14 +1269,16 @@ static u16 packed_last_used(u16 last_used_idx) /* caller must check vring_need_unmap_buffer() */ static void vring_unmap_extra_packed(const struct vring_virtqueue *vq, - const struct vring_desc_extra *extra) + unsigned int i) { + const struct vring_desc_extra *extra = &vq->packed.desc_extra[i]; + const struct vring_desc_dma *dma = &vq->packed.desc_dma[i]; u16 flags; flags = extra->flags; dma_unmap_page(vring_dma_dev(vq), - extra->addr, extra->len, + dma->addr, dma->len, (flags & VRING_DESC_F_WRITE) ? DMA_FROM_DEVICE : DMA_TO_DEVICE); } @@ -1503,8 +1529,8 @@ static inline int virtqueue_add_packed(struct virtqueue *_vq, desc[i].id = cpu_to_le16(id); if (vring_need_unmap_buffer(vq)) { - vq->packed.desc_extra[curr].addr = addr; - vq->packed.desc_extra[curr].len = sg->length; + vq->packed.desc_dma[curr].addr = addr; + vq->packed.desc_dma[curr].len = sg->length; } vq->packed.desc_extra[curr].flags = le16_to_cpu(flags); @@ -1563,7 +1589,7 @@ static inline int virtqueue_add_packed(struct virtqueue *_vq, for (n = 0; n < total_sg; n++) { if (i == err_idx) break; - vring_unmap_extra_packed(vq, &vq->packed.desc_extra[curr]); + vring_unmap_extra_packed(vq, curr); curr = vq->packed.desc_extra[curr].next; i++; if (i >= vq->packed.vring.num) @@ -1644,8 +1670,7 @@ static void detach_buf_packed(struct vring_virtqueue *vq, if (vring_need_unmap_buffer(vq)) { curr = id; for (i = 0; i < state->num; i++) { - vring_unmap_extra_packed(vq, - &vq->packed.desc_extra[curr]); + vring_unmap_extra_packed(vq, curr); curr = vq->packed.desc_extra[curr].next; } } @@ -1958,6 +1983,7 @@ static void vring_free_packed(struct vring_virtqueue_packed *vring_packed, kfree(vring_packed->desc_state); kfree(vring_packed->desc_extra); + kfree(vring_packed->desc_dma); } static int vring_alloc_queue_packed(struct vring_virtqueue_packed *vring_packed, @@ -2014,10 +2040,12 @@ static int vring_alloc_queue_packed(struct vring_virtqueue_packed *vring_packed, return -ENOMEM; } -static int vring_alloc_state_extra_packed(struct vring_virtqueue_packed *vring_packed) +static int vring_alloc_state_extra_packed(struct vring_virtqueue_packed *vring_packed, + bool need_unmap) { struct vring_desc_state_packed *state; struct vring_desc_extra *extra; + struct vring_desc_dma *dma; u32 num = vring_packed->vring.num; state = kmalloc_array(num, sizeof(struct vring_desc_state_packed), GFP_KERNEL); @@ -2026,6 +2054,14 @@ static int vring_alloc_state_extra_packed(struct vring_virtqueue_packed *vring_p memset(state, 0, num * sizeof(struct vring_desc_state_packed)); + if (need_unmap) { + dma = kmalloc_array(num, sizeof(struct vring_desc_dma), GFP_KERNEL); + if (!dma) + goto err_desc_dma; + } else { + dma = NULL; + } + extra = vring_alloc_desc_extra(num); if (!extra) goto err_desc_extra; @@ -2036,6 +2072,8 @@ static int vring_alloc_state_extra_packed(struct vring_virtqueue_packed *vring_p return 0; err_desc_extra: + kfree(dma); +err_desc_dma: kfree(state); err_desc_state: return -ENOMEM; @@ -2127,7 +2165,8 @@ static struct virtqueue *vring_create_virtqueue_packed( if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM)) vq->weak_barriers = false; - err = vring_alloc_state_extra_packed(&vring_packed); + err = vring_alloc_state_extra_packed(&vring_packed, + vring_need_unmap_buffer(vq)); if (err) goto err_state_extra; @@ -2159,7 +2198,8 @@ static int virtqueue_resize_packed(struct virtqueue *_vq, u32 num) if (vring_alloc_queue_packed(&vring_packed, vdev, num, vring_dma_dev(vq))) goto err_ring; - err = vring_alloc_state_extra_packed(&vring_packed); + err = vring_alloc_state_extra_packed(&vring_packed, + vring_need_unmap_buffer(vq)); if (err) goto err_state_extra; @@ -2671,7 +2711,8 @@ static struct virtqueue *__vring_new_virtqueue(unsigned int index, if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM)) vq->weak_barriers = false; - err = vring_alloc_state_extra_split(vring_split); + err = vring_alloc_state_extra_split(vring_split, + vring_need_unmap_buffer(vq)); if (err) { kfree(vq); return NULL; @@ -2831,6 +2872,14 @@ int virtqueue_set_dma_premapped(struct virtqueue *_vq) vq->premapped = true; + if (vq->packed_ring) { + kfree(vq->packed.desc_dma); + vq->packed.desc_dma = NULL; + } else { + kfree(vq->split.desc_dma); + vq->split.desc_dma = NULL; + } + END_USE(vq); return 0; @@ -2920,6 +2969,7 @@ static void vring_free(struct virtqueue *_vq) kfree(vq->packed.desc_state); kfree(vq->packed.desc_extra); + kfree(vq->packed.desc_dma); } else { vring_free_queue(vq->vq.vdev, vq->split.queue_size_in_bytes, @@ -2931,6 +2981,7 @@ static void vring_free(struct virtqueue *_vq) if (!vq->packed_ring) { kfree(vq->split.desc_state); kfree(vq->split.desc_extra); + kfree(vq->split.desc_dma); } } From patchwork Fri Feb 23 08:27:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 13568650 Received: from out30-101.freemail.mail.aliyun.com (out30-101.freemail.mail.aliyun.com [115.124.30.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 24F3722615; Fri, 23 Feb 2024 08:27:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.101 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708676867; cv=none; b=kQ6DIkHsMLobENbGN3+Mj7yyc+CBr5UZzgekavowCA+FoQ/DbXvyaYFjd93MpSWxjPwKBjQ2wSsQsyfbgD+9MciUWQ9noO6UNmbIeaaxQ96Inak8cyMcYq1PLMAHJaidacUO7x7mc4AJqFzm2WUb6P098zMx9cJHbOMKd50iEkQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708676867; c=relaxed/simple; bh=rmq/BFm+uYfi71GX12CYAN4bX1wzMJLiHU6YSooyBlY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=ar5cFWzm+EMKuQu2fMAjG5F11KAn9ZJpP8eUJTykTgabgp1MJm3f0MiidxTbxCT56KzGVFXBLihCz4sqWV4cKUfYXySwEEwcmD1ubxTyqAIwG5Fnglkl3IKK8ghi2WDj+4pNPj7KTTlxi+lJ+3cGI4sf7ZMl+7C7LbdhbCZGakU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=ht98l3Q5; arc=none smtp.client-ip=115.124.30.101 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="ht98l3Q5" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1708676861; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=jPUO+lFSqFBuyvv0ggp34hvW/3uJMmZ02kYac33xX+E=; b=ht98l3Q5I2yp2FjQ+p20wk0E2bxmIKNmskqKJrOnvCXCFNRvKIEN7JSlAo9pdSlYhHe4z0s8IHKxRLdixamjPKDlltEvCD+i9qyVC2ufJr74umxac9/0uIlMVHY/wHwKMOH7Nt7XL+llWDhgv6uNTEieLcXiR3/reV9oXOynfQo= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R611e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046059;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=35;SR=0;TI=SMTPD_---0W13mV-Y_1708676857; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0W13mV-Y_1708676857) by smtp.aliyun-inc.com; Fri, 23 Feb 2024 16:27:38 +0800 From: Xuan Zhuo To: virtualization@lists.linux.dev Cc: Richard Weinberger , Anton Ivanov , Johannes Berg , "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Hans de Goede , =?utf-8?q?Ilpo_J=C3=A4rvinen?= , Vadim Pasternak , Bjorn Andersson , Mathieu Poirier , Cornelia Huck , Halil Pasic , Eric Farman , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , linux-um@lists.infradead.org, netdev@vger.kernel.org, platform-driver-x86@vger.kernel.org, linux-remoteproc@vger.kernel.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH vhost v2 07/19] virtio: find_vqs: pass struct instead of multi parameters Date: Fri, 23 Feb 2024 16:27:14 +0800 Message-Id: <20240223082726.52915-8-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.32.0.3.g01195cf9f In-Reply-To: <20240223082726.52915-1-xuanzhuo@linux.alibaba.com> References: <20240223082726.52915-1-xuanzhuo@linux.alibaba.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Git-Hash: 510995f33855 Now, we pass multi parameters to find_vqs. These parameters may work for transport or work for vring. And find_vqs has multi implements in many places: But every time, arch/um/drivers/virtio_uml.c drivers/platform/mellanox/mlxbf-tmfifo.c drivers/remoteproc/remoteproc_virtio.c drivers/s390/virtio/virtio_ccw.c drivers/virtio/virtio_mmio.c drivers/virtio/virtio_pci_legacy.c drivers/virtio/virtio_pci_modern.c drivers/virtio/virtio_vdpa.c Every time, we try to add a new parameter, that is difficult. We must change every find_vqs implement. One the other side, if we want to pass a parameter to vring, we must change the call path from transport to vring. Too many functions need to be changed. So it is time to refactor the find_vqs. We pass a structure cfg to find_vqs(), that will be passed to vring by transport. Signed-off-by: Xuan Zhuo --- arch/um/drivers/virtio_uml.c | 23 ++++----- drivers/platform/mellanox/mlxbf-tmfifo.c | 13 ++---- drivers/remoteproc/remoteproc_virtio.c | 28 ++++++----- drivers/s390/virtio/virtio_ccw.c | 29 ++++++------ drivers/virtio/virtio_mmio.c | 26 +++++------ drivers/virtio/virtio_pci_common.c | 59 +++++++++++------------- drivers/virtio/virtio_pci_common.h | 9 +--- drivers/virtio/virtio_pci_legacy.c | 11 +++-- drivers/virtio/virtio_pci_modern.c | 19 ++++---- drivers/virtio/virtio_vdpa.c | 36 +++++++-------- include/linux/virtio_config.h | 51 ++++++++++++++++---- 11 files changed, 160 insertions(+), 144 deletions(-) diff --git a/arch/um/drivers/virtio_uml.c b/arch/um/drivers/virtio_uml.c index 8adca2000e51..c13dfeeb90c4 100644 --- a/arch/um/drivers/virtio_uml.c +++ b/arch/um/drivers/virtio_uml.c @@ -937,8 +937,8 @@ static int vu_setup_vq_call_fd(struct virtio_uml_device *vu_dev, } static struct virtqueue *vu_setup_vq(struct virtio_device *vdev, - unsigned index, vq_callback_t *callback, - const char *name, bool ctx) + unsigned index, + struct virtio_vq_config *cfg) { struct virtio_uml_device *vu_dev = to_virtio_uml_device(vdev); struct platform_device *pdev = vu_dev->pdev; @@ -953,10 +953,12 @@ static struct virtqueue *vu_setup_vq(struct virtio_device *vdev, goto error_kzalloc; } snprintf(info->name, sizeof(info->name), "%s.%d-%s", pdev->name, - pdev->id, name); + pdev->id, cfg->names[cfg->cfg_idx]); vq = vring_create_virtqueue(index, num, PAGE_SIZE, vdev, true, true, - ctx, vu_notify, callback, info->name); + cfg->ctx ? cfg->ctx[cfg->cfg_idx] : false, + vu_notify, + cfg->callbacks[cfg->cfg_idx], info->name); if (!vq) { rc = -ENOMEM; goto error_create; @@ -1013,12 +1015,11 @@ static struct virtqueue *vu_setup_vq(struct virtio_device *vdev, return ERR_PTR(rc); } -static int vu_find_vqs(struct virtio_device *vdev, unsigned nvqs, - struct virtqueue *vqs[], vq_callback_t *callbacks[], - const char * const names[], const bool *ctx, - struct irq_affinity *desc) +static int vu_find_vqs(struct virtio_device *vdev, struct virtio_vq_config *cfg) { struct virtio_uml_device *vu_dev = to_virtio_uml_device(vdev); + struct virtqueue **vqs = cfg->vqs; + unsigned int nvqs = cfg->nvqs; int i, queue_idx = 0, rc; struct virtqueue *vq; @@ -1031,13 +1032,13 @@ static int vu_find_vqs(struct virtio_device *vdev, unsigned nvqs, return rc; for (i = 0; i < nvqs; ++i) { - if (!names[i]) { + if (!cfg->names[i]) { vqs[i] = NULL; continue; } - vqs[i] = vu_setup_vq(vdev, queue_idx++, callbacks[i], names[i], - ctx ? ctx[i] : false); + cfg->cfg_idx = i; + vqs[i] = vu_setup_vq(vdev, queue_idx++, cfg); if (IS_ERR(vqs[i])) { rc = PTR_ERR(vqs[i]); goto error_setup; diff --git a/drivers/platform/mellanox/mlxbf-tmfifo.c b/drivers/platform/mellanox/mlxbf-tmfifo.c index b8d1e32e97eb..4252388f52a2 100644 --- a/drivers/platform/mellanox/mlxbf-tmfifo.c +++ b/drivers/platform/mellanox/mlxbf-tmfifo.c @@ -1056,15 +1056,12 @@ static void mlxbf_tmfifo_virtio_del_vqs(struct virtio_device *vdev) /* Create and initialize the virtual queues. */ static int mlxbf_tmfifo_virtio_find_vqs(struct virtio_device *vdev, - unsigned int nvqs, - struct virtqueue *vqs[], - vq_callback_t *callbacks[], - const char * const names[], - const bool *ctx, - struct irq_affinity *desc) + struct virtio_vq_config *cfg) { struct mlxbf_tmfifo_vdev *tm_vdev = mlxbf_vdev_to_tmfifo(vdev); + struct virtqueue **vqs = cfg->vqs; struct mlxbf_tmfifo_vring *vring; + unsigned int nvqs = cfg->nvqs; struct virtqueue *vq; int i, ret, size; @@ -1072,7 +1069,7 @@ static int mlxbf_tmfifo_virtio_find_vqs(struct virtio_device *vdev, return -EINVAL; for (i = 0; i < nvqs; ++i) { - if (!names[i]) { + if (!cfg->names[i]) { ret = -EINVAL; goto error; } @@ -1084,7 +1081,7 @@ static int mlxbf_tmfifo_virtio_find_vqs(struct virtio_device *vdev, vq = vring_new_virtqueue(i, vring->num, vring->align, vdev, false, false, vring->va, mlxbf_tmfifo_virtio_notify, - callbacks[i], names[i]); + cfg->callbacks[i], cfg->names[i]); if (!vq) { dev_err(&vdev->dev, "vring_new_virtqueue failed\n"); ret = -ENOMEM; diff --git a/drivers/remoteproc/remoteproc_virtio.c b/drivers/remoteproc/remoteproc_virtio.c index 83d76915a6ad..57d51c9c7b63 100644 --- a/drivers/remoteproc/remoteproc_virtio.c +++ b/drivers/remoteproc/remoteproc_virtio.c @@ -102,8 +102,7 @@ EXPORT_SYMBOL(rproc_vq_interrupt); static struct virtqueue *rp_find_vq(struct virtio_device *vdev, unsigned int id, - void (*callback)(struct virtqueue *vq), - const char *name, bool ctx) + struct virtio_vq_config *cfg) { struct rproc_vdev *rvdev = vdev_to_rvdev(vdev); struct rproc *rproc = vdev_to_rproc(vdev); @@ -119,7 +118,7 @@ static struct virtqueue *rp_find_vq(struct virtio_device *vdev, if (id >= ARRAY_SIZE(rvdev->vring)) return ERR_PTR(-EINVAL); - if (!name) + if (!cfg->names[cfg->cfg_idx]) return NULL; /* Search allocated memory region by name */ @@ -143,10 +142,12 @@ static struct virtqueue *rp_find_vq(struct virtio_device *vdev, * Create the new vq, and tell virtio we're not interested in * the 'weak' smp barriers, since we're talking with a real device. */ - vq = vring_new_virtqueue(id, num, rvring->align, vdev, false, ctx, - addr, rproc_virtio_notify, callback, name); + vq = vring_new_virtqueue(id, num, rvring->align, vdev, false, + cfg->ctx ? cfg->ctx[cfg->cfg_idx] : false, + addr, rproc_virtio_notify, cfg->callbacks[cfg->cfg_idx], + cfg->names[cfg->cfg_idx]); if (!vq) { - dev_err(dev, "vring_new_virtqueue %s failed\n", name); + dev_err(dev, "vring_new_virtqueue %s failed\n", cfg->names[cfg->cfg_idx]); rproc_free_vring(rvring); return ERR_PTR(-ENOMEM); } @@ -180,23 +181,20 @@ static void rproc_virtio_del_vqs(struct virtio_device *vdev) __rproc_virtio_del_vqs(vdev); } -static int rproc_virtio_find_vqs(struct virtio_device *vdev, unsigned int nvqs, - struct virtqueue *vqs[], - vq_callback_t *callbacks[], - const char * const names[], - const bool * ctx, - struct irq_affinity *desc) +static int rproc_virtio_find_vqs(struct virtio_device *vdev, struct virtio_vq_config *cfg) { + struct virtqueue **vqs = cfg->vqs; + unsigned int nvqs = cfg->nvqs; int i, ret, queue_idx = 0; for (i = 0; i < nvqs; ++i) { - if (!names[i]) { + if (!cfg->names[i]) { vqs[i] = NULL; continue; } - vqs[i] = rp_find_vq(vdev, queue_idx++, callbacks[i], names[i], - ctx ? ctx[i] : false); + cfg->cfg_idx = i; + vqs[i] = rp_find_vq(vdev, queue_idx++, cfg); if (IS_ERR(vqs[i])) { ret = PTR_ERR(vqs[i]); goto error; diff --git a/drivers/s390/virtio/virtio_ccw.c b/drivers/s390/virtio/virtio_ccw.c index ac67576301bf..11eea5086cff 100644 --- a/drivers/s390/virtio/virtio_ccw.c +++ b/drivers/s390/virtio/virtio_ccw.c @@ -499,9 +499,8 @@ static void virtio_ccw_del_vqs(struct virtio_device *vdev) } static struct virtqueue *virtio_ccw_setup_vq(struct virtio_device *vdev, - int i, vq_callback_t *callback, - const char *name, bool ctx, - struct ccw1 *ccw) + int i, struct ccw1 *ccw, + struct virtio_vq_config *cfg) { struct virtio_ccw_device *vcdev = to_vc_device(vdev); bool (*notify)(struct virtqueue *vq); @@ -538,8 +537,11 @@ static struct virtqueue *virtio_ccw_setup_vq(struct virtio_device *vdev, } may_reduce = vcdev->revision > 0; vq = vring_create_virtqueue(i, info->num, KVM_VIRTIO_CCW_RING_ALIGN, - vdev, true, may_reduce, ctx, - notify, callback, name); + vdev, true, may_reduce, + cfg->ctx ? cfg->ctx[cfg->cfg_idx] : false, + notify, + cfg->callbacks[cfg->cfg_idx], + cfg->names[cfg->cfg_idx]); if (!vq) { /* For now, we fail if we can't get the requested size. */ @@ -650,15 +652,13 @@ static int virtio_ccw_register_adapter_ind(struct virtio_ccw_device *vcdev, return ret; } -static int virtio_ccw_find_vqs(struct virtio_device *vdev, unsigned nvqs, - struct virtqueue *vqs[], - vq_callback_t *callbacks[], - const char * const names[], - const bool *ctx, - struct irq_affinity *desc) +static int virtio_ccw_find_vqs(struct virtio_device *vdev, + struct virtio_vq_config *cfg) { struct virtio_ccw_device *vcdev = to_vc_device(vdev); + struct virtqueue **vqs = cfg->vqs; unsigned long *indicatorp = NULL; + unsigned int nvqs = cfg->nvqs; int ret, i, queue_idx = 0; struct ccw1 *ccw; @@ -667,14 +667,13 @@ static int virtio_ccw_find_vqs(struct virtio_device *vdev, unsigned nvqs, return -ENOMEM; for (i = 0; i < nvqs; ++i) { - if (!names[i]) { + if (!cfg->names[i]) { vqs[i] = NULL; continue; } - vqs[i] = virtio_ccw_setup_vq(vdev, queue_idx++, callbacks[i], - names[i], ctx ? ctx[i] : false, - ccw); + cfg->cfg_idx = i; + vqs[i] = virtio_ccw_setup_vq(vdev, queue_idx++, ccw, cfg); if (IS_ERR(vqs[i])) { ret = PTR_ERR(vqs[i]); vqs[i] = NULL; diff --git a/drivers/virtio/virtio_mmio.c b/drivers/virtio/virtio_mmio.c index 59892a31cf76..feb823d279d2 100644 --- a/drivers/virtio/virtio_mmio.c +++ b/drivers/virtio/virtio_mmio.c @@ -370,8 +370,7 @@ static void vm_synchronize_cbs(struct virtio_device *vdev) } static struct virtqueue *vm_setup_vq(struct virtio_device *vdev, unsigned int index, - void (*callback)(struct virtqueue *vq), - const char *name, bool ctx) + struct virtio_vq_config *cfg) { struct virtio_mmio_device *vm_dev = to_virtio_mmio_device(vdev); bool (*notify)(struct virtqueue *vq); @@ -386,7 +385,7 @@ static struct virtqueue *vm_setup_vq(struct virtio_device *vdev, unsigned int in else notify = vm_notify; - if (!name) + if (!cfg->names[index]) return NULL; /* Select the queue we're interested in */ @@ -414,7 +413,11 @@ static struct virtqueue *vm_setup_vq(struct virtio_device *vdev, unsigned int in /* Create the vring */ vq = vring_create_virtqueue(index, num, VIRTIO_MMIO_VRING_ALIGN, vdev, - true, true, ctx, notify, callback, name); + true, true, + cfg->ctx ? cfg->ctx[cfg->cfg_idx] : false, + notify, + cfg->callbacks[cfg->cfg_idx], + cfg->names[cfg->cfg_idx]); if (!vq) { err = -ENOMEM; goto error_new_virtqueue; @@ -487,15 +490,12 @@ static struct virtqueue *vm_setup_vq(struct virtio_device *vdev, unsigned int in return ERR_PTR(err); } -static int vm_find_vqs(struct virtio_device *vdev, unsigned int nvqs, - struct virtqueue *vqs[], - vq_callback_t *callbacks[], - const char * const names[], - const bool *ctx, - struct irq_affinity *desc) +static int vm_find_vqs(struct virtio_device *vdev, struct virtio_vq_config *cfg) { struct virtio_mmio_device *vm_dev = to_virtio_mmio_device(vdev); int irq = platform_get_irq(vm_dev->pdev, 0); + struct virtqueue **vqs = cfg->vqs; + unsigned int nvqs = cfg->nvqs; int i, err, queue_idx = 0; if (irq < 0) @@ -510,13 +510,13 @@ static int vm_find_vqs(struct virtio_device *vdev, unsigned int nvqs, enable_irq_wake(irq); for (i = 0; i < nvqs; ++i) { - if (!names[i]) { + if (!cfg->names[i]) { vqs[i] = NULL; continue; } - vqs[i] = vm_setup_vq(vdev, queue_idx++, callbacks[i], names[i], - ctx ? ctx[i] : false); + cfg->cfg_idx = i; + vqs[i] = vm_setup_vq(vdev, queue_idx++, cfg); if (IS_ERR(vqs[i])) { vm_del_vqs(vdev); return PTR_ERR(vqs[i]); diff --git a/drivers/virtio/virtio_pci_common.c b/drivers/virtio/virtio_pci_common.c index b655fccaf773..a9ae03904dcf 100644 --- a/drivers/virtio/virtio_pci_common.c +++ b/drivers/virtio/virtio_pci_common.c @@ -172,9 +172,7 @@ static int vp_request_msix_vectors(struct virtio_device *vdev, int nvectors, } static struct virtqueue *vp_setup_vq(struct virtio_device *vdev, unsigned int index, - void (*callback)(struct virtqueue *vq), - const char *name, - bool ctx, + struct virtio_vq_config *cfg, u16 msix_vec) { struct virtio_pci_device *vp_dev = to_vp_device(vdev); @@ -186,13 +184,13 @@ static struct virtqueue *vp_setup_vq(struct virtio_device *vdev, unsigned int in if (!info) return ERR_PTR(-ENOMEM); - vq = vp_dev->setup_vq(vp_dev, info, index, callback, name, ctx, + vq = vp_dev->setup_vq(vp_dev, info, index, cfg, msix_vec); if (IS_ERR(vq)) goto out_info; info->vq = vq; - if (callback) { + if (cfg->callbacks[cfg->cfg_idx]) { spin_lock_irqsave(&vp_dev->lock, flags); list_add(&info->node, &vp_dev->virtqueues); spin_unlock_irqrestore(&vp_dev->lock, flags); @@ -284,15 +282,15 @@ void vp_del_vqs(struct virtio_device *vdev) vp_dev->vqs = NULL; } -static int vp_find_vqs_msix(struct virtio_device *vdev, unsigned int nvqs, - struct virtqueue *vqs[], vq_callback_t *callbacks[], - const char * const names[], bool per_vq_vectors, - const bool *ctx, - struct irq_affinity *desc) +static int vp_find_vqs_msix(struct virtio_device *vdev, + struct virtio_vq_config *cfg, + bool per_vq_vectors) { struct virtio_pci_device *vp_dev = to_vp_device(vdev); u16 msix_vec; int i, err, nvectors, allocated_vectors, queue_idx = 0; + struct virtqueue **vqs = cfg->vqs; + unsigned int nvqs = cfg->nvqs; vp_dev->vqs = kcalloc(nvqs, sizeof(*vp_dev->vqs), GFP_KERNEL); if (!vp_dev->vqs) @@ -302,7 +300,7 @@ static int vp_find_vqs_msix(struct virtio_device *vdev, unsigned int nvqs, /* Best option: one for change interrupt, one per vq. */ nvectors = 1; for (i = 0; i < nvqs; ++i) - if (names[i] && callbacks[i]) + if (cfg->names[i] && cfg->callbacks[i]) ++nvectors; } else { /* Second best: one for change, shared for all vqs. */ @@ -310,27 +308,27 @@ static int vp_find_vqs_msix(struct virtio_device *vdev, unsigned int nvqs, } err = vp_request_msix_vectors(vdev, nvectors, per_vq_vectors, - per_vq_vectors ? desc : NULL); + per_vq_vectors ? cfg->desc : NULL); if (err) goto error_find; vp_dev->per_vq_vectors = per_vq_vectors; allocated_vectors = vp_dev->msix_used_vectors; for (i = 0; i < nvqs; ++i) { - if (!names[i]) { + if (!cfg->names[i]) { vqs[i] = NULL; continue; } - if (!callbacks[i]) + if (!cfg->callbacks[i]) msix_vec = VIRTIO_MSI_NO_VECTOR; else if (vp_dev->per_vq_vectors) msix_vec = allocated_vectors++; else msix_vec = VP_MSIX_VQ_VECTOR; - vqs[i] = vp_setup_vq(vdev, queue_idx++, callbacks[i], names[i], - ctx ? ctx[i] : false, - msix_vec); + + cfg->cfg_idx = i; + vqs[i] = vp_setup_vq(vdev, queue_idx++, cfg, msix_vec); if (IS_ERR(vqs[i])) { err = PTR_ERR(vqs[i]); goto error_find; @@ -343,7 +341,7 @@ static int vp_find_vqs_msix(struct virtio_device *vdev, unsigned int nvqs, snprintf(vp_dev->msix_names[msix_vec], sizeof *vp_dev->msix_names, "%s-%s", - dev_name(&vp_dev->vdev.dev), names[i]); + dev_name(&vp_dev->vdev.dev), cfg->names[i]); err = request_irq(pci_irq_vector(vp_dev->pci_dev, msix_vec), vring_interrupt, 0, vp_dev->msix_names[msix_vec], @@ -358,11 +356,11 @@ static int vp_find_vqs_msix(struct virtio_device *vdev, unsigned int nvqs, return err; } -static int vp_find_vqs_intx(struct virtio_device *vdev, unsigned int nvqs, - struct virtqueue *vqs[], vq_callback_t *callbacks[], - const char * const names[], const bool *ctx) +static int vp_find_vqs_intx(struct virtio_device *vdev, struct virtio_vq_config *cfg) { struct virtio_pci_device *vp_dev = to_vp_device(vdev); + struct virtqueue **vqs = cfg->vqs; + unsigned int nvqs = cfg->nvqs; int i, err, queue_idx = 0; vp_dev->vqs = kcalloc(nvqs, sizeof(*vp_dev->vqs), GFP_KERNEL); @@ -377,13 +375,13 @@ static int vp_find_vqs_intx(struct virtio_device *vdev, unsigned int nvqs, vp_dev->intx_enabled = 1; vp_dev->per_vq_vectors = false; for (i = 0; i < nvqs; ++i) { - if (!names[i]) { + if (!cfg->names[i]) { vqs[i] = NULL; continue; } - vqs[i] = vp_setup_vq(vdev, queue_idx++, callbacks[i], names[i], - ctx ? ctx[i] : false, - VIRTIO_MSI_NO_VECTOR); + + cfg->cfg_idx = i; + vqs[i] = vp_setup_vq(vdev, queue_idx++, cfg, VIRTIO_MSI_NO_VECTOR); if (IS_ERR(vqs[i])) { err = PTR_ERR(vqs[i]); goto out_del_vqs; @@ -397,26 +395,23 @@ static int vp_find_vqs_intx(struct virtio_device *vdev, unsigned int nvqs, } /* the config->find_vqs() implementation */ -int vp_find_vqs(struct virtio_device *vdev, unsigned int nvqs, - struct virtqueue *vqs[], vq_callback_t *callbacks[], - const char * const names[], const bool *ctx, - struct irq_affinity *desc) +int vp_find_vqs(struct virtio_device *vdev, struct virtio_vq_config *cfg) { int err; /* Try MSI-X with one vector per queue. */ - err = vp_find_vqs_msix(vdev, nvqs, vqs, callbacks, names, true, ctx, desc); + err = vp_find_vqs_msix(vdev, cfg, true); if (!err) return 0; /* Fallback: MSI-X with one vector for config, one shared for queues. */ - err = vp_find_vqs_msix(vdev, nvqs, vqs, callbacks, names, false, ctx, desc); + err = vp_find_vqs_msix(vdev, cfg, false); if (!err) return 0; /* Is there an interrupt? If not give up. */ if (!(to_vp_device(vdev)->pci_dev->irq)) return err; /* Finally fall back to regular interrupts. */ - return vp_find_vqs_intx(vdev, nvqs, vqs, callbacks, names, ctx); + return vp_find_vqs_intx(vdev, cfg); } const char *vp_bus_name(struct virtio_device *vdev) diff --git a/drivers/virtio/virtio_pci_common.h b/drivers/virtio/virtio_pci_common.h index 7fef52bee455..5ba8b82fb765 100644 --- a/drivers/virtio/virtio_pci_common.h +++ b/drivers/virtio/virtio_pci_common.h @@ -95,9 +95,7 @@ struct virtio_pci_device { struct virtqueue *(*setup_vq)(struct virtio_pci_device *vp_dev, struct virtio_pci_vq_info *info, unsigned int idx, - void (*callback)(struct virtqueue *vq), - const char *name, - bool ctx, + struct virtio_vq_config *vq_cfg, u16 msix_vec); void (*del_vq)(struct virtio_pci_vq_info *info); @@ -126,10 +124,7 @@ bool vp_notify(struct virtqueue *vq); /* the config->del_vqs() implementation */ void vp_del_vqs(struct virtio_device *vdev); /* the config->find_vqs() implementation */ -int vp_find_vqs(struct virtio_device *vdev, unsigned int nvqs, - struct virtqueue *vqs[], vq_callback_t *callbacks[], - const char * const names[], const bool *ctx, - struct irq_affinity *desc); +int vp_find_vqs(struct virtio_device *vdev, struct virtio_vq_config *cfg); const char *vp_bus_name(struct virtio_device *vdev); /* Setup the affinity for a virtqueue: diff --git a/drivers/virtio/virtio_pci_legacy.c b/drivers/virtio/virtio_pci_legacy.c index d9cbb02b35a1..e8d22fce32f5 100644 --- a/drivers/virtio/virtio_pci_legacy.c +++ b/drivers/virtio/virtio_pci_legacy.c @@ -110,9 +110,7 @@ static u16 vp_config_vector(struct virtio_pci_device *vp_dev, u16 vector) static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev, struct virtio_pci_vq_info *info, unsigned int index, - void (*callback)(struct virtqueue *vq), - const char *name, - bool ctx, + struct virtio_vq_config *cfg, u16 msix_vec) { struct virtqueue *vq; @@ -130,8 +128,11 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev, /* create the vring */ vq = vring_create_virtqueue(index, num, VIRTIO_PCI_VRING_ALIGN, &vp_dev->vdev, - true, false, ctx, - vp_notify, callback, name); + true, false, + cfg->ctx ? cfg->ctx[cfg->cfg_idx] : false, + vp_notify, + cfg->callbacks[cfg->cfg_idx], + cfg->names[cfg->cfg_idx]); if (!vq) return ERR_PTR(-ENOMEM); diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c index f62b530aa3b5..04c7782392a3 100644 --- a/drivers/virtio/virtio_pci_modern.c +++ b/drivers/virtio/virtio_pci_modern.c @@ -530,9 +530,7 @@ static bool vp_notify_with_data(struct virtqueue *vq) static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev, struct virtio_pci_vq_info *info, unsigned int index, - void (*callback)(struct virtqueue *vq), - const char *name, - bool ctx, + struct virtio_vq_config *cfg, u16 msix_vec) { @@ -563,8 +561,11 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev, /* create the vring */ vq = vring_create_virtqueue(index, num, SMP_CACHE_BYTES, &vp_dev->vdev, - true, true, ctx, - notify, callback, name); + true, true, + cfg->ctx ? cfg->ctx[cfg->cfg_idx] : false, + notify, + cfg->callbacks[cfg->cfg_idx], + cfg->names[cfg->cfg_idx]); if (!vq) return ERR_PTR(-ENOMEM); @@ -593,15 +594,11 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev, return ERR_PTR(err); } -static int vp_modern_find_vqs(struct virtio_device *vdev, unsigned int nvqs, - struct virtqueue *vqs[], - vq_callback_t *callbacks[], - const char * const names[], const bool *ctx, - struct irq_affinity *desc) +static int vp_modern_find_vqs(struct virtio_device *vdev, struct virtio_vq_config *cfg) { struct virtio_pci_device *vp_dev = to_vp_device(vdev); struct virtqueue *vq; - int rc = vp_find_vqs(vdev, nvqs, vqs, callbacks, names, ctx, desc); + int rc = vp_find_vqs(vdev, cfg); if (rc) return rc; diff --git a/drivers/virtio/virtio_vdpa.c b/drivers/virtio/virtio_vdpa.c index e803db0da307..7f3e173f669c 100644 --- a/drivers/virtio/virtio_vdpa.c +++ b/drivers/virtio/virtio_vdpa.c @@ -142,8 +142,7 @@ static irqreturn_t virtio_vdpa_virtqueue_cb(void *private) static struct virtqueue * virtio_vdpa_setup_vq(struct virtio_device *vdev, unsigned int index, - void (*callback)(struct virtqueue *vq), - const char *name, bool ctx) + struct virtio_vq_config *cfg) { struct virtio_vdpa_device *vd_dev = to_virtio_vdpa_device(vdev); struct vdpa_device *vdpa = vd_get_vdpa(vdev); @@ -161,7 +160,7 @@ virtio_vdpa_setup_vq(struct virtio_device *vdev, unsigned int index, bool may_reduce_num = true; int err; - if (!name) + if (!cfg->names[cfg->cfg_idx]) return NULL; if (index >= vdpa->nvqs) @@ -206,8 +205,12 @@ virtio_vdpa_setup_vq(struct virtio_device *vdev, unsigned int index, else dma_dev = vdpa_get_dma_dev(vdpa); vq = vring_create_virtqueue_dma(index, max_num, align, vdev, - true, may_reduce_num, ctx, - notify, callback, name, dma_dev); + true, may_reduce_num, + cfg->ctx ? cfg->ctx[cfg->cfg_idx] : false, + notify, + cfg->callbacks[cfg->cfg_idx], + cfg->names[cfg->cfg_idx], + dma_dev); if (!vq) { err = -ENOMEM; goto error_new_virtqueue; @@ -216,7 +219,7 @@ virtio_vdpa_setup_vq(struct virtio_device *vdev, unsigned int index, vq->num_max = max_num; /* Setup virtqueue callback */ - cb.callback = callback ? virtio_vdpa_virtqueue_cb : NULL; + cb.callback = cfg->callbacks[cfg->cfg_idx] ? virtio_vdpa_virtqueue_cb : NULL; cb.private = info; cb.trigger = NULL; ops->set_vq_cb(vdpa, index, &cb); @@ -356,12 +359,8 @@ create_affinity_masks(unsigned int nvecs, struct irq_affinity *affd) return masks; } -static int virtio_vdpa_find_vqs(struct virtio_device *vdev, unsigned int nvqs, - struct virtqueue *vqs[], - vq_callback_t *callbacks[], - const char * const names[], - const bool *ctx, - struct irq_affinity *desc) +static int virtio_vdpa_find_vqs(struct virtio_device *vdev, + struct virtio_vq_config *cfg) { struct virtio_vdpa_device *vd_dev = to_virtio_vdpa_device(vdev); struct vdpa_device *vdpa = vd_get_vdpa(vdev); @@ -369,24 +368,25 @@ static int virtio_vdpa_find_vqs(struct virtio_device *vdev, unsigned int nvqs, struct irq_affinity default_affd = { 0 }; struct cpumask *masks; struct vdpa_callback cb; - bool has_affinity = desc && ops->set_vq_affinity; + bool has_affinity = cfg->desc && ops->set_vq_affinity; + struct virtqueue **vqs = cfg->vqs; + unsigned int nvqs = cfg->nvqs; int i, err, queue_idx = 0; if (has_affinity) { - masks = create_affinity_masks(nvqs, desc ? desc : &default_affd); + masks = create_affinity_masks(nvqs, cfg->desc ? cfg->desc : &default_affd); if (!masks) return -ENOMEM; } for (i = 0; i < nvqs; ++i) { - if (!names[i]) { + if (!cfg->names[i]) { vqs[i] = NULL; continue; } - vqs[i] = virtio_vdpa_setup_vq(vdev, queue_idx++, - callbacks[i], names[i], ctx ? - ctx[i] : false); + cfg->cfg_idx = i; + vqs[i] = virtio_vdpa_setup_vq(vdev, queue_idx++, cfg); if (IS_ERR(vqs[i])) { err = PTR_ERR(vqs[i]); goto err_setup_vq; diff --git a/include/linux/virtio_config.h b/include/linux/virtio_config.h index da9b271b54db..1df8634d1258 100644 --- a/include/linux/virtio_config.h +++ b/include/linux/virtio_config.h @@ -96,6 +96,20 @@ typedef void vq_callback_t(struct virtqueue *); * @create_avq: create admin virtqueue resource. * @destroy_avq: destroy admin virtqueue resource. */ + +struct virtio_vq_config { + unsigned int nvqs; + + /* the vq index may not eq to the cfg index of the other array items */ + unsigned int cfg_idx; + + struct virtqueue **vqs; + vq_callback_t **callbacks; + const char *const *names; + const bool *ctx; + struct irq_affinity *desc; +}; + struct virtio_config_ops { void (*get)(struct virtio_device *vdev, unsigned offset, void *buf, unsigned len); @@ -105,10 +119,7 @@ struct virtio_config_ops { u8 (*get_status)(struct virtio_device *vdev); void (*set_status)(struct virtio_device *vdev, u8 status); void (*reset)(struct virtio_device *vdev); - int (*find_vqs)(struct virtio_device *, unsigned nvqs, - struct virtqueue *vqs[], vq_callback_t *callbacks[], - const char * const names[], const bool *ctx, - struct irq_affinity *desc); + int (*find_vqs)(struct virtio_device *vdev, struct virtio_vq_config *cfg); void (*del_vqs)(struct virtio_device *); void (*synchronize_cbs)(struct virtio_device *); u64 (*get_features)(struct virtio_device *vdev); @@ -217,8 +228,14 @@ struct virtqueue *virtio_find_single_vq(struct virtio_device *vdev, vq_callback_t *callbacks[] = { c }; const char *names[] = { n }; struct virtqueue *vq; - int err = vdev->config->find_vqs(vdev, 1, &vq, callbacks, names, NULL, - NULL); + struct virtio_vq_config cfg = {}; + + cfg.nvqs = 1; + cfg.vqs = &vq; + cfg.callbacks = callbacks; + cfg.names = names; + + int err = vdev->config->find_vqs(vdev, &cfg); if (err < 0) return ERR_PTR(err); return vq; @@ -230,7 +247,15 @@ int virtio_find_vqs(struct virtio_device *vdev, unsigned nvqs, const char * const names[], struct irq_affinity *desc) { - return vdev->config->find_vqs(vdev, nvqs, vqs, callbacks, names, NULL, desc); + struct virtio_vq_config cfg = {}; + + cfg.nvqs = nvqs; + cfg.vqs = vqs; + cfg.callbacks = callbacks; + cfg.names = names; + cfg.desc = desc; + + return vdev->config->find_vqs(vdev, &cfg); } static inline @@ -239,8 +264,16 @@ int virtio_find_vqs_ctx(struct virtio_device *vdev, unsigned nvqs, const char * const names[], const bool *ctx, struct irq_affinity *desc) { - return vdev->config->find_vqs(vdev, nvqs, vqs, callbacks, names, ctx, - desc); + struct virtio_vq_config cfg = {}; + + cfg.nvqs = nvqs; + cfg.vqs = vqs; + cfg.callbacks = callbacks; + cfg.names = names; + cfg.ctx = ctx; + cfg.desc = desc; + + return vdev->config->find_vqs(vdev, &cfg); } /** From patchwork Fri Feb 23 08:27:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 13568651 Received: from out30-99.freemail.mail.aliyun.com (out30-99.freemail.mail.aliyun.com [115.124.30.99]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 320DB249F5; Fri, 23 Feb 2024 08:27:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.99 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708676867; cv=none; b=QYAenBjDX8pysbEW/BJqpdMiTdA3UCpSxOD3s+5+aRuRm8yG4/GAMvoJ0eZh9tKsc4/gZy3mX1jQwlrhDwOtWtAkHiEoXWk47qetGh6RI6crGO15YLdEnCdJb7BF4jDw5UsS4Wc9iWmXoLHqiBJCLtwkD0GgYquLv1SOGIgO8UI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708676867; c=relaxed/simple; bh=YFfgc3WNDJrGPF64BDox3LEsQ5mI6kJKUnLSacbLjtQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=RYeNw7Yus+TQOfjw/Oh1qCJ+v0LnMX2vtsEE2v0h4nn5CH7M0avsm+PDG+wyq2Gz4hh+NdL+l2fY1ai/i3Tj0xRWFsM27lzgNX734dEpHDYIc56zbhhkC45Z2WUEkuW0FhcgZaFf6gu7PMbLMKo6rIhyLRIaCsJ9RadEedynhjA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=ZGs7V5FC; arc=none smtp.client-ip=115.124.30.99 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="ZGs7V5FC" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1708676862; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=uEdMiSeda4+sBhC7JuPielwJQteQtKvojs1Lc+xVdZc=; b=ZGs7V5FCVqd8CRWQ3OIif+5lo5MECGBDr1RtTSfeECnJWlIeDTSmoNUGHKVS4pSzvK9CqQ+qesVApqPJb7hu9kpwEAjz4IKHDMAk9PyBdMLySBauKTcFCgHUhPIrY/1zGw+oRirtbVVcp7McoI45GK3FLEwUdVU+/0eL7d25jNA= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R101e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045176;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=35;SR=0;TI=SMTPD_---0W13ndKC_1708676859; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0W13ndKC_1708676859) by smtp.aliyun-inc.com; Fri, 23 Feb 2024 16:27:40 +0800 From: Xuan Zhuo To: virtualization@lists.linux.dev Cc: Richard Weinberger , Anton Ivanov , Johannes Berg , "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Hans de Goede , =?utf-8?q?Ilpo_J=C3=A4rvinen?= , Vadim Pasternak , Bjorn Andersson , Mathieu Poirier , Cornelia Huck , Halil Pasic , Eric Farman , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , linux-um@lists.infradead.org, netdev@vger.kernel.org, platform-driver-x86@vger.kernel.org, linux-remoteproc@vger.kernel.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH vhost v2 08/19] virtio: vring_create_virtqueue: pass struct instead of multi parameters Date: Fri, 23 Feb 2024 16:27:15 +0800 Message-Id: <20240223082726.52915-9-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.32.0.3.g01195cf9f In-Reply-To: <20240223082726.52915-1-xuanzhuo@linux.alibaba.com> References: <20240223082726.52915-1-xuanzhuo@linux.alibaba.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Git-Hash: 510995f33855 Now, we pass multi parameters to vring_create_virtqueue. These parameters may from transport or from driver. vring_create_virtqueue is called by many places. Every time, we try to add a new parameter, that is difficult. If parameters from the driver, that should directly be passed to vring. Then the vring can access the config from driver directly. If parameters from the transport, we squish the parameters to a structure. That will be helpful to add new parameter. Because the virtio_uml.c changes the name, so change the "names" inside the virtio_vq_config from "const char *const *names" to "const char **names". Signed-off-by: Xuan Zhuo --- arch/um/drivers/virtio_uml.c | 14 +++++--- drivers/s390/virtio/virtio_ccw.c | 14 ++++---- drivers/virtio/virtio_mmio.c | 14 ++++---- drivers/virtio/virtio_pci_legacy.c | 15 ++++---- drivers/virtio/virtio_pci_modern.c | 15 ++++---- drivers/virtio/virtio_ring.c | 57 ++++++++++++------------------ drivers/virtio/virtio_vdpa.c | 21 +++++------ include/linux/virtio_config.h | 6 ++-- include/linux/virtio_ring.h | 40 ++++++++------------- 9 files changed, 93 insertions(+), 103 deletions(-) diff --git a/arch/um/drivers/virtio_uml.c b/arch/um/drivers/virtio_uml.c index c13dfeeb90c4..1c2d59d3d02b 100644 --- a/arch/um/drivers/virtio_uml.c +++ b/arch/um/drivers/virtio_uml.c @@ -942,6 +942,7 @@ static struct virtqueue *vu_setup_vq(struct virtio_device *vdev, { struct virtio_uml_device *vu_dev = to_virtio_uml_device(vdev); struct platform_device *pdev = vu_dev->pdev; + struct vq_transport_config tp_cfg = {}; struct virtio_uml_vq_info *info; struct virtqueue *vq; int num = MAX_SUPPORTED_QUEUE_SIZE; @@ -955,10 +956,15 @@ static struct virtqueue *vu_setup_vq(struct virtio_device *vdev, snprintf(info->name, sizeof(info->name), "%s.%d-%s", pdev->name, pdev->id, cfg->names[cfg->cfg_idx]); - vq = vring_create_virtqueue(index, num, PAGE_SIZE, vdev, true, true, - cfg->ctx ? cfg->ctx[cfg->cfg_idx] : false, - vu_notify, - cfg->callbacks[cfg->cfg_idx], info->name); + tp_cfg.num = num; + tp_cfg.vring_align = PAGE_SIZE; + tp_cfg.weak_barriers = true; + tp_cfg.may_reduce_num = true; + tp_cfg.notify = vu_notify; + + cfg->names[cfg->cfg_idx] = info->name; + + vq = vring_create_virtqueue(vdev, index, &tp_cfg, cfg); if (!vq) { rc = -ENOMEM; goto error_create; diff --git a/drivers/s390/virtio/virtio_ccw.c b/drivers/s390/virtio/virtio_ccw.c index 11eea5086cff..f8d5bbd13359 100644 --- a/drivers/s390/virtio/virtio_ccw.c +++ b/drivers/s390/virtio/virtio_ccw.c @@ -503,6 +503,7 @@ static struct virtqueue *virtio_ccw_setup_vq(struct virtio_device *vdev, struct virtio_vq_config *cfg) { struct virtio_ccw_device *vcdev = to_vc_device(vdev); + struct vq_transport_config tp_cfg = {}; bool (*notify)(struct virtqueue *vq); int err; struct virtqueue *vq = NULL; @@ -536,13 +537,14 @@ static struct virtqueue *virtio_ccw_setup_vq(struct virtio_device *vdev, goto out_err; } may_reduce = vcdev->revision > 0; - vq = vring_create_virtqueue(i, info->num, KVM_VIRTIO_CCW_RING_ALIGN, - vdev, true, may_reduce, - cfg->ctx ? cfg->ctx[cfg->cfg_idx] : false, - notify, - cfg->callbacks[cfg->cfg_idx], - cfg->names[cfg->cfg_idx]); + tp_cfg.num = info->num; + tp_cfg.vring_align = KVM_VIRTIO_CCW_RING_ALIGN; + tp_cfg.weak_barriers = true; + tp_cfg.may_reduce_num = may_reduce; + tp_cfg.notify = notify; + + vq = vring_create_virtqueue(vdev, i, &tp_cfg, cfg); if (!vq) { /* For now, we fail if we can't get the requested size. */ dev_warn(&vcdev->cdev->dev, "no vq\n"); diff --git a/drivers/virtio/virtio_mmio.c b/drivers/virtio/virtio_mmio.c index feb823d279d2..cb172fa4d7cc 100644 --- a/drivers/virtio/virtio_mmio.c +++ b/drivers/virtio/virtio_mmio.c @@ -373,6 +373,7 @@ static struct virtqueue *vm_setup_vq(struct virtio_device *vdev, unsigned int in struct virtio_vq_config *cfg) { struct virtio_mmio_device *vm_dev = to_virtio_mmio_device(vdev); + struct vq_transport_config tp_cfg = {}; bool (*notify)(struct virtqueue *vq); struct virtio_mmio_vq_info *info; struct virtqueue *vq; @@ -411,13 +412,14 @@ static struct virtqueue *vm_setup_vq(struct virtio_device *vdev, unsigned int in goto error_new_virtqueue; } + tp_cfg.num = num; + tp_cfg.vring_align = VIRTIO_MMIO_VRING_ALIGN; + tp_cfg.weak_barriers = true; + tp_cfg.may_reduce_num = true; + tp_cfg.notify = notify; + /* Create the vring */ - vq = vring_create_virtqueue(index, num, VIRTIO_MMIO_VRING_ALIGN, vdev, - true, true, - cfg->ctx ? cfg->ctx[cfg->cfg_idx] : false, - notify, - cfg->callbacks[cfg->cfg_idx], - cfg->names[cfg->cfg_idx]); + vq = vring_create_virtqueue(vdev, index, &tp_cfg, cfg); if (!vq) { err = -ENOMEM; goto error_new_virtqueue; diff --git a/drivers/virtio/virtio_pci_legacy.c b/drivers/virtio/virtio_pci_legacy.c index e8d22fce32f5..6fe675b2a5e5 100644 --- a/drivers/virtio/virtio_pci_legacy.c +++ b/drivers/virtio/virtio_pci_legacy.c @@ -113,6 +113,7 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev, struct virtio_vq_config *cfg, u16 msix_vec) { + struct vq_transport_config tp_cfg = {}; struct virtqueue *vq; u16 num; int err; @@ -125,14 +126,14 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev, info->msix_vector = msix_vec; + tp_cfg.num = num; + tp_cfg.vring_align = VIRTIO_PCI_VRING_ALIGN; + tp_cfg.weak_barriers = true; + tp_cfg.may_reduce_num = false; + tp_cfg.notify = vp_notify; + /* create the vring */ - vq = vring_create_virtqueue(index, num, - VIRTIO_PCI_VRING_ALIGN, &vp_dev->vdev, - true, false, - cfg->ctx ? cfg->ctx[cfg->cfg_idx] : false, - vp_notify, - cfg->callbacks[cfg->cfg_idx], - cfg->names[cfg->cfg_idx]); + vq = vring_create_virtqueue(&vp_dev->vdev, index, &tp_cfg, cfg); if (!vq) return ERR_PTR(-ENOMEM); diff --git a/drivers/virtio/virtio_pci_modern.c b/drivers/virtio/virtio_pci_modern.c index 04c7782392a3..e630762f9b77 100644 --- a/drivers/virtio/virtio_pci_modern.c +++ b/drivers/virtio/virtio_pci_modern.c @@ -535,6 +535,7 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev, { struct virtio_pci_modern_device *mdev = &vp_dev->mdev; + struct vq_transport_config tp_cfg = {}; bool (*notify)(struct virtqueue *vq); struct virtqueue *vq; bool is_avq; @@ -558,14 +559,14 @@ static struct virtqueue *setup_vq(struct virtio_pci_device *vp_dev, info->msix_vector = msix_vec; + tp_cfg.num = num; + tp_cfg.vring_align = SMP_CACHE_BYTES; + tp_cfg.weak_barriers = true; + tp_cfg.may_reduce_num = true; + tp_cfg.notify = notify; + /* create the vring */ - vq = vring_create_virtqueue(index, num, - SMP_CACHE_BYTES, &vp_dev->vdev, - true, true, - cfg->ctx ? cfg->ctx[cfg->cfg_idx] : false, - notify, - cfg->callbacks[cfg->cfg_idx], - cfg->names[cfg->cfg_idx]); + vq = vring_create_virtqueue(&vp_dev->vdev, index, &tp_cfg, cfg); if (!vq) return ERR_PTR(-ENOMEM); diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index c1e4edaeb446..b77ee139a5d9 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -2729,43 +2729,32 @@ static struct virtqueue *__vring_new_virtqueue(unsigned int index, return &vq->vq; } -struct virtqueue *vring_create_virtqueue( - unsigned int index, - unsigned int num, - unsigned int vring_align, - struct virtio_device *vdev, - bool weak_barriers, - bool may_reduce_num, - bool context, - bool (*notify)(struct virtqueue *), - void (*callback)(struct virtqueue *), - const char *name) +struct virtqueue *vring_create_virtqueue(struct virtio_device *vdev, + unsigned int index, + struct vq_transport_config *tp_cfg, + struct virtio_vq_config *cfg) { + struct device *dma_dev; + unsigned int num; + unsigned int vring_align; + bool weak_barriers; + bool may_reduce_num; + bool context; + bool (*notify)(struct virtqueue *_); + void (*callback)(struct virtqueue *_); + const char *name; - if (virtio_has_feature(vdev, VIRTIO_F_RING_PACKED)) - return vring_create_virtqueue_packed(index, num, vring_align, - vdev, weak_barriers, may_reduce_num, - context, notify, callback, name, vdev->dev.parent); + dma_dev = tp_cfg->dma_dev ? : vdev->dev.parent; - return vring_create_virtqueue_split(index, num, vring_align, - vdev, weak_barriers, may_reduce_num, - context, notify, callback, name, vdev->dev.parent); -} -EXPORT_SYMBOL_GPL(vring_create_virtqueue); + num = tp_cfg->num; + vring_align = tp_cfg->vring_align; + weak_barriers = tp_cfg->weak_barriers; + may_reduce_num = tp_cfg->may_reduce_num; + notify = tp_cfg->notify; -struct virtqueue *vring_create_virtqueue_dma( - unsigned int index, - unsigned int num, - unsigned int vring_align, - struct virtio_device *vdev, - bool weak_barriers, - bool may_reduce_num, - bool context, - bool (*notify)(struct virtqueue *), - void (*callback)(struct virtqueue *), - const char *name, - struct device *dma_dev) -{ + name = cfg->names[cfg->cfg_idx]; + callback = cfg->callbacks[cfg->cfg_idx]; + context = cfg->ctx ? cfg->ctx[cfg->cfg_idx] : false; if (virtio_has_feature(vdev, VIRTIO_F_RING_PACKED)) return vring_create_virtqueue_packed(index, num, vring_align, @@ -2776,7 +2765,7 @@ struct virtqueue *vring_create_virtqueue_dma( vdev, weak_barriers, may_reduce_num, context, notify, callback, name, dma_dev); } -EXPORT_SYMBOL_GPL(vring_create_virtqueue_dma); +EXPORT_SYMBOL_GPL(vring_create_virtqueue); /** * virtqueue_resize - resize the vring of vq diff --git a/drivers/virtio/virtio_vdpa.c b/drivers/virtio/virtio_vdpa.c index 7f3e173f669c..1ab219fce653 100644 --- a/drivers/virtio/virtio_vdpa.c +++ b/drivers/virtio/virtio_vdpa.c @@ -146,8 +146,8 @@ virtio_vdpa_setup_vq(struct virtio_device *vdev, unsigned int index, { struct virtio_vdpa_device *vd_dev = to_virtio_vdpa_device(vdev); struct vdpa_device *vdpa = vd_get_vdpa(vdev); - struct device *dma_dev; const struct vdpa_config_ops *ops = vdpa->config; + struct vq_transport_config tp_cfg = {}; struct virtio_vdpa_vq_info *info; bool (*notify)(struct virtqueue *vq) = virtio_vdpa_notify; struct vdpa_callback cb; @@ -201,16 +201,17 @@ virtio_vdpa_setup_vq(struct virtio_device *vdev, unsigned int index, align = ops->get_vq_align(vdpa); if (ops->get_vq_dma_dev) - dma_dev = ops->get_vq_dma_dev(vdpa, index); + tp_cfg.dma_dev = ops->get_vq_dma_dev(vdpa, index); else - dma_dev = vdpa_get_dma_dev(vdpa); - vq = vring_create_virtqueue_dma(index, max_num, align, vdev, - true, may_reduce_num, - cfg->ctx ? cfg->ctx[cfg->cfg_idx] : false, - notify, - cfg->callbacks[cfg->cfg_idx], - cfg->names[cfg->cfg_idx], - dma_dev); + tp_cfg.dma_dev = vdpa_get_dma_dev(vdpa); + + tp_cfg.num = max_num; + tp_cfg.vring_align = align; + tp_cfg.weak_barriers = true; + tp_cfg.may_reduce_num = may_reduce_num; + tp_cfg.notify = notify; + + vq = vring_create_virtqueue(vdev, index, &tp_cfg, cfg); if (!vq) { err = -ENOMEM; goto error_new_virtqueue; diff --git a/include/linux/virtio_config.h b/include/linux/virtio_config.h index 1df8634d1258..d47188303d34 100644 --- a/include/linux/virtio_config.h +++ b/include/linux/virtio_config.h @@ -105,7 +105,7 @@ struct virtio_vq_config { struct virtqueue **vqs; vq_callback_t **callbacks; - const char *const *names; + const char **names; const bool *ctx; struct irq_affinity *desc; }; @@ -252,7 +252,7 @@ int virtio_find_vqs(struct virtio_device *vdev, unsigned nvqs, cfg.nvqs = nvqs; cfg.vqs = vqs; cfg.callbacks = callbacks; - cfg.names = names; + cfg.names = (const char **)names; cfg.desc = desc; return vdev->config->find_vqs(vdev, &cfg); @@ -269,7 +269,7 @@ int virtio_find_vqs_ctx(struct virtio_device *vdev, unsigned nvqs, cfg.nvqs = nvqs; cfg.vqs = vqs; cfg.callbacks = callbacks; - cfg.names = names; + cfg.names = (const char **)names; cfg.ctx = ctx; cfg.desc = desc; diff --git a/include/linux/virtio_ring.h b/include/linux/virtio_ring.h index 9b33df741b63..cd8042c79814 100644 --- a/include/linux/virtio_ring.h +++ b/include/linux/virtio_ring.h @@ -5,6 +5,7 @@ #include #include #include +#include /* * Barriers in virtio are tricky. Non-SMP virtio guests can't assume @@ -60,38 +61,25 @@ struct virtio_device; struct virtqueue; struct device; +struct vq_transport_config { + unsigned int num; + unsigned int vring_align; + bool weak_barriers; + bool may_reduce_num; + bool (*notify)(struct virtqueue *vq); + struct device *dma_dev; +}; + /* * Creates a virtqueue and allocates the descriptor ring. If * may_reduce_num is set, then this may allocate a smaller ring than * expected. The caller should query virtqueue_get_vring_size to learn * the actual size of the ring. */ -struct virtqueue *vring_create_virtqueue(unsigned int index, - unsigned int num, - unsigned int vring_align, - struct virtio_device *vdev, - bool weak_barriers, - bool may_reduce_num, - bool ctx, - bool (*notify)(struct virtqueue *vq), - void (*callback)(struct virtqueue *vq), - const char *name); - -/* - * Creates a virtqueue and allocates the descriptor ring with per - * virtqueue DMA device. - */ -struct virtqueue *vring_create_virtqueue_dma(unsigned int index, - unsigned int num, - unsigned int vring_align, - struct virtio_device *vdev, - bool weak_barriers, - bool may_reduce_num, - bool ctx, - bool (*notify)(struct virtqueue *vq), - void (*callback)(struct virtqueue *vq), - const char *name, - struct device *dma_dev); +struct virtqueue *vring_create_virtqueue(struct virtio_device *vdev, + unsigned int index, + struct vq_transport_config *tp_cfg, + struct virtio_vq_config *cfg); /* * Creates a virtqueue with a standard layout but a caller-allocated From patchwork Fri Feb 23 08:27:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 13568652 Received: from out30-133.freemail.mail.aliyun.com (out30-133.freemail.mail.aliyun.com [115.124.30.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 458581863F; Fri, 23 Feb 2024 08:27:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.133 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708676868; cv=none; b=TRc6lBou4LfxSwvqrRjh9VoyMnVwP4vplwh4BGaG+jVAAXZeYuiiCT56cv9yljnjKPmoUwxDXy1jFsp6g0q/dCvpiYdXVNSu5yqUWzOYcYiDdZhEZMGUGLAubEH1B4Z1UHc48i//zDIPUWxWJZ/snk6zbxuq/CmHBFAruWaGD5I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708676868; c=relaxed/simple; bh=BGa3ngyKONG8LuUKOgWRXlfB1O4a8MgChNYUqN4IsQQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=jlspwMuWRFDFhlOG8xkAoU09bLCn0+ZorHq+QzQOyh9hQB9Z2Zoyp8ZmqSfnJIsiN/ppTtPi8Gr8dFjKr9NuSc8Zqz8OEs8KNgzzxy45Qn+49m9LQer9PDlqRW1QO3CVbaJoIdCFWEShK0m+tp/GV0KF41w1VerlWADAf6N1XKQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=cCWuFbhE; arc=none smtp.client-ip=115.124.30.133 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="cCWuFbhE" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1708676863; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=YatZ+z+3WO7Vfe+/3/O0MuxH+VpgqXd3r1LEPxB4vz4=; b=cCWuFbhE1d0ULQ0nd7llj3h1IqNiR4TZ0o2PdRXSB+EC40dZzS3B6k+YR14kTaaOXJY2BR6yp8bmUgbcUM/MAE1or0HUXSWxUDKJCEQKIwjUdCmwubFMxvAoUxApUftZEgj1rXIzWY4vOJ5u78oMKZas5imNQo6SUsf9xL5C0fM= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R171e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045168;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=35;SR=0;TI=SMTPD_---0W13gV7R_1708676861; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0W13gV7R_1708676861) by smtp.aliyun-inc.com; Fri, 23 Feb 2024 16:27:41 +0800 From: Xuan Zhuo To: virtualization@lists.linux.dev Cc: Richard Weinberger , Anton Ivanov , Johannes Berg , "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Hans de Goede , =?utf-8?q?Ilpo_J=C3=A4rvinen?= , Vadim Pasternak , Bjorn Andersson , Mathieu Poirier , Cornelia Huck , Halil Pasic , Eric Farman , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , linux-um@lists.infradead.org, netdev@vger.kernel.org, platform-driver-x86@vger.kernel.org, linux-remoteproc@vger.kernel.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH vhost v2 09/19] virtio: vring_new_virtqueue(): pass struct instead of multi parameters Date: Fri, 23 Feb 2024 16:27:16 +0800 Message-Id: <20240223082726.52915-10-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.32.0.3.g01195cf9f In-Reply-To: <20240223082726.52915-1-xuanzhuo@linux.alibaba.com> References: <20240223082726.52915-1-xuanzhuo@linux.alibaba.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Git-Hash: 510995f33855 Just like find_vqs(), it is time to refactor the vring_new_virtqueue(). We pass the similar struct to vring_new_virtqueue. Signed-off-by: Xuan Zhuo --- drivers/platform/mellanox/mlxbf-tmfifo.c | 13 +++++--- drivers/remoteproc/remoteproc_virtio.c | 11 ++++--- drivers/virtio/virtio_ring.c | 29 +++++++++++----- include/linux/virtio_ring.h | 42 +++++++++++++++++++----- tools/virtio/virtio_test.c | 4 +-- tools/virtio/vringh_test.c | 28 ++++++++-------- 6 files changed, 85 insertions(+), 42 deletions(-) diff --git a/drivers/platform/mellanox/mlxbf-tmfifo.c b/drivers/platform/mellanox/mlxbf-tmfifo.c index 4252388f52a2..36cfce5ba8cf 100644 --- a/drivers/platform/mellanox/mlxbf-tmfifo.c +++ b/drivers/platform/mellanox/mlxbf-tmfifo.c @@ -1059,6 +1059,7 @@ static int mlxbf_tmfifo_virtio_find_vqs(struct virtio_device *vdev, struct virtio_vq_config *cfg) { struct mlxbf_tmfifo_vdev *tm_vdev = mlxbf_vdev_to_tmfifo(vdev); + struct vq_transport_config tp_cfg = {}; struct virtqueue **vqs = cfg->vqs; struct mlxbf_tmfifo_vring *vring; unsigned int nvqs = cfg->nvqs; @@ -1078,10 +1079,14 @@ static int mlxbf_tmfifo_virtio_find_vqs(struct virtio_device *vdev, /* zero vring */ size = vring_size(vring->num, vring->align); memset(vring->va, 0, size); - vq = vring_new_virtqueue(i, vring->num, vring->align, vdev, - false, false, vring->va, - mlxbf_tmfifo_virtio_notify, - cfg->callbacks[i], cfg->names[i]); + + tp_cfg.num = vring->num; + tp_cfg.vring_align = vring->align; + tp_cfg.weak_barriers = false; + tp_cfg.notify = mlxbf_tmfifo_virtio_notify; + + cfg->cfg_idx = i; + vq = vring_new_virtqueue(vdev, i, vring->va, &tp_cfg, cfg); if (!vq) { dev_err(&vdev->dev, "vring_new_virtqueue failed\n"); ret = -ENOMEM; diff --git a/drivers/remoteproc/remoteproc_virtio.c b/drivers/remoteproc/remoteproc_virtio.c index 57d51c9c7b63..70c32837f9dc 100644 --- a/drivers/remoteproc/remoteproc_virtio.c +++ b/drivers/remoteproc/remoteproc_virtio.c @@ -106,6 +106,7 @@ static struct virtqueue *rp_find_vq(struct virtio_device *vdev, { struct rproc_vdev *rvdev = vdev_to_rvdev(vdev); struct rproc *rproc = vdev_to_rproc(vdev); + struct vq_transport_config tp_cfg; struct device *dev = &rproc->dev; struct rproc_mem_entry *mem; struct rproc_vring *rvring; @@ -138,14 +139,16 @@ static struct virtqueue *rp_find_vq(struct virtio_device *vdev, dev_dbg(dev, "vring%d: va %pK qsz %d notifyid %d\n", id, addr, num, rvring->notifyid); + tp_cfg.num = num; + tp_cfg.vring_align = rvring->align; + tp_cfg.weak_barriers = false; + tp_cfg.notify = rproc_virtio_notify; + /* * Create the new vq, and tell virtio we're not interested in * the 'weak' smp barriers, since we're talking with a real device. */ - vq = vring_new_virtqueue(id, num, rvring->align, vdev, false, - cfg->ctx ? cfg->ctx[cfg->cfg_idx] : false, - addr, rproc_virtio_notify, cfg->callbacks[cfg->cfg_idx], - cfg->names[cfg->cfg_idx]); + vq = vring_new_virtqueue(vdev, id, addr, &tp_cfg, cfg); if (!vq) { dev_err(dev, "vring_new_virtqueue %s failed\n", cfg->names[cfg->cfg_idx]); rproc_free_vring(rvring); diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index b77ee139a5d9..e21d506d0f93 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -2909,18 +2909,29 @@ int virtqueue_reset(struct virtqueue *_vq, EXPORT_SYMBOL_GPL(virtqueue_reset); /* Only available for split ring */ -struct virtqueue *vring_new_virtqueue(unsigned int index, - unsigned int num, - unsigned int vring_align, - struct virtio_device *vdev, - bool weak_barriers, - bool context, +struct virtqueue *vring_new_virtqueue(struct virtio_device *vdev, + unsigned int index, void *pages, - bool (*notify)(struct virtqueue *vq), - void (*callback)(struct virtqueue *vq), - const char *name) + struct vq_transport_config *tp_cfg, + struct virtio_vq_config *cfg) { struct vring_virtqueue_split vring_split = {}; + unsigned int num; + unsigned int vring_align; + bool weak_barriers; + bool context; + bool (*notify)(struct virtqueue *_); + void (*callback)(struct virtqueue *_); + const char *name; + + num = tp_cfg->num; + vring_align = tp_cfg->vring_align; + weak_barriers = tp_cfg->weak_barriers; + notify = tp_cfg->notify; + + name = cfg->names[cfg->cfg_idx]; + callback = cfg->callbacks[cfg->cfg_idx]; + context = cfg->ctx ? cfg->ctx[cfg->cfg_idx] : false; if (virtio_has_feature(vdev, VIRTIO_F_RING_PACKED)) return NULL; diff --git a/include/linux/virtio_ring.h b/include/linux/virtio_ring.h index cd8042c79814..616937780785 100644 --- a/include/linux/virtio_ring.h +++ b/include/linux/virtio_ring.h @@ -85,16 +85,40 @@ struct virtqueue *vring_create_virtqueue(struct virtio_device *vdev, * Creates a virtqueue with a standard layout but a caller-allocated * ring. */ -struct virtqueue *vring_new_virtqueue(unsigned int index, - unsigned int num, - unsigned int vring_align, - struct virtio_device *vdev, - bool weak_barriers, - bool ctx, +struct virtqueue *vring_new_virtqueue(struct virtio_device *vdev, + unsigned int index, void *pages, - bool (*notify)(struct virtqueue *vq), - void (*callback)(struct virtqueue *vq), - const char *name); + struct vq_transport_config *tp_cfg, + struct virtio_vq_config *cfg); + +static inline struct virtqueue *vring_new_virtqueue_one(unsigned int index, + unsigned int num, + unsigned int vring_align, + struct virtio_device *vdev, + bool weak_barriers, + bool context, + void *pages, + bool (*notify)(struct virtqueue *vq), + void (*callback)(struct virtqueue *vq), + const char *name) +{ + struct vq_transport_config tp_cfg = {}; + struct virtio_vq_config cfg = {}; + vq_callback_t *callbacks[] = { callback }; + const char *names[] = { name }; + + tp_cfg.num = num; + tp_cfg.vring_align = vring_align; + tp_cfg.weak_barriers = weak_barriers; + tp_cfg.notify = notify; + + cfg.nvqs = 1; + cfg.callbacks = callbacks; + cfg.names = names; + cfg.ctx = &context; + + return vring_new_virtqueue(vdev, index, pages, &tp_cfg, &cfg); +} /* * Destroys a virtqueue. If created with vring_create_virtqueue, this diff --git a/tools/virtio/virtio_test.c b/tools/virtio/virtio_test.c index 028f54e6854a..e41300d71d5e 100644 --- a/tools/virtio/virtio_test.c +++ b/tools/virtio/virtio_test.c @@ -102,8 +102,8 @@ static void vq_reset(struct vq_info *info, int num, struct virtio_device *vdev) memset(info->ring, 0, vring_size(num, 4096)); vring_init(&info->vring, num, info->ring, 4096); - info->vq = vring_new_virtqueue(info->idx, num, 4096, vdev, true, false, - info->ring, vq_notify, vq_callback, "test"); + info->vq = vring_new_virtqueue_one(info->idx, num, 4096, vdev, true, false, + info->ring, vq_notify, vq_callback, "test"); assert(info->vq); info->vq->priv = info; } diff --git a/tools/virtio/vringh_test.c b/tools/virtio/vringh_test.c index 98ff808d6f0c..040689111584 100644 --- a/tools/virtio/vringh_test.c +++ b/tools/virtio/vringh_test.c @@ -316,11 +316,11 @@ static int parallel_test(u64 features, if (sched_setaffinity(getpid(), sizeof(cpu_set), &cpu_set)) err(1, "Could not set affinity to cpu %u", first_cpu); - vq = vring_new_virtqueue(0, RINGSIZE, ALIGN, &gvdev.vdev, true, - false, guest_map, - fast_vringh ? no_notify_host - : parallel_notify_host, - never_callback_guest, "guest vq"); + vq = vring_new_virtqueue_one(0, RINGSIZE, ALIGN, &gvdev.vdev, true, + false, guest_map, + fast_vringh ? no_notify_host + : parallel_notify_host, + never_callback_guest, "guest vq"); /* Don't kfree indirects. */ __kfree_ignore_start = indirects; @@ -485,10 +485,10 @@ int main(int argc, char *argv[]) memset(__user_addr_min, 0, vring_size(RINGSIZE, ALIGN)); /* Set up guest side. */ - vq = vring_new_virtqueue(0, RINGSIZE, ALIGN, &vdev, true, false, - __user_addr_min, - never_notify_host, never_callback_guest, - "guest vq"); + vq = vring_new_virtqueue_one(0, RINGSIZE, ALIGN, &vdev, true, false, + __user_addr_min, + never_notify_host, never_callback_guest, + "guest vq"); /* Set up host side. */ vring_init(&vrh.vring, RINGSIZE, __user_addr_min, ALIGN); @@ -668,11 +668,11 @@ int main(int argc, char *argv[]) /* Force creation of direct, which we modify. */ __virtio_clear_bit(&vdev, VIRTIO_RING_F_INDIRECT_DESC); - vq = vring_new_virtqueue(0, RINGSIZE, ALIGN, &vdev, true, - false, __user_addr_min, - never_notify_host, - never_callback_guest, - "guest vq"); + vq = vring_new_virtqueue_one(0, RINGSIZE, ALIGN, &vdev, true, + false, __user_addr_min, + never_notify_host, + never_callback_guest, + "guest vq"); sg_init_table(guest_sg, 4); sg_set_buf(&guest_sg[0], d, sizeof(*d)*2); From patchwork Fri Feb 23 08:27:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 13568656 Received: from out30-124.freemail.mail.aliyun.com (out30-124.freemail.mail.aliyun.com [115.124.30.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D088C45025; Fri, 23 Feb 2024 08:27:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708676877; cv=none; b=p1Z4AnAW3qdCGa9EsZGhLf5CHxlXJ+N6LcOEp+1eTziPJx50f5q5I9H3h9bQ4EK/ySt3vnO/G9iPQcbR/uG3PhGg0eP6RY81PB8rat8hGol9dYoCpmgPwXaizKWql2vryHsOUKL5x78+Owvsses2GDWvFeyatdYuqvtbaZiwyK8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708676877; c=relaxed/simple; bh=KifZuocjjCmYPt6LN48VMhqh8sB/FAA2M49m55y0beo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=gf4lix7WtEzHaD3vfNcDKbCpvs+a9z9kt4IwUCEI9DR4qy4qTkTmEFziB24Ug2d15AIlsf+8NWwvJy2FDKQbeo7HmMZFGQVFpPg5wr4MxIRIBQBQOLMU7D6KmazWPkpQY7kxaDiJ0l0IY8U0ygrZf80+K41gVJ0jE2UuuIX/it4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=rOxPI/S4; arc=none smtp.client-ip=115.124.30.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="rOxPI/S4" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1708676865; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=5Ngdu8bGb2aWYjBk9XZFjVGx8lBJaDeQ5xeUtZCWaXw=; b=rOxPI/S4P+ByrRhZvApY/+j1dy8b/AdX3/nUkBe5nSFTcgUiyHGWOch3CW+Y/lo+2A2rMKl+fSZhhymz7FVQLng4FJg7zXEq8NnQi6OUa1nNawdu3f43VZaAjZ7uE/BsNoCOeUwVwLwVMOrrAhZWbe5cnpCqz3yx00dVCWT//G4= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R651e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046051;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=35;SR=0;TI=SMTPD_---0W13mV0s_1708676862; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0W13mV0s_1708676862) by smtp.aliyun-inc.com; Fri, 23 Feb 2024 16:27:43 +0800 From: Xuan Zhuo To: virtualization@lists.linux.dev Cc: Richard Weinberger , Anton Ivanov , Johannes Berg , "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Hans de Goede , =?utf-8?q?Ilpo_J=C3=A4rvinen?= , Vadim Pasternak , Bjorn Andersson , Mathieu Poirier , Cornelia Huck , Halil Pasic , Eric Farman , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , linux-um@lists.infradead.org, netdev@vger.kernel.org, platform-driver-x86@vger.kernel.org, linux-remoteproc@vger.kernel.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH vhost v2 10/19] virtio_ring: simplify the parameters of the funcs related to vring_create/new_virtqueue() Date: Fri, 23 Feb 2024 16:27:17 +0800 Message-Id: <20240223082726.52915-11-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.32.0.3.g01195cf9f In-Reply-To: <20240223082726.52915-1-xuanzhuo@linux.alibaba.com> References: <20240223082726.52915-1-xuanzhuo@linux.alibaba.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Git-Hash: 510995f33855 As the refactor of find_vqs()/vring_new_virtqueue()/vring_create_virtqueue the struct cfg/tp_cfg are passed to vring. This patch refactors the vring by these structures. This can simplify the code. Signed-off-by: Xuan Zhuo --- drivers/virtio/virtio_ring.c | 157 +++++++++++------------------------ 1 file changed, 50 insertions(+), 107 deletions(-) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index e21d506d0f93..bbd79fd52885 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -237,15 +237,11 @@ struct vring_virtqueue { #endif }; -static struct virtqueue *__vring_new_virtqueue(unsigned int index, +static struct virtqueue *__vring_new_virtqueue(struct virtio_device *vdev, + unsigned int index, struct vring_virtqueue_split *vring_split, - struct virtio_device *vdev, - bool weak_barriers, - bool context, - bool (*notify)(struct virtqueue *), - void (*callback)(struct virtqueue *), - const char *name, - struct device *dma_dev); + struct vq_transport_config *tp_cfg, + struct virtio_vq_config *cfg); static struct vring_desc_extra *vring_alloc_desc_extra(unsigned int num); static void vring_free(struct virtqueue *_vq); @@ -254,6 +250,8 @@ static void vring_free(struct virtqueue *_vq); */ #define to_vvq(_vq) container_of_const(_vq, struct vring_virtqueue, vq) +#define cfg_vq_val(cfg, key) (cfg->key[cfg->cfg_idx]) +#define cfg_vq_get(cfg, key) (cfg->key ? cfg_vq_val(cfg, key) : false) static bool virtqueue_use_indirect(const struct vring_virtqueue *vq, unsigned int total_sg) @@ -1184,32 +1182,28 @@ static int vring_alloc_queue_split(struct vring_virtqueue_split *vring_split, return 0; } -static struct virtqueue *vring_create_virtqueue_split( - unsigned int index, - unsigned int num, - unsigned int vring_align, - struct virtio_device *vdev, - bool weak_barriers, - bool may_reduce_num, - bool context, - bool (*notify)(struct virtqueue *), - void (*callback)(struct virtqueue *), - const char *name, - struct device *dma_dev) +static struct virtqueue *vring_create_virtqueue_split(struct virtio_device *vdev, + unsigned int index, + struct vq_transport_config *tp_cfg, + struct virtio_vq_config *cfg) { struct vring_virtqueue_split vring_split = {}; struct virtqueue *vq; int err; - err = vring_alloc_queue_split(&vring_split, vdev, num, vring_align, - may_reduce_num, dma_dev); + tp_cfg->dma_dev = tp_cfg->dma_dev ? : vdev->dev.parent; + + err = vring_alloc_queue_split(&vring_split, vdev, + tp_cfg->num, + tp_cfg->vring_align, + tp_cfg->may_reduce_num, + tp_cfg->dma_dev); if (err) return NULL; - vq = __vring_new_virtqueue(index, &vring_split, vdev, weak_barriers, - context, notify, callback, name, dma_dev); + vq = __vring_new_virtqueue(vdev, index, &vring_split, tp_cfg, cfg); if (!vq) { - vring_free_split(&vring_split, vdev, dma_dev); + vring_free_split(&vring_split, vdev, tp_cfg->dma_dev); return NULL; } @@ -2116,38 +2110,33 @@ static void virtqueue_reinit_packed(struct vring_virtqueue *vq) virtqueue_vring_init_packed(&vq->packed, !!vq->vq.callback); } -static struct virtqueue *vring_create_virtqueue_packed( - unsigned int index, - unsigned int num, - unsigned int vring_align, - struct virtio_device *vdev, - bool weak_barriers, - bool may_reduce_num, - bool context, - bool (*notify)(struct virtqueue *), - void (*callback)(struct virtqueue *), - const char *name, - struct device *dma_dev) +static struct virtqueue *vring_create_virtqueue_packed(struct virtio_device *vdev, + unsigned int index, + struct vq_transport_config *tp_cfg, + struct virtio_vq_config *cfg) { struct vring_virtqueue_packed vring_packed = {}; struct vring_virtqueue *vq; + struct device *dma_dev; int err; - if (vring_alloc_queue_packed(&vring_packed, vdev, num, dma_dev)) + dma_dev = tp_cfg->dma_dev ? : vdev->dev.parent; + + if (vring_alloc_queue_packed(&vring_packed, vdev, tp_cfg->num, dma_dev)) goto err_ring; vq = kmalloc(sizeof(*vq), GFP_KERNEL); if (!vq) goto err_vq; - vq->vq.callback = callback; + vq->vq.callback = cfg_vq_val(cfg, callbacks); vq->vq.vdev = vdev; - vq->vq.name = name; + vq->vq.name = cfg_vq_val(cfg, names); vq->vq.index = index; vq->vq.reset = false; vq->we_own_ring = true; - vq->notify = notify; - vq->weak_barriers = weak_barriers; + vq->notify = tp_cfg->notify; + vq->weak_barriers = tp_cfg->weak_barriers; #ifdef CONFIG_VIRTIO_HARDEN_NOTIFICATION vq->broken = true; #else @@ -2159,7 +2148,7 @@ static struct virtqueue *vring_create_virtqueue_packed( vq->premapped = false; vq->indirect = virtio_has_feature(vdev, VIRTIO_RING_F_INDIRECT_DESC) && - !context; + !cfg_vq_get(cfg, ctx); vq->event = virtio_has_feature(vdev, VIRTIO_RING_F_EVENT_IDX); if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM)) @@ -2170,9 +2159,9 @@ static struct virtqueue *vring_create_virtqueue_packed( if (err) goto err_state_extra; - virtqueue_vring_init_packed(&vring_packed, !!callback); + virtqueue_vring_init_packed(&vring_packed, !!cfg_vq_val(cfg, callbacks)); - virtqueue_init(vq, num); + virtqueue_init(vq, tp_cfg->num); virtqueue_vring_attach_packed(vq, &vring_packed); spin_lock(&vdev->vqs_list_lock); @@ -2666,15 +2655,11 @@ irqreturn_t vring_interrupt(int irq, void *_vq) EXPORT_SYMBOL_GPL(vring_interrupt); /* Only available for split ring */ -static struct virtqueue *__vring_new_virtqueue(unsigned int index, +static struct virtqueue *__vring_new_virtqueue(struct virtio_device *vdev, + unsigned int index, struct vring_virtqueue_split *vring_split, - struct virtio_device *vdev, - bool weak_barriers, - bool context, - bool (*notify)(struct virtqueue *), - void (*callback)(struct virtqueue *), - const char *name, - struct device *dma_dev) + struct vq_transport_config *tp_cfg, + struct virtio_vq_config *cfg) { struct vring_virtqueue *vq; int err; @@ -2687,25 +2672,25 @@ static struct virtqueue *__vring_new_virtqueue(unsigned int index, return NULL; vq->packed_ring = false; - vq->vq.callback = callback; + vq->vq.callback = cfg_vq_val(cfg, callbacks); vq->vq.vdev = vdev; - vq->vq.name = name; + vq->vq.name = cfg_vq_val(cfg, names); vq->vq.index = index; vq->vq.reset = false; vq->we_own_ring = false; - vq->notify = notify; - vq->weak_barriers = weak_barriers; + vq->notify = tp_cfg->notify; + vq->weak_barriers = tp_cfg->weak_barriers; #ifdef CONFIG_VIRTIO_HARDEN_NOTIFICATION vq->broken = true; #else vq->broken = false; #endif - vq->dma_dev = dma_dev; + vq->dma_dev = tp_cfg->dma_dev; vq->use_dma_api = vring_use_dma_api(vdev); vq->premapped = false; vq->indirect = virtio_has_feature(vdev, VIRTIO_RING_F_INDIRECT_DESC) && - !context; + !cfg_vq_get(cfg, ctx); vq->event = virtio_has_feature(vdev, VIRTIO_RING_F_EVENT_IDX); if (virtio_has_feature(vdev, VIRTIO_F_ORDER_PLATFORM)) @@ -2734,36 +2719,10 @@ struct virtqueue *vring_create_virtqueue(struct virtio_device *vdev, struct vq_transport_config *tp_cfg, struct virtio_vq_config *cfg) { - struct device *dma_dev; - unsigned int num; - unsigned int vring_align; - bool weak_barriers; - bool may_reduce_num; - bool context; - bool (*notify)(struct virtqueue *_); - void (*callback)(struct virtqueue *_); - const char *name; - - dma_dev = tp_cfg->dma_dev ? : vdev->dev.parent; - - num = tp_cfg->num; - vring_align = tp_cfg->vring_align; - weak_barriers = tp_cfg->weak_barriers; - may_reduce_num = tp_cfg->may_reduce_num; - notify = tp_cfg->notify; - - name = cfg->names[cfg->cfg_idx]; - callback = cfg->callbacks[cfg->cfg_idx]; - context = cfg->ctx ? cfg->ctx[cfg->cfg_idx] : false; - if (virtio_has_feature(vdev, VIRTIO_F_RING_PACKED)) - return vring_create_virtqueue_packed(index, num, vring_align, - vdev, weak_barriers, may_reduce_num, - context, notify, callback, name, dma_dev); + return vring_create_virtqueue_packed(vdev, index, tp_cfg, cfg); - return vring_create_virtqueue_split(index, num, vring_align, - vdev, weak_barriers, may_reduce_num, - context, notify, callback, name, dma_dev); + return vring_create_virtqueue_split(vdev, index, tp_cfg, cfg); } EXPORT_SYMBOL_GPL(vring_create_virtqueue); @@ -2916,30 +2875,14 @@ struct virtqueue *vring_new_virtqueue(struct virtio_device *vdev, struct virtio_vq_config *cfg) { struct vring_virtqueue_split vring_split = {}; - unsigned int num; - unsigned int vring_align; - bool weak_barriers; - bool context; - bool (*notify)(struct virtqueue *_); - void (*callback)(struct virtqueue *_); - const char *name; - - num = tp_cfg->num; - vring_align = tp_cfg->vring_align; - weak_barriers = tp_cfg->weak_barriers; - notify = tp_cfg->notify; - - name = cfg->names[cfg->cfg_idx]; - callback = cfg->callbacks[cfg->cfg_idx]; - context = cfg->ctx ? cfg->ctx[cfg->cfg_idx] : false; if (virtio_has_feature(vdev, VIRTIO_F_RING_PACKED)) return NULL; - vring_init(&vring_split.vring, num, pages, vring_align); - return __vring_new_virtqueue(index, &vring_split, vdev, weak_barriers, - context, notify, callback, name, - vdev->dev.parent); + tp_cfg->dma_dev = vdev->dev.parent; + + vring_init(&vring_split.vring, tp_cfg->num, pages, tp_cfg->vring_align); + return __vring_new_virtqueue(vdev, index, &vring_split, tp_cfg, cfg); } EXPORT_SYMBOL_GPL(vring_new_virtqueue); From patchwork Fri Feb 23 08:27:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 13568653 Received: from out30-110.freemail.mail.aliyun.com (out30-110.freemail.mail.aliyun.com [115.124.30.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 066733C47E; Fri, 23 Feb 2024 08:27:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.110 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708676870; cv=none; b=BDRZoKwXONtkiPPDKNYSiOtVG8yvgetewIy8bWSWt1Wx1fw9vrdS0PkJfUXEWW7kD7Jf9lg3VF2z4oYfWpznmVlyNUfCASOwLRua6EUnZSeDmVpbdf8j8rFzyJdbm0taQ6+nFFpJI24oUio6pYlAUxABOva0dav2HnyCnlafPE0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708676870; c=relaxed/simple; bh=edTitATIxyHSEiW2oo+NUoQMmO0I0Hk4LOfcTXSkfBM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=sNkQylK76a/obuGi39CopEXZpyuMt+lLznHTxBsccvxCctyltT+oCan/uCBbfN6mdZ4r1tjR+6y6aiFTdH7ZHA2DqMpyKGQrmc1ZNMjfo/fj4WiJETMP0P4r+rMaO47TGeSN+h+Ke+iIvcITf/lWMb7oUdUMF425/xRFOHKAlow= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=bN+bZmwF; arc=none smtp.client-ip=115.124.30.110 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="bN+bZmwF" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1708676866; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=INlx7TRy5b87EvD4imFkjgGo8erFnsWbzmY9PKIXOKQ=; b=bN+bZmwFfPCACNw/F7bD+xP1D03718Rd00fWSJsX19zyR4s4ZXD2Pt/U84I0Rqzen0yJCwNj+vKXhl19cR9f5AWuGv6CrNXb9dR54XkNdS5c/WjfLWyEebGxmKk4M7lZlVnKy2siq7VoqSZvPQ9IZ/qpL17ko2rocf0TBdjaido= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R151e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045170;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=35;SR=0;TI=SMTPD_---0W13mV24_1708676864; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0W13mV24_1708676864) by smtp.aliyun-inc.com; Fri, 23 Feb 2024 16:27:44 +0800 From: Xuan Zhuo To: virtualization@lists.linux.dev Cc: Richard Weinberger , Anton Ivanov , Johannes Berg , "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Hans de Goede , =?utf-8?q?Ilpo_J=C3=A4rvinen?= , Vadim Pasternak , Bjorn Andersson , Mathieu Poirier , Cornelia Huck , Halil Pasic , Eric Farman , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , linux-um@lists.infradead.org, netdev@vger.kernel.org, platform-driver-x86@vger.kernel.org, linux-remoteproc@vger.kernel.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH vhost v2 11/19] virtio: find_vqs: add new parameter premapped Date: Fri, 23 Feb 2024 16:27:18 +0800 Message-Id: <20240223082726.52915-12-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.32.0.3.g01195cf9f In-Reply-To: <20240223082726.52915-1-xuanzhuo@linux.alibaba.com> References: <20240223082726.52915-1-xuanzhuo@linux.alibaba.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Git-Hash: 510995f33855 If the premapped mode is enabled, the dma array(struct vring_desc_dma) of virtio core will not be allocated. That is judged when find_vqs() is called. To avoid allocating dma array in find_vqs() and releasing it immediately by virtqueue_set_dma_premapped(). This patch introduces a new parameter to find_vqs(). Then we can judge should we allocate the dma array(struct vring_desc_dma) or not inside find_vqs(). The driver must check the premapped mode of every vq after find_vqs(). Signed-off-by: Xuan Zhuo --- drivers/virtio/virtio_ring.c | 4 ++-- include/linux/virtio_config.h | 1 + 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index bbd79fd52885..dade5c130329 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -2145,7 +2145,7 @@ static struct virtqueue *vring_create_virtqueue_packed(struct virtio_device *vde vq->packed_ring = true; vq->dma_dev = dma_dev; vq->use_dma_api = vring_use_dma_api(vdev); - vq->premapped = false; + vq->premapped = vq->use_dma_api && cfg_vq_get(cfg, premapped); vq->indirect = virtio_has_feature(vdev, VIRTIO_RING_F_INDIRECT_DESC) && !cfg_vq_get(cfg, ctx); @@ -2687,7 +2687,7 @@ static struct virtqueue *__vring_new_virtqueue(struct virtio_device *vdev, #endif vq->dma_dev = tp_cfg->dma_dev; vq->use_dma_api = vring_use_dma_api(vdev); - vq->premapped = false; + vq->premapped = vq->use_dma_api && cfg_vq_get(cfg, premapped); vq->indirect = virtio_has_feature(vdev, VIRTIO_RING_F_INDIRECT_DESC) && !cfg_vq_get(cfg, ctx); diff --git a/include/linux/virtio_config.h b/include/linux/virtio_config.h index d47188303d34..f1f62e57f395 100644 --- a/include/linux/virtio_config.h +++ b/include/linux/virtio_config.h @@ -107,6 +107,7 @@ struct virtio_vq_config { vq_callback_t **callbacks; const char **names; const bool *ctx; + const bool *premapped; struct irq_affinity *desc; }; From patchwork Fri Feb 23 08:27:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 13568657 Received: from out30-112.freemail.mail.aliyun.com (out30-112.freemail.mail.aliyun.com [115.124.30.112]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3CA8146521; Fri, 23 Feb 2024 08:27:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.112 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708676879; cv=none; b=pIDSxMtZkGeoEXvj37vIvNWiOhh5a2YEK3FlGgYv0O/kbYgDcpHC8IODfOx3mIHIPhldQk7XVNm3aswhYi6j1NJuNO5seNoKN7rcb0ziFccHtmnGsgla9wNyzEqdpBbhbKG487/2qHi6ifVl6onG84WacOF2m1YRslJVqHZvavE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708676879; c=relaxed/simple; bh=RHe/EfTN9P/bV6jmDQ3rRmoOKzZKBGuAcvXx//8rB+E=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=u3HQPO9jRyT2/3BlhbWzX+WK1HTsJMdYVSDYZLH8te53gwCeEssP/xKgK+S/qsJZzemnMN5tKCzFag+DdaKfXhxAKwNVndvUDkuT3YGsXJUwkjLDSV9ujWMyXnc9swJh5UkxXaTnhY7dq3ng9JYsniyBqkyfIYF+2FIOznxYSH8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=YyShm7MF; arc=none smtp.client-ip=115.124.30.112 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="YyShm7MF" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1708676868; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=nhDsd+wn0QBsofx/3qFq8QiR22mUJhxgqbGJMfSse1s=; b=YyShm7MFb/2G/O6Gm9jo6KGi+TZseCXpcrUgutAbMe4HGqLwYSmQuYvl0k+aMHKhkCXVrwPZjCCG/0rw8rWOgT5Mk7Rfe7xN9EVTh4MW0l0WMH2t44wlJe79SU3ubbFLwUQvS22E2yv+njO+igwvmVLYi4TNOUIlAIDmU6s+Xqs= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R581e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045168;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=35;SR=0;TI=SMTPD_---0W13gV9r_1708676865; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0W13gV9r_1708676865) by smtp.aliyun-inc.com; Fri, 23 Feb 2024 16:27:46 +0800 From: Xuan Zhuo To: virtualization@lists.linux.dev Cc: Richard Weinberger , Anton Ivanov , Johannes Berg , "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Hans de Goede , =?utf-8?q?Ilpo_J=C3=A4rvinen?= , Vadim Pasternak , Bjorn Andersson , Mathieu Poirier , Cornelia Huck , Halil Pasic , Eric Farman , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , linux-um@lists.infradead.org, netdev@vger.kernel.org, platform-driver-x86@vger.kernel.org, linux-remoteproc@vger.kernel.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH vhost v2 12/19] virtio_ring: export premapped to driver by struct virtqueue Date: Fri, 23 Feb 2024 16:27:19 +0800 Message-Id: <20240223082726.52915-13-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.32.0.3.g01195cf9f In-Reply-To: <20240223082726.52915-1-xuanzhuo@linux.alibaba.com> References: <20240223082726.52915-1-xuanzhuo@linux.alibaba.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Git-Hash: 510995f33855 Export the premapped to drivers, then drivers can check the vq premapped mode after the find_vqs(). Because the find_vqs() just try to enable the vq premapped mode, the driver must check that after find_vqs(). Signed-off-by: Xuan Zhuo --- drivers/virtio/virtio_ring.c | 13 +++++-------- include/linux/virtio.h | 1 + 2 files changed, 6 insertions(+), 8 deletions(-) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index dade5c130329..103b06574e8b 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -191,9 +191,6 @@ struct vring_virtqueue { /* Host publishes avail event idx */ bool event; - /* Do DMA mapping by driver */ - bool premapped; - /* Head of free buffer list. */ unsigned int free_head; /* Number we've added since last sync. */ @@ -311,7 +308,7 @@ static bool vring_use_dma_api(const struct virtio_device *vdev) static bool vring_need_unmap_buffer(const struct vring_virtqueue *vring) { - return vring->use_dma_api && !vring->premapped; + return vring->use_dma_api && !vring->vq.premapped; } size_t virtio_max_dma_size(const struct virtio_device *vdev) @@ -383,7 +380,7 @@ static struct device *vring_dma_dev(const struct vring_virtqueue *vq) static int vring_map_one_sg(const struct vring_virtqueue *vq, struct scatterlist *sg, enum dma_data_direction direction, dma_addr_t *addr) { - if (vq->premapped) { + if (vq->vq.premapped) { *addr = sg_dma_address(sg); return 0; } @@ -2145,7 +2142,7 @@ static struct virtqueue *vring_create_virtqueue_packed(struct virtio_device *vde vq->packed_ring = true; vq->dma_dev = dma_dev; vq->use_dma_api = vring_use_dma_api(vdev); - vq->premapped = vq->use_dma_api && cfg_vq_get(cfg, premapped); + vq->vq.premapped = vq->use_dma_api && cfg_vq_get(cfg, premapped); vq->indirect = virtio_has_feature(vdev, VIRTIO_RING_F_INDIRECT_DESC) && !cfg_vq_get(cfg, ctx); @@ -2687,7 +2684,7 @@ static struct virtqueue *__vring_new_virtqueue(struct virtio_device *vdev, #endif vq->dma_dev = tp_cfg->dma_dev; vq->use_dma_api = vring_use_dma_api(vdev); - vq->premapped = vq->use_dma_api && cfg_vq_get(cfg, premapped); + vq->vq.premapped = vq->use_dma_api && cfg_vq_get(cfg, premapped); vq->indirect = virtio_has_feature(vdev, VIRTIO_RING_F_INDIRECT_DESC) && !cfg_vq_get(cfg, ctx); @@ -2818,7 +2815,7 @@ int virtqueue_set_dma_premapped(struct virtqueue *_vq) return -EINVAL; } - vq->premapped = true; + vq->vq.premapped = true; if (vq->packed_ring) { kfree(vq->packed.desc_dma); diff --git a/include/linux/virtio.h b/include/linux/virtio.h index b0201747a263..407277d5a16b 100644 --- a/include/linux/virtio.h +++ b/include/linux/virtio.h @@ -36,6 +36,7 @@ struct virtqueue { unsigned int num_free; unsigned int num_max; bool reset; + bool premapped; void *priv; }; From patchwork Fri Feb 23 08:27:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 13568654 X-Patchwork-Delegate: kuba@kernel.org Received: from out30-99.freemail.mail.aliyun.com (out30-99.freemail.mail.aliyun.com [115.124.30.99]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1306918E10; Fri, 23 Feb 2024 08:27:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.99 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708676875; cv=none; b=RRsi5rr6HSg4x1OlOqEG3XUXn3JN4QDOaGZAacHlkQ8d+gIeQmh89BDGnTcTqMzzS6yy9Xws96p/N7IDusmpjBBC6H5ajO0oVO5pH03UWbwICIJhcX1U3PxTf0NDNnjYP+A5ilRu1dONOkX2fN3DvOzy9+xym87Kn7bHz0hnPjE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708676875; c=relaxed/simple; bh=MUO8U5SVj3P1Xil/ahjJ/yQETxekSikLccDlysDuyiA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Iu7nBnbb2vgF88Is8TEQHnPB0Jl72/qZ0NlLHfLdg8KkIaRvu7RwxMfSbO+mLiE8GKOsK9VJ1TujR9TFGvgTcf529uD946m8pjVNRUleHh/oCJIbVl06zF1XSPoodHJgh4dh5ts/MfkY6Eomc5mMwU0fdxGBE1dsMhQKlK5dPGg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=t1LNZosJ; arc=none smtp.client-ip=115.124.30.99 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="t1LNZosJ" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1708676869; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=IRSlxqMw0yhAkJ08APM/0uZy3bVrhWlhvpCjnaXaq8Y=; b=t1LNZosJHBLCIZeeUifChXubUsuLD+pu41mOziZXz+utapTHViHq5oxiMLcvlXUf4KOi9nw19/LrGpOTzO1DsLt2Ek0gFAyyKYJIPo++g+cVt6MY6e55YWFSlkUwrgobaxDrpy1mwOuaSNv9m6Dn9GHhn/1QacMGKcglrSMQbVU= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R141e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045192;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=35;SR=0;TI=SMTPD_---0W13ndOb_1708676867; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0W13ndOb_1708676867) by smtp.aliyun-inc.com; Fri, 23 Feb 2024 16:27:47 +0800 From: Xuan Zhuo To: virtualization@lists.linux.dev Cc: Richard Weinberger , Anton Ivanov , Johannes Berg , "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Hans de Goede , =?utf-8?q?Ilpo_J=C3=A4rvinen?= , Vadim Pasternak , Bjorn Andersson , Mathieu Poirier , Cornelia Huck , Halil Pasic , Eric Farman , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , linux-um@lists.infradead.org, netdev@vger.kernel.org, platform-driver-x86@vger.kernel.org, linux-remoteproc@vger.kernel.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH vhost v2 13/19] virtio_net: set premapped mode by find_vqs() Date: Fri, 23 Feb 2024 16:27:20 +0800 Message-Id: <20240223082726.52915-14-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.32.0.3.g01195cf9f In-Reply-To: <20240223082726.52915-1-xuanzhuo@linux.alibaba.com> References: <20240223082726.52915-1-xuanzhuo@linux.alibaba.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Git-Hash: 510995f33855 X-Patchwork-Delegate: kuba@kernel.org Now, the virtio core can set the premapped mode by find_vqs(). If the premapped can be enabled, the dma array will not be allocated. So virtio-net use the api of find_vqs to enable the premapped. Judge the premapped mode by the vq->premapped instead of saving local variable. Signed-off-by: Xuan Zhuo --- drivers/net/virtio_net.c | 57 +++++++++++++++++------------------ include/linux/virtio_config.h | 16 ++-------- 2 files changed, 29 insertions(+), 44 deletions(-) diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index d7ce4a1011ea..140b2c155650 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -208,9 +208,6 @@ struct receive_queue { /* Record the last dma info to free after new pages is allocated. */ struct virtnet_rq_dma *last_dma; - - /* Do dma by self */ - bool do_dma; }; /* This structure can contain rss message with maximum settings for indirection table and keysize @@ -657,7 +654,7 @@ static void *virtnet_rq_get_buf(struct receive_queue *rq, u32 *len, void **ctx) void *buf; buf = virtqueue_get_buf_ctx(rq->vq, len, ctx); - if (buf && rq->do_dma) + if (buf && rq->vq->premapped) virtnet_rq_unmap(rq, buf, *len); return buf; @@ -670,7 +667,7 @@ static void virtnet_rq_init_one_sg(struct receive_queue *rq, void *buf, u32 len) u32 offset; void *head; - if (!rq->do_dma) { + if (!rq->vq->premapped) { sg_init_one(rq->sg, buf, len); return; } @@ -700,7 +697,7 @@ static void *virtnet_rq_alloc(struct receive_queue *rq, u32 size, gfp_t gfp) head = page_address(alloc_frag->page); - if (rq->do_dma) { + if (rq->vq->premapped) { dma = head; /* new pages */ @@ -746,22 +743,6 @@ static void *virtnet_rq_alloc(struct receive_queue *rq, u32 size, gfp_t gfp) return buf; } -static void virtnet_rq_set_premapped(struct virtnet_info *vi) -{ - int i; - - /* disable for big mode */ - if (!vi->mergeable_rx_bufs && vi->big_packets) - return; - - for (i = 0; i < vi->max_queue_pairs; i++) { - if (virtqueue_set_dma_premapped(vi->rq[i].vq)) - continue; - - vi->rq[i].do_dma = true; - } -} - static void virtnet_rq_unmap_free_buf(struct virtqueue *vq, void *buf) { struct virtnet_info *vi = vq->vdev->priv; @@ -770,7 +751,7 @@ static void virtnet_rq_unmap_free_buf(struct virtqueue *vq, void *buf) rq = &vi->rq[i]; - if (rq->do_dma) + if (rq->vq->premapped) virtnet_rq_unmap(rq, buf, 0); virtnet_rq_free_buf(vi, rq, buf); @@ -1865,7 +1846,7 @@ static int add_recvbuf_small(struct virtnet_info *vi, struct receive_queue *rq, err = virtqueue_add_inbuf_ctx(rq->vq, rq->sg, 1, buf, ctx, gfp); if (err < 0) { - if (rq->do_dma) + if (rq->vq->premapped) virtnet_rq_unmap(rq, buf, 0); put_page(virt_to_head_page(buf)); } @@ -1980,7 +1961,7 @@ static int add_recvbuf_mergeable(struct virtnet_info *vi, ctx = mergeable_len_to_ctx(len + room, headroom); err = virtqueue_add_inbuf_ctx(rq->vq, rq->sg, 1, buf, ctx, gfp); if (err < 0) { - if (rq->do_dma) + if (rq->vq->premapped) virtnet_rq_unmap(rq, buf, 0); put_page(virt_to_head_page(buf)); } @@ -4229,7 +4210,7 @@ static void free_receive_page_frags(struct virtnet_info *vi) int i; for (i = 0; i < vi->max_queue_pairs; i++) if (vi->rq[i].alloc_frag.page) { - if (vi->rq[i].do_dma && vi->rq[i].last_dma) + if (vi->rq[i].vq->premapped && vi->rq[i].last_dma) virtnet_rq_unmap(&vi->rq[i], vi->rq[i].last_dma, 0); put_page(vi->rq[i].alloc_frag.page); } @@ -4293,11 +4274,13 @@ static unsigned int mergeable_min_buf_len(struct virtnet_info *vi, struct virtqu static int virtnet_find_vqs(struct virtnet_info *vi) { + struct virtio_vq_config cfg = {}; vq_callback_t **callbacks; struct virtqueue **vqs; const char **names; int ret = -ENOMEM; int total_vqs; + bool *premapped; bool *ctx; u16 i; @@ -4322,8 +4305,13 @@ static int virtnet_find_vqs(struct virtnet_info *vi) ctx = kcalloc(total_vqs, sizeof(*ctx), GFP_KERNEL); if (!ctx) goto err_ctx; + + premapped = kcalloc(total_vqs, sizeof(*premapped), GFP_KERNEL); + if (!ctx) + goto err_premapped; } else { ctx = NULL; + premapped = NULL; } /* Parameters for control virtqueue, if any */ @@ -4342,10 +4330,19 @@ static int virtnet_find_vqs(struct virtnet_info *vi) names[txq2vq(i)] = vi->sq[i].name; if (ctx) ctx[rxq2vq(i)] = true; + + if (premapped) + premapped[rxq2vq(i)] = true; } - ret = virtio_find_vqs_ctx(vi->vdev, total_vqs, vqs, callbacks, - names, ctx, NULL); + cfg.nvqs = total_vqs; + cfg.vqs = vqs; + cfg.callbacks = callbacks; + cfg.names = names; + cfg.ctx = ctx; + cfg.premapped = premapped; + + ret = virtio_find_vqs_cfg(vi->vdev, &cfg); if (ret) goto err_find; @@ -4365,6 +4362,8 @@ static int virtnet_find_vqs(struct virtnet_info *vi) err_find: + kfree(premapped); +err_premapped: kfree(ctx); err_ctx: kfree(names); @@ -4437,8 +4436,6 @@ static int init_vqs(struct virtnet_info *vi) if (ret) goto err_free; - virtnet_rq_set_premapped(vi); - cpus_read_lock(); virtnet_set_affinity(vi); cpus_read_unlock(); diff --git a/include/linux/virtio_config.h b/include/linux/virtio_config.h index f1f62e57f395..e40509fef5fe 100644 --- a/include/linux/virtio_config.h +++ b/include/linux/virtio_config.h @@ -260,21 +260,9 @@ int virtio_find_vqs(struct virtio_device *vdev, unsigned nvqs, } static inline -int virtio_find_vqs_ctx(struct virtio_device *vdev, unsigned nvqs, - struct virtqueue *vqs[], vq_callback_t *callbacks[], - const char * const names[], const bool *ctx, - struct irq_affinity *desc) +int virtio_find_vqs_cfg(struct virtio_device *vdev, struct virtio_vq_config *cfg) { - struct virtio_vq_config cfg = {}; - - cfg.nvqs = nvqs; - cfg.vqs = vqs; - cfg.callbacks = callbacks; - cfg.names = (const char **)names; - cfg.ctx = ctx; - cfg.desc = desc; - - return vdev->config->find_vqs(vdev, &cfg); + return vdev->config->find_vqs(vdev, cfg); } /** From patchwork Fri Feb 23 08:27:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 13568659 Received: from out30-112.freemail.mail.aliyun.com (out30-112.freemail.mail.aliyun.com [115.124.30.112]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DEF6B48CDD; Fri, 23 Feb 2024 08:27:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.112 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708676882; cv=none; b=aYqNSXuMqz5W7n2/OuSTxY2p1QlD/zaCCZQZGmQ/5O2RqZIwttE/62/mMzzReGByCORpGk/4Wpm/+r3ADOVpJya8hmYDsq794XDRRwl8A6ZQHN9QQpcXDnbGZf8PM1i5A1EebWQrKdn6nd/vvUxGBjaOmH3ZhRTUJLZIioIy0xc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708676882; c=relaxed/simple; bh=pjonLh8exRbdrHMa17jNGBBGqb3zvIrKACq7Hq9u+NM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=k0/D2ZXCY34qooD/5Vp8chimdrUV4SKDiMaqaCKTPvbMAXWtJCp5O6233HRoS+ZIw5uzo31EVlGkk66EFWFc2RDD3/pJPUY1R6RKv/RrDxhhPaYz+zk4qbf41t+ezkmn24M3bbZUCDCEcxZoBHRpbVWybCGLlV6BmvfC5FTYX/A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=TRBKIejS; arc=none smtp.client-ip=115.124.30.112 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="TRBKIejS" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1708676872; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=MiRsah0os+XN0E37SyZpCu/ggHFN/seKkb4xKfCwkhI=; b=TRBKIejSch6PQgAAAIWInrgB9pcqqGFLPO+D+N42zZaTEQtonpsCQAZrHUe6MNyPLW0sAPzUe07/1Z+4VHwCF2pliK2RfgmN7dfQrlXXDlFTxczdbrESAjudIJtTYZX1O9VEo7QMC7qvDS5ujpZcSSQEiDswWTLwuv/QwNNDLEk= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R211e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045170;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=35;SR=0;TI=SMTPD_---0W13ndPZ_1708676868; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0W13ndPZ_1708676868) by smtp.aliyun-inc.com; Fri, 23 Feb 2024 16:27:49 +0800 From: Xuan Zhuo To: virtualization@lists.linux.dev Cc: Richard Weinberger , Anton Ivanov , Johannes Berg , "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Hans de Goede , =?utf-8?q?Ilpo_J=C3=A4rvinen?= , Vadim Pasternak , Bjorn Andersson , Mathieu Poirier , Cornelia Huck , Halil Pasic , Eric Farman , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , linux-um@lists.infradead.org, netdev@vger.kernel.org, platform-driver-x86@vger.kernel.org, linux-remoteproc@vger.kernel.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH vhost v2 14/19] virtio_ring: remove api of setting vq premapped Date: Fri, 23 Feb 2024 16:27:21 +0800 Message-Id: <20240223082726.52915-15-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.32.0.3.g01195cf9f In-Reply-To: <20240223082726.52915-1-xuanzhuo@linux.alibaba.com> References: <20240223082726.52915-1-xuanzhuo@linux.alibaba.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Git-Hash: 510995f33855 The virtio-net sets the premapped mode by find_vqs(). So the below API is not used by anyone, we remove it. int virtqueue_set_dma_premapped(struct virtqueue *_vq); Signed-off-by: Xuan Zhuo --- drivers/virtio/virtio_ring.c | 56 ------------------------------------ include/linux/virtio.h | 2 -- 2 files changed, 58 deletions(-) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index 103b06574e8b..63084487c171 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -2775,62 +2775,6 @@ int virtqueue_resize(struct virtqueue *_vq, u32 num, } EXPORT_SYMBOL_GPL(virtqueue_resize); -/** - * virtqueue_set_dma_premapped - set the vring premapped mode - * @_vq: the struct virtqueue we're talking about. - * - * Enable the premapped mode of the vq. - * - * The vring in premapped mode does not do dma internally, so the driver must - * do dma mapping in advance. The driver must pass the dma_address through - * dma_address of scatterlist. When the driver got a used buffer from - * the vring, it has to unmap the dma address. - * - * This function must be called immediately after creating the vq, or after vq - * reset, and before adding any buffers to it. - * - * Caller must ensure we don't call this with other virtqueue operations - * at the same time (except where noted). - * - * Returns zero or a negative error. - * 0: success. - * -EINVAL: vring does not use the dma api, so we can not enable premapped mode. - */ -int virtqueue_set_dma_premapped(struct virtqueue *_vq) -{ - struct vring_virtqueue *vq = to_vvq(_vq); - u32 num; - - START_USE(vq); - - num = vq->packed_ring ? vq->packed.vring.num : vq->split.vring.num; - - if (num != vq->vq.num_free) { - END_USE(vq); - return -EINVAL; - } - - if (!vq->use_dma_api) { - END_USE(vq); - return -EINVAL; - } - - vq->vq.premapped = true; - - if (vq->packed_ring) { - kfree(vq->packed.desc_dma); - vq->packed.desc_dma = NULL; - } else { - kfree(vq->split.desc_dma); - vq->split.desc_dma = NULL; - } - - END_USE(vq); - - return 0; -} -EXPORT_SYMBOL_GPL(virtqueue_set_dma_premapped); - /** * virtqueue_reset - detach and recycle all unused buffers * @_vq: the struct virtqueue we're talking about. diff --git a/include/linux/virtio.h b/include/linux/virtio.h index 407277d5a16b..2167893313f6 100644 --- a/include/linux/virtio.h +++ b/include/linux/virtio.h @@ -82,8 +82,6 @@ bool virtqueue_enable_cb(struct virtqueue *vq); unsigned virtqueue_enable_cb_prepare(struct virtqueue *vq); -int virtqueue_set_dma_premapped(struct virtqueue *_vq); - bool virtqueue_poll(struct virtqueue *vq, unsigned); bool virtqueue_enable_cb_delayed(struct virtqueue *vq); From patchwork Fri Feb 23 08:27:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 13568655 Received: from out30-118.freemail.mail.aliyun.com (out30-118.freemail.mail.aliyun.com [115.124.30.118]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2616817BB7; Fri, 23 Feb 2024 08:27:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.118 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708676877; cv=none; b=U2f+xcdt6ES4DoJMqVVZyzmr/8GQ9Lh+gqchpR07S+IUiVnon3rZ0tvTVS+YEf979M1NQrhLKTpecNeK57WCPrjIQ9uhSOvNNiDNyyWyh8P8757GyB9u7C8woDJRWrVx60NkW/jafdvfiGWjKSL5SWgapriDWSjHBXd64M/MztI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708676877; c=relaxed/simple; bh=V8XsRpioepNKGf3y4dNcVe7pBihQ0P4Gbtoh+MTXVtE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=P3OZzRZ9kpstz/NVJ5Es9prWC1FoF9IwW8kSNM15o058ehOzRJA0k7VNjVaxy1334C/rAuwrbqq0lRuuWnN6z4nsR/8cKjbuEBRLHHzHNIkvHe253aFytT5N0sxeE26uBOkY00jGeSgR12zhtda98iAxg7aSMvHFBpzSyN+z1Tk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=Q1TgiiWQ; arc=none smtp.client-ip=115.124.30.118 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="Q1TgiiWQ" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1708676872; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=wuCB5vDA9pa0b+JuP11RCrcmeZ6iAyZ0CdJ1Q8cuTqA=; b=Q1TgiiWQBuzLLzrr1WpphhTbY/K1ul3pIXcMkhtNJfnfIcvQ2Pu87isgfjRoh47fwJHOTAteADdvpn8fsvRqOC3pDudOquNYxTNZWwh41af5DXWLVQOAWIigaJwInIbZwHwGF0/pnzSqRs+nEqvjibpb7VIcoADP47YaaN9f6b0= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R161e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046051;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=35;SR=0;TI=SMTPD_---0W13UX5R_1708676869; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0W13UX5R_1708676869) by smtp.aliyun-inc.com; Fri, 23 Feb 2024 16:27:50 +0800 From: Xuan Zhuo To: virtualization@lists.linux.dev Cc: Richard Weinberger , Anton Ivanov , Johannes Berg , "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Hans de Goede , =?utf-8?q?Ilpo_J=C3=A4rvinen?= , Vadim Pasternak , Bjorn Andersson , Mathieu Poirier , Cornelia Huck , Halil Pasic , Eric Farman , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , linux-um@lists.infradead.org, netdev@vger.kernel.org, platform-driver-x86@vger.kernel.org, linux-remoteproc@vger.kernel.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH vhost v2 15/19] virtio_ring: introduce dma map api for page Date: Fri, 23 Feb 2024 16:27:22 +0800 Message-Id: <20240223082726.52915-16-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.32.0.3.g01195cf9f In-Reply-To: <20240223082726.52915-1-xuanzhuo@linux.alibaba.com> References: <20240223082726.52915-1-xuanzhuo@linux.alibaba.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Git-Hash: 510995f33855 The virtio-net sq will use these APIs to map the scatterlist. For scatterlist, the page dma APIs are more appropriate. dma_addr_t virtqueue_dma_map_page_attrs(struct virtqueue *_vq, struct page *page, size_t offset, size_t size, enum dma_data_direction dir, unsigned long attrs); void virtqueue_dma_unmap_page_attrs(struct virtqueue *_vq, dma_addr_t addr, size_t size, enum dma_data_direction dir, unsigned long attrs); Signed-off-by: Xuan Zhuo --- drivers/virtio/virtio_ring.c | 52 ++++++++++++++++++++++++++++++++++++ include/linux/virtio.h | 7 +++++ 2 files changed, 59 insertions(+) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index 63084487c171..df7633155558 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -3117,6 +3117,58 @@ void virtqueue_dma_unmap_single_attrs(struct virtqueue *_vq, dma_addr_t addr, } EXPORT_SYMBOL_GPL(virtqueue_dma_unmap_single_attrs); +/** + * virtqueue_dma_map_page_attrs - map DMA for _vq + * @_vq: the struct virtqueue we're talking about. + * @page: the page to do dma + * @offset: the offset inside the page + * @size: the size of the page to do dma + * @dir: DMA direction + * @attrs: DMA Attrs + * + * The caller calls this to do dma mapping in advance. The DMA address can be + * passed to this _vq when it is in pre-mapped mode. + * + * return DMA address. Caller should check that by virtqueue_dma_mapping_error(). + */ +dma_addr_t virtqueue_dma_map_page_attrs(struct virtqueue *_vq, struct page *page, + size_t offset, size_t size, + enum dma_data_direction dir, + unsigned long attrs) +{ + struct vring_virtqueue *vq = to_vvq(_vq); + + if (!vq->use_dma_api) + return page_to_phys(page) + offset; + + return dma_map_page_attrs(vring_dma_dev(vq), page, offset, size, dir, attrs); +} +EXPORT_SYMBOL_GPL(virtqueue_dma_map_page_attrs); + +/** + * virtqueue_dma_unmap_page_attrs - unmap DMA for _vq + * @_vq: the struct virtqueue we're talking about. + * @addr: the dma address to unmap + * @size: the size of the buffer + * @dir: DMA direction + * @attrs: DMA Attrs + * + * Unmap the address that is mapped by the virtqueue_dma_map_* APIs. + * + */ +void virtqueue_dma_unmap_page_attrs(struct virtqueue *_vq, dma_addr_t addr, + size_t size, enum dma_data_direction dir, + unsigned long attrs) +{ + struct vring_virtqueue *vq = to_vvq(_vq); + + if (!vq->use_dma_api) + return; + + dma_unmap_page_attrs(vring_dma_dev(vq), addr, size, dir, attrs); +} +EXPORT_SYMBOL_GPL(virtqueue_dma_unmap_page_attrs); + /** * virtqueue_dma_mapping_error - check dma address * @_vq: the struct virtqueue we're talking about. diff --git a/include/linux/virtio.h b/include/linux/virtio.h index 2167893313f6..b81b93b3eb27 100644 --- a/include/linux/virtio.h +++ b/include/linux/virtio.h @@ -224,6 +224,13 @@ dma_addr_t virtqueue_dma_map_single_attrs(struct virtqueue *_vq, void *ptr, size void virtqueue_dma_unmap_single_attrs(struct virtqueue *_vq, dma_addr_t addr, size_t size, enum dma_data_direction dir, unsigned long attrs); +dma_addr_t virtqueue_dma_map_page_attrs(struct virtqueue *_vq, struct page *page, + size_t offset, size_t size, + enum dma_data_direction dir, + unsigned long attrs); +void virtqueue_dma_unmap_page_attrs(struct virtqueue *_vq, dma_addr_t addr, + size_t size, enum dma_data_direction dir, + unsigned long attrs); int virtqueue_dma_mapping_error(struct virtqueue *_vq, dma_addr_t addr); bool virtqueue_dma_need_sync(struct virtqueue *_vq, dma_addr_t addr); From patchwork Fri Feb 23 08:27:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 13568658 Received: from out30-98.freemail.mail.aliyun.com (out30-98.freemail.mail.aliyun.com [115.124.30.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2B38546450; Fri, 23 Feb 2024 08:27:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.98 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708676880; cv=none; b=El4/N+EHxZCEH5OgUEE3Cbpo4/WMF4rlqZ6rWsJqjjeliPxiakpNvXSSgI3WOso939ETciWd/BJZoSncPrKY4I2+yVq1muyIrvb1LeHgFlR6wyeCfmRpcwpXBen5Mfo3S2cpcDka0UVyk6IA43hO5iySzRzqsE3ZFXgFbkDMTQA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708676880; c=relaxed/simple; bh=nufMRbXLj+YSuSWmljSpMKFk1LTW93GH2wujXElJyos=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Pwt95NDEqsFjPNvJsv3ZpZTIyGt0HymAIhXP2EQfIkG3sPzDNhvhWLX53iNAB7Is5qdqrLLvSxIoJNSKA2HHsMQj8UlWOwJIEAHYhCO7pRNGyBTjTBjpg9u2GO7yrvz8+Rd5X2GN/LHtsbm7uhaaeCvHktl3uZoocRgRKkJveGk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=k2jgJx0w; arc=none smtp.client-ip=115.124.30.98 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="k2jgJx0w" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1708676874; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=d8VgurjiHoufBNHjBHr2ztXStBXU+RItFeIKBycQigI=; b=k2jgJx0wTKh22ktOdbDtoHvLiNlH2OR1X3oXOr142XHFAX3d6XsvfTuAwyIfdKTQGlSPlrkwpumDtp9Gldfl8C7Yb95G1bZ5nQ46IAx/ZpD6pfoQL9n/tYR8bVje0FAS9IvaWEC47NHJv0/mTFi3zFyiRGKpvIWLTfv/IB1WSWQ= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R181e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046049;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=35;SR=0;TI=SMTPD_---0W13gVCV_1708676871; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0W13gVCV_1708676871) by smtp.aliyun-inc.com; Fri, 23 Feb 2024 16:27:52 +0800 From: Xuan Zhuo To: virtualization@lists.linux.dev Cc: Richard Weinberger , Anton Ivanov , Johannes Berg , "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Hans de Goede , =?utf-8?q?Ilpo_J=C3=A4rvinen?= , Vadim Pasternak , Bjorn Andersson , Mathieu Poirier , Cornelia Huck , Halil Pasic , Eric Farman , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , linux-um@lists.infradead.org, netdev@vger.kernel.org, platform-driver-x86@vger.kernel.org, linux-remoteproc@vger.kernel.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH vhost v2 16/19] virtio_ring: introduce virtqueue_dma_map_sg_attrs Date: Fri, 23 Feb 2024 16:27:23 +0800 Message-Id: <20240223082726.52915-17-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.32.0.3.g01195cf9f In-Reply-To: <20240223082726.52915-1-xuanzhuo@linux.alibaba.com> References: <20240223082726.52915-1-xuanzhuo@linux.alibaba.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Git-Hash: 510995f33855 Introduce a helper to do dma map for scatterlist. That can be used by other drivers. Signed-off-by: Xuan Zhuo --- drivers/virtio/virtio_ring.c | 32 ++++++++++++++++++++++++++++++++ include/linux/virtio.h | 3 +++ 2 files changed, 35 insertions(+) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index df7633155558..3f790fa3d56a 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -3187,6 +3187,38 @@ int virtqueue_dma_mapping_error(struct virtqueue *_vq, dma_addr_t addr) } EXPORT_SYMBOL_GPL(virtqueue_dma_mapping_error); +/** + * virtqueue_dma_map_sg_attrs - map scatterlist addr DMA for _vq + * @_vq: the struct virtqueue we're talking about. + * @sg: the scatterlist to do dma + * @dir: DMA direction + * @attrs: DMA Attrs + * + * The caller calls this to do dma mapping in advance. The sg can be + * passed to this _vq when it is in pre-mapped mode. + * + * Returns zero or a negative error. + * 0: success + * -ENOMEM: dma map error + */ +int virtqueue_dma_map_sg_attrs(struct virtqueue *_vq, struct scatterlist *sg, + enum dma_data_direction dir, unsigned long attrs) +{ + dma_addr_t addr; + int err; + + addr = virtqueue_dma_map_page_attrs(_vq, sg_page(sg), sg->offset, + sg->length, dir, attrs); + err = virtqueue_dma_mapping_error(_vq, addr); + if (err) + return err; + + sg->dma_address = addr; + + return 0; +} +EXPORT_SYMBOL_GPL(virtqueue_dma_map_sg_attrs); + /** * virtqueue_dma_need_sync - check a dma address needs sync * @_vq: the struct virtqueue we're talking about. diff --git a/include/linux/virtio.h b/include/linux/virtio.h index b81b93b3eb27..5027eba82fda 100644 --- a/include/linux/virtio.h +++ b/include/linux/virtio.h @@ -233,6 +233,9 @@ void virtqueue_dma_unmap_page_attrs(struct virtqueue *_vq, dma_addr_t addr, unsigned long attrs); int virtqueue_dma_mapping_error(struct virtqueue *_vq, dma_addr_t addr); +int virtqueue_dma_map_sg_attrs(struct virtqueue *_vq, struct scatterlist *sg, + enum dma_data_direction dir, unsigned long attrs); + bool virtqueue_dma_need_sync(struct virtqueue *_vq, dma_addr_t addr); void virtqueue_dma_sync_single_range_for_cpu(struct virtqueue *_vq, dma_addr_t addr, unsigned long offset, size_t size, From patchwork Fri Feb 23 08:27:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 13568682 X-Patchwork-Delegate: kuba@kernel.org Received: from out30-113.freemail.mail.aliyun.com (out30-113.freemail.mail.aliyun.com [115.124.30.113]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 971764CDE5; Fri, 23 Feb 2024 08:28:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.113 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708676885; cv=none; b=tI3HOHUY83OZUkfQPU/lyyS6z1cVzNhaa+aa9yqSqNBJ84TmPwMrthh+uPbY8OlqYvsus63qyhVSITsbLut9EI++fcmQnlEGrK3jxzNjuWJTtYw5G2MazCoYjhuYEOXCuGqaQVYZ0zqJyGaJIAqDRC3hc8NPQbJLcHIr2KfT2eU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708676885; c=relaxed/simple; bh=yPM5tBpfJQI8Ju/RW0nFXCP/h+fVOud1sDDLqpPuNJI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=InBilwd1lieRLMslZbuB79DefeQT09LrW24jQDcM+PeK/OXtOEsk2UjBZQjbuPpSEZ/7BuMzozSTmFY2V5fVunXwrIIfVBXjcxgtErZYqi0upwA+JxnAEDx+zBfv48WdydxOpFNZMZNhWeeXXuoffmuyH8NLCObmpdCChjKN+ns= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=Ydsz84gL; arc=none smtp.client-ip=115.124.30.113 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="Ydsz84gL" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1708676875; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=5W/fyj7ubQoZEKftTvGZ0AfKf1xx3NzxaJdw6hTkCm4=; b=Ydsz84gLZVY+uT5fNxJBzhyewfZCtIgh5RvyJJYBcwtIWTFnmvo/6YEKP6nz7YA3kZjAOe+7NZSktS9JfzLdeWgUQrdGN7UADgNdMx5UvWHbSESTdWRsuUIF04XfjScCaReNwoUGxW6fMNVbCknWTdyCR+dBvWfbaW+NpM5b558= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R361e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046059;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=35;SR=0;TI=SMTPD_---0W13gVDK_1708676873; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0W13gVDK_1708676873) by smtp.aliyun-inc.com; Fri, 23 Feb 2024 16:27:53 +0800 From: Xuan Zhuo To: virtualization@lists.linux.dev Cc: Richard Weinberger , Anton Ivanov , Johannes Berg , "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Hans de Goede , =?utf-8?q?Ilpo_J=C3=A4rvinen?= , Vadim Pasternak , Bjorn Andersson , Mathieu Poirier , Cornelia Huck , Halil Pasic , Eric Farman , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , linux-um@lists.infradead.org, netdev@vger.kernel.org, platform-driver-x86@vger.kernel.org, linux-remoteproc@vger.kernel.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH vhost v2 17/19] virtio_net: unify the code for recycling the xmit ptr Date: Fri, 23 Feb 2024 16:27:24 +0800 Message-Id: <20240223082726.52915-18-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.32.0.3.g01195cf9f In-Reply-To: <20240223082726.52915-1-xuanzhuo@linux.alibaba.com> References: <20240223082726.52915-1-xuanzhuo@linux.alibaba.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Git-Hash: 510995f33855 X-Patchwork-Delegate: kuba@kernel.org There are two completely similar and independent implementations. This is inconvenient for the subsequent addition of new types. So extract a function from this piece of code and call this function uniformly to recover old xmit ptr. Signed-off-by: Xuan Zhuo Acked-by: Jason Wang --- drivers/net/virtio_net.c | 82 +++++++++++++++++++--------------------- 1 file changed, 39 insertions(+), 43 deletions(-) diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index 140b2c155650..62f65e2cacd5 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -80,6 +80,11 @@ struct virtnet_stat_desc { size_t offset; }; +struct virtnet_sq_free_stats { + u64 packets; + u64 bytes; +}; + struct virtnet_sq_stats { struct u64_stats_sync syncp; u64_stats_t packets; @@ -363,6 +368,31 @@ static struct xdp_frame *ptr_to_xdp(void *ptr) return (struct xdp_frame *)((unsigned long)ptr & ~VIRTIO_XDP_FLAG); } +static void __free_old_xmit(struct send_queue *sq, bool in_napi, + struct virtnet_sq_free_stats *stats) +{ + unsigned int len; + void *ptr; + + while ((ptr = virtqueue_get_buf(sq->vq, &len)) != NULL) { + ++stats->packets; + + if (!is_xdp_frame(ptr)) { + struct sk_buff *skb = ptr; + + pr_debug("Sent skb %p\n", skb); + + stats->bytes += skb->len; + napi_consume_skb(skb, in_napi); + } else { + struct xdp_frame *frame = ptr_to_xdp(ptr); + + stats->bytes += xdp_get_frame_len(frame); + xdp_return_frame(frame); + } + } +} + /* Converting between virtqueue no. and kernel tx/rx queue no. * 0:rx0 1:tx0 2:rx1 3:tx1 ... 2N:rxN 2N+1:txN 2N+2:cvq */ @@ -759,37 +789,19 @@ static void virtnet_rq_unmap_free_buf(struct virtqueue *vq, void *buf) static void free_old_xmit_skbs(struct send_queue *sq, bool in_napi) { - unsigned int len; - unsigned int packets = 0; - unsigned int bytes = 0; - void *ptr; + struct virtnet_sq_free_stats stats = {0}; - while ((ptr = virtqueue_get_buf(sq->vq, &len)) != NULL) { - if (likely(!is_xdp_frame(ptr))) { - struct sk_buff *skb = ptr; - - pr_debug("Sent skb %p\n", skb); - - bytes += skb->len; - napi_consume_skb(skb, in_napi); - } else { - struct xdp_frame *frame = ptr_to_xdp(ptr); - - bytes += xdp_get_frame_len(frame); - xdp_return_frame(frame); - } - packets++; - } + __free_old_xmit(sq, in_napi, &stats); /* Avoid overhead when no packets have been processed * happens when called speculatively from start_xmit. */ - if (!packets) + if (!stats.packets) return; u64_stats_update_begin(&sq->stats.syncp); - u64_stats_add(&sq->stats.bytes, bytes); - u64_stats_add(&sq->stats.packets, packets); + u64_stats_add(&sq->stats.bytes, stats.bytes); + u64_stats_add(&sq->stats.packets, stats.packets); u64_stats_update_end(&sq->stats.syncp); } @@ -928,15 +940,12 @@ static int virtnet_xdp_xmit(struct net_device *dev, int n, struct xdp_frame **frames, u32 flags) { struct virtnet_info *vi = netdev_priv(dev); + struct virtnet_sq_free_stats stats = {0}; struct receive_queue *rq = vi->rq; struct bpf_prog *xdp_prog; struct send_queue *sq; - unsigned int len; - int packets = 0; - int bytes = 0; int nxmit = 0; int kicks = 0; - void *ptr; int ret; int i; @@ -955,20 +964,7 @@ static int virtnet_xdp_xmit(struct net_device *dev, } /* Free up any pending old buffers before queueing new ones. */ - while ((ptr = virtqueue_get_buf(sq->vq, &len)) != NULL) { - if (likely(is_xdp_frame(ptr))) { - struct xdp_frame *frame = ptr_to_xdp(ptr); - - bytes += xdp_get_frame_len(frame); - xdp_return_frame(frame); - } else { - struct sk_buff *skb = ptr; - - bytes += skb->len; - napi_consume_skb(skb, false); - } - packets++; - } + __free_old_xmit(sq, false, &stats); for (i = 0; i < n; i++) { struct xdp_frame *xdpf = frames[i]; @@ -988,8 +984,8 @@ static int virtnet_xdp_xmit(struct net_device *dev, } out: u64_stats_update_begin(&sq->stats.syncp); - u64_stats_add(&sq->stats.bytes, bytes); - u64_stats_add(&sq->stats.packets, packets); + u64_stats_add(&sq->stats.bytes, stats.bytes); + u64_stats_add(&sq->stats.packets, stats.packets); u64_stats_add(&sq->stats.xdp_tx, n); u64_stats_add(&sq->stats.xdp_tx_drops, n - nxmit); u64_stats_add(&sq->stats.kicks, kicks); From patchwork Fri Feb 23 08:27:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 13568660 X-Patchwork-Delegate: kuba@kernel.org Received: from out30-110.freemail.mail.aliyun.com (out30-110.freemail.mail.aliyun.com [115.124.30.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 04201495E5; Fri, 23 Feb 2024 08:27:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.110 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708676883; cv=none; b=pb/f5C6tpUMF4Gco+eCYZfqXV7flv7h2s5GRTen8WVugBUPnHmHSY2MLa2ZAtnSpTKIheng7OZvp6epUi3fepZv/gz+FTxLKHPp3f5KIJVzxRocx6iAzVFFSTxsP+0PzpMzsZHe88XVIxVNUKoIlTo0S11YdhpzZ9SVolMWJGvo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708676883; c=relaxed/simple; bh=0Gk9UztMGIIcRnEHpqYyAjKvdnBaolmfK4xirNAtcuc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Wzn7j6ea+8ccZJw2f8BbXdUnMUBGq8LHiVZP5Ptx0v3loGJQ8qghMReeawGUjj5+tc/2YxYfLHqO4a7l37MTAdCRp8sxQ5s4jSTpiNL2WxuLMEGRfePR0Kv8Tns28KpN0qeo9GK84f3gZuiJDPlbd+9L5AQnLe/GtPm3kxSyQQU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=e7B1649N; arc=none smtp.client-ip=115.124.30.110 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="e7B1649N" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1708676878; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=0bf+ondcixqn/GNrn6hzfTiAxwONS6eM86OVYrAkzbE=; b=e7B1649NeW3ZGohkuoRMzvhP15YFzMsDLlHL2aLDubuPNgLG8Hbg1gyzQvgC9LiCK72icEQVfBpV9RwdooQSeF/CgRzICj6xQHFdkwJU5EQUZWO1C16D3VBGA86/A5ueoWbtX810Sd9aZHfosx9eI+vqBK5WW86F6rVDjnUzEqM= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R181e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045176;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=35;SR=0;TI=SMTPD_---0W13ndSM_1708676874; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0W13ndSM_1708676874) by smtp.aliyun-inc.com; Fri, 23 Feb 2024 16:27:55 +0800 From: Xuan Zhuo To: virtualization@lists.linux.dev Cc: Richard Weinberger , Anton Ivanov , Johannes Berg , "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Hans de Goede , =?utf-8?q?Ilpo_J=C3=A4rvinen?= , Vadim Pasternak , Bjorn Andersson , Mathieu Poirier , Cornelia Huck , Halil Pasic , Eric Farman , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , linux-um@lists.infradead.org, netdev@vger.kernel.org, platform-driver-x86@vger.kernel.org, linux-remoteproc@vger.kernel.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH vhost v2 18/19] virtio_net: rename free_old_xmit_skbs to free_old_xmit Date: Fri, 23 Feb 2024 16:27:25 +0800 Message-Id: <20240223082726.52915-19-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.32.0.3.g01195cf9f In-Reply-To: <20240223082726.52915-1-xuanzhuo@linux.alibaba.com> References: <20240223082726.52915-1-xuanzhuo@linux.alibaba.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Git-Hash: 510995f33855 X-Patchwork-Delegate: kuba@kernel.org Since free_old_xmit_skbs not only deals with skb, but also xdp frame and subsequent added xsk, so change the name of this function to free_old_xmit. Signed-off-by: Xuan Zhuo Acked-by: Jason Wang --- drivers/net/virtio_net.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index 62f65e2cacd5..7715bb7032ec 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -787,7 +787,7 @@ static void virtnet_rq_unmap_free_buf(struct virtqueue *vq, void *buf) virtnet_rq_free_buf(vi, rq, buf); } -static void free_old_xmit_skbs(struct send_queue *sq, bool in_napi) +static void free_old_xmit(struct send_queue *sq, bool in_napi) { struct virtnet_sq_free_stats stats = {0}; @@ -841,7 +841,7 @@ static void check_sq_full_and_disable(struct virtnet_info *vi, virtqueue_napi_schedule(&sq->napi, sq->vq); } else if (unlikely(!virtqueue_enable_cb_delayed(sq->vq))) { /* More just got used, free them then recheck. */ - free_old_xmit_skbs(sq, false); + free_old_xmit(sq, false); if (sq->vq->num_free >= 2+MAX_SKB_FRAGS) { netif_start_subqueue(dev, qnum); virtqueue_disable_cb(sq->vq); @@ -2137,7 +2137,7 @@ static void virtnet_poll_cleantx(struct receive_queue *rq) do { virtqueue_disable_cb(sq->vq); - free_old_xmit_skbs(sq, true); + free_old_xmit(sq, true); } while (unlikely(!virtqueue_enable_cb_delayed(sq->vq))); if (sq->vq->num_free >= 2 + MAX_SKB_FRAGS) @@ -2285,7 +2285,7 @@ static int virtnet_poll_tx(struct napi_struct *napi, int budget) txq = netdev_get_tx_queue(vi->dev, index); __netif_tx_lock(txq, raw_smp_processor_id()); virtqueue_disable_cb(sq->vq); - free_old_xmit_skbs(sq, true); + free_old_xmit(sq, true); if (sq->vq->num_free >= 2 + MAX_SKB_FRAGS) netif_tx_wake_queue(txq); @@ -2375,7 +2375,7 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev) if (use_napi) virtqueue_disable_cb(sq->vq); - free_old_xmit_skbs(sq, false); + free_old_xmit(sq, false); } while (use_napi && kick && unlikely(!virtqueue_enable_cb_delayed(sq->vq))); From patchwork Fri Feb 23 08:27:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuan Zhuo X-Patchwork-Id: 13568681 X-Patchwork-Delegate: kuba@kernel.org Received: from out30-119.freemail.mail.aliyun.com (out30-119.freemail.mail.aliyun.com [115.124.30.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 40DB44CB23; Fri, 23 Feb 2024 08:28:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.119 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708676884; cv=none; b=ZZmI4F7O8brZWei2tIlIjm1UnESHRj34ANPRpIvAdndE0aP1KDZqHpsIduhk9FoxSTsJ4O3cTbKSB6X1rixZ9tgeAItre/GtxHmdNFKUJVNPvyofUVfOrxf335fZ4SKLO3Vuc6hRXnlHuEguvYRD6ofLJTNb0SL6hGwPB3J8cEA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708676884; c=relaxed/simple; bh=ebU0mf7uHLesU9rjth4gN2TUDxEGZDMelrjYPH5RKDQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=nWpSt6rUF850iJr3d7u+N2a2LhxQRb8eX4vzgZOsrDT+w/7ncXBYkJgcGsvKVWoTbEBmAnui9RgZkkjqXgOQ8XyjYJcFKwo6VXdOEj8+Bx0T+UqFaM0JsPBzBGcJUaVQ3GRgmWBVJQpaCF16sBDxjlYfp55rtKoaZBzDFPSlQ/8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=kyZ1XRku; arc=none smtp.client-ip=115.124.30.119 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="kyZ1XRku" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1708676879; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=KkT5T/swMYJC+M1uG40XFtpIGk/ayvRaf2PNE7WpniY=; b=kyZ1XRkuy8du/QkOIJ+rldpZSedbXtCzkqDImk5q0c3p7XX2WY7xec7QoOqKaHQezftD6LEI/4u9YXSq5w8/eE0AVi4rjDAHJOuoy7bvT0PLh24bZQdfAsCzD644xHVxvB8t4GXT27IUEd5Rd0oMwcJlEb2a6ZAOaxmGwG/ahLc= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R121e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046050;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=35;SR=0;TI=SMTPD_---0W13ndSm_1708676876; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0W13ndSm_1708676876) by smtp.aliyun-inc.com; Fri, 23 Feb 2024 16:27:56 +0800 From: Xuan Zhuo To: virtualization@lists.linux.dev Cc: Richard Weinberger , Anton Ivanov , Johannes Berg , "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Hans de Goede , =?utf-8?q?Ilpo_J=C3=A4rvinen?= , Vadim Pasternak , Bjorn Andersson , Mathieu Poirier , Cornelia Huck , Halil Pasic , Eric Farman , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , linux-um@lists.infradead.org, netdev@vger.kernel.org, platform-driver-x86@vger.kernel.org, linux-remoteproc@vger.kernel.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH vhost v2 19/19] virtio_net: sq support premapped mode Date: Fri, 23 Feb 2024 16:27:26 +0800 Message-Id: <20240223082726.52915-20-xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 2.32.0.3.g01195cf9f In-Reply-To: <20240223082726.52915-1-xuanzhuo@linux.alibaba.com> References: <20240223082726.52915-1-xuanzhuo@linux.alibaba.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Git-Hash: 510995f33855 X-Patchwork-Delegate: kuba@kernel.org If the xsk is enabling, the xsk tx will share the send queue. But the xsk requires that the send queue use the premapped mode. So the send queue must support premapped mode. cmd: sh samples/pktgen/pktgen_sample01_simple.sh -i eth0 \ -s 16 -d 10.0.0.128 -m 00:16:3e:2c:c8:2e -n 0 -p 100 CPU: Intel(R) Xeon(R) Platinum 8369B CPU @ 2.70GHz Machine: ecs.g7.2xlarge(Aliyun) before: 1600010.00 after(no-premapped): 1599966.00 after(premapped): 1600014.00 Signed-off-by: Xuan Zhuo --- drivers/net/virtio_net.c | 136 +++++++++++++++++++++++++++++++++++++-- 1 file changed, 132 insertions(+), 4 deletions(-) diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index 7715bb7032ec..b83ef6afc4fb 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -146,6 +146,25 @@ struct virtnet_rq_dma { u16 need_sync; }; +struct virtnet_sq_dma { + union { + struct virtnet_sq_dma *next; + void *data; + }; + + u32 num; + + dma_addr_t addr[MAX_SKB_FRAGS + 2]; + u32 len[MAX_SKB_FRAGS + 2]; +}; + +struct virtnet_sq_dma_head { + /* record for kfree */ + void *p; + + struct virtnet_sq_dma *free; +}; + /* Internal representation of a send virtqueue */ struct send_queue { /* Virtqueue associated with this send _queue */ @@ -165,6 +184,8 @@ struct send_queue { /* Record whether sq is in reset state. */ bool reset; + + struct virtnet_sq_dma_head dmainfo; }; /* Internal representation of a receive virtqueue */ @@ -368,6 +389,95 @@ static struct xdp_frame *ptr_to_xdp(void *ptr) return (struct xdp_frame *)((unsigned long)ptr & ~VIRTIO_XDP_FLAG); } +static struct virtnet_sq_dma *virtnet_sq_unmap(struct send_queue *sq, void **data) +{ + struct virtnet_sq_dma *d; + int i; + + d = *data; + *data = d->data; + + for (i = 0; i < d->num; ++i) + virtqueue_dma_unmap_page_attrs(sq->vq, d->addr[i], d->len[i], + DMA_TO_DEVICE, 0); + + d->next = sq->dmainfo.free; + sq->dmainfo.free = d; + + return d; +} + +static struct virtnet_sq_dma *virtnet_sq_map_sg(struct send_queue *sq, + int nents, void *data) +{ + struct virtnet_sq_dma *d; + struct scatterlist *sg; + int i; + + if (!sq->dmainfo.free) + return NULL; + + d = sq->dmainfo.free; + sq->dmainfo.free = d->next; + + for_each_sg(sq->sg, sg, nents, i) { + if (virtqueue_dma_map_sg_attrs(sq->vq, sg, DMA_TO_DEVICE, 0)) + goto err; + + d->addr[i] = sg->dma_address; + d->len[i] = sg->length; + } + + d->data = data; + d->num = i; + return d; + +err: + d->num = i; + virtnet_sq_unmap(sq, (void **)&d); + return NULL; +} + +static int virtnet_add_outbuf(struct send_queue *sq, u32 num, void *data) +{ + int ret; + + if (sq->vq->premapped) { + data = virtnet_sq_map_sg(sq, num, data); + if (!data) + return -ENOMEM; + } + + ret = virtqueue_add_outbuf(sq->vq, sq->sg, num, data, GFP_ATOMIC); + if (ret && sq->vq->premapped) + virtnet_sq_unmap(sq, &data); + + return ret; +} + +static int virtnet_sq_init_dma_mate(struct send_queue *sq) +{ + struct virtnet_sq_dma *d; + int num, i; + + num = virtqueue_get_vring_size(sq->vq); + + sq->dmainfo.free = kcalloc(num, sizeof(*sq->dmainfo.free), GFP_KERNEL); + if (!sq->dmainfo.free) + return -ENOMEM; + + sq->dmainfo.p = sq->dmainfo.free; + + for (i = 0; i < num; ++i) { + d = &sq->dmainfo.free[i]; + d->next = d + 1; + } + + d->next = NULL; + + return 0; +} + static void __free_old_xmit(struct send_queue *sq, bool in_napi, struct virtnet_sq_free_stats *stats) { @@ -377,6 +487,9 @@ static void __free_old_xmit(struct send_queue *sq, bool in_napi, while ((ptr = virtqueue_get_buf(sq->vq, &len)) != NULL) { ++stats->packets; + if (sq->vq->premapped) + virtnet_sq_unmap(sq, &ptr); + if (!is_xdp_frame(ptr)) { struct sk_buff *skb = ptr; @@ -890,8 +1003,7 @@ static int __virtnet_xdp_xmit_one(struct virtnet_info *vi, skb_frag_size(frag), skb_frag_off(frag)); } - err = virtqueue_add_outbuf(sq->vq, sq->sg, nr_frags + 1, - xdp_to_ptr(xdpf), GFP_ATOMIC); + err = virtnet_add_outbuf(sq, nr_frags + 1, xdp_to_ptr(xdpf)); if (unlikely(err)) return -ENOSPC; /* Caller handle free/refcnt */ @@ -2357,7 +2469,7 @@ static int xmit_skb(struct send_queue *sq, struct sk_buff *skb) return num_sg; num_sg++; } - return virtqueue_add_outbuf(sq->vq, sq->sg, num_sg, skb, GFP_ATOMIC); + return virtnet_add_outbuf(sq, num_sg, skb); } static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev) @@ -4166,6 +4278,8 @@ static void virtnet_free_queues(struct virtnet_info *vi) for (i = 0; i < vi->max_queue_pairs; i++) { __netif_napi_del(&vi->rq[i].napi); __netif_napi_del(&vi->sq[i].napi); + + kfree(vi->sq[i].dmainfo.p); } /* We called __netif_napi_del(), @@ -4214,6 +4328,15 @@ static void free_receive_page_frags(struct virtnet_info *vi) static void virtnet_sq_free_unused_buf(struct virtqueue *vq, void *buf) { + struct virtnet_info *vi = vq->vdev->priv; + struct send_queue *sq; + int i = vq2rxq(vq); + + sq = &vi->sq[i]; + + if (sq->vq->premapped) + virtnet_sq_unmap(sq, &buf); + if (!is_xdp_frame(buf)) dev_kfree_skb(buf); else @@ -4327,8 +4450,10 @@ static int virtnet_find_vqs(struct virtnet_info *vi) if (ctx) ctx[rxq2vq(i)] = true; - if (premapped) + if (premapped) { premapped[rxq2vq(i)] = true; + premapped[txq2vq(i)] = true; + } } cfg.nvqs = total_vqs; @@ -4352,6 +4477,9 @@ static int virtnet_find_vqs(struct virtnet_info *vi) vi->rq[i].vq = vqs[rxq2vq(i)]; vi->rq[i].min_buf_len = mergeable_min_buf_len(vi, vi->rq[i].vq); vi->sq[i].vq = vqs[txq2vq(i)]; + + if (vi->sq[i].vq->premapped) + virtnet_sq_init_dma_mate(&vi->sq[i]); } /* run here: ret == 0. */