From patchwork Wed Jun 28 06:59:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cindy Lu X-Patchwork-Id: 13295324 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D96BEEB64D7 for ; Wed, 28 Jun 2023 08:06:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231910AbjF1IEh (ORCPT ); Wed, 28 Jun 2023 04:04:37 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:55576 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232281AbjF1IBw (ORCPT ); Wed, 28 Jun 2023 04:01:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1687939263; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Rjet8Ukl1WyvnmqebMgJ6muk1ycBNL7JSNW8bedLLZI=; b=SMImhwbEG6F8hlscNlTedfggDK//rdQE3nRtDkiymxSNkBUdind3SpDpPd0WULwdfXXCBe 2geYUJWwZo4QntG66qlWsAio4TBQcWlRn5a3CcD7t/jGKTRhRuGbU7nAs2UiFF8nF5hQZw odsNnZNgW3sab4vZT73ILVI0KgV3hVs= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-645-9O6KTV6nMZyR72Za4PuBZQ-1; Wed, 28 Jun 2023 02:59:31 -0400 X-MC-Unique: 9O6KTV6nMZyR72Za4PuBZQ-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D90621C05156; Wed, 28 Jun 2023 06:59:30 +0000 (UTC) Received: from server.redhat.com (ovpn-13-142.pek2.redhat.com [10.72.13.142]) by smtp.corp.redhat.com (Postfix) with ESMTP id 51A62492B02; Wed, 28 Jun 2023 06:59:27 +0000 (UTC) From: Cindy Lu To: lulu@redhat.com, jasowang@redhat.com, mst@redhat.com, maxime.coquelin@redhat.com, xieyongji@bytedance.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org Subject: [RFC 1/4] vduse: Add the struct to save the vq reconnect info Date: Wed, 28 Jun 2023 14:59:16 +0800 Message-Id: <20230628065919.54042-2-lulu@redhat.com> In-Reply-To: <20230628065919.54042-1-lulu@redhat.com> References: <20230628065919.54042-1-lulu@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Your Name this struct is to save the reconnect info struct, in this struct saved the page info that alloc to save the reconnect info Signed-off-by: Cindy Lu --- drivers/vdpa/vdpa_user/vduse_dev.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/drivers/vdpa/vdpa_user/vduse_dev.c b/drivers/vdpa/vdpa_user/vduse_dev.c index 26b7e29cb900..f845dc46b1db 100644 --- a/drivers/vdpa/vdpa_user/vduse_dev.c +++ b/drivers/vdpa/vdpa_user/vduse_dev.c @@ -72,6 +72,12 @@ struct vduse_umem { struct page **pages; struct mm_struct *mm; }; +struct vdpa_reconnect_info { + u32 index; + phys_addr_t addr; + unsigned long vaddr; + phys_addr_t size; +}; struct vduse_dev { struct vduse_vdpa *vdev; @@ -106,6 +112,7 @@ struct vduse_dev { u32 vq_align; struct vduse_umem *umem; struct mutex mem_lock; + struct vdpa_reconnect_info reconnect_info[64]; }; struct vduse_dev_msg { From patchwork Wed Jun 28 06:59:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cindy Lu X-Patchwork-Id: 13295323 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8CC8DEB64D7 for ; Wed, 28 Jun 2023 08:06:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231741AbjF1IEf (ORCPT ); Wed, 28 Jun 2023 04:04:35 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:50045 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232343AbjF1ICB (ORCPT ); Wed, 28 Jun 2023 04:02:01 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1687939273; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=csNRQntj+qYkEtYEdA8LKIDg2tovNr8UB73l+ephPGs=; b=QuiqpCe/zYsnrWuCooc1WBuN7YSAMHirjIDRfHFqolfKAtwUHo7j9Xp5KJ/TxO4VpayGyg hbp7CsqPdx3HcVeyGnxj696DZyNsx0dwotyhidbU70yWKdHrHmwEuMORwz6lvt5r/ZwXhA FIaRxUHX4RA1E2lKkwVgRgxPb9HOzEA= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-86-t2pgzsuqNTeSfM39pUP_9w-1; Wed, 28 Jun 2023 02:59:35 -0400 X-MC-Unique: t2pgzsuqNTeSfM39pUP_9w-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 0DC8B1C08783; Wed, 28 Jun 2023 06:59:35 +0000 (UTC) Received: from server.redhat.com (ovpn-13-142.pek2.redhat.com [10.72.13.142]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7E413492B02; Wed, 28 Jun 2023 06:59:31 +0000 (UTC) From: Cindy Lu To: lulu@redhat.com, jasowang@redhat.com, mst@redhat.com, maxime.coquelin@redhat.com, xieyongji@bytedance.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org Subject: [RFC 2/4] vduse: Add file operation for mmap Date: Wed, 28 Jun 2023 14:59:17 +0800 Message-Id: <20230628065919.54042-3-lulu@redhat.com> In-Reply-To: <20230628065919.54042-1-lulu@redhat.com> References: <20230628065919.54042-1-lulu@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Your Name Add the operation for mmap, The user space APP will use this function to map the pages to userspace Signed-off-by: Cindy Lu --- drivers/vdpa/vdpa_user/vduse_dev.c | 49 ++++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+) diff --git a/drivers/vdpa/vdpa_user/vduse_dev.c b/drivers/vdpa/vdpa_user/vduse_dev.c index f845dc46b1db..1b833bf0ae37 100644 --- a/drivers/vdpa/vdpa_user/vduse_dev.c +++ b/drivers/vdpa/vdpa_user/vduse_dev.c @@ -1313,6 +1313,54 @@ static struct vduse_dev *vduse_dev_get_from_minor(int minor) return dev; } + +static vm_fault_t vduse_vm_fault(struct vm_fault *vmf) +{ + struct vduse_dev *dev = vmf->vma->vm_file->private_data; + struct vm_area_struct *vma = vmf->vma; + u16 index = vma->vm_pgoff; + + struct vdpa_reconnect_info *info; + info = &dev->reconnect_info[index]; + + vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot); + if (remap_pfn_range(vma, vmf->address & PAGE_MASK, PFN_DOWN(info->addr), + PAGE_SIZE, vma->vm_page_prot)) + return VM_FAULT_SIGBUS; + return VM_FAULT_NOPAGE; +} + +static const struct vm_operations_struct vduse_vm_ops = { + .fault = vduse_vm_fault, +}; + +static int vduse_mmap(struct file *file, struct vm_area_struct *vma) +{ + struct vduse_dev *dev = file->private_data; + struct vdpa_reconnect_info *info; + unsigned long index = vma->vm_pgoff; + + if (vma->vm_end - vma->vm_start != PAGE_SIZE) + return -EINVAL; + if ((vma->vm_flags & VM_SHARED) == 0) + return -EINVAL; + + if (index > 65535) + return -EINVAL; + + info = &dev->reconnect_info[index]; + if (info->addr & (PAGE_SIZE - 1)) + return -EINVAL; + if (vma->vm_end - vma->vm_start != info->size) { + return -ENOTSUPP; + } + + vm_flags_set(vma, VM_IO | VM_PFNMAP | VM_DONTEXPAND | VM_DONTDUMP); + vma->vm_ops = &vduse_vm_ops; + + return 0; +} + static int vduse_dev_open(struct inode *inode, struct file *file) { int ret; @@ -1345,6 +1393,7 @@ static const struct file_operations vduse_dev_fops = { .unlocked_ioctl = vduse_dev_ioctl, .compat_ioctl = compat_ptr_ioctl, .llseek = noop_llseek, + .mmap = vduse_mmap, }; static struct vduse_dev *vduse_dev_create(void) From patchwork Wed Jun 28 06:59:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cindy Lu X-Patchwork-Id: 13295322 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 44894EB64D7 for ; Wed, 28 Jun 2023 08:04:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231526AbjF1IEc (ORCPT ); Wed, 28 Jun 2023 04:04:32 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:50996 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232152AbjF1IBG (ORCPT ); Wed, 28 Jun 2023 04:01:06 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1687939215; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Vm8YdVuG5g/R4Upgn7ExptiiekTMB39MDE6nGXwwg5I=; b=F1Bx07nzPaIkb6j4fA2DPfBTkV4xpbMkEt6ECFntR5Q5YO0CI/AmgOxgfGtDag+yO9MThn BmRjs/6o0JjArkN3gnwuIbQsphEDuMK88xVdR+B69EEHrSEtiIybN7u+H8A7ba5Pn5FRHR 6XBRG8t+k1YNrM1UW4ja6kPlPg08Dt8= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-115-wEAlpELWNdCfebbpAG2Kmg-1; Wed, 28 Jun 2023 02:59:39 -0400 X-MC-Unique: wEAlpELWNdCfebbpAG2Kmg-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 379003C025C7; Wed, 28 Jun 2023 06:59:39 +0000 (UTC) Received: from server.redhat.com (ovpn-13-142.pek2.redhat.com [10.72.13.142]) by smtp.corp.redhat.com (Postfix) with ESMTP id A62E0492B02; Wed, 28 Jun 2023 06:59:35 +0000 (UTC) From: Cindy Lu To: lulu@redhat.com, jasowang@redhat.com, mst@redhat.com, maxime.coquelin@redhat.com, xieyongji@bytedance.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org Subject: [RFC 3/4] vduse: Add the function for get/free the mapp pages Date: Wed, 28 Jun 2023 14:59:18 +0800 Message-Id: <20230628065919.54042-4-lulu@redhat.com> In-Reply-To: <20230628065919.54042-1-lulu@redhat.com> References: <20230628065919.54042-1-lulu@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Your Name Add the function for get/free pages, ad this info will saved in dev->reconnect_info Signed-off-by: Cindy Lu --- drivers/vdpa/vdpa_user/vduse_dev.c | 35 ++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/drivers/vdpa/vdpa_user/vduse_dev.c b/drivers/vdpa/vdpa_user/vduse_dev.c index 1b833bf0ae37..3df1256eccb4 100644 --- a/drivers/vdpa/vdpa_user/vduse_dev.c +++ b/drivers/vdpa/vdpa_user/vduse_dev.c @@ -1313,6 +1313,35 @@ static struct vduse_dev *vduse_dev_get_from_minor(int minor) return dev; } +int vduse_get_vq_reconnnect(struct vduse_dev *dev, u16 idx) +{ + struct vdpa_reconnect_info *area; + void *addr = (void *)get_zeroed_page(GFP_KERNEL); + + area = &dev->reconnect_info[idx]; + + area->addr = virt_to_phys(addr); + area->vaddr = (unsigned long)addr; + area->size = PAGE_SIZE; + area->index = idx; + + return 0; +} + +int vduse_free_vq_reconnnect(struct vduse_dev *dev, u16 idx) +{ + struct vdpa_reconnect_info *area; + + area = &dev->reconnect_info[idx]; + if ((area->size == PAGE_SIZE) && (area->addr != NULL)) { + free_page(area->vaddr); + area->size = 0; + area->addr = 0; + area->vaddr = 0; + } + + return 0; +} static vm_fault_t vduse_vm_fault(struct vm_fault *vmf) { @@ -1446,6 +1475,10 @@ static int vduse_destroy_dev(char *name) mutex_unlock(&dev->lock); return -EBUSY; } + for (int i = 0; i < dev->vq_num; i++) { + + vduse_free_vq_reconnnect(dev, i); + } dev->connected = true; mutex_unlock(&dev->lock); @@ -1583,6 +1616,8 @@ static int vduse_create_dev(struct vduse_dev_config *config, INIT_WORK(&dev->vqs[i].kick, vduse_vq_kick_work); spin_lock_init(&dev->vqs[i].kick_lock); spin_lock_init(&dev->vqs[i].irq_lock); + + vduse_get_vq_reconnnect(dev, i); } ret = idr_alloc(&vduse_idr, dev, 1, VDUSE_DEV_MAX, GFP_KERNEL); From patchwork Wed Jun 28 06:59:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cindy Lu X-Patchwork-Id: 13295339 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0F606EB64D7 for ; Wed, 28 Jun 2023 08:06:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232274AbjF1IGI (ORCPT ); Wed, 28 Jun 2023 04:06:08 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:30353 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231588AbjF1ICw (ORCPT ); Wed, 28 Jun 2023 04:02:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1687939321; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JAxb7d8d9hAo9Ep6Vle9AGPfDkrTVJl7Nk2Fi7I0ejI=; b=iPnOD3cO34trA79QROPmXXFfLnFyFR1PbHWH4K29UoghN2DoG6SfKuKgdPNf0VVyyR8tzi DxlMiSut+9TWldpVbBQpgbju/HkEt5IIuoEMKP/+Hq5HpiogJ3hVK/EUtVtHZzScbhOheB zTHG/U4RU08DchEWW/rTwdhQMCTFCXM= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-325-3YrzZBg_P96LFy3lVXhYaA-1; Wed, 28 Jun 2023 03:00:05 -0400 X-MC-Unique: 3YrzZBg_P96LFy3lVXhYaA-1 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 327188ED600; Wed, 28 Jun 2023 07:00:05 +0000 (UTC) Received: from server.redhat.com (ovpn-13-142.pek2.redhat.com [10.72.13.142]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9FB7540BB4D; Wed, 28 Jun 2023 07:00:01 +0000 (UTC) From: Cindy Lu To: lulu@redhat.com, jasowang@redhat.com, mst@redhat.com, maxime.coquelin@redhat.com, xieyongji@bytedance.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org Subject: [RFC 4/4] vduse: update the vq_info in ioctl Date: Wed, 28 Jun 2023 14:59:19 +0800 Message-Id: <20230628065919.54042-5-lulu@redhat.com> In-Reply-To: <20230628065919.54042-1-lulu@redhat.com> References: <20230628065919.54042-1-lulu@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.10 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Your Name in VDUSE_VQ_GET_INFO, driver will sync the last_avail_idx with reconnect info, I have olny test the split mode, so only use this here, will add more information later Signed-off-by: Cindy Lu --- drivers/vdpa/vdpa_user/vduse_dev.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/drivers/vdpa/vdpa_user/vduse_dev.c b/drivers/vdpa/vdpa_user/vduse_dev.c index 3df1256eccb4..b8e453eac0ce 100644 --- a/drivers/vdpa/vdpa_user/vduse_dev.c +++ b/drivers/vdpa/vdpa_user/vduse_dev.c @@ -141,6 +141,11 @@ static u32 allowed_device_id[] = { VIRTIO_ID_NET, }; +struct vhost_reconnect_vring { + uint16_t last_avail_idx; + bool avail_wrap_counter; +}; + static inline struct vduse_dev *vdpa_to_vduse(struct vdpa_device *vdpa) { struct vduse_vdpa *vdev = container_of(vdpa, struct vduse_vdpa, vdpa); @@ -1176,6 +1181,17 @@ static long vduse_dev_ioctl(struct file *file, unsigned int cmd, vq->state.split.avail_index; vq_info.ready = vq->ready; + struct vdpa_reconnect_info *area; + + area = &dev->reconnect_info[index]; + struct vhost_reconnect_vring *log_reconnect; + + log_reconnect = (struct vhost_reconnect_vring *)area->vaddr; + if (log_reconnect->last_avail_idx != + vq_info.split.avail_index) { + vq_info.split.avail_index = + log_reconnect->last_avail_idx; + } ret = -EFAULT; if (copy_to_user(argp, &vq_info, sizeof(vq_info)))