From patchwork Fri May 14 09:59:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Zhu, Lingshan" X-Patchwork-Id: 12257369 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F3FF6C433B4 for ; Fri, 14 May 2021 10:04:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DA40061453 for ; Fri, 14 May 2021 10:04:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232947AbhENKF5 (ORCPT ); Fri, 14 May 2021 06:05:57 -0400 Received: from mga04.intel.com ([192.55.52.120]:4949 "EHLO mga04.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232800AbhENKFx (ORCPT ); Fri, 14 May 2021 06:05:53 -0400 IronPort-SDR: z0S/FA/v/p8vZbUYlCluivB3qonxbG9eilKDqHKLZXkidr7U7dQyHTumbUKGKMIRRcxZvNiOD7 COXCKp5/oaTw== X-IronPort-AV: E=McAfee;i="6200,9189,9983"; a="198195255" X-IronPort-AV: E=Sophos;i="5.82,299,1613462400"; d="scan'208";a="198195255" Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 May 2021 03:04:42 -0700 IronPort-SDR: Os7nvULX0grsqC960ZtdHLoskEzG0z4vyNYpLl7H98YLmerkKXa1c+M6gPUGZxgHgQG5pA1yt1 xQc9H7ECk3Qw== X-IronPort-AV: E=Sophos;i="5.82,299,1613462400"; d="scan'208";a="626910435" Received: from unknown (HELO localhost.localdomain.bj.intel.com) ([10.240.193.73]) by fmsmga005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 May 2021 03:04:41 -0700 From: Zhu Lingshan To: jasowang@redhat.com, mst@redhat.com Cc: virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, kvm@vger.kernel.org, Zhu Lingshan Subject: [PATCH V2 1/2] vDPA/ifcvf: record virtio notify base Date: Fri, 14 May 2021 17:59:12 +0800 Message-Id: <20210514095913.41777-2-lingshan.zhu@intel.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210514095913.41777-1-lingshan.zhu@intel.com> References: <20210514095913.41777-1-lingshan.zhu@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org This commit records virtio notify base physical addr and calculate doorbell physical address for vqs. Signed-off-by: Zhu Lingshan --- drivers/vdpa/ifcvf/ifcvf_base.c | 4 ++++ drivers/vdpa/ifcvf/ifcvf_base.h | 2 ++ 2 files changed, 6 insertions(+) diff --git a/drivers/vdpa/ifcvf/ifcvf_base.c b/drivers/vdpa/ifcvf/ifcvf_base.c index 1a661ab45af5..6e197fe0fcf9 100644 --- a/drivers/vdpa/ifcvf/ifcvf_base.c +++ b/drivers/vdpa/ifcvf/ifcvf_base.c @@ -133,6 +133,8 @@ int ifcvf_init_hw(struct ifcvf_hw *hw, struct pci_dev *pdev) &hw->notify_off_multiplier); hw->notify_bar = cap.bar; hw->notify_base = get_cap_addr(hw, &cap); + hw->notify_base_pa = pci_resource_start(pdev, cap.bar) + + le32_to_cpu(cap.offset); IFCVF_DBG(pdev, "hw->notify_base = %p\n", hw->notify_base); break; @@ -161,6 +163,8 @@ int ifcvf_init_hw(struct ifcvf_hw *hw, struct pci_dev *pdev) notify_off = ifc_ioread16(&hw->common_cfg->queue_notify_off); hw->vring[i].notify_addr = hw->notify_base + notify_off * hw->notify_off_multiplier; + hw->vring[i].notify_pa = hw->notify_base_pa + + notify_off * hw->notify_off_multiplier; } hw->lm_cfg = hw->base[IFCVF_LM_BAR]; diff --git a/drivers/vdpa/ifcvf/ifcvf_base.h b/drivers/vdpa/ifcvf/ifcvf_base.h index 0111bfdeb342..447f4ad9c0bf 100644 --- a/drivers/vdpa/ifcvf/ifcvf_base.h +++ b/drivers/vdpa/ifcvf/ifcvf_base.h @@ -73,6 +73,7 @@ struct vring_info { u16 last_avail_idx; bool ready; void __iomem *notify_addr; + phys_addr_t notify_pa; u32 irq; struct vdpa_callback cb; char msix_name[256]; @@ -87,6 +88,7 @@ struct ifcvf_hw { u8 notify_bar; /* Notificaiton bar address */ void __iomem *notify_base; + phys_addr_t notify_base_pa; u32 notify_off_multiplier; u64 req_features; u64 hw_features; From patchwork Fri May 14 09:59:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Zhu, Lingshan" X-Patchwork-Id: 12257371 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 23E2EC43460 for ; Fri, 14 May 2021 10:04:50 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 132F16101D for ; Fri, 14 May 2021 10:04:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233916AbhENKF7 (ORCPT ); Fri, 14 May 2021 06:05:59 -0400 Received: from mga04.intel.com ([192.55.52.120]:4949 "EHLO mga04.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232968AbhENKFz (ORCPT ); Fri, 14 May 2021 06:05:55 -0400 IronPort-SDR: LVKnD17mxU+qtpbfWtwyUQ2mRfEQicaYcU4M28pFQHcx7SQnTetu2RYWdNJdqblGSqvD8PZ3OW fGoxJAwQj/Lw== X-IronPort-AV: E=McAfee;i="6200,9189,9983"; a="198195258" X-IronPort-AV: E=Sophos;i="5.82,299,1613462400"; d="scan'208";a="198195258" Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 May 2021 03:04:44 -0700 IronPort-SDR: x4XG2Ek49QearfQP2tNfyFnPGfB7y1A9f5DDO4yeMwa+021/nrapZzwN+N0mCxN93YWEMrHQc9 eGMwV6AOiXfA== X-IronPort-AV: E=Sophos;i="5.82,299,1613462400"; d="scan'208";a="626910444" Received: from unknown (HELO localhost.localdomain.bj.intel.com) ([10.240.193.73]) by fmsmga005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 May 2021 03:04:42 -0700 From: Zhu Lingshan To: jasowang@redhat.com, mst@redhat.com Cc: virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, kvm@vger.kernel.org, Zhu Lingshan Subject: [PATCH V2 2/2] vDPA/ifcvf: implement doorbell mapping for ifcvf Date: Fri, 14 May 2021 17:59:13 +0800 Message-Id: <20210514095913.41777-3-lingshan.zhu@intel.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210514095913.41777-1-lingshan.zhu@intel.com> References: <20210514095913.41777-1-lingshan.zhu@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org This commit implements doorbell mapping feature for ifcvf. This feature maps the notify page to userspace, to eliminate vmexit when kick a vq. Signed-off-by: Zhu Lingshan --- drivers/vdpa/ifcvf/ifcvf_main.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/drivers/vdpa/ifcvf/ifcvf_main.c b/drivers/vdpa/ifcvf/ifcvf_main.c index ab0ab5cf0f6e..41c09437602d 100644 --- a/drivers/vdpa/ifcvf/ifcvf_main.c +++ b/drivers/vdpa/ifcvf/ifcvf_main.c @@ -413,6 +413,21 @@ static int ifcvf_vdpa_get_vq_irq(struct vdpa_device *vdpa_dev, return vf->vring[qid].irq; } +static struct vdpa_notification_area ifcvf_get_vq_notification(struct vdpa_device *vdpa_dev, + u16 idx) +{ + struct ifcvf_adapter *adapter = vdpa_to_adapter(vdpa_dev); + struct ifcvf_hw *vf = vdpa_to_vf(vdpa_dev); + struct pci_dev *pdev = adapter->pdev; + struct vdpa_notification_area area; + + area.addr = vf->vring[idx].notify_pa; + area.size = PAGE_SIZE; + if (area.addr % PAGE_SIZE) + IFCVF_DBG(pdev, "vq %u doorbell address is not PAGE_SIZE aligned\n", idx); + return area; +} + /* * IFCVF currently does't have on-chip IOMMU, so not * implemented set_map()/dma_map()/dma_unmap() @@ -440,6 +455,7 @@ static const struct vdpa_config_ops ifc_vdpa_ops = { .get_config = ifcvf_vdpa_get_config, .set_config = ifcvf_vdpa_set_config, .set_config_cb = ifcvf_vdpa_set_config_cb, + .get_vq_notification = ifcvf_get_vq_notification, }; static int ifcvf_probe(struct pci_dev *pdev, const struct pci_device_id *id)