From patchwork Wed Apr 26 10:06:46 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Liu, Yi L" X-Patchwork-Id: 9700989 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 52087603F4 for ; Wed, 26 Apr 2017 10:24:50 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2E541284EE for ; Wed, 26 Apr 2017 10:24:50 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 22F4C285F0; Wed, 26 Apr 2017 10:24:50 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B0E17284EE for ; Wed, 26 Apr 2017 10:24:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2998091AbdDZKYs (ORCPT ); Wed, 26 Apr 2017 06:24:48 -0400 Received: from mga02.intel.com ([134.134.136.20]:20231 "EHLO mga02.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2998096AbdDZKYo (ORCPT ); Wed, 26 Apr 2017 06:24:44 -0400 Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 26 Apr 2017 03:24:44 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.37,254,1488873600"; d="scan'208";a="79066543" Received: from sky-dev.bj.intel.com ([10.238.145.47]) by orsmga002.jf.intel.com with ESMTP; 26 Apr 2017 03:24:41 -0700 From: "Liu, Yi L" To: qemu-devel@nongnu.org, alex.williamson@redhat.com, peterx@redhat.com Cc: kvm@vger.kernel.org, jasowang@redhat.com, iommu@lists.linux-foundation.org, kevin.tian@intel.com, ashok.raj@intel.com, jacob.jun.pan@intel.com, tianyu.lan@intel.com, yi.l.liu@intel.com, jean-philippe.brucker@arm.com, "Liu, Yi L" Subject: [RFC PATCH 16/20] VFIO: Add notifier for propagating IOMMU TLB invalidate Date: Wed, 26 Apr 2017 18:06:46 +0800 Message-Id: <1493201210-14357-17-git-send-email-yi.l.liu@linux.intel.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1493201210-14357-1-git-send-email-yi.l.liu@linux.intel.com> References: <1493201210-14357-1-git-send-email-yi.l.liu@linux.intel.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This patch adds the following items: * add new notifier flag IOMMU_NOTIFIER_IOMMU_TLB_INV = 0x8 * add new IOCTL cmd VFIO_IOMMU_TLB_INVALIDATE attached on container->fd * add vfio_iommu_tlb_invalidate_notify() to propagate IOMMU TLB invalidate to host This new notifier is originated from the requirement of SVM virtualization on VT-d. It is for invalidation of first-level and nested mappings from the IOTLB and the paging-structure-caches. Since the existed MAP/UNMAP notifier is designed for second-level related mappings, it is not suitable for the new requirement. So it is necessary to introduce this new notifier to meet the SVM virtualization requirement. Further detail would be included in the patch below: "intel_iommu: propagate Extended-IOTLB invalidate to host" Signed-off-by: Liu, Yi L --- hw/vfio/pci.c | 37 +++++++++++++++++++++++++++++++++++++ include/exec/memory.h | 2 ++ linux-headers/linux/iommu.h | 5 +++++ linux-headers/linux/vfio.h | 8 ++++++++ 4 files changed, 52 insertions(+) diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index a1e6942..afcefd6 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -2619,6 +2619,33 @@ static void vfio_iommu_bind_pasid_tbl_notify(IOMMUNotifier *n, void *data) g_free(vfio_svm); } +static void vfio_iommu_tlb_invalidate_notify(IOMMUNotifier *n, + void *data) +{ + VFIOGuestIOMMU *giommu = container_of(n, VFIOGuestIOMMU, n); + VFIOContainer *container = giommu->container; + IOMMUNotifierData *iommu_data = (IOMMUNotifierData *) data; + struct vfio_iommu_tlb_invalidate *vfio_tlb_inv; + int argsz; + + argsz = sizeof(*vfio_tlb_inv) + iommu_data->payload_size; + vfio_tlb_inv = g_malloc0(argsz); + vfio_tlb_inv->argsz = argsz; + vfio_tlb_inv->length = iommu_data->payload_size; + + memcpy(&vfio_tlb_inv->data, iommu_data->payload, + iommu_data->payload_size); + + rcu_read_lock(); + if (ioctl(container->fd, VFIO_IOMMU_TLB_INVALIDATE, + vfio_tlb_inv) != 0) { + error_report("vfio_iommu_tlb_invalidate_notify:" + " failed, contanier: %p", container); + } + rcu_read_unlock(); + g_free(vfio_tlb_inv); +} + static void vfio_realize(PCIDevice *pdev, Error **errp) { VFIOPCIDevice *vdev = DO_UPCAST(VFIOPCIDevice, pdev, pdev); @@ -2865,6 +2892,7 @@ static void vfio_realize(PCIDevice *pdev, Error **errp) QTAILQ_FOREACH(subregion, &as->root->subregions, subregions_link) { if (memory_region_is_iommu(subregion)) { IOMMUNotifier n1; + IOMMUNotifier n2; /* FIXME: current iommu notifier is actually designed for @@ -2882,6 +2910,15 @@ static void vfio_realize(PCIDevice *pdev, Error **errp) 0, &n1); + iommu_notifier_init(&n2, vfio_iommu_tlb_invalidate_notify, + IOMMU_NOTIFIER_IOMMU_TLB_INV, + 0, + 0); + vfio_register_notifier(group->container, + subregion, + 0, + &n2); + memory_region_notify_device_record(subregion, &vdev->host); diff --git a/include/exec/memory.h b/include/exec/memory.h index 3b8f487..af15351 100644 --- a/include/exec/memory.h +++ b/include/exec/memory.h @@ -83,6 +83,8 @@ typedef enum { IOMMU_NOTIFIER_MAP = 0x2, /* Notify PASID Table Binding */ IOMMU_NOTIFIER_SVM_PASIDT_BIND = 0x4, + /* Notify IOMMU TLB Invalidation */ + IOMMU_NOTIFIER_IOMMU_TLB_INV = 0x8, } IOMMUNotifierFlag; #define IOMMU_NOTIFIER_MAP_UNMAP (IOMMU_NOTIFIER_MAP | IOMMU_NOTIFIER_UNMAP) diff --git a/linux-headers/linux/iommu.h b/linux-headers/linux/iommu.h index 4519dcf..c2742ba 100644 --- a/linux-headers/linux/iommu.h +++ b/linux-headers/linux/iommu.h @@ -27,4 +27,9 @@ struct pasid_table_info { __u8 opaque[];/* IOMMU-specific details */ }; +struct tlb_invalidate_info { + __u32 model; + __u8 opaque[]; +}; + #endif /* __LINUX_IOMMU_H */ diff --git a/linux-headers/linux/vfio.h b/linux-headers/linux/vfio.h index 9848d63..6c71c4a 100644 --- a/linux-headers/linux/vfio.h +++ b/linux-headers/linux/vfio.h @@ -554,6 +554,14 @@ struct vfio_device_svm { #define VFIO_IOMMU_SVM_BIND_TASK _IO(VFIO_TYPE, VFIO_BASE + 22) +/* For IOMMU Invalidation Passdwon */ +struct vfio_iommu_tlb_invalidate { + __u32 argsz; + __u32 length; + __u8 data[]; +}; + +#define VFIO_IOMMU_TLB_INVALIDATE _IO(VFIO_TYPE, VFIO_BASE + 23) /* -------- Additional API for SPAPR TCE (Server POWERPC) IOMMU -------- */