From patchwork Wed May 4 11:54:15 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 9012821 Return-Path: X-Original-To: patchwork-linux-arm@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id CF19CBF29F for ; Wed, 4 May 2016 11:57:01 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id CB21C203A0 for ; Wed, 4 May 2016 11:57:00 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.9]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C25162039C for ; Wed, 4 May 2016 11:56:59 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1axvOt-0004oz-Ap; Wed, 04 May 2016 11:55:31 +0000 Received: from mail-wm0-x22c.google.com ([2a00:1450:400c:c09::22c]) by bombadil.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1axvOS-0003M2-4m for linux-arm-kernel@lists.infradead.org; Wed, 04 May 2016 11:55:05 +0000 Received: by mail-wm0-x22c.google.com with SMTP id e201so184502410wme.0 for ; Wed, 04 May 2016 04:54:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=kkR7eWNMcjkbRyCxA/QqfEvfom4pZUQJTogOBwmM93M=; b=YNUXwFlJgIV/tDKLwFezgORyqkNBXlfreEVK2800jmzKc8aA01HxGd/evnHXvzW4lu pnsup7E0wPxOP9tWJ2kqOLWlZWAx0VEjjax2iQ4PD4844NV+F4HQO8879v+o02Vr9U+H dNeHAHkDRZzdUJXJyvM2Elmw4GwnKnja6C27Q= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=kkR7eWNMcjkbRyCxA/QqfEvfom4pZUQJTogOBwmM93M=; b=jx/xVxsyMHztPLZ4o6b00YIlIpacBRa9JTaOUOvK18y9hUlexXTmzl3rewS5PdwCG3 lD1f7YPvXe7A/DKrdxhS38k6oiNqPI7kqRn5RVm8tQlJl0upT019pHo6iiJ8XfOEgPVm I5hUJBewTlwjotJqu3l0HiAnT+wOy3XPo7EKgy3QkED0nQXUsBtr1s9cpQ7fr47G4wrM SVdBdlmsC3bLUhFSmCIkxa9RMHHL/uqadi+j7sKBNYbRb4x6Azo1+ip3B6GLKQ4TjC5d QVSHh9y2jhdyrNqkuTL3wGIkt/Ac5kFSJEm5cyicF/w/SVdAgWbwMrlfZnL91UneOGN1 3GvQ== X-Gm-Message-State: AOPr4FUbrYt84iAD1lqPRwsB9VdWRxteXdaQ/ksm+RDPk616rUGCG+CDjaHlGTooCDklA+gB X-Received: by 10.28.218.21 with SMTP id r21mr31067053wmg.100.1462362882685; Wed, 04 May 2016 04:54:42 -0700 (PDT) Received: from new-host-61.home (LMontsouris-657-1-37-90.w80-11.abo.wanadoo.fr. [80.11.198.90]) by smtp.gmail.com with ESMTPSA id ck9sm3816623wjc.22.2016.05.04.04.54.38 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 04 May 2016 04:54:41 -0700 (PDT) From: Eric Auger To: eric.auger@st.com, eric.auger@linaro.org, robin.murphy@arm.com, alex.williamson@redhat.com, will.deacon@arm.com, joro@8bytes.org, tglx@linutronix.de, jason@lakedaemon.net, marc.zyngier@arm.com, christoffer.dall@linaro.org, linux-arm-kernel@lists.infradead.org Subject: [PATCH v9 4/7] vfio: allow reserved msi iova registration Date: Wed, 4 May 2016 11:54:15 +0000 Message-Id: <1462362858-2925-5-git-send-email-eric.auger@linaro.org> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1462362858-2925-1-git-send-email-eric.auger@linaro.org> References: <1462362858-2925-1-git-send-email-eric.auger@linaro.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20160504_045504_560348_652B98EF X-CRM114-Status: GOOD ( 21.97 ) X-Spam-Score: -2.7 (--) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: julien.grall@arm.com, patches@linaro.org, Jean-Philippe.Brucker@arm.com, p.fedin@samsung.com, linux-kernel@vger.kernel.org, Bharat.Bhushan@freescale.com, iommu@lists.linux-foundation.org, pranav.sawargaonkar@gmail.com, yehuday@marvell.com MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Spam-Status: No, score=-6.2 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_MED,RP_MATCHES_RCVD,T_DKIM_INVALID,UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The user is allowed to register a reserved MSI IOVA range by using the DMA MAP API and setting the new flag: VFIO_DMA_MAP_FLAG_MSI_RESERVED_IOVA. This region is stored in the vfio_dma rb tree. At that point the iova range is not mapped to any target address yet. The host kernel will use those iova when needed, typically when MSIs are allocated. Signed-off-by: Eric Auger Signed-off-by: Bharat Bhushan --- v7 -> v8: - use iommu_msi_set_aperture function. There is no notion of unregistration anymore since the reserved msi slot remains until the container gets closed. v6 -> v7: - use iommu_free_reserved_iova_domain - convey prot attributes downto dma-reserved-iommu iova domain creation - reserved bindings teardown now performed on iommu domain destruction - rename VFIO_DMA_MAP_FLAG_MSI_RESERVED_IOVA into VFIO_DMA_MAP_FLAG_RESERVED_MSI_IOVA - change title - pass the protection attribute to dma-reserved-iommu API v3 -> v4: - use iommu_alloc/free_reserved_iova_domain exported by dma-reserved-iommu - protect vfio_register_reserved_iova_range implementation with CONFIG_IOMMU_DMA_RESERVED - handle unregistration by user-space and on vfio_iommu_type1 release v1 -> v2: - set returned value according to alloc_reserved_iova_domain result - free the iova domains in case any error occurs RFC v1 -> v1: - takes into account Alex comments, based on [RFC PATCH 1/6] vfio: Add interface for add/del reserved iova region: - use the existing dma map/unmap ioctl interface with a flag to register a reserved IOVA range. A single reserved iova region is allowed. --- drivers/vfio/vfio_iommu_type1.c | 78 ++++++++++++++++++++++++++++++++++++++++- include/uapi/linux/vfio.h | 10 +++++- 2 files changed, 86 insertions(+), 2 deletions(-) diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c index 94a9916..4d3a6f1 100644 --- a/drivers/vfio/vfio_iommu_type1.c +++ b/drivers/vfio/vfio_iommu_type1.c @@ -36,6 +36,7 @@ #include #include #include +#include #define DRIVER_VERSION "0.2" #define DRIVER_AUTHOR "Alex Williamson " @@ -445,6 +446,20 @@ static void vfio_unmap_unpin(struct vfio_iommu *iommu, struct vfio_dma *dma) vfio_lock_acct(-unlocked); } +static int vfio_set_msi_aperture(struct vfio_iommu *iommu, + dma_addr_t iova, size_t size) +{ + struct vfio_domain *d; + int ret = 0; + + list_for_each_entry(d, &iommu->domain_list, next) { + ret = iommu_msi_set_aperture(d->domain, iova, iova + size - 1); + if (ret) + break; + } + return ret; +} + static void vfio_remove_dma(struct vfio_iommu *iommu, struct vfio_dma *dma) { vfio_unmap_unpin(iommu, dma); @@ -693,6 +708,63 @@ static int vfio_dma_do_map(struct vfio_iommu *iommu, return ret; } +static int vfio_register_msi_range(struct vfio_iommu *iommu, + struct vfio_iommu_type1_dma_map *map) +{ + dma_addr_t iova = map->iova; + size_t size = map->size; + int ret = 0; + struct vfio_dma *dma; + unsigned long order; + uint64_t mask; + + /* Verify that none of our __u64 fields overflow */ + if (map->size != size || map->iova != iova) + return -EINVAL; + + order = __ffs(vfio_pgsize_bitmap(iommu)); + mask = ((uint64_t)1 << order) - 1; + + WARN_ON(mask & PAGE_MASK); + + if (!size || (size | iova) & mask) + return -EINVAL; + + /* Don't allow IOVA address wrap */ + if (iova + size - 1 < iova) + return -EINVAL; + + mutex_lock(&iommu->lock); + + if (vfio_find_dma(iommu, iova, size, VFIO_IOVA_ANY)) { + ret = -EEXIST; + goto unlock; + } + + dma = kzalloc(sizeof(*dma), GFP_KERNEL); + if (!dma) { + ret = -ENOMEM; + goto unlock; + } + + dma->iova = iova; + dma->size = size; + dma->type = VFIO_IOVA_RESERVED; + + ret = vfio_set_msi_aperture(iommu, iova, size); + if (ret) + goto free_unlock; + + vfio_link_dma(iommu, dma); + goto unlock; + +free_unlock: + kfree(dma); +unlock: + mutex_unlock(&iommu->lock); + return ret; +} + static int vfio_bus_type(struct device *dev, void *data) { struct bus_type **bus = data; @@ -1062,7 +1134,8 @@ static long vfio_iommu_type1_ioctl(void *iommu_data, } else if (cmd == VFIO_IOMMU_MAP_DMA) { struct vfio_iommu_type1_dma_map map; uint32_t mask = VFIO_DMA_MAP_FLAG_READ | - VFIO_DMA_MAP_FLAG_WRITE; + VFIO_DMA_MAP_FLAG_WRITE | + VFIO_DMA_MAP_FLAG_RESERVED_MSI_IOVA; minsz = offsetofend(struct vfio_iommu_type1_dma_map, size); @@ -1072,6 +1145,9 @@ static long vfio_iommu_type1_ioctl(void *iommu_data, if (map.argsz < minsz || map.flags & ~mask) return -EINVAL; + if (map.flags & VFIO_DMA_MAP_FLAG_RESERVED_MSI_IOVA) + return vfio_register_msi_range(iommu, &map); + return vfio_dma_do_map(iommu, &map); } else if (cmd == VFIO_IOMMU_UNMAP_DMA) { diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h index 255a211..4a9dbc2 100644 --- a/include/uapi/linux/vfio.h +++ b/include/uapi/linux/vfio.h @@ -498,12 +498,19 @@ struct vfio_iommu_type1_info { * * Map process virtual addresses to IO virtual addresses using the * provided struct vfio_dma_map. Caller sets argsz. READ &/ WRITE required. + * + * In case RESERVED_MSI_IOVA flag is set, the API only aims at registering an + * IOVA region that will be used on some platforms to map the host MSI frames. + * In that specific case, vaddr is ignored. Once registered, an MSI reserved + * IOVA region stays until the container is closed. */ struct vfio_iommu_type1_dma_map { __u32 argsz; __u32 flags; #define VFIO_DMA_MAP_FLAG_READ (1 << 0) /* readable from device */ #define VFIO_DMA_MAP_FLAG_WRITE (1 << 1) /* writable from device */ +/* reserved iova for MSI vectors*/ +#define VFIO_DMA_MAP_FLAG_RESERVED_MSI_IOVA (1 << 2) __u64 vaddr; /* Process virtual address */ __u64 iova; /* IO virtual address */ __u64 size; /* Size of mapping (bytes) */ @@ -519,7 +526,8 @@ struct vfio_iommu_type1_dma_map { * Caller sets argsz. The actual unmapped size is returned in the size * field. No guarantee is made to the user that arbitrary unmaps of iova * or size different from those used in the original mapping call will - * succeed. + * succeed. Once registered, an MSI region cannot be unmapped and stays + * until the container is closed. */ struct vfio_iommu_type1_dma_unmap { __u32 argsz;