From patchwork Mon Feb 12 18:33:25 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 10215227 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 760616055C for ; Tue, 13 Feb 2018 06:00:20 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 648D028E54 for ; Tue, 13 Feb 2018 06:00:20 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 62D2428E35; Tue, 13 Feb 2018 06:00:20 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_MED autolearn=unavailable version=3.3.1 Received: from casper.infradead.org (casper.infradead.org [85.118.1.10]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 4483F28E59 for ; Tue, 13 Feb 2018 06:00:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=casper.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:MIME-Version:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:References:In-Reply-To:Message-Id:Date:Subject:To: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=bW8Exz3xY7kgzjoDAZF4u90/0XBBLOdANHhbjNBcfQ8=; b=qrILbqVPHa8UvePmkbRv44jhdJ Q1P0W0hiKkWjMpCTCDtvva6fzL6pkijMZXjcs3zUfcTDkGcxnKivvMENHjVmN4Tp04OBp+1t3javG SKmBy7+OBygb7s8V0hawVP85kN+bGe0i3v24Pz0oH2UjNx0Us1jhEExFnqczk04ZiQQfmddmK8oCK 5pMNF5JTD++ZYFykwHWUuYQc6DhSbVWm90yaXQxZ/NqF5lNrkWUPT43xctymVHmHhRfRMwkKt0psG NTVWIEh5Ncmi43D/yjIP5GSA/RAvyd/SMCDIF5tLmPvM3zWCApK5cSPpfqUIcwqcTna/ue9JlV9yU wXajPpnw==; Received: from [198.137.202.133] (helo=bombadil.infradead.org) by casper.infradead.org with esmtps (Exim 4.89 #1 (Red Hat Linux)) id 1elLWs-0005ho-Dv; Mon, 12 Feb 2018 21:20:50 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:MIME-Version:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:References:In-Reply-To:Message-Id:Date:Subject:To: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=bW8Exz3xY7kgzjoDAZF4u90/0XBBLOdANHhbjNBcfQ8=; b=uB7LAO4RmE6jqnOcF+A0Az4C7U rn6dmRArKhpNzd0/CFU986m3kVhxdCRhcMNxSQuNGS6xnxSsp9Ng+KkGU0W104kYFnHrpG+VczVfM idoKfAfNRKaKniYAMUndb6btgMeRf6DxLn3uI5wLiq2JgAXQzRYiHd0VkWMc++w8dF3UJWcbYKvtJ QjKWf2H0cYEmyLMmMPncLQO1slams2w9qBENehWJDeybFcU8iI1Xzl8Rm18DhzAl+nm72PdMMidMt BhnsdfGK9/sjOFdK8C3Lw32R6hVbUkAjLeicsU3u4SOSO4jaSq9+RpIIeQntEdcGR98q0MzHZQr14 ic/fZXsQ==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.89 #1 (Red Hat Linux)) id 1elIxo-0002pR-6q; Mon, 12 Feb 2018 18:36:28 +0000 Received: from foss.arm.com ([217.140.101.70]) by bombadil.infradead.org with esmtp (Exim 4.89 #1 (Red Hat Linux)) id 1elItX-0003Mj-Bc for linux-arm-kernel@lists.infradead.org; Mon, 12 Feb 2018 18:32:05 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E9815164F; Mon, 12 Feb 2018 10:31:52 -0800 (PST) Received: from e106794-lin.cambridge.arm.com (e106794-lin.cambridge.arm.com [10.1.210.24]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 17E0A3F24D; Mon, 12 Feb 2018 10:31:46 -0800 (PST) From: Jean-Philippe Brucker To: linux-arm-kernel@lists.infradead.org, linux-pci@vger.kernel.org, linux-acpi@vger.kernel.org, devicetree@vger.kernel.org, iommu@lists.linux-foundation.org, kvm@vger.kernel.org Subject: [PATCH 10/37] iommu/fault: Allow blocking fault handlers Date: Mon, 12 Feb 2018 18:33:25 +0000 Message-Id: <20180212183352.22730-11-jean-philippe.brucker@arm.com> X-Mailer: git-send-email 2.15.1 In-Reply-To: <20180212183352.22730-1-jean-philippe.brucker@arm.com> References: <20180212183352.22730-1-jean-philippe.brucker@arm.com> X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, xieyisheng1@huawei.com, ilias.apalodimas@linaro.org, catalin.marinas@arm.com, xuzaibo@huawei.com, jonathan.cameron@huawei.com, will.deacon@arm.com, okaya@codeaurora.org, yi.l.liu@intel.com, lorenzo.pieralisi@arm.com, ashok.raj@intel.com, tn@semihalf.com, joro@8bytes.org, bharatku@xilinx.com, rfranz@cavium.com, lenb@kernel.org, jacob.jun.pan@linux.intel.com, alex.williamson@redhat.com, robh+dt@kernel.org, thunder.leizhen@huawei.com, bhelgaas@google.com, shunyong.yang@hxt-semitech.com, dwmw2@infradead.org, liubo95@huawei.com, rjw@rjwysocki.net, jcrouse@codeaurora.org, robdclark@gmail.com, hanjun.guo@linaro.org, sudeep.holla@arm.com, robin.murphy@arm.com, christian.koenig@amd.com, nwatters@codeaurora.org MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Allow device driver to register their fault handler at different stages of the handling path. Since we now have a fault workqueue, it is easy to call their handler from blocking context. The API borrows "handler" and "thread" terms from the IRQ subsystem, even though they don't match exactly: some IOMMU driver may report page faults from an IRQ thread instead of handler. But executing blocking fault handlers on the workqueue instead of the IRQ thread is still advantageous, because it allows to unload the low-level fault queue as fast as possible and avoid losing fault events. A driver can request to be called both in blocking and non-blocking context, so it can filter faults early and only execute the blocking code for some of them. Signed-off-by: Jean-Philippe Brucker --- drivers/iommu/io-pgfault.c | 15 +++++++++++++-- drivers/iommu/iommu.c | 12 +++++++++++- include/linux/iommu.h | 24 +++++++++++++++++++----- 3 files changed, 43 insertions(+), 8 deletions(-) diff --git a/drivers/iommu/io-pgfault.c b/drivers/iommu/io-pgfault.c index 484a39710d3f..c8f1d9bdd825 100644 --- a/drivers/iommu/io-pgfault.c +++ b/drivers/iommu/io-pgfault.c @@ -89,10 +89,20 @@ static int iommu_fault_handle_single(struct iommu_fault_context *fault) struct mm_struct *mm; struct vm_area_struct *vma; unsigned int access_flags = 0; + struct device *dev = fault->dev; int ret = IOMMU_PAGE_RESP_INVALID; unsigned int fault_flags = FAULT_FLAG_REMOTE; struct iommu_fault_event *evt = &fault->evt; + if (iommu_has_blocking_device_fault_handler(dev)) { + struct iommu_fault_param *param = dev->iommu_param->fault_param; + + ret = param->thread(evt, param->data); + if (ret != IOMMU_PAGE_RESP_CONTINUE) + return ret; + ret = IOMMU_PAGE_RESP_INVALID; + } + if (!evt->pasid_valid) return ret; @@ -272,7 +282,7 @@ int iommu_report_device_fault(struct device *dev, struct iommu_fault_event *evt) * if upper layers showed interest and installed a fault handler, * invoke it. */ - if (iommu_has_device_fault_handler(dev)) { + if (iommu_has_atomic_device_fault_handler(dev)) { struct iommu_fault_param *param = dev->iommu_param->fault_param; ret = param->handler(evt, param->data); @@ -282,7 +292,8 @@ int iommu_report_device_fault(struct device *dev, struct iommu_fault_event *evt) } /* If the handler is blocking, handle fault in the workqueue */ - if (evt->type == IOMMU_FAULT_PAGE_REQ) + if (evt->type == IOMMU_FAULT_PAGE_REQ || + iommu_has_blocking_device_fault_handler(dev)) ret = iommu_queue_fault(domain, dev, evt); return iommu_fault_complete(domain, dev, evt, ret); diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index 9bec8390694c..7f8395b620b1 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -801,7 +801,8 @@ EXPORT_SYMBOL_GPL(iommu_group_unregister_notifier); /** * iommu_register_device_fault_handler() - Register a device fault handler * @dev: the device - * @handler: the fault handler + * @handler: fault handler that can only be called in atomic context + * @thread: fault handler called from the workqueue and can block * @data: private data passed as argument to the callback * * When an IOMMU fault event is received, call this handler with the fault event @@ -810,14 +811,22 @@ EXPORT_SYMBOL_GPL(iommu_group_unregister_notifier); * fault, or return IOMMU_PAGE_RESP_HANDLED and complete the fault later by * calling iommu_page_response(). * + * At least one of @handler and @thread must be non-NULL. Both may be set, in + * which case the top-half @thread is called from the workqueue iff the + * bottom-half @handler returned IOMMU_PAGE_RESP_CONTINUE. + * * Return 0 if the fault handler was installed successfully, or an error. */ int iommu_register_device_fault_handler(struct device *dev, iommu_dev_fault_handler_t handler, + iommu_dev_fault_handler_t thread, void *data) { struct iommu_param *idata = dev->iommu_param; + if (!handler && !thread) + return -EINVAL; + /* * Device iommu_param should have been allocated when device is * added to its iommu_group. @@ -833,6 +842,7 @@ int iommu_register_device_fault_handler(struct device *dev, if (!idata->fault_param) return -ENOMEM; idata->fault_param->handler = handler; + idata->fault_param->thread = thread; idata->fault_param->data = data; return 0; diff --git a/include/linux/iommu.h b/include/linux/iommu.h index d29991be9401..36fcb579f5ed 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -431,12 +431,13 @@ struct iommu_fault_event { /** * struct iommu_fault_param - per-device IOMMU fault data - * @dev_fault_handler: Callback function to handle IOMMU faults at device level - * @data: handler private data - * + * @handler: Atomic callback to handle IOMMU faults at device level + * @thread: Blocking callback to handle IOMMU faults at device level + * @data: private data for the handler */ struct iommu_fault_param { iommu_dev_fault_handler_t handler; + iommu_dev_fault_handler_t thread; void *data; }; @@ -549,6 +550,7 @@ extern int iommu_group_unregister_notifier(struct iommu_group *group, struct notifier_block *nb); extern int iommu_register_device_fault_handler(struct device *dev, iommu_dev_fault_handler_t handler, + iommu_dev_fault_handler_t thread, void *data); extern int iommu_unregister_device_fault_handler(struct device *dev); @@ -574,7 +576,13 @@ extern void iommu_domain_window_disable(struct iommu_domain *domain, u32 wnd_nr) extern int report_iommu_fault(struct iommu_domain *domain, struct device *dev, unsigned long iova, int flags); -static inline bool iommu_has_device_fault_handler(struct device *dev) +static inline bool iommu_has_blocking_device_fault_handler(struct device *dev) +{ + return dev->iommu_param && dev->iommu_param->fault_param && + dev->iommu_param->fault_param->thread; +} + +static inline bool iommu_has_atomic_device_fault_handler(struct device *dev) { return dev->iommu_param && dev->iommu_param->fault_param && dev->iommu_param->fault_param->handler; @@ -839,6 +847,7 @@ static inline int iommu_group_unregister_notifier(struct iommu_group *group, static inline int iommu_register_device_fault_handler(struct device *dev, iommu_dev_fault_handler_t handler, + iommu_dev_fault_handler_t thread, void *data) { return 0; @@ -849,7 +858,12 @@ static inline int iommu_unregister_device_fault_handler(struct device *dev) return 0; } -static inline bool iommu_has_device_fault_handler(struct device *dev) +static inline bool iommu_has_blocking_device_fault_handler(struct device *dev) +{ + return false; +} + +static inline bool iommu_has_atomic_device_fault_handler(struct device *dev) { return false; }