From patchwork Fri Aug 2 16:39:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oleksandr Tyshchenko X-Patchwork-Id: 11073903 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D2F2D1399 for ; Fri, 2 Aug 2019 16:41:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BF405286AA for ; Fri, 2 Aug 2019 16:41:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B387B28898; Fri, 2 Aug 2019 16:41:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.0 required=2.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 1AD80286AA for ; Fri, 2 Aug 2019 16:41:21 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1htab4-0003r4-Pf; Fri, 02 Aug 2019 16:40:02 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1htab3-0003jG-N3 for xen-devel@lists.xenproject.org; Fri, 02 Aug 2019 16:40:01 +0000 X-Inumbo-ID: 2a9756b4-b544-11e9-8980-bc764e045a96 Received: from mail-lj1-x242.google.com (unknown [2a00:1450:4864:20::242]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id 2a9756b4-b544-11e9-8980-bc764e045a96; Fri, 02 Aug 2019 16:39:59 +0000 (UTC) Received: by mail-lj1-x242.google.com with SMTP id h10so1991828ljg.0 for ; Fri, 02 Aug 2019 09:39:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=tUi536buQuYUiKH6fBHUyB21xHhva7OYwGdSrgDouS8=; b=OhQJYRbC8wPW6QUTfCsBVEphNLWPtFcJk6vLuxjDTGoXt0I76omw/OXteMDk/xRtw3 6UngcBpzvgv+vvMOozbuyEHXajRJF5tGWLfoMzCxYGxGjqfAJCmubCUlFd+zDT4piySG HrYkPQsVOZ/DYkKg+9QQVs2u4Ec9PqD04SFumy2xAzjT/Jx1yiJD/3s1QqgEWCb77wj2 I3GZA9MMK5/N4Dk1Tsav7HBEew4Tr4OY+Y44kAKijQ5XviIAw2HTmfwB7BqDyDGtW3im x+CT6cqVeCA5CD+Vq4WrAenQKWBXBzxzDurkrlteadtmsu1QP9SnV3yuPkztnWL/XPI0 wX2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=tUi536buQuYUiKH6fBHUyB21xHhva7OYwGdSrgDouS8=; b=uaM+0XBRZ5Z8tr7VVadwx0DN/EtpmCZjMml7hBQ/bOCZ5viZ38U66dk7+mQ2RbegmM KE6f60n4GUtnNqc+7+u93JZuZy6ZFw9R0R4Ta0SgQ90m008CUDPp0f1k+cj7HPSw+ejq xPoVkYzW+vj3KHxCU0r5ffitRi7pR7W24ITUyZPqj1SLmF2xhmmMYvLeja/mrn1P+gg+ Mjs+Yw0UjYjFoo1Z8WnHb0nQTiepWtpurR1d5dyDuwr1MTjdfVN2iO55CB6iD0d84IH1 QJGP1OwNWxymfuvWouRSCNexnoeScciYPxrs7KKKdvOt1EwJHqpTqrWe6HezDEwNMy1r Y4Dg== X-Gm-Message-State: APjAAAV9479PG7JjfgiZ2HzWLg0sBr/TUV9CSMI5OaYBnrvelBveGORK 4lMD3AWFM1I37i0wg6ITp/qNJE7UmOY= X-Google-Smtp-Source: APXvYqwP9X/+6muLLHYVaPnqD11YWTQGSf9t8+delRGYlZEGHtX2SSxGmGTXKCw+XHQxsjlno3Cv9Q== X-Received: by 2002:a2e:8849:: with SMTP id z9mr32320025ljj.203.1564763998179; Fri, 02 Aug 2019 09:39:58 -0700 (PDT) Received: from otyshchenko.kyiv.epam.com (ll-22.209.223.85.sovam.net.ua. [85.223.209.22]) by smtp.gmail.com with ESMTPSA id q6sm15378998lji.70.2019.08.02.09.39.57 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 02 Aug 2019 09:39:57 -0700 (PDT) From: Oleksandr Tyshchenko To: xen-devel@lists.xenproject.org Date: Fri, 2 Aug 2019 19:39:40 +0300 Message-Id: <1564763985-20312-2-git-send-email-olekstysh@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1564763985-20312-1-git-send-email-olekstysh@gmail.com> References: <1564763985-20312-1-git-send-email-olekstysh@gmail.com> Subject: [Xen-devel] [PATCH V2 1/6] iommu/arm: Add iommu_helpers.c file to keep common for IOMMUs stuff X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Oleksandr Tyshchenko , julien.grall@arm.com, sstabellini@kernel.org MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Oleksandr Tyshchenko Introduce a separate file to keep various helpers which could be used by more than one IOMMU driver in order not to duplicate code. The first condidates to be moved to the new file are SMMU driver's "map_page/unmap_page" callbacks. There callbacks neither contain any SMMU specific info nor perform any SMMU specific actions and are going to be the same across all IOMMU drivers which H/W IP shares P2M with the CPU like SMMU does. So, move callbacks to iommu_helpers.c for the upcoming IPMMU driver to be able to re-use them. Signed-off-by: Oleksandr Tyshchenko --- xen/drivers/passthrough/arm/Makefile | 2 +- xen/drivers/passthrough/arm/iommu_helpers.c | 78 +++++++++++++++++++++++++++++ xen/drivers/passthrough/arm/smmu.c | 48 +----------------- xen/include/asm-arm/iommu.h | 7 +++ 4 files changed, 88 insertions(+), 47 deletions(-) create mode 100644 xen/drivers/passthrough/arm/iommu_helpers.c diff --git a/xen/drivers/passthrough/arm/Makefile b/xen/drivers/passthrough/arm/Makefile index b3efcfd..4abb87a 100644 --- a/xen/drivers/passthrough/arm/Makefile +++ b/xen/drivers/passthrough/arm/Makefile @@ -1,2 +1,2 @@ -obj-y += iommu.o +obj-y += iommu.o iommu_helpers.o obj-$(CONFIG_ARM_SMMU) += smmu.o diff --git a/xen/drivers/passthrough/arm/iommu_helpers.c b/xen/drivers/passthrough/arm/iommu_helpers.c new file mode 100644 index 0000000..53e8daa --- /dev/null +++ b/xen/drivers/passthrough/arm/iommu_helpers.c @@ -0,0 +1,78 @@ +/* + * xen/drivers/passthrough/arm/iommu_helpers.c + * + * Contains various helpers to be used by IOMMU drivers. + * + * Copyright (C) 2019 EPAM Systems Inc. + * + * This program is free software; you can redistribute it and/or + * modify it under the terms and conditions of the GNU General Public + * License, version 2, as published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * General Public License for more details. + * + * You should have received a copy of the GNU General Public + * License along with this program; If not, see . + */ + +#include +#include +#include +#include + +/* Should only be used if P2M Table is shared between the CPU and the IOMMU. */ +int __must_check arm_iommu_map_page(struct domain *d, dfn_t dfn, mfn_t mfn, + unsigned int flags, + unsigned int *flush_flags) +{ + p2m_type_t t; + + /* + * Grant mappings can be used for DMA requests. The dev_bus_addr + * returned by the hypercall is the MFN (not the IPA). For device + * protected by an IOMMU, Xen needs to add a 1:1 mapping in the domain + * p2m to allow DMA request to work. + * This is only valid when the domain is directed mapped. Hence this + * function should only be used by gnttab code with gfn == mfn == dfn. + */ + BUG_ON(!is_domain_direct_mapped(d)); + BUG_ON(mfn_x(mfn) != dfn_x(dfn)); + + /* We only support readable and writable flags */ + if ( !(flags & (IOMMUF_readable | IOMMUF_writable)) ) + return -EINVAL; + + t = (flags & IOMMUF_writable) ? p2m_iommu_map_rw : p2m_iommu_map_ro; + + /* + * The function guest_physmap_add_entry replaces the current mapping + * if there is already one... + */ + return guest_physmap_add_entry(d, _gfn(dfn_x(dfn)), _mfn(dfn_x(dfn)), 0, t); +} + +/* Should only be used if P2M Table is shared between the CPU and the IOMMU. */ +int __must_check arm_iommu_unmap_page(struct domain *d, dfn_t dfn, + unsigned int *flush_flags) +{ + /* + * This function should only be used by gnttab code when the domain + * is direct mapped (i.e. gfn == mfn == dfn). + */ + if ( !is_domain_direct_mapped(d) ) + return -EINVAL; + + return guest_physmap_remove_page(d, _gfn(dfn_x(dfn)), _mfn(dfn_x(dfn)), 0); +} + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * indent-tabs-mode: nil + * End: + */ diff --git a/xen/drivers/passthrough/arm/smmu.c b/xen/drivers/passthrough/arm/smmu.c index f151b9f..8ae986a 100644 --- a/xen/drivers/passthrough/arm/smmu.c +++ b/xen/drivers/passthrough/arm/smmu.c @@ -2734,50 +2734,6 @@ static void arm_smmu_iommu_domain_teardown(struct domain *d) xfree(xen_domain); } -static int __must_check arm_smmu_map_page(struct domain *d, dfn_t dfn, - mfn_t mfn, unsigned int flags, - unsigned int *flush_flags) -{ - p2m_type_t t; - - /* - * Grant mappings can be used for DMA requests. The dev_bus_addr - * returned by the hypercall is the MFN (not the IPA). For device - * protected by an IOMMU, Xen needs to add a 1:1 mapping in the domain - * p2m to allow DMA request to work. - * This is only valid when the domain is directed mapped. Hence this - * function should only be used by gnttab code with gfn == mfn == dfn. - */ - BUG_ON(!is_domain_direct_mapped(d)); - BUG_ON(mfn_x(mfn) != dfn_x(dfn)); - - /* We only support readable and writable flags */ - if (!(flags & (IOMMUF_readable | IOMMUF_writable))) - return -EINVAL; - - t = (flags & IOMMUF_writable) ? p2m_iommu_map_rw : p2m_iommu_map_ro; - - /* - * The function guest_physmap_add_entry replaces the current mapping - * if there is already one... - */ - return guest_physmap_add_entry(d, _gfn(dfn_x(dfn)), _mfn(dfn_x(dfn)), - 0, t); -} - -static int __must_check arm_smmu_unmap_page(struct domain *d, dfn_t dfn, - unsigned int *flush_flags) -{ - /* - * This function should only be used by gnttab code when the domain - * is direct mapped (i.e. gfn == mfn == dfn). - */ - if ( !is_domain_direct_mapped(d) ) - return -EINVAL; - - return guest_physmap_remove_page(d, _gfn(dfn_x(dfn)), _mfn(dfn_x(dfn)), 0); -} - static const struct iommu_ops arm_smmu_iommu_ops = { .init = arm_smmu_iommu_domain_init, .hwdom_init = arm_smmu_iommu_hwdom_init, @@ -2786,8 +2742,8 @@ static const struct iommu_ops arm_smmu_iommu_ops = { .iotlb_flush_all = arm_smmu_iotlb_flush_all, .assign_device = arm_smmu_assign_dev, .reassign_device = arm_smmu_reassign_dev, - .map_page = arm_smmu_map_page, - .unmap_page = arm_smmu_unmap_page, + .map_page = arm_iommu_map_page, + .unmap_page = arm_iommu_unmap_page, }; static __init const struct arm_smmu_device *find_smmu(const struct device *dev) diff --git a/xen/include/asm-arm/iommu.h b/xen/include/asm-arm/iommu.h index 904c9ae..20d865e 100644 --- a/xen/include/asm-arm/iommu.h +++ b/xen/include/asm-arm/iommu.h @@ -26,6 +26,13 @@ struct arch_iommu const struct iommu_ops *iommu_get_ops(void); void iommu_set_ops(const struct iommu_ops *ops); +/* mapping helpers */ +int __must_check arm_iommu_map_page(struct domain *d, dfn_t dfn, mfn_t mfn, + unsigned int flags, + unsigned int *flush_flags); +int __must_check arm_iommu_unmap_page(struct domain *d, dfn_t dfn, + unsigned int *flush_flags); + #endif /* __ARCH_ARM_IOMMU_H__ */ /* From patchwork Fri Aug 2 16:39:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oleksandr Tyshchenko X-Patchwork-Id: 11073913 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 82B0D13AC for ; Fri, 2 Aug 2019 16:41:44 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7254C287DB for ; Fri, 2 Aug 2019 16:41:44 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 66ABD28874; Fri, 2 Aug 2019 16:41:44 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.0 required=2.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id EC0AF28898 for ; Fri, 2 Aug 2019 16:41:43 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1htab4-0003pL-F0; Fri, 02 Aug 2019 16:40:02 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1htab3-0003jF-N3 for xen-devel@lists.xenproject.org; Fri, 02 Aug 2019 16:40:01 +0000 X-Inumbo-ID: 2b2d476e-b544-11e9-8980-bc764e045a96 Received: from mail-lj1-x241.google.com (unknown [2a00:1450:4864:20::241]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id 2b2d476e-b544-11e9-8980-bc764e045a96; Fri, 02 Aug 2019 16:40:00 +0000 (UTC) Received: by mail-lj1-x241.google.com with SMTP id z28so19232405ljn.4 for ; Fri, 02 Aug 2019 09:40:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=e4pGdvhE+q37x1oVubrx4AHVnvlXN791WoFJjsf1Djk=; b=rU3LhR+4usnQSOaXKDxOdma6fj79EoDO9No9ZHSNYS7B0EZJk/He2moq3nU23lQcru r4flqmOlvXJ09TMiTbBK8DtdWmOknzDNLPrtQsUJtvOmARxGU77XeZHRlI0PjQ2g1HM9 QOPd2OzF4uuQ9wVM7FkbZmFqKIXPF51l5RUKm1hziT5v4lVun7GwmwE3Ipwt4NqClAwW HLTPl3IpmgdsG3PwJNmHd+2NQe6C2Ic52C9tUHqZnG7sp70VHndwS4CiL7PexDsEcoii +eZnM7W0Z47hkhFtqIdwPpmBLNi6h41TPbUDY3LHu8qi7vyhLJOvF4Y78c5oOhyrDXya 7i9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=e4pGdvhE+q37x1oVubrx4AHVnvlXN791WoFJjsf1Djk=; b=WbhCEkHjt6xW6X43WkKozbH4aQXqLtqgKY5aDIovfmOdQOwZu1ooE0SRa/+kj4hzBB TC1N0u3Zqv0qyxxmirBtziZ9Qi7MzSaJlTjZD2k6SfYncdwvqLyPNMVL1RcYQtVKT0CT z7qhnkDG8cdyxepIx/3N/+WAS1YdBA/lQYwx8d7m5XvsI2LG8aQVohpsjbqSr44huy/C rQlvkLyvlhAZk8gyk9B/mlFcfP+UcHn/YSZ85ZmtbDpg0/7cfm6shfHjGyw4VseV3aiO CDo3NqRp6yklnfVQpqZBxsCAbZMyKFBgtwTcPL8icemxGX/uT4aXmdeJrZJ3wmRNDAiQ GxoA== X-Gm-Message-State: APjAAAV0sJMgdNtdN+symL0o2hREPB7ur4Mtqb0ZlwtnYuSJn0OLbO4A z6bo1wrhMFt19aht2bKbMbA88aClmeY= X-Google-Smtp-Source: APXvYqzaPoulz1r9rx+z0DrkiR0BgJsZmwFu3PSLBuy4+fcm2UZT2BtytBbpoj2v+oijjQT4yd7naQ== X-Received: by 2002:a2e:9dc1:: with SMTP id x1mr15792911ljj.0.1564763999230; Fri, 02 Aug 2019 09:39:59 -0700 (PDT) Received: from otyshchenko.kyiv.epam.com (ll-22.209.223.85.sovam.net.ua. [85.223.209.22]) by smtp.gmail.com with ESMTPSA id q6sm15378998lji.70.2019.08.02.09.39.58 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 02 Aug 2019 09:39:58 -0700 (PDT) From: Oleksandr Tyshchenko To: xen-devel@lists.xenproject.org Date: Fri, 2 Aug 2019 19:39:41 +0300 Message-Id: <1564763985-20312-3-git-send-email-olekstysh@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1564763985-20312-1-git-send-email-olekstysh@gmail.com> References: <1564763985-20312-1-git-send-email-olekstysh@gmail.com> Subject: [Xen-devel] [PATCH V2 2/6] iommu/arm: Add ability to handle deferred probing request X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Oleksandr Tyshchenko , julien.grall@arm.com, sstabellini@kernel.org MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Oleksandr Tyshchenko This patch adds minimal required support to General IOMMU framework to be able to handle a case when IOMMU driver requesting deferred probing for a device. In order not to pull Linux's error code (-EPROBE_DEFER) to Xen we have chosen -EAGAIN to be used for indicating that device probing is deferred. This is needed for the upcoming IPMMU driver which may request deferred probing depending on what device will be probed the first (there is some dependency between these devices, Root device must be registered before Cache devices. If not the case, driver will deny further Cache device probes until Root device is registered). As we can't guarantee a fixed pre-defined order for the device nodes in DT, we need to be ready for the situation where devices being probed in "any" order. Signed-off-by: Oleksandr Tyshchenko --- xen/common/device_tree.c | 1 + xen/drivers/passthrough/arm/iommu.c | 35 ++++++++++++++++++++++++++++++++++- xen/include/asm-arm/device.h | 6 +++++- xen/include/xen/device_tree.h | 1 + 4 files changed, 41 insertions(+), 2 deletions(-) diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c index e107c6f..6f37448 100644 --- a/xen/common/device_tree.c +++ b/xen/common/device_tree.c @@ -1774,6 +1774,7 @@ static unsigned long __init unflatten_dt_node(const void *fdt, /* By default the device is not protected */ np->is_protected = false; INIT_LIST_HEAD(&np->domain_list); + INIT_LIST_HEAD(&np->deferred_probe); if ( new_format ) { diff --git a/xen/drivers/passthrough/arm/iommu.c b/xen/drivers/passthrough/arm/iommu.c index 2135233..3195919 100644 --- a/xen/drivers/passthrough/arm/iommu.c +++ b/xen/drivers/passthrough/arm/iommu.c @@ -20,6 +20,12 @@ #include #include +/* + * Used to keep track of devices for which driver requested deferred probing + * (returns -EAGAIN). + */ +static LIST_HEAD(deferred_probe_list); + static const struct iommu_ops *iommu_ops; const struct iommu_ops *iommu_get_ops(void) @@ -42,7 +48,7 @@ void __init iommu_set_ops(const struct iommu_ops *ops) int __init iommu_hardware_setup(void) { - struct dt_device_node *np; + struct dt_device_node *np, *tmp; int rc; unsigned int num_iommus = 0; @@ -51,6 +57,33 @@ int __init iommu_hardware_setup(void) rc = device_init(np, DEVICE_IOMMU, NULL); if ( !rc ) num_iommus++; + else if (rc == -EAGAIN) + /* + * Driver requested deferred probing, so add this device to + * the deferred list for further processing. + */ + list_add(&np->deferred_probe, &deferred_probe_list); + } + + /* + * Process devices in the deferred list if at least one successfully + * probed device is present. + */ + while ( !list_empty(&deferred_probe_list) && num_iommus ) + { + list_for_each_entry_safe ( np, tmp, &deferred_probe_list, + deferred_probe ) + { + rc = device_init(np, DEVICE_IOMMU, NULL); + if ( !rc ) + num_iommus++; + if ( rc != -EAGAIN ) + /* + * Driver didn't request deferred probing, so remove this device + * from the deferred list. + */ + list_del_init(&np->deferred_probe); + } } return ( num_iommus > 0 ) ? 0 : -ENODEV; diff --git a/xen/include/asm-arm/device.h b/xen/include/asm-arm/device.h index 63a0f36..ee1c3bc 100644 --- a/xen/include/asm-arm/device.h +++ b/xen/include/asm-arm/device.h @@ -44,7 +44,11 @@ struct device_desc { enum device_class class; /* List of devices supported by this driver */ const struct dt_device_match *dt_match; - /* Device initialization */ + /* + * Device initialization. + * + * -EAGAIN is used to indicate that device probing is deferred. + */ int (*init)(struct dt_device_node *dev, const void *data); }; diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h index 8315629..71b0e47 100644 --- a/xen/include/xen/device_tree.h +++ b/xen/include/xen/device_tree.h @@ -93,6 +93,7 @@ struct dt_device_node { /* IOMMU specific fields */ bool is_protected; struct list_head domain_list; + struct list_head deferred_probe; struct device dev; }; From patchwork Fri Aug 2 16:39:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oleksandr Tyshchenko X-Patchwork-Id: 11073915 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0B85813AC for ; Fri, 2 Aug 2019 16:41:51 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F02202889C for ; Fri, 2 Aug 2019 16:41:50 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E267A286C8; Fri, 2 Aug 2019 16:41:50 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.0 required=2.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 6DF3B286C8 for ; Fri, 2 Aug 2019 16:41:50 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1htab7-0004AC-2r; Fri, 02 Aug 2019 16:40:05 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1htab5-0003rU-2T for xen-devel@lists.xenproject.org; Fri, 02 Aug 2019 16:40:03 +0000 X-Inumbo-ID: 2bcc9853-b544-11e9-8980-bc764e045a96 Received: from mail-lj1-x242.google.com (unknown [2a00:1450:4864:20::242]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id 2bcc9853-b544-11e9-8980-bc764e045a96; Fri, 02 Aug 2019 16:40:01 +0000 (UTC) Received: by mail-lj1-x242.google.com with SMTP id z28so19232458ljn.4 for ; Fri, 02 Aug 2019 09:40:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=PgnbDeNvPObShiypUWFNiBvgdeoIW4vOdanJkIbquHU=; b=HSXo8fMwXg+H/fqX2ZkHhDY2xTUzwB9OUKQ4yfEj/D0toSAN69fqtPsBjweF6rHLKA duBEFJIiKoRYIVzGFdXSagis9KmgzgwKZMheg1WzHBTqsSFIcvmV6mEDS4KxyYug1Vpl oVkZnP8dFWefZzXevddGUbheIjKuSv5cQBM1d1Wg+rl1F7yR727Yv2PSD3fV+sSvYSDw C2v6bZjLCDZdAyTwZXW5IsHuXphnxlxevBJTj5LoMP7Hyf7zagTOnhozVuKZ03m0aGw4 r+Of03NXaOBrWaT7IcpYaCWVGIAM366cFe0y6ntXAzhp6y3xgDEcQSl8MDrSpr5xpNa/ eMoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=PgnbDeNvPObShiypUWFNiBvgdeoIW4vOdanJkIbquHU=; b=WxUmiqP8UG3lxfC9Jg3/MSHklKx4f6feL6Ua6HhSfjkjFdnP/8JrxELTItjuHPHUhD uPd/lRiRs9xsY78xtb6GPPFRRqkDNGNgQ5nY1PX2+9XUOBdPAjQM0NBdqGkQCfgYnkOF RX2+h/SB8AaBKHhiIQcM1FAQnfo3KOGk4BEv4zXJDLJGjbN6q1tjsGgc6HXGr8EYWagm cPJndj/6H6/xZhP2EMvfUNnwmcssVEvBBV2mDqgG0Q/21ooSM2ZzrtdcUPDmpJZRAdgp 4zs0kQ7aamqPYawXg1YjcssvWOdtxY5vqj0fj6+PHan5Ufr793jxjY4S/VWaXn8RoWeK Decg== X-Gm-Message-State: APjAAAVdUy1QU3F22LXtMdG8uexM3/pxp4JxD9L4fv6S7p7GBNIeZKMW lEfCfG4YJayqa49V5aewkundLFT9zeU= X-Google-Smtp-Source: APXvYqxegN9WyW1h8rNRz4pRvBMM9euzIxGil77e1rujkTwpelpMo+uHq4RcCihb5QeKNPYhQN2Yag== X-Received: by 2002:a2e:3604:: with SMTP id d4mr21245116lja.85.1564764000324; Fri, 02 Aug 2019 09:40:00 -0700 (PDT) Received: from otyshchenko.kyiv.epam.com (ll-22.209.223.85.sovam.net.ua. [85.223.209.22]) by smtp.gmail.com with ESMTPSA id q6sm15378998lji.70.2019.08.02.09.39.59 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 02 Aug 2019 09:39:59 -0700 (PDT) From: Oleksandr Tyshchenko To: xen-devel@lists.xenproject.org Date: Fri, 2 Aug 2019 19:39:42 +0300 Message-Id: <1564763985-20312-4-git-send-email-olekstysh@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1564763985-20312-1-git-send-email-olekstysh@gmail.com> References: <1564763985-20312-1-git-send-email-olekstysh@gmail.com> Subject: [Xen-devel] [PATCH V2 3/6] [RFC] xen/common: Introduce _xrealloc function X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: sstabellini@kernel.org, Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Ian Jackson , Tim Deegan , Oleksandr Tyshchenko , julien.grall@arm.com, Jan Beulich MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Oleksandr Tyshchenko Next patch in this series will make use of it. Original patch was initially posted by Sameer Goel: https://lists.xen.org/archives/html/xen-devel/2017-06/msg00858.html This could be considered as another attempt to add it: https://www.mail-archive.com/kexec@lists.infradead.org/msg21335.html Signed-off-by: Oleksandr Tyshchenko CC: Andrew Cooper CC: George Dunlap CC: Ian Jackson CC: Jan Beulich CC: Julien Grall CC: Konrad Rzeszutek Wilk CC: Stefano Stabellini CC: Tim Deegan CC: Wei Liu --- [As it was previously discussed with Julien in IRC] The reason for this patch to be an RFC is that patch itself is not completely correct and I don't fully understand what/how should be done for this patch to be accepted. Or whether community even wants this to go in. So, to avoid bike shedding, the first target is to collect feedback from the maintainers. In a nutshell, the upcoming "iommu_fwspec" support on ARM is going to use xrealloc when adding new device ID. We really want to have "iommu_fwspec" support which will give us a generic abstract way to add new device to the IOMMU based on the generic IOMMU DT binding. This is how Linux does: https://github.com/torvalds/linux/blob/master/drivers/iommu/iommu.c#L2257 and we are doing the similar in next patch of this thread: "iommu/arm: Add lightweight iommu_fwspec support" --- xen/common/xmalloc_tlsf.c | 21 +++++++++++++++++++++ xen/include/xen/xmalloc.h | 1 + 2 files changed, 22 insertions(+) diff --git a/xen/common/xmalloc_tlsf.c b/xen/common/xmalloc_tlsf.c index 2076953..c080763 100644 --- a/xen/common/xmalloc_tlsf.c +++ b/xen/common/xmalloc_tlsf.c @@ -610,6 +610,27 @@ void *_xzalloc(unsigned long size, unsigned long align) return p ? memset(p, 0, size) : p; } +void *_xrealloc(void *p, unsigned long new_size, unsigned long align) +{ + void *new_p; + + if ( !new_size ) + { + xfree(p); + return NULL; + } + + new_p = _xmalloc(new_size, align); + + if ( new_p && p ) + { + memcpy(new_p, p, new_size); + xfree(p); + } + + return new_p; +} + void xfree(void *p) { struct bhdr *b; diff --git a/xen/include/xen/xmalloc.h b/xen/include/xen/xmalloc.h index b486fe4..63961ef 100644 --- a/xen/include/xen/xmalloc.h +++ b/xen/include/xen/xmalloc.h @@ -51,6 +51,7 @@ extern void xfree(void *); /* Underlying functions */ extern void *_xmalloc(unsigned long size, unsigned long align); extern void *_xzalloc(unsigned long size, unsigned long align); +extern void *_xrealloc(void *p, unsigned long new_size, unsigned long align); static inline void *_xmalloc_array( unsigned long size, unsigned long align, unsigned long num) From patchwork Fri Aug 2 16:39:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oleksandr Tyshchenko X-Patchwork-Id: 11073905 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CE37413AC for ; Fri, 2 Aug 2019 16:41:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BC9B6286B0 for ; Fri, 2 Aug 2019 16:41:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B098A286D0; Fri, 2 Aug 2019 16:41:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.0 required=2.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 0479B286B0 for ; Fri, 2 Aug 2019 16:41:28 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1htab7-0004CB-Lf; Fri, 02 Aug 2019 16:40:05 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1htab6-00045X-JV for xen-devel@lists.xenproject.org; Fri, 02 Aug 2019 16:40:04 +0000 X-Inumbo-ID: 2c5f3edf-b544-11e9-8980-bc764e045a96 Received: from mail-lj1-x241.google.com (unknown [2a00:1450:4864:20::241]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id 2c5f3edf-b544-11e9-8980-bc764e045a96; Fri, 02 Aug 2019 16:40:02 +0000 (UTC) Received: by mail-lj1-x241.google.com with SMTP id h10so1991972ljg.0 for ; Fri, 02 Aug 2019 09:40:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=NwFO0JUoJVO1HNR1vM9cd7WLDQQW3WhdzeYYsj8V4JE=; b=oUJGHIsrM2oUNer+vweAYjHDxMewjF/WWi95Sbsvivg3oelzehvcPYEwvKBMypJuJn +QH7yldXLsTQ3bcoNV38+35dPcoCPm49DLvjcheckHVtfvWRTA8n4RgliRpHQLQKucXK 3F1rPV+ZkEIeMyAUhx4vq8kkU8h87CvDJ6IkpsRdPqx4dNVp0A1mLe+U4Xn26yyH0Kra fztfcDeMioD9yuT+ItKoclZxAoA8AEAVmmaTclHJgWb7JtgE14PcjXW3qpf4T83/ABuV Se7x84ahH8TfgdBsslpKxR1Ztz4fhIdPv+QhvqWtoYLfCN1fsfEFrwOisMhObc6CeHkd lD+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=NwFO0JUoJVO1HNR1vM9cd7WLDQQW3WhdzeYYsj8V4JE=; b=YHmXLbIY3uTEndmP/C/S2+p1YWWBsb44MkRSCSqz7q97PUsA5feyeRWChbIjc34nqt GI1jfj8/98hVhV6XpAW5coj1BYA5gPotnDOWdDCjv/SbNNvzx6HZ8lWZ2c5KRAvjo9jY C18CWSUyVGFppHc1XHS8dqE7xPeL2Q21pUbbg7xKBUQqLIJ8KIcBR1ueG72Aw0FYn0P1 b16enHkmKegnv17Wd25a2CdFwcgR+6BQtTsRRxoVPVdUDWjhvU6Mm+Ffg5DdXCdqK3sV htRHomfQRmLEIXzdiIQX4VZG/VQt84LsYAzEWph6dLYR6Va+DYpTQzPpHkvYLV1u8RBx DjUQ== X-Gm-Message-State: APjAAAXBENj+zKam1rE1fSSX5L8MWCFMc8i8huPxt++rqId7cJoo7Njb XJBcyZw8t496eOozRv0sma1HtQIJeDY= X-Google-Smtp-Source: APXvYqwLtf6T942/1TBvC/LrxsPIOY6a6JDAtge32hzILLYWefQUYUP56UmjKKjBXzUrsJgrkCPKzA== X-Received: by 2002:a2e:9643:: with SMTP id z3mr73065183ljh.43.1564764001224; Fri, 02 Aug 2019 09:40:01 -0700 (PDT) Received: from otyshchenko.kyiv.epam.com (ll-22.209.223.85.sovam.net.ua. [85.223.209.22]) by smtp.gmail.com with ESMTPSA id q6sm15378998lji.70.2019.08.02.09.40.00 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 02 Aug 2019 09:40:00 -0700 (PDT) From: Oleksandr Tyshchenko To: xen-devel@lists.xenproject.org Date: Fri, 2 Aug 2019 19:39:43 +0300 Message-Id: <1564763985-20312-5-git-send-email-olekstysh@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1564763985-20312-1-git-send-email-olekstysh@gmail.com> References: <1564763985-20312-1-git-send-email-olekstysh@gmail.com> Subject: [Xen-devel] [PATCH V2 4/6] iommu/arm: Add lightweight iommu_fwspec support X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Oleksandr Tyshchenko , julien.grall@arm.com, sstabellini@kernel.org MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Oleksandr Tyshchenko We need to have some abstract way to add new device to the IOMMU based on the generic IOMMU DT binding [1] which can be used for both DT (right now) and ACPI (in future). For that reason we can borrow the idea used in Linux these days called "iommu_fwspec". Having this in, it will be possible to configure IOMMU master interfaces of the device (device IDs) from a single common place and avoid keeping almost identifical look-up implementations in each IOMMU driver. There is no need to port the whole implementation of "iommu_fwspec" to Xen, we could, probably, end up with a much simpler solution, some "stripped down" version which fits our requirments. So, this patch adds the following: 1. A common structure "iommu_fwspec" to hold the the per-device firmware data 2. New member "iommu_fwspec" of struct device 3. Functions/helpers to deal with "dev->iommu_fwspec" It should be noted that in comparing with original "iommu_fwspec" Xen's variant doesn't contain some fields, which are not really needed at the moment (ops, flag) and "iommu_fwnode" field was replaced by "iommu_dev" to avoid porting a lot of code (to support "fwnode_handle") with little benefit. Next patch in this series will make use of that support. [1] https://www.kernel.org/doc/Documentation/devicetree/bindings/iommu/iommu.txt Signed-off-by: Oleksandr Tyshchenko --- xen/drivers/passthrough/arm/Makefile | 2 +- xen/drivers/passthrough/arm/iommu_fwspec.c | 91 ++++++++++++++++++++++++++++++ xen/include/asm-arm/device.h | 1 + xen/include/asm-arm/iommu.h | 2 + xen/include/asm-arm/iommu_fwspec.h | 65 +++++++++++++++++++++ 5 files changed, 160 insertions(+), 1 deletion(-) create mode 100644 xen/drivers/passthrough/arm/iommu_fwspec.c create mode 100644 xen/include/asm-arm/iommu_fwspec.h diff --git a/xen/drivers/passthrough/arm/Makefile b/xen/drivers/passthrough/arm/Makefile index 4abb87a..5fbad45 100644 --- a/xen/drivers/passthrough/arm/Makefile +++ b/xen/drivers/passthrough/arm/Makefile @@ -1,2 +1,2 @@ -obj-y += iommu.o iommu_helpers.o +obj-y += iommu.o iommu_helpers.o iommu_fwspec.o obj-$(CONFIG_ARM_SMMU) += smmu.o diff --git a/xen/drivers/passthrough/arm/iommu_fwspec.c b/xen/drivers/passthrough/arm/iommu_fwspec.c new file mode 100644 index 0000000..3474192 --- /dev/null +++ b/xen/drivers/passthrough/arm/iommu_fwspec.c @@ -0,0 +1,91 @@ +/* + * xen/drivers/passthrough/arm/iommu_fwspec.c + * + * Contains functions to maintain per-device firmware data + * + * Based on Linux's iommu_fwspec support you can find at: + * drivers/iommu/iommu.c + * + * Copyright (C) 2019 EPAM Systems Inc. + * + * This program is free software; you can redistribute it and/or + * modify it under the terms and conditions of the GNU General Public + * License, version 2, as published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * General Public License for more details. + * + * You should have received a copy of the GNU General Public + * License along with this program; If not, see . + */ + +#include +#include +#include +#include + +int iommu_fwspec_init(struct device *dev, struct device *iommu_dev) +{ + struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); + + if ( fwspec ) + return 0; + + fwspec = xzalloc(struct iommu_fwspec); + if ( !fwspec ) + return -ENOMEM; + + fwspec->iommu_dev = iommu_dev; + dev_iommu_fwspec_set(dev, fwspec); + + return 0; +} + +void iommu_fwspec_free(struct device *dev) +{ + struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); + + if ( fwspec ) + { + xfree(fwspec); + dev_iommu_fwspec_set(dev, NULL); + } +} + +int iommu_fwspec_add_ids(struct device *dev, uint32_t *ids, int num_ids) +{ + struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); + size_t size; + int i; + + if ( !fwspec ) + return -EINVAL; + + size = offsetof(struct iommu_fwspec, ids[fwspec->num_ids + num_ids]); + if ( size > sizeof(*fwspec) ) + { + fwspec = _xrealloc(fwspec, size, sizeof(void *)); + if ( !fwspec ) + return -ENOMEM; + + dev_iommu_fwspec_set(dev, fwspec); + } + + for ( i = 0; i < num_ids; i++ ) + fwspec->ids[fwspec->num_ids + i] = ids[i]; + + fwspec->num_ids += num_ids; + + return 0; +} + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * indent-tabs-mode: nil + * End: + */ diff --git a/xen/include/asm-arm/device.h b/xen/include/asm-arm/device.h index ee1c3bc..ee7cff2 100644 --- a/xen/include/asm-arm/device.h +++ b/xen/include/asm-arm/device.h @@ -18,6 +18,7 @@ struct device struct dt_device_node *of_node; /* Used by drivers imported from Linux */ #endif struct dev_archdata archdata; + struct iommu_fwspec *iommu_fwspec; /* per-device IOMMU instance data */ }; typedef struct device device_t; diff --git a/xen/include/asm-arm/iommu.h b/xen/include/asm-arm/iommu.h index 20d865e..1853bd9 100644 --- a/xen/include/asm-arm/iommu.h +++ b/xen/include/asm-arm/iommu.h @@ -14,6 +14,8 @@ #ifndef __ARCH_ARM_IOMMU_H__ #define __ARCH_ARM_IOMMU_H__ +#include + struct arch_iommu { /* Private information for the IOMMU drivers */ diff --git a/xen/include/asm-arm/iommu_fwspec.h b/xen/include/asm-arm/iommu_fwspec.h new file mode 100644 index 0000000..0676285 --- /dev/null +++ b/xen/include/asm-arm/iommu_fwspec.h @@ -0,0 +1,65 @@ +/* + * xen/include/asm-arm/iommu_fwspec.h + * + * Contains a common structure to hold the per-device firmware data and + * declaration of functions used to maintain that data + * + * Based on Linux's iommu_fwspec support you can find at: + * include/linux/iommu.h + * + * Copyright (C) 2019 EPAM Systems Inc. + * + * This program is free software; you can redistribute it and/or + * modify it under the terms and conditions of the GNU General Public + * License, version 2, as published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * General Public License for more details. + * + * You should have received a copy of the GNU General Public + * License along with this program; If not, see . + */ + +#ifndef __ARCH_ARM_IOMMU_FWSPEC_H__ +#define __ARCH_ARM_IOMMU_FWSPEC_H__ + +/* per-device IOMMU instance data */ +struct iommu_fwspec { + /* device which represents this IOMMU H/W */ + struct device *iommu_dev; + /* IOMMU driver private data for this device */ + void *iommu_priv; + /* number of associated device IDs */ + unsigned int num_ids; + /* IDs which this device may present to the IOMMU */ + uint32_t ids[1]; +}; + +int iommu_fwspec_init(struct device *dev, struct device *iommu_dev); +void iommu_fwspec_free(struct device *dev); +int iommu_fwspec_add_ids(struct device *dev, uint32_t *ids, int num_ids); + +static inline struct iommu_fwspec *dev_iommu_fwspec_get(struct device *dev) +{ + return dev->iommu_fwspec; +} + +static inline void dev_iommu_fwspec_set(struct device *dev, + struct iommu_fwspec *fwspec) +{ + dev->iommu_fwspec = fwspec; +} + +#endif /* __ARCH_ARM_IOMMU_FWSPEC_H__ */ + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * tab-width: 4 + * indent-tabs-mode: nil + * End: + */ From patchwork Fri Aug 2 16:39:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oleksandr Tyshchenko X-Patchwork-Id: 11073911 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6FA761399 for ; Fri, 2 Aug 2019 16:41:41 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 603F428898 for ; Fri, 2 Aug 2019 16:41:41 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 54A48287DB; Fri, 2 Aug 2019 16:41:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.0 required=2.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id DCFC728897 for ; Fri, 2 Aug 2019 16:41:40 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1htab8-0004EO-3i; Fri, 02 Aug 2019 16:40:06 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1htab6-00045a-K6 for xen-devel@lists.xenproject.org; Fri, 02 Aug 2019 16:40:04 +0000 X-Inumbo-ID: 2ce3f0c6-b544-11e9-8980-bc764e045a96 Received: from mail-lj1-x242.google.com (unknown [2a00:1450:4864:20::242]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id 2ce3f0c6-b544-11e9-8980-bc764e045a96; Fri, 02 Aug 2019 16:40:03 +0000 (UTC) Received: by mail-lj1-x242.google.com with SMTP id m8so39844750lji.7 for ; Fri, 02 Aug 2019 09:40:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=lrPG3Jo0g8IN2TUDjww3PLkkrQlh1VPumDr3dhWVCkk=; b=AzXW2EXtSUupxXxNaZI/YiFCMQp5KsnR3CSGBrX0/FWxjqteAg/NJcHGoV39p22wEK ZD1MTM7ntirqrOURfmnyLK9VYfVZ4QjVVWA3GSZixvoqzQluRbL5NDAcZfI4533z/yg6 nwC16orac8Zixl0zqjfcbcoTuTGHcqiW2M+/bGQ0VwczyuHcclR58k/nq4Ue1ksvM9Di s7JdMSCJ5Snb+aRGsVS6+OIY6AfP+260RktlLTWmLcWEZDzrBCc5dcSm7SjaQCbCjprn k2NSSPQjJy50L5qCLDT9CnZF3wye4TN6MAUkpTxKuFORVDnXAQnoIjFIp4kcS1t7/vmA Ye9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=lrPG3Jo0g8IN2TUDjww3PLkkrQlh1VPumDr3dhWVCkk=; b=OTo5/2hgjgku6DmcOp+j96Ad9FqErIQQuXLHTRoTaXk1dWDwbqnKRTUpWkvISGb6fb oyh51n3UNAjx7iYRV6bmqWB5ZzXqhpejHLqZPnHaQ/5cZC35h29Z8/8OB2eneQxMvE3i JLMe0m3AjiYhWaMe7bvA43593pKk5P/ZBZm9daB1CQolhvqmjstMdOBmvLCF9VTyB6tA Fdr/0SgIe1mFZuAiWDngwiBilySxdbcfao1iw0ZwFXdPg7mEzV1RpkHbWac413ajsHMY Cu1VGAxBp7V0Ex4mank0xXZlqFD53GG1zdTtgQg3SIOGbn0g+7Gwk9XnNmHvfOiia+PM HaoA== X-Gm-Message-State: APjAAAUZ4QfG5/bV3K5zHsp1JLpgFoafHxCXMVlPDXRrO0LqfyOLw/HZ HKJ+cHRdE/Lq6CQmH5YV4PN6FQkpii8= X-Google-Smtp-Source: APXvYqxE60L7NqElsqqOOQetv4stCpru4Ah77SbcciphJ2rp20lCkcoA+xv/WPHXFPa78agvCxlCuA== X-Received: by 2002:a2e:9f0d:: with SMTP id u13mr25560786ljk.186.1564764002116; Fri, 02 Aug 2019 09:40:02 -0700 (PDT) Received: from otyshchenko.kyiv.epam.com (ll-22.209.223.85.sovam.net.ua. [85.223.209.22]) by smtp.gmail.com with ESMTPSA id q6sm15378998lji.70.2019.08.02.09.40.01 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 02 Aug 2019 09:40:01 -0700 (PDT) From: Oleksandr Tyshchenko To: xen-devel@lists.xenproject.org Date: Fri, 2 Aug 2019 19:39:44 +0300 Message-Id: <1564763985-20312-6-git-send-email-olekstysh@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1564763985-20312-1-git-send-email-olekstysh@gmail.com> References: <1564763985-20312-1-git-send-email-olekstysh@gmail.com> Subject: [Xen-devel] [PATCH V2 5/6] iommu/arm: Introduce iommu_add_dt_device API X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Oleksandr Tyshchenko , julien.grall@arm.com, sstabellini@kernel.org MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Oleksandr Tyshchenko This patch adds new iommu_add_dt_device API for adding DT device to the IOMMU using generic IOMMU DT binding [1] and previously added "iommu_fwspec" support. New function parses the DT binding, prepares "dev->iommu_fwspec" with correct information and calls the IOMMU driver using "add_device" callback to register new DT device. The IOMMU driver's responsibility is to check whether "dev->iommu_fwspec" is initialized and mark that device as protected. The additional benefit here is to avoid to go through the whole DT multiple times in IOMMU driver trying to locate master devices which belong to each IOMMU device being probed. The upcoming IPMMU driver will have "add_device" callback implemented. I hope, this patch won't break SMMU driver's functionality, which doesn't have this callback implemented. [1] https://www.kernel.org/doc/Documentation/devicetree/bindings/iommu/iommu.txt Signed-off-by: Oleksandr Tyshchenko --- xen/arch/arm/domain_build.c | 12 ++++++++++ xen/drivers/passthrough/arm/iommu.c | 45 +++++++++++++++++++++++++++++++++++++ xen/include/asm-arm/iommu.h | 3 +++ 3 files changed, 60 insertions(+) diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c index d983677..d67f7d4 100644 --- a/xen/arch/arm/domain_build.c +++ b/xen/arch/arm/domain_build.c @@ -1241,6 +1241,18 @@ static int __init handle_device(struct domain *d, struct dt_device_node *dev, u64 addr, size; bool need_mapping = !dt_device_for_passthrough(dev); + if ( dt_parse_phandle(dev, "iommus", 0) ) + { + dt_dprintk("%s add to iommu\n", dt_node_full_name(dev)); + res = iommu_add_dt_device(dev); + if ( res ) + { + printk(XENLOG_ERR "Failed to add %s to the IOMMU\n", + dt_node_full_name(dev)); + return res; + } + } + nirq = dt_number_of_irq(dev); naddr = dt_number_of_address(dev); diff --git a/xen/drivers/passthrough/arm/iommu.c b/xen/drivers/passthrough/arm/iommu.c index 3195919..19516af 100644 --- a/xen/drivers/passthrough/arm/iommu.c +++ b/xen/drivers/passthrough/arm/iommu.c @@ -113,3 +113,48 @@ int arch_iommu_populate_page_table(struct domain *d) void __hwdom_init arch_iommu_hwdom_init(struct domain *d) { } + +int __init iommu_add_dt_device(struct dt_device_node *np) +{ + const struct iommu_ops *ops = iommu_get_ops(); + struct dt_phandle_args iommu_spec; + struct device *dev = dt_to_dev(np); + int rc = 1, index = 0; + + if ( !iommu_enabled || !ops || !ops->add_device ) + return 0; + + if ( dev_iommu_fwspec_get(dev) ) + return -EEXIST; + + /* According to the Documentation/devicetree/bindings/iommu/iommu.txt */ + while ( !dt_parse_phandle_with_args(np, "iommus", "#iommu-cells", + index, &iommu_spec) ) + { + if ( !dt_device_is_available(iommu_spec.np) ) + break; + + rc = iommu_fwspec_init(dev, &iommu_spec.np->dev); + if ( rc ) + break; + + rc = iommu_fwspec_add_ids(dev, iommu_spec.args, 1); + if ( rc ) + break; + + index++; + } + + /* + * Add DT device to the IOMMU if latter is present and available. + * The IOMMU driver's responsibility is to check whether dev->iommu_fwspec + * field is initialized and mark that device as protected. + */ + if ( !rc ) + rc = ops->add_device(0, dev); + + if ( rc < 0 ) + iommu_fwspec_free(dev); + + return rc < 0 ? rc : 0; +} diff --git a/xen/include/asm-arm/iommu.h b/xen/include/asm-arm/iommu.h index 1853bd9..06b07fa 100644 --- a/xen/include/asm-arm/iommu.h +++ b/xen/include/asm-arm/iommu.h @@ -28,6 +28,9 @@ struct arch_iommu const struct iommu_ops *iommu_get_ops(void); void iommu_set_ops(const struct iommu_ops *ops); +/* helper to add DT device to the IOMMU */ +int iommu_add_dt_device(struct dt_device_node *np); + /* mapping helpers */ int __must_check arm_iommu_map_page(struct domain *d, dfn_t dfn, mfn_t mfn, unsigned int flags, From patchwork Fri Aug 2 16:39:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oleksandr Tyshchenko X-Patchwork-Id: 11073909 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A806A1399 for ; Fri, 2 Aug 2019 16:41:36 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9243728899 for ; Fri, 2 Aug 2019 16:41:36 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 86AB7287DB; Fri, 2 Aug 2019 16:41:36 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.0 required=2.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 3870028897 for ; Fri, 2 Aug 2019 16:41:34 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1htabB-0004KQ-GQ; Fri, 02 Aug 2019 16:40:09 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1htabA-0004Ju-Kz for xen-devel@lists.xenproject.org; Fri, 02 Aug 2019 16:40:08 +0000 X-Inumbo-ID: 2dde1dea-b544-11e9-8980-bc764e045a96 Received: from mail-lj1-x243.google.com (unknown [2a00:1450:4864:20::243]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id 2dde1dea-b544-11e9-8980-bc764e045a96; Fri, 02 Aug 2019 16:40:05 +0000 (UTC) Received: by mail-lj1-x243.google.com with SMTP id i21so73458244ljj.3 for ; Fri, 02 Aug 2019 09:40:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=7fl86rQ4RYRXOdn3LVY+3kxp1DY9QQ71ET6EMKzn0/I=; b=kgzv1VGrDH7bg/Eak3mVmCOT90ECeAmN5mTmWppAvmuwHO7ECHWHE3fvCk9FZI7nIN 9BAClPCqpxx2U/W546cgaVpy6SlWC52TmFiqBcydROpn6oVkaNhiziB5RIEAANTjwRFT QwEwLohfY6oTJeCdh4tMzoL7azdJnpy9BDcYIsCKyE8zg+GD8Ge0Mjgi+OR0XRh41AYz fmpUxGiWoho56skRVgnX0fSrj1KQgi8vV47X/Wpg7U+ed/sMpWp2tro1IrYVW4k95HE3 6/31nKXsIy1GqBDtdMw5/2k/8K/chGCgjAoUNUJnlE3B8jgbJi2iiAxgBXXuqtsmglkf ACmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=7fl86rQ4RYRXOdn3LVY+3kxp1DY9QQ71ET6EMKzn0/I=; b=WlsOWTqsb6KeeEWitF60JClABB7gtnycBkoEywau5GnlQnJlGu+YTxsYFyndbu/UhR EwC3z1eNCkBI5cqiCixdWdfi/iZKOT0lTDThRQ2bFFNcuY31FRWTVPQpSL0QQdjKMX/U UvbVV+xqCW/RdoFoNbzpH+fs8D66b3O1LPv+ruhWRAFjblYKbVlJ9JVGYjW8u5AfXgpS 5Dwv++ZsWrpSuV10s/pnw8NCcmr/h1nXBzGjhMpTG2kOCngPr317wcnXIIS2WvbUhBN5 bl7XhDiwM7lDHFQSFgqjrKrrS+ioEXFOXR1PkZFdczjPQo6j8c8etKAPp4e1t+oENVeW a0DA== X-Gm-Message-State: APjAAAVJpB7B7vIPtzd0MeuS6XUYIIVLBfjPIvW7MqY1JEjoBEvegMYA p7JevlhuNC5f/Cd0KJUuLDTLgTsMrJc= X-Google-Smtp-Source: APXvYqz+x8TNfIkP9neZJLqxF3+h/kchuoP9aKjgp21nPS4AAosTkLoOHnEd8Fa23FlOIP9/96wvXQ== X-Received: by 2002:a2e:9dca:: with SMTP id x10mr70808720ljj.17.1564764003270; Fri, 02 Aug 2019 09:40:03 -0700 (PDT) Received: from otyshchenko.kyiv.epam.com (ll-22.209.223.85.sovam.net.ua. [85.223.209.22]) by smtp.gmail.com with ESMTPSA id q6sm15378998lji.70.2019.08.02.09.40.02 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 02 Aug 2019 09:40:02 -0700 (PDT) From: Oleksandr Tyshchenko To: xen-devel@lists.xenproject.org Date: Fri, 2 Aug 2019 19:39:45 +0300 Message-Id: <1564763985-20312-7-git-send-email-olekstysh@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1564763985-20312-1-git-send-email-olekstysh@gmail.com> References: <1564763985-20312-1-git-send-email-olekstysh@gmail.com> Subject: [Xen-devel] [PATCH V2 6/6] iommu/arm: Add Renesas IPMMU-VMSA support X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Oleksandr Tyshchenko , julien.grall@arm.com, sstabellini@kernel.org, Yoshihiro Shimoda MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Oleksandr Tyshchenko The IPMMU-VMSA is VMSA-compatible I/O Memory Management Unit (IOMMU) which provides address translation and access protection functionalities to processing units and interconnect networks. Please note, current driver is supposed to work only with newest Gen3 SoCs revisions which IPMMU hardware supports stage 2 translation table format and is able to use CPU's P2M table as is if one is 3-level page table (up to 40 bit IPA). The major differences compare to the Linux driver are: 1. Stage 1/Stage 2 translation. Linux driver supports Stage 1 translation only (with Stage 1 translation table format). It manages page table by itself. But Xen driver supports Stage 2 translation (with Stage 2 translation table format) to be able to share the P2M with the CPU. Stage 1 translation is always bypassed in Xen driver. So, Xen driver is supposed to be used with newest Gen3 SoC revisions only (H3 ES3.0, M3 ES3.0, etc.) which IPMMU H/W supports stage 2 translation table format. 2. AArch64 support. Linux driver uses VMSAv8-32 mode, while Xen driver enables Armv8 VMSAv8-64 mode to cover up to 40 bit input address. 3. Context bank (sets of page table) usage. In Xen, each context bank is mapped to one Xen domain. So, all devices being pass throughed to the same Xen domain share the same context bank. Signed-off-by: Oleksandr Tyshchenko CC: Julien Grall CC: Yoshihiro Shimoda --- Changes V1 -> V2: - rewrited driver to use iommu_fwspec - removed DT parsing code for micro-TLBs - removed struct ipmmu_vmsa_master_cfg, dev_archdata macro - added ipmmu_find_mmu_by_dev(), various helpers to access fwspec->iommu_priv - implemented new callback "add_device" to add master device to IPMMU - removed ipmmu_protect_masters() - removed code to locate Root device in the first place, used EAGAIN to request deferred probing - used printk_once for the system wide error messages in ipmmu_init() which don't need to be shown for every device being probed - removed map_page/unmap_page implementation, reused them from iommu_helpers.c - used %pd for printing domaid id - performed various cosmetic fixes - changed u32 -> uint32_t, u64 -> uint64_t, unsigned int -> uint32_t where needed - clarified TODOs - clafiried supported SoC versions in config IPMMU_VMSA, set default to "n" - updated comments in code, provided more accurate description, added new comments where needed - updated patch description by providing differences between Linux/Xen implementations - removed fields for cache snoop transaction when configuring IMTTBCR (update from Renesas BSP) --- xen/arch/arm/platforms/Kconfig | 1 + xen/drivers/passthrough/Kconfig | 13 + xen/drivers/passthrough/arm/Makefile | 1 + xen/drivers/passthrough/arm/ipmmu-vmsa.c | 1342 ++++++++++++++++++++++++++++++ 4 files changed, 1357 insertions(+) create mode 100644 xen/drivers/passthrough/arm/ipmmu-vmsa.c diff --git a/xen/arch/arm/platforms/Kconfig b/xen/arch/arm/platforms/Kconfig index bc0e9cd..c93a6b2 100644 --- a/xen/arch/arm/platforms/Kconfig +++ b/xen/arch/arm/platforms/Kconfig @@ -25,6 +25,7 @@ config RCAR3 bool "Renesas RCar3 support" depends on ARM_64 select HAS_SCIF + select IPMMU_VMSA ---help--- Enable all the required drivers for Renesas RCar3 diff --git a/xen/drivers/passthrough/Kconfig b/xen/drivers/passthrough/Kconfig index a3c0649..3daee16 100644 --- a/xen/drivers/passthrough/Kconfig +++ b/xen/drivers/passthrough/Kconfig @@ -12,4 +12,17 @@ config ARM_SMMU Say Y here if your SoC includes an IOMMU device implementing the ARM SMMU architecture. + +config IPMMU_VMSA + bool "Renesas IPMMU-VMSA found in R-Car Gen3 SoCs" + default n + depends on ARM_64 + ---help--- + Support for implementations of the Renesas IPMMU-VMSA found + in R-Car Gen3 SoCs. + + Say Y here if you are using newest R-Car Gen3 SoCs revisions + (H3 ES3.0, M3 ES3.0, etc) which IPMMU hardware supports stage 2 + translation table format and is able to use CPU's P2M table as is. + endif diff --git a/xen/drivers/passthrough/arm/Makefile b/xen/drivers/passthrough/arm/Makefile index 5fbad45..fcd918e 100644 --- a/xen/drivers/passthrough/arm/Makefile +++ b/xen/drivers/passthrough/arm/Makefile @@ -1,2 +1,3 @@ obj-y += iommu.o iommu_helpers.o iommu_fwspec.o obj-$(CONFIG_ARM_SMMU) += smmu.o +obj-$(CONFIG_IPMMU_VMSA) += ipmmu-vmsa.o diff --git a/xen/drivers/passthrough/arm/ipmmu-vmsa.c b/xen/drivers/passthrough/arm/ipmmu-vmsa.c new file mode 100644 index 0000000..a34a8f8 --- /dev/null +++ b/xen/drivers/passthrough/arm/ipmmu-vmsa.c @@ -0,0 +1,1342 @@ +/* + * xen/drivers/passthrough/arm/ipmmu-vmsa.c + * + * Driver for the Renesas IPMMU-VMSA found in R-Car Gen3 SoCs. + * + * The IPMMU-VMSA is VMSA-compatible I/O Memory Management Unit (IOMMU) + * which provides address translation and access protection functionalities + * to processing units and interconnect networks. + * + * Please note, current driver is supposed to work only with newest Gen3 SoCs + * revisions which IPMMU hardware supports stage 2 translation table format and + * is able to use CPU's P2M table as is. + * + * Based on Linux's IPMMU-VMSA driver from Renesas BSP: + * drivers/iommu/ipmmu-vmsa.c + * you can found at: + * url: git://git.kernel.org/pub/scm/linux/kernel/git/horms/renesas-bsp.git + * branch: v4.14.75-ltsi/rcar-3.9.6 + * commit: e206eb5b81a60e64c35fbc3a999b1a0db2b98044 + * and Xen's SMMU driver: + * xen/drivers/passthrough/arm/smmu.c + * + * Copyright (C) 2016-2019 EPAM Systems Inc. + * + * This program is free software; you can redistribute it and/or + * modify it under the terms and conditions of the GNU General Public + * License, version 2, as published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * General Public License for more details. + * + * You should have received a copy of the GNU General Public + * License along with this program; If not, see . + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#define dev_name(dev) dt_node_full_name(dev_to_dt(dev)) + +/* Device logger functions */ +#define dev_print(dev, lvl, fmt, ...) \ + printk(lvl "ipmmu: %s: " fmt, dev_name(dev), ## __VA_ARGS__) + +#define dev_info(dev, fmt, ...) \ + dev_print(dev, XENLOG_INFO, fmt, ## __VA_ARGS__) +#define dev_warn(dev, fmt, ...) \ + dev_print(dev, XENLOG_WARNING, fmt, ## __VA_ARGS__) +#define dev_err(dev, fmt, ...) \ + dev_print(dev, XENLOG_ERR, fmt, ## __VA_ARGS__) +#define dev_err_ratelimited(dev, fmt, ...) \ + dev_print(dev, XENLOG_ERR, fmt, ## __VA_ARGS__) + +/* + * Gen3 SoCs make use of up to 8 IPMMU contexts (sets of page table) and + * these can be managed independently. Each context is mapped to one Xen domain. + */ +#define IPMMU_CTX_MAX 8 +/* Gen3 SoCs make use of up to 48 micro-TLBs per IPMMU device. */ +#define IPMMU_UTLB_MAX 48 + +/* IPMMU context supports IPA size up to 40 bit. */ +#define IPMMU_MAX_P2M_IPA_BITS 40 + +/* + * Xen's domain IPMMU information stored in dom_iommu(d)->arch.priv + * + * As each context (set of page table) is mapped to one Xen domain, + * all associated IPMMU domains use the same context mapped to this Xen domain. + * This makes all master devices being attached to the same Xen domain share + * the same context (P2M table). + */ +struct ipmmu_vmsa_xen_domain { + /* + * Used to protect everything which belongs to this Xen domain: + * device assignment, domain init/destroy, flush ops, etc + */ + spinlock_t lock; + /* One or more Cache IPMMU domains associated with this Xen domain */ + struct list_head cache_domains; + /* Root IPMMU domain associated with this Xen domain */ + struct ipmmu_vmsa_domain *root_domain; +}; + +/* Xen master device's IPMMU information stored in fwspec->iommu_priv */ +struct ipmmu_vmsa_xen_device { + /* Cache IPMMU domain this master device is logically attached to */ + struct ipmmu_vmsa_domain *domain; + /* Cache IPMMU this master device is physically connected to */ + struct ipmmu_vmsa_device *mmu; +}; + +/* Root/Cache IPMMU device's information */ +struct ipmmu_vmsa_device { + struct device *dev; + void __iomem *base; + struct ipmmu_vmsa_device *root; + struct list_head list; + unsigned int num_utlbs; + unsigned int num_ctx; + spinlock_t lock; /* Protects ctx and domains[] */ + DECLARE_BITMAP(ctx, IPMMU_CTX_MAX); + struct ipmmu_vmsa_domain *domains[IPMMU_CTX_MAX]; +}; + +/* + * Root/Cache IPMMU domain's information + * + * Root IPMMU device is assigned to Root IPMMU domain while Cache IPMMU device + * is assigned to Cache IPMMU domain. Master devices are connected to Cache + * IPMMU devices through specific ports called micro-TLBs. + * All Cache IPMMU devices, in turn, are connected to Root IPMMU device + * which manages IPMMU context. + */ +struct ipmmu_vmsa_domain { + /* + * IPMMU device assigned to this IPMMU domain. + * Either Root device which is located at the main memory bus domain or + * Cache device which is located at each hierarchy bus domain. + */ + struct ipmmu_vmsa_device *mmu; + + /* Context used for this IPMMU domain */ + unsigned int context_id; + + /* Xen domain associated with this IPMMU domain */ + struct domain *d; + + /* The fields below are used for Cache IPMMU domain only */ + + /* + * Used to keep track of the master devices which are attached to this + * IPMMU domain (domain users). Master devices behind the same IPMMU device + * are grouped together by putting into the same IPMMU domain. + * Only when the refcount reaches 0 this IPMMU domain can be destroyed. + */ + unsigned int refcount; + /* Used to link this IPMMU domain for the same Xen domain */ + struct list_head list; +}; + +/* Used to keep track of registered IPMMU devices */ +static LIST_HEAD(ipmmu_devices); +static DEFINE_SPINLOCK(ipmmu_devices_lock); + +#define TLB_LOOP_TIMEOUT 100 /* 100us */ + +/* Registers Definition */ +#define IM_CTX_SIZE 0x40 + +#define IMCTR 0x0000 +/* + * These fields are implemented in IPMMU-MM only. So, can be set for + * Root IPMMU only. + */ +#define IMCTR_VA64 (1 << 29) +#define IMCTR_TRE (1 << 17) +#define IMCTR_AFE (1 << 16) +#define IMCTR_RTSEL_MASK (3 << 4) +#define IMCTR_RTSEL_SHIFT 4 +#define IMCTR_TREN (1 << 3) +/* + * These fields are common for all IPMMU devices. So, can be set for + * Cache IPMMUs as well. + */ +#define IMCTR_INTEN (1 << 2) +#define IMCTR_FLUSH (1 << 1) +#define IMCTR_MMUEN (1 << 0) +#define IMCTR_COMMON_MASK (7 << 0) + +#define IMCAAR 0x0004 + +#define IMTTBCR 0x0008 +#define IMTTBCR_EAE (1 << 31) +#define IMTTBCR_PMB (1 << 30) +#define IMTTBCR_SH1_NON_SHAREABLE (0 << 28) +#define IMTTBCR_SH1_OUTER_SHAREABLE (2 << 28) +#define IMTTBCR_SH1_INNER_SHAREABLE (3 << 28) +#define IMTTBCR_SH1_MASK (3 << 28) +#define IMTTBCR_ORGN1_NC (0 << 26) +#define IMTTBCR_ORGN1_WB_WA (1 << 26) +#define IMTTBCR_ORGN1_WT (2 << 26) +#define IMTTBCR_ORGN1_WB (3 << 26) +#define IMTTBCR_ORGN1_MASK (3 << 26) +#define IMTTBCR_IRGN1_NC (0 << 24) +#define IMTTBCR_IRGN1_WB_WA (1 << 24) +#define IMTTBCR_IRGN1_WT (2 << 24) +#define IMTTBCR_IRGN1_WB (3 << 24) +#define IMTTBCR_IRGN1_MASK (3 << 24) +#define IMTTBCR_TSZ1_MASK (1f << 16) +#define IMTTBCR_TSZ1_SHIFT 16 +#define IMTTBCR_SH0_NON_SHAREABLE (0 << 12) +#define IMTTBCR_SH0_OUTER_SHAREABLE (2 << 12) +#define IMTTBCR_SH0_INNER_SHAREABLE (3 << 12) +#define IMTTBCR_SH0_MASK (3 << 12) +#define IMTTBCR_ORGN0_NC (0 << 10) +#define IMTTBCR_ORGN0_WB_WA (1 << 10) +#define IMTTBCR_ORGN0_WT (2 << 10) +#define IMTTBCR_ORGN0_WB (3 << 10) +#define IMTTBCR_ORGN0_MASK (3 << 10) +#define IMTTBCR_IRGN0_NC (0 << 8) +#define IMTTBCR_IRGN0_WB_WA (1 << 8) +#define IMTTBCR_IRGN0_WT (2 << 8) +#define IMTTBCR_IRGN0_WB (3 << 8) +#define IMTTBCR_IRGN0_MASK (3 << 8) +#define IMTTBCR_SL0_LVL_2 (0 << 6) +#define IMTTBCR_SL0_LVL_1 (1 << 6) +#define IMTTBCR_TSZ0_MASK (0x1f << 0) +#define IMTTBCR_TSZ0_SHIFT 0 + +#define IMTTLBR0 0x0010 +#define IMTTLBR0_TTBR_MASK (0xfffff << 12) +#define IMTTUBR0 0x0014 +#define IMTTUBR0_TTBR_MASK (0xff << 0) +#define IMTTLBR1 0x0018 +#define IMTTLBR1_TTBR_MASK (0xfffff << 12) +#define IMTTUBR1 0x001c +#define IMTTUBR1_TTBR_MASK (0xff << 0) + +#define IMSTR 0x0020 +#define IMSTR_ERRLVL_MASK (3 << 12) +#define IMSTR_ERRLVL_SHIFT 12 +#define IMSTR_ERRCODE_TLB_FORMAT (1 << 8) +#define IMSTR_ERRCODE_ACCESS_PERM (4 << 8) +#define IMSTR_ERRCODE_SECURE_ACCESS (5 << 8) +#define IMSTR_ERRCODE_MASK (7 << 8) +#define IMSTR_MHIT (1 << 4) +#define IMSTR_ABORT (1 << 2) +#define IMSTR_PF (1 << 1) +#define IMSTR_TF (1 << 0) + +#define IMELAR 0x0030 +#define IMEUAR 0x0034 + +#define IMUCTR(n) ((n) < 32 ? IMUCTR0(n) : IMUCTR32(n)) +#define IMUCTR0(n) (0x0300 + ((n) * 16)) +#define IMUCTR32(n) (0x0600 + (((n) - 32) * 16)) +#define IMUCTR_FIXADDEN (1 << 31) +#define IMUCTR_FIXADD_MASK (0xff << 16) +#define IMUCTR_FIXADD_SHIFT 16 +#define IMUCTR_TTSEL_MMU(n) ((n) << 4) +#define IMUCTR_TTSEL_PMB (8 << 4) +#define IMUCTR_TTSEL_MASK (15 << 4) +#define IMUCTR_FLUSH (1 << 1) +#define IMUCTR_MMUEN (1 << 0) + +#define IMUASID(n) ((n) < 32 ? IMUASID0(n) : IMUASID32(n)) +#define IMUASID0(n) (0x0308 + ((n) * 16)) +#define IMUASID32(n) (0x0608 + (((n) - 32) * 16)) +#define IMUASID_ASID8_MASK (0xff << 8) +#define IMUASID_ASID8_SHIFT 8 +#define IMUASID_ASID0_MASK (0xff << 0) +#define IMUASID_ASID0_SHIFT 0 + +#define IMSAUXCTLR 0x0504 +#define IMSAUXCTLR_S2PTE (1 << 3) + +static struct ipmmu_vmsa_device *to_ipmmu(struct device *dev) +{ + struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); + + return fwspec && fwspec->iommu_priv ? + ((struct ipmmu_vmsa_xen_device *)fwspec->iommu_priv)->mmu : NULL; +} + +static void set_ipmmu(struct device *dev, struct ipmmu_vmsa_device *mmu) +{ + struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); + + ((struct ipmmu_vmsa_xen_device *)fwspec->iommu_priv)->mmu = mmu; +} + +static struct ipmmu_vmsa_domain *to_domain(struct device *dev) +{ + struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); + + return fwspec && fwspec->iommu_priv ? + ((struct ipmmu_vmsa_xen_device *)fwspec->iommu_priv)->domain : NULL; +} + +static void set_domain(struct device *dev, struct ipmmu_vmsa_domain *domain) +{ + struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); + + ((struct ipmmu_vmsa_xen_device *)fwspec->iommu_priv)->domain = domain; +} + +static struct ipmmu_vmsa_device *ipmmu_find_mmu_by_dev(struct device *dev) +{ + struct ipmmu_vmsa_device *mmu = NULL; + bool found = false; + + spin_lock(&ipmmu_devices_lock); + + list_for_each_entry ( mmu, &ipmmu_devices, list ) + { + if ( mmu->dev == dev ) + { + found = true; + break; + } + } + + spin_unlock(&ipmmu_devices_lock); + + return found ? mmu : NULL; +} + +/* Root device handling */ +static bool ipmmu_is_root(struct ipmmu_vmsa_device *mmu) +{ + return mmu->root == mmu; +} + +static struct ipmmu_vmsa_device *ipmmu_find_root(void) +{ + struct ipmmu_vmsa_device *mmu = NULL; + bool found = false; + + spin_lock(&ipmmu_devices_lock); + + list_for_each_entry( mmu, &ipmmu_devices, list ) + { + if ( ipmmu_is_root(mmu) ) + { + found = true; + break; + } + } + + spin_unlock(&ipmmu_devices_lock); + + return found ? mmu : NULL; +} + +/* Read/Write Access */ +static uint32_t ipmmu_read(struct ipmmu_vmsa_device *mmu, uint32_t offset) +{ + return readl(mmu->base + offset); +} + +static void ipmmu_write(struct ipmmu_vmsa_device *mmu, uint32_t offset, + uint32_t data) +{ + writel(data, mmu->base + offset); +} + +static uint32_t ipmmu_ctx_read_root(struct ipmmu_vmsa_domain *domain, + uint32_t reg) +{ + return ipmmu_read(domain->mmu->root, + domain->context_id * IM_CTX_SIZE + reg); +} + +static void ipmmu_ctx_write_root(struct ipmmu_vmsa_domain *domain, + uint32_t reg, uint32_t data) +{ + ipmmu_write(domain->mmu->root, + domain->context_id * IM_CTX_SIZE + reg, data); +} + +static void ipmmu_ctx_write_cache(struct ipmmu_vmsa_domain *domain, + uint32_t reg, uint32_t data) +{ + /* We expect only IMCTR value to be passed as a reg. */ + ASSERT(reg == IMCTR); + + /* Mask fields which are implemented in IPMMU-MM only. */ + if ( !ipmmu_is_root(domain->mmu) ) + ipmmu_write(domain->mmu, domain->context_id * IM_CTX_SIZE + reg, + data & IMCTR_COMMON_MASK); +} + +/* + * Write the context to both Root IPMMU and all Cache IPMMUs assigned + * to this Xen domain. + */ +static void ipmmu_ctx_write_all(struct ipmmu_vmsa_domain *domain, + uint32_t reg, uint32_t data) +{ + struct ipmmu_vmsa_xen_domain *xen_domain = dom_iommu(domain->d)->arch.priv; + struct ipmmu_vmsa_domain *cache_domain; + + list_for_each_entry( cache_domain, &xen_domain->cache_domains, list ) + ipmmu_ctx_write_cache(cache_domain, reg, data); + + ipmmu_ctx_write_root(domain, reg, data); +} + +/* TLB and micro-TLB Management */ + +/* Wait for any pending TLB invalidations to complete. */ +static void ipmmu_tlb_sync(struct ipmmu_vmsa_domain *domain) +{ + unsigned int count = 0; + + while ( ipmmu_ctx_read_root(domain, IMCTR) & IMCTR_FLUSH ) + { + cpu_relax(); + if ( ++count == TLB_LOOP_TIMEOUT ) + { + dev_err_ratelimited(domain->mmu->dev, "TLB sync timed out -- MMU may be deadlocked\n"); + return; + } + udelay(1); + } +} + +static void ipmmu_tlb_invalidate(struct ipmmu_vmsa_domain *domain) +{ + uint32_t data; + + data = ipmmu_ctx_read_root(domain, IMCTR); + data |= IMCTR_FLUSH; + ipmmu_ctx_write_all(domain, IMCTR, data); + + ipmmu_tlb_sync(domain); +} + +/* Enable MMU translation for the micro-TLB. */ +static void ipmmu_utlb_enable(struct ipmmu_vmsa_domain *domain, + unsigned int utlb) +{ + struct ipmmu_vmsa_device *mmu = domain->mmu; + + /* + * TODO: Reference-count the micro-TLB as several bus masters can be + * connected to the same micro-TLB. Prevent the use cases where + * the same micro-TLB could be shared between multiple Xen domains. + */ + ipmmu_write(mmu, IMUASID(utlb), 0); + ipmmu_write(mmu, IMUCTR(utlb), ipmmu_read(mmu, IMUCTR(utlb)) | + IMUCTR_TTSEL_MMU(domain->context_id) | IMUCTR_MMUEN); +} + +/* Disable MMU translation for the micro-TLB. */ +static void ipmmu_utlb_disable(struct ipmmu_vmsa_domain *domain, + unsigned int utlb) +{ + struct ipmmu_vmsa_device *mmu = domain->mmu; + + ipmmu_write(mmu, IMUCTR(utlb), 0); +} + +/* Domain/Context Management */ +static int ipmmu_domain_allocate_context(struct ipmmu_vmsa_device *mmu, + struct ipmmu_vmsa_domain *domain) +{ + unsigned long flags; + int ret; + + spin_lock_irqsave(&mmu->lock, flags); + + ret = find_first_zero_bit(mmu->ctx, mmu->num_ctx); + if ( ret != mmu->num_ctx ) + { + mmu->domains[ret] = domain; + set_bit(ret, mmu->ctx); + } + else + ret = -EBUSY; + + spin_unlock_irqrestore(&mmu->lock, flags); + + return ret; +} + +static void ipmmu_domain_free_context(struct ipmmu_vmsa_device *mmu, + unsigned int context_id) +{ + unsigned long flags; + + spin_lock_irqsave(&mmu->lock, flags); + + clear_bit(context_id, mmu->ctx); + mmu->domains[context_id] = NULL; + + spin_unlock_irqrestore(&mmu->lock, flags); +} + +static int ipmmu_domain_init_context(struct ipmmu_vmsa_domain *domain) +{ + uint64_t ttbr; + uint32_t tsz0; + int ret; + + /* Find an unused context. */ + ret = ipmmu_domain_allocate_context(domain->mmu->root, domain); + if ( ret < 0 ) + return ret; + + domain->context_id = ret; + + /* + * TTBR0 + * Use P2M table for this Xen domain. + */ + ASSERT(domain->d != NULL); + ttbr = page_to_maddr(domain->d->arch.p2m.root); + + dev_info(domain->mmu->root->dev, "%pd: Set IPMMU context %u (pgd 0x%"PRIx64")\n", + domain->d, domain->context_id, ttbr); + + ipmmu_ctx_write_root(domain, IMTTLBR0, ttbr & IMTTLBR0_TTBR_MASK); + ipmmu_ctx_write_root(domain, IMTTUBR0, (ttbr >> 32) & IMTTUBR0_TTBR_MASK); + + /* + * TTBCR + * We use long descriptors and allocate the whole "p2m_ipa_bits" IPA space + * to TTBR0. Use 4KB page granule. Start page table walks at first level. + * Always bypass stage 1 translation. + */ + tsz0 = (64 - p2m_ipa_bits) << IMTTBCR_TSZ0_SHIFT; + ipmmu_ctx_write_root(domain, IMTTBCR, IMTTBCR_EAE | IMTTBCR_PMB | + IMTTBCR_SL0_LVL_1 | tsz0); + + /* + * IMSTR + * Clear all interrupt flags. + */ + ipmmu_ctx_write_root(domain, IMSTR, ipmmu_ctx_read_root(domain, IMSTR)); + + /* + * IMCTR + * Enable the MMU and interrupt generation. The long-descriptor + * translation table format doesn't use TEX remapping. Don't enable AF + * software management as we have no use for it. Use VMSAv8-64 mode. + * Enable the context for Root IPMMU only. Flush the TLB as required + * when modifying the context registers. + */ + ipmmu_ctx_write_root(domain, IMCTR, + IMCTR_VA64 | IMCTR_INTEN | IMCTR_FLUSH | IMCTR_MMUEN); + + return 0; +} + +static void ipmmu_domain_destroy_context(struct ipmmu_vmsa_domain *domain) +{ + if ( !domain->mmu ) + return; + + /* + * Disable the context for Root IPMMU only. Flush the TLB as required + * when modifying the context registers. + */ + ipmmu_ctx_write_root(domain, IMCTR, IMCTR_FLUSH); + ipmmu_tlb_sync(domain); + + ipmmu_domain_free_context(domain->mmu->root, domain->context_id); +} + +/* Fault Handling */ +static void ipmmu_domain_irq(struct ipmmu_vmsa_domain *domain) +{ + const uint32_t err_mask = IMSTR_MHIT | IMSTR_ABORT | IMSTR_PF | IMSTR_TF; + struct ipmmu_vmsa_device *mmu = domain->mmu; + uint32_t status; + uint64_t iova; + + status = ipmmu_ctx_read_root(domain, IMSTR); + if ( !(status & err_mask) ) + return; + + iova = ipmmu_ctx_read_root(domain, IMELAR) | + ((uint64_t)ipmmu_ctx_read_root(domain, IMEUAR) << 32); + + /* + * Clear the error status flags. Unlike traditional interrupt flag + * registers that must be cleared by writing 1, this status register + * seems to require 0. The error address register must be read before, + * otherwise its value will be 0. + */ + ipmmu_ctx_write_root(domain, IMSTR, 0); + + /* Log fatal errors. */ + if ( status & IMSTR_MHIT ) + dev_err_ratelimited(mmu->dev, "%pd: Multiple TLB hits @0x%"PRIx64"\n", + domain->d, iova); + if ( status & IMSTR_ABORT ) + dev_err_ratelimited(mmu->dev, "%pd: Page Table Walk Abort @0x%"PRIx64"\n", + domain->d, iova); + + /* Return if it is neither Permission Fault nor Translation Fault. */ + if ( !(status & (IMSTR_PF | IMSTR_TF)) ) + return; + + /* Flush the TLB as required when IPMMU translation error occurred. */ + ipmmu_tlb_invalidate(domain); + + dev_err_ratelimited(mmu->dev, "%pd: Unhandled fault: status 0x%08x iova 0x%"PRIx64"\n", + domain->d, status, iova); +} + +static void ipmmu_irq(int irq, void *dev, struct cpu_user_regs *regs) +{ + struct ipmmu_vmsa_device *mmu = dev; + unsigned int i; + unsigned long flags; + + spin_lock_irqsave(&mmu->lock, flags); + + /* + * When interrupt arrives, we don't know the context it is related to. + * So, check interrupts for all active contexts to locate a context + * with status bits set. + */ + for ( i = 0; i < mmu->num_ctx; i++ ) + { + if ( !mmu->domains[i] ) + continue; + ipmmu_domain_irq(mmu->domains[i]); + } + + spin_unlock_irqrestore(&mmu->lock, flags); +} + +/* Master devices management */ +static int ipmmu_attach_device(struct ipmmu_vmsa_domain *domain, + struct device *dev) +{ + struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); + struct ipmmu_vmsa_device *mmu = to_ipmmu(dev); + unsigned int i; + + if ( !mmu ) + { + dev_err(dev, "Cannot attach to IPMMU\n"); + return -ENXIO; + } + + if ( !domain->mmu ) + { + /* The domain hasn't been used yet, initialize it. */ + domain->mmu = mmu; + + /* + * We have already enabled context for Root IPMMU assigned to this + * Xen domain in ipmmu_domain_init_context(). + * Enable the context for Cache IPMMU only. Flush the TLB as required + * when modifying the context registers. + */ + ipmmu_ctx_write_cache(domain, IMCTR, + ipmmu_ctx_read_root(domain, IMCTR) | IMCTR_FLUSH); + + dev_info(dev, "Using IPMMU context %u\n", domain->context_id); + } + else if ( domain->mmu != mmu ) + { + /* + * Something is wrong, we can't attach two master devices using + * different IOMMUs to the same IPMMU domain. + */ + dev_err(dev, "Can't attach IPMMU %s to domain on IPMMU %s\n", + dev_name(mmu->dev), dev_name(domain->mmu->dev)); + return -EINVAL; + } + else + dev_info(dev, "Reusing IPMMU context %u\n", domain->context_id); + + for ( i = 0; i < fwspec->num_ids; ++i ) + ipmmu_utlb_enable(domain, fwspec->ids[i]); + + return 0; +} + +static void ipmmu_detach_device(struct ipmmu_vmsa_domain *domain, + struct device *dev) +{ + struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); + unsigned int i; + + for ( i = 0; i < fwspec->num_ids; ++i ) + ipmmu_utlb_disable(domain, fwspec->ids[i]); +} + +static void ipmmu_device_reset(struct ipmmu_vmsa_device *mmu) +{ + unsigned int i; + + /* Disable all contexts. */ + for ( i = 0; i < mmu->num_ctx; ++i ) + ipmmu_write(mmu, i * IM_CTX_SIZE + IMCTR, 0); +} + +/* + * This function relies on the fact that Root IPMMU device is being probed + * the first. If not the case, it denies further Cache IPMMU device probes + * (returns the -EAGAIN) until the Root IPMMU device has been registered + * for sure. + */ +static int ipmmu_probe(struct dt_device_node *node) +{ + struct ipmmu_vmsa_device *mmu; + uint64_t addr, size; + int irq, ret; + + mmu = xzalloc(struct ipmmu_vmsa_device); + if ( !mmu ) + { + dev_err(&node->dev, "Cannot allocate device data\n"); + return -ENOMEM; + } + + mmu->dev = &node->dev; + mmu->num_utlbs = IPMMU_UTLB_MAX; + mmu->num_ctx = IPMMU_CTX_MAX; + spin_lock_init(&mmu->lock); + bitmap_zero(mmu->ctx, IPMMU_CTX_MAX); + + /* Map I/O memory and request IRQ. */ + ret = dt_device_get_address(node, 0, &addr, &size); + if ( ret ) + { + dev_err(&node->dev, "Failed to get MMIO\n"); + goto out; + } + + mmu->base = ioremap_nocache(addr, size); + if ( !mmu->base ) + { + dev_err(&node->dev, "Failed to ioremap MMIO (addr 0x%"PRIx64" size 0x%"PRIx64")\n", + addr, size); + ret = -ENOMEM; + goto out; + } + + /* + * Determine if this IPMMU node is a Root device by checking for + * the lack of renesas,ipmmu-main property. + */ + if ( !dt_find_property(node, "renesas,ipmmu-main", NULL) ) + mmu->root = mmu; + else + mmu->root = ipmmu_find_root(); + + /* Wait until the Root device has been registered for sure. */ + if ( !mmu->root ) + { + dev_err(&node->dev, "Root IPMMU hasn't been registered yet\n"); + ret = -EAGAIN; + goto out; + } + + /* Root devices have mandatory IRQs. */ + if ( ipmmu_is_root(mmu) ) + { + irq = platform_get_irq(node, 0); + if ( irq < 0 ) + { + dev_err(&node->dev, "No IRQ found\n"); + ret = irq; + goto out; + } + + ret = request_irq(irq, 0, ipmmu_irq, dev_name(&node->dev), mmu); + if ( ret < 0 ) + { + dev_err(&node->dev, "Failed to request IRQ %d\n", irq); + goto out; + } + + ipmmu_device_reset(mmu); + + /* + * Use stage 2 translation table format when stage 2 translation + * enabled. + */ + ipmmu_write(mmu, IMSAUXCTLR, + ipmmu_read(mmu, IMSAUXCTLR) | IMSAUXCTLR_S2PTE); + + dev_info(&node->dev, "IPMMU context 0 is reserved\n"); + set_bit(0, mmu->ctx); + } + + spin_lock(&ipmmu_devices_lock); + list_add(&mmu->list, &ipmmu_devices); + spin_unlock(&ipmmu_devices_lock); + + dev_info(&node->dev, "Registered %s IPMMU\n", + ipmmu_is_root(mmu) ? "Root" : "Cache"); + + return 0; + +out: + if ( mmu->base ) + iounmap(mmu->base); + xfree(mmu); + + return ret; +} + +/* Xen IOMMU ops */ +static int __must_check ipmmu_iotlb_flush_all(struct domain *d) +{ + struct ipmmu_vmsa_xen_domain *xen_domain = dom_iommu(d)->arch.priv; + + if ( !xen_domain || !xen_domain->root_domain ) + return 0; + + spin_lock(&xen_domain->lock); + ipmmu_tlb_invalidate(xen_domain->root_domain); + spin_unlock(&xen_domain->lock); + + return 0; +} + +static int __must_check ipmmu_iotlb_flush(struct domain *d, dfn_t dfn, + unsigned int page_count, + unsigned int flush_flags) +{ + ASSERT(flush_flags); + + /* The hardware doesn't support selective TLB flush. */ + return ipmmu_iotlb_flush_all(d); +} + +static struct ipmmu_vmsa_domain *ipmmu_get_cache_domain(struct domain *d, + struct device *dev) +{ + struct ipmmu_vmsa_xen_domain *xen_domain = dom_iommu(d)->arch.priv; + struct ipmmu_vmsa_device *mmu = to_ipmmu(dev); + struct ipmmu_vmsa_domain *domain; + + if ( !mmu ) + return NULL; + + /* + * Loop through all Cache IPMMU domains associated with this Xen domain + * to locate an IPMMU domain this IPMMU device is assigned to. + */ + list_for_each_entry( domain, &xen_domain->cache_domains, list ) + { + if ( domain->mmu == mmu ) + return domain; + } + + return NULL; +} + +static struct ipmmu_vmsa_domain *ipmmu_alloc_cache_domain(struct domain *d) +{ + struct ipmmu_vmsa_xen_domain *xen_domain = dom_iommu(d)->arch.priv; + struct ipmmu_vmsa_domain *domain; + + domain = xzalloc(struct ipmmu_vmsa_domain); + if ( !domain ) + return ERR_PTR(-ENOMEM); + + /* + * We don't assign the Cache IPMMU device here, it will be assigned when + * attaching master device to this domain in ipmmu_attach_device(). + * domain->mmu = NULL; + */ + + domain->d = d; + /* Use the same context mapped to this Xen domain. */ + domain->context_id = xen_domain->root_domain->context_id; + + return domain; +} + +static void ipmmu_free_cache_domain(struct ipmmu_vmsa_domain *domain) +{ + list_del(&domain->list); + /* + * Disable the context for Cache IPMMU only. Flush the TLB as required + * when modifying the context registers. + */ + ipmmu_ctx_write_cache(domain, IMCTR, IMCTR_FLUSH); + xfree(domain); +} + +static struct ipmmu_vmsa_domain *ipmmu_alloc_root_domain(struct domain *d) +{ + struct ipmmu_vmsa_domain *domain; + struct ipmmu_vmsa_device *root; + int ret; + + /* If we are here then Root device must has been registered. */ + root = ipmmu_find_root(); + if ( !root ) + { + printk(XENLOG_ERR "ipmmu: Unable to locate Root IPMMU\n"); + return ERR_PTR(-ENODEV); + } + + domain = xzalloc(struct ipmmu_vmsa_domain); + if ( !domain ) + return ERR_PTR(-ENOMEM); + + domain->mmu = root; + domain->d = d; + + /* Initialize the context to be mapped to this Xen domain. */ + ret = ipmmu_domain_init_context(domain); + if ( ret < 0 ) + { + dev_err(root->dev, "%pd: Unable to initialize IPMMU context\n", d); + xfree(domain); + return ERR_PTR(ret); + } + + return domain; +} + +static void ipmmu_free_root_domain(struct ipmmu_vmsa_domain *domain) +{ + ipmmu_domain_destroy_context(domain); + xfree(domain); +} + +static int ipmmu_assign_device(struct domain *d, u8 devfn, struct device *dev, + uint32_t flag) +{ + struct ipmmu_vmsa_xen_domain *xen_domain = dom_iommu(d)->arch.priv; + struct ipmmu_vmsa_domain *domain; + int ret; + + if ( !xen_domain ) + return -EINVAL; + + if ( !to_ipmmu(dev) ) + return -ENODEV; + + spin_lock(&xen_domain->lock); + + /* + * The IPMMU context for the Xen domain is not allocated beforehand + * (at the Xen domain creation time), but on demand only, when the first + * master device being attached to it. + * Create Root IPMMU domain which context will be mapped to this Xen domain + * if not exits yet. + */ + if ( !xen_domain->root_domain ) + { + domain = ipmmu_alloc_root_domain(d); + if ( IS_ERR(domain) ) + { + ret = PTR_ERR(domain); + goto out; + } + + xen_domain->root_domain = domain; + } + + if ( to_domain(dev) ) + { + dev_err(dev, "Already attached to IPMMU domain\n"); + ret = -EEXIST; + goto out; + } + + /* + * Master devices behind the same Cache IPMMU can be attached to the same + * Cache IPMMU domain. + * Before creating new IPMMU domain check to see if the required one + * already exists for this Xen domain. + */ + domain = ipmmu_get_cache_domain(d, dev); + if ( !domain ) + { + /* Create new IPMMU domain this master device will be attached to. */ + domain = ipmmu_alloc_cache_domain(d); + if ( IS_ERR(domain) ) + { + ret = PTR_ERR(domain); + goto out; + } + + /* Chain new IPMMU domain to the Xen domain. */ + list_add(&domain->list, &xen_domain->cache_domains); + } + + ret = ipmmu_attach_device(domain, dev); + if ( ret ) + { + /* + * Destroy Cache IPMMU domain only if there are no master devices + * attached to it. + */ + if ( !domain->refcount ) + ipmmu_free_cache_domain(domain); + } + else + { + domain->refcount++; + set_domain(dev, domain); + } + +out: + spin_unlock(&xen_domain->lock); + + return ret; +} + +static int ipmmu_deassign_device(struct domain *d, struct device *dev) +{ + struct ipmmu_vmsa_xen_domain *xen_domain = dom_iommu(d)->arch.priv; + struct ipmmu_vmsa_domain *domain = to_domain(dev); + + if ( !domain || domain->d != d ) + { + dev_err(dev, "Not attached to %pd\n", d); + return -ESRCH; + } + + spin_lock(&xen_domain->lock); + + ipmmu_detach_device(domain, dev); + set_domain(dev, NULL); + domain->refcount--; + + /* + * Destroy Cache IPMMU domain only if there are no master devices + * attached to it. + */ + if ( !domain->refcount ) + ipmmu_free_cache_domain(domain); + + spin_unlock(&xen_domain->lock); + + return 0; +} + +static int ipmmu_reassign_device(struct domain *s, struct domain *t, + u8 devfn, struct device *dev) +{ + int ret = 0; + + /* Don't allow remapping on other domain than hwdom */ + if ( t && t != hardware_domain ) + return -EPERM; + + if ( t == s ) + return 0; + + ret = ipmmu_deassign_device(s, dev); + if ( ret ) + return ret; + + if ( t ) + { + /* No flags are defined for ARM. */ + ret = ipmmu_assign_device(t, devfn, dev, 0); + if ( ret ) + return ret; + } + + return 0; +} + +static int ipmmu_add_device(u8 devfn, struct device *dev) +{ + struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); + struct ipmmu_vmsa_device *mmu; + unsigned int i; + + if ( !fwspec || !fwspec->iommu_dev ) + return -EINVAL; + + mmu = ipmmu_find_mmu_by_dev(fwspec->iommu_dev); + if ( !mmu ) + return -ENODEV; + + /* + * Perform sanity check of fwspec->num_ids and fwspec->ids[1] fields. + * These fields describe master device's connection to Cache IPMMU + * (micro-TLBs). Each master device gets micro-TLB assignment via + * the "iommus" property in DT. + */ + if ( fwspec->num_ids == 0 ) + return -EINVAL; + + for ( i = 0; i < fwspec->num_ids; ++i ) + { + if ( fwspec->ids[i] >= mmu->num_utlbs ) + return -EINVAL; + } + + if ( to_ipmmu(dev) ) + { + dev_err(dev, "Already added to IPMMU\n"); + return -EEXIST; + } + + fwspec->iommu_priv = xzalloc(struct ipmmu_vmsa_xen_device); + if ( !fwspec->iommu_priv ) + return -ENOMEM; + + set_ipmmu(dev, mmu); + + /* Let Xen know that the master device is protected by an IOMMU. */ + dt_device_set_protected(dev_to_dt(dev)); + + dev_info(dev, "Added master device (IPMMU %s micro-TLBs %u)\n", + dev_name(mmu->dev), fwspec->num_ids); + + return 0; +} + +static int ipmmu_iommu_domain_init(struct domain *d) +{ + struct ipmmu_vmsa_xen_domain *xen_domain; + + xen_domain = xzalloc(struct ipmmu_vmsa_xen_domain); + if ( !xen_domain ) + return -ENOMEM; + + spin_lock_init(&xen_domain->lock); + INIT_LIST_HEAD(&xen_domain->cache_domains); + /* + * We don't create Root IPMMU domain here, it will be created on demand + * only, when attaching the first master device to this Xen domain in + * ipmmu_assign_device(). + * xen_domain->root_domain = NULL; + */ + + dom_iommu(d)->arch.priv = xen_domain; + + return 0; +} + +static void __hwdom_init ipmmu_iommu_hwdom_init(struct domain *d) +{ + /* Set to false options not supported on ARM. */ + if ( iommu_hwdom_inclusive ) + printk(XENLOG_WARNING "ipmmu: map-inclusive dom0-iommu option is not supported on ARM\n"); + iommu_hwdom_inclusive = false; + if ( iommu_hwdom_reserved == 1 ) + printk(XENLOG_WARNING "ipmmu: map-reserved dom0-iommu option is not supported on ARM\n"); + iommu_hwdom_reserved = 0; + + arch_iommu_hwdom_init(d); +} + +static void ipmmu_iommu_domain_teardown(struct domain *d) +{ + struct ipmmu_vmsa_xen_domain *xen_domain = dom_iommu(d)->arch.priv; + + if ( !xen_domain ) + return; + + spin_lock(&xen_domain->lock); + /* + * Destroy Root IPMMU domain which context is mapped to this Xen domain + * if exits. + */ + if ( xen_domain->root_domain ) + ipmmu_free_root_domain(xen_domain->root_domain); + + spin_unlock(&xen_domain->lock); + + /* + * We assume that all master devices have already been detached from + * this Xen domain and there must be no associated Cache IPMMU domains + * in use. + */ + ASSERT(list_empty(&xen_domain->cache_domains)); + xfree(xen_domain); + dom_iommu(d)->arch.priv = NULL; +} + +static const struct iommu_ops ipmmu_iommu_ops = +{ + .init = ipmmu_iommu_domain_init, + .hwdom_init = ipmmu_iommu_hwdom_init, + .teardown = ipmmu_iommu_domain_teardown, + .iotlb_flush = ipmmu_iotlb_flush, + .iotlb_flush_all = ipmmu_iotlb_flush_all, + .assign_device = ipmmu_assign_device, + .reassign_device = ipmmu_reassign_device, + .map_page = arm_iommu_map_page, + .unmap_page = arm_iommu_unmap_page, + .add_device = ipmmu_add_device, +}; + +/* RCAR GEN3 product and cut information. */ +#define RCAR_PRODUCT_MASK 0x00007F00 +#define RCAR_PRODUCT_H3 0x00004F00 +#define RCAR_PRODUCT_M3 0x00005200 +#define RCAR_PRODUCT_M3N 0x00005500 +#define RCAR_CUT_MASK 0x000000FF +#define RCAR_CUT_VER30 0x00000020 + +static __init bool ipmmu_stage2_supported(void) +{ + struct dt_device_node *np; + uint64_t addr, size; + void __iomem *base; + uint32_t product, cut; + static enum + { + UNKNOWN, + SUPPORTED, + NOTSUPPORTED + } stage2_supported = UNKNOWN; + + /* Use the flag to avoid checking for the compatibility more then once. */ + switch ( stage2_supported ) + { + case SUPPORTED: + return true; + + case NOTSUPPORTED: + return false; + + case UNKNOWN: + default: + stage2_supported = NOTSUPPORTED; + break; + } + + np = dt_find_compatible_node(NULL, NULL, "renesas,prr"); + if ( !np ) + { + printk(XENLOG_ERR "ipmmu: Failed to find PRR node\n"); + return false; + } + + if ( dt_device_get_address(np, 0, &addr, &size) ) + { + printk(XENLOG_ERR "ipmmu: Failed to get PRR MMIO\n"); + return false; + } + + base = ioremap_nocache(addr, size); + if ( !base ) + { + printk(XENLOG_ERR "ipmmu: Failed to ioremap PRR MMIO\n"); + return false; + } + + product = readl(base); + cut = product & RCAR_CUT_MASK; + product &= RCAR_PRODUCT_MASK; + + switch ( product ) + { + case RCAR_PRODUCT_H3: + case RCAR_PRODUCT_M3: + if ( cut >= RCAR_CUT_VER30 ) + stage2_supported = SUPPORTED; + break; + + case RCAR_PRODUCT_M3N: + stage2_supported = SUPPORTED; + break; + + default: + printk(XENLOG_ERR "ipmmu: Unsupported SoC version\n"); + break; + } + + iounmap(base); + + return stage2_supported == SUPPORTED; +} + +static const struct dt_device_match ipmmu_dt_match[] __initconst = +{ + DT_MATCH_COMPATIBLE("renesas,ipmmu-r8a7795"), + DT_MATCH_COMPATIBLE("renesas,ipmmu-r8a77965"), + DT_MATCH_COMPATIBLE("renesas,ipmmu-r8a7796"), + { /* sentinel */ }, +}; + +static __init int ipmmu_init(struct dt_device_node *node, const void *data) +{ + int ret; + + /* + * Even if the device can't be initialized, we don't want to give + * the IPMMU device to dom0. + */ + dt_device_set_used_by(node, DOMID_XEN); + + if ( !iommu_hap_pt_share ) + { + printk_once(XENLOG_ERR "ipmmu: P2M table must always be shared between the CPU and the IPMMU\n"); + return -EINVAL; + } + + if ( !ipmmu_stage2_supported() ) + { + printk_once(XENLOG_ERR "ipmmu: P2M sharing is not supported in current SoC revision\n"); + return -EOPNOTSUPP; + } + else + { + /* + * As 4-level translation table is not supported in IPMMU, we need + * to check IPA size used for P2M table beforehand to be sure it is + * 3-level and the IPMMU will be able to use it. + * + * TODO: First initialize the IOMMU and gather the requirements and + * then initialize the P2M. In the P2M code, take into the account + * the IOMMU requirements and restrict "pa_range" if necessary. + */ + if ( IPMMU_MAX_P2M_IPA_BITS < p2m_ipa_bits ) + { + printk_once(XENLOG_ERR "ipmmu: P2M IPA size is not supported (P2M=%u IPMMU=%u)!\n", + p2m_ipa_bits, IPMMU_MAX_P2M_IPA_BITS); + return -EOPNOTSUPP; + } + } + + ret = ipmmu_probe(node); + if ( ret ) + { + dev_err(&node->dev, "Failed to init IPMMU (%d)\n", ret); + return ret; + } + + iommu_set_ops(&ipmmu_iommu_ops); + + return 0; +} + +DT_DEVICE_START(ipmmu, "Renesas IPMMU-VMSA", DEVICE_IOMMU) + .dt_match = ipmmu_dt_match, + .init = ipmmu_init, +DT_DEVICE_END + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * indent-tabs-mode: nil + * End: + */