From patchwork Thu Sep 26 11:20:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oleksandr Tyshchenko X-Patchwork-Id: 11162535 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5C52613B1 for ; Thu, 26 Sep 2019 11:22:28 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 384A6217F4 for ; Thu, 26 Sep 2019 11:22:28 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="O6bIa+fe" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 384A6217F4 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iDRpV-0008Cv-3q; Thu, 26 Sep 2019 11:21:01 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iDRpT-0008Cp-Um for xen-devel@lists.xenproject.org; Thu, 26 Sep 2019 11:21:00 +0000 X-Inumbo-ID: b3fd241e-e04f-11e9-bf31-bc764e2007e4 Received: from mail-lf1-x141.google.com (unknown [2a00:1450:4864:20::141]) by localhost (Halon) with ESMTPS id b3fd241e-e04f-11e9-bf31-bc764e2007e4; Thu, 26 Sep 2019 11:20:54 +0000 (UTC) Received: by mail-lf1-x141.google.com with SMTP id d17so1393922lfa.7 for ; Thu, 26 Sep 2019 04:20:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=b4Rmi9Qvik/48QwmaJHpfPEGtJFeqOMlQXlVzQFrlLM=; b=O6bIa+feR/BQylpM6PzF7Ejd6uREZwmBgx1PFZ6VmsBBqhrzXmffnw9NXqvUdOCBAg cb6GFJJWn+90HgJPvQAqal5uDgPpidL/jXub8cg4iv42a+B7sJSBJj4d59puB43WgWEO xdT5EW3bRSh+zgj1iGJ3TWikuE/ybO11fFGg9YOSDq0tJDtIVZEe75NKkbit1UF5LlPi uuorKN/Go/V/qox3vTaLrBCmyh7WTHVOg3HO6QLr3suv514Z467fLLPJ8bOlEsrT4ctw s5667JGDyYnhvFM/zjt0TJKxswIfnuLzsPU/CFVS13Ux8/HpIiDtPsoJOAbzr7ILh5fo vdmg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=b4Rmi9Qvik/48QwmaJHpfPEGtJFeqOMlQXlVzQFrlLM=; b=isTzXle7sOTF0Dhlnfw9cFdNvCvs4NS3QbUeUeReck2Yh2FiDesusK7N2/rAeUysxe gk7DlZau+jRaHdDyoPAdbaC3/i5k3zdxNdy04mw6Tqz86/02xjHLfkVuXjAtK4tmoYlm eQ44hcV9kYp0yFHN16tYXTuLcg3vx0uPQAIUV9EN5O6D+5PHe2qxUP8Ubj8+yFMKVGhn kIZRT9SbpCvukq3qdYuBOs+HHhmkSA1xFVeETr1iRUp0bdI1Ob5W6d+3gYA3t4MTyFq7 tYNyLbWixUMBVk+nbTNIZdlxAzt9PU0gqn7u1lB7UAq6yhqdt89fjoX0Z20IY3bc5Z++ WhYA== X-Gm-Message-State: APjAAAWvl4uaowy4KaNyqK9JnyRSeBa8PI4DtSKkYfCHwSlM3XIBee0f H1fs0YAXPjTbrPAzFnJ37Sy3TsRU4nk= X-Google-Smtp-Source: APXvYqze9kaQBsqBkJOT7HWuGUAyMlcZY788jIymCpS7Cnvi0EAQS3yAYcnxL/1QoLUMinLzJepw8w== X-Received: by 2002:a19:6001:: with SMTP id u1mr1882905lfb.50.1569496853144; Thu, 26 Sep 2019 04:20:53 -0700 (PDT) Received: from otyshchenko.kyiv.epam.com (ll-22.209.223.85.sovam.net.ua. [85.223.209.22]) by smtp.gmail.com with ESMTPSA id n4sm526810lfi.63.2019.09.26.04.20.51 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 26 Sep 2019 04:20:52 -0700 (PDT) From: Oleksandr Tyshchenko To: xen-devel@lists.xenproject.org Date: Thu, 26 Sep 2019 14:20:26 +0300 Message-Id: <1569496834-7796-1-git-send-email-olekstysh@gmail.com> X-Mailer: git-send-email 2.7.4 Subject: [Xen-devel] [PATCH V6 0/8] iommu/arm: Add Renesas IPMMU-VMSA support + Linux's iommu_fwspec X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Oleksandr Tyshchenko , julien.grall@arm.com, sstabellini@kernel.org, volodymyr_babchuk@epam.com, Yoshihiro Shimoda MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: Oleksandr Tyshchenko The purpose of this patch series is to add IPMMU-VMSA support to Xen on ARM. Besides new IOMMU driver, this series contains "iommu_fwspec" support and new API iommu_add_dt_device() for adding DT device to IOMMU and many other things. The IPMMU-VMSA is VMSA-compatible I/O Memory Management Unit (IOMMU) which provides address translation and access protection functionalities to processing units and interconnect networks. Please note, this driver is supposed to work only with newest R-Car Gen3 SoCs revisions which IPMMU hardware supports stage 2 translation table format and is able to use CPU's P2M table as is if one is 3-level page table (up to 40 bit IPA). ---------- This driver is based on Linux's IPMMU-VMSA driver from Renesas BSP: https://git.kernel.org/pub/scm/linux/kernel/git/horms/renesas-bsp.git/tree/drivers/iommu/ipmmu-vmsa.c?h=v4.14.75-ltsi/rcar-3.9.6 and Xen's SMMU driver: xen/drivers/passthrough/arm/smmu.c Although Xen driver has a lot in common with Linux driver, it is not a "direct ported" copy and should be treated as such. The major differences compare to the Linux driver are: 1. Stage 1/Stage 2 translation. Linux driver supports Stage 1 translation only (with Stage 1 translation table format). It manages page table by itself. But Xen driver supports Stage 2 translation (with Stage 2 translation table format) to be able to share the P2M with the CPU. Stage 1 translation is always bypassed in Xen driver. So, Xen driver is supposed to be used with newest R-Car Gen3 SoC revisions only (H3 ES3.0, M3-W+, etc.) which IPMMU H/W supports stage 2 translation table format. 2. AArch64 support. Linux driver uses VMSAv8-32 mode, while Xen driver enables Armv8 VMSAv8-64 mode to cover up to 40 bit input address. 3. Context bank (sets of page table) usage. In Xen, each context bank is mapped to one Xen domain. So, all devices being pass throughed to the same Xen domain share the same context bank. 4. IPMMU device tracking. In Xen, all IPMMU devices are managed by single driver instance. So, driver uses global list to keep track of registered devices. ---------- Series was tested on R-Car Gen3 H3 ES3.0/M3N ES1.0 based boards using current staging (babde47 introduce a 'passthrough' configuration option to xl.cfg...) in a system with several DMA masters being assigned to different guest domains. Guest domain reboot, destroy/create are functional. You can find the whole series (and [1]) here: repo: https://github.com/otyshchenko1/xen.git branch: ipmmu_upstream6 You can find previous discussions here: [V1] https://lists.xenproject.org/archives/html/xen-devel/2019-06/msg01755.html [V2] https://lists.xenproject.org/archives/html/xen-devel/2019-08/msg00253.html [V3] https://lists.xenproject.org/archives/html/xen-devel/2019-08/msg01948.html [V4] https://lists.xenproject.org/archives/html/xen-devel/2019-09/msg01255.html [V5] https://lists.xenproject.org/archives/html/xen-devel/2019-09/msg02282.html [1] Also, please note, there is a patch in ML which is intended to address the main TODO in the IPMMU-VMSA driver: https://lists.xenproject.org/archives/html/xen-devel/2019-09/msg02304.html For convenience: 1. iommu/arm: Add iommu_helpers.c file to keep common for IOMMUs stuff A-b 2. iommu/arm: Add ability to handle deferred probing request R-b 3. xen/common: Introduce _xrealloc function -none- 4. xen/common: Introduce xrealloc_flex_struct() helper macros R-b 5. iommu/arm: Add lightweight iommu_fwspec support A-b 6. iommu: Order the headers alphabetically in device_tree.c A-b 7. iommu/arm: Introduce iommu_add_dt_device API -none- 8. iommu/arm: Add Renesas IPMMU-VMSA support A-b/R-b Oleksandr Tyshchenko (8): iommu/arm: Add iommu_helpers.c file to keep common for IOMMUs stuff iommu/arm: Add ability to handle deferred probing request xen/common: Introduce _xrealloc function xen/common: Introduce xrealloc_flex_struct() helper macros iommu/arm: Add lightweight iommu_fwspec support iommu: Order the headers alphabetically in device_tree.c iommu/arm: Introduce iommu_add_dt_device API iommu/arm: Add Renesas IPMMU-VMSA support xen/arch/arm/domain_build.c | 25 +- xen/arch/arm/platforms/Kconfig | 1 + xen/common/xmalloc_tlsf.c | 114 ++- xen/drivers/passthrough/Kconfig | 13 + xen/drivers/passthrough/arm/Makefile | 3 +- xen/drivers/passthrough/arm/iommu.c | 56 +- xen/drivers/passthrough/arm/iommu_fwspec.c | 97 ++ xen/drivers/passthrough/arm/iommu_helpers.c | 84 ++ xen/drivers/passthrough/arm/ipmmu-vmsa.c | 1333 +++++++++++++++++++++++++++ xen/drivers/passthrough/arm/smmu.c | 48 +- xen/drivers/passthrough/device_tree.c | 83 +- xen/include/asm-arm/device.h | 7 +- xen/include/asm-arm/iommu.h | 10 + xen/include/asm-arm/iommu_fwspec.h | 68 ++ xen/include/xen/device_tree.h | 7 + xen/include/xen/iommu.h | 21 + xen/include/xen/xmalloc.h | 13 + 17 files changed, 1907 insertions(+), 76 deletions(-) create mode 100644 xen/drivers/passthrough/arm/iommu_fwspec.c create mode 100644 xen/drivers/passthrough/arm/iommu_helpers.c create mode 100644 xen/drivers/passthrough/arm/ipmmu-vmsa.c create mode 100644 xen/include/asm-arm/iommu_fwspec.h