From patchwork Fri Oct 15 13:59:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Bertrand Marquis X-Patchwork-Id: 12562033 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 547B4C433FE for ; Fri, 15 Oct 2021 13:59:51 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E860A6054F for ; Fri, 15 Oct 2021 13:59:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org E860A6054F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.210638.367586 (Exim 4.92) (envelope-from ) id 1mbNkN-000224-B5; Fri, 15 Oct 2021 13:59:43 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 210638.367586; Fri, 15 Oct 2021 13:59:43 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mbNkN-00021x-7w; Fri, 15 Oct 2021 13:59:43 +0000 Received: by outflank-mailman (input) for mailman id 210638; Fri, 15 Oct 2021 13:59:41 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mbNkL-00020z-7t for xen-devel@lists.xenproject.org; Fri, 15 Oct 2021 13:59:41 +0000 Received: from foss.arm.com (unknown [217.140.110.172]) by us1-rack-iad1.inumbo.com (Halon) with ESMTP id a2b38552-6df8-4faf-9778-cada5db9f8e5; Fri, 15 Oct 2021 13:59:40 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 30528147A; Fri, 15 Oct 2021 06:59:40 -0700 (PDT) Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com [10.1.199.62]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 301D93F694; Fri, 15 Oct 2021 06:59:39 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a2b38552-6df8-4faf-9778-cada5db9f8e5 From: Bertrand Marquis To: xen-devel@lists.xenproject.org Cc: iwj@xenproject.org, Paul Durrant , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu Subject: [PATCH v7 1/5] xen/vpci: Move ecam access functions to common code Date: Fri, 15 Oct 2021 14:59:18 +0100 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 PCI standard is using ECAM and not MCFG which is coming from ACPI[1]. Use ECAM/ecam instead of MCFG in common code and in new functions added in common vpci code by this patch. Move vpci_access_allowed from arch/x86/hvm/io.c to drivers/vpci/vpci.c. Create vpci_ecam_{read,write} in drivers/vpci/vpci.c that contains the common code to perform these operations, changed vpci_mmcfg_{read,write} accordingly to make use of these functions. The vpci_ecam_{read,write} functions are returning false on error and true on success. As the x86 code was previously always returning X86EMUL_OKAY the return code is ignored. A comment has been added in the code to show that this is intentional. Those functions will be used in a following patch inside by arm vpci implementation. Rename MMCFG_BDF to VPCI_ECAM_BDF and move it to vpci.h. This macro is only used by functions calling vpci_ecam helpers. No functional change intended with this patch. [1] https://wiki.osdev.org/PCI_Express Suggested-by: Roger Pau Monné Signed-off-by: Bertrand Marquis Acked-by: Roger Pau Monné --- Changes in v7: - Rename vpci_ecam_access_allowed to vpci_access_allowed - Rename vpci_ecam_mmio_{read/write} to vpci_ecam_{read/write} - Replace comment in x86/hvm/io.c with one suggested by Roger - Remove 32bit comments and fixes from this patch and move it to the next one to keep only the moving+renaming in this patch - Change return type of vpci_ecam_{read/write} to bool - rename ECAM_BDF to VPCI_ECAM_BDF and move it to vpci.h Changes in v6: Patch added --- xen/arch/x86/hvm/io.c | 46 ++++----------------------------- xen/drivers/vpci/vpci.c | 54 +++++++++++++++++++++++++++++++++++++++ xen/include/asm-x86/pci.h | 2 -- xen/include/xen/vpci.h | 12 +++++++++ 4 files changed, 71 insertions(+), 43 deletions(-) diff --git a/xen/arch/x86/hvm/io.c b/xen/arch/x86/hvm/io.c index 046a8eb4ed..eb3c80743e 100644 --- a/xen/arch/x86/hvm/io.c +++ b/xen/arch/x86/hvm/io.c @@ -260,20 +260,6 @@ unsigned int hvm_pci_decode_addr(unsigned int cf8, unsigned int addr, return CF8_ADDR_LO(cf8) | (addr & 3); } -/* Do some sanity checks. */ -static bool vpci_access_allowed(unsigned int reg, unsigned int len) -{ - /* Check access size. */ - if ( len != 1 && len != 2 && len != 4 && len != 8 ) - return false; - - /* Check that access is size aligned. */ - if ( (reg & (len - 1)) ) - return false; - - return true; -} - /* vPCI config space IO ports handlers (0xcf8/0xcfc). */ static bool vpci_portio_accept(const struct hvm_io_handler *handler, const ioreq_t *p) @@ -394,7 +380,7 @@ static unsigned int vpci_mmcfg_decode_addr(const struct hvm_mmcfg *mmcfg, paddr_t addr, pci_sbdf_t *sbdf) { addr -= mmcfg->addr; - sbdf->bdf = MMCFG_BDF(addr); + sbdf->bdf = VCPI_ECAM_BDF(addr); sbdf->bus += mmcfg->start_bus; sbdf->seg = mmcfg->segment; @@ -434,25 +420,8 @@ static int vpci_mmcfg_read(struct vcpu *v, unsigned long addr, reg = vpci_mmcfg_decode_addr(mmcfg, addr, &sbdf); read_unlock(&d->arch.hvm.mmcfg_lock); - if ( !vpci_access_allowed(reg, len) || - (reg + len) > PCI_CFG_SPACE_EXP_SIZE ) - return X86EMUL_OKAY; - - /* - * According to the PCIe 3.1A specification: - * - Configuration Reads and Writes must usually be DWORD or smaller - * in size. - * - Because Root Complex implementations are not required to support - * accesses to a RCRB that cross DW boundaries [...] software - * should take care not to cause the generation of such accesses - * when accessing a RCRB unless the Root Complex will support the - * access. - * Xen however supports 8byte accesses by splitting them into two - * 4byte accesses. - */ - *data = vpci_read(sbdf, reg, min(4u, len)); - if ( len == 8 ) - *data |= (uint64_t)vpci_read(sbdf, reg + 4, 4) << 32; + /* Failed reads are not propagated to the caller */ + vpci_ecam_read(sbdf, reg, len, data); return X86EMUL_OKAY; } @@ -476,13 +445,8 @@ static int vpci_mmcfg_write(struct vcpu *v, unsigned long addr, reg = vpci_mmcfg_decode_addr(mmcfg, addr, &sbdf); read_unlock(&d->arch.hvm.mmcfg_lock); - if ( !vpci_access_allowed(reg, len) || - (reg + len) > PCI_CFG_SPACE_EXP_SIZE ) - return X86EMUL_OKAY; - - vpci_write(sbdf, reg, min(4u, len), data); - if ( len == 8 ) - vpci_write(sbdf, reg + 4, 4, data >> 32); + /* Failed writes are not propagated to the caller */ + vpci_ecam_write(sbdf, reg, len, data); return X86EMUL_OKAY; } diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c index cbd1bac7fc..ef690f15a9 100644 --- a/xen/drivers/vpci/vpci.c +++ b/xen/drivers/vpci/vpci.c @@ -478,6 +478,60 @@ void vpci_write(pci_sbdf_t sbdf, unsigned int reg, unsigned int size, spin_unlock(&pdev->vpci->lock); } +/* Helper function to check an access size and alignment on vpci space. */ +bool vpci_access_allowed(unsigned int reg, unsigned int len) +{ + /* Check access size. */ + if ( len != 1 && len != 2 && len != 4 && len != 8 ) + return false; + + /* Check that access is size aligned. */ + if ( (reg & (len - 1)) ) + return false; + + return true; +} + +bool vpci_ecam_write(pci_sbdf_t sbdf, unsigned int reg, unsigned int len, + unsigned long data) +{ + if ( !vpci_access_allowed(reg, len) || + (reg + len) > PCI_CFG_SPACE_EXP_SIZE ) + return false; + + vpci_write(sbdf, reg, min(4u, len), data); + if ( len == 8 ) + vpci_write(sbdf, reg + 4, 4, data >> 32); + + return true; +} + +bool vpci_ecam_read(pci_sbdf_t sbdf, unsigned int reg, unsigned int len, + unsigned long *data) +{ + if ( !vpci_access_allowed(reg, len) || + (reg + len) > PCI_CFG_SPACE_EXP_SIZE ) + return false; + + /* + * According to the PCIe 3.1A specification: + * - Configuration Reads and Writes must usually be DWORD or smaller + * in size. + * - Because Root Complex implementations are not required to support + * accesses to a RCRB that cross DW boundaries [...] software + * should take care not to cause the generation of such accesses + * when accessing a RCRB unless the Root Complex will support the + * access. + * Xen however supports 8byte accesses by splitting them into two + * 4byte accesses. + */ + *data = vpci_read(sbdf, reg, min(4u, len)); + if ( len == 8 ) + *data |= (uint64_t)vpci_read(sbdf, reg + 4, 4) << 32; + + return true; +} + /* * Local variables: * mode: C diff --git a/xen/include/asm-x86/pci.h b/xen/include/asm-x86/pci.h index edd7c3e71a..443f25347d 100644 --- a/xen/include/asm-x86/pci.h +++ b/xen/include/asm-x86/pci.h @@ -6,8 +6,6 @@ #define CF8_ADDR_HI(cf8) ( ((cf8) & 0x0f000000) >> 16) #define CF8_ENABLED(cf8) (!!((cf8) & 0x80000000)) -#define MMCFG_BDF(addr) ( ((addr) & 0x0ffff000) >> 12) - #define IS_SNB_GFX(id) (id == 0x01068086 || id == 0x01168086 \ || id == 0x01268086 || id == 0x01028086 \ || id == 0x01128086 || id == 0x01228086 \ diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h index 9f5b5d52e1..d6cf0baf14 100644 --- a/xen/include/xen/vpci.h +++ b/xen/include/xen/vpci.h @@ -19,6 +19,8 @@ typedef int vpci_register_init_t(struct pci_dev *dev); #define VPCI_PRIORITY_MIDDLE "5" #define VPCI_PRIORITY_LOW "9" +#define VPCI_ECAM_BDF(addr) (((addr) & 0x0ffff000) >> 12) + #define REGISTER_VPCI_INIT(x, p) \ static vpci_register_init_t *const x##_entry \ __used_section(".data.vpci." p) = x @@ -208,6 +210,16 @@ static inline unsigned int vmsix_entry_nr(const struct vpci_msix *msix, { return entry - msix->entries; } + +/* ECAM mmio read/write helpers */ +bool vpci_access_allowed(unsigned int reg, unsigned int len); + +bool vpci_ecam_write(pci_sbdf_t sbdf, unsigned int reg, unsigned int len, + unsigned long data); + +bool vpci_ecam_read(pci_sbdf_t sbdf, unsigned int reg, unsigned int len, + unsigned long *data); + #endif /* __XEN__ */ #else /* !CONFIG_HAS_VPCI */ From patchwork Fri Oct 15 13:59:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Bertrand Marquis X-Patchwork-Id: 12562037 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B7D76C433F5 for ; Fri, 15 Oct 2021 13:59:56 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7C7826054F for ; Fri, 15 Oct 2021 13:59:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 7C7826054F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.210639.367598 (Exim 4.92) (envelope-from ) id 1mbNkR-0002MI-Ko; Fri, 15 Oct 2021 13:59:47 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 210639.367598; Fri, 15 Oct 2021 13:59:47 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mbNkR-0002M7-HD; Fri, 15 Oct 2021 13:59:47 +0000 Received: by outflank-mailman (input) for mailman id 210639; Fri, 15 Oct 2021 13:59:46 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mbNkQ-00020z-76 for xen-devel@lists.xenproject.org; Fri, 15 Oct 2021 13:59:46 +0000 Received: from foss.arm.com (unknown [217.140.110.172]) by us1-rack-iad1.inumbo.com (Halon) with ESMTP id 12d6442e-a78f-46ee-800c-93c651db31c7; Fri, 15 Oct 2021 13:59:42 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 119C614BF; Fri, 15 Oct 2021 06:59:42 -0700 (PDT) Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com [10.1.199.62]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 653133F694; Fri, 15 Oct 2021 06:59:40 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 12d6442e-a78f-46ee-800c-93c651db31c7 From: Bertrand Marquis To: xen-devel@lists.xenproject.org Cc: iwj@xenproject.org, Rahul Singh , Stefano Stabellini , Julien Grall , Volodymyr Babchuk , Andrew Cooper , George Dunlap , Jan Beulich , Wei Liu , Paul Durrant , =?utf-8?q?Roger_Pau_Monn?= =?utf-8?q?=C3=A9?= Subject: [PATCH v7 2/5] xen/arm: Enable the existing x86 virtual PCI support for ARM Date: Fri, 15 Oct 2021 14:59:19 +0100 Message-Id: <847d430fdeb19e695176ddea71eeb11dcef8b23e.1634305870.git.bertrand.marquis@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 From: Rahul Singh The existing VPCI support available for X86 is adapted for Arm. When the device is added to XEN via the hyper call “PHYSDEVOP_pci_device_add”, VPCI handler for the config space access is added to the Xen to emulate the PCI devices config space. A MMIO trap handler for the PCI ECAM space is registered in XEN so that when guest is trying to access the PCI config space,XEN will trap the access and emulate read/write using the VPCI and not the real PCI hardware. For Dom0less systems scan_pci_devices() would be used to discover the PCI device in XEN and VPCI handler will be added during XEN boots. This patch is also doing some small fixes to fix compilation errors on arm32 of vpci and prevent 64bit accesses on 32bit: - use %zu instead of lu in header.c for print - prevent 64bit accesses in vpci_access_allowed - ifdef out using CONFIG_64BIT handling of len 8 in vpci_ecam_{read/write} Signed-off-by: Rahul Singh Signed-off-by: Bertrand Marquis Acked-by: Roger Pau Monné --- Changes in v7: - adapt to changes in vpci generic functions (name and type) - remove call to pci_cleanup_msi on error exit - move pci_add_handlers to be only done when pdev->domain is not NULL - Remove cast to unsigned long in header.c and use %zu for print - Fix non ascii chars in arch-arm.h - Use CONFIG_64BIT inside vpci_access_allowed to prevent 64bit access on 32bit platforms - Use CONFIG_64BIT to compile out 64bit cases in vpci_ecam_{read/write} Changes in v6: - Use new vpci_ecam_ helpers for PCI access - Do not set XEN_DOMCTL_CDF_vpci for dom0 for now (will be done in a future patch once everything is ready) - rename REGISTER_OFFSET to ECAM_REG_OFFSET and move it to pci.h - remove not needed local variables in vpci_mmio_write, the one in read has been kept to ensure proper compilation on arm32 - move call to vpci_add_handlers before iommu init to simplify exit path - move call to pci_cleanup_msi in the out section of pci_add_device if pdev is not NULL and on error - initialize pdev to NULL to handle properly exit path call of pci_cleanup_msi - keep has_vpci to return false for now as CFG_vpci has been removed. Added a comment on top of the definition. - fix compilation errors on arm32 (print in header.c, cast missing in mmio_write. - local variable was kept in vpci_mmio_read on arm to prevent a cast error in arm32. Change in v5: - Add pci_cleanup_msi(pdev) incleanup part. - Added Reviewed-by: Stefano Stabellini Change in v4: - Move addition of XEN_DOMCTL_CDF_vpci flag to separate patch Change in v3: - Use is_pci_passthrough_enabled() in place of pci_passthrough_enabled variable - Reject XEN_DOMCTL_CDF_vpci for x86 in arch_sanitise_domain_config() - Remove IS_ENABLED(CONFIG_HAS_VPCI) from has_vpci() Change in v2: - Add new XEN_DOMCTL_CDF_vpci flag - modify has_vpci() to include XEN_DOMCTL_CDF_vpci - enable vpci support when pci-passthough option is enabled. --- --- xen/arch/arm/Makefile | 1 + xen/arch/arm/domain.c | 4 ++ xen/arch/arm/vpci.c | 77 +++++++++++++++++++++++++++++++++++ xen/arch/arm/vpci.h | 36 ++++++++++++++++ xen/drivers/passthrough/pci.c | 14 +++++++ xen/drivers/vpci/header.c | 2 +- xen/drivers/vpci/vpci.c | 10 +++++ xen/include/asm-arm/domain.h | 1 + xen/include/public/arch-arm.h | 7 ++++ xen/include/xen/pci.h | 2 + 10 files changed, 153 insertions(+), 1 deletion(-) create mode 100644 xen/arch/arm/vpci.c create mode 100644 xen/arch/arm/vpci.h diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile index 64518848b2..07f634508e 100644 --- a/xen/arch/arm/Makefile +++ b/xen/arch/arm/Makefile @@ -7,6 +7,7 @@ ifneq ($(CONFIG_NO_PLAT),y) obj-y += platforms/ endif obj-$(CONFIG_TEE) += tee/ +obj-$(CONFIG_HAS_VPCI) += vpci.o obj-$(CONFIG_HAS_ALTERNATIVE) += alternative.o obj-y += bootfdt.init.o diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c index eef0661beb..96e1b23550 100644 --- a/xen/arch/arm/domain.c +++ b/xen/arch/arm/domain.c @@ -39,6 +39,7 @@ #include #include +#include "vpci.h" #include "vuart.h" DEFINE_PER_CPU(struct vcpu *, curr_vcpu); @@ -773,6 +774,9 @@ int arch_domain_create(struct domain *d, if ( is_hardware_domain(d) && (rc = domain_vuart_init(d)) ) goto fail; + if ( (rc = domain_vpci_init(d)) != 0 ) + goto fail; + return 0; fail: diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c new file mode 100644 index 0000000000..a312d02f3d --- /dev/null +++ b/xen/arch/arm/vpci.c @@ -0,0 +1,77 @@ +/* + * xen/arch/arm/vpci.c + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + */ +#include +#include + +#include + +static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info, + register_t *r, void *p) +{ + pci_sbdf_t sbdf; + /* data is needed to prevent a pointer cast on 32bit */ + unsigned long data; + + /* We ignore segment part and always handle segment 0 */ + sbdf.sbdf = VCPI_ECAM_BDF(info->gpa); + + if ( vpci_ecam_read(sbdf, ECAM_REG_OFFSET(info->gpa), + 1U << info->dabt.size, &data) ) + { + *r = data; + return 1; + } + + *r = ~0ul; + + return 0; +} + +static int vpci_mmio_write(struct vcpu *v, mmio_info_t *info, + register_t r, void *p) +{ + pci_sbdf_t sbdf; + + /* We ignore segment part and always handle segment 0 */ + sbdf.sbdf = VCPI_ECAM_BDF(info->gpa); + + return vpci_ecam_write(sbdf, ECAM_REG_OFFSET(info->gpa), + 1U << info->dabt.size, r); +} + +static const struct mmio_handler_ops vpci_mmio_handler = { + .read = vpci_mmio_read, + .write = vpci_mmio_write, +}; + +int domain_vpci_init(struct domain *d) +{ + if ( !has_vpci(d) ) + return 0; + + register_mmio_handler(d, &vpci_mmio_handler, + GUEST_VPCI_ECAM_BASE, GUEST_VPCI_ECAM_SIZE, NULL); + + return 0; +} + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * indent-tabs-mode: nil + * End: + */ + diff --git a/xen/arch/arm/vpci.h b/xen/arch/arm/vpci.h new file mode 100644 index 0000000000..d8a7b0e3e8 --- /dev/null +++ b/xen/arch/arm/vpci.h @@ -0,0 +1,36 @@ +/* + * xen/arch/arm/vpci.h + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + */ + +#ifndef __ARCH_ARM_VPCI_H__ +#define __ARCH_ARM_VPCI_H__ + +#ifdef CONFIG_HAS_VPCI +int domain_vpci_init(struct domain *d); +#else +static inline int domain_vpci_init(struct domain *d) +{ + return 0; +} +#endif + +#endif /* __ARCH_ARM_VPCI_H__ */ + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * indent-tabs-mode: nil + * End: + */ diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c index 3aa8c3175f..082892c8a2 100644 --- a/xen/drivers/passthrough/pci.c +++ b/xen/drivers/passthrough/pci.c @@ -766,7 +766,21 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn, list_add(&pdev->domain_list, &hardware_domain->pdev_list); } else + { +#ifdef CONFIG_ARM + /* + * On ARM PCI devices discovery will be done by Dom0. Add vpci handler + * when Dom0 inform XEN to add the PCI devices in XEN. + */ + ret = vpci_add_handlers(pdev); + if ( ret ) + { + printk(XENLOG_ERR "Setup of vPCI failed: %d\n", ret); + goto out; + } +#endif iommu_enable_device(pdev); + } pci_enable_acs(pdev); diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c index f8cd55e7c0..40ff79c33f 100644 --- a/xen/drivers/vpci/header.c +++ b/xen/drivers/vpci/header.c @@ -373,7 +373,7 @@ static void bar_write(const struct pci_dev *pdev, unsigned int reg, /* If the value written is the current one avoid printing a warning. */ if ( val != (uint32_t)(bar->addr >> (hi ? 32 : 0)) ) gprintk(XENLOG_WARNING, - "%pp: ignored BAR %lu write with memory decoding enabled\n", + "%pp: ignored BAR %zu write with memory decoding enabled\n", &pdev->sbdf, bar - pdev->vpci->header.bars + hi); return; } diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c index ef690f15a9..decf7d87a1 100644 --- a/xen/drivers/vpci/vpci.c +++ b/xen/drivers/vpci/vpci.c @@ -485,6 +485,12 @@ bool vpci_access_allowed(unsigned int reg, unsigned int len) if ( len != 1 && len != 2 && len != 4 && len != 8 ) return false; +#ifndef CONFIG_64BIT + /* Prevent 64bit accesses on 32bit */ + if ( len == 8 ) + return false; +#endif + /* Check that access is size aligned. */ if ( (reg & (len - 1)) ) return false; @@ -500,8 +506,10 @@ bool vpci_ecam_write(pci_sbdf_t sbdf, unsigned int reg, unsigned int len, return false; vpci_write(sbdf, reg, min(4u, len), data); +#ifdef CONFIG_64BIT if ( len == 8 ) vpci_write(sbdf, reg + 4, 4, data >> 32); +#endif return true; } @@ -526,8 +534,10 @@ bool vpci_ecam_read(pci_sbdf_t sbdf, unsigned int reg, unsigned int len, * 4byte accesses. */ *data = vpci_read(sbdf, reg, min(4u, len)); +#ifdef CONFIG_64BIT if ( len == 8 ) *data |= (uint64_t)vpci_read(sbdf, reg + 4, 4) << 32; +#endif return true; } diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h index 14e575288f..9b3647587a 100644 --- a/xen/include/asm-arm/domain.h +++ b/xen/include/asm-arm/domain.h @@ -263,6 +263,7 @@ static inline void arch_vcpu_block(struct vcpu *v) {} #define arch_vm_assist_valid_mask(d) (1UL << VMASST_TYPE_runstate_update_flag) +/* vPCI is not available on Arm */ #define has_vpci(d) ({ (void)(d); false; }) #endif /* __ASM_DOMAIN_H__ */ diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h index 96ead3de07..b4c615bcdf 100644 --- a/xen/include/public/arch-arm.h +++ b/xen/include/public/arch-arm.h @@ -418,6 +418,13 @@ typedef uint64_t xen_callback_t; #define GUEST_GICV3_GICR0_BASE xen_mk_ullong(0x03020000) /* vCPU0..127 */ #define GUEST_GICV3_GICR0_SIZE xen_mk_ullong(0x01000000) +/* + * 256 MB is reserved for VPCI configuration space based on calculation + * 256 buses x 32 devices x 8 functions x 4 KB = 256 MB + */ +#define GUEST_VPCI_ECAM_BASE xen_mk_ullong(0x10000000) +#define GUEST_VPCI_ECAM_SIZE xen_mk_ullong(0x10000000) + /* ACPI tables physical address */ #define GUEST_ACPI_BASE xen_mk_ullong(0x20000000) #define GUEST_ACPI_SIZE xen_mk_ullong(0x02000000) diff --git a/xen/include/xen/pci.h b/xen/include/xen/pci.h index 70ac25345c..b6d7e454f8 100644 --- a/xen/include/xen/pci.h +++ b/xen/include/xen/pci.h @@ -40,6 +40,8 @@ #define PCI_SBDF3(s,b,df) \ ((pci_sbdf_t){ .sbdf = (((s) & 0xffff) << 16) | PCI_BDF2(b, df) }) +#define ECAM_REG_OFFSET(addr) ((addr) & 0x00000fff) + typedef union { uint32_t sbdf; struct { From patchwork Fri Oct 15 13:59:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bertrand Marquis X-Patchwork-Id: 12562039 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6EF0CC433EF for ; Fri, 15 Oct 2021 14:00:02 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2441D61181 for ; Fri, 15 Oct 2021 14:00:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 2441D61181 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.210641.367620 (Exim 4.92) (envelope-from ) id 1mbNkW-00031O-BH; Fri, 15 Oct 2021 13:59:52 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 210641.367620; Fri, 15 Oct 2021 13:59:52 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mbNkW-000313-6f; Fri, 15 Oct 2021 13:59:52 +0000 Received: by outflank-mailman (input) for mailman id 210641; Fri, 15 Oct 2021 13:59:51 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mbNkV-00020z-7I for xen-devel@lists.xenproject.org; Fri, 15 Oct 2021 13:59:51 +0000 Received: from foss.arm.com (unknown [217.140.110.172]) by us1-rack-iad1.inumbo.com (Halon) with ESMTP id 2d922a5b-17a9-464d-bd3e-0963b84be3d7; Fri, 15 Oct 2021 13:59:43 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2AAB4147A; Fri, 15 Oct 2021 06:59:43 -0700 (PDT) Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com [10.1.199.62]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 452D23F694; Fri, 15 Oct 2021 06:59:42 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 2d922a5b-17a9-464d-bd3e-0963b84be3d7 From: Bertrand Marquis To: xen-devel@lists.xenproject.org Cc: iwj@xenproject.org, Michal Orzel , Wei Liu , Anthony PERARD , Juergen Gross Subject: [PATCH v7 3/5] tools/libxl: Modify libxl__arch_domain_init_hw_description... Date: Fri, 15 Oct 2021 14:59:20 +0100 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 From: Michal Orzel ... to take a second argument of type libxl_domain_config* rather than libxl_domain_build_info*. We need to pass the whole libxl_domain_config structure as this will be needed later on to modify the libxl__prepare_dtb function to also take libxl_domain_config. Signed-off-by: Michal Orzel Suggested-by: Ian Jackson Reviewed-by: Ian Jackson --- tools/libs/light/libxl_arch.h | 2 +- tools/libs/light/libxl_arm.c | 3 ++- tools/libs/light/libxl_dom.c | 2 +- tools/libs/light/libxl_x86.c | 2 +- 4 files changed, 5 insertions(+), 4 deletions(-) diff --git a/tools/libs/light/libxl_arch.h b/tools/libs/light/libxl_arch.h index 8527fc5c6c..1522ecb97f 100644 --- a/tools/libs/light/libxl_arch.h +++ b/tools/libs/light/libxl_arch.h @@ -38,7 +38,7 @@ int libxl__arch_domain_create(libxl__gc *gc, /* setup arch specific hardware description, i.e. DTB on ARM */ _hidden int libxl__arch_domain_init_hw_description(libxl__gc *gc, - libxl_domain_build_info *info, + libxl_domain_config *d_config, libxl__domain_build_state *state, struct xc_dom_image *dom); /* finalize arch specific hardware description. */ diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c index a7801558cf..d5771f98dd 100644 --- a/tools/libs/light/libxl_arm.c +++ b/tools/libs/light/libxl_arm.c @@ -1016,12 +1016,13 @@ out: } int libxl__arch_domain_init_hw_description(libxl__gc *gc, - libxl_domain_build_info *info, + libxl_domain_config *d_config, libxl__domain_build_state *state, struct xc_dom_image *dom) { int rc; uint64_t val; + libxl_domain_build_info *const info = &d_config->b_info; if (info->type != LIBXL_DOMAIN_TYPE_PVH) { LOG(ERROR, "Unsupported Arm guest type %s", diff --git a/tools/libs/light/libxl_dom.c b/tools/libs/light/libxl_dom.c index e9f58ee4b2..fe9f760f71 100644 --- a/tools/libs/light/libxl_dom.c +++ b/tools/libs/light/libxl_dom.c @@ -571,7 +571,7 @@ static int libxl__build_dom(libxl__gc *gc, uint32_t domid, LOG(ERROR, "xc_dom_parse_image failed"); goto out; } - if ( (ret = libxl__arch_domain_init_hw_description(gc, info, state, dom)) != 0 ) { + if ( (ret = libxl__arch_domain_init_hw_description(gc, d_config, state, dom)) != 0 ) { LOGE(ERROR, "libxl__arch_domain_init_hw_description failed"); goto out; } diff --git a/tools/libs/light/libxl_x86.c b/tools/libs/light/libxl_x86.c index 6083878315..1feadebb18 100644 --- a/tools/libs/light/libxl_x86.c +++ b/tools/libs/light/libxl_x86.c @@ -568,7 +568,7 @@ int libxl__arch_extra_memory(libxl__gc *gc, } int libxl__arch_domain_init_hw_description(libxl__gc *gc, - libxl_domain_build_info *info, + libxl_domain_config *d_config, libxl__domain_build_state *state, struct xc_dom_image *dom) { From patchwork Fri Oct 15 13:59:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bertrand Marquis X-Patchwork-Id: 12562041 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 85034C433F5 for ; Fri, 15 Oct 2021 14:00:05 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4C1A76054F for ; Fri, 15 Oct 2021 14:00:05 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 4C1A76054F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.210644.367631 (Exim 4.92) (envelope-from ) id 1mbNkb-0003Zs-MV; Fri, 15 Oct 2021 13:59:57 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 210644.367631; Fri, 15 Oct 2021 13:59:57 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mbNkb-0003Zh-Hf; Fri, 15 Oct 2021 13:59:57 +0000 Received: by outflank-mailman (input) for mailman id 210644; Fri, 15 Oct 2021 13:59:56 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mbNka-00020z-7U for xen-devel@lists.xenproject.org; Fri, 15 Oct 2021 13:59:56 +0000 Received: from foss.arm.com (unknown [217.140.110.172]) by us1-rack-iad1.inumbo.com (Halon) with ESMTP id 3be2732e-6ea7-4ff9-a604-330bc3adcd15; Fri, 15 Oct 2021 13:59:44 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4403014BF; Fri, 15 Oct 2021 06:59:44 -0700 (PDT) Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com [10.1.199.62]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 5E9A03F694; Fri, 15 Oct 2021 06:59:43 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 3be2732e-6ea7-4ff9-a604-330bc3adcd15 From: Bertrand Marquis To: xen-devel@lists.xenproject.org Cc: iwj@xenproject.org, Michal Orzel , Wei Liu , Anthony PERARD , Juergen Gross Subject: [PATCH v7 4/5] tools/libxl_arm: Modify libxl__prepare_dtb... Date: Fri, 15 Oct 2021 14:59:21 +0100 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 From: Michal Orzel ... to take a second argument of type libxl_domain_config* rather than libxl_domain_build_info*. This change will be needed to get access from libxl__prepare_dtb to "num_pcidevs" field of libxl_domain_config to check whether to create a vPCI DT node or not. Signed-off-by: Michal Orzel Suggested-by: Ian Jackson Reviewed-by: Ian Jackson --- tools/libs/light/libxl_arm.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c index d5771f98dd..fdae129605 100644 --- a/tools/libs/light/libxl_arm.c +++ b/tools/libs/light/libxl_arm.c @@ -878,7 +878,7 @@ static int copy_partial_fdt(libxl__gc *gc, void *fdt, void *pfdt) #define FDT_MAX_SIZE (1<<20) -static int libxl__prepare_dtb(libxl__gc *gc, libxl_domain_build_info *info, +static int libxl__prepare_dtb(libxl__gc *gc, libxl_domain_config *d_config, libxl__domain_build_state *state, struct xc_dom_image *dom) { @@ -887,6 +887,7 @@ static int libxl__prepare_dtb(libxl__gc *gc, libxl_domain_build_info *info, int rc, res; size_t fdt_size = 0; int pfdt_size = 0; + libxl_domain_build_info *const info = &d_config->b_info; const libxl_version_info *vers; const struct arch_info *ainfo; @@ -1042,7 +1043,7 @@ int libxl__arch_domain_init_hw_description(libxl__gc *gc, if (rc) return rc; - rc = libxl__prepare_dtb(gc, info, state, dom); + rc = libxl__prepare_dtb(gc, d_config, state, dom); if (rc) goto out; if (!libxl_defbool_val(info->acpi)) { From patchwork Fri Oct 15 13:59:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bertrand Marquis X-Patchwork-Id: 12562035 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 19179C433EF for ; Fri, 15 Oct 2021 13:59:57 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D534B6054F for ; Fri, 15 Oct 2021 13:59:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org D534B6054F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.210640.367609 (Exim 4.92) (envelope-from ) id 1mbNkT-0002eF-0U; Fri, 15 Oct 2021 13:59:49 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 210640.367609; Fri, 15 Oct 2021 13:59:48 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mbNkS-0002e4-TI; Fri, 15 Oct 2021 13:59:48 +0000 Received: by outflank-mailman (input) for mailman id 210640; Fri, 15 Oct 2021 13:59:46 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mbNkQ-0002KR-Fj for xen-devel@lists.xenproject.org; Fri, 15 Oct 2021 13:59:46 +0000 Received: from foss.arm.com (unknown [217.140.110.172]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTP id 2769dd60-2dc0-11ec-8237-12813bfff9fa; Fri, 15 Oct 2021 13:59:46 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C94F8147A; Fri, 15 Oct 2021 06:59:45 -0700 (PDT) Received: from e109506.cambridge.arm.com (e109506.cambridge.arm.com [10.1.199.62]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 78E733F694; Fri, 15 Oct 2021 06:59:44 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 2769dd60-2dc0-11ec-8237-12813bfff9fa From: Bertrand Marquis To: xen-devel@lists.xenproject.org Cc: iwj@xenproject.org, Rahul Singh , Wei Liu , Anthony PERARD , Juergen Gross , Stefano Stabellini , Julien Grall , Volodymyr Babchuk , Michal Orzel Subject: [PATCH v7 5/5] arm/libxl: Emulated PCI device tree node in libxl Date: Fri, 15 Oct 2021 14:59:22 +0100 Message-Id: <4a22121e6474adfb1c5166e0fdaddff47c6dae46.1634305870.git.bertrand.marquis@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 From: Rahul Singh libxl will create an emulated PCI device tree node in the device tree to enable the guest OS to discover the virtual PCI during guest boot. Emulated PCI device tree node will only be created when there is any device assigned to guest. A new area has been reserved in the arm guest physical map at which the VPCI bus is declared in the device tree (reg and ranges parameters of the node). Note that currently we are using num_pcidevs instead of c_info->passthrough to decide whether to create a vPCI DT node. This will be insufficient if and when ARM does PCI hotplug. Add this note inside libxl_create.c where c_info->passthrough is set. Signed-off-by: Rahul Singh Signed-off-by: Michal Orzel Reviewed-by: Julien Grall Acked-by: Ian Jackson --- Changes in v7: -remove arch_arm.vpci -create vpci DT note if (d_config->num_pcidevs) Changes in v6: According to https://marc.info/?l=xen-devel&m=163415838129479&w=2: -do not set XEN_DOMCTL_CDF_vpci -do not enable VPCI support (by setting b_info.arch_arm.vpci) -do not define LIBXL_HAVE_BUILDINFO_ARM_VPCI -keep b_info.arch_arm.vpci, make_vpci_node and its helpers Change in v5: - Move setting the arch_arm.vpci and XEN_DOMCTL_CDF_vpci to libxl_arm.c Change in v4: - Gate code for x86 for setting the XEN_DOMCTL_CDF_vpci for x86. Change in v3: - Make GUEST_VPCI_MEM_ADDR address 2MB aligned Change in v2: - enable doamin_vpci_init() when XEN_DOMCTL_CDF_vpci is set for domain. --- --- tools/libs/light/libxl_arm.c | 103 ++++++++++++++++++++++++++++++++ tools/libs/light/libxl_create.c | 5 ++ xen/include/public/arch-arm.h | 10 ++++ 3 files changed, 118 insertions(+) diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c index fdae129605..eef1de0939 100644 --- a/tools/libs/light/libxl_arm.c +++ b/tools/libs/light/libxl_arm.c @@ -284,6 +284,59 @@ static int fdt_property_reg_placeholder(libxl__gc *gc, void *fdt, return fdt_property(fdt, "reg", regs, sizeof(regs)); } +static int fdt_property_values(libxl__gc *gc, void *fdt, + const char *name, + unsigned num_cells, ...) +{ + uint32_t prop[num_cells]; + be32 *cells = &prop[0]; + int i; + va_list ap; + uint32_t arg; + + va_start(ap, num_cells); + for (i = 0 ; i < num_cells; i++) { + arg = va_arg(ap, uint32_t); + set_cell(&cells, 1, arg); + } + va_end(ap); + + return fdt_property(fdt, name, prop, sizeof(prop)); +} + +static int fdt_property_vpci_ranges(libxl__gc *gc, void *fdt, + unsigned addr_cells, + unsigned size_cells, + unsigned num_regs, ...) +{ + uint32_t regs[num_regs*((addr_cells*2)+size_cells+1)]; + be32 *cells = ®s[0]; + int i; + va_list ap; + uint64_t arg; + + va_start(ap, num_regs); + for (i = 0 ; i < num_regs; i++) { + /* Set the memory bit field */ + arg = va_arg(ap, uint32_t); + set_cell(&cells, 1, arg); + + /* Set the vpci bus address */ + arg = addr_cells ? va_arg(ap, uint64_t) : 0; + set_cell(&cells, addr_cells , arg); + + /* Set the cpu bus address where vpci address is mapped */ + set_cell(&cells, addr_cells, arg); + + /* Set the vpci size requested */ + arg = size_cells ? va_arg(ap, uint64_t) : 0; + set_cell(&cells, size_cells, arg); + } + va_end(ap); + + return fdt_property(fdt, "ranges", regs, sizeof(regs)); +} + static int make_root_properties(libxl__gc *gc, const libxl_version_info *vers, void *fdt) @@ -687,6 +740,53 @@ static int make_vpl011_uart_node(libxl__gc *gc, void *fdt, return 0; } +static int make_vpci_node(libxl__gc *gc, void *fdt, + const struct arch_info *ainfo, + struct xc_dom_image *dom) +{ + int res; + const uint64_t vpci_ecam_base = GUEST_VPCI_ECAM_BASE; + const uint64_t vpci_ecam_size = GUEST_VPCI_ECAM_SIZE; + const char *name = GCSPRINTF("pcie@%"PRIx64, vpci_ecam_base); + + res = fdt_begin_node(fdt, name); + if (res) return res; + + res = fdt_property_compat(gc, fdt, 1, "pci-host-ecam-generic"); + if (res) return res; + + res = fdt_property_string(fdt, "device_type", "pci"); + if (res) return res; + + res = fdt_property_regs(gc, fdt, GUEST_ROOT_ADDRESS_CELLS, + GUEST_ROOT_SIZE_CELLS, 1, vpci_ecam_base, vpci_ecam_size); + if (res) return res; + + res = fdt_property_values(gc, fdt, "bus-range", 2, 0, 255); + if (res) return res; + + res = fdt_property_cell(fdt, "#address-cells", 3); + if (res) return res; + + res = fdt_property_cell(fdt, "#size-cells", 2); + if (res) return res; + + res = fdt_property_string(fdt, "status", "okay"); + if (res) return res; + + res = fdt_property_vpci_ranges(gc, fdt, GUEST_ROOT_ADDRESS_CELLS, + GUEST_ROOT_SIZE_CELLS, 2, + GUEST_VPCI_ADDR_TYPE_MEM, GUEST_VPCI_MEM_ADDR, GUEST_VPCI_MEM_SIZE, + GUEST_VPCI_ADDR_TYPE_PREFETCH_MEM, GUEST_VPCI_PREFETCH_MEM_ADDR, + GUEST_VPCI_PREFETCH_MEM_SIZE); + if (res) return res; + + res = fdt_end_node(fdt); + if (res) return res; + + return 0; +} + static const struct arch_info *get_arch_info(libxl__gc *gc, const struct xc_dom_image *dom) { @@ -991,6 +1091,9 @@ next_resize: if (info->tee == LIBXL_TEE_TYPE_OPTEE) FDT( make_optee_node(gc, fdt) ); + if (d_config->num_pcidevs) + FDT( make_vpci_node(gc, fdt, ainfo, dom) ); + if (pfdt) FDT( copy_partial_fdt(gc, fdt, pfdt) ); diff --git a/tools/libs/light/libxl_create.c b/tools/libs/light/libxl_create.c index 6ebb2bfc76..5a61d01722 100644 --- a/tools/libs/light/libxl_create.c +++ b/tools/libs/light/libxl_create.c @@ -1096,6 +1096,11 @@ int libxl__domain_config_setdefault(libxl__gc *gc, goto error_out; } + /* + * Note: libxl_arm directly examines num_pcidevs to decide whether + * to create a vPCI DT node, rather than using c_info->passthrough. + * This will be insufficient if and when ARM does PCI hotplug. + */ bool need_pt = d_config->num_pcidevs || d_config->num_dtdevs; if (c_info->passthrough == LIBXL_PASSTHROUGH_DEFAULT) { c_info->passthrough = need_pt diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h index b4c615bcdf..94b31511dd 100644 --- a/xen/include/public/arch-arm.h +++ b/xen/include/public/arch-arm.h @@ -433,6 +433,11 @@ typedef uint64_t xen_callback_t; #define GUEST_PL011_BASE xen_mk_ullong(0x22000000) #define GUEST_PL011_SIZE xen_mk_ullong(0x00001000) +/* Guest PCI-PCIe memory space where config space and BAR will be available.*/ +#define GUEST_VPCI_ADDR_TYPE_MEM xen_mk_ullong(0x02000000) +#define GUEST_VPCI_MEM_ADDR xen_mk_ullong(0x23000000) +#define GUEST_VPCI_MEM_SIZE xen_mk_ullong(0x10000000) + /* * 16MB == 4096 pages reserved for guest to use as a region to map its * grant table in. @@ -453,6 +458,11 @@ typedef uint64_t xen_callback_t; #define GUEST_RAM0_BASE xen_mk_ullong(0x40000000) /* 3GB of low RAM @ 1GB */ #define GUEST_RAM0_SIZE xen_mk_ullong(0xc0000000) +/* 4GB @ 4GB Prefetch Memory for VPCI */ +#define GUEST_VPCI_ADDR_TYPE_PREFETCH_MEM xen_mk_ullong(0x42000000) +#define GUEST_VPCI_PREFETCH_MEM_ADDR xen_mk_ullong(0x100000000) +#define GUEST_VPCI_PREFETCH_MEM_SIZE xen_mk_ullong(0x100000000) + #define GUEST_RAM1_BASE xen_mk_ullong(0x0200000000) /* 1016GB of RAM @ 8GB */ #define GUEST_RAM1_SIZE xen_mk_ullong(0xfe00000000)