From patchwork Fri Nov 5 06:33:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oleksandr Andrushchenko X-Patchwork-Id: 12604263 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7F75BC4332F for ; Fri, 5 Nov 2021 06:33:56 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 48F52611EE for ; Fri, 5 Nov 2021 06:33:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 48F52611EE Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.222020.384018 (Exim 4.92) (envelope-from ) id 1misn7-0001Ag-GG; Fri, 05 Nov 2021 06:33:33 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 222020.384018; Fri, 05 Nov 2021 06:33:33 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1misn7-0001AZ-D5; Fri, 05 Nov 2021 06:33:33 +0000 Received: by outflank-mailman (input) for mailman id 222020; Fri, 05 Nov 2021 06:33:32 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1misn6-0000uf-DE for xen-devel@lists.xenproject.org; Fri, 05 Nov 2021 06:33:32 +0000 Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com [2a00:1450:4864:20::52e]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 4b4c8a72-3e02-11ec-a9d2-d9f7a1cc8784; Fri, 05 Nov 2021 07:33:31 +0100 (CET) Received: by mail-ed1-x52e.google.com with SMTP id g10so29445450edj.1 for ; Thu, 04 Nov 2021 23:33:31 -0700 (PDT) Received: from localhost.localdomain ([185.199.97.5]) by smtp.gmail.com with ESMTPSA id e12sm3599870ejs.86.2021.11.04.23.33.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 04 Nov 2021 23:33:30 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 4b4c8a72-3e02-11ec-a9d2-d9f7a1cc8784 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=NpxZeW1ob2oJ1OjlMF1HQC/OBJW7HkL+IWHKhX4cdFQ=; b=VpF5QomEvz3AkIO6BVmFPxKcI93OQn8rhyLo6uT1gpTKm/Sl2UKZNcBwhrCq3iYt4N 0he1AifpyHyfLaPdDfkPQz4X9UCSxf5WFJahzUGsrm2XIE6EPPEaYjzTFY1lM/A/NaYW nbC7urXradjKl/AEnXLqcHfbjeRiMgN7+V0ktv/qfchhPwBXxLtpkfRigySJ4gOUsGHG 44Kg62aPnzk8mCQYd3jYJAmEHGQAFXSJI7aH1uNF0gLSS4FUR4ABmWuNE7yeKE9PaUnA 3Q6IWepGjLAxchz4R8A12nZO5bz9mYre/RZ7rJCdJFoPWd2/R6eKxGTR6KMUFXuZSY6X CU2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=NpxZeW1ob2oJ1OjlMF1HQC/OBJW7HkL+IWHKhX4cdFQ=; b=YzULZWg3ZX0c5oR60p7AjG4F4xRPoKhrXlJzSoYE31sBJ4dVy8gPxsgdRH+d92ssNf p5bkawlFRkwKVJy0omV9SlQlVfaYUwU7apcUpboGMUuIXm4yygjSFzTx7yOVifGtGQHK 5S8TlSOCdeUtIYej35hWFBBlQoGh8raPUgUf7owTAnJtlkVrE1WbEY7k/CzBhgbMsjXZ Dmmap/+FcSQp/2BSh3eSqQnx+Q5N45UJXlnKQ5wuVROjYd9Kx9QPsa9+9jTnHHijdFo8 GjlEBior6Z6o0oOUjmdwNhege9AduWhdNutswXojMz2W2hKqXqvI/Eg5BuKwn5hgpsPS bV/w== X-Gm-Message-State: AOAM5307Cj9WKqyQQrF7GoAhtsKwte8DwT5w62InqMq8KUz6dEG45SCt pkon8WfoFDtJhddkbuR6mEK73M680PwJBw== X-Google-Smtp-Source: ABdhPJwiKXxH8CFi6cbmauliu4A8R5NQ6lsQtohPzmA81YwvSXPzM2kcbskCcVp9emt/qAgTee6K7A== X-Received: by 2002:aa7:cf91:: with SMTP id z17mr55010868edx.193.1636094011048; Thu, 04 Nov 2021 23:33:31 -0700 (PDT) From: Oleksandr Andrushchenko To: xen-devel@lists.xenproject.org Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com, volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com, jbeulich@suse.com, andrew.cooper3@citrix.com, george.dunlap@citrix.com, paul@xen.org, bertrand.marquis@arm.com, rahul.singh@arm.com, Oleksandr Andrushchenko Subject: [PATCH v6 1/7] xen/arm: rename DEVICE_PCI to DEVICE_PCI_HOSTBRIDGE Date: Fri, 5 Nov 2021 08:33:20 +0200 Message-Id: <20211105063326.939843-2-andr2000@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211105063326.939843-1-andr2000@gmail.com> References: <20211105063326.939843-1-andr2000@gmail.com> MIME-Version: 1.0 From: Oleksandr Andrushchenko To better reflect the nature of the device type and not to make any confusion rename DEVICE_PCI to DEVICE_PCI_HOSTBRIDGE which it really is. Signed-off-by: Oleksandr Andrushchenko Suggested-by: Julien Grall Reviewed-by: Julien Grall --- New in v6 --- xen/arch/arm/pci/pci-host-generic.c | 2 +- xen/arch/arm/pci/pci-host-zynqmp.c | 2 +- xen/arch/arm/pci/pci.c | 2 +- xen/include/asm-arm/device.h | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/xen/arch/arm/pci/pci-host-generic.c b/xen/arch/arm/pci/pci-host-generic.c index 33457fbe9615..46de6e43cc72 100644 --- a/xen/arch/arm/pci/pci-host-generic.c +++ b/xen/arch/arm/pci/pci-host-generic.c @@ -32,7 +32,7 @@ static int __init pci_host_generic_probe(struct dt_device_node *dev, return pci_host_common_probe(dev, &pci_generic_ecam_ops); } -DT_DEVICE_START(pci_gen, "PCI HOST GENERIC", DEVICE_PCI) +DT_DEVICE_START(pci_gen, "PCI HOST GENERIC", DEVICE_PCI_HOSTBRIDGE) .dt_match = gen_pci_dt_match, .init = pci_host_generic_probe, DT_DEVICE_END diff --git a/xen/arch/arm/pci/pci-host-zynqmp.c b/xen/arch/arm/pci/pci-host-zynqmp.c index 61a9807d3d58..516982bca833 100644 --- a/xen/arch/arm/pci/pci-host-zynqmp.c +++ b/xen/arch/arm/pci/pci-host-zynqmp.c @@ -49,7 +49,7 @@ static int __init pci_host_generic_probe(struct dt_device_node *dev, return pci_host_common_probe(dev, &nwl_pcie_ops); } -DT_DEVICE_START(pci_gen, "PCI HOST ZYNQMP", DEVICE_PCI) +DT_DEVICE_START(pci_gen, "PCI HOST ZYNQMP", DEVICE_PCI_HOSTBRIDGE) .dt_match = nwl_pcie_dt_match, .init = pci_host_generic_probe, DT_DEVICE_END diff --git a/xen/arch/arm/pci/pci.c b/xen/arch/arm/pci/pci.c index 082c14e127a8..78b97beaef12 100644 --- a/xen/arch/arm/pci/pci.c +++ b/xen/arch/arm/pci/pci.c @@ -46,7 +46,7 @@ static int __init dt_pci_init(void) dt_for_each_device_node(dt_host, np) { - rc = device_init(np, DEVICE_PCI, NULL); + rc = device_init(np, DEVICE_PCI_HOSTBRIDGE, NULL); /* * Ignore the following error codes: * - EBADF: Indicate the current device is not a pci device. diff --git a/xen/include/asm-arm/device.h b/xen/include/asm-arm/device.h index 3782660751b6..086dde13eb6b 100644 --- a/xen/include/asm-arm/device.h +++ b/xen/include/asm-arm/device.h @@ -37,7 +37,7 @@ enum device_class DEVICE_SERIAL, DEVICE_IOMMU, DEVICE_GIC, - DEVICE_PCI, + DEVICE_PCI_HOSTBRIDGE, /* Use for error */ DEVICE_UNKNOWN, }; From patchwork Fri Nov 5 06:33:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oleksandr Andrushchenko X-Patchwork-Id: 12604259 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 42867C433FE for ; Fri, 5 Nov 2021 06:33:56 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D799B611C4 for ; Fri, 5 Nov 2021 06:33:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org D799B611C4 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.222021.384028 (Exim 4.92) (envelope-from ) id 1misn9-0001Re-P0; Fri, 05 Nov 2021 06:33:35 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 222021.384028; Fri, 05 Nov 2021 06:33:35 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1misn9-0001RT-M1; Fri, 05 Nov 2021 06:33:35 +0000 Received: by outflank-mailman (input) for mailman id 222021; Fri, 05 Nov 2021 06:33:34 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1misn8-0001ER-6k for xen-devel@lists.xenproject.org; Fri, 05 Nov 2021 06:33:34 +0000 Received: from mail-ed1-x534.google.com (mail-ed1-x534.google.com [2a00:1450:4864:20::534]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 4c03c8fa-3e02-11ec-9787-a32c541c8605; Fri, 05 Nov 2021 07:33:33 +0100 (CET) Received: by mail-ed1-x534.google.com with SMTP id g10so29445595edj.1 for ; Thu, 04 Nov 2021 23:33:32 -0700 (PDT) Received: from localhost.localdomain ([185.199.97.5]) by smtp.gmail.com with ESMTPSA id e12sm3599870ejs.86.2021.11.04.23.33.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 04 Nov 2021 23:33:31 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 4c03c8fa-3e02-11ec-9787-a32c541c8605 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=OlgpjgahBsbrNbkfAB2Gqbx6wbNdhBt0yFjmP2Yi1zg=; b=NjwUjpR5HC2Bg0akPdE1AxsQePsbsjfXZYA7sDRBtsedkkOWlwLQPkdkz+x9kboXGy M5YFcQfs8nVUDz5uQYw7zAaySt4XLq+bd3hiPlT9x79OrufwJDT/nSZsHalyDN4syhvf icawSkNQD5UiHo78v+auxuFt51ODnNJ+plgo+pjcfX/KIBRAtPLkKYEvHkWnTQaHtMFg bar38eAT+zlAqhs9DCmZLpY+VTfCsjv1tYAwj9HpuAwppp86NNhixVu+1GE9Zz2guVm0 N91ReZaZrdmUYH6nqZPk02tpBGslt3yJWvTCCgaqceTsc6kOaaTnWzyhFMthCltsuZ2U tunw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OlgpjgahBsbrNbkfAB2Gqbx6wbNdhBt0yFjmP2Yi1zg=; b=SCmFJdWIPacI6cbNn6N8n4bwZYAVuw4cyrTGl+Ghhzdj/ufkpt9ESvUtjgqGTPW3QC S66gFy5WWdbTU+qqVaQkjobWXUA4alhfjLiiw79zXypb4PtXwzgzK8jdskHpq/KnPEyF h4uqCMInPI9tcJ+6WorkaRafCUE1pAsMBDcTZhlI3DiQNGk4IH1lVwdQB914hS1VE1lo 0NMlyH3KfiwQ+pPXJjlnqEWdsJ5WtuBvncjDf6sincg1kqnyrItkDKzi55a5jmntJAZh nKzpzamnRtfJlsA0b1jjWU2cc157mHBWfxkecf8k/qLdc93SEZE3O8BGW1oo9dlXLyQK 0p2g== X-Gm-Message-State: AOAM532rxyIfjyhyMvPEpPNhf07YEOGJVTx19MVkJfIAKYaUjH5d4RKw 0UHVtywT+pANtrdEYp3nfD4/RJTCujKLDA== X-Google-Smtp-Source: ABdhPJwtG+PaF+XhSApDY2g23/Yp/gjYGWQ5/anjKkWg4HqvSYC2pJy8ySF6kDMl2A3dJIRSpL48Tw== X-Received: by 2002:a17:906:2cd5:: with SMTP id r21mr71140062ejr.435.1636094012280; Thu, 04 Nov 2021 23:33:32 -0700 (PDT) From: Oleksandr Andrushchenko To: xen-devel@lists.xenproject.org Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com, volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com, jbeulich@suse.com, andrew.cooper3@citrix.com, george.dunlap@citrix.com, paul@xen.org, bertrand.marquis@arm.com, rahul.singh@arm.com, Oleksandr Andrushchenko Subject: [PATCH v6 2/7] xen/arm: add pci-domain for disabled devices Date: Fri, 5 Nov 2021 08:33:21 +0200 Message-Id: <20211105063326.939843-3-andr2000@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211105063326.939843-1-andr2000@gmail.com> References: <20211105063326.939843-1-andr2000@gmail.com> MIME-Version: 1.0 From: Oleksandr Andrushchenko If a PCI host bridge device is present in the device tree, but is disabled, then its PCI host bridge driver was not instantiated. This results in the failure of the pci_get_host_bridge_segment() and the following panic during Xen start: (XEN) Device tree generation failed (-22). (XEN) (XEN) **************************************** (XEN) Panic on CPU 0: (XEN) Could not set up DOM0 guest OS (XEN) **************************************** Fix this by adding "linux,pci-domain" property for all device tree nodes which have "pci" device type, so we know which segments will be used by the guest for which bridges. Fixes: 4cfab4425d39 ("xen/arm: Add linux,pci-domain property for hwdom if not available.") Signed-off-by: Oleksandr Andrushchenko --- New in v6 --- xen/arch/arm/domain_build.c | 15 ++++++++++++++- xen/arch/arm/pci/pci-host-common.c | 2 +- xen/include/asm-arm/pci.h | 8 ++++++++ 3 files changed, 23 insertions(+), 2 deletions(-) diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c index 491f5e2c316e..f7fcb1400c19 100644 --- a/xen/arch/arm/domain_build.c +++ b/xen/arch/arm/domain_build.c @@ -753,9 +753,22 @@ static int __init write_properties(struct domain *d, struct kernel_info *kinfo, { uint16_t segment; + /* + * The node doesn't have "linux,pci-domain" property and it is + * possible that: + * - Xen only has drivers for a part of the host bridges + * - some host bridges are disabled + * Make sure we insert the correct "linux,pci-domain" property + * in any case, so we know which segments will be used + * by Linux for which bridges. + */ res = pci_get_host_bridge_segment(node, &segment); if ( res < 0 ) - return res; + { + segment = pci_get_new_domain_nr(); + printk(XENLOG_DEBUG "Assigned segment %d to %s\n", + segment, node->full_name); + } res = fdt_property_cell(kinfo->fdt, "linux,pci-domain", segment); if ( res ) diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c index d8cbaaaba654..47104b22b221 100644 --- a/xen/arch/arm/pci/pci-host-common.c +++ b/xen/arch/arm/pci/pci-host-common.c @@ -137,7 +137,7 @@ void pci_add_host_bridge(struct pci_host_bridge *bridge) list_add_tail(&bridge->node, &pci_host_bridges); } -static int pci_get_new_domain_nr(void) +int pci_get_new_domain_nr(void) { return atomic_inc_return(&domain_nr); } diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h index 81273e0d87ac..c20eba643d86 100644 --- a/xen/include/asm-arm/pci.h +++ b/xen/include/asm-arm/pci.h @@ -108,6 +108,8 @@ static always_inline bool is_pci_passthrough_enabled(void) void arch_pci_init_pdev(struct pci_dev *pdev); +int pci_get_new_domain_nr(void); + #else /*!CONFIG_HAS_PCI*/ struct arch_pci_dev { }; @@ -128,5 +130,11 @@ static inline int pci_get_host_bridge_segment(const struct dt_device_node *node, return -EINVAL; } +static inline int pci_get_new_domain_nr(void) +{ + ASSERT_UNREACHABLE(); + return -1; +} + #endif /*!CONFIG_HAS_PCI*/ #endif /* __ARM_PCI_H__ */ From patchwork Fri Nov 5 06:33:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oleksandr Andrushchenko X-Patchwork-Id: 12604267 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4E45FC433F5 for ; Fri, 5 Nov 2021 06:33:57 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id F3AAA611C4 for ; Fri, 5 Nov 2021 06:33:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org F3AAA611C4 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.222022.384035 (Exim 4.92) (envelope-from ) id 1misnA-0001Vs-AO; Fri, 05 Nov 2021 06:33:36 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 222022.384035; Fri, 05 Nov 2021 06:33:36 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1misnA-0001VY-3l; Fri, 05 Nov 2021 06:33:36 +0000 Received: by outflank-mailman (input) for mailman id 222022; Fri, 05 Nov 2021 06:33:35 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1misn9-0000uf-0Y for xen-devel@lists.xenproject.org; Fri, 05 Nov 2021 06:33:35 +0000 Received: from mail-ed1-x536.google.com (mail-ed1-x536.google.com [2a00:1450:4864:20::536]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 4cc0448d-3e02-11ec-a9d2-d9f7a1cc8784; Fri, 05 Nov 2021 07:33:34 +0100 (CET) Received: by mail-ed1-x536.google.com with SMTP id b15so10319817edd.7 for ; Thu, 04 Nov 2021 23:33:34 -0700 (PDT) Received: from localhost.localdomain ([185.199.97.5]) by smtp.gmail.com with ESMTPSA id e12sm3599870ejs.86.2021.11.04.23.33.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 04 Nov 2021 23:33:33 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 4cc0448d-3e02-11ec-a9d2-d9f7a1cc8784 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=7DFfi4NJpO6ZXPEa/MCt5rYVrF3eNK6m45b8kORo1QM=; b=KU2BVBauL1W0b/kJmCt+Hr5QQBqzVNc0UslYueGmROgDiwKbNU0oPqWXHhFe0LI4gW o/8iXU7+Dx8ENyZtdb8vH2lbgsSrBukG9TH3JHp5Lwug+65NLBGjcKQBbLumdy3WjNQT 1kNHTFFTEwsmBIRjG/g3LmRlqMD6V/8dU6MUftY5E9gbn7A19FkWZK8cWWh9+/hQgjqP AxH3e23Ad4eotR/kZWprmoQOCCGv0Bhju38poSp/cdFl7lFOieGH9GAL/Fy+qeDmDCJl wjnB6INtfkTho6b9W+TuujtFlEiUfrp/DpevSZXJ85XHO/z5OPaMAbKb4QAW+KcWgVS9 KzCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=7DFfi4NJpO6ZXPEa/MCt5rYVrF3eNK6m45b8kORo1QM=; b=NzG7boXjcofdvd2w1NG2fHGXPqnnaFYRY+u0qqbGX74fW6qrSyz1SqexKU99ueOAup p5ULyGbNvWGQI4ey4eqXWaunTagvpGpgFCmNsYPbw9FWb1QJR3lFDmtFwGu/WFOnaILZ BOAs9/IJALcMdNjlB+m0gAatAgffAnqypXF3rtf/d5eGXXt8Eflks6rPDUEOvJ+1Edkv htljP1tnRBbwyrfMiwuGiQEimcsjs3Zvh7RGc0MuPOuga69bDmusGnhyRvrR2SrNRHnW KrmlAre+PvozGJ8ordFgltyTILA9vf19L98pOVADTNlRi7Ubqvy6d4njFm5UAr+EpbB4 k+Zw== X-Gm-Message-State: AOAM532WOh220kG0PbixkIoa5GzH3xkiE1ItaKOK5MeVMQVgqJk/amed p58lvoAo9x3s8r3IPvOvSEbJQlQMxZYvkQ== X-Google-Smtp-Source: ABdhPJxknzRzOzobhrZIu7O3Gw2GhpxGesAfhPjns3ww0txfV1NaULioEwlrF/dxT5ryKUEXPdt1sg== X-Received: by 2002:a17:906:478e:: with SMTP id cw14mr52604639ejc.46.1636094013511; Thu, 04 Nov 2021 23:33:33 -0700 (PDT) From: Oleksandr Andrushchenko To: xen-devel@lists.xenproject.org Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com, volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com, jbeulich@suse.com, andrew.cooper3@citrix.com, george.dunlap@citrix.com, paul@xen.org, bertrand.marquis@arm.com, rahul.singh@arm.com, Oleksandr Andrushchenko Subject: [PATCH v6 3/7] xen/arm: setup MMIO range trap handlers for hardware domain Date: Fri, 5 Nov 2021 08:33:22 +0200 Message-Id: <20211105063326.939843-4-andr2000@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211105063326.939843-1-andr2000@gmail.com> References: <20211105063326.939843-1-andr2000@gmail.com> MIME-Version: 1.0 From: Oleksandr Andrushchenko In order for vPCI to work it needs to maintain guest and hardware domain's views of the configuration space. For example, BARs and COMMAND registers require emulation for guests and the guest view of the registers needs to be in sync with the real contents of the relevant registers. For that ECAM address space needs to also be trapped for the hardware domain, so we need to implement PCI host bridge specific callbacks to properly setup MMIO handlers for those ranges depending on particular host bridge implementation. Signed-off-by: Oleksandr Andrushchenko --- Since v5: - add vpci_sbdf_from_gpa helper for gpa to SBDF translation - take bridge's bus start into account while calculating SBDF Since v4: - unsigned int for functions working with count - gate number of MMIO handlers needed for CONFIG_HAS_PCI_MSI and fix their number, e.g. single handler for PBA and MSI-X tables (Roger) - re-work code for assigning MMIO handlers to be simpler and account on the fact that there could multiple host bridges exist for the hwdom Since v3: - fixed comment formatting Since v2: - removed unneeded assignment (count = 0) - removed unneeded header inclusion - update commit message Since v1: - Dynamically calculate the number of MMIO handlers required for vPCI and update the total number accordingly - s/clb/cb - Do not introduce a new callback for MMIO handler setup --- xen/arch/arm/domain.c | 2 + xen/arch/arm/pci/pci-host-common.c | 27 ++++++++++++ xen/arch/arm/vpci.c | 66 ++++++++++++++++++++++++++---- xen/arch/arm/vpci.h | 6 +++ xen/include/asm-arm/pci.h | 5 +++ 5 files changed, 98 insertions(+), 8 deletions(-) diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c index 96e1b235501d..92a6c509e5c5 100644 --- a/xen/arch/arm/domain.c +++ b/xen/arch/arm/domain.c @@ -739,6 +739,8 @@ int arch_domain_create(struct domain *d, if ( (rc = domain_vgic_register(d, &count)) != 0 ) goto fail; + count += domain_vpci_get_num_mmio_handlers(d); + if ( (rc = domain_io_init(d, count + MAX_IO_HANDLER)) != 0 ) goto fail; diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c index 47104b22b221..0d271a6e8881 100644 --- a/xen/arch/arm/pci/pci-host-common.c +++ b/xen/arch/arm/pci/pci-host-common.c @@ -289,6 +289,33 @@ int pci_get_host_bridge_segment(const struct dt_device_node *node, return -EINVAL; } +int pci_host_iterate_bridges(struct domain *d, + int (*cb)(struct domain *d, + struct pci_host_bridge *bridge)) +{ + struct pci_host_bridge *bridge; + int err; + + list_for_each_entry( bridge, &pci_host_bridges, node ) + { + err = cb(d, bridge); + if ( err ) + return err; + } + return 0; +} + +unsigned int pci_host_get_num_bridges(void) +{ + struct pci_host_bridge *bridge; + unsigned int count = 0; + + list_for_each_entry( bridge, &pci_host_bridges, node ) + count++; + + return count; +} + /* * Local variables: * mode: C diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c index 23f45386f4b3..5a6ebd8b9868 100644 --- a/xen/arch/arm/vpci.c +++ b/xen/arch/arm/vpci.c @@ -16,16 +16,31 @@ #include +static pci_sbdf_t vpci_sbdf_from_gpa(const struct pci_host_bridge *bridge, + paddr_t gpa) +{ + pci_sbdf_t sbdf; + + if ( bridge ) + { + sbdf.sbdf = VPCI_ECAM_BDF(gpa - bridge->cfg->phys_addr); + sbdf.seg = bridge->segment; + sbdf.bus += bridge->cfg->busn_start; + } + else + sbdf.sbdf = VPCI_ECAM_BDF(gpa - GUEST_VPCI_ECAM_BASE); + + return sbdf; +} + static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info, register_t *r, void *p) { - pci_sbdf_t sbdf; + struct pci_host_bridge *bridge = p; + pci_sbdf_t sbdf = vpci_sbdf_from_gpa(bridge, info->gpa); /* data is needed to prevent a pointer cast on 32bit */ unsigned long data; - /* We ignore segment part and always handle segment 0 */ - sbdf.sbdf = VPCI_ECAM_BDF(info->gpa - GUEST_VPCI_ECAM_BASE); - if ( vpci_ecam_read(sbdf, ECAM_REG_OFFSET(info->gpa), 1U << info->dabt.size, &data) ) { @@ -41,10 +56,8 @@ static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info, static int vpci_mmio_write(struct vcpu *v, mmio_info_t *info, register_t r, void *p) { - pci_sbdf_t sbdf; - - /* We ignore segment part and always handle segment 0 */ - sbdf.sbdf = VPCI_ECAM_BDF(info->gpa - GUEST_VPCI_ECAM_BASE); + struct pci_host_bridge *bridge = p; + pci_sbdf_t sbdf = vpci_sbdf_from_gpa(bridge, info->gpa); return vpci_ecam_write(sbdf, ECAM_REG_OFFSET(info->gpa), 1U << info->dabt.size, r); @@ -55,17 +68,54 @@ static const struct mmio_handler_ops vpci_mmio_handler = { .write = vpci_mmio_write, }; +static int vpci_setup_mmio_handler_cb(struct domain *d, + struct pci_host_bridge *bridge) +{ + struct pci_config_window *cfg = bridge->cfg; + + register_mmio_handler(d, &vpci_mmio_handler, + cfg->phys_addr, cfg->size, bridge); + return 0; +} + int domain_vpci_init(struct domain *d) { if ( !has_vpci(d) ) return 0; + if ( is_hardware_domain(d) ) + return pci_host_iterate_bridges(d, vpci_setup_mmio_handler_cb); + + /* Guest domains use what is programmed in their device tree. */ register_mmio_handler(d, &vpci_mmio_handler, GUEST_VPCI_ECAM_BASE, GUEST_VPCI_ECAM_SIZE, NULL); return 0; } +unsigned int domain_vpci_get_num_mmio_handlers(struct domain *d) +{ + unsigned int count; + + if ( is_hardware_domain(d) ) + /* For each PCI host bridge's configuration space. */ + count = pci_host_get_num_bridges(); + else + /* + * There's a single MSI-X MMIO handler that deals with both PBA + * and MSI-X tables per each PCI device being passed through. + * Maximum number of supported devices is 32 as virtual bus + * topology emulates the devices as embedded endpoints. + * +1 for a single emulated host bridge's configuration space. + */ + count = 1; +#ifdef CONFIG_HAS_PCI_MSI + count += 32; +#endif + + return count; +} + /* * Local variables: * mode: C diff --git a/xen/arch/arm/vpci.h b/xen/arch/arm/vpci.h index d8a7b0e3e802..3c713f3fcdb5 100644 --- a/xen/arch/arm/vpci.h +++ b/xen/arch/arm/vpci.h @@ -17,11 +17,17 @@ #ifdef CONFIG_HAS_VPCI int domain_vpci_init(struct domain *d); +unsigned int domain_vpci_get_num_mmio_handlers(struct domain *d); #else static inline int domain_vpci_init(struct domain *d) { return 0; } + +static inline unsigned int domain_vpci_get_num_mmio_handlers(struct domain *d) +{ + return 0; +} #endif #endif /* __ARCH_ARM_VPCI_H__ */ diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h index c20eba643d86..969333043431 100644 --- a/xen/include/asm-arm/pci.h +++ b/xen/include/asm-arm/pci.h @@ -110,6 +110,11 @@ void arch_pci_init_pdev(struct pci_dev *pdev); int pci_get_new_domain_nr(void); +int pci_host_iterate_bridges(struct domain *d, + int (*clb)(struct domain *d, + struct pci_host_bridge *bridge)); +unsigned int pci_host_get_num_bridges(void); + #else /*!CONFIG_HAS_PCI*/ struct arch_pci_dev { }; From patchwork Fri Nov 5 06:33:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oleksandr Andrushchenko X-Patchwork-Id: 12604269 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 05AF3C433FE for ; Fri, 5 Nov 2021 06:33:58 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A20B86120E for ; Fri, 5 Nov 2021 06:33:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org A20B86120E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.222023.384051 (Exim 4.92) (envelope-from ) id 1misnC-00021Y-Gu; Fri, 05 Nov 2021 06:33:38 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 222023.384051; Fri, 05 Nov 2021 06:33:38 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1misnC-00021K-DK; Fri, 05 Nov 2021 06:33:38 +0000 Received: by outflank-mailman (input) for mailman id 222023; Fri, 05 Nov 2021 06:33:36 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1misnA-0000uf-L6 for xen-devel@lists.xenproject.org; Fri, 05 Nov 2021 06:33:36 +0000 Received: from mail-ed1-x52f.google.com (mail-ed1-x52f.google.com [2a00:1450:4864:20::52f]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 4d889433-3e02-11ec-a9d2-d9f7a1cc8784; Fri, 05 Nov 2021 07:33:35 +0100 (CET) Received: by mail-ed1-x52f.google.com with SMTP id o8so29539932edc.3 for ; Thu, 04 Nov 2021 23:33:35 -0700 (PDT) Received: from localhost.localdomain ([185.199.97.5]) by smtp.gmail.com with ESMTPSA id e12sm3599870ejs.86.2021.11.04.23.33.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 04 Nov 2021 23:33:34 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 4d889433-3e02-11ec-a9d2-d9f7a1cc8784 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ynKiddE/cdPB0rzSPZ2YwMN5+mv/4H5TbvJgrgmAIn4=; b=EpFqyisfx/vBiGnUM/Sj3APtqWb9zBlkaLeZ3pBz7poLyBjzYafEbjd165RQ7qVw3L bq1dPGrKeGJXJdyrlVFoCLiBd7fbiL7jKbVqVkfdVF3GS/84y8tNq0t6AqKHLYFGl9E2 IQOJnddbc4OOuXBi+DKDXn9HY/l5RWdlNVI7SGZT4QZfcG055x4fDlSc0IAC9KDXrelu Olky4subGN13B+KAh5Uz2osi9PtaamqnzHDlS5ZHLL/XFD0ehB+7Isaz45uQfhx7Qg09 j6Ar0pNpx7JRr5I2P5m1V74ve/OYZ9lQwUaQF7SozqKOPj7v5R24Wy721wPCW38iDGmR vWqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ynKiddE/cdPB0rzSPZ2YwMN5+mv/4H5TbvJgrgmAIn4=; b=nGF07MbpCsRCn4Cqe3vWlle5G8fa3+5hxArSpjeFe+jnzETRWij5bRS9v+0ojMepPF J0JRE4/9UeTkFzlCPKF1gb8lSYkp6dMPbj1pDTaQ7R6CzFgzzg7OWUJQSE36h7g2d/0C XiIXcQb21KKa92nvZLlig0fw1ljg4cJMo7LoT8SAddXYlMhu+eHuhFUVi9ZnHC82oFXd yQI1cH4LuVT0blA9A2WyfNfaPCTxcLAVoIHo1TWMxkcQGH/QNV6g2hL0S5Yu+TsIep42 AtgIUJ25Dhj4L4XJANi3Atxz3IbtnvdZRBqlwvlESN96xu51TkrlumNKpcc/cZtV+dWc eGPg== X-Gm-Message-State: AOAM533C3gxqdEQMfwh92ua6pJ69dbgpIJMKI83+uB8GgtnYYF8g8rYF 2WKwJeQwyIQasqXSdtP+M1pUqWxyoT5G/Q== X-Google-Smtp-Source: ABdhPJynu/s2vk/tUF0oWhS/xn0iTnOuUjuSSMnYvzPxrAZZh+AC4SqJZVSM8n+3HbvDQNwY3yhQ1g== X-Received: by 2002:a17:906:3542:: with SMTP id s2mr70316097eja.379.1636094014709; Thu, 04 Nov 2021 23:33:34 -0700 (PDT) From: Oleksandr Andrushchenko To: xen-devel@lists.xenproject.org Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com, volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com, jbeulich@suse.com, andrew.cooper3@citrix.com, george.dunlap@citrix.com, paul@xen.org, bertrand.marquis@arm.com, rahul.singh@arm.com, Oleksandr Andrushchenko Subject: [PATCH v6 4/7] xen/arm: do not map PCI ECAM and MMIO space to Domain-0's p2m Date: Fri, 5 Nov 2021 08:33:23 +0200 Message-Id: <20211105063326.939843-5-andr2000@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211105063326.939843-1-andr2000@gmail.com> References: <20211105063326.939843-1-andr2000@gmail.com> MIME-Version: 1.0 From: Oleksandr Andrushchenko PCI host bridges are special devices in terms of implementing PCI passthrough. According to [1] the current implementation depends on Domain-0 to perform the initialization of the relevant PCI host bridge hardware and perform PCI device enumeration. In order to achieve that one of the required changes is to not map all the memory ranges in map_range_to_domain as we traverse the device tree on startup and perform some additional checks if the range needs to be mapped to Domain-0. The generic PCI host controller device tree binding says [2]: - ranges: As described in IEEE Std 1275-1994, but must provide at least a definition of non-prefetchable memory. One or both of prefetchable Memory and IO Space may also be provided. - reg : The Configuration Space base address and size, as accessed from the parent bus. The base address corresponds to the first bus in the "bus-range" property. If no "bus-range" is specified, this will be bus 0 (the default). From the above none of the memory ranges from the "ranges" property needs to be mapped to Domain-0 at startup as MMIO mapping is going to be handled dynamically by vPCI as we assign PCI devices, e.g. each device assigned to Domain-0/guest will have its MMIOs mapped/unmapped as needed by Xen. The "reg" property covers not only ECAM space, but may also have other then the configuration memory ranges described, for example [3]: - reg: Should contain rc_dbi, config registers location and length. - reg-names: Must include the following entries: "rc_dbi": controller configuration registers; "config": PCIe configuration space registers. This patch makes it possible to not map all the ranges from the "ranges" property and also ECAM from the "reg". All the rest from the "reg" property still needs to be mapped to Domain-0, so the PCI host bridge remains functional in Domain-0. [1] https://lists.xenproject.org/archives/html/xen-devel/2020-07/msg00777.html [2] https://www.kernel.org/doc/Documentation/devicetree/bindings/pci/host-generic-pci.txt [3] https://www.kernel.org/doc/Documentation/devicetree/bindings/pci/hisilicon-pcie.txt Signed-off-by: Oleksandr Andrushchenko --- Since v5: - remove some need_mapping local variables - use own_device in handle_device - add __init for pci_ecam_need_p2m_hwdom_mapping - make pci_host_bridge_mappings use p2m_mmio_direct_dev directly Since v4: - update skip_mapping comment - add comment why we need to map interrupts to Dom0 Since v3: - pass struct map_range_data to map_dt_irq_to_domain - remove redundant check from map_range_to_domain - fix handle_device's .skip_mapping Since v2: - removed check in map_range_to_domain for PCI_DEV and moved it to handle_device, so the code is simpler - s/map_pci_bridge/skip_mapping - extended comment in pci_host_bridge_mappings - minor code restructure in construct_dom0 - s/.need_p2m_mapping/.need_p2m_hwdom_mapping and related callbacks - unsigned int i; in pci_host_bridge_mappings Since v1: - Added better description of why and what needs to be mapped into Domain-0's p2m and what doesn't - Do not do any mappings for PCI devices while traversing the DT - Walk all the bridges and make required mappings in one go --- xen/arch/arm/domain_build.c | 67 +++++++++++++++++------------- xen/arch/arm/pci/ecam.c | 14 +++++++ xen/arch/arm/pci/pci-host-common.c | 50 ++++++++++++++++++++++ xen/arch/arm/pci/pci-host-zynqmp.c | 1 + xen/include/asm-arm/pci.h | 10 +++++ xen/include/asm-arm/setup.h | 13 ++++++ 6 files changed, 126 insertions(+), 29 deletions(-) diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c index f7fcb1400c19..c7d992456ca7 100644 --- a/xen/arch/arm/domain_build.c +++ b/xen/arch/arm/domain_build.c @@ -10,7 +10,6 @@ #include #include #include -#include #include #include #include @@ -51,12 +50,6 @@ static int __init parse_dom0_mem(const char *s) } custom_param("dom0_mem", parse_dom0_mem); -struct map_range_data -{ - struct domain *d; - p2m_type_t p2mt; -}; - /* Override macros from asm/page.h to make them work with mfn_t */ #undef virt_to_mfn #define virt_to_mfn(va) _mfn(__virt_to_mfn(va)) @@ -1676,10 +1669,10 @@ static int __init map_dt_irq_to_domain(const struct dt_device_node *dev, const struct dt_irq *dt_irq, void *data) { - struct domain *d = data; + struct map_range_data *mr_data = data; + struct domain *d = mr_data->d; unsigned int irq = dt_irq->irq; int res; - bool need_mapping = !dt_device_for_passthrough(dev); if ( irq < NR_LOCAL_IRQS ) { @@ -1698,18 +1691,16 @@ static int __init map_dt_irq_to_domain(const struct dt_device_node *dev, return res; } - res = map_irq_to_domain(d, irq, need_mapping, dt_node_name(dev)); + res = map_irq_to_domain(d, irq, !mr_data->skip_mapping, dt_node_name(dev)); return 0; } -static int __init map_range_to_domain(const struct dt_device_node *dev, - u64 addr, u64 len, - void *data) +int __init map_range_to_domain(const struct dt_device_node *dev, + u64 addr, u64 len, void *data) { struct map_range_data *mr_data = data; struct domain *d = mr_data->d; - bool need_mapping = !dt_device_for_passthrough(dev); int res; res = iomem_permit_access(d, paddr_to_pfn(addr), @@ -1723,7 +1714,7 @@ static int __init map_range_to_domain(const struct dt_device_node *dev, return res; } - if ( need_mapping ) + if ( !mr_data->skip_mapping ) { res = map_regions_p2mt(d, gaddr_to_gfn(addr), @@ -1752,23 +1743,21 @@ static int __init map_range_to_domain(const struct dt_device_node *dev, * then we may need to perform additional mappings in order to make * the child resources available to domain 0. */ -static int __init map_device_children(struct domain *d, - const struct dt_device_node *dev, - p2m_type_t p2mt) +static int __init map_device_children(const struct dt_device_node *dev, + struct map_range_data *mr_data) { - struct map_range_data mr_data = { .d = d, .p2mt = p2mt }; - int ret; - if ( dt_device_type_is_equal(dev, "pci") ) { + int ret; + dt_dprintk("Mapping children of %s to guest\n", dt_node_full_name(dev)); - ret = dt_for_each_irq_map(dev, &map_dt_irq_to_domain, d); + ret = dt_for_each_irq_map(dev, &map_dt_irq_to_domain, mr_data); if ( ret < 0 ) return ret; - ret = dt_for_each_range(dev, &map_range_to_domain, &mr_data); + ret = dt_for_each_range(dev, &map_range_to_domain, mr_data); if ( ret < 0 ) return ret; } @@ -1848,14 +1837,28 @@ static int __init handle_device(struct domain *d, struct dt_device_node *dev, unsigned int i; int res; u64 addr, size; - bool need_mapping = !dt_device_for_passthrough(dev); + bool own_device = !dt_device_for_passthrough(dev); + /* + * For PCI passthrough we only need to remap to Dom0 the interrupts + * and memory ranges from "reg" property which cover controller's + * configuration registers and such. PCIe configuration space registers + * of the PCIe Root Complex and PCIe aperture should not be mapped + * automatically to Dom0. + */ + struct map_range_data mr_data = { + .d = d, + .p2mt = p2mt, + .skip_mapping = !own_device || + (is_pci_passthrough_enabled() && + (device_get_class(dev) == DEVICE_PCI_HOSTBRIDGE)) + }; naddr = dt_number_of_address(dev); dt_dprintk("%s passthrough = %d naddr = %u\n", - dt_node_full_name(dev), need_mapping, naddr); + dt_node_full_name(dev), own_device, naddr); - if ( need_mapping ) + if ( own_device ) { dt_dprintk("Check if %s is behind the IOMMU and add it\n", dt_node_full_name(dev)); @@ -1881,14 +1884,13 @@ static int __init handle_device(struct domain *d, struct dt_device_node *dev, } } - res = handle_device_interrupts(d, dev, need_mapping); + res = handle_device_interrupts(d, dev, own_device); if ( res < 0 ) return res; /* Give permission and map MMIOs */ for ( i = 0; i < naddr; i++ ) { - struct map_range_data mr_data = { .d = d, .p2mt = p2mt }; res = dt_device_get_address(dev, i, &addr, &size); if ( res ) { @@ -1902,7 +1904,7 @@ static int __init handle_device(struct domain *d, struct dt_device_node *dev, return res; } - res = map_device_children(d, dev, p2mt); + res = map_device_children(dev, &mr_data); if ( res ) return res; @@ -3060,7 +3062,14 @@ static int __init construct_dom0(struct domain *d) return rc; if ( acpi_disabled ) + { rc = prepare_dtb_hwdom(d, &kinfo); + if ( rc < 0 ) + return rc; +#ifdef CONFIG_HAS_PCI + rc = pci_host_bridge_mappings(d); +#endif + } else rc = prepare_acpi(d, &kinfo); diff --git a/xen/arch/arm/pci/ecam.c b/xen/arch/arm/pci/ecam.c index 602d00799c8d..4f71b11c3057 100644 --- a/xen/arch/arm/pci/ecam.c +++ b/xen/arch/arm/pci/ecam.c @@ -40,6 +40,19 @@ void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge, return base + (PCI_DEVFN2(sbdf.bdf) << devfn_shift) + where; } +bool __init pci_ecam_need_p2m_hwdom_mapping(struct domain *d, + struct pci_host_bridge *bridge, + uint64_t addr) +{ + struct pci_config_window *cfg = bridge->cfg; + + /* + * We do not want ECAM address space to be mapped in Domain-0's p2m, + * so we can trap access to it. + */ + return cfg->phys_addr != addr; +} + /* ECAM ops */ const struct pci_ecam_ops pci_generic_ecam_ops = { .bus_shift = 20, @@ -47,6 +60,7 @@ const struct pci_ecam_ops pci_generic_ecam_ops = { .map_bus = pci_ecam_map_bus, .read = pci_generic_config_read, .write = pci_generic_config_write, + .need_p2m_hwdom_mapping = pci_ecam_need_p2m_hwdom_mapping, } }; diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c index 0d271a6e8881..6af845ab9d6c 100644 --- a/xen/arch/arm/pci/pci-host-common.c +++ b/xen/arch/arm/pci/pci-host-common.c @@ -22,6 +22,8 @@ #include #include +#include + /* * List for all the pci host bridges. */ @@ -316,6 +318,54 @@ unsigned int pci_host_get_num_bridges(void) return count; } +int __init pci_host_bridge_mappings(struct domain *d) +{ + struct pci_host_bridge *bridge; + struct map_range_data mr_data = { + .d = d, + .p2mt = p2m_mmio_direct_dev, + .skip_mapping = false + }; + + /* + * For each PCI host bridge we need to only map those ranges + * which are used by Domain-0 to properly initialize the bridge, + * e.g. we do not want to map ECAM configuration space which lives in + * "reg" device tree property, but we want to map other regions of + * the host bridge. The PCI aperture defined by the "ranges" device + * tree property should also be skipped. + */ + list_for_each_entry( bridge, &pci_host_bridges, node ) + { + const struct dt_device_node *dev = bridge->dt_node; + unsigned int i; + + for ( i = 0; i < dt_number_of_address(dev); i++ ) + { + uint64_t addr, size; + int err; + + err = dt_device_get_address(dev, i, &addr, &size); + if ( err ) + { + printk(XENLOG_ERR + "Unable to retrieve address range index=%u for %s\n", + i, dt_node_full_name(dev)); + return err; + } + + if ( bridge->ops->need_p2m_hwdom_mapping(d, bridge, addr) ) + { + err = map_range_to_domain(dev, addr, size, &mr_data); + if ( err ) + return err; + } + } + } + + return 0; +} + /* * Local variables: * mode: C diff --git a/xen/arch/arm/pci/pci-host-zynqmp.c b/xen/arch/arm/pci/pci-host-zynqmp.c index 516982bca833..101edb8593c1 100644 --- a/xen/arch/arm/pci/pci-host-zynqmp.c +++ b/xen/arch/arm/pci/pci-host-zynqmp.c @@ -34,6 +34,7 @@ const struct pci_ecam_ops nwl_pcie_ops = { .map_bus = pci_ecam_map_bus, .read = pci_generic_config_read, .write = pci_generic_config_write, + .need_p2m_hwdom_mapping = pci_ecam_need_p2m_hwdom_mapping, } }; diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h index 969333043431..3d706fdd1d88 100644 --- a/xen/include/asm-arm/pci.h +++ b/xen/include/asm-arm/pci.h @@ -17,6 +17,8 @@ #ifdef CONFIG_HAS_PCI +#include + #define pci_to_dev(pcidev) (&(pcidev)->arch.dev) extern bool pci_passthrough_enabled; @@ -73,6 +75,9 @@ struct pci_ops { uint32_t reg, uint32_t len, uint32_t *value); int (*write)(struct pci_host_bridge *bridge, pci_sbdf_t sbdf, uint32_t reg, uint32_t len, uint32_t value); + bool (*need_p2m_hwdom_mapping)(struct domain *d, + struct pci_host_bridge *bridge, + uint64_t addr); }; /* @@ -96,6 +101,9 @@ int pci_generic_config_write(struct pci_host_bridge *bridge, pci_sbdf_t sbdf, uint32_t reg, uint32_t len, uint32_t value); void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge, pci_sbdf_t sbdf, uint32_t where); +bool pci_ecam_need_p2m_hwdom_mapping(struct domain *d, + struct pci_host_bridge *bridge, + uint64_t addr); struct pci_host_bridge *pci_find_host_bridge(uint16_t segment, uint8_t bus); struct dt_device_node *pci_find_host_bridge_node(struct device *dev); int pci_get_host_bridge_segment(const struct dt_device_node *node, @@ -115,6 +123,8 @@ int pci_host_iterate_bridges(struct domain *d, struct pci_host_bridge *bridge)); unsigned int pci_host_get_num_bridges(void); +int pci_host_bridge_mappings(struct domain *d); + #else /*!CONFIG_HAS_PCI*/ struct arch_pci_dev { }; diff --git a/xen/include/asm-arm/setup.h b/xen/include/asm-arm/setup.h index 95da0b7ab9cd..88d9673db817 100644 --- a/xen/include/asm-arm/setup.h +++ b/xen/include/asm-arm/setup.h @@ -2,6 +2,8 @@ #define __ARM_SETUP_H_ #include +#include +#include #define MIN_FDT_ALIGN 8 #define MAX_FDT_SIZE SZ_2M @@ -77,6 +79,14 @@ struct bootinfo { #endif }; +struct map_range_data +{ + struct domain *d; + p2m_type_t p2mt; + /* Set if mapping of the memory ranges must be skipped. */ + bool skip_mapping; +}; + extern struct bootinfo bootinfo; extern domid_t max_init_domid; @@ -124,6 +134,9 @@ void device_tree_get_reg(const __be32 **cell, u32 address_cells, u32 device_tree_get_u32(const void *fdt, int node, const char *prop_name, u32 dflt); +int map_range_to_domain(const struct dt_device_node *dev, + u64 addr, u64 len, void *data); + #endif /* * Local variables: From patchwork Fri Nov 5 06:33:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oleksandr Andrushchenko X-Patchwork-Id: 12604265 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C6A16C4321E for ; Fri, 5 Nov 2021 06:33:57 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8C8FD611C4 for ; Fri, 5 Nov 2021 06:33:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 8C8FD611C4 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.222024.384056 (Exim 4.92) (envelope-from ) id 1misnC-000243-UR; Fri, 05 Nov 2021 06:33:38 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 222024.384056; Fri, 05 Nov 2021 06:33:38 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1misnC-00023j-OZ; Fri, 05 Nov 2021 06:33:38 +0000 Received: by outflank-mailman (input) for mailman id 222024; Fri, 05 Nov 2021 06:33:37 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1misnB-0000uf-9c for xen-devel@lists.xenproject.org; Fri, 05 Nov 2021 06:33:37 +0000 Received: from mail-ed1-x532.google.com (mail-ed1-x532.google.com [2a00:1450:4864:20::532]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 4e1ba8e0-3e02-11ec-a9d2-d9f7a1cc8784; Fri, 05 Nov 2021 07:33:36 +0100 (CET) Received: by mail-ed1-x532.google.com with SMTP id g10so29446069edj.1 for ; Thu, 04 Nov 2021 23:33:36 -0700 (PDT) Received: from localhost.localdomain ([185.199.97.5]) by smtp.gmail.com with ESMTPSA id e12sm3599870ejs.86.2021.11.04.23.33.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 04 Nov 2021 23:33:35 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 4e1ba8e0-3e02-11ec-a9d2-d9f7a1cc8784 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=stTfdYmsCC/2Kmn4pDoWefR6cFnWNl9nnA6XN7kP6KA=; b=RrUXMZjfuRVcVwWaN83xyLLOv4csdbKiElqWVwCIZByOoI8zw9325+q2N4a37Antnw JBSWsmLXh5F3pheSFKHoqulolgtSO1RQT5Ym7U2tvGEdK7DPW1o3ALSFMaCA5sLtMAtO VPeuXIu1LANoaf3CKEdmL4UiWDPJ27eqlJCXnnSAbARIoClGPfvHrfrp+iPLNOeburDV MplYa1LjpQ1FcOvWC/czuim/YVDtA3i5h+9XtLp5WIiXP3iCNkMlx2Sm/G3cI62ihJ9V muZlfDBMz/4U21Cb+9fvPmolf0+ZqELYpzj2hRoWNTnPva8eXxdmm564aTYsj9yB28bI 1+sw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=stTfdYmsCC/2Kmn4pDoWefR6cFnWNl9nnA6XN7kP6KA=; b=MnUpQjQPeJJMKhfj9hnYBzmUVQ5nPZGJ5FYyH5bfHradQcgmyuZ+DfO9RPTss5joaa zrc9ejVxzQcFSJMUBFqGweMZbUwn8gLU4tDiCCsR79TtqGPxnb7N3jaGIm4AF2u75Wiz fIASPf7Gwp+eTw9tYlwvE0cpx+RyFoOu7Pn0l0frOYq9lb1T3CQxAlZG1kPM7hbzt6sQ qxlsD3Rdh0ymFQsl7a0Xy/ucd345Re7cKjEdXVPpxJGMro+0xv2Nmc1FkmXSL0SL8GEt RkTUQzNuRaf541PodpC2hDylYfPF7DQclc0QFfJmTMGjGPYJubAt1ywy12CR5BLAkbyZ ST4g== X-Gm-Message-State: AOAM5307D5ofmX++QbaglSgLhbXTiqJ7qJRLJ22b4/pzArlqReyJLjSs Gui1LYU9l4Xh4xRyalS1/LUvh2hge7k2pw== X-Google-Smtp-Source: ABdhPJxrKjutd4WGx5j5BYryIufBgqCfi+owCQzpGgyAM2oQP58FDlI26VRqE+2qFGV4IRSp83bPTg== X-Received: by 2002:a17:906:5641:: with SMTP id v1mr19942107ejr.357.1636094015861; Thu, 04 Nov 2021 23:33:35 -0700 (PDT) From: Oleksandr Andrushchenko To: xen-devel@lists.xenproject.org Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com, volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com, jbeulich@suse.com, andrew.cooper3@citrix.com, george.dunlap@citrix.com, paul@xen.org, bertrand.marquis@arm.com, rahul.singh@arm.com, Oleksandr Andrushchenko Subject: [PATCH v6 5/7] xen/arm: do not map IRQs and memory for disabled devices Date: Fri, 5 Nov 2021 08:33:24 +0200 Message-Id: <20211105063326.939843-6-andr2000@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211105063326.939843-1-andr2000@gmail.com> References: <20211105063326.939843-1-andr2000@gmail.com> MIME-Version: 1.0 From: Oleksandr Andrushchenko Currently Xen maps all IRQs and memory ranges for all devices except those marked for passthrough, e.g. it doesn't pay attention to the "status" property of the node. According to the device tree specification [1]: - "okay" Indicates the device is operational. - "disabled" Indicates that the device is not presently operational, but it might become operational in the future (for example, something is not plugged in, or switched off). Refer to the device binding for details on what disabled means for a given device. So, "disabled" status is device dependent and mapping should be taken by case-by-case approach with that respect. Although in general Xen should map IRQs and memory ranges as the disabled devices might become operational it makes it impossible for the other devices, which are not operational in any case, to skip the mappings. This patch disables mapping for the devices with status = "disabled". [1] https://github.com/devicetree-org/devicetree-specification/releases/download/v0.3/devicetree-specification-v0.3.pdf Signed-off-by: Oleksandr Andrushchenko --- New in v6 --- xen/arch/arm/domain_build.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c index c7d992456ca7..d3a4c0a173b8 100644 --- a/xen/arch/arm/domain_build.c +++ b/xen/arch/arm/domain_build.c @@ -1837,7 +1837,8 @@ static int __init handle_device(struct domain *d, struct dt_device_node *dev, unsigned int i; int res; u64 addr, size; - bool own_device = !dt_device_for_passthrough(dev); + bool own_device = !dt_device_for_passthrough(dev) && + dt_device_is_available(dev); /* * For PCI passthrough we only need to remap to Dom0 the interrupts * and memory ranges from "reg" property which cover controller's From patchwork Fri Nov 5 06:33:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oleksandr Andrushchenko X-Patchwork-Id: 12604273 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1E14BC433EF for ; Fri, 5 Nov 2021 06:33:58 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CA2E9611C4 for ; Fri, 5 Nov 2021 06:33:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org CA2E9611C4 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.222025.384063 (Exim 4.92) (envelope-from ) id 1misnD-0002GP-Ko; Fri, 05 Nov 2021 06:33:39 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 222025.384063; Fri, 05 Nov 2021 06:33:39 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1misnD-0002E9-EJ; Fri, 05 Nov 2021 06:33:39 +0000 Received: by outflank-mailman (input) for mailman id 222025; Fri, 05 Nov 2021 06:33:38 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1misnC-0000uf-BP for xen-devel@lists.xenproject.org; Fri, 05 Nov 2021 06:33:38 +0000 Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com [2a00:1450:4864:20::533]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 4ed02a88-3e02-11ec-a9d2-d9f7a1cc8784; Fri, 05 Nov 2021 07:33:37 +0100 (CET) Received: by mail-ed1-x533.google.com with SMTP id r4so28862671edi.5 for ; Thu, 04 Nov 2021 23:33:37 -0700 (PDT) Received: from localhost.localdomain ([185.199.97.5]) by smtp.gmail.com with ESMTPSA id e12sm3599870ejs.86.2021.11.04.23.33.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 04 Nov 2021 23:33:36 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 4ed02a88-3e02-11ec-a9d2-d9f7a1cc8784 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=sGK8Pm6RypWgXJkvMD4+e3RPRWZCdgGUZOOGFT2QhCc=; b=i7ywQryvHdyLIfZJGgicojKb9SnEXWuT1Bj9bVk6hTWVxo91p52uTyU0HR4FNKFYZi dzeslv82jS4cHhWqhsCvXNG2exjYcpMtDAUG34M7SW1HF0uO+8xJHz2QGPMgtRiDKXxB 2vLLz098HV7e4qaeu21VvmDtBjsxw4gUaTAbAJra8dzuPQ+YQxw3P/WlRDwtPvmaDC2Q vPwdfNV1vYMzdN35G97zbyD2CMph+IHMZXd1rQA5JCUKRrHbw8ykvIbB/EjAcDM8SEJq p+V2alnGb9FUbkYler+gX0VXQssM37lXJu13di3G0eGc2Lg1D2o5TAW2/uI65DBUpkei D3BA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=sGK8Pm6RypWgXJkvMD4+e3RPRWZCdgGUZOOGFT2QhCc=; b=hAbZUx9ZPAKocGyZ+cvNcgaxAHdFZ+2ahqlMA3MnWmhde4M5bg2RAZQlzphb8SCVLN MQUvHbQimbCf1v99jBesJ6Iefz/61wZpBS1NuEWCMgze6lSkpuEoWbywyVU3XioP7E+y XIz6+25k+zyTB9f1T5cBcxSUlBaGCGMBFaKd3RnEhTR4mWpyok2aLs61H4QQd3LYuo0x a7aBB5EWGK8WBZPZy2/iGnjdQ/VUAjtnc98Moy++/4wsoOCEFo/USX9kmy0tTTSBaZX1 aABW3bpND5bb7QUYpNG5U1dArUInoWJ9hdDhgEA9cdzlWCjvpF7xSakJFZpN1DTJvkQT itJA== X-Gm-Message-State: AOAM530AMHmToPNvFqJ/LQvhoOyABCNT2s+Iyg859Iesj6s25fjWgS9I vJw/n/zwxTHhXy5PP4V2+gIvCwsx8dqS3g== X-Google-Smtp-Source: ABdhPJxj576TShH4DBFMpmVbDp7CpVwPbsc9NWydnsmtIkF+CjmzVn+aOPMN4Hce10ZKTtAbYmEdww== X-Received: by 2002:a17:906:c187:: with SMTP id g7mr67130837ejz.534.1636094016970; Thu, 04 Nov 2021 23:33:36 -0700 (PDT) From: Oleksandr Andrushchenko To: xen-devel@lists.xenproject.org Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com, volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com, jbeulich@suse.com, andrew.cooper3@citrix.com, george.dunlap@citrix.com, paul@xen.org, bertrand.marquis@arm.com, rahul.singh@arm.com, Oleksandr Andrushchenko Subject: [PATCH v6 6/7] xen/arm: process pending vPCI map/unmap operations Date: Fri, 5 Nov 2021 08:33:25 +0200 Message-Id: <20211105063326.939843-7-andr2000@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211105063326.939843-1-andr2000@gmail.com> References: <20211105063326.939843-1-andr2000@gmail.com> MIME-Version: 1.0 From: Oleksandr Andrushchenko vPCI may map and unmap PCI device memory (BARs) being passed through which may take a lot of time. For this those operations may be deferred to be performed later, so that they can be safely preempted. Currently this deferred processing is happening in common IOREQ code which doesn't seem to be the right place for x86 and is even more doubtful because IOREQ may not be enabled for Arm at all. So, for Arm the pending vPCI work may have no chance to be executed if the processing is left as is in the common IOREQ code only. For that reason make vPCI processing happen in arch specific code. Please be aware that there are a few outstanding TODOs affecting this code path, see xen/drivers/vpci/header.c:map_range and xen/drivers/vpci/header.c:vpci_process_pending. Signed-off-by: Oleksandr Andrushchenko Acked-by: Jan Beulich Reviewed-by: Julien Grall --- Cc: Andrew Cooper Cc: Paul Durrant Since v5: - check_for_vcpu_work: vPCI addition is moved before the vcpu_ioreq__handle_completion(v). This is to avoid differences with the x86 version. (Julien) Since v2: - update commit message with more insight on x86, IOREQ and Arm - restored order of invocation for IOREQ and vPCI processing (Jan) Since v1: - Moved the check for pending vpci work from the common IOREQ code to hvm_do_resume on x86 - Re-worked the code for Arm to ensure we don't miss pending vPCI work --- xen/arch/arm/traps.c | 13 +++++++++++++ xen/arch/x86/hvm/hvm.c | 6 ++++++ xen/common/ioreq.c | 9 --------- 3 files changed, 19 insertions(+), 9 deletions(-) diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c index 219ab3c3fbde..8757210a798b 100644 --- a/xen/arch/arm/traps.c +++ b/xen/arch/arm/traps.c @@ -34,6 +34,7 @@ #include #include #include +#include #include #include @@ -2290,6 +2291,18 @@ static bool check_for_vcpu_work(void) { struct vcpu *v = current; + if ( has_vpci(v->domain) ) + { + bool pending; + + local_irq_enable(); + pending = vpci_process_pending(v); + local_irq_disable(); + + if ( pending ) + return true; + } + #ifdef CONFIG_IOREQ_SERVER if ( domain_has_ioreq_server(v->domain) ) { diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c index eee365711d63..096a61b7ea02 100644 --- a/xen/arch/x86/hvm/hvm.c +++ b/xen/arch/x86/hvm/hvm.c @@ -546,6 +546,12 @@ void hvm_do_resume(struct vcpu *v) pt_restore_timer(v); + if ( has_vpci(v->domain) && vpci_process_pending(v) ) + { + raise_softirq(SCHEDULE_SOFTIRQ); + return; + } + if ( !vcpu_ioreq_handle_completion(v) ) return; diff --git a/xen/common/ioreq.c b/xen/common/ioreq.c index d732dc045df9..689d256544c8 100644 --- a/xen/common/ioreq.c +++ b/xen/common/ioreq.c @@ -25,9 +25,7 @@ #include #include #include -#include #include -#include #include #include @@ -212,19 +210,12 @@ static bool wait_for_io(struct ioreq_vcpu *sv, ioreq_t *p) bool vcpu_ioreq_handle_completion(struct vcpu *v) { - struct domain *d = v->domain; struct vcpu_io *vio = &v->io; struct ioreq_server *s; struct ioreq_vcpu *sv; enum vio_completion completion; bool res = true; - if ( has_vpci(d) && vpci_process_pending(v) ) - { - raise_softirq(SCHEDULE_SOFTIRQ); - return false; - } - while ( (sv = get_pending_vcpu(v, &s)) != NULL ) if ( !wait_for_io(sv, get_ioreq(s, v)) ) return false; From patchwork Fri Nov 5 06:33:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oleksandr Andrushchenko X-Patchwork-Id: 12604271 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 20EB6C43217 for ; Fri, 5 Nov 2021 06:33:59 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DDCAB611C4 for ; Fri, 5 Nov 2021 06:33:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org DDCAB611C4 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.222026.384081 (Exim 4.92) (envelope-from ) id 1misnF-0002k1-5k; Fri, 05 Nov 2021 06:33:41 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 222026.384081; Fri, 05 Nov 2021 06:33:41 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1misnE-0002i1-S8; Fri, 05 Nov 2021 06:33:40 +0000 Received: by outflank-mailman (input) for mailman id 222026; Fri, 05 Nov 2021 06:33:39 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1misnD-0001ER-Hv for xen-devel@lists.xenproject.org; Fri, 05 Nov 2021 06:33:39 +0000 Received: from mail-ed1-x534.google.com (mail-ed1-x534.google.com [2a00:1450:4864:20::534]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 4f8eac75-3e02-11ec-9787-a32c541c8605; Fri, 05 Nov 2021 07:33:38 +0100 (CET) Received: by mail-ed1-x534.google.com with SMTP id j21so29439095edt.11 for ; Thu, 04 Nov 2021 23:33:38 -0700 (PDT) Received: from localhost.localdomain ([185.199.97.5]) by smtp.gmail.com with ESMTPSA id e12sm3599870ejs.86.2021.11.04.23.33.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 04 Nov 2021 23:33:37 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 4f8eac75-3e02-11ec-9787-a32c541c8605 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=LylF1cpLHbISjBJqhF+mgwyYAEJHR3syGzKHfKTNLi4=; b=Gb1Tzc+FiWeAo4gYgZJI7/zVVhxSsRR70ZC7KSuPQvNGguptQB3sNexSAp5CwJI5VJ /K2eB1ZF8c98NyO/gtNCcQKPCZni/nPcUtnYGaX1mcSNU1P1u839zJDKFkRcYZlAgVxS PfUwixFuQgTRq9ATCMli/7FLRqm/QVwsQ8a//7FSGlnj1JOirpsL0ctL2Nx58pgE6/X6 x+gtsipn21Vr0nhRB5c0q1UtxPSeSe+aqIRW6ZeFKNiEylI37ZWSoz+tuWvO54ratC3m xZ4ihes4NrAbGYrkOTH+FY30KwAgvEH2kPptjphIpP9HTLNEFSszxF3vDm7HKkS6qRGp iWNA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=LylF1cpLHbISjBJqhF+mgwyYAEJHR3syGzKHfKTNLi4=; b=eXtkw8VwrXPw88OZ55aO5RS8DgS+v8kGzOHn8A5hol3dSujPxZD6ObFHbtlQH5KIfG lKFxoA6E2o056IKyuN5iJZUogpnCKpKpCo1y32nyRzbtyNTlETiwAEUY5dkMrDxqsjS6 aB/t3bHXFwFCfmzclZLClMq8ivVHzJ20AOYdY4Eu/3v3fqbqwyZRCPf8ZIGXKpJUfvUg xnIJ/g2GJBVjgzlRR4hmwHtBx7HOFm9UmBlR/STCN40Fbt8h5oq5jwf9v9z36U0ufJ3x Z6K9+jsTiyaghfpuD9zbYO3aItzhXIg0p3QdpK0vNmLC4GZo4hDORThVGRF8Xzt+OCTB 4PkQ== X-Gm-Message-State: AOAM532ei8AhDJeJdt/mKFkalIpY3EruU3MMr/9FHoFgvVgqPps8DF8W WYZCHtfMNDyTKx/COKCyWOlRM/2FhMAVfA== X-Google-Smtp-Source: ABdhPJx6KXyOXUui/Xab210y17uEpTCWAHBBf70htETILb3uWGGNACWCe6wxS5hfGSRplFUVJGz2Tw== X-Received: by 2002:a17:907:6289:: with SMTP id nd9mr68670282ejc.101.1636094018309; Thu, 04 Nov 2021 23:33:38 -0700 (PDT) From: Oleksandr Andrushchenko To: xen-devel@lists.xenproject.org Cc: julien@xen.org, sstabellini@kernel.org, oleksandr_tyshchenko@epam.com, volodymyr_babchuk@epam.com, Artem_Mygaiev@epam.com, roger.pau@citrix.com, jbeulich@suse.com, andrew.cooper3@citrix.com, george.dunlap@citrix.com, paul@xen.org, bertrand.marquis@arm.com, rahul.singh@arm.com, Oleksandr Andrushchenko Subject: [PATCH v6 7/7] xen/arm: do not use void pointer in pci_host_common_probe Date: Fri, 5 Nov 2021 08:33:26 +0200 Message-Id: <20211105063326.939843-8-andr2000@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211105063326.939843-1-andr2000@gmail.com> References: <20211105063326.939843-1-andr2000@gmail.com> MIME-Version: 1.0 From: Oleksandr Andrushchenko There is no reason to use void pointer while passing ECAM ops to the pci_host_common_probe function as it is anyway casted to struct pci_ecam_ops inside. For that reason remove the void pointer and pass struct pci_ecam_ops pointer as is. Signed-off-by: Oleksandr Andrushchenko Reviewed-by: Rahul Singh Tested-by: Rahul Singh Acked-by: Julien Grall --- New in v4 --- xen/arch/arm/pci/ecam.c | 4 ++-- xen/arch/arm/pci/pci-host-common.c | 6 ++---- xen/include/asm-arm/pci.h | 5 +++-- 3 files changed, 7 insertions(+), 8 deletions(-) diff --git a/xen/arch/arm/pci/ecam.c b/xen/arch/arm/pci/ecam.c index 4f71b11c3057..6aeea12a68bf 100644 --- a/xen/arch/arm/pci/ecam.c +++ b/xen/arch/arm/pci/ecam.c @@ -24,8 +24,8 @@ void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge, pci_sbdf_t sbdf, uint32_t where) { const struct pci_config_window *cfg = bridge->cfg; - struct pci_ecam_ops *ops = - container_of(bridge->ops, struct pci_ecam_ops, pci_ops); + const struct pci_ecam_ops *ops = + container_of(bridge->ops, const struct pci_ecam_ops, pci_ops); unsigned int devfn_shift = ops->bus_shift - 8; void __iomem *base; diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c index 6af845ab9d6c..1aad664b213e 100644 --- a/xen/arch/arm/pci/pci-host-common.c +++ b/xen/arch/arm/pci/pci-host-common.c @@ -194,15 +194,13 @@ static int pci_bus_find_domain_nr(struct dt_device_node *dev) return domain; } -int pci_host_common_probe(struct dt_device_node *dev, const void *data) +int pci_host_common_probe(struct dt_device_node *dev, + const struct pci_ecam_ops *ops) { struct pci_host_bridge *bridge; struct pci_config_window *cfg; - struct pci_ecam_ops *ops; int err; - ops = (struct pci_ecam_ops *) data; - bridge = pci_alloc_host_bridge(); if ( !bridge ) return -ENOMEM; diff --git a/xen/include/asm-arm/pci.h b/xen/include/asm-arm/pci.h index 3d706fdd1d88..4199e0267d24 100644 --- a/xen/include/asm-arm/pci.h +++ b/xen/include/asm-arm/pci.h @@ -65,7 +65,7 @@ struct pci_host_bridge { struct list_head node; /* Node in list of host bridges */ uint16_t segment; /* Segment number */ struct pci_config_window* cfg; /* Pointer to the bridge config window */ - struct pci_ops *ops; + const struct pci_ops *ops; }; struct pci_ops { @@ -94,7 +94,8 @@ struct pci_ecam_ops { /* Default ECAM ops */ extern const struct pci_ecam_ops pci_generic_ecam_ops; -int pci_host_common_probe(struct dt_device_node *dev, const void *data); +int pci_host_common_probe(struct dt_device_node *dev, + const struct pci_ecam_ops *ops); int pci_generic_config_read(struct pci_host_bridge *bridge, pci_sbdf_t sbdf, uint32_t reg, uint32_t len, uint32_t *value); int pci_generic_config_write(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,