From patchwork Wed Feb 19 16:48:38 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Roger_Pau_Monn=C3=A9?= X-Patchwork-Id: 13982532 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 79BE6C021B1 for ; Wed, 19 Feb 2025 16:49:02 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.893243.1302157 (Exim 4.92) (envelope-from ) id 1tknFh-0000E9-CF; Wed, 19 Feb 2025 16:48:49 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 893243.1302157; Wed, 19 Feb 2025 16:48:49 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tknFh-0000E2-9i; Wed, 19 Feb 2025 16:48:49 +0000 Received: by outflank-mailman (input) for mailman id 893243; Wed, 19 Feb 2025 16:48:48 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tknFg-0000DW-0W for xen-devel@lists.xenproject.org; Wed, 19 Feb 2025 16:48:48 +0000 Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com [2a00:1450:4864:20::62d]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 62fb6d15-eee1-11ef-9aa8-95dc52dad729; Wed, 19 Feb 2025 17:48:46 +0100 (CET) Received: by mail-ej1-x62d.google.com with SMTP id a640c23a62f3a-aaecf50578eso3688466b.2 for ; Wed, 19 Feb 2025 08:48:46 -0800 (PST) Received: from localhost ([84.78.159.3]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-abb8190d15esm835632166b.16.2025.02.19.08.48.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Feb 2025 08:48:45 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 62fb6d15-eee1-11ef-9aa8-95dc52dad729 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1739983726; x=1740588526; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4HQ77vgWRo/U9WVLabLYV75AdriUUwFMryn5FJ3I67w=; b=cY0goX8bIFuAuOFnkU4OQTIwshEeX1/Aj4O3NVzgbzQ6mp/o2h9YvncsuwX1W03M+d xnupG6mSfVhhbc0j+s93IjA0D39e8t4ruL3K3xS3sPXM0j+LAsp6uR0xl4g4hN2Pbi+a uZQiKHRsAiPDeUP0YqBIdeyJTyVaRW0iimUHg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739983726; x=1740588526; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4HQ77vgWRo/U9WVLabLYV75AdriUUwFMryn5FJ3I67w=; b=nh0G7Lp/+xGiVVWqOd4N4gUKlYZ0ORv6nSu8n93A5vo75XxsJEUbOwUL/8dFoxX1Oq kifiGNIZnovPnRm5P8JV/t/rZxTG+6z3HJOMBQ6EPlu6MCQtltuFXZJGRMt5aNFzuNB1 fW1Soc9GRcj3vOk6CwwtZathQ4oU+VIn7/EMXkQrttYWD0T2NB3ZVcvmGoic5uNgLzBo 2gvFldn+OrnmYijJYh00Z+AL+NF36/ToOxmY7gFrnFkuIKCq40Rcb4AJ62S/BAGVeOcq 3DKPEMlbG05UWlpiDqWd8o/kXKpSOSYXxna7YGgyZwX7+/9v+i6nA0jV+q0+kC0SpS/L 0Aig== X-Gm-Message-State: AOJu0YyiaRYI/6an/hmx7va7+NG7F+NJ507AcCkjpAp1FGlTGlY8hA9H Jf5ocFPs7+axrzhDv/X6DOsA3hqE6XqIGLqbwOv7AjCNwOCNqq9BXbKI7hOMYtCnQotQg+yth8w R X-Gm-Gg: ASbGnctvfaZR7kh9618ysPBbV5Q8eW047CIG9+xCeqI0fAwoSEzhI+CYe2B6WPUyWCy WNdxIAjT2vstpMX/2VJJKIbdWUjxS2G7uC1yfO7y1MnWyKXhGX0fKSslVM2XwmQYMFfsHDwY1eS pnHxVdABtqdoRcFYk+iPU33lUM1XBlWzkzHDM1zYcgt0+ZnJ28QecA5b56CDP9Ip6Ct1Xkw7X3J OK3ZsTivTlDCgx0whRjSwDqR1mrbZcHauCM+d32RafqgoDxl48AmSjFP5I77i74lvkFE6kNal6N LHiSJ0tXu9Vdo9oQsp4VIupTvQ== X-Google-Smtp-Source: AGHT+IEqn/P+qTDgM2IVPcZZGQlbRFzyxFXvh3EM1ihOl44/a6KY+SpzBeoyKXmr9VC5IKtiHxejDw== X-Received: by 2002:a17:906:3118:b0:ab7:63fa:e4ab with SMTP id a640c23a62f3a-abb7053d8cfmr1617742466b.0.1739983726075; Wed, 19 Feb 2025 08:48:46 -0800 (PST) From: Roger Pau Monne To: xen-devel@lists.xenproject.org Cc: Roger Pau Monne , Jan Beulich , Andrew Cooper Subject: [PATCH v3 1/3] x86/dom0: correctly set the maximum ->iomem_caps bound for PVH Date: Wed, 19 Feb 2025 17:48:38 +0100 Message-ID: <20250219164840.94803-2-roger.pau@citrix.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20250219164840.94803-1-roger.pau@citrix.com> References: <20250219164840.94803-1-roger.pau@citrix.com> MIME-Version: 1.0 The logic in dom0_setup_permissions() sets the maximum bound in ->iomem_caps unconditionally using paddr_bits, which is not correct for HVM based domains. Instead use domain_max_paddr_bits() to get the correct maximum paddr bits for each possible domain type. Switch to using PFN_DOWN() instead of PAGE_SHIFT, as that's shorter. Fixes: 53de839fb409 ('x86: constrain MFN range Dom0 may access') Signed-off-by: Roger Pau Monné Reviewed-by: Jan Beulich --- The fixes tag might be dubious, IIRC at that time we had PVHv1 dom0, which would likely also need such adjustment, but not the current PVHv2. --- Changes since v2: - New in this version. --- xen/arch/x86/dom0_build.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/xen/arch/x86/dom0_build.c b/xen/arch/x86/dom0_build.c index 3b9681dc9134..aec596997d5d 100644 --- a/xen/arch/x86/dom0_build.c +++ b/xen/arch/x86/dom0_build.c @@ -481,7 +481,8 @@ int __init dom0_setup_permissions(struct domain *d) /* The hardware domain is initially permitted full I/O capabilities. */ rc = ioports_permit_access(d, 0, 0xFFFF); - rc |= iomem_permit_access(d, 0UL, (1UL << (paddr_bits - PAGE_SHIFT)) - 1); + rc |= iomem_permit_access(d, 0UL, + PFN_DOWN(1UL << domain_max_paddr_bits(d)) - 1); rc |= irqs_permit_access(d, 1, nr_irqs_gsi - 1); /* Modify I/O port access permissions. */ From patchwork Wed Feb 19 16:48:39 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Roger_Pau_Monn=C3=A9?= X-Patchwork-Id: 13982533 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 97191C021B3 for ; Wed, 19 Feb 2025 16:49:02 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.893245.1302178 (Exim 4.92) (envelope-from ) id 1tknFj-0000hQ-Tx; Wed, 19 Feb 2025 16:48:51 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 893245.1302178; Wed, 19 Feb 2025 16:48:51 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tknFj-0000hE-R8; Wed, 19 Feb 2025 16:48:51 +0000 Received: by outflank-mailman (input) for mailman id 893245; Wed, 19 Feb 2025 16:48:50 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tknFi-0000PL-A9 for xen-devel@lists.xenproject.org; Wed, 19 Feb 2025 16:48:50 +0000 Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com [2a00:1450:4864:20::635]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 63a87d46-eee1-11ef-9896-31a8f345e629; Wed, 19 Feb 2025 17:48:48 +0100 (CET) Received: by mail-ej1-x635.google.com with SMTP id a640c23a62f3a-abadccdfe5aso7621566b.0 for ; Wed, 19 Feb 2025 08:48:48 -0800 (PST) Received: from localhost ([84.78.159.3]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-abb8915db0dsm779901866b.145.2025.02.19.08.48.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Feb 2025 08:48:46 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 63a87d46-eee1-11ef-9896-31a8f345e629 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1739983727; x=1740588527; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=PIcVCHNr5p+mIW/1mO/10d/AjhTI/+c0zIyZX/AmKhs=; b=muVylDycbm1qm+6jsUTe98hG7X/1L26y4/Yr1X7r9cmHagb/xiEZy7fHXcU7xCh1Aw 27hikg5hfI0ddXISbkdCVigPbdP33ONjZg4ggXDe9NBlN84B2iUZlDxi7QS8FM1AXuH3 2agFw90Fe9ksxOAHN0/PhFtDkeqtF2nC/VgvI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739983727; x=1740588527; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PIcVCHNr5p+mIW/1mO/10d/AjhTI/+c0zIyZX/AmKhs=; b=XWLIdG0ScBkT4pksrPUsU/OlxTKNnNpPwEQiwl5EYtbYq/HzU0tVAEGiy2bfRBvlNr bjKLpv1kpRWM54QWZz7UTWakck3UtjnwDheEqb1jPJkzu3npTlFEvadkRdIMdZXaePo8 QPCBOqm+kAemlnqWxf7ylIE7xMFssc9RLOUI/QbNPI00LNumltqDZb1UZzdQk2RMfY/C iEu//FRXoSxRPTmheUusEbA4Hf2hCyclTcJEkkJNJ5d8cmMCfCfrE5S2xOSzbJXoRlBs MDMnOu85JqHo04CN6YMHkdSz+yg+kBPfh7xpZZoj6h+bGTg4ZipsJRws1wO8bEJ2W6m4 8NGA== X-Gm-Message-State: AOJu0YyXO5UG4NbXXOUolas+Ji+bdMiT52qW07M3eIk6RoYM81XtaGPZ ci+12wt6K06a1iW+Ea2svsdwG67F9gS+YkBcVWESP2Ey3/ZhZ30icfLf+nWRsSB5Iu3xjZb8KS0 Q X-Gm-Gg: ASbGnctu4tq+mBTauApY6iKlKc4Q2W4MME6RNKASyxRPLYd3hVA2XvgIhXnParixKYY Prl+e7agNn2wiy4GbluoFFOpWpEObafQOQvvSIbJu7XkN+WsSkQgDeKh45kbCOPwrP+67bqsihH dYWo9jg5XARuJcsep3ytJYi6YgBA6PmhU6ixUDV0Uh+qSjhppxuU+nVpOT08ijzGKGnnl3Pfqbn 4zU0Hk9Px0XsiSjvSp51T/8JHW6qeTUJVsKO17XBbUwNbx11dDZDox2J1fLXVWdtFzLOTK30mlu kVC/3FUiI6VhmmIJ/hVdVAS3bQ== X-Google-Smtp-Source: AGHT+IF7eHC8KUW3sQ8ASr/fkhpo6/VTZ9Le7O5onB5o7zUnW5r+MHOz2TR+0XL1tryCPrH5ECKfRQ== X-Received: by 2002:a17:906:4794:b0:ab7:7cf7:9f7a with SMTP id a640c23a62f3a-abb70db8437mr2011960566b.53.1739983727132; Wed, 19 Feb 2025 08:48:47 -0800 (PST) From: Roger Pau Monne To: xen-devel@lists.xenproject.org Cc: Roger Pau Monne , Jan Beulich , Andrew Cooper Subject: [PATCH v3 2/3] x86/iommu: account for IOMEM caps when populating dom0 IOMMU page-tables Date: Wed, 19 Feb 2025 17:48:39 +0100 Message-ID: <20250219164840.94803-3-roger.pau@citrix.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20250219164840.94803-1-roger.pau@citrix.com> References: <20250219164840.94803-1-roger.pau@citrix.com> MIME-Version: 1.0 The current code in arch_iommu_hwdom_init() kind of open-codes the same MMIO permission ranges that are added to the hardware domain ->iomem_caps. Avoid this duplication and use ->iomem_caps in arch_iommu_hwdom_init() to filter which memory regions should be added to the dom0 IOMMU page-tables. Note the IO-APIC and MCFG page(s) must be set as not accessible for a PVH dom0, otherwise the internal Xen emulation for those ranges won't work. This requires adjustments in dom0_setup_permissions(). The call to pvh_setup_mmcfg() in dom0_construct_pvh() must now strictly be done ahead of setting up dom0 permissions, so take the opportunity to also put it inside the existing is_hardware_domain() region. Also the special casing of E820_UNUSABLE regions no longer needs to be done in arch_iommu_hwdom_init(), as those regions are already blocked in ->iomem_caps and thus would be removed from the rangeset as part of ->iomem_caps processing in arch_iommu_hwdom_init(). The E820_UNUSABLE regions below 1Mb are not removed from ->iomem_caps, that's a slight difference for the IOMMU created page-tables, but the aim is to allow access to the same memory either from the CPU or the IOMMU page-tables. Since ->iomem_caps already takes into account the domain max paddr, there's no need to remove any regions past the last address addressable by the domain, as applying ->iomem_caps would have already taken care of that. Suggested-by: Jan Beulich Signed-off-by: Roger Pau Monné Reviewed-by: Jan Beulich --- Changes since v2: - Expand commit message re E820_UNUSABLE. - Use vpci_mmcfg_deny_access(). - Remove max() from map_subtract_iomemcap(). - Use has_vioapic() instead of is_hvm_domain(). Changes since v1: - New in this version. --- xen/arch/x86/dom0_build.c | 11 ++++- xen/arch/x86/hvm/dom0_build.c | 14 +++--- xen/arch/x86/hvm/io.c | 6 +-- xen/arch/x86/include/asm/hvm/io.h | 4 +- xen/drivers/passthrough/x86/iommu.c | 67 ++++++++++++----------------- 5 files changed, 49 insertions(+), 53 deletions(-) diff --git a/xen/arch/x86/dom0_build.c b/xen/arch/x86/dom0_build.c index aec596997d5d..a735e3650c28 100644 --- a/xen/arch/x86/dom0_build.c +++ b/xen/arch/x86/dom0_build.c @@ -558,7 +558,9 @@ int __init dom0_setup_permissions(struct domain *d) for ( i = 0; i < nr_ioapics; i++ ) { mfn = paddr_to_pfn(mp_ioapics[i].mpc_apicaddr); - if ( !rangeset_contains_singleton(mmio_ro_ranges, mfn) ) + /* If emulating IO-APIC(s) make sure the base address is unmapped. */ + if ( has_vioapic(d) || + !rangeset_contains_singleton(mmio_ro_ranges, mfn) ) rc |= iomem_deny_access(d, mfn, mfn); } /* MSI range. */ @@ -599,6 +601,13 @@ int __init dom0_setup_permissions(struct domain *d) rc |= rangeset_add_singleton(mmio_ro_ranges, mfn); } + if ( has_vpci(d) ) + /* + * TODO: runtime added MMCFG regions are not checked to make sure they + * don't overlap with already mapped regions, thus preventing trapping. + */ + rc |= vpci_mmcfg_deny_access(d); + return rc; } diff --git a/xen/arch/x86/hvm/dom0_build.c b/xen/arch/x86/hvm/dom0_build.c index ce5b5c31b318..6a4453103a9a 100644 --- a/xen/arch/x86/hvm/dom0_build.c +++ b/xen/arch/x86/hvm/dom0_build.c @@ -1323,6 +1323,13 @@ int __init dom0_construct_pvh(struct boot_info *bi, struct domain *d) if ( is_hardware_domain(d) ) { + /* + * MMCFG initialization must be performed before setting domain + * permissions, as the MCFG areas must not be part of the domain IOMEM + * accessible regions. + */ + pvh_setup_mmcfg(d); + /* * Setup permissions early so that calls to add MMIO regions to the * p2m as part of vPCI setup don't fail due to permission checks. @@ -1335,13 +1342,6 @@ int __init dom0_construct_pvh(struct boot_info *bi, struct domain *d) } } - /* - * NB: MMCFG initialization needs to be performed before iommu - * initialization so the iommu code can fetch the MMCFG regions used by the - * domain. - */ - pvh_setup_mmcfg(d); - /* * Craft dom0 physical memory map and set the paging allocation. This must * be done before the iommu initializion, since iommu initialization code diff --git a/xen/arch/x86/hvm/io.c b/xen/arch/x86/hvm/io.c index db726b38177b..de6ee6c4dd4d 100644 --- a/xen/arch/x86/hvm/io.c +++ b/xen/arch/x86/hvm/io.c @@ -363,14 +363,14 @@ static const struct hvm_mmcfg *vpci_mmcfg_find(const struct domain *d, return NULL; } -int __hwdom_init vpci_subtract_mmcfg(const struct domain *d, struct rangeset *r) +int __hwdom_init vpci_mmcfg_deny_access(struct domain *d) { const struct hvm_mmcfg *mmcfg; list_for_each_entry ( mmcfg, &d->arch.hvm.mmcfg_regions, next ) { - int rc = rangeset_remove_range(r, PFN_DOWN(mmcfg->addr), - PFN_DOWN(mmcfg->addr + mmcfg->size - 1)); + int rc = iomem_deny_access(d, PFN_DOWN(mmcfg->addr), + PFN_DOWN(mmcfg->addr + mmcfg->size - 1)); if ( rc ) return rc; diff --git a/xen/arch/x86/include/asm/hvm/io.h b/xen/arch/x86/include/asm/hvm/io.h index f2b8431facb0..565bad300d20 100644 --- a/xen/arch/x86/include/asm/hvm/io.h +++ b/xen/arch/x86/include/asm/hvm/io.h @@ -132,8 +132,8 @@ int register_vpci_mmcfg_handler(struct domain *d, paddr_t addr, /* Destroy tracked MMCFG areas. */ void destroy_vpci_mmcfg(struct domain *d); -/* Remove MMCFG regions from a given rangeset. */ -int vpci_subtract_mmcfg(const struct domain *d, struct rangeset *r); +/* Remove MMCFG regions from a domain ->iomem_caps. */ +int vpci_mmcfg_deny_access(struct domain *d); #endif /* __ASM_X86_HVM_IO_H__ */ diff --git a/xen/drivers/passthrough/x86/iommu.c b/xen/drivers/passthrough/x86/iommu.c index 8b1e0596b84a..67f025c1ec6a 100644 --- a/xen/drivers/passthrough/x86/iommu.c +++ b/xen/drivers/passthrough/x86/iommu.c @@ -320,6 +320,26 @@ static int __hwdom_init cf_check map_subtract(unsigned long s, unsigned long e, return rangeset_remove_range(map, s, e); } +struct handle_iomemcap { + struct rangeset *r; + unsigned long last; +}; +static int __hwdom_init cf_check map_subtract_iomemcap(unsigned long s, + unsigned long e, + void *data) +{ + struct handle_iomemcap *h = data; + int rc = 0; + + if ( h->last != s ) + rc = rangeset_remove_range(h->r, h->last, s - 1); + + ASSERT(e < ~0UL); + h->last = e + 1; + + return rc; +} + struct map_data { struct domain *d; unsigned int flush_flags; @@ -400,6 +420,7 @@ void __hwdom_init arch_iommu_hwdom_init(struct domain *d) unsigned int i; struct rangeset *map; struct map_data map_data = { .d = d }; + struct handle_iomemcap iomem = {}; int rc; BUG_ON(!is_hardware_domain(d)); @@ -442,14 +463,6 @@ void __hwdom_init arch_iommu_hwdom_init(struct domain *d) switch ( entry.type ) { - case E820_UNUSABLE: - /* Only relevant for inclusive mode, otherwise this is a no-op. */ - rc = rangeset_remove_range(map, PFN_DOWN(entry.addr), - PFN_DOWN(entry.addr + entry.size - 1)); - if ( rc ) - panic("IOMMU failed to remove unusable memory: %d\n", rc); - continue; - case E820_RESERVED: if ( !iommu_hwdom_inclusive && !iommu_hwdom_reserved ) continue; @@ -475,22 +488,13 @@ void __hwdom_init arch_iommu_hwdom_init(struct domain *d) if ( rc ) panic("IOMMU failed to remove Xen ranges: %d\n", rc); - /* Remove any overlap with the Interrupt Address Range. */ - rc = rangeset_remove_range(map, 0xfee00, 0xfeeff); + iomem.r = map; + rc = rangeset_report_ranges(d->iomem_caps, 0, ~0UL, map_subtract_iomemcap, + &iomem); + if ( !rc && iomem.last < ~0UL ) + rc = rangeset_remove_range(map, iomem.last, ~0UL); if ( rc ) - panic("IOMMU failed to remove Interrupt Address Range: %d\n", rc); - - /* If emulating IO-APIC(s) make sure the base address is unmapped. */ - if ( has_vioapic(d) ) - { - for ( i = 0; i < d->arch.hvm.nr_vioapics; i++ ) - { - rc = rangeset_remove_singleton(map, - PFN_DOWN(domain_vioapic(d, i)->base_address)); - if ( rc ) - panic("IOMMU failed to remove IO-APIC: %d\n", rc); - } - } + panic("IOMMU failed to remove forbidden regions: %d\n", rc); if ( is_pv_domain(d) ) { @@ -506,23 +510,6 @@ void __hwdom_init arch_iommu_hwdom_init(struct domain *d) panic("IOMMU failed to remove read-only regions: %d\n", rc); } - if ( has_vpci(d) ) - { - /* - * TODO: runtime added MMCFG regions are not checked to make sure they - * don't overlap with already mapped regions, thus preventing trapping. - */ - rc = vpci_subtract_mmcfg(d, map); - if ( rc ) - panic("IOMMU unable to remove MMCFG areas: %d\n", rc); - } - - /* Remove any regions past the last address addressable by the domain. */ - rc = rangeset_remove_range(map, PFN_DOWN(1UL << domain_max_paddr_bits(d)), - ~0UL); - if ( rc ) - panic("IOMMU unable to remove unaddressable ranges: %d\n", rc); - if ( iommu_verbose ) printk(XENLOG_INFO "%pd: identity mappings for IOMMU:\n", d); From patchwork Wed Feb 19 16:48:40 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Roger_Pau_Monn=C3=A9?= X-Patchwork-Id: 13982530 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1C226C021B0 for ; Wed, 19 Feb 2025 16:49:01 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.893244.1302167 (Exim 4.92) (envelope-from ) id 1tknFi-0000T2-MV; Wed, 19 Feb 2025 16:48:50 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 893244.1302167; Wed, 19 Feb 2025 16:48:50 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tknFi-0000Sv-JS; Wed, 19 Feb 2025 16:48:50 +0000 Received: by outflank-mailman (input) for mailman id 893244; Wed, 19 Feb 2025 16:48:50 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tknFh-0000DW-Vi for xen-devel@lists.xenproject.org; Wed, 19 Feb 2025 16:48:49 +0000 Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com [2a00:1450:4864:20::62b]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 646123cd-eee1-11ef-9aa8-95dc52dad729; Wed, 19 Feb 2025 17:48:49 +0100 (CET) Received: by mail-ej1-x62b.google.com with SMTP id a640c23a62f3a-ab78e6edb99so3464766b.2 for ; Wed, 19 Feb 2025 08:48:49 -0800 (PST) Received: from localhost ([84.78.159.3]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aba532322aesm1323094666b.17.2025.02.19.08.48.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Feb 2025 08:48:47 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 646123cd-eee1-11ef-9aa8-95dc52dad729 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1739983728; x=1740588528; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=M0Lo7NvC0VRrNHzOZExT/+zaLj5LgD90jhn2nZ96ddg=; b=gE5hZFgYGn8+h3gwzjqCiulpTjBoizu4wyQUZoGAC8wQLSALmJR1nJcPMwcE83ZQSH 6ESHgDoTj/QDgDRXuQ/9+oPXdNhTtWwVLHgA7XtL1Fi1hu4w4ZajFdSoDz4ANb5p/nAx INVvF7WArxSz2eEsruct5wEefLiKAaKbesTFo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739983728; x=1740588528; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=M0Lo7NvC0VRrNHzOZExT/+zaLj5LgD90jhn2nZ96ddg=; b=N+fX2j3viej6HiJfHJyOrda8W2ZfuUzE4AdP+C+y3QmV8Cc0BVaVz8jNUnOyHJqe0S uQPu52qTzcHlqTGaD4fSN7lvejabqG+t7X+WsgwNd0Xjcuzs7/3IE1c1q3IsipSb4cX8 vXt7PmRuKGIB+Kud8KUypN3be/DjETKgSdk2JG4u0E1FtHe0NaSSIMHtT0PG0MfhIMtK Cgp4zmgbXehcyfbh7XLQTctbMan9ErrIWwcXucwuzwS174qsaRhy/MNWgJjRv+lXCcBS W/1gElkG81V+GEswQHVDnoi9zwlbkS3z0wG0CfwCKZaKcaCyOCb0eSJds5uo4A49jb0p jHXQ== X-Gm-Message-State: AOJu0Yz6mRcx6gxQEppgu8TJmCwHVS+g6j/lCiBLJ4uegfai60N4E+zN 5LNJTJHc4Z5mFaAstpuxHk+9DzI09Qcyd6ehCL6yDWNS3Tuc4IMl0V5YxCan9kks7y5t75qx9VZ 6 X-Gm-Gg: ASbGncslsexdSKU4Ukz3lcI8BeV126lGKRAc6nEO7Xzi0u8rZGBupK2Cpj2tFFNZZdr MYNh5jzO6MEbe/U9w3bjboIJe7J/dN1WZrv+sHRUfqYyUWpfp/KlqmceIFvFxt40kIabe98I5TV AVgIOOnWW9rt7l+Q/plvvpfhs2JZYIjDiRGTFtznkoCtVDKbct+QqQFXI8TqY79+BXjkSA0SvYv v79+V8ROV86/IGv2zhdh5GtW14NODUbIaio/MUvCpMeQjPYYrqofuBp9OBPTjtr6YiFN4Dp4+Pn sI5cERxipW0/bF9PBGaRVF0PtA== X-Google-Smtp-Source: AGHT+IFCoMTmLpt40lLnU2dkJWR4NY0jL2YkPmeDYlmpNOicP6g0IpUBTmfb3noiCaj72nEBIRQkcg== X-Received: by 2002:a17:906:308b:b0:ab6:cdc2:bf57 with SMTP id a640c23a62f3a-abbccccb763mr405103066b.1.1739983728251; Wed, 19 Feb 2025 08:48:48 -0800 (PST) From: Roger Pau Monne To: xen-devel@lists.xenproject.org Cc: Roger Pau Monne , Jan Beulich , Andrew Cooper , =?utf-8?b?SsO8cmdlbiBHcm/Dnw==?= Subject: [PATCH v3 3/3] x86/dom0: be less restrictive with the Interrupt Address Range Date: Wed, 19 Feb 2025 17:48:40 +0100 Message-ID: <20250219164840.94803-4-roger.pau@citrix.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20250219164840.94803-1-roger.pau@citrix.com> References: <20250219164840.94803-1-roger.pau@citrix.com> MIME-Version: 1.0 Xen currently prevents dom0 from creating CPU or IOMMU page-table mappings into the interrupt address range [0xfee00000, 0xfeefffff]. This range has two different purposes. For accesses from the CPU is contains the default position of local APIC page at 0xfee00000. For accesses from devices it's the MSI address range, so the address field in the MSI entries (usually) point to an address on that range to trigger an interrupt. There are reports of Lenovo Thinkpad devices placing what seems to be the UCSI shared mailbox at address 0xfeec2000 in the interrupt address range. Attempting to use that device with a Linux PV dom0 leads to an error when Linux kernel maps 0xfeec2000: RIP: e030:xen_mc_flush+0x1e8/0x2b0 xen_leave_lazy_mmu+0x15/0x60 vmap_range_noflush+0x408/0x6f0 __ioremap_caller+0x20d/0x350 acpi_os_map_iomem+0x1a3/0x1c0 acpi_ex_system_memory_space_handler+0x229/0x3f0 acpi_ev_address_space_dispatch+0x17e/0x4c0 acpi_ex_access_region+0x28a/0x510 acpi_ex_field_datum_io+0x95/0x5c0 acpi_ex_extract_from_field+0x36b/0x4e0 acpi_ex_read_data_from_field+0xcb/0x430 acpi_ex_resolve_node_to_value+0x2e0/0x530 acpi_ex_resolve_to_value+0x1e7/0x550 acpi_ds_evaluate_name_path+0x107/0x170 acpi_ds_exec_end_op+0x392/0x860 acpi_ps_parse_loop+0x268/0xa30 acpi_ps_parse_aml+0x221/0x5e0 acpi_ps_execute_method+0x171/0x3e0 acpi_ns_evaluate+0x174/0x5d0 acpi_evaluate_object+0x167/0x440 acpi_evaluate_dsm+0xb6/0x130 ucsi_acpi_dsm+0x53/0x80 ucsi_acpi_read+0x2e/0x60 ucsi_register+0x24/0xa0 ucsi_acpi_probe+0x162/0x1e3 platform_probe+0x48/0x90 really_probe+0xde/0x340 __driver_probe_device+0x78/0x110 driver_probe_device+0x1f/0x90 __driver_attach+0xd2/0x1c0 bus_for_each_dev+0x77/0xc0 bus_add_driver+0x112/0x1f0 driver_register+0x72/0xd0 do_one_initcall+0x48/0x300 do_init_module+0x60/0x220 __do_sys_init_module+0x17f/0x1b0 do_syscall_64+0x82/0x170 Remove the restrictions to create mappings the interrupt address range for dom0. Note that the restriction to map the local APIC page is enforced separately, and that continues to be present. Additionally make sure the emulated local APIC page is also not mapped, in case dom0 is using it. Note that even if the interrupt address range entries are populated in the IOMMU page-tables no device access will reach those pages. Device accesses to the Interrupt Address Range will always be converted into Interrupt Messages and are not subject to DMA remapping. There's also the following restriction noted in Intel VT-d: > Software must not program paging-structure entries to remap any address to > the interrupt address range. Untranslated requests and translation requests > that result in an address in the interrupt range will be blocked with > condition code LGN.4 or SGN.8. Translated requests with an address in the > interrupt address range are treated as Unsupported Request (UR). Similarly for AMD-Vi: > Accesses to the interrupt address range (Table 3) are defined to go through > the interrupt remapping portion of the IOMMU and not through address > translation processing. Therefore, when a transaction is being processed as > an interrupt remapping operation, the transaction attribute of > pretranslated or untranslated is ignored. > > Software Note: The IOMMU should > not be configured such that an address translation results in a special > address such as the interrupt address range. However those restrictions don't apply to the identity mappings possibly created for dom0, since the interrupt address range is never subject to DMA remapping, and hence there's no output address after translation that belongs to the interrupt address range. Reported-by: Jürgen Groß Link: https://lore.kernel.org/xen-devel/baade0a7-e204-4743-bda1-282df74e5f89@suse.com/ Signed-off-by: Roger Pau Monné Acked-by: Jan Beulich --- Changes since v2: - Make sure vlapic page is not mapped. Changes since v1: - No longer needs to modify arch_iommu_hwdom_init(). --- xen/arch/x86/dom0_build.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/xen/arch/x86/dom0_build.c b/xen/arch/x86/dom0_build.c index a735e3650c28..3cda0c2c8765 100644 --- a/xen/arch/x86/dom0_build.c +++ b/xen/arch/x86/dom0_build.c @@ -554,6 +554,12 @@ int __init dom0_setup_permissions(struct domain *d) mfn = paddr_to_pfn(mp_lapic_addr); rc |= iomem_deny_access(d, mfn, mfn); } + /* If using an emulated local APIC make sure its MMIO is unpopulated. */ + if ( has_vlapic(d) ) + { + mfn = paddr_to_pfn(APIC_DEFAULT_PHYS_BASE); + rc |= iomem_deny_access(d, mfn, mfn); + } /* I/O APICs. */ for ( i = 0; i < nr_ioapics; i++ ) { @@ -563,10 +569,6 @@ int __init dom0_setup_permissions(struct domain *d) !rangeset_contains_singleton(mmio_ro_ranges, mfn) ) rc |= iomem_deny_access(d, mfn, mfn); } - /* MSI range. */ - rc |= iomem_deny_access(d, paddr_to_pfn(MSI_ADDR_BASE_LO), - paddr_to_pfn(MSI_ADDR_BASE_LO + - MSI_ADDR_DEST_ID_MASK)); /* HyperTransport range. */ if ( boot_cpu_data.x86_vendor & (X86_VENDOR_AMD | X86_VENDOR_HYGON) ) {