From patchwork Fri Jan 24 12:01:07 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Roger_Pau_Monn=C3=A9?= X-Patchwork-Id: 13949325 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C99B2C02181 for ; Fri, 24 Jan 2025 12:01:37 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.876703.1287061 (Exim 4.92) (envelope-from ) id 1tbING-0007qG-RQ; Fri, 24 Jan 2025 12:01:22 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 876703.1287061; Fri, 24 Jan 2025 12:01:22 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tbING-0007q7-OY; Fri, 24 Jan 2025 12:01:22 +0000 Received: by outflank-mailman (input) for mailman id 876703; Fri, 24 Jan 2025 12:01:21 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tbINF-0007hy-PJ for xen-devel@lists.xenproject.org; Fri, 24 Jan 2025 12:01:21 +0000 Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com [2a00:1450:4864:20::52a]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id ec466dff-da4a-11ef-a0e5-8be0dac302b0; Fri, 24 Jan 2025 13:01:19 +0100 (CET) Received: by mail-ed1-x52a.google.com with SMTP id 4fb4d7f45d1cf-5d3cf094768so3917706a12.0 for ; Fri, 24 Jan 2025 04:01:19 -0800 (PST) Received: from localhost ([84.78.159.3]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5dc18619351sm1097563a12.5.2025.01.24.04.01.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 24 Jan 2025 04:01:18 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: ec466dff-da4a-11ef-a0e5-8be0dac302b0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1737720079; x=1738324879; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9c2aZdvI/tbV2jY/GC3KfB8zoMuoAgzT2lT7lgwZi+o=; b=pLXBBQ8Yb1Ji0UKZVibD6anqwq5PmLrHcq6uBNaT287b7pBz/0DQ0tiEIMc+ZaWIOV 32Dxj51cp9YNiFylI/H4qhwxEB/WDx/pQZsbW4g1J/WlfU2Fr8cXCW6N+UWTYdoiLW7C SxGctemr51Bm0ooktuc8o8Q1DTQf7Pg2GaduU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737720079; x=1738324879; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9c2aZdvI/tbV2jY/GC3KfB8zoMuoAgzT2lT7lgwZi+o=; b=TqIneucwXtEyog3tRl/6MSgr0aF9Cv0vRfItOWLRuFmX4eQNW32h2bhCMt2wXZUAv4 LmeNY8g6k5AnBjWr1cbba4GFOFgVef2uej7fWBDMRC1F9fZtiXewHuTu2yA1pP8K4kyX h/H2t0TYIcLekVJEoEQv07Fp8uHJ2dr/EGxtFHFzUK1+inhUCiJmyEFp1lfggZqAtBGv cpKK8cdjhxcOxm0dfXblOMR4szJQ8EAVcqx5ldwZW6SZXcXIpbk6rvCEbz3n2zodE0kB v7+ROXSa9KhrIq89hkdUni4PTUeG4bDLylT4L2LQK8FVekez7yDh1802To0jUnRtv7OT dFNQ== X-Gm-Message-State: AOJu0Yz1u8LmEw2YLNqpiZe4KzAfaCyiH5ddW6Gal7x8NHyPzJrRc+X0 kSsFUZGc1HrxRLFmn/NAtm6uTXelIj2aX8/+gHti4Q51e3YuUszldwEQFOiRSd3y7ExrtPI8teh V X-Gm-Gg: ASbGncvF+j/d0eO+ifBuSS8kYwTtbpD3EPNUigu2vb/kb2vxHTBnO5xDxCT1TU6Stlk p+ziDgHlKoI5Iq3BfB5T9+T66CN6qppJmVDFekHYiwGgNX4TokL8Jq19zbwKcvYkfMAWu4AU2+q Lnt7r18VZIqS8oVqOPVeGIanTb4cGS7zqFqLJcYI/JkAg0p2j52FQdmcdrOBMhHRmrJ4U7aA4kB AfmBDTy6OVlaSSuQm/DA0M7k2yHK7ooX89n2fkGSxGOhQCMqHI+Nlx6MeP7M3XIablpRqFlt6gU OdSg9mNEa8qEvy4= X-Google-Smtp-Source: AGHT+IGtakv+VJopi6Jb6eXJ3iLF9dXAUCjJf06TVnJ0NdxUWVim75lvthE0amCrD/9kE8DxqYdmCQ== X-Received: by 2002:a05:6402:348b:b0:5db:67a7:e75e with SMTP id 4fb4d7f45d1cf-5db7d2ec452mr27893397a12.5.1737720079038; Fri, 24 Jan 2025 04:01:19 -0800 (PST) From: Roger Pau Monne To: xen-devel@lists.xenproject.org Cc: Teddy Astie , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Subject: [PATCH v2 1/5] x86/iommu: check for CMPXCHG16B when enabling IOMMU Date: Fri, 24 Jan 2025 13:01:07 +0100 Message-ID: <20250124120112.56678-2-roger.pau@citrix.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20250124120112.56678-1-roger.pau@citrix.com> References: <20250124120112.56678-1-roger.pau@citrix.com> MIME-Version: 1.0 From: Teddy Astie All hardware with VT-d/AMD-Vi has CMPXCHG16B support. Check this at initialisation time, and remove the effectively-dead logic for the non-cx16 case. If the local APICs support x2APIC mode the IOMMU support for interrupt remapping will be checked earlier using a specific helper. If no support for CX16 is detected by that earlier hook disable the IOMMU at that point and prevent further poking for CX16 later in the boot process, which would also fail. There's a possible corner case when running virtualized, and the underlying hypervisor exposing an IOMMU but no CMPXCHG16B support. In which case ignoring the IOMMU is fine, albeit the most natural would be for the underlying hypervisor to also expose CMPXCHG16B support if an IOMMU is available to the VM. Note this change only introduces the checks, but doesn't remove the now stale checks for CX16 support sprinkled in the IOMMU code. Further changes will take care of that. Suggested-by: Andrew Cooper Signed-off-by: Teddy Astie Signed-off-by: Roger Pau Monné --- Changes since pickup: - Unify all CX16 checks into a single patch. --- xen/drivers/passthrough/amd/iommu_intr.c | 13 +++++++++++++ xen/drivers/passthrough/amd/pci_amd_iommu.c | 6 ++++++ xen/drivers/passthrough/vtd/intremap.c | 13 +++++++++++++ xen/drivers/passthrough/vtd/iommu.c | 7 +++++++ 4 files changed, 39 insertions(+) diff --git a/xen/drivers/passthrough/amd/iommu_intr.c b/xen/drivers/passthrough/amd/iommu_intr.c index 7fc796dec25b..f07fd9e3d970 100644 --- a/xen/drivers/passthrough/amd/iommu_intr.c +++ b/xen/drivers/passthrough/amd/iommu_intr.c @@ -649,6 +649,19 @@ bool __init cf_check iov_supports_xt(void) if ( !iommu_enable || !iommu_intremap ) return false; + if ( unlikely(!cpu_has_cx16) ) + { + AMD_IOMMU_ERROR("no CMPXCHG16B support, disabling IOMMU\n"); + /* + * Disable IOMMU support at once: there's no reason to check for CX16 + * yet again when attempting to initialize IOMMU DMA remapping + * functionality or interrupt remapping without x2APIC support. + */ + iommu_enable = false; + iommu_intremap = iommu_intremap_off; + return false; + } + if ( amd_iommu_prepare(true) ) return false; diff --git a/xen/drivers/passthrough/amd/pci_amd_iommu.c b/xen/drivers/passthrough/amd/pci_amd_iommu.c index 73dcc4a2dd0c..f96f59440bcc 100644 --- a/xen/drivers/passthrough/amd/pci_amd_iommu.c +++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c @@ -309,6 +309,12 @@ static int __init cf_check iov_detect(void) if ( !iommu_enable && !iommu_intremap ) return 0; + if ( unlikely(!cpu_has_cx16) ) + { + AMD_IOMMU_ERROR("no CMPXCHG16B support, disabling IOMMU\n"); + return -ENODEV; + } + if ( (init_done ? amd_iommu_init_late() : amd_iommu_init(false)) != 0 ) { diff --git a/xen/drivers/passthrough/vtd/intremap.c b/xen/drivers/passthrough/vtd/intremap.c index c504852eb818..233db5cb64de 100644 --- a/xen/drivers/passthrough/vtd/intremap.c +++ b/xen/drivers/passthrough/vtd/intremap.c @@ -150,6 +150,19 @@ bool __init cf_check intel_iommu_supports_eim(void) if ( !iommu_qinval || !iommu_intremap || list_empty(&acpi_drhd_units) ) return false; + if ( unlikely(!cpu_has_cx16) ) + { + printk(XENLOG_ERR VTDPREFIX "no CMPXCHG16B support, disabling IOMMU\n"); + /* + * Disable IOMMU support at once: there's no reason to check for CX16 + * yet again when attempting to initialize IOMMU DMA remapping + * functionality or interrupt remapping without x2APIC support. + */ + iommu_enable = false; + iommu_intremap = iommu_intremap_off; + return false; + } + /* We MUST have a DRHD unit for each IOAPIC. */ for ( apic = 0; apic < nr_ioapics; apic++ ) if ( !ioapic_to_drhd(IO_APIC_ID(apic)) ) diff --git a/xen/drivers/passthrough/vtd/iommu.c b/xen/drivers/passthrough/vtd/iommu.c index 27a4d1640189..3daedc4f5593 100644 --- a/xen/drivers/passthrough/vtd/iommu.c +++ b/xen/drivers/passthrough/vtd/iommu.c @@ -2633,6 +2633,13 @@ static int __init cf_check vtd_setup(void) int ret; bool reg_inval_supported = true; + if ( unlikely(!cpu_has_cx16) ) + { + printk(XENLOG_ERR VTDPREFIX "no CMPXCHG16B support, disabling IOMMU\n"); + ret = -ENODEV; + goto error; + } + if ( list_empty(&acpi_drhd_units) ) { ret = -ENODEV; From patchwork Fri Jan 24 12:01:08 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Roger_Pau_Monn=C3=A9?= X-Patchwork-Id: 13949328 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B1996C0218E for ; Fri, 24 Jan 2025 12:01:38 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.876704.1287071 (Exim 4.92) (envelope-from ) id 1tbINI-00084f-2C; Fri, 24 Jan 2025 12:01:24 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 876704.1287071; Fri, 24 Jan 2025 12:01:24 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tbINH-00084Y-VX; Fri, 24 Jan 2025 12:01:23 +0000 Received: by outflank-mailman (input) for mailman id 876704; Fri, 24 Jan 2025 12:01:22 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tbING-0007hy-Qr for xen-devel@lists.xenproject.org; Fri, 24 Jan 2025 12:01:22 +0000 Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com [2a00:1450:4864:20::62d]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id ed9c120f-da4a-11ef-a0e5-8be0dac302b0; Fri, 24 Jan 2025 13:01:22 +0100 (CET) Received: by mail-ej1-x62d.google.com with SMTP id a640c23a62f3a-ab65fca99b6so392579566b.0 for ; Fri, 24 Jan 2025 04:01:22 -0800 (PST) Received: from localhost ([84.78.159.3]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-ab675e67814sm118162466b.74.2025.01.24.04.01.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 24 Jan 2025 04:01:20 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: ed9c120f-da4a-11ef-a0e5-8be0dac302b0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1737720081; x=1738324881; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zNc8xqr1uTUQn8IFfYe+22YL6fG0Xco7Sp/pdjdKu1o=; b=BOgD76sXrIjdPlAVicXZKECEDeobRT174WkCZNqU3w3O7+wlzTlhrl2hr2Q3wHCQ+q ksTdXdK43RNc3lsJLTRcqlxlF2S67j+fc81S6ocGwXQiRGeXYC1kShbxzEhv1I3/QUzS EYu61ElByuQo9KxVDwmw1UKazAjUcMhz9v8D8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737720081; x=1738324881; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zNc8xqr1uTUQn8IFfYe+22YL6fG0Xco7Sp/pdjdKu1o=; b=N4b4sSuDzrQaTBho+Drp19mebo1X0IwIIz2Lkh7pbdKfyS9dfntMYXHQLNBvvPB3ir F8CgS67J49vRdTaXTWIhWDz39+Af9ydeMG4PSba2sVcT+sCJNVt5rVOrDLxd7wRDH0e4 E+HkW9OJt66CqrQmlQ2I35v4yVzI7aBa7dY/FKQbLY6uIozPmEzGy3zfBAM2bEBgruIm fhTcGt0+UDBvlL5n0iCzshoWY+Heb4jx99zinZR8bCG2neRbwk1gxhDXiD0cXb8ol/dQ AuwGcBzglBa1GQ+zKLt9IQY6D9eiN2phKyjWZ3bnNZekA+EpB/saXCsmIs45LThSbYWr K7bg== X-Gm-Message-State: AOJu0YwtXRn3Xc4O78JdHwVEHfahgVhF3eIYAJSSOG0aOQ1fsUBD8QaK 1SlBBKi7LHHrlPZXwlLTwJAW7Br2Fqqygv1PkxAdbeKvGsyTD/YM9ASE///f8lePSYR+iVptgAh J X-Gm-Gg: ASbGncsSRaxuxtgpPEiIFKQQ5prBU4/bQHVaqhNpZ+iuNga1Z4ls7OSVaR+Tc71iber LRSh1KYcR6Zs+jZqsXRLO67c91D50ua+XImLW9Y42NxOUkR7RJImQXFhUWrTCeUrDoInuNj3iOO SeO57DDQ5MTFPe0pYoUw3iD64U2Lf68ZSQ9c3H1qze2Ys7rKJZuvDPaDV4c/7m3QKnER9lc/S3r /kXC4pC1RHRVUSC2XViVIZ09gB8vXzHBZner162KwVJ+5IU5z9mQfMTFk6epPwu+7K+MWYtaXQM MN/9Z6Qe1eRu20k= X-Google-Smtp-Source: AGHT+IFO3YfpnhWTuhY7pxWYDKGiB6uFLIA/iTPiVnXhamCt+AM7SEGZvi8MUKHqQXMNGwtAo2YyZg== X-Received: by 2002:a17:907:940b:b0:aac:619:7ed8 with SMTP id a640c23a62f3a-ab38b1e651bmr2558293466b.7.1737720080876; Fri, 24 Jan 2025 04:01:20 -0800 (PST) From: Roger Pau Monne To: xen-devel@lists.xenproject.org Cc: Teddy Astie , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Subject: [PATCH v2 2/5] iommu/vtd: remove non-CX16 logic from interrupt remapping Date: Fri, 24 Jan 2025 13:01:08 +0100 Message-ID: <20250124120112.56678-3-roger.pau@citrix.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20250124120112.56678-1-roger.pau@citrix.com> References: <20250124120112.56678-1-roger.pau@citrix.com> MIME-Version: 1.0 From: Teddy Astie As CX16 support is mandatory for IOMMU usage, the checks for CX16 in the interrupt remapping code are stale. Remove them together with the associated code introduced in case CX16 was not available. Note that AMD-Vi support for atomically updating a 128bit IRTE entry is still not implemented, it will be done by further changes. Suggested-by: Andrew Cooper Signed-off-by: Teddy Astie Signed-off-by: Roger Pau Monné --- xen/drivers/passthrough/vtd/intremap.c | 75 +++++--------------------- 1 file changed, 14 insertions(+), 61 deletions(-) diff --git a/xen/drivers/passthrough/vtd/intremap.c b/xen/drivers/passthrough/vtd/intremap.c index 233db5cb64de..820616a8de93 100644 --- a/xen/drivers/passthrough/vtd/intremap.c +++ b/xen/drivers/passthrough/vtd/intremap.c @@ -184,49 +184,26 @@ bool __init cf_check intel_iommu_supports_eim(void) /* * Assume iremap_lock has been acquired. It is to make sure software will not - * change the same IRTE behind us. With this assumption, if only high qword or - * low qword in IRTE is to be updated, this function's atomic variant can - * present an atomic update to VT-d hardware even when cmpxchg16b - * instruction is not supported. + * change the same IRTE behind us. */ static void update_irte(struct vtd_iommu *iommu, struct iremap_entry *entry, const struct iremap_entry *new_ire, bool atomic) { - ASSERT(spin_is_locked(&iommu->intremap.lock)); + __uint128_t ret; + struct iremap_entry old_ire; - if ( cpu_has_cx16 ) - { - __uint128_t ret; - struct iremap_entry old_ire; + ASSERT(spin_is_locked(&iommu->intremap.lock)); - old_ire = *entry; - ret = cmpxchg16b(entry, &old_ire, new_ire); + old_ire = *entry; + ret = cmpxchg16b(entry, &old_ire, new_ire); - /* - * In the above, we use cmpxchg16 to atomically update the 128-bit - * IRTE, and the hardware cannot update the IRTE behind us, so - * the return value of cmpxchg16 should be the same as old_ire. - * This ASSERT validate it. - */ - ASSERT(ret == old_ire.val); - } - else - { - /* - * VT-d hardware doesn't update IRTEs behind us, nor the software - * since we hold iremap_lock. If the caller wants VT-d hardware to - * always see a consistent entry, but we can't meet it, a bug will - * be raised. - */ - if ( entry->lo == new_ire->lo ) - write_atomic(&entry->hi, new_ire->hi); - else if ( entry->hi == new_ire->hi ) - write_atomic(&entry->lo, new_ire->lo); - else if ( !atomic ) - *entry = *new_ire; - else - BUG(); - } + /* + * In the above, we use cmpxchg16 to atomically update the 128-bit + * IRTE, and the hardware cannot update the IRTE behind us, so + * the return value of cmpxchg16 should be the same as old_ire. + * This ASSERT validate it. + */ + ASSERT(ret == old_ire.val); } /* Mark specified intr remap entry as free */ @@ -408,7 +385,6 @@ static int ioapic_rte_to_remap_entry(struct vtd_iommu *iommu, /* Indicate remap format. */ remap_rte->format = 1; - /* If cmpxchg16b is not available the caller must mask the IO-APIC pin. */ update_irte(iommu, iremap_entry, &new_ire, !init && !masked); iommu_sync_cache(iremap_entry, sizeof(*iremap_entry)); iommu_flush_iec_index(iommu, 0, index); @@ -447,38 +423,15 @@ void cf_check io_apic_write_remap_rte( { struct IO_xAPIC_route_entry old_rte = {}, new_rte; struct vtd_iommu *iommu = ioapic_to_iommu(IO_APIC_ID(apic)); - bool masked = true; int rc; - if ( !cpu_has_cx16 ) - { - /* - * Cannot atomically update the IRTE entry: mask the IO-APIC pin to - * avoid interrupts seeing an inconsistent IRTE entry. - */ - old_rte = __ioapic_read_entry(apic, pin, true); - if ( !old_rte.mask ) - { - masked = false; - old_rte.mask = 1; - __ioapic_write_entry(apic, pin, true, old_rte); - } - } - /* Not the initializer, for old gcc to cope. */ new_rte.raw = rte; rc = ioapic_rte_to_remap_entry(iommu, apic, pin, &old_rte, new_rte); if ( rc ) - { - if ( !masked ) - { - /* Recover the original value of 'mask' bit */ - old_rte.mask = 0; - __ioapic_write_entry(apic, pin, true, old_rte); - } return; - } + /* old_rte will contain the updated IO-APIC RTE on success. */ __ioapic_write_entry(apic, pin, true, old_rte); } From patchwork Fri Jan 24 12:01:09 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Roger_Pau_Monn=C3=A9?= X-Patchwork-Id: 13949327 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 294F2C0218D for ; Fri, 24 Jan 2025 12:01:38 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.876705.1287082 (Exim 4.92) (envelope-from ) id 1tbINL-0008Nd-AE; Fri, 24 Jan 2025 12:01:27 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 876705.1287082; Fri, 24 Jan 2025 12:01:27 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tbINL-0008NW-7D; Fri, 24 Jan 2025 12:01:27 +0000 Received: by outflank-mailman (input) for mailman id 876705; Fri, 24 Jan 2025 12:01:25 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tbINJ-0007bK-AC for xen-devel@lists.xenproject.org; Fri, 24 Jan 2025 12:01:25 +0000 Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com [2a00:1450:4864:20::62b]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id ee63f287-da4a-11ef-99a4-01e77a169b0f; Fri, 24 Jan 2025 13:01:23 +0100 (CET) Received: by mail-ej1-x62b.google.com with SMTP id a640c23a62f3a-aa679ad4265so621868266b.0 for ; Fri, 24 Jan 2025 04:01:23 -0800 (PST) Received: from localhost ([84.78.159.3]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-ab675e67814sm118167666b.74.2025.01.24.04.01.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 24 Jan 2025 04:01:21 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: ee63f287-da4a-11ef-99a4-01e77a169b0f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1737720083; x=1738324883; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=DVt6Rj0kKo02aeVOLnGhddbw+hEtfAmvfyBCodPtZ6g=; b=JjODsrszv1U9dsAa9ZtCSc+3fGm8uAtr9j//kzUgExAbu0YBHq76aHArgngrDpcPJY oioCuMphviKhZZTgQ/Q8a42j7EBwgpgmY3LD7NSPSKWFVTaXUUraXHoj1QQ6cUOcqkQP 69G0skxDdJwgaeSN60eoPF7aZniL/+VDw8hsc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737720083; x=1738324883; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DVt6Rj0kKo02aeVOLnGhddbw+hEtfAmvfyBCodPtZ6g=; b=MurSlX3vyiLQv5si8Euj4ULVsE1AbTer5htVUIINxhgfH+VVpmqj+EBM5LE++Opqyr /LS9cFm1KaRrVnGkmAqPQe9a6HMZi5cvGjAwORXenHNV6gy4M8eRZc9lEwnETt6adHoo 9oxvSP36Mn/pg+V6pqGheGTL9ONZ3iqdrZfJELiTW75uPm2ygwFIOOlAvECy2FtFWGZ/ Z2rp5sVQZ1YZdsfkQdWm4HgHIciZUl3KpSUr6owKDepGdyWeiFZgcQGUgddrOGjefsb9 fpSYUEFnZ6OUMlSY6YWNl1BcVYdrO5dA+5lI3DZj2CYgI+Rk3kkub7H0vTpNkflfa+Mo 0Atg== X-Gm-Message-State: AOJu0YyddWyrRp+7WvXOyJ2koWDdRsixLTpsoYjqxMJ9fyPLV7e1Yi5K 02BPFp9ugSz/GORbdROQ0HqiTsMjCQ2vim8oWadhHduGwfgYthpxIm8rTQ2v/O6VlvwwGW3QC0g n X-Gm-Gg: ASbGncvHqKIYM/ikYPd4FjdJQMCOq+sh+n6d+VNULrueC5wDRAXW/Tf0v8cM2+cczXW DCpdlzYta6AcQlaHE9ll3aVAKpZ7zu/r5GDw1F4C0SMzOsDY09RIW848nTHu6yJsnLQCtPRFbR4 9/348KnsCB2YjzDv87o3RWvq62Y53NtUXgJAUm/3zm3c6VRmURBWmn/A4iR27gq6r/vf7JU5fcQ 9zD+weAvBHQqbW0J+/F83EPoU5VRGGFTDmEA5EPKOADZMTh7DcZ9CRVQo5f8e/UnduXT83ouy7c Bdv1MDRf8I0hz7Q= X-Google-Smtp-Source: AGHT+IGR5urwaiLF47OwHM87ZTumEm4WxfrMk8qLxpI+tZWW0hkbPvCEBF+0lO3eKM6AFRu5rh7bEA== X-Received: by 2002:a17:907:3ea0:b0:aa6:88a2:cfbd with SMTP id a640c23a62f3a-ab662a001f0mr501155566b.22.1737720082614; Fri, 24 Jan 2025 04:01:22 -0800 (PST) From: Roger Pau Monne To: xen-devel@lists.xenproject.org Cc: Teddy Astie , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Subject: [PATCH v2 3/5] x86/iommu: remove non-CX16 logic from DMA remapping Date: Fri, 24 Jan 2025 13:01:09 +0100 Message-ID: <20250124120112.56678-4-roger.pau@citrix.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20250124120112.56678-1-roger.pau@citrix.com> References: <20250124120112.56678-1-roger.pau@citrix.com> MIME-Version: 1.0 From: Teddy Astie As CX16 support is mandatory for IOMMU usage, the checks for CX16 in the DMA remapping code are stale. Remove them together with the associated code introduced in case CX16 was not available. Suggested-by: Andrew Cooper Signed-off-by: Teddy Astie Signed-off-by: Roger Pau Monné --- xen/drivers/passthrough/amd/iommu_map.c | 42 +++++---------- xen/drivers/passthrough/vtd/iommu.c | 71 ++++++------------------- 2 files changed, 31 insertions(+), 82 deletions(-) diff --git a/xen/drivers/passthrough/amd/iommu_map.c b/xen/drivers/passthrough/amd/iommu_map.c index 1f0e4167566b..dde393645a62 100644 --- a/xen/drivers/passthrough/amd/iommu_map.c +++ b/xen/drivers/passthrough/amd/iommu_map.c @@ -167,15 +167,14 @@ int amd_iommu_set_root_page_table(struct amd_iommu_dte *dte, { bool valid = flags & SET_ROOT_VALID; - if ( dte->v && dte->tv && - (cpu_has_cx16 || (flags & SET_ROOT_WITH_UNITY_MAP)) ) + if ( dte->v && dte->tv ) { union { struct amd_iommu_dte dte; uint64_t raw64[4]; __uint128_t raw128[2]; } ldte = { .dte = *dte }; - __uint128_t old = ldte.raw128[0]; + __uint128_t res, old = ldte.raw128[0]; int ret = 0; ldte.dte.domain_id = domain_id; @@ -185,33 +184,20 @@ int amd_iommu_set_root_page_table(struct amd_iommu_dte *dte, ldte.dte.paging_mode = paging_mode; ldte.dte.v = valid; - if ( cpu_has_cx16 ) - { - __uint128_t res = cmpxchg16b(dte, &old, &ldte.raw128[0]); + res = cmpxchg16b(dte, &old, &ldte.raw128[0]); - /* - * Hardware does not update the DTE behind our backs, so the - * return value should match "old". - */ - if ( res != old ) - { - printk(XENLOG_ERR - "Dom%d: unexpected DTE %016lx_%016lx (expected %016lx_%016lx)\n", - domain_id, - (uint64_t)(res >> 64), (uint64_t)res, - (uint64_t)(old >> 64), (uint64_t)old); - ret = -EILSEQ; - } - } - else /* Best effort, updating domain_id last. */ + /* + * Hardware does not update the DTE behind our backs, so the + * return value should match "old". + */ + if ( res != old ) { - uint64_t *ptr = (void *)dte; - - write_atomic(ptr + 0, ldte.raw64[0]); - /* No barrier should be needed between these two. */ - write_atomic(ptr + 1, ldte.raw64[1]); - - ret = 1; + printk(XENLOG_ERR + "Dom%d: unexpected DTE %016lx_%016lx (expected %016lx_%016lx)\n", + domain_id, + (uint64_t)(res >> 64), (uint64_t)res, + (uint64_t)(old >> 64), (uint64_t)old); + ret = -EILSEQ; } return ret; diff --git a/xen/drivers/passthrough/vtd/iommu.c b/xen/drivers/passthrough/vtd/iommu.c index 3daedc4f5593..b0963bfcf74e 100644 --- a/xen/drivers/passthrough/vtd/iommu.c +++ b/xen/drivers/passthrough/vtd/iommu.c @@ -1485,7 +1485,7 @@ int domain_context_mapping_one( { struct domain_iommu *hd = dom_iommu(domain); struct context_entry *context, *context_entries, lctxt; - __uint128_t old; + __uint128_t res, old; uint64_t maddr; uint16_t seg = iommu->drhd->segment, prev_did = 0; struct domain *prev_dom = NULL; @@ -1583,55 +1583,23 @@ int domain_context_mapping_one( ASSERT(!context_fault_disable(lctxt)); } - if ( cpu_has_cx16 ) - { - __uint128_t res = cmpxchg16b(context, &old, &lctxt.full); - - /* - * Hardware does not update the context entry behind our backs, - * so the return value should match "old". - */ - if ( res != old ) - { - if ( pdev ) - check_cleanup_domid_map(domain, pdev, iommu); - printk(XENLOG_ERR - "%pp: unexpected context entry %016lx_%016lx (expected %016lx_%016lx)\n", - &PCI_SBDF(seg, bus, devfn), - (uint64_t)(res >> 64), (uint64_t)res, - (uint64_t)(old >> 64), (uint64_t)old); - rc = -EILSEQ; - goto unlock; - } - } - else if ( !prev_dom || !(mode & MAP_WITH_RMRR) ) - { - context_clear_present(*context); - iommu_sync_cache(context, sizeof(*context)); + res = cmpxchg16b(context, &old, &lctxt.full); - write_atomic(&context->hi, lctxt.hi); - /* No barrier should be needed between these two. */ - write_atomic(&context->lo, lctxt.lo); - } - else /* Best effort, updating DID last. */ + /* + * Hardware does not update the context entry behind our backs, + * so the return value should match "old". + */ + if ( res != old ) { - /* - * By non-atomically updating the context entry's DID field last, - * during a short window in time TLB entries with the old domain ID - * but the new page tables may be inserted. This could affect I/O - * of other devices using this same (old) domain ID. Such updating - * therefore is not a problem if this was the only device associated - * with the old domain ID. Diverting I/O of any of a dying domain's - * devices to the quarantine page tables is intended anyway. - */ - if ( !(mode & (MAP_OWNER_DYING | MAP_SINGLE_DEVICE)) ) - printk(XENLOG_WARNING VTDPREFIX - " %pp: reassignment may cause %pd data corruption\n", - &PCI_SBDF(seg, bus, devfn), prev_dom); - - write_atomic(&context->lo, lctxt.lo); - /* No barrier should be needed between these two. */ - write_atomic(&context->hi, lctxt.hi); + if ( pdev ) + check_cleanup_domid_map(domain, pdev, iommu); + printk(XENLOG_ERR + "%pp: unexpected context entry %016lx_%016lx (expected %016lx_%016lx)\n", + &PCI_SBDF(seg, bus, devfn), + (uint64_t)(res >> 64), (uint64_t)res, + (uint64_t)(old >> 64), (uint64_t)old); + rc = -EILSEQ; + goto unlock; } iommu_sync_cache(context, sizeof(struct context_entry)); @@ -2702,12 +2670,7 @@ static int __init cf_check vtd_setup(void) iommu_intremap = iommu_intremap_off; #ifndef iommu_intpost - /* - * We cannot use posted interrupt if X86_FEATURE_CX16 is - * not supported, since we count on this feature to - * atomically update 16-byte IRTE in posted format. - */ - if ( !cap_intr_post(iommu->cap) || !iommu_intremap || !cpu_has_cx16 ) + if ( !cap_intr_post(iommu->cap) || !iommu_intremap ) iommu_intpost = false; #endif From patchwork Fri Jan 24 12:01:10 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Roger_Pau_Monn=C3=A9?= X-Patchwork-Id: 13949329 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D88D1C0218F for ; Fri, 24 Jan 2025 12:01:38 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.876706.1287091 (Exim 4.92) (envelope-from ) id 1tbINM-0000DZ-Ml; Fri, 24 Jan 2025 12:01:28 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 876706.1287091; Fri, 24 Jan 2025 12:01:28 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tbINM-0000Cp-Jq; Fri, 24 Jan 2025 12:01:28 +0000 Received: by outflank-mailman (input) for mailman id 876706; Fri, 24 Jan 2025 12:01:27 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tbINL-0007bK-Gs for xen-devel@lists.xenproject.org; Fri, 24 Jan 2025 12:01:27 +0000 Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com [2a00:1450:4864:20::629]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id efed0514-da4a-11ef-99a4-01e77a169b0f; Fri, 24 Jan 2025 13:01:26 +0100 (CET) Received: by mail-ej1-x629.google.com with SMTP id a640c23a62f3a-aafc9d75f8bso378375766b.2 for ; Fri, 24 Jan 2025 04:01:26 -0800 (PST) Received: from localhost ([84.78.159.3]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-ab675e12770sm119049166b.28.2025.01.24.04.01.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 24 Jan 2025 04:01:24 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: efed0514-da4a-11ef-99a4-01e77a169b0f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1737720085; x=1738324885; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ZG7JEH3gSju1VydKP45xEoqkmeQzG5Nz2nBw/4zt1JY=; b=I2sEsWPX0nNYcf4zGoeeZQpf69Qrh7UQEYyzuIJdTDkoVcXkdXoXV9kO7MoSbjb5a7 3vS63Hu/1rxCVrBsNAhJEX+6lJAR+X+FMfA1lrZF6vOiPW7slaSMnor8xsLMCZ4v3PBG WdBSGOxThvpluHYGBdqS0k100Df6IxbL/9Dkk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737720085; x=1738324885; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZG7JEH3gSju1VydKP45xEoqkmeQzG5Nz2nBw/4zt1JY=; b=GYK27V9ueMwQpMjKMG9YechbmYav7t5zlVPe7dzNuNk193GjyyfAndzMDei4v0/u7g XFIdbzxsJoxVpq/Kkx+HJZivNxn4CJUAfEeBSwoL335t/V+gmvof7Z+w8y0xge87MRrd 0evMAWyZmQ0Wkwbm3grP3y31GBlSfRxJApMlJ1O9RUnZ71yjhjJcIYv3Jz2VFB2cCzx+ K19ddlia0Y6xroOJw12eqz6UBogKRXsW5aAO2/aVJPYYZOYTi9OzBtSzOvePsKMxFTjQ dfneW4HxW1exv6glkR/y1eo2te3HaXVqigJuhDD1OA0AhiDxkLXIeCML/GnTVnK2Qezp 9ezg== X-Gm-Message-State: AOJu0YyCl+lYyKYf+bDhzSsFh6vj3ECadkMyOnaSr1XpS+yKBb+ZjEEQ xd5hC8naNXKHDcA0C9Xd+tUI8/2U6QWhpI1DJpKF88wDb/T1DS/+isWO1Cz/qVw9iJHl1frB1VU C X-Gm-Gg: ASbGnctehdgaCPpvBKCYoVXM7ylH6SbrP7ZVhNtOtDB6FS2YbYQU+nh115YeWQCZGrF cIjkJs+eeOEWb8r3G8SYGb/m3ZSNSvcXEdza116sAJ/BD1Ou3ZkJAhv4Dd8oVlXAVWYfctrt69/ v+/k5tI8QDjbuMdowu8qZiwdniUUdaMBA42aGS5Y9WLFEU3ikuc8dm6Pi4hCByb5bdGjqSs3RWN pM9r+Yva9ZqY1ql2L4pvbnlz+GGO6HfmSHWaRzPhX0SxxmJuP3Eo63YbIyBTSjL4FWsvJZyir4z 5m4WnwtqGbSfACA= X-Google-Smtp-Source: AGHT+IGwobbZw6n/IuGjK/JSvPcW4EXd/vH/ht5HiCBUBeIfVlLQYJB4jj+811xsVLL8IgmHXb0e+g== X-Received: by 2002:a17:907:9408:b0:aa6:a87e:f2e1 with SMTP id a640c23a62f3a-ab38b4b96d0mr2693767166b.56.1737720085229; Fri, 24 Jan 2025 04:01:25 -0800 (PST) From: Roger Pau Monne To: xen-devel@lists.xenproject.org Cc: Teddy Astie , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Subject: [PATCH v2 4/5] iommu/vtd: cleanup MAP_SINGLE_DEVICE and related code Date: Fri, 24 Jan 2025 13:01:10 +0100 Message-ID: <20250124120112.56678-5-roger.pau@citrix.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20250124120112.56678-1-roger.pau@citrix.com> References: <20250124120112.56678-1-roger.pau@citrix.com> MIME-Version: 1.0 From: Teddy Astie This flag was only used in case cx16 is not available, as those code paths no longer exist, this flag now does basically nothing. Signed-off-by: Teddy Astie Signed-off-by: Roger Pau Monné --- xen/drivers/passthrough/vtd/iommu.c | 12 +++--------- xen/drivers/passthrough/vtd/vtd.h | 5 ++--- 2 files changed, 5 insertions(+), 12 deletions(-) diff --git a/xen/drivers/passthrough/vtd/iommu.c b/xen/drivers/passthrough/vtd/iommu.c index b0963bfcf74e..9d7a9977a6a6 100644 --- a/xen/drivers/passthrough/vtd/iommu.c +++ b/xen/drivers/passthrough/vtd/iommu.c @@ -1695,15 +1695,9 @@ static int domain_context_mapping(struct domain *domain, u8 devfn, break; } - if ( domain != pdev->domain && pdev->domain != dom_io ) - { - if ( pdev->domain->is_dying ) - mode |= MAP_OWNER_DYING; - else if ( drhd && - !any_pdev_behind_iommu(pdev->domain, pdev, drhd->iommu) && - !pdev->phantom_stride ) - mode |= MAP_SINGLE_DEVICE; - } + if ( domain != pdev->domain && pdev->domain != dom_io && + pdev->domain->is_dying ) + mode |= MAP_OWNER_DYING; switch ( pdev->type ) { diff --git a/xen/drivers/passthrough/vtd/vtd.h b/xen/drivers/passthrough/vtd/vtd.h index 8aeff8c1f287..b95124517bd3 100644 --- a/xen/drivers/passthrough/vtd/vtd.h +++ b/xen/drivers/passthrough/vtd/vtd.h @@ -28,9 +28,8 @@ */ #define MAP_WITH_RMRR (1u << 0) #define MAP_OWNER_DYING (1u << 1) -#define MAP_SINGLE_DEVICE (1u << 2) -#define MAP_ERROR_RECOVERY (1u << 3) -#define UNMAP_ME_PHANTOM_FUNC (1u << 4) +#define MAP_ERROR_RECOVERY (1u << 2) +#define UNMAP_ME_PHANTOM_FUNC (1u << 3) /* Allow for both IOAPIC and IOSAPIC. */ #define IO_xAPIC_route_entry IO_APIC_route_entry From patchwork Fri Jan 24 12:01:11 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Roger_Pau_Monn=C3=A9?= X-Patchwork-Id: 13949324 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EC424C0218C for ; Fri, 24 Jan 2025 12:01:37 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.876708.1287102 (Exim 4.92) (envelope-from ) id 1tbINO-0000Uz-0a; Fri, 24 Jan 2025 12:01:30 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 876708.1287102; Fri, 24 Jan 2025 12:01:29 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tbINN-0000UY-So; Fri, 24 Jan 2025 12:01:29 +0000 Received: by outflank-mailman (input) for mailman id 876708; Fri, 24 Jan 2025 12:01:29 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tbINN-0007hy-5l for xen-devel@lists.xenproject.org; Fri, 24 Jan 2025 12:01:29 +0000 Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com [2a00:1450:4864:20::62a]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id f14bc8ef-da4a-11ef-a0e5-8be0dac302b0; Fri, 24 Jan 2025 13:01:28 +0100 (CET) Received: by mail-ej1-x62a.google.com with SMTP id a640c23a62f3a-ab2e308a99bso380588366b.1 for ; Fri, 24 Jan 2025 04:01:28 -0800 (PST) Received: from localhost ([84.78.159.3]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-ab6760ab21esm119803566b.91.2025.01.24.04.01.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 24 Jan 2025 04:01:27 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: f14bc8ef-da4a-11ef-a0e5-8be0dac302b0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1737720087; x=1738324887; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=fmWyXMHgHQ0P1pkeRIOSC5BWPtegTo9AV2b/sAitCDA=; b=Qi83Z7QgRg3Smm2WqWxfDEU/tHxOlKlav+sjDWXuyZmMorgcuBfXVglzujK/dUVFMO OajNmYMGiX2zcWDNtkMvrTNj46rDIhORR0V0xmbNxhmtwmH0ZaqDE8ZcZfOBF5qjKpfC 4zeBNj87hl/YzPfcnGjRIhkMzoR6PHTUhbZes= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737720087; x=1738324887; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fmWyXMHgHQ0P1pkeRIOSC5BWPtegTo9AV2b/sAitCDA=; b=RmqJrKXXRjwbPpOulMYi+wozZkzSsH3q8Q3ZMhe1MBlFKONo1UppATIhsFmk0PVugF +7pktf1TFyw8x41GTaxDULnYWIOsh57aOlxT9CucqrEOslsX2+OdOOLYSO4Vj1swBIf1 J6557kfCr8NJ/uC/3hDXSLlVjLKGay8wIu6hDtAo2t8Nfn+oh8N+bTuFUOXRP4dUUk2n 3IUK2c9B0KTi5LxrG5wUna3+gW3ps7IyNxTqVXohzMBa0/v/3GclBXhuJeDxlqnGwAB7 VONFcYmhG6wR1OfuELRWBYBhQnhMKhikGvboxzqs5Cz8Y+lt2rFGuxGOLci/fyAMOPns CHoA== X-Gm-Message-State: AOJu0Yyp6T9AtcHCgXeImh7Cw/cLTjAexEgbCIam5FQc1AhBs11ZYm+h tUvdkpjA01LxbkvBDZoCyKbzf40FEMPg09XPI4Lrm+MapzluY1p0ihHEDDycvTMMqw+gzS+fUQY d X-Gm-Gg: ASbGncvn1ekXpGRrmKtlcjC1Vbex/rtxuC97vtPJgiMP79skCk/AzYNt5dktVvYTzoI gVmTH7BAp99EL01KI0ZV0XYbv50s9NkEYqYHc/YVtNLQVHz4TMQlW+A9knaPLmEYdoJqlmg/6yy q7MN4tjaTJdEsFqR2gGYxoao6SajnOeYCN7oaZRU3Kss6nSP2sbv456UdVrcoq7M5gXy6RabKkf X5HgQAFMcKTyu4ARW3qCur2HbbuCsSmUU+oYt0yh9NZKnSvET4JCusMzA4KIw93I5s6MYChD99L GHQFZm+sLw5cy88= X-Google-Smtp-Source: AGHT+IGBGKgfieeigYW9s9OK8DaGTcbZVqcEXPQFnIqnHZAlx3xjKRNljOF2ZKSB/sGXA6FfIZrH4Q== X-Received: by 2002:a17:907:7e96:b0:aa4:cd1e:c91b with SMTP id a640c23a62f3a-ab6745b8c27mr272502266b.7.1737720087493; Fri, 24 Jan 2025 04:01:27 -0800 (PST) From: Roger Pau Monne To: xen-devel@lists.xenproject.org Cc: Roger Pau Monne , Jan Beulich , Andrew Cooper Subject: [PATCH v2 5/5] iommu/amd: atomically update IRTE Date: Fri, 24 Jan 2025 13:01:11 +0100 Message-ID: <20250124120112.56678-6-roger.pau@citrix.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20250124120112.56678-1-roger.pau@citrix.com> References: <20250124120112.56678-1-roger.pau@citrix.com> MIME-Version: 1.0 Either when using a 32bit Interrupt Remapping Entry or a 128bit one update the entry atomically, by using cmpxchg unconditionally as IOMMU depends on it. No longer disable the entry by setting RemapEn = 0 ahead of updating it. As a consequence of not toggling RemapEn ahead of the update the Interrupt Remapping Table needs to be flushed after the entry update. This avoids a window where the IRTE has RemapEn = 0, which can lead to IO_PAGE_FAULT if the underlying interrupt source is not masked. There's no guidance in AMD-Vi specification about how IRTE update should be performed as opposed to DTE updating which has specific guidance. However DTE updating claims that reads will always be at least 128bits in size, and hence for the purposes here assume that reads and caching of the IRTE entries in either 32 or 128 bit format will be done atomically from the IOMMU. Note that as part of introducing a new raw128 field in the IRTE struct, the current raw field is renamed to raw64 to explicitly contain the size in the field name. Signed-off-by: Roger Pau Monné --- Changes since v1: - cmpxchg is now always available if the IOMMU is enabled. --- xen/drivers/passthrough/amd/iommu_intr.c | 75 ++++++++++-------------- 1 file changed, 32 insertions(+), 43 deletions(-) diff --git a/xen/drivers/passthrough/amd/iommu_intr.c b/xen/drivers/passthrough/amd/iommu_intr.c index f07fd9e3d970..c0273059cb1d 100644 --- a/xen/drivers/passthrough/amd/iommu_intr.c +++ b/xen/drivers/passthrough/amd/iommu_intr.c @@ -39,7 +39,8 @@ union irte32 { }; union irte128 { - uint64_t raw[2]; + uint64_t raw64[2]; + __uint128_t raw128; struct { bool remap_en:1; bool sup_io_pf:1; @@ -187,7 +188,7 @@ static void free_intremap_entry(const struct amd_iommu *iommu, if ( iommu->ctrl.ga_en ) { - ACCESS_ONCE(entry.ptr128->raw[0]) = 0; + ACCESS_ONCE(entry.ptr128->raw64[0]) = 0; /* * Low half (containing RemapEn) needs to be cleared first. Note that * strictly speaking smp_wmb() isn't enough, as conceptually it expands @@ -197,7 +198,7 @@ static void free_intremap_entry(const struct amd_iommu *iommu, * variant will do. */ smp_wmb(); - entry.ptr128->raw[1] = 0; + entry.ptr128->raw64[1] = 0; } else ACCESS_ONCE(entry.ptr32->raw) = 0; @@ -212,7 +213,7 @@ static void update_intremap_entry(const struct amd_iommu *iommu, { if ( iommu->ctrl.ga_en ) { - union irte128 irte = { + const union irte128 irte = { .full = { .remap_en = true, .int_type = int_type, @@ -222,19 +223,26 @@ static void update_intremap_entry(const struct amd_iommu *iommu, .vector = vector, }, }; + __uint128_t old = entry.ptr128->raw128; + __uint128_t res = cmpxchg16b(&entry.ptr128->raw128, &old, + &irte.raw128); - ASSERT(!entry.ptr128->full.remap_en); - entry.ptr128->raw[1] = irte.raw[1]; /* - * High half needs to be set before low one (containing RemapEn). See - * comment in free_intremap_entry() regarding the choice of barrier. + * Hardware does not update the IRTE behind our backs, so the return + * value should match "old". */ - smp_wmb(); - ACCESS_ONCE(entry.ptr128->raw[0]) = irte.raw[0]; + if ( res != old ) + { + printk(XENLOG_ERR + "unexpected IRTE %016lx_%016lx (expected %016lx_%016lx)\n", + (uint64_t)(res >> 64), (uint64_t)res, + (uint64_t)(old >> 64), (uint64_t)old); + ASSERT_UNREACHABLE(); + } } else { - union irte32 irte = { + const union irte32 irte = { .flds = { .remap_en = true, .int_type = int_type, @@ -299,21 +307,13 @@ static int update_intremap_entry_from_ioapic( entry = get_intremap_entry(iommu, req_id, offset); - /* The RemapEn fields match for all formats. */ - while ( iommu->enabled && entry.ptr32->flds.remap_en ) - { - entry.ptr32->flds.remap_en = false; - spin_unlock(lock); - - amd_iommu_flush_intremap(iommu, req_id); - - spin_lock(lock); - } - update_intremap_entry(iommu, entry, vector, delivery_mode, dest_mode, dest); spin_unlock_irqrestore(lock, flags); + if ( !fresh ) + amd_iommu_flush_intremap(iommu, req_id); + set_rte_index(rte, offset); return 0; @@ -322,7 +322,7 @@ static int update_intremap_entry_from_ioapic( void cf_check amd_iommu_ioapic_update_ire( unsigned int apic, unsigned int pin, uint64_t rte) { - struct IO_APIC_route_entry old_rte, new_rte; + struct IO_APIC_route_entry new_rte; int seg, bdf, rc; struct amd_iommu *iommu; unsigned int idx; @@ -346,14 +346,6 @@ void cf_check amd_iommu_ioapic_update_ire( return; } - old_rte = __ioapic_read_entry(apic, pin, true); - /* mask the interrupt while we change the intremap table */ - if ( !old_rte.mask ) - { - old_rte.mask = 1; - __ioapic_write_entry(apic, pin, true, old_rte); - } - /* Update interrupt remapping entry */ rc = update_intremap_entry_from_ioapic( bdf, iommu, &new_rte, @@ -425,6 +417,7 @@ static int update_intremap_entry_from_msi_msg( uint8_t delivery_mode, vector, dest_mode; spinlock_t *lock; unsigned int dest, offset, i; + bool fresh = false; req_id = get_dma_requestor_id(iommu->seg, bdf); alias_id = get_intremap_requestor_id(iommu->seg, bdf); @@ -468,26 +461,21 @@ static int update_intremap_entry_from_msi_msg( return -ENOSPC; } *remap_index = offset; + fresh = true; } entry = get_intremap_entry(iommu, req_id, offset); - /* The RemapEn fields match for all formats. */ - while ( iommu->enabled && entry.ptr32->flds.remap_en ) - { - entry.ptr32->flds.remap_en = false; - spin_unlock(lock); + update_intremap_entry(iommu, entry, vector, delivery_mode, dest_mode, dest); + spin_unlock_irqrestore(lock, flags); + if ( !fresh ) + { amd_iommu_flush_intremap(iommu, req_id); if ( alias_id != req_id ) amd_iommu_flush_intremap(iommu, alias_id); - - spin_lock(lock); } - update_intremap_entry(iommu, entry, vector, delivery_mode, dest_mode, dest); - spin_unlock_irqrestore(lock, flags); - *data = (msg->data & ~(INTREMAP_MAX_ENTRIES - 1)) | offset; /* @@ -735,7 +723,7 @@ static void dump_intremap_table(const struct amd_iommu *iommu, for ( count = 0; count < nr; count++ ) { if ( iommu->ctrl.ga_en - ? !tbl.ptr128[count].raw[0] && !tbl.ptr128[count].raw[1] + ? !tbl.ptr128[count].raw64[0] && !tbl.ptr128[count].raw64[1] : !tbl.ptr32[count].raw ) continue; @@ -748,7 +736,8 @@ static void dump_intremap_table(const struct amd_iommu *iommu, if ( iommu->ctrl.ga_en ) printk(" IRTE[%03x] %016lx_%016lx\n", - count, tbl.ptr128[count].raw[1], tbl.ptr128[count].raw[0]); + count, tbl.ptr128[count].raw64[1], + tbl.ptr128[count].raw64[0]); else printk(" IRTE[%03x] %08x\n", count, tbl.ptr32[count].raw); }