From patchwork Fri Nov 29 11:28:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Roger_Pau_Monn=C3=A9?= X-Patchwork-Id: 11267023 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 449B417E0 for ; Fri, 29 Nov 2019 11:31:03 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2199120833 for ; Fri, 29 Nov 2019 11:31:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="FXKvS/em" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2199120833 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iaeSx-0006qN-Uf; Fri, 29 Nov 2019 11:29:39 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iaeSw-0006qG-Rk for xen-devel@lists.xenproject.org; Fri, 29 Nov 2019 11:29:38 +0000 X-Inumbo-ID: 8615b07e-129b-11ea-a3e3-12813bfff9fa Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 8615b07e-129b-11ea-a3e3-12813bfff9fa; Fri, 29 Nov 2019 11:29:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1575026978; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=XUYFi2R2qXQsUBynqEbA0UN2CXp44zL7+ww6QKkSEyY=; b=FXKvS/emJXthtKGMyAGAndxvP7PM6zmozaq0d75gt6yeFPMlFcc0mj9q vipUCW4SaRAMLM3CdTQkah5reDXm1dQmMd9dod0btmExiUkoUuB32nRh1 bqFnZ2D6yZ/Y1XkHFh1qvvU8R6zMwFLIjVBOhe9l1WjmbopU1PWw9ETtb 4=; Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=roger.pau@citrix.com; spf=Pass smtp.mailfrom=roger.pau@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: None (esa1.hc3370-68.iphmx.com: no sender authenticity information available from domain of roger.pau@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa1.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa1.hc3370-68.iphmx.com: domain of roger.pau@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa1.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa1.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa1.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: zrdiOabcACzS85etsYCph7G63eKZAhY37Mh05bDtkSNhvDVpfeODCdiL5PFQMGln5/Dj/2toBU RmKlGtY56Ax9TN6+oLIm6JPTq5ou1WekGWi8/TnB7lB5xl2fxfndSjsnvzvMlog+Yh4fJavzI9 Zaya4HabWD39jAJGoAT+oOunvHFygja3y8DVHI7G1eZe/JIvKlPmOmrzA/8/MXEaUPxNHt5lrK DR76uzA4jYEFlHI23QAG10Ux1oIKFWcHqhI83Ubhrf32M6H/pB7+rw125RECrmgRLfQQVZqN7H XEM= X-SBRS: 2.7 X-MesageID: 9108564 X-Ironport-Server: esa1.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.69,257,1571716800"; d="scan'208";a="9108564" From: Roger Pau Monne To: Date: Fri, 29 Nov 2019 12:28:48 +0100 Message-ID: <20191129112851.19273-2-roger.pau@citrix.com> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191129112851.19273-1-roger.pau@citrix.com> References: <20191129112851.19273-1-roger.pau@citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [PATCH v2 1/4] x86/ioapic: only use dest32 with x2apic and interrupt remapping enabled X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , Jan Beulich , Roger Pau Monne Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" The IO-APIC code assumes that x2apic being enabled also implies interrupt remapping being enabled, and hence will use the 32bit destination field in the IO-APIC entry. This is safe now, but there's no reason to not enable x2APIC even without interrupt remapping, and hence the IO-APIC code needs to use the 32 bit destination field only when both interrupt remapping and x2APIC are enabled. Signed-off-by: Roger Pau Monné --- Changes since v1: - Fix set_ioapic_affinity_irq. --- xen/arch/x86/io_apic.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/xen/arch/x86/io_apic.c b/xen/arch/x86/io_apic.c index 97cb2d154a..be0b085513 100644 --- a/xen/arch/x86/io_apic.c +++ b/xen/arch/x86/io_apic.c @@ -562,7 +562,7 @@ set_ioapic_affinity_irq(struct irq_desc *desc, const cpumask_t *mask) dest = set_desc_affinity(desc, mask); if (dest != BAD_APICID) { - if ( !x2apic_enabled ) + if ( !iommu_intremap ) dest = SET_APIC_LOGICAL_ID(dest); entry = irq_2_pin + irq; for (;;) { @@ -964,7 +964,7 @@ static hw_irq_controller ioapic_edge_type; #define IOAPIC_LEVEL 1 #define SET_DEST(ent, mode, val) do { \ - if (x2apic_enabled) \ + if (x2apic_enabled && iommu_intremap) \ (ent).dest.dest32 = (val); \ else \ (ent).dest.mode.mode##_dest = (val); \ @@ -1194,14 +1194,14 @@ static void /*__init*/ __print_IO_APIC(bool boot) printk(KERN_DEBUG ".... IRQ redirection table:\n"); printk(KERN_DEBUG " NR %s Msk Trg IRR Pol Stat DstM DelM Vec\n", - x2apic_enabled ? " DestID" : "Dst"); + (x2apic_enabled && iommu_intremap) ? " DestID" : "Dst"); for (i = 0; i <= reg_01.bits.entries; i++) { struct IO_APIC_route_entry entry; entry = ioapic_read_entry(apic, i, 0); - if ( x2apic_enabled ) + if ( x2apic_enabled && iommu_intremap ) printk(KERN_DEBUG " %02x %08x", i, entry.dest.dest32); else printk(KERN_DEBUG " %02x %02x ", i, @@ -2504,9 +2504,9 @@ void dump_ioapic_irq_info(void) rte.dest_mode ? 'L' : 'P', rte.delivery_status, rte.polarity, rte.irr, rte.trigger ? 'L' : 'E', rte.mask, - x2apic_enabled ? 8 : 2, - x2apic_enabled ? rte.dest.dest32 - : rte.dest.logical.logical_dest); + (x2apic_enabled && iommu_intremap) ? 8 : 2, + (x2apic_enabled && iommu_intremap) ? + rte.dest.dest32 : rte.dest.logical.logical_dest); if ( entry->next == 0 ) break; From patchwork Fri Nov 29 11:28:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Roger_Pau_Monn=C3=A9?= X-Patchwork-Id: 11267019 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EB9A6921 for ; Fri, 29 Nov 2019 11:31:01 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C7DAA20833 for ; Fri, 29 Nov 2019 11:31:01 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="bhwbMKrT" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C7DAA20833 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iaeT3-0006qp-Ep; Fri, 29 Nov 2019 11:29:45 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iaeT1-0006qe-MX for xen-devel@lists.xenproject.org; Fri, 29 Nov 2019 11:29:43 +0000 X-Inumbo-ID: 86dca29c-129b-11ea-a3e3-12813bfff9fa Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 86dca29c-129b-11ea-a3e3-12813bfff9fa; Fri, 29 Nov 2019 11:29:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1575026978; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=sQ0XLdjh/Fs15+1JwkEDWJDg2qfluPMHMfcwwCyy4mY=; b=bhwbMKrT4iVjKbjV0xXd29lolkUgWM+Tnrswl1LTpR4IJBLKn3BpksEu CPvu/c0CYYRC0a6ghIXcmGLygNE1G7K9439hleycWSahxx4HsWoeNUfil iSsxhz74iBlt2HWSeRJtqgLbUpmJgBlSJ5gPJEDciPEMruIczSjbEzGWz I=; Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=roger.pau@citrix.com; spf=Pass smtp.mailfrom=roger.pau@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: None (esa5.hc3370-68.iphmx.com: no sender authenticity information available from domain of roger.pau@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa5.hc3370-68.iphmx.com: domain of roger.pau@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa5.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: ZMGsBGbObLOagAGMhhjG7px077dS92xMJBYi92JzuOK4OptHn3Q8lwi6VPy7ZXOoIak6EjNdFP gna21/8r9t/prTk3E15+UTMJ3DD3hZAFnKn6tY6vAGZZyoorH6+xSjiXWqmF2F8aWHsMKN0HNx xj/Hc0QQFLJcJfBvh0EdBjhSKmiIFM1lz3jaL3n8dTOp877qkNIFJBWhiz3jooXKoBbzI4VU5h AefPx81BElZ2Qnc8PRbHvE63jMXXit79rhUTSIZYVogb8OVWGbk/Kia+uuUCyfG3PLH3A2pZLm epQ= X-SBRS: 2.7 X-MesageID: 9338026 X-Ironport-Server: esa5.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.69,257,1571716800"; d="scan'208";a="9338026" From: Roger Pau Monne To: Date: Fri, 29 Nov 2019 12:28:49 +0100 Message-ID: <20191129112851.19273-3-roger.pau@citrix.com> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191129112851.19273-1-roger.pau@citrix.com> References: <20191129112851.19273-1-roger.pau@citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [PATCH v2 2/4] x86/apic: force phys mode if interrupt remapping is disabled X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , Jan Beulich , Roger Pau Monne Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Cluster mode can only be used with interrupt remapping support, since the top 16bits of the APIC ID are filled with the cluster ID, and hence on systems where the physical ID is still smaller than 255 the cluster ID is not. Force x2APIC to use physical mode if there's no interrupt remapping support. Note that this requires a further patch in order to enable x2APIC without interrupt remapping support. Signed-off-by: Roger Pau Monné --- Changes since v1: - New in this version. --- xen/arch/x86/genapic/x2apic.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/xen/arch/x86/genapic/x2apic.c b/xen/arch/x86/genapic/x2apic.c index d5a17f10d5..7e32ee22ff 100644 --- a/xen/arch/x86/genapic/x2apic.c +++ b/xen/arch/x86/genapic/x2apic.c @@ -228,6 +228,14 @@ const struct genapic *__init apic_x2apic_probe(void) if ( x2apic_phys < 0 ) x2apic_phys = !!(acpi_gbl_FADT.flags & ACPI_FADT_APIC_PHYSICAL); + if ( !x2apic_phys && !iommu_intremap ) + /* + * Force physical mode if there's no interrupt remapping support: the + * ID in clustered mode requires a 32 bit destination field due to the + * usage of the high 16 bits to store the cluster ID. + */ + x2apic_phys = true; + if ( x2apic_phys ) return &apic_x2apic_phys; From patchwork Fri Nov 29 11:28:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Roger_Pau_Monn=C3=A9?= X-Patchwork-Id: 11267021 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B360B930 for ; Fri, 29 Nov 2019 11:31:02 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8FF3020833 for ; Fri, 29 Nov 2019 11:31:02 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="WXhEDyoE" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8FF3020833 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iaeT0-0006qY-6h; Fri, 29 Nov 2019 11:29:42 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iaeSy-0006qT-RP for xen-devel@lists.xenproject.org; Fri, 29 Nov 2019 11:29:40 +0000 X-Inumbo-ID: 87c2042c-129b-11ea-b155-bc764e2007e4 Received: from esa2.hc3370-68.iphmx.com (unknown [216.71.145.153]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 87c2042c-129b-11ea-b155-bc764e2007e4; Fri, 29 Nov 2019 11:29:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1575026981; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=M6WVb9qVoCWSq+IksgaLNw4MVZZQraMScTx5Z7NZA6E=; b=WXhEDyoEFzoaKW9wPKo5DbkBw7fPaaYyVwlX38Sc30fKleBSx+S7oR3U NZ2hvZtLmYl+09QOtGh3EFcOewazKilBjcyfG8PRaZlrFgW9rylL6fVNA OHOgROQ8hL1bn/UJcav5SXIewPhzDgK7V96K4besqDNFFMqK80mtQM7el U=; Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=roger.pau@citrix.com; spf=Pass smtp.mailfrom=roger.pau@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: None (esa2.hc3370-68.iphmx.com: no sender authenticity information available from domain of roger.pau@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa2.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa2.hc3370-68.iphmx.com: domain of roger.pau@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa2.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa2.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa2.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: doxhNnF1hksrFHfv1jmCQBG25XC1DX4KXwADCDy2OtipLAWGnrSfDw4D5ZbSvOFa0fXl8rF+/3 as2+F+NqOxhLxLc6iINiW3LpXeZUI9i0AGW2nzZOTg6oGGXyTJrRd7l/wqdtJN2MKKsoeKUciZ wDJvcHsSIeJzOQh6Dtgr5OMe7XMUGW31e+AUma2BpM1KfvkLwIZowxCM6V2IjwO96k2f2vXym7 /VHGW642PV9RWZ+BD6dY2/Wc4XiG9iQ2T6rjz1M76Wm+uXBwMrR7EKk2ZN/lhRFWkxNPyoGnIe ZMs= X-SBRS: 2.7 X-MesageID: 8987262 X-Ironport-Server: esa2.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.69,257,1571716800"; d="scan'208";a="8987262" From: Roger Pau Monne To: Date: Fri, 29 Nov 2019 12:28:50 +0100 Message-ID: <20191129112851.19273-4-roger.pau@citrix.com> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191129112851.19273-1-roger.pau@citrix.com> References: <20191129112851.19273-1-roger.pau@citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [PATCH v2 3/4] x86/smp: check APIC ID on AP bringup X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , Jan Beulich , Roger Pau Monne Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Check that the processor to be woken up APIC ID is addressable in the current APIC mode. Note that in practice systems with APIC IDs > 255 should already have x2APIC enabled by the firmware, and hence this is mostly a safety belt. Signed-off-by: Roger Pau Monné --- xen/arch/x86/smpboot.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c index fa691b6ba0..484d344c44 100644 --- a/xen/arch/x86/smpboot.c +++ b/xen/arch/x86/smpboot.c @@ -1317,6 +1317,13 @@ int __cpu_up(unsigned int cpu) if ( (apicid = x86_cpu_to_apicid[cpu]) == BAD_APICID ) return -ENODEV; + if ( (!x2apic_enabled || !iommu_intremap) && (apicid >> 8) ) + { + printk("Processor with APIC ID %u cannot be onlined in xAPIC mode " + "or without interrupt remapping\n", apicid); + return -EINVAL; + } + if ( (ret = do_boot_cpu(apicid, cpu)) != 0 ) return ret; From patchwork Fri Nov 29 11:28:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Roger_Pau_Monn=C3=A9?= X-Patchwork-Id: 11267025 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7B228921 for ; Fri, 29 Nov 2019 11:31:03 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5799D20833 for ; Fri, 29 Nov 2019 11:31:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="ZqrJrvMW" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5799D20833 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iaeTC-0006rz-1I; Fri, 29 Nov 2019 11:29:54 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iaeTB-0006ro-M8 for xen-devel@lists.xenproject.org; Fri, 29 Nov 2019 11:29:53 +0000 X-Inumbo-ID: 88b44b1a-129b-11ea-a3e3-12813bfff9fa Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 88b44b1a-129b-11ea-a3e3-12813bfff9fa; Fri, 29 Nov 2019 11:29:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1575026981; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=jsBIuccbgrAsEdfwh7i25IjVofpn1BZsmhIrRdMfrQE=; b=ZqrJrvMWwNrNGqtPbLEBLuCMcROzlcWXFupodHlPtETEyrR48r7sYZuc 66Q8NgSgtbtECxmlQ2omqr+v8XWaka/PGVIPdZm10iFQBQDXFT3WiHwHv wVcXpmmzPidaX02ykMxtS+67BH3TmLGZq46txt3Hw+H1jN2rk3nibWqzy s=; Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=roger.pau@citrix.com; spf=Pass smtp.mailfrom=roger.pau@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: None (esa5.hc3370-68.iphmx.com: no sender authenticity information available from domain of roger.pau@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa5.hc3370-68.iphmx.com: domain of roger.pau@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa5.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: AYWYG/PwhFQ1bHMEllGoKRkw3cG+jLqx9DjapJQBra9xchwBJERXk6tqGvaxxyV84YcjKLCQUV kvE1TFqx4ymw4Cfd/cNRrBRPDge7t3iyqEbo/4hq/ZzPIb+ihqvUxCV4rr3taJOZF3b47oRHI2 BP5xjwZcRhx+6sx+AFIBvfaBghp64MbMSXmYqhtPMWmO3wm0ysgnEUbSmsF8Y+/jVfy4IYZIpK cuufBElraTy7VFYNkZh4v+5ogXdaYcBVNx13D9o56+7dcUjHz5uDZteN78pRTw4WyFPq0EVvL2 S6M= X-SBRS: 2.7 X-MesageID: 9338027 X-Ironport-Server: esa5.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.69,257,1571716800"; d="scan'208";a="9338027" From: Roger Pau Monne To: Date: Fri, 29 Nov 2019 12:28:51 +0100 Message-ID: <20191129112851.19273-5-roger.pau@citrix.com> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191129112851.19273-1-roger.pau@citrix.com> References: <20191129112851.19273-1-roger.pau@citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [PATCH v2 4/4] x86/apic: allow enabling x2APIC mode regardless of interrupt remapping X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , Jan Beulich , Roger Pau Monne Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" x2APIC mode doesn't mandate interrupt remapping, and hence can be enabled independently. This patch enables x2APIC when available, regardless of whether there's interrupt remapping support. This is beneficial specially when running on virtualized environments, since it reduces the amount of vmexits. For example when sending an IPI in xAPIC mode Xen performs at least 3 different accesses to the APIC MMIO region, while when using x2APIC mode a single wrmsr is used. The following numbers are from a lock profiling of a Xen PV shim running a Linux PV kernel with 32 vCPUs and xAPIC mode: (XEN) Global lock flush_lock: addr=ffff82d0804af1c0, lockval=03190319, not locked (XEN) lock:656153(892606463454), block:602183(9495067321843) Average lock time: 1360363ns Average block time: 15767743ns While the following are from the same configuration but with the shim using x2APIC mode: (XEN) Global lock flush_lock: addr=ffff82d0804b01c0, lockval=1adb1adb, not locked (XEN) lock:1841883(1375128998543), block:1658716(10193054890781) Average lock time: 746588ns Average block time: 6145147ns Enabling x2APIC has halved the average lock time, thus reducing contention. Signed-off-by: Roger Pau Monné --- Changes since v1: - Fix error paths of iommu_enable_x2apic call in x2apic_bsp_setup. --- NB: should enabling x2APIC without interrupt remapping be limited to running on virtualized environments? The bigger performance benefit is indeed achieved when using x2APIC on virt environments, but I also don't see why we wouldn't want to try using it everywhere where it's supported. --- xen/arch/x86/apic.c | 89 +++++++++++++++++++++------------------------ 1 file changed, 42 insertions(+), 47 deletions(-) diff --git a/xen/arch/x86/apic.c b/xen/arch/x86/apic.c index a8ee18636f..eaf8924585 100644 --- a/xen/arch/x86/apic.c +++ b/xen/arch/x86/apic.c @@ -492,7 +492,8 @@ static void __enable_x2apic(void) static void resume_x2apic(void) { - iommu_enable_x2apic(); + if ( iommu_supports_x2apic() ) + iommu_enable_x2apic(); __enable_x2apic(); } @@ -695,7 +696,8 @@ int lapic_suspend(void) local_irq_save(flags); disable_local_APIC(); - iommu_disable_x2apic(); + if ( iommu_supports_x2apic() ) + iommu_disable_x2apic(); local_irq_restore(flags); return 0; } @@ -875,56 +877,46 @@ void __init x2apic_bsp_setup(void) printk("x2APIC: Already enabled by BIOS: Ignoring cmdline disable.\n"); } - if ( !iommu_supports_x2apic() ) + if ( iommu_supports_x2apic() ) { - if ( !x2apic_enabled ) + if ( (ioapic_entries = alloc_ioapic_entries()) == NULL ) { - printk("Not enabling x2APIC: depends on IOMMU support\n"); - return; + printk("Allocate ioapic_entries failed\n"); + goto out; } - panic("x2APIC: already enabled by BIOS, but no IOMMU support\n"); - } - if ( (ioapic_entries = alloc_ioapic_entries()) == NULL ) - { - printk("Allocate ioapic_entries failed\n"); - goto out; - } - - if ( save_IO_APIC_setup(ioapic_entries) ) - { - printk("Saving IO-APIC state failed\n"); - goto out; - } + if ( save_IO_APIC_setup(ioapic_entries) ) + { + printk("Saving IO-APIC state failed\n"); + goto out; + } - mask_8259A(); - mask_IO_APIC_setup(ioapic_entries); + mask_8259A(); + mask_IO_APIC_setup(ioapic_entries); - switch ( iommu_enable_x2apic() ) - { - case 0: - intremap_enabled = true; - break; - case -ENXIO: /* ACPI_DMAR_X2APIC_OPT_OUT set */ - if ( !x2apic_enabled ) + switch ( iommu_enable_x2apic() ) { + case 0: + intremap_enabled = true; + break; + + case -ENXIO: /* ACPI_DMAR_X2APIC_OPT_OUT set */ + if ( x2apic_enabled ) + panic("IOMMU requests xAPIC mode, but x2APIC already enabled by firmware\n"); + printk("Not enabling x2APIC (upon firmware request)\n"); intremap_enabled = false; goto restore_out; + + default: + printk(XENLOG_ERR "Failed to enable Interrupt Remapping\n"); + intremap_enabled = false; + break; } - /* fall through */ - default: - if ( x2apic_enabled ) - panic("Interrupt remapping could not be enabled while " - "x2APIC is already enabled by BIOS\n"); - - printk(XENLOG_ERR - "Failed to enable Interrupt Remapping: Will not enable x2APIC.\n"); - intremap_enabled = false; - goto restore_out; - } - force_iommu = 1; + if ( intremap_enabled ) + force_iommu = 1; + } if ( !x2apic_enabled ) { @@ -938,13 +930,16 @@ void __init x2apic_bsp_setup(void) printk("Switched to APIC driver %s\n", genapic.name); restore_out: - /* - * NB: do not use raw mode when restoring entries if the iommu has been - * enabled during the process, because the entries need to be translated - * and added to the remapping table in that case. - */ - restore_IO_APIC_setup(ioapic_entries, !intremap_enabled); - unmask_8259A(); + if ( iommu_supports_x2apic() ) + { + /* + * NB: do not use raw mode when restoring entries if the iommu has been + * enabled during the process, because the entries need to be translated + * and added to the remapping table in that case. + */ + restore_IO_APIC_setup(ioapic_entries, !intremap_enabled); + unmask_8259A(); + } out: if ( ioapic_entries )