From patchwork Thu Jul 18 12:09:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 11048899 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0F619746 for ; Thu, 18 Jul 2019 12:14:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F20F728820 for ; Thu, 18 Jul 2019 12:14:03 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E6A5C28821; Thu, 18 Jul 2019 12:14:03 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 807F028803 for ; Thu, 18 Jul 2019 12:14:03 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ho5HC-0002k5-18; Thu, 18 Jul 2019 12:12:46 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ho5HA-0002jq-TZ for xen-devel@lists.xenproject.org; Thu, 18 Jul 2019 12:12:44 +0000 X-Inumbo-ID: 56efb556-a955-11e9-93f3-1b1188db7716 Received: from m4a0040g.houston.softwaregrp.com (unknown [15.124.2.86]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 56efb556-a955-11e9-93f3-1b1188db7716; Thu, 18 Jul 2019 12:12:42 +0000 (UTC) Received: FROM m4a0040g.houston.softwaregrp.com (15.120.17.146) BY m4a0040g.houston.softwaregrp.com WITH ESMTP; Thu, 18 Jul 2019 12:08:48 +0000 Received: from M9W0068.microfocus.com (2002:f79:bf::f79:bf) by M4W0334.microfocus.com (2002:f78:1192::f78:1192) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1591.10; Thu, 18 Jul 2019 12:09:50 +0000 Received: from NAM05-BY2-obe.outbound.protection.outlook.com (15.124.72.13) by M9W0068.microfocus.com (15.121.0.191) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1591.10 via Frontend Transport; Thu, 18 Jul 2019 12:09:50 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=NC7yc/R/TToKE4W4B88esM+pZsmpxJoA9hUJN8oHfWO/kZ8FmW9OpdP9Kv5m98AcaGSgSTcMLdPdy3ZiQzlC1F6aAazTc9Atk9cy0TSX5aqX7DLEZK56z4c+H9gzKaJeoRmlZdzCe/RUmrA0sDCifGYRGgBABbdyztFj2yolpBDC2DXq2/fXm6NNvg01dLxyVxenrY7adRGRryXeYb1zpuoNzDkHA4+PRg9601LVNkdMFkbT4p4xPvDFpmCkHwLKIcohINtfzfyillBXMp84wHjyuc1q3JtJt/1DUBO47qBdV0FpR/n1kAoDM+8doZ/bdx4zVzAtrOpwurEDq6THcw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/77QCDKoT4OyA1szsNFhbW62rB9XBeXO6z6r+BHhLdQ=; b=m2ym+5JbMUuZoQ4pTlJpqDF36D8/qBUM1J8Tk+j6XaMKgNjN6eCw/A+X4cljjB5Iuzfr2WZHoS18v5tJOAsMJAsKaCoSRvIsL7sPqs0YV2d9aSGiQEFHeKiQT1zQzgNSjIujQBcTdfBH2fglkbL5dSYOFk+4/fXtrAZ+LQXMWOL6RnOyzt2s502N6dlFFn2EfiUFJ7mDV7G9pzwvslaZiADjBRM+QG7JdA9gBGF7XjnK4meGaMbB55KAY9+ZVTbZwIJJk/Mt+Pss8rR/o4xi/VLnMIQpkESbKF4rtuAa46qNviztQax1GOtBBkbFWpgg6KrfzKzd6SgN+rVwgWH6Nw== ARC-Authentication-Results: i=1; mx.microsoft.com 1;spf=pass smtp.mailfrom=suse.com;dmarc=pass action=none header.from=suse.com;dkim=pass header.d=suse.com;arc=none Received: from DM6PR18MB3401.namprd18.prod.outlook.com (10.255.174.218) by DM6PR18MB2796.namprd18.prod.outlook.com (20.179.50.83) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2073.14; Thu, 18 Jul 2019 12:09:49 +0000 Received: from DM6PR18MB3401.namprd18.prod.outlook.com ([fe80::1fe:35f6:faf3:78c7]) by DM6PR18MB3401.namprd18.prod.outlook.com ([fe80::1fe:35f6:faf3:78c7%7]) with mapi id 15.20.2073.012; Thu, 18 Jul 2019 12:09:49 +0000 From: Jan Beulich To: "xen-devel@lists.xenproject.org" Thread-Topic: [PATCH 1/2] x86/cpu/intel: Clear cache self-snoop capability in CPUs with known errata Thread-Index: AQHVPWGy1ek8AecLnECtUEoEAro4wg== Date: Thu, 18 Jul 2019 12:09:49 +0000 Message-ID: <05257008-13e3-0d49-cd1d-6a8c9eee2ce5@suse.com> References: <31d4cb3f-6ff0-a13c-00ce-bced77c6dd78@suse.com> In-Reply-To: <31d4cb3f-6ff0-a13c-00ce-bced77c6dd78@suse.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: DB7PR05CA0041.eurprd05.prod.outlook.com (2603:10a6:10:2e::18) To DM6PR18MB3401.namprd18.prod.outlook.com (2603:10b6:5:1cc::26) authentication-results: spf=none (sender IP is ) smtp.mailfrom=JBeulich@suse.com; x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [87.234.252.170] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 6d32ac17-4e91-4cf3-9b47-08d70b78d4d3 x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600148)(711020)(4605104)(1401327)(2017052603328)(7193020); SRVR:DM6PR18MB2796; x-ms-traffictypediagnostic: DM6PR18MB2796: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:1468; x-forefront-prvs: 01026E1310 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(4636009)(366004)(376002)(136003)(346002)(396003)(39860400002)(199004)(189003)(80792005)(478600001)(76176011)(14454004)(66066001)(8676002)(186003)(52116002)(31696002)(476003)(446003)(25786009)(7736002)(2616005)(68736007)(2906002)(6512007)(2501003)(11346002)(305945005)(26005)(6506007)(6436002)(31686004)(3846002)(6486002)(36756003)(386003)(4326008)(486006)(81156014)(316002)(54906003)(81166006)(66476007)(102836004)(256004)(71190400001)(6116002)(2351001)(86362001)(66946007)(66556008)(64756008)(66446008)(71200400001)(5660300002)(99286004)(6916009)(8936002)(53936002)(5640700003); DIR:OUT; SFP:1102; SCL:1; SRVR:DM6PR18MB2796; H:DM6PR18MB3401.namprd18.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: suse.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: LHHCNvFUA/OM4V3wbUXRvCYtwDkOrZ0E4PjPKbcdsG2WUSMpDnCTjmW10MXoEr7QYXeIA6gXiU0QwWhCG/QJMHmlx9zSm4fdGvO9fbz81tBgkrhimUzEH6z74LmOpvIQL1CJMPR98wzSTtF2t3W+lxxuPOiLrplMie+OhR0VKhNUNNMIVuajtxjJUKadHNXVr74KXa57ICq+97ktwIOh9dYlmeceLWBBTkWp5DkZLIRWvaEj42/2rpFOWArMDeM8TGxByWTKXJr+9Nrrx9OO4tg3U7nPNjzS9p4ORWUDP3UD2JYTYfS36qbxop89+lE8PffvmQhuLgyV7yhyqJeW++hU5LfmBgbFeIDdynkiaM4TrYlcyg2rS2q5dmDDoNagPrBdyUisb7yCxFQWOjzh+Fw6IjoRudV2ifFzc24drJA= Content-ID: MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: 6d32ac17-4e91-4cf3-9b47-08d70b78d4d3 X-MS-Exchange-CrossTenant-originalarrivaltime: 18 Jul 2019 12:09:49.1672 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 856b813c-16e5-49a5-85ec-6f081e13b527 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: JBeulich@suse.com X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR18MB2796 X-OriginatorOrg: suse.com Subject: [Xen-devel] [PATCH 1/2] x86/cpu/intel: Clear cache self-snoop capability in CPUs with known errata X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Ricardo Neri Processors which have self-snooping capability can handle conflicting memory type across CPUs by snooping its own cache. However, there exists CPU models in which having conflicting memory types still leads to unpredictable behavior, machine check errors, or hangs. Clear this feature on affected CPUs to prevent its use. Suggested-by: Alan Cox Signed-off-by: Ricardo Neri [Linux commit 1e03bff3600101bd9158d005e4313132e55bdec8] Strip Yonah - as per ark.intel.com it doesn't look to be 64-bit capable. Call check_memory_type_self_snoop_errata() only on the boot CPU. Requested-by: Andrew Cooper Signed-off-by: Jan Beulich --- a/xen/arch/x86/cpu/intel.c +++ b/xen/arch/x86/cpu/intel.c @@ -15,6 +15,32 @@ #include "cpu.h" /* + * Processors which have self-snooping capability can handle conflicting + * memory type across CPUs by snooping its own cache. However, there exists + * CPU models in which having conflicting memory types still leads to + * unpredictable behavior, machine check errors, or hangs. Clear this + * feature to prevent its use on machines with known erratas. + */ +static void __init check_memory_type_self_snoop_errata(void) +{ + switch (boot_cpu_data.x86_model) { + case 0x0f: /* Merom */ + case 0x16: /* Merom L */ + case 0x17: /* Penryn */ + case 0x1d: /* Dunnington */ + case 0x1e: /* Nehalem */ + case 0x1f: /* Auburndale / Havendale */ + case 0x1a: /* Nehalem EP */ + case 0x2e: /* Nehalem EX */ + case 0x25: /* Westmere */ + case 0x2c: /* Westmere EP */ + case 0x2a: /* SandyBridge */ + setup_clear_cpu_cap(X86_FEATURE_SS); + break; + } +} + +/* * Set caps in expected_levelling_cap, probe a specific masking MSR, and set * caps in levelling_caps if it is found, or clobber the MSR index if missing. * If preset, reads the default value into msr_val. @@ -256,8 +282,11 @@ static void early_init_intel(struct cpui (boot_cpu_data.x86_mask == 3 || boot_cpu_data.x86_mask == 4)) paddr_bits = 36; - if (c == &boot_cpu_data) + if (c == &boot_cpu_data) { + check_memory_type_self_snoop_errata(); + intel_init_levelling(); + } ctxt_switch_levelling(NULL); } From patchwork Thu Jul 18 12:10:15 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 11048895 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 41A8513AC for ; Thu, 18 Jul 2019 12:13:13 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 31680285C7 for ; Thu, 18 Jul 2019 12:13:13 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 25A9B28821; Thu, 18 Jul 2019 12:13:13 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id ED51528803 for ; Thu, 18 Jul 2019 12:13:11 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ho5GA-0002bw-8f; Thu, 18 Jul 2019 12:11:42 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ho5G8-0002bo-LQ for xen-devel@lists.xenproject.org; Thu, 18 Jul 2019 12:11:40 +0000 X-Inumbo-ID: 31e555a1-a955-11e9-8980-bc764e045a96 Received: from m4a0039g.houston.softwaregrp.com (unknown [15.124.2.85]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id 31e555a1-a955-11e9-8980-bc764e045a96; Thu, 18 Jul 2019 12:11:39 +0000 (UTC) Received: FROM m4a0039g.houston.softwaregrp.com (15.120.17.147) BY m4a0039g.houston.softwaregrp.com WITH ESMTP; Thu, 18 Jul 2019 12:11:37 +0000 Received: from M9W0067.microfocus.com (2002:f79:be::f79:be) by M4W0335.microfocus.com (2002:f78:1193::f78:1193) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1591.10; Thu, 18 Jul 2019 12:10:17 +0000 Received: from NAM02-BL2-obe.outbound.protection.outlook.com (15.124.72.13) by M9W0067.microfocus.com (15.121.0.190) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1591.10 via Frontend Transport; Thu, 18 Jul 2019 12:10:17 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VzEwoxulEWCM7VLmUTvRbWCLVQdDDWh+xpGdDP9mdhkxenYz2iC2/b4wD/Wt0mqc77r2i2DROSVvWa6OfpzH8HfmaxH6I/o2UXQOBTDN1WwBnqbzx5yt5D4t73/KtVhHpgzF5QqcMXOF8ZL6OzD2Iwy9XHfXotPPZAgeWl0NajWMcLGN92h83t3iMTTR9WwE2Q4M3t+ykpp79chqHyG5Gdt4OSIMYoGHnOomOwJhIeer/y8WO0Hi4iO8t38Ph8NFZwPS5HB2Qom7qbmUa7rjZd+a05R7mwbu4loT3S8mwj/bvlunLebVSrn5o/0hUxrlSpHSmms+j4c3RUjMPQ/hZQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KiOro+tnKuzBsHjfnvOCnAbrWvvNNQ4KGnUw8fME/dA=; b=AH/DbFXOGZJafoxqz2w0qhu7G5Sh/TzyXwnpNvv9ucMljCaYcivzqYG6fdGi/+dZ0Jx9fwPgZCbfe81+Wq0OuJ9PBrC5lhnv+rXRLJ1S+vo8tC3qlmOGNcFDR8LlCZY7Zfzv0rwyORX0X93QalpgCpEFXP3hi4qUW4NDVtp6WKnNVhUohvbH69fMKMKbTrXVvUQWwg4n9U8rmAdIrA5Q9aSQron8JF/7a71kOaRCHqVFm/mt1Fi7rXH2BAns63ucpfN1GkG4DfW52+r5XotkK7mnjp58uujI1CGNY5jmw8NPUt46BU9po+8sVIBjCUSPqpyzIWG8NnO8lW3p/plSgw== ARC-Authentication-Results: i=1; mx.microsoft.com 1;spf=pass smtp.mailfrom=suse.com;dmarc=pass action=none header.from=suse.com;dkim=pass header.d=suse.com;arc=none Received: from DM6PR18MB3401.namprd18.prod.outlook.com (10.255.174.218) by DM6PR18MB2796.namprd18.prod.outlook.com (20.179.50.83) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2073.14; Thu, 18 Jul 2019 12:10:16 +0000 Received: from DM6PR18MB3401.namprd18.prod.outlook.com ([fe80::1fe:35f6:faf3:78c7]) by DM6PR18MB3401.namprd18.prod.outlook.com ([fe80::1fe:35f6:faf3:78c7%7]) with mapi id 15.20.2073.012; Thu, 18 Jul 2019 12:10:16 +0000 From: Jan Beulich To: "xen-devel@lists.xenproject.org" Thread-Topic: [PATCH 2/2] x86/mtrr: Skip cache flushes on CPUs with cache self-snooping Thread-Index: AQHVPWHCwCiDeJzVW0Sy9S0HVfPNTQ== Date: Thu, 18 Jul 2019 12:10:15 +0000 Message-ID: References: <31d4cb3f-6ff0-a13c-00ce-bced77c6dd78@suse.com> In-Reply-To: <31d4cb3f-6ff0-a13c-00ce-bced77c6dd78@suse.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: DB8PR03CA0010.eurprd03.prod.outlook.com (2603:10a6:10:be::23) To DM6PR18MB3401.namprd18.prod.outlook.com (2603:10b6:5:1cc::26) authentication-results: spf=none (sender IP is ) smtp.mailfrom=JBeulich@suse.com; x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [87.234.252.170] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: c8711327-3d2a-44b9-3347-08d70b78e4c6 x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600148)(711020)(4605104)(1401327)(2017052603328)(7193020); SRVR:DM6PR18MB2796; x-ms-traffictypediagnostic: DM6PR18MB2796: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:2582; x-forefront-prvs: 01026E1310 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(4636009)(366004)(376002)(136003)(346002)(396003)(39860400002)(199004)(189003)(80792005)(478600001)(76176011)(14454004)(66066001)(8676002)(186003)(52116002)(31696002)(476003)(446003)(25786009)(7736002)(2616005)(68736007)(2906002)(6512007)(2501003)(11346002)(305945005)(26005)(6506007)(6436002)(31686004)(3846002)(6486002)(36756003)(386003)(4326008)(486006)(81156014)(316002)(54906003)(81166006)(66476007)(102836004)(256004)(71190400001)(6116002)(2351001)(86362001)(66946007)(66556008)(64756008)(66446008)(71200400001)(14444005)(5660300002)(99286004)(6916009)(8936002)(53936002)(5640700003); DIR:OUT; SFP:1102; SCL:1; SRVR:DM6PR18MB2796; H:DM6PR18MB3401.namprd18.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: suse.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: f9q0zI3Fwb8AEbFna1Kr7tT45qjpd4WL/1SjvFahXBy+XnrLGUrl9M/TAf5H5+Lvd+RjGArlqyeFpE8SRVXZuLeYw8DLQzyuMDzWcsU4R6DXXr8vSSYBHtvFJ9ZsrBLjMCreLMU+/EsrVmTjkoWOpQsneoU4Kmba3JwZRT1Hq7c8lHOZlQkTqLrQ/em6voUX3az095nkQEEjGwzFpPfnYN3eroC83SJga0qWeaTRbnHbjv2CUJ58CNAYFAKxrnDYekf/fKw5+zcFshVYgNHWxytmH3UMGAo4BnmfM7jGSed0SJo3MsvafU8ZXp4xlS4upznr7xdi7CMjROEwkxC3FqRdnCm3a/0pRnXBMqmTaeKamuVl/YHcx3JJhxLXFzdtDgehzI0nMSKbHhsArDRWK4zbU+uqfNZk5qXSY3OkLwU= Content-ID: MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: c8711327-3d2a-44b9-3347-08d70b78e4c6 X-MS-Exchange-CrossTenant-originalarrivaltime: 18 Jul 2019 12:10:15.9449 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 856b813c-16e5-49a5-85ec-6f081e13b527 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: JBeulich@suse.com X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR18MB2796 X-OriginatorOrg: suse.com Subject: [Xen-devel] [PATCH 2/2] x86/mtrr: Skip cache flushes on CPUs with cache self-snooping X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Ricardo Neri Programming MTRR registers in multi-processor systems is a rather lengthy process. Furthermore, all processors must program these registers in lock step and with interrupts disabled; the process also involves flushing caches and TLBs twice. As a result, the process may take a considerable amount of time. On some platforms, this can lead to a large skew of the refined-jiffies clock source. Early when booting, if no other clock is available (e.g., booting with hpet=disabled), the refined-jiffies clock source is used to monitor the TSC clock source. If the skew of refined-jiffies is too large, Linux wrongly assumes that the TSC is unstable: clocksource: timekeeping watchdog on CPU1: Marking clocksource 'tsc-early' as unstable because the skew is too large: clocksource: 'refined-jiffies' wd_now: fffedc10 wd_last: fffedb90 mask: ffffffff clocksource: 'tsc-early' cs_now: 5eccfddebc cs_last: 5e7e3303d4 mask: ffffffffffffffff tsc: Marking TSC unstable due to clocksource watchdog As per measurements, around 98% of the time needed by the procedure to program MTRRs in multi-processor systems is spent flushing caches with wbinvd(). As per the Section 11.11.8 of the Intel 64 and IA 32 Architectures Software Developer's Manual, it is not necessary to flush caches if the CPU supports cache self-snooping. Thus, skipping the cache flushes can reduce by several tens of milliseconds the time needed to complete the programming of the MTRR registers: Platform Before After 104-core (208 Threads) Skylake 1437ms 28ms 2-core ( 4 Threads) Haswell 114ms 2ms Reported-by: Mohammad Etemadi Signed-off-by: Ricardo Neri [Linux commit fd329f276ecaad7a371d6f91b9bbea031d0c3440] Use alternatives patching instead of static_cpu_has() (which we don't have [yet]). Interestingly we've been lacking the 2nd wbinvd(), which I'm taking the liberty here. Requested-by: Andrew Cooper Signed-off-by: Jan Beulich Reviewed-by: Andrew Cooper --- a/xen/arch/x86/cpu/mtrr/generic.c +++ b/xen/arch/x86/cpu/mtrr/generic.c @@ -450,7 +450,14 @@ static bool prepare_set(void) /* Enter the no-fill (CD=1, NW=0) cache mode and flush caches. */ write_cr0(read_cr0() | X86_CR0_CD); - wbinvd(); + + /* + * Cache flushing is the most time-consuming step when programming + * the MTRRs. Fortunately, as per the Intel Software Development + * Manual, we can skip it if the processor supports cache self- + * snooping. + */ + alternative("wbinvd", "", X86_FEATURE_SS); cr4 = read_cr4(); if (cr4 & X86_CR4_PGE) @@ -466,6 +473,9 @@ static bool prepare_set(void) /* Disable MTRRs, and set the default type to uncached */ mtrr_wrmsr(MSR_MTRRdefType, deftype & ~0xcff); + /* Again, only flush caches if we have to. */ + alternative("wbinvd", "", X86_FEATURE_SS); + return cr4 & X86_CR4_PGE; }