From patchwork Mon Jun 20 23:15:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Kalra, Ashish" X-Patchwork-Id: 12888549 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AB2B5C433EF for ; Mon, 20 Jun 2022 23:15:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 38C738E0008; Mon, 20 Jun 2022 19:15:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 33C648E0003; Mon, 20 Jun 2022 19:15:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1DDCE8E0008; Mon, 20 Jun 2022 19:15:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 0FC608E0003 for ; Mon, 20 Jun 2022 19:15:29 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay12.hostedemail.com (Postfix) with ESMTP id D65131203C2 for ; Mon, 20 Jun 2022 23:15:28 +0000 (UTC) X-FDA: 79600172736.25.5096156 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2053.outbound.protection.outlook.com [40.107.243.53]) by imf30.hostedemail.com (Postfix) with ESMTP id 72D6480014 for ; Mon, 20 Jun 2022 23:15:28 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ZYwaI0tgXV8iujoQ0STf87VgKmtCejm+HHmvfptTkN9alywJGXnmvYQQZRxmhMuVT5yVMBHlE7Zv64/pEBOYc4YVFNcz3apgpSKa0n9fIO083gVOts/uHDLQrwxZzohR9jrNyT+y91Ed8uNW83AwB2aXD12UowOfvpqStfvXupZiZkMERPHCfijUqfRoiAwBXKrjIqpEJmjDGFRA06klt8+4pEaEb3yhc9AVn7R8UVnkSzPDbKekO4u8be5/PZ0cwb/6K+VoQBkfuQlMfEQphC8FY1gEvDWMZCNO3GuqM8V7Nr86SE1r1Cv9V0O5KPsnW6pj6NOBZnWbUDSPshq47g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=l8mVgporlVHbBNT8Cx4HixJv280dWnam0Bd62mXNBPA=; b=iIVgHZZ3mlX3fcFQlVeWvIZboL/gwNbER+v4xGwRKiUuV1H/TaiJq28dhXrCXtUT/GewtUdan9d7L5dIJ0rvhrjB+gaI7GzylgdE0MzTJ0rVwsnuvouXCPsVesFQQK3gU31hQ5UKX0A+WW0aYiedmJL+wDl1110ISBRBSC+kgqj+wcE3L+SeI8rYTf4H5/1o7cSRNS7X9yKNc9OdVgpia0jjr+OL/GOGZSYzQ0vqkDAfmDUqI/aGtx8++3/LdizMMZx6RiP6CDEtP0c8k2KVEtMeQQ6fGkfNvq2xPxGlj9JdekGnySqM6+hEAB8/KImP5+1Kk3VHLmh9UkVMUSpKIQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=l8mVgporlVHbBNT8Cx4HixJv280dWnam0Bd62mXNBPA=; b=YE7imnvJqYPUpSGlIqaGycrigKYbaSqqmb0ubWvz25gYzm0NFSXMLpOhgxg3bUWn5Ai4bFCXL+uWp+ezxxJPVob25uN4mpRZaRtWRb8W4+9n1go7Vmt3ARxox82u4/1ddhjNVpHxtTED6g9Qd/K2F471EyGghg0xrnXqczPx/OY= Received: from CPYP284CA0023.BRAP284.PROD.OUTLOOK.COM (2603:10d6:103:69::10) by DM5PR12MB1852.namprd12.prod.outlook.com (2603:10b6:3:10a::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5353.17; Mon, 20 Jun 2022 23:15:26 +0000 Received: from DM6NAM11FT048.eop-nam11.prod.protection.outlook.com (2603:10d6:103:69:cafe::e8) by CPYP284CA0023.outlook.office365.com (2603:10d6:103:69::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5353.16 via Frontend Transport; Mon, 20 Jun 2022 23:15:24 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by DM6NAM11FT048.mail.protection.outlook.com (10.13.173.114) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5353.14 via Frontend Transport; Mon, 20 Jun 2022 23:15:24 +0000 Received: from ashkalraubuntuserver.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Mon, 20 Jun 2022 18:15:21 -0500 From: Ashish Kalra To: , , , , , CC: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH Part2 v6 48/49] *debug: warn and retry failed rmpupdates Date: Mon, 20 Jun 2022 23:15:13 +0000 Message-ID: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 6d64c1ae-c0ae-4d38-72f9-08da5312c12c X-MS-TrafficTypeDiagnostic: DM5PR12MB1852:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 0vc00GkCxtEWymMsNyxuWpcPUXgpZwvyX+ftm+XeS65Q+LCs9lD13tQeOS1YkV9MTTqhSYX58Y0il3dgT/v7sRUPZrLHigD5KaI2CgeqEGqtnMTKTANB5xpNW+9S1FwhSOLlP3ed5DMP2oQEeN4RZikb/0nfki+hL/KEK9GO9r5a7Mx/d/D0JyQeVNe5CTUgYSFSF3itKx4A4Yzu6OT060fzVvbl67fnSHwOn5SO+365av2l0Owzj8Duh5GK/l2w4RxemFZyCaSDQlT1BfKYSn23r/8irocAywwKTWRmmf5n8g44EQ+6G1nOBmjWo7yHbSMy0Q00/JqZqg2FWAQJncWdUBjiiQcDL2ykvTcUowlyb8Occa912ZvNP1O+R47TzuhurOW8jEYWVCaAfAYTxDECdeJd+bc+xGZf6YkZ1Z0DHz+PNTUsN6dRWvNnHILiaMglGPO1NsWqtiWLA4kWAAq8YYRR2S/g5WCzytuufhHBr1LV9ZEW0ycs+7EB9Evp+Pi489YJiXw0n3JhfQ9qkIZeI7YIkfEZqg9Wghue1US+xPDUSOnce8gFh2fWb1wBz4fzYbViYpAha1hRUvJ2+lub+oGFVEkbhiliWTBGJrdJwh1BFVFZOdwhl2dlNftlcAGjCzxrWBFZ4oSxHhp8dtIa7q7GldwRFbLOBKfxLk9pcwK13v+wOl+KzNbjUs255In/CfazbLs7nmQKFH6KdjtQyAYkfMjpXnu8d+beVggIuUnhyMRlaR+DsdE4JQDVaKUzLRFoY+w1WPQjMikQTw== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230016)(4636009)(346002)(136003)(376002)(39860400002)(396003)(40470700004)(46966006)(36840700001)(36860700001)(82310400005)(40460700003)(86362001)(356005)(5660300002)(8936002)(82740400003)(81166007)(478600001)(7416002)(7406005)(2906002)(316002)(40480700001)(70586007)(4326008)(8676002)(336012)(6666004)(186003)(47076005)(426003)(70206006)(83380400001)(7696005)(110136005)(2616005)(26005)(41300700001)(16526019)(54906003)(36756003)(2101003)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Jun 2022 23:15:24.0841 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 6d64c1ae-c0ae-4d38-72f9-08da5312c12c X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT048.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR12MB1852 ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1655766928; a=rsa-sha256; cv=pass; b=jn+FXL+IpCfAbFjmkxtnFH4Iq2hMg7sSMQl8MbEtjjvrcWnQzKejmyL5rIgE792+S/7jk8 K3IuQPZDHdDOsRxp0G6ZABmskfrTzi40XWRKs7J2IFZGh92/sJwyxvsZpb5MC9Xq982/3e 5Y0Bj5mGGrUEfbKjZ08tyC7EVef7pWY= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1655766928; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=l8mVgporlVHbBNT8Cx4HixJv280dWnam0Bd62mXNBPA=; b=Nl6PU+pjdFpahmhgsrIMSXHoK7U43qHAUEmemh7WQ/bCXTbRi66dLGFN1lkMnn/3UFmEFB Pz3z6i9fEdyo88OlrqnxhjATGEFAWTQsDvDIi8tYhl2Nyy02j0+2Xzax3kAmzaKSV0EYF+ DrBrBr5sCGJVJJkhAPn2S2TxlYK+GnU= ARC-Authentication-Results: i=2; imf30.hostedemail.com; dkim=pass header.d=amd.com header.s=selector1 header.b=YE7imnvJ; dmarc=pass (policy=quarantine) header.from=amd.com; spf=pass (imf30.hostedemail.com: domain of Ashish.Kalra@amd.com designates 40.107.243.53 as permitted sender) smtp.mailfrom=Ashish.Kalra@amd.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=amd.com header.s=selector1 header.b=YE7imnvJ; dmarc=pass (policy=quarantine) header.from=amd.com; spf=pass (imf30.hostedemail.com: domain of Ashish.Kalra@amd.com designates 40.107.243.53 as permitted sender) smtp.mailfrom=Ashish.Kalra@amd.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") X-Rspam-User: X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 72D6480014 X-Stat-Signature: xu7wuj6egyz3nperxz54rbr7cqbf5qnj X-HE-Tag: 1655766928-843398 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Michael Roth In some cases on B0 hardware exhibits something like the following behavior (where M < 512): Guest A | Guest B |-------------------------------|----------------------------------| | | rc = rmpupdate pfn=N*512,4K,priv | rmpupdate pfn=N*512+M,4K,priv | | rc = FAIL_OVERLAP | rc = SUCCESS The FAIL_OVERLAP might possible be the result of hardware temporarily treating Guest B's rmpupdate for pfn=N*512 as a 2M update, causing the subsequent update from Guest A for pfn=N*512+M to report FAIL_OVERLAP at that particular instance. Retrying the update for N*512+M immediately afterward seems to resolve the FAIL_OVERLAP issue reliably however. A similar failure has also been observed when transitioning pages back to shared during VM destroy. In this case repeating the rmpupdate does not always seem to resolve the failure immediately. Both situations are much more likely to occur if THP is disabled, or if it is enabled/disabled while guests are actively being started/stopped. Include some debug/error information to get a better idea of the behavior on different hardware, and add the rmpupdate retry as a workaround for Milan B0 testing. Signed-off-by: Michael Roth --- arch/x86/kernel/sev.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/arch/x86/kernel/sev.c b/arch/x86/kernel/sev.c index 6640a639fffc..5ae8c9f853c8 100644 --- a/arch/x86/kernel/sev.c +++ b/arch/x86/kernel/sev.c @@ -2530,6 +2530,7 @@ static int rmpupdate(u64 pfn, struct rmpupdate *val) { unsigned long paddr = pfn << PAGE_SHIFT; int ret, level, npages; + int retries = 0; if (!pfn_valid(pfn)) return -EINVAL; @@ -2552,12 +2553,26 @@ static int rmpupdate(u64 pfn, struct rmpupdate *val) } } +retry: /* Binutils version 2.36 supports the RMPUPDATE mnemonic. */ asm volatile(".byte 0xF2, 0x0F, 0x01, 0xFE" : "=a"(ret) : "a"(paddr), "c"((unsigned long)val) : "memory", "cc"); + if (ret) { + if (!retries) { + pr_err("rmpupdate failed, ret: %d, pfn: %llx, npages: %d, level: %d, retrying (max: %d)...\n", + ret, pfn, npages, level, 2 * num_present_cpus()); + dump_stack(); + } + retries++; + if (retries < 2 * num_present_cpus()) + goto retry; + } else if (retries > 0) { + pr_err("rmpupdate for pfn %llx succeeded after %d retries\n", pfn, retries); + } + /* * Restore the direct map after the page is removed from the RMP table. */