From patchwork Fri Aug 20 15:59:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Brijesh Singh X-Patchwork-Id: 12449847 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1DC52C4338F for ; Fri, 20 Aug 2021 16:02:12 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B29B461250 for ; Fri, 20 Aug 2021 16:02:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org B29B461250 Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id E33A98D0025; Fri, 20 Aug 2021 12:01:13 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DBBFC8D0007; Fri, 20 Aug 2021 12:01:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C35FF8D0025; Fri, 20 Aug 2021 12:01:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0107.hostedemail.com [216.40.44.107]) by kanga.kvack.org (Postfix) with ESMTP id A722A8D0007 for ; Fri, 20 Aug 2021 12:01:13 -0400 (EDT) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 4D39E1848227E for ; Fri, 20 Aug 2021 16:01:13 +0000 (UTC) X-FDA: 78495923226.01.AA7C018 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2051.outbound.protection.outlook.com [40.107.237.51]) by imf10.hostedemail.com (Postfix) with ESMTP id CDDB0600198C for ; Fri, 20 Aug 2021 16:01:12 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ZeyREFdj2y/Gz7rNNpyMgvE05Q8zDnzLdsg0V7jOzO5JGoYl7fyXwj+gOAk8woCC0584fEv0zzPLV4knYyCMumVe97syxbRkv3he9iYOJLYjAu/5dCRp8S41CJj5/y8L2WQD8pY+n9owYOivMNGnXlMzZ6g5tlNxEgetLEef6CM3Lq7FKqlWRqYMhGMVD19EVjXeBMzk9Tgs84VQA6w3aOWtiMYeM67hKl+4OLFrSptu2SqlmVG31zEeNicVNW75bKVch8ihcZEqjokhbbPijsdoTu2Fz+qGP/6i/5+NW3oCqGYUNEQW89/RudkI4MUukjBtzd8Y28u8cn9Whr2owA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=oZFq/dGo+kLu7NVgIMt7qlWNCwQLMd7WTut81kZQR7w=; b=MwbNoSzmX41k8xalB3sOs1/8x63obdtQfZyHKEOJxMK92OrH8oYuyZVCPC13mWUWlcqY8cPL11HwDTzJlSwhBSsNt0mAgdNsGj6vigPr85ZlCzPaYOXOvY5EN0YK1MYmlylVCy+TR22hEiMMO9wVTX1g5fxrqGSKqsd0tArv3pi4lRaOy27CQ+Q4+sA5aLThSFzinHO4zWO+8+WTEO0LNAqvKKn73XjFeXVq3+iG2tF5tPUgKikdcyOIcQMpHJHhYxOoQEyK2vsQQttWW42BeAaM8XEKxc/Nyb9qc9gW/jV6uxliHn/dLfoVvs4HNyJK8kpx3VUF1j6MZ7l7B5G7hA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=oZFq/dGo+kLu7NVgIMt7qlWNCwQLMd7WTut81kZQR7w=; b=Qr8o+WFa+4tFiZBghHCrVEePlq+2XqSP/a7KFXlj0amIUMjNNMSnX7WLaOdHoB9UrFUJjV4G9WxayBGfYTQbfF9PnhXABqaDdXu4BYhlRiYNSxoJf/Z0vGu0TSVwx8P0j2LQojAIwBOmilIeEhSN4v2Kzhp6bh35jPc0KcVT51A= Received: from SN6PR12MB2718.namprd12.prod.outlook.com (2603:10b6:805:6f::22) by SA0PR12MB4509.namprd12.prod.outlook.com (2603:10b6:806:9e::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4436.19; Fri, 20 Aug 2021 16:01:09 +0000 Received: from SN6PR12MB2718.namprd12.prod.outlook.com ([fe80::78b7:7336:d363:9be3]) by SN6PR12MB2718.namprd12.prod.outlook.com ([fe80::78b7:7336:d363:9be3%6]) with mapi id 15.20.4436.019; Fri, 20 Aug 2021 16:01:09 +0000 From: Brijesh Singh To: x86@kernel.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-coco@lists.linux.dev, linux-mm@kvack.org, linux-crypto@vger.kernel.org Cc: Thomas Gleixner , Ingo Molnar , Joerg Roedel , Tom Lendacky , "H. Peter Anvin" , Ard Biesheuvel , Paolo Bonzini , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Andy Lutomirski , Dave Hansen , Sergio Lopez , Peter Gonda , Peter Zijlstra , Srinivas Pandruvada , David Rientjes , Dov Murik , Tobin Feldman-Fitzthum , Borislav Petkov , Michael Roth , Vlastimil Babka , "Kirill A . Shutemov" , Andi Kleen , tony.luck@intel.com, marcorr@google.com, sathyanarayanan.kuppuswamy@linux.intel.com, Brijesh Singh Subject: [PATCH Part2 v5 41/45] KVM: SVM: Add support to handle the RMP nested page fault Date: Fri, 20 Aug 2021 10:59:14 -0500 Message-Id: <20210820155918.7518-42-brijesh.singh@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210820155918.7518-1-brijesh.singh@amd.com> References: <20210820155918.7518-1-brijesh.singh@amd.com> X-ClientProxiedBy: SN7P222CA0013.NAMP222.PROD.OUTLOOK.COM (2603:10b6:806:124::11) To SN6PR12MB2718.namprd12.prod.outlook.com (2603:10b6:805:6f::22) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from sbrijesh-desktop.amd.com (165.204.77.1) by SN7P222CA0013.NAMP222.PROD.OUTLOOK.COM (2603:10b6:806:124::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4436.18 via Frontend Transport; Fri, 20 Aug 2021 16:00:45 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 4b031cdf-5472-4765-09de-08d963f3abd5 X-MS-TrafficTypeDiagnostic: SA0PR12MB4509: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:8882; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: q1oNDHqMARHmr9v1JhMNmxaypVk3nPte6KbcUb1Tnk6Qi+DE0Z0Zhqn9d/Q0PDpIKDcvhSYRDcBJUASO4fq3xwi19hBeIvFFNjC1+z1VDSOUpeIRJcHwb7fnl1Y3Snnfko5VlCzOCgQLDkOT2JyfynIaxMTzsAGzs5OelyxHtc089R+wbplEir/ZHin1ZPpvjEsFRHYlX9rexHKDww3Md0rOJql2jLreAEF+seROnvLDkjYOvoZsy6LTi1GZWXLsB0L9BU5aSpOEFRlMmFcrI5fXOnEUeWhN8Q9ud539IEn66C2jC9sMGm70FOqDs8AvI7nfEPD5CO0DPg1Bw1YSa+zUc35UdE47X3AysAZtSylVV2cHdqHiZ5iFazgBC01GpPXNm8/4S/cV7WG3o9VsGL2ZKsYg5/s14cPHfXCtLAEdbhVHe//GBonct3bWG1q+qm5rmpzRv7kiSLQmvYod7YwDOP8HcMGiGu8BqlyvzYH9PVo1p7Al1xIwLTUzmi/POnqulcpcBjAHHXLWIH86GiiTzSUdKk6Pog6Mt2xpiaOxYBKs9L1d0OmwcdSUsf1yITDzjq16zU14bnCNEZuOmJnTo2H4Eh8+lOc4rIti54g4XVHGzvZqwU6HVBz1/qqvVPnlhd+KKIjPWejnebQtwCCU4NFHRXnNUKvTadxMG38tiGrjUW0n9zLW5Kmlw+rxitVA6IFGoDo0CERTiLX8zA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SN6PR12MB2718.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(508600001)(4326008)(36756003)(7416002)(44832011)(54906003)(316002)(66946007)(66556008)(66476007)(86362001)(7406005)(956004)(6486002)(2616005)(2906002)(83380400001)(38350700002)(38100700002)(186003)(5660300002)(8936002)(52116002)(1076003)(7696005)(8676002)(26005)(6666004);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: Qws1+R0TtTyy8+w6UO72Dc6C1ISyqL7UTPb8QVRbiJiKgOJZZIIqk3qaSW2bD6EeX7/wtm1GxOWptLX6MkHfSRkwz8rF5IdGHUoKhJpLVqmHAA8fTdSiLT7msX7mS746vW5scqt5e9Raa1GS4W1O7DPRLZEALzwCyO2WYCdCHzdMR0Bem8A7Mpt0dX4js/1dBXasqquajvlJ5jc1P9jfnCPDMQpYwPzFmT6ss7e9RfhWWk3QWBfimlrri/8NVz5Dbylg0L5G8hywIrHghgz0hePB8M1CwdKrJvJQ/BhciXraJkbb/hKBUX1y7XiAkLMyNjtk+JqjEAudQiLLSzlbLBw6QyAedLXjSYPnl+pCbMOegCwKS360EGi9Z+pR3RvemPj1RRf2iYbc4zHxYGZLEXU9Kc+x/M0+zrqWqQLhZ2ZvZhLmvxdf1CRSomwE2mmQXxMjVkGlAg0KCCXXwM/SXvA8/k3BHFSu3SfuGn/7DquZy6rYf1OHMjUv992edC8fU0PTgHM4YG1ndrKXhc52nXD1gbIfg/ChuAOGnzJe1GmngCXm21whcUAmq7evwl4WrNd7EVmYhLw68j5vr07oabejBtRDQnoZtOyEyy+D0aianCTq9X9ZPp8JcFeTYnvCUV3f2FLVET9G+jv3AMW0BJ56cv3lwpp66Cqy3RB9R7h7jrVLh2wF6rUi/JzwxEk4Cd+Dv1jJ0qsc0i1BEibT7qUJq2YX7gxshmeEnX4Mpb91CSaZHeUPNCG9ITNNlNX4vARH3dSRPPUed1WWgf2THlsjdF/PNCWlKLobtAWWk6JgssJdr3qm4xpICMwI1JJrvF+o2Ucpb9Fpso13zUW1Kzdn9S+Lh13crpwYBZmCd8N0+Xr8W1BkBg/e/my9nvg3TSBkABSzDkruh1SuG2lauK2W4W9JkLIvK6Eu/rYT5Bf70cZpV7hnbd4x0+r4g8ApbsF+9BvhF1rhKQ2acf18QrTB72XbMjLDEL+Gy7d6UTMkAzZoQC+7t+tFe+/lH05gNIJMK4eykLSSn6g10/8Jx3iigrju2zkIK1j3YIPeiqRDyjM4TduKfWgLDf8qS7JmhhCBUjHx5ZAGqUUbupDRJl9XDQrCO1IYAaS1ppZ/6czfE49vwPHgnW85sXgLzzS36abvg8CPyB+anEWTC7y3zhrOFBhkHYx6Q0yz9s3W2u+GoQUzxoaTewqK4vQUV+awdspnFoaBcAluFokjFq0b0ZxLwZzBsSALTps1eM+YBOtxrh2l2slVucrL2jSGiIPmkFelzZ5Tsr3caTZ5OTwBDbdJh65qj2J/NJqkHZsY2pv2ugfsTKoA54mwZkD5vpST X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4b031cdf-5472-4765-09de-08d963f3abd5 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB2718.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Aug 2021 16:00:46.2922 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: hM3NJoOXNQQ6jh/SS7wW53veguG+0idHwKBoz378JLu91HmmHWD04UPNp1qQgDt/0vnaagD+qG3Xiaj6MQ8e6A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4509 Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=amd.com header.s=selector1 header.b=Qr8o+WFa; dmarc=pass (policy=quarantine) header.from=amd.com; spf=pass (imf10.hostedemail.com: domain of brijesh.singh@amd.com designates 40.107.237.51 as permitted sender) smtp.mailfrom=brijesh.singh@amd.com X-Stat-Signature: i71krkpjrbsdbx4tc1sod6j8g7txhsc3 X-Rspamd-Queue-Id: CDDB0600198C X-Rspamd-Server: rspam05 X-HE-Tag: 1629475272-361322 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When SEV-SNP is enabled in the guest, the hardware places restrictions on all memory accesses based on the contents of the RMP table. When hardware encounters RMP check failure caused by the guest memory access it raises the #NPF. The error code contains additional information on the access type. See the APM volume 2 for additional information. Signed-off-by: Brijesh Singh --- arch/x86/kvm/svm/sev.c | 76 ++++++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/svm/svm.c | 14 +++++--- arch/x86/kvm/svm/svm.h | 1 + 3 files changed, 87 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c index 65b578463271..712e8907bc39 100644 --- a/arch/x86/kvm/svm/sev.c +++ b/arch/x86/kvm/svm/sev.c @@ -3651,3 +3651,79 @@ void sev_post_unmap_gfn(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn, int token) srcu_read_unlock(&sev->psc_srcu, token); } + +void handle_rmp_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u64 error_code) +{ + int rmp_level, npt_level, rc, assigned; + struct kvm *kvm = vcpu->kvm; + gfn_t gfn = gpa_to_gfn(gpa); + bool need_psc = false; + enum psc_op psc_op; + kvm_pfn_t pfn; + bool private; + + write_lock(&kvm->mmu_lock); + + if (unlikely(!kvm_mmu_get_tdp_walk(vcpu, gpa, &pfn, &npt_level))) + goto unlock; + + assigned = snp_lookup_rmpentry(pfn, &rmp_level); + if (unlikely(assigned < 0)) + goto unlock; + + private = !!(error_code & PFERR_GUEST_ENC_MASK); + + /* + * If the fault was due to size mismatch, or NPT and RMP page level's + * are not in sync, then use PSMASH to split the RMP entry into 4K. + */ + if ((error_code & PFERR_GUEST_SIZEM_MASK) || + (npt_level == PG_LEVEL_4K && rmp_level == PG_LEVEL_2M && private)) { + rc = snp_rmptable_psmash(kvm, pfn); + if (rc) + pr_err_ratelimited("psmash failed, gpa 0x%llx pfn 0x%llx rc %d\n", + gpa, pfn, rc); + goto out; + } + + /* + * If it's a private access, and the page is not assigned in the + * RMP table, create a new private RMP entry. This can happen if + * guest did not use the PSC VMGEXIT to transition the page state + * before the access. + */ + if (!assigned && private) { + need_psc = 1; + psc_op = SNP_PAGE_STATE_PRIVATE; + goto out; + } + + /* + * If it's a shared access, but the page is private in the RMP table + * then make the page shared in the RMP table. This can happen if + * the guest did not use the PSC VMGEXIT to transition the page + * state before the access. + */ + if (assigned && !private) { + need_psc = 1; + psc_op = SNP_PAGE_STATE_SHARED; + } + +out: + write_unlock(&kvm->mmu_lock); + + if (need_psc) + rc = __snp_handle_page_state_change(vcpu, psc_op, gpa, PG_LEVEL_4K); + + /* + * The fault handler has updated the RMP pagesize, zap the existing + * rmaps for large entry ranges so that nested page table gets rebuilt + * with the updated RMP pagesize. + */ + gfn = gpa_to_gfn(gpa) & ~(KVM_PAGES_PER_HPAGE(PG_LEVEL_2M) - 1); + kvm_zap_gfn_range(kvm, gfn, gfn + PTRS_PER_PMD); + return; + +unlock: + write_unlock(&kvm->mmu_lock); +} diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index 3784d389247b..3ba62f21b113 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -1933,15 +1933,21 @@ static int pf_interception(struct kvm_vcpu *vcpu) static int npf_interception(struct kvm_vcpu *vcpu) { struct vcpu_svm *svm = to_svm(vcpu); + int rc; u64 fault_address = svm->vmcb->control.exit_info_2; u64 error_code = svm->vmcb->control.exit_info_1; trace_kvm_page_fault(fault_address, error_code); - return kvm_mmu_page_fault(vcpu, fault_address, error_code, - static_cpu_has(X86_FEATURE_DECODEASSISTS) ? - svm->vmcb->control.insn_bytes : NULL, - svm->vmcb->control.insn_len); + rc = kvm_mmu_page_fault(vcpu, fault_address, error_code, + static_cpu_has(X86_FEATURE_DECODEASSISTS) ? + svm->vmcb->control.insn_bytes : NULL, + svm->vmcb->control.insn_len); + + if (error_code & PFERR_GUEST_RMP_MASK) + handle_rmp_page_fault(vcpu, fault_address, error_code); + + return rc; } static int db_interception(struct kvm_vcpu *vcpu) diff --git a/arch/x86/kvm/svm/svm.h b/arch/x86/kvm/svm/svm.h index ff91184f9b4a..280072995306 100644 --- a/arch/x86/kvm/svm/svm.h +++ b/arch/x86/kvm/svm/svm.h @@ -626,6 +626,7 @@ struct page *snp_safe_alloc_page(struct kvm_vcpu *vcpu); void sev_rmp_page_level_adjust(struct kvm *kvm, kvm_pfn_t pfn, int *level); int sev_post_map_gfn(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn, int *token); void sev_post_unmap_gfn(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn, int token); +void handle_rmp_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u64 error_code); /* vmenter.S */