From patchwork Mon Oct 16 13:28:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Roth X-Patchwork-Id: 13423348 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F0091CDB465 for ; Mon, 16 Oct 2023 13:43:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 961908D0090; Mon, 16 Oct 2023 09:43:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 911598D0001; Mon, 16 Oct 2023 09:43:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7D9AC8D0090; Mon, 16 Oct 2023 09:43:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 6E9D38D0001 for ; Mon, 16 Oct 2023 09:43:00 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 429181A0935 for ; Mon, 16 Oct 2023 13:43:00 +0000 (UTC) X-FDA: 81351440520.21.00C2AF5 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2079.outbound.protection.outlook.com [40.107.237.79]) by imf20.hostedemail.com (Postfix) with ESMTP id 331391C002B for ; Mon, 16 Oct 2023 13:42:57 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=amd.com header.s=selector1 header.b=ZGY6BC6g; spf=pass (imf20.hostedemail.com: domain of Michael.Roth@amd.com designates 40.107.237.79 as permitted sender) smtp.mailfrom=Michael.Roth@amd.com; dmarc=pass (policy=quarantine) header.from=amd.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1697463777; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=YWWSaTmDjMI9U0l/FGVVcrWjBRDXASnIBVF5pzyJsJU=; b=N8Djb77rSGGypX3/5pX3MYppUsD9V/YNmt4JPP1GsMBL76Mv0m01sj0kcr9EL3n5LTfD4/ eFFsk5+89nXA43kxwXJKjnZBBaNR4d9nUulufFeUC6RJAhA1pNb3zgmZLN5VCHTJzfPyiU FMtXDR+vZRsFtgIP3s7J9IooA25RdeA= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1697463777; a=rsa-sha256; cv=pass; b=20yCWq2P21tXUUpj1I0INQTh79KIarpyHrdXAhA6OA5DY2aa8UjkK7Zv8OLasXQvz8ogPr fqmVeylhjoAJdP1wbzGuljU/s5igDC3SAZtFz3AZz+Wli2kM/bnXNmrDCnFxt+yU/HzvNY UKxMCKdBXkmViIZubqp+LNnS8HcUXP8= ARC-Authentication-Results: i=2; imf20.hostedemail.com; dkim=pass header.d=amd.com header.s=selector1 header.b=ZGY6BC6g; spf=pass (imf20.hostedemail.com: domain of Michael.Roth@amd.com designates 40.107.237.79 as permitted sender) smtp.mailfrom=Michael.Roth@amd.com; dmarc=pass (policy=quarantine) header.from=amd.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Spq8igZs7BzUnErvO7nf7PjxENe5cYDkE/ru8OcpzQWsS8AYExfXGmXHIqUj2Z5Q6O2OxcZMKQFXf3l3sqrf1pgGJnbYCztlPqVGX9SHVVXwQzpfnpCJ1rrxNxNPnDhfEFVG8kGFTLDn6oJUi3u6kaRJd6mwNbHZ9pDeDL2WCvf8rwju+9drKOPpQZzmeCOGPUNKnyXW4UGhan6reedu6O8vlLjEScz3VgetbT5hKkNkO3+7F5RfCHQB/iyJZhOUTDjdX1VQ5e24MocaDYEz72n9UR9gQlPb7RWhChLBfNkVjmmcczUVuQHCrPbBEjSlLvB4LEjU7B4yVUOy2G+xfQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=YWWSaTmDjMI9U0l/FGVVcrWjBRDXASnIBVF5pzyJsJU=; b=MA35UAigcHPOYnVv6k860C+JbvR6Qx7ge8x2a5oiqNEgZdl+CpcqF3l1/w1KvfvRcEFRtlgFqcgCGJpDg3SNekoDYGZeQV3Kzv97B2B+oEqQE8oeSPdHOieZgpTwERNFsqCCS9AnMK6RO4tliI5rEO+R9XEky1zhOcG+XAz7zfoMimF0UPt+hOnfGyvf4gCFTz69Jc/9odyS7PkNp0PD5BcC1/rPnY6piUTXpbfK031tw4LTTB81s7Uk8YUT+q6MOGe7jN+TiKgxE5/RfV8h4gCutMeXlsZUhVW3hp+6VRkjHcnKtd5TYwdq0WyuxvCbdshX6yMxMDuG3kYxw+kWJA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YWWSaTmDjMI9U0l/FGVVcrWjBRDXASnIBVF5pzyJsJU=; b=ZGY6BC6gl/eSbwLcl2DNbMPG9qEmhPF7ZZTwfHTONCwCqWZX7b4KQUU5Syyn5liQ/caDj64iRv0+KtMkobnL/QZc/0xWDs0J/miBmxhTL5Q1AMnWxeBFmKFJi5/dA6cGrXGGCDZv8C3Bvef5EhXOEgnXdcnOJBUB5anzTNE/3d0= Received: from SJ0PR03CA0002.namprd03.prod.outlook.com (2603:10b6:a03:33a::7) by DS0PR12MB7996.namprd12.prod.outlook.com (2603:10b6:8:14f::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6863.47; Mon, 16 Oct 2023 13:42:54 +0000 Received: from SN1PEPF000252A0.namprd05.prod.outlook.com (2603:10b6:a03:33a:cafe::7b) by SJ0PR03CA0002.outlook.office365.com (2603:10b6:a03:33a::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6886.35 via Frontend Transport; Mon, 16 Oct 2023 13:42:54 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by SN1PEPF000252A0.mail.protection.outlook.com (10.167.242.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6838.22 via Frontend Transport; Mon, 16 Oct 2023 13:42:54 +0000 Received: from localhost (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Mon, 16 Oct 2023 08:42:53 -0500 From: Michael Roth To: CC: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v10 39/50] KVM: SEV: Implement gmem hook for initializing private pages Date: Mon, 16 Oct 2023 08:28:08 -0500 Message-ID: <20231016132819.1002933-40-michael.roth@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231016132819.1002933-1-michael.roth@amd.com> References: <20231016132819.1002933-1-michael.roth@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN1PEPF000252A0:EE_|DS0PR12MB7996:EE_ X-MS-Office365-Filtering-Correlation-Id: 375ff03f-4abe-49b1-e727-08dbce4dcc87 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 5GQz8LNG1/Ev9LDgD6TSZ3CJ+aab9O+6w5+vH8VzT8JGbyD8D4/m6KJqWl1jyRwfmQBrXalpgbUZftP0ZfO0Gu+mo3TaMjq5flc9r2EW8bjMmgeW3y4B1oW/QnAcbAXpuyvVbT1cQddCmDo9KzdVuACD2l3cSQqMO8olDJvDf7N3lFLxoKuA4r+/nWJtRAK3Gp8UXD2GheH/hbzT+O8IbXMLYv6cjuzTI0sG7iDxfD+nq238ZmRUQfBxr5hRtr0gkWCcCVdbfKXh871Semk7avzAdnPce2OKVzPoFq8FWrzq4s5UNQGoRcMcnuVf2/cwtVZ2OJi754KimVLcb43OP8TTz2XpKKeIl4slRepqKpBgblzUBmkGxq3P0gefXsSMXQB7ZOMcSDl8k13xb1o9J3pVBMnYeEc5arso0zjabdyg3uGNbtAGegUNKdZLzFzKCrMJ7XGJzn5lKp1J0DWxTVdNvAlsQAPaXoO141s3cRHJstN15QJyUdknyApR2gTfX+UEDeczG327CC5G0mGioaFcNiBqeHId46GJUF+bWFgP/m5YOmrAJeHDvc/c4c/C55e9DTSY+ZHWVAStY/ZZYyHuuA54O9av2Om8JnQH9eEbNDCi514eMRgNqL2WHVN6ZPHFJz8LPy5k11VY/mVzP8psbwUnZI3mQgZWW5k6iO02DyyRKzSSKOX8m0l8mh/9CN8LN5NYz5VmzlUBTORen7AeBo2KSafjdlm2lGSDRfFSQygBsUJEYj1EVYgDDVvXr2fVJ+CRGA0vDlnVzXEFaQ== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(39860400002)(376002)(136003)(346002)(396003)(230922051799003)(64100799003)(451199024)(186009)(82310400011)(1800799009)(40470700004)(46966006)(36840700001)(40480700001)(40460700003)(6666004)(478600001)(336012)(47076005)(86362001)(36860700001)(81166007)(356005)(36756003)(82740400003)(7416002)(7406005)(2906002)(316002)(2616005)(83380400001)(16526019)(426003)(26005)(1076003)(5660300002)(44832011)(70206006)(4326008)(41300700001)(6916009)(54906003)(8676002)(8936002)(70586007)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Oct 2023 13:42:54.1538 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 375ff03f-4abe-49b1-e727-08dbce4dcc87 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SN1PEPF000252A0.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB7996 X-Rspamd-Queue-Id: 331391C002B X-Rspam-User: X-Stat-Signature: x8ri9jp1sc1job4hm7jjpaasjdybkcyg X-Rspamd-Server: rspam03 X-HE-Tag: 1697463777-765691 X-HE-Meta: U2FsdGVkX1/dEuUHrmUIbCHqS26TBSyhFQMhTzYkiJ4w/EQ3HtxTNI1JLxKzA5qSaz4SXZBKv60mIsZFUAifOfQkHDCPQMabDGkUWng2hx3LsOifEazyjeTWE092EtX4w6zm4tXUdRbL9/8+EjDyOQNUjyZLhSjh7a+5qmF0F+twIrkqCu7eF7c+TMVvqWIZb2Rh310Vcp4YLa2C6dCYysI+VytE7lwqiGWSx8CnYU++cHd50LcnK/ESTaczuyaAgaBOH0ruNUXZXoow5fQ9AOyioG45UTGjlGETTyz1lVT/D8gmj1g9QQ5gHTRFV15aQk4KeycO0Pu6hF6V25WLMoXs+oh9pQuR5Qh3Qyv0KhsGWSwJqVRqg3nPIHtwZxwQ8nu0Upo7gSkWSLSrgEz997i5H18ui0bhr+sRwV2d4+RZ+/liKKpVOZBg8n+M/jljnNznzV4CKwdowNerI1I/+BY3JdD4W7zG43POXyStQm7Hg/Q3x4ZSuY+tU5QtgK/i7wB8Xa3gpzQ9QF59FfVDdiop+MZvQzIM/PNTqRomp/EhvqOoKm4F+jH1atgXSRE9luqW6w90XGGLiZAzTTVJGfcZQk+c3WuuZbDv4P83NNfQaWfMm5sr92os5aD5epwshRFZUV/uCahR9nbY23raa3OTsVG37vTRJoMe3ZDEfaLX5MExzjpwflujQRKaVw/j8AlBevJ2nFx1n40oInPSd/Dv/6DtIRF59VKsd7SxfXqrk9mWLirZM/EizxrpWDheRGCVXdTF5fAChaZYbQHle8bV2veVO5Xz9faxvqRHqsA7eRVVVskF2eeQ2wMt80eGSeO9luf2Q14tGLHq6nf3OdaoYqDe6uielf5tsRBvEWuPXq2Tm7YavVeYsddGgpNwQY8HGXfOh1d+vCXHTDz+lPbujEkwn71UXwaaB+wcNVCAWa+7+4/lKPDA4MmFxLtlhxyoU4gyn1lwBrj9VW+ kuLs/BBC rCK+24q1pgOxH9TS6DuDhcdGlV9sNlQoOXPeUaieo/woHa0jCOncahZu1FSOmHl5JSgIR78VCDJ0gZ9FhPr0UVlL6FP2th9LxHG2aPmfGNqVavI2hKbIy1OUhxbHgqjC+zUCkBfcLPoeOi/INild18J4v+g62QW1lyvHUPr9RG8XcE6/NLTNS8xqQyFl6j98FNB6wflGMmKPrcAdpD4py8a2C1dBJWjTWPrWGn/qDtVU6/CAa1Y6K47AINPF2ovOdc01XG582JoUgOaxD/Jul+Hdf5pUrlT6OucJ+4w9XCBSkqkrekd/GkMdtnnGrxV4H8xPjT/v64XxF2aw= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This will handle RMP table updates and direct map changes needed to put a page into a private state before mapping it into an SEV-SNP guest. Signed-off-by: Michael Roth --- arch/x86/kvm/Kconfig | 1 + arch/x86/kvm/svm/sev.c | 95 ++++++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/svm/svm.c | 2 + arch/x86/kvm/svm/svm.h | 1 + 4 files changed, 99 insertions(+) diff --git a/arch/x86/kvm/Kconfig b/arch/x86/kvm/Kconfig index 71dc506aa3fb..8caf2eb6add8 100644 --- a/arch/x86/kvm/Kconfig +++ b/arch/x86/kvm/Kconfig @@ -127,6 +127,7 @@ config KVM_AMD_SEV depends on KVM_AMD && X86_64 depends on CRYPTO_DEV_SP_PSP && !(KVM_AMD=y && CRYPTO_DEV_CCP_DD=m) select KVM_SW_PROTECTED_VM + select HAVE_KVM_GMEM_PREPARE help Provides support for launching Encrypted VMs (SEV) and Encrypted VMs with Encrypted State (SEV-ES) on AMD processors. diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c index 9c38fe796e00..8cf2d19597b1 100644 --- a/arch/x86/kvm/svm/sev.c +++ b/arch/x86/kvm/svm/sev.c @@ -4135,3 +4135,98 @@ void handle_rmp_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u64 error_code) out: put_page(pfn_to_page(pfn)); } + +static bool is_pfn_range_shared(kvm_pfn_t start, kvm_pfn_t end) +{ + kvm_pfn_t pfn = start; + + while (pfn < end) { + int ret, rmp_level; + bool assigned; + + ret = snp_lookup_rmpentry(pfn, &assigned, &rmp_level); + if (ret) { + pr_warn_ratelimited("SEV: Failed to retrieve RMP entry: PFN 0x%llx GFN start 0x%llx GFN end 0x%llx RMP level %d error %d\n", + pfn, start, end, rmp_level, ret); + return false; + } + + if (assigned) { + pr_debug("%s: overlap detected, PFN 0x%llx start 0x%llx end 0x%llx RMP level %d\n", + __func__, pfn, start, end, rmp_level); + return false; + } + + pfn++; + } + + return true; +} + +static u8 max_level_for_order(int order) +{ + if (order >= KVM_HPAGE_GFN_SHIFT(PG_LEVEL_2M)) + return PG_LEVEL_2M; + + return PG_LEVEL_4K; +} + +static bool is_large_rmp_possible(struct kvm *kvm, kvm_pfn_t pfn, int order) +{ + kvm_pfn_t pfn_aligned = ALIGN_DOWN(pfn, PTRS_PER_PMD); + + /* + * If this is a large folio, and the entire 2M range containing the + * PFN is currently shared, then the entire 2M-aligned range can be + * set to private via a single 2M RMP entry. + */ + if (max_level_for_order(order) > PG_LEVEL_4K && + is_pfn_range_shared(pfn_aligned, pfn_aligned + PTRS_PER_PMD)) + return true; + + return false; +} + +int sev_gmem_prepare(struct kvm *kvm, kvm_pfn_t pfn, gfn_t gfn, int max_order) +{ + struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info; + kvm_pfn_t pfn_aligned; + gfn_t gfn_aligned; + int level, rc; + bool assigned; + + if (!sev_snp_guest(kvm)) + return 0; + + rc = snp_lookup_rmpentry(pfn, &assigned, &level); + if (rc) + return rc; + + if (assigned) { + pr_debug("%s: already assigned: gfn %llx pfn %llx max_order %d level %d\n", + __func__, gfn, pfn, max_order, level); + return 0; + } + + if (is_large_rmp_possible(kvm, pfn, max_order)) { + level = PG_LEVEL_2M; + pfn_aligned = ALIGN_DOWN(pfn, PTRS_PER_PMD); + gfn_aligned = ALIGN_DOWN(gfn, PTRS_PER_PMD); + } else { + level = PG_LEVEL_4K; + pfn_aligned = pfn; + gfn_aligned = gfn; + } + + rc = rmp_make_private(pfn_aligned, gfn_to_gpa(gfn_aligned), level, sev->asid, false); + if (rc) { + pr_err_ratelimited("SEV: Failed to update RMP entry: GFN %llx PFN %llx level %d error %d\n", + gfn, pfn, level, rc); + return -EINVAL; + } + + pr_debug("%s: updated: gfn %llx pfn %llx pfn_aligned %llx max_order %d level %d\n", + __func__, gfn, pfn, pfn_aligned, max_order, level); + + return 0; +} diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index f5cdcbd1ba67..b3ed424533b0 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -5041,6 +5041,8 @@ static struct kvm_x86_ops svm_x86_ops __initdata = { .vcpu_deliver_sipi_vector = svm_vcpu_deliver_sipi_vector, .vcpu_get_apicv_inhibit_reasons = avic_vcpu_get_apicv_inhibit_reasons, .alloc_apic_backing_page = svm_alloc_apic_backing_page, + + .gmem_prepare = sev_gmem_prepare, }; /* diff --git a/arch/x86/kvm/svm/svm.h b/arch/x86/kvm/svm/svm.h index f81dfa1594f6..c5cee554176e 100644 --- a/arch/x86/kvm/svm/svm.h +++ b/arch/x86/kvm/svm/svm.h @@ -723,6 +723,7 @@ void sev_es_unmap_ghcb(struct vcpu_svm *svm); struct page *snp_safe_alloc_page(struct kvm_vcpu *vcpu); void handle_rmp_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u64 error_code); void sev_snp_init_protected_guest_state(struct kvm_vcpu *vcpu); +int sev_gmem_prepare(struct kvm *kvm, kvm_pfn_t pfn, gfn_t gfn, int max_order); /* vmenter.S */