From patchwork Thu Apr 18 19:41:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Roth X-Patchwork-Id: 13635309 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3887FC4345F for ; Thu, 18 Apr 2024 19:45:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BFC236B00A4; Thu, 18 Apr 2024 15:45:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BAC1E6B00A5; Thu, 18 Apr 2024 15:45:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A26616B00A6; Thu, 18 Apr 2024 15:45:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 7F6E46B00A4 for ; Thu, 18 Apr 2024 15:45:04 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 3A548C010F for ; Thu, 18 Apr 2024 19:45:04 +0000 (UTC) X-FDA: 82023680928.06.E85F52B Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2043.outbound.protection.outlook.com [40.107.244.43]) by imf16.hostedemail.com (Postfix) with ESMTP id 26F6518000D for ; Thu, 18 Apr 2024 19:45:00 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=amd.com header.s=selector1 header.b=XPJt6iaj; dmarc=pass (policy=quarantine) header.from=amd.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf16.hostedemail.com: domain of Michael.Roth@amd.com designates 40.107.244.43 as permitted sender) smtp.mailfrom=Michael.Roth@amd.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1713469501; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=htzsV/Mg0iJ9hTVHmM5bTePVsoIp+wHnVfb6RMUK+PQ=; b=FdxYWEKq4BMrLe9DCE4CrmabhPbZlc84xoyDowEbayTrz8Rd+Dh+t5sS+n2sE7dbNLKNUt rvPCrl+jnd20x9ElkmY8J7pBKsY71Fsih9V0llrJE2TimUc/BmSxltwd4pkQQgrejgkPsy AVyiQKB7uf2MVeUyPQK8X7hi1QMjsLo= ARC-Authentication-Results: i=2; imf16.hostedemail.com; dkim=pass header.d=amd.com header.s=selector1 header.b=XPJt6iaj; dmarc=pass (policy=quarantine) header.from=amd.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf16.hostedemail.com: domain of Michael.Roth@amd.com designates 40.107.244.43 as permitted sender) smtp.mailfrom=Michael.Roth@amd.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1713469501; a=rsa-sha256; cv=pass; b=rSpmHkCUyUSaVl765JMDU5S/2y364HLP9tziQOTAPF6vVS+gTY58zgQ8t6UMMbrmqHVZcA 1229abhKMCHxFnUc23T/TvpKterFEq4puzS3jg8UCR8TLmOhy4QSGRNt10B0H5l+ZHLuj9 hcQRSPs41bnPs606aHcO2sSC3jAekoY= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Kz64JbjuQJlsfTgXnRuqhX9ZI+4KkoU89OFcXzM4dO6NN0sQFw29NiojIXm2EePL60mm5VejMe1ac4KFOhAarmRGtXQzFOUeRKXP/lf7OXvBtdkCeLhFmdLUQrlhGS3HT8/sRflkC2zVkjjKwJAJAnSaj59xeI/7v/5nrRYHJWjVmG8Kg0Pm4MkDL42a8M1F1YcS8ikzLYJP8elN+bj5u1smDoHmjX7EgTycapyC7RPDmXFUpeHLNV1QHTxwAdzG7JJ4YPCoxmiHGytbCcJnZRdeTjrrET/vTooiXFKfKNO+NY3d+MPz14yXkP+RHSEoM5H8eFdJBqZozlEz3DDdsw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=htzsV/Mg0iJ9hTVHmM5bTePVsoIp+wHnVfb6RMUK+PQ=; b=PCZL6CZIA1mxjByTQyqCzXpp34y1lFqa5nxlDbDlpi+uo3x8u87qYTXDP5cCwelwbRMxHfYyj6dJ/7FtTgB0GttcGOT9IFrNHKTNs9nXlaVu24XFBC7acw4DgKD3w7y2LXGNdeNAdwmC5oLV6YhF7735g35nQCYs16h9OsE1w35W4LHQBQMGFzdHDc106DZmoW34ru67ATTMY/CI1xX52cRwYbZkI2QJpzgoYJgRaFNAWLtjAAKoJvKUgpn/7cAcF9crk2fZNw7IkG6pecnY3th152FAQ0em3Cfs7OLs9Y0JNiInn3WOElq9BQBE8YPZZ8YDIMpucc6fxh9BynM8zw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=htzsV/Mg0iJ9hTVHmM5bTePVsoIp+wHnVfb6RMUK+PQ=; b=XPJt6iaj+48d34SF5T7ezXyoG+sj1A9oWeVAbEkh4Qwvz73qstNHyJGN4GFtcaLWBIKY78xBc70b6rA3zR0f/WGapMKrCzCjxtBSz+HjxIO4Y/Oim+puIvM4+go+Z+O8bwSndrZNfkrPtZ1W65ypWugL4216W/KCJClcBgQyulQ= Received: from BYAPR03CA0032.namprd03.prod.outlook.com (2603:10b6:a02:a8::45) by PH7PR12MB5853.namprd12.prod.outlook.com (2603:10b6:510:1d4::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7472.42; Thu, 18 Apr 2024 19:44:54 +0000 Received: from SJ5PEPF000001CC.namprd05.prod.outlook.com (2603:10b6:a02:a8:cafe::ca) by BYAPR03CA0032.outlook.office365.com (2603:10b6:a02:a8::45) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7495.26 via Frontend Transport; Thu, 18 Apr 2024 19:44:54 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by SJ5PEPF000001CC.mail.protection.outlook.com (10.167.242.41) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7452.22 via Frontend Transport; Thu, 18 Apr 2024 19:44:54 +0000 Received: from localhost (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 18 Apr 2024 14:44:53 -0500 From: Michael Roth To: CC: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Brijesh Singh Subject: [PATCH v13 16/26] KVM: SEV: Support SEV-SNP AP Creation NAE event Date: Thu, 18 Apr 2024 14:41:23 -0500 Message-ID: <20240418194133.1452059-17-michael.roth@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240418194133.1452059-1-michael.roth@amd.com> References: <20240418194133.1452059-1-michael.roth@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ5PEPF000001CC:EE_|PH7PR12MB5853:EE_ X-MS-Office365-Filtering-Correlation-Id: bb92df21-b20b-4e6e-ebf2-08dc5fe00570 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 3yHDRRtl8++G4HEMByQCiMDJ0Cw8i3cIUF+t9Z07rgkzeNjfCzABb9+lrqJX7GdRbHZGBrT6pOmS9OouJscLQKCcw/+ng1PFjqPiGSTYGEe/+Vt+NfYTLd5N8psvFkmJLV4shNfP+I+nQ78NYNBJ1HzYWr2Rqr6oE9CuEhuMNhjMRP0azQynf5U5qzWuGdYWItG/wEL8scRO8xIpieSiagnqmwqPMS97lEKu8Bk76z76wcmRh6wM7w44b8oeinu7HQbVNBm4jDcfOKZFT/f/g1JNwB1YIYz3PiDZpXtHoYnHDWSqg95uKeNiN+Jktjn/BjXQcr5/pChHJQN7l707/31Xvz2aevBMJxVvDf+xFKbDqeXOWfaji24L/1UolnXf7d7Px0NpJwebY6seC+w/6CLnZCIy7Jxg5PXInQqQDxPo6tZn2r/kx0fe3Gq7b5z6jWcS+GzTzX6yG2ugLLKtO1DE0S0/kRgTwm/PgQM5a9dp/PZzuoCcaCUVwdqaDU4/5F1Go+hdWjVOUDuHtb/kgFfNuOa7ZNLfuQs0jm5roXcEYu/vG3zobwGDYuO2yn5nwTvub2Z2WlxeydAEOT/ER3i/ImjO49GToPKaYYuGPilCtVQS/noPaELt3spbLax9ZL6FgRt4hkCuf2JfuJ5RrbCGIv8X9sxEqMconPk21AyQPvvOSQ1i9Xq6ZGW74lYNR0KYv2+iBx+eGgP0TCTEPUYMtJIG0ATroHiSufIa6GyQVVLGRqbpqryjeBAt4ohz X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(7416005)(1800799015)(82310400014)(376005);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Apr 2024 19:44:54.6961 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: bb92df21-b20b-4e6e-ebf2-08dc5fe00570 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SJ5PEPF000001CC.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB5853 X-Rspam-User: X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 26F6518000D X-Stat-Signature: b6j976yfk93ey7tw35nsryi6hekszier X-HE-Tag: 1713469500-990500 X-HE-Meta: U2FsdGVkX18AC8ufJxy6VGvmz/c+agZgE9iia0c3ZRs9VSoVlEzZiJc2SooHu/hmBT5Uh226pgcH7jafMaTKf+nE82twm6NRJ6rvIUh2HkvPDc0ECiDZD/avm7IBq/2CcCOGvsvJQfC4+fqXslPb9rHDv+3UxAGsQKIvv1ScdtfLk9SLmlanwDWSY22aNLuEhybAUQbjxcQcCIU94BLQ/WS6PSxFYPAhJzlCE6Dc4i3gELPKxXkZ5CweCNqnmu2Hf6NaTx1BFV+q83uqQ0YvEXiyCytobrxiyQ4hi0eQi7LiGe0YTCgiqgv1i3oEMI9GTKL6caG2ztMppXt4/Ou+DL258x70dcFTL5B8J8cs92I39AicArLltaWtCsMdz3P5QRTpcCz3BeNod/LlUJWYSlvI+xZNcl+6avYYFptidI+hDqsQ7sEeAiEdhO8eYJTQQUmmeVB96PnR/ylqJJYYFp6uG6wlcU+arHa8jMb876NXl6DWplvKK1bxrU79MlWvDcbk2QhZA+xB2M/aDE0Jpc0tGBwK+aVFqyEZAVnyetcuigDwG+WKDjoDtw7+WHuOeIVHED9oan5oVPvOkFvo7yFuXiDXJ1PgqCPo3R1yujOOCA+GqUFA8zJ+1Wi5bXM6gcISLE3O7qBQ1ltzgSs+WLJ6536TDhzYE2C8DiviZ8wzq8LZJam0IQ0/WMZEocScH794j66fOpT73NZdGqQAmWCM9nkgb+15ufbgyGm5X+JRq3g8DAtSKgPiscvkSkX2tbq3KEQJVp9IfrXy9Z281LlD3/e2tJPNxm//pvwQHBumvAA3XmYm0ta3Hx8vZXp+EYNixfhqDwyeRh9UMD2DaZfSb2I4o/DOH1fTiJywlz+1KvRCtQ0igoSuAerH2vmLXOQaTxeellPocCALZ/bIpI1A14aLGIV0256psYHV9qV1lCCYqeKr3DyOzvW0uE2HahY4UulJgFCTclQEwhW 3BnFd/wu dhnOsyzQk1tOBg6TWa2k4cQoUKjU35lpyBLcTPWnpxXEfE21EuVsU61InnYSoukd9IfA5xUHTZeX1uAt736G1tvg+D/VPQ8v1y/NwkGdM/aPdGVBX4W6oR2jGla9R1juzVDax8sHwTzqwVOAQ2Vej2RjFFFQ1TrxpcyrMVHqiR5ztkxwpbAnmBgGoMGGCTb0TUPRKIzRKTieIaywrlQEYZKtt39HvhBq0CfTV0T7248Tf3LhM3oL4Hy54w2n0KcEFFgv61CJp7Lzu0XXmbAVF3ACiriUJl51inSdFIezpn1VzFsqinJ+X90BLy3xKEKXK91w5beF/tU0+6J8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Tom Lendacky Add support for the SEV-SNP AP Creation NAE event. This allows SEV-SNP guests to alter the register state of the APs on their own. This allows the guest a way of simulating INIT-SIPI. A new event, KVM_REQ_UPDATE_PROTECTED_GUEST_STATE, is created and used so as to avoid updating the VMSA pointer while the vCPU is running. For CREATE The guest supplies the GPA of the VMSA to be used for the vCPU with the specified APIC ID. The GPA is saved in the svm struct of the target vCPU, the KVM_REQ_UPDATE_PROTECTED_GUEST_STATE event is added to the vCPU and then the vCPU is kicked. For CREATE_ON_INIT: The guest supplies the GPA of the VMSA to be used for the vCPU with the specified APIC ID the next time an INIT is performed. The GPA is saved in the svm struct of the target vCPU. For DESTROY: The guest indicates it wishes to stop the vCPU. The GPA is cleared from the svm struct, the KVM_REQ_UPDATE_PROTECTED_GUEST_STATE event is added to vCPU and then the vCPU is kicked. The KVM_REQ_UPDATE_PROTECTED_GUEST_STATE event handler will be invoked as a result of the event or as a result of an INIT. If a new VMSA is to be installed, the VMSA guest page is set as the VMSA in the vCPU VMCB and the vCPU state is set to KVM_MP_STATE_RUNNABLE. If a new VMSA is not to be installed, the VMSA is cleared in the vCPU VMCB and the vCPU state is set to KVM_MP_STATE_HALTED to prevent it from being run. Signed-off-by: Tom Lendacky Co-developed-by: Michael Roth Signed-off-by: Michael Roth Signed-off-by: Brijesh Singh Signed-off-by: Ashish Kalra --- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/include/asm/svm.h | 6 + arch/x86/kvm/svm/sev.c | 229 +++++++++++++++++++++++++++++++- arch/x86/kvm/svm/svm.c | 11 +- arch/x86/kvm/svm/svm.h | 9 ++ arch/x86/kvm/x86.c | 11 ++ 6 files changed, 264 insertions(+), 3 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 6f03e7649780..9943e989fadb 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -121,6 +121,7 @@ KVM_ARCH_REQ_FLAGS(31, KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP) #define KVM_REQ_HV_TLB_FLUSH \ KVM_ARCH_REQ_FLAGS(32, KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP) +#define KVM_REQ_UPDATE_PROTECTED_GUEST_STATE KVM_ARCH_REQ(34) #define CR0_RESERVED_BITS \ (~(unsigned long)(X86_CR0_PE | X86_CR0_MP | X86_CR0_EM | X86_CR0_TS \ diff --git a/arch/x86/include/asm/svm.h b/arch/x86/include/asm/svm.h index 544a43c1cf11..f0dea3750ca9 100644 --- a/arch/x86/include/asm/svm.h +++ b/arch/x86/include/asm/svm.h @@ -286,8 +286,14 @@ static_assert((X2AVIC_MAX_PHYSICAL_ID & AVIC_PHYSICAL_MAX_INDEX_MASK) == X2AVIC_ #define AVIC_HPA_MASK ~((0xFFFULL << 52) | 0xFFF) #define SVM_SEV_FEAT_SNP_ACTIVE BIT(0) +#define SVM_SEV_FEAT_RESTRICTED_INJECTION BIT(3) +#define SVM_SEV_FEAT_ALTERNATE_INJECTION BIT(4) #define SVM_SEV_FEAT_DEBUG_SWAP BIT(5) +#define SVM_SEV_FEAT_INT_INJ_MODES \ + (SVM_SEV_FEAT_RESTRICTED_INJECTION | \ + SVM_SEV_FEAT_ALTERNATE_INJECTION) + struct vmcb_seg { u16 selector; u16 attrib; diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c index 0f70b057bfb8..2de3006fec65 100644 --- a/arch/x86/kvm/svm/sev.c +++ b/arch/x86/kvm/svm/sev.c @@ -37,7 +37,7 @@ #define GHCB_VERSION_MAX 2ULL #define GHCB_VERSION_MIN 1ULL -#define GHCB_HV_FT_SUPPORTED GHCB_HV_FT_SNP +#define GHCB_HV_FT_SUPPORTED (GHCB_HV_FT_SNP | GHCB_HV_FT_SNP_AP_CREATION) /* enable/disable SEV support */ static bool sev_enabled = true; @@ -3261,6 +3261,11 @@ static int sev_es_validate_vmgexit(struct vcpu_svm *svm) if (!kvm_ghcb_sw_scratch_is_valid(svm)) goto vmgexit_err; break; + case SVM_VMGEXIT_AP_CREATION: + if (lower_32_bits(control->exit_info_1) != SVM_VMGEXIT_AP_DESTROY) + if (!kvm_ghcb_rax_is_valid(svm)) + goto vmgexit_err; + break; case SVM_VMGEXIT_NMI_COMPLETE: case SVM_VMGEXIT_AP_HLT_LOOP: case SVM_VMGEXIT_AP_JUMP_TABLE: @@ -3511,6 +3516,205 @@ static int snp_complete_psc(struct kvm_vcpu *vcpu) return 1; /* resume guest */ } +static int __sev_snp_update_protected_guest_state(struct kvm_vcpu *vcpu) +{ + struct vcpu_svm *svm = to_svm(vcpu); + + WARN_ON(!mutex_is_locked(&svm->sev_es.snp_vmsa_mutex)); + + /* Mark the vCPU as offline and not runnable */ + vcpu->arch.pv.pv_unhalted = false; + vcpu->arch.mp_state = KVM_MP_STATE_HALTED; + + /* Clear use of the VMSA */ + svm->vmcb->control.vmsa_pa = INVALID_PAGE; + + if (VALID_PAGE(svm->sev_es.snp_vmsa_gpa)) { + gfn_t gfn = gpa_to_gfn(svm->sev_es.snp_vmsa_gpa); + struct kvm_memory_slot *slot; + kvm_pfn_t pfn; + + slot = gfn_to_memslot(vcpu->kvm, gfn); + if (!slot) + return -EINVAL; + + /* + * The new VMSA will be private memory guest memory, so + * retrieve the PFN from the gmem backend. + */ + if (kvm_gmem_get_pfn(vcpu->kvm, slot, gfn, &pfn, NULL)) + return -EINVAL; + + /* + * From this point forward, the VMSA will always be a + * guest-mapped page rather than the initial one allocated + * by KVM in svm->sev_es.vmsa. In theory, svm->sev_es.vmsa + * could be free'd and cleaned up here, but that involves + * cleanups like wbinvd_on_all_cpus() which would ideally + * be handled during teardown rather than guest boot. + * Deferring that also allows the existing logic for SEV-ES + * VMSAs to be re-used with minimal SNP-specific changes. + */ + svm->sev_es.snp_has_guest_vmsa = true; + + /* Use the new VMSA */ + svm->vmcb->control.vmsa_pa = pfn_to_hpa(pfn); + + /* Mark the vCPU as runnable */ + vcpu->arch.pv.pv_unhalted = false; + vcpu->arch.mp_state = KVM_MP_STATE_RUNNABLE; + + svm->sev_es.snp_vmsa_gpa = INVALID_PAGE; + + /* + * gmem pages aren't currently migratable, but if this ever + * changes then care should be taken to ensure + * svm->sev_es.vmsa is pinned through some other means. + */ + kvm_release_pfn_clean(pfn); + } + + /* + * When replacing the VMSA during SEV-SNP AP creation, + * mark the VMCB dirty so that full state is always reloaded. + */ + vmcb_mark_all_dirty(svm->vmcb); + + return 0; +} + +/* + * Invoked as part of svm_vcpu_reset() processing of an init event. + */ +void sev_snp_init_protected_guest_state(struct kvm_vcpu *vcpu) +{ + struct vcpu_svm *svm = to_svm(vcpu); + int ret; + + if (!sev_snp_guest(vcpu->kvm)) + return; + + mutex_lock(&svm->sev_es.snp_vmsa_mutex); + + if (!svm->sev_es.snp_ap_waiting_for_reset) + goto unlock; + + svm->sev_es.snp_ap_waiting_for_reset = false; + + ret = __sev_snp_update_protected_guest_state(vcpu); + if (ret) + vcpu_unimpl(vcpu, "snp: AP state update on init failed\n"); + +unlock: + mutex_unlock(&svm->sev_es.snp_vmsa_mutex); +} + +static int sev_snp_ap_creation(struct vcpu_svm *svm) +{ + struct kvm_sev_info *sev = &to_kvm_svm(svm->vcpu.kvm)->sev_info; + struct kvm_vcpu *vcpu = &svm->vcpu; + struct kvm_vcpu *target_vcpu; + struct vcpu_svm *target_svm; + unsigned int request; + unsigned int apic_id; + bool kick; + int ret; + + request = lower_32_bits(svm->vmcb->control.exit_info_1); + apic_id = upper_32_bits(svm->vmcb->control.exit_info_1); + + /* Validate the APIC ID */ + target_vcpu = kvm_get_vcpu_by_id(vcpu->kvm, apic_id); + if (!target_vcpu) { + vcpu_unimpl(vcpu, "vmgexit: invalid AP APIC ID [%#x] from guest\n", + apic_id); + return -EINVAL; + } + + ret = 0; + + target_svm = to_svm(target_vcpu); + + /* + * The target vCPU is valid, so the vCPU will be kicked unless the + * request is for CREATE_ON_INIT. For any errors at this stage, the + * kick will place the vCPU in an non-runnable state. + */ + kick = true; + + mutex_lock(&target_svm->sev_es.snp_vmsa_mutex); + + target_svm->sev_es.snp_vmsa_gpa = INVALID_PAGE; + target_svm->sev_es.snp_ap_waiting_for_reset = true; + + /* Interrupt injection mode shouldn't change for AP creation */ + if (request < SVM_VMGEXIT_AP_DESTROY) { + u64 sev_features; + + sev_features = vcpu->arch.regs[VCPU_REGS_RAX]; + sev_features ^= sev->vmsa_features; + + if (sev_features & SVM_SEV_FEAT_INT_INJ_MODES) { + vcpu_unimpl(vcpu, "vmgexit: invalid AP injection mode [%#lx] from guest\n", + vcpu->arch.regs[VCPU_REGS_RAX]); + ret = -EINVAL; + goto out; + } + } + + switch (request) { + case SVM_VMGEXIT_AP_CREATE_ON_INIT: + kick = false; + fallthrough; + case SVM_VMGEXIT_AP_CREATE: + if (!page_address_valid(vcpu, svm->vmcb->control.exit_info_2)) { + vcpu_unimpl(vcpu, "vmgexit: invalid AP VMSA address [%#llx] from guest\n", + svm->vmcb->control.exit_info_2); + ret = -EINVAL; + goto out; + } + + /* + * Malicious guest can RMPADJUST a large page into VMSA which + * will hit the SNP erratum where the CPU will incorrectly signal + * an RMP violation #PF if a hugepage collides with the RMP entry + * of VMSA page, reject the AP CREATE request if VMSA address from + * guest is 2M aligned. + */ + if (IS_ALIGNED(svm->vmcb->control.exit_info_2, PMD_SIZE)) { + vcpu_unimpl(vcpu, + "vmgexit: AP VMSA address [%llx] from guest is unsafe as it is 2M aligned\n", + svm->vmcb->control.exit_info_2); + ret = -EINVAL; + goto out; + } + + target_svm->sev_es.snp_vmsa_gpa = svm->vmcb->control.exit_info_2; + break; + case SVM_VMGEXIT_AP_DESTROY: + break; + default: + vcpu_unimpl(vcpu, "vmgexit: invalid AP creation request [%#x] from guest\n", + request); + ret = -EINVAL; + break; + } + +out: + if (kick) { + kvm_make_request(KVM_REQ_UPDATE_PROTECTED_GUEST_STATE, target_vcpu); + + if (target_vcpu->arch.mp_state == KVM_MP_STATE_UNINITIALIZED) + kvm_make_request(KVM_REQ_UNBLOCK, target_vcpu); + + kvm_vcpu_kick(target_vcpu); + } + + mutex_unlock(&target_svm->sev_es.snp_vmsa_mutex); + + return ret; +} + static int sev_handle_vmgexit_msr_protocol(struct vcpu_svm *svm) { struct vmcb_control_area *control = &svm->vmcb->control; @@ -3754,6 +3958,15 @@ int sev_handle_vmgexit(struct kvm_vcpu *vcpu) vcpu->run->vmgexit.psc.shared_gpa = svm->sev_es.sw_scratch; vcpu->arch.complete_userspace_io = snp_complete_psc; break; + case SVM_VMGEXIT_AP_CREATION: + ret = sev_snp_ap_creation(svm); + if (ret) { + ghcb_set_sw_exit_info_1(svm->sev_es.ghcb, 2); + ghcb_set_sw_exit_info_2(svm->sev_es.ghcb, GHCB_ERR_INVALID_INPUT); + } + + ret = 1; + break; case SVM_VMGEXIT_UNSUPPORTED_EVENT: vcpu_unimpl(vcpu, "vmgexit: unsupported event - exit_info_1=%#llx, exit_info_2=%#llx\n", @@ -3848,7 +4061,7 @@ static void sev_es_init_vmcb(struct vcpu_svm *svm) * the VMSA will be NULL if this vCPU is the destination for intrahost * migration, and will be copied later. */ - if (svm->sev_es.vmsa) + if (!svm->sev_es.snp_has_guest_vmsa) svm->vmcb->control.vmsa_pa = __pa(svm->sev_es.vmsa); /* Can't intercept CR register access, HV can't modify CR registers */ @@ -3921,6 +4134,8 @@ void sev_es_vcpu_reset(struct vcpu_svm *svm) set_ghcb_msr(svm, GHCB_MSR_SEV_INFO(GHCB_VERSION_MAX, GHCB_VERSION_MIN, sev_enc_bit)); + + mutex_init(&svm->sev_es.snp_vmsa_mutex); } void sev_es_prepare_switch_to_guest(struct vcpu_svm *svm, struct sev_es_save_area *hostsa) @@ -4032,6 +4247,16 @@ struct page *snp_safe_alloc_page(struct kvm_vcpu *vcpu) return p; } +void sev_vcpu_unblocking(struct kvm_vcpu *vcpu) +{ + if (!sev_snp_guest(vcpu->kvm)) + return; + + if (kvm_test_request(KVM_REQ_UPDATE_PROTECTED_GUEST_STATE, vcpu) && + vcpu->arch.mp_state == KVM_MP_STATE_UNINITIALIZED) + vcpu->arch.mp_state = KVM_MP_STATE_RUNNABLE; +} + void sev_handle_rmp_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u64 error_code) { struct kvm_memory_slot *slot; diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index 1cddf7a2aec1..9dc929316c5d 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -1398,6 +1398,9 @@ static void svm_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) svm->spec_ctrl = 0; svm->virt_spec_ctrl = 0; + if (init_event) + sev_snp_init_protected_guest_state(vcpu); + init_vmcb(vcpu); if (!init_event) @@ -4939,6 +4942,12 @@ static void *svm_alloc_apic_backing_page(struct kvm_vcpu *vcpu) return page_address(page); } +static void svm_vcpu_unblocking(struct kvm_vcpu *vcpu) +{ + sev_vcpu_unblocking(vcpu); + avic_vcpu_unblocking(vcpu); +} + static struct kvm_x86_ops svm_x86_ops __initdata = { .name = KBUILD_MODNAME, @@ -4961,7 +4970,7 @@ static struct kvm_x86_ops svm_x86_ops __initdata = { .vcpu_load = svm_vcpu_load, .vcpu_put = svm_vcpu_put, .vcpu_blocking = avic_vcpu_blocking, - .vcpu_unblocking = avic_vcpu_unblocking, + .vcpu_unblocking = svm_vcpu_unblocking, .update_exception_bitmap = svm_update_exception_bitmap, .get_msr_feature = svm_get_msr_feature, diff --git a/arch/x86/kvm/svm/svm.h b/arch/x86/kvm/svm/svm.h index d2b0ec27d4fe..81e335dca281 100644 --- a/arch/x86/kvm/svm/svm.h +++ b/arch/x86/kvm/svm/svm.h @@ -210,6 +210,11 @@ struct vcpu_sev_es_state { bool ghcb_sa_free; u64 ghcb_registered_gpa; + + struct mutex snp_vmsa_mutex; /* Used to handle concurrent updates of VMSA. */ + gpa_t snp_vmsa_gpa; + bool snp_ap_waiting_for_reset; + bool snp_has_guest_vmsa; }; struct vcpu_svm { @@ -723,6 +728,8 @@ int sev_cpu_init(struct svm_cpu_data *sd); int sev_dev_get_attr(u32 group, u64 attr, u64 *val); extern unsigned int max_sev_asid; void sev_handle_rmp_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u64 error_code); +void sev_vcpu_unblocking(struct kvm_vcpu *vcpu); +void sev_snp_init_protected_guest_state(struct kvm_vcpu *vcpu); #else static inline struct page *snp_safe_alloc_page(struct kvm_vcpu *vcpu) { return alloc_page(GFP_KERNEL_ACCOUNT | __GFP_ZERO); @@ -737,6 +744,8 @@ static inline int sev_cpu_init(struct svm_cpu_data *sd) { return 0; } static inline int sev_dev_get_attr(u32 group, u64 attr, u64 *val) { return -ENXIO; } #define max_sev_asid 0 static inline void sev_handle_rmp_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u64 error_code) {} +static inline void sev_vcpu_unblocking(struct kvm_vcpu *vcpu) {} +static inline void sev_snp_init_protected_guest_state(struct kvm_vcpu *vcpu) {} #endif diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index a9d014961d2b..436078b9e5aa 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -10938,6 +10938,14 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) if (kvm_check_request(KVM_REQ_UPDATE_CPU_DIRTY_LOGGING, vcpu)) static_call(kvm_x86_update_cpu_dirty_logging)(vcpu); + + if (kvm_check_request(KVM_REQ_UPDATE_PROTECTED_GUEST_STATE, vcpu)) { + kvm_vcpu_reset(vcpu, true); + if (vcpu->arch.mp_state != KVM_MP_STATE_RUNNABLE) { + r = 1; + goto out; + } + } } if (kvm_check_request(KVM_REQ_EVENT, vcpu) || req_int_win || @@ -13145,6 +13153,9 @@ static inline bool kvm_vcpu_has_events(struct kvm_vcpu *vcpu) if (kvm_test_request(KVM_REQ_PMI, vcpu)) return true; + if (kvm_test_request(KVM_REQ_UPDATE_PROTECTED_GUEST_STATE, vcpu)) + return true; + if (kvm_arch_interrupt_allowed(vcpu) && (kvm_cpu_has_interrupt(vcpu) || kvm_guest_apic_has_interrupt(vcpu)))