From patchwork Tue May 16 14:15:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mostafa Saleh X-Patchwork-Id: 13243259 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 58E23C77B7F for ; Tue, 16 May 2023 15:30:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: Mime-Version:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To: References:List-Owner; bh=daZlcMDDj75GVqoPLdUjJWA76/V6ugexBtSxZdlPHlk=; b=OEW vdYpPQhRkzbxe4CRTAuEc3NQe+luUFeYh7B1mb1XpK7Way8dBpFuET9lx0qnzIewkDAgVOCas+YoH GmjrBug2Ysz6JWVQmXNIKkHtF0fwP/Z5ZZa9jHPcEhTuW5DK+7kjzitHJoR3aVBvDdgQAyhBGn/dl xd4yH9MfD6V70LDmfy4ChYHAyF4629cVF9YOwy2CnC1xrFyEvZSfxV9LOM/RD+HgsoaUbPteT7Qqv BvXTzI3plzgimxwVXA3ecPjifBMJqou8Ek8pSN/Z7ZHK2ouCziOFPPCk2kTWqWOs/uTMtEDoRKo1m YE0eI5FxRYpUbWnCebeTAQ6e8K2fnrQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1pywd8-006Fgp-05; Tue, 16 May 2023 15:30:26 +0000 Received: from mail-wm1-x34a.google.com ([2a00:1450:4864:20::34a]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1pyvT7-006354-0b for linux-arm-kernel@lists.infradead.org; Tue, 16 May 2023 14:16:02 +0000 Received: by mail-wm1-x34a.google.com with SMTP id 5b1f17b1804b1-3f433a2308bso21198035e9.0 for ; Tue, 16 May 2023 07:15:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1684246558; x=1686838558; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=B8N8PO1EikxzfdoHKZo5krI5/XOld4YdAWQeBGx5Jjk=; b=Wjo2awxanNESfU3T9n6puAeXqkMRNhZcXHTT6lsKWKqEZypv39qXzz0uLJr+HAgoZp SQelJrcAgx6xycEt86ZmfxjKaKldVb0NQ93xNCBM0qL8DubtX2CQCtpV+s+3Gl+z8ZKY +0E5yRWGqGmX7eQrA2HNLEumgmK5wYPh0BP26DqviRYDH7j9MyaQJbx1FUsBQLRXKdDn V2Nj1RjtPTCKkD3YK3sUFNcVQcKEdhbwUwo2pwyPaP/pYHq0LZnWwVOMaHyLebHA87J8 lY0VjsC4Nd/4qDmL9l4g6cAh74FJkapsw+G3vHztTDBPun+NpxlWiHqEBuxmlxmaN1gO asYQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684246558; x=1686838558; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=B8N8PO1EikxzfdoHKZo5krI5/XOld4YdAWQeBGx5Jjk=; b=gg+/KqshaRA5gsgDDh3KkG5HTQ++7lFFfDsXhYjUQhAVVb+Bt358rIzEB142/FY9US /F5jMsRyUkp1WrZgd95SETRUygnRwxCIi87Fz62Ek4ljVIW3DDqwmrs+2d0PuZHy8GKu y70dEJ+P31MMdA00QdRDmtUhN+uAsEAZpEG2p0BJaEIhw5vcTL47TtnNHhY4X7nCZcew 2PKLdlJ9GdNn0csjl8WEtfgEtYCjp6B+rVxF1VrWKiJSHRSQlLSUDwc8qWsRBMeIAPGr 9gAbuceYPodCYa53s3qkhVAUXkQMzJ7JAu2S9/t2ChadlAwl5Jz3eZtTYKdcaLaRbrQu 6TvA== X-Gm-Message-State: AC+VfDzVKk7VehcHgRIjqgiltJxWhKhHmvsGeU2s+szXwWbv7ag39XAD 8zgxBdZswAbUaiIK3uD/c8n457v0dc4TNg== X-Google-Smtp-Source: ACHHUZ4a8q5ljWXSfSaXs6ygqngOKLgwefkY5eVCtNF1uxBI+dX0LpgsHH1oWB+dpxEG2zYT08nY+GLkIYjHoA== X-Received: from mostafa.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:333c]) (user=smostafa job=sendgmr) by 2002:a05:6000:1b01:b0:306:880c:c7f5 with SMTP id f1-20020a0560001b0100b00306880cc7f5mr6493164wrz.7.1684246558004; Tue, 16 May 2023 07:15:58 -0700 (PDT) Date: Tue, 16 May 2023 14:15:31 +0000 Mime-Version: 1.0 X-Mailer: git-send-email 2.40.1.606.ga4b1b128d6-goog Message-ID: <20230516141531.791492-1-smostafa@google.com> Subject: [PATCH] KVM: arm64: Use different pointer authentication keys for pKVM From: Mostafa Saleh To: maz@kernel.org, oliver.upton@linux.dev, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org Cc: tabba@google.com, kaleshsingh@google.com, will@kernel.org, catalin.marinas@arm.com, yuzenghui@huawei.com, suzuki.poulose@arm.com, james.morse@arm.com, Mostafa Saleh X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230516_071601_246866_F2BE8BE6 X-CRM114-Status: GOOD ( 17.62 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When the kernel is compiled with CONFIG_ARM64_PTR_AUTH_KERNEL, it uses Armv8.3-Pauth for return address protection for the kernel code including nvhe code in EL2. Same keys are used in both kernel(EL1) and nvhe code(EL2), this is fine for nvhe but not when running in protected mode(pKVM) as the host can't be trusted. The keys for the hypervisor are generated from the kernel before it de-privileges, each cpu has different keys, this relies on nvhe code not being migratable while running. This patch adds host/hyp save/restore for the keys. For guest/hyp, they are already handled in common kvm code in __guest_enter, where they are saved/restored if they are not trapped. Signed-off-by: Mostafa Saleh --- arch/arm64/kvm/arm.c | 26 +++++++++++++++++++++++++ arch/arm64/kvm/hyp/nvhe/host.S | 35 +++++++++++++++++++++++++++++++++- 2 files changed, 60 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 14391826241c..dd03b52f035d 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -51,6 +51,8 @@ DECLARE_KVM_HYP_PER_CPU(unsigned long, kvm_hyp_vector); DEFINE_PER_CPU(unsigned long, kvm_arm_hyp_stack_page); DECLARE_KVM_NVHE_PER_CPU(struct kvm_nvhe_init_params, kvm_init_params); +DECLARE_KVM_NVHE_PER_CPU(struct kvm_cpu_context, kvm_hyp_ctxt); + static bool vgic_present; static DEFINE_PER_CPU(unsigned char, kvm_arm_hardware_enabled); @@ -2067,6 +2069,26 @@ static int __init kvm_hyp_init_protection(u32 hyp_va_bits) return 0; } +static void pkvm_hyp_init_ptrauth(void) +{ + struct kvm_cpu_context *hyp_ctxt; + int cpu; + + for_each_possible_cpu(cpu) { + hyp_ctxt = per_cpu_ptr_nvhe_sym(kvm_hyp_ctxt, cpu); + hyp_ctxt->sys_regs[APIAKEYLO_EL1] = get_random_long(); + hyp_ctxt->sys_regs[APIAKEYHI_EL1] = get_random_long(); + hyp_ctxt->sys_regs[APIBKEYLO_EL1] = get_random_long(); + hyp_ctxt->sys_regs[APIBKEYHI_EL1] = get_random_long(); + hyp_ctxt->sys_regs[APDAKEYLO_EL1] = get_random_long(); + hyp_ctxt->sys_regs[APDAKEYHI_EL1] = get_random_long(); + hyp_ctxt->sys_regs[APDBKEYLO_EL1] = get_random_long(); + hyp_ctxt->sys_regs[APDBKEYHI_EL1] = get_random_long(); + hyp_ctxt->sys_regs[APGAKEYLO_EL1] = get_random_long(); + hyp_ctxt->sys_regs[APGAKEYHI_EL1] = get_random_long(); + } +} + /* Inits Hyp-mode on all online CPUs */ static int __init init_hyp_mode(void) { @@ -2228,6 +2250,10 @@ static int __init init_hyp_mode(void) kvm_hyp_init_symbols(); if (is_protected_kvm_enabled()) { + if (IS_ENABLED(CONFIG_ARM64_PTR_AUTH_KERNEL) && + cpus_have_const_cap(ARM64_HAS_ADDRESS_AUTH)) + pkvm_hyp_init_ptrauth(); + init_cpu_logical_map(); if (!init_psci_relay()) { diff --git a/arch/arm64/kvm/hyp/nvhe/host.S b/arch/arm64/kvm/hyp/nvhe/host.S index b6c0188c4b35..255ba4af911b 100644 --- a/arch/arm64/kvm/hyp/nvhe/host.S +++ b/arch/arm64/kvm/hyp/nvhe/host.S @@ -10,6 +10,7 @@ #include #include #include +#include .text @@ -37,10 +38,42 @@ SYM_FUNC_START(__host_exit) /* Save the host context pointer in x29 across the function call */ mov x29, x0 + +#ifdef CONFIG_ARM64_PTR_AUTH_KERNEL +alternative_if_not ARM64_HAS_ADDRESS_AUTH +b __skip_pauth_save +alternative_else_nop_endif + +alternative_if ARM64_KVM_PROTECTED_MODE + /* Save kernel ptrauth keys. */ + add x18, x29, #CPU_APIAKEYLO_EL1 + ptrauth_save_state x18, x19, x20 + + /* Use hyp keys. */ + adr_this_cpu x18, kvm_hyp_ctxt, x19 + add x18, x18, #CPU_APIAKEYLO_EL1 + ptrauth_restore_state x18, x19, x20 +alternative_else_nop_endif +__skip_pauth_save: +#endif /* CONFIG_ARM64_PTR_AUTH_KERNEL */ + bl handle_trap - /* Restore host regs x0-x17 */ __host_enter_restore_full: + /* Restore kernel keys. */ +#ifdef CONFIG_ARM64_PTR_AUTH_KERNEL +alternative_if_not ARM64_HAS_ADDRESS_AUTH +b __skip_pauth_restore +alternative_else_nop_endif + +alternative_if ARM64_KVM_PROTECTED_MODE + add x18, x29, #CPU_APIAKEYLO_EL1 + ptrauth_restore_state x18, x19, x20 +alternative_else_nop_endif +__skip_pauth_restore: +#endif /* CONFIG_ARM64_PTR_AUTH_KERNEL */ + + /* Restore host regs x0-x17 */ ldp x0, x1, [x29, #CPU_XREG_OFFSET(0)] ldp x2, x3, [x29, #CPU_XREG_OFFSET(2)] ldp x4, x5, [x29, #CPU_XREG_OFFSET(4)]