From patchwork Fri Oct 8 13:58:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12545475 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8A037C433F5 for ; Fri, 8 Oct 2021 14:01:07 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5CBE960FD8 for ; Fri, 8 Oct 2021 14:01:07 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 5CBE960FD8 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=6aCnOrsxZoMXAuMQk9XoBo1JY/VXs+TlW4s3WfKkFSE=; b=2RNqLpE229l6mn LAe77jk8fx0tDRBemf6Y/WhOXP/+lPOXE/nZ1YGOIjP3UKg4OBej2flifCtfr0RBSTtDsvvW8iDh8 hwfN5PooEVEuD/OVLQ+l7X7cb/Y6mXBRsd3szybOf8f10T7t9FFnhKuCxsgZOAs/8MGCxhpZmIzn9 0MKjSYgC6c21EpHjXbQ6Tw0PpxMeu/EcT54NesYHWlb63KMI+I1fYPnOqfXJtPGmXg9jChFHYL8WZ 5EcJFwD/ATOD8HkCAG3v168vYUn//WCDMmTIBtpWWlo752+4mF5X+BG6+rWB2RaqpEZGKN53awiCx jYWYLtEmmEKN6IsRnC6A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mYqOt-002yjA-FX; Fri, 08 Oct 2021 13:59:03 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mYqOh-002ygK-Uj for linux-arm-kernel@lists.infradead.org; Fri, 08 Oct 2021 13:58:53 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 2839E61039; Fri, 8 Oct 2021 13:58:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1633701531; bh=NVbzQ35Z52Znz8dA7HLlPUIzmjGtwNnE/zkgMUVad5k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=IG2j3V4PSaXd4QbkhdY5fPc3NLcWzoiyw7dTzNN3I3RLqz67KBvk17f8zPzDKc1eq dXMyRk7AKmt2A56Zh900osINw2J022lubOKwwHIJjAG/8/E07hwEOBVTut+VzPjhrZ mJhuufQEmLHYdN28i5EXQHHZg+RByWyUiTXjgcD12o9tm6cjcXsyZH2gAzm0CEkgB3 791s2EdTkoH2Dc9U05+9YZR7gBZgffPWDF0IHlL3fHslX2FAc1bGkxQHCDeHlJ50gV g0/gi8hMmQLsOlJJWYH/p3CS/JUxP8JsB7t2G1TOsd7MDJaXLH4TlF35YjrOaJAZrJ /KxgRPk3TVJAQ== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Marc Zyngier , Quentin Perret , Catalin Marinas , Alexandru Elisei , Suzuki K Poulose , Mark Rutland , kvmarm@lists.cs.columbia.edu Subject: [PATCH v3 1/6] KVM: arm64: Turn __KVM_HOST_SMCCC_FUNC_* into an enum (mostly) Date: Fri, 8 Oct 2021 14:58:34 +0100 Message-Id: <20211008135839.1193-2-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20211008135839.1193-1-will@kernel.org> References: <20211008135839.1193-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211008_065852_047494_F336BADD X-CRM114-Status: GOOD ( 10.67 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Marc Zyngier __KVM_HOST_SMCCC_FUNC_* is a royal pain, as there is a fair amount of churn around these #defines, and we avoid making it an enum only for the sake of the early init, low level code that requires __KVM_HOST_SMCCC_FUNC___kvm_hyp_init to be usable from assembly. Let's be brave and turn everything but this symbol into an enum, using a bit of arithmetic to avoid any overlap. Acked-by: Will Deacon Link: https://lore.kernel.org/r/877depq9gw.wl-maz@kernel.org Signed-off-by: Marc Zyngier Signed-off-by: Will Deacon --- arch/arm64/include/asm/kvm_asm.h | 44 +++++++++++++++++--------------- 1 file changed, 24 insertions(+), 20 deletions(-) diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h index e86045ac43ba..43b5e213ae43 100644 --- a/arch/arm64/include/asm/kvm_asm.h +++ b/arch/arm64/include/asm/kvm_asm.h @@ -44,31 +44,35 @@ #define KVM_HOST_SMCCC_FUNC(name) KVM_HOST_SMCCC_ID(__KVM_HOST_SMCCC_FUNC_##name) #define __KVM_HOST_SMCCC_FUNC___kvm_hyp_init 0 -#define __KVM_HOST_SMCCC_FUNC___kvm_vcpu_run 1 -#define __KVM_HOST_SMCCC_FUNC___kvm_flush_vm_context 2 -#define __KVM_HOST_SMCCC_FUNC___kvm_tlb_flush_vmid_ipa 3 -#define __KVM_HOST_SMCCC_FUNC___kvm_tlb_flush_vmid 4 -#define __KVM_HOST_SMCCC_FUNC___kvm_flush_cpu_context 5 -#define __KVM_HOST_SMCCC_FUNC___kvm_timer_set_cntvoff 6 -#define __KVM_HOST_SMCCC_FUNC___kvm_enable_ssbs 7 -#define __KVM_HOST_SMCCC_FUNC___vgic_v3_get_gic_config 8 -#define __KVM_HOST_SMCCC_FUNC___vgic_v3_read_vmcr 9 -#define __KVM_HOST_SMCCC_FUNC___vgic_v3_write_vmcr 10 -#define __KVM_HOST_SMCCC_FUNC___vgic_v3_init_lrs 11 -#define __KVM_HOST_SMCCC_FUNC___kvm_get_mdcr_el2 12 -#define __KVM_HOST_SMCCC_FUNC___vgic_v3_save_aprs 13 -#define __KVM_HOST_SMCCC_FUNC___vgic_v3_restore_aprs 14 -#define __KVM_HOST_SMCCC_FUNC___pkvm_init 15 -#define __KVM_HOST_SMCCC_FUNC___pkvm_host_share_hyp 16 -#define __KVM_HOST_SMCCC_FUNC___pkvm_create_private_mapping 17 -#define __KVM_HOST_SMCCC_FUNC___pkvm_cpu_set_vector 18 -#define __KVM_HOST_SMCCC_FUNC___pkvm_prot_finalize 19 -#define __KVM_HOST_SMCCC_FUNC___kvm_adjust_pc 20 #ifndef __ASSEMBLY__ #include +enum __kvm_host_smccc_func { + /* __KVM_HOST_SMCCC_FUNC___kvm_hyp_init */ + __KVM_HOST_SMCCC_FUNC___kvm_vcpu_run = __KVM_HOST_SMCCC_FUNC___kvm_hyp_init + 1, + __KVM_HOST_SMCCC_FUNC___kvm_flush_vm_context, + __KVM_HOST_SMCCC_FUNC___kvm_tlb_flush_vmid_ipa, + __KVM_HOST_SMCCC_FUNC___kvm_tlb_flush_vmid, + __KVM_HOST_SMCCC_FUNC___kvm_flush_cpu_context, + __KVM_HOST_SMCCC_FUNC___kvm_timer_set_cntvoff, + __KVM_HOST_SMCCC_FUNC___kvm_enable_ssbs, + __KVM_HOST_SMCCC_FUNC___vgic_v3_get_gic_config, + __KVM_HOST_SMCCC_FUNC___vgic_v3_read_vmcr, + __KVM_HOST_SMCCC_FUNC___vgic_v3_write_vmcr, + __KVM_HOST_SMCCC_FUNC___vgic_v3_init_lrs, + __KVM_HOST_SMCCC_FUNC___kvm_get_mdcr_el2, + __KVM_HOST_SMCCC_FUNC___vgic_v3_save_aprs, + __KVM_HOST_SMCCC_FUNC___vgic_v3_restore_aprs, + __KVM_HOST_SMCCC_FUNC___pkvm_init, + __KVM_HOST_SMCCC_FUNC___pkvm_host_share_hyp, + __KVM_HOST_SMCCC_FUNC___pkvm_create_private_mapping, + __KVM_HOST_SMCCC_FUNC___pkvm_cpu_set_vector, + __KVM_HOST_SMCCC_FUNC___pkvm_prot_finalize, + __KVM_HOST_SMCCC_FUNC___kvm_adjust_pc, +}; + #define DECLARE_KVM_VHE_SYM(sym) extern char sym[] #define DECLARE_KVM_NVHE_SYM(sym) extern char kvm_nvhe_sym(sym)[] From patchwork Fri Oct 8 13:58:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12545473 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 665EBC433F5 for ; Fri, 8 Oct 2021 14:01:02 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3528861039 for ; Fri, 8 Oct 2021 14:01:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 3528861039 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=18ZkIt36mPNtENqIlkWrqzMEHG5k02RYkWHB6iVATfk=; b=VFNVsZKaebXXIb QnMfMgmlhuSrivCAtihPTnDiJf81NnrW9rQYerrc2eFUR8WmQskzUKoWUyBkNQVepSZItGdizbrLg eMa503vVQMUKC1SNxd09m+mAo0r51GxFElPvLW9eUHbSXyBvJ3KhoQRzNWgv+BGKCUXC2kMkE7HAZ ghaYZ8mJgAXGfJKPjVxyHnbJkh1xbOwQgbLGwAD4xX0/6D5gbb03YYVH53Lf0O+vXqp27DdUTGLIk R4+TDTGglSFlVGdkdn/jp0dK7rin6mo4dZouHjHwZzVuMn7Fhsdho9q4iLZU8SEt+TuyoES5CcsP8 KAGNyl03nJhd4nyaAz6w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mYqP3-002ykS-Kb; Fri, 08 Oct 2021 13:59:13 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mYqOj-002ygn-Ta for linux-arm-kernel@lists.infradead.org; Fri, 08 Oct 2021 13:58:55 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 2062D610CE; Fri, 8 Oct 2021 13:58:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1633701533; bh=4KM8LxAKb5VvinllXj4LLTwxDcCC5aREEGZbHA8T4I4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=BVV5/5Y6SlVKAxflH4ZlZtnx40guLDtRcu1pC5GXvaOIzVXoIK6eWsGI5GW2j2cUZ uM7tHkCPBcRpo9/kiQBep/dcHeAaNqMN2ARnydZRa/pBH+shi7c5HJK3X1nnsjtrLI +HLVTcxpJYt19kmnVJvBok3AHqtWg+ixgBRxpQfB29s4TlsIep/VYAS78e5phnKESo inKzNTw26JzivAmtTmYtiwm7737K/ebrwaJH4TGrBcJuutgs/KHOqwg2NmjeHHCVSA /5nPzDuWAsVssiTIUlRgUCvI15mfeOAK9E0HiY7ial7VXUXVYElal8eabV2vY9VGkN v+Nr8PPTZo4fA== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Marc Zyngier , Quentin Perret , Catalin Marinas , Alexandru Elisei , Suzuki K Poulose , Mark Rutland , kvmarm@lists.cs.columbia.edu Subject: [PATCH v3 2/6] arm64: Prevent kexec and hibernation if is_protected_kvm_enabled() Date: Fri, 8 Oct 2021 14:58:35 +0100 Message-Id: <20211008135839.1193-3-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20211008135839.1193-1-will@kernel.org> References: <20211008135839.1193-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211008_065854_000217_8BF690A4 X-CRM114-Status: GOOD ( 12.37 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When pKVM is enabled, the hypervisor code at EL2 and its data structures are inaccessible to the host kernel and cannot be torn down or replaced as this would defeat the integrity properies which pKVM aims to provide. Furthermore, the ABI between the host and EL2 is flexible and private to whatever the current implementation of KVM requires and so booting a new kernel with an old EL2 component is very likely to end in disaster. In preparation for uninstalling the hyp stub calls which are relied upon to reset EL2, disable kexec and hibernation in the host when protected KVM is enabled. Cc: Marc Zyngier Cc: Quentin Perret Signed-off-by: Will Deacon --- arch/arm64/kernel/smp.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index 6f6ff072acbd..44369b99a57e 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -1128,5 +1128,6 @@ bool cpus_are_stuck_in_kernel(void) { bool smp_spin_tables = (num_possible_cpus() > 1 && !have_cpu_die()); - return !!cpus_stuck_in_kernel || smp_spin_tables; + return !!cpus_stuck_in_kernel || smp_spin_tables || + is_protected_kvm_enabled(); } From patchwork Fri Oct 8 13:58:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12545477 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F07D3C433F5 for ; Fri, 8 Oct 2021 14:01:16 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B4AC46101E for ; Fri, 8 Oct 2021 14:01:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org B4AC46101E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=6T+ZdC7jjIIt5bsrKSekVbMURBQQ7CPtYsZmDbjNDa4=; b=boIEyTCHiTskWb YDWorqPFlDl5NwDVGFPubCUHlMa9/6mC5Uvjzjd53p3AA4vsb75t107lNoQXxdS/XOsflJmcEbTXt u2YOl2b3zM+9CTokN6oqhtIugo0XJkzTSM3q4jpHOQJAfyZeNuiJjzyHwBXNuyfhDuumRAetFxZZz 2kQVXh6C1vWqOUT1R3SeafyZiRrq1q7zbDd+lIWiaW3MWMcNYGs4OA7JKeiFENUH4mB5tbSF6zxPI CMSi6Arw9LHNb9LNbhrpZrQYYsxfS89gfo0r00BUk5RdXoFKfntG3AVpxA7A+U85IxN60MD+cxlrW ZDfGhoECd4CeCXejjtPw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mYqPD-002ymp-RW; Fri, 08 Oct 2021 13:59:23 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mYqOl-002yhE-T7 for linux-arm-kernel@lists.infradead.org; Fri, 08 Oct 2021 13:58:58 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 18E0861056; Fri, 8 Oct 2021 13:58:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1633701535; bh=THAlxcDV712qvLejzcM9lEPGBVTOj85hI6UndSHWDKg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nOWO2PXB0Gu+Ow9FoCztac2S16PKjIm0eujzQk36OfMS96Lcbn033D4OiMXTs8G9c vaSwXz1k51S1DF9Argmhs183BwNAEAw5jlStx8fOHa+MbA194MO6k3M/teNh5Ts3GM syCr5UfKQ5rPEwPkRxAhwWm9ZTNdROYxIuYbe1xaN0FRcVG13RzeAPUnUDmXvpPRjS lHHzd3W1wzExcpRld5I4xs8B2+6zlgwARwg/S6KHTZPQlQOqvdZTVdkflnctMaV90q ZPLKZ4bJmMPid52WWHq/bmg9sCeWpCRpxUtFrn+neSB09nsWcBezt5g5n6yGnnUGFk vqY14/+xltMFA== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Marc Zyngier , Quentin Perret , Catalin Marinas , Alexandru Elisei , Suzuki K Poulose , Mark Rutland , kvmarm@lists.cs.columbia.edu Subject: [PATCH v3 3/6] KVM: arm64: Reject stub hypercalls after pKVM has been initialised Date: Fri, 8 Oct 2021 14:58:36 +0100 Message-Id: <20211008135839.1193-4-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20211008135839.1193-1-will@kernel.org> References: <20211008135839.1193-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211008_065855_994692_51118885 X-CRM114-Status: GOOD ( 13.30 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The stub hypercalls provide mechanisms to reset and replace the EL2 code, so uninstall them once pKVM has been initialised in order to ensure the integrity of the hypervisor code. To ensure pKVM initialisation remains functional, split cpu_hyp_reinit() into two helper functions to separate usage of the stub from usage of pkvm hypercalls either side of __pkvm_init on the boot CPU. Cc: Marc Zyngier Cc: Quentin Perret Signed-off-by: Will Deacon --- arch/arm64/kvm/arm.c | 31 +++++++++++++++++++++++-------- arch/arm64/kvm/hyp/nvhe/host.S | 26 +++++++++++++++++--------- 2 files changed, 40 insertions(+), 17 deletions(-) diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index fe102cd2e518..9506cf88fa0e 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1579,25 +1579,33 @@ static void cpu_set_hyp_vector(void) kvm_call_hyp_nvhe(__pkvm_cpu_set_vector, data->slot); } -static void cpu_hyp_reinit(void) +static void cpu_hyp_init_context(void) { kvm_init_host_cpu_context(&this_cpu_ptr_hyp_sym(kvm_host_data)->host_ctxt); - cpu_hyp_reset(); - - if (is_kernel_in_hyp_mode()) - kvm_timer_init_vhe(); - else + if (!is_kernel_in_hyp_mode()) cpu_init_hyp_mode(); +} +static void cpu_hyp_init_features(void) +{ cpu_set_hyp_vector(); - kvm_arm_init_debug(); + if (is_kernel_in_hyp_mode()) + kvm_timer_init_vhe(); + if (vgic_present) kvm_vgic_init_cpu_hardware(); } +static void cpu_hyp_reinit(void) +{ + cpu_hyp_reset(); + cpu_hyp_init_context(); + cpu_hyp_init_features(); +} + static void _kvm_arch_hardware_enable(void *discard) { if (!__this_cpu_read(kvm_arm_hardware_enabled)) { @@ -1788,10 +1796,17 @@ static int do_pkvm_init(u32 hyp_va_bits) int ret; preempt_disable(); - hyp_install_host_vector(); + cpu_hyp_init_context(); ret = kvm_call_hyp_nvhe(__pkvm_init, hyp_mem_base, hyp_mem_size, num_possible_cpus(), kern_hyp_va(per_cpu_base), hyp_va_bits); + cpu_hyp_init_features(); + + /* + * The stub hypercalls are now disabled, so set our local flag to + * prevent a later re-init attempt in kvm_arch_hardware_enable(). + */ + __this_cpu_write(kvm_arm_hardware_enabled, 1); preempt_enable(); return ret; diff --git a/arch/arm64/kvm/hyp/nvhe/host.S b/arch/arm64/kvm/hyp/nvhe/host.S index 4b652ffb591d..0c6116d34e18 100644 --- a/arch/arm64/kvm/hyp/nvhe/host.S +++ b/arch/arm64/kvm/hyp/nvhe/host.S @@ -110,17 +110,14 @@ SYM_FUNC_START(__hyp_do_panic) b __host_enter_for_panic SYM_FUNC_END(__hyp_do_panic) -.macro host_el1_sync_vect - .align 7 -.L__vect_start\@: - stp x0, x1, [sp, #-16]! - mrs x0, esr_el2 - lsr x0, x0, #ESR_ELx_EC_SHIFT - cmp x0, #ESR_ELx_EC_HVC64 - b.ne __host_exit - +SYM_FUNC_START(__host_hvc) ldp x0, x1, [sp] // Don't fixup the stack yet + /* No stub for you, sonny Jim */ +alternative_if ARM64_KVM_PROTECTED_MODE + b __host_exit +alternative_else_nop_endif + /* Check for a stub HVC call */ cmp x0, #HVC_STUB_HCALL_NR b.hs __host_exit @@ -137,6 +134,17 @@ SYM_FUNC_END(__hyp_do_panic) ldr x5, =__kvm_handle_stub_hvc hyp_pa x5, x6 br x5 +SYM_FUNC_END(__host_hvc) + +.macro host_el1_sync_vect + .align 7 +.L__vect_start\@: + stp x0, x1, [sp, #-16]! + mrs x0, esr_el2 + lsr x0, x0, #ESR_ELx_EC_SHIFT + cmp x0, #ESR_ELx_EC_HVC64 + b.eq __host_hvc + b __host_exit .L__vect_end\@: .if ((.L__vect_end\@ - .L__vect_start\@) > 0x80) .error "host_el1_sync_vect larger than vector entry" From patchwork Fri Oct 8 13:58:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12545479 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 926E2C433F5 for ; Fri, 8 Oct 2021 14:01:20 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5CC226101E for ; Fri, 8 Oct 2021 14:01:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 5CC226101E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=xUlkqYlwBUFNOVGs+V6l3Rj8BIFHv+BsUearK7OzaTU=; b=D6rsSVeCcP3qdu ZOMAud6A5nLgTaAdSLFNfXPQvTDB4VyoIJDuwKU5I9P48zR8el7C5DXpEcf/8dHVj35MPc2+fAJbW exQ1rlvfuRlDW44W6zAYojBmaVind3izpjMQW46/SVE5OZGI9ZqgbtUb6nYq9w4BzGr4VjkYLjMvE 44Ontmm6SYJF/6x+UnkDGmv5qx4GFEp6vBDdKVZZoTqN+mNexH8K38+kzeQ0qBw50UkdwInD7UwD3 cS6rHX6Z4WXjoNBvdXBvs63K/J0C2SSFsVjQEYy0WMsnqCvG11xwJErFHbLSJ7qnwr7vmfu3S2rDS 6B0/tIpb9S67ydJCkZtQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mYqPO-002yq3-Td; Fri, 08 Oct 2021 13:59:35 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mYqOn-002yhW-SB for linux-arm-kernel@lists.infradead.org; Fri, 08 Oct 2021 13:58:59 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 1166C60FD8; Fri, 8 Oct 2021 13:58:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1633701537; bh=d3wFH/yIetGiq5uriHxlgYqLTUnj/21bzxMY8KFLvuE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=gSvmFTxBs3Yv3TsWi3tqX1+3UZfMC5nAidDqLeOMgNhiZwKO3rcX0xDtnTVSqPls8 1WKd7E2cNsrWEuJUcNOegtncGN3biwCWV9+kWuY+DF//z2s+v/+z4t1ZMsq5P5Coey SEG96mqW3SKMHFVwCLFTaWJ7OrBHJqMhfdwIhehq9bA9zvPrcHNM7YbuLTTS4SH9qv uW3OC9bISzBwlAwH5griMOPP3/kdXkWfKVHe8Xga07syTnDPDf9bgxMHUFnzHIDPlU HI1lJZ9b5xg3SDUVuU/nnIT1yCZGYK3XGDbZXXrEyaFTtKBI7T2FJbsxxSWjN4I2MC ZMDtaCgbISNZQ== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Marc Zyngier , Quentin Perret , Catalin Marinas , Alexandru Elisei , Suzuki K Poulose , Mark Rutland , kvmarm@lists.cs.columbia.edu Subject: [PATCH v3 4/6] KVM: arm64: Propagate errors from __pkvm_prot_finalize hypercall Date: Fri, 8 Oct 2021 14:58:37 +0100 Message-Id: <20211008135839.1193-5-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20211008135839.1193-1-will@kernel.org> References: <20211008135839.1193-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211008_065857_950784_D15844AE X-CRM114-Status: GOOD ( 13.96 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org If the __pkvm_prot_finalize hypercall returns an error, we WARN but fail to propagate the failure code back to kvm_arch_init(). Pass a pointer to a zero-initialised return variable so that failure to finalise the pKVM protections on a host CPU can be reported back to KVM. Cc: Marc Zyngier Cc: Quentin Perret Signed-off-by: Will Deacon --- arch/arm64/kvm/arm.c | 30 +++++++++++++++++++----------- 1 file changed, 19 insertions(+), 11 deletions(-) diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 9506cf88fa0e..13bbf35896cd 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1986,9 +1986,25 @@ static int init_hyp_mode(void) return err; } -static void _kvm_host_prot_finalize(void *discard) +static void _kvm_host_prot_finalize(void *arg) { - WARN_ON(kvm_call_hyp_nvhe(__pkvm_prot_finalize)); + int *err = arg; + + if (WARN_ON(kvm_call_hyp_nvhe(__pkvm_prot_finalize))) + WRITE_ONCE(*err, -EINVAL); +} + +static int pkvm_drop_host_privileges(void) +{ + int ret = 0; + + /* + * Flip the static key upfront as that may no longer be possible + * once the host stage 2 is installed. + */ + static_branch_enable(&kvm_protected_mode_initialized); + on_each_cpu(_kvm_host_prot_finalize, &ret, 1); + return ret; } static int finalize_hyp_mode(void) @@ -2002,15 +2018,7 @@ static int finalize_hyp_mode(void) * None of other sections should ever be introspected. */ kmemleak_free_part(__hyp_bss_start, __hyp_bss_end - __hyp_bss_start); - - /* - * Flip the static key upfront as that may no longer be possible - * once the host stage 2 is installed. - */ - static_branch_enable(&kvm_protected_mode_initialized); - on_each_cpu(_kvm_host_prot_finalize, NULL, 1); - - return 0; + return pkvm_drop_host_privileges(); } struct kvm_vcpu *kvm_mpidr_to_vcpu(struct kvm *kvm, unsigned long mpidr) From patchwork Fri Oct 8 13:58:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12545481 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 23024C433F5 for ; Fri, 8 Oct 2021 14:01:41 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E078460EC0 for ; Fri, 8 Oct 2021 14:01:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org E078460EC0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=VmSp73sLKT6aku4vwXcMCCBU1oQoHBdJUIc1wwktQSk=; b=NsNoteZlww0y1q BSSZyQE2TGWy1OtW9vhdj8hpsxzakU+aGynBAxIAtat28qzT7oW8DCnplnmWCjsVVlvQ7VOSOq9ZV Dma8mp5aLiP5QsaiWcMqzZmherU/O1RINgTmIywEUB6lEubiphBs1XZmd7lfVmj3OkT5gV+oTl27N vYKU9xdNRQUCgk40i00GGQVr4t25S/0rPwpgTxfxyYCCrXP5NttSNY7n//MWScrN6KxQbSX/E6VlS eHm3rHXL1hG3QM6av3rOBg+Xb227OX3gXRegGYpSdBTNWjrz4GMVSPxgH6pNAyLFzp6RZGdGXI7A3 3o+sb+qNpvVY5M4ZBFDg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mYqPc-002yuE-A8; Fri, 08 Oct 2021 13:59:48 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mYqOp-002yi5-R9 for linux-arm-kernel@lists.infradead.org; Fri, 08 Oct 2021 13:59:01 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 09CE661108; Fri, 8 Oct 2021 13:58:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1633701539; bh=JQ2vDfTTUv2kZVaFhWcuP9yI9JIJ01mjGFz/l+UPtH8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=o/yRBasuRkcPigzfjLMem83YshHKzzOmSqGMLIlAEeW8n0D+VR73we+TgbGPgGGBA RsiJUjoaqyL7pnTLLD1C8Uj6E4KOYfxDVjIxeVs9GtLOvfwsSeugN5jJ9K2O5BFcmK RK5ghodcW6948U7XQ3890/eL1dtGhJjZu8vb7orzTwxg+jTHE+SKOsa3Eez5Xe8F3p zMuUDru32bs6CeCQAvc5qfxBuUmK86GZm7iuWuDVhkaOZqU5J1uHTsWurF3nPV6i2b SxdX+3UYFjKSeKUtvj063j2vjgJ46CN6BJgHiRzA0ClwLK+GF4PG5su57HflyJQ2NV 8+21lXlOvaJiQ== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Marc Zyngier , Quentin Perret , Catalin Marinas , Alexandru Elisei , Suzuki K Poulose , Mark Rutland , kvmarm@lists.cs.columbia.edu Subject: [PATCH v3 5/6] KVM: arm64: Prevent re-finalisation of pKVM for a given CPU Date: Fri, 8 Oct 2021 14:58:38 +0100 Message-Id: <20211008135839.1193-6-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20211008135839.1193-1-will@kernel.org> References: <20211008135839.1193-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211008_065859_931084_82CF3B05 X-CRM114-Status: GOOD ( 11.12 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org __pkvm_prot_finalize() completes the deprivilege of the host when pKVM is in use by installing a stage-2 translation table for the calling CPU. Issuing the hypercall multiple times for a given CPU makes little sense, but in such a case just return early with -EPERM rather than go through the whole page-table dance again. Cc: Marc Zyngier Cc: Quentin Perret Signed-off-by: Will Deacon --- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c index bacd493a4eac..cafe17e5fa8f 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -123,6 +123,9 @@ int __pkvm_prot_finalize(void) struct kvm_s2_mmu *mmu = &host_kvm.arch.mmu; struct kvm_nvhe_init_params *params = this_cpu_ptr(&kvm_init_params); + if (params->hcr_el2 & HCR_VM) + return -EPERM; + params->vttbr = kvm_get_vttbr(mmu); params->vtcr = host_kvm.arch.vtcr; params->hcr_el2 |= HCR_VM; From patchwork Fri Oct 8 13:58:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12545483 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8DE57C433F5 for ; Fri, 8 Oct 2021 14:01:52 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5CBAD60EC0 for ; Fri, 8 Oct 2021 14:01:52 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 5CBAD60EC0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=5r/LPVbLFPzIDTFHq2sCLYpb4JS39QneBy5oh6VoQLs=; b=qa6PLbNPKlRVeN 7DwAf1z+fDEJtsJWe+SjFfdN0uxlvyUfwiJS4TrfR0uzSpVijglv6YiKohRcp/YsN4sQDyLXhgKFh gQMjUbeCgkWcDwdePnsU3aqaYn7vqWM586sQ0H1UI0gk55hf9WcBmcQx4h9UfBkE2B5YmXsgH/0E/ TG5R7PazULEZ8aubkAHt1N7zE+EWIXGhse9Y0UXgtjppT+06JsgcReBQHRCflNLJ1/vhHNFlfU7nG tyiU5pfFT6Xcc7G+02HZHfaP7fbS1j/aBMRhUT5ra/6OTyMFJYM4cHIwi8eE5JMPTBu3i8875gEa9 blCQZ/mTWfyZlASaTf7w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mYqPt-002z1v-9A; Fri, 08 Oct 2021 14:00:05 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mYqOr-002yir-RM for linux-arm-kernel@lists.infradead.org; Fri, 08 Oct 2021 13:59:03 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 05C2061042; Fri, 8 Oct 2021 13:58:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1633701541; bh=JTv2hBMDVb25nVPPPEEoEpoPB6xSLc/jUtJnXV0tdVQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jZnrC7Gk+glAbDKycPWsmETvhrpPe47c8jrns35zukWVGjdziwzckCHd5BUVOM/ik YP4GjWUemj95E1y7zEh10eE1S7Er1H5ox734i0PZHDbh0xxtziaUKtJ9zVglifuKBF LXdchp+8W3ruKiBr0VeMpnW6RaxqUv8bqEHKgdAzaabN4Th41ulNpyRbvX+dcNb587 +fX4ogzTnjrqhFfH5z2ylJdKBZH7xD/hVNpWIpogQ/xvJe9ziyWkbrpImacdZ8Dyjz hrs5tDcvaMJM/u6Bxw83e0DVeiMOXi/X+EYUmyiEgGrQoz6Ao8KVleL7o1VX52FCuP fy0X6rm0eQ7LA== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Marc Zyngier , Quentin Perret , Catalin Marinas , Alexandru Elisei , Suzuki K Poulose , Mark Rutland , kvmarm@lists.cs.columbia.edu Subject: [PATCH v3 6/6] KVM: arm64: Disable privileged hypercalls after pKVM finalisation Date: Fri, 8 Oct 2021 14:58:39 +0100 Message-Id: <20211008135839.1193-7-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20211008135839.1193-1-will@kernel.org> References: <20211008135839.1193-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211008_065901_943476_A188732C X-CRM114-Status: GOOD ( 16.95 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org After pKVM has been 'finalised' using the __pkvm_prot_finalize hypercall, the calling CPU will have a Stage-2 translation enabled to prevent access to memory pages owned by EL2. Although this forms a significant part of the process to deprivilege the host kernel, we also need to ensure that the hypercall interface is reduced so that the EL2 code cannot, for example, be re-initialised using a new set of vectors. Re-order the hypercalls so that only a suffix remains available after finalisation of pKVM. Cc: Marc Zyngier Cc: Quentin Perret Signed-off-by: Will Deacon --- arch/arm64/include/asm/kvm_asm.h | 25 +++++++++++--------- arch/arm64/kvm/hyp/nvhe/hyp-main.c | 37 +++++++++++++++++++++--------- 2 files changed, 40 insertions(+), 22 deletions(-) diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h index 43b5e213ae43..4654d27fd221 100644 --- a/arch/arm64/include/asm/kvm_asm.h +++ b/arch/arm64/include/asm/kvm_asm.h @@ -50,27 +50,30 @@ #include enum __kvm_host_smccc_func { + /* Hypercalls available only prior to pKVM finalisation */ /* __KVM_HOST_SMCCC_FUNC___kvm_hyp_init */ - __KVM_HOST_SMCCC_FUNC___kvm_vcpu_run = __KVM_HOST_SMCCC_FUNC___kvm_hyp_init + 1, + __KVM_HOST_SMCCC_FUNC___kvm_get_mdcr_el2 = __KVM_HOST_SMCCC_FUNC___kvm_hyp_init + 1, + __KVM_HOST_SMCCC_FUNC___pkvm_init, + __KVM_HOST_SMCCC_FUNC___pkvm_create_private_mapping, + __KVM_HOST_SMCCC_FUNC___pkvm_cpu_set_vector, + __KVM_HOST_SMCCC_FUNC___kvm_enable_ssbs, + __KVM_HOST_SMCCC_FUNC___vgic_v3_init_lrs, + __KVM_HOST_SMCCC_FUNC___vgic_v3_get_gic_config, + __KVM_HOST_SMCCC_FUNC___pkvm_prot_finalize, + + /* Hypercalls available after pKVM finalisation */ + __KVM_HOST_SMCCC_FUNC___pkvm_host_share_hyp, + __KVM_HOST_SMCCC_FUNC___kvm_adjust_pc, + __KVM_HOST_SMCCC_FUNC___kvm_vcpu_run, __KVM_HOST_SMCCC_FUNC___kvm_flush_vm_context, __KVM_HOST_SMCCC_FUNC___kvm_tlb_flush_vmid_ipa, __KVM_HOST_SMCCC_FUNC___kvm_tlb_flush_vmid, __KVM_HOST_SMCCC_FUNC___kvm_flush_cpu_context, __KVM_HOST_SMCCC_FUNC___kvm_timer_set_cntvoff, - __KVM_HOST_SMCCC_FUNC___kvm_enable_ssbs, - __KVM_HOST_SMCCC_FUNC___vgic_v3_get_gic_config, __KVM_HOST_SMCCC_FUNC___vgic_v3_read_vmcr, __KVM_HOST_SMCCC_FUNC___vgic_v3_write_vmcr, - __KVM_HOST_SMCCC_FUNC___vgic_v3_init_lrs, - __KVM_HOST_SMCCC_FUNC___kvm_get_mdcr_el2, __KVM_HOST_SMCCC_FUNC___vgic_v3_save_aprs, __KVM_HOST_SMCCC_FUNC___vgic_v3_restore_aprs, - __KVM_HOST_SMCCC_FUNC___pkvm_init, - __KVM_HOST_SMCCC_FUNC___pkvm_host_share_hyp, - __KVM_HOST_SMCCC_FUNC___pkvm_create_private_mapping, - __KVM_HOST_SMCCC_FUNC___pkvm_cpu_set_vector, - __KVM_HOST_SMCCC_FUNC___pkvm_prot_finalize, - __KVM_HOST_SMCCC_FUNC___kvm_adjust_pc, }; #define DECLARE_KVM_VHE_SYM(sym) extern char sym[] diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c index 2da6aa8da868..8566805ef62c 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -165,36 +165,51 @@ typedef void (*hcall_t)(struct kvm_cpu_context *); #define HANDLE_FUNC(x) [__KVM_HOST_SMCCC_FUNC_##x] = (hcall_t)handle_##x static const hcall_t host_hcall[] = { - HANDLE_FUNC(__kvm_vcpu_run), + /* ___kvm_hyp_init */ + HANDLE_FUNC(__kvm_get_mdcr_el2), + HANDLE_FUNC(__pkvm_init), + HANDLE_FUNC(__pkvm_create_private_mapping), + HANDLE_FUNC(__pkvm_cpu_set_vector), + HANDLE_FUNC(__kvm_enable_ssbs), + HANDLE_FUNC(__vgic_v3_init_lrs), + HANDLE_FUNC(__vgic_v3_get_gic_config), + HANDLE_FUNC(__pkvm_prot_finalize), + + HANDLE_FUNC(__pkvm_host_share_hyp), HANDLE_FUNC(__kvm_adjust_pc), + HANDLE_FUNC(__kvm_vcpu_run), HANDLE_FUNC(__kvm_flush_vm_context), HANDLE_FUNC(__kvm_tlb_flush_vmid_ipa), HANDLE_FUNC(__kvm_tlb_flush_vmid), HANDLE_FUNC(__kvm_flush_cpu_context), HANDLE_FUNC(__kvm_timer_set_cntvoff), - HANDLE_FUNC(__kvm_enable_ssbs), - HANDLE_FUNC(__vgic_v3_get_gic_config), HANDLE_FUNC(__vgic_v3_read_vmcr), HANDLE_FUNC(__vgic_v3_write_vmcr), - HANDLE_FUNC(__vgic_v3_init_lrs), - HANDLE_FUNC(__kvm_get_mdcr_el2), HANDLE_FUNC(__vgic_v3_save_aprs), HANDLE_FUNC(__vgic_v3_restore_aprs), - HANDLE_FUNC(__pkvm_init), - HANDLE_FUNC(__pkvm_cpu_set_vector), - HANDLE_FUNC(__pkvm_host_share_hyp), - HANDLE_FUNC(__pkvm_create_private_mapping), - HANDLE_FUNC(__pkvm_prot_finalize), }; static void handle_host_hcall(struct kvm_cpu_context *host_ctxt) { DECLARE_REG(unsigned long, id, host_ctxt, 0); + unsigned long hcall_min = 0; hcall_t hfn; + /* + * If pKVM has been initialised then reject any calls to the + * early "privileged" hypercalls. Note that we cannot reject + * calls to __pkvm_prot_finalize for two reasons: (1) The static + * key used to determine initialisation must be toggled prior to + * finalisation and (2) finalisation is performed on a per-CPU + * basis. This is all fine, however, since __pkvm_prot_finalize + * returns -EPERM after the first call for a given CPU. + */ + if (static_branch_unlikely(&kvm_protected_mode_initialized)) + hcall_min = __KVM_HOST_SMCCC_FUNC___pkvm_prot_finalize; + id -= KVM_HOST_SMCCC_ID(0); - if (unlikely(id >= ARRAY_SIZE(host_hcall))) + if (unlikely(id < hcall_min || id >= ARRAY_SIZE(host_hcall))) goto inval; hfn = host_hcall[id];