From patchwork Wed Apr 10 08:28:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13623836 Received: from mail-lj1-f178.google.com (mail-lj1-f178.google.com [209.85.208.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B146913D605 for ; Wed, 10 Apr 2024 08:28:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712737693; cv=none; b=pxyLqDsxKZo/cqBJZxa4KQ3+VzXmMdEW55J1uLqy4NlmgPle1WimiMt4Uipjjsazyf+54rtZR9dNFHTZR1Z/PMy1nrd7/737ktLb79f2tgMJ/HyXw35ZFmubR0BRnYmq1jpuxiX70VRjd+KENRviyu+EJGk/joQSIZGidD8I2OU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712737693; c=relaxed/simple; bh=loO222umeudNN/3VRDuIKBD8h6ayrjYQtdXvLf/9SQQ=; h=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type: Content-Disposition; b=hpm6hBZ/NBmFistgVbKaW/t3ze0MfAbIztvYvuG+E3/ZBc3Z/dx8DOeTNOD9yqu9u5kV6fSr954LO2MVPSMqGFijkPp5grQqYNjnjzKx5898PmPeG8S5kiUilPLfUK6TnIbbuac0CCl/CQS9OsWz2lECemmIRtaiZJdURCRl7CM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=t3yfHPkN; arc=none smtp.client-ip=209.85.208.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="t3yfHPkN" Received: by mail-lj1-f178.google.com with SMTP id 38308e7fff4ca-2d700beb60bso104763261fa.1 for ; Wed, 10 Apr 2024 01:28:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1712737690; x=1713342490; darn=vger.kernel.org; h=content-transfer-encoding:content-disposition:mime-version :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=qY+POk7z4HcyMMlZuNM5o7DMNVtuiYxieApfPnvv9HE=; b=t3yfHPkNmqDcRqyZAcdDOe3ajh1HexXyVR8bSm2idQejix23qvqzBL5E+sTVc1Nllo iOf2EyIBpQ1IIh2Vh7/cydahZP2BklGv2dHOMEmP9XPHi5N6K3GZqXXn1ZFTNIFKbEzQ hzv686tvWPyUzuF07nk8ifh38hBBwWhHCyXzWP+ND+17vHRL4cz4cMPoxnv4jOfhPWuM gYu+FMTUjWXGA4YFQmjGR0Qc5+UbVm0afK2Bvy6TgwmC+3R8qmYgKyliTqOiVlvqJ1xJ Ifv4veBuA0cSEor2xd3unkefdfa9yWOmibNtlJwuBV7YqT3jbTg0lCIKdv6B5HTk4c4W vxOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712737690; x=1713342490; h=content-transfer-encoding:content-disposition:mime-version :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qY+POk7z4HcyMMlZuNM5o7DMNVtuiYxieApfPnvv9HE=; b=CdBMkLAB+mUm675CP4APMzOROscQOREqNJH1G2KdC+vFnmHgvA/jLSx4U7Lvvn/pjJ i+50IKu8jJS0DxOVDeQVVGDGsnXpUvNTLSWE960bJ/XKsY6pHVsjIxpXRAvMFNlHRcTy uArh/wX7DBTuv5uRLHcHgGjy3smmASpa38FMssaLTPb0tCB8sazZCxszazjWXrjPVYVZ dE6TdhUCwZRGgysXYkWoEuyxLSUojae6jGTHgacRMqqoSDoVR+Tjd7VfpGk+y0QaGCnI PmSOJJutCJ/rdG9cN4osGwGo8GIBj2pJPYKl6YCTmGbcivIREZiYAZFzeWxSWZGE7/1k 2yFg== X-Forwarded-Encrypted: i=1; AJvYcCWfZ3nuDfFpZ+CYvBhZjG9NOaTcvXIThuXmZOjRNrBYiF98YkHCXecg3zeKVNIOcZOZWh9tPRuTjLWBkGFfA8kHZANq X-Gm-Message-State: AOJu0YyH5YZbBoUUa5ihk2wcwVkculx/KyQvzjgMr/vOnzuqDhojA+Xl BmoqoUGvNwmWUJmAbLfo5BraYXzYRJPqPtnMRsGynK9LDMubMBwHIJCpTWHTag== X-Google-Smtp-Source: AGHT+IEL5qT3PXK3v0HpJZ+zPd2REaRDBNASA0+bLbNO3hFsj9CwSLwbfI4Q9HqWFPT0ZHUDjdBQXg== X-Received: by 2002:a05:651c:2c1:b0:2d8:2d0a:7b9b with SMTP id f1-20020a05651c02c100b002d82d0a7b9bmr1863181ljo.14.1712737689629; Wed, 10 Apr 2024 01:28:09 -0700 (PDT) Received: from google.com (61.134.90.34.bc.googleusercontent.com. [34.90.134.61]) by smtp.gmail.com with ESMTPSA id dr2-20020a170907720200b00a4ea1fbb323sm6656464ejc.98.2024.04.10.01.28.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Apr 2024 01:28:09 -0700 (PDT) Date: Wed, 10 Apr 2024 09:28:05 +0100 From: =?utf-8?q?Pierre-Cl=C3=A9ment?= Tosi To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort Subject: [PATCH v2 01/12] KVM: arm64: Fix clobbered ELR in sync abort/SError Message-ID: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline When the hypervisor receives a SError or synchronous exception (EL2h) while running with the __kvm_hyp_vector and if ELR_EL2 doesn't point to an extable entry, it panics indirectly by overwriting ELR with the address of a panic handler in order for the asm routine it returns to to ERET into the handler. However, this clobbers ELR_EL2 for the handler itself. As a result, hyp_panic(), when retrieving what it believes to be the PC where the exception happened, actually ends up reading the address of the panic handler that called it! This results in an erroneous and confusing panic message where the source of any synchronous exception (e.g. BUG() or kCFI) appears to be __guest_exit_panic, making it hard to locate the actual BRK instruction. Therefore, store the original ELR_EL2 in the per-CPU kvm_hyp_ctxt and point the sysreg to a routine that first restores it to its previous value before running __guest_exit_panic. Fixes: 7db21530479f ("KVM: arm64: Restore hyp when panicking in guest context") Signed-off-by: Pierre-Clément Tosi --- arch/arm64/kernel/asm-offsets.c | 1 + arch/arm64/kvm/hyp/entry.S | 9 +++++++++ arch/arm64/kvm/hyp/include/hyp/switch.h | 5 +++-- 3 files changed, 13 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index 81496083c041..27de1dddb0ab 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -128,6 +128,7 @@ int main(void) DEFINE(VCPU_FAULT_DISR, offsetof(struct kvm_vcpu, arch.fault.disr_el1)); DEFINE(VCPU_HCR_EL2, offsetof(struct kvm_vcpu, arch.hcr_el2)); DEFINE(CPU_USER_PT_REGS, offsetof(struct kvm_cpu_context, regs)); + DEFINE(CPU_ELR_EL2, offsetof(struct kvm_cpu_context, sys_regs[ELR_EL2])); DEFINE(CPU_RGSR_EL1, offsetof(struct kvm_cpu_context, sys_regs[RGSR_EL1])); DEFINE(CPU_GCR_EL1, offsetof(struct kvm_cpu_context, sys_regs[GCR_EL1])); DEFINE(CPU_APIAKEYLO_EL1, offsetof(struct kvm_cpu_context, sys_regs[APIAKEYLO_EL1])); diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S index f3aa7738b477..bcaaf1a11b4e 100644 --- a/arch/arm64/kvm/hyp/entry.S +++ b/arch/arm64/kvm/hyp/entry.S @@ -83,6 +83,15 @@ alternative_else_nop_endif eret sb +SYM_INNER_LABEL(__guest_exit_restore_elr_and_panic, SYM_L_GLOBAL) + // x0-x29,lr: hyp regs + + stp x0, x1, [sp, #-16]! + adr_this_cpu x0, kvm_hyp_ctxt, x1 + ldr x0, [x0, #CPU_ELR_EL2] + msr elr_el2, x0 + ldp x0, x1, [sp], #16 + SYM_INNER_LABEL(__guest_exit_panic, SYM_L_GLOBAL) // x2-x29,lr: vcpu regs // vcpu x0-x1 on the stack diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h index e3fcf8c4d5b4..19a7ca2c1277 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -753,7 +753,7 @@ static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code) static inline void __kvm_unexpected_el2_exception(void) { - extern char __guest_exit_panic[]; + extern char __guest_exit_restore_elr_and_panic[]; unsigned long addr, fixup; struct kvm_exception_table_entry *entry, *end; unsigned long elr_el2 = read_sysreg(elr_el2); @@ -775,7 +775,8 @@ static inline void __kvm_unexpected_el2_exception(void) } /* Trigger a panic after restoring the hyp context. */ - write_sysreg(__guest_exit_panic, elr_el2); + this_cpu_ptr(&kvm_hyp_ctxt)->sys_regs[ELR_EL2] = elr_el2; + write_sysreg(__guest_exit_restore_elr_and_panic, elr_el2); } #endif /* __ARM64_KVM_HYP_SWITCH_H__ */ From patchwork Wed Apr 10 08:28:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13623837 Received: from mail-ej1-f47.google.com (mail-ej1-f47.google.com [209.85.218.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B82A513D61C for ; Wed, 10 Apr 2024 08:28:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.47 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712737704; cv=none; b=YHH6ErTzfYIxlRJnnKNRRTcOmt5ugaHmGpFdT/NRylNWHl7LrKFhLqu99CAA8gOw1VtDbsPJG4ZWgjo9NIbVvrS4fK1FRAHIQ3FgChJC8e64my65CH/ykJjhUCUNN94bucISWzQZzOdEBix4nXuB+yepNMKd/M89lJbkCqt92ZU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712737704; c=relaxed/simple; bh=SNsqoj9eNqjKtvp5/K3OU5JaLw8PBS6M12dONrb5QKc=; h=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type: Content-Disposition; b=TjG7vXzaKMZHLbOJjM30K2TQFb2eftsvFVrjyRx6JJ+msm7V12NoCR6YrLLYoyrblM7xF9pWpvWei41LuHnuiVu91ey9iRLdkIfBV5CrCpyPiZxmU5SZvIGlWJrJKFnu8kjIMOoqNFZ8CEppP5nAWsLWpfu90NnhTrBja31u4uw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=e1WUOx91; arc=none smtp.client-ip=209.85.218.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="e1WUOx91" Received: by mail-ej1-f47.google.com with SMTP id a640c23a62f3a-a52140ea1b5so20168666b.1 for ; Wed, 10 Apr 2024 01:28:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1712737701; x=1713342501; darn=vger.kernel.org; h=content-transfer-encoding:content-disposition:mime-version :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=AcC4Vj21DILBZiNE6/g7j9wakFMnVaxsPp20pxP4UlU=; b=e1WUOx91v37y0xQH63eMQO+Wx1GcQwYFQt7jYIhV6jYbe2F3YJPQzzo/XAIzYx6WAm xxKh3i1uGLdMHg6riiL29XRdsga7KOyxga4DCMVsduyBUyFPuD0HQmSZt1LviYMD4lCN L3dx1ZnGq6M1VWnAo2l1cQjDMuzvcIUmQFsFLZqziI0jfFhe6NzUCL6wmHreMhLrwV/h hS1MDEWhvWSrr6ufuwCZlLBWyV/V39c9smEs4ltJYOtaTtHNeqb5wezcz1dEATl3EGIt 3NdenaOAysGRWfg1Z0JG2VaIv9itfF6fUTmQ1QCsCNwb8YdjBT9iJfNqqBUB3+/AT6Fa Idbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712737701; x=1713342501; h=content-transfer-encoding:content-disposition:mime-version :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AcC4Vj21DILBZiNE6/g7j9wakFMnVaxsPp20pxP4UlU=; b=Q5oh/IEMHXXpjtFBjhLTVz/CRgiTJ6mAhS9QcYlLAYRk+VaOR4+1Hcjrlyx8pyu7Rf 7NAtG8VsediQA7HvKDcgwBXJFRWSOUyNTzbRYone+jbpzfRaY9J/cCvJw6zBH7Fdib1F hSvJ5HDL8s7QawIESyx0ZXqmqxTiwHYf8ZKZ27KcVXG42lF5TOzF3k2+Aa4yUN0bJ83H +AOG5ncljKCT2ZSMLamiFsVkQOz0weejaq/Yzm792QzoewiFd18hk3WJowkiTqS8I079 1hAoLLanDTGOuWipd99T8fBUBJRsiRX4UnQ4fsKFNg8M1GdHJ/OTIp7O8WDHLJ+aA4ly KspQ== X-Forwarded-Encrypted: i=1; AJvYcCU3cRGbsK7Ge2qtzi+2qabE2KDbWKxqlMOohciWi0WAfC6J1oOG5PPFFk4GPhTL6lcI9j0vIhbEERNX0i+hmQFHsf7t X-Gm-Message-State: AOJu0YyWvB+8oX9xr6uRb3ZWeH7rBrpDtOX29hfxIN05PfImlokDgNj4 VzBel1s4rCOBxnXgGXuJdplFOOlD5aH6zf0VyeQIq2HfOXKI860WYVLBevSryAx5dGcFQkt7D1d HEw== X-Google-Smtp-Source: AGHT+IGDOlmZUNa/b3VDkvm8WfG9UbS0FH17m/cxzH6+qrZhejCFvYfRumRMzmA89IrkJpXU6vz+vw== X-Received: by 2002:a17:907:70c1:b0:a4e:a068:7f with SMTP id yk1-20020a17090770c100b00a4ea068007fmr962531ejb.49.1712737700913; Wed, 10 Apr 2024 01:28:20 -0700 (PDT) Received: from google.com (61.134.90.34.bc.googleusercontent.com. [34.90.134.61]) by smtp.gmail.com with ESMTPSA id qq18-20020a17090720d200b00a51cdf560b9sm4255362ejb.37.2024.04.10.01.28.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Apr 2024 01:28:20 -0700 (PDT) Date: Wed, 10 Apr 2024 09:28:16 +0100 From: =?utf-8?q?Pierre-Cl=C3=A9ment?= Tosi To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort Subject: [PATCH v2 02/12] KVM: arm64: Fix __pkvm_init_switch_pgd C signature Message-ID: <22rqvzc34ehoirj42j6q27gfkxj53gjuryvtqqwh2q5d5yggme@rffmqhcxmtvy> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline Update the function declaration to match the asm implementation. Fixes: f320bc742bc2 ("KVM: arm64: Prepare the creation of s1 mappings at EL2") Signed-off-by: Pierre-Clément Tosi --- arch/arm64/include/asm/kvm_hyp.h | 3 +-- arch/arm64/kvm/hyp/nvhe/setup.c | 2 +- 2 files changed, 2 insertions(+), 3 deletions(-) diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h index 3e2a1ac0c9bb..96daf7cf6802 100644 --- a/arch/arm64/include/asm/kvm_hyp.h +++ b/arch/arm64/include/asm/kvm_hyp.h @@ -123,8 +123,7 @@ void __noreturn __hyp_do_panic(struct kvm_cpu_context *host_ctxt, u64 spsr, #endif #ifdef __KVM_NVHE_HYPERVISOR__ -void __pkvm_init_switch_pgd(phys_addr_t phys, unsigned long size, - phys_addr_t pgd, void *sp, void *cont_fn); +void __pkvm_init_switch_pgd(phys_addr_t params, void (*finalize_fn)(void)); int __pkvm_init(phys_addr_t phys, unsigned long size, unsigned long nr_cpus, unsigned long *per_cpu_base, u32 hyp_va_bits); void __noreturn __host_enter(struct kvm_cpu_context *host_ctxt); diff --git a/arch/arm64/kvm/hyp/nvhe/setup.c b/arch/arm64/kvm/hyp/nvhe/setup.c index bc58d1b515af..bcaeb0fafd2d 100644 --- a/arch/arm64/kvm/hyp/nvhe/setup.c +++ b/arch/arm64/kvm/hyp/nvhe/setup.c @@ -316,7 +316,7 @@ int __pkvm_init(phys_addr_t phys, unsigned long size, unsigned long nr_cpus, { struct kvm_nvhe_init_params *params; void *virt = hyp_phys_to_virt(phys); - void (*fn)(phys_addr_t params_pa, void *finalize_fn_va); + typeof(__pkvm_init_switch_pgd) *fn; int ret; BUG_ON(kvm_check_pvm_sysreg_table()); From patchwork Wed Apr 10 08:28:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13623838 Received: from mail-ed1-f41.google.com (mail-ed1-f41.google.com [209.85.208.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 57A0813D888 for ; Wed, 10 Apr 2024 08:28:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712737715; cv=none; b=f4csoaJzLez4DUxb6un5oE36fvCRIETJLKQPo8kpz/GELgsqL0lFANdpoFJibyNqEgQacz8uGvoyypbb6Ey4fh1Sm+J3ccW7mnKMZ6ZllCjwzZCL6hZqxlH4lfTbIK517CojPpVr8PiHAcBQ9bS8bbU8qDZE66yJlRNZQZXhPwI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712737715; c=relaxed/simple; bh=hQbIAYxCN7AHKx4jWK0xjI+853X8oj81NzdgGm0O5U0=; h=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type: Content-Disposition; b=KBFferoOe8GI4IqpMsI68Qw+solvcEqeMpXcJHMhtSEdLjB8DNuezbK3tBffMYU0P/44NKsyNOH2xw1WSb7EalHlG2zV9NQ9goa7EjGLVvhElHJalnyfGU4j4TVhj3RLCuK6yybxqYoxeFe6WSNcbb2hcSYXSMQYtvKogkYO8fk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=17S8qRTW; arc=none smtp.client-ip=209.85.208.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="17S8qRTW" Received: by mail-ed1-f41.google.com with SMTP id 4fb4d7f45d1cf-56e2ac1c16aso6559042a12.0 for ; Wed, 10 Apr 2024 01:28:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1712737710; x=1713342510; darn=vger.kernel.org; h=content-transfer-encoding:content-disposition:mime-version :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=qxLIFexzyxIK5HPZLQRBUNaNzamUF5S8ye0TLbRuvBI=; b=17S8qRTWk3CP7YqAMwAhRFzXk/zx3peK5NrbZIUgEz93KwHdpdHCV/n4BLEutRxWsX NJKigJ9lfKv6jVTW4hLR1VwEYkDxz0mD/Ev6G1x/UrRU/h7jX4cnvO8FArf2ANuDV4C9 ULvcOoacx4QLa5cw8MGQSho60DnHJeytCTQtTdl1cpFKAmaz80mkYiJSCXG5k7u1ET1P qbUZAg3IXCB0nFC1+C8tcXGiytt5Sgt24PsCw0Nk0Ieqz4Gh5icf//zqjSZKMPZk1I8V n7+QZW38TmNXceZWIk0+T6HNBUsx0800YeB421XLc6z9btP4rhpCjGTqakWqseuMxLFM LV2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712737710; x=1713342510; h=content-transfer-encoding:content-disposition:mime-version :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qxLIFexzyxIK5HPZLQRBUNaNzamUF5S8ye0TLbRuvBI=; b=r/JIyo07qA283b4nWEN7Y4bCiEBeHFyI6FXHIaJLpoP2dwuw2s2B1cDiCHWQwAZbpC 4qVcLEWDcTxXUcNRT+qKCEli5QzsH76jXyjY/1wgWnU/wsUjBkMk70vpNZ+Fz3QP6CxT yhG+0P5H/IrlzehFm84PLF+eriu2LtNQTlzUppxLY5UPKleZ11e5viHJwYD1NMSjaaD3 VV/JXmCRhebKDDBslsxUfR45nXRyiUiioIBMNM8KGvsqdbWhOZCjq1dfsZVoQJJ3H26k ga9rghzARV5wD016wLjPTy4it7H7xIx5gMdwhXDWv4tW7MB3jot9eMtI6HOzXz3ftwqI RjVQ== X-Forwarded-Encrypted: i=1; AJvYcCX+4iCRS9+61xU50aVu3rXBiRGpi0gJqtGhxe/20iqTIAOXa3XbWgj3OVRja7+nbExdIMazcNhNIo9YzYbMKQm39bR7 X-Gm-Message-State: AOJu0Yw/FhS7HJlJFgPuseUwneqTUeuMm0/G6RuDXu+I8aoGe8Qb2AKp bucNF/M1kAFMC7RjIVJSqL3Fei0jmxjPGOYNBhjmUtLsVOuvh82MnJ2JbKGDq+h02U5WlRdfFiA OrQ== X-Google-Smtp-Source: AGHT+IGsWsU9bAzX/IPEjmB8SimXIKLnvmyxzGnMrYTatjRSIELLqQwhMG790NNumW8RkBeRsXi0+A== X-Received: by 2002:a50:9ee9:0:b0:56e:418:5559 with SMTP id a96-20020a509ee9000000b0056e04185559mr1277972edf.3.1712737710569; Wed, 10 Apr 2024 01:28:30 -0700 (PDT) Received: from google.com (61.134.90.34.bc.googleusercontent.com. [34.90.134.61]) by smtp.gmail.com with ESMTPSA id q3-20020a056402248300b0056bfca6f1c0sm6154174eda.15.2024.04.10.01.28.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Apr 2024 01:28:30 -0700 (PDT) Date: Wed, 10 Apr 2024 09:28:26 +0100 From: =?utf-8?q?Pierre-Cl=C3=A9ment?= Tosi To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort Subject: [PATCH v2 03/12] KVM: arm64: Pass pointer to __pkvm_init_switch_pgd Message-ID: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline Make the function take a VA pointer, instead of a phys_addr_t, to fully take advantage of the high-level C language and its type checker. Perform all accesses to the kvm_nvhe_init_params before disabling the MMU, removing the need to access it using physical addresses, which was the reason for taking a phys_addr_t. Signed-off-by: Pierre-Clément Tosi --- arch/arm64/include/asm/kvm_hyp.h | 3 ++- arch/arm64/kvm/hyp/nvhe/hyp-init.S | 12 +++++++++--- arch/arm64/kvm/hyp/nvhe/setup.c | 4 +--- 3 files changed, 12 insertions(+), 7 deletions(-) diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h index 96daf7cf6802..c195e71d0746 100644 --- a/arch/arm64/include/asm/kvm_hyp.h +++ b/arch/arm64/include/asm/kvm_hyp.h @@ -123,7 +123,8 @@ void __noreturn __hyp_do_panic(struct kvm_cpu_context *host_ctxt, u64 spsr, #endif #ifdef __KVM_NVHE_HYPERVISOR__ -void __pkvm_init_switch_pgd(phys_addr_t params, void (*finalize_fn)(void)); +void __pkvm_init_switch_pgd(struct kvm_nvhe_init_params *params, + void (*finalize_fn)(void)); int __pkvm_init(phys_addr_t phys, unsigned long size, unsigned long nr_cpus, unsigned long *per_cpu_base, u32 hyp_va_bits); void __noreturn __host_enter(struct kvm_cpu_context *host_ctxt); diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-init.S b/arch/arm64/kvm/hyp/nvhe/hyp-init.S index 2994878d68ea..5a15737b4233 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-init.S +++ b/arch/arm64/kvm/hyp/nvhe/hyp-init.S @@ -265,7 +265,15 @@ alternative_else_nop_endif SYM_CODE_END(__kvm_handle_stub_hvc) +/* + * void __pkvm_init_switch_pgd(struct kvm_nvhe_init_params *params, + * void (*finalize_fn)(void)); + */ SYM_FUNC_START(__pkvm_init_switch_pgd) + /* Load the inputs from the VA pointer before turning the MMU off */ + ldr x5, [x0, #NVHE_INIT_PGD_PA] + ldr x0, [x0, #NVHE_INIT_STACK_HYP_VA] + /* Turn the MMU off */ pre_disable_mmu_workaround mrs x2, sctlr_el2 @@ -276,15 +284,13 @@ SYM_FUNC_START(__pkvm_init_switch_pgd) tlbi alle2 /* Install the new pgtables */ - ldr x3, [x0, #NVHE_INIT_PGD_PA] - phys_to_ttbr x4, x3 + phys_to_ttbr x4, x5 alternative_if ARM64_HAS_CNP orr x4, x4, #TTBR_CNP_BIT alternative_else_nop_endif msr ttbr0_el2, x4 /* Set the new stack pointer */ - ldr x0, [x0, #NVHE_INIT_STACK_HYP_VA] mov sp, x0 /* And turn the MMU back on! */ diff --git a/arch/arm64/kvm/hyp/nvhe/setup.c b/arch/arm64/kvm/hyp/nvhe/setup.c index bcaeb0fafd2d..45b83f3ed012 100644 --- a/arch/arm64/kvm/hyp/nvhe/setup.c +++ b/arch/arm64/kvm/hyp/nvhe/setup.c @@ -314,7 +314,6 @@ void __noreturn __pkvm_init_finalise(void) int __pkvm_init(phys_addr_t phys, unsigned long size, unsigned long nr_cpus, unsigned long *per_cpu_base, u32 hyp_va_bits) { - struct kvm_nvhe_init_params *params; void *virt = hyp_phys_to_virt(phys); typeof(__pkvm_init_switch_pgd) *fn; int ret; @@ -338,9 +337,8 @@ int __pkvm_init(phys_addr_t phys, unsigned long size, unsigned long nr_cpus, update_nvhe_init_params(); /* Jump in the idmap page to switch to the new page-tables */ - params = this_cpu_ptr(&kvm_init_params); fn = (typeof(fn))__hyp_pa(__pkvm_init_switch_pgd); - fn(__hyp_pa(params), __pkvm_init_finalise); + fn(this_cpu_ptr(&kvm_init_params), __pkvm_init_finalise); unreachable(); } From patchwork Wed Apr 10 08:28:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13623839 Received: from mail-ed1-f51.google.com (mail-ed1-f51.google.com [209.85.208.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7598213D8A3 for ; Wed, 10 Apr 2024 08:28:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712737727; cv=none; b=s7OOVaLA+jqDL2YZtsaFCakNCTXyQoie23p15mP3jcf5azYTQc5V6jOa9Vs8WWDlZ5gawjV8zxftvHtsPwRMX8sYIQv169lvm4bh1RUwv/lBS02rPdYFxr/ASqMOhnfgc2EQr2pBazNTaVa/E0JYNhTigA8MeaX1NQqQmIhklKA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712737727; c=relaxed/simple; bh=HHFQuNlgwp5z5UY2GtW1vIEvCtXKdaIbP6MhEmpQwXA=; h=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type: Content-Disposition; b=iGICBhnJERCIDS4xDpFh1FtBMlXtPEe4eyqcer+isJyqk/He5QkYkIEHosvMOtwoSFM676nKauEQSZJ//ffgPNrcadFsjEpT0aLuxkBtRDZFYWc5+5+Zn7T805Mg9LdxixKQ5weZR+lz7RK6GcxUSeNAFOxUc3KK/A4p3EGGWyU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=gG76D+bS; arc=none smtp.client-ip=209.85.208.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="gG76D+bS" Received: by mail-ed1-f51.google.com with SMTP id 4fb4d7f45d1cf-56829f41f81so9285816a12.2 for ; Wed, 10 Apr 2024 01:28:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1712737724; x=1713342524; darn=vger.kernel.org; h=content-transfer-encoding:content-disposition:mime-version :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=mJqNA1qEKMw3ceQcUL51OvRwoS/u84Dci9ztH9xxHDk=; b=gG76D+bSL0LlwpSYAgeC27V+4JSbnMEItFH9zy+Xm/sb6HpMAzavxJJnuCTAUSk2BS gRxfKq73zPFjX6hzfwYNNo5uSt3rgNrHsdSDjR1lsMzUPLeyibeBt8k964xq79tOf8IX GZBL1XjP1moT2jOieXzSBt/9VVYNay1Rf5YrdXD8X4YnBpWKw7Lc1n6AtGSeZt4gT3C0 4uypCY3fBzoxDV2lCGQaSFSxOOmmdV2ZWpoD8N6ogydba9ypnh4SEOZmt3F7KrrS8/wf t99f9ICfZtxEFwfRNAmLtvIn2N4NFqqqh3JVbONyCYDSckd1iE/0ZzUeJmOqFvUIj4lr dg7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712737724; x=1713342524; h=content-transfer-encoding:content-disposition:mime-version :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mJqNA1qEKMw3ceQcUL51OvRwoS/u84Dci9ztH9xxHDk=; b=unqZpYaxwl1v+165gPJub+TAEC/ztmLWl+mWfa5Sqa4GIj7GS0thoyDEA+zU6QHfUs q/RmeM1rkKjL30+RHmhHRUIPelVqAmL1UwUJmDjgP8FsN8ZyyhKQIM3HGRVNO67UtyVe ZpK87JmF09/0QjajGl2oF8d0wdIMTnR8IIlpbDU49K1Pw7KF17Cmprt0kTh1elM2vLpb GNtt5RAnEyVNIk6H5pEMCo3BCzPHuveckaW+24ek9jaltsyP43RMyuevvARVn8GA+/pl MbRHyoKB788IOj1yV71ftHJSFiDPTE0l3/3acOBpibrluC9a8a8IgmFXcskNZaPYHpWy 7F4w== X-Forwarded-Encrypted: i=1; AJvYcCUerM00RUao77Jk+o6ZQ1qlFtA6WfwSHn7wOrtaMsXziiuZ5+AXPVG2S+mTSQGQriMt+ybSPNQLWKxkYnIkAbVKcbbk X-Gm-Message-State: AOJu0YyS+5FnaYs4nvhlNDl1LoNcRqazUVeZ95y6xEd2pAr6fnog5dEF 14KOyRktaUCknxsxz5hWB+luBh874DZo7b6x0OUTAI8pR2CuBdn8ljhDFZH/W7vTrrpEEU1A6cs c2A== X-Google-Smtp-Source: AGHT+IETbmAd/CnDZsb5LGXlCF0alK/wvVK3k2FJoggB8JQHOb1ojR1ZUaJ70prv5IB1E7a0E9CJ/g== X-Received: by 2002:a50:d656:0:b0:56e:246b:2896 with SMTP id c22-20020a50d656000000b0056e246b2896mr1440708edj.3.1712737723651; Wed, 10 Apr 2024 01:28:43 -0700 (PDT) Received: from google.com (61.134.90.34.bc.googleusercontent.com. [34.90.134.61]) by smtp.gmail.com with ESMTPSA id p6-20020a05640210c600b0056e719a9a1bsm1754061edu.16.2024.04.10.01.28.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Apr 2024 01:28:43 -0700 (PDT) Date: Wed, 10 Apr 2024 09:28:39 +0100 From: =?utf-8?q?Pierre-Cl=C3=A9ment?= Tosi To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort Subject: [PATCH v2 04/12] KVM: arm64: nVHE: Simplify __guest_exit_panic path Message-ID: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline Immediately jump to __guest_exit_panic when taking an invalid EL2 exception with the nVHE host vector table instead of first duplicating the vCPU context check that __guest_exit_panic will also perform. Fix the wrong (probably bitrotten) __guest_exit_panic ABI doc to reflect how it is used by VHE and (now) nVHE and rename the routine to __hyp_panic to better reflect that it might not exit through the guest but will always (directly or indirectly) end up executing hyp_panic(). Use CPU_LR_OFFSET to clarify that the routine returns to hyp_panic(). Signed-off-by: Pierre-Clément Tosi --- arch/arm64/kvm/hyp/entry.S | 14 +++++++++----- arch/arm64/kvm/hyp/hyp-entry.S | 2 +- arch/arm64/kvm/hyp/include/hyp/switch.h | 4 ++-- arch/arm64/kvm/hyp/nvhe/host.S | 8 +------- 4 files changed, 13 insertions(+), 15 deletions(-) diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S index bcaaf1a11b4e..6a1ce9d21e5b 100644 --- a/arch/arm64/kvm/hyp/entry.S +++ b/arch/arm64/kvm/hyp/entry.S @@ -83,7 +83,7 @@ alternative_else_nop_endif eret sb -SYM_INNER_LABEL(__guest_exit_restore_elr_and_panic, SYM_L_GLOBAL) +SYM_INNER_LABEL(__hyp_restore_elr_and_panic, SYM_L_GLOBAL) // x0-x29,lr: hyp regs stp x0, x1, [sp, #-16]! @@ -92,13 +92,15 @@ SYM_INNER_LABEL(__guest_exit_restore_elr_and_panic, SYM_L_GLOBAL) msr elr_el2, x0 ldp x0, x1, [sp], #16 -SYM_INNER_LABEL(__guest_exit_panic, SYM_L_GLOBAL) - // x2-x29,lr: vcpu regs - // vcpu x0-x1 on the stack +SYM_INNER_LABEL(__hyp_panic, SYM_L_GLOBAL) + // x0-x29,lr: vcpu regs + + stp x0, x1, [sp, #-16]! // If the hyp context is loaded, go straight to hyp_panic get_loaded_vcpu x0, x1 cbnz x0, 1f + ldp x0, x1, [sp], #16 b hyp_panic 1: @@ -110,10 +112,12 @@ SYM_INNER_LABEL(__guest_exit_panic, SYM_L_GLOBAL) // accurate if the guest had been completely restored. adr_this_cpu x0, kvm_hyp_ctxt, x1 adr_l x1, hyp_panic - str x1, [x0, #CPU_XREG_OFFSET(30)] + str x1, [x0, #CPU_LR_OFFSET] get_vcpu_ptr x1, x0 + // Keep x0-x1 on the stack for __guest_exit + SYM_INNER_LABEL(__guest_exit, SYM_L_GLOBAL) // x0: return code // x1: vcpu diff --git a/arch/arm64/kvm/hyp/hyp-entry.S b/arch/arm64/kvm/hyp/hyp-entry.S index 03f97d71984c..7e65ef738ec9 100644 --- a/arch/arm64/kvm/hyp/hyp-entry.S +++ b/arch/arm64/kvm/hyp/hyp-entry.S @@ -122,7 +122,7 @@ el2_error: eret sb -.macro invalid_vector label, target = __guest_exit_panic +.macro invalid_vector label, target = __hyp_panic .align 2 SYM_CODE_START_LOCAL(\label) b \target diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h index 19a7ca2c1277..9387e3a0b680 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -753,7 +753,7 @@ static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code) static inline void __kvm_unexpected_el2_exception(void) { - extern char __guest_exit_restore_elr_and_panic[]; + extern char __hyp_restore_elr_and_panic[]; unsigned long addr, fixup; struct kvm_exception_table_entry *entry, *end; unsigned long elr_el2 = read_sysreg(elr_el2); @@ -776,7 +776,7 @@ static inline void __kvm_unexpected_el2_exception(void) /* Trigger a panic after restoring the hyp context. */ this_cpu_ptr(&kvm_hyp_ctxt)->sys_regs[ELR_EL2] = elr_el2; - write_sysreg(__guest_exit_restore_elr_and_panic, elr_el2); + write_sysreg(__hyp_restore_elr_and_panic, elr_el2); } #endif /* __ARM64_KVM_HYP_SWITCH_H__ */ diff --git a/arch/arm64/kvm/hyp/nvhe/host.S b/arch/arm64/kvm/hyp/nvhe/host.S index 135cfb294ee5..7397b4f1838a 100644 --- a/arch/arm64/kvm/hyp/nvhe/host.S +++ b/arch/arm64/kvm/hyp/nvhe/host.S @@ -196,19 +196,13 @@ SYM_FUNC_END(__host_hvc) tbz x0, #PAGE_SHIFT, .L__hyp_sp_overflow\@ sub x0, sp, x0 // x0'' = sp' - x0' = (sp + x0) - sp = x0 sub sp, sp, x0 // sp'' = sp' - x0 = (sp + x0) - x0 = sp - /* If a guest is loaded, panic out of it. */ - stp x0, x1, [sp, #-16]! - get_loaded_vcpu x0, x1 - cbnz x0, __guest_exit_panic - add sp, sp, #16 - /* * The panic may not be clean if the exception is taken before the host * context has been saved by __host_exit or after the hyp context has * been partially clobbered by __host_enter. */ - b hyp_panic + b __hyp_panic .L__hyp_sp_overflow\@: /* Switch to the overflow stack */ From patchwork Wed Apr 10 08:28:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13623840 Received: from mail-ej1-f46.google.com (mail-ej1-f46.google.com [209.85.218.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 89D6413D605 for ; Wed, 10 Apr 2024 08:28:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.46 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712737739; cv=none; b=OCMmsg2KlE87MRN7bYidYv1o2MyrLBtBt7+RSTinfTsqZXaV/MDk5B79Xr1GKrJjIdLQV9YBLfCiAltBKVK856ORoY6meIGxbC9ixwjqhBuD2zjh+2ewp/D2th4WZoqcClxyhMycmrrJWXfwASLcSpKYX9KNKKB6QVvqX6JtP8g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712737739; c=relaxed/simple; bh=OKzpp3vLMjhEjznNuaqC33ICK4wZMPNfg/0hzxvwLzY=; h=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type: Content-Disposition; b=RsVzjzgAwASMGuxjI+7O0c41N1iaYb9fFVJmuQwQ8dwFXJRAuK+0n3ig/cPnKwVUOYy2cGSMqD+zB4v2RZoii32xTPN7htWBJzKDeeVw/zC+ALa5JB6Nw+Jvh1NhogDJ+DwvYU4ZJFQY/N0g6tspFtHMqNA0T2cQ4ZPJrrhs6hk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=3iMhqC/M; arc=none smtp.client-ip=209.85.218.46 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="3iMhqC/M" Received: by mail-ej1-f46.google.com with SMTP id a640c23a62f3a-a46ea03c2a5so1070356066b.1 for ; Wed, 10 Apr 2024 01:28:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1712737736; x=1713342536; darn=vger.kernel.org; h=content-transfer-encoding:content-disposition:mime-version :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=henblBD0eejAfMb1rp/7/m+PwpPrco2Rcp0q8OTMVVo=; b=3iMhqC/M9TvdaMWx2wpN/rsCrvBJOyPCxICsfzpXzFvsCrjBPAbeONzz3+YJsBIUGt 1XDvkq8SrMTAoEX7OyTKAgF3TFa/lavrHYG4oZK95hXAe5PCI36zSiI60yJPPv4Brd5+ q5+ebXblrKR7yBTiMEfp0TFvE+fcGyezqjULal3LUxW8l5+GveI4vQb9i/ntpF9Z+wKC p7RSTTP3teSp1/UttUzzc9F5ah4ocoxvAT2IV377NC2l8kySsIKvGCWuX14CF08Koedj WirCaOfQnYzNs+52MtN2db1jwlJuNPS9RVgQ/nrxSjIJKEApL9vgLgPH3ty+P2ku+GMk kqzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712737736; x=1713342536; h=content-transfer-encoding:content-disposition:mime-version :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=henblBD0eejAfMb1rp/7/m+PwpPrco2Rcp0q8OTMVVo=; b=h7mnRbVbgiU5bhcWw3bIg/cOF4/i5H7mv/i5vDJZr/wNie6zfV78ke1fd3HKyRGE0B 0Lg1d4a/TIlW8JN7TZuCV1hYAZb8Gdu0XV54+OuZw7BjQmOCW5bLMEm1WV0SaTuwFfSY l6AaqVuoKRPO0j6FMY4FFYtzjPuVNxRufilFsnxW6kV4/JWxUxHMq5uxwIQbFJLaqZ/q TH/dkr9xrzFpOUMpFaQEQOR6Zhpv9nKuXw9+E9lXxxCieZA4W0qFLWV8cbDXkSmrtfer ArSYXqFMfFyYuENHZuwuis+6PbMYA60KySWPA2OS94BnuscTYQcPmi2HY7uCH8ySya4b gRaQ== X-Forwarded-Encrypted: i=1; AJvYcCVf1PQTajJKQ1EnLRdv5H3kwKM0UftXpBlYaBcQtyavBmi6196aqTbx402gk+IELlVXjHqfSzZM3zy9GAUGxE2xPGzr X-Gm-Message-State: AOJu0YzRN6ggSSc+Vx/RucsjH94y8Fhk/fxhMYgGKV2Ysgwd5bg7GppE O+RB8g6yAFEsvIullrPEMi+EprQiTs818897cImjgq7asknn4ikEGLRRJHv4eQ== X-Google-Smtp-Source: AGHT+IH9EfTrxyP8L2t1ExqRK+qnv2iao0ut7BEh9JmB6lgatiwZJEUZjeaWiTsf3pfjLvZRGVJW3A== X-Received: by 2002:a17:906:fd8d:b0:a51:e188:bced with SMTP id xa13-20020a170906fd8d00b00a51e188bcedmr4535131ejb.37.1712737735696; Wed, 10 Apr 2024 01:28:55 -0700 (PDT) Received: from google.com (61.134.90.34.bc.googleusercontent.com. [34.90.134.61]) by smtp.gmail.com with ESMTPSA id gl2-20020a170906e0c200b00a4df5e48d11sm6698623ejb.72.2024.04.10.01.28.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Apr 2024 01:28:55 -0700 (PDT) Date: Wed, 10 Apr 2024 09:28:51 +0100 From: =?utf-8?q?Pierre-Cl=C3=A9ment?= Tosi To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort Subject: [PATCH v2 05/12] KVM: arm64: nVHE: Add EL2h sync exception handler Message-ID: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline Introduce a handler for EL2h synchronous exceptions distinct from handlers for other "invalid" exceptions when running with the nVHE host vector. This will allow a future patch to handle kCFI (synchronous) errors without affecting other classes of exceptions. Signed-off-by: Pierre-Clément Tosi --- arch/arm64/kvm/hyp/nvhe/host.S | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/arch/arm64/kvm/hyp/nvhe/host.S b/arch/arm64/kvm/hyp/nvhe/host.S index 7397b4f1838a..0613b6e35137 100644 --- a/arch/arm64/kvm/hyp/nvhe/host.S +++ b/arch/arm64/kvm/hyp/nvhe/host.S @@ -183,7 +183,7 @@ SYM_FUNC_END(__host_hvc) .endif .endm -.macro invalid_host_el2_vect +.macro __host_el2_vect handler:req .align 7 /* @@ -202,7 +202,7 @@ SYM_FUNC_END(__host_hvc) * context has been saved by __host_exit or after the hyp context has * been partially clobbered by __host_enter. */ - b __hyp_panic + b \handler .L__hyp_sp_overflow\@: /* Switch to the overflow stack */ @@ -212,6 +212,10 @@ SYM_FUNC_END(__host_hvc) ASM_BUG() .endm +.macro host_el2_sync_vect + __host_el2_vect __hyp_panic +.endm + .macro invalid_host_el1_vect .align 7 mov x0, xzr /* restore_host = false */ @@ -221,6 +225,10 @@ SYM_FUNC_END(__host_hvc) b __hyp_do_panic .endm +.macro invalid_host_el2_vect + __host_el2_vect __hyp_panic +.endm + /* * The host vector does not use an ESB instruction in order to avoid consuming * SErrors that should only be consumed by the host. Guest entry is deferred by @@ -238,7 +246,7 @@ SYM_CODE_START(__kvm_hyp_host_vector) invalid_host_el2_vect // FIQ EL2t invalid_host_el2_vect // Error EL2t - invalid_host_el2_vect // Synchronous EL2h + host_el2_sync_vect // Synchronous EL2h invalid_host_el2_vect // IRQ EL2h invalid_host_el2_vect // FIQ EL2h invalid_host_el2_vect // Error EL2h From patchwork Wed Apr 10 08:29:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13623841 Received: from mail-ej1-f50.google.com (mail-ej1-f50.google.com [209.85.218.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4F72B13D529 for ; Wed, 10 Apr 2024 08:29:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712737751; cv=none; b=Y4P8NMv7a8xCntqgRo9T8M+Oji0WW5t4PA07TJaKls57Hg5Jf0GWEqB15TaS3XdNll7WsAH820wwyEalcjGkulSmIL69p82oofWh3sITTzUeonxCABwkgwBsTvtY8ajcbw2r9S/QLsbLTrIFbJ2xzYmiNA1tlDWnDjDfIu+ogtc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712737751; c=relaxed/simple; bh=kgnqMQDzRwAX/8s3ORh3D/qKYGRwk7ltldtZkzXhbzU=; h=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type: Content-Disposition; b=kb3nBrlgSlPXxN6fkq9LrreQtz9AgvYpT3Q9o/XoEpR/OSv0EKaREexUStNaGhe+EcZj6jCiR/offwjYXErpZZ+fJ6JCps/SVaWwbjHUTKrne79EadJYWEyHVvFT1jksFJz47XdZPLtErPkYWdPjz04vEXiE34HyS1ONKBXYTZQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=MtXkiz7t; arc=none smtp.client-ip=209.85.218.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="MtXkiz7t" Received: by mail-ej1-f50.google.com with SMTP id a640c23a62f3a-a519e1b0e2dso672989766b.2 for ; Wed, 10 Apr 2024 01:29:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1712737747; x=1713342547; darn=vger.kernel.org; h=content-transfer-encoding:content-disposition:mime-version :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=3PGd3bMV1YAEF5Hagbbpcdm+V4PDQlrcvcx2hUHTQ9g=; b=MtXkiz7tTmJzC58djRthp9IB+Hup4dsd3Qz6DAK5204q16qYs+S8rJxwZ0FiyiTKnw 7sL4/ou55n966tSQq2oHb0PkC43ZN8PcqwvtXXDqomHHlxitHE99Yc2J/1aklkPHSyVR GZgfC7oKjiPrdomriA5MLhzvDmWiGisKBRd2XuX5HmVB2Q8nJXRYhyHoT6h3mJPPBjRG wD3VtovyrVQHzMFbB0Q8rgX5w33+7GUFpqNJzgvaMVG1HvZADIaXPFGfXzgcNVslMAIj RgoSfshjJXp/2+fhng7yCIp5MyBcvfRkrOamPsXzbj0+nwodBUbxmf3Cg7yFqMUv3c8i XBcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712737747; x=1713342547; h=content-transfer-encoding:content-disposition:mime-version :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3PGd3bMV1YAEF5Hagbbpcdm+V4PDQlrcvcx2hUHTQ9g=; b=sAokYbDaqFsDsY9x4+eTqY06WBSat3SOUPQ655KX4LqSglrciVB+pGTMWjMs2JS5Od fZ2OBFb/WE+jVLofq7bOCXkjN+ZkLEStvPHtZFo2kPU/Jx7+GfDFER/UYWw6k87S0KTT s/JOIvWIGpTxxcz3tRZAmKcafzk6LYKQaLPMFRepr2/d3Wj9R4wezEH3lYLTqp1oiXuq tGB88QNxBc9nGVM3Rd2WuP31v2HhLmjt23G0o6FPAV24UwnrZIoggZYFUwVPfoLdLxJC 2owY4eyXWaPzXqbeFPfWx39ls/Uo1VSWd4ErVFAeiNKVc2kREkKLLDvZ01RHKQqcGLsC 5sVw== X-Forwarded-Encrypted: i=1; AJvYcCUhbOkH+iN87S0jPUvUsi1wjnvYjfnzDUZCCi7stbruJhAO+ubfJV0V4NRfOuIunNccJs93MSIssOrxPAz39o9IaktE X-Gm-Message-State: AOJu0YwbEJ5xf44UeXOBxLcsscmIwiNqMS1AzPKxDwqCs4xCsqyzXRNh ZH4CdKJfDtwGZJu9IFkQNilwZFyIqf45TkmlIDaAwNCngo3nz2sCr8gDXOq05w== X-Google-Smtp-Source: AGHT+IH61cn+dX4vldfSxdcYm9FVH61Ciw9ZsCjHLsITsN4DKWMU3FJ45IW/shiIDSzz7xSZXKNICg== X-Received: by 2002:a17:906:490:b0:a51:b326:ed41 with SMTP id f16-20020a170906049000b00a51b326ed41mr1201398eja.52.1712737747375; Wed, 10 Apr 2024 01:29:07 -0700 (PDT) Received: from google.com (61.134.90.34.bc.googleusercontent.com. [34.90.134.61]) by smtp.gmail.com with ESMTPSA id jx16-20020a170907761000b00a5197fa2970sm6678498ejc.25.2024.04.10.01.29.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Apr 2024 01:29:07 -0700 (PDT) Date: Wed, 10 Apr 2024 09:29:03 +0100 From: =?utf-8?q?Pierre-Cl=C3=A9ment?= Tosi To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort Subject: [PATCH v2 06/12] KVM: arm64: nVHE: gen-hyprel: Skip R_AARCH64_ABS32 Message-ID: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline Ignore R_AARCH64_ABS32 relocations, instead of panicking, when emitting the relocation table of the hypervisor. The toolchain might produce them when generating function calls with kCFI, to allow type ID resolution across compilation units (between the call-site check and the callee's prefixed u32) at link time. They are therefore not needed in the final (runtime) relocation table. Signed-off-by: Pierre-Clément Tosi --- arch/arm64/kvm/hyp/nvhe/gen-hyprel.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/arch/arm64/kvm/hyp/nvhe/gen-hyprel.c b/arch/arm64/kvm/hyp/nvhe/gen-hyprel.c index 6bc88a756cb7..b63f4e1c1033 100644 --- a/arch/arm64/kvm/hyp/nvhe/gen-hyprel.c +++ b/arch/arm64/kvm/hyp/nvhe/gen-hyprel.c @@ -50,6 +50,9 @@ #ifndef R_AARCH64_ABS64 #define R_AARCH64_ABS64 257 #endif +#ifndef R_AARCH64_ABS32 +#define R_AARCH64_ABS32 258 +#endif #ifndef R_AARCH64_PREL64 #define R_AARCH64_PREL64 260 #endif @@ -383,6 +386,9 @@ static void emit_rela_section(Elf64_Shdr *sh_rela) case R_AARCH64_ABS64: emit_rela_abs64(rela, sh_orig_name); break; + /* Allow 32-bit absolute relocation, for kCFI type hashes. */ + case R_AARCH64_ABS32: + break; /* Allow position-relative data relocations. */ case R_AARCH64_PREL64: case R_AARCH64_PREL32: From patchwork Wed Apr 10 08:29:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13623842 Received: from mail-ed1-f43.google.com (mail-ed1-f43.google.com [209.85.208.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6937A13D529 for ; Wed, 10 Apr 2024 08:29:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712737762; cv=none; b=aksLBhs803wMqrFhRK07vV7h9bEtd2OY1Arfjra/dIRwYyJoWqNEXrrZ1rX1l9+lu2hVGXdgZY3gzKj3I6HjxLfhbu0IQctziLoEfrEQdvQMhAaUSktldMEuqS9ZxDjuwvKPbtV0G0Xfqz8H+q7FsWYPeAY41f8ZRjyPZVN/WuY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712737762; c=relaxed/simple; bh=1xy8JEBzmCvsE0c+vyvVtF1ncyFT2iUpIVYsycSUaUc=; h=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type: Content-Disposition; b=CnVDlmf9SbRV0o5n3F7yU1Q6h6OxOptTrOCQMcnX9mdwfy9flQbajKgFVy/k9i9hRmeUHnYT3REseHa/aMRXUMkXCB4p/HTtMLe+9QEkpQjqygsVwcR/x5rxYI2xF4GEVygEfnFa8Wpd6eI9+4jeaHy6vUlEr5o0Zn8lFYT1uvc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=rIx5NfK5; arc=none smtp.client-ip=209.85.208.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="rIx5NfK5" Received: by mail-ed1-f43.google.com with SMTP id 4fb4d7f45d1cf-56e37503115so4608340a12.1 for ; Wed, 10 Apr 2024 01:29:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1712737759; x=1713342559; darn=vger.kernel.org; h=content-transfer-encoding:content-disposition:mime-version :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=2jh/NV6yW6K+2G6i6rK3aentqLTJHJAzofXzqiefolM=; b=rIx5NfK5fG9LF+7iVfia4KKbZkGkwrMXBBs2FC1nfdSZolZghvyizLzRD8Inlrq1cz s2MpLX0cNKwKwMeyr/BdqHNK9KnXRCv1z2cgF93IFU7ZtZQP4tEmjaq4ZUkDdCBW3KsH +VGE2X7/jT604DeCa3iwUkcAgj6r5LNAE74yAGJnWGrr/liooyrQKvrE7C8rkPp+Mvt6 jkide+dzewojjTPUdTMpLX+enKh/eZv18+MBTgS79Mw601KtMr5CEmXzQOt32uZGbzdS 414crS3E5xv3dmWf2DbOJFBcTEn0PEWDSdqmkw/VeVl3Fax7mR6KW59MLgV7HwR4g9rf TkBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712737759; x=1713342559; h=content-transfer-encoding:content-disposition:mime-version :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2jh/NV6yW6K+2G6i6rK3aentqLTJHJAzofXzqiefolM=; b=ku23I/yE/QThNgdrLYBhSu+sNN2qNQWYURZxxvIznycBnYmM3xVRB/k8y8lNM6wcRn 8mSedUrpsUcQYLrk+eoJ+oc0bwCWifs8EWes/NWAab7d5fxCk/K/xs4ddA2RZb0gGL3T qEYSB5bcHQsDCmFndUlF8K7xZFAy1fECkgM7zyQQycmvtHfTq9qreFVETo6eVCmgDsUk SyMS7lP2vQq4Syr/hCHRniHtBwbkeH7eptF5s3zS6F8sJleHoIAT+j81NxclcurfZM0I FNkXb4F61l/DYi5p7TOFvgNmKly6DBd53mi+MTrsk0mhiSRhUs2s1PLIbhm8Cw14Jahv SoUQ== X-Forwarded-Encrypted: i=1; AJvYcCUi/luD819P6nLBkzTT1bIVNSUmaed9EY/kVtROaVJmue1/SBRWQiXOWY5CfqCBgYTeg/6iijBd5j9T7olAn5AwBnZd X-Gm-Message-State: AOJu0Yz73Md1k7e94XEB6KIxh7X8OHXS5LAYrJOWxoViaAhLQnM0zAbW JhGIHamWEu2vcs4+3nK9n/NsjLGV5pdm+yzkMfNSS/fkHs42nc78M6lItfonJw== X-Google-Smtp-Source: AGHT+IHhEhDmwBmoCNwZPINWnUcQxwKkJgql9hzsRwxXum2LL0s2QGfma9Zc+wwW++uYiLm7gJA5pQ== X-Received: by 2002:a50:871b:0:b0:56e:219a:b49c with SMTP id i27-20020a50871b000000b0056e219ab49cmr1074195edb.32.1712737758727; Wed, 10 Apr 2024 01:29:18 -0700 (PDT) Received: from google.com (61.134.90.34.bc.googleusercontent.com. [34.90.134.61]) by smtp.gmail.com with ESMTPSA id ef5-20020a05640228c500b0056e74af55e0sm1558489edb.83.2024.04.10.01.29.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Apr 2024 01:29:18 -0700 (PDT) Date: Wed, 10 Apr 2024 09:29:14 +0100 From: =?utf-8?q?Pierre-Cl=C3=A9ment?= Tosi To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort Subject: [PATCH v2 07/12] KVM: arm64: VHE: Mark __hyp_call_panic __noreturn Message-ID: <7f4fsc647ve5c4tn5wxosdrss2iqd7fsgmdx3rha4dvruwyi75@kc5yyoyaw7iv> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline Given that the sole purpose of __hyp_call_panic() is to call panic(), a __noreturn function, give it the __noreturn attribute, removing the need for its caller to use unreachable(). Signed-off-by: Pierre-Clément Tosi --- arch/arm64/kvm/hyp/vhe/switch.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c index 1581df6aec87..9db04a286398 100644 --- a/arch/arm64/kvm/hyp/vhe/switch.c +++ b/arch/arm64/kvm/hyp/vhe/switch.c @@ -301,7 +301,7 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu) return ret; } -static void __hyp_call_panic(u64 spsr, u64 elr, u64 par) +static void __noreturn __hyp_call_panic(u64 spsr, u64 elr, u64 par) { struct kvm_cpu_context *host_ctxt; struct kvm_vcpu *vcpu; @@ -326,7 +326,6 @@ void __noreturn hyp_panic(void) u64 par = read_sysreg_par(); __hyp_call_panic(spsr, elr, par); - unreachable(); } asmlinkage void kvm_unexpected_el2_exception(void) From patchwork Wed Apr 10 08:29:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13623843 Received: from mail-ed1-f53.google.com (mail-ed1-f53.google.com [209.85.208.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C277B8C1E for ; Wed, 10 Apr 2024 08:29:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.53 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712737786; cv=none; b=jFjhLVGSSKhNpdsa2+Xn6G83QyNyGUSSpsp5vXb1HptfFRocktzL305GE0wMWz7AEB53X9pT1R4YBxGIOJQ5TJ240dTcHtdPY+KJQaabbCgCoUCnCEZkOCuwekqJeU7IJ8Q6tk50Lf6NVjdIKqdh3fi9oTvyxjHdFdnt/EdGgec= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712737786; c=relaxed/simple; bh=sf0sDasYfNHboeUmoADUV0lyyWhLkNN4THpXFKwDkII=; h=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type: Content-Disposition; b=kSML64Wvit+exQ4hjpg7h41PrDW39VusBamt0BNatTqo0txo7jef9YoreGxBKuwfIQAmYaF7KdVa3qbK1KqWd9KPdbMQXFEfefUeKLkj7E7VT05JrOjC+Nmqx3kGx8BQtzSWNboN+1pseAdeTtDapCfwI6Ujzz6D5D46OQVuRQs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=yhYfRPM9; arc=none smtp.client-ip=209.85.208.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="yhYfRPM9" Received: by mail-ed1-f53.google.com with SMTP id 4fb4d7f45d1cf-56e2b3e114fso6528428a12.2 for ; Wed, 10 Apr 2024 01:29:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1712737783; x=1713342583; darn=vger.kernel.org; h=content-transfer-encoding:content-disposition:mime-version :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=/NZHDu7gaaGLVzR6sjGpXZZ06n1xRkbPQlKF9FVoL20=; b=yhYfRPM9pUwjpU35r+/vlVqbJEntYp4XYdC1zUq8ZtC0o9xeBXQYPwzZhuoMsxmJqt 0lh4n4ARmKwXDjTN8US/X0fngzf3kJSaZGN3A0C0WjgNdRzffRH2GDpQ8Bk8rjSVhgok 7h4UFXeCoGWmz6lY6+u0nWg5mmcO8/qDoOUDibfyKXezDWOhDSRzWG6FyiMcuIqp5F2x QQhepS4zsLK6prYOnnruzvYfQ6bJsMWQws/FG3Dy87LldJkHKwZYPpBAm4KSr3Fb/WmK 6V9e0L9jEtFeu+EaaNieH3ZwsakTHDTVSWMPTG0tX7kVXby58jqFnndOnkr/GZWDg7X+ h/ZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712737783; x=1713342583; h=content-transfer-encoding:content-disposition:mime-version :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/NZHDu7gaaGLVzR6sjGpXZZ06n1xRkbPQlKF9FVoL20=; b=LfXzRTmGuMuGnF+MUNappMQATAVhJDgC6Ho4I9crGeWVmXbCgNUxSSP6MQiLaTI/R6 q5zZWQzZ4vKlIJdmWoKs0wCF1OIpCD6YvMyPozT/tWaSvDl/R/wJVjJqxsD76+ZvYtyB 1ok+y9ZsVIkPItaNyKtRZnI8fPw4lGoLhfDvVyQfdxknAepI4G98r8E70caED6pkODsi d+PyGz4CXjSYFeWKC12IgG5/6SROa/1pGxxCxlm9G6AcFHZXuXeQarMKOCQ3MePDjVSE Arkfwub6b6Dy84hsuu3O2llyykWx+Ibj5qmpIgyaJyeL0VifD/HGEUKaH2hqoifUz/EV 8ZPg== X-Forwarded-Encrypted: i=1; AJvYcCUaSPGTtSYon1iywgCswT52upo7Vq4vXTfn5vQ2xtUAKLQsrqGLybFsOVNo317joOZx41l/0xryPZ2ZuLxq+VjyC8SG X-Gm-Message-State: AOJu0YzQwS/3ZJf00A3DnMcPFcyZCwOgnVeCaUDpisX9qZxA9trFAFuV pRXWQlSWChJw4Tl0ilBn2TtCpQnVL1ZZ/uqBFwGfUsP4HSwd3D1EnwGcjKlzJQ== X-Google-Smtp-Source: AGHT+IHk06Jj4qi1uxqsJ3TKZTuTcrqUHzkgKhb4lhb70e3Hy/FmNeOV4FhI393R0veGubX5c/yefQ== X-Received: by 2002:a17:906:3554:b0:a51:fa56:4fc7 with SMTP id s20-20020a170906355400b00a51fa564fc7mr1066762eja.21.1712737783099; Wed, 10 Apr 2024 01:29:43 -0700 (PDT) Received: from google.com (61.134.90.34.bc.googleusercontent.com. [34.90.134.61]) by smtp.gmail.com with ESMTPSA id c22-20020a170906529600b00a51bbee7e55sm5299754ejm.53.2024.04.10.01.29.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Apr 2024 01:29:42 -0700 (PDT) Date: Wed, 10 Apr 2024 09:29:39 +0100 From: =?utf-8?q?Pierre-Cl=C3=A9ment?= Tosi To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort Subject: [PATCH v2 08/12] arm64: Move esr_comment() to Message-ID: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline As it is already defined twice and is about to be needed for kCFI error detection, move esr_comment() to a header for re-use, with a clearer name. Signed-off-by: Pierre-Clément Tosi --- arch/arm64/include/asm/esr.h | 5 +++++ arch/arm64/kernel/debug-monitors.c | 4 +--- arch/arm64/kernel/traps.c | 8 +++----- arch/arm64/kvm/handle_exit.c | 2 +- 4 files changed, 10 insertions(+), 9 deletions(-) diff --git a/arch/arm64/include/asm/esr.h b/arch/arm64/include/asm/esr.h index 81606bf7d5ac..2bcf216be376 100644 --- a/arch/arm64/include/asm/esr.h +++ b/arch/arm64/include/asm/esr.h @@ -379,6 +379,11 @@ #ifndef __ASSEMBLY__ #include +static inline unsigned long esr_brk_comment(unsigned long esr) +{ + return esr & ESR_ELx_BRK64_ISS_COMMENT_MASK; +} + static inline bool esr_is_data_abort(unsigned long esr) { const unsigned long ec = ESR_ELx_EC(esr); diff --git a/arch/arm64/kernel/debug-monitors.c b/arch/arm64/kernel/debug-monitors.c index 64f2ecbdfe5c..024a7b245056 100644 --- a/arch/arm64/kernel/debug-monitors.c +++ b/arch/arm64/kernel/debug-monitors.c @@ -312,9 +312,7 @@ static int call_break_hook(struct pt_regs *regs, unsigned long esr) * entirely not preemptible, and we can use rcu list safely here. */ list_for_each_entry_rcu(hook, list, node) { - unsigned long comment = esr & ESR_ELx_BRK64_ISS_COMMENT_MASK; - - if ((comment & ~hook->mask) == hook->imm) + if ((esr_brk_comment(esr) & ~hook->mask) == hook->imm) fn = hook->fn; } diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c index 215e6d7f2df8..2652247032ae 100644 --- a/arch/arm64/kernel/traps.c +++ b/arch/arm64/kernel/traps.c @@ -1105,8 +1105,6 @@ static struct break_hook ubsan_break_hook = { }; #endif -#define esr_comment(esr) ((esr) & ESR_ELx_BRK64_ISS_COMMENT_MASK) - /* * Initial handler for AArch64 BRK exceptions * This handler only used until debug_traps_init(). @@ -1115,15 +1113,15 @@ int __init early_brk64(unsigned long addr, unsigned long esr, struct pt_regs *regs) { #ifdef CONFIG_CFI_CLANG - if ((esr_comment(esr) & ~CFI_BRK_IMM_MASK) == CFI_BRK_IMM_BASE) + if ((esr_brk_comment(esr) & ~CFI_BRK_IMM_MASK) == CFI_BRK_IMM_BASE) return cfi_handler(regs, esr) != DBG_HOOK_HANDLED; #endif #ifdef CONFIG_KASAN_SW_TAGS - if ((esr_comment(esr) & ~KASAN_BRK_MASK) == KASAN_BRK_IMM) + if ((esr_brk_comment(esr) & ~KASAN_BRK_MASK) == KASAN_BRK_IMM) return kasan_handler(regs, esr) != DBG_HOOK_HANDLED; #endif #ifdef CONFIG_UBSAN_TRAP - if ((esr_comment(esr) & ~UBSAN_BRK_MASK) == UBSAN_BRK_IMM) + if ((esr_brk_comment(esr) & ~UBSAN_BRK_MASK) == UBSAN_BRK_IMM) return ubsan_handler(regs, esr) != DBG_HOOK_HANDLED; #endif return bug_handler(regs, esr) != DBG_HOOK_HANDLED; diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c index 617ae6dea5d5..0bcafb3179d6 100644 --- a/arch/arm64/kvm/handle_exit.c +++ b/arch/arm64/kvm/handle_exit.c @@ -395,7 +395,7 @@ void __noreturn __cold nvhe_hyp_panic_handler(u64 esr, u64 spsr, if (mode != PSR_MODE_EL2t && mode != PSR_MODE_EL2h) { kvm_err("Invalid host exception to nVHE hyp!\n"); } else if (ESR_ELx_EC(esr) == ESR_ELx_EC_BRK64 && - (esr & ESR_ELx_BRK64_ISS_COMMENT_MASK) == BUG_BRK_IMM) { + esr_brk_comment(esr) == BUG_BRK_IMM) { const char *file = NULL; unsigned int line = 0; From patchwork Wed Apr 10 08:30:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13623844 Received: from mail-ed1-f42.google.com (mail-ed1-f42.google.com [209.85.208.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9E83013D53B for ; Wed, 10 Apr 2024 08:30:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712737816; cv=none; b=cTGFq9TRKQrYyYWYfmpnKFdvlAWjHWnr2xRIPPGsv6x3QI9Fw06qElqGiPpmOkkw/veCOy8QBqXL+SA+bz7In4Q/TR0NzImMAT7rMVYzs+D1OOh0k+B2baQgAIpCQ9iBgBtmMQgHgat6KIOL638SDuswWiWhKRurihXo8sJjQW0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712737816; c=relaxed/simple; bh=njKwkUA9WJ3YIT0OkI3ALUUZV9vR3gHV7QyHX6tpuOU=; h=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type: Content-Disposition; b=eyK4V7D+H7gZy987tOBB7n8AGxbFOb70+a/UQGK9Hgn9g+mTrBBy5BiDYCN5gJYWvqCgGn8+rSGGWzKwy06fLUruw12FZwcxgigacPqxKpPYllnk0lr8EGubVhkRWoOFGoj9MBSSe3e2hRR5OWskLzhKqnXXXy8BEXZ275KK+L0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=XhVMMS+X; arc=none smtp.client-ip=209.85.208.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="XhVMMS+X" Received: by mail-ed1-f42.google.com with SMTP id 4fb4d7f45d1cf-56e1bbdb362so7871981a12.1 for ; Wed, 10 Apr 2024 01:30:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1712737812; x=1713342612; darn=vger.kernel.org; h=content-transfer-encoding:content-disposition:mime-version :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=XU6xcBIsIab8YJPstad3M2dPguqVYWhFpd43H51xWog=; b=XhVMMS+X94BQwX9dL1puQ84iiq6yAOrtsFyS2oQ7NDKhAg/TVHbTa/xd2r8uRra7+x 4doTiHBDwTxi49WavUkv/cxtpEqPTe+2vKx1rn8hoa9Qaab8vLrOM8PElIijQ2OlgCHV URGSb6+Bzp3q/X0/z3xJ/2HGYTOw+uV/xwm6APOBt2vheP8LNmR0n+MD00wFb19gh5rI P6HWaT7sVjkFf0LUVAuR88yu6hjbKgbhT7MmqwEf4GBj1cWfOjuEKyfJXLgJDur8tMZo Who0fb6kTydH6cvaOOHOvMEEefLtf0mamlUWI9Oo6kJWlQFN8l1Uw0f1EUMkCAs/kcf8 0Khg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712737812; x=1713342612; h=content-transfer-encoding:content-disposition:mime-version :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XU6xcBIsIab8YJPstad3M2dPguqVYWhFpd43H51xWog=; b=YLP8dBdM/V4Xjk8oxRj8GLH9GY9Q4a9zgzwLEsEEJrD1Pa/aEz6WlQWuKXQBi+ZcgF 7JhH7qK4D8VTGX8T/HEQds9+lR3q0n1YYEx3+2R2BM+X/cDuPYopIZtFyduIk6uUUdLX niWKXKzJypvYC4DTFfQ2gQI1VzJGEmc9GxN9jGKVaovVvh49f5JwY3I9LSGF52ZXAG3l NWGhXIy28WtV8iXD55IogzSQpBVX06p9DmZtquUtTjqiHT+SB5byWRZOP94ZlOgzrYQS EiPI2HYb7H5Yux+zftjGBdAT0PBbKq1jJ1T4mHsD6ntf9VJBC5LVE+gptMuluF+iR3Ba A+1A== X-Forwarded-Encrypted: i=1; AJvYcCU4HRU5AvJIx9elUB3y7ta6DV/Twx3rGpGvWjWDylZWTa4SFoDpStZErby0TGPTP+BtCfEO1H0aWp5gcEy+remzBzT8 X-Gm-Message-State: AOJu0Yy+raFjuTvVdOcPq8pdkhwk6NLy3o8XDKjsodPBsnSkxk+uzc+I H9OkmKrxEudFLNBtzZW49g043JFMvO+P/RA/tPALlWpyoWteJrRT1a15mTkGyg== X-Google-Smtp-Source: AGHT+IE4gnHyEEJM1drr58SinQXzdNwje+RjQYL3RSrhfdsN00b2mWNmVd56f/js8snWFGYcCXXaAQ== X-Received: by 2002:a50:8d5e:0:b0:56d:f78f:8747 with SMTP id t30-20020a508d5e000000b0056df78f8747mr1696582edt.16.1712737811724; Wed, 10 Apr 2024 01:30:11 -0700 (PDT) Received: from google.com (61.134.90.34.bc.googleusercontent.com. [34.90.134.61]) by smtp.gmail.com with ESMTPSA id ig4-20020a056402458400b0056e51535a2esm4332477edb.82.2024.04.10.01.30.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Apr 2024 01:30:11 -0700 (PDT) Date: Wed, 10 Apr 2024 09:30:07 +0100 From: =?utf-8?q?Pierre-Cl=C3=A9ment?= Tosi To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort Subject: [PATCH v2 09/12] KVM: arm64: VHE: Add test module for hyp kCFI Message-ID: <4webtguoyuc4yzya6vubduzthj5bemvyufryc7erri7wgexgaf@qjnnow76jkre> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In order to easily periodically (and potentially automatically) validate that the hypervisor kCFI feature doesn't bitrot, introduce a way to trigger hypervisor kCFI faults from userspace on test builds of KVM. Add hooks in the hypervisor code to call registered callbacks (intended to trigger kCFI faults either for the callback call itself of from within the callback function) when running with guest or host VBAR_EL2. As the calls are issued from the KVM_RUN ioctl handling path, userspace gains control over when the actual triggering of the fault happens without needing to modify the KVM uAPI. Export kernel functions to register these callbacks from modules and introduce a kernel module intended to contain any testing logic. By limiting the changes to the core kernel to a strict minimum, this architectural split allows tests to be updated (within the module) without the need to redeploy (or recompile) the kernel (hyp) under test. Use the module parameters as the uAPI for configuring the fault condition being tested (i.e. either at insertion or post-insertion using /sys/module/.../parameters), which naturally makes it impossible for userspace to test kCFI without the module (and, inversely, makes the module only - not KVM - responsible for exposing said uAPI). As kCFI is implemented with a caller-side check of a callee-side value, make the module support 4 tests based on the location of the caller and callee (built-in or in-module), for each of the 2 hypervisor contexts (host & guest), selected by userspace using the 'guest' or 'host' module parameter. For this purpose, export symbols which the module can use to configure the callbacks for in-kernel and module-to-built-in kCFI faulting calls. Define the module-to-kernel API to allow the module to detect that it was loaded on a kernel built with support for it but which is running without a hypervisor (-ENXIO) or with one that doesn't use the VHE CPU feature (-EOPNOTSUPP), which is currently the only mode for which KVM supports hypervisor kCFI. Allow kernel build configs to set CONFIG_HYP_CFI_TEST to only support the in-kernel hooks (=y) or also build the test module (=m). Use intermediate internal Kconfig flags (CONFIG_HYP_SUPPORTS_CFI_TEST and CONFIG_HYP_CFI_TEST_MODULE) to simplify the Makefiles and #ifdefs. As the symbols for callback registration are only exported to modules when CONFIG_HYP_CFI_TEST != n, it is impossible for the test module to be non-forcefully inserted on a kernel that doesn't support it. Note that this feature must NOT result in any noticeable change (behavioral or binary size) when HYP_CFI_TEST_MODULE = n. CONFIG_HYP_CFI_TEST is intentionally independent of CONFIG_CFI_CLANG, to avoid arbitrarily limiting the number of flag combinations that can be tested with the module. Also note that, as VHE aliases VBAR_EL1 to VBAR_EL2 for the host, testing hypervisor kCFI in VHE and in host context is equivalent to testing kCFI support of the kernel itself i.e. EL1 in non-VHE and/or in non-virtualized environments. For this reason, CONFIG_CFI_PERMISSIVE **will** prevent the test module from triggering a hyp panic (although a warning still gets printed) in that context. Signed-off-by: Pierre-Clément Tosi --- arch/arm64/include/asm/kvm_cfi.h | 36 ++++++++ arch/arm64/kvm/Kconfig | 22 +++++ arch/arm64/kvm/Makefile | 3 + arch/arm64/kvm/hyp/include/hyp/cfi.h | 47 ++++++++++ arch/arm64/kvm/hyp/vhe/Makefile | 1 + arch/arm64/kvm/hyp/vhe/cfi.c | 37 ++++++++ arch/arm64/kvm/hyp/vhe/switch.c | 7 ++ arch/arm64/kvm/hyp_cfi_test.c | 43 +++++++++ arch/arm64/kvm/hyp_cfi_test_module.c | 133 +++++++++++++++++++++++++++ 9 files changed, 329 insertions(+) create mode 100644 arch/arm64/include/asm/kvm_cfi.h create mode 100644 arch/arm64/kvm/hyp/include/hyp/cfi.h create mode 100644 arch/arm64/kvm/hyp/vhe/cfi.c create mode 100644 arch/arm64/kvm/hyp_cfi_test.c create mode 100644 arch/arm64/kvm/hyp_cfi_test_module.c diff --git a/arch/arm64/include/asm/kvm_cfi.h b/arch/arm64/include/asm/kvm_cfi.h new file mode 100644 index 000000000000..13cc7b19d838 --- /dev/null +++ b/arch/arm64/include/asm/kvm_cfi.h @@ -0,0 +1,36 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2024 - Google Inc + * Author: Pierre-Clément Tosi + */ + +#ifndef __ARM64_KVM_CFI_H__ +#define __ARM64_KVM_CFI_H__ + +#include +#include + +#ifdef CONFIG_HYP_SUPPORTS_CFI_TEST + +int kvm_cfi_test_register_host_ctxt_cb(void (*cb)(void)); +int kvm_cfi_test_register_guest_ctxt_cb(void (*cb)(void)); + +#else + +static inline int kvm_cfi_test_register_host_ctxt_cb(void (*cb)(void)) +{ + return -EOPNOTSUPP; +} + +static inline int kvm_cfi_test_register_guest_ctxt_cb(void (*cb)(void)) +{ + return -EOPNOTSUPP; +} + +#endif /* CONFIG_HYP_SUPPORTS_CFI_TEST */ + +/* Symbols which the host can register as hyp callbacks; see . */ +void hyp_trigger_builtin_cfi_fault(void); +void hyp_builtin_cfi_fault_target(int unused); + +#endif /* __ARM64_KVM_CFI_H__ */ diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig index 58f09370d17e..5daa8079a120 100644 --- a/arch/arm64/kvm/Kconfig +++ b/arch/arm64/kvm/Kconfig @@ -65,4 +65,26 @@ config PROTECTED_NVHE_STACKTRACE If unsure, or not using protected nVHE (pKVM), say N. +config HYP_CFI_TEST + tristate "KVM hypervisor kCFI test support" + depends on KVM + help + Say Y or M here to build KVM with test hooks to support intentionally + triggering hypervisor kCFI faults in guest or host context. + + Say M here to also build a module which registers callbacks triggering + faults and selected by userspace through its parameters. + + Note that this feature is currently only supported in VHE mode. + + If unsure, say N. + +config HYP_SUPPORTS_CFI_TEST + def_bool y + depends on HYP_CFI_TEST + +config HYP_CFI_TEST_MODULE + def_tristate m if HYP_CFI_TEST = m + depends on HYP_CFI_TEST + endif # VIRTUALIZATION diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile index c0c050e53157..d42540ae3ea7 100644 --- a/arch/arm64/kvm/Makefile +++ b/arch/arm64/kvm/Makefile @@ -22,6 +22,7 @@ kvm-y += arm.o mmu.o mmio.o psci.o hypercalls.o pvtime.o \ vgic/vgic-mmio-v3.o vgic/vgic-kvm-device.o \ vgic/vgic-its.o vgic/vgic-debug.o +kvm-$(CONFIG_HYP_SUPPORTS_CFI_TEST) += hyp_cfi_test.o kvm-$(CONFIG_HW_PERF_EVENTS) += pmu-emul.o pmu.o always-y := hyp_constants.h hyp-constants.s @@ -39,3 +40,5 @@ $(obj)/hyp_constants.h: $(obj)/hyp-constants.s FORCE obj-kvm := $(addprefix $(obj)/, $(kvm-y)) $(obj-kvm): $(obj)/hyp_constants.h + +obj-$(CONFIG_HYP_CFI_TEST_MODULE) += hyp_cfi_test_module.o diff --git a/arch/arm64/kvm/hyp/include/hyp/cfi.h b/arch/arm64/kvm/hyp/include/hyp/cfi.h new file mode 100644 index 000000000000..c6536040bc06 --- /dev/null +++ b/arch/arm64/kvm/hyp/include/hyp/cfi.h @@ -0,0 +1,47 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2024 - Google Inc + * Author: Pierre-Clément Tosi + */ + +#ifndef __ARM64_KVM_HYP_CFI_H__ +#define __ARM64_KVM_HYP_CFI_H__ + +#include +#include + +#include + +#ifdef CONFIG_HYP_SUPPORTS_CFI_TEST + +int __kvm_register_cfi_test_cb(void (*cb)(void), bool in_host_ctxt); + +extern void (*hyp_test_host_ctxt_cfi)(void); +extern void (*hyp_test_guest_ctxt_cfi)(void); + +/* Hypervisor callbacks for the host to register. */ +void hyp_trigger_builtin_cfi_fault(void); +void hyp_builtin_cfi_fault_target(int unused); + +#else + +static inline +int __kvm_register_cfi_test_cb(void (*cb)(void), bool in_host_ctxt) +{ + return -EOPNOTSUPP; +} + +#define hyp_test_host_ctxt_cfi ((void(*)(void))(NULL)) +#define hyp_test_guest_ctxt_cfi ((void(*)(void))(NULL)) + +static inline void hyp_trigger_builtin_cfi_fault(void) +{ +} + +static inline void hyp_builtin_cfi_fault_target(int __always_unused unused) +{ +} + +#endif /* CONFIG_HYP_SUPPORTS_CFI_TEST */ + +#endif /* __ARM64_KVM_HYP_CFI_H__ */ diff --git a/arch/arm64/kvm/hyp/vhe/Makefile b/arch/arm64/kvm/hyp/vhe/Makefile index 3b9e5464b5b3..19ca584cc21e 100644 --- a/arch/arm64/kvm/hyp/vhe/Makefile +++ b/arch/arm64/kvm/hyp/vhe/Makefile @@ -9,3 +9,4 @@ ccflags-y := -D__KVM_VHE_HYPERVISOR__ obj-y := timer-sr.o sysreg-sr.o debug-sr.o switch.o tlb.o obj-y += ../vgic-v3-sr.o ../aarch32.o ../vgic-v2-cpuif-proxy.o ../entry.o \ ../fpsimd.o ../hyp-entry.o ../exception.o +obj-$(CONFIG_HYP_SUPPORTS_CFI_TEST) += cfi.o diff --git a/arch/arm64/kvm/hyp/vhe/cfi.c b/arch/arm64/kvm/hyp/vhe/cfi.c new file mode 100644 index 000000000000..5849f239e27f --- /dev/null +++ b/arch/arm64/kvm/hyp/vhe/cfi.c @@ -0,0 +1,37 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2024 - Google Inc + * Author: Pierre-Clément Tosi + */ +#include + +#include + +void (*hyp_test_host_ctxt_cfi)(void); +void (*hyp_test_guest_ctxt_cfi)(void); + +int __kvm_register_cfi_test_cb(void (*cb)(void), bool in_host_ctxt) +{ + if (in_host_ctxt) + hyp_test_host_ctxt_cfi = cb; + else + hyp_test_guest_ctxt_cfi = cb; + + return 0; +} + +void hyp_builtin_cfi_fault_target(int __always_unused unused) +{ +} + +void hyp_trigger_builtin_cfi_fault(void) +{ + /* Intentional UB cast & dereference, to trigger a kCFI fault. */ + void (*target)(void) = (void *)&hyp_builtin_cfi_fault_target; + + /* + * READ_ONCE() prevents this indirect call from being optimized out, + * forcing the compiler to generate the kCFI check before the branch. + */ + READ_ONCE(target)(); +} diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c index 9db04a286398..b3268933b093 100644 --- a/arch/arm64/kvm/hyp/vhe/switch.c +++ b/arch/arm64/kvm/hyp/vhe/switch.c @@ -4,6 +4,7 @@ * Author: Marc Zyngier */ +#include #include #include @@ -221,6 +222,9 @@ static int __kvm_vcpu_run_vhe(struct kvm_vcpu *vcpu) struct kvm_cpu_context *guest_ctxt; u64 exit_code; + if (IS_ENABLED(CONFIG_HYP_SUPPORTS_CFI_TEST) && unlikely(hyp_test_host_ctxt_cfi)) + hyp_test_host_ctxt_cfi(); + host_ctxt = &this_cpu_ptr(&kvm_host_data)->host_ctxt; host_ctxt->__hyp_running_vcpu = vcpu; guest_ctxt = &vcpu->arch.ctxt; @@ -245,6 +249,9 @@ static int __kvm_vcpu_run_vhe(struct kvm_vcpu *vcpu) else vcpu_clear_flag(vcpu, VCPU_HYP_CONTEXT); + if (IS_ENABLED(CONFIG_HYP_SUPPORTS_CFI_TEST) && unlikely(hyp_test_guest_ctxt_cfi)) + hyp_test_guest_ctxt_cfi(); + do { /* Jump in the fire! */ exit_code = __guest_enter(vcpu); diff --git a/arch/arm64/kvm/hyp_cfi_test.c b/arch/arm64/kvm/hyp_cfi_test.c new file mode 100644 index 000000000000..da7b25ca1b1f --- /dev/null +++ b/arch/arm64/kvm/hyp_cfi_test.c @@ -0,0 +1,43 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2024 - Google Inc + * Author: Pierre-Clément Tosi + */ +#include +#include +#include +#include + +#include +#include +#include + +/* For calling directly into the VHE hypervisor; see . */ +int __kvm_register_cfi_test_cb(void (*)(void), bool); + +static int kvm_register_cfi_test_cb(void (*vhe_cb)(void), bool in_host_ctxt) +{ + if (!is_hyp_mode_available()) + return -ENXIO; + + if (is_hyp_nvhe()) + return -EOPNOTSUPP; + + return __kvm_register_cfi_test_cb(vhe_cb, in_host_ctxt); +} + +int kvm_cfi_test_register_host_ctxt_cb(void (*cb)(void)) +{ + return kvm_register_cfi_test_cb(cb, true); +} +EXPORT_SYMBOL(kvm_cfi_test_register_host_ctxt_cb); + +int kvm_cfi_test_register_guest_ctxt_cb(void (*cb)(void)) +{ + return kvm_register_cfi_test_cb(cb, false); +} +EXPORT_SYMBOL(kvm_cfi_test_register_guest_ctxt_cb); + +/* Hypervisor callbacks for the test module to register. */ +EXPORT_SYMBOL(hyp_trigger_builtin_cfi_fault); +EXPORT_SYMBOL(hyp_builtin_cfi_fault_target); diff --git a/arch/arm64/kvm/hyp_cfi_test_module.c b/arch/arm64/kvm/hyp_cfi_test_module.c new file mode 100644 index 000000000000..eeda4be4d3ef --- /dev/null +++ b/arch/arm64/kvm/hyp_cfi_test_module.c @@ -0,0 +1,133 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2024 - Google Inc + * Author: Pierre-Clément Tosi + */ +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt + +#include +#include +#include + +#include +#include +#include +#include + +static int set_host_mode(const char *val, const struct kernel_param *kp); +static int set_guest_mode(const char *val, const struct kernel_param *kp); + +#define M_DESC \ + "\n\t0: none" \ + "\n\t1: built-in caller & built-in callee" \ + "\n\t2: built-in caller & module callee" \ + "\n\t3: module caller & built-in callee" \ + "\n\t4: module caller & module callee" + +static unsigned int host_mode; +module_param_call(host, set_host_mode, param_get_uint, &host_mode, 0644); +MODULE_PARM_DESC(host, + "Hypervisor kCFI fault test case in host context:" M_DESC); + +static unsigned int guest_mode; +module_param_call(guest, set_guest_mode, param_get_uint, &guest_mode, 0644); +MODULE_PARM_DESC(guest, + "Hypervisor kCFI fault test case in guest context:" M_DESC); + +static void trigger_module2module_cfi_fault(void); +static void trigger_module2builtin_cfi_fault(void); +static void hyp_cfi_module2module_test_target(int); +static void hyp_cfi_builtin2module_test_target(int); + +static int set_param_mode(const char *val, const struct kernel_param *kp, + int (*register_cb)(void (*)(void))) +{ + unsigned int *mode = kp->arg; + int err; + + err = param_set_uint(val, kp); + if (err) + return err; + + switch (*mode) { + case 0: + return register_cb(NULL); + case 1: + return register_cb(hyp_trigger_builtin_cfi_fault); + case 2: + return register_cb((void *)hyp_cfi_builtin2module_test_target); + case 3: + return register_cb(trigger_module2builtin_cfi_fault); + case 4: + return register_cb(trigger_module2module_cfi_fault); + default: + return -EINVAL; + } +} + +static int set_host_mode(const char *val, const struct kernel_param *kp) +{ + return set_param_mode(val, kp, kvm_cfi_test_register_host_ctxt_cb); +} + +static int set_guest_mode(const char *val, const struct kernel_param *kp) +{ + return set_param_mode(val, kp, kvm_cfi_test_register_guest_ctxt_cb); +} + +static void __exit exit_hyp_cfi_test(void) +{ + int err; + + err = kvm_cfi_test_register_host_ctxt_cb(NULL); + if (err) + pr_err("Failed to unregister host context trigger: %d\n", err); + + err = kvm_cfi_test_register_guest_ctxt_cb(NULL); + if (err) + pr_err("Failed to unregister guest context trigger: %d\n", err); +} +module_exit(exit_hyp_cfi_test); + +static void trigger_module2builtin_cfi_fault(void) +{ + /* Intentional UB cast & dereference, to trigger a kCFI fault. */ + void (*target)(void) = (void *)&hyp_builtin_cfi_fault_target; + + /* + * READ_ONCE() prevents this indirect call from being optimized out, + * forcing the compiler to generate the kCFI check before the branch. + */ + READ_ONCE(target)(); + + pr_err_ratelimited("%s: Survived a kCFI violation\n", __func__); +} + +static void trigger_module2module_cfi_fault(void) +{ + /* Intentional UB cast & dereference, to trigger a kCFI fault. */ + void (*target)(void) = (void *)&hyp_cfi_module2module_test_target; + + /* + * READ_ONCE() prevents this indirect call from being optimized out, + * forcing the compiler to generate the kCFI check before the branch. + */ + READ_ONCE(target)(); + + pr_err_ratelimited("%s: Survived a kCFI violation\n", __func__); +} + +/* Use different functions, for clearer symbols in kCFI panic reports. */ +static noinline +void hyp_cfi_module2module_test_target(int __always_unused unused) +{ +} + +static noinline +void hyp_cfi_builtin2module_test_target(int __always_unused unused) +{ +} + +MODULE_LICENSE("GPL"); +MODULE_AUTHOR("Pierre-Clément Tosi "); +MODULE_DESCRIPTION("KVM hypervisor kCFI test module"); From patchwork Wed Apr 10 08:30:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13623845 Received: from mail-ej1-f52.google.com (mail-ej1-f52.google.com [209.85.218.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8717913D880 for ; Wed, 10 Apr 2024 08:30:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712737828; cv=none; b=DbNw/WRQnx4pH/p7JUPbeb+1C8dSScAxkyKSkLp4cRH7qr3DFDENSwOI0/lsQYgjbveoY3Krxo7Kh9mfZUwlMvBzwmVN9ND7e4bYdAaYnquaGXMEoGio/7g8Pfvj1LUQpYpRSbNVNL8TeMefcqfBiehVae5XeF1zToI3i++w8bU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712737828; c=relaxed/simple; bh=sjLo+2j08iLCyaP/ZrrXRs8M6s13hs4gKMe6/iy5SyA=; h=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type: Content-Disposition; b=FI0XFM9kSrLjIvWHPt7HHU9eh8FuiQkT6a/c+9dv6TQvDzRPlWnTdKJrwChrjb1AjmFnu0gHYhQwZ6klzKt+YZxkWlKs9J6zBWIrYdlgw1582yF9krwfbXWHVwgDpml5OppJ7RmHL7+WKDIPkdbwlbETFlDCYIQpy1jC+UZhk1E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=4GHy/91n; arc=none smtp.client-ip=209.85.218.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="4GHy/91n" Received: by mail-ej1-f52.google.com with SMTP id a640c23a62f3a-a51c6e91793so573987166b.0 for ; Wed, 10 Apr 2024 01:30:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1712737825; x=1713342625; darn=vger.kernel.org; h=content-transfer-encoding:content-disposition:mime-version :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=v6DrtWKL4HFIVL/Z+tlpZfy75B2ArHvwYVgq7qkQ5Eg=; b=4GHy/91n6yePYMaSwdJJojQnpaAkzjcZSVSfUa0sqCIG2tMFQeGZc/Ovc/ruMyk2P6 LZHE8x++bhWfO0Nijcf8R8Rpn20yLf78GAlFlzWpcswqncV3ZvB9vllp5fTPM/Rx0I9o US/UhttFu7UIRub40gjarw0wYEYXKXPPfTegsT7eVHXqjPeNC9Arkw7MCh8Hm7t/GZhZ 2Ebyn9d7QcicPd9F3S5kGw0fnQGq4LyUCJ9SOI2NwVPvTn9Yq3fJkND6VuDMtmO4aDMG P+1HqI6sUufYf0IdXxGfSXXpZ0V701s97I5H2jSRRR3Mp2Fp6EKxNjoG6noUYY9FuPrn FEiQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712737825; x=1713342625; h=content-transfer-encoding:content-disposition:mime-version :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=v6DrtWKL4HFIVL/Z+tlpZfy75B2ArHvwYVgq7qkQ5Eg=; b=pbPqG7qkWmwHSKxBcTuuCEgsoovBY7OxU+CwOCJn8/pkw4SS9a5cO/4ZCMkysg3mLm cB5wrGxJ6ae5NIu2ZkdtgavEcbs35fYuEHmpuRW1JihnlUFdg7uvE2j6ghrtVVsArBnZ fN+wO3w2QwbzgqNmsfod24bD8B+9am+bU9nYgHnprWsuVyBp9DY0EN/mOycQt2kfPMZB eIW0EpvBwtXFh6dXF/AFCkJGtNcXTz2WRXjiNJzHpv3ElbDQBP6cFB5byC1M1RFGCUID eoSL+0HroxBzm9qZ8y8JTSf7kxDO6rhMK5/7gt/ykL9eb5lK+lodNFYFXiabT1RYbN8U ZL2w== X-Forwarded-Encrypted: i=1; AJvYcCXDyG8QWg7L4TnELw4fqgWsbOT/3gHGrBpkq/JTPPq6Ok+PtOt0v0qesczjSJSzYd9WpLBMPyFtgv4rnZAjTNnFztbF X-Gm-Message-State: AOJu0YyoyTXk1+cTwNRCqBuzFRVKPSwbJPyUip1eUpJuW6JnCGIvLTtm KKo4sr8yWG3s6U06LR3L/XNgRW7Rmshz03CBbg0gLE5D3hOaTLuqQz2RdAYifg== X-Google-Smtp-Source: AGHT+IE+3bFNql+D7KkAv08EKtihoMoVNO9YFCZha/FUwb6uG6uieK7br/du5MzzN0cCL5aBqVM6xQ== X-Received: by 2002:a17:907:8686:b0:a52:b70:454f with SMTP id qa6-20020a170907868600b00a520b70454fmr1429337ejc.15.1712737824552; Wed, 10 Apr 2024 01:30:24 -0700 (PDT) Received: from google.com (61.134.90.34.bc.googleusercontent.com. [34.90.134.61]) by smtp.gmail.com with ESMTPSA id de26-20020a056402309a00b0056c1cca33bfsm6129511edb.6.2024.04.10.01.30.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Apr 2024 01:30:24 -0700 (PDT) Date: Wed, 10 Apr 2024 09:30:20 +0100 From: =?utf-8?q?Pierre-Cl=C3=A9ment?= Tosi To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort Subject: [PATCH v2 10/12] KVM: arm64: nVHE: Support CONFIG_CFI_CLANG at EL2 Message-ID: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline The compiler implements kCFI by adding type information (u32) above every function that might be indirectly called and, whenever a function pointer is called, injects a read-and-compare of that u32 against the value corresponding to the expected type. In case of a mismatch, a BRK instruction gets executed. When the hypervisor triggers such an exception in nVHE, it panics and triggers and exception return to EL1. Therefore, teach nvhe_hyp_panic_handler() to detect kCFI errors from the ESR and report them. If necessary, remind the user that EL2 kCFI is not affected by CONFIG_CFI_PERMISSIVE. Pass $(CC_FLAGS_CFI) to the compiler when building the nVHE hyp code. Use SYM_TYPED_FUNC_START() for __pkvm_init_switch_pgd, as nVHE can't call it directly and must use a PA function pointer from C (because it is part of the idmap page), which would trigger a kCFI failure if the type ID wasn't present. Signed-off-by: Pierre-Clément Tosi --- arch/arm64/include/asm/esr.h | 6 ++++++ arch/arm64/kvm/handle_exit.c | 11 +++++++++++ arch/arm64/kvm/hyp/nvhe/Makefile | 6 +++--- arch/arm64/kvm/hyp/nvhe/hyp-init.S | 6 +++++- 4 files changed, 25 insertions(+), 4 deletions(-) diff --git a/arch/arm64/include/asm/esr.h b/arch/arm64/include/asm/esr.h index 2bcf216be376..9eb9e6aa70cf 100644 --- a/arch/arm64/include/asm/esr.h +++ b/arch/arm64/include/asm/esr.h @@ -391,6 +391,12 @@ static inline bool esr_is_data_abort(unsigned long esr) return ec == ESR_ELx_EC_DABT_LOW || ec == ESR_ELx_EC_DABT_CUR; } +static inline bool esr_is_cfi_brk(unsigned long esr) +{ + return ESR_ELx_EC(esr) == ESR_ELx_EC_BRK64 && + (esr_brk_comment(esr) & ~CFI_BRK_IMM_MASK) == CFI_BRK_IMM_BASE; +} + static inline bool esr_fsc_is_translation_fault(unsigned long esr) { /* Translation fault, level -1 */ diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c index 0bcafb3179d6..0db23a6304ce 100644 --- a/arch/arm64/kvm/handle_exit.c +++ b/arch/arm64/kvm/handle_exit.c @@ -383,6 +383,15 @@ void handle_exit_early(struct kvm_vcpu *vcpu, int exception_index) kvm_handle_guest_serror(vcpu, kvm_vcpu_get_esr(vcpu)); } +static void kvm_nvhe_report_cfi_failure(u64 panic_addr) +{ + kvm_err("nVHE hyp CFI failure at: [<%016llx>] %pB!\n", panic_addr, + (void *)(panic_addr + kaslr_offset())); + + if (IS_ENABLED(CONFIG_CFI_PERMISSIVE)) + kvm_err(" (CONFIG_CFI_PERMISSIVE ignored for hyp failures)\n"); +} + void __noreturn __cold nvhe_hyp_panic_handler(u64 esr, u64 spsr, u64 elr_virt, u64 elr_phys, u64 par, uintptr_t vcpu, @@ -413,6 +422,8 @@ void __noreturn __cold nvhe_hyp_panic_handler(u64 esr, u64 spsr, else kvm_err("nVHE hyp BUG at: [<%016llx>] %pB!\n", panic_addr, (void *)(panic_addr + kaslr_offset())); + } else if (IS_ENABLED(CONFIG_CFI_CLANG) && esr_is_cfi_brk(esr)) { + kvm_nvhe_report_cfi_failure(panic_addr); } else { kvm_err("nVHE hyp panic at: [<%016llx>] %pB!\n", panic_addr, (void *)(panic_addr + kaslr_offset())); diff --git a/arch/arm64/kvm/hyp/nvhe/Makefile b/arch/arm64/kvm/hyp/nvhe/Makefile index 2250253a6429..2eb915d8943f 100644 --- a/arch/arm64/kvm/hyp/nvhe/Makefile +++ b/arch/arm64/kvm/hyp/nvhe/Makefile @@ -89,9 +89,9 @@ quiet_cmd_hyprel = HYPREL $@ quiet_cmd_hypcopy = HYPCOPY $@ cmd_hypcopy = $(OBJCOPY) --prefix-symbols=__kvm_nvhe_ $< $@ -# Remove ftrace, Shadow Call Stack, and CFI CFLAGS. -# This is equivalent to the 'notrace', '__noscs', and '__nocfi' annotations. -KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_FTRACE) $(CC_FLAGS_SCS) $(CC_FLAGS_CFI), $(KBUILD_CFLAGS)) +# Remove ftrace and Shadow Call Stack CFLAGS. +# This is equivalent to the 'notrace' and '__noscs' annotations. +KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_FTRACE) $(CC_FLAGS_SCS), $(KBUILD_CFLAGS)) # Starting from 13.0.0 llvm emits SHT_REL section '.llvm.call-graph-profile' # when profile optimization is applied. gen-hyprel does not support SHT_REL and # causes a build failure. Remove profile optimization flags. diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-init.S b/arch/arm64/kvm/hyp/nvhe/hyp-init.S index 5a15737b4233..33fb5732ab83 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-init.S +++ b/arch/arm64/kvm/hyp/nvhe/hyp-init.S @@ -5,6 +5,7 @@ */ #include +#include #include #include @@ -268,8 +269,11 @@ SYM_CODE_END(__kvm_handle_stub_hvc) /* * void __pkvm_init_switch_pgd(struct kvm_nvhe_init_params *params, * void (*finalize_fn)(void)); + * + * SYM_TYPED_FUNC_START() allows C to call this ID-mapped function indirectly + * using a physical pointer without triggering a kCFI failure. */ -SYM_FUNC_START(__pkvm_init_switch_pgd) +SYM_TYPED_FUNC_START(__pkvm_init_switch_pgd) /* Load the inputs from the VA pointer before turning the MMU off */ ldr x5, [x0, #NVHE_INIT_PGD_PA] ldr x0, [x0, #NVHE_INIT_STACK_HYP_VA] From patchwork Wed Apr 10 08:31:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13623846 Received: from mail-ej1-f41.google.com (mail-ej1-f41.google.com [209.85.218.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F155713F01B for ; Wed, 10 Apr 2024 08:31:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712737881; cv=none; b=OgH3w8VGeh9R1GGB+6RCj+G+QQ6ix8pUz6qwS9xeYvKd9fkS3eDH/e3iTL/b1kEIrx+unn99aY6EqUd1AMWORVyOFvHcUkc+wIvnj/hc0KyUYHa2OkvEoeWKLRe7OGzxP0E70LLRAR0Q2/LuDHu9LZ2p1yUV4NH/B1v3+8VY9HU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712737881; c=relaxed/simple; bh=cpzOqPqp2FWk5BTnYrCSS1LW18sAwwLHJ+v0bKY6LVk=; h=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type: Content-Disposition; b=N21otRbxgQli1VzaHWvkVGkZYNYVoh0AN0jvHzxx0KCdqIyA9qNg42oYzeimsaRb+RbXhhJcsBIzdKsO0HaqhaTYELpomD0g2Yz6aDhX91fmMOINCvY3Vps0B9k8GHGbvqHbbzHH3nSPvMw7D3DwuLyM/Fq9dUzDh/XN/eJiQ0Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=4YXPFTZ4; arc=none smtp.client-ip=209.85.218.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="4YXPFTZ4" Received: by mail-ej1-f41.google.com with SMTP id a640c23a62f3a-a51fc011e8fso179553466b.0 for ; Wed, 10 Apr 2024 01:31:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1712737877; x=1713342677; darn=vger.kernel.org; h=content-transfer-encoding:content-disposition:mime-version :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=RG+NK3kQHpQlT1WXed325+BJ1wFsJ9Qvvp1+whP8THg=; b=4YXPFTZ4z8LAHb3zaqyEevi3zyhazyZflsxZNISJ+thEM3+db1ybdKgX7H0kD9IVOd GNwMFHbsQlxPpJtQoOmf1Bh9wuNStXxdR3UBEQQaMWRmpsDetnA1oDPrui3Wl0Y8ElDX XNEePpslt/xw6D42Pa6Iptl0xx0MY75NAQGVxu5IA1Gew2aDcfQ3eStpxEIfBSTmOQug fKdyLX/zUO1oNF8qCGdlWvCew5ERu/LBBGIvNLjttV/GVDY4OapKTpW9xGEvsR3MhZxX VmtPVncoEE5VFiDWMIyTtLUHKMBRsoFViqsgVMGChPMWlrbr/SNd95tNTlitDgFX0fnD xYBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712737877; x=1713342677; h=content-transfer-encoding:content-disposition:mime-version :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RG+NK3kQHpQlT1WXed325+BJ1wFsJ9Qvvp1+whP8THg=; b=EzBxEGHAqwj8mxFM1zW8QP5TvRwftcpReYmDVYdtEZQtKRSsVNKt46Mk2lOLLMf8Tr aIQD7JcyjZ4oREo9RgDdVeKIWAIVqB0bIuL3y58KwNKesH/a3QFqkztz7ZOrE/YUYw3k pHKKpPfLFNpIAN7ktFG/pCkgWUJO8ZpGXvr8+TBoQCdBf6SVaeCnZLP2oYx+FsbMP5ak bsuOalde6n01REVYJfvcyYNB1dIQBqjwTTVxEhftNFF3FLFL0WtAHDsp9SRUUTEvM/xW 8eZNQH4LQ1xMQF+QKvmOTrUEOkhJAawhnVcBa4AXtkyOtitxLX6KAKHstfbJkC+GwUw6 LY0g== X-Forwarded-Encrypted: i=1; AJvYcCX7r2utQjyreldElAMzJABCPli3DU63E/1gApR7mPr1yX/0JeVztNsBoviFlPYHvvhIHDyDlKAvlmeHlpyCYS+dhhg5 X-Gm-Message-State: AOJu0YznvsY/NUeKCmPtdrufs+crWctsG0f9WRZYWK8aLolQH3jpjPal T0qzSgHt1XQ+9jeBLhwFXZl6Ect5WUw21m4Cjm7QZhQ2+CojPkmxxr71OJyawg== X-Google-Smtp-Source: AGHT+IHsVDjhqLYz4R4KRsAm/rlIfcBL0hwv/AlVUy4a35ojorEEOv6cI9kzisNnwpCqevT93PAxaQ== X-Received: by 2002:a17:907:968b:b0:a51:d235:74cf with SMTP id hd11-20020a170907968b00b00a51d23574cfmr1681270ejc.38.1712737876848; Wed, 10 Apr 2024 01:31:16 -0700 (PDT) Received: from google.com (61.134.90.34.bc.googleusercontent.com. [34.90.134.61]) by smtp.gmail.com with ESMTPSA id ml16-20020a170906cc1000b00a4e670414ffsm6701933ejb.109.2024.04.10.01.31.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Apr 2024 01:31:16 -0700 (PDT) Date: Wed, 10 Apr 2024 09:31:12 +0100 From: =?utf-8?q?Pierre-Cl=C3=A9ment?= Tosi To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort Subject: [PATCH v2 11/12] KVM: arm64: nVHE: Support test module for hyp kCFI Message-ID: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline Extend support for the kCFI test module to nVHE by replicating the hooks on the KVM_RUN handler path currently existing in VHE in the nVHE code, exporting the equivalent callback targets for triggering built-in hyp kCFI faults, and exposing a new CONFIG_HYP_CFI_TEST-only host HVC to implement callback registration with pKVM. Update the test module to register the nVHE equivalent callback for test case '1' (i.e. both EL2 hyp caller and callee are built-in) and document that other cases are not supported outside of VHE, as they require EL2 symbols in the module, which is not currently supported for nVHE. Note that a kernel in protected mode that doesn't support HYP_CFI_TEST will prevent the module from registering nVHE callbacks both by not exporting the necessary symbols (similar to VHE) but also by rejecting the corresponding HVC, if the module tries to issue it directly. Also note that the test module will run in pKVM (with HYP_CFI_TEST) independently of other debug Kconfig flags but that not stacktrace will be printed without PROTECTED_NVHE_STACKTRACE. This allows testing kCFI under conditions closer to release builds, if desired. Signed-off-by: Pierre-Clément Tosi --- arch/arm64/include/asm/kvm_asm.h | 3 ++ arch/arm64/include/asm/kvm_cfi.h | 6 ++-- arch/arm64/kvm/Kconfig | 2 -- arch/arm64/kvm/hyp/{vhe => }/cfi.c | 0 arch/arm64/kvm/hyp/nvhe/Makefile | 1 + arch/arm64/kvm/hyp/nvhe/hyp-main.c | 19 ++++++++++++ arch/arm64/kvm/hyp/nvhe/switch.c | 7 +++++ arch/arm64/kvm/hyp/vhe/Makefile | 2 +- arch/arm64/kvm/hyp_cfi_test.c | 44 ++++++++++++++++++++++++---- arch/arm64/kvm/hyp_cfi_test_module.c | 24 ++++++++------- 10 files changed, 86 insertions(+), 22 deletions(-) rename arch/arm64/kvm/hyp/{vhe => }/cfi.c (100%) diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h index 24b5e6b23417..3256c91ff234 100644 --- a/arch/arm64/include/asm/kvm_asm.h +++ b/arch/arm64/include/asm/kvm_asm.h @@ -81,6 +81,9 @@ enum __kvm_host_smccc_func { __KVM_HOST_SMCCC_FUNC___pkvm_init_vm, __KVM_HOST_SMCCC_FUNC___pkvm_init_vcpu, __KVM_HOST_SMCCC_FUNC___pkvm_teardown_vm, +#ifdef CONFIG_HYP_SUPPORTS_CFI_TEST + __KVM_HOST_SMCCC_FUNC___kvm_register_cfi_test_cb, +#endif }; #define DECLARE_KVM_VHE_SYM(sym) extern char sym[] diff --git a/arch/arm64/include/asm/kvm_cfi.h b/arch/arm64/include/asm/kvm_cfi.h index 13cc7b19d838..ed6422eebce5 100644 --- a/arch/arm64/include/asm/kvm_cfi.h +++ b/arch/arm64/include/asm/kvm_cfi.h @@ -12,8 +12,8 @@ #ifdef CONFIG_HYP_SUPPORTS_CFI_TEST -int kvm_cfi_test_register_host_ctxt_cb(void (*cb)(void)); -int kvm_cfi_test_register_guest_ctxt_cb(void (*cb)(void)); +int kvm_cfi_test_register_host_ctxt_cb(void (*vhe_cb)(void), void *nvhe_cb); +int kvm_cfi_test_register_guest_ctxt_cb(void (*vhe_cb)(void), void *nvhe_cb); #else @@ -31,6 +31,8 @@ static inline int kvm_cfi_test_register_guest_ctxt_cb(void (*cb)(void)) /* Symbols which the host can register as hyp callbacks; see . */ void hyp_trigger_builtin_cfi_fault(void); +DECLARE_KVM_NVHE_SYM(hyp_trigger_builtin_cfi_fault); void hyp_builtin_cfi_fault_target(int unused); +DECLARE_KVM_NVHE_SYM(hyp_builtin_cfi_fault_target); #endif /* __ARM64_KVM_CFI_H__ */ diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig index 5daa8079a120..715c85088c06 100644 --- a/arch/arm64/kvm/Kconfig +++ b/arch/arm64/kvm/Kconfig @@ -75,8 +75,6 @@ config HYP_CFI_TEST Say M here to also build a module which registers callbacks triggering faults and selected by userspace through its parameters. - Note that this feature is currently only supported in VHE mode. - If unsure, say N. config HYP_SUPPORTS_CFI_TEST diff --git a/arch/arm64/kvm/hyp/vhe/cfi.c b/arch/arm64/kvm/hyp/cfi.c similarity index 100% rename from arch/arm64/kvm/hyp/vhe/cfi.c rename to arch/arm64/kvm/hyp/cfi.c diff --git a/arch/arm64/kvm/hyp/nvhe/Makefile b/arch/arm64/kvm/hyp/nvhe/Makefile index 2eb915d8943f..09039d351726 100644 --- a/arch/arm64/kvm/hyp/nvhe/Makefile +++ b/arch/arm64/kvm/hyp/nvhe/Makefile @@ -25,6 +25,7 @@ hyp-obj-y := timer-sr.o sysreg-sr.o debug-sr.o switch.o tlb.o hyp-init.o host.o cache.o setup.o mm.o mem_protect.o sys_regs.o pkvm.o stacktrace.o ffa.o hyp-obj-y += ../vgic-v3-sr.o ../aarch32.o ../vgic-v2-cpuif-proxy.o ../entry.o \ ../fpsimd.o ../hyp-entry.o ../exception.o ../pgtable.o +hyp-obj-$(CONFIG_HYP_SUPPORTS_CFI_TEST) += ../cfi.o hyp-obj-$(CONFIG_LIST_HARDENED) += list_debug.o hyp-obj-y += $(lib-objs) diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c index 2385fd03ed87..431860e8a98d 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -5,6 +5,7 @@ */ #include +#include #include #include @@ -13,6 +14,8 @@ #include #include +#include + #include #include #include @@ -314,6 +317,19 @@ static void handle___pkvm_teardown_vm(struct kvm_cpu_context *host_ctxt) cpu_reg(host_ctxt, 1) = __pkvm_teardown_vm(handle); } +#ifndef CONFIG_HYP_SUPPORTS_CFI_TEST +__always_unused +#endif +static void handle___kvm_register_cfi_test_cb(struct kvm_cpu_context *host_ctxt) +{ + DECLARE_REG(phys_addr_t, cb_phys, host_ctxt, 1); + DECLARE_REG(bool, in_host_ctxt, host_ctxt, 2); + + void (*cb)(void) = cb_phys ? __hyp_va(cb_phys) : NULL; + + cpu_reg(host_ctxt, 1) = __kvm_register_cfi_test_cb(cb, in_host_ctxt); +} + typedef void (*hcall_t)(struct kvm_cpu_context *); #define HANDLE_FUNC(x) [__KVM_HOST_SMCCC_FUNC_##x] = (hcall_t)handle_##x @@ -348,6 +364,9 @@ static const hcall_t host_hcall[] = { HANDLE_FUNC(__pkvm_init_vm), HANDLE_FUNC(__pkvm_init_vcpu), HANDLE_FUNC(__pkvm_teardown_vm), +#ifdef CONFIG_HYP_SUPPORTS_CFI_TEST + HANDLE_FUNC(__kvm_register_cfi_test_cb), +#endif }; static void handle_host_hcall(struct kvm_cpu_context *host_ctxt) diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/switch.c index c50f8459e4fc..160311bf367b 100644 --- a/arch/arm64/kvm/hyp/nvhe/switch.c +++ b/arch/arm64/kvm/hyp/nvhe/switch.c @@ -4,6 +4,7 @@ * Author: Marc Zyngier */ +#include #include #include @@ -253,6 +254,9 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu) bool pmu_switch_needed; u64 exit_code; + if (IS_ENABLED(CONFIG_HYP_SUPPORTS_CFI_TEST) && unlikely(hyp_test_host_ctxt_cfi)) + hyp_test_host_ctxt_cfi(); + /* * Having IRQs masked via PMR when entering the guest means the GIC * will not signal the CPU of interrupts of lower priority, and the @@ -313,6 +317,9 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu) __debug_switch_to_guest(vcpu); + if (IS_ENABLED(CONFIG_HYP_SUPPORTS_CFI_TEST) && unlikely(hyp_test_guest_ctxt_cfi)) + hyp_test_guest_ctxt_cfi(); + do { /* Jump in the fire! */ exit_code = __guest_enter(vcpu); diff --git a/arch/arm64/kvm/hyp/vhe/Makefile b/arch/arm64/kvm/hyp/vhe/Makefile index 19ca584cc21e..951c8c00a685 100644 --- a/arch/arm64/kvm/hyp/vhe/Makefile +++ b/arch/arm64/kvm/hyp/vhe/Makefile @@ -9,4 +9,4 @@ ccflags-y := -D__KVM_VHE_HYPERVISOR__ obj-y := timer-sr.o sysreg-sr.o debug-sr.o switch.o tlb.o obj-y += ../vgic-v3-sr.o ../aarch32.o ../vgic-v2-cpuif-proxy.o ../entry.o \ ../fpsimd.o ../hyp-entry.o ../exception.o -obj-$(CONFIG_HYP_SUPPORTS_CFI_TEST) += cfi.o +obj-$(CONFIG_HYP_SUPPORTS_CFI_TEST) += ../cfi.o diff --git a/arch/arm64/kvm/hyp_cfi_test.c b/arch/arm64/kvm/hyp_cfi_test.c index da7b25ca1b1f..6a02b43c45f6 100644 --- a/arch/arm64/kvm/hyp_cfi_test.c +++ b/arch/arm64/kvm/hyp_cfi_test.c @@ -6,6 +6,7 @@ #include #include #include +#include #include #include @@ -15,29 +16,60 @@ /* For calling directly into the VHE hypervisor; see . */ int __kvm_register_cfi_test_cb(void (*)(void), bool); -static int kvm_register_cfi_test_cb(void (*vhe_cb)(void), bool in_host_ctxt) +static int kvm_register_nvhe_cfi_test_cb(void *cb, bool in_host_ctxt) +{ + extern void *kvm_nvhe_sym(hyp_test_host_ctxt_cfi); + extern void *kvm_nvhe_sym(hyp_test_guest_ctxt_cfi); + + if (is_protected_kvm_enabled()) { + phys_addr_t cb_phys = cb ? virt_to_phys(cb) : 0; + + /* Use HVC as only the hyp can modify its callback pointers. */ + return kvm_call_hyp_nvhe(__kvm_register_cfi_test_cb, cb_phys, + in_host_ctxt); + } + + /* + * In non-protected nVHE, the pKVM HVC is not available but the + * hyp callback pointers can be accessed and modified directly. + */ + if (cb) + cb = kern_hyp_va(kvm_ksym_ref(cb)); + + if (in_host_ctxt) + kvm_nvhe_sym(hyp_test_host_ctxt_cfi) = cb; + else + kvm_nvhe_sym(hyp_test_guest_ctxt_cfi) = cb; + + return 0; +} + +static int kvm_register_cfi_test_cb(void (*vhe_cb)(void), void *nvhe_cb, + bool in_host_ctxt) { if (!is_hyp_mode_available()) return -ENXIO; if (is_hyp_nvhe()) - return -EOPNOTSUPP; + return kvm_register_nvhe_cfi_test_cb(nvhe_cb, in_host_ctxt); return __kvm_register_cfi_test_cb(vhe_cb, in_host_ctxt); } -int kvm_cfi_test_register_host_ctxt_cb(void (*cb)(void)) +int kvm_cfi_test_register_host_ctxt_cb(void (*vhe_cb)(void), void *nvhe_cb) { - return kvm_register_cfi_test_cb(cb, true); + return kvm_register_cfi_test_cb(vhe_cb, nvhe_cb, true); } EXPORT_SYMBOL(kvm_cfi_test_register_host_ctxt_cb); -int kvm_cfi_test_register_guest_ctxt_cb(void (*cb)(void)) +int kvm_cfi_test_register_guest_ctxt_cb(void (*vhe_cb)(void), void *nvhe_cb) { - return kvm_register_cfi_test_cb(cb, false); + return kvm_register_cfi_test_cb(vhe_cb, nvhe_cb, false); } EXPORT_SYMBOL(kvm_cfi_test_register_guest_ctxt_cb); /* Hypervisor callbacks for the test module to register. */ EXPORT_SYMBOL(hyp_trigger_builtin_cfi_fault); +EXPORT_SYMBOL(kvm_nvhe_sym(hyp_trigger_builtin_cfi_fault)); EXPORT_SYMBOL(hyp_builtin_cfi_fault_target); +EXPORT_SYMBOL(kvm_nvhe_sym(hyp_builtin_cfi_fault_target)); diff --git a/arch/arm64/kvm/hyp_cfi_test_module.c b/arch/arm64/kvm/hyp_cfi_test_module.c index eeda4be4d3ef..63a5e99cb164 100644 --- a/arch/arm64/kvm/hyp_cfi_test_module.c +++ b/arch/arm64/kvm/hyp_cfi_test_module.c @@ -20,9 +20,9 @@ static int set_guest_mode(const char *val, const struct kernel_param *kp); #define M_DESC \ "\n\t0: none" \ "\n\t1: built-in caller & built-in callee" \ - "\n\t2: built-in caller & module callee" \ - "\n\t3: module caller & built-in callee" \ - "\n\t4: module caller & module callee" + "\n\t2: built-in caller & module callee (VHE only)" \ + "\n\t3: module caller & built-in callee (VHE only)" \ + "\n\t4: module caller & module callee (VHE only)" static unsigned int host_mode; module_param_call(host, set_host_mode, param_get_uint, &host_mode, 0644); @@ -40,7 +40,7 @@ static void hyp_cfi_module2module_test_target(int); static void hyp_cfi_builtin2module_test_target(int); static int set_param_mode(const char *val, const struct kernel_param *kp, - int (*register_cb)(void (*)(void))) + int (*register_cb)(void (*)(void), void *)) { unsigned int *mode = kp->arg; int err; @@ -51,15 +51,17 @@ static int set_param_mode(const char *val, const struct kernel_param *kp, switch (*mode) { case 0: - return register_cb(NULL); + return register_cb(NULL, NULL); case 1: - return register_cb(hyp_trigger_builtin_cfi_fault); + return register_cb(hyp_trigger_builtin_cfi_fault, + kvm_nvhe_sym(hyp_trigger_builtin_cfi_fault)); case 2: - return register_cb((void *)hyp_cfi_builtin2module_test_target); + return register_cb((void *)hyp_cfi_builtin2module_test_target, + NULL); case 3: - return register_cb(trigger_module2builtin_cfi_fault); + return register_cb(trigger_module2builtin_cfi_fault, NULL); case 4: - return register_cb(trigger_module2module_cfi_fault); + return register_cb(trigger_module2module_cfi_fault, NULL); default: return -EINVAL; } @@ -79,11 +81,11 @@ static void __exit exit_hyp_cfi_test(void) { int err; - err = kvm_cfi_test_register_host_ctxt_cb(NULL); + err = kvm_cfi_test_register_host_ctxt_cb(NULL, NULL); if (err) pr_err("Failed to unregister host context trigger: %d\n", err); - err = kvm_cfi_test_register_guest_ctxt_cb(NULL); + err = kvm_cfi_test_register_guest_ctxt_cb(NULL, NULL); if (err) pr_err("Failed to unregister guest context trigger: %d\n", err); } From patchwork Wed Apr 10 08:31:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= X-Patchwork-Id: 13623847 Received: from mail-ej1-f51.google.com (mail-ej1-f51.google.com [209.85.218.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B97A614431E for ; Wed, 10 Apr 2024 08:31:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712737897; cv=none; b=ksQqO2FLFMggs7tu4rwECllgSxHHazmmV8vQMIG1W0JgFIbqg9tResBm4FaCNynfFipDjilvHltN/d8rSoZmKCDLhtAw0yiWv5XOX0GLXMvgqvQUstKdS4aSgudINe6wE6oyyTJPIqrqwTsbuTJ6By4+mL/4BofRREsty0fcPUk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712737897; c=relaxed/simple; bh=MJ2amgtSwm20ugyZqCjsEgAnfyZd3gx1BXvTPXSiVfg=; h=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type: Content-Disposition; b=loiiZvChQeVx1YePHGJL6GBvsW5HJTBnsnp5VXuwZcOzJ97n0HUdRkCRrAMFKoP11PsgtaFnW+hHbLPGvEMw/bvZrkhozFLz8C05NhiroB2xYQeMx0xIuVit0yyx9x1mKKjv0GnAMhlYA34nEl9omYn8ljz6iXWjTCwnT5eZnBo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=fK3WARyR; arc=none smtp.client-ip=209.85.218.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="fK3WARyR" Received: by mail-ej1-f51.google.com with SMTP id a640c23a62f3a-a5200202c1bso153371766b.0 for ; Wed, 10 Apr 2024 01:31:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1712737894; x=1713342694; darn=vger.kernel.org; h=content-transfer-encoding:content-disposition:mime-version :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=ko/ldX1/RJm34/W6o5zxcM2yrIA8CZLIGcy8oSLFE1o=; b=fK3WARyRDu7TMppxW5ltnjjX1WeiUlLNGGIiazO/RA10XKmfkOT7AnINjKkgogqMed Vx3zduo8qoNYz3VVlBOWpqCO1Dk/8OyUkCvbM3g1ccTkHUzsxZdAIUnHIqRDyXEtoyXx MddwWCJsC1v+q8zBq3kOtXgT9DBr5QROAPn7zZhnd20rAxqSw7WgmlqZvQeY9VUs1Cy+ RrypHIUxkwKComuebtIT+Om4E65ofTFljxbQlAM904Xj+WR8FSlYds8jHlqWwOnZmkmr F7CD+41gt6W13qQ3Abjh9FsBTLfrxTD7x8TIZKYWMn23G3thU3CUbVmxQb3O8+WR/TI5 EkNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712737894; x=1713342694; h=content-transfer-encoding:content-disposition:mime-version :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ko/ldX1/RJm34/W6o5zxcM2yrIA8CZLIGcy8oSLFE1o=; b=qSy4qS38yKY0vKGKfOo1u9O1NKQu/Wi9EPn9Ba5W9eAz5lLAdWN2sI4/X4VEuAPMUw Z5yA7lYddE85z9qWZVoW8ugWbsGJCoGS4VdhiBuNiRAEvghIScS4tRHhEDWXDpP21KuA OnSPBeLaM7fo1EZFjp5dScZMl4TKSvh/d6nteEOb4K5WBGneFh9SJR5DIjIJXaKQ6OYz zJCc8O/44JLbt8Eq4xUPjfdFU6Wh85ZkSFqnjoPjltetEnot6raPL3tC4tFK0dM7k7qC zxnt26DyYsEYbk54rb0OkOjjmiu5tqo8Pr6MqsvE7vvgBEVpsDh7+1lKTVxTwlWMbzpb 6lzw== X-Forwarded-Encrypted: i=1; AJvYcCXYWBktZ3rmVQCCQgVUcCVsna4LOldJU8n/WUalYpZ+j70RjJUCXMK/ifxtT0lktu3sAd1zzcJ6XJz5ZYqD2I2pBP+q X-Gm-Message-State: AOJu0YzCz2Jd0SPE1tK9uBn6QzY0ofO8L3Y4oqxD37tI2Vmu3/stkUn2 px7TOKgITLObnu1zANEIJQOe+PpdDHlFkfhEHGU43tZ0BiUCKpVz0mp953qaouKEZ+FRTI8mbK0 ePg== X-Google-Smtp-Source: AGHT+IF344zY9aJwSy1QipveS6rH/4iNUnhyO2rqBkRNHvzgRM8S3Vb6xHVKK57ErUTVz32k/JGZNw== X-Received: by 2002:a17:906:5786:b0:a51:e1a1:d127 with SMTP id k6-20020a170906578600b00a51e1a1d127mr1552281ejq.26.1712737893551; Wed, 10 Apr 2024 01:31:33 -0700 (PDT) Received: from google.com (61.134.90.34.bc.googleusercontent.com. [34.90.134.61]) by smtp.gmail.com with ESMTPSA id h12-20020a17090634cc00b00a46f95f5849sm6634813ejb.106.2024.04.10.01.31.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Apr 2024 01:31:33 -0700 (PDT) Date: Wed, 10 Apr 2024 09:31:29 +0100 From: =?utf-8?q?Pierre-Cl=C3=A9ment?= Tosi To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: Marc Zyngier , Oliver Upton , Suzuki K Poulose , Vincent Donnefort Subject: [PATCH v2 12/12] KVM: arm64: Improve CONFIG_CFI_CLANG error message Message-ID: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline For kCFI, the compiler encodes in the immediate of the BRK (which the CPU places in ESR_ELx) the indices of the two registers it used to hold (resp.) the function pointer and expected type. Therefore, the kCFI handler must be able to parse the contents of the register file at the point where the exception was triggered. To achieve this, introduce a new hypervisor panic path that first stores the CPU context in the per-CPU kvm_hyp_ctxt before calling (directly or indirectly) hyp_panic() and execute it from all EL2 synchronous exception handlers i.e. - call it directly in host_el2_sync_vect (__kvm_hyp_host_vector, EL2t&h) - call it directly in el2t_sync_invalid (__kvm_hyp_vector, EL2t) - set ELR_EL2 to it in el2_sync (__kvm_hyp_vector, EL2h), which ERETs Teach hyp_panic() to decode the kCFI ESR and extract the target and type from the saved CPU context. In VHE, use that information to panic() with a specialized error message. In nVHE, only report it if the host (EL1) has access to the saved CPU context i.e. iff CONFIG_NVHE_EL2_DEBUG=y, which aligns with the behavior of CONFIG_PROTECTED_NVHE_STACKTRACE. Signed-off-by: Pierre-Clément Tosi --- arch/arm64/kvm/handle_exit.c | 30 +++++++++++++++++++++++-- arch/arm64/kvm/hyp/entry.S | 24 +++++++++++++++++++- arch/arm64/kvm/hyp/hyp-entry.S | 2 +- arch/arm64/kvm/hyp/include/hyp/switch.h | 4 ++-- arch/arm64/kvm/hyp/nvhe/host.S | 2 +- arch/arm64/kvm/hyp/vhe/switch.c | 26 +++++++++++++++++++-- 6 files changed, 79 insertions(+), 9 deletions(-) diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c index 0db23a6304ce..d76e41a07df1 100644 --- a/arch/arm64/kvm/handle_exit.c +++ b/arch/arm64/kvm/handle_exit.c @@ -26,6 +26,8 @@ #define CREATE_TRACE_POINTS #include "trace_handle_exit.h" +DECLARE_KVM_NVHE_PER_CPU(struct kvm_cpu_context, kvm_hyp_ctxt); + typedef int (*exit_handle_fn)(struct kvm_vcpu *); static void kvm_handle_guest_serror(struct kvm_vcpu *vcpu, u64 esr) @@ -383,11 +385,35 @@ void handle_exit_early(struct kvm_vcpu *vcpu, int exception_index) kvm_handle_guest_serror(vcpu, kvm_vcpu_get_esr(vcpu)); } -static void kvm_nvhe_report_cfi_failure(u64 panic_addr) +static void kvm_nvhe_report_cfi_target(struct user_pt_regs *regs, u64 esr, + u64 hyp_offset) +{ + u64 va_mask = GENMASK_ULL(vabits_actual - 1, 0); + u8 type_idx = FIELD_GET(CFI_BRK_IMM_TYPE, esr); + u8 target_idx = FIELD_GET(CFI_BRK_IMM_TARGET, esr); + u32 expected_type = (u32)regs->regs[type_idx]; + u64 target_addr = (regs->regs[target_idx] & va_mask) + hyp_offset; + + kvm_err(" (target: [<%016llx>] %ps, expected type: 0x%08x)\n", + target_addr, (void *)(target_addr + kaslr_offset()), + expected_type); +} + +static void kvm_nvhe_report_cfi_failure(u64 panic_addr, u64 esr, u64 hyp_offset) { + struct user_pt_regs *regs = NULL; + kvm_err("nVHE hyp CFI failure at: [<%016llx>] %pB!\n", panic_addr, (void *)(panic_addr + kaslr_offset())); + if (IS_ENABLED(CONFIG_NVHE_EL2_DEBUG) || !is_protected_kvm_enabled()) + regs = &this_cpu_ptr_nvhe_sym(kvm_hyp_ctxt)->regs; + + if (regs) + kvm_nvhe_report_cfi_target(regs, esr, hyp_offset); + else + kvm_err(" (no target information: !CONFIG_NVHE_EL2_DEBUG)\n"); + if (IS_ENABLED(CONFIG_CFI_PERMISSIVE)) kvm_err(" (CONFIG_CFI_PERMISSIVE ignored for hyp failures)\n"); } @@ -423,7 +449,7 @@ void __noreturn __cold nvhe_hyp_panic_handler(u64 esr, u64 spsr, kvm_err("nVHE hyp BUG at: [<%016llx>] %pB!\n", panic_addr, (void *)(panic_addr + kaslr_offset())); } else if (IS_ENABLED(CONFIG_CFI_CLANG) && esr_is_cfi_brk(esr)) { - kvm_nvhe_report_cfi_failure(panic_addr); + kvm_nvhe_report_cfi_failure(panic_addr, esr, hyp_offset); } else { kvm_err("nVHE hyp panic at: [<%016llx>] %pB!\n", panic_addr, (void *)(panic_addr + kaslr_offset())); diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S index 6a1ce9d21e5b..8838b453b9be 100644 --- a/arch/arm64/kvm/hyp/entry.S +++ b/arch/arm64/kvm/hyp/entry.S @@ -83,7 +83,7 @@ alternative_else_nop_endif eret sb -SYM_INNER_LABEL(__hyp_restore_elr_and_panic, SYM_L_GLOBAL) +SYM_INNER_LABEL(__hyp_restore_elr_save_context_and_panic, SYM_L_GLOBAL) // x0-x29,lr: hyp regs stp x0, x1, [sp, #-16]! @@ -92,6 +92,28 @@ SYM_INNER_LABEL(__hyp_restore_elr_and_panic, SYM_L_GLOBAL) msr elr_el2, x0 ldp x0, x1, [sp], #16 +SYM_INNER_LABEL(__hyp_save_context_and_panic, SYM_L_GLOBAL) + // x0-x29,lr: hyp regs + + stp x0, x1, [sp, #-16]! + + adr_this_cpu x0, kvm_hyp_ctxt, x1 + + stp x2, x3, [x0, #CPU_XREG_OFFSET(2)] + + ldp x2, x3, [sp], #16 + + stp x2, x3, [x0, #CPU_XREG_OFFSET(0)] + stp x4, x5, [x0, #CPU_XREG_OFFSET(4)] + stp x6, x7, [x0, #CPU_XREG_OFFSET(6)] + stp x8, x9, [x0, #CPU_XREG_OFFSET(8)] + stp x10, x11, [x0, #CPU_XREG_OFFSET(10)] + stp x12, x13, [x0, #CPU_XREG_OFFSET(12)] + stp x14, x15, [x0, #CPU_XREG_OFFSET(14)] + stp x16, x17, [x0, #CPU_XREG_OFFSET(16)] + + save_callee_saved_regs x0 + SYM_INNER_LABEL(__hyp_panic, SYM_L_GLOBAL) // x0-x29,lr: vcpu regs diff --git a/arch/arm64/kvm/hyp/hyp-entry.S b/arch/arm64/kvm/hyp/hyp-entry.S index 7e65ef738ec9..d0d90d598338 100644 --- a/arch/arm64/kvm/hyp/hyp-entry.S +++ b/arch/arm64/kvm/hyp/hyp-entry.S @@ -130,7 +130,7 @@ SYM_CODE_END(\label) .endm /* None of these should ever happen */ - invalid_vector el2t_sync_invalid + invalid_vector el2t_sync_invalid, __hyp_save_context_and_panic invalid_vector el2t_irq_invalid invalid_vector el2t_fiq_invalid invalid_vector el2t_error_invalid diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h index 9387e3a0b680..f3d8fbc7a77b 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -753,7 +753,7 @@ static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code) static inline void __kvm_unexpected_el2_exception(void) { - extern char __hyp_restore_elr_and_panic[]; + extern char __hyp_restore_elr_save_context_and_panic[]; unsigned long addr, fixup; struct kvm_exception_table_entry *entry, *end; unsigned long elr_el2 = read_sysreg(elr_el2); @@ -776,7 +776,7 @@ static inline void __kvm_unexpected_el2_exception(void) /* Trigger a panic after restoring the hyp context. */ this_cpu_ptr(&kvm_hyp_ctxt)->sys_regs[ELR_EL2] = elr_el2; - write_sysreg(__hyp_restore_elr_and_panic, elr_el2); + write_sysreg(__hyp_restore_elr_save_context_and_panic, elr_el2); } #endif /* __ARM64_KVM_HYP_SWITCH_H__ */ diff --git a/arch/arm64/kvm/hyp/nvhe/host.S b/arch/arm64/kvm/hyp/nvhe/host.S index 0613b6e35137..ec3e4f5c28cc 100644 --- a/arch/arm64/kvm/hyp/nvhe/host.S +++ b/arch/arm64/kvm/hyp/nvhe/host.S @@ -213,7 +213,7 @@ SYM_FUNC_END(__host_hvc) .endm .macro host_el2_sync_vect - __host_el2_vect __hyp_panic + __host_el2_vect __hyp_save_context_and_panic .endm .macro invalid_host_el1_vect diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c index b3268933b093..17df57580c77 100644 --- a/arch/arm64/kvm/hyp/vhe/switch.c +++ b/arch/arm64/kvm/hyp/vhe/switch.c @@ -18,6 +18,7 @@ #include #include +#include #include #include #include @@ -308,7 +309,24 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu) return ret; } -static void __noreturn __hyp_call_panic(u64 spsr, u64 elr, u64 par) +static void __noreturn __hyp_call_panic_for_cfi(u64 elr, u64 esr) +{ + struct user_pt_regs *regs = &this_cpu_ptr(&kvm_hyp_ctxt)->regs; + u8 type_idx = FIELD_GET(CFI_BRK_IMM_TYPE, esr); + u8 target_idx = FIELD_GET(CFI_BRK_IMM_TARGET, esr); + u32 expected_type = (u32)regs->regs[type_idx]; + u64 target = regs->regs[target_idx]; + + panic("VHE hyp CFI failure at: [<%016llx>] %pB (target: [<%016llx>] %ps, expected type: 0x%08x)\n" +#ifdef CONFIG_CFI_PERMISSIVE + " (CONFIG_CFI_PERMISSIVE ignored for hyp failures)\n" +#endif + , + elr, (void *)elr, target, (void *)target, expected_type); +} +NOKPROBE_SYMBOL(__hyp_call_panic_for_cfi); + +static void __noreturn __hyp_call_panic(u64 spsr, u64 elr, u64 par, u64 esr) { struct kvm_cpu_context *host_ctxt; struct kvm_vcpu *vcpu; @@ -319,6 +337,9 @@ static void __noreturn __hyp_call_panic(u64 spsr, u64 elr, u64 par) __deactivate_traps(vcpu); sysreg_restore_host_state_vhe(host_ctxt); + if (IS_ENABLED(CONFIG_CFI_CLANG) && esr_is_cfi_brk(esr)) + __hyp_call_panic_for_cfi(elr, esr); + panic("HYP panic:\nPS:%08llx PC:%016llx ESR:%08llx\nFAR:%016llx HPFAR:%016llx PAR:%016llx\nVCPU:%p\n", spsr, elr, read_sysreg_el2(SYS_ESR), read_sysreg_el2(SYS_FAR), @@ -331,8 +352,9 @@ void __noreturn hyp_panic(void) u64 spsr = read_sysreg_el2(SYS_SPSR); u64 elr = read_sysreg_el2(SYS_ELR); u64 par = read_sysreg_par(); + u64 esr = read_sysreg_el2(SYS_ESR); - __hyp_call_panic(spsr, elr, par); + __hyp_call_panic(spsr, elr, par, esr); } asmlinkage void kvm_unexpected_el2_exception(void)