From patchwork Wed Dec 2 18:40:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11946743 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B3D0EC8301B for ; Wed, 2 Dec 2020 18:43:18 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 527D92173E for ; Wed, 2 Dec 2020 18:43:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 527D92173E Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=iXLoI4iwH4KHHjY7I28+fL301CDRFHlWeR2OWNTMMgQ=; b=xDmZxyGTL8nP7fQPF1pw6dEHv teyMYE8DY7fzfI9F+dII9iVopjR3P0vN6fUGRGDKLc0d6CjjuAwEDC/7O2B2o0BZJV15kPAoLdAYF PLz+YlBcxAHGptw5unZ4PUlT/qh8hneXXwKSJu3ClIfzaWe3d1OJt+yMgpb59flAwyJS+o3FiMqah BB243z0KSkymPdWgH4/QduGyqEVFiCYS15pHG0GVEEfHkFJ0KYbKIrgJC4DjxikcdRg6M4b+BRrAz SDpv1JLnXRHz9Pxl2vIO8dqMmM2d95Va/QJ0St4ZdraOLo+a0K6Y6+Vq6q4JtElLjhVx2lLngcDqk PpaldQmtw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX4H-0008Aa-PN; Wed, 02 Dec 2020 18:41:33 +0000 Received: from mail-wm1-x344.google.com ([2a00:1450:4864:20::344]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX4E-00089V-TE for linux-arm-kernel@lists.infradead.org; Wed, 02 Dec 2020 18:41:31 +0000 Received: by mail-wm1-x344.google.com with SMTP id e25so11127729wme.0 for ; Wed, 02 Dec 2020 10:41:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Qy+VUI9BXr6HazxoMIsIcdlYXr2Zkw6Es0halgF2Fhw=; b=UAOOKMnPkO17JoHDZkxjQgAVO6zpGftV0wvNiSWjhlY7kOwOa1zo2h+ivb22qOwe+V OUeZ1kaZ2iuLe85sH3vKir/PolrJozCc8f5BEnZclJqkMHEnPBBe5RIxS54Uily8MPnK 2Y1SjWE7pmPOCzK4bYInNyWMy9kenBr5/8ZdVX5elM6EWvAO1SdjrF2QGFKXUd9OqCb+ NtIQnngm0PfEs5d3IgcTf7LhfVPkIPloHQMtfhVabrq5N7Rf/Jb5NOWKYhZgULgWXKhE BGxNfajR8eczlwMYDG1uEtwk+bce5hnubdI0gY+5v+YfNSeDCieQsZ2WmyKqfSlr+VD/ 1/zA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Qy+VUI9BXr6HazxoMIsIcdlYXr2Zkw6Es0halgF2Fhw=; b=VMsw7nR+RIUDraUczt7h6hn5MZTnsomUFmxBnmEfSN7rjocqmuZLYVBScvbXKN6xrf HCx/cymQpmH8YgcwYeLa/KvSjMKssD8uLJCzeS8UQ0tj+DeLnVRI+LLVc5Wx78haP1wQ dZTvtR7FLfSv6XsFNJZy0+Jb8ENU6tdWsp/QstVA5Fc7IXRGxHfaQxF/js1RSy+KDBYh V17DDv0+pPfO05AMLXt8J2mRWbtcggTWaX0xhxfwXdqdjMd8eGHsnFCyvRV/bf/eOiZH /LqZp355e9jDByp2rDybfAyySxEEALbWMUwT07DHyXeVisK953aCoLQSeH5KNLB1pqGo oA9g== X-Gm-Message-State: AOAM532zzb2yW+3dHJJhes20ScCSXnsE48zP1UlCG55jrjubIHYaKcqt UXPA8C3didDBIwR1H5nLWdQC+g== X-Google-Smtp-Source: ABdhPJx5dw7ns+PEDpJmj41ddtOfdNs+k8f+Q+FRgiqp1pfkmjrl7qaCK6KSXenVzyoYRhNeR59CDw== X-Received: by 2002:a1c:b402:: with SMTP id d2mr4422934wmf.38.1606934489275; Wed, 02 Dec 2020 10:41:29 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:5ddd:b7c5:e3c9:e87a]) by smtp.gmail.com with ESMTPSA id o13sm2910262wmc.44.2020.12.02.10.41.27 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 02 Dec 2020 10:41:28 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v4 01/26] kvm: arm64: Add kvm-arm.mode early kernel parameter Date: Wed, 2 Dec 2020 18:40:57 +0000 Message-Id: <20201202184122.26046-2-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201202184122.26046-1-dbrazdil@google.com> References: <20201202184122.26046-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201202_134130_991566_3DB95959 X-CRM114-Status: GOOD ( 18.85 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add an early parameter that allows users to select the mode of operation for KVM/arm64. For now, the only supported value is "protected". By passing this flag users opt into the hypervisor placing additional restrictions on the host kernel. These allow the hypervisor to spawn guests whose state is kept private from the host. Restrictions will include stage-2 address translation to prevent host from accessing guest memory, filtering its SMC calls, etc. Without this parameter, the default behaviour remains selecting VHE/nVHE based on hardware support and CONFIG_ARM64_VHE. Signed-off-by: David Brazdil --- Documentation/admin-guide/kernel-parameters.txt | 10 ++++++++++ arch/arm64/include/asm/kvm_host.h | 9 +++++++++ arch/arm64/kvm/arm.c | 16 ++++++++++++++++ 3 files changed, 35 insertions(+) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index 526d65d8573a..ee9f13776388 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -2259,6 +2259,16 @@ for all guests. Default is 1 (enabled) if in 64-bit or 32-bit PAE mode. + kvm-arm.mode= + [KVM,ARM] Select one of KVM/arm64's modes of operation. + + protected: nVHE-based mode with support for guests whose + state is kept private from the host. + Not valid if the kernel is running in EL2. + + Defaults to VHE/nVHE based on hardware support and + the value of CONFIG_ARM64_VHE. + kvm-arm.vgic_v3_group0_trap= [KVM,ARM] Trap guest accesses to GICv3 group-0 system registers diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 21ce5c420247..bca38ccade58 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -50,6 +50,15 @@ #define KVM_DIRTY_LOG_MANUAL_CAPS (KVM_DIRTY_LOG_MANUAL_PROTECT_ENABLE | \ KVM_DIRTY_LOG_INITIALLY_SET) +/* + * Mode of operation configurable with kvm-arm.mode early param. + * See Documentation/admin-guide/kernel-parameters.txt for more information. + */ +enum kvm_mode { + KVM_MODE_DEFAULT, + KVM_MODE_PROTECTED, +}; + DECLARE_STATIC_KEY_FALSE(userspace_irqchip_in_use); extern unsigned int kvm_sve_max_vl; diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 7e86207fa2fc..01cf6ac5b2cb 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -45,6 +45,8 @@ __asm__(".arch_extension virt"); #endif +static enum kvm_mode kvm_mode = KVM_MODE_DEFAULT; + DECLARE_KVM_HYP_PER_CPU(unsigned long, kvm_hyp_vector); static DEFINE_PER_CPU(unsigned long, kvm_arm_hyp_stack_page); @@ -1868,6 +1870,20 @@ void kvm_arch_exit(void) kvm_perf_teardown(); } +static int __init early_kvm_mode_cfg(char *arg) +{ + if (!arg) + return -EINVAL; + + if (strcmp(arg, "protected") == 0) { + kvm_mode = KVM_MODE_PROTECTED; + return 0; + } + + return -EINVAL; +} +early_param("kvm-arm.mode", early_kvm_mode_cfg); + static int arm_init(void) { int rc = kvm_init(NULL, sizeof(struct kvm_vcpu), 0, THIS_MODULE); From patchwork Wed Dec 2 18:40:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11946741 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5C3EC83017 for ; Wed, 2 Dec 2020 18:43:17 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6B781206D5 for ; Wed, 2 Dec 2020 18:43:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6B781206D5 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=IUJ69dSNpsYAmjzwrE2JDTkm4R0nHe70Q1ieI6jMq6A=; b=xlRcu+N0Dp1y9n8M1GFEz8/AL p0vLhYiAUeMnreSo+WVGYdPfbm93AXWfsLSb+KD+mCmdXoZWs7QNYqYyUDUUdcYBdaanBVMA3oY2F Eu+bmCKcC4faJaL68u+fgNiqmzskvhJwGuSw1xNXGt9Jt4LueK5Tx+XRDTptaFF8Xz/g5+M3H1oOq U4KE9tslQ57cT1aamYBb41BbsZqzSs62VNvZTcgn/FpWRUwB8TNFLWzt51ckmYeNaKVmsTKsYNLpm J7uZQn5vq71jeTdH3oWw/COjkRq7/wnXf5zyWvMMzs1scTcXzDW6ycTn/glVQxkiVKGUrnHW/6alM SRF6aGzRQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX4L-0008BL-Cm; Wed, 02 Dec 2020 18:41:37 +0000 Received: from mail-wr1-x42d.google.com ([2a00:1450:4864:20::42d]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX4H-0008A2-7K for linux-arm-kernel@lists.infradead.org; Wed, 02 Dec 2020 18:41:34 +0000 Received: by mail-wr1-x42d.google.com with SMTP id l1so5136016wrb.9 for ; Wed, 02 Dec 2020 10:41:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=g69TmuIpp+vIm9gRxx12Sh909+m1H1PrZL1qc3JkdQk=; b=WYFmGcSaH/wz/SlleIW3Mcu5nmwmv1kNPSQebgDZz+0eL3IjEcwWqvN80s834KuWzR RT3hly6g09zu5fWXVEkuV1YdbuzgldYW+V/8twi8l0yx8axbff8Ape8T/NR3v99S/UFx UBu7/c8XKsJf4vxFX3puM6VjMfdo+TmAE6IxE4Ang8fcy12BxnfTtg1FdTdeE6rwEhoS YanICjQ8vrCn3rOi1TLnMIhFRy3x7EV/7Dv4MoeSNLxZ2KVazRSH0LJ7mbdhEaezvCt5 1ytVx/ay9QHVAw3P76uQzTzAv86K6uXZ8g0VqxbV1fUq1XJEGMtJpcjaVOearI/QMSyk P8Vg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=g69TmuIpp+vIm9gRxx12Sh909+m1H1PrZL1qc3JkdQk=; b=ihrO5qDT63xH+IafnfLJaNlcoZOZYpt/o8oDAS/Rn8Y+aDfKy8w27bGAxzp1b5w3KP L3oMpx20W+sy3d/CUYq62K8ARM1RUW/DgcOd/rlar82UpXuXI4TVQSaOeOEQMoQQEGQc 20c4bU3inh3YcOK9XepffXxKJJ0gpSIgUTVtrfCxJhmYK1CrGGwjb+KpuqEsy0Ulu97h F4e5KbxMLChiNsE7X6f4pAbNy/vmMEFL1WdoZ5MIAnGZLliM8rMa187/aeHZ8E1quLM6 Pfyd2ihdDt4lIRgdDE8kE9XdlFw4RNdwLpe+GtL0AWmX67Cl4vO1ji40iThg75RH23ND Jnqw== X-Gm-Message-State: AOAM5303V/kYyehp/sBGbheEkz+Cj3f52pqBrXlKFFRyaLqJFAr5XOE5 5PbOQ7oRaO5fylt1qNUhZUjuYw== X-Google-Smtp-Source: ABdhPJxLt3JxRj6qOSsRATpU04zggeVJmgFS/y1Rt/TDhlt67MX8xTg5Vhhj21UyUIwOKHqvCPeRjQ== X-Received: by 2002:adf:dec1:: with SMTP id i1mr4974199wrn.129.1606934491590; Wed, 02 Dec 2020 10:41:31 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:5ddd:b7c5:e3c9:e87a]) by smtp.gmail.com with ESMTPSA id k205sm3004907wmk.4.2020.12.02.10.41.30 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 02 Dec 2020 10:41:30 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v4 02/26] kvm: arm64: Add ARM64_KVM_PROTECTED_MODE CPU capability Date: Wed, 2 Dec 2020 18:40:58 +0000 Message-Id: <20201202184122.26046-3-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201202184122.26046-1-dbrazdil@google.com> References: <20201202184122.26046-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201202_134133_278013_0FFF5562 X-CRM114-Status: GOOD ( 23.61 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Expose the boolean value whether the system is running with KVM in protected mode (nVHE + kernel param). CPU capability was selected over a global variable to allow use in alternatives. Signed-off-by: David Brazdil --- arch/arm64/include/asm/cpucaps.h | 3 ++- arch/arm64/include/asm/kvm_host.h | 1 + arch/arm64/include/asm/virt.h | 8 ++++++++ arch/arm64/kernel/cpufeature.c | 22 ++++++++++++++++++++++ arch/arm64/kvm/arm.c | 9 ++++++++- 5 files changed, 41 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h index 162539d4c8cd..42f850718d4b 100644 --- a/arch/arm64/include/asm/cpucaps.h +++ b/arch/arm64/include/asm/cpucaps.h @@ -66,7 +66,8 @@ #define ARM64_HAS_TLB_RANGE 56 #define ARM64_MTE 57 #define ARM64_WORKAROUND_1508412 58 +#define ARM64_KVM_PROTECTED_MODE 59 -#define ARM64_NCAPS 59 +#define ARM64_NCAPS 60 #endif /* __ASM_CPUCAPS_H */ diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index bca38ccade58..11beda85ee7e 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -58,6 +58,7 @@ enum kvm_mode { KVM_MODE_DEFAULT, KVM_MODE_PROTECTED, }; +enum kvm_mode kvm_get_mode(void); DECLARE_STATIC_KEY_FALSE(userspace_irqchip_in_use); diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h index 6069be50baf9..eb81dcc220b6 100644 --- a/arch/arm64/include/asm/virt.h +++ b/arch/arm64/include/asm/virt.h @@ -97,6 +97,14 @@ static __always_inline bool has_vhe(void) return cpus_have_final_cap(ARM64_HAS_VIRT_HOST_EXTN); } +static __always_inline bool is_protected_kvm_enabled(void) +{ + if (is_vhe_hyp_code()) + return false; + else + return cpus_have_final_cap(ARM64_KVM_PROTECTED_MODE); +} + #endif /* __ASSEMBLY__ */ #endif /* ! __ASM__VIRT_H */ diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 280b10762f6b..5f9e557fa1f7 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -74,6 +74,7 @@ #include #include #include +#include #include #include #include @@ -1709,6 +1710,21 @@ static void cpu_enable_mte(struct arm64_cpu_capabilities const *cap) } #endif /* CONFIG_ARM64_MTE */ +#ifdef CONFIG_KVM +static bool is_kvm_protected_mode(const struct arm64_cpu_capabilities *entry, int __unused) +{ + if (kvm_get_mode() != KVM_MODE_PROTECTED) + return false; + + if (is_kernel_in_hyp_mode()) { + pr_warn("Protected KVM not available with VHE\n"); + return false; + } + + return true; +} +#endif /* CONFIG_KVM */ + /* Internal helper functions to match cpu capability type */ static bool cpucap_late_cpu_optional(const struct arm64_cpu_capabilities *cap) @@ -1822,6 +1838,12 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .field_pos = ID_AA64PFR0_EL1_SHIFT, .min_field_value = ID_AA64PFR0_EL1_32BIT_64BIT, }, + { + .desc = "Protected KVM", + .capability = ARM64_KVM_PROTECTED_MODE, + .type = ARM64_CPUCAP_SYSTEM_FEATURE, + .matches = is_kvm_protected_mode, + }, #endif { .desc = "Kernel page table isolation (KPTI)", diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 01cf6ac5b2cb..d9e8d7dc7e11 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1849,7 +1849,9 @@ int kvm_arch_init(void *opaque) if (err) goto out_hyp; - if (in_hyp_mode) + if (is_protected_kvm_enabled()) + kvm_info("Protected nVHE mode initialized successfully\n"); + else if (in_hyp_mode) kvm_info("VHE mode initialized successfully\n"); else kvm_info("Hyp mode initialized successfully\n"); @@ -1884,6 +1886,11 @@ static int __init early_kvm_mode_cfg(char *arg) } early_param("kvm-arm.mode", early_kvm_mode_cfg); +enum kvm_mode kvm_get_mode(void) +{ + return kvm_mode; +} + static int arm_init(void) { int rc = kvm_init(NULL, sizeof(struct kvm_vcpu), 0, THIS_MODULE); From patchwork Wed Dec 2 18:40:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11946739 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5514AC83016 for ; Wed, 2 Dec 2020 18:43:17 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DBE12206D5 for ; Wed, 2 Dec 2020 18:43:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DBE12206D5 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=U2N8XWK+C/N9b5FHsNHjxEpDp2IzioPiPqvugb9B9Mk=; b=X0VcRPlsWz4G37UIyCS6aSyN9 S02wJ4BTXZiPV5TdYj1CFf6jPckt9BcRIisbnXt71fAWM16JlfvJfedIao8CqtKgJBr3bP17EJR8y 6hZsmmFNsKnxll56qOvcc0t/aTMNtDB6V6527/lNH6YUjhJsLij2K+BdIJgNCk6GMpUrpJlSwmj1/ eyX39zvWAoXwonnEELCEsf9zXd7/SKjg06XlK5LqFK+mBztnM0w34dkn2UEtYZPJspkiPKhrV3+b+ G6Nku931tXrLMk4oaSW6CU6eTXurNNTlEpPKgziN0TDHBePecO6eiooMZ9u2cJwRC7utp23kpBLXW ZU32Fo85Q==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX4N-0008Bm-Ef; Wed, 02 Dec 2020 18:41:39 +0000 Received: from mail-wr1-x443.google.com ([2a00:1450:4864:20::443]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX4J-0008Ah-4A for linux-arm-kernel@lists.infradead.org; Wed, 02 Dec 2020 18:41:36 +0000 Received: by mail-wr1-x443.google.com with SMTP id o1so5161764wrx.7 for ; Wed, 02 Dec 2020 10:41:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=JTPYqEMEJVYu7EDLdYK8M/9yDaVtcEHoKzFLwQ3C9WQ=; b=G9nRp1NClO5ThaoHR0FfWTTV1jX37+O++dpEWQ5M8ktTDAMpJeJccyh4QbRtPHzrgj 3p0wDTK1oNkUPBZWEB48xqyd1DpiPlza78uG3UJmd0FRgJoXsvYEFC5/dJ6p1PDADnej JzxUan8mUbqcytiT8Xsbq58HdA7MIJPBZrBTgotHo8Pn9CTEZK8UNEMPbhDMlXHR7m1s PDfiRnDCZkEazv3nGY2YNAECEwG/U/mnyvg+RXProtzJE1N/fcd9LDxLtq4B8OclAb90 fHQNn4wIZufUltwam9z0S7haohHxuJjP1yQ/G1kC05DlOQE4unx7Y8oRzB8xond/u6h2 YkBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=JTPYqEMEJVYu7EDLdYK8M/9yDaVtcEHoKzFLwQ3C9WQ=; b=qqBcpGWTTKIMp/d9HLkIS2KYfAr3QLLystgQlmFXby9yktt+k6G+8i3LEVZCNcsHhF wmr7fKuDeSBaNJpoGkFxv4HihRtFoqFjqfzxHRqpr22uKK6TxkwbHoePtvbPwE8lefsk t0G1Ct1LCmZGjQJm0Ud/sAp0GVgEgVbcZElN5uOB7GPSp2CnjAzkP+xl4DzKBRe3DCYR 1Rq7d9gxacxZiZe7gjnG8MOvIsIVkuklYyGvfhTmiCnui3Ljo+J+lkKJXp42WmkUaF4I 8aGmD10Hz3Cpn2T9zofUTcfOzo1QzUFWf8kkcYaH3JIzP3zKMlEowxcwmEKAxvdFdAdH Bohw== X-Gm-Message-State: AOAM530Hy/nySTpUKwr/KiU0x0FEL2dURKC1nQ5p1mWdtxtFOctg6Sdq GoFyZu2ljGS8J0a+7g5fEbwtFA== X-Google-Smtp-Source: ABdhPJziIgLZ0tc/W8uCPyhlwdgytrCa5jtp+4HLbEeMEMxZ9Y3AuJ3UEdaL+P3G5xYsvTsXBF1VmQ== X-Received: by 2002:a5d:4610:: with SMTP id t16mr5088666wrq.391.1606934493917; Wed, 02 Dec 2020 10:41:33 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:5ddd:b7c5:e3c9:e87a]) by smtp.gmail.com with ESMTPSA id r21sm3315301wrc.16.2020.12.02.10.41.32 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 02 Dec 2020 10:41:32 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v4 03/26] psci: Support psci_ops.get_version for v0.1 Date: Wed, 2 Dec 2020 18:40:59 +0000 Message-Id: <20201202184122.26046-4-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201202184122.26046-1-dbrazdil@google.com> References: <20201202184122.26046-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201202_134135_516919_E86598B0 X-CRM114-Status: GOOD ( 15.82 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org KVM's host PSCI SMC filter needs to be aware of the PSCI version of the system but currently it is impossible to distinguish between v0.1 and PSCI disabled because both have get_version == NULL. Populate get_version for v0.1 with a function that returns a constant. psci_opt.get_version is currently unused so this has no effect on existing functionality. Acked-by: Mark Rutland Signed-off-by: David Brazdil --- drivers/firmware/psci/psci.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/drivers/firmware/psci/psci.c b/drivers/firmware/psci/psci.c index 00af99b6f97c..ace5b9ac676c 100644 --- a/drivers/firmware/psci/psci.c +++ b/drivers/firmware/psci/psci.c @@ -146,7 +146,12 @@ static int psci_to_linux_errno(int errno) return -EINVAL; } -static u32 psci_get_version(void) +static u32 psci_0_1_get_version(void) +{ + return PSCI_VERSION(0, 1); +} + +static u32 psci_0_2_get_version(void) { return invoke_psci_fn(PSCI_0_2_FN_PSCI_VERSION, 0, 0, 0); } @@ -421,7 +426,7 @@ static void __init psci_init_smccc(void) static void __init psci_0_2_set_functions(void) { pr_info("Using standard PSCI v0.2 function IDs\n"); - psci_ops.get_version = psci_get_version; + psci_ops.get_version = psci_0_2_get_version; psci_function_id[PSCI_FN_CPU_SUSPEND] = PSCI_FN_NATIVE(0_2, CPU_SUSPEND); @@ -450,7 +455,7 @@ static void __init psci_0_2_set_functions(void) */ static int __init psci_probe(void) { - u32 ver = psci_get_version(); + u32 ver = psci_0_2_get_version(); pr_info("PSCIv%d.%d detected in firmware.\n", PSCI_VERSION_MAJOR(ver), @@ -514,6 +519,8 @@ static int __init psci_0_1_init(struct device_node *np) pr_info("Using PSCI v0.1 Function IDs from DT\n"); + psci_ops.get_version = psci_0_1_get_version; + if (!of_property_read_u32(np, "cpu_suspend", &id)) { psci_function_id[PSCI_FN_CPU_SUSPEND] = id; psci_ops.cpu_suspend = psci_cpu_suspend; From patchwork Wed Dec 2 18:41:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11946745 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2C37DC64E7C for ; Wed, 2 Dec 2020 18:43:18 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BF32221D7F for ; Wed, 2 Dec 2020 18:43:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BF32221D7F Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=ER7lekmimM8XvBizEbi85HX/9tkdEMwRBiOzm44qLMA=; b=DjT7BZ1DDxzcKNAkmH2060Td7 CUlOu4toViAJ06peYJJ7lfg7R14UvRm87+b7jzZlPeF1lWB4M7BguezVCCRZJUyJNJC0RPqaqpLV6 FE/zZenhxm0TIVkiavh75qti8+PoMXTsM+DSKdKizgxPPG/UXVi4VB0bHIVT2IycUZwRBIJ4OTqQO gf6EMr5FnDns/S9Sr1FbrH7Id185gAELSVzH1ossGF4e1lLHpSbRebRJEXaqH0YmYkSKk48B9gKmO jGpPfpnJfcGV0ueUqmYc8zT1rmQVqb7puETL38aehrPcEY65rRX2X4r5Y2QEohsaNj2tVd9IDI9rN 3PtBZwr7Q==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX4R-0008DT-6V; Wed, 02 Dec 2020 18:41:43 +0000 Received: from mail-wm1-x343.google.com ([2a00:1450:4864:20::343]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX4L-0008BJ-PC for linux-arm-kernel@lists.infradead.org; Wed, 02 Dec 2020 18:41:39 +0000 Received: by mail-wm1-x343.google.com with SMTP id x22so10028870wmc.5 for ; Wed, 02 Dec 2020 10:41:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=oYr8FdouxPNXJYj6F8T1dGVqWutC568ZBSlYDhNUpeM=; b=NFEcyQF7W9LakzcYiLFbTpnuPAMTiR4UFuRT3IJs/i5DDvUsC/Uk+pNi698+XnR+B3 QvuGNYXhk0dGxab42tIg+hNWSU6kt6X6ohldo4maT1nBg8OCcEsLb7KMRGyrWvFjWZ/H 2MB6Hn6JpAEzYKrPxpWA8Q8hRQJkCsOHXtT+NYu5fSFfNjcA1GfzpsmPzA56wvUk00Cj IZL1Q9wtPhCPc1k8fuvTMtfqVkMsQw16wnU8dYjnzN/A3CT2NCxt5QMNeN/STdRHVymz g9/rpoZBqCJQLXqQDVCHWAEfjit68BnXYNUkvRfxB6WeaygnWo0DN1M+xUvQPp3l3xPn +LmQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=oYr8FdouxPNXJYj6F8T1dGVqWutC568ZBSlYDhNUpeM=; b=f2JS5tN/Emt+iNojjqM7Y/no4jDKY5aLStK3A9eplMMHa9wmnlnfv/WzUBM1qYOUGA udwo4B7SBydNDZJWPfu8S9BWQEgEU3YLvxhgRAvPY4MU7/sBjXllKnrpYfLOBsjx/Pm6 m44GLUkgPDiyGasd8Ob1kCRoONxWpHWYiRt1t+yLn0tnh+cUChyFnWNZ9Mw1/P8fXoI3 VT1k2oKUdLmN0XEcQfuffnynC0l3CvGgN2+GDVl2UXoPR3/RvfluwyJ6dUrUk9Vm3IQT 38wmHPvNvqGbZGNixK1Ga25g6632lfpoh06NSRjcIuz4rFJP+MDtRuqg5ekgfNmNBuTV /+Xw== X-Gm-Message-State: AOAM532eUNbKKEGmQDNBggyMbZgxpweuZ5twv4e6rE/2QFmCIw41b+xw HwEhiFGYKakyYrD6SZ0xF0DEoA== X-Google-Smtp-Source: ABdhPJyA+FeKGAZ0pnrVUnW1W88hwOUW0I0MbdOAm+Y14RWvjrRicJ4tZn4rpByON5cgnGjgoXvNAw== X-Received: by 2002:a1c:f315:: with SMTP id q21mr4543655wmq.1.1606934496105; Wed, 02 Dec 2020 10:41:36 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:5ddd:b7c5:e3c9:e87a]) by smtp.gmail.com with ESMTPSA id h15sm3006773wrw.15.2020.12.02.10.41.34 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 02 Dec 2020 10:41:35 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v4 04/26] psci: Split functions to v0.1 and v0.2+ variants Date: Wed, 2 Dec 2020 18:41:00 +0000 Message-Id: <20201202184122.26046-5-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201202184122.26046-1-dbrazdil@google.com> References: <20201202184122.26046-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201202_134137_895443_DC6523E3 X-CRM114-Status: GOOD ( 16.69 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Refactor implementation of v0.1+ functions (CPU_SUSPEND, CPU_OFF, CPU_ON, MIGRATE) to have two functions psci_0_1_foo / psci_0_2_foo that select the function ID and call a common helper __psci_foo. This is a small cleanup so that the function ID array is only used for v0.1 configurations. Signed-off-by: David Brazdil Acked-by: Mark Rutland --- drivers/firmware/psci/psci.c | 94 +++++++++++++++++++++++------------- 1 file changed, 60 insertions(+), 34 deletions(-) diff --git a/drivers/firmware/psci/psci.c b/drivers/firmware/psci/psci.c index ace5b9ac676c..13b9ed71b446 100644 --- a/drivers/firmware/psci/psci.c +++ b/drivers/firmware/psci/psci.c @@ -168,46 +168,80 @@ int psci_set_osi_mode(bool enable) return psci_to_linux_errno(err); } -static int psci_cpu_suspend(u32 state, unsigned long entry_point) +static int __psci_cpu_suspend(u32 fn, u32 state, unsigned long entry_point) { int err; - u32 fn; - fn = psci_function_id[PSCI_FN_CPU_SUSPEND]; err = invoke_psci_fn(fn, state, entry_point, 0); return psci_to_linux_errno(err); } -static int psci_cpu_off(u32 state) +static int psci_0_1_cpu_suspend(u32 state, unsigned long entry_point) +{ + return __psci_cpu_suspend(psci_function_id[PSCI_FN_CPU_SUSPEND], + state, entry_point); +} + +static int psci_0_2_cpu_suspend(u32 state, unsigned long entry_point) +{ + return __psci_cpu_suspend(PSCI_FN_NATIVE(0_2, CPU_SUSPEND), + state, entry_point); +} + +static int __psci_cpu_off(u32 fn, u32 state) { int err; - u32 fn; - fn = psci_function_id[PSCI_FN_CPU_OFF]; err = invoke_psci_fn(fn, state, 0, 0); return psci_to_linux_errno(err); } -static int psci_cpu_on(unsigned long cpuid, unsigned long entry_point) +static int psci_0_1_cpu_off(u32 state) +{ + return __psci_cpu_off(psci_function_id[PSCI_FN_CPU_OFF], state); +} + +static int psci_0_2_cpu_off(u32 state) +{ + return __psci_cpu_off(PSCI_0_2_FN_CPU_OFF, state); +} + +static int __psci_cpu_on(u32 fn, unsigned long cpuid, unsigned long entry_point) { int err; - u32 fn; - fn = psci_function_id[PSCI_FN_CPU_ON]; err = invoke_psci_fn(fn, cpuid, entry_point, 0); return psci_to_linux_errno(err); } -static int psci_migrate(unsigned long cpuid) +static int psci_0_1_cpu_on(unsigned long cpuid, unsigned long entry_point) +{ + return __psci_cpu_on(psci_function_id[PSCI_FN_CPU_ON], cpuid, entry_point); +} + +static int psci_0_2_cpu_on(unsigned long cpuid, unsigned long entry_point) +{ + return __psci_cpu_on(PSCI_FN_NATIVE(0_2, CPU_ON), cpuid, entry_point); +} + +static int __psci_migrate(u32 fn, unsigned long cpuid) { int err; - u32 fn; - fn = psci_function_id[PSCI_FN_MIGRATE]; err = invoke_psci_fn(fn, cpuid, 0, 0); return psci_to_linux_errno(err); } +static int psci_0_1_migrate(unsigned long cpuid) +{ + return __psci_migrate(psci_function_id[PSCI_FN_MIGRATE], cpuid); +} + +static int psci_0_2_migrate(unsigned long cpuid) +{ + return __psci_migrate(PSCI_FN_NATIVE(0_2, MIGRATE), cpuid); +} + static int psci_affinity_info(unsigned long target_affinity, unsigned long lowest_affinity_level) { @@ -352,7 +386,7 @@ static void __init psci_init_system_suspend(void) static void __init psci_init_cpu_suspend(void) { - int feature = psci_features(psci_function_id[PSCI_FN_CPU_SUSPEND]); + int feature = psci_features(PSCI_FN_NATIVE(0_2, CPU_SUSPEND)); if (feature != PSCI_RET_NOT_SUPPORTED) psci_cpu_suspend_feature = feature; @@ -426,24 +460,16 @@ static void __init psci_init_smccc(void) static void __init psci_0_2_set_functions(void) { pr_info("Using standard PSCI v0.2 function IDs\n"); - psci_ops.get_version = psci_0_2_get_version; - - psci_function_id[PSCI_FN_CPU_SUSPEND] = - PSCI_FN_NATIVE(0_2, CPU_SUSPEND); - psci_ops.cpu_suspend = psci_cpu_suspend; - - psci_function_id[PSCI_FN_CPU_OFF] = PSCI_0_2_FN_CPU_OFF; - psci_ops.cpu_off = psci_cpu_off; - - psci_function_id[PSCI_FN_CPU_ON] = PSCI_FN_NATIVE(0_2, CPU_ON); - psci_ops.cpu_on = psci_cpu_on; - psci_function_id[PSCI_FN_MIGRATE] = PSCI_FN_NATIVE(0_2, MIGRATE); - psci_ops.migrate = psci_migrate; - - psci_ops.affinity_info = psci_affinity_info; - - psci_ops.migrate_info_type = psci_migrate_info_type; + psci_ops = (struct psci_operations){ + .get_version = psci_0_2_get_version, + .cpu_suspend = psci_0_2_cpu_suspend, + .cpu_off = psci_0_2_cpu_off, + .cpu_on = psci_0_2_cpu_on, + .migrate = psci_0_2_migrate, + .affinity_info = psci_affinity_info, + .migrate_info_type = psci_migrate_info_type, + }; arm_pm_restart = psci_sys_reset; @@ -523,22 +549,22 @@ static int __init psci_0_1_init(struct device_node *np) if (!of_property_read_u32(np, "cpu_suspend", &id)) { psci_function_id[PSCI_FN_CPU_SUSPEND] = id; - psci_ops.cpu_suspend = psci_cpu_suspend; + psci_ops.cpu_suspend = psci_0_1_cpu_suspend; } if (!of_property_read_u32(np, "cpu_off", &id)) { psci_function_id[PSCI_FN_CPU_OFF] = id; - psci_ops.cpu_off = psci_cpu_off; + psci_ops.cpu_off = psci_0_1_cpu_off; } if (!of_property_read_u32(np, "cpu_on", &id)) { psci_function_id[PSCI_FN_CPU_ON] = id; - psci_ops.cpu_on = psci_cpu_on; + psci_ops.cpu_on = psci_0_1_cpu_on; } if (!of_property_read_u32(np, "migrate", &id)) { psci_function_id[PSCI_FN_MIGRATE] = id; - psci_ops.migrate = psci_migrate; + psci_ops.migrate = psci_0_1_migrate; } return 0; From patchwork Wed Dec 2 18:41:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11946747 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D6B9FC71155 for ; Wed, 2 Dec 2020 18:43:21 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7612C2173E for ; Wed, 2 Dec 2020 18:43:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7612C2173E Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=gPJku9zZHWf1D7ePMCFHgSEoHUlkjHSwkX2zN99tk5g=; b=PctcpZKgf7U4Htom7z5Bl7iTR JItJ3wWOeiBMRcnfXUuF0RjGEW2nAYp0LAB3EKIACA4+izYwW+xTtrtoqkYg0X+xgHribzYMSnASE UzrNOAl/h5lNOfVaIbNtEBhrkXERtfwOa+ClG9lzHkBUv84pdyC/49otnR1ipLP1xHkvT9CdS27VB WdiIYVptIk/PCdkRr1Jnuar61Q4z7vq/3aDJoZL4w9JJ4JFwbNkZEw8q9eN6Db1aEEodgc83nnPHS OPSZsSrxF9MpSkQUY9bvlbY2Zya9dYl4qCAD5+k5Y9ZhCzK1ShuYr504rEHNNwxCAB+kH4dbCgI1V N3DlJEeew==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX4U-0008EH-1L; Wed, 02 Dec 2020 18:41:46 +0000 Received: from mail-wr1-x442.google.com ([2a00:1450:4864:20::442]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX4N-0008Bj-J8 for linux-arm-kernel@lists.infradead.org; Wed, 02 Dec 2020 18:41:41 +0000 Received: by mail-wr1-x442.google.com with SMTP id s8so5120661wrw.10 for ; Wed, 02 Dec 2020 10:41:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=cK6AK8VTndq2WNu92SDcRQLBfLwMiV+V214MHUEgg2w=; b=Xf/f5iL7f+U7uWYo68hnex0nqviSfyrK8iju0quGui/JaXe3jHIb4ayRSem4PO/dq2 lJGlb+3oWwRKdmbaF5MTjSBGA1a/nZ3DnuYAFU0VXkg3XJeK83UY1px90mjbi3yXR1Mb iM4vVVqC7wOJlGnKkNBM4fXDzNbk9nKHDjGpETmTOggyBJ43ISHXgf96DSwHeZ+e19Zg qzN5AGHZoq4l2bC1PF4CKHiN+laTlUEI/0jdRvB/GCkrYMSfiNf9DuMdmu6iUyf28J74 kWyZuBuapoYx8SF7rQXnFicabHz3K+IUHtgj2IDCOGeZb28eFFDsBHNzY2NUKlGXgXVv XqJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cK6AK8VTndq2WNu92SDcRQLBfLwMiV+V214MHUEgg2w=; b=slNRjpsBv2JpJMsZlBFvHX6kf4cSY59G+UxIg6bNz4zhklpw9IoHFVPdJz6Ig4fhsV AnxN5xeQT/zc5KX/TOpYfK4EJmyb3QhG6zGqwd4pDM2YeoTY7HT+y+mmSeGlcKgsFiHP FdRO/zeSG/kgDwtcntneojtKoF/K5xrz4qjntqqN+W8WziyS7h4X2FmERxZrZtm5bck6 /7lUe4eiDOemA9Aky8Yd300w/AvjGAPNDAT98qXsyTrP9LSoHi7vTtDB8hIgohUQzkX4 YD23yXrpaYXr6GcdMwPAUFm60TPaA5v/u0+sW6YdfmrqiydYGHFrzxjgKgHb4lHTTIwR 0T/A== X-Gm-Message-State: AOAM5337WjUSxzH6UoDITbCBfC/li4zhBGJNdG7kVTyZAyq0uPa3mOpJ 58wYPtsO1y/rOTX5gcYMoBs7Ig== X-Google-Smtp-Source: ABdhPJyx6RaeHoVm3xBFk7dQ2A5SBfxrzx+C7n8O07sr+KP5yugR90H9p2be9APNazqd8m8Fn6a1ww== X-Received: by 2002:adf:dd81:: with SMTP id x1mr5122600wrl.163.1606934498348; Wed, 02 Dec 2020 10:41:38 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:5ddd:b7c5:e3c9:e87a]) by smtp.gmail.com with ESMTPSA id z21sm2967184wmk.20.2020.12.02.10.41.36 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 02 Dec 2020 10:41:37 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v4 05/26] psci: Replace psci_function_id array with a struct Date: Wed, 2 Dec 2020 18:41:01 +0000 Message-Id: <20201202184122.26046-6-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201202184122.26046-1-dbrazdil@google.com> References: <20201202184122.26046-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201202_134139_754510_DD1F8DDD X-CRM114-Status: GOOD ( 15.22 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Small refactor that replaces array of v0.1 function IDs indexed by an enum of function-name constants with a struct of function IDs "indexed" by field names. This is done in preparation for exposing the IDs to other parts of the kernel. Exposing a struct avoids the need for bounds checking. Signed-off-by: David Brazdil Acked-by: Mark Rutland --- drivers/firmware/psci/psci.c | 29 ++++++++++++++--------------- 1 file changed, 14 insertions(+), 15 deletions(-) diff --git a/drivers/firmware/psci/psci.c b/drivers/firmware/psci/psci.c index 13b9ed71b446..593fdd0e09a2 100644 --- a/drivers/firmware/psci/psci.c +++ b/drivers/firmware/psci/psci.c @@ -58,15 +58,14 @@ typedef unsigned long (psci_fn)(unsigned long, unsigned long, unsigned long, unsigned long); static psci_fn *invoke_psci_fn; -enum psci_function { - PSCI_FN_CPU_SUSPEND, - PSCI_FN_CPU_ON, - PSCI_FN_CPU_OFF, - PSCI_FN_MIGRATE, - PSCI_FN_MAX, +struct psci_0_1_function_ids { + u32 cpu_suspend; + u32 cpu_on; + u32 cpu_off; + u32 migrate; }; -static u32 psci_function_id[PSCI_FN_MAX]; +static struct psci_0_1_function_ids psci_0_1_function_ids; #define PSCI_0_2_POWER_STATE_MASK \ (PSCI_0_2_POWER_STATE_ID_MASK | \ @@ -178,7 +177,7 @@ static int __psci_cpu_suspend(u32 fn, u32 state, unsigned long entry_point) static int psci_0_1_cpu_suspend(u32 state, unsigned long entry_point) { - return __psci_cpu_suspend(psci_function_id[PSCI_FN_CPU_SUSPEND], + return __psci_cpu_suspend(psci_0_1_function_ids.cpu_suspend, state, entry_point); } @@ -198,7 +197,7 @@ static int __psci_cpu_off(u32 fn, u32 state) static int psci_0_1_cpu_off(u32 state) { - return __psci_cpu_off(psci_function_id[PSCI_FN_CPU_OFF], state); + return __psci_cpu_off(psci_0_1_function_ids.cpu_off, state); } static int psci_0_2_cpu_off(u32 state) @@ -216,7 +215,7 @@ static int __psci_cpu_on(u32 fn, unsigned long cpuid, unsigned long entry_point) static int psci_0_1_cpu_on(unsigned long cpuid, unsigned long entry_point) { - return __psci_cpu_on(psci_function_id[PSCI_FN_CPU_ON], cpuid, entry_point); + return __psci_cpu_on(psci_0_1_function_ids.cpu_on, cpuid, entry_point); } static int psci_0_2_cpu_on(unsigned long cpuid, unsigned long entry_point) @@ -234,7 +233,7 @@ static int __psci_migrate(u32 fn, unsigned long cpuid) static int psci_0_1_migrate(unsigned long cpuid) { - return __psci_migrate(psci_function_id[PSCI_FN_MIGRATE], cpuid); + return __psci_migrate(psci_0_1_function_ids.migrate, cpuid); } static int psci_0_2_migrate(unsigned long cpuid) @@ -548,22 +547,22 @@ static int __init psci_0_1_init(struct device_node *np) psci_ops.get_version = psci_0_1_get_version; if (!of_property_read_u32(np, "cpu_suspend", &id)) { - psci_function_id[PSCI_FN_CPU_SUSPEND] = id; + psci_0_1_function_ids.cpu_suspend = id; psci_ops.cpu_suspend = psci_0_1_cpu_suspend; } if (!of_property_read_u32(np, "cpu_off", &id)) { - psci_function_id[PSCI_FN_CPU_OFF] = id; + psci_0_1_function_ids.cpu_off = id; psci_ops.cpu_off = psci_0_1_cpu_off; } if (!of_property_read_u32(np, "cpu_on", &id)) { - psci_function_id[PSCI_FN_CPU_ON] = id; + psci_0_1_function_ids.cpu_on = id; psci_ops.cpu_on = psci_0_1_cpu_on; } if (!of_property_read_u32(np, "migrate", &id)) { - psci_function_id[PSCI_FN_MIGRATE] = id; + psci_0_1_function_ids.migrate = id; psci_ops.migrate = psci_0_1_migrate; } From patchwork Wed Dec 2 18:41:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11946749 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 14B05C83018 for ; Wed, 2 Dec 2020 18:43:24 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A6701206D5 for ; Wed, 2 Dec 2020 18:43:23 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A6701206D5 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=Q2gJnPOkSRRCbysWPm6fI+TpT/jSMXIKPrO2Hh6ClP0=; b=U0WeAHBKfEypVmnsf47NvdNr1 Xjzm31EN3DiT0veQBMrqjgjLkYC3KyHbGOOuuEPe6/pB4kzt8UoVfvGHjDF4aSqzUJaBQZSZSCFbU mdeAuXx0mahG9RMltZKJ+tmRnCsaHik+FfivSKuSWDZA0VgwkzltYrOXHhnpU8slUxCY7pRlC3OXj LpphkRyIjerFjaza/y2/4piATf9GKvuC6sZFngd5nopeIXdsdMtJTX/eflJYqL8cc1URMYDyFo863 NGnuUcVJojmPRxJ4veVBPZka2K4uH7yyqY9WKt4/Ly2VnzSu+uAa/ehlI0S40WkBX7BT1XBXDKBvk cPFqM+d9A==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX4X-0008FF-HG; Wed, 02 Dec 2020 18:41:49 +0000 Received: from mail-wr1-x441.google.com ([2a00:1450:4864:20::441]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX4P-0008Ci-SV for linux-arm-kernel@lists.infradead.org; Wed, 02 Dec 2020 18:41:42 +0000 Received: by mail-wr1-x441.google.com with SMTP id i2so5162861wrs.4 for ; Wed, 02 Dec 2020 10:41:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=qXwVi8GgD0h+y0Z1L4Qz/RxstC0DDwAUVzhRN0FU904=; b=i+G98kVudQR6uGePmNxw8xCv7SMJAja1ZSWowPxR1m3WcQORTplZuJmLizNsMjEhCh wEKbRahBqXc7XUcmyktVRkyyRuiuvzzaXLq/c+bjP3tKDlOtoQz2LX6scrNS6qFBjsDZ jJFI5aZZ9jlIWXo5dAO5x0M14XqlhIjLXybyL7t9tmYSDxxm8vj02XMeVsJGeAbRe3Ec bHjOvS6lH4XPiSfInJTeRmGrp8KEenXa0rxR8rBCPjCgXRbh7UKOJ0EfJ0JxTI2zZtoa SeNlCdJAZECI2DajIMd/1/nTwrPVyOC5AzMf8GimCHFwpoSij5LlSefToJPi7j30LqQ8 It/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=qXwVi8GgD0h+y0Z1L4Qz/RxstC0DDwAUVzhRN0FU904=; b=ByMz/6NlqtBvRzVrQy0u/hm305Hv2yyf4s7HfCeapnuTIwfBaocW1YlwEZ/AUNi+pQ nMOtwG3QaZ0C6/syuVHHRPx1eMHTJ0QiCoNmTPPacCAZNCU6YcAEQO7N0nSYsc5L6mkk uOXHPqDCEH49sLR1ZhXsYeP57BWrAxT+/rKfJKZeZrW2Ky6HZ6IA0rkvcILifIB+cEV6 F/YIAUiE4KtO2/SlJifzQqTtvvChGlB+QKGwO1eLH/4qTW0/Tw+0P2NPzBnh1B2c9YJV dZ11eT8ciB1UrH026To0b5TsEWHlUlqJr3xhloLq8uSvLujxaT6m6IxS6RZiPYELUB6x BMEg== X-Gm-Message-State: AOAM5317KVU+M/OrVuElq1LVx7F19gToymkSDzsFl31YFHZ/KWGkEkac rWJV50eZpK5jTmykPsWdiKdq2w== X-Google-Smtp-Source: ABdhPJzo+3XrHLu5MZisvjC9Vlqkp9p6BFe0uyPEE/+3sbo7phn6YCf8SNKtKbXnK5PdAZQg2KF0+w== X-Received: by 2002:a5d:4f0e:: with SMTP id c14mr4996465wru.422.1606934500673; Wed, 02 Dec 2020 10:41:40 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:5ddd:b7c5:e3c9:e87a]) by smtp.gmail.com with ESMTPSA id p11sm3257202wrj.14.2020.12.02.10.41.39 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 02 Dec 2020 10:41:39 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v4 06/26] psci: Add accessor for psci_0_1_function_ids Date: Wed, 2 Dec 2020 18:41:02 +0000 Message-Id: <20201202184122.26046-7-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201202184122.26046-1-dbrazdil@google.com> References: <20201202184122.26046-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201202_134142_058575_5BB7DF03 X-CRM114-Status: GOOD ( 16.29 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Make it possible to retrieve a copy of the psci_0_1_function_ids struct. This is useful for KVM if it is configured to intercept host's PSCI SMCs. Signed-off-by: David Brazdil Acked-by: Mark Rutland --- drivers/firmware/psci/psci.c | 12 +++++------- include/linux/psci.h | 9 +++++++++ 2 files changed, 14 insertions(+), 7 deletions(-) diff --git a/drivers/firmware/psci/psci.c b/drivers/firmware/psci/psci.c index 593fdd0e09a2..f5fc429cae3f 100644 --- a/drivers/firmware/psci/psci.c +++ b/drivers/firmware/psci/psci.c @@ -58,15 +58,13 @@ typedef unsigned long (psci_fn)(unsigned long, unsigned long, unsigned long, unsigned long); static psci_fn *invoke_psci_fn; -struct psci_0_1_function_ids { - u32 cpu_suspend; - u32 cpu_on; - u32 cpu_off; - u32 migrate; -}; - static struct psci_0_1_function_ids psci_0_1_function_ids; +struct psci_0_1_function_ids get_psci_0_1_function_ids(void) +{ + return psci_0_1_function_ids; +} + #define PSCI_0_2_POWER_STATE_MASK \ (PSCI_0_2_POWER_STATE_ID_MASK | \ PSCI_0_2_POWER_STATE_TYPE_MASK | \ diff --git a/include/linux/psci.h b/include/linux/psci.h index 2a1bfb890e58..4ca0060a3fc4 100644 --- a/include/linux/psci.h +++ b/include/linux/psci.h @@ -34,6 +34,15 @@ struct psci_operations { extern struct psci_operations psci_ops; +struct psci_0_1_function_ids { + u32 cpu_suspend; + u32 cpu_on; + u32 cpu_off; + u32 migrate; +}; + +struct psci_0_1_function_ids get_psci_0_1_function_ids(void); + #if defined(CONFIG_ARM_PSCI_FW) int __init psci_dt_init(void); #else From patchwork Wed Dec 2 18:41:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11946751 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4D8ADC83014 for ; Wed, 2 Dec 2020 18:43:30 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DEA4B206D5 for ; Wed, 2 Dec 2020 18:43:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DEA4B206D5 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=LXOAHH4ooQNCXm+EtH+MLutSgomfRAu6VEc6pElUi28=; b=Lxg+LHrOhaj/hJ/vIH1+pKGmv R/e8CTiOJCrovRkkybCMVuMkidsCwlv4gBbKEmh4WIq5j59dplV664wPhxchzwVi8jqugV/UkXwls h6jDCpXa+lvW/Utac08oosaXDrSUYqcCfygpRMeveGvUnAT49nvvYzQAUxV7hu47vxhuK6XbH3RrO w5navoGYr4jM+drTcVxPB2G+PqAiiUliZ3i0R3xyT/xb+T7QyvfXE5MYk4TIiP3KcEAbK95dVd7jN dqQjs8ziVdhGRgrdUnvkq0HMF7FBFFvRZqisiaMLAqUFI3tuLoEnd4R+YUUUG/Y9nkSyuvqkVvPAq Pnp88x3+g==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX4c-0008H4-Oh; Wed, 02 Dec 2020 18:41:54 +0000 Received: from mail-wr1-x442.google.com ([2a00:1450:4864:20::442]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX4R-0008Dg-PN for linux-arm-kernel@lists.infradead.org; Wed, 02 Dec 2020 18:41:44 +0000 Received: by mail-wr1-x442.google.com with SMTP id r3so5174442wrt.2 for ; Wed, 02 Dec 2020 10:41:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=y2p8Ay2XnBVL3vmtVVdiQ6iCXfVcEKgd2S1Ri8Ky5yc=; b=RVZK76j4k6bFwiXLgYuFF1wtJaBoX2V07k5mb0zz2XzuAWmLJ1i+F8W5NohjH00/sj gWlE7qrSAwcbIAEG3hSs7PAR248eOVkY6Xm3SGr2s7GCAbFd1nsA3SXKbGnzIxFSrkg8 irlXEGzEcD3yJ1mQ7deVRYJx/fqkEt/rYVXS4dv7nID08H9qJKazcokqFUOi0xPNWDmc wSiS1Rr8Nrm4KLg51r8BdgmhZJ5lT0++p3Ln9OxqEdQrrLW0MIdiVj2Hn7v3NIVuTRH5 LzsCZkvPa5kTdTcpPz8VJgGlrbBlyP+9VsdgTapGlKyPChgjeyjIeM7KiQROiHGleMKZ qLSQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=y2p8Ay2XnBVL3vmtVVdiQ6iCXfVcEKgd2S1Ri8Ky5yc=; b=NMi2yqAKQDieR0iwtIXpbB8H2cmXOCITCc4GNGlRqf3YU1Av6CNSZ98PzrHQbmmI8S 5FlRtKfyEepeJdcAZPAoJqGBLVgL6gq1wPjtmOf2RYiVlV84yQ3kJFsT0Rd2farKbMXO JRGM5U+JvTmdm4xw0vC8hZThsZ2fsJICjt7aYC4k8Boxq/pJChiUsB+YkMJKXMWJv+mW VzXSe/Yl7obY0aJHa22pSg+gFjiH5GQK/Dm9o2cvuaBtqq91AvKOAHm8ZJ655yRuGbjD POgVjZWtUZsXGVCACxdrUWsgS9oQPiip0R18CxGeqv9EPa1c6ZBf2PlhTEkNmYfBlUCH YpWA== X-Gm-Message-State: AOAM530J1sV95AeN5W096WyQErr2N0+WYorbRBq2lokEdGCsB7CzGHVH yffWJO0v3vcWnVCc5Q1+23pfFg== X-Google-Smtp-Source: ABdhPJxAQnxesV8ENz1UWdK5eE8OCnghkt8xaIi7bsuMXjZf93/bl2MjqeL5MAHnTzF9beINxPa0Kg== X-Received: by 2002:adf:c443:: with SMTP id a3mr5093368wrg.249.1606934502768; Wed, 02 Dec 2020 10:41:42 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:5ddd:b7c5:e3c9:e87a]) by smtp.gmail.com with ESMTPSA id v20sm2813264wmh.44.2020.12.02.10.41.41 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 02 Dec 2020 10:41:41 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v4 07/26] arm64: Make cpu_logical_map() take unsigned int Date: Wed, 2 Dec 2020 18:41:03 +0000 Message-Id: <20201202184122.26046-8-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201202184122.26046-1-dbrazdil@google.com> References: <20201202184122.26046-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201202_134143_919394_AB588E50 X-CRM114-Status: GOOD ( 15.25 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org CPU index should never be negative. Change the signature of (set_)cpu_logical_map to take an unsigned int. This still works even if the users treat the CPU index as an int, and will allow the hypervisor's implementation to check that the index is valid with a single upper-bound check. Signed-off-by: David Brazdil --- arch/arm64/include/asm/smp.h | 4 ++-- arch/arm64/kernel/setup.c | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/arch/arm64/include/asm/smp.h b/arch/arm64/include/asm/smp.h index 2e7f529ec5a6..bcb01ca15325 100644 --- a/arch/arm64/include/asm/smp.h +++ b/arch/arm64/include/asm/smp.h @@ -46,9 +46,9 @@ DECLARE_PER_CPU_READ_MOSTLY(int, cpu_number); * Logical CPU mapping. */ extern u64 __cpu_logical_map[NR_CPUS]; -extern u64 cpu_logical_map(int cpu); +extern u64 cpu_logical_map(unsigned int cpu); -static inline void set_cpu_logical_map(int cpu, u64 hwid) +static inline void set_cpu_logical_map(unsigned int cpu, u64 hwid) { __cpu_logical_map[cpu] = hwid; } diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c index 133257ffd859..2f2973bc67c7 100644 --- a/arch/arm64/kernel/setup.c +++ b/arch/arm64/kernel/setup.c @@ -276,7 +276,7 @@ arch_initcall(reserve_memblock_reserved_regions); u64 __cpu_logical_map[NR_CPUS] = { [0 ... NR_CPUS-1] = INVALID_HWID }; -u64 cpu_logical_map(int cpu) +u64 cpu_logical_map(unsigned int cpu) { return __cpu_logical_map[cpu]; } From patchwork Wed Dec 2 18:41:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11946755 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E2225C83029 for ; Wed, 2 Dec 2020 18:44:06 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 441602173E for ; Wed, 2 Dec 2020 18:44:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 441602173E Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=tQ+PpCeWwy8enwpcMUn8JoSQ8VGiLa7qnK9rYo4tm9M=; b=lQvEViicQKF6I6cErPIWujtPu 76+c1Iu5cIzvUSCYtlr3pEByIVazqeohjoWC3G1IJDwxT2czYm1zEbgK9nfj6P5bNzZzTe87Cuh5A 63YlMWtbOH7piSZl1wisJ7uPAasGk6LZwALreMAZkdFARj2knHiVuXkltyE3Ctp5Ie0PnLHLhsIg8 VLPXoIBe3tnt9PhS0h4z5Y8g46A4aawecanmvezvvqoSp2Cb/RDBtgQQ8fO+9QmEt0sEuAvMmq1/H 2CfiPub0UY6yLwNmOjOV/dCtEtXVp4/XgKnoRgaKve7SJzRIpkTM9XVntpWyf+fA+4oD/by5pvZIX C+/c/S0/g==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX4y-0008Pc-Vi; Wed, 02 Dec 2020 18:42:17 +0000 Received: from mail-wm1-x330.google.com ([2a00:1450:4864:20::330]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX4U-0008ER-Vo for linux-arm-kernel@lists.infradead.org; Wed, 02 Dec 2020 18:41:48 +0000 Received: by mail-wm1-x330.google.com with SMTP id h21so11164141wmb.2 for ; Wed, 02 Dec 2020 10:41:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=5F/Up/Ebf6Bf2bbXNTPpS910fASqQtc48mTj0zAO7xk=; b=SjMo0tJDhHyf0Mrm83VvRqCW1PLSUhEha+EXMzEWAvl4XghqKqgQhhDMZWPERqq8do faEyJm2etTS2u8lTu8J9JR9Z6bJN2JqMxpj0qUUduCl35H6N78nEOsaBysrsvIsoi7ND 2SxzzajH+kd967JO5t5TkVQlAEf0ECHxjt8iRD0E7u8SFQipEB4bViq3fLYGWB1xV7j6 eUaYlnvD7Ivri1VgCZgqTEcfbfFF1GCVvlDb4AyYus/GTEtjxZezjfn9ZWROWOh7IB4q sm/Zf0vuMY+LHm+gXo4cCrO1JiIXryISFhAgA0KDOQdFtjjMpAe5cw4Lb0apI1kBplEs aIUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5F/Up/Ebf6Bf2bbXNTPpS910fASqQtc48mTj0zAO7xk=; b=bCuBx2AUvlnnO5FA/5onTmyORnMp3bQ2V2VCOHwaJDPnNAz9TrVNsYD1nPGzq8xxaa 5VUkLUPyaZqlZZ3U49dEQSy8mNsTMGMOdt1USow2kX3IDLOHbu6rc4HF3//JZTXLWnR6 J/el6/BszClLc59XjF/wIoLY6HBwrftkRDX8W08AfT4w/SAzLlKKjTlg8T7VTJMtIQfJ /rEHrhiQjsWAoVwfoszNLH+W2XgrMsqALn7CaScm+wSQVti8f54tZOI8pwRlUHXiNmWu ChAzY+j7VuZ28HNeXcWKfExguXNGahuQAn+EFv13CJwsi7Nt8J/NDa2MRj8YkkjWyx14 /uwA== X-Gm-Message-State: AOAM531uazei+HI8/mdKni2MG2KTrhIjClePE6BqCC6gqLnSbwp+88CB DHkr93YXaKXXoMv1l3ia1VtiFA== X-Google-Smtp-Source: ABdhPJxXckzKmd0GUcj8deK6tWbGt/buaeRFujJdUjLM1LEwSGt5OIMk48B697gMb8Ns5ghNc0kBaQ== X-Received: by 2002:a1c:2d84:: with SMTP id t126mr4448330wmt.132.1606934505057; Wed, 02 Dec 2020 10:41:45 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:5ddd:b7c5:e3c9:e87a]) by smtp.gmail.com with ESMTPSA id r21sm3315875wrc.16.2020.12.02.10.41.43 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 02 Dec 2020 10:41:44 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v4 08/26] arm64: Extract parts of el2_setup into a macro Date: Wed, 2 Dec 2020 18:41:04 +0000 Message-Id: <20201202184122.26046-9-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201202184122.26046-1-dbrazdil@google.com> References: <20201202184122.26046-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201202_134147_140643_7DB4D3FB X-CRM114-Status: GOOD ( 27.75 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When a CPU is booted in EL2, the kernel checks for VHE support and initializes the CPU core accordingly. For nVHE it also installs the stub vectors and drops down to EL1. Once KVM gains the ability to boot cores without going through the kernel entry point, it will need to initialize the CPU the same way. Extract the relevant bits of el2_setup into an init_el2_state macro with an argument specifying whether to initialize for VHE or nVHE. The following ifdefs are removed: * CONFIG_ARM_GIC_V3 - always selected on arm64 * CONFIG_COMPAT - hstr_el2 can be set even without 32-bit support No functional change intended. Size of el2_setup increased by 148 bytes due to duplication. Signed-off-by: David Brazdil --- arch/arm64/include/asm/el2_setup.h | 182 +++++++++++++++++++++++++++++ arch/arm64/kernel/head.S | 144 +++-------------------- 2 files changed, 198 insertions(+), 128 deletions(-) create mode 100644 arch/arm64/include/asm/el2_setup.h diff --git a/arch/arm64/include/asm/el2_setup.h b/arch/arm64/include/asm/el2_setup.h new file mode 100644 index 000000000000..b6cfd8aa2a81 --- /dev/null +++ b/arch/arm64/include/asm/el2_setup.h @@ -0,0 +1,182 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2012,2013 - ARM Ltd + * Author: Marc Zyngier + */ + +#ifndef __ARM_KVM_INIT_H__ +#define __ARM_KVM_INIT_H__ + +#ifndef __ASSEMBLY__ +#error Assembly-only header +#endif + +#include +#include +#include +#include + +.macro __init_el2_sctlr + mov_q x0, (SCTLR_EL2_RES1 | ENDIAN_SET_EL2) + msr sctlr_el2, x0 + isb +.endm + +/* + * Allow Non-secure EL1 and EL0 to access physical timer and counter. + * This is not necessary for VHE, since the host kernel runs in EL2, + * and EL0 accesses are configured in the later stage of boot process. + * Note that when HCR_EL2.E2H == 1, CNTHCTL_EL2 has the same bit layout + * as CNTKCTL_EL1, and CNTKCTL_EL1 accessing instructions are redefined + * to access CNTHCTL_EL2. This allows the kernel designed to run at EL1 + * to transparently mess with the EL0 bits via CNTKCTL_EL1 access in + * EL2. + */ +.macro __init_el2_timers mode +.ifeqs "\mode", "nvhe" + mrs x0, cnthctl_el2 + orr x0, x0, #3 // Enable EL1 physical timers + msr cnthctl_el2, x0 +.endif + msr cntvoff_el2, xzr // Clear virtual offset +.endm + +.macro __init_el2_debug mode + mrs x1, id_aa64dfr0_el1 + sbfx x0, x1, #ID_AA64DFR0_PMUVER_SHIFT, #4 + cmp x0, #1 + b.lt 1f // Skip if no PMU present + mrs x0, pmcr_el0 // Disable debug access traps + ubfx x0, x0, #11, #5 // to EL2 and allow access to +1: + csel x2, xzr, x0, lt // all PMU counters from EL1 + + /* Statistical profiling */ + ubfx x0, x1, #ID_AA64DFR0_PMSVER_SHIFT, #4 + cbz x0, 3f // Skip if SPE not present + +.ifeqs "\mode", "nvhe" + mrs_s x0, SYS_PMBIDR_EL1 // If SPE available at EL2, + and x0, x0, #(1 << SYS_PMBIDR_EL1_P_SHIFT) + cbnz x0, 2f // then permit sampling of physical + mov x0, #(1 << SYS_PMSCR_EL2_PCT_SHIFT | \ + 1 << SYS_PMSCR_EL2_PA_SHIFT) + msr_s SYS_PMSCR_EL2, x0 // addresses and physical counter +2: + mov x0, #(MDCR_EL2_E2PB_MASK << MDCR_EL2_E2PB_SHIFT) + orr x2, x2, x0 // If we don't have VHE, then + // use EL1&0 translation. +.else + orr x2, x2, #MDCR_EL2_TPMS // For VHE, use EL2 translation + // and disable access from EL1 +.endif + +3: + msr mdcr_el2, x2 // Configure debug traps +.endm + +/* LORegions */ +.macro __init_el2_lor + mrs x1, id_aa64mmfr1_el1 + ubfx x0, x1, #ID_AA64MMFR1_LOR_SHIFT, 4 + cbz x0, 1f + msr_s SYS_LORC_EL1, xzr +1: +.endm + +/* Stage-2 translation */ +.macro __init_el2_stage2 + msr vttbr_el2, xzr +.endm + +/* GICv3 system register access */ +.macro __init_el2_gicv3 + mrs x0, id_aa64pfr0_el1 + ubfx x0, x0, #ID_AA64PFR0_GIC_SHIFT, #4 + cbz x0, 1f + + mrs_s x0, SYS_ICC_SRE_EL2 + orr x0, x0, #ICC_SRE_EL2_SRE // Set ICC_SRE_EL2.SRE==1 + orr x0, x0, #ICC_SRE_EL2_ENABLE // Set ICC_SRE_EL2.Enable==1 + msr_s SYS_ICC_SRE_EL2, x0 + isb // Make sure SRE is now set + mrs_s x0, SYS_ICC_SRE_EL2 // Read SRE back, + tbz x0, #0, 1f // and check that it sticks + msr_s SYS_ICH_HCR_EL2, xzr // Reset ICC_HCR_EL2 to defaults +1: +.endm + +.macro __init_el2_hstr + msr hstr_el2, xzr // Disable CP15 traps to EL2 +.endm + +/* Virtual CPU ID registers */ +.macro __init_el2_nvhe_idregs + mrs x0, midr_el1 + mrs x1, mpidr_el1 + msr vpidr_el2, x0 + msr vmpidr_el2, x1 +.endm + +/* Coprocessor traps */ +.macro __init_el2_nvhe_cptr + mov x0, #0x33ff + msr cptr_el2, x0 // Disable copro. traps to EL2 +.endm + +/* SVE register access */ +.macro __init_el2_nvhe_sve + mrs x1, id_aa64pfr0_el1 + ubfx x1, x1, #ID_AA64PFR0_SVE_SHIFT, #4 + cbz x1, 1f + + bic x0, x0, #CPTR_EL2_TZ // Also disable SVE traps + msr cptr_el2, x0 // Disable copro. traps to EL2 + isb + mov x1, #ZCR_ELx_LEN_MASK // SVE: Enable full vector + msr_s SYS_ZCR_EL2, x1 // length for EL1. +1: +.endm + +.macro __init_el2_nvhe_prepare_eret + mov x0, #(PSR_F_BIT | PSR_I_BIT | PSR_A_BIT | PSR_D_BIT |\ + PSR_MODE_EL1h) + msr spsr_el2, x0 +.endm + +/** + * Initialize EL2 registers to sane values. This should be called early on all + * cores that were booted in EL2. + * + * Regs: x0, x1 and x2 are clobbered. + */ +.macro init_el2_state mode +.ifnes "\mode", "vhe" +.ifnes "\mode", "nvhe" +.error "Invalid 'mode' argument" +.endif +.endif + + __init_el2_sctlr + __init_el2_timers \mode + __init_el2_debug \mode + __init_el2_lor + __init_el2_stage2 + __init_el2_gicv3 + __init_el2_hstr + + /* + * When VHE is not in use, early init of EL2 needs to be done here. + * When VHE _is_ in use, EL1 will not be used in the host and + * requires no configuration, and all non-hyp-specific EL2 setup + * will be done via the _EL1 system register aliases in __cpu_setup. + */ +.ifeqs "\mode", "nvhe" + __init_el2_nvhe_idregs + __init_el2_nvhe_cptr + __init_el2_nvhe_sve + __init_el2_nvhe_prepare_eret +.endif +.endm + +#endif /* __ARM_KVM_INIT_H__ */ diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index d8d9caf02834..da913ce9e89f 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -11,7 +11,6 @@ #include #include -#include #include #include @@ -21,6 +20,7 @@ #include #include #include +#include #include #include #include @@ -493,159 +493,47 @@ SYM_FUNC_START(el2_setup) mrs x0, CurrentEL cmp x0, #CurrentEL_EL2 b.eq 1f + mov_q x0, (SCTLR_EL1_RES1 | ENDIAN_SET_EL1) msr sctlr_el1, x0 mov w0, #BOOT_CPU_MODE_EL1 // This cpu booted in EL1 isb ret -1: mov_q x0, (SCTLR_EL2_RES1 | ENDIAN_SET_EL2) - msr sctlr_el2, x0 - +1: #ifdef CONFIG_ARM64_VHE /* - * Check for VHE being present. For the rest of the EL2 setup, - * x2 being non-zero indicates that we do have VHE, and that the - * kernel is intended to run at EL2. + * Check for VHE being present. x2 being non-zero indicates that we + * do have VHE, and that the kernel is intended to run at EL2. */ mrs x2, id_aa64mmfr1_el1 ubfx x2, x2, #ID_AA64MMFR1_VHE_SHIFT, #4 -#else - mov x2, xzr -#endif + cbz x2, el2_setup_nvhe - /* Hyp configuration. */ - mov_q x0, HCR_HOST_NVHE_FLAGS - cbz x2, set_hcr mov_q x0, HCR_HOST_VHE_FLAGS -set_hcr: msr hcr_el2, x0 isb - /* - * Allow Non-secure EL1 and EL0 to access physical timer and counter. - * This is not necessary for VHE, since the host kernel runs in EL2, - * and EL0 accesses are configured in the later stage of boot process. - * Note that when HCR_EL2.E2H == 1, CNTHCTL_EL2 has the same bit layout - * as CNTKCTL_EL1, and CNTKCTL_EL1 accessing instructions are redefined - * to access CNTHCTL_EL2. This allows the kernel designed to run at EL1 - * to transparently mess with the EL0 bits via CNTKCTL_EL1 access in - * EL2. - */ - cbnz x2, 1f - mrs x0, cnthctl_el2 - orr x0, x0, #3 // Enable EL1 physical timers - msr cnthctl_el2, x0 -1: - msr cntvoff_el2, xzr // Clear virtual offset - -#ifdef CONFIG_ARM_GIC_V3 - /* GICv3 system register access */ - mrs x0, id_aa64pfr0_el1 - ubfx x0, x0, #ID_AA64PFR0_GIC_SHIFT, #4 - cbz x0, 3f - - mrs_s x0, SYS_ICC_SRE_EL2 - orr x0, x0, #ICC_SRE_EL2_SRE // Set ICC_SRE_EL2.SRE==1 - orr x0, x0, #ICC_SRE_EL2_ENABLE // Set ICC_SRE_EL2.Enable==1 - msr_s SYS_ICC_SRE_EL2, x0 - isb // Make sure SRE is now set - mrs_s x0, SYS_ICC_SRE_EL2 // Read SRE back, - tbz x0, #0, 3f // and check that it sticks - msr_s SYS_ICH_HCR_EL2, xzr // Reset ICC_HCR_EL2 to defaults - -3: -#endif - - /* Populate ID registers. */ - mrs x0, midr_el1 - mrs x1, mpidr_el1 - msr vpidr_el2, x0 - msr vmpidr_el2, x1 - -#ifdef CONFIG_COMPAT - msr hstr_el2, xzr // Disable CP15 traps to EL2 -#endif - - /* EL2 debug */ - mrs x1, id_aa64dfr0_el1 - sbfx x0, x1, #ID_AA64DFR0_PMUVER_SHIFT, #4 - cmp x0, #1 - b.lt 4f // Skip if no PMU present - mrs x0, pmcr_el0 // Disable debug access traps - ubfx x0, x0, #11, #5 // to EL2 and allow access to -4: - csel x3, xzr, x0, lt // all PMU counters from EL1 - - /* Statistical profiling */ - ubfx x0, x1, #ID_AA64DFR0_PMSVER_SHIFT, #4 - cbz x0, 7f // Skip if SPE not present - cbnz x2, 6f // VHE? - mrs_s x4, SYS_PMBIDR_EL1 // If SPE available at EL2, - and x4, x4, #(1 << SYS_PMBIDR_EL1_P_SHIFT) - cbnz x4, 5f // then permit sampling of physical - mov x4, #(1 << SYS_PMSCR_EL2_PCT_SHIFT | \ - 1 << SYS_PMSCR_EL2_PA_SHIFT) - msr_s SYS_PMSCR_EL2, x4 // addresses and physical counter -5: - mov x1, #(MDCR_EL2_E2PB_MASK << MDCR_EL2_E2PB_SHIFT) - orr x3, x3, x1 // If we don't have VHE, then - b 7f // use EL1&0 translation. -6: // For VHE, use EL2 translation - orr x3, x3, #MDCR_EL2_TPMS // and disable access from EL1 -7: - msr mdcr_el2, x3 // Configure debug traps - - /* LORegions */ - mrs x1, id_aa64mmfr1_el1 - ubfx x0, x1, #ID_AA64MMFR1_LOR_SHIFT, 4 - cbz x0, 1f - msr_s SYS_LORC_EL1, xzr -1: - - /* Stage-2 translation */ - msr vttbr_el2, xzr - - cbz x2, install_el2_stub + init_el2_state vhe mov w0, #BOOT_CPU_MODE_EL2 // This CPU booted in EL2 isb ret +#endif -SYM_INNER_LABEL(install_el2_stub, SYM_L_LOCAL) - /* - * When VHE is not in use, early init of EL2 and EL1 needs to be - * done here. - * When VHE _is_ in use, EL1 will not be used in the host and - * requires no configuration, and all non-hyp-specific EL2 setup - * will be done via the _EL1 system register aliases in __cpu_setup. - */ - mov_q x0, (SCTLR_EL1_RES1 | ENDIAN_SET_EL1) - msr sctlr_el1, x0 - - /* Coprocessor traps. */ - mov x0, #0x33ff - msr cptr_el2, x0 // Disable copro. traps to EL2 - - /* SVE register access */ - mrs x1, id_aa64pfr0_el1 - ubfx x1, x1, #ID_AA64PFR0_SVE_SHIFT, #4 - cbz x1, 7f - - bic x0, x0, #CPTR_EL2_TZ // Also disable SVE traps - msr cptr_el2, x0 // Disable copro. traps to EL2 +SYM_INNER_LABEL(el2_setup_nvhe, SYM_L_LOCAL) + mov_q x0, HCR_HOST_NVHE_FLAGS + msr hcr_el2, x0 isb - mov x1, #ZCR_ELx_LEN_MASK // SVE: Enable full vector - msr_s SYS_ZCR_EL2, x1 // length for EL1. + + init_el2_state nvhe /* Hypervisor stub */ -7: adr_l x0, __hyp_stub_vectors + adr_l x0, __hyp_stub_vectors msr vbar_el2, x0 - /* spsr */ - mov x0, #(PSR_F_BIT | PSR_I_BIT | PSR_A_BIT | PSR_D_BIT |\ - PSR_MODE_EL1h) - msr spsr_el2, x0 + mov_q x0, (SCTLR_EL1_RES1 | ENDIAN_SET_EL1) + msr sctlr_el1, x0 msr elr_el2, lr mov w0, #BOOT_CPU_MODE_EL2 // This CPU booted in EL2 eret From patchwork Wed Dec 2 18:41:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11946757 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9DD7AC71155 for ; Wed, 2 Dec 2020 18:44:32 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3B23C206D5 for ; Wed, 2 Dec 2020 18:44:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3B23C206D5 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=lS+msw3GJ22QFwxr22AFfG4L3QKoBnEXHbcJ09/pHD0=; b=SMoSyd1FyGoCb9od5kIe5NEZy YW3uk+iWGODmJfpEYHJamlBcOeXYBNNQhZoQ5osjXFefknWJSLuQZfLGJysTgRAu9su5it89bg3Ep MavoE1SI5vO4pKFdk/lli6vcYEbXh3VjwpczWOenBcitqjFFgyMsoWgrOAoNHunqHf1WvTfxatnGc QLtdkGP1ajZHq7GMyN0iyNQYpAtgfQ1flXQEEy8he1xgFLApg1MkmOz8CF6qqVd14lrWRfgN09YCV qIIejL8/8khHk5BoCSC92cjqS8GCLENKd9yGdjW9GcTag0F5OGddttPJS/rGevLca8rStFcMvk7+V cGKtC2MQw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX5H-00006B-SP; Wed, 02 Dec 2020 18:42:36 +0000 Received: from mail-wm1-x342.google.com ([2a00:1450:4864:20::342]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX4W-0008F0-HL for linux-arm-kernel@lists.infradead.org; Wed, 02 Dec 2020 18:41:49 +0000 Received: by mail-wm1-x342.google.com with SMTP id k10so10121936wmi.3 for ; Wed, 02 Dec 2020 10:41:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=pY9jRbioJhJf7MwxwtV14CQQVHe+rZSO6XuQoMfB3LY=; b=NL0mNcnpizqXS1UX9W9Xx72wdHMsPuW8ohq6hfiQ1SgVzyFUmbVUVXO+UIp6cX3GXm 3RCq57FWsycLSju3jZUWCY5zUMAXA5v0bk+c/SHgZ+/DxvtNWejec3R7jeVgNTVOb2HP bZ5FDOGmbS1J0e0Jdjsloi7TwnD2P14i6sFptC1oDSTb6TGcbtiIWyzIngfHvHg0nt67 r0zCYXyh+Hl9naVwqPp7J42lS4ZqCn+t8zbHpiCjyDpyOfMByZmYCs7dHw2V0ZQZar/0 cOrWwZ0Co16GFlvenZ9YCMR2+Dhj3+aApPouoNTu4ibFPYqp0AtKOFRF+GOJULHxAdVc E+tA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pY9jRbioJhJf7MwxwtV14CQQVHe+rZSO6XuQoMfB3LY=; b=S1bIrtiTvtKj6Hm/eUwaijMh+UHSjKtHb7GJ3WzmFafBybATRvb+UOJ5axQpPXDpSH dSuBv/Z2yxQ2IWv18PwFGdwbeUh4ms7bKt54b2J5z/5Dtk3ekRFYVYArcuFhpmM5e+30 3UaIWKR2CmELMHhzgqFnBKyNpLUfxmM4iP+jl62CplWXFVr34rH/Lp/+XtA2JaZxMRKy oGV6YAv2WEyIOgFjPEOmLTcLtOPwDgbzkYhSE1cu6G2j87OZiC/qy6JZZtDiKZRtbZlD 8gBQ7qXu1ej9MmpnhrGEK/3j1Gg3rdkpddJZwI6P2oUNbU/Wf5EqzaRUHmtM1TabU2rP 18zQ== X-Gm-Message-State: AOAM530liH5bI234YDEfzsJFBGi4JhlWpzzOx0nxcQTznNPJbTDqX7GO O8YGbk77fcqbesMpuzHk68F2Qw== X-Google-Smtp-Source: ABdhPJz89vM2fd/nDXaPHt9j0cPFTGj1oyMk3PstTs/+u86wCoX9DNGJNGbi4MtwwUU+yDdDBPf3FQ== X-Received: by 2002:a1c:328a:: with SMTP id y132mr4632122wmy.134.1606934507192; Wed, 02 Dec 2020 10:41:47 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:5ddd:b7c5:e3c9:e87a]) by smtp.gmail.com with ESMTPSA id n128sm3159374wmb.46.2020.12.02.10.41.45 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 02 Dec 2020 10:41:46 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v4 09/26] kvm: arm64: Remove vector_ptr param of hyp-init Date: Wed, 2 Dec 2020 18:41:05 +0000 Message-Id: <20201202184122.26046-10-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201202184122.26046-1-dbrazdil@google.com> References: <20201202184122.26046-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201202_134148_621100_F7D21538 X-CRM114-Status: GOOD ( 19.15 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org KVM precomputes the hyp VA of __kvm_hyp_host_vector, essentially a constant (minus ASLR), before passing it to __kvm_hyp_init. Now that we have alternatives for converting kimg VA to hyp VA, replace this with computing the constant inside __kvm_hyp_init, thus removing the need for an argument. Signed-off-by: David Brazdil --- arch/arm64/include/asm/kvm_asm.h | 2 -- arch/arm64/include/asm/kvm_mmu.h | 24 ++++++++++++++++++++++++ arch/arm64/kvm/arm.c | 4 +--- arch/arm64/kvm/hyp/nvhe/hyp-init.S | 9 ++++++--- 4 files changed, 31 insertions(+), 8 deletions(-) diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h index 4a6a77d8d13e..531f9d04eefd 100644 --- a/arch/arm64/include/asm/kvm_asm.h +++ b/arch/arm64/include/asm/kvm_asm.h @@ -163,10 +163,8 @@ struct kvm_vcpu; struct kvm_s2_mmu; DECLARE_KVM_NVHE_SYM(__kvm_hyp_init); -DECLARE_KVM_NVHE_SYM(__kvm_hyp_host_vector); DECLARE_KVM_HYP_SYM(__kvm_hyp_vector); #define __kvm_hyp_init CHOOSE_NVHE_SYM(__kvm_hyp_init) -#define __kvm_hyp_host_vector CHOOSE_NVHE_SYM(__kvm_hyp_host_vector) #define __kvm_hyp_vector CHOOSE_HYP_SYM(__kvm_hyp_vector) extern unsigned long kvm_arm_hyp_percpu_base[NR_CPUS]; diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h index e298191a854d..e52d82aeadca 100644 --- a/arch/arm64/include/asm/kvm_mmu.h +++ b/arch/arm64/include/asm/kvm_mmu.h @@ -94,6 +94,30 @@ alternative_cb_end sub \reg, \reg, \tmp .endm +/* + * Convert a kernel image address to a hyp VA + * reg: kernel address to be converted in place + * tmp: temporary register + * + * The actual code generation takes place in kvm_get_kimage_voffset, and + * the instructions below are only there to reserve the space and + * perform the register allocation (kvm_update_kimg_phys_offset uses the + * specific registers encoded in the instructions). + */ +.macro kimg_hyp_va reg, tmp +alternative_cb kvm_update_kimg_phys_offset + movz \tmp, #0 + movk \tmp, #0, lsl #16 + movk \tmp, #0, lsl #32 + movk \tmp, #0, lsl #48 +alternative_cb_end + + sub \reg, \reg, \tmp + mov_q \tmp, PAGE_OFFSET + orr \reg, \reg, \tmp + kern_hyp_va \reg +.endm + #else #include diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index d9e8d7dc7e11..9e5d0ea84822 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1386,7 +1386,6 @@ static void cpu_init_hyp_mode(void) { phys_addr_t pgd_ptr; unsigned long hyp_stack_ptr; - unsigned long vector_ptr; unsigned long tpidr_el2; struct arm_smccc_res res; @@ -1404,7 +1403,6 @@ static void cpu_init_hyp_mode(void) pgd_ptr = kvm_mmu_get_httbr(); hyp_stack_ptr = __this_cpu_read(kvm_arm_hyp_stack_page) + PAGE_SIZE; hyp_stack_ptr = kern_hyp_va(hyp_stack_ptr); - vector_ptr = (unsigned long)kern_hyp_va(kvm_ksym_ref(__kvm_hyp_host_vector)); /* * Call initialization code, and switch to the full blown HYP code. @@ -1414,7 +1412,7 @@ static void cpu_init_hyp_mode(void) */ BUG_ON(!system_capabilities_finalized()); arm_smccc_1_1_hvc(KVM_HOST_SMCCC_FUNC(__kvm_hyp_init), - pgd_ptr, tpidr_el2, hyp_stack_ptr, vector_ptr, &res); + pgd_ptr, tpidr_el2, hyp_stack_ptr, &res); WARN_ON(res.a0 != SMCCC_RET_SUCCESS); /* diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-init.S b/arch/arm64/kvm/hyp/nvhe/hyp-init.S index b11a9d7db677..931a8c38f085 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-init.S +++ b/arch/arm64/kvm/hyp/nvhe/hyp-init.S @@ -50,7 +50,6 @@ __invalid: * x1: HYP pgd * x2: per-CPU offset * x3: HYP stack - * x4: HYP vectors */ __do_hyp_init: /* Check for a stub HVC call */ @@ -134,9 +133,13 @@ alternative_else_nop_endif msr sctlr_el2, x0 isb - /* Set the stack and new vectors */ + /* Set the stack */ mov sp, x3 - msr vbar_el2, x4 + + /* Set the host vector */ + ldr x0, =__kvm_hyp_host_vector + kimg_hyp_va x0, x1 + msr vbar_el2, x0 /* Hello, World! */ mov x0, #SMCCC_RET_SUCCESS From patchwork Wed Dec 2 18:41:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11946759 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 507D4C71155 for ; Wed, 2 Dec 2020 18:44:40 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DF79F206D5 for ; Wed, 2 Dec 2020 18:44:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DF79F206D5 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=Tkb7luOZRm7qhp6UZja+3awB4jlsw05amVnQHZ6EOAg=; b=ARpDjtMcmNrQbnza1jCEgpXue S8VXQE63rBZFaiQEDcs86wch0jVarRi/ZJDEL3bR6QntVCvmMN0zZf4sVnO+/l5NHw+F/fzgP1qDw jw8oaKOPikEAiaclNu3STKXRwVHcHOISL1R97KEfpAV2rycfm9TKk1dQymmRMDpJTeQmet8QDsnfw svpsMedZJWl5VrXZ1Ww87pl2etidtVYEcCMJWEXEg5AQItUHLzcJCzLqnpeCpK7sT7xOzzUZpbGbY bccwfIQkct6LdJVcx/B6E647S6nuaxjOFtEF4XWrIkbACnevkBVfVWQtvZIvcHodXOrSfUByi2jnp 9WtT2lBvQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX5Q-0000DZ-Uy; Wed, 02 Dec 2020 18:42:44 +0000 Received: from mail-wr1-x441.google.com ([2a00:1450:4864:20::441]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX4Y-0008Fx-LK for linux-arm-kernel@lists.infradead.org; Wed, 02 Dec 2020 18:41:52 +0000 Received: by mail-wr1-x441.google.com with SMTP id 23so5120752wrc.8 for ; Wed, 02 Dec 2020 10:41:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=y2XR1ngTwyxpwjENRmmRKJhts/5F/OilmxQWITQoZvQ=; b=JzFgXAhlIEgLNkP4PvPB0wdamug+KFMjHLhHpHzd5kVnUCr3t8QDs9NimLkHc7C2o1 NHgnIE3t1mWj8BqptP+PMm6hDZaxCYbA61W91okeustzdOUbrpBE3haZ3EpLu+b4Um9B C6HdAF8lpWpxP6zdmtUviuKmhFUj6Ywy3uhuUFdMiFzvCrXNeTUowX1pkb/61/PpdGYi mi5ulgnhbOZUrzv04KMRwsqsomq3Hq2X00uDYDjoGUXUbr1uO87CLOBRPYMu5eGe3evA 0h9k5iIdnG+5R9qdoaXDx7ucCbBGaDZXX6pISlsr23VUmTinP+qIJEQmBhPC2xjZCh3e gBZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=y2XR1ngTwyxpwjENRmmRKJhts/5F/OilmxQWITQoZvQ=; b=KxOGHZxVa3FV1+Rpdu2OWXNGrQTQ9OYm3t/Dr0P/GoInbyPHY/SyUv9ceHVhvVak5p RL0Z9d2OjWUowKxGSk6D/wHTFeWmhVhdHgg275jgD+GHO2f2Khz/xmgZ3RNRkZXqqfHN bB+9TUZRH4s6+hJMCenabXdK4RcjDMClsvB8v4G6Ytdu4Uf3lYoUsqpTIfLAi4dEwgA1 uWbXEM3/8oRE4c2jbdxRanxDzJsp3zVe3cIhpNdr/FMOsDv6YuWgKMpOHOyKGMPi8ZPN LW14RJdt3Cw6x6ogw50Vlp4drLbWqyNRf8/kA3NVnHY95Dz31298wcELwS4VYU+AnKXQ aW9A== X-Gm-Message-State: AOAM533utAz0e2vyEEVfEY/AElLosaMGZuwS3J5F84//Wf4gB8Vpc4Ww iNUudh0d7mMDeJnCrUeZvsoQjA== X-Google-Smtp-Source: ABdhPJxmCviRUJKT+618pJo3OH+62/YOXf5UY9+1uIzxjn3ExQTbBk2XRMQ55+E62EXgsGkfOiqfVw== X-Received: by 2002:adf:fb06:: with SMTP id c6mr5064164wrr.117.1606934509539; Wed, 02 Dec 2020 10:41:49 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:5ddd:b7c5:e3c9:e87a]) by smtp.gmail.com with ESMTPSA id 90sm3102130wra.95.2020.12.02.10.41.48 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 02 Dec 2020 10:41:48 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v4 10/26] kvm: arm64: Move hyp-init params to a per-CPU struct Date: Wed, 2 Dec 2020 18:41:06 +0000 Message-Id: <20201202184122.26046-11-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201202184122.26046-1-dbrazdil@google.com> References: <20201202184122.26046-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201202_134150_808684_168574C7 X-CRM114-Status: GOOD ( 23.65 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Once we start initializing KVM on newly booted cores before the rest of the kernel, parameters to __do_hyp_init will need to be provided by EL2 rather than EL1. At that point it will not be possible to pass its three arguments directly because PSCI_CPU_ON only supports one context argument. Refactor __do_hyp_init to accept its parameters in a struct. This prepares the code for KVM booting cores as well as removes any limits on the number of __do_hyp_init arguments. Signed-off-by: David Brazdil --- arch/arm64/include/asm/kvm_asm.h | 6 ++++++ arch/arm64/include/asm/kvm_hyp.h | 2 +- arch/arm64/kernel/asm-offsets.c | 3 +++ arch/arm64/kvm/arm.c | 23 +++++++++++++---------- arch/arm64/kvm/hyp/nvhe/hyp-init.S | 16 +++++++--------- arch/arm64/kvm/hyp/nvhe/hyp-main.c | 2 ++ 6 files changed, 32 insertions(+), 20 deletions(-) diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h index 531f9d04eefd..1a7b91534a16 100644 --- a/arch/arm64/include/asm/kvm_asm.h +++ b/arch/arm64/include/asm/kvm_asm.h @@ -148,6 +148,12 @@ extern void *__vhe_undefined_symbol; #endif +struct kvm_nvhe_init_params { + unsigned long tpidr_el2; + unsigned long stack_hyp_va; + phys_addr_t pgd_pa; +}; + /* Translate a kernel address @ptr into its equivalent linear mapping */ #define kvm_ksym_ref(ptr) \ ({ \ diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h index 6b664de5ec1f..cb25c15e3d8d 100644 --- a/arch/arm64/include/asm/kvm_hyp.h +++ b/arch/arm64/include/asm/kvm_hyp.h @@ -14,6 +14,7 @@ DECLARE_PER_CPU(struct kvm_cpu_context, kvm_hyp_ctxt); DECLARE_PER_CPU(unsigned long, kvm_hyp_vector); +DECLARE_PER_CPU(struct kvm_nvhe_init_params, kvm_init_params); #define read_sysreg_elx(r,nvh,vh) \ ({ \ @@ -98,4 +99,3 @@ void __noreturn __hyp_do_panic(bool restore_host, u64 spsr, u64 elr, u64 par); #endif #endif /* __ARM64_KVM_HYP_H__ */ - diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index 7d32fc959b1a..8d6272a01a00 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -110,6 +110,9 @@ int main(void) DEFINE(CPU_APGAKEYLO_EL1, offsetof(struct kvm_cpu_context, sys_regs[APGAKEYLO_EL1])); DEFINE(HOST_CONTEXT_VCPU, offsetof(struct kvm_cpu_context, __hyp_running_vcpu)); DEFINE(HOST_DATA_CONTEXT, offsetof(struct kvm_host_data, host_ctxt)); + DEFINE(NVHE_INIT_TPIDR_EL2, offsetof(struct kvm_nvhe_init_params, tpidr_el2)); + DEFINE(NVHE_INIT_STACK_HYP_VA, offsetof(struct kvm_nvhe_init_params, stack_hyp_va)); + DEFINE(NVHE_INIT_PGD_PA, offsetof(struct kvm_nvhe_init_params, pgd_pa)); #endif #ifdef CONFIG_CPU_PM DEFINE(CPU_CTX_SP, offsetof(struct cpu_suspend_ctx, sp)); diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 9e5d0ea84822..93a408c00249 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -51,6 +51,7 @@ DECLARE_KVM_HYP_PER_CPU(unsigned long, kvm_hyp_vector); static DEFINE_PER_CPU(unsigned long, kvm_arm_hyp_stack_page); unsigned long kvm_arm_hyp_percpu_base[NR_CPUS]; +DECLARE_KVM_NVHE_PER_CPU(struct kvm_nvhe_init_params, kvm_init_params); /* The VMID used in the VTTBR */ static atomic64_t kvm_vmid_gen = ATOMIC64_INIT(1); @@ -1384,9 +1385,7 @@ static int kvm_init_vector_slots(void) static void cpu_init_hyp_mode(void) { - phys_addr_t pgd_ptr; - unsigned long hyp_stack_ptr; - unsigned long tpidr_el2; + struct kvm_nvhe_init_params *params = this_cpu_ptr_nvhe_sym(kvm_init_params); struct arm_smccc_res res; /* Switch from the HYP stub to our own HYP init vector */ @@ -1397,12 +1396,17 @@ static void cpu_init_hyp_mode(void) * kernel's mapping to the linear mapping, and store it in tpidr_el2 * so that we can use adr_l to access per-cpu variables in EL2. */ - tpidr_el2 = (unsigned long)this_cpu_ptr_nvhe_sym(__per_cpu_start) - - (unsigned long)kvm_ksym_ref(CHOOSE_NVHE_SYM(__per_cpu_start)); + params->tpidr_el2 = (unsigned long)this_cpu_ptr_nvhe_sym(__per_cpu_start) - + (unsigned long)kvm_ksym_ref(CHOOSE_NVHE_SYM(__per_cpu_start)); - pgd_ptr = kvm_mmu_get_httbr(); - hyp_stack_ptr = __this_cpu_read(kvm_arm_hyp_stack_page) + PAGE_SIZE; - hyp_stack_ptr = kern_hyp_va(hyp_stack_ptr); + params->stack_hyp_va = kern_hyp_va(__this_cpu_read(kvm_arm_hyp_stack_page) + PAGE_SIZE); + params->pgd_pa = kvm_mmu_get_httbr(); + + /* + * Flush the init params from the data cache because the struct will + * be read while the MMU is off. + */ + kvm_flush_dcache_to_poc(params, sizeof(*params)); /* * Call initialization code, and switch to the full blown HYP code. @@ -1411,8 +1415,7 @@ static void cpu_init_hyp_mode(void) * cpus_have_const_cap() wrapper. */ BUG_ON(!system_capabilities_finalized()); - arm_smccc_1_1_hvc(KVM_HOST_SMCCC_FUNC(__kvm_hyp_init), - pgd_ptr, tpidr_el2, hyp_stack_ptr, &res); + arm_smccc_1_1_hvc(KVM_HOST_SMCCC_FUNC(__kvm_hyp_init), virt_to_phys(params), &res); WARN_ON(res.a0 != SMCCC_RET_SUCCESS); /* diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-init.S b/arch/arm64/kvm/hyp/nvhe/hyp-init.S index 931a8c38f085..e712e317337c 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-init.S +++ b/arch/arm64/kvm/hyp/nvhe/hyp-init.S @@ -47,9 +47,7 @@ __invalid: /* * x0: SMCCC function ID - * x1: HYP pgd - * x2: per-CPU offset - * x3: HYP stack + * x1: struct kvm_nvhe_init_params PA */ __do_hyp_init: /* Check for a stub HVC call */ @@ -70,10 +68,13 @@ __do_hyp_init: mov x0, #SMCCC_RET_NOT_SUPPORTED eret -1: - /* Set tpidr_el2 for use by HYP to free a register */ - msr tpidr_el2, x2 +1: ldr x0, [x1, #NVHE_INIT_TPIDR_EL2] + msr tpidr_el2, x0 + ldr x0, [x1, #NVHE_INIT_STACK_HYP_VA] + mov sp, x0 + + ldr x1, [x1, #NVHE_INIT_PGD_PA] phys_to_ttbr x0, x1 alternative_if ARM64_HAS_CNP orr x0, x0, #TTBR_CNP_BIT @@ -133,9 +134,6 @@ alternative_else_nop_endif msr sctlr_el2, x0 isb - /* Set the stack */ - mov sp, x3 - /* Set the host vector */ ldr x0, =__kvm_hyp_host_vector kimg_hyp_va x0, x1 diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c index 82df7fc24760..a4f1cac714d7 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -16,6 +16,8 @@ #define DECLARE_REG(type, name, ctxt, reg) \ type name = (type)cpu_reg(ctxt, (reg)) +DEFINE_PER_CPU(struct kvm_nvhe_init_params, kvm_init_params); + static void handle___kvm_vcpu_run(struct kvm_cpu_context *host_ctxt) { DECLARE_REG(struct kvm_vcpu *, vcpu, host_ctxt, 1); From patchwork Wed Dec 2 18:41:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11946761 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E6295C71155 for ; Wed, 2 Dec 2020 18:44:54 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 72F6A206D5 for ; Wed, 2 Dec 2020 18:44:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 72F6A206D5 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=tnCwRJsX5+g5kpqdlBgtm3eBWK91NPAU2IzULOA844k=; b=u8SgR7vpwc3N1ozZg9sTeMTTT DkmwqyK5ei++gjYdjabnLaBjAqyf8iBDnrqGI2bVb2+6ykUQkwR+0EvuVLeFnldv2nJ4RouIc+yo9 u3008aPzyAOCfcpc1PXbbb8KZxu53Eg+p47+OU5WBNnpamYQEQuFBNz2axZb2+8iBru7pQXI7Nofk Dbzqo296ySgiAIGu4CVfX5T/Jk+7KuKVubuTc0yxO/P0p2jqKMPiNHGLl2tGLejei/uhH/ikZjIVE h1dpO4qlybbq8VI7XxG/M2CQ3g4zAAAKDDrMcRLnMOh1HtV9HyNaZOfqln1p5lxXOYwZiAWNwSQ8y YnFEW0fCg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX5b-0000JS-K0; Wed, 02 Dec 2020 18:42:55 +0000 Received: from mail-wr1-x441.google.com ([2a00:1450:4864:20::441]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX4a-0008Go-W4 for linux-arm-kernel@lists.infradead.org; Wed, 02 Dec 2020 18:41:55 +0000 Received: by mail-wr1-x441.google.com with SMTP id p8so5162351wrx.5 for ; Wed, 02 Dec 2020 10:41:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=SYG0eeQv/KzkRV71YTMJbmdpg/6//a3ujvezKDQIbk8=; b=I6qPvyU/FEX7FoJaBAPoMr8c0thpJsMeXHt3MTW4C/mlzQSsWMAIep+7zln0pouZ69 oSIqQIHhphrOpyf9KrtyyKVpZ9ws63s0hMyFc+TmZV9P0M0reBlv34xK9/mgCDapBvZ2 HBRTDBpFTKJ2JHroGfWfnXTwvM09Kp5Vt2NzgcNYSoe4gOQvAxueMU4QmxSgQirMH/CQ vfnDSWQq6MXCHPHO+j9hsFHgxgyms8RJGLjJecLRktMuO1/ofiirUVZM614qboCwTRuF q9kPFpqy5ZGaltgubW7eWPEk+L44OIsBifxsw8vU4w66d+3q7Z8ZlWArOrfUwJX0PPZV neWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=SYG0eeQv/KzkRV71YTMJbmdpg/6//a3ujvezKDQIbk8=; b=R3/zA1l9M1z8FlBUKpgV4IHVpd9klb3T8QpR+4GovjqxZhxSt89GF4a0NtIWtDfA07 v7ia4w0P5dOO+yQBoUzeirHDE+j0by7TbTU/lT6D7SqgElNU1NSHJreHeQjhZDZVaT8L /zsUsa/4FnTwKkVsCYZgm+EnvupXBNMjrGLjxma/6WJjGKw7wEa8PBCfHMqQLb7Xvzrm 64Vw0iWhesux1dn0v+INZdxrYbCoz0iNgWzgZb5flJahcU7BvlUdE5ToXYnXF0JhJYO9 c9yF6J84fyC8W3KIOMKeioHuJ5xWgVfcmQEizMBct4KOxagARIy7b+ABh/uWbdP++Mt+ 2WZQ== X-Gm-Message-State: AOAM532ruBAFG8x3ILnYnuvaSqdUo+ZHqq/dGWV0NgQjAUydEyduZB+t gXrpffQ9ossMF/YKgiHPBY9oCA== X-Google-Smtp-Source: ABdhPJzQ4njSIdl4L+IWqJBBxyUMLi1VeNNJl71IrmZZYtKGG+ycdDeDDGy/czwUB2v1OUbzSwBadg== X-Received: by 2002:adf:f8d2:: with SMTP id f18mr4955628wrq.379.1606934511842; Wed, 02 Dec 2020 10:41:51 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:5ddd:b7c5:e3c9:e87a]) by smtp.gmail.com with ESMTPSA id l14sm2962159wmi.33.2020.12.02.10.41.50 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 02 Dec 2020 10:41:50 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v4 11/26] kvm: arm64: Init MAIR/TCR_EL2 from params struct Date: Wed, 2 Dec 2020 18:41:07 +0000 Message-Id: <20201202184122.26046-12-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201202184122.26046-1-dbrazdil@google.com> References: <20201202184122.26046-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201202_134153_128064_70850179 X-CRM114-Status: GOOD ( 22.20 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org MAIR_EL2 and TCR_EL2 are currently initialized from their _EL1 values. This will not work once KVM starts intercepting PSCI ON/SUSPEND SMCs and initializing EL2 state before EL1 state. Obtain the EL1 values during KVM init and store them in the init params struct. The struct will stay in memory and can be used when booting new cores. Take the opportunity to move copying the T0SZ value from idmap_t0sz in KVM init rather than in .hyp.idmap.text. This avoids the need for the idmap_t0sz symbol alias. Signed-off-by: David Brazdil --- arch/arm64/include/asm/kvm_asm.h | 2 ++ arch/arm64/kernel/asm-offsets.c | 2 ++ arch/arm64/kernel/image-vars.h | 3 --- arch/arm64/kvm/arm.c | 22 +++++++++++++++++ arch/arm64/kvm/hyp/nvhe/hyp-init.S | 38 +++++++----------------------- 5 files changed, 34 insertions(+), 33 deletions(-) diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h index 1a7b91534a16..7ccf770c53d9 100644 --- a/arch/arm64/include/asm/kvm_asm.h +++ b/arch/arm64/include/asm/kvm_asm.h @@ -149,6 +149,8 @@ extern void *__vhe_undefined_symbol; #endif struct kvm_nvhe_init_params { + unsigned long mair_el2; + unsigned long tcr_el2; unsigned long tpidr_el2; unsigned long stack_hyp_va; phys_addr_t pgd_pa; diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index 8d6272a01a00..ba01185ef281 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -110,6 +110,8 @@ int main(void) DEFINE(CPU_APGAKEYLO_EL1, offsetof(struct kvm_cpu_context, sys_regs[APGAKEYLO_EL1])); DEFINE(HOST_CONTEXT_VCPU, offsetof(struct kvm_cpu_context, __hyp_running_vcpu)); DEFINE(HOST_DATA_CONTEXT, offsetof(struct kvm_host_data, host_ctxt)); + DEFINE(NVHE_INIT_MAIR_EL2, offsetof(struct kvm_nvhe_init_params, mair_el2)); + DEFINE(NVHE_INIT_TCR_EL2, offsetof(struct kvm_nvhe_init_params, tcr_el2)); DEFINE(NVHE_INIT_TPIDR_EL2, offsetof(struct kvm_nvhe_init_params, tpidr_el2)); DEFINE(NVHE_INIT_STACK_HYP_VA, offsetof(struct kvm_nvhe_init_params, stack_hyp_va)); DEFINE(NVHE_INIT_PGD_PA, offsetof(struct kvm_nvhe_init_params, pgd_pa)); diff --git a/arch/arm64/kernel/image-vars.h b/arch/arm64/kernel/image-vars.h index 4b32588918d9..08e69faedf6c 100644 --- a/arch/arm64/kernel/image-vars.h +++ b/arch/arm64/kernel/image-vars.h @@ -77,9 +77,6 @@ KVM_NVHE_ALIAS(panic); /* Vectors installed by hyp-init on reset HVC. */ KVM_NVHE_ALIAS(__hyp_stub_vectors); -/* IDMAP TCR_EL1.T0SZ as computed by the EL1 init code */ -KVM_NVHE_ALIAS(idmap_t0sz); - /* Kernel symbol used by icache_is_vpipt(). */ KVM_NVHE_ALIAS(__icache_flags); diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 93a408c00249..6c8594378865 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1387,6 +1387,7 @@ static void cpu_init_hyp_mode(void) { struct kvm_nvhe_init_params *params = this_cpu_ptr_nvhe_sym(kvm_init_params); struct arm_smccc_res res; + unsigned long tcr; /* Switch from the HYP stub to our own HYP init vector */ __hyp_set_vectors(kvm_get_idmap_vector()); @@ -1399,6 +1400,27 @@ static void cpu_init_hyp_mode(void) params->tpidr_el2 = (unsigned long)this_cpu_ptr_nvhe_sym(__per_cpu_start) - (unsigned long)kvm_ksym_ref(CHOOSE_NVHE_SYM(__per_cpu_start)); + params->mair_el2 = read_sysreg(mair_el1); + + /* + * The ID map may be configured to use an extended virtual address + * range. This is only the case if system RAM is out of range for the + * currently configured page size and VA_BITS, in which case we will + * also need the extended virtual range for the HYP ID map, or we won't + * be able to enable the EL2 MMU. + * + * However, at EL2, there is only one TTBR register, and we can't switch + * between translation tables *and* update TCR_EL2.T0SZ at the same + * time. Bottom line: we need to use the extended range with *both* our + * translation tables. + * + * So use the same T0SZ value we use for the ID map. + */ + tcr = (read_sysreg(tcr_el1) & TCR_EL2_MASK) | TCR_EL2_RES1; + tcr &= ~TCR_T0SZ_MASK; + tcr |= (idmap_t0sz & GENMASK(TCR_TxSZ_WIDTH - 1, 0)) << TCR_T0SZ_OFFSET; + params->tcr_el2 = tcr; + params->stack_hyp_va = kern_hyp_va(__this_cpu_read(kvm_arm_hyp_stack_page) + PAGE_SIZE); params->pgd_pa = kvm_mmu_get_httbr(); diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-init.S b/arch/arm64/kvm/hyp/nvhe/hyp-init.S index e712e317337c..712f57289357 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-init.S +++ b/arch/arm64/kvm/hyp/nvhe/hyp-init.S @@ -71,48 +71,26 @@ __do_hyp_init: 1: ldr x0, [x1, #NVHE_INIT_TPIDR_EL2] msr tpidr_el2, x0 + ldr x0, [x1, #NVHE_INIT_MAIR_EL2] + msr mair_el2, x0 + ldr x0, [x1, #NVHE_INIT_STACK_HYP_VA] mov sp, x0 - ldr x1, [x1, #NVHE_INIT_PGD_PA] - phys_to_ttbr x0, x1 + ldr x0, [x1, #NVHE_INIT_PGD_PA] + phys_to_ttbr x2, x0 alternative_if ARM64_HAS_CNP - orr x0, x0, #TTBR_CNP_BIT + orr x2, x2, #TTBR_CNP_BIT alternative_else_nop_endif - msr ttbr0_el2, x0 - - mrs x0, tcr_el1 - mov_q x1, TCR_EL2_MASK - and x0, x0, x1 - mov x1, #TCR_EL2_RES1 - orr x0, x0, x1 - - /* - * The ID map may be configured to use an extended virtual address - * range. This is only the case if system RAM is out of range for the - * currently configured page size and VA_BITS, in which case we will - * also need the extended virtual range for the HYP ID map, or we won't - * be able to enable the EL2 MMU. - * - * However, at EL2, there is only one TTBR register, and we can't switch - * between translation tables *and* update TCR_EL2.T0SZ at the same - * time. Bottom line: we need to use the extended range with *both* our - * translation tables. - * - * So use the same T0SZ value we use for the ID map. - */ - ldr_l x1, idmap_t0sz - bfi x0, x1, TCR_T0SZ_OFFSET, TCR_TxSZ_WIDTH + msr ttbr0_el2, x2 /* * Set the PS bits in TCR_EL2. */ + ldr x0, [x1, #NVHE_INIT_TCR_EL2] tcr_compute_pa_size x0, #TCR_EL2_PS_SHIFT, x1, x2 - msr tcr_el2, x0 - mrs x0, mair_el1 - msr mair_el2, x0 isb /* Invalidate the stale TLBs from Bootloader */ From patchwork Wed Dec 2 18:41:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11946763 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3CFF5C71155 for ; Wed, 2 Dec 2020 18:44:58 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 979FD206D5 for ; Wed, 2 Dec 2020 18:44:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 979FD206D5 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=3h67pz/5NZKHPyCfM1r+UFHPilgg70FvjN0TqvUNQQE=; b=K+MGTQz8kWeNEdGgmyefD2O+F P6xXyk/5099c08OEzbPiX1UZTiZKLeRLs8+7dbtI+RGhhM1nBXizTXBIKW4H9ZVWePVOY76El9CDh 1WJRzQEsqN4lf+0RPy/QdyNr93kYSlVupt0pQPhJ0OkMHbJbtzwsWOHRLTUkLzfQ4LR3Ip01Z/evF OoYFbBXw2ODmJnHDxdfIJEtcB1Gmu2I2/QuWpZ2hNyLIX/gDTy/pcq3JQnEd5TutG88PDm2SPmIUJ e6MQ4Mgb2bUjoKRUtUti7TNOIgxWtPYx2CEEwu0fOOzg2iXsU7pPZI1TPPHShurQliHzra9gXgTgN hMTjeyCVA==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX5n-0000Na-P5; Wed, 02 Dec 2020 18:43:08 +0000 Received: from mail-wm1-x342.google.com ([2a00:1450:4864:20::342]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX4d-0008Hn-5z for linux-arm-kernel@lists.infradead.org; Wed, 02 Dec 2020 18:41:57 +0000 Received: by mail-wm1-x342.google.com with SMTP id a6so10141750wmc.2 for ; Wed, 02 Dec 2020 10:41:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=FbhNL96u68QqM7Q1rkLQfNYMQNpyFncRvPJJEVMReqU=; b=aFE7s/N+LJ2214UmKW0aNWOqdMUPnXoa+EPsqwgnoGGxb8gEhdDmTcErY0lO7VX/of Pl4LzHwCPIYUm5h24H+uF+GVjDF8jfobBT3gBZqMIo6DJhMVbKU7Tear3xnCtVJhTA5Z H8UdHSoYKkI3GbRtMA+e9t2FLeHRS1USlflwgf2iXaGNryzWx/cGxM/H1WIW7F0TzFQm kUYSeVPW0o7VN8rQmnKCZzuQhKnOlQdi0fKh13kN7DNpfC847bpT3mlXKhcpfipcvWLM zXmrrrlzQf8R9QP/5CcjD6TobUxbU4+VxaXnueeA5C4NZSfWnZQsKClVCGz391574KZb MG+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FbhNL96u68QqM7Q1rkLQfNYMQNpyFncRvPJJEVMReqU=; b=lkKHoSjixVnFdq7ZrI5M45lGeUcn97Nl2rs7fGtvj0tq+sTIEOhR/6Nm+Z45eWTuoq H9yFrtX8nMQIakdn/mB09P3JyADoajRg6AGPVmBkSc6eOLAEQGRLKp+XijzE94DoKJfF 8WYPuTehGr0rklRXiS0LGFNGUQMBlRXe1DFraWm0s8Sg7sPUFhCn8bccbLwhH0gzIwA+ idDhbAc7M92gwwuXua/MDy+rqKkLcmynZp4lPenpefWjliJKUkwHT0n/AqiRFCX+GhlV yVjTKfnuroiyqBASovWND+sUU2hTSRSSx8U0M3biM9afOFcXRHLZhf9d37rxZiM9x/xd 1ejQ== X-Gm-Message-State: AOAM531Ei50zTllarRfjWLFQHyy97JEBRVn01CPjkUQQ0K6ww+OPQ0Y4 78YWLkP8z07x83AC2wylD0lPzQ== X-Google-Smtp-Source: ABdhPJzDonMAoe1/LUSeGrzjO/oX6VvyeXOfTR80+KWmzVofsLNtAQuxbUf5v7Ho6BG2owrCpnDifw== X-Received: by 2002:a05:600c:2306:: with SMTP id 6mr4532227wmo.53.1606934514097; Wed, 02 Dec 2020 10:41:54 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:5ddd:b7c5:e3c9:e87a]) by smtp.gmail.com with ESMTPSA id z140sm573266wmc.30.2020.12.02.10.41.52 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 02 Dec 2020 10:41:52 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v4 12/26] kvm: arm64: Add .hyp.data..ro_after_init ELF section Date: Wed, 2 Dec 2020 18:41:08 +0000 Message-Id: <20201202184122.26046-13-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201202184122.26046-1-dbrazdil@google.com> References: <20201202184122.26046-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201202_134155_531048_45DA99FF X-CRM114-Status: GOOD ( 20.30 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add rules for renaming the .data..ro_after_init ELF section in KVM nVHE object files to .hyp.data..ro_after_init, linking it into the kernel and mapping it in hyp at runtime. The section is RW to the host, then mapped RO in hyp. The expectation is that the host populates the variables in the section and they are never changed by hyp afterwards. Signed-off-by: David Brazdil --- arch/arm64/include/asm/sections.h | 1 + arch/arm64/kernel/vmlinux.lds.S | 10 ++++++++++ arch/arm64/kvm/arm.c | 8 ++++++++ arch/arm64/kvm/hyp/nvhe/hyp.lds.S | 1 + 4 files changed, 20 insertions(+) diff --git a/arch/arm64/include/asm/sections.h b/arch/arm64/include/asm/sections.h index 3994169985ef..8ff579361731 100644 --- a/arch/arm64/include/asm/sections.h +++ b/arch/arm64/include/asm/sections.h @@ -11,6 +11,7 @@ extern char __alt_instructions[], __alt_instructions_end[]; extern char __hibernate_exit_text_start[], __hibernate_exit_text_end[]; extern char __hyp_idmap_text_start[], __hyp_idmap_text_end[]; extern char __hyp_text_start[], __hyp_text_end[]; +extern char __hyp_data_ro_after_init_start[], __hyp_data_ro_after_init_end[]; extern char __idmap_text_start[], __idmap_text_end[]; extern char __initdata_begin[], __initdata_end[]; extern char __inittext_begin[], __inittext_end[]; diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index 1bda604f4c70..4382b5d0645d 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -30,6 +30,13 @@ jiffies = jiffies_64; *(__kvm_ex_table) \ __stop___kvm_ex_table = .; +#define HYPERVISOR_DATA_SECTIONS \ + HYP_SECTION_NAME(.data..ro_after_init) : { \ + __hyp_data_ro_after_init_start = .; \ + *(HYP_SECTION_NAME(.data..ro_after_init)) \ + __hyp_data_ro_after_init_end = .; \ + } + #define HYPERVISOR_PERCPU_SECTION \ . = ALIGN(PAGE_SIZE); \ HYP_SECTION_NAME(.data..percpu) : { \ @@ -37,6 +44,7 @@ jiffies = jiffies_64; } #else /* CONFIG_KVM */ #define HYPERVISOR_EXTABLE +#define HYPERVISOR_DATA_SECTIONS #define HYPERVISOR_PERCPU_SECTION #endif @@ -234,6 +242,8 @@ SECTIONS _sdata = .; RW_DATA(L1_CACHE_BYTES, PAGE_SIZE, THREAD_ALIGN) + HYPERVISOR_DATA_SECTIONS + /* * Data written with the MMU off but read with the MMU on requires * cache lines to be invalidated, discarding up to a Cache Writeback diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 6c8594378865..40857cbed3d1 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1705,6 +1705,14 @@ static int init_hyp_mode(void) goto out_err; } + err = create_hyp_mappings(kvm_ksym_ref(__hyp_data_ro_after_init_start), + kvm_ksym_ref(__hyp_data_ro_after_init_end), + PAGE_HYP_RO); + if (err) { + kvm_err("Cannot map .hyp.data..ro_after_init section\n"); + goto out_err; + } + err = create_hyp_mappings(kvm_ksym_ref(__start_rodata), kvm_ksym_ref(__end_rodata), PAGE_HYP_RO); if (err) { diff --git a/arch/arm64/kvm/hyp/nvhe/hyp.lds.S b/arch/arm64/kvm/hyp/nvhe/hyp.lds.S index bb2d986ff696..5d76ff2ba63e 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp.lds.S +++ b/arch/arm64/kvm/hyp/nvhe/hyp.lds.S @@ -16,4 +16,5 @@ SECTIONS { HYP_SECTION_NAME(.data..percpu) : { PERCPU_INPUT(L1_CACHE_BYTES) } + HYP_SECTION(.data..ro_after_init) } From patchwork Wed Dec 2 18:41:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11946765 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E2EDC64E7C for ; Wed, 2 Dec 2020 18:45:25 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7541D221FA for ; Wed, 2 Dec 2020 18:45:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7541D221FA Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=B+XgLZu6pWo6xt0Fm+H018RS3SWfsHIOwZX0/iUZT5w=; b=cQAfSram9z6beJw9WGzR2HszC dgfP5BQULoGEeR9ur4hBgUCXJRsbVYhNesQdZLrappraG/6kDBZlobqgNKV9o70YW2fAhD6P/immN mK3v38ZjdC1eBm2BujN/UjCQTtl4jwq2a6WLnN0pAc4OLWjhoaaJElL/7G12VisQvhzqw5K9+6e5e mgU7VP5QGe+3mOCDGtob/RcUw84Rm99UFTH3yfzfZvtx1wzsYwtSSODDAlXsQlJRgaQE0lJkTKJXu Gr4egLJuWGkgGQ5v6AwWkOgRTJrjtguYvbGJYPIa4DKpA0KdzzKxh9WUu8tAewnhWv3p1XOU/YzZd Lb2jFdZwQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX5v-0000VV-73; Wed, 02 Dec 2020 18:43:15 +0000 Received: from mail-wr1-x442.google.com ([2a00:1450:4864:20::442]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX4f-0008Ia-Ln for linux-arm-kernel@lists.infradead.org; Wed, 02 Dec 2020 18:41:59 +0000 Received: by mail-wr1-x442.google.com with SMTP id 23so5121105wrc.8 for ; Wed, 02 Dec 2020 10:41:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=acdB9myIWDy4DJFolYzOe4SZxoOSmlBPTOQxNwh7fkY=; b=FSAcmlxFDzuSUOeF5TGkNCgmVQmWPU2u0IKkkvgG9Ot1WbCofrNuJYCBmgE3OAlNr+ m9Ifb05xO6DRGHvirgILi6saMn/UeeokDDYPbeJbT2J5yM5PY1lXarCs5wZXyPnBKWG6 Qb8ilNqTHFiZXKqSaGDxVKb5sblwFv6YMXqYCJaVzMPqJi4n6knBKqcOu+bSLQy+ieAT xFvIh6NMyrZ5lQMaz5O69R1f60gvYLNVcYiRszXQ9cdUbJ9oIN5m6JKUy0nWYZ9L4381 3vsc4Yt0IHyOp8an6PNdtYWs78pR1CUJmWEcVQRyw/zVl5H1KkfqmPs8qdGhZvTEWMxh qqXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=acdB9myIWDy4DJFolYzOe4SZxoOSmlBPTOQxNwh7fkY=; b=STFS+NyTxikLRFZyCpgWIElMD6kMFOvcfC8u0MQVjZIPb10axGWJN4/OdlbTrqsYeM QriFi4Pik1HqGJC1mgKCHGAYLXg+lkFPg1dbZ9QhzyEsc4dgqeNeImMXSW1/q3FPmqG+ l4JbvmFfW+AAMJrGjv9MuWDjZrSSDdSnquw5f63hY/Jx7D1YGU1ufZB2lhCRJL6Aa55f WSbUJ3Zz0W/rC7ujCUpjJ1ijuSCgCYQaoTwV5lawhBn4AL1wQca813AgYaLQmgm9DX3E IorT2ErAWlKSQC1Czhc7mI4w+f7erMJIsKXO7iRPLZfWv0BtiSvE/a6LBEAgrOErc2LB xOyQ== X-Gm-Message-State: AOAM531U48WkT7SfbuSeNEqR6VZb5KSBkOthYaGWW05R+SjQpOQKQbtY AZDoHeLWMFo6Mfjw5E7KgX/P7g== X-Google-Smtp-Source: ABdhPJx4KfbBae0aYXZ1tphL0nYMHRF+2etuycKW2nu3ecT8P3h0qcnB4uVGsCCsVoiCM+cwIZCQyA== X-Received: by 2002:a5d:4892:: with SMTP id g18mr4992479wrq.365.1606934516331; Wed, 02 Dec 2020 10:41:56 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:5ddd:b7c5:e3c9:e87a]) by smtp.gmail.com with ESMTPSA id n9sm2918860wmd.4.2020.12.02.10.41.54 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 02 Dec 2020 10:41:55 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v4 13/26] kvm: arm64: Support per_cpu_ptr in nVHE hyp code Date: Wed, 2 Dec 2020 18:41:09 +0000 Message-Id: <20201202184122.26046-14-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201202184122.26046-1-dbrazdil@google.com> References: <20201202184122.26046-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201202_134157_798755_2E369C83 X-CRM114-Status: GOOD ( 20.62 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When compiling with __KVM_NVHE_HYPERVISOR__, redefine per_cpu_offset() to __hyp_per_cpu_offset() which looks up the base of the nVHE per-CPU region of the given cpu and computes its offset from the .hyp.data..percpu section. This enables use of per_cpu_ptr() helpers in nVHE hyp code. Until now only this_cpu_ptr() was supported by setting TPIDR_EL2. Signed-off-by: David Brazdil --- arch/arm64/include/asm/percpu.h | 6 ++++++ arch/arm64/kernel/image-vars.h | 3 +++ arch/arm64/kvm/hyp/nvhe/Makefile | 3 ++- arch/arm64/kvm/hyp/nvhe/hyp-smp.c | 24 ++++++++++++++++++++++++ 4 files changed, 35 insertions(+), 1 deletion(-) create mode 100644 arch/arm64/kvm/hyp/nvhe/hyp-smp.c diff --git a/arch/arm64/include/asm/percpu.h b/arch/arm64/include/asm/percpu.h index 1599e17379d8..8f1661603b78 100644 --- a/arch/arm64/include/asm/percpu.h +++ b/arch/arm64/include/asm/percpu.h @@ -239,6 +239,12 @@ PERCPU_RET_OP(add, add, ldadd) #define this_cpu_cmpxchg_8(pcp, o, n) \ _pcp_protect_return(cmpxchg_relaxed, pcp, o, n) +#ifdef __KVM_NVHE_HYPERVISOR__ +extern unsigned long __hyp_per_cpu_offset(unsigned int cpu); +#define __per_cpu_offset +#define per_cpu_offset(cpu) __hyp_per_cpu_offset((cpu)) +#endif + #include /* Redefine macros for nVHE hyp under DEBUG_PREEMPT to avoid its dependencies. */ diff --git a/arch/arm64/kernel/image-vars.h b/arch/arm64/kernel/image-vars.h index 08e69faedf6c..39289d75118d 100644 --- a/arch/arm64/kernel/image-vars.h +++ b/arch/arm64/kernel/image-vars.h @@ -99,6 +99,9 @@ KVM_NVHE_ALIAS(gic_nonsecure_priorities); KVM_NVHE_ALIAS(__start___kvm_ex_table); KVM_NVHE_ALIAS(__stop___kvm_ex_table); +/* Array containing bases of nVHE per-CPU memory regions. */ +KVM_NVHE_ALIAS(kvm_arm_hyp_percpu_base); + #endif /* CONFIG_KVM */ #endif /* __ARM64_KERNEL_IMAGE_VARS_H */ diff --git a/arch/arm64/kvm/hyp/nvhe/Makefile b/arch/arm64/kvm/hyp/nvhe/Makefile index 77b8c4e06f2f..cf11f8182756 100644 --- a/arch/arm64/kvm/hyp/nvhe/Makefile +++ b/arch/arm64/kvm/hyp/nvhe/Makefile @@ -6,7 +6,8 @@ asflags-y := -D__KVM_NVHE_HYPERVISOR__ ccflags-y := -D__KVM_NVHE_HYPERVISOR__ -obj-y := timer-sr.o sysreg-sr.o debug-sr.o switch.o tlb.o hyp-init.o host.o hyp-main.o +obj-y := timer-sr.o sysreg-sr.o debug-sr.o switch.o tlb.o hyp-init.o host.o \ + hyp-main.o hyp-smp.o obj-y += ../vgic-v3-sr.o ../aarch32.o ../vgic-v2-cpuif-proxy.o ../entry.o \ ../fpsimd.o ../hyp-entry.o ../exception.o diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-smp.c b/arch/arm64/kvm/hyp/nvhe/hyp-smp.c new file mode 100644 index 000000000000..7b0363b4857f --- /dev/null +++ b/arch/arm64/kvm/hyp/nvhe/hyp-smp.c @@ -0,0 +1,24 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2020 - Google LLC + * Author: David Brazdil + */ + +#include +#include +#include + +unsigned long __hyp_per_cpu_offset(unsigned int cpu) +{ + unsigned long *cpu_base_array; + unsigned long this_cpu_base; + unsigned long elf_base; + + if (cpu >= ARRAY_SIZE(kvm_arm_hyp_percpu_base)) + hyp_panic(); + + cpu_base_array = (unsigned long *)hyp_symbol_addr(kvm_arm_hyp_percpu_base); + this_cpu_base = kern_hyp_va(cpu_base_array[cpu]); + elf_base = (unsigned long)hyp_symbol_addr(__per_cpu_start); + return this_cpu_base - elf_base; +} From patchwork Wed Dec 2 18:41:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11946767 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9D11FC64E7C for ; Wed, 2 Dec 2020 18:45:30 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 227AD206D5 for ; Wed, 2 Dec 2020 18:45:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 227AD206D5 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=zhwY6OfbOGoLsW01NTREmeusBtjnlqLJsFvb8nnZB80=; b=S6bgFflCC91UwJWTiyKPCjpTb bzg80/HzqiJz9G/bI89iVymudicPkK6fzq1YhkseZOMkcwIw5yWDge7A8i6k/idVrAARb1Q8NQIf7 HP/2L0kZGwAMrpasE/OY1QSrpp0Qcjd1LeQVE6QIp1+PuQF22Np7iZ/AC9IspnFwM51JPc+P5a6k9 KL0wT+SRobSbJWsqESfxGKyEYBStl1tDAprC9lwww8SXaCAg8zUHh75cL+/9CL6ADIAI2oauViBxV LzA7L0gnaNH9BXoK5Kaw5NiNSCX0HgcH+Fg0tJ+XZS1eDTUUIhraZX3wHBsrqJGkCcSjU4ifdNUfM Lza/WHcCQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX64-0000bw-3W; Wed, 02 Dec 2020 18:43:24 +0000 Received: from mail-wm1-x343.google.com ([2a00:1450:4864:20::343]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX4h-0008Jx-S1 for linux-arm-kernel@lists.infradead.org; Wed, 02 Dec 2020 18:42:03 +0000 Received: by mail-wm1-x343.google.com with SMTP id c198so10180590wmd.0 for ; Wed, 02 Dec 2020 10:41:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=X0j+RCoYcy0kA4Cq9TKjA5cl6J4xllDB/E52Sl/VxFI=; b=QcreM1BcCNABhAl7u4JGwSWjJ6gL2zFtNGBBBem5Uz5LSqS20kZNu05BsIN3mdk4kM WmwSgASKeZZf1fSScVelU10Ou/l83oRqNvrWHtKxX57HtR13qLL5HHbTF514eIWpCpMw P0Gy3N3uhn3dqAOv+u3mta3p6YjahXlzn4T7tVKOibfRXgAJf13M/c2ftXFoqM3j22us ffeUS8vBSPDzcOIbAw+5y73xgPZdPgPtA07YEQoX6QjVhtaChsCatVOzVs9MuAzzNIpb ta4/8BaelML3og/7agjZt0z5OZIowiI5vG0V76o+ilQp/3DwYOPyCZFg562dd8fO3tG3 ssSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=X0j+RCoYcy0kA4Cq9TKjA5cl6J4xllDB/E52Sl/VxFI=; b=TrMckytIGsL6kt9T7akqiMzDgZheGlepZIOQQHxDenNQQkVuDgCw8WZnW1iOUoOUMr vAArgrYEhU267Lthd2V9worHOhgE3gcd7ZODuwxc6oaVBNmlGrgpsfGhgbtvZNYSHV48 2f4eZtTwFlHO3dSouZnkVidkQmQ07udhbLaMiaEyYCHwCmF8q65fCyjv8aSAXQEBqvEI Kvv82mgKmR9nzQ1u5nj4nTXbXV8e69OTBRvbgxkhwF+3nZ/rDTACE9FJvYBm2Uai5pJ4 sDNyTk+nkjbGKxT8FEUzpQqbUVde/2bDX30+qIvibl4j7NtDumAQ09j/UCLSj7Z4BfB9 tqHA== X-Gm-Message-State: AOAM533dWmWy87/e+1KibE76l2qhXIyb4iVf6uFs95bD36KivekT4B/y yHggh6USfn13Kb1/krxeYILAAA== X-Google-Smtp-Source: ABdhPJyPFPIxlkDq4z9qel4Mj/fLalo17OXktP8fnmZlxbaiN1V56nYRoNY5X1VLxACl0aApz+oOfw== X-Received: by 2002:a7b:c7c7:: with SMTP id z7mr4615447wmk.48.1606934518563; Wed, 02 Dec 2020 10:41:58 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:5ddd:b7c5:e3c9:e87a]) by smtp.gmail.com with ESMTPSA id 35sm3086773wro.71.2020.12.02.10.41.57 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 02 Dec 2020 10:41:57 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v4 14/26] kvm: arm64: Create nVHE copy of cpu_logical_map Date: Wed, 2 Dec 2020 18:41:10 +0000 Message-Id: <20201202184122.26046-15-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201202184122.26046-1-dbrazdil@google.com> References: <20201202184122.26046-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201202_134200_827078_F1AF794B X-CRM114-Status: GOOD ( 20.15 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When KVM starts validating host's PSCI requests, it will need to map MPIDR back to the CPU ID. To this end, copy cpu_logical_map into nVHE hyp memory when KVM is initialized. Only copy the information for CPUs that are online at the point of KVM initialization so that KVM rejects CPUs whose features were not checked against the finalized capabilities. Signed-off-by: David Brazdil --- arch/arm64/kvm/arm.c | 19 +++++++++++++++++++ arch/arm64/kvm/hyp/nvhe/hyp-smp.c | 16 ++++++++++++++++ 2 files changed, 35 insertions(+) diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 40857cbed3d1..09bb4098502b 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -63,6 +63,8 @@ static bool vgic_present; static DEFINE_PER_CPU(unsigned char, kvm_arm_hardware_enabled); DEFINE_STATIC_KEY_FALSE(userspace_irqchip_in_use); +extern u64 kvm_nvhe_sym(__cpu_logical_map)[NR_CPUS]; + int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu) { return kvm_vcpu_exiting_guest_mode(vcpu) == IN_GUEST_MODE; @@ -1584,6 +1586,20 @@ static inline void hyp_cpu_pm_exit(void) } #endif +static void init_cpu_logical_map(void) +{ + unsigned int cpu; + + /* + * Copy the MPIDR <-> logical CPU ID mapping to hyp. + * Only copy the set of online CPUs whose features have been chacked + * against the finalized system capabilities. The hypervisor will not + * allow any other CPUs from the `possible` set to boot. + */ + for_each_online_cpu(cpu) + kvm_nvhe_sym(__cpu_logical_map)[cpu] = cpu_logical_map(cpu); +} + static int init_common_resources(void) { return kvm_set_ipa_limit(); @@ -1756,6 +1772,9 @@ static int init_hyp_mode(void) } } + if (is_protected_kvm_enabled()) + init_cpu_logical_map(); + return 0; out_err: diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-smp.c b/arch/arm64/kvm/hyp/nvhe/hyp-smp.c index 7b0363b4857f..cbab0c6246e2 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-smp.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-smp.c @@ -8,6 +8,22 @@ #include #include +/* + * nVHE copy of data structures tracking available CPU cores. + * Only entries for CPUs that were online at KVM init are populated. + * Other CPUs should not be allowed to boot because their features were + * not checked against the finalized system capabilities. + */ +u64 __ro_after_init __cpu_logical_map[NR_CPUS] = { [0 ... NR_CPUS-1] = INVALID_HWID }; + +u64 cpu_logical_map(unsigned int cpu) +{ + if (cpu >= ARRAY_SIZE(__cpu_logical_map)) + hyp_panic(); + + return __cpu_logical_map[cpu]; +} + unsigned long __hyp_per_cpu_offset(unsigned int cpu) { unsigned long *cpu_base_array; From patchwork Wed Dec 2 18:41:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11946769 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2DD1AC71155 for ; Wed, 2 Dec 2020 18:45:56 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8AF61206D5 for ; Wed, 2 Dec 2020 18:45:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8AF61206D5 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=UHyV3wzVPL50gD6yyyfQEks2abknFNSRcVTY3bw0T40=; b=XKJapBk51UqzpkLhV2FHYUUb7 8Qv/Xfm1gpaWPlpzMSvVjfQ9CTUcAk+l01ZHbhvju66VVdNsxUvepq3p5veY93n2DrF/BxEgoo2XR wZRizx4S4F0NOVPDwUX4UtTmHAvNylLMsP9Y+jHswtjhhJoR/wUAuiALZZexZQApEPKTY28GIqVFY oUXshrAL1y3C4cElPtC3izIyxQI4DbDlgOymOV8T9+umhg2sG8vwySGFn8JLb2z6TejpGaNgLvomq POmiv0kBS8w2LI80t7V4JIGPV07TmttJr1HPzZ3n+MvyE1gjK5BaSUYSFaRdd3RS1RA27xKnycj59 iZaYdDt7Q==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX6S-0000pO-3k; Wed, 02 Dec 2020 18:43:48 +0000 Received: from mail-wr1-x444.google.com ([2a00:1450:4864:20::444]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX4k-0008Kh-7u for linux-arm-kernel@lists.infradead.org; Wed, 02 Dec 2020 18:42:07 +0000 Received: by mail-wr1-x444.google.com with SMTP id u12so5194576wrt.0 for ; Wed, 02 Dec 2020 10:42:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=6X5Tu43pgKNfdIYIj06Fr+JlpCFzZj/ZURHeuSR11Bw=; b=IAmw5/Y6w/en7YUiVJqEzUbksV4/sxbgv4bnK+MzI6iH7OV6SdZNtRr7IEYZL2boJs 11rJ0wEO34CQVAzo02Vc1m1rTPe8chZU5zhwCjwmrbgxOieXaTiWJatxME0NjTCK+tIW KQP07nH5PKznctiftFYr7s5fGiPHzmRDG9F78EGHbm5d6XAaffuWi4ZxiF9Jv27/TAya /SyK9qECXiwUWdFlzgBP/LVZXOr9gOu2UiODgi1YZl6d2EmVVL8ajBezwoxJ3+f5QxoN 9xF5YFOYDA9SdlVcBWYS1x/NrGphTirebdIi98mz97lZLSx9zo/ExmCsthjnEdHnvp4T DmPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=6X5Tu43pgKNfdIYIj06Fr+JlpCFzZj/ZURHeuSR11Bw=; b=HQnAev9slow+SpPY2ZiCddeWN5eonHl7vu0jrFTptd7bDTqwimZbeZZk/iJ/f4xt7a /mhBJrmpj/gLPcYxndZheWfwUmZX61wbOaHRvC8isaXYBS6zkYSWSNDahEcm5IGXRIUP OlOCjKXohlr7xNGLzh2ga2kHHWnTOTrQks+sT99PsIxYChMRxcza00+/wiIT2m4qPz46 npuiAsCn08aGE0bmfvvvLRHJH5/iiivrBQAKkrhphF9TSOjNcmsuwzbvPfGp8lIMICy5 m2NyKt+NgGPQzz3N239XvDRWnJkv93lHxEqn/NbEvFM//fuP2phMgVxf+CP8y8WIj7L+ a3tA== X-Gm-Message-State: AOAM531ptXqtj/2YhKTEqsv7UjXCN8lAwsNJIHoT+8Y7V5yxwCdt4bTF Nt3L/fof51JCk+Af2ExkEf+X8A== X-Google-Smtp-Source: ABdhPJzPFxnIikpIDzOJVUvZ6LglTEeEOYbM++Awb1eYvwM5zMt7kfYo0dvt6rG0gZ8koyG/p8VLcQ== X-Received: by 2002:a5d:4087:: with SMTP id o7mr4938755wrp.27.1606934520868; Wed, 02 Dec 2020 10:42:00 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:5ddd:b7c5:e3c9:e87a]) by smtp.gmail.com with ESMTPSA id l16sm3085275wrx.5.2020.12.02.10.41.59 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 02 Dec 2020 10:41:59 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v4 15/26] kvm: arm64: Add SMC handler in nVHE EL2 Date: Wed, 2 Dec 2020 18:41:11 +0000 Message-Id: <20201202184122.26046-16-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201202184122.26046-1-dbrazdil@google.com> References: <20201202184122.26046-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201202_134202_401600_02CF4247 X-CRM114-Status: GOOD ( 18.29 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add handler of host SMCs in KVM nVHE trap handler. Forward all SMCs to EL3 and propagate the result back to EL1. This is done in preparation for validating host SMCs in KVM protected mode. The implementation assumes that firmware uses SMCCC v1.2 or older. That means x0-x17 can be used both for arguments and results, other GPRs are preserved. Signed-off-by: David Brazdil --- arch/arm64/kvm/hyp/nvhe/host.S | 38 ++++++++++++++++++++++++++++++ arch/arm64/kvm/hyp/nvhe/hyp-main.c | 35 ++++++++++++++++++++++++--- 2 files changed, 70 insertions(+), 3 deletions(-) diff --git a/arch/arm64/kvm/hyp/nvhe/host.S b/arch/arm64/kvm/hyp/nvhe/host.S index fe2740b224cf..2b56f0bdf874 100644 --- a/arch/arm64/kvm/hyp/nvhe/host.S +++ b/arch/arm64/kvm/hyp/nvhe/host.S @@ -180,3 +180,41 @@ SYM_CODE_START(__kvm_hyp_host_vector) invalid_host_el1_vect // FIQ 32-bit EL1 invalid_host_el1_vect // Error 32-bit EL1 SYM_CODE_END(__kvm_hyp_host_vector) + +/* + * Forward SMC with arguments in struct kvm_cpu_context, and + * store the result into the same struct. Assumes SMCCC 1.2 or older. + * + * x0: struct kvm_cpu_context* + */ +SYM_CODE_START(__kvm_hyp_host_forward_smc) + /* + * Use x18 to keep the pointer to the host context because + * x18 is callee-saved in SMCCC but not in AAPCS64. + */ + mov x18, x0 + + ldp x0, x1, [x18, #CPU_XREG_OFFSET(0)] + ldp x2, x3, [x18, #CPU_XREG_OFFSET(2)] + ldp x4, x5, [x18, #CPU_XREG_OFFSET(4)] + ldp x6, x7, [x18, #CPU_XREG_OFFSET(6)] + ldp x8, x9, [x18, #CPU_XREG_OFFSET(8)] + ldp x10, x11, [x18, #CPU_XREG_OFFSET(10)] + ldp x12, x13, [x18, #CPU_XREG_OFFSET(12)] + ldp x14, x15, [x18, #CPU_XREG_OFFSET(14)] + ldp x16, x17, [x18, #CPU_XREG_OFFSET(16)] + + smc #0 + + stp x0, x1, [x18, #CPU_XREG_OFFSET(0)] + stp x2, x3, [x18, #CPU_XREG_OFFSET(2)] + stp x4, x5, [x18, #CPU_XREG_OFFSET(4)] + stp x6, x7, [x18, #CPU_XREG_OFFSET(6)] + stp x8, x9, [x18, #CPU_XREG_OFFSET(8)] + stp x10, x11, [x18, #CPU_XREG_OFFSET(10)] + stp x12, x13, [x18, #CPU_XREG_OFFSET(12)] + stp x14, x15, [x18, #CPU_XREG_OFFSET(14)] + stp x16, x17, [x18, #CPU_XREG_OFFSET(16)] + + ret +SYM_CODE_END(__kvm_hyp_host_forward_smc) diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c index a4f1cac714d7..f25680ede080 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -18,6 +18,8 @@ DEFINE_PER_CPU(struct kvm_nvhe_init_params, kvm_init_params); +void __kvm_hyp_host_forward_smc(struct kvm_cpu_context *host_ctxt); + static void handle___kvm_vcpu_run(struct kvm_cpu_context *host_ctxt) { DECLARE_REG(struct kvm_vcpu *, vcpu, host_ctxt, 1); @@ -152,12 +154,39 @@ static void handle_host_hcall(struct kvm_cpu_context *host_ctxt) cpu_reg(host_ctxt, 0) = SMCCC_RET_NOT_SUPPORTED; } +static void default_host_smc_handler(struct kvm_cpu_context *host_ctxt) +{ + __kvm_hyp_host_forward_smc(host_ctxt); +} + +static void skip_host_instruction(void) +{ + write_sysreg_el2(read_sysreg_el2(SYS_ELR) + 4, SYS_ELR); +} + +static void handle_host_smc(struct kvm_cpu_context *host_ctxt) +{ + default_host_smc_handler(host_ctxt); + + /* + * Unlike HVC, the return address of an SMC is the instruction's PC. + * Move the return address past the instruction. + */ + skip_host_instruction(); +} + void handle_trap(struct kvm_cpu_context *host_ctxt) { u64 esr = read_sysreg_el2(SYS_ESR); - if (unlikely(ESR_ELx_EC(esr) != ESR_ELx_EC_HVC64)) + switch (ESR_ELx_EC(esr)) { + case ESR_ELx_EC_HVC64: + handle_host_hcall(host_ctxt); + break; + case ESR_ELx_EC_SMC64: + handle_host_smc(host_ctxt); + break; + default: hyp_panic(); - - handle_host_hcall(host_ctxt); + } } From patchwork Wed Dec 2 18:41:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11946805 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4E54FC64E7C for ; Wed, 2 Dec 2020 18:46:31 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C8460206D5 for ; Wed, 2 Dec 2020 18:46:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C8460206D5 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=zEy4ynFvhTKAfcREff1z3Gq61cCQsYNuK77EhxMWjWI=; b=VaL/R56SDNIVD3nu9v5zmHzTm AQGl5GPpYXBL1vj138uzVY/86bweOuml8xwpjgwXJRYQ6ysQm86dHJUtiV3gTxbhYCcFy7Z74lvtX 4Z3PcoC18j2xk8YYrm9nZt83rBdZBLrM+E6bwdkzPBcjGk4yC7AyZVNZg2GT4uGPU6z7aQijoi26r ibFyvrLqd1wNmaoUChNcmBtsMUvpf1yE6xChvsEAefSIUP/C9fGtjchamG+0vKyXZbywTO7FY5PmZ eQsmSgZgdms5NLtIWrEQwd6dEO7PQaLyx0FV7HM5ppnv9sMlbHr+NZ6viPXmm3CtlEh49CTti6zNg Jgr9Nrg4g==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX7I-0001TG-2t; Wed, 02 Dec 2020 18:44:40 +0000 Received: from mail-wm1-x32c.google.com ([2a00:1450:4864:20::32c]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX4n-0008LQ-Ja for linux-arm-kernel@lists.infradead.org; Wed, 02 Dec 2020 18:42:11 +0000 Received: by mail-wm1-x32c.google.com with SMTP id 3so11027882wmg.4 for ; Wed, 02 Dec 2020 10:42:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=6AhUStd7TBYIRUbzKUC2M2YnuRgkt1Z1iWT0O/bo9W0=; b=SCYRPSh79TxoJZNYfwyvOM/BXZdHbkWon/aDDEt3ZW9Q4oq9R/Dm4W4KJlfMOHwrln BiRJLP/i1pQTRQQBGub1VWGlijOCHjhwJ1cUH73e3gxiu3tq5ao5sacWS9lAhhat+lWJ Fkziz6mvwdIha5xDHzagmZVUFnmNr/joylWFtwKOeepPe5Dl4XO/WnqCfW5SXoLvxi0C CHPbG+V/tuh6ggq8fcWu6tT7w3r06y2gx5rW7+0npJVz6HKSWxrc3h3eWkiARPTZv1hV KB1I298E2fte3MlmTytl5cfccjoDq+WdLWpHPOVBeuG1srhhVy8g+0Y3DUDpEWMdOqTU u0YA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=6AhUStd7TBYIRUbzKUC2M2YnuRgkt1Z1iWT0O/bo9W0=; b=HhsevnnyqyottGXoYvc3nONWzwgqGQo0+XuZcQPmcTJR/D89xSDd/ZK8SL2w3Ksn7U GbryE8MX+lgI9yQc2tgd3ftdYih6u73iODc7b5ajPm4IXJyg/5f9W74hbwEk8Bhj+T+f FgNOELajaoopmW85UcTkLlix9Rsem4xOJcipByf3LWYLbvCmw5t9YIO6G7Dp/bE5Wk0i 5O+41oCDv4aycvTaLSv8lPHWmJFQ/F0Z8ir6XSDTMyTZOd4Z5VEGmqJog1nilthG9u2z CNtSaZPPXrZ5z0nNTXLmp22z0STZDOyG96fyohv4Hc9RZtMkJakQ4Xuu8eec7RON9jWd hCLA== X-Gm-Message-State: AOAM531c6h0Z32xaSt9inMct9GHEUhW+yKkziGd54BYJUiwU0BWHc6mn we7wjy7akt7xYyCnoeDrPAk46w== X-Google-Smtp-Source: ABdhPJzrwN/bT6p1dI5tx96H71hjGnZL6cqAMwqfH+7C0aAKLMXtFaO+NLwQmNQoc4uI1EFKzHT1hw== X-Received: by 2002:a1c:454:: with SMTP id 81mr4498180wme.178.1606934523092; Wed, 02 Dec 2020 10:42:03 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:5ddd:b7c5:e3c9:e87a]) by smtp.gmail.com with ESMTPSA id w17sm3109805wru.82.2020.12.02.10.42.01 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 02 Dec 2020 10:42:02 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v4 16/26] kvm: arm64: Bootstrap PSCI SMC handler in nVHE EL2 Date: Wed, 2 Dec 2020 18:41:12 +0000 Message-Id: <20201202184122.26046-17-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201202184122.26046-1-dbrazdil@google.com> References: <20201202184122.26046-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201202_134205_817158_662F575C X-CRM114-Status: GOOD ( 28.23 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add a handler of PSCI SMCs in nVHE hyp code. The handler is initialized with the version used by the host's PSCI driver and the function IDs it was configured with. If the SMC function ID matches one of the configured PSCI calls (for v0.1) or falls into the PSCI function ID range (for v0.2+), the SMC is handled by the PSCI handler. For now, all SMCs return PSCI_RET_NOT_SUPPORTED. Signed-off-by: David Brazdil --- arch/arm64/include/asm/kvm_hyp.h | 2 + arch/arm64/kvm/arm.c | 25 ++++- .../arm64/kvm/hyp/include/nvhe/trap_handler.h | 18 ++++ arch/arm64/kvm/hyp/nvhe/Makefile | 2 +- arch/arm64/kvm/hyp/nvhe/hyp-main.c | 10 +- arch/arm64/kvm/hyp/nvhe/psci-relay.c | 100 ++++++++++++++++++ 6 files changed, 151 insertions(+), 6 deletions(-) create mode 100644 arch/arm64/kvm/hyp/include/nvhe/trap_handler.h create mode 100644 arch/arm64/kvm/hyp/nvhe/psci-relay.c diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h index cb25c15e3d8d..c0450828378b 100644 --- a/arch/arm64/include/asm/kvm_hyp.h +++ b/arch/arm64/include/asm/kvm_hyp.h @@ -93,6 +93,8 @@ void deactivate_traps_vhe_put(void); u64 __guest_enter(struct kvm_vcpu *vcpu); +bool kvm_host_psci_handler(struct kvm_cpu_context *host_ctxt); + void __noreturn hyp_panic(void); #ifdef __KVM_NVHE_HYPERVISOR__ void __noreturn __hyp_do_panic(bool restore_host, u64 spsr, u64 elr, u64 par); diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 09bb4098502b..fda7bfa440d5 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -19,6 +19,7 @@ #include #include #include +#include #include #define CREATE_TRACE_POINTS @@ -64,6 +65,8 @@ static DEFINE_PER_CPU(unsigned char, kvm_arm_hardware_enabled); DEFINE_STATIC_KEY_FALSE(userspace_irqchip_in_use); extern u64 kvm_nvhe_sym(__cpu_logical_map)[NR_CPUS]; +extern u32 kvm_nvhe_sym(kvm_host_psci_version); +extern struct psci_0_1_function_ids kvm_nvhe_sym(kvm_host_psci_0_1_function_ids); int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu) { @@ -1600,6 +1603,22 @@ static void init_cpu_logical_map(void) kvm_nvhe_sym(__cpu_logical_map)[cpu] = cpu_logical_map(cpu); } +static bool init_psci_relay(void) +{ + /* + * If PSCI has not been initialized, protected KVM cannot install + * itself on newly booted CPUs. + */ + if (!psci_ops.get_version) { + kvm_err("Cannot initialize protected mode without PSCI\n"); + return false; + } + + kvm_nvhe_sym(kvm_host_psci_version) = psci_ops.get_version(); + kvm_nvhe_sym(kvm_host_psci_0_1_function_ids) = get_psci_0_1_function_ids(); + return true; +} + static int init_common_resources(void) { return kvm_set_ipa_limit(); @@ -1772,9 +1791,13 @@ static int init_hyp_mode(void) } } - if (is_protected_kvm_enabled()) + if (is_protected_kvm_enabled()) { init_cpu_logical_map(); + if (!init_psci_relay()) + goto out_err; + } + return 0; out_err: diff --git a/arch/arm64/kvm/hyp/include/nvhe/trap_handler.h b/arch/arm64/kvm/hyp/include/nvhe/trap_handler.h new file mode 100644 index 000000000000..1e6d995968a1 --- /dev/null +++ b/arch/arm64/kvm/hyp/include/nvhe/trap_handler.h @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Trap handler helpers. + * + * Copyright (C) 2020 - Google LLC + * Author: Marc Zyngier + */ + +#ifndef __ARM64_KVM_NVHE_TRAP_HANDLER_H__ +#define __ARM64_KVM_NVHE_TRAP_HANDLER_H__ + +#include + +#define cpu_reg(ctxt, r) (ctxt)->regs.regs[r] +#define DECLARE_REG(type, name, ctxt, reg) \ + type name = (type)cpu_reg(ctxt, (reg)) + +#endif /* __ARM64_KVM_NVHE_TRAP_HANDLER_H__ */ diff --git a/arch/arm64/kvm/hyp/nvhe/Makefile b/arch/arm64/kvm/hyp/nvhe/Makefile index cf11f8182756..1f1e351c5fe2 100644 --- a/arch/arm64/kvm/hyp/nvhe/Makefile +++ b/arch/arm64/kvm/hyp/nvhe/Makefile @@ -7,7 +7,7 @@ asflags-y := -D__KVM_NVHE_HYPERVISOR__ ccflags-y := -D__KVM_NVHE_HYPERVISOR__ obj-y := timer-sr.o sysreg-sr.o debug-sr.o switch.o tlb.o hyp-init.o host.o \ - hyp-main.o hyp-smp.o + hyp-main.o hyp-smp.o psci-relay.o obj-y += ../vgic-v3-sr.o ../aarch32.o ../vgic-v2-cpuif-proxy.o ../entry.o \ ../fpsimd.o ../hyp-entry.o ../exception.o diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c index f25680ede080..bde658d51404 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -12,9 +12,7 @@ #include #include -#define cpu_reg(ctxt, r) (ctxt)->regs.regs[r] -#define DECLARE_REG(type, name, ctxt, reg) \ - type name = (type)cpu_reg(ctxt, (reg)) +#include DEFINE_PER_CPU(struct kvm_nvhe_init_params, kvm_init_params); @@ -166,7 +164,11 @@ static void skip_host_instruction(void) static void handle_host_smc(struct kvm_cpu_context *host_ctxt) { - default_host_smc_handler(host_ctxt); + bool handled; + + handled = kvm_host_psci_handler(host_ctxt); + if (!handled) + default_host_smc_handler(host_ctxt); /* * Unlike HVC, the return address of an SMC is the instruction's PC. diff --git a/arch/arm64/kvm/hyp/nvhe/psci-relay.c b/arch/arm64/kvm/hyp/nvhe/psci-relay.c new file mode 100644 index 000000000000..61375d4571c2 --- /dev/null +++ b/arch/arm64/kvm/hyp/nvhe/psci-relay.c @@ -0,0 +1,100 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2020 - Google LLC + * Author: David Brazdil + */ + +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +/* Config options set by the host. */ +__ro_after_init u32 kvm_host_psci_version; +__ro_after_init struct psci_0_1_function_ids kvm_host_psci_0_1_function_ids; + +static u64 get_psci_func_id(struct kvm_cpu_context *host_ctxt) +{ + DECLARE_REG(u64, func_id, host_ctxt, 0); + + return func_id; +} + +static bool is_psci_0_1_call(u64 func_id) +{ + return (func_id == kvm_host_psci_0_1_function_ids.cpu_suspend) || + (func_id == kvm_host_psci_0_1_function_ids.cpu_on) || + (func_id == kvm_host_psci_0_1_function_ids.cpu_off) || + (func_id == kvm_host_psci_0_1_function_ids.migrate); +} + +static bool is_psci_0_2_call(u64 func_id) +{ + /* SMCCC reserves IDs 0x00-1F with the given 32/64-bit base for PSCI. */ + return (PSCI_0_2_FN(0) <= func_id && func_id <= PSCI_0_2_FN(31)) || + (PSCI_0_2_FN64(0) <= func_id && func_id <= PSCI_0_2_FN64(31)); +} + +static bool is_psci_call(u64 func_id) +{ + switch (kvm_host_psci_version) { + case PSCI_VERSION(0, 1): + return is_psci_0_1_call(func_id); + default: + return is_psci_0_2_call(func_id); + } +} + +static unsigned long psci_0_1_handler(u64 func_id, struct kvm_cpu_context *host_ctxt) +{ + return PSCI_RET_NOT_SUPPORTED; +} + +static unsigned long psci_0_2_handler(u64 func_id, struct kvm_cpu_context *host_ctxt) +{ + switch (func_id) { + default: + return PSCI_RET_NOT_SUPPORTED; + } +} + +static unsigned long psci_1_0_handler(u64 func_id, struct kvm_cpu_context *host_ctxt) +{ + switch (func_id) { + default: + return psci_0_2_handler(func_id, host_ctxt); + } +} + +bool kvm_host_psci_handler(struct kvm_cpu_context *host_ctxt) +{ + u64 func_id = get_psci_func_id(host_ctxt); + unsigned long ret; + + if (!is_psci_call(func_id)) + return false; + + switch (kvm_host_psci_version) { + case PSCI_VERSION(0, 1): + ret = psci_0_1_handler(func_id, host_ctxt); + break; + case PSCI_VERSION(0, 2): + ret = psci_0_2_handler(func_id, host_ctxt); + break; + default: + ret = psci_1_0_handler(func_id, host_ctxt); + break; + } + + cpu_reg(host_ctxt, 0) = ret; + cpu_reg(host_ctxt, 1) = 0; + cpu_reg(host_ctxt, 2) = 0; + cpu_reg(host_ctxt, 3) = 0; + return true; +} From patchwork Wed Dec 2 18:41:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11946803 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 15BF2C71156 for ; Wed, 2 Dec 2020 18:46:29 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 769D3206D5 for ; Wed, 2 Dec 2020 18:46:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 769D3206D5 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=E/9M+wrU8paBZ1JYwcI9GQcga7Ug4AlzwLtZ3VEQMz8=; b=YtKA34O9TdORViNi/oOfABuC3 DZ2vuYqJB3rFgeRRERHzZoFY70Op9cNahh5s7ar9HVO8yrizmu3ToT6ji6babbfoOywHrjVxfNGch jR0JJYlfHplw3MSlHsp0wdbgt0qqVYIvvfjUOj+hHcwTh8SdlZt5Pwoj4T0Mtw2owP5yYDuLbXbNG Y9wBZ61kD4FJDKiEHXahAic9xlfxWtAHWQ0Y0feH24nqF3lhZuOiQIXxmb+Gt3CRDvaUFHs27H91K 3MuDo1g3rX7rh0cKg3XahvWyiT5B1vjnMFBKVDfDb8fYz4cGdyInQLgIEyVbjmPL7kgwmRbd97+C2 9TaqfU7bQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX6z-00014P-J5; Wed, 02 Dec 2020 18:44:21 +0000 Received: from mail-wr1-x441.google.com ([2a00:1450:4864:20::441]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX4o-0008MD-TS for linux-arm-kernel@lists.infradead.org; Wed, 02 Dec 2020 18:42:10 +0000 Received: by mail-wr1-x441.google.com with SMTP id p8so5163028wrx.5 for ; Wed, 02 Dec 2020 10:42:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=W2F3XgDMqEgOItGdCRM5+AsTSkKoSbLe12YGzB/3TvE=; b=fU9dMZnq0EEXms7M3KvnzCm8EKKrnGSW67qY+oR7f4a8sd6WG8ahqSKv+9LlTdMSv2 jPe+fM5u1XEChj2kBxLARa7kuONgbg6lySnEtqRvYhT3y/ZQNe8Vpk02CPBXQ0Ky2OfZ Nmy443RidccsbP8jBYjOpNgfHtiwMlp36a1VkRy/wv0RNzVhHX3Qz1dX4li5w/G2UpQn T8PisBjeSEdzFR5SyxZRQh03Qb1evvCEEEKzTsu2SwLWr46cK0sYnIZvGoNdrH0WGbVV N+ScKSKwGwqQKkTet61ZGVYg0yBgffmTTSfbaJT7iZfAAwXkPZmhHwPeClI4FYJ0oRJP KbuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=W2F3XgDMqEgOItGdCRM5+AsTSkKoSbLe12YGzB/3TvE=; b=ndGDfrnNKKHGxwQXLZVaRYeTe9N7fCrjzif+wCjW+Pj/1Z1ta3/fYgqsDilpLrSp5C by+GvZ0JWRSVReRucKKV9L9lZggGKNFaMzZ8NVOzPirTQBhLKnnYftlOiAYydtcVsr2t LVfE+bV+fSBDKu66VwZoYruebV3bfvzWLjtymskoYCAxTL1yFdyXzTjDgPFQ8DYEJrQ/ TheGJFUcs0VSD6W7cUj8Bi7JgVmFR7z97D6BIeBtjjjAHv4rErh7Yu7OmQmFDnhey/mA 1owX9VICeW/3WTzXvq3RTkeTle7r7siPaM9raTC7Aro5GlWmWKWIspcI2ZlcqQWgdaeD o+tQ== X-Gm-Message-State: AOAM533NzT50M6RTs7yfPwIbZ1Ol1w5jM5h3rblyGw7no1G24eB+QgS0 Yjl1LJZGbuyTghgIUsmAd0Vy5g== X-Google-Smtp-Source: ABdhPJxAfhDIldOi3vjK9bjU3teuSjjSL+9tOaDk90A1iO8lPe93m43qHEcAwCAxAn2GJH7EkOTR/Q== X-Received: by 2002:adf:b64b:: with SMTP id i11mr5063389wre.78.1606934525638; Wed, 02 Dec 2020 10:42:05 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:5ddd:b7c5:e3c9:e87a]) by smtp.gmail.com with ESMTPSA id d9sm2453926wrs.26.2020.12.02.10.42.03 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 02 Dec 2020 10:42:04 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v4 17/26] kvm: arm64: Add offset for hyp VA <-> PA conversion Date: Wed, 2 Dec 2020 18:41:13 +0000 Message-Id: <20201202184122.26046-18-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201202184122.26046-1-dbrazdil@google.com> References: <20201202184122.26046-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201202_134207_060147_692BA339 X-CRM114-Status: GOOD ( 17.09 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add a host-initialized constant to KVM nVHE hyp code for converting between EL2 linear map virtual addresses and physical addresses. Also add `__hyp_pa` macro that performs the conversion. Signed-off-by: David Brazdil --- arch/arm64/kvm/hyp/nvhe/psci-relay.c | 3 +++ arch/arm64/kvm/va_layout.c | 30 +++++++++++++++++++++++++--- 2 files changed, 30 insertions(+), 3 deletions(-) diff --git a/arch/arm64/kvm/hyp/nvhe/psci-relay.c b/arch/arm64/kvm/hyp/nvhe/psci-relay.c index 61375d4571c2..70b42f433449 100644 --- a/arch/arm64/kvm/hyp/nvhe/psci-relay.c +++ b/arch/arm64/kvm/hyp/nvhe/psci-relay.c @@ -18,6 +18,9 @@ /* Config options set by the host. */ __ro_after_init u32 kvm_host_psci_version; __ro_after_init struct psci_0_1_function_ids kvm_host_psci_0_1_function_ids; +__ro_after_init s64 hyp_physvirt_offset; + +#define __hyp_pa(x) ((phys_addr_t)((x)) + hyp_physvirt_offset) static u64 get_psci_func_id(struct kvm_cpu_context *host_ctxt) { diff --git a/arch/arm64/kvm/va_layout.c b/arch/arm64/kvm/va_layout.c index 4130b72e6891..d8cc51bd60bf 100644 --- a/arch/arm64/kvm/va_layout.c +++ b/arch/arm64/kvm/va_layout.c @@ -23,6 +23,30 @@ static u8 tag_lsb; static u64 tag_val; static u64 va_mask; +/* + * Compute HYP VA by using the same computation as kern_hyp_va(). + */ +static u64 __early_kern_hyp_va(u64 addr) +{ + addr &= va_mask; + addr |= tag_val << tag_lsb; + return addr; +} + +/* + * Store a hyp VA <-> PA offset into a hyp-owned variable. + */ +static void init_hyp_physvirt_offset(void) +{ + extern s64 kvm_nvhe_sym(hyp_physvirt_offset); + u64 kern_va, hyp_va; + + /* Compute the offset from the hyp VA and PA of a random symbol. */ + kern_va = (u64)kvm_ksym_ref(__hyp_text_start); + hyp_va = __early_kern_hyp_va(kern_va); + CHOOSE_NVHE_SYM(hyp_physvirt_offset) = (s64)__pa(kern_va) - (s64)hyp_va; +} + /* * We want to generate a hyp VA with the following format (with V == * vabits_actual): @@ -54,6 +78,8 @@ __init void kvm_compute_layout(void) tag_val |= get_random_long() & GENMASK_ULL(vabits_actual - 2, tag_lsb); } tag_val >>= tag_lsb; + + init_hyp_physvirt_offset(); } static u32 compute_instruction(int n, u32 rd, u32 rn) @@ -146,9 +172,7 @@ void kvm_patch_vector_branch(struct alt_instr *alt, /* * Compute HYP VA by using the same computation as kern_hyp_va() */ - addr = (uintptr_t)kvm_ksym_ref(__kvm_hyp_vector); - addr &= va_mask; - addr |= tag_val << tag_lsb; + addr = __early_kern_hyp_va((u64)kvm_ksym_ref(__kvm_hyp_vector)); /* Use PC[10:7] to branch to the same vector in KVM */ addr |= ((u64)origptr & GENMASK_ULL(10, 7)); From patchwork Wed Dec 2 18:41:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11946807 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BBDEFC64E7C for ; Wed, 2 Dec 2020 18:46:59 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3558020709 for ; Wed, 2 Dec 2020 18:46:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3558020709 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=/UMg6mmUlhid0pbOAzD3VJEjOG4fxY/0pPeWwLVraGo=; b=0V4XNTowikHlVGkrI1Z467C6Q RuALZXCPIq+BCxAl5buA2W9yRccmkP4biCaGaPZerFlvMYLYPuD1CW8KPonCk5KaKtijtz17rur1q bugqmm3bf8JW+Kbzs0dMjY9GoaTd6Y9+qsSkDZsbwFPRRdzDWBwYL6HiY3HnxNIZjKwRuEKPi1o3F yhPqR56XWL+ghjanGwbSyR+dwqDsL/4Sqtne+C9CJ65F0K18w9uI1iyWcmOn/CQZZZUpIBVVxgzok k5GWxZSmAwlLfDTGNBOkszL7VLcsn9P0//eIzzZ320oDG2F413eLobrAyJjMT8FEvpO7MUOSbMTfP xVpRj8cyA==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX7V-0001c9-EA; Wed, 02 Dec 2020 18:44:53 +0000 Received: from mail-wm1-x341.google.com ([2a00:1450:4864:20::341]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX4r-0008Na-ON for linux-arm-kernel@lists.infradead.org; Wed, 02 Dec 2020 18:42:14 +0000 Received: by mail-wm1-x341.google.com with SMTP id h21so11167695wmb.2 for ; Wed, 02 Dec 2020 10:42:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=qcfRU1YbBQ04ItAE6mdwMxjNOIhFl6zoGdtG5ZWWz6Y=; b=FHW7OMVvnhhhri9l5Ifx4Rrqhfsw/iXEfWL3Xv//E79ZdV/UcgaVz/W0r2HYbZvqeY y2QPeIachgJaHG+rwCcgMXEco0yfQx0/re+sBTjr6jN3QPx9IhOX1g2yZimUEhTvMoB/ YR/6959KrrBVEVUmE+JaqtY45Jz6vNkVJSyz3+1p5vk6xN23TYB9XpSQ6p2AiooWt88K LQ7b0Wm6T9rpwnxpjLpJSDo0mx7Ar0oSQOh22fxFpxsurRK+osX4w8raIkJoQ8iQkpKP 91ESox4eJ95E+WzqyGdvdpGOKHRFxTm9LL8IJqFyGSLj03VMBv6+sx3rJ2bd1jSyLpuW stLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=qcfRU1YbBQ04ItAE6mdwMxjNOIhFl6zoGdtG5ZWWz6Y=; b=KTh3FdWoCV36Ez764wpp1k9n7cYO8MwbOwxBm9EYaAmN27iGNhRuyS+GamyytXVxmP lfx9iKIlitwTxO9KYRzFVrIxhewmlmeV4LqByK29sAuFPdK/aVECKjCvJk/AV5phxA1V SH4eVFYcqDBHBvgneAtTZHTx32GjUduBW2viCToo+ngQck9ynw+5G/RpIzBUKyAgBjOb Dy+VhHSEWmGgo9aDOQNazzOFYs2TW/3WP1JoYaj4oZ7VpmoQx7wMminNm3ttQJ6zpKXK R36hw5kVDdPq8/fjVE1/b3qc/YnBUWTvNTumDPKmA0ZQ4AIRPoZlY/yNfjWr4HMfW4md 63zw== X-Gm-Message-State: AOAM531fOTUnQg+0OsuuGvr16qTY3EKb42WlXrsqNNuXR5k3pBEbjZBr BKz+Wv5oNn650a9ttEXj5VCuqg== X-Google-Smtp-Source: ABdhPJz/Y9ERJYjarEvm1qv0CECj56Dlnsv6FCiSdtWlW6kjM8WK1TRApQh5Lukq91apWPC5bp6dJg== X-Received: by 2002:a1c:bd87:: with SMTP id n129mr4533586wmf.32.1606934527867; Wed, 02 Dec 2020 10:42:07 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:5ddd:b7c5:e3c9:e87a]) by smtp.gmail.com with ESMTPSA id n128sm3160473wmb.46.2020.12.02.10.42.06 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 02 Dec 2020 10:42:06 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v4 18/26] kvm: arm64: Forward safe PSCI SMCs coming from host Date: Wed, 2 Dec 2020 18:41:14 +0000 Message-Id: <20201202184122.26046-19-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201202184122.26046-1-dbrazdil@google.com> References: <20201202184122.26046-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201202_134209_857540_F9FD909D X-CRM114-Status: GOOD ( 19.06 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Forward the following PSCI SMCs issued by host to EL3 as they do not require the hypervisor's intervention. This assumes that EL3 correctly implements the PSCI specification. Only function IDs implemented in Linux are included. Where both 32-bit and 64-bit variants exist, it is assumed that the host will always use the 64-bit variant. * SMCs that only return information about the system * PSCI_VERSION - PSCI version implemented by EL3 * PSCI_FEATURES - optional features supported by EL3 * AFFINITY_INFO - power state of core/cluster * MIGRATE_INFO_TYPE - whether Trusted OS can be migrated * MIGRATE_INFO_UP_CPU - resident core of Trusted OS * operations which do not affect the hypervisor * MIGRATE - migrate Trusted OS to a different core * SET_SUSPEND_MODE - toggle OS-initiated mode * system shutdown/reset * SYSTEM_OFF * SYSTEM_RESET * SYSTEM_RESET2 Signed-off-by: David Brazdil --- arch/arm64/kvm/hyp/nvhe/psci-relay.c | 42 +++++++++++++++++++++++++++- 1 file changed, 41 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kvm/hyp/nvhe/psci-relay.c b/arch/arm64/kvm/hyp/nvhe/psci-relay.c index 70b42f433449..5ad56a875ffa 100644 --- a/arch/arm64/kvm/hyp/nvhe/psci-relay.c +++ b/arch/arm64/kvm/hyp/nvhe/psci-relay.c @@ -54,14 +54,50 @@ static bool is_psci_call(u64 func_id) } } +static unsigned long psci_call(unsigned long fn, unsigned long arg0, + unsigned long arg1, unsigned long arg2) +{ + struct arm_smccc_res res; + + arm_smccc_1_1_smc(fn, arg0, arg1, arg2, &res); + return res.a0; +} + +static unsigned long psci_forward(struct kvm_cpu_context *host_ctxt) +{ + return psci_call(cpu_reg(host_ctxt, 0), cpu_reg(host_ctxt, 1), + cpu_reg(host_ctxt, 2), cpu_reg(host_ctxt, 3)); +} + +static __noreturn unsigned long psci_forward_noreturn(struct kvm_cpu_context *host_ctxt) +{ + psci_forward(host_ctxt); + hyp_panic(); /* unreachable */ +} + static unsigned long psci_0_1_handler(u64 func_id, struct kvm_cpu_context *host_ctxt) { - return PSCI_RET_NOT_SUPPORTED; + if ((func_id == kvm_host_psci_0_1_function_ids.cpu_off) || + (func_id == kvm_host_psci_0_1_function_ids.migrate)) + return psci_forward(host_ctxt); + else + return PSCI_RET_NOT_SUPPORTED; } static unsigned long psci_0_2_handler(u64 func_id, struct kvm_cpu_context *host_ctxt) { switch (func_id) { + case PSCI_0_2_FN_PSCI_VERSION: + case PSCI_0_2_FN_CPU_OFF: + case PSCI_0_2_FN64_AFFINITY_INFO: + case PSCI_0_2_FN64_MIGRATE: + case PSCI_0_2_FN_MIGRATE_INFO_TYPE: + case PSCI_0_2_FN64_MIGRATE_INFO_UP_CPU: + return psci_forward(host_ctxt); + case PSCI_0_2_FN_SYSTEM_OFF: + case PSCI_0_2_FN_SYSTEM_RESET: + psci_forward_noreturn(host_ctxt); + unreachable(); default: return PSCI_RET_NOT_SUPPORTED; } @@ -70,6 +106,10 @@ static unsigned long psci_0_2_handler(u64 func_id, struct kvm_cpu_context *host_ static unsigned long psci_1_0_handler(u64 func_id, struct kvm_cpu_context *host_ctxt) { switch (func_id) { + case PSCI_1_0_FN_PSCI_FEATURES: + case PSCI_1_0_FN_SET_SUSPEND_MODE: + case PSCI_1_1_FN64_SYSTEM_RESET2: + return psci_forward(host_ctxt); default: return psci_0_2_handler(func_id, host_ctxt); } From patchwork Wed Dec 2 18:41:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11946809 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5C078C64E7C for ; Wed, 2 Dec 2020 18:47:06 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D7C5E20709 for ; Wed, 2 Dec 2020 18:47:05 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D7C5E20709 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=FHdBhLOKJLnvV2tXmSqMX3sI0s+XnQm9XPQEOytc60M=; b=XD3IDlV6iE0YT0ZA/JYFtTJS2 6fCeEFR7vot3cbLk+WiZlLax+0RQEesnfav/xZttRxXxDieNfmmDUWYd2VHJKFb84ka//XIVSqn2c F61rZi+BB4YvrpNbdEJ0X/GCj5q1zrha/YzurxKBaDHb9UZ70qSDb/EoDV0ad6pBh8qXzobzlh+x3 Jo2bb6w9ZRToN6+91UD7NnsInyen87H0PLJtqmY/zsIr2sByrQS8H9I7UZJ0+YXNaiNtL3ncYJu2d 8lj95NYVAoIJqZjmGFl9pl46wzlVL9NRDT95nrLOZi7mbu7rC6oSE6fx1f2HVXRJSnGAyspFgsd4o OlKUyJIcw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX7j-0001kn-Sf; Wed, 02 Dec 2020 18:45:07 +0000 Received: from mail-wm1-x341.google.com ([2a00:1450:4864:20::341]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX4t-0008OZ-OD for linux-arm-kernel@lists.infradead.org; Wed, 02 Dec 2020 18:42:14 +0000 Received: by mail-wm1-x341.google.com with SMTP id d3so10153272wmb.4 for ; Wed, 02 Dec 2020 10:42:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=C6am4hqHGk+Vo0ud1IP29/X2lpqX4cUlMBiGKiun4f8=; b=hoQt5BzIWgY7qXX3fYCd3gye89dvXes5OcxZR1oaFHiJQ0uNuctf0EfXcXsXQqJZHi lyMsVJArQ0COsJ3V461k6nveoLNzCLiKZbAL5LUSG4DkZ0mZ4TD+7GR+bvNwn6Oztk+b junmL8dzKUi31Dphr3YG92zBLKleqoCJLPlvFxbGeHyYhHq5d6poYXr1fCeGHotQtBSR HE+mkv/OIjfKLvHevZZVRrqv3+kaP8yMimI6sQpItNZb7BDN+1luqcdtgtiTsgiqIFoE bB/s2tgV4B1tzXzOp8LoYqmDeZbINdCG9fd4QhlOCkBE+gk5X9QdHlI8qQkX3t6UKEwH 0a9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=C6am4hqHGk+Vo0ud1IP29/X2lpqX4cUlMBiGKiun4f8=; b=tBa3wnbnzX6zFS2QtRyFpr2iTZfGx92avlVNamggNrizDDglf/pkB9aHjSrXmT+HYY bdbua6ktMGUe1/n6V55KAE1DVFGjMu0khLS5kg/uzUfgd+jL5GMtVLoTmhJ0ug0hn/H4 mNZcy1QCYv9oAJMIh1gaqgAAyplHjBi5g1l9YMBh9LlSP0MWSisJstUyx+Zx2qJ9ZMTS yA8IoGPs72wWU9x5znv0MfUiti3O5qJrqMaNGCiH0xNvjRgvDf9OPnm2bF3uC1pviSPL GbEEUX1SHbEuh0KiQyJy//NMo4piQRMQZLMCL+Pqypy5HW3IHHPtnUAtnP8+4tGtC+EW Bufg== X-Gm-Message-State: AOAM532qLw5WyxMbN2CH6/uljvZuywtL65SxBG7H12IqjzX3vwi3b3sR 5Idd2rleRFC45NRv2gd9Ovc72A== X-Google-Smtp-Source: ABdhPJzQrJ/LWyt16hrvnyOSjGz1ckoWGV76pEesTOfeJNYR/dIM8PSyGIBinNhf3COj/w3pWBottw== X-Received: by 2002:a7b:ce17:: with SMTP id m23mr4554470wmc.117.1606934530064; Wed, 02 Dec 2020 10:42:10 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:5ddd:b7c5:e3c9:e87a]) by smtp.gmail.com with ESMTPSA id y20sm3136706wma.15.2020.12.02.10.42.08 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 02 Dec 2020 10:42:08 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v4 19/26] kvm: arm64: Extract __do_hyp_init into a helper function Date: Wed, 2 Dec 2020 18:41:15 +0000 Message-Id: <20201202184122.26046-20-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201202184122.26046-1-dbrazdil@google.com> References: <20201202184122.26046-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201202_134211_975915_BED33A9E X-CRM114-Status: GOOD ( 16.20 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In preparation for adding a CPU entry point in nVHE hyp code, extract most of __do_hyp_init hypervisor initialization code into a common helper function. This will be invoked by the entry point to install KVM on the newly booted CPU. Signed-off-by: David Brazdil --- arch/arm64/kvm/hyp/nvhe/hyp-init.S | 47 ++++++++++++++++++++---------- 1 file changed, 32 insertions(+), 15 deletions(-) diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-init.S b/arch/arm64/kvm/hyp/nvhe/hyp-init.S index 712f57289357..b0856b006bc0 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-init.S +++ b/arch/arm64/kvm/hyp/nvhe/hyp-init.S @@ -68,17 +68,36 @@ __do_hyp_init: mov x0, #SMCCC_RET_NOT_SUPPORTED eret -1: ldr x0, [x1, #NVHE_INIT_TPIDR_EL2] - msr tpidr_el2, x0 +1: mov x0, x1 + mov x4, lr + bl ___kvm_hyp_init + mov lr, x4 - ldr x0, [x1, #NVHE_INIT_MAIR_EL2] - msr mair_el2, x0 + /* Hello, World! */ + mov x0, #SMCCC_RET_SUCCESS + eret +SYM_CODE_END(__kvm_hyp_init) + +/* + * Initialize the hypervisor in EL2. + * + * Only uses x0..x3 so as to not clobber callee-saved SMCCC registers + * and leave x4 for the caller. + * + * x0: struct kvm_nvhe_init_params PA + */ +SYM_CODE_START_LOCAL(___kvm_hyp_init) + ldr x1, [x0, #NVHE_INIT_TPIDR_EL2] + msr tpidr_el2, x1 + + ldr x1, [x0, #NVHE_INIT_STACK_HYP_VA] + mov sp, x1 - ldr x0, [x1, #NVHE_INIT_STACK_HYP_VA] - mov sp, x0 + ldr x1, [x0, #NVHE_INIT_MAIR_EL2] + msr mair_el2, x1 - ldr x0, [x1, #NVHE_INIT_PGD_PA] - phys_to_ttbr x2, x0 + ldr x1, [x0, #NVHE_INIT_PGD_PA] + phys_to_ttbr x2, x1 alternative_if ARM64_HAS_CNP orr x2, x2, #TTBR_CNP_BIT alternative_else_nop_endif @@ -87,9 +106,9 @@ alternative_else_nop_endif /* * Set the PS bits in TCR_EL2. */ - ldr x0, [x1, #NVHE_INIT_TCR_EL2] - tcr_compute_pa_size x0, #TCR_EL2_PS_SHIFT, x1, x2 - msr tcr_el2, x0 + ldr x1, [x0, #NVHE_INIT_TCR_EL2] + tcr_compute_pa_size x1, #TCR_EL2_PS_SHIFT, x2, x3 + msr tcr_el2, x1 isb @@ -117,10 +136,8 @@ alternative_else_nop_endif kimg_hyp_va x0, x1 msr vbar_el2, x0 - /* Hello, World! */ - mov x0, #SMCCC_RET_SUCCESS - eret -SYM_CODE_END(__kvm_hyp_init) + ret +SYM_CODE_END(___kvm_hyp_init) SYM_CODE_START(__kvm_handle_stub_hvc) cmp x0, #HVC_SOFT_RESTART From patchwork Wed Dec 2 18:41:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11946811 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5F648C71155 for ; Wed, 2 Dec 2020 18:47:35 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D119721741 for ; Wed, 2 Dec 2020 18:47:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D119721741 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=Zxj8I5wY/RHZ9H+PfDqMSs2OXtw1VOksdniY2dfs2qY=; b=PSaA8pmuGXfcNNwwDWBqzki7B 5Gt4ZqhbP+84sA7fl0x861Sc9iBvR26R1W3y8dpi17rFtvwgyR7hkfBFZROy62PUjGgXE2NaM7XUG zDJ8vvA817cYdWrdhstJNNscwe9XJtafa1L3sFKGh9GIcTRdP75Udxo1Zs5v3+bE+9NhlqnZPhpcq z0zF0xBw8GQFOoaGYvDKgQI4JJE/6vPpb25euhD4OWQ2gtv7TCJkLUtp0ZQJUplBnE+OqsWpYUTJq M/uxrpkBtALfDwyy9D0zqKLIl5jgt1Zh0rLAYxfmlVdnR+b7syER7+tihrrAQbMGYUEMt2+6ZPIqB olJylEDhA==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX87-0001vt-N9; Wed, 02 Dec 2020 18:45:31 +0000 Received: from mail-wr1-x441.google.com ([2a00:1450:4864:20::441]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX4v-0008PT-Bz for linux-arm-kernel@lists.infradead.org; Wed, 02 Dec 2020 18:42:15 +0000 Received: by mail-wr1-x441.google.com with SMTP id o1so5163726wrx.7 for ; Wed, 02 Dec 2020 10:42:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=yTPqxuNIHyHcxIGVDRFuzMb//a26J9RNoM2ThcyKbSM=; b=aCv+2fOzVacnmojuOT7cqdovfGr22fjEzJv+MRvW19pxd/At/2+SeBvsnBovv23E0a 4ejG9x3wrILwl7LMVEeZNAy3Oz04NMiHm2Fj4hdnEr9LAkH0YCPP1X0lywRi3nVmOGD7 nkwvoIwyzCN0kTWt4W3zAUBl+ihGtvBa9iAj7T1vlVaWad8lbsjZhFTC3rh2nTTEeoVN /Lvdr5l6vK5yD3B1B66RYBHWXz9OiYZYIBnuFg7gVGUh0Sw1Q8cGh4Im1B7YS4gfugr6 oyO6bJiThSBLLb5BdbTjsa1DmmAbRvnlt4Ys1b6GgaoLrc+4THiITVFGUu94qhXAsFdT AhxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=yTPqxuNIHyHcxIGVDRFuzMb//a26J9RNoM2ThcyKbSM=; b=mSnCJC1kIMaoUaX/c6lnGCwi+L72IuiS7YFF18siy4zJLIO6mBuaNx8jqFov0uZZ7t c0b1vHsjLmhLRwCxUqla2qUnw3bO8/r0JrYaiF4j2cUDHC6DvZih3osJkVP1R4d/qWsr sGz97x+Ust3ITIiFzQEmVxHDNQ1bFvTN8ncNosUFJVMrC/8tPDj2tAfaU57D8uXtHOXx E54K9pr3p1+VbnMWTb+1HprgojJq2VFgGP9fqAodM37XlxskL17HpvC1d1FZHx/Qn1V4 KeoXjQbkPBtjEjLVWEcxtFby+ztxYDv+CEzU54BFDZsKYDDimkXhTzX6oYqIvgXqXcvE 8Piw== X-Gm-Message-State: AOAM533wWvacN7mCkPgYQ0ckde4c6DnrceI1DhdZw1bzC0fNjOxtx+ou ID5TfIa9o+JOYMXlnUVijW575A== X-Google-Smtp-Source: ABdhPJylrxzOOF0czEtZ7fv5Klvs9l0NUYOehr4kbC+Jl1sFVyJ7nzp854Drk3FK4NwPazSOzLk5iA== X-Received: by 2002:a5d:504f:: with SMTP id h15mr5035439wrt.402.1606934532259; Wed, 02 Dec 2020 10:42:12 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:5ddd:b7c5:e3c9:e87a]) by smtp.gmail.com with ESMTPSA id b200sm3008640wmb.10.2020.12.02.10.42.10 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 02 Dec 2020 10:42:11 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v4 20/26] kvm: arm64: Add function to enter host from KVM nVHE hyp code Date: Wed, 2 Dec 2020 18:41:16 +0000 Message-Id: <20201202184122.26046-21-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201202184122.26046-1-dbrazdil@google.com> References: <20201202184122.26046-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201202_134213_496603_2EC6BADA X-CRM114-Status: GOOD ( 14.35 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org All nVHE hyp code is currently executed as handlers of host's HVCs. This will change as nVHE starts intercepting host's PSCI CPU_ON SMCs. The newly booted CPU will need to initialize EL2 state and then enter the host. Add __host_enter function that branches into the existing host state-restoring code after the trap handler would have returned. Signed-off-by: David Brazdil --- arch/arm64/kvm/hyp/nvhe/host.S | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/arch/arm64/kvm/hyp/nvhe/host.S b/arch/arm64/kvm/hyp/nvhe/host.S index 2b56f0bdf874..a820dfdc9c25 100644 --- a/arch/arm64/kvm/hyp/nvhe/host.S +++ b/arch/arm64/kvm/hyp/nvhe/host.S @@ -39,6 +39,7 @@ SYM_FUNC_START(__host_exit) bl handle_trap /* Restore host regs x0-x17 */ +__host_enter_restore_full: ldp x0, x1, [x29, #CPU_XREG_OFFSET(0)] ldp x2, x3, [x29, #CPU_XREG_OFFSET(2)] ldp x4, x5, [x29, #CPU_XREG_OFFSET(4)] @@ -61,6 +62,14 @@ __host_enter_without_restoring: sb SYM_FUNC_END(__host_exit) +/* + * void __noreturn __host_enter(struct kvm_cpu_context *host_ctxt); + */ +SYM_FUNC_START(__host_enter) + mov x29, x0 + b __host_enter_restore_full +SYM_FUNC_END(__host_enter) + /* * void __noreturn __hyp_do_panic(bool restore_host, u64 spsr, u64 elr, u64 par); */ From patchwork Wed Dec 2 18:41:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11946815 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 44705C71155 for ; Wed, 2 Dec 2020 18:48:40 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A2EF822248 for ; Wed, 2 Dec 2020 18:48:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A2EF822248 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=LC68hNad5jghuXLjm2Uok/inz5Ha/IMtca1y5hz2KHQ=; b=Fh73DcLnGPMDZ0Q1UJ9hxbIe3 vDq5pJbgGRr7NaQxngoL6twhtE22JlO6cqkCyymfVRwTb6w8XWjSWCH5RrOGlewplq1M5qjuXOiz3 jNQu0CRQRgSTPD1rrrmrYua8rhXG8xF/FhpMwMx8ONwbuoYcVTxqk/cvI9zy03jM7j60W0whenNCc tYGzVHlbX+xymdtCy0Hvf0x0pZOplitXqU2l7W4TL0ywQJhRo2TE2ViJF9M7KRifqivxpegKpUotg TxUelrjRlX90NUO6NUaEQIWT8djIlxfhjgnaCRYFPRN67Okn69fTJc7/WJa9dSI/zxFGRJc4C4FpG 71hPMfPVQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX8s-0002Op-Nc; Wed, 02 Dec 2020 18:46:18 +0000 Received: from mail-wr1-x42a.google.com ([2a00:1450:4864:20::42a]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX50-0008Qi-Df for linux-arm-kernel@lists.infradead.org; Wed, 02 Dec 2020 18:42:21 +0000 Received: by mail-wr1-x42a.google.com with SMTP id e7so5160675wrv.6 for ; Wed, 02 Dec 2020 10:42:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=eCJnVbiCjoN/pSAdplV1JGIGBKZGMHYP2ef/XdrHF5Q=; b=hzfSLkwdYdGXdchogjDVmnYDCF/2gpJ4Tp+atdehO+5Cm9Euc6M/aX/Gn62x9Qu8b2 JDB9QgOEaQ8sMcL7F4czqGQT/dq9LhCM/FtMc0yw5kotMl/OLzGLaq932TlhEI9JTbPC YStQGqgocb7+YmWoCwo9cs3yhHWPk8l+K/84tU3GyZEaJfaXgbRUB9eIf1zMRzpWr6S0 Sy0Rn9gAzv4QyT+xbL91UA9bV820Ro4L7w+j/zZt4mV/hC+RxEpBhyvrS8b5V1U0yneM uk4uFvTnK2+VfKflT7WwhLim6kp0CB4y9WtuPeGF9hdob34hF93Tz83unvAzaxC/FtVU A03A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=eCJnVbiCjoN/pSAdplV1JGIGBKZGMHYP2ef/XdrHF5Q=; b=pEWaFZTA/6rvsaFUPLZufcZJvNp78ZSErwM5ZgaUkE7CbjguHOUMYxJNIWfNOuZTF6 HvvSuT+2s2G9l3pzTckKWNm5d7oXF1GLjc0muyEYRaFvNo2dNNmpn9K2MbJImXdum57k 6cGgpQPfRP+O3wAVnhwKrJutqh2M3Q12c6RaNk1aVfjmOXUdyOWp+041kVLc9kfuPPLr Gn4ckcp4qp3ZdloYBNg1UmSPLCrIB7+9FbTPcgffloZBeuNrk2zJ92L8Csdqqmmb7usJ EbgP+WW8cuXo5HNBuj0FJZz2B8WsYoXUR9qRjEFTt53ohd15mbYz1iTR0uU/wWO39Kem k2hg== X-Gm-Message-State: AOAM533rjDOAWyUFVSjaeut+k7jS3NlCClH7Ee3A0d4eeukQ3L6iii6L BfTTAUfZHa7/PkNNOy2a6K5gQg== X-Google-Smtp-Source: ABdhPJzO7xDGtOYCHE/w9ReqAHY2jQUZ9wCSW2QWoyljTFgZ0QsKnjjtpvvcmLrq7Ji0oD+HjGXa2w== X-Received: by 2002:adf:f5c8:: with SMTP id k8mr5258235wrp.2.1606934534601; Wed, 02 Dec 2020 10:42:14 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:5ddd:b7c5:e3c9:e87a]) by smtp.gmail.com with ESMTPSA id z189sm2958120wme.23.2020.12.02.10.42.13 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 02 Dec 2020 10:42:13 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v4 21/26] kvm: arm64: Intercept host's CPU_ON SMCs Date: Wed, 2 Dec 2020 18:41:17 +0000 Message-Id: <20201202184122.26046-22-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201202184122.26046-1-dbrazdil@google.com> References: <20201202184122.26046-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201202_134219_716830_AD4D5BBD X-CRM114-Status: GOOD ( 25.73 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add a handler of the CPU_ON PSCI call from host. When invoked, it looks up the logical CPU ID corresponding to the provided MPIDR and populates the state struct of the target CPU with the provided x0, pc. It then calls CPU_ON itself, with an entry point in hyp that initializes EL2 state before returning ERET to the provided PC in EL1. There is a simple atomic lock around the boot args struct. If it is already locked, CPU_ON will return PENDING_ON error code. Signed-off-by: David Brazdil --- arch/arm64/kvm/hyp/nvhe/hyp-init.S | 48 +++++++++++ arch/arm64/kvm/hyp/nvhe/psci-relay.c | 115 +++++++++++++++++++++++++++ 2 files changed, 163 insertions(+) diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-init.S b/arch/arm64/kvm/hyp/nvhe/hyp-init.S index b0856b006bc0..d07e75f8242e 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-init.S +++ b/arch/arm64/kvm/hyp/nvhe/hyp-init.S @@ -9,6 +9,7 @@ #include #include +#include #include #include #include @@ -139,6 +140,53 @@ alternative_else_nop_endif ret SYM_CODE_END(___kvm_hyp_init) +/* + * PSCI CPU_ON entry point + * + * x0: struct kvm_nvhe_init_params PA + */ +SYM_CODE_START(kvm_hyp_cpu_entry) + mov x1, #1 // is_cpu_on = true + b __kvm_hyp_init_cpu +SYM_CODE_END(kvm_hyp_cpu_entry) + +/* + * Common code for CPU entry points. Initializes EL2 state and + * installs the hypervisor before handing over to a C handler. + * + * x0: struct kvm_nvhe_init_params PA + * x1: bool is_cpu_on + */ +SYM_CODE_START_LOCAL(__kvm_hyp_init_cpu) + mov x28, x0 // Stash arguments + mov x29, x1 + + /* Check that the core was booted in EL2. */ + mrs x0, CurrentEL + cmp x0, #CurrentEL_EL2 + b.eq 2f + + /* The core booted in EL1. KVM cannot be initialized on it. */ +1: wfe + wfi + b 1b + +2: msr SPsel, #1 // We want to use SP_EL{1,2} + + /* Initialize EL2 CPU state to sane values. */ + init_el2_state nvhe // Clobbers x0..x2 + + /* Enable MMU, set vectors and stack. */ + mov x0, x28 + bl ___kvm_hyp_init // Clobbers x0..x3 + + /* Leave idmap. */ + mov x0, x29 + ldr x1, =kvm_host_psci_cpu_entry + kimg_hyp_va x1, x2 + br x1 +SYM_CODE_END(__kvm_hyp_init_cpu) + SYM_CODE_START(__kvm_handle_stub_hvc) cmp x0, #HVC_SOFT_RESTART b.ne 1f diff --git a/arch/arm64/kvm/hyp/nvhe/psci-relay.c b/arch/arm64/kvm/hyp/nvhe/psci-relay.c index 5ad56a875ffa..637e22ed71fc 100644 --- a/arch/arm64/kvm/hyp/nvhe/psci-relay.c +++ b/arch/arm64/kvm/hyp/nvhe/psci-relay.c @@ -9,12 +9,17 @@ #include #include #include +#include #include #include #include #include +void kvm_hyp_cpu_entry(unsigned long r0); + +void __noreturn __host_enter(struct kvm_cpu_context *host_ctxt); + /* Config options set by the host. */ __ro_after_init u32 kvm_host_psci_version; __ro_after_init struct psci_0_1_function_ids kvm_host_psci_0_1_function_ids; @@ -22,6 +27,24 @@ __ro_after_init s64 hyp_physvirt_offset; #define __hyp_pa(x) ((phys_addr_t)((x)) + hyp_physvirt_offset) +#define INVALID_CPU_ID UINT_MAX + +struct psci_boot_args { + atomic_t lock; + unsigned long pc; + unsigned long r0; +}; + +#define PSCI_BOOT_ARGS_UNLOCKED 0 +#define PSCI_BOOT_ARGS_LOCKED 1 + +#define PSCI_BOOT_ARGS_INIT \ + ((struct psci_boot_args){ \ + .lock = ATOMIC_INIT(PSCI_BOOT_ARGS_UNLOCKED), \ + }) + +static DEFINE_PER_CPU(struct psci_boot_args, cpu_on_args) = PSCI_BOOT_ARGS_INIT; + static u64 get_psci_func_id(struct kvm_cpu_context *host_ctxt) { DECLARE_REG(u64, func_id, host_ctxt, 0); @@ -75,11 +98,101 @@ static __noreturn unsigned long psci_forward_noreturn(struct kvm_cpu_context *ho hyp_panic(); /* unreachable */ } +static unsigned int find_cpu_id(u64 mpidr) +{ + unsigned int i; + + /* Reject invalid MPIDRs */ + if (mpidr & ~MPIDR_HWID_BITMASK) + return INVALID_CPU_ID; + + for (i = 0; i < NR_CPUS; i++) { + if (cpu_logical_map(i) == mpidr) + return i; + } + + return INVALID_CPU_ID; +} + +static __always_inline bool try_acquire_boot_args(struct psci_boot_args *args) +{ + return atomic_cmpxchg_acquire(&args->lock, + PSCI_BOOT_ARGS_UNLOCKED, + PSCI_BOOT_ARGS_LOCKED) == + PSCI_BOOT_ARGS_UNLOCKED; +} + +static __always_inline void release_boot_args(struct psci_boot_args *args) +{ + atomic_set_release(&args->lock, PSCI_BOOT_ARGS_UNLOCKED); +} + +static int psci_cpu_on(u64 func_id, struct kvm_cpu_context *host_ctxt) +{ + DECLARE_REG(u64, mpidr, host_ctxt, 1); + DECLARE_REG(unsigned long, pc, host_ctxt, 2); + DECLARE_REG(unsigned long, r0, host_ctxt, 3); + + unsigned int cpu_id; + struct psci_boot_args *boot_args; + struct kvm_nvhe_init_params *init_params; + int ret; + + /* + * Find the logical CPU ID for the given MPIDR. The search set is + * the set of CPUs that were online at the point of KVM initialization. + * Booting other CPUs is rejected because their cpufeatures were not + * checked against the finalized capabilities. This could be relaxed + * by doing the feature checks in hyp. + */ + cpu_id = find_cpu_id(mpidr); + if (cpu_id == INVALID_CPU_ID) + return PSCI_RET_INVALID_PARAMS; + + boot_args = per_cpu_ptr(hyp_symbol_addr(cpu_on_args), cpu_id); + init_params = per_cpu_ptr(hyp_symbol_addr(kvm_init_params), cpu_id); + + /* Check if the target CPU is already being booted. */ + if (!try_acquire_boot_args(boot_args)) + return PSCI_RET_ALREADY_ON; + + boot_args->pc = pc; + boot_args->r0 = r0; + wmb(); + + ret = psci_call(func_id, mpidr, + __hyp_pa(hyp_symbol_addr(kvm_hyp_cpu_entry)), + __hyp_pa(init_params)); + + /* If successful, the lock will be released by the target CPU. */ + if (ret != PSCI_RET_SUCCESS) + release_boot_args(boot_args); + + return ret; +} + +asmlinkage void __noreturn kvm_host_psci_cpu_entry(bool is_cpu_on) +{ + struct psci_boot_args *boot_args; + struct kvm_cpu_context *host_ctxt; + + host_ctxt = &this_cpu_ptr(hyp_symbol_addr(kvm_host_data))->host_ctxt; + boot_args = this_cpu_ptr(hyp_symbol_addr(cpu_on_args)); + + cpu_reg(host_ctxt, 0) = boot_args->r0; + write_sysreg_el2(boot_args->pc, SYS_ELR); + release_boot_args(boot_args); + + __host_enter(host_ctxt); +} + static unsigned long psci_0_1_handler(u64 func_id, struct kvm_cpu_context *host_ctxt) { if ((func_id == kvm_host_psci_0_1_function_ids.cpu_off) || (func_id == kvm_host_psci_0_1_function_ids.migrate)) return psci_forward(host_ctxt); + else if (func_id == kvm_host_psci_0_1_function_ids.cpu_on) + return psci_cpu_on(func_id, host_ctxt); else return PSCI_RET_NOT_SUPPORTED; } @@ -98,6 +211,8 @@ static unsigned long psci_0_2_handler(u64 func_id, struct kvm_cpu_context *host_ case PSCI_0_2_FN_SYSTEM_RESET: psci_forward_noreturn(host_ctxt); unreachable(); + case PSCI_0_2_FN64_CPU_ON: + return psci_cpu_on(func_id, host_ctxt); default: return PSCI_RET_NOT_SUPPORTED; } From patchwork Wed Dec 2 18:41:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11946813 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A8DC8C64E7C for ; Wed, 2 Dec 2020 18:48:10 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 467A321741 for ; Wed, 2 Dec 2020 18:48:10 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 467A321741 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=GOvotSQvjwQzhTQW9snU5Y4Rkki3nOaic7UV8YMA+WA=; b=wAApViSgJIKtdJsmM5gHgkFg9 mpGr+9lTbMtdnwRkU9iYv4LozNEoElHrKoWDtASBsyr1B/Ebr8c0HPNK2SArv74Y20iforE7suHkY JPugiEtchV0n7DY0Mw1iNAMhxWRb1RPTT2s40RVlHWTobTg04vususe+k8RjLnctpSgsXqKlaiehR ez763pC9lm7mD3JbjHH/T4xW6uqU5hgxd5HAA7TQ7ytC2ogUnx6mzA313S8nCrcs0K57LxV2HmmNo 2E44uvrie+M/lwor8gTRvIUHaTO+q84rkIoREAaU0J2Egao72rOykD2Tq7XFUuUeQws0ks5eoFkGg UZmcwq3mg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX8e-0002IV-E1; Wed, 02 Dec 2020 18:46:04 +0000 Received: from mail-wr1-x443.google.com ([2a00:1450:4864:20::443]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX50-0008Rx-4u for linux-arm-kernel@lists.infradead.org; Wed, 02 Dec 2020 18:42:21 +0000 Received: by mail-wr1-x443.google.com with SMTP id 23so5122156wrc.8 for ; Wed, 02 Dec 2020 10:42:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=b4YL0IBplg+FWHBi8Rr0BRJAelmSAxlpGLG7LH6PvWI=; b=BhcEbJ1P8iLmvnw20r8Tmdyd3uTKIEEEWanAgbopFJTNFw5yNcdQvwTUNvQm2zbtm3 MHgBmJr0AE5W0c6JE2xddQ5i5ab1IfU2BwArNg5njgUUrd/uVHEJKFvpIbGcawzT26rf x+PeIZ9V3lFFr9yp1DAwWrivqSPjV1AOPu1JJRL40OkApYw9bcUpUTCI8+LkMHe1yJ6Q L738ogFUJ+t8T4nMrKyXSyNTeghbSuBY23Glw9aJOAi6yklPu/qHPdvV0uiP93uNIwQ/ 8DzpeG2eJs3gxU2aNMJgi5QCywuwe+XzPUdRlq3G0kodHjpRG1cLVuBOQHoTcHxX80C5 daBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=b4YL0IBplg+FWHBi8Rr0BRJAelmSAxlpGLG7LH6PvWI=; b=fCt8Qg3A0ccZDtDSu3eJn58DkFYnfwhOMlvBcOwD56M/hk9QvbgrTygT69n4GNzS09 m5KZE6heK473dH7MHQgHo6U1XOBJBixsKJqpTUSHBui/97qZemYqGy2QNozsZFECGZF+ xTZnNEpL8MNb3d2J8n/nAce7IpSHohRizYiHyRNa0yf2TC2YfjOCEGgvk6+yplGrISLq VppLEXuhsZZMPbzR9UvDe9Od6Zt8mmPA7MpbcuT9brfMQACOoKzcdypBNqpB/2HDUYXG xBhROM73TubtxJWDwb2WNnrZ+8JrPQslJYm/d+qWV64ijMDBDoBOXjH+2Dw9UXKCv5DB pVFQ== X-Gm-Message-State: AOAM530yR84nm69NUDqxrRIn8yPCTLM+10iQwgDECLr82i1QNwLvvSmc pJvH9Dyhl+BdpD0mI2R7fesUfA== X-Google-Smtp-Source: ABdhPJwz6THFS1yYmKpqoEbi0AmdVm+ctA2YRt55ZaPapRgb33xR8DwkQTqRsJm5wJLFpq5eL/rKSg== X-Received: by 2002:adf:f102:: with SMTP id r2mr5065973wro.315.1606934537044; Wed, 02 Dec 2020 10:42:17 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:5ddd:b7c5:e3c9:e87a]) by smtp.gmail.com with ESMTPSA id s133sm3086045wmf.38.2020.12.02.10.42.15 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 02 Dec 2020 10:42:15 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v4 22/26] kvm: arm64: Intercept host's CPU_SUSPEND PSCI SMCs Date: Wed, 2 Dec 2020 18:41:18 +0000 Message-Id: <20201202184122.26046-23-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201202184122.26046-1-dbrazdil@google.com> References: <20201202184122.26046-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201202_134219_585430_9C497A44 X-CRM114-Status: GOOD ( 22.75 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add a handler of CPU_SUSPEND host PSCI SMCs. The SMC can either enter a sleep state indistinguishable from a WFI or a deeper sleep state that behaves like a CPU_OFF+CPU_ON except that the core is still considered online while asleep. The handler saves r0,pc of the host and makes the same call to EL3 with the hyp CPU entry point. It either returns back to the handler and then back to the host, or wakes up into the entry point and initializes EL2 state before dropping back to EL1. No EL2 state needs to be saved/restored for this purpose. CPU_ON and CPU_SUSPEND are both implemented using struct psci_boot_args to store the state upon powerup, with each CPU having separate structs for CPU_ON and CPU_SUSPEND so that CPU_SUSPEND can operate locklessly and so that a CPU_ON call targeting a CPU cannot interfere with a concurrent CPU_SUSPEND call on that CPU. Signed-off-by: David Brazdil --- arch/arm64/kvm/hyp/nvhe/hyp-init.S | 10 +++++++ arch/arm64/kvm/hyp/nvhe/psci-relay.c | 44 ++++++++++++++++++++++++++-- 2 files changed, 52 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-init.S b/arch/arm64/kvm/hyp/nvhe/hyp-init.S index d07e75f8242e..0853f62b052b 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-init.S +++ b/arch/arm64/kvm/hyp/nvhe/hyp-init.S @@ -150,6 +150,16 @@ SYM_CODE_START(kvm_hyp_cpu_entry) b __kvm_hyp_init_cpu SYM_CODE_END(kvm_hyp_cpu_entry) +/* + * PSCI CPU_SUSPEND entry point + * + * x0: struct kvm_nvhe_init_params PA + */ +SYM_CODE_START(kvm_hyp_cpu_resume) + mov x1, #0 // is_cpu_on = false + b __kvm_hyp_init_cpu +SYM_CODE_END(kvm_hyp_cpu_resume) + /* * Common code for CPU entry points. Initializes EL2 state and * installs the hypervisor before handing over to a C handler. diff --git a/arch/arm64/kvm/hyp/nvhe/psci-relay.c b/arch/arm64/kvm/hyp/nvhe/psci-relay.c index 637e22ed71fc..688cf7f40d42 100644 --- a/arch/arm64/kvm/hyp/nvhe/psci-relay.c +++ b/arch/arm64/kvm/hyp/nvhe/psci-relay.c @@ -17,6 +17,7 @@ #include void kvm_hyp_cpu_entry(unsigned long r0); +void kvm_hyp_cpu_resume(unsigned long r0); void __noreturn __host_enter(struct kvm_cpu_context *host_ctxt); @@ -44,6 +45,7 @@ struct psci_boot_args { }) static DEFINE_PER_CPU(struct psci_boot_args, cpu_on_args) = PSCI_BOOT_ARGS_INIT; +static DEFINE_PER_CPU(struct psci_boot_args, suspend_args) = PSCI_BOOT_ARGS_INIT; static u64 get_psci_func_id(struct kvm_cpu_context *host_ctxt) { @@ -171,17 +173,51 @@ static int psci_cpu_on(u64 func_id, struct kvm_cpu_context *host_ctxt) return ret; } +static int psci_cpu_suspend(u64 func_id, struct kvm_cpu_context *host_ctxt) +{ + DECLARE_REG(u64, power_state, host_ctxt, 1); + DECLARE_REG(unsigned long, pc, host_ctxt, 2); + DECLARE_REG(unsigned long, r0, host_ctxt, 3); + + struct psci_boot_args *boot_args; + struct kvm_nvhe_init_params *init_params; + + boot_args = this_cpu_ptr(hyp_symbol_addr(suspend_args)); + init_params = this_cpu_ptr(hyp_symbol_addr(kvm_init_params)); + + /* + * No need to acquire a lock before writing to boot_args because a core + * can only suspend itself. Racy CPU_ON calls use a separate struct. + */ + boot_args->pc = pc; + boot_args->r0 = r0; + + /* + * Will either return if shallow sleep state, or wake up into the entry + * point if it is a deep sleep state. + */ + return psci_call(func_id, power_state, + __hyp_pa(hyp_symbol_addr(kvm_hyp_cpu_resume)), + __hyp_pa(init_params)); +} + asmlinkage void __noreturn kvm_host_psci_cpu_entry(bool is_cpu_on) { struct psci_boot_args *boot_args; struct kvm_cpu_context *host_ctxt; host_ctxt = &this_cpu_ptr(hyp_symbol_addr(kvm_host_data))->host_ctxt; - boot_args = this_cpu_ptr(hyp_symbol_addr(cpu_on_args)); + + if (is_cpu_on) + boot_args = this_cpu_ptr(hyp_symbol_addr(cpu_on_args)); + else + boot_args = this_cpu_ptr(hyp_symbol_addr(suspend_args)); cpu_reg(host_ctxt, 0) = boot_args->r0; write_sysreg_el2(boot_args->pc, SYS_ELR); - release_boot_args(boot_args); + + if (is_cpu_on) + release_boot_args(boot_args); __host_enter(host_ctxt); } @@ -193,6 +229,8 @@ static unsigned long psci_0_1_handler(u64 func_id, struct kvm_cpu_context *host_ return psci_forward(host_ctxt); else if (func_id == kvm_host_psci_0_1_function_ids.cpu_on) return psci_cpu_on(func_id, host_ctxt); + else if (func_id == kvm_host_psci_0_1_function_ids.cpu_suspend) + return psci_cpu_suspend(func_id, host_ctxt); else return PSCI_RET_NOT_SUPPORTED; } @@ -211,6 +249,8 @@ static unsigned long psci_0_2_handler(u64 func_id, struct kvm_cpu_context *host_ case PSCI_0_2_FN_SYSTEM_RESET: psci_forward_noreturn(host_ctxt); unreachable(); + case PSCI_0_2_FN64_CPU_SUSPEND: + return psci_cpu_suspend(func_id, host_ctxt); case PSCI_0_2_FN64_CPU_ON: return psci_cpu_on(func_id, host_ctxt); default: From patchwork Wed Dec 2 18:41:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11946817 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 68A8FC64E7C for ; Wed, 2 Dec 2020 18:49:04 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C1FBA21741 for ; Wed, 2 Dec 2020 18:49:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C1FBA21741 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=VpR1OVSCAG3Oh/4LX09nX+f7eX0M3WJX0DBowfiRfIs=; b=3d7r6SXRH3rqYghKtCVg2B7YJ o3tLRGmTIIyB4+LhgHPf4bxP9+aDfdBW7baF2SRpBFgm/azHZA50DIn9I3fkJSbGTaoBWCGxAzIsi ZbkFlhJRyuy6MB+EyAGNnub3iXX9OLJcww/XozukqS9xmp0Ed6b6NeNF87Y4KvjefR8UbkgFJlO1z euUKWadio+sogJBRqTo//pIlhj3gXRVNP/XQ2P/RDuT4uBDmnoQ9O/JixfLxSCDBmUbdZS91a1zmA KX9yat8oQFMmioblJLY0fsST0i+kdVL1Saxf7okRyZKtBQkatLXGvxJ9lmODFVZk0tb/fpl1lzE8s koUqUsf/w==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX95-0002bn-Sc; Wed, 02 Dec 2020 18:46:34 +0000 Received: from mail-wm1-x344.google.com ([2a00:1450:4864:20::344]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX53-0008Su-Dh for linux-arm-kernel@lists.infradead.org; Wed, 02 Dec 2020 18:42:24 +0000 Received: by mail-wm1-x344.google.com with SMTP id a3so11094045wmb.5 for ; Wed, 02 Dec 2020 10:42:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=FBH6HEW5NPUtTsYV4IlYky2BDGVGm12SrN30lN4PkFY=; b=GwirdQYpnGgq1EVsJzAuWcQcbP11n6x036yOyPPx1vuGLdUpAEBFHj+LcFwdzYyxw+ T8lnqGlHicu8lC1OG+9o9owM1KpTEfxKieiZO+v648VwSXWHshhhM5L5mcoUnNJoGpgP JF9LDYFE971ZkAyI4UJiC9Wj2Xc8p3fiDo1tCV+m8q7oGRHw7LI+kmLYVPoNtx23niwN DKnFEZdky5kwe4pgwVbOEWZKaUBtyvs+ZR+1j/PIZVxPLVLSf3X2PlsluY9qutX26LPO CoDv2JHfB182QCETZ+fYEDp3BzS+9YTS+Cpc2qSLa7FqP4K3kA3VJYq5HemX3PVOurbw WfCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FBH6HEW5NPUtTsYV4IlYky2BDGVGm12SrN30lN4PkFY=; b=DaV8xQdhx7wMyTlQQ0U/2dq2TSu6FECMIrjc35iboJoH2KDx14wCTYOsCZcEFipMbh +/ykd4iIIW942sjix8JstAPEQC82pfClW31E6jaVHNMt7XUvNk+Jje2Mfe6OybGvliND hUkkmLkRz62CYvhRWbHentoZy/xTC8KnAPzTro5Oh8qFK4b2Fd1JF92AoK5dVgEZaBIz lS57tNq2TBL4984yk6NViPLT1I6dDMztgdoUXEmg4ksMFTLXiAQt0+PGOLfraTVwfN72 yUofEzBx1C59RmdP1dAILj4gKL0dSmKywtSJ5UTh2EXY7n3sDQjnbME0pJyXlixGpVE1 42FQ== X-Gm-Message-State: AOAM533u8gNH19kR9GntHuiE3QNNwf1ZGeiVcFeBngNdkqrowpthdtlh IH/Tm4TVz4WAXchwDOIo9OtSKg== X-Google-Smtp-Source: ABdhPJxcoHouQF9MWhIcY2M29LL8SVnxKTEFmpQkXa7h00mrGlfxwPNATVEGfGu2v86fKDMdyLCvLQ== X-Received: by 2002:a1c:a402:: with SMTP id n2mr4626696wme.185.1606934539318; Wed, 02 Dec 2020 10:42:19 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:5ddd:b7c5:e3c9:e87a]) by smtp.gmail.com with ESMTPSA id x9sm3176873wru.55.2020.12.02.10.42.17 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 02 Dec 2020 10:42:18 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v4 23/26] kvm: arm64: Intercept host's SYSTEM_SUSPEND PSCI SMCs Date: Wed, 2 Dec 2020 18:41:19 +0000 Message-Id: <20201202184122.26046-24-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201202184122.26046-1-dbrazdil@google.com> References: <20201202184122.26046-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201202_134222_134690_670F250F X-CRM114-Status: GOOD ( 17.06 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add a handler of SYSTEM_SUSPEND host PSCI SMCs. The semantics are equivalent to CPU_SUSPEND, typically called on the last online CPU. Reuse the same entry point and boot args struct as CPU_SUSPEND. Signed-off-by: David Brazdil --- arch/arm64/kvm/hyp/nvhe/hyp-init.S | 2 +- arch/arm64/kvm/hyp/nvhe/psci-relay.c | 26 ++++++++++++++++++++++++++ 2 files changed, 27 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-init.S b/arch/arm64/kvm/hyp/nvhe/hyp-init.S index 0853f62b052b..a2e251547625 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-init.S +++ b/arch/arm64/kvm/hyp/nvhe/hyp-init.S @@ -151,7 +151,7 @@ SYM_CODE_START(kvm_hyp_cpu_entry) SYM_CODE_END(kvm_hyp_cpu_entry) /* - * PSCI CPU_SUSPEND entry point + * PSCI CPU_SUSPEND / SYSTEM_SUSPEND entry point * * x0: struct kvm_nvhe_init_params PA */ diff --git a/arch/arm64/kvm/hyp/nvhe/psci-relay.c b/arch/arm64/kvm/hyp/nvhe/psci-relay.c index 688cf7f40d42..08dc9de69314 100644 --- a/arch/arm64/kvm/hyp/nvhe/psci-relay.c +++ b/arch/arm64/kvm/hyp/nvhe/psci-relay.c @@ -201,6 +201,30 @@ static int psci_cpu_suspend(u64 func_id, struct kvm_cpu_context *host_ctxt) __hyp_pa(init_params)); } +static int psci_system_suspend(u64 func_id, struct kvm_cpu_context *host_ctxt) +{ + DECLARE_REG(unsigned long, pc, host_ctxt, 1); + DECLARE_REG(unsigned long, r0, host_ctxt, 2); + + struct psci_boot_args *boot_args; + struct kvm_nvhe_init_params *init_params; + + boot_args = this_cpu_ptr(hyp_symbol_addr(suspend_args)); + init_params = this_cpu_ptr(hyp_symbol_addr(kvm_init_params)); + + /* + * No need to acquire a lock before writing to boot_args because a core + * can only suspend itself. Racy CPU_ON calls use a separate struct. + */ + boot_args->pc = pc; + boot_args->r0 = r0; + + /* Will only return on error. */ + return psci_call(func_id, + __hyp_pa(hyp_symbol_addr(kvm_hyp_cpu_resume)), + __hyp_pa(init_params), 0); +} + asmlinkage void __noreturn kvm_host_psci_cpu_entry(bool is_cpu_on) { struct psci_boot_args *boot_args; @@ -265,6 +289,8 @@ static unsigned long psci_1_0_handler(u64 func_id, struct kvm_cpu_context *host_ case PSCI_1_0_FN_SET_SUSPEND_MODE: case PSCI_1_1_FN64_SYSTEM_RESET2: return psci_forward(host_ctxt); + case PSCI_1_0_FN64_SYSTEM_SUSPEND: + return psci_system_suspend(func_id, host_ctxt); default: return psci_0_2_handler(func_id, host_ctxt); } From patchwork Wed Dec 2 18:41:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11946819 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B71D9C71155 for ; Wed, 2 Dec 2020 18:49:47 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 061C22173E for ; Wed, 2 Dec 2020 18:49:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 061C22173E Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=zZSLDLITjg3JFoZjG/+SS3uZK7Frt21GZd3J6Hs1F5M=; b=1kNstwx5Ab/PkwHceen/UbTF4 1YHrr9MZmt9+bGTbUKY5voY39WmjWshPMlD9iyURnqJFpEeR/x2ntRA4cm+qsSvzjuIJ8rdC4I6KM BQDuFJGsb/k7pNloM0Ba3qTgIHzQnYOBma3FQ4hFRkupWf9VEWcl+A0PN5ox2XYs9iwh/WXZLWwpe 0ns0WVBAEkWezYb6G2/nDYVJWbebaZLCOanldWRfQ93Gum5PWu54yvy8+QaxIHOLFeXsL1Y2sH99p CTPJYREE5wpQGJT8m41tp1saXpmONDr/IwzQ0ZFq/w8vkBerhSQCLhVLOSv/bZ/iMb28BAsTRQuLJ k7DPKl6yQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX9U-0002mu-MB; Wed, 02 Dec 2020 18:46:56 +0000 Received: from mail-wm1-x344.google.com ([2a00:1450:4864:20::344]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX56-0008Tc-S0 for linux-arm-kernel@lists.infradead.org; Wed, 02 Dec 2020 18:42:27 +0000 Received: by mail-wm1-x344.google.com with SMTP id w206so1298837wma.0 for ; Wed, 02 Dec 2020 10:42:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=pdq1h6Iz7BaS9XHPJecHzTrxzCvHIq7bY+OCaqaUt9I=; b=KOEK4s1Ox/QOtxtUNQV6lW8tsX7N/Qz11MQmpxX+mgoAtIqI4ROGadur0j1GvfPZ3S qnd7H6icVbP7FDjQA0VS8bb07nR1MeQ9HF9dNdeQKXtjw8SrZGW8CvARHvvIL/pDJYw9 eub897JAH61yZK0cmDaSunb3DJmizow9dhDher5Z/q5p1VMb022tLjTj1d+6gVWvzGkI lV5sPeEzfS+NmliZtrXh/m9lZlnFM4iwGM29uCkyDwzCsWZB16dkh9pbesIly660+TGZ JZZN/EW6Xuubk9oqq8U0SFC0cRqU+n2i1xAYFPOjwGtz5rGvLtCOgvpc80RP5Qr2nhJv CvJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pdq1h6Iz7BaS9XHPJecHzTrxzCvHIq7bY+OCaqaUt9I=; b=cQICjjZCRpXZrRwTF7sNUQyGg9Yoc+bBVOhW6yLAHVYzI5aIBDDsBCh0KEBhZsPt6P rIaWQkRJYqXjSg/VFV1teipiQ7+E7fJEgeBb97ml53Q+AuJGwKYuBEsD3ugyMrfpEcbQ vLZPAVjLusERKRKVLUnKE4fC3JKCFgh3qQ6hxiUk0pM3XMTca654gNDVtCr+1n5Iw0If hdqb1lElOHkI6bseNs1QkhSvovqpW8Cf8KTAJxtbj5IMO1nV0GwsLU1ThCa0A15dIfp+ 63CtWf7oERdQkQragcobN6Zj7KaqaQnYu08+7ooX0e2ENDTbTJRRbAYWwKP/gvxTLw6S RPZw== X-Gm-Message-State: AOAM530SFbd4Vr1IuDoXxIADXe/nFH3cYBdrfatg2ebY2uLUmevqTcz6 frBvdeMO+3DnWiP7PrjZRsnLtg== X-Google-Smtp-Source: ABdhPJyAghBSBueIUZyEOdUpBSWInTgxUTJ9HrAjeXaOhO6iHRoqpQZbX/aG+qsqholRHQQrI3MT8A== X-Received: by 2002:a1c:9ac6:: with SMTP id c189mr4470560wme.137.1606934541552; Wed, 02 Dec 2020 10:42:21 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:5ddd:b7c5:e3c9:e87a]) by smtp.gmail.com with ESMTPSA id o13sm2912807wmc.44.2020.12.02.10.42.20 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 02 Dec 2020 10:42:20 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v4 24/26] kvm: arm64: Keep nVHE EL2 vector installed Date: Wed, 2 Dec 2020 18:41:20 +0000 Message-Id: <20201202184122.26046-25-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201202184122.26046-1-dbrazdil@google.com> References: <20201202184122.26046-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201202_134225_185506_1F776B59 X-CRM114-Status: GOOD ( 16.78 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org KVM by default keeps the stub vector installed and installs the nVHE vector only briefly for init and later on demand. Change this policy to install the vector at init and then never uninstall it if the kernel was given the protected KVM command line parameter. Signed-off-by: David Brazdil --- arch/arm64/kvm/arm.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index fda7bfa440d5..5f8776eefe46 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1531,7 +1531,8 @@ static void _kvm_arch_hardware_disable(void *discard) void kvm_arch_hardware_disable(void) { - _kvm_arch_hardware_disable(NULL); + if (!is_protected_kvm_enabled()) + _kvm_arch_hardware_disable(NULL); } #ifdef CONFIG_CPU_PM @@ -1574,11 +1575,13 @@ static struct notifier_block hyp_init_cpu_pm_nb = { static void __init hyp_cpu_pm_init(void) { - cpu_pm_register_notifier(&hyp_init_cpu_pm_nb); + if (!is_protected_kvm_enabled()) + cpu_pm_register_notifier(&hyp_init_cpu_pm_nb); } static void __init hyp_cpu_pm_exit(void) { - cpu_pm_unregister_notifier(&hyp_init_cpu_pm_nb); + if (!is_protected_kvm_enabled()) + cpu_pm_unregister_notifier(&hyp_init_cpu_pm_nb); } #else static inline void hyp_cpu_pm_init(void) @@ -1666,7 +1669,8 @@ static int init_subsystems(void) kvm_sys_reg_table_init(); out: - on_each_cpu(_kvm_arch_hardware_disable, NULL, 1); + if (err || !is_protected_kvm_enabled()) + on_each_cpu(_kvm_arch_hardware_disable, NULL, 1); return err; } From patchwork Wed Dec 2 18:41:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11946821 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9DEFFC71155 for ; Wed, 2 Dec 2020 18:50:49 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 287B622248 for ; Wed, 2 Dec 2020 18:50:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 287B622248 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=jmTtkW3gQrQEpV6RGfmrE9uD/saS+2Uqfdmkh4GW35U=; b=VZVYUiR8hACsPLu4oiU7LnkzK nQMuatuh1mmj3AvCDB9euHnfdjN59cFwUPUSrsE9LNxvyruRP/ElnygSZTK5PrN+JpZPQBCbKBF3Y fSm4gG05m8hg+tG1yZsu5Yne4ebfDxMvnGr4eZG6GKej8KqdXtj5oEXGShDvquLopK/v375uSSjzG WUrfr4Yma4BTSHOZMU3T0IFdvR4Dqz1QZ/p2vvhx3Rdeb2o85M76UJ6NwJ4r4/mZmVhNowuQO/PnX IeqyFxeVrlwT7tLFBkwGevVl15wLFUN78dOIsKYx1LTCM7AFQ9nAcX54lS02IzrsmqLEzg6Yw+u47 Ps5HuOxDQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkXAW-0003Bx-3d; Wed, 02 Dec 2020 18:48:00 +0000 Received: from mail-wm1-x342.google.com ([2a00:1450:4864:20::342]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX59-0008V9-Kb for linux-arm-kernel@lists.infradead.org; Wed, 02 Dec 2020 18:42:31 +0000 Received: by mail-wm1-x342.google.com with SMTP id h21so11170567wmb.2 for ; Wed, 02 Dec 2020 10:42:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=24QBqzGkP70q8LvHdgbjJ+f63rQSkKFebn+QrdGKy8o=; b=KZbI8e+wbELCtP1nDqFe7mgn9KY48UKckwNmSnl0PwytWNjDtRLIBMhva1x+dBTE2F ncky8aqIUtGjpPLllziKYq/cqu2thcX5CDDVtfGHw8sjqNahjU7LtE0i0TmlN4uCbAaE bPT2DRil1LdObVMx3sZ5MnpPztiPO2tEQLZ8Fb8ep9bbC0hQ+1TygpBboVZkBTOZI8QH 1Zy4uREQPBIU+ri3BBw5xw9uLFgQxJu9+o11KqWj/GF9lCN/OW9EoB+Mvw5CsgHNvLfX gVkMnKuo44a4NiHNcTRZdIn868LOOGz2mS9lvmqJo/e6AT8gr/G9c5htB+JaZ0MtIOAW rp1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=24QBqzGkP70q8LvHdgbjJ+f63rQSkKFebn+QrdGKy8o=; b=dvL/MXhoml62ABpr1cFSuZJSGx7DSAIc5+hN00Yi7ItG3AH3fqjD5eurPDeQ/bYeA6 NIBs0mBX1j8v98x8mxgu7HUJ180pC08oIVTc2vD4whhu+9EpM2Qr0xGObPOlUWZL4PCE vQn/dtUibqWlnkZlo1MftdBHrAS2flC8BgCMfLH8plpfhzSBsKQ7pEL6657gi8LObuCL 5aV3btsqe/QvVQ5MIxKfVox67Df1onk7SXOtqNF1LY4aQFqBLDYV2gDh+cwZUL6YiniJ o4VlDs1vinnTS3EQ2r98BDHDfu18CReUhVvY1KCpDk44OPoHSbTU1QK4vYeqRjdPT5h7 t1fA== X-Gm-Message-State: AOAM530DD958DCpZu1+4fzQqfDtzf31YG6tYH3IHItGN72kDE7aXGsHA tOZmzshujVSArbE9Uy7nTCpOhA== X-Google-Smtp-Source: ABdhPJwIWj1lrrodmGW3N9pVgmHulb+Cio519dRAnYX137VaTJJ4G1ZrC9l3HnzANF4KiUfFUxqSzg== X-Received: by 2002:a7b:cd11:: with SMTP id f17mr4383651wmj.127.1606934544393; Wed, 02 Dec 2020 10:42:24 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:5ddd:b7c5:e3c9:e87a]) by smtp.gmail.com with ESMTPSA id z22sm2940715wml.1.2020.12.02.10.42.22 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 02 Dec 2020 10:42:22 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v4 25/26] kvm: arm64: Trap host SMCs in protected mode Date: Wed, 2 Dec 2020 18:41:21 +0000 Message-Id: <20201202184122.26046-26-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201202184122.26046-1-dbrazdil@google.com> References: <20201202184122.26046-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201202_134229_262661_17C0A73C X-CRM114-Status: GOOD ( 17.09 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org While protected KVM is installed, start trapping all host SMCs. For now these are simply forwarded to EL3, except PSCI CPU_ON/CPU_SUSPEND/SYSTEM_SUSPEND which are intercepted and the hypervisor installed on newly booted cores. Create new constant HCR_HOST_NVHE_PROTECTED_FLAGS with the new set of HCR flags to use while the nVHE vector is installed when the kernel was booted with the protected flag enabled. Switch back to the default HCR flags when switching back to the stub vector. Signed-off-by: David Brazdil --- arch/arm64/include/asm/kvm_arm.h | 1 + arch/arm64/kvm/hyp/nvhe/hyp-init.S | 10 ++++++++++ arch/arm64/kvm/hyp/nvhe/switch.c | 5 ++++- 3 files changed, 15 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h index 64ce29378467..4e90c2debf70 100644 --- a/arch/arm64/include/asm/kvm_arm.h +++ b/arch/arm64/include/asm/kvm_arm.h @@ -80,6 +80,7 @@ HCR_FMO | HCR_IMO | HCR_PTW ) #define HCR_VIRT_EXCP_MASK (HCR_VSE | HCR_VI | HCR_VF) #define HCR_HOST_NVHE_FLAGS (HCR_RW | HCR_API | HCR_APK | HCR_ATA) +#define HCR_HOST_NVHE_PROTECTED_FLAGS (HCR_HOST_NVHE_FLAGS | HCR_TSC) #define HCR_HOST_VHE_FLAGS (HCR_RW | HCR_TGE | HCR_E2H) /* TCR_EL2 Registers bits */ diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-init.S b/arch/arm64/kvm/hyp/nvhe/hyp-init.S index a2e251547625..31b060a44045 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-init.S +++ b/arch/arm64/kvm/hyp/nvhe/hyp-init.S @@ -88,6 +88,11 @@ SYM_CODE_END(__kvm_hyp_init) * x0: struct kvm_nvhe_init_params PA */ SYM_CODE_START_LOCAL(___kvm_hyp_init) +alternative_if ARM64_KVM_PROTECTED_MODE + mov_q x1, HCR_HOST_NVHE_PROTECTED_FLAGS + msr hcr_el2, x1 +alternative_else_nop_endif + ldr x1, [x0, #NVHE_INIT_TPIDR_EL2] msr tpidr_el2, x1 @@ -230,6 +235,11 @@ reset: msr sctlr_el2, x5 isb +alternative_if ARM64_KVM_PROTECTED_MODE + mov_q x5, HCR_HOST_NVHE_FLAGS + msr hcr_el2, x5 +alternative_else_nop_endif + /* Install stub vectors */ adr_l x5, __hyp_stub_vectors msr vbar_el2, x5 diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/switch.c index 3e50ff35aa4f..f3d0e9eca56c 100644 --- a/arch/arm64/kvm/hyp/nvhe/switch.c +++ b/arch/arm64/kvm/hyp/nvhe/switch.c @@ -97,7 +97,10 @@ static void __deactivate_traps(struct kvm_vcpu *vcpu) mdcr_el2 |= MDCR_EL2_E2PB_MASK << MDCR_EL2_E2PB_SHIFT; write_sysreg(mdcr_el2, mdcr_el2); - write_sysreg(HCR_HOST_NVHE_FLAGS, hcr_el2); + if (is_protected_kvm_enabled()) + write_sysreg(HCR_HOST_NVHE_PROTECTED_FLAGS, hcr_el2); + else + write_sysreg(HCR_HOST_NVHE_FLAGS, hcr_el2); write_sysreg(CPTR_EL2_DEFAULT, cptr_el2); write_sysreg(__kvm_hyp_host_vector, vbar_el2); } From patchwork Wed Dec 2 18:41:22 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11946829 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6C821C71155 for ; Wed, 2 Dec 2020 18:51:24 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D152C2173E for ; Wed, 2 Dec 2020 18:51:23 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D152C2173E Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=JtV2qbjPRq6L5R6k183C5Sg1U24zen8YwYjxD/IbdwA=; b=SsaP7UjspmS0uHm0MwG3oK8aG XFtmz3JEtLorajSAch2J390ZYW1k5n2yHEoIXHdZ8FJz7fjrnrzTIkQBZJ9CffJtAeawPjtQmw0sA 3IggO9ffLBAhKrkm5aaeJLtbDW1v4kL3iDGuWYT/eK6anhFqvYY/Kk+7BWwhMnLnI/+Ukyr6vimg/ LOdRDLpY3mJedtt7PaU6n59TY1B85bQlXT5QMyJAa42OPRVBf//NrlubnIqVeXh+fEO48CEgAI8ju imwBbxxMyECtV1aPOhWz2+cq4214IGCBMIcRTGu+XYxPM/k9jyFhLPjgCGkpanKvVfF3BspaeLXVe mjVkbScAw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkXAw-0003TI-80; Wed, 02 Dec 2020 18:48:26 +0000 Received: from mail-wm1-x341.google.com ([2a00:1450:4864:20::341]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kkX5A-0008WO-A7 for linux-arm-kernel@lists.infradead.org; Wed, 02 Dec 2020 18:42:31 +0000 Received: by mail-wm1-x341.google.com with SMTP id f190so11194844wme.1 for ; Wed, 02 Dec 2020 10:42:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=26Ee3tSf6bjWmO+pbW0InldfONK6QPnrCLx1QjKfx/Q=; b=Nu9UWjvhl68YCUsjhbXmp2EFwXOEzYW/jis6nLNCXn/NxJfU7CpOTssijMFw+UsPpd 7oltSpuibe/iHjGG2OpFQ4w4pD1qsmbBGRvFmjW2xuUn2JkuWyVwcg85RRjAp9tIXaTK aP2gAH/bBPYSxRqNlX8Plyl8Gky8pnctQ4ag/a3FWSbjY3S0k6kIfS78g+wEMf0OxlFk v65pBZ5UKBi8p2sRqdfPUqIc6NexyNIU1+i7sK09E/pPGsQ7wP233KAe1++CSZzOLErg 61rDJ75t8vNUKCFB/8Anz2tYqOdXv+9jzvHcdxm4tMFqEUh2BjkE8mApOYf3gCbWJHkp jjPw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=26Ee3tSf6bjWmO+pbW0InldfONK6QPnrCLx1QjKfx/Q=; b=bXTRaNZkp7M9eWsNMvS6hD4tlCFaROtNDA3Uwd5zkWwQka4+MIUULNgdm4NBU7aSig 5rnkDVhSPy/ABzW3OOh/syQbBnZGAVyBHWU/TTLdCTNbJGqYbw0RI0zcM8nr7C6klvva mJkG+yTDFCexjtMyUzJUiwVVZIIdIjHRqQoZmehOPL4ZnUqESgwhK53GKgyaftM3EmCE 7494unKwL4P4qje24uOSKdYn4dGzzsezCX/38Z5VNmoMD8akxG+PYjRJ70Z+DmBq2rTi htnElbiWDNu6nbn5zdVZjvNQG5vpZk933Uirz3uS41Sch1kgNpjawlQXY++cT6r2wOYp 88bQ== X-Gm-Message-State: AOAM531WuuHaAQs9c+EvuYuhIq/L0GkSfClF0AD1mQjnTIlRlZFU39ku ZDkS/kQhuDc0PqZKziJPd1Xj1g== X-Google-Smtp-Source: ABdhPJxp6dGVemgWiRIwNDXC1BFMQeJwThH7w+RQVGXX/e/0zonlc4l4p1zCzJtr2NzkM2SO9rchMQ== X-Received: by 2002:a1c:220a:: with SMTP id i10mr4451281wmi.93.1606934546855; Wed, 02 Dec 2020 10:42:26 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:5ddd:b7c5:e3c9:e87a]) by smtp.gmail.com with ESMTPSA id q12sm3056403wmc.45.2020.12.02.10.42.25 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 02 Dec 2020 10:42:25 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v4 26/26] kvm: arm64: Fix EL2 mode availability checks Date: Wed, 2 Dec 2020 18:41:22 +0000 Message-Id: <20201202184122.26046-27-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201202184122.26046-1-dbrazdil@google.com> References: <20201202184122.26046-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201202_134229_433042_09FBAFA6 X-CRM114-Status: GOOD ( 19.29 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org With protected nVHE hyp code interception host's PSCI SMCs, the host starts seeing new CPUs boot in EL1 instead of EL2. The kernel logic that keeps track of the boot mode needs to be adjusted. Add a static key enabled if KVM protected mode initialization is successful. When the key is enabled, is_hyp_mode_available continues to report `true` because its users either treat it as a check whether KVM will be / was initialized, or whether stub HVCs can be made (eg. hibernate). is_hyp_mode_mismatched is changed to report `false` when the key is enabled. That's because all cores' modes matched at the point of KVM init and KVM will not allow cores not present at init to boot. That said, the function is never used after KVM is initialized. Signed-off-by: David Brazdil --- arch/arm64/include/asm/virt.h | 18 ++++++++++++++++++ arch/arm64/kvm/arm.c | 9 ++++++--- 2 files changed, 24 insertions(+), 3 deletions(-) diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h index eb81dcc220b6..ee6a48df89d9 100644 --- a/arch/arm64/include/asm/virt.h +++ b/arch/arm64/include/asm/virt.h @@ -65,9 +65,19 @@ extern u32 __boot_cpu_mode[2]; void __hyp_set_vectors(phys_addr_t phys_vector_base); void __hyp_reset_vectors(void); +DECLARE_STATIC_KEY_FALSE(kvm_protected_mode_initialized); + /* Reports the availability of HYP mode */ static inline bool is_hyp_mode_available(void) { + /* + * If KVM protected mode is initialized, all CPUs must have been booted + * in EL2. Avoid checking __boot_cpu_mode as CPUs now come up in EL1. + */ + if (IS_ENABLED(CONFIG_KVM) && + static_branch_likely(&kvm_protected_mode_initialized)) + return true; + return (__boot_cpu_mode[0] == BOOT_CPU_MODE_EL2 && __boot_cpu_mode[1] == BOOT_CPU_MODE_EL2); } @@ -75,6 +85,14 @@ static inline bool is_hyp_mode_available(void) /* Check if the bootloader has booted CPUs in different modes */ static inline bool is_hyp_mode_mismatched(void) { + /* + * If KVM protected mode is initialized, all CPUs must have been booted + * in EL2. Avoid checking __boot_cpu_mode as CPUs now come up in EL1. + */ + if (IS_ENABLED(CONFIG_KVM) && + static_branch_likely(&kvm_protected_mode_initialized)) + return false; + return __boot_cpu_mode[0] != __boot_cpu_mode[1]; } diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 5f8776eefe46..6e637d2b4cfb 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -47,6 +47,7 @@ __asm__(".arch_extension virt"); #endif static enum kvm_mode kvm_mode = KVM_MODE_DEFAULT; +DEFINE_STATIC_KEY_FALSE(kvm_protected_mode_initialized); DECLARE_KVM_HYP_PER_CPU(unsigned long, kvm_hyp_vector); @@ -1926,12 +1927,14 @@ int kvm_arch_init(void *opaque) if (err) goto out_hyp; - if (is_protected_kvm_enabled()) + if (is_protected_kvm_enabled()) { + static_branch_enable(&kvm_protected_mode_initialized); kvm_info("Protected nVHE mode initialized successfully\n"); - else if (in_hyp_mode) + } else if (in_hyp_mode) { kvm_info("VHE mode initialized successfully\n"); - else + } else { kvm_info("Hyp mode initialized successfully\n"); + } return 0;