From patchwork Thu Nov 26 15:53:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11934277 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D47EDC63697 for ; Thu, 26 Nov 2020 15:56:02 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7914F21D40 for ; Thu, 26 Nov 2020 15:56:02 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="MOBhOzHb"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="q0DXHENI" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7914F21D40 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=J0WAyU7AMdn3hJ18/SM9hxznKMXPg7MAiDmX9lO1x7M=; b=MOBhOzHb4oKMSmrm5/J570b2g ulLXino2k6DUBGwfSYM4BHhE8uY8DN7NuZHKZTNJgttu2MeiyX91J0D7oaKvk54Ydjv8uRk21rFV2 UnCm1OzJTU9tpMpwbHA6lbka6BA2xoJxIYH4/j32vcUWd7cSZHkHn8lnkCk1AT7LQ2qmNIrIRJDLB JnhM7KsS9Rd1OZ2aIahopsKo6CF9OmNFjfAZzGyJJi2Zv3nDz0K8OSqOwfiJP0Er4FjvY408kmry6 NZaMOjtZ9rKsXI3035ZPU+DCRuulohtcXBIa79oqi2W1IhxnlmjtduXxjwjvjkFMXwO0WfBBrfc4m fNa7Et9hA==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kiJbP-0007fb-Bm; Thu, 26 Nov 2020 15:54:35 +0000 Received: from mail-wr1-x444.google.com ([2a00:1450:4864:20::444]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kiJbJ-0007dt-SA for linux-arm-kernel@lists.infradead.org; Thu, 26 Nov 2020 15:54:31 +0000 Received: by mail-wr1-x444.google.com with SMTP id u12so2662865wrt.0 for ; Thu, 26 Nov 2020 07:54:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9eBkN6lmfdOZ6y3T4AeVQ2xEMCxfbxxoIUc76Qx59Gw=; b=q0DXHENIOaqhJFnVso9J1kN3Eb61iVK8nd3AricHajirGlfnSXv+NmgwWHCMVIELoM /xkucGICwySq83LXla7dhpUSyhhrYAX2WyLB2M0GSO+2LUD2XI2l+m2AypGqvHH1oZWj ar44f6d4mUGekt4sM2d7YlccV/6w5dO+75q5vS8ABmkgJxt4nsK1E5YYxMSRXHZwtqRw CgRzDUSpetP2+yE0F4UC58Vh56PRAdl/j8zQgvJ3gdi0B97IexPJS8ZDk2Z0gcqlwvRq +6NN5jRL2/yL9KuYo3aGBfcKHp1j9xVMqkzoFKdL7JrD/vM8UlDrwr99kKCpPxKPalu6 8LXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9eBkN6lmfdOZ6y3T4AeVQ2xEMCxfbxxoIUc76Qx59Gw=; b=T6q+3eUgNmZ03nEwwQ03QOcbCH4i8VHDHVVc2WxuqasBLssqb5loXranFmJW3J26vE l2/QTVKGCfMcWOcxbjaa30nzH9CYvZ92DgDCcEVfWjVaSZnacmVhESArikUOLZnQvTwE R4MXh6z4jkK61ocwLzXi1n/RyK5SyWbBkZ70yKd3qwG72wJ2nAnEXtvEKsUuSn6m4tHn r2nxHuwm7JogXXssMJiJ26H3vABy4yIWRCWEYSkCnGwTXRkPNXzEvW/6ZqThQN1Uij6+ 09D/UGOxgMjdudDviAcvP03v52LwDZbSrvwfltoqZOpxL3suSnfaNNS9cGCIb0avZKtU 7eEA== X-Gm-Message-State: AOAM5339iendNa+7IkuLhgi2qiw0LMcH+ihDBYW+ks2qIjOUHsWXOYr0 hoo/LkHG1Ih5EDEfIL279tM3YQ== X-Google-Smtp-Source: ABdhPJw/lRW3PEMxi02nhLES66F/ltyrof/LoT+FjVRp6ybB+E4igrjSX8c/nxUUNknF9HL5GfidZA== X-Received: by 2002:a5d:5482:: with SMTP id h2mr4667449wrv.18.1606406068612; Thu, 26 Nov 2020 07:54:28 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:f008:704d:8d4b:9951]) by smtp.gmail.com with ESMTPSA id v7sm2637615wma.26.2020.11.26.07.54.27 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 26 Nov 2020 07:54:27 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v3 01/23] psci: Support psci_ops.get_version for v0.1 Date: Thu, 26 Nov 2020 15:53:59 +0000 Message-Id: <20201126155421.14901-2-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201126155421.14901-1-dbrazdil@google.com> References: <20201126155421.14901-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201126_105429_973221_282AAB6D X-CRM114-Status: GOOD ( 15.53 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org KVM's host PSCI SMC filter needs to be aware of the PSCI version of the system but currently it is impossible to distinguish between v0.1 and PSCI disabled because both have get_version == NULL. Populate get_version for v0.1 with a function that returns a constant. psci_opt.get_version is currently unused so this has no effect on existing functionality. Signed-off-by: David Brazdil Acked-by: Mark Rutland --- drivers/firmware/psci/psci.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/drivers/firmware/psci/psci.c b/drivers/firmware/psci/psci.c index 00af99b6f97c..213c68418a65 100644 --- a/drivers/firmware/psci/psci.c +++ b/drivers/firmware/psci/psci.c @@ -146,6 +146,11 @@ static int psci_to_linux_errno(int errno) return -EINVAL; } +static u32 psci_get_version_0_1(void) +{ + return PSCI_VERSION(0, 1); +} + static u32 psci_get_version(void) { return invoke_psci_fn(PSCI_0_2_FN_PSCI_VERSION, 0, 0, 0); @@ -514,6 +519,8 @@ static int __init psci_0_1_init(struct device_node *np) pr_info("Using PSCI v0.1 Function IDs from DT\n"); + psci_ops.get_version = psci_get_version_0_1; + if (!of_property_read_u32(np, "cpu_suspend", &id)) { psci_function_id[PSCI_FN_CPU_SUSPEND] = id; psci_ops.cpu_suspend = psci_cpu_suspend; From patchwork Thu Nov 26 15:54:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11934297 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 71AD3C56202 for ; Thu, 26 Nov 2020 15:56:21 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0E28E21741 for ; Thu, 26 Nov 2020 15:56:21 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="ljAphfP6"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="RIxckPoe" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0E28E21741 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=THISnnf0RPX893I9IvXBb4OlqztzUKYGwpmyM0jam0A=; b=ljAphfP6aaI0JQfY34aA83IM9 BrdwvVmxrCtjJJsz3B18HfAERzeIf+SMz5b9nF4lxyaG59wiwPcvRS+f0R2nP5ztvDoeD4clD+yHS j6+VRWqDVPCWzV4/Fgb0FCqRsUfQobePe/yonnQsZag/Gb+9Hiws35Xi0SOrp3SyRwd/atAe9B/Ae sLTS1puNpcoeb7pzrsWhfdAP5VaLR+iFZeENevBQdbfD4B60OWqh7EMOt3DPcDP0+mUDr9III5fnA Y6R82spshOpewQN8vXga2iQPCggNNAnn45sveSeffH2xxbjpuUXyAvuJm3SI425vUTyXGeXMvZmZF CIl2oe+qw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kiJbc-0007jK-LM; Thu, 26 Nov 2020 15:54:48 +0000 Received: from mail-wm1-x344.google.com ([2a00:1450:4864:20::344]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kiJbM-0007ee-44 for linux-arm-kernel@lists.infradead.org; Thu, 26 Nov 2020 15:54:32 +0000 Received: by mail-wm1-x344.google.com with SMTP id c198so2578717wmd.0 for ; Thu, 26 Nov 2020 07:54:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=GRbbAoAyT+twJ6DWAMnVPh0yMfbGrEWPbKn8Cs93LQs=; b=RIxckPoe2SCexeONFtE4DSkvr9AstiWpnehcv0VL5bhqQYEg6F/oOLEITVaQ1YoEfP hGk3BAB/mFs/TVGkOysWFzWL2o2OJ63MAD9vHn6ziIvpoT6iU7Kp+f3yRtPmRLGrK3uL 0deZiq22+yVHN3uu+0OcZirO1XVk48C0r0exGjyDri75/b+No5iT5S4xp4Jb/F2siD20 9mbZCjgyOtXgQaVY+Md/r07NG7gzIABxytKiAfgC6hHFqf47OWj1AJA/CygIdN1Nz3RG QjClGrsnaR32TvHvOZhfrzbfM8Kl2yeEXRKWMhh3KYMG0WT+diIy8djka0sD24s2czFl LSkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=GRbbAoAyT+twJ6DWAMnVPh0yMfbGrEWPbKn8Cs93LQs=; b=cAR+NKz0/uq2j+F6UqOKHitxjCE0kOZnt/lEoGglMfDNxRZcayddZdnHKSW62Bikfs tOFVJF/O4oNAdEg9hwuk3aXWPVYdl0mNb5fn+Pn3u0nQoknnDf7Xm9ApPdIe9FJYAEd4 l3PO8huLkgnmMJU4iIsSd/wtp0V6abGe4TelY5BAPPHgqYkLn6XzKIBrECj2quUkGQQw 13Qgo1FybhFZ6Wo0zrKq2vVQpN5U0kp4KFSx85AnJE+bF62zjXUtlC0jxW4csJlm6Dzy 4vbAiTVnFFqVADRaR3OUUU07Ju1YLrClsa7ktrajh2O+q4hYBOwmvPejNxsrEe7hIicl rDFQ== X-Gm-Message-State: AOAM531pRFRDKSRDD0UVEIcjowNGLATAnlR7+dUz655VoqtKA9clcqrM WhpYOd3iYpENIsEUYTgNTuJuVg== X-Google-Smtp-Source: ABdhPJwT9SLDB2L8OAgTeM8M2ZPVm2Y3TUfSJCQdJPrpoyLr/4gGDkZXGYwQZBc52TdBQc78XLbyjQ== X-Received: by 2002:a1c:31c6:: with SMTP id x189mr4071224wmx.50.1606406070792; Thu, 26 Nov 2020 07:54:30 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:f008:704d:8d4b:9951]) by smtp.gmail.com with ESMTPSA id q1sm9612864wrj.8.2020.11.26.07.54.29 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 26 Nov 2020 07:54:29 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v3 02/23] psci: Accessor for configured PSCI function IDs Date: Thu, 26 Nov 2020 15:54:00 +0000 Message-Id: <20201126155421.14901-3-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201126155421.14901-1-dbrazdil@google.com> References: <20201126155421.14901-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201126_105432_234832_6E3E491F X-CRM114-Status: GOOD ( 17.00 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Function IDs used by PSCI are configurable for v0.1 via DT/APCI. If the host is using PSCI v0.1, KVM's host PSCI proxy needs to use the same IDs. Expose the array holding the information with a read-only accessor. Signed-off-by: David Brazdil --- drivers/firmware/psci/psci.c | 16 ++++++++-------- include/linux/psci.h | 10 ++++++++++ 2 files changed, 18 insertions(+), 8 deletions(-) diff --git a/drivers/firmware/psci/psci.c b/drivers/firmware/psci/psci.c index 213c68418a65..40609564595e 100644 --- a/drivers/firmware/psci/psci.c +++ b/drivers/firmware/psci/psci.c @@ -58,16 +58,16 @@ typedef unsigned long (psci_fn)(unsigned long, unsigned long, unsigned long, unsigned long); static psci_fn *invoke_psci_fn; -enum psci_function { - PSCI_FN_CPU_SUSPEND, - PSCI_FN_CPU_ON, - PSCI_FN_CPU_OFF, - PSCI_FN_MIGRATE, - PSCI_FN_MAX, -}; - static u32 psci_function_id[PSCI_FN_MAX]; +u32 psci_get_function_id(enum psci_function fn) +{ + if (WARN_ON_ONCE(fn < 0 || fn >= PSCI_FN_MAX)) + return 0; + + return psci_function_id[fn]; +} + #define PSCI_0_2_POWER_STATE_MASK \ (PSCI_0_2_POWER_STATE_ID_MASK | \ PSCI_0_2_POWER_STATE_TYPE_MASK | \ diff --git a/include/linux/psci.h b/include/linux/psci.h index 2a1bfb890e58..5b49a5c82d6f 100644 --- a/include/linux/psci.h +++ b/include/linux/psci.h @@ -21,6 +21,16 @@ bool psci_power_state_is_valid(u32 state); int psci_set_osi_mode(bool enable); bool psci_has_osi_support(void); +enum psci_function { + PSCI_FN_CPU_SUSPEND, + PSCI_FN_CPU_ON, + PSCI_FN_CPU_OFF, + PSCI_FN_MIGRATE, + PSCI_FN_MAX, +}; + +u32 psci_get_function_id(enum psci_function fn); + struct psci_operations { u32 (*get_version)(void); int (*cpu_suspend)(u32 state, unsigned long entry_point); From patchwork Thu Nov 26 15:54:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11934301 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5F647C56202 for ; Thu, 26 Nov 2020 15:56:32 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 157BC21D1A for ; Thu, 26 Nov 2020 15:56:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="lQZjmjw2"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="WB708Y6Z" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 157BC21D1A Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=ePHVXeV9v2pHvpzSIi3okMBKkj3+FLvVId9ZFfvnt4E=; b=lQZjmjw2eaEP+nNaCzKYxM5Rj T9TrgDDGAapDW+4M1XwOcHHN5N2ID9V23oaVlp+4tOPuQNmyEnQqbfoijeIaccnyN6erkDYVeky7g tgQ7GZrbPLQ2fHNmhEBMJPaSQ6btLXtbH0owukblJNg8B/tiMXaFSsaEowcm86vED3z5NwLNP5lz9 iMWOQL8W5D3tXQQrtvYhuSt9ZuQg/4c+m1EKW1NO+pDSjXowvJMRvOs3MDWtP9LQ1CSkUasMO1Vs/ /w9Dm0vPDih72lWvqoQskCp6H49km5/HLz+Cc7HfbvmtdCLjDv+kQkOV4b38Fvy4tixrxuM9zvD0I Hxs/Zmcpw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kiJbi-0007kq-QC; Thu, 26 Nov 2020 15:54:54 +0000 Received: from mail-wm1-x343.google.com ([2a00:1450:4864:20::343]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kiJbO-0007fI-MN for linux-arm-kernel@lists.infradead.org; Thu, 26 Nov 2020 15:54:36 +0000 Received: by mail-wm1-x343.google.com with SMTP id a186so2569146wme.1 for ; Thu, 26 Nov 2020 07:54:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=t3ZEYk2Uf0v1klDyIVNllC4hj8a8TzhOU5HlrW3d3XY=; b=WB708Y6ZQsVTnPDe3k8SGt1N+Zi1ArG5DNJ8Ikjx+GwIsVyKkyANitm+7v9D8STbCN 4sIx6XZ9WxtYX8XP7V/bVY64iZPta0oEQJhY5JctNUzEqp/siO8Mh7MOaSHI9jTKRX/G ba18CMjyRpguJUKBwLVRI8k5AHvlNVRnB1DKj5K2COahoTLJrC8pp74gWQmVJtRJ8ivp fJffW03dFEtg1jOc/x7+J4NtCy/6j6euZgUM7vcZ5okmBEc+hu9CLbAHOr83RhZZ7PAd Ya9qvceAHvZJLaq5R4L/3OTpAnWZot8Djf5EA45iNDBN22H7c4JKiI1EzSkX09tYRLsS Ahyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=t3ZEYk2Uf0v1klDyIVNllC4hj8a8TzhOU5HlrW3d3XY=; b=Mfd5zrikdgMyOOTwCH8ciK/vPy3aHb8gDyaRECbNd56t+BN21YjHUnQp7Sxl/H299T wW0kCat50+/MmsolU9FaL9TAEuPa0xzS8ZxuUIc/ISXnDXXz2ZTn3tklq4ou7A3xmPDR hAizJlG6H+udi5JJ5qO9Ub+lCRCN7q+9VOKRHfD0EUIpVglc7MC0EeRLl8PcdIrTrHGe khnYt1mWTsPGeNawAMILGkQqZmK1VvyjMuK742VoP1VEmGCQr9rTGuokhslpwLbvpRVD uKEvlHlkDvX+d9q2bOWHtnOF6IO2pqxG2Jm1sgB73FreSGWg9YDcRuFlv2Wf0fy+7Uyx ZN8Q== X-Gm-Message-State: AOAM5318BsnVZ5Po0M4kaySUqQarNGm9rGgqbWjVRg72xIForZ1Syml4 4tkaFZT3QG1Fx9QkC7aaLum8DA== X-Google-Smtp-Source: ABdhPJwqAqu5fiW6MkcfS54w+9X9od3V8YYUykJ0+y44LSiV/rX3jrG5iDECe1WB8DPMFJnBgkCsIA== X-Received: by 2002:a1c:3b07:: with SMTP id i7mr4113356wma.118.1606406072986; Thu, 26 Nov 2020 07:54:32 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:f008:704d:8d4b:9951]) by smtp.gmail.com with ESMTPSA id d8sm8607196wmb.11.2020.11.26.07.54.31 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 26 Nov 2020 07:54:32 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v3 03/23] arm64: Make cpu_logical_map() take unsigned int Date: Thu, 26 Nov 2020 15:54:01 +0000 Message-Id: <20201126155421.14901-4-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201126155421.14901-1-dbrazdil@google.com> References: <20201126155421.14901-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201126_105434_823465_86A04B6C X-CRM114-Status: GOOD ( 14.84 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org CPU index should never be negative. Change the signature of (set_)cpu_logical_map to take an unsigned int. Signed-off-by: David Brazdil --- arch/arm64/include/asm/smp.h | 4 ++-- arch/arm64/kernel/setup.c | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/arch/arm64/include/asm/smp.h b/arch/arm64/include/asm/smp.h index 2e7f529ec5a6..bcb01ca15325 100644 --- a/arch/arm64/include/asm/smp.h +++ b/arch/arm64/include/asm/smp.h @@ -46,9 +46,9 @@ DECLARE_PER_CPU_READ_MOSTLY(int, cpu_number); * Logical CPU mapping. */ extern u64 __cpu_logical_map[NR_CPUS]; -extern u64 cpu_logical_map(int cpu); +extern u64 cpu_logical_map(unsigned int cpu); -static inline void set_cpu_logical_map(int cpu, u64 hwid) +static inline void set_cpu_logical_map(unsigned int cpu, u64 hwid) { __cpu_logical_map[cpu] = hwid; } diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c index 133257ffd859..2f2973bc67c7 100644 --- a/arch/arm64/kernel/setup.c +++ b/arch/arm64/kernel/setup.c @@ -276,7 +276,7 @@ arch_initcall(reserve_memblock_reserved_regions); u64 __cpu_logical_map[NR_CPUS] = { [0 ... NR_CPUS-1] = INVALID_HWID }; -u64 cpu_logical_map(int cpu) +u64 cpu_logical_map(unsigned int cpu) { return __cpu_logical_map[cpu]; } From patchwork Thu Nov 26 15:54:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11934303 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0D257C56202 for ; Thu, 26 Nov 2020 15:57:00 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 355B221D46 for ; Thu, 26 Nov 2020 15:56:59 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="28OHSyip"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="OchURbw1" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 355B221D46 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=7Zz2bYgIH0T2hGxGwVP9ZiLBHgUH7KmNj9AJP1NMqok=; b=28OHSyipx2PmragMyAyqJGtIs TqrCzFAZJy4A+1uh5vw85QFfevMgFqf2KGceJCQ2B99Vle3FDHeJvKYR/lzUCI57VBuoEO9NkA0YQ PsZ3m4D0iBiee4VV4+hjSEEK3YmgW6CXkmmlJMoQj8WGI1nj+bsd2FRv9b2Ujr5vqLPaLebA8hK1H siH/gpJUein6fmYunsIDn7qfhQAGb5+SUtrFstAIwncjZKsua+kHslDkV6S+CNVy1hWNzTUZqpyim URQnqDAALH1olgRnjJq9PXy/EGiClBerwLUga4O+ukugNOpLi9WK1JKgOgW5JaVDEdIfvxpfMn7Zc 221QBd4Ew==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kiJc4-0007sh-CA; Thu, 26 Nov 2020 15:55:16 +0000 Received: from mail-wm1-x344.google.com ([2a00:1450:4864:20::344]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kiJbQ-0007fy-8b for linux-arm-kernel@lists.infradead.org; Thu, 26 Nov 2020 15:54:38 +0000 Received: by mail-wm1-x344.google.com with SMTP id w24so2878989wmi.0 for ; Thu, 26 Nov 2020 07:54:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=HHdsvDKCV+y2T0WHPFGAdN0qhY1OjBTJULnU1VX59iA=; b=OchURbw1mLwFNtApGpEjM6AKBpDfQt9A0SnaSO/IvElpZFNXclLui4HRql7+PGhPzu w04fOAsLJg76P6WZjX/BoPa8t98z/zCXTtsh3+9RM29/ABXswF3vZehbRj8WIt/AgLtP L1k4SXy6vkelZAc1vVbqeOd+r1oeZei7wOPlFer6IGR4IFM5MXaOg8/8g9/rQhagB3ka dpXN3Srhy8SuzemOK5jvdLk4cjfBovVom/uweowmjaI38hHlgTpMbjl2MDXmVHPXSN/S fbBM51yzh1M2Eluao4XM9cf6bSCqn1yHyxPe1V86PBl5py1rObz13Ko5gXZLpnmXc+U2 2dpA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HHdsvDKCV+y2T0WHPFGAdN0qhY1OjBTJULnU1VX59iA=; b=uMyz3Y2ups6/eLnWmh/kIfv2Q43hy2IekpwXhi20saRAxESI9ncj+DpLKogZD1ojMd rWl7fXEQFG2cDjGkXjLz//2MT0FZ+cUx+/RuOtq31/Mf3cItp60oEMahWYRwPdusqq7K ixkb58zbu0xbgIsII8/b0zO4OpRFVujTD9VuPCs0004uuOx8NdOSHfxwm/QQkWrrOzmp F3seUcH6L8aV39zL/6Tf6u0+iGFBga7ifc5YlJ/59YjJjKmk7xO+y8WwvANFHLi6Kyp5 /CMwAIaeWDnjaF+qBGQwgetoYVFIJ25q1nUscFJjfOvKTtt8FjwrA6dQzlePGw8alBPN iJZg== X-Gm-Message-State: AOAM532zoIuaUZ1v/MVXxhJpfIWZUqH1m07RKA4nr3Y2cZMyATIBKEvQ dPH8md/lkDGCUt9xONerocK1qw== X-Google-Smtp-Source: ABdhPJxLdnrPvl0zzUmNu9CSpqRHeZvuF+rg/YRWgtrj7i//XJcOslWwSsG2VuZ6GKRjiQHqNZNJ7A== X-Received: by 2002:a7b:cb58:: with SMTP id v24mr4021372wmj.23.1606406075145; Thu, 26 Nov 2020 07:54:35 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:f008:704d:8d4b:9951]) by smtp.gmail.com with ESMTPSA id a144sm9761056wmd.47.2020.11.26.07.54.33 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 26 Nov 2020 07:54:34 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v3 04/23] arm64: Move MAIR_EL1_SET to asm/memory.h Date: Thu, 26 Nov 2020 15:54:02 +0000 Message-Id: <20201126155421.14901-5-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201126155421.14901-1-dbrazdil@google.com> References: <20201126155421.14901-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201126_105436_405296_212E9739 X-CRM114-Status: GOOD ( 15.59 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org KVM currently initializes MAIR_EL2 to the value of MAIR_EL1. In preparation for initializing MAIR_EL2 before MAIR_EL1, move the constant into a shared header file. Since it is used for EL1 and EL2, rename to MAIR_ELx_SET. Signed-off-by: David Brazdil --- arch/arm64/include/asm/memory.h | 13 +++++++++++++ arch/arm64/mm/proc.S | 15 +-------------- 2 files changed, 14 insertions(+), 14 deletions(-) diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index cd61239bae8c..54a22cb5b17b 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -152,6 +152,19 @@ #define MT_S2_FWB_NORMAL 6 #define MT_S2_FWB_DEVICE_nGnRE 1 +/* + * Default MAIR_ELx. MT_NORMAL_TAGGED is initially mapped as Normal memory and + * changed during __cpu_setup to Normal Tagged if the system supports MTE. + */ +#define MAIR_ELx_SET \ + (MAIR_ATTRIDX(MAIR_ATTR_DEVICE_nGnRnE, MT_DEVICE_nGnRnE) | \ + MAIR_ATTRIDX(MAIR_ATTR_DEVICE_nGnRE, MT_DEVICE_nGnRE) | \ + MAIR_ATTRIDX(MAIR_ATTR_DEVICE_GRE, MT_DEVICE_GRE) | \ + MAIR_ATTRIDX(MAIR_ATTR_NORMAL_NC, MT_NORMAL_NC) | \ + MAIR_ATTRIDX(MAIR_ATTR_NORMAL, MT_NORMAL) | \ + MAIR_ATTRIDX(MAIR_ATTR_NORMAL_WT, MT_NORMAL_WT) | \ + MAIR_ATTRIDX(MAIR_ATTR_NORMAL, MT_NORMAL_TAGGED)) + #ifdef CONFIG_ARM64_4K_PAGES #define IOREMAP_MAX_ORDER (PUD_SHIFT) #else diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index 23c326a06b2d..e3b9aa372b96 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -45,19 +45,6 @@ #define TCR_KASAN_FLAGS 0 #endif -/* - * Default MAIR_EL1. MT_NORMAL_TAGGED is initially mapped as Normal memory and - * changed during __cpu_setup to Normal Tagged if the system supports MTE. - */ -#define MAIR_EL1_SET \ - (MAIR_ATTRIDX(MAIR_ATTR_DEVICE_nGnRnE, MT_DEVICE_nGnRnE) | \ - MAIR_ATTRIDX(MAIR_ATTR_DEVICE_nGnRE, MT_DEVICE_nGnRE) | \ - MAIR_ATTRIDX(MAIR_ATTR_DEVICE_GRE, MT_DEVICE_GRE) | \ - MAIR_ATTRIDX(MAIR_ATTR_NORMAL_NC, MT_NORMAL_NC) | \ - MAIR_ATTRIDX(MAIR_ATTR_NORMAL, MT_NORMAL) | \ - MAIR_ATTRIDX(MAIR_ATTR_NORMAL_WT, MT_NORMAL_WT) | \ - MAIR_ATTRIDX(MAIR_ATTR_NORMAL, MT_NORMAL_TAGGED)) - #ifdef CONFIG_CPU_PM /** * cpu_do_suspend - save CPU registers context @@ -425,7 +412,7 @@ SYM_FUNC_START(__cpu_setup) /* * Memory region attributes */ - mov_q x5, MAIR_EL1_SET + mov_q x5, MAIR_ELx_SET #ifdef CONFIG_ARM64_MTE /* * Update MAIR_EL1, GCR_EL1 and TFSR*_EL1 if MTE is supported From patchwork Thu Nov 26 15:54:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11934305 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B29CFC56202 for ; Thu, 26 Nov 2020 15:57:27 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 310F121D40 for ; Thu, 26 Nov 2020 15:57:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="zx6ChVB5"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="o0AAoClP" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 310F121D40 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=jqHwuZGggvnLZmUlo/Fd4VCXQ0ZUZ/1yqAlNT21oMxk=; b=zx6ChVB5GwaVOV73IX98HriIP 38J4bO3uiAyzaHeM49B3WYXIa6C9Rdsbyw7VEI0xEGn4+yCRAJYk7KsJ2JftAvAQ5djGlO16Y/J/N IifrVQ9eVe+7DrJYQbLTSN+pukk8BmiTlGRtxnKZY9KrHHNKervpz1VwSK8OvPPTWZjM9Y09REtJ+ JDD9JBN6DIJQBaJkznZx6ki7kipNaO0udJ68CiW5gU5syUwoIxuR030ITHkiRsl+WRpWZam1lRSvj jP2gUwpS3bIo9vMbiJKeLMSPH+YwqqGUSEHXF48UsN3v6/XA31KNR9QefhMzqMW/ewdrHzvRuxrvl tMPtvnW/A==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kiJcN-00080N-Fg; Thu, 26 Nov 2020 15:55:35 +0000 Received: from mail-wr1-x441.google.com ([2a00:1450:4864:20::441]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kiJbS-0007gX-Mr for linux-arm-kernel@lists.infradead.org; Thu, 26 Nov 2020 15:54:40 +0000 Received: by mail-wr1-x441.google.com with SMTP id 64so2616107wra.11 for ; Thu, 26 Nov 2020 07:54:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=nvwTQNNZtO31OTVc289HlSj6IfrKQr68R/EeTn2wLTs=; b=o0AAoClPZbIUCVP2EtKok8cJTqxeoawIX1Wd9YfAPAFuA8F2TCi0PItChvL7vxjiHn OzZRwBQa+aU0FdKY+sA25K+2IbuMBayARYNwK/9gvWgxptk9OmcmiJIR+vVVrO9ThZh9 MKYVqN+GcVdZuckTXr5oXkGhuIwsTxuanyrRQdDvDnNhk5iKBhRIoZaucAhhhZ+X+frM X5igaqE6Fl7XA5qPJaifUsszlOFo2CxdUEtAAYG8XJoT2vcIa6i+cHrTU9GqVuUg6p1g 2EyD4CkdHEJZdYUO6VpHejlHPxASWrPYQuM7CngFoGZx/48UOBCIZpOZ/HnYkYsyX2Yh P5fg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=nvwTQNNZtO31OTVc289HlSj6IfrKQr68R/EeTn2wLTs=; b=TWoxoZmPMM8WOdlQycCAtxf9cqX8goKTeERB5Q1idZK5QcBKP2LGtCpnrzwIxYSvw1 5T35SMO0saOEEEGkHHRwMftXiJ68hzJ1dYqkRgdVB9Cq0lfmzqsafDw62ZezL2y3shFZ 3oCAheYIPzlskgUplhETt41vrsNvfM0OhT4+WlJrpsdyjo00/mIUTkE297BHdaUxVbgC LqfztB9fWQa9dWjmYw0MmgvHZKaQoRGwy8xxKbOxInUjr4ftmy2gky7DB+Rgb1blZoGv 0a0/ye3prNC0jvbPfnrIZSpEz0kPxipfNOwZPFRTeLluxD2mwqdHVzdq+Dhscg2jrFpv 1gPw== X-Gm-Message-State: AOAM530EaBxGeQ/iznRR3mPMzjTOoO7XSKjzq5ADC62qs/JqayizpkvJ gJoV2GSEvbrD/hTulr/epEcywQ== X-Google-Smtp-Source: ABdhPJwMxH9eVvxny0iYRpWwzhIl3VMG97fRb8jH6Zk997PRx8L3RSpUnmqsMsequCJqtSeaS0bqXQ== X-Received: by 2002:adf:e502:: with SMTP id j2mr4647857wrm.73.1606406077218; Thu, 26 Nov 2020 07:54:37 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:f008:704d:8d4b:9951]) by smtp.gmail.com with ESMTPSA id t20sm9754665wmi.3.2020.11.26.07.54.35 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 26 Nov 2020 07:54:36 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v3 05/23] arm64: Extract parts of el2_setup into a macro Date: Thu, 26 Nov 2020 15:54:03 +0000 Message-Id: <20201126155421.14901-6-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201126155421.14901-1-dbrazdil@google.com> References: <20201126155421.14901-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201126_105438_889872_8DA7EE58 X-CRM114-Status: GOOD ( 27.68 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When the a CPU is booted in EL2, the kernel checks for VHE support and initializes the CPU core accordingly. For nVHE it also installs the stub vectors and drops down to EL1. Once KVM gains the ability to boot cores without going through the kernel entry point, it will need to initialize the CPU the same way. Extract the relevant bits of el2_setup into an init_el2_state macro with an argument specifying whether to initialize for VHE or nVHE. No functional change. Size of el2_setup increased by 148 bytes due to duplication. Signed-off-by: David Brazdil --- arch/arm64/include/asm/el2_setup.h | 182 +++++++++++++++++++++++++++++ arch/arm64/kernel/head.S | 144 +++-------------------- 2 files changed, 198 insertions(+), 128 deletions(-) create mode 100644 arch/arm64/include/asm/el2_setup.h diff --git a/arch/arm64/include/asm/el2_setup.h b/arch/arm64/include/asm/el2_setup.h new file mode 100644 index 000000000000..b6cfd8aa2a81 --- /dev/null +++ b/arch/arm64/include/asm/el2_setup.h @@ -0,0 +1,182 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2012,2013 - ARM Ltd + * Author: Marc Zyngier + */ + +#ifndef __ARM_KVM_INIT_H__ +#define __ARM_KVM_INIT_H__ + +#ifndef __ASSEMBLY__ +#error Assembly-only header +#endif + +#include +#include +#include +#include + +.macro __init_el2_sctlr + mov_q x0, (SCTLR_EL2_RES1 | ENDIAN_SET_EL2) + msr sctlr_el2, x0 + isb +.endm + +/* + * Allow Non-secure EL1 and EL0 to access physical timer and counter. + * This is not necessary for VHE, since the host kernel runs in EL2, + * and EL0 accesses are configured in the later stage of boot process. + * Note that when HCR_EL2.E2H == 1, CNTHCTL_EL2 has the same bit layout + * as CNTKCTL_EL1, and CNTKCTL_EL1 accessing instructions are redefined + * to access CNTHCTL_EL2. This allows the kernel designed to run at EL1 + * to transparently mess with the EL0 bits via CNTKCTL_EL1 access in + * EL2. + */ +.macro __init_el2_timers mode +.ifeqs "\mode", "nvhe" + mrs x0, cnthctl_el2 + orr x0, x0, #3 // Enable EL1 physical timers + msr cnthctl_el2, x0 +.endif + msr cntvoff_el2, xzr // Clear virtual offset +.endm + +.macro __init_el2_debug mode + mrs x1, id_aa64dfr0_el1 + sbfx x0, x1, #ID_AA64DFR0_PMUVER_SHIFT, #4 + cmp x0, #1 + b.lt 1f // Skip if no PMU present + mrs x0, pmcr_el0 // Disable debug access traps + ubfx x0, x0, #11, #5 // to EL2 and allow access to +1: + csel x2, xzr, x0, lt // all PMU counters from EL1 + + /* Statistical profiling */ + ubfx x0, x1, #ID_AA64DFR0_PMSVER_SHIFT, #4 + cbz x0, 3f // Skip if SPE not present + +.ifeqs "\mode", "nvhe" + mrs_s x0, SYS_PMBIDR_EL1 // If SPE available at EL2, + and x0, x0, #(1 << SYS_PMBIDR_EL1_P_SHIFT) + cbnz x0, 2f // then permit sampling of physical + mov x0, #(1 << SYS_PMSCR_EL2_PCT_SHIFT | \ + 1 << SYS_PMSCR_EL2_PA_SHIFT) + msr_s SYS_PMSCR_EL2, x0 // addresses and physical counter +2: + mov x0, #(MDCR_EL2_E2PB_MASK << MDCR_EL2_E2PB_SHIFT) + orr x2, x2, x0 // If we don't have VHE, then + // use EL1&0 translation. +.else + orr x2, x2, #MDCR_EL2_TPMS // For VHE, use EL2 translation + // and disable access from EL1 +.endif + +3: + msr mdcr_el2, x2 // Configure debug traps +.endm + +/* LORegions */ +.macro __init_el2_lor + mrs x1, id_aa64mmfr1_el1 + ubfx x0, x1, #ID_AA64MMFR1_LOR_SHIFT, 4 + cbz x0, 1f + msr_s SYS_LORC_EL1, xzr +1: +.endm + +/* Stage-2 translation */ +.macro __init_el2_stage2 + msr vttbr_el2, xzr +.endm + +/* GICv3 system register access */ +.macro __init_el2_gicv3 + mrs x0, id_aa64pfr0_el1 + ubfx x0, x0, #ID_AA64PFR0_GIC_SHIFT, #4 + cbz x0, 1f + + mrs_s x0, SYS_ICC_SRE_EL2 + orr x0, x0, #ICC_SRE_EL2_SRE // Set ICC_SRE_EL2.SRE==1 + orr x0, x0, #ICC_SRE_EL2_ENABLE // Set ICC_SRE_EL2.Enable==1 + msr_s SYS_ICC_SRE_EL2, x0 + isb // Make sure SRE is now set + mrs_s x0, SYS_ICC_SRE_EL2 // Read SRE back, + tbz x0, #0, 1f // and check that it sticks + msr_s SYS_ICH_HCR_EL2, xzr // Reset ICC_HCR_EL2 to defaults +1: +.endm + +.macro __init_el2_hstr + msr hstr_el2, xzr // Disable CP15 traps to EL2 +.endm + +/* Virtual CPU ID registers */ +.macro __init_el2_nvhe_idregs + mrs x0, midr_el1 + mrs x1, mpidr_el1 + msr vpidr_el2, x0 + msr vmpidr_el2, x1 +.endm + +/* Coprocessor traps */ +.macro __init_el2_nvhe_cptr + mov x0, #0x33ff + msr cptr_el2, x0 // Disable copro. traps to EL2 +.endm + +/* SVE register access */ +.macro __init_el2_nvhe_sve + mrs x1, id_aa64pfr0_el1 + ubfx x1, x1, #ID_AA64PFR0_SVE_SHIFT, #4 + cbz x1, 1f + + bic x0, x0, #CPTR_EL2_TZ // Also disable SVE traps + msr cptr_el2, x0 // Disable copro. traps to EL2 + isb + mov x1, #ZCR_ELx_LEN_MASK // SVE: Enable full vector + msr_s SYS_ZCR_EL2, x1 // length for EL1. +1: +.endm + +.macro __init_el2_nvhe_prepare_eret + mov x0, #(PSR_F_BIT | PSR_I_BIT | PSR_A_BIT | PSR_D_BIT |\ + PSR_MODE_EL1h) + msr spsr_el2, x0 +.endm + +/** + * Initialize EL2 registers to sane values. This should be called early on all + * cores that were booted in EL2. + * + * Regs: x0, x1 and x2 are clobbered. + */ +.macro init_el2_state mode +.ifnes "\mode", "vhe" +.ifnes "\mode", "nvhe" +.error "Invalid 'mode' argument" +.endif +.endif + + __init_el2_sctlr + __init_el2_timers \mode + __init_el2_debug \mode + __init_el2_lor + __init_el2_stage2 + __init_el2_gicv3 + __init_el2_hstr + + /* + * When VHE is not in use, early init of EL2 needs to be done here. + * When VHE _is_ in use, EL1 will not be used in the host and + * requires no configuration, and all non-hyp-specific EL2 setup + * will be done via the _EL1 system register aliases in __cpu_setup. + */ +.ifeqs "\mode", "nvhe" + __init_el2_nvhe_idregs + __init_el2_nvhe_cptr + __init_el2_nvhe_sve + __init_el2_nvhe_prepare_eret +.endif +.endm + +#endif /* __ARM_KVM_INIT_H__ */ diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index d8d9caf02834..da913ce9e89f 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -11,7 +11,6 @@ #include #include -#include #include #include @@ -21,6 +20,7 @@ #include #include #include +#include #include #include #include @@ -493,159 +493,47 @@ SYM_FUNC_START(el2_setup) mrs x0, CurrentEL cmp x0, #CurrentEL_EL2 b.eq 1f + mov_q x0, (SCTLR_EL1_RES1 | ENDIAN_SET_EL1) msr sctlr_el1, x0 mov w0, #BOOT_CPU_MODE_EL1 // This cpu booted in EL1 isb ret -1: mov_q x0, (SCTLR_EL2_RES1 | ENDIAN_SET_EL2) - msr sctlr_el2, x0 - +1: #ifdef CONFIG_ARM64_VHE /* - * Check for VHE being present. For the rest of the EL2 setup, - * x2 being non-zero indicates that we do have VHE, and that the - * kernel is intended to run at EL2. + * Check for VHE being present. x2 being non-zero indicates that we + * do have VHE, and that the kernel is intended to run at EL2. */ mrs x2, id_aa64mmfr1_el1 ubfx x2, x2, #ID_AA64MMFR1_VHE_SHIFT, #4 -#else - mov x2, xzr -#endif + cbz x2, el2_setup_nvhe - /* Hyp configuration. */ - mov_q x0, HCR_HOST_NVHE_FLAGS - cbz x2, set_hcr mov_q x0, HCR_HOST_VHE_FLAGS -set_hcr: msr hcr_el2, x0 isb - /* - * Allow Non-secure EL1 and EL0 to access physical timer and counter. - * This is not necessary for VHE, since the host kernel runs in EL2, - * and EL0 accesses are configured in the later stage of boot process. - * Note that when HCR_EL2.E2H == 1, CNTHCTL_EL2 has the same bit layout - * as CNTKCTL_EL1, and CNTKCTL_EL1 accessing instructions are redefined - * to access CNTHCTL_EL2. This allows the kernel designed to run at EL1 - * to transparently mess with the EL0 bits via CNTKCTL_EL1 access in - * EL2. - */ - cbnz x2, 1f - mrs x0, cnthctl_el2 - orr x0, x0, #3 // Enable EL1 physical timers - msr cnthctl_el2, x0 -1: - msr cntvoff_el2, xzr // Clear virtual offset - -#ifdef CONFIG_ARM_GIC_V3 - /* GICv3 system register access */ - mrs x0, id_aa64pfr0_el1 - ubfx x0, x0, #ID_AA64PFR0_GIC_SHIFT, #4 - cbz x0, 3f - - mrs_s x0, SYS_ICC_SRE_EL2 - orr x0, x0, #ICC_SRE_EL2_SRE // Set ICC_SRE_EL2.SRE==1 - orr x0, x0, #ICC_SRE_EL2_ENABLE // Set ICC_SRE_EL2.Enable==1 - msr_s SYS_ICC_SRE_EL2, x0 - isb // Make sure SRE is now set - mrs_s x0, SYS_ICC_SRE_EL2 // Read SRE back, - tbz x0, #0, 3f // and check that it sticks - msr_s SYS_ICH_HCR_EL2, xzr // Reset ICC_HCR_EL2 to defaults - -3: -#endif - - /* Populate ID registers. */ - mrs x0, midr_el1 - mrs x1, mpidr_el1 - msr vpidr_el2, x0 - msr vmpidr_el2, x1 - -#ifdef CONFIG_COMPAT - msr hstr_el2, xzr // Disable CP15 traps to EL2 -#endif - - /* EL2 debug */ - mrs x1, id_aa64dfr0_el1 - sbfx x0, x1, #ID_AA64DFR0_PMUVER_SHIFT, #4 - cmp x0, #1 - b.lt 4f // Skip if no PMU present - mrs x0, pmcr_el0 // Disable debug access traps - ubfx x0, x0, #11, #5 // to EL2 and allow access to -4: - csel x3, xzr, x0, lt // all PMU counters from EL1 - - /* Statistical profiling */ - ubfx x0, x1, #ID_AA64DFR0_PMSVER_SHIFT, #4 - cbz x0, 7f // Skip if SPE not present - cbnz x2, 6f // VHE? - mrs_s x4, SYS_PMBIDR_EL1 // If SPE available at EL2, - and x4, x4, #(1 << SYS_PMBIDR_EL1_P_SHIFT) - cbnz x4, 5f // then permit sampling of physical - mov x4, #(1 << SYS_PMSCR_EL2_PCT_SHIFT | \ - 1 << SYS_PMSCR_EL2_PA_SHIFT) - msr_s SYS_PMSCR_EL2, x4 // addresses and physical counter -5: - mov x1, #(MDCR_EL2_E2PB_MASK << MDCR_EL2_E2PB_SHIFT) - orr x3, x3, x1 // If we don't have VHE, then - b 7f // use EL1&0 translation. -6: // For VHE, use EL2 translation - orr x3, x3, #MDCR_EL2_TPMS // and disable access from EL1 -7: - msr mdcr_el2, x3 // Configure debug traps - - /* LORegions */ - mrs x1, id_aa64mmfr1_el1 - ubfx x0, x1, #ID_AA64MMFR1_LOR_SHIFT, 4 - cbz x0, 1f - msr_s SYS_LORC_EL1, xzr -1: - - /* Stage-2 translation */ - msr vttbr_el2, xzr - - cbz x2, install_el2_stub + init_el2_state vhe mov w0, #BOOT_CPU_MODE_EL2 // This CPU booted in EL2 isb ret +#endif -SYM_INNER_LABEL(install_el2_stub, SYM_L_LOCAL) - /* - * When VHE is not in use, early init of EL2 and EL1 needs to be - * done here. - * When VHE _is_ in use, EL1 will not be used in the host and - * requires no configuration, and all non-hyp-specific EL2 setup - * will be done via the _EL1 system register aliases in __cpu_setup. - */ - mov_q x0, (SCTLR_EL1_RES1 | ENDIAN_SET_EL1) - msr sctlr_el1, x0 - - /* Coprocessor traps. */ - mov x0, #0x33ff - msr cptr_el2, x0 // Disable copro. traps to EL2 - - /* SVE register access */ - mrs x1, id_aa64pfr0_el1 - ubfx x1, x1, #ID_AA64PFR0_SVE_SHIFT, #4 - cbz x1, 7f - - bic x0, x0, #CPTR_EL2_TZ // Also disable SVE traps - msr cptr_el2, x0 // Disable copro. traps to EL2 +SYM_INNER_LABEL(el2_setup_nvhe, SYM_L_LOCAL) + mov_q x0, HCR_HOST_NVHE_FLAGS + msr hcr_el2, x0 isb - mov x1, #ZCR_ELx_LEN_MASK // SVE: Enable full vector - msr_s SYS_ZCR_EL2, x1 // length for EL1. + + init_el2_state nvhe /* Hypervisor stub */ -7: adr_l x0, __hyp_stub_vectors + adr_l x0, __hyp_stub_vectors msr vbar_el2, x0 - /* spsr */ - mov x0, #(PSR_F_BIT | PSR_I_BIT | PSR_A_BIT | PSR_D_BIT |\ - PSR_MODE_EL1h) - msr spsr_el2, x0 + mov_q x0, (SCTLR_EL1_RES1 | ENDIAN_SET_EL1) + msr sctlr_el1, x0 msr elr_el2, lr mov w0, #BOOT_CPU_MODE_EL2 // This CPU booted in EL2 eret From patchwork Thu Nov 26 15:54:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11934307 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5448EC63697 for ; Thu, 26 Nov 2020 15:57:39 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D68FB21D40 for ; Thu, 26 Nov 2020 15:57:38 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="kCHJNvk1"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="ZzoISy5I" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D68FB21D40 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=ZTb/6wuc1ppgVdUvEfSzLjYDOsKadqZcpJgn3h/wv6A=; b=kCHJNvk1HUVJEu19/MB+zaNzz HoAbFWL8RdMzp/toQXWQfzL+RlqHUyLfm1e3PaBaDx1M9Ahdf8xK1O7CLwKfrconGA3AMWGvSc1oH 2VaD8gXM05nkNre6OWJFUOXBMdhznfVcqZ3EaCy+JWWj6tnl3Y9Lrn2k4VcxvZOGZV0QwZdjF2pw0 4/j48LcR/DEB4/k4ETgjVIlvM3mR8iikd5sQoqTjw3K9JV8cLRxW5zxx0njxUT4zk4qOz42zmOJqT dFcw3VYKvncV1crUZJGsdu1/4R+fSG9ZFKmQlCeAhv7d6/nlhAVNztqpCOgQjFsPCZZg+7FH3QTW7 7mhGFMVyA==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kiJcX-00084l-Ne; Thu, 26 Nov 2020 15:55:45 +0000 Received: from mail-wr1-x442.google.com ([2a00:1450:4864:20::442]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kiJbU-0007h7-C4 for linux-arm-kernel@lists.infradead.org; Thu, 26 Nov 2020 15:54:41 +0000 Received: by mail-wr1-x442.google.com with SMTP id z7so2645090wrn.3 for ; Thu, 26 Nov 2020 07:54:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=txa3gmLNZHrKqQEh2rRK806HAOB8/IMPENffeRn2Pp8=; b=ZzoISy5IBgKnYUDSKLr4ou3Hmpa7vhh20W36eivbCgG7/560M52OOFTXLUxAKoFV8a PBuwaGiqAm+4JH02iFzu3eb36E9VnlcDWPyev/kMaU5ucflRuB6CSahFxT5c2fcKIrXo OasaMJbdGe1vTJKcamZK55+4r8AGeP5kGb53s8a2v6LsQRwdh+hYYLMCS482yRa43LoZ vUsT+QqEFp0529JcVHSKd6+YmEnRYQD3E41jTl1KFycA4JB/gtHY4NXsHBziHtiOiOaO 2EuNnguAmi6l+obHw6xfXSKVBefA7yCJdjiN+URyOPsHKM0XDVNPWJACJNsnervf0jqO gtqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=txa3gmLNZHrKqQEh2rRK806HAOB8/IMPENffeRn2Pp8=; b=bUJGRBfiV0zwj1+aIs+xMvW6tBr7WhTWs8xDI0rNwwIWxKRzJyjAS0Suu91bEAT8bO yT8LoF5tEHQbWfDGHBLD3Jeas2h1+4qLUQ8MND+wnUnwu1GmE8o2XwNEeh7/+G1cTrR5 A6KwjM185yQKUUjZa1i7ZZKmomiucvLn2PCHfxNaM+JN47BW/82WRTgFs0/LbjA758X0 jYPuqWQFmjphfmGi3pcWqVVfMz1srqEve6k3s6h5W3Huz7j+eUiwUNffWr3/mWvf9Sal ntRbb6sr4JWd0TUZin0fuxjZi2jaxKMknYm9ZGlCrj5C7hncyrtXBOeqMmZVDiNsZ79u G2Qg== X-Gm-Message-State: AOAM5323oLJVc2wAr8cfDCWSOFasUyykGWEvdVPyMd1SPZt0zg9WtO3T p6z5llAD+m3sAa/vzRpJGPxnQQ== X-Google-Smtp-Source: ABdhPJwmwAB0qeB/yoQ0FYSa/sH0XxV6RjnpiuboG/gAt0L9d0uMTFj254pYIT1Ecf1dLKgj9sUIfQ== X-Received: by 2002:a5d:66cd:: with SMTP id k13mr4618637wrw.365.1606406079235; Thu, 26 Nov 2020 07:54:39 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:f008:704d:8d4b:9951]) by smtp.gmail.com with ESMTPSA id o4sm8750577wmh.33.2020.11.26.07.54.37 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 26 Nov 2020 07:54:38 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v3 06/23] kvm: arm64: Add kvm-arm.protected early kernel parameter Date: Thu, 26 Nov 2020 15:54:04 +0000 Message-Id: <20201126155421.14901-7-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201126155421.14901-1-dbrazdil@google.com> References: <20201126155421.14901-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201126_105440_518668_6E6086BF X-CRM114-Status: GOOD ( 23.34 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add an early parameter that allows users to opt into protected KVM mode when using the nVHE hypervisor. In this mode, guest state will be kept private from the host. This will primarily involve enabling stage-2 address translation for the host, restricting DMA to host memory, and filtering host SMCs. Capability ARM64_PROTECTED_KVM is set if the param is passed, CONFIG_KVM is enabled and the kernel was not booted with VHE. Signed-off-by: David Brazdil --- .../admin-guide/kernel-parameters.txt | 5 ++++ arch/arm64/include/asm/cpucaps.h | 3 +- arch/arm64/include/asm/virt.h | 8 +++++ arch/arm64/kernel/cpufeature.c | 29 +++++++++++++++++++ arch/arm64/kvm/arm.c | 4 ++- 5 files changed, 47 insertions(+), 2 deletions(-) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index 526d65d8573a..06c89975c29c 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -2259,6 +2259,11 @@ for all guests. Default is 1 (enabled) if in 64-bit or 32-bit PAE mode. + kvm-arm.protected= + [KVM,ARM] Allow spawning protected guests whose state + is kept private from the host. Only valid for non-VHE. + Default is 0 (disabled). + kvm-arm.vgic_v3_group0_trap= [KVM,ARM] Trap guest accesses to GICv3 group-0 system registers diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h index 162539d4c8cd..9fab6cbffce2 100644 --- a/arch/arm64/include/asm/cpucaps.h +++ b/arch/arm64/include/asm/cpucaps.h @@ -66,7 +66,8 @@ #define ARM64_HAS_TLB_RANGE 56 #define ARM64_MTE 57 #define ARM64_WORKAROUND_1508412 58 +#define ARM64_PROTECTED_KVM 59 -#define ARM64_NCAPS 59 +#define ARM64_NCAPS 60 #endif /* __ASM_CPUCAPS_H */ diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h index 6069be50baf9..2fde1186b962 100644 --- a/arch/arm64/include/asm/virt.h +++ b/arch/arm64/include/asm/virt.h @@ -97,6 +97,14 @@ static __always_inline bool has_vhe(void) return cpus_have_final_cap(ARM64_HAS_VIRT_HOST_EXTN); } +static __always_inline bool is_protected_kvm_enabled(void) +{ + if (is_vhe_hyp_code()) + return false; + else + return cpus_have_final_cap(ARM64_PROTECTED_KVM); +} + #endif /* __ASSEMBLY__ */ #endif /* ! __ASM__VIRT_H */ diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 6f36c4f62f69..dd5bc0f0cf0d 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -1709,6 +1709,29 @@ static void cpu_enable_mte(struct arm64_cpu_capabilities const *cap) } #endif /* CONFIG_ARM64_MTE */ +#ifdef CONFIG_KVM +static bool enable_protected_kvm; + +static bool has_protected_kvm(const struct arm64_cpu_capabilities *entry, int __unused) +{ + if (!enable_protected_kvm) + return false; + + if (is_kernel_in_hyp_mode()) { + pr_warn("Protected KVM not available with VHE\n"); + return false; + } + + return true; +} + +static int __init early_protected_kvm_cfg(char *buf) +{ + return strtobool(buf, &enable_protected_kvm); +} +early_param("kvm-arm.protected", early_protected_kvm_cfg); +#endif /* CONFIG_KVM */ + /* Internal helper functions to match cpu capability type */ static bool cpucap_late_cpu_optional(const struct arm64_cpu_capabilities *cap) @@ -1822,6 +1845,12 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .field_pos = ID_AA64PFR0_EL1_SHIFT, .min_field_value = ID_AA64PFR0_EL1_32BIT_64BIT, }, + { + .desc = "Protected KVM", + .capability = ARM64_PROTECTED_KVM, + .type = ARM64_CPUCAP_SYSTEM_FEATURE, + .matches = has_protected_kvm, + }, #endif { .desc = "Kernel page table isolation (KPTI)", diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 2d0a37c75cda..b25035dc0478 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1818,7 +1818,9 @@ int kvm_arch_init(void *opaque) if (err) goto out_hyp; - if (in_hyp_mode) + if (is_protected_kvm_enabled()) + kvm_info("Protected nVHE mode initialized successfully\n"); + else if (in_hyp_mode) kvm_info("VHE mode initialized successfully\n"); else kvm_info("Hyp mode initialized successfully\n"); From patchwork Thu Nov 26 15:54:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11934309 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F21DDC63697 for ; Thu, 26 Nov 2020 15:57:47 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5FE6621D40 for ; Thu, 26 Nov 2020 15:57:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="JWa6yIvW"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="aDvl7dCl" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5FE6621D40 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=NVbHcJY1/spL8UltefCBJOKgdL2anCn0070a2BpeQw0=; b=JWa6yIvWhQGtq1rb4GRKEyHny CGhMYPPiWW/U2Fd0NVnU4nTPoHEVXKNIwJoOGBhXSPK9gVYiFJCH2nH2o6NJt/j6mhg3SCwOKX1wf dsQNaWGI59KQ6WNnypcYDsR3qcO5X9/orsE78qSoe2u4C9ch9tiDvVy+HBUMjst1GrltHJabUZZUh i8J/ipOeisrNY4NuWoA/GhM9I0vCWFFGWFUCe0/cw6xy/scFQX2xqflD4pGPouPvHlVl4LBQFz2yR WYByG2XWuNQQ92TX9JK97eCWXaZ6Cn3T9Hpz/g/JenJW5ybzui+eqtMgvOPPCNMJ0Aphh9P1bIjN+ ZbErovBgg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kiJci-0008BD-8V; Thu, 26 Nov 2020 15:55:56 +0000 Received: from mail-wm1-x341.google.com ([2a00:1450:4864:20::341]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kiJbW-0007hk-LN for linux-arm-kernel@lists.infradead.org; Thu, 26 Nov 2020 15:54:43 +0000 Received: by mail-wm1-x341.google.com with SMTP id 3so159572wmj.4 for ; Thu, 26 Nov 2020 07:54:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ZrCAS9ygWkC2ufXbj2Wobao4dQO/5x9OhapFi6lV40s=; b=aDvl7dClV2uddSkR11BaXY3pN+uz5HABtJLBS7EVNJEKoJNZV0iBk9o8D59nUGH/+a 3B7FnrMyxDa9I5WhiMmMzlAgUJydOA7B1ixiCwBNjGHS2f23ntZe758HSRDydqFfNx0G AqoHtivWbZj+IUF4KFbqnve9WibgLTwyw/zq0e3ULZgspz4+L2SoR4O0BzQb0HfNfyP+ VYu7VHGair8YbXPnsqfj10NbpQwbS+RfY/5HLENNvh3lTrEkIqim4RFcHVQKoZZ6IQRB oNuaz8WBT941Oa80szhb6hkSCeWKRh0zO+o7NbC2ztdnZAqmYks//F5Q/aWEttvKUeYa +CyA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ZrCAS9ygWkC2ufXbj2Wobao4dQO/5x9OhapFi6lV40s=; b=iqZelwHcjyGH33KgKTDJH5xTLkD0YN0GPvUnO3YuitsWmqZjNUA+cgGRJt8a8PcAmJ QKrdYfBuPGqC7g9f8FV2vxJt0Cm6evWJNY2DTSaipwUuOpjFSyA9GUT94TJqqElmPmm8 aakFrLJMKbconTtV6eiRKl0fWliGQC2CgalEjdvQTTFW47+fD4Px6UW81BT+bDWIu+9N 0RxIeZiHb6sojR28p7XFhYR3uI5U6Hb7n/GrXB55E3vdH1GATxNbYfbG7APrAJMorgI7 O/naxwNOFQtd+6WCzRXmcxw83EytYQL3yO9zbOMbIjoUqXhwmtGlTK2Zgn7mH9uvuC2r Ujig== X-Gm-Message-State: AOAM531C4d+2ZY+MotmEuEC+U0psSv1Td1mOk64CkzZ8nQgcfrYCRTjk ISPQs2jKVGSsjrUm+rUuqsTZsg== X-Google-Smtp-Source: ABdhPJzjuSUApvEOEPjZQvj0RwSRQkwjvpqFgqkQbvkP2wAo7cyRvkYNRsvgHC/RhJaCaStDd8X2AQ== X-Received: by 2002:a05:600c:286:: with SMTP id 6mr4043009wmk.125.1606406081513; Thu, 26 Nov 2020 07:54:41 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:f008:704d:8d4b:9951]) by smtp.gmail.com with ESMTPSA id s2sm9141206wmh.37.2020.11.26.07.54.39 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 26 Nov 2020 07:54:40 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v3 07/23] kvm: arm64: Initialize MAIR_EL2 using a constant Date: Thu, 26 Nov 2020 15:54:05 +0000 Message-Id: <20201126155421.14901-8-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201126155421.14901-1-dbrazdil@google.com> References: <20201126155421.14901-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201126_105442_779413_6D769221 X-CRM114-Status: GOOD ( 16.25 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org MAIR_EL2 is currently initialized to the value of MAIR_EL1, which itself is set to a constant MAIR_ELx_SET. Initialize MAIR_EL2 to MAIR_ELx_SET directly in preparation for allowing KVM to start CPU cores itself before ERETing to EL1. In that case, MAIR_EL2 will be initialized before MAIR_EL1. Signed-off-by: David Brazdil --- arch/arm64/kvm/hyp/nvhe/hyp-init.S | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-init.S b/arch/arm64/kvm/hyp/nvhe/hyp-init.S index b11a9d7db677..17b58dbc3a2f 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-init.S +++ b/arch/arm64/kvm/hyp/nvhe/hyp-init.S @@ -111,7 +111,7 @@ alternative_else_nop_endif msr tcr_el2, x0 - mrs x0, mair_el1 + mov_q x0, MAIR_ELx_SET msr mair_el2, x0 isb From patchwork Thu Nov 26 15:54:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11934311 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A8238C56202 for ; Thu, 26 Nov 2020 15:57:51 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5E45D21D40 for ; Thu, 26 Nov 2020 15:57:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="KjNYwdeV"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="L83CDnry" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5E45D21D40 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=J8iTg6Q00spUy2j3pQdj8OIJuzG6K6BvCV28REbTnZ0=; b=KjNYwdeVFQJxwPUcziRNbCjCd zzHOom3BnciOm37e46pMgcaNrB/9e8vxX98GyblL3EB+ZfOQ28xThZwXHfhGfy/YgFTBR0BXKh9TI aD1XHnTx6oXK9uHh7I7mZFuw/kGCsvizUCyJIvYSEiL3Np0v2px8v89nrLjXrtO7AGdjUSKazjlH4 ar97MD3YPSLtxakSO5fqUIUKybRSOZJOL5fYdSbq4bdjliY8wKrrW0aC1spuZuLFB9dkJMsjn08yy /8Alt3yC74YBdJ1uHHeUQGpqMdk+qd65WS4FPIYH9iARMvEmq3KrK/fuWy4M02CPLNWF5nwqpZej/ Wr69tNFgQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kiJco-0008Cn-Ps; Thu, 26 Nov 2020 15:56:02 +0000 Received: from mail-wr1-x441.google.com ([2a00:1450:4864:20::441]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kiJbY-0007iE-K5 for linux-arm-kernel@lists.infradead.org; Thu, 26 Nov 2020 15:54:45 +0000 Received: by mail-wr1-x441.google.com with SMTP id p8so2639512wrx.5 for ; Thu, 26 Nov 2020 07:54:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=utPvkzoqy6/nIv7yirT7WFlgfKz1OWQA6qiMqU5NSCo=; b=L83CDnryQMuuOUO42YEM1Fei9cFaqRJ6AxwBerQHFEt9tWXMW0lNOkPpS2bejSW/iu tEbbm0lx868wYcgllszhwgBocoNjjhpRethBq4ewq1j2wlI0/ZkmNjLzjNhb/T5wDs0L uE8FlKRHQVHKYNpJJmKdt8Oqd7WY7PiRrcKTAyrLPIZe14wtKCPqpe0/nA/ozRQN8zVy prbEpoF0/+E0KBzkxM79MvkufhuPbIS5u0EoQplVHSL8Zj8YC8wXXkPWfHECdf9KcpuD LGOTL0W9oT4TEWa0ylv9yro0tguEgWkdJ6sWfhkDrJ4ahom5TB2nKKtsgMzPLBwm/SDX ZG5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=utPvkzoqy6/nIv7yirT7WFlgfKz1OWQA6qiMqU5NSCo=; b=g5LJTeOKKggZL0R+T0qUAyiVoJEJfWAzlj9W99qmZTrF6xVYpSt44dIHe19bbGvqqn 1znjBOYjFGsKNTM7In2ieF7+lkqycwluqpPTnl0N6C6G+1iK6SMq4lPjxJpB0BQnrj2B NS0263fzyX2Ievfejyzmz2lMtBwZVHFUOV5HewKZR7yqUobvLA0O2vVNnhK6rN2513vI 08J7nwaiDJQvmFP/gHf9SzhfZ7ZOMotlrJE9YCP7vbTst3g/AQx4WF+SAAC74jXatBy1 ILk6okPg2KwtAL5Rco71+O2W0wNsn+nRKsd50o5xre5qlo6U8YYbFlG1l+q5cb2TDe/N lqUA== X-Gm-Message-State: AOAM532TLCSXe5sQt3ksTSGc5coN1/gx6h4wvhAy5/aUkRKKRYvA5JtV gzlvl16m8Ex8qf3lU1H6y6bhlQ== X-Google-Smtp-Source: ABdhPJxbpHUEbEI7H8KyDN6Qt9yHuAuJ3eZuUVlv4sN6UGfKQTpLTnLejNgi4Q50yjIL+OaOhuCKIw== X-Received: by 2002:a5d:6191:: with SMTP id j17mr4556274wru.299.1606406083603; Thu, 26 Nov 2020 07:54:43 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:f008:704d:8d4b:9951]) by smtp.gmail.com with ESMTPSA id j8sm9486504wrx.11.2020.11.26.07.54.42 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 26 Nov 2020 07:54:42 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v3 08/23] kvm: arm64: Remove vector_ptr param of hyp-init Date: Thu, 26 Nov 2020 15:54:06 +0000 Message-Id: <20201126155421.14901-9-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201126155421.14901-1-dbrazdil@google.com> References: <20201126155421.14901-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201126_105444_712486_835B4088 X-CRM114-Status: GOOD ( 19.58 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org KVM precomputes the hyp VA of __kvm_hyp_host_vector, essentially a constant (minus ASLR), before passing it to __kvm_hyp_init. Now that we have alternatives for converting kimg VA to hyp VA, replace this with computing the constant inside __kvm_hyp_init, thus removing the need for an argument. Signed-off-by: David Brazdil --- arch/arm64/include/asm/kvm_asm.h | 2 -- arch/arm64/include/asm/kvm_mmu.h | 26 +++++++++++++++++++++++++- arch/arm64/kvm/arm.c | 4 +--- arch/arm64/kvm/hyp/nvhe/hyp-init.S | 9 ++++++--- 4 files changed, 32 insertions(+), 9 deletions(-) diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h index 4a6a77d8d13e..531f9d04eefd 100644 --- a/arch/arm64/include/asm/kvm_asm.h +++ b/arch/arm64/include/asm/kvm_asm.h @@ -163,10 +163,8 @@ struct kvm_vcpu; struct kvm_s2_mmu; DECLARE_KVM_NVHE_SYM(__kvm_hyp_init); -DECLARE_KVM_NVHE_SYM(__kvm_hyp_host_vector); DECLARE_KVM_HYP_SYM(__kvm_hyp_vector); #define __kvm_hyp_init CHOOSE_NVHE_SYM(__kvm_hyp_init) -#define __kvm_hyp_host_vector CHOOSE_NVHE_SYM(__kvm_hyp_host_vector) #define __kvm_hyp_vector CHOOSE_HYP_SYM(__kvm_hyp_vector) extern unsigned long kvm_arm_hyp_percpu_base[NR_CPUS]; diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h index 5168a0c516ae..6945faeb68ac 100644 --- a/arch/arm64/include/asm/kvm_mmu.h +++ b/arch/arm64/include/asm/kvm_mmu.h @@ -93,7 +93,31 @@ alternative_cb_end /* reg = __pa(reg) */ sub \reg, \reg, \tmp .endm - + +/* + * Convert a kernel image address to a hyp VA + * reg: kernel address to be converted in place + * tmp: temporary register + * + * The actual code generation takes place in kvm_get_kimage_voffset, and + * the instructions below are only there to reserve the space and + * perform the register allocation (kvm_get_kimage_voffset uses the + * specific registers encoded in the instructions). + */ +.macro kimg_hyp_va reg, tmp +alternative_cb kvm_update_kimg_phys_offset + movz \tmp, #0 + movk \tmp, #0, lsl #16 + movk \tmp, #0, lsl #32 + movk \tmp, #0, lsl #48 +alternative_cb_end + + sub \reg, \reg, \tmp + mov_q \tmp, PAGE_OFFSET + orr \reg, \reg, \tmp + kern_hyp_va \reg +.endm + #else #include diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index b25035dc0478..cd6e5dec203f 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1355,7 +1355,6 @@ static void cpu_init_hyp_mode(void) { phys_addr_t pgd_ptr; unsigned long hyp_stack_ptr; - unsigned long vector_ptr; unsigned long tpidr_el2; struct arm_smccc_res res; @@ -1373,7 +1372,6 @@ static void cpu_init_hyp_mode(void) pgd_ptr = kvm_mmu_get_httbr(); hyp_stack_ptr = __this_cpu_read(kvm_arm_hyp_stack_page) + PAGE_SIZE; hyp_stack_ptr = kern_hyp_va(hyp_stack_ptr); - vector_ptr = (unsigned long)kern_hyp_va(kvm_ksym_ref(__kvm_hyp_host_vector)); /* * Call initialization code, and switch to the full blown HYP code. @@ -1383,7 +1381,7 @@ static void cpu_init_hyp_mode(void) */ BUG_ON(!system_capabilities_finalized()); arm_smccc_1_1_hvc(KVM_HOST_SMCCC_FUNC(__kvm_hyp_init), - pgd_ptr, tpidr_el2, hyp_stack_ptr, vector_ptr, &res); + pgd_ptr, tpidr_el2, hyp_stack_ptr, &res); WARN_ON(res.a0 != SMCCC_RET_SUCCESS); /* diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-init.S b/arch/arm64/kvm/hyp/nvhe/hyp-init.S index 17b58dbc3a2f..b3492a3c4232 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-init.S +++ b/arch/arm64/kvm/hyp/nvhe/hyp-init.S @@ -50,7 +50,6 @@ __invalid: * x1: HYP pgd * x2: per-CPU offset * x3: HYP stack - * x4: HYP vectors */ __do_hyp_init: /* Check for a stub HVC call */ @@ -134,9 +133,13 @@ alternative_else_nop_endif msr sctlr_el2, x0 isb - /* Set the stack and new vectors */ + /* Set the stack */ mov sp, x3 - msr vbar_el2, x4 + + /* Set the host vector */ + ldr x0, =__kvm_hyp_host_vector + kimg_hyp_va x0, x1 + msr vbar_el2, x0 /* Hello, World! */ mov x0, #SMCCC_RET_SUCCESS From patchwork Thu Nov 26 15:54:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11934313 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 02A0CC56202 for ; Thu, 26 Nov 2020 15:58:40 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AC17121D40 for ; Thu, 26 Nov 2020 15:58:39 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="MmQhPYW2"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="np/ka0YJ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AC17121D40 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=bbJRUJ1lyR/7WLU6YIRphok45tOzFiCDmOu7cypI3+k=; b=MmQhPYW29KuBVVK4eVXwZpEmm 1aN+kegxY2IH1GYsoOuFJliG1BHlMvrYq28G+xXDgnK/0ktuSpvvukqIFELiU4tU5Ggh8iW8FsYX5 uHSP3+RljShGi2PEJA0onWAkstEEOFJGfSpCoRrP6mRHdEVE5Q+aHlyjP57+QfXkg9uCJwgrCEEpG xtH3WMrEv8fW8pN1W/P4tkZrqt1UpEqdG7k+m7HZMLhdBohkGKLLkrQFNQIjLxDJrKmBWJeoBXt7/ p6wRQ0VcX7h6uuPkXLbb88TPBwYJNLSj0emKfMUfQPG0g3kqsgKnuUF5KJc/nU6K7z+Deyxzk7EDj OnqRq0H2Q==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kiJdN-0008RC-RI; Thu, 26 Nov 2020 15:56:38 +0000 Received: from mail-wr1-x442.google.com ([2a00:1450:4864:20::442]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kiJba-0007io-PV for linux-arm-kernel@lists.infradead.org; Thu, 26 Nov 2020 15:54:48 +0000 Received: by mail-wr1-x442.google.com with SMTP id u12so2663796wrt.0 for ; Thu, 26 Nov 2020 07:54:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=f3LDgmC45SSw9htMecvZjLzzx6eFZRjVdjLqmJaHMYo=; b=np/ka0YJmEFiD+LjfAs7Dtv9KLiiQXDJ0Wm3by1Kj1qPHJPf1BhLjjQmO3cBJ5VmGV BAkZRGzuYDcXV+u27Y1ltE1v0oCgzZRptwShGbCjyQXrCqKvpBKAFRQ2A+ihJsqD0lRN YTFbiHJ6TnSP+IZKuIdJNMgKkUjvcqDpg3piB6pGOoez2hcMu2AtFisOTidHUbbNSgo5 YotR2JVWY5g6EgaGhS4TqBVe5m1oxBHr/uuN/lxPvn0KeLWnoisurrgQR5ef2KC2d6WK 0NVT3kxTigiSdTKYgWUp4AYXPxoMDVrEzOAaSMMqFN2U8DmzYf+Dgy5USd9SRhmXi1Y6 dg4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=f3LDgmC45SSw9htMecvZjLzzx6eFZRjVdjLqmJaHMYo=; b=IUMvg5jA/YjK5HZMzgb9otc88WiU2pLI72J9qbpfFxEvvu6qhGHXhChaLSfYm1fCLU zba9UpAZpR1x6Ki54uH1D0lmdwkToltlu7DaHbG2fU+P91cXIEPShoGNXPzlN4DqpdwC UGzDlh71mHrupMPpKvRtW1STT4lZiuaHovXvF1PCHgK/9she3hfHNH3GtcTF/QG8CFAH CR9VBdoeNkHe/RmzlunlPqyVoI8GoeTjAoAaJl1zczf8ivfy/zoJredHvTFMicrRi80u rEbe7eQe/CiO1p0qN6mb+WGtO0Gku9iPu/y6mP42Ua3lbGlQ+CZjU1SEWzkYpbxt6jUd SqLQ== X-Gm-Message-State: AOAM530bMXtvSM0aFrUcL2Z2AsYmXal5nnik48oVhzS45h6080RmuXNy dLMjUf+UBa5t/NHUcGAG3LgYxQ== X-Google-Smtp-Source: ABdhPJyM3KLAYj6rk95MI9B6rCbTPkfBbVa7VlRU8nM9FO/ll7O2jCui//tkmPErzKt5hDvdgnB83w== X-Received: by 2002:adf:b64f:: with SMTP id i15mr4720632wre.125.1606406085713; Thu, 26 Nov 2020 07:54:45 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:f008:704d:8d4b:9951]) by smtp.gmail.com with ESMTPSA id l13sm9378005wrm.24.2020.11.26.07.54.44 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 26 Nov 2020 07:54:44 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v3 09/23] kvm: arm64: Move hyp-init params to a per-CPU struct Date: Thu, 26 Nov 2020 15:54:07 +0000 Message-Id: <20201126155421.14901-10-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201126155421.14901-1-dbrazdil@google.com> References: <20201126155421.14901-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201126_105447_780075_E5D5E031 X-CRM114-Status: GOOD ( 24.07 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Once we start initializing KVM on newly booted cores before the rest of the kernel, parameters to __do_hyp_init will need to be provided by EL2 rather than EL1. At that point it will not be possible to pass its three arguments directly because PSCI_CPU_ON only supports one context argument. Refactor __do_hyp_init to accept its parameters in a struct. This prepares the code for KVM booting cores as well as removes any limits on the number of __do_hyp_init arguments. Signed-off-by: David Brazdil --- arch/arm64/include/asm/kvm_asm.h | 6 ++++++ arch/arm64/include/asm/kvm_hyp.h | 2 +- arch/arm64/kernel/asm-offsets.c | 3 +++ arch/arm64/kvm/arm.c | 23 +++++++++++++---------- arch/arm64/kvm/hyp/nvhe/hyp-init.S | 16 +++++++--------- arch/arm64/kvm/hyp/nvhe/hyp-main.c | 2 ++ 6 files changed, 32 insertions(+), 20 deletions(-) diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h index 531f9d04eefd..1a7b91534a16 100644 --- a/arch/arm64/include/asm/kvm_asm.h +++ b/arch/arm64/include/asm/kvm_asm.h @@ -148,6 +148,12 @@ extern void *__vhe_undefined_symbol; #endif +struct kvm_nvhe_init_params { + unsigned long tpidr_el2; + unsigned long stack_hyp_va; + phys_addr_t pgd_pa; +}; + /* Translate a kernel address @ptr into its equivalent linear mapping */ #define kvm_ksym_ref(ptr) \ ({ \ diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h index 6b664de5ec1f..cb25c15e3d8d 100644 --- a/arch/arm64/include/asm/kvm_hyp.h +++ b/arch/arm64/include/asm/kvm_hyp.h @@ -14,6 +14,7 @@ DECLARE_PER_CPU(struct kvm_cpu_context, kvm_hyp_ctxt); DECLARE_PER_CPU(unsigned long, kvm_hyp_vector); +DECLARE_PER_CPU(struct kvm_nvhe_init_params, kvm_init_params); #define read_sysreg_elx(r,nvh,vh) \ ({ \ @@ -98,4 +99,3 @@ void __noreturn __hyp_do_panic(bool restore_host, u64 spsr, u64 elr, u64 par); #endif #endif /* __ARM64_KVM_HYP_H__ */ - diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index 7d32fc959b1a..8d6272a01a00 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -110,6 +110,9 @@ int main(void) DEFINE(CPU_APGAKEYLO_EL1, offsetof(struct kvm_cpu_context, sys_regs[APGAKEYLO_EL1])); DEFINE(HOST_CONTEXT_VCPU, offsetof(struct kvm_cpu_context, __hyp_running_vcpu)); DEFINE(HOST_DATA_CONTEXT, offsetof(struct kvm_host_data, host_ctxt)); + DEFINE(NVHE_INIT_TPIDR_EL2, offsetof(struct kvm_nvhe_init_params, tpidr_el2)); + DEFINE(NVHE_INIT_STACK_HYP_VA, offsetof(struct kvm_nvhe_init_params, stack_hyp_va)); + DEFINE(NVHE_INIT_PGD_PA, offsetof(struct kvm_nvhe_init_params, pgd_pa)); #endif #ifdef CONFIG_CPU_PM DEFINE(CPU_CTX_SP, offsetof(struct cpu_suspend_ctx, sp)); diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index cd6e5dec203f..7b718c6c6c52 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -49,6 +49,7 @@ DECLARE_KVM_HYP_PER_CPU(unsigned long, kvm_hyp_vector); static DEFINE_PER_CPU(unsigned long, kvm_arm_hyp_stack_page); unsigned long kvm_arm_hyp_percpu_base[NR_CPUS]; +DECLARE_KVM_NVHE_PER_CPU(struct kvm_nvhe_init_params, kvm_init_params); /* The VMID used in the VTTBR */ static atomic64_t kvm_vmid_gen = ATOMIC64_INIT(1); @@ -1353,9 +1354,7 @@ static int kvm_init_vector_slots(void) static void cpu_init_hyp_mode(void) { - phys_addr_t pgd_ptr; - unsigned long hyp_stack_ptr; - unsigned long tpidr_el2; + struct kvm_nvhe_init_params *params = this_cpu_ptr_nvhe_sym(kvm_init_params); struct arm_smccc_res res; /* Switch from the HYP stub to our own HYP init vector */ @@ -1366,12 +1365,17 @@ static void cpu_init_hyp_mode(void) * kernel's mapping to the linear mapping, and store it in tpidr_el2 * so that we can use adr_l to access per-cpu variables in EL2. */ - tpidr_el2 = (unsigned long)this_cpu_ptr_nvhe_sym(__per_cpu_start) - - (unsigned long)kvm_ksym_ref(CHOOSE_NVHE_SYM(__per_cpu_start)); + params->tpidr_el2 = (unsigned long)this_cpu_ptr_nvhe_sym(__per_cpu_start) - + (unsigned long)kvm_ksym_ref(CHOOSE_NVHE_SYM(__per_cpu_start)); - pgd_ptr = kvm_mmu_get_httbr(); - hyp_stack_ptr = __this_cpu_read(kvm_arm_hyp_stack_page) + PAGE_SIZE; - hyp_stack_ptr = kern_hyp_va(hyp_stack_ptr); + params->stack_hyp_va = kern_hyp_va(__this_cpu_read(kvm_arm_hyp_stack_page) + PAGE_SIZE); + params->pgd_pa = kvm_mmu_get_httbr(); + + /* + * Flush the init params from the data cache because the struct will + * be read while the MMU is off. + */ + kvm_flush_dcache_to_poc(params, sizeof(*params)); /* * Call initialization code, and switch to the full blown HYP code. @@ -1380,8 +1384,7 @@ static void cpu_init_hyp_mode(void) * cpus_have_const_cap() wrapper. */ BUG_ON(!system_capabilities_finalized()); - arm_smccc_1_1_hvc(KVM_HOST_SMCCC_FUNC(__kvm_hyp_init), - pgd_ptr, tpidr_el2, hyp_stack_ptr, &res); + arm_smccc_1_1_hvc(KVM_HOST_SMCCC_FUNC(__kvm_hyp_init), virt_to_phys(params), &res); WARN_ON(res.a0 != SMCCC_RET_SUCCESS); /* diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-init.S b/arch/arm64/kvm/hyp/nvhe/hyp-init.S index b3492a3c4232..77c983aa90fa 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-init.S +++ b/arch/arm64/kvm/hyp/nvhe/hyp-init.S @@ -47,9 +47,7 @@ __invalid: /* * x0: SMCCC function ID - * x1: HYP pgd - * x2: per-CPU offset - * x3: HYP stack + * x1: struct kvm_nvhe_init_params PA */ __do_hyp_init: /* Check for a stub HVC call */ @@ -70,10 +68,13 @@ __do_hyp_init: mov x0, #SMCCC_RET_NOT_SUPPORTED eret -1: - /* Set tpidr_el2 for use by HYP to free a register */ - msr tpidr_el2, x2 +1: ldr x0, [x1, #NVHE_INIT_TPIDR_EL2] + msr tpidr_el2, x0 + ldr x0, [x1, #NVHE_INIT_STACK_HYP_VA] + mov sp, x0 + + ldr x1, [x1, #NVHE_INIT_PGD_PA] phys_to_ttbr x0, x1 alternative_if ARM64_HAS_CNP orr x0, x0, #TTBR_CNP_BIT @@ -133,9 +134,6 @@ alternative_else_nop_endif msr sctlr_el2, x0 isb - /* Set the stack */ - mov sp, x3 - /* Set the host vector */ ldr x0, =__kvm_hyp_host_vector kimg_hyp_va x0, x1 diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c index 82df7fc24760..a4f1cac714d7 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -16,6 +16,8 @@ #define DECLARE_REG(type, name, ctxt, reg) \ type name = (type)cpu_reg(ctxt, (reg)) +DEFINE_PER_CPU(struct kvm_nvhe_init_params, kvm_init_params); + static void handle___kvm_vcpu_run(struct kvm_cpu_context *host_ctxt) { DECLARE_REG(struct kvm_vcpu *, vcpu, host_ctxt, 1); From patchwork Thu Nov 26 15:54:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11934315 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0B557C63798 for ; Thu, 26 Nov 2020 15:59:55 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 89D62221E2 for ; Thu, 26 Nov 2020 15:59:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="xzOMseg8"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="ZDpOoP1c" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 89D62221E2 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=+Y5ZgxX6uQw52qzEakAvESP4cKYYZUhlbWcBf5K4UFQ=; b=xzOMseg8nT3u3yXuSateiw0ie 2i7VxQOJpa4jZcv2GYpgbL7TvKvA7GyL5CljJUY7bgVYikRZr1nQLFdWGJmYI+jMSiXL3gERyawU2 ncBiVnlPVPqOMUANOCVLvEAySCQeF3WWcf6wsfRoKOayMHMc06LHn2lSxzFgQx/BtPVjKrQzpFxum rihzAqatX/t6lA4ZxiL1BP8H0624BpB4TArlXUYy6FKjbLK+cu6csc4i6PEJhDvjwjfojH4+YfoKo jYOrvcvqYLdo/oUxOS9cIRyHWnsoZpglzL8lOH82tKGNng1zxQNtKn1jG713yLc6lIrK7Dd2WbU5D Q3i1ZS4eQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kiJdw-0000Br-0K; Thu, 26 Nov 2020 15:57:12 +0000 Received: from mail-wr1-x441.google.com ([2a00:1450:4864:20::441]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kiJbd-0007jL-0n for linux-arm-kernel@lists.infradead.org; Thu, 26 Nov 2020 15:54:50 +0000 Received: by mail-wr1-x441.google.com with SMTP id p8so2639787wrx.5 for ; Thu, 26 Nov 2020 07:54:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=zq7DkfhPMt/JJAro9r54YdZwfFGg+H4Ku0Elb+CBars=; b=ZDpOoP1cUv9qcC6WeUnajZTpE2pk/DfIPjgnDHP1fSPmAHBlDq+tp+3dCwFK+NJoSr hujy8nJ43XQNnsV6cOfw8ctviAy9sGNJVkSRJMX25MSifUgsKx0JqZWoK34cRAEjjQ1j Z6qDRdtenNCZJxV46aR/54uPoXGkfFoG4+yEHJ++Syy9+9pHem+bmkxmhqda652EE7S1 +eswk4U8GtrZyc+BYM5rcEq/0NXbBghplq25uRrB7pbqznAVmNGH4OLOidKj/2iH3U9n d4lLTs7kksxsNfuC/Q/AivMiyhMEf/WffyHvuIB2rPexIjJF0P2j2YmJ9S1ES/Mo+6XH NF9A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=zq7DkfhPMt/JJAro9r54YdZwfFGg+H4Ku0Elb+CBars=; b=XTXGHYJxdRE8apyVVDehj0l1/uBf3I8MyFhX7KKa1y3ENRG4bDW6dhJBL77CDLuCAV m2ayHSzHAM3oPidscyd8VAigEenHaHRBZDjdvT+R+qrGGmkHNAt9uRm6Qg4jQAOFPLpQ jIIhdh7/JkZkePZ92z9RdO1VXXaVpTvmF/yOsVyeh8CGLqBm68ot5W/OtwfBvKfAyudB LoNRuzk2e9LCzQ8AQLskEmyNKxyD6DVRbOFvAbLjmWT7M6zxk3pck/2/6eKbzUHoj4mE CsDhwhn53etxQ6zLqpOjoyGJ5NpRca9q/43X/X0G/HwrUKAF1JAjWToiPlpk57iT1YvB PbvQ== X-Gm-Message-State: AOAM532Lj6jneO4LJQ7/8l0jTLmQegNSRyN/9IpwJykTKTf9+p7s8UHH hDeEFaQS5SQiy4v5TeXcv33Stw== X-Google-Smtp-Source: ABdhPJzLv6ad9C+3TJ+xMYmPjbmsjNfbGyokblFmNlEuroXWUirX+UtceeJ4SEAb0WLZLPCge9tRfA== X-Received: by 2002:adf:e912:: with SMTP id f18mr4537063wrm.79.1606406087914; Thu, 26 Nov 2020 07:54:47 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:f008:704d:8d4b:9951]) by smtp.gmail.com with ESMTPSA id v8sm9686147wmg.28.2020.11.26.07.54.46 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 26 Nov 2020 07:54:46 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v3 10/23] kvm: arm64: Add .hyp.data..ro_after_init ELF section Date: Thu, 26 Nov 2020 15:54:08 +0000 Message-Id: <20201126155421.14901-11-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201126155421.14901-1-dbrazdil@google.com> References: <20201126155421.14901-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201126_105449_215917_14ECA97C X-CRM114-Status: GOOD ( 20.81 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add rules for renaming the .data..ro_after_init ELF section in KVM nVHE object files to .hyp.data..ro_after_init, linking it into the kernel and mapping it in hyp at runtime. The section is RW to the host, then mapped RO in hyp. The expectation is that the host populates the variables in the section and they are never changed by hyp afterwards. Signed-off-by: David Brazdil --- arch/arm64/include/asm/sections.h | 1 + arch/arm64/kernel/vmlinux.lds.S | 10 ++++++++++ arch/arm64/kvm/arm.c | 8 ++++++++ arch/arm64/kvm/hyp/nvhe/hyp.lds.S | 1 + 4 files changed, 20 insertions(+) diff --git a/arch/arm64/include/asm/sections.h b/arch/arm64/include/asm/sections.h index 3994169985ef..8ff579361731 100644 --- a/arch/arm64/include/asm/sections.h +++ b/arch/arm64/include/asm/sections.h @@ -11,6 +11,7 @@ extern char __alt_instructions[], __alt_instructions_end[]; extern char __hibernate_exit_text_start[], __hibernate_exit_text_end[]; extern char __hyp_idmap_text_start[], __hyp_idmap_text_end[]; extern char __hyp_text_start[], __hyp_text_end[]; +extern char __hyp_data_ro_after_init_start[], __hyp_data_ro_after_init_end[]; extern char __idmap_text_start[], __idmap_text_end[]; extern char __initdata_begin[], __initdata_end[]; extern char __inittext_begin[], __inittext_end[]; diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index 1bda604f4c70..4382b5d0645d 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -30,6 +30,13 @@ jiffies = jiffies_64; *(__kvm_ex_table) \ __stop___kvm_ex_table = .; +#define HYPERVISOR_DATA_SECTIONS \ + HYP_SECTION_NAME(.data..ro_after_init) : { \ + __hyp_data_ro_after_init_start = .; \ + *(HYP_SECTION_NAME(.data..ro_after_init)) \ + __hyp_data_ro_after_init_end = .; \ + } + #define HYPERVISOR_PERCPU_SECTION \ . = ALIGN(PAGE_SIZE); \ HYP_SECTION_NAME(.data..percpu) : { \ @@ -37,6 +44,7 @@ jiffies = jiffies_64; } #else /* CONFIG_KVM */ #define HYPERVISOR_EXTABLE +#define HYPERVISOR_DATA_SECTIONS #define HYPERVISOR_PERCPU_SECTION #endif @@ -234,6 +242,8 @@ SECTIONS _sdata = .; RW_DATA(L1_CACHE_BYTES, PAGE_SIZE, THREAD_ALIGN) + HYPERVISOR_DATA_SECTIONS + /* * Data written with the MMU off but read with the MMU on requires * cache lines to be invalidated, discarding up to a Cache Writeback diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 7b718c6c6c52..bb07f0401c68 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1652,6 +1652,14 @@ static int init_hyp_mode(void) goto out_err; } + err = create_hyp_mappings(kvm_ksym_ref(__hyp_data_ro_after_init_start), + kvm_ksym_ref(__hyp_data_ro_after_init_end), + PAGE_HYP_RO); + if (err) { + kvm_err("Cannot map .hyp.data..ro_after_init section\n"); + goto out_err; + } + err = create_hyp_mappings(kvm_ksym_ref(__start_rodata), kvm_ksym_ref(__end_rodata), PAGE_HYP_RO); if (err) { diff --git a/arch/arm64/kvm/hyp/nvhe/hyp.lds.S b/arch/arm64/kvm/hyp/nvhe/hyp.lds.S index bb2d986ff696..5d76ff2ba63e 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp.lds.S +++ b/arch/arm64/kvm/hyp/nvhe/hyp.lds.S @@ -16,4 +16,5 @@ SECTIONS { HYP_SECTION_NAME(.data..percpu) : { PERCPU_INPUT(L1_CACHE_BYTES) } + HYP_SECTION(.data..ro_after_init) } From patchwork Thu Nov 26 15:54:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11934317 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0AD0CC56202 for ; Thu, 26 Nov 2020 16:00:05 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 881B121D7E for ; Thu, 26 Nov 2020 16:00:04 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="Lob0LHnO"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="vuRWz2Vz" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 881B121D7E Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=bj/DQEyxQ1kPyX8Y0BXZeCnRYiCAVJKSm587dcyESXc=; b=Lob0LHnOT1ydFqEjn5fXWqvqr m4rvZnVpwkGnY9py2TiQJepwMJxy+ZSLSvwTZ1qxzWNBZRulSZqw2ZySUePVBdB+m+qRDtk6lyshZ 1vN6EHzzFWnCHRqA4H9Tm2mRIdPcCb0OHyXnQ5tWgtNUDNOx832LjSan3yGqxRZKCFp0nBsjcKRVh Pgz2oZbuJClDOF39iivLJnQUfsqrMsVrr9HPxC4s0gNCcZNN06WwsPVa2emrQm0VhFA3nmCynBnru PboQJRG09b2WBU/Po66qPtV0+6K8k0mY0r1NbYJrrYOhrI5xQNhb36TNaf6S7wKY5wNct7PvD9dNM tsSGoJ0eA==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kiJeR-0000Uz-DM; Thu, 26 Nov 2020 15:57:43 +0000 Received: from mail-wr1-x443.google.com ([2a00:1450:4864:20::443]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kiJbf-0007kE-Gn for linux-arm-kernel@lists.infradead.org; Thu, 26 Nov 2020 15:54:52 +0000 Received: by mail-wr1-x443.google.com with SMTP id 64so2616835wra.11 for ; Thu, 26 Nov 2020 07:54:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=e2DA2lb08NjF2kLgte3niLs4IyK5ibsw6X8kGTbt6og=; b=vuRWz2VzCx+NsTyFxiM9DdVajdfpccRX2BWXmB2CRIKJ8IzexQMvbefFsjGqaWVBCk jWYzf55/dDIw1J5itVsU6h1tTPCWRlLRysQ7+3qrYy+EoIg+fWGWDuwzw2ZQVAsIjn0F tl1qerp25yz+79MsbHXnIG7eE0AEPzyspl6b05MfemZ1kEW7LD/IP9Z+g6oyvenjQUyR OYRrcPmdT7/xKDifSqGlleIWs/7ENNoHBYcekTQhjra9BKPI8jSuBf0moV3uceCgIGTE ND4L4ysHhmtYmCnqlon52bBx8YPNEqSXVO2o6mzj9yoL1P2LxOMw54fSC2rF/y7o5xDt LJYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=e2DA2lb08NjF2kLgte3niLs4IyK5ibsw6X8kGTbt6og=; b=jeJoQPhsxdppswpCBNeKdizoXIERBjn96HuY/58tmnelN718wVwwUmoaGyfVpsLMSh OpGXLYlXwfe+q7QHspzVu/RHqEqDk91WRfLsJ+IPPsdtd6gtnIqWkiRPNM6ByauVPUzy OeLlICI1MpNjH14GWbG72+JybTg38mc/7JPJuk1ZvDJ6dpgHHpAoye4dUKCNXEs9PUdX W3fv+SwzUtRBGPSnIZb2ns7CfG2NHfqCnSkGGSGe0E4yddVxhQ5C2FtMp6Y1+ZRogV4q MMLmcE5vrkhLaiaexODqyvePJJiDYHzkrgg8XrlFWxMHi6yzsbMOusgJFj9LUhxrg+T9 hgPw== X-Gm-Message-State: AOAM533nH6i4K0VnJgJTMB/8YXtJF76IhEs4S1MMERYgYWFKWSIG5Ovk T9msUpSiWPHqYIaUMK3ba7R0/A== X-Google-Smtp-Source: ABdhPJx6VhjNHKBFfpNR31U6UDoq1VL+sz6rt7bUK0/RH8aBigcXdj9z3q8fmfoUnFbrGm/vavCgnA== X-Received: by 2002:a5d:4349:: with SMTP id u9mr4496622wrr.319.1606406090191; Thu, 26 Nov 2020 07:54:50 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:f008:704d:8d4b:9951]) by smtp.gmail.com with ESMTPSA id d8sm8608344wmb.11.2020.11.26.07.54.48 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 26 Nov 2020 07:54:49 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v3 11/23] kvm: arm64: Support per_cpu_ptr in nVHE hyp code Date: Thu, 26 Nov 2020 15:54:09 +0000 Message-Id: <20201126155421.14901-12-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201126155421.14901-1-dbrazdil@google.com> References: <20201126155421.14901-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201126_105451_691704_B501B331 X-CRM114-Status: GOOD ( 21.19 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When compiling with __KVM_NVHE_HYPERVISOR__ redefine per_cpu_offset() to __hyp_per_cpu_offset() which looks up the base of the nVHE per-CPU region of the given cpu and computes its offset from the .hyp.data..percpu section. This enables use of per_cpu_ptr() helpers in nVHE hyp code. Until now only this_cpu_ptr() was supported by setting TPIDR_EL2. Signed-off-by: David Brazdil --- arch/arm64/include/asm/percpu.h | 6 ++++++ arch/arm64/kernel/image-vars.h | 3 +++ arch/arm64/kvm/hyp/nvhe/Makefile | 3 ++- arch/arm64/kvm/hyp/nvhe/hyp-smp.c | 24 ++++++++++++++++++++++++ 4 files changed, 35 insertions(+), 1 deletion(-) create mode 100644 arch/arm64/kvm/hyp/nvhe/hyp-smp.c diff --git a/arch/arm64/include/asm/percpu.h b/arch/arm64/include/asm/percpu.h index 1599e17379d8..8f1661603b78 100644 --- a/arch/arm64/include/asm/percpu.h +++ b/arch/arm64/include/asm/percpu.h @@ -239,6 +239,12 @@ PERCPU_RET_OP(add, add, ldadd) #define this_cpu_cmpxchg_8(pcp, o, n) \ _pcp_protect_return(cmpxchg_relaxed, pcp, o, n) +#ifdef __KVM_NVHE_HYPERVISOR__ +extern unsigned long __hyp_per_cpu_offset(unsigned int cpu); +#define __per_cpu_offset +#define per_cpu_offset(cpu) __hyp_per_cpu_offset((cpu)) +#endif + #include /* Redefine macros for nVHE hyp under DEBUG_PREEMPT to avoid its dependencies. */ diff --git a/arch/arm64/kernel/image-vars.h b/arch/arm64/kernel/image-vars.h index 4b32588918d9..8539f34d7538 100644 --- a/arch/arm64/kernel/image-vars.h +++ b/arch/arm64/kernel/image-vars.h @@ -102,6 +102,9 @@ KVM_NVHE_ALIAS(gic_nonsecure_priorities); KVM_NVHE_ALIAS(__start___kvm_ex_table); KVM_NVHE_ALIAS(__stop___kvm_ex_table); +/* Array containing bases of nVHE per-CPU memory regions. */ +KVM_NVHE_ALIAS(kvm_arm_hyp_percpu_base); + #endif /* CONFIG_KVM */ #endif /* __ARM64_KERNEL_IMAGE_VARS_H */ diff --git a/arch/arm64/kvm/hyp/nvhe/Makefile b/arch/arm64/kvm/hyp/nvhe/Makefile index 77b8c4e06f2f..cf11f8182756 100644 --- a/arch/arm64/kvm/hyp/nvhe/Makefile +++ b/arch/arm64/kvm/hyp/nvhe/Makefile @@ -6,7 +6,8 @@ asflags-y := -D__KVM_NVHE_HYPERVISOR__ ccflags-y := -D__KVM_NVHE_HYPERVISOR__ -obj-y := timer-sr.o sysreg-sr.o debug-sr.o switch.o tlb.o hyp-init.o host.o hyp-main.o +obj-y := timer-sr.o sysreg-sr.o debug-sr.o switch.o tlb.o hyp-init.o host.o \ + hyp-main.o hyp-smp.o obj-y += ../vgic-v3-sr.o ../aarch32.o ../vgic-v2-cpuif-proxy.o ../entry.o \ ../fpsimd.o ../hyp-entry.o ../exception.o diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-smp.c b/arch/arm64/kvm/hyp/nvhe/hyp-smp.c new file mode 100644 index 000000000000..c168d86f885a --- /dev/null +++ b/arch/arm64/kvm/hyp/nvhe/hyp-smp.c @@ -0,0 +1,24 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2020 - Google LLC + * Author: David Brazdil + */ + +#include +#include +#include + +unsigned long __hyp_per_cpu_offset(unsigned int cpu) +{ + unsigned long *cpu_base_array; + unsigned long this_cpu_base; + unsigned long elf_base; + + if (cpu >= ARRAY_SIZE(kvm_arm_hyp_percpu_base)) + hyp_panic(); + + cpu_base_array = (unsigned long*)hyp_symbol_addr(kvm_arm_hyp_percpu_base); + this_cpu_base = kern_hyp_va(cpu_base_array[cpu]); + elf_base = (unsigned long)hyp_symbol_addr(__per_cpu_start); + return this_cpu_base - elf_base; +} From patchwork Thu Nov 26 15:54:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11934319 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 78764C56202 for ; Thu, 26 Nov 2020 16:00:19 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id EB43E221E2 for ; Thu, 26 Nov 2020 16:00:18 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="AuGpewYj"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="jqLq4mvu" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EB43E221E2 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=OD/orqvMMBfEaqEfbyC1680dTXQAsCqddNa2TY3iuuU=; b=AuGpewYjUclu1FX9V5lkw5gpQ f4sl8OxZ7/ZGPZ81V9880PyNyeW2BQwsUE+EfTLqmrbSnsGqrc38jAmgPGTs6TIsU1AjxytW4A4Me xoP0HLEobv8Wv4ZuPH0VYgKm4UHistzfcnYkA5OmCqfrZC6FXFxgNgXDtMgTKjjxz9yU4ZQsZH6sj 6TyCaKfcYhUMlDr2Ru0eIp3qPq2FClW9zVsdEMe8zDPYpC3H6ZTL+qf4NYOhtAC5nGurXfTehX2q9 se9xByT6zG51hFBB0JoYTXQecP/fC5Q5GwWD2Ljmkh1tEEIdjlb2LJsrHuc3nDhoL1Z7j4m+gbHG3 Bn+SBaJ9A==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kiJef-0000bn-4S; Thu, 26 Nov 2020 15:57:57 +0000 Received: from mail-wr1-x441.google.com ([2a00:1450:4864:20::441]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kiJbh-0007l2-JO for linux-arm-kernel@lists.infradead.org; Thu, 26 Nov 2020 15:54:54 +0000 Received: by mail-wr1-x441.google.com with SMTP id i2so2641994wrs.4 for ; Thu, 26 Nov 2020 07:54:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=abHZKvAnOIOh0cfJUm3aASRkjBMLNkdSVRPPeTy9mxo=; b=jqLq4mvuFUHRNYTuDA8X4VqzR999IfDP4q84W39BYEKyBQxNgrPCGYrczkPYGhK3xu rAyOQQer3BhE9duh1aWdFCeG+6knjdw5TuROmYdg4HhznGrigbB06JqMnjAT7xfK+est C3hOyQ9Hz6DwLFMNdCGX7L+t1pUqK9ugYB0c9EsrTz5zLyPmstSDlVJslLskorCLfkLJ xnDwNLr1x8JteRXu7nG+IZYBSzTE8FzQbQlInhi+UOtUpSqxM/09so4wFxOqzOzG7W3k fMq1ulRSMT+FhtRSYbgZxKC8qiIf4rnRr+xoJ9PrqtwZUt/DvnRvDjiCJXDqxMKOdZfk IUcA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=abHZKvAnOIOh0cfJUm3aASRkjBMLNkdSVRPPeTy9mxo=; b=NRvuOXo1B+ETbYLVgeSliSqUm9NSz2tWoz5ILX0cKxUYctxny3EDBVbPjl174ovr4y 1CYSCDwCvfcjzTcYtAg2739P/Y9wFMxI9tS8u+m+7ColbFXqExzz2MKGBbptafCL6qMB vshUK740lhqLwxT/Nnafo2t7b+aL03rsHFvb0vrU7ZAfI9OzV9DQ7X3UFmkP9qt9heCd 06zaPNM4GF9D9NFcEQeRJ6MLSBzpBEBl1l8pHwZNmVim1Zh8WQGZqukyDQHOgqzpbx8t puOBCMG6e96Qbq044m/4QRQMcomOgt3wEJgiVGdwwU4qvm9rmA+t/guGmEYzxKeKoFsD gSPw== X-Gm-Message-State: AOAM533HWfulw3oA1M0PAnv7c4DO6Dte2P+G9RuYdxrqlWNgh+/NeBZh Y77CSV7FyMuVSbeR8K5IQJlvhQ== X-Google-Smtp-Source: ABdhPJzBk1cgJvigLTBpzrkzyw93ozQjhaI8GSxKK42oABc32zYGurmL/AUO7+tHdNOzK5VOfFnjDw== X-Received: by 2002:adf:dc83:: with SMTP id r3mr4498991wrj.223.1606406092266; Thu, 26 Nov 2020 07:54:52 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:f008:704d:8d4b:9951]) by smtp.gmail.com with ESMTPSA id r13sm4193297wrm.25.2020.11.26.07.54.50 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 26 Nov 2020 07:54:51 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v3 12/23] kvm: arm64: Create nVHE copy of cpu_logical_map Date: Thu, 26 Nov 2020 15:54:10 +0000 Message-Id: <20201126155421.14901-13-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201126155421.14901-1-dbrazdil@google.com> References: <20201126155421.14901-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201126_105453_786812_B39F7DFA X-CRM114-Status: GOOD ( 20.68 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When KVM starts validating host's PSCI requests, it will need to map MPIDR back to the CPU ID. To this end, copy cpu_logical_map into nVHE hyp memory when KVM is initialized. Only copy the information for CPUs that are online at the point of KVM initialization so that KVM rejects CPUs whose features were not checked against the finalized capabilities. Signed-off-by: David Brazdil --- arch/arm64/kvm/arm.c | 19 +++++++++++++++++++ arch/arm64/kvm/hyp/nvhe/hyp-smp.c | 16 ++++++++++++++++ 2 files changed, 35 insertions(+) diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index bb07f0401c68..bb3c541d3ddb 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -61,6 +61,8 @@ static bool vgic_present; static DEFINE_PER_CPU(unsigned char, kvm_arm_hardware_enabled); DEFINE_STATIC_KEY_FALSE(userspace_irqchip_in_use); +extern u64 kvm_nvhe_sym(__cpu_logical_map)[NR_CPUS]; + int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu) { return kvm_vcpu_exiting_guest_mode(vcpu) == IN_GUEST_MODE; @@ -1531,6 +1533,20 @@ static inline void hyp_cpu_pm_exit(void) } #endif +static void init_cpu_logical_map(void) +{ + unsigned int cpu; + + /* + * Copy the MPIDR <-> logical CPU ID mapping to hyp. + * Only copy the set of online CPUs whose features have been chacked + * against the finalized system capabilities. The hypervisor will not + * allow any other CPUs from the `possible` set to boot. + */ + for_each_online_cpu(cpu) + kvm_nvhe_sym(__cpu_logical_map)[cpu] = cpu_logical_map(cpu); +} + static int init_common_resources(void) { return kvm_set_ipa_limit(); @@ -1703,6 +1719,9 @@ static int init_hyp_mode(void) } } + if (is_protected_kvm_enabled()) + init_cpu_logical_map(); + return 0; out_err: diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-smp.c b/arch/arm64/kvm/hyp/nvhe/hyp-smp.c index c168d86f885a..ceb427aabb91 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-smp.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-smp.c @@ -8,6 +8,22 @@ #include #include +/* + * nVHE copy of data structures tracking available CPU cores. + * Only entries for CPUs that were online at KVM init are populated. + * Other CPUs should not be allowed to boot because their features were + * not checked against the finalized system capabilities. + */ +u64 __ro_after_init __cpu_logical_map[NR_CPUS] = { [0 ... NR_CPUS-1] = INVALID_HWID }; + +u64 cpu_logical_map(unsigned int cpu) +{ + if (cpu >= ARRAY_SIZE(__cpu_logical_map)) + hyp_panic(); + + return __cpu_logical_map[cpu]; +} + unsigned long __hyp_per_cpu_offset(unsigned int cpu) { unsigned long *cpu_base_array; From patchwork Thu Nov 26 15:54:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11934321 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E2983C63798 for ; Thu, 26 Nov 2020 16:00:20 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7AC8C21D40 for ; Thu, 26 Nov 2020 16:00:20 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="VEV9tGBG"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="qZUzcyIm" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7AC8C21D40 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=0DX34Z3K07kc2tiK9KBThM6u8cjR9+iK/qDpNZDhUXw=; b=VEV9tGBGgUv2ykllRTHv2Pul2 bH0nYtpQ8ZY9naj7KvFeVXtAoWUQvdoNtSUcjddXZI3Y0JG7TVSoecll09TGJyKu4VS8V2x6S0q2z 3BmsffvrYAFDerPSyhqETsGbO5sBW/iSNep/3JjamDo6EvNDGeW9TWJ7pq1GZnlcBH/pZYxSgU7cm oLYt3MCpbjUIUYq9VZt7/iBDHPPF0ucLB6YfFIXdEG58gJhZ9JlkKNJSvakfH68iv6ZjnksEr0aXz cur2EBG/iGpeG0FiNlDQlRLZdUNq9jYR7O1W7VE8D81D6GNV6iSfPfLfUnaD3bzSj4nyB3n/POBJL 6h8fkAVIA==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kiJf3-0000le-Ko; Thu, 26 Nov 2020 15:58:21 +0000 Received: from mail-wr1-x442.google.com ([2a00:1450:4864:20::442]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kiJbj-0007lr-Rz for linux-arm-kernel@lists.infradead.org; Thu, 26 Nov 2020 15:54:57 +0000 Received: by mail-wr1-x442.google.com with SMTP id 23so2635814wrc.8 for ; Thu, 26 Nov 2020 07:54:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=oagbJM5Ax/HFickgEo8A1TkVP1JuhAjLvznvg3GZEXE=; b=qZUzcyImvGBn0yrIjOeE6dvV8TTcOgLZ9npbYQvvt7NZJZCYO36QukjaKQ65C91yOa s6tgkRFbVNUlbAQyM/7pZ4EJB2V1rzieMnwHNy9Z3C8bHxhWy5lulXiAWX4nIFRcgwnZ 8JDUw1NkODEBzf/2+Sl5ZC+NtbVdmIe1Xa1R1EWCiKT9iNFd+bW9iMhp7tUNrPDdcfEm IcFm9PjetQN+jfEzQ/p6X99K1DbXM6md/Mno9gTBZRSKaaY4v5CUq9wI3TH+kvyuGbDF NQq4JQ7SE57CJdE5n7JT8TupuWKDm/pQEnVdnWQ7o+BP0HzGpEcUNS9YN0Si+lO2uRD8 Xhlg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=oagbJM5Ax/HFickgEo8A1TkVP1JuhAjLvznvg3GZEXE=; b=mxBDagXVvo+pbfHxVeqFXxH5ZB5stCcE4b2M7v2vkddAKVEG2/EvkOMh+F4HTZ5dXx eZW8q+Af1kK0pS9JuGJYb9BbrnDmHj7B1a2sFpHcfxkKYejqKZhzIP6QneRiPARgvthX opx/F7KUisgdeDJF31PM0zqWy05ZtzmEXexZBVQihB8sZbqMXgxsh+3N7bQYPFJKpyVy r2nWGJpS6p5udCaXmEqtQ1Ct2ffis1TxIlEGtcnm92xu21tTU6++/V2dsG3Y/A8OWJyP Gmc5UgPCT9Q0gEiDqPMQjTFb1N7AEHapwOFmKLZoW/+nn6IIh6w7BD2h3bLdsfQLqexY 0H/g== X-Gm-Message-State: AOAM533tL1Tm20TQiitIpeIpY0DYQIcquiyNN/yg9nN7wDHUmRzVZWJj bIPzAqgcot/D956MjIYJvfMLoA== X-Google-Smtp-Source: ABdhPJzN6596YemtZxOQnXDEasriPsfnG62j/YEP9/S71/zP7vvdUkweGbmtUDi3GsZaFW91d4W0tQ== X-Received: by 2002:adf:f882:: with SMTP id u2mr4653371wrp.271.1606406094562; Thu, 26 Nov 2020 07:54:54 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:f008:704d:8d4b:9951]) by smtp.gmail.com with ESMTPSA id o203sm2654638wmb.0.2020.11.26.07.54.53 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 26 Nov 2020 07:54:53 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v3 13/23] kvm: arm64: Add SMC handler in nVHE EL2 Date: Thu, 26 Nov 2020 15:54:11 +0000 Message-Id: <20201126155421.14901-14-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201126155421.14901-1-dbrazdil@google.com> References: <20201126155421.14901-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201126_105455_985671_2A0A7D9A X-CRM114-Status: GOOD ( 18.79 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add handler of host SMCs in KVM nVHE trap handler. Forward all SMCs to EL3 and propagate the result back to EL1. This is done in preparation for validating host SMCs in KVM nVHE protected mode. The implementation assumes that firmware uses SMCCC v1.2 or older. That means x0-x17 can be used both for arguments and results, other GPRs are preserved. Signed-off-by: David Brazdil --- arch/arm64/kvm/hyp/nvhe/host.S | 38 ++++++++++++++++++++++++++++++ arch/arm64/kvm/hyp/nvhe/hyp-main.c | 35 ++++++++++++++++++++++++--- 2 files changed, 70 insertions(+), 3 deletions(-) diff --git a/arch/arm64/kvm/hyp/nvhe/host.S b/arch/arm64/kvm/hyp/nvhe/host.S index fe2740b224cf..2b56f0bdf874 100644 --- a/arch/arm64/kvm/hyp/nvhe/host.S +++ b/arch/arm64/kvm/hyp/nvhe/host.S @@ -180,3 +180,41 @@ SYM_CODE_START(__kvm_hyp_host_vector) invalid_host_el1_vect // FIQ 32-bit EL1 invalid_host_el1_vect // Error 32-bit EL1 SYM_CODE_END(__kvm_hyp_host_vector) + +/* + * Forward SMC with arguments in struct kvm_cpu_context, and + * store the result into the same struct. Assumes SMCCC 1.2 or older. + * + * x0: struct kvm_cpu_context* + */ +SYM_CODE_START(__kvm_hyp_host_forward_smc) + /* + * Use x18 to keep the pointer to the host context because + * x18 is callee-saved in SMCCC but not in AAPCS64. + */ + mov x18, x0 + + ldp x0, x1, [x18, #CPU_XREG_OFFSET(0)] + ldp x2, x3, [x18, #CPU_XREG_OFFSET(2)] + ldp x4, x5, [x18, #CPU_XREG_OFFSET(4)] + ldp x6, x7, [x18, #CPU_XREG_OFFSET(6)] + ldp x8, x9, [x18, #CPU_XREG_OFFSET(8)] + ldp x10, x11, [x18, #CPU_XREG_OFFSET(10)] + ldp x12, x13, [x18, #CPU_XREG_OFFSET(12)] + ldp x14, x15, [x18, #CPU_XREG_OFFSET(14)] + ldp x16, x17, [x18, #CPU_XREG_OFFSET(16)] + + smc #0 + + stp x0, x1, [x18, #CPU_XREG_OFFSET(0)] + stp x2, x3, [x18, #CPU_XREG_OFFSET(2)] + stp x4, x5, [x18, #CPU_XREG_OFFSET(4)] + stp x6, x7, [x18, #CPU_XREG_OFFSET(6)] + stp x8, x9, [x18, #CPU_XREG_OFFSET(8)] + stp x10, x11, [x18, #CPU_XREG_OFFSET(10)] + stp x12, x13, [x18, #CPU_XREG_OFFSET(12)] + stp x14, x15, [x18, #CPU_XREG_OFFSET(14)] + stp x16, x17, [x18, #CPU_XREG_OFFSET(16)] + + ret +SYM_CODE_END(__kvm_hyp_host_forward_smc) diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c index a4f1cac714d7..f25680ede080 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -18,6 +18,8 @@ DEFINE_PER_CPU(struct kvm_nvhe_init_params, kvm_init_params); +void __kvm_hyp_host_forward_smc(struct kvm_cpu_context *host_ctxt); + static void handle___kvm_vcpu_run(struct kvm_cpu_context *host_ctxt) { DECLARE_REG(struct kvm_vcpu *, vcpu, host_ctxt, 1); @@ -152,12 +154,39 @@ static void handle_host_hcall(struct kvm_cpu_context *host_ctxt) cpu_reg(host_ctxt, 0) = SMCCC_RET_NOT_SUPPORTED; } +static void default_host_smc_handler(struct kvm_cpu_context *host_ctxt) +{ + __kvm_hyp_host_forward_smc(host_ctxt); +} + +static void skip_host_instruction(void) +{ + write_sysreg_el2(read_sysreg_el2(SYS_ELR) + 4, SYS_ELR); +} + +static void handle_host_smc(struct kvm_cpu_context *host_ctxt) +{ + default_host_smc_handler(host_ctxt); + + /* + * Unlike HVC, the return address of an SMC is the instruction's PC. + * Move the return address past the instruction. + */ + skip_host_instruction(); +} + void handle_trap(struct kvm_cpu_context *host_ctxt) { u64 esr = read_sysreg_el2(SYS_ESR); - if (unlikely(ESR_ELx_EC(esr) != ESR_ELx_EC_HVC64)) + switch (ESR_ELx_EC(esr)) { + case ESR_ELx_EC_HVC64: + handle_host_hcall(host_ctxt); + break; + case ESR_ELx_EC_SMC64: + handle_host_smc(host_ctxt); + break; + default: hyp_panic(); - - handle_host_hcall(host_ctxt); + } } From patchwork Thu Nov 26 15:54:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11934325 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 83E95C56202 for ; Thu, 26 Nov 2020 16:00:46 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D215121D7E for ; Thu, 26 Nov 2020 16:00:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="UveuE64m"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="HwVGV83S" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D215121D7E Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=VbGe6U3Ve80Wo6wzCeC3I4s21QTObzlaaVE3rX8ksmI=; b=UveuE64mn7Yo5UWpePSRzeTyl IsuumMBz62vFJT+nfqRPiwQ4DJy9wuoIwBVGMznwj9JODOkvffmo+3dofXR2J7QwBaCDdp21wakit I7v032vFn1zffT1XO3O/cehj6h5k2qvsA72E5RZ6CAMoSk9gIeaKRf37muyqd8yjTyFUsvd5J+m2f wbSJneawIqYfgqgHPJYVBw4e7RQU3BH0mIEkahNzfGWVw8sXcIEMv3ORAFSomNQ9oUheZ4uMGb/wC dvN7bB4q/jPdD/pGm1/vLETyv8xfBEMVqS9W7KSQeM3V6x4i9tGNYwbTcnK3a+ygbg1I6/wgiBhDr UFCH3gz2w==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kiJf8-0000pM-UY; Thu, 26 Nov 2020 15:58:27 +0000 Received: from mail-wr1-x435.google.com ([2a00:1450:4864:20::435]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kiJbn-0007mj-Gr for linux-arm-kernel@lists.infradead.org; Thu, 26 Nov 2020 15:55:03 +0000 Received: by mail-wr1-x435.google.com with SMTP id s8so2623070wrw.10 for ; Thu, 26 Nov 2020 07:54:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=96Y1Pg8NboBYaC6v7mMq4UlUhEivDMpMpRFkrQ3cNDg=; b=HwVGV83SA1pu0EkNWbTp3SpTcyRYOohY63MbncNrkmkpg6bhNUNdS9EwDWsj4Ce67U sX5d/patePE2huLGqwQ6Jie1Q1vG6nL70ED1869nNdm/tCnLnbOUUuaYuYxRvLqWTWlA n+pKGCOOAOUj9OioMDgZ7dFik62hQ4KtzRO8SEpOreGSG4WJdXWKsjeWr0PpEZJ31KQ0 uyakkl+o8i8hSlOLz7STTfHEfJv8cyfx/WkL14oX8ruk6bAw7+AQ7Rq5PEW1A0HMgDKU jphMf3l5iWH8GZG+N9JC+Q0uwPhBd/wdTIHnjLR53pqRZ9C0utsotAEPqbV8Rdju9kFI hegw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=96Y1Pg8NboBYaC6v7mMq4UlUhEivDMpMpRFkrQ3cNDg=; b=VuSEVW0kL5fm4NR8SwVCUEJHuv1kjvfReMKnGiUfxCFiuq+kQmIWam/73JBbbHiuK1 O9k4ohu4KgaDI5OkNV4HPIK9fqzkyLQm8vJklAqtoWsq3WNVLw0HVW+qg28YDAncSoJL i2aXumpcwoA5uLrxrqPH682LxqjMH+dic42lbvCELhdK7mYVbZF9JbhsPFjbr5c9cwqf BqDrdRrO58CPvO9gx6lpeA9Kc5xm72hYtgAHZr/zPzpKn8GKvcj5t5QtJEpvHu00Yw6g fzVcFxg357rT4Y4Vvak2WXX49iREOEX38XLcDvFI0DDSBwhje2QsG3564C71kWByyZ+M kJtA== X-Gm-Message-State: AOAM5333bQL+/GfuaZ3lXV6FtYmsQQqyTVWJLA2E7Uv/a6AjBbjiJW27 Ym+/s6gWzQN37DuzHpUU8rLE+g== X-Google-Smtp-Source: ABdhPJxZM90ubqSRiYq0ls0OqUo2eyyMniWNiLiTqx8tmAhlFoXELKzvYp156Crsd8Y4IBGxE+f1vA== X-Received: by 2002:a5d:570d:: with SMTP id a13mr4513622wrv.193.1606406096673; Thu, 26 Nov 2020 07:54:56 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:f008:704d:8d4b:9951]) by smtp.gmail.com with ESMTPSA id o4sm8751771wmh.33.2020.11.26.07.54.55 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 26 Nov 2020 07:54:55 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v3 14/23] kvm: arm64: Bootstrap PSCI SMC handler in nVHE EL2 Date: Thu, 26 Nov 2020 15:54:12 +0000 Message-Id: <20201126155421.14901-15-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201126155421.14901-1-dbrazdil@google.com> References: <20201126155421.14901-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201126_105459_696520_18695E38 X-CRM114-Status: GOOD ( 29.45 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add a handler of PSCI SMCs in nVHE hyp code. The handler is initialized with the version used by the host's PSCI driver and the function IDs it was configured with. If the SMC function ID matches one of the configured PSCI calls (for v0.1) or falls into the PSCI function ID range (for v0.2+), the SMC is handled by the PSCI handler. For now, all SMCs return PSCI_RET_NOT_SUPPORTED. Signed-off-by: David Brazdil --- arch/arm64/include/asm/kvm_hyp.h | 2 + arch/arm64/kvm/arm.c | 29 ++++- .../arm64/kvm/hyp/include/nvhe/trap_handler.h | 18 +++ arch/arm64/kvm/hyp/nvhe/Makefile | 2 +- arch/arm64/kvm/hyp/nvhe/hyp-main.c | 10 +- arch/arm64/kvm/hyp/nvhe/psci-relay.c | 103 ++++++++++++++++++ 6 files changed, 158 insertions(+), 6 deletions(-) create mode 100644 arch/arm64/kvm/hyp/include/nvhe/trap_handler.h create mode 100644 arch/arm64/kvm/hyp/nvhe/psci-relay.c diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h index cb25c15e3d8d..c0450828378b 100644 --- a/arch/arm64/include/asm/kvm_hyp.h +++ b/arch/arm64/include/asm/kvm_hyp.h @@ -93,6 +93,8 @@ void deactivate_traps_vhe_put(void); u64 __guest_enter(struct kvm_vcpu *vcpu); +bool kvm_host_psci_handler(struct kvm_cpu_context *host_ctxt); + void __noreturn hyp_panic(void); #ifdef __KVM_NVHE_HYPERVISOR__ void __noreturn __hyp_do_panic(bool restore_host, u64 spsr, u64 elr, u64 par); diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index bb3c541d3ddb..7a17b5048454 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -19,6 +19,7 @@ #include #include #include +#include #include #define CREATE_TRACE_POINTS @@ -62,6 +63,8 @@ static DEFINE_PER_CPU(unsigned char, kvm_arm_hardware_enabled); DEFINE_STATIC_KEY_FALSE(userspace_irqchip_in_use); extern u64 kvm_nvhe_sym(__cpu_logical_map)[NR_CPUS]; +extern u32 kvm_nvhe_sym(kvm_host_psci_version); +extern u32 kvm_nvhe_sym(kvm_host_psci_function_id)[PSCI_FN_MAX]; int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu) { @@ -1547,6 +1550,26 @@ static void init_cpu_logical_map(void) kvm_nvhe_sym(__cpu_logical_map)[cpu] = cpu_logical_map(cpu); } +static bool init_psci_relay(void) +{ + int i; + + /* + * If PSCI has not been initialized, protected KVM cannot install + * itself on newly booted CPUs. + */ + if (!psci_ops.get_version) { + kvm_err("Cannot initialize protected mode without PSCI\n"); + return false; + } + + kvm_nvhe_sym(kvm_host_psci_version) = psci_ops.get_version(); + for (i = 0; i < PSCI_FN_MAX; ++i) + kvm_nvhe_sym(kvm_host_psci_function_id)[i] = psci_get_function_id(i); + + return true; +} + static int init_common_resources(void) { return kvm_set_ipa_limit(); @@ -1719,9 +1742,13 @@ static int init_hyp_mode(void) } } - if (is_protected_kvm_enabled()) + if (is_protected_kvm_enabled()) { init_cpu_logical_map(); + if (!init_psci_relay()) + goto out_err; + } + return 0; out_err: diff --git a/arch/arm64/kvm/hyp/include/nvhe/trap_handler.h b/arch/arm64/kvm/hyp/include/nvhe/trap_handler.h new file mode 100644 index 000000000000..92d34675acdb --- /dev/null +++ b/arch/arm64/kvm/hyp/include/nvhe/trap_handler.h @@ -0,0 +1,18 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Trap handler helpers. + * + * Copyright (C) 2020 - Google LLC + * Author: Marc Zyngier + */ + +#ifndef __ARM64_KVM_NVHE_TRAP_HANDLER_H__ +#define __ARM64_KVM_NVHE_TRAP_HANDLER_H__ + +#include + +#define cpu_reg(ctxt, r) (ctxt)->regs.regs[r] +#define DECLARE_REG(type, name, ctxt, reg) \ + type name = (type)cpu_reg(ctxt, (reg)) + +#endif /* __ARM64_KVM_NVHE_TRAP_HANDLER_H__ */ diff --git a/arch/arm64/kvm/hyp/nvhe/Makefile b/arch/arm64/kvm/hyp/nvhe/Makefile index cf11f8182756..1f1e351c5fe2 100644 --- a/arch/arm64/kvm/hyp/nvhe/Makefile +++ b/arch/arm64/kvm/hyp/nvhe/Makefile @@ -7,7 +7,7 @@ asflags-y := -D__KVM_NVHE_HYPERVISOR__ ccflags-y := -D__KVM_NVHE_HYPERVISOR__ obj-y := timer-sr.o sysreg-sr.o debug-sr.o switch.o tlb.o hyp-init.o host.o \ - hyp-main.o hyp-smp.o + hyp-main.o hyp-smp.o psci-relay.o obj-y += ../vgic-v3-sr.o ../aarch32.o ../vgic-v2-cpuif-proxy.o ../entry.o \ ../fpsimd.o ../hyp-entry.o ../exception.o diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c index f25680ede080..bde658d51404 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -12,9 +12,7 @@ #include #include -#define cpu_reg(ctxt, r) (ctxt)->regs.regs[r] -#define DECLARE_REG(type, name, ctxt, reg) \ - type name = (type)cpu_reg(ctxt, (reg)) +#include DEFINE_PER_CPU(struct kvm_nvhe_init_params, kvm_init_params); @@ -166,7 +164,11 @@ static void skip_host_instruction(void) static void handle_host_smc(struct kvm_cpu_context *host_ctxt) { - default_host_smc_handler(host_ctxt); + bool handled; + + handled = kvm_host_psci_handler(host_ctxt); + if (!handled) + default_host_smc_handler(host_ctxt); /* * Unlike HVC, the return address of an SMC is the instruction's PC. diff --git a/arch/arm64/kvm/hyp/nvhe/psci-relay.c b/arch/arm64/kvm/hyp/nvhe/psci-relay.c new file mode 100644 index 000000000000..44a9fb462d24 --- /dev/null +++ b/arch/arm64/kvm/hyp/nvhe/psci-relay.c @@ -0,0 +1,103 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2020 - Google LLC + * Author: David Brazdil + */ + +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +/* Config options set by the host. */ +u32 __ro_after_init kvm_host_psci_version; +u32 __ro_after_init kvm_host_psci_function_id[PSCI_FN_MAX]; + +static u64 get_psci_func_id(struct kvm_cpu_context *host_ctxt) +{ + DECLARE_REG(u64, func_id, host_ctxt, 0); + + return func_id; +} + +static bool is_psci_0_1_call(u64 func_id) +{ + unsigned int i; + + for (i = 0; i < ARRAY_SIZE(kvm_host_psci_function_id); ++i) { + if (func_id == kvm_host_psci_function_id[i]) + return true; + } + return false; +} + +static bool is_psci_0_2_call(u64 func_id) +{ + /* SMCCC reserves IDs 0x00-1F with the given 32/64-bit base for PSCI. */ + return (PSCI_0_2_FN(0) <= func_id && func_id <= PSCI_0_2_FN(31)) || + (PSCI_0_2_FN64(0) <= func_id && func_id <= PSCI_0_2_FN64(31)); +} + +static bool is_psci_call(u64 func_id) +{ + switch (kvm_host_psci_version) { + case PSCI_VERSION(0, 1): + return is_psci_0_1_call(func_id); + default: + return is_psci_0_2_call(func_id); + } +} + +static unsigned long psci_0_1_handler(u64 func_id, struct kvm_cpu_context *host_ctxt) +{ + return PSCI_RET_NOT_SUPPORTED; +} + +static unsigned long psci_0_2_handler(u64 func_id, struct kvm_cpu_context *host_ctxt) +{ + switch (func_id) { + default: + return PSCI_RET_NOT_SUPPORTED; + } +} + +static unsigned long psci_1_0_handler(u64 func_id, struct kvm_cpu_context *host_ctxt) +{ + switch (func_id) { + default: + return psci_0_2_handler(func_id, host_ctxt); + } +} + +bool kvm_host_psci_handler(struct kvm_cpu_context *host_ctxt) +{ + u64 func_id = get_psci_func_id(host_ctxt); + unsigned long ret; + + if (!is_psci_call(func_id)) + return false; + + switch (kvm_host_psci_version) { + case PSCI_VERSION(0, 1): + ret = psci_0_1_handler(func_id, host_ctxt); + break; + case PSCI_VERSION(0, 2): + ret = psci_0_2_handler(func_id, host_ctxt); + break; + default: + ret = psci_1_0_handler(func_id, host_ctxt); + break; + } + + cpu_reg(host_ctxt, 0) = ret; + cpu_reg(host_ctxt, 1) = 0; + cpu_reg(host_ctxt, 2) = 0; + cpu_reg(host_ctxt, 3) = 0; + return true; +} From patchwork Thu Nov 26 15:54:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11934329 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5834AC56202 for ; Thu, 26 Nov 2020 16:00:53 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BF8F1221F1 for ; Thu, 26 Nov 2020 16:00:52 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="aTXJwFP6"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="G+ViC3wP" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BF8F1221F1 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=Hmh82LCFtved76ZGz8HqtW3aJZ4xFyv0DMR+0/r4u1c=; b=aTXJwFP6wlcDP2mzHr25dR1Dk 2VGXuRlwr0Sfrnlinkwk+JOiBK0tbmGeU3woFRAyDRz7OzaM9HEO7HgndEWVQn2Q9ygsDuW8NgurU QCAVofhXlSKTqbHqYkDS+96Xbtiro1IkigLJOwzO1Phti3Fx+Oqo8sSKBPSRgOwMW8qjvLTkeNUkd LDxxnUBzengeqUzsIuf9B6xeCmHMQAv+H5gyHsb7LJOWSWS5IlbXHuscINUbRZ2RFlLPAzhuNDOV+ DlLwzmdwx/rSgYPIJgq1eQyIt4pXrtLWw4qjisYkX3+AN3cGiR+oHQo65lqaKVFv1tUU8O4yNbyuv /KzY9hXRg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kiJfF-0000sb-I9; Thu, 26 Nov 2020 15:58:33 +0000 Received: from mail-wr1-x443.google.com ([2a00:1450:4864:20::443]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kiJbo-0007nT-6U for linux-arm-kernel@lists.infradead.org; Thu, 26 Nov 2020 15:55:06 +0000 Received: by mail-wr1-x443.google.com with SMTP id g14so2605742wrm.13 for ; Thu, 26 Nov 2020 07:54:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=irDOCaoxBC7rYFpDng3539rKfl5EDMRei9R6iacpt5Q=; b=G+ViC3wP1aisQEqSrtFyh4agtf+XRj/S5yCOQSPP+B9cEpF3CEG6DUz/bfIazuPr44 ziINsQardil0WvZ7VvAzP+ViHc8X/aDnyhZgkBWKaSicVgTkjL4koXMMvnWUKwj6Tubf yy+Yx/hK3LwwE2ezmO7wOtcTtG/OXrSJrdAmCkUrUTDGlG25Xz7C8mTrmfItIjst/2Po ItFdY9jyph4RS42NCu18FEzXe2QYDp7fg8a3lGHm5/Bq0J2aPx0WhjJGBCWrdIlkvbXW PRBJQEM/Aw0vr3SF+JCrYwHFypXblFfnJAf0xiWqOkWB+uG+6muEhHq41S+AwQVN50o4 /3AQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=irDOCaoxBC7rYFpDng3539rKfl5EDMRei9R6iacpt5Q=; b=kAD06SHVnbjLB4HwKAeqHMzbh9SLNHG+2NdQrRsVubfwRw5m+PTV8PVfJe2k49wnsP n+J0MlSwh5Qi6HN/l1Qh3ApIdoNjp62s2PktoOjN54hUM2YnYX+iWcIwMxwndhJoXLsm yrfHO4XAkX4c3NfTXpBWlm8suUIxLXY6HDas11E30ZWGrpCmbk7mL6W0dI77xW2BHcQZ fclbK4/MTDi1jjNTMrnlqtxsxoRedAfqB+LwY3vCKS9YxS+nQaJBY+RKEB4m/3ixygkT zz5d5WjanRbihZ2o6UhHXv1r7zdoMsmpwgArHkEffOCzSeMb81PmkPojWIneID5thpxF QyPg== X-Gm-Message-State: AOAM530vlUuyB6TSASTtvcq4+qApxssJ3jMVqshHD8AZLVAZ0OZQURtf /ZYqU6tpF5QIo626fvR5x7/kdA== X-Google-Smtp-Source: ABdhPJy4MBFMpR4y7kRf/sEF0i4eAr3+bsyd1csD1SmT3e7E9ub9c+2WpUvSX9zSC44QRHTnVEfgpQ== X-Received: by 2002:adf:f102:: with SMTP id r2mr4652297wro.315.1606406098667; Thu, 26 Nov 2020 07:54:58 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:f008:704d:8d4b:9951]) by smtp.gmail.com with ESMTPSA id 35sm9328533wro.71.2020.11.26.07.54.57 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 26 Nov 2020 07:54:57 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v3 15/23] kvm: arm64: Add offset for hyp VA <-> PA conversion Date: Thu, 26 Nov 2020 15:54:13 +0000 Message-Id: <20201126155421.14901-16-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201126155421.14901-1-dbrazdil@google.com> References: <20201126155421.14901-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201126_105500_450487_06D23EC8 X-CRM114-Status: GOOD ( 17.60 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add a host-initialized constant to KVM nVHE hyp code for converting between EL2 linear map virtual addresses and physical addresses. Also add `__hyp_pa` macro that performs the conversion. Signed-off-by: David Brazdil --- arch/arm64/kvm/hyp/nvhe/psci-relay.c | 3 +++ arch/arm64/kvm/va_layout.c | 30 +++++++++++++++++++++++++--- 2 files changed, 30 insertions(+), 3 deletions(-) diff --git a/arch/arm64/kvm/hyp/nvhe/psci-relay.c b/arch/arm64/kvm/hyp/nvhe/psci-relay.c index 44a9fb462d24..e7091d89f0fc 100644 --- a/arch/arm64/kvm/hyp/nvhe/psci-relay.c +++ b/arch/arm64/kvm/hyp/nvhe/psci-relay.c @@ -18,6 +18,9 @@ /* Config options set by the host. */ u32 __ro_after_init kvm_host_psci_version; u32 __ro_after_init kvm_host_psci_function_id[PSCI_FN_MAX]; +s64 __ro_after_init hyp_physvirt_offset; + +#define __hyp_pa(x) ((phys_addr_t)((x)) + hyp_physvirt_offset) static u64 get_psci_func_id(struct kvm_cpu_context *host_ctxt) { diff --git a/arch/arm64/kvm/va_layout.c b/arch/arm64/kvm/va_layout.c index 4130b72e6891..d8cc51bd60bf 100644 --- a/arch/arm64/kvm/va_layout.c +++ b/arch/arm64/kvm/va_layout.c @@ -23,6 +23,30 @@ static u8 tag_lsb; static u64 tag_val; static u64 va_mask; +/* + * Compute HYP VA by using the same computation as kern_hyp_va(). + */ +static u64 __early_kern_hyp_va(u64 addr) +{ + addr &= va_mask; + addr |= tag_val << tag_lsb; + return addr; +} + +/* + * Store a hyp VA <-> PA offset into a hyp-owned variable. + */ +static void init_hyp_physvirt_offset(void) +{ + extern s64 kvm_nvhe_sym(hyp_physvirt_offset); + u64 kern_va, hyp_va; + + /* Compute the offset from the hyp VA and PA of a random symbol. */ + kern_va = (u64)kvm_ksym_ref(__hyp_text_start); + hyp_va = __early_kern_hyp_va(kern_va); + CHOOSE_NVHE_SYM(hyp_physvirt_offset) = (s64)__pa(kern_va) - (s64)hyp_va; +} + /* * We want to generate a hyp VA with the following format (with V == * vabits_actual): @@ -54,6 +78,8 @@ __init void kvm_compute_layout(void) tag_val |= get_random_long() & GENMASK_ULL(vabits_actual - 2, tag_lsb); } tag_val >>= tag_lsb; + + init_hyp_physvirt_offset(); } static u32 compute_instruction(int n, u32 rd, u32 rn) @@ -146,9 +172,7 @@ void kvm_patch_vector_branch(struct alt_instr *alt, /* * Compute HYP VA by using the same computation as kern_hyp_va() */ - addr = (uintptr_t)kvm_ksym_ref(__kvm_hyp_vector); - addr &= va_mask; - addr |= tag_val << tag_lsb; + addr = __early_kern_hyp_va((u64)kvm_ksym_ref(__kvm_hyp_vector)); /* Use PC[10:7] to branch to the same vector in KVM */ addr |= ((u64)origptr & GENMASK_ULL(10, 7)); From patchwork Thu Nov 26 15:54:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11934323 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5843CC64E75 for ; Thu, 26 Nov 2020 16:00:46 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A393C21D40 for ; Thu, 26 Nov 2020 16:00:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="kS8jptS/"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="bpKTMIQ4" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A393C21D40 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=bp7xf9ijdhObJUors+ZXWwSx2XHnR0uKyNUKcccJYqU=; b=kS8jptS/G5h7/jsSZiWcRZvNH wwsvGEN3I+NC3WJU0NkBQIi6bOj/VEU4WwhL9BMC2j9a8Yw2kg9WXYwrhW9vR929hULrGh86ErZfx 5aKcn/os0KIKPt0OIOl8ikei2ji/6t8tTR64J2S8LzVEY+L2hTzD/zeykcp+Ioi1X1MihH05nNAda NVeEHVuVThRUV4GXXIDTamRgj0fo86NNOesBzCPQjY8Q5k8hdT0UQLt5UTK5o8w/htsx4xc6YXJcL DGNsTdYQonleBFUiWwUUvPNRv+tH7zOjZKYRkai2JK61YNkm+tXcZZbbACgibCdrUcofikYoensmV A8VmlmDWw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kiJfM-0000wY-2A; Thu, 26 Nov 2020 15:58:40 +0000 Received: from mail-wr1-x444.google.com ([2a00:1450:4864:20::444]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kiJbq-0007oC-KR for linux-arm-kernel@lists.infradead.org; Thu, 26 Nov 2020 15:55:07 +0000 Received: by mail-wr1-x444.google.com with SMTP id g14so2605874wrm.13 for ; Thu, 26 Nov 2020 07:55:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=SfJedK8TmdKV/cyl1d120ny+FNMOujRjMwXhHWy7kEg=; b=bpKTMIQ45FtDnN1sJCDvWAIZ7n/djBhSnLXzmY+GV3rFxtQaEk/jg/HB36CWMm495r 9dyFHXoPlu/6QutYqpZJmOg6wUa99TMcYimuqBqgMfHfcuSGIsViQw1MXVzV82LPd82i Pj43fk2YHVp4/nrQzaAV15yi8dEl9y91D9o67oVTUxotstSuybJnE7DzvY+syCsPyZ0r i7K6L+NiAdodloJMqgZapUI8KuVSrpzLBeQTSEwO+ivclJL7Bslw5QTQUjmKfRssPExa yYzn1XdS7tNvkiZzCkmHrTStfaA2Bdj1iMTHoEVd8DzcAOU0FEV8TW3U03uWNpqgVrWJ v3xw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=SfJedK8TmdKV/cyl1d120ny+FNMOujRjMwXhHWy7kEg=; b=hlco/I4H113GlyhRpD62SUuBH1dZZsYt5DQYVemm5HkMI5+HHICzLrAPcJAviVbn22 pQxwjKj32p4784na/NWB4giOjdjdc3NzQp1hla87SOnPrVWll4Ygd3VT2R6vYFmItYkM 7x2XV30GQcMTL/1HYA5DAQ8UMkNqmTjsCn6ZOXFQapgfOEMsw/WogLS/EkCE/aXHXNK4 qX4Dvq9DQYN1LeUUYvrfcqAmAxfc/4CDrOMbW4b/sypd0dotXMn4cQQh4+nliFCbswwl u+q4ig+Q3doQ1mLYgyvovh5pe9P3NhchwDy4yGY0jkIGbyhT5pxp8TC3CYnFuV7xdsEg GlvA== X-Gm-Message-State: AOAM533ljFupYhgFHRVmHzl70Q0CTIJbjJHDN+nLEn5TAy1OHsaoQmGN E7ErjGj0Hi9brZKbwymq2nqimw== X-Google-Smtp-Source: ABdhPJwSotBfUlMwZQV3OnNmxAKyRRNxCKHByeBTjYDtUfE+GwiqHn2r4E/m/aWfQJpo+cKwRswf6w== X-Received: by 2002:a5d:6743:: with SMTP id l3mr4652330wrw.82.1606406100935; Thu, 26 Nov 2020 07:55:00 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:f008:704d:8d4b:9951]) by smtp.gmail.com with ESMTPSA id z11sm10645961wmc.39.2020.11.26.07.54.59 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 26 Nov 2020 07:54:59 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v3 16/23] kvm: arm64: Forward safe PSCI SMCs coming from host Date: Thu, 26 Nov 2020 15:54:14 +0000 Message-Id: <20201126155421.14901-17-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201126155421.14901-1-dbrazdil@google.com> References: <20201126155421.14901-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201126_105503_390759_028B7074 X-CRM114-Status: GOOD ( 19.45 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Forward the following PSCI SMCs issued by host to EL3 as they do not require the hypervisor's intervention. This assumes that EL3 correctly implements the PSCI specification. Only function IDs implemented in Linux are included. Where both 32-bit and 64-bit variants exist, it is assumed that the host will always use the 64-bit variant. * SMCs that only return information about the system * PSCI_VERSION - PSCI version implemented by EL3 * PSCI_FEATURES - optional features supported by EL3 * AFFINITY_INFO - power state of core/cluster * MIGRATE_INFO_TYPE - whether Trusted OS can be migrated * MIGRATE_INFO_UP_CPU - resident core of Trusted OS * operations which do not affect the hypervisor * MIGRATE - migrate Trusted OS to a different core * SET_SUSPEND_MODE - toggle OS-initiated mode * system shutdown/reset * SYSTEM_OFF * SYSTEM_RESET * SYSTEM_RESET2 Signed-off-by: David Brazdil --- arch/arm64/kvm/hyp/nvhe/psci-relay.c | 43 +++++++++++++++++++++++++++- 1 file changed, 42 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kvm/hyp/nvhe/psci-relay.c b/arch/arm64/kvm/hyp/nvhe/psci-relay.c index e7091d89f0fc..7aa87ab7f5ce 100644 --- a/arch/arm64/kvm/hyp/nvhe/psci-relay.c +++ b/arch/arm64/kvm/hyp/nvhe/psci-relay.c @@ -57,14 +57,51 @@ static bool is_psci_call(u64 func_id) } } +static unsigned long psci_call(unsigned long fn, unsigned long arg0, + unsigned long arg1, unsigned long arg2) +{ + struct arm_smccc_res res; + + arm_smccc_1_1_smc(fn, arg0, arg1, arg2, &res); + return res.a0; +} + +static unsigned long psci_forward(struct kvm_cpu_context *host_ctxt) +{ + return psci_call(cpu_reg(host_ctxt, 0), cpu_reg(host_ctxt, 1), + cpu_reg(host_ctxt, 2), cpu_reg(host_ctxt, 3)); +} + +static __noreturn unsigned long psci_forward_noreturn(struct kvm_cpu_context *host_ctxt) +{ + psci_forward(host_ctxt); + hyp_panic(); /* unreachable */ +} + static unsigned long psci_0_1_handler(u64 func_id, struct kvm_cpu_context *host_ctxt) { - return PSCI_RET_NOT_SUPPORTED; + if (func_id == kvm_host_psci_function_id[PSCI_FN_CPU_OFF]) + return psci_forward(host_ctxt); + else if (func_id == kvm_host_psci_function_id[PSCI_FN_MIGRATE]) + return psci_forward(host_ctxt); + else + return PSCI_RET_NOT_SUPPORTED; } static unsigned long psci_0_2_handler(u64 func_id, struct kvm_cpu_context *host_ctxt) { switch (func_id) { + case PSCI_0_2_FN_PSCI_VERSION: + case PSCI_0_2_FN_CPU_OFF: + case PSCI_0_2_FN64_AFFINITY_INFO: + case PSCI_0_2_FN64_MIGRATE: + case PSCI_0_2_FN_MIGRATE_INFO_TYPE: + case PSCI_0_2_FN64_MIGRATE_INFO_UP_CPU: + return psci_forward(host_ctxt); + case PSCI_0_2_FN_SYSTEM_OFF: + case PSCI_0_2_FN_SYSTEM_RESET: + psci_forward_noreturn(host_ctxt); + unreachable(); default: return PSCI_RET_NOT_SUPPORTED; } @@ -73,6 +110,10 @@ static unsigned long psci_0_2_handler(u64 func_id, struct kvm_cpu_context *host_ static unsigned long psci_1_0_handler(u64 func_id, struct kvm_cpu_context *host_ctxt) { switch (func_id) { + case PSCI_1_0_FN_PSCI_FEATURES: + case PSCI_1_0_FN_SET_SUSPEND_MODE: + case PSCI_1_1_FN64_SYSTEM_RESET2: + return psci_forward(host_ctxt); default: return psci_0_2_handler(func_id, host_ctxt); } From patchwork Thu Nov 26 15:54:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11934327 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1B60AC63697 for ; Thu, 26 Nov 2020 16:00:53 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5B87A221EB for ; Thu, 26 Nov 2020 16:00:52 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="tOgg12mq"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="pbb/yCmE" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5B87A221EB Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=8lv7DIWxbfszMh25fOPXGSwXWhmn+AO7YLqaLZdTE7Y=; b=tOgg12mqomuvL6ATSADzDSLIZ cUFwWvZgEw8JMQaLz5cJIYRO+rETLvJveG5EzTyyC8kA3SbrSIr/9j6NetX71EGhgv6pl6l+S8pBb RaOrcyjBLK1PQwpJqzQtT52Uji1nWWjXU4ZQ/yVcgYCMTOoXb7I2wvwEkJtfsVI0Fy82t0+tnHgVu oHyVf+h3odaBZQWw6kPt70d2dIOA55hXli/W3VAERS8fYgavy1rnHKU7LDbeu+bpbmOaBYAGYOHlj 45iMzKZl9DfpFUZ25qIeDOmDkVjSZ6P66acwjSJJZvYAXAiWHvTTlAfGUgPct/EqO1s33QsJnQ8h6 lhgjbapkg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kiJfW-00012Z-3A; Thu, 26 Nov 2020 15:58:50 +0000 Received: from mail-wr1-x442.google.com ([2a00:1450:4864:20::442]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kiJbt-0007p7-6t for linux-arm-kernel@lists.infradead.org; Thu, 26 Nov 2020 15:55:09 +0000 Received: by mail-wr1-x442.google.com with SMTP id s8so2623442wrw.10 for ; Thu, 26 Nov 2020 07:55:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Pb+ABw8dDs4PbD0OqVCMWkPKj8CiLKPB9XpZniqrrrk=; b=pbb/yCmE0LnjVE/ExQ/pJl7w83OT3LzqwQppQ3HqidXv2SfAJ9Nb1cPbq9a8ZZI4YA X5M/PUSmd0+3fpTDL/InmQ2mCDNpr9piPAuo7p6crV0IeKBHx/ZTIZOgH/esG4VTPy2J vlfVzxrBEn8n6D2S5rPjeXvv6YodXNpHneVC8Nwje61zvBtJxL9jKl+VgVh2Lahc+ksZ iHqtiBbtPHB1T5RFyjVg6EG7jdM6DUm23grUSANYdwjKev67RJFOYVUfOzyGvHvg8snO dasZpwdSKPiXhpsLurhfNgNIsnFGj3Bo6vrCCFJGRdAo6Tb+BYMNEOrvmCeSdXjXLlsU jk1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Pb+ABw8dDs4PbD0OqVCMWkPKj8CiLKPB9XpZniqrrrk=; b=a3F0og5IpfLzv8JGQnFqylVKd7/Sm8R+2M/YGpptcNsflZt3unCgq94V18KeCfNzgI MD/Shf3HG062D5aJUDu4VHmvGf5VBXhjGFfAonxL5k8rxjcUc/z7QM+ZNZHv/OPeUc4s +/QJrfmdcxiliEHg7TXhSWsMrSC3tWDQGfMIcBruqD552tquRf34tScdIYeXFw4HrRqq oeFcQdC9+cyXaRugLYOPRITdAWrH1O1tW4RWSoEsPGimxdxP0vNsIbK899DsZ5cJA4Dc BLcI2OgrjimPYgT38WGkKCzhF6jVfrAD3MBw265Zp+5ARrVQ/0b0xEbzN7TK79Cs/hMp 3qKw== X-Gm-Message-State: AOAM5327xlsa6OUM2w4VteDPH4zKOzhOwVkr4Q52VhPFNBcEVqUdPMyV JBN0/D5vRmct1cTuj3+rd6w5rA== X-Google-Smtp-Source: ABdhPJwxY6DWeyXDaTKzR9vlEHnJK1db+CbK/rNz4u0C73SsrWbM3UVGNV4lTxhZ1MSN5bqYaJbUug== X-Received: by 2002:adf:eb4d:: with SMTP id u13mr4757138wrn.146.1606406103382; Thu, 26 Nov 2020 07:55:03 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:f008:704d:8d4b:9951]) by smtp.gmail.com with ESMTPSA id g25sm8706624wmk.5.2020.11.26.07.55.01 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 26 Nov 2020 07:55:02 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v3 17/23] kvm: arm64: Extract __do_hyp_init into a helper function Date: Thu, 26 Nov 2020 15:54:15 +0000 Message-Id: <20201126155421.14901-18-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201126155421.14901-1-dbrazdil@google.com> References: <20201126155421.14901-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201126_105505_438610_5455879C X-CRM114-Status: GOOD ( 16.66 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In preparation for adding a CPU entry point in nVHE hyp code, extract most of __do_hyp_init hypervisor initialization code into a common helper function. This will be invoked by the entry point to install KVM on the newly booted CPU. Signed-off-by: David Brazdil --- arch/arm64/kvm/hyp/nvhe/hyp-init.S | 35 ++++++++++++++++++++++-------- 1 file changed, 26 insertions(+), 9 deletions(-) diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-init.S b/arch/arm64/kvm/hyp/nvhe/hyp-init.S index 77c983aa90fa..98ce40e17b42 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-init.S +++ b/arch/arm64/kvm/hyp/nvhe/hyp-init.S @@ -68,13 +68,32 @@ __do_hyp_init: mov x0, #SMCCC_RET_NOT_SUPPORTED eret -1: ldr x0, [x1, #NVHE_INIT_TPIDR_EL2] - msr tpidr_el2, x0 +1: mov x0, x1 + mov x4, lr + bl ___kvm_hyp_init + mov lr, x4 - ldr x0, [x1, #NVHE_INIT_STACK_HYP_VA] - mov sp, x0 + /* Hello, World! */ + mov x0, #SMCCC_RET_SUCCESS + eret +SYM_CODE_END(__kvm_hyp_init) + +/* + * Initialize the hypervisor in EL2. + * + * Only uses x0..x3 so as to not clobber callee-saved SMCCC registers + * and leave x4 for the caller. + * + * x0: struct kvm_nvhe_init_params PA + */ +SYM_CODE_START(___kvm_hyp_init) + ldr x1, [x0, #NVHE_INIT_TPIDR_EL2] + msr tpidr_el2, x1 + + ldr x1, [x0, #NVHE_INIT_STACK_HYP_VA] + mov sp, x1 - ldr x1, [x1, #NVHE_INIT_PGD_PA] + ldr x1, [x0, #NVHE_INIT_PGD_PA] phys_to_ttbr x0, x1 alternative_if ARM64_HAS_CNP orr x0, x0, #TTBR_CNP_BIT @@ -139,10 +158,8 @@ alternative_else_nop_endif kimg_hyp_va x0, x1 msr vbar_el2, x0 - /* Hello, World! */ - mov x0, #SMCCC_RET_SUCCESS - eret -SYM_CODE_END(__kvm_hyp_init) + ret +SYM_CODE_END(___kvm_hyp_init) SYM_CODE_START(__kvm_handle_stub_hvc) cmp x0, #HVC_SOFT_RESTART From patchwork Thu Nov 26 15:54:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11934331 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E270DC63798 for ; Thu, 26 Nov 2020 16:01:18 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 61C9921D40 for ; Thu, 26 Nov 2020 16:01:18 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="dpovO1uL"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="mX8bYR+x" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 61C9921D40 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=Zxj8I5wY/RHZ9H+PfDqMSs2OXtw1VOksdniY2dfs2qY=; b=dpovO1uLd77CDH8qEdO1HDA4p 8SXeQdH7wmotjclw2ihfE0QA2BbClDqY9XedjcIFfI1oJA20/tLitUWNrdF8j1e8o4cGrRMdCESVw 7VvCt/Ij97bFjNANgx7OQ/RXMHEnwiGtUCHmbPVq3Rn32Ry58pzeZGB6zVKhJIPvVRGTasHcAcw/G WYwBhY0qEouFHdW4j5PrBW2/LtzEVl3/1SZN9WJ5eo5dN50zJzQxSdO61cs3xRVDM+jhlc6Xea0QT HGpv3jK1warz8D+Ey83AGNsBGfRGNJ5ihsw3eBWqVvAyoltlipwzTd4ppwXDzbktNLHmpkwvjK3rC WgSo5jEvw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kiJfd-00015y-0O; Thu, 26 Nov 2020 15:58:57 +0000 Received: from mail-wr1-x443.google.com ([2a00:1450:4864:20::443]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kiJbu-0007q1-Ne for linux-arm-kernel@lists.infradead.org; Thu, 26 Nov 2020 15:55:09 +0000 Received: by mail-wr1-x443.google.com with SMTP id r3so2658327wrt.2 for ; Thu, 26 Nov 2020 07:55:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=yTPqxuNIHyHcxIGVDRFuzMb//a26J9RNoM2ThcyKbSM=; b=mX8bYR+xBsOyOIWEN8DX7gK/3reSGxDblgkVy4BFuCXdMl/4FvAYp/nmuGo1JoWP9P t6aYDDfb/OykyYNtU+YjrQpGHnafiTiCP67R+SMGTyaiF8vjgYt0Fvys9qkoErbLGFR9 we1WLjGlgUNDIDPEBeUD4Alzz5Om/fnxds/Oxp5CzjqLXbiXrL9tA5FQ1fQEZMm617TK IdSpNFJ2H05e+uXFDCHwheCyX3oSyvQ7+b/+18Q/kDteenUlTRRV+WrhBV712PuxD+Kx wmzbfiP7gvyDFfsWCOAGPdMnOJDwhL/l00+ToHh9zpX87qwlANYNEterelAS3s8rUwBU 5CJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=yTPqxuNIHyHcxIGVDRFuzMb//a26J9RNoM2ThcyKbSM=; b=Ym4yrjEco2Mbgb8huyvKU7JhNTF7nnHFhqV4neQm+82MPuW9sDCzLJb0I9wmD0Io9c yCicln+DHRRuCunvm5yIMrFFp2MxPkFjdROVpvTNFLD9Bo8aDApgp4r3K1OR+W9wjbBa IzFZrqJ6KIXbK5TJ2A5ty4q5Da6/JqzwP3OOWgSnZhDTYs/majP/dRLbmBV/ixJqf4/B nK8vLGbU8hxJhUNSxNFe5ZGXkDpGm1Ly+Rsu7Kzz3nNCdh8UD4P02CZLOj1JbL7A0OTO tNYchm6hhB/IMor8zprLh5BoPifK6ZRgDTDJ/vXOQvR88NPHQZexOZXjVzb3iwmOoRGH z6TQ== X-Gm-Message-State: AOAM53069h26ZVZZq3SB9FBbk2jxLaQgTUggdr4tdysUt0U2HG2ZgKRW q1+mbnTXdIr0Va4WY36zurVpEA== X-Google-Smtp-Source: ABdhPJwX4rEG3aPSmYGdOs6RN45Cetge/Lz5Y7cdyVn5fxjNFMXKsImO/VR5cgpiiWFbabfK0CSr1g== X-Received: by 2002:adf:f888:: with SMTP id u8mr4544222wrp.381.1606406105448; Thu, 26 Nov 2020 07:55:05 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:f008:704d:8d4b:9951]) by smtp.gmail.com with ESMTPSA id s25sm8628255wmh.16.2020.11.26.07.55.04 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 26 Nov 2020 07:55:04 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v3 18/23] kvm: arm64: Add function to enter host from KVM nVHE hyp code Date: Thu, 26 Nov 2020 15:54:16 +0000 Message-Id: <20201126155421.14901-19-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201126155421.14901-1-dbrazdil@google.com> References: <20201126155421.14901-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201126_105506_909689_AE88281C X-CRM114-Status: GOOD ( 15.37 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org All nVHE hyp code is currently executed as handlers of host's HVCs. This will change as nVHE starts intercepting host's PSCI CPU_ON SMCs. The newly booted CPU will need to initialize EL2 state and then enter the host. Add __host_enter function that branches into the existing host state-restoring code after the trap handler would have returned. Signed-off-by: David Brazdil --- arch/arm64/kvm/hyp/nvhe/host.S | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/arch/arm64/kvm/hyp/nvhe/host.S b/arch/arm64/kvm/hyp/nvhe/host.S index 2b56f0bdf874..a820dfdc9c25 100644 --- a/arch/arm64/kvm/hyp/nvhe/host.S +++ b/arch/arm64/kvm/hyp/nvhe/host.S @@ -39,6 +39,7 @@ SYM_FUNC_START(__host_exit) bl handle_trap /* Restore host regs x0-x17 */ +__host_enter_restore_full: ldp x0, x1, [x29, #CPU_XREG_OFFSET(0)] ldp x2, x3, [x29, #CPU_XREG_OFFSET(2)] ldp x4, x5, [x29, #CPU_XREG_OFFSET(4)] @@ -61,6 +62,14 @@ __host_enter_without_restoring: sb SYM_FUNC_END(__host_exit) +/* + * void __noreturn __host_enter(struct kvm_cpu_context *host_ctxt); + */ +SYM_FUNC_START(__host_enter) + mov x29, x0 + b __host_enter_restore_full +SYM_FUNC_END(__host_enter) + /* * void __noreturn __hyp_do_panic(bool restore_host, u64 spsr, u64 elr, u64 par); */ From patchwork Thu Nov 26 15:54:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11934333 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E12EBC56202 for ; Thu, 26 Nov 2020 16:01:40 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 622CD206B5 for ; Thu, 26 Nov 2020 16:01:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="rUf/XXe4"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="nQ8SSmjL" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 622CD206B5 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=KL81gBFq2C9qqHCO75pGNQmczqsskCmce4As4iecbSw=; b=rUf/XXe4tYTcHW9xcidJIYPIJ 4Hege3N8VXpUdRyKfTzDZTmT5/ysoxKa/OPGZK18xfdrCn8Rq6SeRTCzew/nzg37jXhv37owiNj43 N6eJL8tm/TWo7gXYA72hLI3c6C7+nXvVpMYDM6ol0v8GYkiwDS5/2m7pUyiLqzGxfX9aJJNaBD+am s1pyA6ymqidx0A094XLQ1SzkOUceU/uUJJSISJiO1m0u48nFU8N5BmfYYRbTl2Q1yduY0A0PYS8xw jIfa2orqxR1LobyVm/HylBuebK/GmGJqV1/Z4t0PFHEBzca03QjGcIgk1lnTdjsEsLViR5Gb8LQwk k7jc/dKgw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kiJfk-0001AR-Lq; Thu, 26 Nov 2020 15:59:05 +0000 Received: from mail-wr1-x429.google.com ([2a00:1450:4864:20::429]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kiJby-0007qn-Ng for linux-arm-kernel@lists.infradead.org; Thu, 26 Nov 2020 15:55:16 +0000 Received: by mail-wr1-x429.google.com with SMTP id u12so2665024wrt.0 for ; Thu, 26 Nov 2020 07:55:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=OmiGNR2uuCJEGcHKM0d3V1Ye4OF2/smYKtB2F4FGZ9U=; b=nQ8SSmjL1BNQcjaNOG2spLM3KGhqA7Z4p05orE+wNTrlj9pvs+bt8709WH6hGD97Fp MO398OfUu8FCD2Pfu7pCMrAI6fqp3WUQ6fDuJkxFZhy6UOyAMVjrc+DVn+EQsRIpUEMy D5fuMeWEuZWKiDDiS5k/zCbk8oe4n/e71bAlcFH6eE4uhQQAdwjZbtwcU/uFNLVbI5WA /eVYCwlziCS/+HTHGTOp4HlCuR6QA/asEzEo4gtcYuDZtxxQh2w65RbFzgUXj6DZ/5Tt Z+FTUUKQX8v0FVdsfIieakkGjy4cD+VvxnmrrkL3fbZZSBOIDapQ6G+w1GANSluW+Kyt SYUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OmiGNR2uuCJEGcHKM0d3V1Ye4OF2/smYKtB2F4FGZ9U=; b=gxcHP7R5xLVQVTv+1pfsq6l9SzQrSrLWUK49GS170XNobchH6h+MbGVu5G1bQC6Pj4 SxZL3GoufL3pXHqgINyq/eWSGc9534LC2s+4lU88M47Pc2M6aZkgShLXS77RKxbG+Ord ja9K3fYOgD5F5ehAUEhb8mwVnzycV8xb1YCNaGjoIJsnEHrmL0CfHrevA1zT+1JW3EB/ z5G4V3aLLy4aPrkgCW21IHrNFwPzwURbfYKbqsOB5Xv4CE9gYQtQmq7hPArhIWtu8m5B SmUiiY+49tvPy6PdHZJdz4kdxtZI29qVnEbbGhiZVtdExVWF6I53FqYcSUyuyAmos1Qn qW7A== X-Gm-Message-State: AOAM532kH+qFtTz04tja+1pmkjLUUa1iucrO6KCCAVoSndNaNrdMLvJ0 xOaIu0Spt9XkQtBTsGEvYpccZA== X-Google-Smtp-Source: ABdhPJypyppTeU5V6/k3iqU7vPbd+TN9q7mzRr6Yei6CDrxVWBkVZkf5J9/tWnOZ6PCDXQkYs77Tpg== X-Received: by 2002:a5d:6050:: with SMTP id j16mr4697763wrt.158.1606406107573; Thu, 26 Nov 2020 07:55:07 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:f008:704d:8d4b:9951]) by smtp.gmail.com with ESMTPSA id s6sm1779640wme.48.2020.11.26.07.55.06 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 26 Nov 2020 07:55:06 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v3 19/23] kvm: arm64: Intercept host's CPU_ON SMCs Date: Thu, 26 Nov 2020 15:54:17 +0000 Message-Id: <20201126155421.14901-20-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201126155421.14901-1-dbrazdil@google.com> References: <20201126155421.14901-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201126_105510_909696_AD1AC018 X-CRM114-Status: GOOD ( 25.44 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add a handler of the CPU_ON PSCI call from host. When invoked, it looks up the logical CPU ID corresponding to the provided MPIDR and populates the state struct of the target CPU with the provided x0, pc. It then calls CPU_ON itself, with an entry point in hyp that initializes EL2 state before returning ERET to the provided PC in EL1. There is a simple atomic lock around the boot args struct. If it is already locked, CPU_ON will return PENDING_ON error code. Signed-off-by: David Brazdil --- arch/arm64/kvm/hyp/nvhe/hyp-init.S | 30 ++++++++ arch/arm64/kvm/hyp/nvhe/psci-relay.c | 109 +++++++++++++++++++++++++++ 2 files changed, 139 insertions(+) diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-init.S b/arch/arm64/kvm/hyp/nvhe/hyp-init.S index 98ce40e17b42..ea71f653af55 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-init.S +++ b/arch/arm64/kvm/hyp/nvhe/hyp-init.S @@ -9,6 +9,7 @@ #include #include +#include #include #include #include @@ -161,6 +162,35 @@ alternative_else_nop_endif ret SYM_CODE_END(___kvm_hyp_init) +SYM_CODE_START(__kvm_hyp_cpu_on_entry) + msr SPsel, #1 // We want to use SP_EL{1,2} + + /* Check that the core was booted in EL2. */ + mrs x1, CurrentEL + cmp x1, #CurrentEL_EL2 + b.eq 2f + + /* The core booted in EL1. KVM cannot be initialized on it. */ +1: wfe + wfi + b 1b + + /* Initialize EL2 CPU state to sane values. */ +2: mov x29, x0 + init_el2_state nvhe + mov x0, x29 + + /* Enable MMU, set vectors and stack. */ + bl ___kvm_hyp_init + + /* Load address of the C handler. */ + ldr x1, =__kvm_hyp_psci_cpu_entry + kimg_hyp_va x1, x2 + + /* Leave idmap. */ + br x1 +SYM_CODE_END(__kvm_hyp_cpu_on_entry) + SYM_CODE_START(__kvm_handle_stub_hvc) cmp x0, #HVC_SOFT_RESTART b.ne 1f diff --git a/arch/arm64/kvm/hyp/nvhe/psci-relay.c b/arch/arm64/kvm/hyp/nvhe/psci-relay.c index 7aa87ab7f5ce..39e507672e6e 100644 --- a/arch/arm64/kvm/hyp/nvhe/psci-relay.c +++ b/arch/arm64/kvm/hyp/nvhe/psci-relay.c @@ -9,12 +9,17 @@ #include #include #include +#include #include #include #include #include +extern char __kvm_hyp_cpu_on_entry[]; + +void __noreturn __host_enter(struct kvm_cpu_context *host_ctxt); + /* Config options set by the host. */ u32 __ro_after_init kvm_host_psci_version; u32 __ro_after_init kvm_host_psci_function_id[PSCI_FN_MAX]; @@ -22,6 +27,19 @@ s64 __ro_after_init hyp_physvirt_offset; #define __hyp_pa(x) ((phys_addr_t)((x)) + hyp_physvirt_offset) +#define INVALID_CPU_ID UINT_MAX + +#define CPU_UNLOCKED 0 +#define CPU_LOCKED 1 + +struct cpu_boot_args { + unsigned long pc; + unsigned long r0; +}; + +static DEFINE_PER_CPU(atomic_t, cpu_on_lock) = ATOMIC_INIT(0); +static DEFINE_PER_CPU(struct cpu_boot_args, cpu_on_args); + static u64 get_psci_func_id(struct kvm_cpu_context *host_ctxt) { DECLARE_REG(u64, func_id, host_ctxt, 0); @@ -78,10 +96,99 @@ static __noreturn unsigned long psci_forward_noreturn(struct kvm_cpu_context *ho hyp_panic(); /* unreachable */ } +static unsigned int find_cpu_id(u64 mpidr) +{ + unsigned int i; + + /* Reject invalid MPIDRs */ + if (mpidr & ~MPIDR_HWID_BITMASK) + return INVALID_CPU_ID; + + for (i = 0; i < NR_CPUS; i++) { + if (cpu_logical_map(i) == mpidr) + return i; + } + + return INVALID_CPU_ID; +} + +static __always_inline bool try_acquire_cpu_on_lock(atomic_t *l) { + return atomic_cmpxchg_acquire(l, CPU_UNLOCKED, CPU_LOCKED) == CPU_UNLOCKED; +} + +static __always_inline void release_cpu_on_lock(atomic_t *l) { + atomic_set_release(l, CPU_UNLOCKED); +} + +static int psci_cpu_on(u64 func_id, struct kvm_cpu_context *host_ctxt) +{ + DECLARE_REG(u64, mpidr, host_ctxt, 1); + DECLARE_REG(unsigned long, pc, host_ctxt, 2); + DECLARE_REG(unsigned long, r0, host_ctxt, 3); + + unsigned int cpu_id; + atomic_t *lock; + struct cpu_boot_args *boot_args; + struct kvm_nvhe_init_params *init_params; + int ret; + + /* + * Find the logical CPU ID for the given MPIDR. The search set is + * the set of CPUs that were online at the point of KVM initialization. + * Booting other CPUs is rejected because their cpufeatures were not + * checked against the finalized capabilities. This could be relaxed + * by doing the feature checks in hyp. + */ + cpu_id = find_cpu_id(mpidr); + if (cpu_id == INVALID_CPU_ID) + return PSCI_RET_INVALID_PARAMS; + + lock = per_cpu_ptr(hyp_symbol_addr(cpu_on_lock), cpu_id); + boot_args = per_cpu_ptr(hyp_symbol_addr(cpu_on_args), cpu_id); + init_params = per_cpu_ptr(hyp_symbol_addr(kvm_init_params), cpu_id); + + /* Check if the target CPU is already being booted. */ + if (!try_acquire_cpu_on_lock(lock)) + return PSCI_RET_ALREADY_ON; + + *boot_args = (struct cpu_boot_args){ .pc = pc, .r0 = r0 }; + wmb(); + + ret = psci_call(func_id, mpidr, + __hyp_pa(hyp_symbol_addr(__kvm_hyp_cpu_on_entry)), + __hyp_pa(init_params)); + + /* If successful, the lock will be released by the target CPU. */ + if (ret != PSCI_RET_SUCCESS) + release_cpu_on_lock(lock); + + return ret; +} + +asmlinkage void __noreturn __kvm_hyp_psci_cpu_entry(void) +{ + atomic_t *lock; + struct cpu_boot_args *boot_args; + struct kvm_cpu_context *host_ctxt; + + lock = this_cpu_ptr(hyp_symbol_addr(cpu_on_lock)); + boot_args = this_cpu_ptr(hyp_symbol_addr(cpu_on_args)); + host_ctxt = &this_cpu_ptr(hyp_symbol_addr(kvm_host_data))->host_ctxt; + + cpu_reg(host_ctxt, 0) = boot_args->r0; + write_sysreg_el2(boot_args->pc, SYS_ELR); + + release_cpu_on_lock(lock); + + __host_enter(host_ctxt); +} + static unsigned long psci_0_1_handler(u64 func_id, struct kvm_cpu_context *host_ctxt) { if (func_id == kvm_host_psci_function_id[PSCI_FN_CPU_OFF]) return psci_forward(host_ctxt); + else if (func_id == kvm_host_psci_function_id[PSCI_FN_CPU_ON]) + return psci_cpu_on(func_id, host_ctxt); else if (func_id == kvm_host_psci_function_id[PSCI_FN_MIGRATE]) return psci_forward(host_ctxt); else @@ -102,6 +209,8 @@ static unsigned long psci_0_2_handler(u64 func_id, struct kvm_cpu_context *host_ case PSCI_0_2_FN_SYSTEM_RESET: psci_forward_noreturn(host_ctxt); unreachable(); + case PSCI_0_2_FN64_CPU_ON: + return psci_cpu_on(func_id, host_ctxt); default: return PSCI_RET_NOT_SUPPORTED; } From patchwork Thu Nov 26 15:54:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11934337 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A5984C56202 for ; Thu, 26 Nov 2020 16:02:31 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C4FD721D40 for ; Thu, 26 Nov 2020 16:02:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="IM63zcSu"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="ZVUoTU+g" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C4FD721D40 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=RsfFCmOhKsvKi4cO+MHTNEG7YhuHkaXnkxh829lTwaA=; b=IM63zcSu6G8M7tyem/qAt3lsy 5cYhiH7XT+xQzmpf2Ln6bll+ek4oRc1a7gTtrYkjXJOKcGDftL8tQFf7UkMW0XF7nFKqU+QLH+LJ+ QOkJylCnFHnRldmCV/AMTCl4C/2lg5dALYg2yuOCqPF0dq4HN+pCnbkCeEAQD2xH6lo0U+QKM/Iu7 FdhSzgfKJd+PJlXucRixDGNxsEYU48ltezj14gDiDqz7p3BH7uLXm7BbdMRI1RBr73CvvnWRBbads N5chJV/TstboG57iU8Rd/C4lydoVohBnvTbevPqpcpDFxXHNQX9xmzk35qkYmiatf6jkUPFnbtVYm SkHJDJjxQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kiJgG-0001PY-4C; Thu, 26 Nov 2020 15:59:36 +0000 Received: from mail-wr1-x444.google.com ([2a00:1450:4864:20::444]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kiJbz-0007rW-Cl for linux-arm-kernel@lists.infradead.org; Thu, 26 Nov 2020 15:55:16 +0000 Received: by mail-wr1-x444.google.com with SMTP id e7so2642569wrv.6 for ; Thu, 26 Nov 2020 07:55:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=zOoo3Sgmv8F42l97WnsUvAdYKac9P9JfqD+cuM/Sn9g=; b=ZVUoTU+gAuCOCKAA0JLDhF23GWfoPMRCkpRKeYuO2EHe2Onkr6VOoTZUTexi44Rir1 M7Vec/VJjU910zroh1GatIekcjEn1KcoZUYX6zTIR6xbl5/W+BI4ijNuriBY4wzxq6o0 th2sQXhqNToAPL2A4a3jTYDvdhRPGxKoBR1YLOiGZat0oLUmpTjtTL5xf9QYbPo6cRaz LeC0zcowK1CpA5rM9SPW/xuLsByt07lyInueXKnZbfVrraPEbzK2Hp2YKnke9CJtx74L 7eywhbuxnO+at1gvgIEjiycO99JM0jHyGOtXQzmevEWogclk6qZxfgi8ain9mePuu9R6 wdOA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=zOoo3Sgmv8F42l97WnsUvAdYKac9P9JfqD+cuM/Sn9g=; b=RFnluo/rXJlr/kty/ln2HZkh5rjdIzV6DOZaewWSnGKPh8uEOac/vem8P5pg41q2yZ GANFM4lB+j8bex2RbOvRtXa6oIoUg+uhJucRhVDTRbbtPmmqWJfMuuqKJ9WGah4mToHa AUdIrkTbQKiWuzpGHX6r4E/f8O9ip98nl4Ah44j7RNGJVPoXu1SlVC80+g9VDVB+nXN9 qvfUOozcGWVbn7tPSjeuRXc5/hyaPhpA53maMKHbBAusNJflglmDWPCxxt0kA6ELzuvb rkSyav75sauMB2lGuM4JEBO8Dnj7jyyk6oXYabfAUywEswR7YPiW6IT0XwAZuEnlI/Qy DdQw== X-Gm-Message-State: AOAM532VmL4hg87JwesPpxDjU+adJGNrfNIP/rxEu0Zzu5rmsR1QKfTN N4P154EPx8XU5hDpk7GNp+VeuA== X-Google-Smtp-Source: ABdhPJx7GWZaDHcnXn1i3ov34ALXy1EHh5jMhQYPSdtzUtu5jpP/ecukZb3UeFP0dVO5FD7/MCcz8A== X-Received: by 2002:adf:82f5:: with SMTP id 108mr4670137wrc.269.1606406109770; Thu, 26 Nov 2020 07:55:09 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:f008:704d:8d4b:9951]) by smtp.gmail.com with ESMTPSA id d13sm10931063wrb.39.2020.11.26.07.55.08 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 26 Nov 2020 07:55:08 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v3 20/23] kvm: arm64: Intercept host's CPU_SUSPEND PSCI SMCs Date: Thu, 26 Nov 2020 15:54:18 +0000 Message-Id: <20201126155421.14901-21-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201126155421.14901-1-dbrazdil@google.com> References: <20201126155421.14901-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201126_105511_647843_2F38F055 X-CRM114-Status: GOOD ( 22.69 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add a handler of CPU_SUSPEND host PSCI SMCs. The SMC can either enter a sleep state indistinguishable from a WFI or a deeper sleep state that behaves like a CPU_OFF+CPU_ON except that the core is still considered online when asleep. The handler saves r0,pc of the host and makes the same call to EL3 with the hyp CPU entry point. It either returns back to the handler and then back to the host, or wakes up into the entry point and initializes EL2 state before dropping back to EL1. A core can only suspend itself but other cores can concurrently invoke CPU_ON with this core as target. To avoid racing them for the same boot args struct, CPU_SUSPEND uses a different struct instance and entry point. Each entry point selects the corresponding struct to restore host boot args from. This avoids the need for locking in CPU_SUSPEND. Signed-off-by: David Brazdil --- arch/arm64/kvm/hyp/nvhe/hyp-init.S | 9 +++++ arch/arm64/kvm/hyp/nvhe/psci-relay.c | 50 +++++++++++++++++++++++++--- 2 files changed, 54 insertions(+), 5 deletions(-) diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-init.S b/arch/arm64/kvm/hyp/nvhe/hyp-init.S index ea71f653af55..fbb195851fb9 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-init.S +++ b/arch/arm64/kvm/hyp/nvhe/hyp-init.S @@ -162,7 +162,15 @@ alternative_else_nop_endif ret SYM_CODE_END(___kvm_hyp_init) +SYM_CODE_START(__kvm_hyp_cpu_suspend_entry) + mov x28, #0 // is_cpu_on = false + b __kvm_hyp_cpu_common_entry +SYM_CODE_END(__kvm_hyp_cpu_suspend_entry) + SYM_CODE_START(__kvm_hyp_cpu_on_entry) + mov x28, #1 // is_cpu_on = true + +SYM_INNER_LABEL(__kvm_hyp_cpu_common_entry, SYM_L_LOCAL) msr SPsel, #1 // We want to use SP_EL{1,2} /* Check that the core was booted in EL2. */ @@ -188,6 +196,7 @@ SYM_CODE_START(__kvm_hyp_cpu_on_entry) kimg_hyp_va x1, x2 /* Leave idmap. */ + mov x0, x28 br x1 SYM_CODE_END(__kvm_hyp_cpu_on_entry) diff --git a/arch/arm64/kvm/hyp/nvhe/psci-relay.c b/arch/arm64/kvm/hyp/nvhe/psci-relay.c index 39e507672e6e..592c11e9851c 100644 --- a/arch/arm64/kvm/hyp/nvhe/psci-relay.c +++ b/arch/arm64/kvm/hyp/nvhe/psci-relay.c @@ -17,6 +17,7 @@ #include extern char __kvm_hyp_cpu_on_entry[]; +extern char __kvm_hyp_cpu_suspend_entry[]; void __noreturn __host_enter(struct kvm_cpu_context *host_ctxt); @@ -39,6 +40,7 @@ struct cpu_boot_args { static DEFINE_PER_CPU(atomic_t, cpu_on_lock) = ATOMIC_INIT(0); static DEFINE_PER_CPU(struct cpu_boot_args, cpu_on_args); +static DEFINE_PER_CPU(struct cpu_boot_args, cpu_suspend_args); static u64 get_psci_func_id(struct kvm_cpu_context *host_ctxt) { @@ -112,6 +114,34 @@ static unsigned int find_cpu_id(u64 mpidr) return INVALID_CPU_ID; } +static int psci_cpu_suspend(u64 func_id, struct kvm_cpu_context *host_ctxt) +{ + DECLARE_REG(u64, power_state, host_ctxt, 1); + DECLARE_REG(unsigned long, pc, host_ctxt, 2); + DECLARE_REG(unsigned long, r0, host_ctxt, 3); + + struct cpu_boot_args *boot_args; + struct kvm_nvhe_init_params *init_params; + + boot_args = this_cpu_ptr(hyp_symbol_addr(cpu_suspend_args)); + init_params = this_cpu_ptr(hyp_symbol_addr(kvm_init_params)); + + /* + * No need to acquire a lock before writing to boot_args because a core + * can only suspend itself and the racy CPU_ON uses a separate struct. + */ + + *boot_args = (struct cpu_boot_args){ .pc = pc, .r0 = r0 }; + + /* + * Will either return if shallow sleep state, or wake up into the entry + * point if it is a deep sleep state. + */ + return psci_call(func_id, power_state, + __hyp_pa(hyp_symbol_addr(__kvm_hyp_cpu_suspend_entry)), + __hyp_pa(init_params)); +} + static __always_inline bool try_acquire_cpu_on_lock(atomic_t *l) { return atomic_cmpxchg_acquire(l, CPU_UNLOCKED, CPU_LOCKED) == CPU_UNLOCKED; } @@ -165,27 +195,35 @@ static int psci_cpu_on(u64 func_id, struct kvm_cpu_context *host_ctxt) return ret; } -asmlinkage void __noreturn __kvm_hyp_psci_cpu_entry(void) +asmlinkage void __noreturn __kvm_hyp_psci_cpu_entry(bool is_cpu_on) { atomic_t *lock; struct cpu_boot_args *boot_args; struct kvm_cpu_context *host_ctxt; - lock = this_cpu_ptr(hyp_symbol_addr(cpu_on_lock)); - boot_args = this_cpu_ptr(hyp_symbol_addr(cpu_on_args)); + if (is_cpu_on) + boot_args = this_cpu_ptr(hyp_symbol_addr(cpu_on_args)); + else + boot_args = this_cpu_ptr(hyp_symbol_addr(cpu_suspend_args)); + host_ctxt = &this_cpu_ptr(hyp_symbol_addr(kvm_host_data))->host_ctxt; cpu_reg(host_ctxt, 0) = boot_args->r0; write_sysreg_el2(boot_args->pc, SYS_ELR); - release_cpu_on_lock(lock); + if (is_cpu_on) { + lock = this_cpu_ptr(hyp_symbol_addr(cpu_on_lock)); + release_cpu_on_lock(lock); + } __host_enter(host_ctxt); } static unsigned long psci_0_1_handler(u64 func_id, struct kvm_cpu_context *host_ctxt) { - if (func_id == kvm_host_psci_function_id[PSCI_FN_CPU_OFF]) + if (func_id == kvm_host_psci_function_id[PSCI_FN_CPU_SUSPEND]) + return psci_cpu_suspend(func_id, host_ctxt); + else if (func_id == kvm_host_psci_function_id[PSCI_FN_CPU_OFF]) return psci_forward(host_ctxt); else if (func_id == kvm_host_psci_function_id[PSCI_FN_CPU_ON]) return psci_cpu_on(func_id, host_ctxt); @@ -209,6 +247,8 @@ static unsigned long psci_0_2_handler(u64 func_id, struct kvm_cpu_context *host_ case PSCI_0_2_FN_SYSTEM_RESET: psci_forward_noreturn(host_ctxt); unreachable(); + case PSCI_0_2_FN64_CPU_SUSPEND: + return psci_cpu_suspend(func_id, host_ctxt); case PSCI_0_2_FN64_CPU_ON: return psci_cpu_on(func_id, host_ctxt); default: From patchwork Thu Nov 26 15:54:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11934335 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 21ED4C63697 for ; Thu, 26 Nov 2020 16:02:03 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5D950206B5 for ; Thu, 26 Nov 2020 16:02:02 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="E75MtlD4"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="DW6HRBvH" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5D950206B5 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=Mv6eBZ35D/OftctTYZO9NS5XMRjmodC7z6IAWpJLVAI=; b=E75MtlD4wckNms3kusNOcz7er DryJFFkoZvA2y94PnShF3m1qh5fUtXU2fxEBtjf3agpQ4ErdmMhvcE4aoVP/al2OS+qUAkb1u0tI/ ZWyOg+SYCSCLCCJzsoCErO4mcpAm97R0cEcsIGYzdL+8fp2h5CarOOcpWOm+fKYfnZAlWlUtUAB9X UDC6KTlj5Db4+J/HAIcYo4bwwoG0PVuvcR7JmdI8Ww7bpeFp5XmgiMnNtaWi10kPgFVVc1ihKSYT7 ofIw+wW0gGBgmIFVlM6kRmoTFKl3wHUWevsIoc6RzZ/VgPwji3AO9DGHB1mGeKZ6s4l6N6utPOXc0 umqx8VDOg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kiJfx-0001Gn-H2; Thu, 26 Nov 2020 15:59:20 +0000 Received: from mail-wm1-x341.google.com ([2a00:1450:4864:20::341]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kiJc1-0007se-Ah for linux-arm-kernel@lists.infradead.org; Thu, 26 Nov 2020 15:55:16 +0000 Received: by mail-wm1-x341.google.com with SMTP id 3so161008wmj.4 for ; Thu, 26 Nov 2020 07:55:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=yy/LdNdSlEKXNA1O0CJPP3BR+TLcin0LNTWW2mqA/Ws=; b=DW6HRBvH1dMRy9410u25SEcLsIp2cGZuNo6NNar9dsMBlj7Q7JHURHQ2ZWwJ4jXPRq FACFV5c1R7NwwKB0Oib/MpTay9RuvI6yCYOXmhbV+Pr3HgDlXdyvlg4UItg4lhdUNGHa Ld69QoFi9l+cU2d7Jj3v+urROaKpzMnCR+uXap1CF7z5vcz37pqfkvZgvs19n+L89vHB EEJiRfonQQHkdGI2wagoH7mhFUxDpAHkdjQtb7l1WLX+s81JCnUu/JCu2sK0TldwqLJv vWKQT0qJDuODF6PdMxt4duYZuW8RI90XbH/vsvYAybJHI2J7TmqDikGk5/EKWk9vYwAn 9ECg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=yy/LdNdSlEKXNA1O0CJPP3BR+TLcin0LNTWW2mqA/Ws=; b=OsK1HlPj9kM2GwcI43qDksdRmDlFI3kjElqj2QAYcBfRNFDcxrvL+KdjANjp1AsGBX TfUlaOYILPAIQJVaiTcus6kiy8QvhSwMM4nFRLv/CcajoBfBZXSj0fEKP/81JDmR5qd1 0UpDfNIfd7rI1auezBIF82HLBbN/FBv+Zc7N7hZXtu8+PszTeW94wlc9OkVjW7FFakM+ sH7n+yApwbPXe2jotsqb2tWTgP+JpQjsvmIkiprirgJgkjorUU/0fhqt2q8Jo79mSM8O KeLTjrGA8lII2Vh0yfGsM/mI80f4E16m46ZndzQKFrKGbMUoJEUoSzrSLsoSz0i+kP1f XL2g== X-Gm-Message-State: AOAM530KVIp3ZQhe2zMexS8HzLCoHT9vMVau+wNXRRyAhFZMXlFyhxqR nOJNxG+cFPS/7g+an+7es+3rKQ== X-Google-Smtp-Source: ABdhPJzVolDDwaIJ6HnyKtT7uOspSO1I6DuI7m5CXOsy4xcwwUFP5ghKEPUc3bw4ZpFlZEJaa1lvVQ== X-Received: by 2002:a1c:61c2:: with SMTP id v185mr4165642wmb.152.1606406111903; Thu, 26 Nov 2020 07:55:11 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:f008:704d:8d4b:9951]) by smtp.gmail.com with ESMTPSA id q25sm10978629wmq.37.2020.11.26.07.55.10 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 26 Nov 2020 07:55:10 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v3 21/23] kvm: arm64: Keep nVHE EL2 vector installed Date: Thu, 26 Nov 2020 15:54:19 +0000 Message-Id: <20201126155421.14901-22-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201126155421.14901-1-dbrazdil@google.com> References: <20201126155421.14901-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201126_105513_515628_3908F70F X-CRM114-Status: GOOD ( 17.29 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org KVM by default keeps the stub vector installed and installs the nVHE vector only briefly for init and later on demand. Change this policy to install the vector at init and then never uninstall it if the kernel was given the protected KVM command line parameter. Signed-off-by: David Brazdil --- arch/arm64/kvm/arm.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 7a17b5048454..6ec8ddf74643 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1478,7 +1478,8 @@ static void _kvm_arch_hardware_disable(void *discard) void kvm_arch_hardware_disable(void) { - _kvm_arch_hardware_disable(NULL); + if (!is_protected_kvm_enabled()) + _kvm_arch_hardware_disable(NULL); } #ifdef CONFIG_CPU_PM @@ -1521,11 +1522,13 @@ static struct notifier_block hyp_init_cpu_pm_nb = { static void __init hyp_cpu_pm_init(void) { - cpu_pm_register_notifier(&hyp_init_cpu_pm_nb); + if (!is_protected_kvm_enabled()) + cpu_pm_register_notifier(&hyp_init_cpu_pm_nb); } static void __init hyp_cpu_pm_exit(void) { - cpu_pm_unregister_notifier(&hyp_init_cpu_pm_nb); + if (!is_protected_kvm_enabled()) + cpu_pm_unregister_notifier(&hyp_init_cpu_pm_nb); } #else static inline void hyp_cpu_pm_init(void) @@ -1617,7 +1620,8 @@ static int init_subsystems(void) kvm_sys_reg_table_init(); out: - on_each_cpu(_kvm_arch_hardware_disable, NULL, 1); + if (err || !is_protected_kvm_enabled()) + on_each_cpu(_kvm_arch_hardware_disable, NULL, 1); return err; } From patchwork Thu Nov 26 15:54:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11934339 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4E7DAC56202 for ; Thu, 26 Nov 2020 16:03:01 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CC6BC21D40 for ; Thu, 26 Nov 2020 16:03:00 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="wfnymZTN"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="srDcQuvU" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CC6BC21D40 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=ncEiHSSclGm/MD+QvunQ9+AiOw3D83bUVYAhGChDqhQ=; b=wfnymZTN93TBMMKkb2TQsjFYc fPFHI9XIBnmS5T8Gj35sES7Q7r8qCN8FQORH60wowFK9JrCcP5WojEDMDK31AHfSxCymtl4EiSZG4 Ll6LnuHWy/uW36qzCQ4W9gF89SFIhRpk/2uScUBniYiHIM3Fokd7COlQ7atJnngWRhB70sr/46GFK 2PE/xGxzWfW2QYwmthlcIpFaO7WYfI5UKs4JIu7Co4kqMyhL2ZVwKxmyvubxwAXH9b8iXPLdcM+QB V6vZyBjMumOPuXFzDrViPeueb9tJffVia3u5Oetu4kBmI/A/2EzO3+FCJe6l97BrBgLGoZK+Uwobx r61BesZSQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kiJgj-0001dP-Ck; Thu, 26 Nov 2020 16:00:05 +0000 Received: from mail-wr1-x443.google.com ([2a00:1450:4864:20::443]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kiJc3-0007tO-EU for linux-arm-kernel@lists.infradead.org; Thu, 26 Nov 2020 15:55:18 +0000 Received: by mail-wr1-x443.google.com with SMTP id z7so2647018wrn.3 for ; Thu, 26 Nov 2020 07:55:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=oyYLFWQZMneEnEj2VSn1LS09AGJVymYOLIqAbKZwt6U=; b=srDcQuvUhOpoS6XSk1yX5MJj4AVevyOIrmgCRMvuzNjXQLIIJpow8ZYVJGXOAFPuPp dWq6VKySkD2XWUUzXzIG2ru6B6Wv878jxXMs1gdAr3aqnnFjRuIcKatC3IvG5dnwH/Kd z4/S4ecfHLSdmxRm8YG5vzFFd23znMIXdf4yzi6tTZ9Ya3b7nLFTAFa+7ZDf7W6r4CWi EyPt3TlxjLEbVff+OZ+dGgaF0hP77PcnEgxL3ms/56P/1sF0YnhJdeaCxrk4u5MuDO/D tgXZVYJceqIWsNdo+g2jjh64bWPyb1+awZyn/9jfD5UZgLu8SmAj1gfMPiywgQmUfb17 iqdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=oyYLFWQZMneEnEj2VSn1LS09AGJVymYOLIqAbKZwt6U=; b=IQ4AYl/1bqEBXHG9vRMNStJtz8uz1nOJSbGvVrsIuLYvaHNYNN0dOIP5WSqFDy5SjU OfkDwA0B9dQrWIYMegSp49yE6mGxsNOq+5FhWtUaITEFVmA3ULb48kMLKXX0JjWSwjha Ss96QRj2XSoAsLC08uP4g8gIL+chmaFREVcRJd34d7TedHvl46Lyq6DVHcAsVKNVQzJk tvpvd8UFryVbt/n94xbh25aayMfL7YqwZBT+0LSpOnZQSuVsJyYEpcRyMFUOsymbvPZw SvsJC5Wli8PL1LKpqCbbXNlKTFJQa4uq2it/tLby/dVXF+PicUWIKansBfRSaaSkiea7 k6/g== X-Gm-Message-State: AOAM530wnFaOXwvb2/aY6eBIkhjCP/JuJgUeaFivB7A1X39Adrb72aIS keAVT17yUSEnpbVgBIM04MKQlw== X-Google-Smtp-Source: ABdhPJyXJxAcdnGaEOLfWk+JL7WUP6NfvyxPJK4EaBXXt8h/6IhmFqWc5LMhe1Lrb0MJFulrnvz4kg== X-Received: by 2002:a5d:46c6:: with SMTP id g6mr4819602wrs.170.1606406114096; Thu, 26 Nov 2020 07:55:14 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:f008:704d:8d4b:9951]) by smtp.gmail.com with ESMTPSA id b4sm10161280wmc.1.2020.11.26.07.55.12 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 26 Nov 2020 07:55:13 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v3 22/23] kvm: arm64: Trap host SMCs in protected mode Date: Thu, 26 Nov 2020 15:54:20 +0000 Message-Id: <20201126155421.14901-23-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201126155421.14901-1-dbrazdil@google.com> References: <20201126155421.14901-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201126_105515_724698_7B901860 X-CRM114-Status: GOOD ( 17.61 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org While protected nVHE KVM is installed, start trapping all host SMCs. By default, these are simply forwarded to EL3, but PSCI SMCs are validated first. Create new constant HCR_HOST_NVHE_PROTECTED_FLAGS with the new set of HCR flags to use while the nVHE vector is installed when the kernel was booted with the protected flag enabled. Switch back to the default HCR flags when switching back to the stub vector. Signed-off-by: David Brazdil --- arch/arm64/include/asm/kvm_arm.h | 1 + arch/arm64/kvm/hyp/nvhe/hyp-init.S | 10 ++++++++++ arch/arm64/kvm/hyp/nvhe/switch.c | 5 ++++- 3 files changed, 15 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h index 64ce29378467..4e90c2debf70 100644 --- a/arch/arm64/include/asm/kvm_arm.h +++ b/arch/arm64/include/asm/kvm_arm.h @@ -80,6 +80,7 @@ HCR_FMO | HCR_IMO | HCR_PTW ) #define HCR_VIRT_EXCP_MASK (HCR_VSE | HCR_VI | HCR_VF) #define HCR_HOST_NVHE_FLAGS (HCR_RW | HCR_API | HCR_APK | HCR_ATA) +#define HCR_HOST_NVHE_PROTECTED_FLAGS (HCR_HOST_NVHE_FLAGS | HCR_TSC) #define HCR_HOST_VHE_FLAGS (HCR_RW | HCR_TGE | HCR_E2H) /* TCR_EL2 Registers bits */ diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-init.S b/arch/arm64/kvm/hyp/nvhe/hyp-init.S index fbb195851fb9..7af18fa1983d 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-init.S +++ b/arch/arm64/kvm/hyp/nvhe/hyp-init.S @@ -88,6 +88,11 @@ SYM_CODE_END(__kvm_hyp_init) * x0: struct kvm_nvhe_init_params PA */ SYM_CODE_START(___kvm_hyp_init) +alternative_if ARM64_PROTECTED_KVM + mov_q x1, HCR_HOST_NVHE_PROTECTED_FLAGS + msr hcr_el2, x1 +alternative_else_nop_endif + ldr x1, [x0, #NVHE_INIT_TPIDR_EL2] msr tpidr_el2, x1 @@ -233,6 +238,11 @@ reset: msr sctlr_el2, x5 isb +alternative_if ARM64_PROTECTED_KVM + mov_q x5, HCR_HOST_NVHE_FLAGS + msr hcr_el2, x5 +alternative_else_nop_endif + /* Install stub vectors */ adr_l x5, __hyp_stub_vectors msr vbar_el2, x5 diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/switch.c index 3e50ff35aa4f..f3d0e9eca56c 100644 --- a/arch/arm64/kvm/hyp/nvhe/switch.c +++ b/arch/arm64/kvm/hyp/nvhe/switch.c @@ -97,7 +97,10 @@ static void __deactivate_traps(struct kvm_vcpu *vcpu) mdcr_el2 |= MDCR_EL2_E2PB_MASK << MDCR_EL2_E2PB_SHIFT; write_sysreg(mdcr_el2, mdcr_el2); - write_sysreg(HCR_HOST_NVHE_FLAGS, hcr_el2); + if (is_protected_kvm_enabled()) + write_sysreg(HCR_HOST_NVHE_PROTECTED_FLAGS, hcr_el2); + else + write_sysreg(HCR_HOST_NVHE_FLAGS, hcr_el2); write_sysreg(CPTR_EL2_DEFAULT, cptr_el2); write_sysreg(__kvm_hyp_host_vector, vbar_el2); } From patchwork Thu Nov 26 15:54:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11934341 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E471EC63697 for ; Thu, 26 Nov 2020 16:03:19 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6261E21D40 for ; Thu, 26 Nov 2020 16:03:19 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="Ny/n9xZM"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="QM/rbtXw" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6261E21D40 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=NY7OClwwGtW8qi1eX1kNxgi/aFK9jq2rnSlhNjmSAvo=; b=Ny/n9xZMWmPba4Q8Y4AjZu+/U iZot5oELHFOlg5nli+5v+gvVuOHcDUDa7k63QB29n+A5y99BxWWzU9HykD+PKwP0A0Bvqqk89HEyY 0Xd4MppaIh2eVBNnpE7wpUmlnUgn7Q5AkZa7ksK44zjSALtkkU28nZ0ln/hpnBbgIlJVWjZpIzkyU ZTKZz4uDRrUzjZPfUhR57BnxVh5PYY3vm4NerxeyAcu+aIM/71eWvMr4iPuabberiDD0R5qg0DF9S C/P/iXNLnJPmSenoQCda/ZSks5keKCbYmDPAfU45Vv8pNUxVOfKxNFZG7hN1Vr9EQ/iICq+EAKsl9 kvXZqojPg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kiJh4-0001pn-Os; Thu, 26 Nov 2020 16:00:28 +0000 Received: from mail-wm1-x341.google.com ([2a00:1450:4864:20::341]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kiJc7-0007ud-SD for linux-arm-kernel@lists.infradead.org; Thu, 26 Nov 2020 15:55:22 +0000 Received: by mail-wm1-x341.google.com with SMTP id p22so2568289wmg.3 for ; Thu, 26 Nov 2020 07:55:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=aOLJOeUdUyn9Tp5EB3J9cYHwXErbIHpd/Aan89z6ZHs=; b=QM/rbtXwOgu6dblgPCWfx4ngm1/thgvoQ0YW4L2bPmt43izjfGp+feFlrQOkpQS9Om 6IyiMWQDM9+9UTy3hB+J5mL08+VjIOU0v7HHn2JQNnWZfWWWlGmxzXwQJ5YeUMP4URzL 1QtnGLr7B86YDgrOowd0eJ/ZlG7g90/JdTPeSBg4Np5jEsxeOF3riX5Aj+ZJnFB1nucH eGRGKeHAcqx9n6NEDtpKYSogxV+eRc23vpbWc9NRm5ShbBf3SY7zuFqqGKiDs+I9VeCG lbM65gvfMyzKPRVDcbtqP39F0X2J1baGFNxxaE1G/RusVZXby+KoCinFSOxt/dKolC8/ f6PQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=aOLJOeUdUyn9Tp5EB3J9cYHwXErbIHpd/Aan89z6ZHs=; b=kjVIgqodOWOXldQFLrQGaa1HY1xJ+kvxTQP+xaAXrPLlDL0rtBOMyINsTYQjzQjQgK iHrIJal2yAe00YTiwdWj0fR5zCFxuFP169952DHIr5t48VoE7f4M8xRag+WKPx0WN3FP wpH4ynOHg7tN4MDu4EXtSz2smIXK1yKpPLu14W1GkGUFBZi9O66deVZqix473DM8ME36 n4TM5kiyCEPST+avAxoK7i5FURzv9shZQFYjr8GIg4yN1ZymXzsZPgh791hp7TYEdfex ljUBxDjF4BYG2vxYmFd7UzVR2pvUDObDn2KxjBnkZmqXJZNJ/X7oRx0xeGPE4ezi2lw9 TRUA== X-Gm-Message-State: AOAM532xEhhC+H8BiBUIHzOhTHzh46yl1WOYhud86PZACn0sD9zWnyj4 vSNZg3zc5f/9OVSqzSAtTg1gAw== X-Google-Smtp-Source: ABdhPJxNaayzFy0JXM9EQDhPZc7SkErVTVeHwN5g49VsWueupWVWXJ57HuFtoPXgm7dm47oDpr8iqg== X-Received: by 2002:a1c:bd08:: with SMTP id n8mr4124459wmf.136.1606406116246; Thu, 26 Nov 2020 07:55:16 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:f008:704d:8d4b:9951]) by smtp.gmail.com with ESMTPSA id j4sm9290305wrn.83.2020.11.26.07.55.14 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 26 Nov 2020 07:55:15 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v3 23/23] kvm: arm64: Fix EL2 mode availability checks Date: Thu, 26 Nov 2020 15:54:21 +0000 Message-Id: <20201126155421.14901-24-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201126155421.14901-1-dbrazdil@google.com> References: <20201126155421.14901-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201126_105520_154644_67E70B21 X-CRM114-Status: GOOD ( 19.63 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , kernel-team@android.com, Jonathan Corbet , Catalin Marinas , Suzuki K Poulose , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sudeep Holla , James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org With protected nVHE hyp code interception host's PSCI CPU_ON/SUSPEND SMCs, the host starts seeing new CPUs boot in EL1 instead of EL2. The kernel logic that keeps track of the boot mode needs to be adjusted. Add a static key enabled if KVM protected nVHE initialization is successful. When the key is enabled, is_hyp_mode_available continues to report `true` because its users either treat it as a check whether KVM will be / was initialized, or whether stub HVCs can be made (eg. hibernate). is_hyp_mode_mismatched is changed to report `false` when the key is enabled. That's because all cores' modes matched at the point of KVM init and KVM will not allow cores not present at init to boot. That said, the function is never used after KVM is initialized. Signed-off-by: David Brazdil --- arch/arm64/include/asm/virt.h | 18 ++++++++++++++++++ arch/arm64/kvm/arm.c | 10 +++++++--- 2 files changed, 25 insertions(+), 3 deletions(-) diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h index 2fde1186b962..f7cf3f0e5297 100644 --- a/arch/arm64/include/asm/virt.h +++ b/arch/arm64/include/asm/virt.h @@ -65,9 +65,19 @@ extern u32 __boot_cpu_mode[2]; void __hyp_set_vectors(phys_addr_t phys_vector_base); void __hyp_reset_vectors(void); +DECLARE_STATIC_KEY_FALSE(kvm_protected_mode_initialized); + /* Reports the availability of HYP mode */ static inline bool is_hyp_mode_available(void) { + /* + * If KVM protected mode is initialized, all CPUs must have been booted + * in EL2. Avoid checking __boot_cpu_mode as CPUs now come up in EL1. + */ + if (IS_ENABLED(CONFIG_KVM) && + static_branch_likely(&kvm_protected_mode_initialized)) + return true; + return (__boot_cpu_mode[0] == BOOT_CPU_MODE_EL2 && __boot_cpu_mode[1] == BOOT_CPU_MODE_EL2); } @@ -75,6 +85,14 @@ static inline bool is_hyp_mode_available(void) /* Check if the bootloader has booted CPUs in different modes */ static inline bool is_hyp_mode_mismatched(void) { + /* + * If KVM protected mode is initialized, all CPUs must have been booted + * in EL2. Avoid checking __boot_cpu_mode as CPUs now come up in EL1. + */ + if (IS_ENABLED(CONFIG_KVM) && + static_branch_likely(&kvm_protected_mode_initialized)) + return false; + return __boot_cpu_mode[0] != __boot_cpu_mode[1]; } diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 6ec8ddf74643..b153c08e50fa 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -46,6 +46,8 @@ __asm__(".arch_extension virt"); #endif +DEFINE_STATIC_KEY_FALSE(kvm_protected_mode_initialized); + DECLARE_KVM_HYP_PER_CPU(unsigned long, kvm_hyp_vector); static DEFINE_PER_CPU(unsigned long, kvm_arm_hyp_stack_page); @@ -1877,12 +1879,14 @@ int kvm_arch_init(void *opaque) if (err) goto out_hyp; - if (is_protected_kvm_enabled()) + if (is_protected_kvm_enabled()) { + static_branch_enable(&kvm_protected_mode_initialized); kvm_info("Protected nVHE mode initialized successfully\n"); - else if (in_hyp_mode) + } else if (in_hyp_mode) { kvm_info("VHE mode initialized successfully\n"); - else + } else { kvm_info("Hyp mode initialized successfully\n"); + } return 0;