From patchwork Wed Sep 16 17:34:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11780277 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D80B36CA for ; Wed, 16 Sep 2020 17:36:53 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A194120672 for ; Wed, 16 Sep 2020 17:36:53 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="zoDUA3km"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="PCdTbfdD" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A194120672 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=mf+0F4hr4cH3C9yjb+nprgXp8hu8vKetzzUmuQzjvAU=; b=zoDUA3kmjhmBo0cQTcmUbVomK ThiwGLkHyUl+EIV45FrtB3qzLFKKjASmD4OlLojTMg9ze23XgetAQ4jjKp+M+jWeaD4Ly7NRMKGNM 5VLZjUSvDn37SPS+HI2suoatSNJkg/NBVuoEZSfHWbmoL5uezMb4DgDBhZk8/jPGarlmQcWhFNHL5 p3coLYCmYabzFFgRUD9efwh4+AuXlWFulS4AuPIOHknUcBzmIaNlBjI/Oi2+M6eiLRCLTBtWv40+6 L7VkP8eCi4F8KTGZNJlV0nFqLLMH9AshzXb9lr7qex2YCwR4X75fkAmL+eVPnjr8fvfZzRpQQQdFU qHeJx6YLw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kIbKb-0000tm-Ei; Wed, 16 Sep 2020 17:34:57 +0000 Received: from mail-wm1-x32e.google.com ([2a00:1450:4864:20::32e]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kIbKS-0000pF-UK for linux-arm-kernel@lists.infradead.org; Wed, 16 Sep 2020 17:34:50 +0000 Received: by mail-wm1-x32e.google.com with SMTP id d4so3557013wmd.5 for ; Wed, 16 Sep 2020 10:34:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=jeccRoVH2xzd2nQpZdaO6Z+WgudupLqXpR0BDecSqSQ=; b=PCdTbfdDQ/ZI/CI69hr6atdnbbkN1TozI1DD1MkQa16lxMvmQu52uc4k3a9neBl9jh 5nkow1vbp1XzL5hRIFTrHl6gtF7ejYHqYb18RwIHoF+hKN80s197Hu7VywjYXmXbWLMm /sEe6JF5T3Dc2XtbHW64AotI8hwXowHxuZtLJEDTMIRTN655L8AvZxnFuqRqdTDLtafc WhT2AQQp3DBQsic9bNzoZoM9ENaP+DbSEFYaiU5B86/vfNdHhEBSjSR/SXAM4YAgT8+Y ty/yTr3FK4YDXEHSCVra6JmPkydeZ1zmORW7VxGB46T4DW4tnL8OTufsJ0HvX2kPuX4J uTxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=jeccRoVH2xzd2nQpZdaO6Z+WgudupLqXpR0BDecSqSQ=; b=MGknzS+1p3tn3J4qIzaqCsq2DXLsYcxyj47LSvXAiIPBH7ZZ80T+2JNLMlPTcnosM+ yQx9peomcpbxv83lL1GppILPXDY7Gelsoxb/bjncmbcy7kSnxCV9C7yblCrw5i/Lt1WC 8h35YCDRO9dWgLjp4RYbKal52fq5WV9cLttfdn8LnUeL2i+DlRNPnrMcKEtZtoikCftP nByKiT7iLj3NRHwsL0S3KpDxDyB/4Fd582gCFvacD+440g77k7QIoGrs11EdJUexexah 9Ti6EnIpV7GZOo7dRqR/oVLRMuWDa2wSYDfVYfhx6TIMgMEO+bRZBx8RMJ1a5/L1uw0x SKtA== X-Gm-Message-State: AOAM5309YwtzC9B9S3XyxSzO4cyQYXPvYK8IUUF2fEolr4UJM7FcdVBi 2PJkIfA3c4xquaXQiirrH2rwDg== X-Google-Smtp-Source: ABdhPJxtYWNdopesl9Q0NNtQrGFhJvZo052EApCUbh7x2DWg3tiDSUR1Z+UKPrMRUBzu4fiWi1PJnw== X-Received: by 2002:a1c:e90b:: with SMTP id q11mr5819113wmc.39.1600277687253; Wed, 16 Sep 2020 10:34:47 -0700 (PDT) Received: from localhost ([2a01:4b00:8523:2d03:e49d:f6be:d31b:ad3c]) by smtp.gmail.com with ESMTPSA id z83sm6911988wmb.4.2020.09.16.10.34.46 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 16 Sep 2020 10:34:46 -0700 (PDT) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v3 01/11] kvm: arm64: Partially link nVHE hyp code, simplify HYPCOPY Date: Wed, 16 Sep 2020 18:34:29 +0100 Message-Id: <20200916173439.32265-2-dbrazdil@google.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200916173439.32265-1-dbrazdil@google.com> References: <20200916173439.32265-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200916_133449_008924_4A4BA0A3 X-CRM114-Status: GOOD ( 24.63 ) X-Spam-Score: -15.7 (---------------) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (-15.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2a00:1450:4864:20:0:0:0:32e listed in] [list.dnswl.org] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list -7.5 USER_IN_DEF_SPF_WL From: address is in the default SPF white-list -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.5 ENV_AND_HDR_SPF_MATCH Env and Hdr From used in default SPF WL Match -0.0 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kernel-team@android.com, Suzuki K Poulose , Catalin Marinas , linux-kernel@vger.kernel.org, James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org Relying on objcopy to prefix the ELF section names of the nVHE hyp code is brittle and prevents us from using wildcards to match specific section names. Improve the build rules by partially linking all '.nvhe.o' files and prefixing their ELF section names using a linker script. Continue using objcopy for prefixing ELF symbol names. One immediate advantage of this approach is that all subsections matching a pattern can be merged into a single prefixed section, eg. .text and .text.* can be linked into a single '.hyp.text'. This removes the need for -fno-reorder-functions on GCC and will be useful in the future too: LTO builds use .text subsections, compilers routinely generate .rodata subsections, etc. Partially linking all hyp code into a single object file also makes it easier to analyze. Signed-off-by: David Brazdil Acked-by: Will Deacon --- arch/arm64/include/asm/hyp_image.h | 24 ++++++++++++ arch/arm64/kvm/hyp/nvhe/Makefile | 60 ++++++++++++++++-------------- arch/arm64/kvm/hyp/nvhe/hyp.lds.S | 13 +++++++ 3 files changed, 70 insertions(+), 27 deletions(-) create mode 100644 arch/arm64/include/asm/hyp_image.h create mode 100644 arch/arm64/kvm/hyp/nvhe/hyp.lds.S diff --git a/arch/arm64/include/asm/hyp_image.h b/arch/arm64/include/asm/hyp_image.h new file mode 100644 index 000000000000..5b1e3b9ef376 --- /dev/null +++ b/arch/arm64/include/asm/hyp_image.h @@ -0,0 +1,24 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (C) 2020 Google LLC. + * Written by David Brazdil + */ + +#ifndef __ARM64_HYP_IMAGE_H__ +#define __ARM64_HYP_IMAGE_H__ + +#ifdef LINKER_SCRIPT + +/* + * KVM nVHE ELF section names are prefixed with .hyp, to separate them + * from the kernel proper. + */ +#define HYP_SECTION_NAME(NAME) .hyp##NAME + +/* Defines an ELF hyp section from input section @NAME and its subsections. */ +#define HYP_SECTION(NAME) \ + HYP_SECTION_NAME(NAME) : { *(NAME NAME##.[0-9a-zA-Z_]*) } + +#endif /* LINKER_SCRIPT */ + +#endif /* __ARM64_HYP_IMAGE_H__ */ diff --git a/arch/arm64/kvm/hyp/nvhe/Makefile b/arch/arm64/kvm/hyp/nvhe/Makefile index aef76487edc2..2b27b60182f9 100644 --- a/arch/arm64/kvm/hyp/nvhe/Makefile +++ b/arch/arm64/kvm/hyp/nvhe/Makefile @@ -10,40 +10,46 @@ obj-y := timer-sr.o sysreg-sr.o debug-sr.o switch.o tlb.o hyp-init.o obj-y += ../vgic-v3-sr.o ../aarch32.o ../vgic-v2-cpuif-proxy.o ../entry.o \ ../fpsimd.o ../hyp-entry.o -obj-y := $(patsubst %.o,%.hyp.o,$(obj-y)) -extra-y := $(patsubst %.hyp.o,%.hyp.tmp.o,$(obj-y)) +## +## Build rules for compiling nVHE hyp code +## Output of this folder is `kvm_nvhe.o`, a partially linked object +## file containing all nVHE hyp code and data. +## -$(obj)/%.hyp.tmp.o: $(src)/%.c FORCE +hyp-obj := $(patsubst %.o,%.nvhe.o,$(obj-y)) +obj-y := kvm_nvhe.o +extra-y := $(hyp-obj) kvm_nvhe.tmp.o hyp.lds + +# 1) Compile all source files to `.nvhe.o` object files. The file extension +# avoids file name clashes for files shared with VHE. +$(obj)/%.nvhe.o: $(src)/%.c FORCE $(call if_changed_rule,cc_o_c) -$(obj)/%.hyp.tmp.o: $(src)/%.S FORCE +$(obj)/%.nvhe.o: $(src)/%.S FORCE $(call if_changed_rule,as_o_S) -$(obj)/%.hyp.o: $(obj)/%.hyp.tmp.o FORCE - $(call if_changed,hypcopy) -# Disable reordering functions by GCC (enabled at -O2). -# This pass puts functions into '.text.*' sections to aid the linker -# in optimizing ELF layout. See HYPCOPY comment below for more info. -ccflags-y += $(call cc-option,-fno-reorder-functions) +# 2) Compile linker script. +$(obj)/hyp.lds: $(src)/hyp.lds.S FORCE + $(call if_changed_dep,cpp_lds_S) + +# 3) Partially link all '.nvhe.o' files and apply the linker script. +# Prefixes names of ELF sections with '.hyp', eg. '.hyp.text'. +# Note: The following rule assumes that the 'ld' rule puts LDFLAGS before +# the list of dependencies to form '-T $(obj)/hyp.lds'. This is to +# keep the dependency on the target while avoiding an error from +# GNU ld if the linker script is passed to it twice. +LDFLAGS_kvm_nvhe.tmp.o := -r -T +$(obj)/kvm_nvhe.tmp.o: $(obj)/hyp.lds $(addprefix $(obj)/,$(hyp-obj)) FORCE + $(call if_changed,ld) + +# 4) Produce the final 'kvm_nvhe.o', ready to be linked into 'vmlinux'. +# Prefixes names of ELF symbols with '__kvm_nvhe_'. +$(obj)/kvm_nvhe.o: $(obj)/kvm_nvhe.tmp.o FORCE + $(call if_changed,hypcopy) # The HYPCOPY command uses `objcopy` to prefix all ELF symbol names -# and relevant ELF section names to avoid clashes with VHE code/data. -# -# Hyp code is assumed to be in the '.text' section of the input object -# files (with the exception of specialized sections such as -# '.hyp.idmap.text'). This assumption may be broken by a compiler that -# divides code into sections like '.text.unlikely' so as to optimize -# ELF layout. HYPCOPY checks that no such sections exist in the input -# using `objdump`, otherwise they would be linked together with other -# kernel code and not memory-mapped correctly at runtime. +# to avoid clashes with VHE code/data. quiet_cmd_hypcopy = HYPCOPY $@ - cmd_hypcopy = \ - if $(OBJDUMP) -h $< | grep -F '.text.'; then \ - echo "$@: function reordering not supported in nVHE hyp code" >&2; \ - /bin/false; \ - fi; \ - $(OBJCOPY) --prefix-symbols=__kvm_nvhe_ \ - --rename-section=.text=.hyp.text \ - $< $@ + cmd_hypcopy = $(OBJCOPY) --prefix-symbols=__kvm_nvhe_ $< $@ # Remove ftrace and Shadow Call Stack CFLAGS. # This is equivalent to the 'notrace' and '__noscs' annotations. diff --git a/arch/arm64/kvm/hyp/nvhe/hyp.lds.S b/arch/arm64/kvm/hyp/nvhe/hyp.lds.S new file mode 100644 index 000000000000..3b13d1c7cd1a --- /dev/null +++ b/arch/arm64/kvm/hyp/nvhe/hyp.lds.S @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (C) 2020 Google LLC. + * Written by David Brazdil + * + * Linker script used for partial linking of nVHE EL2 object files. + */ + +#include + +SECTIONS { + HYP_SECTION(.text) +} From patchwork Wed Sep 16 17:34:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11780263 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 69AAC746 for ; Wed, 16 Sep 2020 17:35:15 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3C5DA2080C for ; Wed, 16 Sep 2020 17:35:15 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="BCEBK1Ib"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="M2E0n8qJ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3C5DA2080C Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=6jlg0WbaDaIDLdUMUs3ZQK7fJHMCunDrW2fhW+rVhf4=; b=BCEBK1IbNBq+MHFo6L/BOjol6 /ARW0td030bWxL72m+pR5L+3u7kq+6HW7AqZHuvYQaBkP1yy/9DSc4AS784+ZoAGP1AOarSt5kwuX cCkcjpwBOvARx4VwURM/hSmk1/gHL2FY6hBvgDOz2KBfhjMCp6Pyguvo8FRvqRR+WUPWQryIqRLfZ +vT0ljv3UJxU4epGAd1CdlRIfb+f/P7F6fp37uzl43AV9Z2CLDC44vSkugwe2Gzy3Bjv6hHe1rNg9 XURvVTmxscFs02dEyj3GXPYOoxuNKfNjn69W76yoEK0+yD6aM/G0BzwFM2PIzanWgncXG9e0zFOk/ 1od1+dbNw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kIbKf-0000uU-29; Wed, 16 Sep 2020 17:35:01 +0000 Received: from mail-wr1-x444.google.com ([2a00:1450:4864:20::444]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kIbKU-0000pg-8d for linux-arm-kernel@lists.infradead.org; Wed, 16 Sep 2020 17:34:51 +0000 Received: by mail-wr1-x444.google.com with SMTP id z4so7803577wrr.4 for ; Wed, 16 Sep 2020 10:34:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Bb8x+HFH0B/5AJF/3eHINfl7a3Qa0+GTsxwxT6ItzrA=; b=M2E0n8qJMz1MSrSfqdhktCcaOa3AQGk1vmL6Kdjan+YvnN4Uq1JyhmWHaYif2qseKo 7MBBvjOhBg96grbm9ZP7di/5Ks/FE2YU5A0RILohThIR4aygnTZpnOQeqCylD42bNDyZ /GHnv4SVspVP2I7UqGWfOnLnq5PZJXg+7LmAU0usi0LcCoKhWVg20JXWk1gD7Dx9jJ2h urT8MmXA1a4IPXVtGsD2MRKI6+46OC9ZsTGpnmm2FO5qccUyI+KkWdAiMgPH10nR7JU0 V4lLP2rS82e7Q3ZSw1KO5ix3ERwkqgea+/sU+waTdQCUBZqwD4luhpsk5LacwlUFA16t 6RIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Bb8x+HFH0B/5AJF/3eHINfl7a3Qa0+GTsxwxT6ItzrA=; b=NqenQxZ6gyt0Duolfk2X1ncLY0qvVpO2STRpsfOLrNqEUCnZvOpZgSrtvkAa70N0In YReagvYcsuVANp/wgJSCBi1aThVpgHkr1IR512ySyc0CLeAC8XCIsNk53SpTJl1/dGVA hH7XhymRZ0mfv4CkDfjU1M01xThfIn6Fc5PLPrCGBTCbiNP0AI6OoXVD7EKs4aNWi465 5ClPnufiejMPwDlgjnP9CHisSZEopHIiZIhfw9ftkO5qTihGFYcnDnJfCYfFVnWw5YVt cdDJgnKtW9LmdIp7mPSNLfJxL5gr/lGX6CHZ4wm0Yuw9Kt42Xz0+7zd/cDP3rgU+nJ24 mGIQ== X-Gm-Message-State: AOAM530JSLbv6kr7XJnS1v/skYplKCVmiFqotEiwlS94kQvOviv8QPYu K+i9yJMqzORU4HYyVU3xrRr+eg== X-Google-Smtp-Source: ABdhPJyyusmPWRRPVeBCNm47H5a0PGpw8keKMBDtnH6wTG9FWTwQc0RSofvNwO/KVylWvPlmtAdRqQ== X-Received: by 2002:adf:8162:: with SMTP id 89mr28955370wrm.237.1600277689224; Wed, 16 Sep 2020 10:34:49 -0700 (PDT) Received: from localhost ([2a01:4b00:8523:2d03:e49d:f6be:d31b:ad3c]) by smtp.gmail.com with ESMTPSA id d19sm3530522wmd.0.2020.09.16.10.34.47 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 16 Sep 2020 10:34:48 -0700 (PDT) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v3 02/11] kvm: arm64: Move nVHE hyp namespace macros to hyp_image.h Date: Wed, 16 Sep 2020 18:34:30 +0100 Message-Id: <20200916173439.32265-3-dbrazdil@google.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200916173439.32265-1-dbrazdil@google.com> References: <20200916173439.32265-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200916_133450_358515_41E3F359 X-CRM114-Status: GOOD ( 16.59 ) X-Spam-Score: -15.7 (---------------) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (-15.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2a00:1450:4864:20:0:0:0:444 listed in] [list.dnswl.org] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list -7.5 USER_IN_DEF_SPF_WL From: address is in the default SPF white-list -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.5 ENV_AND_HDR_SPF_MATCH Env and Hdr From used in default SPF WL Match -0.0 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kernel-team@android.com, Suzuki K Poulose , Catalin Marinas , linux-kernel@vger.kernel.org, James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org Minor cleanup to move all macros related to prefixing nVHE hyp section and symbol names into one place: hyp_image.h. Signed-off-by: David Brazdil Acked-by: Will Deacon --- arch/arm64/include/asm/hyp_image.h | 12 ++++++++++++ arch/arm64/include/asm/kvm_asm.h | 8 +------- arch/arm64/kernel/image-vars.h | 2 -- 3 files changed, 13 insertions(+), 9 deletions(-) diff --git a/arch/arm64/include/asm/hyp_image.h b/arch/arm64/include/asm/hyp_image.h index 5b1e3b9ef376..9848de588238 100644 --- a/arch/arm64/include/asm/hyp_image.h +++ b/arch/arm64/include/asm/hyp_image.h @@ -7,6 +7,12 @@ #ifndef __ARM64_HYP_IMAGE_H__ #define __ARM64_HYP_IMAGE_H__ +/* + * KVM nVHE code has its own symbol namespace prefixed with __kvm_nvhe_, + * to separate it from the kernel proper. + */ +#define kvm_nvhe_sym(sym) __kvm_nvhe_##sym + #ifdef LINKER_SCRIPT /* @@ -19,6 +25,12 @@ #define HYP_SECTION(NAME) \ HYP_SECTION_NAME(NAME) : { *(NAME NAME##.[0-9a-zA-Z_]*) } +/* + * Defines a linker script alias of a kernel-proper symbol referenced by + * KVM nVHE hyp code. + */ +#define KVM_NVHE_ALIAS(sym) kvm_nvhe_sym(sym) = sym; + #endif /* LINKER_SCRIPT */ #endif /* __ARM64_HYP_IMAGE_H__ */ diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h index 6f98fbd0ac81..c085032e2e3e 100644 --- a/arch/arm64/include/asm/kvm_asm.h +++ b/arch/arm64/include/asm/kvm_asm.h @@ -7,6 +7,7 @@ #ifndef __ARM_KVM_ASM_H__ #define __ARM_KVM_ASM_H__ +#include #include #define VCPU_WORKAROUND_2_FLAG_SHIFT 0 @@ -42,13 +43,6 @@ #include -/* - * Translate name of a symbol defined in nVHE hyp to the name seen - * by kernel proper. All nVHE symbols are prefixed by the build system - * to avoid clashes with the VHE variants. - */ -#define kvm_nvhe_sym(sym) __kvm_nvhe_##sym - #define DECLARE_KVM_VHE_SYM(sym) extern char sym[] #define DECLARE_KVM_NVHE_SYM(sym) extern char kvm_nvhe_sym(sym)[] diff --git a/arch/arm64/kernel/image-vars.h b/arch/arm64/kernel/image-vars.h index 8982b68289b7..76da2ad1010c 100644 --- a/arch/arm64/kernel/image-vars.h +++ b/arch/arm64/kernel/image-vars.h @@ -61,8 +61,6 @@ __efistub__ctype = _ctype; * memory mappings. */ -#define KVM_NVHE_ALIAS(sym) __kvm_nvhe_##sym = sym; - /* Alternative callbacks for init-time patching of nVHE hyp code. */ KVM_NVHE_ALIAS(arm64_enable_wa2_handling); KVM_NVHE_ALIAS(kvm_patch_vector_branch); From patchwork Wed Sep 16 17:34:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11780279 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 09FE5139F for ; Wed, 16 Sep 2020 17:37:04 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9CD2020672 for ; Wed, 16 Sep 2020 17:37:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="J5shT5yP"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="kuz0lgsV" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9CD2020672 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=fRj2cPFhLmCFHTaJKPE4WGeTs+DSj1lL1ELT9I2fD60=; b=J5shT5yPyXtB4fQ1OlIZagTy8 Ux7KJL1/gXC7hQNolI+R98MJ1/OLztneX1aWTk8ttcriQwiKRtJeGq/jl7GSR1bxWqL7Kxuau0T7g YJZbHJERvqpt8rffjHXYvCFS9ER1b+W5mOKLBkwtO4AclGb1FXY9deLqck8UoQ4yzQjH4otBmGf3a l7xqc4ZJYAjZMDMO+5qP9JufeKlullRcteEhqL1Kwf/ffxJy0M5h3m1HBXdLFBVOBxpsszZFY0cIs RWFiRWMiah0fWAXok3HjM3mSclCTFjiKxJ8QJ2tGJkg95EGBNsJlIQUxbHB7szzpGsVPRkTtJOrYf cMRFv8YmQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kIbKl-0000wc-55; Wed, 16 Sep 2020 17:35:07 +0000 Received: from mail-wr1-x444.google.com ([2a00:1450:4864:20::444]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kIbKW-0000qU-4h for linux-arm-kernel@lists.infradead.org; Wed, 16 Sep 2020 17:34:53 +0000 Received: by mail-wr1-x444.google.com with SMTP id a17so7773898wrn.6 for ; Wed, 16 Sep 2020 10:34:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ki62FqIiIgJ8yaBZSKIYHGl1mRsUtoqotQFZS0p5Has=; b=kuz0lgsVij7wLKItfIOGcuCmtvwNys1htJnrFzCNRHaLjnVgElf4udXEubZns0dGAz NqFkPkuvm9TbHN4J5VCtVm7UDYsIf+4wGcbWnp5r2GIAd9r+/SBADS9HcvFM14SFeAoq P2YaGqpnPwU/jtKoPyr2enlSJh9RNiWe4T5dguCXaWfc0N0jpNjqUEXj8iL+3zzKLce5 gQOmsMW24e8arpUJ3wc5JQTLKSJiijZpRkX/RtKzwFR8HA3+DbxQCE5+3Z9HI+eiTgxx FbB0gtfKqTpG/ce9J6iO28HJ/sQI4vk6jtZNWmtWmSFxB01rO8VQFJ/ro1Fry1BHESOm 8VGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ki62FqIiIgJ8yaBZSKIYHGl1mRsUtoqotQFZS0p5Has=; b=FPKustkJEdf2VI30ovhEmdOAgTt2iIjbHet3KNk2H88nyVqMI3nNIhWDl2ea7bW5FK X4rq+rRfoGdyV6llAONyRbK7AI6ImjHMbqq+pxhcTRGLGOZanHplnzjg5TBUaTrmOa9c nbpAxASeng4Nf8rdAUo+WJhGZo8nDdfR5E92uYG1gEwPea5KzG+zL2XbXzNGeDEKUnmY yf7blln1sWv4bPrW7sjUHlFD3DuJ9N64AvYKzutRt7YwbdzjWAKJ/ec8ffEbgm7x8Mew Z2YTvQVpOZl8s5hCS09j5670h1SiSZDU+Rx/up9o4mezoWJrHVsLoTER+/xiqMycrDhE 4QXg== X-Gm-Message-State: AOAM5300bxKYsV5c4Ri9A6WAE3JphzVsHHUZPoBYQslxj/Kc8IHK7knd xav2U8j3SzyIq9NFSrR0aSN3Mg== X-Google-Smtp-Source: ABdhPJzQzJxXIxJM0jRA7b3rUmMwdVhHQPWJyctJSUBa7YGa64yNP8fcblWJOLLn4QkX3rVzCKMVDg== X-Received: by 2002:a05:6000:1184:: with SMTP id g4mr27622757wrx.20.1600277691141; Wed, 16 Sep 2020 10:34:51 -0700 (PDT) Received: from localhost ([2a01:4b00:8523:2d03:e49d:f6be:d31b:ad3c]) by smtp.gmail.com with ESMTPSA id o16sm32123548wrp.52.2020.09.16.10.34.49 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 16 Sep 2020 10:34:50 -0700 (PDT) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v3 03/11] kvm: arm64: Only define __kvm_ex_table for CONFIG_KVM Date: Wed, 16 Sep 2020 18:34:31 +0100 Message-Id: <20200916173439.32265-4-dbrazdil@google.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200916173439.32265-1-dbrazdil@google.com> References: <20200916173439.32265-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200916_133452_230409_6B148A4E X-CRM114-Status: GOOD ( 12.71 ) X-Spam-Score: -15.7 (---------------) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (-15.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2a00:1450:4864:20:0:0:0:444 listed in] [list.dnswl.org] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list -7.5 USER_IN_DEF_SPF_WL From: address is in the default SPF white-list -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.5 ENV_AND_HDR_SPF_MATCH Env and Hdr From used in default SPF WL Match -0.0 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kernel-team@android.com, Suzuki K Poulose , Catalin Marinas , linux-kernel@vger.kernel.org, James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org Minor cleanup that only creates __kvm_ex_table ELF section and related symbols if CONFIG_KVM is enabled. Signed-off-by: David Brazdil Acked-by: Will Deacon --- arch/arm64/kernel/vmlinux.lds.S | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index 7cba7623fcec..0d543570e811 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -21,11 +21,15 @@ ENTRY(_text) jiffies = jiffies_64; +#ifdef CONFIG_KVM #define HYPERVISOR_EXTABLE \ . = ALIGN(SZ_8); \ __start___kvm_ex_table = .; \ *(__kvm_ex_table) \ __stop___kvm_ex_table = .; +#else /* CONFIG_KVM */ +#define HYPERVISOR_EXTABLE +#endif #define HYPERVISOR_TEXT \ /* \ From patchwork Wed Sep 16 17:34:32 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11780281 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3536C139F for ; Wed, 16 Sep 2020 17:37:21 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A9EF020708 for ; Wed, 16 Sep 2020 17:37:20 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="teWhKJwc"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="Z52uL0av" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A9EF020708 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=7sjMf05usrynUZkvSJz1REat5sOooIAjW4bO8A37EGI=; b=teWhKJwcLt2zNxEIAFp6wAuKV u3u9PUw6Fr/k90ayj/Y5XhDSn4UxMv71q8RZAPPq6jTqKdNWLfGr+KP6BdA6+fsXpdYde7EDQKP7H 9cQmffnPPmWZSra7vK1Vr3VvlocNBSRBJa0jZ+J76IRso7X5zzlTvkxdoBOyblOunBE9U0MAcoddo BPL1KRkMCtQxLcAUfFKEw3qjTxHyf+YYe3pyj4Q9PGHXpdC/hvQ5KiYfFdJ/NKkN4FlP/xDzQxVLU ldzL47Ap2eKfMGHZ4qRE5czvd8+xGMaovy9NanWFx4WfUyqhxToAjMTPnX5BPdCaTPsALPGqgEuUS q5YIaAqkQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kIbKu-0000zj-A2; Wed, 16 Sep 2020 17:35:16 +0000 Received: from mail-wr1-x442.google.com ([2a00:1450:4864:20::442]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kIbKY-0000t8-Q0 for linux-arm-kernel@lists.infradead.org; Wed, 16 Sep 2020 17:34:56 +0000 Received: by mail-wr1-x442.google.com with SMTP id o5so7706812wrn.13 for ; Wed, 16 Sep 2020 10:34:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=//UIJKuiVTRRrspCnLdOhX4a4b1lbE5JqtpGTGr3jiw=; b=Z52uL0av7yOsLEyKoqbm0tv4yHwcy/58U0liuI/vgliA5YsLjwCwgMMouzHQn3GKIe qTNMvSzr4j2p1mQd4VOmMVPdomyPgWalVLhFKql9fKekptBcPE784SRU6uPRJ0vKXaLy l8OeDlarQfDToPaJsrreJ78USw9qA4HzEKI79auO0kbaJNZEk8pe7ZmreZUmn1FxZCEs fiGI7n8UsZDRwCV4PSCaDspYD6Fl3vWotR4D7IbOpbHGK4oBpW2mGLvv4LusU7FC+DJ3 JhPHI37Nlg6j6Vr4TTH6F7loZd9dfmsqk+jYUEE1SnEbaK+sdZz93gzLlneJREiiC/vr 2ZbA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=//UIJKuiVTRRrspCnLdOhX4a4b1lbE5JqtpGTGr3jiw=; b=KzfQ56rP8CVvcu/hGAqRk2IRgT/xNx+IwKkZS3mnStlxR/wSCVUJnqWspCxKVFqQgp kIZxfae6OCiihwsaxJ+HxRNIeAg7iwQH6jnMXKZB+VwCaAWAIdsL37utqmF5nYf4e/E3 o2VXImxKa6JViZk94IYqi8s5CbtLQ6aN1USGrDpc2QO36kKkRxATTCfPmtPilTAzP4Sg s/PyiKJw1utQEfF9kJxDwUDC9cFOmXNwJ9pXBCFd5hD9pgRNNmiXXNNhc5qDolZ3paDD Ti7WiD3jn2ZsMbLIMSf8ojFL9flcyskPEsSlN6SR6oMnL1d1YFNluxFFWUUbQKjovI15 Aowg== X-Gm-Message-State: AOAM531mfZ1YpFkwXKZBNH6dxpE3BfdGBF21kwXMZhh3mpV9/4s3X4tP 2wnGqUJgHCNZIE3DoqxW+O1egw== X-Google-Smtp-Source: ABdhPJxIu8CDAGApgUL5mm7ojV/RalVSJ7y0uQFelCq5Y/MJShfCPwbGmmPESt+FreIo6OoaUEZ1Cw== X-Received: by 2002:a5d:540e:: with SMTP id g14mr29024834wrv.148.1600277693139; Wed, 16 Sep 2020 10:34:53 -0700 (PDT) Received: from localhost ([2a01:4b00:8523:2d03:e49d:f6be:d31b:ad3c]) by smtp.gmail.com with ESMTPSA id t16sm34508322wrm.57.2020.09.16.10.34.51 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 16 Sep 2020 10:34:52 -0700 (PDT) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v3 04/11] kvm: arm64: Remove __hyp_this_cpu_read Date: Wed, 16 Sep 2020 18:34:32 +0100 Message-Id: <20200916173439.32265-5-dbrazdil@google.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200916173439.32265-1-dbrazdil@google.com> References: <20200916173439.32265-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200916_133454_925179_CBD33FD5 X-CRM114-Status: GOOD ( 20.50 ) X-Spam-Score: -15.7 (---------------) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (-15.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2a00:1450:4864:20:0:0:0:442 listed in] [list.dnswl.org] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list -7.5 USER_IN_DEF_SPF_WL From: address is in the default SPF white-list -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.5 ENV_AND_HDR_SPF_MATCH Env and Hdr From used in default SPF WL Match -0.0 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kernel-team@android.com, Suzuki K Poulose , Catalin Marinas , linux-kernel@vger.kernel.org, James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry , Andrew Scull Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org this_cpu_ptr is meant for use in kernel proper because it selects between TPIDR_EL1/2 based on nVHE/VHE. __hyp_this_cpu_ptr was used in hyp to always select TPIDR_EL2. Unify all users behind this_cpu_ptr and friends by selecting _EL2 register under __KVM_VHE/NVHE_HYPERVISOR__. Under CONFIG_DEBUG_PREEMPT, the kernel helpers perform a preemption check which is omitted by the hyp helpers. Preserve the behavior for nVHE by overriding the corresponding macros under __KVM_NVHE_HYPERVISOR__. Extend the checks into VHE hyp code. Acked-by: Andrew Scull Signed-off-by: David Brazdil --- arch/arm64/include/asm/kvm_asm.h | 20 -------------- arch/arm64/include/asm/percpu.h | 33 +++++++++++++++++++++-- arch/arm64/kvm/hyp/include/hyp/debug-sr.h | 4 +-- arch/arm64/kvm/hyp/include/hyp/switch.h | 8 +++--- arch/arm64/kvm/hyp/nvhe/switch.c | 2 +- arch/arm64/kvm/hyp/vhe/switch.c | 2 +- arch/arm64/kvm/hyp/vhe/sysreg-sr.c | 4 +-- 7 files changed, 41 insertions(+), 32 deletions(-) diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h index c085032e2e3e..c196eec25498 100644 --- a/arch/arm64/include/asm/kvm_asm.h +++ b/arch/arm64/include/asm/kvm_asm.h @@ -143,26 +143,6 @@ extern char __smccc_workaround_1_smc[__SMCCC_WORKAROUND_1_SMC_SZ]; addr; \ }) -/* - * Home-grown __this_cpu_{ptr,read} variants that always work at HYP, - * provided that sym is really a *symbol* and not a pointer obtained from - * a data structure. As for SHIFT_PERCPU_PTR(), the creative casting keeps - * sparse quiet. - */ -#define __hyp_this_cpu_ptr(sym) \ - ({ \ - void *__ptr; \ - __verify_pcpu_ptr(&sym); \ - __ptr = hyp_symbol_addr(sym); \ - __ptr += read_sysreg(tpidr_el2); \ - (typeof(sym) __kernel __force *)__ptr; \ - }) - -#define __hyp_this_cpu_read(sym) \ - ({ \ - *__hyp_this_cpu_ptr(sym); \ - }) - #define __KVM_EXTABLE(from, to) \ " .pushsection __kvm_ex_table, \"a\"\n" \ " .align 3\n" \ diff --git a/arch/arm64/include/asm/percpu.h b/arch/arm64/include/asm/percpu.h index 0b6409b89e5e..36f304401c38 100644 --- a/arch/arm64/include/asm/percpu.h +++ b/arch/arm64/include/asm/percpu.h @@ -19,7 +19,21 @@ static inline void set_my_cpu_offset(unsigned long off) :: "r" (off) : "memory"); } -static inline unsigned long __my_cpu_offset(void) +static inline unsigned long __hyp_my_cpu_offset(void) +{ + unsigned long off; + + /* + * We want to allow caching the value, so avoid using volatile and + * instead use a fake stack read to hazard against barrier(). + */ + asm("mrs %0, tpidr_el2" : "=r" (off) : + "Q" (*(const unsigned long *)current_stack_pointer)); + + return off; +} + +static inline unsigned long __kern_my_cpu_offset(void) { unsigned long off; @@ -35,7 +49,12 @@ static inline unsigned long __my_cpu_offset(void) return off; } -#define __my_cpu_offset __my_cpu_offset() + +#if defined(__KVM_NVHE_HYPERVISOR__) || defined(__KVM_VHE_HYPERVISOR__) +#define __my_cpu_offset __hyp_my_cpu_offset() +#else +#define __my_cpu_offset __kern_my_cpu_offset() +#endif #define PERCPU_RW_OPS(sz) \ static inline unsigned long __percpu_read_##sz(void *ptr) \ @@ -227,4 +246,14 @@ PERCPU_RET_OP(add, add, ldadd) #include +/* Redefine macros for nVHE hyp under DEBUG_PREEMPT to avoid its dependencies. */ +#if defined(__KVM_NVHE_HYPERVISOR__) && defined(CONFIG_DEBUG_PREEMPT) +#undef this_cpu_ptr +#define this_cpu_ptr raw_cpu_ptr +#undef __this_cpu_read +#define __this_cpu_read raw_cpu_read +#undef __this_cpu_write +#define __this_cpu_write raw_cpu_write +#endif + #endif /* __ASM_PERCPU_H */ diff --git a/arch/arm64/kvm/hyp/include/hyp/debug-sr.h b/arch/arm64/kvm/hyp/include/hyp/debug-sr.h index 5e28ea6aa097..4ebe9f558f3a 100644 --- a/arch/arm64/kvm/hyp/include/hyp/debug-sr.h +++ b/arch/arm64/kvm/hyp/include/hyp/debug-sr.h @@ -135,7 +135,7 @@ static inline void __debug_switch_to_guest_common(struct kvm_vcpu *vcpu) if (!(vcpu->arch.flags & KVM_ARM64_DEBUG_DIRTY)) return; - host_ctxt = &__hyp_this_cpu_ptr(kvm_host_data)->host_ctxt; + host_ctxt = &this_cpu_ptr(&kvm_host_data)->host_ctxt; guest_ctxt = &vcpu->arch.ctxt; host_dbg = &vcpu->arch.host_debug_state.regs; guest_dbg = kern_hyp_va(vcpu->arch.debug_ptr); @@ -154,7 +154,7 @@ static inline void __debug_switch_to_host_common(struct kvm_vcpu *vcpu) if (!(vcpu->arch.flags & KVM_ARM64_DEBUG_DIRTY)) return; - host_ctxt = &__hyp_this_cpu_ptr(kvm_host_data)->host_ctxt; + host_ctxt = &this_cpu_ptr(&kvm_host_data)->host_ctxt; guest_ctxt = &vcpu->arch.ctxt; host_dbg = &vcpu->arch.host_debug_state.regs; guest_dbg = kern_hyp_va(vcpu->arch.debug_ptr); diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h index 5b6b8fa00f0a..f150407fa798 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -386,7 +386,7 @@ static inline bool __hyp_handle_ptrauth(struct kvm_vcpu *vcpu) !esr_is_ptrauth_trap(kvm_vcpu_get_esr(vcpu))) return false; - ctxt = &__hyp_this_cpu_ptr(kvm_host_data)->host_ctxt; + ctxt = &this_cpu_ptr(&kvm_host_data)->host_ctxt; __ptrauth_save_key(ctxt, APIA); __ptrauth_save_key(ctxt, APIB); __ptrauth_save_key(ctxt, APDA); @@ -495,7 +495,7 @@ static inline void __set_guest_arch_workaround_state(struct kvm_vcpu *vcpu) * guest wants it disabled, so be it... */ if (__needs_ssbd_off(vcpu) && - __hyp_this_cpu_read(arm64_ssbd_callback_required)) + __this_cpu_read(arm64_ssbd_callback_required)) arm_smccc_1_1_smc(ARM_SMCCC_ARCH_WORKAROUND_2, 0, NULL); #endif } @@ -507,7 +507,7 @@ static inline void __set_host_arch_workaround_state(struct kvm_vcpu *vcpu) * If the guest has disabled the workaround, bring it back on. */ if (__needs_ssbd_off(vcpu) && - __hyp_this_cpu_read(arm64_ssbd_callback_required)) + __this_cpu_read(arm64_ssbd_callback_required)) arm_smccc_1_1_smc(ARM_SMCCC_ARCH_WORKAROUND_2, 1, NULL); #endif } @@ -521,7 +521,7 @@ static inline void __kvm_unexpected_el2_exception(void) entry = hyp_symbol_addr(__start___kvm_ex_table); end = hyp_symbol_addr(__stop___kvm_ex_table); - host_ctxt = &__hyp_this_cpu_ptr(kvm_host_data)->host_ctxt; + host_ctxt = &this_cpu_ptr(&kvm_host_data)->host_ctxt; while (entry < end) { addr = (unsigned long)&entry->insn + entry->insn; diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/switch.c index 0970442d2dbc..cc4f8e790fb3 100644 --- a/arch/arm64/kvm/hyp/nvhe/switch.c +++ b/arch/arm64/kvm/hyp/nvhe/switch.c @@ -175,7 +175,7 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu) vcpu = kern_hyp_va(vcpu); - host_ctxt = &__hyp_this_cpu_ptr(kvm_host_data)->host_ctxt; + host_ctxt = &this_cpu_ptr(&kvm_host_data)->host_ctxt; host_ctxt->__hyp_running_vcpu = vcpu; guest_ctxt = &vcpu->arch.ctxt; diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c index c1da4f86ccac..575e8054f116 100644 --- a/arch/arm64/kvm/hyp/vhe/switch.c +++ b/arch/arm64/kvm/hyp/vhe/switch.c @@ -108,7 +108,7 @@ static int __kvm_vcpu_run_vhe(struct kvm_vcpu *vcpu) struct kvm_cpu_context *guest_ctxt; u64 exit_code; - host_ctxt = &__hyp_this_cpu_ptr(kvm_host_data)->host_ctxt; + host_ctxt = &this_cpu_ptr(&kvm_host_data)->host_ctxt; host_ctxt->__hyp_running_vcpu = vcpu; guest_ctxt = &vcpu->arch.ctxt; diff --git a/arch/arm64/kvm/hyp/vhe/sysreg-sr.c b/arch/arm64/kvm/hyp/vhe/sysreg-sr.c index 996471e4c138..2a0b8c88d74f 100644 --- a/arch/arm64/kvm/hyp/vhe/sysreg-sr.c +++ b/arch/arm64/kvm/hyp/vhe/sysreg-sr.c @@ -66,7 +66,7 @@ void kvm_vcpu_load_sysregs_vhe(struct kvm_vcpu *vcpu) struct kvm_cpu_context *guest_ctxt = &vcpu->arch.ctxt; struct kvm_cpu_context *host_ctxt; - host_ctxt = &__hyp_this_cpu_ptr(kvm_host_data)->host_ctxt; + host_ctxt = &this_cpu_ptr(&kvm_host_data)->host_ctxt; __sysreg_save_user_state(host_ctxt); /* @@ -100,7 +100,7 @@ void kvm_vcpu_put_sysregs_vhe(struct kvm_vcpu *vcpu) struct kvm_cpu_context *guest_ctxt = &vcpu->arch.ctxt; struct kvm_cpu_context *host_ctxt; - host_ctxt = &__hyp_this_cpu_ptr(kvm_host_data)->host_ctxt; + host_ctxt = &this_cpu_ptr(&kvm_host_data)->host_ctxt; deactivate_traps_vhe_put(); __sysreg_save_el1_state(guest_ctxt); From patchwork Wed Sep 16 17:34:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11780265 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 344FF14F6 for ; Wed, 16 Sep 2020 17:35:36 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id ED56B206A4 for ; Wed, 16 Sep 2020 17:35:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="jMemZjWl"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="qMakas/E" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org ED56B206A4 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=hi+0kd2rMxa6788csJRYwSenAeBlPrlzXhRLVyjd8CA=; b=jMemZjWlqtJLnwIYcT5H2LVtp I5ktvBupY8yFg4CFz8s51Yj6MyeoeoolZnI1owkBq1xDIuOGsfH8Wy1ki1K2YwmEcSsC9UR8HrBty a+6Os2lr4XqS4yCJ7n5SztF8mpJIlIzcK0ItGI9bCbsFjlG+qWq8CadrTwUUPYbbZ4Y7mOwlrKOwr X8KA+S3dpNiFASQ5Uu78a2Bp4rESo0vEjmOATGGTGW48Vk8ktnHSz3w+Iu2YBRJBzHoqKIzTKA6ij 53ZGsoCKtHM3sSu4QD5/CsWu61Y4wio87bJnv8/DEv7gxW/HCmhjKPFf8dQo1xFtOPJRPcu9c2sU0 OXZo4Sb9g==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kIbKz-000115-KT; Wed, 16 Sep 2020 17:35:21 +0000 Received: from mail-wm1-x343.google.com ([2a00:1450:4864:20::343]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kIbKa-0000tQ-Ff for linux-arm-kernel@lists.infradead.org; Wed, 16 Sep 2020 17:34:57 +0000 Received: by mail-wm1-x343.google.com with SMTP id b79so3886094wmb.4 for ; Wed, 16 Sep 2020 10:34:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9sgzGfpFFC64RJRg+974gui8xymUAlAMqzcpem+/CSY=; b=qMakas/EqU8sthwlgBLnrwvnZRAraVI8Sv8JwWPnUfgEnVzZJSdrtVHvMt1ezF7z7H A/CJWmbIe4J9dCW8b0LTLe7Heh9CPHI643HmQCBrpkULf8M+cEXiyc0+aQbcTDAsf1v2 +n07zinh1/QDVCzCcP9y2XRmwPgdEs1CXndG+Zomxw2gl8eA2cp9tt2bxanJ+/59hf2A /m/n28TEXQlZYX3OIhG9T3Sixf56usD+TjQZ/+zNIZJNqfbTjzC6ijgK3F9kXhjOrOS3 4yT1Toi0Q6lOvmlLPznvI5ROFnBl/KAiCFVY5WND4p3AsI8mQUEIZDynexktrozfT8Lx 0RGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9sgzGfpFFC64RJRg+974gui8xymUAlAMqzcpem+/CSY=; b=sp54GnpdpQN36FH2OLOljji7o0tylNaSkkhy3p9DsE6100e+SkEoktRh1gSnvStnQ7 /CRhHH9kUidYpGWRzRUmkVkc40aH3ICKXweGUYdEcVW+kHcEN5tF5KMB0K4gHIMOov2q HAd43nvkNVaCzqOAFSPI2+11InTcicue7RoXav680LR1bz532EnMNq/yAwx3kVgc2uT1 9MyLpj/wBqDxiW/ZGo49N68c+XzPSKJORcEfPOM2PXvgM2OJQbPBOEh9BR7eAiLbrPrw X9SeDzi1eGrA439cTsnMao1J0KbPBFsmkROcyWu7pjQ9VySaSIbNJBPkIU5/1ZUWl5Q6 fxaw== X-Gm-Message-State: AOAM533YfjL8y72raktv/u4xu03BDy8SKXvIWs7YU/mIEKy7fJUzsO6q RLZVZmnZxPAOaYB3qQO/6q4hpw== X-Google-Smtp-Source: ABdhPJxFj6X/hCuUE/N+733RQjEZLOd7n2vySf2bWA+cMRMhZxuMBy1BHCIG9TmmZ1mKvjjqjdC8yA== X-Received: by 2002:a7b:c192:: with SMTP id y18mr5703981wmi.108.1600277695176; Wed, 16 Sep 2020 10:34:55 -0700 (PDT) Received: from localhost ([2a01:4b00:8523:2d03:e49d:f6be:d31b:ad3c]) by smtp.gmail.com with ESMTPSA id m1sm6108804wmc.28.2020.09.16.10.34.53 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 16 Sep 2020 10:34:54 -0700 (PDT) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v3 05/11] kvm: arm64: Remove hyp_adr/ldr_this_cpu Date: Wed, 16 Sep 2020 18:34:33 +0100 Message-Id: <20200916173439.32265-6-dbrazdil@google.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200916173439.32265-1-dbrazdil@google.com> References: <20200916173439.32265-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200916_133456_590698_DE3CBB1D X-CRM114-Status: GOOD ( 14.82 ) X-Spam-Score: -15.7 (---------------) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (-15.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2a00:1450:4864:20:0:0:0:343 listed in] [list.dnswl.org] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list -7.5 USER_IN_DEF_SPF_WL From: address is in the default SPF white-list -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.5 ENV_AND_HDR_SPF_MATCH Env and Hdr From used in default SPF WL Match -0.0 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kernel-team@android.com, Suzuki K Poulose , Catalin Marinas , linux-kernel@vger.kernel.org, James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry , Andrew Scull Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org The hyp_adr/ldr_this_cpu helpers were introduced for use in hyp code because they always needed to use TPIDR_EL2 for base, while adr/ldr_this_cpu from kernel proper would select between TPIDR_EL2 and _EL1 based on VHE/nVHE. Simplify this now that the hyp mode case can be handled using the __KVM_VHE/NVHE_HYPERVISOR__ macros. Acked-by: Andrew Scull Signed-off-by: David Brazdil Acked-by: Will Deacon --- arch/arm64/include/asm/assembler.h | 27 +++++++++++++++++---------- arch/arm64/include/asm/kvm_asm.h | 14 +------------- arch/arm64/kvm/hyp/hyp-entry.S | 2 +- 3 files changed, 19 insertions(+), 24 deletions(-) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 54d181177656..f79231a0f949 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -218,6 +218,21 @@ lr .req x30 // link register str \src, [\tmp, :lo12:\sym] .endm + /* + * @dst: destination register (32 or 64 bit wide) + */ + .macro this_cpu_offset, dst +#if defined(__KVM_NVHE_HYPERVISOR__) || defined(__KVM_VHE_HYPERVISOR__) + mrs \dst, tpidr_el2 +#else +alternative_if_not ARM64_HAS_VIRT_HOST_EXTN + mrs \dst, tpidr_el1 +alternative_else + mrs \dst, tpidr_el2 +alternative_endif +#endif + .endm + /* * @dst: Result of per_cpu(sym, smp_processor_id()) (can be SP) * @sym: The name of the per-cpu variable @@ -226,11 +241,7 @@ lr .req x30 // link register .macro adr_this_cpu, dst, sym, tmp adrp \tmp, \sym add \dst, \tmp, #:lo12:\sym -alternative_if_not ARM64_HAS_VIRT_HOST_EXTN - mrs \tmp, tpidr_el1 -alternative_else - mrs \tmp, tpidr_el2 -alternative_endif + this_cpu_offset \tmp add \dst, \dst, \tmp .endm @@ -241,11 +252,7 @@ alternative_endif */ .macro ldr_this_cpu dst, sym, tmp adr_l \dst, \sym -alternative_if_not ARM64_HAS_VIRT_HOST_EXTN - mrs \tmp, tpidr_el1 -alternative_else - mrs \tmp, tpidr_el2 -alternative_endif + this_cpu_offset \tmp ldr \dst, [\dst, \tmp] .endm diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h index c196eec25498..cf9456663289 100644 --- a/arch/arm64/include/asm/kvm_asm.h +++ b/arch/arm64/include/asm/kvm_asm.h @@ -173,20 +173,8 @@ extern char __smccc_workaround_1_smc[__SMCCC_WORKAROUND_1_SMC_SZ]; #else /* __ASSEMBLY__ */ -.macro hyp_adr_this_cpu reg, sym, tmp - adr_l \reg, \sym - mrs \tmp, tpidr_el2 - add \reg, \reg, \tmp -.endm - -.macro hyp_ldr_this_cpu reg, sym, tmp - adr_l \reg, \sym - mrs \tmp, tpidr_el2 - ldr \reg, [\reg, \tmp] -.endm - .macro get_host_ctxt reg, tmp - hyp_adr_this_cpu \reg, kvm_host_data, \tmp + adr_this_cpu \reg, kvm_host_data, \tmp add \reg, \reg, #HOST_DATA_CONTEXT .endm diff --git a/arch/arm64/kvm/hyp/hyp-entry.S b/arch/arm64/kvm/hyp/hyp-entry.S index 46b4dab933d0..fba91c2ab410 100644 --- a/arch/arm64/kvm/hyp/hyp-entry.S +++ b/arch/arm64/kvm/hyp/hyp-entry.S @@ -132,7 +132,7 @@ alternative_cb_end str x0, [x2, #VCPU_WORKAROUND_FLAGS] /* Check that we actually need to perform the call */ - hyp_ldr_this_cpu x0, arm64_ssbd_callback_required, x2 + ldr_this_cpu x0, arm64_ssbd_callback_required, x2 cbz x0, wa2_end mov w0, #ARM_SMCCC_ARCH_WORKAROUND_2 From patchwork Wed Sep 16 17:34:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11780283 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F40316CA for ; Wed, 16 Sep 2020 17:37:38 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B685920708 for ; Wed, 16 Sep 2020 17:37:38 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="INsw5KVj"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="qo4UjMaG" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B685920708 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=5ebkeWslDjUjWg3fuGKaIpM0Wa/Ps26YPyNmcAPNTUM=; b=INsw5KVjHUyhryCN3V09ssnKj 5i9oL0LPz1nWOKP2d4xp30GeysWG53vk80P3KKKezk1lQ7dI1e/vBGycAR5qeVeK3MciNnyfhh8I6 jcr5++WIkZ7s3SmMuiE0zKZNw/yGKhVNO5w/CjVnl1YfaW5d11CI7ZQsk6Z48HSwZMkyvD08yo8NC 5XVYKgCFuIGUby/PLGGG1hkkXuWj9jHI6++q6rAEkDzL7Sn8/m3Boj5bD6R5r6CjcMYwhbXRtvUKf 4FkmqFaVKJ/w1Kz3GufbLxvDa96ENEwpBLiW2512b6ICIUPJizPTngoZQkr0AP2iys2lARSYf7MQ/ NkB3Hi9HQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kIbL8-00012l-Lp; Wed, 16 Sep 2020 17:35:30 +0000 Received: from mail-wr1-x443.google.com ([2a00:1450:4864:20::443]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kIbKc-0000tt-OC for linux-arm-kernel@lists.infradead.org; Wed, 16 Sep 2020 17:34:59 +0000 Received: by mail-wr1-x443.google.com with SMTP id o5so7706996wrn.13 for ; Wed, 16 Sep 2020 10:34:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=jm9txujGRbU+DIE5pQGENaBdVyNjHKuT2OQRoaKu5mk=; b=qo4UjMaG+LBmqizh/dguDo/1ZgFI6RSCpgvcu7UWxVpteEpYt6NHNQhs2cLgolyhud x3Ot+cX0Sp2hdxCGGm2R7QvodsCr54yNugnLPLPt6pSz/0vqaUvUP1vkJImvPJavE1JU /KWaXjiMd2VyyjXhCcJVrjDOEZ7J/mXd9xq5tVPX0hBQl/hG2W1oMc+sNcdIQEjaEbyg P5irBhmyRSSZ3oYyYC65aFW/vOXx2S2ACJsVISa0BOHj761vLtFGseGu2WvmEE1bh+RI 4Uivcqab87R5WzqTKUqjgCu6wND91b07/46tLA512z6Ayp8j0YmNjO6b4dewhw1zZG7S pKyQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=jm9txujGRbU+DIE5pQGENaBdVyNjHKuT2OQRoaKu5mk=; b=WajtAaZyxgl0tFy9LRc4HkOp6Wm6vOThihPfhWqHgExb0e9+T/3gDH1lG4Z46aUfSa CdzWawRWekWc3CJgcBivp+358FgrzD63W+Ug4RzrSwYWRNirmu3PU8J157KTp9hSG4kf DogAhwPN7E2so2S/8rzmEhrqDQLYjwS3sSRz6eT9sDYevPc5iMkNlNg9vN6wcALI08Ex M4YOyI7co55lmIfbV4E32ASmoMCV0NKiDv0AQDgCm9nKzi2aiVav7AIhp8aaatom/7bt t8eB2uAWUq9yu7MdNLUXwKHiwGNHGzQd9BuXbsyLREcnxwAzQxLIexJmd0os2Up+Bz0z +WUw== X-Gm-Message-State: AOAM530sGS2auMQjJrlzmrjUTjA/4QsgfsNeovES5HgwrStywqwufO+L SbV17/of0Ej0hvzClU8tC9b3Rg== X-Google-Smtp-Source: ABdhPJwU4kbhpWG5C/yjfJonDHsb6tdTDo9eAoD2gMl5GCOkrw+z409Io9EwIzx1e93E00CEZFWrkw== X-Received: by 2002:a05:6000:118a:: with SMTP id g10mr28697445wrx.67.1600277697172; Wed, 16 Sep 2020 10:34:57 -0700 (PDT) Received: from localhost ([2a01:4b00:8523:2d03:e49d:f6be:d31b:ad3c]) by smtp.gmail.com with ESMTPSA id v204sm6596978wmg.20.2020.09.16.10.34.55 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 16 Sep 2020 10:34:56 -0700 (PDT) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v3 06/11] kvm: arm64: Add helpers for accessing nVHE hyp per-cpu vars Date: Wed, 16 Sep 2020 18:34:34 +0100 Message-Id: <20200916173439.32265-7-dbrazdil@google.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200916173439.32265-1-dbrazdil@google.com> References: <20200916173439.32265-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200916_133458_809270_93D6DB54 X-CRM114-Status: GOOD ( 15.03 ) X-Spam-Score: -15.7 (---------------) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (-15.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2a00:1450:4864:20:0:0:0:443 listed in] [list.dnswl.org] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list -7.5 USER_IN_DEF_SPF_WL From: address is in the default SPF white-list -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.5 ENV_AND_HDR_SPF_MATCH Env and Hdr From used in default SPF WL Match -0.0 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kernel-team@android.com, Suzuki K Poulose , Catalin Marinas , linux-kernel@vger.kernel.org, James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org Defining a per-CPU variable in hyp/nvhe will result in its name being prefixed with __kvm_nvhe_. Add helpers for declaring these variables in kernel proper and accessing them with this_cpu_ptr and per_cpu_ptr. Signed-off-by: David Brazdil Acked-by: Will Deacon --- arch/arm64/include/asm/kvm_asm.h | 25 +++++++++++++++++++++++-- 1 file changed, 23 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h index cf9456663289..abc03f386b40 100644 --- a/arch/arm64/include/asm/kvm_asm.h +++ b/arch/arm64/include/asm/kvm_asm.h @@ -54,9 +54,21 @@ DECLARE_KVM_VHE_SYM(sym); \ DECLARE_KVM_NVHE_SYM(sym) +#define DECLARE_KVM_VHE_PER_CPU(type, sym) \ + DECLARE_PER_CPU(type, sym) +#define DECLARE_KVM_NVHE_PER_CPU(type, sym) \ + DECLARE_PER_CPU(type, kvm_nvhe_sym(sym)) + +#define DECLARE_KVM_HYP_PER_CPU(type, sym) \ + DECLARE_KVM_VHE_PER_CPU(type, sym); \ + DECLARE_KVM_NVHE_PER_CPU(type, sym) + #define CHOOSE_VHE_SYM(sym) sym #define CHOOSE_NVHE_SYM(sym) kvm_nvhe_sym(sym) +#define this_cpu_ptr_nvhe(sym) this_cpu_ptr(&kvm_nvhe_sym(sym)) +#define per_cpu_ptr_nvhe(sym, cpu) per_cpu_ptr(&kvm_nvhe_sym(sym), cpu) + #ifndef __KVM_NVHE_HYPERVISOR__ /* * BIG FAT WARNINGS: @@ -69,12 +81,21 @@ * - Don't let the nVHE hypervisor have access to this, as it will * pick the *wrong* symbol (yes, it runs at EL2...). */ -#define CHOOSE_HYP_SYM(sym) (is_kernel_in_hyp_mode() ? CHOOSE_VHE_SYM(sym) \ +#define CHOOSE_HYP_SYM(sym) (is_kernel_in_hyp_mode() \ + ? CHOOSE_VHE_SYM(sym) \ : CHOOSE_NVHE_SYM(sym)) +#define this_cpu_ptr_hyp(sym) (is_kernel_in_hyp_mode() \ + ? this_cpu_ptr(&sym) \ + : this_cpu_ptr_nvhe(sym)) +#define per_cpu_ptr_hyp(sym, cpu) (is_kernel_in_hyp_mode() \ + ? per_cpu_ptr(&sym, cpu) \ + : per_cpu_ptr_nvhe(sym, cpu)) #else /* The nVHE hypervisor shouldn't even try to access anything */ extern void *__nvhe_undefined_symbol; -#define CHOOSE_HYP_SYM(sym) __nvhe_undefined_symbol +#define CHOOSE_HYP_SYM(sym) __nvhe_undefined_symbol +#define this_cpu_ptr_hyp(sym) (&__nvhe_undefined_symbol) +#define per_cpu_ptr_hyp(sym, cpu) (&__nvhe_undefined_symbol) #endif /* Translate a kernel address @ptr into its equivalent linear mapping */ From patchwork Wed Sep 16 17:34:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11780267 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DD59D746 for ; Wed, 16 Sep 2020 17:36:14 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A55C620708 for ; Wed, 16 Sep 2020 17:36:14 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="vH1jsgqa"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="cKP/rHoQ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A55C620708 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=3vDDETbs2wOFYjYGPePswQjfty+2UBZFeHKBRUWKeVI=; b=vH1jsgqantY+AdYoTcUmHK2pu OIsgAw7f578z72oNvd+/zkUgEB6XDftP2WblVNjU+AI++4lugfovuewvqhdXk/DYKwmtBEisIOubw +1wKm3OQNuS62V7pLDa5horUE5/ynFKW0sShp4fA4RwQ+m43zw8f3ATH9wQpiksr+oNoQNgjdHMS8 9ijjvecF3ddU7gHcu3Q12JrrIt7Y1nr7UvWpyYvYMSojzSpX2yOVTlczaJ0/2WT7zPwQyFxvXmZTD wcsGEQIwavU66qxrqbcWe9mRRE4jTwaF4HbDg0IK1LZSnZa9YudwyLOB54lCH2oK0DJOr51A7twZs MCqBejO6A==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kIbLZ-0001BI-0z; Wed, 16 Sep 2020 17:35:57 +0000 Received: from mail-wr1-x444.google.com ([2a00:1450:4864:20::444]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kIbKe-0000uR-5a for linux-arm-kernel@lists.infradead.org; Wed, 16 Sep 2020 17:35:01 +0000 Received: by mail-wr1-x444.google.com with SMTP id m6so7825173wrn.0 for ; Wed, 16 Sep 2020 10:34:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=N/a5esHE2nX/K6yt6KRtONQ/utoU9A1KMk/TmGOQstA=; b=cKP/rHoQVyZyHVqZKyh7DfsE+mZFPUCn8SD3R2cnVqPlL2IpZN/1aLKkpOuKklj1R1 Ff4dtVMf31VqV7eFBh6ys+o6DxOtke8LtPBezFeyWpXkzs3EU+umYnaWkrQnejG9YXnB sHAqmCN4KOZBEQnZDiWlay4r0GDhl5YJGXgS8X0RlMcYif6v2kf7SYPcj7OcW10ScDk1 vjhBdmd1g44WnQjvQUVjyp6PazaRbDijYf2bR5RSFaB6frYEk96o78v/rIri2yBfQGOr AGYUvN3smNU0kV381ttfiFlS2yJpKppiHzhvMlNiMB9av3Maor7OK9Esa5xZHrFb599K 0BZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=N/a5esHE2nX/K6yt6KRtONQ/utoU9A1KMk/TmGOQstA=; b=CZMOCGPZHmeM1/4XxfmhnOsxKfRD0LDFawu+tYE9RqHjRCibUZzts5hc+wiuL6G8Ai 6ieanZlMQayP+Z5jsoRFwQNWWSnmhRCF2a9z/NuxV2yC2vuiNlgJbz6tnEG0Cc9Pair7 wIYsh1ithzyqvyzPK8bsrHublMv2pdurCKR2B1/pc8FqSyxfyOUZ1f6v4bMJ86m2xgpq PIm2lrL3T36aX1bm0pR8oe90i3vYJYK2dwIfTVIrhYRzHLu3C5aoOoBG0pmS8eUdntOf kw+OM9SV4tD28L5DjbvxSGhsiDOBWMPjMzVpWqjmYPXL1IOt9TOKIyfmjeXOcRsn2hP3 TjHQ== X-Gm-Message-State: AOAM533FzVYe6n3qWai/XeGUSzBAzws2ROBakh2p/xuUdmvJJsS6N+3y W7NTOLI4v1vd55jYdiKlVZZJPw== X-Google-Smtp-Source: ABdhPJxomQAbrt08pj/u0T2EZWQsduo0LZzXxFu69uumIM0NIEhf5lzxyS7IebV8zrXzmcgv8Rs77w== X-Received: by 2002:adf:f3c4:: with SMTP id g4mr28414841wrp.168.1600277699045; Wed, 16 Sep 2020 10:34:59 -0700 (PDT) Received: from localhost ([2a01:4b00:8523:2d03:e49d:f6be:d31b:ad3c]) by smtp.gmail.com with ESMTPSA id i3sm33528281wrs.4.2020.09.16.10.34.57 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 16 Sep 2020 10:34:58 -0700 (PDT) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v3 07/11] kvm: arm64: Duplicate arm64_ssbd_callback_required for nVHE hyp Date: Wed, 16 Sep 2020 18:34:35 +0100 Message-Id: <20200916173439.32265-8-dbrazdil@google.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200916173439.32265-1-dbrazdil@google.com> References: <20200916173439.32265-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200916_133500_247153_A15CA45E X-CRM114-Status: GOOD ( 18.05 ) X-Spam-Score: -15.7 (---------------) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (-15.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2a00:1450:4864:20:0:0:0:444 listed in] [list.dnswl.org] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list -7.5 USER_IN_DEF_SPF_WL From: address is in the default SPF white-list -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.5 ENV_AND_HDR_SPF_MATCH Env and Hdr From used in default SPF WL Match -0.0 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kernel-team@android.com, Suzuki K Poulose , Catalin Marinas , linux-kernel@vger.kernel.org, James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org Hyp keeps track of which cores require SSBD callback by accessing a kernel-proper global variable. Create an nVHE symbol of the same name and copy the value from kernel proper to nVHE at KVM init time. Done in preparation for separating percpu memory owned by kernel proper and nVHE. Signed-off-by: David Brazdil --- arch/arm64/include/asm/kvm_mmu.h | 10 +++++++--- arch/arm64/kernel/image-vars.h | 1 - arch/arm64/kvm/arm.c | 2 +- arch/arm64/kvm/hyp/nvhe/switch.c | 3 +++ 4 files changed, 11 insertions(+), 5 deletions(-) diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h index 189839c3706a..9db93da35606 100644 --- a/arch/arm64/include/asm/kvm_mmu.h +++ b/arch/arm64/include/asm/kvm_mmu.h @@ -529,23 +529,27 @@ static inline int kvm_map_vectors(void) #ifdef CONFIG_ARM64_SSBD DECLARE_PER_CPU_READ_MOSTLY(u64, arm64_ssbd_callback_required); +DECLARE_KVM_NVHE_PER_CPU(u64, arm64_ssbd_callback_required); -static inline int hyp_map_aux_data(void) +static inline int hyp_init_aux_data(void) { int cpu, err; for_each_possible_cpu(cpu) { u64 *ptr; - ptr = per_cpu_ptr(&arm64_ssbd_callback_required, cpu); + ptr = per_cpu_ptr_nvhe(arm64_ssbd_callback_required, cpu); err = create_hyp_mappings(ptr, ptr + 1, PAGE_HYP); if (err) return err; + + /* Copy value from kernel to hyp. */ + *ptr = per_cpu(arm64_ssbd_callback_required, cpu); } return 0; } #else -static inline int hyp_map_aux_data(void) +static inline int hyp_init_aux_data(void) { return 0; } diff --git a/arch/arm64/kernel/image-vars.h b/arch/arm64/kernel/image-vars.h index 76da2ad1010c..59d12a0b4622 100644 --- a/arch/arm64/kernel/image-vars.h +++ b/arch/arm64/kernel/image-vars.h @@ -67,7 +67,6 @@ KVM_NVHE_ALIAS(kvm_patch_vector_branch); KVM_NVHE_ALIAS(kvm_update_va_mask); /* Global kernel state accessed by nVHE hyp code. */ -KVM_NVHE_ALIAS(arm64_ssbd_callback_required); KVM_NVHE_ALIAS(kvm_host_data); KVM_NVHE_ALIAS(kvm_vgic_global_state); diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index b588c3b5c2f0..3bdc2661d276 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1552,7 +1552,7 @@ static int init_hyp_mode(void) } } - err = hyp_map_aux_data(); + err = hyp_init_aux_data(); if (err) kvm_err("Cannot map host auxiliary data: %d\n", err); diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/switch.c index cc4f8e790fb3..4662df6330d7 100644 --- a/arch/arm64/kvm/hyp/nvhe/switch.c +++ b/arch/arm64/kvm/hyp/nvhe/switch.c @@ -27,6 +27,9 @@ #include #include +/* Non-VHE copy of the kernel symbol. */ +DEFINE_PER_CPU_READ_MOSTLY(u64, arm64_ssbd_callback_required); + static void __activate_traps(struct kvm_vcpu *vcpu) { u64 val; From patchwork Wed Sep 16 17:34:36 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11780285 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7336F6CA for ; Wed, 16 Sep 2020 17:38:07 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 42CEC20672 for ; Wed, 16 Sep 2020 17:38:07 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="TtEXceCG"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="vr7FzTK3" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 42CEC20672 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=VqHHl/gxfZnpvyPyMeLC6MqMlKiDncpeocwq6XnuQzY=; b=TtEXceCGPMCph81e8jdKDRHaT gY0XRb8Au4qVmLxLxViyzpG4Kc+2B7NdDx1RWlo2hLQUPiZhFgTr2IonoB149Jx038TCJ5sZxM6iC a5Uzp6O+zCTilWltHsD4elEX0dR5P4Cc5XZ6r1kZY6StnecCvBXx4J3AzoY9Pj/hNEjo86S7+KcfD yF7wA2hwIGzpQXMSayyw4JiOPKGiM813kXNNwG/eq3M15Ev32/vrtoz6cTaFu2E8Q0bJ1BnPwXfQs T/OXk1kIaaOTzfuk3KyNJw8blc3TfrRxHSYfa2NobJiCQy+QbtObqNtWWVIJTeb77yJ5sUcS+g7nv mJi/9eurA==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kIbLj-0001Hv-UY; Wed, 16 Sep 2020 17:36:07 +0000 Received: from mail-wr1-x444.google.com ([2a00:1450:4864:20::444]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kIbKg-0000v5-1P for linux-arm-kernel@lists.infradead.org; Wed, 16 Sep 2020 17:35:03 +0000 Received: by mail-wr1-x444.google.com with SMTP id w5so7764873wrp.8 for ; Wed, 16 Sep 2020 10:35:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=UGSLdlYJxr2Is1FgLjfVIxpXo24NFut3vGLdc6XfJ54=; b=vr7FzTK3BDyIMVDHQhfZsXHJsJkfqp5rAFRE/zefTipLlW5H72IEydYiTXENVrBuui Kjrj0ebaq+OUAyb9pOjunUOHcQ91bP3Tx/pPfOonCVlbT/pnzjbxXfDOuDMue9f3Sat0 RPUGXkCv4WobJKNgo+ihJeKmiTgNOIBI2UEbB6KU8JfdPPDVfbnO3YEii8zlCRlwciCZ elxftIaLsepnU/IbXI9dvtIXZQZHUz39jvOh7TBhe7DEMDnYCfICL+JtVpkl61BZXalK VW2L3u/ZfcdVXZwZ7A9xmnfDKZCj1LCLKTJM9fjG52G2pP8KQYwfwfZt4ySzkE03w1gq IR3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=UGSLdlYJxr2Is1FgLjfVIxpXo24NFut3vGLdc6XfJ54=; b=QHpwhbJolNsjhI9kB2Jw3g/NPFuCavtzwOkG72alWoVt+PPppT4uKupaI/8ULw1mTy QUoYZ3zEhgUiMDfXD6KOz4v7bYb4jq1R34ZFnW5cuvprA4+f5fLdtDRSvafpRumjXNhN SgoiLzLnzbuXJJ1L9kYwW4egPUmSnWFmfNICc2MKHcaSzLlUohPM6s2/9XCHe19eY860 COnFraYqu57jv+VV673+jduUN/EmNj23iKi0Ubzj9yS21kDDiGgCt0EvKJNe6moHggCY D2t6CxofxvdEH8TBGgdyjConLas1C/7hbhFU/vYmtETdjCZiRbenY2l8P4zE/a7Xhe/s sRYw== X-Gm-Message-State: AOAM531w3dLwAZW+1ReWsnowwnrZuPRVIN2H02EmeNvBW+jxc3E7upRK Qt7sU15IYqY9j01w99rZPxEmVQ== X-Google-Smtp-Source: ABdhPJznUo72JokJjqhw/LMZEJrjVOkgKKVwMVSNegbpSTKTy3YfAGbOPYWmjGsQFW54VlBCYSt7Mw== X-Received: by 2002:adf:cf01:: with SMTP id o1mr28436465wrj.421.1600277700913; Wed, 16 Sep 2020 10:35:00 -0700 (PDT) Received: from localhost ([2a01:4b00:8523:2d03:e49d:f6be:d31b:ad3c]) by smtp.gmail.com with ESMTPSA id d23sm6221350wmb.6.2020.09.16.10.34.59 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 16 Sep 2020 10:34:59 -0700 (PDT) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v3 08/11] kvm: arm64: Create separate instances of kvm_host_data for VHE/nVHE Date: Wed, 16 Sep 2020 18:34:36 +0100 Message-Id: <20200916173439.32265-9-dbrazdil@google.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200916173439.32265-1-dbrazdil@google.com> References: <20200916173439.32265-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200916_133502_201518_3582DCEF X-CRM114-Status: GOOD ( 17.22 ) X-Spam-Score: -15.7 (---------------) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (-15.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2a00:1450:4864:20:0:0:0:444 listed in] [list.dnswl.org] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list -7.5 USER_IN_DEF_SPF_WL From: address is in the default SPF white-list -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.5 ENV_AND_HDR_SPF_MATCH Env and Hdr From used in default SPF WL Match -0.0 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kernel-team@android.com, Suzuki K Poulose , Catalin Marinas , linux-kernel@vger.kernel.org, James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org Host CPU context is stored in a global per-cpu variable `kvm_host_data`. In preparation for introducing independent per-CPU region for nVHE hyp, create two separate instances of `kvm_host_data`, one for VHE and one for nVHE. Signed-off-by: David Brazdil Acked-by: Will Deacon --- arch/arm64/include/asm/kvm_host.h | 2 +- arch/arm64/kernel/image-vars.h | 1 - arch/arm64/kvm/arm.c | 5 ++--- arch/arm64/kvm/hyp/nvhe/switch.c | 3 +++ arch/arm64/kvm/hyp/vhe/switch.c | 3 +++ arch/arm64/kvm/pmu.c | 8 ++++---- 6 files changed, 13 insertions(+), 9 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 905c2b87e05a..5d8c63f5e97e 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -565,7 +565,7 @@ void kvm_set_sei_esr(struct kvm_vcpu *vcpu, u64 syndrome); struct kvm_vcpu *kvm_mpidr_to_vcpu(struct kvm *kvm, unsigned long mpidr); -DECLARE_PER_CPU(kvm_host_data_t, kvm_host_data); +DECLARE_KVM_HYP_PER_CPU(kvm_host_data_t, kvm_host_data); static inline void kvm_init_host_cpu_context(struct kvm_cpu_context *cpu_ctxt) { diff --git a/arch/arm64/kernel/image-vars.h b/arch/arm64/kernel/image-vars.h index 59d12a0b4622..80da861b8180 100644 --- a/arch/arm64/kernel/image-vars.h +++ b/arch/arm64/kernel/image-vars.h @@ -67,7 +67,6 @@ KVM_NVHE_ALIAS(kvm_patch_vector_branch); KVM_NVHE_ALIAS(kvm_update_va_mask); /* Global kernel state accessed by nVHE hyp code. */ -KVM_NVHE_ALIAS(kvm_host_data); KVM_NVHE_ALIAS(kvm_vgic_global_state); /* Kernel constant needed to compute idmap addresses. */ diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 3bdc2661d276..7af9809fa193 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -46,7 +46,6 @@ __asm__(".arch_extension virt"); #endif -DEFINE_PER_CPU(kvm_host_data_t, kvm_host_data); static DEFINE_PER_CPU(unsigned long, kvm_arm_hyp_stack_page); /* The VMID used in the VTTBR */ @@ -1308,7 +1307,7 @@ static void cpu_hyp_reset(void) static void cpu_hyp_reinit(void) { - kvm_init_host_cpu_context(&this_cpu_ptr(&kvm_host_data)->host_ctxt); + kvm_init_host_cpu_context(&this_cpu_ptr_hyp(kvm_host_data)->host_ctxt); cpu_hyp_reset(); @@ -1543,7 +1542,7 @@ static int init_hyp_mode(void) for_each_possible_cpu(cpu) { kvm_host_data_t *cpu_data; - cpu_data = per_cpu_ptr(&kvm_host_data, cpu); + cpu_data = per_cpu_ptr_hyp(kvm_host_data, cpu); err = create_hyp_mappings(cpu_data, cpu_data + 1, PAGE_HYP); if (err) { diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/switch.c index 4662df6330d7..a7e9b03bd9d1 100644 --- a/arch/arm64/kvm/hyp/nvhe/switch.c +++ b/arch/arm64/kvm/hyp/nvhe/switch.c @@ -30,6 +30,9 @@ /* Non-VHE copy of the kernel symbol. */ DEFINE_PER_CPU_READ_MOSTLY(u64, arm64_ssbd_callback_required); +/* Non-VHE instance of kvm_host_data. */ +DEFINE_PER_CPU(kvm_host_data_t, kvm_host_data); + static void __activate_traps(struct kvm_vcpu *vcpu) { u64 val; diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c index 575e8054f116..0949fc97bf03 100644 --- a/arch/arm64/kvm/hyp/vhe/switch.c +++ b/arch/arm64/kvm/hyp/vhe/switch.c @@ -28,6 +28,9 @@ const char __hyp_panic_string[] = "HYP panic:\nPS:%08llx PC:%016llx ESR:%08llx\nFAR:%016llx HPFAR:%016llx PAR:%016llx\nVCPU:%p\n"; +/* VHE instance of kvm_host_data. */ +DEFINE_PER_CPU(kvm_host_data_t, kvm_host_data); + static void __activate_traps(struct kvm_vcpu *vcpu) { u64 val; diff --git a/arch/arm64/kvm/pmu.c b/arch/arm64/kvm/pmu.c index 3c224162b3dd..6d80ffe1ebfc 100644 --- a/arch/arm64/kvm/pmu.c +++ b/arch/arm64/kvm/pmu.c @@ -31,7 +31,7 @@ static bool kvm_pmu_switch_needed(struct perf_event_attr *attr) */ void kvm_set_pmu_events(u32 set, struct perf_event_attr *attr) { - struct kvm_host_data *ctx = this_cpu_ptr(&kvm_host_data); + struct kvm_host_data *ctx = this_cpu_ptr_hyp(kvm_host_data); if (!kvm_pmu_switch_needed(attr)) return; @@ -47,7 +47,7 @@ void kvm_set_pmu_events(u32 set, struct perf_event_attr *attr) */ void kvm_clr_pmu_events(u32 clr) { - struct kvm_host_data *ctx = this_cpu_ptr(&kvm_host_data); + struct kvm_host_data *ctx = this_cpu_ptr_hyp(kvm_host_data); ctx->pmu_events.events_host &= ~clr; ctx->pmu_events.events_guest &= ~clr; @@ -173,7 +173,7 @@ void kvm_vcpu_pmu_restore_guest(struct kvm_vcpu *vcpu) return; preempt_disable(); - host = this_cpu_ptr(&kvm_host_data); + host = this_cpu_ptr_hyp(kvm_host_data); events_guest = host->pmu_events.events_guest; events_host = host->pmu_events.events_host; @@ -193,7 +193,7 @@ void kvm_vcpu_pmu_restore_host(struct kvm_vcpu *vcpu) if (!has_vhe()) return; - host = this_cpu_ptr(&kvm_host_data); + host = this_cpu_ptr_hyp(kvm_host_data); events_guest = host->pmu_events.events_guest; events_host = host->pmu_events.events_host; From patchwork Wed Sep 16 17:34:37 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11780269 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9215A14F6 for ; Wed, 16 Sep 2020 17:36:26 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5EE4520708 for ; Wed, 16 Sep 2020 17:36:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="uZOtRVpo"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="HNZo35gC" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5EE4520708 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=RYlZBH8XnSHhHgTuXGlV4tfM+xwnaJbV7kxUJcxqCkk=; b=uZOtRVpoAXdlYnAc1yiBQ4MYg CdJM+mXuI25+ysGlRa6pL8N7faQPZu15rZIrO0jgWarJ7Sl6D8dKfShgKHdZdPbFWdvORA+W0SaQG 6rf0v0cITotLlc/VOXG5l9qTQmap8DqQff6RwxIPj+BFTzEtppwMphdkjCew4zROnbrW8kFQGOr7U cIdrJ7LstSo6ALtHvo8VSAh6phz6vGbBLXVRAgIxpyA9RqLYQjb9gP0JKJkRDuESv5YRg4mUjCsFF wZJTmmGKE2e0is95oj5UmUqjCwqJJos8cwRF68iZhFroCMGzwqgz0rVFgm7+/waKZVdYhB4rGALf3 1lmtZKwaw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kIbLn-0001K0-Vd; Wed, 16 Sep 2020 17:36:12 +0000 Received: from mail-wm1-x343.google.com ([2a00:1450:4864:20::343]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kIbKi-0000vr-3m for linux-arm-kernel@lists.infradead.org; Wed, 16 Sep 2020 17:35:05 +0000 Received: by mail-wm1-x343.google.com with SMTP id a9so3902062wmm.2 for ; Wed, 16 Sep 2020 10:35:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3ZouLpoLHZGhnRn3IE3lAdYTtqQtOaDPWcW0BXT72MM=; b=HNZo35gC+J7olQM/J4RAmLlZS6QxVf8SG/2BDEI9P1bohZ+P5Q96wTZVwc7II0n9BL cgE7q3s2HuB8SyeYAwP76jfvS9ilgnaSdWf3BN5CpaP3PpCWpHo7OyCzXoLczzUjYIh+ TAvf72HVFbYYjUfwXRs9KQWyCZzjv5SmQjRTwHXgeAoUK3/2zXTwbrvIkPx4Um1P5RvS TU1N+Ppw0DKArVS8ynQ4bf+jV171VTr1y4ZU53me1jkwW1cniiMaCioHU1u/AMChZdp6 1BnnWaIQvyESEBhMsuSyugrLCEkTdp5crIYFCA9FneqGow+l1YuEeHkCEHhBbzEX3vON E0iQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3ZouLpoLHZGhnRn3IE3lAdYTtqQtOaDPWcW0BXT72MM=; b=eLTFncN2Sw+AMkTwhKpSwv8upD4RgpXGBtBN04m07Q4D2hyXuFZTX9DaEU54x2hEgo j2Xex7dBhLn8YL+sVjY5cfob++DVoAIoZpWEcbcnVypnw1hRzoO0d1Rq4Q/l8wpB+cTl fOolesoIf9K+5tPOaak/2wHr7nkno4iQrVKqeIjBRPMKOwxgLAVKF43Mdd6BXJLRC+nw oSRsL1h4kEyWQV7ADn2iQ+FZ2Zwvsg7fyNCUPMUxBgvV1QphGsRxOZ/OYSkWoraVPoH8 N8943Zv9SOiA5vvnRw4KgrMQIB7Ryrk38aAyIUFk6TARO0FnnAqDJuskrpaNHvhERZHf X50A== X-Gm-Message-State: AOAM533v5IxGKHp0vUNJrP6UUIVJOrEsl8MfVZNtyYBApk1XFta1ZUgK Pt1c84n2cDQ8x+e4Ut6+J52raEAfSi7zFU31 X-Google-Smtp-Source: ABdhPJyjAeHVuIWwfCgiltujOqCdZJciMMvPMbdeAr38UjlrUERhWuN+mDjxXhXo26gYVThTzJRNbg== X-Received: by 2002:a1c:2e08:: with SMTP id u8mr6233481wmu.156.1600277702955; Wed, 16 Sep 2020 10:35:02 -0700 (PDT) Received: from localhost ([2a01:4b00:8523:2d03:e49d:f6be:d31b:ad3c]) by smtp.gmail.com with ESMTPSA id p1sm8808360wma.0.2020.09.16.10.35.01 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 16 Sep 2020 10:35:01 -0700 (PDT) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v3 09/11] kvm: arm64: Mark hyp stack pages reserved Date: Wed, 16 Sep 2020 18:34:37 +0100 Message-Id: <20200916173439.32265-10-dbrazdil@google.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200916173439.32265-1-dbrazdil@google.com> References: <20200916173439.32265-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200916_133504_204589_A7C06652 X-CRM114-Status: GOOD ( 18.67 ) X-Spam-Score: -15.7 (---------------) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (-15.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2a00:1450:4864:20:0:0:0:343 listed in] [list.dnswl.org] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list -7.5 USER_IN_DEF_SPF_WL From: address is in the default SPF white-list -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.5 ENV_AND_HDR_SPF_MATCH Env and Hdr From used in default SPF WL Match -0.0 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kernel-team@android.com, Suzuki K Poulose , Catalin Marinas , linux-kernel@vger.kernel.org, James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org In preparation for unmapping hyp pages from host stage-2, allocate/free hyp stack using new helpers which automatically mark the pages reserved. Signed-off-by: David Brazdil --- arch/arm64/kvm/arm.c | 49 ++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 47 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 7af9809fa193..58d7d614519b 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1456,13 +1456,58 @@ static int init_subsystems(void) return err; } +/* + * Alloc pages and mark them reserved so the kernel never tries to + * take them away from the hypervisor. + */ +static unsigned long alloc_hyp_pages(gfp_t flags, unsigned int order) +{ + struct page *page; + unsigned long i; + + page = alloc_pages(flags, order); + if (!page) + return 0; + + for (i = 0; i < (1ul << order); ++i) + mark_page_reserved(page + i); + + return (unsigned long)page_address(page); +} + +static unsigned long alloc_hyp_page(gfp_t flags) +{ + return alloc_hyp_pages(flags, 0); +} + +/* + * Free pages which were previously marked reserved for the hypervisor. + */ +static void free_hyp_pages(unsigned long addr, unsigned int order) +{ + unsigned long i; + struct page *page; + + if (!addr) + return; + + page = virt_to_page(addr); + for (i = 0; i < (1ul << order); ++i) + free_reserved_page(page + i); +} + +static void free_hyp_page(unsigned long addr) +{ + return free_hyp_pages(addr, 0); +} + static void teardown_hyp_mode(void) { int cpu; free_hyp_pgds(); for_each_possible_cpu(cpu) - free_page(per_cpu(kvm_arm_hyp_stack_page, cpu)); + free_hyp_page(per_cpu(kvm_arm_hyp_stack_page, cpu)); } /** @@ -1486,7 +1531,7 @@ static int init_hyp_mode(void) for_each_possible_cpu(cpu) { unsigned long stack_page; - stack_page = __get_free_page(GFP_KERNEL); + stack_page = alloc_hyp_page(GFP_KERNEL); if (!stack_page) { err = -ENOMEM; goto out_err; From patchwork Wed Sep 16 17:34:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11780287 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3C66D6CA for ; Wed, 16 Sep 2020 17:38:16 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id EC38820708 for ; Wed, 16 Sep 2020 17:38:15 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="AWOchZmX"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="NIFkj+RZ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EC38820708 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=YzxipWq1nd8AqudAcf3QIyt649NQjiwrfGDJQi9b28o=; b=AWOchZmXFjvXjID9la0bG8Oe1 CkSpFheMy0LyYrGK+i3nVLSgUfq1JKXiLnbL2b+rL7JIU2lgibKlILPgqxJ3D7dWDGBYeMddJuVcT aiwkZeIW1GrnsBQNTMNWrtfw5SHnDku3ElEJv1EjEDho+KCBL1PY/LYcGA72wIO8bzRi/ToZvX3qG snOQ/5tWAmYzi2QcGeuxytuFz/f+234EAJm5tgAhGEGg6O+083avopt1FMxI+YjSf+hs8r4bDhDmz zqNqzYjzZXl4HjAIyTe5iNMUtUEm3yKDJIbROle1HM6ReaT91fSDjQk7Ocv3ypL8/EheUZV4gmRId LCr60PCgQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kIbLs-0001Lu-Kp; Wed, 16 Sep 2020 17:36:17 +0000 Received: from mail-wr1-x443.google.com ([2a00:1450:4864:20::443]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kIbKk-0000wd-1i for linux-arm-kernel@lists.infradead.org; Wed, 16 Sep 2020 17:35:07 +0000 Received: by mail-wr1-x443.google.com with SMTP id x14so7731197wrl.12 for ; Wed, 16 Sep 2020 10:35:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Q9UVQQ6DTZHcZFVRtOaPO3UPD1d1Bnoz5yW15JNPfiI=; b=NIFkj+RZcBLfM9uT1KnlW/OhjhMvYayd0/LMbIbIjjbzTiq3iRjUJZDL65n21p4Pvb gmXNhHqPQ+M971nXvZ6jXGdzwFYXNz7M+G2mgbNYqvyhhDqq72Gj8i3nbHkUQ20Qesfx NbC9V/DmTgepxr0SC62j5IN6XHQoNR1C1ww7GnkuwRPPid9feWdOWJmPnhD3wi4hd7rT RBlCtU8A32m6x3kyS9OkzWCpqeMyWB+vGXZ0Qyc6Gx7NYXQpjaGdcLI2lYKAs0D8Bq6Y mqMSaLEYxfWh5BJLQYS7Cv/LkEpqaUCoWZ2hQJskzYleDUAZ+q6UUan4U0WLvKjGf1ND qj9A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Q9UVQQ6DTZHcZFVRtOaPO3UPD1d1Bnoz5yW15JNPfiI=; b=Fh6xo81Cvh7Vr/laW6yvNfR/D6/+kJE6DHISjInx8aY278JTDfBYVcYJjiOE/rxEQq kUtRWeDiYgIXDu0t6pyZ84DmwM9H3xIptRfng3FElYHbp9ZJ6hrwszD1hxGIjsrdMoGB YnsUD0VpLDfvT0CQIlPdQTMC5MqntUclcba5rUFNM/TjNrUBA5DLDpTkrH3UMWKBV+vO NGPPHl1XEmV/CvDgTfxGpPCxkREDFZWDvAPmjSXq+DYFVF1p14hz5w9cnJRmgxMksYs2 D/emQqgIIDR1vyBVmzZptCQzDGARPGcwtXdKt3TIHsjsT7ZscQa3QaNBo4xWn5aOh3vG gD1g== X-Gm-Message-State: AOAM531tBwPyYVBGYjWHVM3MsA5G5wRZnNNo0E7a826ypUbrv/jZCP92 poJrDK6am4FSvhSQurniQn9hRA== X-Google-Smtp-Source: ABdhPJwG1ZOQDzJ0OTA1ogwbtAlSeqzm/T0AT+4qFHXsUmYbr+tGahu6Qso1FQ/U45prv0lKr8VFHA== X-Received: by 2002:a5d:6407:: with SMTP id z7mr27645027wru.65.1600277704925; Wed, 16 Sep 2020 10:35:04 -0700 (PDT) Received: from localhost ([2a01:4b00:8523:2d03:e49d:f6be:d31b:ad3c]) by smtp.gmail.com with ESMTPSA id 8sm35215401wrl.7.2020.09.16.10.35.03 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 16 Sep 2020 10:35:03 -0700 (PDT) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v3 10/11] kvm: arm64: Set up hyp percpu data for nVHE Date: Wed, 16 Sep 2020 18:34:38 +0100 Message-Id: <20200916173439.32265-11-dbrazdil@google.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200916173439.32265-1-dbrazdil@google.com> References: <20200916173439.32265-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200916_133506_151683_DF5352B3 X-CRM114-Status: GOOD ( 24.36 ) X-Spam-Score: -15.7 (---------------) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (-15.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2a00:1450:4864:20:0:0:0:443 listed in] [list.dnswl.org] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list -7.5 USER_IN_DEF_SPF_WL From: address is in the default SPF white-list -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.5 ENV_AND_HDR_SPF_MATCH Env and Hdr From used in default SPF WL Match -0.0 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kernel-team@android.com, Suzuki K Poulose , Catalin Marinas , linux-kernel@vger.kernel.org, James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org Add hyp percpu section to linker script and rename the corresponding ELF sections of hyp/nvhe object files. This moves all nVHE-specific percpu variables to the new hyp percpu section. Allocate sufficient amount of memory for all percpu hyp regions at global KVM init time and create corresponding hyp mappings. The base addresses of hyp percpu regions are kept in a dynamically allocated array in the kernel. Add NULL checks in PMU event-reset code as it may run before KVM memory is initialized. Signed-off-by: David Brazdil --- arch/arm64/include/asm/kvm_asm.h | 19 +++++++++-- arch/arm64/include/asm/sections.h | 1 + arch/arm64/kernel/vmlinux.lds.S | 9 +++++ arch/arm64/kvm/arm.c | 56 +++++++++++++++++++++++++++++-- arch/arm64/kvm/hyp/nvhe/hyp.lds.S | 6 ++++ arch/arm64/kvm/pmu.c | 5 ++- 6 files changed, 90 insertions(+), 6 deletions(-) diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h index abc03f386b40..e0e1e404f6eb 100644 --- a/arch/arm64/include/asm/kvm_asm.h +++ b/arch/arm64/include/asm/kvm_asm.h @@ -66,8 +66,23 @@ #define CHOOSE_VHE_SYM(sym) sym #define CHOOSE_NVHE_SYM(sym) kvm_nvhe_sym(sym) -#define this_cpu_ptr_nvhe(sym) this_cpu_ptr(&kvm_nvhe_sym(sym)) -#define per_cpu_ptr_nvhe(sym, cpu) per_cpu_ptr(&kvm_nvhe_sym(sym), cpu) +/* Array of percpu base addresses. Length of the array is nr_cpu_ids. */ +extern unsigned long *kvm_arm_hyp_percpu_base; + +/* + * Compute pointer to a symbol defined in nVHE percpu region. + * Returns NULL if percpu memory has not been allocated yet. + */ +#define this_cpu_ptr_nvhe(sym) per_cpu_ptr_nvhe(sym, smp_processor_id()) +#define per_cpu_ptr_nvhe(sym, cpu) \ + ({ \ + unsigned long base, off; \ + base = kvm_arm_hyp_percpu_base \ + ? kvm_arm_hyp_percpu_base[cpu] : 0; \ + off = (unsigned long)&kvm_nvhe_sym(sym) - \ + (unsigned long)&kvm_nvhe_sym(__per_cpu_start); \ + base ? (typeof(kvm_nvhe_sym(sym))*)(base + off) : NULL; \ + }) #ifndef __KVM_NVHE_HYPERVISOR__ /* diff --git a/arch/arm64/include/asm/sections.h b/arch/arm64/include/asm/sections.h index 3994169985ef..5062553a6847 100644 --- a/arch/arm64/include/asm/sections.h +++ b/arch/arm64/include/asm/sections.h @@ -18,5 +18,6 @@ extern char __exittext_begin[], __exittext_end[]; extern char __irqentry_text_start[], __irqentry_text_end[]; extern char __mmuoff_data_start[], __mmuoff_data_end[]; extern char __entry_tramp_text_start[], __entry_tramp_text_end[]; +extern char __kvm_nvhe___per_cpu_start[], __kvm_nvhe___per_cpu_end[]; #endif /* __ASM_SECTIONS_H */ diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index 0d543570e811..d52e6b5dbfd3 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -9,6 +9,7 @@ #include #include +#include #include #include #include @@ -27,8 +28,15 @@ jiffies = jiffies_64; __start___kvm_ex_table = .; \ *(__kvm_ex_table) \ __stop___kvm_ex_table = .; + +#define HYPERVISOR_PERCPU_SECTION \ + . = ALIGN(PAGE_SIZE); \ + HYP_SECTION_NAME(.data..percpu) : { \ + *(HYP_SECTION_NAME(.data..percpu)) \ + } #else /* CONFIG_KVM */ #define HYPERVISOR_EXTABLE +#define HYPERVISOR_PERCPU_SECTION #endif #define HYPERVISOR_TEXT \ @@ -194,6 +202,7 @@ SECTIONS } PERCPU_SECTION(L1_CACHE_BYTES) + HYPERVISOR_PERCPU_SECTION .rela.dyn : ALIGN(8) { *(.rela .rela*) diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 58d7d614519b..8293319a32e7 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -47,6 +47,7 @@ __asm__(".arch_extension virt"); #endif static DEFINE_PER_CPU(unsigned long, kvm_arm_hyp_stack_page); +unsigned long *kvm_arm_hyp_percpu_base; /* The VMID used in the VTTBR */ static atomic64_t kvm_vmid_gen = ATOMIC64_INIT(1); @@ -1258,6 +1259,15 @@ long kvm_arch_vm_ioctl(struct file *filp, } } +#define kvm_hyp_percpu_base(cpu) ((unsigned long)per_cpu_ptr_nvhe(__per_cpu_start, cpu)) +#define kvm_hyp_percpu_array_size (nr_cpu_ids * sizeof(*kvm_arm_hyp_percpu_base)) +#define kvm_hyp_percpu_array_order (get_order(kvm_hyp_percpu_array_size)) +#define kvm_hyp_percpu_start (CHOOSE_NVHE_SYM(__per_cpu_start)) +#define kvm_hyp_percpu_end (CHOOSE_NVHE_SYM(__per_cpu_end)) +#define kvm_hyp_percpu_size ((unsigned long)kvm_hyp_percpu_end - \ + (unsigned long)kvm_hyp_percpu_start) +#define kvm_hyp_percpu_order (kvm_hyp_percpu_size ? get_order(kvm_hyp_percpu_size) : 0) + static void cpu_init_hyp_mode(void) { phys_addr_t pgd_ptr; @@ -1273,8 +1283,8 @@ static void cpu_init_hyp_mode(void) * kernel's mapping to the linear mapping, and store it in tpidr_el2 * so that we can use adr_l to access per-cpu variables in EL2. */ - tpidr_el2 = ((unsigned long)this_cpu_ptr(&kvm_host_data) - - (unsigned long)kvm_ksym_ref(&kvm_host_data)); + tpidr_el2 = (unsigned long)this_cpu_ptr_nvhe(__per_cpu_start) - + (unsigned long)kvm_ksym_ref(kvm_hyp_percpu_start); pgd_ptr = kvm_mmu_get_httbr(); hyp_stack_ptr = __this_cpu_read(kvm_arm_hyp_stack_page) + PAGE_SIZE; @@ -1506,8 +1516,11 @@ static void teardown_hyp_mode(void) int cpu; free_hyp_pgds(); - for_each_possible_cpu(cpu) + for_each_possible_cpu(cpu) { free_hyp_page(per_cpu(kvm_arm_hyp_stack_page, cpu)); + free_hyp_pages(kvm_hyp_percpu_base(cpu), kvm_hyp_percpu_order); + } + free_hyp_pages((unsigned long)kvm_arm_hyp_percpu_base, kvm_hyp_percpu_array_order); } /** @@ -1540,6 +1553,28 @@ static int init_hyp_mode(void) per_cpu(kvm_arm_hyp_stack_page, cpu) = stack_page; } + /* + * Allocate and initialize pages for Hypervisor-mode percpu regions. + */ + kvm_arm_hyp_percpu_base = (unsigned long *)alloc_hyp_pages( + GFP_KERNEL | __GFP_ZERO, kvm_hyp_percpu_array_order); + if (!kvm_arm_hyp_percpu_base) { + err = -ENOMEM; + goto out_err; + } + for_each_possible_cpu(cpu) { + unsigned long percpu_base; + + percpu_base = alloc_hyp_pages(GFP_KERNEL, kvm_hyp_percpu_order); + if (!percpu_base) { + err = -ENOMEM; + goto out_err; + } + + memcpy((void *)percpu_base, kvm_hyp_percpu_start, kvm_hyp_percpu_size); + kvm_arm_hyp_percpu_base[cpu] = percpu_base; + } + /* * Map the Hyp-code called directly from the host */ @@ -1584,6 +1619,21 @@ static int init_hyp_mode(void) } } + /* + * Map Hyp percpu pages + */ + for_each_possible_cpu(cpu) { + char *percpu_begin = (char *)kvm_arm_hyp_percpu_base[cpu]; + char *percpu_end = percpu_begin + PAGE_ALIGN(kvm_hyp_percpu_size); + + err = create_hyp_mappings(percpu_begin, percpu_end, PAGE_HYP); + + if (err) { + kvm_err("Cannot map hyp percpu region\n"); + goto out_err; + } + } + for_each_possible_cpu(cpu) { kvm_host_data_t *cpu_data; diff --git a/arch/arm64/kvm/hyp/nvhe/hyp.lds.S b/arch/arm64/kvm/hyp/nvhe/hyp.lds.S index 3b13d1c7cd1a..bb2d986ff696 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp.lds.S +++ b/arch/arm64/kvm/hyp/nvhe/hyp.lds.S @@ -7,7 +7,13 @@ */ #include +#include +#include +#include SECTIONS { HYP_SECTION(.text) + HYP_SECTION_NAME(.data..percpu) : { + PERCPU_INPUT(L1_CACHE_BYTES) + } } diff --git a/arch/arm64/kvm/pmu.c b/arch/arm64/kvm/pmu.c index 6d80ffe1ebfc..cd653091f213 100644 --- a/arch/arm64/kvm/pmu.c +++ b/arch/arm64/kvm/pmu.c @@ -33,7 +33,7 @@ void kvm_set_pmu_events(u32 set, struct perf_event_attr *attr) { struct kvm_host_data *ctx = this_cpu_ptr_hyp(kvm_host_data); - if (!kvm_pmu_switch_needed(attr)) + if (!ctx || !kvm_pmu_switch_needed(attr)) return; if (!attr->exclude_host) @@ -49,6 +49,9 @@ void kvm_clr_pmu_events(u32 clr) { struct kvm_host_data *ctx = this_cpu_ptr_hyp(kvm_host_data); + if (!ctx) + return; + ctx->pmu_events.events_host &= ~clr; ctx->pmu_events.events_guest &= ~clr; } From patchwork Wed Sep 16 17:34:39 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11780275 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DF4D914F6 for ; Wed, 16 Sep 2020 17:36:46 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8B7AA20672 for ; Wed, 16 Sep 2020 17:36:46 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="iCNZ1BAL"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="XBpFPnVn" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8B7AA20672 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=Z9/G6R23NQovlivhdrFw950UwDU01vAMxJfhNyBF0fY=; b=iCNZ1BALxhbZKC9cwfnAyCfA/ wQWtJvJ/giz864nLfI2Xm4gm1o19waAGWRlgpXh8NA5vDlvDYOXseBnl9OpgJIzx4Hb3YQwB6p4Sp kJu7U72i+h9eHZiUmKTaSfM8KeWbgfrwy3pbhDAv68YRgBguSPEiCZzWLubw83j1YX5yK3OIqlXBl 3F0DvzZrdgQgIJdJx5JEcDwUq/ZDD3rIzOKiOyPZNYj9KzVhxHXxrwDXEFEYKzGFFvVF5Z3GVTzgV SvqJ2mgaVd1zFStnVoT1w/ps5lUp2fJpM+Y/t4e+K3i3PlRuXdFPwcKtdEoBdovzMqOEirUBtPffE 38CjK6StA==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kIbM5-0001RM-8c; Wed, 16 Sep 2020 17:36:29 +0000 Received: from mail-wr1-x444.google.com ([2a00:1450:4864:20::444]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kIbKm-0000xO-8M for linux-arm-kernel@lists.infradead.org; Wed, 16 Sep 2020 17:35:09 +0000 Received: by mail-wr1-x444.google.com with SMTP id k15so7761777wrn.10 for ; Wed, 16 Sep 2020 10:35:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=EdexJTeSJ1yzSq2NB9sPeJusVOkXFFsSU8w5tX+wAoY=; b=XBpFPnVnEXotZ9h7Vi96uiegtXa9f6RsxKZhjfRsj/bVOPdAq6fxcKcQEAWKY2yP/+ j3PWyy1esATPnLP3uevSBbvUQKYvsCxNVoLaiGC/5iqudT2qc6itwteJuDQZN39vdob3 QaDJj5Fg7ELcwgZceR5AuZcuIyWWmWGJn9h8tuNgYqFXJvF9GfV92xXB911rmrSm3IJz DEVO8XTVwMYWDmqJ1lvnk1kAgNDeDDzO49zUONaiMZF3cfbTO5K+56OTvnR1AU7UXbMn LvmycZKfMnvi9K9gtcHdirkVA7EoQe+EhLFDqe8HnRz+8V1xqmA+BY1XfOsJpaZD8LH6 UDzA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=EdexJTeSJ1yzSq2NB9sPeJusVOkXFFsSU8w5tX+wAoY=; b=SHjtmGHGjwkcCdCnCYOVDBT6Rn7+oplaeFQHe0CYEDmqGO70CpWdBEarH6JeUoeeIV 75VzoUNTNElBKaILSvNz4K/ySmvyJPIcUD3Rc1FREqR03jGMiDH7GLn5ow2kb/tNqmXW a5WDjRVfnzSAnXyA7pt9B2QDDAVbqn1bsE2EBOxgntScXeRqtzrNgdR+BYwjtJ9AmiCH LzDrObtSc+290YVVLJYTRRytQRwyMccG3KNf/bOYR8RdV6F8YV1yOlOshUhOKRJSXWHS hPnFZnAksEvpxBBPqDYtmKdXdWul6/SAyICE1+DGy6ZIqeyULQLeoR2FansPbUDoxZdN 4jLg== X-Gm-Message-State: AOAM532DU3JJ41/5rGQ3HFZF8xTiti1LIlpsD/fBQ3mCsrL13RewVXGm +MQ4a9vpL5LkryuNUXqig++zAA== X-Google-Smtp-Source: ABdhPJxiNME7o6IoYWEvolPBp1wwKcDxQ2PJwQacH0w0glnZjPPICPGldNZPzVdYDrWU7SS+rM/Ucw== X-Received: by 2002:adf:92c2:: with SMTP id 60mr29983169wrn.220.1600277707032; Wed, 16 Sep 2020 10:35:07 -0700 (PDT) Received: from localhost ([2a01:4b00:8523:2d03:e49d:f6be:d31b:ad3c]) by smtp.gmail.com with ESMTPSA id j10sm35693576wrn.2.2020.09.16.10.35.05 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 16 Sep 2020 10:35:05 -0700 (PDT) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH v3 11/11] kvm: arm64: Remove unnecessary hyp mappings Date: Wed, 16 Sep 2020 18:34:39 +0100 Message-Id: <20200916173439.32265-12-dbrazdil@google.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200916173439.32265-1-dbrazdil@google.com> References: <20200916173439.32265-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200916_133508_525450_99D1F4E3 X-CRM114-Status: GOOD ( 16.36 ) X-Spam-Score: -15.7 (---------------) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (-15.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2a00:1450:4864:20:0:0:0:444 listed in] [list.dnswl.org] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list -7.5 USER_IN_DEF_SPF_WL From: address is in the default SPF white-list -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.5 ENV_AND_HDR_SPF_MATCH Env and Hdr From used in default SPF WL Match -0.0 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kernel-team@android.com, Suzuki K Poulose , Catalin Marinas , linux-kernel@vger.kernel.org, James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , Tejun Heo , Dennis Zhou , Christoph Lameter , David Brazdil , Will Deacon , Julien Thierry , Andrew Scull Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org With all nVHE per-CPU variables being part of the hyp per-CPU region, mapping them individual is not necessary any longer. They are mapped to hyp as part of the overall per-CPU region. Acked-by: Andrew Scull Signed-off-by: David Brazdil --- arch/arm64/include/asm/kvm_mmu.h | 19 +++++-------------- arch/arm64/kvm/arm.c | 17 +---------------- 2 files changed, 6 insertions(+), 30 deletions(-) diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h index 9db93da35606..7d1581bc8b97 100644 --- a/arch/arm64/include/asm/kvm_mmu.h +++ b/arch/arm64/include/asm/kvm_mmu.h @@ -531,28 +531,19 @@ static inline int kvm_map_vectors(void) DECLARE_PER_CPU_READ_MOSTLY(u64, arm64_ssbd_callback_required); DECLARE_KVM_NVHE_PER_CPU(u64, arm64_ssbd_callback_required); -static inline int hyp_init_aux_data(void) +static inline void hyp_init_aux_data(void) { - int cpu, err; + int cpu; + /* Copy arm64_ssbd_callback_required values from kernel to hyp. */ for_each_possible_cpu(cpu) { - u64 *ptr; + u64 *ptr = per_cpu_ptr_nvhe(arm64_ssbd_callback_required, cpu); - ptr = per_cpu_ptr_nvhe(arm64_ssbd_callback_required, cpu); - err = create_hyp_mappings(ptr, ptr + 1, PAGE_HYP); - if (err) - return err; - - /* Copy value from kernel to hyp. */ *ptr = per_cpu(arm64_ssbd_callback_required, cpu); } - return 0; } #else -static inline int hyp_init_aux_data(void) -{ - return 0; -} +static inline void hyp_init_aux_data(void) {} #endif #define kvm_phys_to_vttbr(addr) phys_to_ttbr(addr) diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 8293319a32e7..4483b6aca5d7 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1634,22 +1634,7 @@ static int init_hyp_mode(void) } } - for_each_possible_cpu(cpu) { - kvm_host_data_t *cpu_data; - - cpu_data = per_cpu_ptr_hyp(kvm_host_data, cpu); - err = create_hyp_mappings(cpu_data, cpu_data + 1, PAGE_HYP); - - if (err) { - kvm_err("Cannot map host CPU state: %d\n", err); - goto out_err; - } - } - - err = hyp_init_aux_data(); - if (err) - kvm_err("Cannot map host auxiliary data: %d\n", err); - + hyp_init_aux_data(); return 0; out_err: