From patchwork Wed Jul 22 16:44:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11678933 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C6DB913A4 for ; Wed, 22 Jul 2020 16:47:03 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9C980206C1 for ; Wed, 22 Jul 2020 16:47:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="MTvjVX1Y"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="l8EUVZgC" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9C980206C1 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=iqhFk2nGmShINsSsJ1f60Z+6o8tiFzjjf3PewO3LcEA=; b=MTvjVX1YHkWCRfnLKzX8islO4 g+tJdrbu1bGDCFD5U81AucE0xSLVhfS6uKzIOkP8CAg2s1/5MXVmHXuIMxyIEO2+Systxkdgonr65 EAle8QS/lfkmtUoC1ruthKrM/J+44Vw1c+x8452yD+UivkY223mjuKZogk2He3wdEGGZuQNXVBd0I 3ldFbBgUybER45TMWgK8plWcCoZ8U/K5MuDKN4FNZrcmPLk0kEnt4AwUOX50koemlY5ACf7rqavWt kwAeufEktghIfzRZyGRjbXG7QM/d47Bt/Jsz5r1fDqjOG8vHu0nTxl2y2uLIcgNcOol0CpPZQRfN2 CUDohZa3w==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jyHrg-0001tc-9u; Wed, 22 Jul 2020 16:45:08 +0000 Received: from mail-wm1-x342.google.com ([2a00:1450:4864:20::342]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jyHrb-0001rI-79 for linux-arm-kernel@lists.infradead.org; Wed, 22 Jul 2020 16:45:04 +0000 Received: by mail-wm1-x342.google.com with SMTP id p14so2139012wmg.1 for ; Wed, 22 Jul 2020 09:45:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=6iEzS/K7FKWZpD3ua8htStuWzJAGtakSYjlBsBpxZ/4=; b=l8EUVZgCpIRU1QL7F3sHnMulBx+ndcr+hiyAH/9TDLEfZ3e4nRnvYcMRlw8pJ1RZDD usP3/tJ/orr9olI8t2Qx+Sj/5tEnhSWSaxwGyGsDcxf+AzwpzLvAQgo1GARi5BT9Pose fSVvL492qfWaMefMcn5RNZm4KMBSxUa2R4mITHBHTG4G42tVvAyz5XC8X1EHjDPKKVTC gnO+1g9xzqhkqboMF9feb+jULlJjKVO8BTsdu5IB3Jql+dhjkP+s/fUtYQxamWeUqX3k YELPwGsw80aBmerABn4F7VHRlZuY5XWbtJqzVum5e1OoTgXIAeaBwQQFl2H3zHD4YBs6 8Exw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=6iEzS/K7FKWZpD3ua8htStuWzJAGtakSYjlBsBpxZ/4=; b=dOxD0Y+y2Xgh7YNGn6E+AITIv/nzvgkzGHRTs1PhwLPreBwPQDPTVIA0NHpleMcXzw eTTYTlbI4EzwEDW4ntQEm/NNp9XmVwOzyYKR85HbcQT2y2MbwXx/lNSiBRJufWywhYre VkggAWTpRwpJtGkjkPNCBwp3T/onvcK5H/sob+wZ9pHbI928n5teaoTgbYI/ON14Aur9 dUwxkTnf9Fcu+0cEstKpba2EV4kw+e3G4VBqHfgKvwwrXuoY2PjGHBMjKhWz4H59mt5C n1wXFHO+7zfPbl4lBIGumlT1OTxvW6c2K0n82q42Ez7I3fSOf8BiFqxMblgSya0gqgS+ xWyw== X-Gm-Message-State: AOAM530fDpv+dRWMcXt93E6IL9WZr4qUEVmxV5cw4WRHT2WrfWk7GDUZ HFXjsgi1WOApZI12IX4EPfzoE40UL26PxA== X-Google-Smtp-Source: ABdhPJyvvcrdUvKbE/hxCmWbSibjaaJ7nEtZ3HDL894GJKBpxshjcuwx5vl2E/hVxnCbvqkXRidabQ== X-Received: by 2002:a1c:a756:: with SMTP id q83mr457233wme.168.1595436300851; Wed, 22 Jul 2020 09:45:00 -0700 (PDT) Received: from localhost ([2a01:4b00:8523:2d03:b0ee:900a:e004:b9d0]) by smtp.gmail.com with ESMTPSA id u20sm265826wmm.15.2020.07.22.09.44.59 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 22 Jul 2020 09:45:00 -0700 (PDT) From: David Brazdil To: Catalin Marinas , Will Deacon , Marc Zyngier , Dennis Zhou , Tejun Heo , Christoph Lameter , Arnd Bergmann Subject: [PATCH 1/9] Macros to override naming of percpu symbols and sections Date: Wed, 22 Jul 2020 17:44:16 +0100 Message-Id: <20200722164424.42225-2-dbrazdil@google.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200722164424.42225-1-dbrazdil@google.com> References: <20200722164424.42225-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200722_124503_294133_9670C14A X-CRM114-Status: GOOD ( 14.31 ) X-Spam-Score: -15.7 (---------------) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (-15.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2a00:1450:4864:20:0:0:0:342 listed in] [list.dnswl.org] -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list -7.5 USER_IN_DEF_SPF_WL From: address is in the default SPF white-list 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.5 ENV_AND_HDR_SPF_MATCH Env and Hdr From used in default SPF WL Match -0.0 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-arch@vger.kernel.org, Suzuki K Poulose , linux-kernel@vger.kernel.org, kernel-team@google.com, James Morse , Julien Thierry , David Brazdil , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org Modify generic linker script macros to generate section/symbol names for percpu area using overridable macros. No functional changes. This will allow arm64 linker script to define a second KVM-specific percpu data section using the generic PERCPU_SECTION macro. Signed-off-by: David Brazdil --- include/asm-generic/vmlinux.lds.h | 40 +++++++++++++++++++++---------- 1 file changed, 27 insertions(+), 13 deletions(-) diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h index db600ef218d7..1bfc002ecfce 100644 --- a/include/asm-generic/vmlinux.lds.h +++ b/include/asm-generic/vmlinux.lds.h @@ -892,6 +892,20 @@ #define INIT_RAM_FS #endif +/* + * Macros to override the naming of percpu symbols and sections. + * Used by arm64 linker script to define a separate percpu area for KVM. + */ +#define PERCPU_SECTION_BASE_NAME .data..percpu + +#ifndef PERCPU_SECTION_NAME +#define PERCPU_SECTION_NAME(suffix) PERCPU_SECTION_BASE_NAME ## suffix +#endif + +#ifndef PERCPU_SYMBOL_NAME +#define PERCPU_SYMBOL_NAME(name) name +#endif + /* * Memory encryption operates on a page basis. Since we need to clear * the memory encryption mask for this section, it needs to be aligned @@ -903,7 +917,7 @@ #ifdef CONFIG_AMD_MEM_ENCRYPT #define PERCPU_DECRYPTED_SECTION \ . = ALIGN(PAGE_SIZE); \ - *(.data..percpu..decrypted) \ + *(PERCPU_SECTION_NAME(..decrypted)) \ . = ALIGN(PAGE_SIZE); #else #define PERCPU_DECRYPTED_SECTION @@ -947,17 +961,17 @@ * sharing between subsections for different purposes. */ #define PERCPU_INPUT(cacheline) \ - __per_cpu_start = .; \ - *(.data..percpu..first) \ + PERCPU_SYMBOL_NAME(__per_cpu_start) = .; \ + *(PERCPU_SECTION_NAME(..first)) \ . = ALIGN(PAGE_SIZE); \ - *(.data..percpu..page_aligned) \ + *(PERCPU_SECTION_NAME(..page_aligned)) \ . = ALIGN(cacheline); \ - *(.data..percpu..read_mostly) \ + *(PERCPU_SECTION_NAME(..read_mostly)) \ . = ALIGN(cacheline); \ - *(.data..percpu) \ - *(.data..percpu..shared_aligned) \ + *(PERCPU_SECTION_NAME()) \ + *(PERCPU_SECTION_NAME(..shared_aligned)) \ PERCPU_DECRYPTED_SECTION \ - __per_cpu_end = .; + PERCPU_SYMBOL_NAME(__per_cpu_end) = .; /** * PERCPU_VADDR - define output section for percpu area @@ -984,11 +998,11 @@ * address, use PERCPU_SECTION. */ #define PERCPU_VADDR(cacheline, vaddr, phdr) \ - __per_cpu_load = .; \ - .data..percpu vaddr : AT(__per_cpu_load - LOAD_OFFSET) { \ + PERCPU_SYMBOL_NAME(__per_cpu_load) = .; \ + PERCPU_SECTION_NAME() vaddr : AT(PERCPU_SYMBOL_NAME(__per_cpu_load) - LOAD_OFFSET) { \ PERCPU_INPUT(cacheline) \ } phdr \ - . = __per_cpu_load + SIZEOF(.data..percpu); + . = PERCPU_SYMBOL_NAME(__per_cpu_load) + SIZEOF(PERCPU_SECTION_NAME()); /** * PERCPU_SECTION - define output section for percpu area, simple version @@ -1004,8 +1018,8 @@ */ #define PERCPU_SECTION(cacheline) \ . = ALIGN(PAGE_SIZE); \ - .data..percpu : AT(ADDR(.data..percpu) - LOAD_OFFSET) { \ - __per_cpu_load = .; \ + PERCPU_SECTION_NAME() : AT(ADDR(PERCPU_SECTION_NAME()) - LOAD_OFFSET) { \ + PERCPU_SYMBOL_NAME(__per_cpu_load) = .; \ PERCPU_INPUT(cacheline) \ } From patchwork Wed Jul 22 16:44:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11678929 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 53026138A for ; Wed, 22 Jul 2020 16:46:59 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1E549206C1 for ; Wed, 22 Jul 2020 16:46:59 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="cfB4GQBR"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="nDh5CTG/" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1E549206C1 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=tPeZGRQO3eNqRtoU7Th2WarRxMTTurlnBsJQR5V+u/w=; b=cfB4GQBRNdLnRLzwkEMKSFl2J rjQlU4EWhagQjyOptFQ3WT86tE6GKz9tzEA9Ol+6q+B6q7wZSwJwZTdsLpjAttnpmIQvj3tniYWOY 1LN3VhCNQtAP0ntQHVCkVHQv3RjUBp9kY/uYgp5KwSYRdk7VO0FJRtdmR8hqN2bzHCaGVJxN0pXjq wU0edyDBrwVdM9pxnLbyXqCtzom71YTLJu3uIAYiM2OpfI2a3krPO/F4C99ChKJ5zKZ7/pALYOCnu ygD2cvExrpJS43a1T5jTbtybPR6QJQSS7Wn9lky6k9cm+BOvyusI/ZFkCxN9umTHSi9Qg5wMku7LP WSFcenx4A==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jyHrk-0001vf-Oq; Wed, 22 Jul 2020 16:45:13 +0000 Received: from mail-wr1-x441.google.com ([2a00:1450:4864:20::441]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jyHrc-0001sC-4V for linux-arm-kernel@lists.infradead.org; Wed, 22 Jul 2020 16:45:05 +0000 Received: by mail-wr1-x441.google.com with SMTP id o11so2556588wrv.9 for ; Wed, 22 Jul 2020 09:45:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=eImpnhEO4VwAZh9DFGEK47etMR4IqLUuZar0HHVWIYc=; b=nDh5CTG/ZDTR3Wf0MrlsOp7iOd3+YLoaPlpD6Bd+S6cx9plMTW+TRhrMPSNz0g4YON B5/2L0zyt4puhDMJz8i6g/0fcnTYdFbJxnxZgDwO+8D1f4p5pEoaT6WIWZsWLRM6bfqq Z9gycLLkiEyo1xBqk7+sLoqPv6LI3xNvDihpVjjCunFBjCHJSpqgg+wVlHfLsKtUNSTt 6iygFJoWDq2Do8PRj+neXgNJhvpHt58Ho9lS3sLZXjpIlAR0snEIbO9PjwlFMn5v+RZR QlQzorddi/Bcy8kczzNq8nBrmuBMF2CiOtImpTwevuKkiAJOM44tYdE0qATGIa9HlmtK /hng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=eImpnhEO4VwAZh9DFGEK47etMR4IqLUuZar0HHVWIYc=; b=sXdlmFIvfLhUAK35JwAR5sruOtFJ1IYwGiEJViSNdqu52IXQD692sdO65aSOEu4JQG sCdxXnIxqGgJebzAYVgX3V9mJyckJ5MXkxfSIMtzkmRasIFjUvmMrKKcGnF/fzHQoR5s 8Lz664NBKQ5gpOvHdIbmXUjJJ2zb/jfXJ0h5EqJRasolCrASMrHCeNWpJmmCJyXDX2/W O+2cwHeFcsAiCOz4JSicU+v1fBEUYFn3pBvZetHgb1OtR0Jb7MBZMhL1Yh9GdiOHhbEy hwju0QLV8/xxLbW+pASBPXL/o9t2mTvmpUl6p6p33sYgnbNtEkF10B4FoD1aG/28RciK w+7A== X-Gm-Message-State: AOAM533Fif2l7t7+h6anttPQ09vtbQr7BUbEVArztIeY3NTrGhV9XRlA jmYp1kWn/G2toxGO1ZVHvqXzVg== X-Google-Smtp-Source: ABdhPJyxyJUAxG9reBle7yXKCyrv+T4kQrCRc8RdgMK057wOMWcrkVqWbo9PqLlkpstGMWqATZ6LGw== X-Received: by 2002:adf:bc54:: with SMTP id a20mr371420wrh.227.1595436302760; Wed, 22 Jul 2020 09:45:02 -0700 (PDT) Received: from localhost ([2a01:4b00:8523:2d03:b0ee:900a:e004:b9d0]) by smtp.gmail.com with ESMTPSA id d13sm469069wrq.89.2020.07.22.09.45.01 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 22 Jul 2020 09:45:02 -0700 (PDT) From: David Brazdil To: Catalin Marinas , Will Deacon , Marc Zyngier , Dennis Zhou , Tejun Heo , Christoph Lameter , Arnd Bergmann Subject: [PATCH 2/9] kvm: arm64: Remove __hyp_this_cpu_read Date: Wed, 22 Jul 2020 17:44:17 +0100 Message-Id: <20200722164424.42225-3-dbrazdil@google.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200722164424.42225-1-dbrazdil@google.com> References: <20200722164424.42225-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200722_124504_226186_97CDE9A7 X-CRM114-Status: GOOD ( 18.65 ) X-Spam-Score: -15.7 (---------------) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (-15.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2a00:1450:4864:20:0:0:0:441 listed in] [list.dnswl.org] -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list -7.5 USER_IN_DEF_SPF_WL From: address is in the default SPF white-list 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.5 ENV_AND_HDR_SPF_MATCH Env and Hdr From used in default SPF WL Match -0.0 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-arch@vger.kernel.org, Suzuki K Poulose , linux-kernel@vger.kernel.org, kernel-team@google.com, James Morse , Julien Thierry , David Brazdil , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org this_cpu_ptr is meant for use in kernel proper because it selects between TPIDR_EL1/2 based on nVHE/VHE. __hyp_this_cpu_ptr was used in hyp to always select TPIDR_EL2. Unify all users behind this_cpu_ptr and friends by selecting _EL2 register under __KVM_NVHE_HYPERVISOR__. Under CONFIG_DEBUG_PREEMPT, the kernel helpers perform a preemption check which is omitted by the hyp helpers. Preserve the behavior for nVHE by overriding the corresponding macros under __KVM_NVHE_HYPERVISOR__. Extend the checks into VHE hyp code. Signed-off-by: David Brazdil --- arch/arm64/include/asm/kvm_asm.h | 20 -------------- arch/arm64/include/asm/percpu.h | 33 +++++++++++++++++++++-- arch/arm64/kvm/hyp/include/hyp/debug-sr.h | 4 +-- arch/arm64/kvm/hyp/include/hyp/switch.h | 6 ++--- arch/arm64/kvm/hyp/nvhe/switch.c | 2 +- arch/arm64/kvm/hyp/vhe/switch.c | 2 +- arch/arm64/kvm/hyp/vhe/sysreg-sr.c | 4 +-- 7 files changed, 40 insertions(+), 31 deletions(-) diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h index fb1a922b31ba..da4a0826cacd 100644 --- a/arch/arm64/include/asm/kvm_asm.h +++ b/arch/arm64/include/asm/kvm_asm.h @@ -149,26 +149,6 @@ extern char __smccc_workaround_1_smc[__SMCCC_WORKAROUND_1_SMC_SZ]; addr; \ }) -/* - * Home-grown __this_cpu_{ptr,read} variants that always work at HYP, - * provided that sym is really a *symbol* and not a pointer obtained from - * a data structure. As for SHIFT_PERCPU_PTR(), the creative casting keeps - * sparse quiet. - */ -#define __hyp_this_cpu_ptr(sym) \ - ({ \ - void *__ptr; \ - __verify_pcpu_ptr(&sym); \ - __ptr = hyp_symbol_addr(sym); \ - __ptr += read_sysreg(tpidr_el2); \ - (typeof(sym) __kernel __force *)__ptr; \ - }) - -#define __hyp_this_cpu_read(sym) \ - ({ \ - *__hyp_this_cpu_ptr(sym); \ - }) - #else /* __ASSEMBLY__ */ .macro hyp_adr_this_cpu reg, sym, tmp diff --git a/arch/arm64/include/asm/percpu.h b/arch/arm64/include/asm/percpu.h index 0b6409b89e5e..b4008331475b 100644 --- a/arch/arm64/include/asm/percpu.h +++ b/arch/arm64/include/asm/percpu.h @@ -19,7 +19,21 @@ static inline void set_my_cpu_offset(unsigned long off) :: "r" (off) : "memory"); } -static inline unsigned long __my_cpu_offset(void) +static inline unsigned long __hyp_my_cpu_offset(void) +{ + unsigned long off; + + /* + * We want to allow caching the value, so avoid using volatile and + * instead use a fake stack read to hazard against barrier(). + */ + asm("mrs %0, tpidr_el2" : "=r" (off) : + "Q" (*(const unsigned long *)current_stack_pointer)); + + return off; +} + +static inline unsigned long __kern_my_cpu_offset(void) { unsigned long off; @@ -35,7 +49,12 @@ static inline unsigned long __my_cpu_offset(void) return off; } -#define __my_cpu_offset __my_cpu_offset() + +#ifdef __KVM_NVHE_HYPERVISOR__ +#define __my_cpu_offset __hyp_my_cpu_offset() +#else +#define __my_cpu_offset __kern_my_cpu_offset() +#endif #define PERCPU_RW_OPS(sz) \ static inline unsigned long __percpu_read_##sz(void *ptr) \ @@ -227,4 +246,14 @@ PERCPU_RET_OP(add, add, ldadd) #include +/* Redefine macros for nVHE hyp under DEBUG_PREEMPT to avoid its dependencies. */ +#if defined(__KVM_NVHE_HYPERVISOR__) && defined(CONFIG_DEBUG_PREEMPT) +#undef this_cpu_ptr +#define this_cpu_ptr raw_cpu_ptr +#undef __this_cpu_read +#define __this_cpu_read raw_cpu_read +#undef __this_cpu_write +#define __this_cpu_write raw_cpu_write +#endif + #endif /* __ASM_PERCPU_H */ diff --git a/arch/arm64/kvm/hyp/include/hyp/debug-sr.h b/arch/arm64/kvm/hyp/include/hyp/debug-sr.h index 0297dc63988c..3b2056a225ff 100644 --- a/arch/arm64/kvm/hyp/include/hyp/debug-sr.h +++ b/arch/arm64/kvm/hyp/include/hyp/debug-sr.h @@ -135,7 +135,7 @@ static inline void __debug_switch_to_guest_common(struct kvm_vcpu *vcpu) if (!(vcpu->arch.flags & KVM_ARM64_DEBUG_DIRTY)) return; - host_ctxt = &__hyp_this_cpu_ptr(kvm_host_data)->host_ctxt; + host_ctxt = &this_cpu_ptr(&kvm_host_data)->host_ctxt; guest_ctxt = &vcpu->arch.ctxt; host_dbg = &vcpu->arch.host_debug_state.regs; guest_dbg = kern_hyp_va(vcpu->arch.debug_ptr); @@ -154,7 +154,7 @@ static inline void __debug_switch_to_host_common(struct kvm_vcpu *vcpu) if (!(vcpu->arch.flags & KVM_ARM64_DEBUG_DIRTY)) return; - host_ctxt = &__hyp_this_cpu_ptr(kvm_host_data)->host_ctxt; + host_ctxt = &this_cpu_ptr(&kvm_host_data)->host_ctxt; guest_ctxt = &vcpu->arch.ctxt; host_dbg = &vcpu->arch.host_debug_state.regs; guest_dbg = kern_hyp_va(vcpu->arch.debug_ptr); diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h index 0511af14dc81..e69c2c6098a1 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -382,7 +382,7 @@ static inline bool __hyp_handle_ptrauth(struct kvm_vcpu *vcpu) !esr_is_ptrauth_trap(kvm_vcpu_get_esr(vcpu))) return false; - ctxt = &__hyp_this_cpu_ptr(kvm_host_data)->host_ctxt; + ctxt = &this_cpu_ptr(&kvm_host_data)->host_ctxt; __ptrauth_save_key(ctxt, APIA); __ptrauth_save_key(ctxt, APIB); __ptrauth_save_key(ctxt, APDA); @@ -491,7 +491,7 @@ static inline void __set_guest_arch_workaround_state(struct kvm_vcpu *vcpu) * guest wants it disabled, so be it... */ if (__needs_ssbd_off(vcpu) && - __hyp_this_cpu_read(arm64_ssbd_callback_required)) + __this_cpu_read(arm64_ssbd_callback_required)) arm_smccc_1_1_smc(ARM_SMCCC_ARCH_WORKAROUND_2, 0, NULL); #endif } @@ -503,7 +503,7 @@ static inline void __set_host_arch_workaround_state(struct kvm_vcpu *vcpu) * If the guest has disabled the workaround, bring it back on. */ if (__needs_ssbd_off(vcpu) && - __hyp_this_cpu_read(arm64_ssbd_callback_required)) + __this_cpu_read(arm64_ssbd_callback_required)) arm_smccc_1_1_smc(ARM_SMCCC_ARCH_WORKAROUND_2, 1, NULL); #endif } diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/switch.c index 341be2f2f312..ddb602ffb022 100644 --- a/arch/arm64/kvm/hyp/nvhe/switch.c +++ b/arch/arm64/kvm/hyp/nvhe/switch.c @@ -175,7 +175,7 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu) vcpu = kern_hyp_va(vcpu); - host_ctxt = &__hyp_this_cpu_ptr(kvm_host_data)->host_ctxt; + host_ctxt = &this_cpu_ptr(&kvm_host_data)->host_ctxt; host_ctxt->__hyp_running_vcpu = vcpu; guest_ctxt = &vcpu->arch.ctxt; diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c index c52d714e0d75..746fcc3974c7 100644 --- a/arch/arm64/kvm/hyp/vhe/switch.c +++ b/arch/arm64/kvm/hyp/vhe/switch.c @@ -108,7 +108,7 @@ static int __kvm_vcpu_run_vhe(struct kvm_vcpu *vcpu) struct kvm_cpu_context *guest_ctxt; u64 exit_code; - host_ctxt = &__hyp_this_cpu_ptr(kvm_host_data)->host_ctxt; + host_ctxt = &this_cpu_ptr(&kvm_host_data)->host_ctxt; host_ctxt->__hyp_running_vcpu = vcpu; guest_ctxt = &vcpu->arch.ctxt; diff --git a/arch/arm64/kvm/hyp/vhe/sysreg-sr.c b/arch/arm64/kvm/hyp/vhe/sysreg-sr.c index 996471e4c138..2a0b8c88d74f 100644 --- a/arch/arm64/kvm/hyp/vhe/sysreg-sr.c +++ b/arch/arm64/kvm/hyp/vhe/sysreg-sr.c @@ -66,7 +66,7 @@ void kvm_vcpu_load_sysregs_vhe(struct kvm_vcpu *vcpu) struct kvm_cpu_context *guest_ctxt = &vcpu->arch.ctxt; struct kvm_cpu_context *host_ctxt; - host_ctxt = &__hyp_this_cpu_ptr(kvm_host_data)->host_ctxt; + host_ctxt = &this_cpu_ptr(&kvm_host_data)->host_ctxt; __sysreg_save_user_state(host_ctxt); /* @@ -100,7 +100,7 @@ void kvm_vcpu_put_sysregs_vhe(struct kvm_vcpu *vcpu) struct kvm_cpu_context *guest_ctxt = &vcpu->arch.ctxt; struct kvm_cpu_context *host_ctxt; - host_ctxt = &__hyp_this_cpu_ptr(kvm_host_data)->host_ctxt; + host_ctxt = &this_cpu_ptr(&kvm_host_data)->host_ctxt; deactivate_traps_vhe_put(); __sysreg_save_el1_state(guest_ctxt); From patchwork Wed Jul 22 16:44:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11678939 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 847FA13B6 for ; Wed, 22 Jul 2020 16:47:21 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 59FF3206C1 for ; Wed, 22 Jul 2020 16:47:21 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="U198pHOH"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="MVr23i9E" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 59FF3206C1 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=KtqdmIdPKCSwhf9EqNBmnvlCsi80Yzdi4BrXmr0l13Q=; b=U198pHOHJr2Mq6g7FVveb0TPq +xrt4POIfo/7jO3/gTozqf3JJL2gkH7nD37fLL+6nRUHKynGVa55h61MyTSaqGnxbDNd3YWwz/2/X prPQR5/ZYw48qcagx5QdjNW1uXliRKLT5S1Z6eZfMDjxd1nfaAjUURT02XuHwe3h0ZyeH7mzNF+ZS FfxXEoWnWsXsmu3CdhsXZm+M77nP2m6Sn46hxhccnczdM+F2URak6SWZx0Av5uZHSlE8FB3YaTEf8 d3pmJx+B1WdK+zowOAvpRd6M4P0ctXg2yo/yx9ouz69Ko+KopAGvS2L5G40Vv3UueSkKkINOTtcj2 tkwDnELtQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jyHs1-00021O-5l; Wed, 22 Jul 2020 16:45:29 +0000 Received: from mail-wr1-x441.google.com ([2a00:1450:4864:20::441]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jyHrd-0001sp-JL for linux-arm-kernel@lists.infradead.org; Wed, 22 Jul 2020 16:45:06 +0000 Received: by mail-wr1-x441.google.com with SMTP id f7so2572266wrw.1 for ; Wed, 22 Jul 2020 09:45:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=UzkCazi9IF2R2fovvI4XRbgeSUfI2aZOTLYWCOQmH7Y=; b=MVr23i9EY3adsavlnzqKlwywAbRQmWnYu/O0PqHFO5h2mZINEVU5vdwtCNcmVIzmqc HuDpvw4REHKGW2v9gawHMWOspu7nYKIcm7m1oe8P0DG0AG/jVj8RQJM+fncEU8eAoeFW Q4grx48wtHAnOBBbFmFB/F65EWt+ytE4kp9aCo51/gz5p+jKu5T4002eOYPYmq7uEzE1 NJ+zdUhBvFRvtHM/RZLaDZ3LPvYeZlJRJrm4vb3JYln4uQgm1YmjZZ9WkIcvE4+s0QMo wpBITPMxNDcLOEuTS6TvgjT2x1TCzG34+CS/lKZmQkI5ELNGsHGz5y5Vo26RHLtMkgdF k+3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=UzkCazi9IF2R2fovvI4XRbgeSUfI2aZOTLYWCOQmH7Y=; b=Gap6hfEqHwblGrXnhaCCalDRU0P4GTcNZkcTcGm50F8yaAzjD2oKemZ94MRZWllqfM uAX8aBNw5sV5c8jRgDXWB5btK/IJeRTigjgOlGOaYs92Zt7xvFfpTN3WFkTozyHwfYMB ArPIJxGlSj7sVdV6Jl8rMCZyQGJ3nVFhfg6PrPIPz9LDbFzOBIRbNiulKNnElHzoDMZi SfR2TEYauyT1+zIl+KmgBgGu6xdYClR5FE600QqIGgmiQeanR0PpO/E+ch74BYSFKsY8 Mds3qjRD8mMs96a7kZRP+972tPQC175tITHIfzTd5TH/Tt5dN7WTw561yd9Fi1vlVG+A HPaA== X-Gm-Message-State: AOAM532qJ9SRGXNdsE9HaWZnYKTp4VsES92pMP4Ijbcb59bg7/8uky+Z G/DJk/Wwm881BKktlAc0lcEkvw== X-Google-Smtp-Source: ABdhPJx/qEkJrXuwujIJYy6c8DcTNrb907z4XGsmWjlpmktsrboRRdsmuZeU2BTNkn5zyZPYZ1dJqQ== X-Received: by 2002:a5d:4a45:: with SMTP id v5mr417138wrs.228.1595436304534; Wed, 22 Jul 2020 09:45:04 -0700 (PDT) Received: from localhost ([2a01:4b00:8523:2d03:b0ee:900a:e004:b9d0]) by smtp.gmail.com with ESMTPSA id 2sm250421wmo.44.2020.07.22.09.45.03 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 22 Jul 2020 09:45:03 -0700 (PDT) From: David Brazdil To: Catalin Marinas , Will Deacon , Marc Zyngier , Dennis Zhou , Tejun Heo , Christoph Lameter , Arnd Bergmann Subject: [PATCH 3/9] kvm: arm64: Remove hyp_adr/ldr_this_cpu Date: Wed, 22 Jul 2020 17:44:18 +0100 Message-Id: <20200722164424.42225-4-dbrazdil@google.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200722164424.42225-1-dbrazdil@google.com> References: <20200722164424.42225-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200722_124505_692198_CD856D21 X-CRM114-Status: GOOD ( 12.71 ) X-Spam-Score: -15.7 (---------------) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (-15.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2a00:1450:4864:20:0:0:0:441 listed in] [list.dnswl.org] -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list -7.5 USER_IN_DEF_SPF_WL From: address is in the default SPF white-list 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.5 ENV_AND_HDR_SPF_MATCH Env and Hdr From used in default SPF WL Match -0.0 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-arch@vger.kernel.org, Suzuki K Poulose , linux-kernel@vger.kernel.org, kernel-team@google.com, James Morse , Julien Thierry , David Brazdil , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org The hyp_adr/ldr_this_cpu helpers were introduced for use in hyp code because they always needed to use TPIDR_EL2 for base, while adr/ldr_this_cpu from kernel proper would select between TPIDR_EL2 and _EL1 based on VHE/nVHE. Simplify this now that the nVHE hyp mode case can be handled using the __KVM_NVHE_HYPERVISOR__ macro. VHE hyp code selects _EL2 with alternatives. Signed-off-by: David Brazdil --- arch/arm64/include/asm/assembler.h | 27 +++++++++++++++++---------- arch/arm64/include/asm/kvm_asm.h | 14 +------------- arch/arm64/kvm/hyp/hyp-entry.S | 2 +- 3 files changed, 19 insertions(+), 24 deletions(-) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 54d181177656..b392a977efb6 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -218,6 +218,21 @@ lr .req x30 // link register str \src, [\tmp, :lo12:\sym] .endm + /* + * @dst: destination register (32 or 64 bit wide) + */ + .macro this_cpu_offset, dst +#ifdef __KVM_NVHE_HYPERVISOR__ + mrs \dst, tpidr_el2 +#else +alternative_if_not ARM64_HAS_VIRT_HOST_EXTN + mrs \dst, tpidr_el1 +alternative_else + mrs \dst, tpidr_el2 +alternative_endif +#endif + .endm + /* * @dst: Result of per_cpu(sym, smp_processor_id()) (can be SP) * @sym: The name of the per-cpu variable @@ -226,11 +241,7 @@ lr .req x30 // link register .macro adr_this_cpu, dst, sym, tmp adrp \tmp, \sym add \dst, \tmp, #:lo12:\sym -alternative_if_not ARM64_HAS_VIRT_HOST_EXTN - mrs \tmp, tpidr_el1 -alternative_else - mrs \tmp, tpidr_el2 -alternative_endif + this_cpu_offset \tmp add \dst, \dst, \tmp .endm @@ -241,11 +252,7 @@ alternative_endif */ .macro ldr_this_cpu dst, sym, tmp adr_l \dst, \sym -alternative_if_not ARM64_HAS_VIRT_HOST_EXTN - mrs \tmp, tpidr_el1 -alternative_else - mrs \tmp, tpidr_el2 -alternative_endif + this_cpu_offset \tmp ldr \dst, [\dst, \tmp] .endm diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h index da4a0826cacd..bbd14e205aba 100644 --- a/arch/arm64/include/asm/kvm_asm.h +++ b/arch/arm64/include/asm/kvm_asm.h @@ -151,20 +151,8 @@ extern char __smccc_workaround_1_smc[__SMCCC_WORKAROUND_1_SMC_SZ]; #else /* __ASSEMBLY__ */ -.macro hyp_adr_this_cpu reg, sym, tmp - adr_l \reg, \sym - mrs \tmp, tpidr_el2 - add \reg, \reg, \tmp -.endm - -.macro hyp_ldr_this_cpu reg, sym, tmp - adr_l \reg, \sym - mrs \tmp, tpidr_el2 - ldr \reg, [\reg, \tmp] -.endm - .macro get_host_ctxt reg, tmp - hyp_adr_this_cpu \reg, kvm_host_data, \tmp + adr_this_cpu \reg, kvm_host_data, \tmp add \reg, \reg, #HOST_DATA_CONTEXT .endm diff --git a/arch/arm64/kvm/hyp/hyp-entry.S b/arch/arm64/kvm/hyp/hyp-entry.S index 689fccbc9de7..0a0cb1d3acd3 100644 --- a/arch/arm64/kvm/hyp/hyp-entry.S +++ b/arch/arm64/kvm/hyp/hyp-entry.S @@ -108,7 +108,7 @@ alternative_cb_end str x0, [x2, #VCPU_WORKAROUND_FLAGS] /* Check that we actually need to perform the call */ - hyp_ldr_this_cpu x0, arm64_ssbd_callback_required, x2 + ldr_this_cpu x0, arm64_ssbd_callback_required, x2 cbz x0, wa2_end mov w0, #ARM_SMCCC_ARCH_WORKAROUND_2 From patchwork Wed Jul 22 16:44:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11678935 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0EA1D138A for ; Wed, 22 Jul 2020 16:47:18 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CE0F1206C1 for ; Wed, 22 Jul 2020 16:47:17 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="XhtDPMgw"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="FdmUXfPp" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CE0F1206C1 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=isU831hsT8p0mikRsFJGgDzZSIZAjti8iuqeIEPF/ZU=; b=XhtDPMgw0m4ojHKExL3Lxy51X NA9CyTru6SPS4vgOSZr8L7t3x3B/mB7rbk34T4Cw8OIevHC2tFhX6fYpS9ZDfO9PVzSBoCN88nhWU 13L16IUCgebEMpIOIHQbe/4lY2dutHIgrCECDi/Ok9YW9gY/PMa7g5Xw3abQYsMO4zXNsLDpDMJ2S ffSF0210XBHog+hS1eNtvsqPPGArYwYchnjEBxJuZAh/yUqqDrkOFxg3jILzRaPKipF8kO8T76yJY m39NDNWIIrNvX7xpXl4zlRRYVqeznMTqvkxi+N0PAdRBuinTrgFzHio77X/q190JozGX9syhIYCQo LFBGZEQLw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jyHs4-00022O-9e; Wed, 22 Jul 2020 16:45:33 +0000 Received: from mail-wr1-x441.google.com ([2a00:1450:4864:20::441]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jyHrf-0001tU-Cd for linux-arm-kernel@lists.infradead.org; Wed, 22 Jul 2020 16:45:08 +0000 Received: by mail-wr1-x441.google.com with SMTP id f1so2009086wro.2 for ; Wed, 22 Jul 2020 09:45:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ZXiAbLOq4ubRUcURJC9B1BLv2xJ+3CpTn0EBGii9veo=; b=FdmUXfPpY7o9uEYvSP5nVu4ofg8PL0EkzaQt7c+IphnlbncN6veKPYDXPYLTXRODhI 6vN22+BUNOewLPRoB+VyntfTGU4957QCwvCWV4Rw7/XkrQ5nCgRdII3n6ncKJXismyaG QS0jI990BiiNf12aOl1OO24EJqhL4eaTHvJhZbTDwwVFTC3dpmWz+SUWBIJik8UNPp67 ECuHSrQPykTUctnyqBj2l7S9VS7/jA1LuCpMeq71p8zbNS1+omtsRiI4ZMO+HJbglaMO Mum0PSJ1/kp1e9/TTPm0RAKKWXHtacAuaJYN4Yme80qiYihUdK+xHTp+LbJ5sEzZYVCJ fGDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ZXiAbLOq4ubRUcURJC9B1BLv2xJ+3CpTn0EBGii9veo=; b=TmHoQOgqD8eiQxOqApBOfH/3on18+cv9PvPbyie/rFI8DBNnvLJrD8mnzrq5ldBdfL 35j2K/veWmK4pyMHSR9Bqx3yTnQ9XLKdSZfLzyEBvKbQ5x2KOrJqh8f6VZ1t/ddrcpEu xt8WfnT0gFqwY9ecafii6jXBUQzKfrdp1WkYTiuG2RcgZFNdfVtIbQu7j3yLtV5Tb5RM B0uGrSV/TSNoFAkiuzWsBEyPxyiggtqkX8QVy9QJpDqj85D92S9SATBwm+rwk59hkXrT 6hBvyA6LRCnU2STWveF8ZGxee5ip2v0jN8dkUoaMYX6hbtOVFEuVok06bgv73wj9g10Z B03A== X-Gm-Message-State: AOAM530CY1KMJJgHm0sL9+MMlwJ5yrce5ZPd7tVT40SCtiorHTy7P7VE zAHmZkRhpBshrYlrubKa6bqyTQ== X-Google-Smtp-Source: ABdhPJzJgI+HFT65fE1fBmZ9mEiYEXOQtPMo4T9cTDa2bRmHKYwwP+XUbleLmCwLMIPgGxnThsRfXw== X-Received: by 2002:a5d:6288:: with SMTP id k8mr347278wru.373.1595436306334; Wed, 22 Jul 2020 09:45:06 -0700 (PDT) Received: from localhost ([2a01:4b00:8523:2d03:b0ee:900a:e004:b9d0]) by smtp.gmail.com with ESMTPSA id 33sm590292wri.16.2020.07.22.09.45.05 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 22 Jul 2020 09:45:05 -0700 (PDT) From: David Brazdil To: Catalin Marinas , Will Deacon , Marc Zyngier , Dennis Zhou , Tejun Heo , Christoph Lameter , Arnd Bergmann Subject: [PATCH 4/9] kvm: arm64: Add helpers for accessing nVHE hyp per-cpu vars Date: Wed, 22 Jul 2020 17:44:19 +0100 Message-Id: <20200722164424.42225-5-dbrazdil@google.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200722164424.42225-1-dbrazdil@google.com> References: <20200722164424.42225-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200722_124507_497987_4EA415BA X-CRM114-Status: GOOD ( 13.46 ) X-Spam-Score: -15.7 (---------------) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (-15.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2a00:1450:4864:20:0:0:0:441 listed in] [list.dnswl.org] -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list -7.5 USER_IN_DEF_SPF_WL From: address is in the default SPF white-list 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.5 ENV_AND_HDR_SPF_MATCH Env and Hdr From used in default SPF WL Match -0.0 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-arch@vger.kernel.org, Suzuki K Poulose , linux-kernel@vger.kernel.org, kernel-team@google.com, James Morse , Julien Thierry , David Brazdil , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org Defining a per-CPU variable in hyp/nvhe will result in its name being prefixed with __kvm_nvhe_. Add helpers for declaring these variables in kernel proper and accessing them with this_cpu_ptr and per_cpu_ptr. Signed-off-by: David Brazdil --- arch/arm64/include/asm/kvm_asm.h | 25 +++++++++++++++++++++++-- 1 file changed, 23 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h index bbd14e205aba..3d69cab873e4 100644 --- a/arch/arm64/include/asm/kvm_asm.h +++ b/arch/arm64/include/asm/kvm_asm.h @@ -60,9 +60,21 @@ DECLARE_KVM_VHE_SYM(sym); \ DECLARE_KVM_NVHE_SYM(sym) +#define DECLARE_KVM_VHE_PER_CPU(type, sym) \ + DECLARE_PER_CPU(type, sym) +#define DECLARE_KVM_NVHE_PER_CPU(type, sym) \ + DECLARE_PER_CPU(type, kvm_nvhe_sym(sym)) + +#define DECLARE_KVM_HYP_PER_CPU(type, sym) \ + DECLARE_KVM_VHE_PER_CPU(type, sym); \ + DECLARE_KVM_NVHE_PER_CPU(type, sym) + #define CHOOSE_VHE_SYM(sym) sym #define CHOOSE_NVHE_SYM(sym) kvm_nvhe_sym(sym) +#define this_cpu_ptr_nvhe(sym) this_cpu_ptr(&kvm_nvhe_sym(sym)) +#define per_cpu_ptr_nvhe(sym, cpu) per_cpu_ptr(&kvm_nvhe_sym(sym), cpu) + #ifndef __KVM_NVHE_HYPERVISOR__ /* * BIG FAT WARNINGS: @@ -75,12 +87,21 @@ * - Don't let the nVHE hypervisor have access to this, as it will * pick the *wrong* symbol (yes, it runs at EL2...). */ -#define CHOOSE_HYP_SYM(sym) (is_kernel_in_hyp_mode() ? CHOOSE_VHE_SYM(sym) \ +#define CHOOSE_HYP_SYM(sym) (is_kernel_in_hyp_mode() \ + ? CHOOSE_VHE_SYM(sym) \ : CHOOSE_NVHE_SYM(sym)) +#define this_cpu_ptr_hyp(sym) (is_kernel_in_hyp_mode() \ + ? this_cpu_ptr(&sym) \ + : this_cpu_ptr_nvhe(sym)) +#define per_cpu_ptr_hyp(sym, cpu) (is_kernel_in_hyp_mode() \ + ? per_cpu_ptr(&sym, cpu) \ + : per_cpu_ptr_nvhe(sym, cpu)) #else /* The nVHE hypervisor shouldn't even try to access anything */ extern void *__nvhe_undefined_symbol; -#define CHOOSE_HYP_SYM(sym) __nvhe_undefined_symbol +#define CHOOSE_HYP_SYM(sym) __nvhe_undefined_symbol +#define this_cpu_ptr_hyp(sym) &__nvhe_undefined_symbol +#define per_cpu_ptr_hyp(sym, cpu) &__nvhe_undefined_symbol #endif /* Translate a kernel address @ptr into its equivalent linear mapping */ From patchwork Wed Jul 22 16:44:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11678937 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DE64113A4 for ; Wed, 22 Jul 2020 16:47:20 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B6C3F206C1 for ; Wed, 22 Jul 2020 16:47:20 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="V/cQ4LkI"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="ktMgAg7h" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B6C3F206C1 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=AHq03JpI9FX6uFyjGtDe2XtUv1/w4Clac0KTxqDHfYk=; b=V/cQ4LkINexUMHetNbMcINxDG bt2Q/YiGU+71B3CXc7jKuroJo/mf8oaQ5Tey7Vfo44KHCDTWzm7SP0PhoEKdPKJ2HjK8z3AhNrphw lKEqu8+Iwn/rBe7jrUtjuyIIETI0qH0QSRfLhcN6Vsa/FMq5gZt2xHeImNUroGjKWuTy1LsK9ViU0 VRbJ7GAL0YukEFvUt+ykjqMyR0M8NucApbuXV71eTioiZtUEzN0ErS/esk9Km7ILP0W5qFCHMbV7Y BmUZDPtrEEZXXflna4FLefaTJfrOnFiID3jkidnpQedU1hZC0jGUfkUqztpK4UGHWs2p0ZL32NDzQ HnRo8QEdQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jyHsC-00025B-7s; Wed, 22 Jul 2020 16:45:40 +0000 Received: from mail-wm1-x344.google.com ([2a00:1450:4864:20::344]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jyHrh-0001uP-Fx for linux-arm-kernel@lists.infradead.org; Wed, 22 Jul 2020 16:45:10 +0000 Received: by mail-wm1-x344.google.com with SMTP id a6so4741390wmm.0 for ; Wed, 22 Jul 2020 09:45:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=WoKr+d4xzFa2q5doiKo+Ma/Mr0IRgGSBGk/CKLNfxgY=; b=ktMgAg7h984Iz5zq71o5DN+J0+FK13F2J9HPBy4DmLuxJ090jgw+CMnoSZwyQY49Ma f6yJhxgSogcgx3bETDFW+KqXCwbYzTtFr6O5sgMkcsR2Hq6w19IrrZuEIpXicZJ3EGVV U78Tpy9fh1G98eE0Ys41it7n1SVaRQbQTvPQg94IRPQEXLYEtTYwgo1TOaoUoep+eDoZ H8rq8GLCnQ+9r8ihbCiMv+byr7bxzMtmDd5XPLTWCHHOKBCD5MR8gklAapKIjdYlZp+I wtYhzP8MLjT3vX1DeAvbZBQhzDpQKV5uegI4SY3ESoosnDFzNvhkcvlbO6uy2d3oSVQM fH5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=WoKr+d4xzFa2q5doiKo+Ma/Mr0IRgGSBGk/CKLNfxgY=; b=Eh2WvkEMDdNay2Us54Q8Kkx8FTmoXyO3kuCEmzyZ6k+2vtaowQ8+gr+4gO38re1k38 yRMs4DqH/KooxIoPOkpIXLrMLId0/i7AFHxIImkpfMm798auJiJnZbiCVfRRlFVQkbIJ Xj/BquHBC1lWgSP0IsWiKiZEowUJAsWiQf5/jtbeBe4109T0tEirHQSH9pJWw9ltMyKB GT8nZ/lmNsXVdlkXFKHaclA2jhcuypVKdzGMSa4VNRRHdWzHwKDJUrBxMGcLBBFfrab8 YwYGDilgrosFKcm22NxHeVy5IKsQgrmvCKoFr2vcGSDa3oLziYXLdje+J7ksj5VXLXw0 NfSw== X-Gm-Message-State: AOAM531TAhz2fdsWBqa/q0AbgjZzkepPF97XG1e4svAtreSPhj4dgbXL cpBiCfB4Qlg+ygf2KbmSrsenZg== X-Google-Smtp-Source: ABdhPJxouZ/81xeqagm6vGSDX+SrQHkz8Q+jvvRw7wYTEIvvg6ju/O9IEPfOIvKPbqVtxczJyAFxwg== X-Received: by 2002:a1c:d8:: with SMTP id 207mr432561wma.81.1595436307978; Wed, 22 Jul 2020 09:45:07 -0700 (PDT) Received: from localhost ([2a01:4b00:8523:2d03:b0ee:900a:e004:b9d0]) by smtp.gmail.com with ESMTPSA id g14sm538679wrw.83.2020.07.22.09.45.06 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 22 Jul 2020 09:45:07 -0700 (PDT) From: David Brazdil To: Catalin Marinas , Will Deacon , Marc Zyngier , Dennis Zhou , Tejun Heo , Christoph Lameter , Arnd Bergmann Subject: [PATCH 5/9] kvm: arm64: Duplicate arm64_ssbd_callback_required for nVHE hyp Date: Wed, 22 Jul 2020 17:44:20 +0100 Message-Id: <20200722164424.42225-6-dbrazdil@google.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200722164424.42225-1-dbrazdil@google.com> References: <20200722164424.42225-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200722_124509_584217_CFAE0FE4 X-CRM114-Status: GOOD ( 16.48 ) X-Spam-Score: -15.7 (---------------) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (-15.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2a00:1450:4864:20:0:0:0:344 listed in] [list.dnswl.org] -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list -7.5 USER_IN_DEF_SPF_WL From: address is in the default SPF white-list 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.5 ENV_AND_HDR_SPF_MATCH Env and Hdr From used in default SPF WL Match -0.0 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-arch@vger.kernel.org, Suzuki K Poulose , linux-kernel@vger.kernel.org, kernel-team@google.com, James Morse , Julien Thierry , David Brazdil , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org Hyp keeps track of which cores require SSBD callback by accessing a kernel-proper global variable. Create an nVHE symbol of the same name and copy the value from kernel proper to nVHE at KVM init time. Done in preparation for separating percpu memory owned by kernel proper and nVHE. Signed-off-by: David Brazdil --- arch/arm64/include/asm/kvm_mmu.h | 10 +++++++--- arch/arm64/kernel/image-vars.h | 1 - arch/arm64/kvm/arm.c | 2 +- arch/arm64/kvm/hyp/nvhe/switch.c | 3 +++ 4 files changed, 11 insertions(+), 5 deletions(-) diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h index 22157ded04ca..e9e5875274cb 100644 --- a/arch/arm64/include/asm/kvm_mmu.h +++ b/arch/arm64/include/asm/kvm_mmu.h @@ -529,23 +529,27 @@ static inline int kvm_map_vectors(void) #ifdef CONFIG_ARM64_SSBD DECLARE_PER_CPU_READ_MOSTLY(u64, arm64_ssbd_callback_required); +DECLARE_KVM_NVHE_PER_CPU(u64, arm64_ssbd_callback_required); -static inline int hyp_map_aux_data(void) +static inline int hyp_init_aux_data(void) { int cpu, err; for_each_possible_cpu(cpu) { u64 *ptr; - ptr = per_cpu_ptr(&arm64_ssbd_callback_required, cpu); + ptr = per_cpu_ptr_nvhe(arm64_ssbd_callback_required, cpu); err = create_hyp_mappings(ptr, ptr + 1, PAGE_HYP); if (err) return err; + + /* Copy value from kernel to hyp. */ + *ptr = per_cpu(arm64_ssbd_callback_required, cpu); } return 0; } #else -static inline int hyp_map_aux_data(void) +static inline int hyp_init_aux_data(void) { return 0; } diff --git a/arch/arm64/kernel/image-vars.h b/arch/arm64/kernel/image-vars.h index 9e897c500237..034cf21e67ce 100644 --- a/arch/arm64/kernel/image-vars.h +++ b/arch/arm64/kernel/image-vars.h @@ -69,7 +69,6 @@ KVM_NVHE_ALIAS(kvm_patch_vector_branch); KVM_NVHE_ALIAS(kvm_update_va_mask); /* Global kernel state accessed by nVHE hyp code. */ -KVM_NVHE_ALIAS(arm64_ssbd_callback_required); KVM_NVHE_ALIAS(kvm_host_data); KVM_NVHE_ALIAS(kvm_vgic_global_state); diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 98f05bdac3c1..a53e87305fa0 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1547,7 +1547,7 @@ static int init_hyp_mode(void) } } - err = hyp_map_aux_data(); + err = hyp_init_aux_data(); if (err) kvm_err("Cannot map host auxiliary data: %d\n", err); diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/switch.c index ddb602ffb022..8c2bd04df813 100644 --- a/arch/arm64/kvm/hyp/nvhe/switch.c +++ b/arch/arm64/kvm/hyp/nvhe/switch.c @@ -27,6 +27,9 @@ #include #include +/* Non-VHE copy of the kernel symbol. */ +DEFINE_PER_CPU_READ_MOSTLY(u64, arm64_ssbd_callback_required); + static void __activate_traps(struct kvm_vcpu *vcpu) { u64 val; From patchwork Wed Jul 22 16:44:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11678941 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 26B6713A4 for ; Wed, 22 Jul 2020 16:47:36 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id ED3BD206C1 for ; Wed, 22 Jul 2020 16:47:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="LrbT0I/N"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="ZoZ7p09H" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org ED3BD206C1 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=CZoD+hT2ulL3yaLA0Xu0JZviZwaiZSuNhyUKMPdIlwA=; b=LrbT0I/NC8m7oGGF+AFWXyxw9 l4XRMsU09fGerbWh2JCx5rOhH8jcSkpFaFdt/ncVSEq3RwzsUITYlGF2QhlTVHL2VNndXn6OCvT7Z f5GgJjnvdG2aE/AvCGuKREWmjehhlfz/IEcRWLUJ30ekSSTVEo/TKDWH9QB3QRtg/4s4SX20B9JHq THGjwn1BZct2xS2DlZiVB1nNZKsNq6iZ7ljWwsAE2CSDb/xysaUyOpLrqNQAYMpsxKBofcrK5/vQA VMyIACiXThj2y+MtksC5bqJ6Stol6zJRTGPD4lvZ72K0CtlpmKy0O9u4jVW8nlj0yRMY+rTUMbhan PkIRno30Q==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jyHsM-00028n-Ba; Wed, 22 Jul 2020 16:45:51 +0000 Received: from mail-wm1-x341.google.com ([2a00:1450:4864:20::341]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jyHri-0001v3-Vb for linux-arm-kernel@lists.infradead.org; Wed, 22 Jul 2020 16:45:12 +0000 Received: by mail-wm1-x341.google.com with SMTP id 17so2640246wmo.1 for ; Wed, 22 Jul 2020 09:45:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=DU9IS6flmS7+CCKUEvUpb29VHABYdzHg412IRg1X/w0=; b=ZoZ7p09HWfpUjJlNSAYgo5+THZR1yZ39Tb1hSTFAuI79CRWviUIu2KF9NNnIdD+EwI 6646r5SDYQxnXB+TpAP40VJ6UGdFAvmoYXuCG8G9ehWrrn2ZL19+r2xd6Cgi+QBmNMa5 W+SnpNM7fDifDUxbKd/0pGdA/4l5k+6oLqhpQY/TAvw3w8IgrZoQJcRaSOanCkiCcvBK S1ZGtEK7Xoma+JGFq3q/YsWMBJGdtUwR5PTx9O2DC/rZDgRT+uys/1oJSEEdXu0956gJ W6nnxlJnjkwZqDILf8sdUheSEKq3VS0cpMMfCtXKN1Z6nyQBw7zv5yVe2WOImkYa9dao zhHQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=DU9IS6flmS7+CCKUEvUpb29VHABYdzHg412IRg1X/w0=; b=f5h/CRTYdhzFFnvHSP8EwppfOHoSuH6xhS2tq7O7DqTd6B9LzuoSs8WA0WEXOzUotX WIq5xEqlyqtVukBcY/z0wLcS/YSI4u9/Je7LitWO86h+SksRqlfDvyV5hPjvmbZzuhGZ opyUB1JLUTlQ3gXhnhJlA4+DVS+DUAlBLVM+3T/lSf/asjpCar1fQB0M9WDMZrI4tNr/ XVh9HCxorPL+TN9j0VJMSf88TuFT+AhuaBUk1fXXKJWp6BX6WxE/nW/qJqGJ1eHulITk 9gD9SbZsQ0m1UbsU25hrQPZtiG8CNGGSeBs8sOEzyZtwmgTUBgGFtkeNAfbwblzL1Igq g3Cw== X-Gm-Message-State: AOAM5314LtVToSBS7s1XiwHWqRvN3ybUAjyH3Y0mu4hEL8iStBDRvwz5 jwcsdCujds1VoG1YgvB7x5TDdg== X-Google-Smtp-Source: ABdhPJy9BRTUrJxQau5L70fZlV8vjr6phvTKyvOL3z0vh9LpXeBczB7Rf+5/QVXJY/5OxbBCW4oJ3A== X-Received: by 2002:a1c:7402:: with SMTP id p2mr448503wmc.117.1595436309683; Wed, 22 Jul 2020 09:45:09 -0700 (PDT) Received: from localhost ([2a01:4b00:8523:2d03:b0ee:900a:e004:b9d0]) by smtp.gmail.com with ESMTPSA id q4sm346683wme.31.2020.07.22.09.45.08 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 22 Jul 2020 09:45:08 -0700 (PDT) From: David Brazdil To: Catalin Marinas , Will Deacon , Marc Zyngier , Dennis Zhou , Tejun Heo , Christoph Lameter , Arnd Bergmann Subject: [PATCH 6/9] kvm: arm64: Create separate instances of kvm_host_data for VHE/nVHE Date: Wed, 22 Jul 2020 17:44:21 +0100 Message-Id: <20200722164424.42225-7-dbrazdil@google.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200722164424.42225-1-dbrazdil@google.com> References: <20200722164424.42225-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200722_124511_069841_43253102 X-CRM114-Status: GOOD ( 15.64 ) X-Spam-Score: -15.7 (---------------) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (-15.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2a00:1450:4864:20:0:0:0:341 listed in] [list.dnswl.org] -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list -7.5 USER_IN_DEF_SPF_WL From: address is in the default SPF white-list 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.5 ENV_AND_HDR_SPF_MATCH Env and Hdr From used in default SPF WL Match -0.0 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-arch@vger.kernel.org, Suzuki K Poulose , linux-kernel@vger.kernel.org, kernel-team@google.com, James Morse , Julien Thierry , David Brazdil , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org Host CPU context is stored in a global per-cpu variable `kvm_host_data`. In preparation for introducing independent per-CPU region for nVHE hyp, create two separate instances of `kvm_host_data`, one for VHE and one for nVHE. Signed-off-by: David Brazdil --- arch/arm64/include/asm/kvm_host.h | 2 +- arch/arm64/kernel/image-vars.h | 1 - arch/arm64/kvm/arm.c | 5 ++--- arch/arm64/kvm/hyp/nvhe/switch.c | 3 +++ arch/arm64/kvm/hyp/vhe/switch.c | 3 +++ arch/arm64/kvm/pmu.c | 8 ++++---- 6 files changed, 13 insertions(+), 9 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index e1a32c0707bb..a6d61a708056 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -574,7 +574,7 @@ void kvm_set_sei_esr(struct kvm_vcpu *vcpu, u64 syndrome); struct kvm_vcpu *kvm_mpidr_to_vcpu(struct kvm *kvm, unsigned long mpidr); -DECLARE_PER_CPU(kvm_host_data_t, kvm_host_data); +DECLARE_KVM_HYP_PER_CPU(kvm_host_data_t, kvm_host_data); static inline void kvm_init_host_cpu_context(struct kvm_cpu_context *cpu_ctxt) { diff --git a/arch/arm64/kernel/image-vars.h b/arch/arm64/kernel/image-vars.h index 034cf21e67ce..e23b044c4081 100644 --- a/arch/arm64/kernel/image-vars.h +++ b/arch/arm64/kernel/image-vars.h @@ -69,7 +69,6 @@ KVM_NVHE_ALIAS(kvm_patch_vector_branch); KVM_NVHE_ALIAS(kvm_update_va_mask); /* Global kernel state accessed by nVHE hyp code. */ -KVM_NVHE_ALIAS(kvm_host_data); KVM_NVHE_ALIAS(kvm_vgic_global_state); /* Kernel constant needed to compute idmap addresses. */ diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index a53e87305fa0..0700c3d21b23 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -46,7 +46,6 @@ __asm__(".arch_extension virt"); #endif -DEFINE_PER_CPU(kvm_host_data_t, kvm_host_data); static DEFINE_PER_CPU(unsigned long, kvm_arm_hyp_stack_page); /* The VMID used in the VTTBR */ @@ -1303,7 +1302,7 @@ static void cpu_hyp_reset(void) static void cpu_hyp_reinit(void) { - kvm_init_host_cpu_context(&this_cpu_ptr(&kvm_host_data)->host_ctxt); + kvm_init_host_cpu_context(&this_cpu_ptr_hyp(kvm_host_data)->host_ctxt); cpu_hyp_reset(); @@ -1538,7 +1537,7 @@ static int init_hyp_mode(void) for_each_possible_cpu(cpu) { kvm_host_data_t *cpu_data; - cpu_data = per_cpu_ptr(&kvm_host_data, cpu); + cpu_data = per_cpu_ptr_hyp(kvm_host_data, cpu); err = create_hyp_mappings(cpu_data, cpu_data + 1, PAGE_HYP); if (err) { diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/switch.c index 8c2bd04df813..4488d14de1b5 100644 --- a/arch/arm64/kvm/hyp/nvhe/switch.c +++ b/arch/arm64/kvm/hyp/nvhe/switch.c @@ -30,6 +30,9 @@ /* Non-VHE copy of the kernel symbol. */ DEFINE_PER_CPU_READ_MOSTLY(u64, arm64_ssbd_callback_required); +/* Non-VHE instance of kvm_host_data. */ +DEFINE_PER_CPU(kvm_host_data_t, kvm_host_data); + static void __activate_traps(struct kvm_vcpu *vcpu) { u64 val; diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c index 746fcc3974c7..7927a969eca4 100644 --- a/arch/arm64/kvm/hyp/vhe/switch.c +++ b/arch/arm64/kvm/hyp/vhe/switch.c @@ -28,6 +28,9 @@ const char __hyp_panic_string[] = "HYP panic:\nPS:%08llx PC:%016llx ESR:%08llx\nFAR:%016llx HPFAR:%016llx PAR:%016llx\nVCPU:%p\n"; +/* VHE instance of kvm_host_data. */ +DEFINE_PER_CPU(kvm_host_data_t, kvm_host_data); + static void __activate_traps(struct kvm_vcpu *vcpu) { u64 val; diff --git a/arch/arm64/kvm/pmu.c b/arch/arm64/kvm/pmu.c index b5ae3a5d509e..717941910723 100644 --- a/arch/arm64/kvm/pmu.c +++ b/arch/arm64/kvm/pmu.c @@ -31,7 +31,7 @@ static bool kvm_pmu_switch_needed(struct perf_event_attr *attr) */ void kvm_set_pmu_events(u32 set, struct perf_event_attr *attr) { - struct kvm_host_data *ctx = this_cpu_ptr(&kvm_host_data); + struct kvm_host_data *ctx = this_cpu_ptr_hyp(kvm_host_data); if (!kvm_pmu_switch_needed(attr)) return; @@ -47,7 +47,7 @@ void kvm_set_pmu_events(u32 set, struct perf_event_attr *attr) */ void kvm_clr_pmu_events(u32 clr) { - struct kvm_host_data *ctx = this_cpu_ptr(&kvm_host_data); + struct kvm_host_data *ctx = this_cpu_ptr_hyp(kvm_host_data); ctx->pmu_events.events_host &= ~clr; ctx->pmu_events.events_guest &= ~clr; @@ -169,7 +169,7 @@ void kvm_vcpu_pmu_restore_guest(struct kvm_vcpu *vcpu) if (!has_vhe()) return; - host = this_cpu_ptr(&kvm_host_data); + host = this_cpu_ptr_hyp(kvm_host_data); events_guest = host->pmu_events.events_guest; events_host = host->pmu_events.events_host; @@ -188,7 +188,7 @@ void kvm_vcpu_pmu_restore_host(struct kvm_vcpu *vcpu) if (!has_vhe()) return; - host = this_cpu_ptr(&kvm_host_data); + host = this_cpu_ptr_hyp(kvm_host_data); events_guest = host->pmu_events.events_guest; events_host = host->pmu_events.events_host; From patchwork Wed Jul 22 16:44:22 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11678943 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 505C3138A for ; Wed, 22 Jul 2020 16:47:43 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 243A4206C1 for ; Wed, 22 Jul 2020 16:47:43 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="aXmTm0xR"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="mvQ2D74Q" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 243A4206C1 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=2LsNx2/Q8OnuLmS8clzjlRY1CgWP2EvLMjuoTeTukyk=; b=aXmTm0xR3jZBB64RqH2EMRadf 2tg3OmI3l54qspei4GMS2WFv15r8Oy7OW51p+SFEwglXYQ88RM2NL2voN9txxAEaPvROpQLOdotIE 1qlgmXiM4ddS7eQd3r8SybVsssRruZ4Qht4rVAwozaMNKPF7BGYsfPuw5rLBpjMl/dy7YJzD7Z/7+ m/ZOO+EKk+37HRNBkUx5pj1xGY+Q6Oo6/qa7yWn+/eYYkNXfTse/LkHcLOtSd7Q1JQ6Vwl60RKpSU SIvR5utd+FOReKeuulCWnBBU8DELdffLhicfFwkYloQ1izw1wPoCABKUWQeaHNpRs6h8uledlcIoK /ilpyKDMQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jyHsb-0002Cy-7k; Wed, 22 Jul 2020 16:46:05 +0000 Received: from mail-wr1-x443.google.com ([2a00:1450:4864:20::443]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jyHrk-0001vr-KB for linux-arm-kernel@lists.infradead.org; Wed, 22 Jul 2020 16:45:14 +0000 Received: by mail-wr1-x443.google.com with SMTP id s10so2533804wrw.12 for ; Wed, 22 Jul 2020 09:45:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=bcTngC2Bes9sF1CupaxPgWX3tAw0UoRfYaGJIdlmeeQ=; b=mvQ2D74QyUwCzSPTgbT4sWDjrm7UJoVfL3cHenKVZM4lEyQ4kiJMrsWap47ViZ/Wvr rZDWLr9NFPdHynImP8k+kUbehz7sK/izWI9taaixH28DDyAQxLDE2IVjI4JHth6sqvMd Bm3MyxL4Ev2MPQ719BJsT1suvEsdnYD9LiwdooMdWfuceghQTEMxSuCUAzFCiQNkszgr 776DU9p9GQmi0hKDgGS6X4PrAU7F1kaKRpoHIctLA3l3w5RGa68JqwP9mYrDKItjiXEp YdYfHynQXX5o0Cs0P73flkaXSX7R5VWhnBQrZk7Ew+typZtMNI/jv6YpF1C6khkBO0C0 nNTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bcTngC2Bes9sF1CupaxPgWX3tAw0UoRfYaGJIdlmeeQ=; b=hYXkvpqYueEm0u+v16SQHQ2rVpa0roDuxcKp1Y63rWr6yBn8yqDT/tleiJxFJowmu7 RFBLE2RjXfWNHimP8rQdZZORLHt7HKdD3cKB5GUTIE9jzN/CddoGL70/8QO5Jen1W2KZ r6KLXqP4vQjeftp+3tbX5OKUMcrf8BlH63xGxmZUJ7IqHilNU35yOCYJXk7HtX+D6vaz yk9A9UMpAJ0iHZmTosHt2TyWRnY4aRba5CF2TGQigELptrngPlSdDCaQkxZNsfzcInHp 2+4m/5opQWPiH7IoreMwakFHUTj4NQL9PPAgUEK2oDzfAQ7hjLl2/m323EzvagaEvGKr BtuA== X-Gm-Message-State: AOAM533rZLG25KwTS4ZmFz6HLTbA4vxzgo1ds5oQcavab/+WM3S9yx3E vJDLYlSgoGEZLhgsdr7MxZxvgA== X-Google-Smtp-Source: ABdhPJx2hxeObfvv1sweoqi7kI9BIJM5pRi6WnEDs4mkAmQ3drosmvpJW+pWqVpWlbJoNaizyRi9hA== X-Received: by 2002:a5d:558a:: with SMTP id i10mr388087wrv.146.1595436311404; Wed, 22 Jul 2020 09:45:11 -0700 (PDT) Received: from localhost ([2a01:4b00:8523:2d03:b0ee:900a:e004:b9d0]) by smtp.gmail.com with ESMTPSA id e5sm631557wrc.37.2020.07.22.09.45.10 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 22 Jul 2020 09:45:10 -0700 (PDT) From: David Brazdil To: Catalin Marinas , Will Deacon , Marc Zyngier , Dennis Zhou , Tejun Heo , Christoph Lameter , Arnd Bergmann Subject: [PATCH 7/9] kvm: arm64: Mark hyp stack pages reserved Date: Wed, 22 Jul 2020 17:44:22 +0100 Message-Id: <20200722164424.42225-8-dbrazdil@google.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200722164424.42225-1-dbrazdil@google.com> References: <20200722164424.42225-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200722_124513_666700_53762860 X-CRM114-Status: GOOD ( 17.16 ) X-Spam-Score: -15.7 (---------------) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (-15.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2a00:1450:4864:20:0:0:0:443 listed in] [list.dnswl.org] -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list -7.5 USER_IN_DEF_SPF_WL From: address is in the default SPF white-list 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.5 ENV_AND_HDR_SPF_MATCH Env and Hdr From used in default SPF WL Match -0.0 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-arch@vger.kernel.org, Suzuki K Poulose , linux-kernel@vger.kernel.org, kernel-team@google.com, James Morse , Julien Thierry , David Brazdil , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org In preparation for unmapping hyp pages from host stage-2, allocate/free hyp stack using new helpers which automatically mark the pages reserved. Signed-off-by: David Brazdil --- arch/arm64/kvm/arm.c | 49 ++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 47 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 0700c3d21b23..dc557b380c87 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1451,13 +1451,58 @@ static int init_subsystems(void) return err; } +/* + * Alloc pages and mark them reserved so the kernel never tries to + * take them away from the hypervisor. + */ +static unsigned long alloc_hyp_pages(gfp_t flags, unsigned int order) +{ + struct page *page; + unsigned long i; + + page = alloc_pages(flags, order); + if (!page) + return 0; + + for (i = 0; i < (1ul << order); ++i) + mark_page_reserved(page + i); + + return (unsigned long)page_address(page); +} + +static unsigned long alloc_hyp_page(gfp_t flags) +{ + return alloc_hyp_pages(flags, 0); +} + +/* + * Free pages which were previously marked reserved for the hypervisor. + */ +static void free_hyp_pages(unsigned long addr, unsigned int order) +{ + unsigned long i; + struct page *page; + + if (!addr) + return; + + page = virt_to_page(addr); + for (i = 0; i < (1ul << order); ++i) + free_reserved_page(page + i); +} + +static void free_hyp_page(unsigned long addr) +{ + return free_hyp_pages(addr, 0); +} + static void teardown_hyp_mode(void) { int cpu; free_hyp_pgds(); for_each_possible_cpu(cpu) - free_page(per_cpu(kvm_arm_hyp_stack_page, cpu)); + free_hyp_page(per_cpu(kvm_arm_hyp_stack_page, cpu)); } /** @@ -1481,7 +1526,7 @@ static int init_hyp_mode(void) for_each_possible_cpu(cpu) { unsigned long stack_page; - stack_page = __get_free_page(GFP_KERNEL); + stack_page = alloc_hyp_page(GFP_KERNEL); if (!stack_page) { err = -ENOMEM; goto out_err; From patchwork Wed Jul 22 16:44:23 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11678947 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5F60513B6 for ; Wed, 22 Jul 2020 16:48:00 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2AD3F206F5 for ; Wed, 22 Jul 2020 16:48:00 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="Jdz8jksI"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="O+j9vJMx" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2AD3F206F5 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=ngmSYNU9Vn4xlEDB0P/EeXoFb3wNh5qHN9uwbaUxn68=; b=Jdz8jksINaRASAgDCqSyxqya7 bEJiWoPACRkLqIxI4qMMWvQleF8YdwSVwnkZANIFf8HhZqtJCPfkfWBsqDePDNxL4PkfqfRZniDrC ZxI4SyNFdFvMR58RhsYjsd1/dzyfmSxcb1QLnx1xlMDQoKiIoKv+mnEpDRG1uItBhsm3irxN9bb3w wIShX/kxJ3lcwgq+r7bU3STODVNifGps5QV44I4VqbW+mXQPf0WDcxuX5stO8c7X8KSXClixMn6xh 843GO3YRqJRtKQlJeB4JltlAt1uPXZ1ivZXaFGgjYY+QgXltTziM8yv7dhq8t7X21115po+9ei/jl vDjk7qySg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jyHsi-0002HG-Ku; Wed, 22 Jul 2020 16:46:12 +0000 Received: from mail-wm1-x342.google.com ([2a00:1450:4864:20::342]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jyHrm-0001wN-CK for linux-arm-kernel@lists.infradead.org; Wed, 22 Jul 2020 16:45:16 +0000 Received: by mail-wm1-x342.google.com with SMTP id f139so2619176wmf.5 for ; Wed, 22 Jul 2020 09:45:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=V96ak4i8UKYW1530XCTpq7OSsFZ4sEMYY/vy7g+OwDs=; b=O+j9vJMxDWg4Y7iNQtrBdo/PaCRmPCN0IrEKu0IAOXYNS4QWENM1txkn/akKPubHQt 2Q9bO5sSIjnp7XX8FPY3pwRcbHoMqdMJkGzXaT88liZXeU7cj6sVsFUDq1KO/ckOzlAz QczOXDyjtOBABv852W5Tzx6/2+oXjm/PTaAoZQsN+a7dWEbseiKUqyXAyZZWizDvGnjU 1FZUbG1JADmor77thW2lzkIlb3rOoNPKVh6tb9aPE4gepPFwYQCfH9Zb86Pby+wMTA/O NAZylWO739DSTe2kT8txIfBsXMeHOQKFBra06R5Omocjlvxtbruuk8RbcoziaI81HyVd QAPw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=V96ak4i8UKYW1530XCTpq7OSsFZ4sEMYY/vy7g+OwDs=; b=qaKAtVrNSXdkxMOuF1qJVXi1US0GDxXZ5Ou/UzKJMtG2ze8Q0qfO9Bo/zL8uLTjKXs 5quT4LTL4x0KV+luVCseWbQ5ldgxZvOrM2alzfYaHtL74lcO7h57Sqx1z8hZD6fJlujO LGbanSjkFMtTpeAf6+bbWpYgWQzxcRrKHk2dGkTS/HO5wfhmFX0J9F9kJlHBv+MGZ3k3 j8loEPYJPFuC7RrnEY6RKrW3IqSKfAjRJswCP95osctcbCdTkRaK7xuI0CiaRIuEozG7 1/F+848WF9loIFoE5Rz4ERTfVBsGLmzzGuOxeG8Z8LVJ4Vw+W2zXhvhDLq+k2OZk+b4x VXhQ== X-Gm-Message-State: AOAM533C/jhf0Cewvgn1kFp5RVc02cgi4RWQtCkAsEQkUP5Rd6ZXwDTG NmJOkhCbQd07escTYzjEKM5FKQ== X-Google-Smtp-Source: ABdhPJzzu8w3H0zHy13NXlQtb/DWOK/nCH71MNq45VhKumCJJVIQ8gvuKohqxVj5jKeuYtcbVRZ2Vg== X-Received: by 2002:a1c:2045:: with SMTP id g66mr452060wmg.184.1595436313169; Wed, 22 Jul 2020 09:45:13 -0700 (PDT) Received: from localhost ([2a01:4b00:8523:2d03:b0ee:900a:e004:b9d0]) by smtp.gmail.com with ESMTPSA id h5sm521647wrc.97.2020.07.22.09.45.12 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 22 Jul 2020 09:45:12 -0700 (PDT) From: David Brazdil To: Catalin Marinas , Will Deacon , Marc Zyngier , Dennis Zhou , Tejun Heo , Christoph Lameter , Arnd Bergmann Subject: [PATCH 8/9] kvm: arm64: Set up hyp percpu data for nVHE Date: Wed, 22 Jul 2020 17:44:23 +0100 Message-Id: <20200722164424.42225-9-dbrazdil@google.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200722164424.42225-1-dbrazdil@google.com> References: <20200722164424.42225-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200722_124514_498094_42AD61A9 X-CRM114-Status: GOOD ( 21.16 ) X-Spam-Score: -15.7 (---------------) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (-15.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2a00:1450:4864:20:0:0:0:342 listed in] [list.dnswl.org] -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list -7.5 USER_IN_DEF_SPF_WL From: address is in the default SPF white-list 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.5 ENV_AND_HDR_SPF_MATCH Env and Hdr From used in default SPF WL Match -0.0 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-arch@vger.kernel.org, Suzuki K Poulose , linux-kernel@vger.kernel.org, kernel-team@google.com, James Morse , Julien Thierry , David Brazdil , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org Add hyp percpu section to linker script and rename the corresponding ELF sections of hyp/nvhe object files. This moves all nVHE-specific percpu variables to the new hyp percpu section. Allocate sufficient amount of memory for all percpu hyp regions at global KVM init time, and create corresponding hyp mappings. The base addresses of hyp percpu regions are kept in a dynamically allocated array in the kernel. Add NULL checks in PMU event-reset code as it may run before KVM memory is initialized. Signed-off-by: David Brazdil --- arch/arm64/include/asm/kvm_asm.h | 19 +++++++++-- arch/arm64/include/asm/sections.h | 1 + arch/arm64/kernel/vmlinux.lds.S | 10 ++++++ arch/arm64/kvm/arm.c | 55 +++++++++++++++++++++++++++++-- arch/arm64/kvm/hyp/nvhe/Makefile | 2 ++ arch/arm64/kvm/pmu.c | 5 ++- 6 files changed, 86 insertions(+), 6 deletions(-) diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h index 3d69cab873e4..22e63e651702 100644 --- a/arch/arm64/include/asm/kvm_asm.h +++ b/arch/arm64/include/asm/kvm_asm.h @@ -72,8 +72,23 @@ #define CHOOSE_VHE_SYM(sym) sym #define CHOOSE_NVHE_SYM(sym) kvm_nvhe_sym(sym) -#define this_cpu_ptr_nvhe(sym) this_cpu_ptr(&kvm_nvhe_sym(sym)) -#define per_cpu_ptr_nvhe(sym, cpu) per_cpu_ptr(&kvm_nvhe_sym(sym), cpu) +/* Array of percpu base addresses. Length of the array is nr_cpu_ids. */ +extern unsigned long *kvm_arm_hyp_percpu_base; + +/* + * Compute pointer to a symbol defined in nVHE percpu region. + * Returns NULL if percpu memory has not been allocated yet. + */ +#define this_cpu_ptr_nvhe(sym) per_cpu_ptr_nvhe(sym, smp_processor_id()) +#define per_cpu_ptr_nvhe(sym, cpu) \ + ({ \ + unsigned long base, off; \ + base = kvm_arm_hyp_percpu_base \ + ? kvm_arm_hyp_percpu_base[cpu] : 0; \ + off = (unsigned long)&kvm_nvhe_sym(sym) - \ + (unsigned long)&kvm_nvhe_sym(__per_cpu_start); \ + base ? (typeof(kvm_nvhe_sym(sym))*)(base + off) : NULL; \ + }) #ifndef __KVM_NVHE_HYPERVISOR__ /* diff --git a/arch/arm64/include/asm/sections.h b/arch/arm64/include/asm/sections.h index 3994169985ef..5062553a6847 100644 --- a/arch/arm64/include/asm/sections.h +++ b/arch/arm64/include/asm/sections.h @@ -18,5 +18,6 @@ extern char __exittext_begin[], __exittext_end[]; extern char __irqentry_text_start[], __irqentry_text_end[]; extern char __mmuoff_data_start[], __mmuoff_data_end[]; extern char __entry_tramp_text_start[], __entry_tramp_text_end[]; +extern char __kvm_nvhe___per_cpu_start[], __kvm_nvhe___per_cpu_end[]; #endif /* __ASM_SECTIONS_H */ diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index 6827da7f3aa5..c678615ccd6d 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -16,6 +16,9 @@ #include "image.h" +#define __CONCAT3(x, y, z) x ## y ## z +#define CONCAT3(x, y, z) __CONCAT3(x, y, z) + OUTPUT_ARCH(aarch64) ENTRY(_text) @@ -187,6 +190,13 @@ SECTIONS PERCPU_SECTION(L1_CACHE_BYTES) + /* KVM nVHE per-cpu section */ + #undef PERCPU_SECTION_NAME + #undef PERCPU_SYMBOL_NAME + #define PERCPU_SECTION_NAME(suffix) CONCAT3(.hyp, PERCPU_SECTION_BASE_NAME, suffix) + #define PERCPU_SYMBOL_NAME(name) __kvm_nvhe_ ## name + PERCPU_SECTION(L1_CACHE_BYTES) + .rela.dyn : ALIGN(8) { *(.rela .rela*) } diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index dc557b380c87..bbbc5c1519a9 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -47,6 +47,7 @@ __asm__(".arch_extension virt"); #endif static DEFINE_PER_CPU(unsigned long, kvm_arm_hyp_stack_page); +unsigned long *kvm_arm_hyp_percpu_base; /* The VMID used in the VTTBR */ static atomic64_t kvm_vmid_gen = ATOMIC64_INIT(1); @@ -1253,6 +1254,15 @@ long kvm_arch_vm_ioctl(struct file *filp, } } +#define kvm_hyp_percpu_base(cpu) ((unsigned long)per_cpu_ptr_nvhe(__per_cpu_start, cpu)) +#define kvm_hyp_percpu_array_size (nr_cpu_ids * sizeof(*kvm_arm_hyp_percpu_base)) +#define kvm_hyp_percpu_array_order (get_order(kvm_hyp_percpu_array_size)) +#define kvm_hyp_percpu_begin CHOOSE_NVHE_SYM(__per_cpu_start) +#define kvm_hyp_percpu_size ((unsigned long)CHOOSE_NVHE_SYM(__per_cpu_end) - \ + (unsigned long)CHOOSE_NVHE_SYM(__per_cpu_start)) +#define kvm_hyp_percpu_order (kvm_hyp_percpu_size \ + ? get_order(kvm_hyp_percpu_size) : 0) + static void cpu_init_hyp_mode(void) { phys_addr_t pgd_ptr; @@ -1268,8 +1278,8 @@ static void cpu_init_hyp_mode(void) * kernel's mapping to the linear mapping, and store it in tpidr_el2 * so that we can use adr_l to access per-cpu variables in EL2. */ - tpidr_el2 = ((unsigned long)this_cpu_ptr(&kvm_host_data) - - (unsigned long)kvm_ksym_ref(&kvm_host_data)); + tpidr_el2 = (unsigned long)this_cpu_ptr_nvhe(__per_cpu_start) - + (unsigned long)kvm_ksym_ref(kvm_hyp_percpu_begin); pgd_ptr = kvm_mmu_get_httbr(); hyp_stack_ptr = __this_cpu_read(kvm_arm_hyp_stack_page) + PAGE_SIZE; @@ -1501,8 +1511,11 @@ static void teardown_hyp_mode(void) int cpu; free_hyp_pgds(); - for_each_possible_cpu(cpu) + for_each_possible_cpu(cpu) { free_hyp_page(per_cpu(kvm_arm_hyp_stack_page, cpu)); + free_hyp_pages(kvm_hyp_percpu_base(cpu), kvm_hyp_percpu_order); + } + free_hyp_pages((unsigned long)kvm_arm_hyp_percpu_base, kvm_hyp_percpu_array_order); } /** @@ -1535,6 +1548,28 @@ static int init_hyp_mode(void) per_cpu(kvm_arm_hyp_stack_page, cpu) = stack_page; } + /* + * Allocate and initialize pages for Hypervisor-mode percpu regions. + */ + kvm_arm_hyp_percpu_base = (unsigned long*)alloc_hyp_pages( + GFP_KERNEL | __GFP_ZERO, kvm_hyp_percpu_array_order); + if (!kvm_arm_hyp_percpu_base) { + err = -ENOMEM; + goto out_err; + } + for_each_possible_cpu(cpu) { + unsigned long percpu_base; + + percpu_base = alloc_hyp_pages(GFP_KERNEL, kvm_hyp_percpu_order); + if (!percpu_base) { + err = -ENOMEM; + goto out_err; + } + + memcpy((void*)percpu_base, kvm_hyp_percpu_begin, kvm_hyp_percpu_size); + kvm_arm_hyp_percpu_base[cpu] = percpu_base; + } + /* * Map the Hyp-code called directly from the host */ @@ -1579,6 +1614,20 @@ static int init_hyp_mode(void) } } + /* + * Map Hyp percpu pages + */ + for_each_possible_cpu(cpu) { + char *percpu_begin = (char *)kvm_arm_hyp_percpu_base[cpu]; + char *percpu_end = percpu_begin + PAGE_ALIGN(kvm_hyp_percpu_size); + err = create_hyp_mappings(percpu_begin, percpu_end, PAGE_HYP); + + if (err) { + kvm_err("Cannot map hyp percpu region\n"); + goto out_err; + } + } + for_each_possible_cpu(cpu) { kvm_host_data_t *cpu_data; diff --git a/arch/arm64/kvm/hyp/nvhe/Makefile b/arch/arm64/kvm/hyp/nvhe/Makefile index 0b34414557d6..1d415698e60e 100644 --- a/arch/arm64/kvm/hyp/nvhe/Makefile +++ b/arch/arm64/kvm/hyp/nvhe/Makefile @@ -23,6 +23,8 @@ $(obj)/%.hyp.o: $(obj)/%.hyp.tmp.o FORCE quiet_cmd_hypcopy = HYPCOPY $@ cmd_hypcopy = $(OBJCOPY) --prefix-symbols=__kvm_nvhe_ \ --rename-section=.text=.hyp.text \ + --rename-section=.data..percpu=.hyp.data..percpu \ + --rename-section=.data..percpu..read_mostly=.hyp.data..percpu..read_mostly \ $< $@ # Remove ftrace and Shadow Call Stack CFLAGS. diff --git a/arch/arm64/kvm/pmu.c b/arch/arm64/kvm/pmu.c index 717941910723..b4c8302f0798 100644 --- a/arch/arm64/kvm/pmu.c +++ b/arch/arm64/kvm/pmu.c @@ -33,7 +33,7 @@ void kvm_set_pmu_events(u32 set, struct perf_event_attr *attr) { struct kvm_host_data *ctx = this_cpu_ptr_hyp(kvm_host_data); - if (!kvm_pmu_switch_needed(attr)) + if (!ctx || !kvm_pmu_switch_needed(attr)) return; if (!attr->exclude_host) @@ -49,6 +49,9 @@ void kvm_clr_pmu_events(u32 clr) { struct kvm_host_data *ctx = this_cpu_ptr_hyp(kvm_host_data); + if (!ctx) + return; + ctx->pmu_events.events_host &= ~clr; ctx->pmu_events.events_guest &= ~clr; } From patchwork Wed Jul 22 16:44:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11678945 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 64D8B138A for ; Wed, 22 Jul 2020 16:47:45 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3D2AB207CD for ; Wed, 22 Jul 2020 16:47:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="xdAJ15Pr"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="EHDURTcB" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3D2AB207CD Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=/OpAjPzwD3KKwmZY5LenazGFkJFjYRMc6ZsYEINh3O4=; b=xdAJ15Prtmd+XN4QLTfduFbMr yea3+77ssRJqps9EXmxYHGbFiNHZ/IWvLTrCl4oSuy1N0XRKGnZv9eFwl0oid+x0hd5WzPaqG/3QL oRrMA7h5cMNWGyRiKu8F2n9ar7CbCUfX/U7NTQ5L7AK5h6fX0o02YesByDuAszqsRSK2y5r9axZn0 3f2b50lwAuMnJC7qd+3JwvbQQEFccCu23vhKFYdmFahtKXvslSahvLn6TH0dYGSYADpVp9DqL9JS8 iF/JLu6gX/IRGEou6wIeN+vsSscvzpMUFkI/TDvm8H6CqEl/JVU+wqBemS08tuz/1k8/XlwV53ouG +piV8ppcw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jyHsn-0002JK-Dj; Wed, 22 Jul 2020 16:46:17 +0000 Received: from mail-wr1-x444.google.com ([2a00:1450:4864:20::444]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jyHro-0001xE-2d for linux-arm-kernel@lists.infradead.org; Wed, 22 Jul 2020 16:45:17 +0000 Received: by mail-wr1-x444.google.com with SMTP id s10so2533985wrw.12 for ; Wed, 22 Jul 2020 09:45:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0Zoc4jT5D/PnpX6Ge9Fh8+yTlt5xAHFadpjWGBF1q0w=; b=EHDURTcBkVhJJVi3ujf/kQeTfFBT7h/B9rxoO/V6rVCgVwVesI6SzZIa06WBR4XrFl EEc9WcBPE7DBmvVrqXAG2TZ6AgH9fDPkjcr/cpkZ3YeIKVPt1DBo+pvWJi6VB3G/JmeX fSgzbviAYvO/KtVcj8dE8gdhqC3wxBnOf4JlONMDO93pAlrrfS5v4nv9Avm7ELmCflWd 3AltLvlQCDsboajAG8AeCOm8lC6ewgH11gH6xtNGdjO5UFpz3uUW8GRmqYaHIa02jYdR vH1wWLWsSKtXiU7d9T/LJrWMnSBWw2shzP30T5TVW0Z4QbYHatf6n/Uz3oyeSyz0QFKE 7YXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0Zoc4jT5D/PnpX6Ge9Fh8+yTlt5xAHFadpjWGBF1q0w=; b=LQBHpct+nZnv+h2lU9RNtaZH4oqMI9d999GBhrGizCQRXCmnF1KxL1oybGXp1y9w1M qgm/0010AJWxTXcSS/I/GN3iIroWxImQJOPm/XhGOuMB31Qg2qv65Kjr7uYlFsD+I2nr xYK8HL4VYw8+r+KIRdfkOG//HVhS+IyiMNqzNCTvIQlcqkfM9PR3N4TfEnK7yZDypMJD iBRrNaZ2h7cVQ/xuHS1aXGqXMqho7vB7kq2bRoDIHS3r2GX+TjlXE0mdCKR98kE03r0/ rvb5fqzfylnl9YqGKH/0FprCv6vIYtDTnlRAetP1sKNp8qGoZNya7F2Pjk6NVgl2Tjvj dfXA== X-Gm-Message-State: AOAM533jOSPozGZgxOL7pzJCEQt5kW808gWRJI/oe6a88NybI6+pKa/q OUghwFnPTxZokklCjjsnTAS8Fw== X-Google-Smtp-Source: ABdhPJzLKmKG9pub1HFuNdCsYrh8zqcSQp68qHq5cvDc4U/7vFwUWtKwLOPhPUQibgTF3FLak/t0vg== X-Received: by 2002:a5d:548f:: with SMTP id h15mr365969wrv.331.1595436314869; Wed, 22 Jul 2020 09:45:14 -0700 (PDT) Received: from localhost ([2a01:4b00:8523:2d03:b0ee:900a:e004:b9d0]) by smtp.gmail.com with ESMTPSA id t2sm242165wma.43.2020.07.22.09.45.13 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 22 Jul 2020 09:45:14 -0700 (PDT) From: David Brazdil To: Catalin Marinas , Will Deacon , Marc Zyngier , Dennis Zhou , Tejun Heo , Christoph Lameter , Arnd Bergmann Subject: [PATCH 9/9] kvm: arm64: Remove unnecessary hyp mappings Date: Wed, 22 Jul 2020 17:44:24 +0100 Message-Id: <20200722164424.42225-10-dbrazdil@google.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200722164424.42225-1-dbrazdil@google.com> References: <20200722164424.42225-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200722_124516_218944_09B3A0E4 X-CRM114-Status: GOOD ( 15.39 ) X-Spam-Score: -15.7 (---------------) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (-15.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2a00:1450:4864:20:0:0:0:444 listed in] [list.dnswl.org] -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list -7.5 USER_IN_DEF_SPF_WL From: address is in the default SPF white-list 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.5 ENV_AND_HDR_SPF_MATCH Env and Hdr From used in default SPF WL Match -0.0 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-arch@vger.kernel.org, Suzuki K Poulose , linux-kernel@vger.kernel.org, kernel-team@google.com, James Morse , Julien Thierry , David Brazdil , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org With all nVHE per-CPU variables being part of the hyp per-CPU region, mapping them individual is not necessary any longer. They are mapped to hyp as part of the overall per-CPU region. Signed-off-by: David Brazdil --- arch/arm64/include/asm/kvm_mmu.h | 25 +++++++------------------ arch/arm64/kvm/arm.c | 17 +---------------- 2 files changed, 8 insertions(+), 34 deletions(-) diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h index e9e5875274cb..1a66089cf4f4 100644 --- a/arch/arm64/include/asm/kvm_mmu.h +++ b/arch/arm64/include/asm/kvm_mmu.h @@ -531,28 +531,17 @@ static inline int kvm_map_vectors(void) DECLARE_PER_CPU_READ_MOSTLY(u64, arm64_ssbd_callback_required); DECLARE_KVM_NVHE_PER_CPU(u64, arm64_ssbd_callback_required); -static inline int hyp_init_aux_data(void) +static inline void hyp_init_aux_data(void) { - int cpu, err; + int cpu; - for_each_possible_cpu(cpu) { - u64 *ptr; - - ptr = per_cpu_ptr_nvhe(arm64_ssbd_callback_required, cpu); - err = create_hyp_mappings(ptr, ptr + 1, PAGE_HYP); - if (err) - return err; - - /* Copy value from kernel to hyp. */ - *ptr = per_cpu(arm64_ssbd_callback_required, cpu); - } - return 0; + /* Copy arm64_ssbd_callback_required values from kernel to hyp. */ + for_each_possible_cpu(cpu) + *(per_cpu_ptr_nvhe(arm64_ssbd_callback_required, cpu)) = + per_cpu(arm64_ssbd_callback_required, cpu); } #else -static inline int hyp_init_aux_data(void) -{ - return 0; -} +static inline void hyp_init_aux_data(void) {} #endif #define kvm_phys_to_vttbr(addr) phys_to_ttbr(addr) diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index bbbc5c1519a9..f2e537d99d2b 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1628,22 +1628,7 @@ static int init_hyp_mode(void) } } - for_each_possible_cpu(cpu) { - kvm_host_data_t *cpu_data; - - cpu_data = per_cpu_ptr_hyp(kvm_host_data, cpu); - err = create_hyp_mappings(cpu_data, cpu_data + 1, PAGE_HYP); - - if (err) { - kvm_err("Cannot map host CPU state: %d\n", err); - goto out_err; - } - } - - err = hyp_init_aux_data(); - if (err) - kvm_err("Cannot map host auxiliary data: %d\n", err); - + hyp_init_aux_data(); return 0; out_err: