From patchwork Thu Apr 8 04:05:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12189931 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 09844C433ED for ; Thu, 8 Apr 2021 04:05:43 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7BB97611BE for ; Thu, 8 Apr 2021 04:05:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7BB97611BE Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 10EF56B0080; Thu, 8 Apr 2021 00:05:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 098A86B0081; Thu, 8 Apr 2021 00:05:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E37CB6B0082; Thu, 8 Apr 2021 00:05:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0127.hostedemail.com [216.40.44.127]) by kanga.kvack.org (Postfix) with ESMTP id C65F46B0080 for ; Thu, 8 Apr 2021 00:05:41 -0400 (EDT) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 83ACD181BA3E0 for ; Thu, 8 Apr 2021 04:05:41 +0000 (UTC) X-FDA: 78007860882.18.2E87130 Received: from mail-qk1-f172.google.com (mail-qk1-f172.google.com [209.85.222.172]) by imf04.hostedemail.com (Postfix) with ESMTP id E4B91130 for ; Thu, 8 Apr 2021 04:05:39 +0000 (UTC) Received: by mail-qk1-f172.google.com with SMTP id g15so942999qkl.4 for ; Wed, 07 Apr 2021 21:05:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=2ZHo6bqqoFQHDt1XsnxWzUIHllyauSrjIJToDTBiZLY=; b=X9eAU/eM3040QGUYmUbw7fYBZz3JtF3avBuvT24i1+XpVGvbIi9BNTH2rSlvhxMZ6p vB7bmo+61os1iTwbbNQRR69s0xibH9y2Zh+JxqbT2cEzOMZyXY4XleOC3VU2OQbelf4e qUx64StBUbF+O1txg2vRkmFpmHWn5MioQBG/fANNQ2YzUxTJC8uDaJ4j6/ABFGO80TmL lT7jf5YqjgaPI2Fa4vi35ToQnOb1K/7GhDLWSoB81CHHWcprY9EhPAQcUKCUF69eeTrQ 5rcxFBkWQYTv7dEA+PHoUkVJSA2Bu07e+E9jh6dpYWIsqXcT3kY8zV4f1HpZcTG33b+V 3i2A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2ZHo6bqqoFQHDt1XsnxWzUIHllyauSrjIJToDTBiZLY=; b=h+B8OabvQIFQzjcnjw7XlkPGULtOc3BdndRapayXoJUVgY1SXR+xQbZKTgi4Z85DSa OpTp5F96Tn+65Cug+w8sWm/bGRZpwoxcgjI+9JdUnljUqoMOvut0iXUwnwqxmTQZxmxs lYtEHC9jq9p0u7BYalna9Zos0YFbqRQit0/KBrTNLXH7fgDarvpS4Bto33opeTzXbEb0 FhqKw7N49k0mwwb7fPjROm+NloI2j0EWfHQ7FgHaWYR8JAq1xdW7ht1Asf2AjMgl4kSK kkSepSjEWMqWcu7WOFUdvSzv44sqxzHxwar1Tl52VwQqXhOdNrzPkR5qI5h3/tEUWPqy BQUw== X-Gm-Message-State: AOAM5307df79CeKyFZ1cUh8CaNttl2hflHE4r2GbHX2fdnBr8F1GiQc/ XuylGPI3O2D36kXLcvgzNN7cAA== X-Google-Smtp-Source: ABdhPJxYXvk2FxZ0FX2jOzjDLvti4P2RgEhIeGGR2Vwo3ThVtxJOO1wAhCh/APii2j5zwyiwpg/lrg== X-Received: by 2002:a37:78b:: with SMTP id 133mr6607278qkh.109.1617854740543; Wed, 07 Apr 2021 21:05:40 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id 207sm18177874qkl.125.2021.04.07.21.05.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Apr 2021 21:05:39 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com Subject: [PATCH v13 01/18] arm64: hyp-stub: Check the size of the HYP stub's vectors Date: Thu, 8 Apr 2021 00:05:20 -0400 Message-Id: <20210408040537.2703241-2-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210408040537.2703241-1-pasha.tatashin@soleen.com> References: <20210408040537.2703241-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: E4B91130 X-Stat-Signature: fmj5g1m4p6jjgnphj4hyuc3ougn58km4 Received-SPF: none (soleen.com>: No applicable sender policy available) receiver=imf04; identity=mailfrom; envelope-from=""; helo=mail-qk1-f172.google.com; client-ip=209.85.222.172 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1617854739-832172 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: James Morse Hibernate contains a set of temporary EL2 vectors used to 'park' EL2 somewhere safe while all the memory is thrown in the air. Making kexec do its relocations with the MMU on means they have to be done at EL1, so EL2 has to be parked. This means yet another set of vectors. All these things do is HVC_SET_VECTORS and HVC_SOFT_RESTART, both of which are implemented by the hyp-stub. Lets copy it instead of re-inventing it. To do this the hyp-stub's entrails need to be packed neatly inside its 2K vectors. Start by moving the final 2K alignment inside the end marker, and add a build check that we didn't overflow 2K. Signed-off-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/hyp-stub.S | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kernel/hyp-stub.S b/arch/arm64/kernel/hyp-stub.S index 5eccbd62fec8..572b28646005 100644 --- a/arch/arm64/kernel/hyp-stub.S +++ b/arch/arm64/kernel/hyp-stub.S @@ -41,9 +41,13 @@ SYM_CODE_START(__hyp_stub_vectors) ventry el1_irq_invalid // IRQ 32-bit EL1 ventry el1_fiq_invalid // FIQ 32-bit EL1 ventry el1_error_invalid // Error 32-bit EL1 + .align 11 +SYM_INNER_LABEL(__hyp_stub_vectors_end, SYM_L_LOCAL) SYM_CODE_END(__hyp_stub_vectors) - .align 11 +# Check the __hyp_stub_vectors didn't overflow +.org . - (__hyp_stub_vectors_end - __hyp_stub_vectors) + SZ_2K + SYM_CODE_START_LOCAL(el1_sync) cmp x0, #HVC_SET_VECTORS From patchwork Thu Apr 8 04:05:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12189933 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56156C43460 for ; Thu, 8 Apr 2021 04:05:44 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C1BC061157 for ; Thu, 8 Apr 2021 04:05:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C1BC061157 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5F1286B0081; Thu, 8 Apr 2021 00:05:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 57CAA6B0082; Thu, 8 Apr 2021 00:05:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2E5E86B0083; Thu, 8 Apr 2021 00:05:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0122.hostedemail.com [216.40.44.122]) by kanga.kvack.org (Postfix) with ESMTP id 0E0E26B0081 for ; Thu, 8 Apr 2021 00:05:43 -0400 (EDT) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id CB6368794 for ; Thu, 8 Apr 2021 04:05:42 +0000 (UTC) X-FDA: 78007860924.05.A59E748 Received: from mail-qk1-f180.google.com (mail-qk1-f180.google.com [209.85.222.180]) by imf18.hostedemail.com (Postfix) with ESMTP id 7DE2420001D6 for ; Thu, 8 Apr 2021 04:05:42 +0000 (UTC) Received: by mail-qk1-f180.google.com with SMTP id g20so960753qkk.1 for ; Wed, 07 Apr 2021 21:05:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=Me+IJOmpVZMJE6OZITjjNJXxLybVS0PvpkZtNmZgu9s=; b=HrdrsH+/LJvmodXx6qJUKjxjbprurxBWQX7rF7YH/tot181AndUWuV7gbnA0UKrCI7 P33v6LFzUMT/CxhdJRYj39gFLLn8Kl24eYKMWwa5ryYEp8ZR56p76Dx73+juO2TS5fmp 03vZh8F/ou36y+EHeihspp1zs5O7FTxECJIbzCqeYzBohoD8eVUWtj5V6ci9q3xx7aO5 65ZjTbwF8/uVodijBWQC1fYHP0gWkn7GLjbIKlV1CsDmRzTQV9JCjReAOMPviz5wlbsO gGMCoefM2xm2+wWIHCBHlGKMERalPbcFc8KYZkj4p3rp0rYMbNOKjl4AAZ7ZMZnf7Vwn dEWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Me+IJOmpVZMJE6OZITjjNJXxLybVS0PvpkZtNmZgu9s=; b=hB3CKkeBtnJuJOu4RSTHLc3leOUVz7UIx7nlcWEtrEjx0w1UtGW42+ElkFG+cE62Sl 6pm10OE9v/3Dm5wR+tKlL3o9/zUIOmAijZgvNTjbVNXbTRSU68rrRNJknRkamYeFNrHk CEXT/RCkdICRldE18+oCWtdp6lJM2iqDxs4BCOSnEzFumkLSyDooTvR7Dm5khHvGa1mu 2jJvGPznSbOjgSVSPmW3jQx472Gy8SKQjrn2jyoct+P5KtxeQZH0OinC5ZZ+9YilUzmf 84oZQXh6FN5Cf1l19okdYhStUh6oiFlmqnhcKyNcNZvmDk+vEMTcNhBZ3sA/Dies+uCL 3gKA== X-Gm-Message-State: AOAM532KVmWXhHL33ysA5rFw4gqeXqgwAXBI7odiTZfPwWCni5TejtHR wibTyp1wIzriJO/emX+gdSbMpQ== X-Google-Smtp-Source: ABdhPJxYHvA6IkQyz8orCAuu/6NxJtk1GKdC6ouuVl2yzdNWDbtFSj8FFUMuYUgnnPsyTxVVAcEuTw== X-Received: by 2002:a05:620a:10a6:: with SMTP id h6mr6649236qkk.366.1617854741843; Wed, 07 Apr 2021 21:05:41 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id 207sm18177874qkl.125.2021.04.07.21.05.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Apr 2021 21:05:41 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com Subject: [PATCH v13 02/18] arm64: hyp-stub: Move invalid vector entries into the vectors Date: Thu, 8 Apr 2021 00:05:21 -0400 Message-Id: <20210408040537.2703241-3-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210408040537.2703241-1-pasha.tatashin@soleen.com> References: <20210408040537.2703241-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Stat-Signature: akony6fpcwd55gjgrif3royczcfucqaq X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 7DE2420001D6 Received-SPF: none (soleen.com>: No applicable sender policy available) receiver=imf18; identity=mailfrom; envelope-from=""; helo=mail-qk1-f180.google.com; client-ip=209.85.222.180 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1617854742-226550 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: James Morse Most of the hyp-stub's vector entries are invalid. These are each a unique function that branches to itself. To move these into the vectors, merge the ventry and invalid_vector macros and give each one a unique name. This means we can copy the hyp-stub as it is self contained within its vectors. Signed-off-by: James Morse [Fixed merging issues] Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/hyp-stub.S | 56 +++++++++++++++--------------------- 1 file changed, 23 insertions(+), 33 deletions(-) diff --git a/arch/arm64/kernel/hyp-stub.S b/arch/arm64/kernel/hyp-stub.S index 572b28646005..ff329c5c074d 100644 --- a/arch/arm64/kernel/hyp-stub.S +++ b/arch/arm64/kernel/hyp-stub.S @@ -16,31 +16,38 @@ #include #include +.macro invalid_vector label +SYM_CODE_START_LOCAL(\label) + .align 7 + b \label +SYM_CODE_END(\label) +.endm + .text .pushsection .hyp.text, "ax" .align 11 SYM_CODE_START(__hyp_stub_vectors) - ventry el2_sync_invalid // Synchronous EL2t - ventry el2_irq_invalid // IRQ EL2t - ventry el2_fiq_invalid // FIQ EL2t - ventry el2_error_invalid // Error EL2t + invalid_vector hyp_stub_el2t_sync_invalid // Synchronous EL2t + invalid_vector hyp_stub_el2t_irq_invalid // IRQ EL2t + invalid_vector hyp_stub_el2t_fiq_invalid // FIQ EL2t + invalid_vector hyp_stub_el2t_error_invalid // Error EL2t - ventry el2_sync_invalid // Synchronous EL2h - ventry el2_irq_invalid // IRQ EL2h - ventry el2_fiq_invalid // FIQ EL2h - ventry el2_error_invalid // Error EL2h + invalid_vector hyp_stub_el2h_sync_invalid // Synchronous EL2h + invalid_vector hyp_stub_el2h_irq_invalid // IRQ EL2h + invalid_vector hyp_stub_el2h_fiq_invalid // FIQ EL2h + invalid_vector hyp_stub_el2h_error_invalid // Error EL2h ventry el1_sync // Synchronous 64-bit EL1 - ventry el1_irq_invalid // IRQ 64-bit EL1 - ventry el1_fiq_invalid // FIQ 64-bit EL1 - ventry el1_error_invalid // Error 64-bit EL1 - - ventry el1_sync_invalid // Synchronous 32-bit EL1 - ventry el1_irq_invalid // IRQ 32-bit EL1 - ventry el1_fiq_invalid // FIQ 32-bit EL1 - ventry el1_error_invalid // Error 32-bit EL1 + invalid_vector hyp_stub_el1_irq_invalid // IRQ 64-bit EL1 + invalid_vector hyp_stub_el1_fiq_invalid // FIQ 64-bit EL1 + invalid_vector hyp_stub_el1_error_invalid // Error 64-bit EL1 + + invalid_vector hyp_stub_32b_el1_sync_invalid // Synchronous 32-bit EL1 + invalid_vector hyp_stub_32b_el1_irq_invalid // IRQ 32-bit EL1 + invalid_vector hyp_stub_32b_el1_fiq_invalid // FIQ 32-bit EL1 + invalid_vector hyp_stub_32b_el1_error_invalid // Error 32-bit EL1 .align 11 SYM_INNER_LABEL(__hyp_stub_vectors_end, SYM_L_LOCAL) SYM_CODE_END(__hyp_stub_vectors) @@ -173,23 +180,6 @@ SYM_CODE_END(enter_vhe) .popsection -.macro invalid_vector label -SYM_CODE_START_LOCAL(\label) - b \label -SYM_CODE_END(\label) -.endm - - invalid_vector el2_sync_invalid - invalid_vector el2_irq_invalid - invalid_vector el2_fiq_invalid - invalid_vector el2_error_invalid - invalid_vector el1_sync_invalid - invalid_vector el1_irq_invalid - invalid_vector el1_fiq_invalid - invalid_vector el1_error_invalid - - .popsection - /* * __hyp_set_vectors: Call this after boot to set the initial hypervisor * vectors as part of hypervisor installation. On an SMP system, this should From patchwork Thu Apr 8 04:05:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12189935 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1A778C43461 for ; Thu, 8 Apr 2021 04:05:46 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B3806611C2 for ; Thu, 8 Apr 2021 04:05:45 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B3806611C2 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id AD4736B0082; Thu, 8 Apr 2021 00:05:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A86406B0083; Thu, 8 Apr 2021 00:05:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7ECCC6B0085; Thu, 8 Apr 2021 00:05:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0152.hostedemail.com [216.40.44.152]) by kanga.kvack.org (Postfix) with ESMTP id 5E9AE6B0082 for ; Thu, 8 Apr 2021 00:05:44 -0400 (EDT) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 2495B4DCA for ; Thu, 8 Apr 2021 04:05:44 +0000 (UTC) X-FDA: 78007861008.13.516DD96 Received: from mail-qv1-f46.google.com (mail-qv1-f46.google.com [209.85.219.46]) by imf25.hostedemail.com (Postfix) with ESMTP id 274CE600010B for ; Thu, 8 Apr 2021 04:05:42 +0000 (UTC) Received: by mail-qv1-f46.google.com with SMTP id j3so268458qvs.1 for ; Wed, 07 Apr 2021 21:05:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=ys4BurbzE8f3xOtcZckwDgg+eZ/2ibCIBaYK/PrLc5I=; b=juqdJ1+rPty+mZcJWzuSbk/vYifDHS3s3mojAmWOO+G3qlz7jgNSEEr4f0hG/M5Jpz aw5IKBcea62xB54/m6w+NyeqowI5FwwiwHyTaw6lq6VK5IRdOjzT1vIcWu1vBaBc9Dks LEMgkcImNq1vT7jsoJmB0yPzqkmdvjISYyHKlIglDEXSTmTN+bl3hCkpinJlw9owJSjZ Pt6Guc7V+JIgF6rOLn73KQv2wGXWNbNlCqzxYe7k45r8vnwcZ6SJRPIUBV6qpZjfCqPo bRG1IhRT2MZYsMPCE9cDpsb9zqKsMxgemCJo0vnGi5U5P+iz5i0DiYjYyDasJIOjkdUI XXOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ys4BurbzE8f3xOtcZckwDgg+eZ/2ibCIBaYK/PrLc5I=; b=T9Jw2c7CglHNtV84vYnyNI0K9777sWjiGkj3eLxh7gmebfsm7QV0R12DT0ek/m3Skl MDIl3guuNr3BG/kySk5AjQ5gaEbt0QpdpLvdY4blmKAXotVuljT+Q+BIN/+RKwPJCMpx XXrntMjRGvzqXOODJoE6Pg4OBjG7rCZEAa+vAq59KAT6/buBOQ7fgfimdDQXNPmVBGk0 9nEMHyuK0otmp9UlOCA9iV8hIrLAPaxxLSUFo+N9xlSrFSNbFvZmGVVcptSwEIyU9vq2 X1i04vYLOyPOdcbp+3wo6rX0RnfIkX8W72FSrrZGwcSCsYtNNyC+0KXTd9XFBHdoKKNg CTQg== X-Gm-Message-State: AOAM532R3esVNb7A7mcc2UBYYLCUAt4I2jKoGHPVGtiJLi6Zb+xAF/gV OEdS/mVjeTUH7ltiOrriBT8V0A== X-Google-Smtp-Source: ABdhPJwMb53rZZbQdfaU1akeMPvCbZqVXP8rhlkM/XM+uDXitGdEDxjjUj6mbyKR2RxS/SJppBRTmQ== X-Received: by 2002:a0c:9e0f:: with SMTP id p15mr7047700qve.27.1617854743132; Wed, 07 Apr 2021 21:05:43 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id 207sm18177874qkl.125.2021.04.07.21.05.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Apr 2021 21:05:42 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com Subject: [PATCH v13 03/18] arm64: hyp-stub: Move el1_sync into the vectors Date: Thu, 8 Apr 2021 00:05:22 -0400 Message-Id: <20210408040537.2703241-4-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210408040537.2703241-1-pasha.tatashin@soleen.com> References: <20210408040537.2703241-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 274CE600010B X-Stat-Signature: 8nugsaw78ikfrbjifskgow3stbymdm81 Received-SPF: none (soleen.com>: No applicable sender policy available) receiver=imf25; identity=mailfrom; envelope-from=""; helo=mail-qv1-f46.google.com; client-ip=209.85.219.46 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1617854742-384184 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: James Morse The hyp-stub's el1_sync code doesn't do very much, this can easily fit in the vectors. With this, all of the hyp-stubs behaviour is contained in its vectors. This lets kexec and hibernate copy the hyp-stub when they need its behaviour, instead of re-implementing it. Signed-off-by: James Morse [Fixed merging issues] Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/hyp-stub.S | 59 ++++++++++++++++++------------------ 1 file changed, 29 insertions(+), 30 deletions(-) diff --git a/arch/arm64/kernel/hyp-stub.S b/arch/arm64/kernel/hyp-stub.S index ff329c5c074d..d1a73d0f74e0 100644 --- a/arch/arm64/kernel/hyp-stub.S +++ b/arch/arm64/kernel/hyp-stub.S @@ -21,6 +21,34 @@ SYM_CODE_START_LOCAL(\label) .align 7 b \label SYM_CODE_END(\label) +.endm + +.macro hyp_stub_el1_sync +SYM_CODE_START_LOCAL(hyp_stub_el1_sync) + .align 7 + cmp x0, #HVC_SET_VECTORS + b.ne 2f + msr vbar_el2, x1 + b 9f + +2: cmp x0, #HVC_SOFT_RESTART + b.ne 3f + mov x0, x2 + mov x2, x4 + mov x4, x1 + mov x1, x3 + br x4 // no return + +3: cmp x0, #HVC_RESET_VECTORS + beq 9f // Nothing to reset! + + /* Someone called kvm_call_hyp() against the hyp-stub... */ + mov_q x0, HVC_STUB_ERR + eret + +9: mov x0, xzr + eret +SYM_CODE_END(hyp_stub_el1_sync) .endm .text @@ -39,7 +67,7 @@ SYM_CODE_START(__hyp_stub_vectors) invalid_vector hyp_stub_el2h_fiq_invalid // FIQ EL2h invalid_vector hyp_stub_el2h_error_invalid // Error EL2h - ventry el1_sync // Synchronous 64-bit EL1 + hyp_stub_el1_sync // Synchronous 64-bit EL1 invalid_vector hyp_stub_el1_irq_invalid // IRQ 64-bit EL1 invalid_vector hyp_stub_el1_fiq_invalid // FIQ 64-bit EL1 invalid_vector hyp_stub_el1_error_invalid // Error 64-bit EL1 @@ -55,35 +83,6 @@ SYM_CODE_END(__hyp_stub_vectors) # Check the __hyp_stub_vectors didn't overflow .org . - (__hyp_stub_vectors_end - __hyp_stub_vectors) + SZ_2K - -SYM_CODE_START_LOCAL(el1_sync) - cmp x0, #HVC_SET_VECTORS - b.ne 1f - msr vbar_el2, x1 - b 9f - -1: cmp x0, #HVC_VHE_RESTART - b.eq mutate_to_vhe - -2: cmp x0, #HVC_SOFT_RESTART - b.ne 3f - mov x0, x2 - mov x2, x4 - mov x4, x1 - mov x1, x3 - br x4 // no return - -3: cmp x0, #HVC_RESET_VECTORS - beq 9f // Nothing to reset! - - /* Someone called kvm_call_hyp() against the hyp-stub... */ - mov_q x0, HVC_STUB_ERR - eret - -9: mov x0, xzr - eret -SYM_CODE_END(el1_sync) - // nVHE? No way! Give me the real thing! SYM_CODE_START_LOCAL(mutate_to_vhe) // Sanity check: MMU *must* be off From patchwork Thu Apr 8 04:05:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12189937 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3EDFEC433ED for ; Thu, 8 Apr 2021 04:05:48 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A8215611C0 for ; Thu, 8 Apr 2021 04:05:47 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A8215611C0 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 026086B0083; Thu, 8 Apr 2021 00:05:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F1C9B6B0085; Thu, 8 Apr 2021 00:05:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CA7FF6B0087; Thu, 8 Apr 2021 00:05:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0094.hostedemail.com [216.40.44.94]) by kanga.kvack.org (Postfix) with ESMTP id 9EAC56B0083 for ; Thu, 8 Apr 2021 00:05:45 -0400 (EDT) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 63C7A8248047 for ; Thu, 8 Apr 2021 04:05:45 +0000 (UTC) X-FDA: 78007861050.27.33A033A Received: from mail-qk1-f176.google.com (mail-qk1-f176.google.com [209.85.222.176]) by imf08.hostedemail.com (Postfix) with ESMTP id 42FCC80192D4 for ; Thu, 8 Apr 2021 04:05:37 +0000 (UTC) Received: by mail-qk1-f176.google.com with SMTP id x11so901421qkp.11 for ; Wed, 07 Apr 2021 21:05:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=t5r+nQTDiOBjAcARZmxYD2C/XCnn3pF+1dbE/Rdt76g=; b=fNSvTQaCMuUW4WM7DW+/ylfwvq1Nd8GFZI5DXNtogEMS5GfxQgFo8HftVUvgQ2WW+L XjYfVrYBzVm7gymnWJVpRO+ECRA4CD2OPDTfrymbiIxKBrxQUd5XMEqjZ/iypUUsTFIQ 6X50BQCwfYfbVyhawd2DSswD4JiqBLSqjvnv2Rwzd4aXRn5JR/NNKZikJ7Aegbs8viUH jv4Sndd4hZAC5D8ilmQK3HkGGnaU5EB21eHXJMc2E0f2wAioMpgG8cnLIMFcOM1pGRSF OFnz738dO1OxUG42H7ZbsgXsFE1Ly4cb5QLHOxHHiRFqW+fFwulj1I3sI5tzncrV63RB bWNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=t5r+nQTDiOBjAcARZmxYD2C/XCnn3pF+1dbE/Rdt76g=; b=STBLdWO9KFLezgG4r14Of7HwNn2GlswN85cmwp9i7z3d8Ldz843+WJ+Hy9ByAUlrn1 vHHAdVwhKyNKWFXJxiOM8eFB+WCK170Ns5C7Prj91kHk/tkWkmdHguyDukfZfeQU8EKT Z2pIxGxrxNIWPSKInmnsWMf0A//na5NUlKM3n1YpPFLZWVWC/Me4rL72sUg8i2cayihG eKsbynkp3ABGksgJusDWhNJQCgyHf7vBUjgIKNLlZOHlT1WRgZ6vI0GGQVW4hf0KxAax 3OCdlu8XMxk3ibfzzS77yXPlBAWMt7mg72pNs9GcSEciN9wXMxTGVlEM27CatXnvpZiT ZhMA== X-Gm-Message-State: AOAM533G8PRTubnUc4VLSBF3KWw6TTZymkp5LW69/LNlI2vBn9jc/1Qt Yl/5s4HQILC7SEKYQZGHpObEqw== X-Google-Smtp-Source: ABdhPJwdX428kgXB8v1FqFOp3RPgJRd9YNsFO19DyJ2SqHXbzbyljk/FZrQI44bhTtoFuQP0Gs4fUA== X-Received: by 2002:a05:620a:1497:: with SMTP id w23mr6610916qkj.260.1617854744436; Wed, 07 Apr 2021 21:05:44 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id 207sm18177874qkl.125.2021.04.07.21.05.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Apr 2021 21:05:44 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com Subject: [PATCH v13 04/18] arm64: kernel: add helper for booted at EL2 and not VHE Date: Thu, 8 Apr 2021 00:05:23 -0400 Message-Id: <20210408040537.2703241-5-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210408040537.2703241-1-pasha.tatashin@soleen.com> References: <20210408040537.2703241-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Stat-Signature: ew9fdsn9paxgbw6ftk5ghsa4i6hnm15h X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 42FCC80192D4 Received-SPF: none (soleen.com>: No applicable sender policy available) receiver=imf08; identity=mailfrom; envelope-from=""; helo=mail-qk1-f176.google.com; client-ip=209.85.222.176 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1617854737-159986 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Replace places that contain logic like this: is_hyp_mode_available() && !is_kernel_in_hyp_mode() With a dedicated boolean function is_hyp_callable(). This will be needed later in kexec in order to sooner switch back to EL2. Suggested-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/virt.h | 5 +++++ arch/arm64/kernel/cpu-reset.h | 3 +-- arch/arm64/kernel/hibernate.c | 9 +++------ arch/arm64/kernel/sdei.c | 2 +- 4 files changed, 10 insertions(+), 9 deletions(-) diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h index 7379f35ae2c6..4216c8623538 100644 --- a/arch/arm64/include/asm/virt.h +++ b/arch/arm64/include/asm/virt.h @@ -128,6 +128,11 @@ static __always_inline bool is_protected_kvm_enabled(void) return cpus_have_final_cap(ARM64_KVM_PROTECTED_MODE); } +static inline bool is_hyp_callable(void) +{ + return is_hyp_mode_available() && !is_kernel_in_hyp_mode(); +} + #endif /* __ASSEMBLY__ */ #endif /* ! __ASM__VIRT_H */ diff --git a/arch/arm64/kernel/cpu-reset.h b/arch/arm64/kernel/cpu-reset.h index ed50e9587ad8..1922e7a690f8 100644 --- a/arch/arm64/kernel/cpu-reset.h +++ b/arch/arm64/kernel/cpu-reset.h @@ -20,8 +20,7 @@ static inline void __noreturn cpu_soft_restart(unsigned long entry, { typeof(__cpu_soft_restart) *restart; - unsigned long el2_switch = !is_kernel_in_hyp_mode() && - is_hyp_mode_available(); + unsigned long el2_switch = is_hyp_callable(); restart = (void *)__pa_symbol(__cpu_soft_restart); cpu_install_idmap(); diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index b1cef371df2b..c764574a1acb 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -48,9 +48,6 @@ */ extern int in_suspend; -/* Do we need to reset el2? */ -#define el2_reset_needed() (is_hyp_mode_available() && !is_kernel_in_hyp_mode()) - /* temporary el2 vectors in the __hibernate_exit_text section. */ extern char hibernate_el2_vectors[]; @@ -125,7 +122,7 @@ int arch_hibernation_header_save(void *addr, unsigned int max_size) hdr->reenter_kernel = _cpu_resume; /* We can't use __hyp_get_vectors() because kvm may still be loaded */ - if (el2_reset_needed()) + if (is_hyp_callable()) hdr->__hyp_stub_vectors = __pa_symbol(__hyp_stub_vectors); else hdr->__hyp_stub_vectors = 0; @@ -387,7 +384,7 @@ int swsusp_arch_suspend(void) dcache_clean_range(__idmap_text_start, __idmap_text_end); /* Clean kvm setup code to PoC? */ - if (el2_reset_needed()) { + if (is_hyp_callable()) { dcache_clean_range(__hyp_idmap_text_start, __hyp_idmap_text_end); dcache_clean_range(__hyp_text_start, __hyp_text_end); } @@ -482,7 +479,7 @@ int swsusp_arch_resume(void) * * We can skip this step if we booted at EL1, or are running with VHE. */ - if (el2_reset_needed()) { + if (is_hyp_callable()) { phys_addr_t el2_vectors = (phys_addr_t)hibernate_exit; el2_vectors += hibernate_el2_vectors - __hibernate_exit_text_start; /* offset */ diff --git a/arch/arm64/kernel/sdei.c b/arch/arm64/kernel/sdei.c index 2c7ca449dd51..af0ac2f920cf 100644 --- a/arch/arm64/kernel/sdei.c +++ b/arch/arm64/kernel/sdei.c @@ -200,7 +200,7 @@ unsigned long sdei_arch_get_entry_point(int conduit) * dropped to EL1 because we don't support VHE, then we can't support * SDEI. */ - if (is_hyp_mode_available() && !is_kernel_in_hyp_mode()) { + if (is_hyp_callable()) { pr_err("Not supported on this hardware/boot configuration\n"); goto out_err; } From patchwork Thu Apr 8 04:05:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12189939 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 12AB7C43460 for ; Thu, 8 Apr 2021 04:05:50 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B0F6D6120E for ; Thu, 8 Apr 2021 04:05:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B0F6D6120E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6B5146B0085; Thu, 8 Apr 2021 00:05:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5C89F6B0087; Thu, 8 Apr 2021 00:05:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3F6FF6B0088; Thu, 8 Apr 2021 00:05:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0159.hostedemail.com [216.40.44.159]) by kanga.kvack.org (Postfix) with ESMTP id 097F26B0085 for ; Thu, 8 Apr 2021 00:05:47 -0400 (EDT) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id BC4788248047 for ; Thu, 8 Apr 2021 04:05:46 +0000 (UTC) X-FDA: 78007861092.25.49E4C25 Received: from mail-qt1-f181.google.com (mail-qt1-f181.google.com [209.85.160.181]) by imf16.hostedemail.com (Postfix) with ESMTP id DF1F080192D8 for ; Thu, 8 Apr 2021 04:05:45 +0000 (UTC) Received: by mail-qt1-f181.google.com with SMTP id f12so466586qtq.4 for ; Wed, 07 Apr 2021 21:05:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=+EiR776LGRIVUqs8E6nTUaQXNk2yNVD65j0QM2uVzow=; b=myDJXaOpUVpA8LV8lwPukm1qxzV0VccCr/iZiryb0p12CJdK/BjjSAtNz9PVey5TCf 8TB0NuSbtbOYBJrkUo9jVJKavGXOlXWR9hK+QT9YGnRup03aSNMWNC9/+eMqxXc1jNuU 1Mw7X69o6DYWe3DDKL8oxGUnIbe+GnAioeb1FbKTOqKkZFAKP49DVNBua4GaBwFzgFaY lYqWKxE7CY3d8hs0diMbZU15Aj7eerDOX8d+nL9hWgPwPTGVNnoL8Xr/JMDsTCBLi+Mz 1BU8lY4fuTkGmDFXAAuURPRIt+ItCnjtZpaDBZyjhAoSEA5NpMXRTMSPE4o/BGIISO8k CgTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+EiR776LGRIVUqs8E6nTUaQXNk2yNVD65j0QM2uVzow=; b=kqfxHSC4ZkU5odS9F2ZXmI38qpZggYHJaneBNwXWgqzFDfQg2bQczvjcrZntorY0Qd VtuiGFWSdeuEFCfF4sk2Q7E04h2pZHcInz8yz5QXtkPtwSUfsT6ZwVapwVW0/92hoqK3 RUTPxXKawtplyXnoUQMvY0VnisbPsY24F2qglO32lto6MlirPYSVLZWDwMZF2aCWDOh3 QDZ5bi0/6Dup32livRIGDesBRK4qiDu5jyWGGTiDmoWvbrDoXAKZ4FZilS8XoORP4Vpl NuF+ncgVzAnzGzXVDfPjQw8KVOyIW0jzlHuNEhZicH7WS0EzqJrrOicFpUWXYkoSqg19 2EXg== X-Gm-Message-State: AOAM532UWIIe6MQFy55pXouYvqXFQR1fKpf3zDEFJO9kZIehoAQamxUx skqPXc/t+P5aOsM693I7ShsCFQ== X-Google-Smtp-Source: ABdhPJxd8U7/h/RVyIw++phyu/2DejR7jUeXg4mgQQ8MV4AUR6f/v61HdCxcTfAWRPU9DA+Gu7EhSA== X-Received: by 2002:a05:622a:14c:: with SMTP id v12mr5688069qtw.46.1617854745725; Wed, 07 Apr 2021 21:05:45 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id 207sm18177874qkl.125.2021.04.07.21.05.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Apr 2021 21:05:45 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com Subject: [PATCH v13 05/18] arm64: trans_pgd: hibernate: Add trans_pgd_copy_el2_vectors Date: Thu, 8 Apr 2021 00:05:24 -0400 Message-Id: <20210408040537.2703241-6-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210408040537.2703241-1-pasha.tatashin@soleen.com> References: <20210408040537.2703241-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: DF1F080192D8 X-Stat-Signature: mebs5bbp1b4c4xb6xfxobao5xufj77e7 Received-SPF: none (soleen.com>: No applicable sender policy available) receiver=imf16; identity=mailfrom; envelope-from=""; helo=mail-qt1-f181.google.com; client-ip=209.85.160.181 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1617854745-68562 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Users of trans_pgd may also need a copy of vector table because it is also may be overwritten if a linear map can be overwritten. Move setup of EL2 vectors from hibernate to trans_pgd, so it can be later shared with kexec as well. Suggested-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/trans_pgd.h | 3 +++ arch/arm64/include/asm/virt.h | 3 +++ arch/arm64/kernel/hibernate.c | 28 ++++++++++------------------ arch/arm64/mm/trans_pgd.c | 20 ++++++++++++++++++++ 4 files changed, 36 insertions(+), 18 deletions(-) diff --git a/arch/arm64/include/asm/trans_pgd.h b/arch/arm64/include/asm/trans_pgd.h index 5d08e5adf3d5..e0760e52d36d 100644 --- a/arch/arm64/include/asm/trans_pgd.h +++ b/arch/arm64/include/asm/trans_pgd.h @@ -36,4 +36,7 @@ int trans_pgd_map_page(struct trans_pgd_info *info, pgd_t *trans_pgd, int trans_pgd_idmap_page(struct trans_pgd_info *info, phys_addr_t *trans_ttbr0, unsigned long *t0sz, void *page); +int trans_pgd_copy_el2_vectors(struct trans_pgd_info *info, + phys_addr_t *el2_vectors); + #endif /* _ASM_TRANS_TABLE_H */ diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h index 4216c8623538..bfbb66018114 100644 --- a/arch/arm64/include/asm/virt.h +++ b/arch/arm64/include/asm/virt.h @@ -67,6 +67,9 @@ */ extern u32 __boot_cpu_mode[2]; +extern char __hyp_stub_vectors[]; +#define ARM64_VECTOR_TABLE_LEN SZ_2K + void __hyp_set_vectors(phys_addr_t phys_vector_base); void __hyp_reset_vectors(void); diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index c764574a1acb..0b8bad8bb6eb 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -48,12 +48,6 @@ */ extern int in_suspend; -/* temporary el2 vectors in the __hibernate_exit_text section. */ -extern char hibernate_el2_vectors[]; - -/* hyp-stub vectors, used to restore el2 during resume from hibernate. */ -extern char __hyp_stub_vectors[]; - /* * The logical cpu number we should resume on, initialised to a non-cpu * number. @@ -428,6 +422,7 @@ int swsusp_arch_resume(void) void *zero_page; size_t exit_size; pgd_t *tmp_pg_dir; + phys_addr_t el2_vectors; void __noreturn (*hibernate_exit)(phys_addr_t, phys_addr_t, void *, void *, phys_addr_t, phys_addr_t); struct trans_pgd_info trans_info = { @@ -455,6 +450,14 @@ int swsusp_arch_resume(void) return -ENOMEM; } + if (is_hyp_callable()) { + rc = trans_pgd_copy_el2_vectors(&trans_info, &el2_vectors); + if (rc) { + pr_err("Failed to setup el2 vectors\n"); + return rc; + } + } + exit_size = __hibernate_exit_text_end - __hibernate_exit_text_start; /* * Copy swsusp_arch_suspend_exit() to a safe page. This will generate @@ -467,25 +470,14 @@ int swsusp_arch_resume(void) return rc; } - /* - * The hibernate exit text contains a set of el2 vectors, that will - * be executed at el2 with the mmu off in order to reload hyp-stub. - */ - __flush_dcache_area(hibernate_exit, exit_size); - /* * KASLR will cause the el2 vectors to be in a different location in * the resumed kernel. Load hibernate's temporary copy into el2. * * We can skip this step if we booted at EL1, or are running with VHE. */ - if (is_hyp_callable()) { - phys_addr_t el2_vectors = (phys_addr_t)hibernate_exit; - el2_vectors += hibernate_el2_vectors - - __hibernate_exit_text_start; /* offset */ - + if (is_hyp_callable()) __hyp_set_vectors(el2_vectors); - } hibernate_exit(virt_to_phys(tmp_pg_dir), resume_hdr.ttbr1_el1, resume_hdr.reenter_kernel, restore_pblist, diff --git a/arch/arm64/mm/trans_pgd.c b/arch/arm64/mm/trans_pgd.c index 527f0a39c3da..61549451ed3a 100644 --- a/arch/arm64/mm/trans_pgd.c +++ b/arch/arm64/mm/trans_pgd.c @@ -322,3 +322,23 @@ int trans_pgd_idmap_page(struct trans_pgd_info *info, phys_addr_t *trans_ttbr0, return 0; } + +/* + * Create a copy of the vector table so we can call HVC_SET_VECTORS or + * HVC_SOFT_RESTART from contexts where the table may be overwritten. + */ +int trans_pgd_copy_el2_vectors(struct trans_pgd_info *info, + phys_addr_t *el2_vectors) +{ + void *hyp_stub = trans_alloc(info); + + if (!hyp_stub) + return -ENOMEM; + *el2_vectors = virt_to_phys(hyp_stub); + memcpy(hyp_stub, &__hyp_stub_vectors, ARM64_VECTOR_TABLE_LEN); + __flush_icache_range((unsigned long)hyp_stub, + (unsigned long)hyp_stub + ARM64_VECTOR_TABLE_LEN); + __flush_dcache_area(hyp_stub, ARM64_VECTOR_TABLE_LEN); + + return 0; +} From patchwork Thu Apr 8 04:05:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12189941 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 95D1DC433B4 for ; Thu, 8 Apr 2021 04:05:52 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 35487611C0 for ; Thu, 8 Apr 2021 04:05:52 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 35487611C0 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8FF586B0087; Thu, 8 Apr 2021 00:05:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 888FC6B0088; Thu, 8 Apr 2021 00:05:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 68F256B0089; Thu, 8 Apr 2021 00:05:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0046.hostedemail.com [216.40.44.46]) by kanga.kvack.org (Postfix) with ESMTP id 4541B6B0087 for ; Thu, 8 Apr 2021 00:05:48 -0400 (EDT) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 0802481D4 for ; Thu, 8 Apr 2021 04:05:48 +0000 (UTC) X-FDA: 78007861176.23.E7FAA7E Received: from mail-qt1-f169.google.com (mail-qt1-f169.google.com [209.85.160.169]) by imf06.hostedemail.com (Postfix) with ESMTP id D3713C0007C1 for ; Thu, 8 Apr 2021 04:05:48 +0000 (UTC) Received: by mail-qt1-f169.google.com with SMTP id f12so472085qtf.2 for ; Wed, 07 Apr 2021 21:05:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=3ttZQ3gfSf54pfSUBx1LDfQsXE0pAbLQUWMLzhdR74s=; b=WNp1gMdsw3Tm4wv5ofO098L7+rqsXN0+xuPMLvxyhfRqO2E2ZKhDEGaGLLpQQCVLp6 kRWKAxhxmJ30NKIqBld7gTpjAwpz32kIBNExJ2Et0HZWEE+ivhSVeMWtXqp7UcIZyBCs X/LNtk7ga4MpNqn+3v94T7MU9nnrxtlJ/zYiq0277ls+/aEP16FB1VizUFYUquLa7aw9 GiGHeJD31iKWwANH8mQHHBOHgqAlwmaUPPJC5gCo6msF/l+C4Y8nVRzmZMqDdNYh/mJM 69+TMIvHTm4pme+kPif1x+MlMxbSFgJaabSY0wDQ4Fxs8xJDRaaUL9ndnx4mfcqONRfT nELQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3ttZQ3gfSf54pfSUBx1LDfQsXE0pAbLQUWMLzhdR74s=; b=NvZNWg3nga7NI7QonhoQzDckoS2g/fHIT3mj6LxT09UNrbgeZZALpuEByESVH/wjL3 g1xlaw+A+W3z/XP4FShLCxOb4PA7iqyaci6v1bWbowVhjoe5flto1fbJTrwskP0+sar3 mJg+oZ8cJxTIYsYUpNO3ONr9WIhcfpGE/9L/cTA1AIPtTA92dwN17fYGOy7uxkbT33mC l1RZxxM0oLyXxeT+VFpTixIMousXJ9Q6wX9PrU2owv2p0W2i52VvT9vSKikFf2N6lJUY bdLRlNmpgEbeBmMSysmDFxrnOWN/wkjQS3KUrwkkDe6c9WYTYVGzm/UTx3o+W8XWU7ZI kVrw== X-Gm-Message-State: AOAM533QyGSQWpwZUwBRYDSZYN8B1Q3M06O9q6rCCrocXqIiSj1WMt/S UF9ivj/TZNLzwOZP0lUcqb91ng== X-Google-Smtp-Source: ABdhPJzXmtqxtUQTEJT0lxRGeT3Nceyt+CBZeKYInLcPaN+AnmjXc07TTgCf3AOXx1AjM6gvwPz9Hw== X-Received: by 2002:ac8:6684:: with SMTP id d4mr5596938qtp.34.1617854747044; Wed, 07 Apr 2021 21:05:47 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id 207sm18177874qkl.125.2021.04.07.21.05.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Apr 2021 21:05:46 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com Subject: [PATCH v13 06/18] arm64: hibernate: abstract ttrb0 setup function Date: Thu, 8 Apr 2021 00:05:25 -0400 Message-Id: <20210408040537.2703241-7-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210408040537.2703241-1-pasha.tatashin@soleen.com> References: <20210408040537.2703241-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: D3713C0007C1 X-Stat-Signature: mbekf8q4mh3ti8zw44ag3fht94qreh46 Received-SPF: none (soleen.com>: No applicable sender policy available) receiver=imf06; identity=mailfrom; envelope-from=""; helo=mail-qt1-f169.google.com; client-ip=209.85.160.169 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1617854748-799286 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently, only hibernate sets custom ttbr0 with safe idmaped function. Kexec, is also going to be using this functinality when relocation code is going to be idmapped. Move the setup seqeuence to a dedicated cpu_install_ttbr0() for custom ttbr0. Suggested-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/mmu_context.h | 24 ++++++++++++++++++++++++ arch/arm64/kernel/hibernate.c | 21 +-------------------- 2 files changed, 25 insertions(+), 20 deletions(-) diff --git a/arch/arm64/include/asm/mmu_context.h b/arch/arm64/include/asm/mmu_context.h index bd02e99b1a4c..f64d0d5e1b1f 100644 --- a/arch/arm64/include/asm/mmu_context.h +++ b/arch/arm64/include/asm/mmu_context.h @@ -115,6 +115,30 @@ static inline void cpu_install_idmap(void) cpu_switch_mm(lm_alias(idmap_pg_dir), &init_mm); } +/* + * Load our new page tables. A strict BBM approach requires that we ensure that + * TLBs are free of any entries that may overlap with the global mappings we are + * about to install. + * + * For a real hibernate/resume/kexec cycle TTBR0 currently points to a zero + * page, but TLBs may contain stale ASID-tagged entries (e.g. for EFI runtime + * services), while for a userspace-driven test_resume cycle it points to + * userspace page tables (and we must point it at a zero page ourselves). + * + * We change T0SZ as part of installing the idmap. This is undone by + * cpu_uninstall_idmap() in __cpu_suspend_exit(). + */ +static inline void cpu_install_ttbr0(phys_addr_t ttbr0, unsigned long t0sz) +{ + cpu_set_reserved_ttbr0(); + local_flush_tlb_all(); + __cpu_set_tcr_t0sz(t0sz); + + /* avoid cpu_switch_mm() and its SW-PAN and CNP interactions */ + write_sysreg(ttbr0, ttbr0_el1); + isb(); +} + /* * Atomically replaces the active TTBR1_EL1 PGD with a new VA-compatible PGD, * avoiding the possibility of conflicting TLB entries being allocated. diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index 0b8bad8bb6eb..ded5115bcb63 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -206,26 +206,7 @@ static int create_safe_exec_page(void *src_start, size_t length, if (rc) return rc; - /* - * Load our new page tables. A strict BBM approach requires that we - * ensure that TLBs are free of any entries that may overlap with the - * global mappings we are about to install. - * - * For a real hibernate/resume cycle TTBR0 currently points to a zero - * page, but TLBs may contain stale ASID-tagged entries (e.g. for EFI - * runtime services), while for a userspace-driven test_resume cycle it - * points to userspace page tables (and we must point it at a zero page - * ourselves). - * - * We change T0SZ as part of installing the idmap. This is undone by - * cpu_uninstall_idmap() in __cpu_suspend_exit(). - */ - cpu_set_reserved_ttbr0(); - local_flush_tlb_all(); - __cpu_set_tcr_t0sz(t0sz); - write_sysreg(trans_ttbr0, ttbr0_el1); - isb(); - + cpu_install_ttbr0(trans_ttbr0, t0sz); *phys_dst_addr = virt_to_phys(page); return 0; From patchwork Thu Apr 8 04:05:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12189943 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E6224C433ED for ; Thu, 8 Apr 2021 04:05:54 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 79329611C0 for ; Thu, 8 Apr 2021 04:05:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 79329611C0 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C5A4B6B0088; Thu, 8 Apr 2021 00:05:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BE3AE6B0089; Thu, 8 Apr 2021 00:05:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A10226B008A; Thu, 8 Apr 2021 00:05:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0054.hostedemail.com [216.40.44.54]) by kanga.kvack.org (Postfix) with ESMTP id 84F196B0088 for ; Thu, 8 Apr 2021 00:05:49 -0400 (EDT) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 4B2374DCA for ; Thu, 8 Apr 2021 04:05:49 +0000 (UTC) X-FDA: 78007861218.17.3197529 Received: from mail-qt1-f169.google.com (mail-qt1-f169.google.com [209.85.160.169]) by imf14.hostedemail.com (Postfix) with ESMTP id 16BE1C0001F7 for ; Thu, 8 Apr 2021 04:05:43 +0000 (UTC) Received: by mail-qt1-f169.google.com with SMTP id j7so462641qtx.5 for ; Wed, 07 Apr 2021 21:05:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=FcIEg/sDUKkdURMCkTF1KW5fxOPpkw674yFxDZdydcg=; b=K1NsbXiptJWAFjDZB69MTcxq+sVd357Lm9DKIwBQhxbGV3HftSKBeUqa23Wmi5KKNU zvjcqW4fvbuFxiDUEdiWh3Mcf9hsDD6xZvknyHE0ctG5bfVF8q4qEwPS0v1jcEKtMcbA 8nKTdc1FLRCDhKH+sT2n8fMhr1u+4HPurVK8/xjKe/Z9Cy3im6HxbQ+zh7vl4AwQ97/T ZODphpRxpUyYHVXtt6c71a1yPBcTrl3qHz4ut24V8AGbtwoYkqrh2tIBNNyfurAIblR4 QypVLYj0CXuJ3SBllkLS4gaJWPnDY2g4n0RpZpXaLWcaMPu4cnP4N2D+arBerwRT+OoS Yx6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FcIEg/sDUKkdURMCkTF1KW5fxOPpkw674yFxDZdydcg=; b=RVFH93fqw1j2DH8IX05UBfRIRsrjBYSYt1N1aTdRyxzSVOmTC9MjsNf6nI62UUOC4j Ch3it93lTQ9K+G81JgS9VHOEN0Uu1DHujmLIKRzE4gwFxQ2SFwRw3A4rK1q/Bdu8/tYn PNXPT0ReLGa1mhj1nUGdq2fJnS0sYsderrRzWCp4WXZOp9o3cqcYQQtZ7UsLb5G1w51v oe58kHzxgJIm1FINqrHI3uMdXH/c99XG4rE2eHsfp+qh5lDxMV/UYmdcz8r31a8zeaBZ B7hA3MI3X7vQrY00TOkXYfvAA33/0B+uX/Xf2IjnFbKWLRAsX+rP1aRjiHhvHk3b9Uak s0Iw== X-Gm-Message-State: AOAM530ZZIQeqmpR3a6N0biU1x+haeE8l60qbaeYOc875iFTm/9pCcbx H+IxNrgE+TBWj+0EsOoZQX9S/A== X-Google-Smtp-Source: ABdhPJz+49F5sFnvxO3bcbTDIuYVduCfSWrd4A61z5Tt8rvcCW5QOx1alJlSaDZm0kDJxf714A6VuA== X-Received: by 2002:ac8:5313:: with SMTP id t19mr5657090qtn.148.1617854748347; Wed, 07 Apr 2021 21:05:48 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id 207sm18177874qkl.125.2021.04.07.21.05.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Apr 2021 21:05:47 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com Subject: [PATCH v13 07/18] arm64: kexec: flush image and lists during kexec load time Date: Thu, 8 Apr 2021 00:05:26 -0400 Message-Id: <20210408040537.2703241-8-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210408040537.2703241-1-pasha.tatashin@soleen.com> References: <20210408040537.2703241-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 16BE1C0001F7 X-Stat-Signature: 5a41smx53ibok5c7xeyckrr3sdcj8qbp Received-SPF: none (soleen.com>: No applicable sender policy available) receiver=imf14; identity=mailfrom; envelope-from=""; helo=mail-qt1-f169.google.com; client-ip=209.85.160.169 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1617854743-599239 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently, during kexec load we are copying relocation function and flushing it. However, we can also flush kexec relocation buffers and if new kernel image is already in place (i.e. crash kernel), we can also flush the new kernel image itself. Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/machine_kexec.c | 49 +++++++++++++++---------------- 1 file changed, 23 insertions(+), 26 deletions(-) diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 90a335c74442..3a034bc25709 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -59,23 +59,6 @@ void machine_kexec_cleanup(struct kimage *kimage) /* Empty routine needed to avoid build errors. */ } -int machine_kexec_post_load(struct kimage *kimage) -{ - void *reloc_code = page_to_virt(kimage->control_code_page); - - memcpy(reloc_code, arm64_relocate_new_kernel, - arm64_relocate_new_kernel_size); - kimage->arch.kern_reloc = __pa(reloc_code); - kexec_image_info(kimage); - - /* Flush the reloc_code in preparation for its execution. */ - __flush_dcache_area(reloc_code, arm64_relocate_new_kernel_size); - flush_icache_range((uintptr_t)reloc_code, (uintptr_t)reloc_code + - arm64_relocate_new_kernel_size); - - return 0; -} - /** * machine_kexec_prepare - Prepare for a kexec reboot. * @@ -152,6 +135,29 @@ static void kexec_segment_flush(const struct kimage *kimage) } } +int machine_kexec_post_load(struct kimage *kimage) +{ + void *reloc_code = page_to_virt(kimage->control_code_page); + + /* If in place flush new kernel image, else flush lists and buffers */ + if (kimage->head & IND_DONE) + kexec_segment_flush(kimage); + else + kexec_list_flush(kimage); + + memcpy(reloc_code, arm64_relocate_new_kernel, + arm64_relocate_new_kernel_size); + kimage->arch.kern_reloc = __pa(reloc_code); + kexec_image_info(kimage); + + /* Flush the reloc_code in preparation for its execution. */ + __flush_dcache_area(reloc_code, arm64_relocate_new_kernel_size); + flush_icache_range((uintptr_t)reloc_code, (uintptr_t)reloc_code + + arm64_relocate_new_kernel_size); + + return 0; +} + /** * machine_kexec - Do the kexec reboot. * @@ -169,13 +175,6 @@ void machine_kexec(struct kimage *kimage) WARN(in_kexec_crash && (stuck_cpus || smp_crash_stop_failed()), "Some CPUs may be stale, kdump will be unreliable.\n"); - /* Flush the kimage list and its buffers. */ - kexec_list_flush(kimage); - - /* Flush the new image if already in place. */ - if ((kimage != kexec_crash_image) && (kimage->head & IND_DONE)) - kexec_segment_flush(kimage); - pr_info("Bye!\n"); local_daif_mask(); @@ -250,8 +249,6 @@ void arch_kexec_protect_crashkres(void) { int i; - kexec_segment_flush(kexec_crash_image); - for (i = 0; i < kexec_crash_image->nr_segments; i++) set_memory_valid( __phys_to_virt(kexec_crash_image->segment[i].mem), From patchwork Thu Apr 8 04:05:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12189945 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 551ABC433ED for ; Thu, 8 Apr 2021 04:05:57 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E7FD161157 for ; Thu, 8 Apr 2021 04:05:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E7FD161157 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3A81B6B0089; Thu, 8 Apr 2021 00:05:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3324D6B008A; Thu, 8 Apr 2021 00:05:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 137566B008C; Thu, 8 Apr 2021 00:05:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0222.hostedemail.com [216.40.44.222]) by kanga.kvack.org (Postfix) with ESMTP id EAF106B0089 for ; Thu, 8 Apr 2021 00:05:50 -0400 (EDT) Received: from smtpin37.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id A81367817 for ; Thu, 8 Apr 2021 04:05:50 +0000 (UTC) X-FDA: 78007861260.37.3D3DF8D Received: from mail-qt1-f172.google.com (mail-qt1-f172.google.com [209.85.160.172]) by imf03.hostedemail.com (Postfix) with ESMTP id 0008CC0007C6 for ; Thu, 8 Apr 2021 04:05:47 +0000 (UTC) Received: by mail-qt1-f172.google.com with SMTP id u8so440208qtq.12 for ; Wed, 07 Apr 2021 21:05:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=hiQbfer3Cy+5MCvp23I2bat0V/tLiwOPOJLrW/6qX9o=; b=Vd61oEBV7Vn5WTuBeGbCdJHrwIWCibHMtCk++64snK/ZPDtTyhHe0Nq3a+pjo8uswX Et4Le8VT1I2fif7zEsz3e8Gdw81BQpEQ9IblPTxGPS2Jfqhz9lwSe307RzNh+knkCGtX U+Oiqfv/zT5hYamo3zUvH1r9kESLRvQIuF1PEyslShJ+OFm6TrqzjnvBF5TTn7ibNGl+ +48FhVVIo0tE5For8Dxkpf9z9605OZzVjFN6r/o6N+nQ3g+/M/fHSa90RaVvzZAH8MJk /mj0LHL4p3vKmQFIi9y8+DOJA3AnhqcHU1QgRDtJvhbvjntNHQ7XKTYVbQ8++2ZQu+bc eYSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hiQbfer3Cy+5MCvp23I2bat0V/tLiwOPOJLrW/6qX9o=; b=TUvIlWVz4pEpUMLCTMAwqKdBz9yT6x5TzcGT+oFo5m0AaURvY55bFwgva8/WKyccJO xv0awtu9cakkEaQWkqCUM+XMFQtLkpRIU4IsmmAriCZxZiooILFDYa+w4F7mUZbTfGLL 10VpuMtOgb8GssU0LLOBioMPptpGvu9TNFGvdQvyBKQmafkHIekSvjf24Ug5GR3jPT98 xJfuG718WlA6+555oDp1/KhKLsLtQiLRYAtpPntDO2AUk1VMBnKvv+aeB+E+AWMXpqxj iJkPZ25X2qSMPm0NJnlvlnon+XT5nJQ0R7P59j3YbD4rq0feC1HM3Nxz1cJwgHsYsonX 0Evw== X-Gm-Message-State: AOAM533g8SZHQuqms5rBy2fmdnpu80yb80R9xMKQLoBlrtBJtzNnHLgI NA2br2kNIlUAf2muL4hREaez0w== X-Google-Smtp-Source: ABdhPJwaCeykglUkEXFAvBa1BzVKFnTCf1ecW8WT2k77yBhsz+NHJ2cCh2ZdKNS2R+jDbwph0oQBwQ== X-Received: by 2002:ac8:7611:: with SMTP id t17mr5459416qtq.361.1617854749679; Wed, 07 Apr 2021 21:05:49 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id 207sm18177874qkl.125.2021.04.07.21.05.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Apr 2021 21:05:49 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com Subject: [PATCH v13 08/18] arm64: kexec: skip relocation code for inplace kexec Date: Thu, 8 Apr 2021 00:05:27 -0400 Message-Id: <20210408040537.2703241-9-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210408040537.2703241-1-pasha.tatashin@soleen.com> References: <20210408040537.2703241-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 0008CC0007C6 X-Stat-Signature: yoac7rjgg66qydt6jqyt5kxqo1mk6kde Received-SPF: none (soleen.com>: No applicable sender policy available) receiver=imf03; identity=mailfrom; envelope-from=""; helo=mail-qt1-f172.google.com; client-ip=209.85.160.172 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1617854747-51730 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In case of kdump or when segments are already in place the relocation is not needed, therefore the setup of relocation function and call to it can be skipped. Signed-off-by: Pavel Tatashin Suggested-by: James Morse --- arch/arm64/kernel/machine_kexec.c | 34 ++++++++++++++++++----------- arch/arm64/kernel/relocate_kernel.S | 3 --- 2 files changed, 21 insertions(+), 16 deletions(-) diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 3a034bc25709..b150b65f0b84 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -139,21 +139,23 @@ int machine_kexec_post_load(struct kimage *kimage) { void *reloc_code = page_to_virt(kimage->control_code_page); - /* If in place flush new kernel image, else flush lists and buffers */ - if (kimage->head & IND_DONE) + /* If in place, relocation is not used, only flush next kernel */ + if (kimage->head & IND_DONE) { kexec_segment_flush(kimage); - else - kexec_list_flush(kimage); + kexec_image_info(kimage); + return 0; + } memcpy(reloc_code, arm64_relocate_new_kernel, arm64_relocate_new_kernel_size); kimage->arch.kern_reloc = __pa(reloc_code); - kexec_image_info(kimage); /* Flush the reloc_code in preparation for its execution. */ __flush_dcache_area(reloc_code, arm64_relocate_new_kernel_size); flush_icache_range((uintptr_t)reloc_code, (uintptr_t)reloc_code + arm64_relocate_new_kernel_size); + kexec_list_flush(kimage); + kexec_image_info(kimage); return 0; } @@ -180,19 +182,25 @@ void machine_kexec(struct kimage *kimage) local_daif_mask(); /* - * cpu_soft_restart will shutdown the MMU, disable data caches, then - * transfer control to the kern_reloc which contains a copy of - * the arm64_relocate_new_kernel routine. arm64_relocate_new_kernel - * uses physical addressing to relocate the new image to its final - * position and transfers control to the image entry point when the - * relocation is complete. + * Both restart and cpu_soft_restart will shutdown the MMU, disable data + * caches. However, restart will start new kernel or purgatory directly, + * cpu_soft_restart will transfer control to arm64_relocate_new_kernel * In kexec case, kimage->start points to purgatory assuming that * kernel entry and dtb address are embedded in purgatory by * userspace (kexec-tools). * In kexec_file case, the kernel starts directly without purgatory. */ - cpu_soft_restart(kimage->arch.kern_reloc, kimage->head, kimage->start, - kimage->arch.dtb_mem); + if (kimage->head & IND_DONE) { + typeof(__cpu_soft_restart) *restart; + + cpu_install_idmap(); + restart = (void *)__pa_symbol(__cpu_soft_restart); + restart(is_hyp_callable(), kimage->start, kimage->arch.dtb_mem, + 0, 0); + } else { + cpu_soft_restart(kimage->arch.kern_reloc, kimage->head, + kimage->start, kimage->arch.dtb_mem); + } BUG(); /* Should never get here. */ } diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index b78ea5de97a4..8058fabe0a76 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -32,8 +32,6 @@ SYM_CODE_START(arm64_relocate_new_kernel) mov x16, x0 /* x16 = kimage_head */ mov x14, xzr /* x14 = entry ptr */ mov x13, xzr /* x13 = copy dest */ - /* Check if the new image needs relocation. */ - tbnz x16, IND_DONE_BIT, .Ldone raw_dcache_line_size x15, x1 /* x15 = dcache line size */ .Lloop: and x12, x16, PAGE_MASK /* x12 = addr */ @@ -65,7 +63,6 @@ SYM_CODE_START(arm64_relocate_new_kernel) .Lnext: ldr x16, [x14], #8 /* entry = *ptr++ */ tbz x16, IND_DONE_BIT, .Lloop /* while (!(entry & DONE)) */ -.Ldone: /* wait for writes from copy_page to finish */ dsb nsh ic iallu From patchwork Thu Apr 8 04:05:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12189947 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CD10BC433ED for ; Thu, 8 Apr 2021 04:05:59 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7C96261157 for ; Thu, 8 Apr 2021 04:05:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7C96261157 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 975FE6B008A; Thu, 8 Apr 2021 00:05:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8B1F66B008C; Thu, 8 Apr 2021 00:05:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 72AF26B0092; Thu, 8 Apr 2021 00:05:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0119.hostedemail.com [216.40.44.119]) by kanga.kvack.org (Postfix) with ESMTP id 3FF246B008A for ; Thu, 8 Apr 2021 00:05:52 -0400 (EDT) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id EF13A181C1937 for ; Thu, 8 Apr 2021 04:05:51 +0000 (UTC) X-FDA: 78007861302.13.2BF9356 Received: from mail-qt1-f170.google.com (mail-qt1-f170.google.com [209.85.160.170]) by imf07.hostedemail.com (Postfix) with ESMTP id 6089CA000382 for ; Thu, 8 Apr 2021 04:05:51 +0000 (UTC) Received: by mail-qt1-f170.google.com with SMTP id x9so452401qto.8 for ; Wed, 07 Apr 2021 21:05:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=d9prc1UZUFN5dwsf86cy1HbKMQhioOGgsdfiYkk/1yQ=; b=CKGSSrZOId5j0LqLgbjZCuXMtuV5OgJo1NkH2oQVBjJKC13+u8jXgkVJP0kzp7RP5Z bbOoQmxt7BF6OWd8qL4HSp0IXWbu3tLCeV1Ishweb4EUIuyoem0zSaoWmRQozHihC3Dm VP7NzQQApfjnGgbM1PHOjz/MnPFTiYWNEro2KMw6pbbZnvbU2wP2Cf3WL9WLSSE647dX 3vI7VBAznYsKmkKOYlE4YXmamMCrIkzG58sFWyB5aImgCIRJKwz4PLiRd9lFW+HZJxaf JmP8CmnjAJdpBp4yxQBnhzmCcNYrzOYbPkaqyPYlB0Es2Sx6yeje04DjJ4yrsO4h+R0T bolA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=d9prc1UZUFN5dwsf86cy1HbKMQhioOGgsdfiYkk/1yQ=; b=Kdy4GAfDWk47IMUsTzaIDXMsVJDpik6hxf/7RLf3Ocsz7PH9c8CBBMnXey24jOFTpN z4Aqn6DWdbz6FB2afIA0VELJyd1coylZXTO5OsBkRd/3MFG5qIYtOuRXH83fhDG0COIy 0/0AX0B85zM1UZlZCrrElTgxp2rNofM3XtNM8M5WXf5BMZ864iYK2YD5uUQ4n+x6DLTS axYl72CtztICg+roBXdLuGsPLlnG+Mg+Cn+U9sz8ZEByclCIzZ9hCY2LyIKLf2OTAMPn 0gDjv7eklozBEe4zB4+vafVGB6IJCYLpWvY0X7ddcfRCX/xUuWDpS/5GcdWAcitbgjH8 DNng== X-Gm-Message-State: AOAM530jsqwMDA5wIko5z7RMclWvEQgLmLLBYR0fKVErkhGhLiFhQ2xw Yq5LgWiHnheTMguNMU6NR+MnOA== X-Google-Smtp-Source: ABdhPJy2yjiHv6dpdlu+pGlHHabbICyMKqXxlvfx119ga4c5aETs6NQg38h9Fs3h1H/OOkERem7iRQ== X-Received: by 2002:a05:622a:46:: with SMTP id y6mr5573543qtw.154.1617854750976; Wed, 07 Apr 2021 21:05:50 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id 207sm18177874qkl.125.2021.04.07.21.05.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Apr 2021 21:05:50 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com Subject: [PATCH v13 09/18] arm64: kexec: Use dcache ops macros instead of open-coding Date: Thu, 8 Apr 2021 00:05:28 -0400 Message-Id: <20210408040537.2703241-10-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210408040537.2703241-1-pasha.tatashin@soleen.com> References: <20210408040537.2703241-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 6089CA000382 X-Stat-Signature: 9x64s6q5ybyd4hqjp43h75pgo45qbkhq Received-SPF: none (soleen.com>: No applicable sender policy available) receiver=imf07; identity=mailfrom; envelope-from=""; helo=mail-qt1-f170.google.com; client-ip=209.85.160.170 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1617854751-77504 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: James Morse kexec does dcache maintenance when it re-writes all memory. Our dcache_by_line_op macro depends on reading the sanitised DminLine from memory. Kexec may have overwritten this, so open-codes the sequence. dcache_by_line_op is a whole set of macros, it uses dcache_line_size which uses read_ctr for the sanitsed DminLine. Reading the DminLine is the first thing the dcache_by_line_op does. Rename dcache_by_line_op dcache_by_myline_op and take DminLine as an argument. Kexec can now use the slightly smaller macro. This makes up-coming changes to the dcache maintenance easier on the eye. Code generated by the existing callers is unchanged. Signed-off-by: James Morse [Fixed merging issues] Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/assembler.h | 12 ++++++++---- arch/arm64/kernel/relocate_kernel.S | 13 +++---------- 2 files changed, 11 insertions(+), 14 deletions(-) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index ca31594d3d6c..29061b76aab6 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -371,10 +371,9 @@ alternative_else alternative_endif .endm - .macro dcache_by_line_op op, domain, kaddr, size, tmp1, tmp2 - dcache_line_size \tmp1, \tmp2 + .macro dcache_by_myline_op op, domain, kaddr, size, linesz, tmp2 add \size, \kaddr, \size - sub \tmp2, \tmp1, #1 + sub \tmp2, \linesz, #1 bic \kaddr, \kaddr, \tmp2 9998: .ifc \op, cvau @@ -394,12 +393,17 @@ alternative_endif .endif .endif .endif - add \kaddr, \kaddr, \tmp1 + add \kaddr, \kaddr, \linesz cmp \kaddr, \size b.lo 9998b dsb \domain .endm + .macro dcache_by_line_op op, domain, kaddr, size, tmp1, tmp2 + dcache_line_size \tmp1, \tmp2 + dcache_by_myline_op \op, \domain, \kaddr, \size, \tmp1, \tmp2 + .endm + /* * Macro to perform an instruction cache maintenance for the interval * [start, end) diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index 8058fabe0a76..718037bef560 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -41,16 +41,9 @@ SYM_CODE_START(arm64_relocate_new_kernel) tbz x16, IND_SOURCE_BIT, .Ltest_indirection /* Invalidate dest page to PoC. */ - mov x2, x13 - add x20, x2, #PAGE_SIZE - sub x1, x15, #1 - bic x2, x2, x1 -2: dc ivac, x2 - add x2, x2, x15 - cmp x2, x20 - b.lo 2b - dsb sy - + mov x2, x13 + mov x1, #PAGE_SIZE + dcache_by_myline_op ivac, sy, x2, x1, x15, x20 copy_page x13, x12, x1, x2, x3, x4, x5, x6, x7, x8 b .Lnext .Ltest_indirection: From patchwork Thu Apr 8 04:05:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12189949 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 93A72C433ED for ; Thu, 8 Apr 2021 04:06:02 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 34C9A611BD for ; Thu, 8 Apr 2021 04:06:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 34C9A611BD Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id CAB046B008C; Thu, 8 Apr 2021 00:05:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B98866B0092; Thu, 8 Apr 2021 00:05:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9E8AC6B0093; Thu, 8 Apr 2021 00:05:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0165.hostedemail.com [216.40.44.165]) by kanga.kvack.org (Postfix) with ESMTP id 7F12A6B008C for ; Thu, 8 Apr 2021 00:05:53 -0400 (EDT) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 40643181C1937 for ; Thu, 8 Apr 2021 04:05:53 +0000 (UTC) X-FDA: 78007861386.19.D4BD6B0 Received: from mail-qv1-f45.google.com (mail-qv1-f45.google.com [209.85.219.45]) by imf01.hostedemail.com (Postfix) with ESMTP id 3B2CA5001526 for ; Thu, 8 Apr 2021 04:05:52 +0000 (UTC) Received: by mail-qv1-f45.google.com with SMTP id x27so265229qvd.2 for ; Wed, 07 Apr 2021 21:05:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=tCKT8dQy3j0JXLfUAkLHptP8jUCHy5Mtlem3XeT2uzw=; b=F6JZkU80674Q1VkhfFm80eRIhlT4LTp68/RErf1725v9DYSxSAh29UWw3UDtiBURG4 u0n2ebEhoDt5HtuTO3kincM93W8gSFAACO87UbFvfg+TKnnlfjDa1iPq5U8D9ey2nSYN W4fhKnt/bDMPuHnnAp6P4c6OLXcnEd6kMmzkgMOyEPsn0UhQt+5PjB55hUmuN0Nzpzap +0rOYzqOJLSwje/mdtbOjaL0CrjS19OmEbMqcXf4HNwEkMAZ64+V0SJ6fNu52Frq1oIh KlDteFSrD5AR4g5pbiIAHi0PC+tohcS908vNFFXwicB+eWCfa4xnX2RXlCj6/FZxVMFm scaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tCKT8dQy3j0JXLfUAkLHptP8jUCHy5Mtlem3XeT2uzw=; b=Z87JS2CJX+FLaMzAwBtfox5XIvxs7vK3AuJae3ztsSMPhCV8QLXmbPzuVIA5a+nWCB 6BYyS9ha2agJ4uBzDHkVo5CPSHweVZIA2I+xESFmUc8GlJKXUSR7m+PVlGiuA8V53CAm 1+LhvJaAWYPVO7nIgeffIyUh4QQcyqUL9mievUZME4eT3QMuy3m4PgI0c7O9tveuGDhp Rm1ev1ox3YoUIONbWRofiooCk8HejofgMGsVYF9EwwTVP3E/9VYWHMtJi//XnTWHFVqx 8tfrU/GufSN9frZpVM+n/Myj2jpRgafzGN8yr45cqpLFxYTybggkb4kPiu15XTw31+xz Btag== X-Gm-Message-State: AOAM530NlSbj9JZRHJvdXqZ/DBH8g3EmsAulYeZGHY/CNowNiQN50ltM 5/7Y2heKYGgKVURdG4W8nWilzg== X-Google-Smtp-Source: ABdhPJy34xVqJvda6lNplCUNEP3hwQatijEVRlmosHFcsSuFccjv0kKbw6AfhQKTmbRewTzaiAP2bw== X-Received: by 2002:a0c:80e1:: with SMTP id 88mr6471448qvb.43.1617854752265; Wed, 07 Apr 2021 21:05:52 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id 207sm18177874qkl.125.2021.04.07.21.05.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Apr 2021 21:05:51 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com Subject: [PATCH v13 10/18] arm64: kexec: pass kimage as the only argument to relocation function Date: Thu, 8 Apr 2021 00:05:29 -0400 Message-Id: <20210408040537.2703241-11-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210408040537.2703241-1-pasha.tatashin@soleen.com> References: <20210408040537.2703241-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 3B2CA5001526 X-Stat-Signature: 5embgo7ozna59a9cmyupyhww5tpa18ez Received-SPF: none (soleen.com>: No applicable sender policy available) receiver=imf01; identity=mailfrom; envelope-from=""; helo=mail-qv1-f45.google.com; client-ip=209.85.219.45 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1617854752-931878 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently, kexec relocation function (arm64_relocate_new_kernel) accepts the following arguments: head: start of array that contains relocation information. entry: entry point for new kernel or purgatory. dtb_mem: first and only argument to entry. The number of arguments cannot be easily expended, because this function is also called from HVC_SOFT_RESTART, which preserves only three arguments. And, also arm64_relocate_new_kernel is written in assembly but called without stack, thus no place to move extra arguments to free registers. Soon, we will need to pass more arguments: once we enable MMU we will need to pass information about page tables. Pass kimage to arm64_relocate_new_kernel, and teach it to get the required fields from kimage. Suggested-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/asm-offsets.c | 7 +++++++ arch/arm64/kernel/machine_kexec.c | 6 ++++-- arch/arm64/kernel/relocate_kernel.S | 10 ++++------ 3 files changed, 15 insertions(+), 8 deletions(-) diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index a36e2fc330d4..0c92e193f866 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -9,6 +9,7 @@ #include #include +#include #include #include #include @@ -153,6 +154,12 @@ int main(void) DEFINE(PTRAUTH_USER_KEY_APGA, offsetof(struct ptrauth_keys_user, apga)); DEFINE(PTRAUTH_KERNEL_KEY_APIA, offsetof(struct ptrauth_keys_kernel, apia)); BLANK(); +#endif +#ifdef CONFIG_KEXEC_CORE + DEFINE(KIMAGE_ARCH_DTB_MEM, offsetof(struct kimage, arch.dtb_mem)); + DEFINE(KIMAGE_HEAD, offsetof(struct kimage, head)); + DEFINE(KIMAGE_START, offsetof(struct kimage, start)); + BLANK(); #endif return 0; } diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index b150b65f0b84..2e734e4ae12e 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -83,6 +83,8 @@ static void kexec_list_flush(struct kimage *kimage) { kimage_entry_t *entry; + __flush_dcache_area(kimage, sizeof(*kimage)); + for (entry = &kimage->head; ; entry++) { unsigned int flag; void *addr; @@ -198,8 +200,8 @@ void machine_kexec(struct kimage *kimage) restart(is_hyp_callable(), kimage->start, kimage->arch.dtb_mem, 0, 0); } else { - cpu_soft_restart(kimage->arch.kern_reloc, kimage->head, - kimage->start, kimage->arch.dtb_mem); + cpu_soft_restart(kimage->arch.kern_reloc, virt_to_phys(kimage), + 0, 0); } BUG(); /* Should never get here. */ diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index 718037bef560..36b4496524c3 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -27,9 +27,7 @@ */ SYM_CODE_START(arm64_relocate_new_kernel) /* Setup the list loop variables. */ - mov x18, x2 /* x18 = dtb address */ - mov x17, x1 /* x17 = kimage_start */ - mov x16, x0 /* x16 = kimage_head */ + ldr x16, [x0, #KIMAGE_HEAD] /* x16 = kimage_head */ mov x14, xzr /* x14 = entry ptr */ mov x13, xzr /* x13 = copy dest */ raw_dcache_line_size x15, x1 /* x15 = dcache line size */ @@ -63,12 +61,12 @@ SYM_CODE_START(arm64_relocate_new_kernel) isb /* Start new image. */ - mov x0, x18 + ldr x4, [x0, #KIMAGE_START] /* relocation start */ + ldr x0, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ mov x1, xzr mov x2, xzr mov x3, xzr - br x17 - + br x4 SYM_CODE_END(arm64_relocate_new_kernel) .align 3 /* To keep the 64-bit values below naturally aligned. */ From patchwork Thu Apr 8 04:05:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12189951 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2F310C433ED for ; Thu, 8 Apr 2021 04:06:05 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D1CC5611C0 for ; Thu, 8 Apr 2021 04:06:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D1CC5611C0 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9C2D86B0092; Thu, 8 Apr 2021 00:05:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 925666B0093; Thu, 8 Apr 2021 00:05:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 641846B0095; Thu, 8 Apr 2021 00:05:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0044.hostedemail.com [216.40.44.44]) by kanga.kvack.org (Postfix) with ESMTP id 3C2736B0092 for ; Thu, 8 Apr 2021 00:05:56 -0400 (EDT) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id F2D338778 for ; Thu, 8 Apr 2021 04:05:55 +0000 (UTC) X-FDA: 78007861470.25.576B582 Received: from mail-qv1-f48.google.com (mail-qv1-f48.google.com [209.85.219.48]) by imf26.hostedemail.com (Postfix) with ESMTP id E9DBC40002CF for ; Thu, 8 Apr 2021 04:05:52 +0000 (UTC) Received: by mail-qv1-f48.google.com with SMTP id fn8so261930qvb.5 for ; Wed, 07 Apr 2021 21:05:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=ATHDsIlQRij0i8p3z1/oZchje9DfRT1VSEQgV22ghC4=; b=cZfP06UdFwUbDXzsBkl1RZmPvCxQFFD555CU5BZSd+vAYM+/Oe+q5LF6b+AeDb3hx2 LFsT7XpzjiKY3Si2YsqeQ/IsmJpOiDmMG/HNxXKVtP9KTMFFrpTt/zWcuY6K4+nYd9LP 9ZKSsnmvV+2+q+Pmt93pKHZ0lgdpVdpMXDll0Wv1hibDF4r0T45CgLAGXvLMQJPDkBFz 9IzjdkP1g6YNXrx/gkwrI/Ofe/tf10DE04Grydz47vkZEbw+ldvrJuhINzXNy4tlgWG7 P+aKh8JBXoF2aRQrMgr5pwbBamx/FerBKZ1ntwmSE4A+0xIGTTmKGW08lsKS3Z2H1qJG Decw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ATHDsIlQRij0i8p3z1/oZchje9DfRT1VSEQgV22ghC4=; b=O1tEB87CS2Vze+wZLHJaZ2fbnnTjosbyH3vaEYoatS+gHYtsD9OsURBY947T3BKpEN PHOjg1HyCeudpPbts53nxzU8enXUq4B9oBQ6dcdolKyfHwEgFhVcFKOoSu2Y7mzeKwIq 2syEh9a4IAgGKXeeyb+NQE3RWSr7xqJPlmVfrD686fD3tyJtbARad/bF/r+1Mg+nCHdp ZQEUNZ2iJ6pPSTGC6K0lR7LxDaLIC+5xfkdXURzsm5izMghUAH/0HNrrUpKXLJ4BBppB 6ZqpfFu235jHVJIfBlWXvxjRdJmpI8dOzGgXVdeOmJP/7ceOOqVOeNCznlyeyswmr6fg SEkw== X-Gm-Message-State: AOAM533sJyE6+2aDRQqmUlSewydxxUBsb5YNoNv7zWgBGcxsKCv4JHP9 +7pKg+pLucgq3DABwPvtjOXPRA== X-Google-Smtp-Source: ABdhPJyQo5a7wag6qg1KfAN2lYACfcQS04kFQ7QD+1uKAF30/6EW8kWHUAmbTGpYPmRfnaUQw+xfFQ== X-Received: by 2002:a0c:e2cd:: with SMTP id t13mr7015227qvl.21.1617854754850; Wed, 07 Apr 2021 21:05:54 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id 207sm18177874qkl.125.2021.04.07.21.05.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Apr 2021 21:05:54 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com Subject: [PATCH v13 12/18] arm64: kexec: relocate in EL1 mode Date: Thu, 8 Apr 2021 00:05:31 -0400 Message-Id: <20210408040537.2703241-13-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210408040537.2703241-1-pasha.tatashin@soleen.com> References: <20210408040537.2703241-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: E9DBC40002CF X-Stat-Signature: 4dr61fpjjtcaf44jqwhpftun5ad14acq X-Rspamd-Server: rspam02 Received-SPF: none (soleen.com>: No applicable sender policy available) receiver=imf26; identity=mailfrom; envelope-from=""; helo=mail-qv1-f48.google.com; client-ip=209.85.219.48 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1617854752-819939 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Since we are going to keep MMU enabled during relocation, we need to keep EL1 mode throughout the relocation. Keep EL1 enabled, and switch EL2 only before enterying the new world. Suggested-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/cpu-reset.h | 3 +-- arch/arm64/kernel/machine_kexec.c | 4 ++-- arch/arm64/kernel/relocate_kernel.S | 13 +++++++++++-- 3 files changed, 14 insertions(+), 6 deletions(-) diff --git a/arch/arm64/kernel/cpu-reset.h b/arch/arm64/kernel/cpu-reset.h index 1922e7a690f8..f6d95512fec6 100644 --- a/arch/arm64/kernel/cpu-reset.h +++ b/arch/arm64/kernel/cpu-reset.h @@ -20,11 +20,10 @@ static inline void __noreturn cpu_soft_restart(unsigned long entry, { typeof(__cpu_soft_restart) *restart; - unsigned long el2_switch = is_hyp_callable(); restart = (void *)__pa_symbol(__cpu_soft_restart); cpu_install_idmap(); - restart(el2_switch, entry, arg0, arg1, arg2); + restart(0, entry, arg0, arg1, arg2); unreachable(); } diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index fb03b6676fb9..d5940b7889f8 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -231,8 +231,8 @@ void machine_kexec(struct kimage *kimage) } else { if (is_hyp_callable()) __hyp_set_vectors(kimage->arch.el2_vectors); - cpu_soft_restart(kimage->arch.kern_reloc, virt_to_phys(kimage), - 0, 0); + cpu_soft_restart(kimage->arch.kern_reloc, + virt_to_phys(kimage), 0, 0); } BUG(); /* Should never get here. */ diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index 36b4496524c3..df023b82544b 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -13,6 +13,7 @@ #include #include #include +#include /* * arm64_relocate_new_kernel - Put a 2nd stage image in place and boot it. @@ -61,12 +62,20 @@ SYM_CODE_START(arm64_relocate_new_kernel) isb /* Start new image. */ + ldr x1, [x0, #KIMAGE_ARCH_EL2_VECTORS] /* relocation start */ + cbz x1, .Lel1 + ldr x1, [x0, #KIMAGE_START] /* relocation start */ + ldr x2, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ + mov x3, xzr + mov x4, xzr + mov x0, #HVC_SOFT_RESTART + hvc #0 /* Jumps from el2 */ +.Lel1: ldr x4, [x0, #KIMAGE_START] /* relocation start */ ldr x0, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ - mov x1, xzr mov x2, xzr mov x3, xzr - br x4 + br x4 /* Jumps from el1 */ SYM_CODE_END(arm64_relocate_new_kernel) .align 3 /* To keep the 64-bit values below naturally aligned. */ From patchwork Thu Apr 8 04:05:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12189953 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EB5ADC433ED for ; Thu, 8 Apr 2021 04:06:07 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 93E86611C0 for ; Thu, 8 Apr 2021 04:06:07 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 93E86611C0 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A5F356B0093; Thu, 8 Apr 2021 00:05:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A0E196B0095; Thu, 8 Apr 2021 00:05:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7C3A66B0096; Thu, 8 Apr 2021 00:05:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0018.hostedemail.com [216.40.44.18]) by kanga.kvack.org (Postfix) with ESMTP id 5C2576B0093 for ; Thu, 8 Apr 2021 00:05:57 -0400 (EDT) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 29B99181C1937 for ; Thu, 8 Apr 2021 04:05:57 +0000 (UTC) X-FDA: 78007861554.27.2E59199 Received: from mail-qv1-f54.google.com (mail-qv1-f54.google.com [209.85.219.54]) by imf06.hostedemail.com (Postfix) with ESMTP id 04627C0007CA for ; Thu, 8 Apr 2021 04:05:57 +0000 (UTC) Received: by mail-qv1-f54.google.com with SMTP id t13so258090qvs.7 for ; Wed, 07 Apr 2021 21:05:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=IBZj7dYh8lQRd+RkarkK6Bs3S98VDCxIDzsjaOHMyWU=; b=XWa9sziZNgfipw2p+fBnm+a/QR10cQILmaujAFu4l6VA+vlVaDGjksM7UsE3OmENKV VNucczfet1qHKZ/5/akhn3GRx6lbaaSgBaCSZf1ydSx8NjIkndwUoAG5HuRdZkLBbVIE MkqHHktYZswwr8dW1Djt/3BBInIEvh2P8lufEAXfFrn/Rt/5VfbusPDdCIF2C7eq/2L8 2pSPqmCoZMW3PzGChT+fE1dIZHjIOMOOPu1mI6AC5dXeRe+orGOzpTfQwf6KA2FBhZ3R t9p1rvZn1c56KaacZ1E2JNlRU2HvqbI8E1ZTAYwCMHw+AlhL8gIv9J8FfSZdnatgxjtl 97kQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=IBZj7dYh8lQRd+RkarkK6Bs3S98VDCxIDzsjaOHMyWU=; b=NZ42oCJ46oSKh2uBDShOoleMN2OIuU6YoclfUx7nYUubFBDtATtXpsDdp69H5ZnLQT 4Cr2lT6/VC3VHjUqXBs45d8SML4oURBNFWpN/+dfNpfIQd1og9Ds4jw6uGT35R03xw9x HmHl/ealPcz52y1ArvPhoW7tLoHPoStwNOhb2oCQcjwS3Kwg9rEQ0IzhLIY9rtG/K+du E5yjnCI93evj2kHW0reLzEx5t3q2K362LfEVD6oC7GAdCqDYxWqIxSaOPdhS01cjez6i ojsGsV3myuR9Oh9B+7alGWIEs6qLtW9RxwNbvjvkYbdvEkXAQIeVc+jn3vMPGWGoz97U Mf7A== X-Gm-Message-State: AOAM531PFNL1qBtZO7/QpfA+XPHTSAx0SDFG2ZkCEo8zX/lZnx9+8T5U D/7Ipwe6n4othL8z+NGHrZIqhA== X-Google-Smtp-Source: ABdhPJw4OtlQ8Oa4W1fxbuBoRRZz0Ihai6LuM5gpbSIyFOGLtDUBu9w/nYGR7vBoD5f7IcL9Em6JfQ== X-Received: by 2002:a05:6214:aa6:: with SMTP id ew6mr6754949qvb.2.1617854756198; Wed, 07 Apr 2021 21:05:56 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id 207sm18177874qkl.125.2021.04.07.21.05.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Apr 2021 21:05:55 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com Subject: [PATCH v13 13/18] arm64: kexec: use ld script for relocation function Date: Thu, 8 Apr 2021 00:05:32 -0400 Message-Id: <20210408040537.2703241-14-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210408040537.2703241-1-pasha.tatashin@soleen.com> References: <20210408040537.2703241-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 04627C0007CA X-Stat-Signature: 9sqj3jzfhuf9za8nsd6teqs3u696tyha Received-SPF: none (soleen.com>: No applicable sender policy available) receiver=imf06; identity=mailfrom; envelope-from=""; helo=mail-qv1-f54.google.com; client-ip=209.85.219.54 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1617854757-223681 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently, relocation code declares start and end variables which are used to compute its size. The better way to do this is to use ld script incited, and put relocation function in its own section. Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/sections.h | 1 + arch/arm64/kernel/machine_kexec.c | 14 ++++++-------- arch/arm64/kernel/relocate_kernel.S | 15 ++------------- arch/arm64/kernel/vmlinux.lds.S | 19 +++++++++++++++++++ 4 files changed, 28 insertions(+), 21 deletions(-) diff --git a/arch/arm64/include/asm/sections.h b/arch/arm64/include/asm/sections.h index 2f36b16a5b5d..31e459af89f6 100644 --- a/arch/arm64/include/asm/sections.h +++ b/arch/arm64/include/asm/sections.h @@ -20,5 +20,6 @@ extern char __exittext_begin[], __exittext_end[]; extern char __irqentry_text_start[], __irqentry_text_end[]; extern char __mmuoff_data_start[], __mmuoff_data_end[]; extern char __entry_tramp_text_start[], __entry_tramp_text_end[]; +extern char __relocate_new_kernel_start[], __relocate_new_kernel_end[]; #endif /* __ASM_SECTIONS_H */ diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index d5940b7889f8..f1451d807708 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -20,14 +20,11 @@ #include #include #include +#include #include #include "cpu-reset.h" -/* Global variables for the arm64_relocate_new_kernel routine. */ -extern const unsigned char arm64_relocate_new_kernel[]; -extern const unsigned long arm64_relocate_new_kernel_size; - /** * kexec_image_info - For debugging output. */ @@ -157,6 +154,7 @@ static void *kexec_page_alloc(void *arg) int machine_kexec_post_load(struct kimage *kimage) { void *reloc_code = page_to_virt(kimage->control_code_page); + long reloc_size; struct trans_pgd_info info = { .trans_alloc_page = kexec_page_alloc, .trans_alloc_arg = kimage, @@ -177,14 +175,14 @@ int machine_kexec_post_load(struct kimage *kimage) return rc; } - memcpy(reloc_code, arm64_relocate_new_kernel, - arm64_relocate_new_kernel_size); + reloc_size = __relocate_new_kernel_end - __relocate_new_kernel_start; + memcpy(reloc_code, __relocate_new_kernel_start, reloc_size); kimage->arch.kern_reloc = __pa(reloc_code); /* Flush the reloc_code in preparation for its execution. */ - __flush_dcache_area(reloc_code, arm64_relocate_new_kernel_size); + __flush_dcache_area(reloc_code, reloc_size); flush_icache_range((uintptr_t)reloc_code, (uintptr_t)reloc_code + - arm64_relocate_new_kernel_size); + reloc_size); kexec_list_flush(kimage); kexec_image_info(kimage); diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index df023b82544b..7a600ba33ae1 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -15,6 +15,7 @@ #include #include +.pushsection ".kexec_relocate.text", "ax" /* * arm64_relocate_new_kernel - Put a 2nd stage image in place and boot it. * @@ -77,16 +78,4 @@ SYM_CODE_START(arm64_relocate_new_kernel) mov x3, xzr br x4 /* Jumps from el1 */ SYM_CODE_END(arm64_relocate_new_kernel) - -.align 3 /* To keep the 64-bit values below naturally aligned. */ - -.Lcopy_end: -.org KEXEC_CONTROL_PAGE_SIZE - -/* - * arm64_relocate_new_kernel_size - Number of bytes to copy to the - * control_code_page. - */ -.globl arm64_relocate_new_kernel_size -arm64_relocate_new_kernel_size: - .quad .Lcopy_end - arm64_relocate_new_kernel +.popsection diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index 7eea7888bb02..0d9d5e6af66f 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -12,6 +12,7 @@ #include #include #include +#include #include #include @@ -92,6 +93,16 @@ jiffies = jiffies_64; #define HIBERNATE_TEXT #endif +#ifdef CONFIG_KEXEC_CORE +#define KEXEC_TEXT \ + . = ALIGN(SZ_4K); \ + __relocate_new_kernel_start = .; \ + *(.kexec_relocate.text) \ + __relocate_new_kernel_end = .; +#else +#define KEXEC_TEXT +#endif + #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 #define TRAMP_TEXT \ . = ALIGN(PAGE_SIZE); \ @@ -152,6 +163,7 @@ SECTIONS HYPERVISOR_TEXT IDMAP_TEXT HIBERNATE_TEXT + KEXEC_TEXT TRAMP_TEXT *(.fixup) *(.gnu.warning) @@ -336,3 +348,10 @@ ASSERT(swapper_pg_dir - reserved_pg_dir == RESERVED_SWAPPER_OFFSET, ASSERT(swapper_pg_dir - tramp_pg_dir == TRAMP_SWAPPER_OFFSET, "TRAMP_SWAPPER_OFFSET is wrong!") #endif + +#ifdef CONFIG_KEXEC_CORE +/* kexec relocation code should fit into one KEXEC_CONTROL_PAGE_SIZE */ +ASSERT(__relocate_new_kernel_end - (__relocate_new_kernel_start & ~(SZ_4K - 1)) + <= SZ_4K, "kexec relocation code is too big or misaligned") +ASSERT(KEXEC_CONTROL_PAGE_SIZE >= SZ_4K, "KEXEC_CONTROL_PAGE_SIZE is brokern") +#endif From patchwork Thu Apr 8 04:05:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12189955 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A8471C433B4 for ; Thu, 8 Apr 2021 04:06:10 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4EE57611C0 for ; Thu, 8 Apr 2021 04:06:10 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4EE57611C0 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5C92F6B0095; Thu, 8 Apr 2021 00:06:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 52F0D6B0096; Thu, 8 Apr 2021 00:06:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 330306B0098; Thu, 8 Apr 2021 00:06:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0120.hostedemail.com [216.40.44.120]) by kanga.kvack.org (Postfix) with ESMTP id 169686B0095 for ; Thu, 8 Apr 2021 00:06:00 -0400 (EDT) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id C78A1181C1937 for ; Thu, 8 Apr 2021 04:05:59 +0000 (UTC) X-FDA: 78007861638.01.6C88093 Received: from mail-qt1-f169.google.com (mail-qt1-f169.google.com [209.85.160.169]) by imf07.hostedemail.com (Postfix) with ESMTP id 2EB2EA000391 for ; Thu, 8 Apr 2021 04:05:59 +0000 (UTC) Received: by mail-qt1-f169.google.com with SMTP id f12so472299qtf.2 for ; Wed, 07 Apr 2021 21:05:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=2MwMRNHUNcBKpMF1hToORmuQtXmVKMCROcbjizX1Pao=; b=fpo5AOYCGm3rnQMF4k4Kep/dfrnGm0E++/dJ0VrKIb2VHU6OudZV1VTAwgm8LinIqD yxDQzUb6g4ZO7W1zZuk5Xy1KCgLYe0bDuxlE1I85vxW8d11idj7Koq+4BwzP1LqXvRkT 2dMKemIvcTZ0HaQwfZtVjjuH9s1RAs2kq4RBntnYZxjJOey8rqC25tRwXrVjQzgjG1tZ ukpM0SxnmPuiC/eHb6fb/k9eolxaVTtBTwI9qiJ1Ol/j2GzD4T9XnqADIa0xrddwtB1Z tx2k9hhtZcvliRagOHSKFqvfv1hCu+4aDq7lVINevfwTKrZ0jmPaYk9rLS/UjlnbLCLd mwzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2MwMRNHUNcBKpMF1hToORmuQtXmVKMCROcbjizX1Pao=; b=ZlRex7+njJ+JA7uEcnqzo9NmvGcATl/4HCj5PPBpVRm/uoosiw3+uT3wQ5If13RAYE enkD+GE6fH/B/caCR5ldaIe8OddlELft9sdQkUVA3aZHUu9++JNK4Uq+xWAq92BYWeDJ RXbptvxlXw5C4KJsXO6AAp+Gep6flTeHJneevfB7kheTelNFq9qfh9eHwNZH3/WiMwFv oiOnaiToIm02qsHIS1TqCHswL6Mr5SsZHjeWtVOnf8EwCbTpEd9gOPubtSU4bcKRKZK4 u3x+65Lh1oimHEr+YYCLNyGlnaXZD6tJJB17LHhaCNZ8l19WW6hxTrG7T+NfzyH9ifi7 VPbw== X-Gm-Message-State: AOAM530zTob3DG8VeQoouAyRntVNIGSLM47V7LcY4V/nCvTtcvkaZRN+ IJtJnygXadEKXM7TMTuq7g7PfQ== X-Google-Smtp-Source: ABdhPJxVaJ3kci0H5jEVPHKEH/dGd4Hj0zfNx1cig37iSok8prwWbLZDkAcKO+o7C9aiRkKogLnWYA== X-Received: by 2002:ac8:5c95:: with SMTP id r21mr5804477qta.347.1617854758731; Wed, 07 Apr 2021 21:05:58 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id 207sm18177874qkl.125.2021.04.07.21.05.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Apr 2021 21:05:58 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com Subject: [PATCH v13 15/18] arm64: kexec: keep MMU enabled during kexec relocation Date: Thu, 8 Apr 2021 00:05:34 -0400 Message-Id: <20210408040537.2703241-16-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210408040537.2703241-1-pasha.tatashin@soleen.com> References: <20210408040537.2703241-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Stat-Signature: uhb6wjut59cixnb1898whne7jpm5m6da X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 2EB2EA000391 Received-SPF: none (soleen.com>: No applicable sender policy available) receiver=imf07; identity=mailfrom; envelope-from=""; helo=mail-qt1-f169.google.com; client-ip=209.85.160.169 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1617854759-271477 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Now, that we have linear map page tables configured, keep MMU enabled to allow faster relocation of segments to final destination. Cavium ThunderX2: Kernel Image size: 38M Iniramfs size: 46M Total relocation size: 84M MMU-disabled: relocation 7.489539915s MMU-enabled: relocation 0.03946095s Broadcom Stingray: The performance data: for a moderate size kernel + initramfs: 25M the relocation was taking 0.382s, with enabled MMU it now takes 0.019s only or x20 improvement. The time is proportional to the size of relocation, therefore if initramfs is larger, 100M it could take over a second. Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/kexec.h | 3 +++ arch/arm64/kernel/asm-offsets.c | 1 + arch/arm64/kernel/machine_kexec.c | 16 ++++++++++---- arch/arm64/kernel/relocate_kernel.S | 33 +++++++++++++++++++---------- 4 files changed, 38 insertions(+), 15 deletions(-) diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h index 59ac166daf53..5fc87b51f8a9 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -97,8 +97,11 @@ struct kimage_arch { phys_addr_t dtb_mem; phys_addr_t kern_reloc; phys_addr_t el2_vectors; + phys_addr_t ttbr0; phys_addr_t ttbr1; phys_addr_t zero_page; + unsigned long phys_offset; + unsigned long t0sz; /* Core ELF header buffer */ void *elf_headers; unsigned long elf_headers_mem; diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index 609362b5aa76..ec7bb80aedc8 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -159,6 +159,7 @@ int main(void) DEFINE(KIMAGE_ARCH_DTB_MEM, offsetof(struct kimage, arch.dtb_mem)); DEFINE(KIMAGE_ARCH_EL2_VECTORS, offsetof(struct kimage, arch.el2_vectors)); DEFINE(KIMAGE_ARCH_ZERO_PAGE, offsetof(struct kimage, arch.zero_page)); + DEFINE(KIMAGE_ARCH_PHYS_OFFSET, offsetof(struct kimage, arch.phys_offset)); DEFINE(KIMAGE_ARCH_TTBR1, offsetof(struct kimage, arch.ttbr1)); DEFINE(KIMAGE_HEAD, offsetof(struct kimage, head)); DEFINE(KIMAGE_START, offsetof(struct kimage, start)); diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index c875ef522e53..d5c8aefc66f3 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -190,6 +190,11 @@ int machine_kexec_post_load(struct kimage *kimage) reloc_size = __relocate_new_kernel_end - __relocate_new_kernel_start; memcpy(reloc_code, __relocate_new_kernel_start, reloc_size); kimage->arch.kern_reloc = __pa(reloc_code); + rc = trans_pgd_idmap_page(&info, &kimage->arch.ttbr0, + &kimage->arch.t0sz, reloc_code); + if (rc) + return rc; + kimage->arch.phys_offset = virt_to_phys(kimage) - (long)kimage; /* Flush the reloc_code in preparation for its execution. */ __flush_dcache_area(reloc_code, reloc_size); @@ -223,9 +228,9 @@ void machine_kexec(struct kimage *kimage) local_daif_mask(); /* - * Both restart and cpu_soft_restart will shutdown the MMU, disable data + * Both restart and kernel_reloc will shutdown the MMU, disable data * caches. However, restart will start new kernel or purgatory directly, - * cpu_soft_restart will transfer control to arm64_relocate_new_kernel + * kernel_reloc contains the body of arm64_relocate_new_kernel * In kexec case, kimage->start points to purgatory assuming that * kernel entry and dtb address are embedded in purgatory by * userspace (kexec-tools). @@ -239,10 +244,13 @@ void machine_kexec(struct kimage *kimage) restart(is_hyp_callable(), kimage->start, kimage->arch.dtb_mem, 0, 0); } else { + void (*kernel_reloc)(struct kimage *kimage); + if (is_hyp_callable()) __hyp_set_vectors(kimage->arch.el2_vectors); - cpu_soft_restart(kimage->arch.kern_reloc, - virt_to_phys(kimage), 0, 0); + cpu_install_ttbr0(kimage->arch.ttbr0, kimage->arch.t0sz); + kernel_reloc = (void *)kimage->arch.kern_reloc; + kernel_reloc(kimage); } BUG(); /* Should never get here. */ diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index e83b6380907d..433a57b3d76e 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -4,6 +4,8 @@ * * Copyright (C) Linaro. * Copyright (C) Huawei Futurewei Technologies. + * Copyright (C) 2020, Microsoft Corporation. + * Pavel Tatashin */ #include @@ -15,6 +17,15 @@ #include #include +.macro turn_off_mmu tmp1, tmp2 + mrs \tmp1, sctlr_el1 + mov_q \tmp2, SCTLR_ELx_FLAGS + bic \tmp1, \tmp1, \tmp2 + pre_disable_mmu_workaround + msr sctlr_el1, \tmp1 + isb +.endm + .pushsection ".kexec_relocate.text", "ax" /* * arm64_relocate_new_kernel - Put a 2nd stage image in place and boot it. @@ -32,22 +43,21 @@ SYM_CODE_START(arm64_relocate_new_kernel) ldr x18, [x0, #KIMAGE_ARCH_ZERO_PAGE] /* x18 = zero page for BBM */ ldr x17, [x0, #KIMAGE_ARCH_TTBR1] /* x17 = linear map copy */ ldr x16, [x0, #KIMAGE_HEAD] /* x16 = kimage_head */ - mov x14, xzr /* x14 = entry ptr */ - mov x13, xzr /* x13 = copy dest */ + ldr x22, [x0, #KIMAGE_ARCH_PHYS_OFFSET] /* x22 phys_offset */ raw_dcache_line_size x15, x1 /* x15 = dcache line size */ break_before_make_ttbr_switch x18, x17, x1, x2 /* set linear map */ .Lloop: and x12, x16, PAGE_MASK /* x12 = addr */ - + sub x12, x12, x22 /* Convert x12 to virt */ /* Test the entry flags. */ .Ltest_source: tbz x16, IND_SOURCE_BIT, .Ltest_indirection /* Invalidate dest page to PoC. */ - mov x2, x13 - mov x1, #PAGE_SIZE - dcache_by_myline_op ivac, sy, x2, x1, x15, x20 + mov x19, x13 copy_page x13, x12, x1, x2, x3, x4, x5, x6, x7, x8 + mov x1, #PAGE_SIZE + dcache_by_myline_op civac, sy, x19, x1, x15, x20 b .Lnext .Ltest_indirection: tbz x16, IND_INDIRECTION_BIT, .Ltest_destination @@ -64,19 +74,20 @@ SYM_CODE_START(arm64_relocate_new_kernel) ic iallu dsb nsh isb + ldr x4, [x0, #KIMAGE_START] /* relocation start */ + ldr x1, [x0, #KIMAGE_ARCH_EL2_VECTORS] /* relocation start */ + ldr x0, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ + turn_off_mmu x12, x13 /* Start new image. */ - ldr x1, [x0, #KIMAGE_ARCH_EL2_VECTORS] /* relocation start */ cbz x1, .Lel1 - ldr x1, [x0, #KIMAGE_START] /* relocation start */ - ldr x2, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ + mov x1, x4 /* relocation start */ + mov x2, x0 /* dtb address */ mov x3, xzr mov x4, xzr mov x0, #HVC_SOFT_RESTART hvc #0 /* Jumps from el2 */ .Lel1: - ldr x4, [x0, #KIMAGE_START] /* relocation start */ - ldr x0, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ mov x2, xzr mov x3, xzr br x4 /* Jumps from el1 */ From patchwork Thu Apr 8 04:05:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12189957 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 51037C433B4 for ; Thu, 8 Apr 2021 04:06:13 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id F142E611BD for ; Thu, 8 Apr 2021 04:06:12 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F142E611BD Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id AB06A6B0096; Thu, 8 Apr 2021 00:06:01 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9E9286B0098; Thu, 8 Apr 2021 00:06:01 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8134F6B0099; Thu, 8 Apr 2021 00:06:01 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0072.hostedemail.com [216.40.44.72]) by kanga.kvack.org (Postfix) with ESMTP id 638A06B0096 for ; Thu, 8 Apr 2021 00:06:01 -0400 (EDT) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 314783632 for ; Thu, 8 Apr 2021 04:06:01 +0000 (UTC) X-FDA: 78007861722.16.BB671AE Received: from mail-qk1-f177.google.com (mail-qk1-f177.google.com [209.85.222.177]) by imf12.hostedemail.com (Postfix) with ESMTP id 87808D2 for ; Thu, 8 Apr 2021 04:05:57 +0000 (UTC) Received: by mail-qk1-f177.google.com with SMTP id z10so896438qkz.13 for ; Wed, 07 Apr 2021 21:06:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=mQmK3SsBkKt3YHt5HRugbQ4C32Fw5itH9aDNQf+nmJI=; b=elWVu7ElduDgar7M15pPxIjNtGuFk2IxD56SO/LLqYMadH0bLeFPVdDJrrgXHMQQHY 5O+hPoZGbI2/fKrRlolSphZQemU4b8fO04rT4MR4ge8GzoS//g8BcNoHJnDWpxtXN6m6 K4yt0Ll7DaykR8Tyr8MCv2ALdhZZld2Z8v2GxISaLhL5P8n+yqVTAzsr885OTW/r/CaI ZWy3CSy0Jkc5aLPCQK2fFbp0qgOEgsvZVoIRkykT6rD/cRQui4N8LS/GWlBbLALSY4Jx +gOmt9BWnaITKKFW5XLWpNvETX8hDJzB/44NqbVBwH4mg3CMwDb3PBBB0DlOUtBELbj+ 7j3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=mQmK3SsBkKt3YHt5HRugbQ4C32Fw5itH9aDNQf+nmJI=; b=LFLY+2ODi4IK+tISN2crZP9h1cnm+VAQQI9Cbma7P3btER28ENoU/PaGrKtAePBtcG tACjIdJdEbpAhbqBVRY1Q1g+7vhhJsk8ItJniArI0V9FXUpHzK2SV8Y0mLXtXq6FTqfq iFdnEsuraj+eHPo+mRbpfeXI1tGe974cDMLji1aLqiDMmt4Jv3gfV/aOhe15+DwRVCYb 3/cZWo3FQvKHYaOHC85pF+X/jFvGaLUPxgL+rBf/dvjFWqb/oR8tQux8p1cvMVLfZMLY 3lMlGDFNKgcotsCm6/qHayri6O74ulNAMQpEPTcZz+knp24F4mejukPgTmi+pl2IkOqE VkYQ== X-Gm-Message-State: AOAM532BsUa76Wnp2WnChgWMWdz9URr9/+es/gl7HqmZnGPRQIQOaW7u IHlPfd5uhHh1UswaOS5pwczU0Q== X-Google-Smtp-Source: ABdhPJw8BJbPk7oxJbt5TiRHY20STpCei8pN80U8MVBJiPqRcb2nWUnW5QpsnWL/n+1r5eSXF613pQ== X-Received: by 2002:a37:c92:: with SMTP id 140mr6714935qkm.177.1617854760170; Wed, 07 Apr 2021 21:06:00 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id 207sm18177874qkl.125.2021.04.07.21.05.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Apr 2021 21:05:59 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com Subject: [PATCH v13 16/18] arm64: kexec: remove the pre-kexec PoC maintenance Date: Thu, 8 Apr 2021 00:05:35 -0400 Message-Id: <20210408040537.2703241-17-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210408040537.2703241-1-pasha.tatashin@soleen.com> References: <20210408040537.2703241-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 87808D2 X-Stat-Signature: hp1drzmzw68koq1rkby7xksnbti99z1x Received-SPF: none (soleen.com>: No applicable sender policy available) receiver=imf12; identity=mailfrom; envelope-from=""; helo=mail-qk1-f177.google.com; client-ip=209.85.222.177 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1617854757-632303 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Now that kexec does its relocations with the MMU enabled, we no longer need to clean the relocation data to the PoC. Co-developed-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/machine_kexec.c | 40 ------------------------------- 1 file changed, 40 deletions(-) diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index d5c8aefc66f3..a1c9bee0cddd 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -76,45 +76,6 @@ int machine_kexec_prepare(struct kimage *kimage) return 0; } -/** - * kexec_list_flush - Helper to flush the kimage list and source pages to PoC. - */ -static void kexec_list_flush(struct kimage *kimage) -{ - kimage_entry_t *entry; - - __flush_dcache_area(kimage, sizeof(*kimage)); - - for (entry = &kimage->head; ; entry++) { - unsigned int flag; - void *addr; - - /* flush the list entries. */ - __flush_dcache_area(entry, sizeof(kimage_entry_t)); - - flag = *entry & IND_FLAGS; - if (flag == IND_DONE) - break; - - addr = phys_to_virt(*entry & PAGE_MASK); - - switch (flag) { - case IND_INDIRECTION: - /* Set entry point just before the new list page. */ - entry = (kimage_entry_t *)addr - 1; - break; - case IND_SOURCE: - /* flush the source pages. */ - __flush_dcache_area(addr, PAGE_SIZE); - break; - case IND_DESTINATION: - break; - default: - BUG(); - } - } -} - /** * kexec_segment_flush - Helper to flush the kimage segments to PoC. */ @@ -200,7 +161,6 @@ int machine_kexec_post_load(struct kimage *kimage) __flush_dcache_area(reloc_code, reloc_size); flush_icache_range((uintptr_t)reloc_code, (uintptr_t)reloc_code + reloc_size); - kexec_list_flush(kimage); kexec_image_info(kimage); return 0; From patchwork Thu Apr 8 04:05:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12189959 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E82D9C433ED for ; Thu, 8 Apr 2021 04:06:15 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7747761157 for ; Thu, 8 Apr 2021 04:06:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7747761157 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 61D2E6B0098; Thu, 8 Apr 2021 00:06:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 555516B0099; Thu, 8 Apr 2021 00:06:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2E2466B009A; Thu, 8 Apr 2021 00:06:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0185.hostedemail.com [216.40.44.185]) by kanga.kvack.org (Postfix) with ESMTP id 0FB386B0098 for ; Thu, 8 Apr 2021 00:06:03 -0400 (EDT) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id CA8D0181C65C0 for ; Thu, 8 Apr 2021 04:06:02 +0000 (UTC) X-FDA: 78007861764.27.2692CDA Received: from mail-qk1-f169.google.com (mail-qk1-f169.google.com [209.85.222.169]) by imf20.hostedemail.com (Postfix) with ESMTP id 54CACD6 for ; Thu, 8 Apr 2021 04:05:59 +0000 (UTC) Received: by mail-qk1-f169.google.com with SMTP id 7so927962qka.7 for ; Wed, 07 Apr 2021 21:06:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=xvI3SX7JlP3yoyw6A9a93CvHYXbBacBuW7cN/tDXEXM=; b=PQZpDOWvV+YQX3+RRvSpQFtcUJedqJnM7spQTWj55AWeUxKM7SDx4kPAPVhIti6gUy voTZFJ0QpgeKNepc2Wsw2crxkyVzSebcYCpjVmhiUJHU0EF8Cp2eZYaXKcFetCaQ0nbO 9sgz4EynwexL5fBFi5NdfqUX3KBkDnkUvWB2XRI6lXQ6AD58/hESTOEhAo/WhJJNVKKB XRS9V60S3C0yHEY1oJGn5dzDVLybPzxpWkBY5tt1iidF+UOGgqX3qTEnTHcav/dlpF1D 5IWEsW8FOz8Ysa5jWkyRyxS4DJNl79APieTpJKpMGJjApk1t3Y7Drx2OLvk1ksW2Ww8R hdZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=xvI3SX7JlP3yoyw6A9a93CvHYXbBacBuW7cN/tDXEXM=; b=Ecn3blN37+6Ir/qgYSKCsCFe2UwfO0adss+SoYS74WBYzPGsk7eAx5YdYP9mjpfwEt HKRdKdOLein3tSAyaew9DT6uFl1oJfUXrEmpa0UG3ZrV4JLjeRQzHY0Yu0+0SRkMtulg sVDQxluhy51+uLtttZXknmgK2z4W+Wh9RiKKVnb9AnpHfE2jlK6IWDCnXiySla+42QNr U8Q+OplMElTt171QX5e7z0f0u5ZbIfhlxUIjJpAijdMnn6gcpbRBwIs+o7ETHCO1CSWx JmTAxlkgT427vj8Annq7TOdiyiHdjZ956o6UaAc/7tMP8zhBUrth9n/pfoOyByf5sXrz BXPw== X-Gm-Message-State: AOAM533sT5UbRafvinnaVy8VkoKWwheysX+tWMszQewYmHb1rG3UEe/S GbmUikAu9ORBInrwVl6ZROMM1Q== X-Google-Smtp-Source: ABdhPJyAhGZ8OdBbF6Xwq/FrEvRsf0ZaW/p58HnrVWenn+xQ4Ms5P4taI+EFYNUH/uxAa8e26zbECg== X-Received: by 2002:a37:89c7:: with SMTP id l190mr7002416qkd.361.1617854761446; Wed, 07 Apr 2021 21:06:01 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id 207sm18177874qkl.125.2021.04.07.21.06.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Apr 2021 21:06:01 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com Subject: [PATCH v13 17/18] arm64: kexec: Remove cpu-reset.h Date: Thu, 8 Apr 2021 00:05:36 -0400 Message-Id: <20210408040537.2703241-18-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210408040537.2703241-1-pasha.tatashin@soleen.com> References: <20210408040537.2703241-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Stat-Signature: kkssceoe966i4c1e4arjmh3aqk3h1hab X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 54CACD6 Received-SPF: none (soleen.com>: No applicable sender policy available) receiver=imf20; identity=mailfrom; envelope-from=""; helo=mail-qk1-f169.google.com; client-ip=209.85.222.169 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1617854759-251433 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This header contains only cpu_soft_restart() which is never used directly anymore. So, remove this header, and rename the helper to be cpu_soft_restart(). Suggested-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/kexec.h | 6 ++++++ arch/arm64/kernel/cpu-reset.S | 7 +++---- arch/arm64/kernel/cpu-reset.h | 30 ------------------------------ arch/arm64/kernel/machine_kexec.c | 6 ++---- 4 files changed, 11 insertions(+), 38 deletions(-) delete mode 100644 arch/arm64/kernel/cpu-reset.h diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h index 5fc87b51f8a9..ee71ae3b93ed 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -90,6 +90,12 @@ static inline void crash_prepare_suspend(void) {} static inline void crash_post_resume(void) {} #endif +#if defined(CONFIG_KEXEC_CORE) +void cpu_soft_restart(unsigned long el2_switch, unsigned long entry, + unsigned long arg0, unsigned long arg1, + unsigned long arg2); +#endif + #define ARCH_HAS_KIMAGE_ARCH struct kimage_arch { diff --git a/arch/arm64/kernel/cpu-reset.S b/arch/arm64/kernel/cpu-reset.S index 37721eb6f9a1..5d47d6c92634 100644 --- a/arch/arm64/kernel/cpu-reset.S +++ b/arch/arm64/kernel/cpu-reset.S @@ -16,8 +16,7 @@ .pushsection .idmap.text, "awx" /* - * __cpu_soft_restart(el2_switch, entry, arg0, arg1, arg2) - Helper for - * cpu_soft_restart. + * cpu_soft_restart(el2_switch, entry, arg0, arg1, arg2) * * @el2_switch: Flag to indicate a switch to EL2 is needed. * @entry: Location to jump to for soft reset. @@ -29,7 +28,7 @@ * branch to what would be the reset vector. It must be executed with the * flat identity mapping. */ -SYM_CODE_START(__cpu_soft_restart) +SYM_CODE_START(cpu_soft_restart) /* Clear sctlr_el1 flags. */ mrs x12, sctlr_el1 mov_q x13, SCTLR_ELx_FLAGS @@ -51,6 +50,6 @@ SYM_CODE_START(__cpu_soft_restart) mov x1, x3 // arg1 mov x2, x4 // arg2 br x8 -SYM_CODE_END(__cpu_soft_restart) +SYM_CODE_END(cpu_soft_restart) .popsection diff --git a/arch/arm64/kernel/cpu-reset.h b/arch/arm64/kernel/cpu-reset.h deleted file mode 100644 index f6d95512fec6..000000000000 --- a/arch/arm64/kernel/cpu-reset.h +++ /dev/null @@ -1,30 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0-only */ -/* - * CPU reset routines - * - * Copyright (C) 2015 Huawei Futurewei Technologies. - */ - -#ifndef _ARM64_CPU_RESET_H -#define _ARM64_CPU_RESET_H - -#include - -void __cpu_soft_restart(unsigned long el2_switch, unsigned long entry, - unsigned long arg0, unsigned long arg1, unsigned long arg2); - -static inline void __noreturn cpu_soft_restart(unsigned long entry, - unsigned long arg0, - unsigned long arg1, - unsigned long arg2) -{ - typeof(__cpu_soft_restart) *restart; - - restart = (void *)__pa_symbol(__cpu_soft_restart); - - cpu_install_idmap(); - restart(0, entry, arg0, arg1, arg2); - unreachable(); -} - -#endif diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index a1c9bee0cddd..ef7ba93f2bd6 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -23,8 +23,6 @@ #include #include -#include "cpu-reset.h" - /** * kexec_image_info - For debugging output. */ @@ -197,10 +195,10 @@ void machine_kexec(struct kimage *kimage) * In kexec_file case, the kernel starts directly without purgatory. */ if (kimage->head & IND_DONE) { - typeof(__cpu_soft_restart) *restart; + typeof(cpu_soft_restart) *restart; cpu_install_idmap(); - restart = (void *)__pa_symbol(__cpu_soft_restart); + restart = (void *)__pa_symbol(cpu_soft_restart); restart(is_hyp_callable(), kimage->start, kimage->arch.dtb_mem, 0, 0); } else { From patchwork Thu Apr 8 04:05:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12189961 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7FA5CC433B4 for ; Thu, 8 Apr 2021 04:06:18 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2CF58611BD for ; Thu, 8 Apr 2021 04:06:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2CF58611BD Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5A3CF6B0099; Thu, 8 Apr 2021 00:06:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 553126B009A; Thu, 8 Apr 2021 00:06:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 331476B009B; Thu, 8 Apr 2021 00:06:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0205.hostedemail.com [216.40.44.205]) by kanga.kvack.org (Postfix) with ESMTP id 0B9AC6B0099 for ; Thu, 8 Apr 2021 00:06:04 -0400 (EDT) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id C6F21181C1937 for ; Thu, 8 Apr 2021 04:06:03 +0000 (UTC) X-FDA: 78007861806.05.DB17FD0 Received: from mail-qk1-f170.google.com (mail-qk1-f170.google.com [209.85.222.170]) by imf12.hostedemail.com (Postfix) with ESMTP id 2791DDA for ; Thu, 8 Apr 2021 04:06:00 +0000 (UTC) Received: by mail-qk1-f170.google.com with SMTP id 7so927986qka.7 for ; Wed, 07 Apr 2021 21:06:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=KXQ+yohNMyyqRPEeNONrEzPBCefWBOBuVaWouKl2Q04=; b=hEpX6glagkFAuErJ7yQUGxGTKuEwhpiBYJUVY+mF4vPQ9e8cKN4TsDEZ5/+iZBwsv5 iB7RU8pOFfBpYoimzL4WA8WK8J/rXapc5IOLHDQb+zE0bhEap5LZlBQxZuFhwcGMK63S 2DQBTm9SWhcwg2HWWqUv827UrKERhGot15yaSZtIj1uQK6RO2b2ktv7D9pO2jne7A5vv ioXYlqv1fYjtRrxVNgpy9PO4FG3bavAsYW2etVeizyl2QlU2er0E3nb45su4gKMq6UJc NOMZiK2RWQiRXhdFaS78R4rXW+aveD9Z/y6CN3XhfaGVu+dSgOYUiObLZaVkrWU6WsSz c4Lw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KXQ+yohNMyyqRPEeNONrEzPBCefWBOBuVaWouKl2Q04=; b=J7Jk/wcCv4uSkv9hJ6E2sZdNhK2lY3D9vbYPTq6nsT6h+WRh2jsx9GWPBz8RDYRyi5 7soEAGTaadAJsGV+JKIZeKm5D1dyjpoPKHNek5vxkJAFoMFjCJz7GZM8HjH90EKTOFSl X2iY8THwBsbkVGYr8hj16+zDTWQHgfmZ1WUNLBTNetyS7RwbylPisSLmfhX7C9BLyfiH V5TygoxZrhMpA0XbpzS6FbU+i+jh/yo0fz2PLPVs/ltsrc/vXJJe6PIc7ZHVaCd+vCQT mbD+NBICsW6sxDGQjEfq0EWzyRbfwQ5vRDmruM5Gsbvug00NY5b8SwsiGBg4SE1GC1M8 aP6A== X-Gm-Message-State: AOAM533uIxwUsqWTOpoVI9Vzof6igrBoYct+lGH/AQj3m/EvhgLcj0Y0 FUvE2P36MbNzRSwnkiLkmn5hYqf7kpqhog== X-Google-Smtp-Source: ABdhPJxRILjhMfqmrTfTbUSlaG1IKDkZR0nrqdNhTJMhzMiqtQkU7L/C22a+USt80/gnZrWOBcd4Zg== X-Received: by 2002:a37:9a05:: with SMTP id c5mr6549116qke.16.1617854762799; Wed, 07 Apr 2021 21:06:02 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id 207sm18177874qkl.125.2021.04.07.21.06.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Apr 2021 21:06:02 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com Subject: [PATCH v13 18/18] arm64/mm: remove useless trans_pgd_map_page() Date: Thu, 8 Apr 2021 00:05:37 -0400 Message-Id: <20210408040537.2703241-19-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210408040537.2703241-1-pasha.tatashin@soleen.com> References: <20210408040537.2703241-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Stat-Signature: kbhhfrqcgchpen5motbqgp9frhpazckb X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 2791DDA Received-SPF: none (soleen.com>: No applicable sender policy available) receiver=imf12; identity=mailfrom; envelope-from=""; helo=mail-qk1-f170.google.com; client-ip=209.85.222.170 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1617854760-755204 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Pingfan Liu The intend of trans_pgd_map_page() was to map contigous range of VA memory to the memory that is getting relocated during kexec. However, since we are now using linear map instead of contigous range this function is not needed Signed-off-by: Pingfan Liu [Changed commit message] Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/trans_pgd.h | 5 +-- arch/arm64/mm/trans_pgd.c | 57 ------------------------------ 2 files changed, 1 insertion(+), 61 deletions(-) diff --git a/arch/arm64/include/asm/trans_pgd.h b/arch/arm64/include/asm/trans_pgd.h index e0760e52d36d..234353df2f13 100644 --- a/arch/arm64/include/asm/trans_pgd.h +++ b/arch/arm64/include/asm/trans_pgd.h @@ -15,7 +15,7 @@ /* * trans_alloc_page * - Allocator that should return exactly one zeroed page, if this - * allocator fails, trans_pgd_create_copy() and trans_pgd_map_page() + * allocator fails, trans_pgd_create_copy() and trans_pgd_idmap_page() * return -ENOMEM error. * * trans_alloc_arg @@ -30,9 +30,6 @@ struct trans_pgd_info { int trans_pgd_create_copy(struct trans_pgd_info *info, pgd_t **trans_pgd, unsigned long start, unsigned long end); -int trans_pgd_map_page(struct trans_pgd_info *info, pgd_t *trans_pgd, - void *page, unsigned long dst_addr, pgprot_t pgprot); - int trans_pgd_idmap_page(struct trans_pgd_info *info, phys_addr_t *trans_ttbr0, unsigned long *t0sz, void *page); diff --git a/arch/arm64/mm/trans_pgd.c b/arch/arm64/mm/trans_pgd.c index 61549451ed3a..e24a749013c1 100644 --- a/arch/arm64/mm/trans_pgd.c +++ b/arch/arm64/mm/trans_pgd.c @@ -217,63 +217,6 @@ int trans_pgd_create_copy(struct trans_pgd_info *info, pgd_t **dst_pgdp, return rc; } -/* - * Add map entry to trans_pgd for a base-size page at PTE level. - * info: contains allocator and its argument - * trans_pgd: page table in which new map is added. - * page: page to be mapped. - * dst_addr: new VA address for the page - * pgprot: protection for the page. - * - * Returns 0 on success, and -ENOMEM on failure. - */ -int trans_pgd_map_page(struct trans_pgd_info *info, pgd_t *trans_pgd, - void *page, unsigned long dst_addr, pgprot_t pgprot) -{ - pgd_t *pgdp; - p4d_t *p4dp; - pud_t *pudp; - pmd_t *pmdp; - pte_t *ptep; - - pgdp = pgd_offset_pgd(trans_pgd, dst_addr); - if (pgd_none(READ_ONCE(*pgdp))) { - p4dp = trans_alloc(info); - if (!pgdp) - return -ENOMEM; - pgd_populate(NULL, pgdp, p4dp); - } - - p4dp = p4d_offset(pgdp, dst_addr); - if (p4d_none(READ_ONCE(*p4dp))) { - pudp = trans_alloc(info); - if (!pudp) - return -ENOMEM; - p4d_populate(NULL, p4dp, pudp); - } - - pudp = pud_offset(p4dp, dst_addr); - if (pud_none(READ_ONCE(*pudp))) { - pmdp = trans_alloc(info); - if (!pmdp) - return -ENOMEM; - pud_populate(NULL, pudp, pmdp); - } - - pmdp = pmd_offset(pudp, dst_addr); - if (pmd_none(READ_ONCE(*pmdp))) { - ptep = trans_alloc(info); - if (!ptep) - return -ENOMEM; - pmd_populate_kernel(NULL, pmdp, ptep); - } - - ptep = pte_offset_kernel(pmdp, dst_addr); - set_pte(ptep, pfn_pte(virt_to_pfn(page), pgprot)); - - return 0; -} - /* * The page we want to idmap may be outside the range covered by VA_BITS that * can be built using the kernel's p?d_populate() helpers. As a one off, for a