From patchwork Thu May 27 15:05:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12284527 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9975CC4708A for ; Thu, 27 May 2021 15:05:33 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 48B1D613AC for ; Thu, 27 May 2021 15:05:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 48B1D613AC Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id AA6386B0071; Thu, 27 May 2021 11:05:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A32346B0072; Thu, 27 May 2021 11:05:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8A8DD8D0001; Thu, 27 May 2021 11:05:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0083.hostedemail.com [216.40.44.83]) by kanga.kvack.org (Postfix) with ESMTP id 50B4E6B0071 for ; Thu, 27 May 2021 11:05:32 -0400 (EDT) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id E31B6824999B for ; Thu, 27 May 2021 15:05:31 +0000 (UTC) X-FDA: 78187334862.05.CF054DE Received: from mail-qk1-f172.google.com (mail-qk1-f172.google.com [209.85.222.172]) by imf13.hostedemail.com (Postfix) with ESMTP id 91A07E000816 for ; Thu, 27 May 2021 15:05:23 +0000 (UTC) Received: by mail-qk1-f172.google.com with SMTP id i67so931433qkc.4 for ; Thu, 27 May 2021 08:05:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=ghqRky3xeZiHl8S3DiRVTP9pM6BiSKFavllixIkkvd0=; b=caUhVx3J/tqp+Gal3Xi+7VBVenbj1bAKADWn8txk9KZ20D6QNDbgDjYASPogchOfyj MnR9GvDeTVlU/w7h0MDxRGxJcSxJKoV0mjkK+QdLIg1QR5LED8krF6MVh4Dw7m3PScI9 DH5d0pWUvzXOFaty1vK3wY3s2CaSoIxaYfFpwPApeJBhcfwIAViezHl4cLnrFbUJWTEz CxttZKvFn/yQ47HYBvaRhH+Top3dM3Aos+zj8m2STtvk8DprhmBAFJxZS0AcEiFO3Kb7 DrdN7vRSm71g5/nEb3VcE95apeL0RMCTNaPwasAdOu10HhJ2vq1M/c/C1Yn8L1NpF7Rq MkWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ghqRky3xeZiHl8S3DiRVTP9pM6BiSKFavllixIkkvd0=; b=ouWETiO5moEEmuc+VywX2BcXpj5laLQq5zZ2MOk1SZIJFQ/47Vq0+2UdgLJo2+SEuq Qbf5arO7YesfjoUd4FUcFElt7XVFi5plw8PQwpRwxZmg78a8/rPX6//sQDNFtfnZFVaN fTOmmLMCgK0xLETrTIyxgKK7KdvmZoMl35/NzXNQ3K+vi+o4CxuaJOLezEJ90OF38eX8 nwXauPWCktOnbyieX8bUCqt6FeNSnJsONMSHxXA0cicZrGSk5ki+PQ7rW9F8y9fENXHY TZQSajZaJN4IxQmtWvwoekRApRKLUi30MAVGUnvqQGG519OaEa94thfEdiplo3RQr6i/ /2IQ== X-Gm-Message-State: AOAM533b/mq7a46bcFhV9T9SomfOpfvzwpAi6sJh1mIuaU0sWEF61gjF eBDUpwpvUcyA9YJ4keAprVGS0g== X-Google-Smtp-Source: ABdhPJzkqRktvQM1YvTQKwrzQq+gZ576cPkKRhONnmbg0z/uIwTpNvyWkip4RNp35WexktltcbHpqw== X-Received: by 2002:a37:c15:: with SMTP id 21mr3945299qkm.180.1622127930853; Thu, 27 May 2021 08:05:30 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id x10sm1447468qkh.124.2021.05.27.08.05.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 May 2021 08:05:30 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH 01/18] arm64: hyp-stub: Check the size of the HYP stub's vectors Date: Thu, 27 May 2021 11:05:09 -0400 Message-Id: <20210527150526.271941-2-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210527150526.271941-1-pasha.tatashin@soleen.com> References: <20210527150526.271941-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 91A07E000816 Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=soleen.com header.s=google header.b=caUhVx3J; spf=pass (imf13.hostedemail.com: domain of pasha.tatashin@soleen.com designates 209.85.222.172 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com; dmarc=none X-Rspamd-Server: rspam04 X-Stat-Signature: yf4rb3t6f65xdbfgyjaib3b8dqjdg9t5 X-HE-Tag: 1622127923-996110 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: James Morse Hibernate contains a set of temporary EL2 vectors used to 'park' EL2 somewhere safe while all the memory is thrown in the air. Making kexec do its relocations with the MMU on means they have to be done at EL1, so EL2 has to be parked. This means yet another set of vectors. All these things do is HVC_SET_VECTORS and HVC_SOFT_RESTART, both of which are implemented by the hyp-stub. Lets copy it instead of re-inventing it. To do this the hyp-stub's entrails need to be packed neatly inside its 2K vectors. Start by moving the final 2K alignment inside the end marker, and add a build check that we didn't overflow 2K. Signed-off-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/hyp-stub.S | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kernel/hyp-stub.S b/arch/arm64/kernel/hyp-stub.S index 43d212618834..91aab4121463 100644 --- a/arch/arm64/kernel/hyp-stub.S +++ b/arch/arm64/kernel/hyp-stub.S @@ -41,9 +41,13 @@ SYM_CODE_START(__hyp_stub_vectors) ventry el1_irq_invalid // IRQ 32-bit EL1 ventry el1_fiq_invalid // FIQ 32-bit EL1 ventry el1_error_invalid // Error 32-bit EL1 + .align 11 +SYM_INNER_LABEL(__hyp_stub_vectors_end, SYM_L_LOCAL) SYM_CODE_END(__hyp_stub_vectors) - .align 11 +# Check the __hyp_stub_vectors didn't overflow +.org . - (__hyp_stub_vectors_end - __hyp_stub_vectors) + SZ_2K + SYM_CODE_START_LOCAL(elx_sync) cmp x0, #HVC_SET_VECTORS From patchwork Thu May 27 15:05:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12284529 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A494AC4708B for ; Thu, 27 May 2021 15:05:35 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5119061358 for ; Thu, 27 May 2021 15:05:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5119061358 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 4405B6B0072; Thu, 27 May 2021 11:05:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4157C6B0073; Thu, 27 May 2021 11:05:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1A38D8D0001; Thu, 27 May 2021 11:05:34 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0071.hostedemail.com [216.40.44.71]) by kanga.kvack.org (Postfix) with ESMTP id D183A6B0072 for ; Thu, 27 May 2021 11:05:33 -0400 (EDT) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 583D41811E9E9 for ; Thu, 27 May 2021 15:05:33 +0000 (UTC) X-FDA: 78187334946.20.8B0B164 Received: from mail-qk1-f176.google.com (mail-qk1-f176.google.com [209.85.222.176]) by imf20.hostedemail.com (Postfix) with ESMTP id C4BD22F9 for ; Thu, 27 May 2021 15:05:22 +0000 (UTC) Received: by mail-qk1-f176.google.com with SMTP id 76so898913qkn.13 for ; Thu, 27 May 2021 08:05:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=l3SGBPbH3T8MZIM5VPxw+w4kFGUdmwAVrjo+9WKhb6A=; b=lKkn7FEbTutFu0AJibWJbh7gvLM6Up0Orzxu/3QynB6mWeF920eLv/nRIAjn0HTWbz kRhnM1FhPgEgKDcFLc0RU7+8HDyuVrDf1nOdQR+DxW+V/MU79MqDfiJMev3LvByNYJcG mosH1G9Mvi/NBnrEeQoBPbalZSrRc9viFcC8kFho3Oq48p2L9WDorq4FQts5PrzjhPvY PQXufesw6t3klHivvCa41fH7WyDUbQwQhgjcy69LGoTTy5HoW5rSaakGdTqk4RRi31+u qU/1ksTD7uZt2uZRkEseLan5+B9wF27wnlPJAwpkKXvmMj1unG3unZvMI0rv/GW8p4o1 xqPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=l3SGBPbH3T8MZIM5VPxw+w4kFGUdmwAVrjo+9WKhb6A=; b=UAN3nTNNJMuLN9rf2qb7xlMVoGqthGWd5Jtz2O6OieOFOT7WLL1EV1G/NpVw53nfSx myPPYTuYsSfv0UeMCQJJ6utigKN1Lt6//2GLncZoJ2pMU7lFBNXS4sOaigj7kIaSFtdw pEUVBXCSKkLUgq7AY9ZHO+GrvotqEFTJUCSfM5VrKnuyM37I5mRvjMlbrLxzRnygD5Xh 18WKTNJbBWwYqj4Sv1KRssHFzqc1PavHDsmS+Zc/dl4swX8Irwjfw2iDXKghXCf3AvVd G7d0XIHqEG/8dk5Yc9lVI9plCpv5beBKUKDqJRiH0IRTj9Vst7zcJ/oXdozShTdTW3qU vMMA== X-Gm-Message-State: AOAM531uR86ae9aeHJPOSoRliLxzPGiYg8o2fOeBX1OEwjjQL+AinwDi n9OBb7IhrXohWSZbCepNB4no1A== X-Google-Smtp-Source: ABdhPJxOofNUG3149CvRQe7xgHOAYxhjpmqWLXhWUL/moDKhXxMhTLFFTYyF/8yiB65KCQQlF0YIPQ== X-Received: by 2002:a37:f909:: with SMTP id l9mr2053163qkj.428.1622127932312; Thu, 27 May 2021 08:05:32 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id x10sm1447468qkh.124.2021.05.27.08.05.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 May 2021 08:05:31 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH 02/18] arm64: hyp-stub: Move invalid vector entries into the vectors Date: Thu, 27 May 2021 11:05:10 -0400 Message-Id: <20210527150526.271941-3-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210527150526.271941-1-pasha.tatashin@soleen.com> References: <20210527150526.271941-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: C4BD22F9 Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=soleen.com header.s=google header.b=lKkn7FEb; spf=pass (imf20.hostedemail.com: domain of pasha.tatashin@soleen.com designates 209.85.222.176 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com; dmarc=none X-Rspamd-Server: rspam04 X-Stat-Signature: 7hcrxozh3i7dx69kif8yfwffb58iqhby X-HE-Tag: 1622127922-52576 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: James Morse Most of the hyp-stub's vector entries are invalid. These are each a unique function that branches to itself. To move these into the vectors, merge the ventry and invalid_vector macros and give each one a unique name. This means we can copy the hyp-stub as it is self contained within its vectors. Signed-off-by: James Morse [Fixed merging issues] Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/hyp-stub.S | 62 +++++++++++++++--------------------- 1 file changed, 26 insertions(+), 36 deletions(-) diff --git a/arch/arm64/kernel/hyp-stub.S b/arch/arm64/kernel/hyp-stub.S index 91aab4121463..18a97bee3779 100644 --- a/arch/arm64/kernel/hyp-stub.S +++ b/arch/arm64/kernel/hyp-stub.S @@ -16,31 +16,38 @@ #include #include +.macro invalid_vector label +SYM_CODE_START_LOCAL(\label) + .align 7 + b \label +SYM_CODE_END(\label) +.endm + .text .pushsection .hyp.text, "ax" .align 11 SYM_CODE_START(__hyp_stub_vectors) - ventry el2_sync_invalid // Synchronous EL2t - ventry el2_irq_invalid // IRQ EL2t - ventry el2_fiq_invalid // FIQ EL2t - ventry el2_error_invalid // Error EL2t - - ventry elx_sync // Synchronous EL2h - ventry el2_irq_invalid // IRQ EL2h - ventry el2_fiq_invalid // FIQ EL2h - ventry el2_error_invalid // Error EL2h - - ventry elx_sync // Synchronous 64-bit EL1 - ventry el1_irq_invalid // IRQ 64-bit EL1 - ventry el1_fiq_invalid // FIQ 64-bit EL1 - ventry el1_error_invalid // Error 64-bit EL1 - - ventry el1_sync_invalid // Synchronous 32-bit EL1 - ventry el1_irq_invalid // IRQ 32-bit EL1 - ventry el1_fiq_invalid // FIQ 32-bit EL1 - ventry el1_error_invalid // Error 32-bit EL1 + invalid_vector hyp_stub_el2t_sync_invalid // Synchronous EL2t + invalid_vector hyp_stub_el2t_irq_invalid // IRQ EL2t + invalid_vector hyp_stub_el2t_fiq_invalid // FIQ EL2t + invalid_vector hyp_stub_el2t_error_invalid // Error EL2t + + ventry elx_sync // Synchronous EL2h + invalid_vector hyp_stub_el2h_irq_invalid // IRQ EL2h + invalid_vector hyp_stub_el2h_fiq_invalid // FIQ EL2h + invalid_vector hyp_stub_el2h_error_invalid // Error EL2h + + ventry elx_sync // Synchronous 64-bit EL1 + invalid_vector hyp_stub_el1_irq_invalid // IRQ 64-bit EL1 + invalid_vector hyp_stub_el1_fiq_invalid // FIQ 64-bit EL1 + invalid_vector hyp_stub_el1_error_invalid // Error 64-bit EL1 + + invalid_vector hyp_stub_32b_el1_sync_invalid // Synchronous 32-bit EL1 + invalid_vector hyp_stub_32b_el1_irq_invalid // IRQ 32-bit EL1 + invalid_vector hyp_stub_32b_el1_fiq_invalid // FIQ 32-bit EL1 + invalid_vector hyp_stub_32b_el1_error_invalid // Error 32-bit EL1 .align 11 SYM_INNER_LABEL(__hyp_stub_vectors_end, SYM_L_LOCAL) SYM_CODE_END(__hyp_stub_vectors) @@ -174,23 +181,6 @@ SYM_CODE_END(enter_vhe) .popsection -.macro invalid_vector label -SYM_CODE_START_LOCAL(\label) - b \label -SYM_CODE_END(\label) -.endm - - invalid_vector el2_sync_invalid - invalid_vector el2_irq_invalid - invalid_vector el2_fiq_invalid - invalid_vector el2_error_invalid - invalid_vector el1_sync_invalid - invalid_vector el1_irq_invalid - invalid_vector el1_fiq_invalid - invalid_vector el1_error_invalid - - .popsection - /* * __hyp_set_vectors: Call this after boot to set the initial hypervisor * vectors as part of hypervisor installation. On an SMP system, this should From patchwork Thu May 27 15:05:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12284531 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4A310C47089 for ; Thu, 27 May 2021 15:05:38 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D36006135F for ; Thu, 27 May 2021 15:05:37 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D36006135F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A59106B0073; Thu, 27 May 2021 11:05:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A30F88D0001; Thu, 27 May 2021 11:05:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 814D76B0075; Thu, 27 May 2021 11:05:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0073.hostedemail.com [216.40.44.73]) by kanga.kvack.org (Postfix) with ESMTP id 4ADB36B0073 for ; Thu, 27 May 2021 11:05:35 -0400 (EDT) Received: from smtpin40.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id AE1F01E18 for ; Thu, 27 May 2021 15:05:34 +0000 (UTC) X-FDA: 78187334988.40.15B5324 Received: from mail-qt1-f171.google.com (mail-qt1-f171.google.com [209.85.160.171]) by imf07.hostedemail.com (Postfix) with ESMTP id ED6E5A0000FF for ; Thu, 27 May 2021 15:05:25 +0000 (UTC) Received: by mail-qt1-f171.google.com with SMTP id a15so466370qta.0 for ; Thu, 27 May 2021 08:05:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=Dd4OYNLmvVEa/N4zByXvuOh+6Xq8z1RHPtKQnceMU+s=; b=PAy8oUO7YX8LhrT2i2a4coviv4EjGLjt9zdp4yLK7smhcEYSnOPuWxth8pQ2sJKm+p 2jg8Z8xa3QAfUFb6YPB7i6dNFCgjUSpCNNTF4y2TBPfNZI+7cCG57PyzHQ9Of9f20W6B R7bkFVD0/i3hf1SwqNdYQsnQuKcFwRo7leOcm3+FWAqSlbSv3tKKbTWkEpxfIENLQcP7 Avke/bt4sGbmFGea4UxrsFvQiaqrhp1VhY7VKD6d9SVV7vKuSzxZ9sKf4SCTFCA3py1E o37jHAYoXineM+R2Or5/XjDQtnkuW/wiU2rUyJdY25VcLfzBGnauKIoXd+B4KclFdTl7 0iSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Dd4OYNLmvVEa/N4zByXvuOh+6Xq8z1RHPtKQnceMU+s=; b=OewaDo6IaAirYAJ16Jg8AxO+J7tx2ZfuWJzorv0WjrWCD8ySekRQ6VilkT+dtzdljF eYLaAXGzsTUqx5LEUrnVVS7fXYYMcoREEd4B8kaiRrg+Q9tG2gYIao1hwnb2+grb4NoD FqpPV2t0VPBzFlGfjATyHtLIbbUB5r2+TqWsKmtn9wTzA5yIwbH3/i5ENCLyOKvYP3z+ gQiedpfzJPhZ3cPz2gCj3uhMvcENdGdHKYlKAPEFVrAxyxoIMqslbd9xx1aN6If9/XJS 5/vm4+g7+taMW4YpatEnBk0kk4I2uCA0o0Rrozj2AMnjD874394IEJb8YWjmGPjFDybZ SI5w== X-Gm-Message-State: AOAM530wSKUNSgQyqSgG17HRZe6ex7kStZHVoWfORXwQDvKPH4SYXi9/ IUoGEm//wNSfBweD70H7nGkCLg== X-Google-Smtp-Source: ABdhPJxtuOVFOKAsCuiHh9cOyHIEi9No+IlR64k2qAMP9VTvqHYTpO1ydEtOEgxg9B4D+CYuBIBhhQ== X-Received: by 2002:ac8:74c2:: with SMTP id j2mr3522136qtr.185.1622127933716; Thu, 27 May 2021 08:05:33 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id x10sm1447468qkh.124.2021.05.27.08.05.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 May 2021 08:05:33 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH 03/18] arm64: hyp-stub: Move elx_sync into the vectors Date: Thu, 27 May 2021 11:05:11 -0400 Message-Id: <20210527150526.271941-4-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210527150526.271941-1-pasha.tatashin@soleen.com> References: <20210527150526.271941-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: ED6E5A0000FF Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=soleen.com header.s=google header.b=PAy8oUO7; spf=pass (imf07.hostedemail.com: domain of pasha.tatashin@soleen.com designates 209.85.160.171 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com; dmarc=none X-Rspamd-Server: rspam04 X-Stat-Signature: 91g7yu3f3kd6nmdxjx4nea91xsen45e9 X-HE-Tag: 1622127925-833847 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The hyp-stub's elx_sync code fits in the vector. With this, all of the hyp-stubs behaviour is contained in its vectors. This lets kexec and hibernate copy the hyp-stub when they need its behaviour, instead of re-implementing it. Co-developed-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/hyp-stub.S | 64 +++++++++++++++++++----------------- 1 file changed, 33 insertions(+), 31 deletions(-) diff --git a/arch/arm64/kernel/hyp-stub.S b/arch/arm64/kernel/hyp-stub.S index 18a97bee3779..86af6c4e52b9 100644 --- a/arch/arm64/kernel/hyp-stub.S +++ b/arch/arm64/kernel/hyp-stub.S @@ -21,6 +21,37 @@ SYM_CODE_START_LOCAL(\label) .align 7 b \label SYM_CODE_END(\label) +.endm + +.macro elx_sync_vector label +SYM_CODE_START_LOCAL(\label) + .align 7 + cmp x0, #HVC_SET_VECTORS + b.ne 1f + msr vbar_el2, x1 + b 9f + +1: cmp x0, #HVC_VHE_RESTART + b.eq mutate_to_vhe + +2: cmp x0, #HVC_SOFT_RESTART + b.ne 3f + mov x0, x2 + mov x2, x4 + mov x4, x1 + mov x1, x3 + br x4 // no return + +3: cmp x0, #HVC_RESET_VECTORS + beq 9f // Nothing to reset! + + /* Someone called kvm_call_hyp() against the hyp-stub... */ + mov_q x0, HVC_STUB_ERR + eret + +9: mov x0, xzr + eret +SYM_CODE_END(\label) .endm .text @@ -34,12 +65,12 @@ SYM_CODE_START(__hyp_stub_vectors) invalid_vector hyp_stub_el2t_fiq_invalid // FIQ EL2t invalid_vector hyp_stub_el2t_error_invalid // Error EL2t - ventry elx_sync // Synchronous EL2h + elx_sync_vector el2h_sync // Synchronous EL2h invalid_vector hyp_stub_el2h_irq_invalid // IRQ EL2h invalid_vector hyp_stub_el2h_fiq_invalid // FIQ EL2h invalid_vector hyp_stub_el2h_error_invalid // Error EL2h - ventry elx_sync // Synchronous 64-bit EL1 + elx_sync_vector el1_sync // Synchronous 64-bit EL1 invalid_vector hyp_stub_el1_irq_invalid // IRQ 64-bit EL1 invalid_vector hyp_stub_el1_fiq_invalid // FIQ 64-bit EL1 invalid_vector hyp_stub_el1_error_invalid // Error 64-bit EL1 @@ -55,35 +86,6 @@ SYM_CODE_END(__hyp_stub_vectors) # Check the __hyp_stub_vectors didn't overflow .org . - (__hyp_stub_vectors_end - __hyp_stub_vectors) + SZ_2K - -SYM_CODE_START_LOCAL(elx_sync) - cmp x0, #HVC_SET_VECTORS - b.ne 1f - msr vbar_el2, x1 - b 9f - -1: cmp x0, #HVC_VHE_RESTART - b.eq mutate_to_vhe - -2: cmp x0, #HVC_SOFT_RESTART - b.ne 3f - mov x0, x2 - mov x2, x4 - mov x4, x1 - mov x1, x3 - br x4 // no return - -3: cmp x0, #HVC_RESET_VECTORS - beq 9f // Nothing to reset! - - /* Someone called kvm_call_hyp() against the hyp-stub... */ - mov_q x0, HVC_STUB_ERR - eret - -9: mov x0, xzr - eret -SYM_CODE_END(elx_sync) - // nVHE? No way! Give me the real thing! SYM_CODE_START_LOCAL(mutate_to_vhe) // Sanity check: MMU *must* be off From patchwork Thu May 27 15:05:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12284533 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6FBCEC4708A for ; Thu, 27 May 2021 15:05:40 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1AE1461005 for ; Thu, 27 May 2021 15:05:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1AE1461005 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E9DB56B0074; Thu, 27 May 2021 11:05:36 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E764E6B0075; Thu, 27 May 2021 11:05:36 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C7E0F6B0078; Thu, 27 May 2021 11:05:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0088.hostedemail.com [216.40.44.88]) by kanga.kvack.org (Postfix) with ESMTP id 8A14F6B0074 for ; Thu, 27 May 2021 11:05:36 -0400 (EDT) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 18B0F8249980 for ; Thu, 27 May 2021 15:05:36 +0000 (UTC) X-FDA: 78187335072.20.7C132B3 Received: from mail-qk1-f181.google.com (mail-qk1-f181.google.com [209.85.222.181]) by imf02.hostedemail.com (Postfix) with ESMTP id 6BF0940002E0 for ; Thu, 27 May 2021 15:05:31 +0000 (UTC) Received: by mail-qk1-f181.google.com with SMTP id j189so936163qkf.2 for ; Thu, 27 May 2021 08:05:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=Wg7p4vPv2NoAb+n17TZqII3/Mw2eE/iFM20fouMRLwI=; b=XrzMTNI74leC845PoHu6NBtRzlE8xBimsUOjXLlm0vqEuD994Nb3Pc9xZT4gWtpHt8 6lnP0+AaY2h28uSAyDh1nu4Z+LTcUlllsx5eiqqj5bxu/ktF78lPtnAclLwfIv9SbkiD mxf6ado+CHp4f52cHMoGtOEQ1QFmpN1GY3q5Su+KiQuZ3j8z40XlQF1SLQjKkxpoYSe4 Qoc1WTi0P9N2l1MX2xW5Que69zUhDpRgbA6a0ISuCBIqhTg95a8x65rSXLfqzWFsJIpb sP6Fsxt+vD5yqAz7mf4NLYOIOZlfCeWr0fuDyYPEX6c3bseja8qsl/UQYn1elCzeeUQ0 b/YQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Wg7p4vPv2NoAb+n17TZqII3/Mw2eE/iFM20fouMRLwI=; b=dypghYQbZ4xL3U4dC0XFljwRCgqNVdZObjM0b4HMCFtjJv7c0RmbfRmjZp+GpkPSc4 YU2EG9pobh7jt9RvyDDJaCC4LFdUkQY+7NcdFHCQeDtUbfniMxh0Kqlg1yrJtdIGmHTA zKTQfBxH+/kaMUjXUNoeIvnD7VhzhVgJsKSIXtIba9Jwyk3AZ0F0uZfdzndAA7xBgP1J cTE1MxM8WAN57fNAxPV1bpX1+uKF1asqvgZkfrgdQOJDEJ+tGTsxxDfzavBThit+yCnf i6Eo1G380/NNrNDRDtP7jCvME3rSzcFeBm/PRu5hM7psQ6/kgmZfuLfMBNZ/PhL/3bpY 9uHA== X-Gm-Message-State: AOAM532TCac8K1jH+ZiiZzMa9edtODajC83uXP5J06AFfDPKm45ksn7G mxh16FoyDnxib7kkrSGg0Q/D7w== X-Google-Smtp-Source: ABdhPJycl8DRfGqs/GI/GT6NLoLRwo6sDRsMxgXrzg8L64wopg0OcAVe2PjXq4CFxxycTq8ansLW+g== X-Received: by 2002:a05:620a:14b9:: with SMTP id x25mr3931435qkj.460.1622127935134; Thu, 27 May 2021 08:05:35 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id x10sm1447468qkh.124.2021.05.27.08.05.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 May 2021 08:05:34 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH 04/18] arm64: kernel: add helper for booted at EL2 and not VHE Date: Thu, 27 May 2021 11:05:12 -0400 Message-Id: <20210527150526.271941-5-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210527150526.271941-1-pasha.tatashin@soleen.com> References: <20210527150526.271941-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=soleen.com header.s=google header.b=XrzMTNI7; dmarc=none; spf=pass (imf02.hostedemail.com: domain of pasha.tatashin@soleen.com designates 209.85.222.181 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 6BF0940002E0 X-Stat-Signature: 3dz53guc4w9am1hqdcdi5yah1grgh83b X-HE-Tag: 1622127931-353011 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Replace places that contain logic like this: is_hyp_mode_available() && !is_kernel_in_hyp_mode() With a dedicated boolean function is_hyp_callable(). This will be needed later in kexec in order to sooner switch back to EL2. Suggested-by: James Morse [Fixed merging issues] Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/virt.h | 5 +++++ arch/arm64/kernel/cpu-reset.h | 3 +-- arch/arm64/kernel/hibernate.c | 9 +++------ arch/arm64/kernel/sdei.c | 2 +- 4 files changed, 10 insertions(+), 9 deletions(-) diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h index 7379f35ae2c6..4216c8623538 100644 --- a/arch/arm64/include/asm/virt.h +++ b/arch/arm64/include/asm/virt.h @@ -128,6 +128,11 @@ static __always_inline bool is_protected_kvm_enabled(void) return cpus_have_final_cap(ARM64_KVM_PROTECTED_MODE); } +static inline bool is_hyp_callable(void) +{ + return is_hyp_mode_available() && !is_kernel_in_hyp_mode(); +} + #endif /* __ASSEMBLY__ */ #endif /* ! __ASM__VIRT_H */ diff --git a/arch/arm64/kernel/cpu-reset.h b/arch/arm64/kernel/cpu-reset.h index 9a7b1262ef17..48d0ed48c147 100644 --- a/arch/arm64/kernel/cpu-reset.h +++ b/arch/arm64/kernel/cpu-reset.h @@ -20,8 +20,7 @@ static inline void __noreturn __nocfi cpu_soft_restart(unsigned long entry, { typeof(__cpu_soft_restart) *restart; - unsigned long el2_switch = !is_kernel_in_hyp_mode() && - is_hyp_mode_available(); + unsigned long el2_switch = is_hyp_callable(); restart = (void *)__pa_symbol(function_nocfi(__cpu_soft_restart)); cpu_install_idmap(); diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index b1cef371df2b..c764574a1acb 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -48,9 +48,6 @@ */ extern int in_suspend; -/* Do we need to reset el2? */ -#define el2_reset_needed() (is_hyp_mode_available() && !is_kernel_in_hyp_mode()) - /* temporary el2 vectors in the __hibernate_exit_text section. */ extern char hibernate_el2_vectors[]; @@ -125,7 +122,7 @@ int arch_hibernation_header_save(void *addr, unsigned int max_size) hdr->reenter_kernel = _cpu_resume; /* We can't use __hyp_get_vectors() because kvm may still be loaded */ - if (el2_reset_needed()) + if (is_hyp_callable()) hdr->__hyp_stub_vectors = __pa_symbol(__hyp_stub_vectors); else hdr->__hyp_stub_vectors = 0; @@ -387,7 +384,7 @@ int swsusp_arch_suspend(void) dcache_clean_range(__idmap_text_start, __idmap_text_end); /* Clean kvm setup code to PoC? */ - if (el2_reset_needed()) { + if (is_hyp_callable()) { dcache_clean_range(__hyp_idmap_text_start, __hyp_idmap_text_end); dcache_clean_range(__hyp_text_start, __hyp_text_end); } @@ -482,7 +479,7 @@ int swsusp_arch_resume(void) * * We can skip this step if we booted at EL1, or are running with VHE. */ - if (el2_reset_needed()) { + if (is_hyp_callable()) { phys_addr_t el2_vectors = (phys_addr_t)hibernate_exit; el2_vectors += hibernate_el2_vectors - __hibernate_exit_text_start; /* offset */ diff --git a/arch/arm64/kernel/sdei.c b/arch/arm64/kernel/sdei.c index 2c7ca449dd51..af0ac2f920cf 100644 --- a/arch/arm64/kernel/sdei.c +++ b/arch/arm64/kernel/sdei.c @@ -200,7 +200,7 @@ unsigned long sdei_arch_get_entry_point(int conduit) * dropped to EL1 because we don't support VHE, then we can't support * SDEI. */ - if (is_hyp_mode_available() && !is_kernel_in_hyp_mode()) { + if (is_hyp_callable()) { pr_err("Not supported on this hardware/boot configuration\n"); goto out_err; } From patchwork Thu May 27 15:05:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12284535 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CA1EDC4707F for ; Thu, 27 May 2021 15:05:42 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6F49D613AB for ; Thu, 27 May 2021 15:05:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6F49D613AB Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 015ED6B0075; Thu, 27 May 2021 11:05:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EDCA06B0078; Thu, 27 May 2021 11:05:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 91F246B007B; Thu, 27 May 2021 11:05:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0061.hostedemail.com [216.40.44.61]) by kanga.kvack.org (Postfix) with ESMTP id 5ED5D6B0075 for ; Thu, 27 May 2021 11:05:38 -0400 (EDT) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id E6F3E8E70 for ; Thu, 27 May 2021 15:05:37 +0000 (UTC) X-FDA: 78187335114.18.E471516 Received: from mail-qv1-f47.google.com (mail-qv1-f47.google.com [209.85.219.47]) by imf02.hostedemail.com (Postfix) with ESMTP id 0FB8040F8C19 for ; Thu, 27 May 2021 15:05:32 +0000 (UTC) Received: by mail-qv1-f47.google.com with SMTP id k2so247006qvc.5 for ; Thu, 27 May 2021 08:05:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=+EiR776LGRIVUqs8E6nTUaQXNk2yNVD65j0QM2uVzow=; b=OoTpQqa+5r8BlTD7ijih47CrLq8A6B9RbRxoHM1uCuW1Z7lER6etpTCJW6b36pKuoO /Ib+oumN81PGrjvwhUucnW8PIIX/xfixSd/82C9Ngo0OghsxdhoPMN9YB8akCg4px1aZ XzXeqso3LANa1DVqRQz9crazOLSsLdTmD5xQcuItCBxnRmMyC6R44//8WloWI52+WiPZ UYEUxuuFElnKBjvp82oC1mMOpZWb2Z7QJCVc/Q8PHNr52Va/jYqVOiEiP6SEjdDDqhBD 2jLNo0uWAKW+fjYBYBBF+rnqF8xkm+S4zZh0UrJJEjyMpmAzJzUiWOzMjiANp5dxzvof YSNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+EiR776LGRIVUqs8E6nTUaQXNk2yNVD65j0QM2uVzow=; b=nL8PcDEZpI1dcMgcwlq5sw51M7E7mYQMy/ynyA0zxB/8AL+EPHroXEMMSgihgMvewL uecoop8314e05VlTqV7wyrSuFNthT1eV9e4VA/Xmhd4Pygbhc2HCzYeQdNuDh7GdmoZs elONiu9+Cgmj3CJQBYlaU6GRzP6B0Vg3eQN5stl008qyBb0WeL0UbGaTVaByzb8T2Wqr UtelalLAMKWPWYn8pMq0Sk/4in62PY7OEtE5ekQ9QCOFve2R9Ki3bZmJSXU77ZDzJabb tRnhLboabhvDRoZo1Mrwb1usmbN4QGmmokvrupOrazsUY1MabKPKUrKaHzc8jN2rldiw d8Mg== X-Gm-Message-State: AOAM531pdItGzgK4Qh6m6LMgi0kaX+RuldPCsMWLkUJDA0QfO88/n7cW ypBrOV1ebU8prm7TNF7/8ag4mw== X-Google-Smtp-Source: ABdhPJzTP3oV1R+n8kGbXURQLbsLJZ4X6S2bcFpIfYYBkRufEHUPnqQYpI7uOymwC/kI/WMb3ZRyIw== X-Received: by 2002:ad4:4773:: with SMTP id d19mr4316997qvx.0.1622127936544; Thu, 27 May 2021 08:05:36 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id x10sm1447468qkh.124.2021.05.27.08.05.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 May 2021 08:05:36 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH 05/18] arm64: trans_pgd: hibernate: Add trans_pgd_copy_el2_vectors Date: Thu, 27 May 2021 11:05:13 -0400 Message-Id: <20210527150526.271941-6-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210527150526.271941-1-pasha.tatashin@soleen.com> References: <20210527150526.271941-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=soleen.com header.s=google header.b=OoTpQqa+; dmarc=none; spf=pass (imf02.hostedemail.com: domain of pasha.tatashin@soleen.com designates 209.85.219.47 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 0FB8040F8C19 X-Stat-Signature: qqwugb8gzt68ko7nu5wr41a6py1wdknz X-HE-Tag: 1622127932-119098 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Users of trans_pgd may also need a copy of vector table because it is also may be overwritten if a linear map can be overwritten. Move setup of EL2 vectors from hibernate to trans_pgd, so it can be later shared with kexec as well. Suggested-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/trans_pgd.h | 3 +++ arch/arm64/include/asm/virt.h | 3 +++ arch/arm64/kernel/hibernate.c | 28 ++++++++++------------------ arch/arm64/mm/trans_pgd.c | 20 ++++++++++++++++++++ 4 files changed, 36 insertions(+), 18 deletions(-) diff --git a/arch/arm64/include/asm/trans_pgd.h b/arch/arm64/include/asm/trans_pgd.h index 5d08e5adf3d5..e0760e52d36d 100644 --- a/arch/arm64/include/asm/trans_pgd.h +++ b/arch/arm64/include/asm/trans_pgd.h @@ -36,4 +36,7 @@ int trans_pgd_map_page(struct trans_pgd_info *info, pgd_t *trans_pgd, int trans_pgd_idmap_page(struct trans_pgd_info *info, phys_addr_t *trans_ttbr0, unsigned long *t0sz, void *page); +int trans_pgd_copy_el2_vectors(struct trans_pgd_info *info, + phys_addr_t *el2_vectors); + #endif /* _ASM_TRANS_TABLE_H */ diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h index 4216c8623538..bfbb66018114 100644 --- a/arch/arm64/include/asm/virt.h +++ b/arch/arm64/include/asm/virt.h @@ -67,6 +67,9 @@ */ extern u32 __boot_cpu_mode[2]; +extern char __hyp_stub_vectors[]; +#define ARM64_VECTOR_TABLE_LEN SZ_2K + void __hyp_set_vectors(phys_addr_t phys_vector_base); void __hyp_reset_vectors(void); diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index c764574a1acb..0b8bad8bb6eb 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -48,12 +48,6 @@ */ extern int in_suspend; -/* temporary el2 vectors in the __hibernate_exit_text section. */ -extern char hibernate_el2_vectors[]; - -/* hyp-stub vectors, used to restore el2 during resume from hibernate. */ -extern char __hyp_stub_vectors[]; - /* * The logical cpu number we should resume on, initialised to a non-cpu * number. @@ -428,6 +422,7 @@ int swsusp_arch_resume(void) void *zero_page; size_t exit_size; pgd_t *tmp_pg_dir; + phys_addr_t el2_vectors; void __noreturn (*hibernate_exit)(phys_addr_t, phys_addr_t, void *, void *, phys_addr_t, phys_addr_t); struct trans_pgd_info trans_info = { @@ -455,6 +450,14 @@ int swsusp_arch_resume(void) return -ENOMEM; } + if (is_hyp_callable()) { + rc = trans_pgd_copy_el2_vectors(&trans_info, &el2_vectors); + if (rc) { + pr_err("Failed to setup el2 vectors\n"); + return rc; + } + } + exit_size = __hibernate_exit_text_end - __hibernate_exit_text_start; /* * Copy swsusp_arch_suspend_exit() to a safe page. This will generate @@ -467,25 +470,14 @@ int swsusp_arch_resume(void) return rc; } - /* - * The hibernate exit text contains a set of el2 vectors, that will - * be executed at el2 with the mmu off in order to reload hyp-stub. - */ - __flush_dcache_area(hibernate_exit, exit_size); - /* * KASLR will cause the el2 vectors to be in a different location in * the resumed kernel. Load hibernate's temporary copy into el2. * * We can skip this step if we booted at EL1, or are running with VHE. */ - if (is_hyp_callable()) { - phys_addr_t el2_vectors = (phys_addr_t)hibernate_exit; - el2_vectors += hibernate_el2_vectors - - __hibernate_exit_text_start; /* offset */ - + if (is_hyp_callable()) __hyp_set_vectors(el2_vectors); - } hibernate_exit(virt_to_phys(tmp_pg_dir), resume_hdr.ttbr1_el1, resume_hdr.reenter_kernel, restore_pblist, diff --git a/arch/arm64/mm/trans_pgd.c b/arch/arm64/mm/trans_pgd.c index 527f0a39c3da..61549451ed3a 100644 --- a/arch/arm64/mm/trans_pgd.c +++ b/arch/arm64/mm/trans_pgd.c @@ -322,3 +322,23 @@ int trans_pgd_idmap_page(struct trans_pgd_info *info, phys_addr_t *trans_ttbr0, return 0; } + +/* + * Create a copy of the vector table so we can call HVC_SET_VECTORS or + * HVC_SOFT_RESTART from contexts where the table may be overwritten. + */ +int trans_pgd_copy_el2_vectors(struct trans_pgd_info *info, + phys_addr_t *el2_vectors) +{ + void *hyp_stub = trans_alloc(info); + + if (!hyp_stub) + return -ENOMEM; + *el2_vectors = virt_to_phys(hyp_stub); + memcpy(hyp_stub, &__hyp_stub_vectors, ARM64_VECTOR_TABLE_LEN); + __flush_icache_range((unsigned long)hyp_stub, + (unsigned long)hyp_stub + ARM64_VECTOR_TABLE_LEN); + __flush_dcache_area(hyp_stub, ARM64_VECTOR_TABLE_LEN); + + return 0; +} From patchwork Thu May 27 15:05:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12284537 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D8AE1C47089 for ; Thu, 27 May 2021 15:05:44 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 834EB613AC for ; Thu, 27 May 2021 15:05:44 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 834EB613AC Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2075B6B0078; Thu, 27 May 2021 11:05:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0859A6B007B; Thu, 27 May 2021 11:05:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9BF0A6B007D; Thu, 27 May 2021 11:05:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0123.hostedemail.com [216.40.44.123]) by kanga.kvack.org (Postfix) with ESMTP id 3AC546B0078 for ; Thu, 27 May 2021 11:05:40 -0400 (EDT) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id D22B3180F9840 for ; Thu, 27 May 2021 15:05:39 +0000 (UTC) X-FDA: 78187335198.26.CBBDFAD Received: from mail-qk1-f180.google.com (mail-qk1-f180.google.com [209.85.222.180]) by imf11.hostedemail.com (Postfix) with ESMTP id 00FC82000998 for ; Thu, 27 May 2021 15:05:28 +0000 (UTC) Received: by mail-qk1-f180.google.com with SMTP id i67so931912qkc.4 for ; Thu, 27 May 2021 08:05:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=uRcNIBXOyaSrsQwthVpy1hwauBBMN7Vivxx0utjD54U=; b=baw560aO1zilTRnsPH5ODSQM+umpDLJom0wWm7jrdVnxroTvnRw1ujP0UhZKxl4h16 N38nZKepAcYoE0UAEpaMfhWdP2zvyq4n22H9pFgq0X6dglCZZhNLhxbZjmXHThU2ZZJd FBIZs0K2hc/GsDRn7wCKSuSUDkhiWtOHZ6MQT3KjWhvmW/VAXRRm2CmpHf13D+JUuP5U jngl4FpHRuskS3sFTcK/T1B7mB3OYSga5Z7occBeWpVtPNsXbCtuGJExnBIOgMY8UZJJ adKCXSP8eR+kApqLmL1Xc5egVdLNQ8x4y3pH9ptqd38tuCXkVllnx9kYOl5J496pyK3b lgvQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=uRcNIBXOyaSrsQwthVpy1hwauBBMN7Vivxx0utjD54U=; b=CHX3KZ3X0o7E697NxkCadgBolJkGS9aC80pccSi5SlWysqUIRJ0aW5cPFU7mknlLjM Ac4tRj20l1QiU/f3xVrpVRpiBrSAlHlHvdW/P8CnUmxc/gWKVHAcHZ50w6371dcfvTwR rkBSEQLFudDNkrd0H5T8v86QBLQxMHJI1BL/u0PmCh8ltcEANScVh+ztSys1Wy3/voQm /yBlrRed8zPEFq/L+ISxYg9EGQb/MTKtkx4p2PZKRoruNqgIN7v6f8ETAFY1hURHbbIi JDHXBkXxsgXMBgotjZnCAj/NKJL5ZbQR2qXNEQ4uDNl8jvkHrdx1j5gVRnSkDq7CUmDN Cqkg== X-Gm-Message-State: AOAM530GhILtpwFkCkePZUvavIUdh8Ckr0V1z2W154j4VQnjlWNt1nkE eQ/pafJg5Zm6qtXuqjOoDNYR2w== X-Google-Smtp-Source: ABdhPJxMXmwEgisa5gA4j+w8k9Kb9fYN4u/mCq3bmdSU3qA/vvzD6gnPAIKYpynHFW3vyiVzDRiyAQ== X-Received: by 2002:a05:620a:1126:: with SMTP id p6mr4086588qkk.120.1622127938229; Thu, 27 May 2021 08:05:38 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id x10sm1447468qkh.124.2021.05.27.08.05.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 May 2021 08:05:37 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH 06/18] arm64: hibernate: abstract ttrb0 setup function Date: Thu, 27 May 2021 11:05:14 -0400 Message-Id: <20210527150526.271941-7-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210527150526.271941-1-pasha.tatashin@soleen.com> References: <20210527150526.271941-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 00FC82000998 Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=soleen.com header.s=google header.b=baw560aO; spf=pass (imf11.hostedemail.com: domain of pasha.tatashin@soleen.com designates 209.85.222.180 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com; dmarc=none X-Rspamd-Server: rspam04 X-Stat-Signature: q3u1rg5z4qtp8hfneycz9yss8ohwzgco X-HE-Tag: 1622127928-51048 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently, only hibernate sets custom ttbr0 with safe idmaped function. Kexec, is also going to be using this functinality when relocation code is going to be idmapped. Move the setup seqeuence to a dedicated cpu_install_ttbr0() for custom ttbr0. Suggested-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/mmu_context.h | 24 ++++++++++++++++++++++++ arch/arm64/kernel/hibernate.c | 21 +-------------------- 2 files changed, 25 insertions(+), 20 deletions(-) diff --git a/arch/arm64/include/asm/mmu_context.h b/arch/arm64/include/asm/mmu_context.h index d3cef9133539..85eb92a2ffcb 100644 --- a/arch/arm64/include/asm/mmu_context.h +++ b/arch/arm64/include/asm/mmu_context.h @@ -115,6 +115,30 @@ static inline void cpu_install_idmap(void) cpu_switch_mm(lm_alias(idmap_pg_dir), &init_mm); } +/* + * Load our new page tables. A strict BBM approach requires that we ensure that + * TLBs are free of any entries that may overlap with the global mappings we are + * about to install. + * + * For a real hibernate/resume/kexec cycle TTBR0 currently points to a zero + * page, but TLBs may contain stale ASID-tagged entries (e.g. for EFI runtime + * services), while for a userspace-driven test_resume cycle it points to + * userspace page tables (and we must point it at a zero page ourselves). + * + * We change T0SZ as part of installing the idmap. This is undone by + * cpu_uninstall_idmap() in __cpu_suspend_exit(). + */ +static inline void cpu_install_ttbr0(phys_addr_t ttbr0, unsigned long t0sz) +{ + cpu_set_reserved_ttbr0(); + local_flush_tlb_all(); + __cpu_set_tcr_t0sz(t0sz); + + /* avoid cpu_switch_mm() and its SW-PAN and CNP interactions */ + write_sysreg(ttbr0, ttbr0_el1); + isb(); +} + /* * Atomically replaces the active TTBR1_EL1 PGD with a new VA-compatible PGD, * avoiding the possibility of conflicting TLB entries being allocated. diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index 0b8bad8bb6eb..ded5115bcb63 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -206,26 +206,7 @@ static int create_safe_exec_page(void *src_start, size_t length, if (rc) return rc; - /* - * Load our new page tables. A strict BBM approach requires that we - * ensure that TLBs are free of any entries that may overlap with the - * global mappings we are about to install. - * - * For a real hibernate/resume cycle TTBR0 currently points to a zero - * page, but TLBs may contain stale ASID-tagged entries (e.g. for EFI - * runtime services), while for a userspace-driven test_resume cycle it - * points to userspace page tables (and we must point it at a zero page - * ourselves). - * - * We change T0SZ as part of installing the idmap. This is undone by - * cpu_uninstall_idmap() in __cpu_suspend_exit(). - */ - cpu_set_reserved_ttbr0(); - local_flush_tlb_all(); - __cpu_set_tcr_t0sz(t0sz); - write_sysreg(trans_ttbr0, ttbr0_el1); - isb(); - + cpu_install_ttbr0(trans_ttbr0, t0sz); *phys_dst_addr = virt_to_phys(page); return 0; From patchwork Thu May 27 15:05:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12284539 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3AF95C4707F for ; Thu, 27 May 2021 15:05:47 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id DD81F61005 for ; Thu, 27 May 2021 15:05:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DD81F61005 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 45AFB6B007B; Thu, 27 May 2021 11:05:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3C3286B007D; Thu, 27 May 2021 11:05:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9EDBE6B0080; Thu, 27 May 2021 11:05:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0191.hostedemail.com [216.40.44.191]) by kanga.kvack.org (Postfix) with ESMTP id 421366B007D for ; Thu, 27 May 2021 11:05:41 -0400 (EDT) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id BBF92181AF5D0 for ; Thu, 27 May 2021 15:05:40 +0000 (UTC) X-FDA: 78187335240.29.E654974 Received: from mail-qk1-f172.google.com (mail-qk1-f172.google.com [209.85.222.172]) by imf17.hostedemail.com (Postfix) with ESMTP id 0FDE240B8CF7 for ; Thu, 27 May 2021 15:05:34 +0000 (UTC) Received: by mail-qk1-f172.google.com with SMTP id i67so932030qkc.4 for ; Thu, 27 May 2021 08:05:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=FcIEg/sDUKkdURMCkTF1KW5fxOPpkw674yFxDZdydcg=; b=FoXurcNpRVh8PW9Lk8j7yxcfb9WhsD1odEsRlBENU4YBfTWy5FE7Pci3MirsOpLizl OAwkUiXNxiixolnp818py2J+B/N0h14eTcWXM9L2T3NeUf5C9R5OGcYfwKJsn7sjcW0y RlZ2lY5o9vDQkxgcwfg0mNbVjK/w5ueBm5N8jUlDq5eFUT44aspwJXmgwff2E9EiOGGc 52TL/Kuuj7KVAqFRjlqujmS49xER8gmVrA135D9td7VASDqpb6Ou3adZ8eXy/q+KMWXP ACHn61DU7z5lzrxfnWLT9k5wcrTSFC0FGjeIZyFeGvHPa0mwpY/z2P37hkae3AzL+Xfg PhTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FcIEg/sDUKkdURMCkTF1KW5fxOPpkw674yFxDZdydcg=; b=NSJ9rWimVg8o5vFYYzEvjW8g64VgYwSTQcbmi7BPcjUekP/P9ZuIy9Y96+rKyrKaMs WoKTn1A5t8BTtbFtR7wayXL0hVvndnXgvojkCKGdUFfwiaht280DsQZdR1cxWTta/TpK D208x0gUTCSKmBT4mjdx3PadSdaau+WOHHMzcWFdFiCb86FteytlzWUo89bwA+Tjvpvi ofu13A3et6IfuJX2et0AKeVKDRq5bWBZDH/tmLWnTVHfMZe1IXA6P2MW5cJiZ6HHoZyy QgJyhCxsLz1ugaJqxQa5DUTZ8nQSY0MJ0Gfq+ywP/6+nYtYzL8tjNPO29cWadPyMEc0w YGrA== X-Gm-Message-State: AOAM530L2z3Raatz4Nx+e7pJNUKcExTkP3AlJsf6K3ggWlIckrNJrwT5 8ClXtwnsDhSEziLITQ2gNKZBs8rdBPCrPElE X-Google-Smtp-Source: ABdhPJzfd4jTVGoM6zB9+XxkahnEtHa/XkFTFTWTzzz5x2x/RbHjlysPqBPq0fd50bFxJPKMZb48wA== X-Received: by 2002:a37:30c:: with SMTP id 12mr4018868qkd.355.1622127939672; Thu, 27 May 2021 08:05:39 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id x10sm1447468qkh.124.2021.05.27.08.05.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 May 2021 08:05:39 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH 07/18] arm64: kexec: flush image and lists during kexec load time Date: Thu, 27 May 2021 11:05:15 -0400 Message-Id: <20210527150526.271941-8-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210527150526.271941-1-pasha.tatashin@soleen.com> References: <20210527150526.271941-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 0FDE240B8CF7 Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=soleen.com header.s=google header.b=FoXurcNp; spf=pass (imf17.hostedemail.com: domain of pasha.tatashin@soleen.com designates 209.85.222.172 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com; dmarc=none X-Rspamd-Server: rspam04 X-Stat-Signature: atdai5ykaim5r1h7xbaj4igsc4eaeo63 X-HE-Tag: 1622127934-706305 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently, during kexec load we are copying relocation function and flushing it. However, we can also flush kexec relocation buffers and if new kernel image is already in place (i.e. crash kernel), we can also flush the new kernel image itself. Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/machine_kexec.c | 49 +++++++++++++++---------------- 1 file changed, 23 insertions(+), 26 deletions(-) diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 90a335c74442..3a034bc25709 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -59,23 +59,6 @@ void machine_kexec_cleanup(struct kimage *kimage) /* Empty routine needed to avoid build errors. */ } -int machine_kexec_post_load(struct kimage *kimage) -{ - void *reloc_code = page_to_virt(kimage->control_code_page); - - memcpy(reloc_code, arm64_relocate_new_kernel, - arm64_relocate_new_kernel_size); - kimage->arch.kern_reloc = __pa(reloc_code); - kexec_image_info(kimage); - - /* Flush the reloc_code in preparation for its execution. */ - __flush_dcache_area(reloc_code, arm64_relocate_new_kernel_size); - flush_icache_range((uintptr_t)reloc_code, (uintptr_t)reloc_code + - arm64_relocate_new_kernel_size); - - return 0; -} - /** * machine_kexec_prepare - Prepare for a kexec reboot. * @@ -152,6 +135,29 @@ static void kexec_segment_flush(const struct kimage *kimage) } } +int machine_kexec_post_load(struct kimage *kimage) +{ + void *reloc_code = page_to_virt(kimage->control_code_page); + + /* If in place flush new kernel image, else flush lists and buffers */ + if (kimage->head & IND_DONE) + kexec_segment_flush(kimage); + else + kexec_list_flush(kimage); + + memcpy(reloc_code, arm64_relocate_new_kernel, + arm64_relocate_new_kernel_size); + kimage->arch.kern_reloc = __pa(reloc_code); + kexec_image_info(kimage); + + /* Flush the reloc_code in preparation for its execution. */ + __flush_dcache_area(reloc_code, arm64_relocate_new_kernel_size); + flush_icache_range((uintptr_t)reloc_code, (uintptr_t)reloc_code + + arm64_relocate_new_kernel_size); + + return 0; +} + /** * machine_kexec - Do the kexec reboot. * @@ -169,13 +175,6 @@ void machine_kexec(struct kimage *kimage) WARN(in_kexec_crash && (stuck_cpus || smp_crash_stop_failed()), "Some CPUs may be stale, kdump will be unreliable.\n"); - /* Flush the kimage list and its buffers. */ - kexec_list_flush(kimage); - - /* Flush the new image if already in place. */ - if ((kimage != kexec_crash_image) && (kimage->head & IND_DONE)) - kexec_segment_flush(kimage); - pr_info("Bye!\n"); local_daif_mask(); @@ -250,8 +249,6 @@ void arch_kexec_protect_crashkres(void) { int i; - kexec_segment_flush(kexec_crash_image); - for (i = 0; i < kexec_crash_image->nr_segments; i++) set_memory_valid( __phys_to_virt(kexec_crash_image->segment[i].mem), From patchwork Thu May 27 15:05:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12284541 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7CCE8C4707F for ; Thu, 27 May 2021 15:05:49 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 28574613AB for ; Thu, 27 May 2021 15:05:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 28574613AB Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 32F076B007D; Thu, 27 May 2021 11:05:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 304CA6B007E; Thu, 27 May 2021 11:05:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 097B96B0080; Thu, 27 May 2021 11:05:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0243.hostedemail.com [216.40.44.243]) by kanga.kvack.org (Postfix) with ESMTP id C4CDA6B007D for ; Thu, 27 May 2021 11:05:42 -0400 (EDT) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 60AFF181AEF1F for ; Thu, 27 May 2021 15:05:42 +0000 (UTC) X-FDA: 78187335324.26.3EC9391 Received: from mail-qv1-f43.google.com (mail-qv1-f43.google.com [209.85.219.43]) by imf06.hostedemail.com (Postfix) with ESMTP id 0AF02C005A08 for ; Thu, 27 May 2021 15:05:33 +0000 (UTC) Received: by mail-qv1-f43.google.com with SMTP id 5so276979qvk.0 for ; Thu, 27 May 2021 08:05:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=WOdTNCc8/+vKdXFWJvALf/5jcbGhRyKjqHuE0XK9ac4=; b=d8BkTIMDubNMKo8lhGcNu7AKdRus5J2M4TufcGSW4HtAeu3NbJ3XThPB66dQ+1m9p4 uQSN6tv4/0GLeQQUYYLvH9KTMD/3Y0ymqQQJUncGSGi7O+XI9dAh+bjI/F2JTBUYDI/b hG56Miopzkxf4OQah6VFcnupnjMDqnyYzxqclfNuueFAlJIXCIk/mUQkSy0l9hy7LZ63 8t5Zl2QIKW2FphA5noBR5hutyqZWAAE7ORfjV0j8Vn4IoOBLjWbakCjpSu+mbfcwra9P 9PA5A7XXtZhKASA48HUe+DjXCCyoKtyblRuv7GcVXybzYhpFeowUdCednMkzMNvzqc3m sUNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=WOdTNCc8/+vKdXFWJvALf/5jcbGhRyKjqHuE0XK9ac4=; b=NJent8Gsa6PBoZT/8Ticf/SPy00148KtPqBl2DmiFVCreqvkBlXnNYbGI6DbDpxh31 eTbqLmKMjaOUm8fA9XMKRmhd5jFYSxIkYBPJg5n7oHTbnybxbAzydaAQ+JEzwodHbUHb 56Q1492mteC5CEa2e7WIhyVfEKHA3+mJK7auu68GxzAIQDuOdg1fVO6LQ8ueVIVdUihA ovCBz6OfOltql/IF2ntsyvwDBRH8+zQ2wP8Bm1//gwrOdW8Ty+G4W6mzXhGAH0qMQTZA lPmT1J4XazO082UziEcZ5pH/hpH07XN0ReJP9CIkGw9ZxdlIgN4QaS6PkHhNTWqaqXpg JoVg== X-Gm-Message-State: AOAM531Cns1eQsDiVuUOC9NhJ6OK7TcPo5Xf1zvJd7jR1xX+JdbJdE3n WOcjv4LuJ6xQe/NnD8TfqwN8Ww== X-Google-Smtp-Source: ABdhPJxuuInT5boY5wx2h9r0mi/iNoeH4rIqWoxJwXXRJPR79arNDGrkhRpZr8NydkZli6xNGukztQ== X-Received: by 2002:ad4:4810:: with SMTP id g16mr4197984qvy.21.1622127941083; Thu, 27 May 2021 08:05:41 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id x10sm1447468qkh.124.2021.05.27.08.05.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 May 2021 08:05:40 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH 08/18] arm64: kexec: skip relocation code for inplace kexec Date: Thu, 27 May 2021 11:05:16 -0400 Message-Id: <20210527150526.271941-9-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210527150526.271941-1-pasha.tatashin@soleen.com> References: <20210527150526.271941-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=soleen.com header.s=google header.b=d8BkTIMD; dmarc=none; spf=pass (imf06.hostedemail.com: domain of pasha.tatashin@soleen.com designates 209.85.219.43 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 0AF02C005A08 X-Stat-Signature: hpnru49z7xak6cry3z1knga145qpbwwb X-HE-Tag: 1622127933-793451 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In case of kdump or when segments are already in place the relocation is not needed, therefore the setup of relocation function and call to it can be skipped. Signed-off-by: Pavel Tatashin Suggested-by: James Morse --- arch/arm64/kernel/machine_kexec.c | 34 ++++++++++++++++++----------- arch/arm64/kernel/relocate_kernel.S | 3 --- 2 files changed, 21 insertions(+), 16 deletions(-) diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 3a034bc25709..bde623805866 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -139,21 +139,23 @@ int machine_kexec_post_load(struct kimage *kimage) { void *reloc_code = page_to_virt(kimage->control_code_page); - /* If in place flush new kernel image, else flush lists and buffers */ - if (kimage->head & IND_DONE) + /* If in place, relocation is not used, only flush next kernel */ + if (kimage->head & IND_DONE) { kexec_segment_flush(kimage); - else - kexec_list_flush(kimage); + kexec_image_info(kimage); + return 0; + } memcpy(reloc_code, arm64_relocate_new_kernel, arm64_relocate_new_kernel_size); kimage->arch.kern_reloc = __pa(reloc_code); - kexec_image_info(kimage); /* Flush the reloc_code in preparation for its execution. */ __flush_dcache_area(reloc_code, arm64_relocate_new_kernel_size); flush_icache_range((uintptr_t)reloc_code, (uintptr_t)reloc_code + arm64_relocate_new_kernel_size); + kexec_list_flush(kimage); + kexec_image_info(kimage); return 0; } @@ -180,19 +182,25 @@ void machine_kexec(struct kimage *kimage) local_daif_mask(); /* - * cpu_soft_restart will shutdown the MMU, disable data caches, then - * transfer control to the kern_reloc which contains a copy of - * the arm64_relocate_new_kernel routine. arm64_relocate_new_kernel - * uses physical addressing to relocate the new image to its final - * position and transfers control to the image entry point when the - * relocation is complete. + * Both restart and cpu_soft_restart will shutdown the MMU, disable data + * caches. However, restart will start new kernel or purgatory directly, + * cpu_soft_restart will transfer control to arm64_relocate_new_kernel * In kexec case, kimage->start points to purgatory assuming that * kernel entry and dtb address are embedded in purgatory by * userspace (kexec-tools). * In kexec_file case, the kernel starts directly without purgatory. */ - cpu_soft_restart(kimage->arch.kern_reloc, kimage->head, kimage->start, - kimage->arch.dtb_mem); + if (kimage->head & IND_DONE) { + typeof(__cpu_soft_restart) *restart; + + cpu_install_idmap(); + restart = (void *)__pa_symbol(function_nocfi(__cpu_soft_restart)); + restart(is_hyp_callable(), kimage->start, kimage->arch.dtb_mem, + 0, 0); + } else { + cpu_soft_restart(kimage->arch.kern_reloc, kimage->head, + kimage->start, kimage->arch.dtb_mem); + } BUG(); /* Should never get here. */ } diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index b78ea5de97a4..8058fabe0a76 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -32,8 +32,6 @@ SYM_CODE_START(arm64_relocate_new_kernel) mov x16, x0 /* x16 = kimage_head */ mov x14, xzr /* x14 = entry ptr */ mov x13, xzr /* x13 = copy dest */ - /* Check if the new image needs relocation. */ - tbnz x16, IND_DONE_BIT, .Ldone raw_dcache_line_size x15, x1 /* x15 = dcache line size */ .Lloop: and x12, x16, PAGE_MASK /* x12 = addr */ @@ -65,7 +63,6 @@ SYM_CODE_START(arm64_relocate_new_kernel) .Lnext: ldr x16, [x14], #8 /* entry = *ptr++ */ tbz x16, IND_DONE_BIT, .Lloop /* while (!(entry & DONE)) */ -.Ldone: /* wait for writes from copy_page to finish */ dsb nsh ic iallu From patchwork Thu May 27 15:05:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12284543 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BE800C47089 for ; Thu, 27 May 2021 15:05:51 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 644EF6128D for ; Thu, 27 May 2021 15:05:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 644EF6128D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8AC966B007E; Thu, 27 May 2021 11:05:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 831976B0080; Thu, 27 May 2021 11:05:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 68AA36B0081; Thu, 27 May 2021 11:05:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0074.hostedemail.com [216.40.44.74]) by kanga.kvack.org (Postfix) with ESMTP id 2858E6B007E for ; Thu, 27 May 2021 11:05:44 -0400 (EDT) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id B2C5E181AEF1F for ; Thu, 27 May 2021 15:05:43 +0000 (UTC) X-FDA: 78187335366.11.63B4DED Received: from mail-qt1-f180.google.com (mail-qt1-f180.google.com [209.85.160.180]) by imf12.hostedemail.com (Postfix) with ESMTP id 42BD42C8 for ; Thu, 27 May 2021 15:05:32 +0000 (UTC) Received: by mail-qt1-f180.google.com with SMTP id c15so423305qte.6 for ; Thu, 27 May 2021 08:05:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=I6lkOzPEa6syzQf8r+RZ8XuPpKpclGEbIUwShbrK82U=; b=TnnFfUUvHvEm/etkMIl6FavDDCjxG9hxxxgfKcLeF7n8pbKBfizh0PVGR9SMdhavpQ /qaXI18/FRxFUfElEb2Mw/gUsiyEEVcvLvMB17wq4wMBqThZ8ZW1wxboXPji+jtFXAzA dmFznHFw0mFTEWzs07I5Or3POBDkvY2Li5BAWGSV9TlzSh3tpEA7QozmVmOde0tdRNWJ Ho9DhH92QuT2QsQkhMfXBenFVxdWr5De+0nSVKKvmbALAiuom+VmBP2Ocl+xIXaospwh fhvqjPCz2kafZH+AeYAvRnXpZ979KsJEGkCMNPqV9/MZKUUhRoCgzAp927vHfgXWTBlK QONw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=I6lkOzPEa6syzQf8r+RZ8XuPpKpclGEbIUwShbrK82U=; b=uZqZiY3JjGKy040ZvgUeDYPfgzGdMWZDlzfM82vKD7qBAWGNXkO6+KDaQoYhCOeITm Qj9mfIewvIoOQWZCbtyYzhr+l0N1STzZ87x6aR09pd6yMXYSzI633E8ZrBkw5qLdBcm1 OPSD9p8xMfwvdK/UR2kdAf/xkfbvINDLQBA5B+qLNT/K3DLKG060X3iuqc8IT7w6jDE9 RaPjs2NNMCJkJjexiDmSPzdS20v5V3irAe43Bi3/khou3g/iHunFqWWdQ1y31ursJHeN OU2e0Xz3hdTCYWsQX/X+Cq3FHxqHeTdpf6PV4sVIf9gorWi86PxFWGc77HrTqXwVnZ6e KpsA== X-Gm-Message-State: AOAM532wz3BApBP8mhUvTA+DvTrnaDoufidxN8twfw9D/Git/ksSmVqJ t4T3czYxEFxitLlVATuq7Mwlzg== X-Google-Smtp-Source: ABdhPJyswzPAqJ6yfkg4N9Ue8mJrxUt1+TgPxwsNMw1UD8Ns1Nxidx0p88Kxuz6WsVPrCUipNJJJbg== X-Received: by 2002:ac8:5e98:: with SMTP id r24mr3491247qtx.25.1622127942481; Thu, 27 May 2021 08:05:42 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id x10sm1447468qkh.124.2021.05.27.08.05.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 May 2021 08:05:42 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH 09/18] arm64: kexec: Use dcache ops macros instead of open-coding Date: Thu, 27 May 2021 11:05:17 -0400 Message-Id: <20210527150526.271941-10-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210527150526.271941-1-pasha.tatashin@soleen.com> References: <20210527150526.271941-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=soleen.com header.s=google header.b=TnnFfUUv; dmarc=none; spf=pass (imf12.hostedemail.com: domain of pasha.tatashin@soleen.com designates 209.85.160.180 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com X-Stat-Signature: 4e8qqwuwpdkajz1767jjdoto8nec1be1 X-Rspamd-Queue-Id: 42BD42C8 X-Rspamd-Server: rspam02 X-HE-Tag: 1622127932-965888 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: James Morse kexec does dcache maintenance when it re-writes all memory. Our dcache_by_line_op macro depends on reading the sanitised DminLine from memory. Kexec may have overwritten this, so open-codes the sequence. dcache_by_line_op is a whole set of macros, it uses dcache_line_size which uses read_ctr for the sanitsed DminLine. Reading the DminLine is the first thing the dcache_by_line_op does. Rename dcache_by_line_op dcache_by_myline_op and take DminLine as an argument. Kexec can now use the slightly smaller macro. This makes up-coming changes to the dcache maintenance easier on the eye. Code generated by the existing callers is unchanged. Signed-off-by: James Morse [Fixed merging issues] Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/assembler.h | 12 ++++++++---- arch/arm64/kernel/relocate_kernel.S | 13 +++---------- 2 files changed, 11 insertions(+), 14 deletions(-) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 8418c1bd8f04..f1367863d995 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -393,10 +393,9 @@ alternative_else alternative_endif .endm - .macro dcache_by_line_op op, domain, kaddr, size, tmp1, tmp2 - dcache_line_size \tmp1, \tmp2 + .macro dcache_by_myline_op op, domain, kaddr, size, linesz, tmp2 add \size, \kaddr, \size - sub \tmp2, \tmp1, #1 + sub \tmp2, \linesz, #1 bic \kaddr, \kaddr, \tmp2 9998: .ifc \op, cvau @@ -416,12 +415,17 @@ alternative_endif .endif .endif .endif - add \kaddr, \kaddr, \tmp1 + add \kaddr, \kaddr, \linesz cmp \kaddr, \size b.lo 9998b dsb \domain .endm + .macro dcache_by_line_op op, domain, kaddr, size, tmp1, tmp2 + dcache_line_size \tmp1, \tmp2 + dcache_by_myline_op \op, \domain, \kaddr, \size, \tmp1, \tmp2 + .endm + /* * Macro to perform an instruction cache maintenance for the interval * [start, end) diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index 8058fabe0a76..718037bef560 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -41,16 +41,9 @@ SYM_CODE_START(arm64_relocate_new_kernel) tbz x16, IND_SOURCE_BIT, .Ltest_indirection /* Invalidate dest page to PoC. */ - mov x2, x13 - add x20, x2, #PAGE_SIZE - sub x1, x15, #1 - bic x2, x2, x1 -2: dc ivac, x2 - add x2, x2, x15 - cmp x2, x20 - b.lo 2b - dsb sy - + mov x2, x13 + mov x1, #PAGE_SIZE + dcache_by_myline_op ivac, sy, x2, x1, x15, x20 copy_page x13, x12, x1, x2, x3, x4, x5, x6, x7, x8 b .Lnext .Ltest_indirection: From patchwork Thu May 27 15:05:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12284545 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5288C4708A for ; Thu, 27 May 2021 15:05:53 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 92E21613C9 for ; Thu, 27 May 2021 15:05:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 92E21613C9 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C7A406B0080; Thu, 27 May 2021 11:05:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C50DA6B0081; Thu, 27 May 2021 11:05:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A7FAD8D0001; Thu, 27 May 2021 11:05:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0069.hostedemail.com [216.40.44.69]) by kanga.kvack.org (Postfix) with ESMTP id 6F83E6B0080 for ; Thu, 27 May 2021 11:05:45 -0400 (EDT) Received: from smtpin38.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 15BD98249980 for ; Thu, 27 May 2021 15:05:45 +0000 (UTC) X-FDA: 78187335450.38.337BDDC Received: from mail-qv1-f43.google.com (mail-qv1-f43.google.com [209.85.219.43]) by imf17.hostedemail.com (Postfix) with ESMTP id 499D840F8C18 for ; Thu, 27 May 2021 15:05:39 +0000 (UTC) Received: by mail-qv1-f43.google.com with SMTP id e8so240815qvp.7 for ; Thu, 27 May 2021 08:05:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=wPqWbOC642+FQuLJdmHZY8xtJm1ZVhhgjTRiZDTZY7Q=; b=OYcSb6MwLOEpinFGHYb3FQsN6yhufIjwNbxeyeRaz5wrZtQ/5vpDYqcsw6mj8mqgFS 5UE2pAQSBYMHHc5pzxtMNiWczCnfbE1ewNR25yNAc51ClquEAU42eWpD0wf7yKRwaa8u EJ4J1rzwEIAcPfcFUk17zjO/4bk1E+3laPOMgMNvpcUPtx9wdkjNK16ungcGbtUZZggp R0htuLwV94wC/5xA6en5rnhOodoYgPQYgWwBNINEbi9IsatqBRJokknUJ9QHGsoZbH40 o1WrbfLH0KcBdE54RrkTFHAtvePLYHlUx7qY91Nve36OHBX/DaA3M7dEWr+EOsekJxO5 HojQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wPqWbOC642+FQuLJdmHZY8xtJm1ZVhhgjTRiZDTZY7Q=; b=GCBRO9DTERQMxMjYtEwbRdc5/NaLUPjVCDhVs2hrNyt2YuYFXqq+7LKTIXJusXwsss vrGneKRNZ56tFNscfemYrfgLRh1tLy7cAScO+W4owtfs2li8XfMg28W+npu3QRLdtvMQ LLxfNt3INfGOJIL+dL6Jqze4kEAF1AAcJ/zPJnQyC921fmlSEWCQxWsuhmCPYxvnTcD/ +qRnSyQ+jKe44KMOJwwkTnClpALOHwDUlMqZG8ZbA+lRenfVwJd1+V9ghybtMtCUyh7y bIs9RZeNtvu+omr/krAM+D1/2Rf8haanRDn3mqz8OysT9CvMqrdy1YKF7sjPtpab7jbN uuMA== X-Gm-Message-State: AOAM533oJ7VWGch+3NzQGz3Z6r/HsM/WjmbG7iT1kXM+qndlPxRAsFcU A5wQC/cVT8zZDucIV6PqqRUAcg== X-Google-Smtp-Source: ABdhPJyMEicr+0neWvvsEC3Ec9nXUiLYCTi0Iea32NIOel70L8g//7eWi+fs+fDfMn3EnbeZcrliew== X-Received: by 2002:ad4:536a:: with SMTP id e10mr3765205qvv.9.1622127943895; Thu, 27 May 2021 08:05:43 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id x10sm1447468qkh.124.2021.05.27.08.05.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 May 2021 08:05:43 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH 10/18] arm64: kexec: pass kimage as the only argument to relocation function Date: Thu, 27 May 2021 11:05:18 -0400 Message-Id: <20210527150526.271941-11-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210527150526.271941-1-pasha.tatashin@soleen.com> References: <20210527150526.271941-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=soleen.com header.s=google header.b=OYcSb6Mw; dmarc=none; spf=pass (imf17.hostedemail.com: domain of pasha.tatashin@soleen.com designates 209.85.219.43 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 499D840F8C18 X-Stat-Signature: unojnskcs14qz4ro3nffg8ucag79wui8 X-HE-Tag: 1622127939-197180 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently, kexec relocation function (arm64_relocate_new_kernel) accepts the following arguments: head: start of array that contains relocation information. entry: entry point for new kernel or purgatory. dtb_mem: first and only argument to entry. The number of arguments cannot be easily expended, because this function is also called from HVC_SOFT_RESTART, which preserves only three arguments. And, also arm64_relocate_new_kernel is written in assembly but called without stack, thus no place to move extra arguments to free registers. Soon, we will need to pass more arguments: once we enable MMU we will need to pass information about page tables. Pass kimage to arm64_relocate_new_kernel, and teach it to get the required fields from kimage. Suggested-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/asm-offsets.c | 7 +++++++ arch/arm64/kernel/machine_kexec.c | 6 ++++-- arch/arm64/kernel/relocate_kernel.S | 10 ++++------ 3 files changed, 15 insertions(+), 8 deletions(-) diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index 0cb34ccb6e73..92c2ed846730 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -9,6 +9,7 @@ #include #include +#include #include #include #include @@ -155,6 +156,12 @@ int main(void) DEFINE(PTRAUTH_USER_KEY_APIA, offsetof(struct ptrauth_keys_user, apia)); DEFINE(PTRAUTH_KERNEL_KEY_APIA, offsetof(struct ptrauth_keys_kernel, apia)); BLANK(); +#endif +#ifdef CONFIG_KEXEC_CORE + DEFINE(KIMAGE_ARCH_DTB_MEM, offsetof(struct kimage, arch.dtb_mem)); + DEFINE(KIMAGE_HEAD, offsetof(struct kimage, head)); + DEFINE(KIMAGE_START, offsetof(struct kimage, start)); + BLANK(); #endif return 0; } diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index bde623805866..76a757954a70 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -83,6 +83,8 @@ static void kexec_list_flush(struct kimage *kimage) { kimage_entry_t *entry; + __flush_dcache_area(kimage, sizeof(*kimage)); + for (entry = &kimage->head; ; entry++) { unsigned int flag; void *addr; @@ -198,8 +200,8 @@ void machine_kexec(struct kimage *kimage) restart(is_hyp_callable(), kimage->start, kimage->arch.dtb_mem, 0, 0); } else { - cpu_soft_restart(kimage->arch.kern_reloc, kimage->head, - kimage->start, kimage->arch.dtb_mem); + cpu_soft_restart(kimage->arch.kern_reloc, virt_to_phys(kimage), + 0, 0); } BUG(); /* Should never get here. */ diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index 718037bef560..36b4496524c3 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -27,9 +27,7 @@ */ SYM_CODE_START(arm64_relocate_new_kernel) /* Setup the list loop variables. */ - mov x18, x2 /* x18 = dtb address */ - mov x17, x1 /* x17 = kimage_start */ - mov x16, x0 /* x16 = kimage_head */ + ldr x16, [x0, #KIMAGE_HEAD] /* x16 = kimage_head */ mov x14, xzr /* x14 = entry ptr */ mov x13, xzr /* x13 = copy dest */ raw_dcache_line_size x15, x1 /* x15 = dcache line size */ @@ -63,12 +61,12 @@ SYM_CODE_START(arm64_relocate_new_kernel) isb /* Start new image. */ - mov x0, x18 + ldr x4, [x0, #KIMAGE_START] /* relocation start */ + ldr x0, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ mov x1, xzr mov x2, xzr mov x3, xzr - br x17 - + br x4 SYM_CODE_END(arm64_relocate_new_kernel) .align 3 /* To keep the 64-bit values below naturally aligned. */ From patchwork Thu May 27 15:05:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12284547 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3D4C8C47089 for ; Thu, 27 May 2021 15:05:56 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E77C561358 for ; Thu, 27 May 2021 15:05:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E77C561358 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 885686B0081; Thu, 27 May 2021 11:05:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 834A88D0001; Thu, 27 May 2021 11:05:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 661816B0083; Thu, 27 May 2021 11:05:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0238.hostedemail.com [216.40.44.238]) by kanga.kvack.org (Postfix) with ESMTP id 182EA6B0081 for ; Thu, 27 May 2021 11:05:47 -0400 (EDT) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 83BAC9983 for ; Thu, 27 May 2021 15:05:46 +0000 (UTC) X-FDA: 78187335492.07.868703C Received: from mail-qt1-f177.google.com (mail-qt1-f177.google.com [209.85.160.177]) by imf02.hostedemail.com (Postfix) with ESMTP id 9B63040F8C1F for ; Thu, 27 May 2021 15:05:41 +0000 (UTC) Received: by mail-qt1-f177.google.com with SMTP id i12so415589qtr.7 for ; Thu, 27 May 2021 08:05:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=IRY2JW0Z98EgQbSxHxMAz5C8JAIM2WaC06Y7zNJ3ts0=; b=boiyFkgbfwJ557d627q5so2dUiE6lq9T3OIPmL9hmxupf/IKh//+zJ11bd9/71UrxU 4/yFmZxfMnQEIS+a9xTcY8K45N6bfu78nowipvRRn3DekJXFqP1bNQhrlM61+qnG+6U2 ij8NwJJJetIorTETOnOtqLp31yYSpvkF86AW4MLlWyUbL695KiY8+ZwAQVj/imAl3lpi jPy1zngJ8iwtzSEbmLyKysfKzGJBbciUNcKPi6YPKb1SVC28YKAyeFtS3/mNd1WPrsxg iKZRAMuk4JrqoixsI49a7oYfLibiVjfBVjiN46k/FwAz4CfK7KMbflX0j9QOdzHCqkkt p4JA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=IRY2JW0Z98EgQbSxHxMAz5C8JAIM2WaC06Y7zNJ3ts0=; b=XyoalxBsGEZ6uocfzFzgRxoagTPtfPUmGNp7G/jSL0SKiNt+D0dqjE13dc5EuG7QiW QUJq7XGKa6YDv+cZ9t+t3Rq2RZS95MUzQhV2bqKiht+oq2hU2aLhYTOu7olPyRtq6rd/ l6/SqW7ncnaujI6BYgvLwJ6MNEnFwxNRTrRD1RCLYTY/j1edEwuCE+C0THHgDrK976PL T+JpLWBEGuA6y5W7LUE/bhpXYaCLQQxApBOW6OFibL4PhqAhlguZu0WhneOJh5uMGWDT 0jPo8UfATWlNGAHnWd66b54SrPeE6q/7L1wBW8QmU4bbnoSmZqwnNtI83lOdBVdPDtU2 zYcA== X-Gm-Message-State: AOAM530FMlFb2LLvd8CkRzbbZieiGZ7COiaQ1DtHI40qfvHJ2JrcfC/r 0ZOp+GFV4lM/QX8PcMkZUBNqjA== X-Google-Smtp-Source: ABdhPJxvoayShnrBYhLGN1MUkwd1W/Ry1DebaPAbHABcNwTi2nuw66ZfhC1rl0DYvGfXedawKnn4vw== X-Received: by 2002:a05:622a:28f:: with SMTP id z15mr3591511qtw.260.1622127945305; Thu, 27 May 2021 08:05:45 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id x10sm1447468qkh.124.2021.05.27.08.05.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 May 2021 08:05:44 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH 11/18] arm64: kexec: kexec may require EL2 vectors Date: Thu, 27 May 2021 11:05:19 -0400 Message-Id: <20210527150526.271941-12-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210527150526.271941-1-pasha.tatashin@soleen.com> References: <20210527150526.271941-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 9B63040F8C1F Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=soleen.com header.s=google header.b=boiyFkgb; spf=pass (imf02.hostedemail.com: domain of pasha.tatashin@soleen.com designates 209.85.160.177 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com; dmarc=none X-Rspamd-Server: rspam04 X-Stat-Signature: g8ig9iggoiimz6ouenj8zj7sku3mfd8z X-HE-Tag: 1622127941-911168 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: If we have a EL2 mode without VHE, the EL2 vectors are needed in order to switch to EL2 and jump to new world with hypervisor privileges. In preporation to MMU enabled relocation, configure our EL2 table now. Suggested-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/Kconfig | 2 +- arch/arm64/include/asm/kexec.h | 1 + arch/arm64/kernel/asm-offsets.c | 1 + arch/arm64/kernel/machine_kexec.c | 31 +++++++++++++++++++++++++++++++ 4 files changed, 34 insertions(+), 1 deletion(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 9f1d8566bbf9..e775e6fb03c5 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -1148,7 +1148,7 @@ config CRASH_DUMP config TRANS_TABLE def_bool y - depends on HIBERNATION + depends on HIBERNATION || KEXEC_CORE config XEN_DOM0 def_bool y diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h index 00dbcc71aeb2..753a1c398898 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -96,6 +96,7 @@ struct kimage_arch { void *dtb; phys_addr_t dtb_mem; phys_addr_t kern_reloc; + phys_addr_t el2_vectors; }; #ifdef CONFIG_KEXEC_FILE diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index 92c2ed846730..ee37d0e0fb6d 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -159,6 +159,7 @@ int main(void) #endif #ifdef CONFIG_KEXEC_CORE DEFINE(KIMAGE_ARCH_DTB_MEM, offsetof(struct kimage, arch.dtb_mem)); + DEFINE(KIMAGE_ARCH_EL2_VECTORS, offsetof(struct kimage, arch.el2_vectors)); DEFINE(KIMAGE_HEAD, offsetof(struct kimage, head)); DEFINE(KIMAGE_START, offsetof(struct kimage, start)); BLANK(); diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 76a757954a70..9f74791cb1d0 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -20,6 +20,7 @@ #include #include #include +#include #include "cpu-reset.h" @@ -42,7 +43,9 @@ static void _kexec_image_info(const char *func, int line, pr_debug(" start: %lx\n", kimage->start); pr_debug(" head: %lx\n", kimage->head); pr_debug(" nr_segments: %lu\n", kimage->nr_segments); + pr_debug(" dtb_mem: %pa\n", &kimage->arch.dtb_mem); pr_debug(" kern_reloc: %pa\n", &kimage->arch.kern_reloc); + pr_debug(" el2_vectors: %pa\n", &kimage->arch.el2_vectors); for (i = 0; i < kimage->nr_segments; i++) { pr_debug(" segment[%lu]: %016lx - %016lx, 0x%lx bytes, %lu pages\n", @@ -137,9 +140,27 @@ static void kexec_segment_flush(const struct kimage *kimage) } } +/* Allocates pages for kexec page table */ +static void *kexec_page_alloc(void *arg) +{ + struct kimage *kimage = (struct kimage *)arg; + struct page *page = kimage_alloc_control_pages(kimage, 0); + + if (!page) + return NULL; + + memset(page_address(page), 0, PAGE_SIZE); + + return page_address(page); +} + int machine_kexec_post_load(struct kimage *kimage) { void *reloc_code = page_to_virt(kimage->control_code_page); + struct trans_pgd_info info = { + .trans_alloc_page = kexec_page_alloc, + .trans_alloc_arg = kimage, + }; /* If in place, relocation is not used, only flush next kernel */ if (kimage->head & IND_DONE) { @@ -148,6 +169,14 @@ int machine_kexec_post_load(struct kimage *kimage) return 0; } + kimage->arch.el2_vectors = 0; + if (is_hyp_callable()) { + int rc = trans_pgd_copy_el2_vectors(&info, + &kimage->arch.el2_vectors); + if (rc) + return rc; + } + memcpy(reloc_code, arm64_relocate_new_kernel, arm64_relocate_new_kernel_size); kimage->arch.kern_reloc = __pa(reloc_code); @@ -200,6 +229,8 @@ void machine_kexec(struct kimage *kimage) restart(is_hyp_callable(), kimage->start, kimage->arch.dtb_mem, 0, 0); } else { + if (is_hyp_callable()) + __hyp_set_vectors(kimage->arch.el2_vectors); cpu_soft_restart(kimage->arch.kern_reloc, virt_to_phys(kimage), 0, 0); } From patchwork Thu May 27 15:05:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12284549 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8131AC47089 for ; Thu, 27 May 2021 15:05:58 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2785A6128D for ; Thu, 27 May 2021 15:05:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2785A6128D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A591B6B0082; Thu, 27 May 2021 11:05:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A30716B0083; Thu, 27 May 2021 11:05:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 85C026B0085; Thu, 27 May 2021 11:05:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0202.hostedemail.com [216.40.44.202]) by kanga.kvack.org (Postfix) with ESMTP id 477AF6B0082 for ; Thu, 27 May 2021 11:05:48 -0400 (EDT) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id E1718AF6A for ; Thu, 27 May 2021 15:05:47 +0000 (UTC) X-FDA: 78187335534.12.FAA6DDF Received: from mail-qk1-f173.google.com (mail-qk1-f173.google.com [209.85.222.173]) by imf03.hostedemail.com (Postfix) with ESMTP id 7FE7CC0007C0 for ; Thu, 27 May 2021 15:05:39 +0000 (UTC) Received: by mail-qk1-f173.google.com with SMTP id j184so930934qkd.6 for ; Thu, 27 May 2021 08:05:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=spq5ynnfnR8TM5XEPzllfdybPKsLwK1jYCVhSyBxi9M=; b=kEzFCK/OZq65q45023j25DBKIv6t/KPooiYfjWWPP3IJOjjHeRmCpQtuVLJkoQfk0m LLZVUvatSqsTefwkG844IqhjmqYz+eKjmIWMguCbE//G/PZilV1ovqWDnr3/kiQThoLe 8J5fE0vuDmRoHMifm12gmszoCPbgL/6vKPWVbsXnCVp6POQBaG8mMnHIcVImT2bPCI1y BV/yFm5eIJfxxYVdS1bcl46gMJrIytDeVhMlnOV5TIYrxeOwAZcaQxROJ+GyKDP9GrtS PtTEcv1/j/uuZXYBfqEmqGTXfvUSywNUrIEs2IE6VveTwxDnHBc/1L6sfb2iuapuRDjs +mIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=spq5ynnfnR8TM5XEPzllfdybPKsLwK1jYCVhSyBxi9M=; b=fSSTR6nISxMHvcS9ifxjdwf+IEjlLeqkhpfRdTQqCb6n/S+vhL8dqdtTrIKPkIUOZZ +tv15r8spQ6wBmh+H1ktjdWqMcPnX2YcgCBFS2wSfUUCOBXkQlUTzLWijjxoni83JO+u CHfIOG+0TN5QgVJ01xi+/xADmbRQsF3kX0HDIRs0cPSy5O9+EB+BBZnCWQZ5f3+Bpdry CzZ55v0kp0PjJZsl2v2iMnEFOvYlpEMPNGIA60UwT/epSF4U0l7oDzh0+0Hm1yF6z7hE kaqS7siEFx+DeGEJeJEAhTwE4vSM2mNMpzBoD+iWdNzJdW592V9SGI8Epn3aIXpJxjcm wCKg== X-Gm-Message-State: AOAM533wIGru+zbv7TMm6Ogfha98B790xkqtK03R80C7tq1vhmAL2TxR nMi7QcumEG5v8me5zPdkC7yu9A== X-Google-Smtp-Source: ABdhPJwCPp5AbyvNOecFI+FqtUWaR34fdNcgOcCR3qCe2Y2zX902CBP0VehAshlb1076RC/1us6hfg== X-Received: by 2002:a05:620a:2282:: with SMTP id o2mr3866648qkh.479.1622127946786; Thu, 27 May 2021 08:05:46 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id x10sm1447468qkh.124.2021.05.27.08.05.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 May 2021 08:05:46 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH 12/18] arm64: kexec: relocate in EL1 mode Date: Thu, 27 May 2021 11:05:20 -0400 Message-Id: <20210527150526.271941-13-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210527150526.271941-1-pasha.tatashin@soleen.com> References: <20210527150526.271941-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=soleen.com header.s=google header.b="kEzFCK/O"; dmarc=none; spf=pass (imf03.hostedemail.com: domain of pasha.tatashin@soleen.com designates 209.85.222.173 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com X-Stat-Signature: 3pc7jgnfij4uke7ijjyn9dtj7rgtejb4 X-Rspamd-Queue-Id: 7FE7CC0007C0 X-Rspamd-Server: rspam02 X-HE-Tag: 1622127939-458569 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Since we are going to keep MMU enabled during relocation, we need to keep EL1 mode throughout the relocation. Keep EL1 enabled, and switch EL2 only before enterying the new world. Suggested-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/cpu-reset.h | 3 +-- arch/arm64/kernel/machine_kexec.c | 4 ++-- arch/arm64/kernel/relocate_kernel.S | 13 +++++++++++-- 3 files changed, 14 insertions(+), 6 deletions(-) diff --git a/arch/arm64/kernel/cpu-reset.h b/arch/arm64/kernel/cpu-reset.h index 48d0ed48c147..296abbac7192 100644 --- a/arch/arm64/kernel/cpu-reset.h +++ b/arch/arm64/kernel/cpu-reset.h @@ -20,11 +20,10 @@ static inline void __noreturn __nocfi cpu_soft_restart(unsigned long entry, { typeof(__cpu_soft_restart) *restart; - unsigned long el2_switch = is_hyp_callable(); restart = (void *)__pa_symbol(function_nocfi(__cpu_soft_restart)); cpu_install_idmap(); - restart(el2_switch, entry, arg0, arg1, arg2); + restart(0, entry, arg0, arg1, arg2); unreachable(); } diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 9f74791cb1d0..c9993f029262 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -231,8 +231,8 @@ void machine_kexec(struct kimage *kimage) } else { if (is_hyp_callable()) __hyp_set_vectors(kimage->arch.el2_vectors); - cpu_soft_restart(kimage->arch.kern_reloc, virt_to_phys(kimage), - 0, 0); + cpu_soft_restart(kimage->arch.kern_reloc, + virt_to_phys(kimage), 0, 0); } BUG(); /* Should never get here. */ diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index 36b4496524c3..df023b82544b 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -13,6 +13,7 @@ #include #include #include +#include /* * arm64_relocate_new_kernel - Put a 2nd stage image in place and boot it. @@ -61,12 +62,20 @@ SYM_CODE_START(arm64_relocate_new_kernel) isb /* Start new image. */ + ldr x1, [x0, #KIMAGE_ARCH_EL2_VECTORS] /* relocation start */ + cbz x1, .Lel1 + ldr x1, [x0, #KIMAGE_START] /* relocation start */ + ldr x2, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ + mov x3, xzr + mov x4, xzr + mov x0, #HVC_SOFT_RESTART + hvc #0 /* Jumps from el2 */ +.Lel1: ldr x4, [x0, #KIMAGE_START] /* relocation start */ ldr x0, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ - mov x1, xzr mov x2, xzr mov x3, xzr - br x4 + br x4 /* Jumps from el1 */ SYM_CODE_END(arm64_relocate_new_kernel) .align 3 /* To keep the 64-bit values below naturally aligned. */ From patchwork Thu May 27 15:05:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12284551 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E19B5C4707F for ; Thu, 27 May 2021 15:06:00 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9089A6128D for ; Thu, 27 May 2021 15:06:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9089A6128D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 638956B0083; Thu, 27 May 2021 11:05:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6111B6B0085; Thu, 27 May 2021 11:05:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 464156B0087; Thu, 27 May 2021 11:05:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0139.hostedemail.com [216.40.44.139]) by kanga.kvack.org (Postfix) with ESMTP id EBC846B0083 for ; Thu, 27 May 2021 11:05:49 -0400 (EDT) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 80CEB9405 for ; Thu, 27 May 2021 15:05:49 +0000 (UTC) X-FDA: 78187335618.10.53308E5 Received: from mail-qt1-f177.google.com (mail-qt1-f177.google.com [209.85.160.177]) by imf23.hostedemail.com (Postfix) with ESMTP id 0CD6FA0001ED for ; Thu, 27 May 2021 15:05:41 +0000 (UTC) Received: by mail-qt1-f177.google.com with SMTP id t17so402920qta.11 for ; Thu, 27 May 2021 08:05:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=M+UnYXxw8nkrHqAjav5617WACApgFL7LNjySWDjNsqw=; b=OM8jv6dIvYQ/oU4lPaj545Lb6wcf2ECof0M+ej1+Y+Zo0H/YsWGYtnXhh2nlXF8qzz Ae7FdACu6ubQkw8iOcaZWTKXb3SvgrwUOl1OdUNFT0Wv76oAZtQ5DBjnYrbry7/jXH0/ 5ThzZ5f/USoigM/PsymqzJFSNqfgHrhULDkokiDhB8JL6eVaxHY/QGuuRZcY7M5QPpVR zMh4PH4RvS17sRS25iqSOVMJYGoSlE+H9TeViZoU4y8wqVULNRlAywaMpnrsLA+R7ULL qVC/ljBDadvYtAPXy4npxJ0yHe1c1a21Xt4Tf9L0tQ14+Wfa2HBtlw3qWqNLUAAACY3P o2Yw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=M+UnYXxw8nkrHqAjav5617WACApgFL7LNjySWDjNsqw=; b=OQiTs2ui7g4YCN1UQnUadgO1+M3q0ZueRiRCHabwf/45SNVySislIF8N/Z4u5HHFmj vFQU1pR615ur5Vcne2RSJFv5i42vZ60QDfg4X8H5DXaLZCxtLpjV65PGZynq/RJAMYox U/Fzi8xSUtuCqP6t132xL2Ex/HqBHADF0GrIU6xuuGYIruvYtS2bchhO5xIvfX6EF/6R N0707LtNLSR569bSRIdQc1rAngZ+A67gJ0Ip35wQkz4VGgnJizW9+bUh2IvKAymy112v lHE4cNDySVZL8T7X6v5xRasRmfxs0yH2zbIl4UqbjVh7HJiNKegKIw1jz0ouV6srU2Ti /org== X-Gm-Message-State: AOAM5326NDFIVCCaoHfeKbRTkfBg0+gSOaiOCyZ7YiYBKG8fHIJ/XFT5 g22hu5pUclnsDmxprlTOtHnAqg== X-Google-Smtp-Source: ABdhPJykKRJouKWov8JqAp4TT6mHaHetjX4h/gTdtSRZ4IF/EPEd/iut0BXLaxTvC2S2l28hvd4vGg== X-Received: by 2002:ac8:13c3:: with SMTP id i3mr3613497qtj.322.1622127948200; Thu, 27 May 2021 08:05:48 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id x10sm1447468qkh.124.2021.05.27.08.05.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 May 2021 08:05:47 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH 13/18] arm64: kexec: use ld script for relocation function Date: Thu, 27 May 2021 11:05:21 -0400 Message-Id: <20210527150526.271941-14-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210527150526.271941-1-pasha.tatashin@soleen.com> References: <20210527150526.271941-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=soleen.com header.s=google header.b=OM8jv6dI; dmarc=none; spf=pass (imf23.hostedemail.com: domain of pasha.tatashin@soleen.com designates 209.85.160.177 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com X-Stat-Signature: adzx889mk5au3ztufdofnxofefdgodcw X-Rspamd-Queue-Id: 0CD6FA0001ED X-Rspamd-Server: rspam02 X-HE-Tag: 1622127941-909605 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently, relocation code declares start and end variables which are used to compute its size. The better way to do this is to use ld script incited, and put relocation function in its own section. Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/sections.h | 1 + arch/arm64/kernel/machine_kexec.c | 14 ++++++-------- arch/arm64/kernel/relocate_kernel.S | 15 ++------------- arch/arm64/kernel/vmlinux.lds.S | 19 +++++++++++++++++++ 4 files changed, 28 insertions(+), 21 deletions(-) diff --git a/arch/arm64/include/asm/sections.h b/arch/arm64/include/asm/sections.h index e4ad9db53af1..152cb35bf9df 100644 --- a/arch/arm64/include/asm/sections.h +++ b/arch/arm64/include/asm/sections.h @@ -21,5 +21,6 @@ extern char __exittext_begin[], __exittext_end[]; extern char __irqentry_text_start[], __irqentry_text_end[]; extern char __mmuoff_data_start[], __mmuoff_data_end[]; extern char __entry_tramp_text_start[], __entry_tramp_text_end[]; +extern char __relocate_new_kernel_start[], __relocate_new_kernel_end[]; #endif /* __ASM_SECTIONS_H */ diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index c9993f029262..675d27d7aea4 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -20,14 +20,11 @@ #include #include #include +#include #include #include "cpu-reset.h" -/* Global variables for the arm64_relocate_new_kernel routine. */ -extern const unsigned char arm64_relocate_new_kernel[]; -extern const unsigned long arm64_relocate_new_kernel_size; - /** * kexec_image_info - For debugging output. */ @@ -157,6 +154,7 @@ static void *kexec_page_alloc(void *arg) int machine_kexec_post_load(struct kimage *kimage) { void *reloc_code = page_to_virt(kimage->control_code_page); + long reloc_size; struct trans_pgd_info info = { .trans_alloc_page = kexec_page_alloc, .trans_alloc_arg = kimage, @@ -177,14 +175,14 @@ int machine_kexec_post_load(struct kimage *kimage) return rc; } - memcpy(reloc_code, arm64_relocate_new_kernel, - arm64_relocate_new_kernel_size); + reloc_size = __relocate_new_kernel_end - __relocate_new_kernel_start; + memcpy(reloc_code, __relocate_new_kernel_start, reloc_size); kimage->arch.kern_reloc = __pa(reloc_code); /* Flush the reloc_code in preparation for its execution. */ - __flush_dcache_area(reloc_code, arm64_relocate_new_kernel_size); + __flush_dcache_area(reloc_code, reloc_size); flush_icache_range((uintptr_t)reloc_code, (uintptr_t)reloc_code + - arm64_relocate_new_kernel_size); + reloc_size); kexec_list_flush(kimage); kexec_image_info(kimage); diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index df023b82544b..7a600ba33ae1 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -15,6 +15,7 @@ #include #include +.pushsection ".kexec_relocate.text", "ax" /* * arm64_relocate_new_kernel - Put a 2nd stage image in place and boot it. * @@ -77,16 +78,4 @@ SYM_CODE_START(arm64_relocate_new_kernel) mov x3, xzr br x4 /* Jumps from el1 */ SYM_CODE_END(arm64_relocate_new_kernel) - -.align 3 /* To keep the 64-bit values below naturally aligned. */ - -.Lcopy_end: -.org KEXEC_CONTROL_PAGE_SIZE - -/* - * arm64_relocate_new_kernel_size - Number of bytes to copy to the - * control_code_page. - */ -.globl arm64_relocate_new_kernel_size -arm64_relocate_new_kernel_size: - .quad .Lcopy_end - arm64_relocate_new_kernel +.popsection diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index 709d2c433c5e..9479d66f03bf 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -63,6 +63,7 @@ #include #include #include +#include #include #include @@ -100,6 +101,16 @@ jiffies = jiffies_64; #define HIBERNATE_TEXT #endif +#ifdef CONFIG_KEXEC_CORE +#define KEXEC_TEXT \ + . = ALIGN(SZ_4K); \ + __relocate_new_kernel_start = .; \ + *(.kexec_relocate.text) \ + __relocate_new_kernel_end = .; +#else +#define KEXEC_TEXT +#endif + #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 #define TRAMP_TEXT \ . = ALIGN(PAGE_SIZE); \ @@ -160,6 +171,7 @@ SECTIONS HYPERVISOR_TEXT IDMAP_TEXT HIBERNATE_TEXT + KEXEC_TEXT TRAMP_TEXT *(.fixup) *(.gnu.warning) @@ -348,3 +360,10 @@ ASSERT(swapper_pg_dir - reserved_pg_dir == RESERVED_SWAPPER_OFFSET, ASSERT(swapper_pg_dir - tramp_pg_dir == TRAMP_SWAPPER_OFFSET, "TRAMP_SWAPPER_OFFSET is wrong!") #endif + +#ifdef CONFIG_KEXEC_CORE +/* kexec relocation code should fit into one KEXEC_CONTROL_PAGE_SIZE */ +ASSERT(__relocate_new_kernel_end - (__relocate_new_kernel_start & ~(SZ_4K - 1)) + <= SZ_4K, "kexec relocation code is too big or misaligned") +ASSERT(KEXEC_CONTROL_PAGE_SIZE >= SZ_4K, "KEXEC_CONTROL_PAGE_SIZE is brokern") +#endif From patchwork Thu May 27 15:05:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12284553 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D2B8C4707F for ; Thu, 27 May 2021 15:06:03 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 26D8161005 for ; Thu, 27 May 2021 15:06:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 26D8161005 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E76576B0085; Thu, 27 May 2021 11:05:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E51076B0087; Thu, 27 May 2021 11:05:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BE0B26B0088; Thu, 27 May 2021 11:05:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 819E46B0085 for ; Thu, 27 May 2021 11:05:51 -0400 (EDT) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 1BA589405 for ; Thu, 27 May 2021 15:05:51 +0000 (UTC) X-FDA: 78187335702.13.2FED338 Received: from mail-qk1-f171.google.com (mail-qk1-f171.google.com [209.85.222.171]) by imf05.hostedemail.com (Postfix) with ESMTP id DBAD3E000811 for ; Thu, 27 May 2021 15:05:39 +0000 (UTC) Received: by mail-qk1-f171.google.com with SMTP id o3so924561qke.7 for ; Thu, 27 May 2021 08:05:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=ZG1hbdIv3FvUkXzGyiyJBV6DrXE2wtuUaoQN6LGlj1M=; b=PxjTnw1/0d/OHXwz8yz4wn5Hg8O1MgJq99+RWaGhWnpC2LB6uhK4c/zxkocCCd5/oC e/6xsWOBUGOE90CubgjwvpYuA2I2tZv3Sy5Zrz+zCduX7j0BuclReU68VPVaxOsPdOuh +R5wk3dSn+5+xAy+3iFGLGkGhfTBw0ZoQVJPPIdAi60oUBKmlWltJrC62fx9CgwHPz6M D59VDhjBcie7Ut7+WdA57iZ1SsSDrONvHXGGYTPo9qBTh09kD6AI68kbSEN5uYj5GaNM HXsNg85brLB3CkaI9gBcGyco1YdJubhyzY3XjTbMQBcS6vi6Xnh/IA7FTvLLdlaJenUx jr/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ZG1hbdIv3FvUkXzGyiyJBV6DrXE2wtuUaoQN6LGlj1M=; b=W+q6YtRH2OFvdEDFYsuhQGQ1SyxjtWng7ex5XrMyk0fVXM2ysKU6LDu2tEzpDdR21o 52pdHV+OTvUFLjeeY0FU4uWYNOghvSDpIHFA+EOGwti581vOIc+9rNxmiGh0E44/TL8/ i9F4ILdsVchKWThTI3fdmRn+VU1DkycJp2+cRd5xIGX8XEQUEoI3Ph6EuqQ8yk+bu2Eg Qb9tstknjBq21k4/MHQeGdY4PLMbpcGQuqGoVbjtkFU3IuR50yoRPiW4Oc2M4iiwqQhE 75BBbFLK7o6lcZgNZguxxSx69JfnWDKFEzI+ii+hRNfwbhFxSGsfVsKh9xltYw6pyFUO JyZQ== X-Gm-Message-State: AOAM532ebZtAhET80CFZuYcBHMj08aGFWkOrv1zeAQICSuINsSChqbAJ Vg4DxJG19sGdMJxFqpLruZthXw== X-Google-Smtp-Source: ABdhPJzJ1Sj2T9cni69jm3ehIaxy4vedgsKsmt7wK9VAMhgxQOjax53s6snUh5lU5bh4C1ZyuNA3UA== X-Received: by 2002:a05:620a:1126:: with SMTP id p6mr4087741qkk.120.1622127949626; Thu, 27 May 2021 08:05:49 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id x10sm1447468qkh.124.2021.05.27.08.05.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 May 2021 08:05:49 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH 14/18] arm64: kexec: install a copy of the linear-map Date: Thu, 27 May 2021 11:05:22 -0400 Message-Id: <20210527150526.271941-15-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210527150526.271941-1-pasha.tatashin@soleen.com> References: <20210527150526.271941-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: DBAD3E000811 Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=soleen.com header.s=google header.b="PxjTnw1/"; spf=pass (imf05.hostedemail.com: domain of pasha.tatashin@soleen.com designates 209.85.222.171 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com; dmarc=none X-Rspamd-Server: rspam04 X-Stat-Signature: apkhkymun6rmbpytaq8gmksaz5onz8nh X-HE-Tag: 1622127939-315163 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: To perform the kexec relocations with the MMU enabled, we need a copy of the linear map. Create one, and install it from the relocation code. This has to be done from the assembly code as it will be idmapped with TTBR0. The kernel runs in TTRB1, so can't use the break-before-make sequence on the mapping it is executing from. The makes no difference yet as the relocation code runs with the MMU disabled. Co-developed-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/assembler.h | 19 +++++++++++++++++++ arch/arm64/include/asm/kexec.h | 2 ++ arch/arm64/kernel/asm-offsets.c | 2 ++ arch/arm64/kernel/hibernate-asm.S | 20 -------------------- arch/arm64/kernel/machine_kexec.c | 16 ++++++++++++++-- arch/arm64/kernel/relocate_kernel.S | 3 +++ 6 files changed, 40 insertions(+), 22 deletions(-) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index f1367863d995..7f268bfa98c6 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -447,6 +447,25 @@ USER(\label, ic ivau, \tmp2) // invalidate I line PoU isb .endm +/* + * To prevent the possibility of old and new partial table walks being visible + * in the tlb, switch the ttbr to a zero page when we invalidate the old + * records. D4.7.1 'General TLB maintenance requirements' in ARM DDI 0487A.i + * Even switching to our copied tables will cause a changed output address at + * each stage of the walk. + */ + .macro break_before_make_ttbr_switch zero_page, page_table, tmp, tmp2 + phys_to_ttbr \tmp, \zero_page + msr ttbr1_el1, \tmp + isb + tlbi vmalle1 + dsb nsh + phys_to_ttbr \tmp, \page_table + offset_ttbr1 \tmp, \tmp2 + msr ttbr1_el1, \tmp + isb + .endm + /* * reset_pmuserenr_el0 - reset PMUSERENR_EL0 if PMUv3 present */ diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h index 753a1c398898..d678f0ceb7ee 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -97,6 +97,8 @@ struct kimage_arch { phys_addr_t dtb_mem; phys_addr_t kern_reloc; phys_addr_t el2_vectors; + phys_addr_t ttbr1; + phys_addr_t zero_page; }; #ifdef CONFIG_KEXEC_FILE diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index ee37d0e0fb6d..f9794cd95eef 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -160,6 +160,8 @@ int main(void) #ifdef CONFIG_KEXEC_CORE DEFINE(KIMAGE_ARCH_DTB_MEM, offsetof(struct kimage, arch.dtb_mem)); DEFINE(KIMAGE_ARCH_EL2_VECTORS, offsetof(struct kimage, arch.el2_vectors)); + DEFINE(KIMAGE_ARCH_ZERO_PAGE, offsetof(struct kimage, arch.zero_page)); + DEFINE(KIMAGE_ARCH_TTBR1, offsetof(struct kimage, arch.ttbr1)); DEFINE(KIMAGE_HEAD, offsetof(struct kimage, head)); DEFINE(KIMAGE_START, offsetof(struct kimage, start)); BLANK(); diff --git a/arch/arm64/kernel/hibernate-asm.S b/arch/arm64/kernel/hibernate-asm.S index 8ccca660034e..a31e621ba867 100644 --- a/arch/arm64/kernel/hibernate-asm.S +++ b/arch/arm64/kernel/hibernate-asm.S @@ -15,26 +15,6 @@ #include #include -/* - * To prevent the possibility of old and new partial table walks being visible - * in the tlb, switch the ttbr to a zero page when we invalidate the old - * records. D4.7.1 'General TLB maintenance requirements' in ARM DDI 0487A.i - * Even switching to our copied tables will cause a changed output address at - * each stage of the walk. - */ -.macro break_before_make_ttbr_switch zero_page, page_table, tmp, tmp2 - phys_to_ttbr \tmp, \zero_page - msr ttbr1_el1, \tmp - isb - tlbi vmalle1 - dsb nsh - phys_to_ttbr \tmp, \page_table - offset_ttbr1 \tmp, \tmp2 - msr ttbr1_el1, \tmp - isb -.endm - - /* * Resume from hibernate * diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 675d27d7aea4..fe0f9c383c17 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -153,6 +153,8 @@ static void *kexec_page_alloc(void *arg) int machine_kexec_post_load(struct kimage *kimage) { + int rc; + pgd_t *trans_pgd; void *reloc_code = page_to_virt(kimage->control_code_page); long reloc_size; struct trans_pgd_info info = { @@ -169,12 +171,22 @@ int machine_kexec_post_load(struct kimage *kimage) kimage->arch.el2_vectors = 0; if (is_hyp_callable()) { - int rc = trans_pgd_copy_el2_vectors(&info, - &kimage->arch.el2_vectors); + rc = trans_pgd_copy_el2_vectors(&info, + &kimage->arch.el2_vectors); if (rc) return rc; } + /* Create a copy of the linear map */ + trans_pgd = kexec_page_alloc(kimage); + if (!trans_pgd) + return -ENOMEM; + rc = trans_pgd_create_copy(&info, &trans_pgd, PAGE_OFFSET, PAGE_END); + if (rc) + return rc; + kimage->arch.ttbr1 = __pa(trans_pgd); + kimage->arch.zero_page = __pa(empty_zero_page); + reloc_size = __relocate_new_kernel_end - __relocate_new_kernel_start; memcpy(reloc_code, __relocate_new_kernel_start, reloc_size); kimage->arch.kern_reloc = __pa(reloc_code); diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index 7a600ba33ae1..e83b6380907d 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -29,10 +29,13 @@ */ SYM_CODE_START(arm64_relocate_new_kernel) /* Setup the list loop variables. */ + ldr x18, [x0, #KIMAGE_ARCH_ZERO_PAGE] /* x18 = zero page for BBM */ + ldr x17, [x0, #KIMAGE_ARCH_TTBR1] /* x17 = linear map copy */ ldr x16, [x0, #KIMAGE_HEAD] /* x16 = kimage_head */ mov x14, xzr /* x14 = entry ptr */ mov x13, xzr /* x13 = copy dest */ raw_dcache_line_size x15, x1 /* x15 = dcache line size */ + break_before_make_ttbr_switch x18, x17, x1, x2 /* set linear map */ .Lloop: and x12, x16, PAGE_MASK /* x12 = addr */ From patchwork Thu May 27 15:05:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12284555 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F0A65C4708C for ; Thu, 27 May 2021 15:06:05 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 951836128D for ; Thu, 27 May 2021 15:06:05 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 951836128D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 109786B0087; Thu, 27 May 2021 11:05:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0E2116B0088; Thu, 27 May 2021 11:05:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E51568D0001; Thu, 27 May 2021 11:05:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0207.hostedemail.com [216.40.44.207]) by kanga.kvack.org (Postfix) with ESMTP id AABB06B0087 for ; Thu, 27 May 2021 11:05:53 -0400 (EDT) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 36BB31802DDE4 for ; Thu, 27 May 2021 15:05:53 +0000 (UTC) X-FDA: 78187335786.13.4A68C0C Received: from mail-qt1-f174.google.com (mail-qt1-f174.google.com [209.85.160.174]) by imf12.hostedemail.com (Postfix) with ESMTP id 19C702D1 for ; Thu, 27 May 2021 15:05:40 +0000 (UTC) Received: by mail-qt1-f174.google.com with SMTP id h21so425630qtu.5 for ; Thu, 27 May 2021 08:05:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=AR8EfhndcJhGCfPXw14djNNwxlc4BjhxbHGlWwV0KmE=; b=cuvs0nlIyCBteOSlUbqnu7MxHnc0HMUzOaoXHVjSn2UHqFo4Hy5ar0ZeGkFqsGIbHQ xzJZpkkKF+m2C8zs8onCRxd2q4U46s+Gemio+gH+DUn+AlPyIUKZJoucrJPBQr6E7DcG N99YmAOOFqWCguZ3CDzKq5SQ30+piiw8egDDjWac9+fEafPOpRSvi5Tfwuc+6inQfQS1 SoAg8A3QZOK5Z5In4lBz/74YYIBr8KYpwrtdca9iwVy4i20iE3b8154r5qP7NoPibzoO RdPHEt00M2lkgly6+lSsC5EKnhQt1OFj9ElsVZFY/pe/I6eHNh3/VAQxne50M6M/HfL3 ab9A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=AR8EfhndcJhGCfPXw14djNNwxlc4BjhxbHGlWwV0KmE=; b=A1FLAbSadhXINtSiq/8bNYWeilDg3vMwGSSBOSZ+n/Yaj4jtALtAegPZWT9TNrYbuY VjqbT4b6Q+bxGWQTZt7Nkge/KT7Fmbg1oxNgbcKiILYvlm/wyBFJqb4ljNEbp9Q1s6r2 752s1mbahfMcY/1y6c2cmUPC+4+y1NgY7W3tnhYHGg7c/Qv0XIQDHU/pTAZXNh/x5UmG eYDEaxBc/bXjMc11VSRoBgk189uS2Ji9RIdBlS98p9gTHh7K2/bERE4xhIPIRx7q3bln Wd06ssknKAKbcIPD4Q3CAfLF+n6cNH+WuMTVX6318rR8VM43cu6wMDrOD4Etlktp6QhT 2nnA== X-Gm-Message-State: AOAM531PYmDUF5OzjcfrzwPTPWC0MRuYlfNkAcIwMX4oFboeCU2FMN+O FH4meZzuBMYJehBgsGMc0rAL8A== X-Google-Smtp-Source: ABdhPJyXTJpi0i100Itoz1x1obakqkcvUNpa+7nZ/5NCoxyzGOYGAHt0BiM904+IpFQ0TYxSjtJbhg== X-Received: by 2002:ac8:6e8f:: with SMTP id c15mr3508556qtv.122.1622127951311; Thu, 27 May 2021 08:05:51 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id x10sm1447468qkh.124.2021.05.27.08.05.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 May 2021 08:05:50 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH 15/18] arm64: kexec: keep MMU enabled during kexec relocation Date: Thu, 27 May 2021 11:05:23 -0400 Message-Id: <20210527150526.271941-16-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210527150526.271941-1-pasha.tatashin@soleen.com> References: <20210527150526.271941-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 19C702D1 Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=soleen.com header.s=google header.b=cuvs0nlI; spf=pass (imf12.hostedemail.com: domain of pasha.tatashin@soleen.com designates 209.85.160.174 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com; dmarc=none X-Rspamd-Server: rspam04 X-Stat-Signature: m1q4b9ph547ux9k3sfx9jp3o4jj1soih X-HE-Tag: 1622127940-650262 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Now, that we have linear map page tables configured, keep MMU enabled to allow faster relocation of segments to final destination. Cavium ThunderX2: Kernel Image size: 38M Iniramfs size: 46M Total relocation size: 84M MMU-disabled: relocation 7.489539915s MMU-enabled: relocation 0.03946095s Broadcom Stingray: The performance data: for a moderate size kernel + initramfs: 25M the relocation was taking 0.382s, with enabled MMU it now takes 0.019s only or x20 improvement. The time is proportional to the size of relocation, therefore if initramfs is larger, 100M it could take over a second. Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/kexec.h | 3 +++ arch/arm64/kernel/asm-offsets.c | 1 + arch/arm64/kernel/machine_kexec.c | 16 +++++++++++---- arch/arm64/kernel/relocate_kernel.S | 31 +++++++++++++++++++---------- 4 files changed, 36 insertions(+), 15 deletions(-) diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h index d678f0ceb7ee..dca6dedc3b25 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -97,8 +97,11 @@ struct kimage_arch { phys_addr_t dtb_mem; phys_addr_t kern_reloc; phys_addr_t el2_vectors; + phys_addr_t ttbr0; phys_addr_t ttbr1; phys_addr_t zero_page; + unsigned long phys_offset; + unsigned long t0sz; }; #ifdef CONFIG_KEXEC_FILE diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index f9794cd95eef..bbdf20930491 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -161,6 +161,7 @@ int main(void) DEFINE(KIMAGE_ARCH_DTB_MEM, offsetof(struct kimage, arch.dtb_mem)); DEFINE(KIMAGE_ARCH_EL2_VECTORS, offsetof(struct kimage, arch.el2_vectors)); DEFINE(KIMAGE_ARCH_ZERO_PAGE, offsetof(struct kimage, arch.zero_page)); + DEFINE(KIMAGE_ARCH_PHYS_OFFSET, offsetof(struct kimage, arch.phys_offset)); DEFINE(KIMAGE_ARCH_TTBR1, offsetof(struct kimage, arch.ttbr1)); DEFINE(KIMAGE_HEAD, offsetof(struct kimage, head)); DEFINE(KIMAGE_START, offsetof(struct kimage, start)); diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index fe0f9c383c17..96a478d022b6 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -190,6 +190,11 @@ int machine_kexec_post_load(struct kimage *kimage) reloc_size = __relocate_new_kernel_end - __relocate_new_kernel_start; memcpy(reloc_code, __relocate_new_kernel_start, reloc_size); kimage->arch.kern_reloc = __pa(reloc_code); + rc = trans_pgd_idmap_page(&info, &kimage->arch.ttbr0, + &kimage->arch.t0sz, reloc_code); + if (rc) + return rc; + kimage->arch.phys_offset = virt_to_phys(kimage) - (long)kimage; /* Flush the reloc_code in preparation for its execution. */ __flush_dcache_area(reloc_code, reloc_size); @@ -223,9 +228,9 @@ void machine_kexec(struct kimage *kimage) local_daif_mask(); /* - * Both restart and cpu_soft_restart will shutdown the MMU, disable data + * Both restart and kernel_reloc will shutdown the MMU, disable data * caches. However, restart will start new kernel or purgatory directly, - * cpu_soft_restart will transfer control to arm64_relocate_new_kernel + * kernel_reloc contains the body of arm64_relocate_new_kernel * In kexec case, kimage->start points to purgatory assuming that * kernel entry and dtb address are embedded in purgatory by * userspace (kexec-tools). @@ -239,10 +244,13 @@ void machine_kexec(struct kimage *kimage) restart(is_hyp_callable(), kimage->start, kimage->arch.dtb_mem, 0, 0); } else { + void (*kernel_reloc)(struct kimage *kimage); + if (is_hyp_callable()) __hyp_set_vectors(kimage->arch.el2_vectors); - cpu_soft_restart(kimage->arch.kern_reloc, - virt_to_phys(kimage), 0, 0); + cpu_install_ttbr0(kimage->arch.ttbr0, kimage->arch.t0sz); + kernel_reloc = (void *)kimage->arch.kern_reloc; + kernel_reloc(kimage); } BUG(); /* Should never get here. */ diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index e83b6380907d..fc2348584a54 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -4,6 +4,8 @@ * * Copyright (C) Linaro. * Copyright (C) Huawei Futurewei Technologies. + * Copyright (C) 2020, Microsoft Corporation. + * Pavel Tatashin */ #include @@ -15,6 +17,13 @@ #include #include +.macro turn_off_mmu tmp1, tmp2 + mov_q \tmp1, INIT_SCTLR_EL1_MMU_OFF + pre_disable_mmu_workaround + msr sctlr_el1, \tmp1 + isb +.endm + .pushsection ".kexec_relocate.text", "ax" /* * arm64_relocate_new_kernel - Put a 2nd stage image in place and boot it. @@ -32,22 +41,21 @@ SYM_CODE_START(arm64_relocate_new_kernel) ldr x18, [x0, #KIMAGE_ARCH_ZERO_PAGE] /* x18 = zero page for BBM */ ldr x17, [x0, #KIMAGE_ARCH_TTBR1] /* x17 = linear map copy */ ldr x16, [x0, #KIMAGE_HEAD] /* x16 = kimage_head */ - mov x14, xzr /* x14 = entry ptr */ - mov x13, xzr /* x13 = copy dest */ + ldr x22, [x0, #KIMAGE_ARCH_PHYS_OFFSET] /* x22 phys_offset */ raw_dcache_line_size x15, x1 /* x15 = dcache line size */ break_before_make_ttbr_switch x18, x17, x1, x2 /* set linear map */ .Lloop: and x12, x16, PAGE_MASK /* x12 = addr */ - + sub x12, x12, x22 /* Convert x12 to virt */ /* Test the entry flags. */ .Ltest_source: tbz x16, IND_SOURCE_BIT, .Ltest_indirection /* Invalidate dest page to PoC. */ - mov x2, x13 - mov x1, #PAGE_SIZE - dcache_by_myline_op ivac, sy, x2, x1, x15, x20 + mov x19, x13 copy_page x13, x12, x1, x2, x3, x4, x5, x6, x7, x8 + mov x1, #PAGE_SIZE + dcache_by_myline_op civac, sy, x19, x1, x15, x20 b .Lnext .Ltest_indirection: tbz x16, IND_INDIRECTION_BIT, .Ltest_destination @@ -64,19 +72,20 @@ SYM_CODE_START(arm64_relocate_new_kernel) ic iallu dsb nsh isb + ldr x4, [x0, #KIMAGE_START] /* relocation start */ + ldr x1, [x0, #KIMAGE_ARCH_EL2_VECTORS] /* relocation start */ + ldr x0, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ + turn_off_mmu x12, x13 /* Start new image. */ - ldr x1, [x0, #KIMAGE_ARCH_EL2_VECTORS] /* relocation start */ cbz x1, .Lel1 - ldr x1, [x0, #KIMAGE_START] /* relocation start */ - ldr x2, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ + mov x1, x4 /* relocation start */ + mov x2, x0 /* dtb address */ mov x3, xzr mov x4, xzr mov x0, #HVC_SOFT_RESTART hvc #0 /* Jumps from el2 */ .Lel1: - ldr x4, [x0, #KIMAGE_START] /* relocation start */ - ldr x0, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ mov x2, xzr mov x3, xzr br x4 /* Jumps from el1 */ From patchwork Thu May 27 15:05:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12284557 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8DF7EC4708B for ; Thu, 27 May 2021 15:06:08 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 3612461005 for ; Thu, 27 May 2021 15:06:08 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3612461005 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0B9496B0088; Thu, 27 May 2021 11:05:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 090EF8D0001; Thu, 27 May 2021 11:05:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DFE356B008A; Thu, 27 May 2021 11:05:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0245.hostedemail.com [216.40.44.245]) by kanga.kvack.org (Postfix) with ESMTP id 9DEFB6B0088 for ; Thu, 27 May 2021 11:05:54 -0400 (EDT) Received: from smtpin36.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 403C9180388F0 for ; Thu, 27 May 2021 15:05:54 +0000 (UTC) X-FDA: 78187335828.36.45D9FE4 Received: from mail-qt1-f173.google.com (mail-qt1-f173.google.com [209.85.160.173]) by imf28.hostedemail.com (Postfix) with ESMTP id B41632000981 for ; Thu, 27 May 2021 15:05:45 +0000 (UTC) Received: by mail-qt1-f173.google.com with SMTP id t17so403167qta.11 for ; Thu, 27 May 2021 08:05:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=HZHOJzSFFbUuzx7eGvKvsp2IVzsfaWJrcIfDcXhfmjQ=; b=OE2Nv9rI1yl+eDr9xb/59iX9oopmd3JLdS6dWVrola8Od1OZixPXWsBxDTcSsG/BNj xqiP3/MtToHN8To0OOVfsTQ3FBtribEObmkQCfcGTb2XGrrt2TWIa9A+RUVytD4Z/F1Z kKpuTAqZN0nSiaH9NmXyCZVROaX3De/0QTwv6XiQhtVj2j7cm6+ofwNN8zS5yUCs8bE9 5f7KTwgzvzQxt+xXns0fOIqMOfXSXcBgLh+YZVajTxS7x88UNgl5ov6gRjoqo7qNpKeE aJtrlwWbPDLsQi9BZulXO6txvlllHE94kB0RXJGsAb9bZtu3H53chND0bq2jH+PtgScr gSdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HZHOJzSFFbUuzx7eGvKvsp2IVzsfaWJrcIfDcXhfmjQ=; b=GrMBfgnYwhBP37YXsM9lB8Ip/N/J0oBS1qNJXNaGOVFaSiM7s1FfddPhQyI4xmfv8N bu16/TIsG0iv7aU8UL7vTejvFL9zcqNiX3Ci8PPoxRLl08cnjpEe+QhScQG+b2phURka /XGBuS78CS7zNp7fx3vL1eRExMgCk+/kPZfH3xPfiSpiSsE0OKmW3T3f8FVIynKQnrw3 OfJFTYA/nOBgxJ/PdSfdVLCzuMJbms9UDyYGO/ePlnfJ+tK6wRbtrVFXbGH6kX083yz/ udzb4zg3JQ9xnWhF9ndHgicrrORuOVFFs42eQ9Ozfa/B5feCfMOlfVzeN95geahxmX9s DYTw== X-Gm-Message-State: AOAM5301WLU5nnCpwCQrg0KsHiBrE/UZ/0LRjxKUGL1OxK9m2chYlOD0 iPvZf6p4KDFBVPKLKrVFV6IZWA== X-Google-Smtp-Source: ABdhPJwQ7PTEJ4bfTYMlyGqHPr1GFZ98OMMmMZTDk2VsdokJOvRo2GHvow0yZur+w4JE/untrcRTcA== X-Received: by 2002:ac8:66da:: with SMTP id m26mr3565077qtp.102.1622127952711; Thu, 27 May 2021 08:05:52 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id x10sm1447468qkh.124.2021.05.27.08.05.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 May 2021 08:05:52 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH 16/18] arm64: kexec: remove the pre-kexec PoC maintenance Date: Thu, 27 May 2021 11:05:24 -0400 Message-Id: <20210527150526.271941-17-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210527150526.271941-1-pasha.tatashin@soleen.com> References: <20210527150526.271941-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: B41632000981 Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=soleen.com header.s=google header.b=OE2Nv9rI; dmarc=none; spf=pass (imf28.hostedemail.com: domain of pasha.tatashin@soleen.com designates 209.85.160.173 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com X-Rspamd-Server: rspam03 X-Stat-Signature: to5uoru66ttcdc3gehwqecuqobqfjtf8 X-HE-Tag: 1622127945-732182 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Now that kexec does its relocations with the MMU enabled, we no longer need to clean the relocation data to the PoC. Co-developed-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/machine_kexec.c | 40 ------------------------------- 1 file changed, 40 deletions(-) diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 96a478d022b6..afbde4bb1829 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -76,45 +76,6 @@ int machine_kexec_prepare(struct kimage *kimage) return 0; } -/** - * kexec_list_flush - Helper to flush the kimage list and source pages to PoC. - */ -static void kexec_list_flush(struct kimage *kimage) -{ - kimage_entry_t *entry; - - __flush_dcache_area(kimage, sizeof(*kimage)); - - for (entry = &kimage->head; ; entry++) { - unsigned int flag; - void *addr; - - /* flush the list entries. */ - __flush_dcache_area(entry, sizeof(kimage_entry_t)); - - flag = *entry & IND_FLAGS; - if (flag == IND_DONE) - break; - - addr = phys_to_virt(*entry & PAGE_MASK); - - switch (flag) { - case IND_INDIRECTION: - /* Set entry point just before the new list page. */ - entry = (kimage_entry_t *)addr - 1; - break; - case IND_SOURCE: - /* flush the source pages. */ - __flush_dcache_area(addr, PAGE_SIZE); - break; - case IND_DESTINATION: - break; - default: - BUG(); - } - } -} - /** * kexec_segment_flush - Helper to flush the kimage segments to PoC. */ @@ -200,7 +161,6 @@ int machine_kexec_post_load(struct kimage *kimage) __flush_dcache_area(reloc_code, reloc_size); flush_icache_range((uintptr_t)reloc_code, (uintptr_t)reloc_code + reloc_size); - kexec_list_flush(kimage); kexec_image_info(kimage); return 0; From patchwork Thu May 27 15:05:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12284559 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5EAEFC4707F for ; Thu, 27 May 2021 15:06:11 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0D9976128D for ; Thu, 27 May 2021 15:06:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0D9976128D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 89C616B0089; Thu, 27 May 2021 11:05:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 81AE46B008A; Thu, 27 May 2021 11:05:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0A2636B008C; Thu, 27 May 2021 11:05:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0178.hostedemail.com [216.40.44.178]) by kanga.kvack.org (Postfix) with ESMTP id A915D6B0089 for ; Thu, 27 May 2021 11:05:57 -0400 (EDT) Received: from smtpin14.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 4443D181AEF1F for ; Thu, 27 May 2021 15:05:57 +0000 (UTC) X-FDA: 78187335954.14.9AF9F73 Received: from mail-qt1-f179.google.com (mail-qt1-f179.google.com [209.85.160.179]) by imf19.hostedemail.com (Postfix) with ESMTP id 775AD90009EB for ; Thu, 27 May 2021 15:05:46 +0000 (UTC) Received: by mail-qt1-f179.google.com with SMTP id t20so415764qtx.8 for ; Thu, 27 May 2021 08:05:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=2lq6+MakGPQuirNtQQTDFwD5XrRoYT+d3t/Oh58mP4Q=; b=U1ehG/zeOWYwlR4hoxP5YnaQEmhc2GgjVwIbKmIULFiONY3PaoltZVzgCzAjQs6Cse rHgAqKTLHeMqW/YgTshpwJSBqtCRvtJiKN0ClUGwPUGhbkCYVJeGepTiTy/8wkMlvlb5 O6FMOwriPuaVqJqkKKAdoH5VSffTa2u3TOMeLapsQiGhEsbWdbrLh+ouQplarW+mf/5B gwKH5PslhIVaCy6gjfF3xuACoxFc3zACPGM4DjE44jMUaun/550UV6jajDZvbOkwpxWC Bn+T9v45T5vbIL39tsfl//beVDi5b2Jx/5wSUfXuved3P8Fn99uz79nTcDWD/wj8e3d6 WAxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2lq6+MakGPQuirNtQQTDFwD5XrRoYT+d3t/Oh58mP4Q=; b=NwFJYkU7rgPBLoHDm+TWDGSHMrggy6smr17boyvyl1/eL9RN9TD+6PuxDmQh0aGskD VcxuNLrRra05ZbL6kOAruHOHosw4b7YIX6lduwUsCT9PL/KY0TTjeE6fS1V/ICTi1KHU /K7jDllBZ9jTtrG/5rneRP+qhJ13PbP6SAKvBF4yVKshPeAq+RWsNVtPEtaIoXDCa31Z BmCSe7FqVmKKEoTyWWCnpmcvCY9Nsf0b78JgK8poJv+AGttZTr+lhVz49sLrfxxZRVCm CpiFkDuH/wvfefuS2AoF9+byxAjmzrUt60egEctTSQp3WRVz/AEtmqyrOUrNOSpEyF/e Gxrw== X-Gm-Message-State: AOAM532EPXp7D95jytf4POirtcU5+oemEym2a6SnFIKdVg4r3TIYhIr+ no9zprqj11SmgtwV4EVS3ovnlg== X-Google-Smtp-Source: ABdhPJw0pPua175hZLYTT6CqVpGguaKlKVb56ZVRDoJK9z9ajvuvB1dR7EF+lT603pTxYBhMr1txXQ== X-Received: by 2002:ac8:5e0e:: with SMTP id h14mr3514930qtx.117.1622127954155; Thu, 27 May 2021 08:05:54 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id x10sm1447468qkh.124.2021.05.27.08.05.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 May 2021 08:05:53 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH 17/18] arm64: kexec: Remove cpu-reset.h Date: Thu, 27 May 2021 11:05:25 -0400 Message-Id: <20210527150526.271941-18-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210527150526.271941-1-pasha.tatashin@soleen.com> References: <20210527150526.271941-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 775AD90009EB Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=soleen.com header.s=google header.b="U1ehG/ze"; spf=pass (imf19.hostedemail.com: domain of pasha.tatashin@soleen.com designates 209.85.160.179 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com; dmarc=none X-Rspamd-Server: rspam04 X-Stat-Signature: oe31zoakqho3f1d6j1d7hf6pt9kqxiyd X-HE-Tag: 1622127946-799760 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This header contains only cpu_soft_restart() which is never used directly anymore. So, remove this header, and rename the helper to be cpu_soft_restart(). Suggested-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/kexec.h | 6 ++++++ arch/arm64/kernel/cpu-reset.S | 7 +++---- arch/arm64/kernel/cpu-reset.h | 30 ------------------------------ arch/arm64/kernel/machine_kexec.c | 6 ++---- 4 files changed, 11 insertions(+), 38 deletions(-) delete mode 100644 arch/arm64/kernel/cpu-reset.h diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h index dca6dedc3b25..9839bfc163d7 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -90,6 +90,12 @@ static inline void crash_prepare_suspend(void) {} static inline void crash_post_resume(void) {} #endif +#if defined(CONFIG_KEXEC_CORE) +void cpu_soft_restart(unsigned long el2_switch, unsigned long entry, + unsigned long arg0, unsigned long arg1, + unsigned long arg2); +#endif + #define ARCH_HAS_KIMAGE_ARCH struct kimage_arch { diff --git a/arch/arm64/kernel/cpu-reset.S b/arch/arm64/kernel/cpu-reset.S index d47ff63a5b66..48a8af97faa9 100644 --- a/arch/arm64/kernel/cpu-reset.S +++ b/arch/arm64/kernel/cpu-reset.S @@ -16,8 +16,7 @@ .pushsection .idmap.text, "awx" /* - * __cpu_soft_restart(el2_switch, entry, arg0, arg1, arg2) - Helper for - * cpu_soft_restart. + * cpu_soft_restart(el2_switch, entry, arg0, arg1, arg2) * * @el2_switch: Flag to indicate a switch to EL2 is needed. * @entry: Location to jump to for soft reset. @@ -29,7 +28,7 @@ * branch to what would be the reset vector. It must be executed with the * flat identity mapping. */ -SYM_CODE_START(__cpu_soft_restart) +SYM_CODE_START(cpu_soft_restart) mov_q x12, INIT_SCTLR_EL1_MMU_OFF pre_disable_mmu_workaround /* @@ -48,6 +47,6 @@ SYM_CODE_START(__cpu_soft_restart) mov x1, x3 // arg1 mov x2, x4 // arg2 br x8 -SYM_CODE_END(__cpu_soft_restart) +SYM_CODE_END(cpu_soft_restart) .popsection diff --git a/arch/arm64/kernel/cpu-reset.h b/arch/arm64/kernel/cpu-reset.h deleted file mode 100644 index 296abbac7192..000000000000 --- a/arch/arm64/kernel/cpu-reset.h +++ /dev/null @@ -1,30 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0-only */ -/* - * CPU reset routines - * - * Copyright (C) 2015 Huawei Futurewei Technologies. - */ - -#ifndef _ARM64_CPU_RESET_H -#define _ARM64_CPU_RESET_H - -#include - -void __cpu_soft_restart(unsigned long el2_switch, unsigned long entry, - unsigned long arg0, unsigned long arg1, unsigned long arg2); - -static inline void __noreturn __nocfi cpu_soft_restart(unsigned long entry, - unsigned long arg0, - unsigned long arg1, - unsigned long arg2) -{ - typeof(__cpu_soft_restart) *restart; - - restart = (void *)__pa_symbol(function_nocfi(__cpu_soft_restart)); - - cpu_install_idmap(); - restart(0, entry, arg0, arg1, arg2); - unreachable(); -} - -#endif diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index afbde4bb1829..16996b568fd3 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -23,8 +23,6 @@ #include #include -#include "cpu-reset.h" - /** * kexec_image_info - For debugging output. */ @@ -197,10 +195,10 @@ void machine_kexec(struct kimage *kimage) * In kexec_file case, the kernel starts directly without purgatory. */ if (kimage->head & IND_DONE) { - typeof(__cpu_soft_restart) *restart; + typeof(cpu_soft_restart) *restart; cpu_install_idmap(); - restart = (void *)__pa_symbol(function_nocfi(__cpu_soft_restart)); + restart = (void *)__pa_symbol(function_nocfi(cpu_soft_restart)); restart(is_hyp_callable(), kimage->start, kimage->arch.dtb_mem, 0, 0); } else { From patchwork Thu May 27 15:05:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12284561 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A2D0DC4708A for ; Thu, 27 May 2021 15:06:14 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4E63D61005 for ; Thu, 27 May 2021 15:06:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4E63D61005 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 66BEF6B008A; Thu, 27 May 2021 11:05:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5D4676B0092; Thu, 27 May 2021 11:05:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DA7378D0001; Thu, 27 May 2021 11:05:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 83CCD6B0092 for ; Thu, 27 May 2021 11:05:58 -0400 (EDT) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 1B7F799BF for ; Thu, 27 May 2021 15:05:58 +0000 (UTC) X-FDA: 78187335996.05.51F1802 Received: from mail-qv1-f46.google.com (mail-qv1-f46.google.com [209.85.219.46]) by imf10.hostedemail.com (Postfix) with ESMTP id 73DE740F8C2E for ; Thu, 27 May 2021 15:05:49 +0000 (UTC) Received: by mail-qv1-f46.google.com with SMTP id a7so229616qvf.11 for ; Thu, 27 May 2021 08:05:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=KXQ+yohNMyyqRPEeNONrEzPBCefWBOBuVaWouKl2Q04=; b=Iq2Q5NY3yposm0EzkEbNN3/+GTxojDq2PR6h8IMwJ6xZAn+A1cRGovJwR2aPkV8hn7 w+eFIRjxUsghtCzfIod0bzTDP+pcAgOSje5vB8U3q+aXKe+waXT2d6Ggk1uAUzfsA4gf +QgHKhRNtnebjuIuJILNouevRpewJ1PPo8mfAdzpLMPyW9f25kw4YpwOoRwxkKmbdVa8 OfRtGvLr0zlSKmJy2Vzk7F0qHuaZXgJecrZvHqqRzNFweTj/J9gd2alfHSMSVk+7BIop 627eTfXCJVf+/mfbfdy20rrkK0z6CFDL4pu/MrBdiV/h6BwYsj88TvOrAtjk36oXx7Dh 13cg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KXQ+yohNMyyqRPEeNONrEzPBCefWBOBuVaWouKl2Q04=; b=NEzYVvcyDSPEfBVHCzsWnRUvBNwYu+oX99Yc9VYGtMNGro+c6louvIzEX9Ye5M12rR XYX8tizB5SdWtHRqjh3G+bIKcGoGF3dG+Y0MBeFJmCNSOOazilyOtZCXoI3CxSPnfyss rAmrfZngibPN1SIzJLNSQc6wBpl/mKXERiX/OdG1tWV0fYuNW3wwt48n0zMR6ybzotad mfK6U8VMfNWOSktdUFnQ2xKvWMTf5NMRLd8SwvZk+KUvuTWAQd9Jn0m5Y5O4nF6/68fv jfjWHwPtwjvMMwM9xVrhhj5mVrMa9ebylkl/gKtC9bQ+AT3k0Nr1r1CiGsXq8udfTAm6 iWIA== X-Gm-Message-State: AOAM531c94p87XTqxdaV5OkMU7Xnzb8A8eOCd+zplU14omNty96xZsJ2 ss7wKRLOAjRnn8wTn8r5Wi+SVQ== X-Google-Smtp-Source: ABdhPJzsC2TgTF9Gmqd9ulqOby16oaHRzRYnnCr7Rt0hFZaf2VvH052e/3SGjlSguYtwryEqV/XBTg== X-Received: by 2002:a0c:9e24:: with SMTP id p36mr4019323qve.60.1622127956289; Thu, 27 May 2021 08:05:56 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id x10sm1447468qkh.124.2021.05.27.08.05.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 May 2021 08:05:55 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH 18/18] arm64/mm: remove useless trans_pgd_map_page() Date: Thu, 27 May 2021 11:05:26 -0400 Message-Id: <20210527150526.271941-19-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210527150526.271941-1-pasha.tatashin@soleen.com> References: <20210527150526.271941-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 73DE740F8C2E Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=soleen.com header.s=google header.b=Iq2Q5NY3; spf=pass (imf10.hostedemail.com: domain of pasha.tatashin@soleen.com designates 209.85.219.46 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com; dmarc=none X-Rspamd-Server: rspam04 X-Stat-Signature: 3w8unqu59io1pwci9cbx5194t8e5cfbo X-HE-Tag: 1622127949-624572 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Pingfan Liu The intend of trans_pgd_map_page() was to map contigous range of VA memory to the memory that is getting relocated during kexec. However, since we are now using linear map instead of contigous range this function is not needed Signed-off-by: Pingfan Liu [Changed commit message] Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/trans_pgd.h | 5 +-- arch/arm64/mm/trans_pgd.c | 57 ------------------------------ 2 files changed, 1 insertion(+), 61 deletions(-) diff --git a/arch/arm64/include/asm/trans_pgd.h b/arch/arm64/include/asm/trans_pgd.h index e0760e52d36d..234353df2f13 100644 --- a/arch/arm64/include/asm/trans_pgd.h +++ b/arch/arm64/include/asm/trans_pgd.h @@ -15,7 +15,7 @@ /* * trans_alloc_page * - Allocator that should return exactly one zeroed page, if this - * allocator fails, trans_pgd_create_copy() and trans_pgd_map_page() + * allocator fails, trans_pgd_create_copy() and trans_pgd_idmap_page() * return -ENOMEM error. * * trans_alloc_arg @@ -30,9 +30,6 @@ struct trans_pgd_info { int trans_pgd_create_copy(struct trans_pgd_info *info, pgd_t **trans_pgd, unsigned long start, unsigned long end); -int trans_pgd_map_page(struct trans_pgd_info *info, pgd_t *trans_pgd, - void *page, unsigned long dst_addr, pgprot_t pgprot); - int trans_pgd_idmap_page(struct trans_pgd_info *info, phys_addr_t *trans_ttbr0, unsigned long *t0sz, void *page); diff --git a/arch/arm64/mm/trans_pgd.c b/arch/arm64/mm/trans_pgd.c index 61549451ed3a..e24a749013c1 100644 --- a/arch/arm64/mm/trans_pgd.c +++ b/arch/arm64/mm/trans_pgd.c @@ -217,63 +217,6 @@ int trans_pgd_create_copy(struct trans_pgd_info *info, pgd_t **dst_pgdp, return rc; } -/* - * Add map entry to trans_pgd for a base-size page at PTE level. - * info: contains allocator and its argument - * trans_pgd: page table in which new map is added. - * page: page to be mapped. - * dst_addr: new VA address for the page - * pgprot: protection for the page. - * - * Returns 0 on success, and -ENOMEM on failure. - */ -int trans_pgd_map_page(struct trans_pgd_info *info, pgd_t *trans_pgd, - void *page, unsigned long dst_addr, pgprot_t pgprot) -{ - pgd_t *pgdp; - p4d_t *p4dp; - pud_t *pudp; - pmd_t *pmdp; - pte_t *ptep; - - pgdp = pgd_offset_pgd(trans_pgd, dst_addr); - if (pgd_none(READ_ONCE(*pgdp))) { - p4dp = trans_alloc(info); - if (!pgdp) - return -ENOMEM; - pgd_populate(NULL, pgdp, p4dp); - } - - p4dp = p4d_offset(pgdp, dst_addr); - if (p4d_none(READ_ONCE(*p4dp))) { - pudp = trans_alloc(info); - if (!pudp) - return -ENOMEM; - p4d_populate(NULL, p4dp, pudp); - } - - pudp = pud_offset(p4dp, dst_addr); - if (pud_none(READ_ONCE(*pudp))) { - pmdp = trans_alloc(info); - if (!pmdp) - return -ENOMEM; - pud_populate(NULL, pudp, pmdp); - } - - pmdp = pmd_offset(pudp, dst_addr); - if (pmd_none(READ_ONCE(*pmdp))) { - ptep = trans_alloc(info); - if (!ptep) - return -ENOMEM; - pmd_populate_kernel(NULL, pmdp, ptep); - } - - ptep = pte_offset_kernel(pmdp, dst_addr); - set_pte(ptep, pfn_pte(virt_to_pfn(page), pgprot)); - - return 0; -} - /* * The page we want to idmap may be outside the range covered by VA_BITS that * can be built using the kernel's p?d_populate() helpers. As a one off, for a