From patchwork Mon Aug 2 21:53:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12414855 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 80975C432BE for ; Mon, 2 Aug 2021 21:56:17 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 44C4260187 for ; Mon, 2 Aug 2021 21:56:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 44C4260187 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=2MrSrnG8d2a7lCZFf3JPsoC+fkdR6t+gCSBwuGjlkLQ=; b=Z/Z7JEJDQ4NOxZ 22UAl9jZz4Gb+a7DpK6CGhgLBXxI/xlnGM8ppmgv0WSF6FAV9RWRv4fpZWJJwQLAGnzD/HagW+KB4 spUu59AVg1S6emsJjFd/+uEu/DG74A67kg/AIGV/r2pVb/2kaeghcTFJqWMhsJv1dYyCXpfrnmgz8 dkUtjmG35GohgFCc56chAlaMtR1Cn1se5A/ei1plyb43CYFGpWevmzoKg9Mc3CQZfTqsDBKpk0ZYD 3v7UwSVVs1QpEh8rTekySPyJn4GC3ZWBstvP7riY+2O+ZllBjUrwW2725xOQECqY3izoJXBIFdBTz YmQX3sJHCIqvt855S6oA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mAft3-000Jls-8J; Mon, 02 Aug 2021 21:54:17 +0000 Received: from mail-qk1-x731.google.com ([2607:f8b0:4864:20::731]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mAfsz-000JjP-SL for linux-arm-kernel@lists.infradead.org; Mon, 02 Aug 2021 21:54:15 +0000 Received: by mail-qk1-x731.google.com with SMTP id 129so18126610qkg.4 for ; Mon, 02 Aug 2021 14:54:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=gpKUHaMAwkHB+q+UElCl69jZKnxiURz6Ucy0xkR2ELY=; b=mmGBEfnF9oZBlRuZblnuO8IgGciEd2V3OTlp4jdC//Y9f0WVpenL/OBOeVgveoRMwK wGT90RC7SKYswp9xlxMNjNoEZxUXiYv1oApAYHpcxpWe8tf6N4lQTVgE2Rl8MF1Jhwak IwElBw5j6ZSZWrGaDF64dNenVk8adI9vzpmEdI1vCNgmblGrXwAif7ZQ8wWR87WigxCn MN3qDKfl2RADG4h1zeg2WiGTzf7ZyIXlFU6DWq3NQO/zuQkkRfp581JDmFTlPmXrnmFP zxFEB75mCaToQgA6HJeeqdFJTs3x8N1jTVfZxqyMvfo/U22dsK/mdxKjhxQpuWjj5f6V Sjvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gpKUHaMAwkHB+q+UElCl69jZKnxiURz6Ucy0xkR2ELY=; b=hqS0pXEEU+vOFzEKfQBLklzmp+0dhtDvIJuHwWlTRo0ahBWo+fjw8RgQ6pw8VEruoB IVQZGBRCzoMhccN7simWcCmkdwCXGllep5nhfwqkAyXCRq2Q9KCTpgxPbR6oAZK5jWOc l4KRtMwJnlN8Qyo85lWrvaEzc1IlTnmVX4h03zDhfF5CMCsh7FYZQpgg57vfKwK8OIoG K2nN4C7hyJQCNPP0Z6q7So/1Q7iaOSQcN4nfhRN59kwP3yu4IdbFrMdBX/lY9tNvr3Nu TVzI1YhX7ZAcasrjuxPksRaVpGm02gX+EEzqu2EBEYGRzw1E16jnoJet1FXPXl7rX/k2 eYsQ== X-Gm-Message-State: AOAM532hL7g6m9XNlwyZooH35v0sIMdPPAJjG01quWnRR0xP8w2xUeLH 7DSutsynJobqPNEdi3XLvwJ9Ww== X-Google-Smtp-Source: ABdhPJxUfhb3Dku9KW46dPB118JL9YX+skltVkhp7xJNLax/VRpKXlKrybwlyB5FLGyYAGhSLrA6sQ== X-Received: by 2002:a37:a058:: with SMTP id j85mr12147273qke.471.1627941252530; Mon, 02 Aug 2021 14:54:12 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id v11sm5479216qtc.0.2021.08.02.14.54.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Aug 2021 14:54:12 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH v16 01/15] arm64: kernel: add helper for booted at EL2 and not VHE Date: Mon, 2 Aug 2021 17:53:54 -0400 Message-Id: <20210802215408.804942-2-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210802215408.804942-1-pasha.tatashin@soleen.com> References: <20210802215408.804942-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210802_145413_969292_B1154E19 X-CRM114-Status: GOOD ( 15.85 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Replace places that contain logic like this: is_hyp_mode_available() && !is_kernel_in_hyp_mode() With a dedicated boolean function is_hyp_nvhe(). This will be needed later in kexec in order to sooner switch back to EL2. Suggested-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/virt.h | 5 +++++ arch/arm64/kernel/cpu-reset.h | 3 +-- arch/arm64/kernel/hibernate.c | 2 +- arch/arm64/kernel/sdei.c | 2 +- 4 files changed, 8 insertions(+), 4 deletions(-) diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h index 7379f35ae2c6..a9457e96203c 100644 --- a/arch/arm64/include/asm/virt.h +++ b/arch/arm64/include/asm/virt.h @@ -128,6 +128,11 @@ static __always_inline bool is_protected_kvm_enabled(void) return cpus_have_final_cap(ARM64_KVM_PROTECTED_MODE); } +static inline bool is_hyp_nvhe(void) +{ + return is_hyp_mode_available() && !is_kernel_in_hyp_mode(); +} + #endif /* __ASSEMBLY__ */ #endif /* ! __ASM__VIRT_H */ diff --git a/arch/arm64/kernel/cpu-reset.h b/arch/arm64/kernel/cpu-reset.h index 9a7b1262ef17..81b3d0fe7a63 100644 --- a/arch/arm64/kernel/cpu-reset.h +++ b/arch/arm64/kernel/cpu-reset.h @@ -20,8 +20,7 @@ static inline void __noreturn __nocfi cpu_soft_restart(unsigned long entry, { typeof(__cpu_soft_restart) *restart; - unsigned long el2_switch = !is_kernel_in_hyp_mode() && - is_hyp_mode_available(); + unsigned long el2_switch = is_hyp_nvhe(); restart = (void *)__pa_symbol(function_nocfi(__cpu_soft_restart)); cpu_install_idmap(); diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index 46a0b4d6e251..4c9533f4c0c4 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -49,7 +49,7 @@ extern int in_suspend; /* Do we need to reset el2? */ -#define el2_reset_needed() (is_hyp_mode_available() && !is_kernel_in_hyp_mode()) +#define el2_reset_needed() (is_hyp_nvhe()) /* temporary el2 vectors in the __hibernate_exit_text section. */ extern char hibernate_el2_vectors[]; diff --git a/arch/arm64/kernel/sdei.c b/arch/arm64/kernel/sdei.c index 47f77d1234cb..d20620a1c51a 100644 --- a/arch/arm64/kernel/sdei.c +++ b/arch/arm64/kernel/sdei.c @@ -202,7 +202,7 @@ unsigned long sdei_arch_get_entry_point(int conduit) * dropped to EL1 because we don't support VHE, then we can't support * SDEI. */ - if (is_hyp_mode_available() && !is_kernel_in_hyp_mode()) { + if (is_hyp_nvhe()) { pr_err("Not supported on this hardware/boot configuration\n"); goto out_err; } From patchwork Mon Aug 2 21:53:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12414859 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1A764C4338F for ; Mon, 2 Aug 2021 21:56:43 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D47916023E for ; Mon, 2 Aug 2021 21:56:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org D47916023E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=3JPLbQmdxng9nTFYyYkGbRt898cGGnOfwQcsTCuCBzc=; b=J8lpORPyge3mH3 +37t/XjN5J2CmHlTKbm4dG9wJuqH0Jz8FhC4DthfVFwkgw7r0ZiOStkCyt1mCO1DwxpCR2Yt7idEf UcCv2s0SQ84YJA9VynownarkikONVErtrdCz+Pt/bNulNDetWwNvnk8d3iFDUt+mAyS8tf8UwTwmI 4ILZJrspMutCTY9vHX72MhDe9+yJYuZF49q9G37z1xkpLfKqtklcVgkKtJy4+FzHxgZZDiD7a/SuV gAhwWrK8Nf5SjsmroimqEunn/CDTN/DMoC8vz74dztsmFIJ8RIs8zWUhSER6EzMRRRWpo/xlEmiX+ KQctBac5GGz3Zv6tsdqg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mAftl-000K61-0F; Mon, 02 Aug 2021 21:55:01 +0000 Received: from mail-qv1-xf2b.google.com ([2607:f8b0:4864:20::f2b]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mAft1-000Jkb-Br for linux-arm-kernel@lists.infradead.org; Mon, 02 Aug 2021 21:54:19 +0000 Received: by mail-qv1-xf2b.google.com with SMTP id g6so9628484qvj.8 for ; Mon, 02 Aug 2021 14:54:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=YKdfgXuhu2CUCgXa9ObvPNn0CQc7wFt5Zs9uSkuVOsA=; b=gfWVCyeKZPOId7LZeYo8c368E3yisYg9Xgca9a9MXwLGcV4UeBg2K3NyHRFCulISgX mzOOWsVW5Cu3kVe8g+ivsmp/XkslZNVQiEPBnpeJrtTUaP5EnwQOfPgvitPaJb9A38ox rS9o9gA1l9Nv2Hzyj21wCeqyRVfajIISqCfIWnboqiNYT0W/W+YWZAUAoHqc7ZIAwG1W TDKavvZTfY5W5SCP17yVqwJ1qO7EYdOivc3V8+arrtVLtLfp8VqqFtJBt4HXbcoos7z8 zZO8Ap9KwvLY0TSXIiUTdFBEri21hxWZVyZ1r0cbpMDWJAQ8MLln//7YMR1slBDSg3ec 13OQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=YKdfgXuhu2CUCgXa9ObvPNn0CQc7wFt5Zs9uSkuVOsA=; b=IYB6zh35ZQy2WjIxESEiU9bMcp9Jqg4R9ncqO4urJdOqlyj5+xf/stHZc98MLECwsR gBsIP/JneaEJ41slM5eH5SCnd5wq6VZWzyHGKkoBI7/9V4vo410hqqsayt+rx+JkeGw0 Nqdb3s+DjxKd3oFXXDZnYj/sjzVZ5ZHwLqc770jWGIx5tWxp+fUZjrBAoqV9l/ehdktV 2btIwopUtzV9S34p2M9V8/ZcvdoLVxSzWwVea8bekpcVTVAObqqmzgiUr2C4DD/RWr2i 8p4mADdvH1nHyxME6xe79p4hc/rcox+N5SpdPARq2zfKRIf4T07jLlxTS5DPc8LjZWyP J0qw== X-Gm-Message-State: AOAM532mNHzdkCEU1q+4dStbLj7ENB5YU9xdSr7wlrpDTOjmZKtePKkV rivFA8RziEOFKcOIEUoHJDmNTg== X-Google-Smtp-Source: ABdhPJye8pVKXRr53NzwS56TsDSey/dZjhEjDvIMOivgptTWN5vFfw2CKKx6GPR5mqVzfSe590ht6Q== X-Received: by 2002:a0c:be8e:: with SMTP id n14mr18289595qvi.16.1627941253896; Mon, 02 Aug 2021 14:54:13 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id v11sm5479216qtc.0.2021.08.02.14.54.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Aug 2021 14:54:13 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH v16 02/15] arm64: trans_pgd: hibernate: Add trans_pgd_copy_el2_vectors Date: Mon, 2 Aug 2021 17:53:55 -0400 Message-Id: <20210802215408.804942-3-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210802215408.804942-1-pasha.tatashin@soleen.com> References: <20210802215408.804942-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210802_145415_533189_EAA3277D X-CRM114-Status: GOOD ( 24.56 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Users of trans_pgd may also need a copy of vector table because it is also may be overwritten if a linear map can be overwritten. Move setup of EL2 vectors from hibernate to trans_pgd, so it can be later shared with kexec as well. Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/trans_pgd.h | 7 +++- arch/arm64/include/asm/virt.h | 2 ++ arch/arm64/kernel/hibernate-asm.S | 52 --------------------------- arch/arm64/kernel/hibernate.c | 26 ++++++-------- arch/arm64/mm/Makefile | 1 + arch/arm64/mm/trans_pgd-asm.S | 58 ++++++++++++++++++++++++++++++ arch/arm64/mm/trans_pgd.c | 25 ++++++++++++- 7 files changed, 101 insertions(+), 70 deletions(-) create mode 100644 arch/arm64/mm/trans_pgd-asm.S diff --git a/arch/arm64/include/asm/trans_pgd.h b/arch/arm64/include/asm/trans_pgd.h index 5d08e5adf3d5..411323bda7a4 100644 --- a/arch/arm64/include/asm/trans_pgd.h +++ b/arch/arm64/include/asm/trans_pgd.h @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: GPL-2.0 */ /* - * Copyright (c) 2020, Microsoft Corporation. + * Copyright (c) 2021, Microsoft Corporation. * Pavel Tatashin */ @@ -36,4 +36,9 @@ int trans_pgd_map_page(struct trans_pgd_info *info, pgd_t *trans_pgd, int trans_pgd_idmap_page(struct trans_pgd_info *info, phys_addr_t *trans_ttbr0, unsigned long *t0sz, void *page); +int trans_pgd_copy_el2_vectors(struct trans_pgd_info *info, + phys_addr_t *el2_vectors); + +extern char trans_pgd_stub_vectors[]; + #endif /* _ASM_TRANS_TABLE_H */ diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h index a9457e96203c..3c8af033a997 100644 --- a/arch/arm64/include/asm/virt.h +++ b/arch/arm64/include/asm/virt.h @@ -67,6 +67,8 @@ */ extern u32 __boot_cpu_mode[2]; +#define ARM64_VECTOR_TABLE_LEN SZ_2K + void __hyp_set_vectors(phys_addr_t phys_vector_base); void __hyp_reset_vectors(void); diff --git a/arch/arm64/kernel/hibernate-asm.S b/arch/arm64/kernel/hibernate-asm.S index 81c0186a5e32..a30a2c3f905e 100644 --- a/arch/arm64/kernel/hibernate-asm.S +++ b/arch/arm64/kernel/hibernate-asm.S @@ -112,56 +112,4 @@ alternative_insn "dc cvau, x4", "dc civac, x4", ARM64_WORKAROUND_CLEAN_CACHE hvc #0 3: ret SYM_CODE_END(swsusp_arch_suspend_exit) - -/* - * Restore the hyp stub. - * This must be done before the hibernate page is unmapped by _cpu_resume(), - * but happens before any of the hyp-stub's code is cleaned to PoC. - * - * x24: The physical address of __hyp_stub_vectors - */ -SYM_CODE_START_LOCAL(el1_sync) - msr vbar_el2, x24 - eret -SYM_CODE_END(el1_sync) - -.macro invalid_vector label -SYM_CODE_START_LOCAL(\label) - b \label -SYM_CODE_END(\label) -.endm - - invalid_vector el2_sync_invalid - invalid_vector el2_irq_invalid - invalid_vector el2_fiq_invalid - invalid_vector el2_error_invalid - invalid_vector el1_sync_invalid - invalid_vector el1_irq_invalid - invalid_vector el1_fiq_invalid - invalid_vector el1_error_invalid - -/* el2 vectors - switch el2 here while we restore the memory image. */ - .align 11 -SYM_CODE_START(hibernate_el2_vectors) - ventry el2_sync_invalid // Synchronous EL2t - ventry el2_irq_invalid // IRQ EL2t - ventry el2_fiq_invalid // FIQ EL2t - ventry el2_error_invalid // Error EL2t - - ventry el2_sync_invalid // Synchronous EL2h - ventry el2_irq_invalid // IRQ EL2h - ventry el2_fiq_invalid // FIQ EL2h - ventry el2_error_invalid // Error EL2h - - ventry el1_sync // Synchronous 64-bit EL1 - ventry el1_irq_invalid // IRQ 64-bit EL1 - ventry el1_fiq_invalid // FIQ 64-bit EL1 - ventry el1_error_invalid // Error 64-bit EL1 - - ventry el1_sync_invalid // Synchronous 32-bit EL1 - ventry el1_irq_invalid // IRQ 32-bit EL1 - ventry el1_fiq_invalid // FIQ 32-bit EL1 - ventry el1_error_invalid // Error 32-bit EL1 -SYM_CODE_END(hibernate_el2_vectors) - .popsection diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index 4c9533f4c0c4..b96ef9060e4c 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -51,9 +51,6 @@ extern int in_suspend; /* Do we need to reset el2? */ #define el2_reset_needed() (is_hyp_nvhe()) -/* temporary el2 vectors in the __hibernate_exit_text section. */ -extern char hibernate_el2_vectors[]; - /* hyp-stub vectors, used to restore el2 during resume from hibernate. */ extern char __hyp_stub_vectors[]; @@ -434,6 +431,7 @@ int swsusp_arch_resume(void) void *zero_page; size_t exit_size; pgd_t *tmp_pg_dir; + phys_addr_t el2_vectors; void __noreturn (*hibernate_exit)(phys_addr_t, phys_addr_t, void *, void *, phys_addr_t, phys_addr_t); struct trans_pgd_info trans_info = { @@ -461,6 +459,14 @@ int swsusp_arch_resume(void) return -ENOMEM; } + if (el2_reset_needed()) { + rc = trans_pgd_copy_el2_vectors(&trans_info, &el2_vectors); + if (rc) { + pr_err("Failed to setup el2 vectors\n"); + return rc; + } + } + exit_size = __hibernate_exit_text_end - __hibernate_exit_text_start; /* * Copy swsusp_arch_suspend_exit() to a safe page. This will generate @@ -473,26 +479,14 @@ int swsusp_arch_resume(void) return rc; } - /* - * The hibernate exit text contains a set of el2 vectors, that will - * be executed at el2 with the mmu off in order to reload hyp-stub. - */ - dcache_clean_inval_poc((unsigned long)hibernate_exit, - (unsigned long)hibernate_exit + exit_size); - /* * KASLR will cause the el2 vectors to be in a different location in * the resumed kernel. Load hibernate's temporary copy into el2. * * We can skip this step if we booted at EL1, or are running with VHE. */ - if (el2_reset_needed()) { - phys_addr_t el2_vectors = (phys_addr_t)hibernate_exit; - el2_vectors += hibernate_el2_vectors - - __hibernate_exit_text_start; /* offset */ - + if (el2_reset_needed()) __hyp_set_vectors(el2_vectors); - } hibernate_exit(virt_to_phys(tmp_pg_dir), resume_hdr.ttbr1_el1, resume_hdr.reenter_kernel, restore_pblist, diff --git a/arch/arm64/mm/Makefile b/arch/arm64/mm/Makefile index f188c9092696..ff1e800ba7a1 100644 --- a/arch/arm64/mm/Makefile +++ b/arch/arm64/mm/Makefile @@ -7,6 +7,7 @@ obj-$(CONFIG_HUGETLB_PAGE) += hugetlbpage.o obj-$(CONFIG_PTDUMP_CORE) += ptdump.o obj-$(CONFIG_PTDUMP_DEBUGFS) += ptdump_debugfs.o obj-$(CONFIG_TRANS_TABLE) += trans_pgd.o +obj-$(CONFIG_TRANS_TABLE) += trans_pgd-asm.o obj-$(CONFIG_DEBUG_VIRTUAL) += physaddr.o obj-$(CONFIG_ARM64_MTE) += mteswap.o KASAN_SANITIZE_physaddr.o += n diff --git a/arch/arm64/mm/trans_pgd-asm.S b/arch/arm64/mm/trans_pgd-asm.S new file mode 100644 index 000000000000..831d6369494e --- /dev/null +++ b/arch/arm64/mm/trans_pgd-asm.S @@ -0,0 +1,58 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ + +/* + * Copyright (c) 2021, Microsoft Corporation. + * Pavel Tatashin + */ + +#include +#include +#include + +.macro invalid_vector label +SYM_CODE_START_LOCAL(\label) + .align 7 + b \label +SYM_CODE_END(\label) +.endm + +.macro el1_sync_vector +SYM_CODE_START_LOCAL(el1_sync) + .align 7 + cmp x0, #HVC_SET_VECTORS /* Called from hibernate */ + b.ne 1f + msr vbar_el2, x1 + mov x0, xzr + eret +1: /* Unexpected argument, set an error */ + mov_q x0, HVC_STUB_ERR + eret +SYM_CODE_END(el1_sync) +.endm + +SYM_CODE_START(trans_pgd_stub_vectors) + invalid_vector hyp_stub_el2t_sync_invalid // Synchronous EL2t + invalid_vector hyp_stub_el2t_irq_invalid // IRQ EL2t + invalid_vector hyp_stub_el2t_fiq_invalid // FIQ EL2t + invalid_vector hyp_stub_el2t_error_invalid // Error EL2t + + invalid_vector hyp_stub_el2h_sync_invalid // Synchronous EL2h + invalid_vector hyp_stub_el2h_irq_invalid // IRQ EL2h + invalid_vector hyp_stub_el2h_fiq_invalid // FIQ EL2h + invalid_vector hyp_stub_el2h_error_invalid // Error EL2h + + el1_sync_vector // Synchronous 64-bit EL1 + invalid_vector hyp_stub_el1_irq_invalid // IRQ 64-bit EL1 + invalid_vector hyp_stub_el1_fiq_invalid // FIQ 64-bit EL1 + invalid_vector hyp_stub_el1_error_invalid // Error 64-bit EL1 + + invalid_vector hyp_stub_32b_el1_sync_invalid // Synchronous 32-bit EL1 + invalid_vector hyp_stub_32b_el1_irq_invalid // IRQ 32-bit EL1 + invalid_vector hyp_stub_32b_el1_fiq_invalid // FIQ 32-bit EL1 + invalid_vector hyp_stub_32b_el1_error_invalid // Error 32-bit EL1 + .align 11 +SYM_INNER_LABEL(__trans_pgd_stub_vectors_end, SYM_L_LOCAL) +SYM_CODE_END(trans_pgd_stub_vectors) + +# Check the trans_pgd_stub_vectors didn't overflow +.org . - (__trans_pgd_stub_vectors_end - trans_pgd_stub_vectors) + SZ_2K diff --git a/arch/arm64/mm/trans_pgd.c b/arch/arm64/mm/trans_pgd.c index 527f0a39c3da..a08cf313c858 100644 --- a/arch/arm64/mm/trans_pgd.c +++ b/arch/arm64/mm/trans_pgd.c @@ -5,7 +5,7 @@ * * This file derived from: arch/arm64/kernel/hibernate.c * - * Copyright (c) 2020, Microsoft Corporation. + * Copyright (c) 2021, Microsoft Corporation. * Pavel Tatashin * */ @@ -322,3 +322,26 @@ int trans_pgd_idmap_page(struct trans_pgd_info *info, phys_addr_t *trans_ttbr0, return 0; } + +/* + * Create a copy of the vector table so we can call HVC_SET_VECTORS or + * HVC_SOFT_RESTART from contexts where the table may be overwritten. + */ +int trans_pgd_copy_el2_vectors(struct trans_pgd_info *info, + phys_addr_t *el2_vectors) +{ + void *hyp_stub = trans_alloc(info); + + if (!hyp_stub) + return -ENOMEM; + *el2_vectors = virt_to_phys(hyp_stub); + memcpy(hyp_stub, &trans_pgd_stub_vectors, ARM64_VECTOR_TABLE_LEN); + caches_clean_inval_pou((unsigned long)hyp_stub, + (unsigned long)hyp_stub + + ARM64_VECTOR_TABLE_LEN); + dcache_clean_inval_poc((unsigned long)hyp_stub, + (unsigned long)hyp_stub + + ARM64_VECTOR_TABLE_LEN); + + return 0; +} From patchwork Mon Aug 2 21:53:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12414857 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F2F2BC4338F for ; Mon, 2 Aug 2021 21:56:18 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BB43D6023E for ; Mon, 2 Aug 2021 21:56:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org BB43D6023E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=h6B0XhTPub8m4K4rb4kNLeu9qdZcWc/3DYLkmk/IW50=; b=qvrEc45HvL449I GH6Mo3oO3HN7mSFr9xGC4GoUapiOHAPfdTUc+FnFnqeRx4FDNLd0UgbcdxZ6/gxyXMxTwZoxc3TEV 6hXzTrKy/hpu+yah3SDcvv376Y5XSRnUluh4uqtUqIar/2+v77cSQxuOnYK4wURtYYHcrczMIuMJo a2gwFofvL7kMKjTAvYv33WJ+ERrRAP/wDiUf6J4xDgPPPRr1A7FsamoR5Ut5dg+cKvVfA2gAg9Xu3 USSoYt+iV3WaCFSPci2HQBTcvAZ1S8wxa7f0tAjDSCnlmJZgmdSWfX3U9KjN5nPxWHxCYkaY7yBh2 Eza9LQryIFMndOiNvtOQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mAftY-000K1R-DJ; Mon, 02 Aug 2021 21:54:48 +0000 Received: from mail-qk1-x731.google.com ([2607:f8b0:4864:20::731]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mAft2-000Jlq-0p for linux-arm-kernel@lists.infradead.org; Mon, 02 Aug 2021 21:54:18 +0000 Received: by mail-qk1-x731.google.com with SMTP id z24so18113671qkz.7 for ; Mon, 02 Aug 2021 14:54:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=96Un8yzVfu7KwL+UKPbJdUfuiZIxSkwQ3ffVCXyGsJg=; b=fDKdvVvJFzZLoHCfTztgqUc8ghI0wc0uDmTwEByiClo9Fcrah9+JXI4wlANk0MLP0Z Baythc3XdKwhO6bqGBTm0LqWBDzhSfCqsmEOuAfGPKQbvlzFWkTHH8MV05CYMtSLUBFu 5A4BPjNQfVh86B1fU1SpksUzRvwB4KyS8V1IXEJSxY4A/NqPTIfGZvTSvoApGAai4N+D 40BkEUuNgNrDlgFfukt0M3Bu4OoAVKM1CuG4vnkAP7JypBRhYmitFffAs3WZCtDov+Kp IvrPNUTuRAk4TWlGmF2NZSw6CuXflGh3nUfgH6xcKGnBAyWnyq0R/Z6iB6Zw7gEswAFZ I/sA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=96Un8yzVfu7KwL+UKPbJdUfuiZIxSkwQ3ffVCXyGsJg=; b=nXJDNDUof4qPtv/bdxrinCmw8KyQQ/kOWqdoBORfTdH+W059cELW7WTC1QhxvhGSQs Oarv7lA21Ac4MQiGEzNge/MX0aCwF3TJHJquYcHtMhDMPcovPbsUisULsIO8iq/s9OfO ha1civB3Mu1YA640aIoGjLWqQJAfpwQxmPPzAcnZTu6njvCR9xC4T7vtOdbcdsmomxBF M0FFF2hQPx/5OQkNmBwJw1tqN6G81Q9mmUuDYI/5OO2t+UldBvvD8ehMpez51kpi3qGH 92tksj+7ic2RNY7jhaBe34vYXy8yfT7J+edgovEPbBnMAdMo8Acx59czrJK2E66vK4cW EJSg== X-Gm-Message-State: AOAM533OQxQuhvtZgzFmSHmUYXbY/G5/PiAsT+B76xeWEkOE5H2ENLe0 Mn8QrZnwxDqgPe+4qaHED3earQ== X-Google-Smtp-Source: ABdhPJxxBXqSP3qMfUR18poVpxw/DGnsuk4IsHc07cmFedVkC3VWV8old/VTOr1+C7JPN//f26ZF2A== X-Received: by 2002:a05:620a:164b:: with SMTP id c11mr17543239qko.156.1627941255271; Mon, 02 Aug 2021 14:54:15 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id v11sm5479216qtc.0.2021.08.02.14.54.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Aug 2021 14:54:14 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH v16 03/15] arm64: hibernate: abstract ttrb0 setup function Date: Mon, 2 Aug 2021 17:53:56 -0400 Message-Id: <20210802215408.804942-4-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210802215408.804942-1-pasha.tatashin@soleen.com> References: <20210802215408.804942-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210802_145416_167776_2519FE2E X-CRM114-Status: GOOD ( 15.95 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently, only hibernate sets custom ttbr0 with safe idmaped function. Kexec, is also going to be using this functionality when relocation code is going to be idmapped. Move the setup sequence to a dedicated cpu_install_ttbr0() for custom ttbr0. Suggested-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/mmu_context.h | 24 ++++++++++++++++++++++++ arch/arm64/kernel/hibernate.c | 21 +-------------------- 2 files changed, 25 insertions(+), 20 deletions(-) diff --git a/arch/arm64/include/asm/mmu_context.h b/arch/arm64/include/asm/mmu_context.h index eeb210997149..edb57e075abe 100644 --- a/arch/arm64/include/asm/mmu_context.h +++ b/arch/arm64/include/asm/mmu_context.h @@ -115,6 +115,30 @@ static inline void cpu_install_idmap(void) cpu_switch_mm(lm_alias(idmap_pg_dir), &init_mm); } +/* + * Load our new page tables. A strict BBM approach requires that we ensure that + * TLBs are free of any entries that may overlap with the global mappings we are + * about to install. + * + * For a real hibernate/resume/kexec cycle TTBR0 currently points to a zero + * page, but TLBs may contain stale ASID-tagged entries (e.g. for EFI runtime + * services), while for a userspace-driven test_resume cycle it points to + * userspace page tables (and we must point it at a zero page ourselves). + * + * We change T0SZ as part of installing the idmap. This is undone by + * cpu_uninstall_idmap() in __cpu_suspend_exit(). + */ +static inline void cpu_install_ttbr0(phys_addr_t ttbr0, unsigned long t0sz) +{ + cpu_set_reserved_ttbr0(); + local_flush_tlb_all(); + __cpu_set_tcr_t0sz(t0sz); + + /* avoid cpu_switch_mm() and its SW-PAN and CNP interactions */ + write_sysreg(ttbr0, ttbr0_el1); + isb(); +} + /* * Atomically replaces the active TTBR1_EL1 PGD with a new VA-compatible PGD, * avoiding the possibility of conflicting TLB entries being allocated. diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index b96ef9060e4c..2758f75d6809 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -212,26 +212,7 @@ static int create_safe_exec_page(void *src_start, size_t length, if (rc) return rc; - /* - * Load our new page tables. A strict BBM approach requires that we - * ensure that TLBs are free of any entries that may overlap with the - * global mappings we are about to install. - * - * For a real hibernate/resume cycle TTBR0 currently points to a zero - * page, but TLBs may contain stale ASID-tagged entries (e.g. for EFI - * runtime services), while for a userspace-driven test_resume cycle it - * points to userspace page tables (and we must point it at a zero page - * ourselves). - * - * We change T0SZ as part of installing the idmap. This is undone by - * cpu_uninstall_idmap() in __cpu_suspend_exit(). - */ - cpu_set_reserved_ttbr0(); - local_flush_tlb_all(); - __cpu_set_tcr_t0sz(t0sz); - write_sysreg(trans_ttbr0, ttbr0_el1); - isb(); - + cpu_install_ttbr0(trans_ttbr0, t0sz); *phys_dst_addr = virt_to_phys(page); return 0; From patchwork Mon Aug 2 21:53:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12414863 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 888C1C4338F for ; Mon, 2 Aug 2021 21:57:05 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5357860F9C for ; Mon, 2 Aug 2021 21:57:05 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 5357860F9C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=iWvh0SExoF7yPoTycyoO3Vxkge9P6e+vLOLVwiAmITw=; b=Rfeh/D27qLA1QU 6HygbVU1MPDa3OXmCwqpCZhboCtfeRxeCl5zwPzk6hYV0mJm4TYcB04qeo+P65EkR/Hzobxg8Wc9X /kZHVvTjIvJKX5ld1YasXZuQo6CK6YNtRgjZBdWUUGRjUvMuT+gieTSp2LxmL/sae0q6NROW4AlOh 37SIvA+IWpZ2mktg6olKK44owzjqiLcuvUWVDi/JgRHmec1/PamXpXbRYgfb0H0i0acWpIUUorpBl KOOqtsiLZKpK1yN31bKoqNGZBavHdzm4K8lvsqdv+9JitkcgaizdAPoVGnbQJ98npDY5wjL2KEjX5 nqN98iWfDVcqYixvVQ+Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mAfu8-000KHt-4j; Mon, 02 Aug 2021 21:55:24 +0000 Received: from mail-qt1-x833.google.com ([2607:f8b0:4864:20::833]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mAft4-000JmS-2q for linux-arm-kernel@lists.infradead.org; Mon, 02 Aug 2021 21:54:20 +0000 Received: by mail-qt1-x833.google.com with SMTP id d2so12701452qto.6 for ; Mon, 02 Aug 2021 14:54:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=j+M5hF4GoTM52s3DZq4Ce8jXkOdwrhiLtHbwgSj+FD0=; b=l6aqL1CM2TQZNGjxlG/OMB4a6O9NQjV7sXgli2BxOuiZqOg1+R4L39Hu+XxA1MP2s5 t4dP5Us1cylcgLhIaClJU45qlE2AWSN05gLhWbNEddIvUnvOppiFWBMN2dBLty8Bu7WE 6gaf3VroJKSsrw0OpzaZNzXBElExwfW5ATW9eqbqeBEBmhBNUHbqNvkji2FfJ/IGEybJ ZpKuE68zFf/xMo0Q4srgWww79rMfb7cNXt6JS4zZ6O5a2P7HnD1tEpc/dGa5a406/pco wPgUP5aLSqk1c0Bu/yhNnpFj30p70V3WrBAZsa/YupGYPMXcilEBYoBGGqiU7xVgLZb1 yBcQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=j+M5hF4GoTM52s3DZq4Ce8jXkOdwrhiLtHbwgSj+FD0=; b=mMGc164qBn3v56n+Y8kk/S6LVgWrAltTk0+drlzlUBHyA7gQD0oBlJ04R1iC5F8bCI pYpl7qimeAY+p4lWS0Z4hvjA7nWXKzo9fZorXcJkv7fSit7rdH9zHbzPn0D9Eih6GAK5 gDKwV5m+oPQoZlr5QlvPbyV8IzHS0nIdY8JrD+sPPYcGFHU8wLainn3p93W32j7OKANf Djes8ojyyYbrDeii65d5FYmdMgVBxd3/gKrBqKXPTND08LKoYHkbEsgPtngGUBtHA1Ph 8CUZPA1EOjAwQrpB9cWqN4iObhU66tzlC03AXW2P7m5n84ip7cIELVPWDodJVJNViNmX T5FQ== X-Gm-Message-State: AOAM5301MDaPS7vs/kEzZ2yrbQidf3+n8SmmwsSN60MjJm/h0lI6aA81 9WEVyW5I75uDWEJ/AkjMy4FLaw== X-Google-Smtp-Source: ABdhPJzftvfeQfBpD80+56bQsic+tvcE4+ki6fuiemQ88Y64RUc79X5wGWVVM3Jr7aFMFqrTIyjfxw== X-Received: by 2002:ac8:57c4:: with SMTP id w4mr15810018qta.39.1627941256639; Mon, 02 Aug 2021 14:54:16 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id v11sm5479216qtc.0.2021.08.02.14.54.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Aug 2021 14:54:16 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH v16 04/15] arm64: kexec: flush image and lists during kexec load time Date: Mon, 2 Aug 2021 17:53:57 -0400 Message-Id: <20210802215408.804942-5-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210802215408.804942-1-pasha.tatashin@soleen.com> References: <20210802215408.804942-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210802_145418_169090_ECA8C757 X-CRM114-Status: GOOD ( 16.86 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently, during kexec load we are copying relocation function and flushing it. However, we can also flush kexec relocation buffers and if new kernel image is already in place (i.e. crash kernel), we can also flush the new kernel image itself. Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/machine_kexec.c | 58 ++++++++++++++----------------- 1 file changed, 26 insertions(+), 32 deletions(-) diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 213d56c14f60..b6d5a02cba2e 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -60,29 +60,6 @@ void machine_kexec_cleanup(struct kimage *kimage) /* Empty routine needed to avoid build errors. */ } -int machine_kexec_post_load(struct kimage *kimage) -{ - void *reloc_code = page_to_virt(kimage->control_code_page); - - memcpy(reloc_code, arm64_relocate_new_kernel, - arm64_relocate_new_kernel_size); - kimage->arch.kern_reloc = __pa(reloc_code); - kexec_image_info(kimage); - - /* - * For execution with the MMU off, reloc_code needs to be cleaned to the - * PoC and invalidated from the I-cache. - */ - dcache_clean_inval_poc((unsigned long)reloc_code, - (unsigned long)reloc_code + - arm64_relocate_new_kernel_size); - icache_inval_pou((uintptr_t)reloc_code, - (uintptr_t)reloc_code + - arm64_relocate_new_kernel_size); - - return 0; -} - /** * machine_kexec_prepare - Prepare for a kexec reboot. * @@ -163,6 +140,32 @@ static void kexec_segment_flush(const struct kimage *kimage) } } +int machine_kexec_post_load(struct kimage *kimage) +{ + void *reloc_code = page_to_virt(kimage->control_code_page); + + /* If in place flush new kernel image, else flush lists and buffers */ + if (kimage->head & IND_DONE) + kexec_segment_flush(kimage); + else + kexec_list_flush(kimage); + + memcpy(reloc_code, arm64_relocate_new_kernel, + arm64_relocate_new_kernel_size); + kimage->arch.kern_reloc = __pa(reloc_code); + kexec_image_info(kimage); + + /* Flush the reloc_code in preparation for its execution. */ + dcache_clean_inval_poc((unsigned long)reloc_code, + (unsigned long)reloc_code + + arm64_relocate_new_kernel_size); + icache_inval_pou((uintptr_t)reloc_code, + (uintptr_t)reloc_code + + arm64_relocate_new_kernel_size); + + return 0; +} + /** * machine_kexec - Do the kexec reboot. * @@ -180,13 +183,6 @@ void machine_kexec(struct kimage *kimage) WARN(in_kexec_crash && (stuck_cpus || smp_crash_stop_failed()), "Some CPUs may be stale, kdump will be unreliable.\n"); - /* Flush the kimage list and its buffers. */ - kexec_list_flush(kimage); - - /* Flush the new image if already in place. */ - if ((kimage != kexec_crash_image) && (kimage->head & IND_DONE)) - kexec_segment_flush(kimage); - pr_info("Bye!\n"); local_daif_mask(); @@ -261,8 +257,6 @@ void arch_kexec_protect_crashkres(void) { int i; - kexec_segment_flush(kexec_crash_image); - for (i = 0; i < kexec_crash_image->nr_segments; i++) set_memory_valid( __phys_to_virt(kexec_crash_image->segment[i].mem), From patchwork Mon Aug 2 21:53:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12414865 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 30024C4338F for ; Mon, 2 Aug 2021 21:58:08 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id F398060187 for ; Mon, 2 Aug 2021 21:58:07 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org F398060187 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=fkoo2TV9aw8WAM22c/m6jFtfE64Uxf+w/zgDKCWro+I=; b=zty5gY335BnGuy XfUVToBP5onhXnkyKkpDXdq1jE0iHQwyIifGfcr9kUHs2qVRu/6dmv8UKAM5bOhqtcPG6eaaQHTak GgY6x2nbYx4XXEeFCyKvGAmFRFRqLHCpLn81lVMceRXk5VgeT7oVVRUUtlnEzXj1szkQgAkUDgmsS 3hOWqk67t5kP4NHi2AiU+SyhLapxAyfnCpuud30fnPS5DFHEaqzaWZfcZ6PzQhPoxbeExO8gbN8Oe 3kEt8uuuPpGx0wyR4DfKNMKX9GvXIkgSFMIauYBNw1EdLz3ayi6pzgZ9zxDzfyl1JjL5oGlWnOUTM w/aAfcKyX260obCxsaIQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mAfue-000KZr-Kg; Mon, 02 Aug 2021 21:55:58 +0000 Received: from mail-qk1-x731.google.com ([2607:f8b0:4864:20::731]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mAft4-000JnT-NW for linux-arm-kernel@lists.infradead.org; Mon, 02 Aug 2021 21:54:21 +0000 Received: by mail-qk1-x731.google.com with SMTP id c9so18094852qkc.13 for ; Mon, 02 Aug 2021 14:54:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=ySRVVRwC++hMLjkNftECyAUjQh43zsO4xteYxz9kO+M=; b=dDf0vz0iWjVamQXhs2vdDdV8GxpuCrMHeOrVcKatUZm8kDuGj8aeCscvU0ZKD0PR0u P3pwTsllBPlz2qGvsHN3dUnYVn9dSEo6LBJP4H0RtxYomoXuhtxjmIISfIRQ193rnuhP 7enai/reMUqp8BRJ5yUgJrsJ1VF4UhhHchvnJ/ktN2mn4+i6gV4H1OPJh0qz01OH1/6B OQvr+N3zf28VJq15xRpdvv1UhQgnEzM4s2IEP+eICABab0xDNAQhm8X2FjUx1acmwCXi Yh399GttIvMrf46wXZgEWsv0Srz2KGWWLNr48vP5KF4aBEf9kYJIUn7GwYCoX3x/AROl HE1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ySRVVRwC++hMLjkNftECyAUjQh43zsO4xteYxz9kO+M=; b=CoRzjvuvwYtM8MY37/0xY4uENko+pSz349XdfBI9t/HVJCi0BQKH4jiKmWuxm4S1f/ yXLg124FMZEFq3QQmxjKx+3MFCwIBNerXIz5aPnsZd6T+HhqX3+qGX5o1rQJXpT6krdb Y2gmt/2LRL25NiUFfcMW980uwiCTAWve5hZWNaOO5bBwaMztdG94kAxNRAUjrPwgi8r7 BUCfM9IruZ+gCSNKNBJv6k2Az2DZeNw+fxi2kIchGoEni5+92JxkLiy072KHMLQF7HUC CAt6+LrHQCr1PMNbQBxnU4PWmljasqyYKZbiEu70tEGzwMDev4ci2L+PebpDoDCgWg8T n+SQ== X-Gm-Message-State: AOAM531BM8Z/uj4O8f9wyUnshGzH6jsRYYcOpg4p1F+z8q5ga52lY7zv U1ZtKeMp0ZS/dSIxb1zQlj9Ouw== X-Google-Smtp-Source: ABdhPJycxFDEqftDsNgLCqQw0liKmCH9habEesWjmQ0Ce5hXdnSfdBKjeqzPZ1g5aBX40OQ7MsZwFg== X-Received: by 2002:a05:620a:4d0:: with SMTP id 16mr17965487qks.430.1627941257981; Mon, 02 Aug 2021 14:54:17 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id v11sm5479216qtc.0.2021.08.02.14.54.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Aug 2021 14:54:17 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH v16 05/15] arm64: kexec: skip relocation code for inplace kexec Date: Mon, 2 Aug 2021 17:53:58 -0400 Message-Id: <20210802215408.804942-6-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210802215408.804942-1-pasha.tatashin@soleen.com> References: <20210802215408.804942-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210802_145418_873893_9B17A04D X-CRM114-Status: GOOD ( 19.05 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In case of kdump or when segments are already in place the relocation is not needed, therefore the setup of relocation function and call to it can be skipped. Signed-off-by: Pavel Tatashin Suggested-by: James Morse --- arch/arm64/kernel/machine_kexec.c | 34 ++++++++++++++++++----------- arch/arm64/kernel/relocate_kernel.S | 3 --- 2 files changed, 21 insertions(+), 16 deletions(-) diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index b6d5a02cba2e..7f1cb5a2a463 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -144,16 +144,16 @@ int machine_kexec_post_load(struct kimage *kimage) { void *reloc_code = page_to_virt(kimage->control_code_page); - /* If in place flush new kernel image, else flush lists and buffers */ - if (kimage->head & IND_DONE) + /* If in place, relocation is not used, only flush next kernel */ + if (kimage->head & IND_DONE) { kexec_segment_flush(kimage); - else - kexec_list_flush(kimage); + kexec_image_info(kimage); + return 0; + } memcpy(reloc_code, arm64_relocate_new_kernel, arm64_relocate_new_kernel_size); kimage->arch.kern_reloc = __pa(reloc_code); - kexec_image_info(kimage); /* Flush the reloc_code in preparation for its execution. */ dcache_clean_inval_poc((unsigned long)reloc_code, @@ -162,6 +162,8 @@ int machine_kexec_post_load(struct kimage *kimage) icache_inval_pou((uintptr_t)reloc_code, (uintptr_t)reloc_code + arm64_relocate_new_kernel_size); + kexec_list_flush(kimage); + kexec_image_info(kimage); return 0; } @@ -188,19 +190,25 @@ void machine_kexec(struct kimage *kimage) local_daif_mask(); /* - * cpu_soft_restart will shutdown the MMU, disable data caches, then - * transfer control to the kern_reloc which contains a copy of - * the arm64_relocate_new_kernel routine. arm64_relocate_new_kernel - * uses physical addressing to relocate the new image to its final - * position and transfers control to the image entry point when the - * relocation is complete. + * Both restart and cpu_soft_restart will shutdown the MMU, disable data + * caches. However, restart will start new kernel or purgatory directly, + * cpu_soft_restart will transfer control to arm64_relocate_new_kernel * In kexec case, kimage->start points to purgatory assuming that * kernel entry and dtb address are embedded in purgatory by * userspace (kexec-tools). * In kexec_file case, the kernel starts directly without purgatory. */ - cpu_soft_restart(kimage->arch.kern_reloc, kimage->head, kimage->start, - kimage->arch.dtb_mem); + if (kimage->head & IND_DONE) { + typeof(__cpu_soft_restart) *restart; + + cpu_install_idmap(); + restart = (void *)__pa_symbol(function_nocfi(__cpu_soft_restart)); + restart(is_hyp_nvhe(), kimage->start, kimage->arch.dtb_mem, + 0, 0); + } else { + cpu_soft_restart(kimage->arch.kern_reloc, kimage->head, + kimage->start, kimage->arch.dtb_mem); + } BUG(); /* Should never get here. */ } diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index b78ea5de97a4..8058fabe0a76 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -32,8 +32,6 @@ SYM_CODE_START(arm64_relocate_new_kernel) mov x16, x0 /* x16 = kimage_head */ mov x14, xzr /* x14 = entry ptr */ mov x13, xzr /* x13 = copy dest */ - /* Check if the new image needs relocation. */ - tbnz x16, IND_DONE_BIT, .Ldone raw_dcache_line_size x15, x1 /* x15 = dcache line size */ .Lloop: and x12, x16, PAGE_MASK /* x12 = addr */ @@ -65,7 +63,6 @@ SYM_CODE_START(arm64_relocate_new_kernel) .Lnext: ldr x16, [x14], #8 /* entry = *ptr++ */ tbz x16, IND_DONE_BIT, .Lloop /* while (!(entry & DONE)) */ -.Ldone: /* wait for writes from copy_page to finish */ dsb nsh ic iallu From patchwork Mon Aug 2 21:53:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12414867 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9404EC4338F for ; Mon, 2 Aug 2021 21:58:51 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 62EFD60F9C for ; Mon, 2 Aug 2021 21:58:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 62EFD60F9C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=gtxkTrZNfSioFlEnsu5i9T04Smnr7nqMcuRE6zT4I+o=; b=QUyYlY/FS7Jw9O WbQ5yCsr5cSraWOqVA1w9iAZqfM1F1pnFGO6RZEkPVHiBue50Waj5cWHe8cEGXQiQXuglGk/CZAkl ufIxn6npDdKVXWruY0XuMF+A2KaR3oX+afsT52s+C+G552VtcBH0oQ+xiQx66IRgN7uIeqvfKWl7n oEyU2bA7laj6SRkR5HkbBzAV0QNtG4IC373gSEyruadOubMkaycRLYtFUOSd3wgyC3SslkNU9m5vR /X6Kd0necPRmHwiAlm/1aMycJTBZ8x2fpmuZ55IB+6RNZqMxgqAe/39f3qoxXLOX8f9jXZDSwFWBG KZSi6ylDrijuqacHppUA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mAfvd-000L5V-Bx; Mon, 02 Aug 2021 21:56:57 +0000 Received: from mail-qt1-x82d.google.com ([2607:f8b0:4864:20::82d]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mAft6-000JoN-2n for linux-arm-kernel@lists.infradead.org; Mon, 02 Aug 2021 21:54:23 +0000 Received: by mail-qt1-x82d.google.com with SMTP id m11so12704550qtx.7 for ; Mon, 02 Aug 2021 14:54:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=7WmZNTjCYPreMSwaFFNLECgzpgypzLUDtCmjnNr08GQ=; b=oY0zH9o9YX2CY11bHdfAerRBTkD4lTZ2sAiiEAYvAvCgscc68MPgvS+NSnCgeRBMYX 1nWBbaVU3L5EACElob0GOrNtU65IanIX5XfoK6xoDXFFYf+nTznXDkgogNt1nfT3km5G AbjzKI+tMBAonjcVvhydqSQzTeIKonsZhpZon96oCDQNiUF7QrPA3ZZuUXLZguQw6m6T qFSVvWk8jgCZhkJMN4b9aoIReuzw0kxTTXrVX2GFyi9d3uj9vZRybsngBg79nYSFsNFa xNhDXZuGB2LcP0swLknPS5n/jyX70+MuvS5gS8I/kqdh7iglnn8sUVBVJQZVtKmQgOra zpBQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=7WmZNTjCYPreMSwaFFNLECgzpgypzLUDtCmjnNr08GQ=; b=WaP3bCDKLBuY4iVpZj2PO+FFNROHjhIp3Rnz5mbj6xUVRFyDRH13UlBLEdAFnylsF0 o/i/7ttNILQEz65tBazH5We9dF/t0wAaxEWUIYpHJqPSQ35ZosWTmC83sYQL5dh4LcP+ s6wm4QHFcAKw0jOB6FyE3QwzPFlo5FKxuQSjCfSeD150awISDLg/lGvUe/lsaAAPIo29 eD5TTZjPCxgMQZj5IvZPQ/sVMog6uqEXPM+ws0gvBL7zEsJuZerx2UvTl5EqA79JEev3 XnCMPJ0UsL2x77Ml4YVmzY9V9gsvELgsIsyzrVBpFleWwR+yMzeW9r0dPXVBwwRN0bN8 1iZg== X-Gm-Message-State: AOAM5335+rKbFjNlMVvR+o0hHOlezD5ceI3f88EQBr6h7Ov8srEN5Hhg KxEfKNmoPgp7/HFEWUPkkQRipg== X-Google-Smtp-Source: ABdhPJzTkgygNyFWHsEIO6nLbFDgDOSOhkOQ867R7sIckpKnbLERl8w+s6+lrbKWAnEoeVf/XOiKNw== X-Received: by 2002:ac8:41d2:: with SMTP id o18mr16194864qtm.10.1627941259342; Mon, 02 Aug 2021 14:54:19 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id v11sm5479216qtc.0.2021.08.02.14.54.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Aug 2021 14:54:18 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH v16 06/15] arm64: kexec: Use dcache ops macros instead of open-coding Date: Mon, 2 Aug 2021 17:53:59 -0400 Message-Id: <20210802215408.804942-7-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210802215408.804942-1-pasha.tatashin@soleen.com> References: <20210802215408.804942-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210802_145420_228100_9F298A5D X-CRM114-Status: GOOD ( 14.51 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org kexec does dcache maintenance when it re-writes all memory. Our dcache_by_line_op macro depends on reading the sanitized DminLine from memory. Kexec may have overwritten this, so open-codes the sequence. dcache_by_line_op is a whole set of macros, it uses dcache_line_size which uses read_ctr for the sanitsed DminLine. Reading the DminLine is the first thing the dcache_by_line_op does. Rename dcache_by_line_op dcache_by_myline_op and take DminLine as an argument. Kexec can now use the slightly smaller macro. This makes up-coming changes to the dcache maintenance easier on the eye. Code generated by the existing callers is unchanged. Suggested-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/assembler.h | 30 ++++++++++++++++++++++------- arch/arm64/kernel/relocate_kernel.S | 13 +++---------- 2 files changed, 26 insertions(+), 17 deletions(-) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 89faca0e740d..71999a325055 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -405,19 +405,19 @@ alternative_endif /* * Macro to perform a data cache maintenance for the interval - * [start, end) + * [start, end) with dcache line size explicitly provided. * * op: operation passed to dc instruction * domain: domain used in dsb instruciton * start: starting virtual address of the region * end: end virtual address of the region + * linesz: dcache line size * fixup: optional label to branch to on user fault - * Corrupts: start, end, tmp1, tmp2 + * Corrupts: start, end, tmp */ - .macro dcache_by_line_op op, domain, start, end, tmp1, tmp2, fixup - dcache_line_size \tmp1, \tmp2 - sub \tmp2, \tmp1, #1 - bic \start, \start, \tmp2 + .macro dcache_by_myline_op op, domain, start, end, linesz, tmp, fixup + sub \tmp, \linesz, #1 + bic \start, \start, \tmp .Ldcache_op\@: .ifc \op, cvau __dcache_op_workaround_clean_cache \op, \start @@ -436,7 +436,7 @@ alternative_endif .endif .endif .endif - add \start, \start, \tmp1 + add \start, \start, \linesz cmp \start, \end b.lo .Ldcache_op\@ dsb \domain @@ -444,6 +444,22 @@ alternative_endif _cond_extable .Ldcache_op\@, \fixup .endm +/* + * Macro to perform a data cache maintenance for the interval + * [start, end) + * + * op: operation passed to dc instruction + * domain: domain used in dsb instruciton + * start: starting virtual address of the region + * end: end virtual address of the region + * fixup: optional label to branch to on user fault + * Corrupts: start, end, tmp1, tmp2 + */ + .macro dcache_by_line_op op, domain, start, end, tmp1, tmp2, fixup + dcache_line_size \tmp1, \tmp2 + dcache_by_myline_op \op, \domain, \start, \end, \tmp1, \tmp2, \fixup + .endm + /* * Macro to perform an instruction cache maintenance for the interval * [start, end) diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index 8058fabe0a76..8c43779e8cc6 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -41,16 +41,9 @@ SYM_CODE_START(arm64_relocate_new_kernel) tbz x16, IND_SOURCE_BIT, .Ltest_indirection /* Invalidate dest page to PoC. */ - mov x2, x13 - add x20, x2, #PAGE_SIZE - sub x1, x15, #1 - bic x2, x2, x1 -2: dc ivac, x2 - add x2, x2, x15 - cmp x2, x20 - b.lo 2b - dsb sy - + mov x2, x13 + add x1, x2, #PAGE_SIZE + dcache_by_myline_op ivac, sy, x2, x1, x15, x20 copy_page x13, x12, x1, x2, x3, x4, x5, x6, x7, x8 b .Lnext .Ltest_indirection: From patchwork Mon Aug 2 21:54:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12414869 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 380B2C4338F for ; Mon, 2 Aug 2021 21:59:54 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 03A0A60F9C for ; Mon, 2 Aug 2021 21:59:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 03A0A60F9C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=7b9AuVPrU1ZHE09Ldc6wgI/ojppSnNyeNL7MFsUkNhU=; b=Gx13vnjhcutINA TdsZyk0HsQXOYjNb9gA5u9V4B2MHl6NTx4F0hfPzYpaoifimx1tC8g7vr+bs83ywbFyCRmYynC1zJ RiRdCoTuXPuHmomB1FjkgTNjvP5xX20pmq4e7NeC3FU7tEmTJ+l3V4BGaaJHtIB0EorzBSDQg0ihQ mSjcHQU1SxWahumQIflIVVfpdRSKf9Z1koO6t/xU2W6P/JPCTDwgWLoB1PhzregDbqQYEvgyTrRDe u3G72qX2SgDj4K+HuQMGEvGRM2H+xuJ9D5cyw9sWroq3YgQxrhaEcpuqthRA6z1fukn1GMqr8e50R ZLC9BABVN1+etRm12EZg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mAfwZ-000Lbv-Tf; Mon, 02 Aug 2021 21:57:56 +0000 Received: from mail-qk1-x734.google.com ([2607:f8b0:4864:20::734]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mAft8-000JpI-Vj for linux-arm-kernel@lists.infradead.org; Mon, 02 Aug 2021 21:54:25 +0000 Received: by mail-qk1-x734.google.com with SMTP id 190so18086175qkk.12 for ; Mon, 02 Aug 2021 14:54:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=I2QiiBKBeLsI1fdTqHUVs6c3pFiCblpNdzxEe7/cZMQ=; b=C5ZCA/IZXtCXO/3qfEOXbzGTv9RX4OeD13DHyBOfI9sDrZQZp6vdHaWPVlbKwU1Re8 xipuVWmrEMo9evAXLt1MFogXl5HcbQt53L5FDp+KRxwVC554PuDw2TeM5JdQqK83o73v iGIevs6zJfYWGAaSAIXVMnK8nUygXOm3T/cD0qNnFTwsBTHwGsH+aaDT3dadEB55C6Bw aLxpimreTpXY5tDDsImOXM+B484WwN/gF46Z8FyEtUPnrhwEXsUPLfJaenEAS8i8uFEy D7FPQ3bT229af0quUj3i5LRFb5UMw1x5lUeCvEKChQ7q/j8MpxEtjz674tIrMyL63oOe liYQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=I2QiiBKBeLsI1fdTqHUVs6c3pFiCblpNdzxEe7/cZMQ=; b=qDWPXgYiHGp14UgGOKaxO62BkrxMvc5qY2ID280GOh2fAz2rooKJBTs9spd3yF874a rb7Is2H+L8NZ0nK3SedASz6d2g98K9dCdhlgo46wZ95v1+Hb8STTZvYcGORdzgkN1x/y kfN6uIQde1+Mz9xM8cTCPk4FNhK60JkSGM/WoTYr0dxCw8A/MN239dLBOoDnnUbKYPY1 R3aoKT5OIsL9kR41nYj37TpKiicdEt/tC/2S3zw3xRzr8OBAMbrC+GMpczPwMAkSXk1o dHk40AXFhBMxJGcFtYUBhnGgpehlChedoezwaX2IXfekRqN2u5wouxL1wfi1AMm/IBWU iY0Q== X-Gm-Message-State: AOAM531GiqEECo720R8ieH36WEOcca9THKFBaBSB6oTTmcQRdB4k6ulZ iKkaIEloEjq7730tC8xmc+ObIg== X-Google-Smtp-Source: ABdhPJyCJpMCX9SrR4/FWYeg/8Z/A4s8mVnLr4UX29DlcghfJhjtUwm1ds70DtjKMXOdqRH5RGBQUg== X-Received: by 2002:a05:620a:893:: with SMTP id b19mr17708637qka.487.1627941260749; Mon, 02 Aug 2021 14:54:20 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id v11sm5479216qtc.0.2021.08.02.14.54.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Aug 2021 14:54:20 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH v16 07/15] arm64: kexec: pass kimage as the only argument to relocation function Date: Mon, 2 Aug 2021 17:54:00 -0400 Message-Id: <20210802215408.804942-8-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210802215408.804942-1-pasha.tatashin@soleen.com> References: <20210802215408.804942-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210802_145423_086330_E62B5E71 X-CRM114-Status: GOOD ( 16.46 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently, kexec relocation function (arm64_relocate_new_kernel) accepts the following arguments: head: start of array that contains relocation information. entry: entry point for new kernel or purgatory. dtb_mem: first and only argument to entry. The number of arguments cannot be easily expended, because this function is also called from HVC_SOFT_RESTART, which preserves only three arguments. And, also arm64_relocate_new_kernel is written in assembly but called without stack, thus no place to move extra arguments to free registers. Soon, we will need to pass more arguments: once we enable MMU we will need to pass information about page tables. Pass kimage to arm64_relocate_new_kernel, and teach it to get the required fields from kimage. Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/asm-offsets.c | 7 +++++++ arch/arm64/kernel/machine_kexec.c | 7 +++++-- arch/arm64/kernel/relocate_kernel.S | 10 ++++------ 3 files changed, 16 insertions(+), 8 deletions(-) diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index c85670692afa..b41409b09d04 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -9,6 +9,7 @@ #include #include +#include #include #include #include @@ -170,6 +171,12 @@ int main(void) DEFINE(PTRAUTH_KERNEL_KEY_APIA, offsetof(struct ptrauth_keys_kernel, apia)); #endif BLANK(); +#endif +#ifdef CONFIG_KEXEC_CORE + DEFINE(KIMAGE_ARCH_DTB_MEM, offsetof(struct kimage, arch.dtb_mem)); + DEFINE(KIMAGE_HEAD, offsetof(struct kimage, head)); + DEFINE(KIMAGE_START, offsetof(struct kimage, start)); + BLANK(); #endif return 0; } diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 7f1cb5a2a463..e210b19592c6 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -84,6 +84,9 @@ static void kexec_list_flush(struct kimage *kimage) { kimage_entry_t *entry; + dcache_clean_inval_poc((unsigned long)kimage, + (unsigned long)kimage + sizeof(*kimage)); + for (entry = &kimage->head; ; entry++) { unsigned int flag; unsigned long addr; @@ -206,8 +209,8 @@ void machine_kexec(struct kimage *kimage) restart(is_hyp_nvhe(), kimage->start, kimage->arch.dtb_mem, 0, 0); } else { - cpu_soft_restart(kimage->arch.kern_reloc, kimage->head, - kimage->start, kimage->arch.dtb_mem); + cpu_soft_restart(kimage->arch.kern_reloc, virt_to_phys(kimage), + 0, 0); } BUG(); /* Should never get here. */ diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index 8c43779e8cc6..63ea19868f63 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -27,9 +27,7 @@ */ SYM_CODE_START(arm64_relocate_new_kernel) /* Setup the list loop variables. */ - mov x18, x2 /* x18 = dtb address */ - mov x17, x1 /* x17 = kimage_start */ - mov x16, x0 /* x16 = kimage_head */ + ldr x16, [x0, #KIMAGE_HEAD] /* x16 = kimage_head */ mov x14, xzr /* x14 = entry ptr */ mov x13, xzr /* x13 = copy dest */ raw_dcache_line_size x15, x1 /* x15 = dcache line size */ @@ -63,12 +61,12 @@ SYM_CODE_START(arm64_relocate_new_kernel) isb /* Start new image. */ - mov x0, x18 + ldr x4, [x0, #KIMAGE_START] /* relocation start */ + ldr x0, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ mov x1, xzr mov x2, xzr mov x3, xzr - br x17 - + br x4 SYM_CODE_END(arm64_relocate_new_kernel) .align 3 /* To keep the 64-bit values below naturally aligned. */ From patchwork Mon Aug 2 21:54:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12414873 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E2DC1C432BE for ; Mon, 2 Aug 2021 22:00:46 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id ABF5160187 for ; Mon, 2 Aug 2021 22:00:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org ABF5160187 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=dE6eEExFrT7fREq5y+q5E5rDdCCl1ynyd75Fy2zSkt8=; b=tDqPmqRG6IQkKY pRiZIRWdTr37cM5Y0gBhxxYQuEPSNe+3VzBr3iQkR6iFbKi90YlX8MYnex2j5zWNeKTyi2SZQ/YbO SkmBxhwMsdKmke5mv5dwRLeYUOShyF6jKI2BrfSXfpKRZihYIBh5d3akLg0DHej3YMeBJd6jh9tWz Tds6ck+E+egHX3O+MODZixEGf9n0KsXPn0UKi/zeK+EOOOab2xdzLU6Z6BmcoJTs9BPr6ODF5qXwR SzXpBrI4LXXzkUV6EShAUlWCovGuTFNYJiQTNnQpln7D5AEmvVIiOAuIR4h0xMtGwtAw0B/yRQYYZ z/q9TpbjuDW+fYNLEn1g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mAfxK-000Lxb-S2; Mon, 02 Aug 2021 21:58:43 +0000 Received: from mail-qk1-x72b.google.com ([2607:f8b0:4864:20::72b]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mAft9-000Jpr-Ii for linux-arm-kernel@lists.infradead.org; Mon, 02 Aug 2021 21:54:27 +0000 Received: by mail-qk1-x72b.google.com with SMTP id e14so579475qkg.3 for ; Mon, 02 Aug 2021 14:54:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=vllONSCM3KraKyPXhV20IkRxy2MdqODsPqhupnQLd9g=; b=FCHt4SLjRX4W0uKz1DC2INGR5UyRTa2DOy8SEOiO8odV0LjPUoY+8Xpup7M9GnxJUb M6HoRE2ijwYz85dPiuZGos8rnp6tGoSSiDdIK+KNQPkUPjsGgUKSWEC3/EjNSuDXPf1D xtWvr2VRWAjTPlT+W7jihLZucLsaD28uHw5zj1fOarSfM+vRhtRwZCuKxVeykXpY0e0M hFMcLjZYveT88rLNMJPd772Qm0WC9fOe6KowiYIfy1izlRpTyV92Kz2HIYoYfWwoiznc TvD4oG0+6DA0AcN97ItWekQTJFzr1XShyXpDasCVO0TX1C4KAy8+AaDBlxlYjLTcfQLr ODXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=vllONSCM3KraKyPXhV20IkRxy2MdqODsPqhupnQLd9g=; b=nvjmgMNdMZzqOGf+basFXRyRK99gFaF2AaN98FmJsdfJopak+Wl1V6/aJQKAGMEYLt aMflUVz1RJFOKEwBh1/9wZGxCKP30ywNe6rll4mMqGAF57mwL9djivfrtcgEwf/2Bp19 WNjk/RDCK13fj1QoJunpHUBdphgfazRfLMaVn2VW3MV3hC3m1yXR5G0WjqVpaZ1wS25v nnVe/gwND60AC7oTkXopFg9twI1lMqYzMGHMzUC+rGt7l9hx/0KLsePQ0aEv77e504aa pAH45ZgSX79eF1eP7WnUOt91SxYT5fKFF7Fwg5Fte6dwX3c1I4pot1BPYnysAo7smRi2 zEPA== X-Gm-Message-State: AOAM532PWM/Kxm9PfYkFxOe8LFND4OGSj8vmE66nd5fabaYBX5Cc6jkK d0f39wOufkC1Kg9EtGVI5yPprQ== X-Google-Smtp-Source: ABdhPJy481uCAJREXivorCHdWuWZlyVeqri0ToaKhEh9mzCiyb+NJMFr7CnItwoU5vg1cHArwmCvvQ== X-Received: by 2002:a05:620a:99b:: with SMTP id x27mr18189913qkx.138.1627941262162; Mon, 02 Aug 2021 14:54:22 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id v11sm5479216qtc.0.2021.08.02.14.54.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Aug 2021 14:54:21 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH v16 08/15] arm64: kexec: configure EL2 vectors for kexec Date: Mon, 2 Aug 2021 17:54:01 -0400 Message-Id: <20210802215408.804942-9-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210802215408.804942-1-pasha.tatashin@soleen.com> References: <20210802215408.804942-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210802_145423_665171_F67243EA X-CRM114-Status: GOOD ( 20.05 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org If we have a EL2 mode without VHE, the EL2 vectors are needed in order to switch to EL2 and jump to new world with hypervisor privileges. In preparation to MMU enabled relocation, configure our EL2 table now. Kexec uses #HVC_SOFT_RESTART to branch to the new world, so extend el1_sync vector that is provided by trans_pgd_copy_el2_vectors() to support this case. Signed-off-by: Pavel Tatashin --- arch/arm64/Kconfig | 2 +- arch/arm64/include/asm/kexec.h | 1 + arch/arm64/kernel/asm-offsets.c | 1 + arch/arm64/kernel/machine_kexec.c | 31 +++++++++++++++++++++++++++++++ arch/arm64/mm/trans_pgd-asm.S | 9 ++++++++- 5 files changed, 42 insertions(+), 2 deletions(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index b5b13a932561..86dc6ddd2b8a 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -1138,7 +1138,7 @@ config CRASH_DUMP config TRANS_TABLE def_bool y - depends on HIBERNATION + depends on HIBERNATION || KEXEC_CORE config XEN_DOM0 def_bool y diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h index 00dbcc71aeb2..753a1c398898 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -96,6 +96,7 @@ struct kimage_arch { void *dtb; phys_addr_t dtb_mem; phys_addr_t kern_reloc; + phys_addr_t el2_vectors; }; #ifdef CONFIG_KEXEC_FILE diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index b41409b09d04..47ccb57b787e 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -174,6 +174,7 @@ int main(void) #endif #ifdef CONFIG_KEXEC_CORE DEFINE(KIMAGE_ARCH_DTB_MEM, offsetof(struct kimage, arch.dtb_mem)); + DEFINE(KIMAGE_ARCH_EL2_VECTORS, offsetof(struct kimage, arch.el2_vectors)); DEFINE(KIMAGE_HEAD, offsetof(struct kimage, head)); DEFINE(KIMAGE_START, offsetof(struct kimage, start)); BLANK(); diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index e210b19592c6..59a4b4172b68 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -21,6 +21,7 @@ #include #include #include +#include #include "cpu-reset.h" @@ -43,7 +44,9 @@ static void _kexec_image_info(const char *func, int line, pr_debug(" start: %lx\n", kimage->start); pr_debug(" head: %lx\n", kimage->head); pr_debug(" nr_segments: %lu\n", kimage->nr_segments); + pr_debug(" dtb_mem: %pa\n", &kimage->arch.dtb_mem); pr_debug(" kern_reloc: %pa\n", &kimage->arch.kern_reloc); + pr_debug(" el2_vectors: %pa\n", &kimage->arch.el2_vectors); for (i = 0; i < kimage->nr_segments; i++) { pr_debug(" segment[%lu]: %016lx - %016lx, 0x%lx bytes, %lu pages\n", @@ -143,9 +146,27 @@ static void kexec_segment_flush(const struct kimage *kimage) } } +/* Allocates pages for kexec page table */ +static void *kexec_page_alloc(void *arg) +{ + struct kimage *kimage = (struct kimage *)arg; + struct page *page = kimage_alloc_control_pages(kimage, 0); + + if (!page) + return NULL; + + memset(page_address(page), 0, PAGE_SIZE); + + return page_address(page); +} + int machine_kexec_post_load(struct kimage *kimage) { void *reloc_code = page_to_virt(kimage->control_code_page); + struct trans_pgd_info info = { + .trans_alloc_page = kexec_page_alloc, + .trans_alloc_arg = kimage, + }; /* If in place, relocation is not used, only flush next kernel */ if (kimage->head & IND_DONE) { @@ -154,6 +175,14 @@ int machine_kexec_post_load(struct kimage *kimage) return 0; } + kimage->arch.el2_vectors = 0; + if (is_hyp_nvhe()) { + int rc = trans_pgd_copy_el2_vectors(&info, + &kimage->arch.el2_vectors); + if (rc) + return rc; + } + memcpy(reloc_code, arm64_relocate_new_kernel, arm64_relocate_new_kernel_size); kimage->arch.kern_reloc = __pa(reloc_code); @@ -209,6 +238,8 @@ void machine_kexec(struct kimage *kimage) restart(is_hyp_nvhe(), kimage->start, kimage->arch.dtb_mem, 0, 0); } else { + if (is_hyp_nvhe()) + __hyp_set_vectors(kimage->arch.el2_vectors); cpu_soft_restart(kimage->arch.kern_reloc, virt_to_phys(kimage), 0, 0); } diff --git a/arch/arm64/mm/trans_pgd-asm.S b/arch/arm64/mm/trans_pgd-asm.S index 831d6369494e..c1f2ed1be6de 100644 --- a/arch/arm64/mm/trans_pgd-asm.S +++ b/arch/arm64/mm/trans_pgd-asm.S @@ -24,7 +24,14 @@ SYM_CODE_START_LOCAL(el1_sync) msr vbar_el2, x1 mov x0, xzr eret -1: /* Unexpected argument, set an error */ +1: cmp x0, #HVC_SOFT_RESTART /* Called from kexec */ + b.ne 2f + mov x0, x2 + mov x2, x4 + mov x4, x1 + mov x1, x3 + br x4 +2: /* Unexpected argument, set an error */ mov_q x0, HVC_STUB_ERR eret SYM_CODE_END(el1_sync) From patchwork Mon Aug 2 21:54:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12414875 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2C219C4320A for ; Mon, 2 Aug 2021 22:01:33 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E4DCE60E78 for ; Mon, 2 Aug 2021 22:01:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org E4DCE60E78 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ubHmFXF75IlCttw1+gcLuTtInCWkfik59gsD/oYJsQM=; b=BO8HD3k9pY8cpo 0G7LXx49y0gSEpH1DPgfDRe1NXqdt4XDemjVDOGEojlLznIy/KDz6F2xRzrq8b61lHiw7ph7CUv+I ld2+lM00zo5tNn+F7UmxJ9TsOmByLgatlcYFNR97jxGfp+wsOdHR2v974BKTCDFWcNHoC/dT5NpzH kaWBwm7lXjIBWQ5QUpGBjbc2VmqCISr4DOBv+c6M2eytoV1Msi5n7oHdxBLQ9C+dv9DHLrdSfLEwv 9l0RzqhGn0Nmr2he2bbuIYooKjOvxwbXgfcLg0DaSrNJRDbzp7k+Lm2wtbg2HzOyl2wOzdoKr/lTe GOUMlpawcHOJ+WMsafLw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mAfy2-000MFO-JU; Mon, 02 Aug 2021 21:59:28 +0000 Received: from mail-qk1-x733.google.com ([2607:f8b0:4864:20::733]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mAftB-000JrA-Nn for linux-arm-kernel@lists.infradead.org; Mon, 02 Aug 2021 21:54:27 +0000 Received: by mail-qk1-x733.google.com with SMTP id o13so18106759qkk.9 for ; Mon, 02 Aug 2021 14:54:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=Wj2NAixWvJ4nO1FkxHbUkwp7sIToaWThfISCCFg6+8g=; b=GscCSfNwt44SsxAiPP7NgcgGNAXIbZIzPwC6/zqeK9kfqT8QLhkAwdkgkT2aZY087m 5KxwAVR12ZBJ/ZeCQcNoNKBBF9L+Ds0Hj+DcneIUI8ImzTrkqH9HxKuMPLFmwQsIL2XK bwuxoBK+4RmtzMm5BflhOQZqEsmZxQ0o70Zz2BcWeFPjp8YlBy4MF8vqjY80xbZLZ2jR T/2CwFLk0SR/DPY1g0sIDtYy50PCkJHFZYvSBfOmijbPnYSoHafv4h7czA+iRWjP/ZpV B1cOUnKsTAIUEMluLV/clockW2VvJCE1ndGhTSzVJhSqYABRu+HrJeFpQnkXSVhM/GCk O37Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Wj2NAixWvJ4nO1FkxHbUkwp7sIToaWThfISCCFg6+8g=; b=Aj74m/hYly8I6oReJSJJjJrVhMb2CMK3W1l56g54zAr56RrEmIWzaOoTHYmq+lFh7k hQwF7324ltgkdBOhtx4eH0sBfN2//+kgUic59JkG1oHmy2+6d6dnWh9HHjE4JUKPcln0 kL43mdSpfIDLay/ZNohIjj9pECGMY0podRT01f2Mtm8xrwJSyyAhX8882lJqXm2qAs7G E6LHSfzdLjixs3ph3fICuG8Y9WPW1fen0gH4Cy130eaEmrWAU+qxiU4v5YZg6nk8yjPe jrBM9H5WTJxRHphG0jPo1mq4ipsWukCCd3uX+cSQpAtdr4w7jBHwl7GhscoQ2p7BhfqT 1xYg== X-Gm-Message-State: AOAM530KeZShj37yJjOMq10I23/6ZVwzDDrSN5uX0NC0dYbQbZ+XmtZY plHGvKxDR87+jM0fGs+OQxyR0A== X-Google-Smtp-Source: ABdhPJwzqgDG63F1SdAX7UqdvA9CoqJlGcLHTWFM7eFYSkHjmR9g6tQlWW8DN/xg+wexZixuMpniZg== X-Received: by 2002:a05:620a:a8f:: with SMTP id v15mr17503007qkg.257.1627941263546; Mon, 02 Aug 2021 14:54:23 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id v11sm5479216qtc.0.2021.08.02.14.54.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Aug 2021 14:54:23 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH v16 09/15] arm64: kexec: relocate in EL1 mode Date: Mon, 2 Aug 2021 17:54:02 -0400 Message-Id: <20210802215408.804942-10-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210802215408.804942-1-pasha.tatashin@soleen.com> References: <20210802215408.804942-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210802_145425_859919_EAC17E9E X-CRM114-Status: GOOD ( 13.33 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Since we are going to keep MMU enabled during relocation, we need to keep EL1 mode throughout the relocation. Keep EL1 enabled, and switch EL2 only before entering the new world. Suggested-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/cpu-reset.h | 3 +-- arch/arm64/kernel/machine_kexec.c | 4 ++-- arch/arm64/kernel/relocate_kernel.S | 13 +++++++++++-- 3 files changed, 14 insertions(+), 6 deletions(-) diff --git a/arch/arm64/kernel/cpu-reset.h b/arch/arm64/kernel/cpu-reset.h index 81b3d0fe7a63..296abbac7192 100644 --- a/arch/arm64/kernel/cpu-reset.h +++ b/arch/arm64/kernel/cpu-reset.h @@ -20,11 +20,10 @@ static inline void __noreturn __nocfi cpu_soft_restart(unsigned long entry, { typeof(__cpu_soft_restart) *restart; - unsigned long el2_switch = is_hyp_nvhe(); restart = (void *)__pa_symbol(function_nocfi(__cpu_soft_restart)); cpu_install_idmap(); - restart(el2_switch, entry, arg0, arg1, arg2); + restart(0, entry, arg0, arg1, arg2); unreachable(); } diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 59a4b4172b68..cf5d6f22a041 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -240,8 +240,8 @@ void machine_kexec(struct kimage *kimage) } else { if (is_hyp_nvhe()) __hyp_set_vectors(kimage->arch.el2_vectors); - cpu_soft_restart(kimage->arch.kern_reloc, virt_to_phys(kimage), - 0, 0); + cpu_soft_restart(kimage->arch.kern_reloc, + virt_to_phys(kimage), 0, 0); } BUG(); /* Should never get here. */ diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index 63ea19868f63..b4fb97312a80 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -13,6 +13,7 @@ #include #include #include +#include /* * arm64_relocate_new_kernel - Put a 2nd stage image in place and boot it. @@ -61,12 +62,20 @@ SYM_CODE_START(arm64_relocate_new_kernel) isb /* Start new image. */ + ldr x1, [x0, #KIMAGE_ARCH_EL2_VECTORS] /* relocation start */ + cbz x1, .Lel1 + ldr x1, [x0, #KIMAGE_START] /* relocation start */ + ldr x2, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ + mov x3, xzr + mov x4, xzr + mov x0, #HVC_SOFT_RESTART + hvc #0 /* Jumps from el2 */ +.Lel1: ldr x4, [x0, #KIMAGE_START] /* relocation start */ ldr x0, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ - mov x1, xzr mov x2, xzr mov x3, xzr - br x4 + br x4 /* Jumps from el1 */ SYM_CODE_END(arm64_relocate_new_kernel) .align 3 /* To keep the 64-bit values below naturally aligned. */ From patchwork Mon Aug 2 21:54:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12414877 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EB578C4338F for ; Mon, 2 Aug 2021 22:02:22 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A70EF60E09 for ; Mon, 2 Aug 2021 22:02:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org A70EF60E09 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Jb0ZzDgvewrjcQuxCFZHD1aPJA65PTO8l28VVyhtSF8=; b=a1d2gF7PiUwQox SXwx5ysyBTPH8wjFcHa8NHEjxp/XTjFBM4MxSiK9oHuJZMl6IdKuAgVBTLuSUEn2QQENuZ/FPBXHf 8CHT6ByRBkQFLCeCBT1x3wrvzQ3ytx1kEAfhwNgjLka+gv1W37U0+0Aaa4wFdKMD8u8XkgnmY2ulm B69YTJ3i/Q5ZdX8hW0JOl221naxlNamIDgmvPEyRxByaoMhgOx3FycFOlx0nFMgk4HcmJ3UhVkAeW adlB8zuLHLwaKGMA8Pj4BoWja/9iaTztFxpablH9XrtxylnHJPPIzg1rWbyJtxkGv1YQe/5Chr64O 0LvPuacLoBAmdTRN9fvQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mAfyk-000Mfb-Mk; Mon, 02 Aug 2021 22:00:10 +0000 Received: from mail-qt1-x82e.google.com ([2607:f8b0:4864:20::82e]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mAftC-000Jrf-Fg for linux-arm-kernel@lists.infradead.org; Mon, 02 Aug 2021 21:54:28 +0000 Received: by mail-qt1-x82e.google.com with SMTP id x9so12700480qtw.13 for ; Mon, 02 Aug 2021 14:54:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=AHbNYGEu8ANwWgRt/VGSQHo8ramO8Lj4ax1qOWu6oGE=; b=fB/9GjSl5IWIY25FBlAD9Nack9VOJ70+mV2Zz8FXYpGvrT/DJGI0Ufc013NrfCn6KO lhHWMU8FOtjFp667JF8WnPsIPJdEVMqHbTQGI8f7X9mRtI6UOBrKryK6A2fedjzx9Sos g9T4m/c0GlLAIBN+oheqT5fJ6OwKMQSYTaM6nRq4+9Vf7L5VVKHOSM93fLz89UQ7AZPs UCpadpHsr/D5spjmogfyaXILjG5w/EAQuG7aE0pzqSA7ZdPj1Eh77LVi0aGbnYoQIvR/ /jKLZ7PUVAL00uIDyK1txp79gA+he22IZLFLO25ZkEmsiJSYmGbGM5ZOGewEkCNSj3Iu q5mw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=AHbNYGEu8ANwWgRt/VGSQHo8ramO8Lj4ax1qOWu6oGE=; b=WyFRiXzFf0BIpLDeaJSOpIFRJsa2HLLn1D6+MmGdhiLOLcpnlG44jspiDfC6+NcxnU lM49txHOqgkiQE+DjApYWcroDscfyowbeCck2TOorr2JVXIdQqOhcX3KZMzg+d9VX1iS UE/9wjGfFTEhv+4ffbhtoc48vqK+sMqh8iar3JRufNguREImJrUYcoy4d75WPwNCQ+EE j5prYzmT5iDI0F2r/Q2nF3Ax3H9+WOJXN2CVpXH2c2kQXToMLkzink9PLInXkt5pGMZu CPuowvNHhhBXd8ztmJ4jpXb5NT5xRPEwdoU0bXLS7JSsYUyfSOlBDX2VecA3XuGrSfGe CIuQ== X-Gm-Message-State: AOAM532fRLYZkJ0/yOHDecyZwBnTohWn02LpdvtdBcVSKBqjhA3sPbf0 KIxQ4axVzSiGLHZO+iXUJx9R5A== X-Google-Smtp-Source: ABdhPJwxuI+nZn2PzMpS045eZgGYMvpto3Fq/atQWFEeicu7c33FJ7whiPSMhp/a8gQmKmKUBEo2xw== X-Received: by 2002:ac8:4986:: with SMTP id f6mr15924652qtq.125.1627941264927; Mon, 02 Aug 2021 14:54:24 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id v11sm5479216qtc.0.2021.08.02.14.54.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Aug 2021 14:54:24 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH v16 10/15] arm64: kexec: use ld script for relocation function Date: Mon, 2 Aug 2021 17:54:03 -0400 Message-Id: <20210802215408.804942-11-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210802215408.804942-1-pasha.tatashin@soleen.com> References: <20210802215408.804942-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210802_145426_584313_271A21B4 X-CRM114-Status: GOOD ( 15.35 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently, relocation code declares start and end variables which are used to compute its size. The better way to do this is to use ld script incited, and put relocation function in its own section. Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/sections.h | 1 + arch/arm64/kernel/machine_kexec.c | 16 ++++++---------- arch/arm64/kernel/relocate_kernel.S | 15 ++------------- arch/arm64/kernel/vmlinux.lds.S | 19 +++++++++++++++++++ 4 files changed, 28 insertions(+), 23 deletions(-) diff --git a/arch/arm64/include/asm/sections.h b/arch/arm64/include/asm/sections.h index e4ad9db53af1..152cb35bf9df 100644 --- a/arch/arm64/include/asm/sections.h +++ b/arch/arm64/include/asm/sections.h @@ -21,5 +21,6 @@ extern char __exittext_begin[], __exittext_end[]; extern char __irqentry_text_start[], __irqentry_text_end[]; extern char __mmuoff_data_start[], __mmuoff_data_end[]; extern char __entry_tramp_text_start[], __entry_tramp_text_end[]; +extern char __relocate_new_kernel_start[], __relocate_new_kernel_end[]; #endif /* __ASM_SECTIONS_H */ diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index cf5d6f22a041..83da6045cd45 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -21,14 +21,11 @@ #include #include #include +#include #include #include "cpu-reset.h" -/* Global variables for the arm64_relocate_new_kernel routine. */ -extern const unsigned char arm64_relocate_new_kernel[]; -extern const unsigned long arm64_relocate_new_kernel_size; - /** * kexec_image_info - For debugging output. */ @@ -163,6 +160,7 @@ static void *kexec_page_alloc(void *arg) int machine_kexec_post_load(struct kimage *kimage) { void *reloc_code = page_to_virt(kimage->control_code_page); + long reloc_size; struct trans_pgd_info info = { .trans_alloc_page = kexec_page_alloc, .trans_alloc_arg = kimage, @@ -183,17 +181,15 @@ int machine_kexec_post_load(struct kimage *kimage) return rc; } - memcpy(reloc_code, arm64_relocate_new_kernel, - arm64_relocate_new_kernel_size); + reloc_size = __relocate_new_kernel_end - __relocate_new_kernel_start; + memcpy(reloc_code, __relocate_new_kernel_start, reloc_size); kimage->arch.kern_reloc = __pa(reloc_code); /* Flush the reloc_code in preparation for its execution. */ dcache_clean_inval_poc((unsigned long)reloc_code, - (unsigned long)reloc_code + - arm64_relocate_new_kernel_size); + (unsigned long)reloc_code + reloc_size); icache_inval_pou((uintptr_t)reloc_code, - (uintptr_t)reloc_code + - arm64_relocate_new_kernel_size); + (uintptr_t)reloc_code + reloc_size); kexec_list_flush(kimage); kexec_image_info(kimage); diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index b4fb97312a80..9d2400855ee4 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -15,6 +15,7 @@ #include #include +.pushsection ".kexec_relocate.text", "ax" /* * arm64_relocate_new_kernel - Put a 2nd stage image in place and boot it. * @@ -77,16 +78,4 @@ SYM_CODE_START(arm64_relocate_new_kernel) mov x3, xzr br x4 /* Jumps from el1 */ SYM_CODE_END(arm64_relocate_new_kernel) - -.align 3 /* To keep the 64-bit values below naturally aligned. */ - -.Lcopy_end: -.org KEXEC_CONTROL_PAGE_SIZE - -/* - * arm64_relocate_new_kernel_size - Number of bytes to copy to the - * control_code_page. - */ -.globl arm64_relocate_new_kernel_size -arm64_relocate_new_kernel_size: - .quad .Lcopy_end - arm64_relocate_new_kernel +.popsection diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index 709d2c433c5e..9479d66f03bf 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -63,6 +63,7 @@ #include #include #include +#include #include #include @@ -100,6 +101,16 @@ jiffies = jiffies_64; #define HIBERNATE_TEXT #endif +#ifdef CONFIG_KEXEC_CORE +#define KEXEC_TEXT \ + . = ALIGN(SZ_4K); \ + __relocate_new_kernel_start = .; \ + *(.kexec_relocate.text) \ + __relocate_new_kernel_end = .; +#else +#define KEXEC_TEXT +#endif + #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 #define TRAMP_TEXT \ . = ALIGN(PAGE_SIZE); \ @@ -160,6 +171,7 @@ SECTIONS HYPERVISOR_TEXT IDMAP_TEXT HIBERNATE_TEXT + KEXEC_TEXT TRAMP_TEXT *(.fixup) *(.gnu.warning) @@ -348,3 +360,10 @@ ASSERT(swapper_pg_dir - reserved_pg_dir == RESERVED_SWAPPER_OFFSET, ASSERT(swapper_pg_dir - tramp_pg_dir == TRAMP_SWAPPER_OFFSET, "TRAMP_SWAPPER_OFFSET is wrong!") #endif + +#ifdef CONFIG_KEXEC_CORE +/* kexec relocation code should fit into one KEXEC_CONTROL_PAGE_SIZE */ +ASSERT(__relocate_new_kernel_end - (__relocate_new_kernel_start & ~(SZ_4K - 1)) + <= SZ_4K, "kexec relocation code is too big or misaligned") +ASSERT(KEXEC_CONTROL_PAGE_SIZE >= SZ_4K, "KEXEC_CONTROL_PAGE_SIZE is brokern") +#endif From patchwork Mon Aug 2 21:54:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12414879 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A8C4AC4338F for ; Mon, 2 Aug 2021 22:03:30 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 65F2060E09 for ; Mon, 2 Aug 2021 22:03:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 65F2060E09 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=UQWkEw2sTXyA/LdbVk8jEPN6tnUDXtY+myAyYEJrBZM=; b=CwRCV3EidoTZgh FOArtwI8JF86p0z5axYkq45iGhYf3fPZ2zk/wIyNScW/DD7l8Ql7RuTidTXKoM8e5bm4SGKIaQJlI sgJOja/5uO36hwQdxTzz2jLZHqMUXDWZZwg4vW3iI78CXxEtldFndXlM8yIdCEoPM+SZys7E1+rjw J1OYttp4nWRuQijXtHdvdp4ZuKmMMimc1YQnTbll+BVfVm2H9LvI81fXwyR9Z3O8pbx0uecqnmaoo xiKdWRH28sRouvHVCwE9Ylqofm6DdP2mUOG0p059tXxMv0wwwzjNiTnKCuWs55D7tszS3FQaKC14B JcSUoteYKwIHkmxunUdA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mAfzp-000N7f-QP; Mon, 02 Aug 2021 22:01:19 +0000 Received: from mail-qv1-xf33.google.com ([2607:f8b0:4864:20::f33]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mAftD-000JsO-K8 for linux-arm-kernel@lists.infradead.org; Mon, 02 Aug 2021 21:54:29 +0000 Received: by mail-qv1-xf33.google.com with SMTP id db14so9629540qvb.10 for ; Mon, 02 Aug 2021 14:54:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=d788AXnqtR7xmaTEEUS9xe8Hn4Sx6OLWF7vdJSotwpM=; b=HNBk5sK5nIpwScV6ruErOFK1CnoHkcV1P3nVWVA+EofJS6HR91CV8HytnoU/ushldA MRjHNHRXIM4W8s+NiJIQDZnDMGJIdVUqv2UB0tAAKFikB0jmrT/clURsgEJWX4geXOdz 2WqPpFsHfJQvDcc4rf7QESPXd3q4gpGbOgVh4NP9veiztWV31qIpsMdqEZSVGsOuzjb0 MtaidV+U/5sKOJec1RaWIhLB2+aAFaf8nKGoldWdrjWtcBz0akMXkEM5ixzEh28shuon dcSJdfb/4ZGjju1DaTmmrePZ0fX08LAvpowX/PF1wjrYL/mhzepoVLWuX0kmU/3dp87J gYtw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=d788AXnqtR7xmaTEEUS9xe8Hn4Sx6OLWF7vdJSotwpM=; b=di8i8ipCHl2qQuRZzAlm4I0zXYjkIQOu/tHrLPUgJwL1+dU9jCeHmEfLtlo9OyV51G +6cuLkwAloyL9gQaTHPQBNkzgNtsB5xBhvGZlXB4Ty3BCb2YnBROIS1/ronvZ34E0W5v WM6UE1T3NBn0LCOO1jTtXBKTizPBtVYdOo7CMegtMBwM5ZWI9YOU98cODtXMQUUm4AwL /9g2jVT3sUKEeyKdPMBXKVc6z26LDrgp/B4VA+yo21Had8HS01NZTZYULUX748O5JAPv GzED6S5sDHvwhBDSvhr/Q4VHOiZFwCxmOluQdkIeb0pxnoZK16adZNYrTUSNLjsMK0s/ 0aYQ== X-Gm-Message-State: AOAM533LJeJ5CPNPEvjiTUT8eE0T1pEkHWCu1Sx4sTaYLRezFarmrWPj 3l8Bc5Fiif+q+76ZUOiiHWpbVA== X-Google-Smtp-Source: ABdhPJyLPL30M5BjjZ4euGJ4jPs9KE5OR6RBHD6k3CLzGd5zxsr9ZO+cRB2kEgv0dgLSMx9tlEvHDw== X-Received: by 2002:a0c:c612:: with SMTP id v18mr18349323qvi.20.1627941266265; Mon, 02 Aug 2021 14:54:26 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id v11sm5479216qtc.0.2021.08.02.14.54.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Aug 2021 14:54:25 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH v16 11/15] arm64: kexec: install a copy of the linear-map Date: Mon, 2 Aug 2021 17:54:04 -0400 Message-Id: <20210802215408.804942-12-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210802215408.804942-1-pasha.tatashin@soleen.com> References: <20210802215408.804942-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210802_145427_785126_B278A9D5 X-CRM114-Status: GOOD ( 21.20 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org To perform the kexec relocation with the MMU enabled, we need a copy of the linear map. Create one, and install it from the relocation code. This has to be done from the assembly code as it will be idmapped with TTBR0. The kernel runs in TTRB1, so can't use the break-before-make sequence on the mapping it is executing from. The makes no difference yet as the relocation code runs with the MMU disabled. Suggested-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/assembler.h | 19 +++++++++++++++++++ arch/arm64/include/asm/kexec.h | 2 ++ arch/arm64/kernel/asm-offsets.c | 2 ++ arch/arm64/kernel/hibernate-asm.S | 20 -------------------- arch/arm64/kernel/machine_kexec.c | 16 ++++++++++++++-- arch/arm64/kernel/relocate_kernel.S | 3 +++ 6 files changed, 40 insertions(+), 22 deletions(-) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 71999a325055..4289c4e1c2a3 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -483,6 +483,25 @@ alternative_endif _cond_extable .Licache_op\@, \fixup .endm +/* + * To prevent the possibility of old and new partial table walks being visible + * in the tlb, switch the ttbr to a zero page when we invalidate the old + * records. D4.7.1 'General TLB maintenance requirements' in ARM DDI 0487A.i + * Even switching to our copied tables will cause a changed output address at + * each stage of the walk. + */ + .macro break_before_make_ttbr_switch zero_page, page_table, tmp, tmp2 + phys_to_ttbr \tmp, \zero_page + msr ttbr1_el1, \tmp + isb + tlbi vmalle1 + dsb nsh + phys_to_ttbr \tmp, \page_table + offset_ttbr1 \tmp, \tmp2 + msr ttbr1_el1, \tmp + isb + .endm + /* * reset_pmuserenr_el0 - reset PMUSERENR_EL0 if PMUv3 present */ diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h index 753a1c398898..d678f0ceb7ee 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -97,6 +97,8 @@ struct kimage_arch { phys_addr_t dtb_mem; phys_addr_t kern_reloc; phys_addr_t el2_vectors; + phys_addr_t ttbr1; + phys_addr_t zero_page; }; #ifdef CONFIG_KEXEC_FILE diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index 47ccb57b787e..a98384d13ee7 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -175,6 +175,8 @@ int main(void) #ifdef CONFIG_KEXEC_CORE DEFINE(KIMAGE_ARCH_DTB_MEM, offsetof(struct kimage, arch.dtb_mem)); DEFINE(KIMAGE_ARCH_EL2_VECTORS, offsetof(struct kimage, arch.el2_vectors)); + DEFINE(KIMAGE_ARCH_ZERO_PAGE, offsetof(struct kimage, arch.zero_page)); + DEFINE(KIMAGE_ARCH_TTBR1, offsetof(struct kimage, arch.ttbr1)); DEFINE(KIMAGE_HEAD, offsetof(struct kimage, head)); DEFINE(KIMAGE_START, offsetof(struct kimage, start)); BLANK(); diff --git a/arch/arm64/kernel/hibernate-asm.S b/arch/arm64/kernel/hibernate-asm.S index a30a2c3f905e..0e1d9c3c6a93 100644 --- a/arch/arm64/kernel/hibernate-asm.S +++ b/arch/arm64/kernel/hibernate-asm.S @@ -15,26 +15,6 @@ #include #include -/* - * To prevent the possibility of old and new partial table walks being visible - * in the tlb, switch the ttbr to a zero page when we invalidate the old - * records. D4.7.1 'General TLB maintenance requirements' in ARM DDI 0487A.i - * Even switching to our copied tables will cause a changed output address at - * each stage of the walk. - */ -.macro break_before_make_ttbr_switch zero_page, page_table, tmp, tmp2 - phys_to_ttbr \tmp, \zero_page - msr ttbr1_el1, \tmp - isb - tlbi vmalle1 - dsb nsh - phys_to_ttbr \tmp, \page_table - offset_ttbr1 \tmp, \tmp2 - msr ttbr1_el1, \tmp - isb -.endm - - /* * Resume from hibernate * diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 83da6045cd45..50bc0a265c86 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -159,6 +159,8 @@ static void *kexec_page_alloc(void *arg) int machine_kexec_post_load(struct kimage *kimage) { + int rc; + pgd_t *trans_pgd; void *reloc_code = page_to_virt(kimage->control_code_page); long reloc_size; struct trans_pgd_info info = { @@ -175,12 +177,22 @@ int machine_kexec_post_load(struct kimage *kimage) kimage->arch.el2_vectors = 0; if (is_hyp_nvhe()) { - int rc = trans_pgd_copy_el2_vectors(&info, - &kimage->arch.el2_vectors); + rc = trans_pgd_copy_el2_vectors(&info, + &kimage->arch.el2_vectors); if (rc) return rc; } + /* Create a copy of the linear map */ + trans_pgd = kexec_page_alloc(kimage); + if (!trans_pgd) + return -ENOMEM; + rc = trans_pgd_create_copy(&info, &trans_pgd, PAGE_OFFSET, PAGE_END); + if (rc) + return rc; + kimage->arch.ttbr1 = __pa(trans_pgd); + kimage->arch.zero_page = __pa(empty_zero_page); + reloc_size = __relocate_new_kernel_end - __relocate_new_kernel_start; memcpy(reloc_code, __relocate_new_kernel_start, reloc_size); kimage->arch.kern_reloc = __pa(reloc_code); diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index 9d2400855ee4..a07b737533c3 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -29,10 +29,13 @@ */ SYM_CODE_START(arm64_relocate_new_kernel) /* Setup the list loop variables. */ + ldr x18, [x0, #KIMAGE_ARCH_ZERO_PAGE] /* x18 = zero page for BBM */ + ldr x17, [x0, #KIMAGE_ARCH_TTBR1] /* x17 = linear map copy */ ldr x16, [x0, #KIMAGE_HEAD] /* x16 = kimage_head */ mov x14, xzr /* x14 = entry ptr */ mov x13, xzr /* x13 = copy dest */ raw_dcache_line_size x15, x1 /* x15 = dcache line size */ + break_before_make_ttbr_switch x18, x17, x1, x2 /* set linear map */ .Lloop: and x12, x16, PAGE_MASK /* x12 = addr */ From patchwork Mon Aug 2 21:54:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12414883 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 69681C4338F for ; Mon, 2 Aug 2021 22:05:40 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2B85B60E09 for ; Mon, 2 Aug 2021 22:05:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 2B85B60E09 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=VWG0f89n6pdJdLWf6fmDdLN9aY9FBEPH1/d1EXIpW0M=; b=v7r3gO26pHMgat BDRWtKi4zMvIbEiWsGi2HjC6U+oLrIEOL2YgEXOt7O9ClVStMMM8B/dB9oOsVeVmq93ll+j1TfxTj EwtSDwbshLKyhd4oyUoxf8iwbWNzDc3gWlInHYCPm29FNmxGL5rmd+bdLGph4+hogqiRBVnQB5HKe Tt5GMxYRXeXtRlHm83yQ6E2nZGRlMdK+5RWgdxwHmQMPqgDsJfG+9qbBzqGv67T9MVc1MOXZrr8Xy QTamp/KbFM1QXS53d/lG0CVY3WUwL88lKH7iWyWaAjqlT29gIdBpsZw4l8kBeOdqVkxD9xbOh717i yEWE69zEvB7CuFxlW5vA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mAg1z-000Nyn-RO; Mon, 02 Aug 2021 22:03:33 +0000 Received: from mail-qv1-xf34.google.com ([2607:f8b0:4864:20::f34]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mAftF-000Jtu-1A for linux-arm-kernel@lists.infradead.org; Mon, 02 Aug 2021 21:54:32 +0000 Received: by mail-qv1-xf34.google.com with SMTP id 3so9650545qvd.2 for ; Mon, 02 Aug 2021 14:54:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=g35GBw9jaSD1qco5H/zP0F6z/ZxFUbIC9x16s2RQRMM=; b=PeCelF4CceooL4R0WLeQssxewvhrDwxcoWv1fuEGyNYV2sBkvR33wwrIrNc9PnAHLz vvUY8stQeC6xrhFT5hckDWGiRHWBvBSsrHpwwUwKqZGE9af8eGnLJV7fxri3ZUbBNL1H 604jWltwg88/FE30M4yJS7FmIjAbUGD/DFN3A3Br3p+VCu+yCldNwBwW/e5jhddMZyVx ICehtRETche1bXF/dQh2c4t83jcUj+VdrEwZ4ShfyJ2n8oNJqBlfVLR4NTMH3PlsRMf1 Bs6KI+zw9fu5VAyIxrVz6HdHhPmwuDrZPKcQ2M6yD7ol6tDU2oxT4Lkl25IZoXNQp5x2 7iXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=g35GBw9jaSD1qco5H/zP0F6z/ZxFUbIC9x16s2RQRMM=; b=l1hvIdiFXEDKyRaaBGgWZc0Z11sI3dzS6sMH9I9CdUXhxQ9Lvk1EKT+7pfp+hEnhhO y3wcNu5Anj9vduwFsjvxEB/xzGSjVz8bFDAK0diIcrhg55De8Rjm+y+8GgIbQyBgIB/O Obmp9ADv/KAUuYUwjLAZv5BxOiyxBbbVYGpTyDf22L0dpuCLOhPJTjwb2pvsNAhcmn5y Oq6xjy6VklbFxyjS1Xvb8SUsuVG2pNxe6xhtIVY4WdhnAcSst1tWsPU3pfjS0kIG5YKf X4ncMywt57s94Wlx0pa0jiQ1v+flYpBGYCBm1Pol1oeTipjJFpVY3JWfM0JWfjANZFwW Pr8w== X-Gm-Message-State: AOAM533B0KBQO3WDU/qicKxQGykpV4bcqbKZWGe9UMx7oqeJTTTo5ya5 mFCtG/pYXZMD4peX9udWPAKAUQ== X-Google-Smtp-Source: ABdhPJyntDUQbHFwmfsNTZMv/NAE180vNe44uz2fj4AQUWv/1Zz1sP5bFhh/XK2SXb92W0BRzVINlA== X-Received: by 2002:a0c:fd61:: with SMTP id k1mr18311601qvs.49.1627941267617; Mon, 02 Aug 2021 14:54:27 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id v11sm5479216qtc.0.2021.08.02.14.54.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Aug 2021 14:54:27 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH v16 12/15] arm64: kexec: keep MMU enabled during kexec relocation Date: Mon, 2 Aug 2021 17:54:05 -0400 Message-Id: <20210802215408.804942-13-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210802215408.804942-1-pasha.tatashin@soleen.com> References: <20210802215408.804942-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210802_145429_113179_C954ACD4 X-CRM114-Status: GOOD ( 18.62 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Now, that we have linear map page tables configured, keep MMU enabled to allow faster relocation of segments to final destination. Cavium ThunderX2: Kernel Image size: 38M Iniramfs size: 46M Total relocation size: 84M MMU-disabled: relocation 7.489539915s MMU-enabled: relocation 0.03946095s Broadcom Stingray: The performance data: for a moderate size kernel + initramfs: 25M the relocation was taking 0.382s, with enabled MMU it now takes 0.019s only or x20 improvement. The time is proportional to the size of relocation, therefore if initramfs is larger, 100M it could take over a second. Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/kexec.h | 3 +++ arch/arm64/kernel/asm-offsets.c | 1 + arch/arm64/kernel/machine_kexec.c | 16 +++++++++++---- arch/arm64/kernel/relocate_kernel.S | 31 +++++++++++++++++++---------- 4 files changed, 36 insertions(+), 15 deletions(-) diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h index d678f0ceb7ee..dca6dedc3b25 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -97,8 +97,11 @@ struct kimage_arch { phys_addr_t dtb_mem; phys_addr_t kern_reloc; phys_addr_t el2_vectors; + phys_addr_t ttbr0; phys_addr_t ttbr1; phys_addr_t zero_page; + unsigned long phys_offset; + unsigned long t0sz; }; #ifdef CONFIG_KEXEC_FILE diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index a98384d13ee7..134af7791249 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -176,6 +176,7 @@ int main(void) DEFINE(KIMAGE_ARCH_DTB_MEM, offsetof(struct kimage, arch.dtb_mem)); DEFINE(KIMAGE_ARCH_EL2_VECTORS, offsetof(struct kimage, arch.el2_vectors)); DEFINE(KIMAGE_ARCH_ZERO_PAGE, offsetof(struct kimage, arch.zero_page)); + DEFINE(KIMAGE_ARCH_PHYS_OFFSET, offsetof(struct kimage, arch.phys_offset)); DEFINE(KIMAGE_ARCH_TTBR1, offsetof(struct kimage, arch.ttbr1)); DEFINE(KIMAGE_HEAD, offsetof(struct kimage, head)); DEFINE(KIMAGE_START, offsetof(struct kimage, start)); diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 50bc0a265c86..b43195a7fcb6 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -196,6 +196,11 @@ int machine_kexec_post_load(struct kimage *kimage) reloc_size = __relocate_new_kernel_end - __relocate_new_kernel_start; memcpy(reloc_code, __relocate_new_kernel_start, reloc_size); kimage->arch.kern_reloc = __pa(reloc_code); + rc = trans_pgd_idmap_page(&info, &kimage->arch.ttbr0, + &kimage->arch.t0sz, reloc_code); + if (rc) + return rc; + kimage->arch.phys_offset = virt_to_phys(kimage) - (long)kimage; /* Flush the reloc_code in preparation for its execution. */ dcache_clean_inval_poc((unsigned long)reloc_code, @@ -230,9 +235,9 @@ void machine_kexec(struct kimage *kimage) local_daif_mask(); /* - * Both restart and cpu_soft_restart will shutdown the MMU, disable data + * Both restart and kernel_reloc will shutdown the MMU, disable data * caches. However, restart will start new kernel or purgatory directly, - * cpu_soft_restart will transfer control to arm64_relocate_new_kernel + * kernel_reloc contains the body of arm64_relocate_new_kernel * In kexec case, kimage->start points to purgatory assuming that * kernel entry and dtb address are embedded in purgatory by * userspace (kexec-tools). @@ -246,10 +251,13 @@ void machine_kexec(struct kimage *kimage) restart(is_hyp_nvhe(), kimage->start, kimage->arch.dtb_mem, 0, 0); } else { + void (*kernel_reloc)(struct kimage *kimage); + if (is_hyp_nvhe()) __hyp_set_vectors(kimage->arch.el2_vectors); - cpu_soft_restart(kimage->arch.kern_reloc, - virt_to_phys(kimage), 0, 0); + cpu_install_ttbr0(kimage->arch.ttbr0, kimage->arch.t0sz); + kernel_reloc = (void *)kimage->arch.kern_reloc; + kernel_reloc(kimage); } BUG(); /* Should never get here. */ diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index a07b737533c3..c81b05ac4650 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -4,6 +4,8 @@ * * Copyright (C) Linaro. * Copyright (C) Huawei Futurewei Technologies. + * Copyright (C) 2020, Microsoft Corporation. + * Pavel Tatashin */ #include @@ -15,6 +17,13 @@ #include #include +.macro turn_off_mmu tmp1, tmp2 + mov_q \tmp1, INIT_SCTLR_EL1_MMU_OFF + pre_disable_mmu_workaround + msr sctlr_el1, \tmp1 + isb +.endm + .pushsection ".kexec_relocate.text", "ax" /* * arm64_relocate_new_kernel - Put a 2nd stage image in place and boot it. @@ -32,22 +41,21 @@ SYM_CODE_START(arm64_relocate_new_kernel) ldr x18, [x0, #KIMAGE_ARCH_ZERO_PAGE] /* x18 = zero page for BBM */ ldr x17, [x0, #KIMAGE_ARCH_TTBR1] /* x17 = linear map copy */ ldr x16, [x0, #KIMAGE_HEAD] /* x16 = kimage_head */ - mov x14, xzr /* x14 = entry ptr */ - mov x13, xzr /* x13 = copy dest */ + ldr x22, [x0, #KIMAGE_ARCH_PHYS_OFFSET] /* x22 phys_offset */ raw_dcache_line_size x15, x1 /* x15 = dcache line size */ break_before_make_ttbr_switch x18, x17, x1, x2 /* set linear map */ .Lloop: and x12, x16, PAGE_MASK /* x12 = addr */ - + sub x12, x12, x22 /* Convert x12 to virt */ /* Test the entry flags. */ .Ltest_source: tbz x16, IND_SOURCE_BIT, .Ltest_indirection /* Invalidate dest page to PoC. */ - mov x2, x13 - add x1, x2, #PAGE_SIZE - dcache_by_myline_op ivac, sy, x2, x1, x15, x20 + mov x19, x13 copy_page x13, x12, x1, x2, x3, x4, x5, x6, x7, x8 + add x1, x19, #PAGE_SIZE + dcache_by_myline_op civac, sy, x19, x1, x15, x20 b .Lnext .Ltest_indirection: tbz x16, IND_INDIRECTION_BIT, .Ltest_destination @@ -64,19 +72,20 @@ SYM_CODE_START(arm64_relocate_new_kernel) ic iallu dsb nsh isb + ldr x4, [x0, #KIMAGE_START] /* relocation start */ + ldr x1, [x0, #KIMAGE_ARCH_EL2_VECTORS] /* relocation start */ + ldr x0, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ + turn_off_mmu x12, x13 /* Start new image. */ - ldr x1, [x0, #KIMAGE_ARCH_EL2_VECTORS] /* relocation start */ cbz x1, .Lel1 - ldr x1, [x0, #KIMAGE_START] /* relocation start */ - ldr x2, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ + mov x1, x4 /* relocation start */ + mov x2, x0 /* dtb address */ mov x3, xzr mov x4, xzr mov x0, #HVC_SOFT_RESTART hvc #0 /* Jumps from el2 */ .Lel1: - ldr x4, [x0, #KIMAGE_START] /* relocation start */ - ldr x0, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ mov x2, xzr mov x3, xzr br x4 /* Jumps from el1 */ From patchwork Mon Aug 2 21:54:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12414881 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BCCEEC4338F for ; Mon, 2 Aug 2021 22:04:31 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 79C8C60E09 for ; Mon, 2 Aug 2021 22:04:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 79C8C60E09 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=wSgLh/mr5FY8ErPQLk+DIyLxa2DVArFS2KmdwCFLvyE=; b=w0Y8IakgUUtXde 5kspQVnLomfE2B+S+Vwmmusr4+UDo6jeWD9l6T0bOX09Yv51dhG14FXSap8l8nu1bChnCDQdkeWwA afHVhb2TeMcIaIU7USVhJxSIQLsOVhq0F1OP02XXyTrVDey/6wyH3X2PCe2az4IdEDrXUNPPhSnKA cKrcHNqLwrOHEXRd+zFyl7v7uDMyEogsLCPOaL4T3A2npny0zL6+0c2zwjBbI98r1o4w3+wlqzh1s 4ptsOjWsMZry6LXT1Du9jjGpL37K5Gfq5VEMG5U4IvrtefQA8WrM6NyDV1srQSSJwIPpazuIdHsja w0Cr+lU2QaxiVwEiLNQQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mAg0v-000NZD-Dj; Mon, 02 Aug 2021 22:02:27 +0000 Received: from mail-qv1-xf2b.google.com ([2607:f8b0:4864:20::f2b]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mAftF-000Jur-Qj for linux-arm-kernel@lists.infradead.org; Mon, 02 Aug 2021 21:54:31 +0000 Received: by mail-qv1-xf2b.google.com with SMTP id f91so9638129qva.9 for ; Mon, 02 Aug 2021 14:54:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=r+CYCZCiU1cMcD+VmU9RhDaYUtg7Su3wxwCheymC23U=; b=jctFebjkj3S+TT2x0nauprSV2wyj0PwRV40R5o4/TRvvfjkGSMl96RZP+Aqp2IFQ5m IR9X6dgSW4nno0xAh35XttKT4va3Hvq6tlueaS40BE+3FKdhcycgnE6xBVMQT7TAZqUE 71O9mr3Wl8Mt6kce6WpT8dOMeSrUaOYwFzLXe3sdZHUGXr6ohZfCVt6qsTEC9SNDWKkm R4+sJuqlCeOZZi5H9J3GsMLXN4keopqd+c5IcdI6xf4XJNoEwSiPEuX0wV5kIrPmvVHC 01mW7aNFawtNFDhbUR4mjsfOOn0j1N9LKv2W8Lkg0DREVeqoFcXJ+gaiBNsjZ5hVnO2K kaHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=r+CYCZCiU1cMcD+VmU9RhDaYUtg7Su3wxwCheymC23U=; b=jR6Zp/Vm4N9WZvbznbh6Ib4z/I65Om6LdOxgdvspBylI7Ag9aZQgdUMLiTYn8pRsoq vYgqLaKvpHWa5DiBMub8e8cC1wPc8qYBwYOp/DAx7e/niakSdUL6ghkgRChYDoC5Rojn 3ARIeWeWc09iEKU+gSWmucbgmEhzCz2/XwtZXsPvIOx9dswFDMoVAaH2BLXfrAbZfxWy WsHm9sYptq65kbOhV61bb1iiyN+RxwD04ODC7nl7+HAb9HAuWU8pD5vTF9JBMF8NVTnX lz8MlfUrNtP0kGL1XqhBLGg6EyeIaEfGs8J26AgG4wnFP41tFzJXrPgBoFgxCXogvBHt 2Bvw== X-Gm-Message-State: AOAM532uJYopCBqwsTdJ7Im6lQw4JMnDhB1zC3Cq8+wuCwd80vgMJb5a wi7zGW+ChQ3r2xKKpsdv+R/uGA== X-Google-Smtp-Source: ABdhPJyd/VxX84pagYf1ZcTV+T5GT+qS17GuV4XbvkFGYFpX9dQIB5Z9oz1yWxX+Lk4ixOmma8alIw== X-Received: by 2002:a0c:c2d3:: with SMTP id c19mr18681667qvi.62.1627941268973; Mon, 02 Aug 2021 14:54:28 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id v11sm5479216qtc.0.2021.08.02.14.54.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Aug 2021 14:54:28 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH v16 13/15] arm64: kexec: remove the pre-kexec PoC maintenance Date: Mon, 2 Aug 2021 17:54:06 -0400 Message-Id: <20210802215408.804942-14-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210802215408.804942-1-pasha.tatashin@soleen.com> References: <20210802215408.804942-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210802_145429_968833_04766D4D X-CRM114-Status: GOOD ( 11.79 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Now that kexec does its relocations with the MMU enabled, we no longer need to clean the relocation data to the PoC. Suggested-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/machine_kexec.c | 43 ------------------------------- 1 file changed, 43 deletions(-) diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index b43195a7fcb6..b1856f34cc51 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -77,48 +77,6 @@ int machine_kexec_prepare(struct kimage *kimage) return 0; } -/** - * kexec_list_flush - Helper to flush the kimage list and source pages to PoC. - */ -static void kexec_list_flush(struct kimage *kimage) -{ - kimage_entry_t *entry; - - dcache_clean_inval_poc((unsigned long)kimage, - (unsigned long)kimage + sizeof(*kimage)); - - for (entry = &kimage->head; ; entry++) { - unsigned int flag; - unsigned long addr; - - /* flush the list entries. */ - dcache_clean_inval_poc((unsigned long)entry, - (unsigned long)entry + - sizeof(kimage_entry_t)); - - flag = *entry & IND_FLAGS; - if (flag == IND_DONE) - break; - - addr = (unsigned long)phys_to_virt(*entry & PAGE_MASK); - - switch (flag) { - case IND_INDIRECTION: - /* Set entry point just before the new list page. */ - entry = (kimage_entry_t *)addr - 1; - break; - case IND_SOURCE: - /* flush the source pages. */ - dcache_clean_inval_poc(addr, addr + PAGE_SIZE); - break; - case IND_DESTINATION: - break; - default: - BUG(); - } - } -} - /** * kexec_segment_flush - Helper to flush the kimage segments to PoC. */ @@ -207,7 +165,6 @@ int machine_kexec_post_load(struct kimage *kimage) (unsigned long)reloc_code + reloc_size); icache_inval_pou((uintptr_t)reloc_code, (uintptr_t)reloc_code + reloc_size); - kexec_list_flush(kimage); kexec_image_info(kimage); return 0; From patchwork Mon Aug 2 21:54:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12414885 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 42BC9C4338F for ; Mon, 2 Aug 2021 22:07:09 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0D6F060F41 for ; Mon, 2 Aug 2021 22:07:09 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 0D6F060F41 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=NZx4lmkP5f/6wkJTcn/IOKht+psWvNeDqeSIYKRnlg0=; b=NO3kXkCPDXxIuk TJE1CppMBKskL2c+Vrxpjj2t04pcJ9e8fChUfbJyu2su6hKJoHxQxoQIvg02//b9HeQJ3q6RFTtiE FXygoqp95SOXl+tfcW79N9tpRWmQ4RI7mm4lvlGncfaEFRhVZy+BDCaiBx6mgSxj3oIxFZFkNovI8 u9r6h7UMus/FmFIfcwutZOuZK5Zk0vY/ySQTEdFKmQM5TopOyBolPrdTijRVGE1G3pO0N/L5tRvc9 NzRUPMvhzPAE6cVHETy4pP/VEjTVPRCWGcDq/vRLRnu1X4KhlpnU/koa6mmpTIFXaXr7aOfTJLB0T JpHGUICSL4MHqEI+2Nug==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mAg3H-000ORz-3M; Mon, 02 Aug 2021 22:04:53 +0000 Received: from mail-qk1-x72e.google.com ([2607:f8b0:4864:20::72e]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mAftH-000Jvi-Jy for linux-arm-kernel@lists.infradead.org; Mon, 02 Aug 2021 21:54:33 +0000 Received: by mail-qk1-x72e.google.com with SMTP id x3so18114783qkl.6 for ; Mon, 02 Aug 2021 14:54:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=O89Po0drPUY9w7yyZAa+dEG9G9+uNuq122i+2KNLn6I=; b=AhhJGOFC0mCf92dgLqLR23/WMp6qpg0D29roEbUTuAhmrsZjB5M8IFHXczuIzmBDPJ 4wg/JT6123cHq8KXyq2mjDiNfi2Ny+l/PrX8tweyH0P2p/n+aTS4EC20NW1zZbTSMkGK d4h4MQU6Ne6Fsr4v0Eu4aq6eoljx1Dj8OdSJ/KVrTrpg0YA7vuDLNygKeFexVNckt6nc uJrDQ7/q9xlRiESlJ0p/8YMYgzvW4jX1ks60Nj29fUiXCdqS+2O+bvuMRkE+E4nlMmGH a5roi/phQ5pvHqDJ2bMyGuM5g8qvx+SxTm7GCeBy/by+vwGuk+itmOn8SwqJBdV1RcTZ 8l+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=O89Po0drPUY9w7yyZAa+dEG9G9+uNuq122i+2KNLn6I=; b=tC1t4VvPMSF5ILf20OIYO5AEjXIHzfaGGbk/C9K4uQRJkkNNaSuLgtr8tKBvBNRWzu 0IBMV8dNWjLOVpxnWVT7EJXwZEvrIlqEyXyBPZm7eO1EZm4rU1PiSQPN4wOF3dPKRrte 1rJfXQZMTUjuJEHOqqfgGBHF7YRrAqgJd4CgavcpfN+LsxTBvP9yOW+NJWkvvSJiElU6 nTOL2J4B0fc48AInTxyeEi4J8ZtDwr7haQI57iNBe4UCnJKoJrghJm8L4cMkc2DNI7oy HxJYcXFyyyVplpP8s6BU8UZ0ycTrcmT3lroDrjNBACaGH/8NvSTIgq8mqvAakJU+Slgu kT5w== X-Gm-Message-State: AOAM531RBw2Jfr1XHyGEZp1oNLSGl4LT79TsUU8eD4aDBie9X0GHD0me AFdUbs8qEEDyYEnaChiQvSXijQ== X-Google-Smtp-Source: ABdhPJwqF/XZ768WVx8wxVwUzI1LAFFuR84TgeimwZlzbZ1HRdjZBNi/fKDE8KSYixSU8NQSGqhwOQ== X-Received: by 2002:a37:a5c4:: with SMTP id o187mr17607427qke.179.1627941270295; Mon, 02 Aug 2021 14:54:30 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id v11sm5479216qtc.0.2021.08.02.14.54.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Aug 2021 14:54:29 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH v16 14/15] arm64: kexec: remove cpu-reset.h Date: Mon, 2 Aug 2021 17:54:07 -0400 Message-Id: <20210802215408.804942-15-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210802215408.804942-1-pasha.tatashin@soleen.com> References: <20210802215408.804942-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210802_145431_683394_D2DDA966 X-CRM114-Status: GOOD ( 15.82 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This header contains only cpu_soft_restart() which is never used directly anymore. So, remove this header, and rename the helper to be cpu_soft_restart(). Suggested-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/kexec.h | 6 ++++++ arch/arm64/kernel/cpu-reset.S | 7 +++---- arch/arm64/kernel/cpu-reset.h | 30 ------------------------------ arch/arm64/kernel/machine_kexec.c | 6 ++---- 4 files changed, 11 insertions(+), 38 deletions(-) delete mode 100644 arch/arm64/kernel/cpu-reset.h diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h index dca6dedc3b25..9839bfc163d7 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -90,6 +90,12 @@ static inline void crash_prepare_suspend(void) {} static inline void crash_post_resume(void) {} #endif +#if defined(CONFIG_KEXEC_CORE) +void cpu_soft_restart(unsigned long el2_switch, unsigned long entry, + unsigned long arg0, unsigned long arg1, + unsigned long arg2); +#endif + #define ARCH_HAS_KIMAGE_ARCH struct kimage_arch { diff --git a/arch/arm64/kernel/cpu-reset.S b/arch/arm64/kernel/cpu-reset.S index d47ff63a5b66..48a8af97faa9 100644 --- a/arch/arm64/kernel/cpu-reset.S +++ b/arch/arm64/kernel/cpu-reset.S @@ -16,8 +16,7 @@ .pushsection .idmap.text, "awx" /* - * __cpu_soft_restart(el2_switch, entry, arg0, arg1, arg2) - Helper for - * cpu_soft_restart. + * cpu_soft_restart(el2_switch, entry, arg0, arg1, arg2) * * @el2_switch: Flag to indicate a switch to EL2 is needed. * @entry: Location to jump to for soft reset. @@ -29,7 +28,7 @@ * branch to what would be the reset vector. It must be executed with the * flat identity mapping. */ -SYM_CODE_START(__cpu_soft_restart) +SYM_CODE_START(cpu_soft_restart) mov_q x12, INIT_SCTLR_EL1_MMU_OFF pre_disable_mmu_workaround /* @@ -48,6 +47,6 @@ SYM_CODE_START(__cpu_soft_restart) mov x1, x3 // arg1 mov x2, x4 // arg2 br x8 -SYM_CODE_END(__cpu_soft_restart) +SYM_CODE_END(cpu_soft_restart) .popsection diff --git a/arch/arm64/kernel/cpu-reset.h b/arch/arm64/kernel/cpu-reset.h deleted file mode 100644 index 296abbac7192..000000000000 --- a/arch/arm64/kernel/cpu-reset.h +++ /dev/null @@ -1,30 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0-only */ -/* - * CPU reset routines - * - * Copyright (C) 2015 Huawei Futurewei Technologies. - */ - -#ifndef _ARM64_CPU_RESET_H -#define _ARM64_CPU_RESET_H - -#include - -void __cpu_soft_restart(unsigned long el2_switch, unsigned long entry, - unsigned long arg0, unsigned long arg1, unsigned long arg2); - -static inline void __noreturn __nocfi cpu_soft_restart(unsigned long entry, - unsigned long arg0, - unsigned long arg1, - unsigned long arg2) -{ - typeof(__cpu_soft_restart) *restart; - - restart = (void *)__pa_symbol(function_nocfi(__cpu_soft_restart)); - - cpu_install_idmap(); - restart(0, entry, arg0, arg1, arg2); - unreachable(); -} - -#endif diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index b1856f34cc51..6f1da3fd5320 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -24,8 +24,6 @@ #include #include -#include "cpu-reset.h" - /** * kexec_image_info - For debugging output. */ @@ -201,10 +199,10 @@ void machine_kexec(struct kimage *kimage) * In kexec_file case, the kernel starts directly without purgatory. */ if (kimage->head & IND_DONE) { - typeof(__cpu_soft_restart) *restart; + typeof(cpu_soft_restart) *restart; cpu_install_idmap(); - restart = (void *)__pa_symbol(function_nocfi(__cpu_soft_restart)); + restart = (void *)__pa_symbol(function_nocfi(cpu_soft_restart)); restart(is_hyp_nvhe(), kimage->start, kimage->arch.dtb_mem, 0, 0); } else { From patchwork Mon Aug 2 21:54:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12414887 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 529A9C4338F for ; Mon, 2 Aug 2021 22:08:52 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 175B060EBC for ; Mon, 2 Aug 2021 22:08:52 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 175B060EBC Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=4Y6rYSc9WVCDuvVHdP/myg2sZ6vLffpS0utuOwj2BIk=; b=LUQ2TlxKaP/kYt 9DCy0dDkb/mtGaJW5Kmm39qtxl1Ik95UTkjw4IOhTZuSaJoYftyHndAjCyuZcLSbgBLhosjJVWQCc 1q3BOpOzoPNLY6yMGiS5OpL61qrHoLWnd32pLnUrI1GaIJAT1cXoN7wWB2M7Uh1F1UwJwFDGSjzTN OzdXSf4Xym80GOect9P4qZ3WPbDwF7ANU5fLvGo9HJAMsp2I0vg453r8JmNWrtf5oQez17b34O/+Y etQsOzMPewlzlA1LhOsQec7MsVYvSdoFXKQxc4+z+GJNiipyxWkASeG56o+cFcvVyUJQvl56fKso2 CeVQKYKVCLWQK6WI6RrA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mAg4y-000P1u-AA; Mon, 02 Aug 2021 22:06:37 +0000 Received: from mail-qk1-x72b.google.com ([2607:f8b0:4864:20::72b]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mAftI-000JwX-J0 for linux-arm-kernel@lists.infradead.org; Mon, 02 Aug 2021 21:54:34 +0000 Received: by mail-qk1-x72b.google.com with SMTP id 184so18152307qkh.1 for ; Mon, 02 Aug 2021 14:54:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=KO+UtZYDZ93vRTmLSDKEJjOCFuS9p62EhZfFf9ovch8=; b=XKau4jwWMiCZa409w4MdI2PXmAxjIEEUEVgVNb4oqyKZvs4pzBFvMRyLou4UBwv+GQ TW+EJsuAd8LEq3UppSwoLKj2w/0TeuiJe4IGEABWnYRmbKKtlk+PE10R+S+6L1hDBKmv wmIiEFWWfvfUwhttS6zbIupZGyaYZbGyF6fOYqANIalWNad/7SCqTkmdzRxrAY+gL2Jy xTv09Un0W96gj6gvwoFBxqm0z+HDNJ88BsgZIOf27WfvUy40ipC/X3AoI0U2/OAuXPXk ZDgZy1N8l724D+q4JmH6H4itZ/vmMpWrskk2LUXOqmfm+DIkDhOpU+EHXoyaGbTObG0h 2U9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KO+UtZYDZ93vRTmLSDKEJjOCFuS9p62EhZfFf9ovch8=; b=N7brWKvMQh3KYKhDx4NaKq6kyH55ti3AtDyGs62TQkKimJh9Lhg1eWKMGxJi8c5Wzi 8lnJAzKOqBtdNjZ/GLmQdOvWvz+ivtxi/FeDUd/KmxwV7krmCVPyAyELl9U22IHx9in5 791k+cuqm3C8PraX4ahxPu5ssTT+ermqThq2u/ZCHR4Zp5mMKIdYYZr/V4G5Z2jCKMXW pPcpO5WufBlHPWr7jiEnAICkzLyIJi0qs/KIOIETZUMxAXxIRMHKnHBM75mZikj5k5qw pQqs4GfZX5/NnTQZM09EwYfZBR55KQKmCV9/vS7xQlYVMaBSzO7RWcPVY3jiqLh0K5Pr VQtw== X-Gm-Message-State: AOAM5306rlkfDkP7ZfLcU9aSjpgBOdav8H4dYlY6CAc5Hko7Ec+jct6+ lzttInNw5FVglXVZYF7nCbYF0g== X-Google-Smtp-Source: ABdhPJyLFm5KeldobIkuT9ocF0wPdPlFXom0qmn5x07S4Ban/u4jSm4NOTGd+dyu0FMWOe62wYnHYw== X-Received: by 2002:a05:620a:1327:: with SMTP id p7mr17704056qkj.483.1627941271676; Mon, 02 Aug 2021 14:54:31 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id v11sm5479216qtc.0.2021.08.02.14.54.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Aug 2021 14:54:31 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH v16 15/15] arm64: trans_pgd: remove trans_pgd_map_page() Date: Mon, 2 Aug 2021 17:54:08 -0400 Message-Id: <20210802215408.804942-16-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210802215408.804942-1-pasha.tatashin@soleen.com> References: <20210802215408.804942-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210802_145432_681685_9F76FE8F X-CRM114-Status: GOOD ( 13.78 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The intend of trans_pgd_map_page() was to map contiguous range of VA memory to the memory that is getting relocated during kexec. However, since we are now using linear map instead of contiguous range this function is not needed Suggested-by: Pingfan Liu Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/trans_pgd.h | 5 +-- arch/arm64/mm/trans_pgd.c | 57 ------------------------------ 2 files changed, 1 insertion(+), 61 deletions(-) diff --git a/arch/arm64/include/asm/trans_pgd.h b/arch/arm64/include/asm/trans_pgd.h index 411323bda7a4..c373d43c07ec 100644 --- a/arch/arm64/include/asm/trans_pgd.h +++ b/arch/arm64/include/asm/trans_pgd.h @@ -15,7 +15,7 @@ /* * trans_alloc_page * - Allocator that should return exactly one zeroed page, if this - * allocator fails, trans_pgd_create_copy() and trans_pgd_map_page() + * allocator fails, trans_pgd_create_copy() and trans_pgd_idmap_page() * return -ENOMEM error. * * trans_alloc_arg @@ -30,9 +30,6 @@ struct trans_pgd_info { int trans_pgd_create_copy(struct trans_pgd_info *info, pgd_t **trans_pgd, unsigned long start, unsigned long end); -int trans_pgd_map_page(struct trans_pgd_info *info, pgd_t *trans_pgd, - void *page, unsigned long dst_addr, pgprot_t pgprot); - int trans_pgd_idmap_page(struct trans_pgd_info *info, phys_addr_t *trans_ttbr0, unsigned long *t0sz, void *page); diff --git a/arch/arm64/mm/trans_pgd.c b/arch/arm64/mm/trans_pgd.c index a08cf313c858..b70588a42c29 100644 --- a/arch/arm64/mm/trans_pgd.c +++ b/arch/arm64/mm/trans_pgd.c @@ -217,63 +217,6 @@ int trans_pgd_create_copy(struct trans_pgd_info *info, pgd_t **dst_pgdp, return rc; } -/* - * Add map entry to trans_pgd for a base-size page at PTE level. - * info: contains allocator and its argument - * trans_pgd: page table in which new map is added. - * page: page to be mapped. - * dst_addr: new VA address for the page - * pgprot: protection for the page. - * - * Returns 0 on success, and -ENOMEM on failure. - */ -int trans_pgd_map_page(struct trans_pgd_info *info, pgd_t *trans_pgd, - void *page, unsigned long dst_addr, pgprot_t pgprot) -{ - pgd_t *pgdp; - p4d_t *p4dp; - pud_t *pudp; - pmd_t *pmdp; - pte_t *ptep; - - pgdp = pgd_offset_pgd(trans_pgd, dst_addr); - if (pgd_none(READ_ONCE(*pgdp))) { - p4dp = trans_alloc(info); - if (!pgdp) - return -ENOMEM; - pgd_populate(NULL, pgdp, p4dp); - } - - p4dp = p4d_offset(pgdp, dst_addr); - if (p4d_none(READ_ONCE(*p4dp))) { - pudp = trans_alloc(info); - if (!pudp) - return -ENOMEM; - p4d_populate(NULL, p4dp, pudp); - } - - pudp = pud_offset(p4dp, dst_addr); - if (pud_none(READ_ONCE(*pudp))) { - pmdp = trans_alloc(info); - if (!pmdp) - return -ENOMEM; - pud_populate(NULL, pudp, pmdp); - } - - pmdp = pmd_offset(pudp, dst_addr); - if (pmd_none(READ_ONCE(*pmdp))) { - ptep = trans_alloc(info); - if (!ptep) - return -ENOMEM; - pmd_populate_kernel(NULL, pmdp, ptep); - } - - ptep = pte_offset_kernel(pmdp, dst_addr); - set_pte(ptep, pfn_pte(virt_to_pfn(page), pgprot)); - - return 0; -} - /* * The page we want to idmap may be outside the range covered by VA_BITS that * can be built using the kernel's p?d_populate() helpers. As a one off, for a