From patchwork Sat Aug 27 15:58:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 12957049 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F27AAC0502A for ; Sat, 27 Aug 2022 16:01:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=L5u72sYOgv92wQnWfiYlvSDMOJfMR2Nei023g1StSpk=; b=X76KextfNmYey3 e4RTvkHkkM005QoE5UODQvtRCMAI/ZS/QqYBc6N3QSfQZ4oRz2paH/vbNPZB7W1BAR4UxPhpA67pz hErVEy3NezjwQgD7CORclKGghfHOeYZObDh1CBtM66BQND4h/O+cNKrv3P4WMFLeIrfnky2/P/+En 8d1rLiaed+6KM3WryPLtpr0ZwXQY/RBFv+M7o1B74MHi6MxChq9q/HT9kPEELbF3NWERx9tgq7Pti CN3M9YV9h8/a0chPCyOb7Cg/qE1X+22wmU7+itFxunu8ks/Gq+nx2jT7imXb/aSLoFxBgX4ma5wEi xcMBB1dmZDHtH2gEjQsA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oRyEE-00CvZH-ME; Sat, 27 Aug 2022 16:00:10 +0000 Received: from ams.source.kernel.org ([2604:1380:4601:e00::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oRyDJ-00Cuf3-ON for linux-arm-kernel@lists.infradead.org; Sat, 27 Aug 2022 15:59:15 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 6E6ECB80958; Sat, 27 Aug 2022 15:59:12 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 87074C4347C; Sat, 27 Aug 2022 15:59:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1661615951; bh=rduofxKv06cjLI775l5wG9qfruh5afXftQCd8hJnRpA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=EBnMsf305e96EibBb7MqOJmdqTgJs+79l/VLq9kQIMnJOQr/b2pzI1PrvsM03oEHc PyvyfS7CqYvwCgAOnxN0ZzJc3O2ozZVS3DgG76xsuB0EBrdDh5u20zLajvO8Z4SLOn tWK0HkzaDB334DXVX98iu2eC165TVUY1sT/+Sl3RVK2Rc7Yje/aoUTsr5iN+B0cbnN UVr+WRanhQGM1WSFBJhbTrmcLE+Xr0oFOQJHN+U7k/xGMKw05imRjisE4YvXdxQR+j 1fmdLidlHAeMZq6OT6EfpWrBRvw4mYHfx6ji6qklL67+Y1XUSKErGupUNCuuHTCbQC 09JQLG8Zjixaw== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, maz@kernel.org, mark.rutland@arm.com, linux-efi@vger.kernel.org, keescook@chromium.org, Ard Biesheuvel Subject: [PATCH v3 4/7] arm64: head: avoid cache invalidation when entering with the MMU on Date: Sat, 27 Aug 2022 17:58:49 +0200 Message-Id: <20220827155852.3338551-5-ardb@kernel.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220827155852.3338551-1-ardb@kernel.org> References: <20220827155852.3338551-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1782; i=ardb@kernel.org; h=from:subject; bh=rduofxKv06cjLI775l5wG9qfruh5afXftQCd8hJnRpA=; b=owEB7QES/pANAwAKAcNPIjmS2Y8kAcsmYgBjCj81T3FObq5Tunuz9GbT4RppS6Hs19oKsFHKEq/c JNCiTomJAbMEAAEKAB0WIQT72WJ8QGnJQhU3VynDTyI5ktmPJAUCYwo/NQAKCRDDTyI5ktmPJFjVC/ 9/RUDvV+gb7ZZFRsSJG2OI7+D1DeKpRGa/19kufY5DIHXtbhENCW3RHCnzvotUz7/q68u7SMwro0sY uaUmI+Yqd1eh4RLpX93IC2BbagXvxlD9dI2c8nu6eodia2jpk3omBrWQPWbH9s+llYkowOuMzLvPG/ gFQ4jlgZRM+wz7t3MixevoDZK15EOfqCOB6ry50XXcbqp78P1mQr0Kf7mbphYNzEuW/w1BjTfLROqI MT2MV2/6d8QEAUrff0f22zkimID+0IVBRaaTTJ9iu1oQm7viBb4HryHt4ocTZhkVRHo/DfuAfd56JQ 9kcRsVO9gkRv9E3OUrn+hoNoTGQ8XYMp25sj+OHqfvBCy0dhZie9M2zffTbeYLG+uOF6W5waWaXu9h hywv6Zg3l6n9e6MUMhejMQZnU14h9FZa3rnHsOcienVO04k2TSbLopC3yAVe6kdLhXqXfDFcitaIGk GEmEDv2tgZiaTBSf2v2VScMTtYA46gpx6WIuHOzy0dj+M= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220827_085914_117146_837DB18F X-CRM114-Status: GOOD ( 12.23 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org If we enter with the MMU on, there is no need for explicit cache invalidation for stores to memory, as they will be coherent with the caches. Let's take advantage of this, and create the ID map with the MMU still enabled if that is how we entered, and avoid any cache invalidation calls in that case. Signed-off-by: Ard Biesheuvel --- arch/arm64/kernel/head.S | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index 8e26f2deb78b..4c5a5692c1e4 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -89,9 +89,9 @@ SYM_CODE_START(primary_entry) bl record_mmu_state bl preserve_boot_args + bl create_idmap bl init_kernel_el // w0=cpu_boot_mode mov x20, x0 - bl create_idmap /* * The following calls CPU setup code, see arch/arm64/mm/proc.S for @@ -133,11 +133,13 @@ SYM_CODE_START_LOCAL(preserve_boot_args) stp x21, x1, [x0] // x0 .. x3 at kernel entry stp x2, x3, [x0, #16] + cbnz x19, 0f // skip cache invalidation if MMU is on dmb sy // needed before dc ivac with // MMU off add x1, x0, #0x20 // 4 x 8 bytes b dcache_inval_poc // tail call +0: ret SYM_CODE_END(preserve_boot_args) SYM_FUNC_START_LOCAL(clear_page_tables) @@ -374,12 +376,13 @@ SYM_FUNC_START_LOCAL(create_idmap) * accesses (MMU disabled), invalidate those tables again to * remove any speculatively loaded cache lines. */ + cbnz x19, 0f // skip cache invalidation if MMU is on dmb sy adrp x0, init_idmap_pg_dir adrp x1, init_idmap_pg_end bl dcache_inval_poc - ret x28 +0: ret x28 SYM_FUNC_END(create_idmap) SYM_FUNC_START_LOCAL(create_kernel_mapping)