From patchwork Tue Feb 18 16:44:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 11388949 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D6F1513A4 for ; Tue, 18 Feb 2020 16:45:40 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B2BAA208C4 for ; Tue, 18 Feb 2020 16:45:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="JcflcyZB"; dkim=fail reason="signature verification failed" (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="EFJ8So8Y" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B2BAA208C4 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=fS37CM/+7edq0rf63+OtfKUs5Ugd5mYn/IrK386jOfg=; b=JcflcyZB75Sh39bhiOffG2dmhv OeOp9sFKhaFYWiMxo4EqdbyoM94L8yqumA8auPqDxpAgf6Fyns2kwE6woGIx2iX/PBurgWYdDNVxp JvATjx38bTnmnxBNmaO8R/3oCGvGyZK9SSZPfeT5WJl9dWASf6eAqkrvFrYurlWbDEyd5+Nxn4sM3 cX4AN5QOwTVxkeeHOeytXlTcaxmf/EPxkhKB8/zNzdvuOu7Gwse/4GNnR8LNtI/pvwpkSvBAfjsNK nvNj4aQMRzW19PMoxwaJq3J/DtisQt3mcdcoUpXsLOKOljqzv4xLgSM79SgZDjt5P9X1dY6J1WRFv 6PqBkrNg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1j4605-0006kz-Tf; Tue, 18 Feb 2020 16:45:33 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1j45zu-0006cK-Ft for linux-arm-kernel@lists.infradead.org; Tue, 18 Feb 2020 16:45:23 +0000 Received: from e123331-lin.home (amontpellier-657-1-18-247.w109-210.abo.wanadoo.fr [109.210.65.247]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id C63CD22B48; Tue, 18 Feb 2020 16:45:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1582044322; bh=hm+90oUf47SPlQ5U0GH9SzXcGf83AzvmE+6vDU7VQQg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=EFJ8So8YFCpjuTqC6XE7Acup/qcg1SPKdPJj78HPvjVRUgD8LZRFI+AFPjJuhlfma ZnOHOKKA9dFUlQ+lY8uDfTr5L7CiEPdpiCjVR8zYPV/PDCzMOkhDMe9vhY+gRsWdqQ 04AD36HBBEW99KviUT+Cr/r3e7oIky3qHFTIEVls= From: Ard Biesheuvel To: linux-efi@vger.kernel.org Subject: [PATCH 1/2] ARM: decompressor: prepare cache_clean_flush for doing by-VA maintenance Date: Tue, 18 Feb 2020 17:44:29 +0100 Message-Id: <20200218164430.32671-2-ardb@kernel.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200218164430.32671-1-ardb@kernel.org> References: <20200218164430.32671-1-ardb@kernel.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200218_084522_549526_17780895 X-CRM114-Status: GOOD ( 10.72 ) X-Spam-Score: -5.2 (-----) X-Spam-Report: SpamAssassin version 3.4.3 on bombadil.infradead.org summary: Content analysis details: (-5.2 points) pts rule name description ---- ---------------------- -------------------------------------------------- -5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at https://www.dnswl.org/, high trust [198.145.29.99 listed in list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.0 DKIMWL_WL_HIGH DKIMwl.org - Whitelisted High sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Nicolas Pitre , Marc Zyngier , Russell King , Catalin Marinas , Ard Biesheuvel , linux-arm-kernel@lists.infradead.org MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org In preparation of turning the decompressor's cache clean/flush operations into proper by-VA maintenance for v7 cores, pass the start and end addresses of the regions that need cache maintenance into cache_clean_flush in registers r0 and r1. Currently, all implementations of cache_clean_flush ignore these values, so no functional change is expected as a result of this patch. Signed-off-by: Ard Biesheuvel --- arch/arm/boot/compressed/head.S | 28 +++++++++++++++++--- 1 file changed, 25 insertions(+), 3 deletions(-) diff --git a/arch/arm/boot/compressed/head.S b/arch/arm/boot/compressed/head.S index 7b86a2e1acce..935799b92198 100644 --- a/arch/arm/boot/compressed/head.S +++ b/arch/arm/boot/compressed/head.S @@ -525,12 +525,15 @@ dtb_check_done: /* cache_clean_flush may use the stack, so relocate it */ add sp, sp, r6 #endif - + mov r0, r9 + ldr r1, 0f + add r1, r1, r0 bl cache_clean_flush badr r0, restart add r0, r0, r6 mov pc, r0 +0: .long _edata - restart wont_overwrite: /* @@ -622,6 +625,21 @@ not_relocated: mov r0, #0 add r2, sp, #0x10000 @ 64k max mov r3, r7 bl decompress_kernel + + mov r0, r4 @ base of inflated image + adr r1, LC0 @ actual LC0 + ldr r2, [r1] @ linktime LC0 + sub r2, r1, r2 @ LC0 delta + ldr r1, [r1, #16] @ link time inflated size offset + ldr r1, [r1, r2] @ actual inflated size (LE) +#ifdef __ARMEB__ + /* convert to big endian */ + eor r2, r1, r1, ror #16 + bic r2, r2, #0x00ff0000 + mov r1, r1, ror #8 + eor r1, r1, r2, lsr #8 +#endif + add r1, r1, r0 @ end of inflated image bl cache_clean_flush bl cache_off @@ -1439,6 +1457,7 @@ reloc_code_end: #ifdef CONFIG_EFI_STUB .align 2 _start: .long start - . +__edata: .long _edata - . ENTRY(efi_stub_entry) @ allocate space on stack for passing current zImage address @@ -1470,8 +1489,11 @@ ENTRY(efi_stub_entry) .align 2 0: .long start - (. + 4) - @ Preserve return value of efi_entry() in r4 - mov r4, r0 + mov r4, r0 @ preserve DTB pointer + mov r0, r1 @ start of image + adr r2, __edata + ldr r1, [r2] + add r1, r1, r2 @ end of image bl cache_clean_flush bl cache_off From patchwork Tue Feb 18 16:44:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 11388951 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1FA301395 for ; Tue, 18 Feb 2020 16:45:51 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D5A1F208C4 for ; Tue, 18 Feb 2020 16:45:50 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="H7lndM7a"; dkim=fail reason="signature verification failed" (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="b07MY+/r" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D5A1F208C4 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=dM2XBVnmVTLLkwbwXTpBDN5M5aYdPTUX+hQe69itd2U=; b=H7lndM7ag7oeacBsYDn9jieJyU R9c6dWC/FMbRvtPmlLPQAYtyKYYc/fw9OTMzRLlOn2ho8+jHKwd2NZN9yJCE1E79aiaY24P/unYnW YkTiOq6bl016+iMLNVy/g/vxc7TlckIfLI5M+mh64F4+tYZI4NSCosbxqEkQjR32A/+ZepIIKEchh E54kqefLHhTbJjx8l4M3upTJ5rMcM6RvGjVR4mGJqe8TimEDZLALYoIp65voXvX14o5ca8ZMnCdHP MwC0OQkrvTC/VDZ+Z8gewTPuHvLPgW4rcuBHXXePN8iwpLoBLxWs68U2GOCjOoEEvXDOaht3Lj3xU 8UJwFxqg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1j460J-00071z-F2; Tue, 18 Feb 2020 16:45:47 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1j45zw-0006dE-Bn for linux-arm-kernel@lists.infradead.org; Tue, 18 Feb 2020 16:45:26 +0000 Received: from e123331-lin.home (amontpellier-657-1-18-247.w109-210.abo.wanadoo.fr [109.210.65.247]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 9C5F124658; Tue, 18 Feb 2020 16:45:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1582044324; bh=1grcBjHUkpwVZbAnNw+vpuR0VTZNDyp5sMtm5PUWRC4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=b07MY+/rTziRDkZnmlGwytrrX+TpXR0FLm1dUep9vXutmCa3XtPCBPQStopgtMpIZ t4dqBMnliRX+j2StwvD6YYzd8zYO9n1yWK8C7aHxoc2Vfp8RchBl2UNHgsdxjLGny7 IjfF3hJOX0m/rpCNR3SjFLG6rbNFMho6exATvYG4= From: Ard Biesheuvel To: linux-efi@vger.kernel.org Subject: [PATCH 2/2] ARM: decompressor: switch to by-VA cache maintenance for v7 cores Date: Tue, 18 Feb 2020 17:44:30 +0100 Message-Id: <20200218164430.32671-3-ardb@kernel.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200218164430.32671-1-ardb@kernel.org> References: <20200218164430.32671-1-ardb@kernel.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200218_084524_448431_1DA9406E X-CRM114-Status: GOOD ( 13.42 ) X-Spam-Score: -5.2 (-----) X-Spam-Report: SpamAssassin version 3.4.3 on bombadil.infradead.org summary: Content analysis details: (-5.2 points) pts rule name description ---- ---------------------- -------------------------------------------------- -5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at https://www.dnswl.org/, high trust [198.145.29.99 listed in list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.0 DKIMWL_WL_HIGH DKIMwl.org - Whitelisted High sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Nicolas Pitre , Marc Zyngier , Russell King , Catalin Marinas , Ard Biesheuvel , linux-arm-kernel@lists.infradead.org MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org Update the v7 cache_clean_flush routine to take into account the memory range passed in r0/r1, and perform cache maintenance by virtual address on this range instead of set/way maintenance, which is inappropriate for the purpose of maintaining the cached state of memory contents. Signed-off-by: Ard Biesheuvel --- arch/arm/boot/compressed/head.S | 77 ++++++++------------ 1 file changed, 29 insertions(+), 48 deletions(-) diff --git a/arch/arm/boot/compressed/head.S b/arch/arm/boot/compressed/head.S index 935799b92198..df93c9f0a19a 100644 --- a/arch/arm/boot/compressed/head.S +++ b/arch/arm/boot/compressed/head.S @@ -685,6 +685,24 @@ params: ldr r0, =0x10000100 @ params_phys for RPC .align #endif +/* + * dcache_line_size - get the minimum D-cache line size from the CTR register + * on ARMv7. + */ + .macro dcache_line_size, reg, tmp +#ifdef CONFIG_CPU_V7M + movw \tmp, #:lower16:BASEADDR_V7M_SCB + V7M_SCB_CTR + movt \tmp, #:upper16:BASEADDR_V7M_SCB + V7M_SCB_CTR + ldr \tmp, [\tmp] +#else + mrc p15, 0, \tmp, c0, c0, 1 @ read ctr +#endif + lsr \tmp, \tmp, #16 + and \tmp, \tmp, #0xf @ cache line size encoding + mov \reg, #4 @ bytes per word + mov \reg, \reg, lsl \tmp @ actual cache line size + .endm + /* * Turn on the cache. We need to setup some page tables so that we * can have both the I and D caches on. @@ -1177,8 +1195,6 @@ __armv7_mmu_cache_off: bic r0, r0, #0x000c #endif mcr p15, 0, r0, c1, c0 @ turn MMU and cache off - mov r12, lr - bl __armv7_mmu_cache_flush mov r0, #0 #ifdef CONFIG_MMU mcr p15, 0, r0, c8, c7, 0 @ invalidate whole TLB @@ -1186,7 +1202,7 @@ __armv7_mmu_cache_off: mcr p15, 0, r0, c7, c5, 6 @ invalidate BTC mcr p15, 0, r0, c7, c10, 4 @ DSB mcr p15, 0, r0, c7, c5, 4 @ ISB - mov pc, r12 + mov pc, lr /* * Clean and flush the cache to maintain consistency. @@ -1199,6 +1215,7 @@ __armv7_mmu_cache_off: .align 5 cache_clean_flush: mov r3, #16 + mov r11, r1 b call_cache_fn __armv4_mpu_cache_flush: @@ -1249,51 +1266,15 @@ __armv7_mmu_cache_flush: mcr p15, 0, r10, c7, c14, 0 @ clean+invalidate D b iflush hierarchical: - mcr p15, 0, r10, c7, c10, 5 @ DMB - stmfd sp!, {r0-r7, r9-r11} - mrc p15, 1, r0, c0, c0, 1 @ read clidr - ands r3, r0, #0x7000000 @ extract loc from clidr - mov r3, r3, lsr #23 @ left align loc bit field - beq finished @ if loc is 0, then no need to clean - mov r10, #0 @ start clean at cache level 0 -loop1: - add r2, r10, r10, lsr #1 @ work out 3x current cache level - mov r1, r0, lsr r2 @ extract cache type bits from clidr - and r1, r1, #7 @ mask of the bits for current cache only - cmp r1, #2 @ see what cache we have at this level - blt skip @ skip if no cache, or just i-cache - mcr p15, 2, r10, c0, c0, 0 @ select current cache level in cssr - mcr p15, 0, r10, c7, c5, 4 @ isb to sych the new cssr&csidr - mrc p15, 1, r1, c0, c0, 0 @ read the new csidr - and r2, r1, #7 @ extract the length of the cache lines - add r2, r2, #4 @ add 4 (line length offset) - ldr r4, =0x3ff - ands r4, r4, r1, lsr #3 @ find maximum number on the way size - clz r5, r4 @ find bit position of way size increment - ldr r7, =0x7fff - ands r7, r7, r1, lsr #13 @ extract max number of the index size -loop2: - mov r9, r4 @ create working copy of max way size -loop3: - ARM( orr r11, r10, r9, lsl r5 ) @ factor way and cache number into r11 - ARM( orr r11, r11, r7, lsl r2 ) @ factor index number into r11 - THUMB( lsl r6, r9, r5 ) - THUMB( orr r11, r10, r6 ) @ factor way and cache number into r11 - THUMB( lsl r6, r7, r2 ) - THUMB( orr r11, r11, r6 ) @ factor index number into r11 - mcr p15, 0, r11, c7, c14, 2 @ clean & invalidate by set/way - subs r9, r9, #1 @ decrement the way - bge loop3 - subs r7, r7, #1 @ decrement the index - bge loop2 -skip: - add r10, r10, #2 @ increment cache number - cmp r3, r10 - bgt loop1 -finished: - ldmfd sp!, {r0-r7, r9-r11} - mov r10, #0 @ switch back to cache level 0 - mcr p15, 2, r10, c0, c0, 0 @ select current cache level in cssr + dcache_line_size r1, r2 @ r1 := dcache min line size + sub r2, r1, #1 @ r2 := line size mask + bic r0, r0, r2 @ round down start to line size + bic r11, r11, r2 @ round down end to line size +0: cmp r0, r11 @ finished? + bgt iflush + mcr p15, 0, r0, c7, c10, 1 @ clean line at r0 from Dcache + add r0, r0, r1 + b 0b iflush: mcr p15, 0, r10, c7, c10, 4 @ DSB mcr p15, 0, r10, c7, c5, 0 @ invalidate I+BTB