From patchwork Tue Jul 26 15:44:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Norov X-Patchwork-Id: 12929434 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 32226C00140 for ; Tue, 26 Jul 2022 15:45:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:Cc :To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=lIiUZIR8Qz9XINxlBcFbwsFfjg5eHQCFjqsf0yiFMw8=; b=bgQ18NdNd3e4Mc uwc8A9gAHMxPQQHgLuUJw//t5EU7QOIUoZZgLdn6px8NeRpw5Yrk6/BE6frvXXbTfqcMHiq5Fo5cK oUZAkPpVe7VDfQ8k0VqTsyGqVX2EmO4zWzoTQBymSF3o/TMVdHL0z5bQIs8Qct+AqwpZV7qWJt+9K LdVfNk9srgyeapDgmEAxiH3s4eEFC/EBxrUc83VmSA2QZvu/4pfMQK/1VB+yiy48ufUVod6tlMwzc E6YkIWc2pZ3DMowM4mYksIwiBcLOMhFJIj9HhdiUNhOxbLwCaDGAVKDmFZ/S0OGNwowFTFtKkCL4P A6ROxo+xpgo30OhTX2cA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oGMjI-000hU7-5q; Tue, 26 Jul 2022 15:44:16 +0000 Received: from mail-qv1-xf34.google.com ([2607:f8b0:4864:20::f34]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oGMjE-000hR7-9w for linux-arm-kernel@lists.infradead.org; Tue, 26 Jul 2022 15:44:14 +0000 Received: by mail-qv1-xf34.google.com with SMTP id i4so10900072qvv.7 for ; Tue, 26 Jul 2022 08:44:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=BexC9/AbcP3o6Ymn2Ewp4h96F58LBV0iCsv1+O+Xgow=; b=U0lbfOUC9m2barn1E18hRc+8hedwwhAsqBjx+mMnkrx9FyA0Z4MVn3nnG8/5QGKcBT EjvrPB3CbDZsS3BZLWxXVzHg1972G/j67E7SMZ+Xkr8lFSiYopvbXwLrFTzKus1hY6bG KZB+xAw7juawSmsQW2hrT41amILq5uEYTXrM80QNSQtiFkT8yAaQQApYSIgUZ7+De3s0 3ws/gaD85sLboongTpBUcTG+aQb8LOKp16LdDkIV8TCljAXPoVhps4wsYu3iMoO1ZMbI Ku+bzeACs3cna3SWd3C8O/13muHtQLIaD1l2tkpr4+5VUd0cseHV31AahPbdQGmQKsXA 1u2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=BexC9/AbcP3o6Ymn2Ewp4h96F58LBV0iCsv1+O+Xgow=; b=jchgmgt5p4XdOp5Pz8NCzCBiUDozkD3YuF6CzsAzl0EbsinvF/jzuWeZJHksl/tUg2 1HZWZaYW3HGHsc/FRBXuK2YEO6F2QpBM7NIK4CnziNiUsnn/AiWBjiC56dFEdoTs2BgQ EQLof9y1km/iK7WlveEKL/M2ayehUX1D7fbAuxjzbLfpj2l3ABGTRLG2h9+NpmevPfaE dRZa3E2L2Swe9bWcjdEn9YqkpH435DTdq4MNh95HkqS0fq/fh4RB004kb3BrtRqmceZr tMUmPy2WTTdONNsDGEdFSwfcvniK0pn4CkSqAbCvZBjW2H6J5Xm96ol+P+5ipV2vFOSj cHqg== X-Gm-Message-State: AJIora9pPhSf784md6HeFzXJcfpQMFVD7fsRFHZ+DafXP/LY04bIJ0/u GJM083J4rMdZvGHaud90WUU= X-Google-Smtp-Source: AGRyM1thh9Gw+loukSHoxH3TzUo0DXR1Sx7xP1j9+jZiKzPT7hvUmO35dSBTwlTZOrL1XtvWOXcc3w== X-Received: by 2002:a05:6214:20a9:b0:473:6e47:956 with SMTP id 9-20020a05621420a900b004736e470956mr15372560qvd.31.1658850249319; Tue, 26 Jul 2022 08:44:09 -0700 (PDT) Received: from localhost (c-69-254-185-160.hsd1.fl.comcast.net. [69.254.185.160]) by smtp.gmail.com with ESMTPSA id q30-20020a37f71e000000b006a79d8c8198sm10861396qkj.135.2022.07.26.08.44.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Jul 2022 08:44:08 -0700 (PDT) From: Yury Norov To: linux-kernel@vger.kernel.org, Linus Torvalds Cc: Yury Norov , Guenter Roeck , Dennis Zhou , Russell King , Catalin Marinas , linux-arm-kernel@lists.infradead.org Subject: [PATCH] arm: drop arch implementation for find_bit() functions Date: Tue, 26 Jul 2022 08:44:07 -0700 Message-Id: <20220726154407.3464776-1-yury.norov@gmail.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220726_084412_393780_1B1A1411 X-CRM114-Status: GOOD ( 17.46 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org find_next_bit(bitmap, nbits, off) shouldn't touch memory if nbits == 0 or off >= nbits to avoid out-of-boundary access. Generic implementation has explicit check for this, but arm doesn't, which is spotted by KFENCE when running test_bitmap. Instead of fixing arm implementation, this patch switches arm to use generic code. It's better optimized for small bitmaps with small_const_nbits(), and for long bitmaps too because generic code fetches words with LDR, while arch code fetches bytes with LDRB. The KFENCE report: BUG: KFENCE: out-of-bounds read in _find_next_bit_le (arch/arm/lib/findbit.S:88) Out-of-bounds read at 0xef59e000 (4096B right of kfence-#93): _find_next_bit_le (arch/arm/lib/findbit.S:88) kfence-#93: 0xef59d000-0xef59dfff, size=4096, cache=kmalloc-4k allocated by task 1 on cpu 1 at 18.432911s: test_bitmap_printlist (./include/linux/slab.h:600 lib/test_bitmap.c:452) test_bitmap_init (lib/test_bitmap.c:883 lib/test_bitmap.c:889) do_one_initcall (./include/linux/jump_label.h:261 ./include/linux/jump_label.h:271 ./include/trace/events/initcall.h:48 init/main.c:1296) kernel_init_freeable (init/main.c:1367 init/main.c:1384 init/main.c:1403 init/main.c:1610) kernel_init (init/main.c:1501) ret_from_fork (arch/arm/kernel/entry-common.S:149) 0x0 CPU: 1 PID: 1 Comm: swapper/0 Not tainted 5.19.0-rc8 #1 Hardware name: Samsung Exynos (Flattened Device Tree) PC is at _find_next_bit_le (arch/arm/lib/findbit.S:88) LR is at bitmap_list_string.constprop.0 (lib/vsprintf.c:1246) pc : lr : psr: 20000113 sp : f082dc70 ip : 00000001 fp : 00000001 r10: 00000000 r9 : 0000002d r8 : ef59d000 r7 : c0e55514 r6 : c2215000 r5 : 00008000 r4 : 00008000 r3 : 845cac12 r2 : 00008001 r1 : 00008000 r0 : ef59d000 Flags: nzCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment none Control: 10c5387d Table: 4000406a DAC: 00000051 CPU: 1 PID: 1 Comm: swapper/0 Not tainted 5.19.0-rc8 #1 Hardware name: Samsung Exynos (Flattened Device Tree) unwind_backtrace from show_stack (arch/arm/kernel/traps.c:255) show_stack from dump_stack_lvl (lib/dump_stack.c:107) dump_stack_lvl from kfence_report_error (mm/kfence/report.c:262) kfence_report_error from kfence_handle_page_fault (mm/kfence/core.c:1151) kfence_handle_page_fault from __do_kernel_fault.part.0 (arch/arm/mm/fault.c:143) __do_kernel_fault.part.0 from do_page_fault (arch/arm/mm/fault.c:380) do_page_fault from do_DataAbort (arch/arm/mm/fault.c:539) do_DataAbort from __dabt_svc (arch/arm/kernel/entry-armv.S:214) Exception stack(0xf082dc20 to 0xf082dc68) dc20: ef59d000 00008000 00008001 845cac12 00008000 00008000 c2215000 c0e55514 dc40: ef59d000 0000002d 00000000 00000001 00000001 f082dc70 c0715930 c06ff18c dc60: 20000113 ffffffff __dabt_svc from _find_next_bit_le (arch/arm/lib/findbit.S:88) CC: Guenter Roeck CC: Dennis Zhou CC: Russell King CC: Catalin Marinas CC: linux-arm-kernel@lists.infradead.org Reported-by: Guenter Roeck Suggested-by: Linus Torvalds Signed-off-by: Yury Norov Tested-by: Guenter Roeck --- arch/arm/include/asm/bitops.h | 61 ----------- arch/arm/kernel/armksyms.c | 11 -- arch/arm/lib/Makefile | 2 +- arch/arm/lib/findbit.S | 193 ---------------------------------- 4 files changed, 1 insertion(+), 266 deletions(-) delete mode 100644 arch/arm/lib/findbit.S diff --git a/arch/arm/include/asm/bitops.h b/arch/arm/include/asm/bitops.h index 8e94fe7ab5eb..23a7cde3422e 100644 --- a/arch/arm/include/asm/bitops.h +++ b/arch/arm/include/asm/bitops.h @@ -157,22 +157,6 @@ extern int _test_and_set_bit(int nr, volatile unsigned long * p); extern int _test_and_clear_bit(int nr, volatile unsigned long * p); extern int _test_and_change_bit(int nr, volatile unsigned long * p); -/* - * Little endian assembly bitops. nr = 0 -> byte 0 bit 0. - */ -extern int _find_first_zero_bit_le(const unsigned long *p, unsigned size); -extern int _find_next_zero_bit_le(const unsigned long *p, int size, int offset); -extern int _find_first_bit_le(const unsigned long *p, unsigned size); -extern int _find_next_bit_le(const unsigned long *p, int size, int offset); - -/* - * Big endian assembly bitops. nr = 0 -> byte 3 bit 0. - */ -extern int _find_first_zero_bit_be(const unsigned long *p, unsigned size); -extern int _find_next_zero_bit_be(const unsigned long *p, int size, int offset); -extern int _find_first_bit_be(const unsigned long *p, unsigned size); -extern int _find_next_bit_be(const unsigned long *p, int size, int offset); - #ifndef CONFIG_SMP /* * The __* form of bitops are non-atomic and may be reordered. @@ -193,26 +177,6 @@ extern int _find_next_bit_be(const unsigned long *p, int size, int offset); #define test_and_clear_bit(nr,p) ATOMIC_BITOP(test_and_clear_bit,nr,p) #define test_and_change_bit(nr,p) ATOMIC_BITOP(test_and_change_bit,nr,p) -#ifndef __ARMEB__ -/* - * These are the little endian, atomic definitions. - */ -#define find_first_zero_bit(p,sz) _find_first_zero_bit_le(p,sz) -#define find_next_zero_bit(p,sz,off) _find_next_zero_bit_le(p,sz,off) -#define find_first_bit(p,sz) _find_first_bit_le(p,sz) -#define find_next_bit(p,sz,off) _find_next_bit_le(p,sz,off) - -#else -/* - * These are the big endian, atomic definitions. - */ -#define find_first_zero_bit(p,sz) _find_first_zero_bit_be(p,sz) -#define find_next_zero_bit(p,sz,off) _find_next_zero_bit_be(p,sz,off) -#define find_first_bit(p,sz) _find_first_bit_be(p,sz) -#define find_next_bit(p,sz,off) _find_next_bit_be(p,sz,off) - -#endif - #if __LINUX_ARM_ARCH__ < 5 #include @@ -235,35 +199,10 @@ extern int _find_next_bit_be(const unsigned long *p, int size, int offset); #endif #include - #include - #include #include #include - -#ifdef __ARMEB__ - -static inline int find_first_zero_bit_le(const void *p, unsigned size) -{ - return _find_first_zero_bit_le(p, size); -} -#define find_first_zero_bit_le find_first_zero_bit_le - -static inline int find_next_zero_bit_le(const void *p, int size, int offset) -{ - return _find_next_zero_bit_le(p, size, offset); -} -#define find_next_zero_bit_le find_next_zero_bit_le - -static inline int find_next_bit_le(const void *p, int size, int offset) -{ - return _find_next_bit_le(p, size, offset); -} -#define find_next_bit_le find_next_bit_le - -#endif - #include /* diff --git a/arch/arm/kernel/armksyms.c b/arch/arm/kernel/armksyms.c index 82e96ac83684..10130987d388 100644 --- a/arch/arm/kernel/armksyms.c +++ b/arch/arm/kernel/armksyms.c @@ -150,17 +150,6 @@ EXPORT_SYMBOL(_clear_bit); EXPORT_SYMBOL(_test_and_clear_bit); EXPORT_SYMBOL(_change_bit); EXPORT_SYMBOL(_test_and_change_bit); -EXPORT_SYMBOL(_find_first_zero_bit_le); -EXPORT_SYMBOL(_find_next_zero_bit_le); -EXPORT_SYMBOL(_find_first_bit_le); -EXPORT_SYMBOL(_find_next_bit_le); - -#ifdef __ARMEB__ -EXPORT_SYMBOL(_find_first_zero_bit_be); -EXPORT_SYMBOL(_find_next_zero_bit_be); -EXPORT_SYMBOL(_find_first_bit_be); -EXPORT_SYMBOL(_find_next_bit_be); -#endif #ifdef CONFIG_FUNCTION_TRACER EXPORT_SYMBOL(__gnu_mcount_nc); diff --git a/arch/arm/lib/Makefile b/arch/arm/lib/Makefile index 6d2ba454f25b..8b152b1a3014 100644 --- a/arch/arm/lib/Makefile +++ b/arch/arm/lib/Makefile @@ -7,7 +7,7 @@ lib-y := changebit.o csumipv6.o csumpartial.o \ csumpartialcopy.o csumpartialcopyuser.o clearbit.o \ - delay.o delay-loop.o findbit.o memchr.o memcpy.o \ + delay.o delay-loop.o memchr.o memcpy.o \ memmove.o memset.o setbit.o \ strchr.o strrchr.o \ testchangebit.o testclearbit.o testsetbit.o \ diff --git a/arch/arm/lib/findbit.S b/arch/arm/lib/findbit.S deleted file mode 100644 index b5e8b9ae4c7d..000000000000 --- a/arch/arm/lib/findbit.S +++ /dev/null @@ -1,193 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0-only */ -/* - * linux/arch/arm/lib/findbit.S - * - * Copyright (C) 1995-2000 Russell King - * - * 16th March 2001 - John Ripley - * Fixed so that "size" is an exclusive not an inclusive quantity. - * All users of these functions expect exclusive sizes, and may - * also call with zero size. - * Reworked by rmk. - */ -#include -#include - .text - -/* - * Purpose : Find a 'zero' bit - * Prototype: int find_first_zero_bit(void *addr, unsigned int maxbit); - */ -ENTRY(_find_first_zero_bit_le) - teq r1, #0 - beq 3f - mov r2, #0 -1: - ARM( ldrb r3, [r0, r2, lsr #3] ) - THUMB( lsr r3, r2, #3 ) - THUMB( ldrb r3, [r0, r3] ) - eors r3, r3, #0xff @ invert bits - bne .L_found @ any now set - found zero bit - add r2, r2, #8 @ next bit pointer -2: cmp r2, r1 @ any more? - blo 1b -3: mov r0, r1 @ no free bits - ret lr -ENDPROC(_find_first_zero_bit_le) - -/* - * Purpose : Find next 'zero' bit - * Prototype: int find_next_zero_bit(void *addr, unsigned int maxbit, int offset) - */ -ENTRY(_find_next_zero_bit_le) - teq r1, #0 - beq 3b - ands ip, r2, #7 - beq 1b @ If new byte, goto old routine - ARM( ldrb r3, [r0, r2, lsr #3] ) - THUMB( lsr r3, r2, #3 ) - THUMB( ldrb r3, [r0, r3] ) - eor r3, r3, #0xff @ now looking for a 1 bit - movs r3, r3, lsr ip @ shift off unused bits - bne .L_found - orr r2, r2, #7 @ if zero, then no bits here - add r2, r2, #1 @ align bit pointer - b 2b @ loop for next bit -ENDPROC(_find_next_zero_bit_le) - -/* - * Purpose : Find a 'one' bit - * Prototype: int find_first_bit(const unsigned long *addr, unsigned int maxbit); - */ -ENTRY(_find_first_bit_le) - teq r1, #0 - beq 3f - mov r2, #0 -1: - ARM( ldrb r3, [r0, r2, lsr #3] ) - THUMB( lsr r3, r2, #3 ) - THUMB( ldrb r3, [r0, r3] ) - movs r3, r3 - bne .L_found @ any now set - found zero bit - add r2, r2, #8 @ next bit pointer -2: cmp r2, r1 @ any more? - blo 1b -3: mov r0, r1 @ no free bits - ret lr -ENDPROC(_find_first_bit_le) - -/* - * Purpose : Find next 'one' bit - * Prototype: int find_next_zero_bit(void *addr, unsigned int maxbit, int offset) - */ -ENTRY(_find_next_bit_le) - teq r1, #0 - beq 3b - ands ip, r2, #7 - beq 1b @ If new byte, goto old routine - ARM( ldrb r3, [r0, r2, lsr #3] ) - THUMB( lsr r3, r2, #3 ) - THUMB( ldrb r3, [r0, r3] ) - movs r3, r3, lsr ip @ shift off unused bits - bne .L_found - orr r2, r2, #7 @ if zero, then no bits here - add r2, r2, #1 @ align bit pointer - b 2b @ loop for next bit -ENDPROC(_find_next_bit_le) - -#ifdef __ARMEB__ - -ENTRY(_find_first_zero_bit_be) - teq r1, #0 - beq 3f - mov r2, #0 -1: eor r3, r2, #0x18 @ big endian byte ordering - ARM( ldrb r3, [r0, r3, lsr #3] ) - THUMB( lsr r3, #3 ) - THUMB( ldrb r3, [r0, r3] ) - eors r3, r3, #0xff @ invert bits - bne .L_found @ any now set - found zero bit - add r2, r2, #8 @ next bit pointer -2: cmp r2, r1 @ any more? - blo 1b -3: mov r0, r1 @ no free bits - ret lr -ENDPROC(_find_first_zero_bit_be) - -ENTRY(_find_next_zero_bit_be) - teq r1, #0 - beq 3b - ands ip, r2, #7 - beq 1b @ If new byte, goto old routine - eor r3, r2, #0x18 @ big endian byte ordering - ARM( ldrb r3, [r0, r3, lsr #3] ) - THUMB( lsr r3, #3 ) - THUMB( ldrb r3, [r0, r3] ) - eor r3, r3, #0xff @ now looking for a 1 bit - movs r3, r3, lsr ip @ shift off unused bits - bne .L_found - orr r2, r2, #7 @ if zero, then no bits here - add r2, r2, #1 @ align bit pointer - b 2b @ loop for next bit -ENDPROC(_find_next_zero_bit_be) - -ENTRY(_find_first_bit_be) - teq r1, #0 - beq 3f - mov r2, #0 -1: eor r3, r2, #0x18 @ big endian byte ordering - ARM( ldrb r3, [r0, r3, lsr #3] ) - THUMB( lsr r3, #3 ) - THUMB( ldrb r3, [r0, r3] ) - movs r3, r3 - bne .L_found @ any now set - found zero bit - add r2, r2, #8 @ next bit pointer -2: cmp r2, r1 @ any more? - blo 1b -3: mov r0, r1 @ no free bits - ret lr -ENDPROC(_find_first_bit_be) - -ENTRY(_find_next_bit_be) - teq r1, #0 - beq 3b - ands ip, r2, #7 - beq 1b @ If new byte, goto old routine - eor r3, r2, #0x18 @ big endian byte ordering - ARM( ldrb r3, [r0, r3, lsr #3] ) - THUMB( lsr r3, #3 ) - THUMB( ldrb r3, [r0, r3] ) - movs r3, r3, lsr ip @ shift off unused bits - bne .L_found - orr r2, r2, #7 @ if zero, then no bits here - add r2, r2, #1 @ align bit pointer - b 2b @ loop for next bit -ENDPROC(_find_next_bit_be) - -#endif - -/* - * One or more bits in the LSB of r3 are assumed to be set. - */ -.L_found: -#if __LINUX_ARM_ARCH__ >= 5 - rsb r0, r3, #0 - and r3, r3, r0 - clz r3, r3 - rsb r3, r3, #31 - add r0, r2, r3 -#else - tst r3, #0x0f - addeq r2, r2, #4 - movne r3, r3, lsl #4 - tst r3, #0x30 - addeq r2, r2, #2 - movne r3, r3, lsl #2 - tst r3, #0x40 - addeq r2, r2, #1 - mov r0, r2 -#endif - cmp r1, r0 @ Clamp to maxbit - movlo r0, r1 - ret lr -