From patchwork Thu Oct 20 03:20:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Norov X-Patchwork-Id: 13012587 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 31321C4332F for ; Thu, 20 Oct 2022 03:26:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=1k/TI/wof4aS/3/gubrExo/utBF0WY99oAvyk4aI+ME=; b=H0Zk+xaw7MwnuC WTTmxnZ95khWkVIQzzGce6i6+acnpNbxwhWvvi0nqEtWRao861yOLTXIFCVoPfAJnibbRR4X5kMpL oCB2TP6qnYo/lY3UNHekdUQCoRYPEHtNBjsaI8KbTYfihvCg5eYD6nZIB1quDOIiAib601+ayBsUu 04YACbFZ+R824oJ0eZNQZ4q6GPG9cCQBaPAbeKUiS4QWc0u93YvY5/R3GJsRXfINR+OW6XJex61kL VY/BXnECN4Bzna3q7ALpIj/t9MP5cxgrqu5ZZhRJUc+ws8wj++a5ylBaFwdISxBcY1XwMS1sAge7R eGdKsU/zyZnoGj4HxdLQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1olMBL-009wbC-RI; Thu, 20 Oct 2022 03:25:20 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1olMBJ-009wap-Ui for linux-arm-kernel@bombadil.infradead.org; Thu, 20 Oct 2022 03:25:18 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Transfer-Encoding:MIME-Version :References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=OczjChBmGkUsoBfYt2005Z8T2ggpJvwmMFUd3PcsLKA=; b=bEXzypo6IN1neUzRfGYfdBccc1 NxMImEYEGzwud2Q0hTDp3PnECFt3Oj52m4XKdAPBNuCwpv4+PawPoVZevd+0CvqCyr4f5Hav7YtJa M0Lbmbtwn8gXaAJ2tlGJUVDigoncof7vDvTSTajensvWH0LDFpXN2ziS+1JvVvwNGU26YzXRlKeYe rsK62a9vnSzuL1Lo3r9WE3VjUihx60VAKAHrW7Z6+hCRuEArLDNlayDfqYNcMRIYJugILug1DK1Xk uI5V4kXd5SzGtXb1Yy5alPExBr2AF5Co8GFrC9u9PRRh9Op9nPuHaAFQElg29CQnsdQlR+6yCJCRa xftnvJCg==; Received: from mail-oi1-x22a.google.com ([2607:f8b0:4864:20::22a]) by desiato.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1olMB3-004wOt-LI for linux-arm-kernel@lists.infradead.org; Thu, 20 Oct 2022 03:25:05 +0000 Received: by mail-oi1-x22a.google.com with SMTP id g130so21476636oia.13 for ; Wed, 19 Oct 2022 20:24:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=OczjChBmGkUsoBfYt2005Z8T2ggpJvwmMFUd3PcsLKA=; b=Tc1hvzFMF23lt+BFLnxI7WX4sMpYoLkFxJzxHhJETh1MSfcQuJmAP341e2KQwFTJsJ EmLReN2GxQD4lz92lcUHSQnuNWQr+mBaXao4mrC+eJMAnpohWbG/2WgPmvzOzZWHP9wH 6y5TEKgsLspmLNtSj9KNcWSRSj7Z8e8+O1Mq6mXoC8pfEJhSTLc1v96I+wIo4/qn6tsh Y/p71MWFrA/EB1d1oVFjdb10h2jTvoId9QBa67aCSbWtUNWbJSY/syidxU0LOd2jj0qi Z7ElXW8pg1UF7dTM0vhyEWbpSjd3/yZWoC4ApPo5xsHPCego3IonNbj+cWRTaZVgR0tl 1sxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=OczjChBmGkUsoBfYt2005Z8T2ggpJvwmMFUd3PcsLKA=; b=1ZXYdwCc5smxvlqPMULbdEvX7yOzT92B4yCSDvNT8e4qbgGsnJ0wrsK4c8rr1TJzEW qpWOyfBl6W2pPlfJV5s0kPW1asP+GZN2Cj9CsGxvPTsYv3XKGSxV39v1rmb826tmekzv AbOqIx3awkBgnWJSv4oII//XFpmCKyjYxPwqijlIgTlsLu/KeJH5K4Yk/HVzjGPsfg8u D9TPKtc4XQJcPOmm6VEyD5EGnMoNZOo5m9g8q90MF9NGexeeW+KYGgUaw3c/+9lW4HcI JESvkBSZwlBuQV5YPs+ygH0pApmHRn4kJJNFl01NTRrzJQUMlc2BgbAjCsXMLekFpLK8 iNMw== X-Gm-Message-State: ACrzQf2iwN4L2BQWbv1zuBsav6lYJfcmAIqE018nM11YESiRnvdsZpVl YOEP94DMMrtRV+j2VYNEoKDOTFfYkSo= X-Google-Smtp-Source: AMsMyM4YQSPRdBzk2mRZPiUYN065g9bPt1mlN95Zv1RoRTX0R7OQlWZ+UIEQbpb+X8hjxt3dEjGzog== X-Received: by 2002:aca:df8b:0:b0:350:e9a8:1627 with SMTP id w133-20020acadf8b000000b00350e9a81627mr20314745oig.68.1666236295217; Wed, 19 Oct 2022 20:24:55 -0700 (PDT) Received: from localhost ([12.97.180.36]) by smtp.gmail.com with ESMTPSA id y3-20020a056870418300b0011f400edb17sm8450627oac.4.2022.10.19.20.24.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Oct 2022 20:24:54 -0700 (PDT) From: Yury Norov To: "Russell King (Oracle)" , Catalin Marinas , Mark Rutland , Will Deacon , linux-arm-kernel@lists.infradead.org Cc: Yury Norov , Linux Kernel Mailing List , Alexey Klimov , Andy Shevchenko , Andy Whitcroft , Dennis Zhou , Geert Uytterhoeven , Guenter Roeck , Kees Cook , Linus Torvalds , Rasmus Villemoes Subject: [PATCH 1/2] bitmap: add sanity check function for find_bit() Date: Wed, 19 Oct 2022 20:20:23 -0700 Message-Id: <20221020032024.1804535-2-yury.norov@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221020032024.1804535-1-yury.norov@gmail.com> References: <20221020032024.1804535-1-yury.norov@gmail.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221020_042502_503096_E2385127 X-CRM114-Status: GOOD ( 15.56 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org find_bit() requires a pointer aligned to it's size. However some subsystems (fs, for example) cast char* variables to unsigned long* before passing them to find_bit(). Many architectures allow unaligned pointers with the cost of performance degradation. This patch adds runtime check for the pointers to be aligned. Signed-off-by: Yury Norov --- include/linux/find.h | 35 +++++++++++++++++++++++++++++++++++ lib/Kconfig.debug | 7 +++++++ 2 files changed, 42 insertions(+) diff --git a/include/linux/find.h b/include/linux/find.h index ccaf61a0f5fd..2d8f5419d787 100644 --- a/include/linux/find.h +++ b/include/linux/find.h @@ -7,6 +7,7 @@ #endif #include +#include unsigned long _find_next_bit(const unsigned long *addr1, unsigned long nbits, unsigned long start); @@ -35,6 +36,14 @@ unsigned long _find_next_bit_le(const unsigned long *addr, unsigned long size, unsigned long offset); #endif +static __always_inline +void check_find_bit(const unsigned long *addr) +{ +#ifdef CONFIG_DEBUG_BITMAP + WARN_ON_ONCE(!IS_ALIGNED((unsigned long)addr, sizeof(unsigned long))); +#endif +} + #ifndef find_next_bit /** * find_next_bit - find the next set bit in a memory region @@ -49,6 +58,8 @@ static inline unsigned long find_next_bit(const unsigned long *addr, unsigned long size, unsigned long offset) { + check_find_bit(addr); + if (small_const_nbits(size)) { unsigned long val; @@ -79,6 +90,9 @@ unsigned long find_next_and_bit(const unsigned long *addr1, const unsigned long *addr2, unsigned long size, unsigned long offset) { + check_find_bit(addr1); + check_find_bit(addr2); + if (small_const_nbits(size)) { unsigned long val; @@ -138,6 +152,8 @@ static inline unsigned long find_next_zero_bit(const unsigned long *addr, unsigned long size, unsigned long offset) { + check_find_bit(addr); + if (small_const_nbits(size)) { unsigned long val; @@ -164,6 +180,8 @@ unsigned long find_next_zero_bit(const unsigned long *addr, unsigned long size, static inline unsigned long find_first_bit(const unsigned long *addr, unsigned long size) { + check_find_bit(addr); + if (small_const_nbits(size)) { unsigned long val = *addr & GENMASK(size - 1, 0); @@ -270,6 +288,9 @@ unsigned long find_first_and_bit(const unsigned long *addr1, const unsigned long *addr2, unsigned long size) { + check_find_bit(addr1); + check_find_bit(addr2); + if (small_const_nbits(size)) { unsigned long val = *addr1 & *addr2 & GENMASK(size - 1, 0); @@ -292,6 +313,8 @@ unsigned long find_first_and_bit(const unsigned long *addr1, static inline unsigned long find_first_zero_bit(const unsigned long *addr, unsigned long size) { + check_find_bit(addr); + if (small_const_nbits(size)) { unsigned long val = *addr | ~GENMASK(size - 1, 0); @@ -313,6 +336,8 @@ unsigned long find_first_zero_bit(const unsigned long *addr, unsigned long size) static inline unsigned long find_last_bit(const unsigned long *addr, unsigned long size) { + check_find_bit(addr); + if (small_const_nbits(size)) { unsigned long val = *addr & GENMASK(size - 1, 0); @@ -417,18 +442,24 @@ extern unsigned long find_next_clump8(unsigned long *clump, static inline unsigned long find_next_zero_bit_le(const void *addr, unsigned long size, unsigned long offset) { + check_find_bit(addr); + return find_next_zero_bit(addr, size, offset); } static inline unsigned long find_next_bit_le(const void *addr, unsigned long size, unsigned long offset) { + check_find_bit(addr); + return find_next_bit(addr, size, offset); } static inline unsigned long find_first_zero_bit_le(const void *addr, unsigned long size) { + check_find_bit(addr); + return find_first_zero_bit(addr, size); } @@ -439,6 +470,8 @@ static inline unsigned long find_next_zero_bit_le(const void *addr, unsigned long size, unsigned long offset) { + check_find_bit(addr); + if (small_const_nbits(size)) { unsigned long val = *(const unsigned long *)addr; @@ -472,6 +505,8 @@ static inline unsigned long find_next_bit_le(const void *addr, unsigned long size, unsigned long offset) { + check_find_bit(addr); + if (small_const_nbits(size)) { unsigned long val = *(const unsigned long *)addr; diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index 3fc7abffc7aa..1c7dcd33fc2a 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -543,6 +543,13 @@ endmenu # "Compiler options" menu "Generic Kernel Debugging Instruments" +config DEBUG_BITMAP + bool "Debug bitmaps" + help + Say Y here if you want to check bitmap functions parameters at + the runtime. Enable CONFIG_DEBUG_BITMAP only for debugging because + it may affect performance. + config MAGIC_SYSRQ bool "Magic SysRq key" depends on !UML From patchwork Thu Oct 20 03:20:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Norov X-Patchwork-Id: 13012589 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 86740C433FE for ; Thu, 20 Oct 2022 03:28:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=mS138QwDogxupxZ6XO7w/OYhmxILguWrxkf0kFOKbc4=; b=OwbOV6wafkft7B c6H6FqyojxJzwllmlgJJE6CsWwijnwJlEdTQSBVst7iLlPm+vqEM8UBAUvqnC94WFiCKR+93DibEi E9XBv4TIj8OX6fwK2jNj9XDoYJp8S3gSg5R45Ih/DcyDLNMTNkRExQrDZx+mBlwbBHKipXcpdZFAP 0K2qbYYWgpEmtt+qYb9kW6Ft9hzQxqlEjZlP1vL4pN5ZQZoU7J9VwWB9/8uFvYLSyNiUj8IL+jNFX j3bZIMd2GKBoXTEr+w1XdWWm2oZWPMCurYBzAnYl1I6qa7fHD7gzVotggR5BB3C8SsDgcnpYZV+cb bflt94uj6CIGAI1+NfAw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1olMDF-009wtr-JH; Thu, 20 Oct 2022 03:27:17 +0000 Received: from mail-oi1-x22c.google.com ([2607:f8b0:4864:20::22c]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1olMDC-009wsL-1E for linux-arm-kernel@lists.infradead.org; Thu, 20 Oct 2022 03:27:16 +0000 Received: by mail-oi1-x22c.google.com with SMTP id g130so21480575oia.13 for ; Wed, 19 Oct 2022 20:27:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=LrTrr/AbOZpQ6QHnwNWQS/WlfAPQBvIDtPQnyjJ/xrI=; b=Z2p3UItqZ5oQssgEbNAIUAQcyChFU6Ml9yFLtFvHDFLBras8kWTXQ5dvU7auQrQ+rA 7PIwcACmz41UbdFXofMB5rrPDDEFtZgk9KdMeaJAC9LyGgoJby/sn5m4NUgSH9vFndSn QYwd7Ylrztp+ax2tVGAKKBD1PWZlBE825Y+QCoSIP+cEPfrSqXVZMgvDDw4wiPFjyH4j glCgMR5tkbI/tpImqb5yJWYbeabrIpwQfcpJa8zRYpNl9ob3mmz7HH/D6Gm5DNFK68DI WOsoEECAZ4qpFSyFBGcnDIuXZxOuKPmqvmWRy4ODGcaKVg5tLgznlUbdrnKWaXWBlpw+ kfbw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LrTrr/AbOZpQ6QHnwNWQS/WlfAPQBvIDtPQnyjJ/xrI=; b=gWeStkroXdRppVa4Gx2vY0rvJihAPbvBGj102h3y/Hv4ub3HEOWt51gZftad8lSdDL m3sY8Trp6VPWtNZhy9q7PgPtOTSG5h7YkWAmuRbbDqAkTbsCkFNYrckP03HzwLGi+PTm s94tgATf+Tf1uKuBDxd6P91af6p4zhXEiw0zeX2FCZ3BXgez0bmnb2tmgqAxboimn5dv 1Qr3eW/fX5BEDgXfDkfB2isbYO4Zxo9F/LULOnpREsLMfmpyIXGEZjwso0C79oj3y8+n D5glrH8LmJ57sH0aWwsKZdiW17dg2NWA11/kFCLYcTEgXgsSpSA++dwA3laUIptb37o8 s8yg== X-Gm-Message-State: ACrzQf0TINMPk7SuHkKkdGp3I2Wc80moNJytIfclBMr549uCLKxn5v4D fHYsSFDMxqiZzA2M9MDvCG8= X-Google-Smtp-Source: AMsMyM7eB9Y0BQM2stTC+vPJTidAiabmFHMNHba1bGRtSr0BlTQZA5SaIvTQAZgi0swIcZDHavN1yA== X-Received: by 2002:a05:6808:1587:b0:355:3525:8ed with SMTP id t7-20020a056808158700b00355352508edmr6340031oiw.153.1666236426316; Wed, 19 Oct 2022 20:27:06 -0700 (PDT) Received: from localhost ([12.97.180.36]) by smtp.gmail.com with ESMTPSA id x17-20020a056830245100b006393ea22c1csm7642057otr.16.2022.10.19.20.27.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Oct 2022 20:27:05 -0700 (PDT) From: Yury Norov To: "Russell King (Oracle)" , Catalin Marinas , Mark Rutland , Will Deacon , linux-arm-kernel@lists.infradead.org Cc: Yury Norov , Linux Kernel Mailing List , Alexey Klimov , Andy Shevchenko , Andy Whitcroft , Dennis Zhou , Geert Uytterhoeven , Guenter Roeck , Kees Cook , Linus Torvalds , Rasmus Villemoes Subject: [PATCH 2/2] arm: drop arch implementation for find_bit() functions Date: Wed, 19 Oct 2022 20:20:24 -0700 Message-Id: <20221020032024.1804535-3-yury.norov@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221020032024.1804535-1-yury.norov@gmail.com> References: <20221020032024.1804535-1-yury.norov@gmail.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221019_202714_119317_F3ED4313 X-CRM114-Status: GOOD ( 20.03 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Generic code works on par with arch or better, according to my testing, and with recent improvements merged in v6.1, it should be even faster. ARM already uses many generic find_bit() functions - those that it doesn't implement. So we are talking about migrating a subset of the API; most of find_bit() family has only generic implementation on ARM. This patch switches ARM to generic find_bit() entirely. CC: Guenter Roeck CC: Dennis Zhou CC: Russell King CC: Catalin Marinas CC: linux-arm-kernel@lists.infradead.org Reported-by: Guenter Roeck Suggested-by: Linus Torvalds Signed-off-by: Yury Norov --- arch/arm/include/asm/bitops.h | 63 ----------- arch/arm/kernel/armksyms.c | 11 -- arch/arm/lib/Makefile | 2 +- arch/arm/lib/findbit.S | 193 ---------------------------------- 4 files changed, 1 insertion(+), 268 deletions(-) delete mode 100644 arch/arm/lib/findbit.S diff --git a/arch/arm/include/asm/bitops.h b/arch/arm/include/asm/bitops.h index 714440fa2fc6..23a7cde3422e 100644 --- a/arch/arm/include/asm/bitops.h +++ b/arch/arm/include/asm/bitops.h @@ -157,24 +157,6 @@ extern int _test_and_set_bit(int nr, volatile unsigned long * p); extern int _test_and_clear_bit(int nr, volatile unsigned long * p); extern int _test_and_change_bit(int nr, volatile unsigned long * p); -/* - * Little endian assembly bitops. nr = 0 -> byte 0 bit 0. - */ -unsigned long _find_first_zero_bit_le(const unsigned long *p, unsigned long size); -unsigned long _find_next_zero_bit_le(const unsigned long *p, - unsigned long size, unsigned long offset); -unsigned long _find_first_bit_le(const unsigned long *p, unsigned long size); -unsigned long _find_next_bit_le(const unsigned long *p, unsigned long size, unsigned long offset); - -/* - * Big endian assembly bitops. nr = 0 -> byte 3 bit 0. - */ -unsigned long _find_first_zero_bit_be(const unsigned long *p, unsigned long size); -unsigned long _find_next_zero_bit_be(const unsigned long *p, - unsigned long size, unsigned long offset); -unsigned long _find_first_bit_be(const unsigned long *p, unsigned long size); -unsigned long _find_next_bit_be(const unsigned long *p, unsigned long size, unsigned long offset); - #ifndef CONFIG_SMP /* * The __* form of bitops are non-atomic and may be reordered. @@ -195,26 +177,6 @@ unsigned long _find_next_bit_be(const unsigned long *p, unsigned long size, unsi #define test_and_clear_bit(nr,p) ATOMIC_BITOP(test_and_clear_bit,nr,p) #define test_and_change_bit(nr,p) ATOMIC_BITOP(test_and_change_bit,nr,p) -#ifndef __ARMEB__ -/* - * These are the little endian, atomic definitions. - */ -#define find_first_zero_bit(p,sz) _find_first_zero_bit_le(p,sz) -#define find_next_zero_bit(p,sz,off) _find_next_zero_bit_le(p,sz,off) -#define find_first_bit(p,sz) _find_first_bit_le(p,sz) -#define find_next_bit(p,sz,off) _find_next_bit_le(p,sz,off) - -#else -/* - * These are the big endian, atomic definitions. - */ -#define find_first_zero_bit(p,sz) _find_first_zero_bit_be(p,sz) -#define find_next_zero_bit(p,sz,off) _find_next_zero_bit_be(p,sz,off) -#define find_first_bit(p,sz) _find_first_bit_be(p,sz) -#define find_next_bit(p,sz,off) _find_next_bit_be(p,sz,off) - -#endif - #if __LINUX_ARM_ARCH__ < 5 #include @@ -237,35 +199,10 @@ unsigned long _find_next_bit_be(const unsigned long *p, unsigned long size, unsi #endif #include - #include - #include #include #include - -#ifdef __ARMEB__ - -static inline int find_first_zero_bit_le(const void *p, unsigned size) -{ - return _find_first_zero_bit_le(p, size); -} -#define find_first_zero_bit_le find_first_zero_bit_le - -static inline int find_next_zero_bit_le(const void *p, int size, int offset) -{ - return _find_next_zero_bit_le(p, size, offset); -} -#define find_next_zero_bit_le find_next_zero_bit_le - -static inline int find_next_bit_le(const void *p, int size, int offset) -{ - return _find_next_bit_le(p, size, offset); -} -#define find_next_bit_le find_next_bit_le - -#endif - #include /* diff --git a/arch/arm/kernel/armksyms.c b/arch/arm/kernel/armksyms.c index 82e96ac83684..10130987d388 100644 --- a/arch/arm/kernel/armksyms.c +++ b/arch/arm/kernel/armksyms.c @@ -150,17 +150,6 @@ EXPORT_SYMBOL(_clear_bit); EXPORT_SYMBOL(_test_and_clear_bit); EXPORT_SYMBOL(_change_bit); EXPORT_SYMBOL(_test_and_change_bit); -EXPORT_SYMBOL(_find_first_zero_bit_le); -EXPORT_SYMBOL(_find_next_zero_bit_le); -EXPORT_SYMBOL(_find_first_bit_le); -EXPORT_SYMBOL(_find_next_bit_le); - -#ifdef __ARMEB__ -EXPORT_SYMBOL(_find_first_zero_bit_be); -EXPORT_SYMBOL(_find_next_zero_bit_be); -EXPORT_SYMBOL(_find_first_bit_be); -EXPORT_SYMBOL(_find_next_bit_be); -#endif #ifdef CONFIG_FUNCTION_TRACER EXPORT_SYMBOL(__gnu_mcount_nc); diff --git a/arch/arm/lib/Makefile b/arch/arm/lib/Makefile index 6d2ba454f25b..8b152b1a3014 100644 --- a/arch/arm/lib/Makefile +++ b/arch/arm/lib/Makefile @@ -7,7 +7,7 @@ lib-y := changebit.o csumipv6.o csumpartial.o \ csumpartialcopy.o csumpartialcopyuser.o clearbit.o \ - delay.o delay-loop.o findbit.o memchr.o memcpy.o \ + delay.o delay-loop.o memchr.o memcpy.o \ memmove.o memset.o setbit.o \ strchr.o strrchr.o \ testchangebit.o testclearbit.o testsetbit.o \ diff --git a/arch/arm/lib/findbit.S b/arch/arm/lib/findbit.S deleted file mode 100644 index 7fd3600db8ef..000000000000 --- a/arch/arm/lib/findbit.S +++ /dev/null @@ -1,193 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0-only */ -/* - * linux/arch/arm/lib/findbit.S - * - * Copyright (C) 1995-2000 Russell King - * - * 16th March 2001 - John Ripley - * Fixed so that "size" is an exclusive not an inclusive quantity. - * All users of these functions expect exclusive sizes, and may - * also call with zero size. - * Reworked by rmk. - */ -#include -#include - .text - -/* - * Purpose : Find a 'zero' bit - * Prototype: int find_first_zero_bit(void *addr, unsigned int maxbit); - */ -ENTRY(_find_first_zero_bit_le) - teq r1, #0 - beq 3f - mov r2, #0 -1: - ARM( ldrb r3, [r0, r2, lsr #3] ) - THUMB( lsr r3, r2, #3 ) - THUMB( ldrb r3, [r0, r3] ) - eors r3, r3, #0xff @ invert bits - bne .L_found @ any now set - found zero bit - add r2, r2, #8 @ next bit pointer -2: cmp r2, r1 @ any more? - blo 1b -3: mov r0, r1 @ no free bits - ret lr -ENDPROC(_find_first_zero_bit_le) - -/* - * Purpose : Find next 'zero' bit - * Prototype: int find_next_zero_bit(void *addr, unsigned int maxbit, int offset) - */ -ENTRY(_find_next_zero_bit_le) - cmp r2, r1 - bhs 3b - ands ip, r2, #7 - beq 1b @ If new byte, goto old routine - ARM( ldrb r3, [r0, r2, lsr #3] ) - THUMB( lsr r3, r2, #3 ) - THUMB( ldrb r3, [r0, r3] ) - eor r3, r3, #0xff @ now looking for a 1 bit - movs r3, r3, lsr ip @ shift off unused bits - bne .L_found - orr r2, r2, #7 @ if zero, then no bits here - add r2, r2, #1 @ align bit pointer - b 2b @ loop for next bit -ENDPROC(_find_next_zero_bit_le) - -/* - * Purpose : Find a 'one' bit - * Prototype: int find_first_bit(const unsigned long *addr, unsigned int maxbit); - */ -ENTRY(_find_first_bit_le) - teq r1, #0 - beq 3f - mov r2, #0 -1: - ARM( ldrb r3, [r0, r2, lsr #3] ) - THUMB( lsr r3, r2, #3 ) - THUMB( ldrb r3, [r0, r3] ) - movs r3, r3 - bne .L_found @ any now set - found zero bit - add r2, r2, #8 @ next bit pointer -2: cmp r2, r1 @ any more? - blo 1b -3: mov r0, r1 @ no free bits - ret lr -ENDPROC(_find_first_bit_le) - -/* - * Purpose : Find next 'one' bit - * Prototype: int find_next_zero_bit(void *addr, unsigned int maxbit, int offset) - */ -ENTRY(_find_next_bit_le) - cmp r2, r1 - bhs 3b - ands ip, r2, #7 - beq 1b @ If new byte, goto old routine - ARM( ldrb r3, [r0, r2, lsr #3] ) - THUMB( lsr r3, r2, #3 ) - THUMB( ldrb r3, [r0, r3] ) - movs r3, r3, lsr ip @ shift off unused bits - bne .L_found - orr r2, r2, #7 @ if zero, then no bits here - add r2, r2, #1 @ align bit pointer - b 2b @ loop for next bit -ENDPROC(_find_next_bit_le) - -#ifdef __ARMEB__ - -ENTRY(_find_first_zero_bit_be) - teq r1, #0 - beq 3f - mov r2, #0 -1: eor r3, r2, #0x18 @ big endian byte ordering - ARM( ldrb r3, [r0, r3, lsr #3] ) - THUMB( lsr r3, #3 ) - THUMB( ldrb r3, [r0, r3] ) - eors r3, r3, #0xff @ invert bits - bne .L_found @ any now set - found zero bit - add r2, r2, #8 @ next bit pointer -2: cmp r2, r1 @ any more? - blo 1b -3: mov r0, r1 @ no free bits - ret lr -ENDPROC(_find_first_zero_bit_be) - -ENTRY(_find_next_zero_bit_be) - cmp r2, r1 - bhs 3b - ands ip, r2, #7 - beq 1b @ If new byte, goto old routine - eor r3, r2, #0x18 @ big endian byte ordering - ARM( ldrb r3, [r0, r3, lsr #3] ) - THUMB( lsr r3, #3 ) - THUMB( ldrb r3, [r0, r3] ) - eor r3, r3, #0xff @ now looking for a 1 bit - movs r3, r3, lsr ip @ shift off unused bits - bne .L_found - orr r2, r2, #7 @ if zero, then no bits here - add r2, r2, #1 @ align bit pointer - b 2b @ loop for next bit -ENDPROC(_find_next_zero_bit_be) - -ENTRY(_find_first_bit_be) - teq r1, #0 - beq 3f - mov r2, #0 -1: eor r3, r2, #0x18 @ big endian byte ordering - ARM( ldrb r3, [r0, r3, lsr #3] ) - THUMB( lsr r3, #3 ) - THUMB( ldrb r3, [r0, r3] ) - movs r3, r3 - bne .L_found @ any now set - found zero bit - add r2, r2, #8 @ next bit pointer -2: cmp r2, r1 @ any more? - blo 1b -3: mov r0, r1 @ no free bits - ret lr -ENDPROC(_find_first_bit_be) - -ENTRY(_find_next_bit_be) - cmp r2, r1 - bhs 3b - ands ip, r2, #7 - beq 1b @ If new byte, goto old routine - eor r3, r2, #0x18 @ big endian byte ordering - ARM( ldrb r3, [r0, r3, lsr #3] ) - THUMB( lsr r3, #3 ) - THUMB( ldrb r3, [r0, r3] ) - movs r3, r3, lsr ip @ shift off unused bits - bne .L_found - orr r2, r2, #7 @ if zero, then no bits here - add r2, r2, #1 @ align bit pointer - b 2b @ loop for next bit -ENDPROC(_find_next_bit_be) - -#endif - -/* - * One or more bits in the LSB of r3 are assumed to be set. - */ -.L_found: -#if __LINUX_ARM_ARCH__ >= 5 - rsb r0, r3, #0 - and r3, r3, r0 - clz r3, r3 - rsb r3, r3, #31 - add r0, r2, r3 -#else - tst r3, #0x0f - addeq r2, r2, #4 - movne r3, r3, lsl #4 - tst r3, #0x30 - addeq r2, r2, #2 - movne r3, r3, lsl #2 - tst r3, #0x40 - addeq r2, r2, #1 - mov r0, r2 -#endif - cmp r1, r0 @ Clamp to maxbit - movlo r0, r1 - ret lr -