From patchwork Fri Nov 10 20:41:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13452780 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 204CBC4332F for ; Fri, 10 Nov 2023 20:41:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AD1ED8D00FA; Fri, 10 Nov 2023 15:41:41 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A83568D0005; Fri, 10 Nov 2023 15:41:41 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 886B48D00FA; Fri, 10 Nov 2023 15:41:41 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 786218D0005 for ; Fri, 10 Nov 2023 15:41:41 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 4443B1A048E for ; Fri, 10 Nov 2023 20:41:41 +0000 (UTC) X-FDA: 81443215602.27.4107F7B Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf03.hostedemail.com (Postfix) with ESMTP id 8092720005 for ; Fri, 10 Nov 2023 20:41:39 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=vZbf4qLU; dmarc=none; spf=none (imf03.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1699648899; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Xki740aAoX9yBxyCdgA26lHDCnTzivKGn1Gv5eBSmHQ=; b=ejx5kKhmLB5rCpcmIW60/I3Y0fGINu1i1Yqf6biIVP17OOLX6RyWspSxgWg0PoDRqfhYv6 Bd7ht06BG2yfIYSS4Zr6yqExzdRED+EBY6WDYGFDIMhEgOeZPzlSkucp7twRUT+HxhTNoT 1KqragyA78aPShzWEbsnISs2zkuXfT8= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=vZbf4qLU; dmarc=none; spf=none (imf03.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1699648899; a=rsa-sha256; cv=none; b=oHpqeKJH0Gk0JgWfM5e//OXqc2CUvA+3ep4TufWhG8Bvjle+E8vVJWIjldqoACibKUchdM F8UFFohi/4QG15bnXEB8LaLLMgbxqkdn8871Mu9ldJ4bjHsK/huwEF0NXUaGxvfbcIND56 bytrvEYGvzNujt/AoMHDngV6vOgLHtI= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=Xki740aAoX9yBxyCdgA26lHDCnTzivKGn1Gv5eBSmHQ=; b=vZbf4qLUD8BS9Dl7W1IRtLtjat xawi8qeBavSyl8iXANIozdeDZwFbgA/BRLWIdBz/nBloRyrsb6NH178+2p4YNsDS8Z3Hv5/bxkx1n s5fre0aG20Bfw5KLbwnVR2mgyIWjiVt+SWZ7Pqo1uCoo/3bcj6XCO+Y9UAKAMZGbS3QdtcWP3C4T+ vRiGf6JNzq2JDBCoZe2gydqfci30qTZptXtZQ99uKoKjsEiyRxjEjySoVVdYFUaA2VeKfD5riRcNF 3GagWA8b08j58LozOizxsJk6JKYtbKYBlAhkgqBXWrJh9FZDPGeon+ORMuksMkbUP7QUP/z7Unog7 fiqjjzRg==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1r1YJd-00FUT4-0t; Fri, 10 Nov 2023 20:41:21 +0000 From: "Matthew Wilcox (Oracle)" To: Peter Zijlstra , Ingo Molnar , Will Deacon , Waiman Long Cc: "Matthew Wilcox (Oracle)" , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Chandan Babu R , "Darrick J . Wong" , linux-xfs@vger.kernel.org, Mateusz Guzik Subject: [PATCH v3 1/4] locking: Add rwsem_assert_held() and rwsem_assert_held_write() Date: Fri, 10 Nov 2023 20:41:16 +0000 Message-Id: <20231110204119.3692023-2-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20231110204119.3692023-1-willy@infradead.org> References: <20231110204119.3692023-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 8092720005 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: nbkhrgwxj3dbgfy5xcgmwhe5y8ibcamn X-HE-Tag: 1699648899-73394 X-HE-Meta: U2FsdGVkX1/S8GCmuELVnLKb5K67WQRmN+JpIlkqjf8voQw35cjpwi3OwHGZ6MQKNm7ydqGEEuNJg4RsR4IDFumfXFgIz56bcXckBh2KZmuWkWqbb3l3bGyR02fPX3ssVq/zPHyQuKEprGb0PyMDy+/X+iccjDZ2OZWwvS7EuMfT4yRE8HgkFJSazzYQk8fwJ9TS70T+QKAsbhJa7xeN4/mppBSi7P6SsOo44laVZ3gao4Oaw/A6waKFyQ9tCm4KaChdrqZk0Q7plxi/I62q437Gsu4UwtFL47RU8nJPpzS7F5UboorCWTuyJCYpbCwOIv0GRVAKVAqOzNEzUBYZwAQe8t5S3UjYztjkRiHU1Ze+0g6L28Y7PGwoM1QK13ZB0hxSf5UKborDgoM6hKEr90lczaxcUt3zU6F8RN6sHWLnu75x3EYWaQXF+ZlJIsNxOKpzpseAUOxv8+zDnPq38fI0HVUiWZRjVGoUdPrfA0eD05Ipb8d/STc0t829UieBJSDsDstQwVMKihg9SQchZw0/+8K6lk9xn0dBpguC9oK1phy6W1j3fZQZaJ5RL9WRTdLJf3QYXsqNBajFSdwc/pmmhBS2V5wowSsXl7uBI+Z81TcFH7c3LC13y3bhfLVXVHeXJF5zxnsF7Ph2cbJdo3fuvFZvpu4XBQvKccBoVJ9r6A2MNpnY5DVjidBxY6M5gViB3W9jVsg/VvkJs4aI1d4vt86wmAC1wQ7Ms7nf5WALEgr9hluYd4PusXdYh4nBdltRjzbzH6Raa1f/sDlRxvOuB+7MGdbAxTJq7Jty61o+VFi+NLIZSJSRedYNNkxT0/2NJGm5L6eIlq9JJSHPc1QSdL09yMXgGCbb8MP4S/kVJk6ZaWJLCD1CaRCBVtRBo15E5F1qOVWJMKqJ1bTwKrRGNvY4xNZLX1cxqw5RRX/DCzm1RoiRGbMYEBR8MEs83HQ9R+zZ3q3u1PtSiD9 3L7pd+yA ZKMaqhYAcmW6hYBjdwG7YLAy0fxsf/hJohKl4w5IjW9BDA/YHyla+HnkA8s7HAvhWv/EuyUMitg9XbofZzmuiXSLaOkhu+vAROnN1cP6gKr9R8xf7bJseNY1SqKSbNcCHuVkybf52+MfWwMhXs0R+8pG1OGWF7JuRLpvxzHgpS9BC2n/ahVOTPNAl2nhx3PasyANqLe5FZv1O4+aEyrKps6OXSe2Gh99cvOemFx6BGK0XNh0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Modelled after lockdep_assert_held() and lockdep_assert_held_write(), but are always active, even when lockdep is disabled. Of course, they don't test that _this_ thread is the owner, but it's sufficient to catch many bugs and doesn't incur the same performance penalty as lockdep. Signed-off-by: Matthew Wilcox (Oracle) --- include/linux/rwbase_rt.h | 9 ++++++-- include/linux/rwsem.h | 46 ++++++++++++++++++++++++++++++++++----- 2 files changed, 48 insertions(+), 7 deletions(-) diff --git a/include/linux/rwbase_rt.h b/include/linux/rwbase_rt.h index 1d264dd08625..a04acd85705b 100644 --- a/include/linux/rwbase_rt.h +++ b/include/linux/rwbase_rt.h @@ -26,12 +26,17 @@ struct rwbase_rt { } while (0) -static __always_inline bool rw_base_is_locked(struct rwbase_rt *rwb) +static __always_inline bool rw_base_is_locked(const struct rwbase_rt *rwb) { return atomic_read(&rwb->readers) != READER_BIAS; } -static __always_inline bool rw_base_is_contended(struct rwbase_rt *rwb) +static inline void rw_base_assert_held_write(const struct rwbase_rt *rwb) +{ + BUG_ON(atomic_read(&rwb->readers) != WRITER_BIAS); +} + +static __always_inline bool rw_base_is_contended(const struct rwbase_rt *rwb) { return atomic_read(&rwb->readers) > 0; } diff --git a/include/linux/rwsem.h b/include/linux/rwsem.h index 1dd530ce8b45..b5b34cca86f3 100644 --- a/include/linux/rwsem.h +++ b/include/linux/rwsem.h @@ -66,14 +66,24 @@ struct rw_semaphore { #endif }; -/* In all implementations count != 0 means locked */ +#define RWSEM_UNLOCKED_VALUE 0UL +#define RWSEM_WRITER_LOCKED (1UL << 0) +#define __RWSEM_COUNT_INIT(name) .count = ATOMIC_LONG_INIT(RWSEM_UNLOCKED_VALUE) + static inline int rwsem_is_locked(struct rw_semaphore *sem) { - return atomic_long_read(&sem->count) != 0; + return atomic_long_read(&sem->count) != RWSEM_UNLOCKED_VALUE; } -#define RWSEM_UNLOCKED_VALUE 0L -#define __RWSEM_COUNT_INIT(name) .count = ATOMIC_LONG_INIT(RWSEM_UNLOCKED_VALUE) +static inline void rwsem_assert_held_nolockdep(const struct rw_semaphore *sem) +{ + WARN_ON(atomic_long_read(&sem->count) == RWSEM_UNLOCKED_VALUE); +} + +static inline void rwsem_assert_held_write_nolockdep(const struct rw_semaphore *sem) +{ + WARN_ON(!(atomic_long_read(&sem->count) & RWSEM_WRITER_LOCKED)); +} /* Common initializer macros and functions */ @@ -152,11 +162,21 @@ do { \ __init_rwsem((sem), #sem, &__key); \ } while (0) -static __always_inline int rwsem_is_locked(struct rw_semaphore *sem) +static __always_inline int rwsem_is_locked(const struct rw_semaphore *sem) { return rw_base_is_locked(&sem->rwbase); } +static inline void rwsem_assert_held_nolockdep(const struct rw_semaphore *sem) +{ + BUG_ON(!rwsem_is_locked(sem)); +} + +static inline void rwsem_assert_held_write_nolockdep(const struct rw_semaphore *sem) +{ + rw_base_assert_held_write(sem); +} + static __always_inline int rwsem_is_contended(struct rw_semaphore *sem) { return rw_base_is_contended(&sem->rwbase); @@ -169,6 +189,22 @@ static __always_inline int rwsem_is_contended(struct rw_semaphore *sem) * the RT specific variant. */ +static inline void rwsem_assert_held(const struct rw_semaphore *sem) +{ + if (IS_ENABLED(CONFIG_LOCKDEP)) + lockdep_assert_held(sem); + else + rwsem_assert_held_nolockdep(sem); +} + +static inline void rwsem_assert_held_write(const struct rw_semaphore *sem) +{ + if (IS_ENABLED(CONFIG_LOCKDEP)) + lockdep_assert_held_write(sem); + else + rwsem_assert_held_write_nolockdep(sem); +} + /* * lock for reading */ From patchwork Fri Nov 10 20:41:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13452777 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 04879C4332F for ; Fri, 10 Nov 2023 20:41:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3A57E8D00F9; Fri, 10 Nov 2023 15:41:33 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 387EE8D00FA; Fri, 10 Nov 2023 15:41:33 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2444F8D00F9; Fri, 10 Nov 2023 15:41:33 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 1706D8D0005 for ; Fri, 10 Nov 2023 15:41:33 -0500 (EST) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id D540B40118 for ; Fri, 10 Nov 2023 20:41:32 +0000 (UTC) X-FDA: 81443215224.18.A08DF20 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf14.hostedemail.com (Postfix) with ESMTP id E33A010000D for ; Fri, 10 Nov 2023 20:41:30 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=VtlHsuyG; dmarc=none; spf=none (imf14.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1699648891; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=25QfWharHbMWrIl6caRn0CZ+MZ75oP4Z9qoseEg2zjc=; b=O95g19Ylf7pai+yQaWqVPZyLbNKfuJsxi3bX1rEHFObJ+igwyA6yfL6hK4kj90/qMZdG3A cVvmBWuFMf06lZu1eiPMteUSXN5FbVYTz5UOLN7bMVXEnEbxEi3C4psuUEiTpuAXUCod70 ochh9iBbQVsaHXuvJ01RmMeIsvIByMg= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=VtlHsuyG; dmarc=none; spf=none (imf14.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1699648891; a=rsa-sha256; cv=none; b=i+MzTdQmP4zLy+fsu+89hsdFObPCHUyd5FyOocN/ST2KxWiO+Zo2hEvOLTIW9RReRK/oU2 8GU6vjbyltuTgt0IOo32y8bsnSarGaSlAWCCH6OEFJJ6tg+n+vQIvVCGbHvGNgAr9Omf5m pbK+yoMwIIPhDAtpmmOTMJrvcbw0AoU= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=25QfWharHbMWrIl6caRn0CZ+MZ75oP4Z9qoseEg2zjc=; b=VtlHsuyG81kZTqbBNIglKsUA0j NhxzGv8c91tUS6E+YAnLht71Pq8PLu7ZDmoMvu106yZEX5Kl7ViDhN7Qx+Cb+X8hODeZldt8xfMd+ bT4PCX2OFJrqE0sUJ0OZ3bx78GDIcbAJ2wfvrTPW0psqAXRuZFkr6WADJUBlQuhRYf3P/sRmjM2xx nQzc1dQq43nkYvCVN2v6thH/fsFm59/CUZOhiEgN8Rc9AlKRmnwqa1pac1ygpGKR007h2pAVu/7VE cxvROJ3Ixoz+pKs7BIRw+5LjUk/0dmGjChbBgW+KE26RME/Nn5uTd0ehSIJ9ltZr47+ZV2c3QZd7C Qp9DOl8g==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1r1YJd-00FUT6-51; Fri, 10 Nov 2023 20:41:21 +0000 From: "Matthew Wilcox (Oracle)" To: Peter Zijlstra , Ingo Molnar , Will Deacon , Waiman Long Cc: "Matthew Wilcox (Oracle)" , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Chandan Babu R , "Darrick J . Wong" , linux-xfs@vger.kernel.org, Mateusz Guzik Subject: [PATCH v3 2/4] mm: Use rwsem assertion macros for mmap_lock Date: Fri, 10 Nov 2023 20:41:17 +0000 Message-Id: <20231110204119.3692023-3-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20231110204119.3692023-1-willy@infradead.org> References: <20231110204119.3692023-1-willy@infradead.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: E33A010000D X-Stat-Signature: 6goqdbn7srue6yo7n7bp8387jn94x9cp X-HE-Tag: 1699648890-120959 X-HE-Meta: U2FsdGVkX192vDCUtl8zgjHlb15KWtrlkiGXwhnrzHpMbjW/NUdfelOOyUocO6Zs96mv3qR3Gy1dLjBqZ9pWXV3k5zXugRVnxosR7wGqM3/xLx6NVeehrviie4FHQn/2+eZ2TlFgGtk8WDSB4CT8CkbPCKrxv7ZjZvxQzIjvwkikfZ5TvYyhDVRZboayx1TGm1UGLyBcbeq+P/plWuX9tl4Rcm5gtO+QqScFwdSbuGa5/WEe3Q0JIsITymBDr5pwdyGzBYi32SrLBX6GEdnipWD+Lh55kM8ZZeLH0ncOP5YhfURyVSNSsr5wHAATjj9bGgH3w71nWK1MxSseeXlHPyQtlpXn47bu1LwkD2OhDNV/m8rBwZfT46wPOeiuB8W6Ifxe0pu9tIq8Xf+Yh6t3q5JgQh2ZP0oKLdl90hY40FY3aW8xwbQBXqFVXbxlJsuStL0noPpEQsO5K0PKPQQ8mWEMtLmfkcMqYgx5Ludyv+/WSrKe2ubVGtihaWu6aw75mc4L6jlUIx3ZlfNZiIb0uoyH7yFv0eO26RHooB28vTTyLzyZ6d/rPO2gYoQwx0YHsFSo3f6nOBfoGOrFYg12wVKZztbVLZaOu/tKhCDbNQXl5jA2Z3SSO0gGW05dCCmMSvX6WbuDcj/YhaXx620HgcNJZcIP5FfW6CbLwskGOCviTOIXZjjyrCRUjle1awliLft1/IGD4Ux2mYGoWEksE1Jyl07cicXjoeB/88L3u+eJ3lpL/DDP7ksKL3xWYE39zVZRDLMu0XDZ/w9xRv5EzLaXC2sKh/ZDrMXJxTVE0U80yqcIJSqclm6L8lR8UOMjvr9bPBXGi+XggjLQ0B8pm/WF0erPBSVg9Hq+yqoFi8qGwHovW966b2Uo4w7yQANZ7xzHWkY3G6XniuE+YGKePnilMZ1oz830abGHM1RxnDJBo8iKBIDvdLtHsKpTSy+elXWyXlX1D7AdSx2woTN 3H3emaGO 9ZxELnAJbRqKZ8rvEN5oIMou2GRg6RJzmy9hRZ3u0nzZDq/IwyVxKCW1HcUG12CkPX1z3nlB1g5cWiblljkGNSA03fThNvvUTrW6dkEAs1Lh6wia5yZaHywMob4mEeTkm4q0cUeUWvAqNQ6Sar87jUPzvGmFsgCE23DG1vzQvMwcdmaQgbI34Sy7u5z4WbfZshU/Rf38HcF9NIdKJGtVQ7bnT3kLDyT1A3jyg X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This slightly strengthens our write assertion when lockdep is disabled. It also downgrades us from BUG_ON to WARN_ON, but I think that's an improvement. I don't think dumping the mm_struct was all that valuable; the call chain is what's important. Signed-off-by: Matthew Wilcox (Oracle) --- include/linux/mmap_lock.h | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/include/linux/mmap_lock.h b/include/linux/mmap_lock.h index 8d38dcb6d044..de9dc20b01ba 100644 --- a/include/linux/mmap_lock.h +++ b/include/linux/mmap_lock.h @@ -60,16 +60,14 @@ static inline void __mmap_lock_trace_released(struct mm_struct *mm, bool write) #endif /* CONFIG_TRACING */ -static inline void mmap_assert_locked(struct mm_struct *mm) +static inline void mmap_assert_locked(const struct mm_struct *mm) { - lockdep_assert_held(&mm->mmap_lock); - VM_BUG_ON_MM(!rwsem_is_locked(&mm->mmap_lock), mm); + rwsem_assert_held(&mm->mmap_lock); } -static inline void mmap_assert_write_locked(struct mm_struct *mm) +static inline void mmap_assert_write_locked(const struct mm_struct *mm) { - lockdep_assert_held_write(&mm->mmap_lock); - VM_BUG_ON_MM(!rwsem_is_locked(&mm->mmap_lock), mm); + rwsem_assert_held_write(&mm->mmap_lock); } #ifdef CONFIG_PER_VMA_LOCK From patchwork Fri Nov 10 20:41:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13452778 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 14299C4167B for ; Fri, 10 Nov 2023 20:41:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 83B32280009; Fri, 10 Nov 2023 15:41:33 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7EA198D0005; Fri, 10 Nov 2023 15:41:33 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 68A998D00FA; Fri, 10 Nov 2023 15:41:33 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 34C658D0005 for ; Fri, 10 Nov 2023 15:41:33 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id E65AD1404E6 for ; Fri, 10 Nov 2023 20:41:32 +0000 (UTC) X-FDA: 81443215224.17.0305DB1 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf30.hostedemail.com (Postfix) with ESMTP id EFC9680012 for ; Fri, 10 Nov 2023 20:41:30 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b="V/0LsvqH"; spf=none (imf30.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1699648891; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=zIWhQPPVGGNsaH/zRjrbtFp7YhWrYOd/f9YxTGaAZsk=; b=7l2611Nc9VXQ/RCEYz+juBxa02AMZvLObRWOHvZ5IP72yVBAWNwckX2mKIBkbVhnvdJ5lQ 8XRzeYVc+DY/psazc28hEsp84AIhoEezk0d7nwQo9lGiJ8FLHtmej3HqewOxdt+1KSk8/l A5ZKqlNIk7rDjXUwpyfeZHKI96WBves= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b="V/0LsvqH"; spf=none (imf30.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1699648891; a=rsa-sha256; cv=none; b=uLHwRjQ5te7TRLH+UEcNToptQdgE41E0mfbWchKWj8enohu065zUPyEKSG5ohUo7XHgTDM o4r76bjV00WMzWn1n8nxpGJIIYmmFM1vzHcn10fPJ2AJ5y/FYOCVVS/6p4gMW6L+7vnEKN jwngiRBhys6b//HNmv8T5RS+RPVRNmY= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=zIWhQPPVGGNsaH/zRjrbtFp7YhWrYOd/f9YxTGaAZsk=; b=V/0LsvqHZxuH1WQBbyi1o8z9ob khRauoUl0ia4ev2SntjAolsMV+pTeyIM6ND3ENfIiVNwIuBvpGpuTD7BIeJnU01XB4qQlvavyMN56 phj4yqYpnOfE+xivPYnjQqNlh622XeuBNdimGTYJqMtRkJA/0xJfLP/kTna/KjzmzYzRXS8FLf/Jw 6OjsWNF0X+tTLixXMKlA3ysqh4tzhBCE+us7CMebzE1+yPghfsjI2KlmCGkln1pfwyVehm7PICMFg 5CVV3HceK8SJwsjFFcZ4FA85DrpsIR9Oli15msYV3+hxt7Ta1yau9tjNzDZ862EAZL8mtlxO4dw3e aqbnUyHQ==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1r1YJd-00FUT8-7j; Fri, 10 Nov 2023 20:41:21 +0000 From: "Matthew Wilcox (Oracle)" To: Peter Zijlstra , Ingo Molnar , Will Deacon , Waiman Long Cc: "Matthew Wilcox (Oracle)" , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Chandan Babu R , "Darrick J . Wong" , linux-xfs@vger.kernel.org, Mateusz Guzik Subject: [PATCH v3 3/4] xfs: Replace xfs_isilocked with xfs_assert_ilocked Date: Fri, 10 Nov 2023 20:41:18 +0000 Message-Id: <20231110204119.3692023-4-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20231110204119.3692023-1-willy@infradead.org> References: <20231110204119.3692023-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: EFC9680012 X-Rspam-User: X-Stat-Signature: eu4ep7b73t5x3xnfoibmcuychri9swfs X-Rspamd-Server: rspam01 X-HE-Tag: 1699648890-269391 X-HE-Meta: U2FsdGVkX1+A6z0652QRgm+CRp3l9mRtJnShfr88oV7vn9bmOS5CbrUzWDJTi2GblDy4/ErU/Wut44JcsG6abs2VF4mL5ncMnlraZSE5XtRpw3OFG9GbNUdYHNPsePOtgL5whZHwX/fXC63Lzhv06njyysQ+4NmP5+8JlSh5fai3EUlFiBQ2JxS0dW+t7WHexFm4RX1JDTM5t6hmANBbn+aUHasQCYdXrlJHL68oYr7sfndFnvbhaARHncstaJHS9VQeKNbOm6YhHBLJCnZ23nlbYd9IBbp8U4m8vWNnSVmDcCxLA5AgRFivs3jsYruzJiwLc5F3wIlRHFYwPOKDMaVOJYzaAQgW0QnOW0fXpkv5fv7D0HgJIse5dmXs5ZVbgESwIlPnA/adONUzozqSUSNG2djYv0NTwtapcVOgX+G1uBO604xpBkhdGmBqgm2OUYB6RDDHEp/HDfrV1ow1ZTWnR8s0XbjmZ2dVbaciXBvUWjsIBu/d6DlBnpz9h+xpxuc9IAgmZe42dteSq/k7yR9FxnqR8aolYAKWGdyHf0RhqV8aNKn8RsggAPgcOWA970I32jYCG4aAGUls+oY35QyYscFxGYseMCO6jBZHPANHqtQkNeO2zrDpT2pcxrSXNIc6coQdrs0KO4G4C3H/SOylsnBbRXvNZ4ZitEbhM918XMwlJ89hP+ZZIveOz71Pwv4T/nGeMcXaF6L0TwADaRYoMpfIlOC2tWu881xdWBhtDY2ZbrecJujrt1cn5yDCIRjUv8oO/wBZgW9bPd5K99fXoeGG7vvpj2H4X4+eqRVdcmFuHwefAEib9+R93sSanhkgQQEnlSOhKhTzLpOmYm5jhO+2g8i8RBrP959O5ekYrV7rY9xQSSgvSRoLKG9I+/DhBzjbiWD8qMrrenm0KLida+8XK2tRVW5q5zvCjgsDhIiymfF9qPuadS9KnvS8rjDq9viETjxPIxdZDYf R7L5ed8L EJDBQbzniJx3XYn1nMBo8UNtS+Ty9FcpcAlHoAyEUZq252H5oHtZ5O6310AEO9Lyq0UUTcVwtRgpcrA1ROKVUlngrxRMDueKZ0bsETMyE4vLnSDnlaAaYbmnO/QOGWb3O2NxbayH+BUdHESOjKZecjY/1TVZvVGT1JLiGce2nZa1Ejv7zKFRtmatgGVIXeQu2Yw6iKsXYqI/DTNfOtO4Ksa46ItueyitHZkSj X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To use the new rwsem_assert_held()/rwsem_assert_held_write(), we can't use the existing ASSERT macro. Add a new xfs_assert_ilocked() and convert all the callers. Fix an apparent bug in xfs_isilocked(): If the caller specifies XFS_IOLOCK_EXCL | XFS_ILOCK_EXCL, xfs_assert_ilocked() will check both the IOLOCK and the ILOCK are held for write. xfs_isilocked() only checked that the ILOCK was held for write. xfs_assert_ilocked() is always on, even if DEBUG or XFS_WARN aren't defined. It's a cheap check, so I don't think it's worth defining it away. Signed-off-by: Matthew Wilcox (Oracle) --- fs/xfs/libxfs/xfs_attr.c | 2 +- fs/xfs/libxfs/xfs_attr_remote.c | 2 +- fs/xfs/libxfs/xfs_bmap.c | 19 +++++---- fs/xfs/libxfs/xfs_defer.c | 2 +- fs/xfs/libxfs/xfs_inode_fork.c | 2 +- fs/xfs/libxfs/xfs_rtbitmap.c | 2 +- fs/xfs/libxfs/xfs_trans_inode.c | 6 +-- fs/xfs/scrub/readdir.c | 4 +- fs/xfs/xfs_attr_list.c | 2 +- fs/xfs/xfs_bmap_util.c | 10 ++--- fs/xfs/xfs_dir2_readdir.c | 2 +- fs/xfs/xfs_dquot.c | 4 +- fs/xfs/xfs_file.c | 4 +- fs/xfs/xfs_inode.c | 72 +++++++++++---------------------- fs/xfs/xfs_inode.h | 2 +- fs/xfs/xfs_inode_item.c | 4 +- fs/xfs/xfs_iops.c | 3 +- fs/xfs/xfs_qm.c | 10 ++--- fs/xfs/xfs_reflink.c | 2 +- fs/xfs/xfs_rtalloc.c | 4 +- fs/xfs/xfs_symlink.c | 2 +- fs/xfs/xfs_trans_dquot.c | 2 +- 22 files changed, 66 insertions(+), 96 deletions(-) diff --git a/fs/xfs/libxfs/xfs_attr.c b/fs/xfs/libxfs/xfs_attr.c index e28d93d232de..ebf0722d8963 100644 --- a/fs/xfs/libxfs/xfs_attr.c +++ b/fs/xfs/libxfs/xfs_attr.c @@ -224,7 +224,7 @@ int xfs_attr_get_ilocked( struct xfs_da_args *args) { - ASSERT(xfs_isilocked(args->dp, XFS_ILOCK_SHARED | XFS_ILOCK_EXCL)); + xfs_assert_ilocked(args->dp, XFS_ILOCK_SHARED | XFS_ILOCK_EXCL); if (!xfs_inode_hasattr(args->dp)) return -ENOATTR; diff --git a/fs/xfs/libxfs/xfs_attr_remote.c b/fs/xfs/libxfs/xfs_attr_remote.c index d440393b40eb..1c007ebf153a 100644 --- a/fs/xfs/libxfs/xfs_attr_remote.c +++ b/fs/xfs/libxfs/xfs_attr_remote.c @@ -545,7 +545,7 @@ xfs_attr_rmtval_stale( struct xfs_buf *bp; int error; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); if (XFS_IS_CORRUPT(mp, map->br_startblock == DELAYSTARTBLOCK) || XFS_IS_CORRUPT(mp, map->br_startblock == HOLESTARTBLOCK)) diff --git a/fs/xfs/libxfs/xfs_bmap.c b/fs/xfs/libxfs/xfs_bmap.c index be62acffad6c..eaa1dbe9a6b3 100644 --- a/fs/xfs/libxfs/xfs_bmap.c +++ b/fs/xfs/libxfs/xfs_bmap.c @@ -1189,7 +1189,7 @@ xfs_iread_extents( if (!xfs_need_iread_extents(ifp)) return 0; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); ir.loaded = 0; xfs_iext_first(ifp, &ir.icur); @@ -3883,7 +3883,7 @@ xfs_bmapi_read( ASSERT(*nmap >= 1); ASSERT(!(flags & ~(XFS_BMAPI_ATTRFORK | XFS_BMAPI_ENTIRE))); - ASSERT(xfs_isilocked(ip, XFS_ILOCK_SHARED|XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_SHARED|XFS_ILOCK_EXCL); if (WARN_ON_ONCE(!ifp)) return -EFSCORRUPTED; @@ -4354,7 +4354,7 @@ xfs_bmapi_write( ASSERT(tp != NULL); ASSERT(len > 0); ASSERT(ifp->if_format != XFS_DINODE_FMT_LOCAL); - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); ASSERT(!(flags & XFS_BMAPI_REMAP)); /* zeroing is for currently only for data extents, not metadata */ @@ -4651,7 +4651,7 @@ xfs_bmapi_remap( ifp = xfs_ifork_ptr(ip, whichfork); ASSERT(len > 0); ASSERT(len <= (xfs_filblks_t)XFS_MAX_BMBT_EXTLEN); - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); ASSERT(!(flags & ~(XFS_BMAPI_ATTRFORK | XFS_BMAPI_PREALLOC | XFS_BMAPI_NORMAP))); ASSERT((flags & (XFS_BMAPI_ATTRFORK | XFS_BMAPI_PREALLOC)) != @@ -5287,7 +5287,7 @@ __xfs_bunmapi( if (xfs_is_shutdown(mp)) return -EIO; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); ASSERT(len > 0); ASSERT(nexts >= 0); @@ -5631,8 +5631,7 @@ xfs_bmse_merge( blockcount = left->br_blockcount + got->br_blockcount; - ASSERT(xfs_isilocked(ip, XFS_IOLOCK_EXCL)); - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_IOLOCK_EXCL | XFS_ILOCK_EXCL); ASSERT(xfs_bmse_can_merge(left, got, shift)); new = *left; @@ -5760,7 +5759,7 @@ xfs_bmap_collapse_extents( if (xfs_is_shutdown(mp)) return -EIO; - ASSERT(xfs_isilocked(ip, XFS_IOLOCK_EXCL | XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_IOLOCK_EXCL | XFS_ILOCK_EXCL); error = xfs_iread_extents(tp, ip, whichfork); if (error) @@ -5833,7 +5832,7 @@ xfs_bmap_can_insert_extents( int is_empty; int error = 0; - ASSERT(xfs_isilocked(ip, XFS_IOLOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_IOLOCK_EXCL); if (xfs_is_shutdown(ip->i_mount)) return -EIO; @@ -5875,7 +5874,7 @@ xfs_bmap_insert_extents( if (xfs_is_shutdown(mp)) return -EIO; - ASSERT(xfs_isilocked(ip, XFS_IOLOCK_EXCL | XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_IOLOCK_EXCL | XFS_ILOCK_EXCL); error = xfs_iread_extents(tp, ip, whichfork); if (error) diff --git a/fs/xfs/libxfs/xfs_defer.c b/fs/xfs/libxfs/xfs_defer.c index bcfb6a4203cd..7927a721dc86 100644 --- a/fs/xfs/libxfs/xfs_defer.c +++ b/fs/xfs/libxfs/xfs_defer.c @@ -744,7 +744,7 @@ xfs_defer_ops_capture( * transaction. */ for (i = 0; i < dfc->dfc_held.dr_inos; i++) { - ASSERT(xfs_isilocked(dfc->dfc_held.dr_ip[i], XFS_ILOCK_EXCL)); + xfs_assert_ilocked(dfc->dfc_held.dr_ip[i], XFS_ILOCK_EXCL); ihold(VFS_I(dfc->dfc_held.dr_ip[i])); } diff --git a/fs/xfs/libxfs/xfs_inode_fork.c b/fs/xfs/libxfs/xfs_inode_fork.c index 5a2e7ddfa76d..14193e044f3d 100644 --- a/fs/xfs/libxfs/xfs_inode_fork.c +++ b/fs/xfs/libxfs/xfs_inode_fork.c @@ -563,7 +563,7 @@ xfs_iextents_copy( struct xfs_bmbt_irec rec; int64_t copied = 0; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL | XFS_ILOCK_SHARED)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL | XFS_ILOCK_SHARED); ASSERT(ifp->if_bytes > 0); for_each_xfs_iext(ifp, &icur, &rec) { diff --git a/fs/xfs/libxfs/xfs_rtbitmap.c b/fs/xfs/libxfs/xfs_rtbitmap.c index c269d704314d..0b094a44c6e2 100644 --- a/fs/xfs/libxfs/xfs_rtbitmap.c +++ b/fs/xfs/libxfs/xfs_rtbitmap.c @@ -946,7 +946,7 @@ xfs_rtfree_extent( struct timespec64 atime; ASSERT(mp->m_rbmip->i_itemp != NULL); - ASSERT(xfs_isilocked(mp->m_rbmip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(mp->m_rbmip, XFS_ILOCK_EXCL); error = xfs_rtcheck_alloc_range(&args, start, len); if (error) diff --git a/fs/xfs/libxfs/xfs_trans_inode.c b/fs/xfs/libxfs/xfs_trans_inode.c index 70e97ea6eee7..69fc5b981352 100644 --- a/fs/xfs/libxfs/xfs_trans_inode.c +++ b/fs/xfs/libxfs/xfs_trans_inode.c @@ -31,7 +31,7 @@ xfs_trans_ijoin( { struct xfs_inode_log_item *iip; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); if (ip->i_itemp == NULL) xfs_inode_item_init(ip, ip->i_mount); iip = ip->i_itemp; @@ -60,7 +60,7 @@ xfs_trans_ichgtime( struct timespec64 tv; ASSERT(tp); - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); tv = current_time(inode); @@ -90,7 +90,7 @@ xfs_trans_log_inode( struct inode *inode = VFS_I(ip); ASSERT(iip); - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); ASSERT(!xfs_iflags_test(ip, XFS_ISTALE)); tp->t_flags |= XFS_TRANS_DIRTY; diff --git a/fs/xfs/scrub/readdir.c b/fs/xfs/scrub/readdir.c index e51c1544be63..0b200bab04c8 100644 --- a/fs/xfs/scrub/readdir.c +++ b/fs/xfs/scrub/readdir.c @@ -283,7 +283,7 @@ xchk_dir_walk( return -EIO; ASSERT(S_ISDIR(VFS_I(dp)->i_mode)); - ASSERT(xfs_isilocked(dp, XFS_ILOCK_SHARED | XFS_ILOCK_EXCL)); + xfs_assert_ilocked(dp, XFS_ILOCK_SHARED | XFS_ILOCK_EXCL); if (dp->i_df.if_format == XFS_DINODE_FMT_LOCAL) return xchk_dir_walk_sf(sc, dp, dirent_fn, priv); @@ -334,7 +334,7 @@ xchk_dir_lookup( return -EIO; ASSERT(S_ISDIR(VFS_I(dp)->i_mode)); - ASSERT(xfs_isilocked(dp, XFS_ILOCK_SHARED | XFS_ILOCK_EXCL)); + xfs_assert_ilocked(dp, XFS_ILOCK_SHARED | XFS_ILOCK_EXCL); if (dp->i_df.if_format == XFS_DINODE_FMT_LOCAL) { error = xfs_dir2_sf_lookup(&args); diff --git a/fs/xfs/xfs_attr_list.c b/fs/xfs/xfs_attr_list.c index 99bbbe1a0e44..235dd125c92f 100644 --- a/fs/xfs/xfs_attr_list.c +++ b/fs/xfs/xfs_attr_list.c @@ -505,7 +505,7 @@ xfs_attr_list_ilocked( { struct xfs_inode *dp = context->dp; - ASSERT(xfs_isilocked(dp, XFS_ILOCK_SHARED | XFS_ILOCK_EXCL)); + xfs_assert_ilocked(dp, XFS_ILOCK_SHARED | XFS_ILOCK_EXCL); /* * Decide on what work routines to call based on the inode size. diff --git a/fs/xfs/xfs_bmap_util.c b/fs/xfs/xfs_bmap_util.c index 731260a5af6d..0887fb37d84f 100644 --- a/fs/xfs/xfs_bmap_util.c +++ b/fs/xfs/xfs_bmap_util.c @@ -649,8 +649,8 @@ xfs_can_free_eofblocks( * Caller must either hold the exclusive io lock; or be inactivating * the inode, which guarantees there are no other users of the inode. */ - ASSERT(xfs_isilocked(ip, XFS_IOLOCK_EXCL) || - (VFS_I(ip)->i_state & I_FREEING)); + if (!(VFS_I(ip)->i_state & I_FREEING)) + xfs_assert_ilocked(ip, XFS_IOLOCK_EXCL); /* prealloc/delalloc exists only on regular files */ if (!S_ISREG(VFS_I(ip)->i_mode)) @@ -1106,8 +1106,7 @@ xfs_collapse_file_space( xfs_fileoff_t shift_fsb = XFS_B_TO_FSB(mp, len); bool done = false; - ASSERT(xfs_isilocked(ip, XFS_IOLOCK_EXCL)); - ASSERT(xfs_isilocked(ip, XFS_MMAPLOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_IOLOCK_EXCL | XFS_MMAPLOCK_EXCL); trace_xfs_collapse_file_space(ip); @@ -1176,8 +1175,7 @@ xfs_insert_file_space( xfs_fileoff_t shift_fsb = XFS_B_TO_FSB(mp, len); bool done = false; - ASSERT(xfs_isilocked(ip, XFS_IOLOCK_EXCL)); - ASSERT(xfs_isilocked(ip, XFS_MMAPLOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_IOLOCK_EXCL | XFS_MMAPLOCK_EXCL); trace_xfs_insert_file_space(ip); diff --git a/fs/xfs/xfs_dir2_readdir.c b/fs/xfs/xfs_dir2_readdir.c index 9f3ceb461515..95cd8b9cf3dc 100644 --- a/fs/xfs/xfs_dir2_readdir.c +++ b/fs/xfs/xfs_dir2_readdir.c @@ -521,7 +521,7 @@ xfs_readdir( return -EIO; ASSERT(S_ISDIR(VFS_I(dp)->i_mode)); - ASSERT(xfs_isilocked(dp, XFS_IOLOCK_SHARED | XFS_IOLOCK_EXCL)); + xfs_assert_ilocked(dp, XFS_IOLOCK_SHARED | XFS_IOLOCK_EXCL); XFS_STATS_INC(dp->i_mount, xs_dir_getdents); args.dp = dp; diff --git a/fs/xfs/xfs_dquot.c b/fs/xfs/xfs_dquot.c index ac6ba646624d..4b2f1b82badc 100644 --- a/fs/xfs/xfs_dquot.c +++ b/fs/xfs/xfs_dquot.c @@ -949,7 +949,7 @@ xfs_qm_dqget_inode( if (error) return error; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); ASSERT(xfs_inode_dquot(ip, type) == NULL); id = xfs_qm_id_for_quotatype(ip, type); @@ -1006,7 +1006,7 @@ xfs_qm_dqget_inode( } dqret: - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); trace_xfs_dqget_miss(dqp); *O_dqpp = dqp; return 0; diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c index e33e5e13b95f..632653e00906 100644 --- a/fs/xfs/xfs_file.c +++ b/fs/xfs/xfs_file.c @@ -879,7 +879,7 @@ xfs_break_dax_layouts( { struct page *page; - ASSERT(xfs_isilocked(XFS_I(inode), XFS_MMAPLOCK_EXCL)); + xfs_assert_ilocked(XFS_I(inode), XFS_MMAPLOCK_EXCL); page = dax_layout_busy_page(inode->i_mapping); if (!page) @@ -900,7 +900,7 @@ xfs_break_layouts( bool retry; int error; - ASSERT(xfs_isilocked(XFS_I(inode), XFS_IOLOCK_SHARED|XFS_IOLOCK_EXCL)); + xfs_assert_ilocked(XFS_I(inode), XFS_IOLOCK_SHARED | XFS_IOLOCK_EXCL); do { retry = false; diff --git a/fs/xfs/xfs_inode.c b/fs/xfs/xfs_inode.c index c0f1c89786c2..1ed6bed19bec 100644 --- a/fs/xfs/xfs_inode.c +++ b/fs/xfs/xfs_inode.c @@ -333,52 +333,26 @@ xfs_ilock_demote( trace_xfs_ilock_demote(ip, lock_flags, _RET_IP_); } -#if defined(DEBUG) || defined(XFS_WARN) -static inline bool -__xfs_rwsem_islocked( - struct rw_semaphore *rwsem, - bool shared) -{ - if (!debug_locks) - return rwsem_is_locked(rwsem); - - if (!shared) - return lockdep_is_held_type(rwsem, 0); - - /* - * We are checking that the lock is held at least in shared - * mode but don't care that it might be held exclusively - * (i.e. shared | excl). Hence we check if the lock is held - * in any mode rather than an explicit shared mode. - */ - return lockdep_is_held_type(rwsem, -1); -} - -bool -xfs_isilocked( +void +xfs_assert_ilocked( struct xfs_inode *ip, uint lock_flags) { - if (lock_flags & (XFS_ILOCK_EXCL|XFS_ILOCK_SHARED)) { - if (!(lock_flags & XFS_ILOCK_SHARED)) - return !!ip->i_lock.mr_writer; - return rwsem_is_locked(&ip->i_lock.mr_lock); - } - - if (lock_flags & (XFS_MMAPLOCK_EXCL|XFS_MMAPLOCK_SHARED)) { - return __xfs_rwsem_islocked(&VFS_I(ip)->i_mapping->invalidate_lock, - (lock_flags & XFS_MMAPLOCK_SHARED)); - } - - if (lock_flags & (XFS_IOLOCK_EXCL | XFS_IOLOCK_SHARED)) { - return __xfs_rwsem_islocked(&VFS_I(ip)->i_rwsem, - (lock_flags & XFS_IOLOCK_SHARED)); - } - - ASSERT(0); - return false; + if (lock_flags & XFS_ILOCK_SHARED) + rwsem_assert_held(&ip->i_lock.mr_lock); + else if (lock_flags & XFS_ILOCK_EXCL) + ASSERT(ip->i_lock.mr_writer); + + if (lock_flags & XFS_MMAPLOCK_SHARED) + rwsem_assert_held(&VFS_I(ip)->i_mapping->invalidate_lock); + else if (lock_flags & XFS_MMAPLOCK_EXCL) + rwsem_assert_held_write(&VFS_I(ip)->i_mapping->invalidate_lock); + + if (lock_flags & XFS_IOLOCK_SHARED) + rwsem_assert_held(&VFS_I(ip)->i_rwsem); + else if (lock_flags & XFS_IOLOCK_EXCL) + rwsem_assert_held_write(&VFS_I(ip)->i_rwsem); } -#endif /* * xfs_lockdep_subclass_ok() is only used in an ASSERT, so is only called when @@ -1342,9 +1316,9 @@ xfs_itruncate_extents_flags( xfs_filblks_t unmap_len; int error = 0; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); - ASSERT(!atomic_read(&VFS_I(ip)->i_count) || - xfs_isilocked(ip, XFS_IOLOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); + if (atomic_read(&VFS_I(ip)->i_count)) + xfs_assert_ilocked(ip, XFS_IOLOCK_EXCL); ASSERT(new_size <= XFS_ISIZE(ip)); ASSERT(tp->t_flags & XFS_TRANS_PERM_LOG_RES); ASSERT(ip->i_itemp != NULL); @@ -1605,7 +1579,7 @@ xfs_inactive_ifree( xfs_trans_ijoin(tp, ip, XFS_ILOCK_EXCL); error = xfs_ifree(tp, ip); - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); if (error) { /* * If we fail to free the inode, shut down. The cancel @@ -2359,7 +2333,7 @@ xfs_ifree( struct xfs_inode_log_item *iip = ip->i_itemp; int error; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); ASSERT(VFS_I(ip)->i_nlink == 0); ASSERT(ip->i_df.if_nextents == 0); ASSERT(ip->i_disk_size == 0 || !S_ISREG(VFS_I(ip)->i_mode)); @@ -2428,7 +2402,7 @@ static void xfs_iunpin( struct xfs_inode *ip) { - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL|XFS_ILOCK_SHARED)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL | XFS_ILOCK_SHARED); trace_xfs_inode_unpin_nowait(ip, _RET_IP_); @@ -3189,7 +3163,7 @@ xfs_iflush( struct xfs_mount *mp = ip->i_mount; int error; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL|XFS_ILOCK_SHARED)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL|XFS_ILOCK_SHARED); ASSERT(xfs_iflags_test(ip, XFS_IFLUSHING)); ASSERT(ip->i_df.if_format != XFS_DINODE_FMT_BTREE || ip->i_df.if_nextents > XFS_IFORK_MAXEXT(ip, XFS_DATA_FORK)); diff --git a/fs/xfs/xfs_inode.h b/fs/xfs/xfs_inode.h index 3dc47937da5d..dd8b8339ba1b 100644 --- a/fs/xfs/xfs_inode.h +++ b/fs/xfs/xfs_inode.h @@ -523,7 +523,7 @@ void xfs_ilock(xfs_inode_t *, uint); int xfs_ilock_nowait(xfs_inode_t *, uint); void xfs_iunlock(xfs_inode_t *, uint); void xfs_ilock_demote(xfs_inode_t *, uint); -bool xfs_isilocked(struct xfs_inode *, uint); +void xfs_assert_ilocked(struct xfs_inode *, uint); uint xfs_ilock_data_map_shared(struct xfs_inode *); uint xfs_ilock_attr_map_shared(struct xfs_inode *); diff --git a/fs/xfs/xfs_inode_item.c b/fs/xfs/xfs_inode_item.c index cd7803fda8b1..3e125be356c1 100644 --- a/fs/xfs/xfs_inode_item.c +++ b/fs/xfs/xfs_inode_item.c @@ -649,7 +649,7 @@ xfs_inode_item_pin( { struct xfs_inode *ip = INODE_ITEM(lip)->ili_inode; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); ASSERT(lip->li_buf); trace_xfs_inode_pin(ip, _RET_IP_); @@ -755,7 +755,7 @@ xfs_inode_item_release( unsigned short lock_flags; ASSERT(ip->i_itemp != NULL); - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); lock_flags = iip->ili_lock_flags; iip->ili_lock_flags = 0; diff --git a/fs/xfs/xfs_iops.c b/fs/xfs/xfs_iops.c index fdfda4fba12b..e711668d601b 100644 --- a/fs/xfs/xfs_iops.c +++ b/fs/xfs/xfs_iops.c @@ -796,8 +796,7 @@ xfs_setattr_size( uint lock_flags = 0; bool did_zeroing = false; - ASSERT(xfs_isilocked(ip, XFS_IOLOCK_EXCL)); - ASSERT(xfs_isilocked(ip, XFS_MMAPLOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_IOLOCK_EXCL | XFS_MMAPLOCK_EXCL); ASSERT(S_ISREG(inode->i_mode)); ASSERT((iattr->ia_valid & (ATTR_UID|ATTR_GID|ATTR_ATIME|ATTR_ATIME_SET| ATTR_MTIME_SET|ATTR_TIMES_SET)) == 0); diff --git a/fs/xfs/xfs_qm.c b/fs/xfs/xfs_qm.c index 94a7932ac570..45cf9a557eb9 100644 --- a/fs/xfs/xfs_qm.c +++ b/fs/xfs/xfs_qm.c @@ -254,7 +254,7 @@ xfs_qm_dqattach_one( struct xfs_dquot *dqp; int error; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); error = 0; /* @@ -322,7 +322,7 @@ xfs_qm_dqattach_locked( if (!xfs_qm_need_dqattach(ip)) return 0; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); if (XFS_IS_UQUOTA_ON(mp) && !ip->i_udquot) { error = xfs_qm_dqattach_one(ip, XFS_DQTYPE_USER, @@ -353,7 +353,7 @@ xfs_qm_dqattach_locked( * Don't worry about the dquots that we may have attached before any * error - they'll get detached later if it has not already been done. */ - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); return error; } @@ -1809,7 +1809,7 @@ xfs_qm_vop_chown( XFS_TRANS_DQ_RTBCOUNT : XFS_TRANS_DQ_BCOUNT; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); ASSERT(XFS_IS_QUOTA_ON(ip->i_mount)); /* old dquot */ @@ -1897,7 +1897,7 @@ xfs_qm_vop_create_dqattach( if (!XFS_IS_QUOTA_ON(mp)) return; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); if (udqp && XFS_IS_UQUOTA_ON(mp)) { ASSERT(ip->i_udquot == NULL); diff --git a/fs/xfs/xfs_reflink.c b/fs/xfs/xfs_reflink.c index 658edee8381d..d10eba9548dc 100644 --- a/fs/xfs/xfs_reflink.c +++ b/fs/xfs/xfs_reflink.c @@ -527,7 +527,7 @@ xfs_reflink_allocate_cow( int error; bool found; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); if (!ip->i_cowfp) { ASSERT(!xfs_is_reflink_inode(ip)); xfs_ifork_init_cow(ip); diff --git a/fs/xfs/xfs_rtalloc.c b/fs/xfs/xfs_rtalloc.c index 88c48de5c9c8..5e5eedff71c9 100644 --- a/fs/xfs/xfs_rtalloc.c +++ b/fs/xfs/xfs_rtalloc.c @@ -1182,7 +1182,7 @@ xfs_rtallocate_extent( int error; /* error value */ xfs_rtxnum_t r; /* result allocated rtext */ - ASSERT(xfs_isilocked(args.mp->m_rbmip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(args.mp->m_rbmip, XFS_ILOCK_EXCL); ASSERT(minlen > 0 && minlen <= maxlen); /* @@ -1425,7 +1425,7 @@ xfs_rtpick_extent( uint64_t seq; /* sequence number of file creation */ struct timespec64 ts; /* timespec in inode */ - ASSERT(xfs_isilocked(mp->m_rbmip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(mp->m_rbmip, XFS_ILOCK_EXCL); ts = inode_get_atime(VFS_I(mp->m_rbmip)); if (!(mp->m_rbmip->i_diflags & XFS_DIFLAG_NEWRTBM)) { diff --git a/fs/xfs/xfs_symlink.c b/fs/xfs/xfs_symlink.c index 85e433df6a3f..2ca157de4a73 100644 --- a/fs/xfs/xfs_symlink.c +++ b/fs/xfs/xfs_symlink.c @@ -43,7 +43,7 @@ xfs_readlink_bmap_ilocked( int fsblocks = 0; int offset; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_SHARED | XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_SHARED | XFS_ILOCK_EXCL); fsblocks = xfs_symlink_blocks(mp, pathlen); error = xfs_bmapi_read(ip, 0, fsblocks, mval, &nmaps, 0); diff --git a/fs/xfs/xfs_trans_dquot.c b/fs/xfs/xfs_trans_dquot.c index aa00cf67ad72..9c159d016ecf 100644 --- a/fs/xfs/xfs_trans_dquot.c +++ b/fs/xfs/xfs_trans_dquot.c @@ -796,7 +796,7 @@ xfs_trans_reserve_quota_nblks( return 0; ASSERT(!xfs_is_quota_inode(&mp->m_sb, ip->i_ino)); - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); if (force) qflags |= XFS_QMOPT_FORCE_RES; From patchwork Fri Nov 10 20:41:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13452779 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 34F93C4167D for ; Fri, 10 Nov 2023 20:41:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C314F28001D; Fri, 10 Nov 2023 15:41:39 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id BE1DA8D0005; Fri, 10 Nov 2023 15:41:39 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AA9DE28001D; Fri, 10 Nov 2023 15:41:39 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 9BF7D8D0005 for ; Fri, 10 Nov 2023 15:41:39 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 5973F1CBC7A for ; Fri, 10 Nov 2023 20:41:39 +0000 (UTC) X-FDA: 81443215518.19.2992411 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf03.hostedemail.com (Postfix) with ESMTP id 9CAA120006 for ; Fri, 10 Nov 2023 20:41:37 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=NXjTGPNA; dmarc=none; spf=none (imf03.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1699648897; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=xbcUpDlbQzcZtzSAWcv3Q7LJrTanIECAc3h72m1YlgE=; b=hHdpBpCWfPE7mkWWqKNgtLudUfX5bxEnxnDAyRDzYAmy13qjd5aQowEy1kKMeMDf9/dH1f SAiCNs8mwbgWklFVPjFeCevBiO5PBIotfAjN2T5w9o0kfrPYtAIv+PAe/UDN1TaXRb0idl +Wpxz2qV0Mv8ICh75dt2gprzihhvpZg= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=NXjTGPNA; dmarc=none; spf=none (imf03.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1699648897; a=rsa-sha256; cv=none; b=XtbB9L4vBAr+ofvL2Gyec29usz/zS540DwIAm/JNcMu3d4j4oLLK4rg12YuYOZ9uqOwN6D xkI0Ipm6O1lGgwSofdGwsAW2IdVQzXpWx2F17P4Fsgmj2bGLDMuH+gT7zdLz7UzauEW/0m ZrcuXKWK21JTUgUXgz08PRJbj8nnVrM= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=xbcUpDlbQzcZtzSAWcv3Q7LJrTanIECAc3h72m1YlgE=; b=NXjTGPNA2/J+S0B69M9GryHSZy JEUbI2a0TcAYRnGh16+i9v4DFgBnH83q3GPcVmkFYWeoL2adHr3jmgVG2axEPdWLdI+jor9FpoEB4 WOd9Nh4rje6m4Be73wVz7JajVwB7Jan9En5P94sI1bn6IbzMWrOZvu1ynUfk1Exyo9UtvvRaTOuli UT4d0/S9wkysIlvVu4vAARkoMPnt9Ip/T/fESfI7hCRzWsnPwgaNjQq6wxrjMZc/qLz8aiJ6qT2cc HBY6Q7ud4STl3KlqSrCt/oXDuxacuZuLcBHlCvpCK6j8ogowkjqKuJiYsJ7n9Cjn0Ra/nTJqxskGj aq72S09w==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1r1YJd-00FUTA-Bl; Fri, 10 Nov 2023 20:41:21 +0000 From: "Matthew Wilcox (Oracle)" To: Peter Zijlstra , Ingo Molnar , Will Deacon , Waiman Long Cc: "Matthew Wilcox (Oracle)" , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Chandan Babu R , "Darrick J . Wong" , linux-xfs@vger.kernel.org, Mateusz Guzik Subject: [PATCH v3 4/4] xfs: Remove mrlock wrapper Date: Fri, 10 Nov 2023 20:41:19 +0000 Message-Id: <20231110204119.3692023-5-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20231110204119.3692023-1-willy@infradead.org> References: <20231110204119.3692023-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 9CAA120006 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: igw3u3rwtc8tj8gmfdxh7bu465xgedou X-HE-Tag: 1699648897-947870 X-HE-Meta: U2FsdGVkX18ew2aaqwRjHGchZxgi9Q3ECsXPjzV4a44U7JYxlpM5gWRpekL3Z8wa/g5WGP/fR4oAe24ITymiJ75mQYQn0jSAyiL09eKNIy3C2Tub9/RfrIRugZc9cz+HfIWq9ZYyJ3mH9MYzxfo6BimrcjOX5MOoUpZ+DBAoOfeE/ij4v2Og1fSgPJNQQ0a80+6WyXFVC/GlX6wNE5Jzz6O7G+Z4SRLDrmcYQDfNi9RFnguxl91NrWDYQ/Ug0yAoEaHtPY3X/Gao63kbaoAu/ZTWS15v6BO1xpPwjZanzKfVqyxB0WoINymsM3Z6W5IZlFhW2jU6cW3ZBd/GgU0tusgvEFIAXxDs7DeCK/INmtwVPrlVU06yDBbPPydzDdMauggzb9nNJFE3vGE4DIb9BA3X0h5yALFUvId5QhIIVdAHhOQYAOhvmFPz1CSyzFOaW28JJzpcyHKkdh6CPxS+3VMnomiSrXC9Xi1S2UhPiwtaBqXZnV2zf9ihR1APm5gnuD04U6Xx4lT7BBB0C+ng2wZc2szQNXD4q/KvjAQFpC2ZObcE6uEmAuHopIRP9NJevnvAUm3FzkllBIQLFda10t/BkkSc8IrXQLU+B5WnfEEat2SPTLiJYWZGxbwBZcuQcE58k73AQJLWEB5X3H0RP9DfG/R7qIPY+GpA1OD/U9QuUpHlVFbC/8KtY/zRsHtBmPFg66dbjht+KWRw/jgdOiIrZ9hgopN7saZOj9ILPfUdtwZ/zqsK7rU9Dfl2dTvnUgUZlp1YNq1e9++wgJYyKsXH7TdIx8ojsTEfp8yKS1bxbsvVL1y96N6gK7q/N3M1vWpQlkc82b/v1cr1ngEpB8nEu7VNeGFiesuQgNOa9BEQDjJHBlzJ6uZ8yN3gStBplJruoXiH+nbnWGkoNCXxGDW2Na+KDHZQvUndtOsaCOg8lfvZ98Vl0D9B2UkFQI5INsSBuEZ4RLAjx9BBWzG xy7CHUhQ tSIfDu1bMIRdmiN2MSceC8OEE3Hol+VCwoiQLpgOuOEmJ0/lAfQju8u+kRD46K5d4GJ0S/VnHQvCLwwTQk7EVAP66eNJRD98ew+r+/2graLhdaw2wpaJO+nD7Q3SHS5xQO4VDGbDKdJN1X4WZNkpxFzDHN6CU7kklq89EEZOTeiuTGwSfOlVCcnMZxA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: mrlock was an rwsem wrapper that also recorded whether the lock was held for read or write. Now that we can ask the generic code whether the lock is held for read or write, we can remove this wrapper and use an rwsem directly. As the comment says, we can't use lockdep to assert that the ILOCK is held for write, because we might be in a workqueue, and we aren't able to tell lockdep that we do in fact own the lock. Signed-off-by: Matthew Wilcox (Oracle) --- fs/xfs/mrlock.h | 78 ---------------------------------------------- fs/xfs/xfs_inode.c | 22 +++++++------ fs/xfs/xfs_inode.h | 2 +- fs/xfs/xfs_iops.c | 4 +-- fs/xfs/xfs_linux.h | 2 +- fs/xfs/xfs_super.c | 4 +-- 6 files changed, 18 insertions(+), 94 deletions(-) delete mode 100644 fs/xfs/mrlock.h diff --git a/fs/xfs/mrlock.h b/fs/xfs/mrlock.h deleted file mode 100644 index 79155eec341b..000000000000 --- a/fs/xfs/mrlock.h +++ /dev/null @@ -1,78 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * Copyright (c) 2000-2006 Silicon Graphics, Inc. - * All Rights Reserved. - */ -#ifndef __XFS_SUPPORT_MRLOCK_H__ -#define __XFS_SUPPORT_MRLOCK_H__ - -#include - -typedef struct { - struct rw_semaphore mr_lock; -#if defined(DEBUG) || defined(XFS_WARN) - int mr_writer; -#endif -} mrlock_t; - -#if defined(DEBUG) || defined(XFS_WARN) -#define mrinit(mrp, name) \ - do { (mrp)->mr_writer = 0; init_rwsem(&(mrp)->mr_lock); } while (0) -#else -#define mrinit(mrp, name) \ - do { init_rwsem(&(mrp)->mr_lock); } while (0) -#endif - -#define mrlock_init(mrp, t,n,s) mrinit(mrp, n) -#define mrfree(mrp) do { } while (0) - -static inline void mraccess_nested(mrlock_t *mrp, int subclass) -{ - down_read_nested(&mrp->mr_lock, subclass); -} - -static inline void mrupdate_nested(mrlock_t *mrp, int subclass) -{ - down_write_nested(&mrp->mr_lock, subclass); -#if defined(DEBUG) || defined(XFS_WARN) - mrp->mr_writer = 1; -#endif -} - -static inline int mrtryaccess(mrlock_t *mrp) -{ - return down_read_trylock(&mrp->mr_lock); -} - -static inline int mrtryupdate(mrlock_t *mrp) -{ - if (!down_write_trylock(&mrp->mr_lock)) - return 0; -#if defined(DEBUG) || defined(XFS_WARN) - mrp->mr_writer = 1; -#endif - return 1; -} - -static inline void mrunlock_excl(mrlock_t *mrp) -{ -#if defined(DEBUG) || defined(XFS_WARN) - mrp->mr_writer = 0; -#endif - up_write(&mrp->mr_lock); -} - -static inline void mrunlock_shared(mrlock_t *mrp) -{ - up_read(&mrp->mr_lock); -} - -static inline void mrdemote(mrlock_t *mrp) -{ -#if defined(DEBUG) || defined(XFS_WARN) - mrp->mr_writer = 0; -#endif - downgrade_write(&mrp->mr_lock); -} - -#endif /* __XFS_SUPPORT_MRLOCK_H__ */ diff --git a/fs/xfs/xfs_inode.c b/fs/xfs/xfs_inode.c index 1ed6bed19bec..7661084ca568 100644 --- a/fs/xfs/xfs_inode.c +++ b/fs/xfs/xfs_inode.c @@ -208,9 +208,9 @@ xfs_ilock( } if (lock_flags & XFS_ILOCK_EXCL) - mrupdate_nested(&ip->i_lock, XFS_ILOCK_DEP(lock_flags)); + down_write_nested(&ip->i_lock, XFS_ILOCK_DEP(lock_flags)); else if (lock_flags & XFS_ILOCK_SHARED) - mraccess_nested(&ip->i_lock, XFS_ILOCK_DEP(lock_flags)); + down_read_nested(&ip->i_lock, XFS_ILOCK_DEP(lock_flags)); } /* @@ -251,10 +251,10 @@ xfs_ilock_nowait( } if (lock_flags & XFS_ILOCK_EXCL) { - if (!mrtryupdate(&ip->i_lock)) + if (!down_write_trylock(&ip->i_lock)) goto out_undo_mmaplock; } else if (lock_flags & XFS_ILOCK_SHARED) { - if (!mrtryaccess(&ip->i_lock)) + if (!down_read_trylock(&ip->i_lock)) goto out_undo_mmaplock; } return 1; @@ -303,9 +303,9 @@ xfs_iunlock( up_read(&VFS_I(ip)->i_mapping->invalidate_lock); if (lock_flags & XFS_ILOCK_EXCL) - mrunlock_excl(&ip->i_lock); + up_write(&ip->i_lock); else if (lock_flags & XFS_ILOCK_SHARED) - mrunlock_shared(&ip->i_lock); + up_read(&ip->i_lock); trace_xfs_iunlock(ip, lock_flags, _RET_IP_); } @@ -324,7 +324,7 @@ xfs_ilock_demote( ~(XFS_IOLOCK_EXCL|XFS_MMAPLOCK_EXCL|XFS_ILOCK_EXCL)) == 0); if (lock_flags & XFS_ILOCK_EXCL) - mrdemote(&ip->i_lock); + downgrade_write(&ip->i_lock); if (lock_flags & XFS_MMAPLOCK_EXCL) downgrade_write(&VFS_I(ip)->i_mapping->invalidate_lock); if (lock_flags & XFS_IOLOCK_EXCL) @@ -338,10 +338,14 @@ xfs_assert_ilocked( struct xfs_inode *ip, uint lock_flags) { + /* + * Sometimes we assert the ILOCK is held exclusively, but we're in + * a workqueue, so lockdep doesn't know we're the owner. + */ if (lock_flags & XFS_ILOCK_SHARED) - rwsem_assert_held(&ip->i_lock.mr_lock); + rwsem_assert_held(&ip->i_lock); else if (lock_flags & XFS_ILOCK_EXCL) - ASSERT(ip->i_lock.mr_writer); + rwsem_assert_held_write_nolockdep(&ip->i_lock); if (lock_flags & XFS_MMAPLOCK_SHARED) rwsem_assert_held(&VFS_I(ip)->i_mapping->invalidate_lock); diff --git a/fs/xfs/xfs_inode.h b/fs/xfs/xfs_inode.h index dd8b8339ba1b..3cade5903fda 100644 --- a/fs/xfs/xfs_inode.h +++ b/fs/xfs/xfs_inode.h @@ -39,7 +39,7 @@ typedef struct xfs_inode { /* Transaction and locking information. */ struct xfs_inode_log_item *i_itemp; /* logging information */ - mrlock_t i_lock; /* inode lock */ + struct rw_semaphore i_lock; /* inode lock */ atomic_t i_pincount; /* inode pin count */ struct llist_node i_gclist; /* deferred inactivation list */ diff --git a/fs/xfs/xfs_iops.c b/fs/xfs/xfs_iops.c index e711668d601b..79010d60ee21 100644 --- a/fs/xfs/xfs_iops.c +++ b/fs/xfs/xfs_iops.c @@ -1284,9 +1284,9 @@ xfs_setup_inode( */ lockdep_set_class(&inode->i_rwsem, &inode->i_sb->s_type->i_mutex_dir_key); - lockdep_set_class(&ip->i_lock.mr_lock, &xfs_dir_ilock_class); + lockdep_set_class(&ip->i_lock, &xfs_dir_ilock_class); } else { - lockdep_set_class(&ip->i_lock.mr_lock, &xfs_nondir_ilock_class); + lockdep_set_class(&ip->i_lock, &xfs_nondir_ilock_class); } /* diff --git a/fs/xfs/xfs_linux.h b/fs/xfs/xfs_linux.h index d7873e0360f0..ec3c6c138a63 100644 --- a/fs/xfs/xfs_linux.h +++ b/fs/xfs/xfs_linux.h @@ -22,7 +22,6 @@ typedef __u32 xfs_nlink_t; #include "xfs_types.h" #include "kmem.h" -#include "mrlock.h" #include #include @@ -51,6 +50,7 @@ typedef __u32 xfs_nlink_t; #include #include #include +#include #include #include #include diff --git a/fs/xfs/xfs_super.c b/fs/xfs/xfs_super.c index 764304595e8b..cd0200ed51f0 100644 --- a/fs/xfs/xfs_super.c +++ b/fs/xfs/xfs_super.c @@ -724,9 +724,7 @@ xfs_fs_inode_init_once( /* xfs inode */ atomic_set(&ip->i_pincount, 0); spin_lock_init(&ip->i_flags_lock); - - mrlock_init(&ip->i_lock, MRLOCK_ALLOW_EQUAL_PRI|MRLOCK_BARRIER, - "xfsino", ip->i_ino); + init_rwsem(&ip->i_lock); } /*