From patchwork Fri Apr 18 01:59:15 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Davidlohr Bueso X-Patchwork-Id: 14056539 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A2037C369CF for ; Fri, 18 Apr 2025 01:59:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9C53F6B029F; Thu, 17 Apr 2025 21:59:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9752C6B02A1; Thu, 17 Apr 2025 21:59:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6DD7C6B02A3; Thu, 17 Apr 2025 21:59:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 4AF886B029F for ; Thu, 17 Apr 2025 21:59:37 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 47E0F1CBBF9 for ; Fri, 18 Apr 2025 01:59:38 +0000 (UTC) X-FDA: 83345508036.08.0E7EE84 Received: from serval.cherry.relay.mailchannels.net (serval.cherry.relay.mailchannels.net [23.83.223.163]) by imf28.hostedemail.com (Postfix) with ESMTP id 0C6E2C0004 for ; Fri, 18 Apr 2025 01:59:35 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=stgolabs.net header.s=dreamhost header.b=p1xyuGdf; dmarc=none; spf=pass (imf28.hostedemail.com: domain of dave@stgolabs.net designates 23.83.223.163 as permitted sender) smtp.mailfrom=dave@stgolabs.net; arc=pass ("mailchannels.net:s=arc-2022:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744941576; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=l0OkMrrCNfiNSUNqllJ63trdw8uvDJBD9sB7Chep1qQ=; b=q/JChSWP+Ti3FHIi0ZwY5fRvppkPlv9UPY9qDpxgNPLE+wfl+R7LxPwS43txBdV97qDHEy q3nzTlAxxArg2j/V2jy2vkEBhq6RGUVQZD0xzhPXwG9MitTGzycDW5SBE0+3d6purHILxe p0uZz41Wf/kSEdZLKFoLqChs7ZnQI+c= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1744941576; a=rsa-sha256; cv=pass; b=g8cz5sK99ATmV8d9eIZEt1HzGNGtViFvLs78aCn6vk7GccoJuyaaKvCNhOJHcg6GHot6af PqXhJawhn2ha6aSIFUYflFw3FDJMxNqluGRyJpo6yLacpQtFikb0Uk11fXtqEwAkrFysl/ dnXmkHM8Z2SVZZREXOc66jS7z9BmzIs= ARC-Authentication-Results: i=2; imf28.hostedemail.com; dkim=pass header.d=stgolabs.net header.s=dreamhost header.b=p1xyuGdf; dmarc=none; spf=pass (imf28.hostedemail.com: domain of dave@stgolabs.net designates 23.83.223.163 as permitted sender) smtp.mailfrom=dave@stgolabs.net; arc=pass ("mailchannels.net:s=arc-2022:i=1") X-Sender-Id: dreamhost|x-authsender|dave@stgolabs.net Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id DA7C04E3E21; Fri, 18 Apr 2025 01:59:34 +0000 (UTC) Received: from pdx1-sub0-mail-a285.dreamhost.com (100-107-95-229.trex-nlb.outbound.svc.cluster.local [100.107.95.229]) (Authenticated sender: dreamhost) by relay.mailchannels.net (Postfix) with ESMTPA id 76E904E3C8C; Fri, 18 Apr 2025 01:59:34 +0000 (UTC) ARC-Seal: i=1; s=arc-2022; d=mailchannels.net; t=1744941574; a=rsa-sha256; cv=none; b=nTINgWWCpNRS6h4rQ+hDdWix5ieHLh2YvkU/HGmrRHhrLXnmmFZYLGoAKqREEQLf6NlqEo IzGtcgb2AnszYccBiRaabLwBIwo7+lMSPD3JJ5wICmwx/PiSzTOuJ+5o4y51QKPJuAJpkX KKIZXiOJwxomBWoCnIbE9vXkCbEElRWRtxp9iAhedw/AtxYHXrFD3P8ChDcfUVXY/gZU06 kQj0eWMmb2Ziic72MAVTxz1tOeT6r4jhxkFEa/LNf11beexjbvymPWznMDidxHEb3vcAlm AQKslS6e9CGQIS2daqtl5icWh5OFYhmfhYgRXpoflG0yiZTpA/Vi35hA7hrMBw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=mailchannels.net; s=arc-2022; t=1744941574; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=l0OkMrrCNfiNSUNqllJ63trdw8uvDJBD9sB7Chep1qQ=; b=4D5orQYmCxpExXLwDca3eoowuwJ5J7hePJCnS6NJ7850Xdly79QCPLRnrjnpcQo8i+RAFw OoMYQQUncr+AJoKJvYGbQmClNFpFF4ZRwe3qJ1GEzg7HWtFu2x133WRCWQwsgTLQ9hECGy 3XyoweS1nFZbrx4O1vtDjBOfalspQ/FkZ6GuFv35y22WCHvmEN9m9JlQL/uX7FndLspiRO XjjYrgIZF9n9eq3iP3bPXe8Z1LJil829bqpCxwTrHD1vRBcibyx2Bc/PkHOH1twekBw3NF /6K0m97KKSmHZvSxArWSieBCHyoG2CsQlb7SaYIpJluIPoPCxSItWlFZ4meL+w== ARC-Authentication-Results: i=1; rspamd-7bd9ff6c58-mj7hx; auth=pass smtp.auth=dreamhost smtp.mailfrom=dave@stgolabs.net X-Sender-Id: dreamhost|x-authsender|dave@stgolabs.net X-MC-Relay: Neutral X-MailChannels-SenderId: dreamhost|x-authsender|dave@stgolabs.net X-MailChannels-Auth-Id: dreamhost X-Broad-White: 7d820e6c16f23b4a_1744941574795_2082964327 X-MC-Loop-Signature: 1744941574795:136927829 X-MC-Ingress-Time: 1744941574795 Received: from pdx1-sub0-mail-a285.dreamhost.com (pop.dreamhost.com [64.90.62.162]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384) by 100.107.95.229 (trex/7.0.3); Fri, 18 Apr 2025 01:59:34 +0000 Received: from localhost.localdomain (ip72-199-50-187.sd.sd.cox.net [72.199.50.187]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: dave@stgolabs.net) by pdx1-sub0-mail-a285.dreamhost.com (Postfix) with ESMTPSA id 4Zdycj3m31zC4; Thu, 17 Apr 2025 18:59:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=stgolabs.net; s=dreamhost; t=1744941574; bh=l0OkMrrCNfiNSUNqllJ63trdw8uvDJBD9sB7Chep1qQ=; h=From:To:Cc:Subject:Date:Content-Transfer-Encoding; b=p1xyuGdfjY+kIZiUDVQxbDoSeumqeD6CRGZS5xt8zFdXGLyWeymMlwNP2bcj49nSW SRsKRijsknB7yuhD91VoidYVvK5Pd4U3BBlR6uxo1nLW76h+jtKoK2j9kJsZlRIqXU QnjWjkTfEWWLLTIQyz9nMXvH6bDSgkKDTT85dGUeAb9SPZ3D49XBTWYQnuSWyCwr1N oaUKT6pKiNEliKnxRCD2Mdc7nnhNcM/aA3L8ETWJpgAnN3cbSfNddVgxTMqPbNdOpz Glib8yBke2/A1WzVVrmTlaUBvp7vEgQ5gCvGpoIy9wfqFonA3uxgDH0RfC2B1ETaKE q96Ib+xiJLoHw== From: Davidlohr Bueso To: jack@suse.cz, tytso@mit.edu, adilger.kernel@dilger.ca, brauner@kernel.org Cc: mcgrof@kernel.org, willy@infradead.org, hare@suse.de, djwong@kernel.org, linux-ext4@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, Davidlohr Bueso Subject: [PATCH 1/7] fs/buffer: split locking for pagecache lookups Date: Thu, 17 Apr 2025 18:59:15 -0700 Message-Id: <20250418015921.132400-2-dave@stgolabs.net> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250418015921.132400-1-dave@stgolabs.net> References: <20250418015921.132400-1-dave@stgolabs.net> MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 0C6E2C0004 X-Rspam-User: X-Stat-Signature: w8d7bc63oh56m945ax3esqyibpe3dkm8 X-HE-Tag: 1744941575-28719 X-HE-Meta: U2FsdGVkX1/Dgo/N86M0dmOxD8Z5M5VwlrvDgrhGAkQUhk40aTf+0eBdJDXKk5JQMCFM5cK5inzFK4nPckWLS0YnkLt60pdwUZIls0Z2wKRKjL8wqnK01wr32IBDL55mJh10AwyF+zJdXqWTfeYrm5jd4VuOlr0dv/VMCgG0aJ1K5z7oc8oRxcXGwWntOz8aJ6oA7+aFTVBOxlFwZJ28TTDrkRmx910LmieXkLpmHd08R5bvxRrQxGm/X6BszPneQEeR1wFLwW4Z0/svQo8P3Mehi77ZmscB7m4K08Sj0SIZgnBt60CyDa098vPq3VgsgtpkSPzEmNZ5XsSzCfEBUzjc9waqwbo0kZnU5QcpJ4N8RmpvWGdoCZoe9plBTcR4aFxveI2WW6/OEpDcYV2SizgFMbq3zXNNDqh825PU8wmLnS1sQ7xJTNXgwBHtvRAxHqb/r2hsjD9OI82qjqN06rk424JkhzF5BCbxj+VCR7c2w1oWzUVc8NigF4QDMWdE3vAxd+r5b1YOJaGWnFZPOVyQ8Dfu+Nq2DOI5hOTIZNK3d7fvKKjslAqe91315QO6/k5VtVY3GbQSDkdRoJyPehYcHSZouOq7ekrm24nzdoSftY1i2JYxWnOyuZEpaEZIbD2BuDvtNeg6lQSb+VyEiFZ6JifJmCOMDfiptSLVYtUfqJcLJ0V7EnIL1bYtvarkFDyIEuUonD0gbSuZe2b84/5fq7+EZA4HuSS/ZWOE9yyXiaxsclQ3eySSRQKCrK/+PBsu660cwjdnojBS/yhxjYEOzX0CEd17PI+/XcBQUmlKZDwoWAsy1XsCiGc3yefNwziHkU96MoeLSWiYEBbi+O3wurkmGLHhbHSEvL77weOz5KyYOHlulmBcfCOYVgqLAW4itlJVHwhLg4B2NDh/xDeBTrHfM7dQ552ejglnvHpIDbgHjLYwWdF9tS3ZlVD/OynXXmWChXeUlccggPl 4v0wJ3VS 6lzBYJUODTRIkzWgg1oeVca4P/vdEcowo9lZ/O4DC+ADw3IXO5QXCYY7BH3GS6VXW84wYi+H4FmglNMsGjEkpFWsYNG3aVPaxXO2HNnAl6t7BJDzfVgJpY3ITHSJvXsoBLci3Of1fZ2zHiNS1x3Hhz7fYU1s3TXdya8y4TjzA+o4gfuqbnWCuV7X+WrhR4xAYjXPJv0EjesaLVahI8ht0KPSe3GsMiHSrM4mgjGM4+tNxdmzNzR32j0U2oLXvTGVxA2HVudPGZq2mS09KwkKfYPWrJI9WfGsMvyfORSKpAjU19eZOiMnqVM5tlS4fW6n9kRxtumYgqQvHLi/vMBg3uSG8GQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Callers of __find_get_block() may or may not allow for blocking semantics, and is currently assumed that it will not. Layout two paths based on this. The the private_lock scheme will continued to be used for atomic contexts. Otherwise take the folio lock instead, which protects the buffers, such as vs migration and try_to_free_buffers(). Per the "hack idea", the latter can alleviate contention on the private_lock for bdev mappings. For reasons of determinism and avoid making bugs hard to reproduce, the trylocking is not attempted. No change in semantics. All lookup users still take the spinlock. Reviewed-by: Jan Kara Signed-off-by: Davidlohr Bueso --- fs/buffer.c | 41 +++++++++++++++++++++++++---------------- 1 file changed, 25 insertions(+), 16 deletions(-) diff --git a/fs/buffer.c b/fs/buffer.c index c7abb4a029dc..f8fcffdbe5d9 100644 --- a/fs/buffer.c +++ b/fs/buffer.c @@ -176,18 +176,8 @@ void end_buffer_write_sync(struct buffer_head *bh, int uptodate) } EXPORT_SYMBOL(end_buffer_write_sync); -/* - * Various filesystems appear to want __find_get_block to be non-blocking. - * But it's the page lock which protects the buffers. To get around this, - * we get exclusion from try_to_free_buffers with the blockdev mapping's - * i_private_lock. - * - * Hack idea: for the blockdev mapping, i_private_lock contention - * may be quite high. This code could TryLock the page, and if that - * succeeds, there is no need to take i_private_lock. - */ static struct buffer_head * -__find_get_block_slow(struct block_device *bdev, sector_t block) +__find_get_block_slow(struct block_device *bdev, sector_t block, bool atomic) { struct address_space *bd_mapping = bdev->bd_mapping; const int blkbits = bd_mapping->host->i_blkbits; @@ -204,7 +194,16 @@ __find_get_block_slow(struct block_device *bdev, sector_t block) if (IS_ERR(folio)) goto out; - spin_lock(&bd_mapping->i_private_lock); + /* + * Folio lock protects the buffers. Callers that cannot block + * will fallback to serializing vs try_to_free_buffers() via + * the i_private_lock. + */ + if (atomic) + spin_lock(&bd_mapping->i_private_lock); + else + folio_lock(folio); + head = folio_buffers(folio); if (!head) goto out_unlock; @@ -236,7 +235,10 @@ __find_get_block_slow(struct block_device *bdev, sector_t block) 1 << blkbits); } out_unlock: - spin_unlock(&bd_mapping->i_private_lock); + if (atomic) + spin_unlock(&bd_mapping->i_private_lock); + else + folio_unlock(folio); folio_put(folio); out: return ret; @@ -1388,14 +1390,15 @@ lookup_bh_lru(struct block_device *bdev, sector_t block, unsigned size) * it in the LRU and mark it as accessed. If it is not present then return * NULL */ -struct buffer_head * -__find_get_block(struct block_device *bdev, sector_t block, unsigned size) +static struct buffer_head * +find_get_block_common(struct block_device *bdev, sector_t block, + unsigned size, bool atomic) { struct buffer_head *bh = lookup_bh_lru(bdev, block, size); if (bh == NULL) { /* __find_get_block_slow will mark the page accessed */ - bh = __find_get_block_slow(bdev, block); + bh = __find_get_block_slow(bdev, block, atomic); if (bh) bh_lru_install(bh); } else @@ -1403,6 +1406,12 @@ __find_get_block(struct block_device *bdev, sector_t block, unsigned size) return bh; } + +struct buffer_head * +__find_get_block(struct block_device *bdev, sector_t block, unsigned size) +{ + return find_get_block_common(bdev, block, size, true); +} EXPORT_SYMBOL(__find_get_block); /**