From patchwork Thu Apr 10 01:49:39 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luis Chamberlain X-Patchwork-Id: 14045749 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5FD8C369A6 for ; Thu, 10 Apr 2025 01:50:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 741006B024D; Wed, 9 Apr 2025 21:49:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1D9AF6B0250; Wed, 9 Apr 2025 21:49:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A0F396B0250; Wed, 9 Apr 2025 21:49:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 50AE46B024E for ; Wed, 9 Apr 2025 21:49:57 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 8E4D781817 for ; Thu, 10 Apr 2025 01:49:58 +0000 (UTC) X-FDA: 83316453276.09.3D95D28 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf23.hostedemail.com (Postfix) with ESMTP id D758D140006 for ; Thu, 10 Apr 2025 01:49:56 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b="b/XLETIB"; dmarc=fail reason="No valid SPF, DKIM not aligned (relaxed)" header.from=kernel.org (policy=quarantine); spf=none (imf23.hostedemail.com: domain of mcgrof@infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=mcgrof@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744249796; a=rsa-sha256; cv=none; b=2/8XJDndemrwlC4Dg+5kQcaW3/aihSLZC3nxfIIJFMNEAlCAINrgu71gISzFov0iOguoFe ussXbXB3ynIhxKp1IqVahsjwQ6JwhZu+G6UXPhra0z2hk+tFsTnSW8t1WlbrFb/6n6PJDD mRTcPfi+kHgrySCkQv9K9zZk0xNx5XI= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b="b/XLETIB"; dmarc=fail reason="No valid SPF, DKIM not aligned (relaxed)" header.from=kernel.org (policy=quarantine); spf=none (imf23.hostedemail.com: domain of mcgrof@infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=mcgrof@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744249796; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=1ZkQCLEfvSJJMcZUEvIWp91J3xZQH1qlm2s7wYby9N4=; b=db+fIVI0fGH2OofvoR1FHyjMkgaktgug2h23VgzTlYQKM2Sncb52peAM9//dLZhE0hYVHu 15ZYzMiuqWa+7KQgoTeNeovLy1GsSqq+1Tm2QTCt8JyKx19hiyINbo+WOhZsXcnR/weu+S eiOP55foocWH0GngfkqUVob96SCtq4g= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=1ZkQCLEfvSJJMcZUEvIWp91J3xZQH1qlm2s7wYby9N4=; b=b/XLETIB5KCHfGv4ObhFsAtZeg 9PIOhDozbrjIiwkz8FCrVudBNh2xFJuJPCMRAZeWfEuT0B65+YA0mPvNkelPKXGYW70ztF80jFtrk +zk3ijWBp87lt6bqq3DJhsxWX9gTuF74Jnd9NmvK1ryYmCOivexotCIUtVE5uDZGyi3XDwy0R42J6 hmwni0rd8Rx/vXlbvB/t2/Lj/REqtA8EpzSkQsol1GK/rc9TKvQvQBSVIAHKqYN5G9bju7WsHJS9q Qc3PJM/BXxCWo0oGP+Feqe48qlCXA0oi9dt8kdTsUCrc9P/4T1Apl74rogmqpTvs/ISX1C7WGRDtl KWtwGWeQ==; Received: from mcgrof by bombadil.infradead.org with local (Exim 4.98.2 #2 (Red Hat Linux)) id 1u2h35-00000008yvH-3Pl7; Thu, 10 Apr 2025 01:49:47 +0000 From: Luis Chamberlain To: brauner@kernel.org, jack@suse.cz, tytso@mit.edu, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, riel@surriel.com Cc: dave@stgolabs.net, willy@infradead.org, hannes@cmpxchg.org, oliver.sang@intel.com, david@redhat.com, axboe@kernel.dk, hare@suse.de, david@fromorbit.com, djwong@kernel.org, ritesh.list@gmail.com, linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-mm@kvack.org, gost.dev@samsung.com, p.raghav@samsung.com, da.gomez@samsung.com, mcgrof@kernel.org Subject: [PATCH v2 2/8] fs/buffer: try to use folio lock for pagecache lookups Date: Wed, 9 Apr 2025 18:49:39 -0700 Message-ID: <20250410014945.2140781-3-mcgrof@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250410014945.2140781-1-mcgrof@kernel.org> References: <20250410014945.2140781-1-mcgrof@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: D758D140006 X-Stat-Signature: 5dfwhun5bmk49gqseqfomwq63cn9ifnq X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Pre-Result: action=add header; module=dmarc; Action set by DMARC X-Rspam: Yes X-HE-Tag: 1744249796-705245 X-HE-Meta: U2FsdGVkX19VQ3JLv4kFcprW35ypw01Zd6kjZuyU9res2GyGn9yOvUDqZTPAbmqxZcLeG55g5h8ss9XcYdCeKjK+EcqTjsRhWb4/lpC3zW8D6ZeeI88PbANzzIPTRoCrrCxGQ+UlfUBWb5vgWB1Pz+6zowuPRjBHkRb18bSuICXPpoHZ4Dxp3cRxVu3YopwBdme40WK3OrijjgfWeXkZJUjBTQw9IKNC8miIF061kOg1DUiF3T2GMtHUrmiwzEmeJQ9pZEyLQ8gL+gM/9do6wuQD7MLDC8eKK9NQrLRoi+gcOTJpAKdj4FyoxTotyIk6KbbhuoUDYfayB5Q09YCMgzz3j7IQHvRBiaOznoIpozD0PMi/141yqwQi+Pw6KbTfVe+L1B8MRf+parBgT10GR8qF1GDs9OJekD+ZkwOs5IFPgo9gqe7IjnXAJAPjgAf9NXBS5y1ZXOr+jpLgMeJtNoHoyHad9A2D26lfqy08EgXlVQbjjOTTT0YKFVzTIkVAKKloPnNwZMTnvA86HKdkFJI3k3UUoJ7EH3u1khHtqWBXIelkwkotnXWNDvS4tJACHP1QT1hnaZ1cDQid0fvo9dp7YYakK71OAyqFAYCcLbgT5F2241vyEzzXkzWT6cRfbc+a7w5z7G7PXQHzeL+AbpAgN/lVtWRpVsv7aXjQMh9k0vdujgmznPeeypIk+8pWZzM1C1xhL76umuOeGa2Thj06IaTiyphlj3IpNFsYfcvEI8JsHF7+kPJPsgk/oGR1DAOdpYZMh/pc5v6+KW94QC0BDeiphiwT2887fsTs4cSwYhe09AY3pzwwYXL1s+46BPVnYGJp68oQlreugKqk5Woqw86eIyUjjjXoDb/IahtBqKiPdTRFkeBleQgo9soaPUiJ/RPWnfnnq0tDyfcsqjsPJSaJjeAzop9JD9oHd5TA7wVsHS1DoB7nCsPmq5A5YOOf7hivuGZt/c99mdL w0pjSkPR rjd6RYFkiWz2i0qFLBe5YGKWdcVPK5iVZcd4vzvR7e6PCGRKntEvafrSDaXU5KRpstuBes6oOnYLb++P7HW5nmNeFCM6Nss4LL+tLnPW68KHgAv+M2I84FobxP/yrbvNSmAVYQ9J07FUjmoI82RRqiYgSsjOCBQQBl2FfT2zY7LzEmKD3YGjoTGwGzXdiuD2jeGI1wNxerCtaO53ygWQFa/K087J0fk3FS/n3l7HXuM2ybM6DwvXpWPpkz+uwEBokwrkzeOUZ0laqyxyY1js+V7sq7UI+/FAgsw9YDnJAiw3cb/mQ0t3SjoJB43FAPBTCNcF83SWq7ipo9emltl4V5D1LtGf5r690vH1gDEDlGS04a5zV6tUaa+vsz2fIqyvbRJiTN9uGdlWydzVMApTzKB6xfH/7nm1ihfNwS5PWubRmO2FzpghQlzAVo50DEZg0B9qOPhI3yTbrv8Q8kPp6EIA8u6nsvo0H+GvvdmalQ01Z45L0/T4qPE7WnKCuwcpCykHOkxEANeUyVCmcsZSJwY1OE4TJ8E/gQNEmInlCBglXQwgwnN1fSSvwzw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Davidlohr Bueso Callers of __find_get_block() may or may not allow for blocking semantics, and is currently assumed that it will not. Layout two paths based on this. Ultimately the i_private_lock scheme will be used as a fallback in non-blocking contexts. Otherwise always take the folio lock instead. The suggested trylock idea is implemented, thereby potentially reducing i_private_lock contention in addition to later enabling future migration support around with large folios and noref migration. No change in semantics. All lookup users are non-blocking. Signed-off-by: Davidlohr Bueso Signed-off-by: Luis Chamberlain --- fs/buffer.c | 45 +++++++++++++++++++++++++++++---------------- 1 file changed, 29 insertions(+), 16 deletions(-) diff --git a/fs/buffer.c b/fs/buffer.c index c7abb4a029dc..5a1a37a6840a 100644 --- a/fs/buffer.c +++ b/fs/buffer.c @@ -176,18 +176,8 @@ void end_buffer_write_sync(struct buffer_head *bh, int uptodate) } EXPORT_SYMBOL(end_buffer_write_sync); -/* - * Various filesystems appear to want __find_get_block to be non-blocking. - * But it's the page lock which protects the buffers. To get around this, - * we get exclusion from try_to_free_buffers with the blockdev mapping's - * i_private_lock. - * - * Hack idea: for the blockdev mapping, i_private_lock contention - * may be quite high. This code could TryLock the page, and if that - * succeeds, there is no need to take i_private_lock. - */ static struct buffer_head * -__find_get_block_slow(struct block_device *bdev, sector_t block) +__find_get_block_slow(struct block_device *bdev, sector_t block, bool atomic) { struct address_space *bd_mapping = bdev->bd_mapping; const int blkbits = bd_mapping->host->i_blkbits; @@ -197,6 +187,7 @@ __find_get_block_slow(struct block_device *bdev, sector_t block) struct buffer_head *head; struct folio *folio; int all_mapped = 1; + bool folio_locked = true; static DEFINE_RATELIMIT_STATE(last_warned, HZ, 1); index = ((loff_t)block << blkbits) / PAGE_SIZE; @@ -204,7 +195,19 @@ __find_get_block_slow(struct block_device *bdev, sector_t block) if (IS_ERR(folio)) goto out; - spin_lock(&bd_mapping->i_private_lock); + /* + * Folio lock protects the buffers. Callers that cannot block + * will fallback to serializing vs try_to_free_buffers() via + * the i_private_lock. + */ + if (!folio_trylock(folio)) { + if (atomic) { + spin_lock(&bd_mapping->i_private_lock); + folio_locked = false; + } else + folio_lock(folio); + } + head = folio_buffers(folio); if (!head) goto out_unlock; @@ -236,7 +239,10 @@ __find_get_block_slow(struct block_device *bdev, sector_t block) 1 << blkbits); } out_unlock: - spin_unlock(&bd_mapping->i_private_lock); + if (folio_locked) + folio_unlock(folio); + else + spin_unlock(&bd_mapping->i_private_lock); folio_put(folio); out: return ret; @@ -1388,14 +1394,15 @@ lookup_bh_lru(struct block_device *bdev, sector_t block, unsigned size) * it in the LRU and mark it as accessed. If it is not present then return * NULL */ -struct buffer_head * -__find_get_block(struct block_device *bdev, sector_t block, unsigned size) +static struct buffer_head * +find_get_block_common(struct block_device *bdev, sector_t block, + unsigned size, bool atomic) { struct buffer_head *bh = lookup_bh_lru(bdev, block, size); if (bh == NULL) { /* __find_get_block_slow will mark the page accessed */ - bh = __find_get_block_slow(bdev, block); + bh = __find_get_block_slow(bdev, block, atomic); if (bh) bh_lru_install(bh); } else @@ -1403,6 +1410,12 @@ __find_get_block(struct block_device *bdev, sector_t block, unsigned size) return bh; } + +struct buffer_head * +__find_get_block(struct block_device *bdev, sector_t block, unsigned size) +{ + return find_get_block_common(bdev, block, size, true); +} EXPORT_SYMBOL(__find_get_block); /**