From patchwork Wed Oct 4 16:53:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13409066 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8E60CE7C4E2 for ; Wed, 4 Oct 2023 16:53:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 26AB36B026E; Wed, 4 Oct 2023 12:53:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 131C26B026F; Wed, 4 Oct 2023 12:53:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DCD096B026F; Wed, 4 Oct 2023 12:53:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id CA50F6B026E for ; Wed, 4 Oct 2023 12:53:24 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 916691A03D1 for ; Wed, 4 Oct 2023 16:53:24 +0000 (UTC) X-FDA: 81308374728.19.E502F0B Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf25.hostedemail.com (Postfix) with ESMTP id C0FF0A0027 for ; Wed, 4 Oct 2023 16:53:22 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=Kf1rS9d9; dmarc=none; spf=none (imf25.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696438402; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=8+vjrjpTMU20NjTh21ymbSWyQ+A6oOAP0PoW8Flxz4A=; b=Z4AMq1aEu7q+cD0waapP50VpVaWOhxjRFAOso7ZuxmVXgK+JuyVqBr9Hdhor0zph3zDrI5 yLInR+1GAEZ/hFwZksLhtP2eDnGL+CmQOTs94TsRz77k2ktNvCKuF40kKrIlil1HBDXkwm IthEaTYK34Xzvkk9kW9fiVSfDmIF0bA= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=Kf1rS9d9; dmarc=none; spf=none (imf25.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696438402; a=rsa-sha256; cv=none; b=k0h8MxYj0OfoMv5eWP0j6RHUDiRhPPNBa1fPrq2ZR0boRA6pFOEtrkxvLJOpHsRQccETTI B5J2tOXjJAZvTGpAMQnU7dHxNUC7MYbR7XQnjOGalQpVY4b2mNAVstHqVdWiR4IoszKiBo fA/slOhK5sfVgQ79oaL1SWqqkveTrQo= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=8+vjrjpTMU20NjTh21ymbSWyQ+A6oOAP0PoW8Flxz4A=; b=Kf1rS9d9vz2PJFuFKvV7rph0p1 HCyHPGHA+FbW1KAMoLhpQu/T9u3tpDlH9vyzTv4mDkTX0yV9Dpe2uN3ldxUDZ+9/i+xmwogS5qTLX kqFEqo9od39oL/2d6fXstvwLvJTaE4kIBipi7WvWwrlGAP/Qsrxq0k8cweo7AhCb9Y7zHrF2PA72x D9qgc0osOC51gblLOkrYKh983huTeODe/DozdfPHgqA/F5MUrc8QAVjCmkZY6+l0ikP6FRn5y5JLs IxZNcoqdFknci4ZLy6qYxJXmgtgyvGn5D5UmxyUKH2SeMwDmUoeBQ4QZRdI8YDQgrgROf03t6vi4R La+kEGjg==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1qo57h-004SGM-0x; Wed, 04 Oct 2023 16:53:21 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, torvalds@linux-foundation.org, npiggin@gmail.com Subject: [PATCH v2 17/17] mm: Use folio_xor_flags_has_waiters() in folio_end_writeback() Date: Wed, 4 Oct 2023 17:53:17 +0100 Message-Id: <20231004165317.1061855-18-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20231004165317.1061855-1-willy@infradead.org> References: <20231004165317.1061855-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: C0FF0A0027 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: h8hifuqorw38hysuj7gtta64b5mt36oe X-HE-Tag: 1696438402-171243 X-HE-Meta: U2FsdGVkX19m1QsudW0+HdPfuk/fZmzzAFl8MJckxEPOzx0kg7dzUxwa7O8Nszv0L1+Gdcs5c9nUZHQ6JivyAyo+yPlvfR+jjdICRrA2aNJzAM62rk+606vrwovD+oWc9BxMFXnTxmK01mKpWUnxBG1L0cz7nq30FmLWE6+TM3qBfetYLepOTONXz/96/z0oSPMru3QNzMLEKWm+WnDvxkNPQcz11b1LilAIxjPhHJb/3MXwUO04VuQOmMizSZ4VwhnyPac7IpzTFe+G9KyAsNGXKcgoPI/s1jFJKSBlkN/4dGEs1ipxsUlCIZ9xSLORsXeUqWH3X4FpNeN6SRsl4FfB4p7OgLH1zxvRuPe/254AeK0oe3fsaSib1w/z2E0RUWIlnJ9TleXYnfhtX+VaLe6PN20c6afaWKpKCPht/5P9F334FRUq6zgjsllDyBN2mJgEzH7AvmX5tYwQ03rjmR93EoJp5nvZx71WfiRjE96wg8DfQJb3P0332b97UTV/fFjgZZLbfpcMOCi6kJ0BKa92weB0JShzHTTpmWXzPzOvbGOPeAcMYm7YUsL8IUFIsVZOo7XENQi4+P8l6kf3Z+pUPavIhKODHs8zQYm5ASYOvIWWbLQTyLYmEDLC8ppvm4ertKqR3x5IeglMt8EaEjGXFQalI51y31UoIMz/MvdqdUZhSN6146FahMSnxAZy50bgrXQ/vEL5SBbv6NAEGdI6J8lUOqqwMkAZ2BLxD5Fkxc3cTvVoDBEq5PtOuqC5JsaH/L35Wi043v69m4rJbv46FHVD+sLN+2RcPIWXWjf2U8yMMw8BhlGv9u4XxIGKrzlBDOU/d7/4crsvqNVRxTPoEldhG35/EoA2fzqacvU3n8XZKDLzA6BDi4b3a8vi3PKjMAf+eveHYLJSoqVAlVs5wgFagCl3Xqim4jzr1eGZiiJjtp+jRZdgZTd6ve/lOQRK2Xaez7k14ThFUgK RTFqC9Nl 8tjnVvxTYFgb/F2wwCPpxlfnmuF6X4gfZLHMKXPg+wZj1xpOkItQSeAdsckm4BJjZFuGWJqes4ow8pzTUmLQP4sTOZLJvNM+Yj4wxM05v22JRf7jQzWku3y1nr/p2nwo1ut16RowERcsAIGK7i2VE4LqNOFTjUUw4jNlAkepOQtaoMkBCocJulrpnd1RJdO6Yu9MK98AmlsYI0oCmUi3sqpIjEeseM2zPLGsH X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Match how folio_unlock() works by combining the test for PG_waiters with the clearing of PG_writeback. This should have a small performance win, and removes the last user of folio_wake(). Signed-off-by: Matthew Wilcox (Oracle) --- mm/filemap.c | 15 +++------------ mm/internal.h | 2 +- mm/page-writeback.c | 9 ++++++--- 3 files changed, 10 insertions(+), 16 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index ddcced4638b5..56bfba546be7 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1177,13 +1177,6 @@ static void folio_wake_bit(struct folio *folio, int bit_nr) spin_unlock_irqrestore(&q->lock, flags); } -static void folio_wake(struct folio *folio, int bit) -{ - if (!folio_test_waiters(folio)) - return; - folio_wake_bit(folio, bit); -} - /* * A choice of three behaviors for folio_wait_bit_common(): */ @@ -1620,13 +1613,11 @@ void folio_end_writeback(struct folio *folio) * Writeback does not hold a folio reference of its own, relying * on truncation to wait for the clearing of PG_writeback. * But here we must make sure that the folio is not freed and - * reused before the folio_wake(). + * reused before the folio_wake_bit(). */ folio_get(folio); - __folio_end_writeback(folio); - - smp_mb__after_atomic(); - folio_wake(folio, PG_writeback); + if (__folio_end_writeback(folio)) + folio_wake_bit(folio, PG_writeback); acct_reclaim_writeback(folio); folio_put(folio); } diff --git a/mm/internal.h b/mm/internal.h index ccb08dd9b5ec..30cf724ddbce 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -105,7 +105,7 @@ static inline void wake_throttle_isolated(pg_data_t *pgdat) vm_fault_t do_swap_page(struct vm_fault *vmf); void folio_rotate_reclaimable(struct folio *folio); -void __folio_end_writeback(struct folio *folio); +bool __folio_end_writeback(struct folio *folio); void deactivate_file_folio(struct folio *folio); void folio_activate(struct folio *folio); diff --git a/mm/page-writeback.c b/mm/page-writeback.c index 410b53e888e3..c52514fe1d23 100644 --- a/mm/page-writeback.c +++ b/mm/page-writeback.c @@ -2940,10 +2940,11 @@ static void wb_inode_writeback_end(struct bdi_writeback *wb) spin_unlock_irqrestore(&wb->work_lock, flags); } -void __folio_end_writeback(struct folio *folio) +bool __folio_end_writeback(struct folio *folio) { long nr = folio_nr_pages(folio); struct address_space *mapping = folio_mapping(folio); + bool ret; folio_memcg_lock(folio); if (mapping && mapping_use_writeback_tags(mapping)) { @@ -2952,7 +2953,7 @@ void __folio_end_writeback(struct folio *folio) unsigned long flags; xa_lock_irqsave(&mapping->i_pages, flags); - folio_test_clear_writeback(folio); + ret = folio_xor_flags_has_waiters(folio, 1 << PG_writeback); __xa_clear_mark(&mapping->i_pages, folio_index(folio), PAGECACHE_TAG_WRITEBACK); if (bdi->capabilities & BDI_CAP_WRITEBACK_ACCT) { @@ -2970,13 +2971,15 @@ void __folio_end_writeback(struct folio *folio) xa_unlock_irqrestore(&mapping->i_pages, flags); } else { - folio_test_clear_writeback(folio); + ret = folio_xor_flags_has_waiters(folio, 1 << PG_writeback); } lruvec_stat_mod_folio(folio, NR_WRITEBACK, -nr); zone_stat_mod_folio(folio, NR_ZONE_WRITE_PENDING, -nr); node_stat_mod_folio(folio, NR_WRITTEN, nr); folio_memcg_unlock(folio); + + return ret; } bool __folio_start_writeback(struct folio *folio, bool keep_write)