From patchwork Mon Aug 21 20:44:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13359801 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DADACEE49AA for ; Mon, 21 Aug 2023 20:44:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BA90E940014; Mon, 21 Aug 2023 16:44:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B0BDD940010; Mon, 21 Aug 2023 16:44:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9353E940014; Mon, 21 Aug 2023 16:44:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 82607940013 for ; Mon, 21 Aug 2023 16:44:41 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 5852D1C9C33 for ; Mon, 21 Aug 2023 20:44:41 +0000 (UTC) X-FDA: 81149290362.26.A80ACC0 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf23.hostedemail.com (Postfix) with ESMTP id 949C214000C for ; Mon, 21 Aug 2023 20:44:39 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=Ceq6MGoy; dmarc=none; spf=none (imf23.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1692650679; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=pSrtzJTe3IU8p+1cAoPm0gIDtB8SqdaEhhREkvHFPcI=; b=xkgmLN+Lb4lM8WJgFVhUqVv4wCOJHJJDz9TvpMyQFrSsw5vlrVf1IsfylzFVTotjVCsEF0 v8F9AEbyH4AUtron5+jcPvH27vGv5KX0L8A+xLEEgfQwc0sfKDqpo4Ngr25+H8R4bugd0K UyPvfZ1ncctruou0Fek1ob2COASvyUk= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=Ceq6MGoy; dmarc=none; spf=none (imf23.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1692650679; a=rsa-sha256; cv=none; b=PsSb2np162xJ5bWfASvgWOF1cfKGJLU5uVvcLSYpI9J3OIEnrT7/ePS4KFC0QWz223EJ8M sZA9F4XhOyONP6zfCX6LPatPXxC7XHhCvOFUkpGYi6Zl8sbNayINDg1tfvQK4QWxTtwvZS F/4JNpXCsJxYV7hZ0r+MspI6RR1GhVo= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=pSrtzJTe3IU8p+1cAoPm0gIDtB8SqdaEhhREkvHFPcI=; b=Ceq6MGoywPDgt/xWZ3yRxd/WSL RkzO1yVGelML9XeqBKiR4+8SdApxY7F+p7ljHaoG0D6HLhXJ/sG+nrVXduVmXoAnrfZEpqL80lGco PoD2MLxNIaGzT2hN7NjKhPe5egBOm4TJfy5VI1r7rVR9CmQKcSySZHP3AUk2xjdkJ77tCaAyHS0vx ib8C07J9p0tmPvrdpd8PxYCPF2p/XZzUiI4BGZZXoZlZbjFfdiSg/7BiSvdoFBaKTt3KLueh+mhML 4uATzCFca0yv3rmigKvdTN8NCsXT3KZJSUPYy4NcdFmF/mR/U4BIKVyhYr4O78PyWou7n71PDyXbO kuyzOn5g==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1qYBlF-00CL0b-Tr; Mon, 21 Aug 2023 20:44:30 +0000 From: "Matthew Wilcox (Oracle)" To: Mateusz Guzik , linux-kernel@vger.kernel.org, dennis@kernel.org, tj@kernel.org, cl@linux.com, akpm@linux-foundation.org, shakeelb@google.com, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" Subject: [PATCH 7/7] mm: Use free_unref_folios() in put_pages_list() Date: Mon, 21 Aug 2023 21:44:25 +0100 Message-Id: <20230821204425.2940496-7-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20230821204425.2940496-1-willy@infradead.org> References: <20230821204425.2940496-1-willy@infradead.org> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: 3yo4ofz5y3uoqya78s4b5nziprwp1xmm X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 949C214000C X-HE-Tag: 1692650679-145649 X-HE-Meta: U2FsdGVkX18C3/aOLfeG8zaDUamFwoXov/qdvyGsbDAw1K9Y6pzure8hv/tLLjXu862NwNEorwX8sn6rUCTM+qNyaOFQp8gIlIOxxSYZDNBvFml1wK0nViyvKfHNo9DcRZtwEWO/Arg2PKzlDXX2rSwmIfUtR97XCYHiv1p1udM3qkjF1DSvt7OKbr2vakACi9IopOBKE69G26v3+7G/FevqBDhDy3O0+xMbtYi1q1VZop8+/W7ch1JoNvDqiqQ0QE67lyJUZhP3cCBbOqjo+2jo4LtKk0RxQApqS164LLVGAn7CilrlzLkYOmkUCzX3vzJcjKCzWvNzZ218wTmp9URN+Y8rRtpmx/vbnscgBGL+1WdcrgthA70DpT2A6DPyPOst2rwat8C9T4oxEFisSvu37d7vus8GrGZHOG2nRfNbYmREU2ZEunbUQue0ehB6p9rNYEn97BnBiUxIwUAUB5DOzTEDVVJ4KWO1cJSgSHo7wvA1HQnqp8D/MKgCxMH1uHodam5PUh3xqzrbL4tCgJY43Zg0hjajErtEFJ9Jq2qXP7Ph4R/f0TV1KtOljFVbQJebooxeyeZtVKpKAgB6BzI99KgO2pp+X7eaNclylEJMznhmKX79zISLzHjqPyyJLbj5pNRWHvJuIaQpcgPi/VvYKZERTrqojb4e1CA1L1wuL6iPHpQVO1JwzfZfyfXz7bkvXzthnxWqnXKks15bcJ7s+MdHsrSBjDiWuZpUZ29oalsI2DFcSLMgUIQ3dXmIaz3rPlbb0GD8MOwrGvqpTZ4MstDAas+8O4D9aST0PPsdyJxI8kNSjrV5pwPJIiFC4V90C2dpK8tcYAfTpP+7egxiExvsfpGPDGkr67vonQOvHVFk3SSfFgEeNg34VY5QjzZdAJOoK/de+ExPwqVpChCC56dPWBp1XN2ny/nDkZ96flC5mGViJq7pyBB1n8Qmvuu2gPCN0Kx9DCmQBNb V8/aMcWJ Mft2p7Sl3rEvqBlpGWyWmYrrtBiDASgjIgu+hMUH1BYOx590XJsuiiicW0GuUEX8aEoacTT3q51/y8nqk+DisNw5SDD1GlTOdzNLJnSEzpbzph9qZ4ATEFFErCv3vHh+pca0xD1vD2DVdThxUFwWzLr1SbWNNiPR63r+4NZ8RgG8kMeQWbA9zsAnYetdegcsuZ4mdegjFEkxZJPRy3O2iZDJltX8Ohzlk6ny6 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Break up the list of folios into batches here so that the folios are more likely to be cache hot when doing the rest of the processing. Signed-off-by: Matthew Wilcox (Oracle) --- mm/swap.c | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/mm/swap.c b/mm/swap.c index 8ebb10d44de7..51a613f477a9 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -138,22 +138,26 @@ EXPORT_SYMBOL(__folio_put); */ void put_pages_list(struct list_head *pages) { - struct folio *folio, *next; + struct folio_batch fbatch; + struct folio *folio; - list_for_each_entry_safe(folio, next, pages, lru) { - if (!folio_put_testzero(folio)) { - list_del(&folio->lru); + folio_batch_init(&fbatch); + list_for_each_entry(folio, pages, lru) { + if (!folio_put_testzero(folio)) continue; - } if (folio_test_large(folio)) { - list_del(&folio->lru); __folio_put_large(folio); continue; } /* LRU flag must be clear because it's passed using the lru */ + if (folio_batch_add(&fbatch, folio) > 0) + continue; + free_unref_folios(&fbatch); + fbatch.nr = 0; } - free_unref_page_list(pages); + if (fbatch.nr) + free_unref_folios(&fbatch); INIT_LIST_HEAD(pages); } EXPORT_SYMBOL(put_pages_list);