From patchwork Tue Feb 27 17:42:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Matthew Wilcox (Oracle)" X-Patchwork-Id: 13574203 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D6146C54798 for ; Tue, 27 Feb 2024 17:43:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4D3B66B00EB; Tue, 27 Feb 2024 12:43:05 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 11B316B00EC; Tue, 27 Feb 2024 12:43:04 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A3FE96B00F0; Tue, 27 Feb 2024 12:43:04 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 4BFA06B00E7 for ; Tue, 27 Feb 2024 12:43:04 -0500 (EST) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id E4A731A0C97 for ; Tue, 27 Feb 2024 17:43:03 +0000 (UTC) X-FDA: 81838304646.04.28A133B Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf24.hostedemail.com (Postfix) with ESMTP id 65A00180028 for ; Tue, 27 Feb 2024 17:43:02 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=ggDylXov; dmarc=none; spf=none (imf24.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709055782; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ga6KYsObfe/VGxSB5IYxXXQ8b3liJ4Nmvd2PEithDXA=; b=3nmFek8zT6QcvYNHTTp4fKHIkDNqCvd077dRYOpMAHje+P500U2aD7g65DUSaCwSstHHr3 NhyOzufomPJnEnrD5fmtlg0yzN7UE2Yz481KWh+fCJKxJKqpz3jDWxeNELEFyEfL7tM+wj cBlzj/WOPjydBb+szsVHFoxsC1aEDqM= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=ggDylXov; dmarc=none; spf=none (imf24.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709055782; a=rsa-sha256; cv=none; b=bSnz4tpnI852FDYNVDPxAN352EkQoUpcB/+NUWFiXqQJVaQR6kpglphdOwNoUBpkFchzvY I0NG7nkh0MCYdtoWP0TgxrYqgQaDLp8dJEqWbVh6JcAOrWJtLdQGy6MVLuGPhXGWMmM7Es z7CyD90GhExYlkBY2QcaVXGjc7xpWNo= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=ga6KYsObfe/VGxSB5IYxXXQ8b3liJ4Nmvd2PEithDXA=; b=ggDylXovC1rEQMSlqMxuv1t8SK LHOHViTEyr3CFrNJ5AWSkIQ8/ZXkZ8FlquuktIsxdImpEgPj9q3OMfwVamYOGLuTLvXDiosDXB/xr 55gh5TmU3USZAb/3Wi26clzbmOFKR/G3bzNtKwPkKtiAB35gFDYidzqBVEEFcr/dgbhCq8LrZLwM2 4EcjwSg+9WyINRUg9CakMWUJv8Z0b8ZsYl+y5UqaIAijItBe28YK4H6tEX9urt3OBcpuRbjHuUoJA JKD/87LjxaFxcTx8izgUznW6iAk6quS9N54kYgKWo2K7/JMH8/T+b11Gejo9/f33BFfCyXzpyKbyw QBEBruqg==; Received: from willy by casper.infradead.org with local (Exim 4.97.1 #2 (Red Hat Linux)) id 1rf1Tn-00000002yre-0A0L; Tue, 27 Feb 2024 17:42:59 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, Mel Gorman Subject: [PATCH v3 11/18] mm: Free folios in a batch in shrink_folio_list() Date: Tue, 27 Feb 2024 17:42:45 +0000 Message-ID: <20240227174254.710559-12-willy@infradead.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240227174254.710559-1-willy@infradead.org> References: <20240227174254.710559-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 65A00180028 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: 5fbf3s7q418qmhudztx8x3t91thy4ue1 X-HE-Tag: 1709055782-592276 X-HE-Meta: U2FsdGVkX18Ka2hTIGO7z+9Dr0GfdDajJM+k12f5Hl0doXQVV7r3AvzSTdeVcSRSw20MVwyIPmbTzWMwFcG8abivLupjxpOqvXoZ8gIgiK7Iu6f1VLkN0c3x8/RyJME5S1sPRjR0BzzknS+zVhjCYEBg29BwJ5jP9OVz0MasY0l5/0TU9mJzrVZUqmNrYi/zD5ZVwbiLtL53Wyptxsm13upA3wLn9VOn9ANzh9xbBk3NqxBJG1yZr299Zw8AmrfAa2GbriWm8vkCFXmuoI6fa5FZnHHcfg7rvl9aDPfwsrFqdipjAreyVApFQECSFs4gRA1c1zkPlGlr7l04IoMqOyZ4AASE75X58xTey4ZYR3kY//vH0CmEpTw8H3ftvWMtxhpv4qUjfovvMXVPjhR8EVXTk+uC6RjOglVDtOi9mqHctbOOu8j1pp2gXDuw0P4zdn59G8h0jXgGohHHymKFQzeKOXbKHiteY81Ub5WIx/aSd2YV/6iSSkFlODFh2iA2Z/o4OOc6cUsIBEH0XJUb7svZ8Kvpf/BDqnCy2eNeUDuZfkaIV3F0jSoFZRRYLuArEasYU450NtfE8j+UlRQQCEIQ8BjaKQonCkm9SsEWArakLQM7BQK7aC2IKU9Ophv1dqoQtKJxd8jj65no78qzt9ABusdU5vEccIFB8ecF0PEMxIVNnR5FWM1D0Rcwyc/yf9xclA9dz3tbU+GdOvg3ud9REGQ5Hh8ZQ20dhaQobwS6/HiaJfgxKEPC3OhP/e27WJ/pEIyr8jPO8qu1JpSAIKEtMyQOzufhiOlRVAQ9LOGOWPLaEiq1w9+QU0Xk3x3KHegtEDsMTgXHXyqp07g4xcntn3hmMduUvcBIXqY9wqA2agN7OUBOzhbL0/LxUGa1/Jj56NcKYoQl/rP87XYrYnWlXYEdCLxG X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Use free_unref_page_batch() to free the folios. This may increase the number of IPIs from calling try_to_unmap_flush() more often, but that's going to be very workload-dependent. It may even reduce the number of IPIs as we now batch-free large folios instead of freeing them one at a time. Signed-off-by: Matthew Wilcox (Oracle) Cc: Mel Gorman --- mm/vmscan.c | 20 +++++++++----------- 1 file changed, 9 insertions(+), 11 deletions(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index d3c6e84475b9..0c88cb23cc40 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -1026,14 +1026,15 @@ static unsigned int shrink_folio_list(struct list_head *folio_list, struct pglist_data *pgdat, struct scan_control *sc, struct reclaim_stat *stat, bool ignore_references) { + struct folio_batch free_folios; LIST_HEAD(ret_folios); - LIST_HEAD(free_folios); LIST_HEAD(demote_folios); unsigned int nr_reclaimed = 0; unsigned int pgactivate = 0; bool do_demote_pass; struct swap_iocb *plug = NULL; + folio_batch_init(&free_folios); memset(stat, 0, sizeof(*stat)); cond_resched(); do_demote_pass = can_demote(pgdat->node_id, sc); @@ -1432,14 +1433,11 @@ static unsigned int shrink_folio_list(struct list_head *folio_list, */ nr_reclaimed += nr_pages; - /* - * Is there need to periodically free_folio_list? It would - * appear not as the counts should be low - */ - if (unlikely(folio_test_large(folio))) - destroy_large_folio(folio); - else - list_add(&folio->lru, &free_folios); + if (folio_batch_add(&free_folios, folio) == 0) { + mem_cgroup_uncharge_folios(&free_folios); + try_to_unmap_flush(); + free_unref_folios(&free_folios); + } continue; activate_locked_split: @@ -1503,9 +1501,9 @@ static unsigned int shrink_folio_list(struct list_head *folio_list, pgactivate = stat->nr_activate[0] + stat->nr_activate[1]; - mem_cgroup_uncharge_list(&free_folios); + mem_cgroup_uncharge_folios(&free_folios); try_to_unmap_flush(); - free_unref_page_list(&free_folios); + free_unref_folios(&free_folios); list_splice(&ret_folios, folio_list); count_vm_events(PGACTIVATE, pgactivate);