From patchwork Mon Feb 13 07:53:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Zhao X-Patchwork-Id: 13137934 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5B8D9C636D4 for ; Mon, 13 Feb 2023 07:53:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 81BE56B0071; Mon, 13 Feb 2023 02:53:28 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7CCF36B0074; Mon, 13 Feb 2023 02:53:28 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 693836B0075; Mon, 13 Feb 2023 02:53:28 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 5A9306B0071 for ; Mon, 13 Feb 2023 02:53:28 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 2634B1A0155 for ; Mon, 13 Feb 2023 07:53:28 +0000 (UTC) X-FDA: 80461503696.19.151B424 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) by imf11.hostedemail.com (Postfix) with ESMTP id 77BCF40004 for ; Mon, 13 Feb 2023 07:53:26 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=optKyx+O; spf=pass (imf11.hostedemail.com: domain of 3dezpYwYKCBwQMR92G8GG8D6.4GEDAFMP-EECN24C.GJ8@flex--yuzhao.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=3dezpYwYKCBwQMR92G8GG8D6.4GEDAFMP-EECN24C.GJ8@flex--yuzhao.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676274806; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=NvhGAhuoLUXZRBDLvVFPku75YEeAYrautV9kwkbVm+U=; b=CEORzEv7RHKJpfl7ILOjfBae6uJxutFGtqrk4d7McTOU6vIS/ceMf6wNp61ZAElxcmz8u4 itjnXi7HHdgPx8qV3zoTQWmhutk8V3FKZEdRJKeQXmKZXpYNNOFqG+guWrdeC6J9rk4OrZ di5A4TMVCEgX8ZM9fgc28vqbJ8/h1ZA= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=optKyx+O; spf=pass (imf11.hostedemail.com: domain of 3dezpYwYKCBwQMR92G8GG8D6.4GEDAFMP-EECN24C.GJ8@flex--yuzhao.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=3dezpYwYKCBwQMR92G8GG8D6.4GEDAFMP-EECN24C.GJ8@flex--yuzhao.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676274806; a=rsa-sha256; cv=none; b=RSfkBdOM8SRhrannrL5tpF7vuletQ+N92kA5e3uF+Q0NVjAJFzaPAtJV9wQuAIgKpTbBmv jAJSYP/u9vMwK6l4ljnhae5U38j2PpXPc1heTlCXRSMu6HXZ+Kx77DUEysUlydw5N6lFK5 OpKlX8+3C4YEH0a2UOm0YvvCunD+Jcw= Received: by mail-yb1-f202.google.com with SMTP id a4-20020a5b0004000000b006fdc6aaec4fso11880299ybp.20 for ; Sun, 12 Feb 2023 23:53:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:mime-version:message-id:date:from:to:cc:subject :date:message-id:reply-to; bh=NvhGAhuoLUXZRBDLvVFPku75YEeAYrautV9kwkbVm+U=; b=optKyx+Ol4vgXo9n9HECwvIRntjr3OUydQJaoxkHSsFuotk0exbyE/mdWNa18ffL/h qQmFuqJAhnLXTtEyCMP8wNz4mAakzwVU4q94mV8Eb8IFVJ+0aBHIa2GtgstgoQ8rRmle ZwjA6uZ6kVYx40fca2ORN7hr4T6JZTiBfUmY6DSxVTlpg4FGiulHGtFN3JM6VHlNRDyI PFoiyLYbHEbYKeXi7+LuOUPsT0PxNlbMuZH8JZiwTJ/vQ5GyrmGG4E2PlXIKAbHUWLhh Zlxp5yjsxYzXDqxwf8kdiOk1LJk33G+qspbVqUpAPZGKN3VTQA32H1LgXhuK0fIJHmI0 5PrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:mime-version:message-id:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=NvhGAhuoLUXZRBDLvVFPku75YEeAYrautV9kwkbVm+U=; b=5O8vPXdlzO5x6XODrlbnLXNtSJOEq2wRl5s14AiDeVEf49Ia6vFumxa+PcPxXiFwo1 jcSBCiRusSlRHK+C7FRQnLN3/4YYCWIhPreU15krASwCEfTlkmJZXkpjmhiRWCNZZZyU zZWdMcbAtzUBHUdEFuUbGmNj40gkDhPqit3wgHVnDjzM+P6iO62pRleW1ibMDIMLg8Kr 1Tdge9ValL081Y+4rG0uLls9gOCLnXogmfj9uYdGzRJnADoXB7nGn5oXPuj8D3wIZXFD mG6ib4GlCjgq8MVGJxHUrJaQqu81KZKHV/uXKAICv2KK3tj8Bk4C6Y/TjDuTSp16d2++ th8A== X-Gm-Message-State: AO0yUKUkkCPwe0Pf/9JF/qjIoCUJ4wUWXXljXBntsrzvyL328j+z1voL KGUQNSUpkWD9RZNT9hNYrl0Vm+g/TRo= X-Google-Smtp-Source: AK7set/guU6artniwycMRZB/B3vkY/jJjJNmh0MMnseMPOIWtKnpO+3lWWTtlzgawUqAURmh5dclb/fIQ84= X-Received: from yuzhao.bld.corp.google.com ([2620:15c:183:200:7197:d392:1707:74d9]) (user=yuzhao job=sendgmr) by 2002:a81:a10:0:b0:521:daa4:d687 with SMTP id 16-20020a810a10000000b00521daa4d687mr30ywk.0.1676274805040; Sun, 12 Feb 2023 23:53:25 -0800 (PST) Date: Mon, 13 Feb 2023 00:53:22 -0700 Message-Id: <20230213075322.1416966-1-yuzhao@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.39.1.581.gbfd45094c4-goog Subject: [PATCH mm-unstable v1] mm: multi-gen LRU: avoid futile retries From: Yu Zhao To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-mm@google.com, Yu Zhao , "T . J . Mercier" X-Stat-Signature: 936tgp4c86opubbqjk9h8ianaahfpxzq X-Rspam-User: X-Rspamd-Queue-Id: 77BCF40004 X-Rspamd-Server: rspam06 X-HE-Tag: 1676274806-747135 X-HE-Meta: U2FsdGVkX1/z7Xk7utGCB6181HPLSLsAvxR6wIAcdPF3YGWFSMg9i/arVYUrOCdIj36Pj4l3XKvfHgfw2zmj4skX66xAKiyrG7w+bLCPnDmXlRYpmIL9fYVG4flhokDdXhxAc2uDsR/PsXjhtztSe1bQWHA50bkSJxrWAKHQF5vl6MWsQHyghWc50YfV0+xkAZzS7iSRHodRFtlcFlPbSd0qUgyvldNtC9lgIrSpNlWnlp8fj0FtjUXRl57Z7BSEYpwKMxEgEUDt8WWM2lm/N/UPXUjXo0kRtSdRnMtqu3ctw452erVUcrab+hC8rXWE4hHakTQu8sQNnuRYbz3rblt51Zw9+a25X0XYbqEefDXk8zWmEzQSp6+TUPlINhd7x3YVt2ZG2mDLg9VtCnV81VudguKEA6z+45K4uE6qXcG8m8pbs9sRbKEm2t19uJ0tTb7yOWNzFuf1AoBZwGFlxkZuPDCIiytLoAxuajqjBmLrJeGPOhZROjM0WNlNRCWkwDTgWGCyf3nzKdqwesKBgejjF2x341DTaCHCy81X2FxD5jfEMrCpwjxYFtQRyszBHs6Cb5NGpFBzXKtAxV/2pZcnEUCtpnWJHbOJjA4AtnhF1srFadJ3ZJw8jJZ9oXAlrKfM7nOsV7T57mztBPqfl2gTjXnmwYi7uf+E3DPJuKx9jXD2a1MpXn8w1jcLnv4mQflwQZ8Ds7VU9J9SMX49wGrQlzLzoIE72tH2IU/cx043WtdJ/b3RtdkI7jSQPaw72RUyz8B//O8QNrPC6v1s3oE/IWkTEuuUnYpgtFH3eaDocVyT8mGfRQGkljDYJW6DgfSM05E9zU3XGlSD/MY786MhMM8RjHIwwzZ+jCQKz08m456w8DaRSR/1vNvNMdocbjRCNjfgVdMwM2CGFkwiM/1AkLiG0gWZ6a3CbcK5zTYpGsFQkzy2l+t92CpPjofbpZ913SXKhmqr6YDqM/6 tsFp41yB ITFZqrmqsPvJ/+7V+/OjCkwhn7eT5vJJNWwL3qQnrOxHkU4HQ1eRfOYVN7NR5W+/pgPzTP5oZbhqHwa7kMKKVUG9c+Co3lQ9lg5VMcWGWbI25Y9ad/hHWJb/m1B6RuujvBJg9Z0bxCvqPNf6/vkhBGg8RVIvK/UaGAxjU+lJzTt+6PM66jsXwpV8jvAt53Ommkxavf3tFlVklmPhD/HPX6m5ZB+ShfNpI+4dLPSc3T6MQh485+avfQQ6o0H562o6GNP+0rKno/f4cL0QUwUpvxz2187S077XRW8xRLe7K7ww3gu7mTTH2YL5A2ZaWWwtJMUg/U1QZnIA6wrOlQ+Dn/wqH4mTH+09bZ61wPDNm7yMJ7i792o7qfvwKsrk5e1pw8wWr2kg5uZDfwLsz9wDlKdAoJ3bVUbq+UmPtO6YA7iTas+gAEtZyLl2brI0m924uQiAXlML0lBVv6w+cJ1q0ypT1bt9W02rON4ad2mxj0rzSQ2qLI/Lp/ofgx3Uf0/T+UVG3MeTikSzaAJgog4bGzFBA2w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Recall that the per-node memcg LRU has two generations and they alternate when the last memcg (of a given node) is moved from one to the other. Each generation is also sharded into multiple bins to improve scalability. A reclaimer starts with a random bin (in the old generation) and, if it fails, it will retry, i.e., to try the rest of the bins. If a reclaimer fails with the last memcg, it should move this memcg to the young generation first, which causes the generations to alternate, and then retry. Otherwise, the retries will be futile because all other bins are empty. Fixes: e4dde56cd208 ("mm: multi-gen LRU: per-node lru_gen_folio lists") Reported-by: T.J. Mercier Signed-off-by: Yu Zhao --- mm/vmscan.c | 25 +++++++++++++++---------- 1 file changed, 15 insertions(+), 10 deletions(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index d4b9fd1ae0ed..34535bbd4fe9 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -5356,18 +5356,20 @@ static int shrink_one(struct lruvec *lruvec, struct scan_control *sc) static void shrink_many(struct pglist_data *pgdat, struct scan_control *sc) { + int op; int gen; int bin; int first_bin; struct lruvec *lruvec; struct lru_gen_folio *lrugen; + struct mem_cgroup *memcg; const struct hlist_nulls_node *pos; - int op = 0; - struct mem_cgroup *memcg = NULL; unsigned long nr_to_reclaim = get_nr_to_reclaim(sc); bin = first_bin = get_random_u32_below(MEMCG_NR_BINS); restart: + op = 0; + memcg = NULL; gen = get_memcg_gen(READ_ONCE(pgdat->memcg_lru.seq)); rcu_read_lock(); @@ -5391,14 +5393,22 @@ static void shrink_many(struct pglist_data *pgdat, struct scan_control *sc) op = shrink_one(lruvec, sc); - if (sc->nr_reclaimed >= nr_to_reclaim) - goto success; - rcu_read_lock(); + + if (sc->nr_reclaimed >= nr_to_reclaim) + break; } rcu_read_unlock(); + if (op) + lru_gen_rotate_memcg(lruvec, op); + + mem_cgroup_put(memcg); + + if (sc->nr_reclaimed >= nr_to_reclaim) + return; + /* restart if raced with lru_gen_rotate_memcg() */ if (gen != get_nulls_value(pos)) goto restart; @@ -5407,11 +5417,6 @@ static void shrink_many(struct pglist_data *pgdat, struct scan_control *sc) bin = get_memcg_bin(bin + 1); if (bin != first_bin) goto restart; -success: - if (op) - lru_gen_rotate_memcg(lruvec, op); - - mem_cgroup_put(memcg); } static void lru_gen_shrink_lruvec(struct lruvec *lruvec, struct scan_control *sc)