From patchwork Tue Sep 12 18:45:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 13382048 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 86178EE3F0B for ; Tue, 12 Sep 2023 18:45:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1E3236B014C; Tue, 12 Sep 2023 14:45:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 190836B014D; Tue, 12 Sep 2023 14:45:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 00A9A6B014E; Tue, 12 Sep 2023 14:45:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id DEB426B014C for ; Tue, 12 Sep 2023 14:45:48 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id BA1951CA929 for ; Tue, 12 Sep 2023 18:45:48 +0000 (UTC) X-FDA: 81228824376.22.AABBF49 Received: from mail-pf1-f176.google.com (mail-pf1-f176.google.com [209.85.210.176]) by imf13.hostedemail.com (Postfix) with ESMTP id DC0102000C for ; Tue, 12 Sep 2023 18:45:46 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b=nFajmyWk; spf=pass (imf13.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.210.176 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1694544346; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=zmtAWomE9UHhxu4UCSwc3QxDlD+DtTBiD3PMp93+3PE=; b=g4USYwLhbS5GOIVbdlMy+L1JhZrupEoIoJKm+CJhQRCyeq9peuZq65Bk6uJRPfucP6klkP Xh3h6Z/FUxsdGJX0Gr+X8BpL+/R9WOnZJdp52i1RQ4JAlnVHmgNXHrmu3fkztuBoxglGZf XKK3y/g1hTvPOTzBwMzLN+J0P/80hio= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1694544346; a=rsa-sha256; cv=none; b=b/RqnNIkzp81w5lmTMyPSzxS8HBuwd/hcBzaWdnFyc7lMZ4rGc+2ExbZxlgzRmlMXYrp20 MFvtzDIe90D3n5Ks97mmFmhSmAA8uk+xljS4hIEVbPhls+eq0WurHNr6WFZwHgcJi+Dz3m GPTckKOrjekoTswVoEZgGJrLrOJdXtI= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b=nFajmyWk; spf=pass (imf13.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.210.176 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-pf1-f176.google.com with SMTP id d2e1a72fcca58-68fbd31d9ddso2050398b3a.0 for ; Tue, 12 Sep 2023 11:45:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1694544345; x=1695149145; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=zmtAWomE9UHhxu4UCSwc3QxDlD+DtTBiD3PMp93+3PE=; b=nFajmyWk9y9S9omjDygcSYVTffnCvTU93NzZQtx+bi6SrsO5TAO732/dXGpihsgiVU F2V5TBAN/2aGqf339i5Z0dbjRCYKZfaHyxYZyjU/g9Qdtgk1/TxEW8BsaXcCFJ1MJYWA wuMJ+rJ22DCRaKKPApnfVGiaZt3K5Uq02tiIvF57uPXeeu8cLURO81LYGo2Hrnr/Xuda V6CAQUR52RtfrgWRiYttdRashGaY1REe6s5u1dFO05rP/qG97iCItnYmrSyz3FyUJ6C1 4c5iIEqnLRlSMcArhZvaSAL3uJwuApXGk58lQ6COcO0ei2BYxkfnzKUg3RqdaaB47mPY 9YHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694544345; x=1695149145; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=zmtAWomE9UHhxu4UCSwc3QxDlD+DtTBiD3PMp93+3PE=; b=O9uyKe4qfetAnqA4H+DpVnRIJ2x5YEqwKgbP/p+RiAq8hyi3ab+yvsW9NoZ3Nu3r7r /iUgoTlvgesWZEaP44ubBoDtLCvpqZBWXHMVCjlo6c4nIktoJo4abMnoePy5DMHSRdee 1Fx3NyFChlctwSBzCxqziEbcxZ62UPzJKlmSexB2+KF3NidESDrr5jIYXZLERN9sUUoZ Dmgj70yxKcP4v8vLlyL0TXCeEwLm3RhgN+RpzqCScZYghIJhjTtRLCJMVv6O7mwP2VM8 IZc4KciGa/0FbjjmK0JppWiZjo4J/U/yFHsmGHw5l+1fgVnL9gGAqWT8wFNtX39WUEOs 7j+Q== X-Gm-Message-State: AOJu0Yx615I37FV0obpVW+92qPo9Ir95ViI8udcPaP6L66GRt+Z5xYtd ezLgj3cq5cT9sPbr0pi/5Q2+HKkFuHtURYD/Ej/Slw== X-Google-Smtp-Source: AGHT+IFTl+e/Goi7iXFwtLdVoeLHUo9ApJcrx6N3ePfxaq6hFANf6bjQXpHJPVMd0VLiRBWMDtZbew== X-Received: by 2002:a05:6a20:914a:b0:155:1221:a3d5 with SMTP id x10-20020a056a20914a00b001551221a3d5mr341766pzc.5.1694544345053; Tue, 12 Sep 2023 11:45:45 -0700 (PDT) Received: from KASONG-MB2.tencent.com ([124.127.145.18]) by smtp.gmail.com with ESMTPSA id q18-20020a63bc12000000b00553b9e0510esm7390605pge.60.2023.09.12.11.45.41 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 12 Sep 2023 11:45:44 -0700 (PDT) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , Yu Zhao , Roman Gushchin , Johannes Weiner , Michal Hocko , Hugh Dickins , Nhat Pham , Yuanchu Xie , Suren Baghdasaryan , "T . J . Mercier" , linux-kernel@vger.kernel.orng, Kairui Song Subject: [RFC PATCH v2 4/5] lru_gen: convert avg_total and avg_refaulted to atomic Date: Wed, 13 Sep 2023 02:45:10 +0800 Message-ID: <20230912184511.49333-5-ryncsn@gmail.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230912184511.49333-1-ryncsn@gmail.com> References: <20230912184511.49333-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Rspamd-Queue-Id: DC0102000C X-Rspam-User: X-Stat-Signature: 5dp379bwjgx6xd3smcwywcz55jed41wu X-Rspamd-Server: rspam03 X-HE-Tag: 1694544346-87181 X-HE-Meta: U2FsdGVkX1+qytR9TjkAoTzOl0wvBkyUtZctCZgWZ2PfV+fo2vaEdMTr3hrVUBQH/W53TkcH0K+C1Kb3MFbIi9WVTZNBsP+5MNcO/OfGJDpfxmYVANf9kLp4dx7iyrsI/Pp4kDlL+uMV4XTr9xxjOMbWM3m2h0QG3cqN0znB/dKZK1VVzqJ+QKwGy9QgPTkb/zz9TSyVsPUL4B1NAZNdcSQFn1gC0V5c3Sok2pqv2NTSBZd/vr/GnHLhbWLjGAbNLJye3VWVf6VZXtGk/mqKC3kdBZ91y4i5quXNybyTAlGhLYdGjDZ0aBrfo2i3/45g4Dy+AY8447O79U5OWO8dnGbt/C6jj0isr+rvETRLORhj0y0+454xSYaZCELlOwnpOLpYhUKJ8Fd/rYRhmqSlY5w5QJSNEnLiqputlC6NwQRsi0Xswm3cutvLMP4NrsC6WNClVJ97ywz5j1mkQJ7ZnqnrUfPyvp5tRfz5++Tt/33BHloz5PvRy4L5pKRNnFMzLi23JpahxKUBP5yr4/cVK1Ffd9MZsH/jWPx8ts89ChpW/ti3L8a3gy169VDeHYWLQIenDW0tjzG4++FczGFWdGXIj20/LKovKThR5rEd/i0qmAVmFu769IsRFOpnP6yJNzcBfJ+iIboDs2c27xsr4Yrdd22d0hGoNqrWvJZw+poFK6PlV6MWAk6D/ZTozawrD3v3T1T/uqeBIGN8SC7nZ1p5o10UzbBy794hz0Oy4e9y06mvfX3t4mAA5lGf9nSl3Xwlwh1tLuBPBxAAHf8wODNlHXMZsF0gHND72QTsY1DIjRUwboFYOm4W1SjMeSq7EsZOLrxbQdvC+yg+Vie8Ox/odd9igsF5/JAy7evLOgk4Ibq30kyOxZ6zrlZRQ8d+VzbJc3DvlVnRAnCZAfp2qA46IBXfpBxTB9l51GMNbpkAEPZLpzfjy+Ko6S6slFWwt68dQWLZz45XHlO5OJ7 qI9uYgu2 vEcKeBRxrcnVVwLexNCAtYbNQV1eX0sscnorYc79LpSwSSKMkPSPQJqlDX4RJb29kdAXmrro4go5FcnFOEYkV69Q2YRem66iK+aG+wL2E/N/G2ZukLS+5VH3Db1N9Q58yqA6ypBopVWD+fd8qIwG+9ZUaQ6p13ON3iQ4nbU6AF/ocAb8EPsR2+q8wXst/dUlDgdAUK/ElDpQdTpArhLKk4s9qxFaa76IMdZbsrRyM0TAx88hnLZgkg8bDeYRzXT2Gk5stxm/j0yOLUP8GuYIpp3psQX7R8JoGDawNpEFuzwy+hLiLWrUgEeactvEML4CyRgvjhYiopx6ICcy6XWq/Ah7Oh83N2LeIa4AQ6f9cOlCVpzlN9FIQXA3alxRB8SxadSfI8/BNj89ZLV6Hzx68042eiFNiCeR/cum9Q4nItiojjMJgfF5I04hLxOp0iIVh2TIkjgn0Va9HVYY4xhz4OO3BU3NsvXVf/LsQUqufiaImze4/7fi95MFw2IQf85m7bVYFPi9iiLCIpNqAaoPt+GcHC1vPUBPAg6DR/YZxr7I09h/0DNwfLGcsC1iOqzzGodtWMEriS4ze5YhKcldSn0cKHu9KRhTUeoHvWKZzE0vehkoyoUz4KhZtcDZ6K/QIg4zq+B+SHqZ+I6xkTNQQKoMrx964LsAAhC5a9Ozadja7L4o= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Kairui Song No feature change, prepare for later patch. Signed-off-by: Kairui Song --- include/linux/mmzone.h | 4 ++-- mm/vmscan.c | 16 ++++++++-------- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 4106fbc5b4b3..d944987b67d3 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -425,9 +425,9 @@ struct lru_gen_folio { /* the multi-gen LRU sizes, eventually consistent */ long nr_pages[MAX_NR_GENS][ANON_AND_FILE][MAX_NR_ZONES]; /* the exponential moving average of refaulted */ - unsigned long avg_refaulted[ANON_AND_FILE][MAX_NR_TIERS]; + atomic_long_t avg_refaulted[ANON_AND_FILE][MAX_NR_TIERS]; /* the exponential moving average of evicted+protected */ - unsigned long avg_total[ANON_AND_FILE][MAX_NR_TIERS]; + atomic_long_t avg_total[ANON_AND_FILE][MAX_NR_TIERS]; /* the first tier doesn't need protection, hence the minus one */ unsigned long protected[NR_HIST_GENS][ANON_AND_FILE][MAX_NR_TIERS - 1]; /* can be modified without holding the LRU lock */ diff --git a/mm/vmscan.c b/mm/vmscan.c index 3f4de75e5186..82acc1934c86 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -3705,9 +3705,9 @@ static void read_ctrl_pos(struct lruvec *lruvec, int type, int tier, int gain, struct lru_gen_folio *lrugen = &lruvec->lrugen; int hist = lru_hist_from_seq(lrugen->min_seq[type]); - pos->refaulted = lrugen->avg_refaulted[type][tier] + + pos->refaulted = atomic_long_read(&lrugen->avg_refaulted[type][tier]) + atomic_long_read(&lrugen->refaulted[hist][type][tier]); - pos->total = lrugen->avg_total[type][tier] + + pos->total = atomic_long_read(&lrugen->avg_total[type][tier]) + atomic_long_read(&lrugen->evicted[hist][type][tier]); if (tier) pos->total += lrugen->protected[hist][type][tier - 1]; @@ -3732,15 +3732,15 @@ static void reset_ctrl_pos(struct lruvec *lruvec, int type, bool carryover) if (carryover) { unsigned long sum; - sum = lrugen->avg_refaulted[type][tier] + + sum = atomic_long_read(&lrugen->avg_refaulted[type][tier]) + atomic_long_read(&lrugen->refaulted[hist][type][tier]); - WRITE_ONCE(lrugen->avg_refaulted[type][tier], sum / 2); + atomic_long_set(&lrugen->avg_refaulted[type][tier], sum / 2); - sum = lrugen->avg_total[type][tier] + + sum = atomic_long_read(&lrugen->avg_total[type][tier]) + atomic_long_read(&lrugen->evicted[hist][type][tier]); if (tier) sum += lrugen->protected[hist][type][tier - 1]; - WRITE_ONCE(lrugen->avg_total[type][tier], sum / 2); + atomic_long_set(&lrugen->avg_total[type][tier], sum / 2); } if (clear) { @@ -5885,8 +5885,8 @@ static void lru_gen_seq_show_full(struct seq_file *m, struct lruvec *lruvec, if (seq == max_seq) { s = "RT "; - n[0] = READ_ONCE(lrugen->avg_refaulted[type][tier]); - n[1] = READ_ONCE(lrugen->avg_total[type][tier]); + n[0] = atomic_long_read(&lrugen->avg_refaulted[type][tier]); + n[1] = atomic_long_read(&lrugen->avg_total[type][tier]); } else if (seq == min_seq[type] || NR_HIST_GENS > 1) { s = "rep"; n[0] = atomic_long_read(&lrugen->refaulted[hist][type][tier]);