From patchwork Wed Aug 2 02:56:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kalesh Singh X-Patchwork-Id: 13337549 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9C2A2EB64DD for ; Wed, 2 Aug 2023 02:56:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: Mime-Version:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To: References:List-Owner; bh=HaQUp1PfG/3BNIiXDD4+sa+t53rahLsHIkqIUpmqQjo=; b=Tor 5XvCNcRJ3G+stRamajZNpSfGEORDdsPEmthxrapwjjJi7qKTLi2d5su4MdfgDxydHiNUR5E2AwfC6 w/2UVA8IBy9T82gcihNoC0j7tMykHj8zeDwES6duz796aiXyO8AZeLr4vftIjx7qhRGUNOqShCYt6 w/fv81TK7FDMCZmmnBC8ZLXR/4qGNRgIqeVIpnlICoIx/S/wUWvpBjFxXYOtokunPYcosWwYayIy5 NitskrEfaLfzlzkQ1siu775tn6koErUIjwShhZor5Mkl+b06cLngGfEa4VpPqWo2/UShf8qPzWgHv 1NW6rgBjnCTh5yOqjj7hqgGIPdpNmYw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qR228-003p7i-2L; Wed, 02 Aug 2023 02:56:20 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qR225-003p6O-1o for linux-arm-kernel@lists.infradead.org; Wed, 02 Aug 2023 02:56:19 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-d3ae4f43dceso529083276.0 for ; Tue, 01 Aug 2023 19:56:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1690944975; x=1691549775; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=aqehfJB+hF+Fne+AdGJurbsZIYsQCeOte57mRi2GAJw=; b=33TVVDzXg6eBl7/Vpt8F6zz+Hj/NOmdXdllc0HT3+fXbEBhkPYtEkaq37eNBBfQ//f huzyAhMvUorpNH56dDYRwA06JZVdYTEAbzfGjbqw3gBWiic0SbSYYIRLI7+dUO6KJeP/ QbLSh8xuVcrhWjh4oWX9BfGiIQ2WhcFjEVln6zdtkNbTrb8Vr/FaB0bbjGe719xWxqpC hL3EQTp/eSWIIOfcyzhoc5/BO66sBZZeI+o1Tma4ICZmtHB5eUiGscJSklwG5yeK8aOP pOTWKEoNcoXV0uOzF2WA7ODrKPaZuVgMaXvYUoulWRPVnXT17L+Vmq1QR0HZFgWxuJek tQWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690944975; x=1691549775; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=aqehfJB+hF+Fne+AdGJurbsZIYsQCeOte57mRi2GAJw=; b=RA8TM1zIU/l0oX3zFjI+sNx4vJzj9M2DJRY3aUnW7Xyt9xWKcVwI9+1pc4tssm6zPj CAVFaKEwWuXEWQ3u1H/qGTBbKyUU7VJK9TskPYbzOTNgOPvPSCWW6hXOpFxuL65NQS+U TQAXwUpNhfdeG1FVPyTBMVn8RETD/+XlsJhyVfCrZzlWCwSPLr/zKrr93MbWkmjlWVbE aocRr2eJMNUQBYQ8h9/+Q7YYDKcb2GHcFOaBuI4E6zWvd+X0RLqnyASJkQ5/s73u+a6O n3QZVHjrEmyQLh5vyAIuxJ0EQTrFbKUBLS8Fy2ZPRZCLRE4wRFT5Egu1xURWhD7yd9RT Dp+g== X-Gm-Message-State: ABy/qLb/EyfEv7wriQBUMFAbHxRlvns5ZIs3rPuRuY8Pi6jR+e/77KKs YG828XM5guHJjETUELCDBOw/Psv/gObtKC6upw== X-Google-Smtp-Source: APBJJlGu4CGYprEhx9EAJQV2bkMTULOViWk9fNZltzmcAuDxS7zR2BS8SgTh3oU+2KBCXMRrlqL63XtCKu/qeEPcYg== X-Received: from kalesh.mtv.corp.google.com ([2620:15c:211:201:aa5f:ff75:318:255]) (user=kaleshsingh job=sendgmr) by 2002:a05:6902:100f:b0:cf9:3564:33cc with SMTP id w15-20020a056902100f00b00cf9356433ccmr116986ybt.13.1690944975112; Tue, 01 Aug 2023 19:56:15 -0700 (PDT) Date: Tue, 1 Aug 2023 19:56:02 -0700 Mime-Version: 1.0 X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog Message-ID: <20230802025606.346758-1-kaleshsingh@google.com> Subject: [PATCH v2 1/3] mm-unstable: Multi-gen LRU: Fix per-zone reclaim From: Kalesh Singh To: yuzhao@google.com, akpm@linux-foundation.org Cc: surenb@google.com, android-mm@google.com, kernel-team@android.com, Kalesh Singh , stable@vger.kernel.org, Charan Teja Kalla , Lecopzer Chen , Matthias Brugger , AngeloGioacchino Del Regno , Suleiman Souhlal , Oleksandr Natalenko , "Jan Alexander Steffens (heftig)" , Qi Zheng , Steven Barrett , Brian Geffon , Barry Song , linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mediatek@lists.infradead.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230801_195617_615604_EB0E75AD X-CRM114-Status: GOOD ( 22.58 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org MGLRU has a LRU list for each zone for each type (anon/file) in each generation: long nr_pages[MAX_NR_GENS][ANON_AND_FILE][MAX_NR_ZONES]; The min_seq (oldest generation) can progress independently for each type but the max_seq (youngest generation) is shared for both anon and file. This is to maintain a common frame of reference. In order for eviction to advance the min_seq of a type, all the per-zone lists in the oldest generation of that type must be empty. The eviction logic only considers pages from eligible zones for eviction or promotion. scan_folios() { ... for (zone = sc->reclaim_idx; zone >= 0; zone--) { ... sort_folio(); // Promote ... isolate_folio(); // Evict } ... } Consider the system has the movable zone configured and default 4 generations. The current state of the system is as shown below (only illustrating one type for simplicity): Type: ANON Zone DMA32 Normal Movable Device Gen 0 0 0 4GB 0 Gen 1 0 1GB 1MB 0 Gen 2 1MB 4GB 1MB 0 Gen 3 1MB 1MB 1MB 0 Now consider there is a GFP_KERNEL allocation request (eligible zone index <= Normal), evict_folios() will return without doing any work since there are no pages to scan in the eligible zones of the oldest generation. Reclaim won't make progress until triggered from a ZONE_MOVABLE allocation request; which may not happen soon if there is a lot of free memory in the movable zone. This can lead to OOM kills, although there is 1GB pages in the Normal zone of Gen 1 that we have not yet tried to reclaim. This issue is not seen in the conventional active/inactive LRU since there are no per-zone lists. If there are no (not enough) folios to scan in the eligible zones, move folios from ineligible zone (zone_index > reclaim_index) to the next generation. This allows for the progression of min_seq and reclaiming from the next generation (Gen 1). Qualcomm, Mediatek and raspberrypi [1] discovered this issue independently. [1] https://github.com/raspberrypi/linux/issues/5395 Fixes: ac35a4902374 ("mm: multi-gen LRU: minimal implementation") Cc: stable@vger.kernel.org Cc: Yu Zhao Cc: Andrew Morton Reported-by: Charan Teja Kalla Reported-by: Lecopzer Chen Signed-off-by: Kalesh Singh Tested-by: AngeloGioacchino Del Regno Tested-by: Charan Teja Kalla --- Changes in v2: - Add Fixes tag and cc stable mm/vmscan.c | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index 4039620d30fe..489a4fc7d9b1 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -4889,7 +4889,8 @@ static int lru_gen_memcg_seg(struct lruvec *lruvec) * the eviction ******************************************************************************/ -static bool sort_folio(struct lruvec *lruvec, struct folio *folio, int tier_idx) +static bool sort_folio(struct lruvec *lruvec, struct folio *folio, struct scan_control *sc, + int tier_idx) { bool success; int gen = folio_lru_gen(folio); @@ -4939,6 +4940,13 @@ static bool sort_folio(struct lruvec *lruvec, struct folio *folio, int tier_idx) return true; } + /* ineligible */ + if (zone > sc->reclaim_idx) { + gen = folio_inc_gen(lruvec, folio, false); + list_move_tail(&folio->lru, &lrugen->folios[gen][type][zone]); + return true; + } + /* waiting for writeback */ if (folio_test_locked(folio) || folio_test_writeback(folio) || (type == LRU_GEN_FILE && folio_test_dirty(folio))) { @@ -4987,7 +4995,8 @@ static bool isolate_folio(struct lruvec *lruvec, struct folio *folio, struct sca static int scan_folios(struct lruvec *lruvec, struct scan_control *sc, int type, int tier, struct list_head *list) { - int gen, zone; + int i; + int gen; enum vm_event_item item; int sorted = 0; int scanned = 0; @@ -5003,9 +5012,10 @@ static int scan_folios(struct lruvec *lruvec, struct scan_control *sc, gen = lru_gen_from_seq(lrugen->min_seq[type]); - for (zone = sc->reclaim_idx; zone >= 0; zone--) { + for (i = MAX_NR_ZONES; i > 0; i--) { LIST_HEAD(moved); int skipped = 0; + int zone = (sc->reclaim_idx + i) % MAX_NR_ZONES; struct list_head *head = &lrugen->folios[gen][type][zone]; while (!list_empty(head)) { @@ -5019,7 +5029,7 @@ static int scan_folios(struct lruvec *lruvec, struct scan_control *sc, scanned += delta; - if (sort_folio(lruvec, folio, tier)) + if (sort_folio(lruvec, folio, sc, tier)) sorted += delta; else if (isolate_folio(lruvec, folio, sc)) { list_add(&folio->lru, list); From patchwork Wed Aug 2 02:56:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kalesh Singh X-Patchwork-Id: 13337550 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A35C3C001DF for ; Wed, 2 Aug 2023 02:56:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=RIF2tvI/CPT31zEmgf41pKsMdN4HPUP3T4dXr3RuWQU=; b=Ge6DrSCkqA/x5fx89zf4Dj/34A b6n6R/nAhDR/Y9msWRX0mimVotZue9YNitZzWIUroLpB4RdOk6DriYqxnIrJ8mNK2WPuWB4oTTHdY KRJMO26MqUIpcMUYXzoWwtDwADfHWToS7wVJlygryWfT0hNKPFwRbj1b0yjqvtwwsCgTiyJuIFfdW fazC9oHZjI7jkg8K5H0vJeI5Q2TtxoChMYL4RXQIzE0VM/d64vQ15QDQYct1VauEQf72CKDK31dM+ LwwgO9c8atM5lHM5zb6aYf/1GDBLoq/Lrb/2s+fbzDlgLY06ic9aeHoUQngxYJIplI2gA5iH0n/Dm 6L/hQwlA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qR22D-003pA0-08; Wed, 02 Aug 2023 02:56:25 +0000 Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qR22A-003p7h-0d for linux-arm-kernel@lists.infradead.org; Wed, 02 Aug 2023 02:56:23 +0000 Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-d0d27cd9db9so875802276.0 for ; Tue, 01 Aug 2023 19:56:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1690944980; x=1691549780; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=J6cxPzKelBRogs1RcqlDIpKMeY4RfiCOB4MpGgBN/Dg=; b=Jrr8nzKXxGfH9rZ05coei/Z5XkXPQKYffsRvq0JJPMkpPWKSPML6OO06eZDIof3ZxE FqR3adURv8HbiY0CSWCrzORDmomvxAk0WLRY9cf18uCIkdSyJjl/VvbdCY4q9Wy7Oc4w +pskRXdmBEFfcC42q/5pqN3PxM0TzWr4gK4sE9xc5wu50UZc34qQkFTahmwhRGZJVytQ iQt96NsYT5jF6oFb+S9xqxKcAjWxbIwls+nZQCaZUiFG50rjkxYUVoWP/KSzWH+XfZJr RVj4ctwi+migEpHkjEJI3k5GUHjwOtKhv4c0gSFmtgVSST9QuZmYcyIVrwx+qWlaHlyG UnUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690944980; x=1691549780; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=J6cxPzKelBRogs1RcqlDIpKMeY4RfiCOB4MpGgBN/Dg=; b=bx4wb1my4LlZe9uJ6tcJOOZ+UHKdIgH1YvIy2ByQvKQYD5dLvdakjKnyg9d9jYbN+s u6nflX4yH5H1aDCof54m2BM89MPbAh9hOTIPz4RZ2PQBffbUlWyfnMfMxL1b6whEReae yFdWCeuN5nGq9rh9kpzEZ9FFLwZ8eBD1+eDZddoncexSyI7yEFO+bsOtirnqs0WHCewD Vh5+gIgpPDn7wDV7ml4uH9Q+LWUQ871NORdceGwBxgumBgACV4Wd4pFY8PuvAClKmJEr URWCCsWlamqzxI9qwYFQcK48JhhtOcxmXaew0HPxRh264ebirdQwepCwG4bVRl17HKNn wq5Q== X-Gm-Message-State: ABy/qLb46JhmHNbPxw0oT+vLAy12HnIUx6M/qnf1C1zmaEFQiQHKRKzy V2MwjfksAVTgCU0ihlJDpVLZaL9HS/Rahw5teQ== X-Google-Smtp-Source: APBJJlHvG4mc4jsYVOJYfZ7h42Tv/EwdyD/HOeVXUVvtcmr8y48s75g7rEDYvVAnOWGuPl5UYIq68sSDD86PJyMPYA== X-Received: from kalesh.mtv.corp.google.com ([2620:15c:211:201:aa5f:ff75:318:255]) (user=kaleshsingh job=sendgmr) by 2002:a25:b084:0:b0:bd5:dc2d:9d7f with SMTP id f4-20020a25b084000000b00bd5dc2d9d7fmr167401ybj.4.1690944980290; Tue, 01 Aug 2023 19:56:20 -0700 (PDT) Date: Tue, 1 Aug 2023 19:56:03 -0700 In-Reply-To: <20230802025606.346758-1-kaleshsingh@google.com> Mime-Version: 1.0 References: <20230802025606.346758-1-kaleshsingh@google.com> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog Message-ID: <20230802025606.346758-2-kaleshsingh@google.com> Subject: [PATCH v2 2/3] mm-unstable: Multi-gen LRU: Avoid race in inc_min_seq() From: Kalesh Singh To: yuzhao@google.com, akpm@linux-foundation.org Cc: surenb@google.com, android-mm@google.com, kernel-team@android.com, Kalesh Singh , stable@vger.kernel.org, Aneesh Kumar K V , Matthias Brugger , AngeloGioacchino Del Regno , "Jan Alexander Steffens (heftig)" , Oleksandr Natalenko , Brian Geffon , Steven Barrett , Suleiman Souhlal , Qi Zheng , linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mediatek@lists.infradead.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230801_195622_346974_169561BE X-CRM114-Status: GOOD ( 16.83 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org inc_max_seq() will try to inc_min_seq() if nr_gens == MAX_NR_GENS. This is because the generations are reused (the last oldest now empty generation will become the next youngest generation). inc_min_seq() is retried until successful, dropping the lru_lock and yielding the CPU on each failure, and retaking the lock before trying again: while (!inc_min_seq(lruvec, type, can_swap)) { spin_unlock_irq(&lruvec->lru_lock); cond_resched(); spin_lock_irq(&lruvec->lru_lock); } However, the initial condition that required incrementing the min_seq (nr_gens == MAX_NR_GENS) is not retested. This can change by another call to inc_max_seq() from run_aging() with force_scan=true from the debugfs interface. Since the eviction stalls when the nr_gens == MIN_NR_GENS, avoid unnecessarily incrementing the min_seq by rechecking the number of generations before each attempt. This issue was uncovered in previous discussion on the list by Yu Zhao and Aneesh Kumar [1]. [1] https://lore.kernel.org/linux-mm/CAOUHufbO7CaVm=xjEb1avDhHVvnC8pJmGyKcFf2iY_dpf+zR3w@mail.gmail.com/ Fixes: d6c3af7d8a2b ("mm: multi-gen LRU: debugfs interface") Cc: stable@vger.kernel.org Cc: Yu Zhao Cc: Aneesh Kumar K V Cc: Andrew Morton Signed-off-by: Kalesh Singh --- Changes in v2: - Add Fixes tag and cc stable mm/vmscan.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index 489a4fc7d9b1..6eecd291756c 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -4439,7 +4439,7 @@ static void inc_max_seq(struct lruvec *lruvec, bool can_swap, bool force_scan) int prev, next; int type, zone; struct lru_gen_folio *lrugen = &lruvec->lrugen; - +restart: spin_lock_irq(&lruvec->lru_lock); VM_WARN_ON_ONCE(!seq_is_valid(lruvec)); @@ -4450,11 +4450,12 @@ static void inc_max_seq(struct lruvec *lruvec, bool can_swap, bool force_scan) VM_WARN_ON_ONCE(!force_scan && (type == LRU_GEN_FILE || can_swap)); - while (!inc_min_seq(lruvec, type, can_swap)) { - spin_unlock_irq(&lruvec->lru_lock); - cond_resched(); - spin_lock_irq(&lruvec->lru_lock); - } + if (inc_min_seq(lruvec, type, can_swap)) + continue; + + spin_unlock_irq(&lruvec->lru_lock); + cond_resched(); + goto restart; } /* From patchwork Wed Aug 2 02:56:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kalesh Singh X-Patchwork-Id: 13337551 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 71373EB64DD for ; Wed, 2 Aug 2023 02:56:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=RakYEJxzwAT4IEFFV9jneV7D+O7+AtKp7c2tCX9Yjvc=; b=o2JOj7jO18RjTwNVYae98kew+Z z4Ty0bZN67TO5CnHummuBB0KVWuTMRyMXndU9axRRfQPz1nGh5bC9z5MFuOVeC+jvVz9Nr/bGjlx6 +rj0jllYVYBh0FTqO248vYXv1vh3SJ/tu4sv0rxzxuB4BaXCuv7S6d07I4S5qJNG+ciaRYssX6ZUN XCbDl60TOCDwOK0vlTptbv6dDFIUkzA6CZ+RXqE4Z2e2j8U7ZG8MydA8RiQhx7CUpq1YzAOunb5qB iMZ4iuf2lkdjI5LJsRndTQ79k4ycY3i/wgCFWcZ0CeVvSLuQD2RpXOeOqLKd9fg878bRzRqfJyXRF 4zTsuV8w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qR22I-003pD1-0C; Wed, 02 Aug 2023 02:56:30 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qR22E-003p9l-3A for linux-arm-kernel@lists.infradead.org; Wed, 02 Aug 2023 02:56:28 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-583c1903ad3so74215767b3.2 for ; Tue, 01 Aug 2023 19:56:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1690944984; x=1691549784; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=/hIR85jBvXabpNDlh2e7xtBRwmCRiojtCshBAYjOX1Y=; b=NaC24ZANDfp+9XAVYJ2zthvJ8rUBTQ6l5oOsseG6mfA3cvF+arl71fQG1qHAhLbqJ9 9qeh16JOmABIOlkWmOSbROyLEpFlmq9EQUA6/uFoeGgx/95B08yoDHueOaoh/i5155mY XVuhu3IiHZBf1PozmtFvavLXWlrDny3HMWCrGbcbi3wm/Avg9RqMK4JG4u4VTr67QtHd KbHI/PNLuqrNbAwO5JvECyVMzM5KsSmQk2W4uE2b/bpTqBDpICou5hRnR2Ky2iDO0WiX xNEzbL/OrqT3RifP1ne+1qALr+MKSfhXAnsSEvF0phJYNv0q7iAwZlhSP5UEXbHyOUib Cf7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690944984; x=1691549784; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=/hIR85jBvXabpNDlh2e7xtBRwmCRiojtCshBAYjOX1Y=; b=k0PeL1K9yUb5mq8I+Pg9ZsYx1njdVS8MeEXuxWrPqnOlNc6Y/UZIlNhnpayFOVaFPp stmiE+eSnnGEnzdmw0XQAOUOlh8zUyJ4HCgg/fUa3CZNnAFy8g6lEhKyAipvJqsw6JY7 c4TUB+BaJCwo8SPKBJwIYdQbVRGq+Nbi7RUnc8xsU+H9GILJwy6UJYFMc3n8vKvG6Phh 9dGBDkMKJmODzpw/FTB36YZVBf98M3VlGE53JE1D0moCPhO+YmMLgSMnAxVDBhNH2DzW oj5c9SslVAyfS7NzgkF8Rj8rwXRI0efb5hFMMWAjZYs2cFVzpgWxzMpGn/+351qm2lb4 FY4g== X-Gm-Message-State: ABy/qLbzF4rs0WQNu0VroOKyXBdapvAm2kE+85YVKRk2Y2i7v1hjFWtG 1zxF4axlmuWhrSK9tfCrtkgJqWoXXp4pkWU+Jw== X-Google-Smtp-Source: APBJJlHj0tBKgoeUZ7ci0leYLXMuHGiQZrYMbH4Cl/uWTIvoLc5trBzIvMofv2F1eZ7phbcd71NDee6Wx5e539GPkA== X-Received: from kalesh.mtv.corp.google.com ([2620:15c:211:201:aa5f:ff75:318:255]) (user=kaleshsingh job=sendgmr) by 2002:a81:a706:0:b0:583:4f82:b9d9 with SMTP id e6-20020a81a706000000b005834f82b9d9mr120587ywh.5.1690944984182; Tue, 01 Aug 2023 19:56:24 -0700 (PDT) Date: Tue, 1 Aug 2023 19:56:04 -0700 In-Reply-To: <20230802025606.346758-1-kaleshsingh@google.com> Mime-Version: 1.0 References: <20230802025606.346758-1-kaleshsingh@google.com> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog Message-ID: <20230802025606.346758-3-kaleshsingh@google.com> Subject: [PATCH v2 3/3] mm-unstable: Multi-gen LRU: Fix can_swap in lru_gen_look_around() From: Kalesh Singh To: yuzhao@google.com, akpm@linux-foundation.org Cc: surenb@google.com, android-mm@google.com, kernel-team@android.com, Kalesh Singh , Matthias Brugger , AngeloGioacchino Del Regno , Suleiman Souhlal , Steven Barrett , Oleksandr Natalenko , "Jan Alexander Steffens (heftig)" , Qi Zheng , Brian Geffon , Barry Song , linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mediatek@lists.infradead.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230801_195627_017078_8253C424 X-CRM114-Status: GOOD ( 13.39 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org walk->can_swap might be invalid since it's not guaranteed to be initialized for the particular lruvec. Instead deduce it from the folio type (anon/file). Fixes: 018ee47f1489 ("mm: multi-gen LRU: exploit locality in rmap") Cc: Yu Zhao Cc: Andrew Morton Signed-off-by: Kalesh Singh --- Changes in v2: - Add Fixes tag mm/vmscan.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index 6eecd291756c..b4329f93a682 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -4656,6 +4656,7 @@ void lru_gen_look_around(struct page_vma_mapped_walk *pvmw) pte_t *pte = pvmw->pte; unsigned long addr = pvmw->address; struct folio *folio = pfn_folio(pvmw->pfn); + bool can_swap = !folio_is_file_lru(folio); struct mem_cgroup *memcg = folio_memcg(folio); struct pglist_data *pgdat = folio_pgdat(folio); struct lruvec *lruvec = mem_cgroup_lruvec(memcg, pgdat); @@ -4704,7 +4705,7 @@ void lru_gen_look_around(struct page_vma_mapped_walk *pvmw) if (!pte_young(ptent)) continue; - folio = get_pfn_folio(pfn, memcg, pgdat, !walk || walk->can_swap); + folio = get_pfn_folio(pfn, memcg, pgdat, can_swap); if (!folio) continue;