From patchwork Sun Apr 2 10:42:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Aneesh Kumar K.V" X-Patchwork-Id: 13197365 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6ADE9C7619A for ; Sun, 2 Apr 2023 10:43:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 10CB26B0078; Sun, 2 Apr 2023 06:43:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0BCC76B007B; Sun, 2 Apr 2023 06:43:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E9F8F6B007D; Sun, 2 Apr 2023 06:43:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id DB7E26B0078 for ; Sun, 2 Apr 2023 06:43:21 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 61F41AB774 for ; Sun, 2 Apr 2023 10:43:21 +0000 (UTC) X-FDA: 80636114202.26.C67B53E Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by imf07.hostedemail.com (Postfix) with ESMTP id 05E2C40006 for ; Sun, 2 Apr 2023 10:43:18 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=tj18M0pB; dmarc=pass (policy=none) header.from=ibm.com; spf=pass (imf07.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1680432199; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=5qxkyCYd49dRRXkdSjfBTAzu3lls6ES4h0/iKkFPgcE=; b=R9Pr/oI2XGiWdfcJZ2tm50JiMSsv+kT5AzDxUUDusicqvOu64nHOJBpySUXW8Yy1Ell3se hGXJUN1sfxeMmo0ZLi+oIAUnTSsWBshhHn12ayRGLupsn2MY48yOsvZ3XfL5SSUKUU5vys w/ctczCy3FIqY1lriDHKlynfPYUS0gc= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=tj18M0pB; dmarc=pass (policy=none) header.from=ibm.com; spf=pass (imf07.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1680432199; a=rsa-sha256; cv=none; b=fQov2bj3nzwRVXufXepFHdLlYILtB8cZcbtQrU/Y/qXfC7fTZpfu9GFIDL/ibDpfs5Vz+0 KAWV9EZ/hbHlJba2X04pvl823YTs+nCj5oxrtxXQoTbr5bWrsqORx2kghCwyhTUOOP5ZW/ d/0VFQvXIWYOsucYlXKuhWhdq6LZyTs= Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 3327P8bT019990; Sun, 2 Apr 2023 10:43:13 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=5qxkyCYd49dRRXkdSjfBTAzu3lls6ES4h0/iKkFPgcE=; b=tj18M0pBVsy0g6uRtKK9DGlgCinaHUACvdYgHxUAXcLL1kE0kgLAX15zLobanSihFHtJ 1xWzmp/NFNo6dtceDR5tsTLWcaA94DLZGSFRkUKJkbGENlGXoQX5siH1pzEDS+KUh9bq UVBwqOwh+fdMaPvBPiJNczI+R5bi3VRRdJcT6/9qyknFM4PDwxvVM3sdojYuLG5b9z5q M4Rlu3AR99a9Tv6IdncynOOeH/g4oNu1FmXtVtS1171BDlNJ1eAwPJDTQlI+Qes0WQcA 4wfk+nXf57gDVIac+dLSltbM6tyBHKOXnSHMhq33ia8bHRyGpFY6Mdj9o8pBsVVuZjKW Ig== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3ppxdtfb2y-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sun, 02 Apr 2023 10:43:12 +0000 Received: from m0098409.ppops.net (m0098409.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 332Ae5Z4013827; Sun, 2 Apr 2023 10:43:12 GMT Received: from ppma01dal.us.ibm.com (83.d6.3fa9.ip4.static.sl-reverse.com [169.63.214.131]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3ppxdtfb2q-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sun, 02 Apr 2023 10:43:12 +0000 Received: from pps.filterd (ppma01dal.us.ibm.com [127.0.0.1]) by ppma01dal.us.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 3328bWKO016965; Sun, 2 Apr 2023 10:43:11 GMT Received: from smtprelay03.wdc07v.mail.ibm.com ([9.208.129.113]) by ppma01dal.us.ibm.com (PPS) with ESMTPS id 3ppc88fj6y-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sun, 02 Apr 2023 10:43:11 +0000 Received: from smtpav03.dal12v.mail.ibm.com (smtpav03.dal12v.mail.ibm.com [10.241.53.102]) by smtprelay03.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 332Ah9sC24904410 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sun, 2 Apr 2023 10:43:09 GMT Received: from smtpav03.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 861BE58064; Sun, 2 Apr 2023 10:43:09 +0000 (GMT) Received: from smtpav03.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B98D158056; Sun, 2 Apr 2023 10:43:05 +0000 (GMT) Received: from skywalker.ibmuc.com (unknown [9.43.8.200]) by smtpav03.dal12v.mail.ibm.com (Postfix) with ESMTP; Sun, 2 Apr 2023 10:43:05 +0000 (GMT) From: "Aneesh Kumar K.V" To: linux-mm@kvack.org, akpm@linux-foundation.org Cc: Dave Hansen , Johannes Weiner , Matthew Wilcox , Mel Gorman , Yu Zhao , Wei Xu , Guru Anbalagane , "Aneesh Kumar K.V" Subject: [RFC PATCH v1 3/7] mm: multi-gen LRU: avoid using generation stored in page flags for generation Date: Sun, 2 Apr 2023 16:12:36 +0530 Message-Id: <20230402104240.1734931-4-aneesh.kumar@linux.ibm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230402104240.1734931-1-aneesh.kumar@linux.ibm.com> References: <20230402104240.1734931-1-aneesh.kumar@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: IEDsy_NM0kJEcyGtdsH50S6N8qCe916z X-Proofpoint-GUID: 7iFHbo6PJVUYWe52UvWmk7XU7G6C7mrL X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-31_07,2023-03-31_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 clxscore=1015 bulkscore=0 spamscore=0 suspectscore=0 adultscore=0 mlxlogscore=999 impostorscore=0 malwarescore=0 mlxscore=0 phishscore=0 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2303200000 definitions=main-2304020092 X-Rspamd-Queue-Id: 05E2C40006 X-Rspamd-Server: rspam09 X-Rspam-User: X-Stat-Signature: t7gg4pwiwbt3d11zeni4mggxkpkp78j5 X-HE-Tag: 1680432198-732378 X-HE-Meta: U2FsdGVkX1+26leUjCY9L5ZkWVNEBm3YXwKMO+WxSxkyIXAQAv9Zg9HQEpr0u6ethM5RI3HHPnaHk0pywzVtZbWbGcwDbSy8rMWjivTRTd/UPtVENSatpbOG9DpFVNWNAZllMC2mafVRIIhFSKxSbZd0Vsuwg2dPPyJh/J2w8cdwF3AM8k5Aq5IPXQY6LhbL5kLCgxJZljFRui9dDABbbO7U6uJayl8imUmmTlug8X85/H6YssqkOHGKoVdHjLK8DNiucw3tVXTL5bFl14Ym7MA2xUzeNpo57+Bfwfl0sjCOnsGXsgRc7DHcdz+SWhnBj9JKSHErCc6QImHZ4wOpos/9WnRuOfbHU/CnvZ6KHlbrbgf/zfsHVfHnd/4gc9DP7ne3v9BtbEt/rocZJgGCeRne9UigZdz+POl2d9myaIfSRUnha7SNtwyY0OmFHWY4sPmaEMvJiJLCzi/x+YNXHQSrwws3wDUD2MYtpzRzxB0xx5cyxFATRyaOrtbaLx4j9ECN9gxHUazM317JgcgWmsoGR1zVV7iSzWRIU0e/ObEvSj1hghhjwQBzK8kjrEUEWpblYMgb9tuVlrWUXL+X4fowBmCQu0tQOiBOy6URQ7KGE2q1I1c3caD/f3MNFYbKIfMzofRtueNi18hlYRf9FWMvGt6N+pzimhOXrycmi372pUu6GMsB10FfkuN46nsB6V7imj+HsJqEJ3+GiOSgLtsmwIr3kl3o5Jwdw/2YYfLsJ84x7l6gVSX/Sf3BmYSMwGs/RV3eXaQ9Cos/YNhmtT8ez5ejSeIwCRXsFBVHMKRsns4tp53Qzen6VSXV2KnaPHVKc3kEhgCm9fqQ7Mg13nkuJ8rCLBYpz1loLvewBOfgIRABtA7lcYOqD2bFnIsHiOineSysskuE9ylsVP4tVBruwmSTnhzKvHpldAoU4WosPLsuPDr5FuvHYmnnacabxpcVmneFbuTFu/r+VHc VzBpHcg1 wnzwCGIKom/A5mVnVao8NO294kXZsL1yXlOEQ5yFkZ5nH6BPEMmk8DeydzmpCEe5ysXfMw67Xrf9HJY9E04iiuorsxNfcb4pa01/8HIWAT4PuHhn+JEfy2w+qi3pIkXTBSFebWCNuyY/WirXDeHEULbuGuoS4pI4VN1J3 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Some architectures can handle different methods for determining page access count. In such case, we may not really need to use page flags for tracking generation. We can possibly derive generation directly from arch-supported access count values. Hence avoid using page flags to store generation in that case. Signed-off-by: Aneesh Kumar K.V --- include/linux/page_aging.h | 26 ++++++++++++++++++++++++++ mm/rmap.c | 4 +++- mm/vmscan.c | 34 +++++++++++++++++++++++++--------- 3 files changed, 54 insertions(+), 10 deletions(-) create mode 100644 include/linux/page_aging.h diff --git a/include/linux/page_aging.h b/include/linux/page_aging.h new file mode 100644 index 000000000000..ab77f4578916 --- /dev/null +++ b/include/linux/page_aging.h @@ -0,0 +1,26 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ + +#ifndef _LINUX_PAGE_AGING_H +#define _LINUX_PAGE_AGING_H + +#ifndef arch_supports_page_access_count +static inline bool arch_supports_page_access_count(void) +{ + return false; +} +#endif + +#ifdef CONFIG_LRU_GEN +#ifndef arch_get_lru_gen_seq +static inline unsigned long arch_get_lru_gen_seq(struct lruvec *lruvec, struct folio *folio) +{ + int type = folio_is_file_lru(folio); + + return lruvec->lrugen.min_seq[type]; +} +#endif +#endif /* CONFIG_LRU_GEN */ + +#endif + + diff --git a/mm/rmap.c b/mm/rmap.c index b616870a09be..1ef3cb8119d5 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -74,6 +74,7 @@ #include #include #include +#include #include @@ -825,7 +826,8 @@ static bool folio_referenced_one(struct folio *folio, if (pvmw.pte) { if (lru_gen_enabled() && pte_young(*pvmw.pte) && !(vma->vm_flags & (VM_SEQ_READ | VM_RAND_READ))) { - lru_gen_look_around(&pvmw); + if (!arch_supports_page_access_count()) + lru_gen_look_around(&pvmw); referenced++; } diff --git a/mm/vmscan.c b/mm/vmscan.c index f92b689af2a5..518d1482f6ab 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -62,6 +62,7 @@ #include #include #include +#include #include "internal.h" #include "swap.h" @@ -4934,7 +4935,7 @@ void lru_gen_look_around(struct page_vma_mapped_walk *pvmw) static bool sort_folio(struct lruvec *lruvec, struct folio *folio, int tier_idx) { bool success; - int gen = folio_lru_gen(folio); + int gen; int type = folio_is_file_lru(folio); int zone = folio_zonenum(folio); int delta = folio_nr_pages(folio); @@ -4942,7 +4943,6 @@ static bool sort_folio(struct lruvec *lruvec, struct folio *folio, int tier_idx) int tier = lru_tier_from_refs(refs); struct lru_gen_struct *lrugen = &lruvec->lrugen; - VM_WARN_ON_ONCE_FOLIO(gen >= MAX_NR_GENS, folio); /* unevictable */ if (!folio_evictable(folio)) { @@ -4963,8 +4963,14 @@ static bool sort_folio(struct lruvec *lruvec, struct folio *folio, int tier_idx) return true; } - /* promoted */ + if (!arch_supports_page_access_count()) { + gen = folio_lru_gen(folio); + VM_WARN_ON_ONCE_FOLIO(gen >= MAX_NR_GENS, folio); + } else + gen = lru_gen_from_seq(arch_get_lru_gen_seq(lruvec, folio)); + if (gen != lru_gen_from_seq(lrugen->min_seq[type])) { + /* promote the folio */ list_move(&folio->lru, &lrugen->lists[gen][type][zone]); return true; } @@ -5464,12 +5470,22 @@ bool lru_gen_add_folio(struct lruvec *lruvec, struct folio *folio, bool reclaimi */ if (folio_test_active(folio)) seq = lrugen->max_seq; - else if ((type == LRU_GEN_ANON && !folio_test_swapcache(folio)) || - (folio_test_reclaim(folio) && - (folio_test_dirty(folio) || folio_test_writeback(folio)))) - seq = lrugen->min_seq[type] + 1; - else - seq = lrugen->min_seq[type]; + else { + /* + * For a non active folio use the arch based + * aging details to derive the MGLRU generation. + */ + seq = arch_get_lru_gen_seq(lruvec, folio); + + if (seq == lrugen->min_seq[type]) { + if ((type == LRU_GEN_ANON && + !folio_test_swapcache(folio)) || + (folio_test_reclaim(folio) && + (folio_test_dirty(folio) || + folio_test_writeback(folio)))) + seq = lrugen->min_seq[type] + 1; + } + } gen = lru_gen_from_seq(seq); flags = (gen + 1UL) << LRU_GEN_PGOFF;