From patchwork Tue Dec 13 09:27:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13071850 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C3C56C4167B for ; Tue, 13 Dec 2022 09:11:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A24758E0002; Tue, 13 Dec 2022 04:11:26 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7003B8E0006; Tue, 13 Dec 2022 04:11:26 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3CD648E0002; Tue, 13 Dec 2022 04:11:26 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 1BBB48E0006 for ; Tue, 13 Dec 2022 04:11:26 -0500 (EST) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id DA529140D46 for ; Tue, 13 Dec 2022 09:11:25 +0000 (UTC) X-FDA: 80236714530.09.020CCB3 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by imf12.hostedemail.com (Postfix) with ESMTP id 47AF440005 for ; Tue, 13 Dec 2022 09:11:22 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf12.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1670922684; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dUXwDIxyAE99qPFcVmtGA/Gfbue+3uIg4UM27pTsTzo=; b=YRIZa4tC3vfqyZcwnH28m9l1/hOVSsaDO8TuDYTWSS7gQAv5CbU6QPqjwBY1wiCmyGqldy kuaOukGYg9Stq+v4dMGnLro5ii9jgizpZ06aud44w1suy9SoR1Eiv0zjdr4Yk95GoQ+29h zfHKeBFnOtXU/Qmy7APdgvNNPkCIJPs= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf12.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1670922684; a=rsa-sha256; cv=none; b=IvkJjc6PHRLM2sNEs8L0F43DpSiHSNY5N5ZdJjw4AgzFlChVTlVs7iTkAhKaHw7ygxC6Rl Itm4Z6gxyEkM+sxgTjQkYLLvPf5jGVFWWMUBQPCNYNEFpTbnDHDY0AYWsl+laqSS6rLEG0 2x8Kp3w7D/u36KkI+5FJ9YVZ/y+1Cww= Received: from dggpemm500001.china.huawei.com (unknown [172.30.72.54]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4NWXgW3V50zqT0X; Tue, 13 Dec 2022 17:07:03 +0800 (CST) Received: from localhost.localdomain.localdomain (10.175.113.25) by dggpemm500001.china.huawei.com (7.185.36.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Tue, 13 Dec 2022 17:11:19 +0800 From: Kefeng Wang To: Andrew Morton , David Hildenbrand , Oscar Salvador , SeongJae Park CC: , , , , , Kefeng Wang Subject: [PATCH -next 2/8] mm: page_idle: Convert page idle to use folios Date: Tue, 13 Dec 2022 17:27:29 +0800 Message-ID: <20221213092735.187924-3-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20221213092735.187924-1-wangkefeng.wang@huawei.com> References: <20221213092735.187924-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.113.25] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm500001.china.huawei.com (7.185.36.107) X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 47AF440005 X-Stat-Signature: zp9sohit53phjosihqk4skzgcm37zps9 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1670922682-285278 X-HE-Meta: U2FsdGVkX1+R37bT1/PsrywemRR+tIGHBUkFos5WR532x4ucBNRe1AqwckTrjSRU7L6cvSQ4LdAeqrdYgzm2QhvGPffbonWHxaHGwyZ3i5WYXoKKgU3tu/MlrKbtPLOpDwnUCeDdDMh38TWhHIUssemK4GAXAxwEyCj6jN6+p49liu2XEFgUSWc/6uRrREIF6VnPTLbIhK7c3zyuKIoNTU6A6EGtBUGNByZV/Mj8kwfnqbKN4zQNpMZgMSaQYM82asf3/w7JX0qk3s2Xu1Rx1KthQJMMtW8W3/HNhPFqKVOF52OpllliiLp7XopNu5dI3Y3hNp0S227UlYNzmC6YeGCdzCWPWEzde9cHCcQTpgsogUAEDtI8OS+l9DPvSWavks6D2OjSvQBd5tU7gJgTrkTnj8o+2u3NFwqUZZnNskipeZY34+NErYgoA+fKcUYyZ57v+BJzuJUakWSq8hk3h6w41H50XN3UJ9Zmp8wiUx5tZWzDAasvVrgrW+vPDNlYldlRRIzJMQrKXOWC+bN1ArLB7125bl9DSdgJYirry6IM0N0a2w2G4eJjqOyOWbecaNOvmB4tHf41QwLtXwSY8lBgKzLbyKAhejwmabEWHT2j2QhqWn8MpqX2NgNKg6UlR7qKgVxt3BqqXbFChxpS28Kxc0qi/d1Sd5X6FHfmYod5OC0m7/InEgDhCIxYHQDnRPPal7zY7QFf2xH/NX+4M0Zo5dg4LTDlfZIkgh3VNkjIOz5TJu/+2KfQkTlri1XFTEAlGT80EvsHxBC7umMS1cWlBnI9zLD4rh/jQGLdhK2meydV/cAuxVpYTRXRYkwDhYNuMxUvn/aejsul8ipkMyKpbNG/l48V+LUKj2bXWvd496CmPsC7hddGLa2PlFJ/1XOFBggtG012ty/FozVlVN2ER1HfezPOQMcUJ963pP8ENw4/tRWIE7IJxpH6pBrK3Wwzi8WuWVUqUmAC70X aLqoXYWo E4oKss8kSJwBN/o0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This change and rename all page related function to use folio, which save many compound_head() calls and kernel text. Signed-off-by: Kefeng Wang --- mm/page_idle.c | 91 ++++++++++++++++++++++++-------------------------- 1 file changed, 44 insertions(+), 47 deletions(-) diff --git a/mm/page_idle.c b/mm/page_idle.c index bc08332a609c..38641f94cf49 100644 --- a/mm/page_idle.c +++ b/mm/page_idle.c @@ -19,34 +19,33 @@ #define BITMAP_CHUNK_BITS (BITMAP_CHUNK_SIZE * BITS_PER_BYTE) /* - * Idle page tracking only considers user memory pages, for other types of - * pages the idle flag is always unset and an attempt to set it is silently + * Idle folio tracking only considers user memory folios, for other types of + * folios the idle flag is always unset and an attempt to set it is silently * ignored. * - * We treat a page as a user memory page if it is on an LRU list, because it is - * always safe to pass such a page to rmap_walk(), which is essential for idle - * page tracking. With such an indicator of user pages we can skip isolated - * pages, but since there are not usually many of them, it will hardly affect + * We treat a folio as a user memory folio if it is on an LRU list, because it is + * always safe to pass such a folio to rmap_walk(), which is essential for idle + * folio tracking. With such an indicator of user folios we can skip isolated + * folios, but since there are not usually many of them, it will hardly affect * the overall result. * - * This function tries to get a user memory page by pfn as described above. + * This function tries to get a user memory folio by pfn as described above. */ -static struct page *page_idle_get_page(unsigned long pfn) +static struct folio *folio_idle_get_folio(unsigned long pfn) { - struct page *page = pfn_to_online_page(pfn); + struct folio *folio = pfn_to_online_folio(pfn); - if (!page || !PageLRU(page) || - !get_page_unless_zero(page)) + if (!folio || !folio_test_lru(folio) || !folio_try_get(folio)) return NULL; - if (unlikely(!PageLRU(page))) { - put_page(page); - page = NULL; + if (unlikely(!folio_test_lru(folio))) { + folio_put(folio); + folio = NULL; } - return page; + return folio; } -static bool page_idle_clear_pte_refs_one(struct folio *folio, +static bool folio_idle_clear_pte_refs_one(struct folio *folio, struct vm_area_struct *vma, unsigned long addr, void *arg) { @@ -74,8 +73,8 @@ static bool page_idle_clear_pte_refs_one(struct folio *folio, if (referenced) { folio_clear_idle(folio); /* - * We cleared the referenced bit in a mapping to this page. To - * avoid interference with page reclaim, mark it young so that + * We cleared the referenced bit in a mapping to this folio. To + * avoid interference with folio reclaim, mark it young so that * folio_referenced() will return > 0. */ folio_set_young(folio); @@ -83,16 +82,14 @@ static bool page_idle_clear_pte_refs_one(struct folio *folio, return true; } -static void page_idle_clear_pte_refs(struct page *page) +static void folio_idle_clear_pte_refs(struct folio *folio) { - struct folio *folio = page_folio(page); - /* * Since rwc.try_lock is unused, rwc is effectively immutable, so we * can make it static to save some cycles and stack. */ static struct rmap_walk_control rwc = { - .rmap_one = page_idle_clear_pte_refs_one, + .rmap_one = folio_idle_clear_pte_refs_one, .anon_lock = folio_lock_anon_vma_read, }; bool need_lock; @@ -110,12 +107,12 @@ static void page_idle_clear_pte_refs(struct page *page) folio_unlock(folio); } -static ssize_t page_idle_bitmap_read(struct file *file, struct kobject *kobj, +static ssize_t folio_idle_bitmap_read(struct file *file, struct kobject *kobj, struct bin_attribute *attr, char *buf, loff_t pos, size_t count) { u64 *out = (u64 *)buf; - struct page *page; + struct folio *folio; unsigned long pfn, end_pfn; int bit; @@ -134,19 +131,19 @@ static ssize_t page_idle_bitmap_read(struct file *file, struct kobject *kobj, bit = pfn % BITMAP_CHUNK_BITS; if (!bit) *out = 0ULL; - page = page_idle_get_page(pfn); - if (page) { - if (page_is_idle(page)) { + folio = folio_idle_get_folio(pfn); + if (folio) { + if (folio_test_idle(folio)) { /* - * The page might have been referenced via a + * The folio might have been referenced via a * pte, in which case it is not idle. Clear * refs and recheck. */ - page_idle_clear_pte_refs(page); - if (page_is_idle(page)) + folio_idle_clear_pte_refs(folio); + if (folio_test_idle(folio)) *out |= 1ULL << bit; } - put_page(page); + folio_put(folio); } if (bit == BITMAP_CHUNK_BITS - 1) out++; @@ -155,12 +152,12 @@ static ssize_t page_idle_bitmap_read(struct file *file, struct kobject *kobj, return (char *)out - buf; } -static ssize_t page_idle_bitmap_write(struct file *file, struct kobject *kobj, +static ssize_t folio_idle_bitmap_write(struct file *file, struct kobject *kobj, struct bin_attribute *attr, char *buf, loff_t pos, size_t count) { const u64 *in = (u64 *)buf; - struct page *page; + struct folio *folio; unsigned long pfn, end_pfn; int bit; @@ -178,11 +175,11 @@ static ssize_t page_idle_bitmap_write(struct file *file, struct kobject *kobj, for (; pfn < end_pfn; pfn++) { bit = pfn % BITMAP_CHUNK_BITS; if ((*in >> bit) & 1) { - page = page_idle_get_page(pfn); - if (page) { - page_idle_clear_pte_refs(page); - set_page_idle(page); - put_page(page); + folio = folio_idle_get_folio(pfn); + if (folio) { + folio_idle_clear_pte_refs(folio); + folio_set_idle(folio); + folio_put(folio); } } if (bit == BITMAP_CHUNK_BITS - 1) @@ -192,29 +189,29 @@ static ssize_t page_idle_bitmap_write(struct file *file, struct kobject *kobj, return (char *)in - buf; } -static struct bin_attribute page_idle_bitmap_attr = +static struct bin_attribute folio_idle_bitmap_attr = __BIN_ATTR(bitmap, 0600, - page_idle_bitmap_read, page_idle_bitmap_write, 0); + folio_idle_bitmap_read, folio_idle_bitmap_write, 0); -static struct bin_attribute *page_idle_bin_attrs[] = { - &page_idle_bitmap_attr, +static struct bin_attribute *folio_idle_bin_attrs[] = { + &folio_idle_bitmap_attr, NULL, }; -static const struct attribute_group page_idle_attr_group = { - .bin_attrs = page_idle_bin_attrs, +static const struct attribute_group folio_idle_attr_group = { + .bin_attrs = folio_idle_bin_attrs, .name = "page_idle", }; -static int __init page_idle_init(void) +static int __init folio_idle_init(void) { int err; - err = sysfs_create_group(mm_kobj, &page_idle_attr_group); + err = sysfs_create_group(mm_kobj, &folio_idle_attr_group); if (err) { pr_err("page_idle: register sysfs failed\n"); return err; } return 0; } -subsys_initcall(page_idle_init); +subsys_initcall(folio_idle_init);