From patchwork Tue Sep 26 00:52:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13398585 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9F0F8E6FE31 for ; Tue, 26 Sep 2023 00:53:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 15C7B8D002E; Mon, 25 Sep 2023 20:53:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 10BCE8D0005; Mon, 25 Sep 2023 20:53:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F3CDE8D002E; Mon, 25 Sep 2023 20:53:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id E42958D0005 for ; Mon, 25 Sep 2023 20:53:16 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id BCD761A0D9D for ; Tue, 26 Sep 2023 00:53:16 +0000 (UTC) X-FDA: 81276924792.16.00C550B Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by imf17.hostedemail.com (Postfix) with ESMTP id 3BEAB40005 for ; Tue, 26 Sep 2023 00:53:13 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=none; spf=pass (imf17.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1695689595; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LSYNIqiX5/dP1JwV7wGTtypK5JC/Iahb15M57nxl0fM=; b=AK9d4HeofoZt59FvHuSSUCwMW98IZdlY6KpDdv+8A4CV+o/As/tb8ccd109+HoHbQqopLc qf9ymMJYcFekProxUpb5BSfK9qfXHZG9/H2GuF9pK3VJav7vxAPQXbKaykN65xZQ+DEnue pc9K1TCXDyEE8mrZbGmYi+/Qrbr2QE8= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=none; spf=pass (imf17.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1695689595; a=rsa-sha256; cv=none; b=7IgojwaiqrfAQiN6155+nGNaXLB3Pqd9SKJ21WUIC/EXTj6dai+oQSPQp0BQYMFbwxLFD1 xtF+gXKgIkffWSmIe63Fq+HkUr7lT7VSJ8yFN0ojfe3NNj0zXpABFDPT/ZAZUXFeZHgusj QEE1wfzM2/hkRIjJ9zYSzv2o6CGJ5n8= Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.54]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4Rvh3Z2N7HzVlCM; Tue, 26 Sep 2023 08:50:02 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Tue, 26 Sep 2023 08:53:08 +0800 From: Kefeng Wang To: Andrew Morton CC: Mike Rapoport , Matthew Wilcox , David Hildenbrand , , , , Zi Yan , Kefeng Wang Subject: [PATCH -next 1/9] mm_types: add _last_cpupid into folio Date: Tue, 26 Sep 2023 08:52:46 +0800 Message-ID: <20230926005254.2861577-2-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20230926005254.2861577-1-wangkefeng.wang@huawei.com> References: <20230926005254.2861577-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 3BEAB40005 X-Rspam-User: X-Stat-Signature: 8oj4rrfb6exrjsn4dne6wr7d4uos5fmx X-Rspamd-Server: rspam01 X-HE-Tag: 1695689593-789340 X-HE-Meta: U2FsdGVkX1/cPx7DX8jjkF/Z5o7n4H8X9UeRMUUsJhsLs6NUBvwqJt/SKckYlvhR8eaQtny7KUP9CVN1VJl3xlV0+IZa+6tZozR8S1VKL07pqSKr9boZJRfpZhg2TLXMklgRawZsyjR6Xs6xe6athaZXiQH8sTQl3ExmG1HPr63P4HofLvZGgxBCfgcVa23NI2JUu5f0wt0qMytXYMfY/PItclYjLY/xqwELCxwvl4jbI12Ycjlk7gdUBv9AiZ86XagdBaq9/ckYYqdfjBe5Xb8+W8+Oa6CtXIygrBnwK7Xwj42fQXtviFUU8/x1ynvHOkdBWY0ZgBotnufP5BBbLCCSzivD+a4jL5VUXEK7DdH9Kxqe3ff3O0KEvl1jOacekEUDBZCbEDmqURoW8vT1TxQpfUzQ2n/DR3IMhqMA5Mte/mKx2ReukXy51ivMlrjQ8AzV1WqVasTp6pyXTsfryFCgZ5wcpBfmEJvR0Ic84/c8UEl7zTOXg1bUXMxZIcd7l9qXDQKlqoxJcMOBNbOawUOO03/mtxaUJt8620LddwlUkWolA7IDPyjs4MOOKb+dyYO4hzZziivM1ZRE96bWfZVL4pSih5B8u/4SNlD0yQ4hpCv9ZWnoWCKfh8fftiqSK8y2qok83CkcMWo+Q4/r9hk34/RJp6Dbl3pviVJ2LvayJPY2+hLqX8eYNzvfo5ZS/O0HTa9PEqzrRzdDg0N6qOd+2nWIDg32fj/ri9XXU5uRyfFHm+DyNwkivWMSyDb4sK/m6EAde7+19Zt8gogLddSwdMFFwiSqRi+AMLURYPFfNiDrhvRDBMpX3OaDIG395iwH3EqTzdOLKTUv2KE1xWfn83rFbZuoM6o9TtljunYFZ1PCrl0WUC4Zzmd5aS0WqA60zn3SabDNyF6egM+QbQDIaAk5RrhcTm6wWmm03l/aizVPi3oGqHDviBUkTHZUgFKaxzmmyp+yZ4+taot 4XlDnP5z D2Uf51G5xo7Whc/WOq8BBKWTfLhX/j9V1FbxR8/K7XMWTkRbPt05qDdZxSuKkm7STReBMLvyaKvRf+GwAr5EafVakcMJ+KHxc6LKcI3u90kQaCI7cDuoG/gPfVpPOA/XK8yUJ2Bu2qbOtIpfRK6egXMGlViyl3/undHowyymMl68uSRnRwje/3nt5KWRBxXb3kzBn X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: At present, only arc/sparc/m68k define WANT_PAGE_VIRTUAL, both of them don't support numa balancing, and the page struct is aligned to _struct_page_alignment, it is safe to move _last_cpupid before 'virtual' in page, meanwhile, add it into folio, which make us to use folio->_last_cpupid directly. Signed-off-by: Kefeng Wang --- include/linux/mm_types.h | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 5a995089cbf5..2fdfddd8264a 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -183,6 +183,9 @@ struct page { #ifdef CONFIG_MEMCG unsigned long memcg_data; #endif +#ifdef LAST_CPUPID_NOT_IN_PAGE_FLAGS + int _last_cpupid; +#endif /* * On machines where all RAM is mapped into kernel address space, @@ -210,10 +213,6 @@ struct page { struct page *kmsan_shadow; struct page *kmsan_origin; #endif - -#ifdef LAST_CPUPID_NOT_IN_PAGE_FLAGS - int _last_cpupid; -#endif } _struct_page_alignment; /* @@ -328,6 +327,9 @@ struct folio { atomic_t _refcount; #ifdef CONFIG_MEMCG unsigned long memcg_data; +#endif +#ifdef LAST_CPUPID_NOT_IN_PAGE_FLAGS + int _last_cpupid; #endif /* private: the union with struct page is transitional */ }; @@ -384,6 +386,9 @@ FOLIO_MATCH(_refcount, _refcount); #ifdef CONFIG_MEMCG FOLIO_MATCH(memcg_data, memcg_data); #endif +#ifdef LAST_CPUPID_NOT_IN_PAGE_FLAGS +FOLIO_MATCH(_last_cpupid, _last_cpupid); +#endif #undef FOLIO_MATCH #define FOLIO_MATCH(pg, fl) \ static_assert(offsetof(struct folio, fl) == \ From patchwork Tue Sep 26 00:52:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13398586 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 95010E81806 for ; Tue, 26 Sep 2023 00:53:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D6A818D005C; Mon, 25 Sep 2023 20:53:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CF3558D0005; Mon, 25 Sep 2023 20:53:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BBA958D005C; Mon, 25 Sep 2023 20:53:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id A8BFB8D0005 for ; Mon, 25 Sep 2023 20:53:17 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 7CE6480D39 for ; Tue, 26 Sep 2023 00:53:17 +0000 (UTC) X-FDA: 81276924834.17.5AD7DDF Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by imf03.hostedemail.com (Postfix) with ESMTP id 762A920013 for ; Tue, 26 Sep 2023 00:53:13 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf03.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1695689595; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fVfwzDbo5Kc4i4rlTbkYOzLkB4fLvg8axYuIJPcNuUQ=; b=CBn4tkjPz1LGc3aXGdMoZ9rxjPmGrtaY+7oDcghpVHwQrO2ryVn87lSeXYmRVrsPTithv0 zgIvA1S824hJun9JeGUtaV0jcfTag0ZMxVgNGrWtn0qP8ZrEmB7oWDx1XAwiw7Oq4cpUWL MiWeM8wewX6wS9rsWRNIquIAxeNloXQ= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf03.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1695689595; a=rsa-sha256; cv=none; b=arU7lbmt7oxSWaqjw5ujEIjEyIfv9hWlmCnUDN02S56hokas8u5fV3BDDXBwSrSHL1KWIv TKA799CRaXyRopj0xY7dVt1xJO9CBHLJNhtEFw72GW1zdRoeAts7bGb4Gs7/N873bV0N+f xaX1EbrFPXU48huX3huyQv8QNVffLr0= Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.57]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4Rvh286hNcztT42; Tue, 26 Sep 2023 08:48:48 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Tue, 26 Sep 2023 08:53:08 +0800 From: Kefeng Wang To: Andrew Morton CC: Mike Rapoport , Matthew Wilcox , David Hildenbrand , , , , Zi Yan , Kefeng Wang Subject: [PATCH -next 2/9] mm: mprotect: use a folio in change_pte_range() Date: Tue, 26 Sep 2023 08:52:47 +0800 Message-ID: <20230926005254.2861577-3-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20230926005254.2861577-1-wangkefeng.wang@huawei.com> References: <20230926005254.2861577-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 762A920013 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: tps5jhgxb35zyqhkxwmnnr7rdbjwby5c X-HE-Tag: 1695689593-247053 X-HE-Meta: U2FsdGVkX19PYxWKvjIWnWd+wPFAQWb6OBL0FDukiCJblS0FAHB2QQM3WfOB4lzNui+eFUobIt52PoGbL7WHQua9RScbE01A1hCw659dn/CF8+lrcHDPyqTse+8RkUx2+b0r89t3DfTS9c3EGVGxU1AlwCcb9fdZ7sHvNdrPJwnLdsR9HMPynbOew9IadF2ubrOSIFJ6jdAZFwBeMYxy3Mx8eWFvCDAdd7as2XUZ2Bt3cINISupZQE2MmkHOQH2cfndv5zgTu2Mv0WbN+jqCQ2mqW8VpZ+7WWe9YW0f02bTx77cPyjQ6fcClLOxI92tHJgr4Zdxc2PNHVp6p2uNtstlCf2oYsYpGBy98rI9qWTP1mv6ezRCtgiXN/iEcLlphIFjEd0Co7Anuhxzuej/qjl3bUZcp4rlWPZUxn/TNQ37j9ob+Egz2QXJnSe6p1X2C9jd/rUZvRDvm002wgKpanR5I1YTM9f8cm3P0HvxmT41JJft4jiZhCpzSNbY49ZkhrAlToZ9Gi4IDrCyLprkHrSrwfVmQeI6/+a73I+0y1h1qxGSyW+naNT7CwlIfPPkXnc23K97YrAKlqcZuZ8SbkzYu7LrWRIaip2d/VWNHjevf0fdFVnt9swWXb9HSogYWiys0aKgi04YE4qe4QNftcjjalLcXO1tTaDSSnbjOW+sbyYrKV9NuaNmmevBQ2exA9Zp8o0dVbvCwz8j6trZ9YYo0i0+MgEkVwAv8SDqHdpK5ZF5+91JQ4hXRmtNwBr7heQnBnD7gNYGpncJ94PcLcgv45uyoTfbsQkzMzRdrAgSLNxTukOuQOH9nQQe8zlaPPVZDU4Q8fyQAse1050NtGk+LDikX3z8Uv9ke7Qf0Z1VCdVYofHsaNvh2qSfUmBb7ozH6wsZC1j/wlkZ9uJFOxqAkIO95s49d0N6NwmCShGGh9tlVfdzwhevPmRf9nzwXiTsyPmEkhsJ06o6X+iP ZgudCu1c wP5kY1tVRozB6A8Wl5P8oC5hpqTqT2sOnh5Gj0AT6xxLm6X3OHxcwXH3nw6ERIv8J2aBqmVKjThkaUXC/almtkym9kQUfJEXP4yhNp2WwgZsHyVM8Cgfv4mnB5s28Kxleypha7cX1oR6V3SioUxZbMOffRAsTzBpruhf+0YIxh7f0N7ef+LZ66mKQjZnDIBqswfq1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Use a folio in change_pte_range() to save three compound_head() calls. Signed-off-by: Kefeng Wang --- mm/mprotect.c | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/mm/mprotect.c b/mm/mprotect.c index b94fbb45d5c7..459daa987131 100644 --- a/mm/mprotect.c +++ b/mm/mprotect.c @@ -114,7 +114,7 @@ static long change_pte_range(struct mmu_gather *tlb, * pages. See similar comment in change_huge_pmd. */ if (prot_numa) { - struct page *page; + struct folio *folio; int nid; bool toptier; @@ -122,13 +122,14 @@ static long change_pte_range(struct mmu_gather *tlb, if (pte_protnone(oldpte)) continue; - page = vm_normal_page(vma, addr, oldpte); - if (!page || is_zone_device_page(page) || PageKsm(page)) + folio = vm_normal_folio(vma, addr, oldpte); + if (!folio || folio_is_zone_device(folio) || + folio_test_ksm(folio)) continue; /* Also skip shared copy-on-write pages */ if (is_cow_mapping(vma->vm_flags) && - page_count(page) != 1) + folio_ref_count(folio) != 1) continue; /* @@ -136,14 +137,15 @@ static long change_pte_range(struct mmu_gather *tlb, * it cannot move them all from MIGRATE_ASYNC * context. */ - if (page_is_file_lru(page) && PageDirty(page)) + if (folio_is_file_lru(folio) && + folio_test_dirty(folio)) continue; /* * Don't mess with PTEs if page is already on the node * a single-threaded process is running on. */ - nid = page_to_nid(page); + nid = folio_nid(folio); if (target_node == nid) continue; toptier = node_is_toptier(nid); @@ -157,7 +159,7 @@ static long change_pte_range(struct mmu_gather *tlb, continue; if (sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING && !toptier) - xchg_page_access_time(page, + xchg_page_access_time(&folio->page, jiffies_to_msecs(jiffies)); } From patchwork Tue Sep 26 00:52:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13398591 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5ED24E81806 for ; Tue, 26 Sep 2023 00:53:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 44C0D8D0005; Mon, 25 Sep 2023 20:53:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3355E8D0060; Mon, 25 Sep 2023 20:53:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1F62A8D0005; Mon, 25 Sep 2023 20:53:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 06D858D005C for ; Mon, 25 Sep 2023 20:53:47 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id C7BBF120D4A for ; Tue, 26 Sep 2023 00:53:46 +0000 (UTC) X-FDA: 81276926052.23.593A31B Received: from szxga08-in.huawei.com (szxga08-in.huawei.com [45.249.212.255]) by imf28.hostedemail.com (Postfix) with ESMTP id 99E56C0008 for ; Tue, 26 Sep 2023 00:53:44 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=none; spf=pass (imf28.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.255 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1695689625; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xLDhMjCrVSLoYbaa4gzJ4uRL0hqY2r45hKznONkN8B8=; b=tbh3yAqiBgBg30UHQPL9bHVLMw/HjdUmyGNNGvXMdkyCLRy6jIOnBHjxr69l3RKZEI6gQc xH7PANbG3OF4J62tap2NdMbttyPaXOt5bdiq/z+eKLPMDHE4YvEGVB6Q2C4aHeMFrnPgn3 RWw4Dt34MaMkLESh1/eOSfilrtKaXs4= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=none; spf=pass (imf28.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.255 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1695689625; a=rsa-sha256; cv=none; b=L8SY8ID5d3WglsizC2zxyXDC01QaE5h/j1r3CW9Scg4ZCj+Tm5agtHau7j1Ve1GqJmzSxd UEdi4AlC/hyD6py06zFXOMhzDmhbZ34TD4LVFIhPUHExpjnzmN4Paf/dYxec9IroeEYTSW 2H/NxSwo2tp5D2ZTaT2QyyD1rUEmo6M= Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.56]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4Rvh4c4Mp8z15Mq5; Tue, 26 Sep 2023 08:50:56 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Tue, 26 Sep 2023 08:53:09 +0800 From: Kefeng Wang To: Andrew Morton CC: Mike Rapoport , Matthew Wilcox , David Hildenbrand , , , , Zi Yan , Kefeng Wang Subject: [PATCH -next 3/9] mm: huge_memory: use a folio in change_huge_pmd() Date: Tue, 26 Sep 2023 08:52:48 +0800 Message-ID: <20230926005254.2861577-4-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20230926005254.2861577-1-wangkefeng.wang@huawei.com> References: <20230926005254.2861577-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 99E56C0008 X-Rspam-User: X-Stat-Signature: 1bge7hzys1wo93e5qr1q5cmbottshdou X-Rspamd-Server: rspam01 X-HE-Tag: 1695689624-420526 X-HE-Meta: U2FsdGVkX1+C2eB0ykUjaInEV1A44GNyVMGBwmBy7hQAaqpNvxsj+P64BfoQsd6D2CFh7J+eiQ85rArb4cNjhCFNnmLZ/xgInMCd+q8pEjjMZkig3gU4KBQIAN889JWo/PIxQd03nnCn2RYztxazU/EOMMqoQTlaQ5Vkj6GMAXTZ/dbSam1SNCZLhg2JlHnbx9P/Mp266rGRlCrtKIq3a6ukwThezMr9sEQU75QjzNPSW6vNBmy7GlOlalgCw6xKetVrxWFkrloXyR7He/tKlfHwnK0qE7w6OLf7R8UYD70iH6ZnuvMbk8lUsDd7DYUHCgjYcb0dOcQ5h8PzwULFm7chKF1+aQbDTry/2dnBVO/gxMcdFWLAUckpDQp3eAFG6MGrnDqlOueGpGyYpWUbqxYM92L9qpdHD586r5dDJHchiE0ROiHBHpqDDeeVG6DJYfrvRIiTeFBrycfruanJANOrcLN4B60ht1r4mUkw7p0XMHkEciUaoyYK72Sx8JjWly7/q88cb1nNZm/a6RP58mgPzhIySHYiVOydPri2G7TiIb7ENKLoNSVvjahoNyyi56x9K41uL8fd5I5GX3NIRO2eVkiFACqXXR0O4SgyXsjFv2xMgsg/n3gIuTCnU1AiyaJ0pr3TncuMd0jA/A+h/yzAKVepUp1MR1WVg77HN5GJzShW/IMWovFXg4BTfZWQU/X1KsaUviiOTlnEOVTqVYw39u08IFefwCfXfBdpykHu8vrOXPDW04NYnrfbATi+8WUqR1YHLHWT20G0in8WnYHammu+pRhL9sHeXlKa2gnHxQvN15bvh6nGRm91g0RtXEKJJm8jEly4iIugF07i0JssfhjR1Bu1tWDwz6sVpjyKtF+RFD+An5vq7ZApUoEkh0X/jr0zZCw5pi6FcK4hAIka0y9nV47jAoroRZpEtH3nVVMBdbb6NZUj3knBzGgZwZylheYglh/Os7nFf8o 1wCFOOrm fFVPMgQB2Zc5qVap/3YR1+7QBPCc1Mbux0N7ZC+bJ2RJ8wV+8RC+WgjdhWqWH6TQWQkaS4Y+MaQhp4/J04xnSgyDwsCL8L8ceSlKnQjH8Lo7lVoS/855gbTwY2XgJq9m0S+oXrX4oXDkJperRAFPrnK4Rma4pGtBBrqGiHEKrc7HNfoUqzzo69njShqsPu6fWQJN6 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Use a folio in change_huge_pmd(), this is in preparation for xchg_page_access_time() to folio conversion. Signed-off-by: Kefeng Wang --- mm/huge_memory.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 0f93a73115f7..c7efa214add8 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1849,7 +1849,7 @@ int change_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, #ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION if (is_swap_pmd(*pmd)) { swp_entry_t entry = pmd_to_swp_entry(*pmd); - struct page *page = pfn_swap_entry_to_page(entry); + struct folio *folio = page_folio(pfn_swap_entry_to_page(entry)); pmd_t newpmd; VM_BUG_ON(!is_pmd_migration_entry(*pmd)); @@ -1858,7 +1858,7 @@ int change_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, * A protection check is difficult so * just be safe and disable write */ - if (PageAnon(page)) + if (folio_test_anon(folio)) entry = make_readable_exclusive_migration_entry(swp_offset(entry)); else entry = make_readable_migration_entry(swp_offset(entry)); @@ -1880,7 +1880,7 @@ int change_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, #endif if (prot_numa) { - struct page *page; + struct folio *folio; bool toptier; /* * Avoid trapping faults against the zero page. The read-only @@ -1893,8 +1893,8 @@ int change_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, if (pmd_protnone(*pmd)) goto unlock; - page = pmd_page(*pmd); - toptier = node_is_toptier(page_to_nid(page)); + folio = page_folio(pmd_page(*pmd)); + toptier = node_is_toptier(folio_nid(folio)); /* * Skip scanning top tier node if normal numa * balancing is disabled @@ -1905,7 +1905,8 @@ int change_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, if (sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING && !toptier) - xchg_page_access_time(page, jiffies_to_msecs(jiffies)); + xchg_page_access_time(&folio->page, + jiffies_to_msecs(jiffies)); } /* * In case prot_numa, we are under mmap_read_lock(mm). It's critical From patchwork Tue Sep 26 00:52:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13398589 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4C450E6FE31 for ; Tue, 26 Sep 2023 00:53:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E546F8D005C; Mon, 25 Sep 2023 20:53:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E047A8D0005; Mon, 25 Sep 2023 20:53:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CF37A8D005C; Mon, 25 Sep 2023 20:53:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id C22148D0005 for ; Mon, 25 Sep 2023 20:53:33 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 9472DB4078 for ; Tue, 26 Sep 2023 00:53:33 +0000 (UTC) X-FDA: 81276925506.30.E8427DE Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by imf25.hostedemail.com (Postfix) with ESMTP id 29DADA0007 for ; Tue, 26 Sep 2023 00:53:30 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf25.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1695689611; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1NMXgpblLezaPosE82Jei/cXHdbr/hhXwa/WKnoyZJg=; b=eLSJzYaeLQGmpCec5FakZs4be5wQpxdzXoWeuvQ4WaZhKWu++1IF3RPhKa76Ij8e8QaNmc +SlcDOmjIUKegkA4q+oNO8S1uUR45h2cGnA4vzGWNzEzReuusFDQ0G/mmzvJI1C+mrd0BO THuLZabc80ZsYrOnofmtUlDtEzYd2lw= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf25.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1695689611; a=rsa-sha256; cv=none; b=GoK9Xx8huq31LpdPj1tN9p09IBsV0Y/1w3tf1bZ+DOLNl9221BPVScPnzUh7UgkuO4++I4 wzL4ESiSWQN0Uz4SxjeepzCAjD1u8atX2w3uqrnam3RwjZ/ig2r0e0DX2F7cW2E9dYmMtg Ga5MnySy/645y4h6vajnPLH8F34H66E= Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.57]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4Rvh2n28SpzNnv8; Tue, 26 Sep 2023 08:49:21 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Tue, 26 Sep 2023 08:53:09 +0800 From: Kefeng Wang To: Andrew Morton CC: Mike Rapoport , Matthew Wilcox , David Hildenbrand , , , , Zi Yan , Kefeng Wang Subject: [PATCH -next 4/9] mm: convert xchg_page_access_time to xchg_folio_access_time() Date: Tue, 26 Sep 2023 08:52:49 +0800 Message-ID: <20230926005254.2861577-5-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20230926005254.2861577-1-wangkefeng.wang@huawei.com> References: <20230926005254.2861577-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Rspam-User: X-Stat-Signature: krcp4zxse7rb4xbj91phkm6ixdza4iw8 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 29DADA0007 X-HE-Tag: 1695689610-246513 X-HE-Meta: U2FsdGVkX1/WBIW9miRxfGAEEv0hFarUqsp9WeUQ9bjPz31MsG0xabEusOcVAXJW4CnTJ5i9nW4U02F51txdWvtjjS73lHtDWBFj7VmCUpai+JdavgJ/ofrwMh5at/Lin+yTsY633Qnls1om9b0IrxsNIM2/qfEvth8aZepDJTzKcUwF4l6OGHYg0EV2mp8uXRZfIww317iENb9s2s8j8vc5keG8iXhW60PZJv4w7dmz8zCuifBRC+F5MOIjWgyfU/lmMNZEj/fkRn+B2FsCT68Uu8EtuNxxhdd9dOIxegZDDnusMf5iEdW76MQNZ4/cWVnrgENWindfFOcdYyDWogBUX4oU9y2VFLKnFBnXTRICakCAQLySbeEJZ4tLjUweEx5xgxF52NJKggR8uxe84F4WPy9bMkBbZXehvqr97btWsoEX1AIXyD8Wunt50069oYk4Sxf/lxsS/WCrs0ZTDxfLsubJ0ZvePQrQBsE3/vw820GZA5nUsnnpYRw78IqPIMkgKu1rvxpslFLLQbFJ325GGeD1nEn5qEZkNBWqVIHy3QKJ6xw8Fb+O/cbbcX+Lo2SVfruIlZDE1XVxB30QP0WTe++bWqsq1GzE4P9wdnJbqqj09WWkcnDoXqciofU3SkEzRK05g0lFAGdENjK8U0Lg+6xYAFisUCjTmbuXPq4JE/1Wvr4NyR2NfueX1T7sZK3SzUcP+o27hChg7tJBcBNY3J98sZlJi8JdK4kn/+XXvxNL/1JLp6uvL4SHOEZscV0PexpGnbAoHigBmnWtgk5kvmB3aK37WzCiF1ohMBfnGvuh0vUvPavJIHDZFaIl7oPsAIGowLqtGz9Ym/2Ygj5lZvA7hD63uBoGOVePythVz28VxpL2rIMCpT1qaIE+SrP0TrDx0dUoXv5GEShgUcAzTtmgtdjnVPUhC0aYYxFoblQW6wLnLuPeJuEZenFtRmh7XRWiltqQNLaduqG srlCAWol MssIOfiuwD1rYD3lSrbucxSJkpUT98J7APQhyc848CEGM18lWsE8d2cRKXrizVoc04xYk5UeH68RYRJLrtbfTjt+jrJf19bLmqC3/R5/0VcI+SpcMyuY+jasDrIMtckkKah9W8H73OEamu6noNZpyLBm3Y2ROI6jM0BODxFSmIJlwTtBu71pdYWpzpCzXa1vUK7ZQ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Make xchg_page_access_time to take a folio, and rename it to xchg_folio_access_time() since all callers with a folio. Signed-off-by: Kefeng Wang --- include/linux/mm.h | 7 ++++--- kernel/sched/fair.c | 2 +- mm/huge_memory.c | 4 ++-- mm/mprotect.c | 2 +- 4 files changed, 8 insertions(+), 7 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index a1d0c82ac9a7..49b9fa383e7d 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1711,11 +1711,12 @@ static inline void page_cpupid_reset_last(struct page *page) } #endif /* LAST_CPUPID_NOT_IN_PAGE_FLAGS */ -static inline int xchg_page_access_time(struct page *page, int time) +static inline int xchg_folio_access_time(struct folio *folio, int time) { int last_time; - last_time = page_cpupid_xchg_last(page, time >> PAGE_ACCESS_TIME_BUCKETS); + last_time = page_cpupid_xchg_last(&folio->page, + time >> PAGE_ACCESS_TIME_BUCKETS); return last_time << PAGE_ACCESS_TIME_BUCKETS; } @@ -1734,7 +1735,7 @@ static inline int page_cpupid_xchg_last(struct page *page, int cpupid) return page_to_nid(page); /* XXX */ } -static inline int xchg_page_access_time(struct page *page, int time) +static inline int xchg_folio_access_time(struct folio *folio, int time) { return 0; } diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index b507ec29e1e1..afb9dc98a8ee 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -1714,7 +1714,7 @@ static int numa_hint_fault_latency(struct folio *folio) int last_time, time; time = jiffies_to_msecs(jiffies); - last_time = xchg_page_access_time(&folio->page, time); + last_time = xchg_folio_access_time(folio, time); return (time - last_time) & PAGE_ACCESS_TIME_MASK; } diff --git a/mm/huge_memory.c b/mm/huge_memory.c index c7efa214add8..c4f4951615fd 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1905,8 +1905,8 @@ int change_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, if (sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING && !toptier) - xchg_page_access_time(&folio->page, - jiffies_to_msecs(jiffies)); + xchg_folio_access_time(folio, + jiffies_to_msecs(jiffies)); } /* * In case prot_numa, we are under mmap_read_lock(mm). It's critical diff --git a/mm/mprotect.c b/mm/mprotect.c index 459daa987131..1c556651888a 100644 --- a/mm/mprotect.c +++ b/mm/mprotect.c @@ -159,7 +159,7 @@ static long change_pte_range(struct mmu_gather *tlb, continue; if (sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING && !toptier) - xchg_page_access_time(&folio->page, + xchg_folio_access_time(folio, jiffies_to_msecs(jiffies)); } From patchwork Tue Sep 26 00:52:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13398590 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8DCA6E6FE31 for ; Tue, 26 Sep 2023 00:53:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 295678D005F; Mon, 25 Sep 2023 20:53:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 26A448D005C; Mon, 25 Sep 2023 20:53:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0CA2B8D005F; Mon, 25 Sep 2023 20:53:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id EE8EB8D0005 for ; Mon, 25 Sep 2023 20:53:46 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id BD92B1C9D10 for ; Tue, 26 Sep 2023 00:53:46 +0000 (UTC) X-FDA: 81276926052.19.00B49FC Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by imf24.hostedemail.com (Postfix) with ESMTP id C34A8180014 for ; Tue, 26 Sep 2023 00:53:43 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf24.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1695689625; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=a0yIiMoZ+BRAnMLaOS2qTLnjbKRujtjPqddXRR5KgqA=; b=y4S1gD0lvTeEZ88/LAJbr5IkCJjDE9Ar85rJLoANg58N5irmh8E1J9vHY8mvXOwKWbW5kz 20Np4yND/+KzywyDXrFLac4F8miirn7pT1c43n17vCm0uYvD58DTgcXzEsiBEUJ2Mh9kJw +19tvW2g95SMhGHRO0mKXBgjsm3sJ8g= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf24.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1695689625; a=rsa-sha256; cv=none; b=OaDwMZ+K6nsRu2vHvlQJ9KlM89X92jUWsVK9vme8N8VwFI2Gs4p/5S/5796gAH58GzXdvO j+ZyVVfgbKWdM5KtLx/buI7ZZHz4ZANM/kFrsLKc7USMaU5JroclaP6ZwtzkjkEmkrm39C Xy0HYvAebyQSyyMcPHwSVLXF/cb9Drw= Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.53]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4Rvh4c62kfzrSw6; Tue, 26 Sep 2023 08:50:56 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Tue, 26 Sep 2023 08:53:10 +0800 From: Kefeng Wang To: Andrew Morton CC: Mike Rapoport , Matthew Wilcox , David Hildenbrand , , , , Zi Yan , Kefeng Wang Subject: [PATCH -next 5/9] mm: convert page_cpupid_last() to folio_cpupid_last() Date: Tue, 26 Sep 2023 08:52:50 +0800 Message-ID: <20230926005254.2861577-6-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20230926005254.2861577-1-wangkefeng.wang@huawei.com> References: <20230926005254.2861577-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Rspam-User: X-Stat-Signature: 337monrt6eja8oq4b9t3gqzorpciqa4t X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: C34A8180014 X-HE-Tag: 1695689623-190879 X-HE-Meta: U2FsdGVkX19B1hDjQX+UOVZMHAWBqA5fPt6Bb3ylDVA9LhFBFAsT+vOBuVIMDVHqTb2lbAH242rhbuzq4yfAf96t3rM2Owf3mXNs587JeG9reRoCAJ0+1CMfo6p7Q4YRkSceyVsgVK7g6LYEiy2NAab+B5QrubDsuxqqHXxbYC/yyfBVYs8ueIVsAf2qrP3/OjfKC56wghKwA+Xw9ERdx5TLrPnMc1gjrNCeGcwouKjx/jpObVI8s5P8D9fNa0FhUH+8aVCiISw82+TYjyAcM34RC20JwSvYgK+wQL0rvRFVyTontB5AsRT2yEkuFnH8GeBaxsrvotkNk3t9+hB+vButQEHvrcIO3E/TNdN7FvHla65euT18+cilWEneJtnadfiZc4XGxbFfMgxQ6iLD5rl3lPdc4q6Ydt8gxuvXQNO8wqLrmQBVgsESEnwYxc9kKGe6pofN92BHotIM8igp5jJz5tb8lPPB4DM8ZmAflRu+gSUMxOCmWQ6ZDmMjAWQJ9QKkAKOPqW96oR865cn9Vjirbr3yqIbCmRRRNJbf25PXX/+Cbv0oFo2dxs2DVaBlV4yYvBdf3PYCXujPHXPT4rNLSJx4C3eXnFLVIn9NwH+C97OUdWZzj9b7l2wRQU3b1g8i5ECEulYMpFvAGKtzeWZChCbQxOZ0CTrcfYLFxXmJOmr3RgmNJVNUnJZaW+vCvThkjOuBTJOe3Hd+xh73derle1C+/L2TprCVAI25kmn9vhQJqqLtLhYPZXVZOL177/bb5V/U56sZPlMQ6nfWbcwVgQ/plL0AuDRQPd4j89cEboYzxYgF0ObtnZENNZ41Nryei+gKPv3eVa4Jil8TZM5zryTRLYlXbM5Pv8QJga9GDv2NOY1b1ZieipIhIH5aKUM2FZeajePKZ/jYWs5zyXrn/eCPUpUNZAYhCMVDSyTU+njJIbIabSG5AlYVASl9lq+ttbPn8+odLFyi0Wo P1LPjCkc h31A+vgj1+gIBgH4h1KERwvqn+079ckPTUiEeyV9NG0nHF/wuLHRHLvBOhWyPzxysILRiRYODU9d2ZHh4bIM1fOPf3Z0ofa1tf5Ao3UIVj73wuiL4+iq0QOV4p9G0HOulynqhlZmGSFKuIMc23VsAmy/3OlGk5IZrertBPm5BObDY09Bgjm6tOHEJhA/0bX41k1Lo X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Make page_cpupid_last() to take a folio, and rename it to folio_cpupid_last() since all callers with a folio. Signed-off-by: Kefeng Wang --- include/linux/mm.h | 12 ++++++------ mm/huge_memory.c | 4 ++-- mm/memory.c | 2 +- 3 files changed, 9 insertions(+), 9 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 49b9fa383e7d..aa7fdda1b56c 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1689,18 +1689,18 @@ static inline int page_cpupid_xchg_last(struct page *page, int cpupid) return xchg(&page->_last_cpupid, cpupid & LAST_CPUPID_MASK); } -static inline int page_cpupid_last(struct page *page) +static inline int folio_cpupid_last(struct folio *folio) { - return page->_last_cpupid; + return folio->_last_cpupid; } static inline void page_cpupid_reset_last(struct page *page) { page->_last_cpupid = -1 & LAST_CPUPID_MASK; } #else -static inline int page_cpupid_last(struct page *page) +static inline int folio_cpupid_last(struct folio *folio) { - return (page->flags >> LAST_CPUPID_PGSHIFT) & LAST_CPUPID_MASK; + return (folio->flags >> LAST_CPUPID_PGSHIFT) & LAST_CPUPID_MASK; } extern int page_cpupid_xchg_last(struct page *page, int cpupid); @@ -1740,9 +1740,9 @@ static inline int xchg_folio_access_time(struct folio *folio, int time) return 0; } -static inline int page_cpupid_last(struct page *page) +static inline int folio_cpupid_last(struct folio *folio) { - return page_to_nid(page); /* XXX */ + return folio_nid(folio); /* XXX */ } static inline int cpupid_to_nid(int cpupid) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index c4f4951615fd..93981a759daf 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1555,7 +1555,7 @@ vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf) * to record page access time. So use default value. */ if (node_is_toptier(nid)) - last_cpupid = page_cpupid_last(&folio->page); + last_cpupid = folio_cpupid_last(folio); target_nid = numa_migrate_prep(folio, vma, haddr, nid, &flags); if (target_nid == NUMA_NO_NODE) { folio_put(folio); @@ -2508,7 +2508,7 @@ static void __split_huge_page_tail(struct folio *folio, int tail, if (page_is_idle(head)) set_page_idle(page_tail); - page_cpupid_xchg_last(page_tail, page_cpupid_last(head)); + page_cpupid_xchg_last(page_tail, folio_cpupid_last(folio)); /* * always add to the tail because some iterators expect new diff --git a/mm/memory.c b/mm/memory.c index 29c5618c91e5..5ab6e8d45a7d 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4814,7 +4814,7 @@ static vm_fault_t do_numa_page(struct vm_fault *vmf) !node_is_toptier(nid)) last_cpupid = (-1 & LAST_CPUPID_MASK); else - last_cpupid = page_cpupid_last(&folio->page); + last_cpupid = folio_cpupid_last(folio); target_nid = numa_migrate_prep(folio, vma, vmf->address, nid, &flags); if (target_nid == NUMA_NO_NODE) { folio_put(folio); From patchwork Tue Sep 26 00:52:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13398594 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 05775E6FE31 for ; Tue, 26 Sep 2023 01:11:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 319118D005C; Mon, 25 Sep 2023 21:11:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2A1C48D002E; Mon, 25 Sep 2023 21:11:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 11C2B8D005C; Mon, 25 Sep 2023 21:11:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id ED7C78D0005 for ; Mon, 25 Sep 2023 21:11:53 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id C9BBF40D4C for ; Tue, 26 Sep 2023 01:11:53 +0000 (UTC) X-FDA: 81276971706.01.1668978 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by imf16.hostedemail.com (Postfix) with ESMTP id 393CF18000F for ; Tue, 26 Sep 2023 01:11:50 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=none; spf=pass (imf16.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1695690712; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DXDkkBvU79MWZKI+0442dVxorWxgJcWylyLHViP7wEs=; b=Ij0N9rzDYtNXtMbyc5z2Ym/UoI35fcLiXVw0dPHOxtqsPdsYcF3gUAd36YxBoIOSGD8fgy Zu0Ny8YVmdhdb8PPqMHVl359WOed5NZiSs3ccscCQfnj0eVNxkGvoVyedXKgCwMErZryRN W2fIF1GXPwDba5PyEMxeL8mymerNb3Y= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1695690712; a=rsa-sha256; cv=none; b=Fq5hGti/dcK8Q92QXM2IAefFksu8PZOSL5Xh8fDNcMX3FyfQFhbAe029Fh3Xsa3v7GDOSJ e5Psap61moOa+ZvBiUdVEl34JXTdMdHL00Za8P7Pj+Zh5u9XCungbSiN2gI4aj881d9kz1 M6xGxgMkpsZfO8WTiAipZY8Kmd35GLI= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=none; spf=pass (imf16.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.53]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4Rvh2B6SqZztT0Z; Tue, 26 Sep 2023 08:48:50 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Tue, 26 Sep 2023 08:53:10 +0800 From: Kefeng Wang To: Andrew Morton CC: Mike Rapoport , Matthew Wilcox , David Hildenbrand , , , , Zi Yan , Kefeng Wang Subject: [PATCH -next 6/9] mm: make wp_page_reuse() and finish_mkwrite_fault() to take a folio Date: Tue, 26 Sep 2023 08:52:51 +0800 Message-ID: <20230926005254.2861577-7-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20230926005254.2861577-1-wangkefeng.wang@huawei.com> References: <20230926005254.2861577-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Stat-Signature: 6bdzx8t9u5gxujzi4wztdpy4zjsktqs7 X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 393CF18000F X-Rspam-User: X-HE-Tag: 1695690710-688750 X-HE-Meta: U2FsdGVkX18KUjHPf5mmW54L27BRblj6VRXqJWUg64SbUzZ8KBNlxOscz6WJUBTGm40n47aV5X1I6exAw8FZjHK0phCecLUcFJhh42UgoeheHE4/ZMTQmya6J8C3gjKOXIqdYOqFuGJy9hCm/Qwdu54AokYfAxRyym/VpTkUlQql85XN9Yq2OqMQwTLXl1HwRXvCTas+/uly+V4P/O/yfYV1wbXMpHLM6wKLkSqun8meqD7Qdc5ZqNCNYwq19scfLdWs44WY2V5keByUkUeEgkT3nUTHE1h8DaxffClHpZ+mc8xkwcJlbzt8VUYGnJUfmYVGcV8nuBvjjvfV0WrwLruqI0173KYoZQN5fxvPf2rqI1Rj2bJ3Rzel1yeIWESfWattyUD/WQbXPrTeaJKhI7jq2C4M6egyGp1AiyWFFu5pVaClQOF5GouiT8SCA024e7r4mwmU314jOwsxlFbo2lu0+OUuDEvQpBkh1JlqRxGfr8TGvlHC/HElj07I/gy4eS5pm//fzQhpLGawD6665CqOmK0iRYElCeeC4EGn1aC9Yz1Slm89N47UoBiBNbzVkXLUDvCdcFaFk8i4FzlDTcWcyzb1YzPI/7tI4jVjSZ/q2kdwftEjKZD5og7cMkC/1VEDhXTjyqap9BFcK0bTkKOUzoffDUtZRCqDYFnD4tQx3fg/o/PFXiZVM9BFJMcBSwc3MxV+p+IQG7srAknoDeOkE2qhybX7oLOAjSKhHEjPtsvBHiSGja/3gbyUvPg0/k/jLftAZQTjObnWYeTkVoTG7fZ6GQNpI9dwn6X2UnjtEdfrWKJHdW0hmG/6kchj9HXboai2crGZ2qRO+Dfp6KKlzXvwFodkhGJUJn3sq510BpsgPDMhv14UiY30WoZv59pkn2eJiRWP5xm7vdG49geiAnbHMhyxmdBuPNWnRjIziWJBxFtv1MU+/R+kLc3ESw6/zgNEnffcjnWbH+K 1XAEyQZA 6KLq/k8iVpS3TQBJCdjXhBJish5rbS7TcpauDlZkEwtt4B9WoVTzsMKwKx1GY22TWa3ZFuImcooe6OpVsJRxR/fGIUWtJPrOhTz766ipOW4Z4UYyG/R1kwKO1XfekSIypqyzD6rvOB2r5+qc1dt01s/54hiSbM6x9i3v4ykEDru1ekcJgDNs93Qk5TS0X9EjN8twE X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Make finish_mkwrite_fault() to a static function, and convert wp_page_reuse() and finish_mkwrite_fault() to take a folio in preparation for page_cpupid_xchg_last() to folio conversion. Signed-off-by: Kefeng Wang --- include/linux/mm.h | 1 - mm/memory.c | 37 ++++++++++++++++++++----------------- 2 files changed, 20 insertions(+), 18 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index aa7fdda1b56c..9933f6345e66 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1335,7 +1335,6 @@ void set_pte_range(struct vm_fault *vmf, struct folio *folio, struct page *page, unsigned int nr, unsigned long addr); vm_fault_t finish_fault(struct vm_fault *vmf); -vm_fault_t finish_mkwrite_fault(struct vm_fault *vmf); #endif /* diff --git a/mm/memory.c b/mm/memory.c index 5ab6e8d45a7d..119c40e4465e 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3014,23 +3014,24 @@ static vm_fault_t fault_dirty_shared_page(struct vm_fault *vmf) * case, all we need to do here is to mark the page as writable and update * any related book-keeping. */ -static inline void wp_page_reuse(struct vm_fault *vmf) +static inline void wp_page_reuse(struct vm_fault *vmf, struct folio *folio) __releases(vmf->ptl) { struct vm_area_struct *vma = vmf->vma; - struct page *page = vmf->page; pte_t entry; VM_BUG_ON(!(vmf->flags & FAULT_FLAG_WRITE)); - VM_BUG_ON(page && PageAnon(page) && !PageAnonExclusive(page)); + if (folio) { + VM_BUG_ON(folio_test_anon(folio) && + !PageAnonExclusive(vmf->page)); - /* - * Clear the pages cpupid information as the existing - * information potentially belongs to a now completely - * unrelated process. - */ - if (page) - page_cpupid_xchg_last(page, (1 << LAST_CPUPID_SHIFT) - 1); + /* + * Clear the pages cpupid information as the existing + * information potentially belongs to a now completely + * unrelated process. + */ + page_cpupid_xchg_last(vmf->page, (1 << LAST_CPUPID_SHIFT) - 1); + } flush_cache_page(vma, vmf->address, pte_pfn(vmf->orig_pte)); entry = pte_mkyoung(vmf->orig_pte); @@ -3223,6 +3224,7 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf) * writeable once the page is prepared * * @vmf: structure describing the fault + * @folio: the folio of vmf->page * * This function handles all that is needed to finish a write page fault in a * shared mapping due to PTE being read-only once the mapped page is prepared. @@ -3234,7 +3236,8 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf) * Return: %0 on success, %VM_FAULT_NOPAGE when PTE got changed before * we acquired PTE lock. */ -vm_fault_t finish_mkwrite_fault(struct vm_fault *vmf) +static vm_fault_t finish_mkwrite_fault(struct vm_fault *vmf, + struct folio *folio) { WARN_ON_ONCE(!(vmf->vma->vm_flags & VM_SHARED)); vmf->pte = pte_offset_map_lock(vmf->vma->vm_mm, vmf->pmd, vmf->address, @@ -3250,7 +3253,7 @@ vm_fault_t finish_mkwrite_fault(struct vm_fault *vmf) pte_unmap_unlock(vmf->pte, vmf->ptl); return VM_FAULT_NOPAGE; } - wp_page_reuse(vmf); + wp_page_reuse(vmf, folio); return 0; } @@ -3275,9 +3278,9 @@ static vm_fault_t wp_pfn_shared(struct vm_fault *vmf) ret = vma->vm_ops->pfn_mkwrite(vmf); if (ret & (VM_FAULT_ERROR | VM_FAULT_NOPAGE)) return ret; - return finish_mkwrite_fault(vmf); + return finish_mkwrite_fault(vmf, NULL); } - wp_page_reuse(vmf); + wp_page_reuse(vmf, NULL); return 0; } @@ -3305,14 +3308,14 @@ static vm_fault_t wp_page_shared(struct vm_fault *vmf, struct folio *folio) folio_put(folio); return tmp; } - tmp = finish_mkwrite_fault(vmf); + tmp = finish_mkwrite_fault(vmf, folio); if (unlikely(tmp & (VM_FAULT_ERROR | VM_FAULT_NOPAGE))) { folio_unlock(folio); folio_put(folio); return tmp; } } else { - wp_page_reuse(vmf); + wp_page_reuse(vmf, folio); folio_lock(folio); } ret |= fault_dirty_shared_page(vmf); @@ -3436,7 +3439,7 @@ static vm_fault_t do_wp_page(struct vm_fault *vmf) pte_unmap_unlock(vmf->pte, vmf->ptl); return 0; } - wp_page_reuse(vmf); + wp_page_reuse(vmf, folio); return 0; } copy: From patchwork Tue Sep 26 00:52:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13398587 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 772E0E81807 for ; Tue, 26 Sep 2023 00:53:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C7F588D005D; Mon, 25 Sep 2023 20:53:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C09118D0005; Mon, 25 Sep 2023 20:53:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A823F8D005D; Mon, 25 Sep 2023 20:53:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 9B76A8D0005 for ; Mon, 25 Sep 2023 20:53:19 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 68F17C0D75 for ; Tue, 26 Sep 2023 00:53:19 +0000 (UTC) X-FDA: 81276924918.05.8A0433B Received: from szxga08-in.huawei.com (szxga08-in.huawei.com [45.249.212.255]) by imf30.hostedemail.com (Postfix) with ESMTP id 1D9BC80006 for ; Tue, 26 Sep 2023 00:53:15 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=none; spf=pass (imf30.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.255 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1695689596; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iGP1uZCLGGtg16MJd/pOyCwveUPyD+uTEMShphh8Lhg=; b=JSFinvScqCAgdve4RrQjjljHKDF977s81VBUQI0CTohJERPigpIFXKbr/BkFtM0K+Co0jf TaUhvZPmoHrjONVogbM7PFYs6toPYIckH6kGeBJ6cm+5o7lklp+m1rE4992Ccht0qFZ2D3 lWIJjXwIZXbL0yfpslP2/cuj9yt8XJQ= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=none; spf=pass (imf30.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.255 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1695689596; a=rsa-sha256; cv=none; b=viabk9k60KarP10sNrq/iRcLyL+NvTaaalFXP1NED8WER5IgLY5lgnyW+8mZubGcCtDI+9 MpImwCP0Hck60PoGgdBMy05yIhY+MVs9Y3qjJkpWNnJ4vo4qAlUBshzcFUyBtj5ocQmHzS RVEV1CmQa9K99yQVGZ71OtJEDFn6OHg= Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.57]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4Rvh4f4Jj7z15NQw; Tue, 26 Sep 2023 08:50:58 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Tue, 26 Sep 2023 08:53:11 +0800 From: Kefeng Wang To: Andrew Morton CC: Mike Rapoport , Matthew Wilcox , David Hildenbrand , , , , Zi Yan , Kefeng Wang Subject: [PATCH -next 7/9] mm: convert page_cpupid_xchg_last() to folio_cpupid_xchg_last() Date: Tue, 26 Sep 2023 08:52:52 +0800 Message-ID: <20230926005254.2861577-8-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20230926005254.2861577-1-wangkefeng.wang@huawei.com> References: <20230926005254.2861577-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 1D9BC80006 X-Rspam-User: X-Stat-Signature: jqdrtoj57hfr6pyjf6q7thctamoyiici X-Rspamd-Server: rspam01 X-HE-Tag: 1695689595-913301 X-HE-Meta: U2FsdGVkX1/oMdQOG8+t2haWOMkcC2phu/SfFMEGWySC0B58+4J7QUMstYaevm4iMvKkvVvyhD7+86BlM86nKxTRujMVtnfa4C0zGQopVq5rhNOZQebGHRx2zDiPLmBCQBEI+L/ZDlXeSD7XTrGw1gyqt3OZil6nROMeth40RKu65y5Kvr9jngqJGjjv4F5K2pgtYtVGuooxT9MRet0kWI33i3Ini4R1hUM2/IKpo6B1bTiFF4yM2skvk5RGOknBnvX2z7EcoTzGt6YKXCqSCU6QlrSvP+j5Ps0K21PVWDBlGlhNrxLUcO0CvvX0bTAYpWVU67jkGKJHnTt0v1IyN59jP5Dg//GJ+EYADpTgO8wGppR3akHZ9qc/Mtan7gB4Y0XaHhhhTXSMA1cZGhXQmj1jMYjQWwT7zUT3OfWfsTcS2jDssAolYnEnglHxUTfE5kdHcpf8tnyQubKzcQ2rPbgilKpgDbiIQJASSq0HVpOxGn76BL+zGWXJUzTNP1OOcjhCjdvRO9/Hu6B9qHXK+SB3cVSPKIdfq16Fr/SMSPkgasl0Rw/RRMCxxxxE0dOpAiIy4QRgSX2Ptr/c3rcA2iMZDC8lclQ7vZF5AOZCKzhVgPJ8txsO4OadetYSO4SZLISoOnYDLBpD4JqZGjzzSoMfp72DUX4DkkhxRqy5gJfZs4QOOBGAmwUiVUswUQ73RGFf/WxXz80Fv1Q3ol0ram4+Yq5fwyFR9wtFrcEfuU8eWnIA0oaIJlaZDbMFMK7awLiW/KW4aT09Q7Ye8zuzHlLhkSJJi77ys09AyJSegaj3UrizXfY4KC08zZvZz40k651HkpLyW8yn+1R4xJ/5PjwBFU1mqvFCGFnikOri6kXVFmVzCcRxwi25KJB0TWVWPlSVRLEp2rOuxJSGv1syunstsg8MlA+8LqDQ3X3alTHcMcDRtJ6xdMVPNDxU5dIsDYe5CSm/2trL6C/j9YL h/nSHaIe Gzuc1RciRxv8z2Kss+o8zT+wza/p1qFno241atBRRrekKpW9DVZ9SeEAknXVasjDforW68XQIVImGy/r6s+oYZlgSpEwYZ5Spwb54rQQM6YoakmJpaG+zhYBKrDoVjmkCT9geaM+77miD8+MMN/VSPvLCHodsSt3Hm50BhUnsLNCfdABxthwkQQSLN5RREW6ikDCA X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Make page_cpupid_xchg_last() to take a folio, and rename it to olio_cpupid_xchg_last() since all callers with a folio. Signed-off-by: Kefeng Wang --- include/linux/mm.h | 14 +++++++------- kernel/sched/fair.c | 2 +- mm/huge_memory.c | 2 +- mm/memory.c | 2 +- mm/migrate.c | 4 ++-- mm/mmzone.c | 6 +++--- 6 files changed, 15 insertions(+), 15 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 9933f6345e66..a6f4b55bf469 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1683,9 +1683,9 @@ static inline bool __cpupid_match_pid(pid_t task_pid, int cpupid) #define cpupid_match_pid(task, cpupid) __cpupid_match_pid(task->pid, cpupid) #ifdef LAST_CPUPID_NOT_IN_PAGE_FLAGS -static inline int page_cpupid_xchg_last(struct page *page, int cpupid) +static inline int folio_cpupid_xchg_last(struct folio *folio, int cpupid) { - return xchg(&page->_last_cpupid, cpupid & LAST_CPUPID_MASK); + return xchg(&folio->_last_cpupid, cpupid & LAST_CPUPID_MASK); } static inline int folio_cpupid_last(struct folio *folio) @@ -1702,7 +1702,7 @@ static inline int folio_cpupid_last(struct folio *folio) return (folio->flags >> LAST_CPUPID_PGSHIFT) & LAST_CPUPID_MASK; } -extern int page_cpupid_xchg_last(struct page *page, int cpupid); +extern int folio_cpupid_xchg_last(struct folio *folio, int cpupid); static inline void page_cpupid_reset_last(struct page *page) { @@ -1714,8 +1714,8 @@ static inline int xchg_folio_access_time(struct folio *folio, int time) { int last_time; - last_time = page_cpupid_xchg_last(&folio->page, - time >> PAGE_ACCESS_TIME_BUCKETS); + last_time = folio_cpupid_xchg_last(folio, + time >> PAGE_ACCESS_TIME_BUCKETS); return last_time << PAGE_ACCESS_TIME_BUCKETS; } @@ -1729,9 +1729,9 @@ static inline void vma_set_access_pid_bit(struct vm_area_struct *vma) } } #else /* !CONFIG_NUMA_BALANCING */ -static inline int page_cpupid_xchg_last(struct page *page, int cpupid) +static inline int folio_cpupid_xchg_last(struct folio *folio, int cpupid) { - return page_to_nid(page); /* XXX */ + return folio_nid(folio); /* XXX */ } static inline int xchg_folio_access_time(struct folio *folio, int time) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index afb9dc98a8ee..dca1546aa9c1 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -1810,7 +1810,7 @@ bool should_numa_migrate_memory(struct task_struct *p, struct folio *folio, } this_cpupid = cpu_pid_to_cpupid(dst_cpu, current->pid); - last_cpupid = page_cpupid_xchg_last(&folio->page, this_cpupid); + last_cpupid = folio_cpupid_xchg_last(folio, this_cpupid); if (!(sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING) && !node_is_toptier(src_nid) && !cpupid_valid(last_cpupid)) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 93981a759daf..89e65ff46ad4 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2508,7 +2508,7 @@ static void __split_huge_page_tail(struct folio *folio, int tail, if (page_is_idle(head)) set_page_idle(page_tail); - page_cpupid_xchg_last(page_tail, folio_cpupid_last(folio)); + folio_cpupid_xchg_last(new_folio, folio_cpupid_last(folio)); /* * always add to the tail because some iterators expect new diff --git a/mm/memory.c b/mm/memory.c index 119c40e4465e..bf07ebdc24a0 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3030,7 +3030,7 @@ static inline void wp_page_reuse(struct vm_fault *vmf, struct folio *folio) * information potentially belongs to a now completely * unrelated process. */ - page_cpupid_xchg_last(vmf->page, (1 << LAST_CPUPID_SHIFT) - 1); + folio_cpupid_xchg_last(folio, (1 << LAST_CPUPID_SHIFT) - 1); } flush_cache_page(vma, vmf->address, pte_pfn(vmf->orig_pte)); diff --git a/mm/migrate.c b/mm/migrate.c index 7d1804c4a5d9..d41139ccbd3f 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -588,7 +588,7 @@ void folio_migrate_flags(struct folio *newfolio, struct folio *folio) * Copy NUMA information to the new page, to prevent over-eager * future migrations of this same page. */ - cpupid = page_cpupid_xchg_last(&folio->page, -1); + cpupid = folio_cpupid_xchg_last(folio, -1); /* * For memory tiering mode, when migrate between slow and fast * memory node, reset cpupid, because that is used to record @@ -601,7 +601,7 @@ void folio_migrate_flags(struct folio *newfolio, struct folio *folio) if (f_toptier != t_toptier) cpupid = -1; } - page_cpupid_xchg_last(&newfolio->page, cpupid); + folio_cpupid_xchg_last(newfolio, cpupid); folio_migrate_ksm(newfolio, folio); /* diff --git a/mm/mmzone.c b/mm/mmzone.c index 68e1511be12d..cd473f82b647 100644 --- a/mm/mmzone.c +++ b/mm/mmzone.c @@ -93,19 +93,19 @@ void lruvec_init(struct lruvec *lruvec) } #if defined(CONFIG_NUMA_BALANCING) && !defined(LAST_CPUPID_NOT_IN_PAGE_FLAGS) -int page_cpupid_xchg_last(struct page *page, int cpupid) +int folio_cpupid_xchg_last(struct folio *folio, int cpupid) { unsigned long old_flags, flags; int last_cpupid; - old_flags = READ_ONCE(page->flags); + old_flags = READ_ONCE(folio->flags); do { flags = old_flags; last_cpupid = (flags >> LAST_CPUPID_PGSHIFT) & LAST_CPUPID_MASK; flags &= ~(LAST_CPUPID_MASK << LAST_CPUPID_PGSHIFT); flags |= (cpupid & LAST_CPUPID_MASK) << LAST_CPUPID_PGSHIFT; - } while (unlikely(!try_cmpxchg(&page->flags, &old_flags, flags))); + } while (unlikely(!try_cmpxchg(&folio->flags, &old_flags, flags))); return last_cpupid; } From patchwork Tue Sep 26 00:52:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13398595 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D221FE81806 for ; Tue, 26 Sep 2023 01:11:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 625BE8D0005; Mon, 25 Sep 2023 21:11:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 516688D005D; Mon, 25 Sep 2023 21:11:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1B67E8D0005; Mon, 25 Sep 2023 21:11:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 013A58D002E for ; Mon, 25 Sep 2023 21:11:53 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id CAC3E1CAA44 for ; Tue, 26 Sep 2023 01:11:53 +0000 (UTC) X-FDA: 81276971706.29.409E16C Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by imf22.hostedemail.com (Postfix) with ESMTP id 3785DC000F for ; Tue, 26 Sep 2023 01:11:50 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf22.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1695690711; a=rsa-sha256; cv=none; b=sPuoNg4aWUcjVThixkveVwip6QoSUsVk688m6F6BSkKRZEnDfJlixgQqjX7rdCS9Msay5g T2+mCwrPApNS+n42T8jP+OplQO1sipV4BKbSw02O0+7W4aHhGyr4AbwULT1LwGTB8hOisy JnIW7t3pxf0cTZEn3ee+KqlbqMUply8= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf22.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1695690711; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Z4qWbMQVyNDvnH/0/qkU92K86v2t3NC3Tu9QW6Z7yRg=; b=zwcxtzGxNrSaGwwjaD5E2Tnq1unN278OBLn7PGX+LK0mlqmdRK46Tw/xsxv7BCwcUKd/8d Nf7bo70HQ1AKvpIBmBhjg+5ifT96leMt4nbLlPXbly4ObokaBQ5BPSiyvKFy4E2bGQGCq9 3I5bpyRP/7uiAZFNELdAneNSvFhhzwA= Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.55]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4Rvh2C6FzGztT3x; Tue, 26 Sep 2023 08:48:51 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Tue, 26 Sep 2023 08:53:11 +0800 From: Kefeng Wang To: Andrew Morton CC: Mike Rapoport , Matthew Wilcox , David Hildenbrand , , , , Zi Yan , Kefeng Wang Subject: [PATCH -next 8/9] mm: page_alloc: use a folio in free_pages_prepare() Date: Tue, 26 Sep 2023 08:52:53 +0800 Message-ID: <20230926005254.2861577-9-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20230926005254.2861577-1-wangkefeng.wang@huawei.com> References: <20230926005254.2861577-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 3785DC000F X-Stat-Signature: gisgx5aaqoafrrswy4m7rgg3fbj7qkst X-Rspam-User: X-HE-Tag: 1695690710-574779 X-HE-Meta: U2FsdGVkX195w3I9Jck3B2nJ0byNBgS1QbM4zUl/Bdv64icWrbKOKHaGz4pbmuR+APyXRHyPIRuaKh9ERY60Dz45a6QvLLOhNurYTIbbOFFRq3DOVbY9HtqqI9gaYy+nxs2UleeCeZsFEAtbzK5YuCgklJXl1/SI+n+Xi2d3nMgP2tIA5pCeCgdyjG8C9HYTvShb/AlgkmYcTy2oXnBkVPdLXj34yC/qPruJYOKNNfAZWInVNxzS+8FZgdlHChe0Pwcld8fzvCfWKvUuMvf4yU0z2MpexipHiJsfe6dQdXGjS3CjAzoq2lozTYs75VYV5Dq5SBAFfIJIaZVJp0v7710JPvp6MERuSvMkgMqoHUA9yhodzNNe0UWxRifbPJ2+FB9DAaibxpl1CshK6jXvClQM+tD4mJOpNZ9zyiB6Xm6fCYPt6RDfPUhbnOq6siIs7DxOGTo0pPUwfhrOoCNcI7df232uVAOBNny62csuKG1HO/CFeCbBTN8UJ+yiUCndDCsEkmxPeWeqFXusnU0GgYZREiKjpkZjgLPsGtwz7fG/CDk5/5u/5MtKn/85CJsqipK/ncoI2N0PZ6q46LgTC7lZS+MofwwqsqRj1zokM0ZDmGi7ztjKBpfgCn6UM2aRxcVXdUbKBqg+kPSYs5P+qUISsCPOTf7zB+0/E/yAp0Rbbyjn4JFb6NcBxZTLuzJRFtEeoO0tHWlIgFIGGQIUGsIe4Ob8+fUAavlz7TxI4xQYYqxTTyZTtRpNvotpgIUhDnP/T3+zoGbJTuilVRcNd6DKlakyXq4e+v89KFhUvyL7AdsZr9WeIqDs6fsuFtRJGhEP7h3C1R2YoSsh6q6T83JzjzZTWRphiZzfB6+Ipg28DFHm3gjY4QMJOIortyWoZRNcZad+lsuzyWo/Wxx648fcSHZXKNQ6KtqhLEro6bUkFZyjsNbDmjxIwOnPbUq11wCnj5tVLTnH5O6g4eQ Wl5ig9k6 Y0WidNR8MwRfl1PAiMGWzXv0o5eASVWceOz0KSKSNHofJWCFZ0ycb9/SOOMwKtkMPQRKTGcBg7uOr1ySwoQQmcBkdR4ueHYcyD4ED9pDon320z9EQrl/raY1RqeQn6UnpipqYeSaUJygrUJpGLsDffH1Dmw6/GVTEi1KFc+LDX3YwtKvqQEe2QiDZBHKfULEnC5Yy X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The page should not a tail page in free_pages_prepare(), let's use a folio in free_pages_prepare() to save several compound_head() calls. Signed-off-by: Kefeng Wang --- mm/page_alloc.c | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 06be8821d833..a888b9d57751 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1070,6 +1070,7 @@ static __always_inline bool free_pages_prepare(struct page *page, unsigned int order, fpi_t fpi_flags) { int bad = 0; + struct folio *folio = page_folio(page); bool skip_kasan_poison = should_skip_kasan_poison(page, fpi_flags); bool init = want_init_on_free(); @@ -1078,12 +1079,12 @@ static __always_inline bool free_pages_prepare(struct page *page, trace_mm_page_free(page, order); kmsan_free_page(page, order); - if (unlikely(PageHWPoison(page)) && !order) { + if (unlikely(folio_test_hwpoison(folio)) && !order) { /* * Do not let hwpoison pages hit pcplists/buddy * Untie memcg state and reset page's owner */ - if (memcg_kmem_online() && PageMemcgKmem(page)) + if (memcg_kmem_online() && folio_memcg_kmem(folio)) __memcg_kmem_uncharge_page(page, order); reset_page_owner(page, order); page_table_check_free(page, order); @@ -1095,10 +1096,10 @@ static __always_inline bool free_pages_prepare(struct page *page, * avoid checking PageCompound for order-0 pages. */ if (unlikely(order)) { - bool compound = PageCompound(page); + bool compound = folio_test_large(folio); int i; - VM_BUG_ON_PAGE(compound && compound_order(page) != order, page); + VM_BUG_ON_FOLIO(compound && folio_order(folio) != order, folio); if (compound) page[1].flags &= ~PAGE_FLAGS_SECOND; @@ -1114,9 +1115,9 @@ static __always_inline bool free_pages_prepare(struct page *page, (page + i)->flags &= ~PAGE_FLAGS_CHECK_AT_PREP; } } - if (PageMappingFlags(page)) + if (folio_mapping_flags(folio)) page->mapping = NULL; - if (memcg_kmem_online() && PageMemcgKmem(page)) + if (memcg_kmem_online() && folio_memcg_kmem(folio)) __memcg_kmem_uncharge_page(page, order); if (is_check_pages_enabled()) { if (free_page_is_bad(page)) @@ -1130,7 +1131,7 @@ static __always_inline bool free_pages_prepare(struct page *page, reset_page_owner(page, order); page_table_check_free(page, order); - if (!PageHighMem(page)) { + if (!folio_test_highmem(folio)) { debug_check_no_locks_freed(page_address(page), PAGE_SIZE << order); debug_check_no_obj_freed(page_address(page), From patchwork Tue Sep 26 00:52:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13398588 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5D3E4E6FE31 for ; Tue, 26 Sep 2023 00:53:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DE55D8D005E; Mon, 25 Sep 2023 20:53:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D19D78D0005; Mon, 25 Sep 2023 20:53:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BE1FA8D005E; Mon, 25 Sep 2023 20:53:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id B01178D0005 for ; Mon, 25 Sep 2023 20:53:21 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 83FEB40D38 for ; Tue, 26 Sep 2023 00:53:21 +0000 (UTC) X-FDA: 81276925002.27.7590313 Received: from szxga03-in.huawei.com (szxga03-in.huawei.com [45.249.212.189]) by imf15.hostedemail.com (Postfix) with ESMTP id 26891A0036 for ; Tue, 26 Sep 2023 00:53:17 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf15.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.189 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1695689599; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0mBqhTUBjVNOhn3gad4QEbyJ4wj4oLKXxijp0qqMZg4=; b=PhJ67D6A+iIEJDuq1qLTacUUIdhGHEVwtnteLQUd6AhOOuc384wcLSKrf4zl4GGZ85Ld+q FD3vry/VqX7GQn+F6hpy8RJkIF+ir67EGb5nFhjxRcJ1Fj+YkiKVXjuCxczKJ14BY0By2b CC0aYA6p//IHS7ht2UMhK8QxPMibfNQ= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf15.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.189 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1695689599; a=rsa-sha256; cv=none; b=AIXM3NNF109+vg0GldE7NDQei5IJpm+ih7n94VA2bUmLN5aHpLDHiRyFV041oojP8mrrQN dGe740z+DYON5+FSK4wrJkFaWjct9BYhehIfZwfIpNU7OqG8idmIJuDlfDVz363KK/TTCq HfU4h+5Usk0Jf5i8/pZ5J3vdZL6nx0M= Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.53]) by szxga03-in.huawei.com (SkyGuard) with ESMTP id 4Rvh2z45FGzMlpj; Tue, 26 Sep 2023 08:49:31 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Tue, 26 Sep 2023 08:53:12 +0800 From: Kefeng Wang To: Andrew Morton CC: Mike Rapoport , Matthew Wilcox , David Hildenbrand , , , , Zi Yan , Kefeng Wang Subject: [PATCH -next 9/9] mm: convert page_cpupid_reset_last() to folio_cpupid_reset_last() Date: Tue, 26 Sep 2023 08:52:54 +0800 Message-ID: <20230926005254.2861577-10-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20230926005254.2861577-1-wangkefeng.wang@huawei.com> References: <20230926005254.2861577-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 26891A0036 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: 8c7eukpihgt71da4jofsmq7pm5qsonkz X-HE-Tag: 1695689597-618698 X-HE-Meta: U2FsdGVkX19KkCCnubHgFjsePcIjCRhvCsxNODtFjc8dtDefNDRsqpoMoVY2qMwR9iS2eNCVV/furZvJdr2JMAG5zR5ybj6EHHS+vBtBeS4eY9FppgUrHFXalBTVFVwpdv+MW/xCALxxRxwaekcEkGttpMzWG9vdNiPHnmg+Lk637EIwKwgbIylFTfImeKFw5dVs5V69z/NNxxcZFgLn4xMwHhpuz+7TKD4pgi53cIPh6aSsfTNtNIsMk+YoWb0ig8nOuNJRyJAX++MG84xMrg0A73xHvGyp00EBBGyjN9ZmYPUn+Q+tHh7xn4QqSfckwHeznYzTjqCYR2VyKeqmgTeKhwkMw4TuJhPhRNAFA0bgd3OXnpyJhuZNOSSzqnL851+wwJJzzBgFUQExBoXutbXx7H2EBMaRQifVG8h7YKSlY1lQw3f95n6bYQ/LuAxaDCMqkDoxVteh3M4zfYRUhfPKnv8ZxrtguJAd6TKdtee0f6gm0RTvRyXGI1SRAak09K1saDabna53rVxBs8wcww3MWzbdxf60SZYc0gLAQ3YFWIxy8xf/6MbUvzHqyS1s0ZvcUcr2dMXwKtM91ez8cLXYc5+FYZrzyNdfwfmyJA3nFzWdtssqM7aPE/iBAstTwAkvRV8Z6AjPPYMaNLbbpDnnuuI+8tXkz2sCTCCZQpfZNq8JfGqJCT0ERFvNg5yzSW4RR+7WI4EBPEyK3jIZEfjiJ75VWP+nn2fxiLbplcXklXz8CfD0wjA/ehP/N0iOPRSNrzkchKXrZ17o8KEQdl62HM2bssv7pSDHCKzIy4KzDWESh4/rGYsfjJz/xkKFcslYTUVBus0bp3Xa0x8XAR6MnuNIihynn7N+e38jCC3EUAm5P1coPsSZ9xInxLUSHiwLNxWXY68XgT1sRubxCd9+ITn0OjoJp/xp+ETWzlFf+i758ADPz/l5jW7vbR1iKQPGbiXvyjBr1/eBtau v2IYAb2S N7MKpMO7Fq/oeuxFCrxd4+UvU9uRadibAwb4kxLcjEjrZETRU+EOBqEFtsb2yqzDf5mm9t45HMtTaGDzjgr+Vre3NGII6N/U6Pi1AkswmPUeQ0gYtTRYdDfEAgpYP1k/bObTv5AbbTKrgokYvY8ybEIDw1uZZmkihRnDw/OHyt4GegdEaXzOFgT8zn2BMlpHJNP2r X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: It isn't need to fill the default cpupid value for all the struct page, since cpupid is only used for numa balancing, and the pages for numa balancing are all from buddy, page_cpupid_reset_last() is already called by free_pages_prepare() to initialize it, so let's drop the page_cpupid_reset_last() in __init_single_page(), then make page_cpupid_reset_last() to take a folio and rename it to folio_cpupid_reset_last(). Signed-off-by: Kefeng Wang --- include/linux/mm.h | 10 +++++----- mm/mm_init.c | 1 - mm/page_alloc.c | 2 +- 3 files changed, 6 insertions(+), 7 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index a6f4b55bf469..ca66a05eb2ed 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1692,9 +1692,9 @@ static inline int folio_cpupid_last(struct folio *folio) { return folio->_last_cpupid; } -static inline void page_cpupid_reset_last(struct page *page) +static inline void folio_cpupid_reset_last(struct folio *folio) { - page->_last_cpupid = -1 & LAST_CPUPID_MASK; + folio->_last_cpupid = -1 & LAST_CPUPID_MASK; } #else static inline int folio_cpupid_last(struct folio *folio) @@ -1704,9 +1704,9 @@ static inline int folio_cpupid_last(struct folio *folio) extern int folio_cpupid_xchg_last(struct folio *folio, int cpupid); -static inline void page_cpupid_reset_last(struct page *page) +static inline void folio_cpupid_reset_last(struct folio *folio) { - page->flags |= LAST_CPUPID_MASK << LAST_CPUPID_PGSHIFT; + folio->flags |= LAST_CPUPID_MASK << LAST_CPUPID_PGSHIFT; } #endif /* LAST_CPUPID_NOT_IN_PAGE_FLAGS */ @@ -1769,7 +1769,7 @@ static inline bool cpupid_pid_unset(int cpupid) return true; } -static inline void page_cpupid_reset_last(struct page *page) +static inline void folio_cpupid_reset_last(struct folio *folio) { } diff --git a/mm/mm_init.c b/mm/mm_init.c index 06a72c223bce..74c0dc27fbf1 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -563,7 +563,6 @@ void __meminit __init_single_page(struct page *page, unsigned long pfn, set_page_links(page, zone, nid, pfn); init_page_count(page); page_mapcount_reset(page); - page_cpupid_reset_last(page); page_kasan_tag_reset(page); INIT_LIST_HEAD(&page->lru); diff --git a/mm/page_alloc.c b/mm/page_alloc.c index a888b9d57751..852fc78ddb34 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1126,7 +1126,7 @@ static __always_inline bool free_pages_prepare(struct page *page, return false; } - page_cpupid_reset_last(page); + folio_cpupid_reset_last(folio); page->flags &= ~PAGE_FLAGS_CHECK_AT_PREP; reset_page_owner(page, order); page_table_check_free(page, order);