From patchwork Wed Oct 18 14:07:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13427156 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 91934C46CA1 for ; Wed, 18 Oct 2023 14:09:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A455E80071; Wed, 18 Oct 2023 10:09:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9F8108006D; Wed, 18 Oct 2023 10:09:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 848B180071; Wed, 18 Oct 2023 10:09:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 7594F8006D for ; Wed, 18 Oct 2023 10:09:39 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 3D97B1601FB for ; Wed, 18 Oct 2023 14:09:39 +0000 (UTC) X-FDA: 81358765278.06.B50A8B1 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by imf09.hostedemail.com (Postfix) with ESMTP id B931F14000C for ; Wed, 18 Oct 2023 14:09:36 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=none; spf=pass (imf09.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1697638177; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=42AZB0UhRDkADivyLQI4LDJLes6XAOA7/Y7eZodG6fA=; b=ivltJRNOSTOZYawFxJdAVykekoQ81jzv+SJda/g940uU5sforA3FdlG0+U197BrMzv7r/4 WlaKsv2I9A0h30wxx8fSAkaGgFgwCR5igr8yfXVnY4LvwVdTGill2UfT8Jr11V8NW+/QPJ +Vrhh+Q2SlQkU97JnmdNrjVxUAUpIK0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1697638177; a=rsa-sha256; cv=none; b=Qgw7YmHfFrsldJyaZOvVJv00pFeLJ7VeC6p9fPGhL2Ys03RdXtfg834JU3bsVWJ0QneR9M 73Hm7iN+HinGmzN2+525/7ebcq9kmaX1MrWMQKkajNsTmUuLp+Lftekh5o4V7Ay/RAl1qv qchNr8SBRa40NoCctvIwcg4X532I8ZE= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=none; spf=pass (imf09.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.55]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4S9Xh81z84zrTMP; Wed, 18 Oct 2023 22:06:16 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Wed, 18 Oct 2023 22:08:59 +0800 From: Kefeng Wang To: Andrew Morton CC: , , , , , Zi Yan , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Kefeng Wang Subject: [PATCH v3 01/19] mm_types: add virtual and _last_cpupid into struct folio Date: Wed, 18 Oct 2023 22:07:48 +0800 Message-ID: <20231018140806.2783514-2-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231018140806.2783514-1-wangkefeng.wang@huawei.com> References: <20231018140806.2783514-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: B931F14000C X-Rspam-User: X-Stat-Signature: c969u366zdynauymk951m8ng8xmuzd3y X-Rspamd-Server: rspam03 X-HE-Tag: 1697638176-710609 X-HE-Meta: U2FsdGVkX1/RtL9KWz/AKl+Ir3dNBC0eqM2XoCfY/ZGjd/7e1N7it9W0jHj1aeRTpdjyeoi1DyyT4OJ56OX4cr9Zv87uqeBc7Ie8uk2YlwgVOCLVwclK35k8rjwSoh76UuP8/YR0b5q/ME756/0MkAaLbexTbuKzvLJDYxMF4jagz9ZDnSVWTL7nFNUpLmXIB5N7y20ofrlm63R8A39SoqNKPws+pmKdvrM8k9odOSADrBe3FeWswd9bYqr1RnOItNpg/K3H9Yl/6VuDNYvLtMwjxC5dFdIG2jWISZDr64xHtIU4trWD/Twp3IWQjbV2frNKJFSgToq5KeDplV7q07iCocqYdQ2HnIuxm22GWQAc+q7hCGcXf6w9XNcO8vWQRZabVGT1/d/9YbPAKcxFqg9qhtDXs1beCv1yw1pHQ/MmuwdVKtElbSRsOlX08sefl4zr87fxYLNrg45l3BNtpaK4ige6a7Y8ncw0BkydXfaequgLBqgCqQuw1UefSU0GPOZMcUFICr8COiiCQt/uTHGM1Q3x5vTva05+5Dkd2DngFJvv5o6BboWY1acENmKWR85ypI8Y/aVefhXUXkUjp5CCDowdcXD0MQ0Y3bzzHUEbEgt5pRI2r9AaVAQpLv4AbiGY7fMriugdnsf4yS94EtboIU0Lcy8QVEmGqty7A3v7JXHzPjrH+PFjQI80+qPZY8ofcBLY5HimwJYiE5k3dw0Y1T/WlGyy4CHk9YIWWMDKguqsfhd6tGYInego3TTeVpUdZ2hMvES9+Ck16TfKYuPZzcnFCunL4/uFweo95v0VKrtow8VVMlqFyf2Ghmk9MQkU6swXfdV34EU70zC0FbJ1cwYN3a+t6nlnQtrEgzHThrCIsXK8J1k45xZRXYtYXQBF/aPJ5iKnoX/PUa8wvpDxLZdYLkkRPFyRUqkQ6dT8iP9DxStVG1MFV50JkYiAjNtgYyNpPx5Hgq/lvMQ 5a3VEkAq Geu5sw1t+1DfYWP3KnxIRPa/qT+rZfatS4jcuoY4GQP0ipxa/+bVez8AhkEujp84MJnG7/WTPTMQ5PtrXCeyvfxdw1W7yN6MDRD3Z2gNS4xUOqq2ibVOPsugmtOiB6Q2LfJvcJTXu69OVW8gvFX6eWqs3goG0Fi1LwI9XuuuQHyvfdQLcuaNdE9pLN1oeRmTL4dzFbH2K27fFQfS7uxCBARlZmqaz3dQ0KSMoaGX+YXpjbAEm/qFgcQaiIGbVilf0nQ4aj7f1RyjXALh/JQqHLNuv5ZNykS+xd7K7 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: If WANT_PAGE_VIRTUAL and LAST_CPUPID_NOT_IN_PAGE_FLAGS defined, the 'virtual' and '_last_cpupid' are in struct page, and since _last_cpupid is used by numa balancing feature, it is better to move it before KMSAN metadata from struct page, also add them into struct folio to make us to access them from folio directly. Signed-off-by: Kefeng Wang --- include/linux/mm_types.h | 22 ++++++++++++++++++---- 1 file changed, 18 insertions(+), 4 deletions(-) diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index cc8bb767c003..34466be945a9 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -199,6 +199,10 @@ struct page { not kmapped, ie. highmem) */ #endif /* WANT_PAGE_VIRTUAL */ +#ifdef LAST_CPUPID_NOT_IN_PAGE_FLAGS + int _last_cpupid; +#endif + #ifdef CONFIG_KMSAN /* * KMSAN metadata for this page: @@ -210,10 +214,6 @@ struct page { struct page *kmsan_shadow; struct page *kmsan_origin; #endif - -#ifdef LAST_CPUPID_NOT_IN_PAGE_FLAGS - int _last_cpupid; -#endif } _struct_page_alignment; /* @@ -272,6 +272,8 @@ typedef struct { * @_refcount: Do not access this member directly. Use folio_ref_count() * to find how many references there are to this folio. * @memcg_data: Memory Control Group data. + * @virtual: Virtual address in the kernel direct map. + * @_last_cpupid: IDs of last CPU and last process that accessed the folio. * @_entire_mapcount: Do not use directly, call folio_entire_mapcount(). * @_nr_pages_mapped: Do not use directly, call folio_mapcount(). * @_pincount: Do not use directly, call folio_maybe_dma_pinned(). @@ -317,6 +319,12 @@ struct folio { atomic_t _refcount; #ifdef CONFIG_MEMCG unsigned long memcg_data; +#endif +#if defined(WANT_PAGE_VIRTUAL) + void *virtual; +#endif +#ifdef LAST_CPUPID_NOT_IN_PAGE_FLAGS + int _last_cpupid; #endif /* private: the union with struct page is transitional */ }; @@ -373,6 +381,12 @@ FOLIO_MATCH(_refcount, _refcount); #ifdef CONFIG_MEMCG FOLIO_MATCH(memcg_data, memcg_data); #endif +#if defined(WANT_PAGE_VIRTUAL) +FOLIO_MATCH(virtual, virtual); +#endif +#ifdef LAST_CPUPID_NOT_IN_PAGE_FLAGS +FOLIO_MATCH(_last_cpupid, _last_cpupid); +#endif #undef FOLIO_MATCH #define FOLIO_MATCH(pg, fl) \ static_assert(offsetof(struct folio, fl) == \ From patchwork Wed Oct 18 14:07:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13427125 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 877FDCDB482 for ; Wed, 18 Oct 2023 14:09:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 16CCB6B01A4; Wed, 18 Oct 2023 10:09:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0F5A88D015C; Wed, 18 Oct 2023 10:09:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EB4838D0016; Wed, 18 Oct 2023 10:09:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id D2EBF6B01A4 for ; Wed, 18 Oct 2023 10:09:08 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 95ABE1A01FD for ; Wed, 18 Oct 2023 14:09:08 +0000 (UTC) X-FDA: 81358763976.07.0BB278B Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by imf20.hostedemail.com (Postfix) with ESMTP id A32341C0003 for ; Wed, 18 Oct 2023 14:09:05 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=none; spf=pass (imf20.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1697638146; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=z3XKsRcFGkwnebwYgONpauQeYaDIMFmQ038Pa3oEk3k=; b=4JlmqRaAlOMIWF/q/+okV4iTcGNtwSwGlV3SQjfpXWljdLC1FuwONCnaD0s42Pff3WGaE1 AhgErHIaNUr8cGAZdHBnVoLUnQT/GHlIeT+xOLbLpMAmTwABUDRndSrw9wLOh8dBlXD361 pxZbaMF8uqQBTDl4u1U1HHjvi+BHNsc= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1697638146; a=rsa-sha256; cv=none; b=d0q0EoW/xpB2Gft9enxhjuZDrBJtMdNJaJTnNgrDyyZwuWzbJk5RCMPg6Ni+1wQpvrfPyv CanCXCe742Cxxcpc06YAy9L0Qk6WFKlgkJ4/4pyuvfRibHLzym/2wJbKkO4kQMqqDkoIPF H0of0l3xJXbsEic9CRMpJedEh6FhM6A= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=none; spf=pass (imf20.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.54]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4S9Xfj0G7qzCrRT; Wed, 18 Oct 2023 22:05:01 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Wed, 18 Oct 2023 22:09:00 +0800 From: Kefeng Wang To: Andrew Morton CC: , , , , , Zi Yan , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Kefeng Wang Subject: [PATCH v3 02/19] mm: add folio_last_cpupid() Date: Wed, 18 Oct 2023 22:07:49 +0800 Message-ID: <20231018140806.2783514-3-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231018140806.2783514-1-wangkefeng.wang@huawei.com> References: <20231018140806.2783514-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Stat-Signature: ws8og9b7p6tseknr1ce9ix6pf9wuj8fz X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: A32341C0003 X-Rspam-User: X-HE-Tag: 1697638145-521274 X-HE-Meta: U2FsdGVkX18pga3D3bqvjU2+9DL8roosvHXR5gaHLpHQrkR4VItSKFcHSi+NRWhw2PIBD5oc+GzfOAV3mrKu5Rgar//5S2BHIaOQVWT8OYwib7vk0oS9+lhK8O7KVClA/Pg0nBmIO6JGMc3u+/qdAFTG8P7SxLvLQADK3Y8bFKGa0SpwLU4Zan7xM5hJkjKS1YqUY/wXYfO4PK0+zzHfWTpDQGnkFTQ6Z5/Rb6Mx01aikALRol/pRRYrNdjUfycbCuotlbGXqooxsxs8y8gJVhsE+O1am1jsptZYZaH7mwO7paRE5piiB6v/88xwHMxBjl6NXVloWA84rGNGuOH1xzTnOubVMjtRdEPcR+x6uRu642uzOJokYE/7D1gFwdx8n7AGqEhzmauvyFmg6qX6WyLqw3VrqmLvi6jK7cBWpCObzVIB4+dfXQuqAMjan06+r5fDMthsLvZSUKoBjNmGqJPrprjkTQfvXm673wnhwsOL1J8j8GYE21BONwsNmhY7SIb0Tnx/6B6is66M1j3lro1lns17cfhsPw3d8768JO/RJsVihLdS2KbgkQnxyHIOONLTH0jr/bv7cvs1Zo+zfgwfOrLpln+1U0rbnw4JYWkX9ud5EEDC9xFVtE/vuz4Jo5pJXWsnPM+VE3ZQx4D5TxieDT27sA0x7157oGWVaCCjiyW6R5bwfunWNnW7QiNDA9y3X5V8wWMtgbZaImDYQkk/9ktHvX6b9hjEmT5Qfso6yQY8xFTF0BetVnZAfiB5maaXYGGxGdKUxJHW7dlYlpmNyFJmCdFmK4sJyb70qwRf4hSUvPHzBdp6jTlEkjuaIB74WWjFbmuuOhCK74gquMpH0aIwXjJFIrTsoPWRW18coEdsUJerCPJWxu9qE8T0RXoauGtRUy9/jXgYf/sWzQ9G5SWlYyhsenD5h+63Qi0l0qHNL7XYaSfI7pwgF7CGLm/xwIsYrQfooZbEgPG QyVtyTvZ AamKlkV8VqWVm9UvIU6cMSBMm66jIaUKpWJCe3QvBtra0pXh1LxkKJUBBhZMAkxty2EeExI/8+/ONFqoSJm0CxApCfas9Hb5e60evZvGDydeXJT3xCe3gZzaM/asUsj/xxy303W+SU7eKY8FnwsZQtNFPqNObS5EL7k3nYlqrCsgH1M8z6dRgEg/3wgs+zNjp32BB X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add folio_last_cpupid() wrapper, which is required to convert page_cpupid_last() to folio vertion later in the series. Signed-off-by: Kefeng Wang --- include/linux/mm.h | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index d005beccaa5d..1c393a72037b 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1794,6 +1794,11 @@ static inline void vma_set_access_pid_bit(struct vm_area_struct *vma) } #endif /* CONFIG_NUMA_BALANCING */ +static inline int folio_last_cpupid(struct folio *folio) +{ + return page_cpupid_last(&folio->page); +} + #if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS) /* From patchwork Wed Oct 18 14:07:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13427134 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1A659CDB47E for ; Wed, 18 Oct 2023 14:09:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4AAE76B01A9; Wed, 18 Oct 2023 10:09:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2A40F80017; Wed, 18 Oct 2023 10:09:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E62E66B01A9; Wed, 18 Oct 2023 10:09:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id C215D80016 for ; Wed, 18 Oct 2023 10:09:28 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 8350740150 for ; Wed, 18 Oct 2023 14:09:28 +0000 (UTC) X-FDA: 81358764816.03.77A6C2C Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by imf24.hostedemail.com (Postfix) with ESMTP id C56AF18001E for ; Wed, 18 Oct 2023 14:09:24 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf24.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1697638166; a=rsa-sha256; cv=none; b=sjyfYO8M6f1l5a1xqxd13olEdqyTB4fZd53DfY+HfMODhc39n7opxXZkmpbT7ePIXVtcoh avdbfNTdQrzdm+8HC5zqfDCTAxbNkg6nKQK6mCVeLdxjase1BGQE0JeNw4ZFQYadEhNZYY KMn4R0JDtsxtmMZXVdjMJK/V0zJKcVw= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf24.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1697638166; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=y9mdiRTkF16A23MSwCD/zcfcV62lDhs7zm5d0Ve5GvQ=; b=HacjNipMVXh84vQVlFu3Gpqdl5fqwTu6/kRB1jIpli1JdaA0bv+UV2Og1fuZCCzKLZQ3CT kV3KsrX+0VwEUCIuvCa2y40WwusmH67j9G9kIaPZRcl9/rAuqwqQGLsL7vr9Ejo6vn/yoK YgxJnOF9pI9Uf1GC2CGgii/W9TlvqnQ= Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.56]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4S9Xh93L0qzrTNS; Wed, 18 Oct 2023 22:06:17 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Wed, 18 Oct 2023 22:09:00 +0800 From: Kefeng Wang To: Andrew Morton CC: , , , , , Zi Yan , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Kefeng Wang Subject: [PATCH v3 03/19] mm: memory: use folio_last_cpupid() in do_numa_page() Date: Wed, 18 Oct 2023 22:07:50 +0800 Message-ID: <20231018140806.2783514-4-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231018140806.2783514-1-wangkefeng.wang@huawei.com> References: <20231018140806.2783514-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: C56AF18001E X-Stat-Signature: xaryqe1oihtr3yw4qt7d9d16uia39utc X-Rspam-User: X-HE-Tag: 1697638164-183923 X-HE-Meta: U2FsdGVkX1+9nFXPGwtnHbEne3HhGys7I1FSnufOolRwi8+r4tsI9tvWnird+eVRvaBr9fluEDZS0MJg0RfWolt6eOUYuitrmStsknnIujXhpwiqzXIelG0Am+KQRAqEyiSvQB20aI3kzhKq+WGqgz3PTrhUEpnqXd/eDqX0utTFLwAH8+eJO8d2CElhAb8yzn588gMGQTSfBe8Ag9Cx/MLK3h6woL52Ybbv52PB7zhauL3t4GLMGIAQLDM49E2md4Cu/7xGcyjG4cDThejKy4Nr8mdqzxWc+LwBNsoQIaGSKjOmPqce6YXKe5HZ0NWLrPjtTlbRWmr3utN14Dtjq5M2hfldKEXXonIKyx+WzmKF0WgDo5VaI81sARYHUxeiGCZ+xAmTpli02FO7qDAnV1gsfapYxB++1RuC2dG0SoBgm+K4e6yZXM0CmeeCCBCUz1OKGGk6O6436vzq+GTyaZ/mHX3FPXncXxZ+Go/ILZ0IHtWbWjJD7g7WHAr4fs6sj+pNa3GUrCutbn7688Myd3wQZm+wWKpEdcEAOf9N6R9gRf8Xstv6pzIFGsVyHaywVJbzrr8v+Nj8KNS8lXFd8i3h8krPRhYoW/sw0LzRX6m3LULFM31HF5CB07tFSh9vM9iafCF3w4NHJFtV56Wil2dxzgdDyhRI+UsUUY6bBk3oQy8z/Q4NXJzPmu01MxL+WUEW0a14xMByPT1rvR+Nr1Qaqg3a0Z7IMp+W3rFtiQCyQ4BRaf7wM1ewvaLzqz9BDYYrkhkYMFVvJZBHxyqcNJbVhvy2cKebY1h7T4auJHBzrCZ78zHwsXPuvux/FH0dzMrUiSXHXhYZBwR1XZS4e5AZWgRWzpjoQyHMigJpGAGGN//be32kyEHYCIgdDs42GGn95qtvdmJSlwdnUPnUx61sVjfu50t+p4ruNF5Ip1KkDTnUqLpNgnwpc5/MUTgf8tacvXOReEpHd3Vxwri GemGB4Xq xL8Uh5fZj94yo4GMpaYE8N/Cis0WxgpZMcBjJS8p7uipabVQOM0VMSEP3K5MCTkjfaYKLpm0ZkjF5WBWamwLH0sBop78yjvC+whoga2nC4p64OYp6gNlkpitrAeaqXbLcovST0oaQy/EtE9725+ghRHzqq3YJg1sijEeiWrPTbD37fU0HFK4nLvxRPU2FiCj8S8FhHPx2pjedVNXvkPa/ZrqtM4eKDU808bKTOgtKhgMcjiIcxU79JOSY7sdgKzSGD73I8DS/O7bS6E13uSLwb4ct3ebRra4T/+d3 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Convert to use folio_last_cpupid() in do_numa_page(). Signed-off-by: Kefeng Wang --- mm/memory.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/memory.c b/mm/memory.c index c4b4aa4c1180..a1cf25a3ff16 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4861,7 +4861,7 @@ static vm_fault_t do_numa_page(struct vm_fault *vmf) !node_is_toptier(nid)) last_cpupid = (-1 & LAST_CPUPID_MASK); else - last_cpupid = page_cpupid_last(&folio->page); + last_cpupid = folio_last_cpupid(folio); target_nid = numa_migrate_prep(folio, vma, vmf->address, nid, &flags); if (target_nid == NUMA_NO_NODE) { folio_put(folio); From patchwork Wed Oct 18 14:07:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13427155 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D77BECDB47E for ; Wed, 18 Oct 2023 14:09:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0F7406B0205; Wed, 18 Oct 2023 10:09:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0A6606B0206; Wed, 18 Oct 2023 10:09:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D78908006D; Wed, 18 Oct 2023 10:09:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id BC3D16B0205 for ; Wed, 18 Oct 2023 10:09:37 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 915E7C01AE for ; Wed, 18 Oct 2023 14:09:37 +0000 (UTC) X-FDA: 81358765194.26.242FAAC Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by imf06.hostedemail.com (Postfix) with ESMTP id 20872180021 for ; Wed, 18 Oct 2023 14:09:34 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=none; spf=pass (imf06.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1697638175; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CXnUonKq8y+vcUdw1/au85alxquWMu9WZTGSC9Nd5bU=; b=UYcONFNgTgZAuOfnwYugwmZt0la3op0tfp8cH9nCtWK9vkyDvK/WVpXA2xJuFoJ63vFtQ0 +4DSEduaDYlSmjpLkq6ndERTotvD7l37G2pEeka1kfrNJbzoUfDiSsV+9fc5W1u325es5S 9/7y4jznayZWU5ciwTgx/F9sI6byKYQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1697638175; a=rsa-sha256; cv=none; b=QfNXFG4aroHHn0RXd613nnzWMECdOdJeFBwMHaukbLmu0+g/tw76/FVXIyX5lM8Rms6lVS SWW9w1Yz6v7+nJNantFs9twDRMu2OeSw1eL3r0RH0G8q8oGNsNJQj1J9aOf76woDS5Utir SKImcA7wqfx19yFxMKs2pGt9l7X6dVE= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=none; spf=pass (imf06.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.57]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4S9XhB0sD5zrTNh; Wed, 18 Oct 2023 22:06:18 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Wed, 18 Oct 2023 22:09:01 +0800 From: Kefeng Wang To: Andrew Morton CC: , , , , , Zi Yan , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Kefeng Wang Subject: [PATCH v3 04/19] mm: huge_memory: use folio_last_cpupid() in do_huge_pmd_numa_page() Date: Wed, 18 Oct 2023 22:07:51 +0800 Message-ID: <20231018140806.2783514-5-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231018140806.2783514-1-wangkefeng.wang@huawei.com> References: <20231018140806.2783514-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Stat-Signature: ipetwcxsyksudsbxe33f1kpe8yakh3hu X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 20872180021 X-Rspam-User: X-HE-Tag: 1697638174-97827 X-HE-Meta: U2FsdGVkX18aQLsXF/ub8CpXwM9uHBBygamhhkfj+5ksa6IoedbCVvLlaLpFhahl5VF9LHIM7IJDYNAwcPLWIDu1izqVZLxieQaVIhcknmBuoxOp5yzWE3zcRuKkObeK8U8Cs7Jq646+pdh2DELPBX1/kRclTdWZbnq8p0cuJmm/3/beU20Voax5VUvTpyCx4z+vcxR0dCAuiQH45ANPe73790ZXQGOvlXWloyJ42gpMF/REUL3S8pr7bJiHsXzG4WmdSs6ltp4P8TSiyh7GrtsbozAieJ3zHRb08qNvnK6hCDZn1zCzDfnOi+856Kr5N6euMOg4oQpnNiKsqWbGCFk2l5lYrn57KmNnS2FUQfqGhOfZqOl07+aev2u1acuJDzNoB0A4iYef0RyWPOvGd4R9VgyFy0gqDuau4ymZHPOpyZVDkjECZzccx6O24KdHPJQ1JiR9thVxesSA/uoUaJi8SqQA66j321a5FLOJHhA+UpYZB6juPUggwGPMXevRHDgXi9SFBszV0XkELPM9s38D2PwNrOswM4C8jJQhTofGJNGTft6ikkX8dMSxQlBqZXeE4elPZMeQE80cajkgM7LQbSrDrV35CFQT4LCcU5f6q1VL5Br7SJHX1BKQTzWIgHJln9BN68+BEgsHaXCEtpjcS8JCSArcpJgO12Dwhdxywn3iPz8O7HpMvDanSLahUMLB7aiXOyMIKV/ZGuQltn/r0TNpq60cKyaoG9eD9Lfzriev10snFkmDGuFYDpw94eWmLiSWpWnYKSC/TO92JpZfzES1CIVTQqfJXVLHM2q2W1NbIbIRahHtCNCyJXxtz+/ix7mtjjdO1R5j8TG3m/F7f3qK2pz3dTOHwWp+T04TADclpdE7n2hivlNNdjIXf1G9KD4bkje0Kg/x3vTuOGjBemvvHs87ix/RKnusubuE0YHTPFDtYQeONHyMxtMqzPg6wkurNALCEwFetX4 GKpQC5pC lBozaY/ecpLZpfd1ovlfhAVTh9GJ/lyHpxP5GOvGTMD2gG994AUKEkoXKKuq/7KBnvcNPj/BZp4O6YXrrlgUcE6AZt2yPaAQ/M4dONN1qRjxgDUe3cq8QeeRJBmKEGX3Z67X9NSFrZOmvPdsDj6ECHExgy5sWnapzbKA5lvJecf8Ywh4P0F1qUUOwBD3pPo+MTnZWee2kPv8IZ1SCT16FW6dMmIG/2CYwingXR+0spC6RtpLlFKcbsrVBbTEiZEM0uDzhc2t1mLAt/y2mvN52N4fWity6oM3yANdNzcCLCuBzPUIiSf5pysKIrw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Convert to use folio_last_cpupid() in do_huge_pmd_numa_page(). Signed-off-by: Kefeng Wang --- mm/huge_memory.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index c9cbcbf6697e..f9571bf92603 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1562,7 +1562,7 @@ vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf) * to record page access time. So use default value. */ if (node_is_toptier(nid)) - last_cpupid = page_cpupid_last(&folio->page); + last_cpupid = folio_last_cpupid(folio); target_nid = numa_migrate_prep(folio, vma, haddr, nid, &flags); if (target_nid == NUMA_NO_NODE) { folio_put(folio); From patchwork Wed Oct 18 14:07:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13427157 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F07A4C41513 for ; Wed, 18 Oct 2023 14:09:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E8B5A80072; Wed, 18 Oct 2023 10:09:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E14698006D; Wed, 18 Oct 2023 10:09:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D055E80072; Wed, 18 Oct 2023 10:09:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id BCBED8006D for ; Wed, 18 Oct 2023 10:09:42 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 864ACC0280 for ; Wed, 18 Oct 2023 14:09:42 +0000 (UTC) X-FDA: 81358765404.15.A74043D Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by imf11.hostedemail.com (Postfix) with ESMTP id B614C40024 for ; Wed, 18 Oct 2023 14:09:38 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=none; spf=pass (imf11.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1697638180; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0YHxzJua+p5mlcs/k2mq03WYuq2lzIWkVKq0L7/b+fw=; b=kGm8L8QV36jHAGvYp7Ccn5N5IPSNkzHXygg/ck2g1TUOoGQXY9K0qv0kNXIO4NV/uN2GUk aJJRlsoTNIuiH6A4IuxEjnC0KucZNWyZ8gvV2PLXPAhSXL+l9gTgBpEUsKioDMRNEgX5JA uGyqey3hfywfNKLdDLFk6K3m9Ih4mKs= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1697638180; a=rsa-sha256; cv=none; b=rA2dhDfKO7/0m73+7KFQ0r6hQw/blOA2kNq7KTRFw4hEIZcVySuqtYCSmcpz7jr6FozQ0b VOfs+lslUNNmT+9ORp8M8Z8BrvCXdVxtHWuksZQI5v+gtBPR/QRrwWkf2OBduCBm+EOgJy 2iJWSwMUGXS2c+8J5WWtN3NPcwjIF68= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=none; spf=pass (imf11.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.53]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4S9Xg561TqzRt3s; Wed, 18 Oct 2023 22:05:21 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Wed, 18 Oct 2023 22:09:01 +0800 From: Kefeng Wang To: Andrew Morton CC: , , , , , Zi Yan , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Kefeng Wang Subject: [PATCH v3 05/19] mm: huge_memory: use folio_last_cpupid() in __split_huge_page_tail() Date: Wed, 18 Oct 2023 22:07:52 +0800 Message-ID: <20231018140806.2783514-6-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231018140806.2783514-1-wangkefeng.wang@huawei.com> References: <20231018140806.2783514-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Stat-Signature: s453tuob9gpdqgpqqd9qrmsckx1kcu9f X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: B614C40024 X-Rspam-User: X-HE-Tag: 1697638178-168289 X-HE-Meta: U2FsdGVkX19r5yxTzOJYxmhIreF3FSF0JoePkcEW+iJm6QuIITcfbUtG5OaHVnMhtGhyZmTLdVA7mmvAC1fQD4yw4WQP7bYVTKKqimeeyH1s2K+fMlZAWbg5s0J0Bzv0+i1Sh6p/4qyuTZJO/7mn8zzKMeiI5Zo09M4lvhUkIkvUJgj7ueCdHJ3nEBRRr1se0zNXbimDTJMHxxxm6ouu5yPl0HYzveCRVXCUPV7kSFtIgMv0l8hqyTnTBHST8P5QcEK3w1Q+pZQR0TiDHhviynGBsjFXW9FY/ehGheZqa3k3bImT+Ezsj2fbj4MtoFDrlyfta/2TYxP3L7LcBZhVJWX2SBHx9Fwldg8qoaWgEP/xAbVp3kdqJZUcD32EaGRv+PafCZ6t0/5YUkAnvwMvBjlyJ6TJV1KPH9zvqrp2KzTxVqNX9chJ7yS+nrHDC1IpOdFs5dbFtdmLwAtiiVTnutIimkCjQRdG2dbdhrNO2JReLRGTiLuy060BAh8/e7RY+M4MZwafEyr4Rg8fr+m/oqiCoz88Y7XUJqvd5BG5jGQg0C2sPRCdZrlxg3SG2qtQxuAi3CSzuZO7x+UofchaH1UPvXqRihjWd448IUrfgIQP7jlMYTV1bAXhhEJSUN7OA9eq4ZKd9LVoUGFE3yK/hPCVcFC4h+3BxrkCcp9cJ4KsHTFJ6BSVQ6qgXRu6hfxvBHv80IxBBJ5CqwuCdmxkZxcjd7SOC8WF++rACCKIiOkg6vYAhXA9bTjZGVmhmvdziRhiYO2NLKqwYnUe93+36gI64dxqgpHpxUIrKdoBK997HeBKXz4Vu+TOmMwj7YCCg23Fnr8vy5OnnBLugti3h0TRePez2hukpN6d2qRe6/yxHLfQO2jwUbWMcSdUrSsLsoww8yRl5yHOsyg7mA9LQXrtHASXKKB8S8GpFw76UJx7zPXO/792c9ByWOgQjLGVSsFtUByS707KrXb0VPR Acc1KAiH uvGhPnBIV2JpH6kOhRXsE3e5GrxUtTQmKU61aADjGfL+zBY3G5i5y8VmdtNqRsc3TJoGV148Nhie+EQcPTMEg5kovQaN2liuNxiHRZAKaU4IUNO36f0f/LOA5Rg6echZxsTt8fMV/zWRR7ITHa6sqFcg3L50gmqPXpXjF3VHiLk63+yreRoe82hge96rB/eOq7I0R X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Convert to use folio_last_cpupid() in __split_huge_page_tail(). Signed-off-by: Kefeng Wang --- mm/huge_memory.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index f9571bf92603..5455dfe4c3c7 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2514,7 +2514,7 @@ static void __split_huge_page_tail(struct folio *folio, int tail, if (page_is_idle(head)) set_page_idle(page_tail); - page_cpupid_xchg_last(page_tail, page_cpupid_last(head)); + page_cpupid_xchg_last(page_tail, folio_last_cpupid(folio)); /* * always add to the tail because some iterators expect new From patchwork Wed Oct 18 14:07:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13427132 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 26A7ACDB47E for ; Wed, 18 Oct 2023 14:09:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E65996B01AB; Wed, 18 Oct 2023 10:09:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DF31280018; Wed, 18 Oct 2023 10:09:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BF2336B0203; Wed, 18 Oct 2023 10:09:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 9491F80017 for ; Wed, 18 Oct 2023 10:09:28 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 55E7BA01C4 for ; Wed, 18 Oct 2023 14:09:28 +0000 (UTC) X-FDA: 81358764816.04.F94438A Received: from szxga03-in.huawei.com (szxga03-in.huawei.com [45.249.212.189]) by imf17.hostedemail.com (Postfix) with ESMTP id C75FB4001C for ; Wed, 18 Oct 2023 14:09:25 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf17.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.189 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1697638166; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Ew90GRduGLuwlGQ2gfJPl9ppfNJrebvIabpuV6V4jCE=; b=bhB8qvhveOGgWQVfdmKM2zWzaz3yV2c8G3wVnxpwyL+ngnpHu+RQYturIvM/NF/tRehXdP 72z7r21/GEQH+La0voOc8k1nUblYbAMpNRXOnKSUHlqyZpJXo70o1qIMI4ZHlbHGGfPB9Q qeMjjTlXxng40WEInPTZNuDmGnjYils= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf17.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.189 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1697638166; a=rsa-sha256; cv=none; b=VFg2d4XErH1ab6tOWMmyuZ+4AQn0pdHCj790/S3wU3vTz7fxOAkMif8PuMZZYgTYVM74He anoujHGT4OALxyJkmQA32EsGwhXpEj+i+2IR6CDG/Is22qXENIoJDKvAHc2r37kRjwrLsW 1fuKbngY6NMUeK9NKrvPft2SU3L2+Bk= Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.54]) by szxga03-in.huawei.com (SkyGuard) with ESMTP id 4S9Xfd2h8yzLp6H; Wed, 18 Oct 2023 22:04:57 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Wed, 18 Oct 2023 22:09:02 +0800 From: Kefeng Wang To: Andrew Morton CC: , , , , , Zi Yan , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Kefeng Wang Subject: [PATCH v3 06/19] mm: remove page_cpupid_last() Date: Wed, 18 Oct 2023 22:07:53 +0800 Message-ID: <20231018140806.2783514-7-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231018140806.2783514-1-wangkefeng.wang@huawei.com> References: <20231018140806.2783514-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: C75FB4001C X-Stat-Signature: 7paujftzw4qf1sn4i7odetf9foh3hgna X-HE-Tag: 1697638165-772209 X-HE-Meta: U2FsdGVkX1+S+rx5idA1YnjdVyZVge/feVqdUcZ2Gf4rgPaDHJMvfKlVThBMxLdsjHHpksTljVq2FRWJXkAJ71QI1rLgNzM64Q5hGEZ6k1MJkbdqDFC50dxWkxjW8VcY5b4IdFs9QjebhFrKP0v0nl8pnG6cpvFDqLit6lhXcFFAjAE7EAOQqxlqGKTen8hAH6AtXwoyV4T2RjshCkvpp2bIFKk9Z1btAMrpZwQAqX8I9h9QsZZeqEcur5Lukpf/yJ7ygOB5rPwHH982vDAIXV09XAd+MPFuZnhuzMDgJXz/ad/o1zJHoGswsiRbPLz/O7brD4MCGkUspFgznwnAvlpt3SpIBoXjDdzHqFCbkS4eIw99cC/jY6vQ3iel0gGOJaMOrUFAx+GRMyf8qa9IzCq2YVITMBylu7jivF2HA+e6ETtHpIndMezEm4U8GJzpfm0Hbpa7ASH2/skrAWIzyEhkDw6t3Ub5yyW3zRjclAhSFdQLe3JELDC6GbEJfzTW9tIlARc2r04dNBjdJoKexbQA9EhbWu5XwnQB+2LOYCEvYsCAJi1QiyZbJcYimHGEx/88GMjejLNdNjnpoEu975c7/3IgVo9OUsKdRWSILcMKF9fFRzpJES8ntR1n95DnJeOdv4rjMOLl8LQUenrIAYa/cBFyUMTqUmI/EwXCYzezfz8sSja7+aHdlMydAZk2q5WwzFhHjOiuaATUij4vP/Yp4RP9Ua0g6Uuy4V8lC1oznHhRA1E7yKba8dYZH7+wJi8nR/awQgzgZ/2978yux4QwVUoVirmyATDAF3a56GoItc/dpgrp+MvzoEDoo2riFjwjdtQRRuHeIBiG5RGBsNG96HMbstAibPfVElZJeSp6jJCri/TiDejEyd87gmXRPrVQ3gubQfP8mJrL7NRyT9Mah3g4w8JFD+4KphrGonaNTUd5lBJ9sSaSkcVCkOl6Zp3cgGwiQqI88UVtrwa 4fVvBeRD 8faixJOJkZlT+Ol2gmS0pu00fBBViT9bwVhQeJRRdN/J22sQiJXCp60ZklbpbRRX7khaA79hWH39fZj+Gyp4UhAwr7xYkmIlymKu29k1u/TYo+LSJFW9EXRii9IlwnB3xOmSTuYlQAfyICEPSCUeXEvfc6suxfR1Z/G/iIB4DCEFZcflvjmZE3BZMuVcxpSPg6clV X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Since all calls use folio_last_cpupid(), remove page_cpupid_last(). Signed-off-by: Kefeng Wang --- include/linux/mm.h | 17 ++++++----------- 1 file changed, 6 insertions(+), 11 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 1c393a72037b..1d56a818b212 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1700,18 +1700,18 @@ static inline int page_cpupid_xchg_last(struct page *page, int cpupid) return xchg(&page->_last_cpupid, cpupid & LAST_CPUPID_MASK); } -static inline int page_cpupid_last(struct page *page) +static inline int folio_last_cpupid(struct folio *folio) { - return page->_last_cpupid; + return folio->_last_cpupid; } static inline void page_cpupid_reset_last(struct page *page) { page->_last_cpupid = -1 & LAST_CPUPID_MASK; } #else -static inline int page_cpupid_last(struct page *page) +static inline int folio_last_cpupid(struct folio *folio) { - return (page->flags >> LAST_CPUPID_PGSHIFT) & LAST_CPUPID_MASK; + return (folio->flags >> LAST_CPUPID_PGSHIFT) & LAST_CPUPID_MASK; } extern int page_cpupid_xchg_last(struct page *page, int cpupid); @@ -1750,9 +1750,9 @@ static inline int xchg_page_access_time(struct page *page, int time) return 0; } -static inline int page_cpupid_last(struct page *page) +static inline int folio_last_cpupid(struct folio *folio) { - return page_to_nid(page); /* XXX */ + return folio_nid(folio); /* XXX */ } static inline int cpupid_to_nid(int cpupid) @@ -1794,11 +1794,6 @@ static inline void vma_set_access_pid_bit(struct vm_area_struct *vma) } #endif /* CONFIG_NUMA_BALANCING */ -static inline int folio_last_cpupid(struct folio *folio) -{ - return page_cpupid_last(&folio->page); -} - #if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS) /* From patchwork Wed Oct 18 14:07:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13427126 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DE77BCDB482 for ; Wed, 18 Oct 2023 14:09:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 69C1D8D0016; Wed, 18 Oct 2023 10:09:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 64C9D6B01A7; Wed, 18 Oct 2023 10:09:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 513F08D0016; Wed, 18 Oct 2023 10:09:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 409606B01A6 for ; Wed, 18 Oct 2023 10:09:12 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 14333B5F3F for ; Wed, 18 Oct 2023 14:09:12 +0000 (UTC) X-FDA: 81358764144.19.B10D980 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by imf08.hostedemail.com (Postfix) with ESMTP id C68DC160035 for ; Wed, 18 Oct 2023 14:09:07 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=none; spf=pass (imf08.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1697638149; a=rsa-sha256; cv=none; b=uATRx6nAJ8b+Yv6ZmGl8V+ZFkjW4eqmNU6J9bWntixR0pMIY13RBOBgqOR+8UGuxwpR2iv 0kPDqrekF/WKtmiPxnQ5FdZSkUIV/JF8MyxdFEJE8FVgPJJPGJoXd6AEsgRVdev6tOifgH M7x57I3IyiSJ4CbItup3cjpHfqQm9hk= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=none; spf=pass (imf08.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1697638149; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MKSp5EBF6+Ov7gSZHW50N+nAf20gVCjFGTQvZhuqW6o=; b=fzIh1QJ22cZ8bPAivoJxT5pziyOeqimVGmU46sBkwWQ3oGCPk23hZAC4ctbU0oWCiZK2ze yQUDR+S0QnBdpnDX56pDZBj+5Jzz5Et+NLnR9ZQFc0twWOL9xe1vmTgjQmal4oy5AyVTGz eWB0viOK9S5/MgdKLBxgj7Uudu104Ck= Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.55]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4S9Xdt5dLmzvQ8j; Wed, 18 Oct 2023 22:04:18 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Wed, 18 Oct 2023 22:09:03 +0800 From: Kefeng Wang To: Andrew Morton CC: , , , , , Zi Yan , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Kefeng Wang Subject: [PATCH v3 07/19] mm: add folio_xchg_access_time() Date: Wed, 18 Oct 2023 22:07:54 +0800 Message-ID: <20231018140806.2783514-8-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231018140806.2783514-1-wangkefeng.wang@huawei.com> References: <20231018140806.2783514-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: C68DC160035 X-Stat-Signature: rixmr18ff3ekmkpy878mf931i1g5ijx9 X-HE-Tag: 1697638147-723251 X-HE-Meta: U2FsdGVkX19PPk/vuJzgIwGyz/NeYl2uur0l6Z58d40wX97gq30130wPHyJ4EBvcSnKpyMU8I/qGoaefOu5X1St1HFHhrmJfk7EVAb+M0qlBoaMpnhapALzAAoKBxm1ETpzaTI8Y492J9Xj4xdpiQDXb4IU642QCd++WlCyHcfC6OarPnGEhbCk0g+y7ZGCRwu7nsESLyapUV8s+xZTzybNG0CgoWEQuW0BupkDCAIGc+oc6CY/SynjHxsm5EQDge/KNWtVYNsMLDR0o0Wmf9nggvcMLKjIJIw57c6J6q9/QwV8cEyJNb3lP3569PiqhJpsugl5TTpsKWV2CUDfGutrFtxrbyBmUqTAPOzIlJDQXTzHYcHDthLZciBI82zp/hzihOh5jY6CcDAY/XHILFfOQbxHLEiYKTcPOvG/djbptRILLBN7HtnHFX9pn0WwDfKyFkPSir56UjTlFLv0IMljtq2FMnBNs32JL6PtUskG7hRxi6FQibWM0Yzn5ovi/aa/2w7vMxOE3bXHKU2HbsRIFRHRsLHE2oCt0PVVez224QQvr6EuB8dkDz2yQUVFexF8B8pPZSGArNxwKdQTbSQx3gNhaSh8QHFaggjr2rhHIIjy14WiVGY1IaFp0o/EhgTSHIExrqJhr5auc/tN0MRMZVK7L2ovDImj0YQFNcGwij66ZbzRlPMtmPt1BxrqF43QcqH2T6Y/4j+fV6quFXjc9Ct5N0iumXnRz7VX2Nx5T40pZnEVrnI3HK1ThfQjTYYo7+rCetD6QiShfXHF9BHcHiGc/t1gpftIFE0QRlujjTbGiA1wSOzQI6NhPHKsH9rQJNHAom5UGAyXXjrhmvJfQk0sjkTs6Ty/qhHwxZ2EbczXp7OIYmVoqrJ3aDSgJro9wBsaYNXbrdFK4e8hRC7akcUZvUZuptwP9z2PGf0IFe3rdn6uT7W5gZm7m7TuQmV14rlWUeKH/BIoIhFY mTnCK4GS xbsb6QtB7gYk2l87WJ7TZXfS89s0hFOv2OuoE5Q0gDia6EgItN5bT/2fsv4exl4nWF1cEVa1syMhjufJECMQSlsnRw+1Gtd0YUDfi6gtLlWno7sJQOJ7pKnLJk+dDjuFfj/ItD7ATIsRCpTA2oCYlq3QVJbelAy0cMsIG2EkWxdHJm8MSz6k6Is38jOUef5FEjdBUxqFubiO/k7UFLgDdkNJ2CzilyPeIAFxXjN1NsSaTC11ON+Dt9zdZhM4K7Flh+g1lCR7wPr3TECR1PRvWrBaW2DBFTSVWfLPQ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add folio_xchg_access_time() wrapper, which is required to convert xchg_page_access_time() to folio vertion later in the series. Signed-off-by: Kefeng Wang --- include/linux/mm.h | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index 1d56a818b212..1238ab784d8b 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1794,6 +1794,11 @@ static inline void vma_set_access_pid_bit(struct vm_area_struct *vma) } #endif /* CONFIG_NUMA_BALANCING */ +static inline int folio_xchg_access_time(struct folio *folio, int time) +{ + return xchg_page_access_time(&folio->page, time); +} + #if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS) /* From patchwork Wed Oct 18 14:07:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13427174 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D6EB0CDB47E for ; Wed, 18 Oct 2023 14:28:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5E43E8D015C; Wed, 18 Oct 2023 10:28:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 593FC8D0016; Wed, 18 Oct 2023 10:28:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4837F8D015C; Wed, 18 Oct 2023 10:28:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 36C6C8D0016 for ; Wed, 18 Oct 2023 10:28:33 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id D9821C04EF for ; Wed, 18 Oct 2023 14:28:32 +0000 (UTC) X-FDA: 81358812864.05.E656F8E Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by imf23.hostedemail.com (Postfix) with ESMTP id 7832914000A for ; Wed, 18 Oct 2023 14:28:29 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=none; spf=pass (imf23.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1697639311; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MSt120ujaCbI5iqH61p99iQMj5WX9vWzUelkG1NabO0=; b=cd3f7FguKvw3X860xmcoHSC1WSFG6K118AKre6WAfACdWfqurSsvmT2bVJUO5z6D6gEfpg gEQ8lPPZzl4C+yJ4ML58YwXqpDhIRWA1M0mzHm3Da9pOjo7KMawGoDEFUPkenaEc69YudW 05QqRlUaiG72shK4BBfSzen9Ol0D3og= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1697639311; a=rsa-sha256; cv=none; b=TyXbwJlVXrqsQm/qsbFOAowXzDa+vCi4+zCyAuux40kO1boLVClv4NUHdVrEjCNbsf2Brd 8TRc6ouokzc7/jW+P2nNCCf5l6kK2w3jH1pHq7lFoRInP8vmeb5yIN0oKjKhZ0GB7l68+M MQsoDPMxGUSX9KLhbIkIM6T7E0b6Wmw= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=none; spf=pass (imf23.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.57]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4S9Xfm5FCJzCrSB; Wed, 18 Oct 2023 22:05:04 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Wed, 18 Oct 2023 22:09:03 +0800 From: Kefeng Wang To: Andrew Morton CC: , , , , , Zi Yan , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Kefeng Wang Subject: [PATCH v3 08/19] sched/fair: use folio_xchg_access_time() in numa_hint_fault_latency() Date: Wed, 18 Oct 2023 22:07:55 +0800 Message-ID: <20231018140806.2783514-9-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231018140806.2783514-1-wangkefeng.wang@huawei.com> References: <20231018140806.2783514-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 7832914000A X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: pcxf9cywz58xriph7g8gbyjrkdj4idtz X-HE-Tag: 1697639309-118937 X-HE-Meta: U2FsdGVkX199kiU+GlUj1Nuy6rqa/aZTNdY6lh4y4d397PFid4se7gevb9BQr1vZkvK/o1FcfklFO8xwRG35GrNuz3Or8misQxAgmht+qICxzYWh8oUPiibKnt6LqbUD4xa4zWvXbKBhjepW6Uq/rKBaVFS04je5x+WOmPec8UL0/9zYlOQdRwikaoU1Rmi2hqOExlN0K2YaSxuNG0JdKhY8wt5hNNnhjYqc7fRh1o3gp6EvlaAU3eS+xzFgRPPVkEEm+VLWtrZ8+xeyVTGBRcwW1ICxHlGb+FRoGod7Olj3ESqfYlKTW8GOAYcuOOP2pEEDAnEjbNu1tZn8+Cyq/iKd4fMAZQ6UoCmK3Vr5rvxZfgU6NurhLdekbeVY6LVYS3atX87y+Ly9Fo73ZH4XpGfgV5SlJP7oIVRoFphFAMjPV6tx6VP58KKVQvbME/pddIoWyMjfPwrIG10QYyBrzqL/rfA1TQSs7L+/CygxGHxPTfvct6ZWkTuiEV58XoSG8tXKVjsK1UAyWZ8lMS6Tv4GzDBgFS9L3kSrMRFjAk800BR+IhM0m5wj+BGRrjUC4pR3yKBzOT45yjpEZiq0gCbhEyFac/NDYgKthufBAxBQPbrufbLyMbgq6+QZmcxbQOvLbFfpJvkWxDrQFsaATdgM4pW8xXuibq9mkdq5h4JKPDQLn1uvbeQdBWghG0r5paPfBKTCTc83rlicDHdN2abXYh233VSDnrWD2ka+ZBt+FMbZs1nUHcm8jqb7gQq4cdWS79chNvVun0RY7/Yyzt+oImeWKxOOAjB53Ad6cc2KYrm+bN+VVg6vz1XtQyT8nTSeepoOUuE82RYwoudVF4LyiYcmnVd8essgXCci2u3xnRczB0tfHhL0IETo4Ynm/r+ASv4+Q/dyZun2+28847RY97QdMkznOszjm8H/kq54PS/4dSTT0XDjc9m4ID5Rkcp/Zf4bzjYdBPIxGQAH pb99JMXP 4LIIVC9+5YmbCvhnEm/vrsplTiNY2SV3ccf0yuadx7KfwFN527yf2lH7EgmngzFD9T7hWrqx0+nElZ7sUGIEmpyzM/N7jg/imX9clI27l2gRWr6bWSP66qcIqR7kfQHg8fDlB0x+fuMeaQ9M2NR9Vg3Tu3+YssP6tAWx/3i3tyGNNp4+dGkfmFzpqkmJuNl7Hch1u2Re/R3/9PQbDgssawRIOYIrAIx6n19rx X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Convert to use folio_xchg_access_time() in numa_hint_fault_latency(). Signed-off-by: Kefeng Wang --- kernel/sched/fair.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index c50271f39d91..b8c52893b3d9 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -1766,7 +1766,7 @@ static int numa_hint_fault_latency(struct folio *folio) int last_time, time; time = jiffies_to_msecs(jiffies); - last_time = xchg_page_access_time(&folio->page, time); + last_time = folio_xchg_access_time(folio, time); return (time - last_time) & PAGE_ACCESS_TIME_MASK; } From patchwork Wed Oct 18 14:07:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13427127 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E2E21C41513 for ; Wed, 18 Oct 2023 14:09:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BA64B80012; Wed, 18 Oct 2023 10:09:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B54066B01A7; Wed, 18 Oct 2023 10:09:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9F5BF80014; Wed, 18 Oct 2023 10:09:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 8BA7280012 for ; Wed, 18 Oct 2023 10:09:12 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 524E814027D for ; Wed, 18 Oct 2023 14:09:12 +0000 (UTC) X-FDA: 81358764144.20.1416653 Received: from szxga08-in.huawei.com (szxga08-in.huawei.com [45.249.212.255]) by imf05.hostedemail.com (Postfix) with ESMTP id 27309100016 for ; Wed, 18 Oct 2023 14:09:08 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=none; spf=pass (imf05.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.255 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1697638149; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+SExSAFdXGenEFq34IiGWjXXWvTPNv7RgaHg/hl0yTY=; b=iYgx4WQm8DYmgC2ra5lv6pQLbptzXkUaInkW6+95T3bl7Jn5/jQsryt7y4EMbYn2AtIIBE GZbk4WHD2Sa1MmwZHBBKxdtADEl+fIuXuvOzfuL+ZF9J7edWmoHthSQludY+UjuHlbS74l 4bV3uzaSOjTxZYmuIJ65luDfmujRapo= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1697638149; a=rsa-sha256; cv=none; b=AUXkvSx+sv5QoOEOjCrm23OAMjfYogP1iXNlZYLjalA1PSA1VyKn9/STAH0chsmFe/VDsY UT+qWTO7htuzK6+8a1RQfAq6oUXyTugyDP8TsD5ApoArfqzqkL+TAZ1wG+O9HzvYYFwWdu 0Rg+uapguEYa//OtTer2BxBnoEOIZrs= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=none; spf=pass (imf05.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.255 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.57]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4S9XhF6Tn5z15NWX; Wed, 18 Oct 2023 22:06:21 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Wed, 18 Oct 2023 22:09:04 +0800 From: Kefeng Wang To: Andrew Morton CC: , , , , , Zi Yan , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Kefeng Wang Subject: [PATCH v3 09/19] mm: mprotect: use a folio in change_pte_range() Date: Wed, 18 Oct 2023 22:07:56 +0800 Message-ID: <20231018140806.2783514-10-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231018140806.2783514-1-wangkefeng.wang@huawei.com> References: <20231018140806.2783514-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 27309100016 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: 6dtrss1okwrqbebkqxb4pu5wggzxagu3 X-HE-Tag: 1697638148-155970 X-HE-Meta: U2FsdGVkX19Xo9s/9k5vBX4ABCrKel6Cb0uODNsl3NaW8Fg2q3HmFzziEI4S7RyxEy5gU2u/XHq4OxWmS85GWZHGOrW+ifTl/UCtThPB9+phUXTwnPUp50f8N9mMMgiGLuGABqbG5p7KF1qIA3a5Ldn5F+WE8rfF/nP2Tl2m6Y0kN44boCRdUTSq2WedsF8UosckaarRU/uNyyUDZ5a6snB3dyLQ4rk8mr5+l2af3KpuX2VbQv/TB3WGT38xvyrQhORW6iqc0O5JqOcwgrNIz1YwFQzW4LJwWHVwjjyCFECj/w723jQiX2VdvrSsyQa6p/FvtHsuaDZXT0pdsLQQSslDvdTY78P27ljH1UOZ2hWfHCqbvJz4KAKEd4BaA9zJScwYvA5uEmIp8pESRZ2KI6G1FBGFhqJ23dV7gFkJWkYkb8M9RydbMuDEz3j9VUg6Z3CnS+Rys8lLo2WH4H6Yfsj9JqVBUUumcSqOKNjtjmCL18asunqFnA7pEJrMryroFM7YqpVp5WJjpjRk+bU07rhnAHXQ3nghzxmyZyEQw/c254sZ9y8rN1b/9fg1lSra5JIoqyPL1d+ZAjtCAPx2wwK3Z8fLsMaLlwVi+TTSiHSU7/zgiXO3Z4T39akUOYj1ZMq7X7yvyl86a3gYAyLrRcDlDZ7SkaU1cwF/s61tSGhIJ+lB1dKAA7Ekd/YsLPXZ41YpBZFgzqKu0xRnMgR4ql4/zBQ0rQWwKNzAzLjPZNet7JZKxS6F1gutvooXTTpHftv+Xdxb+XZS0V17i42rJXOMbKCx0QE9ZQ9Ffmf96jIkBg2w184veYc14q60KfY/BcJ+guZvZjqU0vzFaJO6JhqvBY9b7wgQGdV0BvFzN8Xor2nddBHqzM1Yd8FBvqGWwwK3D2T8Tn3DQDSd5jvX4JyTHrnWKOp1l5wYST3LNzizSaA6JSLkYIXeW3PK1Hh2KfFSSyho1mSDfaUxOyq 5PQMKdI3 4/nn7iOuIqU2a3vbL1jo7RiqeQIKvbCEfhtfj09eI1nMyh0aoUHlsTiBW1P1Qn6/ri299l5vZJgVaY6VoG0ZYoRrx68hqW7+O8o/EAozkweWVNOPH0EAFor+a10uPauJBZwkLjU28WXGGaKjx6j51Sa5m8N7C8GVfN5VG363hS0NsnFzuD72L4Cbf2T0mylITGvJc X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Use a folio in change_pte_range() to save three compound_head() calls. Since now only normal and PMD-mapped page is handled by numa balancing, it is enough to only update the entire folio's access time. Signed-off-by: Kefeng Wang --- mm/mprotect.c | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/mm/mprotect.c b/mm/mprotect.c index f1dc8f8c84ef..81991102f785 100644 --- a/mm/mprotect.c +++ b/mm/mprotect.c @@ -114,7 +114,7 @@ static long change_pte_range(struct mmu_gather *tlb, * pages. See similar comment in change_huge_pmd. */ if (prot_numa) { - struct page *page; + struct folio *folio; int nid; bool toptier; @@ -122,13 +122,14 @@ static long change_pte_range(struct mmu_gather *tlb, if (pte_protnone(oldpte)) continue; - page = vm_normal_page(vma, addr, oldpte); - if (!page || is_zone_device_page(page) || PageKsm(page)) + folio = vm_normal_folio(vma, addr, oldpte); + if (!folio || folio_is_zone_device(folio) || + folio_test_ksm(folio)) continue; /* Also skip shared copy-on-write pages */ if (is_cow_mapping(vma->vm_flags) && - page_count(page) != 1) + folio_ref_count(folio) != 1) continue; /* @@ -136,14 +137,15 @@ static long change_pte_range(struct mmu_gather *tlb, * it cannot move them all from MIGRATE_ASYNC * context. */ - if (page_is_file_lru(page) && PageDirty(page)) + if (folio_is_file_lru(folio) && + folio_test_dirty(folio)) continue; /* * Don't mess with PTEs if page is already on the node * a single-threaded process is running on. */ - nid = page_to_nid(page); + nid = folio_nid(folio); if (target_node == nid) continue; toptier = node_is_toptier(nid); @@ -157,7 +159,7 @@ static long change_pte_range(struct mmu_gather *tlb, continue; if (sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING && !toptier) - xchg_page_access_time(page, + folio_xchg_access_time(folio, jiffies_to_msecs(jiffies)); } From patchwork Wed Oct 18 14:07:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13427172 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46A6DCDB47E for ; Wed, 18 Oct 2023 14:26:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C728F8D015C; Wed, 18 Oct 2023 10:26:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C22B88D0016; Wed, 18 Oct 2023 10:26:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B111F8D015C; Wed, 18 Oct 2023 10:26:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id A07768D0016 for ; Wed, 18 Oct 2023 10:26:57 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 50E57802D7 for ; Wed, 18 Oct 2023 14:26:57 +0000 (UTC) X-FDA: 81358808874.29.E2CABE8 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by imf26.hostedemail.com (Postfix) with ESMTP id 110B5140021 for ; Wed, 18 Oct 2023 14:26:44 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf26.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1697639215; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ms2pbGLo7mG2bKDY2JPOUHEMK8SMoQKLkUCNA44AIfg=; b=feiwDez5V130mnsTe/0rKrXi6oqeB6WF7mgAeBfrp2Wge5ot5oizeAGAfK7wjO5fdgYUl3 VqePJtTCaWaHT1yYIAaZOiaMBTEYXquFNsFBwMlH4aqX63nqXkT6BgaxoPpjFTeT39vzW8 7YKfhdog++LJDOWVcksiqQf6091uuBc= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf26.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1697639215; a=rsa-sha256; cv=none; b=qrTjMLT6oV1qa6rJEZAov6SyHTGcKvjBeN316hUsyCmAE3/mvEvaKkB0axgvsHt9tT2JPt y3DkskGqkgoX+KFxqGoRvTfpxTmsq1rW6TjHGhF0l2uoK/3mP5cmEL63hsTn1JMKIThddX tvK+GXSpw75fUDayeobuweGmjkwRQqs= Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.54]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4S9Xdw4TSzzvQ9v; Wed, 18 Oct 2023 22:04:20 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Wed, 18 Oct 2023 22:09:04 +0800 From: Kefeng Wang To: Andrew Morton CC: , , , , , Zi Yan , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Kefeng Wang Subject: [PATCH v3 10/19] mm: huge_memory: use a folio in change_huge_pmd() Date: Wed, 18 Oct 2023 22:07:57 +0800 Message-ID: <20231018140806.2783514-11-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231018140806.2783514-1-wangkefeng.wang@huawei.com> References: <20231018140806.2783514-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 110B5140021 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: xfwpe1qw7qrxbwa3czj5bkoszziffgx4 X-HE-Tag: 1697639204-421139 X-HE-Meta: U2FsdGVkX1+IosNz9GOvoYuVV7xs51ToJj8ZhEzaUv9fP4tidkAMeGzOwarVQu95ZoAebxHeBicDFztaGi24kwNLSlbFZUPOIYYj6V1fhLrMas+BBnGXuyJxyFy2mii/11ZCmf2nsBvaBndy43ryWkYr0O3+XCHzgEnoOknWmokcLg4zpXmPVSYnq2ta3wIYorepgAaLOl+3l0Ljrn72aPETKG8htENw3zJvZ3dQnVHhxNBuDSPd7ud6K2l3m4cIrSoGSW0e6R1LdYb7u44YTY24rwSx4uiu+MrtN/Z/vWzzPLhrYHNbczbUXF2QzjKJBjfeaJW9xCjD1wfe1wul/KmhD5+mVN4R6xinIIk+Itc+DFJVbk1p3YsL5a6WyuKeAZe9gV/aMY54avpQQa0A6L+ggQSmdybWrSgfDAZ7kCSQH5nGMQFD6NxL32TVRvduLCCfZIThxKm78v1BLX5R4AxF0lGatHNfnok523mwZ9fTsGmxyQ1wT/keMHXeyjEyWfHX2UVanYqdtkGLxxYoPYUr2ciq/ZD2hn3QA3JxXxy9bVI5AtERdY7o8L7K+7xibjQMZ7S6kqLq0wUijgTfKw2VmKWkrzdyzHa3NbFz8IpZs+ZiJlTzlDMHOzOLIr+y5TLY4hRhx4U9rIB1ZYHC96rOcKC6rR/8GL5r0gyxUZ2llqqHMMxfybEMTUy0WhKNwopnEwHzb6gUwhLXd7cf2fHy01Ip0IW8gV/vrDfFO7KyZasrtJHlYcLyg2wxxtl6Jg3SJKDXYLLVnMfJj70Z5GB2BlvMHxHJBkNcB6Xv1DCreCp2VzBdMklH5ErfVFfnFCuyMZnlA4Y8lCEj1XSX3gpjb97F/erHdtNCfflOmnJ74tuegZanpb9NRB20ZWw/yFYQ7AD316mUYJgwJKmEIojQMcJp/uAosYWuYir4gtJFF3ECtDIusLB356xleXuS41o54WflZoldWzblsXF H8QIYUqL J7SBJmATwRNARnEnPWHQ+JorN48hDv+ywUKJxIev33S4PPEMR8vHKJpbBjbKEMF/owegQSVXwJFuQsxmVOgh0smoYFcaZ4RiQO6ysB6ADPWHzIrvYxpVe0NurVf8Pj6sRNPKD76wwQaizpEdF9ltQrxDqPiqbFoOGOLA55iywmpmTuMRXvK45b15N2qKjh5+NqFSFZSc/vtg52TUXHW1clX7y/7/vHZ8F9/6jOpjSBusmBujdJbFuQUgVnFpl1Lj7Yf0XZHNXOSlrCR3bTrvR+fqbsidYySCt26pI X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Use a folio in change_huge_pmd(), which helps to remove last xchg_page_access_time() caller. Signed-off-by: Kefeng Wang --- mm/huge_memory.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 5455dfe4c3c7..f01f345141da 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1856,7 +1856,7 @@ int change_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, #ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION if (is_swap_pmd(*pmd)) { swp_entry_t entry = pmd_to_swp_entry(*pmd); - struct page *page = pfn_swap_entry_to_page(entry); + struct folio *folio = page_folio(pfn_swap_entry_to_page(entry)); pmd_t newpmd; VM_BUG_ON(!is_pmd_migration_entry(*pmd)); @@ -1865,7 +1865,7 @@ int change_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, * A protection check is difficult so * just be safe and disable write */ - if (PageAnon(page)) + if (folio_test_anon(folio)) entry = make_readable_exclusive_migration_entry(swp_offset(entry)); else entry = make_readable_migration_entry(swp_offset(entry)); @@ -1887,7 +1887,7 @@ int change_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, #endif if (prot_numa) { - struct page *page; + struct folio *folio; bool toptier; /* * Avoid trapping faults against the zero page. The read-only @@ -1900,8 +1900,8 @@ int change_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, if (pmd_protnone(*pmd)) goto unlock; - page = pmd_page(*pmd); - toptier = node_is_toptier(page_to_nid(page)); + folio = page_folio(pmd_page(*pmd)); + toptier = node_is_toptier(folio_nid(folio)); /* * Skip scanning top tier node if normal numa * balancing is disabled @@ -1912,7 +1912,8 @@ int change_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, if (sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING && !toptier) - xchg_page_access_time(page, jiffies_to_msecs(jiffies)); + folio_xchg_access_time(folio, + jiffies_to_msecs(jiffies)); } /* * In case prot_numa, we are under mmap_read_lock(mm). It's critical From patchwork Wed Oct 18 14:07:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13427131 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CE84ACDB47E for ; Wed, 18 Oct 2023 14:09:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A98FF6B01A8; Wed, 18 Oct 2023 10:09:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 98DE680018; Wed, 18 Oct 2023 10:09:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7D32480016; Wed, 18 Oct 2023 10:09:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 68B376B01A8 for ; Wed, 18 Oct 2023 10:09:28 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 3129A1A01ED for ; Wed, 18 Oct 2023 14:09:28 +0000 (UTC) X-FDA: 81358764816.05.2AFA02B Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by imf12.hostedemail.com (Postfix) with ESMTP id 7EF8F40025 for ; Wed, 18 Oct 2023 14:09:24 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=none; spf=pass (imf12.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1697638166; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dav5PI7iEms8jeP4a1XhSbNqrplMPXmJIKIe/iyLzYk=; b=V2KeGi7SaH2Fs2Cehu+v41CbKRHSsFFFu0C73RK2b/pR7/LMa51/W8FZss+Oj5/4dopZUF u7caH++xrqQpHyYQeZUzg02Rkf8hvGtTMpJ/x0GLAy7heHWzt+dhQZdcO8C319+vuB/twC g/tIXEdfjhJpmllOEvteVXPTW4Z6kfo= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1697638166; a=rsa-sha256; cv=none; b=Y5GzyEc2nl+1jE0LbUc09IrwbMIgMZjpFJ8nw/a+JWJ1S6g7krolgjLAZMoqDiN/YLTon1 XygZolbsI50hcE33KL+tvtgbhtKIhSBHvGuQvx7UkFJo/OC6G+y0qH4fn+2rOWQpMFtof+ ffq++/mBaawcnhw6hTQxPscspO71yBc= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=none; spf=pass (imf12.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.57]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4S9Xg93f9RzRt5H; Wed, 18 Oct 2023 22:05:25 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Wed, 18 Oct 2023 22:09:05 +0800 From: Kefeng Wang To: Andrew Morton CC: , , , , , Zi Yan , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Kefeng Wang Subject: [PATCH v3 11/19] mm: remove xchg_page_access_time() Date: Wed, 18 Oct 2023 22:07:58 +0800 Message-ID: <20231018140806.2783514-12-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231018140806.2783514-1-wangkefeng.wang@huawei.com> References: <20231018140806.2783514-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 7EF8F40025 X-Rspam-User: X-Stat-Signature: gxsa1oxk7k9mont9z5bhsjyji9z9ptqx X-Rspamd-Server: rspam03 X-HE-Tag: 1697638164-727529 X-HE-Meta: U2FsdGVkX1/BZTN/k1PH4QZpsbVx67Ia0YRBBjCieJcU9Bqml+1TZ3W6A/3n+8DhzYAfe8TaHDRhFdB2wBhCBLVep6Ms4Q4YZxyw/NypUxuPsQ8YuWRAr6F60y0UfajN7Wz0xkwAsbg9h6t3nEjm+kWM7+dhPV+7Q6mRoRvfRKNrcSC3GPj+wQUh/urmzMi+gWSzu2xMXOZWCPW66IYUpcykjv3pS9KM2l6DxfzdMD3hWavEslV1GrXNqEcslnorhjOzME6RuEWwTSQPMxyTRCW7Zfx/NBZ4/I3a/tXuL4i09+Sa+f3AtjI8ZNcQKkJblNg4NMjSH6ImTBuPl4MW5FkZO3x8iF2ZwR12n9GvSX1f7hHGHpzXijNFNu8Nv7MZGG6NYSv3oOKpyHzDf5x84KE+i3Zd4ybSBC34sSKzpR4s/lz9zWK8kFBU89m0yAeQPmw+bs6CZMUFWBEY+PCt7Ywuzb4gZ/KvVhw9M01dVvIPw1Elpokm4Am0hsctDlqOK5enfOu0X5FCWtK95T6l86CpecRsLW9anGF2OSkeKclM+Zpvd0iAQRQd75SKDmchbO4lUUY5OGBScBjMAExHzo2iUOBNbuLtbpOXIede9qNjp/VOXwWQ6kIXsynLjUEePeidOmgXoO5HGYNpuRyzTKsf/b3ohId+AQkwIrF1G48qKTiFu2yUqWqHT6tr+vMDVK4PmzCod4PcWSP/Shh25VrnVB/Ikka3KX+mL+U0M5HBkQBD6FZyqJ3Il36t26Q8jD4sPnuxN80jfFPBppvtUfEwOrQLFNvuz2nbHblD9Us/sh8KR717p8iFVUtGDx6eOpaj1mVOS2TJnyE+vpC2nOBZfbc+JR0gvZRHfPvoiwXEBgRIx7DBdWjNR56CrltUr5wumebVYfFWxXoRPNOMchg4E1lIA5fFiHZHHF0whgmOo4heqjEu+P4etpKryRM2+MEpndr2O9UJW9uzsEc fbYbZLwA cI3lzrEPcEp73A/h7Th6CKiYmmk2yxFf6u6SaoH24BC60/mkJzyFU86qmUUHiKiLzr35/9liKisnwD5Nl1TMHe7J0JEDqQr+jG6u0ydT9sJqzhJ3CEdBcZLtavk53ZSx+0lg06ELEOFRHBZfEHw3h4kCcwkQbsXAsNoZhqcl69UJ8BHZCeRtYns+jYnkmXRfr4dq9 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Since all calls use folio_xchg_access_time(), remove xchg_page_access_time(). Signed-off-by: Kefeng Wang --- include/linux/mm.h | 12 ++++-------- 1 file changed, 4 insertions(+), 8 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 1238ab784d8b..8a2ff345338b 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1722,11 +1722,12 @@ static inline void page_cpupid_reset_last(struct page *page) } #endif /* LAST_CPUPID_NOT_IN_PAGE_FLAGS */ -static inline int xchg_page_access_time(struct page *page, int time) +static inline int folio_xchg_access_time(struct folio *folio, int time) { int last_time; - last_time = page_cpupid_xchg_last(page, time >> PAGE_ACCESS_TIME_BUCKETS); + last_time = page_cpupid_xchg_last(&folio->page, + time >> PAGE_ACCESS_TIME_BUCKETS); return last_time << PAGE_ACCESS_TIME_BUCKETS; } @@ -1745,7 +1746,7 @@ static inline int page_cpupid_xchg_last(struct page *page, int cpupid) return page_to_nid(page); /* XXX */ } -static inline int xchg_page_access_time(struct page *page, int time) +static inline int folio_xchg_access_time(struct folio *folio, int time) { return 0; } @@ -1794,11 +1795,6 @@ static inline void vma_set_access_pid_bit(struct vm_area_struct *vma) } #endif /* CONFIG_NUMA_BALANCING */ -static inline int folio_xchg_access_time(struct folio *folio, int time) -{ - return xchg_page_access_time(&folio->page, time); -} - #if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS) /* From patchwork Wed Oct 18 14:07:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13427133 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2D4B0CDB47E for ; Wed, 18 Oct 2023 14:09:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2A81280018; Wed, 18 Oct 2023 10:09:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0AEC880016; Wed, 18 Oct 2023 10:09:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D7B1080017; Wed, 18 Oct 2023 10:09:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id B555D6B01AB for ; Wed, 18 Oct 2023 10:09:28 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 5B4BB40224 for ; Wed, 18 Oct 2023 14:09:28 +0000 (UTC) X-FDA: 81358764816.05.94B4555 Received: from szxga08-in.huawei.com (szxga08-in.huawei.com [45.249.212.255]) by imf12.hostedemail.com (Postfix) with ESMTP id D0EEB40017 for ; Wed, 18 Oct 2023 14:09:25 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf12.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.255 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1697638166; a=rsa-sha256; cv=none; b=ZVV4l6aP8QvFDixodnCs16gJsW/sgf6UTWbGE1fBzTEo28jwayhMYSe0MxSgUsDSHKUpRw R9v9kCHx4kgOI4NQ4z1DJz0e4JIV8S0+svPugWcN6coGsh5znqnSaRTG1aXFojn/zYcJUk LyyEg/LDooxm9NQGnRdGwZavcxY2wF4= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf12.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.255 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1697638166; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DNlrD185RwWjuMxj0G5WMekXxP2MZsx6AcQ/7SuQlaE=; b=dqoEcz/KJuuvn8Kacdk43l2ysnNQ8VxkulIqlw0VesNGlxzAcxkCgJudI+PnQwqHL0HAyQ BEfPKOUJ/kD2mf6BHp9GTMS0Dnxep/95zyNnPzKa8exxIQZRUnpPVagdof3TwnemMcK3yo yHS3x20GBfLA4nOuHeUau7vmwuLKP+Y= Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.57]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4S9XhH50l8z15NWq; Wed, 18 Oct 2023 22:06:23 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Wed, 18 Oct 2023 22:09:06 +0800 From: Kefeng Wang To: Andrew Morton CC: , , , , , Zi Yan , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Kefeng Wang Subject: [PATCH v3 12/19] mm: add folio_xchg_last_cpupid() Date: Wed, 18 Oct 2023 22:07:59 +0800 Message-ID: <20231018140806.2783514-13-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231018140806.2783514-1-wangkefeng.wang@huawei.com> References: <20231018140806.2783514-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: D0EEB40017 X-Stat-Signature: bkshqf6ejbnqwpz793nsenjsttwygz16 X-Rspam-User: X-HE-Tag: 1697638165-280559 X-HE-Meta: U2FsdGVkX19vN2z9hTrja8T9ozfMufJnRu2HAH74FEQNNxqeQvnC80Me5oeGAyEFycC91HzzbBf6Cba65D7sDuwFghj90YuocdWJbgeSpuM2cSHbS9WdpyQuadAE/5S7Xrnh/qd5GTZDtI1sgAtiCaVMzdmTXZtz76tTGMOUGYgQCC43dxOdjKLswhRziFB5aWYC33X7LzCaKBCDxHo+LtkrK6Cjrd+654cNNB6+lSdBKKLs32hg5nnyjemRJupRqkUjSxYY3Z1CJoZP3e5JjNFCgJaxNprCdQyqHz6lmH2WEXs3JAkhJAZkPTnGd9JjwF2yiZADpLDmSvsYFkSKtoe/737MZOyIDRZvNRP+MaT7vj996HJ2twHVm8t+oZZCcFpp3niuWHU7G8be1fFIBmVsYiJe54PyC44unsKwnLLQkcKEWwcYEjOHVldrp+hSZk1ToAvHV4/uQ1XRqQT7LWBq57+6vsZDdjznlA8JeE+EeRRadWdyshCFXeW0gee6lCr3e7OA+RqAtHzLSER6p0bhOm6IDvcHakJJL0PNwjLsimDvXbcvBA2k5MAXCGJRdPtJlMpfQ6rDzSgI1IhHz2YDgtWkuiqXPmxMyxaHPMbOg1ZaIH50r/w4GRdjP2zZONujTX2s1EJv6SIhyYMVIyUtadF8E0vMa+3Kfo3t/D0kTCd6fEtnru8ELjxZpTC+QHiQgibBQS6kr6JbOIE3/dm1/fVdsRuImqlrYGArtjtIpssnGhxFW1QKB0n3hJ3F20F3gzARlNbeT15nwondiIXYktoP8ggXxPM8TC266CeIGpYjYigYUso7jlf/LMyR58cjGMpNuoIc+5FpUWfTBRbChjplfM2BMEpPizthMx/c72jQhAsQ71Bkatt55GislU4NwhqMkODSE+h9L+t55zgwGlBxKvqOmU97TR/zbRYLJGnO9ftbBAwwSsoYZ2eEZTycgnsyCw6RP5ZVFY1 Ndl5j5Us y8oA+nMX0u9sEfBHtIEeX6IaUzLydKVBPT6Er2TY5jWugUHhFczSvnY8TYRig8geWC7B1itq1BhV4Yve9dFTwqq0FW7tO6ByU3pHk+rdjoBo5ZAoZ0VIDmyS1+weyQeskU67gs4B6AnFJ7U7XcAEsTWEibFdde06/sfZQWxlM2gN8FWkng56oxYQ7rLCXoIRzv5WA X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add folio_xchg_last_cpupid() wrapper, which is required to convert page_cpupid_xchg_last() to folio vertion later in the series. Signed-off-by: Kefeng Wang --- include/linux/mm.h | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index 8a2ff345338b..8229137e093b 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1795,6 +1795,11 @@ static inline void vma_set_access_pid_bit(struct vm_area_struct *vma) } #endif /* CONFIG_NUMA_BALANCING */ +static inline int folio_xchg_last_cpupid(struct folio *folio, int cpupid) +{ + return page_cpupid_xchg_last(&folio->page, cpupid); +} + #if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS) /* From patchwork Wed Oct 18 14:08:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13427153 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8010CCDB47E for ; Wed, 18 Oct 2023 14:09:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C1D0B6B0203; Wed, 18 Oct 2023 10:09:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id ABE7980017; Wed, 18 Oct 2023 10:09:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 847676B0205; Wed, 18 Oct 2023 10:09:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 7115480016 for ; Wed, 18 Oct 2023 10:09:29 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 4BE87B57AF for ; Wed, 18 Oct 2023 14:09:29 +0000 (UTC) X-FDA: 81358764858.20.D79CFAC Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by imf16.hostedemail.com (Postfix) with ESMTP id 01AA618001E for ; Wed, 18 Oct 2023 14:09:25 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf16.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1697638167; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RlBbZypv6yE/hTj8++hHCKN5K1x2FYqLwI5an1ekiHg=; b=RI5K0m/kOtT69DZetblsgFIQIm7ZU/9iNagjK3XblqciQzGesqeTEXKcuq9CCF7mwScpks 7+Y6Tj/WwSvDAHv60iYqbbd+RIUw79nKDy3R9SAOTx7O5e5CXCnyDkV25yBqs9mePcQLa8 I2AYN1Ean60S7Shy00Yvdkk/oPqm+c4= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf16.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1697638167; a=rsa-sha256; cv=none; b=2EbdKC8acAi2B9UaYHOYSE1BrW31voJnBt9EwS2f3lpoF7MRSfJ7zr+Z2b0eEi//oe6Djm UFcm3CFN45urqo6ZlfDuloHObXrw4+OFGw0/hIzkLrasTB3TOZBJPoCdf1y2W+fpgohGL+ ScdUpBzCjD6xfMfyZO+aAvu0I0GVbEQ= Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.54]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4S9Xfq5SM4zCrSG; Wed, 18 Oct 2023 22:05:07 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Wed, 18 Oct 2023 22:09:06 +0800 From: Kefeng Wang To: Andrew Morton CC: , , , , , Zi Yan , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Kefeng Wang Subject: [PATCH v3 13/19] sched/fair: use folio_xchg_last_cpupid() in should_numa_migrate_memory() Date: Wed, 18 Oct 2023 22:08:00 +0800 Message-ID: <20231018140806.2783514-14-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231018140806.2783514-1-wangkefeng.wang@huawei.com> References: <20231018140806.2783514-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 01AA618001E X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: pmdrp85x8i39snmbtm6zsnmpy5d5sumb X-HE-Tag: 1697638165-848734 X-HE-Meta: U2FsdGVkX1/+JjFckpIzjcK+iOZBY6byWJJhKxNhXO71hZTOG/90TFFMPyF9Z7KpoVgP3s7sruGz6PsdYJE/d5v1QUFJT8XaMkDf/LZ58l/uoIvalyWn7B/Gs4VxREiWovSWhfdLHJ3kvzsJWbPrWCF0k3MqlIKTt2msWDc63jdmSKy0w3mEvxjMkcIgORVdD4XiHn6V/bF5wwqOGgrgaPr3RyTvLKEOhkbIvtINU21nMSgFu7hdQbpiNQjN4WamHNW6MWRrkXctOjsnP2VcsdBy8HeP3uxzk/5cPcMntD9yDMtQr4eakd9/I7kug+jCKs4wllurc+uOlcQWBN4eHg9gP01rATnvSKC/sTpyDG2v7q5YwZQt9ZL+EzYuPjkBd3eOp5g+3o+PqJIbG4Ee+c8+y8v44AXPjnN8aw5r1spyKlS0dT9XPqeY3x41FxUlWvktHEPMcKlobKSDCf9Z3G+qkLmyc1IMnG7dkfwBOypBxNzF/kqCt8e0wviTaqef1kDabT88zUEnTReimKksHblXDa9K0c9QfAk0b6ESyVOVyk2PVlcriRmd2dxac2/ITGCl1PQxnzhbN+fEOblD+738J/vfHVs31PDb1RiDBNXLczcDwj8iyptsDJVPzVcE3U8tm/n63u9qa/dZRgnWvp5pp7aQzGUQgEJpkS8u6G/hrpwkrUkG7MVmebGWWTUwY556+A9abu7iFdvEkZ5UuaHOkq+hl1i1CKVWAXmEifHtywCokflqBn3s7H4S+6Z1XP6tTZnixyRdEsixF/507IWUYCQ+f4KcFxKicZ0iC7/BMIEEzzdiFsT2VL+aPwcvQaZx+8/10oE7iUwCXQtgYMB1NoY6/6xJm4tmi1o7JUKdzoVbBU+toPG/4IlhNNETN/bpGI3POBEklBR+VBTClo83xs3q/iwcDtMwu0RBi2YVBHL9Fx5NxJFEHnYk+Dax8l1Y+yQut73Yai6z82I 6BQwiety dTICLM82L9XHKgJ2wNIdLvVhw6+fP+cz6AA+drtx3/lgkZR743j7n1RkTOwvWsX9LrojaF2WIdOLtdtFCsn7QKSC1gNF8na99r6fok9zOdV0cUKzEqUyHjUrERnP+DrkYdaJRoZ6R06CzBh9SlR/2HufvX5GjJ6O1Jbr9DDwHMIXL3EudrN21dv+Z5Wicrw95L0EZ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Convert to use folio_xchg_last_cpupid() in should_numa_migrate_memory(). Signed-off-by: Kefeng Wang --- kernel/sched/fair.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index b8c52893b3d9..bafac84f7be9 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -1862,7 +1862,7 @@ bool should_numa_migrate_memory(struct task_struct *p, struct folio *folio, } this_cpupid = cpu_pid_to_cpupid(dst_cpu, current->pid); - last_cpupid = page_cpupid_xchg_last(&folio->page, this_cpupid); + last_cpupid = folio_xchg_last_cpupid(folio, this_cpupid); if (!(sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING) && !node_is_toptier(src_nid) && !cpupid_valid(last_cpupid)) From patchwork Wed Oct 18 14:08:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13427128 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A5D8DCDB483 for ; Wed, 18 Oct 2023 14:09:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 032BB6B01A6; Wed, 18 Oct 2023 10:09:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F25F880015; Wed, 18 Oct 2023 10:09:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DEEB980014; Wed, 18 Oct 2023 10:09:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id D01C66B01A6 for ; Wed, 18 Oct 2023 10:09:13 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id A95B0801BC for ; Wed, 18 Oct 2023 14:09:13 +0000 (UTC) X-FDA: 81358764186.28.FDEE810 Received: from szxga08-in.huawei.com (szxga08-in.huawei.com [45.249.212.255]) by imf19.hostedemail.com (Postfix) with ESMTP id 325161A0028 for ; Wed, 18 Oct 2023 14:09:10 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=none; spf=pass (imf19.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.255 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1697638151; a=rsa-sha256; cv=none; b=rLkQcfg3bRlBMBT/Yz97AGSiyVNwctnbcpYrPzberd9sYMKqt8cQ7l/l0Ei7zX402jNuLX BIbSH5prqqmZEpoztlvI7rPSwmmlU8dcg0waeJA9npBGPwh0mISJvEb6sYH5Pm21twkqNx oQ0WHnfpc/xcHbXqSNXjuyYdge6Y9os= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=none; spf=pass (imf19.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.255 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1697638151; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ONiFhuoJXZyIBus5P7d1IKd0msDLCi+tp53vVhwi8Pw=; b=op8Q72vCOPqJzV9EatI2kYc0/dPACyIHkPTKt2qmrFJgQ4qdWCi3j8tio/xLUdzZzzHXO5 1MUTXDGAwwQ05fJ2T4mY7JFxUBuED8gzhklxkhZPbNt8ySLwuJ/C5spG8UzL7+O7awD/TD QfhBK3lM4fLvxKjKPCQ1Qmc6JTI71DQ= Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.56]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4S9XhJ6mgGz15NX0; Wed, 18 Oct 2023 22:06:24 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Wed, 18 Oct 2023 22:09:07 +0800 From: Kefeng Wang To: Andrew Morton CC: , , , , , Zi Yan , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Kefeng Wang Subject: [PATCH v3 14/19] mm: migrate: use folio_xchg_last_cpupid() in folio_migrate_flags() Date: Wed, 18 Oct 2023 22:08:01 +0800 Message-ID: <20231018140806.2783514-15-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231018140806.2783514-1-wangkefeng.wang@huawei.com> References: <20231018140806.2783514-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 325161A0028 X-Stat-Signature: pezuoyfbydj3g33d94j8h9az3g567565 X-HE-Tag: 1697638150-357804 X-HE-Meta: U2FsdGVkX18whbFzMNV0fAVF9/NhMCE8fniiOm7JyFQW00eI9/e1vTNLnmUKeRwaq+El/2sV3TCFjs7NiNm/FZF0K41zormCBOAsr/SOnSYwH2nu6L5JxGuva8iVIBKfPmFXU/bLSxUxAT7TjMGtPSQ12W79hnP4br+8EWWcSRIEF48iHXcxTXu8plvGwOzlvM2YBDbAWYvOCkGoNjsGFsgHj0iwi9ZkWdWnRibFHh1EMYOOis2/NOuqkYiJZutd6ombkDvzpiVViUPNpqRGzTmvzIAw6FoYdeHI7AMpeEqDWubbMR8W+7SmWCsXLja9+OAl3JBRLpHipYSk65v6a3kEh9BHmSPaNrEMK2NRxZ4/OVlFOMaQHaIJFazxu3McHgW06pKVYoe3sJbTHe4+QwFnvy7Re3C5bHhKDjvkwe6PRb1FoER0iiTsfd3lnRKbXC2GrpgNlMvXY4f632Dti9dj5ixyWS6g1d6m6KTGfyQfB0kQxloVmh38dItiTebTdv92st87XHszMcslz3D4WXA3NxnjLAkyUDmyWwhyG7gKmvLgXhMeNbzH/3GErSeznPDy/uYcCJaNgdJRD1347PSihAWdxTaAeyEAJKNMe+au22KZd8IYEzToAnEoEtK+BcgIWPAP4g5+J4t+rQBz1C3hyrnqOK43cRO5AxWe4XWxTs/+4lk2kHQ9twmxK6DRB4z2mDiKGxKDzS7A40fiaoiSXP9DP6LwOx7+wEcmvUnizwEMZbm17/am7pxqJ1CkQRXtbkV2ENV5Ukg98yJAP4XUOJDXQUz5nQ218GtQlxmlEVcSvmgDD4BWmQTXb3JlbhjeCywON5Ld3utpPsgm/0dzdxaRD22gdVrp46PMhW4mVCwVWYOFf7LTX7NejO9bsNV2ImOQfLXtpkvpF/Ur2Do4ksnoWGyuckAVoDmZQkEpSdbsd0Fzbu0K6GMYCKB/rDkeQYAR79w4WFOpCsx Mk7MmDp0 kPDve89nBHV72vGZr6BWPNKqL9tfk81qELE2SDHIot2oAGDjxMFZ8gRmL6VHcR9Boa1S9DSPWi05km47th8x5yjL6n0h29YF9kOSXNl/uxVzY0pGAbJCrCWRN6MbT9qENe8Di8zK8+Dh40ehXHmZDprLD2+/tLhPkTAvy+jOET0p3QQYY5ZxT9yUu80/fj6XSq32k X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Convert to use folio_xchg_last_cpupid() in folio_migrate_flags(), also directly use folio_nid() instead of page_to_nid(&folio->page). Signed-off-by: Kefeng Wang --- mm/migrate.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/mm/migrate.c b/mm/migrate.c index 5348827bd958..821c42d61ed0 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -588,20 +588,20 @@ void folio_migrate_flags(struct folio *newfolio, struct folio *folio) * Copy NUMA information to the new page, to prevent over-eager * future migrations of this same page. */ - cpupid = page_cpupid_xchg_last(&folio->page, -1); + cpupid = folio_xchg_last_cpupid(folio, -1); /* * For memory tiering mode, when migrate between slow and fast * memory node, reset cpupid, because that is used to record * page access time in slow memory node. */ if (sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING) { - bool f_toptier = node_is_toptier(page_to_nid(&folio->page)); - bool t_toptier = node_is_toptier(page_to_nid(&newfolio->page)); + bool f_toptier = node_is_toptier(folio_nid(folio)); + bool t_toptier = node_is_toptier(folio_nid(newfolio)); if (f_toptier != t_toptier) cpupid = -1; } - page_cpupid_xchg_last(&newfolio->page, cpupid); + folio_xchg_last_cpupid(newfolio, cpupid); folio_migrate_ksm(newfolio, folio); /* From patchwork Wed Oct 18 14:08:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13427135 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1094ACDB483 for ; Wed, 18 Oct 2023 14:09:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9340380016; Wed, 18 Oct 2023 10:09:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8E5DC6B0203; Wed, 18 Oct 2023 10:09:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7B1F280017; Wed, 18 Oct 2023 10:09:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 5F0EF6B0203 for ; Wed, 18 Oct 2023 10:09:29 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 3E9C81601FB for ; Wed, 18 Oct 2023 14:09:29 +0000 (UTC) X-FDA: 81358764858.25.0E70648 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by imf30.hostedemail.com (Postfix) with ESMTP id C624A80017 for ; Wed, 18 Oct 2023 14:09:26 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf30.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1697638167; a=rsa-sha256; cv=none; b=DikffGrJQBscZN09ak+yfPSEd9HCthor3KEn2SveDLwKGVnMYaUgqxgrNJH4dPGq1vjKjq bGJIaHKD/1d979KBu0cNGcUPgycsSs4rfpmySRUmyOuQYSdvMm9+1rCABZC6uNkZtkB51G 9F3xTNE1jci5w7SjR33X90qnW5vxxgY= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf30.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1697638167; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=J7hLCzxP+vfH5k3zHwo4He37UOBmYzrfi4mtBVqDm2w=; b=wbVwmCijcs+7E1FwE+van7TtPM5AfMercDrb8BCL4gXQuVbWYMjoUzxEhxubj2ww9hil73 hvwaC0jzafqj0OwtJtbDqW46UK0+PpbN1oGEmmCdVzUNpnptL3QMqOeJ18/64KeoE4U4eu BeL8kzPKsy2zi7eLgALyC7tiF2g9naM= Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.57]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4S9Xfr71CvzCrSJ; Wed, 18 Oct 2023 22:05:08 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Wed, 18 Oct 2023 22:09:07 +0800 From: Kefeng Wang To: Andrew Morton CC: , , , , , Zi Yan , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Kefeng Wang Subject: [PATCH v3 15/19] mm: huge_memory: use folio_xchg_last_cpupid() in __split_huge_page_tail() Date: Wed, 18 Oct 2023 22:08:02 +0800 Message-ID: <20231018140806.2783514-16-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231018140806.2783514-1-wangkefeng.wang@huawei.com> References: <20231018140806.2783514-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: C624A80017 X-Stat-Signature: tceehhx6cgf1gdp54wtwmbofyt5q3csn X-Rspam-User: X-HE-Tag: 1697638166-167882 X-HE-Meta: U2FsdGVkX186e9hioEkaO3TWEuRA63wAwP5jks/ep4YmAG0e4weGHb7K8uBOW9Uxb/zqbmw0exzkwsdbyahFZfO37qkItyGK8mNzGOstpiTVUhmIlPBE9an1NGfJmrVdQH+nvdoQ3maDhs/hTsi6S2ppfir8XUqjIPEbLfiAVM50Smb8oDqjt8SxszVbB8sNflcIKviGpbdZHdwFWI6LM7vGSODX5igOaVjjBvO9alOiSZx4xkHub67jgGMf8xP1qsemxoSBFMcaxCW8k6Lcm0yEvcGs46vM8kFQhQVFNsqx3MycUFhYK9B8wzYvbPGISWaKeQWWb8J9NNxHbusHfzmcEGewB9QgYV8RQzCgXjoTsNqP/dqOStW6QQ0I+hdwV9qqyEC9RrIfivTqJpBFytoCfWD5ERkTJl+R1Xj+a1Py3EZdqvZjGoJzv8byLAnS3i4Y/TtBz/VeC2+XT8OzT9vnlzVekrkDi4TnMSLIXfpDms6KDMiHW1oxNkdcTA5/Gk6bY4ipQNYNLFc9LvdJpRNdwBcvHix1wn+YCz0ARZz+Tyfi5zajuvgXHLAppAOVeIMF223FrrhWov+EDCpCikGtxVRCFuh1uFeK1HjC8Xt6Mzyv9ngODtHE0oqEbh+fFG/VzE66MBf65E8XUAyCRGJB7lTRcrsREN6aBoF7vpL4pVhXwmYK9rj65p3GLWo9emiP6lZ+zH12mlLihHNVa7eIBBtUuDPkUydRbxbD8dfozf7YducqELrSuJTMN7CYcI294YFA8kGFL0BYo5f6BmgdNndBcLQgZq7pCyqpwmx6RBIhd9EtfLNisnSgV6tG2kQtrUXVmjNK8PEZ4KixgNm56I/F6a8FH1hEu5mkZYY99A50JzDc+HQLthCCkGTuD35SrGtaBykzCOdejjxACtKYdwF7q8eFR+xmCF7UeO0mjAGi6oKmaBNd5qwvNVaGqYGk1xiEKxd2nC1J1BR HlDU2k++ I99S6rspG9gZHXgWMnUvgW1xiZpwsa4t/XpYJZONxFA+Ml77Xq2qU7PvqGqwHdFdwvImL4jM3aFWo0kA/p2538lqondzkb1yUOGEiJf9NlcZuyDxsCc2jtCU5zq6W3jUEAmGkrxUGTkHHq0Dhlxch8SVJiwQGVaS6/vjW8xN+5tFt0MJvstgB+p4mum4GHdqNzrQb73A241aznmM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Convert to use folio_xchg_last_cpupid() in __split_huge_page_tail(). Signed-off-by: Kefeng Wang --- mm/huge_memory.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index f01f345141da..f31f02472396 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2515,7 +2515,7 @@ static void __split_huge_page_tail(struct folio *folio, int tail, if (page_is_idle(head)) set_page_idle(page_tail); - page_cpupid_xchg_last(page_tail, folio_last_cpupid(folio)); + folio_xchg_last_cpupid(new_folio, folio_last_cpupid(folio)); /* * always add to the tail because some iterators expect new From patchwork Wed Oct 18 14:08:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13427129 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 93013CDB482 for ; Wed, 18 Oct 2023 14:09:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 935236B01A7; Wed, 18 Oct 2023 10:09:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8C02F80015; Wed, 18 Oct 2023 10:09:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7126D80014; Wed, 18 Oct 2023 10:09:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 58B156B01A7 for ; Wed, 18 Oct 2023 10:09:15 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 3CE0B1A025B for ; Wed, 18 Oct 2023 14:09:15 +0000 (UTC) X-FDA: 81358764270.10.3CB7953 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by imf17.hostedemail.com (Postfix) with ESMTP id A73A040025 for ; Wed, 18 Oct 2023 14:09:12 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf17.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1697638153; a=rsa-sha256; cv=none; b=VbYHJ9FCGnWBUc3dotCf2Ix2knTu9KvQQEHyFr1+rET3CSbBiMVSBayDXVDvVXPEvJGD7K YVDPw5pN0s9NFtzlNJAbQr6mRHPY+fdWRRNknxpwfReoWYff9VIOHk8iqB6O7R93cB8r/S j61bxtmJtHMZ9zPBzQc0u+48UCv1Tlk= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf17.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1697638153; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MUSi80tZ4yNk8CECWcRvMJAQ5YmLm4C/rY6QQ6q7lE8=; b=jrVqjG5AZkNi+nzV1JDMIggRQ8GvPvJTqO++67SLBRGkXOzb75Xuo4gT/zqnllfXbZaGrn Y64Vifl9sdXWuriIpEHqYSZSwziNqajWUxMsjGVsCCfGdBLURKkGyP4cJPpme8IxA9Wqqq AK467MzMjG4AOKdBFVgpQs77vLBkVtk= Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.56]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4S9Xf01sH2zvQBS; Wed, 18 Oct 2023 22:04:24 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Wed, 18 Oct 2023 22:09:08 +0800 From: Kefeng Wang To: Andrew Morton CC: , , , , , Zi Yan , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Kefeng Wang Subject: [PATCH v3 16/19] mm: make finish_mkwrite_fault() static Date: Wed, 18 Oct 2023 22:08:03 +0800 Message-ID: <20231018140806.2783514-17-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231018140806.2783514-1-wangkefeng.wang@huawei.com> References: <20231018140806.2783514-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: A73A040025 X-Stat-Signature: fcj4yx1yekqyg6f6c41zwg8kg57em4s6 X-Rspam-User: X-HE-Tag: 1697638152-362698 X-HE-Meta: U2FsdGVkX1/bGvYSkb+f+XZgK+rUdZ0WUj3B0+cqMCgQ+yuSj9yqXnSi3GjOV+UrdxLyAD9rNLDncs1vbSsIi4qaHn1LeWBY+uLY8KmxmJHHbvQaflZ7uNoHDshWEdgSD2+9tE5YyeT6wGrAp+e37zY5ol22seyCeaseJFdrFlQxKQxwUNLDmp5Frojix6DS0SrHfK459mC0dBm7RxJlTFKq7TegaWfbmlEagzoP2aWYsuWTCtG1lw1IGJF5EqheXg8BdBsWxgM1W9xuyB46vxiS0mL05PxZklBlyEHquGVfTKPVOg8wyM4RQ3dctTR5hd0WBPbifegFCrp2DgVEv7Cy4JD3EAGd64IA1sFT1V3B7xQkprI7TIi+tk8HHi9kSCkB2Nej/d92xqdPdiY6imLt8IYNcEPvt1epOETwB6ozVXzVgG70qyblFI8ggBrX66JA+AeOHRdtrMs7l24Dq3o15Ez0CuDHDXnBviwkeg7U8z9ADQMPXPMSQUfjHfZlv9Xsbz+zkrxAdR/s3LP7SMRotqHdVSZz6WcgyGfOIvS9tANeJDBH/8+evnFDtv+dHsNfww8TgLHoCPOVQ+Fhibgta5VaieYEPWXuhiEDKdFto9namD/w3bVJ+jxuWGJbS4qhkoYR0yC5/C07nsAzxr8ZRYDe3ZJ2uE9oiqX/73SbyEUoX97LPrPQcHljYC6LZqYAZbnfXYW1UuRNIRaocqhTZYeNNfI7VZwqz6pzjws6D00UdbS5UaTXow2MQHQ+b+8u4jO6MQJUYQHqaMdpMZdGeJWVAGDvfGjoQp+lNopfiLTrwIPWxnBjHSiwv6R4Xvf5Xt4cM/YtzTHCHVLjH94Bdhy+BdW9RQtUD37zp+A7iPpGCkkiWlY0/A5U9FJcF4IZ/0ZJHiD37JHtKW2sn6xnuJa6RbVnef3h5yij46/1iltQORgj2UFzoqOofBHhkgdVykhbtiiG1VfWExS numD8ZxX kkKe4IxwyuopPZa6TcZRPKsqiJQv2C/SfuzNVwFGttCy1UbgWXZ+IZiWF1qtEMNoF+TbgsFaOl7NT0xkYdE7wXOMRvDGv39YvkxCcDAEPCkNPQOgbVBgRAk1RPrgRGjqg+ap8BfkE/NDuh0xD1NJe13bMg6MiaADtLl9DpFZ8EQulXV4/NswgvtcTMrZumw/mt5ly+IieYgRT50DNXvntp5WVDVbNlEYzbHsjZOJLHBDTqf4uAPP+0VYdg/Wua2ixYkHC5JDjp2A6SpFxh+jPf1DySm44knnihzYm X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Make finish_mkwrite_fault static since it is not used outside of memory.c. Signed-off-by: Kefeng Wang --- include/linux/mm.h | 1 - mm/memory.c | 2 +- 2 files changed, 1 insertion(+), 2 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 8229137e093b..70eae2e7d5e5 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1346,7 +1346,6 @@ void set_pte_range(struct vm_fault *vmf, struct folio *folio, struct page *page, unsigned int nr, unsigned long addr); vm_fault_t finish_fault(struct vm_fault *vmf); -vm_fault_t finish_mkwrite_fault(struct vm_fault *vmf); #endif /* diff --git a/mm/memory.c b/mm/memory.c index a1cf25a3ff16..b6cc24257683 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3272,7 +3272,7 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf) * Return: %0 on success, %VM_FAULT_NOPAGE when PTE got changed before * we acquired PTE lock. */ -vm_fault_t finish_mkwrite_fault(struct vm_fault *vmf) +static vm_fault_t finish_mkwrite_fault(struct vm_fault *vmf) { WARN_ON_ONCE(!(vmf->vma->vm_flags & VM_SHARED)); vmf->pte = pte_offset_map_lock(vmf->vma->vm_mm, vmf->pmd, vmf->address, From patchwork Wed Oct 18 14:08:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13427154 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 35157CDB482 for ; Wed, 18 Oct 2023 14:09:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 20B7180017; Wed, 18 Oct 2023 10:09:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1C4A06B0206; Wed, 18 Oct 2023 10:09:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 031A180017; Wed, 18 Oct 2023 10:09:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id E46E06B0205 for ; Wed, 18 Oct 2023 10:09:32 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id BEE961CBC28 for ; Wed, 18 Oct 2023 14:09:32 +0000 (UTC) X-FDA: 81358764984.05.10A1928 Received: from szxga08-in.huawei.com (szxga08-in.huawei.com [45.249.212.255]) by imf07.hostedemail.com (Postfix) with ESMTP id 13BA44001F for ; Wed, 18 Oct 2023 14:09:29 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf07.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.255 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1697638170; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Ydl5R/Os1oCYcNiMi7C125hjKrYLYWYSyaLxBm8zeC0=; b=tR6zJ0y55rv3qfAQm4etf9TwrhHEHHGcSoo0ninht7u7MiWHNytfEqx3nKLI1l2jOaTSoc GqaqjfTO1CEDurotkUWT4f+9O7h7r3yXEBYtma6G8s0QuF1/kf8G+XcB7HNiyRrCoBDeeO mE1vuEcjxZ9CgLw22Duh3TWp36B0m1I= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf07.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.255 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1697638170; a=rsa-sha256; cv=none; b=Lssp5Bhos3r/JV18bwCAzjOBXcuTqbPssj1cGwGL0a904RWvo9oP0xkqkEtxGQrwRRIJH7 Ga/OWgKnp/F/TD/kIt71VX3RzBTszu+5tsRXkiq9GY+8Kdo48Be4nEJbIXY+AOO7/5uY9B XyB7akycwkiK4i/ckA42uO8JrWi7jW0= Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.56]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4S9XhL5QYkz15NX8; Wed, 18 Oct 2023 22:06:26 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Wed, 18 Oct 2023 22:09:09 +0800 From: Kefeng Wang To: Andrew Morton CC: , , , , , Zi Yan , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Kefeng Wang Subject: [PATCH v3 17/19] mm: convert wp_page_reuse() and finish_mkwrite_fault() to take a folio Date: Wed, 18 Oct 2023 22:08:04 +0800 Message-ID: <20231018140806.2783514-18-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231018140806.2783514-1-wangkefeng.wang@huawei.com> References: <20231018140806.2783514-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 13BA44001F X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: f6zc7hmy9a1ce3fmhwticmfjumhea5yw X-HE-Tag: 1697638169-414265 X-HE-Meta: U2FsdGVkX19ILv5iMJ+5t7dFJjkG/2l9PLmzHHZlDj+D5FGWuUUtT1f/YSn1Z3I9lZZLrzpCDQL/hoV4P3hrHf8xMDE2WOGP9ICLTXdtS/ov9nRuR6/L9hLlvRMcEOcbgqGgxeFwP3S/L6bS1wzfFqDQ7eVWG9vVRZwxOpuYAHnRoANJaKmdNfx5ufmQiMUpwTIsn5PIQ62pdcracfN+4w2PIgk9OGR4otbY6iNx2DeDAaKTFvttyldOAysilz0qEYjeRZebbN5BzrTWZACpYBFtq9BHDvPnqJW4S1iIB+4jwQFx7z8stw9LK6OmJbm2Cf6c5fxZ5dZNjWp8izDO3UGOe6yJQTM7Ft+uITdZncse56xrGr+ju1N/RIqej4ZOuUbGYXLAUjnRDEpwccDiAEJiHUqCWaEhZmI7cuwIrFSoTEA8F65shg0CFjHGRI+fOZqqIEQyB4Pd53bzBv4qu/SVD4pgHIIKfNw2oDTFhZhMq3QczfqEqDdYHJdO+D2qAO3vgOyjKezDtAoJKolpp6Q0WapgGgg43T8wIQpApAQsjHFgUGG4HbgLjol4Bb3Oa1dP3sZUIncJAqJTcPAWs/lLlpWcNxmy+6OuKH8+XOchdF7rnkWVbD37++QqvnMav4H7TVmzF4hZeId5/TxcAj+w2/KpxJjIElfSXM8nNWdk2dVzIEo7l421X4r/GTmqHwr6veVsa94hhDEPSlzHvh20MRO2g0mhwmiPPBcJinz/TsDVBe91lYLYm7PjYN6YdiwJ5Yh7gtSBBO/HlfDyVzH9Uz6hInzOvkpnjzdo1vEXRTI8SVfBy4fem66wSjFyBOI9McLsDtPhN+ktu6YFXKG7edD79499zbwn4uxJ60gC5qGv4H8j/AOubr0dnaY3I1MPOlOXuoHSJY6uOmSB46GwcIuaYBQ0rV3h8D9EvJ8kqQLIOHUxjtU0oezQ+NMcvpkfrgOvbnxnloIjx7H wibZbzF9 pN5JSzQnmPRgXXXUZALJv/QLhiFbk+r7juVNSYm3DaHmfoHpS05Og3SkIJeSnEkcrdGtc5ujMTTUgs4zOr1j9TuNmvN39BqXiEf+jOqHOZpSLPagKPdEAH6gbv/EJByldQ9cIQnedTGxwOauDhilx9lHII8yi+d9JdysaP+8HY5XNarF9QFzxVSXNnFhsOlpMsl/F X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Saves one compound_head() call, also in preparation for page_cpupid_xchg_last() conversion. Signed-off-by: Kefeng Wang --- mm/memory.c | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index b6cc24257683..037da118f1d1 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3018,7 +3018,7 @@ static vm_fault_t fault_dirty_shared_page(struct vm_fault *vmf) * case, all we need to do here is to mark the page as writable and update * any related book-keeping. */ -static inline void wp_page_reuse(struct vm_fault *vmf) +static inline void wp_page_reuse(struct vm_fault *vmf, struct folio *folio) __releases(vmf->ptl) { struct vm_area_struct *vma = vmf->vma; @@ -3026,7 +3026,7 @@ static inline void wp_page_reuse(struct vm_fault *vmf) pte_t entry; VM_BUG_ON(!(vmf->flags & FAULT_FLAG_WRITE)); - VM_BUG_ON(page && PageAnon(page) && !PageAnonExclusive(page)); + VM_BUG_ON(folio && folio_test_anon(folio) && !PageAnonExclusive(page)); /* * Clear the pages cpupid information as the existing @@ -3261,6 +3261,7 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf) * writeable once the page is prepared * * @vmf: structure describing the fault + * @folio: the folio of vmf->page * * This function handles all that is needed to finish a write page fault in a * shared mapping due to PTE being read-only once the mapped page is prepared. @@ -3272,7 +3273,7 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf) * Return: %0 on success, %VM_FAULT_NOPAGE when PTE got changed before * we acquired PTE lock. */ -static vm_fault_t finish_mkwrite_fault(struct vm_fault *vmf) +static vm_fault_t finish_mkwrite_fault(struct vm_fault *vmf, struct folio *folio) { WARN_ON_ONCE(!(vmf->vma->vm_flags & VM_SHARED)); vmf->pte = pte_offset_map_lock(vmf->vma->vm_mm, vmf->pmd, vmf->address, @@ -3288,7 +3289,7 @@ static vm_fault_t finish_mkwrite_fault(struct vm_fault *vmf) pte_unmap_unlock(vmf->pte, vmf->ptl); return VM_FAULT_NOPAGE; } - wp_page_reuse(vmf); + wp_page_reuse(vmf, folio); return 0; } @@ -3312,9 +3313,9 @@ static vm_fault_t wp_pfn_shared(struct vm_fault *vmf) ret = vma->vm_ops->pfn_mkwrite(vmf); if (ret & (VM_FAULT_ERROR | VM_FAULT_NOPAGE)) return ret; - return finish_mkwrite_fault(vmf); + return finish_mkwrite_fault(vmf, NULL); } - wp_page_reuse(vmf); + wp_page_reuse(vmf, NULL); return 0; } @@ -3342,14 +3343,14 @@ static vm_fault_t wp_page_shared(struct vm_fault *vmf, struct folio *folio) folio_put(folio); return tmp; } - tmp = finish_mkwrite_fault(vmf); + tmp = finish_mkwrite_fault(vmf, folio); if (unlikely(tmp & (VM_FAULT_ERROR | VM_FAULT_NOPAGE))) { folio_unlock(folio); folio_put(folio); return tmp; } } else { - wp_page_reuse(vmf); + wp_page_reuse(vmf, folio); folio_lock(folio); } ret |= fault_dirty_shared_page(vmf); @@ -3494,7 +3495,7 @@ static vm_fault_t do_wp_page(struct vm_fault *vmf) pte_unmap_unlock(vmf->pte, vmf->ptl); return 0; } - wp_page_reuse(vmf); + wp_page_reuse(vmf, folio); return 0; } /* From patchwork Wed Oct 18 14:08:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13427130 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 53B7ECDB484 for ; Wed, 18 Oct 2023 14:09:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4A07180014; Wed, 18 Oct 2023 10:09:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 402246B01A9; Wed, 18 Oct 2023 10:09:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2F0D280014; Wed, 18 Oct 2023 10:09:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 1D6336B01A8 for ; Wed, 18 Oct 2023 10:09:19 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id E4D3740D20 for ; Wed, 18 Oct 2023 14:09:18 +0000 (UTC) X-FDA: 81358764396.16.FAD4A23 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by imf10.hostedemail.com (Postfix) with ESMTP id 53C26C000C for ; Wed, 18 Oct 2023 14:09:14 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf10.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1697638157; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=z6mhHbia0tgVbcgOllnNbvBMUqnXaQ9x0wWBWtbmQ/I=; b=xsV0ZE7aq8PJya1sXZjn1ggjasIGjJM1qMAJ47bRzMMtvDR0lQE9d9gZbnWaBbZoHJib5m xT0l+DsjBYNoUb4gtG5JYpi/mAupxdZREYbx/s3y0qC3TfX0HfHeFXYV/RbIdRmLrAaR3L hpa5Q5TQMpiaAJtFkIp189WhbdPcn2o= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf10.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1697638157; a=rsa-sha256; cv=none; b=xO8PjLoGd89uK9UaZWMxlqxqRUk3oBt1V+dmYe9RJL35Pky5wxiADdWQsuxBrtFs/vlOrF XmjiOsAnx3KfL0rR8Xc5TZPCyljd6QOt+r0rxMK+PEwvDuKiw0ExQpPL34JaynnDnvXLef dYjCN25RhvdfQiGc4JvzO2xZkufmCO4= Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.54]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4S9XhL4X7YzrTQF; Wed, 18 Oct 2023 22:06:26 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Wed, 18 Oct 2023 22:09:09 +0800 From: Kefeng Wang To: Andrew Morton CC: , , , , , Zi Yan , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Kefeng Wang Subject: [PATCH v3 18/19] mm: use folio_xchg_last_cpupid() in wp_page_reuse() Date: Wed, 18 Oct 2023 22:08:05 +0800 Message-ID: <20231018140806.2783514-19-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231018140806.2783514-1-wangkefeng.wang@huawei.com> References: <20231018140806.2783514-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Rspam-User: X-Stat-Signature: nqref839ir776km1ms7xjixmmmrseyd4 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 53C26C000C X-HE-Tag: 1697638154-277723 X-HE-Meta: U2FsdGVkX19473okdeEpnJWMA9ahC9E+Meq6RxhjL5gDCqNtPtiXtIIG6uxgsBbjtbxSKcI3FwHFIclYR503B+Xe5+qbGDTS1z0VnpyM7C3EjzN2XFMVT7VlyC6WvPQRX7yxfZB+O0U8TipQb9MfezYA4VY/H6VeCinFRAs/US7j2UskRmK0ypP1zVCrEKWZmIfzfTElx9gOpb+qCHWO/qJDzCtWaXjrDK4TzW760EA4MiJ4xcdimD8OplbnBVsLyCJnjwVSs1Bg+ryFfb1/0FTB2kCYd1dg7MRtGvd3X5QlIPnKzrloXSSihogwN7CkYg6HU5uPIMO+P8T0W9BNJ/dMT6Xwude4mu6ZbA1xvygyerRYomzy9CDiRrMxx73V9A/Ivy7ZRCol2rE2mqJ7/xfRSAOlPC9TJNClIor5GwmbVsOCiLfa+tNSSAEbRZ+XPi/qJenayEcYL7YZdgkwULY1NWHkgFQLgai1R+eJXYvdXU3L//3N5EiOdZ+wB0FK7Xsk1FRtL/cYq2CY8vJe7vsVosrnzuxXFY7hc7KbihqePxxqQXplzZ9/R+pOwbcfvo6PrEd4pt9X8niEjFanhUk2nIfdzEhfruttLiB1zcO9kOU73GFyx1l34Z9a2qWmY72jiwPZ15btTeYRiYaQax/TBX9esompDFulLyLV49N9U1bBsUOQroO3KhhvA6hlR2YMFOMd7nNTW7Al0vrDJ93nq8UyaHky2f8RZTc2yete72X2w43pbWmu33yzkiFfFBFDFh70k9k7/MmS46LrUJGFl7LvvuX4/cxI1cCmhCcRCHGfQxIW8tLz5chlH4e5qlliJoPPitB8e0IsRadvcf01QhXBhwlubdRVeIzHV2hHQRkef7xxKy6ZwOZH7Pm5uMSHrzYVEFi+OkOgnJ7v6p20JgjRH4bXKNf/LuwxjxEkMqtxxJanWSdowBxS4B0BxxLNmb5tHP9o9XLniK2 HAIMBY2R nndFcKpQhvEz6+wDDNsypdod2e8qGApFCjZ164u9PAmgi9b6FAG+rd9XAWmkFI+zE1tam4MJ/UD5DH7JeTM/Q1gMSCX7JXI3TWWUi6EriaL1CxdJ1wQpeipcuXnjpQYKQ2dJEVHjL958t5rVHbT+nFRr+AGTqLeIunr18p7Yw5LEwKJkpSRFUbw4w9cNJe8HfjQfoZgU9tA5C/DZ/PvX6+oDUCgNFNVrhOjyiPkh1uippnzLvZP7tAL1+lYb88Cwzu5pAAxP41DKJR6/tUo+ADZJdRrslXLJ8Q/gW X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Convert to use folio_xchg_last_cpupid() in wp_page_reuse(), and remove page variable. Since now only normal and PMD-mapped page is handled by numa balancing, it's enough to only update the entire folio's last cpupid. Signed-off-by: Kefeng Wang --- mm/memory.c | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 037da118f1d1..4f0dfbd5e4bf 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3022,19 +3022,20 @@ static inline void wp_page_reuse(struct vm_fault *vmf, struct folio *folio) __releases(vmf->ptl) { struct vm_area_struct *vma = vmf->vma; - struct page *page = vmf->page; pte_t entry; VM_BUG_ON(!(vmf->flags & FAULT_FLAG_WRITE)); - VM_BUG_ON(folio && folio_test_anon(folio) && !PageAnonExclusive(page)); - /* - * Clear the pages cpupid information as the existing - * information potentially belongs to a now completely - * unrelated process. - */ - if (page) - page_cpupid_xchg_last(page, (1 << LAST_CPUPID_SHIFT) - 1); + if (folio) { + VM_BUG_ON(folio_test_anon(folio) && + !PageAnonExclusive(vmf->page)); + /* + * Clear the folio's cpupid information as the existing + * information potentially belongs to a now completely + * unrelated process. + */ + folio_xchg_last_cpupid(folio, (1 << LAST_CPUPID_SHIFT) - 1); + } flush_cache_page(vma, vmf->address, pte_pfn(vmf->orig_pte)); entry = pte_mkyoung(vmf->orig_pte); From patchwork Wed Oct 18 14:08:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13427173 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F292BCDB482 for ; Wed, 18 Oct 2023 14:28:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8CA8080017; Wed, 18 Oct 2023 10:28:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 87B5E8D0016; Wed, 18 Oct 2023 10:28:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 768E480017; Wed, 18 Oct 2023 10:28:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 621378D0016 for ; Wed, 18 Oct 2023 10:28:28 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 2A1E21CBBD0 for ; Wed, 18 Oct 2023 14:28:28 +0000 (UTC) X-FDA: 81358812696.26.8E31699 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by imf04.hostedemail.com (Postfix) with ESMTP id 67C2040010 for ; Wed, 18 Oct 2023 14:28:23 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf04.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1697639305; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KMKcdjwd08Xk8ND6ULlLoLx+XBYNhLeLKyS742C9v9I=; b=KQuSJCu2GYgH0O9vuio1DYHC46CKsV8/mwJ66qaQVy/38tluZDAkhxjN48VHwG/1WmNUtH WZoIV8aStsqQpZFYtuRuYPJGlm/Q80sBlIJ2QrlK2ZiahhkxUfiS2QSK+hMQkS6/Oj1sbh HZ1BLYWN/5cBOBqzszZGy6WZBuHFvcI= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf04.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1697639305; a=rsa-sha256; cv=none; b=IFJpJ+QEsWPERVYhZEfvFf4xz2NBN4tFEEMZP9AwE+A30RE8STBw7pqklpJlfv6PcumAMx 4EC+t1ruk4HTabMxKN6YSpqKMKPRCk4o8MoqiNyT8y9SY4eZVrrcz2qnNymRKvQeiKXlxJ kZpgb+C3SSarQfRDCXJznT6LnjXBRHI= Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.53]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4S9XgG2pm6zRt5M; Wed, 18 Oct 2023 22:05:30 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Wed, 18 Oct 2023 22:09:10 +0800 From: Kefeng Wang To: Andrew Morton CC: , , , , , Zi Yan , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Kefeng Wang Subject: [PATCH v3 19/19] mm: remove page_cpupid_xchg_last() Date: Wed, 18 Oct 2023 22:08:06 +0800 Message-ID: <20231018140806.2783514-20-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231018140806.2783514-1-wangkefeng.wang@huawei.com> References: <20231018140806.2783514-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 67C2040010 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: t3icra7pyaheyh7pki5ycc9s17ron8gp X-HE-Tag: 1697639303-719073 X-HE-Meta: U2FsdGVkX1/RdsSiXclF7GBpgF22IzdudS3JzwzJtKffUIkSDpqmrY0S4OvyI4/0xxk/sHgJnHJ24kz6Tn6OoepKBkXaLX9fPiNmpP2rf57BZZ/wW5fxQc+3q7u6tCi0qVEchYr1y8DDZvv8ynGUD0aDIHL+RnYrD/7h46a+e5O/pBOtLwL9QIKNk7b9FvbfC+Tqu1Cu6Y0LCAEs1eBJi50RNcmbLIKPi+wK6/5TbZz/vT3ZE0Mgj66w31LsoogHZ6Z/OdA+i2ecvwodxvGYcDa1KuCMo1Y+sRUk+OG8nrhRjfRR2HyqOAuzB5R0r+VvAvsNsLy+SwJ6gwR1asA15zUFqc6fT/3RjMew01KgQCxcUlccL/howyxUKfKgjE6bBcMOhXID+C5GbgggK/oawGga+jTOqSNTPHCix+iTMGOD+ilyOl2Qlk+nBYeaL26PDoDmjJzwSbm6NE1Vjnid/hx/AvThsouD+yQcd2+sfjTH+xeEjM4yC8EadZMIInoMujrmveWWKlc1M4ZdORaZMIhcv/1Jn9IwKbwhtZJv9Vpx1Uyq3pjrGdzV8WHjDta9XXp060F7tsBecmHXAYg4UBwo/L2ovHnGpOpcGibdwEX4co6UxK9EvrX2YBS1cDYySfEg0hWVqyVIQg4rIZBga12ATkz7n16B69ajIhIdcpbV3EEOWaWr3z9229FN69m8uq0KxJRbclb/rwl/pXMYc74V+idtDeWPQ5MlpSOLaj6hcsCBsC1M8IPaPtVbdub0fDyDxEvqFd07fATegsIIayyo+gVH5dr7asu1FqGngIwF90w4nPS0cZNGg5iLGqFnZ1440/TSOffZWL8l/R0JN3hEw8pq6Z3FNHr8GLXsHRCdGcUVMfRmLQ/yt6VOF9Hoy+/S1Psk0flWNibIV91P3PhKFJSM6b4uTuj/zvJKmkW0w0rOi9pEoSPlmngvpWxH+bkM+B03RkOK04yRlm/ uKnhgO8h m4GFNuNtOfKgX/v9t75aTywplmKe/i4w4uZeSRRYgRlmu7d0K9BBWbZFjbNTzLmiu5WMW2Rgny+UZrbIHBXdD5HPkJs1kkW1emxAzW3sa5uHPFzeKV7TlESFzanEogDt2GZy4zWDzpf24+GRxjn2KRSmM9qrFSQBKLvONBgUhfMh+yQDxHc+QB0ZBbZ3hhAWvU1Js X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Since all calls use folio_xchg_last_cpupid(), remove page_cpupid_xchg_last(). Signed-off-by: Kefeng Wang --- include/linux/mm.h | 19 +++++++------------ mm/mmzone.c | 6 +++--- 2 files changed, 10 insertions(+), 15 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 70eae2e7d5e5..287d52ace444 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1694,9 +1694,9 @@ static inline bool __cpupid_match_pid(pid_t task_pid, int cpupid) #define cpupid_match_pid(task, cpupid) __cpupid_match_pid(task->pid, cpupid) #ifdef LAST_CPUPID_NOT_IN_PAGE_FLAGS -static inline int page_cpupid_xchg_last(struct page *page, int cpupid) +static inline int folio_xchg_last_cpupid(struct folio *folio, int cpupid) { - return xchg(&page->_last_cpupid, cpupid & LAST_CPUPID_MASK); + return xchg(&folio->_last_cpupid, cpupid & LAST_CPUPID_MASK); } static inline int folio_last_cpupid(struct folio *folio) @@ -1713,7 +1713,7 @@ static inline int folio_last_cpupid(struct folio *folio) return (folio->flags >> LAST_CPUPID_PGSHIFT) & LAST_CPUPID_MASK; } -extern int page_cpupid_xchg_last(struct page *page, int cpupid); +int folio_xchg_last_cpupid(struct folio *folio, int cpupid); static inline void page_cpupid_reset_last(struct page *page) { @@ -1725,8 +1725,8 @@ static inline int folio_xchg_access_time(struct folio *folio, int time) { int last_time; - last_time = page_cpupid_xchg_last(&folio->page, - time >> PAGE_ACCESS_TIME_BUCKETS); + last_time = folio_xchg_last_cpupid(folio, + time >> PAGE_ACCESS_TIME_BUCKETS); return last_time << PAGE_ACCESS_TIME_BUCKETS; } @@ -1740,9 +1740,9 @@ static inline void vma_set_access_pid_bit(struct vm_area_struct *vma) } } #else /* !CONFIG_NUMA_BALANCING */ -static inline int page_cpupid_xchg_last(struct page *page, int cpupid) +static inline int folio_xchg_last_cpupid(struct folio *folio, int cpupid) { - return page_to_nid(page); /* XXX */ + return folio_nid(folio); /* XXX */ } static inline int folio_xchg_access_time(struct folio *folio, int time) @@ -1794,11 +1794,6 @@ static inline void vma_set_access_pid_bit(struct vm_area_struct *vma) } #endif /* CONFIG_NUMA_BALANCING */ -static inline int folio_xchg_last_cpupid(struct folio *folio, int cpupid) -{ - return page_cpupid_xchg_last(&folio->page, cpupid); -} - #if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS) /* diff --git a/mm/mmzone.c b/mm/mmzone.c index 68e1511be12d..b594d3f268fe 100644 --- a/mm/mmzone.c +++ b/mm/mmzone.c @@ -93,19 +93,19 @@ void lruvec_init(struct lruvec *lruvec) } #if defined(CONFIG_NUMA_BALANCING) && !defined(LAST_CPUPID_NOT_IN_PAGE_FLAGS) -int page_cpupid_xchg_last(struct page *page, int cpupid) +int folio_xchg_last_cpupid(struct folio *folio, int cpupid) { unsigned long old_flags, flags; int last_cpupid; - old_flags = READ_ONCE(page->flags); + old_flags = READ_ONCE(folio->flags); do { flags = old_flags; last_cpupid = (flags >> LAST_CPUPID_PGSHIFT) & LAST_CPUPID_MASK; flags &= ~(LAST_CPUPID_MASK << LAST_CPUPID_PGSHIFT); flags |= (cpupid & LAST_CPUPID_MASK) << LAST_CPUPID_PGSHIFT; - } while (unlikely(!try_cmpxchg(&page->flags, &old_flags, flags))); + } while (unlikely(!try_cmpxchg(&folio->flags, &old_flags, flags))); return last_cpupid; }