From patchwork Mon Nov 6 15:49:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13447140 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4C4BCC4167B for ; Mon, 6 Nov 2023 15:50:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 490C98D0027; Mon, 6 Nov 2023 10:50:11 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 43EFB8D0028; Mon, 6 Nov 2023 10:50:11 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 133A28D0027; Mon, 6 Nov 2023 10:50:11 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id F1D6B8D0028 for ; Mon, 6 Nov 2023 10:50:10 -0500 (EST) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id C12AAC06EF for ; Mon, 6 Nov 2023 15:50:10 +0000 (UTC) X-FDA: 81427965780.11.96AD08C Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by imf15.hostedemail.com (Postfix) with ESMTP id 1C653A0012 for ; Mon, 6 Nov 2023 15:50:07 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf15.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1699285809; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oZZ54VdEIlacFfuk1r0uZ44GYKJKe8HsZ6PDThbFMXE=; b=Yg3xIdvhsodRW2bxVR33LRd5pjQZM1em31FAyxhNCNSh2ZcuFKYRAoJ2oIbSaQAPdbuTIz 8s5F8VDhqXuds3TWqW4NwtJ9Xe1TYf0980zJYTzXK0DlB1zj4YatYrafDHQ2tYU1myYnAE pAD/9Q1/DyjEitWKks24L8nn8/4U52o= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf15.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1699285809; a=rsa-sha256; cv=none; b=WFKhhxQhbRXaNeo1snU0CMrVsOKEsCwkG9PjLvZJ+py/9DkMR6Iq5c83fFthXspv4dy9s1 pFGa+zlB02FjuPHmdKJKGCm71tfy9ccT1YuiEOeqaZ5bapvQ0ZsB6LPAzRgG0CszvPKMyM 3MZBct5yFDllTFTuG2tgUVf4ycBK3g4= Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.53]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4SPG4s51RDzVlYW; Mon, 6 Nov 2023 23:49:49 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Mon, 6 Nov 2023 23:49:59 +0800 From: Kefeng Wang To: Andrew Morton CC: , , Matthew Wilcox , David Hildenbrand , , Kefeng Wang Subject: [PATCH 01/10] mm: swap: introduce pfn_swap_entry_to_folio() Date: Mon, 6 Nov 2023 23:49:41 +0800 Message-ID: <20231106154950.3399469-2-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231106154950.3399469-1-wangkefeng.wang@huawei.com> References: <20231106154950.3399469-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 1C653A0012 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: gzcrjcpcydp7zj7negaezqu35rhunss8 X-HE-Tag: 1699285807-234962 X-HE-Meta: U2FsdGVkX18wSqZ1qMj0eLiKNEwweP0/VDUj8XjF0VCEc10DRL0Fp5GxGQGgx3BoKm+nSSaPS+W9TVuuCbA19ZlxWc2zqAtpPQhgF+Hg0MpknVbAhqHBY4Ke20X0Mb+zKEsuv3wsLncMp+1T2nCJoffPB+ytxH9EoEZ6v7ghq1owF9y4nNE6CiqB91Y8/JCDCYBjnxZzahAflyO1WDCrPp7uul7U2eylwJqnqwhu2WtCfmyFqMOrPFgEXZY9PJn1scLD41SRJb0UQhP0FGpYo/aNnytPYLNXopi/Vxa5rtZMVHtw5WABOOanKI6YQ78bV87YPepnLfwbLoM0nKwsqfxx1EE2X0x2NtFqBHAYL2UdR84Ew9UeNEGA6/MLDM/CCrshDYsJFBTDMydvJiJguCki3rLh42yftbDRVWgq4ZLD0HZ5iyPffjMLemA94eh3LE9KNUu69SyJ6SyonW6GNUVHOSqUAxKk4kWJraHSywd28qTjE97VVWI8wudjTw2A74wFmtssWJ8hJvWJxuAIvg1LQtW3It4aPgao5725JJ3bAWXG4YA6pmOAAdTUzATAEUoJvmRlTUhZ2PhVHC4G89Z1Vx5bbThMsi/wTrYzulcJoXRXcZYCADbhqTD50DfFPsusbHtSs+FiEKE6kh3dtVWt4TVqDa2rpeIdhrR9d08ytAB0vb+yoD6i7HnM8Aedw4OHKqZMRM+SEXoG/CwHJhlLDNAH18YldXMQpqDtzHQ+iqWpt24yjwq/rpdMpJX63Oacmzj2pqNQofk2X/MJnQIRw+i+G3JnXnyrdWdxZkpbg1cACuosaiYOsbB7T0WOGz6WemvNfReN944zGm36OELiAkovDyBZR9KWL0E4pXqSxPIZtWUxdTlS7qRp9CIZZUf3FtCk62CKRxPzsKDMoGy+XKCD+E3hdutnCTDNrOUpITig9WqrRDop6TGUPzeJL3NCDaN6z7M+lr6+fuP DD4Z+ydI V7crvjlK4WXRWjeGEUXprpxLUopJ/xShywCzCZV2gRFe9K6r1oNoD86l62fbN31fbNvLfXJnHmuGyxq9TAXFm3Wou747Eo/a7Nm44Zm+Q3oLmLjGw7etRPd0dLIFXxLevzF9R3QlZ9ZWoRX5RNoYHByPwxxYXjdZBTjaTihIGXFZZVotVR9q+345oaYJodhdsDweI X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Introduce a new pfn_swap_entry_to_folio(), it is similar to pfn_swap_entry_to_page(), but return a folio, which allow us to completely replace the struct page variables with struct folio variables. Signed-off-by: Kefeng Wang --- include/linux/swapops.h | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/include/linux/swapops.h b/include/linux/swapops.h index bff1e8d97de0..85cb84e4be95 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -468,6 +468,19 @@ static inline struct page *pfn_swap_entry_to_page(swp_entry_t entry) return p; } +static inline struct folio *pfn_swap_entry_to_folio(swp_entry_t entry) +{ + struct folio *folio = pfn_folio(swp_offset_pfn(entry)); + + /* + * Any use of migration entries may only occur while the + * corresponding folio is locked + */ + BUG_ON(is_migration_entry(entry) && !folio_test_locked(folio)); + + return folio; +} + /* * A pfn swap entry is a special type of swap entry that always has a pfn stored * in the swap offset. They are used to represent unaddressable device memory From patchwork Mon Nov 6 15:49:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13447138 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E3AAEC4332F for ; Mon, 6 Nov 2023 15:50:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BE7AD8D0002; Mon, 6 Nov 2023 10:50:10 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id AD07B8D0027; Mon, 6 Nov 2023 10:50:10 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8F8F48D0028; Mon, 6 Nov 2023 10:50:10 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 6DA128D0027 for ; Mon, 6 Nov 2023 10:50:10 -0500 (EST) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 2647A1A078A for ; Mon, 6 Nov 2023 15:50:10 +0000 (UTC) X-FDA: 81427965780.18.BFEEEB2 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by imf18.hostedemail.com (Postfix) with ESMTP id C57761C0022 for ; Mon, 6 Nov 2023 15:50:05 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf18.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1699285806; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UmujaejCASjfQtJbBR1EzDPhPNeOSdCiPLhgFsCit5A=; b=tCgiSJeHmwQvWLqgDfjYqn2zTtf1aXZyZGTmGg1tIYTKqVE7QUVfjqRbmgmY+uu1hB1ijv NGn5u+GK+fEz56sbVLygin5AbAdmOEbiYtjqg+Hm4qvhBk19F+24AZlLnpZxi9tH1pIlQR hkmOdoUpPFz00+lwhpPxFAn7VNeGf0c= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf18.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1699285806; a=rsa-sha256; cv=none; b=pwaRJi+xqAcWCVO/TWHNG1Ri+qize+K1FwXiR5WyM8q66y7bprfD5Hv9ePWga4zoCIWtJe DC2yClUMyMP14X9ZxZ3wl3VejJL+0fuGENDYr9YNvQ6B+gobV36KT9VQJABVx6qGPxtYtT dzcCEs7DFH6guAfdr4faPN+Ov2MbZFE= Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.55]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4SPG4t11wVzVlmQ; Mon, 6 Nov 2023 23:49:50 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Mon, 6 Nov 2023 23:49:59 +0800 From: Kefeng Wang To: Andrew Morton CC: , , Matthew Wilcox , David Hildenbrand , , Kefeng Wang Subject: [PATCH 02/10] s390: pgtable: use a folio in ptep_zap_swap_entry() Date: Mon, 6 Nov 2023 23:49:42 +0800 Message-ID: <20231106154950.3399469-3-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231106154950.3399469-1-wangkefeng.wang@huawei.com> References: <20231106154950.3399469-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: C57761C0022 X-Stat-Signature: knc8y3b9cck6yqg4irqn4bznt473cwb4 X-HE-Tag: 1699285805-450556 X-HE-Meta: U2FsdGVkX185OJlvhJS/sn/85fa5q/rK6bZRlOPOWw1v6B1BVdAjBuz+LgffaBME+brH0irDo0/GUDqBQw/66WlnCWkNWpRlVuYpRpWb7Mk/zS/ahBWZAilHzTcewnACpkKZOUJ1XdOEqG9MIiMkq9uq+TUs6HtiGvMtLcCpI87JEDp3lV4Q3XntZpG/uxZibs2kJjUwbJGO1PaThGrzBL0ieWY7reqRMwgHJL2Xz1i1udURZ/tnOE9sTpIkq1kLInZOHdSBG5u964UNNGorsQZNsQXqmEQXpH9NQqpYCb/tE/WjkNEqeF8Wbj4LSBISr46qcQ+HVVvB2uM5hvVLRY0N2j5u10vG8b1nxTc5YnwZ4MY+zbuNuT4U55lHg1Hd/+5hZ5/oDn8Vz3GdYuHo7PVDvRQdWLdz9ubTF2MxnVjeANBBxXuKukS7GpiZKyKX98xTp7Ooy8gPqKB+jtqsmu254bmzMFtcgHAG4pjnHrxlLP51f3LbFKK07JIqz/tPe8NHVGHyqU1AqEBWpZqNmIOJ9kA0TWkU20am41BdNs4nHmyUmxxYq1C0j9cdY/wRNin/wEVsngg1Sqfeej109n0faRjDH6XHW9fVYVp3HEyhG5U9IGIyFNFEAqgMg4RJkjv1+/1+EuUOFrqhpCRM8tm+sCYKUb/rxBaITz14OZYKOh98MVk0lp/TW1kW75hNQYMgH7CBlaPuB0cMefz3ZNPRGpaSGzdxqgWE8LDzo2WY2Ol4p8SvY1463J0VfW8vW6Ov0RmkcF8VcPlmArraqsYgI/pGztiBzBRR3T20d10QmV+eFphbw7FAKRNqhnRY7pfCylT0rKnmVvaYUNbpNr2PXfzObJpnwjTszloWmtUHDKiipF1g4Fps588WdEbvif/uqxByeywkHxXTF8TRt5jAXAqXMgG8YajxLl/ozw+eMWx/0RKdpNAkcwe0CMeNXEM51I0Es/BGqsVmnG9 k8g4k9XX HsWwGoq1XStwFe7U4TtwIYUvdHa4CfajnyjX4oxXEj6U/OnupSPn2yBNbyR6sorJdQtslwPrIa2bTf/e6bhKshCX/O9RR/sULNtomYgJb9YXAVe48ckItnHjqxgmgKxXEGVi0QxyffKT2o4PatlMPH2TbiI2htkLj9XkFG431JyUPVKVFMvOHj4PsbdfcJx/jNPA8 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Use a folio in ptep_zap_swap_entry(), which is preparetion for converting mm counter functions to take a folio. Signed-off-by: Kefeng Wang --- arch/s390/mm/pgtable.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/s390/mm/pgtable.c b/arch/s390/mm/pgtable.c index 3bd2ab2a9a34..2f946b493fff 100644 --- a/arch/s390/mm/pgtable.c +++ b/arch/s390/mm/pgtable.c @@ -730,9 +730,9 @@ static void ptep_zap_swap_entry(struct mm_struct *mm, swp_entry_t entry) if (!non_swap_entry(entry)) dec_mm_counter(mm, MM_SWAPENTS); else if (is_migration_entry(entry)) { - struct page *page = pfn_swap_entry_to_page(entry); + struct folio *folio = pfn_swap_entry_to_folio(entry); - dec_mm_counter(mm, mm_counter(page)); + dec_mm_counter(mm, mm_counter(&folio->page)); } free_swap_and_cache(entry); } From patchwork Mon Nov 6 15:49:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13447136 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EB3DEC0018C for ; Mon, 6 Nov 2023 15:50:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6D9EE8D0025; Mon, 6 Nov 2023 10:50:10 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6AF6A8D0024; Mon, 6 Nov 2023 10:50:10 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 50FF08D0025; Mon, 6 Nov 2023 10:50:10 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 3AA1B8D0002 for ; Mon, 6 Nov 2023 10:50:10 -0500 (EST) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 0E12BA0585 for ; Mon, 6 Nov 2023 15:50:10 +0000 (UTC) X-FDA: 81427965780.18.5410BB6 Received: from szxga08-in.huawei.com (szxga08-in.huawei.com [45.249.212.255]) by imf04.hostedemail.com (Postfix) with ESMTP id D75574002B for ; Mon, 6 Nov 2023 15:50:06 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf04.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.255 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1699285807; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MPGmfuGIWa6O6otMqJXbXH0YFlYM5vFEP+zlcU8Yh7w=; b=Vo0QAjPaOOwhRX85dvlJsEoLmU0MnwVHq2ck4lb4vz97u8cOc/XB0NESotpUNfeQ1EF7+h b4315jNqKigTaAG8zhASIJ924Mwg3ORfx0Eca9eM7suUqwvFlGuyDsDs6Pxn1JSXOYQgy1 wmKF7Ie22pdDlEod8gKhaS6Tmpj3434= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf04.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.255 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1699285807; a=rsa-sha256; cv=none; b=rS8TdAmS6ctpDGHvcXZkkiFgvhcEFTh1ECMgFBR0JGbGC5yRSgYmWOQsbanI8UOQxDKFQY 5141Ld4ByCIZJlbPF/t/30SgBTvhl1BkHIn+Isw7b52/Hqhc6idjRkEBm4/p151paeTCH6 O54nPT9KKBBRFoawJMRJ57DTkCq7E1A= Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.56]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4SPG1S1t8Mz1P7vj; Mon, 6 Nov 2023 23:46:52 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Mon, 6 Nov 2023 23:49:59 +0800 From: Kefeng Wang To: Andrew Morton CC: , , Matthew Wilcox , David Hildenbrand , , Kefeng Wang Subject: [PATCH 03/10] mm: huge_memory: use a folio in __split_huge_pmd_locked() Date: Mon, 6 Nov 2023 23:49:43 +0800 Message-ID: <20231106154950.3399469-4-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231106154950.3399469-1-wangkefeng.wang@huawei.com> References: <20231106154950.3399469-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: D75574002B X-Stat-Signature: jhhhiztfh1eqb5571o6jpbo9ndsmrsc6 X-Rspam-User: X-HE-Tag: 1699285806-167049 X-HE-Meta: U2FsdGVkX1/hVgXLX8nhOcPYOVjhliI2lKnCgxDsqCovPR0Z5GY0zGL3B8OkPBmXwZxii1oTo5KWK980wI7wBRZFTEAng73dzp/RzYdW+pncuCs8DHRI3Fuur9qJlv7+ZF18lu4ZeCE+7FMgrI1rcOUr+S5+WI6Cfdo65gsJEOoVI6nSf3r79krHKEvkm81tl1CYVvOAQ/6Z/Zj8aoxaUHpcUOB2G/+HKkIPso7Uuz+L7hbJxXNI7UaZAGRQloivY60e6vaGbw0HAgXLwCu9VwmoZAwrweQ0ojyUYaXHtCAQScdyAme0kYeBA0eUAVyCfs6Qq7aFP63hGmerPGOLud4fn+vD86MsskNkeBjRXsY3otdSC2W4fCxGLIGy1ucBxu7rJ9w2dQhqzl7UVLNVLPwVah5MhqmRgqvL48I8vLRKwjhz1prZNC4GTxkSWBwkdWqDUSK+gNOpQ/CNiOUbCGB8BexfRanqqzKvOxTW0ZMHCXK32Z+Mq10p4iapexhYkR2rFRpRxsDWyU6JwKSYFJ9rfzbuGUrxunDz2PiSWBy5j4NSHL1r8DrdPwn116N5P50AR6/+kOmk8rPYFZ0aA6hf9V0q+wSyv08JYfu2V9jhwhZyY35xqp3UDLuyR4ADW6wwFs6M1LLr1Q6mg3bxWaA5qS8k5m1rNiSQs2Gv4xkYdH8saqzwNmfkNs98tHFa4FNGnJt2/sFEutfUXvYtwK9WVQagGaNA2AMI10/Ovh0f49JcEpKHm6urLhflOS51o66eyuAgAsA0pesw3A+2h9xJoNUTzG8UZ92lTW9QxkPXEvTcUdNRwIlCDQGnKC6M7HxvXNNOOxWjyCJ9RM/8Q5Tp3MOzuzmnaTzoSW/gGCxGgcRDs1lkqqCA3R+fCM7J5X30IAaiWVPSDVjuQjbCUa12N26wydlcpcPzpJLycPdadVAXi4gxP7Wdv4LXzJk04lP9WD5Q/2FHTvzmPxx 5SnF8Pkn VA3Aam0gBevfPARg4OlP2XQaZhO6RoYByz+AYIksSqsih24r0gaGMe8vOISFbFRovXw3MiGovhNKBnd5vsoObtcVwJf63XUYBTI16nnKOj+tuebJ33g0MtlNgAy7zzkFXU8UxNbDqZ8GzW73/5E058Qn34pmo11NVoBwRYD256aWIwZ5YUUodtE6wZNL/X5tl54Wp X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Use a folio in __split_huge_pmd_locked() which replaces five compound_head() call with two page_folio() calls. Signed-off-by: Kefeng Wang --- mm/huge_memory.c | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index f31f02472396..34dd01970927 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2117,6 +2117,7 @@ static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd, count_vm_event(THP_SPLIT_PMD); if (!vma_is_anonymous(vma)) { + struct folio *folio; old_pmd = pmdp_huge_clear_flush(vma, haddr, pmd); /* * We are going to unmap this huge page. So @@ -2130,17 +2131,17 @@ static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd, swp_entry_t entry; entry = pmd_to_swp_entry(old_pmd); - page = pfn_swap_entry_to_page(entry); + folio = pfn_swap_entry_to_folio(entry); } else { - page = pmd_page(old_pmd); - if (!PageDirty(page) && pmd_dirty(old_pmd)) - set_page_dirty(page); - if (!PageReferenced(page) && pmd_young(old_pmd)) - SetPageReferenced(page); - page_remove_rmap(page, vma, true); - put_page(page); + folio = page_folio(pmd_page(old_pmd)); + if (!folio_test_dirty(folio) && pmd_dirty(old_pmd)) + folio_set_dirty(folio); + if (!folio_test_referenced(folio) && pmd_young(old_pmd)) + folio_set_referenced(folio); + page_remove_rmap(&folio->page, vma, true); + folio_put(folio); } - add_mm_counter(mm, mm_counter_file(page), -HPAGE_PMD_NR); + add_mm_counter(mm, mm_counter_file(&folio->page), -HPAGE_PMD_NR); return; } From patchwork Mon Nov 6 15:49:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13447143 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BC853C4332F for ; Mon, 6 Nov 2023 15:50:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 973D48E0003; Mon, 6 Nov 2023 10:50:23 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8D5788E0001; Mon, 6 Nov 2023 10:50:23 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 74D1C8E0003; Mon, 6 Nov 2023 10:50:23 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 654978E0001 for ; Mon, 6 Nov 2023 10:50:23 -0500 (EST) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 3F15CB57FF for ; Mon, 6 Nov 2023 15:50:23 +0000 (UTC) X-FDA: 81427966326.28.1A05F98 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by imf07.hostedemail.com (Postfix) with ESMTP id B04DB4000C for ; Mon, 6 Nov 2023 15:50:20 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf07.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1699285821; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tP9NWM/ekJcLRFo0kqQKalmLDZuh2TT+P+my24sgZwk=; b=DWqe1/6NZd2Xu1FxvaIV2sC0K21Q4RJLonJIAf7badBhjUMFS2ZucqrVmo5dGLEnZ+fvB/ GG+hML9RLZP8L59FI0x+TZddOap+xG+4XgZpJDNyaWrtPgRKP/OgEVxsWraaIa1eWmIjzj ljYsRA4AVpBT+COpDjTcPMpepUoLZYw= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf07.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1699285821; a=rsa-sha256; cv=none; b=qsKyoYwegf9rk6BNEpG11z1oLjeOdeOgfaK+ePt7FrzzgtbSDp+kw92DwmLEImhCoz8cFo Od714Sjcsk1R9fggyXqQqSBEFKFugbao9zznhU6Jnyoj4GMJrGdk3zTdl9G0iIBnuqHTcC wI+meHAQDuHu5KPzlIip1RjhXJ/od/0= Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.56]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4SPG4x5P3BzvQTB; Mon, 6 Nov 2023 23:49:53 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Mon, 6 Nov 2023 23:50:00 +0800 From: Kefeng Wang To: Andrew Morton CC: , , Matthew Wilcox , David Hildenbrand , , Kefeng Wang Subject: [PATCH 04/10] mm: huge_memory: use a folio in zap_huge_pmd() Date: Mon, 6 Nov 2023 23:49:44 +0800 Message-ID: <20231106154950.3399469-5-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231106154950.3399469-1-wangkefeng.wang@huawei.com> References: <20231106154950.3399469-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: B04DB4000C X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: 7iyqy79z84ykwnxmbujdn31tgbshshqp X-HE-Tag: 1699285820-304369 X-HE-Meta: U2FsdGVkX1+N7wyYLyurc4PsUP+NvAAuMCDND4n1NbD6RbPaFZSe9WodjOcemnAuJaw++ZJAmJLWiZkY+47rWcvgWFt3jPLbAA6K2xeCRKzGOR2vuLXYDEZMnJKH9WzuPqbn3/ox11puVRr64CIIqaWyqsIoTZ9WW1Fj28r+mSzXfBnBsRG4ZAByJKq/cGGcr6uFzfR7W9jVBFmb5rlOoRVBrkW608wsAb5LLpDDtR8mipoLUYJWDYW2bpVe3bpMElm/jm5H7ZFlFaT8fwCmxFrv3dH2Z3/56/KqE+NA71hM06LmvJsV1VEuJy286qGehJNFcPX/CSrgZvW+z4OZyFGHHMe+p7pj7+5MsVLy2gpOSnCXD6x6daKitE/x1T/MFKJvA7ht2LdOgBx0QNY3FOKCkvGD3QxpI15kB7c2tJeeXYMOLWlP/wYmaijhghv+FQej8EoQnsuNLW2yuyRX6PhN28nvXPN4kPMRIywl7zlcr00woDi1s8Enwfi8AYc/zYPKLAzVOyCOBHKKhTSQZ5qA5yT23vY1QbLIgQpsM/HRIu4TctiBKWWWrZSj5UXhxb++a4it+ojTxmmeebaDbTZ4TEZA3kdWUTjOIX1e/94dMUV5/9jBNkKrYidE6NiV5INJGC6ppw1v170BLtjpzuHfRvKQuKlegUphC0GjhQHZO8PFgVibtyYdse/f5V2/pJgXwt3SvQOMRuKkOvwkIf9Y+V/dhNOvnGf9/90Rw8kuMEMaHkU7rEKC60ZvUyhrX+55O0Puao7JPN8qZ79oqBk31klfcK+r8zfGCjfYH2M4D3HvvZ3I028DWDW8dMHZLjGch9famJbZxqrPjeLd3QYOtBChDQnj4JuRdtTMnEvbYBj/Wo4jBxuLkJdBHFfbXDC+/BER+ZRYQCqPsbomfSYWnXTwiQNvWBBkCeuUSxB+dTGY7S9rXwLI0PVRefVbUDEE9sEPkf4MFvbBq52 uSflSnrA mfLJXL6pU+rucml4KetSeTB0F3335b0Muv9AkRDFFX4Tc6tQWvApdUaohykmJYKVLReEMQzsSmlFre+LpBzB8ACFij+TMtweNjjZPR/sdf0PXCoVZEhuDKsVJH97HuFtvZC0qpAkA1spKEOlJ4jGI0o9dBEpUWZmZQl71DRN6gPsDJl0URjCJpnetHeHkNEPyhrXn X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Use a folio in zap_huge_pmd(), which is a preparetion for converting mm counter functions to take a folio. Signed-off-by: Kefeng Wang --- mm/huge_memory.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 34dd01970927..78a00fe22c2d 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1716,11 +1716,13 @@ int zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, zap_deposited_table(tlb->mm, pmd); spin_unlock(ptl); } else { - struct page *page = NULL; + struct folio *folio = NULL; int flush_needed = 1; if (pmd_present(orig_pmd)) { - page = pmd_page(orig_pmd); + struct page *page = pmd_page(orig_pmd); + + folio = page_folio(page); page_remove_rmap(page, vma, true); VM_BUG_ON_PAGE(page_mapcount(page) < 0, page); VM_BUG_ON_PAGE(!PageHead(page), page); @@ -1729,12 +1731,12 @@ int zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, VM_BUG_ON(!is_pmd_migration_entry(orig_pmd)); entry = pmd_to_swp_entry(orig_pmd); - page = pfn_swap_entry_to_page(entry); + folio = pfn_swap_entry_to_folio(entry); flush_needed = 0; } else WARN_ONCE(1, "Non present huge pmd without pmd migration enabled!"); - if (PageAnon(page)) { + if (folio_test_anon(folio)) { zap_deposited_table(tlb->mm, pmd); add_mm_counter(tlb->mm, MM_ANONPAGES, -HPAGE_PMD_NR); } else { @@ -1745,7 +1747,7 @@ int zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, spin_unlock(ptl); if (flush_needed) - tlb_remove_page_size(tlb, page, HPAGE_PMD_SIZE); + tlb_remove_page_size(tlb, &folio->page, HPAGE_PMD_SIZE); } return 1; } From patchwork Mon Nov 6 15:49:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13447141 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D30F1C4332F for ; Mon, 6 Nov 2023 15:50:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2BB168D0028; Mon, 6 Nov 2023 10:50:14 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 26B9A8E0001; Mon, 6 Nov 2023 10:50:14 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0E3BE8D0029; Mon, 6 Nov 2023 10:50:14 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id EA4B08D0028 for ; Mon, 6 Nov 2023 10:50:13 -0500 (EST) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id C6FA7140774 for ; Mon, 6 Nov 2023 15:50:13 +0000 (UTC) X-FDA: 81427965906.06.B7E41C6 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by imf29.hostedemail.com (Postfix) with ESMTP id 93CF0120009 for ; Mon, 6 Nov 2023 15:50:11 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf29.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1699285812; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=S/5dklcr5ixKzN/mRuVptDUFzLJu2EfnoxXfaOnnEM4=; b=wmTy9YAvW3oZZr2Vz7UeJwakhafYitrHk7+kJEWy40BDLvLdW1zuBR4mAY8m4REvOi3j6B ychgNdA+CYr9PwR2kguILbOx+VUcjRic/82vYoGmpiWTkwh+NCuF+Eq9t79wiOlUmIhDdO dwixcBBjpDeKqXTCVEWYdm+2enXV/mY= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf29.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1699285812; a=rsa-sha256; cv=none; b=aVREa6eaSZoV5X71MeWGIfhN2/E9pQoGEtcQ6Q6w7hLE5k+w+e6vsNAHGBRc3yDzlNQaSJ pDr++v9hTMQI1grkEBbqWPcug+Xx6pG4fpLiRUxwIRswAdVxxMPWLW65UPuqsAibKHBaWO RVVz7zyL5KRsEvC6vDQEC1Zk9lMAjuI= Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.53]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4SPG4v34Z0zVlpd; Mon, 6 Nov 2023 23:49:51 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Mon, 6 Nov 2023 23:50:00 +0800 From: Kefeng Wang To: Andrew Morton CC: , , Matthew Wilcox , David Hildenbrand , , Kefeng Wang Subject: [PATCH 05/10] mm: memory: use a folio in copy_nonpresent_pte() Date: Mon, 6 Nov 2023 23:49:45 +0800 Message-ID: <20231106154950.3399469-6-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231106154950.3399469-1-wangkefeng.wang@huawei.com> References: <20231106154950.3399469-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 93CF0120009 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: 3b4ypkxjz4fq9zkbigphhwkpakzhxdsd X-HE-Tag: 1699285811-660564 X-HE-Meta: U2FsdGVkX1+zxTIm56tumGVM0kXtkVVXOeW685tHht11WLHzHaslXnhi/crIXgVStJ3E5V3YaUVt0V/qSOu+F4vw10CzSkyXCthkl+mRjLk+L44Df12/UVNVw8Q5B0HeDAGG6VwyC6xJ1Nm2dlewbWQ8kLZkwqvpIAHC+CZ5ORZcFPZJK9j5I4ytZIiUWxDKKCrvOz5d3EtyhoReP9pOOV7tsbVcDbl4tNcixtkBDUk2PORgudkpzBeovlOLZ3yDmO5uRD1O33tKzj+x+j7kplo9fCZGhotzOPyCnE0uJFVB0rLpVaeVHV3V4d0Dro26+TfYDl2AXDPH83fWpreUz3zv6OnxpBxaOoD6ahiMXIVYHFFgTUJXUzqPwbM7gq6FFKHT2yVTDfNVdXAyrzs9CtyRpDxsGgc6XFHR5Ks71l20TM0TZnROqgP86imByqJfboJdfI9ioH6NA/KtC372nEILCXItOTNfMCNbzHQD6cxagNmHsyoPT52nMRLZKLGURdrTk1EOT+yDFBjUzWvkf50mUPcrHFD45q+yncLAqmfS8CX9lI6pvX+KQ6oaf7MYmlNc2xhXSRlk0DGmxgjtYZo+SERyPSMsgReDH8+Xo5hl3a4uJv+ktFyfMj/JnkhB02qRcaXKQdxGfO5Q76coP4sfrUHHR1qbVq49fu9gyNdD0qmqsLCzrSjwZ6rUf+/0qD9fa+COq+oLmbqgenwLU9CZdNerm9UrfJZccpRZOVBbOfQKbmpFOi4ZTDeIRAkj8uA3MAkNUvxSD9nkayTbUDSFVVAuvBmZlAPHFmab7i4xR+mDIphydwb8oe5B25JGUGpLd30mnJ0hPCyoZg4XhTQ6qMDvanWtsgT9rECfqQfmOrzl4fZffVgHqcDsxQKC9kRB2zAmzNzF/IJbgtHHY7NQfBR9/D5Qj/7pydlZBvevDJePDArv/D3Qw08IVUOkNRDRn2IoFFmAtNQ21+M k6yQn3Dw WT2XvGT3PM+/Brw6w8zC9VciGsCZUFSXbzCEoVEBMl5InAJFv+lptks/kZq9/rcRfr9a+cHnf4RpWfmYCAhaCaMMFA3HhgrKHynfvKjU2miGjFKGXRO7t+oAjnWRSsHbj/SHrL+4Ee2M1UEhdHnXg4EWwrc3liusQYyZoeMDp90nSczzSmHsJoRWfT+6Pby7UXvyY X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Use a folio in copy_nonpresent_pte(), which is a preparetion for converting mm counter functions to take a folio. Signed-off-by: Kefeng Wang --- mm/memory.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 1f18ed4a5497..914353d1c7f1 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -779,7 +779,7 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct mm_struct *src_mm, unsigned long vm_flags = dst_vma->vm_flags; pte_t orig_pte = ptep_get(src_pte); pte_t pte = orig_pte; - struct page *page; + struct folio *folio; swp_entry_t entry = pte_to_swp_entry(orig_pte); if (likely(!non_swap_entry(entry))) { @@ -801,9 +801,9 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct mm_struct *src_mm, } rss[MM_SWAPENTS]++; } else if (is_migration_entry(entry)) { - page = pfn_swap_entry_to_page(entry); + folio = pfn_swap_entry_to_folio(entry); - rss[mm_counter(page)]++; + rss[mm_counter(&folio->page)]++; if (!is_readable_migration_entry(entry) && is_cow_mapping(vm_flags)) { @@ -822,8 +822,9 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct mm_struct *src_mm, set_pte_at(src_mm, addr, src_pte, pte); } } else if (is_device_private_entry(entry)) { - page = pfn_swap_entry_to_page(entry); + struct page *page = pfn_swap_entry_to_page(entry); + folio = page_folio(page); /* * Update rss count even for unaddressable pages, as * they should treated just like normal pages in this @@ -833,7 +834,7 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct mm_struct *src_mm, * for unaddressable pages, at some point. But for now * keep things as they are. */ - get_page(page); + folio_get(folio); rss[mm_counter(page)]++; /* Cannot fail as these pages cannot get pinned. */ BUG_ON(page_try_dup_anon_rmap(page, false, src_vma)); From patchwork Mon Nov 6 15:49:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13447137 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D1B7FC41535 for ; Mon, 6 Nov 2023 15:50:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A0F2C8D0024; Mon, 6 Nov 2023 10:50:10 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 999A98D0002; Mon, 6 Nov 2023 10:50:10 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 63A3A8D0002; Mon, 6 Nov 2023 10:50:10 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 4CE0F8D0024 for ; Mon, 6 Nov 2023 10:50:10 -0500 (EST) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 169CF40807 for ; Mon, 6 Nov 2023 15:50:10 +0000 (UTC) X-FDA: 81427965780.22.1ED2CDD Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by imf04.hostedemail.com (Postfix) with ESMTP id E9A1D40023 for ; Mon, 6 Nov 2023 15:50:05 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf04.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1699285807; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PwuD5h4HDp7gyAC3F3z00/3MQZmSXYSpa4jfbyyWmuo=; b=zeowSVXGUTfGJ/f4oNgtWp0qeqNAo17ldAoW6vbVkojqHgUGuK8CLtQL4oj1YDJyLGuUZU C48EPsvfIuNvZPXzFlfJTzZ7rfyjGTMRCwl/nTvehJUswCY0ls6HorlrSZpctC3jzMgmtQ gMUT4i42X6RInbQrTEX5Qx4GqweWPmY= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf04.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1699285807; a=rsa-sha256; cv=none; b=sYB8OUoOpN/wMDDR6DTQ31holEZv72RiF5UH1JaKLkQx+wmOiXlPZ/9RcCRmLCO8VAb5F+ sxccTwcCo4NP9ZkDuED67lGBp2kVJ96GHvUu8YyX377wKSdUaszWWQaIMCceI9bRtVgaJ1 sW74nJgsAXzPBqUj7ff+1IX2ItxgKDY= Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.55]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4SPG0K1YGGzPnbL; Mon, 6 Nov 2023 23:45:53 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Mon, 6 Nov 2023 23:50:01 +0800 From: Kefeng Wang To: Andrew Morton CC: , , Matthew Wilcox , David Hildenbrand , , Kefeng Wang Subject: [PATCH 06/10] mm: memory: use a folio in zap_pte_range() Date: Mon, 6 Nov 2023 23:49:46 +0800 Message-ID: <20231106154950.3399469-7-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231106154950.3399469-1-wangkefeng.wang@huawei.com> References: <20231106154950.3399469-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Rspam-User: X-Stat-Signature: bhaxdhms9fm9giof6gkwz8rc6f7xsgi5 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: E9A1D40023 X-HE-Tag: 1699285805-5286 X-HE-Meta: U2FsdGVkX1+l7vfZMhv8vkjGHmMOOmOMqUfjJlK5+/a50DzZSc2KerLxi5CA//l97HblTBeVYnGFfVOw2nLoXeCRK4bEd+v1z5VMbe9wMta5keKhPKqH7iEEb2mzjFky3idlVGQLoBYQy0TSkY0CELMo8fap9GEcpBcxc3BcM2OxkH2C/YE4nWmyVDriRvLpDbJugdqmdS8E3UGkDXj+UKmTwg6AYKMK5HrVQ/d0gzS/wguNBstXkBbSLhiuO2yFt4STKCV1cB3NI+/mDXouKOT2wnVYAAu+0/ck+KRBdPoTLrkzckQ/GNvRiQN2E/oE47TIcqmTOuZR5pAJBmO05a8Dd8uHBtni2FRSPJhgWECdpNJasWtEfzM2rvBlKI08h5hLAs4/VW0I0qMYOjxm3UZxm7FDgEifhQBOmquo3DIiAEJbWRR3xBl0Ya6cpOyLEKgrnSW9OwiBSZwVKM6ml6OZJNyUEaEW36F0owU/Ny/iJ+XGCouTVRR4XBqLmaZQqJY29NExw3PL+JjakZyOnQ6XSb2A3gq2pHhTFuxPwiKbpBNS0AINQbZOsePdzG0JlAmRiavOmN3TPkew/xN8Gjms3Rkrat1OGMLlNG+79hKLOjH5etqVXNb7ZHV9unX0ZZCPOsQiIYGGiR87Dfe62c5vHbDnQtNWqxIXEiNJKgFt7Sn1MRUibTckjft6fvpv8wg9cgJE+9VDINXHzc30nsYvkTTFwZdjudZoC2d9TkEMkMhTN+e6fNYwtM4vPfgWih2QkgxUxJe9n5kgMFxQTwGW1usawzy6YoallhMrJZFbi3GdD4zbCvPCPEJA3jAPUcpYTMmseZ/3evV7gtqIK+7P7cb6ySrb1levYDxvJF9U4nlZXUVk+oJsdxwpRy7qyLoFLkz498mSxx7qc9CfbqaaG61blqQhuQbX4hjM2tT87ALDiZLF89WO+lQ3FukE+i3MQ0y1Ot3JEkgRU34 dphDkHEM e1u8zGhrit2JY+sHpwSChwxm9yQicY6d0rPV2NOCxw/D3IPmK+pg1h/6ksF6TaQPvb9uZOXylO9hsXaEWIPJ3MF/GasTXqIj44vUh5b0IETYcePGSOigTbW2MNPS+MwZTTAm1TVcJrUN9JZ4laxDENWFGEXjTTQU2WUGhXDiH41s1Wsy3FAlzdEcxS/kNSLomKbK8 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Make should_zap_page() to take a folio and use a folio in zap_pte_range(), which save several compound_head() calls. Signed-off-by: Kefeng Wang --- mm/memory.c | 44 ++++++++++++++++++++++++++------------------ 1 file changed, 26 insertions(+), 18 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 914353d1c7f1..669f167f45cd 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1359,19 +1359,20 @@ static inline bool should_zap_cows(struct zap_details *details) return details->even_cows; } -/* Decides whether we should zap this page with the page pointer specified */ -static inline bool should_zap_page(struct zap_details *details, struct page *page) +/* Decides whether we should zap this folio with the folio pointer specified */ +static inline bool should_zap_folio(struct zap_details *details, + struct folio *folio) { - /* If we can make a decision without *page.. */ + /* If we can make a decision without *folio.. */ if (should_zap_cows(details)) return true; - /* E.g. the caller passes NULL for the case of a zero page */ - if (!page) + /* E.g. the caller passes NULL for the case of a zero folio */ + if (!folio) return true; - /* Otherwise we should only zap non-anon pages */ - return !PageAnon(page); + /* Otherwise we should only zap non-anon folios */ + return !folio_test_anon(folio); } static inline bool zap_drop_file_uffd_wp(struct zap_details *details) @@ -1424,7 +1425,7 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, arch_enter_lazy_mmu_mode(); do { pte_t ptent = ptep_get(pte); - struct page *page; + struct folio *folio = NULL; if (pte_none(ptent)) continue; @@ -1434,9 +1435,13 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, if (pte_present(ptent)) { unsigned int delay_rmap; + struct page *page; page = vm_normal_page(vma, addr, ptent); - if (unlikely(!should_zap_page(details, page))) + if (page) + folio = page_folio(page); + + if (unlikely(!should_zap_folio(details, folio))) continue; ptent = ptep_get_and_clear_full(mm, addr, pte, tlb->fullmm); @@ -1450,16 +1455,16 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, } delay_rmap = 0; - if (!PageAnon(page)) { + if (!folio_test_anon(folio)) { if (pte_dirty(ptent)) { - set_page_dirty(page); + folio_set_dirty(folio); if (tlb_delay_rmap(tlb)) { delay_rmap = 1; force_flush = 1; } } if (pte_young(ptent) && likely(vma_has_recency(vma))) - mark_page_accessed(page); + folio_mark_accessed(folio); } rss[mm_counter(page)]--; if (!delay_rmap) { @@ -1478,9 +1483,12 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, entry = pte_to_swp_entry(ptent); if (is_device_private_entry(entry) || is_device_exclusive_entry(entry)) { - page = pfn_swap_entry_to_page(entry); - if (unlikely(!should_zap_page(details, page))) + struct page *page = pfn_swap_entry_to_page(entry); + + folio = page_folio(page); + if (unlikely(!should_zap_folio(details, folio))) continue; + /* * Both device private/exclusive mappings should only * work with anonymous page so far, so we don't need to @@ -1491,7 +1499,7 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, rss[mm_counter(page)]--; if (is_device_private_entry(entry)) page_remove_rmap(page, vma, false); - put_page(page); + folio_put(folio); } else if (!non_swap_entry(entry)) { /* Genuine swap entry, hence a private anon page */ if (!should_zap_cows(details)) @@ -1500,10 +1508,10 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, if (unlikely(!free_swap_and_cache(entry))) print_bad_pte(vma, addr, ptent, NULL); } else if (is_migration_entry(entry)) { - page = pfn_swap_entry_to_page(entry); - if (!should_zap_page(details, page)) + folio = pfn_swap_entry_to_folio(entry); + if (!should_zap_folio(details, folio)) continue; - rss[mm_counter(page)]--; + rss[mm_counter(&folio->page)]--; } else if (pte_marker_entry_uffd_wp(entry)) { /* * For anon: always drop the marker; for file: only From patchwork Mon Nov 6 15:49:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13447139 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E98A1C4167D for ; Mon, 6 Nov 2023 15:50:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 136FA8D002A; Mon, 6 Nov 2023 10:50:11 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 06DCC8D0029; Mon, 6 Nov 2023 10:50:10 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E50918D0027; Mon, 6 Nov 2023 10:50:10 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id C9DCE8D0028 for ; Mon, 6 Nov 2023 10:50:10 -0500 (EST) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id AB858A07E2 for ; Mon, 6 Nov 2023 15:50:10 +0000 (UTC) X-FDA: 81427965780.13.BE45F63 Received: from szxga03-in.huawei.com (szxga03-in.huawei.com [45.249.212.189]) by imf12.hostedemail.com (Postfix) with ESMTP id 67CB340008 for ; Mon, 6 Nov 2023 15:50:07 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=none; spf=pass (imf12.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.189 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1699285808; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PXinfkgX9Mk6qMgnf/9ujGJB/UjAvuMExZucM1FxJLU=; b=JW+UPDInRuVhnqitnya7zrVvvtMS8NOBJDawmo6/zXLIAHxoE8h8y16GjsWDnJDvy1ShKZ IOqo7Hd32veS8GZ3XGxbBAks1ymusyGrgoUovjBSs3Zu3suV9jdwQTvjKuMxQjICDgiFVi TTuhIadQopbWbiuFkBNsWCmF86mIsyM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1699285808; a=rsa-sha256; cv=none; b=1G+O3sd2GoXiQZto620xLwsLALuOu5i6lBVyZwrC7LHSkZDoXtXUgR3sWTth+2YrlmfOHu FxFjaTZDIMAui91YZUhDU9hchMGowhc2tkOKAkuT6bhBswOioFLkABJDUgJFLwyJCrpFQl 3eziqYznYq/X4wvagHVUhyhDheCaQQs= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=none; spf=pass (imf12.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.189 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.53]) by szxga03-in.huawei.com (SkyGuard) with ESMTP id 4SPFzz4mWdzMmWr; Mon, 6 Nov 2023 23:45:35 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Mon, 6 Nov 2023 23:50:01 +0800 From: Kefeng Wang To: Andrew Morton CC: , , Matthew Wilcox , David Hildenbrand , , Kefeng Wang Subject: [PATCH 07/10] mm: memory: use a folio in do_set_pmd() Date: Mon, 6 Nov 2023 23:49:47 +0800 Message-ID: <20231106154950.3399469-8-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231106154950.3399469-1-wangkefeng.wang@huawei.com> References: <20231106154950.3399469-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 67CB340008 X-Rspam-User: X-Stat-Signature: spa7qkgxpc1wnxw8igqwkdxabknm45ah X-Rspamd-Server: rspam03 X-HE-Tag: 1699285807-79787 X-HE-Meta: U2FsdGVkX1+JeM3udWgTATbUKXVb/F0PW/z+DADHhXay+DHmKK+I9PjlECDSExKU6zj/BgwEVSNooyg8r0SP0iDimw+TaRp5nG5YJeLm30bocAB+vWVNL7hAnKcUnl+JLW22LEebOgVIFB4RpWl0ppHP3auQIk0X5MAtfc2q3ThCacGNf+qtt79Ny2RDnbz+ZRFi/KUyaqnc4tDC1rE+w3KvTS/r5mOR2UKPx1BXmMuReQPBaWBslKL9p5vYA9TQZfuXwydYExCQ6wwEX7f8oQLBn5lO2s0o1KqOh0VkWRVtaUZN2AivBg/XwqPFnGfDN0WNz8CymJ0sfwIlpxSnirn2kcuAJnVXW6+UY+jxDuBiy28gIySlZQvwZ0AkICNgmCTwhw6hmWFsfxJFrXi1seDPOWEsW6EOajDPAkh+BDqm6uGNNQyuNjsHOlXW69XULCzgduK1zjYw1Zvf6uM/a5wgD8E/0CQ6SV3BhKPEraAUXSkVL/DhF4xOX71lZ9/E1xPHT607Q7fWQOTjwrHBfq/hdd4E8ZqXv6ruFMCcG2sQxQQGM2MkI0Qs2/tF3tqRIxLjSh68xhcfJ0pVy5cXdy+tvB3xrIgUkZo7bEkxPzJtowhO5OYS4406n8lgFlx4XoWlPAMGkryXQLq75o8qVsLhjeUap3A9/xU3SXxKoupic95JrBVOe89exLsFWGjS1/mpJDqb+Otudo9RexnoQdeWZmN4p6xRefG5udXxNHNLbRuHgXrwPyLH0nGoFNbUimdnrZ96Unxug/A/YzwG/2JqqVtjI2AYBOrTv+T9jU7BPLK5oNc7u9Wn+4CI+7bU1mFcux51yS7uARmfAhY/7wIytob64fNPXwr/bn/P6FAV12/Zc+dxem1f0INWGaiqPJpUT8HyEzlmGtFouZNpNDROxNV+oLgR7ECBxwNf30W0eRJO+5af8aYsxOnqwDtI9ISoxpY0s5ZWd3a+qMe I8vn9ImJ 9574D/LQ6uunmHqygDxxuDe3rVyD5DpPswl8PAROWUynpwK7Lf3y3vJwgSpVK1SuIQ81COhqc4qfCdGG4VMsSVNIghmGd5WPsnn0qmtgLR7W5CpGP7pVa97XJtoCis1jepyxDkCnvRiXDdi1KmtiseqancG98+BeKgSUA6JZbfXhWLtv/QXO005St3Kwekk4nONaU X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Use a folio in do_set_pmd(), which is a preparetion for converting mm counter functions to take a folio. Signed-off-by: Kefeng Wang --- mm/memory.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 669f167f45cd..fbed32a09fec 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4322,12 +4322,13 @@ vm_fault_t do_set_pmd(struct vm_fault *vmf, struct page *page) unsigned long haddr = vmf->address & HPAGE_PMD_MASK; pmd_t entry; vm_fault_t ret = VM_FAULT_FALLBACK; + struct folio *folio; if (!transhuge_vma_suitable(vma, haddr)) return ret; - page = compound_head(page); - if (compound_order(page) != HPAGE_PMD_ORDER) + folio = page_folio(page); + if (folio_order(folio) != HPAGE_PMD_ORDER) return ret; /* From patchwork Mon Nov 6 15:49:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13447142 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2D847C4167B for ; Mon, 6 Nov 2023 15:50:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C305A8E0002; Mon, 6 Nov 2023 10:50:14 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id BE1838E0001; Mon, 6 Nov 2023 10:50:14 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 948A88E0002; Mon, 6 Nov 2023 10:50:14 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 800568E0001 for ; Mon, 6 Nov 2023 10:50:14 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 0161A8068F for ; Mon, 6 Nov 2023 15:50:13 +0000 (UTC) X-FDA: 81427965948.08.313DF48 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by imf01.hostedemail.com (Postfix) with ESMTP id 0B81440021 for ; Mon, 6 Nov 2023 15:50:10 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf01.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1699285812; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8W4DGONJ14Bibgm3pNXmI4YyRAKX5ZNYD4yPVpkOEtg=; b=8rYzuJ4rVGEfUs+IO7pi3dLKtdjcIXVGLsyIkCDi6xBeBc1CksNz0fjvqKGA4cuqiUSB/u LfNzVt7idxEtWI6yAYzFeaIET5MGE+NyhRCZxjwg9rtgQWI+RQbIp9W7iC0aViDdHeZW8i tdm8prgIzcOIt2YUaV9jkUL6LIA6ME0= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf01.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1699285812; a=rsa-sha256; cv=none; b=FAYW82lRcosyuUgy/lC9e8KZTMUO6Rm/ere1739VMhWZLIntbxBwNXnJnpL5MjnBdDxyeA M44+JyEXWD1yoT5tJjBS3y+PBbwc8o2RJbK5Zmj6tA4KH4+HzT+1AdnwgKPx6zNvMKEBFz Cbcoo60jPztIPAQaBHSrbdkMG3Qpscg= Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.56]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4SPG0L0dT2zPntQ; Mon, 6 Nov 2023 23:45:54 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Mon, 6 Nov 2023 23:50:02 +0800 From: Kefeng Wang To: Andrew Morton CC: , , Matthew Wilcox , David Hildenbrand , , Kefeng Wang Subject: [PATCH 08/10] mm: memory: use a folio in insert_page_into_pte_locked() Date: Mon, 6 Nov 2023 23:49:48 +0800 Message-ID: <20231106154950.3399469-9-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231106154950.3399469-1-wangkefeng.wang@huawei.com> References: <20231106154950.3399469-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 0B81440021 X-Stat-Signature: ysjomr9beu83y5d18om3r1wrwwtqfhb9 X-Rspam-User: X-HE-Tag: 1699285810-431889 X-HE-Meta: U2FsdGVkX1/dRRS90qwr5XYn2orWQ9dyNycxgTnoLN5u8qJqbSIH5sOmVOAbFOLKXkUdFkGM8tCaxGVuBiSXxJm9aNNsuEbkw8+Vy8tMvw/He8PFldCDiEa9EGrZiEbuE7G/OjFdeGJaM/xQ022kP64kEDVxBl41Ozn2BpA/aSk+jEdrvP6NKZfCxLcjCrKDm+AAvsLnzi0iyyOKdYj+5Rh+PYud5NXbjhrC+XG0YHuE1UxT2zgK9LmNgpQlTQxmxyVOJs9RowrY6cNTOy3+Hxqh8fV5NAbW3LW/CC9BNnNwMYGpFdpcz8chOgQ+x2zv7nBfFa4oT72R7Idk8jZY7HB81SlnbC+726Y0Eudmwt8xzHi3fXQfM0uVXAmHZNDOp6seeNEj4t3QkBw22W4I3nP+QIXIpdUcXsn2UT0ZxAVKAQLN+UgiHVDh+rZePWJ5txudTkf1Wpk7dp/wUu0d7qlMKYrNwUQfa7fuCGz1eI5kIeajt6LNyaO9cYOrPDUanByzUdhi1LUuRQWW/38dZHeHCY6/y35dlclJlGHI6Ao/yEGMgA6AqMmllfTViWsy4TYJ4uJ9fMNJYyWaG2Khhjz6GX7t5KaqI7RP7ymHfHpkHpRRmCS8rpLjpWSEq6DlDIp1Ycjb/Gjl7cVH7cxMa5a79h+vEfblIHAesmRWKfsSXNAuLeyAAHUQA5fdg1msR9jQX3q9KXov8nk1acKF/vgGIUY2T1B4YDNzuiObs3ABjF1T+MF6d3QuzQ9uJPiyUI+wtOX6nNa8VNT2vB9PqN11quIJqBMlrEghWb5422FveBi7mbPloLmlqdPc9kQ8yVsED6AC048xcYvUzaEFb2sF/Z6XVcSoH8VY+U4kr0pkyVyzDB4WQ7SexUwZVhSk2FDqVOMq7vP1v2ZSCNOSFVJ3LAYK16VQ30f1X45CqN4AEdrR6YFgF51Hx455NhZeILxHz547+hZ/Ho8fndB SwjnAl9U i/SZsS4wwtdqMOJC6+vJwRwcv+sVj0jfunyfrLTYeYTW7veDfdMD6QSaRUwrkQQCvj+iYqfEEoAftTV3T5qNsnkmE8IPjEW6EuY5OYKf5+F/hfA0t8mEdHEyd+KVx+cjbAwD/YV6cuGnRlCde0EtrtO6T2YSrKyVsyJznNaAArmHcofUOLeICiOYk4VyOSJbdHya55zHNaFIj9irWmzbA1MTwdw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Use a folio in insert_page_into_pte_locked(), which is preparetion for converting mm counter functions to take a folio. Signed-off-by: Kefeng Wang --- mm/memory.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/mm/memory.c b/mm/memory.c index fbed32a09fec..dd3760988e02 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1854,10 +1854,13 @@ static int validate_page_before_insert(struct page *page) static int insert_page_into_pte_locked(struct vm_area_struct *vma, pte_t *pte, unsigned long addr, struct page *page, pgprot_t prot) { + struct folio *folio; + if (!pte_none(ptep_get(pte))) return -EBUSY; + folio = page_folio(page); /* Ok, finally just insert the thing.. */ - get_page(page); + folio_get(folio); inc_mm_counter(vma->vm_mm, mm_counter_file(page)); page_add_file_rmap(page, vma, false); set_pte_at(vma->vm_mm, addr, pte, mk_pte(page, prot)); From patchwork Mon Nov 6 15:49:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13447160 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 57FB3C4332F for ; Mon, 6 Nov 2023 16:07:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E0A5B8E000A; Mon, 6 Nov 2023 11:07:55 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id DB9BB8E0001; Mon, 6 Nov 2023 11:07:55 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C5C9A8E000A; Mon, 6 Nov 2023 11:07:55 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id B78B28E0001 for ; Mon, 6 Nov 2023 11:07:55 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 783A614081C for ; Mon, 6 Nov 2023 16:07:55 +0000 (UTC) X-FDA: 81428010510.08.1BF0C27 Received: from szxga08-in.huawei.com (szxga08-in.huawei.com [45.249.212.255]) by imf13.hostedemail.com (Postfix) with ESMTP id 2BFB82001C for ; Mon, 6 Nov 2023 16:07:50 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=none; spf=pass (imf13.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.255 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1699286872; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=113j91Z/CFywLfCKAhAiKZT7Zsk5X2/WxRslA1lSS0I=; b=1gWY/QwMPYSjlzwMVtuCxEo8pJm7OhBYaS3dkhhwutv1Iu7lqy3AkSc0X0Z0Q+0PRvuu1o x/GNhSv0j42cwjI2e4QIokQrXg3FlRDFFtlbiHhksz4i/wzzgQwsJfSKWqJrTZ+vUJOSeD GQAIqs8K/L3AbSw5VI+ieCpAUCcl564= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=none; spf=pass (imf13.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.255 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1699286872; a=rsa-sha256; cv=none; b=puzttEYSwhqg8BZB991Rwn+1Lg2L3Z4SZ8zy8Uiq3u/P7Az7UpbRCiJIYsvWGgLp8vltdV JQRsGPppYLyMTeni9X7hLAa2yd+H2Wa95D+fNaibpnvVOTHn3WFEmLMtH/9YW+V8By5dYw 71mMUTEFandwdg6kQDJ+m/Wjm3KGDTA= Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.54]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4SPG1X25YHz1P7x5; Mon, 6 Nov 2023 23:46:56 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Mon, 6 Nov 2023 23:50:02 +0800 From: Kefeng Wang To: Andrew Morton CC: , , Matthew Wilcox , David Hildenbrand , , Kefeng Wang Subject: [PATCH 09/10] mm: convert mm_counter() to take a folio Date: Mon, 6 Nov 2023 23:49:49 +0800 Message-ID: <20231106154950.3399469-10-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231106154950.3399469-1-wangkefeng.wang@huawei.com> References: <20231106154950.3399469-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 2BFB82001C X-Rspam-User: X-Stat-Signature: b71mbopuh7sawiwf6mgd17d74ts4ff85 X-Rspamd-Server: rspam01 X-HE-Tag: 1699286870-613555 X-HE-Meta: U2FsdGVkX186wiBuoNJEbuEqlF9gGjeYHYzTqrakf2mIK4Bxvwypcp5KtR1KKUdnNAB1Dr1RnZFbETF6OjZq30Pxx39IvR2r0L7EtZXbU3JSyvoKdeFhHWds32iLZwqRKZLmOAK9A2IlNGNjChnhlECfydn0JqpmifoctrWwLilxOHD6CF0DiodIU4UI4T80o/J1Cfs8gFXEgoRaSUY707vkY1vhFF/KcRfUtjNdcbNUWdyIJQHSew2cFKMH211YbSjoj3H2vcxY7Mg+hMlRTfrSYLKQ78wkeNSl9vCijGb+YXdu6wge8tJtakFu+dlywsYoHhNI2qd+QMBZuXW9prkYqOV5Gmwib8qSdibI5qtnNm1NZEaVgGXNIfXyie7ZhbIxkT6Yo15IOC4oakJr8v8QH152YgjYLBDm7h7/E61Y7omB5yRSzw/QszGGL4WrJWfSN6/Zd/S6XUeFDF+XlKDpJV4q7crOsXcKbJ5vvs5LBMmzjo+CJ0lHdPuRRFLLfRQV919fmxMjvynwwME05doSDY32buL4rrMeaDOCoPyqTLMdKE0wT3CA0nIhe3kmQUDQSF3XkWkU5kSkaPaiQnvfb6pG/EQeVwk1tczQDk37HS5bJ8oIKDLZwlKIfyV/n2roUc0IblJMvcHC5l2PwwtHcYz6FTi/pljoxwzMvnBNCUUdvVlHOPMfq3J9ZltwDKzmyu1JeQ1fUdcq3CcwMIM8lWxZRaNDzWhSy2vMHwpl4w8JcPrzZp47F0A0KS3ES4ul6xPgwmtIhKyPKDreKY7pCDiHpKPuArG2zyuxZGOpgeOEa50GZ2VB57DRCI4Q8+LvBbbIGDWEmhOo5GMiY4QvASSu8UWzEC8TleXVJjpiNab6vm9ecYrVV3q+YiRlcdKvPT9o2Fp4YFAhHwShie6MHKr5ysc4gTWG5AMmYY0U5pTMmTvbnR9Hkc1PL4CZe77tPj1pI8brL7MyrFn p3rn0FEm jFXGxUPGmxOHfYwxeHYrK6QmGn6HItIgZVb7E+SHop6mvj2Pb5Q5cx3J/1frXTdrEyeezML4IV1BZSBvzTnIQx0DDgOCA0yQUnYkGlbUDnYtHrVTJ/TC4zut9r2+DtzquJAUKOQCG64g91iPZNszNXNmqPLrrLNN/NOcR8KismMA2Y51hM8o3BVsAgY0MREpCEsc9 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Since all mm_counter() callers with a folio, let's convert mm_counter() to take a folio. Signed-off-by: Kefeng Wang --- arch/s390/mm/pgtable.c | 2 +- include/linux/mm.h | 6 +++--- mm/memory.c | 10 +++++----- mm/rmap.c | 8 ++++---- mm/userfaultfd.c | 2 +- 5 files changed, 14 insertions(+), 14 deletions(-) diff --git a/arch/s390/mm/pgtable.c b/arch/s390/mm/pgtable.c index 2f946b493fff..54b184648db6 100644 --- a/arch/s390/mm/pgtable.c +++ b/arch/s390/mm/pgtable.c @@ -732,7 +732,7 @@ static void ptep_zap_swap_entry(struct mm_struct *mm, swp_entry_t entry) else if (is_migration_entry(entry)) { struct folio *folio = pfn_swap_entry_to_folio(entry); - dec_mm_counter(mm, mm_counter(&folio->page)); + dec_mm_counter(mm, mm_counter(folio)); } free_swap_and_cache(entry); } diff --git a/include/linux/mm.h b/include/linux/mm.h index 418d26608ece..fea78900bf84 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2591,11 +2591,11 @@ static inline int mm_counter_file(struct page *page) return MM_FILEPAGES; } -static inline int mm_counter(struct page *page) +static inline int mm_counter(struct folio *folio) { - if (PageAnon(page)) + if (folio_test_anon(folio)) return MM_ANONPAGES; - return mm_counter_file(page); + return mm_counter_file(&folio->page); } static inline unsigned long get_mm_rss(struct mm_struct *mm) diff --git a/mm/memory.c b/mm/memory.c index dd3760988e02..3ffef84dd7bb 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -803,7 +803,7 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct mm_struct *src_mm, } else if (is_migration_entry(entry)) { folio = pfn_swap_entry_to_folio(entry); - rss[mm_counter(&folio->page)]++; + rss[mm_counter(folio)]++; if (!is_readable_migration_entry(entry) && is_cow_mapping(vm_flags)) { @@ -835,7 +835,7 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct mm_struct *src_mm, * keep things as they are. */ folio_get(folio); - rss[mm_counter(page)]++; + rss[mm_counter(folio)]++; /* Cannot fail as these pages cannot get pinned. */ BUG_ON(page_try_dup_anon_rmap(page, false, src_vma)); @@ -1466,7 +1466,7 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, if (pte_young(ptent) && likely(vma_has_recency(vma))) folio_mark_accessed(folio); } - rss[mm_counter(page)]--; + rss[mm_counter(folio)]--; if (!delay_rmap) { page_remove_rmap(page, vma, false); if (unlikely(page_mapcount(page) < 0)) @@ -1496,7 +1496,7 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, * see zap_install_uffd_wp_if_needed(). */ WARN_ON_ONCE(!vma_is_anonymous(vma)); - rss[mm_counter(page)]--; + rss[mm_counter(folio)]--; if (is_device_private_entry(entry)) page_remove_rmap(page, vma, false); folio_put(folio); @@ -1511,7 +1511,7 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, folio = pfn_swap_entry_to_folio(entry); if (!should_zap_folio(details, folio)) continue; - rss[mm_counter(&folio->page)]--; + rss[mm_counter(folio)]--; } else if (pte_marker_entry_uffd_wp(entry)) { /* * For anon: always drop the marker; for file: only diff --git a/mm/rmap.c b/mm/rmap.c index 7a27a2b41802..7a563490ce08 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1678,7 +1678,7 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma, set_huge_pte_at(mm, address, pvmw.pte, pteval, hsz); } else { - dec_mm_counter(mm, mm_counter(&folio->page)); + dec_mm_counter(mm, mm_counter(folio)); set_pte_at(mm, address, pvmw.pte, pteval); } @@ -1693,7 +1693,7 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma, * migration) will not expect userfaults on already * copied pages. */ - dec_mm_counter(mm, mm_counter(&folio->page)); + dec_mm_counter(mm, mm_counter(folio)); } else if (folio_test_anon(folio)) { swp_entry_t entry = page_swap_entry(subpage); pte_t swp_pte; @@ -2075,7 +2075,7 @@ static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma, set_huge_pte_at(mm, address, pvmw.pte, pteval, hsz); } else { - dec_mm_counter(mm, mm_counter(&folio->page)); + dec_mm_counter(mm, mm_counter(folio)); set_pte_at(mm, address, pvmw.pte, pteval); } @@ -2090,7 +2090,7 @@ static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma, * migration) will not expect userfaults on already * copied pages. */ - dec_mm_counter(mm, mm_counter(&folio->page)); + dec_mm_counter(mm, mm_counter(folio)); } else { swp_entry_t entry; pte_t swp_pte; diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index 96d9eae5c7cc..9a6759fa9b06 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -124,7 +124,7 @@ int mfill_atomic_install_pte(pmd_t *dst_pmd, * Must happen after rmap, as mm_counter() checks mapping (via * PageAnon()), which is set by __page_set_anon_rmap(). */ - inc_mm_counter(dst_mm, mm_counter(page)); + inc_mm_counter(dst_mm, mm_counter(folio)); set_pte_at(dst_mm, dst_addr, dst_pte, _dst_pte); From patchwork Mon Nov 6 15:49:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13447145 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E26B7C4332F for ; Mon, 6 Nov 2023 15:51:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 810FA8E0006; Mon, 6 Nov 2023 10:51:01 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7C0A58E0001; Mon, 6 Nov 2023 10:51:01 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6B04D8E0006; Mon, 6 Nov 2023 10:51:01 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 5CB958E0001 for ; Mon, 6 Nov 2023 10:51:01 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 142A5160706 for ; Mon, 6 Nov 2023 15:51:01 +0000 (UTC) X-FDA: 81427967922.25.014D6DC Received: from szxga08-in.huawei.com (szxga08-in.huawei.com [45.249.212.255]) by imf25.hostedemail.com (Postfix) with ESMTP id 85665A001B for ; Mon, 6 Nov 2023 15:50:58 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf25.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.255 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1699285859; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6ZbcxIplLz2/cFjSStN8lhzNVnpkbIW45XoB8UcboTY=; b=RdGERe1Nwtd87SwaWVXyWl6zbITUb9ru7INfuGYD9YM9U7TcyR4mDikePNq9xO6moSDrdy +qUnVmkUwYnOc9i5X+FprhkyIAnY7eZJPRO9zZyBdSVvUJ0yuVsnrSevaovkENOtWvvNcg PqkttwVo8f5AUEL+YOETFDYGFYLdq0k= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf25.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.255 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1699285859; a=rsa-sha256; cv=none; b=KcPJHXE/EFHYuIw884Y9mZ89n+gL/xctsDX4GWtDZ0pSNFSgWgA0F4npGJulSUXX6JEg6V W+ntCydVffop8pIHA9Q/2iNMFRzZ6Clki1a1df7wUdhFBEK4iba6CsSwndcan67eONgmst FoyX7ouznq89e+1i1QPP+5wQmu7CW/0= Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.54]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4SPG1b3kWqz1P7xD; Mon, 6 Nov 2023 23:46:59 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Mon, 6 Nov 2023 23:50:02 +0800 From: Kefeng Wang To: Andrew Morton CC: , , Matthew Wilcox , David Hildenbrand , , Kefeng Wang Subject: [PATCH 10/10] mm: convert mm_counter_file() to take a folio Date: Mon, 6 Nov 2023 23:49:50 +0800 Message-ID: <20231106154950.3399469-11-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231106154950.3399469-1-wangkefeng.wang@huawei.com> References: <20231106154950.3399469-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 85665A001B X-Stat-Signature: hbhetac3mm1gh1xa71ehweti35ogksgf X-HE-Tag: 1699285858-990559 X-HE-Meta: U2FsdGVkX18safudIAlJF2OnTaZPYbrJGUZCvHp3CGcq5O6D6RAj6fQXDUMaUSkGG0HPMdn5pq15sz+61L9ZKJOlE4F4p3WLVUcOJ5l16rqf7Offr7SeWJAMJUSiXtcoDPi5xfAyloPgey0/5ESF8Ti7PGqzgUJ9a4Me0aNcawfs8fbavU57tFQycPYZ27rhec0XWUYbIzrHgZBvu2ysQQjodYufAfTF7hVL36G+hv2r1KD125Gp4B7ebbhbdoN752oSbVxmUykhtBpU0T73lfYSduDCWtDOI5dufRO5ACAGjoqJz1U7i42PYQvRxwvVn40vfUkoa/tcTYMlpUAYE7wyMI7r1OeUBeEJjV57g4MF34LX6wWWJCiHyRsJvRZqdhxxkWeduzb217I8RD+YetzAZJl8tkL5tq9mrR8+fQ9jOoePTDhjbDR9DUEzsNTKV7h2OCkQEeckj84AwB8KpOaNSa9MVZqUJLpg/RG35Eus5/M4BWQbfToP7egJHuwdTxpgjzKPMIWioYsTd6WG7WUG+Z56UUfSom7zaCB+kDmx3HeXlnwUWsNpWp7lO9e0vk4hA6rWhSKd8xz5HNw3rblir4Aghe86n/0aF1H94tZnea4zXOAL5EacpyX9YezZFRGRk0uajGCHuWDkK3DcLNaHcCLSgcHCpSOPSJAlOPmgKz07npFm+v6suJ83j/bVPixYqKjCnPVNUVxyLqJzvaU/mjpcAkgzTYT4QImWl1fDr+TivQxuMUG/Zt8Aht0KX+FGs6aGASCmmma5A+dQyi2NfZ54c/AyPm49V9qAnup/srttVqdPEWLY/vFVbbOd0x6caoI2XYHGEHrfb0mObozrqpJjiY9SZmMseXFP/aSxyGrPhQkrIfrWnNJFYCTODGnYMDeLECSxsCFBFGiH7PCSD7dmsfUDAV9YgIJeAEi70szotz91tUHK5c6POPl3I+WAHpGBJVTXq5CfKFN LzEoV4xh QKnyvdpv1YneLJ7pnuB7JXjmE05Tuu6Pic7wVW4QO5ruoCYabTuGVh1noaoEQt+90bbhpZGUzVGU05LHrtT8aLjQLhRcOzbe4juGhxl+KlUtCf2EtVxWdZt9YTLxwDCpGb9BaHWNiODK9ZXqvmTFeKhpUS0515f0IjsrcRNO3L3lFzsUemO8y5P7P73Z2Ozh4vtmz X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Since all mm_counter_file() callers with a folio, let's convert mm_counter_file() to take a folio. Signed-off-by: Kefeng Wang --- include/linux/mm.h | 8 ++++---- kernel/events/uprobes.c | 2 +- mm/huge_memory.c | 5 +++-- mm/khugepaged.c | 4 ++-- mm/memory.c | 10 +++++----- mm/rmap.c | 2 +- 6 files changed, 16 insertions(+), 15 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index fea78900bf84..95573065a46b 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2583,10 +2583,10 @@ static inline void dec_mm_counter(struct mm_struct *mm, int member) mm_trace_rss_stat(mm, member); } -/* Optimized variant when page is already known not to be PageAnon */ -static inline int mm_counter_file(struct page *page) +/* Optimized variant when folio is already known not to be anon */ +static inline int mm_counter_file(struct folio *folio) { - if (PageSwapBacked(page)) + if (folio_test_swapbacked(folio)) return MM_SHMEMPAGES; return MM_FILEPAGES; } @@ -2595,7 +2595,7 @@ static inline int mm_counter(struct folio *folio) { if (folio_test_anon(folio)) return MM_ANONPAGES; - return mm_counter_file(&folio->page); + return mm_counter_file(folio); } static inline unsigned long get_mm_rss(struct mm_struct *mm) diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index 435aac1d8c27..ce251e3a4ae6 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -188,7 +188,7 @@ static int __replace_page(struct vm_area_struct *vma, unsigned long addr, dec_mm_counter(mm, MM_ANONPAGES); if (!folio_test_anon(old_folio)) { - dec_mm_counter(mm, mm_counter_file(old_page)); + dec_mm_counter(mm, mm_counter_file(old_folio)); inc_mm_counter(mm, MM_ANONPAGES); } diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 78a00fe22c2d..88420d067477 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1742,7 +1742,8 @@ int zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, } else { if (arch_needs_pgtable_deposit()) zap_deposited_table(tlb->mm, pmd); - add_mm_counter(tlb->mm, mm_counter_file(page), -HPAGE_PMD_NR); + add_mm_counter(tlb->mm, mm_counter_file(folio), + -HPAGE_PMD_NR); } spin_unlock(ptl); @@ -2143,7 +2144,7 @@ static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd, page_remove_rmap(&folio->page, vma, true); folio_put(folio); } - add_mm_counter(mm, mm_counter_file(&folio->page), -HPAGE_PMD_NR); + add_mm_counter(mm, mm_counter_file(folio), -HPAGE_PMD_NR); return; } diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 064654717843..39393f4262b2 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1630,7 +1630,7 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, /* step 3: set proper refcount and mm_counters. */ if (nr_ptes) { folio_ref_sub(folio, nr_ptes); - add_mm_counter(mm, mm_counter_file(&folio->page), -nr_ptes); + add_mm_counter(mm, mm_counter_file(folio), -nr_ptes); } /* step 4: remove empty page table */ @@ -1661,7 +1661,7 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, if (nr_ptes) { flush_tlb_mm(mm); folio_ref_sub(folio, nr_ptes); - add_mm_counter(mm, mm_counter_file(&folio->page), -nr_ptes); + add_mm_counter(mm, mm_counter_file(folio), -nr_ptes); } if (start_pte) pte_unmap_unlock(start_pte, ptl); diff --git a/mm/memory.c b/mm/memory.c index 3ffef84dd7bb..bf4ea31150a9 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -961,7 +961,7 @@ copy_present_pte(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma, } else if (page) { folio_get(folio); page_dup_file_rmap(page, false); - rss[mm_counter_file(page)]++; + rss[mm_counter_file(folio)]++; } /* @@ -1861,7 +1861,7 @@ static int insert_page_into_pte_locked(struct vm_area_struct *vma, pte_t *pte, folio = page_folio(page); /* Ok, finally just insert the thing.. */ folio_get(folio); - inc_mm_counter(vma->vm_mm, mm_counter_file(page)); + inc_mm_counter(vma->vm_mm, mm_counter_file(folio)); page_add_file_rmap(page, vma, false); set_pte_at(vma->vm_mm, addr, pte, mk_pte(page, prot)); return 0; @@ -3170,7 +3170,7 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf) if (likely(vmf->pte && pte_same(ptep_get(vmf->pte), vmf->orig_pte))) { if (old_folio) { if (!folio_test_anon(old_folio)) { - dec_mm_counter(mm, mm_counter_file(&old_folio->page)); + dec_mm_counter(mm, mm_counter_file(old_folio)); inc_mm_counter(mm, MM_ANONPAGES); } } else { @@ -4363,7 +4363,7 @@ vm_fault_t do_set_pmd(struct vm_fault *vmf, struct page *page) if (write) entry = maybe_pmd_mkwrite(pmd_mkdirty(entry), vma); - add_mm_counter(vma->vm_mm, mm_counter_file(page), HPAGE_PMD_NR); + add_mm_counter(vma->vm_mm, mm_counter_file(folio), HPAGE_PMD_NR); page_add_file_rmap(page, vma, true); /* @@ -4426,7 +4426,7 @@ void set_pte_range(struct vm_fault *vmf, struct folio *folio, folio_add_new_anon_rmap(folio, vma, addr); folio_add_lru_vma(folio, vma); } else { - add_mm_counter(vma->vm_mm, mm_counter_file(page), nr); + add_mm_counter(vma->vm_mm, mm_counter_file(folio), nr); folio_add_file_rmap_range(folio, page, nr, vma, false); } set_ptes(vma->vm_mm, addr, vmf->pte, entry, nr); diff --git a/mm/rmap.c b/mm/rmap.c index 7a563490ce08..9e3d0eff8b05 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1801,7 +1801,7 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma, * * See Documentation/mm/mmu_notifier.rst */ - dec_mm_counter(mm, mm_counter_file(&folio->page)); + dec_mm_counter(mm, mm_counter_file(folio)); } discard: page_remove_rmap(subpage, vma, folio_test_hugetlb(folio));