From patchwork Mon Jul 31 07:48:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13333950 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A433DC001DE for ; Mon, 31 Jul 2023 07:39:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D8CBC28001C; Mon, 31 Jul 2023 03:39:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D3CB228001A; Mon, 31 Jul 2023 03:39:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BDBAB28001C; Mon, 31 Jul 2023 03:39:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id A2B2F28001A for ; Mon, 31 Jul 2023 03:39:25 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 7BD35401A0 for ; Mon, 31 Jul 2023 07:39:25 +0000 (UTC) X-FDA: 81071106690.08.AA65F2B Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by imf27.hostedemail.com (Postfix) with ESMTP id 49C6740015 for ; Mon, 31 Jul 2023 07:39:22 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf27.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1690789163; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Yj4az2ALhhEyvky0TbaSWKAJzQx3pWTWZySo+ZOZ/q0=; b=DyuIud4t4rzv4lW4lkh6gULsacrYrRm1L8uc4EkUSVI2ZP3Ie2JjNPP0/IenQazDS47gs7 /TBlLVwv11280AQlD6wPXd9ri+L42ywDjjbffBXLnCkZV5GbshTv9LGtFvfyvTuBvjfyXY DRYwxN17Hogldx6La5pHaunUljzSwng= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf27.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1690789163; a=rsa-sha256; cv=none; b=dTzfGQT0i/FYxVmHGtkzH4aDefjcIE8S3ke5zBjyGqqCsfm7YW6lqFA+vq7iO3Z9FkWLG1 HAYLjL51l40OhBBJOOYNFXNhGjb1Kw56cALwlijBN9k13Y51KyZJrU+EBotPfcI5LVAC8i 3WUipPLxXWSKHBzR2+E1iyo2eW4SUxI= Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.57]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4RDqm627jmzNmP9; Mon, 31 Jul 2023 15:35:50 +0800 (CST) Received: from localhost.localdomain.localdomain (10.175.113.25) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Mon, 31 Jul 2023 15:39:14 +0800 From: Kefeng Wang To: Andrew Morton , Catalin Marinas , Will Deacon , Mike Kravetz , Muchun Song , Mina Almasry , , , , , CC: , , Kefeng Wang Subject: [PATCH 1/4] mm: hugetlb: use flush_hugetlb_tlb_range() in move_hugetlb_page_tables() Date: Mon, 31 Jul 2023 15:48:26 +0800 Message-ID: <20230731074829.79309-2-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230731074829.79309-1-wangkefeng.wang@huawei.com> References: <20230731074829.79309-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.113.25] X-ClientProxiedBy: dggems702-chm.china.huawei.com (10.3.19.179) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 49C6740015 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: ntf6qnh4if8sp8ymfjstztyr3h44ihzx X-HE-Tag: 1690789162-24396 X-HE-Meta: U2FsdGVkX1/b6kPo8dowcR8g+C47ql1/8ag0ltB7eP4HGWTOHbQdrmh4oNJ9aShdtWLb7g2Ll+7etHsyNM0aoaK4JFEqYW3dx9aSUPKXe3zH5f0VYiofR/kGKm4EV0Ow5e07ugQIapSVewdY7QYsytF5RtXeu31yklxTvCRbTRtbW7Ad2yWtIprpRhLUpxA3maxeEOcCAlHWZN7DEiNwmHXCi3Aj+kPRYPsNa0T8e1pw+2ZFmdepVPosgdl5qRypBgxN/DieTiv7f6F0Jb3mx54wU0Mq53A6qvetqcpKhbLyrjcbuXfF/cSN/MXns6vnWD1jHvVvpZCEsjiaGhOiA4ct25m9B7DFVVwwagPuQXFrOSzOQMiBITS/Twwl2BptZVRRzNlp0qSptDembxa2Sjj386pR6jNBGC7A9D2Of35RCJwwV4z8OwPVgbrbg9xFaYuNF/UCuURexmgZXt2NpVqRBW9Lw6ihH0OnXyFmslONHcSt3k4PNYe22YZds2s8yhJxbRCcuPaUq804KMDk+s+wxZJizmmmWxsiT9VFh3VbaGkWdC3cN2NqwNY4jEolsQkQkucad/3C1vm9S96Yb0P7QP8HPyzPKugloubg9msWLkhBXB3wOmL6VYvS93GI7epnCIIoq+Yrx7cs1EaqhHJcXpHppuI9Z+hECk6SxrxX+Ub3Y952dWGolIjEkl8qYU1lnhacHlZEVP01y/y/R7oYzX+KAYzyEm0hVQ48zxQnmZxOi0Mxhd+GlShRO2SSBTD/86KXe86OUwIRAVLwP2N+JSlS1wnYX1vHh2izxz6TXvtSmr5eqJI1Z15oM5tCbSCNaWwe3jM44RjtiRKr4ATjkpJEp4fKDH7G4ggVPvhINgzZouatfcqGYupRBjp4ieehJQqACckcsyBmxn07LNfjPUsqfjMvvfwsVWgvqnhTQDcJyCNMllbnhZK0DvstUb6egb6MED3E8xQNFFX 4TLukUSz jdBZoOrfJE7ojMpbJ/f3WtcSzh6D/RFrkE9LBwC4HE6ro8Hbd9FUTASW9NfrCOdagGXEQK71ejEammYyhBFwpOMhkRqhRVZBc+ryHmE9ksH5qWy+LWHgqJIkqSfmP0tIYIV+AlSRTgGlQRaSo0QrZURYW6K2Wgn1eFbmLoN80blAbjlvLlVYz4JwCZ00GUOOi2idR X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Archs may need to do special things when flushing hugepage tlb, so use the more applicable flush_hugetlb_tlb_range() instead of flush_tlb_range(). Fixes: 550a7d60bd5e ("mm, hugepages: add mremap() support for hugepage backed vma") Signed-off-by: Kefeng Wang Reviewed-by: Mike Kravetz Acked-by: Muchun Song --- mm/hugetlb.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 64a3239b6407..ac876bfba340 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -5281,9 +5281,9 @@ int move_hugetlb_page_tables(struct vm_area_struct *vma, } if (shared_pmd) - flush_tlb_range(vma, range.start, range.end); + flush_hugetlb_tlb_range(vma, range.start, range.end); else - flush_tlb_range(vma, old_end - len, old_end); + flush_hugetlb_tlb_range(vma, old_end - len, old_end); mmu_notifier_invalidate_range_end(&range); i_mmap_unlock_write(mapping); hugetlb_vma_unlock_write(vma); From patchwork Mon Jul 31 07:48:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13333949 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 92CEFC001DF for ; Mon, 31 Jul 2023 07:39:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 32CA1280019; Mon, 31 Jul 2023 03:39:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 267D628001A; Mon, 31 Jul 2023 03:39:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CC62A280019; Mon, 31 Jul 2023 03:39:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 9DEBA280019 for ; Mon, 31 Jul 2023 03:39:24 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id B9DC3B1F2F for ; Mon, 31 Jul 2023 07:39:23 +0000 (UTC) X-FDA: 81071106606.23.78E4BBA Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by imf28.hostedemail.com (Postfix) with ESMTP id 43BD7C0009 for ; Mon, 31 Jul 2023 07:39:20 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=none; spf=pass (imf28.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1690789161; a=rsa-sha256; cv=none; b=rSFK3M4sf+6zybGsif+hqs03VQBqvKbnAK8L6aW55sRYTHxBuDdTpVV35ZTseimfHYo6KF yjLWWAW0dB+UN8nVawRXFXiNriTtlBlB4MDJ2uDpQF7Vdz21p3XGr6FIXkPItv+HBDTtul 4mcwUT4saGW30A+LzRM9KHCRTuV1pFY= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=none; spf=pass (imf28.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1690789161; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QnjtTek5fZ/4QtwvL41dNaJX0C5UFKPKxH+gNlzHWXo=; b=5UvczpVsdPnCDoJm+EKozoCCCYFkBGt+X5Rzq2DfFRS3IWbbUpqCqRsdw593JutFwClaTx 9h5q0Lv51V4keGc5b4AbsWyqJ05UryqwdIDxD/BbC6bpjonDa+CiPKS5k0hSexSwLNspJn W5JgWEVbcsitEItRXCwjJ44ZOg++zV4= Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.54]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4RDqmD2YBwztQSD; Mon, 31 Jul 2023 15:35:56 +0800 (CST) Received: from localhost.localdomain.localdomain (10.175.113.25) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Mon, 31 Jul 2023 15:39:15 +0800 From: Kefeng Wang To: Andrew Morton , Catalin Marinas , Will Deacon , Mike Kravetz , Muchun Song , Mina Almasry , , , , , CC: , , Kefeng Wang Subject: [PATCH 2/4] mm: mremap: use flush_pmd_tlb_range() in move_normal_pmd() Date: Mon, 31 Jul 2023 15:48:27 +0800 Message-ID: <20230731074829.79309-3-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230731074829.79309-1-wangkefeng.wang@huawei.com> References: <20230731074829.79309-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.113.25] X-ClientProxiedBy: dggems702-chm.china.huawei.com (10.3.19.179) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 43BD7C0009 X-Stat-Signature: st3d6sanqg4xr9jcsazgewoef1htyp6s X-HE-Tag: 1690789160-354093 X-HE-Meta: U2FsdGVkX19JYDY0QpkENLVZr0thHjOwsvkKG3HN8hQdFb0Mu/e7E/dF1R3PD96IFMFWQNBbzi/6zoDFEf0EwOmiA6HCp8Ah79rnxE7yQ+zCc3PQT/e65+UPqXQI3ix15Xj7hQNy/Le2FYbG48704pP4YFg9ffm5A6XDQkPSnS5XhJwzMerGZBFBb+KlK8TAMTkyLxH73RylJLtcBwpEQdZ0E2UqoGpJV8gFfVdcjx+WqlLkRJ8/P8VlB6Hd22qISbZywjcu8xpTeGDJzqcBvTHVRoxKQS1+tMrHiZoNrIlzxbixKoGlE6NikIYK10vbY1oJkXhlLYrQM9PW6NJip+bZnUZHxJaTHVum34Xo18S9fjVtHOM6UvQQW0ON7y1vdUpMNZYoEPjNoFjAsEZNm9tlbSiGxIIuoI5U1lwad7RA5ouEnjbGZnyARK5n4/cy0+xz5YchoNKCkEwDqevVSJ6t7Q4AROdkGSgz0713F0y79HinUw2AceXEWxUbiMexbuDlnQAYVlI/ZXZ8pqVD/feZm6qp3BjCdkXGj5Bg8mgEUDw07CbeXP5BTuc8eln2sT82HfDNpemS5SaVrBTPpvKikWXgwi3COLxEEL65FsMZTzhQX/FTn5JsIG2bzWzqBG6pxN/kaBzPraq7MlYxtLAQ3cnB9lAtsB26fqDLoiI9ZOjWiHiCOI1SoZTQz3DpbWg5ba9/kaYGBN6a48wHIQRIRt4JTW9aDm37rjJ0a/qcPJ3vBU3An4nPyWChBtB86Dh5vbBmF8GPliLYD3vbqKD/2G1Kigd9iILQ2gEDP+LxS6TYJBQiQ2gxosxtgH/Iar9Xb58Q+FvJgDdKwQN6IAWBdNHeMIcj4Sg5bxD+bSAUD50NJ2gqS2srRv7p8YwqP4CaRgcJg8/f6OeVJFZFRNBLpHzhN9Wm226aQ2/i4T7UYDfiYT0RirgVwqdYffGbtIoqtIhkluH7lTlTUN5 yt6VaQJM GL7uN2N5/T3E1kS+FP1s3xIdfGo4fr8U/4UZhWG5FLt3izhYmgJp90Lgq0X+bva48EIrQs0i3Zxu9WSgFqsCuV5HWocAb7CkW9xVaRpXPOrUQVQo9oJc2XQVSNf+kysKdHP21oSeOjQlNMMfrdhhVP9G7CdXwYvcIc+naxVQWAsgE2k+SOCu5GoO9OBLxU2Nr/jbz X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Archs may need to do special things when flushing thp tlb, so use the more applicable flush_pud_tlb_range() instead of flush_tlb_range(). Fixes: 2c91bd4a4e2e ("mm: speed up mremap by 20x on large regions") Signed-off-by: Kefeng Wang --- mm/mremap.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/mremap.c b/mm/mremap.c index 11e06e4ab33b..1883205fa22b 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -284,7 +284,7 @@ static bool move_normal_pmd(struct vm_area_struct *vma, unsigned long old_addr, VM_BUG_ON(!pmd_none(*new_pmd)); pmd_populate(mm, new_pmd, pmd_pgtable(pmd)); - flush_tlb_range(vma, old_addr, old_addr + PMD_SIZE); + flush_pmd_tlb_range(vma, old_addr, old_addr + PMD_SIZE); if (new_ptl != old_ptl) spin_unlock(new_ptl); spin_unlock(old_ptl); From patchwork Mon Jul 31 07:48:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13333947 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E633C04A94 for ; Mon, 31 Jul 2023 07:39:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CC99728001B; Mon, 31 Jul 2023 03:39:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C77CA28001A; Mon, 31 Jul 2023 03:39:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ACBC728001B; Mon, 31 Jul 2023 03:39:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 9CC6D280012 for ; Mon, 31 Jul 2023 03:39:24 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 5B1ECA0440 for ; Mon, 31 Jul 2023 07:39:24 +0000 (UTC) X-FDA: 81071106648.16.2F37875 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by imf23.hostedemail.com (Postfix) with ESMTP id B71EE140014 for ; Mon, 31 Jul 2023 07:39:21 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf23.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1690789162; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=y2xwMHmp2+AF+LplUKlYJQ4KHQF98DXrKqXWA6DlewM=; b=3O1fgeBJ7uyddqKdQgFP0UzZPrcWZTclja0aTIdma3KCC6vWrwdmkQAzcixuHvuGCFK0je eghn4sjjqiD0uZIu2cheaNcqSorzwWMUpE2WE/igXlvH1IxefIJ1YyYLlRdSgRTiUgoPoT 9Xlz5C90TOrNhBYN/tJuHnk9ffFXvPc= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf23.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1690789162; a=rsa-sha256; cv=none; b=vMemt5EOtqjkEWEW8EJ+S3/WDHcWaJkX59Xx4/EafesTw+7SlEM03MRQUGTNoe6WFtOdFb BMhm/m5twxiXl8pNsUYVAQ18bjD2zdjJ0hd+CWRu/+J+wQ3O+FLB8d2lB+XgpArXtW+Sj8 xu0UiE+1Zjt+m4HaiFYejlu5/lwuPF4= Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.54]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4RDqm74d7YzNmbC; Mon, 31 Jul 2023 15:35:51 +0800 (CST) Received: from localhost.localdomain.localdomain (10.175.113.25) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Mon, 31 Jul 2023 15:39:15 +0800 From: Kefeng Wang To: Andrew Morton , Catalin Marinas , Will Deacon , Mike Kravetz , Muchun Song , Mina Almasry , , , , , CC: , , Kefeng Wang Subject: [PATCH 3/4] mm: mremap: use flush_pud_tlb_range in move_normal_pud() Date: Mon, 31 Jul 2023 15:48:28 +0800 Message-ID: <20230731074829.79309-4-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230731074829.79309-1-wangkefeng.wang@huawei.com> References: <20230731074829.79309-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.113.25] X-ClientProxiedBy: dggems702-chm.china.huawei.com (10.3.19.179) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Rspam-User: X-Stat-Signature: w45g6ksqp3x9ga3untfsahbwagqqjdhs X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: B71EE140014 X-HE-Tag: 1690789161-854567 X-HE-Meta: U2FsdGVkX18JmyR6ii3zu7n+b+dsx4jLHCBzPfXdQzAA7nKKHA1UwChgFe/vUJtV44hEFNZ7kzHzVtWwAGDZibK+mXKzSzc8TyiltRUkOGQ0mZfril/l/jlBznZwyD3GMOVDpkAFeoQGeeEHH+6miWKTKk7ynQhlVWXCW1cDc6YhmMwo9Aryqi4NZTxkFMWJMwQxNWNdOufhXi1k9t22b/aoA6H8n1iBc6OJG5EV9QNi5OURpC/EydDAKkyOxOdFzxWttqPDU39WNTa9DK7/6hciPJQ+l/5fpwFV9qccGuzZTIAqShj5MzNgEbTNxmo5ZafPv1eKmF3+X49UfPKldIlfZGot3u3lyLMhPCo9egaufxpTTKrLIP4OiJtwrrLY5E3y+LOixQo1xs3rgd2l+QGgm5SN4gFCjsnNqBk3rUtu2n5NZCkyes7EAqdDi6SXutXwPAxxuu247rn+ZLmvqx4AmhOFBoGT57Q/BI/DZBFqHzAPhH9ni5w8yeli1lgDrSFG+zbbmDqiAvSaxzNGG+32T6Te9jVCOwQvZX4o04Gtwxro3T1t/KBRKmOrplWKEQ0pTDZck95fqfQwTdviLf6x44EOm6k9fzMxEOt1Piq9aUS0xl7N7ovSlBa9kFKiUasySxToIyrldqYsA+579UXJcej4sJNRfhpnCZNr7sG7hswAeyd6E/AnnezLPqG2L+BVfGidnahmrDFxk4exniMXpdhGp3d/TafmCXDKc5CUTfROJKwR4PgCQtSMQwTKqv9+FGO4kyY49Wa2Cbd55Ii0AoYBf807i8bEUf1xT4LP0GFVxfvwhsF2oGehJt79kq2tTsXKOBImmGKTwTr9fsuHSm7HIYtLgRmShg3xEG6RVS8XQJE5x1r7shyDumLcJw+fkdLkoVqRjAsRexsz4uvkQ9TEzXkS/J1YkGREcECo8nPeXyssxN8XRhq9GW22bcEyyJvoQLjyn7ccTbE nykUqAfv KLAveMuylvjkWW4R8do1H92eK+AW4oIZPUp4PmyxBZ0hcqTq8dFndWZ5vRTy/cdXSqtsa4XQgnhqTdS4mLCGwl1EZYb5b5wpBGgDNGxjdR1LHGdJcmG/HCHGljzV7HN3OGlDIkoWJtdt0lJJB0M2a2cW6NLIcrpoNiYFXGq5mgnq9WZA6vrGOH/yVfvn3jxvNokSJ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Archs may need to do special things when flushing thp tlb, so use the more applicable flush_pud_tlb_range() instead of flush_tlb_range(). Fixes: c49dd3401802 ("mm: speedup mremap on 1GB or larger regions") Signed-off-by: Kefeng Wang --- mm/mremap.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/mremap.c b/mm/mremap.c index 1883205fa22b..25114e56901f 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -333,7 +333,7 @@ static bool move_normal_pud(struct vm_area_struct *vma, unsigned long old_addr, VM_BUG_ON(!pud_none(*new_pud)); pud_populate(mm, new_pud, pud_pgtable(pud)); - flush_tlb_range(vma, old_addr, old_addr + PUD_SIZE); + flush_pud_tlb_range(vma, old_addr, old_addr + PUD_SIZE); if (new_ptl != old_ptl) spin_unlock(new_ptl); spin_unlock(old_ptl); From patchwork Mon Jul 31 07:48:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13333948 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 677CDC04E69 for ; Mon, 31 Jul 2023 07:39:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 01B3A280012; Mon, 31 Jul 2023 03:39:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E0D7428001C; Mon, 31 Jul 2023 03:39:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B8E38280012; Mon, 31 Jul 2023 03:39:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 9EA4128001A for ; Mon, 31 Jul 2023 03:39:24 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 4D01780107 for ; Mon, 31 Jul 2023 07:39:24 +0000 (UTC) X-FDA: 81071106648.17.FA86951 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by imf25.hostedemail.com (Postfix) with ESMTP id 0CF72A0019 for ; Mon, 31 Jul 2023 07:39:21 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf25.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1690789162; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Y3tb33D7Yj/stY7JznHb58iHPQ693NkVRx8ipzJMNtk=; b=q4ujG1j7nzX1PEmTyKIXltLRZRWnc9j4qdAWlSO6DMU/YfB5/3Fimn3K+QsZcZXhgusam3 zmWr8QRD6VfiLALGfMJp0p8j6n6tZfvneV3qwhV48Ix3lo91InVxASTrEhG/CWDNG9eUfn pI2v5eK/RfymW6HsyTun6ok64tUXVq4= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf25.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1690789162; a=rsa-sha256; cv=none; b=MAA0naQdtDzWhSD+HObCpLqWzyQIJYINW3nGNWAK/tTqRxzpdpVcDdUkQStq05pKI9DW4G aZgNmdBMm40FcuB2t9qACSENy5U4b7xK2XHHD55TixRWUVebkh4TgX4/Fqciq8+hw5hUnJ MZ/jOyoUvjrdFeBHtEykLSHEfkuZlmE= Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.53]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4RDqp763FHzVjdw; Mon, 31 Jul 2023 15:37:35 +0800 (CST) Received: from localhost.localdomain.localdomain (10.175.113.25) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Mon, 31 Jul 2023 15:39:16 +0800 From: Kefeng Wang To: Andrew Morton , Catalin Marinas , Will Deacon , Mike Kravetz , Muchun Song , Mina Almasry , , , , , CC: , , Kefeng Wang Subject: [PATCH 4/4] arm64: tlb: set huge page size to stride for hugepage Date: Mon, 31 Jul 2023 15:48:29 +0800 Message-ID: <20230731074829.79309-5-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230731074829.79309-1-wangkefeng.wang@huawei.com> References: <20230731074829.79309-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.113.25] X-ClientProxiedBy: dggems702-chm.china.huawei.com (10.3.19.179) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 0CF72A0019 X-Stat-Signature: diphinxmduncpprg5gbfn9ui63jt57qh X-Rspam-User: X-HE-Tag: 1690789161-613569 X-HE-Meta: U2FsdGVkX18o4/fJ0roERZ0w8tFOh0iXlNiKwl0tyugU4Zir/nxhQHraz5Vi62hqhpZMGt9LsB3jliByq5qd44PIWUPL8nKx0QSE4MvSSjGORzOxO94BXkz03htg9L8dXVrHH7rl7D3eG8cUGvTNbFYZ4m0luhsKY6L9f+qTeBDCfoNtja+f58V/rH2J3qb0Q45wCJzlwP75x6yONpAFPu+F8G8FuIJIivjPsKU7yzShwkWBWBlJ157/bagl0BVo3C9fC/6qVbmqkfDif1yexK/WWYx8q+d6HNfIh9IBppda42nt5uYCAfq9mRUGlDS7DhMZQ5ldP1m9GFNe/pWxarM4KJaT0vnakVbv3a8kXVEBhj6VjrMGKaifjH8xxtOG37UYl2aetPtxTvf4PB94hUzeqhEOLOCx2tYuwvWXirsnRRSaAjVK7APL26BuZGPa6E/NHPGfVZ9PMLbVbRX4mIqDbdVZce+Y6zPnV63THimk5WEjzEXvDtsifHvr5iEWTGwHfWX1c39JQZ7oyhMw4omtGGYQ4JT6tiC2g+H/eC7N7/SZLbUpzTopx9iEidgccG0ohUdJiMd8TblrokPRNSxMEL7WZhZ7rGK4kzBULqp8Tpf53sw0liO/b5jGRpiXluM8GwbHsinIKkBxRBTY924J3+flw0ynvctx5AhxYmNM2bSV6u8qHjtW/itSVjvOzwn0b7Okj/GWeqfxI2PO/RgvuxubyAKKja3sMkUx79G0jBoITOPNO6HQA/TADci+FmmTWCmTlSEQ7t8AS9W179ACoExtC5gYBdsgop1SjyxePKzy9b+pCNBW8w/KJKV6ZjrY7+OeldmrQF8NaiIvUL2eGufof7c0M56wKj9aiTpLbDGgMK1w+3CegMo/3DhGJuL+7IYpYZP2tCEFS7B0pxBH/xUBGXLjzFWt8sNESg8y3lIAwc1xDtE9pE4B/JQsCceathS9AwXaHYdgvWZ 5XF7uRKr 605PPJd5R2aTVHOxxfvJPfy8MaRTPKvLxAIQ0SBnJiQz622OMyFY8LKFc7i8pPnLCjBSbCCYEuOQ1Mj+cZNWhim/5MGDCpitNDyPqg+6JCV7lNsANtE/1OC0mespcUKtYPPkht1bbkTXIlgp9hBf80Yl+Sg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: It is better to use huge_page_size() for hugepage(HugeTLB) instead of PAGE_SIZE for stride, which has been done in flush_pmd/pud_tlb_range(), it could reduce the loop in __flush_tlb_range(). Signed-off-by: Kefeng Wang --- arch/arm64/include/asm/tlbflush.h | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/arch/arm64/include/asm/tlbflush.h b/arch/arm64/include/asm/tlbflush.h index 412a3b9a3c25..25e35e6f8093 100644 --- a/arch/arm64/include/asm/tlbflush.h +++ b/arch/arm64/include/asm/tlbflush.h @@ -360,16 +360,17 @@ static inline void __flush_tlb_range(struct vm_area_struct *vma, dsb(ish); } -static inline void flush_tlb_range(struct vm_area_struct *vma, - unsigned long start, unsigned long end) -{ - /* - * We cannot use leaf-only invalidation here, since we may be invalidating - * table entries as part of collapsing hugepages or moving page tables. - * Set the tlb_level to 0 because we can not get enough information here. - */ - __flush_tlb_range(vma, start, end, PAGE_SIZE, false, 0); -} +/* + * We cannot use leaf-only invalidation here, since we may be invalidating + * table entries as part of collapsing hugepages or moving page tables. + * Set the tlb_level to 0 because we can not get enough information here. + */ +#define flush_tlb_range(vma, start, end) \ + __flush_tlb_range(vma, start, end, \ + ((vma)->vm_flags & VM_HUGETLB) \ + ? huge_page_size(hstate_vma(vma)) \ + : PAGE_SIZE, false, 0) + static inline void flush_tlb_kernel_range(unsigned long start, unsigned long end) {