From patchwork Wed Aug 2 01:27:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13337472 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7A8A2C41513 for ; Wed, 2 Aug 2023 01:16:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CADD028010D; Tue, 1 Aug 2023 21:16:02 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C5DE12800C8; Tue, 1 Aug 2023 21:16:02 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B4E1528010D; Tue, 1 Aug 2023 21:16:02 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id A52112800C8 for ; Tue, 1 Aug 2023 21:16:02 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 68D8E406A7 for ; Wed, 2 Aug 2023 01:16:02 +0000 (UTC) X-FDA: 81077398164.20.035B322 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by imf29.hostedemail.com (Postfix) with ESMTP id 43FEB12000F for ; Wed, 2 Aug 2023 01:15:58 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=none; spf=pass (imf29.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1690938960; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zRWiOuTxTFzdqZXoMk9vXPPqbk7dboQ/Mz9S/oX+99Q=; b=F5cBx394auC+mTVlHCVsYDzzDGhIpLbhUEh+WTPPk2KhRE2hOkUSw8M15Sue7A/fSb3Dmd y6iNL1GFQ9y1YxhIGxBqqFQ8FOiBLToC6xbZYMpg4Bzb5UGgtOQeAim4+70mZ3y+uuvVJm RUm+hlswZA5W4xJYRLhN00rQ/2llpqM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1690938960; a=rsa-sha256; cv=none; b=H2/dFnLLq6kALFDqq0M6eT2vdAqzQXSID8+xRGMAL4KYliEfgEjwlZFthhZjUnYRBBSdUJ OesKlBsG1gmPdDiB5Jk0Q7YctDZCjrFA3xFxeFWCpuC3eQrFlZgvF1jz4FJ58Ve4XMnedy 3ZeI0hvpmw6mKTvdk/kW8qk0l6DtsfY= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=none; spf=pass (imf29.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.54]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4RFvBq2H6VzVjyY; Wed, 2 Aug 2023 09:14:11 +0800 (CST) Received: from localhost.localdomain.localdomain (10.175.113.25) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Wed, 2 Aug 2023 09:15:54 +0800 From: Kefeng Wang To: Andrew Morton , Catalin Marinas , Will Deacon , Mike Kravetz , Muchun Song , Mina Almasry , , , , , CC: , , <21cnbao@gmail.com>, Kefeng Wang Subject: [PATCH v4] arm64: hugetlb: enable __HAVE_ARCH_FLUSH_HUGETLB_TLB_RANGE Date: Wed, 2 Aug 2023 09:27:31 +0800 Message-ID: <20230802012731.62512-1-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230801135616.165067-1-wangkefeng.wang@huawei.com> References: <20230801135616.165067-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.113.25] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Stat-Signature: bhcxr8rq53j9ztkhx6rdhz5cqhj8wg9q X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 43FEB12000F X-Rspam-User: X-HE-Tag: 1690938958-424093 X-HE-Meta: U2FsdGVkX19mpa93RMbiHcxRtELvjgsg83k0ZKrCbSybFmVqq8Lpu75L2mMA4rwj2zhmr8RALga4w98E/IfxyoiXQBrGvVYSUj3MSqyyFJ8Eg5p0q7NNlKzLo8nTQpChJCGbXVKBcmVI8/seam+/TYaZv4f0FoLrjzBR5619TyfT7tDt5WMydAPoYv12Vqg6ZZgidsuo6goxQhsS6UZwfXHMRozWRJov5qJ1uhvnxbv7tNZ0VEEEeSzcAlo2zRN0YzMEIxKCstQsSNSdkmyFYuckwIhWO+FxaeZZOELVEs8Bpp5I/uO0TUqBcyty9gJzuCZFcEBgTEFweXXEay/w68puUnynodwvv8gNIdn4bA8K+ZJcug/4qJjdgc2CFo2rggIyDG8w52vATVPs6xtoHBuVCXtnKD7JnnZUD2kr3qrFUTEGXvDKHLlZESo8Rsh48W1eg+3kc9HCXq4Qr21sCrRIl5USjbPR0Jmo9pANRFP64686TBPEJ54CD84KkxCi08uR4Pik2Dkwxkz7oubERGdza9ZRxez5KRgCtM/9/SAOuyAMKrrlMeyaYdiCvV7K4rLOs/xhluv2BL6RYSB927Sod8s5j0ixE7R+faeDeM42SgGx2f3IusuUbBp3k/W/Jz2QcuDjAguwYOSoDWMFpNHc7K1B4WFmwujtvCc36r61SxOth7X3OgAaPp9ANGC4S31g2xYLqxu8zFZnAuvzMJW6m2M8Qhaw7whuX6T02hmueNvBaz7ceZzX5KyN1DbwxttbunYiKnwUMfaN33y27JoqFUcyrwnGxqR7m9Bqx1wB5CFBncC/H04oH6utelZISlGaT9cnjTqOkjPS+51yLXHOXiB93T2Id3msGefIb6Kwr9mm6lMaTKZ2pXsF1zNPO4ifZQhDeWg+ULVBw3dJYd1Zv1hEsUXNPz/aY/U/HCDswRRIiYdBo1ng26hUVA5/Uymya2eGzgbEyO+GUhI TXTNfOId 7IlN7I2wnDwJV1yI4TDcQ5RRYt9XZddNlRfrPzafNPw4fFl0bZg4SeYwu8U23MLkxPTtjIXrQZp6+r+QWGiobJ2Q6AGgFGftoVNGlJILVEXXARcMOdGl2VHg9OtWC8hijoZ+16ntG+jCjYR3ZzO7mHr+yN9L0xsEn9hU7TAdsasRT3BtdAiwJvuWqIFZAJuLoJAtX X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: It is better to use huge page size instead of PAGE_SIZE for stride when flush hugepage, which reduces the loop in __flush_tlb_range(). Let's support arch's flush_hugetlb_tlb_range(), which is used in hugetlb_unshare_all_pmds(), move_hugetlb_page_tables() and hugetlb_change_protection() for now. Note, for hugepages based on contiguous bit, it has to be invalidated individually since the contiguous PTE bit is just a hint, the hardware may or may not take it into account. Signed-off-by: Kefeng Wang Reviewed-by: Muchun Song Reviewed-by: Catalin Marinas --- v4: directly pass tlb_level to __flush_tlb_range() with PMD/PUD size, suggested by Catalin v3: add tlb_level hint by using flush_pud/pmd_tlb_range, suggested by Catalin arch/arm64/include/asm/hugetlb.h | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/arch/arm64/include/asm/hugetlb.h b/arch/arm64/include/asm/hugetlb.h index 6a4a1ab8eb23..a91d6219aa78 100644 --- a/arch/arm64/include/asm/hugetlb.h +++ b/arch/arm64/include/asm/hugetlb.h @@ -60,4 +60,19 @@ extern void huge_ptep_modify_prot_commit(struct vm_area_struct *vma, #include +#define __HAVE_ARCH_FLUSH_HUGETLB_TLB_RANGE +static inline void flush_hugetlb_tlb_range(struct vm_area_struct *vma, + unsigned long start, + unsigned long end) +{ + unsigned long stride = huge_page_size(hstate_vma(vma)); + + if (stride == PMD_SIZE) + __flush_tlb_range(vma, start, end, stride, false, 2); + else if (stride == PUD_SIZE) + __flush_tlb_range(vma, start, end, stride, false, 1); + else + __flush_tlb_range(vma, start, end, PAGE_SIZE, false, 0); +} + #endif /* __ASM_HUGETLB_H */