| Message ID | 20230802012731.62512-1-wangkefeng.wang@huawei.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | [v4] arm64: hugetlb: enable __HAVE_ARCH_FLUSH_HUGETLB_TLB_RANGE | expand |
> On Aug 2, 2023, at 09:27, Kefeng Wang <wangkefeng.wang@huawei.com> wrote: > > It is better to use huge page size instead of PAGE_SIZE > for stride when flush hugepage, which reduces the loop > in __flush_tlb_range(). > > Let's support arch's flush_hugetlb_tlb_range(), which is > used in hugetlb_unshare_all_pmds(), move_hugetlb_page_tables() > and hugetlb_change_protection() for now. > > Note, for hugepages based on contiguous bit, it has to be > invalidated individually since the contiguous PTE bit is > just a hint, the hardware may or may not take it into account. > > Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com> Reviewed-by: Muchun Song <songmuchun@bytedance.com>
On Wed, Aug 02, 2023 at 09:27:31AM +0800, Kefeng Wang wrote: > It is better to use huge page size instead of PAGE_SIZE > for stride when flush hugepage, which reduces the loop > in __flush_tlb_range(). > > Let's support arch's flush_hugetlb_tlb_range(), which is > used in hugetlb_unshare_all_pmds(), move_hugetlb_page_tables() > and hugetlb_change_protection() for now. > > Note, for hugepages based on contiguous bit, it has to be > invalidated individually since the contiguous PTE bit is > just a hint, the hardware may or may not take it into account. > > Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com> Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
diff --git a/arch/arm64/include/asm/hugetlb.h b/arch/arm64/include/asm/hugetlb.h index 6a4a1ab8eb23..a91d6219aa78 100644 --- a/arch/arm64/include/asm/hugetlb.h +++ b/arch/arm64/include/asm/hugetlb.h @@ -60,4 +60,19 @@ extern void huge_ptep_modify_prot_commit(struct vm_area_struct *vma, #include <asm-generic/hugetlb.h> +#define __HAVE_ARCH_FLUSH_HUGETLB_TLB_RANGE +static inline void flush_hugetlb_tlb_range(struct vm_area_struct *vma, + unsigned long start, + unsigned long end) +{ + unsigned long stride = huge_page_size(hstate_vma(vma)); + + if (stride == PMD_SIZE) + __flush_tlb_range(vma, start, end, stride, false, 2); + else if (stride == PUD_SIZE) + __flush_tlb_range(vma, start, end, stride, false, 1); + else + __flush_tlb_range(vma, start, end, PAGE_SIZE, false, 0); +} + #endif /* __ASM_HUGETLB_H */
It is better to use huge page size instead of PAGE_SIZE for stride when flush hugepage, which reduces the loop in __flush_tlb_range(). Let's support arch's flush_hugetlb_tlb_range(), which is used in hugetlb_unshare_all_pmds(), move_hugetlb_page_tables() and hugetlb_change_protection() for now. Note, for hugepages based on contiguous bit, it has to be invalidated individually since the contiguous PTE bit is just a hint, the hardware may or may not take it into account. Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com> --- v4: directly pass tlb_level to __flush_tlb_range() with PMD/PUD size, suggested by Catalin v3: add tlb_level hint by using flush_pud/pmd_tlb_range, suggested by Catalin arch/arm64/include/asm/hugetlb.h | 15 +++++++++++++++ 1 file changed, 15 insertions(+)