| Message ID | 1399390209-1756-5-git-send-email-steve.capper@linaro.org (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
Hi Steve, On 05/06/2014 11:30 AM, Steve Capper wrote: > Activate the RCU fast_gup for ARM. We also need to force THP splits to > broadcast an IPI s.t. we block in the fast_gup page walker. As THP > splits are comparatively rare, this should not lead to a noticeable > performance degradation. > diff --git a/arch/arm/mm/flush.c b/arch/arm/mm/flush.c > index 3387e60..91a2b59 100644 > --- a/arch/arm/mm/flush.c > +++ b/arch/arm/mm/flush.c > @@ -377,3 +377,22 @@ void __flush_anon_page(struct vm_area_struct *vma, struct page *page, unsigned l > */ > __cpuc_flush_dcache_area(page_address(page), PAGE_SIZE); > } > + > +#ifdef CONFIG_TRANSPARENT_HUGEPAGE > +#ifdef CONFIG_HAVE_RCU_TABLE_FREE This is trivia, but I for one find the form #if defined(a) && defined(b) easier to read. (Applies to the A64 version as well). Christopher
On 13 May 2014 16:31, Christopher Covington <cov@codeaurora.org> wrote: > Hi Steve, > > On 05/06/2014 11:30 AM, Steve Capper wrote: >> Activate the RCU fast_gup for ARM. We also need to force THP splits to >> broadcast an IPI s.t. we block in the fast_gup page walker. As THP >> splits are comparatively rare, this should not lead to a noticeable >> performance degradation. > >> diff --git a/arch/arm/mm/flush.c b/arch/arm/mm/flush.c >> index 3387e60..91a2b59 100644 >> --- a/arch/arm/mm/flush.c >> +++ b/arch/arm/mm/flush.c >> @@ -377,3 +377,22 @@ void __flush_anon_page(struct vm_area_struct *vma, struct page *page, unsigned l >> */ >> __cpuc_flush_dcache_area(page_address(page), PAGE_SIZE); >> } >> + >> +#ifdef CONFIG_TRANSPARENT_HUGEPAGE >> +#ifdef CONFIG_HAVE_RCU_TABLE_FREE > > This is trivia, but I for one find the form #if defined(a) && defined(b) > easier to read. (Applies to the A64 version as well). > Thank you Christopher, I agree that looks nicer. Cheers,
diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig index 6cfdb3b..d0572f1 100644 --- a/arch/arm/Kconfig +++ b/arch/arm/Kconfig @@ -1803,6 +1803,9 @@ config ARCH_SELECT_MEMORY_MODEL config HAVE_ARCH_PFN_VALID def_bool ARCH_HAS_HOLES_MEMORYMODEL || !SPARSEMEM +config HAVE_RCU_GUP + def_bool y + config HIGHMEM bool "High Memory Support" depends on MMU diff --git a/arch/arm/include/asm/pgtable-3level.h b/arch/arm/include/asm/pgtable-3level.h index b286ba9..fdc4a4f 100644 --- a/arch/arm/include/asm/pgtable-3level.h +++ b/arch/arm/include/asm/pgtable-3level.h @@ -226,6 +226,12 @@ static inline pte_t pte_mkspecial(pte_t pte) #ifdef CONFIG_TRANSPARENT_HUGEPAGE #define pmd_trans_huge(pmd) (pmd_val(pmd) && !(pmd_val(pmd) & PMD_TABLE_BIT)) #define pmd_trans_splitting(pmd) (pmd_val(pmd) & PMD_SECT_SPLITTING) + +#ifdef CONFIG_HAVE_RCU_TABLE_FREE +#define __HAVE_ARCH_PMDP_SPLITTING_FLUSH +void pmdp_splitting_flush(struct vm_area_struct *vma, unsigned long address, + pmd_t *pmdp); +#endif #endif #define PMD_BIT_FUNC(fn,op) \ diff --git a/arch/arm/mm/flush.c b/arch/arm/mm/flush.c index 3387e60..91a2b59 100644 --- a/arch/arm/mm/flush.c +++ b/arch/arm/mm/flush.c @@ -377,3 +377,22 @@ void __flush_anon_page(struct vm_area_struct *vma, struct page *page, unsigned l */ __cpuc_flush_dcache_area(page_address(page), PAGE_SIZE); } + +#ifdef CONFIG_TRANSPARENT_HUGEPAGE +#ifdef CONFIG_HAVE_RCU_TABLE_FREE +static void thp_splitting_flush_sync(void *arg) +{ +} + +void pmdp_splitting_flush(struct vm_area_struct *vma, unsigned long address, + pmd_t *pmdp) +{ + pmd_t pmd = pmd_mksplitting(*pmdp); + VM_BUG_ON(address & ~PMD_MASK); + set_pmd_at(vma->vm_mm, address, pmdp, pmd); + + /* dummy IPI to serialise against fast_gup */ + smp_call_function(thp_splitting_flush_sync, NULL, 1); +} +#endif /* CONFIG_HAVE_RCU_TABLE_FREE */ +#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
Activate the RCU fast_gup for ARM. We also need to force THP splits to broadcast an IPI s.t. we block in the fast_gup page walker. As THP splits are comparatively rare, this should not lead to a noticeable performance degradation. Signed-off-by: Steve Capper <steve.capper@linaro.org> --- arch/arm/Kconfig | 3 +++ arch/arm/include/asm/pgtable-3level.h | 6 ++++++ arch/arm/mm/flush.c | 19 +++++++++++++++++++ 3 files changed, 28 insertions(+)