From patchwork Fri Oct 11 10:14:10 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anshuman Khandual X-Patchwork-Id: 11185171 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 42C0017EE for ; Fri, 11 Oct 2019 10:14:04 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0DAA721D82 for ; Fri, 11 Oct 2019 10:14:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0DAA721D82 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 435038E0006; Fri, 11 Oct 2019 06:14:03 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 3BECB8E0001; Fri, 11 Oct 2019 06:14:03 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 286BB8E0006; Fri, 11 Oct 2019 06:14:03 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0107.hostedemail.com [216.40.44.107]) by kanga.kvack.org (Postfix) with ESMTP id ECBAE8E0001 for ; Fri, 11 Oct 2019 06:14:02 -0400 (EDT) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 9EE83181AC9AE for ; Fri, 11 Oct 2019 10:14:02 +0000 (UTC) X-FDA: 76031093124.26.jar95_7a44438e6ad42 X-Spam-Summary: 50,0,0,19f09cdbd28bf8b1,d41d8cd98f00b204,anshuman.khandual@arm.com,::linux-kernel@vger.kernel.org:linux-arm-kernel@lists.infradead.org:akpm@linux-foundation.org:catalin.marinas@arm.com:will@kernel.org:mark.rutland@arm.com:david@redhat.com:cai@lca.pw:logang@deltatee.com:cpandya@codeaurora.org:arunks@codeaurora.org:dan.j.williams@intel.com:mgorman@techsingularity.net:osalvador@suse.de:ard.biesheuvel@arm.com:steve.capper@arm.com:broonie@kernel.org:valentin.schneider@arm.com:robin.murphy@arm.com:steven.price@arm.com:suzuki.poulose@arm.com:ira.weiny@intel.com,RULES_HIT:2:41:142:355:379:541:960:966:967:968:973:988:989:1260:1261:1345:1437:1535:1605:1730:1747:1777:1792:1963:2196:2198:2199:2200:2393:2525:2568:2634:2682:2685:2731:2859:2890:2895:2897:2906:2911:2924:2925:2926:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4042:4050:4120:4250:4321:4385:4425:4605 :5007:61 X-HE-Tag: jar95_7a44438e6ad42 X-Filterd-Recvd-Size: 9341 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf47.hostedemail.com (Postfix) with ESMTP for ; Fri, 11 Oct 2019 10:14:00 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C6E0A28; Fri, 11 Oct 2019 03:13:58 -0700 (PDT) Received: from p8cg001049571a15.blr.arm.com (p8cg001049571a15.blr.arm.com [10.162.41.129]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 78FB23F703; Fri, 11 Oct 2019 03:13:51 -0700 (PDT) From: Anshuman Khandual To: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, akpm@linux-foundation.org, catalin.marinas@arm.com, will@kernel.org Cc: mark.rutland@arm.com, david@redhat.com, cai@lca.pw, logang@deltatee.com, cpandya@codeaurora.org, arunks@codeaurora.org, dan.j.williams@intel.com, mgorman@techsingularity.net, osalvador@suse.de, ard.biesheuvel@arm.com, steve.capper@arm.com, broonie@kernel.org, valentin.schneider@arm.com, Robin.Murphy@arm.com, steven.price@arm.com, suzuki.poulose@arm.com, ira.weiny@intel.com Subject: [PATCH V10 0/2] arm64/mm: Enable memory hot remove Date: Fri, 11 Oct 2019 15:44:10 +0530 Message-Id: <1570788852-12402-1-git-send-email-anshuman.khandual@arm.com> X-Mailer: git-send-email 2.7.4 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This series enables memory hot remove on arm64 after fixing a possible arm64 platform specific kernel page table race condition. This series is based on linux-next (next-20191011). Concurrent vmalloc() and hot-remove conflict: As pointed out earlier on the v5 thread [2] there can be potential conflict between concurrent vmalloc() and memory hot-remove operation. The problem here is caused by inadequate locking in vmalloc() which protects installation of a page table page but not the walk or the leaf entry modification. Now free_empty_tables() and it's children functions take into account a maximum possible range on which it operates as a floor-ceiling boundary. This makes sure that no page table page is freed unless its fully within the maximum possible range as decided by the caller. Testing: Memory hot remove has been tested on arm64 for 4K, 16K, 64K page config options with all possible CONFIG_ARM64_VA_BITS and CONFIG_PGTABLE_LEVELS combinations. Changes in V10: - Perform just single TLBI invalidation for PMD or PUD block mappings per Catalin - Added comment in free_empty_pte_table() while validating PTE level clears per Catalin - Added comments in free_empty_pxx_table() while checking for non-clear entries per Catalin Changes in V9: (https://lkml.org/lkml/2019/10/9/131) - Dropped ACK tags from Steve and David as this series has changed since - Dropped WARN(!page) in free_hotplug_page_range() per Matthew Wilcox - Replaced pxx_page() with virt_to_page() in free_pxx_table() per Catalin - Dropped page and call virt_to_page() in free_hotplug_pgtable_page() - Replaced sparse_vmap with free_mapped per Catalin - Dropped ternary operators in all unmap_hotplug_pxx_range() per Catalin - Collapsed all free_pxx_table() into free_empty_pxx_table() per Catalin Changes in V8: (https://lkml.org/lkml/2019/9/23/22) - Dropped the first patch (memblock_[free|remove] reorder) from the series which is no longer needed for arm64 hot-remove enablement and was posted separately as (https://patchwork.kernel.org/patch/11146361/) - Dropped vmalloc-vmemmap detection and subsequent skipping of free_empty_tables() - Changed free_empty_[pxx]_tables() functions which now accepts a possible maximum floor-ceiling address range on which it operates. Also changed free_pxx_table() functions to check against required alignment as well as maximum floor-ceiling range as another prerequisite before freeing the page table page. - Dropped remove_pagetable(), instead call it's constituent functions directly Changes in V7: (https://lkml.org/lkml/2019/9/3/326) - vmalloc_vmemmap_overlap gets evaluated early during boot for a given config - free_empty_tables() gets conditionally called based on vmalloc_vmemmap_overlap Changes in V6: (https://lkml.org/lkml/2019/7/15/36) - Implemented most of the suggestions from Mark Rutland - Added in ptdump - remove_pagetable() now has two distinct passes over the kernel page table - First pass unmap_hotplug_range() removes leaf level entries at all level - Second pass free_empty_tables() removes empty page table pages - Kernel page table lock has been dropped completely - vmemmap_free() does not call freee_empty_tables() to avoid conflict with vmalloc() - All address range scanning are converted to do {} while() loop - Added 'unsigned long end' in __remove_pgd_mapping() - Callers need not provide starting pointer argument to free_[pte|pmd|pud]_table() - Drop the starting pointer argument from free_[pte|pmd|pud]_table() functions - Fetching pxxp[i] in free_[pte|pmd|pud]_table() is wrapped around in READ_ONCE() - free_[pte|pmd|pud]_table() now computes starting pointer inside the function - Fixed TLB handling while freeing huge page section mappings at PMD or PUD level - Added WARN_ON(!page) in free_hotplug_page_range() - Added WARN_ON(![pm|pud]_table(pud|pmd)) when there is no section mapping - [PATCH 1/3] mm/hotplug: Reorder memblock_[free|remove]() calls in try_remove_memory() - Request earlier for separate merger (https://patchwork.kernel.org/patch/10986599/) - s/__remove_memory/try_remove_memory in the subject line - s/arch_remove_memory/memblock_[free|remove] in the subject line - A small change in the commit message as re-order happens now for memblock remove functions not for arch_remove_memory() Changes in V5: (https://lkml.org/lkml/2019/5/29/218) - Have some agreement [1] over using memory_hotplug_lock for arm64 ptdump - Change 7ba36eccb3f8 ("arm64/mm: Inhibit huge-vmap with ptdump") already merged - Dropped the above patch from this series - Fixed indentation problem in arch_[add|remove]_memory() as per David - Collected all new Acked-by tags Changes in V4: (https://lkml.org/lkml/2019/5/20/19) - Implemented most of the suggestions from Mark Rutland - Interchanged patch [PATCH 2/4] <---> [PATCH 3/4] and updated commit message - Moved CONFIG_PGTABLE_LEVELS inside free_[pud|pmd]_table() - Used READ_ONCE() in missing instances while accessing page table entries - s/p???_present()/p???_none() for checking valid kernel page table entries - WARN_ON() when an entry is !p???_none() and !p???_present() at the same time - Updated memory hot-remove commit message with additional details as suggested - Rebased the series on 5.2-rc1 with hotplug changes from David and Michal Hocko - Collected all new Acked-by tags Changes in V3: (https://lkml.org/lkml/2019/5/14/197) - Implemented most of the suggestions from Mark Rutland for remove_pagetable() - Fixed applicable PGTABLE_LEVEL wrappers around pgtable page freeing functions - Replaced 'direct' with 'sparse_vmap' in remove_pagetable() with inverted polarity - Changed pointer names ('p' at end) and removed tmp from iterations - Perform intermediate TLB invalidation while clearing pgtable entries - Dropped flush_tlb_kernel_range() in remove_pagetable() - Added flush_tlb_kernel_range() in remove_pte_table() instead - Renamed page freeing functions for pgtable page and mapped pages - Used page range size instead of order while freeing mapped or pgtable pages - Removed all PageReserved() handling while freeing mapped or pgtable pages - Replaced XXX_index() with XXX_offset() while walking the kernel page table - Used READ_ONCE() while fetching individual pgtable entries - Taken overall init_mm.page_table_lock instead of just while changing an entry - Dropped previously added [pmd|pud]_index() which are not required anymore - Added a new patch to protect kernel page table race condition for ptdump - Added a new patch from Mark Rutland to prevent huge-vmap with ptdump Changes in V2: (https://lkml.org/lkml/2019/4/14/5) - Added all received review and ack tags - Split the series from ZONE_DEVICE enablement for better review - Moved memblock re-order patch to the front as per Robin Murphy - Updated commit message on memblock re-order patch per Michal Hocko - Dropped [pmd|pud]_large() definitions - Used existing [pmd|pud]_sect() instead of earlier [pmd|pud]_large() - Removed __meminit and __ref tags as per Oscar Salvador - Dropped unnecessary 'ret' init in arch_add_memory() per Robin Murphy - Skipped calling into pgtable_page_dtor() for linear mapping page table pages and updated all relevant functions Changes in V1: (https://lkml.org/lkml/2019/4/3/28) References: [1] https://lkml.org/lkml/2019/5/28/584 [2] https://lkml.org/lkml/2019/6/11/709 Anshuman Khandual (2): arm64/mm: Hold memory hotplug lock while walking for kernel page table dump arm64/mm: Enable memory hot remove arch/arm64/Kconfig | 3 + arch/arm64/include/asm/memory.h | 1 + arch/arm64/mm/mmu.c | 303 ++++++++++++++++++++++++++++++++++++++-- arch/arm64/mm/ptdump_debugfs.c | 4 + 4 files changed, 302 insertions(+), 9 deletions(-)