From patchwork Tue Jan 26 04:54:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 12045225 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E2BD5C433DB for ; Tue, 26 Jan 2021 04:54:17 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 764A222B51 for ; Tue, 26 Jan 2021 04:54:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 764A222B51 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 205008D0089; Mon, 25 Jan 2021 23:54:17 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1B5588D0076; Mon, 25 Jan 2021 23:54:17 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 056778D0089; Mon, 25 Jan 2021 23:54:17 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0248.hostedemail.com [216.40.44.248]) by kanga.kvack.org (Postfix) with ESMTP id E47918D0076 for ; Mon, 25 Jan 2021 23:54:16 -0500 (EST) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id AA5E3181AF5C3 for ; Tue, 26 Jan 2021 04:54:16 +0000 (UTC) X-FDA: 77746709712.27.dock13_5d0c5b02758b Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin27.hostedemail.com (Postfix) with ESMTP id 90F5B3D663 for ; Tue, 26 Jan 2021 04:54:16 +0000 (UTC) X-HE-Tag: dock13_5d0c5b02758b X-Filterd-Recvd-Size: 10617 Received: from mail-pf1-f181.google.com (mail-pf1-f181.google.com [209.85.210.181]) by imf12.hostedemail.com (Postfix) with ESMTP for ; Tue, 26 Jan 2021 04:54:16 +0000 (UTC) Received: by mail-pf1-f181.google.com with SMTP id 11so9809898pfu.4 for ; Mon, 25 Jan 2021 20:54:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=4aFsNeXQ/i7Fj8qIza5kcen5kjFa9Ve5fg30RPhukTw=; b=ce4C/3fTVP7qFzH8XJIen9MuOjrG+JistpkikOqziLr8j1goHHT7iog9ZX6Ywg3Fep qUwOuyPf9UHl0XAc6Wrb0XGXbvqegp2qrW5NZa9gGoSpSAp8sjpJ5v2PYo1jm+ZiPL3Z 9HQ4o/xILrIyBPuWEyFYVJH1upZQ7ql7e2aLwPTbTbCeLHryvt4ttbn7FCwUjoujIsc1 frFJuYVJ8garJRRYPBzuMUwQKEFzwey/krTWoZE/ZNQbAciCwBQj8sp4fgKlz5agzYye zdAAQWKCY/T+keNL3sdWNS85Wi8bLa4kizHTLicdgkOFzf+oOrPYCeyXwWTL6NV1Hji5 dwoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4aFsNeXQ/i7Fj8qIza5kcen5kjFa9Ve5fg30RPhukTw=; b=HMzN04a7pfw+O3UiLD4QLLLK4ApWQh09QbxRCfGaQyzNi/QWbCiruwn9BxUDlL42M9 2ZJczBGhBDTB4XyujKMyNe9O/e7K1fo2s5JBbiOX9BtHnshtUE9RqXiwrM17M7+D8stn F0uFqgsmamoEs1MR0pSwwTlFQKWIrWGcW4ICypQQbwkfqVrnYLPSyo19+A3k/xsh9F++ YxCYEP6re74eUSjRR9oz5DPqhIUH6zxguPQvjCJGhkXFU5BamA9gWXGwKwBZNV24H4AN +k/Wm/r+eb2Su0FO2A7/y3F8yabu6Dj0Y+0U2OY0IqEiwiT+ejuqvNaZJNI3bvlYfLgY PQog== X-Gm-Message-State: AOAM533eWaBeGCvCGCds5mKLYBPcqX0sSPco3vREQnPOt+5pgz2LdBHD vSrqSH2OkEW1So8Pp1zj9U4O6xDr+VA= X-Google-Smtp-Source: ABdhPJwOY9GEbO+VwkQYL9vvVzoFRPj/2nPjkKBGDhJyj1Mw9kSKI3/aZ5hBVh8cQNQQB9Qxh1zpYA== X-Received: by 2002:a63:e442:: with SMTP id i2mr3859050pgk.12.1611636855014; Mon, 25 Jan 2021 20:54:15 -0800 (PST) Received: from bobo.ozlabs.ibm.com (192.156.221.203.dial.dynamic.acc50-nort-cbr.comindico.com.au. [203.221.156.192]) by smtp.gmail.com with ESMTPSA id w184sm18803382pgb.71.2021.01.25.20.54.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Jan 2021 20:54:14 -0800 (PST) From: Nicholas Piggin To: linux-mm@kvack.org, Andrew Morton Cc: Nicholas Piggin , linux-kernel@vger.kernel.org, Christoph Hellwig Subject: [PATCH 1/5] mm/vmalloc: remove map_kernel_range Date: Tue, 26 Jan 2021 14:54:00 +1000 Message-Id: <20210126045404.2492588-2-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210126045404.2492588-1-npiggin@gmail.com> References: <20210126045404.2492588-1-npiggin@gmail.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This is a shim around vmap_pages_range, get rid of it. Move the main API comment from the _noflush variant to the normal variant, and make _noflush internal to mm/. Signed-off-by: Nicholas Piggin Reviewed-by: Christoph Hellwig --- Documentation/core-api/cachetlb.rst | 2 +- include/linux/vmalloc.h | 11 ----- mm/internal.h | 6 +++ mm/percpu-vm.c | 5 ++- mm/vmalloc.c | 65 +++++++++++++---------------- 5 files changed, 38 insertions(+), 51 deletions(-) diff --git a/Documentation/core-api/cachetlb.rst b/Documentation/core-api/cachetlb.rst index a1582cc79f0f..756f7bcf8191 100644 --- a/Documentation/core-api/cachetlb.rst +++ b/Documentation/core-api/cachetlb.rst @@ -213,7 +213,7 @@ Here are the routines, one by one: there will be no entries in the cache for the kernel address space for virtual addresses in the range 'start' to 'end-1'. - The first of these two routines is invoked after map_kernel_range() + The first of these two routines is invoked after vmap_range() has installed the page table entries. The second is invoked before unmap_kernel_range() deletes the page table entries. diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h index 93270adf5db5..913c9d4f5e03 100644 --- a/include/linux/vmalloc.h +++ b/include/linux/vmalloc.h @@ -217,10 +217,6 @@ static inline bool is_vm_area_hugepages(const void *addr) int vmap_range(unsigned long addr, unsigned long end, phys_addr_t phys_addr, pgprot_t prot, unsigned int max_page_shift); -extern int map_kernel_range_noflush(unsigned long start, unsigned long size, - pgprot_t prot, struct page **pages); -int map_kernel_range(unsigned long start, unsigned long size, pgprot_t prot, - struct page **pages); extern void unmap_kernel_range_noflush(unsigned long addr, unsigned long size); extern void unmap_kernel_range(unsigned long addr, unsigned long size); static inline void set_vm_flush_reset_perms(void *addr) @@ -232,13 +228,6 @@ static inline void set_vm_flush_reset_perms(void *addr) } #else -static inline int -map_kernel_range_noflush(unsigned long start, unsigned long size, - pgprot_t prot, struct page **pages) -{ - return size >> PAGE_SHIFT; -} -#define map_kernel_range map_kernel_range_noflush static inline void unmap_kernel_range_noflush(unsigned long addr, unsigned long size) { diff --git a/mm/internal.h b/mm/internal.h index 25d2b2439f19..3ce246945882 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -623,4 +623,10 @@ struct migration_target_control { gfp_t gfp_mask; }; +/* + * mm/vmalloc.c + */ +int vmap_pages_range_noflush(unsigned long addr, unsigned long end, + pgprot_t prot, struct page **pages, unsigned int page_shift); + #endif /* __MM_INTERNAL_H */ diff --git a/mm/percpu-vm.c b/mm/percpu-vm.c index e46f7a6917f9..88a53eb68a94 100644 --- a/mm/percpu-vm.c +++ b/mm/percpu-vm.c @@ -8,6 +8,7 @@ * Chunks are mapped into vmalloc areas and populated page by page. * This is the default chunk allocator. */ +#include "internal.h" static struct page *pcpu_chunk_page(struct pcpu_chunk *chunk, unsigned int cpu, int page_idx) @@ -192,8 +193,8 @@ static void pcpu_post_unmap_tlb_flush(struct pcpu_chunk *chunk, static int __pcpu_map_pages(unsigned long addr, struct page **pages, int nr_pages) { - return map_kernel_range_noflush(addr, nr_pages << PAGE_SHIFT, - PAGE_KERNEL, pages); + return vmap_pages_range_noflush(addr, addr + (nr_pages << PAGE_SHIFT), + PAGE_KERNEL, pages, PAGE_SHIFT); } /** diff --git a/mm/vmalloc.c b/mm/vmalloc.c index e9a28de04182..78dc3ec457ef 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -523,7 +523,16 @@ static int vmap_small_pages_range_noflush(unsigned long addr, unsigned long end, return 0; } -static int vmap_pages_range_noflush(unsigned long addr, unsigned long end, +/* + * vmap_pages_range_noflush is similar to vmap_pages_range, but does not + * flush caches. + * + * The caller is responsible for calling flush_cache_vmap() after this + * function returns successfully and before the addresses are accessed. + * + * This is an internal function only. Do not use outside mm/. + */ +int vmap_pages_range_noflush(unsigned long addr, unsigned long end, pgprot_t prot, struct page **pages, unsigned int page_shift) { unsigned int i, nr = (end - addr) >> PAGE_SHIFT; @@ -549,48 +558,26 @@ static int vmap_pages_range_noflush(unsigned long addr, unsigned long end, return 0; } -static int vmap_pages_range(unsigned long addr, unsigned long end, - pgprot_t prot, struct page **pages, unsigned int page_shift) -{ - int err; - - err = vmap_pages_range_noflush(addr, end, prot, pages, page_shift); - flush_cache_vmap(addr, end); - return err; -} - /** - * map_kernel_range_noflush - map kernel VM area with the specified pages + * vmap_pages_range - map pages to a kernel virtual address * @addr: start of the VM area to map - * @size: size of the VM area to map + * @end: end of the VM area to map (non-inclusive) * @prot: page protection flags to use - * @pages: pages to map - * - * Map PFN_UP(@size) pages at @addr. The VM area @addr and @size specify should - * have been allocated using get_vm_area() and its friends. - * - * NOTE: - * This function does NOT do any cache flushing. The caller is responsible for - * calling flush_cache_vmap() on to-be-mapped areas before calling this - * function. + * @pages: pages to map (always PAGE_SIZE pages) + * @page_shift: maximum shift that the pages may be mapped with, @pages must + * be aligned and contiguous up to at least this shift. * * RETURNS: * 0 on success, -errno on failure. */ -int map_kernel_range_noflush(unsigned long addr, unsigned long size, - pgprot_t prot, struct page **pages) -{ - return vmap_pages_range_noflush(addr, addr + size, prot, pages, PAGE_SHIFT); -} - -int map_kernel_range(unsigned long start, unsigned long size, pgprot_t prot, - struct page **pages) +static int vmap_pages_range(unsigned long addr, unsigned long end, + pgprot_t prot, struct page **pages, unsigned int page_shift) { - int ret; + int err; - ret = map_kernel_range_noflush(start, size, prot, pages); - flush_cache_vmap(start, start + size); - return ret; + err = vmap_pages_range_noflush(addr, end, prot, pages, page_shift); + flush_cache_vmap(addr, end); + return err; } int is_vmalloc_or_module_addr(const void *x) @@ -2156,10 +2143,12 @@ void *vm_map_ram(struct page **pages, unsigned int count, int node) kasan_unpoison_vmalloc(mem, size); - if (map_kernel_range(addr, size, PAGE_KERNEL, pages) < 0) { + if (vmap_pages_range(addr, addr + size, PAGE_KERNEL, + pages, PAGE_SHIFT) < 0) { vm_unmap_ram(mem, count); return NULL; } + return mem; } EXPORT_SYMBOL(vm_map_ram); @@ -2703,6 +2692,7 @@ void *vmap(struct page **pages, unsigned int count, unsigned long flags, pgprot_t prot) { struct vm_struct *area; + unsigned long addr; unsigned long size; /* In bytes */ might_sleep(); @@ -2715,8 +2705,9 @@ void *vmap(struct page **pages, unsigned int count, if (!area) return NULL; - if (map_kernel_range((unsigned long)area->addr, size, pgprot_nx(prot), - pages) < 0) { + addr = (unsigned long)area->addr; + if (vmap_pages_range(addr, addr + size, pgprot_nx(prot), + pages, PAGE_SHIFT) < 0) { vunmap(area->addr); return NULL; } From patchwork Tue Jan 26 04:54:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 12045227 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CCA56C433E0 for ; Tue, 26 Jan 2021 04:54:21 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6635222D04 for ; Tue, 26 Jan 2021 04:54:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6635222D04 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E39A78D008A; Mon, 25 Jan 2021 23:54:20 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id DE9588D0076; Mon, 25 Jan 2021 23:54:20 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CFF528D008A; Mon, 25 Jan 2021 23:54:20 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0038.hostedemail.com [216.40.44.38]) by kanga.kvack.org (Postfix) with ESMTP id B79618D0076 for ; Mon, 25 Jan 2021 23:54:20 -0500 (EST) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 83C6D1EE6 for ; Tue, 26 Jan 2021 04:54:20 +0000 (UTC) X-FDA: 77746709880.12.act37_1c14ed52758b Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin12.hostedemail.com (Postfix) with ESMTP id 6737718022F23 for ; Tue, 26 Jan 2021 04:54:20 +0000 (UTC) X-HE-Tag: act37_1c14ed52758b X-Filterd-Recvd-Size: 3843 Received: from mail-pj1-f54.google.com (mail-pj1-f54.google.com [209.85.216.54]) by imf25.hostedemail.com (Postfix) with ESMTP for ; Tue, 26 Jan 2021 04:54:19 +0000 (UTC) Received: by mail-pj1-f54.google.com with SMTP id jx18so1502438pjb.5 for ; Mon, 25 Jan 2021 20:54:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=a69BMfJQ9E4oHULh93aNwSAV8hiyHTyNrIA1sCczbko=; b=mNfngmN/q7ViCvgKIo8bo3NBWuQB90CjvHeINbBALSaZ9eQfiKiHDbOkzIlmSFVJsr S1Swe0LlSEQdH19e70KOsf8TqXZl6qUlsmhl8B8BadbI4RpVDG+3PMVsTsXZljl9fTez nDIu2af4HTSc7/qULyahGyHx7G1jqNlinR55/bJusztTdX4F96cnWe64sDxGUjr8mP2x 0G0mp360Y5bYDAsNb62nazgCY6L9MoSuAUCEJk/uXxsqC3EJv70Yq6f0FbtwQ6O/rE06 aJmTWlzwFplZhScxww31Fji6SUnUGDFL3PUz1KF5NEBsKS+eF7yMd1x0TjLd/fyG1J1s eQTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=a69BMfJQ9E4oHULh93aNwSAV8hiyHTyNrIA1sCczbko=; b=coT3JDSDUDANGKdj7zG9/+zaiM4OR4hAzKN/p0KlrDp8tGw2ACnWnM9RBpsZn72+2v feC57/VVqQc9SjcUtNt1Rv3E1UpjcaEevY2RFkNPMRojYT0VmmFbjsOaH97vtjkc2uYl jaXCXjLZANnBRl9v6I8MA8iRnKa5PbVoPIw9XNUD1WT1qutw6d9vHpDpq22ak5S6yPQE hXKrQYahNOgXUKlgqTF55D7QbWMBqwPIVKovev8hNnbR9jtnvDrTkPex7UCv8LGHL3ze G+NehuWjOCqJCXdtiMdtJ5EwSjkYkAGjS8OdqM6/oz8k03WGdLWa966VhGo5l8cr7KkQ nQUg== X-Gm-Message-State: AOAM532V1t47VmnrqUSxHJpBpzR5EpgHBJxEevFneJW1aDEjS6uhRK59 kGyNQ7mZ/WLsA22WzjPZmJevdInsq3w= X-Google-Smtp-Source: ABdhPJwSa2nXR/aKoIRxXCZWsKkoMqrHw7m/qOUyKn2wDN2zSos1dL5DiF7wqcAQ5jV1WhPBUCJ4ow== X-Received: by 2002:a17:902:7c10:b029:e0:7a2:38c7 with SMTP id x16-20020a1709027c10b02900e007a238c7mr4251545pll.79.1611636858988; Mon, 25 Jan 2021 20:54:18 -0800 (PST) Received: from bobo.ozlabs.ibm.com (192.156.221.203.dial.dynamic.acc50-nort-cbr.comindico.com.au. [203.221.156.192]) by smtp.gmail.com with ESMTPSA id w184sm18803382pgb.71.2021.01.25.20.54.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Jan 2021 20:54:18 -0800 (PST) From: Nicholas Piggin To: linux-mm@kvack.org, Andrew Morton Cc: Nicholas Piggin , linux-kernel@vger.kernel.org, Christoph Hellwig , Marek Szyprowski , Robin Murphy , iommu@lists.linux-foundation.org Subject: [PATCH 2/5] kernel/dma: remove unnecessary unmap_kernel_range Date: Tue, 26 Jan 2021 14:54:01 +1000 Message-Id: <20210126045404.2492588-3-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210126045404.2492588-1-npiggin@gmail.com> References: <20210126045404.2492588-1-npiggin@gmail.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: vunmap will remove ptes. Cc: Christoph Hellwig Cc: Marek Szyprowski Cc: Robin Murphy Cc: iommu@lists.linux-foundation.org Signed-off-by: Nicholas Piggin Reviewed-by: Christoph Hellwig --- kernel/dma/remap.c | 1 - 1 file changed, 1 deletion(-) diff --git a/kernel/dma/remap.c b/kernel/dma/remap.c index 905c3fa005f1..b4526668072e 100644 --- a/kernel/dma/remap.c +++ b/kernel/dma/remap.c @@ -66,6 +66,5 @@ void dma_common_free_remap(void *cpu_addr, size_t size) return; } - unmap_kernel_range((unsigned long)cpu_addr, PAGE_ALIGN(size)); vunmap(cpu_addr); } From patchwork Tue Jan 26 04:54:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 12045229 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9C907C433DB for ; Tue, 26 Jan 2021 04:54:25 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2B2D722D04 for ; Tue, 26 Jan 2021 04:54:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2B2D722D04 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B57148D008B; Mon, 25 Jan 2021 23:54:24 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id AE1AC8D0076; Mon, 25 Jan 2021 23:54:24 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9F1E88D008B; Mon, 25 Jan 2021 23:54:24 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0089.hostedemail.com [216.40.44.89]) by kanga.kvack.org (Postfix) with ESMTP id 8820A8D0076 for ; Mon, 25 Jan 2021 23:54:24 -0500 (EST) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 4BA918249980 for ; Tue, 26 Jan 2021 04:54:24 +0000 (UTC) X-FDA: 77746710048.09.slave56_120f2162758b Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin09.hostedemail.com (Postfix) with ESMTP id 1A872180ACF9A for ; Tue, 26 Jan 2021 04:54:24 +0000 (UTC) X-HE-Tag: slave56_120f2162758b X-Filterd-Recvd-Size: 4167 Received: from mail-pj1-f50.google.com (mail-pj1-f50.google.com [209.85.216.50]) by imf05.hostedemail.com (Postfix) with ESMTP for ; Tue, 26 Jan 2021 04:54:23 +0000 (UTC) Received: by mail-pj1-f50.google.com with SMTP id cq1so1322665pjb.4 for ; Mon, 25 Jan 2021 20:54:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kxNEOYhgKwvRvVfu2/k3ZgwDNvcTYU3oc8R54p0XWwA=; b=to0uKAF7Nj1QlCsWyg4IjE5lSdoDnjb+l9Ld6oFvxCEr+TdnupQouEo+9kBXa/CDTa CcWbs9nZMugSmys1AYrg0YzRl7gGvYIGqFcWo3m+tnDDoNP1eEytLw8r3y7GSwMyrmCe D1MkaErva8TKJARzEmQ3TRygPOnZWkae0oQNXH8nuBTl1tLbbILHBtIsNbb52vxMEmBG HQHRvco4f3ttmI5jpwD72uQoDxUVtsLsRLoXAOuHMtgh+IGcrxP5DwZ+4nZ3hWnf9Yc7 BPYalDoaBtLHOmX56hB5YjYixLyMljyMwoz86Q8ScLyG6lL2uoJfpDdBof0J7WJ4hrPV oSuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kxNEOYhgKwvRvVfu2/k3ZgwDNvcTYU3oc8R54p0XWwA=; b=jOJG2lHpvHq6xg1tCmuVP7Qw0Ft8Y6dOX2+/LS7n60wxk6pc/idRVRfo8NJ41/hFgT b3MFT1Jdw4isMgg+iGREF78Po50ntyfJvlHP610XL0E3WjMIy2lnbSj9oU3CbTRggkpj olr3UkN301GO9iuoYBw5ETbGxD4WnJCtO1A3hY1daE+R2sfdHub605NJWljQVjy9qRb+ FRg5Qlxub1I/nYAhaEd3R9WavJGrtwPA2jvEfHJGjivhrHl7c4aPXdclIF98/XRhorrB +u2/c37U3tIvf5ex08/DsG9StZBe6ZyYmIiAeYESPQ5xtUu7U4nrWhH3N7476bBcbmY/ N8Pw== X-Gm-Message-State: AOAM531oV4yFT8SSHtjBP3hLIdIN3Ly4hJ4FXnMWArC7mei1vfEgkmg+ aT3o+SO662A07UW2lGBC0u9+Pxy7Q8Q= X-Google-Smtp-Source: ABdhPJwtUyXXER9bmRZ+e2DThvE7nOQ7xV1I38TK3IzCWyGCRxyYZp+rA6/30YI7Bw+AQQF+pC05Lw== X-Received: by 2002:a17:902:e844:b029:de:5abb:7df1 with SMTP id t4-20020a170902e844b02900de5abb7df1mr4028710plg.55.1611636862738; Mon, 25 Jan 2021 20:54:22 -0800 (PST) Received: from bobo.ozlabs.ibm.com (192.156.221.203.dial.dynamic.acc50-nort-cbr.comindico.com.au. [203.221.156.192]) by smtp.gmail.com with ESMTPSA id w184sm18803382pgb.71.2021.01.25.20.54.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Jan 2021 20:54:22 -0800 (PST) From: Nicholas Piggin To: linux-mm@kvack.org, Andrew Morton Cc: Nicholas Piggin , linux-kernel@vger.kernel.org, Christoph Hellwig , =?utf-8?q?C=C3=A9dric_Le_Goater?= , linuxppc-dev@lists.ozlabs.org Subject: [PATCH 3/5] powerpc/xive: remove unnecessary unmap_kernel_range Date: Tue, 26 Jan 2021 14:54:02 +1000 Message-Id: <20210126045404.2492588-4-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210126045404.2492588-1-npiggin@gmail.com> References: <20210126045404.2492588-1-npiggin@gmail.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: iounmap will remove ptes. Cc: "Cédric Le Goater" Cc: linuxppc-dev@lists.ozlabs.org Signed-off-by: Nicholas Piggin Reviewed-by: Christoph Hellwig Acked-by: Cédric Le Goater --- arch/powerpc/sysdev/xive/common.c | 4 ---- 1 file changed, 4 deletions(-) diff --git a/arch/powerpc/sysdev/xive/common.c b/arch/powerpc/sysdev/xive/common.c index 595310e056f4..d6c2069cc828 100644 --- a/arch/powerpc/sysdev/xive/common.c +++ b/arch/powerpc/sysdev/xive/common.c @@ -959,16 +959,12 @@ EXPORT_SYMBOL_GPL(is_xive_irq); void xive_cleanup_irq_data(struct xive_irq_data *xd) { if (xd->eoi_mmio) { - unmap_kernel_range((unsigned long)xd->eoi_mmio, - 1u << xd->esb_shift); iounmap(xd->eoi_mmio); if (xd->eoi_mmio == xd->trig_mmio) xd->trig_mmio = NULL; xd->eoi_mmio = NULL; } if (xd->trig_mmio) { - unmap_kernel_range((unsigned long)xd->trig_mmio, - 1u << xd->esb_shift); iounmap(xd->trig_mmio); xd->trig_mmio = NULL; } From patchwork Tue Jan 26 04:54:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 12045231 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2E4F7C433DB for ; Tue, 26 Jan 2021 04:54:29 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id ABA8D22D04 for ; Tue, 26 Jan 2021 04:54:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org ABA8D22D04 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 427298D008C; Mon, 25 Jan 2021 23:54:28 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3D6908D0076; Mon, 25 Jan 2021 23:54:28 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2C5F78D008C; Mon, 25 Jan 2021 23:54:28 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0196.hostedemail.com [216.40.44.196]) by kanga.kvack.org (Postfix) with ESMTP id 18CC08D0076 for ; Mon, 25 Jan 2021 23:54:28 -0500 (EST) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id C6A088249980 for ; Tue, 26 Jan 2021 04:54:27 +0000 (UTC) X-FDA: 77746710174.12.cover97_4c0c0252758b Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin12.hostedemail.com (Postfix) with ESMTP id A1FEA18022F23 for ; Tue, 26 Jan 2021 04:54:27 +0000 (UTC) X-HE-Tag: cover97_4c0c0252758b X-Filterd-Recvd-Size: 12960 Received: from mail-pj1-f41.google.com (mail-pj1-f41.google.com [209.85.216.41]) by imf08.hostedemail.com (Postfix) with ESMTP for ; Tue, 26 Jan 2021 04:54:27 +0000 (UTC) Received: by mail-pj1-f41.google.com with SMTP id s24so7186pjp.5 for ; Mon, 25 Jan 2021 20:54:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=wGEryhrm5c1wqfxot2kAYpl/XWvHAkbpC5WQ3NIWIAU=; b=qfwpbyqlt6zOAaUEZxTqpvbXz5OoP0/xUb0/QnP2xTRAo/J5CCB31Sei5jFQPW/gnO FCqSgCivxck59xSdPwNpioaxmWln/fz5+wnCaEZYOvbu5+mb3IGG1alccVcPCG/PT12P +Xyaf2XClLgEcFa6T/wEVmauzmybyPi8xiXtBXN72jW+Y4SGCYnyZ3s2awnhFgH04ufp qADTTnUlYxcVOoRL6nZ3hgIZpHAfCJIk85NQkT73UiTc2nrHbgIh0jIl33zn0V2xihOr +gysALVmTiTksdyT/AbJm3XERhqzZnVLfE5rtpYcNAw9c43nhqqRD+njK7yKcrsmD5nE INJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wGEryhrm5c1wqfxot2kAYpl/XWvHAkbpC5WQ3NIWIAU=; b=KCBcFg6MRIoOkiErhaCJi1GcRGRqBNM60AwdCA+nqmwwpMZGp/hHDFMe96LcxGFs65 RXi9klYLKEiOGuFrXzNN/YhyUSvDbQoiuBDDHHfcF8PXxYgAAQ9oGThoSShxBzEVZGiV emJ9ozUUtrGTnxsDMC7HDi+egVsZf81rDId7dkS9IeuD9GQY+MLYqixePW65UuzmdRwS yRZAScZ1d2G92klM5w7MzqLe/lC0MdH/BSaGjnA2RcjpsduUr51zMfFYRT+hj40yD/6o n06XYNYBV5VMtAWE5YEBejHGAzQrEF3v1Ne6+GyUeUek2REN4j+4A8theAnNAmR407KC AWoA== X-Gm-Message-State: AOAM5339y+RcxKFQwx0j9hOfr1dIh/P7rrQClWFpqTRRRJ7LSiPiuCk6 73D7HwumfEosp5RKl4xtN8B8bcRKOw4= X-Google-Smtp-Source: ABdhPJzmZbawnaJ6T51tQobT8KneXcFflspxxOZksfckoxk0gdsxd/RQkXROfH4VVEHDTAzSOOS6jQ== X-Received: by 2002:a17:90a:7345:: with SMTP id j5mr4087606pjs.176.1611636866026; Mon, 25 Jan 2021 20:54:26 -0800 (PST) Received: from bobo.ozlabs.ibm.com (192.156.221.203.dial.dynamic.acc50-nort-cbr.comindico.com.au. [203.221.156.192]) by smtp.gmail.com with ESMTPSA id w184sm18803382pgb.71.2021.01.25.20.54.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Jan 2021 20:54:25 -0800 (PST) From: Nicholas Piggin To: linux-mm@kvack.org, Andrew Morton Cc: Nicholas Piggin , linux-kernel@vger.kernel.org, Christoph Hellwig Subject: [PATCH 4/5] mm/vmalloc: remove unmap_kernel_range Date: Tue, 26 Jan 2021 14:54:03 +1000 Message-Id: <20210126045404.2492588-5-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210126045404.2492588-1-npiggin@gmail.com> References: <20210126045404.2492588-1-npiggin@gmail.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This is a shim around vunmap_range, get rid of it. Move the main API comment from the _noflush variant to the normal variant, and make _noflush internal to mm/. Signed-off-by: Nicholas Piggin Reviewed-by: Christoph Hellwig --- Documentation/core-api/cachetlb.rst | 2 +- arch/arm64/mm/init.c | 2 +- arch/powerpc/kernel/isa-bridge.c | 4 +- arch/powerpc/kernel/pci_64.c | 2 +- arch/powerpc/mm/ioremap.c | 2 +- drivers/pci/pci.c | 2 +- include/linux/vmalloc.h | 8 +--- mm/internal.h | 1 + mm/percpu-vm.c | 2 +- mm/vmalloc.c | 59 ++++++++++++++--------------- 10 files changed, 38 insertions(+), 46 deletions(-) diff --git a/Documentation/core-api/cachetlb.rst b/Documentation/core-api/cachetlb.rst index 756f7bcf8191..fe4290e26729 100644 --- a/Documentation/core-api/cachetlb.rst +++ b/Documentation/core-api/cachetlb.rst @@ -215,7 +215,7 @@ Here are the routines, one by one: The first of these two routines is invoked after vmap_range() has installed the page table entries. The second is invoked - before unmap_kernel_range() deletes the page table entries. + before vunmap_range() deletes the page table entries. There exists another whole class of cpu cache issues which currently require a whole different set of interfaces to handle properly. diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c index 709d98fea90c..7fe0a5074205 100644 --- a/arch/arm64/mm/init.c +++ b/arch/arm64/mm/init.c @@ -498,7 +498,7 @@ void free_initmem(void) * prevents the region from being reused for kernel modules, which * is not supported by kallsyms. */ - unmap_kernel_range((u64)__init_begin, (u64)(__init_end - __init_begin)); + vunmap_range((u64)__init_begin, (u64)__init_end); } void dump_mem_limit(void) diff --git a/arch/powerpc/kernel/isa-bridge.c b/arch/powerpc/kernel/isa-bridge.c index 2257d24e6a26..39c625737c09 100644 --- a/arch/powerpc/kernel/isa-bridge.c +++ b/arch/powerpc/kernel/isa-bridge.c @@ -48,7 +48,7 @@ static void remap_isa_base(phys_addr_t pa, unsigned long size) if (slab_is_available()) { if (ioremap_page_range(ISA_IO_BASE, ISA_IO_BASE + size, pa, pgprot_noncached(PAGE_KERNEL))) - unmap_kernel_range(ISA_IO_BASE, size); + vunmap_range(ISA_IO_BASE, ISA_IO_BASE + size); } else { early_ioremap_range(ISA_IO_BASE, pa, size, pgprot_noncached(PAGE_KERNEL)); @@ -311,7 +311,7 @@ static void isa_bridge_remove(void) isa_bridge_pcidev = NULL; /* Unmap the ISA area */ - unmap_kernel_range(ISA_IO_BASE, 0x10000); + vunmap_range(ISA_IO_BASE, ISA_IO_BASE + 0x10000); } /** diff --git a/arch/powerpc/kernel/pci_64.c b/arch/powerpc/kernel/pci_64.c index 9312e6eda7ff..3fb7e572abed 100644 --- a/arch/powerpc/kernel/pci_64.c +++ b/arch/powerpc/kernel/pci_64.c @@ -140,7 +140,7 @@ void __iomem *ioremap_phb(phys_addr_t paddr, unsigned long size) addr = (unsigned long)area->addr; if (ioremap_page_range(addr, addr + size, paddr, pgprot_noncached(PAGE_KERNEL))) { - unmap_kernel_range(addr, size); + vunmap_range(addr, addr + size); return NULL; } diff --git a/arch/powerpc/mm/ioremap.c b/arch/powerpc/mm/ioremap.c index b1a0aebe8c48..57342154d2b0 100644 --- a/arch/powerpc/mm/ioremap.c +++ b/arch/powerpc/mm/ioremap.c @@ -93,7 +93,7 @@ void __iomem *do_ioremap(phys_addr_t pa, phys_addr_t offset, unsigned long size, if (!ret) return (void __iomem *)area->addr + offset; - unmap_kernel_range(va, size); + vunmap_range(va, va + size); free_vm_area(area); return NULL; diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index b9fecc25d213..d1e5ee09b381 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -4107,7 +4107,7 @@ void pci_unmap_iospace(struct resource *res) #if defined(PCI_IOBASE) && defined(CONFIG_MMU) unsigned long vaddr = (unsigned long)PCI_IOBASE + res->start; - unmap_kernel_range(vaddr, resource_size(res)); + vunmap_range(vaddr, vaddr + resource_size(res)); #endif } EXPORT_SYMBOL(pci_unmap_iospace); diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h index 913c9d4f5e03..b569a13c9960 100644 --- a/include/linux/vmalloc.h +++ b/include/linux/vmalloc.h @@ -217,8 +217,7 @@ static inline bool is_vm_area_hugepages(const void *addr) int vmap_range(unsigned long addr, unsigned long end, phys_addr_t phys_addr, pgprot_t prot, unsigned int max_page_shift); -extern void unmap_kernel_range_noflush(unsigned long addr, unsigned long size); -extern void unmap_kernel_range(unsigned long addr, unsigned long size); +void vunmap_range(unsigned long addr, unsigned long end); static inline void set_vm_flush_reset_perms(void *addr) { struct vm_struct *vm = find_vm_area(addr); @@ -228,11 +227,6 @@ static inline void set_vm_flush_reset_perms(void *addr) } #else -static inline void -unmap_kernel_range_noflush(unsigned long addr, unsigned long size) -{ -} -#define unmap_kernel_range unmap_kernel_range_noflush static inline void set_vm_flush_reset_perms(void *addr) { } diff --git a/mm/internal.h b/mm/internal.h index 3ce246945882..3d4449f79582 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -628,5 +628,6 @@ struct migration_target_control { */ int vmap_pages_range_noflush(unsigned long addr, unsigned long end, pgprot_t prot, struct page **pages, unsigned int page_shift); +void vunmap_range_noflush(unsigned long start, unsigned long end); #endif /* __MM_INTERNAL_H */ diff --git a/mm/percpu-vm.c b/mm/percpu-vm.c index 88a53eb68a94..8d3844bc0c7c 100644 --- a/mm/percpu-vm.c +++ b/mm/percpu-vm.c @@ -134,7 +134,7 @@ static void pcpu_pre_unmap_flush(struct pcpu_chunk *chunk, static void __pcpu_unmap_pages(unsigned long addr, int nr_pages) { - unmap_kernel_range_noflush(addr, nr_pages << PAGE_SHIFT); + vunmap_range_noflush(addr, addr + (nr_pages << PAGE_SHIFT)); } /** diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 78dc3ec457ef..5ff190590fe4 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -378,22 +378,20 @@ static void vunmap_p4d_range(pgd_t *pgd, unsigned long addr, unsigned long end, } while (p4d++, addr = next, addr != end); } -/** - * unmap_kernel_range_noflush - unmap kernel VM area - * @start: start of the VM area to unmap - * @size: size of the VM area to unmap +/* + * vunmap_range_noflush is similar to vunmap_range, but does not + * flush caches or TLBs. * - * Unmap PFN_UP(@size) pages at @addr. The VM area @addr and @size specify - * should have been allocated using get_vm_area() and its friends. + * The caller is responsible for calling flush_cache_vmap() before calling + * this function, and flush_tlb_kernel_range after it has returned + * successfully (and before the addresses are expected to cause a page fault + * or be re-mapped for something else, if TLB flushes are being delayed or + * coalesced). * - * NOTE: - * This function does NOT do any cache flushing. The caller is responsible - * for calling flush_cache_vunmap() on to-be-mapped areas before calling this - * function and flush_tlb_kernel_range() after. + * This is an internal function only. Do not use outside mm/. */ -void unmap_kernel_range_noflush(unsigned long start, unsigned long size) +void vunmap_range_noflush(unsigned long start, unsigned long end) { - unsigned long end = start + size; unsigned long next; pgd_t *pgd; unsigned long addr = start; @@ -414,6 +412,22 @@ void unmap_kernel_range_noflush(unsigned long start, unsigned long size) arch_sync_kernel_mappings(start, end); } +/** + * vunmap_range_noflush - unmap kernel virtual addresses + * @addr: start of the VM area to unmap + * @end: end of the VM area to unmap (non-inclusive) + * + * Clears any present PTEs in the virtual address range, flushes TLBs and + * caches. Any subsequent access to the address before it has been re-mapped + * is a kernel bug. + */ +void vunmap_range(unsigned long addr, unsigned long end) +{ + flush_cache_vunmap(addr, end); + vunmap_range_noflush(addr, end); + flush_tlb_kernel_range(addr, end); +} + static int vmap_pages_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end, pgprot_t prot, struct page **pages, int *nr, pgtbl_mod_mask *mask) @@ -1712,7 +1726,7 @@ static void free_vmap_area_noflush(struct vmap_area *va) static void free_unmap_vmap_area(struct vmap_area *va) { flush_cache_vunmap(va->va_start, va->va_end); - unmap_kernel_range_noflush(va->va_start, va->va_end - va->va_start); + vunmap_range_noflush(va->va_start, va->va_end); if (debug_pagealloc_enabled_static()) flush_tlb_kernel_range(va->va_start, va->va_end); @@ -1990,7 +2004,7 @@ static void vb_free(unsigned long addr, unsigned long size) offset = (addr & (VMAP_BLOCK_SIZE - 1)) >> PAGE_SHIFT; vb = xa_load(&vmap_blocks, addr_to_vb_idx(addr)); - unmap_kernel_range_noflush(addr, size); + vunmap_range_noflush(addr, addr + size); if (debug_pagealloc_enabled_static()) flush_tlb_kernel_range(addr, addr + size); @@ -2307,23 +2321,6 @@ void __init vmalloc_init(void) vmap_initialized = true; } -/** - * unmap_kernel_range - unmap kernel VM area and flush cache and TLB - * @addr: start of the VM area to unmap - * @size: size of the VM area to unmap - * - * Similar to unmap_kernel_range_noflush() but flushes vcache before - * the unmapping and tlb after. - */ -void unmap_kernel_range(unsigned long addr, unsigned long size) -{ - unsigned long end = addr + size; - - flush_cache_vunmap(addr, end); - unmap_kernel_range_noflush(addr, size); - flush_tlb_kernel_range(addr, end); -} - static inline void setup_vmalloc_vm_locked(struct vm_struct *vm, struct vmap_area *va, unsigned long flags, const void *caller) { From patchwork Tue Jan 26 04:54:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 12045233 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DCE7BC433E6 for ; Tue, 26 Jan 2021 04:54:31 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 885F32256F for ; Tue, 26 Jan 2021 04:54:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 885F32256F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 251968D008D; Mon, 25 Jan 2021 23:54:31 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 203058D0076; Mon, 25 Jan 2021 23:54:31 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0CA688D008D; Mon, 25 Jan 2021 23:54:31 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id ECC018D0076 for ; Mon, 25 Jan 2021 23:54:30 -0500 (EST) Received: from smtpin02.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id A965B181AF5C3 for ; Tue, 26 Jan 2021 04:54:30 +0000 (UTC) X-FDA: 77746710300.02.sort51_4e1415d2758b Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin02.hostedemail.com (Postfix) with ESMTP id 8121B10097AA0 for ; Tue, 26 Jan 2021 04:54:30 +0000 (UTC) X-HE-Tag: sort51_4e1415d2758b X-Filterd-Recvd-Size: 6543 Received: from mail-pj1-f47.google.com (mail-pj1-f47.google.com [209.85.216.47]) by imf08.hostedemail.com (Postfix) with ESMTP for ; Tue, 26 Jan 2021 04:54:29 +0000 (UTC) Received: by mail-pj1-f47.google.com with SMTP id gx1so1522676pjb.1 for ; Mon, 25 Jan 2021 20:54:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=zWzZEjtMI7dPvku+j5LOMBapAYYLqsSnngDI0xcSPtU=; b=mdZXnij8j2OhoSuz0AxfBhIxqiB+8yZ9jo3y8Eyh3ODyLDaCAyDWBPMrglhVBGkODv Crv1+LoHFbqcgWZZnnx6/1VH42DX6IXP0AbtgUKK9Hwhj8tjohWGrg0iNVALj8E40VLK z6I8tpUKtdoi+7EbBR7z/JUJLBMD3LUmZCurMjMSOZR8P46fh+ToDppHamydxYc8CWVv qQQ1O7kYy1617y4/KRAXlD3Dehj3QYGfjPvkDOZ6Slt4w9wxY1jLKpBsZbzzv/Ypq2Hv oIZDmQy5/Ze9lBQxTJ5oVqFmvxbJD35P165x33/BDl34KS+z/TIglWhWZ3uMMtmNbIwl XY8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=zWzZEjtMI7dPvku+j5LOMBapAYYLqsSnngDI0xcSPtU=; b=udycvqTa4lpG5aGEAYaDEgpJWAxXkM67pRmSNvLQvTZdgmPQ2UuuWvqYmjOsHdWl7F lF5yENr4IGBUveE86dmA5AXS+0XP2bINGVfG1V2/Zgttkjbp9VKmp9FAubI3WBnH8PKv fIyXPpFPDeA4anbizag/Po5UKOLXhojvhvZTQ2cBBpvL74sYB/CR2ovQGxRFUWM+db5R FEJPEsfpr6amD+N6xb2TTdXpa6IKUPhEE3R/xt0dOn3WjCk3DXXKeSD9EGrygLEvJSBG tNMGkXKXq0GbdrYqvNTzmN+cKwUWRpLCCVTV6olPeNz51md6AGz417WIdZji39H0Glqk lFlQ== X-Gm-Message-State: AOAM532aYindlvFdxM1ZnIi7mJZ3H4R/h658123LHCG4UetkkHC4lcyH PESrCHC6iZXQ/OeshAgUcuIAe4dp5os= X-Google-Smtp-Source: ABdhPJywTkPitOcbrsL64q10OuxjK3xV19FLJoWY/kuHyIiloXjVdY125pYsW/JbsEdfQlC4T2Htig== X-Received: by 2002:a17:90a:4209:: with SMTP id o9mr4130449pjg.75.1611636869086; Mon, 25 Jan 2021 20:54:29 -0800 (PST) Received: from bobo.ozlabs.ibm.com (192.156.221.203.dial.dynamic.acc50-nort-cbr.comindico.com.au. [203.221.156.192]) by smtp.gmail.com with ESMTPSA id w184sm18803382pgb.71.2021.01.25.20.54.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Jan 2021 20:54:28 -0800 (PST) From: Nicholas Piggin To: linux-mm@kvack.org, Andrew Morton Cc: Nicholas Piggin , linux-kernel@vger.kernel.org, Christoph Hellwig Subject: [PATCH 5/5] mm/vmalloc: improve allocation failure error messages Date: Tue, 26 Jan 2021 14:54:04 +1000 Message-Id: <20210126045404.2492588-6-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210126045404.2492588-1-npiggin@gmail.com> References: <20210126045404.2492588-1-npiggin@gmail.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: There are several reasons why a vmalloc can fail, virtual space exhausted, page array allocation failure, page allocation failure, and kernel page table allocation failure. Add distinct warning messages for the main causes of failure, with some added information like page order or allocation size where applicable. Signed-off-by: Nicholas Piggin Reviewed-by: Christoph Hellwig --- mm/vmalloc.c | 40 ++++++++++++++++++++++++++++------------ 1 file changed, 28 insertions(+), 12 deletions(-) diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 5ff190590fe4..4facf582a3be 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -2790,6 +2790,10 @@ static void *__vmalloc_area_node(struct vm_struct *area, gfp_t gfp_mask, if (!pages) { free_vm_area(area); + warn_alloc(gfp_mask, NULL, + "vmalloc size %lu allocation failure: " + "page array size %lu allocation failed", + area->nr_pages * PAGE_SIZE, array_size); return NULL; } @@ -2813,6 +2817,10 @@ static void *__vmalloc_area_node(struct vm_struct *area, gfp_t gfp_mask, /* Successfully allocated i pages, free them in __vfree() */ area->nr_pages = i; atomic_long_add(area->nr_pages, &nr_vmalloc_pages); + warn_alloc(gfp_mask, NULL, + "vmalloc size %lu allocation failure: " + "page order %u allocation failed", + area->nr_pages * PAGE_SIZE, page_order); goto fail; } @@ -2824,15 +2832,17 @@ static void *__vmalloc_area_node(struct vm_struct *area, gfp_t gfp_mask, } atomic_long_add(area->nr_pages, &nr_vmalloc_pages); - if (vmap_pages_range(addr, addr + size, prot, pages, page_shift) < 0) + if (vmap_pages_range(addr, addr + size, prot, pages, page_shift) < 0) { + warn_alloc(gfp_mask, NULL, + "vmalloc size %lu allocation failure: " + "failed to map pages", + area->nr_pages * PAGE_SIZE); goto fail; + } return area->addr; fail: - warn_alloc(gfp_mask, NULL, - "vmalloc: allocation failure, allocated %ld of %ld bytes", - (area->nr_pages*PAGE_SIZE), size); __vfree(area->addr); return NULL; } @@ -2866,8 +2876,15 @@ void *__vmalloc_node_range(unsigned long size, unsigned long align, unsigned long real_align = align; unsigned int shift = PAGE_SHIFT; - if (!size || (size >> PAGE_SHIFT) > totalram_pages()) - goto fail; + if (WARN_ON_ONCE(!size)) + return NULL; + + if ((size >> PAGE_SHIFT) > totalram_pages()) { + warn_alloc(gfp_mask, NULL, + "vmalloc size %lu allocation failure: " + "exceeds total pages", real_size); + return NULL; + } if (vmap_allow_huge && !(vm_flags & VM_NO_HUGE_VMAP) && arch_vmap_pmd_supported(prot)) { @@ -2894,8 +2911,12 @@ void *__vmalloc_node_range(unsigned long size, unsigned long align, size = PAGE_ALIGN(size); area = __get_vm_area_node(size, align, VM_ALLOC | VM_UNINITIALIZED | vm_flags, start, end, node, gfp_mask, caller); - if (!area) + if (!area) { + warn_alloc(gfp_mask, NULL, + "vmalloc size %lu allocation failure: " + "vm_struct allocation failed", real_size); goto fail; + } addr = __vmalloc_area_node(area, gfp_mask, prot, shift, node); if (!addr) @@ -2920,11 +2941,6 @@ void *__vmalloc_node_range(unsigned long size, unsigned long align, goto again; } - if (!area) { - /* Warn for area allocation, page allocations already warn */ - warn_alloc(gfp_mask, NULL, - "vmalloc: allocation failure: %lu bytes", real_size); - } return NULL; }