From patchwork Sun Aug 20 19:40:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13358885 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 45F86EE4996 for ; Sun, 20 Aug 2023 19:40:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 567398E0005; Sun, 20 Aug 2023 15:40:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 516968D0001; Sun, 20 Aug 2023 15:40:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3DE658E0005; Sun, 20 Aug 2023 15:40:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 2A31B8D0001 for ; Sun, 20 Aug 2023 15:40:41 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id F06F9140789 for ; Sun, 20 Aug 2023 19:40:40 +0000 (UTC) X-FDA: 81145500240.11.1CE991E Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf13.hostedemail.com (Postfix) with ESMTP id 4547720005 for ; Sun, 20 Aug 2023 19:40:39 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=I0Bjq8yy; dmarc=none; spf=none (imf13.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1692560439; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=yffIQNHXKypeq1zIbfQo6NWIljVNd0Nx7ZtnKKW0FkU=; b=TwOQGzEvaFO+wQvVoHSE0Xi0UfiJOE+gKxaPjM51E4icHEFJifpPXPMStBEbVi4ccIq1Gr CFauXd1uIyibP0dVPrAJSeDvbxuhRE+GutJt8+tS4xxVrO4a9D20Y8RdUcT0+orueFM4ht o6KXaZyUJR6n9H54kf+RsYxS3LwVSbI= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=I0Bjq8yy; dmarc=none; spf=none (imf13.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1692560439; a=rsa-sha256; cv=none; b=qogRgGMl3jHyd6Fr9eDUjGQ+aQMHCSGOC3fEmo9ecMJOtaTynBziM/MobrqsNY8kWJYci+ Uw/I0MVPMbMasodV03hL1A+CxMGAyJVJZcbPD/jmtmJodSM0IAwZBKJP++PSVHK0u2ygnz iZlXtVS0eih/x69kaE94lNj0xResRvw= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:In-Reply-To:References; bh=yffIQNHXKypeq1zIbfQo6NWIljVNd0Nx7ZtnKKW0FkU=; b=I0Bjq8yycB77341iiR/zTAxcIT vO+8N98DOUbc+RGu/Hqc9NyFi8XxjM7Ml3FihLSP6eYTbFqteKBbanuDDIeqYHLBc1UqJv/PPnI/b EuMHcb5LEPsYDFkawKb47igSVrxW9de7/ZdedEsqOok3aN7UsiBVnaR9C6QO9fXOIIhAt3Osayuh/ o4dREqomZL9IE0AcG5x1+9nEr82plu1WwZzXQEkskM+PkoJguDPKst2yPvfzrGq8yQACK1k2b+xl8 otLkyw28y1S9EpqrT4tB88+5sMSLxVjr0fA/n5TzLXr4irModmrbclMIhIhdgi1b4elpL7nlF9Mn0 IDgQeMbw==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1qXoHq-005r8H-Gc; Sun, 20 Aug 2023 19:40:34 +0000 From: "Matthew Wilcox (Oracle)" To: Uladzislau Rezki Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH] mm: Add vmalloc_user_node() Date: Sun, 20 Aug 2023 20:40:31 +0100 Message-Id: <20230820194031.1395870-1-willy@infradead.org> X-Mailer: git-send-email 2.37.1 MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: h8bqm1mn47966bcz6sxjd8f77upy9jx9 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 4547720005 X-HE-Tag: 1692560439-439976 X-HE-Meta: U2FsdGVkX1/zcJjxoKtA+Y+s7QswT5PvJp28tJHGhb5M3R63bxlKixNJp/8YJQUm8zVrC3+H0IuEIcUVONsiauM36B4iBeQtuh+vj0G0oyFb/nq+TrsDpjEl69b2QIvjyAn1OP4Cji/6zO4BlVvDVDhGWJHf8hSsevtjc9gJaN8rnuxIZB8rTTFdHnqrF731M0Uw5l3zmFuY46PJyO1gsQaivUqQpMqr8+sKVkf0ha/hY0i6sPzaz3ePQeGmFLeNZVXWRtdBB9wzcwyRFmUFsL1V6EkXD2UwAV6XEKdJ0apJg6+P/UHbUDaGzN/p5O/HzPXWeicboV3em0PBd/0Qw3YDYjrayHVxjEf3A2R2/decSXQNtgh/5quSxaoYm4H/FxHebfeokd3tIUJ5L3Tgk+9pW8HGq9UFeAtnUbEFS5y8rjtM2ocR8FvUddF3nB2VeHU/NlBsOWgZI7AHFaEwprkntbJbTweeD6865IwvieDTsIgMN62WYieqgv2jW0xOpNg6KJHUTIPaXrSPU6IKa9CcuyVyy/+5KE8/pkIGKXbSKkZr/dct4hV2THPenOyDOUvWzsR6uwxSqu6GFhAFp7AtsIGGWDtC+oL+1XD7WXFD4DmecRGRwhNASlMWY2BYDaVdBs2BcMS7QjjiFTn6pFu4Rs/wcnfAEAG0URq0HKDvThiwo2ov4amJlSF+AsPzP/4zVIKdxkcH1q6TT8RDHuFqxbLyJV+fxCIj1ZH43Y+/pnut8qVKuV2/CKJPfBkY1JxOCfBP+H0330SNc6lYysisdgxeayFTdbq2QXpeAJEkbFmJzWcbCiICX6j3eX65U+DLHaP/vb4MRxUgzxa4TTL4ikbip7JQpv4ZJIWNCIo+dIj8XtNJ7zfvCjpsoJtNIh7XmCNx17FAt28So3L4Xj/wcrZt4WNJtljz0HYCL0WDtWmJ3VShG9cBI9DIzh5weQEz7VaWdiraIMK8yzo D21t3aPD SxIbUEz1L3VjoWltN+aZEraW2nT2k6hAU1mEbfC+8fpwV+63UC0k1mJy0UHKrguAZ5hOKdplcYF1vsVgnil582zO1uKE59sh28SZT9HptRbB+39wf0MYAYrNrK58ov+9CCt00wlGLMKq4ySC27PNL3Pv+KPkoI/ZGN8bcDL9TR+Xo1mLnQEDoe40B7+sdXxjjBb3EK/DHvVNDK+w36NikV3nRyHZRx7Wq1LHgc/uWBCQeFdiGVYUQg4ZS8Eok8qSKEfY50Ti1j/C0Ksk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Allow memory to be allocated on a specified node. Use it in the perf ring-buffer code. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: David Hildenbrand Reviewed-by: Lorenzo Stoakes --- I was looking at the perf ringbuffer code for other reasons, and I noticed that everything else gets allocated on the specified node ... except there's no way to do this for vmalloc. I assume it'd be helpful here too. include/linux/vmalloc.h | 17 ++++++++++++++++- kernel/events/ring_buffer.c | 2 +- mm/vmalloc.c | 9 +++++---- 3 files changed, 22 insertions(+), 6 deletions(-) diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h index c720be70c8dd..030bfe1a60ab 100644 --- a/include/linux/vmalloc.h +++ b/include/linux/vmalloc.h @@ -6,6 +6,7 @@ #include #include #include +#include #include /* pgprot_t */ #include #include @@ -139,7 +140,7 @@ static inline unsigned long vmalloc_nr_pages(void) { return 0; } extern void *vmalloc(unsigned long size) __alloc_size(1); extern void *vzalloc(unsigned long size) __alloc_size(1); -extern void *vmalloc_user(unsigned long size) __alloc_size(1); +extern void *vmalloc_user_node(unsigned long size, int node) __alloc_size(1); extern void *vmalloc_node(unsigned long size, int node) __alloc_size(1); extern void *vzalloc_node(unsigned long size, int node) __alloc_size(1); extern void *vmalloc_32(unsigned long size) __alloc_size(1); @@ -158,6 +159,20 @@ extern void *vmalloc_array(size_t n, size_t size) __alloc_size(1, 2); extern void *__vcalloc(size_t n, size_t size, gfp_t flags) __alloc_size(1, 2); extern void *vcalloc(size_t n, size_t size) __alloc_size(1, 2); +/** + * vmalloc_user - allocate zeroed virtually contiguous memory for userspace + * @size: allocation size + * + * The resulting memory area is zeroed so it can be mapped to userspace + * without leaking data. + * + * Return: pointer to the allocated memory or %NULL on error + */ +static inline void *vmalloc_user(size_t size) +{ + return vmalloc_user_node(size, NUMA_NO_NODE); +} + extern void vfree(const void *addr); extern void vfree_atomic(const void *addr); diff --git a/kernel/events/ring_buffer.c b/kernel/events/ring_buffer.c index cc90d5299005..c73add132618 100644 --- a/kernel/events/ring_buffer.c +++ b/kernel/events/ring_buffer.c @@ -918,7 +918,7 @@ struct perf_buffer *rb_alloc(int nr_pages, long watermark, int cpu, int flags) INIT_WORK(&rb->work, rb_free_work); - all_buf = vmalloc_user((nr_pages + 1) * PAGE_SIZE); + all_buf = vmalloc_user_node((nr_pages + 1) * PAGE_SIZE, node); if (!all_buf) goto fail_all_buf; diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 228a4a5312f2..3616bfe4348f 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -3461,22 +3461,23 @@ void *vzalloc(unsigned long size) EXPORT_SYMBOL(vzalloc); /** - * vmalloc_user - allocate zeroed virtually contiguous memory for userspace + * vmalloc_user_node - allocate zeroed virtually contiguous memory for userspace * @size: allocation size + * @node: NUMA node * * The resulting memory area is zeroed so it can be mapped to userspace * without leaking data. * * Return: pointer to the allocated memory or %NULL on error */ -void *vmalloc_user(unsigned long size) +void *vmalloc_user_node(unsigned long size, int node) { return __vmalloc_node_range(size, SHMLBA, VMALLOC_START, VMALLOC_END, GFP_KERNEL | __GFP_ZERO, PAGE_KERNEL, - VM_USERMAP, NUMA_NO_NODE, + VM_USERMAP, node, __builtin_return_address(0)); } -EXPORT_SYMBOL(vmalloc_user); +EXPORT_SYMBOL(vmalloc_user_node); /** * vmalloc_node - allocate memory on a specific node