From patchwork Fri Feb 10 15:47:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 13135958 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 13404C05027 for ; Fri, 10 Feb 2023 15:48:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7CAB36B0153; Fri, 10 Feb 2023 10:47:59 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 753F46B0155; Fri, 10 Feb 2023 10:47:59 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5CD516B0156; Fri, 10 Feb 2023 10:47:59 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 44FCE6B0153 for ; Fri, 10 Feb 2023 10:47:59 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id C2384403CE for ; Fri, 10 Feb 2023 15:47:58 +0000 (UTC) X-FDA: 80451813036.27.4EE647A Received: from mail-pf1-f181.google.com (mail-pf1-f181.google.com [209.85.210.181]) by imf23.hostedemail.com (Postfix) with ESMTP id DC51B140022 for ; Fri, 10 Feb 2023 15:47:56 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=oZ0tIoXB; spf=pass (imf23.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.210.181 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676044076; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=i3DzMoNfpXAhki9P1E058MwckxQ0M07ZDtVShvGycKA=; b=cCI2Zvjq1ZcLtvZk1jdKQxiixcpxn3E1JdgY2H8peaVelP3xd+sEfpjVC+JjNXHCbb2xaj fK/lX+G5iTOH7C/oQO/qg1Im+wQvuzjnbwVW7RouqB2CudOXkp1lgB54QhmI98N1ohuTWE tddGLihQx7QE/cJSD6GjIxJI5rB29a8= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=oZ0tIoXB; spf=pass (imf23.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.210.181 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676044076; a=rsa-sha256; cv=none; b=BQZh/mWg3bnFg3kQTzcq9mMkDllkjQ5cjUxcFYA0yivsjiR8BS4MLyTcEH50knyz0Qj9en petOQU5M6go2reHIosHkmOMUIBqunXBtnbXgMIRCOKYtEgjvLO8NsG0ByVVlj8BzKElHGT ITi+0ArZyKbjGow99xg2FqOUliuBzoY= Received: by mail-pf1-f181.google.com with SMTP id ay1so3719386pfb.7 for ; Fri, 10 Feb 2023 07:47:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=i3DzMoNfpXAhki9P1E058MwckxQ0M07ZDtVShvGycKA=; b=oZ0tIoXBZYNSfCI21Hayv+qmA7VSMOoMMOB6Lb5LFDSNKWa5ZA1OvywGXFbCWh8rAn fveeyllUwSACZz2OlzAqiEo59qwmLf7GZhgB7W3iSPCLWMmsW6NA4ZnSXreyGxulD3/y 9BMmX3XJwetCzbqJ0iYfMiIavh+R1uAh+K3hErbzqpOKmoIYaCalasRZ8UQ2h7L6Ne2i nkGZnjmlwKEJDDtCSqQOZJFTk3tODlmIZL5QpGwW6jhKIrJzTnsshBKamVpjVcGittNQ 6qyTM5pH4ToQwYnaUukshYiUudfoiKYDUGugo9i8aciEhz/+nkNDHKspgmXIQmp4xwow ncQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=i3DzMoNfpXAhki9P1E058MwckxQ0M07ZDtVShvGycKA=; b=zXMkRLYm9IAai8auZfzVOamBMOMav9OQ+CzHvahSvoAvsU+TSt4T1AbPn3Z/406G91 HFre0Y0KSEFWKnr7682kSR1xsR/+xQYXL/awM7fBLTou9XAaaM+RkFFZusSrDFyR1dZ7 yyl14XGfvBS0F1m7kqQuz4Q51SE+cCINljWsF7M2z/Cvd8y8zJ4af/rhh9XMpnv8rCaQ egGyExW/2BxaDPIiTDIsdtIamd43MbMstQU7Nctbqm06LpWZ1VGVonFSrHlrX6pOSkbd ck/pDXvRde/mI3VQqwfuiTjCIrNhC1uZizFSZJ0qFMjoDogGip9tizbV1fM3f+byLQSz J/QA== X-Gm-Message-State: AO0yUKWWV/29i97PMIqW9zhdyvn3bGPyxyw8DHQMN4vk6ITNcKAA/7WX pMAfDjSHpSNXsqx+LwnpKJg= X-Google-Smtp-Source: AK7set/wDZlxvee4YGZQEWH6VZZuQhWxaDntwzaB44hF8rNeick+jUa8cfDxbgwrQy6oPT1YaThwYg== X-Received: by 2002:a62:380b:0:b0:5a8:58b5:bfa9 with SMTP id f11-20020a62380b000000b005a858b5bfa9mr5165943pfa.12.1676044075752; Fri, 10 Feb 2023 07:47:55 -0800 (PST) Received: from vultr.guest ([2001:19f0:7001:2f6a:5400:4ff:fe4c:e050]) by smtp.gmail.com with ESMTPSA id t20-20020aa79394000000b005921c46cbadsm3520069pfe.99.2023.02.10.07.47.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Feb 2023 07:47:55 -0800 (PST) From: Yafang Shao To: tj@kernel.org, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, hannes@cmpxchg.org, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, muchun.song@linux.dev, akpm@linux-foundation.org Cc: bpf@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org, Yafang Shao Subject: [PATCH bpf-next v2 2/4] bpf: use bpf_map_kvcalloc in bpf_local_storage Date: Fri, 10 Feb 2023 15:47:32 +0000 Message-Id: <20230210154734.4416-3-laoar.shao@gmail.com> X-Mailer: git-send-email 2.39.0 In-Reply-To: <20230210154734.4416-1-laoar.shao@gmail.com> References: <20230210154734.4416-1-laoar.shao@gmail.com> MIME-Version: 1.0 X-Stat-Signature: gxq6bxee7njif1oa8ffu1pjyjkncnn13 X-Rspam-User: X-Rspamd-Queue-Id: DC51B140022 X-Rspamd-Server: rspam06 X-HE-Tag: 1676044076-319532 X-HE-Meta: U2FsdGVkX182fOQuTZTZhrn53m4nzYcb0nfHLJBeXXfLBBBo7EwlBROh9bLZhZCCFL4Q/PuyxQ2bNdV2L1/UKkXTb5c0Q/R6xyWT4oTleqEk/vNNJFQDIsHzHBn3mRzj4h7K20+l0vFXrvCxC2ptXyv9rCJIXK9oxHBzUtkGqh5sfeTYCE07FkVUd+TiQkAUxH6hxdmctaGOAkt9cU0dWeyj/cfP9/ddhWlIJ2i+xJDKobQZhsXQF4XdkBLJ7wcjZeAMsVBtC9wwFND/pwKBKLthR0f0J8hMrS7huuiUA9inqneKqHswQA96P4vR05PVZE1cKr8U1DA2wquaShF0uoXnJqR1Juhs7MeFnxAvxxFsuKol9OpMKKjkc/jwq3C3Cehz1YygLMcpMI2r6w0tDAWAlcwEuMNuAdJhALdTlQguXNyvrejGeL2SI4rAR8rKRiFjiNuIQXhxqyYgExC0vBxQWJQV1mW4tCmK8KfnX7p/REQA6Q7bJj9sdDNFtOBpiS4w8sDyiXsJM5lOdyNFGa8fsQhSZrUU+lhpOBdAbZLkBHbNdkmbiiNTgBMeVn9oA6KEeWnnw9DwBbOL/3Ab0CcCJADDEZCrgQyl+gQamUCmIhytKKbKwYGEIlhuUEWYpQV4Kt/Ad14cjXm4BGkLZ/Dt5xJTpPqpujENb2hBbYpLaS+oViVQOxhN5toyFxGWV+AZvPbdNcSX3n6e2NYuFdbYP0q2AgFMXzw1nNOzD9l6oYnoCk8IOQjvBC/gNIBATzUgytYDY5YH9nh+NQyEqj/yBoukYmnjOgjD+Tbqd6Kc6feHwAb+oZWhVQl7pzwb0XYnIDxG+7S8RhmYHwHdLRBzTlKjft5MdYzNG8YiRBZ8BrfjkU2Xagp7LLd+2E61zJevlC/wbPnCBq1mirjt2hMkO5oZhP3hx4TLBQBTia+Tf92t+EY21iZ/RDcgNi/eq3O9IsuEpFaaoGrKYqk GwRgq66q bMqa9dO0WWVve7tOGNq1IKHHDnBXaoBmOBIM5JoiSAJS9/r5+E788krV6XrF+BPOIRzGFBLpEeerE+NxNDJE8sCEJVlnlx+/jKtBG8603m6vA+lbRWUY4EaGqeiJnJPRRkimdrlrJ804TUYX5N5o79gZ+kPoJSFTD+mFVu6tk2yriFqTUk//lwHZa9exj8kb5U5jXoHh9e1a/gRfkV6TS7q8vpaUzx6i/MfVRrklkCSNi19YnOTEIxGdQDYHxjxGaiI6kzOS3/i68iv7NAEaytetMFsrRCkET7vedQTwnea+yO2SpxyjqvE7FXUmI4pjHlevI2+b+CuHwADnd2iKsTAI1gM+eatHbysUJ0L330yKBothxx23LxUOUAzLRAWKol4ZxisVl4lBD/rK5TZxHA4XCKS+2QwZsqf+Q8y4CTwFcQVR3lsTd0qH918nchj0gdIKDZiqTG3r33x1TheYfqHOsuwsdfgEfAk17PSvT46J3tR9b4nhQnhdwfbW3LMDQYbHowwdKERPK1aLv182bwre6f87VCVQl8eZA/5M8VbNNu9JJKT+NLgwAJTZdhQn+yuVLpbQthR6R7nvpI7PNHcB6ft/dAYsCiPJifYb0Bshg0H1Z+fbWLHVHiyklquPxwGyI8IJEWUW6N+CgNigifBvZgA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Introduce new helper bpf_map_kvcalloc() for the memory allocation in bpf_local_storage(). Then the allocation will charge the memory from the map instead of from current, though currently they are the same thing as it is only used in map creation path now. By charging map's memory into the memcg from the map, it will be more clear. Signed-off-by: Yafang Shao Acked-by: Johannes Weiner Acked-by: Roman Gushchin --- include/linux/bpf.h | 8 ++++++++ kernel/bpf/bpf_local_storage.c | 4 ++-- kernel/bpf/syscall.c | 15 +++++++++++++++ 3 files changed, 25 insertions(+), 2 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 35c18a9..fe0bf48 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -1886,6 +1886,8 @@ int generic_map_delete_batch(struct bpf_map *map, void *bpf_map_kmalloc_node(const struct bpf_map *map, size_t size, gfp_t flags, int node); void *bpf_map_kzalloc(const struct bpf_map *map, size_t size, gfp_t flags); +void *bpf_map_kvcalloc(struct bpf_map *map, size_t n, size_t size, + gfp_t flags); void __percpu *bpf_map_alloc_percpu(const struct bpf_map *map, size_t size, size_t align, gfp_t flags); #else @@ -1902,6 +1904,12 @@ void __percpu *bpf_map_alloc_percpu(const struct bpf_map *map, size_t size, return kzalloc(size, flags); } +static inline void * +bpf_map_kvcalloc(struct bpf_map *map, size_t n, size_t size, gfp_t flags) +{ + return kvcalloc(n, size, flags); +} + static inline void __percpu * bpf_map_alloc_percpu(const struct bpf_map *map, size_t size, size_t align, gfp_t flags) diff --git a/kernel/bpf/bpf_local_storage.c b/kernel/bpf/bpf_local_storage.c index 373c3c2..35f4138 100644 --- a/kernel/bpf/bpf_local_storage.c +++ b/kernel/bpf/bpf_local_storage.c @@ -568,8 +568,8 @@ static struct bpf_local_storage_map *__bpf_local_storage_map_alloc(union bpf_att nbuckets = max_t(u32, 2, nbuckets); smap->bucket_log = ilog2(nbuckets); - smap->buckets = kvcalloc(sizeof(*smap->buckets), nbuckets, - GFP_USER | __GFP_NOWARN | __GFP_ACCOUNT); + smap->buckets = bpf_map_kvcalloc(&smap->map, sizeof(*smap->buckets), + nbuckets, GFP_USER | __GFP_NOWARN); if (!smap->buckets) { bpf_map_area_free(smap); return ERR_PTR(-ENOMEM); diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index bcc9761..9d94a35 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -464,6 +464,21 @@ void *bpf_map_kzalloc(const struct bpf_map *map, size_t size, gfp_t flags) return ptr; } +void *bpf_map_kvcalloc(struct bpf_map *map, size_t n, size_t size, + gfp_t flags) +{ + struct mem_cgroup *memcg, *old_memcg; + void *ptr; + + memcg = bpf_map_get_memcg(map); + old_memcg = set_active_memcg(memcg); + ptr = kvcalloc(n, size, flags | __GFP_ACCOUNT); + set_active_memcg(old_memcg); + mem_cgroup_put(memcg); + + return ptr; +} + void __percpu *bpf_map_alloc_percpu(const struct bpf_map *map, size_t size, size_t align, gfp_t flags) {