From patchwork Thu Jul 7 22:35:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Song Liu X-Patchwork-Id: 12910355 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2BEC4C43334 for ; Thu, 7 Jul 2022 22:36:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236797AbiGGWga convert rfc822-to-8bit (ORCPT ); Thu, 7 Jul 2022 18:36:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35620 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236800AbiGGWg3 (ORCPT ); Thu, 7 Jul 2022 18:36:29 -0400 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 59286DF5 for ; Thu, 7 Jul 2022 15:36:28 -0700 (PDT) Received: from pps.filterd (m0109334.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 267KPopX010715 for ; Thu, 7 Jul 2022 15:36:28 -0700 Received: from maileast.thefacebook.com ([163.114.130.16]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3h5ashmbgn-13 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Thu, 07 Jul 2022 15:36:27 -0700 Received: from twshared0725.22.frc3.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:83::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Thu, 7 Jul 2022 15:36:20 -0700 Received: by devbig932.frc1.facebook.com (Postfix, from userid 4523) id 573139D349C0; Thu, 7 Jul 2022 15:36:02 -0700 (PDT) From: Song Liu To: , , CC: , , , , , , Song Liu Subject: [PATCH v6 bpf-next 1/5] module: introduce module_alloc_huge Date: Thu, 7 Jul 2022 15:35:42 -0700 Message-ID: <20220707223546.4124919-2-song@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220707223546.4124919-1-song@kernel.org> References: <20220707223546.4124919-1-song@kernel.org> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-ORIG-GUID: Air8S56dKx8b7YWLXyZcvzZesVoAUuYv X-Proofpoint-GUID: Air8S56dKx8b7YWLXyZcvzZesVoAUuYv X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-07-07_17,2022-06-28_01,2022-06-22_01 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Introduce module_alloc_huge, which allocates huge page backed memory in module memory space. The primary user of this memory is bpf_prog_pack (multiple BPF programs sharing a huge page). Signed-off-by: Song Liu --- arch/x86/kernel/module.c | 21 +++++++++++++++++++++ include/linux/moduleloader.h | 5 +++++ kernel/module/main.c | 8 ++++++++ 3 files changed, 34 insertions(+) diff --git a/arch/x86/kernel/module.c b/arch/x86/kernel/module.c index b98ffcf4d250..63f6a16c70dc 100644 --- a/arch/x86/kernel/module.c +++ b/arch/x86/kernel/module.c @@ -86,6 +86,27 @@ void *module_alloc(unsigned long size) return p; } +void *module_alloc_huge(unsigned long size) +{ + gfp_t gfp_mask = GFP_KERNEL; + void *p; + + if (PAGE_ALIGN(size) > MODULES_LEN) + return NULL; + + p = __vmalloc_node_range(size, MODULE_ALIGN, + MODULES_VADDR + get_module_load_offset(), + MODULES_END, gfp_mask, PAGE_KERNEL, + VM_DEFER_KMEMLEAK | VM_ALLOW_HUGE_VMAP, + NUMA_NO_NODE, __builtin_return_address(0)); + if (p && (kasan_alloc_module_shadow(p, size, gfp_mask) < 0)) { + vfree(p); + return NULL; + } + + return p; +} + #ifdef CONFIG_X86_32 int apply_relocate(Elf32_Shdr *sechdrs, const char *strtab, diff --git a/include/linux/moduleloader.h b/include/linux/moduleloader.h index 9e09d11ffe5b..d34743a88938 100644 --- a/include/linux/moduleloader.h +++ b/include/linux/moduleloader.h @@ -26,6 +26,11 @@ unsigned int arch_mod_section_prepend(struct module *mod, unsigned int section); sections. Returns NULL on failure. */ void *module_alloc(unsigned long size); +/* Allocator used for allocating memory in module memory space. If size is + * greater than PMD_SIZE, allow using huge pages. Returns NULL on failure. + */ +void *module_alloc_huge(unsigned long size); + /* Free memory returned from module_alloc. */ void module_memfree(void *module_region); diff --git a/kernel/module/main.c b/kernel/module/main.c index fed58d30725d..349b2a8bd20f 100644 --- a/kernel/module/main.c +++ b/kernel/module/main.c @@ -1613,6 +1613,14 @@ void * __weak module_alloc(unsigned long size) NUMA_NO_NODE, __builtin_return_address(0)); } +void * __weak module_alloc_huge(unsigned long size) +{ + return __vmalloc_node_range(size, 1, VMALLOC_START, VMALLOC_END, + GFP_KERNEL, PAGE_KERNEL_EXEC, + VM_FLUSH_RESET_PERMS | VM_ALLOW_HUGE_VMAP, + NUMA_NO_NODE, __builtin_return_address(0)); +} + bool __weak module_init_section(const char *name) { return strstarts(name, ".init"); From patchwork Thu Jul 7 22:35:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Song Liu X-Patchwork-Id: 12910354 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C001BC433EF for ; Thu, 7 Jul 2022 22:36:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236665AbiGGWg3 convert rfc822-to-8bit (ORCPT ); Thu, 7 Jul 2022 18:36:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35614 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236797AbiGGWg2 (ORCPT ); Thu, 7 Jul 2022 18:36:28 -0400 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B02D6D86 for ; Thu, 7 Jul 2022 15:36:27 -0700 (PDT) Received: from pps.filterd (m0044010.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 267KPkcQ013753 for ; Thu, 7 Jul 2022 15:36:27 -0700 Received: from maileast.thefacebook.com ([163.114.130.16]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3h4uaqsjpa-5 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Thu, 07 Jul 2022 15:36:27 -0700 Received: from twshared30313.14.frc2.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:83::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Thu, 7 Jul 2022 15:36:24 -0700 Received: by devbig932.frc1.facebook.com (Postfix, from userid 4523) id 79C5B9D349C1; Thu, 7 Jul 2022 15:36:04 -0700 (PDT) From: Song Liu To: , , CC: , , , , , , Song Liu Subject: [PATCH v6 bpf-next 2/5] bpf: use module_alloc_huge for bpf_prog_pack Date: Thu, 7 Jul 2022 15:35:43 -0700 Message-ID: <20220707223546.4124919-3-song@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220707223546.4124919-1-song@kernel.org> References: <20220707223546.4124919-1-song@kernel.org> X-FB-Internal: Safe X-Proofpoint-GUID: TMtU17Gf5bBUxhH3eEvQOPOP2z9wM1PF X-Proofpoint-ORIG-GUID: TMtU17Gf5bBUxhH3eEvQOPOP2z9wM1PF X-Proofpoint-UnRewURL: 0 URL was un-rewritten MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-07-07_17,2022-06-28_01,2022-06-22_01 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Use module_alloc_huge for bpf_prog_pack so that BPF programs sit on PMD_SIZE pages. This benefits system performance by reducing iTLB miss rate. Benchmark of a real web service workload shows this change gives another ~0.2% performance boost on top of PAGE_SIZE bpf_prog_pack (which improve system throughput by ~0.5%). Also, remove set_vm_flush_reset_perms() from alloc_new_pack() and use set_memory_[nx|rw] in bpf_prog_pack_free(). This is because VM_FLUSH_RESET_PERMS does not work with huge pages yet. [1] [1] https://lore.kernel.org/bpf/aeeeaf0b7ec63fdba55d4834d2f524d8bf05b71b.camel@intel.com/ Suggested-by: Rick Edgecombe Signed-off-by: Song Liu --- kernel/bpf/core.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c index 805c2ad5c793..d1f32ac354d3 100644 --- a/kernel/bpf/core.c +++ b/kernel/bpf/core.c @@ -860,7 +860,7 @@ static size_t select_bpf_prog_pack_size(void) void *ptr; size = BPF_HPAGE_SIZE * num_online_nodes(); - ptr = module_alloc(size); + ptr = module_alloc_huge(size); /* Test whether we can get huge pages. If not just use PAGE_SIZE * packs. @@ -884,7 +884,7 @@ static struct bpf_prog_pack *alloc_new_pack(bpf_jit_fill_hole_t bpf_fill_ill_ins GFP_KERNEL); if (!pack) return NULL; - pack->ptr = module_alloc(bpf_prog_pack_size); + pack->ptr = module_alloc_huge(bpf_prog_pack_size); if (!pack->ptr) { kfree(pack); return NULL; @@ -893,7 +893,6 @@ static struct bpf_prog_pack *alloc_new_pack(bpf_jit_fill_hole_t bpf_fill_ill_ins bitmap_zero(pack->bitmap, bpf_prog_pack_size / BPF_PROG_CHUNK_SIZE); list_add_tail(&pack->list, &pack_list); - set_vm_flush_reset_perms(pack->ptr); set_memory_ro((unsigned long)pack->ptr, bpf_prog_pack_size / PAGE_SIZE); set_memory_x((unsigned long)pack->ptr, bpf_prog_pack_size / PAGE_SIZE); return pack; @@ -912,10 +911,9 @@ static void *bpf_prog_pack_alloc(u32 size, bpf_jit_fill_hole_t bpf_fill_ill_insn if (size > bpf_prog_pack_size) { size = round_up(size, PAGE_SIZE); - ptr = module_alloc(size); + ptr = module_alloc_huge(size); if (ptr) { bpf_fill_ill_insns(ptr, size); - set_vm_flush_reset_perms(ptr); set_memory_ro((unsigned long)ptr, size / PAGE_SIZE); set_memory_x((unsigned long)ptr, size / PAGE_SIZE); } @@ -952,6 +950,8 @@ static void bpf_prog_pack_free(struct bpf_binary_header *hdr) mutex_lock(&pack_mutex); if (hdr->size > bpf_prog_pack_size) { + set_memory_nx((unsigned long)hdr, hdr->size / PAGE_SIZE); + set_memory_rw((unsigned long)hdr, hdr->size / PAGE_SIZE); module_memfree(hdr); goto out; } @@ -978,6 +978,8 @@ static void bpf_prog_pack_free(struct bpf_binary_header *hdr) if (bitmap_find_next_zero_area(pack->bitmap, bpf_prog_chunk_count(), 0, bpf_prog_chunk_count(), 0) == 0) { list_del(&pack->list); + set_memory_nx((unsigned long)pack->ptr, bpf_prog_pack_size / PAGE_SIZE); + set_memory_rw((unsigned long)pack->ptr, bpf_prog_pack_size / PAGE_SIZE); module_memfree(pack->ptr); kfree(pack); } From patchwork Thu Jul 7 22:35:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Song Liu X-Patchwork-Id: 12910356 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CE3A0C433EF for ; Thu, 7 Jul 2022 22:36:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236988AbiGGWga convert rfc822-to-8bit (ORCPT ); Thu, 7 Jul 2022 18:36:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35622 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236824AbiGGWg3 (ORCPT ); Thu, 7 Jul 2022 18:36:29 -0400 Received: from mx0b-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B1285DFB for ; Thu, 7 Jul 2022 15:36:28 -0700 (PDT) Received: from pps.filterd (m0109332.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 267KPhSC030988 for ; Thu, 7 Jul 2022 15:36:28 -0700 Received: from maileast.thefacebook.com ([163.114.130.16]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3h5nw2f4te-8 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Thu, 07 Jul 2022 15:36:27 -0700 Received: from twshared0725.22.frc3.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:82::f) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Thu, 7 Jul 2022 15:36:24 -0700 Received: by devbig932.frc1.facebook.com (Postfix, from userid 4523) id 8C1649D349C2; Thu, 7 Jul 2022 15:36:06 -0700 (PDT) From: Song Liu To: , , CC: , , , , , , Song Liu Subject: [PATCH v6 bpf-next 3/5] vmalloc: WARN for set_vm_flush_reset_perms() on huge pages Date: Thu, 7 Jul 2022 15:35:44 -0700 Message-ID: <20220707223546.4124919-4-song@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220707223546.4124919-1-song@kernel.org> References: <20220707223546.4124919-1-song@kernel.org> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: f9-7NUeE_fDNVTHmDHb3pOCgR_2Q63Vf X-Proofpoint-ORIG-GUID: f9-7NUeE_fDNVTHmDHb3pOCgR_2Q63Vf X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-07-07_17,2022-06-28_01,2022-06-22_01 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net VM_FLUSH_RESET_PERMS is not yet ready for huge pages, add a WARN to catch misuse soon. Suggested-by: Rick Edgecombe Signed-off-by: Song Liu --- include/linux/vmalloc.h | 1 + 1 file changed, 1 insertion(+) diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h index 096d48aa3437..59d3e1f3e108 100644 --- a/include/linux/vmalloc.h +++ b/include/linux/vmalloc.h @@ -239,6 +239,7 @@ static inline void set_vm_flush_reset_perms(void *addr) { struct vm_struct *vm = find_vm_area(addr); + WARN_ON_ONCE(is_vm_area_hugepages(addr)); if (vm) vm->flags |= VM_FLUSH_RESET_PERMS; } From patchwork Thu Jul 7 22:35:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Song Liu X-Patchwork-Id: 12910357 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8C901C43334 for ; Thu, 7 Jul 2022 22:36:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236798AbiGGWgd convert rfc822-to-8bit (ORCPT ); Thu, 7 Jul 2022 18:36:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35670 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236824AbiGGWgb (ORCPT ); Thu, 7 Jul 2022 18:36:31 -0400 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D2BCCDF5 for ; Thu, 7 Jul 2022 15:36:30 -0700 (PDT) Received: from pps.filterd (m0044010.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 267KPkcW013753 for ; Thu, 7 Jul 2022 15:36:30 -0700 Received: from maileast.thefacebook.com ([163.114.130.16]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3h4uaqsjpa-11 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Thu, 07 Jul 2022 15:36:30 -0700 Received: from twshared34609.14.frc2.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:83::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Thu, 7 Jul 2022 15:36:28 -0700 Received: by devbig932.frc1.facebook.com (Postfix, from userid 4523) id 40CF59D349C4; Thu, 7 Jul 2022 15:36:08 -0700 (PDT) From: Song Liu To: , , CC: , , , , , , Song Liu Subject: [PATCH v6 bpf-next 4/5] vmalloc: introduce huge_vmalloc_supported Date: Thu, 7 Jul 2022 15:35:45 -0700 Message-ID: <20220707223546.4124919-5-song@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220707223546.4124919-1-song@kernel.org> References: <20220707223546.4124919-1-song@kernel.org> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: 7Yy5BknbRAkDQeTYQZ6ZFoYzbQDQHY6X X-Proofpoint-ORIG-GUID: 7Yy5BknbRAkDQeTYQZ6ZFoYzbQDQHY6X X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-07-07_17,2022-06-28_01,2022-06-22_01 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net huge_vmalloc_supported() exposes vmap_allow_huge so that users of vmalloc APIs could know whether vmalloc will return huge pages. Suggested-by: Rick Edgecombe Signed-off-by: Song Liu --- include/linux/vmalloc.h | 6 ++++++ mm/vmalloc.c | 5 +++++ 2 files changed, 11 insertions(+) diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h index 59d3e1f3e108..aa2182959fc5 100644 --- a/include/linux/vmalloc.h +++ b/include/linux/vmalloc.h @@ -243,11 +243,17 @@ static inline void set_vm_flush_reset_perms(void *addr) if (vm) vm->flags |= VM_FLUSH_RESET_PERMS; } +bool huge_vmalloc_supported(void); #else static inline void set_vm_flush_reset_perms(void *addr) { } + +static inline bool huge_vmalloc_supported(void) +{ + return false; +} #endif /* for /proc/kcore */ diff --git a/mm/vmalloc.c b/mm/vmalloc.c index effd1ff6a4b4..0a5add4b5b2d 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -72,6 +72,11 @@ early_param("nohugevmalloc", set_nohugevmalloc); static const bool vmap_allow_huge = false; #endif /* CONFIG_HAVE_ARCH_HUGE_VMALLOC */ +bool huge_vmalloc_supported(void) +{ + return vmap_allow_huge; +} + bool is_vmalloc_addr(const void *x) { unsigned long addr = (unsigned long)kasan_reset_tag(x); From patchwork Thu Jul 7 22:35:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Song Liu X-Patchwork-Id: 12910358 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F0DD9CCA480 for ; Thu, 7 Jul 2022 22:36:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237003AbiGGWge convert rfc822-to-8bit (ORCPT ); Thu, 7 Jul 2022 18:36:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35696 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237008AbiGGWgc (ORCPT ); Thu, 7 Jul 2022 18:36:32 -0400 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C102AE8B for ; Thu, 7 Jul 2022 15:36:31 -0700 (PDT) Received: from pps.filterd (m0044012.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 267KPxCC000753 for ; Thu, 7 Jul 2022 15:36:31 -0700 Received: from maileast.thefacebook.com ([163.114.130.16]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3h5kgpqxk0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Thu, 07 Jul 2022 15:36:31 -0700 Received: from twshared0725.22.frc3.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:82::c) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Thu, 7 Jul 2022 15:36:30 -0700 Received: by devbig932.frc1.facebook.com (Postfix, from userid 4523) id 4AFEB9D349C5; Thu, 7 Jul 2022 15:36:10 -0700 (PDT) From: Song Liu To: , , CC: , , , , , , Song Liu Subject: [PATCH v6 bpf-next 5/5] bpf: simplify select_bpf_prog_pack_size Date: Thu, 7 Jul 2022 15:35:46 -0700 Message-ID: <20220707223546.4124919-6-song@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220707223546.4124919-1-song@kernel.org> References: <20220707223546.4124919-1-song@kernel.org> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-ORIG-GUID: e2DinLnuuuKmS9JxtjresLyXPCrI2FCf X-Proofpoint-GUID: e2DinLnuuuKmS9JxtjresLyXPCrI2FCf X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-07-07_17,2022-06-28_01,2022-06-22_01 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Use huge_vmalloc_supported to simplify select_bpf_prog_pack_size, so that we don't allocate some huge pages and free them immediately. Suggested-by: Rick Edgecombe Signed-off-by: Song Liu --- kernel/bpf/core.c | 15 ++++----------- 1 file changed, 4 insertions(+), 11 deletions(-) diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c index d1f32ac354d3..e1f8d36fb95c 100644 --- a/kernel/bpf/core.c +++ b/kernel/bpf/core.c @@ -857,22 +857,15 @@ static LIST_HEAD(pack_list); static size_t select_bpf_prog_pack_size(void) { size_t size; - void *ptr; - - size = BPF_HPAGE_SIZE * num_online_nodes(); - ptr = module_alloc_huge(size); - /* Test whether we can get huge pages. If not just use PAGE_SIZE - * packs. - */ - if (!ptr || !is_vm_area_hugepages(ptr)) { + if (huge_vmalloc_supported()) { + size = BPF_HPAGE_SIZE * num_online_nodes(); + bpf_prog_pack_mask = BPF_HPAGE_MASK; + } else { size = PAGE_SIZE; bpf_prog_pack_mask = PAGE_MASK; - } else { - bpf_prog_pack_mask = BPF_HPAGE_MASK; } - vfree(ptr); return size; }