From patchwork Fri May 20 23:57:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Song Liu X-Patchwork-Id: 12857581 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3DFC5C4332F for ; Sat, 21 May 2022 00:00:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CB5396B0075; Fri, 20 May 2022 20:00:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C3C9E6B0078; Fri, 20 May 2022 20:00:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A67906B007D; Fri, 20 May 2022 20:00:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 91EAC6B0075 for ; Fri, 20 May 2022 20:00:39 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay11.hostedemail.com (Postfix) with ESMTP id 7436780175 for ; Sat, 21 May 2022 00:00:39 +0000 (UTC) X-FDA: 79487793798.24.824D22C Received: from mx0a-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) by imf23.hostedemail.com (Postfix) with ESMTP id 2090D1400CB for ; Sat, 21 May 2022 00:00:19 +0000 (UTC) Received: from pps.filterd (m0001303.ppops.net [127.0.0.1]) by m0001303.ppops.net (8.17.1.5/8.17.1.5) with ESMTP id 24KMsAmD021874 for ; Fri, 20 May 2022 17:00:38 -0700 Received: from mail.thefacebook.com ([163.114.132.120]) by m0001303.ppops.net (PPS) with ESMTPS id 3g5wkrg2ru-13 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Fri, 20 May 2022 17:00:38 -0700 Received: from twshared8307.18.frc3.facebook.com (2620:10d:c085:208::11) by mail.thefacebook.com (2620:10d:c085:21d::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Fri, 20 May 2022 17:00:34 -0700 Received: by devbig932.frc1.facebook.com (Postfix, from userid 4523) id 1FA317E2217E; Fri, 20 May 2022 16:58:07 -0700 (PDT) From: Song Liu To: , , CC: , , , , , , , Song Liu Subject: [PATCH v4 bpf-next 0/8] bpf_prog_pack followup Date: Fri, 20 May 2022 16:57:50 -0700 Message-ID: <20220520235758.1858153-1-song@kernel.org> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: VXvfaK-78qh-FgSJtpg7Lhg9Rfh0oA9F X-Proofpoint-ORIG-GUID: VXvfaK-78qh-FgSJtpg7Lhg9Rfh0oA9F X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.874,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-05-20_08,2022-05-20_02,2022-02-23_01 Authentication-Results: imf23.hostedemail.com; dkim=none; spf=none (imf23.hostedemail.com: domain of "prvs=6140678c23=songliubraving@fb.com" has no SPF policy when checking 67.231.153.30) smtp.mailfrom="prvs=6140678c23=songliubraving@fb.com"; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=kernel.org (policy=none) X-Rspam-User: X-Stat-Signature: ejbic3nz31xmjhnbsghjoun64u6fymki X-Rspamd-Queue-Id: 2090D1400CB X-Rspamd-Server: rspam01 X-HE-Tag: 1653091219-641490 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Changes v3 => v4: 1. Shorten CC list on 4/8, so it is not dropped by the mail list. Changes v2 => v3: 1. Fix issues reported by kernel test robot . Changes v1 => v2: 1. Add WARN to set_vm_flush_reset_perms() on huge pages. (Rick Edgecombe) 2. Simplify select_bpf_prog_pack_size. (Rick Edgecombe) As of 5.18-rc6, x86_64 uses bpf_prog_pack on 4kB pages. This set contains a few followups: 1/8 - 3/8 fills unused part of bpf_prog_pack with illegal instructions. 4/8 - 5/8 enables bpf_prog_pack on 2MB pages. The primary goal of bpf_prog_pack is to reduce iTLB miss rate and reduce direct memory mapping fragmentation. This leads to non-trivial performance improvements. For our web service production benchmark, bpf_prog_pack on 4kB pages gives 0.5% to 0.7% more throughput than not using bpf_prog_pack. bpf_prog_pack on 2MB pages 0.6% to 0.9% more throughput than not using bpf_prog_pack. Note that 0.5% is a huge improvement for our fleet. I believe this is also significant for other companies with many thousand servers. bpf_prog_pack on 2MB pages may use slightly more memory for systems without many BPF programs. However, such waste in memory (<2MB) is within noisy for modern x86_64 systems. Song Liu (8): bpf: fill new bpf_prog_pack with illegal instructions x86/alternative: introduce text_poke_set bpf: introduce bpf_arch_text_invalidate for bpf_prog_pack module: introduce module_alloc_huge bpf: use module_alloc_huge for bpf_prog_pack vmalloc: WARN for set_vm_flush_reset_perms() on huge pages vmalloc: introduce huge_vmalloc_supported bpf: simplify select_bpf_prog_pack_size arch/x86/include/asm/text-patching.h | 1 + arch/x86/kernel/alternative.c | 67 +++++++++++++++++++++++----- arch/x86/kernel/module.c | 21 +++++++++ arch/x86/net/bpf_jit_comp.c | 5 +++ include/linux/bpf.h | 1 + include/linux/moduleloader.h | 5 +++ include/linux/vmalloc.h | 7 +++ kernel/bpf/core.c | 43 ++++++++++-------- kernel/module.c | 8 ++++ mm/vmalloc.c | 5 +++ 10 files changed, 134 insertions(+), 29 deletions(-) --- 2.30.2