From patchwork Mon May 16 05:40:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Song Liu X-Patchwork-Id: 12850265 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C007CC433EF for ; Mon, 16 May 2022 05:41:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229799AbiEPFlW convert rfc822-to-8bit (ORCPT ); Mon, 16 May 2022 01:41:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56768 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239655AbiEPFlJ (ORCPT ); Mon, 16 May 2022 01:41:09 -0400 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C8A98DF48 for ; Sun, 15 May 2022 22:41:08 -0700 (PDT) Received: from pps.filterd (m0044010.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 24FLEhGt019278 for ; Sun, 15 May 2022 22:41:08 -0700 Received: from maileast.thefacebook.com ([163.114.130.16]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3g27rnqmb0-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Sun, 15 May 2022 22:41:08 -0700 Received: from twshared13345.18.frc3.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:83::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Sun, 15 May 2022 22:41:00 -0700 Received: by devbig932.frc1.facebook.com (Postfix, from userid 4523) id 348F37AEBBF4; Sun, 15 May 2022 22:40:56 -0700 (PDT) From: Song Liu To: , CC: , , , , , , , Song Liu Subject: [PATCH bpf-next 0/5] bpf_prog_pack followup Date: Sun, 15 May 2022 22:40:46 -0700 Message-ID: <20220516054051.114490-1-song@kernel.org> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-ORIG-GUID: o3GorWvJTB8RgDvjLygtASGTgxmH5nPQ X-Proofpoint-GUID: o3GorWvJTB8RgDvjLygtASGTgxmH5nPQ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.858,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-05-15_11,2022-05-13_01,2022-02-23_01 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Resending the set, as the original ones didn't make through the maillist. As of 5.18-rc6, x86_64 uses bpf_prog_pack on 4kB pages. This set contains two followups: 1/5 - 3/5 fills unused part of bpf_prog_pack with illegal instructions. 4/5 - 5/5 enables bpf_prog_pack on 2MB pages. The primary goal of bpf_prog_pack is to reduce iTLB miss rate and reduce direct memory mapping fragmentation. This leads to non-trivial performance improvements. For our web service production benchmark, bpf_prog_pack on 4kB pages gives 0.5% to 0.7% more throughput than not using bpf_prog_pack. bpf_prog_pack on 2MB pages 0.6% to 0.9% more throughput than not using bpf_prog_pack. Note that 0.5% is a huge improvement for our fleet. I believe this is also significant for other companies with many thousand servers. bpf_prog_pack on 2MB pages may use slightly more memory for systems without many BPF programs. However, such waste in memory (<2MB) is within noisy for modern x86_64 systems. Song Liu (5): bpf: fill new bpf_prog_pack with illegal instructions x86/alternative: introduce text_poke_set bpf: introduce bpf_arch_text_invalidate for bpf_prog_pack module: introduce module_alloc_huge bpf: use module_alloc_huge for bpf_prog_pack arch/x86/include/asm/text-patching.h | 1 + arch/x86/kernel/alternative.c | 70 ++++++++++++++++++++++++---- arch/x86/kernel/module.c | 21 +++++++++ arch/x86/net/bpf_jit_comp.c | 5 ++ include/linux/bpf.h | 1 + include/linux/moduleloader.h | 5 ++ kernel/bpf/core.c | 30 ++++++++---- kernel/module.c | 8 ++++ 8 files changed, 122 insertions(+), 19 deletions(-) --- 2.30.2