From patchwork Mon Jan 24 02:52:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peng Liu X-Patchwork-Id: 12721456 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D9753C4332F for ; Mon, 24 Jan 2022 02:37:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2E7816B0087; Sun, 23 Jan 2022 21:37:48 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2BD7E6B0088; Sun, 23 Jan 2022 21:37:48 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1B0936B008A; Sun, 23 Jan 2022 21:37:48 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id F10506B0088 for ; Sun, 23 Jan 2022 21:37:47 -0500 (EST) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 9FAF18249980 for ; Mon, 24 Jan 2022 02:37:47 +0000 (UTC) X-FDA: 79063620174.05.895A9ED Received: from szxga03-in.huawei.com (szxga03-in.huawei.com [45.249.212.189]) by imf20.hostedemail.com (Postfix) with ESMTP id D7F331C0020 for ; Mon, 24 Jan 2022 02:37:46 +0000 (UTC) Received: from kwepemi500013.china.huawei.com (unknown [172.30.72.54]) by szxga03-in.huawei.com (SkyGuard) with ESMTP id 4JhvG070gtz8wTb; Mon, 24 Jan 2022 10:34:48 +0800 (CST) Received: from kwepemm600017.china.huawei.com (7.193.23.234) by kwepemi500013.china.huawei.com (7.221.188.120) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.21; Mon, 24 Jan 2022 10:37:43 +0800 Received: from localhost.localdomain (10.175.112.125) by kwepemm600017.china.huawei.com (7.193.23.234) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.21; Mon, 24 Jan 2022 10:37:42 +0800 From: Peng Liu To: , , , , , , CC: , , , , , Subject: [PATCH RFC 2/3] kfence: Optimize branches prediction when sample interval is zero Date: Mon, 24 Jan 2022 02:52:04 +0000 Message-ID: <20220124025205.329752-3-liupeng256@huawei.com> X-Mailer: git-send-email 2.18.0.huawei.25 In-Reply-To: <20220124025205.329752-1-liupeng256@huawei.com> References: <20220124025205.329752-1-liupeng256@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To kwepemm600017.china.huawei.com (7.193.23.234) X-CFilter-Loop: Reflected X-Stat-Signature: tby89k8y5ytpfn8zjznpx3rjhgs61fsk Authentication-Results: imf20.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf20.hostedemail.com: domain of liupeng256@huawei.com designates 45.249.212.189 as permitted sender) smtp.mailfrom=liupeng256@huawei.com X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: D7F331C0020 X-HE-Tag: 1642991866-238645 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In order to release a uniform kernel with KFENCE, it is good to compile it with CONFIG_KFENCE_SAMPLE_INTERVAL = 0. For a group of produtions who don't want to use KFENCE, they can use kernel just as original vesion without KFENCE. For KFENCE users, they can open it by setting the kernel boot parameter kfence.sample_interval. Hence, set KFENCE sample interval default to zero is convenient. The current KFENCE is supportted to adjust sample interval via the kernel boot parameter. However, branches prediction in kfence_alloc is not good for situation with CONFIG_KFENCE_SAMPLE_INTERVAL = 0 and boot parameter kfence.sample_interval != 0, which is because the current kfence_alloc is likely to return NULL when CONFIG_KFENCE_SAMPLE_INTERVAL = 0. To optimize branches prediction in this situation, kfence_enabled will check firstly. Signed-off-by: Peng Liu --- include/linux/kfence.h | 5 ++++- mm/kfence/core.c | 2 +- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/include/linux/kfence.h b/include/linux/kfence.h index aec4f6b247b5..bf91b76b87ee 100644 --- a/include/linux/kfence.h +++ b/include/linux/kfence.h @@ -17,6 +17,7 @@ #include #include +extern bool kfence_enabled; extern unsigned long kfence_num_objects; /* * We allocate an even number of pages, as it simplifies calculations to map @@ -115,7 +116,9 @@ void *__kfence_alloc(struct kmem_cache *s, size_t size, gfp_t flags); */ static __always_inline void *kfence_alloc(struct kmem_cache *s, size_t size, gfp_t flags) { -#if defined(CONFIG_KFENCE_STATIC_KEYS) || CONFIG_KFENCE_SAMPLE_INTERVAL == 0 + if (!kfence_enabled) + return NULL; +#if defined(CONFIG_KFENCE_STATIC_KEYS) if (!static_branch_unlikely(&kfence_allocation_key)) return NULL; #else diff --git a/mm/kfence/core.c b/mm/kfence/core.c index 4655bcc0306e..2301923182b8 100644 --- a/mm/kfence/core.c +++ b/mm/kfence/core.c @@ -48,7 +48,7 @@ /* === Data ================================================================= */ -static bool kfence_enabled __read_mostly; +bool kfence_enabled __read_mostly; static unsigned long kfence_sample_interval __read_mostly = CONFIG_KFENCE_SAMPLE_INTERVAL;