From patchwork Tue Mar 8 12:20:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marco Elver X-Patchwork-Id: 12773608 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 34F65C433EF for ; Tue, 8 Mar 2022 12:20:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B722C8D0003; Tue, 8 Mar 2022 07:20:30 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B213A8D0001; Tue, 8 Mar 2022 07:20:30 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A0FF38D0003; Tue, 8 Mar 2022 07:20:30 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0158.hostedemail.com [216.40.44.158]) by kanga.kvack.org (Postfix) with ESMTP id 9274A8D0001 for ; Tue, 8 Mar 2022 07:20:30 -0500 (EST) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 5353FAA078 for ; Tue, 8 Mar 2022 12:20:30 +0000 (UTC) X-FDA: 79221127020.26.615BFF1 Received: from mail-ej1-f73.google.com (mail-ej1-f73.google.com [209.85.218.73]) by imf18.hostedemail.com (Postfix) with ESMTP id D66211C0005 for ; Tue, 8 Mar 2022 12:20:29 +0000 (UTC) Received: by mail-ej1-f73.google.com with SMTP id i14-20020a17090639ce00b006dabe6a112fso4835188eje.13 for ; Tue, 08 Mar 2022 04:20:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:message-id:mime-version:subject:from:to:cc; bh=kl2sXhR1ZPN1zpyjxLJOZF6G4P4aj2NkEcWtAzkOw34=; b=e8o5OZ/PjBLUrn4XXwdWmsvAptgAay2fZ8FGw0JI2jB/j4+ttCbC9bXWl62EQm2oyP 9yCMJ4m5v3HeKN1Fo2CZY8vRNiH4fXUB1Go3eiGEWu5jigu1DRlDDhFbn135L4oF0AR8 v4mrb7nZ5rA/cZAJP1NhYaLVva3DJ5hIt2MC7CjnpV0GVhcYkjRq8fa5xhKekix+DVcI TfbogjWkfWGm5At5NnaXkm7JweN+BjlL47iGT8+MowaKQuEdMKh9G87pAFcFkabYlhPN ksyVUEz+rWPBxg2bty70dnJOxXuo57PdCaYSZgzKt63SxCeVDcFY82+zM3UMmtSjy2/t jsvg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:message-id:mime-version:subject:from:to:cc; bh=kl2sXhR1ZPN1zpyjxLJOZF6G4P4aj2NkEcWtAzkOw34=; b=w68pHu+X3y5yY7hqnYhx3m+3jN1PFyXe1/tCEVi+DrC3qLUZaxPSYlwr/eGiUSq2o7 zAk6s2ZWstQ4Fp1rIB7LedF0bjUUqduYSYlbX1ZaYOulVd9F6k/To8NY6nzdUpvPQI81 QW1VBKPKAXgVD/VpGEhQHhlzAr3uUva+nQimX4Oii+ByFG0u2KvlEvO5u1Ew1rgDZvg9 SLm+0fUq1prURyG6eH4gDssA3OTO741cbm16MyBeikZLmP4a30dLhG2CB2JYjfEmgMUy Og5GPys3EL3lTpkaC/jHF138gPZjuEcgaM7h7eri7pOHyIDheW23b6oUZMnrxPKB9eeu 8Tng== X-Gm-Message-State: AOAM533odFdcg2C/vsa0yfgrG0dq0kNhYNqKRD52WLYQjDJzKzRXtTqM tDzSCScfHADB71BL6xoIzLfhmEr5Ug== X-Google-Smtp-Source: ABdhPJxfkP2/aCUWzQbI6nd5lClz2NRqyNjoTupTqovem5C6WFkLWF7MoURPx1fFC6GsUpAmk1n+lQpAEw== X-Received: from elver.muc.corp.google.com ([2a00:79e0:15:13:edd7:29ac:9b6f:1abd]) (user=elver job=sendgmr) by 2002:a05:6402:50c8:b0:416:2ae0:4bb3 with SMTP id h8-20020a05640250c800b004162ae04bb3mr14109102edb.132.1646742028577; Tue, 08 Mar 2022 04:20:28 -0800 (PST) Date: Tue, 8 Mar 2022 13:20:23 +0100 Message-Id: <20220308122023.3068150-1-elver@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.35.1.616.g0bdcbb4464-goog Subject: [PATCH] kfence: allow use of a deferrable timer From: Marco Elver To: elver@google.com, Andrew Morton Cc: Alexander Potapenko , Dmitry Vyukov , kasan-dev@googlegroups.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org X-Rspamd-Queue-Id: D66211C0005 X-Stat-Signature: uiua8h3tqmxjshijr19yt18bwbdsozyd X-Rspam-User: Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b="e8o5OZ/P"; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf18.hostedemail.com: domain of 3DEonYgUKCFU18I1E3BB381.zB985AHK-997Ixz7.BE3@flex--elver.bounces.google.com designates 209.85.218.73 as permitted sender) smtp.mailfrom=3DEonYgUKCFU18I1E3BB381.zB985AHK-997Ixz7.BE3@flex--elver.bounces.google.com X-Rspamd-Server: rspam03 X-HE-Tag: 1646742029-232555 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Allow the use of a deferrable timer, which does not force CPU wake-ups when the system is idle. A consequence is that the sample interval becomes very unpredictable, to the point that it is not guaranteed that the KFENCE KUnit test still passes. Nevertheless, on power-constrained systems this may be preferable, so let's give the user the option should they accept the above trade-off. Signed-off-by: Marco Elver Reviewed-by: Alexander Potapenko --- lib/Kconfig.kfence | 12 ++++++++++++ mm/kfence/core.c | 15 +++++++++++++-- 2 files changed, 25 insertions(+), 2 deletions(-) diff --git a/lib/Kconfig.kfence b/lib/Kconfig.kfence index 912f252a41fc..1cf2ea2a1ac7 100644 --- a/lib/Kconfig.kfence +++ b/lib/Kconfig.kfence @@ -45,6 +45,18 @@ config KFENCE_NUM_OBJECTS pages are required; with one containing the object and two adjacent ones used as guard pages. +config KFENCE_DEFERRABLE + bool "Use a deferrable timer to trigger allocations" if EXPERT + help + Use a deferrable timer to trigger allocations. This avoids forcing + CPU wake-ups if the system is idle, at the risk of a less predictable + sample interval. + + Warning: The KUnit test suite fails with this option enabled - due to + the unpredictability of the sample interval! + + Say N if you are unsure. + config KFENCE_STATIC_KEYS bool "Use static keys to set up allocations" if EXPERT depends on JUMP_LABEL diff --git a/mm/kfence/core.c b/mm/kfence/core.c index f126b53b9b85..451277b41bfb 100644 --- a/mm/kfence/core.c +++ b/mm/kfence/core.c @@ -95,6 +95,10 @@ module_param_cb(sample_interval, &sample_interval_param_ops, &kfence_sample_inte static unsigned long kfence_skip_covered_thresh __read_mostly = 75; module_param_named(skip_covered_thresh, kfence_skip_covered_thresh, ulong, 0644); +/* If true, use a deferrable timer at the risk of unpredictable sample intervals. */ +static bool kfence_deferrable __read_mostly = IS_ENABLED(CONFIG_KFENCE_DEFERRABLE); +module_param_named(deferrable, kfence_deferrable, bool, 0444); + /* The pool of pages used for guard pages and objects. */ char *__kfence_pool __read_mostly; EXPORT_SYMBOL(__kfence_pool); /* Export for test modules. */ @@ -740,6 +744,8 @@ late_initcall(kfence_debugfs_init); /* === Allocation Gate Timer ================================================ */ +static struct delayed_work kfence_timer; + #ifdef CONFIG_KFENCE_STATIC_KEYS /* Wait queue to wake up allocation-gate timer task. */ static DECLARE_WAIT_QUEUE_HEAD(allocation_wait); @@ -762,7 +768,6 @@ static DEFINE_IRQ_WORK(wake_up_kfence_timer_work, wake_up_kfence_timer); * avoids IPIs, at the cost of not immediately capturing allocations if the * instructions remain cached. */ -static struct delayed_work kfence_timer; static void toggle_allocation_gate(struct work_struct *work) { if (!READ_ONCE(kfence_enabled)) @@ -790,7 +795,6 @@ static void toggle_allocation_gate(struct work_struct *work) queue_delayed_work(system_unbound_wq, &kfence_timer, msecs_to_jiffies(kfence_sample_interval)); } -static DECLARE_DELAYED_WORK(kfence_timer, toggle_allocation_gate); /* === Public interface ===================================================== */ @@ -809,8 +813,15 @@ static void kfence_init_enable(void) { if (!IS_ENABLED(CONFIG_KFENCE_STATIC_KEYS)) static_branch_enable(&kfence_allocation_key); + + if (kfence_deferrable) + INIT_DEFERRABLE_WORK(&kfence_timer, toggle_allocation_gate); + else + INIT_DELAYED_WORK(&kfence_timer, toggle_allocation_gate); + WRITE_ONCE(kfence_enabled, true); queue_delayed_work(system_unbound_wq, &kfence_timer, 0); + pr_info("initialized - using %lu bytes for %d objects at 0x%p-0x%p\n", KFENCE_POOL_SIZE, CONFIG_KFENCE_NUM_OBJECTS, (void *)__kfence_pool, (void *)(__kfence_pool + KFENCE_POOL_SIZE));