From patchwork Mon Apr 19 08:50:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marco Elver X-Patchwork-Id: 12211215 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 42C42C433B4 for ; Mon, 19 Apr 2021 08:50:59 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C08F761107 for ; Mon, 19 Apr 2021 08:50:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C08F761107 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 47CD46B006E; Mon, 19 Apr 2021 04:50:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4061C6B0070; Mon, 19 Apr 2021 04:50:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2341B6B0071; Mon, 19 Apr 2021 04:50:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0199.hostedemail.com [216.40.44.199]) by kanga.kvack.org (Postfix) with ESMTP id 02B606B006E for ; Mon, 19 Apr 2021 04:50:57 -0400 (EDT) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id B73108249980 for ; Mon, 19 Apr 2021 08:50:57 +0000 (UTC) X-FDA: 78048496554.24.590C771 Received: from mail-wr1-f74.google.com (mail-wr1-f74.google.com [209.85.221.74]) by imf01.hostedemail.com (Postfix) with ESMTP id 5F6A9500152E for ; Mon, 19 Apr 2021 08:50:56 +0000 (UTC) Received: by mail-wr1-f74.google.com with SMTP id 93-20020adf80e60000b0290106fab45006so4162045wrl.20 for ; Mon, 19 Apr 2021 01:50:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=vYl7G57j5M+nH2KzFf9qyrbxlx0FOF9ni4b2Z/4AoS0=; b=j0gtczwPaI+cs4X0SI5bpT9FOnjc+W1uVLyPs+aAetSjO6f/HrBMPoDZh+5dwauc8V kTvpy2Rw30R4hN4dZp79bF54547BHLL5vfc8Ddi8dgyAMd32lQuK1DXC7o24cIxc4rD0 wJN7MOgPITZgUKAYi/eqGlBnLoS5ODMaseG7ew4M80eB7J0Et6VGG68cXC8dNLkyf7+1 mlHjE1wEFTENGwOCoLTjzRawVMw8mhMf7wfJPKh3oxjoQ8dsdW2DU9SC6EIeNsBcrSWE CuCA0uD5l6ccw2Iu0dGpmG88BxIntliqMSt4CRRJhmr0HEjtEl+pP4LfAJ6EpPmjps4B GiiA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=vYl7G57j5M+nH2KzFf9qyrbxlx0FOF9ni4b2Z/4AoS0=; b=E7s90/0GKFu8vQLuUdKoi+4QY6ZQe3jDjG+CBTCoxsP3Nfv6C4032g3/i9sYAW6E2+ 6zJXO/HpcVPN5t6NwFmJs8OxpMm1mZJnSVzbC6zMXDV72DDxNbJJ/Aw4/zuNfYLlEgA3 BuPsrmVrwSwV1j/yCRmSq8nB/CAxmQrZL0hcsS0FTMc6Fw0fVyv83cH+trD3L1FQnDFO 7nvr7prPdTigZyZTUFCbW7ms5yVHBrBBwdd47QRs7Kt3/mdlJ/IKqHJtD2PJ1CcCRCqb hR/i1XvM3tcmHudS+MFGRTTtf57Kq9uMHzRH9zk2QCB8uc0H2a3UgE6Oau3DazYi/UzV n0wg== X-Gm-Message-State: AOAM530Ap/4wbxz//bLRS48z9S+YmQmHFzAu2/MeUDdu6FVVNGDRZR30 Wb5e4YPbZO9oJfVXSP1RxdPf94/zTQ== X-Google-Smtp-Source: ABdhPJzkMlFib2sTj44vDpZc4hhB+ZlXfwBn2wMs7xBakSpQW9OwsdQEzis16G7Q5uQB4QEPzu3OjKS06Q== X-Received: from elver.muc.corp.google.com ([2a00:79e0:15:13:92f8:c03b:1448:ada5]) (user=elver job=sendgmr) by 2002:a05:600c:48a6:: with SMTP id j38mr20532545wmp.99.1618822256477; Mon, 19 Apr 2021 01:50:56 -0700 (PDT) Date: Mon, 19 Apr 2021 10:50:25 +0200 In-Reply-To: <20210419085027.761150-1-elver@google.com> Message-Id: <20210419085027.761150-2-elver@google.com> Mime-Version: 1.0 References: <20210419085027.761150-1-elver@google.com> X-Mailer: git-send-email 2.31.1.368.gbe11c130af-goog Subject: [PATCH 1/3] kfence: await for allocation using wait_event From: Marco Elver To: elver@google.com, akpm@linux-foundation.org Cc: glider@google.com, dvyukov@google.com, jannh@google.com, mark.rutland@arm.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, kasan-dev@googlegroups.com X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 5F6A9500152E X-Stat-Signature: wgfa1pc9g5x7t1zarc57ras8gxqa35oq Received-SPF: none (flex--elver.bounces.google.com>: No applicable sender policy available) receiver=imf01; identity=mailfrom; envelope-from="<3cER9YAUKCPcdkudqfnnfkd.bnlkhmtw-lljuZbj.nqf@flex--elver.bounces.google.com>"; helo=mail-wr1-f74.google.com; client-ip=209.85.221.74 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1618822256-107478 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On mostly-idle systems, we have observed that toggle_allocation_gate() is a cause of frequent wake-ups, preventing an otherwise idle CPU to go into a lower power state. A late change in KFENCE's development, due to a potential deadlock [1], required changing the scheduling-friendly wait_event_timeout() and wake_up() to an open-coded wait-loop using schedule_timeout(). [1] https://lkml.kernel.org/r/000000000000c0645805b7f982e4@google.com To avoid unnecessary wake-ups, switch to using wait_event_timeout(). Unfortunately, we still cannot use a version with direct wake_up() in __kfence_alloc() due to the same potential for deadlock as in [1]. Instead, add a level of indirection via an irq_work that is scheduled if we determine that the kfence_timer requires a wake_up(). Fixes: 0ce20dd84089 ("mm: add Kernel Electric-Fence infrastructure") Signed-off-by: Marco Elver --- lib/Kconfig.kfence | 1 + mm/kfence/core.c | 58 +++++++++++++++++++++++++++++++++------------- 2 files changed, 43 insertions(+), 16 deletions(-) diff --git a/lib/Kconfig.kfence b/lib/Kconfig.kfence index 78f50ccb3b45..e641add33947 100644 --- a/lib/Kconfig.kfence +++ b/lib/Kconfig.kfence @@ -7,6 +7,7 @@ menuconfig KFENCE bool "KFENCE: low-overhead sampling-based memory safety error detector" depends on HAVE_ARCH_KFENCE && (SLAB || SLUB) select STACKTRACE + select IRQ_WORK help KFENCE is a low-overhead sampling-based detector of heap out-of-bounds access, use-after-free, and invalid-free errors. KFENCE is designed diff --git a/mm/kfence/core.c b/mm/kfence/core.c index 768dbd58170d..5f0a56041549 100644 --- a/mm/kfence/core.c +++ b/mm/kfence/core.c @@ -10,6 +10,7 @@ #include #include #include +#include #include #include #include @@ -587,6 +588,20 @@ late_initcall(kfence_debugfs_init); /* === Allocation Gate Timer ================================================ */ +#ifdef CONFIG_KFENCE_STATIC_KEYS +/* Wait queue to wake up allocation-gate timer task. */ +static DECLARE_WAIT_QUEUE_HEAD(allocation_wait); + +static void wake_up_kfence_timer(struct irq_work *work) +{ + wake_up(&allocation_wait); +} +static DEFINE_IRQ_WORK(wake_up_kfence_timer_work, wake_up_kfence_timer); + +/* Indicate if timer task is waiting, to avoid unnecessary irq_work. */ +static bool kfence_timer_waiting; +#endif + /* * Set up delayed work, which will enable and disable the static key. We need to * use a work queue (rather than a simple timer), since enabling and disabling a @@ -604,25 +619,16 @@ static void toggle_allocation_gate(struct work_struct *work) if (!READ_ONCE(kfence_enabled)) return; - /* Enable static key, and await allocation to happen. */ atomic_set(&kfence_allocation_gate, 0); #ifdef CONFIG_KFENCE_STATIC_KEYS + /* Enable static key, and await allocation to happen. */ static_branch_enable(&kfence_allocation_key); - /* - * Await an allocation. Timeout after 1 second, in case the kernel stops - * doing allocations, to avoid stalling this worker task for too long. - */ - { - unsigned long end_wait = jiffies + HZ; - - do { - set_current_state(TASK_UNINTERRUPTIBLE); - if (atomic_read(&kfence_allocation_gate) != 0) - break; - schedule_timeout(1); - } while (time_before(jiffies, end_wait)); - __set_current_state(TASK_RUNNING); - } + + WRITE_ONCE(kfence_timer_waiting, true); + smp_mb(); /* See comment in __kfence_alloc(). */ + wait_event_timeout(allocation_wait, atomic_read(&kfence_allocation_gate), HZ); + smp_store_release(&kfence_timer_waiting, false); /* Order after wait_event(). */ + /* Disable static key and reset timer. */ static_branch_disable(&kfence_allocation_key); #endif @@ -729,6 +735,26 @@ void *__kfence_alloc(struct kmem_cache *s, size_t size, gfp_t flags) */ if (atomic_read(&kfence_allocation_gate) || atomic_inc_return(&kfence_allocation_gate) > 1) return NULL; +#ifdef CONFIG_KFENCE_STATIC_KEYS + /* + * Read of kfence_timer_waiting must be ordered after write to + * kfence_allocation_gate (fully ordered per atomic_inc_return()). + * + * Conversely, the write to kfence_timer_waiting must be ordered before + * the check of kfence_allocation_gate in toggle_allocation_gate(). + * + * This ensures that toggle_allocation_gate() always sees the updated + * kfence_allocation_gate, or we see that the timer is waiting and will + * queue the work to wake it up. + */ + if (READ_ONCE(kfence_timer_waiting)) { + /* + * Calling wake_up() here may deadlock when allocations happen + * from within timer code. Use an irq_work to defer it. + */ + irq_work_queue(&wake_up_kfence_timer_work); + } +#endif if (!READ_ONCE(kfence_enabled)) return NULL; From patchwork Mon Apr 19 08:50:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marco Elver X-Patchwork-Id: 12211217 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 605E8C433ED for ; Mon, 19 Apr 2021 08:51:01 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 022D461107 for ; Mon, 19 Apr 2021 08:51:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 022D461107 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8C3636B0070; Mon, 19 Apr 2021 04:51:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 89B6B6B0071; Mon, 19 Apr 2021 04:51:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 714846B0072; Mon, 19 Apr 2021 04:51:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0074.hostedemail.com [216.40.44.74]) by kanga.kvack.org (Postfix) with ESMTP id 58EBE6B0070 for ; Mon, 19 Apr 2021 04:51:00 -0400 (EDT) Received: from smtpin37.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 1D609180AD830 for ; Mon, 19 Apr 2021 08:51:00 +0000 (UTC) X-FDA: 78048496680.37.ED409D4 Received: from mail-qt1-f201.google.com (mail-qt1-f201.google.com [209.85.160.201]) by imf02.hostedemail.com (Postfix) with ESMTP id AD0BA40002CF for ; Mon, 19 Apr 2021 08:50:38 +0000 (UTC) Received: by mail-qt1-f201.google.com with SMTP id z5-20020a05622a0285b02901b943be06b5so496527qtw.17 for ; Mon, 19 Apr 2021 01:50:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=1R4Jr4Y0fNVqtFmGdn7fmPgc472QE3HJBlQ1cDdhzt4=; b=TMBkauBIAfAP9XjeDFGJnDsoFke73tSPJe4f2mEkud+WWPz6Mmq/1Nmg9lVrWBd39+ wVZW66TOGiC/4Ki0pXPQjVxoAR3nGwKHALc1Ct9l34Kt0v1OvcKspHmEYuiTNhUHlDoc GdRPD0UYmpqr3hdeh3FftJau7nFiZGQrtf6jjq7py7gEI0eNQZCPn4wYZcki74Z99Jqu I2rECNJ42gZOOteD2/QXjDdO2WkkqubnFqYsY3bVuIb8W90FgIa4Pxxd5Bue+mLBI+yq JAaaAr30YToUOB+CPUGwKJ9GaqomjYg3czrf7Zu+35YAkKMEoWrKlDLTIVYnKeDKZQPK sgWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=1R4Jr4Y0fNVqtFmGdn7fmPgc472QE3HJBlQ1cDdhzt4=; b=JuRj/OLa0C3Cs0LiXWLXjVUUo+qCcG4AcAXGZnNyV6ehKJ6SBYAmQoUamC//g/mqCD G5fP9r4zBt7z5VuGJr7cFF+xt7l5uOLUE93FkQyiPWLc61SZc9LixS+FLejyFb0VG6lw oZ94au3bk+n44AT4wgB78kKKm5sROsieyWy9nwiWL3gufshd0q1Ey3k8R+/5VxgJHJ2H mk8mH0h+Kwp3w88LAZV3CJWaAm1Qovsodq3/S4LdTF81h8tmdo221wWrS7piygQRU2sC oE4qExWfTJ9WnSTj2JzOvWk+dj2aztDGzDY+DYRNg0bIhyMFY/pCrXkThbaFdi8LFHnG LsRQ== X-Gm-Message-State: AOAM532nb+/vRJTMCiTq4QasGVA2hNlSpGNeJRkZXdi40jNhEgcnUUwH 2AbYu3sgSxZvJOOcYs1hZV+yxM5PiA== X-Google-Smtp-Source: ABdhPJyfu1Y3YRhkFKwn8cRYHRc0nLt0o6kvgfbB8Wf5OVUUnfByumSzTxb1FCZL47h5AmiWMHIJhPJIaA== X-Received: from elver.muc.corp.google.com ([2a00:79e0:15:13:92f8:c03b:1448:ada5]) (user=elver job=sendgmr) by 2002:a0c:a98d:: with SMTP id a13mr4682439qvb.39.1618822258947; Mon, 19 Apr 2021 01:50:58 -0700 (PDT) Date: Mon, 19 Apr 2021 10:50:26 +0200 In-Reply-To: <20210419085027.761150-1-elver@google.com> Message-Id: <20210419085027.761150-3-elver@google.com> Mime-Version: 1.0 References: <20210419085027.761150-1-elver@google.com> X-Mailer: git-send-email 2.31.1.368.gbe11c130af-goog Subject: [PATCH 2/3] kfence: maximize allocation wait timeout duration From: Marco Elver To: elver@google.com, akpm@linux-foundation.org Cc: glider@google.com, dvyukov@google.com, jannh@google.com, mark.rutland@arm.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, kasan-dev@googlegroups.com X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: AD0BA40002CF X-Stat-Signature: on3wdgcdboieex743qohet717597ttwu Received-SPF: none (flex--elver.bounces.google.com>: No applicable sender policy available) receiver=imf02; identity=mailfrom; envelope-from="<3ckR9YAUKCPkfmwfshpphmf.dpnmjovy-nnlwbdl.psh@flex--elver.bounces.google.com>"; helo=mail-qt1-f201.google.com; client-ip=209.85.160.201 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1618822238-819152 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The allocation wait timeout was initially added because of warnings due to CONFIG_DETECT_HUNG_TASK=y [1]. While the 1 sec timeout is sufficient to resolve the warnings (given the hung task timeout must be 1 sec or larger) it may cause unnecessary wake-ups if the system is idle. [1] https://lkml.kernel.org/r/CADYN=9J0DQhizAGB0-jz4HOBBh+05kMBXb4c0cXMS7Qi5NAJiw@mail.gmail.com Fix it by computing the timeout duration in terms of the current sysctl_hung_task_timeout_secs value. Signed-off-by: Marco Elver --- mm/kfence/core.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/mm/kfence/core.c b/mm/kfence/core.c index 5f0a56041549..73e7b621fb36 100644 --- a/mm/kfence/core.c +++ b/mm/kfence/core.c @@ -20,6 +20,7 @@ #include #include #include +#include #include #include #include @@ -626,7 +627,16 @@ static void toggle_allocation_gate(struct work_struct *work) WRITE_ONCE(kfence_timer_waiting, true); smp_mb(); /* See comment in __kfence_alloc(). */ - wait_event_timeout(allocation_wait, atomic_read(&kfence_allocation_gate), HZ); + if (sysctl_hung_task_timeout_secs) { + /* + * During low activity with no allocations we might wait a + * while; let's avoid the hung task warning. + */ + wait_event_timeout(allocation_wait, atomic_read(&kfence_allocation_gate), + sysctl_hung_task_timeout_secs * HZ / 2); + } else { + wait_event(allocation_wait, atomic_read(&kfence_allocation_gate)); + } smp_store_release(&kfence_timer_waiting, false); /* Order after wait_event(). */ /* Disable static key and reset timer. */ From patchwork Mon Apr 19 08:50:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marco Elver X-Patchwork-Id: 12211219 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46A80C43460 for ; Mon, 19 Apr 2021 08:51:04 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D483261107 for ; Mon, 19 Apr 2021 08:51:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D483261107 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6C6A06B0071; Mon, 19 Apr 2021 04:51:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 69D796B0072; Mon, 19 Apr 2021 04:51:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 564E26B0073; Mon, 19 Apr 2021 04:51:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0068.hostedemail.com [216.40.44.68]) by kanga.kvack.org (Postfix) with ESMTP id 3DD436B0071 for ; Mon, 19 Apr 2021 04:51:03 -0400 (EDT) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id F07E7180AD830 for ; Mon, 19 Apr 2021 08:51:02 +0000 (UTC) X-FDA: 78048496764.25.DE82D4E Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) by imf21.hostedemail.com (Postfix) with ESMTP id 3AC8CE000108 for ; Mon, 19 Apr 2021 08:51:00 +0000 (UTC) Received: by mail-wm1-f74.google.com with SMTP id v65-20020a1cde440000b029012853a35ee7so3835716wmg.2 for ; Mon, 19 Apr 2021 01:51:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=p3g1jw5/hGDj5wABho0AViDXSgtC45K4NGeFzanUCY4=; b=i3pEmtzl9Y9yr4qP/FhB0UAxWQWx/TnwZWDJLCOZjiYThK2wDLH3szcJMq2jBHWGJx 8lFlpz7eCUTbgNuCkNQ/hQMlCZlFckio/wKwBGDbzGmVMwZfhRiNgrp6fhGIvchgW4CT XgW3GA3rjsfNWlg9Ak/qghjRMZzJTTPTZLC15L+71r2yQ3d3x1dJieJjnmrHUfKPvHNj 80c1OeguGlK61udIO/VqiVdPCwrOzA1MVB00mGpwxLMHiPLokUL6rHh3h8K54EZ+5jQV Xj4CaPbXrHqInHKQL3DbmVxUm4bBEvj4y0mZRN14HGYW1CSp8X4Tf/Z/yJ7JMQNTcAoS 9EFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=p3g1jw5/hGDj5wABho0AViDXSgtC45K4NGeFzanUCY4=; b=ZzLE78Fp8P/dcQng+0OAzNReswPW0R2kZU3CanORv4Sb57jxvfiB8pJaq6ma+XBVO8 aCfLW+GGNJKoYZ7Q6RJJmB+iPfIk1HZmN7/ByVCzP8wb05FbCNNFSa8amntksE7bN0t9 9k0oZe2sq8yawYh3dBBKrWP5XOK2WDEvsMejOKFR0noRfc+MSvVqpZzHYmxQmRm2ne7h s9a4b+WUoGFd0Phwz/LWEgK3UnKEAdIrttlfYkdUFuMp0dwmWNZhCq5AD6DHDfGt0yMA jSXEaODpZsBQK84t6sq1fAOYY1Eo2iMggWg1pFGmRGORKEIL1D8ZjjPii5BswZWG6QBq WdlQ== X-Gm-Message-State: AOAM531hsmKzReFWdskeRhzZz7zqn2ZkTs14Lnp/uKlhEoTfPoR78IgR 0/fifnmGFj4DiQpx9duTYGMnhbvO1w== X-Google-Smtp-Source: ABdhPJw7p0IZNRzHEhoYVbNtBdDWX3bV3Cz4INszTsP30Q+/zUXpsgM1fhE69590WodfIcGOTAxlKDhPmw== X-Received: from elver.muc.corp.google.com ([2a00:79e0:15:13:92f8:c03b:1448:ada5]) (user=elver job=sendgmr) by 2002:a1c:4d0e:: with SMTP id o14mr20041525wmh.141.1618822261465; Mon, 19 Apr 2021 01:51:01 -0700 (PDT) Date: Mon, 19 Apr 2021 10:50:27 +0200 In-Reply-To: <20210419085027.761150-1-elver@google.com> Message-Id: <20210419085027.761150-4-elver@google.com> Mime-Version: 1.0 References: <20210419085027.761150-1-elver@google.com> X-Mailer: git-send-email 2.31.1.368.gbe11c130af-goog Subject: [PATCH 3/3] kfence: use power-efficient work queue to run delayed work From: Marco Elver To: elver@google.com, akpm@linux-foundation.org Cc: glider@google.com, dvyukov@google.com, jannh@google.com, mark.rutland@arm.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, kasan-dev@googlegroups.com X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 3AC8CE000108 X-Stat-Signature: 3338sd7w7bpce78ortg1t7kxjmxhihia Received-SPF: none (flex--elver.bounces.google.com>: No applicable sender policy available) receiver=imf21; identity=mailfrom; envelope-from="<3dUR9YAUKCPwipzivksskpi.gsqpmry1-qqozego.svk@flex--elver.bounces.google.com>"; helo=mail-wm1-f74.google.com; client-ip=209.85.128.74 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1618822260-915166 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000049, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Use the power-efficient work queue, to avoid the pathological case where we keep pinning ourselves on the same possibly idle CPU on systems that want to be power-efficient [1]. [1] https://lwn.net/Articles/731052/ Signed-off-by: Marco Elver --- mm/kfence/core.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/mm/kfence/core.c b/mm/kfence/core.c index 73e7b621fb36..7e20cd9690a2 100644 --- a/mm/kfence/core.c +++ b/mm/kfence/core.c @@ -642,7 +642,8 @@ static void toggle_allocation_gate(struct work_struct *work) /* Disable static key and reset timer. */ static_branch_disable(&kfence_allocation_key); #endif - schedule_delayed_work(&kfence_timer, msecs_to_jiffies(kfence_sample_interval)); + queue_delayed_work(system_power_efficient_wq, &kfence_timer, + msecs_to_jiffies(kfence_sample_interval)); } static DECLARE_DELAYED_WORK(kfence_timer, toggle_allocation_gate); @@ -671,7 +672,7 @@ void __init kfence_init(void) } WRITE_ONCE(kfence_enabled, true); - schedule_delayed_work(&kfence_timer, 0); + queue_delayed_work(system_power_efficient_wq, &kfence_timer, 0); pr_info("initialized - using %lu bytes for %d objects at 0x%p-0x%p\n", KFENCE_POOL_SIZE, CONFIG_KFENCE_NUM_OBJECTS, (void *)__kfence_pool, (void *)(__kfence_pool + KFENCE_POOL_SIZE));