From patchwork Wed Apr 21 10:51:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marco Elver X-Patchwork-Id: 12215967 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-21.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT, USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E0AABC433B4 for ; Wed, 21 Apr 2021 10:52:02 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 70576613FB for ; Wed, 21 Apr 2021 10:52:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 70576613FB Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id DE3466B006E; Wed, 21 Apr 2021 06:52:01 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DB90E6B0070; Wed, 21 Apr 2021 06:52:01 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C80C26B0071; Wed, 21 Apr 2021 06:52:01 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0117.hostedemail.com [216.40.44.117]) by kanga.kvack.org (Postfix) with ESMTP id AE8536B006E for ; Wed, 21 Apr 2021 06:52:01 -0400 (EDT) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 6AFE53644 for ; Wed, 21 Apr 2021 10:52:01 +0000 (UTC) X-FDA: 78056059242.20.1B74231 Received: from mail-qt1-f202.google.com (mail-qt1-f202.google.com [209.85.160.202]) by imf04.hostedemail.com (Postfix) with ESMTP id B5B073C1 for ; Wed, 21 Apr 2021 10:51:58 +0000 (UTC) Received: by mail-qt1-f202.google.com with SMTP id y10-20020a05622a004ab029019d4ad3437cso12861242qtw.12 for ; Wed, 21 Apr 2021 03:52:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:message-id:mime-version:subject:from:to:cc; bh=sw+9jYBxrqSEYlhC6g7s4hihWn7xQEI251/x8DCExM4=; b=niS9o6XJW15BgeAt66tRA+u87VsppXQgRDjyUi7vi7+RzcRN5tGboc3Y4Dux+70z2K jqRWXIKAvKhitjw8mDVCUyt+YkxMgXeEmSh1o8wgjQFDMYPk+a64EW2UmvcTWWi+P+Vo ix2X81SAy6qwLKkub9epNGmLn3zlprAfWf9V8hruwqDXQjUIiQl5i0egdjHCI6A4WsPI ZIkaOT7zXuulEz1aMpl8A1yo1kcGq6gBru5BPD1Lmpgajdu+cc+8OmUFE1uGxYgZHOLN z87b4xk+EI8zqoYIUoHYkmpHsV6MI9hNUbp/UJgCoO0rAvpESrv0vBQzj0hasrdUPiZQ WkXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:mime-version:subject:from:to:cc; bh=sw+9jYBxrqSEYlhC6g7s4hihWn7xQEI251/x8DCExM4=; b=aNnJJe3RqqHwNr5/u6YYLpzjOL0WTW6rluGjaEQ0Xjd7tBiAIZWl0KELqvZWKX5qrd cr0ICOIz0+aBrVYF0BTgaVzVCM9kXx2VbjLKJ7XVuMLViqA8B9HmOd/FuNbtQRdoCJnW /CxO2dvZ0QWtNQm0e/nHn68irvzgqMpQz8+T4Jiy6xbge4Ic4YN98KKpNTO/frwUxuTk QXtaOsstJoWeqywVRlua86CUcsFmSTvAixKHF8XgPzMTmRPTDjeJhuenTnye0y1TxcjY ulSEWrRP7oPcsXBA4K7+YW3mO9AuUKK93LQN1rXGsCd2RbfL9swjWueLAd7dF3DgczGT oL/A== X-Gm-Message-State: AOAM531+yvi0NW0eSJ0U2DwpWyuZXyRjY3ijp8fWciV++jz7qipXVuGu Trclj3numRqxEwn1mUxJf1sCDvl5yg== X-Google-Smtp-Source: ABdhPJxbhjQSZUVIqBwWaNHt9KmfKEu61gegKhBYh8G8q+ZgtJQobtOJZHrdlreDY95xs18AvjUrvcoyEg== X-Received: from elver.muc.corp.google.com ([2a00:79e0:15:13:c552:ee7c:6a14:80cc]) (user=elver job=sendgmr) by 2002:ad4:5be1:: with SMTP id k1mr12623987qvc.55.1619002320199; Wed, 21 Apr 2021 03:52:00 -0700 (PDT) Date: Wed, 21 Apr 2021 12:51:29 +0200 Message-Id: <20210421105132.3965998-1-elver@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.31.1.368.gbe11c130af-goog Subject: [PATCH v2 0/3] kfence: optimize timer scheduling From: Marco Elver To: elver@google.com, akpm@linux-foundation.org Cc: glider@google.com, dvyukov@google.com, jannh@google.com, mark.rutland@arm.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, kasan-dev@googlegroups.com, hdanton@sina.com X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: B5B073C1 X-Stat-Signature: jxgss1k1k46ogytzso9eboguoi78x899 Received-SPF: none (flex--elver.bounces.google.com>: No applicable sender policy available) receiver=imf04; identity=mailfrom; envelope-from="<30AOAYAUKCOEHOYHUJRRJOH.FRPOLQXa-PPNYDFN.RUJ@flex--elver.bounces.google.com>"; helo=mail-qt1-f202.google.com; client-ip=209.85.160.202 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1619002318-295463 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: We have observed that mostly-idle systems with KFENCE enabled wake up otherwise idle CPUs, preventing such to enter a lower power state. Debugging revealed that KFENCE spends too much active time in toggle_allocation_gate(). While the first version of KFENCE was using all the right bits to be scheduling optimal, and thus power efficient, by simply using wait_event() + wake_up(), that code was unfortunately removed. As KFENCE was exposed to various different configs and tests, the scheduling optimal code slowly disappeared. First because of hung task warnings, and finally because of deadlocks when an allocation is made by timer code with debug objects enabled. Clearly, the "fixes" were not too friendly for devices that want to be power efficient. Therefore, let's try a little harder to fix the hung task and deadlock problems that we have with wait_event() + wake_up(), while remaining as scheduling friendly and power efficient as possible. Crucially, we need to defer the wake_up() to an irq_work, avoiding any potential for deadlock. The result with this series is that on the devices where we observed a power regression, power usage returns back to baseline levels. Changelog --------- v2: * Replace kfence_timer_waiting with simpler waitqueue_active() check. v1: https://lkml.kernel.org/r/20210419085027.761150-1-elver@google.com Marco Elver (3): kfence: await for allocation using wait_event kfence: maximize allocation wait timeout duration kfence: use power-efficient work queue to run delayed work lib/Kconfig.kfence | 1 + mm/kfence/core.c | 58 ++++++++++++++++++++++++++++++++-------------- 2 files changed, 42 insertions(+), 17 deletions(-)