From patchwork Thu Sep 23 14:09:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pingfan Liu X-Patchwork-Id: 12513057 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.7 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 50080C433EF for ; Thu, 23 Sep 2021 14:12:35 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1F30561019 for ; Thu, 23 Sep 2021 14:12:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 1F30561019 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=sujlmqfs0YlIwWalTw9i5UcNp7ZftS3BQAGf0sX4r4Q=; b=Cx7TOXi7VWnjoR JamJVx4rJDH4CorRbNmTB0Ux8rw9kv6dn+Ag3sAyRv2v9LiKkezRnZh6X7E3fDv0B8EPfhrD5H2CY o5otjqfyUBD2uNLSXQ5puULeN1kqknt0ZycG2hsIU111dr69ffZZRFJjYYLWwW1Zh/T0ElacQ8uMu NU/Y3/fwwxOxOs/iiAAWORwsf75ZKoVv2kDMyfniN4ucTmj1Z/7CzQrxcFJ1TFqw9EH/XqHF3lkPh 7QcXtK0gTv3vaZjw2whNcqLcdUIHhvebVF9zk/LPWibfZMkBNNwOlWsFVao7lnWMTSKMKWqh7owaZ r5YVC6kL5QecY8upoUew==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mTPRG-00BmNG-Cs; Thu, 23 Sep 2021 14:11:02 +0000 Received: from mail-pf1-x434.google.com ([2607:f8b0:4864:20::434]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mTPQq-00BmDs-8T for linux-arm-kernel@lists.infradead.org; Thu, 23 Sep 2021 14:10:37 +0000 Received: by mail-pf1-x434.google.com with SMTP id w14so5866570pfu.2 for ; Thu, 23 Sep 2021 07:10:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=4uC2eWg7aEhw9OQsJm+DyTckWnK1d1EQQKLFN4MrYQw=; b=nZcWLyFX6b4h1Cn/kDTSOUiZYJB63rCYafEeVclmX1skSVyMr5tul8uelIiYPoPQoE qAn6MslpW+m0PQ1NmIdZSCRnuLaIKwBzHQMRIGp89JF6vqOgVHWkvQRzWwx5mY3Z687F 1WAtFXZnSPgLjLUHuWOp8+xgBcfW9/XwQ6HMyoornzPYfjc25ncVI84+fJhrmVfa0CeM t2ZUZjFkkp+FJGLgb2rpJ0kGvHQ41li/JhuZSeeAOGrnTM5BxoSqVvT9Z9HasNoP0v4F iaZHqiSZmfjr649EPagTN1+sv7nEvyvzdG45Lb+7pampaxFx/wtw4atEMzDywhsNJLYO +zcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4uC2eWg7aEhw9OQsJm+DyTckWnK1d1EQQKLFN4MrYQw=; b=FvUj3oH75I/eJyTfxCkk8YabU8EhDtvOfBVUzSu+6VSKcQbb2fUIUYRN/7g5GxRP+v MGypj+jFhxW+APEhbdO5euNZQyVUVlktpVx5GW8zlRuFytRoafmrAf5R+8KtU/ZMUPxW Ld6349eQRYO2RnstPD+BJsH621jap5KINXEcTC/bU05DetH3V+EN5/RQmIKylAnHkwNG ktMPDBtPpzfDM2LhgNr4hZ18AfvdKhHU1HtGtxEMmZsujy1CDqtawGdqQ2WmiRuLNmVe ri59QO9gWKcK4XJgDgMB/p2jFa9qH36rI9wVCowTnFiBR0v6/24vyRCOFTRxbUnq2Tud 53wg== X-Gm-Message-State: AOAM532zNoIj+z9IfEoAjJiv40NSbNHC7eB8d/7Sj3kA3VFBimEb+STK Q8JTczo0d2c6ZWNTel1Uvw== X-Google-Smtp-Source: ABdhPJwmNSh05Gr/CtxmPWIkNtONd+/aOhaL6bSUzPs/gvjn+O8HsGhBu1NC2ey/m1AQaTZn2uSBCg== X-Received: by 2002:a62:1e43:0:b0:447:cb0b:4c6e with SMTP id e64-20020a621e43000000b00447cb0b4c6emr4535184pfe.1.1632406235819; Thu, 23 Sep 2021 07:10:35 -0700 (PDT) Received: from piliu.users.ipa.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id j6sm5800766pfn.107.2021.09.23.07.10.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 Sep 2021 07:10:34 -0700 (PDT) From: Pingfan Liu To: linux-kernel@vger.kernel.org Cc: Pingfan Liu , Sumit Garg , Catalin Marinas , Will Deacon , Ingo Molnar , Arnaldo Carvalho de Melo , Mark Rutland , Alexander Shishkin , Jiri Olsa , Namhyung Kim , Marc Zyngier , Kees Cook , Masahiro Yamada , Sami Tolvanen , Petr Mladek , Andrew Morton , Wang Qing , "Peter Zijlstra (Intel)" , Santosh Sivaraj , linux-arm-kernel@lists.infradead.org Subject: [PATCHv2 3/4] kernel/watchdog: adapt the watchdog_hld interface for async model Date: Thu, 23 Sep 2021 22:09:50 +0800 Message-Id: <20210923140951.35902-4-kernelfans@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210923140951.35902-1-kernelfans@gmail.com> References: <20210923140951.35902-1-kernelfans@gmail.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210923_071036_397792_1CD9EE1E X-CRM114-Status: GOOD ( 19.37 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When lockup_detector_init()->watchdog_nmi_probe(), PMU may be not ready yet. E.g. on arm64, PMU is not ready until device_initcall(armv8_pmu_driver_init). And it is deeply integrated with the driver model and cpuhp. Hence it is hard to push this initialization before smp_init(). But it is easy to take an opposite approach by enabling watchdog_hld to get the capability of PMU async. The async model is achieved by expanding watchdog_nmi_probe() with -EBUSY, and a re-initializing work_struct which waits on a wait_queue_head. Signed-off-by: Pingfan Liu Cc: Sumit Garg Cc: Catalin Marinas Cc: Will Deacon Cc: Ingo Molnar Cc: Arnaldo Carvalho de Melo Cc: Mark Rutland Cc: Alexander Shishkin Cc: Jiri Olsa Cc: Namhyung Kim Cc: Marc Zyngier Cc: Kees Cook Cc: Masahiro Yamada Cc: Sami Tolvanen Cc: Petr Mladek Cc: Andrew Morton Cc: Wang Qing Cc: "Peter Zijlstra (Intel)" Cc: Santosh Sivaraj Cc: linux-arm-kernel@lists.infradead.org To: linux-kernel@vger.kernel.org --- include/linux/nmi.h | 3 +++ kernel/watchdog.c | 37 +++++++++++++++++++++++++++++++++++-- 2 files changed, 38 insertions(+), 2 deletions(-) diff --git a/include/linux/nmi.h b/include/linux/nmi.h index b7bcd63c36b4..270d440fe4b7 100644 --- a/include/linux/nmi.h +++ b/include/linux/nmi.h @@ -118,6 +118,9 @@ static inline int hardlockup_detector_perf_init(void) { return 0; } void watchdog_nmi_stop(void); void watchdog_nmi_start(void); + +extern bool hld_detector_delay_initialized; +extern struct wait_queue_head hld_detector_wait; int watchdog_nmi_probe(void); void watchdog_nmi_enable(unsigned int cpu); void watchdog_nmi_disable(unsigned int cpu); diff --git a/kernel/watchdog.c b/kernel/watchdog.c index 6e6dd5f0bc3e..bd4ae1839b72 100644 --- a/kernel/watchdog.c +++ b/kernel/watchdog.c @@ -103,7 +103,10 @@ void __weak watchdog_nmi_disable(unsigned int cpu) hardlockup_detector_perf_disable(); } -/* Return 0, if a NMI watchdog is available. Error code otherwise */ +/* + * Return 0, if a NMI watchdog is available. -EBUSY if not ready. + * Other negative value if not support. + */ int __weak __init watchdog_nmi_probe(void) { return hardlockup_detector_perf_init(); @@ -739,15 +742,45 @@ int proc_watchdog_cpumask(struct ctl_table *table, int write, } #endif /* CONFIG_SYSCTL */ +static void lockup_detector_delay_init(struct work_struct *work); +bool hld_detector_delay_initialized __initdata; + +struct wait_queue_head hld_detector_wait __initdata = + __WAIT_QUEUE_HEAD_INITIALIZER(hld_detector_wait); + +static struct work_struct detector_work __initdata = + __WORK_INITIALIZER(detector_work, lockup_detector_delay_init); + +static void __init lockup_detector_delay_init(struct work_struct *work) +{ + int ret; + + wait_event(hld_detector_wait, hld_detector_delay_initialized); + ret = watchdog_nmi_probe(); + if (!ret) { + nmi_watchdog_available = true; + lockup_detector_setup(); + } else { + WARN_ON(ret == -EBUSY); + pr_info("Perf NMI watchdog permanently disabled\n"); + } +} + void __init lockup_detector_init(void) { + int ret; + if (tick_nohz_full_enabled()) pr_info("Disabling watchdog on nohz_full cores by default\n"); cpumask_copy(&watchdog_cpumask, housekeeping_cpumask(HK_FLAG_TIMER)); - if (!watchdog_nmi_probe()) + ret = watchdog_nmi_probe(); + if (!ret) nmi_watchdog_available = true; + else if (ret == -EBUSY) + queue_work_on(smp_processor_id(), system_wq, &detector_work); + lockup_detector_setup(); }