From patchwork Fri Aug 7 21:29:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Adams X-Patchwork-Id: 11706109 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 458D91392 for ; Fri, 7 Aug 2020 21:30:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2E8D222CB3 for ; Fri, 7 Aug 2020 21:30:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="FbP0RxUp" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727893AbgHGVap (ORCPT ); Fri, 7 Aug 2020 17:30:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53482 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726787AbgHGV3l (ORCPT ); Fri, 7 Aug 2020 17:29:41 -0400 Received: from mail-qv1-xf49.google.com (mail-qv1-xf49.google.com [IPv6:2607:f8b0:4864:20::f49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AF864C061A2E for ; Fri, 7 Aug 2020 14:29:40 -0700 (PDT) Received: by mail-qv1-xf49.google.com with SMTP id l18so2409050qvq.16 for ; Fri, 07 Aug 2020 14:29:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=Xwhl6oTAS7prpbXCRZdZshg5xz57JrEbZ056Cu9NiLQ=; b=FbP0RxUpuIXc9TPHeycYqgrsWHy+edJZOunO3xzH9xgd3XMesTsydOHKMec+qxUJFA FlXnrJi21QCaDcqgirJGJT0eBkQ0v3Vq6LIzdTyTYf6I4CV8iPofNCGuawph3tfsMnfh rVST9SGjyClH8eIIJqualzLP+dJgXyLWWuF3VRTq6ejnpWajSiynlSZz5Y285bh+tnSW ZxEvaLhRHfrOvoRGHyx1uzFMrik/lFM8Yi0f5EOQG7sM80OuBBVAKF1eI2Nt5evM5J/V ZSfJtPKRCyz4Ary7BmVgPJgTRfuuh+/bHoAJlAFzxb/0O1Zu489t0ofM9yH0JSH6UL05 9KPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=Xwhl6oTAS7prpbXCRZdZshg5xz57JrEbZ056Cu9NiLQ=; b=V1913pRhCNKGw/EaPWGwKBl4ZLDezID4ZTOBl4BVk4PAkjX9JZpYBxDmxQAZ9ZFuGd Rc8E00mw2EOZT+TE8TyMlYt3rP/RBb2mn+nR7avprnBBiuxGYogVPFESKLLR7XyH7PTl 56lpwbFRG2YVjwF/8W1iKWUIuQt93IZqRmTQqH9hZ137AwiAdHu4xOzoa/Oo1Dn1+J1f 8W+Np7sVokZplPEmw/bqCn6ox2jHM1NsQC0ia6z8EoYMmqen0P0gKl9BtAvmNormNbDT i2TnnLYZfQhch+EomzpG7fjt7DN/ozBVPo6+D17H5MD+ZilOaOiMxpo4Y0h6DTsaf3ya NiNQ== X-Gm-Message-State: AOAM5300TiDNlpUGChwEvAJWed67BeDhf+CogxxvYcCMyE/OFbAJBsOr 9VBowZa83WjgEbSQ8uBN/gePATuIT9U= X-Google-Smtp-Source: ABdhPJyLLX7R17IOCOR3CY0ZoHgSxM9yzJvgcbYl6HaF+KCt80M13Jn4ki0ErUlYpRFS5LWuyydOwJIuc87a X-Received: by 2002:ad4:5502:: with SMTP id az2mr16402089qvb.148.1596835779896; Fri, 07 Aug 2020 14:29:39 -0700 (PDT) Date: Fri, 7 Aug 2020 14:29:12 -0700 In-Reply-To: <20200807212916.2883031-1-jwadams@google.com> Message-Id: <20200807212916.2883031-4-jwadams@google.com> Mime-Version: 1.0 References: <20200807212916.2883031-1-jwadams@google.com> X-Mailer: git-send-email 2.28.0.236.gb10cc79966-goog Subject: [RFC PATCH 3/7] core/metricfs: metric for kernel warnings From: Jonathan Adams To: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org Cc: netdev@vger.kernel.org, kvm@vger.kernel.org, Paolo Bonzini , Greg KH , Jim Mattson , David Rientjes , Jonathan Adams Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Count kernel warnings by function name of the caller. Each time WARN() is called, which includes WARN_ON(), increment a counter in a 256-entry hash table. The table key is the entry point of the calling function, which is found using kallsyms. We store the name of the function in the table (because it may be a module address); reporting the metric just walks the table and prints the values. The "warnings" metric is cumulative. Signed-off-by: Jonathan Adams --- jwadams@google.com: rebased to 5.8-rc6, removed google-isms, added lockdep_assert_held(), NMI handling, ..._unknown*_counts and locking in warn_tbl_fn(); renamed warn_metric... to warn_tbl... The original work was done in 2012 by an engineer no longer at Google. --- kernel/panic.c | 131 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 131 insertions(+) diff --git a/kernel/panic.c b/kernel/panic.c index e2157ca387c8..c019b41ab387 100644 --- a/kernel/panic.c +++ b/kernel/panic.c @@ -31,6 +31,9 @@ #include #include #include +#include +#include +#include #include #define PANIC_TIMER_STEP 100 @@ -568,6 +571,133 @@ void oops_exit(void) kmsg_dump(KMSG_DUMP_OOPS); } +#ifdef CONFIG_METRICFS + +/* + * Hash table from function address to count of WARNs called within that + * function. + * So far this is an add-only hash table (ie, entries never removed), so some + * simplifying assumptions are made. + */ +#define WARN_TBL_BITS (8) +#define WARN_TBL_SIZE (1<= 0) + warn_tbl[entry].count++; + else + warn_tbl_unknown_count++; + + spin_unlock_irqrestore(&warn_tbl_lock, flags); +} + +/* + * Export the hash table to metricfs. + */ +static void warn_tbl_fn(struct metric_emitter *e) +{ + int i; + unsigned long flags; + int unknown_count = READ_ONCE(warn_tbl_unknown_count) + + atomic_read(&warn_tbl_unknown_nmi_count) + + atomic_read(&warn_tbl_unknown_lookup_count); + + if (unknown_count != 0) + METRIC_EMIT_INT(e, unknown_count, "(unknown)", NULL); + + spin_lock_irqsave(&warn_tbl_lock, flags); + for (i = 0; i < WARN_TBL_SIZE; i++) { + unsigned long fn = (unsigned long)warn_tbl[i].function; + const char *function_name = warn_tbl[i].function_name; + int count = warn_tbl[i].count; + + if (!fn) + continue; + + // function_name[] is constant once function is non-NULL + spin_unlock_irqrestore(&warn_tbl_lock, flags); + METRIC_EMIT_INT(e, count, function_name, NULL); + spin_lock_irqsave(&warn_tbl_lock, flags); + } + spin_unlock_irqrestore(&warn_tbl_lock, flags); +} +METRIC_EXPORT_COUNTER(warnings, "Count of calls to WARN().", + "function", NULL, warn_tbl_fn); + +static int __init metricfs_panic_init(void) +{ + metric_init_warnings(NULL); + return 0; +} +late_initcall(metricfs_panic_init); + +#else /* CONFIG_METRICFS */ +inline void tbl_increment(void *caller) {} +#endif + struct warn_args { const char *fmt; va_list args; @@ -576,6 +706,7 @@ struct warn_args { void __warn(const char *file, int line, void *caller, unsigned taint, struct pt_regs *regs, struct warn_args *args) { + tbl_increment(caller); disable_trace_on_warning(); if (file)