From patchwork Mon Jul 24 13:22:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Leizhen (ThunderTown)" X-Patchwork-Id: 13324781 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9ABAC001DF for ; Mon, 24 Jul 2023 13:23:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230461AbjGXNXt (ORCPT ); Mon, 24 Jul 2023 09:23:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60954 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229535AbjGXNXs (ORCPT ); Mon, 24 Jul 2023 09:23:48 -0400 Received: from dggsgout12.his.huawei.com (unknown [45.249.212.56]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5C8EBE0; Mon, 24 Jul 2023 06:23:47 -0700 (PDT) Received: from mail02.huawei.com (unknown [172.30.67.143]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTP id 4R8gph1hJ1z4f3nyV; Mon, 24 Jul 2023 21:23:40 +0800 (CST) Received: from huaweicloud.com (unknown [10.174.178.55]) by APP4 (Coremail) with SMTP id gCh0CgBn0LNbe75kcavTOg--.2237S5; Mon, 24 Jul 2023 21:23:42 +0800 (CST) From: thunder.leizhen@huaweicloud.com To: "Paul E . McKenney" , Frederic Weisbecker , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Zqiang , rcu@vger.kernel.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org Cc: Zhen Lei Subject: [PATCH 1/2] softirq: fix integer overflow in function show_stat() Date: Mon, 24 Jul 2023 21:22:23 +0800 Message-Id: <20230724132224.916-2-thunder.leizhen@huaweicloud.com> X-Mailer: git-send-email 2.37.3.windows.1 In-Reply-To: <20230724132224.916-1-thunder.leizhen@huaweicloud.com> References: <20230724132224.916-1-thunder.leizhen@huaweicloud.com> MIME-Version: 1.0 X-CM-TRANSID: gCh0CgBn0LNbe75kcavTOg--.2237S5 X-Coremail-Antispam: 1UD129KBjvJXoWxJryDGFy7KryxurWxWF4fAFb_yoW8WFy5pa 43Kw1jvrW8Cw17XFs7JF1jgry8JF98JayavFyfG342qFyUJ3Z0gFyfKFZ0gFWjgrWrC3yr Aa17KryUurWDX3JanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUBKb4IE77IF4wAFF20E14v26rWj6s0DM7CY07I20VC2zVCF04k2 6cxKx2IYs7xG6r1F6r1fM7CIcVAFz4kK6r1j6r18M28IrcIa0xkI8VA2jI8067AKxVWUGw A2048vs2IY020Ec7CjxVAFwI0_Gr0_Xr1l8cAvFVAK0II2c7xJM28CjxkF64kEwVA0rcxS w2x7M28EF7xvwVC0I7IYx2IY67AKxVWDJVCq3wA2z4x0Y4vE2Ix0cI8IcVCY1x0267AKxV W8Jr0_Cr1UM28EF7xvwVC2z280aVAFwI0_GcCE3s1l84ACjcxK6I8E87Iv6xkF7I0E14v2 6rxl6s0DM2AIxVAIcxkEcVAq07x20xvEncxIr21l5I8CrVACY4xI64kE6c02F40Ex7xfMc Ij6xIIjxv20xvE14v26r1j6r18McIj6I8E87Iv67AKxVWUJVW8JwAm72CE4IkC6x0Yz7v_ Jr0_Gr1lF7xvr2IYc2Ij64vIr41lFIxGxcIEc7CjxVA2Y2ka0xkIwI1lw4CEc2x0rVAKj4 xxMxAIw28IcxkI7VAKI48JMxC20s026xCaFVCjc4AY6r1j6r4UMI8I3I0E5I8CrVAFwI0_ Jr0_Jr4lx2IqxVCjr7xvwVAFwI0_JrI_JrWlx4CE17CEb7AF67AKxVWUtVW8ZwCIc40Y0x 0EwIxGrwCI42IY6xIIjxv20xvE14v26r1j6r1xMIIF0xvE2Ix0cI8IcVCY1x0267AKxVW8 JVWxJwCI42IY6xAIw20EY4v20xvaj40_Jr0_JF4lIxAIcVC2z280aVAFwI0_Jr0_Gr1lIx AIcVC2z280aVCY1x0267AKxVW8JVW8JrUvcSsGvfC2KfnxnUUI43ZEXa7IU8qFAJUUUUU= = X-CM-SenderInfo: hwkx0vthuozvpl2kv046kxt4xhlfz01xgou0bp/ X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: rcu@vger.kernel.org From: Zhen Lei The statistics function of softirq is supported by commit aa0ce5bbc2db ("softirq: introduce statistics for softirq") in 2009. At that time, 64-bit processors should not have many cores and would not face significant count overflow problems. Now it's common for a processor to have hundreds of cores. Assume that there are 100 cores and 10 TIMER_SOFTIRQ are generated per second, then the 32-bit sum will be overflowed after 50 days. For example: seq_put_decimal_ull(p, "softirq ", (unsigned long long)sum_softirq); for (i = 0; i < NR_SOFTIRQS; i++) seq_put_decimal_ull(p, " ", per_softirq_sums[i]); $ cat /proc/stat | tail -n 1 softirq 22929066124 9 2963267579 4128150 2618598635 546358555 0 \ 629391610 1326100278 74637 1956244783 Here, the sum of per_softirq_sums[] is 10044164236 and is not equal to 22929066124. Because integers overflowed. Therefore, change the type of local variable per_softirq_sums[] to u64. Fixes: d3d64df21d3d ("proc: export statistics for softirq to /proc") Signed-off-by: Zhen Lei --- fs/proc/stat.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/proc/stat.c b/fs/proc/stat.c index da60956b2915645..84aac577a50cabb 100644 --- a/fs/proc/stat.c +++ b/fs/proc/stat.c @@ -86,7 +86,7 @@ static int show_stat(struct seq_file *p, void *v) u64 guest, guest_nice; u64 sum = 0; u64 sum_softirq = 0; - unsigned int per_softirq_sums[NR_SOFTIRQS] = {0}; + u64 per_softirq_sums[NR_SOFTIRQS] = {0}; struct timespec64 boottime; user = nice = system = idle = iowait = From patchwork Mon Jul 24 13:22:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Leizhen (ThunderTown)" X-Patchwork-Id: 13324779 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EA356C001B0 for ; Mon, 24 Jul 2023 13:23:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230441AbjGXNXt (ORCPT ); Mon, 24 Jul 2023 09:23:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60956 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230310AbjGXNXs (ORCPT ); Mon, 24 Jul 2023 09:23:48 -0400 Received: from dggsgout12.his.huawei.com (unknown [45.249.212.56]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2CC58DA; Mon, 24 Jul 2023 06:23:47 -0700 (PDT) Received: from mail02.huawei.com (unknown [172.30.67.143]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTP id 4R8gph6cbxz4f3nyk; Mon, 24 Jul 2023 21:23:40 +0800 (CST) Received: from huaweicloud.com (unknown [10.174.178.55]) by APP4 (Coremail) with SMTP id gCh0CgBn0LNbe75kcavTOg--.2237S6; Mon, 24 Jul 2023 21:23:43 +0800 (CST) From: thunder.leizhen@huaweicloud.com To: "Paul E . McKenney" , Frederic Weisbecker , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Zqiang , rcu@vger.kernel.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org Cc: Zhen Lei Subject: [PATCH 2/2] softirq: redefine the type of kernel_stat.softirqs[] as unsigned long Date: Mon, 24 Jul 2023 21:22:24 +0800 Message-Id: <20230724132224.916-3-thunder.leizhen@huaweicloud.com> X-Mailer: git-send-email 2.37.3.windows.1 In-Reply-To: <20230724132224.916-1-thunder.leizhen@huaweicloud.com> References: <20230724132224.916-1-thunder.leizhen@huaweicloud.com> MIME-Version: 1.0 X-CM-TRANSID: gCh0CgBn0LNbe75kcavTOg--.2237S6 X-Coremail-Antispam: 1UD129KBjvJXoWxXF43tF48KF13JF1UJryUAwb_yoW7Jw17pF yqkFy7Kw4UGa4jvas7JF4DuryUJwn5Ga4akwsxt34fta45Jr1FgFn3u34qqrWjgrW8Ga1S yFWayrWjg3yDW3JanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUBKb4IE77IF4wAFF20E14v26rWj6s0DM7CY07I20VC2zVCF04k2 6cxKx2IYs7xG6r1F6r1fM7CIcVAFz4kK6r1j6r18M28IrcIa0xkI8VA2jI8067AKxVWUXw A2048vs2IY020Ec7CjxVAFwI0_Xr0E3s1l8cAvFVAK0II2c7xJM28CjxkF64kEwVA0rcxS w2x7M28EF7xvwVC0I7IYx2IY67AKxVWDJVCq3wA2z4x0Y4vE2Ix0cI8IcVCY1x0267AKxV W8Jr0_Cr1UM28EF7xvwVC2z280aVAFwI0_GcCE3s1l84ACjcxK6I8E87Iv6xkF7I0E14v2 6rxl6s0DM2AIxVAIcxkEcVAq07x20xvEncxIr21l5I8CrVACY4xI64kE6c02F40Ex7xfMc Ij6xIIjxv20xvE14v26r1j6r18McIj6I8E87Iv67AKxVWUJVW8JwAm72CE4IkC6x0Yz7v_ Jr0_Gr1lF7xvr2IYc2Ij64vIr41lFIxGxcIEc7CjxVA2Y2ka0xkIwI1lw4CEc2x0rVAKj4 xxMxAIw28IcxkI7VAKI48JMxC20s026xCaFVCjc4AY6r1j6r4UMI8I3I0E5I8CrVAFwI0_ Jr0_Jr4lx2IqxVCjr7xvwVAFwI0_JrI_JrWlx4CE17CEb7AF67AKxVWUtVW8ZwCIc40Y0x 0EwIxGrwCI42IY6xIIjxv20xvE14v26r1j6r1xMIIF0xvE2Ix0cI8IcVCY1x0267AKxVW8 JVWxJwCI42IY6xAIw20EY4v20xvaj40_Jr0_JF4lIxAIcVC2z280aVAFwI0_Jr0_Gr1lIx AIcVC2z280aVCY1x0267AKxVW8JVW8JrUvcSsGvfC2KfnxnUUI43ZEXa7IU1yEEUUUUUU= = X-CM-SenderInfo: hwkx0vthuozvpl2kv046kxt4xhlfz01xgou0bp/ X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: rcu@vger.kernel.org From: Zhen Lei As commit aa0ce5bbc2db ("softirq: introduce statistics for softirq") mentioned, the number of one softirq can exceed 100 per second. At this rate, the 32-bit sum will overflow after 1.5 years. This problem can be avoided if the type of 'softirqs[]' is changed to u64, but the atomicity of the counting operation cannot be guaranteed on 32-bit processors. Changing to unsigned long can safely solve the problem on 64-bit processors, and it is the same as the type of 'irqs_sum'. Signed-off-by: Zhen Lei --- fs/proc/softirqs.c | 2 +- fs/proc/stat.c | 2 +- include/linux/kernel_stat.h | 8 ++++---- kernel/rcu/tree.h | 2 +- kernel/rcu/tree_stall.h | 6 +++--- 5 files changed, 10 insertions(+), 10 deletions(-) diff --git a/fs/proc/softirqs.c b/fs/proc/softirqs.c index f4616083faef3bb..985be6904d748ab 100644 --- a/fs/proc/softirqs.c +++ b/fs/proc/softirqs.c @@ -20,7 +20,7 @@ static int show_softirqs(struct seq_file *p, void *v) for (i = 0; i < NR_SOFTIRQS; i++) { seq_printf(p, "%12s:", softirq_to_name[i]); for_each_possible_cpu(j) - seq_printf(p, " %10u", kstat_softirqs_cpu(i, j)); + seq_printf(p, " %10lu", kstat_softirqs_cpu(i, j)); seq_putc(p, '\n'); } return 0; diff --git a/fs/proc/stat.c b/fs/proc/stat.c index 84aac577a50cabb..7d40d98471d5ab2 100644 --- a/fs/proc/stat.c +++ b/fs/proc/stat.c @@ -116,7 +116,7 @@ static int show_stat(struct seq_file *p, void *v) sum += arch_irq_stat_cpu(i); for (j = 0; j < NR_SOFTIRQS; j++) { - unsigned int softirq_stat = kstat_softirqs_cpu(j, i); + unsigned long softirq_stat = kstat_softirqs_cpu(j, i); per_softirq_sums[j] += softirq_stat; sum_softirq += softirq_stat; diff --git a/include/linux/kernel_stat.h b/include/linux/kernel_stat.h index 9935f7ecbfb9e31..b6c5723a1149cd6 100644 --- a/include/linux/kernel_stat.h +++ b/include/linux/kernel_stat.h @@ -40,7 +40,7 @@ struct kernel_cpustat { struct kernel_stat { unsigned long irqs_sum; - unsigned int softirqs[NR_SOFTIRQS]; + unsigned long softirqs[NR_SOFTIRQS]; }; DECLARE_PER_CPU(struct kernel_stat, kstat); @@ -63,15 +63,15 @@ static inline void kstat_incr_softirqs_this_cpu(unsigned int irq) __this_cpu_inc(kstat.softirqs[irq]); } -static inline unsigned int kstat_softirqs_cpu(unsigned int irq, int cpu) +static inline unsigned long kstat_softirqs_cpu(unsigned int irq, int cpu) { return kstat_cpu(cpu).softirqs[irq]; } -static inline unsigned int kstat_cpu_softirqs_sum(int cpu) +static inline unsigned long kstat_cpu_softirqs_sum(int cpu) { int i; - unsigned int sum = 0; + unsigned long sum = 0; for (i = 0; i < NR_SOFTIRQS; i++) sum += kstat_softirqs_cpu(i, cpu); diff --git a/kernel/rcu/tree.h b/kernel/rcu/tree.h index 192536916f9a607..ce51640cd34f66b 100644 --- a/kernel/rcu/tree.h +++ b/kernel/rcu/tree.h @@ -268,7 +268,7 @@ struct rcu_data { unsigned long rcuc_activity; /* 7) Diagnostic data, including RCU CPU stall warnings. */ - unsigned int softirq_snap; /* Snapshot of softirq activity. */ + unsigned long softirq_snap; /* Snapshot of softirq activity. */ /* ->rcu_iw* fields protected by leaf rcu_node ->lock. */ struct irq_work rcu_iw; /* Check for non-irq activity. */ bool rcu_iw_pending; /* Is ->rcu_iw pending? */ diff --git a/kernel/rcu/tree_stall.h b/kernel/rcu/tree_stall.h index 0ade7f9dcaa18da..d5b22700abcc385 100644 --- a/kernel/rcu/tree_stall.h +++ b/kernel/rcu/tree_stall.h @@ -446,7 +446,7 @@ static void print_cpu_stat_info(int cpu) rsr.cputime_system = kcpustat_field(kcsp, CPUTIME_SYSTEM, cpu); pr_err("\t hardirqs softirqs csw/system\n"); - pr_err("\t number: %8ld %10d %12lld\n", + pr_err("\t number: %8ld %10ld %12lld\n", kstat_cpu_irqs_sum(cpu) - rsrp->nr_hardirqs, kstat_cpu_softirqs_sum(cpu) - rsrp->nr_softirqs, nr_context_switches_cpu(cpu) - rsrp->nr_csw); @@ -498,7 +498,7 @@ static void print_cpu_stall_info(int cpu) rcuc_starved = rcu_is_rcuc_kthread_starving(rdp, &j); if (rcuc_starved) sprintf(buf, " rcuc=%ld jiffies(starved)", j); - pr_err("\t%d-%c%c%c%c: (%lu %s) idle=%04x/%ld/%#lx softirq=%u/%u fqs=%ld%s%s\n", + pr_err("\t%d-%c%c%c%c: (%lu %s) idle=%04x/%ld/%#lx softirq=%lu/%lu fqs=%ld%s%s\n", cpu, "O."[!!cpu_online(cpu)], "o."[!!(rdp->grpmask & rdp->mynode->qsmaskinit)], @@ -575,7 +575,7 @@ static void rcu_check_gp_kthread_expired_fqs_timer(void) data_race(rcu_state.gp_flags), gp_state_getname(RCU_GP_WAIT_FQS), RCU_GP_WAIT_FQS, data_race(READ_ONCE(gpk->__state))); - pr_err("\tPossible timer handling issue on cpu=%d timer-softirq=%u\n", + pr_err("\tPossible timer handling issue on cpu=%d timer-softirq=%lu\n", cpu, kstat_softirqs_cpu(TIMER_SOFTIRQ, cpu)); } }