From patchwork Mon Aug 26 22:45:37 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stephen Boyd X-Patchwork-Id: 2849825 Return-Path: X-Original-To: patchwork-linux-pm@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork2.web.kernel.org (Postfix) with ESMTP id F2F16BF546 for ; Mon, 26 Aug 2013 22:45:50 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 1006520362 for ; Mon, 26 Aug 2013 22:45:50 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 0FF9E20326 for ; Mon, 26 Aug 2013 22:45:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752287Ab3HZWpm (ORCPT ); Mon, 26 Aug 2013 18:45:42 -0400 Received: from smtp.codeaurora.org ([198.145.11.231]:44633 "EHLO smtp.codeaurora.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751835Ab3HZWpl (ORCPT ); Mon, 26 Aug 2013 18:45:41 -0400 Received: from smtp.codeaurora.org (localhost [127.0.0.1]) by smtp.codeaurora.org (Postfix) with ESMTP id 1BD4D13EE0C; Mon, 26 Aug 2013 22:45:41 +0000 (UTC) Received: by smtp.codeaurora.org (Postfix, from userid 486) id 0E69B13EF89; Mon, 26 Aug 2013 22:45:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Spam-Level: X-Spam-Status: No, score=-9.3 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 Received: from sboyd-linux.qualcomm.com (i-global252.qualcomm.com [199.106.103.252]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: sboyd@smtp.codeaurora.org) by smtp.codeaurora.org (Postfix) with ESMTPSA id 7D63113EEE6; Mon, 26 Aug 2013 22:45:40 +0000 (UTC) From: Stephen Boyd To: "Rafael J . Wysocki" , Viresh Kumar Cc: linux-kernel@vger.kernel.org, cpufreq@vger.kernel.org, linux-pm@vger.kernel.org Subject: [PATCH] cpufreq: Fix timer/workqueue corruption due to double queueing Date: Mon, 26 Aug 2013 15:45:37 -0700 Message-Id: <1377557137-1157-1-git-send-email-sboyd@codeaurora.org> X-Mailer: git-send-email 1.8.4.rc3.2.g2c2b664 X-Virus-Scanned: ClamAV using ClamSMTP Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP When a CPU is hot removed we'll cancel all the delayed work items via gov_cancel_work(). Normally this will just cancel a delayed timer on each CPU that the policy is managing and the work won't run, but if the work is already running the workqueue code will wait for the work to finish before continuing to prevent the work items from re-queuing themselves like they normally do. This scheme will work most of the time, except for the case where the work function determines that it should adjust the delay for all other CPUs that the policy is managing. If this scenario occurs, the canceling CPU will cancel its own work but queue up the other CPUs works to run. For example: CPU0 CPU1 ---- ---- cpu_down() ... __cpufreq_remove_dev() cpufreq_governor_dbs() case CPUFREQ_GOV_STOP: gov_cancel_work(dbs_data, policy); cpu0 work is canceled timer is canceled cpu1 work is canceled od_dbs_timer() gov_queue_work(*, *, true); cpu0 work queued cpu1 work queued cpu2 work queued ... cpu1 work is canceled cpu2 work is canceled ... At the end of the GOV_STOP case cpu0 still has a work queued to run although the code is expecting all of the works to be canceled. __cpufreq_remove_dev() will then proceed to re-initialize all the other CPUs works except for the CPU that is going down. The CPUFREQ_GOV_START case in cpufreq_governor_dbs() will trample over the queued work and debugobjects will spit out a warning: WARNING: at lib/debugobjects.c:260 debug_print_object+0x94/0xbc() ODEBUG: init active (active state 0) object type: timer_list hint: delayed_work_timer_fn+0x0/0x10 Modules linked in: CPU: 0 PID: 1491 Comm: sh Tainted: G W 3.10.0 #19 [] (unwind_backtrace+0x0/0x11c) from [] (show_stack+0x10/0x14) [] (show_stack+0x10/0x14) from [] (warn_slowpath_common+0x4c/0x6c) [] (warn_slowpath_common+0x4c/0x6c) from [] (warn_slowpath_fmt+0x2c/0x3c) [] (warn_slowpath_fmt+0x2c/0x3c) from [] (debug_print_object+0x94/0xbc) [] (debug_print_object+0x94/0xbc) from [] (__debug_object_init+0x2d0/0x340) [] (__debug_object_init+0x2d0/0x340) from [] (init_timer_key+0x14/0xb0) [] (init_timer_key+0x14/0xb0) from [] (cpufreq_governor_dbs+0x3e8/0x5f8) [] (cpufreq_governor_dbs+0x3e8/0x5f8) from [] (__cpufreq_governor+0xdc/0x1a4) [] (__cpufreq_governor+0xdc/0x1a4) from [] (__cpufreq_remove_dev.isra.10+0x3b4/0x434) [] (__cpufreq_remove_dev.isra.10+0x3b4/0x434) from [] (cpufreq_cpu_callback+0x60/0x80) [] (cpufreq_cpu_callback+0x60/0x80) from [] (notifier_call_chain+0x38/0x68) [] (notifier_call_chain+0x38/0x68) from [] (__cpu_notify+0x28/0x40) [] (__cpu_notify+0x28/0x40) from [] (_cpu_down+0x7c/0x2c0) [] (_cpu_down+0x7c/0x2c0) from [] (cpu_down+0x24/0x40) [] (cpu_down+0x24/0x40) from [] (store_online+0x2c/0x74) [] (store_online+0x2c/0x74) from [] (dev_attr_store+0x18/0x24) [] (dev_attr_store+0x18/0x24) from [] (sysfs_write_file+0x100/0x148) [] (sysfs_write_file+0x100/0x148) from [] (vfs_write+0xcc/0x174) [] (vfs_write+0xcc/0x174) from [] (SyS_write+0x38/0x64) [] (SyS_write+0x38/0x64) from [] (ret_fast_syscall+0x0/0x30) The simplest fix is to check and see if the governor is being stopped and ignore the all_cpus flag so that only the work that's being canceled has the chance to re-queue itself. Signed-off-by: Stephen Boyd --- This should probably go to stable. I think this all started happening in commit 031299b3be30f3ec (cpufreq: governors: Avoid unnecessary per cpu timer interrupts, 2013-02-27). drivers/cpufreq/cpufreq_governor.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/cpufreq/cpufreq_governor.c b/drivers/cpufreq/cpufreq_governor.c index 7b839a8..0375a3c 100644 --- a/drivers/cpufreq/cpufreq_governor.c +++ b/drivers/cpufreq/cpufreq_governor.c @@ -133,7 +133,7 @@ void gov_queue_work(struct dbs_data *dbs_data, struct cpufreq_policy *policy, { int i; - if (!all_cpus) { + if (!all_cpus || !policy->governor_enabled) { __gov_queue_work(smp_processor_id(), dbs_data, delay); } else { for_each_cpu(i, policy->cpus)