From patchwork Wed Apr 25 10:59:31 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shilpasri G Bhat X-Patchwork-Id: 10362539 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 6F4CC6038F for ; Wed, 25 Apr 2018 11:00:18 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5FE9F28F7F for ; Wed, 25 Apr 2018 11:00:18 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5356628F71; Wed, 25 Apr 2018 11:00:18 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00, MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CEF2728F7F for ; Wed, 25 Apr 2018 11:00:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753018AbeDYLAE (ORCPT ); Wed, 25 Apr 2018 07:00:04 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:44588 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1754095AbeDYK7w (ORCPT ); Wed, 25 Apr 2018 06:59:52 -0400 Received: from pps.filterd (m0098417.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w3PAxQlL021970 for ; Wed, 25 Apr 2018 06:59:51 -0400 Received: from e06smtp11.uk.ibm.com (e06smtp11.uk.ibm.com [195.75.94.107]) by mx0a-001b2d01.pphosted.com with ESMTP id 2hjp4h6ev8-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 25 Apr 2018 06:59:51 -0400 Received: from localhost by e06smtp11.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 25 Apr 2018 11:59:49 +0100 Received: from b06cxnps3075.portsmouth.uk.ibm.com (9.149.109.195) by e06smtp11.uk.ibm.com (192.168.101.141) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Wed, 25 Apr 2018 11:59:46 +0100 Received: from d06av22.portsmouth.uk.ibm.com (d06av22.portsmouth.uk.ibm.com [9.149.105.58]) by b06cxnps3075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w3PAxjXL590234; Wed, 25 Apr 2018 10:59:45 GMT Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7F69C4C04E; Wed, 25 Apr 2018 11:52:05 +0100 (BST) Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5A45B4C059; Wed, 25 Apr 2018 11:52:01 +0100 (BST) Received: from oc4502181600.in.ibm.com (unknown [9.84.225.83]) by d06av22.portsmouth.uk.ibm.com (Postfix) with ESMTP; Wed, 25 Apr 2018 11:52:01 +0100 (BST) From: Shilpasri G Bhat To: rjw@rjwysocki.net, viresh.kumar@linaro.org Cc: npiggin@gmail.com, benh@kernel.crashing.org, mpe@ellerman.id.au, linux-pm@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, ppaidipe@linux.vnet.ibm.com, svaidy@linux.vnet.ibm.com, Shilpasri G Bhat , Subject: [PATCH V3] cpufreq: powernv: Fix the hardlockup by synchronus smp_call in timer interrupt Date: Wed, 25 Apr 2018 16:29:31 +0530 X-Mailer: git-send-email 1.8.3.1 X-TM-AS-GCONF: 00 x-cbid: 18042510-0040-0000-0000-000004513A3D X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18042510-0041-0000-0000-000020F5B131 Message-Id: <1524653971-4919-1-git-send-email-shilpa.bhat@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2018-04-25_03:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1709140000 definitions=main-1804250104 Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP gpstate_timer_handler() uses synchronous smp_call to set the pstate on the requested core. This causes the below hard lockup: [c000003fe566b320] [c0000000001d5340] smp_call_function_single+0x110/0x180 (unreliable) [c000003fe566b390] [c0000000001d55e0] smp_call_function_any+0x180/0x250 [c000003fe566b3f0] [c000000000acd3e8] gpstate_timer_handler+0x1e8/0x580 [c000003fe566b4a0] [c0000000001b46b0] call_timer_fn+0x50/0x1c0 [c000003fe566b520] [c0000000001b4958] expire_timers+0x138/0x1f0 [c000003fe566b590] [c0000000001b4bf8] run_timer_softirq+0x1e8/0x270 [c000003fe566b630] [c000000000d0d6c8] __do_softirq+0x158/0x3e4 [c000003fe566b710] [c000000000114be8] irq_exit+0xe8/0x120 [c000003fe566b730] [c000000000024d0c] timer_interrupt+0x9c/0xe0 [c000003fe566b760] [c000000000009014] decrementer_common+0x114/0x120 -- interrupt: 901 at doorbell_global_ipi+0x34/0x50 LR = arch_send_call_function_ipi_mask+0x120/0x130 [c000003fe566ba50] [c00000000004876c] arch_send_call_function_ipi_mask+0x4c/0x130 [c000003fe566ba90] [c0000000001d59f0] smp_call_function_many+0x340/0x450 [c000003fe566bb00] [c000000000075f18] pmdp_invalidate+0x98/0xe0 [c000003fe566bb30] [c0000000003a1120] change_huge_pmd+0xe0/0x270 [c000003fe566bba0] [c000000000349278] change_protection_range+0xb88/0xe40 [c000003fe566bcf0] [c0000000003496c0] mprotect_fixup+0x140/0x340 [c000003fe566bdb0] [c000000000349a74] SyS_mprotect+0x1b4/0x350 [c000003fe566be30] [c00000000000b184] system_call+0x58/0x6c One way to avoid this is removing the smp-call. We can ensure that the timer always runs on one of the policy-cpus. If the timer gets migrated to a cpu outside the policy then re-queue it back on the policy->cpus. This way we can get rid of the smp-call which was being used to set the pstate on the policy->cpus. Fixes: 7bc54b652f13 (timers, cpufreq/powernv: Initialize the gpstate timer as pinned) Cc: [4.8+] Reported-by: Nicholas Piggin Reported-by: Pridhiviraj Paidipeddi Signed-off-by: Shilpasri G Bhat Acked-by: Nicholas Piggin Acked-by: Viresh Kumar Acked-by: Vaidyanathan Srinivasan --- Changes from V2: - Remove the check for active policy while requeing the migrated timer Changes from V1: - Remove smp_call in the pstate handler. drivers/cpufreq/powernv-cpufreq.c | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/drivers/cpufreq/powernv-cpufreq.c b/drivers/cpufreq/powernv-cpufreq.c index 71f8682..e368e1f 100644 --- a/drivers/cpufreq/powernv-cpufreq.c +++ b/drivers/cpufreq/powernv-cpufreq.c @@ -679,6 +679,16 @@ void gpstate_timer_handler(struct timer_list *t) if (!spin_trylock(&gpstates->gpstate_lock)) return; + /* + * If the timer has migrated to the different cpu then bring + * it back to one of the policy->cpus + */ + if (!cpumask_test_cpu(raw_smp_processor_id(), policy->cpus)) { + gpstates->timer.expires = jiffies + msecs_to_jiffies(1); + add_timer_on(&gpstates->timer, cpumask_first(policy->cpus)); + spin_unlock(&gpstates->gpstate_lock); + return; + } /* * If PMCR was last updated was using fast_swtich then @@ -718,10 +728,8 @@ void gpstate_timer_handler(struct timer_list *t) if (gpstate_idx != gpstates->last_lpstate_idx) queue_gpstate_timer(gpstates); + set_pstate(&freq_data); spin_unlock(&gpstates->gpstate_lock); - - /* Timer may get migrated to a different cpu on cpu hot unplug */ - smp_call_function_any(policy->cpus, set_pstate, &freq_data, 1); } /*