From patchwork Thu Jan 12 19:43:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 13099345 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BA502C61DB3 for ; Thu, 12 Jan 2023 20:15:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4FE268E0002; Thu, 12 Jan 2023 15:15:05 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 486FE8E0001; Thu, 12 Jan 2023 15:15:05 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 264A88E0002; Thu, 12 Jan 2023 15:15:05 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 110438E0001 for ; Thu, 12 Jan 2023 15:15:05 -0500 (EST) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id BF5AFA0770 for ; Thu, 12 Jan 2023 20:15:04 +0000 (UTC) X-FDA: 80347250928.11.69E5051 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf01.hostedemail.com (Postfix) with ESMTP id F242A4001C for ; Thu, 12 Jan 2023 20:15:02 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=EPZzE64m; spf=none (imf01.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=peterz@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1673554503; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:references:dkim-signature; bh=CWtXQ9ScESvHC863qag21jQX+KK5xw/Ija2/01ku3yk=; b=YmAwrkZiNC07TDglphIDbCXCudpAOCyQufjqQTvDxUzo1AaIEhiNOVx55JolMJ2bFhWMPq 1ESuqlrp0J0WKUOSXGp8gCr+O+UnOtpTJ80ADbUnNvy0D64q9hFmw8aHaP20KV1snw0hjW AQzKrhWK0QgtAzUj6/mA8XY+1+NiprA= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=EPZzE64m; spf=none (imf01.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=peterz@infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1673554503; a=rsa-sha256; cv=none; b=SCCbOwMKU0XQ4qpCqguBppHB4n+C3NnpHTFxnG3cYoy3EqwDFjLIUB6HNiKUFHm1uKksTo vtvZuQmGcF8Ddh4Ak/jr+K5fdLWcDcQX/zHNBxkdq3mLt0oI7uOo06nIt6AVP/SNQNLNHY dLHawR61dc+mBmeMoyv89e4/SqPdZ9o= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=CWtXQ9ScESvHC863qag21jQX+KK5xw/Ija2/01ku3yk=; b=EPZzE64m3oCXia+0wohNVHPJNI f4ZNUmyXPrJXWH7PL/wiKrgoxQtuCyfusu/y3sfTkworJKy8iifd1vPpHuP8cALb+9eexAF9i/bVo uujPfq/+tsg4qnvDB8OfG48pQXqBC3CNDcGcVnIOkcOltZ3//Ulq9Y9bpaRAZg+03QAYPkLcPO1s4 wONcCHCuHmLBFH0E7HMRRRMXZzsgmsm4f6hMvQuS12REPp2ZASTvrZVy/uINUGIc+agphKV1b+2yC JtrqWDPadPqhgKcTuFKtk4zf466ryZibze46GIPvhsYOTho1iX+RO3kAi0kCHINTSxmQEJ1fXTrIl AOBjoGlw==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1pG3hY-005Oce-9l; Thu, 12 Jan 2023 19:57:28 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 1E366303403; Thu, 12 Jan 2023 20:57:13 +0100 (CET) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id BAFC42CCF1F5C; Thu, 12 Jan 2023 20:57:07 +0100 (CET) Message-ID: <20230112195540.007918454@infradead.org> User-Agent: quilt/0.66 Date: Thu, 12 Jan 2023 20:43:25 +0100 From: Peter Zijlstra To: peterz@infradead.org Cc: richard.henderson@linaro.org, ink@jurassic.park.msu.ru, mattst88@gmail.com, vgupta@kernel.org, linux@armlinux.org.uk, nsekhar@ti.com, brgl@bgdev.pl, ulli.kroll@googlemail.com, linus.walleij@linaro.org, shawnguo@kernel.org, Sascha Hauer , kernel@pengutronix.de, festevam@gmail.com, linux-imx@nxp.com, tony@atomide.com, khilman@kernel.org, krzysztof.kozlowski@linaro.org, alim.akhtar@samsung.com, catalin.marinas@arm.com, will@kernel.org, guoren@kernel.org, bcain@quicinc.com, chenhuacai@kernel.org, kernel@xen0n.name, geert@linux-m68k.org, sammy@sammy.net, monstr@monstr.eu, tsbogend@alpha.franken.de, dinguyen@kernel.org, jonas@southpole.se, stefan.kristiansson@saunalahti.fi, shorne@gmail.com, James.Bottomley@HansenPartnership.com, deller@gmx.de, mpe@ellerman.id.au, npiggin@gmail.com, christophe.leroy@csgroup.eu, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, hca@linux.ibm.com, gor@linux.ibm.com, agordeev@linux.ibm.com, borntraeger@linux.ibm.com, svens@linux.ibm.com, ysato@users.sourceforge.jp, dalias@libc.org, davem@davemloft.net, richard@nod.at, anton.ivanov@cambridgegreys.com, johannes@sipsolutions.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, acme@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, namhyung@kernel.org, jgross@suse.com, srivatsa@csail.mit.edu, amakhalov@vmware.com, pv-drivers@vmware.com, boris.ostrovsky@oracle.com, chris@zankel.net, jcmvbkbc@gmail.com, rafael@kernel.org, lenb@kernel.org, pavel@ucw.cz, gregkh@linuxfoundation.org, mturquette@baylibre.com, sboyd@kernel.org, daniel.lezcano@linaro.org, lpieralisi@kernel.org, sudeep.holla@arm.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, anup@brainfault.org, thierry.reding@gmail.com, jonathanh@nvidia.com, jacob.jun.pan@linux.intel.com, atishp@atishpatra.org, Arnd Bergmann , yury.norov@gmail.com, andriy.shevchenko@linux.intel.com, linux@rasmusvillemoes.dk, dennis@kernel.org, tj@kernel.org, cl@linux.com, rostedt@goodmis.org, mhiramat@kernel.org, frederic@kernel.org, paulmck@kernel.org, pmladek@suse.com, senozhatsky@chromium.org, john.ogness@linutronix.de, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, vschneid@redhat.com, ryabinin.a.a@gmail.com, glider@google.com, andreyknvl@gmail.com, dvyukov@google.com, vincenzo.frascino@arm.com, Andrew Morton , jpoimboe@kernel.org, linux-alpha@vger.kernel.org, linux-kernel@vger.kernel.org, linux-snps-arc@lists.infradead.org, linux-omap@vger.kernel.org, linux-samsung-soc@vger.kernel.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, linux-ia64@vger.kernel.org, loongarch@lists.linux.dev, linux-m68k@lists.linux-m68k.org, linux-mips@vger.kernel.org, openrisc@lists.librecores.org, linux-parisc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-sh@vger.kernel.org, sparclinux@vger.kernel.org, linux-um@lists.infradead.org, linux-perf-users@vger.kernel.org, virtualization@lists.linux-foundation.org, linux-xtensa@linux-xtensa.org, linux-acpi@vger.kernel.org, linux-pm@vger.kernel.org, linux-clk@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-tegra@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-trace-kernel@vger.kernel.org, kasan-dev@googlegroups.com, "Rafael J. Wysocki" , Ulf Hansson Subject: [PATCH v3 11/51] cpuidle,omap4: Push RCU-idle into driver References: <20230112194314.845371875@infradead.org> MIME-Version: 1.0 X-Stat-Signature: 6mh4mwfu36obod963kgex87ab9n815qb X-Rspam-User: X-Rspamd-Queue-Id: F242A4001C X-Rspamd-Server: rspam06 X-HE-Tag: 1673554502-357533 X-HE-Meta: U2FsdGVkX18v0GcWE0/wOz0Xc3LOhbCFuzFjZSjBP24J2mBgSHKqqgie5foAAV3ubPNv7vvvVsedqABiLsaN4kfj5T4i0JxYV+f7qt7s2RLM3H0oJMm9w4HkCbqw88H5m/2O3wcm0xt+3Lr+E8TY83XS2xfS8L1QLRTunOnkELFSsFrPWmHGmfGJj5h1xX8csGrI5O34vNWgQanY5bU0vgjPIjlWiz1zzJV9tePZ8hxrJ+rpGd47NtkmSMobS2wSyCyU0oVtYDKHqfQmVS6797die29X9A8CWQ9NCVkGfpI+8qAiGlpZ7ahWBX7gltUAwvG9CYvjjuVjf1EqIJY3YXGYgZstjuQ4dKkrNLOsn3P1sWqmmYPc8oJ8sOqe4crHbTtbxzPHEpdDZGTg1cnXX54l9XOcQbb8pamuh8SyqgrLd6VlvnKDRPGW4t0WAMcdczX16tLSqEtGhYD4ENkSb8A/fyEFEs1TPexltmbqL8AkDtf9cCeFi96Fe/WbVmsshnZVDEJLTnkvpXjtuHIUvrTVJ5roEUii0cOLAycm0v1dyAleWnWkdVwL1H/jgRjDsETVj3tSh8KhtEUOBl1J/+GxOlPamaiKJjSUjiUxyV5GID8OZqMPe4oQIVwC5BkxJn//sIyI6I4x8dCoDqnNC8LyL0I0hvafXEedLXTbDh3lVmasOY7AVSQI68CCoBCaZya3ZQSesh2C+izconkJSCA9L7vWuPjIAHr2bHQY+rhs4RL03yFFfMaceIrJjpzUSaWzd8QlJl7WVdjGO0lYsXDsRuS0BKPbEOvPYcBTEuIbF+f3vfBxcolnE3f+92TcAFHXlEa6RKMvdZBdOiSnUzuovaDfsPFWVd4fMk7tZ+7lVZw2Q+hRZ4EP4+w+e0h0TgVpZ2eoRnEpd6BhQ45blWEVksNmO2KjdWLW5bZd3SgNyppT3EIjqO/0wkOJWsQ1zMgw72RY4xp9rakn0Na b77tk5JP tNqfasyX05wmwpY6lyeS+ZJcZ8n3ea2EpMcLy+XF5K+eX2RnVH9Et9y7srdvfAJ/tdD23qFtu111VbdElKm+EBef1QtlYWtZoLhY06tGuLBfOPIzPutURe4jraLWCYyyHZSgrQnK1WyzcxacHuHfF59anZWqiHKRPXMQRw9BAAu9krDI7jX0pzLZJHJfYXB9aNTww5B+9kId/iNpqIm8hbNNUGdSJOTAoEh+NtkUTFQ2H9BStK/ugf0jy8N3FCPWGUj7Rh6fhs5e1Giku3R8kl9aEc5gR7XH0csy4GsS14n/yW6UEv6phxdamY1PZ5Lb6rbn2Bxjgnto7Y2Sl3NMsMSeBTTbtqDPNesNY1JLiNO/GKT1Pi9RacqxZ6Xt3mQfXSVI6D3uV1rSCuKxC+XYd2tcDEq1r2H/cp/ieWidwTbrIj9Vw3HMJ6m77OZhxMB6Q8VtOrRwFBRFYny7IyI7Fu3J9jMNeFdtGMkiVZ9S5aVuqIfiFSFxWuUHcrGB/vMXcVtrVcNZXK3thfuGwA5mn3dQ6DLt8jMvLDtwlBqc1SO69WAJO9kGoXrbWPhGHUIiLbHUV22JnDHR8Q/NYF+GGNvtNf9lN3b/Y8ustB17vnAzSaNDVERikCH+jX22/auGvOhM883EsYH+t1ybpPJ1gb1po/9ZCfL+WZCQJogtJhtrY5PbwYyf7JVjrcNJrFV+FKUGKrN2zIGAAmS54c9ITrU9HOxozA/B6woHmxfTx//m9aA1BvsFkODUoTt+S+5L1S9Atxx1iW+nll+J8LbsWhiNgFAW2+4xMwc8RhK4Mq0GTlXBJjqzWzjpDGNHWHEezEUd33gXhE4qWknNbCiRkT8c1ZlBt7hZcVTmqvLF+BLDVqruGC3iHvsuixIUxkXYpNhIIM62TFwciuBxweu9wNFuOwdn+8ls8sY05urv3fjtRcbM22kmnIREifCpDtwZrphadFw6ZpMghVZgminrRXYSz/oaI Mi0WNOYU TUDyVMNxL13yhZFwAZqNVo5txNkChcoa9COuq9HkYfr0gkkqjYrBjc/SLS7H0sb/pkA3NdTHH6vSXpnt2LQUKVa86UlOenB+Mjz1+y7zwWq07owzAZW0ikjUSPYP+a26BNmKUeFoxxrGE0IsQWGbmeeZ4tXWaNxCSycQvzT4UQiSe0jPX0XhA7CZ2552U8lid0f06Fd/g9bu1AS13+B/EcfukpnlthSSjgDb17pyEuJ3TetqdEbvnoEjqHbJi3vt0D+GZmQLXZx5bmw+OpolzBCqTywSq9W7xVrAEIbQWcGl2MYKHw+f0HWIPud2GH4eDb9V/hLcodlnoZuOZjTU/f/BaXQoyJNzB51NYd6tP7llU/x54XePD55LnRRe0rQPnOgchr/B4NTSlr4IcH7GqjNojpSOV6RAuk6CZoUzYpgICo1py3qK4aFiDQwUC0z8vp459n7EUFse5NgzvfuZ3dmrJoQinyN79ZLEtFKqc3jCLw/EldbWlQWh0m8EJ6GOtGLIcqyt6pgaVEdkytyJigKsmSc8GmfHKkSgUlPq/iKVrPt7yPUnwNUrnKpFQu7ty2xQbGXIXv+fKn9d3IoELc9p4C5s4dC5jr9uZqAP1fp/dpn/Jj5sAGdxOrVYCroJ3SnE8HSHp7ULUut/BMp/Mq+s5svoIpwMkD99wuHBQlNCR3n+39E+pF7/mZS2+jDVPxpsLe5tN1xjlHc214WAIvHSx5kvB3hg/UrbvIN4U/fFkDi9XPhPc319ZoCU/E+g21pLxcbZAzFp8slH3bjAP3W3aSV8FR2V6ZCpacAM91Gh+bTJYpLrNsDhyIokulJA/9IpeAt39y+Tt3ytTEK7fpRdvSxnuDVTe0uCFbcXJvmKS+EzQZAAbCrtNvGrgwC6wf71fJTlaX1MK7AD2Mkg/MdV4eKaMKfq9ERXosP3QdsQ09GCSzvEsKUDZ/piilCJUpoDeeyp+bUcmq9mndGWmj34oh7MP OIF+GfOR 0vYlTb/CLznQ/lWsokPWhl/g1sBDX09ZoXW8N82MYdaqtpFsEl5eFQPHf+zplidXaWq26yzVq6YowvGQASQzmNs3RoPUk8SkG+YdHY+O5MhOO2WSmTIXTgUXnsNC6sLbF94H+Ue4qeiqGecfldn8f2i3zQ/ggoSW81ayc1wSuBqcOtKRulv2G1byo9ScNcuem1u9jkL09b4pWnIKljwt3/eojwAzaVspzqsJgXMUeLrqfnd404UGU0ZaM4sKUazUjvL8bHnv6apXVsaxvpc3GFvkCx3jU8WJNlJWPHtYgw70G0NntWMzJDAGO1sTSe0BZfMreZNDsBjQwO5NPFn/uypSBMbLubnuEKzs0cO0Q5T2xI8PBB/mZP2hXrCRoJ9DCObHQ553ufHdvO/FQ1HfOeX0E52MWAqlLIEIpaL8mDrcTQgztGgW66Ca/sNPR3HHFeM1g4w9Z6+TUXCGIz227XPQa9WakKesTk7AI/xZ7kTzz86KQo/I+dHv+kt8ebllxvq/WWTg2NqUOHNBPtPqqTL+JsOdsti+sKpvphVgR5KlBA6fA50sw8EAYYKEKuH5E33NJVhTFnILHpVBcEHqHZvSNmVku0ozLmBYZuDWlzov6J1RF6bP5aVggjkidigLVGWDZa8PLKy14zb3kYPWKIydkDkFpUCuWncSSaAe7q0tPaDQtnuXB6o6z/W+G1Thi72NnY/6h99EkZ3pblonZdXDaJr3vibdaz0V1a6R9jxMxnMlTphLSY2DOinGOUirKMxr8H9NsWLJq30DUx5OMBUl59IJzldxYm2+s51y3ScWt1P1NoyvBaY52aTh9WLJ4DQd54fUF6VbIN0dP2tUym6pA2gpNQwpfwikxIQZ/NKHYRee4EJRqVvwQgUYx4E1j/QFAJmwPKENQxh5Gsnl1Xj6Gsb2UVBdqvQsugvTsJYu5KhlLwE3ZPJXf7J/POCTYsxz+F/v5SrLq+n5Iecm5M6pjBR1+ ZCUs/PX0 FYSmeuEOhiNqnCGrTjT/O0WwTvCHcCOoOyeuYaW13qTK+aI/RZrjXrME/GmnEz8QH+2BxeLoGakOg2nsRwRg5ITOLpad9RnzlI+Fs2NFsB6A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Doing RCU-idle outside the driver, only to then temporarily enable it again, some *four* times, before going idle is daft. Notably three times explicitly using RCU_NONIDLE() and once implicitly through cpu_pm_*(). Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: Frederic Weisbecker Reviewed-by: Tony Lindgren Acked-by: Rafael J. Wysocki Tested-by: Tony Lindgren Tested-by: Ulf Hansson --- arch/arm/mach-omap2/cpuidle44xx.c | 29 ++++++++++++++++++----------- 1 file changed, 18 insertions(+), 11 deletions(-) --- a/arch/arm/mach-omap2/cpuidle44xx.c +++ b/arch/arm/mach-omap2/cpuidle44xx.c @@ -105,7 +105,9 @@ static int omap_enter_idle_smp(struct cp } raw_spin_unlock_irqrestore(&mpu_lock, flag); + ct_idle_enter(); omap4_enter_lowpower(dev->cpu, cx->cpu_state); + ct_idle_exit(); raw_spin_lock_irqsave(&mpu_lock, flag); if (cx->mpu_state_vote == num_online_cpus()) @@ -151,10 +153,10 @@ static int omap_enter_idle_coupled(struc (cx->mpu_logic_state == PWRDM_POWER_OFF); /* Enter broadcast mode for periodic timers */ - RCU_NONIDLE(tick_broadcast_enable()); + tick_broadcast_enable(); /* Enter broadcast mode for one-shot timers */ - RCU_NONIDLE(tick_broadcast_enter()); + tick_broadcast_enter(); /* * Call idle CPU PM enter notifier chain so that @@ -166,7 +168,7 @@ static int omap_enter_idle_coupled(struc if (dev->cpu == 0) { pwrdm_set_logic_retst(mpu_pd, cx->mpu_logic_state); - RCU_NONIDLE(omap_set_pwrdm_state(mpu_pd, cx->mpu_state)); + omap_set_pwrdm_state(mpu_pd, cx->mpu_state); /* * Call idle CPU cluster PM enter notifier chain @@ -178,14 +180,16 @@ static int omap_enter_idle_coupled(struc index = 0; cx = state_ptr + index; pwrdm_set_logic_retst(mpu_pd, cx->mpu_logic_state); - RCU_NONIDLE(omap_set_pwrdm_state(mpu_pd, cx->mpu_state)); + omap_set_pwrdm_state(mpu_pd, cx->mpu_state); mpuss_can_lose_context = 0; } } } + ct_idle_enter(); omap4_enter_lowpower(dev->cpu, cx->cpu_state); cpu_done[dev->cpu] = true; + ct_idle_exit(); /* Wakeup CPU1 only if it is not offlined */ if (dev->cpu == 0 && cpumask_test_cpu(1, cpu_online_mask)) { @@ -194,9 +198,9 @@ static int omap_enter_idle_coupled(struc mpuss_can_lose_context) gic_dist_disable(); - RCU_NONIDLE(clkdm_deny_idle(cpu_clkdm[1])); - RCU_NONIDLE(omap_set_pwrdm_state(cpu_pd[1], PWRDM_POWER_ON)); - RCU_NONIDLE(clkdm_allow_idle(cpu_clkdm[1])); + clkdm_deny_idle(cpu_clkdm[1]); + omap_set_pwrdm_state(cpu_pd[1], PWRDM_POWER_ON); + clkdm_allow_idle(cpu_clkdm[1]); if (IS_PM44XX_ERRATUM(PM_OMAP4_ROM_SMP_BOOT_ERRATUM_GICD) && mpuss_can_lose_context) { @@ -222,7 +226,7 @@ static int omap_enter_idle_coupled(struc cpu_pm_exit(); cpu_pm_out: - RCU_NONIDLE(tick_broadcast_exit()); + tick_broadcast_exit(); fail: cpuidle_coupled_parallel_barrier(dev, &abort_barrier); @@ -247,7 +251,8 @@ static struct cpuidle_driver omap4_idle_ /* C2 - CPU0 OFF + CPU1 OFF + MPU CSWR */ .exit_latency = 328 + 440, .target_residency = 960, - .flags = CPUIDLE_FLAG_COUPLED, + .flags = CPUIDLE_FLAG_COUPLED | + CPUIDLE_FLAG_RCU_IDLE, .enter = omap_enter_idle_coupled, .name = "C2", .desc = "CPUx OFF, MPUSS CSWR", @@ -256,7 +261,8 @@ static struct cpuidle_driver omap4_idle_ /* C3 - CPU0 OFF + CPU1 OFF + MPU OSWR */ .exit_latency = 460 + 518, .target_residency = 1100, - .flags = CPUIDLE_FLAG_COUPLED, + .flags = CPUIDLE_FLAG_COUPLED | + CPUIDLE_FLAG_RCU_IDLE, .enter = omap_enter_idle_coupled, .name = "C3", .desc = "CPUx OFF, MPUSS OSWR", @@ -282,7 +288,8 @@ static struct cpuidle_driver omap5_idle_ /* C2 - CPU0 RET + CPU1 RET + MPU CSWR */ .exit_latency = 48 + 60, .target_residency = 100, - .flags = CPUIDLE_FLAG_TIMER_STOP, + .flags = CPUIDLE_FLAG_TIMER_STOP | + CPUIDLE_FLAG_RCU_IDLE, .enter = omap_enter_idle_smp, .name = "C2", .desc = "CPUx CSWR, MPUSS CSWR",