From patchwork Tue Mar 17 14:40:38 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 6032221 Return-Path: X-Original-To: patchwork-linux-arm@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 7C7BDBF90F for ; Tue, 17 Mar 2015 14:43:33 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 4FA7520412 for ; Tue, 17 Mar 2015 14:43:32 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.9]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 038D2203FB for ; Tue, 17 Mar 2015 14:43:31 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1YXsg6-0006SZ-4w; Tue, 17 Mar 2015 14:41:06 +0000 Received: from smtprelay0195.hostedemail.com ([216.40.44.195] helo=smtprelay.hostedemail.com) by bombadil.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1YXsg2-0006OB-4q for linux-arm-kernel@lists.infradead.org; Tue, 17 Mar 2015 14:41:03 +0000 Received: from filter.hostedemail.com (unknown [216.40.38.60]) by smtprelay02.hostedemail.com (Postfix) with ESMTP id 5987412BA13; Tue, 17 Mar 2015 14:40:40 +0000 (UTC) X-Session-Marker: 726F737465647440676F6F646D69732E6F7267 X-Spam-Summary: 50, 0, 0, , d41d8cd98f00b204, rostedt@goodmis.org, :::::::::::::::, RULES_HIT:41:152:355:379:541:800:960:967:973:988:989:1260:1263:1277:1311:1313:1314:1345:1359:1437:1513:1515:1516:1518:1521:1534:1542:1593:1594:1711:1730:1747:1777:1792:1981:2194:2199:2393:2525:2560:2563:2682:2685:2859:2893:2897:2902:2904:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3352:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:5007:6261:7903:8599:8660:8957:9025:9040:10004:10400:10848:10913:11026:11232:11473:11658:11914:12043:12296:12438:12517:12519:12555:12679:12783:13148:13161:13221:13229:13230:14096:14097:14394:21080:21088, 0, RBL:none, CacheIP:none, Bayesian:0.5, 0.5, 0.5, Netcheck:none, DomainCache:0, MSF:not bulk, SPF:fn, MSBL:0, DNSBL:none, Custom_rules:0:0:0 X-HE-Tag: fruit71_22c9af7101827 X-Filterd-Recvd-Size: 4142 Received: from gandalf.local.home (cpe-67-246-153-56.stny.res.rr.com [67.246.153.56]) (Authenticated sender: rostedt@goodmis.org) by omf12.hostedemail.com (Postfix) with ESMTPA; Tue, 17 Mar 2015 14:40:38 +0000 (UTC) Date: Tue, 17 Mar 2015 10:40:38 -0400 From: Steven Rostedt To: Christoph Lameter Subject: [PATCH v2] ring-buffer: Replace this_cpu_*() with __this_cpu_*() Message-ID: <20150317104038.312e73d1@gandalf.local.home> In-Reply-To: <20150317101113.32f5618a@gandalf.local.home> References: <20150316173154.537b80ee@gandalf.local.home> <20150317081341.0f9a8b4c@gandalf.local.home> <20150317101113.32f5618a@gandalf.local.home> X-Mailer: Claws Mail 3.11.1 (GTK+ 2.24.25; x86_64-pc-linux-gnu) MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20150317_074102_388486_67CB6844 X-CRM114-Status: GOOD ( 11.30 ) X-Spam-Score: -0.0 (/) Cc: Russell King , "H. Peter Anvin" , LKML , Uwe =?UTF-8?B?S2xlaW5lLUvDtm5pZw==?= , Thomas Gleixner , Ingo Molnar , linux-arm-kernel@lists.infradead.org X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_MED, T_RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP It has come to my attention that this_cpu_read/write are horrible on architectures other than x86. Worse yet, they actually disable preemption or interrupts! This caused some unexpected tracing results on ARM. 101.356868: preempt_count_add <-ring_buffer_lock_reserve 101.356870: preempt_count_sub <-ring_buffer_lock_reserve The ring_buffer_lock_reserve has recursion protection that requires accessing a per cpu variable. But since preempt_disable() is traced, it too got traced while accessing the variable that is suppose to prevent recursion like this. The generic version of this_cpu_read() and write() are: #define _this_cpu_generic_read(pcp) \ ({ typeof(pcp) ret__; \ preempt_disable(); \ ret__ = *this_cpu_ptr(&(pcp)); \ preempt_enable(); \ ret__; \ }) #define _this_cpu_generic_to_op(pcp, val, op) \ do { \ unsigned long flags; \ raw_local_irq_save(flags); \ *__this_cpu_ptr(&(pcp)) op val; \ raw_local_irq_restore(flags); \ } while (0) Which is unacceptable for locations that know they are within preempt disabled or interrupt disabled locations. Paul McKenney stated that __this_cpu_() versions produce much better code on other architectures than this_cpu_() does, if we know that the call is done in a preempt disabled location. I also changed the recursive_unlock() to use two local variables instead of accessing the per_cpu variable twice. Link: http://lkml.kernel.org/r/20150317114411.GE3589@linux.vnet.ibm.com Cc: stable@vger.kernel.org Cc: Christoph Lameter Reported-by: Uwe Kleine-König Signed-off-by: Steven Rostedt Acked-by: Christoph Lameter --- Changes since v1: Use __this_cpu_*() instead of this_cpu_ptr() diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index 5040d44fe5a3..363b9ec58aae 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -2679,7 +2679,7 @@ static DEFINE_PER_CPU(unsigned int, current_context); static __always_inline int trace_recursive_lock(void) { - unsigned int val = this_cpu_read(current_context); + unsigned int val = __this_cpu_read(current_context); int bit; if (in_interrupt()) { @@ -2696,18 +2696,19 @@ static __always_inline int trace_recursive_lock(void) return 1; val |= (1 << bit); - this_cpu_write(current_context, val); + __this_cpu_write(current_context, val); return 0; } static __always_inline void trace_recursive_unlock(void) { - unsigned int val = this_cpu_read(current_context); + unsigned int val = __this_cpu_read(current_context); + unsigned int val2; - val--; - val &= this_cpu_read(current_context); - this_cpu_write(current_context, val); + val2 = val - 1; + val &= val2; + __this_cpu_write(current_context, val); } #else