From patchwork Tue Oct 1 21:53:05 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Burton X-Patchwork-Id: 11169845 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6150513B1 for ; Tue, 1 Oct 2019 21:55:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 3F14521A4C for ; Tue, 1 Oct 2019 21:55:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=wavecomp.com header.i=@wavecomp.com header.b="u6aFAbnB" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728010AbfJAVxL (ORCPT ); Tue, 1 Oct 2019 17:53:11 -0400 Received: from mail-eopbgr730112.outbound.protection.outlook.com ([40.107.73.112]:19088 "EHLO NAM05-DM3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727837AbfJAVxK (ORCPT ); Tue, 1 Oct 2019 17:53:10 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Dq54d+9cWDFeSOL298QY+OBknY20t24Yn+cwL/1icQwBl/SEDDs/dAWL4A9hxbu0DGF0uPTfCRxhupqI8eFZASGmbmnq6UclNxgbr/AciHVciDp12zHhuWtomjSpZUcf3bOGb37JcdrzGbLxy/LPp3cp9SQ+obfghfJLPKaur65QMqzBdaghNYv6I65GO1OmtsEmi0P4ehb9VqXzcrtLw545ZNAUwiH6XlIb4xOXHbcGvtSKQgI7YUaAvWcIduTJq61C5ocOdCUJEtpqp6vOiVBVfjM2080vHbZry5OkiGyGMpmagRu3FffLKLU8pXEePtoa1HwYf5GPbXwVwImE5Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9vI9gNCglkfhF5VFQxv/Hh5DetZ54lJmxxhImH/eGP0=; b=Jvvk3XNqS81dSYr2EqnqL9MOvEvj7SIMTrIPrB2pgrbkYSzfUVYBq3A2tpi2GFxNxu3YwnxLzw47C3I5h/oP06aRe1RuLpaBmIkjEuyv0hscqhoyiofiwCu3rk2pwAHGf5OB+gpk4F9LIOdwM8cFJgkS1JAV+f4I+soivBoisJN3E5RXWgfZ2YvzxZbQpSyzXM315nylzgTofD7613gBbhoewTRwdSPp2YN0tAJ2Z2t1rarmEVDBP6sRvvPqcJfHWi5NChH+NTgGXrLYbhljh8QfQ/xNuk/wYbmDfrlu5+LEAqDA/VevmSO8e5QmLC3M/FJBMSfil6EtsfytGWesUw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wavecomp.com; dmarc=pass action=none header.from=mips.com; dkim=pass header.d=mips.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wavecomp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9vI9gNCglkfhF5VFQxv/Hh5DetZ54lJmxxhImH/eGP0=; b=u6aFAbnB+7vGyD+9pMdpavGFRLOV/E2gGJfvHgEXsNoNssuZnNwx6h7DSL9ep685tmvqUPBsyFMgxB09j2bV8ayb0tbW7Fjlj5iTXoDtCsrxQah7NPDi2lVGMs3BUoGvtXKHxeBsYCY53j7s47rNsrXPiXbvhl2owEeUIrOMol8= Received: from MWHPR2201MB1277.namprd22.prod.outlook.com (10.172.60.12) by MWHPR2201MB1213.namprd22.prod.outlook.com (10.172.61.148) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2305.20; Tue, 1 Oct 2019 21:53:06 +0000 Received: from MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481]) by MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481%2]) with mapi id 15.20.2305.022; Tue, 1 Oct 2019 21:53:06 +0000 From: Paul Burton To: "linux-mips@vger.kernel.org" CC: Huacai Chen , Jiaxun Yang , "linux-kernel@vger.kernel.org" , Paul Burton Subject: [PATCH v2 01/36] MIPS: Unify sc beqz definition Thread-Topic: [PATCH v2 01/36] MIPS: Unify sc beqz definition Thread-Index: AQHVeKKbqA5jK9SbWEK9RRYhOQ0O0w== Date: Tue, 1 Oct 2019 21:53:05 +0000 Message-ID: <20191001215249.4157062-2-paul.burton@mips.com> References: <20191001215249.4157062-1-paul.burton@mips.com> In-Reply-To: <20191001215249.4157062-1-paul.burton@mips.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BYAPR07CA0093.namprd07.prod.outlook.com (2603:10b6:a03:12b::34) To MWHPR2201MB1277.namprd22.prod.outlook.com (2603:10b6:301:18::12) authentication-results: spf=none (sender IP is ) smtp.mailfrom=pburton@wavecomp.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.23.0 x-originating-ip: [12.94.197.246] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 6fda94e5-9dc2-4aec-8c45-08d746b9bd24 x-ms-traffictypediagnostic: MWHPR2201MB1213: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:4502; x-forefront-prvs: 0177904E6B x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(346002)(396003)(366004)(376002)(136003)(39840400004)(199004)(189003)(6512007)(5640700003)(54906003)(2501003)(3846002)(6486002)(5660300002)(6436002)(66066001)(6116002)(1076003)(50226002)(2906002)(76176011)(52116002)(6506007)(107886003)(2351001)(4326008)(6916009)(102836004)(71190400001)(8936002)(386003)(25786009)(36756003)(44832011)(8676002)(71200400001)(81166006)(81156014)(186003)(256004)(11346002)(14444005)(2616005)(7736002)(316002)(42882007)(305945005)(99286004)(66446008)(64756008)(66556008)(478600001)(66946007)(66476007)(486006)(446003)(476003)(14454004)(26005);DIR:OUT;SFP:1102;SCL:1;SRVR:MWHPR2201MB1213;H:MWHPR2201MB1277.namprd22.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; received-spf: None (protection.outlook.com: wavecomp.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: RTsIeGHEJOf0i3VoV0EpuIzPujTXvd11t6EV21zEhSSGON4HhHAcmoxKv7PrxJsT2hmkVueAzNSSQHSmINRld2clKRLiYDR9GZsUYL/AaOnBNFAx5fXf2g2uM0zxKe1XUpRxoMuT6BiX3F1eQuOTygsb8ijpqO4pWqDPeOo61n+20lUwgQNgmU8ZmMaFYbzCoQOm90kA8YtVx91F8z+SB9nXHQsjlqK6nmkkxEUPoSTyPYYE78MJiSUPlRXfiqWfg3LCceIsJ/Jh9F8wOE9BVuBhjJ0en07LaR0Ri2ezbnmydhNEOyz4g4DAIuUFOw9aQc0mWFMU46DauAbRjoevXg145ZTYsOyVJz3tkvOE/fZIOGjduPyrY1QWfBuQqXgkte34bjoYQ0xGvmjmfMVqDCqNHL3592WKKQFrBVQOA6c= MIME-Version: 1.0 X-OriginatorOrg: mips.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6fda94e5-9dc2-4aec-8c45-08d746b9bd24 X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Oct 2019 21:53:05.9304 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 463607d3-1db3-40a0-8a29-970c56230104 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: xbqATM/wk+P06dP9xxd47OaCc0chIjw/vFI2uYijkGVH5nk2S1b3HeHaX0cz67ch90cJzHkPL8Y5Rf4UAqfJdQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR2201MB1213 Sender: linux-mips-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-mips@vger.kernel.org We currently duplicate the definition of __scbeqz in asm/atomic.h & asm/cmpxchg.h. Move it to asm/llsc.h & rename it to __SC_BEQZ to fit better with the existing __SC macro provided there. We include a tab in the string in order to avoid the need for users to indent code any further to include whitespace of their own after the instruction mnemonic. Signed-off-by: Paul Burton --- Changes in v2: None arch/mips/include/asm/atomic.h | 28 +++++++++------------------- arch/mips/include/asm/cmpxchg.h | 20 ++++---------------- arch/mips/include/asm/llsc.h | 11 +++++++++++ 3 files changed, 24 insertions(+), 35 deletions(-) diff --git a/arch/mips/include/asm/atomic.h b/arch/mips/include/asm/atomic.h index bb8658cc7f12..7578c807ef98 100644 --- a/arch/mips/include/asm/atomic.h +++ b/arch/mips/include/asm/atomic.h @@ -20,19 +20,9 @@ #include #include #include +#include #include -/* - * Using a branch-likely instruction to check the result of an sc instruction - * works around a bug present in R10000 CPUs prior to revision 3.0 that could - * cause ll-sc sequences to execute non-atomically. - */ -#if R10000_LLSC_WAR -# define __scbeqz "beqzl" -#else -# define __scbeqz "beqz" -#endif - #define ATOMIC_INIT(i) { (i) } /* @@ -65,7 +55,7 @@ static __inline__ void atomic_##op(int i, atomic_t * v) \ "1: ll %0, %1 # atomic_" #op " \n" \ " " #asm_op " %0, %2 \n" \ " sc %0, %1 \n" \ - "\t" __scbeqz " %0, 1b \n" \ + "\t" __SC_BEQZ "%0, 1b \n" \ " .set pop \n" \ : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (v->counter) \ : "Ir" (i) : __LLSC_CLOBBER); \ @@ -93,7 +83,7 @@ static __inline__ int atomic_##op##_return_relaxed(int i, atomic_t * v) \ "1: ll %1, %2 # atomic_" #op "_return \n" \ " " #asm_op " %0, %1, %3 \n" \ " sc %0, %2 \n" \ - "\t" __scbeqz " %0, 1b \n" \ + "\t" __SC_BEQZ "%0, 1b \n" \ " " #asm_op " %0, %1, %3 \n" \ " .set pop \n" \ : "=&r" (result), "=&r" (temp), \ @@ -127,7 +117,7 @@ static __inline__ int atomic_fetch_##op##_relaxed(int i, atomic_t * v) \ "1: ll %1, %2 # atomic_fetch_" #op " \n" \ " " #asm_op " %0, %1, %3 \n" \ " sc %0, %2 \n" \ - "\t" __scbeqz " %0, 1b \n" \ + "\t" __SC_BEQZ "%0, 1b \n" \ " .set pop \n" \ " move %0, %1 \n" \ : "=&r" (result), "=&r" (temp), \ @@ -205,7 +195,7 @@ static __inline__ int atomic_sub_if_positive(int i, atomic_t * v) " .set push \n" " .set "MIPS_ISA_LEVEL" \n" " sc %1, %2 \n" - "\t" __scbeqz " %1, 1b \n" + "\t" __SC_BEQZ "%1, 1b \n" "2: \n" " .set pop \n" : "=&r" (result), "=&r" (temp), @@ -267,7 +257,7 @@ static __inline__ void atomic64_##op(s64 i, atomic64_t * v) \ "1: lld %0, %1 # atomic64_" #op " \n" \ " " #asm_op " %0, %2 \n" \ " scd %0, %1 \n" \ - "\t" __scbeqz " %0, 1b \n" \ + "\t" __SC_BEQZ "%0, 1b \n" \ " .set pop \n" \ : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (v->counter) \ : "Ir" (i) : __LLSC_CLOBBER); \ @@ -295,7 +285,7 @@ static __inline__ s64 atomic64_##op##_return_relaxed(s64 i, atomic64_t * v) \ "1: lld %1, %2 # atomic64_" #op "_return\n" \ " " #asm_op " %0, %1, %3 \n" \ " scd %0, %2 \n" \ - "\t" __scbeqz " %0, 1b \n" \ + "\t" __SC_BEQZ "%0, 1b \n" \ " " #asm_op " %0, %1, %3 \n" \ " .set pop \n" \ : "=&r" (result), "=&r" (temp), \ @@ -329,7 +319,7 @@ static __inline__ s64 atomic64_fetch_##op##_relaxed(s64 i, atomic64_t * v) \ "1: lld %1, %2 # atomic64_fetch_" #op "\n" \ " " #asm_op " %0, %1, %3 \n" \ " scd %0, %2 \n" \ - "\t" __scbeqz " %0, 1b \n" \ + "\t" __SC_BEQZ "%0, 1b \n" \ " move %0, %1 \n" \ " .set pop \n" \ : "=&r" (result), "=&r" (temp), \ @@ -404,7 +394,7 @@ static __inline__ s64 atomic64_sub_if_positive(s64 i, atomic64_t * v) " move %1, %0 \n" " bltz %0, 1f \n" " scd %1, %2 \n" - "\t" __scbeqz " %1, 1b \n" + "\t" __SC_BEQZ "%1, 1b \n" "1: \n" " .set pop \n" : "=&r" (result), "=&r" (temp), diff --git a/arch/mips/include/asm/cmpxchg.h b/arch/mips/include/asm/cmpxchg.h index 79bf34efbc04..5d3f0e3513b4 100644 --- a/arch/mips/include/asm/cmpxchg.h +++ b/arch/mips/include/asm/cmpxchg.h @@ -11,19 +11,9 @@ #include #include #include +#include #include -/* - * Using a branch-likely instruction to check the result of an sc instruction - * works around a bug present in R10000 CPUs prior to revision 3.0 that could - * cause ll-sc sequences to execute non-atomically. - */ -#if R10000_LLSC_WAR -# define __scbeqz "beqzl" -#else -# define __scbeqz "beqz" -#endif - /* * These functions doesn't exist, so if they are called you'll either: * @@ -57,7 +47,7 @@ extern unsigned long __xchg_called_with_bad_pointer(void) " move $1, %z3 \n" \ " .set " MIPS_ISA_ARCH_LEVEL " \n" \ " " st " $1, %1 \n" \ - "\t" __scbeqz " $1, 1b \n" \ + "\t" __SC_BEQZ "$1, 1b \n" \ " .set pop \n" \ : "=&r" (__ret), "=" GCC_OFF_SMALL_ASM() (*m) \ : GCC_OFF_SMALL_ASM() (*m), "Jr" (val) \ @@ -130,7 +120,7 @@ static inline unsigned long __xchg(volatile void *ptr, unsigned long x, " move $1, %z4 \n" \ " .set "MIPS_ISA_ARCH_LEVEL" \n" \ " " st " $1, %1 \n" \ - "\t" __scbeqz " $1, 1b \n" \ + "\t" __SC_BEQZ "$1, 1b \n" \ " .set pop \n" \ "2: \n" \ : "=&r" (__ret), "=" GCC_OFF_SMALL_ASM() (*m) \ @@ -268,7 +258,7 @@ static inline unsigned long __cmpxchg64(volatile void *ptr, /* Attempt to store new at ptr */ " scd %L1, %2 \n" /* If we failed, loop! */ - "\t" __scbeqz " %L1, 1b \n" + "\t" __SC_BEQZ "%L1, 1b \n" " .set pop \n" "2: \n" : "=&r"(ret), @@ -311,6 +301,4 @@ static inline unsigned long __cmpxchg64(volatile void *ptr, # endif /* !CONFIG_SMP */ #endif /* !CONFIG_64BIT */ -#undef __scbeqz - #endif /* __ASM_CMPXCHG_H */ diff --git a/arch/mips/include/asm/llsc.h b/arch/mips/include/asm/llsc.h index c6d17d171147..9b19f38562ac 100644 --- a/arch/mips/include/asm/llsc.h +++ b/arch/mips/include/asm/llsc.h @@ -25,4 +25,15 @@ #define __EXT "dext " #endif +/* + * Using a branch-likely instruction to check the result of an sc instruction + * works around a bug present in R10000 CPUs prior to revision 3.0 that could + * cause ll-sc sequences to execute non-atomically. + */ +#if R10000_LLSC_WAR +# define __SC_BEQZ "beqzl " +#else +# define __SC_BEQZ "beqz " +#endif + #endif /* __ASM_LLSC_H */ From patchwork Tue Oct 1 21:53:06 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Burton X-Patchwork-Id: 11169843 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DF1D91709 for ; Tue, 1 Oct 2019 21:55:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id BD6BC21A4A for ; Tue, 1 Oct 2019 21:55:31 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=wavecomp.com header.i=@wavecomp.com header.b="ltRIJU7g" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728028AbfJAVxN (ORCPT ); Tue, 1 Oct 2019 17:53:13 -0400 Received: from mail-eopbgr730120.outbound.protection.outlook.com ([40.107.73.120]:2512 "EHLO NAM05-DM3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728009AbfJAVxM (ORCPT ); Tue, 1 Oct 2019 17:53:12 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VWndTvRWe5nLMIb6lb3jh6G6dRQTu087WbnJAf6YGQqLAajloQwLbfk4PL2BzN79ItA46wDncSknfdyhJ1ZuwsUo1W26wIS6lyO/t/2Ep324IB+lL/Mxu6DQefMZEPw/Ve/1B/a7ce/QVuwFA+xR6IrQEHtmLU/tM/0i+92JHkNeKR+dM/jbs9NDDusj2uOl+orDzVfl45mlzE0BvkD1YxpYT9uPne7d4QC6q+V/e/+BitnQgGtvDDlQMWSmppOR8v47W65zRgeeviyFx9VHUgpfXiBptK9ov9W1w8kKXaAS+UamaWFAoAN0DwtUkntQH+hD87WBZgXtU98K6VG9mg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NNUO8WeJjpFVl6pijXSPOS4DqYCVTGhmAbqOX/Q9qoM=; b=NdMM0kaq+bhJ8xDkXzuasf+eq871xcbvTuPz2FYRMhOaasrlGSTNU2L/oW4YyZ4WyGHGJW/QZrvKBwA+7wAB9wBBhbhG4mZXwLeVqdjEp8cx5eloTRI5y7jnYPsymoZApyZjZznJ8OteCjJB0EyjTrBPABAxyY+EO2CgoTItOezki1WMohzft+lWwv7HZ+p6GlQ1jQkWx8nmY2bsBXPm/3IieDb+bmuzafT1K4yajCoaypWXu7uv5tf8vN194C4mW2Rd3Inoqz58MsQM8z/YKOpoJioQOKPscmW4+1TXAxJhm6Ln1AU5jVmCptm7tueMrT3KTJIygvHoipuHwTqA/g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wavecomp.com; dmarc=pass action=none header.from=mips.com; dkim=pass header.d=mips.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wavecomp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NNUO8WeJjpFVl6pijXSPOS4DqYCVTGhmAbqOX/Q9qoM=; b=ltRIJU7ghz2bXFNUWhVwp/+5BS96ydNkfVhcJazS/QlKHGwFUZ/S7Dqlehyo3E1aIXoyr3Sxa10FAXNKaHB9NfjnNi/e+Xd4de0RsFpA0KUXfwrmecUxPlAlQt59d/fCnZ6hW/x02zMZ5QhHpsa8eYWs/NaS7lvGSFOopSjiS4U= Received: from MWHPR2201MB1277.namprd22.prod.outlook.com (10.172.60.12) by MWHPR2201MB1213.namprd22.prod.outlook.com (10.172.61.148) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2305.20; Tue, 1 Oct 2019 21:53:07 +0000 Received: from MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481]) by MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481%2]) with mapi id 15.20.2305.022; Tue, 1 Oct 2019 21:53:07 +0000 From: Paul Burton To: "linux-mips@vger.kernel.org" CC: Huacai Chen , Jiaxun Yang , "linux-kernel@vger.kernel.org" , Paul Burton Subject: [PATCH v2 02/36] MIPS: Use compact branch for LL/SC loops on MIPSr6+ Thread-Topic: [PATCH v2 02/36] MIPS: Use compact branch for LL/SC loops on MIPSr6+ Thread-Index: AQHVeKKb9mKJgp21ekqbovI5ofLUZw== Date: Tue, 1 Oct 2019 21:53:06 +0000 Message-ID: <20191001215249.4157062-3-paul.burton@mips.com> References: <20191001215249.4157062-1-paul.burton@mips.com> In-Reply-To: <20191001215249.4157062-1-paul.burton@mips.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BYAPR07CA0093.namprd07.prod.outlook.com (2603:10b6:a03:12b::34) To MWHPR2201MB1277.namprd22.prod.outlook.com (2603:10b6:301:18::12) authentication-results: spf=none (sender IP is ) smtp.mailfrom=pburton@wavecomp.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.23.0 x-originating-ip: [12.94.197.246] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 864ad0ae-5e78-49d9-52f4-08d746b9be09 x-ms-traffictypediagnostic: MWHPR2201MB1213: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:4714; x-forefront-prvs: 0177904E6B x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(346002)(396003)(366004)(376002)(136003)(39840400004)(199004)(189003)(6512007)(5640700003)(54906003)(2501003)(4744005)(3846002)(6486002)(5660300002)(6436002)(66066001)(6116002)(1076003)(50226002)(2906002)(76176011)(52116002)(6506007)(107886003)(2351001)(4326008)(6916009)(102836004)(71190400001)(8936002)(386003)(25786009)(36756003)(44832011)(8676002)(71200400001)(81166006)(81156014)(186003)(256004)(11346002)(2616005)(7736002)(316002)(42882007)(305945005)(99286004)(66446008)(64756008)(66556008)(478600001)(66946007)(66476007)(486006)(446003)(476003)(14454004)(26005);DIR:OUT;SFP:1102;SCL:1;SRVR:MWHPR2201MB1213;H:MWHPR2201MB1277.namprd22.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; received-spf: None (protection.outlook.com: wavecomp.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: WxNXCvplV5lKm3I0atZNFXktHkQ6yyiy2NYXGJZI4o+TvkWFpsZwqrVr/iKeF4B7ItR0B/Bgx2YXNk56HH7CxN7CKbIXonUu5J+sfmEVoduRmrlZ2wQB9qDftIor1y+VimWKJP5QQA+K/TMD5BJf4oXwkSyD2xwOXaX78u7mfvNaECkprpQshWyu59ZajBzeO2SNZp8IOUVmJZSj2cIGbd1sMez/S2SBwTsF2MkhmJwOin86LZNTgfvlylhYAFl+aiaslkp2c/se5bMP20y6nP8gZvtEi9vHGqxUKP4SsvvETwsHaXKRBe+kJifR489tlR833MjjOhoFsQRT6nNxH4/q1jsSg8TfPkXb0ggWg7Rj/6OAsrU2XAL5xqAZteWMRr9ehEROU3XIXwjgtOvsNsDa26oUUMUT6kppG7+PyVY= MIME-Version: 1.0 X-OriginatorOrg: mips.com X-MS-Exchange-CrossTenant-Network-Message-Id: 864ad0ae-5e78-49d9-52f4-08d746b9be09 X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Oct 2019 21:53:06.9079 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 463607d3-1db3-40a0-8a29-970c56230104 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: Ke5PqEn2hiVAAOymUP4pDXc9Xqddg46Ch+uFeQsoyvZunaT9LKcAUXyuCCCQbKMqF2op0RpOpWi20w4h4rkDow== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR2201MB1213 Sender: linux-mips-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-mips@vger.kernel.org When targeting MIPSr6 or higher make use of a compact branch in LL/SC loops, preventing the insertion of a delay slot nop that only serves to waste space. Signed-off-by: Paul Burton --- Changes in v2: None arch/mips/include/asm/llsc.h | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/arch/mips/include/asm/llsc.h b/arch/mips/include/asm/llsc.h index 9b19f38562ac..d240a4a2d1c4 100644 --- a/arch/mips/include/asm/llsc.h +++ b/arch/mips/include/asm/llsc.h @@ -9,6 +9,8 @@ #ifndef __ASM_LLSC_H #define __ASM_LLSC_H +#include + #if _MIPS_SZLONG == 32 #define SZLONG_LOG 5 #define SZLONG_MASK 31UL @@ -32,6 +34,8 @@ */ #if R10000_LLSC_WAR # define __SC_BEQZ "beqzl " +#elif MIPS_ISA_REV >= 6 +# define __SC_BEQZ "beqzc " #else # define __SC_BEQZ "beqz " #endif From patchwork Tue Oct 1 21:53:07 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Burton X-Patchwork-Id: 11169841 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B3FD91599 for ; Tue, 1 Oct 2019 21:55:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 71EF721A4A for ; Tue, 1 Oct 2019 21:55:31 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=wavecomp.com header.i=@wavecomp.com header.b="OeSma3KS" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728033AbfJAVz1 (ORCPT ); Tue, 1 Oct 2019 17:55:27 -0400 Received: from mail-eopbgr730120.outbound.protection.outlook.com ([40.107.73.120]:2512 "EHLO NAM05-DM3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727837AbfJAVxO (ORCPT ); Tue, 1 Oct 2019 17:53:14 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=N9nqUkVbTQAMeP/EqPbt4ZCc7ntZuBhZ605MRf8TUdfOFvHdCT0wTHCSGSYwIIsLoOfxV6LGzxp4qIviz7VQD5nbDoYCZg0PNzueUf4wXZiD0FNhwVnuIIZ/eVShJqO+p2Fl3+uxGapkRc/42VMQ1AzMwQWo3GFPPl7/tga/qaUGka23zIoqQPBnKSwI0/8L75aCj1HC7y1XUC1YpZ6WOQsQyXAjf1DCKhATK5hVyGzm97ROi/Vzk7SsFfy3sBCVQ3RjyRTxQizBSFyne9YSEWE53kIcLX6KloUjxVITYXTkcdW46RqzHG6pV9+CqMPeqfEV/M4PR38bqioeWGhFVA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ZijVaOdshKvJY5MXwlhr+TyDa9K427v6WimtCe1uBuo=; b=RiyMiM7rIa2p4Wi8TOP0oUALZCyZHVGRJCL8ogJZf418o6dSM3pY6qZF4228jBA2Ysm9u6I1ziflKAtFYhFWwI2n0VC7PPOfFUVSArH8RCXZs4Dx5QhQw0zIuzI6xuyjL+AXUHC7Z1JlNnhV9sK0kzm5dxiYMYCLvZGE0N9qCSHt5XlHtonHGqkyfvA3urOCP93tYDoCG/kP/mo3gYZy1T33k9qNY/cj1aLZTLqGTLyuee0XLJVHCb/KwgIibCumfp4vVNMzPQqC8EkwFo5vkNq9xbiQG8DGaoZJgOtO2RvmDuKy30C3dPyYHa3usAzVx5pmmX+kRdobLSCuuBsAfA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wavecomp.com; dmarc=pass action=none header.from=mips.com; dkim=pass header.d=mips.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wavecomp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ZijVaOdshKvJY5MXwlhr+TyDa9K427v6WimtCe1uBuo=; b=OeSma3KSyS3BEoIKfDXcqQi+SbWMDIHGuWnemR/rrDe+bn8aTMpMUmGsRFvbCZ2QeAOFjkUXgADuUy93FrKAZUdksVi5mPP6tg+Y9PV/V4B3DrM63UtmQbf2Y7wq/qBUVdVLVbpW8+vopO0K2GazRUL3oJG8vssOXbb6WP5xnFs= Received: from MWHPR2201MB1277.namprd22.prod.outlook.com (10.172.60.12) by MWHPR2201MB1213.namprd22.prod.outlook.com (10.172.61.148) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2305.20; Tue, 1 Oct 2019 21:53:07 +0000 Received: from MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481]) by MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481%2]) with mapi id 15.20.2305.022; Tue, 1 Oct 2019 21:53:07 +0000 From: Paul Burton To: "linux-mips@vger.kernel.org" CC: Huacai Chen , Jiaxun Yang , "linux-kernel@vger.kernel.org" , Paul Burton Subject: [PATCH v2 03/36] MIPS: barrier: Add __SYNC() infrastructure Thread-Topic: [PATCH v2 03/36] MIPS: barrier: Add __SYNC() infrastructure Thread-Index: AQHVeKKc6Rl/g0nKvEmoZDEKJD6FHg== Date: Tue, 1 Oct 2019 21:53:07 +0000 Message-ID: <20191001215249.4157062-4-paul.burton@mips.com> References: <20191001215249.4157062-1-paul.burton@mips.com> In-Reply-To: <20191001215249.4157062-1-paul.burton@mips.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BYAPR07CA0093.namprd07.prod.outlook.com (2603:10b6:a03:12b::34) To MWHPR2201MB1277.namprd22.prod.outlook.com (2603:10b6:301:18::12) authentication-results: spf=none (sender IP is ) smtp.mailfrom=pburton@wavecomp.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.23.0 x-originating-ip: [12.94.197.246] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 8fd0d350-ea89-4bd5-1dca-08d746b9be8b x-ms-traffictypediagnostic: MWHPR2201MB1213: x-ms-exchange-purlcount: 1 x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-forefront-prvs: 0177904E6B x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(346002)(396003)(366004)(376002)(136003)(39840400004)(199004)(189003)(6306002)(6512007)(5640700003)(54906003)(2501003)(3846002)(6486002)(5660300002)(30864003)(6436002)(66066001)(6116002)(1076003)(50226002)(2906002)(76176011)(52116002)(6506007)(107886003)(2351001)(4326008)(6916009)(102836004)(71190400001)(8936002)(386003)(25786009)(36756003)(44832011)(8676002)(71200400001)(81166006)(81156014)(186003)(256004)(11346002)(14444005)(2616005)(7736002)(316002)(966005)(42882007)(305945005)(99286004)(66446008)(64756008)(66556008)(478600001)(66946007)(66476007)(486006)(446003)(476003)(14454004)(26005)(579004);DIR:OUT;SFP:1102;SCL:1;SRVR:MWHPR2201MB1213;H:MWHPR2201MB1277.namprd22.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; received-spf: None (protection.outlook.com: wavecomp.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: W2Z+Nt6gpkpW9Pe4FN27YZRYO4LNbqz1n08LLuZwxkGyXHQugmypk9o8Hy4FHLpN+Y/Phk5QH+gF3QMtVj1dCjsNJOiKKE5j2UUXatMwnsdfUvi8XL7cwOIC2UX+rMkz5HplEz4FVWgTBekRFLR/NKV9Q5eQon3191xLM7/4StxKRCJvC5BBCYM1+onxcZX+fZWa+Ow9/1ysBWw3D6V4HPWg65NC4qwmZkPJQO9I5BGxx6htGRtZvz0Kc+32AdbwCf0sw0ZECmXWEFvltdu4DHhJKCdJy32nA5GDOLLlaMMBV41HVIKrIQiThOEAprVOmaB2VL2wsIyO+Nak4oQuRBMZHwWGhRISwaoTPCTGFsmH/hakPlk1XKjmqtJfhibucy9Es5npGNTUuTSjEqSMmQjym0ddZ47H/TPFTsK4R42E+PKffwNi6ePLP6bcgnwI4Hfz9lNWghw1AyvoyRx8uA== MIME-Version: 1.0 X-OriginatorOrg: mips.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8fd0d350-ea89-4bd5-1dca-08d746b9be8b X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Oct 2019 21:53:07.8615 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 463607d3-1db3-40a0-8a29-970c56230104 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: ZCjK89sGPdvAo6U8T+qcnZ3hS7fbKWcb+LGONNtJNUdURJq6I+V+xCqYjc3DR4P6oF4LKNriM7+iw/BauG1CZQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR2201MB1213 Sender: linux-mips-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-mips@vger.kernel.org Introduce an asm/sync.h header which provides infrastructure that can be used to generate sync instructions of various types, and for various reasons. For example if we need a sync instruction that provides a full completion barrier but only on systems which have weak memory ordering, we can generate the appropriate assembly code using: __SYNC(full, weak_ordering) When the kernel is configured to run on systems with weak memory ordering (ie. CONFIG_WEAK_ORDERING is selected) we'll emit a sync instruction. When the kernel is configured to run on systems with strong memory ordering (ie. CONFIG_WEAK_ORDERING is not selected) we'll emit nothing. The caller doesn't need to know which happened - it simply says what it needs & when, with no concern for checking the kernel configuration. There are some scenarios in which we may want to emit code only when we *didn't* emit a sync instruction. For example, some Loongson3 CPUs suffer from a bug that requires us to emit a sync instruction prior to each ll instruction (enabled by CONFIG_CPU_LOONGSON3_WORKAROUNDS). In cases where this bug workaround is enabled, it's wasteful to then have more generic code emit another sync instruction to provide barriers we need in general. A __SYNC_ELSE() macro allows for this, providing an extra argument that contains code to be assembled only in cases where the sync instruction was not emitted. For example if we have a scenario in which we generally want to emit a release barrier but for affected Loongson3 configurations upgrade that to a full completion barrier, we can do that like so: __SYNC_ELSE(full, loongson3_war, __SYNC(rl, always)) The assembly generated by these macros can be used either as inline assembly or in assembly source files. Differing types of sync as provided by MIPSr6 are defined, but currently they all generate a full completion barrier except in kernels configured for Cavium Octeon systems. There the wmb sync-type is used, and rmb syncs are omitted, as has been the case since commit 6b07d38aaa52 ("MIPS: Octeon: Use optimized memory barrier primitives."). Using __SYNC() with the wmb or rmb types will abstract away the Octeon specific behavior and allow us to later clean up asm/barrier.h code that currently includes a plethora of #ifdef's. Signed-off-by: Paul Burton --- Changes in v2: None arch/mips/include/asm/barrier.h | 113 +---------------- arch/mips/include/asm/sync.h | 207 ++++++++++++++++++++++++++++++++ arch/mips/kernel/pm-cps.c | 20 +-- 3 files changed, 219 insertions(+), 121 deletions(-) create mode 100644 arch/mips/include/asm/sync.h diff --git a/arch/mips/include/asm/barrier.h b/arch/mips/include/asm/barrier.h index 9228f7386220..5ad39bfd3b6d 100644 --- a/arch/mips/include/asm/barrier.h +++ b/arch/mips/include/asm/barrier.h @@ -9,116 +9,7 @@ #define __ASM_BARRIER_H #include - -/* - * Sync types defined by the MIPS architecture (document MD00087 table 6.5) - * These values are used with the sync instruction to perform memory barriers. - * Types of ordering guarantees available through the SYNC instruction: - * - Completion Barriers - * - Ordering Barriers - * As compared to the completion barrier, the ordering barrier is a - * lighter-weight operation as it does not require the specified instructions - * before the SYNC to be already completed. Instead it only requires that those - * specified instructions which are subsequent to the SYNC in the instruction - * stream are never re-ordered for processing ahead of the specified - * instructions which are before the SYNC in the instruction stream. - * This potentially reduces how many cycles the barrier instruction must stall - * before it completes. - * Implementations that do not use any of the non-zero values of stype to define - * different barriers, such as ordering barriers, must make those stype values - * act the same as stype zero. - */ - -/* - * Completion barriers: - * - Every synchronizable specified memory instruction (loads or stores or both) - * that occurs in the instruction stream before the SYNC instruction must be - * already globally performed before any synchronizable specified memory - * instructions that occur after the SYNC are allowed to be performed, with - * respect to any other processor or coherent I/O module. - * - * - The barrier does not guarantee the order in which instruction fetches are - * performed. - * - * - A stype value of zero will always be defined such that it performs the most - * complete set of synchronization operations that are defined.This means - * stype zero always does a completion barrier that affects both loads and - * stores preceding the SYNC instruction and both loads and stores that are - * subsequent to the SYNC instruction. Non-zero values of stype may be defined - * by the architecture or specific implementations to perform synchronization - * behaviors that are less complete than that of stype zero. If an - * implementation does not use one of these non-zero values to define a - * different synchronization behavior, then that non-zero value of stype must - * act the same as stype zero completion barrier. This allows software written - * for an implementation with a lighter-weight barrier to work on another - * implementation which only implements the stype zero completion barrier. - * - * - A completion barrier is required, potentially in conjunction with SSNOP (in - * Release 1 of the Architecture) or EHB (in Release 2 of the Architecture), - * to guarantee that memory reference results are visible across operating - * mode changes. For example, a completion barrier is required on some - * implementations on entry to and exit from Debug Mode to guarantee that - * memory effects are handled correctly. - */ - -/* - * stype 0 - A completion barrier that affects preceding loads and stores and - * subsequent loads and stores. - * Older instructions which must reach the load/store ordering point before the - * SYNC instruction completes: Loads, Stores - * Younger instructions which must reach the load/store ordering point only - * after the SYNC instruction completes: Loads, Stores - * Older instructions which must be globally performed when the SYNC instruction - * completes: Loads, Stores - */ -#define STYPE_SYNC 0x0 - -/* - * Ordering barriers: - * - Every synchronizable specified memory instruction (loads or stores or both) - * that occurs in the instruction stream before the SYNC instruction must - * reach a stage in the load/store datapath after which no instruction - * re-ordering is possible before any synchronizable specified memory - * instruction which occurs after the SYNC instruction in the instruction - * stream reaches the same stage in the load/store datapath. - * - * - If any memory instruction before the SYNC instruction in program order, - * generates a memory request to the external memory and any memory - * instruction after the SYNC instruction in program order also generates a - * memory request to external memory, the memory request belonging to the - * older instruction must be globally performed before the time the memory - * request belonging to the younger instruction is globally performed. - * - * - The barrier does not guarantee the order in which instruction fetches are - * performed. - */ - -/* - * stype 0x10 - An ordering barrier that affects preceding loads and stores and - * subsequent loads and stores. - * Older instructions which must reach the load/store ordering point before the - * SYNC instruction completes: Loads, Stores - * Younger instructions which must reach the load/store ordering point only - * after the SYNC instruction completes: Loads, Stores - * Older instructions which must be globally performed when the SYNC instruction - * completes: N/A - */ -#define STYPE_SYNC_MB 0x10 - -/* - * stype 0x14 - A completion barrier specific to global invalidations - * - * When a sync instruction of this type completes any preceding GINVI or GINVT - * operation has been globalized & completed on all coherent CPUs. Anything - * that the GINV* instruction should invalidate will have been invalidated on - * all coherent CPUs when this instruction completes. It is implementation - * specific whether the GINV* instructions themselves will ensure completion, - * or this sync type will. - * - * In systems implementing global invalidates (ie. with Config5.GI == 2 or 3) - * this sync type also requires that previous SYNCI operations have completed. - */ -#define STYPE_GINV 0x14 +#include #ifdef CONFIG_CPU_HAS_SYNC #define __sync() \ @@ -286,7 +177,7 @@ static inline void sync_ginv(void) { - asm volatile("sync\t%0" :: "i"(STYPE_GINV)); + asm volatile("sync\t%0" :: "i"(__SYNC_ginv)); } #include diff --git a/arch/mips/include/asm/sync.h b/arch/mips/include/asm/sync.h new file mode 100644 index 000000000000..7c6a1095f556 --- /dev/null +++ b/arch/mips/include/asm/sync.h @@ -0,0 +1,207 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef __MIPS_ASM_SYNC_H__ +#define __MIPS_ASM_SYNC_H__ + +/* + * sync types are defined by the MIPS64 Instruction Set documentation in Volume + * II-A of the MIPS Architecture Reference Manual, which can be found here: + * + * https://www.mips.com/?do-download=the-mips64-instruction-set-v6-06 + * + * Two types of barrier are provided: + * + * 1) Completion barriers, which ensure that a memory operation has actually + * completed & often involve stalling the CPU pipeline to do so. + * + * 2) Ordering barriers, which only ensure that affected memory operations + * won't be reordered in the CPU pipeline in a manner that violates the + * restrictions imposed by the barrier. + * + * Ordering barriers can be more efficient than completion barriers, since: + * + * a) Ordering barriers only require memory access instructions which preceed + * them in program order (older instructions) to reach a point in the + * load/store datapath beyond which reordering is not possible before + * allowing memory access instructions which follow them (younger + * instructions) to be performed. That is, older instructions don't + * actually need to complete - they just need to get far enough that all + * other coherent CPUs will observe their completion before they observe + * the effects of younger instructions. + * + * b) Multiple variants of ordering barrier are provided which allow the + * effects to be restricted to different combinations of older or younger + * loads or stores. By way of example, if we only care that stores older + * than a barrier are observed prior to stores that are younger than a + * barrier & don't care about the ordering of loads then the 'wmb' + * ordering barrier can be used. Limiting the barrier's effects to stores + * allows loads to continue unaffected & potentially allows the CPU to + * make progress faster than if younger loads had to wait for older stores + * to complete. + */ + +/* + * No sync instruction at all; used to allow code to nullify the effect of the + * __SYNC() macro without needing lots of #ifdefery. + */ +#define __SYNC_none -1 + +/* + * A full completion barrier; all memory accesses appearing prior to this sync + * instruction in program order must complete before any memory accesses + * appearing after this sync instruction in program order. + */ +#define __SYNC_full 0x00 + +/* + * For now we use a full completion barrier to implement all sync types, until + * we're satisfied that lightweight ordering barriers defined by MIPSr6 are + * sufficient to uphold our desired memory model. + */ +#define __SYNC_aq __SYNC_full +#define __SYNC_rl __SYNC_full +#define __SYNC_mb __SYNC_full + +/* + * ...except on Cavium Octeon CPUs, which have been using the 'wmb' ordering + * barrier since 2010 & omit 'rmb' barriers because the CPUs don't perform + * speculative reads. + */ +#ifdef CONFIG_CPU_CAVIUM_OCTEON +# define __SYNC_rmb __SYNC_none +# define __SYNC_wmb 0x04 +#else +# define __SYNC_rmb __SYNC_full +# define __SYNC_wmb __SYNC_full +#endif + +/* + * A GINV sync is a little different; it doesn't relate directly to loads or + * stores, but instead causes synchronization of an icache or TLB global + * invalidation operation triggered by the ginvi or ginvt instructions + * respectively. In cases where we need to know that a ginvi or ginvt operation + * has been performed by all coherent CPUs, we must issue a sync instruction of + * this type. Once this instruction graduates all coherent CPUs will have + * observed the invalidation. + */ +#define __SYNC_ginv 0x14 + +/* Trivial; indicate that we always need this sync instruction. */ +#define __SYNC_always (1 << 0) + +/* + * Indicate that we need this sync instruction only on systems with weakly + * ordered memory access. In general this is most MIPS systems, but there are + * exceptions which provide strongly ordered memory. + */ +#ifdef CONFIG_WEAK_ORDERING +# define __SYNC_weak_ordering (1 << 1) +#else +# define __SYNC_weak_ordering 0 +#endif + +/* + * Indicate that we need this sync instruction only on systems where LL/SC + * don't implicitly provide a memory barrier. In general this is most MIPS + * systems. + */ +#ifdef CONFIG_WEAK_REORDERING_BEYOND_LLSC +# define __SYNC_weak_llsc (1 << 2) +#else +# define __SYNC_weak_llsc 0 +#endif + +/* + * Some Loongson 3 CPUs have a bug wherein execution of a memory access (load, + * store or prefetch) in between an LL & SC can cause the SC instruction to + * erroneously succeed, breaking atomicity. Whilst it's unusual to write code + * containing such sequences, this bug bites harder than we might otherwise + * expect due to reordering & speculation: + * + * 1) A memory access appearing prior to the LL in program order may actually + * be executed after the LL - this is the reordering case. + * + * In order to avoid this we need to place a memory barrier (ie. a SYNC + * instruction) prior to every LL instruction, in between it and any earlier + * memory access instructions. + * + * This reordering case is fixed by 3A R2 CPUs, ie. 3A2000 models and later. + * + * 2) If a conditional branch exists between an LL & SC with a target outside + * of the LL-SC loop, for example an exit upon value mismatch in cmpxchg() + * or similar, then misprediction of the branch may allow speculative + * execution of memory accesses from outside of the LL-SC loop. + * + * In order to avoid this we need a memory barrier (ie. a SYNC instruction) + * at each affected branch target. + * + * This case affects all current Loongson 3 CPUs. + * + * The above described cases cause an error in the cache coherence protocol; + * such that the Invalidate of a competing LL-SC goes 'missing' and SC + * erroneously observes its core still has Exclusive state and lets the SC + * proceed. + * + * Therefore the error only occurs on SMP systems. + */ +#ifdef CONFIG_CPU_LOONGSON3_WORKAROUNDS +# define __SYNC_loongson3_war (1 << 31) +#else +# define __SYNC_loongson3_war 0 +#endif + +/* + * Some Cavium Octeon CPUs suffer from a bug that causes a single wmb ordering + * barrier to be ineffective, requiring the use of 2 in sequence to provide an + * effective barrier as noted by commit 6b07d38aaa52 ("MIPS: Octeon: Use + * optimized memory barrier primitives."). Here we specify that the affected + * sync instructions should be emitted twice. + */ +#ifdef CONFIG_CPU_CAVIUM_OCTEON +# define __SYNC_rpt(type) (1 + (type == __SYNC_wmb)) +#else +# define __SYNC_rpt(type) 1 +#endif + +/* + * The main event. Here we actually emit a sync instruction of a given type, if + * reason is non-zero. + * + * In future we have the option of emitting entries in a fixups-style table + * here that would allow us to opportunistically remove some sync instructions + * when we detect at runtime that we're running on a CPU that doesn't need + * them. + */ +#ifdef CONFIG_CPU_HAS_SYNC +# define ____SYNC(_type, _reason, _else) \ + .if (( _type ) != -1) && ( _reason ); \ + .set push; \ + .set MIPS_ISA_LEVEL_RAW; \ + .rept __SYNC_rpt(_type); \ + sync _type; \ + .endr; \ + .set pop; \ + .else; \ + _else; \ + .endif +#else +# define ____SYNC(_type, _reason, _else) +#endif + +/* + * Preprocessor magic to expand macros used as arguments before we insert them + * into assembly code. + */ +#ifdef __ASSEMBLY__ +# define ___SYNC(type, reason, else) \ + ____SYNC(type, reason, else) +#else +# define ___SYNC(type, reason, else) \ + __stringify(____SYNC(type, reason, else)) +#endif + +#define __SYNC(type, reason) \ + ___SYNC(__SYNC_##type, __SYNC_##reason, ) +#define __SYNC_ELSE(type, reason, else) \ + ___SYNC(__SYNC_##type, __SYNC_##reason, else) + +#endif /* __MIPS_ASM_SYNC_H__ */ diff --git a/arch/mips/kernel/pm-cps.c b/arch/mips/kernel/pm-cps.c index a26f40db15d0..9bf60d7d44d3 100644 --- a/arch/mips/kernel/pm-cps.c +++ b/arch/mips/kernel/pm-cps.c @@ -307,7 +307,7 @@ static int cps_gen_flush_fsb(u32 **pp, struct uasm_label **pl, } /* Barrier ensuring previous cache invalidates are complete */ - uasm_i_sync(pp, STYPE_SYNC); + uasm_i_sync(pp, __SYNC_full); uasm_i_ehb(pp); /* Check whether the pipeline stalled due to the FSB being full */ @@ -397,7 +397,7 @@ static void *cps_gen_entry_code(unsigned cpu, enum cps_pm_state state) if (coupled_coherence) { /* Increment ready_count */ - uasm_i_sync(&p, STYPE_SYNC_MB); + uasm_i_sync(&p, __SYNC_mb); uasm_build_label(&l, p, lbl_incready); uasm_i_ll(&p, t1, 0, r_nc_count); uasm_i_addiu(&p, t2, t1, 1); @@ -406,7 +406,7 @@ static void *cps_gen_entry_code(unsigned cpu, enum cps_pm_state state) uasm_i_addiu(&p, t1, t1, 1); /* Barrier ensuring all CPUs see the updated r_nc_count value */ - uasm_i_sync(&p, STYPE_SYNC_MB); + uasm_i_sync(&p, __SYNC_mb); /* * If this is the last VPE to become ready for non-coherence @@ -473,7 +473,7 @@ static void *cps_gen_entry_code(unsigned cpu, enum cps_pm_state state) Index_Writeback_Inv_D, lbl_flushdcache); /* Barrier ensuring previous cache invalidates are complete */ - uasm_i_sync(&p, STYPE_SYNC); + uasm_i_sync(&p, __SYNC_full); uasm_i_ehb(&p); if (mips_cm_revision() < CM_REV_CM3) { @@ -487,7 +487,7 @@ static void *cps_gen_entry_code(unsigned cpu, enum cps_pm_state state) uasm_i_lw(&p, t0, 0, r_pcohctl); /* Barrier to ensure write to coherence control is complete */ - uasm_i_sync(&p, STYPE_SYNC); + uasm_i_sync(&p, __SYNC_full); uasm_i_ehb(&p); } @@ -534,7 +534,7 @@ static void *cps_gen_entry_code(unsigned cpu, enum cps_pm_state state) } /* Barrier to ensure write to CPC command is complete */ - uasm_i_sync(&p, STYPE_SYNC); + uasm_i_sync(&p, __SYNC_full); uasm_i_ehb(&p); } @@ -572,13 +572,13 @@ static void *cps_gen_entry_code(unsigned cpu, enum cps_pm_state state) uasm_i_lw(&p, t0, 0, r_pcohctl); /* Barrier to ensure write to coherence control is complete */ - uasm_i_sync(&p, STYPE_SYNC); + uasm_i_sync(&p, __SYNC_full); uasm_i_ehb(&p); if (coupled_coherence && (state == CPS_PM_NC_WAIT)) { /* Decrement ready_count */ uasm_build_label(&l, p, lbl_decready); - uasm_i_sync(&p, STYPE_SYNC_MB); + uasm_i_sync(&p, __SYNC_mb); uasm_i_ll(&p, t1, 0, r_nc_count); uasm_i_addiu(&p, t2, t1, -1); uasm_i_sc(&p, t2, 0, r_nc_count); @@ -586,7 +586,7 @@ static void *cps_gen_entry_code(unsigned cpu, enum cps_pm_state state) uasm_i_andi(&p, v0, t1, (1 << fls(smp_num_siblings)) - 1); /* Barrier ensuring all CPUs see the updated r_nc_count value */ - uasm_i_sync(&p, STYPE_SYNC_MB); + uasm_i_sync(&p, __SYNC_mb); } if (coupled_coherence && (state == CPS_PM_CLOCK_GATED)) { @@ -608,7 +608,7 @@ static void *cps_gen_entry_code(unsigned cpu, enum cps_pm_state state) uasm_build_label(&l, p, lbl_secondary_cont); /* Barrier ensuring all CPUs see the updated r_nc_count value */ - uasm_i_sync(&p, STYPE_SYNC_MB); + uasm_i_sync(&p, __SYNC_mb); } /* The core is coherent, time to return to C code */ From patchwork Tue Oct 1 21:53:08 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Burton X-Patchwork-Id: 11169777 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6D94F1709 for ; Tue, 1 Oct 2019 21:53:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 42A8C21924 for ; Tue, 1 Oct 2019 21:53:18 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=wavecomp.com header.i=@wavecomp.com header.b="IcMD8WtW" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728069AbfJAVxR (ORCPT ); Tue, 1 Oct 2019 17:53:17 -0400 Received: from mail-eopbgr730120.outbound.protection.outlook.com ([40.107.73.120]:2512 "EHLO NAM05-DM3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728041AbfJAVxR (ORCPT ); Tue, 1 Oct 2019 17:53:17 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Bpw5rkOdN9n3LvgmSuj+Z+YzY1xhW9xdYxnjfM7mmVKiXPJxRrMILeI+ZFc8DCKFdfG1N1Ji2ElogKr9Z428YaRI/Wp7fBsOBftBB3UG8thN/9jnoWu+D4MQhJzJJ4C+KgpDEo9SXcjGjMarCWGLOar7qlfP5vdYEFNpd7BL43Wdn0+uGUI+oKzHP2lNv9Tfx22mofqtHr3EDStHY6RBoHXsdil0SfuFVCJ1pJiSCJMKS2xZhxxE5++/6bL0AnH7klNHaXUagog0zT2rolXLwTTBuPgLqUoxDY2bY3bT7y2Xywt91+6OGYipKdpicLNT0gS33Mm3FmJ5BywSDsTjEA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+5ZgLdYPRBTcgvtPfNv7KwgHqKOev7ZMnWj11NjJxGg=; b=nYhAJ5lsYt4MekIA5crRlCi/zRUVc/5AUuVgZWGUYZiUFD+p4H1oGiuVWUXyrcRSl43xmd1jjFHC3h7Pj8CfqKYz+UyCmiYa6Zh/eePky03h40893L118AUxYs6BpstoEhiaHnPFYFUxrDhZNDyImZJS/ZKEwP34r0iH8eIGzo7LSMLP+55Q6uK199quj5TAt5MVmrKYQWBqnG4V+TM9V6/LeTNHd7NOLBOCCFlmkZ0LM0SSDXpx9PoP5x8qkpkK1iIgbMe4DdsaakqfHlbkSuw0bqig+YbIndph34x4/C2qNPSM9R63DUvy4+KCKhiyHToGEZnkt9aNQomN472J+w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wavecomp.com; dmarc=pass action=none header.from=mips.com; dkim=pass header.d=mips.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wavecomp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+5ZgLdYPRBTcgvtPfNv7KwgHqKOev7ZMnWj11NjJxGg=; b=IcMD8WtWITHiSiS/ylrtRyCEcprBJ1Ol2TNDu7S4X+0xnAifiysFPzyftRBwOTJSlc4/Rqu4hpBiw73vRz1UVnRUibME6AP/5xCqHxuQv3SfFKPAk01TDk9JSbTdcTMnSmNPyc8lBY++3VoizClBepNoLRwrmOKLVqJPGS5BYzs= Received: from MWHPR2201MB1277.namprd22.prod.outlook.com (10.172.60.12) by MWHPR2201MB1213.namprd22.prod.outlook.com (10.172.61.148) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2305.20; Tue, 1 Oct 2019 21:53:08 +0000 Received: from MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481]) by MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481%2]) with mapi id 15.20.2305.022; Tue, 1 Oct 2019 21:53:08 +0000 From: Paul Burton To: "linux-mips@vger.kernel.org" CC: Huacai Chen , Jiaxun Yang , "linux-kernel@vger.kernel.org" , Paul Burton Subject: [PATCH v2 04/36] MIPS: barrier: Clean up rmb() & wmb() definitions Thread-Topic: [PATCH v2 04/36] MIPS: barrier: Clean up rmb() & wmb() definitions Thread-Index: AQHVeKKcp1ytqV6lDUWfIgPgu4vc+Q== Date: Tue, 1 Oct 2019 21:53:08 +0000 Message-ID: <20191001215249.4157062-5-paul.burton@mips.com> References: <20191001215249.4157062-1-paul.burton@mips.com> In-Reply-To: <20191001215249.4157062-1-paul.burton@mips.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BYAPR07CA0093.namprd07.prod.outlook.com (2603:10b6:a03:12b::34) To MWHPR2201MB1277.namprd22.prod.outlook.com (2603:10b6:301:18::12) authentication-results: spf=none (sender IP is ) smtp.mailfrom=pburton@wavecomp.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.23.0 x-originating-ip: [12.94.197.246] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: e8974825-8baf-498f-d225-08d746b9bf1f x-ms-traffictypediagnostic: MWHPR2201MB1213: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:4941; x-forefront-prvs: 0177904E6B x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(346002)(396003)(366004)(376002)(136003)(39840400004)(199004)(189003)(6512007)(5640700003)(54906003)(2501003)(3846002)(6486002)(5660300002)(6436002)(66066001)(6116002)(1076003)(50226002)(2906002)(76176011)(52116002)(6506007)(107886003)(2351001)(4326008)(6916009)(102836004)(71190400001)(8936002)(386003)(25786009)(36756003)(44832011)(8676002)(71200400001)(81166006)(81156014)(186003)(256004)(11346002)(2616005)(7736002)(316002)(42882007)(305945005)(99286004)(66446008)(64756008)(66556008)(478600001)(66946007)(66476007)(486006)(446003)(476003)(14454004)(26005);DIR:OUT;SFP:1102;SCL:1;SRVR:MWHPR2201MB1213;H:MWHPR2201MB1277.namprd22.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; received-spf: None (protection.outlook.com: wavecomp.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: UCyCNu77ncsgdme9okacj04MFkDblRbxO8RRwzXKK6lz57dQqH6HG4QipUellyNMINUeTeecy8I5UoE/LelV2VOXY53oFZEY204JeRQ2B3D0OMUJBxkMesvmLfTt59nadjl6T1nhaC+WkFJ2qdnC7qWLKYp/dQ5KUQs0y+MFO4YyKDjEm2BAbDGlpcm5XeL9RvHQ66vtXLUSzBzgCyCMwh2Mdk7ZOOk9S+Hm4uKxJYoMoR6aOIllUi7A30i3wJFwc/6BJ1CdlY61fR+vXmugvrdoTsNN5CPDnzMN9pu9XsIsA2JuZd1JKazMaFCapdg4NVLZ9HKLMr21eC+wISL+QAiDXrEX9S6/Sag1NLZTuOU2JYkLsoQaJls1CwAq+vgtcrrtSuW7TVDYZXLWfRoerpgwHfwTO4pasg9Rabvlqy8= MIME-Version: 1.0 X-OriginatorOrg: mips.com X-MS-Exchange-CrossTenant-Network-Message-Id: e8974825-8baf-498f-d225-08d746b9bf1f X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Oct 2019 21:53:08.6881 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 463607d3-1db3-40a0-8a29-970c56230104 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: kkqL1OIGQPy7vZOHUtAhl+UQHuBvcK4czqDmW1bKLZs2CSda6VTzCaktFRsKnTsajDeXjXtPevJsmPU4Ub1ztw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR2201MB1213 Sender: linux-mips-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-mips@vger.kernel.org Simplify our definitions of rmb() & wmb() using the new __SYNC() infrastructure. The fast_rmb() & fast_wmb() macros are removed, since they only provided a level of indirection that made the code less readable & weren't directly used anywhere in the kernel tree. The Octeon #ifdef'ery is removed, since the "syncw" instruction previously used is merely an alias for "sync 4" which __SYNC() will emit for the wmb sync type when the kernel is configured for an Octeon CPU. Similarly __SYNC() will emit nothing for the rmb sync type in Octeon configurations. Signed-off-by: Paul Burton --- Changes in v2: None arch/mips/include/asm/barrier.h | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/arch/mips/include/asm/barrier.h b/arch/mips/include/asm/barrier.h index 5ad39bfd3b6d..f36cab87cfde 100644 --- a/arch/mips/include/asm/barrier.h +++ b/arch/mips/include/asm/barrier.h @@ -26,6 +26,18 @@ #define __sync() do { } while(0) #endif +static inline void rmb(void) +{ + asm volatile(__SYNC(rmb, always) ::: "memory"); +} +#define rmb rmb + +static inline void wmb(void) +{ + asm volatile(__SYNC(wmb, always) ::: "memory"); +} +#define wmb wmb + #define __fast_iob() \ __asm__ __volatile__( \ ".set push\n\t" \ @@ -37,16 +49,9 @@ : "m" (*(int *)CKSEG1) \ : "memory") #ifdef CONFIG_CPU_CAVIUM_OCTEON -# define OCTEON_SYNCW_STR ".set push\n.set arch=octeon\nsyncw\nsyncw\n.set pop\n" -# define __syncw() __asm__ __volatile__(OCTEON_SYNCW_STR : : : "memory") - -# define fast_wmb() __syncw() -# define fast_rmb() barrier() # define fast_mb() __sync() # define fast_iob() do { } while (0) #else /* ! CONFIG_CPU_CAVIUM_OCTEON */ -# define fast_wmb() __sync() -# define fast_rmb() __sync() # define fast_mb() __sync() # ifdef CONFIG_SGI_IP28 # define fast_iob() \ @@ -83,19 +88,14 @@ #endif /* !CONFIG_CPU_HAS_WB */ -#define wmb() fast_wmb() -#define rmb() fast_rmb() - #if defined(CONFIG_WEAK_ORDERING) # ifdef CONFIG_CPU_CAVIUM_OCTEON # define __smp_mb() __sync() -# define __smp_rmb() barrier() -# define __smp_wmb() __syncw() # else # define __smp_mb() __asm__ __volatile__("sync" : : :"memory") -# define __smp_rmb() __asm__ __volatile__("sync" : : :"memory") -# define __smp_wmb() __asm__ __volatile__("sync" : : :"memory") # endif +# define __smp_rmb() rmb() +# define __smp_wmb() wmb() #else #define __smp_mb() barrier() #define __smp_rmb() barrier() From patchwork Tue Oct 1 21:53:10 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Burton X-Patchwork-Id: 11169835 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0B80D1599 for ; Tue, 1 Oct 2019 21:55:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id DBA2021924 for ; Tue, 1 Oct 2019 21:55:12 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=wavecomp.com header.i=@wavecomp.com header.b="b4MOgLCx" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728605AbfJAVzM (ORCPT ); Tue, 1 Oct 2019 17:55:12 -0400 Received: from mail-eopbgr730120.outbound.protection.outlook.com ([40.107.73.120]:2512 "EHLO NAM05-DM3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728024AbfJAVxS (ORCPT ); Tue, 1 Oct 2019 17:53:18 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LgV22Konmbnockrb4wPYzNOtOF8/VzN2Fq9Y7b5QfiUbyrCeyGCwIcJaLpuV1L90+jutn8nHvJXc58k6jupn7hCNgBJa9e/rysUdvTmoxcshbuc6JSJV3s4SMqKz91ZajYb/Uy3DB76r+fMZB2dHKVr1bcpYybpSZKqWj5CY26s/4o8yLZ4TfZ9wcD1JwMqBzPgJgISMYP0oLm7oxbj3V8bZl1Wyw2LYzv30BRDPBXZ2hU2vCWVInlMC/WlKrItLfa1yG8QMUdnZ0xKRqxxTAnuTFfvWpZmw3szDbI4MxPI+1ESBwLj07+fNQMSZ9fGYeXjvYTVK1DBqjo3wDqU9gA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=HErJtKN+ZByZN+F9FXcEwN0O0QpNIL2oUDRyjUdVTrs=; b=Y/FQJ1yuLm4ZoLG5aZVQp3EQjGMRrQoV0zJ8DLY31vObjvf5wBlAchujWwG0PxRlcFEpwbjpYHJ+MSmlhCkq6e+5BTfddmzZm7Cn2XXmoqZAUfJ3cMvrmSJokN+isLkDkM5yoT08BnE+4paJmcyMMNjCkFvBxp+7rTie49B5Rf6AkNt3c4pz/sluy/4WVMK87n7qijbjGonq5VHD84g+pgyIB5lhhbpxEkzaMrzAWaCKAXgYeAHQfGlNbL0RB3S941WALvRsajdSNGbFcNYYIaCDHVZKx+MC9dmTbpMkvocY5yJInSIqoHy9O31rcr0HFwxhuVkUy+UHS/159Wg/hg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wavecomp.com; dmarc=pass action=none header.from=mips.com; dkim=pass header.d=mips.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wavecomp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=HErJtKN+ZByZN+F9FXcEwN0O0QpNIL2oUDRyjUdVTrs=; b=b4MOgLCx1lP5GctnId8KafRy925SDUN0qJDJ3gNkjlpo6ddRyf4Wf98EZzrgy+ajCbzmMSkMDj+YSJMuEUSoTfUApgdHn71weW6ixJu/oIPBKnbxWt9Fn5jTAPoaVJlSzXlcDCLBVLuz/pKibmKak10atT6aiDEdpPlMmxiP+OE= Received: from MWHPR2201MB1277.namprd22.prod.outlook.com (10.172.60.12) by MWHPR2201MB1213.namprd22.prod.outlook.com (10.172.61.148) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2305.20; Tue, 1 Oct 2019 21:53:10 +0000 Received: from MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481]) by MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481%2]) with mapi id 15.20.2305.022; Tue, 1 Oct 2019 21:53:10 +0000 From: Paul Burton To: "linux-mips@vger.kernel.org" CC: Huacai Chen , Jiaxun Yang , "linux-kernel@vger.kernel.org" , Paul Burton Subject: [PATCH v2 05/36] MIPS: barrier: Clean up __smp_mb() definition Thread-Topic: [PATCH v2 05/36] MIPS: barrier: Clean up __smp_mb() definition Thread-Index: AQHVeKKdE9HQxFlerUGI5C03xj1djA== Date: Tue, 1 Oct 2019 21:53:10 +0000 Message-ID: <20191001215249.4157062-6-paul.burton@mips.com> References: <20191001215249.4157062-1-paul.burton@mips.com> In-Reply-To: <20191001215249.4157062-1-paul.burton@mips.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BYAPR07CA0093.namprd07.prod.outlook.com (2603:10b6:a03:12b::34) To MWHPR2201MB1277.namprd22.prod.outlook.com (2603:10b6:301:18::12) authentication-results: spf=none (sender IP is ) smtp.mailfrom=pburton@wavecomp.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.23.0 x-originating-ip: [12.94.197.246] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: af156861-480a-4fd9-31bf-08d746b9bfec x-ms-traffictypediagnostic: MWHPR2201MB1213: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:7691; x-forefront-prvs: 0177904E6B x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(346002)(396003)(366004)(376002)(136003)(39840400004)(199004)(189003)(6512007)(5640700003)(54906003)(2501003)(3846002)(6486002)(5660300002)(6436002)(66066001)(6116002)(1076003)(50226002)(2906002)(76176011)(52116002)(6506007)(107886003)(2351001)(4326008)(6916009)(102836004)(71190400001)(8936002)(386003)(25786009)(36756003)(44832011)(8676002)(71200400001)(81166006)(81156014)(186003)(256004)(11346002)(14444005)(2616005)(7736002)(316002)(42882007)(305945005)(99286004)(66446008)(64756008)(66556008)(478600001)(66946007)(66476007)(486006)(446003)(476003)(14454004)(26005);DIR:OUT;SFP:1102;SCL:1;SRVR:MWHPR2201MB1213;H:MWHPR2201MB1277.namprd22.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; received-spf: None (protection.outlook.com: wavecomp.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: R/Ey45PSJ9zmvijFSRxzjFMvh064BGWOIpWbXci9/Sp7NEdzCnTGpvfRaiJ65uzVq46OqAUMdWSHSRuUXKQYf7DEBvE9PvKjCBWha7vuzp+K078CrHz67x1LezCm+kAMS8JITt5Em+5LSOvyC2q1XG2eX6PUc7ua8DaUFmi8LlvvzcgjTnxofaOlInsvgPkpuWEhrarRP+gNVNaxCxGsxHuJvHXsJrZDEuGURQN+rzMsF4Rhhkt+JnW90qjj6AvqrAewLySl+ZQ6Jda2ZjnMKF/C2JbQyl5/zmFN3lN0BpIP7flE7RvtjvtBBnH3c0QVvbs2iIF6JzxrmXcJHxS+8nzwz1ARDFdCS+F/UaPPyVBnwFf/2YaGMDxs5cDDqHm9/lUi2bxAPbOOKqVmkWf/0+OWneu+MiDFKISeyrt2I1M= MIME-Version: 1.0 X-OriginatorOrg: mips.com X-MS-Exchange-CrossTenant-Network-Message-Id: af156861-480a-4fd9-31bf-08d746b9bfec X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Oct 2019 21:53:10.0665 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 463607d3-1db3-40a0-8a29-970c56230104 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: AWnEJOUppU1cuO/DusSSdf+eMJsmSieBLvZP8wsV0wuUhne+GMSirmKGqXjp6GbNOHHITdwOKXRwfsFoDl1Wcg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR2201MB1213 Sender: linux-mips-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-mips@vger.kernel.org We #ifdef on Cavium Octeon CPUs, but emit the same sync instruction in both cases. Remove the #ifdef & simply expand to the __sync() macro. Whilst here indent the strong ordering case definitions to match the indentation of the weak ordering ones, helping readability. Signed-off-by: Paul Burton --- Changes in v2: None arch/mips/include/asm/barrier.h | 12 ++++-------- 1 file changed, 4 insertions(+), 8 deletions(-) diff --git a/arch/mips/include/asm/barrier.h b/arch/mips/include/asm/barrier.h index f36cab87cfde..8a5abc1c85a6 100644 --- a/arch/mips/include/asm/barrier.h +++ b/arch/mips/include/asm/barrier.h @@ -89,17 +89,13 @@ static inline void wmb(void) #endif /* !CONFIG_CPU_HAS_WB */ #if defined(CONFIG_WEAK_ORDERING) -# ifdef CONFIG_CPU_CAVIUM_OCTEON -# define __smp_mb() __sync() -# else -# define __smp_mb() __asm__ __volatile__("sync" : : :"memory") -# endif +# define __smp_mb() __sync() # define __smp_rmb() rmb() # define __smp_wmb() wmb() #else -#define __smp_mb() barrier() -#define __smp_rmb() barrier() -#define __smp_wmb() barrier() +# define __smp_mb() barrier() +# define __smp_rmb() barrier() +# define __smp_wmb() barrier() #endif /* From patchwork Tue Oct 1 21:53:11 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Burton X-Patchwork-Id: 11169837 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8F42813B1 for ; Tue, 1 Oct 2019 21:55:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 634E721A4A for ; Tue, 1 Oct 2019 21:55:23 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=wavecomp.com header.i=@wavecomp.com header.b="Kns37M1v" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728035AbfJAVxO (ORCPT ); Tue, 1 Oct 2019 17:53:14 -0400 Received: from mail-eopbgr730127.outbound.protection.outlook.com ([40.107.73.127]:55809 "EHLO NAM05-DM3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728024AbfJAVxO (ORCPT ); Tue, 1 Oct 2019 17:53:14 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=a/J5dX0pL1h6O+/f8ezZ3Q/fU1kxUB27gFKwawMBDFNqdv36p3YTKrvhWmIUqpfjGuOBZm7yvuEk0UpYqgryBQN4+MST5gC0BFNO8xs7LKJMuEE0rWciyiJZ6LOjTGv0vE1v+mqfy9gW2JShnk5FcrG8+Xqm9ID83ZFD9RwWZla59qmNUfnoqG0i+dMsXxqNZGRXKkxxoITYLUYX0JVyPBt3hjye086eZGA3nl28I6vrmIZ36SHFyM9hkxOOPaOGoMCZ1Hy1MXqry6UQP50nJaZN8+LVOiUT0aWIzISTGb8TmL8zNjPibM+WYhNOEY977BjFAh2AWcgCqq6LwSG14Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=6e7ebd9MXfR2Sz/oJu/bw9Jr1OuQWNHWSBxeBRXCAmQ=; b=hHkTgWt1531+EMJG9ZUR6vleT2eiQSu+7lQaGIvhCkrTzAgTEaprrWsl7Oks8usfvh9Gz7ovNOAf0QVILNK7Fe6tA/9EgSXmBRP6JkAMsPpmOb2WlLBCJA0ZOsyJ6Qte5EQF0IXKQuBGCH03e9dJwjCGlGk4FyNurz/tKH3FVOjuEJm+2J2GBZo7XsKE1TAfG/s567V7myDHmLi8tonCowCQ0xI46IA5IcNvsD9y2FaenrPVZHZK0FeMjSpmLrhFqK2phTbUbX+HliEcVTYeErbgLReS5nHas5GWgYjiGV9Tp/JuIRbwBUp0HJs95t1BkwQ4ULVjnT32vfrSkY5wcQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wavecomp.com; dmarc=pass action=none header.from=mips.com; dkim=pass header.d=mips.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wavecomp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=6e7ebd9MXfR2Sz/oJu/bw9Jr1OuQWNHWSBxeBRXCAmQ=; b=Kns37M1vHgMAMNZyM7lO8Ht4P6yrEHqHKvW8my7u0g41liiE0A3sTeaPfgA3vqutsa/QiW/+l5K9WLcP5yMXnoCLTn2Hru4DdV+eT3KQUqVUqZoT9oAtkp8M9yTA98RAQqMR4//dHZzKlwbcfZ4jKA71K+ePUxY8+OEy1b91RRw= Received: from MWHPR2201MB1277.namprd22.prod.outlook.com (10.172.60.12) by MWHPR2201MB1439.namprd22.prod.outlook.com (10.174.169.166) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2305.20; Tue, 1 Oct 2019 21:53:11 +0000 Received: from MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481]) by MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481%2]) with mapi id 15.20.2305.022; Tue, 1 Oct 2019 21:53:11 +0000 From: Paul Burton To: "linux-mips@vger.kernel.org" CC: Huacai Chen , Jiaxun Yang , "linux-kernel@vger.kernel.org" , Paul Burton Subject: [PATCH v2 06/36] MIPS: barrier: Remove fast_mb() Octeon #ifdef'ery Thread-Topic: [PATCH v2 06/36] MIPS: barrier: Remove fast_mb() Octeon #ifdef'ery Thread-Index: AQHVeKKecuBJXFEExEuaIM5meiefFQ== Date: Tue, 1 Oct 2019 21:53:11 +0000 Message-ID: <20191001215249.4157062-7-paul.burton@mips.com> References: <20191001215249.4157062-1-paul.burton@mips.com> In-Reply-To: <20191001215249.4157062-1-paul.burton@mips.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BYAPR07CA0093.namprd07.prod.outlook.com (2603:10b6:a03:12b::34) To MWHPR2201MB1277.namprd22.prod.outlook.com (2603:10b6:301:18::12) authentication-results: spf=none (sender IP is ) smtp.mailfrom=pburton@wavecomp.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.23.0 x-originating-ip: [12.94.197.246] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 941c3f8c-3e9f-42f8-2bbc-08d746b9c07c x-ms-traffictypediagnostic: MWHPR2201MB1439: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:4502; x-forefront-prvs: 0177904E6B x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(396003)(39840400004)(366004)(346002)(136003)(376002)(199004)(189003)(186003)(11346002)(6512007)(1076003)(486006)(66066001)(26005)(6486002)(316002)(2906002)(5640700003)(6436002)(44832011)(42882007)(4744005)(25786009)(446003)(6916009)(386003)(6506007)(76176011)(478600001)(52116002)(2616005)(102836004)(476003)(5660300002)(305945005)(64756008)(66446008)(2351001)(7736002)(99286004)(54906003)(14454004)(71190400001)(71200400001)(66946007)(66556008)(66476007)(36756003)(3846002)(6116002)(107886003)(4326008)(50226002)(81166006)(81156014)(8676002)(2501003)(8936002)(256004);DIR:OUT;SFP:1102;SCL:1;SRVR:MWHPR2201MB1439;H:MWHPR2201MB1277.namprd22.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; received-spf: None (protection.outlook.com: wavecomp.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: QqBWANbODkvJAoLJDkCtW5LaFywMrHLyYCi1au3LJSFtk7BizZfSDuEgPfv0kbM9AcLi6sfkFkx45KGeD9QByONoXEygbHc8NnMSsBFrQsETbqLo4AZPAjAlWS9t8e9ZLx1eUTyH7lzaMNII+9rAPIvifSeexR7KzIywfYrL/CJ2UKKVgj79RU2+AiuVkX29QPIaUxxDaIeVMAxa8+UkJr1sVTxp5wcgD/KyqyEK2IQLf3W7ASfmDmKHgbepGqvh6+Kv3jKRYCN4CbjSBg1mdzQxtoO6bpcDicXvhfkFVJUvKEGr+y32uVXjvjWyeR23YEWOTsaBGATlk47YKGMUoDQpuXXAz5/HzSdC0Q8qCl0hquiNTO6kHIP73nQp0cuMwkhwGa+P37s2szD0++RQ0BPLwtYt7Hjjl7vVMcMX7YU= MIME-Version: 1.0 X-OriginatorOrg: mips.com X-MS-Exchange-CrossTenant-Network-Message-Id: 941c3f8c-3e9f-42f8-2bbc-08d746b9c07c X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Oct 2019 21:53:11.1000 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 463607d3-1db3-40a0-8a29-970c56230104 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: NLH36cKMPrbCp5wABLkP9iAnOAxVbw14cwrc6DQV1/7T5CXrFC7zEKqdQAo4u4Y574hc2NKJi9IKva4bhiZiDg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR2201MB1439 Sender: linux-mips-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-mips@vger.kernel.org The definition of fast_mb() is the same in both the Octeon & non-Octeon cases, so remove the duplication & define it only once. Signed-off-by: Paul Burton --- Changes in v2: None arch/mips/include/asm/barrier.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/mips/include/asm/barrier.h b/arch/mips/include/asm/barrier.h index 8a5abc1c85a6..657ec01120a4 100644 --- a/arch/mips/include/asm/barrier.h +++ b/arch/mips/include/asm/barrier.h @@ -38,6 +38,8 @@ static inline void wmb(void) } #define wmb wmb +#define fast_mb() __sync() + #define __fast_iob() \ __asm__ __volatile__( \ ".set push\n\t" \ @@ -49,10 +51,8 @@ static inline void wmb(void) : "m" (*(int *)CKSEG1) \ : "memory") #ifdef CONFIG_CPU_CAVIUM_OCTEON -# define fast_mb() __sync() # define fast_iob() do { } while (0) #else /* ! CONFIG_CPU_CAVIUM_OCTEON */ -# define fast_mb() __sync() # ifdef CONFIG_SGI_IP28 # define fast_iob() \ __asm__ __volatile__( \ From patchwork Tue Oct 1 21:53:13 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Burton X-Patchwork-Id: 11169839 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 96CA413B1 for ; Tue, 1 Oct 2019 21:55:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 6C05121A4A for ; Tue, 1 Oct 2019 21:55:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=wavecomp.com header.i=@wavecomp.com header.b="Kuc5VAuL" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726568AbfJAVzX (ORCPT ); Tue, 1 Oct 2019 17:55:23 -0400 Received: from mail-eopbgr730127.outbound.protection.outlook.com ([40.107.73.127]:55809 "EHLO NAM05-DM3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728033AbfJAVxP (ORCPT ); Tue, 1 Oct 2019 17:53:15 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SowhO0dinLJzxPD7NCevQwJHs87NN1cs58/YNOk7S3UJheI6Unm7CCaf/mMfndC2UjU1EorT3n6rIdRuVhZ6Z7ff3wiTvV1GsbM6yma3MK6vfRlwPzprBPCzK8H+U1XphfXUEUNIBarf6bg+mdPiOdFvYYK9xl9zFZyivZGRvzdzfVQuOudG+85EwWa8fVeYSFfmX6rwBegpdzq1qKeI7KO3VEOzmwYla0n/gKj6beXpv7nEeYCfGtAQH9peXL1rrVqPZ8Y5jVt0sMqHIMiWYJS4FcdHRiwk/mBfhDpYkbr22KSgbEc7Lnj/xIKnn8rrXRMNIrzjJoZf6irnEOTK5A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=npxfvJbix+azbh5wI9YeQRTK16AogeGdIvvTh4ZSUuE=; b=AXzpY9bOEMMhQgoTBw5DY0B3OvxCJj3901n/hhHdh/wDXlGo6E3zCS6ditQkWOhSAO09t7X2G6XwrSn4XpuJQuYaztpryBCeIcqy5IQA/hrW7wSAVxyAnZrTBa2SUpdx2Idxu/2K95Zy748VGc0LLc1JFCNBlij4yLNOKJxhX+g/PNxl/qC5JrY7uUWjqJVz2BbnXM1tH0LWt6U91KTk5DAvBpncs1x83OPBGuXMLCO+SqDpOV+L8irTBNVkqNXjlxyS7KNG46VH4YH9yeHgmkLSmEQfLO9nYPDzkZYnVWSUaGlcyTuw4NM/YjOeE9BQkS9oT9dacIUeQ+PB4A4UAQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wavecomp.com; dmarc=pass action=none header.from=mips.com; dkim=pass header.d=mips.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wavecomp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=npxfvJbix+azbh5wI9YeQRTK16AogeGdIvvTh4ZSUuE=; b=Kuc5VAuLygiqCZmb+XBqdE7Eia8Wn3uURYYX6dmsRGbku1R6LIF8f2LUu36Nfo6XfDIkru9Gi732k8mqW0hWXLS8edU4+EWmnDLnZ2gGHXpt6PW3X9tPEl5Iy2anKU8dISgRQeiPnqn//rcahTJTMG7ngAn8/xlwNUx8v916UdY= Received: from MWHPR2201MB1277.namprd22.prod.outlook.com (10.172.60.12) by MWHPR2201MB1439.namprd22.prod.outlook.com (10.174.169.166) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2305.20; Tue, 1 Oct 2019 21:53:13 +0000 Received: from MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481]) by MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481%2]) with mapi id 15.20.2305.022; Tue, 1 Oct 2019 21:53:13 +0000 From: Paul Burton To: "linux-mips@vger.kernel.org" CC: Huacai Chen , Jiaxun Yang , "linux-kernel@vger.kernel.org" , Paul Burton Subject: [PATCH v2 07/36] MIPS: barrier: Clean up __sync() definition Thread-Topic: [PATCH v2 07/36] MIPS: barrier: Clean up __sync() definition Thread-Index: AQHVeKKfcgi1tDGHuEGmrvUl70AtLw== Date: Tue, 1 Oct 2019 21:53:13 +0000 Message-ID: <20191001215249.4157062-8-paul.burton@mips.com> References: <20191001215249.4157062-1-paul.burton@mips.com> In-Reply-To: <20191001215249.4157062-1-paul.burton@mips.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BYAPR07CA0093.namprd07.prod.outlook.com (2603:10b6:a03:12b::34) To MWHPR2201MB1277.namprd22.prod.outlook.com (2603:10b6:301:18::12) authentication-results: spf=none (sender IP is ) smtp.mailfrom=pburton@wavecomp.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.23.0 x-originating-ip: [12.94.197.246] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: aff50c2e-5e22-4610-195f-08d746b9c1cb x-ms-traffictypediagnostic: MWHPR2201MB1439: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:5236; x-forefront-prvs: 0177904E6B x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(396003)(39840400004)(366004)(346002)(136003)(376002)(199004)(189003)(43544003)(186003)(11346002)(6512007)(1076003)(486006)(66066001)(26005)(6486002)(316002)(2906002)(5640700003)(6436002)(44832011)(42882007)(25786009)(446003)(6916009)(386003)(6506007)(76176011)(478600001)(52116002)(2616005)(102836004)(476003)(5660300002)(305945005)(64756008)(66446008)(2351001)(7736002)(99286004)(54906003)(14454004)(71190400001)(71200400001)(66946007)(66556008)(66476007)(36756003)(3846002)(6116002)(107886003)(4326008)(50226002)(81166006)(81156014)(8676002)(2501003)(8936002)(256004);DIR:OUT;SFP:1102;SCL:1;SRVR:MWHPR2201MB1439;H:MWHPR2201MB1277.namprd22.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; received-spf: None (protection.outlook.com: wavecomp.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: Ihd+JeTVdMlc5S8OM7tq7GWLDo++5uSwu+UeTrHk8FQZuORS5qjelJaPbC6Rz3fFoQPE5qlqN2KhuYSnkjelo8WiRqFCDSG3YC94A+HZ4bywxegQTazyKdvVSyTHp6spuoWCExCL6pd63mOp8+59P5HY6JNUFeOTwiurU7Wo78TDFzsQb9AjnJ3Wt26+SWNdVLjsJ5dHvKavTdcgvpahJFHWltdH97WVI9l0BevcR2PPX2SFV+R+YM04+J5yCKvAQbm+WWbWy/71G3b8i8Di0q6duyfBo8M8Uh5eOW6NNbSAE5IWRVjMECcVxkpHTWxjZaOvIWzAJZf7KrxTBdoh8SScqyRSZu6dUBsLLA8IPHJ4CgcJLAvECyv7xPBBOUcKIX4d77LWCoVlaj0Xu37wN9je7LVUSC9aYyuD6Xdu4ps= MIME-Version: 1.0 X-OriginatorOrg: mips.com X-MS-Exchange-CrossTenant-Network-Message-Id: aff50c2e-5e22-4610-195f-08d746b9c1cb X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Oct 2019 21:53:13.2130 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 463607d3-1db3-40a0-8a29-970c56230104 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: OlH20ZiEk46gHlE5jOmgKg8OsUx+JQ3LBJL8aO5s/lxEWiNQd5FVwTh1RFGSTaeALY/o1xIjQ+HFr+/5ndL5vQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR2201MB1439 Sender: linux-mips-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-mips@vger.kernel.org Implement __sync() using the new __SYNC() infrastructure, which will take care of not emitting an instruction for old R3k CPUs that don't support it. The only behavioral difference is that __sync() will now provide a compiler barrier on these old CPUs, but that seems like reasonable behavior anyway. Signed-off-by: Paul Burton --- Changes in v2: None arch/mips/include/asm/barrier.h | 18 ++++-------------- 1 file changed, 4 insertions(+), 14 deletions(-) diff --git a/arch/mips/include/asm/barrier.h b/arch/mips/include/asm/barrier.h index 657ec01120a4..a117c6d95038 100644 --- a/arch/mips/include/asm/barrier.h +++ b/arch/mips/include/asm/barrier.h @@ -11,20 +11,10 @@ #include #include -#ifdef CONFIG_CPU_HAS_SYNC -#define __sync() \ - __asm__ __volatile__( \ - ".set push\n\t" \ - ".set noreorder\n\t" \ - ".set mips2\n\t" \ - "sync\n\t" \ - ".set pop" \ - : /* no output */ \ - : /* no input */ \ - : "memory") -#else -#define __sync() do { } while(0) -#endif +static inline void __sync(void) +{ + asm volatile(__SYNC(full, always) ::: "memory"); +} static inline void rmb(void) { From patchwork Tue Oct 1 21:53:14 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Burton X-Patchwork-Id: 11169775 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 13AA013B1 for ; Tue, 1 Oct 2019 21:53:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id DBB6E2133F for ; Tue, 1 Oct 2019 21:53:17 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=wavecomp.com header.i=@wavecomp.com header.b="KGRY0GUy" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728064AbfJAVxR (ORCPT ); Tue, 1 Oct 2019 17:53:17 -0400 Received: from mail-eopbgr730127.outbound.protection.outlook.com ([40.107.73.127]:55809 "EHLO NAM05-DM3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728045AbfJAVxR (ORCPT ); Tue, 1 Oct 2019 17:53:17 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=DxYVwHSaAZgQICLEgrI4keSQVVIr81dTFIU+I2m2AVzbcIVlgP4ibsGnIhtBe3RPcTK6efI4DO7ZOydv7irHgH+WpFKYQtigkDweC0BCSqRCZ+gK4D5889n7cP69gpy8VEcp9GjFRRcT8XYiwjqSrH5BmwKzf9iVUzdBkZ2DgILY4MskTisK6rfPsCfbkgY5fwqr5PGPatt0uU5tdSLFDgegxl2I4/LIA+7hQZcINDapZOJcDW5LFuQUlDxg9SSzvL1cQoGlrUqGzUIrjvHhhmL82HN20XrZANsHTyUmLM7sN9BOeZZ8CGEessAkQXk4Hd0aYZs9zjvNnKXoK6TtLg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=1cwWhMELYLQJHbgf/OCb1XluOho2mW2rRT2RERpUepI=; b=Zp2I1Si+kNoT12J6BjO8dz8yim6T3bQNGoTXXjVA+2vKRR8+2afZBiUTT0xvEvk7sW973LpbeAohdow0Nehltxl9NdwPgjpRElCfAW1LQzeyunBQURDwDw9eV1QIriOhg4zuUWvqNFEvTk0S6y3ecZiAdltaW8HiDnKf18cVZ4pEhKuCywmTPdFdkVEo0mRITIFb3vFNPFcmtLuPqH5syhs1nmNmdDRrXISEq/piiOV3bH7nmNtdRvwOsJY6DygyTOffcHOwRGlia4sna7iImIsYD60bBjRfU5/C97CIm/vzYulKDOrHql4dfSNDgXczE60IwJ+22qmhMWA37tYoLQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wavecomp.com; dmarc=pass action=none header.from=mips.com; dkim=pass header.d=mips.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wavecomp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=1cwWhMELYLQJHbgf/OCb1XluOho2mW2rRT2RERpUepI=; b=KGRY0GUyQ8/YESfw8tLKenT2QV5lszqfZibgpJ5HU9NH1PGI+Iv8gK4o6wulN7I7GDZwY/OCEmPuQKPqOIuj4AsiXxgjh14mZUeQsDj3oP8Hjz8Vuwez0TI4edFFq3kTiYgoHtJd2mU2AJnKaipsuEKnbZJcaLdMIGqQOdYUdps= Received: from MWHPR2201MB1277.namprd22.prod.outlook.com (10.172.60.12) by MWHPR2201MB1439.namprd22.prod.outlook.com (10.174.169.166) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2305.20; Tue, 1 Oct 2019 21:53:14 +0000 Received: from MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481]) by MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481%2]) with mapi id 15.20.2305.022; Tue, 1 Oct 2019 21:53:14 +0000 From: Paul Burton To: "linux-mips@vger.kernel.org" CC: Huacai Chen , Jiaxun Yang , "linux-kernel@vger.kernel.org" , Paul Burton Subject: [PATCH v2 08/36] MIPS: barrier: Clean up sync_ginv() Thread-Topic: [PATCH v2 08/36] MIPS: barrier: Clean up sync_ginv() Thread-Index: AQHVeKKfhd9UvyaMP0GSx91fZ7LfQg== Date: Tue, 1 Oct 2019 21:53:14 +0000 Message-ID: <20191001215249.4157062-9-paul.burton@mips.com> References: <20191001215249.4157062-1-paul.burton@mips.com> In-Reply-To: <20191001215249.4157062-1-paul.burton@mips.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BYAPR07CA0093.namprd07.prod.outlook.com (2603:10b6:a03:12b::34) To MWHPR2201MB1277.namprd22.prod.outlook.com (2603:10b6:301:18::12) authentication-results: spf=none (sender IP is ) smtp.mailfrom=pburton@wavecomp.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.23.0 x-originating-ip: [12.94.197.246] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 5cb0c0b4-4343-4fc1-365f-08d746b9c254 x-ms-traffictypediagnostic: MWHPR2201MB1439: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:2958; x-forefront-prvs: 0177904E6B x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(396003)(39840400004)(366004)(346002)(136003)(376002)(199004)(189003)(186003)(11346002)(6512007)(1076003)(486006)(66066001)(26005)(6486002)(316002)(2906002)(5640700003)(6436002)(44832011)(42882007)(4744005)(25786009)(446003)(6916009)(386003)(6506007)(76176011)(478600001)(52116002)(2616005)(102836004)(476003)(5660300002)(305945005)(64756008)(66446008)(14444005)(2351001)(7736002)(99286004)(54906003)(14454004)(71190400001)(71200400001)(66946007)(66556008)(66476007)(36756003)(3846002)(6116002)(107886003)(4326008)(50226002)(81166006)(81156014)(8676002)(2501003)(8936002)(256004);DIR:OUT;SFP:1102;SCL:1;SRVR:MWHPR2201MB1439;H:MWHPR2201MB1277.namprd22.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; received-spf: None (protection.outlook.com: wavecomp.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: P8I4hq1ao6dyv71Ya/7C2qFkkFjtxocDUVTo/tCTd7m+huXOnqgfguLvT/0i+Hrt3GxgjKPOLaysF97PQl981VwSLC/qoIROImlzSRJ3eDmWRGOwzqlsdor6iw7x3aVa1pTDD83UsAfHdXsATK5bDYI7vOUcft3N1kDjxVWSDrY5gliOnC9xGimBme7quOWyoIplxIOAj+3jp+j0+CUR6NHQ/1SMb2yzc6MIYokQn3bEGuU+WsilQ/9B6Dp9AYCY/8EcRBb2KDWzzKzKjO+xcSAH0ItxMHsJA7L4S4l6ckWswcSbkg6CsJNaUpRwsRcVh1iRem0DbkfYmUqsp31Y47xCvvmYklBKu1lgc01M4Zpkkr9RdjSM9dYeA39YRgUtlBNNs7nFh4PNpkz7QfSpZQ4LQkLu2nOWDQTQHOJIblg= MIME-Version: 1.0 X-OriginatorOrg: mips.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5cb0c0b4-4343-4fc1-365f-08d746b9c254 X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Oct 2019 21:53:14.2046 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 463607d3-1db3-40a0-8a29-970c56230104 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: K8sZonESf0DLrqL1qqir9CJWY81GGZMkFK42TJPklgfrhKQe6H10Au6Vh+7p02kKefkn86/+cEjrbLZHG88Q3w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR2201MB1439 Sender: linux-mips-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-mips@vger.kernel.org Use the new __SYNC() infrastructure to implement sync_ginv(), for consistency with much of the rest of the asm/barrier.h. Signed-off-by: Paul Burton --- Changes in v2: None arch/mips/include/asm/barrier.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/mips/include/asm/barrier.h b/arch/mips/include/asm/barrier.h index a117c6d95038..c7e05e832da9 100644 --- a/arch/mips/include/asm/barrier.h +++ b/arch/mips/include/asm/barrier.h @@ -163,7 +163,7 @@ static inline void wmb(void) static inline void sync_ginv(void) { - asm volatile("sync\t%0" :: "i"(__SYNC_ginv)); + asm volatile(__SYNC(ginv, always)); } #include From patchwork Tue Oct 1 21:53:15 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Burton X-Patchwork-Id: 11169833 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D22B213B1 for ; Tue, 1 Oct 2019 21:55:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id A620121924 for ; Tue, 1 Oct 2019 21:55:12 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=wavecomp.com header.i=@wavecomp.com header.b="VckPaciM" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727196AbfJAVzM (ORCPT ); Tue, 1 Oct 2019 17:55:12 -0400 Received: from mail-eopbgr730127.outbound.protection.outlook.com ([40.107.73.127]:55809 "EHLO NAM05-DM3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728063AbfJAVxS (ORCPT ); Tue, 1 Oct 2019 17:53:18 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=CpOPtTu+XozlYl4hrNAObIDM65bgyFUAdYmEBP6DEiXrd2a9rb7W5umbRO6+4qHm2YhjDEKliq2ppiZYS0VXWU3TbKjyhj6e6DzsMkXmUDGM6U6SpGvkuq168ks+knCxRkUklBIBEGOoXQ1J/vma8aFRgucByxEiC9rUAFyR+u9jqo+c36pYzSSfD7vBSPH1jaffJzejV+LSgn+BugSKxOft3+uat6tuF+UrlRvYI5skmQpUejmDmnyH/BxM/pMMf5IlMythtGl/jFL2q4XEqyadU3RjaBAJjRFkgQp1mDxq/I4m0HczTewDikgpVYzmz4oyFUMixwIpWsLo6etwcg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=i0dgxjl7VAQ6KGAg/hOjwYcyvgPNvW+txUWQu7qTaU0=; b=IBlwgda7DQk1eLW27dqfa79U7gN72Gef1FIJLW8CLYVacB7epH2zxYjcwdPSwFlwAiL+IBqMEDbAPmasWTcx5pyRuRNlp/wWuMm/xnfYhuFY/0oEPTylfAmpkCoZg70AfQqRAdqkHHwE6gTGYrPr66hwMGeT74IKkUt6f42S8bO6WgIUxUoXWTLl8oxDhkbMblOZotc/2WPPf51qNPtw9f49nVdO884tcKUqBZ8oQJOKR+ouGpGHaZw8dsdZcUzSk4d+HSfm6ntLbUepXaGvBrPIdIaX5CQrVF9xzHfQJU4q0JeySduUh12NKZioEwfy+nJAqHncCwsO+ypWpE7UOA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wavecomp.com; dmarc=pass action=none header.from=mips.com; dkim=pass header.d=mips.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wavecomp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=i0dgxjl7VAQ6KGAg/hOjwYcyvgPNvW+txUWQu7qTaU0=; b=VckPaciM59iXdlx2yfk07ZeuPK6Kp6k/rhOmu3QwdfK3LNQWlNQdFkF0r2dCQKogH+lWTtTF3xs8M0NxtTOvVt2pF66qzZ+9VWPpURFFH1fsoP0TYPFuB6IYX8qSPys94fmb4s1xev4OkdkV31nAWm041V07DDmkyTY627xYA30= Received: from MWHPR2201MB1277.namprd22.prod.outlook.com (10.172.60.12) by MWHPR2201MB1439.namprd22.prod.outlook.com (10.174.169.166) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2305.20; Tue, 1 Oct 2019 21:53:16 +0000 Received: from MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481]) by MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481%2]) with mapi id 15.20.2305.022; Tue, 1 Oct 2019 21:53:16 +0000 From: Paul Burton To: "linux-mips@vger.kernel.org" CC: Huacai Chen , Jiaxun Yang , "linux-kernel@vger.kernel.org" , Paul Burton Subject: [PATCH v2 09/36] MIPS: atomic: Fix whitespace in ATOMIC_OP macros Thread-Topic: [PATCH v2 09/36] MIPS: atomic: Fix whitespace in ATOMIC_OP macros Thread-Index: AQHVeKKgGWCu9gnKukKr6j/vfu7GEg== Date: Tue, 1 Oct 2019 21:53:15 +0000 Message-ID: <20191001215249.4157062-10-paul.burton@mips.com> References: <20191001215249.4157062-1-paul.burton@mips.com> In-Reply-To: <20191001215249.4157062-1-paul.burton@mips.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BYAPR07CA0093.namprd07.prod.outlook.com (2603:10b6:a03:12b::34) To MWHPR2201MB1277.namprd22.prod.outlook.com (2603:10b6:301:18::12) authentication-results: spf=none (sender IP is ) smtp.mailfrom=pburton@wavecomp.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.23.0 x-originating-ip: [12.94.197.246] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 096b7edd-2821-4e59-fffa-08d746b9c35a x-ms-traffictypediagnostic: MWHPR2201MB1439: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:1388; x-forefront-prvs: 0177904E6B x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(396003)(39840400004)(366004)(346002)(136003)(376002)(199004)(189003)(186003)(11346002)(6512007)(1076003)(486006)(66066001)(26005)(6486002)(316002)(2906002)(5640700003)(6436002)(44832011)(42882007)(25786009)(446003)(6916009)(386003)(6506007)(76176011)(478600001)(52116002)(2616005)(102836004)(476003)(5660300002)(305945005)(64756008)(66446008)(14444005)(2351001)(7736002)(99286004)(54906003)(14454004)(71190400001)(71200400001)(66946007)(66556008)(66476007)(36756003)(3846002)(6116002)(107886003)(4326008)(50226002)(81166006)(81156014)(8676002)(2501003)(8936002)(256004);DIR:OUT;SFP:1102;SCL:1;SRVR:MWHPR2201MB1439;H:MWHPR2201MB1277.namprd22.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; received-spf: None (protection.outlook.com: wavecomp.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: Vq5wucUqF0O6RiYp4ZItsrJ1duNuX75rvrw0bxl58Bo4YlYmq4xVS5KGQu51Lk499Wgi90r6qqpTCAZ8+SKxXc3sek5C9luZJibgSgof9sqsot7yGWMvRDpMgkm8+I7MiLIAiLjhv95HPgr7djc2lDbi/WTIPq/QYv/VPoCGXcLMHgzwt8c+I/NuKwAJlGMWpKbfUgUINz4583WNJzpXHKx1ROpgZRjWYIIQbj6YOest0oq7T+Qa5kaDNsZGhj7kVR8EY8KsrA25y33OGHZG40cvN08EwV8Jn9KYiUD9Y92DL7jPkA18JwUBm8KB0X+j3BhrkIC2IfdUhmToFDcFaWAl6GjebgR6duQQGYnXlxNzJpWbMGyVQjc8kjVWgM7PLhlnBzq8LDnyaD8+NNG+tc5Y4MZtPFTHWveXeGsKMLM= MIME-Version: 1.0 X-OriginatorOrg: mips.com X-MS-Exchange-CrossTenant-Network-Message-Id: 096b7edd-2821-4e59-fffa-08d746b9c35a X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Oct 2019 21:53:15.8418 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 463607d3-1db3-40a0-8a29-970c56230104 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 0v+fwGJCHWtF8r9dOyOqYQoL5l0h+nvcIeUbhVYoYBIOD7jkU65pUsWOLxQGx9u1BMh16LuyyhWGBUaDNvO98A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR2201MB1439 Sender: linux-mips-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-mips@vger.kernel.org We define macros in asm/atomic.h which end each line with space characters before a backslash to continue on the next line. Remove the space characters leaving tabs as the whitespace used for conformity with coding convention. Signed-off-by: Paul Burton --- Changes in v2: None arch/mips/include/asm/atomic.h | 184 ++++++++++++++++----------------- 1 file changed, 92 insertions(+), 92 deletions(-) diff --git a/arch/mips/include/asm/atomic.h b/arch/mips/include/asm/atomic.h index 7578c807ef98..2d2a8a74c51b 100644 --- a/arch/mips/include/asm/atomic.h +++ b/arch/mips/include/asm/atomic.h @@ -42,102 +42,102 @@ */ #define atomic_set(v, i) WRITE_ONCE((v)->counter, (i)) -#define ATOMIC_OP(op, c_op, asm_op) \ -static __inline__ void atomic_##op(int i, atomic_t * v) \ -{ \ - if (kernel_uses_llsc) { \ - int temp; \ - \ - loongson_llsc_mb(); \ - __asm__ __volatile__( \ - " .set push \n" \ - " .set "MIPS_ISA_LEVEL" \n" \ - "1: ll %0, %1 # atomic_" #op " \n" \ - " " #asm_op " %0, %2 \n" \ - " sc %0, %1 \n" \ - "\t" __SC_BEQZ "%0, 1b \n" \ - " .set pop \n" \ - : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (v->counter) \ - : "Ir" (i) : __LLSC_CLOBBER); \ - } else { \ - unsigned long flags; \ - \ - raw_local_irq_save(flags); \ - v->counter c_op i; \ - raw_local_irq_restore(flags); \ - } \ +#define ATOMIC_OP(op, c_op, asm_op) \ +static __inline__ void atomic_##op(int i, atomic_t * v) \ +{ \ + if (kernel_uses_llsc) { \ + int temp; \ + \ + loongson_llsc_mb(); \ + __asm__ __volatile__( \ + " .set push \n" \ + " .set "MIPS_ISA_LEVEL" \n" \ + "1: ll %0, %1 # atomic_" #op " \n" \ + " " #asm_op " %0, %2 \n" \ + " sc %0, %1 \n" \ + "\t" __SC_BEQZ "%0, 1b \n" \ + " .set pop \n" \ + : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (v->counter) \ + : "Ir" (i) : __LLSC_CLOBBER); \ + } else { \ + unsigned long flags; \ + \ + raw_local_irq_save(flags); \ + v->counter c_op i; \ + raw_local_irq_restore(flags); \ + } \ } -#define ATOMIC_OP_RETURN(op, c_op, asm_op) \ -static __inline__ int atomic_##op##_return_relaxed(int i, atomic_t * v) \ -{ \ - int result; \ - \ - if (kernel_uses_llsc) { \ - int temp; \ - \ - loongson_llsc_mb(); \ - __asm__ __volatile__( \ - " .set push \n" \ - " .set "MIPS_ISA_LEVEL" \n" \ - "1: ll %1, %2 # atomic_" #op "_return \n" \ - " " #asm_op " %0, %1, %3 \n" \ - " sc %0, %2 \n" \ - "\t" __SC_BEQZ "%0, 1b \n" \ - " " #asm_op " %0, %1, %3 \n" \ - " .set pop \n" \ - : "=&r" (result), "=&r" (temp), \ - "+" GCC_OFF_SMALL_ASM() (v->counter) \ - : "Ir" (i) : __LLSC_CLOBBER); \ - } else { \ - unsigned long flags; \ - \ - raw_local_irq_save(flags); \ - result = v->counter; \ - result c_op i; \ - v->counter = result; \ - raw_local_irq_restore(flags); \ - } \ - \ - return result; \ +#define ATOMIC_OP_RETURN(op, c_op, asm_op) \ +static __inline__ int atomic_##op##_return_relaxed(int i, atomic_t * v) \ +{ \ + int result; \ + \ + if (kernel_uses_llsc) { \ + int temp; \ + \ + loongson_llsc_mb(); \ + __asm__ __volatile__( \ + " .set push \n" \ + " .set "MIPS_ISA_LEVEL" \n" \ + "1: ll %1, %2 # atomic_" #op "_return \n" \ + " " #asm_op " %0, %1, %3 \n" \ + " sc %0, %2 \n" \ + "\t" __SC_BEQZ "%0, 1b \n" \ + " " #asm_op " %0, %1, %3 \n" \ + " .set pop \n" \ + : "=&r" (result), "=&r" (temp), \ + "+" GCC_OFF_SMALL_ASM() (v->counter) \ + : "Ir" (i) : __LLSC_CLOBBER); \ + } else { \ + unsigned long flags; \ + \ + raw_local_irq_save(flags); \ + result = v->counter; \ + result c_op i; \ + v->counter = result; \ + raw_local_irq_restore(flags); \ + } \ + \ + return result; \ } -#define ATOMIC_FETCH_OP(op, c_op, asm_op) \ -static __inline__ int atomic_fetch_##op##_relaxed(int i, atomic_t * v) \ -{ \ - int result; \ - \ - if (kernel_uses_llsc) { \ - int temp; \ - \ - loongson_llsc_mb(); \ - __asm__ __volatile__( \ - " .set push \n" \ - " .set "MIPS_ISA_LEVEL" \n" \ - "1: ll %1, %2 # atomic_fetch_" #op " \n" \ - " " #asm_op " %0, %1, %3 \n" \ - " sc %0, %2 \n" \ - "\t" __SC_BEQZ "%0, 1b \n" \ - " .set pop \n" \ - " move %0, %1 \n" \ - : "=&r" (result), "=&r" (temp), \ - "+" GCC_OFF_SMALL_ASM() (v->counter) \ - : "Ir" (i) : __LLSC_CLOBBER); \ - } else { \ - unsigned long flags; \ - \ - raw_local_irq_save(flags); \ - result = v->counter; \ - v->counter c_op i; \ - raw_local_irq_restore(flags); \ - } \ - \ - return result; \ +#define ATOMIC_FETCH_OP(op, c_op, asm_op) \ +static __inline__ int atomic_fetch_##op##_relaxed(int i, atomic_t * v) \ +{ \ + int result; \ + \ + if (kernel_uses_llsc) { \ + int temp; \ + \ + loongson_llsc_mb(); \ + __asm__ __volatile__( \ + " .set push \n" \ + " .set "MIPS_ISA_LEVEL" \n" \ + "1: ll %1, %2 # atomic_fetch_" #op " \n" \ + " " #asm_op " %0, %1, %3 \n" \ + " sc %0, %2 \n" \ + "\t" __SC_BEQZ "%0, 1b \n" \ + " .set pop \n" \ + " move %0, %1 \n" \ + : "=&r" (result), "=&r" (temp), \ + "+" GCC_OFF_SMALL_ASM() (v->counter) \ + : "Ir" (i) : __LLSC_CLOBBER); \ + } else { \ + unsigned long flags; \ + \ + raw_local_irq_save(flags); \ + result = v->counter; \ + v->counter c_op i; \ + raw_local_irq_restore(flags); \ + } \ + \ + return result; \ } -#define ATOMIC_OPS(op, c_op, asm_op) \ - ATOMIC_OP(op, c_op, asm_op) \ - ATOMIC_OP_RETURN(op, c_op, asm_op) \ +#define ATOMIC_OPS(op, c_op, asm_op) \ + ATOMIC_OP(op, c_op, asm_op) \ + ATOMIC_OP_RETURN(op, c_op, asm_op) \ ATOMIC_FETCH_OP(op, c_op, asm_op) ATOMIC_OPS(add, +=, addu) @@ -149,8 +149,8 @@ ATOMIC_OPS(sub, -=, subu) #define atomic_fetch_sub_relaxed atomic_fetch_sub_relaxed #undef ATOMIC_OPS -#define ATOMIC_OPS(op, c_op, asm_op) \ - ATOMIC_OP(op, c_op, asm_op) \ +#define ATOMIC_OPS(op, c_op, asm_op) \ + ATOMIC_OP(op, c_op, asm_op) \ ATOMIC_FETCH_OP(op, c_op, asm_op) ATOMIC_OPS(and, &=, and) From patchwork Tue Oct 1 21:53:16 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Burton X-Patchwork-Id: 11169779 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A578213B1 for ; Tue, 1 Oct 2019 21:53:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 7A1922133F for ; Tue, 1 Oct 2019 21:53:21 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=wavecomp.com header.i=@wavecomp.com header.b="EbH/L4Mh" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728083AbfJAVxV (ORCPT ); Tue, 1 Oct 2019 17:53:21 -0400 Received: from mail-eopbgr730127.outbound.protection.outlook.com ([40.107.73.127]:55809 "EHLO NAM05-DM3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728078AbfJAVxT (ORCPT ); Tue, 1 Oct 2019 17:53:19 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=a+poWWrzh6BF6Tj9lPI1WRAV6nUD13ps4VfavvOH/FOiIJiWqaaybvbsBw7BaDSK1PjUuyKH8R9C1+JnGKtoDQvs9IQP8z+Dw4nlHtDvN4VS2dw5DpPji97nWxbS7Gi8VKQkJmtYsoUpSyvtkDDIMQrxcqSNRc/TEa4zo6Tx3IgOCzbaZTciVuqhdM8GeOaFyBADsQUS9DiI7phX3LqVcGrc6ju/+gdTLzepj8wNx++V69pqBGATK+w3UmUUkMuhe8ZlGhnaMNq7r1dHR90/PQBY0ooucjnNfv3roEVWgh7luCcoqNzCapVU4ft2GKGZONnuPv2DoK9Z5toRdl1zpA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=hlM+lUTj9OAkDM61xJmFNZDp9Ocv+/vFn06O1gXXcWQ=; b=OulNr0HgDI8RpiCv7avlBh/yoPJqZJJRf+tlwgMIMrwyh9u8/NH8d0LbUxDwI4KYFCxow5Q9sVzcEDqdybeyFJMT11ilpy7U8PEkcgJNPbv3Hvovm1W/Rc8Ezd4jV3S1j8L2kF+Z1RyyQZj7HjVhagCcIgx1JjtrsGCMZ1im/jXvThkej/AexCUulsdRgqfOzNbclubKb4ysbbl82K+cwYiBc58yq8ioJ7AUT6Ey0EVKR02zf71dJT02XA2aUMVuj0FUzASmc3t5OSYiLqRZpi5BiSEJtRRCCyFmUUOgHTv3YRM8C/UEE+z3dA/3C4dqoOCe/FyTKaIqqkjpeUwGnw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wavecomp.com; dmarc=pass action=none header.from=mips.com; dkim=pass header.d=mips.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wavecomp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=hlM+lUTj9OAkDM61xJmFNZDp9Ocv+/vFn06O1gXXcWQ=; b=EbH/L4Mh445+MzvHVOZFDRjZPUF21OGC1f33D39HXep5KzLrtFzvnpm3x2az+OILGw/Oh8bmsPvjs4qgy922U1o4neJ+pH269S6K9zggIUe8SxnyBuljTG+9sxkEQA0rBUNFuS7cruWWsj948kWbdRfr+DABWt3G3aUdqXzHXGM= Received: from MWHPR2201MB1277.namprd22.prod.outlook.com (10.172.60.12) by MWHPR2201MB1439.namprd22.prod.outlook.com (10.174.169.166) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2305.20; Tue, 1 Oct 2019 21:53:17 +0000 Received: from MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481]) by MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481%2]) with mapi id 15.20.2305.022; Tue, 1 Oct 2019 21:53:17 +0000 From: Paul Burton To: "linux-mips@vger.kernel.org" CC: Huacai Chen , Jiaxun Yang , "linux-kernel@vger.kernel.org" , Paul Burton Subject: [PATCH v2 10/36] MIPS: atomic: Handle !kernel_uses_llsc first Thread-Topic: [PATCH v2 10/36] MIPS: atomic: Handle !kernel_uses_llsc first Thread-Index: AQHVeKKhvkClO5qosUCaGdrXmajwZw== Date: Tue, 1 Oct 2019 21:53:16 +0000 Message-ID: <20191001215249.4157062-11-paul.burton@mips.com> References: <20191001215249.4157062-1-paul.burton@mips.com> In-Reply-To: <20191001215249.4157062-1-paul.burton@mips.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BYAPR07CA0093.namprd07.prod.outlook.com (2603:10b6:a03:12b::34) To MWHPR2201MB1277.namprd22.prod.outlook.com (2603:10b6:301:18::12) authentication-results: spf=none (sender IP is ) smtp.mailfrom=pburton@wavecomp.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.23.0 x-originating-ip: [12.94.197.246] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 22f1026f-7b7c-42db-359e-08d746b9c405 x-ms-traffictypediagnostic: MWHPR2201MB1439: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:989; x-forefront-prvs: 0177904E6B x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(396003)(39840400004)(366004)(346002)(136003)(376002)(199004)(189003)(186003)(11346002)(6512007)(1076003)(486006)(66066001)(26005)(6486002)(316002)(2906002)(5640700003)(6436002)(44832011)(42882007)(25786009)(446003)(6916009)(386003)(6506007)(76176011)(478600001)(52116002)(2616005)(102836004)(476003)(5660300002)(305945005)(64756008)(66446008)(14444005)(2351001)(7736002)(99286004)(54906003)(14454004)(71190400001)(71200400001)(66946007)(66556008)(66476007)(36756003)(3846002)(6116002)(107886003)(4326008)(50226002)(81166006)(81156014)(8676002)(2501003)(8936002)(256004);DIR:OUT;SFP:1102;SCL:1;SRVR:MWHPR2201MB1439;H:MWHPR2201MB1277.namprd22.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; received-spf: None (protection.outlook.com: wavecomp.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: Mc/uP0zYlFO1yz3lm4347BeOFBM0qilgWQ2dx18Jbup+O59FSqM2/PnOPd708KJ5WnW8+XCPs1+b8+hXgHclmvrVyyWQOSvSLf1JdmiaXDa8TZ1eBESMAWfV33I0BiwBQ4tgxr7QTC/cMGRLoS3BD1lBPbJwnFeiXDjFd54FYiIyh1b7IcfPX+4uIP/YgQ8lZ3zgzUbDXCZQCSaQ+MYGOh8c9wxhAAfCNHBEpesIEpV8rBaiH5ywg9mcxJ1waUZ0CeRLjsx7z83zzq3FKeNKJUfcnKyGYwqJjpHTCv1b6gIjHAvKsiJlTOtNNRWcxhRuYLiP7fvEq3XeGp3+g2opeiuPUTw8qlETXjjq+2jKu4yQTJ3MEN18IbnEWkERi4cJnkkCNNOUdFt5c/TJwcQoqQiw6aE1+EzFWgqbe+26oYg= MIME-Version: 1.0 X-OriginatorOrg: mips.com X-MS-Exchange-CrossTenant-Network-Message-Id: 22f1026f-7b7c-42db-359e-08d746b9c405 X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Oct 2019 21:53:17.0752 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 463607d3-1db3-40a0-8a29-970c56230104 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: NxqOt6hWJ8aqjDTY3cQu8qxjSJyBmEElHklu9FuJ6J5KVp+xEybqgKyJ5mSrbIfRDQ8ROEYpbi7pQS2mUN2vaQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR2201MB1439 Sender: linux-mips-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-mips@vger.kernel.org Handle the !kernel_uses_llsc path first in our ATOMIC_OP(), ATOMIC_OP_RETURN() & ATOMIC_FETCH_OP() macros & return from within the block. This allows us to de-indent the kernel_uses_llsc path by one level which will be useful when making further changes. Signed-off-by: Paul Burton --- Changes in v2: None arch/mips/include/asm/atomic.h | 99 +++++++++++++++++----------------- 1 file changed, 49 insertions(+), 50 deletions(-) diff --git a/arch/mips/include/asm/atomic.h b/arch/mips/include/asm/atomic.h index 2d2a8a74c51b..ace2ea005588 100644 --- a/arch/mips/include/asm/atomic.h +++ b/arch/mips/include/asm/atomic.h @@ -45,51 +45,36 @@ #define ATOMIC_OP(op, c_op, asm_op) \ static __inline__ void atomic_##op(int i, atomic_t * v) \ { \ - if (kernel_uses_llsc) { \ - int temp; \ + int temp; \ \ - loongson_llsc_mb(); \ - __asm__ __volatile__( \ - " .set push \n" \ - " .set "MIPS_ISA_LEVEL" \n" \ - "1: ll %0, %1 # atomic_" #op " \n" \ - " " #asm_op " %0, %2 \n" \ - " sc %0, %1 \n" \ - "\t" __SC_BEQZ "%0, 1b \n" \ - " .set pop \n" \ - : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (v->counter) \ - : "Ir" (i) : __LLSC_CLOBBER); \ - } else { \ + if (!kernel_uses_llsc) { \ unsigned long flags; \ \ raw_local_irq_save(flags); \ v->counter c_op i; \ raw_local_irq_restore(flags); \ + return; \ } \ + \ + loongson_llsc_mb(); \ + __asm__ __volatile__( \ + " .set push \n" \ + " .set " MIPS_ISA_LEVEL " \n" \ + "1: ll %0, %1 # atomic_" #op " \n" \ + " " #asm_op " %0, %2 \n" \ + " sc %0, %1 \n" \ + "\t" __SC_BEQZ "%0, 1b \n" \ + " .set pop \n" \ + : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (v->counter) \ + : "Ir" (i) : __LLSC_CLOBBER); \ } #define ATOMIC_OP_RETURN(op, c_op, asm_op) \ static __inline__ int atomic_##op##_return_relaxed(int i, atomic_t * v) \ { \ - int result; \ - \ - if (kernel_uses_llsc) { \ - int temp; \ + int temp, result; \ \ - loongson_llsc_mb(); \ - __asm__ __volatile__( \ - " .set push \n" \ - " .set "MIPS_ISA_LEVEL" \n" \ - "1: ll %1, %2 # atomic_" #op "_return \n" \ - " " #asm_op " %0, %1, %3 \n" \ - " sc %0, %2 \n" \ - "\t" __SC_BEQZ "%0, 1b \n" \ - " " #asm_op " %0, %1, %3 \n" \ - " .set pop \n" \ - : "=&r" (result), "=&r" (temp), \ - "+" GCC_OFF_SMALL_ASM() (v->counter) \ - : "Ir" (i) : __LLSC_CLOBBER); \ - } else { \ + if (!kernel_uses_llsc) { \ unsigned long flags; \ \ raw_local_irq_save(flags); \ @@ -97,41 +82,55 @@ static __inline__ int atomic_##op##_return_relaxed(int i, atomic_t * v) \ result c_op i; \ v->counter = result; \ raw_local_irq_restore(flags); \ + return result; \ } \ \ + loongson_llsc_mb(); \ + __asm__ __volatile__( \ + " .set push \n" \ + " .set " MIPS_ISA_LEVEL " \n" \ + "1: ll %1, %2 # atomic_" #op "_return \n" \ + " " #asm_op " %0, %1, %3 \n" \ + " sc %0, %2 \n" \ + "\t" __SC_BEQZ "%0, 1b \n" \ + " " #asm_op " %0, %1, %3 \n" \ + " .set pop \n" \ + : "=&r" (result), "=&r" (temp), \ + "+" GCC_OFF_SMALL_ASM() (v->counter) \ + : "Ir" (i) : __LLSC_CLOBBER); \ + \ return result; \ } #define ATOMIC_FETCH_OP(op, c_op, asm_op) \ static __inline__ int atomic_fetch_##op##_relaxed(int i, atomic_t * v) \ { \ - int result; \ + int temp, result; \ \ - if (kernel_uses_llsc) { \ - int temp; \ - \ - loongson_llsc_mb(); \ - __asm__ __volatile__( \ - " .set push \n" \ - " .set "MIPS_ISA_LEVEL" \n" \ - "1: ll %1, %2 # atomic_fetch_" #op " \n" \ - " " #asm_op " %0, %1, %3 \n" \ - " sc %0, %2 \n" \ - "\t" __SC_BEQZ "%0, 1b \n" \ - " .set pop \n" \ - " move %0, %1 \n" \ - : "=&r" (result), "=&r" (temp), \ - "+" GCC_OFF_SMALL_ASM() (v->counter) \ - : "Ir" (i) : __LLSC_CLOBBER); \ - } else { \ + if (!kernel_uses_llsc) { \ unsigned long flags; \ \ raw_local_irq_save(flags); \ result = v->counter; \ v->counter c_op i; \ raw_local_irq_restore(flags); \ + return result; \ } \ \ + loongson_llsc_mb(); \ + __asm__ __volatile__( \ + " .set push \n" \ + " .set "MIPS_ISA_LEVEL" \n" \ + "1: ll %1, %2 # atomic_fetch_" #op " \n" \ + " " #asm_op " %0, %1, %3 \n" \ + " sc %0, %2 \n" \ + "\t" __SC_BEQZ "%0, 1b \n" \ + " .set pop \n" \ + " move %0, %1 \n" \ + : "=&r" (result), "=&r" (temp), \ + "+" GCC_OFF_SMALL_ASM() (v->counter) \ + : "Ir" (i) : __LLSC_CLOBBER); \ + \ return result; \ } From patchwork Tue Oct 1 21:53:18 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Burton X-Patchwork-Id: 11169831 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8D6CB1599 for ; Tue, 1 Oct 2019 21:55:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 5742221A4A for ; Tue, 1 Oct 2019 21:55:11 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=wavecomp.com header.i=@wavecomp.com header.b="ieMwNRpc" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728104AbfJAVxZ (ORCPT ); Tue, 1 Oct 2019 17:53:25 -0400 Received: from mail-eopbgr730127.outbound.protection.outlook.com ([40.107.73.127]:55809 "EHLO NAM05-DM3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728085AbfJAVxW (ORCPT ); Tue, 1 Oct 2019 17:53:22 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=f0DoO3aqfI3qnBGW6jVYRjRuRWPFnY7L3RK6COyntOjlVT4kX4iycgoE6CXh+9kliTFcfuKUQzKR+wsqRiBRErQRlqJ5NBupiYrNurXOYrsOMbHJLQU+c3Jjo/3FtoIoBxAVlI2DqDOY8XaK5lOL5MUfVZKbYTNbJb6KiQpr23AzbIKZZdUZe0GD4a+57LjeAoVlCtAxkCJYejzqSQQw/eH+G6xMOS6isq8aPm4qIuFmMqcVjqaA30jrNcWjEjCfFenvu5rwxp5TDfEWqVMYhbBXrq7mL713P0+P5DUqA6M+qRgYA+q4mNx6ajpF9bQ/EsIbNTGaXs9ecUT9ZxKdCg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3so5Qt3Lo3QN1t8xjuC8bWCMsfm5kOHTcsNrsjDRuaE=; b=dUOyj32QTJ/Bu4Ez5ETRHQizTzXipwTJQUH/xi2wQ/1XwNZbHhcBl4QZtZ+3pPvVIAGIjWxrmdrDzC3KC4jkTzgYEBYyEIKG7r1j1OnqXDgIGfuCLITKs0FkxLQh2B4g9Nl53pVYpZjIzRP9cRnVAVbSCwC1nJnwXVw6RdQAMV+OU7U+bbdVcNctLwew+rX1MPuqSyAt2ZtKxGiGgzn53rhUDMNa8OPOogNXLHwgi9rJVyFzbK+xHsb3kRNsytt3lk+QCQie1n17GG9/4NK1y1J03xVOC92bG0UdcNRQ6MQczl1e0BpJ8EZOD50dozofl6Vl6ZHl8++BRYOoWQVZLQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wavecomp.com; dmarc=pass action=none header.from=mips.com; dkim=pass header.d=mips.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wavecomp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3so5Qt3Lo3QN1t8xjuC8bWCMsfm5kOHTcsNrsjDRuaE=; b=ieMwNRpcgb/LAleG5e07BhBsL0bmbckmgs874hBXbIA76d2FxQuKBtJJc7tM8pFwrHghCcd8oYZ6jvN8Y6F+eq6/pVwK1jitA5cURxDKw521tR/54Ysw9m7O+dG15tgjZG4z0HR774vDFcJu7Bz9vNXYA5IDX382316ed3Qmys0= Received: from MWHPR2201MB1277.namprd22.prod.outlook.com (10.172.60.12) by MWHPR2201MB1439.namprd22.prod.outlook.com (10.174.169.166) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2305.20; Tue, 1 Oct 2019 21:53:18 +0000 Received: from MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481]) by MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481%2]) with mapi id 15.20.2305.022; Tue, 1 Oct 2019 21:53:18 +0000 From: Paul Burton To: "linux-mips@vger.kernel.org" CC: Huacai Chen , Jiaxun Yang , "linux-kernel@vger.kernel.org" , Paul Burton Subject: [PATCH v2 11/36] MIPS: atomic: Use one macro to generate 32b & 64b functions Thread-Topic: [PATCH v2 11/36] MIPS: atomic: Use one macro to generate 32b & 64b functions Thread-Index: AQHVeKKirPSs9DL0w0uLssz8WQ7moQ== Date: Tue, 1 Oct 2019 21:53:18 +0000 Message-ID: <20191001215249.4157062-12-paul.burton@mips.com> References: <20191001215249.4157062-1-paul.burton@mips.com> In-Reply-To: <20191001215249.4157062-1-paul.burton@mips.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BYAPR07CA0093.namprd07.prod.outlook.com (2603:10b6:a03:12b::34) To MWHPR2201MB1277.namprd22.prod.outlook.com (2603:10b6:301:18::12) authentication-results: spf=none (sender IP is ) smtp.mailfrom=pburton@wavecomp.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.23.0 x-originating-ip: [12.94.197.246] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 572cc263-79bf-40c5-add9-08d746b9c4f2 x-ms-traffictypediagnostic: MWHPR2201MB1439: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:298; x-forefront-prvs: 0177904E6B x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(396003)(39840400004)(366004)(346002)(136003)(376002)(199004)(189003)(186003)(11346002)(6512007)(1076003)(486006)(66066001)(26005)(6486002)(316002)(2906002)(5640700003)(6436002)(44832011)(42882007)(25786009)(446003)(6916009)(386003)(6506007)(76176011)(478600001)(52116002)(2616005)(102836004)(476003)(5660300002)(305945005)(64756008)(66446008)(14444005)(2351001)(7736002)(99286004)(54906003)(14454004)(71190400001)(71200400001)(66946007)(66556008)(66476007)(36756003)(3846002)(6116002)(107886003)(4326008)(50226002)(81166006)(81156014)(8676002)(2501003)(8936002)(256004);DIR:OUT;SFP:1102;SCL:1;SRVR:MWHPR2201MB1439;H:MWHPR2201MB1277.namprd22.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; received-spf: None (protection.outlook.com: wavecomp.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: YiQrRRWydu3omWQFlxwJgMIn2nOkNS9odaPON+YiZR6TWeOQlXuV2YPWm0AhTWa+9hwF/PEqCzUwBBNHKBu6Tn/WACvlRxBFF51f167vKno4G5F+h7PAX9f2hrdi7xFuq9uh7bHNsLWzSQEhkzzfS96JNUG6FgWcgzBpFouBJV8qZ9iuRXtXeaZU7hWwQH3It+Nfkha/BVlAJGzoBImVuzQxzGHmHWFF5h0tIi7Vr5sOEnDyEHsTO/HlfwgDsjmjShwP1KRWrkZ7p5HAjMuUQdRoOz5OgGfDvF8cc6HBj5G/OHfupUEa8vIuxr62klC4bk1fb8xKy4CMRBb95sxNfJPxPzi0RQvkdyT7vguewluBphTf68mjf5ys60VEDK78PhNF4Acj0kS7z5xv1ll+4+kcDnDainOFRYFIYzXvrhA= MIME-Version: 1.0 X-OriginatorOrg: mips.com X-MS-Exchange-CrossTenant-Network-Message-Id: 572cc263-79bf-40c5-add9-08d746b9c4f2 X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Oct 2019 21:53:18.6565 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 463607d3-1db3-40a0-8a29-970c56230104 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: vPsZgScZZm/KXR0Psx9FdSVYSNHQibwqgGmi9FQYwGaAJ53G4LQ696Gp5vcwK5StMhYXCSfqKyGRG+X/LkzP8A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR2201MB1439 Sender: linux-mips-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-mips@vger.kernel.org Cut down on duplication by generalizing the ATOMIC_OP(), ATOMIC_OP_RETURN() & ATOMIC_FETCH_OP() macros to work for both 32b & 64b atomics, and removing the ATOMIC64_ variants. This ensures consistency between our atomic_* & atomic64_* functions. Signed-off-by: Paul Burton --- Changes in v2: None arch/mips/include/asm/atomic.h | 196 ++++++++------------------------- 1 file changed, 45 insertions(+), 151 deletions(-) diff --git a/arch/mips/include/asm/atomic.h b/arch/mips/include/asm/atomic.h index ace2ea005588..b834af5a7382 100644 --- a/arch/mips/include/asm/atomic.h +++ b/arch/mips/include/asm/atomic.h @@ -42,10 +42,10 @@ */ #define atomic_set(v, i) WRITE_ONCE((v)->counter, (i)) -#define ATOMIC_OP(op, c_op, asm_op) \ -static __inline__ void atomic_##op(int i, atomic_t * v) \ +#define ATOMIC_OP(pfx, op, type, c_op, asm_op, ll, sc) \ +static __inline__ void pfx##_##op(type i, pfx##_t * v) \ { \ - int temp; \ + type temp; \ \ if (!kernel_uses_llsc) { \ unsigned long flags; \ @@ -60,19 +60,19 @@ static __inline__ void atomic_##op(int i, atomic_t * v) \ __asm__ __volatile__( \ " .set push \n" \ " .set " MIPS_ISA_LEVEL " \n" \ - "1: ll %0, %1 # atomic_" #op " \n" \ + "1: " #ll " %0, %1 # " #pfx "_" #op " \n" \ " " #asm_op " %0, %2 \n" \ - " sc %0, %1 \n" \ + " " #sc " %0, %1 \n" \ "\t" __SC_BEQZ "%0, 1b \n" \ " .set pop \n" \ : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (v->counter) \ : "Ir" (i) : __LLSC_CLOBBER); \ } -#define ATOMIC_OP_RETURN(op, c_op, asm_op) \ -static __inline__ int atomic_##op##_return_relaxed(int i, atomic_t * v) \ +#define ATOMIC_OP_RETURN(pfx, op, type, c_op, asm_op, ll, sc) \ +static __inline__ type pfx##_##op##_return_relaxed(type i, pfx##_t * v) \ { \ - int temp, result; \ + type temp, result; \ \ if (!kernel_uses_llsc) { \ unsigned long flags; \ @@ -89,9 +89,9 @@ static __inline__ int atomic_##op##_return_relaxed(int i, atomic_t * v) \ __asm__ __volatile__( \ " .set push \n" \ " .set " MIPS_ISA_LEVEL " \n" \ - "1: ll %1, %2 # atomic_" #op "_return \n" \ + "1: " #ll " %1, %2 # " #pfx "_" #op "_return\n" \ " " #asm_op " %0, %1, %3 \n" \ - " sc %0, %2 \n" \ + " " #sc " %0, %2 \n" \ "\t" __SC_BEQZ "%0, 1b \n" \ " " #asm_op " %0, %1, %3 \n" \ " .set pop \n" \ @@ -102,8 +102,8 @@ static __inline__ int atomic_##op##_return_relaxed(int i, atomic_t * v) \ return result; \ } -#define ATOMIC_FETCH_OP(op, c_op, asm_op) \ -static __inline__ int atomic_fetch_##op##_relaxed(int i, atomic_t * v) \ +#define ATOMIC_FETCH_OP(pfx, op, type, c_op, asm_op, ll, sc) \ +static __inline__ type pfx##_fetch_##op##_relaxed(type i, pfx##_t * v) \ { \ int temp, result; \ \ @@ -120,10 +120,10 @@ static __inline__ int atomic_fetch_##op##_relaxed(int i, atomic_t * v) \ loongson_llsc_mb(); \ __asm__ __volatile__( \ " .set push \n" \ - " .set "MIPS_ISA_LEVEL" \n" \ - "1: ll %1, %2 # atomic_fetch_" #op " \n" \ + " .set " MIPS_ISA_LEVEL " \n" \ + "1: " #ll " %1, %2 # " #pfx "_fetch_" #op "\n" \ " " #asm_op " %0, %1, %3 \n" \ - " sc %0, %2 \n" \ + " " #sc " %0, %2 \n" \ "\t" __SC_BEQZ "%0, 1b \n" \ " .set pop \n" \ " move %0, %1 \n" \ @@ -134,32 +134,50 @@ static __inline__ int atomic_fetch_##op##_relaxed(int i, atomic_t * v) \ return result; \ } -#define ATOMIC_OPS(op, c_op, asm_op) \ - ATOMIC_OP(op, c_op, asm_op) \ - ATOMIC_OP_RETURN(op, c_op, asm_op) \ - ATOMIC_FETCH_OP(op, c_op, asm_op) +#define ATOMIC_OPS(pfx, op, type, c_op, asm_op, ll, sc) \ + ATOMIC_OP(pfx, op, type, c_op, asm_op, ll, sc) \ + ATOMIC_OP_RETURN(pfx, op, type, c_op, asm_op, ll, sc) \ + ATOMIC_FETCH_OP(pfx, op, type, c_op, asm_op, ll, sc) -ATOMIC_OPS(add, +=, addu) -ATOMIC_OPS(sub, -=, subu) +ATOMIC_OPS(atomic, add, int, +=, addu, ll, sc) +ATOMIC_OPS(atomic, sub, int, -=, subu, ll, sc) #define atomic_add_return_relaxed atomic_add_return_relaxed #define atomic_sub_return_relaxed atomic_sub_return_relaxed #define atomic_fetch_add_relaxed atomic_fetch_add_relaxed #define atomic_fetch_sub_relaxed atomic_fetch_sub_relaxed +#ifdef CONFIG_64BIT +ATOMIC_OPS(atomic64, add, s64, +=, daddu, lld, scd) +ATOMIC_OPS(atomic64, sub, s64, -=, dsubu, lld, scd) +# define atomic64_add_return_relaxed atomic64_add_return_relaxed +# define atomic64_sub_return_relaxed atomic64_sub_return_relaxed +# define atomic64_fetch_add_relaxed atomic64_fetch_add_relaxed +# define atomic64_fetch_sub_relaxed atomic64_fetch_sub_relaxed +#endif /* CONFIG_64BIT */ + #undef ATOMIC_OPS -#define ATOMIC_OPS(op, c_op, asm_op) \ - ATOMIC_OP(op, c_op, asm_op) \ - ATOMIC_FETCH_OP(op, c_op, asm_op) +#define ATOMIC_OPS(pfx, op, type, c_op, asm_op, ll, sc) \ + ATOMIC_OP(pfx, op, type, c_op, asm_op, ll, sc) \ + ATOMIC_FETCH_OP(pfx, op, type, c_op, asm_op, ll, sc) -ATOMIC_OPS(and, &=, and) -ATOMIC_OPS(or, |=, or) -ATOMIC_OPS(xor, ^=, xor) +ATOMIC_OPS(atomic, and, int, &=, and, ll, sc) +ATOMIC_OPS(atomic, or, int, |=, or, ll, sc) +ATOMIC_OPS(atomic, xor, int, ^=, xor, ll, sc) #define atomic_fetch_and_relaxed atomic_fetch_and_relaxed #define atomic_fetch_or_relaxed atomic_fetch_or_relaxed #define atomic_fetch_xor_relaxed atomic_fetch_xor_relaxed +#ifdef CONFIG_64BIT +ATOMIC_OPS(atomic64, and, s64, &=, and, lld, scd) +ATOMIC_OPS(atomic64, or, s64, |=, or, lld, scd) +ATOMIC_OPS(atomic64, xor, s64, ^=, xor, lld, scd) +# define atomic64_fetch_and_relaxed atomic64_fetch_and_relaxed +# define atomic64_fetch_or_relaxed atomic64_fetch_or_relaxed +# define atomic64_fetch_xor_relaxed atomic64_fetch_xor_relaxed +#endif + #undef ATOMIC_OPS #undef ATOMIC_FETCH_OP #undef ATOMIC_OP_RETURN @@ -243,130 +261,6 @@ static __inline__ int atomic_sub_if_positive(int i, atomic_t * v) */ #define atomic64_set(v, i) WRITE_ONCE((v)->counter, (i)) -#define ATOMIC64_OP(op, c_op, asm_op) \ -static __inline__ void atomic64_##op(s64 i, atomic64_t * v) \ -{ \ - if (kernel_uses_llsc) { \ - s64 temp; \ - \ - loongson_llsc_mb(); \ - __asm__ __volatile__( \ - " .set push \n" \ - " .set "MIPS_ISA_LEVEL" \n" \ - "1: lld %0, %1 # atomic64_" #op " \n" \ - " " #asm_op " %0, %2 \n" \ - " scd %0, %1 \n" \ - "\t" __SC_BEQZ "%0, 1b \n" \ - " .set pop \n" \ - : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (v->counter) \ - : "Ir" (i) : __LLSC_CLOBBER); \ - } else { \ - unsigned long flags; \ - \ - raw_local_irq_save(flags); \ - v->counter c_op i; \ - raw_local_irq_restore(flags); \ - } \ -} - -#define ATOMIC64_OP_RETURN(op, c_op, asm_op) \ -static __inline__ s64 atomic64_##op##_return_relaxed(s64 i, atomic64_t * v) \ -{ \ - s64 result; \ - \ - if (kernel_uses_llsc) { \ - s64 temp; \ - \ - loongson_llsc_mb(); \ - __asm__ __volatile__( \ - " .set push \n" \ - " .set "MIPS_ISA_LEVEL" \n" \ - "1: lld %1, %2 # atomic64_" #op "_return\n" \ - " " #asm_op " %0, %1, %3 \n" \ - " scd %0, %2 \n" \ - "\t" __SC_BEQZ "%0, 1b \n" \ - " " #asm_op " %0, %1, %3 \n" \ - " .set pop \n" \ - : "=&r" (result), "=&r" (temp), \ - "+" GCC_OFF_SMALL_ASM() (v->counter) \ - : "Ir" (i) : __LLSC_CLOBBER); \ - } else { \ - unsigned long flags; \ - \ - raw_local_irq_save(flags); \ - result = v->counter; \ - result c_op i; \ - v->counter = result; \ - raw_local_irq_restore(flags); \ - } \ - \ - return result; \ -} - -#define ATOMIC64_FETCH_OP(op, c_op, asm_op) \ -static __inline__ s64 atomic64_fetch_##op##_relaxed(s64 i, atomic64_t * v) \ -{ \ - s64 result; \ - \ - if (kernel_uses_llsc) { \ - s64 temp; \ - \ - loongson_llsc_mb(); \ - __asm__ __volatile__( \ - " .set push \n" \ - " .set "MIPS_ISA_LEVEL" \n" \ - "1: lld %1, %2 # atomic64_fetch_" #op "\n" \ - " " #asm_op " %0, %1, %3 \n" \ - " scd %0, %2 \n" \ - "\t" __SC_BEQZ "%0, 1b \n" \ - " move %0, %1 \n" \ - " .set pop \n" \ - : "=&r" (result), "=&r" (temp), \ - "+" GCC_OFF_SMALL_ASM() (v->counter) \ - : "Ir" (i) : __LLSC_CLOBBER); \ - } else { \ - unsigned long flags; \ - \ - raw_local_irq_save(flags); \ - result = v->counter; \ - v->counter c_op i; \ - raw_local_irq_restore(flags); \ - } \ - \ - return result; \ -} - -#define ATOMIC64_OPS(op, c_op, asm_op) \ - ATOMIC64_OP(op, c_op, asm_op) \ - ATOMIC64_OP_RETURN(op, c_op, asm_op) \ - ATOMIC64_FETCH_OP(op, c_op, asm_op) - -ATOMIC64_OPS(add, +=, daddu) -ATOMIC64_OPS(sub, -=, dsubu) - -#define atomic64_add_return_relaxed atomic64_add_return_relaxed -#define atomic64_sub_return_relaxed atomic64_sub_return_relaxed -#define atomic64_fetch_add_relaxed atomic64_fetch_add_relaxed -#define atomic64_fetch_sub_relaxed atomic64_fetch_sub_relaxed - -#undef ATOMIC64_OPS -#define ATOMIC64_OPS(op, c_op, asm_op) \ - ATOMIC64_OP(op, c_op, asm_op) \ - ATOMIC64_FETCH_OP(op, c_op, asm_op) - -ATOMIC64_OPS(and, &=, and) -ATOMIC64_OPS(or, |=, or) -ATOMIC64_OPS(xor, ^=, xor) - -#define atomic64_fetch_and_relaxed atomic64_fetch_and_relaxed -#define atomic64_fetch_or_relaxed atomic64_fetch_or_relaxed -#define atomic64_fetch_xor_relaxed atomic64_fetch_xor_relaxed - -#undef ATOMIC64_OPS -#undef ATOMIC64_FETCH_OP -#undef ATOMIC64_OP_RETURN -#undef ATOMIC64_OP - /* * atomic64_sub_if_positive - conditionally subtract integer from atomic * variable From patchwork Tue Oct 1 21:53:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Burton X-Patchwork-Id: 11169781 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 65C3F13B1 for ; Tue, 1 Oct 2019 21:53:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 398202133F for ; Tue, 1 Oct 2019 21:53:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=wavecomp.com header.i=@wavecomp.com header.b="XWSA86Cu" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728108AbfJAVx0 (ORCPT ); Tue, 1 Oct 2019 17:53:26 -0400 Received: from mail-eopbgr730113.outbound.protection.outlook.com ([40.107.73.113]:17171 "EHLO NAM05-DM3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728097AbfJAVxZ (ORCPT ); Tue, 1 Oct 2019 17:53:25 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=DVS/yz6ktY075wTCOFM3aP0aOjOzzkxNlKZDQ+UmYICT2KECbsRx07pE2XoxxMsUC4vismuWpkFuV3f/NdGiuh7YTjcKG54kzvRdZcSKT9ZgEPk6ZhQ+HSHtYWWKnqlPLv9K0QHgnwTexo0bgZqeqKTfkFbDqzZBh0lwgbN22VGAd9oKfuCLGkIQWeOIPYWmWwe2e839CYcU3agw7Re7m717HOqciNgu2To3hDQKxHCFhhGod8ywuypTJax+a05s76S2s+BmC+qMddPdph0Hl0DPLqdSSkazhk6ScCfilWnOFapxSnRmhE1p237BGe44BPoSaNZbNGnkVKPkAnrxdw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=J30odTe+XebbxzQL8i/yALQ5p9SuUs5yaDqAG4FQBl4=; b=fjqKrDF3G/2fmDQB47TT67lQsXMp4krhRR/Apgb/U4T03EJrDj6/mTfYn6gf/5a0RNvI5EfhRsmx8DSc2qyY2RzDhpl3lX2VqbVMMzmTJ+M9KG9lBvhJbztS+W1OHtShQTtMKXKizSkV7/nkSST01yKlnjdFjF3grHn5kOuBRe2ynQA0xMO4AubVc77wg6jQ2v+GDGCYJLMdEc8nzPWnT689BXN17en0R5Bee0rfa49duE8KhktwG8viaflQoz8H2h4yrfzOzBKOohaAZO2ctYRiEX7JgJXWGAOehlYRpSVC7YoAm5wb4Qi4e0Sa7SdhegTJlptn3KN3Y8zM/WdS2Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wavecomp.com; dmarc=pass action=none header.from=mips.com; dkim=pass header.d=mips.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wavecomp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=J30odTe+XebbxzQL8i/yALQ5p9SuUs5yaDqAG4FQBl4=; b=XWSA86CuFoEEgVAnhTFpSxBF0qsoxwiRdOi/vDHAc2aKDNedbLflDTXcWhLK3oH9QAwHXZpbst9JERxJzw490xi4ka7715HNFi8i74qFq+rwWA8Z70BZv2WEL3Ec9zwejxV4Wg8HtVFl7adsiJZgtLSyd3MP6eoBFBsWK/AmDTo= Received: from MWHPR2201MB1277.namprd22.prod.outlook.com (10.172.60.12) by MWHPR2201MB1439.namprd22.prod.outlook.com (10.174.169.166) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2305.20; Tue, 1 Oct 2019 21:53:20 +0000 Received: from MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481]) by MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481%2]) with mapi id 15.20.2305.022; Tue, 1 Oct 2019 21:53:20 +0000 From: Paul Burton To: "linux-mips@vger.kernel.org" CC: Huacai Chen , Jiaxun Yang , "linux-kernel@vger.kernel.org" , Paul Burton Subject: [PATCH v2 12/36] MIPS: atomic: Emit Loongson3 sync workarounds within asm Thread-Topic: [PATCH v2 12/36] MIPS: atomic: Emit Loongson3 sync workarounds within asm Thread-Index: AQHVeKKjF/6Q5rRGb0K/TMnkoH0pMw== Date: Tue, 1 Oct 2019 21:53:20 +0000 Message-ID: <20191001215249.4157062-13-paul.burton@mips.com> References: <20191001215249.4157062-1-paul.burton@mips.com> In-Reply-To: <20191001215249.4157062-1-paul.burton@mips.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BYAPR07CA0093.namprd07.prod.outlook.com (2603:10b6:a03:12b::34) To MWHPR2201MB1277.namprd22.prod.outlook.com (2603:10b6:301:18::12) authentication-results: spf=none (sender IP is ) smtp.mailfrom=pburton@wavecomp.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.23.0 x-originating-ip: [12.94.197.246] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: ca7395f4-4828-4b47-d562-08d746b9c61e x-ms-traffictypediagnostic: MWHPR2201MB1439: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:3968; x-forefront-prvs: 0177904E6B x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(396003)(39840400004)(366004)(346002)(136003)(376002)(199004)(189003)(186003)(11346002)(6512007)(1076003)(486006)(66066001)(26005)(6486002)(316002)(2906002)(5640700003)(6436002)(44832011)(42882007)(25786009)(446003)(6916009)(386003)(6506007)(76176011)(478600001)(52116002)(2616005)(102836004)(476003)(5660300002)(305945005)(64756008)(66446008)(14444005)(2351001)(7736002)(99286004)(54906003)(14454004)(71190400001)(71200400001)(66946007)(66556008)(66476007)(36756003)(3846002)(6116002)(107886003)(4326008)(50226002)(81166006)(81156014)(8676002)(2501003)(8936002)(256004);DIR:OUT;SFP:1102;SCL:1;SRVR:MWHPR2201MB1439;H:MWHPR2201MB1277.namprd22.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; received-spf: None (protection.outlook.com: wavecomp.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: rHRrDQXCVg7JxC3pCmxsRUaQLQDWtfdgda9Af9+smRLwkbuIxCv5RtgDFQcEZNpgkhHkDS5MdEvOGLL9Lta6PDHP9EhPtz0Y6E7bFS0NooA8Ew4u+Nw/7eOnyiF1OfuVLfVpEyA/IklRRvRegnPdQL+HOSbWYuvrRt3C/E5yyLPkKhnk7iInDv7QFkdu9Ex8dmmP+jIzx1/1I3WlNZ7HlgKjDqKeXvGRvTpgS+RTLQDDyUfmVI1gNfFT2RVtD686oJdgCX1/Qu+xN2Wya6l0lsbhdJqD3cSotvA3CpgH6FcyyVmzrsX4bHH7QUflsU1OMjAiOFXR1BPsgFbBi/F70PBsCeBot60VV2HxYZNPW1+DyLZpN8mpEEy4c1IhZRxjq/W7xbT6nZbAkigLT8lB/ifIwSsbefqGXljt22LGPf0= MIME-Version: 1.0 X-OriginatorOrg: mips.com X-MS-Exchange-CrossTenant-Network-Message-Id: ca7395f4-4828-4b47-d562-08d746b9c61e X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Oct 2019 21:53:20.4087 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 463607d3-1db3-40a0-8a29-970c56230104 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 12gd/BhFbsEk5J8yx4q284umlN2xPxNLrlBTtG9pLHLo5NLBgZD+pasS8nO6ueXKdSLgG1kuepqjKZM5hN5njA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR2201MB1439 Sender: linux-mips-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-mips@vger.kernel.org Generate the sync instructions required to workaround Loongson3 LL/SC errata within inline asm blocks, which feels a little safer than doing it from C where strictly speaking the compiler would be well within its rights to insert a memory access between the separate asm statements we previously had, containing sync & ll instructions respectively. Signed-off-by: Paul Burton --- Changes in v2: None arch/mips/include/asm/atomic.h | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-) diff --git a/arch/mips/include/asm/atomic.h b/arch/mips/include/asm/atomic.h index b834af5a7382..841ff274ada6 100644 --- a/arch/mips/include/asm/atomic.h +++ b/arch/mips/include/asm/atomic.h @@ -21,6 +21,7 @@ #include #include #include +#include #include #define ATOMIC_INIT(i) { (i) } @@ -56,10 +57,10 @@ static __inline__ void pfx##_##op(type i, pfx##_t * v) \ return; \ } \ \ - loongson_llsc_mb(); \ __asm__ __volatile__( \ " .set push \n" \ " .set " MIPS_ISA_LEVEL " \n" \ + " " __SYNC(full, loongson3_war) " \n" \ "1: " #ll " %0, %1 # " #pfx "_" #op " \n" \ " " #asm_op " %0, %2 \n" \ " " #sc " %0, %1 \n" \ @@ -85,10 +86,10 @@ static __inline__ type pfx##_##op##_return_relaxed(type i, pfx##_t * v) \ return result; \ } \ \ - loongson_llsc_mb(); \ __asm__ __volatile__( \ " .set push \n" \ " .set " MIPS_ISA_LEVEL " \n" \ + " " __SYNC(full, loongson3_war) " \n" \ "1: " #ll " %1, %2 # " #pfx "_" #op "_return\n" \ " " #asm_op " %0, %1, %3 \n" \ " " #sc " %0, %2 \n" \ @@ -117,10 +118,10 @@ static __inline__ type pfx##_fetch_##op##_relaxed(type i, pfx##_t * v) \ return result; \ } \ \ - loongson_llsc_mb(); \ __asm__ __volatile__( \ " .set push \n" \ " .set " MIPS_ISA_LEVEL " \n" \ + " " __SYNC(full, loongson3_war) " \n" \ "1: " #ll " %1, %2 # " #pfx "_fetch_" #op "\n" \ " " #asm_op " %0, %1, %3 \n" \ " " #sc " %0, %2 \n" \ @@ -200,10 +201,10 @@ static __inline__ int atomic_sub_if_positive(int i, atomic_t * v) if (kernel_uses_llsc) { int temp; - loongson_llsc_mb(); __asm__ __volatile__( " .set push \n" " .set "MIPS_ISA_LEVEL" \n" + " " __SYNC(full, loongson3_war) " \n" "1: ll %1, %2 # atomic_sub_if_positive\n" " .set pop \n" " subu %0, %1, %3 \n" @@ -213,7 +214,7 @@ static __inline__ int atomic_sub_if_positive(int i, atomic_t * v) " .set "MIPS_ISA_LEVEL" \n" " sc %1, %2 \n" "\t" __SC_BEQZ "%1, 1b \n" - "2: \n" + "2: " __SYNC(full, loongson3_war) " \n" " .set pop \n" : "=&r" (result), "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (v->counter) @@ -229,7 +230,14 @@ static __inline__ int atomic_sub_if_positive(int i, atomic_t * v) raw_local_irq_restore(flags); } - smp_llsc_mb(); + /* + * In the Loongson3 workaround case we already have a completion + * barrier at 2: above, which is needed due to the bltz that can branch + * to code outside of the LL/SC loop. As such, we don't need to emit + * another barrier here. + */ + if (!__SYNC_loongson3_war) + smp_llsc_mb(); return result; } From patchwork Tue Oct 1 21:53:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Burton X-Patchwork-Id: 11169829 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C35E81599 for ; Tue, 1 Oct 2019 21:55:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id A0CF121A4A for ; Tue, 1 Oct 2019 21:55:10 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=wavecomp.com header.i=@wavecomp.com header.b="nIft/n9i" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728112AbfJAVx1 (ORCPT ); Tue, 1 Oct 2019 17:53:27 -0400 Received: from mail-eopbgr730113.outbound.protection.outlook.com ([40.107.73.113]:17171 "EHLO NAM05-DM3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728102AbfJAVx0 (ORCPT ); Tue, 1 Oct 2019 17:53:26 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VUJkZ/EBDxdUUsVK9IpdyNyDL10bAWh4gtOsgW5GHPybiBlpvmKoEXJrL1i1FWdVHB0bELNWx0n6qCQt4GKGn2zitPdhK+oqsmurgZc2YDH4DeosqFC7KXO36dFQPeqlyRuTVD9uBTKZl4WO5zACbydyXdaPvURaqF7jlcbNcCwFv0a8DukT9yQP0/nbzELBgp8S0YsB22IudnnGPr7Dp2KsBuMYaAnriyYIwsKNQA0o7GFjnrfe0/Zbw+I/0kZ9UyKDYYLlXUGgb8OEtagZrsn5i8m5u7f4MzJRroavun7mris/p6pKojEACL3w2MtJyiGkZz15eURCzXBqfeeiJg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=49Jd5Z3FfPcepodFqHQoJO3RNX9e6jeeL5/ciXdkBxQ=; b=WsouRD4w/d4uhMq/Vz9E8SstA2rTGCuY4Lh8tJsemB6ALXESFjnKQ2fkERQJey8BCFipkxYeHGX8GvXtYL/2ppDeWl6iFDAUPiYpMBhzL9L0J2j/nbXkRuKBJscqL9Sc97zrSM1JQMluRqLCAuDZ0lNXYnwh55st/xAMnScjLqNiG/MIsPihDwN96SG1O0qwT+TDSK26QXi3wMNn1PnM+SErGkysk/d8+CF2hSZrzxPDO68pwJFrlhNHm4P+SH0rwmjEB9AU0ybFUf8bhzgwWl7V1U3kuE+KdFe1MxKxijQuRYP5AqrZPWOXMCO3GEilN6sGehb/L2CCiiiGXXn9vA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wavecomp.com; dmarc=pass action=none header.from=mips.com; dkim=pass header.d=mips.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wavecomp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=49Jd5Z3FfPcepodFqHQoJO3RNX9e6jeeL5/ciXdkBxQ=; b=nIft/n9i3XyezoKBDZMOfKqc8/S5OgVdkgdcWTmc1tjeEo5CgJ/9Jb6nSRADWOJr2jUEWMTeUiYdxHN5pZxGQ7ZrVeSSTjaMPMywFL38WB8MLYILIOKnvRN9K9rw9PDbpu4C88lH6ChMVQcwRWuJKvVXDe8NtG2nw8XwYLb1Vyw= Received: from MWHPR2201MB1277.namprd22.prod.outlook.com (10.172.60.12) by MWHPR2201MB1439.namprd22.prod.outlook.com (10.174.169.166) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2305.20; Tue, 1 Oct 2019 21:53:22 +0000 Received: from MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481]) by MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481%2]) with mapi id 15.20.2305.022; Tue, 1 Oct 2019 21:53:22 +0000 From: Paul Burton To: "linux-mips@vger.kernel.org" CC: Huacai Chen , Jiaxun Yang , "linux-kernel@vger.kernel.org" , Paul Burton Subject: [PATCH v2 13/36] MIPS: atomic: Use _atomic barriers in atomic_sub_if_positive() Thread-Topic: [PATCH v2 13/36] MIPS: atomic: Use _atomic barriers in atomic_sub_if_positive() Thread-Index: AQHVeKKkiaHoVT4NRUe9RobCldAC8Q== Date: Tue, 1 Oct 2019 21:53:21 +0000 Message-ID: <20191001215249.4157062-14-paul.burton@mips.com> References: <20191001215249.4157062-1-paul.burton@mips.com> In-Reply-To: <20191001215249.4157062-1-paul.burton@mips.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BYAPR07CA0093.namprd07.prod.outlook.com (2603:10b6:a03:12b::34) To MWHPR2201MB1277.namprd22.prod.outlook.com (2603:10b6:301:18::12) authentication-results: spf=none (sender IP is ) smtp.mailfrom=pburton@wavecomp.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.23.0 x-originating-ip: [12.94.197.246] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: a263dcc0-275a-4389-b765-08d746b9c6e5 x-ms-traffictypediagnostic: MWHPR2201MB1439: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:3276; x-forefront-prvs: 0177904E6B x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(396003)(39840400004)(366004)(346002)(136003)(376002)(199004)(189003)(186003)(11346002)(6512007)(1076003)(486006)(66066001)(26005)(6486002)(316002)(2906002)(5640700003)(6436002)(44832011)(42882007)(4744005)(25786009)(446003)(6916009)(386003)(6506007)(76176011)(478600001)(52116002)(2616005)(102836004)(476003)(5660300002)(305945005)(64756008)(66446008)(2351001)(7736002)(99286004)(54906003)(14454004)(71190400001)(71200400001)(66946007)(66556008)(66476007)(36756003)(3846002)(6116002)(107886003)(4326008)(50226002)(81166006)(81156014)(8676002)(2501003)(8936002)(256004);DIR:OUT;SFP:1102;SCL:1;SRVR:MWHPR2201MB1439;H:MWHPR2201MB1277.namprd22.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; received-spf: None (protection.outlook.com: wavecomp.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: gB4PwjDE8ulZ/lAKowSsbNbas0JCeNDe6aNOs4Wqyag23x/uK4zp04AceK51GQJ7qx9WcTVJw7IYfCioc9U3Qf8FUDwijMZN0U1S58zZU5+ejE7oWRfh2KCO1uLA9k56qt0VGaUv7pPCth0ChW2HwQ5a/QvoHUCdrvnqgawLqyb0vnmIwWDRB6FHHx3iEakpApexFr6EADQw4vqTX/PbtwF60M3oHf1n493mD0udZbHNcEJ7ZUfVWz3av0OUFwLuKgC03Hcod7tn7wSFXjFgLUncpzat3WPZwnghOT/HoBy2fYJh/7cBxSGz3VqgToLGr054MhhFaFhTLvvCRq6D623zeVHQfKszBb1By3sbqrAdx7CpbeGnzU/2409QgL+qi9iZJo5lGesLhLi72h0RrSMHg/ZYeAYhnARN75wmAgA= MIME-Version: 1.0 X-OriginatorOrg: mips.com X-MS-Exchange-CrossTenant-Network-Message-Id: a263dcc0-275a-4389-b765-08d746b9c6e5 X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Oct 2019 21:53:21.9650 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 463607d3-1db3-40a0-8a29-970c56230104 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 3d1GWdc5N58ymI22MT29lTxfJtcd8+iMeDKZ7y9PEeb9RbzFusvnaBUoKczrkncKlMNIPvF0CXWv/VMsBYnO1A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR2201MB1439 Sender: linux-mips-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-mips@vger.kernel.org Use smp_mb__before_atomic() & smp_mb__after_atomic() in atomic_sub_if_positive() rather than the equivalent smp_mb__before_llsc() & smp_llsc_mb(). The former are more standard & this preps us for avoiding redundant duplicate barriers on Loongson3 in a later patch. Signed-off-by: Paul Burton --- Changes in v2: None arch/mips/include/asm/atomic.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/mips/include/asm/atomic.h b/arch/mips/include/asm/atomic.h index 841ff274ada6..24443ef29337 100644 --- a/arch/mips/include/asm/atomic.h +++ b/arch/mips/include/asm/atomic.h @@ -196,7 +196,7 @@ static __inline__ int atomic_sub_if_positive(int i, atomic_t * v) { int result; - smp_mb__before_llsc(); + smp_mb__before_atomic(); if (kernel_uses_llsc) { int temp; @@ -237,7 +237,7 @@ static __inline__ int atomic_sub_if_positive(int i, atomic_t * v) * another barrier here. */ if (!__SYNC_loongson3_war) - smp_llsc_mb(); + smp_mb__after_atomic(); return result; } From patchwork Tue Oct 1 21:53:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Burton X-Patchwork-Id: 11169783 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 92C3E1599 for ; Tue, 1 Oct 2019 21:53:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 5D70C21D7B for ; Tue, 1 Oct 2019 21:53:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=wavecomp.com header.i=@wavecomp.com header.b="anQiEHWk" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728119AbfJAVx2 (ORCPT ); Tue, 1 Oct 2019 17:53:28 -0400 Received: from mail-eopbgr730113.outbound.protection.outlook.com ([40.107.73.113]:17171 "EHLO NAM05-DM3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728107AbfJAVx2 (ORCPT ); Tue, 1 Oct 2019 17:53:28 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FpwfJgj3xZubnIgtc1o/99v2211BulRrSGkD3Kj+DMhGUSTAEYvBsO+ziliM3Dm/c/2fOeWd8qkuGw6qX5RSAJY821OlGooPxtOl+SiU6xZFI3lDsEletAOmNrQVVMJWCTvordWxvjI8+rYBLifQ767kIRMuK22e5kK3+rwDYP0RtzjONBa/VfQiFJomqDlRamByZXhAvVBEeTlCw0b6jXPhlpHw2EMUYEmcFK/+a1PH5AWHW0WzkxjknsNiqrx9tfZpHdMJyZXLGX16xVb4qVunqzBi5eotKeRB+cmE5gCQGChiuGQJjIhGzMWsX80/uDgkQgF+JarBIlElzKvWwA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/oIdoh72+TuqYxw0e6v85QgBl6Han8Vh4G1I/SBvwDg=; b=lg441+636AFT+PKtOkKacyvJ3HLQqjNhHMkbHBa9uy/6d/mdzj+k73obmhRohc1fcufQVVS9oZ8QU+hivF3ZZUc5v0yO7dkWnCy+skjkUeDkvgfcRtZIw/EuZt944DSTIMHjZyhSNOSCXsgBKWm+PUQUNgQBn8sXt6qcLPjo9+vUVEqSZtlv1EMqDHaZ+fo3PAT9LLZWWJdw5CRH+05snVnN2zD8cwFfNsGL04n3JvEVACLjEDbvjO1ZZV7kx3jsVu9TLzUxal4w9ikRSqJMMz91isdjhFwUA6SGrOZsvJieCrWEAg8X09k2VtlcsTnDlvJFJWeHK9sCv2AlrqNfwg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wavecomp.com; dmarc=pass action=none header.from=mips.com; dkim=pass header.d=mips.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wavecomp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/oIdoh72+TuqYxw0e6v85QgBl6Han8Vh4G1I/SBvwDg=; b=anQiEHWkG+Cm5rT9cscMPbXU8wTgk5VLGYL1vGqSXj7YZ6Pcw+vs26Bcp0VOavZD+Ewk4Iqpid8+6rSIIY5ULyLhMACi1xq9We8x0lFB/qpv8tTtenyhRajcF/k94cMbiwm6djgiPsf+Bj9O8qvI7mCaQUSw+WxJNamH19LoMrI= Received: from MWHPR2201MB1277.namprd22.prod.outlook.com (10.172.60.12) by MWHPR2201MB1439.namprd22.prod.outlook.com (10.174.169.166) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2305.20; Tue, 1 Oct 2019 21:53:23 +0000 Received: from MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481]) by MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481%2]) with mapi id 15.20.2305.022; Tue, 1 Oct 2019 21:53:23 +0000 From: Paul Burton To: "linux-mips@vger.kernel.org" CC: Huacai Chen , Jiaxun Yang , "linux-kernel@vger.kernel.org" , Paul Burton Subject: [PATCH v2 14/36] MIPS: atomic: Unify 32b & 64b sub_if_positive Thread-Topic: [PATCH v2 14/36] MIPS: atomic: Unify 32b & 64b sub_if_positive Thread-Index: AQHVeKKlEcpsFkxrr0SAHmIRa77Nfg== Date: Tue, 1 Oct 2019 21:53:23 +0000 Message-ID: <20191001215249.4157062-15-paul.burton@mips.com> References: <20191001215249.4157062-1-paul.burton@mips.com> In-Reply-To: <20191001215249.4157062-1-paul.burton@mips.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BYAPR07CA0093.namprd07.prod.outlook.com (2603:10b6:a03:12b::34) To MWHPR2201MB1277.namprd22.prod.outlook.com (2603:10b6:301:18::12) authentication-results: spf=none (sender IP is ) smtp.mailfrom=pburton@wavecomp.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.23.0 x-originating-ip: [12.94.197.246] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: d8603796-3fe2-4034-1765-08d746b9c7a9 x-ms-traffictypediagnostic: MWHPR2201MB1439: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:4303; x-forefront-prvs: 0177904E6B x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(396003)(39840400004)(366004)(346002)(136003)(376002)(199004)(189003)(186003)(11346002)(6512007)(1076003)(486006)(66066001)(26005)(6486002)(316002)(2906002)(5640700003)(6436002)(44832011)(42882007)(25786009)(446003)(6916009)(386003)(6506007)(76176011)(478600001)(52116002)(2616005)(102836004)(476003)(5660300002)(305945005)(64756008)(66446008)(14444005)(2351001)(7736002)(99286004)(54906003)(14454004)(71190400001)(71200400001)(66946007)(66556008)(66476007)(36756003)(3846002)(6116002)(107886003)(4326008)(50226002)(81166006)(81156014)(8676002)(2501003)(8936002)(256004);DIR:OUT;SFP:1102;SCL:1;SRVR:MWHPR2201MB1439;H:MWHPR2201MB1277.namprd22.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; received-spf: None (protection.outlook.com: wavecomp.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: VhFf1wWa3ekmr4zjGTkOJlAZMP9X9gL0/PYJOaLIlj/guMtEQQsh/nE73A41UGx7VHeear45kecdCduwErrgG1LGxzmDYahy+9DPKhtdibjqPPg6E8522O8v8lsAdTVeCKlbDrq0wHCFUqbUlxuZls9nr7o970DNzwYOX9uDOjstVfG1/upiD6qhvGh3bgll1iMEQF06Oo+c5dTY3Ly3xRXpdRRGh2XGkpQHSWDhfbHLVEUC8LKOM+Gu0wWjExsiyFcZAKEPJJfGgN8J6nNFHSJ9a8vP0gSmVu06OFR98IC6tO7klIbRo/meRbp6oKmTikeAz3m/QiJrBXsmUZ5301PLFOmUUeZ5byMqcwDUpvOoZX6c2EqStOoCd5hD2hrPnCDHzJqhcBHzarlurZOfR5Jg1Ei16744eoIz5wJVXMo= MIME-Version: 1.0 X-OriginatorOrg: mips.com X-MS-Exchange-CrossTenant-Network-Message-Id: d8603796-3fe2-4034-1765-08d746b9c7a9 X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Oct 2019 21:53:23.1914 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 463607d3-1db3-40a0-8a29-970c56230104 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: hS7P8MaCWkY9yvjZxPV9VIzNQkTK0zmaKDN20f3pTS0E49PAxvx/O6+y2VtC/O2Ias0kwFQkSzZKjQo2fczQwA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR2201MB1439 Sender: linux-mips-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-mips@vger.kernel.org Unify the definitions of atomic_sub_if_positive() & atomic64_sub_if_positive() using a macro like we do for most other atomic functions. This allows us to share the implementation ensuring consistency between the two. Notably this provides the appropriate loongson3_war barriers in the atomic64_sub_if_positive() case which were previously missing. The code is rearranged a little to handle the !kernel_uses_llsc case first in order to de-indent the LL/SC case & allow us not to go over 80 characters per line. Signed-off-by: Paul Burton --- Changes in v2: None arch/mips/include/asm/atomic.h | 164 ++++++++++++--------------------- 1 file changed, 58 insertions(+), 106 deletions(-) diff --git a/arch/mips/include/asm/atomic.h b/arch/mips/include/asm/atomic.h index 24443ef29337..96ef50fa2817 100644 --- a/arch/mips/include/asm/atomic.h +++ b/arch/mips/include/asm/atomic.h @@ -192,65 +192,71 @@ ATOMIC_OPS(atomic64, xor, s64, ^=, xor, lld, scd) * Atomically test @v and subtract @i if @v is greater or equal than @i. * The function returns the old value of @v minus @i. */ -static __inline__ int atomic_sub_if_positive(int i, atomic_t * v) -{ - int result; - - smp_mb__before_atomic(); - - if (kernel_uses_llsc) { - int temp; - - __asm__ __volatile__( - " .set push \n" - " .set "MIPS_ISA_LEVEL" \n" - " " __SYNC(full, loongson3_war) " \n" - "1: ll %1, %2 # atomic_sub_if_positive\n" - " .set pop \n" - " subu %0, %1, %3 \n" - " move %1, %0 \n" - " bltz %0, 2f \n" - " .set push \n" - " .set "MIPS_ISA_LEVEL" \n" - " sc %1, %2 \n" - "\t" __SC_BEQZ "%1, 1b \n" - "2: " __SYNC(full, loongson3_war) " \n" - " .set pop \n" - : "=&r" (result), "=&r" (temp), - "+" GCC_OFF_SMALL_ASM() (v->counter) - : "Ir" (i) : __LLSC_CLOBBER); - } else { - unsigned long flags; +#define ATOMIC_SIP_OP(pfx, type, op, ll, sc) \ +static __inline__ int pfx##_sub_if_positive(type i, pfx##_t * v) \ +{ \ + type temp, result; \ + \ + smp_mb__before_atomic(); \ + \ + if (!kernel_uses_llsc) { \ + unsigned long flags; \ + \ + raw_local_irq_save(flags); \ + result = v->counter; \ + result -= i; \ + if (result >= 0) \ + v->counter = result; \ + raw_local_irq_restore(flags); \ + smp_mb__after_atomic(); \ + return result; \ + } \ + \ + __asm__ __volatile__( \ + " .set push \n" \ + " .set " MIPS_ISA_LEVEL " \n" \ + " " __SYNC(full, loongson3_war) " \n" \ + "1: " #ll " %1, %2 # atomic_sub_if_positive\n" \ + " .set pop \n" \ + " " #op " %0, %1, %3 \n" \ + " move %1, %0 \n" \ + " bltz %0, 2f \n" \ + " .set push \n" \ + " .set " MIPS_ISA_LEVEL " \n" \ + " " #sc " %1, %2 \n" \ + " " __SC_BEQZ "%1, 1b \n" \ + "2: " __SYNC(full, loongson3_war) " \n" \ + " .set pop \n" \ + : "=&r" (result), "=&r" (temp), \ + "+" GCC_OFF_SMALL_ASM() (v->counter) \ + : "Ir" (i) \ + : __LLSC_CLOBBER); \ + \ + /* \ + * In the Loongson3 workaround case we already have a \ + * completion barrier at 2: above, which is needed due to the \ + * bltz that can branch to code outside of the LL/SC loop. As \ + * such, we don't need to emit another barrier here. \ + */ \ + if (!__SYNC_loongson3_war) \ + smp_mb__after_atomic(); \ + \ + return result; \ +} - raw_local_irq_save(flags); - result = v->counter; - result -= i; - if (result >= 0) - v->counter = result; - raw_local_irq_restore(flags); - } +ATOMIC_SIP_OP(atomic, int, subu, ll, sc) +#define atomic_dec_if_positive(v) atomic_sub_if_positive(1, v) - /* - * In the Loongson3 workaround case we already have a completion - * barrier at 2: above, which is needed due to the bltz that can branch - * to code outside of the LL/SC loop. As such, we don't need to emit - * another barrier here. - */ - if (!__SYNC_loongson3_war) - smp_mb__after_atomic(); +#ifdef CONFIG_64BIT +ATOMIC_SIP_OP(atomic64, s64, dsubu, lld, scd) +#define atomic64_dec_if_positive(v) atomic64_sub_if_positive(1, v) +#endif - return result; -} +#undef ATOMIC_SIP_OP #define atomic_cmpxchg(v, o, n) (cmpxchg(&((v)->counter), (o), (n))) #define atomic_xchg(v, new) (xchg(&((v)->counter), (new))) -/* - * atomic_dec_if_positive - decrement by 1 if old value positive - * @v: pointer of type atomic_t - */ -#define atomic_dec_if_positive(v) atomic_sub_if_positive(1, v) - #ifdef CONFIG_64BIT #define ATOMIC64_INIT(i) { (i) } @@ -269,64 +275,10 @@ static __inline__ int atomic_sub_if_positive(int i, atomic_t * v) */ #define atomic64_set(v, i) WRITE_ONCE((v)->counter, (i)) -/* - * atomic64_sub_if_positive - conditionally subtract integer from atomic - * variable - * @i: integer value to subtract - * @v: pointer of type atomic64_t - * - * Atomically test @v and subtract @i if @v is greater or equal than @i. - * The function returns the old value of @v minus @i. - */ -static __inline__ s64 atomic64_sub_if_positive(s64 i, atomic64_t * v) -{ - s64 result; - - smp_mb__before_llsc(); - - if (kernel_uses_llsc) { - s64 temp; - - __asm__ __volatile__( - " .set push \n" - " .set "MIPS_ISA_LEVEL" \n" - "1: lld %1, %2 # atomic64_sub_if_positive\n" - " dsubu %0, %1, %3 \n" - " move %1, %0 \n" - " bltz %0, 1f \n" - " scd %1, %2 \n" - "\t" __SC_BEQZ "%1, 1b \n" - "1: \n" - " .set pop \n" - : "=&r" (result), "=&r" (temp), - "+" GCC_OFF_SMALL_ASM() (v->counter) - : "Ir" (i)); - } else { - unsigned long flags; - - raw_local_irq_save(flags); - result = v->counter; - result -= i; - if (result >= 0) - v->counter = result; - raw_local_irq_restore(flags); - } - - smp_llsc_mb(); - - return result; -} - #define atomic64_cmpxchg(v, o, n) \ ((__typeof__((v)->counter))cmpxchg(&((v)->counter), (o), (n))) #define atomic64_xchg(v, new) (xchg(&((v)->counter), (new))) -/* - * atomic64_dec_if_positive - decrement by 1 if old value positive - * @v: pointer of type atomic64_t - */ -#define atomic64_dec_if_positive(v) atomic64_sub_if_positive(1, v) - #endif /* CONFIG_64BIT */ #endif /* _ASM_ATOMIC_H */ From patchwork Tue Oct 1 21:53:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Burton X-Patchwork-Id: 11169827 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D6A1213B1 for ; Tue, 1 Oct 2019 21:55:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id AA38821A4A for ; Tue, 1 Oct 2019 21:55:09 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=wavecomp.com header.i=@wavecomp.com header.b="PmsZpBHY" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727244AbfJAVzB (ORCPT ); Tue, 1 Oct 2019 17:55:01 -0400 Received: from mail-eopbgr730113.outbound.protection.outlook.com ([40.107.73.113]:17171 "EHLO NAM05-DM3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728116AbfJAVx3 (ORCPT ); Tue, 1 Oct 2019 17:53:29 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=IEufhwe8NhiuB90fwBr0dUUUWK4jqSHlby/ZQtA/5YIepMbzIBu/q9eGKApu+Ed/MLiuJ9wAdsnpV+8gueyJ0PJiiVqTD8PmoM1SnQS6FXSxPedXMjcVCb+aqExLgVdNArNSo7SkugDqn9iDnSluMrythdSxr6gNHjUgQPKAUlhUXS2S3/7ycqfHi0WqdROsGfK/ULK1NRxdBBmYHoawLsmr1CCLJPbS64Cv0JCLtRsJPJjeZgfC6oepHGwdeSueGKsd2QcUOdv+l6QB4F4jQVlwfABW/vGlI7NvmmBSkjpf67pIX0JTNVG3gAxR1MQYK1QJdIYyssPm9af3RsWD2A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=dho5hzus8m/2lkImn8ZIXDv7gok3Vo42gJWi78e//V4=; b=DB8Pf1FiTNKr+qRd4rZBVX8Ze+4joCE3l4160pCQ63lQtMZx1dx+Jl3g34Es4G0dJOI76xcBdAe/ZsHkU5rJVlyncNRymGvkzGSKRk530iDhG0+bJQ0wovYce4GKDC2pMoYnK5Gj13qMgEuby8LDaFb+42YNLH44hLKpEmZ/BWJz+rflRk3voc8yYUV9D5hV+EEb8K29Pr4jXYZBeYLbvCo/CWnWoeN/gqn/Gl9ELU4m0ceQhb4CbOSaaPje8wWSeg5TGgZEE/tOIYGpZbzWh/PMfRYFOmgB/heo66tlwhqtEoLnRJ96h9MAMX8vuf5KgubCNA5R9t5qRQ/lmDLflw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wavecomp.com; dmarc=pass action=none header.from=mips.com; dkim=pass header.d=mips.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wavecomp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=dho5hzus8m/2lkImn8ZIXDv7gok3Vo42gJWi78e//V4=; b=PmsZpBHYK11++JtOfhV7JK7iu05A2QQpkJTSWg8LfaWNqllb7vKFD0tT0RXuCPDX0yvC9MA1A4SnVJXIg6mBerdEmT8xkn/Ktw34/1IHkkqYz6yVj+zajMiJQekBJVEECBQa2+9FPup6W2lxi9jx4bBP0pI/lVfXroBn4COTplA= Received: from MWHPR2201MB1277.namprd22.prod.outlook.com (10.172.60.12) by MWHPR2201MB1439.namprd22.prod.outlook.com (10.174.169.166) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2305.20; Tue, 1 Oct 2019 21:53:24 +0000 Received: from MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481]) by MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481%2]) with mapi id 15.20.2305.022; Tue, 1 Oct 2019 21:53:24 +0000 From: Paul Burton To: "linux-mips@vger.kernel.org" CC: Huacai Chen , Jiaxun Yang , "linux-kernel@vger.kernel.org" , Paul Burton Subject: [PATCH v2 15/36] MIPS: atomic: Deduplicate 32b & 64b read, set, xchg, cmpxchg Thread-Topic: [PATCH v2 15/36] MIPS: atomic: Deduplicate 32b & 64b read, set, xchg, cmpxchg Thread-Index: AQHVeKKlJxT7iMVLuk6sBsJVvUiXBQ== Date: Tue, 1 Oct 2019 21:53:24 +0000 Message-ID: <20191001215249.4157062-16-paul.burton@mips.com> References: <20191001215249.4157062-1-paul.burton@mips.com> In-Reply-To: <20191001215249.4157062-1-paul.burton@mips.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BYAPR07CA0093.namprd07.prod.outlook.com (2603:10b6:a03:12b::34) To MWHPR2201MB1277.namprd22.prod.outlook.com (2603:10b6:301:18::12) authentication-results: spf=none (sender IP is ) smtp.mailfrom=pburton@wavecomp.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.23.0 x-originating-ip: [12.94.197.246] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: e36d3218-ddf0-4a29-109c-08d746b9c84b x-ms-traffictypediagnostic: MWHPR2201MB1439: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:4502; x-forefront-prvs: 0177904E6B x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(396003)(39840400004)(366004)(346002)(136003)(376002)(199004)(189003)(186003)(11346002)(6512007)(1076003)(486006)(66066001)(26005)(6486002)(316002)(2906002)(5640700003)(6436002)(44832011)(42882007)(25786009)(446003)(6916009)(386003)(6506007)(76176011)(478600001)(52116002)(2616005)(102836004)(476003)(5660300002)(305945005)(64756008)(66446008)(2351001)(7736002)(99286004)(54906003)(14454004)(71190400001)(71200400001)(66946007)(66556008)(66476007)(36756003)(3846002)(6116002)(107886003)(4326008)(50226002)(81166006)(81156014)(8676002)(2501003)(8936002)(256004)(17423001)(156123004);DIR:OUT;SFP:1102;SCL:1;SRVR:MWHPR2201MB1439;H:MWHPR2201MB1277.namprd22.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; received-spf: None (protection.outlook.com: wavecomp.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: hJHMGK9rixxdn8S+HITneOMVgWtCKJV0XWmk9f4z246tOU7i0LgXnZm5Q3lIGPxxhpNh2HcDoPb3Epby09Tarb6mfM70tzSPum6XgNotstOEGhmfMamCKrTDBcjBZJy8Sg3uDjREgFM3US0iNtbd9GiSFl6QSXqDGlol4s3XZIuA864Zb7lXdwGfBCPXUDwPcYjPupmd5Srw5ZAV/4PNoBhYEwrEvljb5sxjWlnYjS4pQbQQNhb4/SjhIrrCPVXgALXegRRAm3HPYRBC8pWJpnldE9HYt8ErUVHqzQm4RHTXHOwmUo+nUGQ4S5GraHtCHJZ0BsVO3Rnawf8k5Cz5e/zBKxItW8NoYOhL6PyMIUXmQyC6WQUZotQiAxis2gnCVOz88lGCzpdH1/oWk4DzZJBSXyJat3i54/LL83SDk4Y= MIME-Version: 1.0 X-OriginatorOrg: mips.com X-MS-Exchange-CrossTenant-Network-Message-Id: e36d3218-ddf0-4a29-109c-08d746b9c84b X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Oct 2019 21:53:24.2479 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 463607d3-1db3-40a0-8a29-970c56230104 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: Vgw/i5OgStAW4NH8lB6S2dTTaBXJBL8FntPeixBdf7rcLz3FPXianjrvx/gPYDwEvxfJMSj1LOgoI9IdoYHz2w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR2201MB1439 Sender: linux-mips-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-mips@vger.kernel.org Remove the remaining duplication between 32b & 64b in asm/atomic.h by making use of an ATOMIC_OPS() macro to generate: - atomic_read()/atomic64_read() - atomic_set()/atomic64_set() - atomic_cmpxchg()/atomic64_cmpxchg() - atomic_xchg()/atomic64_xchg() This is consistent with the way all other functions in asm/atomic.h are generated, and ensures consistency between the 32b & 64b functions. Of note is that this results in the above now being static inline functions rather than macros. Signed-off-by: Paul Burton --- Changes in v2: None arch/mips/include/asm/atomic.h | 70 +++++++++++++--------------------- 1 file changed, 27 insertions(+), 43 deletions(-) diff --git a/arch/mips/include/asm/atomic.h b/arch/mips/include/asm/atomic.h index 96ef50fa2817..e5ac88392d1f 100644 --- a/arch/mips/include/asm/atomic.h +++ b/arch/mips/include/asm/atomic.h @@ -24,24 +24,34 @@ #include #include -#define ATOMIC_INIT(i) { (i) } +#define ATOMIC_OPS(pfx, type) \ +static __always_inline type pfx##_read(const pfx##_t *v) \ +{ \ + return READ_ONCE(v->counter); \ +} \ + \ +static __always_inline void pfx##_set(pfx##_t *v, type i) \ +{ \ + WRITE_ONCE(v->counter, i); \ +} \ + \ +static __always_inline type pfx##_cmpxchg(pfx##_t *v, type o, type n) \ +{ \ + return cmpxchg(&v->counter, o, n); \ +} \ + \ +static __always_inline type pfx##_xchg(pfx##_t *v, type n) \ +{ \ + return xchg(&v->counter, n); \ +} -/* - * atomic_read - read atomic variable - * @v: pointer of type atomic_t - * - * Atomically reads the value of @v. - */ -#define atomic_read(v) READ_ONCE((v)->counter) +#define ATOMIC_INIT(i) { (i) } +ATOMIC_OPS(atomic, int) -/* - * atomic_set - set atomic variable - * @v: pointer of type atomic_t - * @i: required value - * - * Atomically sets the value of @v to @i. - */ -#define atomic_set(v, i) WRITE_ONCE((v)->counter, (i)) +#ifdef CONFIG_64BIT +# define ATOMIC64_INIT(i) { (i) } +ATOMIC_OPS(atomic64, s64) +#endif #define ATOMIC_OP(pfx, op, type, c_op, asm_op, ll, sc) \ static __inline__ void pfx##_##op(type i, pfx##_t * v) \ @@ -135,6 +145,7 @@ static __inline__ type pfx##_fetch_##op##_relaxed(type i, pfx##_t * v) \ return result; \ } +#undef ATOMIC_OPS #define ATOMIC_OPS(pfx, op, type, c_op, asm_op, ll, sc) \ ATOMIC_OP(pfx, op, type, c_op, asm_op, ll, sc) \ ATOMIC_OP_RETURN(pfx, op, type, c_op, asm_op, ll, sc) \ @@ -254,31 +265,4 @@ ATOMIC_SIP_OP(atomic64, s64, dsubu, lld, scd) #undef ATOMIC_SIP_OP -#define atomic_cmpxchg(v, o, n) (cmpxchg(&((v)->counter), (o), (n))) -#define atomic_xchg(v, new) (xchg(&((v)->counter), (new))) - -#ifdef CONFIG_64BIT - -#define ATOMIC64_INIT(i) { (i) } - -/* - * atomic64_read - read atomic variable - * @v: pointer of type atomic64_t - * - */ -#define atomic64_read(v) READ_ONCE((v)->counter) - -/* - * atomic64_set - set atomic variable - * @v: pointer of type atomic64_t - * @i: required value - */ -#define atomic64_set(v, i) WRITE_ONCE((v)->counter, (i)) - -#define atomic64_cmpxchg(v, o, n) \ - ((__typeof__((v)->counter))cmpxchg(&((v)->counter), (o), (n))) -#define atomic64_xchg(v, new) (xchg(&((v)->counter), (new))) - -#endif /* CONFIG_64BIT */ - #endif /* _ASM_ATOMIC_H */ From patchwork Tue Oct 1 21:53:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Burton X-Patchwork-Id: 11169785 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EDD701599 for ; Tue, 1 Oct 2019 21:53:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id ACFF921D80 for ; Tue, 1 Oct 2019 21:53:33 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=wavecomp.com header.i=@wavecomp.com header.b="aKGRDpIF" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728159AbfJAVxd (ORCPT ); Tue, 1 Oct 2019 17:53:33 -0400 Received: from mail-eopbgr730113.outbound.protection.outlook.com ([40.107.73.113]:17171 "EHLO NAM05-DM3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728135AbfJAVxc (ORCPT ); Tue, 1 Oct 2019 17:53:32 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ZebZc8497xg5jL1sorMVjGXfrVeCtoAMJPaofccsG62wg9v2O1Ger4LSIFr3NU8c3sbgfa32s0rlXCs9eYzfA6tHgTwCwSYBIG1akJidb98i5k82Z2xAchE+iARurR1pBiOhVk1a8KY9bGQTCz1LjRCLbFze8SjK/Pwd9xqB9BoEr1bEF6Y7cb5Gllm36OcWmscOpPbB21oFpxcwWCIEIF92hcby+2L6DbL3xwRDohBksi3JLZUmvW5yv+FjSt6WoM9OyG0T6AxCa0SFlPIUCy4LSpMuCCUKBUSyYjq7YPmAe0eXxbaDYavt/dGL4IfaO7k2Ap3qmDi/MIYQt9YD2A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=EkvCPIfFuh5pfYvK0a3TRFuqTs94DeVNZNPt98OTCww=; b=NP2Y26BPXQzUniLDUnnlOHMx+l7UHhAE9QY947VXv9h69MCzrexnSVm2URR0wzlD+ZL0a3S10chT5tu3oSwOeY3MRIYfgEAgB/o4C3tpwVDdItdN8MdJHO6OzSTjr0JTabL/Ll1xHwJ8hkI14Dt7d/kZWX8WAyYA70HlS7BUa2WcR9XryuxSR/gXGbDI8OExZGCDbAz+JNRpi4FicgGFcRvfljLS/URADjn928Wf34y6yjNBCqgsI+HIlIs/kvNGkNwMncColhmyNtNJcLdvTgfQn5rM+qmJAR925amw7hndNXD0CurDB/HpuTS1VgQxN4np7k6nVU2ORXWzOWO9Eg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wavecomp.com; dmarc=pass action=none header.from=mips.com; dkim=pass header.d=mips.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wavecomp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=EkvCPIfFuh5pfYvK0a3TRFuqTs94DeVNZNPt98OTCww=; b=aKGRDpIF+NjWJWvukJwe87GoJALSdJGd+nUZ6Fqld/YZFjPu259kbX03ipnNkSAMa6EDPrCyP5stOVeCg4tWbNJ1uDuprlCM1ui+0wtVr8l9Jc+KbfXpsn/7436g/gl5r6Ag7jPNZN0K4DGJ7lMbYME+2JIzOoU1o1Rk6nZEp/M= Received: from MWHPR2201MB1277.namprd22.prod.outlook.com (10.172.60.12) by MWHPR2201MB1439.namprd22.prod.outlook.com (10.174.169.166) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2305.20; Tue, 1 Oct 2019 21:53:25 +0000 Received: from MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481]) by MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481%2]) with mapi id 15.20.2305.022; Tue, 1 Oct 2019 21:53:25 +0000 From: Paul Burton To: "linux-mips@vger.kernel.org" CC: Huacai Chen , Jiaxun Yang , "linux-kernel@vger.kernel.org" , Paul Burton Subject: [PATCH v2 16/36] MIPS: bitops: Handle !kernel_uses_llsc first Thread-Topic: [PATCH v2 16/36] MIPS: bitops: Handle !kernel_uses_llsc first Thread-Index: AQHVeKKmaw5LyazGw02kyUdZnEF8yg== Date: Tue, 1 Oct 2019 21:53:25 +0000 Message-ID: <20191001215249.4157062-17-paul.burton@mips.com> References: <20191001215249.4157062-1-paul.burton@mips.com> In-Reply-To: <20191001215249.4157062-1-paul.burton@mips.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BYAPR07CA0093.namprd07.prod.outlook.com (2603:10b6:a03:12b::34) To MWHPR2201MB1277.namprd22.prod.outlook.com (2603:10b6:301:18::12) authentication-results: spf=none (sender IP is ) smtp.mailfrom=pburton@wavecomp.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.23.0 x-originating-ip: [12.94.197.246] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 46fc2c4b-0b12-40f6-5e86-08d746b9c91f x-ms-traffictypediagnostic: MWHPR2201MB1439: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:2399; x-forefront-prvs: 0177904E6B x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(396003)(39840400004)(366004)(346002)(136003)(376002)(199004)(189003)(186003)(11346002)(6512007)(1076003)(486006)(66066001)(26005)(6486002)(316002)(2906002)(5640700003)(6436002)(44832011)(42882007)(25786009)(446003)(6916009)(30864003)(386003)(6506007)(76176011)(478600001)(52116002)(2616005)(102836004)(476003)(5660300002)(305945005)(64756008)(66446008)(14444005)(2351001)(7736002)(99286004)(54906003)(14454004)(71190400001)(71200400001)(66946007)(66556008)(66476007)(36756003)(3846002)(6116002)(107886003)(4326008)(50226002)(81166006)(81156014)(8676002)(2501003)(8936002)(256004);DIR:OUT;SFP:1102;SCL:1;SRVR:MWHPR2201MB1439;H:MWHPR2201MB1277.namprd22.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; received-spf: None (protection.outlook.com: wavecomp.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: SkRMbwgMKuoaMn1Sji9B4p8QeGdeplqM3LQEYZwwYBKRdCpSEFsbMJ1US29mM/lZGx8NQBncdw1N6A7eiW7qWIbD90kRcrUicC/dBoWyPE+JG1mMB68JRNSeWBm1oEGbuaFX3/GWy7Yl2wh3ItLmm2NjNegy6ozVnT4+G3eKI3ur62ymcoyegxS8g/VGYKK4lAl8iy+kyLOvQ5dpSKRmmFx3KcmB6nVP36weRJNxY2vMXPoR2yBBDTXspbqKltnbyIOKTTRh+a2/vooJqpM9up25HloTUTBhEl7XPalYEl2Uj595GKyieByv3hyJI4v8fhU37C+VzIoom7+v12tTsj5Sl6swxSm32iLxhrFu3V0sMKAo9toJjC9g9YegJ5l7kpZacVjr1f2bZ9xmGsE6Ic9PQQi9WBaUkJHoO6gslC4= MIME-Version: 1.0 X-OriginatorOrg: mips.com X-MS-Exchange-CrossTenant-Network-Message-Id: 46fc2c4b-0b12-40f6-5e86-08d746b9c91f X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Oct 2019 21:53:25.6343 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 463607d3-1db3-40a0-8a29-970c56230104 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: JYK8AKt0342jdOZZoD5rh0KO3Pca1wj6+rT809ZFFl5RXraE61Pw+j95rklxbXjyNatUG3RoJ3pPzF55Jtij+Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR2201MB1439 Sender: linux-mips-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-mips@vger.kernel.org Reorder conditions in our various bitops functions that check kernel_uses_llsc such that they handle the !kernel_uses_llsc case first. This allows us to avoid the need to duplicate the kernel_uses_llsc check in all the other cases. For functions that don't involve barriers common to the various implementations, we switch to returning from within each if block making each case easier to read in isolation. Signed-off-by: Paul Burton --- Changes in v2: None arch/mips/include/asm/bitops.h | 213 ++++++++++++++++----------------- 1 file changed, 105 insertions(+), 108 deletions(-) diff --git a/arch/mips/include/asm/bitops.h b/arch/mips/include/asm/bitops.h index 985d6a02f9ea..e300960717e0 100644 --- a/arch/mips/include/asm/bitops.h +++ b/arch/mips/include/asm/bitops.h @@ -52,11 +52,16 @@ int __mips_test_and_change_bit(unsigned long nr, */ static inline void set_bit(unsigned long nr, volatile unsigned long *addr) { - unsigned long *m = ((unsigned long *) addr) + (nr >> SZLONG_LOG); + unsigned long *m = ((unsigned long *)addr) + (nr >> SZLONG_LOG); int bit = nr & SZLONG_MASK; unsigned long temp; - if (kernel_uses_llsc && R10000_LLSC_WAR) { + if (!kernel_uses_llsc) { + __mips_set_bit(nr, addr); + return; + } + + if (R10000_LLSC_WAR) { __asm__ __volatile__( " .set push \n" " .set arch=r4000 \n" @@ -68,8 +73,11 @@ static inline void set_bit(unsigned long nr, volatile unsigned long *addr) : "=&r" (temp), "=" GCC_OFF_SMALL_ASM() (*m) : "ir" (1UL << bit), GCC_OFF_SMALL_ASM() (*m) : __LLSC_CLOBBER); + return; + } + #if defined(CONFIG_CPU_MIPSR2) || defined(CONFIG_CPU_MIPSR6) - } else if (kernel_uses_llsc && __builtin_constant_p(bit)) { + if (__builtin_constant_p(bit)) { loongson_llsc_mb(); do { __asm__ __volatile__( @@ -80,23 +88,23 @@ static inline void set_bit(unsigned long nr, volatile unsigned long *addr) : "ir" (bit), "r" (~0) : __LLSC_CLOBBER); } while (unlikely(!temp)); + return; + } #endif /* CONFIG_CPU_MIPSR2 || CONFIG_CPU_MIPSR6 */ - } else if (kernel_uses_llsc) { - loongson_llsc_mb(); - do { - __asm__ __volatile__( - " .set push \n" - " .set "MIPS_ISA_ARCH_LEVEL" \n" - " " __LL "%0, %1 # set_bit \n" - " or %0, %2 \n" - " " __SC "%0, %1 \n" - " .set pop \n" - : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (*m) - : "ir" (1UL << bit) - : __LLSC_CLOBBER); - } while (unlikely(!temp)); - } else - __mips_set_bit(nr, addr); + + loongson_llsc_mb(); + do { + __asm__ __volatile__( + " .set push \n" + " .set "MIPS_ISA_ARCH_LEVEL" \n" + " " __LL "%0, %1 # set_bit \n" + " or %0, %2 \n" + " " __SC "%0, %1 \n" + " .set pop \n" + : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (*m) + : "ir" (1UL << bit) + : __LLSC_CLOBBER); + } while (unlikely(!temp)); } /* @@ -111,11 +119,16 @@ static inline void set_bit(unsigned long nr, volatile unsigned long *addr) */ static inline void clear_bit(unsigned long nr, volatile unsigned long *addr) { - unsigned long *m = ((unsigned long *) addr) + (nr >> SZLONG_LOG); + unsigned long *m = ((unsigned long *)addr) + (nr >> SZLONG_LOG); int bit = nr & SZLONG_MASK; unsigned long temp; - if (kernel_uses_llsc && R10000_LLSC_WAR) { + if (!kernel_uses_llsc) { + __mips_clear_bit(nr, addr); + return; + } + + if (R10000_LLSC_WAR) { __asm__ __volatile__( " .set push \n" " .set arch=r4000 \n" @@ -127,8 +140,11 @@ static inline void clear_bit(unsigned long nr, volatile unsigned long *addr) : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (*m) : "ir" (~(1UL << bit)) : __LLSC_CLOBBER); + return; + } + #if defined(CONFIG_CPU_MIPSR2) || defined(CONFIG_CPU_MIPSR6) - } else if (kernel_uses_llsc && __builtin_constant_p(bit)) { + if (__builtin_constant_p(bit)) { loongson_llsc_mb(); do { __asm__ __volatile__( @@ -139,23 +155,23 @@ static inline void clear_bit(unsigned long nr, volatile unsigned long *addr) : "ir" (bit) : __LLSC_CLOBBER); } while (unlikely(!temp)); + return; + } #endif /* CONFIG_CPU_MIPSR2 || CONFIG_CPU_MIPSR6 */ - } else if (kernel_uses_llsc) { - loongson_llsc_mb(); - do { - __asm__ __volatile__( - " .set push \n" - " .set "MIPS_ISA_ARCH_LEVEL" \n" - " " __LL "%0, %1 # clear_bit \n" - " and %0, %2 \n" - " " __SC "%0, %1 \n" - " .set pop \n" - : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (*m) - : "ir" (~(1UL << bit)) - : __LLSC_CLOBBER); - } while (unlikely(!temp)); - } else - __mips_clear_bit(nr, addr); + + loongson_llsc_mb(); + do { + __asm__ __volatile__( + " .set push \n" + " .set "MIPS_ISA_ARCH_LEVEL" \n" + " " __LL "%0, %1 # clear_bit \n" + " and %0, %2 \n" + " " __SC "%0, %1 \n" + " .set pop \n" + : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (*m) + : "ir" (~(1UL << bit)) + : __LLSC_CLOBBER); + } while (unlikely(!temp)); } /* @@ -183,12 +199,16 @@ static inline void clear_bit_unlock(unsigned long nr, volatile unsigned long *ad */ static inline void change_bit(unsigned long nr, volatile unsigned long *addr) { + unsigned long *m = ((unsigned long *)addr) + (nr >> SZLONG_LOG); int bit = nr & SZLONG_MASK; + unsigned long temp; - if (kernel_uses_llsc && R10000_LLSC_WAR) { - unsigned long *m = ((unsigned long *) addr) + (nr >> SZLONG_LOG); - unsigned long temp; + if (!kernel_uses_llsc) { + __mips_change_bit(nr, addr); + return; + } + if (R10000_LLSC_WAR) { __asm__ __volatile__( " .set push \n" " .set arch=r4000 \n" @@ -200,25 +220,22 @@ static inline void change_bit(unsigned long nr, volatile unsigned long *addr) : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (*m) : "ir" (1UL << bit) : __LLSC_CLOBBER); - } else if (kernel_uses_llsc) { - unsigned long *m = ((unsigned long *) addr) + (nr >> SZLONG_LOG); - unsigned long temp; + return; + } - loongson_llsc_mb(); - do { - __asm__ __volatile__( - " .set push \n" - " .set "MIPS_ISA_ARCH_LEVEL" \n" - " " __LL "%0, %1 # change_bit \n" - " xor %0, %2 \n" - " " __SC "%0, %1 \n" - " .set pop \n" - : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (*m) - : "ir" (1UL << bit) - : __LLSC_CLOBBER); - } while (unlikely(!temp)); - } else - __mips_change_bit(nr, addr); + loongson_llsc_mb(); + do { + __asm__ __volatile__( + " .set push \n" + " .set "MIPS_ISA_ARCH_LEVEL" \n" + " " __LL "%0, %1 # change_bit \n" + " xor %0, %2 \n" + " " __SC "%0, %1 \n" + " .set pop \n" + : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (*m) + : "ir" (1UL << bit) + : __LLSC_CLOBBER); + } while (unlikely(!temp)); } /* @@ -232,15 +249,15 @@ static inline void change_bit(unsigned long nr, volatile unsigned long *addr) static inline int test_and_set_bit(unsigned long nr, volatile unsigned long *addr) { + unsigned long *m = ((unsigned long *)addr) + (nr >> SZLONG_LOG); int bit = nr & SZLONG_MASK; - unsigned long res; + unsigned long res, temp; smp_mb__before_llsc(); - if (kernel_uses_llsc && R10000_LLSC_WAR) { - unsigned long *m = ((unsigned long *) addr) + (nr >> SZLONG_LOG); - unsigned long temp; - + if (!kernel_uses_llsc) { + res = __mips_test_and_set_bit(nr, addr); + } else if (R10000_LLSC_WAR) { __asm__ __volatile__( " .set push \n" " .set arch=r4000 \n" @@ -253,10 +270,7 @@ static inline int test_and_set_bit(unsigned long nr, : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (*m), "=&r" (res) : "r" (1UL << bit) : __LLSC_CLOBBER); - } else if (kernel_uses_llsc) { - unsigned long *m = ((unsigned long *) addr) + (nr >> SZLONG_LOG); - unsigned long temp; - + } else { loongson_llsc_mb(); do { __asm__ __volatile__( @@ -272,8 +286,7 @@ static inline int test_and_set_bit(unsigned long nr, } while (unlikely(!res)); res = temp & (1UL << bit); - } else - res = __mips_test_and_set_bit(nr, addr); + } smp_llsc_mb(); @@ -291,13 +304,13 @@ static inline int test_and_set_bit(unsigned long nr, static inline int test_and_set_bit_lock(unsigned long nr, volatile unsigned long *addr) { + unsigned long *m = ((unsigned long *)addr) + (nr >> SZLONG_LOG); int bit = nr & SZLONG_MASK; - unsigned long res; - - if (kernel_uses_llsc && R10000_LLSC_WAR) { - unsigned long *m = ((unsigned long *) addr) + (nr >> SZLONG_LOG); - unsigned long temp; + unsigned long res, temp; + if (!kernel_uses_llsc) { + res = __mips_test_and_set_bit_lock(nr, addr); + } else if (R10000_LLSC_WAR) { __asm__ __volatile__( " .set push \n" " .set arch=r4000 \n" @@ -310,11 +323,7 @@ static inline int test_and_set_bit_lock(unsigned long nr, : "=&r" (temp), "+m" (*m), "=&r" (res) : "r" (1UL << bit) : __LLSC_CLOBBER); - } else if (kernel_uses_llsc) { - unsigned long *m = ((unsigned long *) addr) + (nr >> SZLONG_LOG); - unsigned long temp; - - loongson_llsc_mb(); + } else { do { __asm__ __volatile__( " .set push \n" @@ -329,8 +338,7 @@ static inline int test_and_set_bit_lock(unsigned long nr, } while (unlikely(!res)); res = temp & (1UL << bit); - } else - res = __mips_test_and_set_bit_lock(nr, addr); + } smp_llsc_mb(); @@ -347,15 +355,15 @@ static inline int test_and_set_bit_lock(unsigned long nr, static inline int test_and_clear_bit(unsigned long nr, volatile unsigned long *addr) { + unsigned long *m = ((unsigned long *)addr) + (nr >> SZLONG_LOG); int bit = nr & SZLONG_MASK; - unsigned long res; + unsigned long res, temp; smp_mb__before_llsc(); - if (kernel_uses_llsc && R10000_LLSC_WAR) { - unsigned long *m = ((unsigned long *) addr) + (nr >> SZLONG_LOG); - unsigned long temp; - + if (!kernel_uses_llsc) { + res = __mips_test_and_clear_bit(nr, addr); + } else if (R10000_LLSC_WAR) { __asm__ __volatile__( " .set push \n" " .set arch=r4000 \n" @@ -370,10 +378,7 @@ static inline int test_and_clear_bit(unsigned long nr, : "r" (1UL << bit) : __LLSC_CLOBBER); #if defined(CONFIG_CPU_MIPSR2) || defined(CONFIG_CPU_MIPSR6) - } else if (kernel_uses_llsc && __builtin_constant_p(nr)) { - unsigned long *m = ((unsigned long *) addr) + (nr >> SZLONG_LOG); - unsigned long temp; - + } else if (__builtin_constant_p(nr)) { loongson_llsc_mb(); do { __asm__ __volatile__( @@ -386,10 +391,7 @@ static inline int test_and_clear_bit(unsigned long nr, : __LLSC_CLOBBER); } while (unlikely(!temp)); #endif - } else if (kernel_uses_llsc) { - unsigned long *m = ((unsigned long *) addr) + (nr >> SZLONG_LOG); - unsigned long temp; - + } else { loongson_llsc_mb(); do { __asm__ __volatile__( @@ -406,8 +408,7 @@ static inline int test_and_clear_bit(unsigned long nr, } while (unlikely(!res)); res = temp & (1UL << bit); - } else - res = __mips_test_and_clear_bit(nr, addr); + } smp_llsc_mb(); @@ -425,15 +426,15 @@ static inline int test_and_clear_bit(unsigned long nr, static inline int test_and_change_bit(unsigned long nr, volatile unsigned long *addr) { + unsigned long *m = ((unsigned long *)addr) + (nr >> SZLONG_LOG); int bit = nr & SZLONG_MASK; - unsigned long res; + unsigned long res, temp; smp_mb__before_llsc(); - if (kernel_uses_llsc && R10000_LLSC_WAR) { - unsigned long *m = ((unsigned long *) addr) + (nr >> SZLONG_LOG); - unsigned long temp; - + if (!kernel_uses_llsc) { + res = __mips_test_and_change_bit(nr, addr); + } else if (R10000_LLSC_WAR) { __asm__ __volatile__( " .set push \n" " .set arch=r4000 \n" @@ -446,10 +447,7 @@ static inline int test_and_change_bit(unsigned long nr, : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (*m), "=&r" (res) : "r" (1UL << bit) : __LLSC_CLOBBER); - } else if (kernel_uses_llsc) { - unsigned long *m = ((unsigned long *) addr) + (nr >> SZLONG_LOG); - unsigned long temp; - + } else { loongson_llsc_mb(); do { __asm__ __volatile__( @@ -465,8 +463,7 @@ static inline int test_and_change_bit(unsigned long nr, } while (unlikely(!res)); res = temp & (1UL << bit); - } else - res = __mips_test_and_change_bit(nr, addr); + } smp_llsc_mb(); From patchwork Tue Oct 1 21:53:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Burton X-Patchwork-Id: 11169787 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4F3D113B1 for ; Tue, 1 Oct 2019 21:53:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 2C23021D7B for ; Tue, 1 Oct 2019 21:53:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=wavecomp.com header.i=@wavecomp.com header.b="K9jTvt4k" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728172AbfJAVxe (ORCPT ); Tue, 1 Oct 2019 17:53:34 -0400 Received: from mail-eopbgr730113.outbound.protection.outlook.com ([40.107.73.113]:17171 "EHLO NAM05-DM3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728158AbfJAVxe (ORCPT ); Tue, 1 Oct 2019 17:53:34 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=H4oAd3TwORqGAaoeZKwF7uVar4dD3eJ9sbjHwRDI5dY+3EaDC52TydqZ+wLxQNvqTFRtiijv5bkT03XW4g7txlknu5Ft+TO0ZQMOSvu3u0rOhNPqTZf/wvs3AGWV2Yfk+RLUVQHQx2afZ1orb3OI2qhBa27rWt3j5QIvTvwOJHQcIwc4ElVXNbJLcPE1xQWWicVyi/2UxTPkqSIRemYLx0RlnmtpnfNUMzK10FTsLXVIDvG0pEdi0bMjH+GLR+3zK5RhuvFmj7xqxL9Xh8YghKI/xeK0TjMR/d4odbVLz2P2TTzOakGiXpWs51zoWtOjyCaN8Ur+pa7TDsRYUhMECQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=IfiRsDiQqxRk1eif9MMujcDB1YCLPcoEvYoKJruLovM=; b=iClaUV3jqnD/kRlwe1A33KUsRTvFqYtewfYKfNXHsaC5ITeZ1DYiDCHdQjMT4MEGSaVsey6EbaIttYN+BNxFevBrtpKwsels8PK/133fMxWoHm4HygXonkoJtqw4EIBlPeYF2kMUGWCYZ1mJVtvoOmkcp+WRxkoEYxZ6V9ITrHnvCYQZkb7DjWAXfvqiwMwWGlnFs08EsV8fHOPpnA7MHCbNggmuAfDjaGNIy+ybYDRhWBFrtVxx2u9yEMb1nH4zBD2Np2QNlIQIho25NAUBOjfGmu7hwJ2eXW/8Ejj364iv41OJD9dhpTWAbjOh94UZHUSr629okmguJf5Ys6GpFg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wavecomp.com; dmarc=pass action=none header.from=mips.com; dkim=pass header.d=mips.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wavecomp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=IfiRsDiQqxRk1eif9MMujcDB1YCLPcoEvYoKJruLovM=; b=K9jTvt4k4vAmGTz3AvIBw9hmCJ9vznLZb0W5Aubdn5ebjSD91A0vxlNwkL7yZvIxSM4Cl5VDUGfkhwkey4NfXQRyNgR5PXWSDce+M1wLt6eBlMxqOV3nbLZ2Z1wxZiWExmmpopi+fpMN/buq0KPEI1yJ5ESAVts1NI4JrtIB+Ew= Received: from MWHPR2201MB1277.namprd22.prod.outlook.com (10.172.60.12) by MWHPR2201MB1439.namprd22.prod.outlook.com (10.174.169.166) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2305.20; Tue, 1 Oct 2019 21:53:26 +0000 Received: from MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481]) by MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481%2]) with mapi id 15.20.2305.022; Tue, 1 Oct 2019 21:53:26 +0000 From: Paul Burton To: "linux-mips@vger.kernel.org" CC: Huacai Chen , Jiaxun Yang , "linux-kernel@vger.kernel.org" , Paul Burton Subject: [PATCH v2 17/36] MIPS: bitops: Only use ins for bit 16 or higher Thread-Topic: [PATCH v2 17/36] MIPS: bitops: Only use ins for bit 16 or higher Thread-Index: AQHVeKKn4vPBf6q7eEKYatE8kpS2FQ== Date: Tue, 1 Oct 2019 21:53:26 +0000 Message-ID: <20191001215249.4157062-18-paul.burton@mips.com> References: <20191001215249.4157062-1-paul.burton@mips.com> In-Reply-To: <20191001215249.4157062-1-paul.burton@mips.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BYAPR07CA0093.namprd07.prod.outlook.com (2603:10b6:a03:12b::34) To MWHPR2201MB1277.namprd22.prod.outlook.com (2603:10b6:301:18::12) authentication-results: spf=none (sender IP is ) smtp.mailfrom=pburton@wavecomp.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.23.0 x-originating-ip: [12.94.197.246] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 75f2181c-5fb9-4009-2844-08d746b9c9bc x-ms-traffictypediagnostic: MWHPR2201MB1439: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:7219; x-forefront-prvs: 0177904E6B x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(396003)(39840400004)(366004)(346002)(136003)(376002)(199004)(189003)(186003)(11346002)(6512007)(1076003)(486006)(66066001)(26005)(6486002)(316002)(2906002)(5640700003)(6436002)(44832011)(42882007)(25786009)(446003)(6916009)(386003)(6506007)(76176011)(478600001)(52116002)(2616005)(102836004)(476003)(5660300002)(305945005)(64756008)(66446008)(14444005)(2351001)(7736002)(99286004)(54906003)(14454004)(71190400001)(71200400001)(66946007)(66556008)(66476007)(36756003)(3846002)(6116002)(107886003)(4326008)(50226002)(81166006)(81156014)(8676002)(2501003)(8936002)(256004);DIR:OUT;SFP:1102;SCL:1;SRVR:MWHPR2201MB1439;H:MWHPR2201MB1277.namprd22.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; received-spf: None (protection.outlook.com: wavecomp.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: xwFmUgZRnN/fc2FQIqrdzqBOEWrULQgJflhTSoP83DIvAJaxlAVQRfTQyP5PuvNSSE7oyoZBeGyOHMgAGXNVBC70ECptFRr9UNV8iF0zq8Z1ny1eudSxSBE4jSTq3Hhz6c6MJgsaj0ANGNoenwja7tesH+LF6kj1gHN8RQ6/VDeDqcVa/sM7OgbzYUmm+JHAN9FPOCh38newSfua7lnc9krH6C5F4t8xhqyNFPdkHnV/ER4NjtSQWZmZTMxJ2yD5+S+7gGyig/3C+8RTL1otoX60LzgXFdk7CBYWd+uQBv4i8y+Am8jEu7PxToxvWqvnUdULPyqJIyUIoodx808lac5po5Tut/0ReWjgqozgKxRfWi/zDm+XNldXwM2WPuWkbNq78Z45TilDqIOdQ0DEHva/DFqVAgHjO6U4HbjKo2o= MIME-Version: 1.0 X-OriginatorOrg: mips.com X-MS-Exchange-CrossTenant-Network-Message-Id: 75f2181c-5fb9-4009-2844-08d746b9c9bc X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Oct 2019 21:53:26.5429 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 463607d3-1db3-40a0-8a29-970c56230104 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: j+0eaYBHxe5dgOc8tlNJe8CqCwyDQZvTo6lEmR+gAg19D5GChaYlUL10igXucTWi+giYgWuyxem9mL8OARnaxA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR2201MB1439 Sender: linux-mips-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-mips@vger.kernel.org set_bit() can set bits 0-15 using an ori instruction, rather than loading the value -1 into a register & then using an ins instruction. That is, rather than the following: li t0, -1 ll t1, 0(t2) ins t1, t0, 4, 1 sc t1, 0(t2) We can have the simpler: ll t1, 0(t2) ori t1, t1, 0x10 sc t1, 0(t2) The or path already allows immediates to be used, so simply restricting the ins path to bits that don't fit in immediates is sufficient to take advantage of this. Signed-off-by: Paul Burton --- Changes in v2: None arch/mips/include/asm/bitops.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/mips/include/asm/bitops.h b/arch/mips/include/asm/bitops.h index e300960717e0..1e5739191ddf 100644 --- a/arch/mips/include/asm/bitops.h +++ b/arch/mips/include/asm/bitops.h @@ -77,7 +77,7 @@ static inline void set_bit(unsigned long nr, volatile unsigned long *addr) } #if defined(CONFIG_CPU_MIPSR2) || defined(CONFIG_CPU_MIPSR6) - if (__builtin_constant_p(bit)) { + if (__builtin_constant_p(bit) && (bit >= 16)) { loongson_llsc_mb(); do { __asm__ __volatile__( From patchwork Tue Oct 1 21:53:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Burton X-Patchwork-Id: 11169825 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1D0921599 for ; Tue, 1 Oct 2019 21:55:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id E525221A4A for ; Tue, 1 Oct 2019 21:55:02 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=wavecomp.com header.i=@wavecomp.com header.b="LiXFfHKT" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728544AbfJAVzC (ORCPT ); Tue, 1 Oct 2019 17:55:02 -0400 Received: from mail-eopbgr780114.outbound.protection.outlook.com ([40.107.78.114]:49402 "EHLO NAM03-BY2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728122AbfJAVx3 (ORCPT ); Tue, 1 Oct 2019 17:53:29 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Z5VXRcMMj+vhzrIbeaty0ivIBBqfwhJxUgaNnbeKDitZ/k0NCS3MPLv/nwToBdWE6GRCvA9MZlaxZLtEKPbcDgoajGv+LkTnbeJ+wpCPv/0JN1O0oN0xwN2eyXEIb1moAAM06mphSf/wuSkwEcSjweKV9wFtlen1Z/ZiSgiOOoKaud/BqDNv5uWSBncJJoX17aYQ+JC5R+vSKDTzc4x6vjBits3m3UL1bQ81SudEz84hUMwJisPBe9SvVGO8CJKVYSGqss06uwfIII5FeRFNGOWjgqDWTTwXmw6k6QvprXqUOyOnrK+SIMXCwsfzK+rgs8YmKAFUj7KPOdEeGkhaAw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=VVeqRdQx8uxlqlfyAFwW6/7HHFA0hQKyR9N3fC+u/Cg=; b=DmkVAfyNjs3S1LUGJcfFRLrFws4wQU3SZqjKA17BcPpvDFMO6v7sZToP+LaIreJl5JGjEd8TdNqGATd6rEsIBier6AbF2c1DYi8Fm7nqoCzYT/KTYbhDMa5srwXrD4eh7uvgHp3BAO/O5FluQTiAv0HmG0LO33Q5q1/kBLrEsnR6z5ARET5vwMiqAhC6j3uXw6KcpbMtVv0jXOYBl9Gu7h6YhjQ8uvJpGRjE3apBIqDX1RQ6u9ZKmH0FfOcrrApO+M3lPIxUp3AzIepcJ5AD5DlO5bQ58FN9hBljZD3sOvuJC6NTJUA7l5T5pSes2xRqYTp/a7ZqEmffSm25j5zu0g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wavecomp.com; dmarc=pass action=none header.from=mips.com; dkim=pass header.d=mips.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wavecomp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=VVeqRdQx8uxlqlfyAFwW6/7HHFA0hQKyR9N3fC+u/Cg=; b=LiXFfHKTf/c7u6gL7HJ2fjVyuL1nnTpOGgzkyxFavQS1I7Npz112SW/B5TPjMHs8B+4V9jHagG582mBrUcPJQXBOZFN/ezM1FzaTh+TBV0LJVLV/7tg8NAulaV0TJvKmN/uR3Zo1UAtu99XkjqVyvnGhT5XNeRJWU6t8ojmFzUY= Received: from MWHPR2201MB1277.namprd22.prod.outlook.com (10.172.60.12) by MWHPR2201MB1213.namprd22.prod.outlook.com (10.172.61.148) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2305.20; Tue, 1 Oct 2019 21:53:27 +0000 Received: from MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481]) by MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481%2]) with mapi id 15.20.2305.022; Tue, 1 Oct 2019 21:53:27 +0000 From: Paul Burton To: "linux-mips@vger.kernel.org" CC: Huacai Chen , Jiaxun Yang , "linux-kernel@vger.kernel.org" , Paul Burton Subject: [PATCH v2 18/36] MIPS: bitops: Use MIPS_ISA_REV, not #ifdefs Thread-Topic: [PATCH v2 18/36] MIPS: bitops: Use MIPS_ISA_REV, not #ifdefs Thread-Index: AQHVeKKngmDrZVAa20Gg4WzhINML+w== Date: Tue, 1 Oct 2019 21:53:27 +0000 Message-ID: <20191001215249.4157062-19-paul.burton@mips.com> References: <20191001215249.4157062-1-paul.burton@mips.com> In-Reply-To: <20191001215249.4157062-1-paul.burton@mips.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BYAPR07CA0093.namprd07.prod.outlook.com (2603:10b6:a03:12b::34) To MWHPR2201MB1277.namprd22.prod.outlook.com (2603:10b6:301:18::12) authentication-results: spf=none (sender IP is ) smtp.mailfrom=pburton@wavecomp.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.23.0 x-originating-ip: [12.94.197.246] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: fa24ba13-7f02-4d32-56f4-08d746b9ca3d x-ms-traffictypediagnostic: MWHPR2201MB1213: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:3044; x-forefront-prvs: 0177904E6B x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(346002)(396003)(366004)(376002)(136003)(39840400004)(199004)(189003)(6512007)(5640700003)(54906003)(2501003)(3846002)(6486002)(5660300002)(6436002)(66066001)(6116002)(1076003)(50226002)(2906002)(76176011)(52116002)(6506007)(107886003)(2351001)(4326008)(6916009)(102836004)(71190400001)(8936002)(386003)(25786009)(36756003)(44832011)(8676002)(71200400001)(81166006)(81156014)(186003)(256004)(11346002)(14444005)(2616005)(7736002)(316002)(42882007)(305945005)(99286004)(66446008)(64756008)(66556008)(478600001)(66946007)(66476007)(486006)(446003)(476003)(14454004)(26005);DIR:OUT;SFP:1102;SCL:1;SRVR:MWHPR2201MB1213;H:MWHPR2201MB1277.namprd22.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; received-spf: None (protection.outlook.com: wavecomp.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: SfrP+nZnYz4gpBrMwLAlWVyW3i+IXquODu17ejqBVmc1L/ep8b2Bqn/DaqyXr0c2/Z9iIRBOxSed/ZvSEH5zVHO+khgBtVm3A5IeAhyJ5UMYX6Y6b7qD2e+Qms8ZptIOvVA6bLMwwNxD0GC8QNT2zhQ+E+FjgXVjOToxSzWO1zs1tPMu5f/ONZUeUkJbCOhu0gH2NODyBBgb2MGR/FGTBB0AkqJeW6CyPH+fOKOVhoKhG3aCTOXOzfqCRDqpB09QDH6ZOegGTQ/NAw/04mtkn47zfGRFgDCfZ1hNceLRtntCbtakbjQSm93IrjTgTgLzoVrcOB5FYbyEMcdtnwNp88muav5NFIrh9upv7Q45aNC2mS/iJJcrulD52WrPwbRI6uTS9Puqpl1Pbp1KwIzPP4gxIrpvGE5LVVGYaKpIMqk= MIME-Version: 1.0 X-OriginatorOrg: mips.com X-MS-Exchange-CrossTenant-Network-Message-Id: fa24ba13-7f02-4d32-56f4-08d746b9ca3d X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Oct 2019 21:53:27.3745 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 463607d3-1db3-40a0-8a29-970c56230104 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 54EYtv8P97pQ41RHp63z20ChkRbdI6DCw4a5rohz6lbmpBk8NykOz56046eXC2fZOFdwxUZjIAXUDq/8F0pNpQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR2201MB1213 Sender: linux-mips-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-mips@vger.kernel.org Rather than #ifdef on CONFIG_CPU_* to determine whether the ins instruction is supported we can simply check MIPS_ISA_REV to discover whether we're targeting MIPSr2 or higher. Do so in order to clean up the code. Signed-off-by: Paul Burton --- Changes in v2: None arch/mips/include/asm/bitops.h | 13 ++++--------- 1 file changed, 4 insertions(+), 9 deletions(-) diff --git a/arch/mips/include/asm/bitops.h b/arch/mips/include/asm/bitops.h index 1e5739191ddf..0f5329e32e87 100644 --- a/arch/mips/include/asm/bitops.h +++ b/arch/mips/include/asm/bitops.h @@ -19,6 +19,7 @@ #include /* sigh ... */ #include #include +#include #include #include #include @@ -76,8 +77,7 @@ static inline void set_bit(unsigned long nr, volatile unsigned long *addr) return; } -#if defined(CONFIG_CPU_MIPSR2) || defined(CONFIG_CPU_MIPSR6) - if (__builtin_constant_p(bit) && (bit >= 16)) { + if ((MIPS_ISA_REV >= 2) && __builtin_constant_p(bit) && (bit >= 16)) { loongson_llsc_mb(); do { __asm__ __volatile__( @@ -90,7 +90,6 @@ static inline void set_bit(unsigned long nr, volatile unsigned long *addr) } while (unlikely(!temp)); return; } -#endif /* CONFIG_CPU_MIPSR2 || CONFIG_CPU_MIPSR6 */ loongson_llsc_mb(); do { @@ -143,8 +142,7 @@ static inline void clear_bit(unsigned long nr, volatile unsigned long *addr) return; } -#if defined(CONFIG_CPU_MIPSR2) || defined(CONFIG_CPU_MIPSR6) - if (__builtin_constant_p(bit)) { + if ((MIPS_ISA_REV >= 2) && __builtin_constant_p(bit)) { loongson_llsc_mb(); do { __asm__ __volatile__( @@ -157,7 +155,6 @@ static inline void clear_bit(unsigned long nr, volatile unsigned long *addr) } while (unlikely(!temp)); return; } -#endif /* CONFIG_CPU_MIPSR2 || CONFIG_CPU_MIPSR6 */ loongson_llsc_mb(); do { @@ -377,8 +374,7 @@ static inline int test_and_clear_bit(unsigned long nr, : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (*m), "=&r" (res) : "r" (1UL << bit) : __LLSC_CLOBBER); -#if defined(CONFIG_CPU_MIPSR2) || defined(CONFIG_CPU_MIPSR6) - } else if (__builtin_constant_p(nr)) { + } else if ((MIPS_ISA_REV >= 2) && __builtin_constant_p(nr)) { loongson_llsc_mb(); do { __asm__ __volatile__( @@ -390,7 +386,6 @@ static inline int test_and_clear_bit(unsigned long nr, : "ir" (bit) : __LLSC_CLOBBER); } while (unlikely(!temp)); -#endif } else { loongson_llsc_mb(); do { From patchwork Tue Oct 1 21:53:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Burton X-Patchwork-Id: 11169823 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5020713B1 for ; Tue, 1 Oct 2019 21:54:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 24656222C9 for ; Tue, 1 Oct 2019 21:54:56 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=wavecomp.com header.i=@wavecomp.com header.b="o42ZpeGP" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728190AbfJAVxh (ORCPT ); Tue, 1 Oct 2019 17:53:37 -0400 Received: from mail-eopbgr730113.outbound.protection.outlook.com ([40.107.73.113]:17171 "EHLO NAM05-DM3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728177AbfJAVxg (ORCPT ); Tue, 1 Oct 2019 17:53:36 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=RJCab2LgaLrvJyYrUnFOSa9RcFME5qBcRP6NKxM5c/jx+DVUIOaNfX8oaLTah2KWiom/1f9duZhf4Wu4tMu2zxEc1FsWGgIJOVqOFOgp0ZSZj5T1dZPKkpOMkaMMUOM57NFrvwpXA3SVTJm+ngMtv7QO6AW6kXrt7UCr0etDwzIPddjkqBVjy1/CJoKLEYV9c/o0TZe934oOGxiHj1fJX8SId7EwtiOoz4kwf+hxiAlbQG5hSs4iIyNupIyFUZrMXKA60kKNX6mSiikNB4kXV5r33gbTmIIzVJn1b8Hl6sFrGv8he+Mt8EiaHfzculKabdDcZKWhfU3mj5ILeCduAA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wfQFj92XwiM5Yll6ITwBRnUHmK79aZPRXQ36V53lwPs=; b=afQ8TXxmy/FqSd5so196E4HVcUeiyt9jiacGyT5Py7251Uu9Rk/IAIBSANQz4l2joyiEbZbf5wpMKJ++yRzZ2a0V5oEYGuia2jxPxEIPn15A6WwXkQtZw4Kjp/WHdRU7tUhKkt4qZhhYoWIO+EPrgSw4NTM38zEaF5ml1LyPY/hxLjNIFw1cn98CDxt2A/tYUvmPdK425uKJetDBOveVM4Ugjuw0JtK2RK7QSquHdyGVMqVHpfU9gxFwiTMOsEgCYucuz8phkDuK2yovNhyOLDsAHk/VwoLLPDNFhZKiHXDw0Cm/NzHs7MW6YY0klIhstQC9L3zp3PJQf4jRWW0MtQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wavecomp.com; dmarc=pass action=none header.from=mips.com; dkim=pass header.d=mips.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wavecomp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wfQFj92XwiM5Yll6ITwBRnUHmK79aZPRXQ36V53lwPs=; b=o42ZpeGPUWIHtUH6cCDu9WTgYZjE/OdtdRPUqDUQoAQ9rh54kTstPBNf+xt4qtoqBVpKRUp1TIFfCpr29yFpOrYucYepOWpRBLkhdWvze5yy14FUl+2Qa0SWdycOYiyBhpIJFzSN+z9ZXBCkpkPhApkz/Cge6KMGkMr/wif5D/E= Received: from MWHPR2201MB1277.namprd22.prod.outlook.com (10.172.60.12) by MWHPR2201MB1439.namprd22.prod.outlook.com (10.174.169.166) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2305.20; Tue, 1 Oct 2019 21:53:29 +0000 Received: from MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481]) by MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481%2]) with mapi id 15.20.2305.022; Tue, 1 Oct 2019 21:53:29 +0000 From: Paul Burton To: "linux-mips@vger.kernel.org" CC: Huacai Chen , Jiaxun Yang , "linux-kernel@vger.kernel.org" , Paul Burton Subject: [PATCH v2 19/36] MIPS: bitops: ins start position is always an immediate Thread-Topic: [PATCH v2 19/36] MIPS: bitops: ins start position is always an immediate Thread-Index: AQHVeKKpGFkNJ1wM/EuB/qgdkW07aw== Date: Tue, 1 Oct 2019 21:53:29 +0000 Message-ID: <20191001215249.4157062-20-paul.burton@mips.com> References: <20191001215249.4157062-1-paul.burton@mips.com> In-Reply-To: <20191001215249.4157062-1-paul.burton@mips.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BYAPR07CA0093.namprd07.prod.outlook.com (2603:10b6:a03:12b::34) To MWHPR2201MB1277.namprd22.prod.outlook.com (2603:10b6:301:18::12) authentication-results: spf=none (sender IP is ) smtp.mailfrom=pburton@wavecomp.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.23.0 x-originating-ip: [12.94.197.246] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 468e0ca3-ee65-49c3-4969-08d746b9cb81 x-ms-traffictypediagnostic: MWHPR2201MB1439: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:2887; x-forefront-prvs: 0177904E6B x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(396003)(39840400004)(366004)(346002)(136003)(376002)(199004)(189003)(186003)(11346002)(6512007)(1076003)(486006)(66066001)(26005)(6486002)(316002)(2906002)(5640700003)(6436002)(44832011)(42882007)(25786009)(446003)(6916009)(386003)(6506007)(76176011)(478600001)(52116002)(2616005)(102836004)(476003)(5660300002)(305945005)(64756008)(66446008)(2351001)(7736002)(99286004)(54906003)(14454004)(71190400001)(71200400001)(66946007)(66556008)(66476007)(36756003)(3846002)(6116002)(107886003)(4326008)(50226002)(81166006)(81156014)(8676002)(2501003)(8936002)(256004)(14773001);DIR:OUT;SFP:1102;SCL:1;SRVR:MWHPR2201MB1439;H:MWHPR2201MB1277.namprd22.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; received-spf: None (protection.outlook.com: wavecomp.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: c6ks9cX5dYcPxi7uPV1n4z1Wm5ToFFHkt2GHB7u935/bVkaYx9XI7oecMEACZCaKUO04Hcg+MXKnoJQ2UA4TP/NPQ3eyZamjRD7bMY+OvVTenrSSgRMcMPrJgg6lSGelRI61VdJ8GM7yKGhduFHccV7NsODXxiq+R4Ic8mhZkgN4zw88BZ/WiW29T305Vtmrmlg/sObDsKFrGJBwfl79mOQSudIjDt6YWzM+07BVYRmkBdZEeYg8j6o/wz3T19PgEi66yem3pR0b/7GjIrr4Rz55CB1Hur+9XB44yS7s0Bhk4zvoNSEpt70Eklv3tZMt3u667YuRBUeBhR4ajhSShH+G2BwLqmDPzP7pwzWoTwGVsh8/7YpO8niSBpQfpUUYF92DcTaf+2vxZDbSkfM1XyU3Id/E2pKxjdBvTpG5RyQ= MIME-Version: 1.0 X-OriginatorOrg: mips.com X-MS-Exchange-CrossTenant-Network-Message-Id: 468e0ca3-ee65-49c3-4969-08d746b9cb81 X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Oct 2019 21:53:29.4475 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 463607d3-1db3-40a0-8a29-970c56230104 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: OX7LwPCZnvbTKg2EQrBMW4QMfNLCw3lQ3TirWoN4tqszFt5imivMI69r9VDmmkaop6kqyT26RnfEJqi0DV08cQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR2201MB1439 Sender: linux-mips-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-mips@vger.kernel.org The start position for an ins instruction is always encoded as an immediate, so allowing registers to be used by the inline asm makes no sense. It should never happen anyway since a bit index should always be small enough to be treated as an immediate, but remove the nonsensical "r" for sanity. Signed-off-by: Paul Burton --- Changes in v2: None arch/mips/include/asm/bitops.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/arch/mips/include/asm/bitops.h b/arch/mips/include/asm/bitops.h index 0f5329e32e87..03532ae9f528 100644 --- a/arch/mips/include/asm/bitops.h +++ b/arch/mips/include/asm/bitops.h @@ -85,7 +85,7 @@ static inline void set_bit(unsigned long nr, volatile unsigned long *addr) " " __INS "%0, %3, %2, 1 \n" " " __SC "%0, %1 \n" : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (*m) - : "ir" (bit), "r" (~0) + : "i" (bit), "r" (~0) : __LLSC_CLOBBER); } while (unlikely(!temp)); return; @@ -150,7 +150,7 @@ static inline void clear_bit(unsigned long nr, volatile unsigned long *addr) " " __INS "%0, $0, %2, 1 \n" " " __SC "%0, %1 \n" : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (*m) - : "ir" (bit) + : "i" (bit) : __LLSC_CLOBBER); } while (unlikely(!temp)); return; @@ -383,7 +383,7 @@ static inline int test_and_clear_bit(unsigned long nr, " " __INS "%0, $0, %3, 1 \n" " " __SC "%0, %1 \n" : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (*m), "=&r" (res) - : "ir" (bit) + : "i" (bit) : __LLSC_CLOBBER); } while (unlikely(!temp)); } else { From patchwork Tue Oct 1 21:53:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Burton X-Patchwork-Id: 11169791 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 90ED11709 for ; Tue, 1 Oct 2019 21:53:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 6E94B2133F for ; Tue, 1 Oct 2019 21:53:46 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=wavecomp.com header.i=@wavecomp.com header.b="mcmvXfwi" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728241AbfJAVxp (ORCPT ); Tue, 1 Oct 2019 17:53:45 -0400 Received: from mail-eopbgr730113.outbound.protection.outlook.com ([40.107.73.113]:17171 "EHLO NAM05-DM3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728188AbfJAVxi (ORCPT ); Tue, 1 Oct 2019 17:53:38 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LEQMBzS2/VV3yC2wtCoLAWGYjZW7OfyD2sBK9ribcgJ4kz/4/GzsbGgilWV/Tbu6/0Hrp2DPvwjC34lIxmHC7TQhuuIApKJN+GXIkr3oP2+35amX9j+ELMXCKpzgFtdrdlfK2aaR2sXCc3w+PKqFOgC5e8eMXa1DDeure3AyNe3f2ecD3ugGNSiUuEiMbcSeZg5G/AtbEIuDYWff8sIQMWkmDVZMMW3hB4O9GtaYjbCfziq7DsaBtszYQiLUnjcDjUSOa+8S3HXhq1L42e43RxPTF593ejlY0fE1VMblVIQsishAbx08vBfruHPtdeADdYXC9KN8fLZyMWhFcyl3nQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=lWYJd2itdUXXQdiCnhaWS4h86JUAe1WbuhxmdT9ZDs4=; b=fQ8WgVW1Viwytd0PmUkSUD7kqpWzQsRBJUPmV/jqNOvn68VtoPf8DbxzKJh3p2LEeL7WqcyOG/17BvhE5kquGNVt7jXFwKMgwxcjojHiA6aw2UMcB1hpE8gPXC6RK6RTVJxujJGesjz82nkWFAuGHWBgPi89E7sxnxYlcxtaTKaCNLFw2uexLAXONYEBlF7Dz1kJjASajNT6ADJ42+XqtduM1PSKEEHRZnPP1olajcz+2CH2bAG1eED3HaFe+/KhWhmTjybBynm5Z4t2fdA1rtVlqBQFuJeYRF8RjqGcdSUb7xpTe7wddTGC8EbfcjlVJpNVJv+wr2lges7LHD+tOw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wavecomp.com; dmarc=pass action=none header.from=mips.com; dkim=pass header.d=mips.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wavecomp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=lWYJd2itdUXXQdiCnhaWS4h86JUAe1WbuhxmdT9ZDs4=; b=mcmvXfwiMkkMUQXMkqEKgC8/uoPDwlFU10/MojuGZto7uoMtB/C2NSHQ4iN42EsD7kCPz7YOz71zKDzc/bT7bZvnnZkvLoAn91Kj2nMOT38RgBOraziQzi+x0A2I+fyMFY0ltGNzBTjXzNjdtIGfw3DtsbTLHsA/3rWaFWczXFU= Received: from MWHPR2201MB1277.namprd22.prod.outlook.com (10.172.60.12) by MWHPR2201MB1439.namprd22.prod.outlook.com (10.174.169.166) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2305.20; Tue, 1 Oct 2019 21:53:30 +0000 Received: from MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481]) by MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481%2]) with mapi id 15.20.2305.022; Tue, 1 Oct 2019 21:53:30 +0000 From: Paul Burton To: "linux-mips@vger.kernel.org" CC: Huacai Chen , Jiaxun Yang , "linux-kernel@vger.kernel.org" , Paul Burton Subject: [PATCH v2 20/36] MIPS: bitops: Implement test_and_set_bit() in terms of _lock variant Thread-Topic: [PATCH v2 20/36] MIPS: bitops: Implement test_and_set_bit() in terms of _lock variant Thread-Index: AQHVeKKpuvngwP/IhkelpS2BN9urcQ== Date: Tue, 1 Oct 2019 21:53:30 +0000 Message-ID: <20191001215249.4157062-21-paul.burton@mips.com> References: <20191001215249.4157062-1-paul.burton@mips.com> In-Reply-To: <20191001215249.4157062-1-paul.burton@mips.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BYAPR07CA0093.namprd07.prod.outlook.com (2603:10b6:a03:12b::34) To MWHPR2201MB1277.namprd22.prod.outlook.com (2603:10b6:301:18::12) authentication-results: spf=none (sender IP is ) smtp.mailfrom=pburton@wavecomp.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.23.0 x-originating-ip: [12.94.197.246] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: bc2eeb68-7ce4-4f10-03b6-08d746b9cbf0 x-ms-traffictypediagnostic: MWHPR2201MB1439: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:4714; x-forefront-prvs: 0177904E6B x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(396003)(39840400004)(366004)(346002)(136003)(376002)(199004)(189003)(186003)(11346002)(6512007)(1076003)(486006)(66066001)(26005)(6486002)(316002)(2906002)(5640700003)(6436002)(44832011)(42882007)(25786009)(446003)(6916009)(386003)(6506007)(76176011)(478600001)(52116002)(2616005)(102836004)(476003)(5660300002)(305945005)(64756008)(66446008)(14444005)(2351001)(7736002)(99286004)(54906003)(14454004)(71190400001)(71200400001)(66946007)(66556008)(66476007)(36756003)(3846002)(6116002)(107886003)(4326008)(50226002)(81166006)(81156014)(8676002)(2501003)(8936002)(256004);DIR:OUT;SFP:1102;SCL:1;SRVR:MWHPR2201MB1439;H:MWHPR2201MB1277.namprd22.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; received-spf: None (protection.outlook.com: wavecomp.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: bN/IaQ7QKyX4FJLZOavGPVqKE3gv3Ima4B7Xl+0i7eKHwJB7R6jnOA3DkSI9oj3iVkQK3pxDKxjPknDx7OWs0CjVbktC5dy5OAEwtCNsAq+JLWlTQKCEBV24xtHCHKvURHREbwKB2dyUaL6v8UO5ST6y0at5UtsdEq/AFPSxLu0buA2jO5h3sYUOJ9ehF7KtQjkYRj2B4khYCEN/o7e/mklTHUOyQKs1/aUpUmTKbEycllTDMZxeUCCW2hDLzQv4pFdYppDNyVRqre380wy88XeE693G3eZEED+22znVtIZuazuvDE8izh/0whBZH87cqo0SSWldL5s5DS2AL++pLUrdEPgbChk0T1BKsRUjjzIWgtqWK0TGuclVY2RnZRC23yDJAKQfeFGdbJ2vkCy0ztdxKNHoOYGMCwt8pN+Fzrg= MIME-Version: 1.0 X-OriginatorOrg: mips.com X-MS-Exchange-CrossTenant-Network-Message-Id: bc2eeb68-7ce4-4f10-03b6-08d746b9cbf0 X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Oct 2019 21:53:30.2262 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 463607d3-1db3-40a0-8a29-970c56230104 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: B7rF5kz8P1ugKeaKZdQcMA5d0jkDINu2d8qkFqAzzfGrlbOM6b+reWRNU0utovLC7Cvrcjvz/ScRfsu+tSz1jQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR2201MB1439 Sender: linux-mips-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-mips@vger.kernel.org The only difference between test_and_set_bit() & test_and_set_bit_lock() is memory ordering barrier semantics - the former provides a full barrier whilst the latter only provides acquire semantics. We can therefore implement test_and_set_bit() in terms of test_and_set_bit_lock() with the addition of the extra memory barrier. Do this in order to avoid duplicating logic. Signed-off-by: Paul Burton --- Changes in v2: None arch/mips/include/asm/bitops.h | 66 +++++++--------------------------- arch/mips/lib/bitops.c | 26 -------------- 2 files changed, 13 insertions(+), 79 deletions(-) diff --git a/arch/mips/include/asm/bitops.h b/arch/mips/include/asm/bitops.h index 03532ae9f528..ea35a2e87b6d 100644 --- a/arch/mips/include/asm/bitops.h +++ b/arch/mips/include/asm/bitops.h @@ -31,8 +31,6 @@ void __mips_set_bit(unsigned long nr, volatile unsigned long *addr); void __mips_clear_bit(unsigned long nr, volatile unsigned long *addr); void __mips_change_bit(unsigned long nr, volatile unsigned long *addr); -int __mips_test_and_set_bit(unsigned long nr, - volatile unsigned long *addr); int __mips_test_and_set_bit_lock(unsigned long nr, volatile unsigned long *addr); int __mips_test_and_clear_bit(unsigned long nr, @@ -236,24 +234,22 @@ static inline void change_bit(unsigned long nr, volatile unsigned long *addr) } /* - * test_and_set_bit - Set a bit and return its old value + * test_and_set_bit_lock - Set a bit and return its old value * @nr: Bit to set * @addr: Address to count from * - * This operation is atomic and cannot be reordered. - * It also implies a memory barrier. + * This operation is atomic and implies acquire ordering semantics + * after the memory operation. */ -static inline int test_and_set_bit(unsigned long nr, +static inline int test_and_set_bit_lock(unsigned long nr, volatile unsigned long *addr) { unsigned long *m = ((unsigned long *)addr) + (nr >> SZLONG_LOG); int bit = nr & SZLONG_MASK; unsigned long res, temp; - smp_mb__before_llsc(); - if (!kernel_uses_llsc) { - res = __mips_test_and_set_bit(nr, addr); + res = __mips_test_and_set_bit_lock(nr, addr); } else if (R10000_LLSC_WAR) { __asm__ __volatile__( " .set push \n" @@ -264,7 +260,7 @@ static inline int test_and_set_bit(unsigned long nr, " beqzl %2, 1b \n" " and %2, %0, %3 \n" " .set pop \n" - : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (*m), "=&r" (res) + : "=&r" (temp), "+m" (*m), "=&r" (res) : "r" (1UL << bit) : __LLSC_CLOBBER); } else { @@ -291,56 +287,20 @@ static inline int test_and_set_bit(unsigned long nr, } /* - * test_and_set_bit_lock - Set a bit and return its old value + * test_and_set_bit - Set a bit and return its old value * @nr: Bit to set * @addr: Address to count from * - * This operation is atomic and implies acquire ordering semantics - * after the memory operation. + * This operation is atomic and cannot be reordered. + * It also implies a memory barrier. */ -static inline int test_and_set_bit_lock(unsigned long nr, +static inline int test_and_set_bit(unsigned long nr, volatile unsigned long *addr) { - unsigned long *m = ((unsigned long *)addr) + (nr >> SZLONG_LOG); - int bit = nr & SZLONG_MASK; - unsigned long res, temp; - - if (!kernel_uses_llsc) { - res = __mips_test_and_set_bit_lock(nr, addr); - } else if (R10000_LLSC_WAR) { - __asm__ __volatile__( - " .set push \n" - " .set arch=r4000 \n" - "1: " __LL "%0, %1 # test_and_set_bit \n" - " or %2, %0, %3 \n" - " " __SC "%2, %1 \n" - " beqzl %2, 1b \n" - " and %2, %0, %3 \n" - " .set pop \n" - : "=&r" (temp), "+m" (*m), "=&r" (res) - : "r" (1UL << bit) - : __LLSC_CLOBBER); - } else { - do { - __asm__ __volatile__( - " .set push \n" - " .set "MIPS_ISA_ARCH_LEVEL" \n" - " " __LL "%0, %1 # test_and_set_bit \n" - " or %2, %0, %3 \n" - " " __SC "%2, %1 \n" - " .set pop \n" - : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (*m), "=&r" (res) - : "r" (1UL << bit) - : __LLSC_CLOBBER); - } while (unlikely(!res)); - - res = temp & (1UL << bit); - } - - smp_llsc_mb(); - - return res != 0; + smp_mb__before_llsc(); + return test_and_set_bit_lock(nr, addr); } + /* * test_and_clear_bit - Clear a bit and return its old value * @nr: Bit to clear diff --git a/arch/mips/lib/bitops.c b/arch/mips/lib/bitops.c index 3b2a1e78a543..fba402c0879d 100644 --- a/arch/mips/lib/bitops.c +++ b/arch/mips/lib/bitops.c @@ -77,32 +77,6 @@ void __mips_change_bit(unsigned long nr, volatile unsigned long *addr) EXPORT_SYMBOL(__mips_change_bit); -/** - * __mips_test_and_set_bit - Set a bit and return its old value. This is - * called by test_and_set_bit() if it cannot find a faster solution. - * @nr: Bit to set - * @addr: Address to count from - */ -int __mips_test_and_set_bit(unsigned long nr, - volatile unsigned long *addr) -{ - unsigned long *a = (unsigned long *)addr; - unsigned bit = nr & SZLONG_MASK; - unsigned long mask; - unsigned long flags; - int res; - - a += nr >> SZLONG_LOG; - mask = 1UL << bit; - raw_local_irq_save(flags); - res = (mask & *a) != 0; - *a |= mask; - raw_local_irq_restore(flags); - return res; -} -EXPORT_SYMBOL(__mips_test_and_set_bit); - - /** * __mips_test_and_set_bit_lock - Set a bit and return its old value. This is * called by test_and_set_bit_lock() if it cannot find a faster solution. From patchwork Tue Oct 1 21:53:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Burton X-Patchwork-Id: 11169807 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9B78D1599 for ; Tue, 1 Oct 2019 21:54:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 6F89021924 for ; Tue, 1 Oct 2019 21:54:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=wavecomp.com header.i=@wavecomp.com header.b="P36tm/mb" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728461AbfJAVy1 (ORCPT ); Tue, 1 Oct 2019 17:54:27 -0400 Received: from mail-eopbgr760115.outbound.protection.outlook.com ([40.107.76.115]:25806 "EHLO NAM02-CY1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728204AbfJAVyU (ORCPT ); Tue, 1 Oct 2019 17:54:20 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LpmWaynSO8nNhi+S9tkXzGOiuvFNTDe1vOrXXVKwQzdEkcV4otEM7ydyiS+CkPMDdK+uggf+YoZUaUJCR/KBEINixQK1ojNaeM7AsPV5Eb03pdp1WD/h6jdlqQ4Ws00+SbZGojYl8aHoOabcJNnWs5eUOPEzDkBJxKWEKnrKA34yxgK9q7i4rmD0Ia1iWpwCwkc4RD/9yC3xl+BqV4Rsf8RfjIMthqY1Q/PgtP+A0P9yvEwfWqZaUiyOufGxIRo7KJozDhs6XTE9XD+3vA2Mgotr50f5jF4aiV9FzCMQzb3jEwq21WZH1F+RZNkZsT56JLTdiXmp21jlRr/dS1pBrA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=LNYKj1hxnjMXXzl1DYbP6CGgH1/iGl4p4wsUEP3r9cY=; b=bS7dvq9ff0Tor8eLZ0l38iLBAKJOx8lKkfJjCnTz2f0YriNwrqfbnTyjgE3vMNh04vNxYju8VfvQDtqJeE8wOwBa87/K1SQOZXvaqGqyiiYprir39aRHujCWtBc43tAuZWWnl1YYWBI1ldXBX4lZ69lYyPBLdi4qM5ZQMhIsPmBxrOr4DcnxangHdjZuW5I8BCeOhVfY2M0p0jUE4ckef5fR/5E8PPE56GK4yi6xArzkxA4V/hhHQEHq0LqNL96ATEhRsQH72UBveCAbB7sIRABcQrBxCoeGN4OPOX4pLl8oTYNVdczSVhaM2iUKO0sEEJOss23LxrNh6/6z1Fgsfw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wavecomp.com; dmarc=pass action=none header.from=mips.com; dkim=pass header.d=mips.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wavecomp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=LNYKj1hxnjMXXzl1DYbP6CGgH1/iGl4p4wsUEP3r9cY=; b=P36tm/mbiPqOGn99iu0S+U6ZZnSvaJC4KW7lvkEA6amQr2b2x6ifKwel2hOTgyQGJAGTaXiBIp2ZfSALtDQQXKx+WFlkrfbXHr6hxaRRXcsl7vUvYhwXYDY8A/o8LQsAykOjclSSy9ehAlN73NkhhrWY7kmrvECGiDrPdZ87YfM= Received: from MWHPR2201MB1277.namprd22.prod.outlook.com (10.172.60.12) by MWHPR2201MB1439.namprd22.prod.outlook.com (10.174.169.166) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2305.20; Tue, 1 Oct 2019 21:53:31 +0000 Received: from MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481]) by MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481%2]) with mapi id 15.20.2305.022; Tue, 1 Oct 2019 21:53:31 +0000 From: Paul Burton To: "linux-mips@vger.kernel.org" CC: Huacai Chen , Jiaxun Yang , "linux-kernel@vger.kernel.org" , Paul Burton Subject: [PATCH v2 21/36] MIPS: bitops: Allow immediates in test_and_{set,clear,change}_bit Thread-Topic: [PATCH v2 21/36] MIPS: bitops: Allow immediates in test_and_{set,clear,change}_bit Thread-Index: AQHVeKKqDW9GPW66cUyyL3ilTsVu8g== Date: Tue, 1 Oct 2019 21:53:31 +0000 Message-ID: <20191001215249.4157062-22-paul.burton@mips.com> References: <20191001215249.4157062-1-paul.burton@mips.com> In-Reply-To: <20191001215249.4157062-1-paul.burton@mips.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BYAPR07CA0093.namprd07.prod.outlook.com (2603:10b6:a03:12b::34) To MWHPR2201MB1277.namprd22.prod.outlook.com (2603:10b6:301:18::12) authentication-results: spf=none (sender IP is ) smtp.mailfrom=pburton@wavecomp.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.23.0 x-originating-ip: [12.94.197.246] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: e7d8f364-760b-4534-4c8f-08d746b9cc6e x-ms-traffictypediagnostic: MWHPR2201MB1439: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:3173; x-forefront-prvs: 0177904E6B x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(979002)(396003)(39840400004)(366004)(346002)(136003)(376002)(199004)(189003)(186003)(11346002)(6512007)(1076003)(486006)(66066001)(26005)(6486002)(316002)(2906002)(5640700003)(6436002)(44832011)(42882007)(25786009)(446003)(6916009)(386003)(6506007)(76176011)(478600001)(52116002)(2616005)(102836004)(476003)(5660300002)(305945005)(64756008)(66446008)(14444005)(2351001)(7736002)(99286004)(54906003)(14454004)(71190400001)(71200400001)(66946007)(66556008)(66476007)(36756003)(3846002)(6116002)(107886003)(4326008)(50226002)(81166006)(81156014)(8676002)(2501003)(8936002)(256004)(14773001)(142933001)(969003)(989001)(999001)(1009001)(1019001);DIR:OUT;SFP:1102;SCL:1;SRVR:MWHPR2201MB1439;H:MWHPR2201MB1277.namprd22.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; received-spf: None (protection.outlook.com: wavecomp.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: mVFS3Nozpc+lygteNzkhBnnOD8rG2y6id7lAu+AEjLdPGl3nXReGMrZkL4t+WgJZ6aXWtXTwXE+fbXf11qi72opElY+G9Hsg0Nf4TuqlUZOEmnHlMFCk4lEztwqktvFh8OFHY55V2jm5vxQAnyga+1YDlkhopJuqQ/ee6ay7C0Mi3ak7hfqtm1XfRA+6We1aw+a1CP95FOQ78AEsP9Q2wgHQ6OoZzY1lOjxqbVekZaDD7MLHF0niDIfodd/eiwehBpQLaBVsFdlfKPXQQGQlKKsRguNf8COhF+w1zGCZJ693d3d0V/zWX1lWtr3NPbJchkc0Zazu/QEuE2uDzqnFol53Be7jk8ph/J0OObvCisvNgisU7d+ZI8xALalnlggGmFVhXIvzUD0tYJNvRSB6eqsxo902U1ZALwdsjUf1dyg= MIME-Version: 1.0 X-OriginatorOrg: mips.com X-MS-Exchange-CrossTenant-Network-Message-Id: e7d8f364-760b-4534-4c8f-08d746b9cc6e X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Oct 2019 21:53:31.0568 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 463607d3-1db3-40a0-8a29-970c56230104 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: VP3IRFtiBfl5s4Z8Oc4OpVOE9u3otJHiImyX+JX9zx/wwmon3IenpmqsNnCPa1GfoizqIvI9iE+ojOufpL60yg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR2201MB1439 Sender: linux-mips-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-mips@vger.kernel.org The logical operations or & xor used in the test_and_set_bit_lock(), test_and_clear_bit() & test_and_change_bit() functions currently force the value 1< --- Changes in v2: None arch/mips/include/asm/bitops.h | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/arch/mips/include/asm/bitops.h b/arch/mips/include/asm/bitops.h index ea35a2e87b6d..7314ba5a3683 100644 --- a/arch/mips/include/asm/bitops.h +++ b/arch/mips/include/asm/bitops.h @@ -261,7 +261,7 @@ static inline int test_and_set_bit_lock(unsigned long nr, " and %2, %0, %3 \n" " .set pop \n" : "=&r" (temp), "+m" (*m), "=&r" (res) - : "r" (1UL << bit) + : "ir" (1UL << bit) : __LLSC_CLOBBER); } else { loongson_llsc_mb(); @@ -274,7 +274,7 @@ static inline int test_and_set_bit_lock(unsigned long nr, " " __SC "%2, %1 \n" " .set pop \n" : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (*m), "=&r" (res) - : "r" (1UL << bit) + : "ir" (1UL << bit) : __LLSC_CLOBBER); } while (unlikely(!res)); @@ -332,7 +332,7 @@ static inline int test_and_clear_bit(unsigned long nr, " and %2, %0, %3 \n" " .set pop \n" : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (*m), "=&r" (res) - : "r" (1UL << bit) + : "ir" (1UL << bit) : __LLSC_CLOBBER); } else if ((MIPS_ISA_REV >= 2) && __builtin_constant_p(nr)) { loongson_llsc_mb(); @@ -358,7 +358,7 @@ static inline int test_and_clear_bit(unsigned long nr, " " __SC "%2, %1 \n" " .set pop \n" : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (*m), "=&r" (res) - : "r" (1UL << bit) + : "ir" (1UL << bit) : __LLSC_CLOBBER); } while (unlikely(!res)); @@ -400,7 +400,7 @@ static inline int test_and_change_bit(unsigned long nr, " and %2, %0, %3 \n" " .set pop \n" : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (*m), "=&r" (res) - : "r" (1UL << bit) + : "ir" (1UL << bit) : __LLSC_CLOBBER); } else { loongson_llsc_mb(); @@ -413,7 +413,7 @@ static inline int test_and_change_bit(unsigned long nr, " " __SC "\t%2, %1 \n" " .set pop \n" : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (*m), "=&r" (res) - : "r" (1UL << bit) + : "ir" (1UL << bit) : __LLSC_CLOBBER); } while (unlikely(!res)); From patchwork Tue Oct 1 21:53:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Burton X-Patchwork-Id: 11169801 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0633A13B1 for ; Tue, 1 Oct 2019 21:54:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id CDFFF2133F for ; Tue, 1 Oct 2019 21:54:11 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=wavecomp.com header.i=@wavecomp.com header.b="Zir1FnQD" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728231AbfJAVxp (ORCPT ); Tue, 1 Oct 2019 17:53:45 -0400 Received: from mail-eopbgr730113.outbound.protection.outlook.com ([40.107.73.113]:17171 "EHLO NAM05-DM3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728200AbfJAVxk (ORCPT ); Tue, 1 Oct 2019 17:53:40 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jhlmrJtnFZS1uRLa/L/tibdnZROlxPEQ5wQG+jI781nhIys3WUnWms74P5ZeMreAOSZQis3QPRGzYyBwepDw665wcIhLXv54AZPwQyCzDxzPMqFp7fvBfwbYq3RO1nkeZ2T5GTuhVu6iIx3FqLQ8pPBp1DgBkKsn3utXn1rBIWbeAIm2j9nJGboqgN4XCs5/q0RPf5Nrog/FknNxT1J8FKnXe5WEZBtrJiRe5cE7U7KlQj2dekDKRoNMSGaZuHpNia4yOcz0SH+71tU5yrpMOQ1yT2N8bAcCYNPZ+btBBU8r0ovTy6a9l1UbB+oKnc8ZgQUIDIHU4Rki8mIoXuhdIQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ci/15PmHn5aOkSH4PGDpm2VAoB19B/XPaRo/e5fUaXA=; b=JX6LY+WSa54PYRK7E1y/CDQazB6OEbmXi4OYqkfWDyziOmwVjPMDnV+dt3Ue+4Hg/65BqMrG1QcHdOgvoDGpeRQgkMoQW3IGjklOZdh23kFx5aM7/SqyGljPv7yw067343oIKY/ivQ5BSedgMaj7p1CrHIHPcwXkKZfWjczenHiLY70Agns0CJhpKF7NPPcmaVHsCJ6K0HeczyTbiC3ACFKJyTrE5t9eFlrrpBJshZ6UXXjVrF2/eoTk9jo5S9glyKB7+uyxz6TZglx79ZYmg9H7ro6sVs/ANIv0AnIg9UbnZupowqKTpGMBT97mida+6BWU5drLzvIKzK+kMi9W3A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wavecomp.com; dmarc=pass action=none header.from=mips.com; dkim=pass header.d=mips.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wavecomp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ci/15PmHn5aOkSH4PGDpm2VAoB19B/XPaRo/e5fUaXA=; b=Zir1FnQDp4HaGyU5Al1Csalw5RaMiQgqtO8dc1PavhdBrNZC9tWwuDfjE9EZV/4THOLtOSFrgtYYpyzQukpz50JkKm5EB4aY66clWNltTkcmw6oxug1JOR8UV6hQPv+1kEOXqCCJeZO1FHFU7fkSmSoDKM50GAb5vJP404nPuaI= Received: from MWHPR2201MB1277.namprd22.prod.outlook.com (10.172.60.12) by MWHPR2201MB1439.namprd22.prod.outlook.com (10.174.169.166) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2305.20; Tue, 1 Oct 2019 21:53:32 +0000 Received: from MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481]) by MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481%2]) with mapi id 15.20.2305.022; Tue, 1 Oct 2019 21:53:32 +0000 From: Paul Burton To: "linux-mips@vger.kernel.org" CC: Huacai Chen , Jiaxun Yang , "linux-kernel@vger.kernel.org" , Paul Burton Subject: [PATCH v2 22/36] MIPS: bitops: Use the BIT() macro Thread-Topic: [PATCH v2 22/36] MIPS: bitops: Use the BIT() macro Thread-Index: AQHVeKKqA6Bv8sCnekWZwAB+VsQHGA== Date: Tue, 1 Oct 2019 21:53:31 +0000 Message-ID: <20191001215249.4157062-23-paul.burton@mips.com> References: <20191001215249.4157062-1-paul.burton@mips.com> In-Reply-To: <20191001215249.4157062-1-paul.burton@mips.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BYAPR07CA0093.namprd07.prod.outlook.com (2603:10b6:a03:12b::34) To MWHPR2201MB1277.namprd22.prod.outlook.com (2603:10b6:301:18::12) authentication-results: spf=none (sender IP is ) smtp.mailfrom=pburton@wavecomp.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.23.0 x-originating-ip: [12.94.197.246] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 1450bc32-0afc-4610-589c-08d746b9cced x-ms-traffictypediagnostic: MWHPR2201MB1439: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:1201; x-forefront-prvs: 0177904E6B x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(979002)(396003)(39840400004)(366004)(346002)(136003)(376002)(199004)(189003)(186003)(11346002)(6512007)(1076003)(486006)(66066001)(26005)(6486002)(316002)(2906002)(5640700003)(6436002)(44832011)(42882007)(25786009)(446003)(6916009)(386003)(6506007)(76176011)(478600001)(52116002)(2616005)(102836004)(476003)(5660300002)(305945005)(64756008)(66446008)(14444005)(2351001)(7736002)(99286004)(54906003)(14454004)(71190400001)(71200400001)(66946007)(66556008)(66476007)(36756003)(3846002)(6116002)(107886003)(4326008)(50226002)(81166006)(81156014)(8676002)(2501003)(8936002)(256004)(969003)(989001)(999001)(1009001)(1019001);DIR:OUT;SFP:1102;SCL:1;SRVR:MWHPR2201MB1439;H:MWHPR2201MB1277.namprd22.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; received-spf: None (protection.outlook.com: wavecomp.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: SZLAzBOYsmF7EkeXRPnIpcbQ0VmxULLbUV99kglxFa7lxAJVsa1MHnGQYQqduLyyqtW5D7JTPdaOYeX8EYMp3jOnlB7I960jAUpOZdQI2F/kYUSIyTqbR5szrNRbGcV21JYw6/jw+tlGni06R+9NHPG7uCveyL2XEtWXHVbU9RLy20s7iMGvd0FinQaOwHGDV2PmEWI+Mn8s8m0iB2klUQmH3UIfk+RlIe3uyFTfhZLRumkhENqv0zN+rOhgrq63QnYKb0jfjzYrEP5Iz3L7ovwB8yRrRDKhrrGMC2DNTk5aQEjdz9ZMCnOADrXnuf9kAlYrTCcIZ6syvFXdamdxwECQB3jtoSvclpBbJcyV/uH1+pYVN8d3aONHQzRC6P4J4YpkR5e3+xXni17Bndd+EsiPbiN0m+uTMRwVyvAsC1E= MIME-Version: 1.0 X-OriginatorOrg: mips.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1450bc32-0afc-4610-589c-08d746b9cced X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Oct 2019 21:53:31.9024 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 463607d3-1db3-40a0-8a29-970c56230104 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: nHVVRwVUr1tCqku9VBwEj933BjwgzbXejnjPpgZ5XcyKM+P5ow0yuG1e9CHrSCX43OYKLF7XQ+Uzm37np4jU3A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR2201MB1439 Sender: linux-mips-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-mips@vger.kernel.org Use the BIT() macro in asm/bitops.h rather than open-coding its equivalent. Signed-off-by: Paul Burton --- Changes in v2: None arch/mips/include/asm/bitops.h | 31 ++++++++++++++++--------------- 1 file changed, 16 insertions(+), 15 deletions(-) diff --git a/arch/mips/include/asm/bitops.h b/arch/mips/include/asm/bitops.h index 7314ba5a3683..0f8ff896e86b 100644 --- a/arch/mips/include/asm/bitops.h +++ b/arch/mips/include/asm/bitops.h @@ -13,6 +13,7 @@ #error only can be included directly #endif +#include #include #include #include @@ -70,7 +71,7 @@ static inline void set_bit(unsigned long nr, volatile unsigned long *addr) " beqzl %0, 1b \n" " .set pop \n" : "=&r" (temp), "=" GCC_OFF_SMALL_ASM() (*m) - : "ir" (1UL << bit), GCC_OFF_SMALL_ASM() (*m) + : "ir" (BIT(bit)), GCC_OFF_SMALL_ASM() (*m) : __LLSC_CLOBBER); return; } @@ -99,7 +100,7 @@ static inline void set_bit(unsigned long nr, volatile unsigned long *addr) " " __SC "%0, %1 \n" " .set pop \n" : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (*m) - : "ir" (1UL << bit) + : "ir" (BIT(bit)) : __LLSC_CLOBBER); } while (unlikely(!temp)); } @@ -135,7 +136,7 @@ static inline void clear_bit(unsigned long nr, volatile unsigned long *addr) " beqzl %0, 1b \n" " .set pop \n" : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (*m) - : "ir" (~(1UL << bit)) + : "ir" (~(BIT(bit))) : __LLSC_CLOBBER); return; } @@ -164,7 +165,7 @@ static inline void clear_bit(unsigned long nr, volatile unsigned long *addr) " " __SC "%0, %1 \n" " .set pop \n" : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (*m) - : "ir" (~(1UL << bit)) + : "ir" (~(BIT(bit))) : __LLSC_CLOBBER); } while (unlikely(!temp)); } @@ -213,7 +214,7 @@ static inline void change_bit(unsigned long nr, volatile unsigned long *addr) " beqzl %0, 1b \n" " .set pop \n" : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (*m) - : "ir" (1UL << bit) + : "ir" (BIT(bit)) : __LLSC_CLOBBER); return; } @@ -228,7 +229,7 @@ static inline void change_bit(unsigned long nr, volatile unsigned long *addr) " " __SC "%0, %1 \n" " .set pop \n" : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (*m) - : "ir" (1UL << bit) + : "ir" (BIT(bit)) : __LLSC_CLOBBER); } while (unlikely(!temp)); } @@ -261,7 +262,7 @@ static inline int test_and_set_bit_lock(unsigned long nr, " and %2, %0, %3 \n" " .set pop \n" : "=&r" (temp), "+m" (*m), "=&r" (res) - : "ir" (1UL << bit) + : "ir" (BIT(bit)) : __LLSC_CLOBBER); } else { loongson_llsc_mb(); @@ -274,11 +275,11 @@ static inline int test_and_set_bit_lock(unsigned long nr, " " __SC "%2, %1 \n" " .set pop \n" : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (*m), "=&r" (res) - : "ir" (1UL << bit) + : "ir" (BIT(bit)) : __LLSC_CLOBBER); } while (unlikely(!res)); - res = temp & (1UL << bit); + res = temp & BIT(bit); } smp_llsc_mb(); @@ -332,7 +333,7 @@ static inline int test_and_clear_bit(unsigned long nr, " and %2, %0, %3 \n" " .set pop \n" : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (*m), "=&r" (res) - : "ir" (1UL << bit) + : "ir" (BIT(bit)) : __LLSC_CLOBBER); } else if ((MIPS_ISA_REV >= 2) && __builtin_constant_p(nr)) { loongson_llsc_mb(); @@ -358,11 +359,11 @@ static inline int test_and_clear_bit(unsigned long nr, " " __SC "%2, %1 \n" " .set pop \n" : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (*m), "=&r" (res) - : "ir" (1UL << bit) + : "ir" (BIT(bit)) : __LLSC_CLOBBER); } while (unlikely(!res)); - res = temp & (1UL << bit); + res = temp & BIT(bit); } smp_llsc_mb(); @@ -400,7 +401,7 @@ static inline int test_and_change_bit(unsigned long nr, " and %2, %0, %3 \n" " .set pop \n" : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (*m), "=&r" (res) - : "ir" (1UL << bit) + : "ir" (BIT(bit)) : __LLSC_CLOBBER); } else { loongson_llsc_mb(); @@ -413,11 +414,11 @@ static inline int test_and_change_bit(unsigned long nr, " " __SC "\t%2, %1 \n" " .set pop \n" : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (*m), "=&r" (res) - : "ir" (1UL << bit) + : "ir" (BIT(bit)) : __LLSC_CLOBBER); } while (unlikely(!res)); - res = temp & (1UL << bit); + res = temp & BIT(bit); } smp_llsc_mb(); From patchwork Tue Oct 1 21:53:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Burton X-Patchwork-Id: 11169821 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9AF331709 for ; Tue, 1 Oct 2019 21:54:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 797F421A4A for ; Tue, 1 Oct 2019 21:54:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=wavecomp.com header.i=@wavecomp.com header.b="quuuvoVT" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728354AbfJAVyM (ORCPT ); Tue, 1 Oct 2019 17:54:12 -0400 Received: from mail-eopbgr730113.outbound.protection.outlook.com ([40.107.73.113]:17171 "EHLO NAM05-DM3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728177AbfJAVxo (ORCPT ); Tue, 1 Oct 2019 17:53:44 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=P4u2ANLHWaywxHKlGlUbFpdEqZR0SWNwXuA4z0dlpAbcwIBLOj13963QxRRLKxwCBWbpbD45pLLDnVSCf0ZpEi0o/7MXT8gbOZDh8dioEUAXLTUVpnVpnc0TeQTbplZQDe6ekJseMYLEIYvK9EzsIpVa3g6n8oz5eRv6kvfN8iv7fcfZl+5cVvvH1GG2lddKp2FQy19nmDADrLF3cm4u12uquuAlp1rvfSivLLmvaY8sbqcNcp6jAN684NaiKyEm5pogZ+okotCDRYrQ08pgYmAEPM1HreNI6lu95DSjpR1iFBCKjJOQdLC79uuLuygicRBKOTbxuc6G/pP5g1exAg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=08aXOuUCv1QBQ78zSHt4gRJFznTUvA/Oy5HXdvyZIf8=; b=CYgvxIpOnllsx1RvjIn1h1xNFX8P775vTM/j+Kb7GiMnKYbZi0PK6FHU4rVi3L30c3+E3gNPtt9E5XOWbd1vcePPl9fL7VzRiBIEiW00LeQGjlCJ7FEDSWWJuEIOyrJ5GV/W+mbNJHdo3MX/JEOb4kjFCofeyYWj+4FbuDYmjuevB9dRch6iqr23MErzCXXjtizuq1jvkcWBcYb6PMJlWks7PqvJby1PFUv1cGrvf42Ia9rTRoJAboSkda0IdMz2J0OAc+aLgn8QEOCvZAkuwaosjk2ulD44ky7p42eYWwjDr19nstDbCwXz6uRxtfEzNpOswILFBujlhpUGTtgTaQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wavecomp.com; dmarc=pass action=none header.from=mips.com; dkim=pass header.d=mips.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wavecomp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=08aXOuUCv1QBQ78zSHt4gRJFznTUvA/Oy5HXdvyZIf8=; b=quuuvoVTt5nSQX5Q3BCm/owf9hVlbiVjGHwncWB3nSk+PfXTkWRzXBbYqKUATYZoELukE3yNwfT3B/v+tq7pLyuX7nxHDeEUgDzeWccsI/G//lt1sllJb0wNE0IDpbgQZ8JOZ6mLbHfRdwDGknsCg2Epmi+i55qfkU1M8JxDiyM= Received: from MWHPR2201MB1277.namprd22.prod.outlook.com (10.172.60.12) by MWHPR2201MB1439.namprd22.prod.outlook.com (10.174.169.166) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2305.20; Tue, 1 Oct 2019 21:53:32 +0000 Received: from MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481]) by MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481%2]) with mapi id 15.20.2305.022; Tue, 1 Oct 2019 21:53:32 +0000 From: Paul Burton To: "linux-mips@vger.kernel.org" CC: Huacai Chen , Jiaxun Yang , "linux-kernel@vger.kernel.org" , Paul Burton Subject: [PATCH v2 23/36] MIPS: bitops: Avoid redundant zero-comparison for non-LLSC Thread-Topic: [PATCH v2 23/36] MIPS: bitops: Avoid redundant zero-comparison for non-LLSC Thread-Index: AQHVeKKr/ZciCZFEkku5+pQr0Jn66g== Date: Tue, 1 Oct 2019 21:53:32 +0000 Message-ID: <20191001215249.4157062-24-paul.burton@mips.com> References: <20191001215249.4157062-1-paul.burton@mips.com> In-Reply-To: <20191001215249.4157062-1-paul.burton@mips.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BYAPR07CA0093.namprd07.prod.outlook.com (2603:10b6:a03:12b::34) To MWHPR2201MB1277.namprd22.prod.outlook.com (2603:10b6:301:18::12) authentication-results: spf=none (sender IP is ) smtp.mailfrom=pburton@wavecomp.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.23.0 x-originating-ip: [12.94.197.246] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: c89cc258-248f-4811-ae57-08d746b9cd6e x-ms-traffictypediagnostic: MWHPR2201MB1439: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:883; x-forefront-prvs: 0177904E6B x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(396003)(39840400004)(366004)(346002)(136003)(376002)(199004)(189003)(186003)(11346002)(6512007)(1076003)(486006)(66066001)(26005)(6486002)(316002)(2906002)(5640700003)(6436002)(44832011)(42882007)(25786009)(446003)(6916009)(386003)(6506007)(76176011)(478600001)(52116002)(2616005)(102836004)(476003)(5660300002)(305945005)(64756008)(66446008)(14444005)(2351001)(7736002)(99286004)(54906003)(14454004)(71190400001)(71200400001)(66946007)(66556008)(66476007)(36756003)(3846002)(6116002)(107886003)(4326008)(50226002)(81166006)(81156014)(8676002)(2501003)(8936002)(256004);DIR:OUT;SFP:1102;SCL:1;SRVR:MWHPR2201MB1439;H:MWHPR2201MB1277.namprd22.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; received-spf: None (protection.outlook.com: wavecomp.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: WxbgGVYN1Nvhe47PK0tlnvGGMDFotd0jF396+tnuQOdSj45qGfyfoqVD3SkQDkYJDUOB0Le6mN5Wb8FKh441m70nFxa9S0nt+wrMI+1Nz9qsZfk7XF66JojG3RmHaLUWUtijFeeGH5bBemmu1ryU0rW0Hb8GO8HLPJdRmS7wrJUMnAR8USRpCvIDzrfsA1GUTHBhPh19Ltjdsc/r0PfNZCBSdLTQZYPdyutEqiIenYADc4f/b4aybZjFDiVEyieHWgUXnETO9mFc7HujsGy20QOOmh/LXTXoMWIuKK9WS7BWPSuS5tTZSRJoaWfpP4MRzcLHn4QapYAaGaCRh0q2rFxLPd2UaVu2a7OLTQOTf7HttItdlQG4QAtb1ZIlEAI0Ik1xPupstSWCnRXlqlxFx1VjdxHL/TLK4Qc7VFQPzM4= MIME-Version: 1.0 X-OriginatorOrg: mips.com X-MS-Exchange-CrossTenant-Network-Message-Id: c89cc258-248f-4811-ae57-08d746b9cd6e X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Oct 2019 21:53:32.7210 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 463607d3-1db3-40a0-8a29-970c56230104 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: rbU+xxcf3H8sHO4b/F3ewENS/Zz0rwk5ySqKGxX/HcooT8qbnzDBXOonUEyJf7qWRIMFc0gO6ZN6X2YNbmlQlQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR2201MB1439 Sender: linux-mips-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-mips@vger.kernel.org The IRQ-disabling non-LLSC fallbacks for bitops on UP systems already return a zero or one, so there's no need to perform another comparison against zero. Move these comparisons into the LLSC paths to avoid the redundant work. Signed-off-by: Paul Burton --- Changes in v2: None arch/mips/include/asm/bitops.h | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/arch/mips/include/asm/bitops.h b/arch/mips/include/asm/bitops.h index 0f8ff896e86b..7671db2a7b73 100644 --- a/arch/mips/include/asm/bitops.h +++ b/arch/mips/include/asm/bitops.h @@ -264,6 +264,8 @@ static inline int test_and_set_bit_lock(unsigned long nr, : "=&r" (temp), "+m" (*m), "=&r" (res) : "ir" (BIT(bit)) : __LLSC_CLOBBER); + + res = res != 0; } else { loongson_llsc_mb(); do { @@ -279,12 +281,12 @@ static inline int test_and_set_bit_lock(unsigned long nr, : __LLSC_CLOBBER); } while (unlikely(!res)); - res = temp & BIT(bit); + res = (temp & BIT(bit)) != 0; } smp_llsc_mb(); - return res != 0; + return res; } /* @@ -335,6 +337,8 @@ static inline int test_and_clear_bit(unsigned long nr, : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (*m), "=&r" (res) : "ir" (BIT(bit)) : __LLSC_CLOBBER); + + res = res != 0; } else if ((MIPS_ISA_REV >= 2) && __builtin_constant_p(nr)) { loongson_llsc_mb(); do { @@ -363,12 +367,12 @@ static inline int test_and_clear_bit(unsigned long nr, : __LLSC_CLOBBER); } while (unlikely(!res)); - res = temp & BIT(bit); + res = (temp & BIT(bit)) != 0; } smp_llsc_mb(); - return res != 0; + return res; } /* @@ -403,6 +407,8 @@ static inline int test_and_change_bit(unsigned long nr, : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (*m), "=&r" (res) : "ir" (BIT(bit)) : __LLSC_CLOBBER); + + res = res != 0; } else { loongson_llsc_mb(); do { @@ -418,12 +424,12 @@ static inline int test_and_change_bit(unsigned long nr, : __LLSC_CLOBBER); } while (unlikely(!res)); - res = temp & BIT(bit); + res = (temp & BIT(bit)) != 0; } smp_llsc_mb(); - return res != 0; + return res; } #include From patchwork Tue Oct 1 21:53:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Burton X-Patchwork-Id: 11169799 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AAD6E13B1 for ; Tue, 1 Oct 2019 21:54:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 752F921924 for ; Tue, 1 Oct 2019 21:54:10 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=wavecomp.com header.i=@wavecomp.com header.b="gdycOskh" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728337AbfJAVyG (ORCPT ); Tue, 1 Oct 2019 17:54:06 -0400 Received: from mail-eopbgr730113.outbound.protection.outlook.com ([40.107.73.113]:17171 "EHLO NAM05-DM3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728237AbfJAVxq (ORCPT ); Tue, 1 Oct 2019 17:53:46 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=AcPxqESVHSoHM0i3T5N8q4MTgwL6ytd6tXz8MsEnE1fNN/gBjK00YRyY2klqOs5gf+ZzA/I7V4VTXCPoxpRXMyO8Zp9cQNiVAKgedNeSDqOtynRP3yT5pt5je0Q/2+2sh+liT+mUkOUiLn44B6RpcLnJS82trEBCWg6HEcjDO676fs08TOOl/oI4KwubuIThjWwrMergdOD9VtB+XnADy1O+hH8iOc3e68nfHsJq75vS9oKYEOC9ed0t/1i/hSXDjUahNitKcZ2s0nbYr6n9FOKd+EFI1fOTqVoQCO0koW3VRIjQVSiUUn+h7owxOIb9aQHsD4y/SeqtSoeSdDqY7Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=npOaMSJyJF5W9F6cpP/sgOA+pcyXNaIKTv58DqWOnDU=; b=NpLozVbQ5BaJo4VrPDmd4ewDPSzubFORcu1zzUrx0Sdg0I9ysPWww7kvoyN8pIh9Kdoor3hiXTAtzD+S0VGsjUBCzEjXHGNEEYLtNHcy/+GjMESOFbJ1xuuunV0RgB9R0dLbi/CIHQl2I+Z2HPGhqhF8XsWmHd/lp43A7cSbVimCGVzGUVnIRm6hHPGWMg+uIsSPhDwQ/6HkDElO4ZVuc0+ygWKAqkJbfoldZuzZlgoVDxLZryh20PS5tD9GG9MNscdjFFCdJ7wW8cpgtRMyUPfoT4Iocd3PHyejAnxcMLAtDEniJ/Ynlqrj5hVrqNGhlvUGK6UhhP5iszomnuq7Ow== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wavecomp.com; dmarc=pass action=none header.from=mips.com; dkim=pass header.d=mips.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wavecomp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=npOaMSJyJF5W9F6cpP/sgOA+pcyXNaIKTv58DqWOnDU=; b=gdycOskhuE/BcM5qmFx0GY0d3uM7LEWqaoe0Z7cfD1E1rmNhmwxiTCxc6hHWx2QrkX3HxQd9XDnnvHPwo4DhqSbYZjQojSqBSB8vgjanQA1rsTLwn11Lqo0H66PWPv28utmlwpYr9ueDsilqxsJX62hd9I3EPszKqxMWIs0alcU= Received: from MWHPR2201MB1277.namprd22.prod.outlook.com (10.172.60.12) by MWHPR2201MB1439.namprd22.prod.outlook.com (10.174.169.166) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2305.20; Tue, 1 Oct 2019 21:53:33 +0000 Received: from MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481]) by MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481%2]) with mapi id 15.20.2305.022; Tue, 1 Oct 2019 21:53:33 +0000 From: Paul Burton To: "linux-mips@vger.kernel.org" CC: Huacai Chen , Jiaxun Yang , "linux-kernel@vger.kernel.org" , Paul Burton Subject: [PATCH v2 24/36] MIPS: bitops: Abstract LL/SC loops Thread-Topic: [PATCH v2 24/36] MIPS: bitops: Abstract LL/SC loops Thread-Index: AQHVeKKrVOaQd8iJw06uvKmFE7Np4A== Date: Tue, 1 Oct 2019 21:53:33 +0000 Message-ID: <20191001215249.4157062-25-paul.burton@mips.com> References: <20191001215249.4157062-1-paul.burton@mips.com> In-Reply-To: <20191001215249.4157062-1-paul.burton@mips.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BYAPR07CA0093.namprd07.prod.outlook.com (2603:10b6:a03:12b::34) To MWHPR2201MB1277.namprd22.prod.outlook.com (2603:10b6:301:18::12) authentication-results: spf=none (sender IP is ) smtp.mailfrom=pburton@wavecomp.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.23.0 x-originating-ip: [12.94.197.246] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 4db24a35-7ef5-469e-bed5-08d746b9cdeb x-ms-traffictypediagnostic: MWHPR2201MB1439: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:1388; x-forefront-prvs: 0177904E6B x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(396003)(39840400004)(366004)(346002)(136003)(376002)(199004)(189003)(186003)(11346002)(6512007)(1076003)(486006)(66066001)(26005)(6486002)(316002)(2906002)(5640700003)(6436002)(44832011)(42882007)(25786009)(446003)(6916009)(386003)(6506007)(76176011)(478600001)(52116002)(2616005)(102836004)(476003)(5660300002)(305945005)(64756008)(66446008)(14444005)(2351001)(7736002)(99286004)(54906003)(14454004)(71190400001)(71200400001)(66946007)(66556008)(66476007)(36756003)(3846002)(6116002)(107886003)(4326008)(50226002)(81166006)(81156014)(8676002)(2501003)(8936002)(256004);DIR:OUT;SFP:1102;SCL:1;SRVR:MWHPR2201MB1439;H:MWHPR2201MB1277.namprd22.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; received-spf: None (protection.outlook.com: wavecomp.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 9vbB7nk0euZmgbvf1SktAW1mJ8GTobIhTuZefLO8iwsu5C1D/IdmTjOXeUv5uTwa5EmI8aTXroquSHpqY1vrUDxQKSIylu1ZZYs4Gd+Up8+U31jEaEgvI4TNEnzvfNVVFleap7Mop+m7Le6IzCB5DyyqIdCQI4tphQ8mYvhaQq1M74CB7d52BE2c0uKuU8pkZSFjr0ow5k3JxH0ygH2iqjb6PVWIwciPPWtGrnx8tVYvEb/6tEvU0bUDq2FzSNh41cO8puzKlmohbgPiuxwl2qFmKJEg42HrLahj/4SK1BGDvCaZldkY/OL46j9ddKWhezmdFK6QvKP2FNkrJmZdAKecpN226VKGyjDFvivb5baYz8gydXLj1HYClAO1CJFhskqSkK859Tdtc+a61UZ7OKHhhz8+Y3f6sOIikng8za8= MIME-Version: 1.0 X-OriginatorOrg: mips.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4db24a35-7ef5-469e-bed5-08d746b9cdeb X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Oct 2019 21:53:33.6646 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 463607d3-1db3-40a0-8a29-970c56230104 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: YHleMTU+6SGvttFkP2opPvhKUairgdGCJVRJ1CKHv5gHxNeOribS5fGuDDx6JoBgxhS5poXXzH6W/jRfTs8Z7A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR2201MB1439 Sender: linux-mips-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-mips@vger.kernel.org Introduce __bit_op() & __test_bit_op() macros which abstract away the implementation of LL/SC loops. This cuts down on a lot of duplicate boilerplate code, and also allows R10000_LLSC_WAR to be handled outside of the individual bitop functions. Signed-off-by: Paul Burton --- Changes in v2: None arch/mips/include/asm/bitops.h | 267 ++++++++------------------------- 1 file changed, 63 insertions(+), 204 deletions(-) diff --git a/arch/mips/include/asm/bitops.h b/arch/mips/include/asm/bitops.h index 7671db2a7b73..fba0a842b98a 100644 --- a/arch/mips/include/asm/bitops.h +++ b/arch/mips/include/asm/bitops.h @@ -25,6 +25,41 @@ #include #include +#define __bit_op(mem, insn, inputs...) do { \ + unsigned long temp; \ + \ + asm volatile( \ + " .set push \n" \ + " .set " MIPS_ISA_LEVEL " \n" \ + "1: " __LL "%0, %1 \n" \ + " " insn " \n" \ + " " __SC "%0, %1 \n" \ + " " __SC_BEQZ "%0, 1b \n" \ + " .set pop \n" \ + : "=&r"(temp), "+" GCC_OFF_SMALL_ASM()(mem) \ + : inputs \ + : __LLSC_CLOBBER); \ +} while (0) + +#define __test_bit_op(mem, ll_dst, insn, inputs...) ({ \ + unsigned long orig, temp; \ + \ + asm volatile( \ + " .set push \n" \ + " .set " MIPS_ISA_LEVEL " \n" \ + "1: " __LL ll_dst ", %2 \n" \ + " " insn " \n" \ + " " __SC "%1, %2 \n" \ + " " __SC_BEQZ "%1, 1b \n" \ + " .set pop \n" \ + : "=&r"(orig), "=&r"(temp), \ + "+" GCC_OFF_SMALL_ASM()(mem) \ + : inputs \ + : __LLSC_CLOBBER); \ + \ + orig; \ +}) + /* * These are the "slower" versions of the functions and are in bitops.c. * These functions call raw_local_irq_{save,restore}(). @@ -54,55 +89,20 @@ static inline void set_bit(unsigned long nr, volatile unsigned long *addr) { unsigned long *m = ((unsigned long *)addr) + (nr >> SZLONG_LOG); int bit = nr & SZLONG_MASK; - unsigned long temp; if (!kernel_uses_llsc) { __mips_set_bit(nr, addr); return; } - if (R10000_LLSC_WAR) { - __asm__ __volatile__( - " .set push \n" - " .set arch=r4000 \n" - "1: " __LL "%0, %1 # set_bit \n" - " or %0, %2 \n" - " " __SC "%0, %1 \n" - " beqzl %0, 1b \n" - " .set pop \n" - : "=&r" (temp), "=" GCC_OFF_SMALL_ASM() (*m) - : "ir" (BIT(bit)), GCC_OFF_SMALL_ASM() (*m) - : __LLSC_CLOBBER); - return; - } - if ((MIPS_ISA_REV >= 2) && __builtin_constant_p(bit) && (bit >= 16)) { loongson_llsc_mb(); - do { - __asm__ __volatile__( - " " __LL "%0, %1 # set_bit \n" - " " __INS "%0, %3, %2, 1 \n" - " " __SC "%0, %1 \n" - : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (*m) - : "i" (bit), "r" (~0) - : __LLSC_CLOBBER); - } while (unlikely(!temp)); + __bit_op(*m, __INS "%0, %3, %2, 1", "i"(bit), "r"(~0)); return; } loongson_llsc_mb(); - do { - __asm__ __volatile__( - " .set push \n" - " .set "MIPS_ISA_ARCH_LEVEL" \n" - " " __LL "%0, %1 # set_bit \n" - " or %0, %2 \n" - " " __SC "%0, %1 \n" - " .set pop \n" - : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (*m) - : "ir" (BIT(bit)) - : __LLSC_CLOBBER); - } while (unlikely(!temp)); + __bit_op(*m, "or\t%0, %2", "ir"(BIT(bit))); } /* @@ -119,55 +119,20 @@ static inline void clear_bit(unsigned long nr, volatile unsigned long *addr) { unsigned long *m = ((unsigned long *)addr) + (nr >> SZLONG_LOG); int bit = nr & SZLONG_MASK; - unsigned long temp; if (!kernel_uses_llsc) { __mips_clear_bit(nr, addr); return; } - if (R10000_LLSC_WAR) { - __asm__ __volatile__( - " .set push \n" - " .set arch=r4000 \n" - "1: " __LL "%0, %1 # clear_bit \n" - " and %0, %2 \n" - " " __SC "%0, %1 \n" - " beqzl %0, 1b \n" - " .set pop \n" - : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (*m) - : "ir" (~(BIT(bit))) - : __LLSC_CLOBBER); - return; - } - if ((MIPS_ISA_REV >= 2) && __builtin_constant_p(bit)) { loongson_llsc_mb(); - do { - __asm__ __volatile__( - " " __LL "%0, %1 # clear_bit \n" - " " __INS "%0, $0, %2, 1 \n" - " " __SC "%0, %1 \n" - : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (*m) - : "i" (bit) - : __LLSC_CLOBBER); - } while (unlikely(!temp)); + __bit_op(*m, __INS "%0, $0, %2, 1", "i"(bit)); return; } loongson_llsc_mb(); - do { - __asm__ __volatile__( - " .set push \n" - " .set "MIPS_ISA_ARCH_LEVEL" \n" - " " __LL "%0, %1 # clear_bit \n" - " and %0, %2 \n" - " " __SC "%0, %1 \n" - " .set pop \n" - : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (*m) - : "ir" (~(BIT(bit))) - : __LLSC_CLOBBER); - } while (unlikely(!temp)); + __bit_op(*m, "and\t%0, %2", "ir"(~BIT(bit))); } /* @@ -197,41 +162,14 @@ static inline void change_bit(unsigned long nr, volatile unsigned long *addr) { unsigned long *m = ((unsigned long *)addr) + (nr >> SZLONG_LOG); int bit = nr & SZLONG_MASK; - unsigned long temp; if (!kernel_uses_llsc) { __mips_change_bit(nr, addr); return; } - if (R10000_LLSC_WAR) { - __asm__ __volatile__( - " .set push \n" - " .set arch=r4000 \n" - "1: " __LL "%0, %1 # change_bit \n" - " xor %0, %2 \n" - " " __SC "%0, %1 \n" - " beqzl %0, 1b \n" - " .set pop \n" - : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (*m) - : "ir" (BIT(bit)) - : __LLSC_CLOBBER); - return; - } - loongson_llsc_mb(); - do { - __asm__ __volatile__( - " .set push \n" - " .set "MIPS_ISA_ARCH_LEVEL" \n" - " " __LL "%0, %1 # change_bit \n" - " xor %0, %2 \n" - " " __SC "%0, %1 \n" - " .set pop \n" - : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (*m) - : "ir" (BIT(bit)) - : __LLSC_CLOBBER); - } while (unlikely(!temp)); + __bit_op(*m, "xor\t%0, %2", "ir"(BIT(bit))); } /* @@ -247,41 +185,16 @@ static inline int test_and_set_bit_lock(unsigned long nr, { unsigned long *m = ((unsigned long *)addr) + (nr >> SZLONG_LOG); int bit = nr & SZLONG_MASK; - unsigned long res, temp; + unsigned long res, orig; if (!kernel_uses_llsc) { res = __mips_test_and_set_bit_lock(nr, addr); - } else if (R10000_LLSC_WAR) { - __asm__ __volatile__( - " .set push \n" - " .set arch=r4000 \n" - "1: " __LL "%0, %1 # test_and_set_bit \n" - " or %2, %0, %3 \n" - " " __SC "%2, %1 \n" - " beqzl %2, 1b \n" - " and %2, %0, %3 \n" - " .set pop \n" - : "=&r" (temp), "+m" (*m), "=&r" (res) - : "ir" (BIT(bit)) - : __LLSC_CLOBBER); - - res = res != 0; } else { loongson_llsc_mb(); - do { - __asm__ __volatile__( - " .set push \n" - " .set "MIPS_ISA_ARCH_LEVEL" \n" - " " __LL "%0, %1 # test_and_set_bit \n" - " or %2, %0, %3 \n" - " " __SC "%2, %1 \n" - " .set pop \n" - : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (*m), "=&r" (res) - : "ir" (BIT(bit)) - : __LLSC_CLOBBER); - } while (unlikely(!res)); - - res = (temp & BIT(bit)) != 0; + orig = __test_bit_op(*m, "%0", + "or\t%1, %0, %3", + "ir"(BIT(bit))); + res = (orig & BIT(bit)) != 0; } smp_llsc_mb(); @@ -317,57 +230,25 @@ static inline int test_and_clear_bit(unsigned long nr, { unsigned long *m = ((unsigned long *)addr) + (nr >> SZLONG_LOG); int bit = nr & SZLONG_MASK; - unsigned long res, temp; + unsigned long res, orig; smp_mb__before_llsc(); if (!kernel_uses_llsc) { res = __mips_test_and_clear_bit(nr, addr); - } else if (R10000_LLSC_WAR) { - __asm__ __volatile__( - " .set push \n" - " .set arch=r4000 \n" - "1: " __LL "%0, %1 # test_and_clear_bit \n" - " or %2, %0, %3 \n" - " xor %2, %3 \n" - " " __SC "%2, %1 \n" - " beqzl %2, 1b \n" - " and %2, %0, %3 \n" - " .set pop \n" - : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (*m), "=&r" (res) - : "ir" (BIT(bit)) - : __LLSC_CLOBBER); - - res = res != 0; } else if ((MIPS_ISA_REV >= 2) && __builtin_constant_p(nr)) { loongson_llsc_mb(); - do { - __asm__ __volatile__( - " " __LL "%0, %1 # test_and_clear_bit \n" - " " __EXT "%2, %0, %3, 1 \n" - " " __INS "%0, $0, %3, 1 \n" - " " __SC "%0, %1 \n" - : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (*m), "=&r" (res) - : "i" (bit) - : __LLSC_CLOBBER); - } while (unlikely(!temp)); + res = __test_bit_op(*m, "%1", + __EXT "%0, %1, %3, 1;" + __INS "%1, $0, %3, 1", + "i"(bit)); } else { loongson_llsc_mb(); - do { - __asm__ __volatile__( - " .set push \n" - " .set "MIPS_ISA_ARCH_LEVEL" \n" - " " __LL "%0, %1 # test_and_clear_bit \n" - " or %2, %0, %3 \n" - " xor %2, %3 \n" - " " __SC "%2, %1 \n" - " .set pop \n" - : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (*m), "=&r" (res) - : "ir" (BIT(bit)) - : __LLSC_CLOBBER); - } while (unlikely(!res)); - - res = (temp & BIT(bit)) != 0; + orig = __test_bit_op(*m, "%0", + "or\t%1, %0, %3;" + "xor\t%1, %1, %3", + "ir"(BIT(bit))); + res = (orig & BIT(bit)) != 0; } smp_llsc_mb(); @@ -388,43 +269,18 @@ static inline int test_and_change_bit(unsigned long nr, { unsigned long *m = ((unsigned long *)addr) + (nr >> SZLONG_LOG); int bit = nr & SZLONG_MASK; - unsigned long res, temp; + unsigned long res, orig; smp_mb__before_llsc(); if (!kernel_uses_llsc) { res = __mips_test_and_change_bit(nr, addr); - } else if (R10000_LLSC_WAR) { - __asm__ __volatile__( - " .set push \n" - " .set arch=r4000 \n" - "1: " __LL "%0, %1 # test_and_change_bit \n" - " xor %2, %0, %3 \n" - " " __SC "%2, %1 \n" - " beqzl %2, 1b \n" - " and %2, %0, %3 \n" - " .set pop \n" - : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (*m), "=&r" (res) - : "ir" (BIT(bit)) - : __LLSC_CLOBBER); - - res = res != 0; } else { loongson_llsc_mb(); - do { - __asm__ __volatile__( - " .set push \n" - " .set "MIPS_ISA_ARCH_LEVEL" \n" - " " __LL "%0, %1 # test_and_change_bit \n" - " xor %2, %0, %3 \n" - " " __SC "\t%2, %1 \n" - " .set pop \n" - : "=&r" (temp), "+" GCC_OFF_SMALL_ASM() (*m), "=&r" (res) - : "ir" (BIT(bit)) - : __LLSC_CLOBBER); - } while (unlikely(!res)); - - res = (temp & BIT(bit)) != 0; + orig = __test_bit_op(*m, "%0", + "xor\t%1, %0, %3", + "ir"(BIT(bit))); + res = (orig & BIT(bit)) != 0; } smp_llsc_mb(); @@ -432,6 +288,9 @@ static inline int test_and_change_bit(unsigned long nr, return res; } +#undef __bit_op +#undef __test_bit_op + #include /* From patchwork Tue Oct 1 21:53:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Burton X-Patchwork-Id: 11169797 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3436A1709 for ; Tue, 1 Oct 2019 21:54:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id F3DE52133F for ; Tue, 1 Oct 2019 21:54:04 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=wavecomp.com header.i=@wavecomp.com header.b="U7gEdpFG" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728269AbfJAVxs (ORCPT ); Tue, 1 Oct 2019 17:53:48 -0400 Received: from mail-eopbgr730113.outbound.protection.outlook.com ([40.107.73.113]:17171 "EHLO NAM05-DM3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728256AbfJAVxs (ORCPT ); Tue, 1 Oct 2019 17:53:48 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jYHGFkRlb3yIf2LG0p0UI+FELIlufJeT97UQN/GS1OcEII4SLCQqlCEjnFXAip5dbbosO2gbsMa27ZQBcyrx2xD2mUeSfJRuyVdtLwrEhT34Dj7RRi9BEs+Q1TU0zpAE8wQIeb/BbiahUU7KcuUaXS2QYU9cFDhwf+qvVBQN9pHXQaN2MutfLfQK+0o8uWxL2Ih7aso5k65plAM7nc57Xsdhk+s5ObpE/4oEefWyuF/Q487aquo1v9xFlJBXJxtF3edoRKRPSm4feyJmQ+GiqssRfd+nFQGWt/6dQStlG2iUlPg3yM4riI3B6qwqQ/QG9sWr6bOtH4umWNXQWpRqng== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=daRCgn13zIlHccXug0wPnms++7PKm6/04xIJWExUjNQ=; b=ZHBDvFoD+deEi7nu1hwukUCZPssl6tPylX01oI5j1UdDZ2Svb1lsyjaaKcL6IBmHY/iV5Kq9WGKBzRXcB7ui/+GYEytIvZzA/2FKW8GmmQANQbiZzTmskR/qobZSbEWFYa6mKwPxCmNTE3L/AAzSRxEK7Y12bWySS/4SU6vSi1tz8HlnomncBcTOQFHd13+UrUg9TElT4Yam2ZKtPifQUDwdjv9JWdge2fwmQoKX55k588m/LtyOulsEd8XdE3uBGsAkcRacGhL6bzfZc6LRWIl8d5cLyShlziQFr1bL/BRjdWehPKOyGU9p6suUiiBtTTOkv+YyRi3UFyJQKAr2VQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wavecomp.com; dmarc=pass action=none header.from=mips.com; dkim=pass header.d=mips.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wavecomp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=daRCgn13zIlHccXug0wPnms++7PKm6/04xIJWExUjNQ=; b=U7gEdpFGtss0zaQGOugOzdqe76v3Qqkw/mHOL/OJFOk8FHxDzQaBKfSaqqphmpcyy0KuzqZVwAhzVW3h8IbxSonb56SE8W4oulAWsJEI741/rX0s2zvYjhufLxt6Yes5QwcohHlw/XJIHz7WcR/0osYbFRLfab89e9LmErtts0o= Received: from MWHPR2201MB1277.namprd22.prod.outlook.com (10.172.60.12) by MWHPR2201MB1439.namprd22.prod.outlook.com (10.174.169.166) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2305.20; Tue, 1 Oct 2019 21:53:35 +0000 Received: from MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481]) by MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481%2]) with mapi id 15.20.2305.022; Tue, 1 Oct 2019 21:53:34 +0000 From: Paul Burton To: "linux-mips@vger.kernel.org" CC: Huacai Chen , Jiaxun Yang , "linux-kernel@vger.kernel.org" , Paul Burton Subject: [PATCH v2 25/36] MIPS: bitops: Use BIT_WORD() & BITS_PER_LONG Thread-Topic: [PATCH v2 25/36] MIPS: bitops: Use BIT_WORD() & BITS_PER_LONG Thread-Index: AQHVeKKsakmoK+XUGUmcY9FGfdcSIQ== Date: Tue, 1 Oct 2019 21:53:34 +0000 Message-ID: <20191001215249.4157062-26-paul.burton@mips.com> References: <20191001215249.4157062-1-paul.burton@mips.com> In-Reply-To: <20191001215249.4157062-1-paul.burton@mips.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BYAPR07CA0093.namprd07.prod.outlook.com (2603:10b6:a03:12b::34) To MWHPR2201MB1277.namprd22.prod.outlook.com (2603:10b6:301:18::12) authentication-results: spf=none (sender IP is ) smtp.mailfrom=pburton@wavecomp.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.23.0 x-originating-ip: [12.94.197.246] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 8f01e9a7-0ca9-45b5-b469-08d746b9ce7d x-ms-traffictypediagnostic: MWHPR2201MB1439: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:7219; x-forefront-prvs: 0177904E6B x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(1496009)(396003)(39840400004)(366004)(346002)(136003)(376002)(199004)(189003)(186003)(11346002)(6512007)(1076003)(486006)(66066001)(26005)(6486002)(316002)(2906002)(5640700003)(6436002)(44832011)(42882007)(25786009)(446003)(6916009)(386003)(6506007)(76176011)(478600001)(52116002)(2616005)(102836004)(476003)(5660300002)(305945005)(64756008)(66446008)(14444005)(2351001)(7736002)(99286004)(54906003)(14454004)(71190400001)(71200400001)(66946007)(66556008)(66476007)(36756003)(3846002)(6116002)(107886003)(4326008)(50226002)(81166006)(81156014)(8676002)(2501003)(8936002)(256004);DIR:OUT;SFP:1102;SCL:1;SRVR:MWHPR2201MB1439;H:MWHPR2201MB1277.namprd22.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; received-spf: None (protection.outlook.com: wavecomp.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 7rlgR6FLkNBfAWHISw5m4fjEfeZZjJS4V6UeN3DiXdrniIQTAnUq2TEc7eCQR6rnu9+J7JwZaxZBEDdOOrboFd4EpEXQ24+wsjKwHQLFEVg83QxEKNiR3Gac2VXycLerZeiUt1DJ5WZKLtYAiT5mENAEsBdsO1cw5Zf82vA3xJ4Svb/NZ8frH68TuMSk7yUw4r2RMDSo/WyjQrCk8PY6MXJmxp6k0G3tdubj2cU91snEwZxFHdWHTb8v55wWLk/Pq0eOAqyqeA8xAKfgoA+tLzLzSy3UIXMrt8eZHg7LL63/CxcRzoXtDN/n9qkPaciTrv/cMGBNQW/eKn5H14gdaIRdOQWT3XcR3p3k9dLSKJlN28+U99HgPgCY3eLeb+3QFGJ5oOo6pEAWG+n9RB0s6Wtq7cV6gj3AMT7fyYdo8Hc= MIME-Version: 1.0 X-OriginatorOrg: mips.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8f01e9a7-0ca9-45b5-b469-08d746b9ce7d X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Oct 2019 21:53:34.4902 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 463607d3-1db3-40a0-8a29-970c56230104 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: oBuGi01PXFjRRVLGtoiyiwGvCZ+Fw9GyrTG7U+SrLQbV6A7zjP/Wr5RnxqRCXweeK0/mIpC15xxJQIixlecppg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR2201MB1439 Sender: linux-mips-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-mips@vger.kernel.org Rather than using custom SZLONG_LOG & SZLONG_MASK macros to shift & mask a bit index to form word & bit offsets respectively, make use of the standard BIT_WORD() & BITS_PER_LONG macros for the same purpose. volatile is added to the definition of pointers to the long-sized word we'll operate on, in order to prevent the compiler complaining that we cast away the volatile qualifier of the addr argument. This should have no effect on generated code, which in the LL/SC case is inline asm anyway & in the non-LLSC case access is constrained by compiler barriers provided by raw_local_irq_{save,restore}(). Signed-off-by: Paul Burton --- Changes in v2: None arch/mips/include/asm/bitops.h | 24 ++++++++++++------------ arch/mips/include/asm/llsc.h | 4 ---- arch/mips/lib/bitops.c | 31 +++++++++++++------------------ 3 files changed, 25 insertions(+), 34 deletions(-) diff --git a/arch/mips/include/asm/bitops.h b/arch/mips/include/asm/bitops.h index fba0a842b98a..d39fca2def60 100644 --- a/arch/mips/include/asm/bitops.h +++ b/arch/mips/include/asm/bitops.h @@ -87,8 +87,8 @@ int __mips_test_and_change_bit(unsigned long nr, */ static inline void set_bit(unsigned long nr, volatile unsigned long *addr) { - unsigned long *m = ((unsigned long *)addr) + (nr >> SZLONG_LOG); - int bit = nr & SZLONG_MASK; + volatile unsigned long *m = &addr[BIT_WORD(nr)]; + int bit = nr % BITS_PER_LONG; if (!kernel_uses_llsc) { __mips_set_bit(nr, addr); @@ -117,8 +117,8 @@ static inline void set_bit(unsigned long nr, volatile unsigned long *addr) */ static inline void clear_bit(unsigned long nr, volatile unsigned long *addr) { - unsigned long *m = ((unsigned long *)addr) + (nr >> SZLONG_LOG); - int bit = nr & SZLONG_MASK; + volatile unsigned long *m = &addr[BIT_WORD(nr)]; + int bit = nr % BITS_PER_LONG; if (!kernel_uses_llsc) { __mips_clear_bit(nr, addr); @@ -160,8 +160,8 @@ static inline void clear_bit_unlock(unsigned long nr, volatile unsigned long *ad */ static inline void change_bit(unsigned long nr, volatile unsigned long *addr) { - unsigned long *m = ((unsigned long *)addr) + (nr >> SZLONG_LOG); - int bit = nr & SZLONG_MASK; + volatile unsigned long *m = &addr[BIT_WORD(nr)]; + int bit = nr % BITS_PER_LONG; if (!kernel_uses_llsc) { __mips_change_bit(nr, addr); @@ -183,8 +183,8 @@ static inline void change_bit(unsigned long nr, volatile unsigned long *addr) static inline int test_and_set_bit_lock(unsigned long nr, volatile unsigned long *addr) { - unsigned long *m = ((unsigned long *)addr) + (nr >> SZLONG_LOG); - int bit = nr & SZLONG_MASK; + volatile unsigned long *m = &addr[BIT_WORD(nr)]; + int bit = nr % BITS_PER_LONG; unsigned long res, orig; if (!kernel_uses_llsc) { @@ -228,8 +228,8 @@ static inline int test_and_set_bit(unsigned long nr, static inline int test_and_clear_bit(unsigned long nr, volatile unsigned long *addr) { - unsigned long *m = ((unsigned long *)addr) + (nr >> SZLONG_LOG); - int bit = nr & SZLONG_MASK; + volatile unsigned long *m = &addr[BIT_WORD(nr)]; + int bit = nr % BITS_PER_LONG; unsigned long res, orig; smp_mb__before_llsc(); @@ -267,8 +267,8 @@ static inline int test_and_clear_bit(unsigned long nr, static inline int test_and_change_bit(unsigned long nr, volatile unsigned long *addr) { - unsigned long *m = ((unsigned long *)addr) + (nr >> SZLONG_LOG); - int bit = nr & SZLONG_MASK; + volatile unsigned long *m = &addr[BIT_WORD(nr)]; + int bit = nr % BITS_PER_LONG; unsigned long res, orig; smp_mb__before_llsc(); diff --git a/arch/mips/include/asm/llsc.h b/arch/mips/include/asm/llsc.h index d240a4a2d1c4..c49738bc3bda 100644 --- a/arch/mips/include/asm/llsc.h +++ b/arch/mips/include/asm/llsc.h @@ -12,15 +12,11 @@ #include #if _MIPS_SZLONG == 32 -#define SZLONG_LOG 5 -#define SZLONG_MASK 31UL #define __LL "ll " #define __SC "sc " #define __INS "ins " #define __EXT "ext " #elif _MIPS_SZLONG == 64 -#define SZLONG_LOG 6 -#define SZLONG_MASK 63UL #define __LL "lld " #define __SC "scd " #define __INS "dins " diff --git a/arch/mips/lib/bitops.c b/arch/mips/lib/bitops.c index fba402c0879d..116d0bd8b2ae 100644 --- a/arch/mips/lib/bitops.c +++ b/arch/mips/lib/bitops.c @@ -7,6 +7,7 @@ * Copyright (c) 1999, 2000 Silicon Graphics, Inc. */ #include +#include #include #include @@ -19,12 +20,11 @@ */ void __mips_set_bit(unsigned long nr, volatile unsigned long *addr) { - unsigned long *a = (unsigned long *)addr; - unsigned bit = nr & SZLONG_MASK; + volatile unsigned long *a = &addr[BIT_WORD(nr)]; + unsigned int bit = nr % BITS_PER_LONG; unsigned long mask; unsigned long flags; - a += nr >> SZLONG_LOG; mask = 1UL << bit; raw_local_irq_save(flags); *a |= mask; @@ -41,12 +41,11 @@ EXPORT_SYMBOL(__mips_set_bit); */ void __mips_clear_bit(unsigned long nr, volatile unsigned long *addr) { - unsigned long *a = (unsigned long *)addr; - unsigned bit = nr & SZLONG_MASK; + volatile unsigned long *a = &addr[BIT_WORD(nr)]; + unsigned int bit = nr % BITS_PER_LONG; unsigned long mask; unsigned long flags; - a += nr >> SZLONG_LOG; mask = 1UL << bit; raw_local_irq_save(flags); *a &= ~mask; @@ -63,12 +62,11 @@ EXPORT_SYMBOL(__mips_clear_bit); */ void __mips_change_bit(unsigned long nr, volatile unsigned long *addr) { - unsigned long *a = (unsigned long *)addr; - unsigned bit = nr & SZLONG_MASK; + volatile unsigned long *a = &addr[BIT_WORD(nr)]; + unsigned int bit = nr % BITS_PER_LONG; unsigned long mask; unsigned long flags; - a += nr >> SZLONG_LOG; mask = 1UL << bit; raw_local_irq_save(flags); *a ^= mask; @@ -86,13 +84,12 @@ EXPORT_SYMBOL(__mips_change_bit); int __mips_test_and_set_bit_lock(unsigned long nr, volatile unsigned long *addr) { - unsigned long *a = (unsigned long *)addr; - unsigned bit = nr & SZLONG_MASK; + volatile unsigned long *a = &addr[BIT_WORD(nr)]; + unsigned int bit = nr % BITS_PER_LONG; unsigned long mask; unsigned long flags; int res; - a += nr >> SZLONG_LOG; mask = 1UL << bit; raw_local_irq_save(flags); res = (mask & *a) != 0; @@ -111,13 +108,12 @@ EXPORT_SYMBOL(__mips_test_and_set_bit_lock); */ int __mips_test_and_clear_bit(unsigned long nr, volatile unsigned long *addr) { - unsigned long *a = (unsigned long *)addr; - unsigned bit = nr & SZLONG_MASK; + volatile unsigned long *a = &addr[BIT_WORD(nr)]; + unsigned int bit = nr % BITS_PER_LONG; unsigned long mask; unsigned long flags; int res; - a += nr >> SZLONG_LOG; mask = 1UL << bit; raw_local_irq_save(flags); res = (mask & *a) != 0; @@ -136,13 +132,12 @@ EXPORT_SYMBOL(__mips_test_and_clear_bit); */ int __mips_test_and_change_bit(unsigned long nr, volatile unsigned long *addr) { - unsigned long *a = (unsigned long *)addr; - unsigned bit = nr & SZLONG_MASK; + volatile unsigned long *a = &addr[BIT_WORD(nr)]; + unsigned int bit = nr % BITS_PER_LONG; unsigned long mask; unsigned long flags; int res; - a += nr >> SZLONG_LOG; mask = 1UL << bit; raw_local_irq_save(flags); res = (mask & *a) != 0; From patchwork Tue Oct 1 21:53:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Burton X-Patchwork-Id: 11169795 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EB62A1599 for ; Tue, 1 Oct 2019 21:54:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id BE1302133F for ; Tue, 1 Oct 2019 21:54:04 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=wavecomp.com header.i=@wavecomp.com header.b="hjcM0gd+" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728256AbfJAVyA (ORCPT ); Tue, 1 Oct 2019 17:54:00 -0400 Received: from mail-eopbgr730113.outbound.protection.outlook.com ([40.107.73.113]:17171 "EHLO NAM05-DM3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728267AbfJAVxs (ORCPT ); Tue, 1 Oct 2019 17:53:48 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=gEkrCZRYhqja6gjw3X1B9TMrcD3FMSQJJfwlwwABOz7B38zuZNdrXjj9cZVM6OYtKb0Gjap0CTcDczv3jpGXsSy8H+y1bzHCTeTHfi2RHpjU7dgoSpLfHJLfJpEDqaEwiLRGSYGOeM3NRiZri+0p2HADYe+/Wy9fmqSi6PjO0wI0KCL+7xf4nl3W8IVIMcW57JaAe1xKuCfsxyI22zyA4u+WLpJU5kW5vMvkmZSkBGkD5Y+flNw7KgGmhQSZa9iwZI5DVPWao2hVBg6oKWUY6yfMZ+Lu4W+E5aaTA7akzZO1WpZbM9K8eZvviTjOlJ11+d1O7YS29wGDWjHiKuki+A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4wKkI6JMqdWbvnxqs2rDUzI1Mrt/HuPnZG7sFlZTi/A=; b=ZJjyb1J5aQyMUR7I6mFD3dciCcbEmdQKl5xlwujuMdj9gOSccDzdBrFGC6VjxgSLLcToyUJPa7gomJXsRrn/Zz9rPPSZVehn6h53mpU3zWkF4wSOUIbG67SsdbbVB0LWeeMuGB/0A/gKyF7Gw488Ujy7RgCKV+rkyfUraxNTK2DOL+hm9qnZRPu5zmD8PaA07YMyfE5bY7WZSap7+A3oKfX44so7UWH3WFPjKAeTKgZIo/U/BdxaEiLxUJoG9xVPVtAxNMR3aBAEpHWAk9XXz0mG7CSuSlwScd6ww1kOS3mgY0hYRwBas3e1mfsnmTcpRuPi+ffnjF0yDnWaP1M/7A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wavecomp.com; dmarc=pass action=none header.from=mips.com; dkim=pass header.d=mips.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wavecomp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4wKkI6JMqdWbvnxqs2rDUzI1Mrt/HuPnZG7sFlZTi/A=; b=hjcM0gd+3WebPEJM8pn99JujlHKeOB99RZ/JGyPoYXavNmbcOJxNxR6RAXd9DtGqTONdQP0J073i6S08dMH3/hPn91IdP99MSQtFtVBhC3ImM3dBpoqe3ims2U02NKSDH6tr7ei00KYgYynKzvP/44ZEvu6c3OgWJE41g9Bo724= Received: from MWHPR2201MB1277.namprd22.prod.outlook.com (10.172.60.12) by MWHPR2201MB1439.namprd22.prod.outlook.com (10.174.169.166) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2305.20; Tue, 1 Oct 2019 21:53:36 +0000 Received: from MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481]) by MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481%2]) with mapi id 15.20.2305.022; Tue, 1 Oct 2019 21:53:35 +0000 From: Paul Burton To: "linux-mips@vger.kernel.org" CC: Huacai Chen , Jiaxun Yang , "linux-kernel@vger.kernel.org" , Paul Burton Subject: [PATCH v2 26/36] MIPS: bitops: Emit Loongson3 sync workarounds within asm Thread-Topic: [PATCH v2 26/36] MIPS: bitops: Emit Loongson3 sync workarounds within asm Thread-Index: AQHVeKKsNoYb7CyIfESjM7bTex9Tow== Date: Tue, 1 Oct 2019 21:53:35 +0000 Message-ID: <20191001215249.4157062-27-paul.burton@mips.com> References: <20191001215249.4157062-1-paul.burton@mips.com> In-Reply-To: <20191001215249.4157062-1-paul.burton@mips.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BYAPR07CA0093.namprd07.prod.outlook.com (2603:10b6:a03:12b::34) To MWHPR2201MB1277.namprd22.prod.outlook.com (2603:10b6:301:18::12) authentication-results: spf=none (sender IP is ) smtp.mailfrom=pburton@wavecomp.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.23.0 x-originating-ip: [12.94.197.246] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 0e8d0cd4-70c4-44d1-aa66-08d746b9cf06 x-ms-traffictypediagnostic: MWHPR2201MB1439: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:6108; x-forefront-prvs: 0177904E6B x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(1496009)(396003)(39840400004)(366004)(346002)(136003)(376002)(199004)(189003)(186003)(11346002)(6512007)(1076003)(486006)(66066001)(26005)(6486002)(316002)(2906002)(5640700003)(6436002)(44832011)(42882007)(25786009)(446003)(6916009)(386003)(6506007)(76176011)(478600001)(52116002)(2616005)(102836004)(476003)(5660300002)(305945005)(64756008)(66446008)(14444005)(2351001)(7736002)(99286004)(54906003)(14454004)(71190400001)(71200400001)(66946007)(66556008)(66476007)(36756003)(3846002)(6116002)(107886003)(4326008)(50226002)(81166006)(81156014)(8676002)(2501003)(8936002)(256004);DIR:OUT;SFP:1102;SCL:1;SRVR:MWHPR2201MB1439;H:MWHPR2201MB1277.namprd22.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; received-spf: None (protection.outlook.com: wavecomp.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 0FO8A+Xf3fzwtQi8vp1mpk/ZoC/jA8qIP2i7d3+xsY8puTJIia6am0YmEutrGSbTv5qMgkX5mkzIigPbb/p/y8zNwHLvFPAEe5OOklZYX1mQ3avtA3MOy/D1rGVWcII6dHKZttw8DLH0b2E2nEgbr5haEU1zDhJOOsNKju6c3Lt4/KoDYo8twZr15ZKGjcBpRrTuJ3degm6CNgBuTOuTWhu0HbXER0XA+rqDV9ZmSEhdHGYKQmfIxnhHeuJ+sYCDQyBptCgI7jNssHPzAJ5Uh+HZkcMNQ1wIaAkePG76N7rV/rdnawtmI4+zV4WMq5qwoOl9st639P//FCPErC1bGBdXw7/04HzpjRemRznoEZOcvpzPn1P1bmujf9pUAdxqH8cgfrcx8tmEVsmfgCWtkoYvwwG/hyk2jqSyrs+LNZU= MIME-Version: 1.0 X-OriginatorOrg: mips.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0e8d0cd4-70c4-44d1-aa66-08d746b9cf06 X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Oct 2019 21:53:35.3768 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 463607d3-1db3-40a0-8a29-970c56230104 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: ZVhj/JR8I/oxJxOaMM7zHpMQsI6W7wxddYiAzhPfClcAg5Q10WXAei7OUnkEm9Ux639VfQUkDzt5xb7UJ/8JFQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR2201MB1439 Sender: linux-mips-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-mips@vger.kernel.org Generate the sync instructions required to workaround Loongson3 LL/SC errata within inline asm blocks, which feels a little safer than doing it from C where strictly speaking the compiler would be well within its rights to insert a memory access between the separate asm statements we previously had, containing sync & ll instructions respectively. Signed-off-by: Paul Burton --- Changes in v2: None arch/mips/include/asm/bitops.h | 11 ++--------- 1 file changed, 2 insertions(+), 9 deletions(-) diff --git a/arch/mips/include/asm/bitops.h b/arch/mips/include/asm/bitops.h index d39fca2def60..c08b6d225f10 100644 --- a/arch/mips/include/asm/bitops.h +++ b/arch/mips/include/asm/bitops.h @@ -31,6 +31,7 @@ asm volatile( \ " .set push \n" \ " .set " MIPS_ISA_LEVEL " \n" \ + " " __SYNC(full, loongson3_war) " \n" \ "1: " __LL "%0, %1 \n" \ " " insn " \n" \ " " __SC "%0, %1 \n" \ @@ -47,6 +48,7 @@ asm volatile( \ " .set push \n" \ " .set " MIPS_ISA_LEVEL " \n" \ + " " __SYNC(full, loongson3_war) " \n" \ "1: " __LL ll_dst ", %2 \n" \ " " insn " \n" \ " " __SC "%1, %2 \n" \ @@ -96,12 +98,10 @@ static inline void set_bit(unsigned long nr, volatile unsigned long *addr) } if ((MIPS_ISA_REV >= 2) && __builtin_constant_p(bit) && (bit >= 16)) { - loongson_llsc_mb(); __bit_op(*m, __INS "%0, %3, %2, 1", "i"(bit), "r"(~0)); return; } - loongson_llsc_mb(); __bit_op(*m, "or\t%0, %2", "ir"(BIT(bit))); } @@ -126,12 +126,10 @@ static inline void clear_bit(unsigned long nr, volatile unsigned long *addr) } if ((MIPS_ISA_REV >= 2) && __builtin_constant_p(bit)) { - loongson_llsc_mb(); __bit_op(*m, __INS "%0, $0, %2, 1", "i"(bit)); return; } - loongson_llsc_mb(); __bit_op(*m, "and\t%0, %2", "ir"(~BIT(bit))); } @@ -168,7 +166,6 @@ static inline void change_bit(unsigned long nr, volatile unsigned long *addr) return; } - loongson_llsc_mb(); __bit_op(*m, "xor\t%0, %2", "ir"(BIT(bit))); } @@ -190,7 +187,6 @@ static inline int test_and_set_bit_lock(unsigned long nr, if (!kernel_uses_llsc) { res = __mips_test_and_set_bit_lock(nr, addr); } else { - loongson_llsc_mb(); orig = __test_bit_op(*m, "%0", "or\t%1, %0, %3", "ir"(BIT(bit))); @@ -237,13 +233,11 @@ static inline int test_and_clear_bit(unsigned long nr, if (!kernel_uses_llsc) { res = __mips_test_and_clear_bit(nr, addr); } else if ((MIPS_ISA_REV >= 2) && __builtin_constant_p(nr)) { - loongson_llsc_mb(); res = __test_bit_op(*m, "%1", __EXT "%0, %1, %3, 1;" __INS "%1, $0, %3, 1", "i"(bit)); } else { - loongson_llsc_mb(); orig = __test_bit_op(*m, "%0", "or\t%1, %0, %3;" "xor\t%1, %1, %3", @@ -276,7 +270,6 @@ static inline int test_and_change_bit(unsigned long nr, if (!kernel_uses_llsc) { res = __mips_test_and_change_bit(nr, addr); } else { - loongson_llsc_mb(); orig = __test_bit_op(*m, "%0", "xor\t%1, %0, %3", "ir"(BIT(bit))); From patchwork Tue Oct 1 21:53:36 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Burton X-Patchwork-Id: 11169789 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 65B3B1599 for ; Tue, 1 Oct 2019 21:53:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 41BD52133F for ; Tue, 1 Oct 2019 21:53:46 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=wavecomp.com header.i=@wavecomp.com header.b="qcWHbxwp" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728235AbfJAVxp (ORCPT ); Tue, 1 Oct 2019 17:53:45 -0400 Received: from mail-eopbgr750107.outbound.protection.outlook.com ([40.107.75.107]:29167 "EHLO NAM02-BL2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728206AbfJAVxk (ORCPT ); Tue, 1 Oct 2019 17:53:40 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=cfEnthP9G2b7/xZ19kw37z/vIN8nS+7xeNoqDD+qfa4wZsfBCghkgGldO8AZLKllkMCMF8exJXztGAKBuyjKvJTNEkIx6156rvBROLGIahk6aNjDx8cnU36E4ZUzOtbMBVFM8ZiySKziMBBC3PR98ozcWU7VJaJQwPAKJgTxvEv5IUb/vd34A+bx8lRFIV1gI5lsdRZ8mNcmMSWb0P3GrO2WsPUSwbXDc66ZuUsM0MQp8kRv8wiQ63trvzGfLgjLiaf4KriJ7JKp2GumaHEgDe4Olrff/7ZRewDQQZjHaM7fernh06v1DCm9RF8PTtSI29pUqFEX6vtzq0e3gG2lig== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cesVpm9iaWg8/mEmkyc7BFfg1I1588Rr4QlIPaQRPiU=; b=PIYI/V4+EEQ9+N9k9mnyMPT2XcMl4eLQXpw2NMg17rbv2fl4/6/XbMCUepweHPbetoGnqDWi+rKzib+kTLPVLMwz6ERJYAVoKdGpjSgyQHTnhfB7qXc8SB6LgMOZj1/1rhmL1tcd1NTdx3tDcEnZQ4lXcB4RrchAlwOMa5bggj0BfIas1mfyOaJcXC/EK/l3iTE6Tukiz1ys3ci00o+wB3SPFrgOMlMZyPUmC1Sv3k8S96+PKxEVI6jNrsx2HtJu29nU17NEshn1bbZtRMzLVzMohijaPqUuvI6WxOEujpsnqkriiVN9JBjCy8qJ4EG7qecHOc9I/FuUPKWFi85yaQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wavecomp.com; dmarc=pass action=none header.from=mips.com; dkim=pass header.d=mips.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wavecomp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cesVpm9iaWg8/mEmkyc7BFfg1I1588Rr4QlIPaQRPiU=; b=qcWHbxwp/j4I39V+mQega4XfN9N+icoYbc2Z+yZGD7o9QqQRNWggO6/Kcv/I7RvaGT+xnNrkrmcrTkorkiPzs/+dP108rlNqcq5+/J1Pjq7AGJ/t+OjMSuk3L3FU7oe80BDvfrpHsNGC+XvhbIgwbpnDthb+sThQDsJIJKr8T6I= Received: from MWHPR2201MB1277.namprd22.prod.outlook.com (10.172.60.12) by MWHPR2201MB1581.namprd22.prod.outlook.com (10.174.167.30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2305.15; Tue, 1 Oct 2019 21:53:36 +0000 Received: from MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481]) by MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481%2]) with mapi id 15.20.2305.022; Tue, 1 Oct 2019 21:53:36 +0000 From: Paul Burton To: "linux-mips@vger.kernel.org" CC: Huacai Chen , Jiaxun Yang , "linux-kernel@vger.kernel.org" , Paul Burton Subject: [PATCH v2 27/36] MIPS: bitops: Use smp_mb__before_atomic in test_* ops Thread-Topic: [PATCH v2 27/36] MIPS: bitops: Use smp_mb__before_atomic in test_* ops Thread-Index: AQHVeKKtOOr6hECUOEGysD9KJdfMKg== Date: Tue, 1 Oct 2019 21:53:36 +0000 Message-ID: <20191001215249.4157062-28-paul.burton@mips.com> References: <20191001215249.4157062-1-paul.burton@mips.com> In-Reply-To: <20191001215249.4157062-1-paul.burton@mips.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BYAPR07CA0093.namprd07.prod.outlook.com (2603:10b6:a03:12b::34) To MWHPR2201MB1277.namprd22.prod.outlook.com (2603:10b6:301:18::12) authentication-results: spf=none (sender IP is ) smtp.mailfrom=pburton@wavecomp.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.23.0 x-originating-ip: [12.94.197.246] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 293abf96-075f-4007-0363-08d746b9cfa1 x-ms-traffictypediagnostic: MWHPR2201MB1581: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:2582; x-forefront-prvs: 0177904E6B x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(136003)(39850400004)(366004)(346002)(396003)(376002)(189003)(199004)(8936002)(26005)(478600001)(6512007)(8676002)(81166006)(81156014)(6916009)(386003)(76176011)(186003)(6506007)(25786009)(52116002)(6436002)(99286004)(6486002)(316002)(102836004)(54906003)(14454004)(5640700003)(2501003)(486006)(66476007)(7736002)(256004)(14444005)(66556008)(66946007)(1076003)(64756008)(36756003)(107886003)(66066001)(66446008)(4326008)(42882007)(2906002)(3846002)(6116002)(476003)(446003)(2616005)(11346002)(2351001)(71190400001)(44832011)(5660300002)(50226002)(305945005)(71200400001)(101420200001);DIR:OUT;SFP:1102;SCL:1;SRVR:MWHPR2201MB1581;H:MWHPR2201MB1277.namprd22.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;MX:1;A:1; received-spf: None (protection.outlook.com: wavecomp.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: ItjOJvWp4BzWFI1Kdp2svZtYDlGbNf5zWk5vyVobfr0yWrXuGcAigXsAKqGKkbnwVYseDlNQPe3TKMrBw0VCEs2vthGPkbvSypTgDyi8wFeYzi2Koe4DN3OK+AxsbpIEhIwn7ObPc3bcvRHk+ZnMBoNpgMfacKA1kLgSoMvgWexz4o0ag8EuSR7XG7t9f9XoN+Vl0XyU3kSnIGpID6cvXPL73zyX0vZMt4n/Ik9VWDdI2ljZK7bJOycO0yOo/LxfvAbbzYxCjYxIQ+j2o78tAK06pwe8qQ/Drl26F9i5ridaW3bhCXLAZubADfZr9jw4njiBNgthj7lrLn3NuhoRIBYJJb9fEZ3AIaVLCJM3QOkDstEVsxgpj/UMAkuqr5kXBTpuL8SOww9t5qvB+kuhR+r2R4Z9NdRVMXlT5YI102k= MIME-Version: 1.0 X-OriginatorOrg: mips.com X-MS-Exchange-CrossTenant-Network-Message-Id: 293abf96-075f-4007-0363-08d746b9cfa1 X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Oct 2019 21:53:36.5353 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 463607d3-1db3-40a0-8a29-970c56230104 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: x88NzXtKH7dNZFJRPqCEn3lQXqdzogmacEMlSPoq95EOrDOeODy66qwEkokxlJcaB1H7c/brgwQLsrGdGJjPGw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR2201MB1581 Sender: linux-mips-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-mips@vger.kernel.org Use smp_mb__before_atomic() rather than smp_mb__before_llsc() in test_and_set_bit(), test_and_clear_bit() & test_and_change_bit(). The _atomic() versions make semantic sense in these cases, and will allow a later patch to omit redundant barriers for Loongson3 systems that already include a barrier within __test_bit_op(). Signed-off-by: Paul Burton --- Changes in v2: None arch/mips/include/asm/bitops.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/arch/mips/include/asm/bitops.h b/arch/mips/include/asm/bitops.h index c08b6d225f10..a74769940fbd 100644 --- a/arch/mips/include/asm/bitops.h +++ b/arch/mips/include/asm/bitops.h @@ -209,7 +209,7 @@ static inline int test_and_set_bit_lock(unsigned long nr, static inline int test_and_set_bit(unsigned long nr, volatile unsigned long *addr) { - smp_mb__before_llsc(); + smp_mb__before_atomic(); return test_and_set_bit_lock(nr, addr); } @@ -228,7 +228,7 @@ static inline int test_and_clear_bit(unsigned long nr, int bit = nr % BITS_PER_LONG; unsigned long res, orig; - smp_mb__before_llsc(); + smp_mb__before_atomic(); if (!kernel_uses_llsc) { res = __mips_test_and_clear_bit(nr, addr); @@ -265,7 +265,7 @@ static inline int test_and_change_bit(unsigned long nr, int bit = nr % BITS_PER_LONG; unsigned long res, orig; - smp_mb__before_llsc(); + smp_mb__before_atomic(); if (!kernel_uses_llsc) { res = __mips_test_and_change_bit(nr, addr); From patchwork Tue Oct 1 21:53:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Burton X-Patchwork-Id: 11169819 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 535C31599 for ; Tue, 1 Oct 2019 21:54:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 26D2721A4A for ; Tue, 1 Oct 2019 21:54:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=wavecomp.com header.i=@wavecomp.com header.b="Y1Jdq1Zw" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728356AbfJAVyM (ORCPT ); Tue, 1 Oct 2019 17:54:12 -0400 Received: from mail-eopbgr750107.outbound.protection.outlook.com ([40.107.75.107]:29167 "EHLO NAM02-BL2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728228AbfJAVxo (ORCPT ); Tue, 1 Oct 2019 17:53:44 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=IPJgBCyoQtWND11ksud+SdcKXI9nyGTBdY3O60WAF/IcvfwjRpeqjd7+4rXyQ0FJsXRM4RLIcUeSzo1Fo/VmygV3vCZKHevNwkLTzw0qvUQODnBnn4MTn2Icvv6EF9ULSOCs0yKS7I1+KZw+U0TiiqMWIv4h2pkgakvO/roRylQN5SCfWKJHK5sjACO25Lc2hbL1HN9/R2f7/10qYmlv/lFHzCOyYkDzra2Dtt46J3cup81My1kcMgj4qs246NzOOP02eHqDvGmQ5gljUNdHTMiieum3u0A/sfOu6T9e8D7grlP8/l+dTlitTC9JJF5/sK7uDwhZ9prYSYUaIi8thg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5ofa9KCl/0RsNTSD7ocqa4+xq30RwjFidGpIUp1H1oQ=; b=EPjUEfM6Q0WyY84AQQwAmKsb9RqmaBAvKMSWji+ZHUJlt+i5fYlWvjnD2R+AiX/76j01VLVMr9fklEnIK9IIelYY4bpc5wXNzB7PxArulLdLtIHxG7u5f64/SYiXZ+Y10eSACIlXOKAnOnqHfm0TQz8p73YLlQRlBxjjJRl+GyxZQm3dyVqBq/nQHecNn9zRFHfjMMR8D+rhCN4CW312S+r3BZ00Z3AXdIEp4zWl297GvsQQVu42QBtKQT5erpFZffRKlna6Oxkvd4zndcxckdc800jmiKLyXoVsYoIGWBVt8LnwnckyvdxjzZv7ssJVPKOLUq3ny3KwmsuRHP1b2w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wavecomp.com; dmarc=pass action=none header.from=mips.com; dkim=pass header.d=mips.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wavecomp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5ofa9KCl/0RsNTSD7ocqa4+xq30RwjFidGpIUp1H1oQ=; b=Y1Jdq1ZwggnILB0UsQjQe45LcQU073LjnroZY4JoRWPpsE4jg5okUDnXXr0RButUojBEP02NJzH0nJWs3Tvrg+LRKbpR0jJrOCcCqareHZHYoOjxCOrYd4jl5OraadY/0PdlfLDVJ2iRAdE+lsHQiuaLjhyGG7+gP83H+KgU9C0= Received: from MWHPR2201MB1277.namprd22.prod.outlook.com (10.172.60.12) by MWHPR2201MB1581.namprd22.prod.outlook.com (10.174.167.30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2305.15; Tue, 1 Oct 2019 21:53:37 +0000 Received: from MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481]) by MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481%2]) with mapi id 15.20.2305.022; Tue, 1 Oct 2019 21:53:37 +0000 From: Paul Burton To: "linux-mips@vger.kernel.org" CC: Huacai Chen , Jiaxun Yang , "linux-kernel@vger.kernel.org" , Paul Burton Subject: [PATCH v2 28/36] MIPS: cmpxchg: Emit Loongson3 sync workarounds within asm Thread-Topic: [PATCH v2 28/36] MIPS: cmpxchg: Emit Loongson3 sync workarounds within asm Thread-Index: AQHVeKKt1X3uRc/5eE62lTuyLiifoQ== Date: Tue, 1 Oct 2019 21:53:37 +0000 Message-ID: <20191001215249.4157062-29-paul.burton@mips.com> References: <20191001215249.4157062-1-paul.burton@mips.com> In-Reply-To: <20191001215249.4157062-1-paul.burton@mips.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BYAPR07CA0093.namprd07.prod.outlook.com (2603:10b6:a03:12b::34) To MWHPR2201MB1277.namprd22.prod.outlook.com (2603:10b6:301:18::12) authentication-results: spf=none (sender IP is ) smtp.mailfrom=pburton@wavecomp.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.23.0 x-originating-ip: [12.94.197.246] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: c58e5c79-619e-48ae-a209-08d746b9d036 x-ms-traffictypediagnostic: MWHPR2201MB1581: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:3631; x-forefront-prvs: 0177904E6B x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(136003)(39850400004)(366004)(346002)(396003)(376002)(189003)(199004)(8936002)(26005)(478600001)(6512007)(8676002)(81166006)(81156014)(6916009)(386003)(76176011)(186003)(6506007)(25786009)(52116002)(6436002)(99286004)(6486002)(316002)(102836004)(54906003)(14454004)(5640700003)(2501003)(486006)(66476007)(7736002)(256004)(14444005)(66556008)(66946007)(1076003)(64756008)(36756003)(107886003)(66066001)(66446008)(4326008)(42882007)(2906002)(3846002)(6116002)(476003)(446003)(2616005)(11346002)(2351001)(71190400001)(44832011)(5660300002)(50226002)(305945005)(71200400001)(133343001);DIR:OUT;SFP:1102;SCL:1;SRVR:MWHPR2201MB1581;H:MWHPR2201MB1277.namprd22.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;MX:1;A:1; received-spf: None (protection.outlook.com: wavecomp.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: uokMrjLGcZqhtqEk6EJEbDfx9NCSvdfxmqECgNOimUkSN5cghJ7dQmVbnC3oIeu9TdSt2YjARW9I5Fnsi9hPpEmVBan2BK0ccw7R6foRg7QTFIYpjgBMQRtG/HRTa+c2DRaDEteqxsTnl3gErCkOi6JniZx9nMaW1S4gXXrly8RjY3RToJQF0cs2D8xnEsefJv0nNdOEk8XCFjfjpRacravcZwU5owj4D0wiawrDYJBwTT0L4VXHask3vjq+8VJ2IAL5p096fpMZir/hRgKQ8iEZAN5RRols8cZeS/f1c/8xxQPR1XI10aXSsN+bIZEuIBThVCEwQdR0oxhbIZfcpJNk4STWFAydJ8vGuSMbGHvPltNmR4kXG7ap99Z/wc3J3QpXTq2MJlUzyaQHaKAfajUSEYlHVhIcumrs6TLmmZA= MIME-Version: 1.0 X-OriginatorOrg: mips.com X-MS-Exchange-CrossTenant-Network-Message-Id: c58e5c79-619e-48ae-a209-08d746b9d036 X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Oct 2019 21:53:37.3819 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 463607d3-1db3-40a0-8a29-970c56230104 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: UnD2oHWrKwYRPLzL/3aF1jbfGp/UXvvxfSkbnCFpVCmMC4CSSkZQvY06VoYJN2ERI0csphibNqxw8WYnSh8/Fw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR2201MB1581 Sender: linux-mips-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-mips@vger.kernel.org Generate the sync instructions required to workaround Loongson3 LL/SC errata within inline asm blocks, which feels a little safer than doing it from C where strictly speaking the compiler would be well within its rights to insert a memory access between the separate asm statements we previously had, containing sync & ll instructions respectively. Signed-off-by: Paul Burton --- Changes in v2: None arch/mips/include/asm/cmpxchg.h | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/arch/mips/include/asm/cmpxchg.h b/arch/mips/include/asm/cmpxchg.h index 5d3f0e3513b4..fc121d20a980 100644 --- a/arch/mips/include/asm/cmpxchg.h +++ b/arch/mips/include/asm/cmpxchg.h @@ -12,6 +12,7 @@ #include #include #include +#include #include /* @@ -36,12 +37,12 @@ extern unsigned long __xchg_called_with_bad_pointer(void) __typeof(*(m)) __ret; \ \ if (kernel_uses_llsc) { \ - loongson_llsc_mb(); \ __asm__ __volatile__( \ " .set push \n" \ " .set noat \n" \ " .set push \n" \ " .set " MIPS_ISA_ARCH_LEVEL " \n" \ + " " __SYNC(full, loongson3_war) " \n" \ "1: " ld " %0, %2 # __xchg_asm \n" \ " .set pop \n" \ " move $1, %z3 \n" \ @@ -108,12 +109,12 @@ static inline unsigned long __xchg(volatile void *ptr, unsigned long x, __typeof(*(m)) __ret; \ \ if (kernel_uses_llsc) { \ - loongson_llsc_mb(); \ __asm__ __volatile__( \ " .set push \n" \ " .set noat \n" \ " .set push \n" \ " .set "MIPS_ISA_ARCH_LEVEL" \n" \ + " " __SYNC(full, loongson3_war) " \n" \ "1: " ld " %0, %2 # __cmpxchg_asm \n" \ " bne %0, %z3, 2f \n" \ " .set pop \n" \ @@ -122,11 +123,10 @@ static inline unsigned long __xchg(volatile void *ptr, unsigned long x, " " st " $1, %1 \n" \ "\t" __SC_BEQZ "$1, 1b \n" \ " .set pop \n" \ - "2: \n" \ + "2: " __SYNC(full, loongson3_war) " \n" \ : "=&r" (__ret), "=" GCC_OFF_SMALL_ASM() (*m) \ : GCC_OFF_SMALL_ASM() (*m), "Jr" (old), "Jr" (new) \ : __LLSC_CLOBBER); \ - loongson_llsc_mb(); \ } else { \ unsigned long __flags; \ \ @@ -222,11 +222,11 @@ static inline unsigned long __cmpxchg64(volatile void *ptr, */ local_irq_save(flags); - loongson_llsc_mb(); asm volatile( " .set push \n" " .set " MIPS_ISA_ARCH_LEVEL " \n" /* Load 64 bits from ptr */ + " " __SYNC(full, loongson3_war) " \n" "1: lld %L0, %3 # __cmpxchg64 \n" /* * Split the 64 bit value we loaded into the 2 registers that hold the @@ -260,7 +260,7 @@ static inline unsigned long __cmpxchg64(volatile void *ptr, /* If we failed, loop! */ "\t" __SC_BEQZ "%L1, 1b \n" " .set pop \n" - "2: \n" + "2: " __SYNC(full, loongson3_war) " \n" : "=&r"(ret), "=&r"(tmp), "=" GCC_OFF_SMALL_ASM() (*(unsigned long long *)ptr) @@ -268,7 +268,6 @@ static inline unsigned long __cmpxchg64(volatile void *ptr, "r" (old), "r" (new) : "memory"); - loongson_llsc_mb(); local_irq_restore(flags); return ret; From patchwork Tue Oct 1 21:53:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Burton X-Patchwork-Id: 11169793 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0215513B1 for ; Tue, 1 Oct 2019 21:53:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id CA67C2133F for ; Tue, 1 Oct 2019 21:53:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=wavecomp.com header.i=@wavecomp.com header.b="LJdoGSNl" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728257AbfJAVxr (ORCPT ); Tue, 1 Oct 2019 17:53:47 -0400 Received: from mail-eopbgr750107.outbound.protection.outlook.com ([40.107.75.107]:29167 "EHLO NAM02-BL2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728240AbfJAVxq (ORCPT ); Tue, 1 Oct 2019 17:53:46 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Frz/9ntv4VE04FuzGA9mmRhK6HhFpja33WvQN4SyY4JLrafKVtsjlGCING4STKaY6IOccs/y3HHc7gXxZaNhAry8eqyPrVNZjPZYqGwaMTFKBBTQ5N0qo4pnGfpTzAxDb3DEzglAs3u/YhLxGFXox/xf3SMzLHD6z/24if53t5cFGzmSBT0HtOdbVr6hW29HN7V4KE4EyyAHrFKSSIVxlx4STrIeJQDL3NhJGj7gbEDlCsvHPfCdDM2wvpx5Qgwit/3mr4HDj3P8BMW5Y1ZgZzLDdZ4V1MMAF5vUPxMjWc5z5p4EFIxgvGplyPhJ9kltMipReeaLn8aCbaZekSNu7w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=A9dOVhBTnEotVmDabnalZVGySzNoS0tZ4vxHpl0prAg=; b=VgfyZSsx641Gv92bRrDpFTsmkLf17RlT2DX+fKrVV/YwFnDkFNkF5cXXsMAPBGFZAWZi07dvwVpwA2GLqFrfZPy1neUOx9tUyU59y1Nie46fgonHHQW2Inb+qJpP426ZGpFwg6Evn6BCx/+Oe4Y3mpauyMjJDKBxEAH9EzPeEOmAvnKZzP5mQy5cCMxyw6ZfFKgsyNa0iWQR61js23WQW5jV1xJ26U1VKs5wEuEhW9DA+IvBnubVXTY5f8bys6b5uXzLK+Snx2mNXSkqTr1iuGnsc2BDgBkyyqLNfdEH3Bd833dO67aMpo6l7HX6/rMQ3+VYNfrn9Two5Bk8UQ3G6w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wavecomp.com; dmarc=pass action=none header.from=mips.com; dkim=pass header.d=mips.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wavecomp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=A9dOVhBTnEotVmDabnalZVGySzNoS0tZ4vxHpl0prAg=; b=LJdoGSNlu/J1cDF0+q5c3xMX/a/9mN+k1Mvhe+U11x7bS2JZ7tFt1qJloyMZlEBVV5Gpl+vRHZVuzD+PuWB7zXGreUVIP70JADVLaHQWbRT1oySU1H4qln+Adgx2yhG0kvVw/cOOQsXtJcuz6FRSuqgocIyBpdc+fin1ua2+c6I= Received: from MWHPR2201MB1277.namprd22.prod.outlook.com (10.172.60.12) by MWHPR2201MB1581.namprd22.prod.outlook.com (10.174.167.30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2305.15; Tue, 1 Oct 2019 21:53:38 +0000 Received: from MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481]) by MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481%2]) with mapi id 15.20.2305.022; Tue, 1 Oct 2019 21:53:38 +0000 From: Paul Burton To: "linux-mips@vger.kernel.org" CC: Huacai Chen , Jiaxun Yang , "linux-kernel@vger.kernel.org" , Paul Burton Subject: [PATCH v2 29/36] MIPS: cmpxchg: Omit redundant barriers for Loongson3 Thread-Topic: [PATCH v2 29/36] MIPS: cmpxchg: Omit redundant barriers for Loongson3 Thread-Index: AQHVeKKu2dehK6fuHUiwbbrrNcwXQQ== Date: Tue, 1 Oct 2019 21:53:38 +0000 Message-ID: <20191001215249.4157062-30-paul.burton@mips.com> References: <20191001215249.4157062-1-paul.burton@mips.com> In-Reply-To: <20191001215249.4157062-1-paul.burton@mips.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BYAPR07CA0093.namprd07.prod.outlook.com (2603:10b6:a03:12b::34) To MWHPR2201MB1277.namprd22.prod.outlook.com (2603:10b6:301:18::12) authentication-results: spf=none (sender IP is ) smtp.mailfrom=pburton@wavecomp.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.23.0 x-originating-ip: [12.94.197.246] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 6def6961-48d1-434a-3951-08d746b9d0ac x-ms-traffictypediagnostic: MWHPR2201MB1581: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:7219; x-forefront-prvs: 0177904E6B x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(136003)(39850400004)(366004)(346002)(396003)(376002)(189003)(199004)(8936002)(26005)(478600001)(6512007)(8676002)(81166006)(81156014)(6916009)(386003)(76176011)(186003)(6506007)(25786009)(52116002)(6436002)(99286004)(6486002)(316002)(102836004)(54906003)(14454004)(5640700003)(2501003)(486006)(66476007)(7736002)(256004)(66556008)(66946007)(1076003)(64756008)(36756003)(107886003)(66066001)(66446008)(4326008)(42882007)(2906002)(3846002)(6116002)(476003)(446003)(2616005)(11346002)(2351001)(71190400001)(44832011)(5660300002)(50226002)(305945005)(71200400001)(133343001);DIR:OUT;SFP:1102;SCL:1;SRVR:MWHPR2201MB1581;H:MWHPR2201MB1277.namprd22.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;MX:1;A:1; received-spf: None (protection.outlook.com: wavecomp.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: Pape8/43qeLdaKIgHbl0tp2QJFgt5oMix2kbxrYN7fbNuIu1503DxCD4FHkNrjJeys5hJE3bLlpGOK4nfdrsSW1vmV1lmeZ8eqdA0PghzkWbozyF2WmWO4GR1qdm2uz52RLHDJW2H56eVMfmmVcxJubhtm4wr+z4jew+NIhV8SYplqddlsOArXhAlxHLEy+UTiHdHFUulqj4I/1Xvf2VX2FEX2pkYukfq5I2tWJtoixvFeMIz+vQ28wvy4aDCo0ePyYw2cU56kcc1xIMJXIkU9FQs0gLCekJaqBvts5PD44JApqlzfXdimUBYfXn7pY13DNWAf7QT8C+TnmO52vhixadpCD3BS67n074219fIAO5DbEzg40aKGGs6NdP9dWIecjzCqGV/EKyBfgDLJeE6970D9qXdu0Hb6NA9ywOyPQ= MIME-Version: 1.0 X-OriginatorOrg: mips.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6def6961-48d1-434a-3951-08d746b9d0ac X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Oct 2019 21:53:38.1375 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 463607d3-1db3-40a0-8a29-970c56230104 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: wV4dJa3CvVECHtkp8LQpIycJ1DnnUA/AgRiEjPfP5pOb9XfuwHmIwLtq1HlabnYJZ69oZN+CKfQrQiuPyOmUCQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR2201MB1581 Sender: linux-mips-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-mips@vger.kernel.org When building a kernel configured to support Loongson3 LL/SC workarounds (ie. CONFIG_CPU_LOONGSON3_WORKAROUNDS=y) the inline assembly in __xchg_asm() & __cmpxchg_asm() already emits completion barriers, and as such we don't need to emit extra barriers from the xchg() or cmpxchg() macros. Add compile-time constant checks causing us to omit the redundant memory barriers. Signed-off-by: Paul Burton --- Changes in v2: None arch/mips/include/asm/cmpxchg.h | 26 +++++++++++++++++++++++--- 1 file changed, 23 insertions(+), 3 deletions(-) diff --git a/arch/mips/include/asm/cmpxchg.h b/arch/mips/include/asm/cmpxchg.h index fc121d20a980..820df68e32e1 100644 --- a/arch/mips/include/asm/cmpxchg.h +++ b/arch/mips/include/asm/cmpxchg.h @@ -94,7 +94,13 @@ static inline unsigned long __xchg(volatile void *ptr, unsigned long x, ({ \ __typeof__(*(ptr)) __res; \ \ - smp_mb__before_llsc(); \ + /* \ + * In the Loongson3 workaround case __xchg_asm() already \ + * contains a completion barrier prior to the LL, so we don't \ + * need to emit an extra one here. \ + */ \ + if (!__SYNC_loongson3_war) \ + smp_mb__before_llsc(); \ \ __res = (__typeof__(*(ptr))) \ __xchg((ptr), (unsigned long)(x), sizeof(*(ptr))); \ @@ -179,9 +185,23 @@ static inline unsigned long __cmpxchg(volatile void *ptr, unsigned long old, ({ \ __typeof__(*(ptr)) __res; \ \ - smp_mb__before_llsc(); \ + /* \ + * In the Loongson3 workaround case __cmpxchg_asm() already \ + * contains a completion barrier prior to the LL, so we don't \ + * need to emit an extra one here. \ + */ \ + if (!__SYNC_loongson3_war) \ + smp_mb__before_llsc(); \ + \ __res = cmpxchg_local((ptr), (old), (new)); \ - smp_llsc_mb(); \ + \ + /* \ + * In the Loongson3 workaround case __cmpxchg_asm() already \ + * contains a completion barrier after the SC, so we don't \ + * need to emit an extra one here. \ + */ \ + if (!__SYNC_loongson3_war) \ + smp_llsc_mb(); \ \ __res; \ }) From patchwork Tue Oct 1 21:53:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Burton X-Patchwork-Id: 11169817 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CA9581709 for ; Tue, 1 Oct 2019 21:54:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 9ED9221A4A for ; Tue, 1 Oct 2019 21:54:53 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=wavecomp.com header.i=@wavecomp.com header.b="WAeikLFL" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728390AbfJAVyR (ORCPT ); Tue, 1 Oct 2019 17:54:17 -0400 Received: from mail-eopbgr750119.outbound.protection.outlook.com ([40.107.75.119]:27367 "EHLO NAM02-BL2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728374AbfJAVyP (ORCPT ); Tue, 1 Oct 2019 17:54:15 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=N3bvrSnV/nzmSbMJfNyJHhwS/82TCs8xRyUlun3CoH6h2KZYwU+mhbdPFQ8RWN7m+w/8qIDvccgRh5kZgLHp9eH1LbZMoGcUUCz+unGKgqMgjsJuzLL2JpG3R+q8BXanwcbRTActJumPEPblLxnsyCGTSwHN+ztFiBeEKBLS1E8Uf/eDGl8jxixzFH4LAQ8A6yI+g6O1IEHnKBG74z2wfjtk2rWi1JfZTc9yJOxewW93czsa6XOY9vTvVrGbUKCOjrjcogKTtJXFQmnik61W9/mxkhV+5hYHxZGw85pZCJREzxvRilhB1f+54TSmeu+Qyj21K9XOLRKVTmYEUPPfVg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BvKp4k19ls4+6tnMRA/6GTdZbR5v9xxSyD4Fo6DCAgc=; b=oXqW67f4GRW6SE4XbVxKLyiPPkevVmd5lar/z7HJs6OV8tqCekXFu4tOdqSRl9uk/d9bLfcXa47CoDUZGouXhalnvs96xCPwgb7h82GHGR1l3gqYUhnYY4N2B0aOW+tAvwXGiUmqMOEUlDn41fNdParnbsB+zfmcMsxwkcq89hw5fVdN6s0GukRopl+JZmhsfY3JXxoPwdmFgzqTbvzEJcu7/IZ1TFpaJ/Hd9kV6R5RQ/CuKXazqYbikIQOrD0/lH3chNQAAXu9lGFF36U9KRTcMPr6iBTg06j8wDWxN4clkaG0xRxhLfdOxEN8jXP0sQifTrll3tcwuQygnjckXlg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wavecomp.com; dmarc=pass action=none header.from=mips.com; dkim=pass header.d=mips.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wavecomp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BvKp4k19ls4+6tnMRA/6GTdZbR5v9xxSyD4Fo6DCAgc=; b=WAeikLFLynD7GhQWiTh4BRAbnQzLSUWyMF38WiECfwSqikhP2UwSm6MzIV6H1m1b5tHVUNibl+e1pyoaYBajbaKWR77QXRSKHPLmVxJT8590WhRkmWyq8tuHlAasaV0B7xaUcFY8ram2F7zY2nyY8f9m+2SGSXdrrnnuIYk0Hok= Received: from MWHPR2201MB1277.namprd22.prod.outlook.com (10.172.60.12) by MWHPR2201MB1581.namprd22.prod.outlook.com (10.174.167.30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2305.15; Tue, 1 Oct 2019 21:54:09 +0000 Received: from MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481]) by MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481%2]) with mapi id 15.20.2305.022; Tue, 1 Oct 2019 21:54:09 +0000 From: Paul Burton To: "linux-mips@vger.kernel.org" CC: Huacai Chen , Jiaxun Yang , "linux-kernel@vger.kernel.org" , Paul Burton Subject: [PATCH v2 30/36] MIPS: futex: Emit Loongson3 sync workarounds within asm Thread-Topic: [PATCH v2 30/36] MIPS: futex: Emit Loongson3 sync workarounds within asm Thread-Index: AQHVeKKu1sY4yaAX30mGgrwia3pl8g== Date: Tue, 1 Oct 2019 21:53:38 +0000 Message-ID: <20191001215249.4157062-31-paul.burton@mips.com> References: <20191001215249.4157062-1-paul.burton@mips.com> In-Reply-To: <20191001215249.4157062-1-paul.burton@mips.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BYAPR07CA0093.namprd07.prod.outlook.com (2603:10b6:a03:12b::34) To MWHPR2201MB1277.namprd22.prod.outlook.com (2603:10b6:301:18::12) authentication-results: spf=none (sender IP is ) smtp.mailfrom=pburton@wavecomp.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.23.0 x-originating-ip: [12.94.197.246] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 46063039-e7f3-4061-08fe-08d746b9d123 x-ms-traffictypediagnostic: MWHPR2201MB1581: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:2449; x-forefront-prvs: 0177904E6B x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(136003)(39850400004)(366004)(346002)(396003)(376002)(189003)(199004)(8936002)(26005)(478600001)(6512007)(8676002)(81166006)(81156014)(6916009)(386003)(76176011)(186003)(6506007)(25786009)(52116002)(6436002)(99286004)(6486002)(316002)(102836004)(54906003)(14454004)(5640700003)(2501003)(486006)(66476007)(7736002)(256004)(14444005)(66556008)(66946007)(1076003)(64756008)(36756003)(107886003)(66066001)(66446008)(4326008)(42882007)(2906002)(6666004)(3846002)(6116002)(476003)(446003)(2616005)(11346002)(2351001)(71190400001)(44832011)(5660300002)(50226002)(305945005)(71200400001);DIR:OUT;SFP:1102;SCL:1;SRVR:MWHPR2201MB1581;H:MWHPR2201MB1277.namprd22.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;MX:1;A:1; received-spf: None (protection.outlook.com: wavecomp.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: pSWApVL7msZAe8xVsRPOignrSKZbcKO9lxKgKC5ohxE4VemMM+EpvfrMJrOfxWHcFuPm0mgM4C6UCpTGsdnJdfGVCk3Ofw+MnBbkGCJ8Yuv+bVuAAW48silcTHQNzI8VmcCsatZFj8sc2kYs19ngmIz0b5DaUwy1kTaSu9hEGigkJSCJeIdmYu2ZOa4EgPTGNlq183cp1e5gRNI+e6L/hduVR9FfJGVW+xtrGuICBe3MUmobyG9+iaoo/JjpQeNmUFGqX5oDStjKD8LkSjXvI3fEdYEJ0KM/tS3QKiE1nOwNopKtK/DbR/JUU4p1sRNJgDmWYaEtbEIYL0qKL+7aekpzYfMVz6PiYatm5vcQ/xHGWX+9s5IXC1rL3dcBk/SvAzwYE0OVBaTVMBULDs+RRPFewlpc2e2PSHVhhN8vRPQ= MIME-Version: 1.0 X-OriginatorOrg: mips.com X-MS-Exchange-CrossTenant-Network-Message-Id: 46063039-e7f3-4061-08fe-08d746b9d123 X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Oct 2019 21:53:39.0201 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 463607d3-1db3-40a0-8a29-970c56230104 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: /CHeDW8NAzx90tYYcwEXctd8NA7Tl12/6PFJ6VL9z65Vic/32mCyM2ygEd+pQWHeH9MMO7/mPMKCuotGCqPVSw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR2201MB1581 Sender: linux-mips-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-mips@vger.kernel.org Generate the sync instructions required to workaround Loongson3 LL/SC errata within inline asm blocks, which feels a little safer than doing it from C where strictly speaking the compiler would be well within its rights to insert a memory access between the separate asm statements we previously had, containing sync & ll instructions respectively. Signed-off-by: Paul Burton --- Changes in v2: - De-string __WEAK_LLSC_MB to allow its use with __SYNC_ELSE(). arch/mips/include/asm/barrier.h | 13 +++++++------ arch/mips/include/asm/futex.h | 15 +++++++-------- 2 files changed, 14 insertions(+), 14 deletions(-) diff --git a/arch/mips/include/asm/barrier.h b/arch/mips/include/asm/barrier.h index c7e05e832da9..133afd565067 100644 --- a/arch/mips/include/asm/barrier.h +++ b/arch/mips/include/asm/barrier.h @@ -95,13 +95,14 @@ static inline void wmb(void) * ordering will be done by smp_llsc_mb() and friends. */ #if defined(CONFIG_WEAK_REORDERING_BEYOND_LLSC) && defined(CONFIG_SMP) -#define __WEAK_LLSC_MB " sync \n" -#define smp_llsc_mb() __asm__ __volatile__(__WEAK_LLSC_MB : : :"memory") -#define __LLSC_CLOBBER +# define __WEAK_LLSC_MB sync +# define smp_llsc_mb() \ + __asm__ __volatile__(__stringify(__WEAK_LLSC_MB) : : :"memory") +# define __LLSC_CLOBBER #else -#define __WEAK_LLSC_MB " \n" -#define smp_llsc_mb() do { } while (0) -#define __LLSC_CLOBBER "memory" +# define __WEAK_LLSC_MB +# define smp_llsc_mb() do { } while (0) +# define __LLSC_CLOBBER "memory" #endif #ifdef CONFIG_CPU_CAVIUM_OCTEON diff --git a/arch/mips/include/asm/futex.h b/arch/mips/include/asm/futex.h index b83b0397462d..54cf20530931 100644 --- a/arch/mips/include/asm/futex.h +++ b/arch/mips/include/asm/futex.h @@ -16,6 +16,7 @@ #include #include #include +#include #include #define __futex_atomic_op(insn, ret, oldval, uaddr, oparg) \ @@ -32,7 +33,7 @@ " .set arch=r4000 \n" \ "2: sc $1, %2 \n" \ " beqzl $1, 1b \n" \ - __WEAK_LLSC_MB \ + __stringify(__WEAK_LLSC_MB) \ "3: \n" \ " .insn \n" \ " .set pop \n" \ @@ -50,19 +51,19 @@ "i" (-EFAULT) \ : "memory"); \ } else if (cpu_has_llsc) { \ - loongson_llsc_mb(); \ __asm__ __volatile__( \ " .set push \n" \ " .set noat \n" \ " .set push \n" \ " .set "MIPS_ISA_ARCH_LEVEL" \n" \ + " " __SYNC(full, loongson3_war) " \n" \ "1: "user_ll("%1", "%4")" # __futex_atomic_op\n" \ " .set pop \n" \ " " insn " \n" \ " .set "MIPS_ISA_ARCH_LEVEL" \n" \ "2: "user_sc("$1", "%2")" \n" \ " beqz $1, 1b \n" \ - __WEAK_LLSC_MB \ + __stringify(__WEAK_LLSC_MB) \ "3: \n" \ " .insn \n" \ " .set pop \n" \ @@ -147,7 +148,7 @@ futex_atomic_cmpxchg_inatomic(u32 *uval, u32 __user *uaddr, " .set arch=r4000 \n" "2: sc $1, %2 \n" " beqzl $1, 1b \n" - __WEAK_LLSC_MB + __stringify(__WEAK_LLSC_MB) "3: \n" " .insn \n" " .set pop \n" @@ -164,13 +165,13 @@ futex_atomic_cmpxchg_inatomic(u32 *uval, u32 __user *uaddr, "i" (-EFAULT) : "memory"); } else if (cpu_has_llsc) { - loongson_llsc_mb(); __asm__ __volatile__( "# futex_atomic_cmpxchg_inatomic \n" " .set push \n" " .set noat \n" " .set push \n" " .set "MIPS_ISA_ARCH_LEVEL" \n" + " " __SYNC(full, loongson3_war) " \n" "1: "user_ll("%1", "%3")" \n" " bne %1, %z4, 3f \n" " .set pop \n" @@ -178,8 +179,7 @@ futex_atomic_cmpxchg_inatomic(u32 *uval, u32 __user *uaddr, " .set "MIPS_ISA_ARCH_LEVEL" \n" "2: "user_sc("$1", "%2")" \n" " beqz $1, 1b \n" - __WEAK_LLSC_MB - "3: \n" + "3: " __SYNC_ELSE(full, loongson3_war, __WEAK_LLSC_MB) "\n" " .insn \n" " .set pop \n" " .section .fixup,\"ax\" \n" @@ -194,7 +194,6 @@ futex_atomic_cmpxchg_inatomic(u32 *uval, u32 __user *uaddr, : GCC_OFF_SMALL_ASM() (*uaddr), "Jr" (oldval), "Jr" (newval), "i" (-EFAULT) : "memory"); - loongson_llsc_mb(); } else return -ENOSYS; From patchwork Tue Oct 1 21:53:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Burton X-Patchwork-Id: 11169803 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B4C8B13B1 for ; Tue, 1 Oct 2019 21:54:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 8990E21924 for ; Tue, 1 Oct 2019 21:54:18 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=wavecomp.com header.i=@wavecomp.com header.b="SLBwptn5" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728393AbfJAVyS (ORCPT ); Tue, 1 Oct 2019 17:54:18 -0400 Received: from mail-eopbgr750119.outbound.protection.outlook.com ([40.107.75.119]:27367 "EHLO NAM02-BL2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728385AbfJAVyQ (ORCPT ); Tue, 1 Oct 2019 17:54:16 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=e0IjH+BPGM8yZ6kiQcoVYoBgh2Ik0LGkGtnMvxHVZtNBE6Nn5IYpGM+JoWfwCJuemKe1VXhKvo7TD75l35AhsM50HNZ8cZXhdpRbIIy+75mB4UFM/Hrx0RTDtzE4ofjUxODV7fYgwiXrVUS5teK0/zOYSssmFi/2q4mRPyb05I8vFjIYeTbbb9sW48DlnFrUwVlmgVPYCVTvzJ29YdHpY0H4I/8z5I1z8+wRxrpDjhEouvRzgvFMLL4p7CbVqQDXHaXBe+VX5DFNCPzGsRloroED1q2vPCAQwHNDBj87CiPt6gyIAzqJWExXf7FJihh8xX37vRqknBb2XMe0k4uOHA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=bm7wZ1vEmtJG3B5kaj+MwIDQ5nFxyg3K/vPx+nBAnUY=; b=LnSDGrmchIRz2d5VDFsIaqGobGJfllvYMofzKyr5Dig6uu6ss4+3QtRac+T471zHB5bH/Ve30ZRfAyNGcVN9q6pQU0BxtePyI8VH9bosRxPRvUiSWhkNL6k/KS+idqVWcSOvreI3LuFP2Xdcj1uEri4RZaAVYcmn+IPlWp3E9uS/4TTcG5XwnQMF+rYf8rxTjgcxQeAmogw9Y/yLPMxmIf0FvLJlct6sJMPP2rLlVVwwCykjbbBL15qEqJhPvLcUGJlenrSg0U1j42TwkeM05SbPDHAR0FrgDVOb32esQWM5bT1peaXlyc/dH+sklNgO7yFlk8ueGxyLuY1OFmQWWg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wavecomp.com; dmarc=pass action=none header.from=mips.com; dkim=pass header.d=mips.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wavecomp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=bm7wZ1vEmtJG3B5kaj+MwIDQ5nFxyg3K/vPx+nBAnUY=; b=SLBwptn5SdnRwV/kDFBZPDNDbWiCcfwEAgJDoSq9BNGp2MO5jkeZrFKpbDCVu3b9nayqTtTnNmqj4atiZmVaot9jO+hyuLuQBgskB2m1XN2H/uKCetRzwWnpWJN0DE4SkoCtS5P37IIVMBdiEG3to835SFK/PMCl5sPqSpDyT78= Received: from MWHPR2201MB1277.namprd22.prod.outlook.com (10.172.60.12) by MWHPR2201MB1581.namprd22.prod.outlook.com (10.174.167.30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2305.15; Tue, 1 Oct 2019 21:54:13 +0000 Received: from MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481]) by MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481%2]) with mapi id 15.20.2305.022; Tue, 1 Oct 2019 21:54:13 +0000 From: Paul Burton To: "linux-mips@vger.kernel.org" CC: Huacai Chen , Jiaxun Yang , "linux-kernel@vger.kernel.org" , Paul Burton Subject: [PATCH v2 31/36] MIPS: syscall: Emit Loongson3 sync workarounds within asm Thread-Topic: [PATCH v2 31/36] MIPS: syscall: Emit Loongson3 sync workarounds within asm Thread-Index: AQHVeKKvagu7Xn/Sc0OasjlKTL7UqA== Date: Tue, 1 Oct 2019 21:53:39 +0000 Message-ID: <20191001215249.4157062-32-paul.burton@mips.com> References: <20191001215249.4157062-1-paul.burton@mips.com> In-Reply-To: <20191001215249.4157062-1-paul.burton@mips.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BYAPR07CA0093.namprd07.prod.outlook.com (2603:10b6:a03:12b::34) To MWHPR2201MB1277.namprd22.prod.outlook.com (2603:10b6:301:18::12) authentication-results: spf=none (sender IP is ) smtp.mailfrom=pburton@wavecomp.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.23.0 x-originating-ip: [12.94.197.246] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: a1f7219c-b493-435e-07c7-08d746b9d1a8 x-ms-traffictypediagnostic: MWHPR2201MB1581: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:6108; x-forefront-prvs: 0177904E6B x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(136003)(39850400004)(366004)(346002)(396003)(376002)(189003)(199004)(8936002)(26005)(478600001)(6512007)(8676002)(81166006)(81156014)(6916009)(386003)(76176011)(186003)(6506007)(25786009)(52116002)(6436002)(99286004)(6486002)(316002)(102836004)(54906003)(14454004)(5640700003)(2501003)(486006)(66476007)(7736002)(256004)(14444005)(66556008)(66946007)(1076003)(64756008)(36756003)(107886003)(66066001)(66446008)(4326008)(42882007)(2906002)(6666004)(3846002)(6116002)(476003)(446003)(2616005)(11346002)(2351001)(71190400001)(44832011)(5660300002)(50226002)(305945005)(71200400001);DIR:OUT;SFP:1102;SCL:1;SRVR:MWHPR2201MB1581;H:MWHPR2201MB1277.namprd22.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;MX:1;A:1; received-spf: None (protection.outlook.com: wavecomp.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 3F11ALBr7yTQ0oLvEluNaCWmUoK+jnAPI5RwXJWun0s2hhKyCx1/E06dRXvMgb8x9KvVKFACHkkDzSznmJrNqyYD9PRXSDKd0CC/ExOZu0YQ4z7g5PQ9BITdM/eDFikH9KEfShIAsUFTl78XQrjq7OxUkGzD2FKfSSFDtS0FMXAmoWXi6XR6xXw/kIF6v6JUBSGxOeorM6sU4bYjIdgCuspI/jKs3TFJbBE/pmX5hGPjWbu8LsnZvlmV4bO1xbhu2G5Ckc2oRdA8dCV2uktn4GayH9DzJxAsq8hCE4f1/AX4yX3vMWdEvHgp0cYAApW/vePWLjhXyvc8aFLR3NXhjwImqLdlRaDeZ7O33KdjjZ88gU+acnjUs/vmgRotteQeZBzMwqiT04k3jCCoffJ94wPvpzQPzsrswDqU+clH4O0= MIME-Version: 1.0 X-OriginatorOrg: mips.com X-MS-Exchange-CrossTenant-Network-Message-Id: a1f7219c-b493-435e-07c7-08d746b9d1a8 X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Oct 2019 21:53:39.8347 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 463607d3-1db3-40a0-8a29-970c56230104 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: rZD8yQWWFXdDhlxnSVfHNpPZVXxDFifzhTzrE9N2m2vRvc+q3tRjLV5H7C19SFRGkrzlSZCoW7K6KyORpUsmqg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR2201MB1581 Sender: linux-mips-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-mips@vger.kernel.org Generate the sync instructions required to workaround Loongson3 LL/SC errata within inline asm blocks, which feels a little safer than doing it from C where strictly speaking the compiler would be well within its rights to insert a memory access between the separate asm statements we previously had, containing sync & ll instructions respectively. Signed-off-by: Paul Burton --- Changes in v2: None arch/mips/kernel/syscall.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/mips/kernel/syscall.c b/arch/mips/kernel/syscall.c index b0e25e913bdb..3ea288ca35f1 100644 --- a/arch/mips/kernel/syscall.c +++ b/arch/mips/kernel/syscall.c @@ -37,6 +37,7 @@ #include #include #include +#include #include #include @@ -132,12 +133,12 @@ static inline int mips_atomic_set(unsigned long addr, unsigned long new) [efault] "i" (-EFAULT) : "memory"); } else if (cpu_has_llsc) { - loongson_llsc_mb(); __asm__ __volatile__ ( " .set push \n" " .set "MIPS_ISA_ARCH_LEVEL" \n" " li %[err], 0 \n" "1: \n" + " " __SYNC(full, loongson3_war) " \n" user_ll("%[old]", "(%[addr])") " move %[tmp], %[new] \n" "2: \n" From patchwork Tue Oct 1 21:53:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Burton X-Patchwork-Id: 11169815 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3C52813B1 for ; Tue, 1 Oct 2019 21:54:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 0FC3A21924 for ; Tue, 1 Oct 2019 21:54:53 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=wavecomp.com header.i=@wavecomp.com header.b="LPNzDjm2" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727306AbfJAVys (ORCPT ); Tue, 1 Oct 2019 17:54:48 -0400 Received: from mail-eopbgr750119.outbound.protection.outlook.com ([40.107.75.119]:27367 "EHLO NAM02-BL2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728228AbfJAVyS (ORCPT ); Tue, 1 Oct 2019 17:54:18 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ItEcVh/Rqqr2fr6Ob0XvUWplq8StD4J4iAixgVEJ8e1DXsCGhxsVvV9uPBF/1Mun4ixBth0L7++teKFf1enNdtqMTCd90CzEviC/cB1d0/bMuTR1vCAuut1U6jR9LqVenkcfHFXn35W43RGMXcjTknbtqikFf9uihkD9xeFuizznJYd2Z2bV3qW0xkm2R9IPn3DPBPj0sgeXDJZw4DqkYs9gNZjEozJpC40oPsIf0JiZKtBHM//qvHBzxFSMLets09rq6FBPZ5J/1MU/NkXBf0Y4K0Qi7j43CK78A9gyw/sujh0xONpO2i0B7jTROWGdLqZ1FxdzQx49wbwyY3xK6w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=jsap2fxNVD+tllp6cUv0r68bu6emowL+UFRcAVsZBKc=; b=BVISKLY+88b+JAMMXAfdHv/U2fRqIXURxdCewkTwMXjPvR3oq+dFx7qY21AYx13nK3wmRxV9m2Bi5499kb8BT5vU1cmWSpbgxD+dbSj4CvrGH5Kc4FZcFKIpMp2S4WB9r+bo5n55sPdbhHdP7BZFSxvzSyNEszjH1SXRbN2P5wmXlUM772aJfWr1ILC5eg288PwuvvMFOJogOyTNn12WjcXkCWSYABJp7HfJ1UP3Q7rr2uoLEq6m+x0Wa4QzyH6KQppx14DAdd4ITQZOpglneUYRneiUPyh27d1lTpRiBhVijyVedB3iIcFPoX3VoViZVLSFzja6J9ljSlhffqgTZQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wavecomp.com; dmarc=pass action=none header.from=mips.com; dkim=pass header.d=mips.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wavecomp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=jsap2fxNVD+tllp6cUv0r68bu6emowL+UFRcAVsZBKc=; b=LPNzDjm29DmAQJnGIv15+qxz0//ZabJYQ+GD+KTsx4i0tK+DNcQNyzUnTeZDnvguHiVc3Wdgm0MnexVtItqfIHZGN10GMBplU5axhc9K/ST2FBNmiE0wh1QZu8emvnppqJsdGSvEf6ORp4CLdCk3kRfmIkywS9Wn1mksmRC/0ms= Received: from MWHPR2201MB1277.namprd22.prod.outlook.com (10.172.60.12) by MWHPR2201MB1581.namprd22.prod.outlook.com (10.174.167.30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2305.15; Tue, 1 Oct 2019 21:54:14 +0000 Received: from MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481]) by MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481%2]) with mapi id 15.20.2305.022; Tue, 1 Oct 2019 21:54:14 +0000 From: Paul Burton To: "linux-mips@vger.kernel.org" CC: Huacai Chen , Jiaxun Yang , "linux-kernel@vger.kernel.org" , Paul Burton Subject: [PATCH v2 32/36] MIPS: barrier: Remove loongson_llsc_mb() Thread-Topic: [PATCH v2 32/36] MIPS: barrier: Remove loongson_llsc_mb() Thread-Index: AQHVeKKvI6s8IftE90ee2xJAk8okiw== Date: Tue, 1 Oct 2019 21:53:40 +0000 Message-ID: <20191001215249.4157062-33-paul.burton@mips.com> References: <20191001215249.4157062-1-paul.burton@mips.com> In-Reply-To: <20191001215249.4157062-1-paul.burton@mips.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BYAPR07CA0093.namprd07.prod.outlook.com (2603:10b6:a03:12b::34) To MWHPR2201MB1277.namprd22.prod.outlook.com (2603:10b6:301:18::12) authentication-results: spf=none (sender IP is ) smtp.mailfrom=pburton@wavecomp.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.23.0 x-originating-ip: [12.94.197.246] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 09afb3fd-110d-456f-184a-08d746b9d225 x-ms-traffictypediagnostic: MWHPR2201MB1581: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-forefront-prvs: 0177904E6B x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(136003)(39850400004)(366004)(346002)(396003)(376002)(189003)(199004)(8936002)(26005)(478600001)(6512007)(8676002)(81166006)(81156014)(6916009)(386003)(76176011)(186003)(6506007)(25786009)(52116002)(6436002)(99286004)(6486002)(316002)(102836004)(54906003)(14454004)(5640700003)(2501003)(486006)(66476007)(7736002)(256004)(14444005)(66556008)(66946007)(1076003)(64756008)(36756003)(107886003)(66066001)(66446008)(4326008)(42882007)(2906002)(6666004)(3846002)(6116002)(476003)(446003)(2616005)(11346002)(2351001)(71190400001)(44832011)(5660300002)(50226002)(305945005)(71200400001);DIR:OUT;SFP:1102;SCL:1;SRVR:MWHPR2201MB1581;H:MWHPR2201MB1277.namprd22.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;MX:1;A:1; received-spf: None (protection.outlook.com: wavecomp.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: B+BAVSanta6ZOUd5Vp/ih4ulA+KhIcYMlH46PsilOGLTsI9UcL6yycovvGUksYtP8I/9SrU4i6L3/Fiuzfvc5jcC0zovf5fgV/JXaG1BEfQWeVBcCCgDgoWFKx/Eu5cDQB6Kg4h0vSsjFVpYH5gL1yVZFKsW/2TYNOzqm4ugBv0lcVvZU9OQgjiuZO8b0yy074F10p32CsZsUgXsSxvRvx9De/aH5usip7oNHFjO4aKykun4DUw1LfLJ9wTNjrKfMYzohmyggGFOPVL078gQgwxSJ3HCzVv2fEyMA9tbYqaFedj5sIw2VZxKmx0oCH9WdOF8pQjhwFinmAIdGpYqvkDL0MfONZqOznS54KA1Qsu6VZLh569E7Vt5DN6Ip9EbGcPBAKUc1MVRfpx4gOb/NWO5AZVyRBNDErmp6QQBwZk= MIME-Version: 1.0 X-OriginatorOrg: mips.com X-MS-Exchange-CrossTenant-Network-Message-Id: 09afb3fd-110d-456f-184a-08d746b9d225 X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Oct 2019 21:53:40.7923 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 463607d3-1db3-40a0-8a29-970c56230104 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 6vkulqkgoJHdhKtX7+jMxcnrnslmxtWD0jjTq5SK7xJ5WBjtaF2u429T+kbZu3hakXHOhjq6NayjcqJadC+Avg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR2201MB1581 Sender: linux-mips-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-mips@vger.kernel.org The loongson_llsc_mb() macro is no longer used - instead barriers are emitted as part of inline asm using the __SYNC() macro. Remove the now-defunct loongson_llsc_mb() macro. Signed-off-by: Paul Burton --- Changes in v2: None arch/mips/include/asm/barrier.h | 40 --------------------------------- arch/mips/loongson64/Platform | 2 +- 2 files changed, 1 insertion(+), 41 deletions(-) diff --git a/arch/mips/include/asm/barrier.h b/arch/mips/include/asm/barrier.h index 133afd565067..6d92d5ccdafa 100644 --- a/arch/mips/include/asm/barrier.h +++ b/arch/mips/include/asm/barrier.h @@ -122,46 +122,6 @@ static inline void wmb(void) #define __smp_mb__before_atomic() __smp_mb__before_llsc() #define __smp_mb__after_atomic() smp_llsc_mb() -/* - * Some Loongson 3 CPUs have a bug wherein execution of a memory access (load, - * store or prefetch) in between an LL & SC can cause the SC instruction to - * erroneously succeed, breaking atomicity. Whilst it's unusual to write code - * containing such sequences, this bug bites harder than we might otherwise - * expect due to reordering & speculation: - * - * 1) A memory access appearing prior to the LL in program order may actually - * be executed after the LL - this is the reordering case. - * - * In order to avoid this we need to place a memory barrier (ie. a SYNC - * instruction) prior to every LL instruction, in between it and any earlier - * memory access instructions. - * - * This reordering case is fixed by 3A R2 CPUs, ie. 3A2000 models and later. - * - * 2) If a conditional branch exists between an LL & SC with a target outside - * of the LL-SC loop, for example an exit upon value mismatch in cmpxchg() - * or similar, then misprediction of the branch may allow speculative - * execution of memory accesses from outside of the LL-SC loop. - * - * In order to avoid this we need a memory barrier (ie. a SYNC instruction) - * at each affected branch target, for which we also use loongson_llsc_mb() - * defined below. - * - * This case affects all current Loongson 3 CPUs. - * - * The above described cases cause an error in the cache coherence protocol; - * such that the Invalidate of a competing LL-SC goes 'missing' and SC - * erroneously observes its core still has Exclusive state and lets the SC - * proceed. - * - * Therefore the error only occurs on SMP systems. - */ -#ifdef CONFIG_CPU_LOONGSON3_WORKAROUNDS /* Loongson-3's LLSC workaround */ -#define loongson_llsc_mb() __asm__ __volatile__("sync" : : :"memory") -#else -#define loongson_llsc_mb() do { } while (0) -#endif - static inline void sync_ginv(void) { asm volatile(__SYNC(ginv, always)); diff --git a/arch/mips/loongson64/Platform b/arch/mips/loongson64/Platform index c1a4d4dc4665..28172500f95a 100644 --- a/arch/mips/loongson64/Platform +++ b/arch/mips/loongson64/Platform @@ -27,7 +27,7 @@ cflags-$(CONFIG_CPU_LOONGSON3) += -Wa,--trap # # Some versions of binutils, not currently mainline as of 2019/02/04, support # an -mfix-loongson3-llsc flag which emits a sync prior to each ll instruction -# to work around a CPU bug (see loongson_llsc_mb() in asm/barrier.h for a +# to work around a CPU bug (see __SYNC_loongson3_war in asm/sync.h for a # description). # # We disable this in order to prevent the assembler meddling with the From patchwork Tue Oct 1 21:53:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Burton X-Patchwork-Id: 11169805 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C3E7313B1 for ; Tue, 1 Oct 2019 21:54:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 97EBF21924 for ; Tue, 1 Oct 2019 21:54:23 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=wavecomp.com header.i=@wavecomp.com header.b="EQa5TUCA" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728428AbfJAVyV (ORCPT ); Tue, 1 Oct 2019 17:54:21 -0400 Received: from mail-eopbgr750115.outbound.protection.outlook.com ([40.107.75.115]:30851 "EHLO NAM02-BL2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728374AbfJAVyU (ORCPT ); Tue, 1 Oct 2019 17:54:20 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=nzh4IqhE1htTmSF4CB79RsewnHCreBEpX9YOTQ2U4Jllt5COHLN3d/UMS5e20JmjaOD8lrXK4xReXh2UCE70hmwYELc8vk37LiHThWuvHJ3PFX91DDxtmb+tRvRsu52u8FNWjrsKdjXHVbxVL9yr9FIplj/av2aLWYqbyMgRLXK4si5xbqbj7c+/+7FbvmKStysZzh7+JJ24Crl/bKjyC2M09Iu48F/uYD0U6SLVrHJyLcNRLs0XGZ9O8NB03qzTzdZDl5dze3tyzhCytH/fcWRYzlWfRCzQ+abJh4+6vYBI92QZXzBGxoNRuNi4l3AdrQP5GQn/pFpjnsrwmWwyjA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=sh5vG+SPU2Hm/waIrlSJpgSDR+tZl3iQ1MmEoawt9qU=; b=MXndv7POotaxw2ddgacQa8H8euLldL6urNTd8yZWL9LFdCV3e296xgCUsWf/XMTBc2qlfR+Rqy2OP22OWmulzCc0BGacHjosbETlXDAT/OihYUm+piQSH8AVsXs0HjwsXq93u8EV+SnbVLwUpF/HvYd+xAu2JwMexvVOi87I4v2LhOqXRTyU38J/lFt0ZHvSwapQEX0KAuG0IPSFjQU77mbR3ztWbKR8byLC17N7bJK6v6SDGZjXkcJ2H2HKrJkI27zLbQ4Ci1Qt69zv1qKE6KGFsrJavM1Kk20tH+HpVyJpZnIF9PVkub1qlTo98ao2kc6QWiGqkTH11csBgn/IKQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wavecomp.com; dmarc=pass action=none header.from=mips.com; dkim=pass header.d=mips.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wavecomp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=sh5vG+SPU2Hm/waIrlSJpgSDR+tZl3iQ1MmEoawt9qU=; b=EQa5TUCAuabV+Utiqb4z5qqTeAzyW2YyUuxOVHqqwy8HFwi9qaWwWj7adN9nOW+1NgMzOMfS8KaMga5z+s3olIc+Ynk1f9Ekgqje+lT7ygwGCWqdv3uDAaUSupuoSeIyZO1B0+J/n5pHxo1AEoSDh0QaS1MKtsfHwzLOm72buXE= Received: from MWHPR2201MB1277.namprd22.prod.outlook.com (10.172.60.12) by MWHPR2201MB1581.namprd22.prod.outlook.com (10.174.167.30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2305.15; Tue, 1 Oct 2019 21:54:15 +0000 Received: from MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481]) by MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481%2]) with mapi id 15.20.2305.022; Tue, 1 Oct 2019 21:54:15 +0000 From: Paul Burton To: "linux-mips@vger.kernel.org" CC: Huacai Chen , Jiaxun Yang , "linux-kernel@vger.kernel.org" , Paul Burton Subject: [PATCH v2 33/36] MIPS: barrier: Make __smp_mb__before_atomic() a no-op for Loongson3 Thread-Topic: [PATCH v2 33/36] MIPS: barrier: Make __smp_mb__before_atomic() a no-op for Loongson3 Thread-Index: AQHVeKKwhbfdOsKkPE+4wKLTFI1E5g== Date: Tue, 1 Oct 2019 21:53:41 +0000 Message-ID: <20191001215249.4157062-34-paul.burton@mips.com> References: <20191001215249.4157062-1-paul.burton@mips.com> In-Reply-To: <20191001215249.4157062-1-paul.burton@mips.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BYAPR07CA0093.namprd07.prod.outlook.com (2603:10b6:a03:12b::34) To MWHPR2201MB1277.namprd22.prod.outlook.com (2603:10b6:301:18::12) authentication-results: spf=none (sender IP is ) smtp.mailfrom=pburton@wavecomp.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.23.0 x-originating-ip: [12.94.197.246] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: b18c0c9a-9b53-47a5-e6d1-08d746b9d2b6 x-ms-traffictypediagnostic: MWHPR2201MB1581: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:5797; x-forefront-prvs: 0177904E6B x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(136003)(39850400004)(366004)(346002)(396003)(376002)(189003)(199004)(8936002)(26005)(478600001)(6512007)(8676002)(81166006)(81156014)(6916009)(386003)(76176011)(186003)(6506007)(25786009)(52116002)(6436002)(99286004)(6486002)(316002)(102836004)(54906003)(14454004)(5640700003)(2501003)(486006)(66476007)(7736002)(256004)(14444005)(66556008)(66946007)(1076003)(64756008)(36756003)(107886003)(66066001)(66446008)(4326008)(42882007)(2906002)(6666004)(3846002)(6116002)(476003)(446003)(2616005)(11346002)(2351001)(71190400001)(44832011)(5660300002)(50226002)(305945005)(71200400001)(101420200001);DIR:OUT;SFP:1102;SCL:1;SRVR:MWHPR2201MB1581;H:MWHPR2201MB1277.namprd22.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;MX:1;A:1; received-spf: None (protection.outlook.com: wavecomp.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 4gXRBRX+x+cUbJzD8n8EFzKlQjJ4NYJhGim73DkwiYanwP2H+EV1HSEIifteVV5AN6RMWlDuu6Q6QGK69HJ/m/9mKMEV+VWP3Wxjo9lbp5nijbcX8QJOqzjPO9iVINeOFd8aBmvE/CY4rYCKgLLXVY5OKVBf64kpGZCLHy7c2fGyF17W+sCM/39iSLJu48f5n9lM9Ua1F9LzCSNqALDLt1/C8KqId3r1ZRG7hYB3biKAwb+sWoVaMos6mgVUy2BgIt9/c6Rm8/O5qnCWygT8WKULatSRUbZ5f59D9aaEsmtqMqqN5bGpsscI+n7wGl6acF80BTIRAKueDQ2lB0OnAH7yU+WAU5Ab/vWGVGVVRASfG+r5HmQ3/cWtxCFV2WdIwC373NMjKdf+WhXIB9ocz3hJ1CdDWUIt/FQYaHH8bDg= MIME-Version: 1.0 X-OriginatorOrg: mips.com X-MS-Exchange-CrossTenant-Network-Message-Id: b18c0c9a-9b53-47a5-e6d1-08d746b9d2b6 X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Oct 2019 21:53:41.6609 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 463607d3-1db3-40a0-8a29-970c56230104 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: Wut9jReBp+CsVA459Y1Wnpf5Ka+nTOxJUOxZWf5xbs8n+NtOaRbszJEuoiZeI5v4pRmOkaQ7FfrYgyZK46qKLg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR2201MB1581 Sender: linux-mips-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-mips@vger.kernel.org Loongson3 systems with CONFIG_CPU_LOONGSON3_WORKAROUNDS enabled already emit a full completion barrier as part of the inline assembly containing LL/SC loops for atomic operations. As such the barrier emitted by __smp_mb__before_atomic() is redundant, and we can remove it. Signed-off-by: Paul Burton --- Changes in v2: None arch/mips/include/asm/barrier.h | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/arch/mips/include/asm/barrier.h b/arch/mips/include/asm/barrier.h index 6d92d5ccdafa..49ff172a72b9 100644 --- a/arch/mips/include/asm/barrier.h +++ b/arch/mips/include/asm/barrier.h @@ -119,7 +119,17 @@ static inline void wmb(void) #define nudge_writes() mb() #endif -#define __smp_mb__before_atomic() __smp_mb__before_llsc() +/* + * In the Loongson3 LL/SC workaround case, all of our LL/SC loops already have + * a completion barrier immediately preceding the LL instruction. Therefore we + * can skip emitting a barrier from __smp_mb__before_atomic(). + */ +#ifdef CONFIG_CPU_LOONGSON3_WORKAROUNDS +# define __smp_mb__before_atomic() +#else +# define __smp_mb__before_atomic() __smp_mb__before_llsc() +#endif + #define __smp_mb__after_atomic() smp_llsc_mb() static inline void sync_ginv(void) From patchwork Tue Oct 1 21:53:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Burton X-Patchwork-Id: 11169811 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CD2BC13B1 for ; Tue, 1 Oct 2019 21:54:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id A0FD421A4A for ; Tue, 1 Oct 2019 21:54:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=wavecomp.com header.i=@wavecomp.com header.b="W1wn3Znz" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728455AbfJAVy0 (ORCPT ); Tue, 1 Oct 2019 17:54:26 -0400 Received: from mail-eopbgr750115.outbound.protection.outlook.com ([40.107.75.115]:30851 "EHLO NAM02-BL2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728425AbfJAVyW (ORCPT ); Tue, 1 Oct 2019 17:54:22 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=HUl6jQ//W19rsoIqZaY0BQCNcXnP4ASTai2TIGMWhxalgVQheJCPKsQ0LJe/E6oh7OcTMvK75txz0HBuIP6isiyH5IPgZ5k3JG1wNGfr7io5qB31coyXlX210LNVHNLxCdJTPcnoLilrSQ3k64yDZKVygjJNvKt4ewOPklZr47NXPIUtYUTF3+/MAzw0lV1P3W5BLAsbU5rC91eRSEVuCaPPomYZZ7QBvNWR1fcP1LsHwniofpBVl8p1Pj4coJw7c1yMthX3wibOjbABe5DLTJEqV23/Gr7SiZG1vrqhuYQgqFBy78Gw8YLI6xeDc5rDJsdDh8DSNRVJyev933QREA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/+BP8EbJp0rVcdy7AFdozelvMtBsULC5QYdNbsMvZFo=; b=Ranmi2QLqhuKN2M5e9JGowrGjsJXqCyzpbq3oQ/qDubk51W+QTawlmG7ryZLz21Bscj28JJu2l4FV5BbzA2IUDqGftm3pscOSkcj60s+dQWR8hzWmYN3O0ustq2z+o4XyoUp1Udjhx/3ig/pKu2rfNZQG4f9+LOw61PCOPdfWzPRtDgp3aEGVIcBDmEIsT9ti7uCbtuRXbaWEThWuwUCb4CLx4Z4y2YujEgc67a24L+IU2C4PwmqzyL95ENV5J0Joh58b+Y7FXO5HS6S49oN6Ped6dtfv39SBV0N0E4ON9twfBod3hXoStx1NkxxTKmoryKWGy282fBeHrK970x0/w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wavecomp.com; dmarc=pass action=none header.from=mips.com; dkim=pass header.d=mips.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wavecomp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/+BP8EbJp0rVcdy7AFdozelvMtBsULC5QYdNbsMvZFo=; b=W1wn3Znz6NJd7mvW86PJOD/i5Ll9GiWvBD6LPpswZM3rlCVM3kTVDkj1/oQu2QSugpXjuv/TT6z5hnXV76V/prTjiZ2MUDwgyIxkm7KyHBBn8JDtQECnV+mCJjA9oIqOpu47Iq6xkGbxTBjhhpo2Z/z/HGC0vA8zIbvFEHc057U= Received: from MWHPR2201MB1277.namprd22.prod.outlook.com (10.172.60.12) by MWHPR2201MB1581.namprd22.prod.outlook.com (10.174.167.30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2305.15; Tue, 1 Oct 2019 21:54:16 +0000 Received: from MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481]) by MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481%2]) with mapi id 15.20.2305.022; Tue, 1 Oct 2019 21:54:16 +0000 From: Paul Burton To: "linux-mips@vger.kernel.org" CC: Huacai Chen , Jiaxun Yang , "linux-kernel@vger.kernel.org" , Paul Burton Subject: [PATCH v2 34/36] MIPS: genex: Add Loongson3 LL/SC workaround to ejtag_debug_handler Thread-Topic: [PATCH v2 34/36] MIPS: genex: Add Loongson3 LL/SC workaround to ejtag_debug_handler Thread-Index: AQHVeKKwzu5kY6X3IUqruCoqAj70MQ== Date: Tue, 1 Oct 2019 21:53:42 +0000 Message-ID: <20191001215249.4157062-35-paul.burton@mips.com> References: <20191001215249.4157062-1-paul.burton@mips.com> In-Reply-To: <20191001215249.4157062-1-paul.burton@mips.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BYAPR07CA0093.namprd07.prod.outlook.com (2603:10b6:a03:12b::34) To MWHPR2201MB1277.namprd22.prod.outlook.com (2603:10b6:301:18::12) authentication-results: spf=none (sender IP is ) smtp.mailfrom=pburton@wavecomp.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.23.0 x-originating-ip: [12.94.197.246] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: bf9d3143-be94-4bbe-8a42-08d746b9d341 x-ms-traffictypediagnostic: MWHPR2201MB1581: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:6430; x-forefront-prvs: 0177904E6B x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(136003)(39850400004)(366004)(346002)(396003)(376002)(189003)(199004)(8936002)(26005)(478600001)(6512007)(8676002)(81166006)(81156014)(6916009)(386003)(76176011)(186003)(6506007)(25786009)(52116002)(6436002)(99286004)(6486002)(316002)(102836004)(54906003)(14454004)(5640700003)(2501003)(486006)(4744005)(66476007)(7736002)(256004)(14444005)(66556008)(66946007)(1076003)(64756008)(36756003)(107886003)(66066001)(66446008)(4326008)(42882007)(2906002)(6666004)(3846002)(6116002)(476003)(446003)(2616005)(11346002)(2351001)(71190400001)(44832011)(5660300002)(50226002)(305945005)(71200400001);DIR:OUT;SFP:1102;SCL:1;SRVR:MWHPR2201MB1581;H:MWHPR2201MB1277.namprd22.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;MX:1;A:1; received-spf: None (protection.outlook.com: wavecomp.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: Uaw5EaBerF5xwd5YTlTFeJlsR4XQK800G1X/DWfMHO2l1+qfQlzxLX31OfrIE/AF1+j2DMXzZ4DLktEhxlqEDrZnVKym/ORZE4GNBRfb0pp/3uUpPwaMWwddb91cqwiL7WDUOo2ofGeq1Ym0WISce+4O1W15BWhlJWYsevesOB1+aHP4cEHWA7HNSkFtU35hWY9ochFQhF35qyOSZvfbYZiupDR1ltyxI2bn52OyFM+q53U+Kq/2YpHQfmT4iOIbLKmv+XJUe9i2ryKAGCUr0WZb/6B94ygq3zYaR2xZTDqbL/zdHWouy2OQ6HuCYrZDZXg10xEHpbqHaVFX/j/JfKRW0o726pWNcjLGImOyyoU10MYwjpL13PL1Hir8X68ZvmRxvHm+YXywt1CvOXemri+i4tBFPM2PahNzfcxxixc= MIME-Version: 1.0 X-OriginatorOrg: mips.com X-MS-Exchange-CrossTenant-Network-Message-Id: bf9d3143-be94-4bbe-8a42-08d746b9d341 X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Oct 2019 21:53:42.5305 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 463607d3-1db3-40a0-8a29-970c56230104 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: KYYekGG+Fe40NXL31TgHTn6mCGZQ5OPJDL/qGIsXdK/Snn1tEsED5jzM0STidMrvyczg5sq82C8v9VM3ZdJgbw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR2201MB1581 Sender: linux-mips-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-mips@vger.kernel.org In ejtag_debug_handler we use LL & SC instructions to acquire & release an open-coded spinlock. For Loongson3 systems affected by LL/SC errata this requires that we insert a sync instruction prior to the LL in order to ensure correct behavior of the LL/SC loop. Signed-off-by: Paul Burton --- Changes in v2: None arch/mips/kernel/genex.S | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/mips/kernel/genex.S b/arch/mips/kernel/genex.S index efde27c99414..ac4f2b835165 100644 --- a/arch/mips/kernel/genex.S +++ b/arch/mips/kernel/genex.S @@ -18,6 +18,7 @@ #include #include #include +#include #include #include @@ -353,6 +354,7 @@ NESTED(ejtag_debug_handler, PT_SIZE, sp) #ifdef CONFIG_SMP 1: PTR_LA k0, ejtag_debug_buffer_spinlock + __SYNC(full, loongson3_war) ll k0, 0(k0) bnez k0, 1b PTR_LA k0, ejtag_debug_buffer_spinlock From patchwork Tue Oct 1 21:53:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Burton X-Patchwork-Id: 11169813 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D36F713B1 for ; Tue, 1 Oct 2019 21:54:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id B147A21924 for ; Tue, 1 Oct 2019 21:54:46 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=wavecomp.com header.i=@wavecomp.com header.b="FHC/iLND" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728432AbfJAVy0 (ORCPT ); Tue, 1 Oct 2019 17:54:26 -0400 Received: from mail-eopbgr750115.outbound.protection.outlook.com ([40.107.75.115]:30851 "EHLO NAM02-BL2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728447AbfJAVyZ (ORCPT ); Tue, 1 Oct 2019 17:54:25 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=DCcrQDCNyI39OPoBimBqB7XUQS2Ga2MXGZbBue0u4s9NgufINqiQseC1/GvA52VozhYukiaItxrmuA/rGFCRD9aionJKOlpyL4f4Tw34zZtwpcsIkJEHHmUNZzdGXGzrvMqhrkkHtqbmEl/fy0edvoaEkbCiRGOAu4KtM98jnKbWDg3d9222R6U67VuO6MOtg+HiUIQ3Q7M2C5rfKLperA2g5PjlgyIUcgU2cB7erRaaTiqDJ4ujg/+0jxZFMuF/PiCOUym10PVP/fUMR+U2OpGi3RfylWUEbM+9g1Sk4qr/FYJZGzXO32Iwx8AaFlvnp1B+Gqr1lvvzD4NdCz0IHw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=UEuJ/46XhwVU2eIrIfOwmeNx92pxUuflO1ZQt5HbWCo=; b=N0ecvU2KKDBxd/Uw1Cqjm8elo7cONLBlZYt5jWuebFZrf0r6VXpCJGJRoaRokpHFgQRVz/KYFdcFf6nZz6Bcs02FEoSoeY/J6OL6us4wX75zcpFEvl/8l3HKbOp+Zhicksu4Xb/g1Dqt/hg2DZFWleOzb72u319CHcbfESZ8WckoV9wDWzidbGE1btXKymXj6YI7LjSFSa37uFYZERup98mVKIJ5h7QzboU47TrPT2jjTrQvXLvLgYQ3PcpE7YxvGi6QGO9BFLQ61q19+sk0Uf0jaAxiiwFhPpS7+Obk/dsnTq4qr237pN64fciROmCAoCsDUpSvbmo/t+jG3fEXqA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wavecomp.com; dmarc=pass action=none header.from=mips.com; dkim=pass header.d=mips.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wavecomp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=UEuJ/46XhwVU2eIrIfOwmeNx92pxUuflO1ZQt5HbWCo=; b=FHC/iLNDqEgtRmQiQCdbLqBjCa/gUsqTymtp0wRtMVRbY3Ih8K8yOcFD2zyX2ohvwV3rqvQTW2iD2evozu9Xt1WHcOZsUDI8u54AJrUswUM6g1A5AGgZDqYk+dwfeKGnoxeXSwNl8bHoUKwXqOMYVgvKxCrvppIFRcDUlSuzEi4= Received: from MWHPR2201MB1277.namprd22.prod.outlook.com (10.172.60.12) by MWHPR2201MB1581.namprd22.prod.outlook.com (10.174.167.30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2305.15; Tue, 1 Oct 2019 21:54:16 +0000 Received: from MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481]) by MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481%2]) with mapi id 15.20.2305.022; Tue, 1 Oct 2019 21:54:16 +0000 From: Paul Burton To: "linux-mips@vger.kernel.org" CC: Huacai Chen , Jiaxun Yang , "linux-kernel@vger.kernel.org" , Paul Burton Subject: [PATCH v2 35/36] MIPS: genex: Don't reload address unnecessarily Thread-Topic: [PATCH v2 35/36] MIPS: genex: Don't reload address unnecessarily Thread-Index: AQHVeKKxGksHXgvWN0Cq1FTc0sIMpQ== Date: Tue, 1 Oct 2019 21:53:43 +0000 Message-ID: <20191001215249.4157062-36-paul.burton@mips.com> References: <20191001215249.4157062-1-paul.burton@mips.com> In-Reply-To: <20191001215249.4157062-1-paul.burton@mips.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BYAPR07CA0093.namprd07.prod.outlook.com (2603:10b6:a03:12b::34) To MWHPR2201MB1277.namprd22.prod.outlook.com (2603:10b6:301:18::12) authentication-results: spf=none (sender IP is ) smtp.mailfrom=pburton@wavecomp.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.23.0 x-originating-ip: [12.94.197.246] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 4e443a24-2d61-4a1f-d2b9-08d746b9d3c8 x-ms-traffictypediagnostic: MWHPR2201MB1581: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8882; x-forefront-prvs: 0177904E6B x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(136003)(39850400004)(366004)(346002)(396003)(376002)(189003)(199004)(8936002)(26005)(478600001)(6512007)(8676002)(81166006)(81156014)(6916009)(386003)(76176011)(186003)(6506007)(25786009)(52116002)(6436002)(99286004)(6486002)(316002)(102836004)(54906003)(14454004)(5640700003)(2501003)(486006)(66476007)(7736002)(256004)(14444005)(66556008)(66946007)(1076003)(64756008)(36756003)(107886003)(66066001)(66446008)(4326008)(42882007)(2906002)(6666004)(3846002)(6116002)(476003)(446003)(2616005)(11346002)(2351001)(71190400001)(44832011)(5660300002)(50226002)(305945005)(71200400001);DIR:OUT;SFP:1102;SCL:1;SRVR:MWHPR2201MB1581;H:MWHPR2201MB1277.namprd22.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;MX:1;A:1; received-spf: None (protection.outlook.com: wavecomp.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: iVsi1yyHGIykh5K0ZPf/KofbYdCDYPPAMmpIm2u0Auy8AVha9V6a2JemHewtZ/tqRnCKl56PHcruuW92fZa9H6k8tc1a6scDR2YmMubylULzzAVSIsC6G4ryJ2fq8d8tXK8CiSF6+Hvr2g6O91MIE2oQrNTbjljPtQEUfneUFF4qgRTLF4El+mJSzNjicljLPFoXxLyvHSTDVkfdILouMD2yUiwGEs0axGlHGNR9cfFhoF5fwozFHXvOhfFoCDIAKNPPoLYRaFre4Mkdz+5mwODY7pPjozvGkkFpTMFMhOWqZBf1njOOQT/G9RRQp6S2/wyi0yhQxcIZclbK0hjeKFhnMTuNAxQFMwKFlZYkbySlYJqNltPZadQrDzYeeHNl7xVi93dFXYru4ArjZOp+vBRLul3XmXcvpvsS5CNLVIE= MIME-Version: 1.0 X-OriginatorOrg: mips.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4e443a24-2d61-4a1f-d2b9-08d746b9d3c8 X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Oct 2019 21:53:43.4191 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 463607d3-1db3-40a0-8a29-970c56230104 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: XbXSJizRjekxJcODd79Rvz4k1Zj1vDSB2datmgPnhsVv/bv8nU4fvZHaCSC2k4Tw09UyPk6nk2ZUAD5Ng3n0iA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR2201MB1581 Sender: linux-mips-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-mips@vger.kernel.org In ejtag_debug_handler() we must reload the address of ejtag_debug_buffer_spinlock if an sc fails, since the address in k0 will have been clobbered by the result of the sc instruction. In the case where we simply load a non-zero value (ie. there's contention for the lock) the address will not be clobbered & we can simply branch back to repeat the load from memory without reloading the address into k0. The primary motivation for this change is that it moves the target of the bnez instruction to an instruction within the LL/SC loop (the LL itself), which we know contains no other memory accesses & therefore isn't affected by Loongson3 LL/SC errata. Signed-off-by: Paul Burton --- Changes in v2: None arch/mips/kernel/genex.S | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/mips/kernel/genex.S b/arch/mips/kernel/genex.S index ac4f2b835165..60ede6b75a3b 100644 --- a/arch/mips/kernel/genex.S +++ b/arch/mips/kernel/genex.S @@ -355,8 +355,8 @@ NESTED(ejtag_debug_handler, PT_SIZE, sp) #ifdef CONFIG_SMP 1: PTR_LA k0, ejtag_debug_buffer_spinlock __SYNC(full, loongson3_war) - ll k0, 0(k0) - bnez k0, 1b +2: ll k0, 0(k0) + bnez k0, 2b PTR_LA k0, ejtag_debug_buffer_spinlock sc k0, 0(k0) beqz k0, 1b From patchwork Tue Oct 1 21:53:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Burton X-Patchwork-Id: 11169809 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 550E413B1 for ; Tue, 1 Oct 2019 21:54:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 1CC7921924 for ; Tue, 1 Oct 2019 21:54:43 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=wavecomp.com header.i=@wavecomp.com header.b="jd6MUnj3" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728474AbfJAVy3 (ORCPT ); Tue, 1 Oct 2019 17:54:29 -0400 Received: from mail-eopbgr750115.outbound.protection.outlook.com ([40.107.75.115]:30851 "EHLO NAM02-BL2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728374AbfJAVy2 (ORCPT ); Tue, 1 Oct 2019 17:54:28 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=e6JyXmKYfAM2pBepbyziyoJTpfpovQXu3GNeDGQ5+YcLjOcd0ppYD84iKXM6rLyavT95jGts5IOQKlsEQ71zktKM8tDJnVtbW9ILzyERzA5m9MfeKQzt+z/6AxHdc3e6eFD9KWay4dzPJBtrJLYdiKBJ95Ndq41YPvGWr1Ljcnc9Nr2+s6eYhK8rc/5CkAbS7orEpQqIMJ3h881ev4/8MnWubaxjjLp43KHHFKd0FKzRODY2SmX1Bv1cOm/ttN9tw6wZqWSnOWgy4tl52PfYbU4gTIb/+ODMHvpsXtoDyFQ/y/B5FuFmqBo7eqNUlO+rxJK4TPlzwgyPkVqM2hosgA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=c0HFqTmB5HlOC7377lZyro2MRboHr5sygckm1ojSSmc=; b=INOyJG9UuQueR4mVeQzyvavc9zW0QEETNpjuodyqM4/LUW8T45uUfEylQIRpYfsYwRyQDB5jAVxwMlXh4sq5401APOlKr4v0YzrAUTXf4ynxaaPWbMeaD/vFSl/jb+Al9W1KTXCczmLyuRE/nXcaZQQQ9spko35h+fZVwxs9z23eEOnQIpVROIm3tByZY1K4/ClihJ1yCnlzDWqfWayGQHcHHwdfJOqGy8D+6+STUb+Kk6vMlxd3Rxs4PCis5bwHT0vWo21l7mauDF+tRjouP4dhT/in3brDa5q75v/D+lk5XDVxon7Y92uCw1beKA4cKCiSr+hLWILF2vzCHl6RgA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wavecomp.com; dmarc=pass action=none header.from=mips.com; dkim=pass header.d=mips.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wavecomp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=c0HFqTmB5HlOC7377lZyro2MRboHr5sygckm1ojSSmc=; b=jd6MUnj3izzRbeFHAHUKMSLAuMofi17CnjkjE0vt0FZW6PxcqlYShp5iKTq7m5t1zKSL76ltTYxNDGv6kYIYFQLNePl+PkekEqOc17qvDu10SjNB0NhLZ/Mu9PRjlP4JUOY8C9jqje4L0BYAK15D6iMjMNfV8a813IVxlblalks= Received: from MWHPR2201MB1277.namprd22.prod.outlook.com (10.172.60.12) by MWHPR2201MB1581.namprd22.prod.outlook.com (10.174.167.30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2305.15; Tue, 1 Oct 2019 21:54:17 +0000 Received: from MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481]) by MWHPR2201MB1277.namprd22.prod.outlook.com ([fe80::498b:c2cd:e816:1481%2]) with mapi id 15.20.2305.022; Tue, 1 Oct 2019 21:54:17 +0000 From: Paul Burton To: "linux-mips@vger.kernel.org" CC: Huacai Chen , Jiaxun Yang , "linux-kernel@vger.kernel.org" , Paul Burton Subject: [PATCH v2 36/36] MIPS: Check Loongson3 LL/SC errata workaround correctness Thread-Topic: [PATCH v2 36/36] MIPS: Check Loongson3 LL/SC errata workaround correctness Thread-Index: AQHVeKKxElSaWtuqP0WF1bkQzrRwhQ== Date: Tue, 1 Oct 2019 21:53:44 +0000 Message-ID: <20191001215249.4157062-37-paul.burton@mips.com> References: <20191001215249.4157062-1-paul.burton@mips.com> In-Reply-To: <20191001215249.4157062-1-paul.burton@mips.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BYAPR07CA0093.namprd07.prod.outlook.com (2603:10b6:a03:12b::34) To MWHPR2201MB1277.namprd22.prod.outlook.com (2603:10b6:301:18::12) authentication-results: spf=none (sender IP is ) smtp.mailfrom=pburton@wavecomp.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.23.0 x-originating-ip: [12.94.197.246] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 0bb4137f-0dcc-42f3-b3b9-08d746b9d440 x-ms-traffictypediagnostic: MWHPR2201MB1581: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:2150; x-forefront-prvs: 0177904E6B x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(136003)(39850400004)(366004)(346002)(396003)(376002)(189003)(199004)(8936002)(26005)(478600001)(6512007)(8676002)(81166006)(81156014)(6916009)(386003)(76176011)(186003)(6506007)(25786009)(52116002)(6436002)(99286004)(6486002)(316002)(102836004)(54906003)(14454004)(5640700003)(2501003)(486006)(66476007)(7736002)(256004)(14444005)(66556008)(66946007)(1076003)(64756008)(36756003)(107886003)(66066001)(66446008)(4326008)(42882007)(2906002)(6666004)(3846002)(6116002)(476003)(446003)(2616005)(11346002)(2351001)(71190400001)(44832011)(5660300002)(50226002)(305945005)(71200400001);DIR:OUT;SFP:1102;SCL:1;SRVR:MWHPR2201MB1581;H:MWHPR2201MB1277.namprd22.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;MX:1;A:1; received-spf: None (protection.outlook.com: wavecomp.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: Gp9EukrRI//ySsEbyxFC7DwWSrHGX+BHBDcIKvhPYwYfH3x45jVV7Aj6pPsM71QzPeWg1vpduACibv6gjYqOsOBc7Xq1r77Vx6kCT7VTJuVmEV79r7tdAIoyxF9urw5YCY0tozDPQuBHGCRMPCosdsOw4UhsUtFjbdIjDZOZ0HAF9DmuK26iUI2RtoOWAuRnRGHHCyR/coFUZVCvcmq8ZLPZnqSyqCEI0mwpyQIa+z0YkOFzRCHgMPC9Wp8uO0HocNl2mOXIIESLiTma5TlM/oePIHMHAm5vhxHoAZ6ZcC3TnqyX7hsEyH1TTBNINzXriKv7w6T6RYEyTcrewtDKZstwgH1RJmQkGSRbAlC3OaXR8zZA+6bqUq1cE4UQQnpZ9z+ZGW5A0E4lTYuRn9tPqb7AN/NVUXaKmZXzSdnanuY= MIME-Version: 1.0 X-OriginatorOrg: mips.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0bb4137f-0dcc-42f3-b3b9-08d746b9d440 X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Oct 2019 21:53:44.2067 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 463607d3-1db3-40a0-8a29-970c56230104 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: KL3NDdSO6my+cfFkU9PIXHW4Qjr0f2kDoxJKpNqDQttws39kVBbAwSYlSXxu49XFH3W8UXB145+iEOmb7HMj6g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR2201MB1581 Sender: linux-mips-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-mips@vger.kernel.org When Loongson3 LL/SC errata workarounds are enabled (ie. CONFIG_CPU_LOONGSON3_WORKAROUNDS=y) run a tool to scan through the compiled kernel & ensure that the workaround is applied correctly. That is, ensure that: - Every LL or LLD instruction is preceded by a sync instruction. - Any branches from within an LL/SC loop to outside of that loop target a sync instruction. Reasoning for these conditions can be found by reading the comment above the definition of __SYNC_loongson3_war in arch/mips/include/asm/sync.h. This tool will help ensure that we don't inadvertently introduce code paths that miss the required workarounds. Signed-off-by: Paul Burton --- Changes in v2: - Only try to build loongson3-llsc-check from arch/mips/Makefile when CONFIG_CPU_LOONGSON3_WORKAROUNDS is enabled. arch/mips/Makefile | 3 + arch/mips/Makefile.postlink | 10 +- arch/mips/tools/.gitignore | 1 + arch/mips/tools/Makefile | 5 + arch/mips/tools/loongson3-llsc-check.c | 307 +++++++++++++++++++++++++ 5 files changed, 325 insertions(+), 1 deletion(-) create mode 100644 arch/mips/tools/loongson3-llsc-check.c diff --git a/arch/mips/Makefile b/arch/mips/Makefile index cdc09b71febe..0a5eab626260 100644 --- a/arch/mips/Makefile +++ b/arch/mips/Makefile @@ -14,6 +14,9 @@ archscripts: scripts_basic $(Q)$(MAKE) $(build)=arch/mips/tools elf-entry +ifeq ($(CONFIG_CPU_LOONGSON3_WORKAROUNDS),y) + $(Q)$(MAKE) $(build)=arch/mips/tools loongson3-llsc-check +endif $(Q)$(MAKE) $(build)=arch/mips/boot/tools relocs KBUILD_DEFCONFIG := 32r2el_defconfig diff --git a/arch/mips/Makefile.postlink b/arch/mips/Makefile.postlink index 4eea4188cb20..f03fdc95143e 100644 --- a/arch/mips/Makefile.postlink +++ b/arch/mips/Makefile.postlink @@ -3,7 +3,8 @@ # Post-link MIPS pass # =========================================================================== # -# 1. Insert relocations into vmlinux +# 1. Check that Loongson3 LL/SC workarounds are applied correctly +# 2. Insert relocations into vmlinux PHONY := __archpost __archpost: @@ -11,6 +12,10 @@ __archpost: -include include/config/auto.conf include scripts/Kbuild.include +CMD_LS3_LLSC = arch/mips/tools/loongson3-llsc-check +quiet_cmd_ls3_llsc = LLSCCHK $@ + cmd_ls3_llsc = $(CMD_LS3_LLSC) $@ + CMD_RELOCS = arch/mips/boot/tools/relocs quiet_cmd_relocs = RELOCS $@ cmd_relocs = $(CMD_RELOCS) $@ @@ -19,6 +24,9 @@ quiet_cmd_relocs = RELOCS $@ vmlinux: FORCE @true +ifeq ($(CONFIG_CPU_LOONGSON3_WORKAROUNDS),y) + $(call if_changed,ls3_llsc) +endif ifeq ($(CONFIG_RELOCATABLE),y) $(call if_changed,relocs) endif diff --git a/arch/mips/tools/.gitignore b/arch/mips/tools/.gitignore index 56d34ccccce4..b0209450d9ff 100644 --- a/arch/mips/tools/.gitignore +++ b/arch/mips/tools/.gitignore @@ -1 +1,2 @@ elf-entry +loongson3-llsc-check diff --git a/arch/mips/tools/Makefile b/arch/mips/tools/Makefile index 3baee4bc6775..aaef688749f5 100644 --- a/arch/mips/tools/Makefile +++ b/arch/mips/tools/Makefile @@ -3,3 +3,8 @@ hostprogs-y := elf-entry PHONY += elf-entry elf-entry: $(obj)/elf-entry @: + +hostprogs-$(CONFIG_CPU_LOONGSON3_WORKAROUNDS) += loongson3-llsc-check +PHONY += loongson3-llsc-check +loongson3-llsc-check: $(obj)/loongson3-llsc-check + @: diff --git a/arch/mips/tools/loongson3-llsc-check.c b/arch/mips/tools/loongson3-llsc-check.c new file mode 100644 index 000000000000..0ebddd0ae46f --- /dev/null +++ b/arch/mips/tools/loongson3-llsc-check.c @@ -0,0 +1,307 @@ +// SPDX-License-Identifier: GPL-2.0-only +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#ifdef be32toh +/* If libc provides le{16,32,64}toh() then we'll use them */ +#elif BYTE_ORDER == LITTLE_ENDIAN +# define le16toh(x) (x) +# define le32toh(x) (x) +# define le64toh(x) (x) +#elif BYTE_ORDER == BIG_ENDIAN +# define le16toh(x) bswap_16(x) +# define le32toh(x) bswap_32(x) +# define le64toh(x) bswap_64(x) +#endif + +/* MIPS opcodes, in bits 31:26 of an instruction */ +#define OP_SPECIAL 0x00 +#define OP_REGIMM 0x01 +#define OP_BEQ 0x04 +#define OP_BNE 0x05 +#define OP_BLEZ 0x06 +#define OP_BGTZ 0x07 +#define OP_BEQL 0x14 +#define OP_BNEL 0x15 +#define OP_BLEZL 0x16 +#define OP_BGTZL 0x17 +#define OP_LL 0x30 +#define OP_LLD 0x34 +#define OP_SC 0x38 +#define OP_SCD 0x3c + +/* Bits 20:16 of OP_REGIMM instructions */ +#define REGIMM_BLTZ 0x00 +#define REGIMM_BGEZ 0x01 +#define REGIMM_BLTZL 0x02 +#define REGIMM_BGEZL 0x03 +#define REGIMM_BLTZAL 0x10 +#define REGIMM_BGEZAL 0x11 +#define REGIMM_BLTZALL 0x12 +#define REGIMM_BGEZALL 0x13 + +/* Bits 5:0 of OP_SPECIAL instructions */ +#define SPECIAL_SYNC 0x0f + +static void usage(FILE *f) +{ + fprintf(f, "Usage: loongson3-llsc-check /path/to/vmlinux\n"); +} + +static int se16(uint16_t x) +{ + return (int16_t)x; +} + +static bool is_ll(uint32_t insn) +{ + switch (insn >> 26) { + case OP_LL: + case OP_LLD: + return true; + + default: + return false; + } +} + +static bool is_sc(uint32_t insn) +{ + switch (insn >> 26) { + case OP_SC: + case OP_SCD: + return true; + + default: + return false; + } +} + +static bool is_sync(uint32_t insn) +{ + /* Bits 31:11 should all be zeroes */ + if (insn >> 11) + return false; + + /* Bits 5:0 specify the SYNC special encoding */ + if ((insn & 0x3f) != SPECIAL_SYNC) + return false; + + return true; +} + +static bool is_branch(uint32_t insn, int *off) +{ + switch (insn >> 26) { + case OP_BEQ: + case OP_BEQL: + case OP_BNE: + case OP_BNEL: + case OP_BGTZ: + case OP_BGTZL: + case OP_BLEZ: + case OP_BLEZL: + *off = se16(insn) + 1; + return true; + + case OP_REGIMM: + switch ((insn >> 16) & 0x1f) { + case REGIMM_BGEZ: + case REGIMM_BGEZL: + case REGIMM_BGEZAL: + case REGIMM_BGEZALL: + case REGIMM_BLTZ: + case REGIMM_BLTZL: + case REGIMM_BLTZAL: + case REGIMM_BLTZALL: + *off = se16(insn) + 1; + return true; + + default: + return false; + } + + default: + return false; + } +} + +static int check_ll(uint64_t pc, uint32_t *code, size_t sz) +{ + ssize_t i, max, sc_pos; + int off; + + /* + * Every LL must be preceded by a sync instruction in order to ensure + * that instruction reordering doesn't allow a prior memory access to + * execute after the LL & cause erroneous results. + */ + if (!is_sync(le32toh(code[-1]))) { + fprintf(stderr, "%" PRIx64 ": LL not preceded by sync\n", pc); + return -EINVAL; + } + + /* Find the matching SC instruction */ + max = sz / 4; + for (sc_pos = 0; sc_pos < max; sc_pos++) { + if (is_sc(le32toh(code[sc_pos]))) + break; + } + if (sc_pos >= max) { + fprintf(stderr, "%" PRIx64 ": LL has no matching SC\n", pc); + return -EINVAL; + } + + /* + * Check branches within the LL/SC loop target sync instructions, + * ensuring that speculative execution can't generate memory accesses + * due to instructions outside of the loop. + */ + for (i = 0; i < sc_pos; i++) { + if (!is_branch(le32toh(code[i]), &off)) + continue; + + /* + * If the branch target is within the LL/SC loop then we don't + * need to worry about it. + */ + if ((off >= -i) && (off <= sc_pos)) + continue; + + /* If the branch targets a sync instruction we're all good... */ + if (is_sync(le32toh(code[i + off]))) + continue; + + /* ...but if not, we have a problem */ + fprintf(stderr, "%" PRIx64 ": Branch target not a sync\n", + pc + (i * 4)); + return -EINVAL; + } + + return 0; +} + +static int check_code(uint64_t pc, uint32_t *code, size_t sz) +{ + int err = 0; + + if (sz % 4) { + fprintf(stderr, "%" PRIx64 ": Section size not a multiple of 4\n", + pc); + err = -EINVAL; + sz -= (sz % 4); + } + + if (is_ll(le32toh(code[0]))) { + fprintf(stderr, "%" PRIx64 ": First instruction in section is an LL\n", + pc); + err = -EINVAL; + } + +#define advance() ( \ + code++, \ + pc += 4, \ + sz -= 4 \ +) + + /* + * Skip the first instructionm allowing check_ll to look backwards + * unconditionally. + */ + advance(); + + /* Now scan through the code looking for LL instructions */ + for (; sz; advance()) { + if (is_ll(le32toh(code[0]))) + err |= check_ll(pc, code, sz); + } + + return err; +} + +int main(int argc, char *argv[]) +{ + int vmlinux_fd, status, err, i; + const char *vmlinux_path; + struct stat st; + Elf64_Ehdr *eh; + Elf64_Shdr *sh; + void *vmlinux; + + status = EXIT_FAILURE; + + if (argc < 2) { + usage(stderr); + goto out_ret; + } + + vmlinux_path = argv[1]; + vmlinux_fd = open(vmlinux_path, O_RDONLY); + if (vmlinux_fd == -1) { + perror("Unable to open vmlinux"); + goto out_ret; + } + + err = fstat(vmlinux_fd, &st); + if (err) { + perror("Unable to stat vmlinux"); + goto out_close; + } + + vmlinux = mmap(NULL, st.st_size, PROT_READ, MAP_PRIVATE, vmlinux_fd, 0); + if (vmlinux == MAP_FAILED) { + perror("Unable to mmap vmlinux"); + goto out_close; + } + + eh = vmlinux; + if (memcmp(eh->e_ident, ELFMAG, SELFMAG)) { + fprintf(stderr, "vmlinux is not an ELF?\n"); + goto out_munmap; + } + + if (eh->e_ident[EI_CLASS] != ELFCLASS64) { + fprintf(stderr, "vmlinux is not 64b?\n"); + goto out_munmap; + } + + if (eh->e_ident[EI_DATA] != ELFDATA2LSB) { + fprintf(stderr, "vmlinux is not little endian?\n"); + goto out_munmap; + } + + for (i = 0; i < le16toh(eh->e_shnum); i++) { + sh = vmlinux + le64toh(eh->e_shoff) + (i * le16toh(eh->e_shentsize)); + + if (sh->sh_type != SHT_PROGBITS) + continue; + if (!(sh->sh_flags & SHF_EXECINSTR)) + continue; + + err = check_code(le64toh(sh->sh_addr), + vmlinux + le64toh(sh->sh_offset), + le64toh(sh->sh_size)); + if (err) + goto out_munmap; + } + + status = EXIT_SUCCESS; +out_munmap: + munmap(vmlinux, st.st_size); +out_close: + close(vmlinux_fd); +out_ret: + return status; +}