From patchwork Wed Mar 16 23:25:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Palmer Dabbelt X-Patchwork-Id: 12783290 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C650CC433F5 for ; Wed, 16 Mar 2022 23:28:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:To:From:Cc:MIME-Version:References: In-Reply-To:Message-Id:Date:Subject:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=I3BveWqhYo/+kmlES4y408sMG0mYWY3v06WHMyr26gw=; b=MKgOF6muyOElsP 4WkV/JZG2CnlUypB5xG/sjfJrQFUHKun5Cc2FdQwHISET4qRw5eN2qg1gpq9sbuC/oEq58odOZuOI E2k1LZoSZQ1HW0O6njE/DEatVY+YeS1gztxMqqYHw8rT42jISQ4SSI/6eoApIIW/XgOT//V1gSoEc QcQy6jVwN39yZuAg7RNOokBk9PNwrjoH2MuE+eMSNDi4f6e20LzOYxVimPmjSVL7ZHH19XejSHsEo 6aN9qz7Orh2MXJUH21VFF2V7COzZWLkUq0OkuRqbYWx77wxMyudcpiWrXDhPD1QP3Q39JWCYI+Bg4 A1zWb7GqYqi8d5XYzwBw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nUd4B-00EUAM-1V; Wed, 16 Mar 2022 23:28:31 +0000 Received: from mail-pf1-x435.google.com ([2607:f8b0:4864:20::435]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nUd46-00EU6h-DG for linux-riscv@lists.infradead.org; Wed, 16 Mar 2022 23:28:27 +0000 Received: by mail-pf1-x435.google.com with SMTP id l8so5421685pfu.1 for ; Wed, 16 Mar 2022 16:28:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; h=subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding:cc:from:to; bh=ofUFlPn1fFr6Pz05IWJLvQM8LyheEnwZgHIcbcC4aPw=; b=S5EoqvKolpNViVFVC6Nzsj4aBgAWTlxXUWEEMY7p3wEb0057ehZpzaUnP8f4GDTFJs zse0vx0vaol845GXW+EPh58H5hafan3CLwS2NebIMBjef8U0HNWEx3yRz3jPiV3uJlEC YXWJrMiwVe+oiaYZPhnxkOPG0gNmZ72sB2JgKC7u3KkkHP6Mrza8LqpBn+wgIhU0+wXO eiGMxzeY0dkF1fMYunyzzQlM6whMlxgT8yXppR7Shy3VF4INxmMTGDdTecXUZiBbWpJi KxadBRMOObu8VavJ+IpQtZPAu2dQ5o1nwyDfes8oJeqdvEAFgSJuP+TqCD4jDajWtIKl w7BQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:cc:from:to; bh=ofUFlPn1fFr6Pz05IWJLvQM8LyheEnwZgHIcbcC4aPw=; b=eYaTmHkFH45dsld3Ndgev9Ut1tvbJu88uEOYYwCGsxRLh9ZTeSq/t/B+y1oV/5fpIU GR8tdIghuWyLOxYP7f5SqebNGWJJ7edWyL6oeIiKna13h7YVBJeUgYp4unAN8te1utwW vHXmzXD5vD5MEIMHWMEO/qP16NCqHJwLCCDyhY/Mx2PI2+cmhbRiGlVBps+SlXUnPvAs m1umVNuupN9XZPpZd4svS2txzTo5ahCGFY8fRBuV4hGPTRMzNwwd8qliYtA33h2diQWM M4XKTJoY7VT70ly0CMJb45oWnhqh/t2pfmWMmxLpCRQ+GWyP2SL4q/A7dkfJzhkeNvGP jMMg== X-Gm-Message-State: AOAM532h21nQQE8eut/JY9iQ6r0hG5KpoWlSr+x7jZjymc08bWr9dmdN ZectRqjowRNLPhyivgofJJPRHw== X-Google-Smtp-Source: ABdhPJxYRrhM5pW5WVasmxAVWyXeEsoX5VSiHkFO1pWtO6NNC1CxKjdOixv6tMKvEgnHK4ZptvG9zQ== X-Received: by 2002:a63:5855:0:b0:380:a9f7:e373 with SMTP id i21-20020a635855000000b00380a9f7e373mr1418979pgm.557.1647473305604; Wed, 16 Mar 2022 16:28:25 -0700 (PDT) Received: from localhost ([12.3.194.138]) by smtp.gmail.com with ESMTPSA id l186-20020a633ec3000000b003820485172asm145763pga.65.2022.03.16.16.28.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 16 Mar 2022 16:28:25 -0700 (PDT) Subject: [PATCH 1/5] asm-generic: qspinlock: Indicate the use of mixed-size atomics Date: Wed, 16 Mar 2022 16:25:56 -0700 Message-Id: <20220316232600.20419-2-palmer@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220316232600.20419-1-palmer@rivosinc.com> References: <20220316232600.20419-1-palmer@rivosinc.com> MIME-Version: 1.0 Cc: jonas@southpole.se, stefan.kristiansson@saunalahti.fi, shorne@gmail.com, mingo@redhat.com, Will Deacon , longman@redhat.com, boqun.feng@gmail.com, Paul Walmsley , Palmer Dabbelt , aou@eecs.berkeley.edu, Arnd Bergmann , jszhang@kernel.org, wangkefeng.wang@huawei.com, openrisc@lists.librecores.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, Palmer Dabbelt From: Palmer Dabbelt To: linux-riscv@lists.infradead.org, peterz@infradead.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220316_162826_475781_5B3BA3EC X-CRM114-Status: GOOD ( 12.77 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org From: Peter Zijlstra The qspinlock implementation depends on having well behaved mixed-size atomics. This is true on the more widely-used platforms, but these requirements are somewhat subtle and may not be satisfied by all the platforms that qspinlock is used on. Document these requirements, so ports that use qspinlock can more easily determine if they meet these requirements. Signed-off-by: Palmer Dabbelt Acked-by: Waiman Long --- I have specifically not included Peter's SOB on this, as he sent his original patch without one. --- include/asm-generic/qspinlock.h | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/include/asm-generic/qspinlock.h b/include/asm-generic/qspinlock.h index d74b13825501..a7a1296b0b4d 100644 --- a/include/asm-generic/qspinlock.h +++ b/include/asm-generic/qspinlock.h @@ -2,6 +2,36 @@ /* * Queued spinlock * + * A 'generic' spinlock implementation that is based on MCS locks. An + * architecture that's looking for a 'generic' spinlock, please first consider + * ticket-lock.h and only come looking here when you've considered all the + * constraints below and can show your hardware does actually perform better + * with qspinlock. + * + * + * It relies on atomic_*_release()/atomic_*_acquire() to be RCsc (or no weaker + * than RCtso if you're power), where regular code only expects atomic_t to be + * RCpc. + * + * It relies on a far greater (compared to ticket-lock.h) set of atomic + * operations to behave well together, please audit them carefully to ensure + * they all have forward progress. Many atomic operations may default to + * cmpxchg() loops which will not have good forward progress properties on + * LL/SC architectures. + * + * One notable example is atomic_fetch_or_acquire(), which x86 cannot (cheaply) + * do. Carefully read the patches that introduced queued_fetch_set_pending_acquire(). + * + * It also heavily relies on mixed size atomic operations, in specific it + * requires architectures to have xchg16; something which many LL/SC + * architectures need to implement as a 32bit and+or in order to satisfy the + * forward progress guarantees mentioned above. + * + * Further reading on mixed size atomics that might be relevant: + * + * http://www.cl.cam.ac.uk/~pes20/popl17/mixed-size.pdf + * + * * (C) Copyright 2013-2015 Hewlett-Packard Development Company, L.P. * (C) Copyright 2015 Hewlett-Packard Enterprise Development LP * From patchwork Wed Mar 16 23:25:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Palmer Dabbelt X-Patchwork-Id: 12783289 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6184EC43219 for ; Wed, 16 Mar 2022 23:28:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:To:From:Cc:MIME-Version:References: In-Reply-To:Message-Id:Date:Subject:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ZPKYgmGQTiWRADrr56DGddfGtGJHxNXUZg7kdN+eKMo=; b=Cw8pPMFo+Eg5Qk aQpyFacfHrkvkBtA84pHzueFPcun2rMzdwUwANS981pzXfozKEX1IGDfEl64GI+xOlTlCNkhlAX7w sHZzw0XOrCGn2pXfzXh3554Y5ICVeIGXW6k5ad8H2fv7dAqRvsTLZcM46Yg2SXcZlM5pDepXdoCbR fmBMu1ak1j1+zGaVaisRXPK1prOni9o04+ESMgce/tNxOrQyn4UOrLSw4gvhJIYdFC+Lb061yecez JC8YblxXEabrsvXyp7jpGtv7ep/X/Ccmfm9q4H5R2+Lljv1Yu9diSa5DpRB0olMphuQmsAsHXMacY I9wUMA3PDOOaDf32SnDw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nUd4D-00EUBe-5Y; Wed, 16 Mar 2022 23:28:33 +0000 Received: from mail-pf1-x429.google.com ([2607:f8b0:4864:20::429]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nUd47-00EU7R-Mt for linux-riscv@lists.infradead.org; Wed, 16 Mar 2022 23:28:29 +0000 Received: by mail-pf1-x429.google.com with SMTP id d19so5365761pfv.7 for ; Wed, 16 Mar 2022 16:28:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; h=subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding:cc:from:to; bh=FX0vp498CDh+Q8VYRYD8Az++zOHOkwOZmBFCOJ/MPQk=; b=g7RGlpUtsYp0dL6ZURcJ9XqT0FTjuYFUg3R+dEpF965ZohDKCeoeGHfjtjiVSkGxEj lANdfMQl9/IBkJ/9UMg+SPHYCtMWSo9c4xaW9U1+sxAE128S6sObkonPfJPoKIzymiIw 09sK8rm+D+QlLwCSg7j5kwRZZ5Ze8MI5Jek627pUrimcrgKq4K2jJlUa/Pj8a2SElTV9 xl4ffwNapdts7992kA3feRtQzz/jnL60ZUCdKR4LQ7AoV/Y7AfVaxzeYDr7myiwH00f4 SRKTk+WG9MddPkkdOopQzO/1xBaFNEzmYY2D9huskq0oyJ4kYPwaGuIBtGRfGZgtAFr0 gBJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:cc:from:to; bh=FX0vp498CDh+Q8VYRYD8Az++zOHOkwOZmBFCOJ/MPQk=; b=hqQ+Y5bhVtKz4Nngz+t/qyMqLnrXJBDDmax23tRV8ygcfMWtliZmuiJztyfL1+nerM vZcGKFDqrOfxbxOFTuReyKr7XLE+6FG+SSRUu0cYQ94Dh9UsMzxYQWehJTFhGWW3eKNd xtlVV4QjBJ3SsgKDRhRz2DhXKokRE2nJCDUQvxyn8Z6YXrBMG/4Cc/4D/HN4lR4rWt/k nBbjiiEUG8JgBws8/rHu3vRA6uFJCyt3zjyWuy/BKFWKjwwbz3goZRX1Obx3Yngwgri2 tCGknCshKagZL0UzXRflUUnqrbz0yigTNVjuNfaiPnFRhu8bph04JNUwGQ1WYSHFlST+ Dk8w== X-Gm-Message-State: AOAM530CoGHqI57dBHq1dp5XrTGGnzWXU1hvOScMXfTCJrrnwK22q+bb UZgVEBBpBoDmLiT5SBOQIH6axg== X-Google-Smtp-Source: ABdhPJxPEnRTV51QRb/xegbfDS8UXc6ttxfzGlax57AVGxnNF0PWp6AH0Fu+hZMGJACkhYsvidNMYQ== X-Received: by 2002:a05:6a00:729:b0:4f7:77ed:c256 with SMTP id 9-20020a056a00072900b004f777edc256mr2013933pfm.1.1647473306932; Wed, 16 Mar 2022 16:28:26 -0700 (PDT) Received: from localhost ([12.3.194.138]) by smtp.gmail.com with ESMTPSA id ce3-20020a17090aff0300b001c51f47840csm3560019pjb.0.2022.03.16.16.28.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 16 Mar 2022 16:28:26 -0700 (PDT) Subject: [PATCH 2/5] asm-generic: ticket-lock: New generic ticket-based spinlock Date: Wed, 16 Mar 2022 16:25:57 -0700 Message-Id: <20220316232600.20419-3-palmer@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220316232600.20419-1-palmer@rivosinc.com> References: <20220316232600.20419-1-palmer@rivosinc.com> MIME-Version: 1.0 Cc: jonas@southpole.se, stefan.kristiansson@saunalahti.fi, shorne@gmail.com, mingo@redhat.com, Will Deacon , longman@redhat.com, boqun.feng@gmail.com, Paul Walmsley , Palmer Dabbelt , aou@eecs.berkeley.edu, Arnd Bergmann , jszhang@kernel.org, wangkefeng.wang@huawei.com, openrisc@lists.librecores.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, Palmer Dabbelt From: Palmer Dabbelt To: linux-riscv@lists.infradead.org, peterz@infradead.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220316_162827_792902_21E9EB98 X-CRM114-Status: GOOD ( 15.93 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org From: Peter Zijlstra This is a simple, fair spinlock. Specifically it doesn't have all the subtle memory model dependencies that qspinlock has, which makes it more suitable for simple systems as it is more likely to be correct. [Palmer: commit text] Signed-off-by: Palmer Dabbelt Signed-off-by: Peter Zijlstra (Intel) Acked-by: Waiman Long --- I have specifically not included Peter's SOB on this, as he sent his original patch without one. --- include/asm-generic/ticket-lock-types.h | 11 ++++ include/asm-generic/ticket-lock.h | 86 +++++++++++++++++++++++++ 2 files changed, 97 insertions(+) create mode 100644 include/asm-generic/ticket-lock-types.h create mode 100644 include/asm-generic/ticket-lock.h diff --git a/include/asm-generic/ticket-lock-types.h b/include/asm-generic/ticket-lock-types.h new file mode 100644 index 000000000000..829759aedda8 --- /dev/null +++ b/include/asm-generic/ticket-lock-types.h @@ -0,0 +1,11 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef __ASM_GENERIC_TICKET_LOCK_TYPES_H +#define __ASM_GENERIC_TICKET_LOCK_TYPES_H + +#include +typedef atomic_t arch_spinlock_t; + +#define __ARCH_SPIN_LOCK_UNLOCKED ATOMIC_INIT(0) + +#endif /* __ASM_GENERIC_TICKET_LOCK_TYPES_H */ diff --git a/include/asm-generic/ticket-lock.h b/include/asm-generic/ticket-lock.h new file mode 100644 index 000000000000..3f0d53e21a37 --- /dev/null +++ b/include/asm-generic/ticket-lock.h @@ -0,0 +1,86 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +/* + * 'Generic' ticket-lock implementation. + * + * It relies on atomic_fetch_add() having well defined forward progress + * guarantees under contention. If your architecture cannot provide this, stick + * to a test-and-set lock. + * + * It also relies on atomic_fetch_add() being safe vs smp_store_release() on a + * sub-word of the value. This is generally true for anything LL/SC although + * you'd be hard pressed to find anything useful in architecture specifications + * about this. If your architecture cannot do this you might be better off with + * a test-and-set. + * + * It further assumes atomic_*_release() + atomic_*_acquire() is RCpc and hence + * uses atomic_fetch_add() which is SC to create an RCsc lock. + * + * The implementation uses smp_cond_load_acquire() to spin, so if the + * architecture has WFE like instructions to sleep instead of poll for word + * modifications be sure to implement that (see ARM64 for example). + * + */ + +#ifndef __ASM_GENERIC_TICKET_LOCK_H +#define __ASM_GENERIC_TICKET_LOCK_H + +#include +#include + +static __always_inline void ticket_lock(arch_spinlock_t *lock) +{ + u32 val = atomic_fetch_add(1<<16, lock); /* SC, gives us RCsc */ + u16 ticket = val >> 16; + + if (ticket == (u16)val) + return; + + atomic_cond_read_acquire(lock, ticket == (u16)VAL); +} + +static __always_inline bool ticket_trylock(arch_spinlock_t *lock) +{ + u32 old = atomic_read(lock); + + if ((old >> 16) != (old & 0xffff)) + return false; + + return atomic_try_cmpxchg(lock, &old, old + (1<<16)); /* SC, for RCsc */ +} + +static __always_inline void ticket_unlock(arch_spinlock_t *lock) +{ + u16 *ptr = (u16 *)lock + __is_defined(__BIG_ENDIAN); + u32 val = atomic_read(lock); + + smp_store_release(ptr, (u16)val + 1); +} + +static __always_inline int ticket_is_locked(arch_spinlock_t *lock) +{ + u32 val = atomic_read(lock); + + return ((val >> 16) != (val & 0xffff)); +} + +static __always_inline int ticket_is_contended(arch_spinlock_t *lock) +{ + u32 val = atomic_read(lock); + + return (s16)((val >> 16) - (val & 0xffff)) > 1; +} + +static __always_inline int ticket_value_unlocked(arch_spinlock_t lock) +{ + return !ticket_is_locked(&lock); +} + +#define arch_spin_lock(l) ticket_lock(l) +#define arch_spin_trylock(l) ticket_trylock(l) +#define arch_spin_unlock(l) ticket_unlock(l) +#define arch_spin_is_locked(l) ticket_is_locked(l) +#define arch_spin_is_contended(l) ticket_is_contended(l) +#define arch_spin_value_unlocked(l) ticket_value_unlocked(l) + +#endif /* __ASM_GENERIC_TICKET_LOCK_H */ From patchwork Wed Mar 16 23:25:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Palmer Dabbelt X-Patchwork-Id: 12783291 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BD95DC4332F for ; Wed, 16 Mar 2022 23:28:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:To:From:Cc:MIME-Version:References: In-Reply-To:Message-Id:Date:Subject:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=2vQ6pmLKEutJFy0k2NGjbASdYQ3jEtUi8s/zEnv4D9A=; b=z0Cw5p5+nHNki4 xkyZJeMbmao1HWXfCIT9ZvbVjkyYTV8QOf5qLmZ+055YBsTiY+kL+WqiAqkoM8xo3+tq1ywDEEOBE dGJJ6dhJhkSUxcn0WpvKTzSeiE4NF1j3rp8mG4HCYq4jTFYbgSUlJuszQnJMPMjrNfa3x+awI61yF n9ylDYIKXrk0CeHxsxVKJoheoDAi2HOAZzTS18c0X55pNhecuQRvSP1sx5/7M9HdVEiJ2ym0tRBRi LNM6T9ydvYsaRImnooAtjk2Vx76jTlyH3SG0tlV2VjymI64ifTw5DOxCpZsEaIf2sRgFYNcX4qS55 IXTjMaC8Q2CR9U9HNHjw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nUd4F-00EUCl-Mw; Wed, 16 Mar 2022 23:28:35 +0000 Received: from mail-pf1-x430.google.com ([2607:f8b0:4864:20::430]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nUd49-00EU8H-UE for linux-riscv@lists.infradead.org; Wed, 16 Mar 2022 23:28:31 +0000 Received: by mail-pf1-x430.google.com with SMTP id s11so5328140pfu.13 for ; Wed, 16 Mar 2022 16:28:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; h=subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding:cc:from:to; bh=TemgJXj/dxR4o9xjuRPFVAtvCpC+RuCkyAl1o75jaTw=; b=5NAASYUylgaEPqqdOEpfJYNUGMgV0gFaeMRax5C5dSSZLOGAdqEJZnhq0wtD7zPoW0 G5ob4+N2yFpwbpch0DWhTknq6SOGVQuAlyx5O3Q4z+CabPYf0Q0nUxJYIRUm+1UXldia CUfn33UFTj/UQ6niZ6ytLeUqcODuQ8vUjiiE5/y4Cga4QKby77CDm6r7N0bPNRi39KC4 O/zvHvMm0khtUBVhRQ1vvm59igRk3I3+zGiCD9QZkEZ4zSYXLICNZRMaGnxXlOOI5pc8 96R6MSsMUafEA9jSOCcTsTp6f+fYDXiBsUqlxz1cBR4CM7sz772W9JvY9llYBqeaxTjL 2xnw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:cc:from:to; bh=TemgJXj/dxR4o9xjuRPFVAtvCpC+RuCkyAl1o75jaTw=; b=smNz1bghFcP5BJfpS8bC1+/MY6rUW9FpNam4InOPhSU/gUo5HZTojQkvDy3kWDHsg2 FtlrwRL8BrSMJ5Sl7+xGVFtKr55yw05FScDbKnRkeM7RocuqMwG4/wXmhWpQ3Fxxjmha VZ+m+fFxLffXaXzEYDFHFet7nuVkK11n3z5hVyNWGMFHoORrY2ToJ5bjpvBIruA7BjSN 3C4T3MooA9PG+fcMIHp2hKeiiuZOfdludJdUeBid78TwNxsrZ4qVSeAYVlPry1h3zKFU rtRw4n0qcXMx1QUx7H4jmot6Wk4h3FMGfoQW45EeaPT3Igl91czkT+B6tEO0HDFFdldE 4MYA== X-Gm-Message-State: AOAM531nIP3OJXTYKx4+C9W2Bq7tEBgckEZB15/vovDakEYEEm8bJIN+ TQEMsQ7ALHhye0N4aGgs+YOtRA== X-Google-Smtp-Source: ABdhPJzu2zL3dDL4hiAgSK4z9Dcw0idEICTak8foXBS3lSnxmO9pbBmuyGxzNgKWSyVr5slYVp3eQg== X-Received: by 2002:a05:6a00:1a89:b0:4f7:b90b:17ee with SMTP id e9-20020a056a001a8900b004f7b90b17eemr2233942pfv.46.1647473308139; Wed, 16 Mar 2022 16:28:28 -0700 (PDT) Received: from localhost ([12.3.194.138]) by smtp.gmail.com with ESMTPSA id d16-20020a056a00245000b004f7728a4346sm4436763pfj.79.2022.03.16.16.28.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 16 Mar 2022 16:28:27 -0700 (PDT) Subject: [PATCH 3/5] openrisc: Move to ticket-spinlock Date: Wed, 16 Mar 2022 16:25:58 -0700 Message-Id: <20220316232600.20419-4-palmer@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220316232600.20419-1-palmer@rivosinc.com> References: <20220316232600.20419-1-palmer@rivosinc.com> MIME-Version: 1.0 Cc: jonas@southpole.se, stefan.kristiansson@saunalahti.fi, shorne@gmail.com, mingo@redhat.com, Will Deacon , longman@redhat.com, boqun.feng@gmail.com, Paul Walmsley , Palmer Dabbelt , aou@eecs.berkeley.edu, Arnd Bergmann , jszhang@kernel.org, wangkefeng.wang@huawei.com, openrisc@lists.librecores.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, Palmer Dabbelt From: Palmer Dabbelt To: linux-riscv@lists.infradead.org, peterz@infradead.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220316_162830_056388_53E83AED X-CRM114-Status: GOOD ( 11.69 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org From: Peter Zijlstra We have no indications that openrisc meets the qspinlock requirements, so move to ticket-spinlock as that is more likey to be correct. Signed-off-by: Palmer Dabbelt Signed-off-by: Peter Zijlstra (Intel) --- I have specifically not included Peter's SOB on this, as he sent his original patch without one. --- arch/openrisc/Kconfig | 1 - arch/openrisc/include/asm/Kbuild | 5 ++--- arch/openrisc/include/asm/spinlock.h | 3 +-- arch/openrisc/include/asm/spinlock_types.h | 2 +- 4 files changed, 4 insertions(+), 7 deletions(-) diff --git a/arch/openrisc/Kconfig b/arch/openrisc/Kconfig index f724b3f1aeed..f5fa226362f6 100644 --- a/arch/openrisc/Kconfig +++ b/arch/openrisc/Kconfig @@ -30,7 +30,6 @@ config OPENRISC select HAVE_DEBUG_STACKOVERFLOW select OR1K_PIC select CPU_NO_EFFICIENT_FFS if !OPENRISC_HAVE_INST_FF1 - select ARCH_USE_QUEUED_SPINLOCKS select ARCH_USE_QUEUED_RWLOCKS select OMPIC if SMP select ARCH_WANT_FRAME_POINTERS diff --git a/arch/openrisc/include/asm/Kbuild b/arch/openrisc/include/asm/Kbuild index ca5987e11053..cb260e7d73db 100644 --- a/arch/openrisc/include/asm/Kbuild +++ b/arch/openrisc/include/asm/Kbuild @@ -1,9 +1,8 @@ # SPDX-License-Identifier: GPL-2.0 generic-y += extable.h generic-y += kvm_para.h -generic-y += mcs_spinlock.h -generic-y += qspinlock_types.h -generic-y += qspinlock.h +generic-y += ticket-lock.h +generic-y += ticket-lock-types.h generic-y += qrwlock_types.h generic-y += qrwlock.h generic-y += user.h diff --git a/arch/openrisc/include/asm/spinlock.h b/arch/openrisc/include/asm/spinlock.h index 264944a71535..40e4c9fdc349 100644 --- a/arch/openrisc/include/asm/spinlock.h +++ b/arch/openrisc/include/asm/spinlock.h @@ -15,8 +15,7 @@ #ifndef __ASM_OPENRISC_SPINLOCK_H #define __ASM_OPENRISC_SPINLOCK_H -#include - +#include #include #define arch_spin_relax(lock) cpu_relax() diff --git a/arch/openrisc/include/asm/spinlock_types.h b/arch/openrisc/include/asm/spinlock_types.h index 7c6fb1208c88..58ea31fa65ce 100644 --- a/arch/openrisc/include/asm/spinlock_types.h +++ b/arch/openrisc/include/asm/spinlock_types.h @@ -1,7 +1,7 @@ #ifndef _ASM_OPENRISC_SPINLOCK_TYPES_H #define _ASM_OPENRISC_SPINLOCK_TYPES_H -#include +#include #include #endif /* _ASM_OPENRISC_SPINLOCK_TYPES_H */ From patchwork Wed Mar 16 23:25:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Palmer Dabbelt X-Patchwork-Id: 12783292 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 25595C433FE for ; Wed, 16 Mar 2022 23:28:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:To:From:Cc:MIME-Version:References: In-Reply-To:Message-Id:Date:Subject:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=e7zQ2AwOScWuG5sQIBko7A3I1Mm5GMPqU9nRYgZDths=; b=hRp4qtlL+2TrjB 3sZyUu4d2TKDyxgsL2mdQQCvpl8XQVGtUl6Lval1UBbK49EMMTBov+nr4h2mNyHXs+vLtCpveVqxy ItxikqMZ9fjBdGx8vqBgyxeqD72/Tv7KdBcHRF5ACcPA9ItwDGa+pNinsIrojO8a1qIkwdxRSp6iD 3HOQoBznYv6WJAR54Xwzpqws2pYIvCFUAdmZxMIQ2ESFVEQ33Gb5pTUMGKUsFr+fFnVYUCgIvwEql puYs9uBBqOU7m9f7QMN7y13ZZtjpkWx6WT0B6bP9OWOP4Iq0ZnQApPTA1crn/Jwf1UWYyp1DRPLFh +eJ3bnYIyxOLl8b+H+5g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nUd4H-00EUDo-RJ; Wed, 16 Mar 2022 23:28:37 +0000 Received: from mail-pj1-x1030.google.com ([2607:f8b0:4864:20::1030]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nUd4A-00EU8d-7f for linux-riscv@lists.infradead.org; Wed, 16 Mar 2022 23:28:33 +0000 Received: by mail-pj1-x1030.google.com with SMTP id z12-20020a17090ad78c00b001bf022b69d6so3950599pju.2 for ; Wed, 16 Mar 2022 16:28:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; h=subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding:cc:from:to; bh=QjUbWpYW3rVnM/AEVOfQY7Hb1962iPWLkT7GwNacZrs=; b=OPswRH2lRihBIfhRfn+/3FQDyS4CIDrFb0a6W/SFptnXwSb197U4E15ipwXAHAI0Ta TNyvuSpdp0Dl7Udy9D2GfQMyBJv1EAU0uSev1KuzOjG/5dK0WBhl8Mbqd7ahL1zjtniM Y1SqPUOdGayiPKAU6YQE0nUK5pJvsw2r8oicK3P3V5ZhK6gt2WLY4clS8KYMvMnsh/qH t1+CgGbeMH5nQ/NzMh338/CKlZC6QrLpB6Ol4xXtu3THVc5XuN017XXitVKtMXD9lQqc x1VUjFX+iePIisj0C7WmE0/fu1p8BI2Bk3rrSnzxy2JNeg8GQP2Fx2KEuKzIbkX7wL9d r1iA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:cc:from:to; bh=QjUbWpYW3rVnM/AEVOfQY7Hb1962iPWLkT7GwNacZrs=; b=U79jEICau+FNOgkpeVzOVUrKSO7X8nfJzk4IDoia1AuRJqIXQMnidCMyZkGKI32iHX sAtuWKiJDm/unw5m1RSdZdhgO+ObCCLTnE4ZthbM0m5plPP6RpC8dLMjoBQBiWCMSKf+ 69gidJOa5HO0/gO5TZBMZI5i8oG5beeK8kcOYMuKBD4FQGvpWlKuTNnvcqjBcnwVYly7 QrN9UDC06gOYSV3vIJCBmsrxStazyQvzMgZtGL8LDMYrGUeXH38umTQqrgL9QEM8vvPL PvhHXycMO5n2XGjnzEy5UQzjNBCosWHny0evR0IzJr1ThHiUmNOeSqn+RsIeuLEaPSBB O1QA== X-Gm-Message-State: AOAM531OeDXFc53K6tT06JH1cZqFG0OR7dOpcRfFgQ49P3Ewc8wE3y6o 7wWBvn1cRsK+nLAqQNlpdcD16A== X-Google-Smtp-Source: ABdhPJx15zZafntzrXc+qFtN7TCKqNirxAH51x/OX1Z7DAEkhf7MHHxu+PKBTGIRQSByMNHXy1V3iA== X-Received: by 2002:a17:903:1206:b0:151:7d67:2924 with SMTP id l6-20020a170903120600b001517d672924mr1783497plh.45.1647473309449; Wed, 16 Mar 2022 16:28:29 -0700 (PDT) Received: from localhost ([12.3.194.138]) by smtp.gmail.com with ESMTPSA id pj13-20020a17090b4f4d00b001bf2ff56430sm8078615pjb.30.2022.03.16.16.28.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 16 Mar 2022 16:28:29 -0700 (PDT) Subject: [PATCH 4/5] RISC-V: Move to ticket-spinlocks Date: Wed, 16 Mar 2022 16:25:59 -0700 Message-Id: <20220316232600.20419-5-palmer@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220316232600.20419-1-palmer@rivosinc.com> References: <20220316232600.20419-1-palmer@rivosinc.com> MIME-Version: 1.0 Cc: jonas@southpole.se, stefan.kristiansson@saunalahti.fi, shorne@gmail.com, mingo@redhat.com, Will Deacon , longman@redhat.com, boqun.feng@gmail.com, Paul Walmsley , Palmer Dabbelt , aou@eecs.berkeley.edu, Arnd Bergmann , jszhang@kernel.org, wangkefeng.wang@huawei.com, openrisc@lists.librecores.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, Palmer Dabbelt From: Palmer Dabbelt To: linux-riscv@lists.infradead.org, peterz@infradead.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220316_162830_306580_1B136270 X-CRM114-Status: GOOD ( 11.89 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org From: Palmer Dabbelt Our existing spinlocks aren't fair and replacing them has been on the TODO list for a long time. This moves to the recently-introduced ticket spinlocks, which are simple enough that they are likely to be correct and fast on the vast majority of extant implementations. Signed-off-by: Palmer Dabbelt --- arch/riscv/include/asm/Kbuild | 2 ++ arch/riscv/include/asm/spinlock.h | 41 +------------------------ arch/riscv/include/asm/spinlock_types.h | 6 +--- 3 files changed, 4 insertions(+), 45 deletions(-) diff --git a/arch/riscv/include/asm/Kbuild b/arch/riscv/include/asm/Kbuild index 57b86fd9916c..42b1961af1a6 100644 --- a/arch/riscv/include/asm/Kbuild +++ b/arch/riscv/include/asm/Kbuild @@ -2,5 +2,7 @@ generic-y += early_ioremap.h generic-y += flat.h generic-y += kvm_para.h +generic-y += ticket-lock.h +generic-y += ticket-lock-types.h generic-y += user.h generic-y += vmlinux.lds.h diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h index f4f7fa1b7ca8..38089cbdea92 100644 --- a/arch/riscv/include/asm/spinlock.h +++ b/arch/riscv/include/asm/spinlock.h @@ -10,46 +10,7 @@ #include #include #include - -/* - * Simple spin lock operations. These provide no fairness guarantees. - */ - -/* FIXME: Replace this with a ticket lock, like MIPS. */ - -#define arch_spin_is_locked(x) (READ_ONCE((x)->lock) != 0) - -static inline void arch_spin_unlock(arch_spinlock_t *lock) -{ - smp_store_release(&lock->lock, 0); -} - -static inline int arch_spin_trylock(arch_spinlock_t *lock) -{ - int tmp = 1, busy; - - __asm__ __volatile__ ( - " amoswap.w %0, %2, %1\n" - RISCV_ACQUIRE_BARRIER - : "=r" (busy), "+A" (lock->lock) - : "r" (tmp) - : "memory"); - - return !busy; -} - -static inline void arch_spin_lock(arch_spinlock_t *lock) -{ - while (1) { - if (arch_spin_is_locked(lock)) - continue; - - if (arch_spin_trylock(lock)) - break; - } -} - -/***********************************************************/ +#include static inline void arch_read_lock(arch_rwlock_t *lock) { diff --git a/arch/riscv/include/asm/spinlock_types.h b/arch/riscv/include/asm/spinlock_types.h index 5a35a49505da..431ee08e26c4 100644 --- a/arch/riscv/include/asm/spinlock_types.h +++ b/arch/riscv/include/asm/spinlock_types.h @@ -10,11 +10,7 @@ # error "please don't include this file directly" #endif -typedef struct { - volatile unsigned int lock; -} arch_spinlock_t; - -#define __ARCH_SPIN_LOCK_UNLOCKED { 0 } +#include typedef struct { volatile unsigned int lock; From patchwork Wed Mar 16 23:26:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Palmer Dabbelt X-Patchwork-Id: 12783293 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D3548C433F5 for ; Wed, 16 Mar 2022 23:28:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:To:From:Cc:MIME-Version:References: In-Reply-To:Message-Id:Date:Subject:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=BdF8D0OcJvVpoBzre09XxF17FzzBpp6Cw9n2FjO7cb0=; b=xBMWXj5sZwMVOn UT6xYjBScOMONOUXsLQBer8xK91r34ytQzSAqYDtfnWuqzX8GQt+g2uwpPLR9/7YAGx/rDTrAABHz dtYk1XrTXHjrCvIcxYVAB1OWGj+QsauVST6ytXMDEdN8OGYEQhnLbq+CDCpUiEwKlBCsecnGZAZxR f1Clnq0jGFeo7ZKe/G66m7c0bn47DU503TIvB9aDkigLlSwnZWqPo1vzjdUN3JJ5yUE1iEk7aQSBf HuZxJ64fS0tTOR2aoXvffIk4jy/WbkbkOVz4GxsE4ue47O3SfKuu4Pbc39QzKLLswWrOriHxTRy5s pAZCMg4Zh35kb7inYLhQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nUd4J-00EUFI-VM; Wed, 16 Mar 2022 23:28:39 +0000 Received: from mail-pj1-x1035.google.com ([2607:f8b0:4864:20::1035]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nUd4B-00EUAY-HY for linux-riscv@lists.infradead.org; Wed, 16 Mar 2022 23:28:35 +0000 Received: by mail-pj1-x1035.google.com with SMTP id m22so3499726pja.0 for ; Wed, 16 Mar 2022 16:28:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; h=subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding:cc:from:to; bh=llt+unTQxxB47mMK9cTVf/sVk7YwdP8ahVPZ8vUzo+E=; b=P13QgI0zd+4RAOt3pbwZknViRr2CVhiVAFIRqM5bOGjAu9nzvjSnZusFhYRC5h9qSQ 9JCxra+cscVH+cq1tVm8CjiuMJlxW877K2j6AlCHNuh0DRBNi9EE4AcotfiNoC3XOOLK dhnYjAvGq4qBYfG/ws5BM4JI/1zMJoKWHl+gPx8IkFDWC/CuJ+vSOxvmg3M8IpV/BOKi Oz6nK2qVkXNZ02AyIkvsE3D3PHdimNMIXmdl8ruKlmv4JHoGQeGhFIjnxn/Mq7G4QX+y 2JDewQpoUe01b9wf6IjFRTdrHwJVTBH6ES8hoo446YGPwdfpUHyka6ajuquwNdtxfM3P F7kA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:cc:from:to; bh=llt+unTQxxB47mMK9cTVf/sVk7YwdP8ahVPZ8vUzo+E=; b=eYID6HUJNfsgoTFfUav9HC/C4zNCgBShcXVfy6JVRiOE569ToOxUE60swqC76kFvWm YEhm9wxNfB79XpQjOlE7T3lNXR4Da8ZfCbluzJzFvNn6ZoKhJXfAg8ij1CkHMd+VOoj/ BcewXTnjZrXiZaKNgK8cZzzDQ0UnqI38+Rtsu/ELlg2DsyQfJ2ML74/d01AnFvrjUQC9 g/ZQcglRlhdmHxXoTXMtWE11Oz9SKEUVEz04605HFQ59QJyqzbgYljUWYbWCw4bcgEAN Vp2ffzHJkjFf0vY80r/xgvRfhFx3BTnQmQFpDS7H5MdnhPwngbschEpWSsWTkxnZ/WiK c6jQ== X-Gm-Message-State: AOAM533SSkhZNKJvE1URuVwWmP1KBMpLmnIJNvOstcpwohFw8HjUZ03B /0f7pjHVzmqU134cQ7BPnLL9bg== X-Google-Smtp-Source: ABdhPJzyPV7prsqRJmPgkY9DrfVu2bdBaNJER6RIDfsjbbPdF+/QlL1mJF+2to5VA22BRA7tJfuvxA== X-Received: by 2002:a17:903:11c9:b0:151:9521:d5c7 with SMTP id q9-20020a17090311c900b001519521d5c7mr1923340plh.73.1647473310711; Wed, 16 Mar 2022 16:28:30 -0700 (PDT) Received: from localhost ([12.3.194.138]) by smtp.gmail.com with ESMTPSA id o5-20020a056a0015c500b004f7988f16c3sm4696034pfu.30.2022.03.16.16.28.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 16 Mar 2022 16:28:30 -0700 (PDT) Subject: [PATCH 5/5] RISC-V: Move to queued RW locks Date: Wed, 16 Mar 2022 16:26:00 -0700 Message-Id: <20220316232600.20419-6-palmer@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220316232600.20419-1-palmer@rivosinc.com> References: <20220316232600.20419-1-palmer@rivosinc.com> MIME-Version: 1.0 Cc: jonas@southpole.se, stefan.kristiansson@saunalahti.fi, shorne@gmail.com, mingo@redhat.com, Will Deacon , longman@redhat.com, boqun.feng@gmail.com, Paul Walmsley , Palmer Dabbelt , aou@eecs.berkeley.edu, Arnd Bergmann , jszhang@kernel.org, wangkefeng.wang@huawei.com, openrisc@lists.librecores.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, Palmer Dabbelt From: Palmer Dabbelt To: linux-riscv@lists.infradead.org, peterz@infradead.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220316_162831_595212_5B614AF8 X-CRM114-Status: GOOD ( 11.65 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org From: Palmer Dabbelt With the move to fair spinlocks, we might as well move to fair rwlocks. Signed-off-by: Palmer Dabbelt --- arch/riscv/Kconfig | 1 + arch/riscv/include/asm/Kbuild | 2 + arch/riscv/include/asm/spinlock.h | 82 +------------------------ arch/riscv/include/asm/spinlock_types.h | 7 +-- 4 files changed, 5 insertions(+), 87 deletions(-) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 5adcbd9b5e88..feb7030cfb6d 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -38,6 +38,7 @@ config RISCV select ARCH_SUPPORTS_DEBUG_PAGEALLOC if MMU select ARCH_SUPPORTS_HUGETLBFS if MMU select ARCH_USE_MEMTEST + select ARCH_USE_QUEUED_RWLOCKS select ARCH_WANT_DEFAULT_TOPDOWN_MMAP_LAYOUT if MMU select ARCH_WANT_FRAME_POINTERS select ARCH_WANT_HUGE_PMD_SHARE if 64BIT diff --git a/arch/riscv/include/asm/Kbuild b/arch/riscv/include/asm/Kbuild index 42b1961af1a6..e8714070cbb9 100644 --- a/arch/riscv/include/asm/Kbuild +++ b/arch/riscv/include/asm/Kbuild @@ -4,5 +4,7 @@ generic-y += flat.h generic-y += kvm_para.h generic-y += ticket-lock.h generic-y += ticket-lock-types.h +generic-y += qrwlock.h +generic-y += qrwlock_types.h generic-y += user.h generic-y += vmlinux.lds.h diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h index 38089cbdea92..97dfb150d18c 100644 --- a/arch/riscv/include/asm/spinlock.h +++ b/arch/riscv/include/asm/spinlock.h @@ -11,86 +11,6 @@ #include #include #include - -static inline void arch_read_lock(arch_rwlock_t *lock) -{ - int tmp; - - __asm__ __volatile__( - "1: lr.w %1, %0\n" - " bltz %1, 1b\n" - " addi %1, %1, 1\n" - " sc.w %1, %1, %0\n" - " bnez %1, 1b\n" - RISCV_ACQUIRE_BARRIER - : "+A" (lock->lock), "=&r" (tmp) - :: "memory"); -} - -static inline void arch_write_lock(arch_rwlock_t *lock) -{ - int tmp; - - __asm__ __volatile__( - "1: lr.w %1, %0\n" - " bnez %1, 1b\n" - " li %1, -1\n" - " sc.w %1, %1, %0\n" - " bnez %1, 1b\n" - RISCV_ACQUIRE_BARRIER - : "+A" (lock->lock), "=&r" (tmp) - :: "memory"); -} - -static inline int arch_read_trylock(arch_rwlock_t *lock) -{ - int busy; - - __asm__ __volatile__( - "1: lr.w %1, %0\n" - " bltz %1, 1f\n" - " addi %1, %1, 1\n" - " sc.w %1, %1, %0\n" - " bnez %1, 1b\n" - RISCV_ACQUIRE_BARRIER - "1:\n" - : "+A" (lock->lock), "=&r" (busy) - :: "memory"); - - return !busy; -} - -static inline int arch_write_trylock(arch_rwlock_t *lock) -{ - int busy; - - __asm__ __volatile__( - "1: lr.w %1, %0\n" - " bnez %1, 1f\n" - " li %1, -1\n" - " sc.w %1, %1, %0\n" - " bnez %1, 1b\n" - RISCV_ACQUIRE_BARRIER - "1:\n" - : "+A" (lock->lock), "=&r" (busy) - :: "memory"); - - return !busy; -} - -static inline void arch_read_unlock(arch_rwlock_t *lock) -{ - __asm__ __volatile__( - RISCV_RELEASE_BARRIER - " amoadd.w x0, %1, %0\n" - : "+A" (lock->lock) - : "r" (-1) - : "memory"); -} - -static inline void arch_write_unlock(arch_rwlock_t *lock) -{ - smp_store_release(&lock->lock, 0); -} +#include #endif /* _ASM_RISCV_SPINLOCK_H */ diff --git a/arch/riscv/include/asm/spinlock_types.h b/arch/riscv/include/asm/spinlock_types.h index 431ee08e26c4..3779f13706fa 100644 --- a/arch/riscv/include/asm/spinlock_types.h +++ b/arch/riscv/include/asm/spinlock_types.h @@ -11,11 +11,6 @@ #endif #include - -typedef struct { - volatile unsigned int lock; -} arch_rwlock_t; - -#define __ARCH_RW_LOCK_UNLOCKED { 0 } +#include #endif /* _ASM_RISCV_SPINLOCK_TYPES_H */