From patchwork Fri Mar 13 08:05:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?b?SsO8cmdlbiBHcm/Dnw==?= X-Patchwork-Id: 11436189 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 098836CA for ; Fri, 13 Mar 2020 08:06:15 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E4006206E2 for ; Fri, 13 Mar 2020 08:06:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E4006206E2 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1jCfJr-0002Nl-6g; Fri, 13 Mar 2020 08:05:23 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1jCfJq-0002NV-5q for xen-devel@lists.xenproject.org; Fri, 13 Mar 2020 08:05:22 +0000 X-Inumbo-ID: 621af450-6501-11ea-b29b-12813bfff9fa Received: from mx2.suse.de (unknown [195.135.220.15]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 621af450-6501-11ea-b29b-12813bfff9fa; Fri, 13 Mar 2020 08:05:20 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx2.suse.de (Postfix) with ESMTP id CAB13ACD0; Fri, 13 Mar 2020 08:05:19 +0000 (UTC) From: Juergen Gross To: xen-devel@lists.xenproject.org Date: Fri, 13 Mar 2020 09:05:16 +0100 Message-Id: <20200313080517.28728-2-jgross@suse.com> X-Mailer: git-send-email 2.16.4 In-Reply-To: <20200313080517.28728-1-jgross@suse.com> References: <20200313080517.28728-1-jgross@suse.com> Subject: [Xen-devel] [PATCH 1/2] xen/rwlocks: call preempt_disable() when taking a rwlock X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Juergen Gross , Stefano Stabellini , Julien Grall , Wei Liu , Andrew Cooper , Ian Jackson , George Dunlap , Jan Beulich MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Similar to spinlocks preemption should be disabled while holding a rwlock. Signed-off-by: Juergen Gross --- xen/include/xen/rwlock.h | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-) diff --git a/xen/include/xen/rwlock.h b/xen/include/xen/rwlock.h index 1c221dd0d9..4ee341a182 100644 --- a/xen/include/xen/rwlock.h +++ b/xen/include/xen/rwlock.h @@ -2,6 +2,7 @@ #define __RWLOCK_H__ #include +#include #include #include @@ -57,10 +58,12 @@ static inline int _read_trylock(rwlock_t *lock) cnts = atomic_read(&lock->cnts); if ( likely(_can_read_lock(cnts)) ) { + preempt_disable(); cnts = (u32)atomic_add_return(_QR_BIAS, &lock->cnts); if ( likely(_can_read_lock(cnts)) ) return 1; atomic_sub(_QR_BIAS, &lock->cnts); + preempt_enable(); } return 0; } @@ -73,6 +76,7 @@ static inline void _read_lock(rwlock_t *lock) { u32 cnts; + preempt_disable(); cnts = atomic_add_return(_QR_BIAS, &lock->cnts); if ( likely(_can_read_lock(cnts)) ) return; @@ -106,6 +110,7 @@ static inline void _read_unlock(rwlock_t *lock) * Atomically decrement the reader count */ atomic_sub(_QR_BIAS, &lock->cnts); + preempt_enable(); } static inline void _read_unlock_irq(rwlock_t *lock) @@ -137,6 +142,7 @@ static inline unsigned int _write_lock_val(void) static inline void _write_lock(rwlock_t *lock) { /* Optimize for the unfair lock case where the fair flag is 0. */ + preempt_disable(); if ( atomic_cmpxchg(&lock->cnts, 0, _write_lock_val()) == 0 ) return; @@ -172,13 +178,21 @@ static inline int _write_trylock(rwlock_t *lock) if ( unlikely(cnts) ) return 0; - return likely(atomic_cmpxchg(&lock->cnts, 0, _write_lock_val()) == 0); + preempt_disable(); + if ( unlikely(atomic_cmpxchg(&lock->cnts, 0, _write_lock_val()) != 0) ) + { + preempt_enable(); + return 0; + } + + return 1; } static inline void _write_unlock(rwlock_t *lock) { ASSERT(_is_write_locked_by_me(atomic_read(&lock->cnts))); atomic_and(~(_QW_CPUMASK | _QW_WMASK), &lock->cnts); + preempt_enable(); } static inline void _write_unlock_irq(rwlock_t *lock) @@ -274,6 +288,7 @@ static inline void _percpu_read_lock(percpu_rwlock_t **per_cpudata, } /* Indicate this cpu is reading. */ + preempt_disable(); this_cpu_ptr(per_cpudata) = percpu_rwlock; smp_mb(); /* Check if a writer is waiting. */ @@ -308,6 +323,7 @@ static inline void _percpu_read_unlock(percpu_rwlock_t **per_cpudata, return; } this_cpu_ptr(per_cpudata) = NULL; + preempt_enable(); smp_wmb(); } From patchwork Fri Mar 13 08:05:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?b?SsO8cmdlbiBHcm/Dnw==?= X-Patchwork-Id: 11436193 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2AA3E913 for ; Fri, 13 Mar 2020 08:06:24 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 11ABE206E2 for ; Fri, 13 Mar 2020 08:06:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 11ABE206E2 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1jCfJw-0002PQ-Fd; Fri, 13 Mar 2020 08:05:28 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1jCfJv-0002P5-0F for xen-devel@lists.xenproject.org; Fri, 13 Mar 2020 08:05:27 +0000 X-Inumbo-ID: 621a4456-6501-11ea-bec1-bc764e2007e4 Received: from mx2.suse.de (unknown [195.135.220.15]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 621a4456-6501-11ea-bec1-bc764e2007e4; Fri, 13 Mar 2020 08:05:20 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx2.suse.de (Postfix) with ESMTP id 0456CAD03; Fri, 13 Mar 2020 08:05:20 +0000 (UTC) From: Juergen Gross To: xen-devel@lists.xenproject.org Date: Fri, 13 Mar 2020 09:05:17 +0100 Message-Id: <20200313080517.28728-3-jgross@suse.com> X-Mailer: git-send-email 2.16.4 In-Reply-To: <20200313080517.28728-1-jgross@suse.com> References: <20200313080517.28728-1-jgross@suse.com> Subject: [Xen-devel] [PATCH 2/2] xen/spinlocks: fix placement of preempt_[dis|en]able() X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Juergen Gross , Stefano Stabellini , Julien Grall , Wei Liu , Andrew Cooper , Ian Jackson , George Dunlap , Jan Beulich MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" In case Xen ever gains preemption support the spinlock coding's placement of preempt_disable() and preempt_enable() should be outside of the locked section. Signed-off-by: Juergen Gross --- xen/common/spinlock.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/xen/common/spinlock.c b/xen/common/spinlock.c index 344981c54a..f05fb068cd 100644 --- a/xen/common/spinlock.c +++ b/xen/common/spinlock.c @@ -160,6 +160,7 @@ void inline _spin_lock_cb(spinlock_t *lock, void (*cb)(void *), void *data) LOCK_PROFILE_VAR; check_lock(&lock->debug); + preempt_disable(); tickets.head_tail = arch_fetch_and_add(&lock->tickets.head_tail, tickets.head_tail); while ( tickets.tail != observe_head(&lock->tickets) ) @@ -171,7 +172,6 @@ void inline _spin_lock_cb(spinlock_t *lock, void (*cb)(void *), void *data) } got_lock(&lock->debug); LOCK_PROFILE_GOT; - preempt_disable(); arch_lock_acquire_barrier(); } @@ -199,10 +199,10 @@ unsigned long _spin_lock_irqsave(spinlock_t *lock) void _spin_unlock(spinlock_t *lock) { arch_lock_release_barrier(); - preempt_enable(); LOCK_PROFILE_REL; rel_lock(&lock->debug); add_sized(&lock->tickets.head, 1); + preempt_enable(); arch_lock_signal(); } @@ -242,15 +242,18 @@ int _spin_trylock(spinlock_t *lock) return 0; new = old; new.tail++; + preempt_disable(); if ( cmpxchg(&lock->tickets.head_tail, old.head_tail, new.head_tail) != old.head_tail ) + { + preempt_enable(); return 0; + } got_lock(&lock->debug); #ifdef CONFIG_DEBUG_LOCK_PROFILE if (lock->profile) lock->profile->time_locked = NOW(); #endif - preempt_disable(); /* * cmpxchg() is a full barrier so no need for an * arch_lock_acquire_barrier().