From patchwork Fri Mar 13 08:05:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?b?SsO8cmdlbiBHcm/Dnw==?= X-Patchwork-Id: 11436189 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 098836CA for ; Fri, 13 Mar 2020 08:06:15 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E4006206E2 for ; Fri, 13 Mar 2020 08:06:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E4006206E2 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1jCfJr-0002Nl-6g; Fri, 13 Mar 2020 08:05:23 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1jCfJq-0002NV-5q for xen-devel@lists.xenproject.org; Fri, 13 Mar 2020 08:05:22 +0000 X-Inumbo-ID: 621af450-6501-11ea-b29b-12813bfff9fa Received: from mx2.suse.de (unknown [195.135.220.15]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 621af450-6501-11ea-b29b-12813bfff9fa; Fri, 13 Mar 2020 08:05:20 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx2.suse.de (Postfix) with ESMTP id CAB13ACD0; Fri, 13 Mar 2020 08:05:19 +0000 (UTC) From: Juergen Gross To: xen-devel@lists.xenproject.org Date: Fri, 13 Mar 2020 09:05:16 +0100 Message-Id: <20200313080517.28728-2-jgross@suse.com> X-Mailer: git-send-email 2.16.4 In-Reply-To: <20200313080517.28728-1-jgross@suse.com> References: <20200313080517.28728-1-jgross@suse.com> Subject: [Xen-devel] [PATCH 1/2] xen/rwlocks: call preempt_disable() when taking a rwlock X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Juergen Gross , Stefano Stabellini , Julien Grall , Wei Liu , Andrew Cooper , Ian Jackson , George Dunlap , Jan Beulich MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Similar to spinlocks preemption should be disabled while holding a rwlock. Signed-off-by: Juergen Gross --- xen/include/xen/rwlock.h | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-) diff --git a/xen/include/xen/rwlock.h b/xen/include/xen/rwlock.h index 1c221dd0d9..4ee341a182 100644 --- a/xen/include/xen/rwlock.h +++ b/xen/include/xen/rwlock.h @@ -2,6 +2,7 @@ #define __RWLOCK_H__ #include +#include #include #include @@ -57,10 +58,12 @@ static inline int _read_trylock(rwlock_t *lock) cnts = atomic_read(&lock->cnts); if ( likely(_can_read_lock(cnts)) ) { + preempt_disable(); cnts = (u32)atomic_add_return(_QR_BIAS, &lock->cnts); if ( likely(_can_read_lock(cnts)) ) return 1; atomic_sub(_QR_BIAS, &lock->cnts); + preempt_enable(); } return 0; } @@ -73,6 +76,7 @@ static inline void _read_lock(rwlock_t *lock) { u32 cnts; + preempt_disable(); cnts = atomic_add_return(_QR_BIAS, &lock->cnts); if ( likely(_can_read_lock(cnts)) ) return; @@ -106,6 +110,7 @@ static inline void _read_unlock(rwlock_t *lock) * Atomically decrement the reader count */ atomic_sub(_QR_BIAS, &lock->cnts); + preempt_enable(); } static inline void _read_unlock_irq(rwlock_t *lock) @@ -137,6 +142,7 @@ static inline unsigned int _write_lock_val(void) static inline void _write_lock(rwlock_t *lock) { /* Optimize for the unfair lock case where the fair flag is 0. */ + preempt_disable(); if ( atomic_cmpxchg(&lock->cnts, 0, _write_lock_val()) == 0 ) return; @@ -172,13 +178,21 @@ static inline int _write_trylock(rwlock_t *lock) if ( unlikely(cnts) ) return 0; - return likely(atomic_cmpxchg(&lock->cnts, 0, _write_lock_val()) == 0); + preempt_disable(); + if ( unlikely(atomic_cmpxchg(&lock->cnts, 0, _write_lock_val()) != 0) ) + { + preempt_enable(); + return 0; + } + + return 1; } static inline void _write_unlock(rwlock_t *lock) { ASSERT(_is_write_locked_by_me(atomic_read(&lock->cnts))); atomic_and(~(_QW_CPUMASK | _QW_WMASK), &lock->cnts); + preempt_enable(); } static inline void _write_unlock_irq(rwlock_t *lock) @@ -274,6 +288,7 @@ static inline void _percpu_read_lock(percpu_rwlock_t **per_cpudata, } /* Indicate this cpu is reading. */ + preempt_disable(); this_cpu_ptr(per_cpudata) = percpu_rwlock; smp_mb(); /* Check if a writer is waiting. */ @@ -308,6 +323,7 @@ static inline void _percpu_read_unlock(percpu_rwlock_t **per_cpudata, return; } this_cpu_ptr(per_cpudata) = NULL; + preempt_enable(); smp_wmb(); }