From patchwork Wed Feb 8 18:00:25 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Waiman Long X-Patchwork-Id: 9562951 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 5E88E60434 for ; Wed, 8 Feb 2017 18:03:22 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 684491FF14 for ; Wed, 8 Feb 2017 18:03:22 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5CAC628511; Wed, 8 Feb 2017 18:03:22 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id C7D5C1FF14 for ; Wed, 8 Feb 2017 18:03:21 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cbWYA-0005o1-Mi; Wed, 08 Feb 2017 18:01:02 +0000 Received: from mail6.bemta3.messagelabs.com ([195.245.230.39]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cbWY9-0005nW-BO for xen-devel@lists.xenproject.org; Wed, 08 Feb 2017 18:01:01 +0000 Received: from [85.158.137.68] by server-7.bemta-3.messagelabs.com id 70/64-23854-CDC5B985; Wed, 08 Feb 2017 18:01:00 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrILMWRWlGSWpSXmKPExsVysWW7jO7tmNk RBltvWFt83zKZyYHR4/CHKywBjFGsmXlJ+RUJrBnTJl5mLjghWPH9z3/mBsYTfF2MnBxCAnuZ JDY9ieti5AKyDzBKfPtziBkkwSagJvHnVicriC0icJxJ4vcMJZAiZoFnzBJXezvYQRLCAmESv zcdYgOxWQRUJY41TQSzeQWcJKbu/wPUzMHBKeAscXBWMYgpBBT+et0cpEJCQFvi1Mbv7BB2H6 PEzKmVExh5FjAyrGLUKE4tKkst0jUy0UsqykzPKMlNzMzRNTQw1stNLS5OTE/NSUwq1kvOz93 ECPR6PQMD4w7GV8f9DjFKcjApifLmRc2OEOJLyk+pzEgszogvKs1JLT7EKMPBoSTBeywaKCdY lJqeWpGWmQMMP5i0BAePkgjvJZA0b3FBYm5xZjpE6hSjopQ4rxUwaIUEQBIZpXlwbbCQv8QoK yXMy8jAwCDEU5BalJtZgir/ilGcg1FJmPcVyHiezLwSuOmvgBYzAS2+fnoWyOKSRISUVAPjCu Pz762sfF9Zr+LiTVLxmv5R/buxsC9ft0Sd+Dd9vwV9Zl45BxirP5kpC62ZUNgaI2IvUXLBU5B hpYbGiqveZx4EKb2L6v8/OaDz+7dV6p+Z1efO/rvLXav508kz89yF/ea6qM38GLLK3VJp6bVr 79y/WhudPGoSY3S9Q72Ee6PNS7UZofxKLMUZiYZazEXFiQDXqBpedAIAAA== X-Env-Sender: longman@redhat.com X-Msg-Ref: server-4.tower-31.messagelabs.com!1486576858!26573840!1 X-Originating-IP: [209.132.183.28] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogMjA5LjEzMi4xODMuMjggPT4gNTQwNjQ=\n X-StarScan-Received: X-StarScan-Version: 9.1.1; banners=-,-,- X-VirusChecked: Checked Received: (qmail 40208 invoked from network); 8 Feb 2017 18:00:59 -0000 Received: from mx1.redhat.com (HELO mx1.redhat.com) (209.132.183.28) by server-4.tower-31.messagelabs.com with DHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 8 Feb 2017 18:00:59 -0000 Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id D8EAFC05AA4E; Wed, 8 Feb 2017 18:00:58 +0000 (UTC) Received: from llong.com (dhcp-17-128.bos.redhat.com [10.18.17.128]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v18I0q3L031426; Wed, 8 Feb 2017 13:00:56 -0500 From: Waiman Long To: Jeremy Fitzhardinge , Chris Wright , Alok Kataria , Rusty Russell , Peter Zijlstra , Ingo Molnar , Thomas Gleixner , "H. Peter Anvin" Date: Wed, 8 Feb 2017 13:00:25 -0500 Message-Id: <1486576825-17058-2-git-send-email-longman@redhat.com> In-Reply-To: <1486576825-17058-1-git-send-email-longman@redhat.com> References: <1486576825-17058-1-git-send-email-longman@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Wed, 08 Feb 2017 18:00:59 +0000 (UTC) Cc: linux-arch@vger.kernel.org, Juergen Gross , kvm@vger.kernel.org, =?UTF-8?q?Radim=20Kr=C4=8Dm=C3=A1=C5=99?= , Pan Xinhui , x86@kernel.org, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, Waiman Long , Paolo Bonzini , xen-devel@lists.xenproject.org, Boris Ostrovsky Subject: [Xen-devel] [PATCH 2/2] locking/mutex, rwsem: Reduce vcpu_is_preempted() calling frequency X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP As the vcpu_is_preempted() call is pretty costly compared with other checks within mutex_spin_on_owner() and rwsem_spin_on_owner(), they are done at a reduce frequency of once every 256 iterations. Signed-off-by: Waiman Long --- kernel/locking/mutex.c | 5 ++++- kernel/locking/rwsem-xadd.c | 6 ++++-- 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/kernel/locking/mutex.c b/kernel/locking/mutex.c index ad2d9e2..2ece0c4 100644 --- a/kernel/locking/mutex.c +++ b/kernel/locking/mutex.c @@ -423,6 +423,7 @@ bool mutex_spin_on_owner(struct mutex *lock, struct task_struct *owner, struct ww_acquire_ctx *ww_ctx, struct mutex_waiter *waiter) { bool ret = true; + int loop = 0; rcu_read_lock(); while (__mutex_owner(lock) == owner) { @@ -436,9 +437,11 @@ bool mutex_spin_on_owner(struct mutex *lock, struct task_struct *owner, /* * Use vcpu_is_preempted to detect lock holder preemption issue. + * As vcpu_is_preempted is more costly to use, it is called at + * a reduced frequencey (once every 256 iterations). */ if (!owner->on_cpu || need_resched() || - vcpu_is_preempted(task_cpu(owner))) { + (!(++loop & 0xff) && vcpu_is_preempted(task_cpu(owner)))) { ret = false; break; } diff --git a/kernel/locking/rwsem-xadd.c b/kernel/locking/rwsem-xadd.c index 2ad8d8d..7a884a6 100644 --- a/kernel/locking/rwsem-xadd.c +++ b/kernel/locking/rwsem-xadd.c @@ -351,6 +351,7 @@ static inline bool rwsem_can_spin_on_owner(struct rw_semaphore *sem) static noinline bool rwsem_spin_on_owner(struct rw_semaphore *sem) { struct task_struct *owner = READ_ONCE(sem->owner); + int loop = 0; if (!rwsem_owner_is_writer(owner)) goto out; @@ -367,10 +368,11 @@ static noinline bool rwsem_spin_on_owner(struct rw_semaphore *sem) /* * abort spinning when need_resched or owner is not running or - * owner's cpu is preempted. + * owner's cpu is preempted. The preemption check is done at + * a lower frequencey because of its high cost. */ if (!owner->on_cpu || need_resched() || - vcpu_is_preempted(task_cpu(owner))) { + (!(++loop & 0xff) && vcpu_is_preempted(task_cpu(owner)))) { rcu_read_unlock(); return false; }