From patchwork Mon Nov 28 12:20:12 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Nicolai_H=C3=A4hnle?= X-Patchwork-Id: 9449487 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 9CD74600CB for ; Mon, 28 Nov 2016 12:21:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8CEF020453 for ; Mon, 28 Nov 2016 12:21:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 81CA727BFF; Mon, 28 Nov 2016 12:21:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=2.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, FREEMAIL_FROM, RCVD_IN_DNSWL_MED, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 385B320453 for ; Mon, 28 Nov 2016 12:21:28 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 55AD289C9A; Mon, 28 Nov 2016 12:21:26 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from mail-wj0-x241.google.com (mail-wj0-x241.google.com [IPv6:2a00:1450:400c:c01::241]) by gabe.freedesktop.org (Postfix) with ESMTPS id 691A56E2BD for ; Mon, 28 Nov 2016 12:20:34 +0000 (UTC) Received: by mail-wj0-x241.google.com with SMTP id kp2so13843918wjc.0 for ; Mon, 28 Nov 2016 04:20:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=d+iyAsmAkY8CDATXuOcIFlZXrWjGurT7GjQVNFz+GqI=; b=XJ0OT9gBGw30zv6VJIyroSqQAC+rBarNC8E1abfLWU97SXABdlDPdW6YOufvxxIZm6 VLcmdOxAMmFjQPjTLaKQNOy9LcjXzxvWUAahUmVhQ2DGJiT6T7V61U26szA19mbFkTCn geyKDjcTmwbgS1LEDvKIMH23ey+yv3wMdetX5n8noFNX/9A0Ix5SX3FLGDh7hF7LwX85 mYHSzn7hhXYBcCxdiY6mc3MdUddflQ4zLU200t9k0l0W2kx3DpG8KJRpC/Rx1badu2rk gZD1W35Txt5cR0k1sdcSOaQH+h/WHLiuLn1bYLtee3s6/EwTW09E+8wZy5ZN6219/WGl oEQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=d+iyAsmAkY8CDATXuOcIFlZXrWjGurT7GjQVNFz+GqI=; b=TBQwcC5xmp8RULIj3Fo5vDQpSqRu53lmj5pkHSqQxV4UWuMrZT+SCnQLMWffaF+CaY gKFoBM/XHzusnsHJQir6J2Wngsdcb+uRM+1fUp4Glj5vDEDFdgQuCeDDHi7ntgZ+iBQD EzSjPtovr/Fxo+Ef/JallbOltjgdQvvQBNEgWq45SsrrLKVwVIIVAPvmcKJE9gX/oNF8 03pVGtEi/KKX0HvKlpcN46EnAC5sh2z+JSCwblSpMqcoj9unb7fTIQBTTxIPCocNhIRz WnSwScVkR0GOlYnKD2+n6CrCUcNldRlZbGG3WGrgvgA7LGAH9Bfc7vKGSC54Lo3m0WlF P4MA== X-Gm-Message-State: AKaTC00RVWBoFUm3+C+cpLsJ9UF2wFqXYfLLKW7ZCBn56ss8vB0X2/lYTnLYCZsLalu1ug== X-Received: by 10.194.105.228 with SMTP id gp4mr438722wjb.208.1480335632862; Mon, 28 Nov 2016 04:20:32 -0800 (PST) Received: from cassiopeia.fritz.box ([2001:a61:1119:dc01:d88b:9432:f601:27ed]) by smtp.gmail.com with ESMTPSA id a13sm28830511wma.18.2016.11.28.04.20.31 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 28 Nov 2016 04:20:32 -0800 (PST) From: =?UTF-8?q?Nicolai=20H=C3=A4hnle?= To: linux-kernel@vger.kernel.org Subject: [PATCH 11/11] [rfc] locking/ww_mutex: Always spin optimistically for the first waiter Date: Mon, 28 Nov 2016 13:20:12 +0100 Message-Id: <1480335612-12069-12-git-send-email-nhaehnle@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1480335612-12069-1-git-send-email-nhaehnle@gmail.com> References: <1480335612-12069-1-git-send-email-nhaehnle@gmail.com> MIME-Version: 1.0 Cc: Maarten Lankhorst , =?UTF-8?q?Nicolai=20H=C3=A4hnle?= , Peter Zijlstra , dri-devel@lists.freedesktop.org, Ingo Molnar X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Nicolai Hähnle Check the current owner's context once against our stamp. If our stamp is lower, we continue to spin optimistically instead of backing off. This is correct with respect to deadlock detection because while the (owner, ww_ctx) pair may re-appear if the owner task manages to unlock and re-acquire the lock while we're spinning, the context could only have been re-initialized with an even higher stamp. We also still detect when we have to back off for other waiters that join the list while we're spinning. But taking the wait_lock in mutex_spin_on_owner feels iffy, even if it is done only once. Median timings taken of a contention-heavy GPU workload: Before: real 0m53.086s user 0m7.360s sys 1m46.204s After: real 0m52.577s user 0m7.544s sys 1m49.200s Cc: Peter Zijlstra Cc: Ingo Molnar Cc: Maarten Lankhorst Cc: Daniel Vetter Cc: Chris Wilson Cc: dri-devel@lists.freedesktop.org Signed-off-by: Nicolai Hähnle --- kernel/locking/mutex.c | 35 ++++++++++++++++++++++++++++++++--- 1 file changed, 32 insertions(+), 3 deletions(-) diff --git a/kernel/locking/mutex.c b/kernel/locking/mutex.c index ee84007..e7d5fac 100644 --- a/kernel/locking/mutex.c +++ b/kernel/locking/mutex.c @@ -378,6 +378,28 @@ bool mutex_spin_on_owner(struct mutex *lock, struct task_struct *owner, struct mutex_waiter *waiter) { bool ret = true; + struct ww_acquire_ctx *owner_ww_ctx = NULL; + + if (use_ww_ctx && ww_ctx && ww_ctx->acquired > 0) { + struct ww_mutex *ww; + unsigned long flags; + + ww = container_of(lock, struct ww_mutex, base); + + /* + * Check the stamp of the current owner once. This allows us + * to spin optimistically in the case where the current owner + * has a higher stamp than us. + */ + spin_lock_mutex(&lock->wait_lock, flags); + owner_ww_ctx = ww->ctx; + if (owner_ww_ctx && + __ww_mutex_stamp_after(ww_ctx, owner_ww_ctx)) { + spin_unlock_mutex(&lock->wait_lock, flags); + return false; + } + spin_unlock_mutex(&lock->wait_lock, flags); + } rcu_read_lock(); while (__mutex_owner(lock) == owner) { @@ -414,9 +436,16 @@ bool mutex_spin_on_owner(struct mutex *lock, struct task_struct *owner, * Check this in every inner iteration because we may * be racing against another thread's ww_mutex_lock. */ - if (ww_ctx->acquired > 0 && READ_ONCE(ww->ctx)) { - ret = false; - break; + if (ww_ctx->acquired > 0) { + struct ww_acquire_ctx *current_ctx; + + current_ctx = READ_ONCE(ww->ctx); + + if (current_ctx && + current_ctx != owner_ww_ctx) { + ret = false; + break; + } } /*