From patchwork Tue Jul 16 20:19:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11046691 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6ADF617E0 for ; Tue, 16 Jul 2019 20:19:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5B39C28657 for ; Tue, 16 Jul 2019 20:19:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4F7E1286BF; Tue, 16 Jul 2019 20:19:37 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DB51028657 for ; Tue, 16 Jul 2019 20:19:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728579AbfGPUTg (ORCPT ); Tue, 16 Jul 2019 16:19:36 -0400 Received: from mail-qt1-f193.google.com ([209.85.160.193]:39038 "EHLO mail-qt1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728137AbfGPUTg (ORCPT ); Tue, 16 Jul 2019 16:19:36 -0400 Received: by mail-qt1-f193.google.com with SMTP id l9so20978412qtu.6 for ; Tue, 16 Jul 2019 13:19:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=ZWvLpiy8LsbkNpgxUihtSHfUQdquxnVHPH9fwkxLy2U=; b=rQjEeIi3+bfHage+YBJh/Hl4NhqEclZqjhzRRdUWBzduXxbVOamfeXMfZ4+HpzkE5g vLBAQMwsghkmFvCvyWdcuRjRqEekqrh6bwXBRwtJO5cz5itSGTifrQK7aViKRavn85RZ 62/nqxG+tgMsm18Y7Ialoupxwb+xXVZK32EX/eZL09yFCOHYDvwJA66DbQM7OfLxY6Bu +WsmXW06O7wgsuOfFbrXD44ILmxH1+wvqVcBryd8CyyJtCX2XgV6XB6VDmFjRIpeoXd3 3Oaws/VeTyCLco1Sd3LFOabGp5l9or6xNT2K0yreMyS6QH7RH5Lmij+A4hKAtooGP2Nz X0Hw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=ZWvLpiy8LsbkNpgxUihtSHfUQdquxnVHPH9fwkxLy2U=; b=e1jnt9Pt/56h0jdi5Kz5bXrnWMEuclqwCBuElPQD5YhNwNg+I2EDfyAjnuaUkTpGmH IuKd2q1iA5Vkw8XojgjjNctlFTj/14K6SiBjUtCjs+QMeAz9ThVvrOiwY9aus2ELBgjc tiW3a++T8Mx/xEa/6dGVQW5V8y90iYZIL/0/iLn9dORn3cbc4OKisGNJiXDdexSRpDxa 4GV3BATWvG2uid/eSnIpxnhsSjuQflQdpQB9V5uI1OaXd23vj8+TScgOIBCzpvmwVoxj psTGUj66oWVaifUAhrcJrVOa+v7J8FYjM0PSTMh68ul2h1SzAQbLQCUUPGepu2Ywx/eK QuOg== X-Gm-Message-State: APjAAAU1Xz0suP7zqFJmfriaMR9qw+76vGJWnM/JwmLQeEYkotTDWSvQ athayBx2n3joKx5J/eyk0Jg= X-Google-Smtp-Source: APXvYqwTXHy1G9KQCSKkWAoL3wLWOeIyHfWtkmdoI1getlh48C3Ypjb1f2yZmQXw54xF482q/rQDeg== X-Received: by 2002:a0c:e001:: with SMTP id j1mr25989191qvk.110.1563308375432; Tue, 16 Jul 2019 13:19:35 -0700 (PDT) Received: from localhost ([2620:10d:c091:480::d7f3]) by smtp.gmail.com with ESMTPSA id l19sm13326988qtb.6.2019.07.16.13.19.34 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 16 Jul 2019 13:19:34 -0700 (PDT) From: Josef Bacik To: axboe@kernel.dk, kernel-team@fb.com, linux-block@vger.kernel.org, peterz@infradead.org, oleg@redhat.com Subject: [PATCH 1/5] wait: add wq_has_single_sleeper helper Date: Tue, 16 Jul 2019 16:19:25 -0400 Message-Id: <20190716201929.79142-2-josef@toxicpanda.com> X-Mailer: git-send-email 2.13.5 In-Reply-To: <20190716201929.79142-1-josef@toxicpanda.com> References: <20190716201929.79142-1-josef@toxicpanda.com> Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP rq-qos sits in the io path so we want to take locks as sparingly as possible. To accomplish this we try not to take the waitqueue head lock unless we are sure we need to go to sleep, and we have an optimization to make sure that we don't starve out existing waiters. Since we check if there are existing waiters locklessly we need to be able to update our view of the waitqueue list after we've added ourselves to the waitqueue. Accomplish this by adding this helper to see if there is more than just ourselves on the list. Signed-off-by: Josef Bacik --- include/linux/wait.h | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/include/linux/wait.h b/include/linux/wait.h index b6f77cf60dd7..30c515520fb2 100644 --- a/include/linux/wait.h +++ b/include/linux/wait.h @@ -126,6 +126,19 @@ static inline int waitqueue_active(struct wait_queue_head *wq_head) return !list_empty(&wq_head->head); } +/** + * wq_has_single_sleeper - check if there is only one sleeper + * @wq_head: wait queue head + * + * Returns true of wq_head has only one sleeper on the list. + * + * Please refer to the comment for waitqueue_active. + */ +static inline bool wq_has_single_sleeper(struct wait_queue_head *wq_head) +{ + return list_is_singular(&wq_head->head); +} + /** * wq_has_sleeper - check if there are any waiting processes * @wq_head: wait queue head From patchwork Tue Jul 16 20:19:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11046693 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 487E5912 for ; Tue, 16 Jul 2019 20:19:39 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 386CD28657 for ; Tue, 16 Jul 2019 20:19:39 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2CD53286BF; Tue, 16 Jul 2019 20:19:39 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D048628657 for ; Tue, 16 Jul 2019 20:19:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388518AbfGPUTi (ORCPT ); Tue, 16 Jul 2019 16:19:38 -0400 Received: from mail-qt1-f196.google.com ([209.85.160.196]:36077 "EHLO mail-qt1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728137AbfGPUTi (ORCPT ); Tue, 16 Jul 2019 16:19:38 -0400 Received: by mail-qt1-f196.google.com with SMTP id z4so21006784qtc.3 for ; Tue, 16 Jul 2019 13:19:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=MB4gVXqWNYaKJnrlVX20etvhbnggMPAbdHNsnWIPeXs=; b=E0TsVungQ0CRAB8CPusKzkgnknOYKTXIDOYzfPHQXdujxSJjUiCBTmQrMRVdGc5Eei cklNvU3oWTctho8seddoap//2XJC7ukV1tKaw3fGfd1PZZJ4eQRuT34UBcD5EOnM2YMA tBNsVumhOIEhFoTGm0WmERWdpAeQb4VwvIYfPzWJEMi4xSzs+Utha4IsoLYvNOjziKCC TiK2K41IYoc729YHqEZsfjJr7OaPZ56UgKTzzoJiJvSbeXyKCvIy9pz5SP/aJC0juYJA 6onZVWyvS1sFAUl6nmGtveO2nyZVZE+HumuHK335UcJ3jIJ4S0c1IUX6ksyaEFxJip5C nWbQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=MB4gVXqWNYaKJnrlVX20etvhbnggMPAbdHNsnWIPeXs=; b=HM3rPcRqJZQuRf+FGJWoSaQY25TG4cOjJTxNSfZN2zm3DxPiPQTfDT6P84XJyQKjtH 6DyrpmHCpB8V5JNkY8MeEJnyQAGYgotEZjutguuSW9xefITT++YSt/qI+3m2wdDNwhES L70hWXxBGrct59HzmnK0cjH0oKRb5dRIuLFGAu2UQGvtY84/E1yuS5fB1ohM6+lyxaG1 lgjvY7vSAjWaAyQfvywDgO2vkQxJKzBRvCSlvQ46g9xQi8VRZsDXWLLLExZAjTnMYJS5 1AYZeGpUYDFbB/Q/d0S4iZs22Y7sTz7439I+D7KCFKEhLcrblJe87YlUbB/FKZC3ZHSP 8llA== X-Gm-Message-State: APjAAAWQolazDRrg1JJtJQm7qfnsWE0K+TwRzmJx5n/UUkhL7jkJ6XwN u+u4T3oJ0/Y/0AQ4+qDGdx8= X-Google-Smtp-Source: APXvYqzJJtrhLlNXUoj6dGJO4DPR80pYH7IJTLsI4yzx7sjdGMgUa65FO3qLauCNZ0onWm787lf+Xw== X-Received: by 2002:a0c:9214:: with SMTP id a20mr26002629qva.195.1563308377340; Tue, 16 Jul 2019 13:19:37 -0700 (PDT) Received: from localhost ([2620:10d:c091:480::d7f3]) by smtp.gmail.com with ESMTPSA id m5sm9285237qkb.117.2019.07.16.13.19.36 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 16 Jul 2019 13:19:36 -0700 (PDT) From: Josef Bacik To: axboe@kernel.dk, kernel-team@fb.com, linux-block@vger.kernel.org, peterz@infradead.org, oleg@redhat.com Subject: [PATCH 2/5] rq-qos: fix missed wake-ups in rq_qos_throttle Date: Tue, 16 Jul 2019 16:19:26 -0400 Message-Id: <20190716201929.79142-3-josef@toxicpanda.com> X-Mailer: git-send-email 2.13.5 In-Reply-To: <20190716201929.79142-1-josef@toxicpanda.com> References: <20190716201929.79142-1-josef@toxicpanda.com> Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We saw a hang in production with WBT where there was only one waiter in the throttle path and no outstanding IO. This is because of the has_sleepers optimization that is used to make sure we don't steal an inflight counter for new submitters when there are people already on the list. We can race with our check to see if the waitqueue has any waiters (this is done locklessly) and the time we actually add ourselves to the waitqueue. If this happens we'll go to sleep and never be woken up because nobody is doing IO to wake us up. Fix this by checking if the waitqueue has a single sleeper on the list after we add ourselves, that way we have an uptodate view of the list. Signed-off-by: Josef Bacik --- block/blk-rq-qos.c | 1 + 1 file changed, 1 insertion(+) diff --git a/block/blk-rq-qos.c b/block/blk-rq-qos.c index 659ccb8b693f..67a0a4c07060 100644 --- a/block/blk-rq-qos.c +++ b/block/blk-rq-qos.c @@ -244,6 +244,7 @@ void rq_qos_wait(struct rq_wait *rqw, void *private_data, return; prepare_to_wait_exclusive(&rqw->wait, &data.wq, TASK_UNINTERRUPTIBLE); + has_sleeper = !wq_has_single_sleeper(&rqw->wait); do { if (data.got_token) break; From patchwork Tue Jul 16 20:19:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11046695 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5ADEF912 for ; Tue, 16 Jul 2019 20:19:42 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4B3F728657 for ; Tue, 16 Jul 2019 20:19:42 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3FFBA286BF; Tue, 16 Jul 2019 20:19:42 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1C95428657 for ; Tue, 16 Jul 2019 20:19:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388536AbfGPUTk (ORCPT ); Tue, 16 Jul 2019 16:19:40 -0400 Received: from mail-qk1-f195.google.com ([209.85.222.195]:42442 "EHLO mail-qk1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728137AbfGPUTk (ORCPT ); Tue, 16 Jul 2019 16:19:40 -0400 Received: by mail-qk1-f195.google.com with SMTP id 201so15650176qkm.9 for ; Tue, 16 Jul 2019 13:19:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=bTSNcfyeNuJX18rXrTxBmXYwd6To1Z5x97hHlskS8FA=; b=c01+0JKPZsveVFXSuvUZNzxEVDTTpKypRmqi6thGNAe0l4M3PZEL6o/0LyGgoEGgDN T6oVXpY1K9F5Yz0LAU9Epm0tDZgM5LxfKn+fBB0Rr6JokfVL4P2q0OiCQvHd4yp7pUbt UWjO+hS5lCKKD+X3BNnbQeSXrnKL12ltWbL7RIOSCuv6o7NdV6YXjnTyv+At1JZUeIa6 Xs9mx4WKqsRUWSbqh0nkEKrjjIbDbQxuvY58cZH1aUixYVMKjVeXIR6AKbHvGHL3M7no +vc9nceng2OruCu9pllIvEIOdosXvW7kCpLVTdKHjN58b6T2RFV1cqfl1Gnr1BW6CDLe tipw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=bTSNcfyeNuJX18rXrTxBmXYwd6To1Z5x97hHlskS8FA=; b=Ztmuw55t2aTflS5XOh5So0W3dweOmYeK4zCk0oB/lSWfzpbvCbIrBcF2Ceyqu5Gaua m2wCJSlQ3lWp4OZIB0WuPu+eB4nine9XMNHxcr+p8b0H7XeymGhQt0Eqzj/CEH40fYKk mzX2Jo+Q28abjxaPN2X/dGMz33/f7DhvV3DycNC6hIEZ+QS8+MzxC/Agv4s916S/MUV0 6+MQZLJNmhzYXwlixg7TqKgktJL84GRfY3hBxC/Q5+wUffpn38gz+qaFMgE4NxivWpsC JIsPs/0YGzULKpcG+oza0R/30cF8jMkGOUYr/1D2+vOIiGa0BHFMtWRcXzcSWs5AaqzB OqjA== X-Gm-Message-State: APjAAAWRFK21xfr73M9iOduAtW9X1IZq63HYO4tRQ5LL5EjZ0VUa8StV hgxi9FLP8zlUZkXRGuOjnDI= X-Google-Smtp-Source: APXvYqzeRRDzZEKx2VIjYdjatndvqEEZjHV7SSgPYpff27eEQHUmXd74Sj0M3yhnCN2TUo6erAEh+A== X-Received: by 2002:a37:de18:: with SMTP id h24mr23039925qkj.147.1563308379253; Tue, 16 Jul 2019 13:19:39 -0700 (PDT) Received: from localhost ([2620:10d:c091:480::d7f3]) by smtp.gmail.com with ESMTPSA id z50sm11688579qtz.36.2019.07.16.13.19.38 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 16 Jul 2019 13:19:38 -0700 (PDT) From: Josef Bacik To: axboe@kernel.dk, kernel-team@fb.com, linux-block@vger.kernel.org, peterz@infradead.org, oleg@redhat.com Subject: [PATCH 3/5] rq-qos: don't reset has_sleepers on spurious wakeups Date: Tue, 16 Jul 2019 16:19:27 -0400 Message-Id: <20190716201929.79142-4-josef@toxicpanda.com> X-Mailer: git-send-email 2.13.5 In-Reply-To: <20190716201929.79142-1-josef@toxicpanda.com> References: <20190716201929.79142-1-josef@toxicpanda.com> Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP If we raced with somebody else getting an inflight counter we could fail to get an inflight counter with no sleepers on the list, and thus need to go to sleep. In this case has_sleepers should be true because we are now relying on the waker to get our inflight counter for us. And in the case of spurious wakeups we'd still want this to be the case. So set has_sleepers to true if we went to sleep to make sure we're woken up the proper way. Signed-off-by: Josef Bacik --- block/blk-rq-qos.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/blk-rq-qos.c b/block/blk-rq-qos.c index 67a0a4c07060..69a0f0b77795 100644 --- a/block/blk-rq-qos.c +++ b/block/blk-rq-qos.c @@ -261,7 +261,7 @@ void rq_qos_wait(struct rq_wait *rqw, void *private_data, break; } io_schedule(); - has_sleeper = false; + has_sleeper = true; } while (1); finish_wait(&rqw->wait, &data.wq); } From patchwork Tue Jul 16 20:19:28 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11046697 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 057CF112C for ; Tue, 16 Jul 2019 20:19:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EA6AF28657 for ; Tue, 16 Jul 2019 20:19:42 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DEFBB286BF; Tue, 16 Jul 2019 20:19:42 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9796828657 for ; Tue, 16 Jul 2019 20:19:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388616AbfGPUTm (ORCPT ); Tue, 16 Jul 2019 16:19:42 -0400 Received: from mail-qt1-f193.google.com ([209.85.160.193]:46391 "EHLO mail-qt1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728799AbfGPUTm (ORCPT ); Tue, 16 Jul 2019 16:19:42 -0400 Received: by mail-qt1-f193.google.com with SMTP id h21so20935346qtn.13 for ; Tue, 16 Jul 2019 13:19:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=TFVLDA77cnSRzhHRobnrF66rGEp70nX1OfcS3CZKAFI=; b=UgS/qVNbKj+IdrlB+0y5xCamIZXfpG2tqApZ4CX0TmdlcmNyz+G39Gz2NYqUdEXEgE zzwkP6Sv/Osqt5db1JaP3BGQQaiLxfh/gr7hHzge1hVpltXKlAwK0O+ut2qo/wlZ9K4/ uv7cerlINyxYQwpi9fGE3h2lKaBXsZA2W1ndKMiTBF+Ry4MUdhpwAVv7Z9PlkiIii3TN aStRLPSzGHcPGTCwXw8/PGZFoAz4e0fjmrcNpNtggTmPO+xFCh6+AiSZqp5acgxQnExJ 8oZd1jN4fKJSXCgeS1/UQ60MH+bW3pDDQQz/lPuCp6XfXjoN/u2yP474bx08QyX2FuPt Bi/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=TFVLDA77cnSRzhHRobnrF66rGEp70nX1OfcS3CZKAFI=; b=HU3q+yfVi58+ezsXmXZmNx7VbbqDg2iaEU7R4A3BWebTsMDR5xG73X3rKmYsktJ4gf mdR+k+g577Etuy8D/boXfBW80Yfn3PALTke2yWzy7+Lc/XrM/fVGuCeLtv7qgvGC66P3 0gb7qBtqTfbwKXC0dFuRryJjhw15Y7qTwUDoBSp9fkhlTSmtZwzyxHqVfcjeZBhORXg3 jWu0s9OW2n68+917w648uVyjH7V8H5Cxi7oDwXHpBdEBa4Qk4RWj9zNeIBHyuM4ESalS 90dPmJrU0M70LFiF026vIKtZfTZrSjOr36/SDDOxO733bnuBam6SsaBG2bF3rGy/faIc 7SBg== X-Gm-Message-State: APjAAAVm3HCQTZdZf0pD79q5cmjXzJWFXHfI9kzst6Mtm9kGbbtV0Ggr EFOFTHyEMUobkB2ER/b7Mw0= X-Google-Smtp-Source: APXvYqwNmctRE10wLXD43RT8dNa9p6irhDGe14B9B7ZEH12rQoA3ypjBJ2wkinKbJKpvcRav+aDeyA== X-Received: by 2002:ac8:303c:: with SMTP id f57mr24652233qte.294.1563308381048; Tue, 16 Jul 2019 13:19:41 -0700 (PDT) Received: from localhost ([2620:10d:c091:480::d7f3]) by smtp.gmail.com with ESMTPSA id m44sm12314962qtm.54.2019.07.16.13.19.40 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 16 Jul 2019 13:19:40 -0700 (PDT) From: Josef Bacik To: axboe@kernel.dk, kernel-team@fb.com, linux-block@vger.kernel.org, peterz@infradead.org, oleg@redhat.com Subject: [PATCH 4/5] rq-qos: set ourself TASK_UNINTERRUPTIBLE after we schedule Date: Tue, 16 Jul 2019 16:19:28 -0400 Message-Id: <20190716201929.79142-5-josef@toxicpanda.com> X-Mailer: git-send-email 2.13.5 In-Reply-To: <20190716201929.79142-1-josef@toxicpanda.com> References: <20190716201929.79142-1-josef@toxicpanda.com> Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP In case we get a spurious wakeup we need to make sure to re-set ourselves to TASK_UNINTERRUPTIBLE so we don't busy wait. Signed-off-by: Josef Bacik --- block/blk-rq-qos.c | 1 + 1 file changed, 1 insertion(+) diff --git a/block/blk-rq-qos.c b/block/blk-rq-qos.c index 69a0f0b77795..c450b8952eae 100644 --- a/block/blk-rq-qos.c +++ b/block/blk-rq-qos.c @@ -262,6 +262,7 @@ void rq_qos_wait(struct rq_wait *rqw, void *private_data, } io_schedule(); has_sleeper = true; + set_current_state(TASK_UNINTERRUPTIBLE); } while (1); finish_wait(&rqw->wait, &data.wq); } From patchwork Tue Jul 16 20:19:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11046699 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D3532112C for ; Tue, 16 Jul 2019 20:19:44 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C31C928657 for ; Tue, 16 Jul 2019 20:19:44 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B7B7C286BF; Tue, 16 Jul 2019 20:19:44 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 64BFC28657 for ; Tue, 16 Jul 2019 20:19:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728799AbfGPUTn (ORCPT ); Tue, 16 Jul 2019 16:19:43 -0400 Received: from mail-qt1-f193.google.com ([209.85.160.193]:39056 "EHLO mail-qt1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728137AbfGPUTn (ORCPT ); Tue, 16 Jul 2019 16:19:43 -0400 Received: by mail-qt1-f193.google.com with SMTP id l9so20978794qtu.6 for ; Tue, 16 Jul 2019 13:19:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=kr2XG802Hn4GL9ZD3rFYFWnpPNcGVnzMvAnWfTnII4Q=; b=t2F7t4vpZf3J8aVWGrSGt5Kl3Os4II0Ymsw/XqA8yZ3zMBVK0mZSddvn9jkU8CE7WY E5dyFhNX9cLuzRRgrYy1imo/6WpFc9QYnW5Lbwu6MD5VdcaJFfkjsnc0JpBUed4PicAZ M7/B3P5xulAp8DdMfLmeeO60G+qR35EQZsM65/7P6qv1rpTAdv8EoqFi4vX/NWb9b+rJ KaWBta7036V2SWkjBEhhJxtU/THrx3A5kAXjWfLrHXytCYNDd6NyUxfC51Dg+LKwTeqw yV7AkbkflOCLuwWKk64TnoEiYiUqOHSpKoV5q5dTVdBInSzME213HOzTg31azZGSC/6o Omog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=kr2XG802Hn4GL9ZD3rFYFWnpPNcGVnzMvAnWfTnII4Q=; b=Ap/7BuDBwv1HrY3+VeV3x9G1ppjYaqTfL0D1s53bXkM+GT/9VlZzN5YKrnjq4g2Ngh lInGg+FgMEqpNtAFGCIFb+FTD6oIXfdBjfBLxQugQoMUFc78qiQ6yNLdyXZ1vtwcxkjY 0XCezsb73CRaG6zq09o9c2cHVxE9p/o//SJDY1epgK7+ssH8p8KLYUX0ebEmN5fnrzi4 V76tMybeVRw8s2+zktja/2jjr6ykNwqh375mlK8SByEaSLudXIoXBK+WPkfIT6PbOH1u 61TQpRR0vuL6/cyJqJhX7R4XxadGG078lEueqAiNmdgB3LkmNb5I+7ZK6uNOH6NUuqrC 35SA== X-Gm-Message-State: APjAAAXGK4x174Dv6BWBwx2Z82xRswFtLNmUW0uvsfuznVfiuiCMBbI9 VQ/92P1J8xn6Hb0pRunR6dg= X-Google-Smtp-Source: APXvYqxWVJbsw2SX+E/ilTHH9Jf7CD/xtBadcxZ6FOT6Hjfmuzwz5IY2/7dCww9jSpMqVewH/sBoDw== X-Received: by 2002:ac8:2d08:: with SMTP id n8mr24177161qta.383.1563308382867; Tue, 16 Jul 2019 13:19:42 -0700 (PDT) Received: from localhost ([2620:10d:c091:480::d7f3]) by smtp.gmail.com with ESMTPSA id k25sm7574655qta.78.2019.07.16.13.19.41 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 16 Jul 2019 13:19:42 -0700 (PDT) From: Josef Bacik To: axboe@kernel.dk, kernel-team@fb.com, linux-block@vger.kernel.org, peterz@infradead.org, oleg@redhat.com Subject: [PATCH 5/5] rq-qos: use a mb for got_token Date: Tue, 16 Jul 2019 16:19:29 -0400 Message-Id: <20190716201929.79142-6-josef@toxicpanda.com> X-Mailer: git-send-email 2.13.5 In-Reply-To: <20190716201929.79142-1-josef@toxicpanda.com> References: <20190716201929.79142-1-josef@toxicpanda.com> Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Oleg noticed that our checking of data.got_token is unsafe in the cleanup case, and should really use a memory barrier. Use a wmb on the write side, and a rmb() on the read side. We don't need one in the main loop since we're saved by set_current_state(). Signed-off-by: Josef Bacik --- block/blk-rq-qos.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/block/blk-rq-qos.c b/block/blk-rq-qos.c index c450b8952eae..3954c0dc1443 100644 --- a/block/blk-rq-qos.c +++ b/block/blk-rq-qos.c @@ -202,6 +202,7 @@ static int rq_qos_wake_function(struct wait_queue_entry *curr, return -1; data->got_token = true; + smp_wmb(); list_del_init(&curr->entry); wake_up_process(data->task); return 1; @@ -246,6 +247,7 @@ void rq_qos_wait(struct rq_wait *rqw, void *private_data, prepare_to_wait_exclusive(&rqw->wait, &data.wq, TASK_UNINTERRUPTIBLE); has_sleeper = !wq_has_single_sleeper(&rqw->wait); do { + /* The memory barrier in set_task_state saves us here. */ if (data.got_token) break; if (!has_sleeper && acquire_inflight_cb(rqw, private_data)) { @@ -256,6 +258,7 @@ void rq_qos_wait(struct rq_wait *rqw, void *private_data, * which means we now have two. Put our local token * and wake anyone else potentially waiting for one. */ + smp_rmb(); if (data.got_token) cleanup_cb(rqw, private_data); break;