From patchwork Mon Apr 2 19:00:53 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 10320211 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 58FAE60116 for ; Mon, 2 Apr 2018 19:01:44 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3BE6C28AFE for ; Mon, 2 Apr 2018 19:01:44 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3A68C28B3E; Mon, 2 Apr 2018 19:01:44 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4942C28C77 for ; Mon, 2 Apr 2018 19:00:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753224AbeDBTA6 (ORCPT ); Mon, 2 Apr 2018 15:00:58 -0400 Received: from mail-yw0-f194.google.com ([209.85.161.194]:36868 "EHLO mail-yw0-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753177AbeDBTA5 (ORCPT ); Mon, 2 Apr 2018 15:00:57 -0400 Received: by mail-yw0-f194.google.com with SMTP id y23so5314191ywy.4; Mon, 02 Apr 2018 12:00:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:mime-version :content-disposition:user-agent; bh=EMaUJwvlh5KF41dk/NHLIfo2oDqok1MBrKxSxlxKN2k=; b=tMBKQxgGWUUqnDRtURnGuunM4tl4QjtCsci2w5cWd6pPdbChdwNjXmHR966WtP7rMB fdlOzR82nr2PXojPtPy9de87cRX7n33Gxl5Ft2gcEwoiJQXz44q8MYyWv4nWBi3TRRHK NQrNpCzE08ASm48qfA94Sgqt7YHvkbI5cjMIR46CbGwxAFkQ7iCkAig911DSRLVCDs98 8WBGhwjQq2qKMA6yOpDehgIJmwB9baBFXi3hVQ02iWtp5ySss3ssscQpUb1KJmLo0eY5 3VO0LtgFbya+fvRGD5fwGtDE93XsgOuWTK0+Y9ytxs9WFNuJThCIMm9TxdBPadMRnxDt c5xQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :mime-version:content-disposition:user-agent; bh=EMaUJwvlh5KF41dk/NHLIfo2oDqok1MBrKxSxlxKN2k=; b=VDR/upMFZIJ0OXRNM8EkFYiwxxbSCsqJhfP6UlodF0rlGPJ7CLYHirzas9cH5lE0wg FFG2/WzgOOOSxWHEkxjiCrvpdrworqP0lH+mhKIcjTSY4hTMIvB5dEeo/7iyrqIhf/2A H2i8Nxmt9GGQOgXNLhCHenvgw1HWTD6WuC6GWytTsJ8GH5nvkyVyluZk9utN5zXdlHJe mcFQRLQryt449GaYYqV0R3owt4BC6i13OotzKL1+ftxYupl3un7Pf2V5r1NbXK6huSS7 kyjeaaKhnCoBj1EOPBc4Oh9Uxoe5hNdYGOLRb253xTF6/9v5Mmv7h6iT64bBL6zTLoif 2JTA== X-Gm-Message-State: ALQs6tDiU+zPrD2n4yTTzmzI+dA8hZjaXCqYU+LxwSj7kjH8kIlms7dp d+Lmmhm2wlCww1uDei+NHIU= X-Google-Smtp-Source: AIpwx48vj6bBPBE4mvPPjV7arR/c/KKTgdDC4gPr0Hz8LXyNTyKBcuqI9dqUMCL3DWKVn40H/sJIyw== X-Received: by 10.13.217.214 with SMTP id b205mr6307726ywe.220.1522695656148; Mon, 02 Apr 2018 12:00:56 -0700 (PDT) Received: from localhost ([2620:10d:c091:180::1:23de]) by smtp.gmail.com with ESMTPSA id r12sm414696ywl.108.2018.04.02.12.00.54 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 02 Apr 2018 12:00:54 -0700 (PDT) Date: Mon, 2 Apr 2018 12:00:53 -0700 From: Tejun Heo To: Jens Axboe Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 1/2] blk-mq: Factor out [s]rcu synchronization Message-ID: <20180402190053.GC388343@devbig577.frc2.facebook.com> MIME-Version: 1.0 Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Factor out [s]rcu synchronization in blk_mq_timeout_work() into blk_mq_timeout_sync_rcu(). This is to add another user in the future and doesn't cause any functional changes. Signed-off-by: Tejun Heo Cc: Bart Van Assche Reviewed-by: Bart Van Assche --- Hello, We were tracking this down in the following thread http://lkml.kernel.org/r/20180207011133.25957-1-bart.vanassche@wdc.com but lost the reproducer in the middle and couldn't fully verify these two patches fix the problem; however, the identified race is real and a bug, so I think it'd be best to apply these two patches. Given the lack of further reports on this front, I don't think it's necessary to rush these patches. I think it'd be best to apply these once the merge window closes. If we need to backport these to -stable, we can tag them later. Thanks. block/blk-mq.c | 39 +++++++++++++++++++++++---------------- include/linux/blk-mq.h | 2 +- 2 files changed, 24 insertions(+), 17 deletions(-) --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -876,13 +876,34 @@ static void blk_mq_check_expired(struct time_after_eq(jiffies, deadline)) { blk_mq_rq_update_aborted_gstate(rq, gstate); data->nr_expired++; - hctx->nr_expired++; + hctx->need_sync_rcu = true; } else if (!data->next_set || time_after(data->next, deadline)) { data->next = deadline; data->next_set = 1; } } +static void blk_mq_timeout_sync_rcu(struct request_queue *q) +{ + struct blk_mq_hw_ctx *hctx; + bool has_rcu = false; + int i; + + queue_for_each_hw_ctx(q, hctx, i) { + if (!hctx->need_sync_rcu) + continue; + + if (!(hctx->flags & BLK_MQ_F_BLOCKING)) + has_rcu = true; + else + synchronize_srcu(hctx->srcu); + + hctx->need_sync_rcu = false; + } + if (has_rcu) + synchronize_rcu(); +} + static void blk_mq_terminate_expired(struct blk_mq_hw_ctx *hctx, struct request *rq, void *priv, bool reserved) { @@ -930,27 +951,13 @@ static void blk_mq_timeout_work(struct w blk_mq_queue_tag_busy_iter(q, blk_mq_check_expired, &data); if (data.nr_expired) { - bool has_rcu = false; - /* * Wait till everyone sees ->aborted_gstate. The * sequential waits for SRCUs aren't ideal. If this ever * becomes a problem, we can add per-hw_ctx rcu_head and * wait in parallel. */ - queue_for_each_hw_ctx(q, hctx, i) { - if (!hctx->nr_expired) - continue; - - if (!(hctx->flags & BLK_MQ_F_BLOCKING)) - has_rcu = true; - else - synchronize_srcu(hctx->srcu); - - hctx->nr_expired = 0; - } - if (has_rcu) - synchronize_rcu(); + blk_mq_timeout_sync_rcu(q); /* terminate the ones we won */ blk_mq_queue_tag_busy_iter(q, blk_mq_terminate_expired, NULL); --- a/include/linux/blk-mq.h +++ b/include/linux/blk-mq.h @@ -51,7 +51,7 @@ struct blk_mq_hw_ctx { unsigned int queue_num; atomic_t nr_active; - unsigned int nr_expired; + bool need_sync_rcu; struct hlist_node cpuhp_dead; struct kobject kobj;