From patchwork Wed Oct 18 18:09:32 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 10015153 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 2794A60215 for ; Wed, 18 Oct 2017 18:09:41 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1E58128FA8 for ; Wed, 18 Oct 2017 18:09:41 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 11C69290E4; Wed, 18 Oct 2017 18:09:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C008F290C5 for ; Wed, 18 Oct 2017 18:09:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1750991AbdJRSJi (ORCPT ); Wed, 18 Oct 2017 14:09:38 -0400 Received: from esa6.hgst.iphmx.com ([216.71.154.45]:3646 "EHLO esa6.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751091AbdJRSJf (ORCPT ); Wed, 18 Oct 2017 14:09:35 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1508350176; x=1539886176; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-id:content-transfer-encoding: mime-version; bh=wYmRUWzMmlv/6NxkMgr+PRymRwZ3KrVGAYMRvAQvJv4=; b=AQFvWGWP+WESPYu2PiXkNji4bFgVJe71Fe1WhRX30jcc8Imfwu9jUUFU FTJuMLAA02TkoYHyhU/XqeAwTbtaZp7+zS9vleBvzqzlmImcIOa2gDb1q u5JIzgIuhYvFo2CJ4nF06SE4QI7MD2Mi6e9lCpMWVzJuaaJCQtBqsOjIO BA7RccRM0uKj8be26LAQucs9Zu2GCndfor1uChxbuyR95ArLhz5IQLgqb XFOTSc7ST89MUl2Jmgj6LWqYXW7VLki9xK/uMizp3v2oo//0fSkth3kYf SfeP3pk9stQqzrH7KVtM0bvj/xONbE3ULqnxa36VLyi0caWmLqiiXNQ+t g==; X-IronPort-AV: E=Sophos;i="5.43,397,1503331200"; d="scan'208";a="58961151" Received: from mail-bl2nam02lp0086.outbound.protection.outlook.com (HELO NAM02-BL2-obe.outbound.protection.outlook.com) ([207.46.163.86]) by ob1.hgst.iphmx.com with ESMTP; 19 Oct 2017 02:09:35 +0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sharedspace.onmicrosoft.com; s=selector1-wdc-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=wYmRUWzMmlv/6NxkMgr+PRymRwZ3KrVGAYMRvAQvJv4=; b=deh2R7kdNGSwXAUm8cGkfYg61GNLd37QxQwppS1rjrLmAVT6fTDmAi2bjOoygqYXjpYX3tEk43CsNVfBscC2wVVohXwTyg1vuMdBDrempAwY7cTxJ1OGeaPsLIROUSAaLgypj8AfiaYTSOR+uCRQDaRaPHZTd047T3QetTrZc2g= Received: from CY1PR0401MB1536.namprd04.prod.outlook.com (10.163.19.154) by CY1PR0401MB1533.namprd04.prod.outlook.com (10.163.19.151) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.77.7; Wed, 18 Oct 2017 18:09:32 +0000 Received: from CY1PR0401MB1536.namprd04.prod.outlook.com ([10.163.19.154]) by CY1PR0401MB1536.namprd04.prod.outlook.com ([10.163.19.154]) with mapi id 15.20.0077.022; Wed, 18 Oct 2017 18:09:32 +0000 From: Bart Van Assche To: "chenxiang66@hisilicon.com" , "hch@infradead.org" , "axboe@kernel.dk" CC: "linux-block@vger.kernel.org" , "linuxarm@huawei.com" Subject: Re: [bug report] A race between blk_cleanup_queue and blk_timeout_work Thread-Topic: [bug report] A race between blk_cleanup_queue and blk_timeout_work Thread-Index: AQHTRuubATKgNDrSLES+sLNDlVQRpKLp0V+AgAAZRQA= Date: Wed, 18 Oct 2017 18:09:32 +0000 Message-ID: <1508350170.2540.18.camel@wdc.com> References: <1508344743.2540.11.camel@wdc.com> In-Reply-To: <1508344743.2540.11.camel@wdc.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=Bart.VanAssche@wdc.com; x-originating-ip: [63.163.107.100] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; CY1PR0401MB1533; 20:OCndfCPHarS0AYNGFo3fUERBY2T/N1S7xF75BHIshqYSo/zDqGOnvy638ZZhrINnUHctIOFowj/x3RH7wOdLgBJgM/1yE8BX6cM9U56RhS/EFz82BWBkv0RAeu5E6M916M/XVtB1jYHt/oI1frMj3yJF5bXlooHa6jdE4ZUl0eo= x-ms-exchange-antispam-srfa-diagnostics: SSOS; x-ms-office365-filtering-correlation-id: b797cb51-c16d-41f6-c579-08d5165361f6 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(2017030254152)(48565401081)(2017052603199)(201703131423075)(201703031133081)(201702281549075); SRVR:CY1PR0401MB1533; x-ms-traffictypediagnostic: CY1PR0401MB1533: wdcipoutbound: EOP-TRUE x-exchange-antispam-report-test: UriScan:(166708455590820); x-microsoft-antispam-prvs: x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(5005006)(8121501046)(3002001)(93006095)(93001095)(10201501046)(100000703101)(100105400095)(6055026)(6041248)(20161123555025)(20161123564025)(20161123562025)(20161123558100)(20161123560025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:CY1PR0401MB1533; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:CY1PR0401MB1533; x-forefront-prvs: 0464DBBBC4 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(6009001)(346002)(39860400002)(376002)(377424004)(51884002)(24454002)(51914003)(189002)(199003)(8676002)(6436002)(8936002)(14454004)(3280700002)(68736007)(5660300001)(189998001)(25786009)(4001150100001)(101416001)(54356999)(50986999)(2900100001)(966005)(99286003)(76176999)(3660700001)(2950100002)(478600001)(81156014)(72206003)(81166006)(36756003)(229853002)(66066001)(2501003)(6512007)(305945005)(6306002)(106356001)(7736002)(6116002)(102836003)(86362001)(575784001)(4326008)(2201001)(54906003)(110136005)(103116003)(6246003)(97736004)(6486002)(77096006)(316002)(6506006)(105586002)(2906002)(33646002)(53936002)(3846002); DIR:OUT; SFP:1102; SCL:1; SRVR:CY1PR0401MB1533; H:CY1PR0401MB1536.namprd04.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-ID: MIME-Version: 1.0 X-OriginatorOrg: wdc.com X-MS-Exchange-CrossTenant-originalarrivaltime: 18 Oct 2017 18:09:32.2577 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b61c8803-16f3-4c35-9b17-6f65f441df86 X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR0401MB1533 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On Wed, 2017-10-18 at 09:39 -0700, Bart Van Assche wrote: > On Tue, 2017-10-17 at 09:55 +0800, chenxiang (M) wrote: > > I find there is a race between blk_cleanup_queue and blk_timeout_work > > Thanks for the detailed report. Would it be possible to test the patch below > (applies on top of https://github.com/bvanassche/linux/tree/blk-mq-pm-2017-10-18-v10)? Here is a better version of the patch included in the previous e-mail: Subject: [PATCH] block: Fix race between blk_cleanup_queue() and timeout handling Make sure that if the timeout timer fires after a queue has been marked "dying" that the affected requests are finished. --- block/blk-core.c | 2 ++ block/blk-timeout.c | 3 --- 2 files changed, 2 insertions(+), 3 deletions(-) -- 2.14.2 diff --git a/block/blk-core.c b/block/blk-core.c index 4fc9d6d44edd..0a0fdaa474e3 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -333,6 +333,7 @@ EXPORT_SYMBOL(blk_stop_queue); void blk_sync_queue(struct request_queue *q) { del_timer_sync(&q->timeout); + cancel_work_sync(&q->timeout_work); if (q->mq_ops) { struct blk_mq_hw_ctx *hctx; @@ -905,6 +906,7 @@ struct request_queue *blk_alloc_queue_node(gfp_t gfp_mask, int node_id) setup_timer(&q->backing_dev_info->laptop_mode_wb_timer, laptop_mode_timer_fn, (unsigned long) q); setup_timer(&q->timeout, blk_rq_timed_out_timer, (unsigned long) q); + INIT_WORK(&q->timeout_work, NULL); INIT_LIST_HEAD(&q->queue_head); INIT_LIST_HEAD(&q->timeout_list); INIT_LIST_HEAD(&q->icq_list); diff --git a/block/blk-timeout.c b/block/blk-timeout.c index 1eba71486716..764ecf9aeb30 100644 --- a/block/blk-timeout.c +++ b/block/blk-timeout.c @@ -134,8 +134,6 @@ void blk_timeout_work(struct work_struct *work) struct request *rq, *tmp; int next_set = 0; - if (blk_queue_enter(q, BLK_MQ_REQ_NOWAIT | BLK_MQ_REQ_PREEMPT)) - return; spin_lock_irqsave(q->queue_lock, flags); list_for_each_entry_safe(rq, tmp, &q->timeout_list, timeout_list) @@ -145,7 +143,6 @@ void blk_timeout_work(struct work_struct *work) mod_timer(&q->timeout, round_jiffies_up(next)); spin_unlock_irqrestore(q->queue_lock, flags); - blk_queue_exit(q); } /**