From patchwork Wed Oct 18 16:39:05 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 10015031 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 2BBA860215 for ; Wed, 18 Oct 2017 16:39:27 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1C865291A6 for ; Wed, 18 Oct 2017 16:39:27 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0FB6A291B4; Wed, 18 Oct 2017 16:39:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 505AD291A6 for ; Wed, 18 Oct 2017 16:39:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751420AbdJRQjZ (ORCPT ); Wed, 18 Oct 2017 12:39:25 -0400 Received: from esa2.hgst.iphmx.com ([68.232.143.124]:6746 "EHLO esa2.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750800AbdJRQjX (ORCPT ); Wed, 18 Oct 2017 12:39:23 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1508345915; x=1539881915; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-id:content-transfer-encoding: mime-version; bh=eSOUCniVOI078O1JXftMJAMVWCF8lrYJH5DZe6wqcKI=; b=YgUjvYWudc0w2a4vPrdLJv31A09VOVKhdnsYlXPZN9wpMMzKcEWVQ3uv s/6Zgz2mwb8EAaAbO/SrhfwkwDBCIn9zf/VG9fnypzDz2kJuRd5u8w6qU jEEQlodLf3NdzaMu3ZJesHplAuLqBdn7Lauo7oBjAWqke4wOXiL4phcFZ S6FcMg7Po2VXxBqZmqQHXEI+JMoYvKKhDPwuW8PP0a0Udd+VxwMWcbD4Y JknMUE0FYAlL8drHb90sG7Wn6zG3OG59CcetuEgsQ6nKtxj+Z6fIzRGDe EDpk1BiCjbxX6kmEyCF1ELSwAbGkjinnIHVXZX8bH7lHNx9R6ARs5U393 A==; X-IronPort-AV: E=Sophos;i="5.43,397,1503331200"; d="scan'208";a="154667300" Received: from mail-cys01nam02lp0047.outbound.protection.outlook.com (HELO NAM02-CY1-obe.outbound.protection.outlook.com) ([207.46.163.47]) by ob1.hgst.iphmx.com with ESMTP; 19 Oct 2017 00:58:10 +0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sharedspace.onmicrosoft.com; s=selector1-wdc-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=eSOUCniVOI078O1JXftMJAMVWCF8lrYJH5DZe6wqcKI=; b=gzTKCH6fXYuIGEen3ZadSmGNdpVAE5uywWoPkBMrTDq/fH9/L8D5yz/epGrtkKEUV+p3fSBob78phDAvZq2tuevwKDxCHbwR+n/eRUNa69zaNau4K7KZAEmWjZvqN6x92c2yXq27hUziqvHYDh/y6qfTDLvq7Sqs69T1ARc3W1w= Received: from CY1PR0401MB1536.namprd04.prod.outlook.com (10.163.19.154) by CY1PR0401MB1535.namprd04.prod.outlook.com (10.163.19.153) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.77.7; Wed, 18 Oct 2017 16:39:05 +0000 Received: from CY1PR0401MB1536.namprd04.prod.outlook.com ([10.163.19.154]) by CY1PR0401MB1536.namprd04.prod.outlook.com ([10.163.19.154]) with mapi id 15.20.0077.022; Wed, 18 Oct 2017 16:39:05 +0000 From: Bart Van Assche To: "chenxiang66@hisilicon.com" , "hch@infradead.org" , "axboe@kernel.dk" CC: "linux-block@vger.kernel.org" , "linuxarm@huawei.com" Subject: Re: [bug report] A race between blk_cleanup_queue and blk_timeout_work Thread-Topic: [bug report] A race between blk_cleanup_queue and blk_timeout_work Thread-Index: AQHTRuubATKgNDrSLES+sLNDlVQRpKLp0V+A Date: Wed, 18 Oct 2017 16:39:05 +0000 Message-ID: <1508344743.2540.11.camel@wdc.com> References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=Bart.VanAssche@wdc.com; x-originating-ip: [63.163.107.100] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; CY1PR0401MB1535; 20:HFs1xDUaJORx0iXd9W3Fu6Dj7SE1cA26mOUTj0r+vSbRZJvv0SuDVPmoKsKZV7ynnEHPmQ907HWzt6nb3byIodOc6j6bx52QIyfknV8hZrxTg9MPhty4Lx9CL/3PHO6N/eXhDCsDxuUkh/xLoNRzTm111hEBIsP8lEbD1kGxiN8= x-ms-exchange-antispam-srfa-diagnostics: SSOS; x-ms-office365-filtering-correlation-id: 74c4f892-0838-4bec-f3ad-08d51646bf28 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(2017030254152)(48565401081)(2017052603199)(201703131423075)(201703031133081)(201702281549075); SRVR:CY1PR0401MB1535; x-ms-traffictypediagnostic: CY1PR0401MB1535: wdcipoutbound: EOP-TRUE x-exchange-antispam-report-test: UriScan:(166708455590820); x-microsoft-antispam-prvs: x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(5005006)(8121501046)(100000703101)(100105400095)(10201501046)(3002001)(93006095)(93001095)(6055026)(6041248)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123564025)(20161123555025)(20161123562025)(20161123560025)(20161123558100)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:CY1PR0401MB1535; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:CY1PR0401MB1535; x-forefront-prvs: 0464DBBBC4 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(6009001)(346002)(39860400002)(376002)(199003)(377424004)(51884002)(51914003)(24454002)(189002)(50986999)(6116002)(2906002)(8936002)(86362001)(101416001)(25786009)(5660300001)(2950100002)(189998001)(3280700002)(76176999)(68736007)(3660700001)(2900100001)(4326008)(102836003)(305945005)(3846002)(7736002)(6246003)(2501003)(14454004)(110136005)(54906003)(81156014)(54356999)(97736004)(2201001)(36756003)(81166006)(106356001)(4001150100001)(77096006)(478600001)(6512007)(6486002)(72206003)(99286003)(66066001)(6506006)(105586002)(6306002)(8676002)(6436002)(103116003)(33646002)(53936002)(316002)(966005)(229853002); DIR:OUT; SFP:1102; SCL:1; SRVR:CY1PR0401MB1535; H:CY1PR0401MB1536.namprd04.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-ID: <48C33AB35F652442B22879E75CEEFFD0@namprd04.prod.outlook.com> MIME-Version: 1.0 X-OriginatorOrg: wdc.com X-MS-Exchange-CrossTenant-originalarrivaltime: 18 Oct 2017 16:39:05.1884 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b61c8803-16f3-4c35-9b17-6f65f441df86 X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR0401MB1535 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On Tue, 2017-10-17 at 09:55 +0800, chenxiang (M) wrote: > I find there is a race between blk_cleanup_queue and blk_timeout_work Thanks for the detailed report. Would it be possible to test the patch below (applies on top of https://github.com/bvanassche/linux/tree/blk-mq-pm-2017-10-18-v10)? Thanks, Bart. Subject: [PATCH] block: Fix race between blk_cleanup_queue() and timeout handling Make sure that if the timeout timer fires after a queue has been marked "dying" that the affected requests are finished. Reported-by: chenxiang (M) Signed-off-by: Bart Van Assche Cc: Christoph Hellwig Cc: Hannes Reinecke Cc: Ming Lei Cc: Johannes Thumshirn --- block/blk-core.c | 1 + block/blk-timeout.c | 3 --- 2 files changed, 1 insertion(+), 3 deletions(-) diff --git a/block/blk-core.c b/block/blk-core.c index 4fc9d6d44edd..16f2aa73696e 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -333,6 +333,7 @@ EXPORT_SYMBOL(blk_stop_queue); void blk_sync_queue(struct request_queue *q) { del_timer_sync(&q->timeout); + cancel_work_sync(&q->timeout_work); if (q->mq_ops) { struct blk_mq_hw_ctx *hctx; diff --git a/block/blk-timeout.c b/block/blk-timeout.c index 1eba71486716..764ecf9aeb30 100644 --- a/block/blk-timeout.c +++ b/block/blk-timeout.c @@ -134,8 +134,6 @@ void blk_timeout_work(struct work_struct *work) struct request *rq, *tmp; int next_set = 0; - if (blk_queue_enter(q, BLK_MQ_REQ_NOWAIT | BLK_MQ_REQ_PREEMPT)) - return; spin_lock_irqsave(q->queue_lock, flags); list_for_each_entry_safe(rq, tmp, &q->timeout_list, timeout_list) @@ -145,7 +143,6 @@ void blk_timeout_work(struct work_struct *work) mod_timer(&q->timeout, round_jiffies_up(next)); spin_unlock_irqrestore(q->queue_lock, flags); - blk_queue_exit(q); } /**