From patchwork Tue Dec 5 16:08:20 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 10093321 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 1B3DC60327 for ; Tue, 5 Dec 2017 16:08:27 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 10767295AB for ; Tue, 5 Dec 2017 16:08:27 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 053702968C; Tue, 5 Dec 2017 16:08:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AC22C295AB for ; Tue, 5 Dec 2017 16:08:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752151AbdLEQIY (ORCPT ); Tue, 5 Dec 2017 11:08:24 -0500 Received: from esa6.hgst.iphmx.com ([216.71.154.45]:24217 "EHLO esa6.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752209AbdLEQIX (ORCPT ); Tue, 5 Dec 2017 11:08:23 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1512490103; x=1544026103; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-id:content-transfer-encoding: mime-version; bh=D+IrFMKVcnUGIg7d/0cXs2Fyk1l7a8CW8H/zh/GZMzk=; b=Tc2so6tq0BiOtCrMv4SqYoQWtLunr5Rp31XqwbPaKyBtJDZSZNCNkLhc 760hl9+xOVJPauPmIahC047IBBIW08nyLMurmn61hY9rV1SU+QGUtRluD poljAD77Sv5XmubsZdph0ab62yUgRzvCs55zdTYB0eB0jXdVKf2DavvI8 AJzL1XERSwTHilKLYX7oFa0FqyoMt+O4ceHPnmsKRFqGUFz6lQV2MIBDw 8okI5fN7mxjkkEscNmS6TrdVQ9yYnvDotTLYmOdruo1pX4kMn1Z/050q9 01IBXaaVkNEqOL6BeM1rGRbG8zcNMPn7HsIAZd9kzOeceSvqEFMjR9TP9 A==; X-IronPort-AV: E=Sophos;i="5.45,364,1508774400"; d="scan'208";a="64679016" Received: from mail-sn1nam01lp0112.outbound.protection.outlook.com (HELO NAM01-SN1-obe.outbound.protection.outlook.com) ([207.46.163.112]) by ob1.hgst.iphmx.com with ESMTP; 06 Dec 2017 00:08:22 +0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sharedspace.onmicrosoft.com; s=selector1-wdc-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=D+IrFMKVcnUGIg7d/0cXs2Fyk1l7a8CW8H/zh/GZMzk=; b=gwK83jMpQon25Ca3ke6Qa8PrjRaeoitDqEAJNR+IoVmnlUUaAoc4Zb1Xg2iTAXBd27r1PJkV3aSSwFh3tuy4PW0UOG3GmtP445ZDrEJfQxx8NWn3O+jsUHLdLY5WdFB+Ut00iBe0/qL1gg0LUmF7OiBKNQ47MtoklT+KACrgwe8= Received: from CY1PR0401MB1536.namprd04.prod.outlook.com (10.163.19.154) by CY1PR0401MB1535.namprd04.prod.outlook.com (10.163.19.153) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.282.5; Tue, 5 Dec 2017 16:08:20 +0000 Received: from CY1PR0401MB1536.namprd04.prod.outlook.com ([10.163.19.154]) by CY1PR0401MB1536.namprd04.prod.outlook.com ([10.163.19.154]) with mapi id 15.20.0282.012; Tue, 5 Dec 2017 16:08:20 +0000 From: Bart Van Assche To: "linux-scsi@vger.kernel.org" , "hch@infradead.org" , "jejb@linux.vnet.ibm.com" , "linux-block@vger.kernel.org" , "axboe@fb.com" , "ming.lei@redhat.com" , "martin.petersen@oracle.com" CC: "linux-kernel@vger.kernel.org" , "hare@suse.com" , "holger@applied-asynchrony.com" Subject: Re: [PATCH] SCSI: run queue if SCSI device queue isn't ready and queue is idle Thread-Topic: [PATCH] SCSI: run queue if SCSI device queue isn't ready and queue is idle Thread-Index: AQHTbZ4g7T3yQq9yCkKeO51Qk7DLeaM060uA Date: Tue, 5 Dec 2017 16:08:20 +0000 Message-ID: <1512490099.2660.6.camel@sandisk.com> References: <20171205075256.10319-1-ming.lei@redhat.com> In-Reply-To: <20171205075256.10319-1-ming.lei@redhat.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=Bart.VanAssche@wdc.com; x-originating-ip: [199.255.44.171] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; CY1PR0401MB1535; 20:lkdq2RRf1Q437sxyRyKoSa2DPXy1JpKbvvjh56n4E2qnDfcQRmfTQUs5E+1yA27cCENywtj3upzURdJJ14eulUJEJBhzQk92Q76EwMW6uHLEK6tIGx4F6sCCvxMQbulhkhOOs83DBvKo5nm1YukRYHqwu30IZZvhXWIhtq3ccU8= x-ms-exchange-antispam-srfa-diagnostics: SSOS; x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: ea624bef-1972-44f8-1a6c-08d53bfa676b x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(4534020)(4602075)(4627115)(201703031133081)(201702281549075)(48565401081)(5600026)(4604075)(2017052603286); SRVR:CY1PR0401MB1535; x-ms-traffictypediagnostic: CY1PR0401MB1535: wdcipoutbound: EOP-TRUE x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:; x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040450)(2401047)(8121501046)(5005006)(3231022)(93006095)(93001095)(3002001)(10201501046)(6055026)(6041248)(20161123560025)(20161123562025)(20161123555025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123558100)(20161123564025)(6072148)(201708071742011); SRVR:CY1PR0401MB1535; BCL:0; PCL:0; RULEID:(100000803101)(100110400095); SRVR:CY1PR0401MB1535; x-forefront-prvs: 0512CC5201 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(6009001)(366004)(376002)(346002)(39860400002)(377424004)(24454002)(189002)(199003)(3280700002)(3660700001)(53936002)(33896003)(2906002)(6246003)(97736004)(229853002)(2950100002)(99286004)(2501003)(316002)(2201001)(77096006)(4326008)(6486002)(5660300001)(110136005)(14454004)(54906003)(7736002)(6436002)(76176011)(86362001)(68736007)(305945005)(7416002)(6506006)(25786009)(8676002)(36756003)(2900100001)(101416001)(103116003)(478600001)(189998001)(105586002)(33646002)(81166006)(8936002)(9686003)(106356001)(102836003)(6512007)(66066001)(81156014)(3846002)(6116002)(72206003); DIR:OUT; SFP:1102; SCL:1; SRVR:CY1PR0401MB1535; H:CY1PR0401MB1536.namprd04.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-ID: <0C64A389DF03B445955E906C4745E347@namprd04.prod.outlook.com> MIME-Version: 1.0 X-OriginatorOrg: wdc.com X-MS-Exchange-CrossTenant-Network-Message-Id: ea624bef-1972-44f8-1a6c-08d53bfa676b X-MS-Exchange-CrossTenant-originalarrivaltime: 05 Dec 2017 16:08:20.4478 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b61c8803-16f3-4c35-9b17-6f65f441df86 X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR0401MB1535 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On Tue, 2017-12-05 at 15:52 +0800, Ming Lei wrote: > diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c > index db9556662e27..1816dd8259b3 100644 > --- a/drivers/scsi/scsi_lib.c > +++ b/drivers/scsi/scsi_lib.c > @@ -1967,6 +1967,8 @@ static bool scsi_mq_get_budget(struct blk_mq_hw_ctx *hctx) > out_put_device: > put_device(&sdev->sdev_gendev); > out: > + if (atomic_read(&sdev->device_busy) == 0 && !scsi_device_blocked(sdev)) > + blk_mq_delay_run_hw_queue(hctx, SCSI_QUEUE_DELAY); > return false; > } This cannot work since multiple threads can call scsi_mq_get_budget() concurrently and hence it can happen that none of them sees atomic_read(&sdev->device_busy) == 0. BTW, the above patch is something I consider as a workaround. Have you considered to fix the root cause and to add blk_mq_sched_mark_restart_hctx() where these are missing, e.g. in blk_mq_sched_dispatch_requests()? The patch below is not a full solution but resulted in a significant improvement in my tests: Bart. Reviewed-by: Bart Van Assche diff --git a/block/blk-mq-sched.c b/block/blk-mq-sched.c index 69e3226e66ca..9d86876ec503 100644 --- a/block/blk-mq-sched.c +++ b/block/blk-mq-sched.c @@ -226,6 +226,7 @@ void blk_mq_sched_dispatch_requests(struct blk_mq_hw_ctx *hctx) * TODO: get more budgets, and dequeue more requests in * one time. */ + blk_mq_sched_mark_restart_hctx(hctx); blk_mq_do_dispatch_ctx(hctx); } else { blk_mq_flush_busy_ctxs(hctx, &rq_list);