From patchwork Wed Dec 5 07:43:59 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "jianchao.wang" X-Patchwork-Id: 10713101 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 185DA13BF for ; Wed, 5 Dec 2018 07:44:48 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EBAF52CC2D for ; Wed, 5 Dec 2018 07:44:47 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C5EEC2CC57; Wed, 5 Dec 2018 07:44:47 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6799A2CC2D for ; Wed, 5 Dec 2018 07:44:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726351AbeLEHoq (ORCPT ); Wed, 5 Dec 2018 02:44:46 -0500 Received: from userp2120.oracle.com ([156.151.31.85]:42996 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726102AbeLEHoq (ORCPT ); Wed, 5 Dec 2018 02:44:46 -0500 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.22/8.16.0.22) with SMTP id wB57i48q129835; Wed, 5 Dec 2018 07:44:42 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=B2zNxyNngGK6JzHyyb7FVpzffaYVgPFHJVpgTzXKJBc=; b=JG4JBfwN7u0lw5RuFiI8zYp6y1xK7j59W6eGoq4nSdoQv7N1M1xCBPbduBmSuTtoiroY /MutwyjE+hof4KvkXlHhNmsfjmOWbz76ds7whKltF2dEqb3EP8ls5xlVjmKuatfr1ErN WZ83t0JwbDlJd3LGBlGgsMvtSt9JewreEl1/O74PBt6lUxmCfQ+EyuhEp19cbqI7k5e6 1Tz1Mj8Vy5x6AVYfO4Iri7MMQa08m5jD1d08Cvn//m7IbYpvX45axKYr+6vNyUCMZrW9 gLJeRDJ3QxQF/OXyVzJN7c/QbDeFy+HZsRLMdhF/UsEmlZwFEUM3N5Rm4xMWpjGGLEzQ YQ== Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by userp2120.oracle.com with ESMTP id 2p3jxrgpvj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 05 Dec 2018 07:44:42 +0000 Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by aserv0021.oracle.com (8.14.4/8.14.4) with ESMTP id wB57ifUb002756 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 5 Dec 2018 07:44:41 GMT Received: from abhmp0011.oracle.com (abhmp0011.oracle.com [141.146.116.17]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id wB57ifi4025491; Wed, 5 Dec 2018 07:44:41 GMT Received: from will-ThinkCentre-M93p.cn.oracle.com (/10.182.70.234) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 04 Dec 2018 23:44:40 -0800 From: Jianchao Wang To: axboe@kernel.dk Cc: ming.lei@redhat.com, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH V9 1/4] blk-mq: insert to hctx dispatch list when bypass_insert is true Date: Wed, 5 Dec 2018 15:43:59 +0800 Message-Id: <1543995842-20704-2-git-send-email-jianchao.w.wang@oracle.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1543995842-20704-1-git-send-email-jianchao.w.wang@oracle.com> References: <1543995842-20704-1-git-send-email-jianchao.w.wang@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9097 signatures=668686 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=1 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1812050072 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We don't allow direct dispatch of anything but regular reads/writes and insert all of non-read-write requests. However, this is not correct for 'bypass_insert == true' case where inserting is ignored and BLK_STS_RESOURCE is returned. The caller will fail forever. Fix it with inserting the non-read-write request to hctx dispatch list to avoid to involve merge and io scheduler when bypass_insert is true. Signed-off-by: Jianchao Wang --- block/blk-mq.c | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 9005505..153af90 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -1822,6 +1822,7 @@ static blk_status_t __blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, { struct request_queue *q = rq->q; bool run_queue = true; + bool force = false; /* * RCU or SRCU read lock is needed before checking quiesced flag. @@ -1836,9 +1837,19 @@ static blk_status_t __blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, goto insert; } - if (!blk_rq_can_direct_dispatch(rq) || (q->elevator && !bypass_insert)) + if (q->elevator && !bypass_insert) goto insert; + if (!blk_rq_can_direct_dispatch(rq)) { + /* + * For 'bypass_insert == true' case, insert request into hctx + * dispatch list. + */ + if (bypass_insert) + force = true; + goto insert; + } + if (!blk_mq_get_dispatch_budget(hctx)) goto insert; @@ -1849,8 +1860,12 @@ static blk_status_t __blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, return __blk_mq_issue_directly(hctx, rq, cookie, last); insert: - if (bypass_insert) + if (force) { + blk_mq_request_bypass_insert(rq, run_queue); + return BLK_STS_OK; + } else if (bypass_insert) { return BLK_STS_RESOURCE; + } blk_mq_sched_insert_request(rq, false, run_queue, false); return BLK_STS_OK; From patchwork Wed Dec 5 07:44:00 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "jianchao.wang" X-Patchwork-Id: 10713103 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 55B98109C for ; Wed, 5 Dec 2018 07:44:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 403732CC2D for ; Wed, 5 Dec 2018 07:44:57 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 312592CC57; Wed, 5 Dec 2018 07:44:57 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A1FC62CC2D for ; Wed, 5 Dec 2018 07:44:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726874AbeLEHoy (ORCPT ); Wed, 5 Dec 2018 02:44:54 -0500 Received: from userp2120.oracle.com ([156.151.31.85]:43102 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727328AbeLEHox (ORCPT ); Wed, 5 Dec 2018 02:44:53 -0500 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.22/8.16.0.22) with SMTP id wB57i48r129835; Wed, 5 Dec 2018 07:44:50 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=CQZ6XaLgDcM57YO2cVSVXHzhyMl0Ti+QXzcKCbaYUys=; b=HQB+GSSCP9SH76dlZyOHjmabYneExCmnVmP/zj8ixkuknPsSODltoEDeVqZ8vh0fbySt cnaJtVENcOUONHoGFDAVR7R/J8O48qSaHcVW3z3D//ujcmYDEgDO95CX8pDU4PxzLiug IjfIk39q/j8L0X943dV3q9WIusF3RVtXgmrAhVyf0YbPlifTrRT+5I+Uz4X24kf3yfZs OE7Q8SBQ70gEULsnVLMNeQId11Nw9Vs4xFmwQopbpLy9qhKDb9ZGEe0W1jub8DdJ83Vj 404Nu+cldLm34Hd1T5zT6ibJPh/WMtO2xEdCPNYvfBPmBvQ/gCKVFl/UlnXJh6cYTymW sw== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2120.oracle.com with ESMTP id 2p3jxrgpw4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 05 Dec 2018 07:44:49 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id wB57ihDd009942 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 5 Dec 2018 07:44:44 GMT Received: from abhmp0011.oracle.com (abhmp0011.oracle.com [141.146.116.17]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id wB57ihFK029665; Wed, 5 Dec 2018 07:44:43 GMT Received: from will-ThinkCentre-M93p.cn.oracle.com (/10.182.70.234) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 04 Dec 2018 23:44:42 -0800 From: Jianchao Wang To: axboe@kernel.dk Cc: ming.lei@redhat.com, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH V9 2/4] blk-mq: refactor the code of issue request directly Date: Wed, 5 Dec 2018 15:44:00 +0800 Message-Id: <1543995842-20704-3-git-send-email-jianchao.w.wang@oracle.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1543995842-20704-1-git-send-email-jianchao.w.wang@oracle.com> References: <1543995842-20704-1-git-send-email-jianchao.w.wang@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9097 signatures=668686 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=1 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1812050072 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Merge blk_mq_try_issue_directly and __blk_mq_try_issue_directly into one interface to unify the interfaces to issue requests directly. The merged interface takes over the requests totally, it could insert, end or do nothing based on the return value of .queue_rq and 'bypass' parameter. Then caller needn't any other handling any more and then code could be cleaned up. Signed-off-by: Jianchao Wang --- block/blk-mq.c | 116 +++++++++++++++++++++++++++------------------------------ 1 file changed, 54 insertions(+), 62 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 153af90..fe92e52 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -1815,93 +1815,85 @@ static bool blk_rq_can_direct_dispatch(struct request *rq) return req_op(rq) == REQ_OP_READ || req_op(rq) == REQ_OP_WRITE; } -static blk_status_t __blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, +static blk_status_t blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, struct request *rq, blk_qc_t *cookie, - bool bypass_insert, bool last) + bool bypass, bool last) { struct request_queue *q = rq->q; bool run_queue = true; + blk_status_t ret = BLK_STS_RESOURCE; + int srcu_idx; bool force = false; + if (!blk_rq_can_direct_dispatch(rq)) { + /* + * Insert request to hctx dispatch list for 'bypass == true' + * case, otherwise, the caller will fail forever. + */ + if (bypass) + force = true; + goto out; + } + + hctx_lock(hctx, &srcu_idx); /* - * RCU or SRCU read lock is needed before checking quiesced flag. + * hctx_lock is needed before checking quiesced flag. * - * When queue is stopped or quiesced, ignore 'bypass_insert' from - * blk_mq_request_issue_directly(), and return BLK_STS_OK to caller, - * and avoid driver to try to dispatch again. + * When queue is stopped or quiesced, ignore 'bypass', insert + * and return BLK_STS_OK to caller, and avoid driver to try to + * dispatch again. */ - if (blk_mq_hctx_stopped(hctx) || blk_queue_quiesced(q)) { + if (unlikely(blk_mq_hctx_stopped(hctx) || blk_queue_quiesced(q))) { run_queue = false; - bypass_insert = false; - goto insert; + bypass = false; + goto out_unlock; } - if (q->elevator && !bypass_insert) - goto insert; - - if (!blk_rq_can_direct_dispatch(rq)) { - /* - * For 'bypass_insert == true' case, insert request into hctx - * dispatch list. - */ - if (bypass_insert) - force = true; - goto insert; - } + if (unlikely(q->elevator && !bypass)) + goto out_unlock; if (!blk_mq_get_dispatch_budget(hctx)) - goto insert; + goto out_unlock; if (!blk_mq_get_driver_tag(rq)) { blk_mq_put_dispatch_budget(hctx); - goto insert; + goto out_unlock; } - return __blk_mq_issue_directly(hctx, rq, cookie, last); -insert: - if (force) { - blk_mq_request_bypass_insert(rq, run_queue); - return BLK_STS_OK; - } else if (bypass_insert) { - return BLK_STS_RESOURCE; + ret = __blk_mq_issue_directly(hctx, rq, cookie, last); +out_unlock: + hctx_unlock(hctx, srcu_idx); +out: + switch (ret) { + case BLK_STS_OK: + break; + case BLK_STS_DEV_RESOURCE: + case BLK_STS_RESOURCE: + if (force) { + blk_mq_request_bypass_insert(rq, run_queue); + ret = BLK_STS_OK; + } else if (!bypass) { + blk_mq_sched_insert_request(rq, false, run_queue, false); + ret = BLK_STS_OK; + } + break; + default: + if (!bypass) { + blk_mq_end_request(rq, ret); + ret = BLK_STS_OK; + } + break; } - blk_mq_sched_insert_request(rq, false, run_queue, false); - return BLK_STS_OK; -} - -static void blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, - struct request *rq, blk_qc_t *cookie) -{ - blk_status_t ret; - int srcu_idx; - - might_sleep_if(hctx->flags & BLK_MQ_F_BLOCKING); - - hctx_lock(hctx, &srcu_idx); - - ret = __blk_mq_try_issue_directly(hctx, rq, cookie, false, true); - if (ret == BLK_STS_RESOURCE || ret == BLK_STS_DEV_RESOURCE) - blk_mq_sched_insert_request(rq, false, true, false); - else if (ret != BLK_STS_OK) - blk_mq_end_request(rq, ret); - - hctx_unlock(hctx, srcu_idx); + return ret; } blk_status_t blk_mq_request_issue_directly(struct request *rq, bool last) { - blk_status_t ret; - int srcu_idx; - blk_qc_t unused_cookie; - struct blk_mq_hw_ctx *hctx = rq->mq_hctx; + blk_qc_t unused; - hctx_lock(hctx, &srcu_idx); - ret = __blk_mq_try_issue_directly(hctx, rq, &unused_cookie, true, last); - hctx_unlock(hctx, srcu_idx); - - return ret; + return blk_mq_try_issue_directly(rq->mq_hctx, rq, &unused, true, last); } void blk_mq_try_issue_list_directly(struct blk_mq_hw_ctx *hctx, @@ -2044,13 +2036,13 @@ static blk_qc_t blk_mq_make_request(struct request_queue *q, struct bio *bio) if (same_queue_rq) { data.hctx = same_queue_rq->mq_hctx; blk_mq_try_issue_directly(data.hctx, same_queue_rq, - &cookie); + &cookie, false, true); } } else if ((q->nr_hw_queues > 1 && is_sync) || (!q->elevator && !data.hctx->dispatch_busy)) { blk_mq_put_ctx(data.ctx); blk_mq_bio_to_request(rq, bio); - blk_mq_try_issue_directly(data.hctx, rq, &cookie); + blk_mq_try_issue_directly(data.hctx, rq, &cookie, false, true); } else { blk_mq_put_ctx(data.ctx); blk_mq_bio_to_request(rq, bio); From patchwork Wed Dec 5 07:44:01 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "jianchao.wang" X-Patchwork-Id: 10713107 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3AECC109C for ; Wed, 5 Dec 2018 07:45:12 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2C3242CC2D for ; Wed, 5 Dec 2018 07:45:12 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1F8DC2CC57; Wed, 5 Dec 2018 07:45:12 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AD3A52CC2D for ; Wed, 5 Dec 2018 07:45:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727170AbeLEHou (ORCPT ); Wed, 5 Dec 2018 02:44:50 -0500 Received: from userp2120.oracle.com ([156.151.31.85]:43048 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726102AbeLEHou (ORCPT ); Wed, 5 Dec 2018 02:44:50 -0500 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.22/8.16.0.22) with SMTP id wB57iR0b129950; Wed, 5 Dec 2018 07:44:46 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=9g3aOswRkBa0Sy+nHjpOtR1GfeY+RHsrrC/jHEcEm3A=; b=c1iBGReXJZzrgmkwg8B78NItEOsf4TIfrvSyC3MU1tWzYf/IwWcHKY0QNXu5aMCoJAB1 WiJ+wp92MyxDrTzqIdGYlvRCdfHtMDWmgIVM8nWrLHYOqQgeP7xIaDNWHJ1uy2XZJaFl wq2CmnJQECEz0sWOmYfSdJz7sv/FyNzinT+ASB859PXRfHZEzhkJ7ixxJulfTkCPuUlG VN/GYRaynKdqUCGqRdFVWZI36AE/1spQFhValHp+7ywhsBh+tKVkQt4Xzl7R1JN3NfqX W/tAEIOhJdLRjDNtTwmHqXBD4NjFgvIc1dO6qsyR3ksUNvLPSS/WKZX8k7lFOukl01Yf Sw== Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by userp2120.oracle.com with ESMTP id 2p3jxrgpvy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 05 Dec 2018 07:44:46 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id wB57ijp7020882 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 5 Dec 2018 07:44:45 GMT Received: from abhmp0011.oracle.com (abhmp0011.oracle.com [141.146.116.17]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id wB57ijeX032102; Wed, 5 Dec 2018 07:44:45 GMT Received: from will-ThinkCentre-M93p.cn.oracle.com (/10.182.70.234) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 04 Dec 2018 23:44:44 -0800 From: Jianchao Wang To: axboe@kernel.dk Cc: ming.lei@redhat.com, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH V9 3/4] blk-mq: issue directly with bypass 'false' in blk_mq_sched_insert_requests Date: Wed, 5 Dec 2018 15:44:01 +0800 Message-Id: <1543995842-20704-4-git-send-email-jianchao.w.wang@oracle.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1543995842-20704-1-git-send-email-jianchao.w.wang@oracle.com> References: <1543995842-20704-1-git-send-email-jianchao.w.wang@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9097 signatures=668686 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=1 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1812050072 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP It is not necessary to issue request directly with bypass 'true' in blk_mq_sched_insert_requests and handle the non-issued requests itself. Just set bypass to 'false' and let blk_mq_try_issue_directly handle them totally. Remove the blk_rq_can_direct_dispatch check, because blk_mq_try_issue_directly can handle it well. With respect to commit_rqs hook, we only need to care about the last request's result. If it is inserted, invoke commit_rqs. We identify the actual result of blk_mq_try_issue_directly with outputed cookie. Signed-off-by: Jianchao Wang --- block/blk-mq-sched.c | 8 +++----- block/blk-mq.c | 25 ++++++++----------------- include/linux/blk_types.h | 1 + 3 files changed, 12 insertions(+), 22 deletions(-) diff --git a/block/blk-mq-sched.c b/block/blk-mq-sched.c index f096d898..5b4d52d 100644 --- a/block/blk-mq-sched.c +++ b/block/blk-mq-sched.c @@ -417,12 +417,10 @@ void blk_mq_sched_insert_requests(struct blk_mq_hw_ctx *hctx, * busy in case of 'none' scheduler, and this way may save * us one extra enqueue & dequeue to sw queue. */ - if (!hctx->dispatch_busy && !e && !run_queue_async) { + if (!hctx->dispatch_busy && !e && !run_queue_async) blk_mq_try_issue_list_directly(hctx, list); - if (list_empty(list)) - return; - } - blk_mq_insert_requests(hctx, ctx, list); + else + blk_mq_insert_requests(hctx, ctx, list); } blk_mq_run_hw_queue(hctx, run_queue_async); diff --git a/block/blk-mq.c b/block/blk-mq.c index fe92e52..0dfa269 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -1899,32 +1899,23 @@ blk_status_t blk_mq_request_issue_directly(struct request *rq, bool last) void blk_mq_try_issue_list_directly(struct blk_mq_hw_ctx *hctx, struct list_head *list) { + blk_qc_t cookie = BLK_QC_T_INVALID; + while (!list_empty(list)) { - blk_status_t ret; struct request *rq = list_first_entry(list, struct request, queuelist); - if (!blk_rq_can_direct_dispatch(rq)) - break; - list_del_init(&rq->queuelist); - ret = blk_mq_request_issue_directly(rq, list_empty(list)); - if (ret != BLK_STS_OK) { - if (ret == BLK_STS_RESOURCE || - ret == BLK_STS_DEV_RESOURCE) { - list_add(&rq->queuelist, list); - break; - } - blk_mq_end_request(rq, ret); - } + blk_mq_try_issue_directly(hctx, rq, &cookie, false, + list_empty(list)); } /* - * If we didn't flush the entire list, we could have told - * the driver there was more coming, but that turned out to - * be a lie. + * cookie is set to a valid value only when reqeust is issued successfully. + * We only need to care about the last request's result, if it is inserted, + * kick the hardware with commit_rqs hook. */ - if (!list_empty(list) && hctx->queue->mq_ops->commit_rqs) + if ((cookie == BLK_QC_T_INVALID) && hctx->queue->mq_ops->commit_rqs) hctx->queue->mq_ops->commit_rqs(hctx); } diff --git a/include/linux/blk_types.h b/include/linux/blk_types.h index c0ba1a0..a0a467a41 100644 --- a/include/linux/blk_types.h +++ b/include/linux/blk_types.h @@ -414,6 +414,7 @@ static inline int op_stat_group(unsigned int op) } typedef unsigned int blk_qc_t; +#define BLK_QC_T_INVALID -2U #define BLK_QC_T_NONE -1U #define BLK_QC_T_SHIFT 16 #define BLK_QC_T_INTERNAL (1U << 31) From patchwork Wed Dec 5 07:44:02 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "jianchao.wang" X-Patchwork-Id: 10713105 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7070E13BB for ; Wed, 5 Dec 2018 07:45:06 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 60B502CC2D for ; Wed, 5 Dec 2018 07:45:06 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 515CE2CC57; Wed, 5 Dec 2018 07:45:06 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E33A42CC2D for ; Wed, 5 Dec 2018 07:45:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727299AbeLEHov (ORCPT ); Wed, 5 Dec 2018 02:44:51 -0500 Received: from userp2120.oracle.com ([156.151.31.85]:43070 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726866AbeLEHov (ORCPT ); Wed, 5 Dec 2018 02:44:51 -0500 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.22/8.16.0.22) with SMTP id wB57i68x129846; Wed, 5 Dec 2018 07:44:47 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=qJYgPBIAJdCMkFJRCLSasTDUdZ0NkwekdNARv/TDnXg=; b=DJvrQBMEPjAxIksSkUqueQh6LIdaAootVA9LNxvU4EmfXkTdA73ubclNZou+3M5c87dp Nq/Rfgt7d97kCoApFKL/oxeeavThb9rLIsRS0DCUaqBLjGmdCCxF1N0n74JvRhHKdhTD bA1od8SA7pg1qEwJZqDHCvkkXIRMjoqhbd/h9g723r46LiDn/xyg1n65VHTQxQMpZ0MZ i++AUE3i8RNHpIJDMOpBPFUbsieI7ZHnDEx1f+b+mpGNNEvmijYMjhk/jRV7UUS5R9Vj joWJTTpkeP70JfF0335fwA7qJaov1QJTsT0ZaJ3Kgq+eyc2lzu5ezmesvzDVUQ4IBG3m wQ== Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by userp2120.oracle.com with ESMTP id 2p3jxrgpw3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 05 Dec 2018 07:44:47 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id wB57ilTI016761 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 5 Dec 2018 07:44:47 GMT Received: from abhmp0011.oracle.com (abhmp0011.oracle.com [141.146.116.17]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id wB57ilxA032108; Wed, 5 Dec 2018 07:44:47 GMT Received: from will-ThinkCentre-M93p.cn.oracle.com (/10.182.70.234) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 04 Dec 2018 23:44:46 -0800 From: Jianchao Wang To: axboe@kernel.dk Cc: ming.lei@redhat.com, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH V9 4/4] blk-mq: replace and kill blk_mq_request_issue_directly Date: Wed, 5 Dec 2018 15:44:02 +0800 Message-Id: <1543995842-20704-5-git-send-email-jianchao.w.wang@oracle.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1543995842-20704-1-git-send-email-jianchao.w.wang@oracle.com> References: <1543995842-20704-1-git-send-email-jianchao.w.wang@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9097 signatures=668686 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=1 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=963 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1812050072 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Replace blk_mq_request_issue_directly with blk_mq_try_issue_directly in blk_insert_cloned_request and kill it as nobody uses it any more. Signed-off-by: Jianchao Wang --- block/blk-core.c | 4 +++- block/blk-mq.c | 9 +-------- block/blk-mq.h | 6 ++++-- 3 files changed, 8 insertions(+), 11 deletions(-) diff --git a/block/blk-core.c b/block/blk-core.c index ad59102..c92d866 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -1297,6 +1297,8 @@ static int blk_cloned_rq_check_limits(struct request_queue *q, */ blk_status_t blk_insert_cloned_request(struct request_queue *q, struct request *rq) { + blk_qc_t unused; + if (blk_cloned_rq_check_limits(q, rq)) return BLK_STS_IOERR; @@ -1312,7 +1314,7 @@ blk_status_t blk_insert_cloned_request(struct request_queue *q, struct request * * bypass a potential scheduler on the bottom device for * insert. */ - return blk_mq_request_issue_directly(rq, true); + return blk_mq_try_issue_directly(rq->mq_hctx, rq, &unused, true, true); } EXPORT_SYMBOL_GPL(blk_insert_cloned_request); diff --git a/block/blk-mq.c b/block/blk-mq.c index 0dfa269..9d5c949 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -1815,7 +1815,7 @@ static bool blk_rq_can_direct_dispatch(struct request *rq) return req_op(rq) == REQ_OP_READ || req_op(rq) == REQ_OP_WRITE; } -static blk_status_t blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, +blk_status_t blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, struct request *rq, blk_qc_t *cookie, bool bypass, bool last) @@ -1889,13 +1889,6 @@ static blk_status_t blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, return ret; } -blk_status_t blk_mq_request_issue_directly(struct request *rq, bool last) -{ - blk_qc_t unused; - - return blk_mq_try_issue_directly(rq->mq_hctx, rq, &unused, true, last); -} - void blk_mq_try_issue_list_directly(struct blk_mq_hw_ctx *hctx, struct list_head *list) { diff --git a/block/blk-mq.h b/block/blk-mq.h index a664ea4..b81e619 100644 --- a/block/blk-mq.h +++ b/block/blk-mq.h @@ -68,8 +68,10 @@ void blk_mq_request_bypass_insert(struct request *rq, bool run_queue); void blk_mq_insert_requests(struct blk_mq_hw_ctx *hctx, struct blk_mq_ctx *ctx, struct list_head *list); -/* Used by blk_insert_cloned_request() to issue request directly */ -blk_status_t blk_mq_request_issue_directly(struct request *rq, bool last); +blk_status_t blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, + struct request *rq, + blk_qc_t *cookie, + bool bypass, bool last); void blk_mq_try_issue_list_directly(struct blk_mq_hw_ctx *hctx, struct list_head *list);