From patchwork Fri Dec 7 03:09:42 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "jianchao.wang" X-Patchwork-Id: 10717385 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 92E411923 for ; Fri, 7 Dec 2018 03:10:38 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 81FD02E944 for ; Fri, 7 Dec 2018 03:10:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 762EF2E9AB; Fri, 7 Dec 2018 03:10:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1AC242E944 for ; Fri, 7 Dec 2018 03:10:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726029AbeLGDKc (ORCPT ); Thu, 6 Dec 2018 22:10:32 -0500 Received: from userp2130.oracle.com ([156.151.31.86]:51060 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725939AbeLGDKb (ORCPT ); Thu, 6 Dec 2018 22:10:31 -0500 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.22/8.16.0.22) with SMTP id wB72xGNu019052; Fri, 7 Dec 2018 03:10:27 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=S3+/7CXRFbPTfIbccA8dMMcAQ3GGP2dz6DsxuSvojyM=; b=huBzHI/JwkY/zIvuJTAuXSpJ9HbuvvhT6NIXvr3/3gk0LMmYCW0oZkDKBnk/ogv9JOV8 cZ7Jft0dS9NNTtaLd9WIJ5Z9xA2su8vJzzQnRWRZYH0cih0nPOxT4koeD7ABiH7dFOpr 0oeITpwBtvPFxmz7YPaDuFw4+BQfHp4haOVi/JM3TV7yp9pQFVLKTAQlNx9ZpnKaxZ4O D2hE4pna9nsY0Zw8b/VEVPXIxe0QTEACmzjMan28Fay0Dei/P0xwJYBgRX2x1VIo+1jR dsVA5cKTKxxPoti6lSkA17RwJCX1dBjEafQCbrXIuhPknoPM/+9Ik+XFUZ2nGfaVbKo6 nA== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2130.oracle.com with ESMTP id 2p3hqubkbt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 07 Dec 2018 03:10:27 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id wB73AQPP001627 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 7 Dec 2018 03:10:26 GMT Received: from abhmp0005.oracle.com (abhmp0005.oracle.com [141.146.116.11]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id wB73APx6032566; Fri, 7 Dec 2018 03:10:25 GMT Received: from will-ThinkCentre-M93p.cn.oracle.com (/10.182.70.234) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 06 Dec 2018 19:10:25 -0800 From: Jianchao Wang To: axboe@kernel.dk Cc: ming.lei@redhat.com, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH V11 1/4] blk-mq: insert to hctx dispatch list when bypass_insert is true Date: Fri, 7 Dec 2018 11:09:42 +0800 Message-Id: <1544152185-32667-2-git-send-email-jianchao.w.wang@oracle.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1544152185-32667-1-git-send-email-jianchao.w.wang@oracle.com> References: <1544152185-32667-1-git-send-email-jianchao.w.wang@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9099 signatures=668679 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=1 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1812070024 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We don't allow direct dispatch of anything but regular reads/writes and insert all of non-read-write requests. However, this is not correct for 'bypass_insert == true' case where inserting is ignored and BLK_STS_RESOURCE is returned. The caller will fail forever. Fix it with inserting the non-read-write request to hctx dispatch list to avoid to involve merge and io scheduler when bypass_insert is true. Signed-off-by: Jianchao Wang --- block/blk-mq.c | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 9005505..01802bf 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -1822,6 +1822,7 @@ static blk_status_t __blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, { struct request_queue *q = rq->q; bool run_queue = true; + bool force = false; /* * RCU or SRCU read lock is needed before checking quiesced flag. @@ -1836,9 +1837,18 @@ static blk_status_t __blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, goto insert; } - if (!blk_rq_can_direct_dispatch(rq) || (q->elevator && !bypass_insert)) + if (q->elevator && !bypass_insert) goto insert; + if (!blk_rq_can_direct_dispatch(rq)) { + /* + * For 'bypass_insert == true' case, insert request into hctx + * dispatch list. + */ + force = bypass_insert; + goto insert; + } + if (!blk_mq_get_dispatch_budget(hctx)) goto insert; @@ -1849,8 +1859,12 @@ static blk_status_t __blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, return __blk_mq_issue_directly(hctx, rq, cookie, last); insert: - if (bypass_insert) + if (force) { + blk_mq_request_bypass_insert(rq, run_queue); + return BLK_STS_OK; + } else if (bypass_insert) { return BLK_STS_RESOURCE; + } blk_mq_sched_insert_request(rq, false, run_queue, false); return BLK_STS_OK; From patchwork Fri Dec 7 03:09:43 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "jianchao.wang" X-Patchwork-Id: 10717391 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3F4C413AF for ; Fri, 7 Dec 2018 03:10:54 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3078E2E944 for ; Fri, 7 Dec 2018 03:10:54 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 24FCE2E9AB; Fri, 7 Dec 2018 03:10:54 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 972F12E944 for ; Fri, 7 Dec 2018 03:10:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726039AbeLGDKc (ORCPT ); Thu, 6 Dec 2018 22:10:32 -0500 Received: from userp2120.oracle.com ([156.151.31.85]:60962 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726022AbeLGDKb (ORCPT ); Thu, 6 Dec 2018 22:10:31 -0500 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.22/8.16.0.22) with SMTP id wB72x1PW177228; Fri, 7 Dec 2018 03:10:28 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=MiJ0eHjm0z5juPu6Vb83GsMn2AocDMhet+vwBF8v2mE=; b=RB77gS6rAX6DiNoD6SeNd2zekfwt+IToh5PJ9PBWfN9XE5tW0SYABxyxLDZox3Lhp1oK vtPR/G1FPFiB0mCuqp+bCODyamQ2DemH8req/CRAvEnuv+kQ0TWIq4zs8QLsMluArdtE IFfNHGtyWtmiGxFpGzB9xNtg7kTUf8PEj8lnL8jXaKYhbEPXIwQniEL962cSXH7sxLr2 jGw35GeV3A+8fINdowphAQmeFOiGR28oLKA/WGhiK9fe2UBfE4rPKtuiqYVsWUmmZ828 +pRA4F8wHV4T3AhIqgfjnXUXco5MRj/rkYRe1uL1PixGxKUBv0TyuNw4avth0UQtVzTn MQ== Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by userp2120.oracle.com with ESMTP id 2p3jxrufq8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 07 Dec 2018 03:10:28 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id wB73ARpU014655 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 7 Dec 2018 03:10:27 GMT Received: from abhmp0005.oracle.com (abhmp0005.oracle.com [141.146.116.11]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id wB73ARKj032569; Fri, 7 Dec 2018 03:10:27 GMT Received: from will-ThinkCentre-M93p.cn.oracle.com (/10.182.70.234) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 06 Dec 2018 19:10:27 -0800 From: Jianchao Wang To: axboe@kernel.dk Cc: ming.lei@redhat.com, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH V11 2/4] blk-mq: refactor the code of issue request directly Date: Fri, 7 Dec 2018 11:09:43 +0800 Message-Id: <1544152185-32667-3-git-send-email-jianchao.w.wang@oracle.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1544152185-32667-1-git-send-email-jianchao.w.wang@oracle.com> References: <1544152185-32667-1-git-send-email-jianchao.w.wang@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9099 signatures=668679 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=1 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1812070024 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Merge blk_mq_try_issue_directly and __blk_mq_try_issue_directly into one interface to unify the interfaces to issue requests directly. The merged interface takes over the requests totally, it could insert, end or do nothing based on the return value of .queue_rq and 'bypass' parameter. Then caller needn't any other handling any more and then code could be cleaned up. Signed-off-by: Jianchao Wang --- block/blk-mq.c | 120 ++++++++++++++++++++++++++++----------------------------- 1 file changed, 59 insertions(+), 61 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 01802bf..88ee447 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -1815,92 +1815,90 @@ static bool blk_rq_can_direct_dispatch(struct request *rq) return req_op(rq) == REQ_OP_READ || req_op(rq) == REQ_OP_WRITE; } -static blk_status_t __blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, +static blk_status_t blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, struct request *rq, blk_qc_t *cookie, - bool bypass_insert, bool last) + bool bypass, bool last) { struct request_queue *q = rq->q; bool run_queue = true; + blk_status_t ret = BLK_STS_RESOURCE; + int srcu_idx; bool force = false; + if (!blk_rq_can_direct_dispatch(rq)) { + /* + * Insert request to hctx dispatch list and return + * BLK_STS_OK for 'bypass == true' case, otherwise, + * the caller will fail forever. + */ + force = bypass; + goto out; + } + + hctx_lock(hctx, &srcu_idx); /* - * RCU or SRCU read lock is needed before checking quiesced flag. + * hctx_lock is needed before checking quiesced flag. * - * When queue is stopped or quiesced, ignore 'bypass_insert' from - * blk_mq_request_issue_directly(), and return BLK_STS_OK to caller, - * and avoid driver to try to dispatch again. + * When queue is stopped or quiesced, ignore 'bypass', insert + * and return BLK_STS_OK to caller, and avoid driver to try to + * dispatch again. */ - if (blk_mq_hctx_stopped(hctx) || blk_queue_quiesced(q)) { + if (unlikely(blk_mq_hctx_stopped(hctx) || blk_queue_quiesced(q))) { run_queue = false; - bypass_insert = false; - goto insert; + bypass = false; + goto out_unlock; } - if (q->elevator && !bypass_insert) - goto insert; - - if (!blk_rq_can_direct_dispatch(rq)) { - /* - * For 'bypass_insert == true' case, insert request into hctx - * dispatch list. - */ - force = bypass_insert; - goto insert; - } + if (unlikely(q->elevator && !bypass)) + goto out_unlock; if (!blk_mq_get_dispatch_budget(hctx)) - goto insert; + goto out_unlock; if (!blk_mq_get_driver_tag(rq)) { blk_mq_put_dispatch_budget(hctx); - goto insert; + goto out_unlock; } - return __blk_mq_issue_directly(hctx, rq, cookie, last); -insert: - if (force) { - blk_mq_request_bypass_insert(rq, run_queue); - return BLK_STS_OK; - } else if (bypass_insert) { - return BLK_STS_RESOURCE; + /* + * If the request is issued unsuccessfully with + * BLK_STS_DEV_RESOURCE or BLK_STS_RESOURCE, insert + * the request to hctx dispatch list due to attached + * lldd resource. + */ + force = true; + ret = __blk_mq_issue_directly(hctx, rq, cookie, last); +out_unlock: + hctx_unlock(hctx, srcu_idx); +out: + switch (ret) { + case BLK_STS_OK: + break; + case BLK_STS_DEV_RESOURCE: + case BLK_STS_RESOURCE: + if (force) { + blk_mq_request_bypass_insert(rq, run_queue); + ret = bypass ? BLK_STS_OK : ret; + } else if (!bypass) { + blk_mq_sched_insert_request(rq, false, + run_queue, false); + } + break; + default: + if (!bypass) + blk_mq_end_request(rq, ret); + break; } - blk_mq_sched_insert_request(rq, false, run_queue, false); - return BLK_STS_OK; -} - -static void blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, - struct request *rq, blk_qc_t *cookie) -{ - blk_status_t ret; - int srcu_idx; - - might_sleep_if(hctx->flags & BLK_MQ_F_BLOCKING); - - hctx_lock(hctx, &srcu_idx); - - ret = __blk_mq_try_issue_directly(hctx, rq, cookie, false, true); - if (ret == BLK_STS_RESOURCE || ret == BLK_STS_DEV_RESOURCE) - blk_mq_sched_insert_request(rq, false, true, false); - else if (ret != BLK_STS_OK) - blk_mq_end_request(rq, ret); - - hctx_unlock(hctx, srcu_idx); + return ret; } blk_status_t blk_mq_request_issue_directly(struct request *rq, bool last) { - blk_status_t ret; - int srcu_idx; - blk_qc_t unused_cookie; - struct blk_mq_hw_ctx *hctx = rq->mq_hctx; + blk_qc_t unused; - hctx_lock(hctx, &srcu_idx); - ret = __blk_mq_try_issue_directly(hctx, rq, &unused_cookie, true, last); - hctx_unlock(hctx, srcu_idx); - - return ret; + return blk_mq_try_issue_directly(rq->mq_hctx, rq, &unused, true, last); } void blk_mq_try_issue_list_directly(struct blk_mq_hw_ctx *hctx, @@ -2043,13 +2041,13 @@ static blk_qc_t blk_mq_make_request(struct request_queue *q, struct bio *bio) if (same_queue_rq) { data.hctx = same_queue_rq->mq_hctx; blk_mq_try_issue_directly(data.hctx, same_queue_rq, - &cookie); + &cookie, false, true); } } else if ((q->nr_hw_queues > 1 && is_sync) || (!q->elevator && !data.hctx->dispatch_busy)) { blk_mq_put_ctx(data.ctx); blk_mq_bio_to_request(rq, bio); - blk_mq_try_issue_directly(data.hctx, rq, &cookie); + blk_mq_try_issue_directly(data.hctx, rq, &cookie, false, true); } else { blk_mq_put_ctx(data.ctx); blk_mq_bio_to_request(rq, bio); From patchwork Fri Dec 7 03:09:44 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "jianchao.wang" X-Patchwork-Id: 10717389 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3CC321932 for ; Fri, 7 Dec 2018 03:10:50 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2E1942E944 for ; Fri, 7 Dec 2018 03:10:50 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 22B8D2E9AB; Fri, 7 Dec 2018 03:10:50 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BBC6A2E944 for ; Fri, 7 Dec 2018 03:10:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726061AbeLGDKd (ORCPT ); Thu, 6 Dec 2018 22:10:33 -0500 Received: from userp2120.oracle.com ([156.151.31.85]:60988 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726043AbeLGDKd (ORCPT ); Thu, 6 Dec 2018 22:10:33 -0500 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.22/8.16.0.22) with SMTP id wB72x5TN177303; Fri, 7 Dec 2018 03:10:30 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=1HrGOB7n42joPy8UiZcuIER2fmv6qujifgn9jg+k/0w=; b=t5HQJMfMEfE8jKO91qHin+Shf5zRBmRLsvCJhcUZBm8H3W4b/7VHoSLvKDRALlYThr5u YqY7HjiIF2W51M7Atdg6tV3oQMnukVVXhv5F2wTx3PMo98vL/7ka/zHgSs/WdG03QQXx S/VFH5hhymfocIrDxT7kiLrwl90o0QFLGvpMU6pjcK6Idb7dE9KIyZniG5H/YxCI++5M 3gF6qaLdAQd0uVKLoQvTZVsdVL/0tviaEtte1IcbzxVv0EurMBe6DvMvoItFZqNQkjjX dMeIlshLVsTByWJ7zJYj0KJ1IJeQz+fNKqLkHy8wWsQvCcjQG0jbN5HdyyPzNBVz9/UA ZA== Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by userp2120.oracle.com with ESMTP id 2p3jxrufqa-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 07 Dec 2018 03:10:30 +0000 Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id wB73ATf8019723 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 7 Dec 2018 03:10:29 GMT Received: from abhmp0005.oracle.com (abhmp0005.oracle.com [141.146.116.11]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id wB73ATlR028311; Fri, 7 Dec 2018 03:10:29 GMT Received: from will-ThinkCentre-M93p.cn.oracle.com (/10.182.70.234) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 06 Dec 2018 19:10:28 -0800 From: Jianchao Wang To: axboe@kernel.dk Cc: ming.lei@redhat.com, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH V11 3/4] blk-mq: issue directly with bypass 'false' in blk_mq_sched_insert_requests Date: Fri, 7 Dec 2018 11:09:44 +0800 Message-Id: <1544152185-32667-4-git-send-email-jianchao.w.wang@oracle.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1544152185-32667-1-git-send-email-jianchao.w.wang@oracle.com> References: <1544152185-32667-1-git-send-email-jianchao.w.wang@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9099 signatures=668679 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=1 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1812070024 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP It is not necessary to issue request directly with bypass 'true' in blk_mq_sched_insert_requests and handle the non-issued requests itself. Just set bypass to 'false' and let blk_mq_try_issue_directly handle them totally. Remove the blk_rq_can_direct_dispatch check, because blk_mq_try_issue_directly can handle it well.If request is direct-issued unsuccessfully, insert the reset. Signed-off-by: Jianchao Wang --- block/blk-mq-sched.c | 8 +++----- block/blk-mq.c | 29 ++++++++++------------------- 2 files changed, 13 insertions(+), 24 deletions(-) diff --git a/block/blk-mq-sched.c b/block/blk-mq-sched.c index f096d898..5b4d52d 100644 --- a/block/blk-mq-sched.c +++ b/block/blk-mq-sched.c @@ -417,12 +417,10 @@ void blk_mq_sched_insert_requests(struct blk_mq_hw_ctx *hctx, * busy in case of 'none' scheduler, and this way may save * us one extra enqueue & dequeue to sw queue. */ - if (!hctx->dispatch_busy && !e && !run_queue_async) { + if (!hctx->dispatch_busy && !e && !run_queue_async) blk_mq_try_issue_list_directly(hctx, list); - if (list_empty(list)) - return; - } - blk_mq_insert_requests(hctx, ctx, list); + else + blk_mq_insert_requests(hctx, ctx, list); } blk_mq_run_hw_queue(hctx, run_queue_async); diff --git a/block/blk-mq.c b/block/blk-mq.c index 88ee447..3265e30 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -1904,32 +1904,23 @@ blk_status_t blk_mq_request_issue_directly(struct request *rq, bool last) void blk_mq_try_issue_list_directly(struct blk_mq_hw_ctx *hctx, struct list_head *list) { + blk_qc_t unused; + blk_status_t ret = BLK_STS_OK; + while (!list_empty(list)) { - blk_status_t ret; struct request *rq = list_first_entry(list, struct request, queuelist); - if (!blk_rq_can_direct_dispatch(rq)) - break; - list_del_init(&rq->queuelist); - ret = blk_mq_request_issue_directly(rq, list_empty(list)); - if (ret != BLK_STS_OK) { - if (ret == BLK_STS_RESOURCE || - ret == BLK_STS_DEV_RESOURCE) { - list_add(&rq->queuelist, list); - break; - } - blk_mq_end_request(rq, ret); - } + if (ret == BLK_STS_OK) + ret = blk_mq_try_issue_directly(hctx, rq, &unused, + false, + list_empty(list)); + else + blk_mq_sched_insert_request(rq, false, true, false); } - /* - * If we didn't flush the entire list, we could have told - * the driver there was more coming, but that turned out to - * be a lie. - */ - if (!list_empty(list) && hctx->queue->mq_ops->commit_rqs) + if (ret != BLK_STS_OK && hctx->queue->mq_ops->commit_rqs) hctx->queue->mq_ops->commit_rqs(hctx); } From patchwork Fri Dec 7 03:09:45 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "jianchao.wang" X-Patchwork-Id: 10717387 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 95A8A18A7 for ; Fri, 7 Dec 2018 03:10:47 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 822292E944 for ; Fri, 7 Dec 2018 03:10:47 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7687D2E9AB; Fri, 7 Dec 2018 03:10:47 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1C3F72E944 for ; Fri, 7 Dec 2018 03:10:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725986AbeLGDKl (ORCPT ); Thu, 6 Dec 2018 22:10:41 -0500 Received: from userp2130.oracle.com ([156.151.31.86]:51182 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726043AbeLGDKk (ORCPT ); Thu, 6 Dec 2018 22:10:40 -0500 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.22/8.16.0.22) with SMTP id wB72x7Xc018748; Fri, 7 Dec 2018 03:10:37 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=OeMPcaxDTjTZYrQXzZRWlHK02ZDBV0lCZIhqry/BzG0=; b=2CavLLGlQ/w1kn0veZGRzZcIT1tqld+PAPkf0uV5UmjA3PJKnUxt9ZRmhGvIhpY1unSM SmXjFADt9lKEz34aW9hI914JHfobzWk7l1eA8kI4RJU++Lm3lGT1VviUhojxWVXMxyOS s5DP1AumrHhSOGr6ByU1kC0Nq2V+CU9weKMMpp99poqWCoVMADXlIHTSLaCLXSzzb1YI /6Wre9T+jNAA64XdVBnQCq2Xdb+KVkcEcTLywduXw1gq/RVEzh/Zhfic4bguNa6Znwmd ZAeJPgWBzLAwExsngOFJViyXTVxEAVNCVTDRjKVXlB02Im5FiI85dcqKyRyz5JOnyBIy 7g== Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by userp2130.oracle.com with ESMTP id 2p3hqubkc5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 07 Dec 2018 03:10:37 +0000 Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id wB73AVdo014748 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 7 Dec 2018 03:10:31 GMT Received: from abhmp0005.oracle.com (abhmp0005.oracle.com [141.146.116.11]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id wB73AV29028318; Fri, 7 Dec 2018 03:10:31 GMT Received: from will-ThinkCentre-M93p.cn.oracle.com (/10.182.70.234) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 06 Dec 2018 19:10:30 -0800 From: Jianchao Wang To: axboe@kernel.dk Cc: ming.lei@redhat.com, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH V11 4/4] blk-mq: replace and kill blk_mq_request_issue_directly Date: Fri, 7 Dec 2018 11:09:45 +0800 Message-Id: <1544152185-32667-5-git-send-email-jianchao.w.wang@oracle.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1544152185-32667-1-git-send-email-jianchao.w.wang@oracle.com> References: <1544152185-32667-1-git-send-email-jianchao.w.wang@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9099 signatures=668679 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=1 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=959 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1812070024 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Replace blk_mq_request_issue_directly with blk_mq_try_issue_directly in blk_insert_cloned_request and kill it as nobody uses it any more. Signed-off-by: Jianchao Wang --- block/blk-core.c | 4 +++- block/blk-mq.c | 9 +-------- block/blk-mq.h | 6 ++++-- 3 files changed, 8 insertions(+), 11 deletions(-) diff --git a/block/blk-core.c b/block/blk-core.c index ad59102..c92d866 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -1297,6 +1297,8 @@ static int blk_cloned_rq_check_limits(struct request_queue *q, */ blk_status_t blk_insert_cloned_request(struct request_queue *q, struct request *rq) { + blk_qc_t unused; + if (blk_cloned_rq_check_limits(q, rq)) return BLK_STS_IOERR; @@ -1312,7 +1314,7 @@ blk_status_t blk_insert_cloned_request(struct request_queue *q, struct request * * bypass a potential scheduler on the bottom device for * insert. */ - return blk_mq_request_issue_directly(rq, true); + return blk_mq_try_issue_directly(rq->mq_hctx, rq, &unused, true, true); } EXPORT_SYMBOL_GPL(blk_insert_cloned_request); diff --git a/block/blk-mq.c b/block/blk-mq.c index 3265e30..32102d0 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -1815,7 +1815,7 @@ static bool blk_rq_can_direct_dispatch(struct request *rq) return req_op(rq) == REQ_OP_READ || req_op(rq) == REQ_OP_WRITE; } -static blk_status_t blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, +blk_status_t blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, struct request *rq, blk_qc_t *cookie, bool bypass, bool last) @@ -1894,13 +1894,6 @@ static blk_status_t blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, return ret; } -blk_status_t blk_mq_request_issue_directly(struct request *rq, bool last) -{ - blk_qc_t unused; - - return blk_mq_try_issue_directly(rq->mq_hctx, rq, &unused, true, last); -} - void blk_mq_try_issue_list_directly(struct blk_mq_hw_ctx *hctx, struct list_head *list) { diff --git a/block/blk-mq.h b/block/blk-mq.h index a664ea4..b81e619 100644 --- a/block/blk-mq.h +++ b/block/blk-mq.h @@ -68,8 +68,10 @@ void blk_mq_request_bypass_insert(struct request *rq, bool run_queue); void blk_mq_insert_requests(struct blk_mq_hw_ctx *hctx, struct blk_mq_ctx *ctx, struct list_head *list); -/* Used by blk_insert_cloned_request() to issue request directly */ -blk_status_t blk_mq_request_issue_directly(struct request *rq, bool last); +blk_status_t blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, + struct request *rq, + blk_qc_t *cookie, + bool bypass, bool last); void blk_mq_try_issue_list_directly(struct blk_mq_hw_ctx *hctx, struct list_head *list);