From patchwork Mon Oct 29 08:33:30 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "jianchao.wang" X-Patchwork-Id: 10658831 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C60FD13A9 for ; Mon, 29 Oct 2018 08:34:14 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B43B6296CE for ; Mon, 29 Oct 2018 08:34:14 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A87A129733; Mon, 29 Oct 2018 08:34:14 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0AD03296CE for ; Mon, 29 Oct 2018 08:34:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729435AbeJ2RVt (ORCPT ); Mon, 29 Oct 2018 13:21:49 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:37218 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729286AbeJ2RVs (ORCPT ); Mon, 29 Oct 2018 13:21:48 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.22/8.16.0.22) with SMTP id w9T8Skd6158539; Mon, 29 Oct 2018 08:34:07 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=/UAA579QFMm6BaKM6RhEXltpFuVXLtDa+CQ+VYFIe5A=; b=tL+FbmB3mhlaAQKbkVsCMm/mDhHu97IQjlRfNlnYzsBJeag6pf99cROFHgNC5PTztioR TL8XHRWLTZ7qgpxjqbVlcZB3x3X/HoqCitRz5QQvpx2gAJgAjuAwASiMuo7AxTPzJJ7o bXt/4sGPvUy/hNGnZSRSX6SCR1mokiNw8YqM9OJ6BYCPOkrNHokwjcQfYBQVoKaeRngT N5XSqUOTikFaNFbXFnkdpPzV9I5jyNjwsDkDjNUfsDCaT5xT/Xiv0ZyQLqTQ0fW308W4 oDWnpgmUe+uSjjJ2Cxl9BS0YcgFWKJPjBPRo7MzQk7AXR+lmGmciopYYD/OPZYdf4iQH fg== Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by userp2120.oracle.com with ESMTP id 2ncgnqm449-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 29 Oct 2018 08:34:07 +0000 Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id w9T8Y6oK012964 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 29 Oct 2018 08:34:07 GMT Received: from abhmp0007.oracle.com (abhmp0007.oracle.com [141.146.116.13]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id w9T8Y6k8020152; Mon, 29 Oct 2018 08:34:06 GMT Received: from will-ThinkCentre-M93p.cn.oracle.com (/10.182.70.234) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 29 Oct 2018 01:34:06 -0700 From: Jianchao Wang To: axboe@kernel.dk Cc: ming.lei@redhat.com, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH V3 1/4] blk-mq: refactor the code of issue request directly Date: Mon, 29 Oct 2018 16:33:30 +0800 Message-Id: <1540802013-2921-2-git-send-email-jianchao.w.wang@oracle.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1540802013-2921-1-git-send-email-jianchao.w.wang@oracle.com> References: <1540802013-2921-1-git-send-email-jianchao.w.wang@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9060 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=1 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1807170000 definitions=main-1810290082 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Merge blk_mq_try_issue_directly and __blk_mq_try_issue_directly into one interface which is able to handle the return value from .queue_rq callback. To make the code clearer, introduce new helpers blk_mq_make_decision and enum mq_decision. Signed-off-by: Jianchao Wang --- block/blk-mq.c | 104 ++++++++++++++++++++++++++++++----------------------- block/blk-mq.c.rej | 60 +++++++++++++++++++++++++++++++ 2 files changed, 120 insertions(+), 44 deletions(-) create mode 100644 block/blk-mq.c.rej diff --git a/block/blk-mq.c b/block/blk-mq.c index dcf10e3..0f6328b 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -1711,7 +1711,6 @@ static blk_status_t __blk_mq_issue_directly(struct blk_mq_hw_ctx *hctx, switch (ret) { case BLK_STS_OK: blk_mq_update_dispatch_busy(hctx, false); - *cookie = new_cookie; break; case BLK_STS_RESOURCE: case BLK_STS_DEV_RESOURCE: @@ -1720,86 +1719,103 @@ static blk_status_t __blk_mq_issue_directly(struct blk_mq_hw_ctx *hctx, break; default: blk_mq_update_dispatch_busy(hctx, false); - *cookie = BLK_QC_T_NONE; + new_cookie = BLK_QC_T_NONE; break; } + if (cookie) + *cookie = new_cookie; return ret; } -static blk_status_t __blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, +enum mq_decision { + MQ_INSERT_QUEUE, + MQ_END_REQUEST, + MQ_DO_NOTHING, +}; + +static inline enum mq_decision + blk_mq_make_dicision(blk_status_t ret, bool bypass_insert) +{ + enum mq_decision dec; + + switch(ret) { + case BLK_STS_OK: + dec = MQ_DO_NOTHING; + break; + case BLK_STS_DEV_RESOURCE: + case BLK_STS_RESOURCE: + dec = bypass_insert ? MQ_DO_NOTHING : MQ_INSERT_QUEUE; + break; + default: + dec = bypass_insert ? MQ_DO_NOTHING : MQ_END_REQUEST; + break; + } + + return dec; +} + +static blk_status_t blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, struct request *rq, blk_qc_t *cookie, bool bypass_insert) { struct request_queue *q = rq->q; bool run_queue = true; + enum mq_decision dec; + blk_status_t ret = BLK_STS_RESOURCE; + int srcu_idx; + hctx_lock(hctx, &srcu_idx); /* - * RCU or SRCU read lock is needed before checking quiesced flag. + * hctx_lock is needed before checking quiesced flag. * - * When queue is stopped or quiesced, ignore 'bypass_insert' from - * blk_mq_request_issue_directly(), and return BLK_STS_OK to caller, - * and avoid driver to try to dispatch again. + * When queue is stopped or quiesced, ignore 'bypass_insert', + * and return BLK_STS_OK to caller, and avoid driver to try to + * dispatch again. */ if (blk_mq_hctx_stopped(hctx) || blk_queue_quiesced(q)) { run_queue = false; bypass_insert = false; - goto insert; + goto out_unlock; } if (q->elevator && !bypass_insert) - goto insert; + goto out_unlock; if (!blk_mq_get_dispatch_budget(hctx)) - goto insert; + goto out_unlock; if (!blk_mq_get_driver_tag(rq)) { blk_mq_put_dispatch_budget(hctx); - goto insert; + goto out_unlock; } - return __blk_mq_issue_directly(hctx, rq, cookie); -insert: - if (bypass_insert) - return BLK_STS_RESOURCE; - - blk_mq_sched_insert_request(rq, false, run_queue, false); - return BLK_STS_OK; -} - -static void blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, - struct request *rq, blk_qc_t *cookie) -{ - blk_status_t ret; - int srcu_idx; - - might_sleep_if(hctx->flags & BLK_MQ_F_BLOCKING); - - hctx_lock(hctx, &srcu_idx); + ret = __blk_mq_issue_directly(hctx, rq, cookie); +out_unlock: + hctx_unlock(hctx, srcu_idx); - ret = __blk_mq_try_issue_directly(hctx, rq, cookie, false); - if (ret == BLK_STS_RESOURCE || ret == BLK_STS_DEV_RESOURCE) - blk_mq_sched_insert_request(rq, false, true, false); - else if (ret != BLK_STS_OK) + dec = blk_mq_make_dicision(ret, bypass_insert); + switch(dec) { + case MQ_INSERT_QUEUE: + blk_mq_sched_insert_request(rq, false, run_queue, false); + break; + case MQ_END_REQUEST: blk_mq_end_request(rq, ret); + break; + default: + return ret; + } - hctx_unlock(hctx, srcu_idx); + return BLK_STS_OK; } blk_status_t blk_mq_request_issue_directly(struct request *rq) { - blk_status_t ret; - int srcu_idx; - blk_qc_t unused_cookie; struct blk_mq_ctx *ctx = rq->mq_ctx; struct blk_mq_hw_ctx *hctx = blk_mq_map_queue(rq->q, ctx->cpu); - hctx_lock(hctx, &srcu_idx); - ret = __blk_mq_try_issue_directly(hctx, rq, &unused_cookie, true); - hctx_unlock(hctx, srcu_idx); - - return ret; + return blk_mq_try_issue_directly(hctx, rq, NULL, true); } void blk_mq_try_issue_list_directly(struct blk_mq_hw_ctx *hctx, @@ -1921,13 +1937,13 @@ static blk_qc_t blk_mq_make_request(struct request_queue *q, struct bio *bio) data.hctx = blk_mq_map_queue(q, same_queue_rq->mq_ctx->cpu); blk_mq_try_issue_directly(data.hctx, same_queue_rq, - &cookie); + &cookie, false); } } else if ((q->nr_hw_queues > 1 && is_sync) || (!q->elevator && !data.hctx->dispatch_busy)) { blk_mq_put_ctx(data.ctx); blk_mq_bio_to_request(rq, bio); - blk_mq_try_issue_directly(data.hctx, rq, &cookie); + blk_mq_try_issue_directly(data.hctx, rq, &cookie, false); } else { blk_mq_put_ctx(data.ctx); blk_mq_bio_to_request(rq, bio); diff --git a/block/blk-mq.c.rej b/block/blk-mq.c.rej new file mode 100644 index 0000000..7813cd9 --- /dev/null +++ b/block/blk-mq.c.rej @@ -0,0 +1,60 @@ +--- block/blk-mq.c ++++ block/blk-mq.c +@@ -1728,13 +1728,14 @@ static blk_status_t __blk_mq_issue_directly(struct blk_mq_hw_ctx *hctx, + } + + enum mq_decision { ++ MQ_INSERT_DISPATCH, + MQ_INSERT_QUEUE, + MQ_END_REQUEST, + MQ_DO_NOTHING, + }; + +-static inline enum mq_decision +- blk_mq_make_dicision(blk_status_t ret, bool bypass_insert) ++static inline enum mq_decision blk_mq_make_dicision(blk_status_t ret, ++ bool bypass_insert, bool force_insert) + { + enum mq_decision dec; + +@@ -1744,7 +1745,10 @@ static inline enum mq_decision + break; + case BLK_STS_DEV_RESOURCE: + case BLK_STS_RESOURCE: +- dec = bypass_insert ? MQ_DO_NOTHING : MQ_INSERT_QUEUE; ++ if (force_insert) ++ dec = bypass_insert ? MQ_INSERT_DISPATCH : MQ_INSERT_QUEUE; ++ else ++ dec = bypass_insert ? MQ_DO_NOTHING : MQ_INSERT_QUEUE; + break; + default: + dec = bypass_insert ? MQ_DO_NOTHING : MQ_END_REQUEST; +@@ -1760,7 +1764,7 @@ static blk_status_t blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, + bool bypass_insert) + { + struct request_queue *q = rq->q; +- bool run_queue = true; ++ bool run_queue = true, force_insert = false; + enum mq_decision dec; + blk_status_t ret = BLK_STS_RESOURCE; + int srcu_idx; +@@ -1775,7 +1779,7 @@ static blk_status_t blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, + */ + if (blk_mq_hctx_stopped(hctx) || blk_queue_quiesced(q)) { + run_queue = false; +- bypass_insert = false; ++ force_insert = true; + goto out_unlock; + } + +@@ -1795,7 +1799,10 @@ static blk_status_t blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, + hctx_unlock(hctx, srcu_idx); + + dec = blk_mq_make_dicision(ret, bypass_insert); + switch(pos) { ++ case MQ_INSERT_DISPATCH: ++ blk_mq_request_bypass_insert(rq, run_queue); ++ break; + case MQ_INSERT_QUEUE: + blk_mq_sched_insert_request(rq, false, run_queue, false); + break; From patchwork Mon Oct 29 08:33:31 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "jianchao.wang" X-Patchwork-Id: 10658837 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4EA20174F for ; Mon, 29 Oct 2018 08:34:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3BE1128B00 for ; Mon, 29 Oct 2018 08:34:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2D5A0295FA; Mon, 29 Oct 2018 08:34:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BFD23295FA for ; Mon, 29 Oct 2018 08:34:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726036AbeJ2RVw (ORCPT ); Mon, 29 Oct 2018 13:21:52 -0400 Received: from aserp2120.oracle.com ([141.146.126.78]:37816 "EHLO aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729436AbeJ2RVu (ORCPT ); Mon, 29 Oct 2018 13:21:50 -0400 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.22/8.16.0.22) with SMTP id w9T8TTwq013617; Mon, 29 Oct 2018 08:34:09 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=ljr1AVEDeyqNV2z//1XrWvkspXgCo1lAbTrhPYSDkVY=; b=4iZysp28Gnx43sHBnyCb1jPyW2Q+9mlOOb+NLiAnv5O93edK8fZS4uxX6lICXNRZ/3Az ju0C1wmIyzndC4jgIDtj/zM/IJ9qjPiwt5gM3JBN6d8w3zQwFE1cFUXu82ELSs+O46nR L82pRQbh8iYAF1nMK7VEz2qQpqa0i9e5mqkjZoGzX+RpZh/sTgaR7R3NfF7Oe74R3xwj 56XyPvR/ZrZ1alXCPB7pgULf6As+yjj2lvZj3QGNQ2DX513/TlY7SKuCzRsE2M0cwpaY DgGNuTeFqBSIhxnUgyt4O4lDWoBGxNuxi3Ay/bS3bebAuuW5V4UJrRP2Ov2Rt/IJn3gg nw== Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by aserp2120.oracle.com with ESMTP id 2ncfypm5vd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 29 Oct 2018 08:34:09 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id w9T8Y8jt013068 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 29 Oct 2018 08:34:08 GMT Received: from abhmp0007.oracle.com (abhmp0007.oracle.com [141.146.116.13]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id w9T8Y8ud010591; Mon, 29 Oct 2018 08:34:08 GMT Received: from will-ThinkCentre-M93p.cn.oracle.com (/10.182.70.234) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 29 Oct 2018 01:34:08 -0700 From: Jianchao Wang To: axboe@kernel.dk Cc: ming.lei@redhat.com, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH V3 2/4] blk-mq: insert request without involving any io scheduler Date: Mon, 29 Oct 2018 16:33:31 +0800 Message-Id: <1540802013-2921-3-git-send-email-jianchao.w.wang@oracle.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1540802013-2921-1-git-send-email-jianchao.w.wang@oracle.com> References: <1540802013-2921-1-git-send-email-jianchao.w.wang@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9060 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=1 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1807170000 definitions=main-1810290082 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP When try to issue request directly, if the queue is stopped or quiesced, 'bypass_insert' will be ignored and return BLK_STS_OK to caller to avoid it dispatch request again. Then the request will be inserted with blk_mq_sched_insert_request. This is not correct for dm-rq case where we should avoid to pass through the underlying paths' io scheduler. To fix it, use blk_mq_request_bypass_insert to insert the request into hctx->dispatch directly. Signed-off-by: Jianchao Wang --- block/blk-mq.c | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 0f6328b..e0aa068 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -1729,13 +1729,14 @@ static blk_status_t __blk_mq_issue_directly(struct blk_mq_hw_ctx *hctx, } enum mq_decision { + MQ_INSERT_DISPATCH, MQ_INSERT_QUEUE, MQ_END_REQUEST, MQ_DO_NOTHING, }; static inline enum mq_decision - blk_mq_make_dicision(blk_status_t ret, bool bypass_insert) + blk_mq_make_dicision(blk_status_t ret, bool bypass_insert, bool force) { enum mq_decision dec; @@ -1745,7 +1746,10 @@ static inline enum mq_decision break; case BLK_STS_DEV_RESOURCE: case BLK_STS_RESOURCE: - dec = bypass_insert ? MQ_DO_NOTHING : MQ_INSERT_QUEUE; + if (force) + dec = bypass_insert ? MQ_INSERT_DISPATCH : MQ_INSERT_QUEUE; + else + dec = bypass_insert ? MQ_DO_NOTHING : MQ_INSERT_QUEUE; break; default: dec = bypass_insert ? MQ_DO_NOTHING : MQ_END_REQUEST; @@ -1761,7 +1765,7 @@ static blk_status_t blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, bool bypass_insert) { struct request_queue *q = rq->q; - bool run_queue = true; + bool run_queue = true, force = false; enum mq_decision dec; blk_status_t ret = BLK_STS_RESOURCE; int srcu_idx; @@ -1776,7 +1780,7 @@ static blk_status_t blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, */ if (blk_mq_hctx_stopped(hctx) || blk_queue_quiesced(q)) { run_queue = false; - bypass_insert = false; + force = true; goto out_unlock; } @@ -1795,8 +1799,11 @@ static blk_status_t blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, out_unlock: hctx_unlock(hctx, srcu_idx); - dec = blk_mq_make_dicision(ret, bypass_insert); + dec = blk_mq_make_dicision(ret, bypass_insert, force); switch(dec) { + case MQ_INSERT_DISPATCH: + blk_mq_request_bypass_insert(rq, run_queue); + break; case MQ_INSERT_QUEUE: blk_mq_sched_insert_request(rq, false, run_queue, false); break; From patchwork Mon Oct 29 08:33:32 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "jianchao.wang" X-Patchwork-Id: 10658833 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C1E4513A9 for ; Mon, 29 Oct 2018 08:34:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AF1BA28B00 for ; Mon, 29 Oct 2018 08:34:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A2F382960E; Mon, 29 Oct 2018 08:34:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 92A1828B00 for ; Mon, 29 Oct 2018 08:34:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729255AbeJ2RV6 (ORCPT ); Mon, 29 Oct 2018 13:21:58 -0400 Received: from aserp2120.oracle.com ([141.146.126.78]:37968 "EHLO aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729149AbeJ2RV5 (ORCPT ); Mon, 29 Oct 2018 13:21:57 -0400 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.22/8.16.0.22) with SMTP id w9T8Spjm013186; Mon, 29 Oct 2018 08:34:16 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=FeJiBIM4gj+Uv4+k6aC45dq/teyiiFKjX9alY/b/HNs=; b=g833a1kppqZdB0Qna3OBPACCftFIIYLwAJY5hXtVkGzzXwCCoZu7+jrJ8lmv0J/6SKJV zjzDq98hUVt0ia6UXcgGgUe+HlMSExoR9nDV30ddDlz/2yXnffR0hlXCKw0Coj/33tcd OPU4bVT8XG2sg7F84IZ/jwEINt5OuTXLT352T12ugbZ27TaeQ6VULAOCkRH92GwEMGgE tPzkBdkdJ0YvdtNFrzuIEG9MX5I2gi65UGWLjqz+yRhHBakmP+AGfOfx8lDpWeebGkCE ZhZlS70Sd0BQIZTUX6eTXeSLiNkWOF88b8kgV5cwjL+w53UXk5yAz7WZgnvKr/AklHFX yQ== Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by aserp2120.oracle.com with ESMTP id 2ncfypm5ws-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 29 Oct 2018 08:34:16 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id w9T8YAXE027565 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 29 Oct 2018 08:34:10 GMT Received: from abhmp0007.oracle.com (abhmp0007.oracle.com [141.146.116.13]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id w9T8YAJH008228; Mon, 29 Oct 2018 08:34:10 GMT Received: from will-ThinkCentre-M93p.cn.oracle.com (/10.182.70.234) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 29 Oct 2018 01:34:09 -0700 From: Jianchao Wang To: axboe@kernel.dk Cc: ming.lei@redhat.com, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH V3 3/4] blk-mq: issue directly with bypass_insert 'false' in blk_mq_sched_insert_requests Date: Mon, 29 Oct 2018 16:33:32 +0800 Message-Id: <1540802013-2921-4-git-send-email-jianchao.w.wang@oracle.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1540802013-2921-1-git-send-email-jianchao.w.wang@oracle.com> References: <1540802013-2921-1-git-send-email-jianchao.w.wang@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9060 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=1 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1807170000 definitions=main-1810290082 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP It is not necessary to issue request directly with bypass_insert 'true' in blk_mq_sched_insert_requests and insert the non-issued requests itself. Just set bypass_insert to 'false' and let blk_mq_try_issue_directly handle them. Signed-off-by: Jianchao Wang --- block/blk-mq-sched.c | 8 +++----- block/blk-mq.c | 10 +--------- 2 files changed, 4 insertions(+), 14 deletions(-) diff --git a/block/blk-mq-sched.c b/block/blk-mq-sched.c index 29bfe80..23cd97e 100644 --- a/block/blk-mq-sched.c +++ b/block/blk-mq-sched.c @@ -411,12 +411,10 @@ void blk_mq_sched_insert_requests(struct request_queue *q, * busy in case of 'none' scheduler, and this way may save * us one extra enqueue & dequeue to sw queue. */ - if (!hctx->dispatch_busy && !e && !run_queue_async) { + if (!hctx->dispatch_busy && !e && !run_queue_async) blk_mq_try_issue_list_directly(hctx, list); - if (list_empty(list)) - return; - } - blk_mq_insert_requests(hctx, ctx, list); + else + blk_mq_insert_requests(hctx, ctx, list); } blk_mq_run_hw_queue(hctx, run_queue_async); diff --git a/block/blk-mq.c b/block/blk-mq.c index e0aa068..9c6c858 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -1834,15 +1834,7 @@ void blk_mq_try_issue_list_directly(struct blk_mq_hw_ctx *hctx, queuelist); list_del_init(&rq->queuelist); - ret = blk_mq_request_issue_directly(rq); - if (ret != BLK_STS_OK) { - if (ret == BLK_STS_RESOURCE || - ret == BLK_STS_DEV_RESOURCE) { - list_add(&rq->queuelist, list); - break; - } - blk_mq_end_request(rq, ret); - } + ret = blk_mq_try_issue_directly(hctx, rq, NULL, false); } } From patchwork Mon Oct 29 08:33:33 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "jianchao.wang" X-Patchwork-Id: 10658835 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8C37713A9 for ; Mon, 29 Oct 2018 08:34:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 789FA295FA for ; Mon, 29 Oct 2018 08:34:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6CBCD2960E; Mon, 29 Oct 2018 08:34:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5CAC6295FA for ; Mon, 29 Oct 2018 08:34:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729436AbeJ2RVx (ORCPT ); Mon, 29 Oct 2018 13:21:53 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:46636 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729286AbeJ2RVw (ORCPT ); Mon, 29 Oct 2018 13:21:52 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.22/8.16.0.22) with SMTP id w9T8Snnk008905; Mon, 29 Oct 2018 08:34:12 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=ioT/yFIjJIy+Qg/6RzlFcZD3/CAktU68agYXTE+or0o=; b=PsPj23KBL6Mx+3Coxb/n6/FbbyCx1+vXrU7UDo4lsNt4svS6F0DGPKN6a8u5GGYPIA6x BZTyvVdQiCPHnC2zgU0rjmxEZMM8CiZiJab3Vq12mKMTOeEJ5zIAXhG2+jn1BgLA/ALS aGZ/Suk0Gg3eHYsqiUVASmzAzevElYaehcaZdL6aHOXWVnMEqSey0pKHm5Ykow5HfeIJ qgsjEj51INPLjoTd2BJFvQYT6cIhvJaE0eYlQrzMSAkcsAMxgmBuySDEJaOoxVmIRAec 8fAqaB7QwOhoBAVGqAHWYR9d+Jgd2vG1TkzBDutkYYnbEbkh2IulN6/qG/sNDn1rXs6k fQ== Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by userp2130.oracle.com with ESMTP id 2nduckrt2u-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 29 Oct 2018 08:34:12 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id w9T8YCqW013222 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 29 Oct 2018 08:34:12 GMT Received: from abhmp0007.oracle.com (abhmp0007.oracle.com [141.146.116.13]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id w9T8YBQH010597; Mon, 29 Oct 2018 08:34:11 GMT Received: from will-ThinkCentre-M93p.cn.oracle.com (/10.182.70.234) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 29 Oct 2018 01:34:11 -0700 From: Jianchao Wang To: axboe@kernel.dk Cc: ming.lei@redhat.com, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH V3 4/4] blk-mq: ensure hctx to be ran on mapped cpu when issue directly Date: Mon, 29 Oct 2018 16:33:33 +0800 Message-Id: <1540802013-2921-5-git-send-email-jianchao.w.wang@oracle.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1540802013-2921-1-git-send-email-jianchao.w.wang@oracle.com> References: <1540802013-2921-1-git-send-email-jianchao.w.wang@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9060 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=1 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=947 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1807170000 definitions=main-1810290082 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP When issue request directly and the task is migrated out of the original cpu where it allocates request, hctx could be ran on the cpu where it is not mapped. To fix this, insert the request if BLK_MQ_F_BLOCKING is set, check whether the current is mapped to the hctx and invoke __blk_mq_issue_directly under preemption disabled. Signed-off-by: Jianchao Wang --- block/blk-mq.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 9c6c858..ced3346 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -1770,6 +1770,17 @@ static blk_status_t blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, blk_status_t ret = BLK_STS_RESOURCE; int srcu_idx; + if (hctx->flags & BLK_MQ_F_BLOCKING) { + force = true; + goto out; + } + + if (!cpumask_test_cpu(get_cpu(), hctx->cpumask)) { + put_cpu(); + force = true; + goto out; + } + hctx_lock(hctx, &srcu_idx); /* * hctx_lock is needed before checking quiesced flag. @@ -1798,7 +1809,8 @@ static blk_status_t blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, ret = __blk_mq_issue_directly(hctx, rq, cookie); out_unlock: hctx_unlock(hctx, srcu_idx); - + put_cpu(); +out: dec = blk_mq_make_dicision(ret, bypass_insert, force); switch(dec) { case MQ_INSERT_DISPATCH: