From patchwork Mon Nov 12 09:23:25 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "jianchao.wang" X-Patchwork-Id: 10678371 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 344821759 for ; Mon, 12 Nov 2018 09:24:53 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2429E29FA8 for ; Mon, 12 Nov 2018 09:24:53 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 17EE829FAA; Mon, 12 Nov 2018 09:24:53 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 88A1D29FA8 for ; Mon, 12 Nov 2018 09:24:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726161AbeKLTQf (ORCPT ); Mon, 12 Nov 2018 14:16:35 -0500 Received: from userp2120.oracle.com ([156.151.31.85]:55398 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727204AbeKLTQd (ORCPT ); Mon, 12 Nov 2018 14:16:33 -0500 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.22/8.16.0.22) with SMTP id wAC9OApX073753; Mon, 12 Nov 2018 09:24:10 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=oUUhis9lyTm6Y3TOYyVY5eEkjDm3iPjB9EJ1NqiJNLE=; b=mDVh2yRV59XHELXojZqS8B45c4xMlOJAaUDr3BJqIna2EEBtkgzXN/fN4d/tW6Mose0b aZOIr9uA0lqpgT0F9IKioifmtt+/Ck7Zne3a12l2HNFcTF4DluRA923nwFaNVvkBvOFT sbRliJUxOyAcht3BhS48lDkdSF7Exe5l07lhM4nH299tEZ2RyIVxrh6qLU1dN4cgdEWO wifTYOndTxVp8/25Um7JO2Lvlx8r96/flJ7Biegr7G3mpKOZALKMUtmkrJNs6rqwnyrT BUcKC1lFiEIy3+w8lyL8oTyGSvzB1ctLW2GYxeMSZ7F2piUe5W9FawPRCIs6U7hki6UL rQ== Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by userp2120.oracle.com with ESMTP id 2nnwc0c4kd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 12 Nov 2018 09:24:10 +0000 Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id wAC9O936009708 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 12 Nov 2018 09:24:10 GMT Received: from abhmp0012.oracle.com (abhmp0012.oracle.com [141.146.116.18]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id wAC9O9un007287; Mon, 12 Nov 2018 09:24:09 GMT Received: from will-ThinkCentre-M93p.cn.oracle.com (/10.182.70.234) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 12 Nov 2018 01:24:09 -0800 From: Jianchao Wang To: axboe@kernel.dk Cc: ming.lei@redhat.com, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH V5 2/6] blk-mq: refactor the code of issue request directly Date: Mon, 12 Nov 2018 17:23:25 +0800 Message-Id: <1542014609-1769-3-git-send-email-jianchao.w.wang@oracle.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1542014609-1769-1-git-send-email-jianchao.w.wang@oracle.com> References: <1542014609-1769-1-git-send-email-jianchao.w.wang@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9074 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=1 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1807170000 definitions=main-1811120086 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Merge blk_mq_try_issue_directly and __blk_mq_try_issue_directly into one interface to unify the interfaces to issue requests directly. The merged interface takes over the requests totally, it could insert, end or do nothing based on the return value of .queue_rq and 'bypass' parameter. Then caller needn't any other handling any more. To make code clearer, introduce new helpers enum mq_issue_decision and blk_mq_make_decision to decide how to handle the non-issued requests. Signed-off-by: Jianchao Wang --- block/blk-mq.c | 108 +++++++++++++++++++++++++++++++++------------------------ 1 file changed, 63 insertions(+), 45 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 364a53f..48b7a7c 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -1766,77 +1766,95 @@ static blk_status_t __blk_mq_issue_directly(struct blk_mq_hw_ctx *hctx, return ret; } -static blk_status_t __blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, +enum mq_issue_decision { + MQ_ISSUE_INSERT_QUEUE, + MQ_ISSUE_END_REQUEST, + MQ_ISSUE_DO_NOTHING, +}; + +static inline enum mq_issue_decision + blk_mq_make_dicision(blk_status_t ret, bool bypass) +{ + enum mq_issue_decision dec; + + switch(ret) { + case BLK_STS_OK: + dec = MQ_ISSUE_DO_NOTHING; + break; + case BLK_STS_DEV_RESOURCE: + case BLK_STS_RESOURCE: + dec = bypass ? MQ_ISSUE_DO_NOTHING : MQ_ISSUE_INSERT_QUEUE; + break; + default: + dec = bypass ? MQ_ISSUE_DO_NOTHING : MQ_ISSUE_END_REQUEST; + break; + } + + return dec; +} + +static blk_status_t blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, struct request *rq, blk_qc_t *cookie, - bool bypass_insert) + bool bypass) { struct request_queue *q = rq->q; bool run_queue = true; + blk_status_t ret = BLK_STS_RESOURCE; + enum mq_issue_decision dec; + int srcu_idx; + hctx_lock(hctx, &srcu_idx); /* - * RCU or SRCU read lock is needed before checking quiesced flag. + * hctx_lock is needed before checking quiesced flag. * - * When queue is stopped or quiesced, ignore 'bypass_insert' from - * blk_mq_request_issue_directly(), and return BLK_STS_OK to caller, - * and avoid driver to try to dispatch again. + * When queue is stopped or quiesced, ignore 'bypass', insert + * and return BLK_STS_OK to caller, and avoid driver to try to + * dispatch again. */ if (blk_mq_hctx_stopped(hctx) || blk_queue_quiesced(q)) { run_queue = false; - bypass_insert = false; - goto insert; + bypass = false; + goto out_unlock; } - if (q->elevator && !bypass_insert) - goto insert; + /* + * Bypass the potential scheduler on the bottom device. + */ + if (q->elevator && !bypass) + goto out_unlock; if (!blk_mq_get_dispatch_budget(hctx)) - goto insert; + goto out_unlock; if (!blk_mq_get_driver_tag(rq)) { blk_mq_put_dispatch_budget(hctx); - goto insert; + goto out_unlock; } - return __blk_mq_issue_directly(hctx, rq, cookie); -insert: - if (bypass_insert) - return BLK_STS_RESOURCE; - - blk_mq_sched_insert_request(rq, false, run_queue, false); - return BLK_STS_OK; -} - -static void blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, - struct request *rq, blk_qc_t *cookie) -{ - blk_status_t ret; - int srcu_idx; + ret = __blk_mq_issue_directly(hctx, rq, cookie); - might_sleep_if(hctx->flags & BLK_MQ_F_BLOCKING); - - hctx_lock(hctx, &srcu_idx); +out_unlock: + hctx_unlock(hctx, srcu_idx); - ret = __blk_mq_try_issue_directly(hctx, rq, cookie, false); - if (ret == BLK_STS_RESOURCE || ret == BLK_STS_DEV_RESOURCE) - blk_mq_sched_insert_request(rq, false, true, false); - else if (ret != BLK_STS_OK) + dec = blk_mq_make_dicision(ret, bypass); + switch(dec) { + case MQ_ISSUE_INSERT_QUEUE: + blk_mq_sched_insert_request(rq, false, run_queue, false); + break; + case MQ_ISSUE_END_REQUEST: blk_mq_end_request(rq, ret); + break; + default: + return ret; + } - hctx_unlock(hctx, srcu_idx); + return BLK_STS_OK; } blk_status_t blk_mq_request_issue_directly(struct request *rq) { - blk_status_t ret; - int srcu_idx; - struct blk_mq_hw_ctx *hctx = rq->mq_hctx; - - hctx_lock(hctx, &srcu_idx); - ret = __blk_mq_try_issue_directly(hctx, rq, NULL, true); - hctx_unlock(hctx, srcu_idx); - - return ret; + return blk_mq_try_issue_directly(rq->mq_hctx, rq, NULL, true); } void blk_mq_try_issue_list_directly(struct blk_mq_hw_ctx *hctx, @@ -1957,13 +1975,13 @@ static blk_qc_t blk_mq_make_request(struct request_queue *q, struct bio *bio) if (same_queue_rq) { data.hctx = same_queue_rq->mq_hctx; blk_mq_try_issue_directly(data.hctx, same_queue_rq, - &cookie); + &cookie, false); } } else if ((q->nr_hw_queues > 1 && is_sync) || (!q->elevator && !data.hctx->dispatch_busy)) { blk_mq_put_ctx(data.ctx); blk_mq_bio_to_request(rq, bio); - blk_mq_try_issue_directly(data.hctx, rq, &cookie); + blk_mq_try_issue_directly(data.hctx, rq, &cookie, false); } else { blk_mq_put_ctx(data.ctx); blk_mq_bio_to_request(rq, bio);