From patchwork Fri Mar 15 08:57:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "jianchao.wang" X-Patchwork-Id: 10854325 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 27C451669 for ; Fri, 15 Mar 2019 09:07:05 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1162E28AD7 for ; Fri, 15 Mar 2019 09:07:05 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 056042A8DE; Fri, 15 Mar 2019 09:07:05 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8E1FF28AD7 for ; Fri, 15 Mar 2019 09:07:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726468AbfCOJHC (ORCPT ); Fri, 15 Mar 2019 05:07:02 -0400 Received: from aserp2130.oracle.com ([141.146.126.79]:44886 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728640AbfCOJGx (ORCPT ); Fri, 15 Mar 2019 05:06:53 -0400 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2F8xIDk057534; Fri, 15 Mar 2019 09:06:16 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=OC/PdQhEPXLumSdNoF/bK0ObwxgKZgaL5SrQOG5BdtQ=; b=VQ0ttMj3Q7gxdNh6WbdeF8gSVgCqxu3LVd0I/9nmYWAQe3vhHatIxpvaD9DlR8X+4EnZ 7vADoSf7Mj7ap5cYxRjRjAcw+JPnECYGvdKDOl9/VpGy6jWpPqm2PjAmUnJeXxTXnXKE tnINDT5W6J5EM0Zn0HNjxaXxsJsw8R5F8pFOZb92izlwyKDdq8Dnxn/oO95IlZQKzKkD XiLtC/YuIAxe/FpjZK100Ytej5FMArnFsmU4bcuWttxycvP/ofhQJl56wEj4kHoelSQ5 fxh25m6AUmKmJSEPOjaSCIhtMEk668z8udyd9m4DDxbhPyqqpyqDfBQnXF4FxdbheuxI Ow== Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by aserp2130.oracle.com with ESMTP id 2r430f61hs-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 15 Mar 2019 09:06:15 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x2F96D9K022654 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 15 Mar 2019 09:06:13 GMT Received: from abhmp0015.oracle.com (abhmp0015.oracle.com [141.146.116.21]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x2F96CLJ021411; Fri, 15 Mar 2019 09:06:12 GMT Received: from will-ThinkCentre-M93p.cn.oracle.com (/10.182.71.12) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 15 Mar 2019 09:06:11 +0000 From: Jianchao Wang To: axboe@kernel.dk Cc: hch@lst.de, jthumshirn@suse.de, hare@suse.de, josef@toxicpanda.com, bvanassche@acm.org, sagi@grimberg.me, keith.busch@intel.com, jsmart2021@gmail.com, linux-block@vger.kernel.org, linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH 1/8] blk-mq: get rid of the synchronize_rcu in __blk_mq_update_nr_hw_queues Date: Fri, 15 Mar 2019 16:57:37 +0800 Message-Id: <1552640264-26101-2-git-send-email-jianchao.w.wang@oracle.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1552640264-26101-1-git-send-email-jianchao.w.wang@oracle.com> References: <1552640264-26101-1-git-send-email-jianchao.w.wang@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9195 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903150067 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP In commit 530ca2c (blk-mq: Allow blocking queue tag iter callbacks), we try to get a non-zero q_usage_counter to avoid access hctxs that being modified. So the synchronize_rcu is useless and should be removed. Signed-off-by: Jianchao Wang Reviewed-by: Ming Lei --- block/blk-mq-tag.c | 4 +--- block/blk-mq.c | 4 ---- 2 files changed, 1 insertion(+), 7 deletions(-) diff --git a/block/blk-mq-tag.c b/block/blk-mq-tag.c index 2089c6c..b792537 100644 --- a/block/blk-mq-tag.c +++ b/block/blk-mq-tag.c @@ -384,9 +384,7 @@ void blk_mq_queue_tag_busy_iter(struct request_queue *q, busy_iter_fn *fn, /* * __blk_mq_update_nr_hw_queues() updates nr_hw_queues and queue_hw_ctx * while the queue is frozen. So we can use q_usage_counter to avoid - * racing with it. __blk_mq_update_nr_hw_queues() uses - * synchronize_rcu() to ensure this function left the critical section - * below. + * racing with it. */ if (!percpu_ref_tryget(&q->q_usage_counter)) return; diff --git a/block/blk-mq.c b/block/blk-mq.c index 9437a5e..07584a9 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -3208,10 +3208,6 @@ static void __blk_mq_update_nr_hw_queues(struct blk_mq_tag_set *set, list_for_each_entry(q, &set->tag_list, tag_set_list) blk_mq_freeze_queue(q); /* - * Sync with blk_mq_queue_tag_busy_iter. - */ - synchronize_rcu(); - /* * Switch IO scheduler to 'none', cleaning up the data associated * with the previous scheduler. We will switch back once we are done * updating the new sw to hw queue mappings. From patchwork Fri Mar 15 08:57:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "jianchao.wang" X-Patchwork-Id: 10854319 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1D0B21669 for ; Fri, 15 Mar 2019 09:06:42 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0595B28AD7 for ; Fri, 15 Mar 2019 09:06:42 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id ED5852A8D9; Fri, 15 Mar 2019 09:06:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BC5EF28AD7 for ; Fri, 15 Mar 2019 09:06:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727501AbfCOJGi (ORCPT ); Fri, 15 Mar 2019 05:06:38 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:40492 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727470AbfCOJGi (ORCPT ); Fri, 15 Mar 2019 05:06:38 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2F8xH1R027925; Fri, 15 Mar 2019 09:06:18 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=Jp9oYfN+g9q7N3MGFia2Plxo4iGg52/WEXTjkCnOtjo=; b=cdVHaVivR5c03QUaKgrbQoGUcG4p9h7KUPCkFYHkcor6kFlo5a6DF0KpBAdmYey6P6cC rM22XqUmUMse0qCf+g7W8Y8v7Mucy9X5w2muQWHG4IvRy7AcXZ6ovmstbKdpEuysy6dF 5QD5uofljtBieaevDRQD/XTtfbOp7M6qIt4rjmT1PHPBDjigcjBJxn6c9gFlrP+armLb 10mqSWsz3QN7v0K+Mht003nwBjoT8tu9a/+RAuABBvwV1gwFhpRI9yj+IMY+aLxp0aJv MatUY3JlC1cnzVsKv7AwvPI/gbcN59pj0JVlsm1YizO0PrE06I2hvNNBb2lsw5YMj8S9 Gg== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2120.oracle.com with ESMTP id 2r464rwhsq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 15 Mar 2019 09:06:18 +0000 Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x2F96H5m005618 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 15 Mar 2019 09:06:17 GMT Received: from abhmp0015.oracle.com (abhmp0015.oracle.com [141.146.116.21]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x2F96FrN005261; Fri, 15 Mar 2019 09:06:15 GMT Received: from will-ThinkCentre-M93p.cn.oracle.com (/10.182.71.12) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 15 Mar 2019 09:06:14 +0000 From: Jianchao Wang To: axboe@kernel.dk Cc: hch@lst.de, jthumshirn@suse.de, hare@suse.de, josef@toxicpanda.com, bvanassche@acm.org, sagi@grimberg.me, keith.busch@intel.com, jsmart2021@gmail.com, linux-block@vger.kernel.org, linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH 2/8] blk-mq: change the method of iterating busy tags of a request_queue Date: Fri, 15 Mar 2019 16:57:38 +0800 Message-Id: <1552640264-26101-3-git-send-email-jianchao.w.wang@oracle.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1552640264-26101-1-git-send-email-jianchao.w.wang@oracle.com> References: <1552640264-26101-1-git-send-email-jianchao.w.wang@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9195 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=3 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903150067 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP tags->rqs[] will not been cleaned when free driver tag and there is a window between get driver tag and write tags->rqs[], so we may see stale rq in tags->rqs[] which may have been freed, as following case, blk_mq_get_request blk_mq_queue_tag_busy_iter -> blk_mq_get_tag -> bt_for_each -> bt_iter -> rq = taags->rqs[] -> rq->q -> blk_mq_rq_ctx_init -> data->hctx->tags->rqs[rq->tag] = rq; To fix this, the blk_mq_queue_tag_busy_iter is changed in this patch to use tags->static_rqs[] instead of tags->rqs[]. We have to identify whether there is a io scheduler attached to decide to use hctx->tags or hctx->sched_tags. And we will try to get a non-zero q_usage_counter before that, so it is safe to access them. Add 'inflight' parameter to determine to iterate in-flight requests or just busy tags. A correction here is that part_in_flight should count the busy tags instead of rqs that have got driver tags. Moreover, get rid of the 'hctx' parameter in iter function as we could get it from rq->mq_hctx and export this interface for drivers. Signed-off-by: Jianchao Wang --- block/blk-mq-tag.c | 76 +++++++++++++++++++++++++++++++++----------------- block/blk-mq-tag.h | 2 -- block/blk-mq.c | 31 ++++++++------------ include/linux/blk-mq.h | 5 ++-- 4 files changed, 64 insertions(+), 50 deletions(-) diff --git a/block/blk-mq-tag.c b/block/blk-mq-tag.c index b792537..cdec2cd 100644 --- a/block/blk-mq-tag.c +++ b/block/blk-mq-tag.c @@ -216,26 +216,38 @@ struct bt_iter_data { busy_iter_fn *fn; void *data; bool reserved; + bool inflight; }; static bool bt_iter(struct sbitmap *bitmap, unsigned int bitnr, void *data) { struct bt_iter_data *iter_data = data; struct blk_mq_hw_ctx *hctx = iter_data->hctx; - struct blk_mq_tags *tags = hctx->tags; bool reserved = iter_data->reserved; + struct blk_mq_tags *tags; struct request *rq; + tags = hctx->sched_tags ? hctx->sched_tags : hctx->tags; + if (!reserved) bitnr += tags->nr_reserved_tags; - rq = tags->rqs[bitnr]; + /* + * Because tags->rqs[] will not been cleaned when free driver tag + * and there is a window between get driver tag and write tags->rqs[], + * so we may see stale rq in tags->rqs[] which may have been freed. + * Using static_rqs[] is safer. + */ + rq = tags->static_rqs[bitnr]; /* - * We can hit rq == NULL here, because the tagging functions - * test and set the bit before assigning ->rqs[]. + * There is a small window between get tag and blk_mq_rq_ctx_init, + * so rq->q and rq->mq_hctx maybe different. */ - if (rq && rq->q == hctx->queue) - return iter_data->fn(hctx, rq, iter_data->data, reserved); + if (rq && rq->q == hctx->queue && + rq->mq_hctx == hctx && + (!iter_data->inflight || + blk_mq_rq_state(rq) == MQ_RQ_IN_FLIGHT)) + return iter_data->fn(rq, iter_data->data, reserved); return true; } @@ -246,7 +258,7 @@ static bool bt_iter(struct sbitmap *bitmap, unsigned int bitnr, void *data) * or the bitmap_tags member of struct blk_mq_tags. * @fn: Pointer to the function that will be called for each request * associated with @hctx that has been assigned a driver tag. - * @fn will be called as follows: @fn(@hctx, rq, @data, @reserved) + * @fn will be called as follows: @fn(rq, @data, @reserved) * where rq is a pointer to a request. Return true to continue * iterating tags, false to stop. * @data: Will be passed as third argument to @fn. @@ -254,13 +266,14 @@ static bool bt_iter(struct sbitmap *bitmap, unsigned int bitnr, void *data) * bitmap_tags member of struct blk_mq_tags. */ static void bt_for_each(struct blk_mq_hw_ctx *hctx, struct sbitmap_queue *bt, - busy_iter_fn *fn, void *data, bool reserved) + busy_iter_fn *fn, void *data, bool reserved, bool inflight) { struct bt_iter_data iter_data = { .hctx = hctx, .fn = fn, .data = data, .reserved = reserved, + .inflight = inflight, }; sbitmap_for_each_set(&bt->sb, bt_iter, &iter_data); @@ -362,36 +375,33 @@ void blk_mq_tagset_busy_iter(struct blk_mq_tag_set *tagset, EXPORT_SYMBOL(blk_mq_tagset_busy_iter); /** - * blk_mq_queue_tag_busy_iter - iterate over all requests with a driver tag + * blk_mq_queue_tag_busy_iter - iterate over all busy tags * @q: Request queue to examine. - * @fn: Pointer to the function that will be called for each request - * on @q. @fn will be called as follows: @fn(hctx, rq, @priv, - * reserved) where rq is a pointer to a request and hctx points - * to the hardware queue associated with the request. 'reserved' - * indicates whether or not @rq is a reserved request. + * @fn: Pointer to the function that will be called for each + * in-flight request issued by @q. @fn will be called as + * follows: + * @fn(rq, @priv, reserved) + * rq is a pointer to a request.'reserved' indicates whether or + * not @rq is a reserved request. * @priv: Will be passed as third argument to @fn. - * - * Note: if @q->tag_set is shared with other request queues then @fn will be - * called for all requests on all queues that share that tag set and not only - * for requests associated with @q. */ void blk_mq_queue_tag_busy_iter(struct request_queue *q, busy_iter_fn *fn, - void *priv) + void *priv, bool inflight) { struct blk_mq_hw_ctx *hctx; int i; /* - * __blk_mq_update_nr_hw_queues() updates nr_hw_queues and queue_hw_ctx - * while the queue is frozen. So we can use q_usage_counter to avoid - * racing with it. + * Get a reference of the queue unless it has been zero. We use this + * to avoid the race with the code that would modify the hctxs after + * freeze and drain the queue, including updating nr_hw_queues, io + * scheduler switching and queue clean up. */ if (!percpu_ref_tryget(&q->q_usage_counter)) return; queue_for_each_hw_ctx(q, hctx, i) { - struct blk_mq_tags *tags = hctx->tags; - + struct blk_mq_tags *tags; /* * If no software queues are currently mapped to this * hardware queue, there's nothing to check @@ -399,12 +409,26 @@ void blk_mq_queue_tag_busy_iter(struct request_queue *q, busy_iter_fn *fn, if (!blk_mq_hw_queue_mapped(hctx)) continue; + tags = hctx->sched_tags ? hctx->sched_tags : hctx->tags; + if (tags->nr_reserved_tags) - bt_for_each(hctx, &tags->breserved_tags, fn, priv, true); - bt_for_each(hctx, &tags->bitmap_tags, fn, priv, false); + bt_for_each(hctx, &tags->breserved_tags, + fn, priv, true, inflight); + bt_for_each(hctx, &tags->bitmap_tags, + fn, priv, false, inflight); + /* + * flush_rq represents the rq with REQ_PREFLUSH and REQ_FUA + * (if FUA is not supported by device) to be issued to + * device. So we need to consider it when iterate inflight + * rqs, but needn't to count it when iterate busy tags. + */ + if (inflight && + blk_mq_rq_state(hctx->fq->flush_rq) == MQ_RQ_IN_FLIGHT) + fn(hctx->fq->flush_rq, priv, false); } blk_queue_exit(q); } +EXPORT_SYMBOL(blk_mq_queue_tag_busy_iter); static int bt_alloc(struct sbitmap_queue *bt, unsigned int depth, bool round_robin, int node) diff --git a/block/blk-mq-tag.h b/block/blk-mq-tag.h index 61deab0..5af7ff9 100644 --- a/block/blk-mq-tag.h +++ b/block/blk-mq-tag.h @@ -33,8 +33,6 @@ extern int blk_mq_tag_update_depth(struct blk_mq_hw_ctx *hctx, struct blk_mq_tags **tags, unsigned int depth, bool can_grow); extern void blk_mq_tag_wakeup_all(struct blk_mq_tags *tags, bool); -void blk_mq_queue_tag_busy_iter(struct request_queue *q, busy_iter_fn *fn, - void *priv); static inline struct sbq_wait_state *bt_wait_ptr(struct sbitmap_queue *bt, struct blk_mq_hw_ctx *hctx) diff --git a/block/blk-mq.c b/block/blk-mq.c index 07584a9..9144ec1 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -93,8 +93,7 @@ struct mq_inflight { unsigned int *inflight; }; -static bool blk_mq_check_inflight(struct blk_mq_hw_ctx *hctx, - struct request *rq, void *priv, +static bool blk_mq_check_inflight(struct request *rq, void *priv, bool reserved) { struct mq_inflight *mi = priv; @@ -114,13 +113,12 @@ unsigned int blk_mq_in_flight(struct request_queue *q, struct hd_struct *part) struct mq_inflight mi = { .part = part, .inflight = inflight, }; inflight[0] = inflight[1] = 0; - blk_mq_queue_tag_busy_iter(q, blk_mq_check_inflight, &mi); + blk_mq_queue_tag_busy_iter(q, blk_mq_check_inflight, &mi, false); return inflight[0]; } -static bool blk_mq_check_inflight_rw(struct blk_mq_hw_ctx *hctx, - struct request *rq, void *priv, +static bool blk_mq_check_inflight_rw(struct request *rq, void *priv, bool reserved) { struct mq_inflight *mi = priv; @@ -137,7 +135,7 @@ void blk_mq_in_flight_rw(struct request_queue *q, struct hd_struct *part, struct mq_inflight mi = { .part = part, .inflight = inflight, }; inflight[0] = inflight[1] = 0; - blk_mq_queue_tag_busy_iter(q, blk_mq_check_inflight_rw, &mi); + blk_mq_queue_tag_busy_iter(q, blk_mq_check_inflight_rw, &mi, false); } void blk_freeze_queue_start(struct request_queue *q) @@ -813,28 +811,22 @@ struct request *blk_mq_tag_to_rq(struct blk_mq_tags *tags, unsigned int tag) } EXPORT_SYMBOL(blk_mq_tag_to_rq); -static bool blk_mq_rq_inflight(struct blk_mq_hw_ctx *hctx, struct request *rq, - void *priv, bool reserved) +static bool blk_mq_rq_inflight(struct request *rq, void *priv, bool reserved) { + bool *busy = priv; /* * If we find a request that is inflight and the queue matches, * we know the queue is busy. Return false to stop the iteration. */ - if (rq->state == MQ_RQ_IN_FLIGHT && rq->q == hctx->queue) { - bool *busy = priv; - - *busy = true; - return false; - } - - return true; + *busy = true; + return false; } bool blk_mq_queue_inflight(struct request_queue *q) { bool busy = false; - blk_mq_queue_tag_busy_iter(q, blk_mq_rq_inflight, &busy); + blk_mq_queue_tag_busy_iter(q, blk_mq_rq_inflight, &busy, true); return busy; } EXPORT_SYMBOL_GPL(blk_mq_queue_inflight); @@ -874,8 +866,7 @@ static bool blk_mq_req_expired(struct request *rq, unsigned long *next) return false; } -static bool blk_mq_check_expired(struct blk_mq_hw_ctx *hctx, - struct request *rq, void *priv, bool reserved) +static bool blk_mq_check_expired(struct request *rq, void *priv, bool reserved) { unsigned long *next = priv; @@ -936,7 +927,7 @@ static void blk_mq_timeout_work(struct work_struct *work) if (!percpu_ref_tryget(&q->q_usage_counter)) return; - blk_mq_queue_tag_busy_iter(q, blk_mq_check_expired, &next); + blk_mq_queue_tag_busy_iter(q, blk_mq_check_expired, &next, true); if (next != 0) { mod_timer(&q->timeout, next); diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h index 0e030f5..d6beeb5 100644 --- a/include/linux/blk-mq.h +++ b/include/linux/blk-mq.h @@ -132,8 +132,7 @@ typedef int (init_request_fn)(struct blk_mq_tag_set *set, struct request *, typedef void (exit_request_fn)(struct blk_mq_tag_set *set, struct request *, unsigned int); -typedef bool (busy_iter_fn)(struct blk_mq_hw_ctx *, struct request *, void *, - bool); +typedef bool (busy_iter_fn)(struct request *, void *, bool); typedef bool (busy_tag_iter_fn)(struct request *, void *, bool); typedef int (poll_fn)(struct blk_mq_hw_ctx *); typedef int (map_queues_fn)(struct blk_mq_tag_set *set); @@ -322,6 +321,8 @@ bool blk_mq_run_hw_queue(struct blk_mq_hw_ctx *hctx, bool async); void blk_mq_run_hw_queues(struct request_queue *q, bool async); void blk_mq_tagset_busy_iter(struct blk_mq_tag_set *tagset, busy_tag_iter_fn *fn, void *priv); +void blk_mq_queue_tag_busy_iter(struct request_queue *q, busy_iter_fn *fn, + void *priv, bool inflight); void blk_mq_freeze_queue(struct request_queue *q); void blk_mq_unfreeze_queue(struct request_queue *q); void blk_freeze_queue_start(struct request_queue *q); From patchwork Fri Mar 15 08:57:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "jianchao.wang" X-Patchwork-Id: 10854327 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7B36314DE for ; Fri, 15 Mar 2019 09:07:30 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 61A5328AD7 for ; Fri, 15 Mar 2019 09:07:30 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 535752A8DC; Fri, 15 Mar 2019 09:07:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F0F4C28AD7 for ; Fri, 15 Mar 2019 09:07:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728487AbfCOJHO (ORCPT ); Fri, 15 Mar 2019 05:07:14 -0400 Received: from aserp2130.oracle.com ([141.146.126.79]:44864 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728659AbfCOJGv (ORCPT ); Fri, 15 Mar 2019 05:06:51 -0400 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2F8xIY1057463; Fri, 15 Mar 2019 09:06:19 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=fbPDd3Ab9z/qNbAoNclCyGS1HDGYOqc/WGMQxm0WqUo=; b=DWsW8a2mddlAgjY2fyj3TUc36Qu7ijpHf+mKUTjhARmB2THE+yf8oI+EtwDoKByd1PVU DZRX4+krJpazXOwER9ZrQULT9KOGlRTjxPWrUeVHIh0lQpOuPKnO8Aocg1GELtI4oWdT 5ZkHB4DD8LniFgggCbN19wb+fIiw6dASO9+L09ZT3sUF2MWoplYwVsxOCf3WVwga0/Th 3qn8QUiUHTOYkbjChGFmTImHkWCVDhC2l3hMz054HyQnyInW0NF6BKwKffXabUuJcNco V7H550l/M32a8mfmbXZbAnxObQ1V16mE0ZOnYZCLXGdNdxMPj7IKsNrH7Hik4jO4ETV/ 6w== Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by aserp2130.oracle.com with ESMTP id 2r430f61j9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 15 Mar 2019 09:06:19 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by aserv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x2F96ISZ007932 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 15 Mar 2019 09:06:19 GMT Received: from abhmp0015.oracle.com (abhmp0015.oracle.com [141.146.116.21]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x2F96IBg017585; Fri, 15 Mar 2019 09:06:18 GMT Received: from will-ThinkCentre-M93p.cn.oracle.com (/10.182.71.12) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 15 Mar 2019 09:06:18 +0000 From: Jianchao Wang To: axboe@kernel.dk Cc: hch@lst.de, jthumshirn@suse.de, hare@suse.de, josef@toxicpanda.com, bvanassche@acm.org, sagi@grimberg.me, keith.busch@intel.com, jsmart2021@gmail.com, linux-block@vger.kernel.org, linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH 3/8] blk-mq: use blk_mq_queue_tag_busy_iter in debugfs Date: Fri, 15 Mar 2019 16:57:39 +0800 Message-Id: <1552640264-26101-4-git-send-email-jianchao.w.wang@oracle.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1552640264-26101-1-git-send-email-jianchao.w.wang@oracle.com> References: <1552640264-26101-1-git-send-email-jianchao.w.wang@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9195 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=964 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903150067 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP blk_mq_tagset_busy_iter is not safe that it could get stale request in tags->rqs[]. Use blk_mq_queue_tag_busy_iter here. Signed-off-by: Jianchao Wang --- block/blk-mq-debugfs.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/block/blk-mq-debugfs.c b/block/blk-mq-debugfs.c index 7921573..7c4e771 100644 --- a/block/blk-mq-debugfs.c +++ b/block/blk-mq-debugfs.c @@ -442,8 +442,8 @@ static int hctx_busy_show(void *data, struct seq_file *m) struct blk_mq_hw_ctx *hctx = data; struct show_busy_params params = { .m = m, .hctx = hctx }; - blk_mq_tagset_busy_iter(hctx->queue->tag_set, hctx_show_busy_rq, - ¶ms); + blk_mq_queue_tag_busy_iter(hctx->queue, hctx_show_busy_rq, + ¶ms, true); return 0; } From patchwork Fri Mar 15 08:57:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "jianchao.wang" X-Patchwork-Id: 10854321 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D3C4514DE for ; Fri, 15 Mar 2019 09:06:54 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BD78828AD7 for ; Fri, 15 Mar 2019 09:06:54 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B104B2A8D9; Fri, 15 Mar 2019 09:06:54 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5818828AD7 for ; Fri, 15 Mar 2019 09:06:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728666AbfCOJGx (ORCPT ); Fri, 15 Mar 2019 05:06:53 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:50542 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728643AbfCOJGt (ORCPT ); Fri, 15 Mar 2019 05:06:49 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2F8xInG020924; Fri, 15 Mar 2019 09:06:22 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=hBt3meiso9L9P5SHWZuowZExw8d7d0HPQjxMIiWYeD0=; b=hgGRRlmn3dy45uwU4Xl18NFuh9RXTQDCH3rvtyRZq5H/uPoruw9z293cUHabP/ul5ei+ fWqNKrdy3UMZrjrWnQALGr2hnMUQk8jeoKz7PjDK7EAD4FQKulAD12rOq1xM0+xyjkSj wbUKSfkcXu4trb6RmxV/Cn1/uQpyou2NZt70BEdtKccwJuW08pJB05PzOU9S9ky+O3U3 KyCtLz8Jiz2JFZP0KTwcKK3zRXcAFTYEWKjtaJjRqVWulHLjuEB1rJ+T8hpLxjpQt47N BQeqGIJPHAwGiZgPtCfheh5+83+DPzZ0ZUDfNy/x6NAZIWB5W7jIszcsbFpAdvUIIdhf tQ== Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by userp2130.oracle.com with ESMTP id 2r44wunjt4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 15 Mar 2019 09:06:22 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x2F96LG4029141 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 15 Mar 2019 09:06:21 GMT Received: from abhmp0015.oracle.com (abhmp0015.oracle.com [141.146.116.21]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x2F96LVa017601; Fri, 15 Mar 2019 09:06:21 GMT Received: from will-ThinkCentre-M93p.cn.oracle.com (/10.182.71.12) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 15 Mar 2019 09:06:21 +0000 From: Jianchao Wang To: axboe@kernel.dk Cc: hch@lst.de, jthumshirn@suse.de, hare@suse.de, josef@toxicpanda.com, bvanassche@acm.org, sagi@grimberg.me, keith.busch@intel.com, jsmart2021@gmail.com, linux-block@vger.kernel.org, linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH 4/8] mtip32xx: use blk_mq_queue_tag_busy_iter Date: Fri, 15 Mar 2019 16:57:40 +0800 Message-Id: <1552640264-26101-5-git-send-email-jianchao.w.wang@oracle.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1552640264-26101-1-git-send-email-jianchao.w.wang@oracle.com> References: <1552640264-26101-1-git-send-email-jianchao.w.wang@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9195 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=800 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903150067 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP blk_mq_tagset_busy_iter is not safe that it could get stale request in tags->rqs[]. Use blk_mq_queue_tag_busy_iter here. Signed-off-by: Jianchao Wang --- drivers/block/mtip32xx/mtip32xx.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/drivers/block/mtip32xx/mtip32xx.c b/drivers/block/mtip32xx/mtip32xx.c index 88e8440..6e356f7 100644 --- a/drivers/block/mtip32xx/mtip32xx.c +++ b/drivers/block/mtip32xx/mtip32xx.c @@ -2770,13 +2770,13 @@ static int mtip_service_thread(void *data) blk_mq_quiesce_queue(dd->queue); - blk_mq_tagset_busy_iter(&dd->tags, mtip_queue_cmd, dd); + blk_mq_queue_tag_busy_iter(dd->queue, mtip_queue_cmd, dd, true); set_bit(MTIP_PF_ISSUE_CMDS_BIT, &dd->port->flags); if (mtip_device_reset(dd)) - blk_mq_tagset_busy_iter(&dd->tags, - mtip_abort_cmd, dd); + blk_mq_queue_tag_busy_iter(dd->queue, + mtip_abort_cmd, dd, true); clear_bit(MTIP_PF_TO_ACTIVE_BIT, &dd->port->flags); @@ -3907,7 +3907,7 @@ static int mtip_block_remove(struct driver_data *dd) blk_freeze_queue_start(dd->queue); blk_mq_quiesce_queue(dd->queue); - blk_mq_tagset_busy_iter(&dd->tags, mtip_no_dev_cleanup, dd); + blk_mq_queue_tag_busy_iter(dd->queue, mtip_no_dev_cleanup, dd, true); blk_mq_unquiesce_queue(dd->queue); /* From patchwork Fri Mar 15 08:57:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "jianchao.wang" X-Patchwork-Id: 10854333 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 876591669 for ; Fri, 15 Mar 2019 09:07:41 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7094B28AD7 for ; Fri, 15 Mar 2019 09:07:41 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6498D2A8D9; Fri, 15 Mar 2019 09:07:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1BB6428AD7 for ; Fri, 15 Mar 2019 09:07:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728621AbfCOJGo (ORCPT ); Fri, 15 Mar 2019 05:06:44 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:40616 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727470AbfCOJGn (ORCPT ); Fri, 15 Mar 2019 05:06:43 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2F8xI5J027981; Fri, 15 Mar 2019 09:06:30 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=3fE6qdcH4Ccpw7H6cCg51Qtvf/v1uSWjMXLedQ7vr3Q=; b=Kzesn5fHkKDXfgXY2nsAsO/mT2PR7yZ3KBhaIASyr3oDt81pjiCLWn767XJPY2641ecm XGv2Hrh6rmodqfBqDU6AOU8WJMLhDPz2c8FmLLg92CRt6uR6g+oX4808KzXvA/9JWGg7 Cq+FKKiZfH40pjKEDk3zOdjh0hRaCYL7f8W8ZsesIJkxV2s1AaF2I5T2BYQhIln9KncU MgRZ9jYBgxu0/9XGlw8LQ9lZ+CBnGowApHCCqwWGrB0pyTcpefgamU5LrAXmvJv63J9S AMCThMbivyreEy8NHrIn+zuwsM2/ky6m4EAnB6xFzb5ptKpYWYhdA0NRRDNCsLzSujdu ng== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2120.oracle.com with ESMTP id 2r464rwhtr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 15 Mar 2019 09:06:30 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x2F96OEc005837 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 15 Mar 2019 09:06:24 GMT Received: from abhmp0015.oracle.com (abhmp0015.oracle.com [141.146.116.21]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x2F96OHK017621; Fri, 15 Mar 2019 09:06:24 GMT Received: from will-ThinkCentre-M93p.cn.oracle.com (/10.182.71.12) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 15 Mar 2019 09:06:24 +0000 From: Jianchao Wang To: axboe@kernel.dk Cc: hch@lst.de, jthumshirn@suse.de, hare@suse.de, josef@toxicpanda.com, bvanassche@acm.org, sagi@grimberg.me, keith.busch@intel.com, jsmart2021@gmail.com, linux-block@vger.kernel.org, linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH 5/8] nbd: use blk_mq_queue_tag_busy_iter Date: Fri, 15 Mar 2019 16:57:41 +0800 Message-Id: <1552640264-26101-6-git-send-email-jianchao.w.wang@oracle.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1552640264-26101-1-git-send-email-jianchao.w.wang@oracle.com> References: <1552640264-26101-1-git-send-email-jianchao.w.wang@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9195 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=857 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903150067 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP blk_mq_tagset_busy_iter is not safe that it could get stale request in tags->rqs[]. Use blk_mq_queue_tag_busy_iter here. Signed-off-by: Jianchao Wang --- drivers/block/nbd.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c index 7c9a949..9e7e828 100644 --- a/drivers/block/nbd.c +++ b/drivers/block/nbd.c @@ -747,7 +747,7 @@ static bool nbd_clear_req(struct request *req, void *data, bool reserved) static void nbd_clear_que(struct nbd_device *nbd) { blk_mq_quiesce_queue(nbd->disk->queue); - blk_mq_tagset_busy_iter(&nbd->tag_set, nbd_clear_req, NULL); + blk_mq_queue_tag_busy_iter(nbd->disk->queue, nbd_clear_req, NULL, true); blk_mq_unquiesce_queue(nbd->disk->queue); dev_dbg(disk_to_dev(nbd->disk), "queue cleared\n"); } From patchwork Fri Mar 15 08:57:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "jianchao.wang" X-Patchwork-Id: 10854331 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8D0BA14DE for ; Fri, 15 Mar 2019 09:07:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7736B2A8D8 for ; Fri, 15 Mar 2019 09:07:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6B5BA2A8DC; Fri, 15 Mar 2019 09:07:37 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 20A0F2A8D8 for ; Fri, 15 Mar 2019 09:07:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728045AbfCOJHa (ORCPT ); Fri, 15 Mar 2019 05:07:30 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:50544 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728644AbfCOJGt (ORCPT ); Fri, 15 Mar 2019 05:06:49 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2F8xImn020939; Fri, 15 Mar 2019 09:06:28 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=hodO79sVdUVbs8R2wgveaoHWI6y3it+JXm1fmyjQDVk=; b=xqP793vYQW7JOCT2ovXf0NeKSjjV6Sk1vbWBMc5KCIqFzaaWcWjxPhN64zTOSSVBd2V+ KypriZ9GXCMVv1PmuqpDuA+wBvsUbajU/FDam8RBe8pUXDW6jCQL7MayLe/La3L4T219 /48pkhDmxnU7ch2QlJDXhd4H73t1SpTVWwehhNTuLzeraQPltvG+eBfPnb6hYfunlmkI tvQKJXre8anR0Bn8g35omJdWXe7PNLs8nqU7LvtVn4DLJ8tOeuvDtHDrRGQ8xrwT8EBD EMvifZ/1sIAz2RvmC1E+KgJHqCwLaNzG7QPEP+7ylKKjx85axOlrFvNjoM1m4RlQXrZU dA== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2130.oracle.com with ESMTP id 2r44wunjte-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 15 Mar 2019 09:06:28 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x2F96RjV006011 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 15 Mar 2019 09:06:27 GMT Received: from abhmp0015.oracle.com (abhmp0015.oracle.com [141.146.116.21]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x2F96Rit017635; Fri, 15 Mar 2019 09:06:27 GMT Received: from will-ThinkCentre-M93p.cn.oracle.com (/10.182.71.12) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 15 Mar 2019 09:06:27 +0000 From: Jianchao Wang To: axboe@kernel.dk Cc: hch@lst.de, jthumshirn@suse.de, hare@suse.de, josef@toxicpanda.com, bvanassche@acm.org, sagi@grimberg.me, keith.busch@intel.com, jsmart2021@gmail.com, linux-block@vger.kernel.org, linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH 6/8] skd: use blk_mq_queue_tag_busy_iter Date: Fri, 15 Mar 2019 16:57:42 +0800 Message-Id: <1552640264-26101-7-git-send-email-jianchao.w.wang@oracle.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1552640264-26101-1-git-send-email-jianchao.w.wang@oracle.com> References: <1552640264-26101-1-git-send-email-jianchao.w.wang@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9195 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903150067 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP blk_mq_tagset_busy_iter is not safe that it could get stale request in tags->rqs[]. Use blk_mq_queue_tag_busy_iter here. Signed-off-by: Jianchao Wang --- drivers/block/skd_main.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/block/skd_main.c b/drivers/block/skd_main.c index ab893a7..60c34ff 100644 --- a/drivers/block/skd_main.c +++ b/drivers/block/skd_main.c @@ -395,7 +395,7 @@ static int skd_in_flight(struct skd_device *skdev) { int count = 0; - blk_mq_tagset_busy_iter(&skdev->tag_set, skd_inc_in_flight, &count); + blk_mq_queue_tag_busy_iter(skdev->queue, skd_inc_in_flight, &count, true); return count; } @@ -1916,7 +1916,7 @@ static bool skd_recover_request(struct request *req, void *data, bool reserved) static void skd_recover_requests(struct skd_device *skdev) { - blk_mq_tagset_busy_iter(&skdev->tag_set, skd_recover_request, skdev); + blk_mq_queue_tag_busy_iter(skdev->queue, skd_recover_request, skdev, true); } static void skd_isr_msg_from_dev(struct skd_device *skdev) From patchwork Fri Mar 15 08:57:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "jianchao.wang" X-Patchwork-Id: 10854329 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0E4F61823 for ; Fri, 15 Mar 2019 09:07:31 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E9E6C28AD7 for ; Fri, 15 Mar 2019 09:07:30 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DE0C42A8D9; Fri, 15 Mar 2019 09:07:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 406512A8D8 for ; Fri, 15 Mar 2019 09:07:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728028AbfCOJHO (ORCPT ); Fri, 15 Mar 2019 05:07:14 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:40720 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728664AbfCOJGw (ORCPT ); Fri, 15 Mar 2019 05:06:52 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2F8xHfW027865; Fri, 15 Mar 2019 09:06:32 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=wPnBoZe+aGWRd8JQ9uU6MKrXmOk7pVf3H6ud0bgcG0M=; b=rlZwi5KSXn8/FlnjWvKWbjjvh3qFNJ4df5UrQbheInNoI+iCZ311iIPdZHDLCMO1laxp HOHBhzgGgNZbFO1n2KCkTCy9kEUp0nagtRkU/LJ4+qgD1FX1tnM8TrdDO89Hm7ZdtXXR NuJgPPOg47ZNRZseFKzAwnIQ4/RnrFFCBD0SnRCGAypR2VKtg+hNqyT9h6FoDPQUFkZQ P1BcjGYH+WutXJDdp8U3cm7ahjyubAbK+aHWNEJp6XD49aKzd2DwLGLqyjXSVbUufHl0 xdV79F34dMa2QaGkkX9KqXTK5pNY5RnpUoXa7mp3bifBHhccYGY8INmwpRzqWD9U9ZCi GQ== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2120.oracle.com with ESMTP id 2r464rwhtu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 15 Mar 2019 09:06:31 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x2F96Uj5006122 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 15 Mar 2019 09:06:31 GMT Received: from abhmp0015.oracle.com (abhmp0015.oracle.com [141.146.116.21]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x2F96UBv017660; Fri, 15 Mar 2019 09:06:30 GMT Received: from will-ThinkCentre-M93p.cn.oracle.com (/10.182.71.12) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 15 Mar 2019 09:06:30 +0000 From: Jianchao Wang To: axboe@kernel.dk Cc: hch@lst.de, jthumshirn@suse.de, hare@suse.de, josef@toxicpanda.com, bvanassche@acm.org, sagi@grimberg.me, keith.busch@intel.com, jsmart2021@gmail.com, linux-block@vger.kernel.org, linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH 7/8] nvme: use blk_mq_queue_tag_busy_iter Date: Fri, 15 Mar 2019 16:57:43 +0800 Message-Id: <1552640264-26101-8-git-send-email-jianchao.w.wang@oracle.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1552640264-26101-1-git-send-email-jianchao.w.wang@oracle.com> References: <1552640264-26101-1-git-send-email-jianchao.w.wang@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9195 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903150067 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP blk_mq_tagset_busy_iter is not safe that it could get stale request in tags->rqs[]. Use blk_mq_queue_tag_busy_iter here. A new helper interface nvme_iterate_inflight_rqs is introduced to iterate all of the ns under a ctrl. Signed-off-by: Jianchao Wang --- drivers/nvme/host/core.c | 12 ++++++++++++ drivers/nvme/host/fc.c | 12 ++++++------ drivers/nvme/host/nvme.h | 2 ++ drivers/nvme/host/pci.c | 5 +++-- drivers/nvme/host/rdma.c | 6 +++--- drivers/nvme/host/tcp.c | 5 +++-- drivers/nvme/target/loop.c | 6 +++--- 7 files changed, 32 insertions(+), 16 deletions(-) diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index 6a9dd68..5760fad 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -3816,6 +3816,18 @@ void nvme_start_queues(struct nvme_ctrl *ctrl) } EXPORT_SYMBOL_GPL(nvme_start_queues); +void nvme_iterate_inflight_rqs(struct nvme_ctrl *ctrl, + busy_iter_fn *fn, void *data) +{ + struct nvme_ns *ns; + + down_read(&ctrl->namespaces_rwsem); + list_for_each_entry(ns, &ctrl->namespaces, list) + blk_mq_queue_tag_busy_iter(ns->queue, fn, data, true); + up_read(&ctrl->namespaces_rwsem); +} +EXPORT_SYMBOL_GPL(nvme_iterate_inflight_rqs); + int __init nvme_core_init(void) { int result = -ENOMEM; diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c index 89accc7..02b2de0 100644 --- a/drivers/nvme/host/fc.c +++ b/drivers/nvme/host/fc.c @@ -2368,7 +2368,7 @@ nvme_fc_complete_rq(struct request *rq) /* * This routine is used by the transport when it needs to find active * io on a queue that is to be terminated. The transport uses - * blk_mq_tagset_busy_itr() to find the busy requests, which then invoke + * blk_mq_queue_tag_busy_iter() to find the busy requests, which then invoke * this routine to kill them on a 1 by 1 basis. * * As FC allocates FC exchange for each io, the transport must contact @@ -2729,7 +2729,7 @@ nvme_fc_delete_association(struct nvme_fc_ctrl *ctrl) * If io queues are present, stop them and terminate all outstanding * ios on them. As FC allocates FC exchange for each io, the * transport must contact the LLDD to terminate the exchange, - * thus releasing the FC exchange. We use blk_mq_tagset_busy_itr() + * thus releasing the FC exchange. We use blk_mq_queue_tag_busy_iter * to tell us what io's are busy and invoke a transport routine * to kill them with the LLDD. After terminating the exchange * the LLDD will call the transport's normal io done path, but it @@ -2739,7 +2739,7 @@ nvme_fc_delete_association(struct nvme_fc_ctrl *ctrl) */ if (ctrl->ctrl.queue_count > 1) { nvme_stop_queues(&ctrl->ctrl); - blk_mq_tagset_busy_iter(&ctrl->tag_set, + nvme_iterate_inflight_rqs(&ctrl->ctrl, nvme_fc_terminate_exchange, &ctrl->ctrl); } @@ -2757,12 +2757,12 @@ nvme_fc_delete_association(struct nvme_fc_ctrl *ctrl) /* * clean up the admin queue. Same thing as above. - * use blk_mq_tagset_busy_itr() and the transport routine to + * use blk_mq_queue_tag_busy_iter() and the transport routine to * terminate the exchanges. */ blk_mq_quiesce_queue(ctrl->ctrl.admin_q); - blk_mq_tagset_busy_iter(&ctrl->admin_tag_set, - nvme_fc_terminate_exchange, &ctrl->ctrl); + blk_mq_queue_tag_busy_iter(ctrl->ctrl.admin_q, + nvme_fc_terminate_exchange, &ctrl->ctrl, true); /* kill the aens as they are a separate path */ nvme_fc_abort_aen_ops(ctrl); diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h index c4a1bb4..6dc4ff8 100644 --- a/drivers/nvme/host/nvme.h +++ b/drivers/nvme/host/nvme.h @@ -441,6 +441,8 @@ void nvme_unfreeze(struct nvme_ctrl *ctrl); void nvme_wait_freeze(struct nvme_ctrl *ctrl); void nvme_wait_freeze_timeout(struct nvme_ctrl *ctrl, long timeout); void nvme_start_freeze(struct nvme_ctrl *ctrl); +void nvme_iterate_inflight_rqs(struct nvme_ctrl *ctrl, + busy_iter_fn *fn, void *data); #define NVME_QID_ANY -1 struct request *nvme_alloc_request(struct request_queue *q, diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c index 7fee665..0f0e361 100644 --- a/drivers/nvme/host/pci.c +++ b/drivers/nvme/host/pci.c @@ -2476,8 +2476,9 @@ static void nvme_dev_disable(struct nvme_dev *dev, bool shutdown) nvme_suspend_queue(&dev->queues[0]); nvme_pci_disable(dev); - blk_mq_tagset_busy_iter(&dev->tagset, nvme_cancel_request, &dev->ctrl); - blk_mq_tagset_busy_iter(&dev->admin_tagset, nvme_cancel_request, &dev->ctrl); + nvme_iterate_inflight_rqs(&dev->ctrl, nvme_cancel_request, &dev->ctrl); + blk_mq_queue_tag_busy_iter(dev->ctrl.admin_q, + nvme_cancel_request, &dev->ctrl, true); /* * The driver will not be starting up queues again if shutting down so diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c index 52abc3a..1be5688 100644 --- a/drivers/nvme/host/rdma.c +++ b/drivers/nvme/host/rdma.c @@ -922,8 +922,8 @@ static void nvme_rdma_teardown_admin_queue(struct nvme_rdma_ctrl *ctrl, { blk_mq_quiesce_queue(ctrl->ctrl.admin_q); nvme_rdma_stop_queue(&ctrl->queues[0]); - blk_mq_tagset_busy_iter(&ctrl->admin_tag_set, nvme_cancel_request, - &ctrl->ctrl); + blk_mq_queue_tag_busy_iter(ctrl->ctrl.admin_q, nvme_cancel_request, + &ctrl->ctrl, true); blk_mq_unquiesce_queue(ctrl->ctrl.admin_q); nvme_rdma_destroy_admin_queue(ctrl, remove); } @@ -934,7 +934,7 @@ static void nvme_rdma_teardown_io_queues(struct nvme_rdma_ctrl *ctrl, if (ctrl->ctrl.queue_count > 1) { nvme_stop_queues(&ctrl->ctrl); nvme_rdma_stop_io_queues(ctrl); - blk_mq_tagset_busy_iter(&ctrl->tag_set, nvme_cancel_request, + nvme_iterate_inflight_rqs(&ctrl->ctrl, nvme_cancel_request, &ctrl->ctrl); if (remove) nvme_start_queues(&ctrl->ctrl); diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c index 5f0a004..75eb069 100644 --- a/drivers/nvme/host/tcp.c +++ b/drivers/nvme/host/tcp.c @@ -1686,7 +1686,8 @@ static void nvme_tcp_teardown_admin_queue(struct nvme_ctrl *ctrl, { blk_mq_quiesce_queue(ctrl->admin_q); nvme_tcp_stop_queue(ctrl, 0); - blk_mq_tagset_busy_iter(ctrl->admin_tagset, nvme_cancel_request, ctrl); + blk_mq_queue_tag_busy_iter(ctrl->admin_q, + nvme_cancel_request, ctrl, true); blk_mq_unquiesce_queue(ctrl->admin_q); nvme_tcp_destroy_admin_queue(ctrl, remove); } @@ -1698,7 +1699,7 @@ static void nvme_tcp_teardown_io_queues(struct nvme_ctrl *ctrl, return; nvme_stop_queues(ctrl); nvme_tcp_stop_io_queues(ctrl); - blk_mq_tagset_busy_iter(ctrl->tagset, nvme_cancel_request, ctrl); + nvme_iterate_inflight_rqs(ctrl, nvme_cancel_request, ctrl); if (remove) nvme_start_queues(ctrl); nvme_tcp_destroy_io_queues(ctrl, remove); diff --git a/drivers/nvme/target/loop.c b/drivers/nvme/target/loop.c index 4aac1b4..0d25083 100644 --- a/drivers/nvme/target/loop.c +++ b/drivers/nvme/target/loop.c @@ -429,7 +429,7 @@ static void nvme_loop_shutdown_ctrl(struct nvme_loop_ctrl *ctrl) { if (ctrl->ctrl.queue_count > 1) { nvme_stop_queues(&ctrl->ctrl); - blk_mq_tagset_busy_iter(&ctrl->tag_set, + nvme_iterate_inflight_rqs(&ctrl->ctrl, nvme_cancel_request, &ctrl->ctrl); nvme_loop_destroy_io_queues(ctrl); } @@ -438,8 +438,8 @@ static void nvme_loop_shutdown_ctrl(struct nvme_loop_ctrl *ctrl) nvme_shutdown_ctrl(&ctrl->ctrl); blk_mq_quiesce_queue(ctrl->ctrl.admin_q); - blk_mq_tagset_busy_iter(&ctrl->admin_tag_set, - nvme_cancel_request, &ctrl->ctrl); + blk_mq_queue_tag_busy_iter(ctrl->ctrl.admin_q, + nvme_cancel_request, &ctrl->ctrl, true); blk_mq_unquiesce_queue(ctrl->ctrl.admin_q); nvme_loop_destroy_admin_queue(ctrl); } From patchwork Fri Mar 15 08:57:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "jianchao.wang" X-Patchwork-Id: 10854323 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 979F114DE for ; Fri, 15 Mar 2019 09:07:02 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 80F522A8D8 for ; Fri, 15 Mar 2019 09:07:02 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 74ADA2A8DC; Fri, 15 Mar 2019 09:07:02 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E62E82A8D8 for ; Fri, 15 Mar 2019 09:07:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728643AbfCOJG4 (ORCPT ); Fri, 15 Mar 2019 05:06:56 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:50658 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726468AbfCOJGz (ORCPT ); Fri, 15 Mar 2019 05:06:55 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2F8xHXe020839; Fri, 15 Mar 2019 09:06:36 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=gFXOPmEFeouNmRWga/lmVGO0Wj+K/sZgIY+UwDko9R4=; b=wwtQO51fc+o8NST/Txo2SuZsHkhkbfJgF398I0nvDqt8nduGiO/n64vp7NFDEy5ptrLJ 2YEa5kooySXMuXo86MAR27QDPIxUksK5jEX6wwhY6TeBIhnCAUuNwkkhDTp9ywVxL3K0 faqOX6NCFWLSuiekdlDNWGw/hFKlE6A5FoDsBZIdbnYZbeekhnooOQWxKrtpqBX+52fi GxoQM1YY9QtzY6ObkE8XBVbHzRSMmfNQUyWvjt7jIhqCQClZTfWDP1Vfac35yXL+lO0A +gjClHAkMJXPMM4JdA3eVe6E5UZeJkl8NUtJCweyz2yo3jflcFmXeJcaQnbxtquSXZQl FQ== Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by userp2130.oracle.com with ESMTP id 2r44wunjts-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 15 Mar 2019 09:06:36 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by aserv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x2F96Yfq009241 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 15 Mar 2019 09:06:35 GMT Received: from abhmp0015.oracle.com (abhmp0015.oracle.com [141.146.116.21]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x2F96YTR021509; Fri, 15 Mar 2019 09:06:34 GMT Received: from will-ThinkCentre-M93p.cn.oracle.com (/10.182.71.12) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 15 Mar 2019 09:06:33 +0000 From: Jianchao Wang To: axboe@kernel.dk Cc: hch@lst.de, jthumshirn@suse.de, hare@suse.de, josef@toxicpanda.com, bvanassche@acm.org, sagi@grimberg.me, keith.busch@intel.com, jsmart2021@gmail.com, linux-block@vger.kernel.org, linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH 8/8] blk-mq: remove blk_mq_tagset_busy_iter Date: Fri, 15 Mar 2019 16:57:44 +0800 Message-Id: <1552640264-26101-9-git-send-email-jianchao.w.wang@oracle.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1552640264-26101-1-git-send-email-jianchao.w.wang@oracle.com> References: <1552640264-26101-1-git-send-email-jianchao.w.wang@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9195 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903150067 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP As nobody uses blk_mq_tagset_busy_iter, remove it. Signed-off-by: Jianchao Wang --- block/blk-mq-tag.c | 95 -------------------------------------------------- include/linux/blk-mq.h | 2 -- 2 files changed, 97 deletions(-) diff --git a/block/blk-mq-tag.c b/block/blk-mq-tag.c index cdec2cd..b5a14ce 100644 --- a/block/blk-mq-tag.c +++ b/block/blk-mq-tag.c @@ -279,101 +279,6 @@ static void bt_for_each(struct blk_mq_hw_ctx *hctx, struct sbitmap_queue *bt, sbitmap_for_each_set(&bt->sb, bt_iter, &iter_data); } -struct bt_tags_iter_data { - struct blk_mq_tags *tags; - busy_tag_iter_fn *fn; - void *data; - bool reserved; -}; - -static bool bt_tags_iter(struct sbitmap *bitmap, unsigned int bitnr, void *data) -{ - struct bt_tags_iter_data *iter_data = data; - struct blk_mq_tags *tags = iter_data->tags; - bool reserved = iter_data->reserved; - struct request *rq; - - if (!reserved) - bitnr += tags->nr_reserved_tags; - - /* - * We can hit rq == NULL here, because the tagging functions - * test and set the bit before assining ->rqs[]. - */ - rq = tags->rqs[bitnr]; - if (rq && blk_mq_request_started(rq)) - return iter_data->fn(rq, iter_data->data, reserved); - - return true; -} - -/** - * bt_tags_for_each - iterate over the requests in a tag map - * @tags: Tag map to iterate over. - * @bt: sbitmap to examine. This is either the breserved_tags member - * or the bitmap_tags member of struct blk_mq_tags. - * @fn: Pointer to the function that will be called for each started - * request. @fn will be called as follows: @fn(rq, @data, - * @reserved) where rq is a pointer to a request. Return true - * to continue iterating tags, false to stop. - * @data: Will be passed as second argument to @fn. - * @reserved: Indicates whether @bt is the breserved_tags member or the - * bitmap_tags member of struct blk_mq_tags. - */ -static void bt_tags_for_each(struct blk_mq_tags *tags, struct sbitmap_queue *bt, - busy_tag_iter_fn *fn, void *data, bool reserved) -{ - struct bt_tags_iter_data iter_data = { - .tags = tags, - .fn = fn, - .data = data, - .reserved = reserved, - }; - - if (tags->rqs) - sbitmap_for_each_set(&bt->sb, bt_tags_iter, &iter_data); -} - -/** - * blk_mq_all_tag_busy_iter - iterate over all started requests in a tag map - * @tags: Tag map to iterate over. - * @fn: Pointer to the function that will be called for each started - * request. @fn will be called as follows: @fn(rq, @priv, - * reserved) where rq is a pointer to a request. 'reserved' - * indicates whether or not @rq is a reserved request. Return - * true to continue iterating tags, false to stop. - * @priv: Will be passed as second argument to @fn. - */ -static void blk_mq_all_tag_busy_iter(struct blk_mq_tags *tags, - busy_tag_iter_fn *fn, void *priv) -{ - if (tags->nr_reserved_tags) - bt_tags_for_each(tags, &tags->breserved_tags, fn, priv, true); - bt_tags_for_each(tags, &tags->bitmap_tags, fn, priv, false); -} - -/** - * blk_mq_tagset_busy_iter - iterate over all started requests in a tag set - * @tagset: Tag set to iterate over. - * @fn: Pointer to the function that will be called for each started - * request. @fn will be called as follows: @fn(rq, @priv, - * reserved) where rq is a pointer to a request. 'reserved' - * indicates whether or not @rq is a reserved request. Return - * true to continue iterating tags, false to stop. - * @priv: Will be passed as second argument to @fn. - */ -void blk_mq_tagset_busy_iter(struct blk_mq_tag_set *tagset, - busy_tag_iter_fn *fn, void *priv) -{ - int i; - - for (i = 0; i < tagset->nr_hw_queues; i++) { - if (tagset->tags && tagset->tags[i]) - blk_mq_all_tag_busy_iter(tagset->tags[i], fn, priv); - } -} -EXPORT_SYMBOL(blk_mq_tagset_busy_iter); - /** * blk_mq_queue_tag_busy_iter - iterate over all busy tags * @q: Request queue to examine. diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h index d6beeb5..3a048db 100644 --- a/include/linux/blk-mq.h +++ b/include/linux/blk-mq.h @@ -319,8 +319,6 @@ void blk_mq_unquiesce_queue(struct request_queue *q); void blk_mq_delay_run_hw_queue(struct blk_mq_hw_ctx *hctx, unsigned long msecs); bool blk_mq_run_hw_queue(struct blk_mq_hw_ctx *hctx, bool async); void blk_mq_run_hw_queues(struct request_queue *q, bool async); -void blk_mq_tagset_busy_iter(struct blk_mq_tag_set *tagset, - busy_tag_iter_fn *fn, void *priv); void blk_mq_queue_tag_busy_iter(struct request_queue *q, busy_iter_fn *fn, void *priv, bool inflight); void blk_mq_freeze_queue(struct request_queue *q);