From patchwork Wed Jan 11 21:39:59 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 9511527 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id F127460710 for ; Wed, 11 Jan 2017 21:42:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E3571286A1 for ; Wed, 11 Jan 2017 21:42:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D8559286A4; Wed, 11 Jan 2017 21:42:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A093F286A1 for ; Wed, 11 Jan 2017 21:42:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1761574AbdAKVl5 (ORCPT ); Wed, 11 Jan 2017 16:41:57 -0500 Received: from mx0b-00082601.pphosted.com ([67.231.153.30]:57358 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1761543AbdAKVkP (ORCPT ); Wed, 11 Jan 2017 16:40:15 -0500 Received: from pps.filterd (m0001255.ppops.net [127.0.0.1]) by mx0b-00082601.pphosted.com (8.16.0.17/8.16.0.17) with SMTP id v0BLVxOb014310; Wed, 11 Jan 2017 13:40:12 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=facebook; bh=BPIxK6+5S9Z7kph88Gz89+VQUmr/tZ3hk5axHzFs/uc=; b=oaeX5ms6EJjD2z+9pT2C2ygLpUiX8Ye4HgV+O9Qn5RK5MW7ldrdH79ao52t9iw0uoWpn 2I/+MWuKzRf7n+0o38teoBF3+0V1d8a5tpRYf7MfvXKkeSlhbpyxqJyPGuX9dZdjV199 5EDWPBD50OawRzswrqMS1a6BMoYFzop7E5k= Received: from mail.thefacebook.com ([199.201.64.23]) by mx0b-00082601.pphosted.com with ESMTP id 27wutb08ku-4 (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=NOT); Wed, 11 Jan 2017 13:40:12 -0800 Received: from localhost.localdomain (192.168.54.13) by mail.thefacebook.com (192.168.16.22) with Microsoft SMTP Server (TLS) id 14.3.294.0; Wed, 11 Jan 2017 13:40:10 -0800 From: Jens Axboe To: , CC: , , Jens Axboe Subject: [PATCH 06/10] blk-mq-tag: cleanup the normal/reserved tag allocation Date: Wed, 11 Jan 2017 14:39:59 -0700 Message-ID: <1484170803-9311-7-git-send-email-axboe@fb.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1484170803-9311-1-git-send-email-axboe@fb.com> References: <1484170803-9311-1-git-send-email-axboe@fb.com> MIME-Version: 1.0 X-Originating-IP: [192.168.54.13] X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-01-11_17:, , signatures=0 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This is in preparation for having another tag set available. Cleanup the parameters, and allow passing in of tags fo blk_mq_put_tag(). Signed-off-by: Jens Axboe Reviewed-by: Bart Van Assche -- --- block/blk-mq-tag.c | 58 ++++++++++++++++++++++++++++++------------------------ block/blk-mq-tag.h | 4 ++-- block/blk-mq.c | 2 +- block/blk-mq.h | 5 +++++ 4 files changed, 40 insertions(+), 29 deletions(-) diff --git a/block/blk-mq-tag.c b/block/blk-mq-tag.c index dcf5ce3ba4bf..0ff8df347bfb 100644 --- a/block/blk-mq-tag.c +++ b/block/blk-mq-tag.c @@ -97,25 +97,35 @@ static int __bt_get(struct blk_mq_hw_ctx *hctx, struct sbitmap_queue *bt) return __sbitmap_queue_get(bt); } -static int bt_get(struct blk_mq_alloc_data *data, struct sbitmap_queue *bt, - struct blk_mq_hw_ctx *hctx, struct blk_mq_tags *tags) +static struct sbitmap_queue *bt_from_data(struct blk_mq_alloc_data *data) { + struct blk_mq_tags *tags = blk_mq_tags_from_data(data); + + if (data->flags & BLK_MQ_REQ_RESERVED) + return &tags->breserved_tags; + + return &tags->bitmap_tags; +} + +static int bt_get(struct blk_mq_alloc_data *data) +{ + struct sbitmap_queue *bt = bt_from_data(data); struct sbq_wait_state *ws; DEFINE_WAIT(wait); int tag; - tag = __bt_get(hctx, bt); + tag = __bt_get(data->hctx, bt); if (tag != -1) return tag; if (data->flags & BLK_MQ_REQ_NOWAIT) return -1; - ws = bt_wait_ptr(bt, hctx); + ws = bt_wait_ptr(bt, data->hctx); do { prepare_to_wait(&ws->wait, &wait, TASK_UNINTERRUPTIBLE); - tag = __bt_get(hctx, bt); + tag = __bt_get(data->hctx, bt); if (tag != -1) break; @@ -125,14 +135,14 @@ static int bt_get(struct blk_mq_alloc_data *data, struct sbitmap_queue *bt, * some to complete. Note that hctx can be NULL here for * reserved tag allocation. */ - if (hctx) - blk_mq_run_hw_queue(hctx, false); + if (data->hctx) + blk_mq_run_hw_queue(data->hctx, false); /* * Retry tag allocation after running the hardware queue, * as running the queue may also have found completions. */ - tag = __bt_get(hctx, bt); + tag = __bt_get(data->hctx, bt); if (tag != -1) break; @@ -142,14 +152,10 @@ static int bt_get(struct blk_mq_alloc_data *data, struct sbitmap_queue *bt, data->ctx = blk_mq_get_ctx(data->q); data->hctx = blk_mq_map_queue(data->q, data->ctx->cpu); - if (data->flags & BLK_MQ_REQ_RESERVED) { - bt = &data->hctx->tags->breserved_tags; - } else { - hctx = data->hctx; - bt = &hctx->tags->bitmap_tags; - } + bt = bt_from_data(data); + finish_wait(&ws->wait, &wait); - ws = bt_wait_ptr(bt, hctx); + ws = bt_wait_ptr(bt, data->hctx); } while (1); finish_wait(&ws->wait, &wait); @@ -160,25 +166,27 @@ static unsigned int __blk_mq_get_tag(struct blk_mq_alloc_data *data) { int tag; - tag = bt_get(data, &data->hctx->tags->bitmap_tags, data->hctx, - data->hctx->tags); - if (tag >= 0) - return tag + data->hctx->tags->nr_reserved_tags; + tag = bt_get(data); + if (tag >= 0) { + struct blk_mq_tags *tags = blk_mq_tags_from_data(data); + + return tag + tags->nr_reserved_tags; + } return BLK_MQ_TAG_FAIL; } static unsigned int __blk_mq_get_reserved_tag(struct blk_mq_alloc_data *data) { + struct blk_mq_tags *tags = blk_mq_tags_from_data(data); int tag; - if (unlikely(!data->hctx->tags->nr_reserved_tags)) { + if (unlikely(!tags->nr_reserved_tags)) { WARN_ON_ONCE(1); return BLK_MQ_TAG_FAIL; } - tag = bt_get(data, &data->hctx->tags->breserved_tags, NULL, - data->hctx->tags); + tag = bt_get(data); if (tag < 0) return BLK_MQ_TAG_FAIL; @@ -192,11 +200,9 @@ unsigned int blk_mq_get_tag(struct blk_mq_alloc_data *data) return __blk_mq_get_tag(data); } -void blk_mq_put_tag(struct blk_mq_hw_ctx *hctx, struct blk_mq_ctx *ctx, - unsigned int tag) +void blk_mq_put_tag(struct blk_mq_hw_ctx *hctx, struct blk_mq_tags *tags, + struct blk_mq_ctx *ctx, unsigned int tag) { - struct blk_mq_tags *tags = hctx->tags; - if (tag >= tags->nr_reserved_tags) { const int real_tag = tag - tags->nr_reserved_tags; diff --git a/block/blk-mq-tag.h b/block/blk-mq-tag.h index d1662734dc53..923602dd3bfb 100644 --- a/block/blk-mq-tag.h +++ b/block/blk-mq-tag.h @@ -24,8 +24,8 @@ extern struct blk_mq_tags *blk_mq_init_tags(unsigned int nr_tags, unsigned int r extern void blk_mq_free_tags(struct blk_mq_tags *tags); extern unsigned int blk_mq_get_tag(struct blk_mq_alloc_data *data); -extern void blk_mq_put_tag(struct blk_mq_hw_ctx *hctx, struct blk_mq_ctx *ctx, - unsigned int tag); +extern void blk_mq_put_tag(struct blk_mq_hw_ctx *hctx, struct blk_mq_tags *tags, + struct blk_mq_ctx *ctx, unsigned int tag); extern bool blk_mq_has_free_tags(struct blk_mq_tags *tags); extern ssize_t blk_mq_tag_sysfs_show(struct blk_mq_tags *tags, char *page); extern int blk_mq_tag_update_depth(struct blk_mq_tags *tags, unsigned int depth); diff --git a/block/blk-mq.c b/block/blk-mq.c index 9fc521755e22..6fab8e9c724f 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -335,7 +335,7 @@ void __blk_mq_free_request(struct blk_mq_hw_ctx *hctx, struct blk_mq_ctx *ctx, clear_bit(REQ_ATOM_STARTED, &rq->atomic_flags); clear_bit(REQ_ATOM_POLL_SLEPT, &rq->atomic_flags); - blk_mq_put_tag(hctx, ctx, tag); + blk_mq_put_tag(hctx, hctx->tags, ctx, tag); blk_queue_exit(q); } diff --git a/block/blk-mq.h b/block/blk-mq.h index e59f5ca520a2..48b7771eb192 100644 --- a/block/blk-mq.h +++ b/block/blk-mq.h @@ -118,6 +118,11 @@ static inline void blk_mq_set_alloc_data(struct blk_mq_alloc_data *data, data->hctx = hctx; } +static inline struct blk_mq_tags *blk_mq_tags_from_data(struct blk_mq_alloc_data *data) +{ + return data->hctx->tags; +} + /* * Internal helpers for request allocation/init/free */