From patchwork Fri Aug 4 15:04:20 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 9881497 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 7016B6031B for ; Fri, 4 Aug 2017 15:04:32 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 637A1288D1 for ; Fri, 4 Aug 2017 15:04:32 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 58534289E1; Fri, 4 Aug 2017 15:04:32 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.4 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, RCVD_IN_DNSWL_HI, RCVD_IN_SORBS_SPAM autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DABD0288D1 for ; Fri, 4 Aug 2017 15:04:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752070AbdHDPEb (ORCPT ); Fri, 4 Aug 2017 11:04:31 -0400 Received: from mail-io0-f170.google.com ([209.85.223.170]:36658 "EHLO mail-io0-f170.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752064AbdHDPEa (ORCPT ); Fri, 4 Aug 2017 11:04:30 -0400 Received: by mail-io0-f170.google.com with SMTP id g35so6843495ioi.3 for ; Fri, 04 Aug 2017 08:04:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=SYXmD2dEgCOQQs8mUbGLLzndTslGqEvwg4JZrCtBLjg=; b=yWREazQ/pdjF+FR77CI6xRPBOsFdR+Mj08vAVD6zxdRkZa4shrUrzftdwoYtRagEmE 7k7cn/YK0lM/WyJ2x2UGThgsYg4L9NklVu5199mflg0vodvyPmTqEPuoanu6kZZn2e2n pLeSafJM/1N1RbKIfNffIhpvuaO7IK3Y3YC6MgDhiJlW7gODRSyxbuGkeYay3noQmmiL ClT5YDvr+xy/nvtS5s939/bo6AZhvXfh5pH8QJxmZAxbdS7k3b9D4Jesm7cHFVgLeOHL B9/WfRRJ5xTwcpK8R3yhktMIVF0cIXJcPT1OkilXyY9BPY6RoLZ+PZlI8vhNobs2JRCN AqDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=SYXmD2dEgCOQQs8mUbGLLzndTslGqEvwg4JZrCtBLjg=; b=VP78bvPTJ2F2cRR+hrNGKV6kCH6mCT6K+8i95Pq/U3Z+H67UXDbx3fCEas0o4RQ9Bl gZ4xzGJnfYwGJ1UHVKOasQXqykwgaNV4JtKOJIqoS9wzqZwPH3zX+j3Ys2V3Pcv+KXmh BMkqNpM6xiNxYnFp8ogIfqtLdKRJ+56BA4Nk6aEK/JVMSIp7aEMQREkW4I2jEmnYS1GY +288MEdnVFmLLt3X/bZ328lKMIA9zTYko9bY+TA2Y1X6rDdjh+8IlA+mpDKz2jBRGSQm 9kR8C1XA13dri+sGyBzzoJpQflYR3iYvrz6QpQBYOPWuCxGX0A7kVYIsQe2OY9o0UTuf PStg== X-Gm-Message-State: AHYfb5i5LJ+U9nuoTGA8CR9g+baC+5+Qnv2rYBwllLvyVVKiY3R3CxJ1 ZVGFRYbYZXLryDIxCHTJ4A== X-Received: by 10.107.19.229 with SMTP id 98mr2987634iot.148.1501859069603; Fri, 04 Aug 2017 08:04:29 -0700 (PDT) Received: from localhost.localdomain ([216.160.245.98]) by smtp.gmail.com with ESMTPSA id w207sm763245itc.34.2017.08.04.08.04.28 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 04 Aug 2017 08:04:28 -0700 (PDT) From: Jens Axboe To: linux-block@vger.kernel.org Cc: brking@linux.vnet.ibm.com, bart.vanassche@wdc.com, Jens Axboe Subject: [PATCH 4/6] blk-mq: provide internal in-flight variant Date: Fri, 4 Aug 2017 09:04:20 -0600 Message-Id: <1501859062-11120-5-git-send-email-axboe@kernel.dk> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1501859062-11120-1-git-send-email-axboe@kernel.dk> References: <1501859062-11120-1-git-send-email-axboe@kernel.dk> Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We don't have to inc/dec some counter, since we can just iterate the tags. That makes inc/dec a noop, but means we have to iterate busy tags to get an in-flight count. Reviewed-by: Bart Van Assche Signed-off-by: Jens Axboe Reviewed-by: Omar Sandoval --- block/blk-mq.c | 31 +++++++++++++++++++++++++++++++ block/blk-mq.h | 3 +++ block/genhd.c | 37 +++++++++++++++++++++++++++++++++++++ include/linux/genhd.h | 34 ++++++---------------------------- 4 files changed, 77 insertions(+), 28 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index a5d369dc7622..fe1aa1f5f069 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -83,6 +83,37 @@ static void blk_mq_hctx_clear_pending(struct blk_mq_hw_ctx *hctx, sbitmap_clear_bit(&hctx->ctx_map, ctx->index_hw); } +struct mq_inflight { + struct hd_struct *part; + unsigned int *inflight; +}; + +static void blk_mq_check_inflight(struct blk_mq_hw_ctx *hctx, + struct request *rq, void *priv, + bool reserved) +{ + struct mq_inflight *mi = priv; + + if (!test_bit(REQ_ATOM_STARTED, &rq->atomic_flags)) + return; + + /* + * Count as inflight if it either matches the partition we asked + * for, or if it's the root + */ + if (rq->part == mi->part || mi->part->partno) + mi->inflight[0]++; +} + +void blk_mq_in_flight(struct request_queue *q, struct hd_struct *part, + unsigned int inflight[2]) +{ + struct mq_inflight mi = { .part = part, .inflight = inflight, }; + + inflight[0] = 0; + blk_mq_queue_tag_busy_iter(q, blk_mq_check_inflight, &mi); +} + void blk_freeze_queue_start(struct request_queue *q) { int freeze_depth; diff --git a/block/blk-mq.h b/block/blk-mq.h index 60b01c0309bc..98252b79b80b 100644 --- a/block/blk-mq.h +++ b/block/blk-mq.h @@ -133,4 +133,7 @@ static inline bool blk_mq_hw_queue_mapped(struct blk_mq_hw_ctx *hctx) return hctx->nr_ctx && hctx->tags; } +void blk_mq_in_flight(struct request_queue *q, struct hd_struct *part, + unsigned int inflight[2]); + #endif diff --git a/block/genhd.c b/block/genhd.c index 822f65f95e2a..3dc4d115480f 100644 --- a/block/genhd.c +++ b/block/genhd.c @@ -45,6 +45,43 @@ static void disk_add_events(struct gendisk *disk); static void disk_del_events(struct gendisk *disk); static void disk_release_events(struct gendisk *disk); +void part_inc_in_flight(struct request_queue *q, struct hd_struct *part, int rw) +{ + if (q->mq_ops) + return; + + atomic_inc(&part->in_flight[rw]); + if (part->partno) + atomic_inc(&part_to_disk(part)->part0.in_flight[rw]); +} + +void part_dec_in_flight(struct request_queue *q, struct hd_struct *part, int rw) +{ + if (q->mq_ops) + return; + + atomic_dec(&part->in_flight[rw]); + if (part->partno) + atomic_dec(&part_to_disk(part)->part0.in_flight[rw]); +} + +void part_in_flight(struct request_queue *q, struct hd_struct *part, + unsigned int inflight[2]) +{ + if (q->mq_ops) { + blk_mq_in_flight(q, part, inflight); + return; + } + + inflight[0] = atomic_read(&part->in_flight[0]) + + atomic_read(&part->in_flight[1]); + if (part->partno) { + part = &part_to_disk(part)->part0; + inflight[1] = atomic_read(&part->in_flight[0]) + + atomic_read(&part->in_flight[1]); + } +} + /** * disk_get_part - get partition * @disk: disk to look partition from diff --git a/include/linux/genhd.h b/include/linux/genhd.h index a9c8ea632fdc..ea652bfcd675 100644 --- a/include/linux/genhd.h +++ b/include/linux/genhd.h @@ -362,34 +362,12 @@ static inline void free_part_stats(struct hd_struct *part) #define part_stat_sub(cpu, gendiskp, field, subnd) \ part_stat_add(cpu, gendiskp, field, -subnd) -static inline void part_inc_in_flight(struct request_queue *q, - struct hd_struct *part, int rw) -{ - atomic_inc(&part->in_flight[rw]); - if (part->partno) - atomic_inc(&part_to_disk(part)->part0.in_flight[rw]); -} - -static inline void part_dec_in_flight(struct request_queue *q, - struct hd_struct *part, int rw) -{ - atomic_dec(&part->in_flight[rw]); - if (part->partno) - atomic_dec(&part_to_disk(part)->part0.in_flight[rw]); -} - -static inline void part_in_flight(struct request_queue *q, - struct hd_struct *part, - unsigned int inflight[2]) -{ - inflight[0] = atomic_read(&part->in_flight[0]) + - atomic_read(&part->in_flight[1]); - if (part->partno) { - part = &part_to_disk(part)->part0; - inflight[1] = atomic_read(&part->in_flight[0]) + - atomic_read(&part->in_flight[1]); - } -} +void part_in_flight(struct request_queue *q, struct hd_struct *part, + unsigned int inflight[2]); +void part_dec_in_flight(struct request_queue *q, struct hd_struct *part, + int rw); +void part_inc_in_flight(struct request_queue *q, struct hd_struct *part, + int rw); static inline struct partition_meta_info *alloc_part_info(struct gendisk *disk) {