From patchwork Tue Apr 4 16:49:56 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Omar Sandoval X-Patchwork-Id: 9662091 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id E340A60353 for ; Tue, 4 Apr 2017 16:51:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DAB0827F10 for ; Tue, 4 Apr 2017 16:51:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CFDB928418; Tue, 4 Apr 2017 16:51:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.4 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, RCVD_IN_DNSWL_HI, RCVD_IN_SORBS_SPAM autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9ED7027F10 for ; Tue, 4 Apr 2017 16:51:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754234AbdDDQvL (ORCPT ); Tue, 4 Apr 2017 12:51:11 -0400 Received: from mail-pg0-f44.google.com ([74.125.83.44]:34812 "EHLO mail-pg0-f44.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753154AbdDDQvJ (ORCPT ); Tue, 4 Apr 2017 12:51:09 -0400 Received: by mail-pg0-f44.google.com with SMTP id 21so158018666pgg.1 for ; Tue, 04 Apr 2017 09:51:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osandov-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :in-reply-to:references; bh=7QqQX+csrBjdDy7Z03g938ycatFXklBzEXBqfkuHHx4=; b=MDBO8YHQciQF2tRFU+Nrt0ZcA2xckS/Ai5GQ9O0QRG0f7D4xywB8s79Rx9BwRkduTN WeKa+RoPS6bLqoA+ew4FQTTTh9bDp87ZAMoW/MU9lrjZUXelqrSq7mg9jil3Y8UoBXsM soWdYO+XTCQmN7ebCp7frhN6wx58gf1d2ywUAOBMruCTXJYRfKWcumfKruOqv6QxDzT9 stDUCtQK0Jg56F4n19VcH8EzUt3ztzAdX4c3lxy9TeCIsYOrRlY0KileRj8PMhnEtXCq DCca1yTYbRaxNqvaI74uEnqbUPuqUlYl44vWoXAaQ/+HVqmDFWrq8yhixnZ166MWi4Py PG6A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:in-reply-to:references; bh=7QqQX+csrBjdDy7Z03g938ycatFXklBzEXBqfkuHHx4=; b=CHQw8RSm8VW70XPG5Mi4G69i3gGChnAX7KDcXVX9kwBY73x6bsUAl7Q9RmxrpxY4P/ gqUHYvgCrNbUkHK04aodpi8GMultzrdJEG0+F+s+1cvN1rTgHl/h1uA5B7dJ704pASU8 b/WHJkV8qhBGtQxEbY6wHt8nnwpcqA6IWTkBZKxE33IAOFMWGm6p2CuzQVFhMG4BnFHR D6uK6jzfmcdgoJD2d4Ng2IE1Ieahf/h6X/yEwjtVAav3D4lrkv9IY4/avFgeF5zZyC2V myE2ByGfiuVp7jzEArvT0OVZ3yWr4Ol3NWx6NsJiFkmmkL49kwQpZIiDDdK4seWLgLQz Xd4Q== X-Gm-Message-State: AFeK/H3rdesmD/AUi8K9r19GxN34bc4Jyr+3OB80YWWeseA7U/Qn7iGRz4YoVXTMJKKuW0hc X-Received: by 10.98.219.2 with SMTP id f2mr24487095pfg.23.1491324664222; Tue, 04 Apr 2017 09:51:04 -0700 (PDT) Received: from vader.thefacebook.com ([2620:10d:c090:200::e:987e]) by smtp.gmail.com with ESMTPSA id z25sm32835297pfi.28.2017.04.04.09.51.03 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 04 Apr 2017 09:51:03 -0700 (PDT) From: Omar Sandoval To: Jens Axboe , linux-block@vger.kernel.org Cc: kernel-team@fb.com Subject: [PATCH v2 3/5] blk-mq-sched: make completed_request() callback more useful Date: Tue, 4 Apr 2017 09:49:56 -0700 Message-Id: X-Mailer: git-send-email 2.12.2 In-Reply-To: References: In-Reply-To: References: Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Omar Sandoval Currently, this callback is called right after put_request() and has no distinguishable purpose. Instead, let's call it before put_request() as soon as I/O has completed on the request, before we account it in blk-stat. With this, Kyber can enable stats when it sees a latency outlier and make sure the outlier gets accounted. Signed-off-by: Omar Sandoval --- block/blk-mq-sched.h | 11 +++-------- block/blk-mq.c | 5 ++++- include/linux/elevator.h | 2 +- 3 files changed, 8 insertions(+), 10 deletions(-) diff --git a/block/blk-mq-sched.h b/block/blk-mq-sched.h index c6e760df0fb4..d846e07bb564 100644 --- a/block/blk-mq-sched.h +++ b/block/blk-mq-sched.h @@ -82,17 +82,12 @@ blk_mq_sched_allow_merge(struct request_queue *q, struct request *rq, return true; } -static inline void -blk_mq_sched_completed_request(struct blk_mq_hw_ctx *hctx, struct request *rq) +static inline void blk_mq_sched_completed_request(struct request *rq) { - struct elevator_queue *e = hctx->queue->elevator; + struct elevator_queue *e = rq->q->elevator; if (e && e->type->ops.mq.completed_request) - e->type->ops.mq.completed_request(hctx, rq); - - BUG_ON(rq->internal_tag == -1); - - blk_mq_put_tag(hctx, hctx->sched_tags, rq->mq_ctx, rq->internal_tag); + e->type->ops.mq.completed_request(rq); } static inline void blk_mq_sched_started_request(struct request *rq) diff --git a/block/blk-mq.c b/block/blk-mq.c index b49fdfde7e06..3dbfed6d0e5b 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -350,7 +350,7 @@ void __blk_mq_finish_request(struct blk_mq_hw_ctx *hctx, struct blk_mq_ctx *ctx, if (rq->tag != -1) blk_mq_put_tag(hctx, hctx->tags, ctx, rq->tag); if (sched_tag != -1) - blk_mq_sched_completed_request(hctx, rq); + blk_mq_put_tag(hctx, hctx->sched_tags, ctx, sched_tag); blk_mq_sched_restart_queues(hctx); blk_queue_exit(q); } @@ -443,6 +443,9 @@ static void __blk_mq_complete_request(struct request *rq) { struct request_queue *q = rq->q; + if (rq->internal_tag != -1) + blk_mq_sched_completed_request(rq); + blk_mq_stat_add(rq); if (!q->softirq_done_fn) diff --git a/include/linux/elevator.h b/include/linux/elevator.h index b7ec315ee7e7..3a216318ae73 100644 --- a/include/linux/elevator.h +++ b/include/linux/elevator.h @@ -106,7 +106,7 @@ struct elevator_mq_ops { void (*insert_requests)(struct blk_mq_hw_ctx *, struct list_head *, bool); struct request *(*dispatch_request)(struct blk_mq_hw_ctx *); bool (*has_work)(struct blk_mq_hw_ctx *); - void (*completed_request)(struct blk_mq_hw_ctx *, struct request *); + void (*completed_request)(struct request *); void (*started_request)(struct request *); void (*requeue_request)(struct request *); struct request *(*former_request)(struct request_queue *, struct request *);