From patchwork Wed May 2 17:34:21 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Omar Sandoval X-Patchwork-Id: 10376125 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 2625F60384 for ; Wed, 2 May 2018 17:34:46 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 15C9428ECC for ; Wed, 2 May 2018 17:34:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0A4AD28F29; Wed, 2 May 2018 17:34:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CEB9928F1D for ; Wed, 2 May 2018 17:34:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751020AbeEBReo (ORCPT ); Wed, 2 May 2018 13:34:44 -0400 Received: from mail-pg0-f65.google.com ([74.125.83.65]:42389 "EHLO mail-pg0-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750925AbeEBRen (ORCPT ); Wed, 2 May 2018 13:34:43 -0400 Received: by mail-pg0-f65.google.com with SMTP id p9-v6so8229372pgc.9 for ; Wed, 02 May 2018 10:34:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osandov-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :in-reply-to:references; bh=G0iwHCfg9qPBksTNJwzft5MXQyIYLzZEQN7+UHpxukw=; b=dqUAx5TgfNy5Qa5/QwQBDpyYu0NMbwJx/wffxoxn+syhKjsik5EKIZF8o/OFwNZCF5 sO6oAW33IloRHiNSFD1a9W5PC1jtJ24zM/v5x00HxwfDMr/wE3Y1mzKdKuBRlabKKUTA 6O1yhBkxr7+EfUCD43kIeaBbTJeDZnFkFUz7ZVpidYMO86CYvnPKRnKVgkD/JHZYmeHG VidevElEJdOcQI/mh58pIhoWQXe/AE36hIxDWYgn9OAEc9f/stZaBrpMqwtDCh6V66H0 GgIc9WItF17/FCKYzPGY3tD4yLyIAv8Krx+r5KBU1Uj1LzWVM8NCoYIosowV6QKJVkAq /mWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:in-reply-to:references; bh=G0iwHCfg9qPBksTNJwzft5MXQyIYLzZEQN7+UHpxukw=; b=aEZ54GTM0bgLXrmlKcDtFYCwfr814CrQeydkdUmyrf7PMPTLmNNRVTW0RlT+0z/hzJ Jj6Gnd3eEReuRf1jcE9kv62qdypjdpLEVg054qAFaxz2V0pgJxk4q+UnPin63pVbReNr iqGy51EI/qhU8UkLQDHtOBrA7cosz32wP9gW80jNsSlRavnmeJ3gxr9jtbuYMTA+yLc7 6uNOdsQcYaKzDMnIvrvrVd//ejBY9CoxdKDiquaJlhYh7LCuwbKAV9bXlDCNvz1iuUED R4iKbyR616cJT5NqEwBIYkykJ5CEYjMiy1VCBEy8v2QEdWfWBuBLxgabwgiz2Y9z12SC 0crw== X-Gm-Message-State: ALQs6tBJaG/7d9SZfAGA30GEmZ41CT9Rrpqdv0WDzjPbzXJ7jAEMNlur 6wuielCKxDUW5EXLD/qk+kZC/DKH3co= X-Google-Smtp-Source: AB8JxZrDm/saTcK/vpEDIwkphtVsvATK/LEuNmSGZUwiAh2NiiHAvOuSoliQaTiy6EZOY/77k6lxbg== X-Received: by 10.98.75.139 with SMTP id d11mr20255746pfj.244.1525282482089; Wed, 02 May 2018 10:34:42 -0700 (PDT) Received: from vader.thefacebook.com ([2620:10d:c090:180::1:24e1]) by smtp.gmail.com with ESMTPSA id w184-v6sm17757388pgb.20.2018.05.02.10.34.40 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 02 May 2018 10:34:41 -0700 (PDT) From: Omar Sandoval To: linux-block@vger.kernel.org Cc: Jens Axboe , kernel-team@fb.com Subject: [PATCH 2/7] block: pass struct request instead of struct blk_issue_stat to wbt Date: Wed, 2 May 2018 10:34:21 -0700 Message-Id: <42fc4271b0d124982df9fd4d1dd66f1bcb1c528e.1525282392.git.osandov@fb.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: References: In-Reply-To: References: Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Omar Sandoval issue_stat is going to go away, so first make writeback throttling take the containing request, update the internal wbt helpers accordingly, and change rwb->sync_cookie to be the request pointer instead of the issue_stat pointer. No functional change. Signed-off-by: Omar Sandoval --- block/blk-core.c | 10 ++++----- block/blk-mq.c | 10 ++++----- block/blk-wbt.c | 53 ++++++++++++++++++++++++------------------------ block/blk-wbt.h | 18 ++++++++-------- 4 files changed, 45 insertions(+), 46 deletions(-) diff --git a/block/blk-core.c b/block/blk-core.c index 85909b431eb0..851ea3c0c0b4 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -1660,7 +1660,7 @@ void blk_requeue_request(struct request_queue *q, struct request *rq) blk_delete_timer(rq); blk_clear_rq_complete(rq); trace_block_rq_requeue(q, rq); - wbt_requeue(q->rq_wb, &rq->issue_stat); + wbt_requeue(q->rq_wb, rq); if (rq->rq_flags & RQF_QUEUED) blk_queue_end_tag(q, rq); @@ -1767,7 +1767,7 @@ void __blk_put_request(struct request_queue *q, struct request *req) /* this is a bio leak */ WARN_ON(req->bio != NULL); - wbt_done(q->rq_wb, &req->issue_stat); + wbt_done(q->rq_wb, req); /* * Request may not have originated from ll_rw_blk. if not, @@ -2078,7 +2078,7 @@ static blk_qc_t blk_queue_bio(struct request_queue *q, struct bio *bio) goto out_unlock; } - wbt_track(&req->issue_stat, wb_acct); + wbt_track(req, wb_acct); /* * After dropping the lock and possibly sleeping here, our request @@ -2994,7 +2994,7 @@ void blk_start_request(struct request *req) if (test_bit(QUEUE_FLAG_STATS, &req->q->queue_flags)) { blk_stat_set_issue(&req->issue_stat, blk_rq_sectors(req)); req->rq_flags |= RQF_STATS; - wbt_issue(req->q->rq_wb, &req->issue_stat); + wbt_issue(req->q->rq_wb, req); } BUG_ON(blk_rq_is_complete(req)); @@ -3213,7 +3213,7 @@ void blk_finish_request(struct request *req, blk_status_t error) blk_account_io_done(req); if (req->end_io) { - wbt_done(req->q->rq_wb, &req->issue_stat); + wbt_done(req->q->rq_wb, req); req->end_io(req, error); } else { if (blk_bidi_rq(req)) diff --git a/block/blk-mq.c b/block/blk-mq.c index c3621453ad87..5c9e3ea6cc4e 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -472,7 +472,7 @@ void blk_mq_free_request(struct request *rq) if (unlikely(laptop_mode && !blk_rq_is_passthrough(rq))) laptop_io_completion(q->backing_dev_info); - wbt_done(q->rq_wb, &rq->issue_stat); + wbt_done(q->rq_wb, rq); if (blk_rq_rl(rq)) blk_put_rl(blk_rq_rl(rq)); @@ -492,7 +492,7 @@ inline void __blk_mq_end_request(struct request *rq, blk_status_t error) blk_account_io_done(rq); if (rq->end_io) { - wbt_done(rq->q->rq_wb, &rq->issue_stat); + wbt_done(rq->q->rq_wb, rq); rq->end_io(rq, error); } else { if (unlikely(blk_bidi_rq(rq))) @@ -655,7 +655,7 @@ void blk_mq_start_request(struct request *rq) if (test_bit(QUEUE_FLAG_STATS, &q->queue_flags)) { blk_stat_set_issue(&rq->issue_stat, blk_rq_sectors(rq)); rq->rq_flags |= RQF_STATS; - wbt_issue(q->rq_wb, &rq->issue_stat); + wbt_issue(q->rq_wb, rq); } WARN_ON_ONCE(blk_mq_rq_state(rq) != MQ_RQ_IDLE); @@ -703,7 +703,7 @@ static void __blk_mq_requeue_request(struct request *rq) blk_mq_put_driver_tag(rq); trace_block_rq_requeue(q, rq); - wbt_requeue(q->rq_wb, &rq->issue_stat); + wbt_requeue(q->rq_wb, rq); if (blk_mq_rq_state(rq) != MQ_RQ_IDLE) { blk_mq_rq_update_state(rq, MQ_RQ_IDLE); @@ -1866,7 +1866,7 @@ static blk_qc_t blk_mq_make_request(struct request_queue *q, struct bio *bio) return BLK_QC_T_NONE; } - wbt_track(&rq->issue_stat, wb_acct); + wbt_track(rq, wb_acct); cookie = request_to_qc_t(data.hctx, rq); diff --git a/block/blk-wbt.c b/block/blk-wbt.c index 030d6bdcaa4c..10aecfadc57d 100644 --- a/block/blk-wbt.c +++ b/block/blk-wbt.c @@ -29,24 +29,24 @@ #define CREATE_TRACE_POINTS #include -static inline void wbt_clear_state(struct blk_issue_stat *stat) +static inline void wbt_clear_state(struct request *rq) { - stat->stat &= ~BLK_STAT_RES_MASK; + rq->issue_stat.stat &= ~BLK_STAT_RES_MASK; } -static inline enum wbt_flags wbt_stat_to_mask(struct blk_issue_stat *stat) +static inline enum wbt_flags wbt_flags(struct request *rq) { - return (stat->stat & BLK_STAT_RES_MASK) >> BLK_STAT_RES_SHIFT; + return (rq->issue_stat.stat & BLK_STAT_RES_MASK) >> BLK_STAT_RES_SHIFT; } -static inline bool wbt_is_tracked(struct blk_issue_stat *stat) +static inline bool wbt_is_tracked(struct request *rq) { - return (stat->stat >> BLK_STAT_RES_SHIFT) & WBT_TRACKED; + return (rq->issue_stat.stat >> BLK_STAT_RES_SHIFT) & WBT_TRACKED; } -static inline bool wbt_is_read(struct blk_issue_stat *stat) +static inline bool wbt_is_read(struct request *rq) { - return (stat->stat >> BLK_STAT_RES_SHIFT) & WBT_READ; + return (rq->issue_stat.stat >> BLK_STAT_RES_SHIFT) & WBT_READ; } enum { @@ -185,24 +185,24 @@ void __wbt_done(struct rq_wb *rwb, enum wbt_flags wb_acct) * Called on completion of a request. Note that it's also called when * a request is merged, when the request gets freed. */ -void wbt_done(struct rq_wb *rwb, struct blk_issue_stat *stat) +void wbt_done(struct rq_wb *rwb, struct request *rq) { if (!rwb) return; - if (!wbt_is_tracked(stat)) { - if (rwb->sync_cookie == stat) { + if (!wbt_is_tracked(rq)) { + if (rwb->sync_cookie == rq) { rwb->sync_issue = 0; rwb->sync_cookie = NULL; } - if (wbt_is_read(stat)) + if (wbt_is_read(rq)) wb_timestamp(rwb, &rwb->last_comp); } else { - WARN_ON_ONCE(stat == rwb->sync_cookie); - __wbt_done(rwb, wbt_stat_to_mask(stat)); + WARN_ON_ONCE(rq == rwb->sync_cookie); + __wbt_done(rwb, wbt_flags(rq)); } - wbt_clear_state(stat); + wbt_clear_state(rq); } /* @@ -629,30 +629,29 @@ enum wbt_flags wbt_wait(struct rq_wb *rwb, struct bio *bio, spinlock_t *lock) return ret | WBT_TRACKED; } -void wbt_issue(struct rq_wb *rwb, struct blk_issue_stat *stat) +void wbt_issue(struct rq_wb *rwb, struct request *rq) { if (!rwb_enabled(rwb)) return; /* - * Track sync issue, in case it takes a long time to complete. Allows - * us to react quicker, if a sync IO takes a long time to complete. - * Note that this is just a hint. 'stat' can go away when the - * request completes, so it's important we never dereference it. We - * only use the address to compare with, which is why we store the - * sync_issue time locally. + * Track sync issue, in case it takes a long time to complete. Allows us + * to react quicker, if a sync IO takes a long time to complete. Note + * that this is just a hint. The request can go away when it completes, + * so it's important we never dereference it. We only use the address to + * compare with, which is why we store the sync_issue time locally. */ - if (wbt_is_read(stat) && !rwb->sync_issue) { - rwb->sync_cookie = stat; - rwb->sync_issue = blk_stat_time(stat); + if (wbt_is_read(rq) && !rwb->sync_issue) { + rwb->sync_cookie = rq; + rwb->sync_issue = blk_stat_time(&rq->issue_stat); } } -void wbt_requeue(struct rq_wb *rwb, struct blk_issue_stat *stat) +void wbt_requeue(struct rq_wb *rwb, struct request *rq) { if (!rwb_enabled(rwb)) return; - if (stat == rwb->sync_cookie) { + if (rq == rwb->sync_cookie) { rwb->sync_issue = 0; rwb->sync_cookie = NULL; } diff --git a/block/blk-wbt.h b/block/blk-wbt.h index d3f47a721aec..dd9211fa853f 100644 --- a/block/blk-wbt.h +++ b/block/blk-wbt.h @@ -84,19 +84,19 @@ static inline unsigned int wbt_inflight(struct rq_wb *rwb) #ifdef CONFIG_BLK_WBT -static inline void wbt_track(struct blk_issue_stat *stat, enum wbt_flags flags) +static inline void wbt_track(struct request *rq, enum wbt_flags flags) { - stat->stat |= ((u64)flags) << BLK_STAT_RES_SHIFT; + rq->issue_stat.stat |= ((u64)flags) << BLK_STAT_RES_SHIFT; } void __wbt_done(struct rq_wb *, enum wbt_flags); -void wbt_done(struct rq_wb *, struct blk_issue_stat *); +void wbt_done(struct rq_wb *, struct request *); enum wbt_flags wbt_wait(struct rq_wb *, struct bio *, spinlock_t *); int wbt_init(struct request_queue *); void wbt_exit(struct request_queue *); void wbt_update_limits(struct rq_wb *); -void wbt_requeue(struct rq_wb *, struct blk_issue_stat *); -void wbt_issue(struct rq_wb *, struct blk_issue_stat *); +void wbt_requeue(struct rq_wb *, struct request *); +void wbt_issue(struct rq_wb *, struct request *); void wbt_disable_default(struct request_queue *); void wbt_enable_default(struct request_queue *); @@ -107,13 +107,13 @@ u64 wbt_default_latency_nsec(struct request_queue *); #else -static inline void wbt_track(struct blk_issue_stat *stat, enum wbt_flags flags) +static inline void wbt_track(struct request *rq, enum wbt_flags flags) { } static inline void __wbt_done(struct rq_wb *rwb, enum wbt_flags flags) { } -static inline void wbt_done(struct rq_wb *rwb, struct blk_issue_stat *stat) +static inline void wbt_done(struct rq_wb *rwb, struct request *rq) { } static inline enum wbt_flags wbt_wait(struct rq_wb *rwb, struct bio *bio, @@ -131,10 +131,10 @@ static inline void wbt_exit(struct request_queue *q) static inline void wbt_update_limits(struct rq_wb *rwb) { } -static inline void wbt_requeue(struct rq_wb *rwb, struct blk_issue_stat *stat) +static inline void wbt_requeue(struct rq_wb *rwb, struct request *rq) { } -static inline void wbt_issue(struct rq_wb *rwb, struct blk_issue_stat *stat) +static inline void wbt_issue(struct rq_wb *rwb, struct request *rq) { } static inline void wbt_disable_default(struct request_queue *q)