From patchwork Wed May 2 21:32:41 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Omar Sandoval X-Patchwork-Id: 10376663 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 574C8603B4 for ; Wed, 2 May 2018 21:33:19 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 46CB52866D for ; Wed, 2 May 2018 21:33:19 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 389F928995; Wed, 2 May 2018 21:33:19 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 58D7B29112 for ; Wed, 2 May 2018 21:33:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751814AbeEBVdH (ORCPT ); Wed, 2 May 2018 17:33:07 -0400 Received: from mail-pg0-f67.google.com ([74.125.83.67]:42530 "EHLO mail-pg0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751339AbeEBVdG (ORCPT ); Wed, 2 May 2018 17:33:06 -0400 Received: by mail-pg0-f67.google.com with SMTP id p9-v6so8655901pgc.9 for ; Wed, 02 May 2018 14:33:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osandov-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :in-reply-to:references; bh=L71TYlBghQWX0vMQZ7vIQpnhuZUAVmBd/6sLPwvjDRo=; b=wtm5smOP0akw3V+PMz7w2brq2+m6EotmiEnOpuD9hDJuzfNXyH/+N+ZiGIJWxSkAhC DHU/QnQ3nCrwGYT1dik66kuDqy5gzjsRCk2e7kmAtV7fU5vTa7CNgK44mvHj3MSwuBoG cySqJqwdTiii7JXAYM7EPpAQ9wSdfChMgOaXcTDbdDdwYZ+1NAMB96or3ZOX0RFJRWkF ZfsQAZstrvY40HpLdKKMtZAKuSJfo2JEwSVKSxOmr+JOjRuZBa1kd+6MFnMuaMj5Jtai KwJBXv5S9UnidTQo+qoTZ8kPwi1ja0rlY6hFBAmfR1C4s8acQeTJEaiaVeOmYGa1Z9aE fcFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:in-reply-to:references; bh=L71TYlBghQWX0vMQZ7vIQpnhuZUAVmBd/6sLPwvjDRo=; b=czi/s/4ArDo2Pdd63XX2HBw4+/vPsv3Y2he0qw++Cs7jpg8xgMLDFfJIHZ+Sd54V6o lbZFG1i0hH/ApS8gYPrzd8CBlG/wftqUThbW9LfJy5Lt/7aSZOkz19vHlyZEKWg9k9+F vz/sWDmNYT3fyDdVzbqjJb3jLrtjKUudwr3+TMnQL0tCjK+hPn2VGzaJfNlIqLjekWcR mKQxt5UXa/5NxMmFrrPxGD3WBUjf/naIZtsOfVbnPWus/HzdViUWzftrz/p8c6DwRcgu TZQUjA9zOKFlqAwZkL9RoAfGY+OsioLCO79xLy6BLDpoeZWogdMHGZOtWh5G+JoquMn1 t1cA== X-Gm-Message-State: ALQs6tA8Mm2F6gLrjYHWNJ0raDtJA92IxSWv24mUWVPX40hkMrkAqPgQ G5LOL8BrAUAx19kZRq85zpYxEw0QyIU= X-Google-Smtp-Source: AB8JxZofIJXfaP6UW7kaxgavbZwPe6V5sI4P72ncBFaFjlHbq2x3VxwtTE4//k88tZPFXqUWyDaBVQ== X-Received: by 2002:a63:a60a:: with SMTP id t10-v6mr17601022pge.357.1525296785756; Wed, 02 May 2018 14:33:05 -0700 (PDT) Received: from vader.thefacebook.com ([2620:10d:c090:180::1:24e1]) by smtp.gmail.com with ESMTPSA id s6-v6sm9640492pgq.19.2018.05.02.14.33.04 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 02 May 2018 14:33:05 -0700 (PDT) From: Omar Sandoval To: linux-block@vger.kernel.org Cc: Jens Axboe , kernel-team@fb.com, Josef Bacik Subject: [PATCH v2 3/7] block: replace bio->bi_issue_stat with bio-specific type Date: Wed, 2 May 2018 14:32:41 -0700 Message-Id: <7dabf9eaa28c5dbf4a83a9e95da386d302e7b560.1525296089.git.osandov@fb.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: References: In-Reply-To: References: Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Omar Sandoval struct blk_issue_stat is going away, and bio->bi_issue_stat doesn't even use the blk-stats interface, so we can provide a separate implementation specific for bios. The helpers work the same way as the blk-stats helpers. Signed-off-by: Omar Sandoval --- block/blk-throttle.c | 19 +++++++-------- include/linux/blk_types.h | 49 ++++++++++++++++++++++++++++++++++++++- 2 files changed, 56 insertions(+), 12 deletions(-) diff --git a/block/blk-throttle.c b/block/blk-throttle.c index c5a131673733..35f9b8ff40d7 100644 --- a/block/blk-throttle.c +++ b/block/blk-throttle.c @@ -36,8 +36,6 @@ static int throtl_quantum = 32; */ #define LATENCY_FILTERED_HD (1000L) /* 1ms */ -#define SKIP_LATENCY (((u64)1) << BLK_STAT_RES_SHIFT) - static struct blkcg_policy blkcg_policy_throtl; /* A workqueue to queue throttle related work */ @@ -2139,7 +2137,7 @@ static void blk_throtl_assoc_bio(struct throtl_grp *tg, struct bio *bio) bio->bi_cg_private = tg; blkg_get(tg_to_blkg(tg)); } - blk_stat_set_issue(&bio->bi_issue_stat, bio_sectors(bio)); + bio_issue_init(&bio->bi_issue, bio_sectors(bio)); #endif } @@ -2251,7 +2249,7 @@ bool blk_throtl_bio(struct request_queue *q, struct blkcg_gq *blkg, #ifdef CONFIG_BLK_DEV_THROTTLING_LOW if (throttled || !td->track_bio_latency) - bio->bi_issue_stat.stat |= SKIP_LATENCY; + bio->bi_issue.value |= BIO_ISSUE_THROTL_SKIP_LATENCY; #endif return throttled; } @@ -2302,8 +2300,8 @@ void blk_throtl_bio_endio(struct bio *bio) finish_time_ns = ktime_get_ns(); tg->last_finish_time = finish_time_ns >> 10; - start_time = blk_stat_time(&bio->bi_issue_stat) >> 10; - finish_time = __blk_stat_time(finish_time_ns) >> 10; + start_time = bio_issue_time(&bio->bi_issue) >> 10; + finish_time = __bio_issue_time(finish_time_ns) >> 10; if (!start_time || finish_time <= start_time) { blkg_put(tg_to_blkg(tg)); return; @@ -2311,16 +2309,15 @@ void blk_throtl_bio_endio(struct bio *bio) lat = finish_time - start_time; /* this is only for bio based driver */ - if (!(bio->bi_issue_stat.stat & SKIP_LATENCY)) - throtl_track_latency(tg->td, blk_stat_size(&bio->bi_issue_stat), - bio_op(bio), lat); + if (!(bio->bi_issue.value & BIO_ISSUE_THROTL_SKIP_LATENCY)) + throtl_track_latency(tg->td, bio_issue_size(&bio->bi_issue), + bio_op(bio), lat); if (tg->latency_target && lat >= tg->td->filtered_latency) { int bucket; unsigned int threshold; - bucket = request_bucket_index( - blk_stat_size(&bio->bi_issue_stat)); + bucket = request_bucket_index(bio_issue_size(&bio->bi_issue)); threshold = tg->td->avg_buckets[rw][bucket].latency + tg->latency_target; if (lat > threshold) diff --git a/include/linux/blk_types.h b/include/linux/blk_types.h index 17b18b91ebac..b6f1d53cf113 100644 --- a/include/linux/blk_types.h +++ b/include/linux/blk_types.h @@ -8,6 +8,7 @@ #include #include +#include struct bio_set; struct bio; @@ -94,6 +95,52 @@ struct blk_issue_stat { u64 stat; }; +/* + * From most significant bit: + * 1 bit: reserved for other usage, see below + * 12 bits: original size of bio + * 51 bits: issue time of bio + */ +#define BIO_ISSUE_RES_BITS 1 +#define BIO_ISSUE_SIZE_BITS 12 +#define BIO_ISSUE_RES_SHIFT (64 - BIO_ISSUE_RES_BITS) +#define BIO_ISSUE_SIZE_SHIFT (BIO_ISSUE_RES_SHIFT - BIO_ISSUE_SIZE_BITS) +#define BIO_ISSUE_TIME_MASK ((1ULL << BIO_ISSUE_SIZE_SHIFT) - 1) +#define BIO_ISSUE_SIZE_MASK \ + (((1ULL << BIO_ISSUE_SIZE_BITS) - 1) << BIO_ISSUE_SIZE_SHIFT) +#define BIO_ISSUE_RES_MASK (~((1ULL << BIO_ISSUE_RES_SHIFT) - 1)) + +/* Reserved bit for blk-throtl */ +#define BIO_ISSUE_THROTL_SKIP_LATENCY (1ULL << 63) + +struct bio_issue { + u64 value; +}; + +static inline u64 __bio_issue_time(u64 time) +{ + return time & BIO_ISSUE_TIME_MASK; +} + +static inline u64 bio_issue_time(struct bio_issue *issue) +{ + return __bio_issue_time(issue->value); +} + +static inline sector_t bio_issue_size(struct bio_issue *issue) +{ + return ((issue->value & BIO_ISSUE_SIZE_MASK) >> BIO_ISSUE_SIZE_SHIFT); +} + +static inline void bio_issue_init(struct bio_issue *issue, + sector_t size) +{ + size &= (1ULL << BIO_ISSUE_SIZE_BITS) - 1; + issue->value = ((issue->value & BIO_ISSUE_RES_MASK) | + (ktime_get_ns() & BIO_ISSUE_TIME_MASK) | + ((u64)size << BIO_ISSUE_SIZE_SHIFT)); +} + /* * main unit of I/O for the block layer and lower layers (ie drivers and * stacking drivers) @@ -138,7 +185,7 @@ struct bio { struct cgroup_subsys_state *bi_css; #ifdef CONFIG_BLK_DEV_THROTTLING_LOW void *bi_cg_private; - struct blk_issue_stat bi_issue_stat; + struct bio_issue bi_issue; #endif #endif union {