From patchwork Wed Apr 8 20:14:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 11480825 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 98D3792A for ; Wed, 8 Apr 2020 20:15:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 61EC720771 for ; Wed, 8 Apr 2020 20:15:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1586376902; bh=90GiN1r6+R8LM37vCtkDKSVICTgRZwiGBRzc55CK7no=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=fJgJt5s09lRo2kybwtn3BM512NI0Wc7cRKys9YgO0XFpDs1JrU0KZMeNiMZIezH+P RN0lJ5Pbz7T4rNYcKrAOETIXhDaSjtkBTRsz/i4DWTdyhMbSKkPBibUVscrDArQtRl 8AtEDbeMpU8CT2W+moe76w0kzGE/eQM93Mg0Wzug= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730295AbgDHUPB (ORCPT ); Wed, 8 Apr 2020 16:15:01 -0400 Received: from mail-qk1-f196.google.com ([209.85.222.196]:47065 "EHLO mail-qk1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728853AbgDHUPB (ORCPT ); Wed, 8 Apr 2020 16:15:01 -0400 Received: by mail-qk1-f196.google.com with SMTP id g74so1616382qke.13; Wed, 08 Apr 2020 13:14:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=alpxsM/abPDA1/mjXIVosAx6OTqAHRRnuUUJrYCfVtA=; b=JpVYHe+hBeOTw/UTMJ/u+ltNZG1zrXA1nkJhyBcw0TYcodxkSu3qqPgDAfqxTlosIP IOZ+dxch6p+vEiT1gAAFKX2EpXlObPAyCG0QxBEYhMt4SYvbFV+cHfe1JRG4jMWTk2sk r0C9oHK2xMZ2Kn9/okBz1zl2KS9edOhYjNcbbQkQkrdwIEaXD5ANmrM00fH7D4c00TyI heTWDSRMSY/k1m80XCJg4Acf/XTG1grj4E/VO/ts8l3aJj6HTuptMFb89nXCCQ0cs2/t A6e3wnW4sTqACSK3k5UVcUH9Si3hsoIjWqxF/knUGaq1jDZpXsX75zLwOU1bodlZ1HTR r7DQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=alpxsM/abPDA1/mjXIVosAx6OTqAHRRnuUUJrYCfVtA=; b=q4J0zwxiZuhCSwtW8JSLmJsmt6dzsJe6DuBlcGZ2A7UX7T5RUNKx549y1YH0kYW4AN 4fCbpYWkPk4zrgT2BIlH0tWPrbsRNbngZj/fjUOJE07Qpf8z5NtTr4i8w6fXc5FTQC25 tQzT96vrvTDOjs+TmhUN6oMwoVEqJCq4PfUuVfTMLRTbEuEBUc5RREolreU+zQIw6Fpe qxtXVP/JQdy1R2f2hS36v7Qug4+L441DpgunQaDmdzm+9N0KqToawoOXRwlFfgFWjW3D CpuIs65/m8LvB6N0G6bsWRUxNlVWYmVSoSo6BTVo65bjNfQgwGXJ/FZMBJkaKz2Pwfqi VeIw== X-Gm-Message-State: AGi0PubctY3mwVx2qJe/h06PwvJ6knE4VGb3aXyIuiqS+4S+D+aLaoEh vkZ9kgJIxC/hfnUDw+sgQ24= X-Google-Smtp-Source: APiQypL7YKWZiJ/tElbRGmVG3Z4C+0a0XxHMonMtkzDlMd48yW5WfD8YFwmz19PfSGrNOJT9l4Qegw== X-Received: by 2002:a05:620a:1084:: with SMTP id g4mr9196377qkk.52.1586376897570; Wed, 08 Apr 2020 13:14:57 -0700 (PDT) Received: from localhost ([2620:10d:c091:480::36b2]) by smtp.gmail.com with ESMTPSA id 65sm1701622qkf.79.2020.04.08.13.14.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Apr 2020 13:14:56 -0700 (PDT) From: Tejun Heo To: axboe@kernel.dk Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, kernel-team@vger.kernel.org, cgroups@vger.kernel.org, newella@fb.com, josef@toxicpanda.com, Tejun Heo Subject: [PATCH 1/5] blk-iocost: switch to fixed non-auto-decaying use_delay Date: Wed, 8 Apr 2020 16:14:46 -0400 Message-Id: <20200408201450.3959560-2-tj@kernel.org> X-Mailer: git-send-email 2.25.2 In-Reply-To: <20200408201450.3959560-1-tj@kernel.org> References: <20200408201450.3959560-1-tj@kernel.org> MIME-Version: 1.0 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org The use_delay mechanism was introduced by blk-iolatency to hold memory allocators accountable for the reclaim and other shared IOs they cause. The duration of the delay is dynamically balanced between iolatency increasing the value on each target miss and it auto-decaying as time passes and threads get delayed on it. While this works well for iolatency, iocost's control model isn't compatible with it. There is no repeated "violation" events which can be balanced against auto-decaying. iocost instead knows how much a given cgroup is over budget and wants to prevent that cgroup from issuing IOs while over budget. Until now, iocost has been adding the cost of force-issued IOs. However, this doesn't reflect the amount which is already over budget and is simply not enough to counter the auto-decaying allowing anon-memory leaking low priority cgroup to go over its alloted share of IOs. As auto-decaying doesn't make much sense for iocost, this patch introduces a different mode of operation for use_delay - when blkcg_set_delay() are used insted of blkcg_add/use_delay(), the delay duration is not auto-decayed until it is explicitly cleared with blkcg_clear_delay(). iocost is updated to keep the delay duration synchronized to the budget overage amount. With this change, iocost can effectively police cgroups which generate significant amount of force-issued IOs. Signed-off-by: Tejun Heo Cc: Josef Bacik --- block/blk-cgroup.c | 6 ++++++ block/blk-iocost.c | 23 ++++++++------------ include/linux/blk-cgroup.h | 43 +++++++++++++++++++++++++++++--------- 3 files changed, 48 insertions(+), 24 deletions(-) diff --git a/block/blk-cgroup.c b/block/blk-cgroup.c index c15a26096038..3ee9df979ff2 100644 --- a/block/blk-cgroup.c +++ b/block/blk-cgroup.c @@ -1514,6 +1514,10 @@ static void blkcg_scale_delay(struct blkcg_gq *blkg, u64 now) { u64 old = atomic64_read(&blkg->delay_start); + /* negative use_delay means no scaling, see blkcg_set_delay() */ + if (atomic_read(&blkg->use_delay) < 0) + return; + /* * We only want to scale down every second. The idea here is that we * want to delay people for min(delay_nsec, NSEC_PER_SEC) in a certain @@ -1701,6 +1705,8 @@ void blkcg_schedule_throttle(struct request_queue *q, bool use_memdelay) */ void blkcg_add_delay(struct blkcg_gq *blkg, u64 now, u64 delta) { + if (WARN_ON_ONCE(atomic_read(&blkg->use_delay) < 0)) + return; blkcg_scale_delay(blkg, now); atomic64_add(delta, &blkg->delay_nsec); } diff --git a/block/blk-iocost.c b/block/blk-iocost.c index db35ee682294..a8e99ef76a08 100644 --- a/block/blk-iocost.c +++ b/block/blk-iocost.c @@ -1209,14 +1209,14 @@ static enum hrtimer_restart iocg_waitq_timer_fn(struct hrtimer *timer) return HRTIMER_NORESTART; } -static bool iocg_kick_delay(struct ioc_gq *iocg, struct ioc_now *now, u64 cost) +static bool iocg_kick_delay(struct ioc_gq *iocg, struct ioc_now *now) { struct ioc *ioc = iocg->ioc; struct blkcg_gq *blkg = iocg_to_blkg(iocg); u64 vtime = atomic64_read(&iocg->vtime); u64 vmargin = ioc->margin_us * now->vrate; u64 margin_ns = ioc->margin_us * NSEC_PER_USEC; - u64 expires, oexpires; + u64 delta_ns, expires, oexpires; u32 hw_inuse; /* debt-adjust vtime */ @@ -1233,15 +1233,10 @@ static bool iocg_kick_delay(struct ioc_gq *iocg, struct ioc_now *now, u64 cost) return false; /* use delay */ - if (cost) { - u64 cost_ns = DIV64_U64_ROUND_UP(cost * NSEC_PER_USEC, - now->vrate); - blkcg_add_delay(blkg, now->now_ns, cost_ns); - } - blkcg_use_delay(blkg); - - expires = now->now_ns + DIV64_U64_ROUND_UP(vtime - now->vnow, - now->vrate) * NSEC_PER_USEC; + delta_ns = DIV64_U64_ROUND_UP(vtime - now->vnow, + now->vrate) * NSEC_PER_USEC; + blkcg_set_delay(blkg, delta_ns); + expires = now->now_ns + delta_ns; /* if already active and close enough, don't bother */ oexpires = ktime_to_ns(hrtimer_get_softexpires(&iocg->delay_timer)); @@ -1260,7 +1255,7 @@ static enum hrtimer_restart iocg_delay_timer_fn(struct hrtimer *timer) struct ioc_now now; ioc_now(iocg->ioc, &now); - iocg_kick_delay(iocg, &now, 0); + iocg_kick_delay(iocg, &now); return HRTIMER_NORESTART; } @@ -1378,7 +1373,7 @@ static void ioc_timer_fn(struct timer_list *timer) atomic64_read(&iocg->abs_vdebt)) { /* might be oversleeping vtime / hweight changes, kick */ iocg_kick_waitq(iocg, &now); - iocg_kick_delay(iocg, &now, 0); + iocg_kick_delay(iocg, &now); } else if (iocg_is_idle(iocg)) { /* no waiter and idle, deactivate */ iocg->last_inuse = iocg->inuse; @@ -1737,7 +1732,7 @@ static void ioc_rqos_throttle(struct rq_qos *rqos, struct bio *bio) */ if (bio_issue_as_root_blkg(bio) || fatal_signal_pending(current)) { atomic64_add(abs_cost, &iocg->abs_vdebt); - if (iocg_kick_delay(iocg, &now, cost)) + if (iocg_kick_delay(iocg, &now)) blkcg_schedule_throttle(rqos->q, (bio->bi_opf & REQ_SWAP) == REQ_SWAP); return; diff --git a/include/linux/blk-cgroup.h b/include/linux/blk-cgroup.h index e4a6949fd171..a95cd4c8a3fb 100644 --- a/include/linux/blk-cgroup.h +++ b/include/linux/blk-cgroup.h @@ -638,6 +638,8 @@ static inline bool blkcg_bio_issue_check(struct request_queue *q, static inline void blkcg_use_delay(struct blkcg_gq *blkg) { + if (WARN_ON_ONCE(atomic_read(&blkg->use_delay) < 0)) + return; if (atomic_add_return(1, &blkg->use_delay) == 1) atomic_inc(&blkg->blkcg->css.cgroup->congestion_count); } @@ -646,6 +648,8 @@ static inline int blkcg_unuse_delay(struct blkcg_gq *blkg) { int old = atomic_read(&blkg->use_delay); + if (WARN_ON_ONCE(old < 0)) + return 0; if (old == 0) return 0; @@ -670,20 +674,39 @@ static inline int blkcg_unuse_delay(struct blkcg_gq *blkg) return 1; } +/** + * blkcg_set_delay - Enable allocator delay mechanism with the specified delay amount + * @blkg: target blkg + * @delay: delay duration in nsecs + * + * When enabled with this function, the delay is not decayed and must be + * explicitly cleared with blkcg_clear_delay(). Must not be mixed with + * blkcg_[un]use_delay() and blkcg_add_delay() usages. + */ +static inline void blkcg_set_delay(struct blkcg_gq *blkg, u64 delay) +{ + int old = atomic_read(&blkg->use_delay); + + /* We only want 1 person setting the congestion count for this blkg. */ + if (!old && atomic_cmpxchg(&blkg->use_delay, old, -1) == old) + atomic_inc(&blkg->blkcg->css.cgroup->congestion_count); + + atomic64_set(&blkg->delay_nsec, delay); +} + +/** + * blkcg_clear_delay - Disable allocator delay mechanism + * @blkg: target blkg + * + * Disable use_delay mechanism. See blkcg_set_delay(). + */ static inline void blkcg_clear_delay(struct blkcg_gq *blkg) { int old = atomic_read(&blkg->use_delay); - if (!old) - return; + /* We only want 1 person clearing the congestion count for this blkg. */ - while (old) { - int cur = atomic_cmpxchg(&blkg->use_delay, old, 0); - if (cur == old) { - atomic_dec(&blkg->blkcg->css.cgroup->congestion_count); - break; - } - old = cur; - } + if (old && atomic_cmpxchg(&blkg->use_delay, old, 0) == old) + atomic_dec(&blkg->blkcg->css.cgroup->congestion_count); } void blkcg_add_delay(struct blkcg_gq *blkg, u64 now, u64 delta); From patchwork Wed Apr 8 20:14:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 11480831 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F09C31805 for ; Wed, 8 Apr 2020 20:15:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id CEFE720753 for ; Wed, 8 Apr 2020 20:15:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1586376913; bh=uGBf4NhVW/uFPCgYJ2UWlMv5dzZbtb9yrAOcmbviKoY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=GiU7QNNnsg6t/TnmYyLZ3okuEQM2KOGOc/1Mq8QbQJMf3r3P8cPWmb0MI9cVWFjHM mdeQq+x5xzcjpkenvunB3Xo6e2Ct8joOAUH1pDfL2sq98UvygWBlwnU3sKyMSlBKZw WL6KxSCy/tm3p1SLmP06PSGPIdO3f7yy8wgC5fYo= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730315AbgDHUPD (ORCPT ); Wed, 8 Apr 2020 16:15:03 -0400 Received: from mail-qt1-f194.google.com ([209.85.160.194]:41178 "EHLO mail-qt1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730302AbgDHUPC (ORCPT ); Wed, 8 Apr 2020 16:15:02 -0400 Received: by mail-qt1-f194.google.com with SMTP id i3so977726qtv.8; Wed, 08 Apr 2020 13:14:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Ih5upVjhUxfYMhdnP4A5/T8a9y3p/p63bXQ1Cpw5ptI=; b=ahdDMthtvdSscTfVMy/vrgXkEE6ZLB/M6Kqlcr4R1o5OQHWIEAP1lo7TbzBshjqpCi ijZq34NRYb1Wu/xEtN7bQcDJNh6Gv/5gE9N3nKqoQdfjvMtx9Gohf9fADM/CRUc3c/CZ 9rKw14KxaKSEjoftno62YvA0eZ0wBdQOFpRG0QiJgot3SK8khszhzpQKBdBkDxhpXX7u pXyrkGGwxgGc35KEFON+hvf2NVTqYgULNGVY6GczJRq/gohaYTmvIkP5/Jn6AD8sm5jA UFDc3fRkF6wXTWUq86zn7wMUK4PuykXM3UtoOcF6+Nw2CiBwktkCj78chNE6Gia7VXd3 2qhw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=Ih5upVjhUxfYMhdnP4A5/T8a9y3p/p63bXQ1Cpw5ptI=; b=MZWUO0RFPr9rQ8Tir/1h1QKDsfqQxuPpAMCMtpZEKE78NRhXqSWFpwaedzg82izJIu IhCgDJaBmPssG0U5brolH9H7gJ5FXqYnrIE/ET2LzmS42IZquQioKwZ4WRbhG+HTwVB+ Lwknezsd8wjCOC0ROhVJcCWWqCidyLD+gucWEZMMZgOBNPzeONH6wB5GXjx9vuhHB/we XRMQBOs/JeRecQt72nmi4eY8NgDkLBDRhDP82M5e3MHly/UPdUwiCitm8ySoAmsSzS+L U6nxlYz1Gqg7oaiFt4AKvKYPBaXm78p4kALJ2CnVX/btHXmyMdkauUykJGbJTFa6WRti 0DHQ== X-Gm-Message-State: AGi0PuZy3cxPWyNG2UpXhDHnJlEq8BLoFyR+gEMHpkrmu8DM71HhPktl id0Q196ffdjbGUyfue22scw= X-Google-Smtp-Source: APiQypJ+V51HeM9lIksTMHTJr0O0aWzVoY1i89+H2QOsGYkxE3JJ60mBLO+vpQ06kVuTddKxSWEEaw== X-Received: by 2002:aed:3981:: with SMTP id m1mr9341568qte.35.1586376899337; Wed, 08 Apr 2020 13:14:59 -0700 (PDT) Received: from localhost ([2620:10d:c091:480::36b2]) by smtp.gmail.com with ESMTPSA id l22sm20476361qkj.120.2020.04.08.13.14.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Apr 2020 13:14:58 -0700 (PDT) From: Tejun Heo To: axboe@kernel.dk Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, kernel-team@vger.kernel.org, cgroups@vger.kernel.org, newella@fb.com, josef@toxicpanda.com, Tejun Heo Subject: [PATCH 2/5] block: add request->io_data_len Date: Wed, 8 Apr 2020 16:14:47 -0400 Message-Id: <20200408201450.3959560-3-tj@kernel.org> X-Mailer: git-send-email 2.25.2 In-Reply-To: <20200408201450.3959560-1-tj@kernel.org> References: <20200408201450.3959560-1-tj@kernel.org> MIME-Version: 1.0 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Currently, at the time of completeion, there's no way of knowing how big a request was. blk-iocost will need this information to account for IO size when calculating expected latencies. This patch adds rq->io_data_len which remembers blk_rq_bytes() at the time the request gets issued. The field is enabled iff CONFIG_BLK_IO_DATA_LEN is set and doesn't increase the size of the struct even when enabled. Signed-off-by: Tejun Heo --- block/Kconfig | 3 +++ block/blk-mq.c | 6 ++++++ include/linux/blkdev.h | 8 ++++++++ 3 files changed, 17 insertions(+) diff --git a/block/Kconfig b/block/Kconfig index 3bc76bb113a0..48308e600dc8 100644 --- a/block/Kconfig +++ b/block/Kconfig @@ -26,6 +26,9 @@ menuconfig BLOCK if BLOCK +config BLK_RQ_IO_DATA_LEN + bool + config BLK_RQ_ALLOC_TIME bool diff --git a/block/blk-mq.c b/block/blk-mq.c index f6291ceedee4..64ed22712fe4 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -415,6 +415,9 @@ struct request *blk_mq_alloc_request(struct request_queue *q, unsigned int op, return ERR_PTR(-EWOULDBLOCK); rq->__data_len = 0; +#ifdef CONFIG_BLK_RQ_IO_DATA_LEN + rq->io_data_len = 0; +#endif rq->__sector = (sector_t) -1; rq->bio = rq->biotail = NULL; return rq; @@ -655,6 +658,9 @@ void blk_mq_start_request(struct request *rq) trace_block_rq_issue(q, rq); +#ifdef CONFIG_BLK_RQ_IO_DATA_LEN + rq->io_data_len = blk_rq_bytes(rq); +#endif if (test_bit(QUEUE_FLAG_STATS, &q->queue_flags)) { rq->io_start_time_ns = ktime_get_ns(); rq->stats_sectors = blk_rq_sectors(rq); diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 32868fbedc9e..bfd34c6a27ef 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -142,6 +142,14 @@ struct request { /* the following two fields are internal, NEVER access directly */ unsigned int __data_len; /* total data len */ +#ifdef CONFIG_BLK_RQ_IO_DATA_LEN + /* + * Total data len at the time of issue. This doesn't get deducted by + * blk_update_request() and can be used by completion path to determine + * the request size. + */ + unsigned int io_data_len; +#endif sector_t __sector; /* sector cursor */ struct bio *bio; From patchwork Wed Apr 8 20:14:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 11480833 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 42B0392A for ; Wed, 8 Apr 2020 20:15:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 15E45208E4 for ; Wed, 8 Apr 2020 20:15:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1586376919; bh=w747uuIg1jdrXmp0aDyB3OiMnM/0qgxwIntGycBEGOg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=qXT0ASDumn9rBZsGRMVv6+ov5CMBvWyeWIolCML0/jcudXhgy5gFBTDdWaFL5c47y MA8rFsiwjbZ7flDu8n7VwyIOPFx5tdkZCKlZGJ32dN7PXMXeMHxbWMHipxtTSyLvzi 01tK/gbMogQC1oWA5pwJWitnYPWDl9kd/hORGJfk= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730361AbgDHUPO (ORCPT ); Wed, 8 Apr 2020 16:15:14 -0400 Received: from mail-qk1-f194.google.com ([209.85.222.194]:45567 "EHLO mail-qk1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730309AbgDHUPC (ORCPT ); Wed, 8 Apr 2020 16:15:02 -0400 Received: by mail-qk1-f194.google.com with SMTP id m67so1624862qke.12; Wed, 08 Apr 2020 13:15:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=JdXZ3lJLN6mj2DFzKWWl+veWK2Db945pbS3QdWvrLY0=; b=vSHKOPOY1fZ0RWyLB4sv9rKbE91lJyvINDh9HA66upkxrCZCsFqamFz/DYSOhOgdYg rhxi9bNKEd+fNPZK7od8vDCK/+vuOvEozB6DDyDXPKN3i9608HYrOHwc9TS5SYRR+l3d ErVkebnrtKaYJY5OOYkBmWrYSxBv2fa1p+zB8dGYc77HKNJ23/vu5Miief6DgyIlWk1c Dteif6bDq7sYypOvnwPEX+FdMG/0OayCjgzzePAgz3Bi6wXWO6CBuPeAOQWTOqyQm2AZ l7AmSsojdFv3eC7jugIOdcAItndujAc3JMFc19DwwdGI6D3ZC1lI8HliuiuKc+UY7W2q LoaQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=JdXZ3lJLN6mj2DFzKWWl+veWK2Db945pbS3QdWvrLY0=; b=ZMIdiRDhVlVhdl1keSalvdCKGlKuUE+oXG1lvPH3cN+YKhR1NgDtOvFLRn7TaYprQA 5a1RzB+Bk1kHc+C0d41sigQnRNtNPhOFPQ40G4ojuCp5ESsj0v+uNIaaA4PIzDpmUgwn l6HmdnmelO0u5xkA86JxCHfPdtOk2axG2nswZDO8JZldqKmvDGS+5ZXxEvqJ4ojI7j6P lYqDfjAph6N3GjNexfIfSZVjIdFcgF5ScZTETmQarGDzA9h/CZ3z/UZ0Orzhg4soL6/a QvHkMEevY3Tnlm64Bi5/JqUq71q0LceBXgNSBbtduDNurUjOmYsx9T28ef2m2K+pnweF r1MQ== X-Gm-Message-State: AGi0PuZjoyY2XuXoKImdqI7CKZ7MWx91qJiiM2G1vJwjdQ79Seg+sXmd ZsiBT6/AsK3aR8fLKWNLtrrAc1qO9BI= X-Google-Smtp-Source: APiQypJ/jtm3TeWXnX5UBgw/Ym2V/O1SmHKO9XstdQMbbCDR0m1fA2SbLHvoG/TnqtB90OV+DDwCNw== X-Received: by 2002:a37:a4c3:: with SMTP id n186mr1612922qke.62.1586376901312; Wed, 08 Apr 2020 13:15:01 -0700 (PDT) Received: from localhost ([2620:10d:c091:480::36b2]) by smtp.gmail.com with ESMTPSA id z11sm19391343qti.23.2020.04.08.13.15.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Apr 2020 13:15:00 -0700 (PDT) From: Tejun Heo To: axboe@kernel.dk Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, kernel-team@vger.kernel.org, cgroups@vger.kernel.org, newella@fb.com, josef@toxicpanda.com, Tejun Heo Subject: [PATCH 3/5] blk-iocost: account for IO size when testing latencies Date: Wed, 8 Apr 2020 16:14:48 -0400 Message-Id: <20200408201450.3959560-4-tj@kernel.org> X-Mailer: git-send-email 2.25.2 In-Reply-To: <20200408201450.3959560-1-tj@kernel.org> References: <20200408201450.3959560-1-tj@kernel.org> MIME-Version: 1.0 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org On each IO completion, iocost decides whether the IO met or missed its latency target. Currently, the targets are fixed numbers per IO type. While this can be good enough for loose latency targets way higher than typical completion latencies, the effect of IO size makes it difficult to tighten the latency target - a target adequate for 4k IOs might be too tight for 512k IOs and vice-versa. iocost already has all the necessary information to account for different IO sizes when testing whether the latency target is met as iocost can calculate the size vtime cost of a given IO. This patch updates the completion path to calculate the size vtime cost of the IO, deduct the nsec equivalent from the observed latency and use the adjusted value to decide whether the target is met. This makes latency targets independent from IO size and enables determining adequate latency targets with fixed size fio runs. Signed-off-by: Tejun Heo Cc: Andy Newell --- block/Kconfig | 1 + block/blk-iocost.c | 31 +++++++++++++++++++++++++++++-- 2 files changed, 30 insertions(+), 2 deletions(-) diff --git a/block/Kconfig b/block/Kconfig index 48308e600dc8..3b0b698ca254 100644 --- a/block/Kconfig +++ b/block/Kconfig @@ -149,6 +149,7 @@ config BLK_CGROUP_IOLATENCY config BLK_CGROUP_IOCOST bool "Enable support for cost model based cgroup IO controller" depends on BLK_CGROUP=y + select BLK_RQ_IO_DATA_LEN select BLK_RQ_ALLOC_TIME ---help--- Enabling this option enables the .weight interface for cost diff --git a/block/blk-iocost.c b/block/blk-iocost.c index a8e99ef76a08..76e8738f7bb5 100644 --- a/block/blk-iocost.c +++ b/block/blk-iocost.c @@ -260,6 +260,7 @@ enum { VTIME_PER_SEC_SHIFT = 37, VTIME_PER_SEC = 1LLU << VTIME_PER_SEC_SHIFT, VTIME_PER_USEC = VTIME_PER_SEC / USEC_PER_SEC, + VTIME_PER_NSEC = VTIME_PER_SEC / NSEC_PER_SEC, /* bound vrate adjustments within two orders of magnitude */ VRATE_MIN_PPM = 10000, /* 1% */ @@ -1668,6 +1669,30 @@ static u64 calc_vtime_cost(struct bio *bio, struct ioc_gq *iocg, bool is_merge) return cost; } +static void calc_size_vtime_cost_builtin(struct request *rq, struct ioc *ioc, u64 *costp) +{ + u64 pages = max_t(u64, rq->io_data_len >> IOC_PAGE_SHIFT, 1); + + switch (req_op(rq)) { + case REQ_OP_READ: + *costp = pages * ioc->params.lcoefs[LCOEF_RPAGE]; + break; + case REQ_OP_WRITE: + *costp = pages * ioc->params.lcoefs[LCOEF_WPAGE]; + break; + default: + *costp = 0; + } +} + +static u64 calc_size_vtime_cost(struct request *rq, struct ioc *ioc) +{ + u64 cost; + + calc_size_vtime_cost_builtin(rq, ioc, &cost); + return cost; +} + static void ioc_rqos_throttle(struct rq_qos *rqos, struct bio *bio) { struct blkcg_gq *blkg = bio->bi_blkg; @@ -1837,7 +1862,7 @@ static void ioc_rqos_done_bio(struct rq_qos *rqos, struct bio *bio) static void ioc_rqos_done(struct rq_qos *rqos, struct request *rq) { struct ioc *ioc = rqos_to_ioc(rqos); - u64 on_q_ns, rq_wait_ns; + u64 on_q_ns, rq_wait_ns, size_nsec; int pidx, rw; if (!ioc->enabled || !rq->alloc_time_ns || !rq->start_time_ns) @@ -1858,8 +1883,10 @@ static void ioc_rqos_done(struct rq_qos *rqos, struct request *rq) on_q_ns = ktime_get_ns() - rq->alloc_time_ns; rq_wait_ns = rq->start_time_ns - rq->alloc_time_ns; + size_nsec = div64_u64(calc_size_vtime_cost(rq, ioc), VTIME_PER_NSEC); - if (on_q_ns <= ioc->params.qos[pidx] * NSEC_PER_USEC) + if (on_q_ns <= size_nsec || + on_q_ns - size_nsec <= ioc->params.qos[pidx] * NSEC_PER_USEC) this_cpu_inc(ioc->pcpu_stat->missed[rw].nr_met); else this_cpu_inc(ioc->pcpu_stat->missed[rw].nr_missed); From patchwork Wed Apr 8 20:14:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 11480829 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 51D7515AB for ; Wed, 8 Apr 2020 20:15:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 2EDB320784 for ; Wed, 8 Apr 2020 20:15:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1586376913; bh=8VmrQ6jz3iEwcnir3jeM3aawg7hgug8AzuR6T+WgcgI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=MhkX1ThaJZedESe6ToTO9eaE3slBHbSvTRJBghuNHKEb9AbEfvIIl7Z7OzeV5Qqo0 69ovK/pBQvyMFa/6kJ4FdqSKqZhkzbA5zz/Xxp/8XVaitKLWiC4RoNOb0Bxru7JQWy RJnfR+IVgtdKw4LW/9Os5L/+vTj2ebMKvg4qbj1Y= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730327AbgDHUPF (ORCPT ); Wed, 8 Apr 2020 16:15:05 -0400 Received: from mail-qk1-f194.google.com ([209.85.222.194]:35389 "EHLO mail-qk1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730322AbgDHUPE (ORCPT ); Wed, 8 Apr 2020 16:15:04 -0400 Received: by mail-qk1-f194.google.com with SMTP id c63so1694922qke.2; Wed, 08 Apr 2020 13:15:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=vrEanY8wxDtok+RTWBGcNDaSQ4QB8ANJ03cLIKdV4tg=; b=ex4iZHeH4DOVQuYBmfQMJJYYtzF1WGnnyQk5oSf5kQrIww5/9iaUeRu+xrFcO35QBm uTEZUmfC8pKrLoa6CIk++HA89Fr1LmqClmiDzN9WbnfFagMGE44s/rkrv9pp8Ez3VOaL rYDsmvOpo6L1AUjud8ciac302OSNuwwlHqWUmgAYPehNYqfBKvjNUkRh79ZIcW0cr/xP HGSpOuYozYntvjo/7Wq4Pa/GI11OHsXmIQa2eLRaJ23D16kgN/GmYDGXACIPyUzyQsme mWTyM2ehCqgLfRKNhKnyQWWwhUhCsgWwlUjpZX6XpYvz3DKyIhCmpO6sSDNjZB6RGAb5 HCVg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=vrEanY8wxDtok+RTWBGcNDaSQ4QB8ANJ03cLIKdV4tg=; b=a+r3F/um07AK3YKhnYp1oG5qN7gysGh/T/j67EZ2m2jkTTn9idiaxZy4TUa4XCsw3e 4KAZts5WYT7iC+m0Jb//afxUunugV2M2Vl0y61IXymWSm42eQlGyJO3zMDxwmrtzFNIr EmBNC9UnQaUEQ8Ch2V6KGz9FTCB6oIA3mpniWl3ycZnBxc4Yf56QbX46imt9hZjs4//f xiivSj87e4L5QdRezOlrzetLvLJBgrBpMuImUYu3oTTCsfT/3OZ1BpwXYwdCIfiQze0X ISaBQAT7+ytWtFI6bMWZqwGCvglZcj2fbX7XgNCmX0lR8nDRx2u48iIdle97WCU43XsJ zbhg== X-Gm-Message-State: AGi0PuYgEEToing1P6E+BperhhZuHU/pbIrI228aRPwxyGxKjXKFDyK7 Z4bTVLEHFioS7fvWNG+g+Zs= X-Google-Smtp-Source: APiQypJafVvTukLL2cplRfE6mMKqbxOimy4N5GBauz/EWg7/lRjK1fmuOArhy09GhZPxWY4iS0SmMw== X-Received: by 2002:a37:d0a:: with SMTP id 10mr9249174qkn.288.1586376903087; Wed, 08 Apr 2020 13:15:03 -0700 (PDT) Received: from localhost ([2620:10d:c091:480::36b2]) by smtp.gmail.com with ESMTPSA id o128sm14651009qkf.116.2020.04.08.13.15.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Apr 2020 13:15:02 -0700 (PDT) From: Tejun Heo To: axboe@kernel.dk Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, kernel-team@vger.kernel.org, cgroups@vger.kernel.org, newella@fb.com, josef@toxicpanda.com, Tejun Heo Subject: [PATCH 4/5] iocost_monitor: exit successfully if interval is zero Date: Wed, 8 Apr 2020 16:14:49 -0400 Message-Id: <20200408201450.3959560-5-tj@kernel.org> X-Mailer: git-send-email 2.25.2 In-Reply-To: <20200408201450.3959560-1-tj@kernel.org> References: <20200408201450.3959560-1-tj@kernel.org> MIME-Version: 1.0 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org This is to help external tools to decide whether iocost_monitor has all its requirements met or not based on the exit status of an -i0 run. Signed-off-by: Tejun Heo --- tools/cgroup/iocost_monitor.py | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/tools/cgroup/iocost_monitor.py b/tools/cgroup/iocost_monitor.py index 7427a5ee761b..eb2363b868c5 100644 --- a/tools/cgroup/iocost_monitor.py +++ b/tools/cgroup/iocost_monitor.py @@ -28,7 +28,8 @@ parser.add_argument('devname', metavar='DEV', parser.add_argument('--cgroup', action='append', metavar='REGEX', help='Regex for target cgroups, ') parser.add_argument('--interval', '-i', metavar='SECONDS', type=float, default=1, - help='Monitoring interval in seconds') + help='Monitoring interval in seconds (0 exits immediately ' + 'after checking requirements)') parser.add_argument('--json', action='store_true', help='Output in json') args = parser.parse_args() @@ -243,6 +244,9 @@ ioc = None if ioc is None: err(f'Could not find ioc for {devname}'); +if interval == 0: + sys.exit(0) + # Keep printing while True: now = time.time() From patchwork Wed Apr 8 20:14:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 11480827 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B1AB492A for ; Wed, 8 Apr 2020 20:15:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 8564F20784 for ; Wed, 8 Apr 2020 20:15:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1586376911; bh=tq7HfA9O4mPilqeaDrt2JRpmzIf/oyv7/XuxMyJ6uQM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=DAQ8OIHA5bKvhDkRYvYc5sxBDzrNsNapMU8B4+CeHdcXEAlYlavAP633itC3G1Gii M8B3OP01IK1vXmYDRAXcJs76XvSVDAQmxl8JwyZEfGzOxYYtph9APwb8DcrlItmB++ uH1cID/k4qDnvfpAC4n7/EQ5oQ5BDUm4NCT7Iv+M= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730342AbgDHUPH (ORCPT ); Wed, 8 Apr 2020 16:15:07 -0400 Received: from mail-qk1-f196.google.com ([209.85.222.196]:34808 "EHLO mail-qk1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730331AbgDHUPG (ORCPT ); Wed, 8 Apr 2020 16:15:06 -0400 Received: by mail-qk1-f196.google.com with SMTP id i186so1708434qke.1; Wed, 08 Apr 2020 13:15:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=+BQj1JBsSqI5aXq/pqViTsj/4QvWCDvVD92VTzp4arE=; b=qKboPNy2/dnjhxkRbRhKQPht9hywFpKyf4i8CXsAR9zg0nhM5XrScSCIvpHz7F0eji 41v6ZqceIMREH0c+nwCHi/waR3Ax0Z7sTe6ayepIgXW86yw+orBD/g82g9KzmM/mt7Yj m0ZW8Z3YolR7v0+IcLE06O2273fnL6vM0Hx/8bihQL47lmFAqRRjUELts2RQ99MA9z0/ bIcqmVBOHyjQcf56HDHd/4w3zJzm29ZiAxwDKFbiPTJCKLXgWyTAraN3q7jQekCPAror cvCUcZxY2v+v7Vi4wuY70fic7qGswgC1yMzn9eLdmP4ah89UPOUE/XZU96WuffzSkn9E tn9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=+BQj1JBsSqI5aXq/pqViTsj/4QvWCDvVD92VTzp4arE=; b=CHWv0VuXLcqPAx/sQsStGLMvkh+ddP6aVK9iR9G/isMcnWWKXhwdPQ0jIODyjJPy/t F40JefQWFSWXwnxZH56XDwNybPgYeAIgM9BE65uZk0dSf79ft6HEFgfr9OEyClWzG2Uw 0PrSpaCHrh2okAO4hUbbECyl8vFw9I822jfH4J1ifv/95We9XvAgc2do7LtlaSD6S0E4 o0PIOU7C5vCJpSMXCQP+Sv4vPHZD9mJdihd2K+58ajXUVzn48fUwvjdIGllK2yPgjdCH gR3HZhA11m8RaumyD9kXNXxhLmga9MRxgPdrZniAVUIIZJpN5UUtDpRaRjbFpaTrL4Jf 5SpA== X-Gm-Message-State: AGi0PubOur88+VrWAZF8T+SDtZtvgurMsjFASBjdlg8tpbXx0Mro/se5 TzP6ziBv0gfEg0FxC8vfLls= X-Google-Smtp-Source: APiQypJSKn0E6p5g9aODEGUakCg+0fdp0wYkfpYJVtzATTSKzyku2CbfjyLqyEwrUo39mE8s2Qx5VA== X-Received: by 2002:a37:a5c2:: with SMTP id o185mr9299768qke.219.1586376904902; Wed, 08 Apr 2020 13:15:04 -0700 (PDT) Received: from localhost ([2620:10d:c091:480::36b2]) by smtp.gmail.com with ESMTPSA id v33sm10416277qtd.88.2020.04.08.13.15.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Apr 2020 13:15:04 -0700 (PDT) From: Tejun Heo To: axboe@kernel.dk Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, kernel-team@vger.kernel.org, cgroups@vger.kernel.org, newella@fb.com, josef@toxicpanda.com, Tejun Heo Subject: [PATCH 5/5] iocost_monitor: drop string wrap around numbers when outputting json Date: Wed, 8 Apr 2020 16:14:50 -0400 Message-Id: <20200408201450.3959560-6-tj@kernel.org> X-Mailer: git-send-email 2.25.2 In-Reply-To: <20200408201450.3959560-1-tj@kernel.org> References: <20200408201450.3959560-1-tj@kernel.org> MIME-Version: 1.0 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Wrapping numbers in strings is used by some to work around bit-width issues in some enviroments. The problem isn't innate to json and the workaround seems to cause more integration problems than help. Let's drop the string wrapping. Signed-off-by: Tejun Heo --- tools/cgroup/iocost_monitor.py | 42 +++++++++++++++++----------------- 1 file changed, 21 insertions(+), 21 deletions(-) diff --git a/tools/cgroup/iocost_monitor.py b/tools/cgroup/iocost_monitor.py index eb2363b868c5..188b3379b9a1 100644 --- a/tools/cgroup/iocost_monitor.py +++ b/tools/cgroup/iocost_monitor.py @@ -113,14 +113,14 @@ autop_names = { def dict(self, now): return { 'device' : devname, - 'timestamp' : str(now), - 'enabled' : str(int(self.enabled)), - 'running' : str(int(self.running)), - 'period_ms' : str(self.period_ms), - 'period_at' : str(self.period_at), - 'period_vtime_at' : str(self.vperiod_at), - 'busy_level' : str(self.busy_level), - 'vrate_pct' : str(self.vrate_pct), } + 'timestamp' : now, + 'enabled' : self.enabled, + 'running' : self.running, + 'period_ms' : self.period_ms, + 'period_at' : self.period_at, + 'period_vtime_at' : self.vperiod_at, + 'busy_level' : self.busy_level, + 'vrate_pct' : self.vrate_pct, } def table_preamble_str(self): state = ('RUN' if self.running else 'IDLE') if self.enabled else 'OFF' @@ -175,19 +175,19 @@ autop_names = { def dict(self, now, path): out = { 'cgroup' : path, - 'timestamp' : str(now), - 'is_active' : str(int(self.is_active)), - 'weight' : str(self.weight), - 'weight_active' : str(self.active), - 'weight_inuse' : str(self.inuse), - 'hweight_active_pct' : str(self.hwa_pct), - 'hweight_inuse_pct' : str(self.hwi_pct), - 'inflight_pct' : str(self.inflight_pct), - 'debt_ms' : str(self.debt_ms), - 'use_delay' : str(self.use_delay), - 'delay_ms' : str(self.delay_ms), - 'usage_pct' : str(self.usage), - 'address' : str(hex(self.address)) } + 'timestamp' : now, + 'is_active' : self.is_active, + 'weight' : self.weight, + 'weight_active' : self.active, + 'weight_inuse' : self.inuse, + 'hweight_active_pct' : self.hwa_pct, + 'hweight_inuse_pct' : self.hwi_pct, + 'inflight_pct' : self.inflight_pct, + 'debt_ms' : self.debt_ms, + 'use_delay' : self.use_delay, + 'delay_ms' : self.delay_ms, + 'usage_pct' : self.usage, + 'address' : self.address } for i in range(len(self.usages)): out[f'usage_pct_{i}'] = str(self.usages[i]) return out