From patchwork Fri Sep 18 00:44:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 11783767 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B35BF6CB for ; Fri, 18 Sep 2020 00:45:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9A3C1208C3 for ; Fri, 18 Sep 2020 00:45:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1600389912; bh=G8+eX5VSAkNzWOrJJqAtXLoM8BtC4FDnvPq6T4Aa3vM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=jouk9YdTu4RNfCu5Qgn596ytEdK35PZrdrCkaE7Qz/ElL7nUcDU2rx9DuYQw8JAka QtDxigUDpZV8PHJeuY/kYYbHF5ha7q/kderow8IpHw0jDA1QLbzmTftR/CbgLwM/xY 1efI58Jy2zvHbmWMayHIK0iFVQUJSHoNAKNGJy6c= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726201AbgIRApE (ORCPT ); Thu, 17 Sep 2020 20:45:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41316 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726007AbgIRApD (ORCPT ); Thu, 17 Sep 2020 20:45:03 -0400 Received: from mail-qt1-x843.google.com (mail-qt1-x843.google.com [IPv6:2607:f8b0:4864:20::843]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0AB33C06174A; Thu, 17 Sep 2020 17:45:03 -0700 (PDT) Received: by mail-qt1-x843.google.com with SMTP id h6so3591687qtd.6; Thu, 17 Sep 2020 17:45:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=fpm0/fWCrM1vVyvEQSMysbCeUT3o5wyc4wLO+mMphzI=; b=EAkqxPk9U1A/FBgCCtf77AqQVFTNOOZPp+CyRxx9IQl/l8Lmbav99naCkjEybqG8Ra 4KcUi/0Zpq+8+NM6rqcx1BqCqJjbUF80x8X6Vk5Z7NghUcAPxutVuKp1TL7E4xFsOWwy 8jIMGhFSKalQU8ky4mSciIceYca1kGVrxHAdhAN2kNYdlxTGVrBMkIVBstZcbLRZ4ddk pFBnqqMN3JyR3VDod6GBpM3ELjWtb0mUs/KPcPE4i5TqQzxOdebikE0/SWX4q/4XhOSf VHD3Yb5sg4upVKv0QPQs+2Fr7iMSwd6Z4XfMqjGwb/3GwCCgs4XK3oKLivk+x6mGyV8V 7Rcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=fpm0/fWCrM1vVyvEQSMysbCeUT3o5wyc4wLO+mMphzI=; b=UxHPX4C9TkjPxnqDqkr7MYIMMLBc31dsLYg3mWWo9gdt+QQx0IGScqbfUw+oZz39F2 P7tkjm4MqIEmwXGzRd9TUPknSNXSfHfeKEtMRwGR3q/R7DIN565Ry/lAZOROHNYZDcwu /rQ4Jp1aECxZ3yhdN0SJ5zKBxiCgoJw7U5tSt+BKPP4daYtdYuGrP6YCIgwh5wzIeRM4 7xc2j7rP5erZt/tEDG7zfREtqak7hF7I0XxMWMeJqX5xf3B6w2J1H0x1rBIUH5S8l66n UC9Ofbc3mA2Qp3CfSvlca8niZXykbHMdzQzGKSk12ICmovVsLfn3PLRWNI2zEvyhQw0y ngBw== X-Gm-Message-State: AOAM531LwnPLvZOE79QeBNOW1Ak6h703CNtjraREyfiIq7nIbYds6p74 5VLVUge6oqLmYs6bm9+fjN2dHLQHdIJfFA== X-Google-Smtp-Source: ABdhPJxFr1BGHZEuXI816xZfhdR3MHSYta9ZEh10uPfzJvn1z1mOKNoOF9aIPiLwW3Hv8PiCsdzIwA== X-Received: by 2002:ac8:4d01:: with SMTP id w1mr31942466qtv.357.1600389902084; Thu, 17 Sep 2020 17:45:02 -0700 (PDT) Received: from localhost ([2620:10d:c091:480::1:6893]) by smtp.gmail.com with ESMTPSA id f189sm1032980qkd.20.2020.09.17.17.45.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 17 Sep 2020 17:45:01 -0700 (PDT) From: Tejun Heo To: axboe@kernel.dk Cc: linux-block@vger.kernel.org, cgroups@vger.kernel.org, kernel-team@fb.com, linux-kernel@vger.kernel.org, Tejun Heo Subject: [PATCH 1/5] iocost: factor out ioc_forgive_debts() Date: Thu, 17 Sep 2020 20:44:52 -0400 Message-Id: <20200918004456.593983-2-tj@kernel.org> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200918004456.593983-1-tj@kernel.org> References: <20200918004456.593983-1-tj@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Debt reduction logic is going to be improved and expanded. Factor it out into ioc_forgive_debts() and generalize the comment a bit. No functional change. Signed-off-by: Tejun Heo --- block/blk-iocost.c | 66 ++++++++++++++++++++++++---------------------- 1 file changed, 35 insertions(+), 31 deletions(-) diff --git a/block/blk-iocost.c b/block/blk-iocost.c index ef9476fca1d8..bbf30bb06c07 100644 --- a/block/blk-iocost.c +++ b/block/blk-iocost.c @@ -1979,6 +1979,40 @@ static void transfer_surpluses(struct list_head *surpluses, struct ioc_now *now) list_del_init(&iocg->walk_list); } +/* + * A low weight iocg can amass a large amount of debt, for example, when + * anonymous memory gets reclaimed aggressively. If the system has a lot of + * memory paired with a slow IO device, the debt can span multiple seconds or + * more. If there are no other subsequent IO issuers, the in-debt iocg may end + * up blocked paying its debt while the IO device is idle. + * + * The following protects against such cases. If the device has been + * sufficiently idle for a while, the debts are halved. + */ +static void ioc_forgive_debts(struct ioc *ioc, u64 usage_us_sum, int nr_debtors, + int nr_shortages, struct ioc_now *now) +{ + if (nr_shortages || + div64_u64(100 * usage_us_sum, now->now - ioc->period_at) >= + DEBT_BUSY_USAGE_PCT) + ioc->debt_busy_at = now->now; + + if (nr_debtors && + now->now - ioc->debt_busy_at >= DEBT_REDUCTION_IDLE_DUR) { + struct ioc_gq *iocg; + + list_for_each_entry(iocg, &ioc->active_iocgs, active_list) { + if (iocg->abs_vdebt) { + spin_lock(&iocg->waitq.lock); + iocg->abs_vdebt /= 2; + iocg_kick_waitq(iocg, true, now); + spin_unlock(&iocg->waitq.lock); + } + } + ioc->debt_busy_at = now->now; + } +} + static void ioc_timer_fn(struct timer_list *timer) { struct ioc *ioc = container_of(timer, struct ioc, timer); @@ -2171,37 +2205,7 @@ static void ioc_timer_fn(struct timer_list *timer) list_for_each_entry_safe(iocg, tiocg, &surpluses, surplus_list) list_del_init(&iocg->surplus_list); - /* - * A low weight iocg can amass a large amount of debt, for example, when - * anonymous memory gets reclaimed aggressively. If the system has a lot - * of memory paired with a slow IO device, the debt can span multiple - * seconds or more. If there are no other subsequent IO issuers, the - * in-debt iocg may end up blocked paying its debt while the IO device - * is idle. - * - * The following protects against such pathological cases. If the device - * has been sufficiently idle for a substantial amount of time, the - * debts are halved. The criteria are on the conservative side as we - * want to resolve the rare extreme cases without impacting regular - * operation by forgiving debts too readily. - */ - if (nr_shortages || - div64_u64(100 * usage_us_sum, now.now - ioc->period_at) >= - DEBT_BUSY_USAGE_PCT) - ioc->debt_busy_at = now.now; - - if (nr_debtors && - now.now - ioc->debt_busy_at >= DEBT_REDUCTION_IDLE_DUR) { - list_for_each_entry(iocg, &ioc->active_iocgs, active_list) { - if (iocg->abs_vdebt) { - spin_lock(&iocg->waitq.lock); - iocg->abs_vdebt /= 2; - iocg_kick_waitq(iocg, true, &now); - spin_unlock(&iocg->waitq.lock); - } - } - ioc->debt_busy_at = now.now; - } + ioc_forgive_debts(ioc, usage_us_sum, nr_debtors, nr_shortages, &now); /* * If q is getting clogged or we're missing too much, we're issuing From patchwork Fri Sep 18 00:44:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 11783765 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 659D16CB for ; Fri, 18 Sep 2020 00:45:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4CCE5208C3 for ; Fri, 18 Sep 2020 00:45:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1600389910; bh=gcI9x8qnm7aJusu+RAjLffERO7PHORt+Dc+DwTrhx/U=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=LitAX+M5o2CAEyswsHJNr4cIVZEfAwvRj2UtmKp5RTh20zkHEL4MdBk7PtSCq8wIN 4rs84utzmhubdCJFAuGy0uGcP9vX5pRRc8Sb4R395lPw3oW86s7QMGs+5tPkxuYQ3l aaUrNqy7TCOylSLF5ddaznACzf+ChfxeLtDyRRvw= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726245AbgIRApF (ORCPT ); Thu, 17 Sep 2020 20:45:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41322 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726007AbgIRApF (ORCPT ); Thu, 17 Sep 2020 20:45:05 -0400 Received: from mail-qk1-x744.google.com (mail-qk1-x744.google.com [IPv6:2607:f8b0:4864:20::744]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1DE42C06174A; Thu, 17 Sep 2020 17:45:05 -0700 (PDT) Received: by mail-qk1-x744.google.com with SMTP id w12so4366434qki.6; Thu, 17 Sep 2020 17:45:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=j/zKXBEr6TfdKAVuGNBQ0JBMHe4IOIVvKgYa0IBCbeM=; b=V+eukaeBQ9EymY1UpurIpKChUF9k/7A5HpI5/TyACkajuXTHx7jxGZYyFFS4HHakSz nZTlahmQZU2BrlkbxdryiyrNSNTZYN4Q3cS8mMllkeSjCt7+ygWZm+WBsRa96E22GKe5 xwSeNq6ftFaRDQL7gHjJLdFHKBTbIBQVfGKbckFdEYSDxBglwYcWuQKTeYHWD088XX0S gMX2DCEaHHGym14WixlbHcyFItlTD+EccCdxeRR0xJrR0LL6iCVD/CPf6FbLcnIIm7sN FGQH5TccRjAysF7Tn9HalVYbLkFf/Mp6HIuH1N2SbQevxwTI9QZPdSBR/24ktpX2WpAk SkPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=j/zKXBEr6TfdKAVuGNBQ0JBMHe4IOIVvKgYa0IBCbeM=; b=YhSGKQ91xp8v2EKIywIbasQYArAtkXEYnzvMwX+ipCckuTZQFAOdLn+Q9C92vCr8MT EGAaC2wo/uRG5uwD/Ujh7u6vZACt2GTZYiiBgaKG1mZ5K9SXeO37P/sv4BeUSFkbCwpx rCaRzuMRj/Lxwhoa31q99DWKdJz1cIY13sNSVP/ZbfNVRdHkgYUwSTR/MU2E4ul/MxW8 NglweU7R0w0+ZLaIh6CPbjSALa9cMaVgzJVV5qhcW5OH6Kuc0dmeMGbJAkaLrh+BGARw rj98krtKY2vQEdmEtj/XyBSH2X6QFL8FoMMw/eOwQyJIF6//iZtG3elDWjE8W+Kka7Aq 6Ugg== X-Gm-Message-State: AOAM533+ksn1rUl3itBRXC/A+8bEtPCaZp5CC0dLSz0dAJjMEQO0GCua A7ghUvu7KgwkI7f4l4kNlpE= X-Google-Smtp-Source: ABdhPJx9F4osQFUKiKrsAy9zgGKkumTulERLM/MfTr25TsVyTtPN0lvPXXRmQ2S5tVuJqsJqx7wr0A== X-Received: by 2002:a37:52c1:: with SMTP id g184mr8650621qkb.425.1600389904204; Thu, 17 Sep 2020 17:45:04 -0700 (PDT) Received: from localhost ([2620:10d:c091:480::1:6893]) by smtp.gmail.com with ESMTPSA id 85sm1032669qkn.64.2020.09.17.17.45.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 17 Sep 2020 17:45:03 -0700 (PDT) From: Tejun Heo To: axboe@kernel.dk Cc: linux-block@vger.kernel.org, cgroups@vger.kernel.org, kernel-team@fb.com, linux-kernel@vger.kernel.org, Tejun Heo Subject: [PATCH 2/5] iocost: replace nr_shortages cond in ioc_forgive_debts() with busy_level one Date: Thu, 17 Sep 2020 20:44:53 -0400 Message-Id: <20200918004456.593983-3-tj@kernel.org> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200918004456.593983-1-tj@kernel.org> References: <20200918004456.593983-1-tj@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Debt reduction was blocked if any iocg was short on budget in the past period to avoid reducing debts while some iocgs are saturated. However, this ends up unnecessarily blocking debt reduction due to temporary local imbalances when the device is generally being underutilized, while also failing to block when the underlying device is overwhelmed and the usage becomes low from high latency. Given that debt accumulation mostly happens with swapout bursts which can significantly deteriorate the underlying device's latency response, the current logic is not great. Let's replace it with ioc->busy_level based condition so that we block debt reduction when the underlying device is being saturated. ioc_forgive_debts() call is moved after busy_level determination. Signed-off-by: Tejun Heo --- block/blk-iocost.c | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/block/blk-iocost.c b/block/blk-iocost.c index bbf30bb06c07..c0499c294da9 100644 --- a/block/blk-iocost.c +++ b/block/blk-iocost.c @@ -287,10 +287,7 @@ enum { MIN_DELAY = 250, MAX_DELAY = 250 * USEC_PER_MSEC, - /* - * Halve debts if total usage keeps staying under 25% w/o any shortages - * for over 100ms. - */ + /* halve debts if total usage keeps staying under 25% for over 100ms */ DEBT_BUSY_USAGE_PCT = 25, DEBT_REDUCTION_IDLE_DUR = 100 * USEC_PER_MSEC, @@ -1990,9 +1987,9 @@ static void transfer_surpluses(struct list_head *surpluses, struct ioc_now *now) * sufficiently idle for a while, the debts are halved. */ static void ioc_forgive_debts(struct ioc *ioc, u64 usage_us_sum, int nr_debtors, - int nr_shortages, struct ioc_now *now) + struct ioc_now *now) { - if (nr_shortages || + if (ioc->busy_level < 0 || div64_u64(100 * usage_us_sum, now->now - ioc->period_at) >= DEBT_BUSY_USAGE_PCT) ioc->debt_busy_at = now->now; @@ -2205,8 +2202,6 @@ static void ioc_timer_fn(struct timer_list *timer) list_for_each_entry_safe(iocg, tiocg, &surpluses, surplus_list) list_del_init(&iocg->surplus_list); - ioc_forgive_debts(ioc, usage_us_sum, nr_debtors, nr_shortages, &now); - /* * If q is getting clogged or we're missing too much, we're issuing * too much IO and should lower vtime rate. If we're not missing @@ -2301,6 +2296,8 @@ static void ioc_timer_fn(struct timer_list *timer) ioc_refresh_params(ioc, false); + ioc_forgive_debts(ioc, usage_us_sum, nr_debtors, &now); + /* * This period is done. Move onto the next one. If nothing's * going on with the device, stop the timer. From patchwork Fri Sep 18 00:44:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 11783773 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1105A6CB for ; Fri, 18 Sep 2020 00:45:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EBFE121734 for ; Fri, 18 Sep 2020 00:45:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1600389928; bh=dEiyggvyGcVDW49PH0Zhalut3L6FboOKzrU2Ldfjoqk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=PbiJGgQ9BJT3XrsoQaALEbqVcKDNkVpk4jjJBTEh+A1oUiwe09NS4tOBBS8CIsTm4 JE+j/uznqj+rV3OICYUmXETojGq1gPnOoTyPDu72o2kyNB1oLdxNQvNbHHjzGeeIq+ k1dVNh9SOa/25CEdjlTMAPXp5SWTZ8oHHhxAERjc= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726301AbgIRApN (ORCPT ); Thu, 17 Sep 2020 20:45:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41330 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726267AbgIRApH (ORCPT ); Thu, 17 Sep 2020 20:45:07 -0400 Received: from mail-qk1-x742.google.com (mail-qk1-x742.google.com [IPv6:2607:f8b0:4864:20::742]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2A1D5C06174A; Thu, 17 Sep 2020 17:45:07 -0700 (PDT) Received: by mail-qk1-x742.google.com with SMTP id q63so4376144qkf.3; Thu, 17 Sep 2020 17:45:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ESbogjuwusL6bG2HVAbKfWJT6Gw8rmatSqGwZlkX8AM=; b=KqiKjo8ece3zoqRHEClfRRBiJtjRdILsq3RELTwgDJTBWNqze7+GKFNAm9neqmm+OQ 52HxffH349WPSwZiaNBwfEyEsEamhZjpUqxpN3ieLKeofn5LOBX6kxj0OKSY3m1LIbpd qFra6bKLan2OQC7JKa8wFCMbLb6T7x+36jvPnVfrzB6lP6q8/v9iqUYTrVZN9CQQY2Z+ VEYXy9ugIPOGpmri/WTCKNgpvxYnK13V3F3GNtS86ZIS32JadBE4xJ1xqgMvvDpzqKbj b6EV/XPGPg8QCjW6+EX6jQRjJzQTTGGnngfemHP5O59D/yLoYAVvpv6g5C0C7juRoq87 Sxhw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=ESbogjuwusL6bG2HVAbKfWJT6Gw8rmatSqGwZlkX8AM=; b=JaCHKVQh7SyNWOWRgcnZr2IXlifQn3jpxAixGZywLU3S7n9PX4jXdhLqMTH+j0Bxwl 706O0mllZ+MfLtR3eEijMk2ck+seKa4o2i0/5QNBQKnrVYBPs9hecxScSbKoSkZl8Qd9 SnIA9zMqltZQ44E4KETP1ark/SSrvAbSmPD25XfMlTOxTvBza1s6aEJ8sWrqy88nemRU 9dx/XvRf9ODpCk58tB8MkcYwLH0HUJFIA/cxJnXwcErPOLPd20a1cYzyvKUmrmK81bWG xtjOkzxVmFe5pElV30b+X9uG4TXmMbJlKZbBPMZOfLidtpUGBSNFALGHGwrIHJauHHvD TkSA== X-Gm-Message-State: AOAM531WrEVWLqTVEPGo5QwryxTJw/uGlxPVACet2zq5CGxZxr+kbiOQ 0BYyiqqWSJAgsJAf/OQUjfk= X-Google-Smtp-Source: ABdhPJxw5JyhewjwQ+K9jESelogQrwQJJ6oATw4UQK3z550KIq+hs2nwK0XFHfm6IJhd+C+XHJusVQ== X-Received: by 2002:a37:cc7:: with SMTP id 190mr31141841qkm.388.1600389906216; Thu, 17 Sep 2020 17:45:06 -0700 (PDT) Received: from localhost ([2620:10d:c091:480::1:6893]) by smtp.gmail.com with ESMTPSA id 2sm1041620qtg.20.2020.09.17.17.45.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 17 Sep 2020 17:45:05 -0700 (PDT) From: Tejun Heo To: axboe@kernel.dk Cc: linux-block@vger.kernel.org, cgroups@vger.kernel.org, kernel-team@fb.com, linux-kernel@vger.kernel.org, Tejun Heo Subject: [PATCH 3/5] iocost: recalculate delay after debt reduction Date: Thu, 17 Sep 2020 20:44:54 -0400 Message-Id: <20200918004456.593983-4-tj@kernel.org> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200918004456.593983-1-tj@kernel.org> References: <20200918004456.593983-1-tj@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Debt sets the initial delay duration which is decayed over time. The current debt reduction halved the debt but didn't change the delay. It prevented future debts from increasing delay but didn't do anything to lower the existing delay, limiting the mechanism's ability to reduce unnecessary idling. Reset iocg->delay to 0 after debt reduction so that iocg_kick_waitq() recalculates new delay value based on the reduced debt amount. Signed-off-by: Tejun Heo --- block/blk-iocost.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/block/blk-iocost.c b/block/blk-iocost.c index c0499c294da9..ffcb78126ab7 100644 --- a/block/blk-iocost.c +++ b/block/blk-iocost.c @@ -1984,7 +1984,8 @@ static void transfer_surpluses(struct list_head *surpluses, struct ioc_now *now) * up blocked paying its debt while the IO device is idle. * * The following protects against such cases. If the device has been - * sufficiently idle for a while, the debts are halved. + * sufficiently idle for a while, the debts are halved and delays are + * recalculated. */ static void ioc_forgive_debts(struct ioc *ioc, u64 usage_us_sum, int nr_debtors, struct ioc_now *now) @@ -2002,6 +2003,7 @@ static void ioc_forgive_debts(struct ioc *ioc, u64 usage_us_sum, int nr_debtors, if (iocg->abs_vdebt) { spin_lock(&iocg->waitq.lock); iocg->abs_vdebt /= 2; + iocg->delay = 0; /* kick_waitq will recalc */ iocg_kick_waitq(iocg, true, now); spin_unlock(&iocg->waitq.lock); } From patchwork Fri Sep 18 00:44:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 11783771 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D60CF92C for ; Fri, 18 Sep 2020 00:45:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B96F4208DB for ; Fri, 18 Sep 2020 00:45:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1600389914; bh=SAcT7GJyApzIjdgql5OUVarhWTtMbXQWvh34Lvpt9mo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=sN0sOQV1c7K009t9CWvfLLPPb+JYs9XPpFrWlzEUBLEmpocPeAsmNieVWcy0BoiP5 wxrE8bVvQWFuvmyDWWqpteSfFcZKbADOdXJqJ0bhRr9APk3WdWTvjDtvKJJO2IcA+M dXASbcj15N1dJ/NzUazAxz2OOIyg0U9zOBO8CW6o= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726304AbgIRApO (ORCPT ); Thu, 17 Sep 2020 20:45:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41336 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726007AbgIRApJ (ORCPT ); Thu, 17 Sep 2020 20:45:09 -0400 Received: from mail-qk1-x741.google.com (mail-qk1-x741.google.com [IPv6:2607:f8b0:4864:20::741]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1B880C061756; Thu, 17 Sep 2020 17:45:09 -0700 (PDT) Received: by mail-qk1-x741.google.com with SMTP id o5so4327891qke.12; Thu, 17 Sep 2020 17:45:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3VfMGSmp+708OVCB0V/rCITNRvHxvu/O226lr9+Sv6M=; b=G0EYqkC81T77PGGEYaNCMWDYz+gv2r9Qb5LDYOJ+IPNstZZa5g3bo2ZlOyOiuPCBCh X8nom7pA3onATBwoxojLEJzQV9qlb/bd5JjcYstQmQWHw2cw0K2CiEqfy6b3VLxBjwHS ZZSSeGSqJM0xHKi18lUTGZTgpG769TYc53BpjUVKOS5tQzTiDO93mXXmnE1BFaEzGFWB J3FNSSjj5LfRhCjQ+gVsADYyK2JDVWK3tBNNxqHdxvY86fYLDXKpSxnDnD7giQ4XA4Oq cTGqnrYHqu4cOtzO/J4gx/Sma/mBI97O4C4UhwEsQxVJD4HWtyZFuia4HQyneZ1SQZ4E 3nJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=3VfMGSmp+708OVCB0V/rCITNRvHxvu/O226lr9+Sv6M=; b=KlPQjE8Fo/Az/o6GsM00LY1B2gLRcNVkdZFryK8QJWrnl8KIhTueI8wjm1E0/GVVlm lhxEZBOEwSipFaPVFankDFdSMYV06GN7BYKMJPE+v2LDBlOQ1bYpUmWEUFKwzXU1nWSc MyOBA1MEYlJGeaF5ioT2wDikWabI2Cmu2d23Vp5UhXvVnL/DlE3zqpxJs9qGqMTFmL2H Mp6K4s8mCD8R8AMqVdIwRK1dtFSjy/GRBHjFKENfkgToF98JS4eVBRu9dz1TxNjLiWOE evK1yr2vS8TUZ95vNHNotwGVWgp77NRnYx/uFW4T8mwusWAkv2C+IuRFjj1/XCHwRJHj lBlw== X-Gm-Message-State: AOAM532rk8xVbSwH7S5Jjs3VHVYK80RQbNp/eZCZb8kq5q7X93jKF8hn hDvZlUSAKo6zbOJLEi7WPHc= X-Google-Smtp-Source: ABdhPJzbqgI4sz5Kd4ncKMIa6TPuapf0ORRDv1V3v/s447Mjk/b6EZRDHVWHhdGg/CWl6UpHtYjhiw== X-Received: by 2002:a05:620a:545:: with SMTP id o5mr31602684qko.172.1600389908210; Thu, 17 Sep 2020 17:45:08 -0700 (PDT) Received: from localhost ([2620:10d:c091:480::1:6893]) by smtp.gmail.com with ESMTPSA id z37sm1069740qtz.67.2020.09.17.17.45.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 17 Sep 2020 17:45:07 -0700 (PDT) From: Tejun Heo To: axboe@kernel.dk Cc: linux-block@vger.kernel.org, cgroups@vger.kernel.org, kernel-team@fb.com, linux-kernel@vger.kernel.org, Tejun Heo Subject: [PATCH 4/5] iocost: reimplement debt forgiveness using average usage Date: Thu, 17 Sep 2020 20:44:55 -0400 Message-Id: <20200918004456.593983-5-tj@kernel.org> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200918004456.593983-1-tj@kernel.org> References: <20200918004456.593983-1-tj@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Debt forgiveness logic was counting the number of consecutive !busy periods as the trigger condition. While this usually works, it can easily be thrown off by temporary fluctuations especially on configurations w/ short periods. This patch reimplements debt forgiveness so that: * Use the average usage over the forgiveness period instead of counting consecutive periods. * Debt is reduced at around the target rate (1/2 every 100ms) regardless of ioc period duration. * Usage threshold is raised to 50%. Combined with the preceding changes and the switch to average usage, this makes debt forgivness a lot more effective at reducing the amount of unnecessary idleness. * Constants are renamed with DFGV_ prefix. Signed-off-by: Tejun Heo --- block/blk-iocost.c | 94 ++++++++++++++++++++++++++++++++++------------ 1 file changed, 69 insertions(+), 25 deletions(-) diff --git a/block/blk-iocost.c b/block/blk-iocost.c index ffcb78126ab7..9b1f94499432 100644 --- a/block/blk-iocost.c +++ b/block/blk-iocost.c @@ -287,9 +287,9 @@ enum { MIN_DELAY = 250, MAX_DELAY = 250 * USEC_PER_MSEC, - /* halve debts if total usage keeps staying under 25% for over 100ms */ - DEBT_BUSY_USAGE_PCT = 25, - DEBT_REDUCTION_IDLE_DUR = 100 * USEC_PER_MSEC, + /* halve debts if avg usage over 100ms is under 50% */ + DFGV_USAGE_PCT = 50, + DFGV_PERIOD = 100 * USEC_PER_MSEC, /* don't let cmds which take a very long time pin lagging for too long */ MAX_LAGGING_PERIODS = 10, @@ -433,8 +433,10 @@ struct ioc { bool weights_updated; atomic_t hweight_gen; /* for lazy hweights */ - /* the last time debt cancel condition wasn't met */ - u64 debt_busy_at; + /* debt forgivness */ + u64 dfgv_period_at; + u64 dfgv_period_rem; + u64 dfgv_usage_us_sum; u64 autop_too_fast_at; u64 autop_too_slow_at; @@ -1251,7 +1253,8 @@ static bool iocg_activate(struct ioc_gq *iocg, struct ioc_now *now) if (ioc->running == IOC_IDLE) { ioc->running = IOC_RUNNING; - ioc->debt_busy_at = now->now; + ioc->dfgv_period_at = now->now; + ioc->dfgv_period_rem = 0; ioc_start_period(ioc, now); } @@ -1990,25 +1993,66 @@ static void transfer_surpluses(struct list_head *surpluses, struct ioc_now *now) static void ioc_forgive_debts(struct ioc *ioc, u64 usage_us_sum, int nr_debtors, struct ioc_now *now) { - if (ioc->busy_level < 0 || - div64_u64(100 * usage_us_sum, now->now - ioc->period_at) >= - DEBT_BUSY_USAGE_PCT) - ioc->debt_busy_at = now->now; - - if (nr_debtors && - now->now - ioc->debt_busy_at >= DEBT_REDUCTION_IDLE_DUR) { - struct ioc_gq *iocg; - - list_for_each_entry(iocg, &ioc->active_iocgs, active_list) { - if (iocg->abs_vdebt) { - spin_lock(&iocg->waitq.lock); - iocg->abs_vdebt /= 2; - iocg->delay = 0; /* kick_waitq will recalc */ - iocg_kick_waitq(iocg, true, now); - spin_unlock(&iocg->waitq.lock); - } - } - ioc->debt_busy_at = now->now; + struct ioc_gq *iocg; + u64 dur, usage_pct, nr_cycles; + + /* if no debtor, reset the cycle */ + if (!nr_debtors) { + ioc->dfgv_period_at = now->now; + ioc->dfgv_period_rem = 0; + ioc->dfgv_usage_us_sum = 0; + return; + } + + /* + * Debtors can pass through a lot of writes choking the device and we + * don't want to be forgiving debts while the device is struggling from + * write bursts. If we're missing latency targets, consider the device + * fully utilized. + */ + if (ioc->busy_level > 0) + usage_us_sum = max_t(u64, usage_us_sum, ioc->period_us); + + ioc->dfgv_usage_us_sum += usage_us_sum; + if (time_before64(now->now, ioc->dfgv_period_at + DFGV_PERIOD)) + return; + + /* + * At least DFGV_PERIOD has passed since the last period. Calculate the + * average usage and reset the period counters. + */ + dur = now->now - ioc->dfgv_period_at; + usage_pct = div64_u64(100 * ioc->dfgv_usage_us_sum, dur); + + ioc->dfgv_period_at = now->now; + ioc->dfgv_usage_us_sum = 0; + + /* if was too busy, reset everything */ + if (usage_pct > DFGV_USAGE_PCT) { + ioc->dfgv_period_rem = 0; + return; + } + + /* + * Usage is lower than threshold. Let's forgive some debts. Debt + * forgiveness runs off of the usual ioc timer but its period usually + * doesn't match ioc's. Compensate the difference by performing the + * reduction as many times as would fit in the duration since the last + * run and carrying over the left-over duration in @ioc->dfgv_period_rem + * - if ioc period is 75% of DFGV_PERIOD, one out of three consecutive + * reductions is doubled. + */ + nr_cycles = dur + ioc->dfgv_period_rem; + ioc->dfgv_period_rem = do_div(nr_cycles, DFGV_PERIOD); + + list_for_each_entry(iocg, &ioc->active_iocgs, active_list) { + if (!iocg->abs_vdebt) + continue; + spin_lock(&iocg->waitq.lock); + iocg->abs_vdebt >>= nr_cycles; + iocg->delay = 0; /* kick_waitq will recalc */ + iocg_kick_waitq(iocg, true, now); + spin_unlock(&iocg->waitq.lock); } } From patchwork Fri Sep 18 00:44:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 11783769 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 06BE36CB for ; Fri, 18 Sep 2020 00:45:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E078520838 for ; Fri, 18 Sep 2020 00:45:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1600389913; bh=QXHL2178dP8PXW/HjKnPeEmvBvs4esmALFbzHF46vDo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=OG0ZkIe6UwdI5Cs2rzmLl6vd17cO7BoiEMJSIJJGqj9yxcNpnBvlez2E+2GnhqqC3 tuCPextP+V0jyIWWJx5oN0PwlPrWYht1H+fKmOlk1rV4+Di2wELK29CXhC2yKvCWdN 3P+4bYMDjjSJvyz2v1aLwyVxpJzutIpEUX9LY2DI= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726281AbgIRApM (ORCPT ); Thu, 17 Sep 2020 20:45:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41346 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726279AbgIRApL (ORCPT ); Thu, 17 Sep 2020 20:45:11 -0400 Received: from mail-qk1-x741.google.com (mail-qk1-x741.google.com [IPv6:2607:f8b0:4864:20::741]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F1460C06178A; Thu, 17 Sep 2020 17:45:10 -0700 (PDT) Received: by mail-qk1-x741.google.com with SMTP id g72so4354107qke.8; Thu, 17 Sep 2020 17:45:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=mKtA7cFsHNh8hYHY7vxbi2vKeig9nNldo1RjIq6TDkU=; b=qEUz8l3TjyhsGosmXReNkCXnNOUMnbuhorgcWpRVoMfslfF8hO0Z5BXLsGJ1eOA/YQ B/FJJyDVE+YoC/n5s19IhlY5KOzaDWsulPXgRRPl7m/2HV8UYEPh+Y7deLQjqLMoogD5 q/I2xRsP1X1d9aMY0QgP2C+Eohq109DY5E4dtT6MZWUc5vLE8gF0gOB8xokWY68MiENL OOTeZVRCNo4yFZ9ZZ6jSDJGKf73eKQpEV8U+D6gLO532hS38I1AwwrnGxI6LK75Lei+F SFYqsZ1NDgmJtBFZbeMnCGOKsef3Pc4dcwqjay5YgwrIERh9W80ioJxhA7i4yoyUAClm WeTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=mKtA7cFsHNh8hYHY7vxbi2vKeig9nNldo1RjIq6TDkU=; b=GLaeVKx7ux0eM8L7Fsy1LGwcSdi/gYa7awloTdfZQ2+XUVrXkfbwI13yLbPk4J7Jns m1jqtcdZrbhueHKro/rGM3MxH/X29k9U3pihOMCbS3jLFj6ZIvcX2A475E4ggkmSB5JI 6rt4/3Jfs0+fQmhQZpQxHEouVz7+B38hGsMwAPmb39RpILjLMzriPIVAHTvy2WCsxiCw 8g9skdptRAIZw8LnMGASc3aC/MuqVFJJiQA0Lr3pPY3lHn1ExYuCDXIqtgOymIrvRN9c KWM+qJHyAeqve3i09WM3Zsc2t55RbL7tB64PeGRDyOsFd9lxGzt3IVo6g+Zr7WLsGM1o kWOg== X-Gm-Message-State: AOAM533gqtViGjUhMgTbdKdVPO9j5tDSxsqMf1xM6zOn9f5gzOVhci3O UZ9dW5k68jZSU6rOtzihycU= X-Google-Smtp-Source: ABdhPJyz9fH33yXfplmTghMymIBJ67u130kv7OxYBvln1bqjnQ2+xHKNecNuQseRlxGekhe8CIveVA== X-Received: by 2002:a37:2715:: with SMTP id n21mr31314550qkn.401.1600389910170; Thu, 17 Sep 2020 17:45:10 -0700 (PDT) Received: from localhost ([2620:10d:c091:480::1:6893]) by smtp.gmail.com with ESMTPSA id a24sm1024239qko.82.2020.09.17.17.45.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 17 Sep 2020 17:45:09 -0700 (PDT) From: Tejun Heo To: axboe@kernel.dk Cc: linux-block@vger.kernel.org, cgroups@vger.kernel.org, kernel-team@fb.com, linux-kernel@vger.kernel.org, Tejun Heo Subject: [PATCH 5/5] iocost: add iocg_forgive_debt tracepoint Date: Thu, 17 Sep 2020 20:44:56 -0400 Message-Id: <20200918004456.593983-6-tj@kernel.org> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200918004456.593983-1-tj@kernel.org> References: <20200918004456.593983-1-tj@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Signed-off-by: Tejun Heo --- block/blk-iocost.c | 12 ++++++++++ include/trace/events/iocost.h | 41 +++++++++++++++++++++++++++++++++++ 2 files changed, 53 insertions(+) diff --git a/block/blk-iocost.c b/block/blk-iocost.c index 9b1f94499432..328ae805e85f 100644 --- a/block/blk-iocost.c +++ b/block/blk-iocost.c @@ -2046,12 +2046,24 @@ static void ioc_forgive_debts(struct ioc *ioc, u64 usage_us_sum, int nr_debtors, ioc->dfgv_period_rem = do_div(nr_cycles, DFGV_PERIOD); list_for_each_entry(iocg, &ioc->active_iocgs, active_list) { + u64 __maybe_unused old_debt, __maybe_unused old_delay; + if (!iocg->abs_vdebt) continue; + spin_lock(&iocg->waitq.lock); + + old_debt = iocg->abs_vdebt; + old_delay = iocg->delay; + iocg->abs_vdebt >>= nr_cycles; iocg->delay = 0; /* kick_waitq will recalc */ iocg_kick_waitq(iocg, true, now); + + TRACE_IOCG_PATH(iocg_forgive_debt, iocg, now, usage_pct, + old_debt, iocg->abs_vdebt, + old_delay, iocg->delay); + spin_unlock(&iocg->waitq.lock); } } diff --git a/include/trace/events/iocost.h b/include/trace/events/iocost.h index b350860d2e71..0b6869980ba2 100644 --- a/include/trace/events/iocost.h +++ b/include/trace/events/iocost.h @@ -164,6 +164,47 @@ TRACE_EVENT(iocost_ioc_vrate_adj, ) ); +TRACE_EVENT(iocost_iocg_forgive_debt, + + TP_PROTO(struct ioc_gq *iocg, const char *path, struct ioc_now *now, + u32 usage_pct, u64 old_debt, u64 new_debt, + u64 old_delay, u64 new_delay), + + TP_ARGS(iocg, path, now, usage_pct, + old_debt, new_debt, old_delay, new_delay), + + TP_STRUCT__entry ( + __string(devname, ioc_name(iocg->ioc)) + __string(cgroup, path) + __field(u64, now) + __field(u64, vnow) + __field(u32, usage_pct) + __field(u64, old_debt) + __field(u64, new_debt) + __field(u64, old_delay) + __field(u64, new_delay) + ), + + TP_fast_assign( + __assign_str(devname, ioc_name(iocg->ioc)); + __assign_str(cgroup, path); + __entry->now = now->now; + __entry->vnow = now->vnow; + __entry->usage_pct = usage_pct; + __entry->old_debt = old_debt; + __entry->new_debt = new_debt; + __entry->old_delay = old_delay; + __entry->new_delay = new_delay; + ), + + TP_printk("[%s:%s] now=%llu:%llu usage=%u debt=%llu->%llu delay=%llu->%llu", + __get_str(devname), __get_str(cgroup), + __entry->now, __entry->vnow, __entry->usage_pct, + __entry->old_debt, __entry->new_debt, + __entry->old_delay, __entry->new_delay + ) +); + #endif /* _TRACE_BLK_IOCOST_H */ /* This part must be outside protection */ From patchwork Fri Sep 18 18:41:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 11785671 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BB03E746 for ; Fri, 18 Sep 2020 18:41:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 97EB9221EC for ; Fri, 18 Sep 2020 18:41:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1600454491; bh=2ySdKpNv4H3xjExvqcqBkdQV6ayk69DC/t/KNpodubc=; h=Date:From:To:Cc:Subject:References:In-Reply-To:List-ID:From; b=1hOrTsyWRXKtiHEiX+5ixq1byFMeZGiB8Pbs1pNlQGEWvRx9ajKJ4nTgtb8sC2BRa Uc/GtBAmcL6eiHNr2lnTu5eC7lGYNz93sVGwPdXlGWWybVi9tOZ/mKFyKAjIBwNTfG 2+LDvSBdiuYXOX0LDby6MIe7dIUn+184coptDwEI= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726192AbgIRSla (ORCPT ); Fri, 18 Sep 2020 14:41:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38348 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726115AbgIRSla (ORCPT ); Fri, 18 Sep 2020 14:41:30 -0400 Received: from mail-qv1-xf43.google.com (mail-qv1-xf43.google.com [IPv6:2607:f8b0:4864:20::f43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8B19FC0613CF; Fri, 18 Sep 2020 11:41:30 -0700 (PDT) Received: by mail-qv1-xf43.google.com with SMTP id db4so3483752qvb.4; Fri, 18 Sep 2020 11:41:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=RyuK5UieZne+pKhpGinbT1xLnPtjZ4n2yvvqOQQR/RM=; b=bijR7TvXbgX48PfH9waDRVC/Tbn3PSFLH/s/gHvqLfGFatnCdUy9xUMEhd8hR+12c6 bM4G7PDeskHdUznZLA0Y1RXttv8Zf5Kc4gWb0xfOf7BiW2JvYcY8zQoWk/EWjxWDr9Co F6lzPp0d/WanDs+PEy5McGVjmqp9J4aOu1Mj6fu/oo7Zb28uMf6HGPvK3eQekiroIocI qgnEXOWua96Vs6spW1iBrlpEY1kxchcSzIEAAer60R4lQGVJsXv55z2V/a4PjsGPbf7R 0pHxW2in/M7Xd1ZBpmWUarETt52ESRjqTDHazLcEMmOk0JD1lIYUj0mqE7Idd144OnLF 94sQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition:in-reply-to; bh=RyuK5UieZne+pKhpGinbT1xLnPtjZ4n2yvvqOQQR/RM=; b=R6Igt2pcSBpeIJSiGiK++72Kc+2hfNI7Gg0GMRG5pf1EY9BccHzgEmZViiPJhpfnIk +nc05zhCbC5CJKOJB1BksU/7MretGyaNRH9Nd5Jtsx8HqrB6+bvqTl2QN6Kyl8bvBxBz npDzEm5wmjQOCJO5oXA1iYKgkOB/RCDllEnGghlPViOlkb1csyq0Uq3u7HIGNATUG03m Oh2EIuwQTEHzrySAQ8mexQhzX8N7qs3s5KxMRqKmXKXHPw01EoOy8AMLSgmh7l76OJdZ YNbNY7NCkPNkG+2g9/zVZdkkXPENbUlWNR9DvKDUgBJfSL7FCopkwoSaeEBYjhH9h9op VAOg== X-Gm-Message-State: AOAM530IL2qpZuoziGyMoUB62AHLxPC7F6wWpjbSY9d5+HfHgQBvQKFA poh6GX6GoTsrtOL9W35iDn0= X-Google-Smtp-Source: ABdhPJyrfE5zh7qVv/Ct8VYZ7c9haa82D17TT639IsPr+2OBmTZj5Zp7at5HyuG8X09vdAQnQQeKVQ== X-Received: by 2002:a0c:8ec6:: with SMTP id y6mr23235555qvb.24.1600454489667; Fri, 18 Sep 2020 11:41:29 -0700 (PDT) Received: from localhost ([2620:10d:c091:480::1:602a]) by smtp.gmail.com with ESMTPSA id 18sm2584248qkd.120.2020.09.18.11.41.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Sep 2020 11:41:29 -0700 (PDT) Date: Fri, 18 Sep 2020 14:41:27 -0400 From: Tejun Heo To: axboe@kernel.dk Cc: linux-block@vger.kernel.org, cgroups@vger.kernel.org, kernel-team@fb.com, linux-kernel@vger.kernel.org Subject: [PATCH 6/5] iocost: consider iocgs with active delays for debt forgiveness Message-ID: <20200918184127.GB4247@mtj.thefacebook.com> References: <20200918004456.593983-1-tj@kernel.org> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20200918004456.593983-1-tj@kernel.org> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org An iocg may have 0 debt but non-zero delay. The current debt forgiveness logic doesn't act on such iocgs. This can lead to unexpected behaviors - an iocg with a little bit of debt will have its delay canceled through debt forgiveness but one w/o any debt but active delay will have to wait out until its delay decays out. This patch updates the debt handling logic so that it treats delays the same as debts. If either debt or delay is active, debt forgiveness logic kicks in and acts on both the same way. Also, avoid turning the debt and delay directly to zero as that can confuse state transitions. Signed-off-by: Tejun Heo --- Jens, a follow up patch to the series. The git tree is also updated. Thanks. block/blk-iocost.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) --- a/block/blk-iocost.c +++ b/block/blk-iocost.c @@ -2048,7 +2048,7 @@ static void ioc_forgive_debts(struct ioc list_for_each_entry(iocg, &ioc->active_iocgs, active_list) { u64 __maybe_unused old_debt, __maybe_unused old_delay; - if (!iocg->abs_vdebt) + if (!iocg->abs_vdebt && !iocg->delay) continue; spin_lock(&iocg->waitq.lock); @@ -2056,8 +2056,11 @@ static void ioc_forgive_debts(struct ioc old_debt = iocg->abs_vdebt; old_delay = iocg->delay; - iocg->abs_vdebt >>= nr_cycles; - iocg->delay = 0; /* kick_waitq will recalc */ + if (iocg->abs_vdebt) + iocg->abs_vdebt = iocg->abs_vdebt >> nr_cycles ?: 1; + if (iocg->delay) + iocg->delay = iocg->delay >> nr_cycles ?: 1; + iocg_kick_waitq(iocg, true, now); TRACE_IOCG_PATH(iocg_forgive_debt, iocg, now, usage_pct, @@ -2129,7 +2132,7 @@ static void ioc_timer_fn(struct timer_li iocg->delay) { /* might be oversleeping vtime / hweight changes, kick */ iocg_kick_waitq(iocg, true, &now); - if (iocg->abs_vdebt) + if (iocg->abs_vdebt || iocg->delay) nr_debtors++; } else if (iocg_is_idle(iocg)) { /* no waiter and idle, deactivate */