From patchwork Fri Oct 13 23:35:37 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Lyle X-Patchwork-Id: 10006133 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 02A7C60230 for ; Fri, 13 Oct 2017 23:36:16 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E78A529009 for ; Fri, 13 Oct 2017 23:36:15 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DCA2D2918E; Fri, 13 Oct 2017 23:36:15 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.4 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, RCVD_IN_DNSWL_HI, RCVD_IN_SORBS_SPAM autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8415329009 for ; Fri, 13 Oct 2017 23:36:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753089AbdJMXgN (ORCPT ); Fri, 13 Oct 2017 19:36:13 -0400 Received: from mail-pg0-f66.google.com ([74.125.83.66]:43886 "EHLO mail-pg0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753232AbdJMXgK (ORCPT ); Fri, 13 Oct 2017 19:36:10 -0400 Received: by mail-pg0-f66.google.com with SMTP id s75so1499478pgs.0 for ; Fri, 13 Oct 2017 16:36:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lyle-org.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=Wvbs+Eozy1TrZs1lhuRPIph6Y5kaFJD3AKvXbJItdSI=; b=ocnd4AlQb85YrR+7VPLzwbOdB/RJFW5lo8wiLmf294w4vfbcj9vVaTrm4bMUQ49ARh KTkPY4aL4h5GpuKHxyrYAllCzGIcnAbfIzIWefVQ9dOrSf8f9L5+vm300OEl7UdEgM0u PQoiws0pvuqWIQokcSlmbXJp2wymjeovfSquHyYYGUDyBIqXiofFSsyYSCfDfLJteReJ pvKVzTdG9pBjy6ouyjPr+nSmAcAve6b/VzrGrISwuxNYzr7UIYvoLeGtFCsuvl5krfQZ MPld7YthQXjnBpD0v7JgSUMskctkqQiV+5vrn2lEOL+d5ZgXAzRv/pYNboY43pnVJT6O fwaQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Wvbs+Eozy1TrZs1lhuRPIph6Y5kaFJD3AKvXbJItdSI=; b=rv41Wl+dDRdLHTU2V1zC0SMGmHQsBEULRyf/6boeIiPqBF0ZpwO68vzz9QihAOOur/ 2qgnIwPyvcF4V1HrFTVIN+dKoTZgXNFvjnad/lGklRbSOTl1fGNqlyYeIHQR2IXRYB/S zMHFzNQbk5vkZb1CJfLVVU0MSbaYu1eSPPeVsPcfLrLcrZmwxkC5Le8h5eL9YCR0XsZX AYYVZqR/nthAEyuQG40+xli7QSRy7WIcB/4U+h/X9Og8glTME8kAv5RpxlKHTyKrwCzm fP61344iR6wALeUpNs5jwP23kp356Kiok6Zc6dNp+ftn8ZBWqyjDDxL3HshCTuBh2eW1 FUCQ== X-Gm-Message-State: AMCzsaW/r2aSeTT2cQJCyhoXXULNIwpqOYtISzt10WdvjrYu6pUus345 oU6ABecFgVjGJ/gXSmmF7vUIQw== X-Google-Smtp-Source: AOwi7QA8l5YnDbtXpC1MO9SYlfYFOa+KXKaO/N42yK6bdIFHsVWBOYfJu8D+Y108Ifqq5jtHEYVG7A== X-Received: by 10.84.173.4 with SMTP id o4mr2679195plb.152.1507937769901; Fri, 13 Oct 2017 16:36:09 -0700 (PDT) Received: from localhost.localdomain (68-189-67-104.dhcp.prtv.ca.charter.com. [68.189.67.104]) by smtp.gmail.com with ESMTPSA id 74sm4950718pft.184.2017.10.13.16.36.08 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 13 Oct 2017 16:36:09 -0700 (PDT) From: Michael Lyle To: linux-bcache@vger.kernel.org, linux-block@vger.kernel.org Cc: axboe@fb.com, Michael Lyle Subject: [PATCH 10/15] bcache: smooth writeback rate control Date: Fri, 13 Oct 2017 16:35:37 -0700 Message-Id: <20171013233542.20938-11-mlyle@lyle.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20171013233542.20938-1-mlyle@lyle.org> References: <20171013233542.20938-1-mlyle@lyle.org> Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This works in conjunction with the new PI controller. Currently, in real-world workloads, the rate controller attempts to write back 1 sector per second. In practice, these minimum-rate writebacks are between 4k and 60k in test scenarios, since bcache aggregates and attempts to do contiguous writes and because filesystems on top of bcachefs typically write 4k or more. Previously, bcache used to guarantee to write at least once per second. This means that the actual writeback rate would exceed the configured amount by a factor of 8-120 or more. This patch adjusts to be willing to sleep up to 2.5 seconds, and to target writing 4k/second. On the smallest writes, it will sleep 1 second like before, but many times it will sleep longer and load the backing device less. This keeps the loading on the cache and backing device related to writeback more consistent when writing back at low rates. Signed-off-by: Michael Lyle Reviewed-by: Coly Li --- drivers/md/bcache/util.c | 10 ++++++++-- drivers/md/bcache/writeback.c | 2 +- 2 files changed, 9 insertions(+), 3 deletions(-) diff --git a/drivers/md/bcache/util.c b/drivers/md/bcache/util.c index 176d3c2ef5f5..4dbe37e82877 100644 --- a/drivers/md/bcache/util.c +++ b/drivers/md/bcache/util.c @@ -232,8 +232,14 @@ uint64_t bch_next_delay(struct bch_ratelimit *d, uint64_t done) d->next += div_u64(done * NSEC_PER_SEC, d->rate); - if (time_before64(now + NSEC_PER_SEC, d->next)) - d->next = now + NSEC_PER_SEC; + /* Bound the time. Don't let us fall further than 2 seconds behind + * (this prevents unnecessary backlog that would make it impossible + * to catch up). If we're ahead of the desired writeback rate, + * don't let us sleep more than 2.5 seconds (so we can notice/respond + * if the control system tells us to speed up!). + */ + if (time_before64(now + NSEC_PER_SEC * 5 / 2, d->next)) + d->next = now + NSEC_PER_SEC * 5 / 2; if (time_after64(now - NSEC_PER_SEC * 2, d->next)) d->next = now - NSEC_PER_SEC * 2; diff --git a/drivers/md/bcache/writeback.c b/drivers/md/bcache/writeback.c index cac8678da5d0..8deb721c355e 100644 --- a/drivers/md/bcache/writeback.c +++ b/drivers/md/bcache/writeback.c @@ -526,7 +526,7 @@ void bch_cached_dev_writeback_init(struct cached_dev *dc) dc->writeback_percent = 10; dc->writeback_delay = 30; dc->writeback_rate.rate = 1024; - dc->writeback_rate_minimum = 1; + dc->writeback_rate_minimum = 8; dc->writeback_rate_update_seconds = 5; dc->writeback_rate_p_term_inverse = 40;