From patchwork Tue Apr 23 03:46:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kemeng Shi X-Patchwork-Id: 13639286 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A7A00C04FFE for ; Tue, 23 Apr 2024 03:47:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 087E86B009B; Mon, 22 Apr 2024 23:46:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BCC8C6B0098; Mon, 22 Apr 2024 23:46:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 846A36B009C; Mon, 22 Apr 2024 23:46:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 4E5C86B009D for ; Mon, 22 Apr 2024 23:46:55 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 01BCBC0A4C for ; Tue, 23 Apr 2024 03:46:54 +0000 (UTC) X-FDA: 82039410390.27.45F6493 Received: from dggsgout11.his.huawei.com (unknown [45.249.212.51]) by imf29.hostedemail.com (Postfix) with ESMTP id E2A83120006 for ; Tue, 23 Apr 2024 03:46:50 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf29.hostedemail.com: domain of shikemeng@huaweicloud.com designates 45.249.212.51 as permitted sender) smtp.mailfrom=shikemeng@huaweicloud.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1713844013; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EaZdIV8Ypvj7Pk6yHhSXBjg1GMBCsv+k1WT11475tIY=; b=vz4d7xKXk+fWEh3LlkQwhQVTC/pRk6XbkRKNidhXuHnzK0QHHdrlenppTUzI4VcdnVkEX2 kbNPEvq858XwjksOZbFMrwwFdAdAkn/KodRi4JUklMJ1a2ExG0E5CddqepkzF8BNlIAank gwgHyVd7+waQIVNtSpO9/5aIgqD/ngE= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf29.hostedemail.com: domain of shikemeng@huaweicloud.com designates 45.249.212.51 as permitted sender) smtp.mailfrom=shikemeng@huaweicloud.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1713844013; a=rsa-sha256; cv=none; b=pRc4HVPacxL7TC3jXl51cUqrRgeMiCwXgvEYyRb/YnU9lMpL6Z2yLT5XAfynrWxSi3bSSo Nv835rproNt1wBChFF/LTTM2aabakPQF+kvyvUPB3utxs3WCHByFh4uqTB961gCQ4AAhos iWsVS5yjZjACpq0bNRYhKOuLmocQPJk= Received: from mail.maildlp.com (unknown [172.19.163.235]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTP id 4VNp2T5XVwz4f3kny for ; Tue, 23 Apr 2024 11:46:41 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.75]) by mail.maildlp.com (Postfix) with ESMTP id 92C1F1A0568 for ; Tue, 23 Apr 2024 11:46:46 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.101.6]) by APP2 (Coremail) with SMTP id Syh0CgA3Ww4kLydmKkDYKw--.11241S4; Tue, 23 Apr 2024 11:46:46 +0800 (CST) From: Kemeng Shi To: akpm@linux-foundation.org, willy@infradead.org, jack@suse.cz, bfoster@redhat.com, tj@kernel.org Cc: dsterba@suse.com, mjguzik@gmail.com, dhowells@redhat.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Subject: [PATCH v5 2/5] writeback: support retrieving per group debug writeback stats of bdi Date: Tue, 23 Apr 2024 11:46:40 +0800 Message-Id: <20240423034643.141219-3-shikemeng@huaweicloud.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20240423034643.141219-1-shikemeng@huaweicloud.com> References: <20240423034643.141219-1-shikemeng@huaweicloud.com> MIME-Version: 1.0 X-CM-TRANSID: Syh0CgA3Ww4kLydmKkDYKw--.11241S4 X-Coremail-Antispam: 1UD129KBjvJXoW3Xw1rCry8Kw1kCr4ftF4fXwb_yoW7uF45pF Z8Gw15tr4UZr17WFZxAFW2qry5tw48trW7X3s7u3yftFnFgry3tFy8ury8Ary5AF93AFy3 Jan8Cry8GrW8KrJanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUB0b4IE77IF4wAFF20E14v26rWj6s0DM7CY07I20VC2zVCF04k2 6cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M28IrcIa0xkI8VA2jI8067AKxVWUXw A2048vs2IY020Ec7CjxVAFwI0_Xr0E3s1l8cAvFVAK0II2c7xJM28CjxkF64kEwVA0rcxS w2x7M28EF7xvwVC0I7IYx2IY67AKxVWDJVCq3wA2z4x0Y4vE2Ix0cI8IcVCY1x0267AKxV W8Jr0_Cr1UM28EF7xvwVC2z280aVAFwI0_GcCE3s1l84ACjcxK6I8E87Iv6xkF7I0E14v2 6rxl6s0DM2AIxVAIcxkEcVAq07x20xvEncxIr21l5I8CrVACY4xI64kE6c02F40Ex7xfMc Ij6xIIjxv20xvE14v26r1j6r18McIj6I8E87Iv67AKxVWUJVW8JwAm72CE4IkC6x0Yz7v_ Jr0_Gr1lF7xvr2IYc2Ij64vIr41lFIxGxcIEc7CjxVA2Y2ka0xkIwI1l42xK82IYc2Ij64 vIr41l4I8I3I0E4IkC6x0Yz7v_Jr0_Gr1lx2IqxVAqx4xG67AKxVWUJVWUGwC20s026x8G jcxK67AKxVWUGVWUWwC2zVAF1VAY17CE14v26r1q6r43MIIYrxkI7VAKI48JMIIF0xvE2I x0cI8IcVAFwI0_Jr0_JF4lIxAIcVC0I7IYx2IY6xkF7I0E14v26r4j6F4UMIIF0xvE42xK 8VAvwI8IcIk0rVWUJVWUCwCI42IY6I8E87Iv67AKxVWUJVW8JwCI42IY6I8E87Iv6xkF7I 0E14v26r4j6r4UJbIYCTnIWIevJa73UjIFyTuYvjxUFa9-UUUUU X-CM-SenderInfo: 5vklyvpphqwq5kxd4v5lfo033gof0z/ X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: E2A83120006 X-Rspam-User: X-Stat-Signature: dixgipued1hgjk669455pehjywgoc56d X-HE-Tag: 1713844010-156974 X-HE-Meta: U2FsdGVkX18YJ0yoHyxYIuGMw1U8p8ldWDox0k4J7qJzGQDzC+xGLFhY6SWWwM5Nms+KMqyEQEAbR2mlnLfJF6NhrvVpb1H+TS2Jr9SaWXsSPo5WINasd5CVjQGq2NfeNMj6afGcPT1RukmJ7sfWCiMHk+UjoOZ/EKDI5/K4a92tWY7eSle4kdZh2UHYhC2sWCst2GZbjy3LU0v9sz5gmO1AxqViZ8L/zjXXK8bN9USDT9eN7au0avuTVknw5O8Z6LCsVsO5s56rdC/AgV740hjhMLt2a5JojUqGEpqWNCEWaaIlZN+XwYzBvzTEXEFXOXbJ7myCi3ab4TOlOYA8HlIBwcFMZoASvD4cFbvw5PTScQId2IESo9GaYxFnkQvgchwpA7BnYR1alum9hOEmKteZUpbSfVEloL9Zz67guoSz2mhXb2a2NvyhtsF5gYNaJLyqv/mkahprF41BzvJRI+qLyG4T21BKu5EK8JzhUSUwL9302vrppxQEbIMpYsIlwmD6VsLJua/wEXpFUOmvpVGfgrZjQuQN7piyMTDCJUoaTAMctnbdGpw9UOr5VbW5CDU1tqg52WD/vwHAkluY8TQVkc572TphXvwY1p+IOwH6qs6zaYUejmFgmGhMqzt1vmdlbh6dVstth3TA1tcbW9JVbPvghPSh737A/Xu5YSRgDBhPWn9ju2SzBsZMnuAbPpoCePkFDeJFa/JVCuOA5Ef0sA9454LbPfNugRthm48vmY7dMLTOLaoaGVTSLnQIK9Aq84lZdxc74B2o+X9rAqMj5HZS+KEDHrphE5ssyVxKYnR4w3JNuJ4nY+ECg9oQZmE86fnJXni/j/KijUliUkyAna/gyVpCgHwowJuD8LSFKDUXIN3Rckf4L/3zjf23aS46LwEaMkr5Q6n8Vcn+M5yt4fqKs/O4nO5TcdNCx8b1wIU4Rvj4EbQT78HUI7eU1LdQXBAvNRjCFOlrUtb vSqAcON7 3fv3ymO/2AYtlCnXJFXUUAaH3t9TVHJJYWU4m6RJ0RERctTGOEL8VV9crczbh/Z/iUYtUjxx3oQc577VJPxcQs9fh5U3Od6Y6i5KvL8P7wPLykjuhjlxWGI/2xDcUJpKiev2SCJharQzzcoKI1HZi3vo67t74rEXes9iLDtXreFCkUfj+T34J/DlWVJRrdqWezOAhq1ZRY0Fbp3kD4+KqiXTJejwt+GeZvJn1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add /sys/kernel/debug/bdi/xxx/wb_stats to show per group writeback stats of bdi. Following domain hierarchy is tested: global domain (320G) / \ cgroup domain1(10G) cgroup domain2(10G) | | bdi wb1 wb2 /* per wb writeback info of bdi is collected */ cat wb_stats WbCgIno: 1 WbWriteback: 0 kB WbReclaimable: 0 kB WbDirtyThresh: 0 kB WbDirtied: 0 kB WbWritten: 0 kB WbWriteBandwidth: 102400 kBps b_dirty: 0 b_io: 0 b_more_io: 0 b_dirty_time: 0 state: 1 WbCgIno: 4091 WbWriteback: 1792 kB WbReclaimable: 820512 kB WbDirtyThresh: 6004692 kB WbDirtied: 1820448 kB WbWritten: 999488 kB WbWriteBandwidth: 169020 kBps b_dirty: 0 b_io: 0 b_more_io: 1 b_dirty_time: 0 state: 5 WbCgIno: 4131 WbWriteback: 1120 kB WbReclaimable: 820064 kB WbDirtyThresh: 6004728 kB WbDirtied: 1822688 kB WbWritten: 1002400 kB WbWriteBandwidth: 153520 kBps b_dirty: 0 b_io: 0 b_more_io: 1 b_dirty_time: 0 state: 5 Signed-off-by: Kemeng Shi --- include/linux/writeback.h | 1 + mm/backing-dev.c | 78 ++++++++++++++++++++++++++++++++++++++- mm/page-writeback.c | 19 ++++++++++ 3 files changed, 96 insertions(+), 2 deletions(-) diff --git a/include/linux/writeback.h b/include/linux/writeback.h index 9845cb62e40b..112d806ddbe4 100644 --- a/include/linux/writeback.h +++ b/include/linux/writeback.h @@ -355,6 +355,7 @@ int dirtytime_interval_handler(struct ctl_table *table, int write, void global_dirty_limits(unsigned long *pbackground, unsigned long *pdirty); unsigned long wb_calc_thresh(struct bdi_writeback *wb, unsigned long thresh); +unsigned long cgwb_calc_thresh(struct bdi_writeback *wb); void wb_update_bandwidth(struct bdi_writeback *wb); diff --git a/mm/backing-dev.c b/mm/backing-dev.c index 089146feb830..6ecd11bdce6e 100644 --- a/mm/backing-dev.c +++ b/mm/backing-dev.c @@ -155,19 +155,93 @@ static int bdi_debug_stats_show(struct seq_file *m, void *v) } DEFINE_SHOW_ATTRIBUTE(bdi_debug_stats); +static void wb_stats_show(struct seq_file *m, struct bdi_writeback *wb, + struct wb_stats *stats) +{ + + seq_printf(m, + "WbCgIno: %10lu\n" + "WbWriteback: %10lu kB\n" + "WbReclaimable: %10lu kB\n" + "WbDirtyThresh: %10lu kB\n" + "WbDirtied: %10lu kB\n" + "WbWritten: %10lu kB\n" + "WbWriteBandwidth: %10lu kBps\n" + "b_dirty: %10lu\n" + "b_io: %10lu\n" + "b_more_io: %10lu\n" + "b_dirty_time: %10lu\n" + "state: %10lx\n\n", + cgroup_ino(wb->memcg_css->cgroup), + K(stats->nr_writeback), + K(stats->nr_reclaimable), + K(stats->wb_thresh), + K(stats->nr_dirtied), + K(stats->nr_written), + K(wb->avg_write_bandwidth), + stats->nr_dirty, + stats->nr_io, + stats->nr_more_io, + stats->nr_dirty_time, + wb->state); +} + +static int cgwb_debug_stats_show(struct seq_file *m, void *v) +{ + struct backing_dev_info *bdi = m->private; + unsigned long background_thresh; + unsigned long dirty_thresh; + struct bdi_writeback *wb; + struct wb_stats stats; + + global_dirty_limits(&background_thresh, &dirty_thresh); + + rcu_read_lock(); + list_for_each_entry_rcu(wb, &bdi->wb_list, bdi_node) { + struct wb_stats stats = { .dirty_thresh = dirty_thresh }; + + if (!wb_tryget(wb)) + continue; + + collect_wb_stats(&stats, wb); + + /* + * Calculate thresh of wb in writeback cgroup which is min of + * thresh in global domain and thresh in cgroup domain. Drop + * rcu lock because cgwb_calc_thresh may sleep in + * cgroup_rstat_flush. We can do so here because we have a ref. + */ + if (mem_cgroup_wb_domain(wb)) { + rcu_read_unlock(); + stats.wb_thresh = min(stats.wb_thresh, cgwb_calc_thresh(wb)); + rcu_read_lock(); + } + + wb_stats_show(m, wb, &stats); + + wb_put(wb); + } + rcu_read_unlock(); + + return 0; +} +DEFINE_SHOW_ATTRIBUTE(cgwb_debug_stats); + static void bdi_debug_register(struct backing_dev_info *bdi, const char *name) { bdi->debug_dir = debugfs_create_dir(name, bdi_debug_root); debugfs_create_file("stats", 0444, bdi->debug_dir, bdi, &bdi_debug_stats_fops); + debugfs_create_file("wb_stats", 0444, bdi->debug_dir, bdi, + &cgwb_debug_stats_fops); } static void bdi_debug_unregister(struct backing_dev_info *bdi) { debugfs_remove_recursive(bdi->debug_dir); } -#else +#else /* CONFIG_DEBUG_FS */ static inline void bdi_debug_init(void) { } @@ -178,7 +252,7 @@ static inline void bdi_debug_register(struct backing_dev_info *bdi, static inline void bdi_debug_unregister(struct backing_dev_info *bdi) { } -#endif +#endif /* CONFIG_DEBUG_FS */ static ssize_t read_ahead_kb_store(struct device *dev, struct device_attribute *attr, diff --git a/mm/page-writeback.c b/mm/page-writeback.c index 3e19b87049db..3bb3bed102ef 100644 --- a/mm/page-writeback.c +++ b/mm/page-writeback.c @@ -892,6 +892,25 @@ unsigned long wb_calc_thresh(struct bdi_writeback *wb, unsigned long thresh) return __wb_calc_thresh(&gdtc); } +unsigned long cgwb_calc_thresh(struct bdi_writeback *wb) +{ + struct dirty_throttle_control gdtc = { GDTC_INIT_NO_WB }; + struct dirty_throttle_control mdtc = { MDTC_INIT(wb, &gdtc) }; + unsigned long filepages = 0, headroom = 0, writeback = 0; + + gdtc.avail = global_dirtyable_memory(); + gdtc.dirty = global_node_page_state(NR_FILE_DIRTY) + + global_node_page_state(NR_WRITEBACK); + + mem_cgroup_wb_stats(wb, &filepages, &headroom, + &mdtc.dirty, &writeback); + mdtc.dirty += writeback; + mdtc_calc_avail(&mdtc, filepages, headroom); + domain_dirty_limits(&mdtc); + + return __wb_calc_thresh(&mdtc); +} + /* * setpoint - dirty 3 * f(dirty) := 1.0 + (----------------)