From patchwork Mon Apr 22 16:05:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kemeng Shi X-Patchwork-Id: 13637731 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 69673C4345F for ; Mon, 22 Apr 2024 07:10:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D33206B009C; Mon, 22 Apr 2024 03:10:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CBE026B009D; Mon, 22 Apr 2024 03:10:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B5F416B009F; Mon, 22 Apr 2024 03:10:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 915FB6B009C for ; Mon, 22 Apr 2024 03:10:22 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 4328FC0A32 for ; Mon, 22 Apr 2024 07:10:22 +0000 (UTC) X-FDA: 82036294284.27.9ADD4F4 Received: from dggsgout11.his.huawei.com (unknown [45.249.212.51]) by imf11.hostedemail.com (Postfix) with ESMTP id EE0744000B for ; Mon, 22 Apr 2024 07:10:18 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf11.hostedemail.com: domain of shikemeng@huaweicloud.com designates 45.249.212.51 as permitted sender) smtp.mailfrom=shikemeng@huaweicloud.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1713769820; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references; bh=1tjYsy79oyXOhQYwG6NlI56XnA13MurIJ8fBAyjSJ/s=; b=kEUgqycAEE1jFjoh3l1YVbDLojpxKw5cd9hZw9bD6/5nw56XTho4l0cQQ7foamE47fwGJS 0e0g035pYRQBrNI6ye1+bKv5Kmh6mr4OlRtnlkawOwVk30AVJh5Sn7OqkpVm0BFembAlWr pgo7WNZ/u9urvCIjyb9Ww0TVGgv8UhA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1713769820; a=rsa-sha256; cv=none; b=PpSW3xuEvSQ5/Z5n1tlaK9FLJ3fosmo67GEaT83PJX+k3ft+ZWu8JRopzITccD7gHED11X oZNitauzOB6vf7fwI08xxyJRodN4S+gFrKHI7+7twKFn36+VrXIbrOkt79859TeFnrKMKS CB4lSKP3qoR75dp9TFS85Se3sTjoY6U= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf11.hostedemail.com: domain of shikemeng@huaweicloud.com designates 45.249.212.51 as permitted sender) smtp.mailfrom=shikemeng@huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.216]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTP id 4VNGbc3n2nz4f3l1m for ; Mon, 22 Apr 2024 15:10:04 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.75]) by mail.maildlp.com (Postfix) with ESMTP id A10D51A1050 for ; Mon, 22 Apr 2024 15:10:13 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.101.6]) by APP2 (Coremail) with SMTP id Syh0CgAHdwpTDSZmt5eIKw--.7343S2; Mon, 22 Apr 2024 15:10:13 +0800 (CST) From: Kemeng Shi To: akpm@linux-foundation.org, willy@infradead.org, jack@suse.cz, bfoster@redhat.com, tj@kernel.org Cc: dsterba@suse.com, mjguzik@gmail.com, dhowells@redhat.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Subject: [ATCH v3 0/4] Improve visibility of writeback Date: Tue, 23 Apr 2024 00:05:35 +0800 Message-Id: <20240422160539.3340-1-shikemeng@huaweicloud.com> X-Mailer: git-send-email 2.30.0 MIME-Version: 1.0 X-CM-TRANSID: Syh0CgAHdwpTDSZmt5eIKw--.7343S2 X-Coremail-Antispam: 1UD129KBjvJXoWxWFy5ur1UAF43ZrWUAw47urg_yoW5uw4rpF Z5Aw15tr1UZw1xAFn3Ca42gry5K3y8tFy7Xr9xZrW2q3Z0qr1Dtr95Wa4rAr15C3sayFy3 JFsxZry8Gr4v9F7anT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUvvb4IE77IF4wAFF20E14v26r4j6ryUM7CY07I20VC2zVCF04k2 6cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M280x2IEY4vEnII2IxkI6r1a6r45M2 8lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E 3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F4UJwA2z4x0Y4vEx4A2jsIE14v26r xl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq3wAS0I0E0xvYzxvE52x082IY62kv 0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7IYx2IY67AKxVWUJVWUGwAv7VC2z2 80aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4UM4x0Y48IcxkI7VAKI48JM4IIrI8v 6xkF7I0E8cxan2IY04v7MxAIw28IcxkI7VAKI48JMxC20s026xCaFVCjc4AY6r1j6r4UMI 8I3I0E5I8CrVAFwI0_Jr0_Jr4lx2IqxVCjr7xvwVAFwI0_JrI_JrWlx4CE17CEb7AF67AK xVWUtVW8ZwCIc40Y0x0EwIxGrwCI42IY6xIIjxv20xvE14v26r1j6r1xMIIF0xvE2Ix0cI 8IcVCY1x0267AKxVW8JVWxJwCI42IY6xAIw20EY4v20xvaj40_WFyUJVCq3wCI42IY6I8E 87Iv67AKxVWUJVW8JwCI42IY6I8E87Iv6xkF7I0E14v26r4j6r4UJbIYCTnIWIevJa73Uj IFyTuYvjxUIf-PUUUUU X-CM-SenderInfo: 5vklyvpphqwq5kxd4v5lfo033gof0z/ X-Stat-Signature: 1irf1jed3hdujtadunpsnh3j64e6b9qt X-Rspamd-Queue-Id: EE0744000B X-Rspamd-Server: rspam06 X-Rspam-User: X-HE-Tag: 1713769818-288299 X-HE-Meta: U2FsdGVkX1+bEG/egyMO4pkAmUT00ZjxlbwDcP8KJK0omxRgNqGFhN3PSGhH//kZHcNyWBb0UfMxcAbwkWD+OC/svjfP6IFTjM4IAbf70MAuQPan3+532YPiL2BhHd5mes5lOsmRlCXeP6H9P0DXY8vyh62f23QZIvtSed+Sqd4McnKKxUGLgrWeITRJCFsGjynfNrTAEUVqg7UrqMrFDjXGyT8Q97y8ENpAXFwDWaoyjEt+fRTX8fy+2hl4ZNGGpCtBxf0e4fXV+baTb5Ya9FD63TtdzIf8wTM2p0uEwiWPA0oCHBt0pGK1rZl5o7ew5DpTwaQkkTN1JZfkv+LBQvkwG9n9JEtZyAUfXOfGFAPkRYkJP4iEEyY8trKYur71wxoTiIICwUTLS69GPNI0wii4ByDnlKFIaNivH4I8FpUCRTyrCGWslhhgv3caxoCB95dnUxMMqxXBtqtEdacyuObVrQmO0wtVUa6ICnUUDEwZv9wB7uQt3aUKIfycjAiAnWKvfb1W4pID/idTSGATxUwV/uym6jBZsi11HfV01E789kpvZFmdyo3GFouEc+mJmup15UwQnw9r9TWAtmgKsC4jukrGbvJxwo5mvoiYD9S1ynqNTW85QGZP00Te4636FZ1sT+G/Ml/KpaZGTtdPj3mL5eM0gBmu3NF0P9ApzunaSQYVcrud4hesKcgP1SXw67w959wba4tmLK86SkalL1JguiLstkx88v7aW6zOYJ4RZezsmuEBzK45ZtAI84WwteqrdDGAU8+5svUPod38E5bno0hO1fau0futtShzaKCGp+jA88OeONTbFR1WolSM5XRqjBCQ83WFc3IgCWLzcvOVQDRiIxLyCAwfu0DR+Ru5gyNen2bWNjYc7OiwyuoFF6zQYPAvmlRknUSnfRgPshuLDN+yfTR4KV2qZFzmD3SDbdIpS3Mx6wNd1b74VBeVML1xnJzUe8UXpR6po+a IO+zs+SQ wFzLuUG02lrfBSyFhis1WbOgpXVAr3S6oIZBX/9qleRv6CVV77efL1JOBGpxU+2G7gDvWC4nGJFOvxpwFsmbTyM3EA5jHD9BBF3a8zS3wZTCXE6LaX68OewwP/LWOjgOCFwTFF7b9CcrnV3j+2HUpkQyOmPO+NLb06WuMVPLR5UYrRjeqLNdeapzkQnBmaY3zyC/O X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: v2->v3: -Drop patches to protect non-exist race and to define GDTC_INIT_NO_WB to null. -Add wb_tryget to wb from which we collect stats to bdi stats. -Create wb_stats when CONFIG_CGROUP_WRITEBACK is no enabled. -Add a blank line between two wb stats in wb_stats. v1->v2: -Send cleanup to wq_monitor.py separately. -Add patch to avoid use after free of bdi. -Rename wb_calc_cg_thresh to cgwb_calc_thresh as Tejun suggested. -Use rcu walk to avoid use after free. -Add debug output to each related patches. This series tries to improve visilibity of writeback. Patch 1 make /sys/kernel/debug/bdi/xxx/stats show writeback info of whole bdi instead of only writeback info in root cgroup. Patch 2 add a new debug file /sys/kernel/debug/bdi/xxx/wb_stats to show per wb writeback info. Patch 3 add wb_monitor.py to monitor basic writeback info of running system, more info could be added on demand. Patch 4 is a random cleanup. More details can be found in respective patches. Thanks! Following domain hierarchy is tested: global domain (320G) / \ cgroup domain1(10G) cgroup domain2(10G) | | bdi wb1 wb2 /* all writeback info of bdi is successfully collected */ cat stats BdiWriteback: 4704 kB BdiReclaimable: 1294496 kB BdiDirtyThresh: 204208088 kB DirtyThresh: 195259944 kB BackgroundThresh: 32503588 kB BdiDirtied: 48519296 kB BdiWritten: 47225696 kB BdiWriteBandwidth: 1173892 kBps b_dirty: 1 b_io: 0 b_more_io: 1 b_dirty_time: 0 bdi_list: 1 state: 1 /* per wb writeback info of bdi is collected */ cat /sys/kernel/debug/bdi/252:16/wb_stats WbCgIno: 1 WbWriteback: 0 kB WbReclaimable: 0 kB WbDirtyThresh: 0 kB WbDirtied: 0 kB WbWritten: 0 kB WbWriteBandwidth: 102400 kBps b_dirty: 0 b_io: 0 b_more_io: 0 b_dirty_time: 0 state: 1 WbCgIno: 4208 WbWriteback: 59808 kB WbReclaimable: 676480 kB WbDirtyThresh: 6004624 kB WbDirtied: 23348192 kB WbWritten: 22614592 kB WbWriteBandwidth: 593204 kBps b_dirty: 1 b_io: 1 b_more_io: 0 b_dirty_time: 0 state: 7 WbCgIno: 4249 WbWriteback: 144256 kB WbReclaimable: 432096 kB WbDirtyThresh: 6004344 kB WbDirtied: 25727744 kB WbWritten: 25154752 kB WbWriteBandwidth: 577904 kBps b_dirty: 0 b_io: 1 b_more_io: 0 b_dirty_time: 0 state: 7 The wb_monitor.py script output is as following: ./wb_monitor.py 252:16 -c writeback reclaimable dirtied written avg_bw 252:16_1 0 0 0 0 102400 252:16_4284 672 820064 9230368 8410304 685612 252:16_4325 896 819840 10491264 9671648 652348 252:16 1568 1639904 19721632 18081952 1440360 writeback reclaimable dirtied written avg_bw 252:16_1 0 0 0 0 102400 252:16_4284 672 820064 9230368 8410304 685612 252:16_4325 896 819840 10491264 9671648 652348 252:16 1568 1639904 19721632 18081952 1440360 ... Kemeng Shi (4): writeback: collect stats of all wb of bdi in bdi_debug_stats_show writeback: support retrieving per group debug writeback stats of bdi writeback: add wb_monitor.py script to monitor writeback info on bdi writeback: rename nr_reclaimable to nr_dirty in balance_dirty_pages include/linux/writeback.h | 1 + mm/backing-dev.c | 174 +++++++++++++++++++++++++++++----- mm/page-writeback.c | 27 +++++- tools/writeback/wb_monitor.py | 172 +++++++++++++++++++++++++++++++++ 4 files changed, 345 insertions(+), 29 deletions(-) create mode 100644 tools/writeback/wb_monitor.py