From patchwork Tue Jul 20 13:12:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 12388269 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E09BCC636C8 for ; Tue, 20 Jul 2021 13:13:55 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7895E61186 for ; Tue, 20 Jul 2021 13:13:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7895E61186 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0C5F26B0036; Tue, 20 Jul 2021 09:13:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 09D606B005D; Tue, 20 Jul 2021 09:13:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E59766B006C; Tue, 20 Jul 2021 09:13:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id BD05E6B0036 for ; Tue, 20 Jul 2021 09:13:55 -0400 (EDT) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 43C8D183B95AD for ; Tue, 20 Jul 2021 13:13:54 +0000 (UTC) X-FDA: 78383008788.22.9BE51D2 Received: from mail-qt1-f176.google.com (mail-qt1-f176.google.com [209.85.160.176]) by imf30.hostedemail.com (Postfix) with ESMTP id F18A8E0019B2 for ; Tue, 20 Jul 2021 13:13:53 +0000 (UTC) Received: by mail-qt1-f176.google.com with SMTP id k9so3634679qtx.3 for ; Tue, 20 Jul 2021 06:13:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=vycya5wChR8O11SPvwfrbgcahHWj2DvmeexI6w+aeRE=; b=AXjurdksSSyEZKFZ4dwI7bqIy7rFRNuiCDsvXU0YkuocK0QEXajhTDLdW4z1z/e65T LTLEY6rfvFw2zhW9t6YjY7pJAjqcinHqBHc+fLPFdMRjVhHRZY3JEDBzx0pGr0y45zCU iMbw3FsUPACZLzgC4Ivq0MGqY+ZiR2YUf2t8qzEmccEijSZfWJAdOZQw6+fudK9PdaVL vHFg5lAVpphObFQG7UQNG2Y7Zo2Wq6azf1aFxNlccsJv0JcY+wecDmxG65lP2Bd2K52V 0l/f7AZ2VcijTD9MfXEdPXS0oRr+XOwZHi1mI/PTW7FAtdTPFRyz8UwmWuFcQwEhzPy9 otjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=vycya5wChR8O11SPvwfrbgcahHWj2DvmeexI6w+aeRE=; b=Nznq2WKgEEtNu3UjXz9gBn/zUkkCW2gXMBQ7Jl1TCxB2sY9VEWQOvS0pTC/u3KntDu 0vVMmqw29Er18R49AI/i6SsbUHNL6AOkM8Vya04+5yYUMb6O4djw9EMYovl3S7x9ZWio 4bfpB4vDox1+Wkkp8pqvVZBH473+ww4W9FUaI0XFqwoCr/7c6HJB53Ux0UX08u9X6+Rh S/DiwhF1LIZeyFQRNxgHu13lL+2WlHGXGyKPdfQcCEF1T4lWI8Juqatm8NcVywol86C7 fwMuwqZM2IZZQcAdF1qrS7/3oaSzuPNuKybaWVAeEmKaHPQqMA73NsrKRrJwjUDlETN4 6/+Q== X-Gm-Message-State: AOAM531/3+U4A29flFwe5oLhdks20PiBZKI0zvazg4Be0Og95lJarpBR QG5IUKFBJOUmoRgdr2VQ8Bc= X-Google-Smtp-Source: ABdhPJy0LjG6uDWiu+ZbyuX11opxgLX2i0q6/CnN0YojkXcrGAiVtf0xl2dZoqfsPzdIQwBuOwt9DA== X-Received: by 2002:ac8:7645:: with SMTP id i5mr26265284qtr.133.1626786833321; Tue, 20 Jul 2021 06:13:53 -0700 (PDT) Received: from localhost.localdomain (ec2-35-169-212-159.compute-1.amazonaws.com. [35.169.212.159]) by smtp.gmail.com with ESMTPSA id g17sm9701225qkm.34.2021.07.20.06.13.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Jul 2021 06:13:52 -0700 (PDT) From: SeongJae Park To: akpm@linux-foundation.org Cc: SeongJae Park , Jonathan.Cameron@Huawei.com, acme@kernel.org, alexander.shishkin@linux.intel.com, amit@kernel.org, benh@kernel.crashing.org, brendanhiggins@google.com, corbet@lwn.net, david@redhat.com, dwmw@amazon.com, elver@google.com, fan.du@intel.com, foersleo@amazon.de, greg@kroah.com, gthelen@google.com, guoju.fgj@alibaba-inc.com, jgowans@amazon.com, joe@perches.com, mgorman@suse.de, mheyne@amazon.de, minchan@kernel.org, mingo@redhat.com, namhyung@kernel.org, peterz@infradead.org, riel@surriel.com, rientjes@google.com, rostedt@goodmis.org, rppt@kernel.org, shakeelb@google.com, shuah@kernel.org, sieberf@amazon.com, sj38.park@gmail.com, snu@zelle79.org, vbabka@suse.cz, vdavydov.dev@gmail.com, zgf574564920@gmail.com, linux-damon@amazon.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [RFC v3 01/15] mm/damon/paddr: Support the pageout scheme Date: Tue, 20 Jul 2021 13:12:55 +0000 Message-Id: <20210720131309.22073-2-sj38.park@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210720131309.22073-1-sj38.park@gmail.com> References: <20210720131309.22073-1-sj38.park@gmail.com> Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=AXjurdks; spf=pass (imf30.hostedemail.com: domain of sj38park@gmail.com designates 209.85.160.176 as permitted sender) smtp.mailfrom=sj38park@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Stat-Signature: djuhmfwpemo3w4zgc6hxjucaac878d1m X-Rspamd-Queue-Id: F18A8E0019B2 X-Rspamd-Server: rspam01 X-HE-Tag: 1626786833-168765 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park This commit makes the DAMON primitives for physical address space to support the pageout action for DAMON-based Operation Schemes. IOW, now the users can implement their own data access-aware reclamations for whole system using DAMOS. Signed-off-by: SeongJae Park --- mm/damon/paddr.c | 38 +++++++++++++++++++++++++++++++++++++- mm/damon/prmtv-common.c | 2 +- mm/damon/prmtv-common.h | 2 ++ 3 files changed, 40 insertions(+), 2 deletions(-) diff --git a/mm/damon/paddr.c b/mm/damon/paddr.c index b92b07a3ce53..303db372e53b 100644 --- a/mm/damon/paddr.c +++ b/mm/damon/paddr.c @@ -7,6 +7,9 @@ #define pr_fmt(fmt) "damon-pa: " fmt +#include + +#include "../internal.h" #include "prmtv-common.h" /* @@ -85,6 +88,39 @@ bool damon_pa_target_valid(void *t) return true; } +int damon_pa_apply_scheme(struct damon_ctx *ctx, struct damon_target *t, + struct damon_region *r, struct damos *scheme) +{ + unsigned long addr; + LIST_HEAD(page_list); + + if (scheme->action != DAMOS_PAGEOUT) + return -EINVAL; + + for (addr = r->ar.start; addr < r->ar.end; addr += PAGE_SIZE) { + struct page *page = damon_get_page(PHYS_PFN(addr)); + + if (!page) + continue; + + ClearPageReferenced(page); + test_and_clear_page_young(page); + if (isolate_lru_page(page)) { + put_page(page); + continue; + } + if (PageUnevictable(page)) { + putback_lru_page(page); + } else { + list_add(&page->lru, &page_list); + put_page(page); + } + } + reclaim_pages(&page_list); + cond_resched(); + return 0; +} + void damon_pa_set_primitives(struct damon_ctx *ctx) { ctx->primitive.init = NULL; @@ -94,5 +130,5 @@ void damon_pa_set_primitives(struct damon_ctx *ctx) ctx->primitive.reset_aggregated = NULL; ctx->primitive.target_valid = damon_pa_target_valid; ctx->primitive.cleanup = NULL; - ctx->primitive.apply_scheme = NULL; + ctx->primitive.apply_scheme = damon_pa_apply_scheme; } diff --git a/mm/damon/prmtv-common.c b/mm/damon/prmtv-common.c index 08e9318d67ed..01c1c1b37859 100644 --- a/mm/damon/prmtv-common.c +++ b/mm/damon/prmtv-common.c @@ -14,7 +14,7 @@ * The body of this function is stolen from the 'page_idle_get_page()'. We * steal rather than reuse it because the code is quite simple. */ -static struct page *damon_get_page(unsigned long pfn) +struct page *damon_get_page(unsigned long pfn) { struct page *page = pfn_to_online_page(pfn); diff --git a/mm/damon/prmtv-common.h b/mm/damon/prmtv-common.h index 939c41af6b59..ba0c4eecbb79 100644 --- a/mm/damon/prmtv-common.h +++ b/mm/damon/prmtv-common.h @@ -18,6 +18,8 @@ /* Get a random number in [l, r) */ #define damon_rand(l, r) (l + prandom_u32_max(r - l)) +struct page *damon_get_page(unsigned long pfn); + void damon_va_mkold(struct mm_struct *mm, unsigned long addr); bool damon_va_young(struct mm_struct *mm, unsigned long addr, unsigned long *page_sz); From patchwork Tue Jul 20 13:12:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 12388271 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D82DFC07E95 for ; Tue, 20 Jul 2021 13:14:00 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 58709610CC for ; Tue, 20 Jul 2021 13:14:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 58709610CC Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 139C76B005D; Tue, 20 Jul 2021 09:14:01 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1144D6B006C; Tue, 20 Jul 2021 09:14:01 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EA7136B0070; Tue, 20 Jul 2021 09:14:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0059.hostedemail.com [216.40.44.59]) by kanga.kvack.org (Postfix) with ESMTP id C30ED6B005D for ; Tue, 20 Jul 2021 09:14:00 -0400 (EDT) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 52CF61845707B for ; Tue, 20 Jul 2021 13:13:59 +0000 (UTC) X-FDA: 78383008998.24.FA78012 Received: from mail-qt1-f171.google.com (mail-qt1-f171.google.com [209.85.160.171]) by imf21.hostedemail.com (Postfix) with ESMTP id 04E32D00B7D9 for ; Tue, 20 Jul 2021 13:13:56 +0000 (UTC) Received: by mail-qt1-f171.google.com with SMTP id w26so15270573qto.9 for ; Tue, 20 Jul 2021 06:13:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=eLmnd6iZieuSdj2C5koxMI8ybXWCYl9uIJX/s9kWwgk=; b=SPESqdfw1g2LlbIqfwQWGMNAqLDDUpO9U/iALe+hhuZKKmYbrK6PoXtSbYzvoBto8j Mqb6PVx2ytiFguZeAmDxUzvhmXWsnlm7KrW9/4IqDcE2dgaY/pXXka1KKl8fH5vJUDmd Qw/1lE+4gM2ENKmc1/Ns0LJn6Jlj8zV/05wJGydGwpGCrdeYp0tMVz1hPKpnVqwtChoh PQU7fa8e/2oYPO38SRIpBypj903wQkE3FAxl2aa17AOFAjiH09gs3GwY8COqJ260Pu5T 2yJQmp1k1fxahEVxy1BGsS281WSgU415q42LAUe76lhlJhIV+V5TgrzXZpDLq6b+n6IH Pi/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=eLmnd6iZieuSdj2C5koxMI8ybXWCYl9uIJX/s9kWwgk=; b=dKLYB0wndnxA8RX5/sPWPeB03pmW9R0wTdFKdXx88z/LsFS0fMKhFcbmQVV9w1dncd QyKasQsX4sQz8V5lrmiE8jyQXacOigY1hjyaQJhILr2Zqd/ymKIXcBFDrlnhQx09xhK+ iXS1HDpZu0gE6ZNsUgJnWdkWKxRTwrPDHWJRKJkEx+RTDnp8isB8jArSwCUEOxxo7dWS u0cOQFOz8c5i+hXGIqZAW2RmkMU162Lo6Qm+29O5zypYGnuB8qPZ8TaHYZsnK3gDL/7z Uze6uunjsHviQoKIwrLfTrNr069jpe3yJSaCaBhceU2n+b4PJS0OMK6EfAGr00HhsLwI Fncg== X-Gm-Message-State: AOAM533nc/fB/pBlQta+Oe64jQ9NwXYNO+y0vtILppGwnRHuaSEd/TAy uGxS/hHhcugIsAx/Oow7GKo= X-Google-Smtp-Source: ABdhPJyElUl/ZW/jTKmwGkPtCxiExGimBuugukAYvyfYBT3iFiQT3lMQSoR5kNXpchEQELOw+NNtrw== X-Received: by 2002:ac8:7f07:: with SMTP id f7mr25960026qtk.120.1626786836336; Tue, 20 Jul 2021 06:13:56 -0700 (PDT) Received: from localhost.localdomain (ec2-35-169-212-159.compute-1.amazonaws.com. [35.169.212.159]) by smtp.gmail.com with ESMTPSA id g17sm9701225qkm.34.2021.07.20.06.13.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Jul 2021 06:13:55 -0700 (PDT) From: SeongJae Park To: akpm@linux-foundation.org Cc: SeongJae Park , Jonathan.Cameron@Huawei.com, acme@kernel.org, alexander.shishkin@linux.intel.com, amit@kernel.org, benh@kernel.crashing.org, brendanhiggins@google.com, corbet@lwn.net, david@redhat.com, dwmw@amazon.com, elver@google.com, fan.du@intel.com, foersleo@amazon.de, greg@kroah.com, gthelen@google.com, guoju.fgj@alibaba-inc.com, jgowans@amazon.com, joe@perches.com, mgorman@suse.de, mheyne@amazon.de, minchan@kernel.org, mingo@redhat.com, namhyung@kernel.org, peterz@infradead.org, riel@surriel.com, rientjes@google.com, rostedt@goodmis.org, rppt@kernel.org, shakeelb@google.com, shuah@kernel.org, sieberf@amazon.com, sj38.park@gmail.com, snu@zelle79.org, vbabka@suse.cz, vdavydov.dev@gmail.com, zgf574564920@gmail.com, linux-damon@amazon.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [RFC v3 02/15] mm/damon/damos: Make schemes aggressiveness controllable Date: Tue, 20 Jul 2021 13:12:56 +0000 Message-Id: <20210720131309.22073-3-sj38.park@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210720131309.22073-1-sj38.park@gmail.com> References: <20210720131309.22073-1-sj38.park@gmail.com> Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=SPESqdfw; spf=pass (imf21.hostedemail.com: domain of sj38park@gmail.com designates 209.85.160.171 as permitted sender) smtp.mailfrom=sj38park@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspamd-Server: rspam02 X-Stat-Signature: 1bhmfgi3ck9wer16ehaefcxneepac4rk X-Rspamd-Queue-Id: 04E32D00B7D9 X-HE-Tag: 1626786836-650626 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park If there are too large memory regions fulfilling the target data access pattern of a DAMON-based operation scheme, applying the action of the scheme could consume too much CPU. To avoid that, this commit implements a limit for the action application speed. Using the feature, the client can set up to how much amount of memory regions the action could applied within specific time duration. Signed-off-by: SeongJae Park --- include/linux/damon.h | 36 +++++++++++++++++++++++--- mm/damon/core.c | 60 +++++++++++++++++++++++++++++++++++++------ mm/damon/dbgfs.c | 4 ++- 3 files changed, 87 insertions(+), 13 deletions(-) diff --git a/include/linux/damon.h b/include/linux/damon.h index 6eb265717fd4..9c996adb02b8 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -89,6 +89,26 @@ enum damos_action { DAMOS_STAT, /* Do nothing but only record the stat */ }; +/** + * struct damos_quota - Controls the aggressiveness of the given scheme. + * @sz: Maximum bytes of memory that the action can be applied. + * @reset_interval: Charge reset interval in milliseconds. + * + * To avoid consuming too much CPU time or IO resources for applying the + * &struct damos->action to large memory, DAMON allows users to set a size + * quota. The quota can be set by writing non-zero values to &sz. If the size + * quota is set, DAMON tries to apply the action only up to &sz bytes within + * &reset_interval. + */ +struct damos_quota { + unsigned long sz; + unsigned long reset_interval; + +/* private: For charging the quota */ + unsigned long charged_sz; + unsigned long charged_from; +}; + /** * struct damos - Represents a Data Access Monitoring-based Operation Scheme. * @min_sz_region: Minimum size of target regions. @@ -98,13 +118,20 @@ enum damos_action { * @min_age_region: Minimum age of target regions. * @max_age_region: Maximum age of target regions. * @action: &damo_action to be applied to the target regions. + * @quota: Control the aggressiveness of this scheme. * @stat_count: Total number of regions that this scheme is applied. * @stat_sz: Total size of regions that this scheme is applied. * @list: List head for siblings. * - * For each aggregation interval, DAMON applies @action to monitoring target - * regions fit in the condition and updates the statistics. Note that both - * the minimums and the maximums are inclusive. + * For each aggregation interval, DAMON finds regions which fit in the + * condition (&min_sz_region, &max_sz_region, &min_nr_accesses, + * &max_nr_accesses, &min_age_region, &max_age_region) and applies &action to + * those. To avoid consuming too much CPU time or IO resources for the + * &action, "a is used. + * + * After applying the &action to each region, &stat_count and &stat_sz is + * updated to reflect the number of regions and total size of regions that the + * &action is applied. */ struct damos { unsigned long min_sz_region; @@ -114,6 +141,7 @@ struct damos { unsigned int min_age_region; unsigned int max_age_region; enum damos_action action; + struct damos_quota quota; unsigned long stat_count; unsigned long stat_sz; struct list_head list; @@ -338,7 +366,7 @@ struct damos *damon_new_scheme( unsigned long min_sz_region, unsigned long max_sz_region, unsigned int min_nr_accesses, unsigned int max_nr_accesses, unsigned int min_age_region, unsigned int max_age_region, - enum damos_action action); + enum damos_action action, struct damos_quota *quota); void damon_add_scheme(struct damon_ctx *ctx, struct damos *s); void damon_destroy_scheme(struct damos *s); diff --git a/mm/damon/core.c b/mm/damon/core.c index c45827e853cf..00804a1e5e2a 100644 --- a/mm/damon/core.c +++ b/mm/damon/core.c @@ -89,7 +89,7 @@ struct damos *damon_new_scheme( unsigned long min_sz_region, unsigned long max_sz_region, unsigned int min_nr_accesses, unsigned int max_nr_accesses, unsigned int min_age_region, unsigned int max_age_region, - enum damos_action action) + enum damos_action action, struct damos_quota *quota) { struct damos *scheme; @@ -107,6 +107,11 @@ struct damos *damon_new_scheme( scheme->stat_sz = 0; INIT_LIST_HEAD(&scheme->list); + scheme->quota.sz = quota->sz; + scheme->quota.reset_interval = quota->reset_interval; + scheme->quota.charged_sz = 0; + scheme->quota.charged_from = 0; + return scheme; } @@ -535,15 +540,25 @@ static void kdamond_reset_aggregated(struct damon_ctx *c) } } +static void damon_split_region_at(struct damon_ctx *ctx, + struct damon_target *t, struct damon_region *r, + unsigned long sz_r); + static void damon_do_apply_schemes(struct damon_ctx *c, struct damon_target *t, struct damon_region *r) { struct damos *s; - unsigned long sz; damon_for_each_scheme(s, c) { - sz = r->ar.end - r->ar.start; + struct damos_quota *quota = &s->quota; + unsigned long sz = r->ar.end - r->ar.start; + + /* Check the quota */ + if (quota->sz && quota->charged_sz >= quota->sz) + continue; + + /* Check the target regions condition */ if (sz < s->min_sz_region || s->max_sz_region < sz) continue; if (r->nr_accesses < s->min_nr_accesses || @@ -551,22 +566,51 @@ static void damon_do_apply_schemes(struct damon_ctx *c, continue; if (r->age < s->min_age_region || s->max_age_region < r->age) continue; - s->stat_count++; - s->stat_sz += sz; - if (c->primitive.apply_scheme) + + /* Apply the scheme */ + if (c->primitive.apply_scheme) { + if (quota->sz && quota->charged_sz + sz > quota->sz) { + sz = ALIGN_DOWN(quota->sz - quota->charged_sz, + DAMON_MIN_REGION); + if (!sz) + goto update_stat; + damon_split_region_at(c, t, r, sz); + } c->primitive.apply_scheme(c, t, r, s); + quota->charged_sz += sz; + } if (s->action != DAMOS_STAT) r->age = 0; + +update_stat: + s->stat_count++; + s->stat_sz += sz; } } static void kdamond_apply_schemes(struct damon_ctx *c) { struct damon_target *t; - struct damon_region *r; + struct damon_region *r, *next_r; + struct damos *s; + + damon_for_each_scheme(s, c) { + struct damos_quota *quota = &s->quota; + + if (!quota->sz) + continue; + + /* New charge window starts */ + if (time_after_eq(jiffies, quota->charged_from + + msecs_to_jiffies( + quota->reset_interval))) { + quota->charged_from = jiffies; + quota->charged_sz = 0; + } + } damon_for_each_target(t, c) { - damon_for_each_region(r, t) + damon_for_each_region_safe(r, next_r, t) damon_do_apply_schemes(c, t, r); } } diff --git a/mm/damon/dbgfs.c b/mm/damon/dbgfs.c index c244e7689c70..ac0de2de1987 100644 --- a/mm/damon/dbgfs.c +++ b/mm/damon/dbgfs.c @@ -310,6 +310,8 @@ static struct damos **str_to_schemes(const char *str, ssize_t len, *nr_schemes = 0; while (pos < len && *nr_schemes < max_nr_schemes) { + struct damos_quota quota = {}; + ret = sscanf(&str[pos], "%lu %lu %u %u %u %u %u%n", &min_sz, &max_sz, &min_nr_a, &max_nr_a, &min_age, &max_age, &action, &parsed); @@ -322,7 +324,7 @@ static struct damos **str_to_schemes(const char *str, ssize_t len, pos += parsed; scheme = damon_new_scheme(min_sz, max_sz, min_nr_a, max_nr_a, - min_age, max_age, action); + min_age, max_age, action, "a); if (!scheme) goto fail; From patchwork Tue Jul 20 13:12:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 12388273 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4E7C8C07E95 for ; Tue, 20 Jul 2021 13:14:03 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 092DE61164 for ; Tue, 20 Jul 2021 13:14:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 092DE61164 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id BC1506B006C; Tue, 20 Jul 2021 09:14:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B721A6B0070; Tue, 20 Jul 2021 09:14:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9EC3A6B0071; Tue, 20 Jul 2021 09:14:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 79FAB6B006C for ; Tue, 20 Jul 2021 09:14:03 -0400 (EDT) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id DEA5A1846CA04 for ; Tue, 20 Jul 2021 13:14:01 +0000 (UTC) X-FDA: 78383009082.24.2142C43 Received: from mail-qt1-f182.google.com (mail-qt1-f182.google.com [209.85.160.182]) by imf16.hostedemail.com (Postfix) with ESMTP id 8F580F000097 for ; Tue, 20 Jul 2021 13:14:01 +0000 (UTC) Received: by mail-qt1-f182.google.com with SMTP id d1so15315194qto.4 for ; Tue, 20 Jul 2021 06:14:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=d+FN8BCXq1Njo+3jWyl699IvNI4knnaapCBiI4jANDs=; b=sKV3Sub0ZWw3J1A9H/aM4MCg5TP6Cvdgb2f7j3O/vH1twUs7qUxGwfjlPc0VoLhBhf 2S3YMwYU+fYaMmkQNYqu6QhhxGxlprKZxcsoJ22fQ11snxI27MW2ZXHJJ3neqsKVeuIH zvkcT5OIscQO/gbb9s48YC+nIUdo8hh7dEqm9htIbKh1bW/jTrqZr726zllpFT9yRspS 2IFTgDPtJt2075MHXgPyv5K1a2C6NuShpJ5v72Cpmp838BXVuFtabI3s8TQjHghL6K78 ukovvHU4whQsk5WpK0lihTKtn4eKBwysGwHFlEI8DHeVsSR3ryLtVE2YaWMRq8gt/bOD mIrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=d+FN8BCXq1Njo+3jWyl699IvNI4knnaapCBiI4jANDs=; b=Zgqd5WVxuZDI/01JFjoS7ump9mz8J1Z/ExxvFqsHCljvDYpj00Q2GD3HalhJRvsOGK KQNN4EBMtNt2LH8QHjSxHLNxHsFrBm/iVFejwehCxKHlBxj7AYEI1tJf5c+R8erBtKSG qkTJ0p05g9XgNyEHZzD/M3+bUOXMRKAwH51eX4c6eZDkROG80F7bdBq2IQgh6ZA9bmg7 4LEm83kyurLP989Zd6BOFy0s41AV1Oi4Ce68KHnDiaG8flIrPafbFc9cLYtH92o3gY4f GEKYHIsl4FRsbhyol+KjUdDWTtmFY+j7jbXjUImNgz3Vvv8Xp21aXX9BFuLM5Vn7tCNn n7iw== X-Gm-Message-State: AOAM533QjT6tgu0GOrIUFOvhb3vsOyMExcqVcx7griMCF3EN32bDxP3b l8x7ZCHaij0XrOu1EC24uAQ= X-Google-Smtp-Source: ABdhPJya1WocZ2Abu2uH1JFsFZnGgG+TUYEiCkCCCkaOrXAB7pUilYbjSriLMVokCmVpZDpmD3CNpw== X-Received: by 2002:a05:622a:207:: with SMTP id b7mr19490392qtx.377.1626786840957; Tue, 20 Jul 2021 06:14:00 -0700 (PDT) Received: from localhost.localdomain (ec2-35-169-212-159.compute-1.amazonaws.com. [35.169.212.159]) by smtp.gmail.com with ESMTPSA id g17sm9701225qkm.34.2021.07.20.06.13.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Jul 2021 06:14:00 -0700 (PDT) From: SeongJae Park To: akpm@linux-foundation.org Cc: SeongJae Park , Jonathan.Cameron@Huawei.com, acme@kernel.org, alexander.shishkin@linux.intel.com, amit@kernel.org, benh@kernel.crashing.org, brendanhiggins@google.com, corbet@lwn.net, david@redhat.com, dwmw@amazon.com, elver@google.com, fan.du@intel.com, foersleo@amazon.de, greg@kroah.com, gthelen@google.com, guoju.fgj@alibaba-inc.com, jgowans@amazon.com, joe@perches.com, mgorman@suse.de, mheyne@amazon.de, minchan@kernel.org, mingo@redhat.com, namhyung@kernel.org, peterz@infradead.org, riel@surriel.com, rientjes@google.com, rostedt@goodmis.org, rppt@kernel.org, shakeelb@google.com, shuah@kernel.org, sieberf@amazon.com, sj38.park@gmail.com, snu@zelle79.org, vbabka@suse.cz, vdavydov.dev@gmail.com, zgf574564920@gmail.com, linux-damon@amazon.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [RFC v3 03/15] damon/core/schemes: Skip already charged targets and regions Date: Tue, 20 Jul 2021 13:12:57 +0000 Message-Id: <20210720131309.22073-4-sj38.park@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210720131309.22073-1-sj38.park@gmail.com> References: <20210720131309.22073-1-sj38.park@gmail.com> Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=sKV3Sub0; spf=pass (imf16.hostedemail.com: domain of sj38park@gmail.com designates 209.85.160.182 as permitted sender) smtp.mailfrom=sj38park@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspamd-Server: rspam05 X-Stat-Signature: 4ur78z8qyjtgrfuf1s3h9gyjng3u4msg X-Rspamd-Queue-Id: 8F580F000097 X-HE-Tag: 1626786841-76143 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park If DAMOS stopped applying action to memory regions due to the speed limit, it does nothing until next charge window starts. Then, it starts the work from the beginning of the address space. If there is a huge memory region at the beginning of the address space and it fulfills the scheme target data access pattern, the action will applied to only the region. This commit mitigates the case by skipping memory regions that charged in previous charge window at the beginning of current charge window. Signed-off-by: SeongJae Park --- include/linux/damon.h | 5 +++++ mm/damon/core.c | 37 +++++++++++++++++++++++++++++++++++++ 2 files changed, 42 insertions(+) diff --git a/include/linux/damon.h b/include/linux/damon.h index 9c996adb02b8..7b1fa506e7a6 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -107,6 +107,8 @@ struct damos_quota { /* private: For charging the quota */ unsigned long charged_sz; unsigned long charged_from; + struct damon_target *charge_target_from; + unsigned long charge_addr_from; }; /** @@ -335,6 +337,9 @@ struct damon_ctx { #define damon_prev_region(r) \ (container_of(r->list.prev, struct damon_region, list)) +#define damon_last_region(t) \ + (list_last_entry(&t->regions_list, struct damon_region, list)) + #define damon_for_each_region(r, t) \ list_for_each_entry(r, &t->regions_list, list) diff --git a/mm/damon/core.c b/mm/damon/core.c index 00804a1e5e2a..a41eb9d885bb 100644 --- a/mm/damon/core.c +++ b/mm/damon/core.c @@ -111,6 +111,8 @@ struct damos *damon_new_scheme( scheme->quota.reset_interval = quota->reset_interval; scheme->quota.charged_sz = 0; scheme->quota.charged_from = 0; + scheme->quota.charge_target_from = NULL; + scheme->quota.charge_addr_from = 0; return scheme; } @@ -558,6 +560,37 @@ static void damon_do_apply_schemes(struct damon_ctx *c, if (quota->sz && quota->charged_sz >= quota->sz) continue; + /* Skip previously charged regions */ + if (quota->charge_target_from) { + if (t != quota->charge_target_from) + continue; + if (r == damon_last_region(t)) { + quota->charge_target_from = NULL; + quota->charge_addr_from = 0; + continue; + } + if (quota->charge_addr_from && + r->ar.end <= quota->charge_addr_from) + continue; + + if (quota->charge_addr_from && r->ar.start < + quota->charge_addr_from) { + sz = ALIGN_DOWN(quota->charge_addr_from - + r->ar.start, DAMON_MIN_REGION); + if (!sz) { + if (r->ar.end - r->ar.start <= + DAMON_MIN_REGION) + continue; + sz = DAMON_MIN_REGION; + } + damon_split_region_at(c, t, r, sz); + r = damon_next_region(r); + sz = r->ar.end - r->ar.start; + } + quota->charge_target_from = NULL; + quota->charge_addr_from = 0; + } + /* Check the target regions condition */ if (sz < s->min_sz_region || s->max_sz_region < sz) continue; @@ -578,6 +611,10 @@ static void damon_do_apply_schemes(struct damon_ctx *c, } c->primitive.apply_scheme(c, t, r, s); quota->charged_sz += sz; + if (quota->sz && quota->charged_sz >= quota->sz) { + quota->charge_target_from = t; + quota->charge_addr_from = r->ar.end + 1; + } } if (s->action != DAMOS_STAT) r->age = 0; From patchwork Tue Jul 20 13:12:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 12388275 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 01ED0C636C8 for ; Tue, 20 Jul 2021 13:14:09 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8792A610CC for ; Tue, 20 Jul 2021 13:14:07 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8792A610CC Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 447B66B0070; Tue, 20 Jul 2021 09:14:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 41FFA6B0071; Tue, 20 Jul 2021 09:14:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2996A6B0072; Tue, 20 Jul 2021 09:14:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0146.hostedemail.com [216.40.44.146]) by kanga.kvack.org (Postfix) with ESMTP id 040B66B0070 for ; Tue, 20 Jul 2021 09:14:07 -0400 (EDT) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 8009A8248047 for ; Tue, 20 Jul 2021 13:14:06 +0000 (UTC) X-FDA: 78383009292.04.C6F0648 Received: from mail-qk1-f176.google.com (mail-qk1-f176.google.com [209.85.222.176]) by imf09.hostedemail.com (Postfix) with ESMTP id 379313000109 for ; Tue, 20 Jul 2021 13:14:06 +0000 (UTC) Received: by mail-qk1-f176.google.com with SMTP id bm6so7132770qkb.1 for ; Tue, 20 Jul 2021 06:14:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=YceUl9gK1f+DUw5lLJnUOuiqygVMuUWiniuXFoUnVS4=; b=Ieb+Cvc3zbFZSyAoPxvbLvuq08w5aQSh69ss+eMxt4RYknpj0ExKOY/oKTfWFturrB E43GJ5eaTG976f21rtJ2nnBi/lEBxkiGThyU+u5BRB2pt7lbhLThFDtmOfkK68KXQqZG 59Lr+XBFW7kK5PtlDmyapxVAAzXxTyH4Do5B6vxse4+xuC0N3XzrxvXZafCr8CTJiTPb i1rh9+JKUi10lC7hDFaX9Weh/nO48gwFA0LEAwbt1aSOClkYCVQFQFWTjGBJy3EkuOIz ISttkTcsq1km+U9TyI8B447iB6Qpwy1KRklTgzmdyNNnj8WjdAkhmTzO1Jjv/kxxvD3W tAJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=YceUl9gK1f+DUw5lLJnUOuiqygVMuUWiniuXFoUnVS4=; b=FA03FX/Kr6Zutq3TLh62pAY50b6zfiibGshK7BcbyRd8E3AFCULJ0z13mIN8DFcV+f 2NWYcR+Vku+qHWK6wMJbgtK6L/CXoBzoyXQXXd3YPf3VyUZMPgVPHyN5tvxePh+XwZTt pjQiy5ixv6Sd6L+2jRMgR7zWhQxTVHb5aJrr5q5+idAvbo0f2eGPJPVEEgXuztf+p60O qHmQ4kvXA59P81qL4IklBB6gqvV6CmPpuNdE96/PgO+gAEIETAzLzWEqyotB2hPAk0ed 6uvLwFaNaFbp4zJ0QVMSv2q/s1uhf+6KpdyDiUx3EEszYhFIUYavuR/30nvIji7aL3m/ Ak6w== X-Gm-Message-State: AOAM530xdtkrnfLZaLYWRDYP/q3/f3mQ7NczzAlWBUrgeV5+azWKgfAz /NJWcqKrORUfYxkc40V4fMo= X-Google-Smtp-Source: ABdhPJzdTEEl3SbX2pZcS9sDsNYjQP7I8ks/LX6G+x/G+B90qTjGmlMr22grI2h627KFW3x/REOsyQ== X-Received: by 2002:a05:620a:629:: with SMTP id 9mr15185106qkv.501.1626786845624; Tue, 20 Jul 2021 06:14:05 -0700 (PDT) Received: from localhost.localdomain (ec2-35-169-212-159.compute-1.amazonaws.com. [35.169.212.159]) by smtp.gmail.com with ESMTPSA id g17sm9701225qkm.34.2021.07.20.06.14.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Jul 2021 06:14:05 -0700 (PDT) From: SeongJae Park To: akpm@linux-foundation.org Cc: SeongJae Park , Jonathan.Cameron@Huawei.com, acme@kernel.org, alexander.shishkin@linux.intel.com, amit@kernel.org, benh@kernel.crashing.org, brendanhiggins@google.com, corbet@lwn.net, david@redhat.com, dwmw@amazon.com, elver@google.com, fan.du@intel.com, foersleo@amazon.de, greg@kroah.com, gthelen@google.com, guoju.fgj@alibaba-inc.com, jgowans@amazon.com, joe@perches.com, mgorman@suse.de, mheyne@amazon.de, minchan@kernel.org, mingo@redhat.com, namhyung@kernel.org, peterz@infradead.org, riel@surriel.com, rientjes@google.com, rostedt@goodmis.org, rppt@kernel.org, shakeelb@google.com, shuah@kernel.org, sieberf@amazon.com, sj38.park@gmail.com, snu@zelle79.org, vbabka@suse.cz, vdavydov.dev@gmail.com, zgf574564920@gmail.com, linux-damon@amazon.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [RFC v3 04/15] mm/damon/schemes: Implement time quota Date: Tue, 20 Jul 2021 13:12:58 +0000 Message-Id: <20210720131309.22073-5-sj38.park@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210720131309.22073-1-sj38.park@gmail.com> References: <20210720131309.22073-1-sj38.park@gmail.com> Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=Ieb+Cvc3; spf=pass (imf09.hostedemail.com: domain of sj38park@gmail.com designates 209.85.222.176 as permitted sender) smtp.mailfrom=sj38park@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Stat-Signature: faeub89y7u3ratqezapwmeombr57rqb8 X-Rspamd-Queue-Id: 379313000109 X-Rspamd-Server: rspam01 X-HE-Tag: 1626786846-893783 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park This commit implements time-based quota for DAMON-based Operation Schemes. If the quota is set, DAMOS tries to use only up to the user-defined quota within the 'reset_interval' milliseconds. Signed-off-by: SeongJae Park --- include/linux/damon.h | 25 +++++++++++++++++++----- mm/damon/core.c | 45 ++++++++++++++++++++++++++++++++++++++----- 2 files changed, 60 insertions(+), 10 deletions(-) diff --git a/include/linux/damon.h b/include/linux/damon.h index 7b1fa506e7a6..d2dd36b9dd6c 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -91,20 +91,35 @@ enum damos_action { /** * struct damos_quota - Controls the aggressiveness of the given scheme. + * @ms: Maximum milliseconds that the scheme can use. * @sz: Maximum bytes of memory that the action can be applied. * @reset_interval: Charge reset interval in milliseconds. * * To avoid consuming too much CPU time or IO resources for applying the - * &struct damos->action to large memory, DAMON allows users to set a size - * quota. The quota can be set by writing non-zero values to &sz. If the size - * quota is set, DAMON tries to apply the action only up to &sz bytes within - * &reset_interval. + * &struct damos->action to large memory, DAMON allows users to set time and/or + * size quotas. The quotas can be set by writing non-zero values to &ms and + * &sz, respectively. If the time quota is set, DAMON tries to use only up to + * &ms milliseconds within &reset_interval for applying the action. If the + * size quota is set, DAMON tries to apply the action only up to &sz bytes + * within &reset_interval. + * + * Internally, the time quota is transformed to a size quota using estimated + * throughput of the scheme's action. DAMON then compares it against &sz and + * uses smaller one as the effective quota. */ struct damos_quota { + unsigned long ms; unsigned long sz; unsigned long reset_interval; -/* private: For charging the quota */ +/* private: */ + /* For throughput estimation */ + unsigned long total_charged_sz; + unsigned long total_charged_ns; + + unsigned long esz; /* Effective size quota in bytes */ + + /* For charging the quota */ unsigned long charged_sz; unsigned long charged_from; struct damon_target *charge_target_from; diff --git a/mm/damon/core.c b/mm/damon/core.c index a41eb9d885bb..321523604ef6 100644 --- a/mm/damon/core.c +++ b/mm/damon/core.c @@ -107,8 +107,12 @@ struct damos *damon_new_scheme( scheme->stat_sz = 0; INIT_LIST_HEAD(&scheme->list); + scheme->quota.ms = quota->ms; scheme->quota.sz = quota->sz; scheme->quota.reset_interval = quota->reset_interval; + scheme->quota.total_charged_sz = 0; + scheme->quota.total_charged_ns = 0; + scheme->quota.esz = 0; scheme->quota.charged_sz = 0; scheme->quota.charged_from = 0; scheme->quota.charge_target_from = NULL; @@ -555,9 +559,10 @@ static void damon_do_apply_schemes(struct damon_ctx *c, damon_for_each_scheme(s, c) { struct damos_quota *quota = &s->quota; unsigned long sz = r->ar.end - r->ar.start; + struct timespec64 begin, end; /* Check the quota */ - if (quota->sz && quota->charged_sz >= quota->sz) + if (quota->esz && quota->charged_sz >= quota->esz) continue; /* Skip previously charged regions */ @@ -602,16 +607,21 @@ static void damon_do_apply_schemes(struct damon_ctx *c, /* Apply the scheme */ if (c->primitive.apply_scheme) { - if (quota->sz && quota->charged_sz + sz > quota->sz) { - sz = ALIGN_DOWN(quota->sz - quota->charged_sz, + if (quota->esz && + quota->charged_sz + sz > quota->esz) { + sz = ALIGN_DOWN(quota->esz - quota->charged_sz, DAMON_MIN_REGION); if (!sz) goto update_stat; damon_split_region_at(c, t, r, sz); } + ktime_get_coarse_ts64(&begin); c->primitive.apply_scheme(c, t, r, s); + ktime_get_coarse_ts64(&end); + quota->total_charged_ns += timespec64_to_ns(&end) - + timespec64_to_ns(&begin); quota->charged_sz += sz; - if (quota->sz && quota->charged_sz >= quota->sz) { + if (quota->esz && quota->charged_sz >= quota->esz) { quota->charge_target_from = t; quota->charge_addr_from = r->ar.end + 1; } @@ -625,6 +635,29 @@ static void damon_do_apply_schemes(struct damon_ctx *c, } } +/* Shouldn't be called if quota->ms and quota->sz are zero */ +static void damos_set_effective_quota(struct damos_quota *quota) +{ + unsigned long throughput; + unsigned long esz; + + if (!quota->ms) { + quota->esz = quota->sz; + return; + } + + if (quota->total_charged_ns) + throughput = quota->total_charged_sz * 1000000 / + quota->total_charged_ns; + else + throughput = PAGE_SIZE * 1024; + esz = throughput * quota->ms; + + if (quota->sz && quota->sz < esz) + esz = quota->sz; + quota->esz = esz; +} + static void kdamond_apply_schemes(struct damon_ctx *c) { struct damon_target *t; @@ -634,15 +667,17 @@ static void kdamond_apply_schemes(struct damon_ctx *c) damon_for_each_scheme(s, c) { struct damos_quota *quota = &s->quota; - if (!quota->sz) + if (!quota->ms && !quota->sz) continue; /* New charge window starts */ if (time_after_eq(jiffies, quota->charged_from + msecs_to_jiffies( quota->reset_interval))) { + quota->total_charged_sz += quota->charged_sz; quota->charged_from = jiffies; quota->charged_sz = 0; + damos_set_effective_quota(quota); } } From patchwork Tue Jul 20 13:12:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 12388277 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5DB83C636C8 for ; Tue, 20 Jul 2021 13:14:14 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0CA48610CC for ; Tue, 20 Jul 2021 13:14:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0CA48610CC Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id BE8556B0071; Tue, 20 Jul 2021 09:14:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BBE816B0072; Tue, 20 Jul 2021 09:14:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A3A816B0073; Tue, 20 Jul 2021 09:14:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0153.hostedemail.com [216.40.44.153]) by kanga.kvack.org (Postfix) with ESMTP id 7CBAE6B0071 for ; Tue, 20 Jul 2021 09:14:14 -0400 (EDT) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 0E68E274DB for ; Tue, 20 Jul 2021 13:14:13 +0000 (UTC) X-FDA: 78383009586.26.2285110 Received: from mail-qt1-f172.google.com (mail-qt1-f172.google.com [209.85.160.172]) by imf09.hostedemail.com (Postfix) with ESMTP id BE162300010A for ; Tue, 20 Jul 2021 13:14:12 +0000 (UTC) Received: by mail-qt1-f172.google.com with SMTP id d1so15315577qto.4 for ; Tue, 20 Jul 2021 06:14:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=IGNtAVR3k8XF7vgLV1Ypck6ucmf98cXrZIWCxjZwhEs=; b=GAvfz/BMO3wrFZlsSYPAxDLPt9E1IVrmnf78IudxRM3j8OdhKm/+plUr4YXsQBCWGp yUD6hbEqONcdSu+5I/Bdzbc65Xaojyv5E4RAF8ydd9jxhD1/5sgP/Jq8li09Sas/S5ZN LsknWXI6t/85mjINrn2UAE6TAZNnFq9AyxGBKIA9d+YR4PGNpNImmgDJPLy/j0o3cOlU V9bueOkTJWmFO76vT9zJk6pC+AzDO06zY1CiEwsRrdgDOMN+e9v/PDYF6ZW1XxJBTdRD rXYKVt31gh7ghliXZF0HlG5JgAnvQ0p7XG6uMeCMTVEaIEsaA3NpAxwChB8lXz0BgOGA JQDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=IGNtAVR3k8XF7vgLV1Ypck6ucmf98cXrZIWCxjZwhEs=; b=GoXPcKtTiQmgplVMbBSwzdF7AmVozxlScbDDEOZTVjwURQ6jXcG66UxE8nVX7kzJtF ZhfVQOO+Ij6jjgyne8Bbi4B2NhmyUeThpSWSlDX/DhqgkqFY+/uJD7mQdczBmzSP1fYh BBGUeE+56cn7jkge5X+woQZtttmM2aT7FPf0dZTmHMAPxrfuB6kYMh/yEpVs1Jua97lm vAHTSAIJMTufFxyWt7TJPwqzNx8t8piaRlNsBgekdVejxO5llJNRNd20BqTz1RyjmpH7 0yV78LxoNagYIwzAC8QPpF0GacJnaL70y+qNNp8la4nYRmXvOv/gPFxFgNsEEioUHm4W 5COQ== X-Gm-Message-State: AOAM530oumfiEi3bhxSx+LkBP64h7HfJl6DvQY1Q49zPES2xWt/GccfQ IteU+nI6rJx5gwAm1NAU8Fg= X-Google-Smtp-Source: ABdhPJxvig77cKBciRJ2ldiG0pfJ9Ic/jAu3PCw2ZeX8znA6+2zHm5JQDCb9lPnGPoXl7zwhNPDLWg== X-Received: by 2002:ac8:47c5:: with SMTP id d5mr26165672qtr.231.1626786852145; Tue, 20 Jul 2021 06:14:12 -0700 (PDT) Received: from localhost.localdomain (ec2-35-169-212-159.compute-1.amazonaws.com. [35.169.212.159]) by smtp.gmail.com with ESMTPSA id g17sm9701225qkm.34.2021.07.20.06.14.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Jul 2021 06:14:11 -0700 (PDT) From: SeongJae Park To: akpm@linux-foundation.org Cc: SeongJae Park , Jonathan.Cameron@Huawei.com, acme@kernel.org, alexander.shishkin@linux.intel.com, amit@kernel.org, benh@kernel.crashing.org, brendanhiggins@google.com, corbet@lwn.net, david@redhat.com, dwmw@amazon.com, elver@google.com, fan.du@intel.com, foersleo@amazon.de, greg@kroah.com, gthelen@google.com, guoju.fgj@alibaba-inc.com, jgowans@amazon.com, joe@perches.com, mgorman@suse.de, mheyne@amazon.de, minchan@kernel.org, mingo@redhat.com, namhyung@kernel.org, peterz@infradead.org, riel@surriel.com, rientjes@google.com, rostedt@goodmis.org, rppt@kernel.org, shakeelb@google.com, shuah@kernel.org, sieberf@amazon.com, sj38.park@gmail.com, snu@zelle79.org, vbabka@suse.cz, vdavydov.dev@gmail.com, zgf574564920@gmail.com, linux-damon@amazon.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [RFC v3 05/15] mm/damon/dbgfs: Support schemes' time/IO quotas Date: Tue, 20 Jul 2021 13:12:59 +0000 Message-Id: <20210720131309.22073-6-sj38.park@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210720131309.22073-1-sj38.park@gmail.com> References: <20210720131309.22073-1-sj38.park@gmail.com> Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=gmail.com header.s=20161025 header.b="GAvfz/BM"; spf=pass (imf09.hostedemail.com: domain of sj38park@gmail.com designates 209.85.160.172 as permitted sender) smtp.mailfrom=sj38park@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Stat-Signature: 46fqe848mnhjjwdy4ykjherj1ey8c4s4 X-Rspamd-Queue-Id: BE162300010A X-Rspamd-Server: rspam01 X-HE-Tag: 1626786852-288757 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park This commit makes the debugfs interface of DAMON to support the schemes' time/IO quotas by chaning the format of the input for the schemes file. Signed-off-by: SeongJae Park --- mm/damon/dbgfs.c | 32 +++++++++++++++++++++++++------- 1 file changed, 25 insertions(+), 7 deletions(-) diff --git a/mm/damon/dbgfs.c b/mm/damon/dbgfs.c index ac0de2de1987..3cd253a07116 100644 --- a/mm/damon/dbgfs.c +++ b/mm/damon/dbgfs.c @@ -227,11 +227,14 @@ static ssize_t sprint_schemes(struct damon_ctx *c, char *buf, ssize_t len) damon_for_each_scheme(s, c) { rc = scnprintf(&buf[written], len - written, - "%lu %lu %u %u %u %u %d %lu %lu\n", + "%lu %lu %u %u %u %u %d %lu %lu %lu %lu %lu\n", s->min_sz_region, s->max_sz_region, s->min_nr_accesses, s->max_nr_accesses, s->min_age_region, s->max_age_region, - s->action, s->stat_count, s->stat_sz); + s->action, + s->quota.ms, s->quota.sz, + s->quota.reset_interval, + s->stat_count, s->stat_sz); if (!rc) return -ENOMEM; @@ -312,10 +315,11 @@ static struct damos **str_to_schemes(const char *str, ssize_t len, while (pos < len && *nr_schemes < max_nr_schemes) { struct damos_quota quota = {}; - ret = sscanf(&str[pos], "%lu %lu %u %u %u %u %u%n", + ret = sscanf(&str[pos], "%lu %lu %u %u %u %u %u %lu %lu %lu%n", &min_sz, &max_sz, &min_nr_a, &max_nr_a, - &min_age, &max_age, &action, &parsed); - if (ret != 7) + &min_age, &max_age, &action, "a.ms, + "a.sz, "a.reset_interval, &parsed); + if (ret != 10) break; if (!damos_action_valid(action)) { pr_err("wrong action %d\n", action); @@ -1137,6 +1141,15 @@ static ssize_t dbgfs_monitor_on_write(struct file *file, return ret; } +/* + * v1: Add the scheme speed limit + */ +static ssize_t dbgfs_version_read(struct file *file, + char __user *buf, size_t count, loff_t *ppos) +{ + return simple_read_from_buffer(buf, count, ppos, "1\n", 2); +} + static const struct file_operations mk_contexts_fops = { .write = dbgfs_mk_context_write, }; @@ -1150,13 +1163,18 @@ static const struct file_operations monitor_on_fops = { .write = dbgfs_monitor_on_write, }; +static const struct file_operations version_fops = { + .owner = THIS_MODULE, + .read = dbgfs_version_read, +}; + static int __init __damon_dbgfs_init(void) { struct dentry *dbgfs_root; const char * const file_names[] = {"mk_contexts", "rm_contexts", - "monitor_on"}; + "monitor_on", "version"}; const struct file_operations *fops[] = {&mk_contexts_fops, - &rm_contexts_fops, &monitor_on_fops}; + &rm_contexts_fops, &monitor_on_fops, &version_fops}; int i; dbgfs_root = debugfs_create_dir("damon", NULL); From patchwork Tue Jul 20 13:13:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 12388279 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6E368C636C9 for ; Tue, 20 Jul 2021 13:14:18 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0C31D61106 for ; Tue, 20 Jul 2021 13:14:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0C31D61106 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id BC2146B0072; Tue, 20 Jul 2021 09:14:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B98476B0073; Tue, 20 Jul 2021 09:14:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A14566B0074; Tue, 20 Jul 2021 09:14:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0041.hostedemail.com [216.40.44.41]) by kanga.kvack.org (Postfix) with ESMTP id 76F176B0072 for ; Tue, 20 Jul 2021 09:14:18 -0400 (EDT) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id F3379181F24A4 for ; Tue, 20 Jul 2021 13:14:16 +0000 (UTC) X-FDA: 78383009754.20.09E32B8 Received: from mail-qt1-f170.google.com (mail-qt1-f170.google.com [209.85.160.170]) by imf04.hostedemail.com (Postfix) with ESMTP id A0EA650000AD for ; Tue, 20 Jul 2021 13:14:16 +0000 (UTC) Received: by mail-qt1-f170.google.com with SMTP id k3so15288236qtq.7 for ; Tue, 20 Jul 2021 06:14:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=6X6ejMAiX1gTqZemYodAPU0FplHO/tuxF/5lomt1hjU=; b=tf4s/IAcedqyKa5fvHgtNtrGASUG219LraSto3HNwbBVrNJU+nMFn+PkEJbLd6nXlQ 98/KaDnk2ND9d7mbHxkV5+hEM9s/CXsVNf/qpGkxzNtsDgZ9xjNHzBW0aU2uZFXbu/kg 7+gcPwUTvewHHnOYexqtadBZVxY3lJAniiYEAaKrq/V+RR7K3pbRh4aWJCHNHiPnnjmD y1hR1ll5fyIu/w0y2naIiRn59RBzWEjGWbCsRd6UYh14W97IeXyEHGwXC50f0+P80b0d EtuFPqgBSv3TG8VeWFea5gValIaA0VFcEDcIWC1JcGPuCuHM8pWkyGJtiBAfsn5haanv 0wLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=6X6ejMAiX1gTqZemYodAPU0FplHO/tuxF/5lomt1hjU=; b=Y0Bnz/IR8XxOdhk+qx5UOdVUyRipJW8F+L1HmdVyAuC3Qd4n2cIWC0r4FJoYBgFuI/ o4o67J2C+5Hv9IOosNukb5lV/rnksY3wVbUif6U3ZSlQW9lbL3N5ZVb7RwQpZMvY71c8 vGqv3XxTFvKsV47ysCn1mUJB6/Ji6tMPfQnU/GbKxrF8P8SJEICiK8XOK4Aqntrvf/zT eO7lBp2QgbyTM0Qx5Co37QIEpnrRRDpJli54xUZaea2i6FQJZhb8PCef2kXog98UzGcO Nj4UaaKgkJ7TxRJ6yLJF0fV7f75CNRADxGL0yh97cxomzJdkclYSJ261s40xEwwmBzPU HwJg== X-Gm-Message-State: AOAM530QB+yALnbx3iP0vdkNGSaGLU2ub3jxL3AnqBnFmZhgZeK85KfC ked2P7BR79Cr+dey3FmwGR0= X-Google-Smtp-Source: ABdhPJwsAlEOMXXQ2Y7PmLM2hx/Iw/tBsTpoGx1Mclm+eVfAI860b1H7MueLWvBl2anRMaE2Ex9fHQ== X-Received: by 2002:ac8:734b:: with SMTP id q11mr26406393qtp.105.1626786856080; Tue, 20 Jul 2021 06:14:16 -0700 (PDT) Received: from localhost.localdomain (ec2-35-169-212-159.compute-1.amazonaws.com. [35.169.212.159]) by smtp.gmail.com with ESMTPSA id g17sm9701225qkm.34.2021.07.20.06.14.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Jul 2021 06:14:15 -0700 (PDT) From: SeongJae Park To: akpm@linux-foundation.org Cc: SeongJae Park , Jonathan.Cameron@Huawei.com, acme@kernel.org, alexander.shishkin@linux.intel.com, amit@kernel.org, benh@kernel.crashing.org, brendanhiggins@google.com, corbet@lwn.net, david@redhat.com, dwmw@amazon.com, elver@google.com, fan.du@intel.com, foersleo@amazon.de, greg@kroah.com, gthelen@google.com, guoju.fgj@alibaba-inc.com, jgowans@amazon.com, joe@perches.com, mgorman@suse.de, mheyne@amazon.de, minchan@kernel.org, mingo@redhat.com, namhyung@kernel.org, peterz@infradead.org, riel@surriel.com, rientjes@google.com, rostedt@goodmis.org, rppt@kernel.org, shakeelb@google.com, shuah@kernel.org, sieberf@amazon.com, sj38.park@gmail.com, snu@zelle79.org, vbabka@suse.cz, vdavydov.dev@gmail.com, zgf574564920@gmail.com, linux-damon@amazon.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [RFC v3 06/15] mm/damon/selftests: Support schemes quotas Date: Tue, 20 Jul 2021 13:13:00 +0000 Message-Id: <20210720131309.22073-7-sj38.park@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210720131309.22073-1-sj38.park@gmail.com> References: <20210720131309.22073-1-sj38.park@gmail.com> Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=gmail.com header.s=20161025 header.b="tf4s/IAc"; spf=pass (imf04.hostedemail.com: domain of sj38park@gmail.com designates 209.85.160.170 as permitted sender) smtp.mailfrom=sj38park@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: A0EA650000AD X-Stat-Signature: mae9wwkkausmq7m7b8qyxxudko6i1xdj X-HE-Tag: 1626786856-145497 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park This commit updates DAMON selftests to support updated schemes debugfs file format for the quotas. Signed-off-by: SeongJae Park --- tools/testing/selftests/damon/debugfs_attrs.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/damon/debugfs_attrs.sh b/tools/testing/selftests/damon/debugfs_attrs.sh index 63908a4c397b..b6a09ed6a1fc 100755 --- a/tools/testing/selftests/damon/debugfs_attrs.sh +++ b/tools/testing/selftests/damon/debugfs_attrs.sh @@ -78,10 +78,10 @@ echo "$orig_content" > "$file" file="$DBGFS/schemes" orig_content=$(cat "$file") -test_write_succ "$file" "1 2 3 4 5 6 4" \ +test_write_succ "$file" "1 2 3 4 5 6 4 0 0 0" \ "$orig_content" "valid input" test_write_fail "$file" "1 2 -3 4 5 6 3" "$orig_content" "multi lines" +3 4 5 6 3 0 0 0" "$orig_content" "multi lines" test_write_succ "$file" "" "$orig_content" "disabling" echo "$orig_content" > "$file" From patchwork Tue Jul 20 13:13:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 12388281 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 45AE7C07E95 for ; Tue, 20 Jul 2021 13:14:23 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E7AD7610D2 for ; Tue, 20 Jul 2021 13:14:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E7AD7610D2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A3AFC6B0073; Tue, 20 Jul 2021 09:14:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A11FE6B0074; Tue, 20 Jul 2021 09:14:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8661B6B0075; Tue, 20 Jul 2021 09:14:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0033.hostedemail.com [216.40.44.33]) by kanga.kvack.org (Postfix) with ESMTP id 609A46B0073 for ; Tue, 20 Jul 2021 09:14:22 -0400 (EDT) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id E53A920432 for ; Tue, 20 Jul 2021 13:14:20 +0000 (UTC) X-FDA: 78383009880.20.20EA81D Received: from mail-qv1-f44.google.com (mail-qv1-f44.google.com [209.85.219.44]) by imf08.hostedemail.com (Postfix) with ESMTP id 9CC353000241 for ; Tue, 20 Jul 2021 13:14:20 +0000 (UTC) Received: by mail-qv1-f44.google.com with SMTP id o31so4666643qvo.1 for ; Tue, 20 Jul 2021 06:14:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=TVJbfwQeVmP57kGKh2Jm/+lhdbG5Ku1DhtHyjROhhmk=; b=Pdiy77ZG9jJof1KC+UVNQqgI+mCfBeV/zyt+MEZ6GvEw4D1OMt3LiHFKQG/dRZxhEY 2FOyN52bTyGVzs2j7EBUgHEvXVtSJa83SguNvDda9E7cNxQSx+IBgw8IuuRsMxChKQtU +f6BizUuFlZ0o58BOWSrj823OvO0E+ZzbGWkf5pkNbkOTBVWXEQfLD/R2/mhjU6inAkv UVyjLAQKVlDRxpCqX2zhbmEyGFrw4GK+jUPowzUNDVHaKUR2ij34er2rb3vOMdoYvD0o RRG4P8d35s2f7y4OEE29kpsBp7S6Zfn0ioqmBaZacWS9D0Axg9TfVWeCQPYHgipjebzt Ha4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=TVJbfwQeVmP57kGKh2Jm/+lhdbG5Ku1DhtHyjROhhmk=; b=qTMI0p3ihC6VkTV3C3T6SeLmw/ZSRI9ooxfSUhg11jqG9V8HSOs8xTbFkrAf0mR3kb G/8dwLmpTV+TfTC399KP8to24rp+UURew5QFZEXUzOfcATeZ36YFmEmbVuw7XcwlWcyT uKwuv7gPDK6+PP8ory20cAgMibmZK5TFpc3E0B20Xo7uacNjJNwWkYroEfaVetsMr12r Xi9oLKa4ghfpsiGhQA4zenqus9R6uPQqkg0B4H0Vr8HLnUqX1oI4YKZ814jfLWk3PC8z Ehz0+jGsBCfaLPnWAQIHF9Zcc4ZbpEzwo0bnkvm04blhOHLys6g4H90fPcuj7OcuDQ7g wSfA== X-Gm-Message-State: AOAM531zDcdCttB5q7v68/Ghvj0Kp0Qv/WTTh+TFPvvzcXm9mYQKVN8v NNNut68nQHADVITktW3GdVo= X-Google-Smtp-Source: ABdhPJyfpwlUWDGamTt4dKokvdEMonQWcrKIGMjq/B9Ap4tCF0T60jVU78KSzRZtI+M+A6bAzsg1/w== X-Received: by 2002:a05:6214:184b:: with SMTP id d11mr30172069qvy.5.1626786859968; Tue, 20 Jul 2021 06:14:19 -0700 (PDT) Received: from localhost.localdomain (ec2-35-169-212-159.compute-1.amazonaws.com. [35.169.212.159]) by smtp.gmail.com with ESMTPSA id g17sm9701225qkm.34.2021.07.20.06.14.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Jul 2021 06:14:19 -0700 (PDT) From: SeongJae Park To: akpm@linux-foundation.org Cc: SeongJae Park , Jonathan.Cameron@Huawei.com, acme@kernel.org, alexander.shishkin@linux.intel.com, amit@kernel.org, benh@kernel.crashing.org, brendanhiggins@google.com, corbet@lwn.net, david@redhat.com, dwmw@amazon.com, elver@google.com, fan.du@intel.com, foersleo@amazon.de, greg@kroah.com, gthelen@google.com, guoju.fgj@alibaba-inc.com, jgowans@amazon.com, joe@perches.com, mgorman@suse.de, mheyne@amazon.de, minchan@kernel.org, mingo@redhat.com, namhyung@kernel.org, peterz@infradead.org, riel@surriel.com, rientjes@google.com, rostedt@goodmis.org, rppt@kernel.org, shakeelb@google.com, shuah@kernel.org, sieberf@amazon.com, sj38.park@gmail.com, snu@zelle79.org, vbabka@suse.cz, vdavydov.dev@gmail.com, zgf574564920@gmail.com, linux-damon@amazon.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [RFC v3 07/15] mm/damon/schemes: Prioritize regions within the quotas Date: Tue, 20 Jul 2021 13:13:01 +0000 Message-Id: <20210720131309.22073-8-sj38.park@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210720131309.22073-1-sj38.park@gmail.com> References: <20210720131309.22073-1-sj38.park@gmail.com> Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=Pdiy77ZG; spf=pass (imf08.hostedemail.com: domain of sj38park@gmail.com designates 209.85.219.44 as permitted sender) smtp.mailfrom=sj38park@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspamd-Server: rspam02 X-Stat-Signature: nyicnwq6izyfrjnq1tnpdqepmh8134zi X-Rspamd-Queue-Id: 9CC353000241 X-HE-Tag: 1626786860-524552 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park This commit makes DAMON to apply schemes to regions having higher priority first, if it cannot apply schemes to all regions due to the quotas. The prioritization function should be implemented in each monitoring primitive. Those would commonly calculate the priority of the region using attributes of regions, namely 'size', 'nr_accesses', and 'age'. For example, some primitive would calculate the priority of each region using a weighted sum of 'nr_accesses' and 'age' of the region. The optimal weights would depend on give environments, so this commit allows it to be customizable. Nevertheless, the score calculation functions are only encouraged to respect the weights, not mandated. So, the customization might not work for some primitives. Signed-off-by: SeongJae Park --- include/linux/damon.h | 26 ++++++++++++++++++ mm/damon/core.c | 62 ++++++++++++++++++++++++++++++++++++++----- 2 files changed, 81 insertions(+), 7 deletions(-) diff --git a/include/linux/damon.h b/include/linux/damon.h index d2dd36b9dd6c..ddea4b4f3bfa 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -14,6 +14,8 @@ /* Minimal region size. Every damon_region is aligned by this. */ #define DAMON_MIN_REGION PAGE_SIZE +/* Max priority score for DAMON-based operation schemes */ +#define DAMOS_MAX_SCORE (99) /** * struct damon_addr_range - Represents an address region of [@start, @end). @@ -95,6 +97,10 @@ enum damos_action { * @sz: Maximum bytes of memory that the action can be applied. * @reset_interval: Charge reset interval in milliseconds. * + * @weight_sz: Weight of the region's size for prioritization. + * @weight_nr_accesses: Weight of the region's nr_accesses for prioritization. + * @weight_age: Weight of the region's age for prioritization. + * * To avoid consuming too much CPU time or IO resources for applying the * &struct damos->action to large memory, DAMON allows users to set time and/or * size quotas. The quotas can be set by writing non-zero values to &ms and @@ -106,12 +112,22 @@ enum damos_action { * Internally, the time quota is transformed to a size quota using estimated * throughput of the scheme's action. DAMON then compares it against &sz and * uses smaller one as the effective quota. + * + * For selecting regions within the quota, DAMON prioritizes current scheme's + * target memory regions using the &struct damon_primitive->get_scheme_score. + * You could customize the prioritization logic by setting &weight_sz, + * &weight_nr_accesses, and &weight_age, because monitoring primitives are + * encouraged to respect those. */ struct damos_quota { unsigned long ms; unsigned long sz; unsigned long reset_interval; + unsigned int weight_sz; + unsigned int weight_nr_accesses; + unsigned int weight_age; + /* private: */ /* For throughput estimation */ unsigned long total_charged_sz; @@ -124,6 +140,10 @@ struct damos_quota { unsigned long charged_from; struct damon_target *charge_target_from; unsigned long charge_addr_from; + + /* For prioritization */ + unsigned long histogram[DAMOS_MAX_SCORE + 1]; + unsigned int min_score; }; /** @@ -174,6 +194,7 @@ struct damon_ctx; * @prepare_access_checks: Prepare next access check of target regions. * @check_accesses: Check the accesses to target regions. * @reset_aggregated: Reset aggregated accesses monitoring results. + * @get_scheme_score: Get the score of a region for a scheme. * @apply_scheme: Apply a DAMON-based operation scheme. * @target_valid: Determine if the target is valid. * @cleanup: Clean up the context. @@ -201,6 +222,8 @@ struct damon_ctx; * of its update. The value will be used for regions adjustment threshold. * @reset_aggregated should reset the access monitoring results that aggregated * by @check_accesses. + * @get_scheme_score should return the priority score of a region for a scheme + * as an integer in [0, &DAMOS_MAX_SCORE]. * @apply_scheme is called from @kdamond when a region for user provided * DAMON-based operation scheme is found. It should apply the scheme's action * to the region. This is not used for &DAMON_ARBITRARY_TARGET case. @@ -215,6 +238,9 @@ struct damon_primitive { void (*prepare_access_checks)(struct damon_ctx *context); unsigned int (*check_accesses)(struct damon_ctx *context); void (*reset_aggregated)(struct damon_ctx *context); + int (*get_scheme_score)(struct damon_ctx *context, + struct damon_target *t, struct damon_region *r, + struct damos *scheme); int (*apply_scheme)(struct damon_ctx *context, struct damon_target *t, struct damon_region *r, struct damos *scheme); bool (*target_valid)(void *target); diff --git a/mm/damon/core.c b/mm/damon/core.c index 321523604ef6..4e8b44fe02a0 100644 --- a/mm/damon/core.c +++ b/mm/damon/core.c @@ -12,6 +12,7 @@ #include #include #include +#include #define CREATE_TRACE_POINTS #include @@ -110,6 +111,9 @@ struct damos *damon_new_scheme( scheme->quota.ms = quota->ms; scheme->quota.sz = quota->sz; scheme->quota.reset_interval = quota->reset_interval; + scheme->quota.weight_sz = quota->weight_sz; + scheme->quota.weight_nr_accesses = quota->weight_nr_accesses; + scheme->quota.weight_age = quota->weight_age; scheme->quota.total_charged_sz = 0; scheme->quota.total_charged_ns = 0; scheme->quota.esz = 0; @@ -550,6 +554,28 @@ static void damon_split_region_at(struct damon_ctx *ctx, struct damon_target *t, struct damon_region *r, unsigned long sz_r); +static bool __damos_valid_target(struct damon_region *r, struct damos *s) +{ + unsigned long sz; + + sz = r->ar.end - r->ar.start; + return s->min_sz_region <= sz && sz <= s->max_sz_region && + s->min_nr_accesses <= r->nr_accesses && + r->nr_accesses <= s->max_nr_accesses && + s->min_age_region <= r->age && r->age <= s->max_age_region; +} + +static bool damos_valid_target(struct damon_ctx *c, struct damon_target *t, + struct damon_region *r, struct damos *s) +{ + bool ret = __damos_valid_target(r, s); + + if (!ret || !s->quota.esz || !c->primitive.get_scheme_score) + return ret; + + return c->primitive.get_scheme_score(c, t, r, s) >= s->quota.min_score; +} + static void damon_do_apply_schemes(struct damon_ctx *c, struct damon_target *t, struct damon_region *r) @@ -596,13 +622,7 @@ static void damon_do_apply_schemes(struct damon_ctx *c, quota->charge_addr_from = 0; } - /* Check the target regions condition */ - if (sz < s->min_sz_region || s->max_sz_region < sz) - continue; - if (r->nr_accesses < s->min_nr_accesses || - s->max_nr_accesses < r->nr_accesses) - continue; - if (r->age < s->min_age_region || s->max_age_region < r->age) + if (!damos_valid_target(c, t, r, s)) continue; /* Apply the scheme */ @@ -666,6 +686,8 @@ static void kdamond_apply_schemes(struct damon_ctx *c) damon_for_each_scheme(s, c) { struct damos_quota *quota = &s->quota; + unsigned long cumulated_sz; + unsigned int score, max_score = 0; if (!quota->ms && !quota->sz) continue; @@ -679,6 +701,32 @@ static void kdamond_apply_schemes(struct damon_ctx *c) quota->charged_sz = 0; damos_set_effective_quota(quota); } + + if (!c->primitive.get_scheme_score) + continue; + + /* Fill up the score histogram */ + memset(quota->histogram, 0, sizeof(quota->histogram)); + damon_for_each_target(t, c) { + damon_for_each_region(r, t) { + if (!__damos_valid_target(r, s)) + continue; + score = c->primitive.get_scheme_score( + c, t, r, s); + quota->histogram[score] += + r->ar.end - r->ar.start; + if (score > max_score) + max_score = score; + } + } + + /* Set the min score limit */ + for (cumulated_sz = 0, score = max_score; ; score--) { + cumulated_sz += quota->histogram[score]; + if (cumulated_sz >= quota->esz || !score) + break; + } + quota->min_score = score; } damon_for_each_target(t, c) { From patchwork Tue Jul 20 13:13:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 12388283 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7AB68C636C9 for ; Tue, 20 Jul 2021 13:14:26 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1BEC8610D2 for ; Tue, 20 Jul 2021 13:14:26 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1BEC8610D2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C85D06B0075; Tue, 20 Jul 2021 09:14:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C5C0A6B0078; Tue, 20 Jul 2021 09:14:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AD67E6B007B; Tue, 20 Jul 2021 09:14:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0093.hostedemail.com [216.40.44.93]) by kanga.kvack.org (Postfix) with ESMTP id 856A06B0075 for ; Tue, 20 Jul 2021 09:14:26 -0400 (EDT) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 145048248047 for ; Tue, 20 Jul 2021 13:14:25 +0000 (UTC) X-FDA: 78383010090.16.EE3A66A Received: from mail-qv1-f54.google.com (mail-qv1-f54.google.com [209.85.219.54]) by imf20.hostedemail.com (Postfix) with ESMTP id C502CD000099 for ; Tue, 20 Jul 2021 13:14:24 +0000 (UTC) Received: by mail-qv1-f54.google.com with SMTP id ay16so10019298qvb.12 for ; Tue, 20 Jul 2021 06:14:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=JHj2ZHEcS9mBmBj4jaozpyE80ZCeNHUxWZ52QHE8I0I=; b=XCcGcB1VWhx+FVmMWzQW+oKjCj63Hi3DokARapYCmKWY+d72h1cIyuWtouMTmekjqv 35a/dfYZ57s0A1y2Kg57mYgt9Y7ZIh81KVCfJ7CGneylbt07Xt+xKtAf6RrfLLhJhsMu y9zBfzj0l9Y12GMvh+hJO1b+z8T01IyFM+RjVLG5FJksl4GXgbfRo7WXgeQZM3gRsBIQ osWqpQ0MYTo4THRVj5km6R+MIpZaiPOp7UC9TP9ljqZ9SdEsVy+yrBhlIZvaPJ/xeKED uQoYQXvVUMRoA5Zv1VYGt/E6frXOXYqHGcIisFjYLIbjj37vzK3K+AsQ48gciPKs5Q+K C/3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=JHj2ZHEcS9mBmBj4jaozpyE80ZCeNHUxWZ52QHE8I0I=; b=AyQ5UhzHZZLRzuy57TJPh/zt8mFhYE0hD2l2CRUCGeDoAkHi2To/GL/ccN5ZUGnKHK GwQmg7uazi/D0Dp2hqtCJjaTbCwXFLWFIaMAqpK8fbzZRvaZLsV+KqJu7+FLUywSIOzy KF4+F69OsBu5P7d4HtCCBwn1OORgXdfIZCJWVScTE6AdkGZ4IzGpm4mVY44pYX1Xk20M WbsFWQH1AMQh4uezgLpLQG6IjYcd2oLJMyGcLt+r9iKavXyGkByAjl/M4i4VX0g5Pk1/ YceIcscErsO748sM3Rm1Cl/MlPOKslOXNkJPGynlYCk/33jIO8xgsn7HIkFil3MuwuTJ bJrA== X-Gm-Message-State: AOAM532MOn0cEzKb+1E+CuddoZbm1bl7et6pcNGs50nteENVqR117VkA RNnJx/84jM4crmS35K5i+LY= X-Google-Smtp-Source: ABdhPJwlwW4Oth3j8RLu6cIQkn4HCGL7sOwB7VQUf8JwQ2A7MK/6BNtR8TdBjcVwRh7YofvDLnOjSw== X-Received: by 2002:a05:6214:242b:: with SMTP id gy11mr30339892qvb.9.1626786864225; Tue, 20 Jul 2021 06:14:24 -0700 (PDT) Received: from localhost.localdomain (ec2-35-169-212-159.compute-1.amazonaws.com. [35.169.212.159]) by smtp.gmail.com with ESMTPSA id g17sm9701225qkm.34.2021.07.20.06.14.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Jul 2021 06:14:23 -0700 (PDT) From: SeongJae Park To: akpm@linux-foundation.org Cc: SeongJae Park , Jonathan.Cameron@Huawei.com, acme@kernel.org, alexander.shishkin@linux.intel.com, amit@kernel.org, benh@kernel.crashing.org, brendanhiggins@google.com, corbet@lwn.net, david@redhat.com, dwmw@amazon.com, elver@google.com, fan.du@intel.com, foersleo@amazon.de, greg@kroah.com, gthelen@google.com, guoju.fgj@alibaba-inc.com, jgowans@amazon.com, joe@perches.com, mgorman@suse.de, mheyne@amazon.de, minchan@kernel.org, mingo@redhat.com, namhyung@kernel.org, peterz@infradead.org, riel@surriel.com, rientjes@google.com, rostedt@goodmis.org, rppt@kernel.org, shakeelb@google.com, shuah@kernel.org, sieberf@amazon.com, sj38.park@gmail.com, snu@zelle79.org, vbabka@suse.cz, vdavydov.dev@gmail.com, zgf574564920@gmail.com, linux-damon@amazon.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [RFC v3 08/15] mm/damon/vaddr,paddr: Support pageout prioritization Date: Tue, 20 Jul 2021 13:13:02 +0000 Message-Id: <20210720131309.22073-9-sj38.park@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210720131309.22073-1-sj38.park@gmail.com> References: <20210720131309.22073-1-sj38.park@gmail.com> Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=XCcGcB1V; spf=pass (imf20.hostedemail.com: domain of sj38park@gmail.com designates 209.85.219.54 as permitted sender) smtp.mailfrom=sj38park@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspamd-Server: rspam02 X-Stat-Signature: 84597wb17xegt44xbjq44mudet9h33n5 X-Rspamd-Queue-Id: C502CD000099 X-HE-Tag: 1626786864-652726 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park This commit makes the default monitoring primitives for virtual address spaces and the physical address sapce to support memory regions prioritization for 'PAGEOUT' DAMOS action. It calculates hotness of each region as weighted sum of 'nr_accesses' and 'age' of the region and get the priority score as reverse of the hotness, so that cold regions can be paged out first. Signed-off-by: SeongJae Park --- include/linux/damon.h | 4 ++++ mm/damon/paddr.c | 14 +++++++++++++ mm/damon/prmtv-common.c | 46 +++++++++++++++++++++++++++++++++++++++++ mm/damon/prmtv-common.h | 3 +++ mm/damon/vaddr.c | 15 ++++++++++++++ 5 files changed, 82 insertions(+) diff --git a/include/linux/damon.h b/include/linux/damon.h index ddea4b4f3bfa..f56f04b63b1c 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -449,6 +449,8 @@ bool damon_va_target_valid(void *t); void damon_va_cleanup(struct damon_ctx *ctx); int damon_va_apply_scheme(struct damon_ctx *context, struct damon_target *t, struct damon_region *r, struct damos *scheme); +int damon_va_scheme_score(struct damon_ctx *context, struct damon_target *t, + struct damon_region *r, struct damos *scheme); void damon_va_set_primitives(struct damon_ctx *ctx); #endif /* CONFIG_DAMON_VADDR */ @@ -459,6 +461,8 @@ void damon_va_set_primitives(struct damon_ctx *ctx); void damon_pa_prepare_access_checks(struct damon_ctx *ctx); unsigned int damon_pa_check_accesses(struct damon_ctx *ctx); bool damon_pa_target_valid(void *t); +int damon_pa_scheme_score(struct damon_ctx *context, struct damon_target *t, + struct damon_region *r, struct damos *scheme); void damon_pa_set_primitives(struct damon_ctx *ctx); #endif /* CONFIG_DAMON_PADDR */ diff --git a/mm/damon/paddr.c b/mm/damon/paddr.c index 303db372e53b..99a579e8d046 100644 --- a/mm/damon/paddr.c +++ b/mm/damon/paddr.c @@ -121,6 +121,19 @@ int damon_pa_apply_scheme(struct damon_ctx *ctx, struct damon_target *t, return 0; } +int damon_pa_scheme_score(struct damon_ctx *context, struct damon_target *t, + struct damon_region *r, struct damos *scheme) +{ + switch (scheme->action) { + case DAMOS_PAGEOUT: + return damon_pageout_score(context, r, scheme); + default: + break; + } + + return DAMOS_MAX_SCORE; +} + void damon_pa_set_primitives(struct damon_ctx *ctx) { ctx->primitive.init = NULL; @@ -131,4 +144,5 @@ void damon_pa_set_primitives(struct damon_ctx *ctx) ctx->primitive.target_valid = damon_pa_target_valid; ctx->primitive.cleanup = NULL; ctx->primitive.apply_scheme = damon_pa_apply_scheme; + ctx->primitive.get_scheme_score = damon_pa_scheme_score; } diff --git a/mm/damon/prmtv-common.c b/mm/damon/prmtv-common.c index 01c1c1b37859..1df31d1f8590 100644 --- a/mm/damon/prmtv-common.c +++ b/mm/damon/prmtv-common.c @@ -236,3 +236,49 @@ bool damon_pa_young(unsigned long paddr, unsigned long *page_sz) *page_sz = result.page_sz; return result.accessed; } + +#define DAMON_MAX_SUBSCORE (100) +#define DAMON_MAX_AGE_IN_LOG (32) + +int damon_pageout_score(struct damon_ctx *c, struct damon_region *r, + struct damos *s) +{ + unsigned int max_nr_accesses; + int freq_subscore; + unsigned int age_in_sec; + int age_in_log, age_subscore; + unsigned int freq_weight = s->quota.weight_nr_accesses; + unsigned int age_weight = s->quota.weight_age; + int hotness; + + max_nr_accesses = c->aggr_interval / c->sample_interval; + freq_subscore = r->nr_accesses * DAMON_MAX_SUBSCORE / max_nr_accesses; + + age_in_sec = (unsigned long)r->age * c->aggr_interval / 1000000; + for (age_in_log = 0; age_in_log < DAMON_MAX_AGE_IN_LOG && age_in_sec; + age_in_log++, age_in_sec >>= 1) + ; + + /* If frequency is 0, higher age means it's colder */ + if (freq_subscore == 0) + age_in_log *= -1; + + /* + * Now age_in_log is in [-DAMON_MAX_AGE_IN_LOG, DAMON_MAX_AGE_IN_LOG]. + * Scale it to be in [0, 100] and set it as age subscore. + */ + age_in_log += DAMON_MAX_AGE_IN_LOG; + age_subscore = age_in_log * DAMON_MAX_SUBSCORE / + DAMON_MAX_AGE_IN_LOG / 2; + + hotness = (freq_weight * freq_subscore + age_weight * age_subscore); + if (freq_weight + age_weight) + hotness /= freq_weight + age_weight; + /* + * Transform it to fit in [0, DAMOS_MAX_SCORE] + */ + hotness = hotness * DAMOS_MAX_SCORE / DAMON_MAX_SUBSCORE; + + /* Return coldness of the region */ + return DAMOS_MAX_SCORE - hotness; +} diff --git a/mm/damon/prmtv-common.h b/mm/damon/prmtv-common.h index ba0c4eecbb79..b27c4e94917e 100644 --- a/mm/damon/prmtv-common.h +++ b/mm/damon/prmtv-common.h @@ -26,3 +26,6 @@ bool damon_va_young(struct mm_struct *mm, unsigned long addr, void damon_pa_mkold(unsigned long paddr); bool damon_pa_young(unsigned long paddr, unsigned long *page_sz); + +int damon_pageout_score(struct damon_ctx *c, struct damon_region *r, + struct damos *s); diff --git a/mm/damon/vaddr.c b/mm/damon/vaddr.c index ee4cbe12ac30..38d5434315e4 100644 --- a/mm/damon/vaddr.c +++ b/mm/damon/vaddr.c @@ -515,6 +515,20 @@ int damon_va_apply_scheme(struct damon_ctx *ctx, struct damon_target *t, return damos_madvise(t, r, madv_action); } +int damon_va_scheme_score(struct damon_ctx *context, struct damon_target *t, + struct damon_region *r, struct damos *scheme) +{ + + switch (scheme->action) { + case DAMOS_PAGEOUT: + return damon_pageout_score(context, r, scheme); + default: + break; + } + + return DAMOS_MAX_SCORE; +} + void damon_va_set_primitives(struct damon_ctx *ctx) { ctx->primitive.init = damon_va_init; @@ -525,6 +539,7 @@ void damon_va_set_primitives(struct damon_ctx *ctx) ctx->primitive.target_valid = damon_va_target_valid; ctx->primitive.cleanup = NULL; ctx->primitive.apply_scheme = damon_va_apply_scheme; + ctx->primitive.get_scheme_score = damon_va_scheme_score; } #include "vaddr-test.h" From patchwork Tue Jul 20 13:13:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 12388285 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E7326C636C8 for ; Tue, 20 Jul 2021 13:14:29 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9202160551 for ; Tue, 20 Jul 2021 13:14:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9202160551 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 4E3996B0078; Tue, 20 Jul 2021 09:14:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4BA936B007B; Tue, 20 Jul 2021 09:14:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 335046B007D; Tue, 20 Jul 2021 09:14:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0173.hostedemail.com [216.40.44.173]) by kanga.kvack.org (Postfix) with ESMTP id 0CFB46B0078 for ; Tue, 20 Jul 2021 09:14:30 -0400 (EDT) Received: from smtpin33.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 851498248047 for ; Tue, 20 Jul 2021 13:14:28 +0000 (UTC) X-FDA: 78383010216.33.E413285 Received: from mail-qv1-f42.google.com (mail-qv1-f42.google.com [209.85.219.42]) by imf16.hostedemail.com (Postfix) with ESMTP id 302E4F00009D for ; Tue, 20 Jul 2021 13:14:28 +0000 (UTC) Received: by mail-qv1-f42.google.com with SMTP id ck17so10039439qvb.9 for ; Tue, 20 Jul 2021 06:14:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=QWr2BUkC4vvi9PM11ezntCMBhcCSyWA7DgM4Q5kuaLU=; b=Utq+RpuMRXgMXcdrvC6umPzmRtG3+auvypucw0Mh9V5KsDQKasPqaKQZhixrOKk1hJ ZS7kXhz6SXpwJGP18CvVNsIsbd9UGqtsOS0mvqwodSY1bz5CNud6AvNz7A/R/kms7qD6 P+YiMKGadHslMcq4Z5RMuKRtVL61+M7kvkKGoaODW5vCwZnEdVA2ecB6LZClGKv/oW4a ainJ7pgrwoSLp0trcw4w6KNiElr6L7G3EKqfU+wvgM/LIIezdbkKzQQCS+sVAlx6eeeT yMb5d5oLL4Cui6iwx8cAAT4x8psgvyGQdSlQMxgx+21ctylM0GXxmNjZWfdX8oIZ0fX6 achA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=QWr2BUkC4vvi9PM11ezntCMBhcCSyWA7DgM4Q5kuaLU=; b=Joj4JZKfyZcM8jIqbC1FkavqMGJNmeNReXo9o9sZMhLpJpflGXg8c7g8ge8D1nKHo1 1MfEVWC0JjFpLoZmwo7/eto/T6SyVpftmnBqIeIThUSZ0uTQnHEdnbgzbjBJclbgZg0U X+6MGhZ6P2dVEc7nf6iwDz94fqfa0J29Fhj0MH4EOHbltj7V/cR6S0BwXZcMaEWUeDPI iV5ocyH2gUQJB9oVeodRnznm58tqAmL4PVAiz4STi4ah+DWqxQAKZ3A79vvGIgFPmQae N117Xpiy8Ai2+jWDXW7X+ik47v/NCDZ6afSZnlQEJoYfs7yqNnbs5EHJDDH4EyzvkTGp kRog== X-Gm-Message-State: AOAM532pGKKXTgIU4vjXh8aFhueKKiJwu4jdEeE4nvsE+mGZ5yTNHdrj w4dabFFza+Ynx4wtEOU1jJ4= X-Google-Smtp-Source: ABdhPJzU/oZsWMgvR36q8SniWa6pLaFlm6s2cnx4Cx9k+RudbNGPvCx5QEqWVowKYTRc2Qo3plfjng== X-Received: by 2002:a0c:fa08:: with SMTP id q8mr25163182qvn.8.1626786867608; Tue, 20 Jul 2021 06:14:27 -0700 (PDT) Received: from localhost.localdomain (ec2-35-169-212-159.compute-1.amazonaws.com. [35.169.212.159]) by smtp.gmail.com with ESMTPSA id g17sm9701225qkm.34.2021.07.20.06.14.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Jul 2021 06:14:27 -0700 (PDT) From: SeongJae Park To: akpm@linux-foundation.org Cc: SeongJae Park , Jonathan.Cameron@Huawei.com, acme@kernel.org, alexander.shishkin@linux.intel.com, amit@kernel.org, benh@kernel.crashing.org, brendanhiggins@google.com, corbet@lwn.net, david@redhat.com, dwmw@amazon.com, elver@google.com, fan.du@intel.com, foersleo@amazon.de, greg@kroah.com, gthelen@google.com, guoju.fgj@alibaba-inc.com, jgowans@amazon.com, joe@perches.com, mgorman@suse.de, mheyne@amazon.de, minchan@kernel.org, mingo@redhat.com, namhyung@kernel.org, peterz@infradead.org, riel@surriel.com, rientjes@google.com, rostedt@goodmis.org, rppt@kernel.org, shakeelb@google.com, shuah@kernel.org, sieberf@amazon.com, sj38.park@gmail.com, snu@zelle79.org, vbabka@suse.cz, vdavydov.dev@gmail.com, zgf574564920@gmail.com, linux-damon@amazon.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [RFC v3 09/15] mm/damon/dbgfs: Support prioritization weights Date: Tue, 20 Jul 2021 13:13:03 +0000 Message-Id: <20210720131309.22073-10-sj38.park@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210720131309.22073-1-sj38.park@gmail.com> References: <20210720131309.22073-1-sj38.park@gmail.com> Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=Utq+RpuM; spf=pass (imf16.hostedemail.com: domain of sj38park@gmail.com designates 209.85.219.42 as permitted sender) smtp.mailfrom=sj38park@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspamd-Server: rspam02 X-Stat-Signature: aqojihu8tzdm1mypue5z33nf76qfeomu X-Rspamd-Queue-Id: 302E4F00009D X-HE-Tag: 1626786868-547068 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park This commit allows DAMON debugfs interface users set the prioritization weights by putting three more numbers to the 'schemes' file. Signed-off-by: SeongJae Park --- mm/damon/dbgfs.c | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/mm/damon/dbgfs.c b/mm/damon/dbgfs.c index 3cd253a07116..93ac15cb5786 100644 --- a/mm/damon/dbgfs.c +++ b/mm/damon/dbgfs.c @@ -227,13 +227,16 @@ static ssize_t sprint_schemes(struct damon_ctx *c, char *buf, ssize_t len) damon_for_each_scheme(s, c) { rc = scnprintf(&buf[written], len - written, - "%lu %lu %u %u %u %u %d %lu %lu %lu %lu %lu\n", + "%lu %lu %u %u %u %u %d %lu %lu %lu %u %u %u %lu %lu\n", s->min_sz_region, s->max_sz_region, s->min_nr_accesses, s->max_nr_accesses, s->min_age_region, s->max_age_region, s->action, s->quota.ms, s->quota.sz, s->quota.reset_interval, + s->quota.weight_sz, + s->quota.weight_nr_accesses, + s->quota.weight_age, s->stat_count, s->stat_sz); if (!rc) return -ENOMEM; @@ -315,11 +318,14 @@ static struct damos **str_to_schemes(const char *str, ssize_t len, while (pos < len && *nr_schemes < max_nr_schemes) { struct damos_quota quota = {}; - ret = sscanf(&str[pos], "%lu %lu %u %u %u %u %u %lu %lu %lu%n", + ret = sscanf(&str[pos], + "%lu %lu %u %u %u %u %u %lu %lu %lu %u %u %u%n", &min_sz, &max_sz, &min_nr_a, &max_nr_a, &min_age, &max_age, &action, "a.ms, - "a.sz, "a.reset_interval, &parsed); - if (ret != 10) + "a.sz, "a.reset_interval, + "a.weight_sz, "a.weight_nr_accesses, + "a.weight_age, &parsed); + if (ret != 13) break; if (!damos_action_valid(action)) { pr_err("wrong action %d\n", action); @@ -1147,7 +1153,7 @@ static ssize_t dbgfs_monitor_on_write(struct file *file, static ssize_t dbgfs_version_read(struct file *file, char __user *buf, size_t count, loff_t *ppos) { - return simple_read_from_buffer(buf, count, ppos, "1\n", 2); + return simple_read_from_buffer(buf, count, ppos, "2\n", 2); } static const struct file_operations mk_contexts_fops = { From patchwork Tue Jul 20 13:13:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 12388287 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EEBC8C636C9 for ; Tue, 20 Jul 2021 13:14:33 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9D7E2610D2 for ; Tue, 20 Jul 2021 13:14:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9D7E2610D2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5CDEA6B007B; Tue, 20 Jul 2021 09:14:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5A5036B007D; Tue, 20 Jul 2021 09:14:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3F8796B007E; Tue, 20 Jul 2021 09:14:34 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0128.hostedemail.com [216.40.44.128]) by kanga.kvack.org (Postfix) with ESMTP id 1CEB36B007B for ; Tue, 20 Jul 2021 09:14:34 -0400 (EDT) Received: from smtpin15.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id A62C58248047 for ; Tue, 20 Jul 2021 13:14:32 +0000 (UTC) X-FDA: 78383010384.15.C987866 Received: from mail-qk1-f176.google.com (mail-qk1-f176.google.com [209.85.222.176]) by imf03.hostedemail.com (Postfix) with ESMTP id 5CCE330002CA for ; Tue, 20 Jul 2021 13:14:32 +0000 (UTC) Received: by mail-qk1-f176.google.com with SMTP id bm6so7134060qkb.1 for ; Tue, 20 Jul 2021 06:14:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=N4OySLzN00OyxAmBozCmrLJ4FDwzHDu/TgQ7eqRxySc=; b=L4iKNe414B39IjYtYRV+736mv2j5dwbTHDQDz9OhJ+YCqXFJivxcxVtLLgmYptKayY ICY4eEAdTgYvcHLm4FvGg90Mb/bCZRsiId9X/7f7cfcljtYuq461c3ucbUHdqLzloodK zkGwYwKbyVgjjh80pwe3GkhGKZvkWMHZoMqwunOPBRR+NRH/xM1Ymx2BycOrcrbQZYN/ 9H+LqlyRWbMBPJrmEZ51NJrE6M/PKMeX7c91Zy9EI+UdN/LrWpCiofceIcfaTfuz42GC E657VxPLlhVkuXXFNHdWSccPUCd8q8MjUucAr6weT3oxUlebuy4UR0CzY9zQjHL+cse1 wsNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=N4OySLzN00OyxAmBozCmrLJ4FDwzHDu/TgQ7eqRxySc=; b=lQEMSU7ILz36aLRD4iDt+8BV41gWvtxdfJuzO4Jt30+8+zmK35Li7qnwSRbRiGVZ/9 XqpnXL4THbCfE9WB7rIFlIkh95Y/CnyLEWTnouh+gu7LsMj9dnOEKnrswguT5NkATudC PVyKNzpF3GpijpWopS6N5jya9vY5ngvHEO1/6LWXcAXqZYymC1YsLhmc9e74VWfs1hOr b67SlMx3OheuS5IDUBbp9mF7T7J7O/641VFfHYZqWI80HTEeTZERZKfuypeFeTqds/Jt Q2ijFs5s662/dGSL9fLnvBJRy8r9Wr+WMZiPRA6AJP7ODemt9/C4/DkHnuc+oabUm/CU HrMQ== X-Gm-Message-State: AOAM532IpdmhF5mt6ChQ3uAYO5ABvmEUKN492JqVPi7JNKQWRRuxkJ+i +LV1EJYdqVuuF92dgDYI/ME= X-Google-Smtp-Source: ABdhPJxUFKWtsMXkkyZQ8/2AFphId8tuwlo+lJcqhyphDa44DBS0Yw9n1wF3tMxCY49TNhlpMKT1Sg== X-Received: by 2002:a05:620a:4e1:: with SMTP id b1mr29376604qkh.363.1626786871796; Tue, 20 Jul 2021 06:14:31 -0700 (PDT) Received: from localhost.localdomain (ec2-35-169-212-159.compute-1.amazonaws.com. [35.169.212.159]) by smtp.gmail.com with ESMTPSA id g17sm9701225qkm.34.2021.07.20.06.14.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Jul 2021 06:14:31 -0700 (PDT) From: SeongJae Park To: akpm@linux-foundation.org Cc: SeongJae Park , Jonathan.Cameron@Huawei.com, acme@kernel.org, alexander.shishkin@linux.intel.com, amit@kernel.org, benh@kernel.crashing.org, brendanhiggins@google.com, corbet@lwn.net, david@redhat.com, dwmw@amazon.com, elver@google.com, fan.du@intel.com, foersleo@amazon.de, greg@kroah.com, gthelen@google.com, guoju.fgj@alibaba-inc.com, jgowans@amazon.com, joe@perches.com, mgorman@suse.de, mheyne@amazon.de, minchan@kernel.org, mingo@redhat.com, namhyung@kernel.org, peterz@infradead.org, riel@surriel.com, rientjes@google.com, rostedt@goodmis.org, rppt@kernel.org, shakeelb@google.com, shuah@kernel.org, sieberf@amazon.com, sj38.park@gmail.com, snu@zelle79.org, vbabka@suse.cz, vdavydov.dev@gmail.com, zgf574564920@gmail.com, linux-damon@amazon.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [RFC v3 10/15] tools/selftests/damon: Update for regions prioritization of schemes Date: Tue, 20 Jul 2021 13:13:04 +0000 Message-Id: <20210720131309.22073-11-sj38.park@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210720131309.22073-1-sj38.park@gmail.com> References: <20210720131309.22073-1-sj38.park@gmail.com> X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 5CCE330002CA X-Stat-Signature: 7qpxmbq84c8meo3pnhir3fp3wif8r3zj Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=L4iKNe41; spf=pass (imf03.hostedemail.com: domain of sj38park@gmail.com designates 209.85.222.176 as permitted sender) smtp.mailfrom=sj38park@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-HE-Tag: 1626786872-945471 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park This commit updates the DAMON selftests for 'schemes' debugfs file, as the file format is updated. Signed-off-by: SeongJae Park --- tools/testing/selftests/damon/debugfs_attrs.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/damon/debugfs_attrs.sh b/tools/testing/selftests/damon/debugfs_attrs.sh index b6a09ed6a1fc..4c69d8709e3b 100755 --- a/tools/testing/selftests/damon/debugfs_attrs.sh +++ b/tools/testing/selftests/damon/debugfs_attrs.sh @@ -78,10 +78,10 @@ echo "$orig_content" > "$file" file="$DBGFS/schemes" orig_content=$(cat "$file") -test_write_succ "$file" "1 2 3 4 5 6 4 0 0 0" \ +test_write_succ "$file" "1 2 3 4 5 6 4 0 0 0 1 2 3" \ "$orig_content" "valid input" test_write_fail "$file" "1 2 -3 4 5 6 3 0 0 0" "$orig_content" "multi lines" +3 4 5 6 3 0 0 0 1 2 3" "$orig_content" "multi lines" test_write_succ "$file" "" "$orig_content" "disabling" echo "$orig_content" > "$file" From patchwork Tue Jul 20 13:13:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 12388289 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 395D2C07E95 for ; Tue, 20 Jul 2021 13:14:37 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CF5E460551 for ; Tue, 20 Jul 2021 13:14:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CF5E460551 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8B4256B0011; Tue, 20 Jul 2021 09:14:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 88A8C6B007D; Tue, 20 Jul 2021 09:14:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6DE676B007E; Tue, 20 Jul 2021 09:14:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0191.hostedemail.com [216.40.44.191]) by kanga.kvack.org (Postfix) with ESMTP id 4912E6B0011 for ; Tue, 20 Jul 2021 09:14:37 -0400 (EDT) Received: from smtpin31.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id B4B7B267CF for ; Tue, 20 Jul 2021 13:14:35 +0000 (UTC) X-FDA: 78383010510.31.F7D4AC9 Received: from mail-qt1-f171.google.com (mail-qt1-f171.google.com [209.85.160.171]) by imf09.hostedemail.com (Postfix) with ESMTP id 7E2253000103 for ; Tue, 20 Jul 2021 13:14:35 +0000 (UTC) Received: by mail-qt1-f171.google.com with SMTP id d1so15316315qto.4 for ; Tue, 20 Jul 2021 06:14:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=OTGTiZSl1O9O/Ean2SQx4cmBjaBEQg8/xZ0NCYTf44g=; b=W6C3k5x7UsbXKLJL0gC9dn0dduAtptG/v5QsLa6igKgcGs+Ed1SqHlNgnOToXPz77Z Uu8GIIQkz6wQLQxP3bf6YiMGXCrRcDKh4kI5We0HerXTutv88/xgHdTt2V/AghOz3We0 bAAVBMO1+7NAIKAUwYJMkspP7ynShOlXx4Wm/18lXELx1ffUy0mUMPOWymP8Qu7VE+ZU e3jIMhLYBlkL5IGJacJrYbYDnqky/nF1Bx45DyFWCaDDIDd0AFLLh55KRZwiUcW8ytCM IoBkf9UenhQabTyFQpnkhTo8P4v07TdEyoDWDxNxLVfnCJS4UWL6va5LCSooJ0QEBbLj mM+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=OTGTiZSl1O9O/Ean2SQx4cmBjaBEQg8/xZ0NCYTf44g=; b=tfEWDAzZcHVqjYABRq2JexNMhU73CJzRzyIt28PIBdhHUdZN/Uzdav21zCC1MzI7K+ 3vxa6FuB4PIpRoebYjZfOGd6zIHjHfBB9AwfkR6yTkNUmxAHTHdTh4RVLTA0J7AdYjB6 96dV9dsviQ+Dc6ctl9A5XekgXL7jhS9v5Uh4Bunv2Qgs2q4U90V2uSQitegrvoksaQmp oriO0DmypWYrsOkx1Qsqn5n0qld4qTxADLAOaUlu7uwRxMSSeHM9xMhhhTTRm7IT3LhA k6JoRDnH5YjTzX6gknHoEdHa83KN4nBE7S2zSqqtCofDhdkSlkzi9ja+NWZaIBaK5lgh 3k1Q== X-Gm-Message-State: AOAM530toPUM4llJWseXjfUFf6fhSPg0ky+RrKpF8jm7xl1yBrZZw9yD OGCQFfvqkSJCcF+mnxObRHw= X-Google-Smtp-Source: ABdhPJwNcM1/z0EWqSKPmg8xNFXtFIOUqz7YJp0kh1NgNTE3y+RF1LpBKAJP+S3huxPL5H1oYCle4g== X-Received: by 2002:a05:622a:310:: with SMTP id q16mr26438668qtw.168.1626786874645; Tue, 20 Jul 2021 06:14:34 -0700 (PDT) Received: from localhost.localdomain (ec2-35-169-212-159.compute-1.amazonaws.com. [35.169.212.159]) by smtp.gmail.com with ESMTPSA id g17sm9701225qkm.34.2021.07.20.06.14.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Jul 2021 06:14:34 -0700 (PDT) From: SeongJae Park To: akpm@linux-foundation.org Cc: SeongJae Park , Jonathan.Cameron@Huawei.com, acme@kernel.org, alexander.shishkin@linux.intel.com, amit@kernel.org, benh@kernel.crashing.org, brendanhiggins@google.com, corbet@lwn.net, david@redhat.com, dwmw@amazon.com, elver@google.com, fan.du@intel.com, foersleo@amazon.de, greg@kroah.com, gthelen@google.com, guoju.fgj@alibaba-inc.com, jgowans@amazon.com, joe@perches.com, mgorman@suse.de, mheyne@amazon.de, minchan@kernel.org, mingo@redhat.com, namhyung@kernel.org, peterz@infradead.org, riel@surriel.com, rientjes@google.com, rostedt@goodmis.org, rppt@kernel.org, shakeelb@google.com, shuah@kernel.org, sieberf@amazon.com, sj38.park@gmail.com, snu@zelle79.org, vbabka@suse.cz, vdavydov.dev@gmail.com, zgf574564920@gmail.com, linux-damon@amazon.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [RFC v3 11/15] mm/damon/schemes: Activate schemes based on a watermarks mechanism Date: Tue, 20 Jul 2021 13:13:05 +0000 Message-Id: <20210720131309.22073-12-sj38.park@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210720131309.22073-1-sj38.park@gmail.com> References: <20210720131309.22073-1-sj38.park@gmail.com> Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=W6C3k5x7; spf=pass (imf09.hostedemail.com: domain of sj38park@gmail.com designates 209.85.160.171 as permitted sender) smtp.mailfrom=sj38park@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspamd-Server: rspam05 X-Stat-Signature: qojsf1mzdmiupqf1idaz9ee54hxqqjaz X-Rspamd-Queue-Id: 7E2253000103 X-HE-Tag: 1626786875-482471 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park DAMON-based operation schemes need to be manually turned on and off. In some use cases, however, the condition for turning a scheme on and off would depend on the system's situation. For example, schemes for proactive pages reclamation would need to be turned on when some memory pressure is detected, and turned off when the system has enough free memory. For easier control of schemes activation based on the system situation, this commit introduces a watermarks-based mechanism. The client can describe the watermark metric (e.g., amount of free memory in the system), watermark check interval, and three watermarks, namely high, mid, and low. If the scheme is deactivated, it only gets the metric and compare that to the three watermarks for every check interval. If the metric is higher than the high watermark, the scheme is deactivated. If the metric is between the mid watermark and the low watermark, the scheme is activated. If the metric is lower than the low watermark, the scheme is deactivated again. This is to allow users fall back to traditional page-granularity mechanisms. Signed-off-by: SeongJae Park --- include/linux/damon.h | 52 ++++++++++++++++++++++- mm/damon/core.c | 97 ++++++++++++++++++++++++++++++++++++++++++- mm/damon/dbgfs.c | 5 ++- 3 files changed, 151 insertions(+), 3 deletions(-) diff --git a/include/linux/damon.h b/include/linux/damon.h index f56f04b63b1c..b33a5ee94f62 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -146,6 +146,45 @@ struct damos_quota { unsigned int min_score; }; +/** + * enum damos_wmark_metric - Represents the watermark metric. + * + * @DAMOS_WMARK_NONE: Ignore the watermarks of the given scheme. + * @DAMOS_WMARK_FREE_MEM_RATE: Free memory rate of the system in [0,1000]. + */ +enum damos_wmark_metric { + DAMOS_WMARK_NONE, + DAMOS_WMARK_FREE_MEM_RATE, +}; + +/** + * struct damos_watermarks - Controls when a given scheme should be activated. + * @metric: Metric for the watermarks. + * @interval: Watermarks check time interval in microseconds. + * @high: High watermark. + * @mid: Middle watermark. + * @low: Low watermark. + * + * If &metric is &DAMOS_WMARK_NONE, the scheme is always active. Being active + * means DAMON does monitoring and applying the action of the scheme to + * appropriate memory regions. Else, DAMON checks &metric of the system for at + * least every &interval microseconds and works as below. + * + * If &metric is higher than &high, the scheme is inactivated. If &metric is + * between &mid and &low, the scheme is activated. If &metric is lower than + * &low, the scheme is inactivated. + */ +struct damos_watermarks { + enum damos_wmark_metric metric; + unsigned long interval; + unsigned long high; + unsigned long mid; + unsigned long low; + +/* private: */ + bool activated; +}; + /** * struct damos - Represents a Data Access Monitoring-based Operation Scheme. * @min_sz_region: Minimum size of target regions. @@ -156,6 +195,7 @@ struct damos_quota { * @max_age_region: Maximum age of target regions. * @action: &damo_action to be applied to the target regions. * @quota: Control the aggressiveness of this scheme. + * @wmarks: Watermarks for automated (in)activation of this scheme. * @stat_count: Total number of regions that this scheme is applied. * @stat_sz: Total size of regions that this scheme is applied. * @list: List head for siblings. @@ -166,6 +206,14 @@ struct damos_quota { * those. To avoid consuming too much CPU time or IO resources for the * &action, "a is used. * + * To do the work only when needed, schemes can be activated for specific + * system situations using &wmarks. If all schemes that registered to the + * monitoring context are inactive, DAMON stops monitoring either, and just + * repeatedly checks the watermarks. + * + * If all schemes that registered to a &struct damon_ctx are inactive, DAMON + * stops monitoring and just repeatedly checks the watermarks. + * * After applying the &action to each region, &stat_count and &stat_sz is * updated to reflect the number of regions and total size of regions that the * &action is applied. @@ -179,6 +227,7 @@ struct damos { unsigned int max_age_region; enum damos_action action; struct damos_quota quota; + struct damos_watermarks wmarks; unsigned long stat_count; unsigned long stat_sz; struct list_head list; @@ -412,7 +461,8 @@ struct damos *damon_new_scheme( unsigned long min_sz_region, unsigned long max_sz_region, unsigned int min_nr_accesses, unsigned int max_nr_accesses, unsigned int min_age_region, unsigned int max_age_region, - enum damos_action action, struct damos_quota *quota); + enum damos_action action, struct damos_quota *quota, + struct damos_watermarks *wmarks); void damon_add_scheme(struct damon_ctx *ctx, struct damos *s); void damon_destroy_scheme(struct damos *s); diff --git a/mm/damon/core.c b/mm/damon/core.c index 4e8b44fe02a0..5b578f65c109 100644 --- a/mm/damon/core.c +++ b/mm/damon/core.c @@ -10,6 +10,7 @@ #include #include #include +#include #include #include #include @@ -90,7 +91,8 @@ struct damos *damon_new_scheme( unsigned long min_sz_region, unsigned long max_sz_region, unsigned int min_nr_accesses, unsigned int max_nr_accesses, unsigned int min_age_region, unsigned int max_age_region, - enum damos_action action, struct damos_quota *quota) + enum damos_action action, struct damos_quota *quota, + struct damos_watermarks *wmarks) { struct damos *scheme; @@ -122,6 +124,13 @@ struct damos *damon_new_scheme( scheme->quota.charge_target_from = NULL; scheme->quota.charge_addr_from = 0; + scheme->wmarks.metric = wmarks->metric; + scheme->wmarks.interval = wmarks->interval; + scheme->wmarks.high = wmarks->high; + scheme->wmarks.mid = wmarks->mid; + scheme->wmarks.low = wmarks->low; + scheme->wmarks.activated = true; + return scheme; } @@ -587,6 +596,9 @@ static void damon_do_apply_schemes(struct damon_ctx *c, unsigned long sz = r->ar.end - r->ar.start; struct timespec64 begin, end; + if (!s->wmarks.activated) + continue; + /* Check the quota */ if (quota->esz && quota->charged_sz >= quota->esz) continue; @@ -689,6 +701,9 @@ static void kdamond_apply_schemes(struct damon_ctx *c) unsigned long cumulated_sz; unsigned int score, max_score = 0; + if (!s->wmarks.activated) + continue; + if (!quota->ms && !quota->sz) continue; @@ -932,6 +947,83 @@ static bool kdamond_need_stop(struct damon_ctx *ctx) return true; } +static unsigned long damos_wmark_metric_value(enum damos_wmark_metric metric) +{ + struct sysinfo i; + + switch (metric) { + case DAMOS_WMARK_FREE_MEM_RATE: + si_meminfo(&i); + return i.freeram * 1000 / i.totalram; + default: + break; + } + return -EINVAL; +} + +/* + * Returns zero if the scheme is active. Else, returns time to wait for next + * watermark check in micro-seconds. + */ +static unsigned long damos_wmark_wait_us(struct damos *scheme) +{ + unsigned long metric; + + if (scheme->wmarks.metric == DAMOS_WMARK_NONE) + return 0; + + metric = damos_wmark_metric_value(scheme->wmarks.metric); + /* higher than high watermark or lower than low watermark */ + if (metric > scheme->wmarks.high || scheme->wmarks.low > metric) { + if (scheme->wmarks.activated) + pr_info("inactivate a scheme (%d) for %s wmark\n", + scheme->action, + metric > scheme->wmarks.high ? + "high" : "low"); + scheme->wmarks.activated = false; + return scheme->wmarks.interval; + } + + /* inactive and higher than middle watermark */ + if ((scheme->wmarks.high >= metric && metric >= scheme->wmarks.mid) && + !scheme->wmarks.activated) + return scheme->wmarks.interval; + + if (!scheme->wmarks.activated) + pr_info("activate a scheme (%d)\n", scheme->action); + scheme->wmarks.activated = true; + return 0; +} + +static void kdamond_usleep(unsigned long usecs) +{ + if (usecs > 100 * 1000) + schedule_timeout_interruptible(usecs_to_jiffies(usecs)); + else + usleep_range(usecs, usecs + 1); +} + +/* Returns negative error code if it's not activated but should return */ +static int kdamond_wait_activation(struct damon_ctx *ctx) +{ + struct damos *s; + unsigned long wait_time; + unsigned long min_wait_time = 0; + + while (!kdamond_need_stop(ctx)) { + damon_for_each_scheme(s, ctx) { + wait_time = damos_wmark_wait_us(s); + if (!min_wait_time || wait_time < min_wait_time) + min_wait_time = wait_time; + } + if (!min_wait_time) + return 0; + + kdamond_usleep(min_wait_time); + } + return -EBUSY; +} + static void set_kdamond_stop(struct damon_ctx *ctx) { mutex_lock(&ctx->kdamond_lock); @@ -962,6 +1054,9 @@ static int kdamond_fn(void *data) sz_limit = damon_region_sz_limit(ctx); while (!kdamond_need_stop(ctx)) { + if (kdamond_wait_activation(ctx)) + continue; + if (ctx->primitive.prepare_access_checks) ctx->primitive.prepare_access_checks(ctx); if (ctx->callback.after_sampling && diff --git a/mm/damon/dbgfs.c b/mm/damon/dbgfs.c index 93ac15cb5786..2509c0b634b4 100644 --- a/mm/damon/dbgfs.c +++ b/mm/damon/dbgfs.c @@ -317,6 +317,9 @@ static struct damos **str_to_schemes(const char *str, ssize_t len, *nr_schemes = 0; while (pos < len && *nr_schemes < max_nr_schemes) { struct damos_quota quota = {}; + struct damos_watermarks wmarks = { + .metric = DAMOS_WMARK_NONE, + }; ret = sscanf(&str[pos], "%lu %lu %u %u %u %u %u %lu %lu %lu %u %u %u%n", @@ -334,7 +337,7 @@ static struct damos **str_to_schemes(const char *str, ssize_t len, pos += parsed; scheme = damon_new_scheme(min_sz, max_sz, min_nr_a, max_nr_a, - min_age, max_age, action, "a); + min_age, max_age, action, "a, &wmarks); if (!scheme) goto fail; From patchwork Tue Jul 20 13:13:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 12388291 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 14FD8C07E95 for ; Tue, 20 Jul 2021 13:14:44 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id BFB866108B for ; Tue, 20 Jul 2021 13:14:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BFB866108B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7A2336B007D; Tue, 20 Jul 2021 09:14:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 778106B007E; Tue, 20 Jul 2021 09:14:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5CC396B0080; Tue, 20 Jul 2021 09:14:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0195.hostedemail.com [216.40.44.195]) by kanga.kvack.org (Postfix) with ESMTP id 3ACAC6B007D for ; Tue, 20 Jul 2021 09:14:43 -0400 (EDT) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id C94DF1848895E for ; Tue, 20 Jul 2021 13:14:41 +0000 (UTC) X-FDA: 78383010762.23.EE97EFA Received: from mail-qt1-f181.google.com (mail-qt1-f181.google.com [209.85.160.181]) by imf06.hostedemail.com (Postfix) with ESMTP id 50C5A801F912 for ; Tue, 20 Jul 2021 13:14:38 +0000 (UTC) Received: by mail-qt1-f181.google.com with SMTP id c13so12659876qtc.10 for ; Tue, 20 Jul 2021 06:14:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=VVbmUrxw+u6qngGhg59S6mmfEHwMiesMjAfWbg/XAgs=; b=cExFOH1nBjn1ObWW/rCGWAbZARXNv9jlHIeIlBAjTD98oLaE8iOzHo+Om4oafQnCH6 rT5fK4H8sQu2prO4+fznCC0hl/5IXpsu0Oh73i7ozhaqPd1ZLg+FdhebR4ID7L2uCu6W 93JtUEN4gtapZD2xRRPotQfbcNzxiGHHYqPuIKHPlYgEWQF1rYO0mFRi/iYsG2kZkaGK TEw37uLftGQ2NWaeBwehJGA4Q3AVCQw1jxUJmc7aGBSCUd6AWYx+1NcvkQvRsbV8wDzI Fz+WTVjMNY+7yLv7WXovmVWideZgLxiRDnK3bj9ZqzRUsm21PHz6mSLgjyb5ZusjuAmB wJYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=VVbmUrxw+u6qngGhg59S6mmfEHwMiesMjAfWbg/XAgs=; b=sP75yMwA6Gnth0t1oV1xBeVuI3Daxz12WzLM6Jy8G8v+D/W2+914lnokizbOaIghdN 2uU3ZKVMMJg/dbg2Qne6RHYqWVXkDIfuOlZKYmRxBT5AIA+MkmZbI+bvKWA/FDxlacel 7fmq+nyUqqncHSkyi4VsbrP3Rq4+uedoLkpwM2nPcXmVDVrYUPtsFD6UzpH9EJcS1G/Z qWIPVwqPjhsSamDSWhoBNcpd/iKBaVxovK8omZn2P+I5892Zzz0CIA+kNgKBEOb6qCw5 c89EHFn1kg4ZvWPq3jcdDhcvnY40V7uPKXb5Ts1iu9UyiBxMS/Vlu6YGptQShLJRH/kh sa/g== X-Gm-Message-State: AOAM5321ZpSa+ZrYGrjoQZM5jTKb0KTqrHi+t9HP7fu3GC6yzLnZZz6Q 94opn9XavvZasNa6H+BdQ4U= X-Google-Smtp-Source: ABdhPJwaqkELLyml0pvKZt5ZF5XE8mHpJ4f+Mn/CnQd/+B+Z8jr+2HjvimhLkAR2vf0YZd63SwaH4w== X-Received: by 2002:ac8:47c5:: with SMTP id d5mr26167300qtr.231.1626786877747; Tue, 20 Jul 2021 06:14:37 -0700 (PDT) Received: from localhost.localdomain (ec2-35-169-212-159.compute-1.amazonaws.com. [35.169.212.159]) by smtp.gmail.com with ESMTPSA id g17sm9701225qkm.34.2021.07.20.06.14.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Jul 2021 06:14:37 -0700 (PDT) From: SeongJae Park To: akpm@linux-foundation.org Cc: SeongJae Park , Jonathan.Cameron@Huawei.com, acme@kernel.org, alexander.shishkin@linux.intel.com, amit@kernel.org, benh@kernel.crashing.org, brendanhiggins@google.com, corbet@lwn.net, david@redhat.com, dwmw@amazon.com, elver@google.com, fan.du@intel.com, foersleo@amazon.de, greg@kroah.com, gthelen@google.com, guoju.fgj@alibaba-inc.com, jgowans@amazon.com, joe@perches.com, mgorman@suse.de, mheyne@amazon.de, minchan@kernel.org, mingo@redhat.com, namhyung@kernel.org, peterz@infradead.org, riel@surriel.com, rientjes@google.com, rostedt@goodmis.org, rppt@kernel.org, shakeelb@google.com, shuah@kernel.org, sieberf@amazon.com, sj38.park@gmail.com, snu@zelle79.org, vbabka@suse.cz, vdavydov.dev@gmail.com, zgf574564920@gmail.com, linux-damon@amazon.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [RFC v3 12/15] mm/damon/dbgfs: Support watermarks Date: Tue, 20 Jul 2021 13:13:06 +0000 Message-Id: <20210720131309.22073-13-sj38.park@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210720131309.22073-1-sj38.park@gmail.com> References: <20210720131309.22073-1-sj38.park@gmail.com> X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 50C5A801F912 X-Stat-Signature: s8pt5db4g8sx8zohh5m15ftc8qgg51t8 Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=cExFOH1n; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf06.hostedemail.com: domain of sj38park@gmail.com designates 209.85.160.181 as permitted sender) smtp.mailfrom=sj38park@gmail.com X-HE-Tag: 1626786878-885311 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park This commit updates DAMON debugfs interface to support the watermarks based schemes activation. For this, now 'schemes' file receives five more values. Signed-off-by: SeongJae Park --- mm/damon/dbgfs.c | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/mm/damon/dbgfs.c b/mm/damon/dbgfs.c index 2509c0b634b4..168bd6406a3f 100644 --- a/mm/damon/dbgfs.c +++ b/mm/damon/dbgfs.c @@ -227,7 +227,7 @@ static ssize_t sprint_schemes(struct damon_ctx *c, char *buf, ssize_t len) damon_for_each_scheme(s, c) { rc = scnprintf(&buf[written], len - written, - "%lu %lu %u %u %u %u %d %lu %lu %lu %u %u %u %lu %lu\n", + "%lu %lu %u %u %u %u %d %lu %lu %lu %u %u %u %d %lu %lu %lu %lu %lu %lu\n", s->min_sz_region, s->max_sz_region, s->min_nr_accesses, s->max_nr_accesses, s->min_age_region, s->max_age_region, @@ -237,6 +237,8 @@ static ssize_t sprint_schemes(struct damon_ctx *c, char *buf, ssize_t len) s->quota.weight_sz, s->quota.weight_nr_accesses, s->quota.weight_age, + s->wmarks.metric, s->wmarks.interval, + s->wmarks.high, s->wmarks.mid, s->wmarks.low, s->stat_count, s->stat_sz); if (!rc) return -ENOMEM; @@ -317,18 +319,18 @@ static struct damos **str_to_schemes(const char *str, ssize_t len, *nr_schemes = 0; while (pos < len && *nr_schemes < max_nr_schemes) { struct damos_quota quota = {}; - struct damos_watermarks wmarks = { - .metric = DAMOS_WMARK_NONE, - }; + struct damos_watermarks wmarks; ret = sscanf(&str[pos], - "%lu %lu %u %u %u %u %u %lu %lu %lu %u %u %u%n", + "%lu %lu %u %u %u %u %u %lu %lu %lu %u %u %u %u %lu %lu %lu %lu%n", &min_sz, &max_sz, &min_nr_a, &max_nr_a, &min_age, &max_age, &action, "a.ms, "a.sz, "a.reset_interval, "a.weight_sz, "a.weight_nr_accesses, - "a.weight_age, &parsed); - if (ret != 13) + "a.weight_age, &wmarks.metric, + &wmarks.interval, &wmarks.high, &wmarks.mid, + &wmarks.low, &parsed); + if (ret != 18) break; if (!damos_action_valid(action)) { pr_err("wrong action %d\n", action); From patchwork Tue Jul 20 13:13:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 12388293 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 362B2C636C8 for ; Tue, 20 Jul 2021 13:14:44 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E11E760551 for ; Tue, 20 Jul 2021 13:14:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E11E760551 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6B93E6B007E; Tue, 20 Jul 2021 09:14:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5F49C6B0080; Tue, 20 Jul 2021 09:14:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3AA106B0081; Tue, 20 Jul 2021 09:14:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0074.hostedemail.com [216.40.44.74]) by kanga.kvack.org (Postfix) with ESMTP id 163A36B007E for ; Tue, 20 Jul 2021 09:14:44 -0400 (EDT) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 97B12276EE for ; Tue, 20 Jul 2021 13:14:42 +0000 (UTC) X-FDA: 78383010804.20.E5943E9 Received: from mail-qk1-f172.google.com (mail-qk1-f172.google.com [209.85.222.172]) by imf20.hostedemail.com (Postfix) with ESMTP id 5A4EBD000099 for ; Tue, 20 Jul 2021 13:14:42 +0000 (UTC) Received: by mail-qk1-f172.google.com with SMTP id q15so6347767qkm.8 for ; Tue, 20 Jul 2021 06:14:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=4JGp05b24ihTRHgklBj4UKyWFFV1V2kZsG7t8cqA7wo=; b=jYobwTtDo3NVFwh2/V12GOKGtD3WnOJKH8wi9aW9n9EE4lNglOClw++Igx3yDrJVrf S58vCPA8O+wOVncPXOzKMP/9n5qCaCgVyDIz0+RNgQ6dNV8mHu4gxzmYoQm7CQut/o14 h2Um4lXcjH2SgXGcBjxgceVA/dQ269Pf90MkyAd7DQW8FhVhCqNByGvS59aL5HHiYXlu EbIJacrudRnd+8lmAmAYyYjBX5agXgVaZzpr+KfljjozLMM8q0Nmc+AZR073z9u40oCe 4VVBDu5o4NBvVy6VkzaJn601x4I94VIH+KituqkfQKMR4Lyutg//L8J5hjpYhToaqOYk xLpg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=4JGp05b24ihTRHgklBj4UKyWFFV1V2kZsG7t8cqA7wo=; b=b/7uhwDCx2Vm6HeCcRfrXkZU2HlLeBxwa1bhlGCxlas+U+NKfdY0aMHrXOxoBgnBiC sSfGZoUWkMgKkEtT+UuFE/JLMl88lMHSfBh8sIOGVWQukK4rMjpDg+V7aV7vru/fIFbu USYUCLdeFwHHHIgBvilFUK77rDbQnQzc+kBxDZHOGGKelz3G6Olw2js+qTpux7bol/P5 T77wEHg2Z0OD3Kwy0XcrYwCzuH/iKtmYbZY4goQSIVeZlTmGnKurKN8Wp8Zej4OGZVJ8 ++nX5NMoQ1HYc0C8XZSvlfNf78VPAsQNSsPONtH2Oxq0FPIhiCaX49nxAX/Q3AN0VIji yVww== X-Gm-Message-State: AOAM530/LnLO30kuxbhlgQ8WWc2KGn09/7z2bSGEUhUNsC/LDIndfNiO Ob4SpLtxh8Rpge2Ng8eYgis= X-Google-Smtp-Source: ABdhPJzhnbTUOjZXn+05iSY6IRs7WUNu24W17TsbD/hSW5y6sEDFtOjgxQpkq3qOjfN0u4cR9ga9SA== X-Received: by 2002:a05:620a:d5d:: with SMTP id o29mr29562774qkl.157.1626786881788; Tue, 20 Jul 2021 06:14:41 -0700 (PDT) Received: from localhost.localdomain (ec2-35-169-212-159.compute-1.amazonaws.com. [35.169.212.159]) by smtp.gmail.com with ESMTPSA id g17sm9701225qkm.34.2021.07.20.06.14.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Jul 2021 06:14:41 -0700 (PDT) From: SeongJae Park To: akpm@linux-foundation.org Cc: SeongJae Park , Jonathan.Cameron@Huawei.com, acme@kernel.org, alexander.shishkin@linux.intel.com, amit@kernel.org, benh@kernel.crashing.org, brendanhiggins@google.com, corbet@lwn.net, david@redhat.com, dwmw@amazon.com, elver@google.com, fan.du@intel.com, foersleo@amazon.de, greg@kroah.com, gthelen@google.com, guoju.fgj@alibaba-inc.com, jgowans@amazon.com, joe@perches.com, mgorman@suse.de, mheyne@amazon.de, minchan@kernel.org, mingo@redhat.com, namhyung@kernel.org, peterz@infradead.org, riel@surriel.com, rientjes@google.com, rostedt@goodmis.org, rppt@kernel.org, shakeelb@google.com, shuah@kernel.org, sieberf@amazon.com, sj38.park@gmail.com, snu@zelle79.org, vbabka@suse.cz, vdavydov.dev@gmail.com, zgf574564920@gmail.com, linux-damon@amazon.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [RFC v3 13/15] selftests/damon: Support watermarks Date: Tue, 20 Jul 2021 13:13:07 +0000 Message-Id: <20210720131309.22073-14-sj38.park@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210720131309.22073-1-sj38.park@gmail.com> References: <20210720131309.22073-1-sj38.park@gmail.com> Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=jYobwTtD; spf=pass (imf20.hostedemail.com: domain of sj38park@gmail.com designates 209.85.222.172 as permitted sender) smtp.mailfrom=sj38park@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspamd-Server: rspam02 X-Stat-Signature: j4ir94xgwb3b9yc6zegfpss8z1tbbxsb X-Rspamd-Queue-Id: 5A4EBD000099 X-HE-Tag: 1626786882-450231 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park This commit updates DAMON selftests for 'schemes' debugfs file to reflect the changes in the format. Signed-off-by: SeongJae Park --- tools/testing/selftests/damon/debugfs_attrs.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/damon/debugfs_attrs.sh b/tools/testing/selftests/damon/debugfs_attrs.sh index 4c69d8709e3b..154f9f76f84a 100755 --- a/tools/testing/selftests/damon/debugfs_attrs.sh +++ b/tools/testing/selftests/damon/debugfs_attrs.sh @@ -78,10 +78,10 @@ echo "$orig_content" > "$file" file="$DBGFS/schemes" orig_content=$(cat "$file") -test_write_succ "$file" "1 2 3 4 5 6 4 0 0 0 1 2 3" \ +test_write_succ "$file" "1 2 3 4 5 6 4 0 0 0 1 2 3 1 100 3 2 1" \ "$orig_content" "valid input" test_write_fail "$file" "1 2 -3 4 5 6 3 0 0 0 1 2 3" "$orig_content" "multi lines" +3 4 5 6 3 0 0 0 1 2 3 1 100 3 2 1" "$orig_content" "multi lines" test_write_succ "$file" "" "$orig_content" "disabling" echo "$orig_content" > "$file" From patchwork Tue Jul 20 13:13:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 12388297 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E9EEBC636CE for ; Tue, 20 Jul 2021 13:14:46 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8F6BC60551 for ; Tue, 20 Jul 2021 13:14:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8F6BC60551 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 4E33B6B0080; Tue, 20 Jul 2021 09:14:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4BC8D6B0081; Tue, 20 Jul 2021 09:14:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3111F6B0082; Tue, 20 Jul 2021 09:14:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0042.hostedemail.com [216.40.44.42]) by kanga.kvack.org (Postfix) with ESMTP id 09F6D6B0080 for ; Tue, 20 Jul 2021 09:14:47 -0400 (EDT) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 8D0FA26DF4 for ; Tue, 20 Jul 2021 13:14:45 +0000 (UTC) X-FDA: 78383010930.05.140EFBB Received: from mail-qv1-f47.google.com (mail-qv1-f47.google.com [209.85.219.47]) by imf15.hostedemail.com (Postfix) with ESMTP id 3999AD0000A5 for ; Tue, 20 Jul 2021 13:14:45 +0000 (UTC) Received: by mail-qv1-f47.google.com with SMTP id c15so10054121qvw.6 for ; Tue, 20 Jul 2021 06:14:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=M4zJ88ccAMF8JuzRAtdnkdZF1CqPuLIAgD1DSEm1rbE=; b=YUUK8GhClXqWngv7zXnAZce2VwjbZ3dp9fx1Ejna0LHhId8H29J94+8VvtuoJPgmj3 1HixAY3GBBmm2cnm93WjzgrlG+42J5dA8BTNomwhiBMDo1xIrgqP3wtkZTYOMw3QGWHg TFczcHREJI780XQGgjlIG/ns34DN6Vf/duf/J6ijsozFsDRVq+x5xvJHsRi7kF2Sp7/0 o88NYfYq6WuKeuc77G15LAuicjY9J6WLM0cZtGrd+OHtJkh6IRMd9Mu+5O950RBi7Psl BMLKZ+bEQhb3WmqtWRZj45mM7pIC3OBxbUB00Ez7SWAkcI95uwt086k0uFxEZYaTxhtk fbaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=M4zJ88ccAMF8JuzRAtdnkdZF1CqPuLIAgD1DSEm1rbE=; b=Ls8yTADOWnLLijDMW7C9ij6XoIzrUZYTqSkEplcNwljt56CrzIg0m0d0vpy7g+zuSL i+SMintUQI6MeDm+W32ttQFSApyfZi2adK/lHoen0hjsJvYgJLewui2cI5JJ+M6yqMBY thqfTP4K/3UI6jYsRq74MZ/LxUcJVUNknldyxHbhfDIuNsN8c8Wl945rf/zhL8n6ITVL H8U6Lb1jq5Hb9X/A9/DVedcy7w8M1jTKDWsOXQ3rJqjuVkPzZYrEXLc+1/YQW7Y2c52D 2p+JpiwyF83w5icdcrwHUewcXltlIg6s2+6fKANuKoP+OKFdopuDrjTr9JqlQ32VU3ZV gMTA== X-Gm-Message-State: AOAM532eSUsEdU5RTbqW35BqfTqjXlQHIGWfEdcQieZNdatz8HwKZCWa Iorv5Kw08ZSz+PJrF68dgIk= X-Google-Smtp-Source: ABdhPJxz/AR3LRboVuwsHGRuWmdfmxKoqde57fBaVus1XPErGIwYFDPfulYa/xSTzbAO+vQS4qh0pw== X-Received: by 2002:ad4:4654:: with SMTP id y20mr29892806qvv.21.1626786884534; Tue, 20 Jul 2021 06:14:44 -0700 (PDT) Received: from localhost.localdomain (ec2-35-169-212-159.compute-1.amazonaws.com. [35.169.212.159]) by smtp.gmail.com with ESMTPSA id g17sm9701225qkm.34.2021.07.20.06.14.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Jul 2021 06:14:44 -0700 (PDT) From: SeongJae Park To: akpm@linux-foundation.org Cc: SeongJae Park , Jonathan.Cameron@Huawei.com, acme@kernel.org, alexander.shishkin@linux.intel.com, amit@kernel.org, benh@kernel.crashing.org, brendanhiggins@google.com, corbet@lwn.net, david@redhat.com, dwmw@amazon.com, elver@google.com, fan.du@intel.com, foersleo@amazon.de, greg@kroah.com, gthelen@google.com, guoju.fgj@alibaba-inc.com, jgowans@amazon.com, joe@perches.com, mgorman@suse.de, mheyne@amazon.de, minchan@kernel.org, mingo@redhat.com, namhyung@kernel.org, peterz@infradead.org, riel@surriel.com, rientjes@google.com, rostedt@goodmis.org, rppt@kernel.org, shakeelb@google.com, shuah@kernel.org, sieberf@amazon.com, sj38.park@gmail.com, snu@zelle79.org, vbabka@suse.cz, vdavydov.dev@gmail.com, zgf574564920@gmail.com, linux-damon@amazon.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [RFC v3 14/15] mm/damon: Introduce DAMON-based reclamation Date: Tue, 20 Jul 2021 13:13:08 +0000 Message-Id: <20210720131309.22073-15-sj38.park@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210720131309.22073-1-sj38.park@gmail.com> References: <20210720131309.22073-1-sj38.park@gmail.com> Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=YUUK8GhC; spf=pass (imf15.hostedemail.com: domain of sj38park@gmail.com designates 209.85.219.47 as permitted sender) smtp.mailfrom=sj38park@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspamd-Server: rspam05 X-Stat-Signature: nxxsne3jrtirjc64xbuhrhxfn8pcm9ps X-Rspamd-Queue-Id: 3999AD0000A5 X-HE-Tag: 1626786885-781510 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park This commit implements a new kernel subsystem that finds cold memory regions using DAMON and reclaims those immediately. It is intended to be used as proactive lightweigh reclamation logic for light memory pressure. For heavy memory pressure, it could be inactivated and fall back to the traditional page-scanning based reclamation. It's implemented on top of DAMON framework to use the DAMON-based Operation Schemes (DAMOS) feature. It utilizes all the DAMOS features including speed limit, prioritization, and watermarks. It could be enabled and tuned in build time via the kernel configuration, in boot time via the kernel boot parameter, and in run time via its module parameter ('/sys/module/damon_reclaim/parameters/') interface. Signed-off-by: SeongJae Park --- mm/damon/Kconfig | 12 ++ mm/damon/Makefile | 1 + mm/damon/reclaim.c | 354 +++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 367 insertions(+) create mode 100644 mm/damon/reclaim.c diff --git a/mm/damon/Kconfig b/mm/damon/Kconfig index eeefb5b633b6..d9cd88810279 100644 --- a/mm/damon/Kconfig +++ b/mm/damon/Kconfig @@ -84,4 +84,16 @@ config DAMON_DBGFS_KUNIT_TEST If unsure, say N. +config DAMON_RECLAIM + bool "Build DAMON-based reclaim (DAMON_RECLAIM)" + depends on DAMON_PADDR + help + This builds the DAMON-based reclamation subsystem. It finds pages + that not accessed for a long time (cold) using DAMON and reclaim + those if enabled. + + This is suggested to be used as a proactive and lightweight + reclamation under light memory pressure, while the traditional page + scanning-based reclamation is used for heavy pressure. + endmenu diff --git a/mm/damon/Makefile b/mm/damon/Makefile index 017799e5670a..39433e7d570c 100644 --- a/mm/damon/Makefile +++ b/mm/damon/Makefile @@ -5,3 +5,4 @@ obj-$(CONFIG_DAMON_VADDR) += prmtv-common.o vaddr.o obj-$(CONFIG_DAMON_PADDR) += prmtv-common.o paddr.o obj-$(CONFIG_DAMON_PGIDLE) += prmtv-common.o pgidle.o obj-$(CONFIG_DAMON_DBGFS) += dbgfs.o +obj-$(CONFIG_DAMON_RECLAIM) += reclaim.o diff --git a/mm/damon/reclaim.c b/mm/damon/reclaim.c new file mode 100644 index 000000000000..147fd1304b7b --- /dev/null +++ b/mm/damon/reclaim.c @@ -0,0 +1,354 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * DAMON-based page reclamation + * + * Author: SeongJae Park + */ + +#define pr_fmt(fmt) "damon-reclaim: " fmt + +#include +#include +#include +#include +#include + +#ifdef MODULE_PARAM_PREFIX +#undef MODULE_PARAM_PREFIX +#endif +#define MODULE_PARAM_PREFIX "damon_reclaim." + +/* + * Enable or disable DAMON_RECLAIM. + * + * You can enable DAMON_RCLAIM by setting the value of this parameter as ``Y``. + * Setting it as ``N`` disables DAMON_RECLAIM. Note that DAMON_RECLAIM could + * do no real monitoring and reclamation due to the watermarks-based activation + * condition. Refer to below descriptions for the watermarks parameter for + * this. + */ +static bool enabled __read_mostly; +module_param(enabled, bool, 0600); + +/* + * Time threshold for cold memory regions identification in microseconds. + * + * If a memory region is not accessed for this or longer time, DAMON_RECLAIM + * identifies the region as cold, and reclaims. 5 seconds by default. + */ +static unsigned long min_age __read_mostly = 5000000; +module_param(min_age, ulong, 0600); + +/* + * Limit of time for trying the reclamation in milliseconds. + * + * DAMON_RECLAIM tries to use only up to this time within a time window + * (quota_reset_interval_ms) for trying reclamation of cold pages. This can be + * used for limiting CPU consumption of DAMON_RECLAIM. If the value is zero, + * the limit is disabled. + * + * 100 ms by default. + */ +static unsigned long quota_ms __read_mostly = 100; +module_param(quota_ms, ulong, 0600); + +/* + * Limit of size of memory for the reclamation in bytes. + * + * DAMON_RECLAIM charges amount of memory which it tried to reclaim within a + * time window (quota_reset_interval_ms) and makes no more than this limit is + * tried. This can be used for limiting consumption of CPU and IO. If this + * value is zero, the limit is disabled. + * + * 1 GiB by default. + */ +static unsigned long quota_sz __read_mostly = 1024 * 1024 * 1024; +module_param(quota_sz, ulong, 0600); + +/* + * The time/size quota charge reset interval in milliseconds. + * + * The charge reset interval for the quota of time (quota_ms) and size + * (quota_sz). That is, DAMON_RECLAIM does not try reclamation for more than + * quota_ms milliseconds or quota_sz bytes within quota_reset_interval_ms + * milliseconds. + * + * 1 second by default. + */ +static unsigned long quota_reset_interval_ms __read_mostly = 1000; +module_param(quota_reset_interval_ms, ulong, 0600); + +/* + * The watermarks check time interval in microseconds. + * + * Minimal time to wait before checking the watermarks, when DAMON_RECLAIM is + * enabled but inactive due to its watermarks rule. 5 seconds by default. + */ +static unsigned long wmarks_interval __read_mostly = 5000000; +module_param(wmarks_interval, ulong, 0600); + +/* + * Free memory rate (per thousand) for the high watermark. + * + * If free memory of the system in bytes per thousand bytes is higher than + * this, DAMON_RECLAIM becomes inactive, so it does nothing but periodically + * checks the watermarks. 500 (50%) by default. + */ +static unsigned long wmarks_high __read_mostly = 500; +module_param(wmarks_high, ulong, 0600); + +/* + * Free memory rate (per thousand) for the middle watermark. + * + * If free memory of the system in bytes per thousand bytes is between this and + * the low watermark, DAMON_RECLAIM becomes active, so starts the monitoring + * and the reclaiming. 400 (40%) by default. + */ +static unsigned long wmarks_mid __read_mostly = 400; +module_param(wmarks_mid, ulong, 0600); + +/* + * Free memory rate (per thousand) for the low watermark. + * + * If free memory of the system in bytes per thousand bytes is lower than this, + * DAMON_RECLAIM becomes inactive, so it does nothing but periodically checks + * the watermarks. In the case, the system falls back to the LRU-based page + * granularity reclamation logic. 200 (20%) by default. + */ +static unsigned long wmarks_low __read_mostly = 200; +module_param(wmarks_low, ulong, 0600); + +/* + * Sampling interval for the monitoring in microseconds. + * + * The sampling interval of DAMON for the cold memory monitoring. Please refer + * to the DAMON documentation for more detail. 5 ms by default. + */ +static unsigned long sample_interval __read_mostly = 5000; +module_param(sample_interval, ulong, 0600); + +/* + * Aggregation interval for the monitoring in microseconds. + * + * The aggregation interval of DAMON for the cold memory monitoring. Please + * refer to the DAMON documentation for more detail. 100 ms by default. + */ +static unsigned long aggr_interval __read_mostly = 100000; +module_param(aggr_interval, ulong, 0600); + +/* + * Minimum number of monitoring regions. + * + * The minimal number of monitoring regions of DAMON for the cold memory + * monitoring. This can be used to set lower-bound of the monitoring quality. + * But, setting this too high could result in increased monitoring overhead. + * Please refer to the DAMON documentation for more detail. 10 by default. + */ +static unsigned long min_nr_regions __read_mostly = 10; +module_param(min_nr_regions, ulong, 0600); + +/* + * Maximum number of monitoring regions. + * + * The maximum number of monitoring regions of DAMON for the cold memory + * monitoring. This can be used to set upper-bound of the monitoring overhead. + * However, setting this too low could result in bad monitoring quality. + * Please refer to the DAMON documentation for more detail. 1000 by default. + */ +static unsigned long max_nr_regions __read_mostly = 1000; +module_param(max_nr_regions, ulong, 0600); + +/* + * Start of the target memory region in physical address. + * + * The start physical address of memory region that DAMON_RECLAIM will do work + * against. By default, biggest System RAM is used as the region. + */ +static unsigned long monitor_region_start __read_mostly; +module_param(monitor_region_start, ulong, 0600); + +/* + * End of the target memory region in physical address. + * + * The end physical address of memory region that DAMON_RECLAIM will do work + * against. By default, biggest System RAM is used as the region. + */ +static unsigned long monitor_region_end __read_mostly; +module_param(monitor_region_end, ulong, 0600); + +/* + * PID of the DAMON thread + * + * If DAMON_RECLAIM is enabled, this becomes the PID of the worker thread. + * Else, -1. + */ +static int kdamond_pid __read_mostly = -1; +module_param(kdamond_pid, int, 0400); + +static struct damon_ctx *ctx; +static struct damon_target *target; + +struct damon_reclaim_ram_walk_arg { + unsigned long start; + unsigned long end; +}; + +static int walk_system_ram(struct resource *res, void *arg) +{ + struct damon_reclaim_ram_walk_arg *a = arg; + + if (a->end - a->start < res->end - res->start) { + a->start = res->start; + a->end = res->end; + } + return 0; +} + +/* + * Find biggest 'System RAM' resource and store its start and end address in + * @start and @end, respectively. If no System RAM is found, returns false. + */ +static bool get_monitoring_region(unsigned long *start, unsigned long *end) +{ + struct damon_reclaim_ram_walk_arg arg = {}; + + walk_system_ram_res(0, ULONG_MAX, &arg, walk_system_ram); + if (arg.end <= arg.start) + return false; + + *start = arg.start; + *end = arg.end; + return true; +} + +static struct damos *damon_reclaim_new_scheme(void) +{ + struct damos_watermarks wmarks = { + .metric = DAMOS_WMARK_FREE_MEM_RATE, + .interval = wmarks_interval, + .high = wmarks_high, + .mid = wmarks_mid, + .low = wmarks_low, + }; + struct damos_quota quota = { + /* + * Do not try reclamation for more than quota_ms milliseconds + * or quota_sz bytes within quota_reset_interval_ms. + */ + .ms = quota_ms, + .sz = quota_sz, + .reset_interval = quota_reset_interval_ms, + /* Within the quota, page out older regions first. */ + .weight_sz = 0, + .weight_nr_accesses = 0, + .weight_age = 1 + }; + struct damos *scheme = damon_new_scheme( + /* Find regions having PAGE_SIZE or larger size */ + PAGE_SIZE, ULONG_MAX, + /* and not accessed at all */ + 0, 0, + /* for min_age or more micro-seconds, and */ + min_age / aggr_interval, UINT_MAX, + /* page out those, as soon as found */ + DAMOS_PAGEOUT, + /* under the quota. */ + "a, + /* (De)activate this according to the watermarks. */ + &wmarks); + + return scheme; +} + +static int damon_reclaim_turn(bool on) +{ + struct damon_region *region; + struct damos *scheme; + int err; + + if (!on) { + err = damon_stop(&ctx, 1); + if (!err) + kdamond_pid = -1; + return err; + } + + err = damon_set_attrs(ctx, sample_interval, aggr_interval, 0, + min_nr_regions, max_nr_regions); + if (err) + return err; + + if (monitor_region_start > monitor_region_end) + return -EINVAL; + if (!monitor_region_start && !monitor_region_end && + !get_monitoring_region(&monitor_region_start, + &monitor_region_end)) + return -EINVAL; + /* DAMON will free this on its own when finish monitoring */ + region = damon_new_region(monitor_region_start, monitor_region_end); + if (!region) + return -ENOMEM; + damon_add_region(region, target); + + /* Will be freed by 'damon_set_schemes()' below */ + scheme = damon_reclaim_new_scheme(); + if (!scheme) + goto free_region_out; + err = damon_set_schemes(ctx, &scheme, 1); + if (err) + goto free_scheme_out; + + err = damon_start(&ctx, 1); + if (!err) { + kdamond_pid = ctx->kdamond->pid; + return 0; + } + +free_scheme_out: + damon_destroy_scheme(scheme); +free_region_out: + damon_destroy_region(region, target); + return err; +} + +#define ENABLE_CHECK_INTERVAL_MS 1000 +static struct delayed_work damon_reclaim_timer; +static void damon_reclaim_timer_fn(struct work_struct *work) +{ + static bool last_enabled; + bool now_enabled; + + now_enabled = enabled; + if (last_enabled != now_enabled) { + if (!damon_reclaim_turn(now_enabled)) + last_enabled = now_enabled; + else + enabled = last_enabled; + } + + schedule_delayed_work(&damon_reclaim_timer, + msecs_to_jiffies(ENABLE_CHECK_INTERVAL_MS)); +} +static DECLARE_DELAYED_WORK(damon_reclaim_timer, damon_reclaim_timer_fn); + +static int __init damon_reclaim_init(void) +{ + ctx = damon_new_ctx(DAMON_ADAPTIVE_TARGET); + if (!ctx) + return -ENOMEM; + + damon_pa_set_primitives(ctx); + + /* 4242 means nothing but fun */ + target = damon_new_target(4242); + if (!target) { + damon_destroy_ctx(ctx); + return -ENOMEM; + } + damon_add_target(ctx, target); + + schedule_delayed_work(&damon_reclaim_timer, 0); + return 0; +} + +module_init(damon_reclaim_init); From patchwork Tue Jul 20 13:13:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 12388295 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 09584C07E9B for ; Tue, 20 Jul 2021 13:14:50 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A13D06108B for ; Tue, 20 Jul 2021 13:14:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A13D06108B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 61B876B0081; Tue, 20 Jul 2021 09:14:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5F5336B0082; Tue, 20 Jul 2021 09:14:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 46D1C6B0083; Tue, 20 Jul 2021 09:14:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0220.hostedemail.com [216.40.44.220]) by kanga.kvack.org (Postfix) with ESMTP id 2246F6B0081 for ; Tue, 20 Jul 2021 09:14:50 -0400 (EDT) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id AD649184FA06D for ; Tue, 20 Jul 2021 13:14:48 +0000 (UTC) X-FDA: 78383011056.20.C982D25 Received: from mail-qt1-f178.google.com (mail-qt1-f178.google.com [209.85.160.178]) by imf25.hostedemail.com (Postfix) with ESMTP id 49C99B000577 for ; Tue, 20 Jul 2021 13:14:48 +0000 (UTC) Received: by mail-qt1-f178.google.com with SMTP id c13so521329qtd.2 for ; Tue, 20 Jul 2021 06:14:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=2KUp38YsvXkumqr3XMuR3ixgpLpciARDVQEewv37dA8=; b=aVQsoNLLEUlYbW+7TK5d9IdEAm0+426B5ooqKpDUWLQ2TcGYH7ou17/g1YRTRu/aka YTJWjWj6qDwXdawDVTviUeJQu52HZWmqH0NOk44qkCsRqAFSJsS2MYYsVggAhO2t9Bli kzYNZh1IluvcnsyTK4G4fYElr3eaOtKlzHdZ0EgM4WpzolPUZc+pSuAFmHP6XrWrANzd 6EXpFYhVy40duZvLFmFqyeY2YKEoKWZ6aGU+1N6PzQ95pH9XKZvBRABfa0iA74cCeu07 cqHnhigqXEPCpeRR2Qtcy+hhGTPKCMs2iMYZICwXFtDntdmqn+8VI9GrBeqP96mlVWYN +nQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=2KUp38YsvXkumqr3XMuR3ixgpLpciARDVQEewv37dA8=; b=uTsf3t/izu9W8ebmZd6Pfr1M9GeuISboKI1QOBFGIOTK2YUKAfEPdHNlW2QaNAPg8y Xn8SbHkyLBEAu7v6Qqs70Fdno2JsdC9QGGlOnVRC3gj2eHIPLQnznlJqIbn0Y3OymOAG fp+ZwTVPMuEV/hqcUk+ChUSJVzHwkh8sI43Zez8tohD+q1SB3k+2KD/Ofn3QnoP0exib dhpPIx/ahZE1thUQr34W/SJlWQLErH6mu3OQ01gp3oO5HA6dO4GoExHZRLOuGWj1mBEj 1N2M2Ele2SgNVn/euUfH5db7+4L6XiYW7Ho3APN758NMvRrHHXF7roVHowEgq/vgEUBA qxqQ== X-Gm-Message-State: AOAM533cY6wVvcR+OkzFg4T7d2ekIAV743yKMgLkmwr8mFf+lSmJuatB bTH+el8SfIJFDhj5eUmyjzM= X-Google-Smtp-Source: ABdhPJzOM11CvnWbx/+kPZxTtC4it9bcXXkHiOXCvsRsm1pi23w+6aTKOCIzymxrnG2/6q33limkdw== X-Received: by 2002:ac8:7f07:: with SMTP id f7mr25963209qtk.120.1626786887598; Tue, 20 Jul 2021 06:14:47 -0700 (PDT) Received: from localhost.localdomain (ec2-35-169-212-159.compute-1.amazonaws.com. [35.169.212.159]) by smtp.gmail.com with ESMTPSA id g17sm9701225qkm.34.2021.07.20.06.14.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Jul 2021 06:14:47 -0700 (PDT) From: SeongJae Park To: akpm@linux-foundation.org Cc: SeongJae Park , Jonathan.Cameron@Huawei.com, acme@kernel.org, alexander.shishkin@linux.intel.com, amit@kernel.org, benh@kernel.crashing.org, brendanhiggins@google.com, corbet@lwn.net, david@redhat.com, dwmw@amazon.com, elver@google.com, fan.du@intel.com, foersleo@amazon.de, greg@kroah.com, gthelen@google.com, guoju.fgj@alibaba-inc.com, jgowans@amazon.com, joe@perches.com, mgorman@suse.de, mheyne@amazon.de, minchan@kernel.org, mingo@redhat.com, namhyung@kernel.org, peterz@infradead.org, riel@surriel.com, rientjes@google.com, rostedt@goodmis.org, rppt@kernel.org, shakeelb@google.com, shuah@kernel.org, sieberf@amazon.com, sj38.park@gmail.com, snu@zelle79.org, vbabka@suse.cz, vdavydov.dev@gmail.com, zgf574564920@gmail.com, linux-damon@amazon.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [RFC v3 15/15] Documentation/admin-guide/mm/damon: Add a document for DAMON_RECLAIM Date: Tue, 20 Jul 2021 13:13:09 +0000 Message-Id: <20210720131309.22073-16-sj38.park@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210720131309.22073-1-sj38.park@gmail.com> References: <20210720131309.22073-1-sj38.park@gmail.com> Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=aVQsoNLL; spf=pass (imf25.hostedemail.com: domain of sj38park@gmail.com designates 209.85.160.178 as permitted sender) smtp.mailfrom=sj38park@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspamd-Server: rspam02 X-Stat-Signature: 1egasba6cj1bq14hk8oc8xuh59zgkoxe X-Rspamd-Queue-Id: 49C99B000577 X-HE-Tag: 1626786888-676917 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park This commit adds an admin-guide document for DAMON-based Reclamation. Signed-off-by: SeongJae Park --- Documentation/admin-guide/mm/damon/index.rst | 1 + .../admin-guide/mm/damon/reclaim.rst | 233 ++++++++++++++++++ 2 files changed, 234 insertions(+) create mode 100644 Documentation/admin-guide/mm/damon/reclaim.rst diff --git a/Documentation/admin-guide/mm/damon/index.rst b/Documentation/admin-guide/mm/damon/index.rst index 8c5dde3a5754..61aff88347f3 100644 --- a/Documentation/admin-guide/mm/damon/index.rst +++ b/Documentation/admin-guide/mm/damon/index.rst @@ -13,3 +13,4 @@ optimize those. start usage + reclaim diff --git a/Documentation/admin-guide/mm/damon/reclaim.rst b/Documentation/admin-guide/mm/damon/reclaim.rst new file mode 100644 index 000000000000..1f4ad79919ca --- /dev/null +++ b/Documentation/admin-guide/mm/damon/reclaim.rst @@ -0,0 +1,233 @@ +.. SPDX-License-Identifier: GPL-2.0 + +======================= +DAMON-based Reclamation +======================= + +DAMON-based Reclamation (DAMON_RECLAIM) is a static kernel module that aimed to +be used for proactive and lightweight reclamation under light memory pressure. +It doesn't aim to replace the LRU-list based page_granularity reclamation, but +to be selectively used for different level of memory pressure and requirements. + +Where Proactive Reclamation is Required? +======================================== + +On general memory over-committed systems, proactively reclaiming cold pages +helps saving memory and reducing latency spikes that incurred by the direct +reclaim of the process or CPU consumption of kswapd, while incurring only +minimal performance degradation [1]_ [2]_ . + +Free Pages Reporting [3]_ based memory over-commit virtualization systems are +good example of the cases. In such systems, the guest VMs reports their free +memory to host, and the host reallocates the reported memory to other guests. +As a result, the memory of the systems are fully utilized. However, the +guests could be not so memory-frugal, mainly because some kernel subsystems and +user-space applications are designed to use as much memory as available. Then, +guests could report only small amount of memory as free to host, results in +memory utilization drop of the systems. Running the proactive reclamation in +guests could mitigate this problem. + +How It Works? +============= + +DAMON_RECLAIM finds memory regions that didn't accessed for specific time +duration and page out. To avoid it consuming too much CPU for the paging out +operation, a speed limit can be configured. Under the speed limit, it pages +out memory regions that didn't accessed longer time first. System +administrators can also configure under what situation this scheme should +automatically activated and deactivated with three memory pressure watermarks. + +Interface: Module Parameters +============================ + +To use this feature, you should first ensure your system is running on a kernel +that is built with ``CONFIG_DAMON_RECLAIM=y``. + +To let sysadmins enable or disable it and tune for the given system, +DAMON_RECLAIM utilizes module parameters. That is, you can put +``damon_reclaim.=`` on the kernel boot command line or write +proper values to ``/sys/modules/damon_reclaim/parameters/`` files. + +Note that the parameter values except ``enabled`` are applied only when +DAMON_RECLAIM starts. Therefore, if you want to apply new parameter values in +runtime and DAMON_RECLAIM is already enabled, you should disable and re-enable +it via ``enabled`` parameter file. Writing of the new values to proper +parameter values should be done before the re-enablement. + +Below are the description of each parameter. + +enabled +------- + +Enable or disable DAMON_RECLAIM. + +You can enable DAMON_RCLAIM by setting the value of this parameter as ``Y``. +Setting it as ``N`` disables DAMON_RECLAIM. Note that DAMON_RECLAIM could do +no real monitoring and reclamation due to the watermarks-based activation +condition. Refer to below descriptions for the watermarks parameter for this. + +min_age +------- + +Time threshold for cold memory regions identification in microseconds. + +If a memory region is not accessed for this or longer time, DAMON_RECLAIM +identifies the region as cold, and reclaims it. + +quota_ms +-------- + +Limit of time for the reclamation in milliseconds. + +DAMON_RECLAIM tries to use only up to this time within a time window +(quota_reset_interval_ms) for trying reclamation of cold pages. This can be +used for limiting CPU consumption of DAMON_RECLAIM. If the value is zero, the +limit is disabled. + +100 ms by default. + +quota_sz +-------- + +Limit of size of memory for the reclamation in bytes. + +DAMON_RECLAIM charges amount of memory which it tried to reclaim within a time +window (quota_reset_interval_ms) and makes no more than this limit is tried. +This can be used for limiting consumption of CPU and IO. If this value is +zero, the limit is disabled. + +1 GiB by default. + +quota_reset_interval_ms +----------------------- + +The time/size quota charge reset interval in milliseconds. + +The charget reset interval for the quota of time (quota_ms) and size +(quota_sz). That is, DAMON_RECLAIM does not try reclamation for more than +quota_ms milliseconds or quota_sz bytes within quota_reset_interval_ms +milliseconds. + +1 second by default. + +wmarks_interval +--------------- + +Minimal time to wait before checking the watermarks, when DAMON_RECLAIM is +enabled but inactive due to its watermarks rule. + +wmarks_high +----------- + +Free memory rate (per thousand) for the high watermark. + +If free memory of the system in bytes per thousand bytes is higher than this, +DAMON_RECLAIM becomes inactive, so it does nothing but only periodically checks +the watermarks. + +wmarks_mid +---------- + +Free memory rate (per thousand) for the middle watermark. + +If free memory of the system in bytes per thousand bytes is between this and +the low watermark, DAMON_RECLAIM becomes active, so starts the monitoring and +the reclaiming. + +wmarks_low +---------- + +Free memory rate (per thousand) for the low watermark. + +If free memory of the system in bytes per thousand bytes is lower than this, +DAMON_RECLAIM becomes inactive, so it does nothing but periodically checks the +watermarks. In the case, the system falls back to the LRU-list based page +granularity reclamation logic. + +sample_interval +--------------- + +Sampling interval for the monitoring in microseconds. + +The sampling interval of DAMON for the cold memory monitoring. Please refer to +the DAMON documentation (:doc:`usage`) for more detail. + +aggr_interval +------------- + +Aggregation interval for the monitoring in microseconds. + +The aggregation interval of DAMON for the cold memory monitoring. Please +refer to the DAMON documentation (:doc:`usage`) for more detail. + +min_nr_regions +-------------- + +Minimum number of monitoring regions. + +The minimal number of monitoring regions of DAMON for the cold memory +monitoring. This can be used to set lower-bound of the monitoring quality. +But, setting this too high could result in increased monitoring overhead. +Please refer to the DAMON documentation (:doc:`usage`) for more detail. + +max_nr_regions +-------------- + +Maximum number of monitoring regions. + +The maximum number of monitoring regions of DAMON for the cold memory +monitoring. This can be used to set upper-bound of the monitoring overhead. +However, setting this too low could result in bad monitoring quality. Please +refer to the DAMON documentation (:doc:`usage`) for more detail. + +monitor_region_start +-------------------- + +Start of target memory region in physical address. + +The start physical address of memory region that DAMON_RECLAIM will do work +against. That is, DAMON_RECLAIM will find cold memory regions in this region +and reclaims. By default, biggest System RAM is used as the region. + +monitor_region_end +------------------ + +End of target memory region in physical address. + +The end physical address of memory region that DAMON_RECLAIM will do work +against. That is, DAMON_RECLAIM will find cold memory regions in this region +and reclaims. By default, biggest System RAM is used as the region. + +kdamond_pid +----------- + +PID of the DAMON thread. + +If DAMON_RECLAIM is enabled, this becomes the PID of the worker thread. Else, +-1. + +Example +======= + +Below runtime example commands make DAMON_RECLAIM to find memory regions that +not accessed for 30 seconds or more and pages out. The reclamation is limited +to be done only up to 1 GiB per second to avoid DAMON_RECLAIM consuming too +much CPU time for the paging out operation. It also asks DAMON_RECLAIM to do +nothing if the system's free memory rate is more than 50%, but start the real +works if it becomes lower than 40%. If DAMON_RECLAIM doesn't make progress and +therefore the free memory rate becomes lower than 20%, it asks DAMON_RECLAIM to +do nothing again, so that we can fall back to the LRU-list based page +granularity reclamation. :: + + # cd /sys/modules/damon_reclaim/parameters + # echo 30000000 > min_age + # echo $((1 * 1024 * 1024 * 1024)) > quota_sz + # echo 1000 > quota_reset_interval_ms + # echo 500 > wmarks_high + # echo 400 > wmarks_mid + # echo 200 > wmarks_low + # echo Y > enabled + +.. [1] https://research.google/pubs/pub48551/ +.. [2] https://lwn.net/Articles/787611/ +.. [3] https://www.kernel.org/doc/html/latest/vm/free_page_reporting.html