From patchwork Mon Feb 19 19:44:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 13563009 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CC8B4C48BC3 for ; Mon, 19 Feb 2024 19:44:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E12916B008C; Mon, 19 Feb 2024 14:44:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D9A956B0093; Mon, 19 Feb 2024 14:44:46 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C15926B0092; Mon, 19 Feb 2024 14:44:46 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id A62486B0089 for ; Mon, 19 Feb 2024 14:44:46 -0500 (EST) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 548BE12038D for ; Mon, 19 Feb 2024 19:44:46 +0000 (UTC) X-FDA: 81809580972.26.BB19D8D Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf11.hostedemail.com (Postfix) with ESMTP id 9A8AD4000F for ; Mon, 19 Feb 2024 19:44:44 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Z7CKpYy3; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf11.hostedemail.com: domain of sj@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=sj@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1708371884; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=vwLHs0PKGhGSIgKerYJ521PT+13VxaMIHLgN3P/NRzk=; b=4giMN5MsWHXTtNafKTs9Poq6xxuLyasYgHK7kzP4F2ZlHlvu/+vHJipG1dAguvg5HYSARK +pKLGP8yhWDjmbJTppgnl+CxfDOX9SAPIytJMWlZx5E/42jd+XN8Jvl0q3Gx7LjmzZS75U qv1HAquQv5MZLHoRLUDmEknNcIC7AMI= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Z7CKpYy3; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf11.hostedemail.com: domain of sj@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=sj@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1708371884; a=rsa-sha256; cv=none; b=xm8+wJJuEV7E3S0GsijIpCMMIv03yMw4x9KBef+/9rx+U5qYsn6Tj9hM3qeQYAXvM1Ti1J wV7uIMfa+nOZQ/vPJoz87fg0onwHVWCzSpexTPwYAckSXgiIWUhlCRBJBZ16gBs5V/ouWj WR/Ra1FJWK8TAfhktb3l8JghQjrMIfM= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id D0AA860F78; Mon, 19 Feb 2024 19:44:43 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 46B6EC433B2; Mon, 19 Feb 2024 19:44:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1708371883; bh=oql1XzmeT1+l2YfyGdM6ybnvM0NrgJg8QAeQ/QOsNwg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Z7CKpYy3zZMnNZ2RLrlm/AhKAtiDKdltHNeltt9YG/c//avZ/C21btvKo+6IMHC96 LuwXn48YUa67+phLsPiSdim1cxKHYuV0Kb8Q9wGuJtXotzBoNuJ3dItv/tLRJeL+QQ d/ph0VDegeSUVff2L1TWYXgi8lgOT8AP06X9xZHbaMikLZS7a0IzQBAVQx/htTH6Vg WvzON9voOco8RI33NMG3fCC//hezOSpAo+mrz2fhCQP1CWpuofBL8v+QUxd6zewgej /QJNyZg1x6+1scwJCV3Ql/pIaF55xQUw6DDSxXLKge0+xX8eYYgdvgZ3QfHD/Dd+Z0 tBvtii6hP0Qew== From: SeongJae Park To: Andrew Morton Cc: SeongJae Park , damon@lists.linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 08/20] mm/damon/core: add multiple goals per damos_quota and helpers for those Date: Mon, 19 Feb 2024 11:44:19 -0800 Message-Id: <20240219194431.159606-9-sj@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240219194431.159606-1-sj@kernel.org> References: <20240219194431.159606-1-sj@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 9A8AD4000F X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: fzyeam5879mar5xe9139qicjanzkzgy3 X-HE-Tag: 1708371884-909591 X-HE-Meta: U2FsdGVkX198ntTBQhDa62emXbyolkxexuM+K7lWfj7AqgAbtzuI63sPrl3FT8l1/NUMhFVlv7WSVdLZOQkKs3y3EcaQpRXkpHlRE1n4dVBoWPtqczCC6LicKSm90NQ83tPopmgnRBWMLj/l8si9YHyVh+iczvHnubHuIo8sgXQ2NJEnJLsRh9tzBahaJ/11YzEPVcrvjAXtyZzOYpYKRV3GS+8fhyNCh901NLX+PuHudg4Gt1iPmfr6kx7F7VnllsVZAV50VfvNHVbVmjU40qA5Yks9F5BzPtfyBsgD2TxEqCbJ2Iy0NX5gzQeGHTaxM7tTngUN+T0WYSgDyxAnjFRrALrDUt9awF1qqBfMq04NDvR6CkKWGeIpdRhVP6DqaiVn2q31gahR2L/PvjrB3BTfLYa6XK07XgaS+iNwfaLoB/kI1bkMyPHDGcMqXFHkbTI2Hu/REZvHdcJdzMlkCU+IOTyv0Vvxm5xVmaLxjk4tsaXnDQxHDwFiCQCE4DbRlquxkWNwtLskE+Yzi83kF5EqqjIkxa0ocAsrEf9Yk0UKJUcVOJGiy2n81nQuDl9i5nSf7Y29Phw0FO4yFMNkVXwpn8IjmwWBgDAhHvB1ynt/Dm55z0czsV4XocKaFewpQ079GnV+eegRjZDL6VuA2A2+7kcaMSHeEAORLdKbWfrTfS1W9MFYjp5SL3TNaX7S4N30N9pSWUa3NAEwTQC7JERFNXHbvFYovuUgTf5McRhuVyA81DhhXc9gxUyTQP1mP2xOOC5NS4dBt5o8yfFXCqWvDGWCSbIDhlBDwl2vKKTtPMS2AD7j8mZeaHmJnlHI6r+7JBgVxBUFIflw7RIJXHehBqicq3GzOILPyLDzhDWOGGpFyDgYrMGvHeI3AOJ90dmCKpj2iQeuuJsQ1lv8DI2sRg8/59ub2qYzRANqdgKFCpaXsD4k9D/gAf8MBLn6BSfFqOKyMtQXKvT3gKN hZxoA4MQ 2+lA3//aOlCZA2SDHXg6X7TbkOjkksjGyGW9jhVMUWlHxK9YMNjPTCq3D5g76SVCdqwM+5WNCu8i1uG6hjEhoZVqUA7zq6krPsWE1WN/EyHnnbRDpxj8BFQJlv4nm/Nl/x9BtRVICgOaz8QxVcWaBb75vErNI701sHPtzwYzjwxNO8QULg7jPOSlaeScpPdQ4RFt0OnobxUkZ/J79rInARUfvXwnrpxWADynpF1Lett1k536gyIcwmGFEOOnetsBdMeAHQ1Y/C976vJmeS4NxYDgXp8WHSd53TwbX X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The feedback-driven DAMOS quota auto-tuning feature allows only single goal to the DAMON kernel API users. The API users could implement multiple goals for the end-users on their level, and that's what DAMON sysfs interface is doing. More DAMON kernel API users such as DAMON_RECLAIM would need to do similar work. To reduce unnecessary future duplciated efforts, support multiple goals from DAMOS core layer. To make the support in minimum non-destructive change, keep the old single goal setup interface, and add multiple goals setup. The single goal will treated as one of the multiple goals, so old API users are not required to make any change. Signed-off-by: SeongJae Park --- include/linux/damon.h | 17 ++++++++++ mm/damon/core.c | 78 +++++++++++++++++++++++++++++++++++++++---- 2 files changed, 88 insertions(+), 7 deletions(-) diff --git a/include/linux/damon.h b/include/linux/damon.h index 2fe345adf6b2..4bd898eaf80e 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -130,6 +130,7 @@ enum damos_action { * struct damos_quota_goal - DAMOS scheme quota auto-tuning goal. * @get_score: Function for getting current score of the goal. * @get_score_arg: Parameter for @get_score + * @list: List head for siblings. * * Data structure for getting the current score of the quota tuning goal. * Calling @get_score with @get_score_arg as the parameter should return the @@ -140,6 +141,7 @@ enum damos_action { struct damos_quota_goal { unsigned long (*get_score)(void *arg); void *get_score_arg; + struct list_head list; }; /** @@ -148,6 +150,7 @@ struct damos_quota_goal { * @ms: Maximum milliseconds that the scheme can use. * @sz: Maximum bytes of memory that the action can be applied. * @goal: Quota auto-tuning goal. + * @goals: Head of quota tuning goals (&damos_quota_goal) list. * @esz: Effective size quota in bytes. * * @weight_sz: Weight of the region's size for prioritization. @@ -171,6 +174,8 @@ struct damos_quota_goal { * every @reset_interval. Then, if the new size quota is smaller than the * effective quota, it uses the new size quota as the effective quota. * + * If @goals is not empty, same action is taken for each goal of the list. + * * The resulting effective size quota in bytes is set to @esz. * * For selecting regions within the quota, DAMON prioritizes current scheme's @@ -184,6 +189,7 @@ struct damos_quota { unsigned long ms; unsigned long sz; struct damos_quota_goal goal; + struct list_head goals; unsigned long esz; unsigned int weight_sz; @@ -648,6 +654,12 @@ static inline unsigned long damon_sz_region(struct damon_region *r) #define damon_for_each_scheme_safe(s, next, ctx) \ list_for_each_entry_safe(s, next, &(ctx)->schemes, list) +#define damos_for_each_quota_goal(goal, quota) \ + list_for_each_entry(goal, "a->goals, list) + +#define damos_for_each_quota_goal_safe(goal, next, quota) \ + list_for_each_entry_safe(goal, next, &(quota)->goals, list) + #define damos_for_each_filter(f, scheme) \ list_for_each_entry(f, &(scheme)->filters, list) @@ -681,6 +693,11 @@ struct damos_filter *damos_new_filter(enum damos_filter_type type, void damos_add_filter(struct damos *s, struct damos_filter *f); void damos_destroy_filter(struct damos_filter *f); +struct damos_quota_goal *damos_new_quota_goal( + unsigned long (*get_score)(void *), void *get_score_arg); +void damos_add_quota_goal(struct damos_quota *q, struct damos_quota_goal *g); +void damos_destroy_quota_goal(struct damos_quota_goal *goal); + struct damos *damon_new_scheme(struct damos_access_pattern *pattern, enum damos_action action, unsigned long apply_interval_us, diff --git a/mm/damon/core.c b/mm/damon/core.c index fe4209672121..b6cd99b64e85 100644 --- a/mm/damon/core.c +++ b/mm/damon/core.c @@ -299,6 +299,41 @@ void damos_destroy_filter(struct damos_filter *f) damos_free_filter(f); } +struct damos_quota_goal *damos_new_quota_goal( + unsigned long (*get_score)(void *), void *get_score_arg) +{ + struct damos_quota_goal *goal; + + goal = kmalloc(sizeof(*goal), GFP_KERNEL); + if (!goal) + return NULL; + goal->get_score = get_score; + goal->get_score_arg = get_score_arg; + INIT_LIST_HEAD(&goal->list); + return goal; +} + +void damos_add_quota_goal(struct damos_quota *q, struct damos_quota_goal *g) +{ + list_add_tail(&g->list, &q->goals); +} + +static void damos_del_quota_goal(struct damos_quota_goal *g) +{ + list_del(&g->list); +} + +static void damos_free_quota_goal(struct damos_quota_goal *g) +{ + kfree(g); +} + +void damos_destroy_quota_goal(struct damos_quota_goal *g) +{ + damos_del_quota_goal(g); + damos_free_quota_goal(g); +} + /* initialize fields of @quota that normally API users wouldn't set */ static struct damos_quota *damos_quota_init(struct damos_quota *quota) { @@ -337,6 +372,8 @@ struct damos *damon_new_scheme(struct damos_access_pattern *pattern, INIT_LIST_HEAD(&scheme->list); scheme->quota = *(damos_quota_init(quota)); + /* quota.goals should be separately set by caller */ + INIT_LIST_HEAD(&scheme->quota.goals); scheme->wmarks = *wmarks; scheme->wmarks.activated = true; @@ -373,8 +410,12 @@ static void damon_free_scheme(struct damos *s) void damon_destroy_scheme(struct damos *s) { + struct damos_quota_goal *g, *g_next; struct damos_filter *f, *next; + damos_for_each_quota_goal_safe(g, g_next, &s->quota) + damos_destroy_quota_goal(g); + damos_for_each_filter_safe(f, next, s) damos_destroy_filter(f); damon_del_scheme(s); @@ -1083,22 +1124,44 @@ static unsigned long damon_feed_loop_next_input(unsigned long last_input, return min_input; } -/* Called only if quota->ms, quota->sz, or quota->goal.get_score are set */ +/* Return the highest score since it makes schemes least aggressive */ +static unsigned long damos_quota_score(struct damos_quota *quota) +{ + struct damos_quota_goal *goal; + unsigned long highest_score = 0; + + if (quota->goal.get_score) + highest_score = quota->goal.get_score( + quota->goal.get_score_arg); + + damos_for_each_quota_goal(goal, quota) + highest_score = max(highest_score, + goal->get_score(goal->get_score_arg)); + + return highest_score; +} + +/* + * Called only if quota->ms, quota->sz, or quota->goal.get_score are set, or + * quota->goals is not empty + */ static void damos_set_effective_quota(struct damos_quota *quota) { unsigned long throughput; unsigned long esz; - if (!quota->ms && !quota->goal.get_score) { + if (!quota->ms && !quota->goal.get_score && + list_empty("a->goals)) { quota->esz = quota->sz; return; } - if (quota->goal.get_score) { + if (quota->goal.get_score || !list_empty("a->goals)) { + unsigned long score = damos_quota_score(quota); + quota->esz_bp = damon_feed_loop_next_input( max(quota->esz_bp, 10000UL), - quota->goal.get_score( - quota->goal.get_score_arg)); + score); esz = quota->esz_bp / 10000; } @@ -1108,7 +1171,7 @@ static void damos_set_effective_quota(struct damos_quota *quota) quota->total_charged_ns; else throughput = PAGE_SIZE * 1024; - if (quota->goal.get_score) + if (quota->goal.get_score || !list_empty("a->goals)) esz = min(throughput * quota->ms, esz); else esz = throughput * quota->ms; @@ -1128,7 +1191,8 @@ static void damos_adjust_quota(struct damon_ctx *c, struct damos *s) unsigned long cumulated_sz; unsigned int score, max_score = 0; - if (!quota->ms && !quota->sz && !quota->goal.get_score) + if (!quota->ms && !quota->sz && !quota->goal.get_score && + list_empty("a->goals)) return; /* New charge window starts */