From patchwork Fri Feb 28 22:03:22 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 13997207 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7C7E9C282D0 for ; Fri, 28 Feb 2025 22:03:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DDAEC280004; Fri, 28 Feb 2025 17:03:36 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D3BF5280001; Fri, 28 Feb 2025 17:03:36 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B8F1E280004; Fri, 28 Feb 2025 17:03:36 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 9DF5E280001 for ; Fri, 28 Feb 2025 17:03:36 -0500 (EST) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 45EC514135A for ; Fri, 28 Feb 2025 22:03:36 +0000 (UTC) X-FDA: 83170730832.20.AE559D5 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf02.hostedemail.com (Postfix) with ESMTP id 9078080011 for ; Fri, 28 Feb 2025 22:03:34 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=vGfRkR+Y; spf=pass (imf02.hostedemail.com: domain of sj@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=sj@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740780214; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=zmGs8Er9RYV4HDgxFiKDVP+pssCzJiVA9KZtO+ha3js=; b=5DNH0azDOYAyVsA43yw8NJyUPMZtB2TRznAvLok5KZR2JI8BFrK+6DZiSJl6byHkUo5oD+ vzEaUW2gn38iizQwVo1Gd74reLZGbjBNs1ZyVjrkrQ/hhbP1Ko4N9UHTfmREiFR3EYIPWP dRg4vuzFThmIElIAIeUb0yAwM7i8pF8= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=vGfRkR+Y; spf=pass (imf02.hostedemail.com: domain of sj@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=sj@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740780214; a=rsa-sha256; cv=none; b=NZOAEn1gz+mMLUhOdnewXBCsSR4760gYytd412lfXOA8zPkqysYTU8qfWksmiANtFYoy8D l4nGHSOPIVXq2DaocrrA4/KVa48Ql+nOuWiyCiUEVXtZXvFjQ8MF0H7SHYwUS97mtPgoju CaX3OVNfU2jNMAvxzXiEIKAeCw/GcK8= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id F2A0C5C6A40; Fri, 28 Feb 2025 22:01:16 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 781B7C4CEE2; Fri, 28 Feb 2025 22:03:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1740780213; bh=w21tstzzb4FfHupcVwHDLvqkq/QXU7C/Es38OxUzGJE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=vGfRkR+YJIBl3Vea7jC1iBcMljBOVbkTePmmNC8lkKlsV7rX0LY8snKXVd+qbhs/0 uMu5P8J3qPX6bnBq+lua2Djqx6g/S5QxO6G2/wGhW9ze/UmZOUqmW27vQbSnQPyp8v AxZUiJSIOp5uGFJDYsRcTrIi4aSBY9Se+BZeT9HplmzY02OL3VR512RWaaQgVeWCZ4 G+r3xT2IcgOR1AAilvNuSDoEb1zLS5rW6Vop0tRrfA6lauWKbkTwho6XF5uqmjcP3u BH2nWDJZD6DegKZqBfPB2ulMTD1R+0JGgNmUT/ks4s9JK36V76ceRgGMbp5BWR1ujv sv1J4TYVU+Y+g== From: SeongJae Park To: Cc: SeongJae Park , Andrew Morton , damon@lists.linux.dev, kernel-team@meta.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v2 2/8] mm/damon/core: implement intervals auto-tuning Date: Fri, 28 Feb 2025 14:03:22 -0800 Message-Id: <20250228220328.49438-3-sj@kernel.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250228220328.49438-1-sj@kernel.org> References: <20250228220328.49438-1-sj@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 9078080011 X-Rspamd-Server: rspam09 X-Stat-Signature: 9t1dad4f7edgemb9rg6kmwbj9uohng1w X-HE-Tag: 1740780214-636716 X-HE-Meta: U2FsdGVkX19fYqqauTURnvFRDPKtqxoj5rEi3RFA8PfqBJ6jAcIBrCRfhqg8/e6SPORYrzje55IUcdUwScM8MHPNt+AeX2MtxpPQN8yqS7CZt8yYyapHhxOmrfsJ+gmNxuHeTbBy3nZvper1riV3Mo00DhR1+m66yniqgmKWBDfgXD0UvrAw2HgXjAmI4RH2gMOpiocK4PLjBfswxzgD9V0FLyeO5MwTEbCSdxMQry6BVmqdKVzGyLwCbVmasIuVv8QbPlqO1heweNls3XGmEExAk9cekNoXem+JsXYM8PIuT02sX7I/Rq+e+dTXLLFhS3UP/Wiuoth98ugt4APw9JJ4eoiUji2DRrKHtYDLHbMfUP+8qcO4eOGumkHwt/vM7tnXFlCMcm+vc8hvKrLaY4ft3kRu26vqI/E9tK6dG+6HQBuupqpztUN5ASk+RMeD0HFFTeXNYkfgy9baKLywtcklp1kALUulEs70qoeWs06sOZoMXB/pBFPj5JG8WJjCrXpcs01wFUKSe/YOWk1t6FJ8mbksQbNG0imGT90vgpUoZnd/8OXheY5shTdg01Z7AJaFs+4HTNcopNw9ev/gPDuGYMmTjJb08w4RM63cZa5DqFZRms7uVRN17Sh7tgEvtmmiB6+QRS0+DBT4kYVKuHhRgQOayDpK0+HsWGblxS7zTV+AdYxE4IvoD/mVKLQo7uxfRItxH61MUNT0ghY3Gy1JDbqKfB9Z4ihPUBXdH3lJoI2KI24G9azt4QQQSAh4g57LAznZuKc1SdLK5lcDNPq0KtDI3kC9aRa2o4K/IqvHj80g5N9+jjb+wRU4tzV1uokk1M5ZmbIp6lGQkYV4ED8637drSFt6Mj3002cgIul3i1c8Am3FyeAusuEBURyOLKT438yGM91bolmTml4wMkqKJVDu2LScXYhiDZG5K3L6b4iNBLzrUrc+Piibk1ThLcP7UiiKb0SNpcglae9 ech6+wW6 kafsY4+xfJ6VF9UMTiMwHXBgq9cYylxrFsWMpur/hG6YIICNcoZ0NiWKaXqzZpfY9iiN06Zeu86LApTwRwaPtLRC8rCiwYvEhisdsAf/H91FOco1oq/xsvBt+olHsPIGhguZQKH4Ou/L0W+Hx9SYcORO2Uk6jnfzaSW7IFfW2ItKHnB/ju9nbFP/s8CmOVoNfOkBadkzy/Nme6rLWPePV81MiqoQ9JRaCOk8dwegKyV9gaPw= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Implement the DAMON sampling and aggregation intervals auto-tuning mechanism as briefly described on 'struct damon_intervals_goal'. The core part for deciding the direction and amount of the changes is implemented reusing the feedback loop function which is being used for DAMOS quotas auto-tuning. Unlike the DAMOS quotas auto-tuning use case, limit the maximum decreasing amount after the adjustment to 50% of the current value, though. This is because the intervals have no good merits at rapid reductions since it could unnecessarily increase the monitoring overhead. Signed-off-by: SeongJae Park --- include/linux/damon.h | 16 +++++++++ mm/damon/core.c | 76 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 92 insertions(+) diff --git a/include/linux/damon.h b/include/linux/damon.h index 130c3e01089c..eed008b64a23 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -718,6 +718,17 @@ struct damon_attrs { struct damon_intervals_goal intervals_goal; unsigned long min_nr_regions; unsigned long max_nr_regions; +/* private: internal use only */ + /* + * @aggr_interval to @sample_interval ratio. + * Core-external components call damon_set_attrs() with &damon_attrs + * that this field is unset. In the case, damon_set_attrs() sets this + * field of resulting &damon_attrs. Core-internal components such as + * kdamond_tune_intervals() calls damon_set_attrs() with &damon_attrs + * that this field is set. In the case, damon_set_attrs() just keep + * it. + */ + unsigned long aggr_samples; }; /** @@ -766,6 +777,11 @@ struct damon_ctx { * update */ unsigned long next_ops_update_sis; + /* + * number of sample intervals that should be passed before next + * intervals tuning + */ + unsigned long next_intervals_tune_sis; /* for waiting until the execution of the kdamond_fn is started */ struct completion kdamond_started; /* for scheme quotas prioritization */ diff --git a/mm/damon/core.c b/mm/damon/core.c index b11595c01c37..d4fc2c907cfc 100644 --- a/mm/damon/core.c +++ b/mm/damon/core.c @@ -687,6 +687,10 @@ int damon_set_attrs(struct damon_ctx *ctx, struct damon_attrs *attrs) if (attrs->sample_interval > attrs->aggr_interval) return -EINVAL; + /* calls from core-external doesn't set this. */ + if (!attrs->aggr_samples) + attrs->aggr_samples = attrs->aggr_interval / sample_interval; + ctx->next_aggregation_sis = ctx->passed_sample_intervals + attrs->aggr_interval / sample_interval; ctx->next_ops_update_sis = ctx->passed_sample_intervals + @@ -1392,6 +1396,65 @@ static void kdamond_reset_aggregated(struct damon_ctx *c) } } +static unsigned long damon_get_intervals_score(struct damon_ctx *c) +{ + struct damon_target *t; + struct damon_region *r; + unsigned long sz_region, max_access_events = 0, access_events = 0; + unsigned long target_access_events; + unsigned long goal_bp = c->attrs.intervals_goal.access_bp; + + damon_for_each_target(t, c) { + damon_for_each_region(r, t) { + sz_region = damon_sz_region(r); + max_access_events += sz_region * c->attrs.aggr_samples; + access_events += sz_region * r->nr_accesses; + } + } + target_access_events = max_access_events * goal_bp / 10000; + return access_events * 10000 / target_access_events; +} + +static unsigned long damon_feed_loop_next_input(unsigned long last_input, + unsigned long score); + +static unsigned long damon_get_intervals_adaptation_bp(struct damon_ctx *c) +{ + unsigned long score_bp, adaptation_bp; + + score_bp = damon_get_intervals_score(c); + adaptation_bp = damon_feed_loop_next_input(100000000, score_bp) / + 10000; + /* + * adaptaion_bp ranges from 1 to 20,000. Avoid too rapid reduction of + * the intervals by rescaling [1,10,000] to [5000, 10,000]. + */ + if (adaptation_bp <= 10000) + adaptation_bp = 5000 + adaptation_bp / 2; + return adaptation_bp; +} + +static void kdamond_tune_intervals(struct damon_ctx *c) +{ + unsigned long adaptation_bp; + struct damon_attrs new_attrs; + struct damon_intervals_goal *goal; + + adaptation_bp = damon_get_intervals_adaptation_bp(c); + if (adaptation_bp == 10000) + return; + + new_attrs = c->attrs; + goal = &c->attrs.intervals_goal; + new_attrs.sample_interval = min(goal->max_sample_us, + c->attrs.sample_interval * adaptation_bp / 10000); + new_attrs.sample_interval = max(goal->min_sample_us, + new_attrs.sample_interval); + new_attrs.aggr_interval = new_attrs.sample_interval * + c->attrs.aggr_samples; + damon_set_attrs(c, &new_attrs); +} + static void damon_split_region_at(struct damon_target *t, struct damon_region *r, unsigned long sz_r); @@ -2300,6 +2363,8 @@ static void kdamond_init_intervals_sis(struct damon_ctx *ctx) ctx->next_aggregation_sis = ctx->attrs.aggr_interval / sample_interval; ctx->next_ops_update_sis = ctx->attrs.ops_update_interval / sample_interval; + ctx->next_intervals_tune_sis = ctx->next_aggregation_sis * + ctx->attrs.intervals_goal.aggrs; damon_for_each_scheme(scheme, ctx) { apply_interval = scheme->apply_interval_us ? @@ -2384,6 +2449,17 @@ static int kdamond_fn(void *data) sample_interval = ctx->attrs.sample_interval ? ctx->attrs.sample_interval : 1; if (ctx->passed_sample_intervals >= next_aggregation_sis) { + if (ctx->attrs.intervals_goal.aggrs && + ctx->passed_sample_intervals >= + ctx->next_intervals_tune_sis) { + ctx->next_intervals_tune_sis += + ctx->attrs.aggr_samples * + ctx->attrs.intervals_goal.aggrs; + kdamond_tune_intervals(ctx); + sample_interval = ctx->attrs.sample_interval ? + ctx->attrs.sample_interval : 1; + + } ctx->next_aggregation_sis = next_aggregation_sis + ctx->attrs.aggr_interval / sample_interval;