From patchwork Fri Dec 13 21:53:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 13907892 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EA61CE77183 for ; Fri, 13 Dec 2024 21:53:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3ED896B0088; Fri, 13 Dec 2024 16:53:23 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 377C66B0089; Fri, 13 Dec 2024 16:53:23 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 12C206B008C; Fri, 13 Dec 2024 16:53:23 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id E6A7B6B0088 for ; Fri, 13 Dec 2024 16:53:22 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 9EAD1A124D for ; Fri, 13 Dec 2024 21:53:22 +0000 (UTC) X-FDA: 82891286856.05.0B5B0A9 Received: from nyc.source.kernel.org (nyc.source.kernel.org [147.75.193.91]) by imf02.hostedemail.com (Postfix) with ESMTP id 5A26180007 for ; Fri, 13 Dec 2024 21:52:31 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=X7Hv9Kwf; spf=pass (imf02.hostedemail.com: domain of sj@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=sj@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1734126789; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=gpUHTOmwJQN4qPYrzfbajSWNaKk48fT0ZDaV60ybycU=; b=2g6ZTpOYc0vCmIZ92dbcgQupU6oED5NXfAQ1efXHLGRXIeWRKIxVg8L1EAZf05SLQKFoMw CyfJgfOCB+V9/pkXd+835J3KoGI7d/U3h4Ca4b8kD2j9pNsRaOd4h5M6y8o2d7PbffgssJ 5Rj8yXSqAG5Owz8b1WSoNku8Ty5Tzpw= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=X7Hv9Kwf; spf=pass (imf02.hostedemail.com: domain of sj@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=sj@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1734126789; a=rsa-sha256; cv=none; b=skvm9b9/23jDEsor1Re3CQOSw3zarK36G/2wOf6JE+FySKk5Z3tI8kf+CnpjhW8vqJybCO 32XtxnNhdOy7XR2+duiWVuvIO72mAgBrzX1wIgCdIkhy57hOMWQIrW9GQTn5CXKTGReKFW eV9Y1Bkgm0Mcj5zgDIF2RDdZLfVahtI= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id 41CE0A42E07 for ; Fri, 13 Dec 2024 21:51:29 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C62C1C4CED6; Fri, 13 Dec 2024 21:53:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1734126800; bh=CuegZeDM4FxKE/fdac0FkRb85mikc51KmPkJxxesedI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=X7Hv9Kwf5zS9WP/rJRXgSfMY+UFevhNx7vSTcspG6xNCHaSbxxbX2m3bTvmne9I0G BEvAeYFmxKA621fItAOJdn4kpDz0A4qUQUeXuluyldhTqUnDKw7BwOuVF8wFUfs6Yl 3Cl5lVBeiP4uU7Rq9MSLiaMXsTNTIls70NWqvi+EcxgSt7vlJ5IYqBrWStt0jb/5VJ 7xaL2ZIRyCgzR5tL2wNDcrbyHFcGZs9+rQ3CQFLgF2c5x7vl7hljZaNitST7UlmboJ eSj3i3dA2IJNvIsqZYMVBY0Inj6CPul/Wu5Q1UoFb3hKd+ZAPBjyzbsXV+zkcXZlkd gUqLS67FqgmNQ== From: SeongJae Park To: Cc: SeongJae Park , damon@lists.linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [RFC PATCH 3/9] mm/damon/core: implement damon_call() Date: Fri, 13 Dec 2024 13:53:00 -0800 Message-Id: <20241213215306.54778-4-sj@kernel.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20241213215306.54778-1-sj@kernel.org> References: <20241213215306.54778-1-sj@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 5A26180007 X-Stat-Signature: 8ymnjm47ptjaqgfyfi1tyu8k6og9bqjt X-Rspam-User: X-Rspamd-Server: rspam11 X-HE-Tag: 1734126751-758815 X-HE-Meta: U2FsdGVkX19ve6/WIViEB2e/JX8P142G7dRr3Zjm8VQYgtxCZUcIiUyUUvhqDEwWxHthwFtoYC+bYeBfwvcM1ZO05CTNS1hwCzSeDy/hR6DnJIL9Kutv4dUvTGuMjIS95XnTBtSaPT1pA4JRprs3ElqQSDsrtNDrhZAyjCT8SETnaKyx76iU7O5cR6450L0CPbNkrdDL+DVj4eHcemiL+NF6TYHzYazvNXIsxuR0ttaffKTi4oEcTcx5SI0kcz4Fk8eTtDnqyGa7Pu7NKiis4+J4k9ZzORnp2K6VvFVKKp3bmGa6ONbWX5uDnRoODyTbaeZUb1TaVNMrBINQB6zTrWmdn2wI/XMQEWyJuh2m+FPwNN1ogS27IusLp8XXYhG9G0p4XEpte9drrd/NaC8i5fdHo6OC8ZaPIfWQQd/gJVYwiIXh7IyikIez6CkvNtIk0r/1EWwYSV/kC9eXW0PqRUO6+MqrrG3KOIFo7alAfjdyLlCripvSna+9FIdpJXmpZtijFG1d2GmmGZAcjF/nAU55AWrBpQ7UqrehxDTV7ZQNbUa4dZADtpZWyLfndAErmQ8P7fSmRoFgHlUiJtjyt27yQAu0S4epfr+yifl2HaZffvxKb80ITc7fBEEuw3uff5GLGMVbOGx6wlvL1SWOO9FYc9P0/uaypdiHhnNzT7SQ9GRBULfeyzZmoiB6LfwwcuWf2KUUFKfB6hgKU/FwELESbW0YHZZ4Oi03rZvk362RZsIlP1OKyeBuHaM7F1tli7Rfkxu3wh3ic3Y5cGIIiUcZ86iAZCg+0sPqf75IrBkYRjTkNQGyw0GYoMpM+yxPZhuNm6STQfYV5DpL1QL4gfHj2yoAaMeuoQ+xqJ4b888PcIhjksejzhy7PmEAfSsehiO+k0pTPXRDZLXIla7nk+hCJewJaRu5cnu/Kclc+3GDUuhi+k41V9/A541q+UV4kZN0HsulnrcD7QZ18SB OrSp306d bdXvCIaIheYXE4QcOq0t5t2gqDvXlwW2aadGTWaDHkmNo+HyedBav5150Wb/k+Rm9A5LUIb3EhhPfk9XT/uJpewfm19k45U6j1ziBNgOb8AhWezxAq2dqrJo634y0kDb+9nwoEYWxx8hUGpuUbdLLrnsLgeKnXMxbOlpjZPkLulKkWr47TFbHxcaNYv1bwq8BnbhppFDxwY7zQU7EtPFqLjO8N3INrKyJziVKFRi0qEMWOBVGDdAbArAT+BXGfBoA8XcgmQLRoX5OiPoQmIaOn/1OVw33X4yW6GLXWEurYUHd++KbhI5AQlGsenbp6lIW1Oag2ZWfW1LXEmaMH7GzZ1XbO3gax06j4Q/8 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Introduce a new DAMON core API function, damon_call(). It aims to replace some damon_callback usages that access damon_ctx of ongoing kdamond with additional synchronizations. It receives a function pointer, let the parallel kdamond invokes the function, and returns after the invocation is finished, or canceled due to some races. kdamond invokes the function inside the main loop after sampling is done. If it is deactivated by DAMOS watermarks or already out of the main loop, mark the request as canceled so that damon_call() can wakeup and return. Signed-off-by: SeongJae Park --- include/linux/damon.h | 26 +++++++++++++ mm/damon/core.c | 86 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 112 insertions(+) diff --git a/include/linux/damon.h b/include/linux/damon.h index 10fc6df52111..529ea578f2d5 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -590,6 +590,27 @@ struct damon_callback { void (*before_terminate)(struct damon_ctx *context); }; +/* + * struct damon_call_control - Control damon_call(). + * + * @fn: Function to be called back. + * @data: Data that will be passed to @fn. + * @return_code: Return code from @fn invocation. + * + * Control damon_call(), which requests specific kdamond to invoke a given + * function. Refer to damon_call() for more details. + */ +struct damon_call_control { + int (*fn)(void *data); + void *data; + int return_code; +/* private: internal use only */ + /* informs if the kdamond finished handling of the request */ + struct completion completion; + /* informs if the kdamond canceled @fn infocation */ + bool canceled; +}; + /** * struct damon_attrs - Monitoring attributes for accuracy/overhead control. * @@ -670,6 +691,9 @@ struct damon_ctx { /* for scheme quotas prioritization */ unsigned long *regions_score_histogram; + struct damon_call_control *call_control; + struct mutex call_control_lock; + /* public: */ struct task_struct *kdamond; struct mutex kdamond_lock; @@ -817,6 +841,8 @@ static inline unsigned int damon_max_nr_accesses(const struct damon_attrs *attrs int damon_start(struct damon_ctx **ctxs, int nr_ctxs, bool exclusive); int damon_stop(struct damon_ctx **ctxs, int nr_ctxs); +int damon_call(struct damon_ctx *ctx, struct damon_call_control *control); + int damon_set_region_biggest_system_ram_default(struct damon_target *t, unsigned long *start, unsigned long *end); diff --git a/mm/damon/core.c b/mm/damon/core.c index bf04987a91c6..89a679c06e30 100644 --- a/mm/damon/core.c +++ b/mm/damon/core.c @@ -533,6 +533,7 @@ struct damon_ctx *damon_new_ctx(void) ctx->next_ops_update_sis = 0; mutex_init(&ctx->kdamond_lock); + mutex_init(&ctx->call_control_lock); ctx->attrs.min_nr_regions = 10; ctx->attrs.max_nr_regions = 1000; @@ -1183,6 +1184,54 @@ int damon_stop(struct damon_ctx **ctxs, int nr_ctxs) return err; } +static bool damon_is_running(struct damon_ctx *ctx) +{ + bool running; + + mutex_lock(&ctx->kdamond_lock); + running = ctx->kdamond != NULL; + mutex_unlock(&ctx->kdamond_lock); + return running; +} + +/** + * damon_call() - Invoke a given function on DAMON worker thread (kdamond). + * @ctx: DAMON context to call the function for. + * @control: Control variable of the call request. + * + * Ask DAMON worker thread (kdamond) of @ctx to call a function with an + * argument data that respectively passed via &damon_call_control->fn and + * &damon_call_control->data of @control, and wait until the kdamond finishes + * handling of the request. + * + * The kdamond executes the function with the argument in the main loop, just + * after a sampling of the iteration is finished. The function can hence + * safely access the internal data of the &struct damon_ctx without additional + * synchronization. The return value of the function will be saved in + * &damon_call_control->return_code. + * + * Return: 0 on success, negative error code otherwise. + */ +int damon_call(struct damon_ctx *ctx, struct damon_call_control *control) +{ + init_completion(&control->completion); + control->canceled = false; + + mutex_lock(&ctx->call_control_lock); + if (ctx->call_control) { + mutex_unlock(&ctx->call_control_lock); + return -EBUSY; + } + ctx->call_control = control; + mutex_unlock(&ctx->call_control_lock); + if (!damon_is_running(ctx)) + return -EINVAL; + wait_for_completion(&control->completion); + if (control->canceled) + return -ECANCELED; + return 0; +} + /* * Reset the aggregated monitoring results ('nr_accesses' of each region). */ @@ -1970,6 +2019,39 @@ static void kdamond_usleep(unsigned long usecs) usleep_range_idle(usecs, usecs + 1); } +/* + * kdamond_call() - handle damon_call_control. + * @ctx: The &struct damon_ctx of the kdamond. + * @cancel: Whether to cancel the invocation of the function. + * + * If there is a &struct damon_call_control request that registered via + * &damon_call() on @ctx, do or cancel the invocation of the function depending + * on @cancel. @cancel is set when the kdamond is deactivated by DAMOS + * watermarks, or the kdamond is already out of the main loop and therefore + * will be terminated. + */ +static void kdamond_call(struct damon_ctx *ctx, bool cancel) +{ + struct damon_call_control *control; + int ret = 0; + + mutex_lock(&ctx->call_control_lock); + control = ctx->call_control; + mutex_unlock(&ctx->call_control_lock); + if (!control) + return; + if (cancel) { + control->canceled = true; + } else { + ret = control->fn(control->data); + control->return_code = ret; + } + complete(&control->completion); + mutex_lock(&ctx->call_control_lock); + ctx->call_control = NULL; + mutex_unlock(&ctx->call_control_lock); +} + /* Returns negative error code if it's not activated but should return */ static int kdamond_wait_activation(struct damon_ctx *ctx) { @@ -1994,6 +2076,7 @@ static int kdamond_wait_activation(struct damon_ctx *ctx) if (ctx->callback.after_wmarks_check && ctx->callback.after_wmarks_check(ctx)) break; + kdamond_call(ctx, true); } return -EBUSY; } @@ -2065,6 +2148,7 @@ static int kdamond_fn(void *data) if (ctx->callback.after_sampling && ctx->callback.after_sampling(ctx)) break; + kdamond_call(ctx, false); kdamond_usleep(sample_interval); ctx->passed_sample_intervals++; @@ -2126,6 +2210,8 @@ static int kdamond_fn(void *data) ctx->kdamond = NULL; mutex_unlock(&ctx->kdamond_lock); + kdamond_call(ctx, true); + mutex_lock(&damon_lock); nr_running_ctxs--; if (!nr_running_ctxs && running_exclusive_ctxs)