From patchwork Thu Mar 18 10:08:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 12147791 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 927ECC433E0 for ; Thu, 18 Mar 2021 10:10:04 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 010D161580 for ; Thu, 18 Mar 2021 10:10:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 010D161580 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5AE8D6B0072; Thu, 18 Mar 2021 06:10:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 584C76B0073; Thu, 18 Mar 2021 06:10:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 38A986B0074; Thu, 18 Mar 2021 06:10:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 154AE6B0072 for ; Thu, 18 Mar 2021 06:10:03 -0400 (EDT) Received: from smtpin15.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id BCAA38249980 for ; Thu, 18 Mar 2021 10:10:02 +0000 (UTC) X-FDA: 77932574202.15.E5CEAFB Received: from mail-qv1-f53.google.com (mail-qv1-f53.google.com [209.85.219.53]) by imf07.hostedemail.com (Postfix) with ESMTP id 08AC1A0009EC for ; Thu, 18 Mar 2021 10:10:01 +0000 (UTC) Received: by mail-qv1-f53.google.com with SMTP id d10so2876803qve.7 for ; Thu, 18 Mar 2021 03:10:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=H7n6s7LW8bwtbqEUo8cPgMrkiWuu99SWLGdG8VaRQkw=; b=cA3RwQkGjwjsQ1qo2xKoNGzNRY/EmT5UQ0wCoukBAnJS634QXHjpEEeli98DJTWVap TrskxHxbBjKy11RBA3hu93Tjr+2AL8+ffoEq4cYAs3u/Q7nHxNAb7EtOjfWsYXILY0lH pJ8spbo6RMDCXsrU6JCEqske2R+GVGNEY36QID0FuTEcXlqUByiBGm/48niiMW/a2kAi 1Dd3Akdzbm0mZAlOriyGReF4Afgt6MvRqlJDlSrKw1FFydYKqWpV+y6Fv06MAF2ZEAgE FMlMpjhBVp9H2jFlZw8KY7+t2iB+9VudXCRkGbGUv7vcvPm8usONuUogTpDe/gkIb+wE U8FA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=H7n6s7LW8bwtbqEUo8cPgMrkiWuu99SWLGdG8VaRQkw=; b=GeYEbzNbxX7Aps/ulYO3Zf9hrOSCQzA7uBRIo8VWXXtwLUa0yfT7ExeAHlTXfyrphb Jz5uimq1u8rbFGGc72c8IPOsIbhUVhH03RZpR4CJoSy4Q4lpvBfMSo1AwheUMM2/JEfz c5wfPpgT0GBGudzmzKCCGg8SG4hsWXDxM7jnoOU238lB2IGMPf/YbgUE6Vgj3aGNnxrS 5ix7xZwVzPQbla+bC+vXtoqX9e2Ia6ac+vXp4WUrE6xQbEaTSraVeyTlMSwjZGuXCMyq tzt2XI0Qn1R5Z1BPI577c0gKFSyAE3eSwOaBNseBlwjbOJQQNZUCMgtix2dcT2SYzf1Y aGUw== X-Gm-Message-State: AOAM530MokJu6DnKlw/cSIFVPBfjVR9+CG7ZOFPiA66Fa25C79Mrtcn1 8dF75CMaCCC87e5jhy+XpN0= X-Google-Smtp-Source: ABdhPJx/pvfIIdFA8MUDyVgZmLopF50WSuwBDrcklwzp8j6zBv9E1JV6WfSBLIFVMdn6C5gJdZcA1w== X-Received: by 2002:a05:6214:242f:: with SMTP id gy15mr3465830qvb.17.1616062201199; Thu, 18 Mar 2021 03:10:01 -0700 (PDT) Received: from localhost.localdomain (ec2-35-169-212-159.compute-1.amazonaws.com. [35.169.212.159]) by smtp.gmail.com with ESMTPSA id d16sm1358101qka.43.2021.03.18.03.10.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 18 Mar 2021 03:10:00 -0700 (PDT) From: sj38.park@gmail.com To: akpm@linux-foundation.org Cc: SeongJae Park , Jonathan.Cameron@Huawei.com, acme@kernel.org, alexander.shishkin@linux.intel.com, amit@kernel.org, benh@kernel.crashing.org, brendanhiggins@google.com, corbet@lwn.net, david@redhat.com, dwmw@amazon.com, elver@google.com, fan.du@intel.com, foersleo@amazon.de, gthelen@google.com, mgorman@suse.de, minchan@kernel.org, mingo@redhat.com, namhyung@kernel.org, peterz@infradead.org, riel@surriel.com, rientjes@google.com, rostedt@goodmis.org, rppt@kernel.org, shakeelb@google.com, shuah@kernel.org, sj38.park@gmail.com, snu@amazon.de, vbabka@suse.cz, vdavydov.dev@gmail.com, zgf574564920@gmail.com, linux-damon@amazon.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v25 01/13] mm: Introduce Data Access MONitor (DAMON) Date: Thu, 18 Mar 2021 10:08:44 +0000 Message-Id: <20210318100856.34715-2-sj38.park@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210318100856.34715-1-sj38.park@gmail.com> References: <20210318100856.34715-1-sj38.park@gmail.com> X-Stat-Signature: gq568xjj68ogrrw1nank1wh8mn9e1x56 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 08AC1A0009EC Received-SPF: none (gmail.com>: No applicable sender policy available) receiver=imf07; identity=mailfrom; envelope-from=""; helo=mail-qv1-f53.google.com; client-ip=209.85.219.53 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1616062201-633284 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park DAMON is a data access monitoring framework for the Linux kernel. The core mechanisms of DAMON make it - accurate (the monitoring output is useful enough for DRAM level performance-centric memory management; It might be inappropriate for CPU cache levels, though), - light-weight (the monitoring overhead is normally low enough to be applied online), and - scalable (the upper-bound of the overhead is in constant range regardless of the size of target workloads). Using this framework, hence, we can easily write efficient kernel space data access monitoring applications. For example, the kernel's memory management mechanisms can make advanced decisions using this. Experimental data access aware optimization works that incurring high access monitoring overhead could again be implemented on top of this. Due to its simple and flexible interface, providing user space interface would be also easy. Then, user space users who have some special workloads can write personalized applications for better understanding and optimizations of their workloads and systems. === Nevertheless, this commit is defining and implementing only basic access check part without the overhead-accuracy handling core logic. The basic access check is as below. The output of DAMON says what memory regions are how frequently accessed for a given duration. The resolution of the access frequency is controlled by setting ``sampling interval`` and ``aggregation interval``. In detail, DAMON checks access to each page per ``sampling interval`` and aggregates the results. In other words, counts the number of the accesses to each region. After each ``aggregation interval`` passes, DAMON calls callback functions that previously registered by users so that users can read the aggregated results and then clears the results. This can be described in below simple pseudo-code:: init() while monitoring_on: for page in monitoring_target: if accessed(page): nr_accesses[page] += 1 if time() % aggregation_interval == 0: for callback in user_registered_callbacks: callback(monitoring_target, nr_accesses) for page in monitoring_target: nr_accesses[page] = 0 if time() % update_interval == 0: update() sleep(sampling interval) The target regions constructed at the beginning of the monitoring and updated after each ``regions_update_interval``, because the target regions could be dynamically changed (e.g., mmap() or memory hotplug). The monitoring overhead of this mechanism will arbitrarily increase as the size of the target workload grows. The basic monitoring primitives for actual access check and dynamic target regions construction aren't in the core part of DAMON. Instead, it allows users to implement their own primitives that are optimized for their use case and configure DAMON to use those. In other words, users cannot use current version of DAMON without some additional works. Following commits will implement the core mechanisms for the overhead-accuracy control and default primitives implementations. Signed-off-by: SeongJae Park Reviewed-by: Leonard Foerster --- include/linux/damon.h | 167 ++++++++++++++++++++++ mm/Kconfig | 2 + mm/Makefile | 1 + mm/damon/Kconfig | 15 ++ mm/damon/Makefile | 3 + mm/damon/core.c | 318 ++++++++++++++++++++++++++++++++++++++++++ 6 files changed, 506 insertions(+) create mode 100644 include/linux/damon.h create mode 100644 mm/damon/Kconfig create mode 100644 mm/damon/Makefile create mode 100644 mm/damon/core.c diff --git a/include/linux/damon.h b/include/linux/damon.h new file mode 100644 index 000000000000..2f652602b1ea --- /dev/null +++ b/include/linux/damon.h @@ -0,0 +1,167 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * DAMON api + * + * Author: SeongJae Park + */ + +#ifndef _DAMON_H_ +#define _DAMON_H_ + +#include +#include +#include + +struct damon_ctx; + +/** + * struct damon_primitive Monitoring primitives for given use cases. + * + * @init: Initialize primitive-internal data structures. + * @update: Update primitive-internal data structures. + * @prepare_access_checks: Prepare next access check of target regions. + * @check_accesses: Check the accesses to target regions. + * @reset_aggregated: Reset aggregated accesses monitoring results. + * @target_valid: Determine if the target is valid. + * @cleanup: Clean up the context. + * + * DAMON can be extended for various address spaces and usages. For this, + * users should register the low level primitives for their target address + * space and usecase via the &damon_ctx.primitive. Then, the monitoring thread + * (&damon_ctx.kdamond) calls @init and @prepare_access_checks before starting + * the monitoring, @update after each &damon_ctx.primitive_update_interval, and + * @check_accesses, @target_valid and @prepare_access_checks after each + * &damon_ctx.sample_interval. Finally, @reset_aggregated is called after each + * &damon_ctx.aggr_interval. + * + * @init should initialize primitive-internal data structures. For example, + * this could be used to construct proper monitoring target regions and link + * those to @damon_ctx.target. + * @update should update the primitive-internal data structures. For example, + * this could be used to update monitoring target regions for current status. + * @prepare_access_checks should manipulate the monitoring regions to be + * prepared for the next access check. + * @check_accesses should check the accesses to each region that made after the + * last preparation and update the number of observed accesses of each region. + * @reset_aggregated should reset the access monitoring results that aggregated + * by @check_accesses. + * @target_valid should check whether the target is still valid for the + * monitoring. + * @cleanup is called from @kdamond just before its termination. + */ +struct damon_primitive { + void (*init)(struct damon_ctx *context); + void (*update)(struct damon_ctx *context); + void (*prepare_access_checks)(struct damon_ctx *context); + void (*check_accesses)(struct damon_ctx *context); + void (*reset_aggregated)(struct damon_ctx *context); + bool (*target_valid)(void *target); + void (*cleanup)(struct damon_ctx *context); +}; + +/* + * struct damon_callback Monitoring events notification callbacks. + * + * @before_start: Called before starting the monitoring. + * @after_sampling: Called after each sampling. + * @after_aggregation: Called after each aggregation. + * @before_terminate: Called before terminating the monitoring. + * @private: User private data. + * + * The monitoring thread (&damon_ctx.kdamond) calls @before_start and + * @before_terminate just before starting and finishing the monitoring, + * respectively. Therefore, those are good places for installing and cleaning + * @private. + * + * The monitoring thread calls @after_sampling and @after_aggregation for each + * of the sampling intervals and aggregation intervals, respectively. + * Therefore, users can safely access the monitoring results without additional + * protection. For the reason, users are recommended to use these callback for + * the accesses to the results. + * + * If any callback returns non-zero, monitoring stops. + */ +struct damon_callback { + void *private; + + int (*before_start)(struct damon_ctx *context); + int (*after_sampling)(struct damon_ctx *context); + int (*after_aggregation)(struct damon_ctx *context); + int (*before_terminate)(struct damon_ctx *context); +}; + +/** + * struct damon_ctx - Represents a context for each monitoring. This is the + * main interface that allows users to set the attributes and get the results + * of the monitoring. + * + * @sample_interval: The time between access samplings. + * @aggr_interval: The time between monitor results aggregations. + * @primitive_update_interval: The time between monitoring primitive updates. + * + * For each @sample_interval, DAMON checks whether each region is accessed or + * not. It aggregates and keeps the access information (number of accesses to + * each region) for @aggr_interval time. DAMON also checks whether the target + * memory regions need update (e.g., by ``mmap()`` calls from the application, + * in case of virtual memory monitoring) and applies the changes for each + * @primitive_update_interval. All time intervals are in micro-seconds. + * Please refer to &struct damon_primitive and &struct damon_callback for more + * detail. + * + * @kdamond: Kernel thread who does the monitoring. + * @kdamond_stop: Notifies whether kdamond should stop. + * @kdamond_lock: Mutex for the synchronizations with @kdamond. + * + * For each monitoring context, one kernel thread for the monitoring is + * created. The pointer to the thread is stored in @kdamond. + * + * Once started, the monitoring thread runs until explicitly required to be + * terminated or every monitoring target is invalid. The validity of the + * targets is checked via the &damon_primitive.target_valid of @primitive. The + * termination can also be explicitly requested by writing non-zero to + * @kdamond_stop. The thread sets @kdamond to NULL when it terminates. + * Therefore, users can know whether the monitoring is ongoing or terminated by + * reading @kdamond. Reads and writes to @kdamond and @kdamond_stop from + * outside of the monitoring thread must be protected by @kdamond_lock. + * + * Note that the monitoring thread protects only @kdamond and @kdamond_stop via + * @kdamond_lock. Accesses to other fields must be protected by themselves. + * + * @primitive: Set of monitoring primitives for given use cases. + * @callback: Set of callbacks for monitoring events notifications. + * + * @target: Pointer to the user-defined monitoring target. + */ +struct damon_ctx { + unsigned long sample_interval; + unsigned long aggr_interval; + unsigned long primitive_update_interval; + +/* private: internal use only */ + struct timespec64 last_aggregation; + struct timespec64 last_primitive_update; + +/* public: */ + struct task_struct *kdamond; + bool kdamond_stop; + struct mutex kdamond_lock; + + struct damon_primitive primitive; + struct damon_callback callback; + + void *target; +}; + +#ifdef CONFIG_DAMON + +struct damon_ctx *damon_new_ctx(void); +void damon_destroy_ctx(struct damon_ctx *ctx); +int damon_set_attrs(struct damon_ctx *ctx, unsigned long sample_int, + unsigned long aggr_int, unsigned long primitive_upd_int); + +int damon_start(struct damon_ctx **ctxs, int nr_ctxs); +int damon_stop(struct damon_ctx **ctxs, int nr_ctxs); + +#endif /* CONFIG_DAMON */ + +#endif /* _DAMON_H */ diff --git a/mm/Kconfig b/mm/Kconfig index 3823a2314256..fa8c95a54346 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -893,4 +893,6 @@ config KMAP_LOCAL config SECRETMEM def_bool ARCH_HAS_SET_DIRECT_MAP && !EMBEDDED +source "mm/damon/Kconfig" + endmenu diff --git a/mm/Makefile b/mm/Makefile index 788c5ce5c0ef..baef68fd2e2c 100644 --- a/mm/Makefile +++ b/mm/Makefile @@ -125,3 +125,4 @@ obj-$(CONFIG_MAPPING_DIRTY_HELPERS) += mapping_dirty_helpers.o obj-$(CONFIG_PTDUMP_CORE) += ptdump.o obj-$(CONFIG_PAGE_REPORTING) += page_reporting.o obj-$(CONFIG_SECRETMEM) += secretmem.o +obj-$(CONFIG_DAMON) += damon/ diff --git a/mm/damon/Kconfig b/mm/damon/Kconfig new file mode 100644 index 000000000000..d00e99ac1a15 --- /dev/null +++ b/mm/damon/Kconfig @@ -0,0 +1,15 @@ +# SPDX-License-Identifier: GPL-2.0-only + +menu "Data Access Monitoring" + +config DAMON + bool "DAMON: Data Access Monitoring Framework" + help + This builds a framework that allows kernel subsystems to monitor + access frequency of each memory region. The information can be useful + for performance-centric DRAM level memory management. + + See https://damonitor.github.io/doc/html/latest-damon/index.html for + more information. + +endmenu diff --git a/mm/damon/Makefile b/mm/damon/Makefile new file mode 100644 index 000000000000..4fd2edb4becf --- /dev/null +++ b/mm/damon/Makefile @@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0 + +obj-$(CONFIG_DAMON) := core.o diff --git a/mm/damon/core.c b/mm/damon/core.c new file mode 100644 index 000000000000..693e51ebc05a --- /dev/null +++ b/mm/damon/core.c @@ -0,0 +1,318 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Data Access Monitor + * + * Author: SeongJae Park + */ + +#define pr_fmt(fmt) "damon: " fmt + +#include +#include +#include +#include + +static DEFINE_MUTEX(damon_lock); +static int nr_running_ctxs; + +struct damon_ctx *damon_new_ctx(void) +{ + struct damon_ctx *ctx; + + ctx = kzalloc(sizeof(*ctx), GFP_KERNEL); + if (!ctx) + return NULL; + + ctx->sample_interval = 5 * 1000; + ctx->aggr_interval = 100 * 1000; + ctx->primitive_update_interval = 1000 * 1000; + + ktime_get_coarse_ts64(&ctx->last_aggregation); + ctx->last_primitive_update = ctx->last_aggregation; + + mutex_init(&ctx->kdamond_lock); + + ctx->target = NULL; + + return ctx; +} + +void damon_destroy_ctx(struct damon_ctx *ctx) +{ + if (ctx->primitive.cleanup) + ctx->primitive.cleanup(ctx); + kfree(ctx); +} + +/** + * damon_set_attrs() - Set attributes for the monitoring. + * @ctx: monitoring context + * @sample_int: time interval between samplings + * @aggr_int: time interval between aggregations + * @primitive_upd_int: time interval between monitoring primitive updates + * + * This function should not be called while the kdamond is running. + * Every time interval is in micro-seconds. + * + * Return: 0 on success, negative error code otherwise. + */ +int damon_set_attrs(struct damon_ctx *ctx, unsigned long sample_int, + unsigned long aggr_int, unsigned long primitive_upd_int) +{ + ctx->sample_interval = sample_int; + ctx->aggr_interval = aggr_int; + ctx->primitive_update_interval = primitive_upd_int; + + return 0; +} + +static bool damon_kdamond_running(struct damon_ctx *ctx) +{ + bool running; + + mutex_lock(&ctx->kdamond_lock); + running = ctx->kdamond != NULL; + mutex_unlock(&ctx->kdamond_lock); + + return running; +} + +static int kdamond_fn(void *data); + +/* + * __damon_start() - Starts monitoring with given context. + * @ctx: monitoring context + * + * This function should be called while damon_lock is hold. + * + * Return: 0 on success, negative error code otherwise. + */ +static int __damon_start(struct damon_ctx *ctx) +{ + int err = -EBUSY; + + mutex_lock(&ctx->kdamond_lock); + if (!ctx->kdamond) { + err = 0; + ctx->kdamond_stop = false; + ctx->kdamond = kthread_create(kdamond_fn, ctx, "kdamond.%d", + nr_running_ctxs); + if (IS_ERR(ctx->kdamond)) + err = PTR_ERR(ctx->kdamond); + else + wake_up_process(ctx->kdamond); + } + mutex_unlock(&ctx->kdamond_lock); + + return err; +} + +/** + * damon_start() - Starts the monitorings for a given group of contexts. + * @ctxs: an array of the pointers for contexts to start monitoring + * @nr_ctxs: size of @ctxs + * + * This function starts a group of monitoring threads for a group of monitoring + * contexts. One thread per each context is created and run in parallel. The + * caller should handle synchronization between the threads by itself. If a + * group of threads that created by other 'damon_start()' call is currently + * running, this function does nothing but returns -EBUSY. + * + * Return: 0 on success, negative error code otherwise. + */ +int damon_start(struct damon_ctx **ctxs, int nr_ctxs) +{ + int i; + int err = 0; + + mutex_lock(&damon_lock); + if (nr_running_ctxs) { + mutex_unlock(&damon_lock); + return -EBUSY; + } + + for (i = 0; i < nr_ctxs; i++) { + err = __damon_start(ctxs[i]); + if (err) + break; + nr_running_ctxs++; + } + mutex_unlock(&damon_lock); + + return err; +} + +/* + * __damon_stop() - Stops monitoring of given context. + * @ctx: monitoring context + * + * Return: 0 on success, negative error code otherwise. + */ +static int __damon_stop(struct damon_ctx *ctx) +{ + mutex_lock(&ctx->kdamond_lock); + if (ctx->kdamond) { + ctx->kdamond_stop = true; + mutex_unlock(&ctx->kdamond_lock); + while (damon_kdamond_running(ctx)) + usleep_range(ctx->sample_interval, + ctx->sample_interval * 2); + return 0; + } + mutex_unlock(&ctx->kdamond_lock); + + return -EPERM; +} + +/** + * damon_stop() - Stops the monitorings for a given group of contexts. + * @ctxs: an array of the pointers for contexts to stop monitoring + * @nr_ctxs: size of @ctxs + * + * Return: 0 on success, negative error code otherwise. + */ +int damon_stop(struct damon_ctx **ctxs, int nr_ctxs) +{ + int i, err = 0; + + for (i = 0; i < nr_ctxs; i++) { + /* nr_running_ctxs is decremented in kdamond_fn */ + err = __damon_stop(ctxs[i]); + if (err) + return err; + } + + return err; +} + +/* + * damon_check_reset_time_interval() - Check if a time interval is elapsed. + * @baseline: the time to check whether the interval has elapsed since + * @interval: the time interval (microseconds) + * + * See whether the given time interval has passed since the given baseline + * time. If so, it also updates the baseline to current time for next check. + * + * Return: true if the time interval has passed, or false otherwise. + */ +static bool damon_check_reset_time_interval(struct timespec64 *baseline, + unsigned long interval) +{ + struct timespec64 now; + + ktime_get_coarse_ts64(&now); + if ((timespec64_to_ns(&now) - timespec64_to_ns(baseline)) < + interval * 1000) + return false; + *baseline = now; + return true; +} + +/* + * Check whether it is time to flush the aggregated information + */ +static bool kdamond_aggregate_interval_passed(struct damon_ctx *ctx) +{ + return damon_check_reset_time_interval(&ctx->last_aggregation, + ctx->aggr_interval); +} + +/* + * Check whether it is time to check and apply the target monitoring regions + * + * Returns true if it is. + */ +static bool kdamond_need_update_primitive(struct damon_ctx *ctx) +{ + return damon_check_reset_time_interval(&ctx->last_primitive_update, + ctx->primitive_update_interval); +} + +/* + * Check whether current monitoring should be stopped + * + * The monitoring is stopped when either the user requested to stop, or all + * monitoring targets are invalid. + * + * Returns true if need to stop current monitoring. + */ +static bool kdamond_need_stop(struct damon_ctx *ctx) +{ + bool stop; + + mutex_lock(&ctx->kdamond_lock); + stop = ctx->kdamond_stop; + mutex_unlock(&ctx->kdamond_lock); + if (stop) + return true; + + if (!ctx->primitive.target_valid) + return false; + + return !ctx->primitive.target_valid(ctx->target); +} + +static void set_kdamond_stop(struct damon_ctx *ctx) +{ + mutex_lock(&ctx->kdamond_lock); + ctx->kdamond_stop = true; + mutex_unlock(&ctx->kdamond_lock); +} + +/* + * The monitoring daemon that runs as a kernel thread + */ +static int kdamond_fn(void *data) +{ + struct damon_ctx *ctx = (struct damon_ctx *)data; + + pr_info("kdamond (%d) starts\n", ctx->kdamond->pid); + + if (ctx->primitive.init) + ctx->primitive.init(ctx); + if (ctx->callback.before_start && ctx->callback.before_start(ctx)) + set_kdamond_stop(ctx); + + while (!kdamond_need_stop(ctx)) { + if (ctx->primitive.prepare_access_checks) + ctx->primitive.prepare_access_checks(ctx); + if (ctx->callback.after_sampling && + ctx->callback.after_sampling(ctx)) + set_kdamond_stop(ctx); + + usleep_range(ctx->sample_interval, ctx->sample_interval + 1); + + if (ctx->primitive.check_accesses) + ctx->primitive.check_accesses(ctx); + + if (kdamond_aggregate_interval_passed(ctx)) { + if (ctx->callback.after_aggregation && + ctx->callback.after_aggregation(ctx)) + set_kdamond_stop(ctx); + if (ctx->primitive.reset_aggregated) + ctx->primitive.reset_aggregated(ctx); + } + + if (kdamond_need_update_primitive(ctx)) { + if (ctx->primitive.update) + ctx->primitive.update(ctx); + } + } + + if (ctx->callback.before_terminate && + ctx->callback.before_terminate(ctx)) + set_kdamond_stop(ctx); + if (ctx->primitive.cleanup) + ctx->primitive.cleanup(ctx); + + pr_debug("kdamond (%d) finishes\n", ctx->kdamond->pid); + mutex_lock(&ctx->kdamond_lock); + ctx->kdamond = NULL; + mutex_unlock(&ctx->kdamond_lock); + + mutex_lock(&damon_lock); + nr_running_ctxs--; + mutex_unlock(&damon_lock); + + do_exit(0); +} From patchwork Thu Mar 18 10:08:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 12147793 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6ABF3C433E6 for ; Thu, 18 Mar 2021 10:10:07 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D881164E86 for ; Thu, 18 Mar 2021 10:10:06 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D881164E86 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6C0046B0073; Thu, 18 Mar 2021 06:10:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6977D6B0074; Thu, 18 Mar 2021 06:10:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4E99B6B0075; Thu, 18 Mar 2021 06:10:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0070.hostedemail.com [216.40.44.70]) by kanga.kvack.org (Postfix) with ESMTP id 31B606B0073 for ; Thu, 18 Mar 2021 06:10:06 -0400 (EDT) Received: from smtpin37.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id DF29C8789 for ; Thu, 18 Mar 2021 10:10:05 +0000 (UTC) X-FDA: 77932574370.37.B123452 Received: from mail-qt1-f173.google.com (mail-qt1-f173.google.com [209.85.160.173]) by imf18.hostedemail.com (Postfix) with ESMTP id 4E0022000241 for ; Thu, 18 Mar 2021 10:10:05 +0000 (UTC) Received: by mail-qt1-f173.google.com with SMTP id c6so3624422qtc.1 for ; Thu, 18 Mar 2021 03:10:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=kPY0GkE9S3FRLIHJStl59h2p5t9zcOUJ2P3qtKum3yg=; b=uifa1zNjdbvpbTXmZcIKvBHVGeyYEFADJwyoHeuQNZqV1kDmwB0V91RV/2VREZyGbs qLG3vDdd4IOI5UVQABqKsyeSAuHn4HPkV5FtPQJ1ve0R/B16S+XfSsD34iknZIFNQrW1 0SQDULowYjwAhVQXV63/5Ow/0BZyvHBAyYX87iUuITvDLE6t6WNTqnvd2Z0WGJwFgOKY b6CYcghHnNyNSxZncImChF8TE6sXeXczWJmW/KEegOWScQaotwVbaFn/bbv+UgVmtljZ Rc4ZZbsRtd2x8M1H4Bu0hmUYIiLtKOdLVyy2SP/hkNr5pt/H798mzqpJ2xZ9YgeQoRu4 zNzA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=kPY0GkE9S3FRLIHJStl59h2p5t9zcOUJ2P3qtKum3yg=; b=U1KeE341YSxqm944/aJh+QxKuwaustDKUBpPqxp0//M/tb0gp50pKLZ5NvKjdtYbtH Tj5KynfBeTlHWzWeuJQ5k3e2ToYC8cLvTNAj9CuY0S0TnYEy94fw7zc/GxfDlhHDqWgS UIzO0J4Us/T7EWC7uGRckB3eaw6qM02wqXztLI2IinfhR/EfRG6eNFEYRABy/8C629im AroxNcJGx+A+7kZ8zVz2vkQo6VJqeWJIwHCoGZ3OkqaDRjaRV4PH0mgLArvfbJGhK5b3 pGtYrKZmo25G89wdi2E8C9el7gw6KVor07ROqYTyfVsLm93zZCwC0d57F0FDsRwiChJQ QeGg== X-Gm-Message-State: AOAM532JuTh+4t4feoWPkD8/2WA3KlF/+O+orerDSSQv4OgX6tT3HwFT C1/9S9bhmrdrhHuu02qOPzs= X-Google-Smtp-Source: ABdhPJx9BuUeb9sqCnoNK/EsEHuSf53fJCziFiNq4nAQvp5P9LSCCsjFW4GiOQc6OhsWiK4VFctTkA== X-Received: by 2002:ac8:53c3:: with SMTP id c3mr3056797qtq.118.1616062204665; Thu, 18 Mar 2021 03:10:04 -0700 (PDT) Received: from localhost.localdomain (ec2-35-169-212-159.compute-1.amazonaws.com. [35.169.212.159]) by smtp.gmail.com with ESMTPSA id d16sm1358101qka.43.2021.03.18.03.10.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 18 Mar 2021 03:10:04 -0700 (PDT) From: sj38.park@gmail.com To: akpm@linux-foundation.org Cc: SeongJae Park , Jonathan.Cameron@Huawei.com, acme@kernel.org, alexander.shishkin@linux.intel.com, amit@kernel.org, benh@kernel.crashing.org, brendanhiggins@google.com, corbet@lwn.net, david@redhat.com, dwmw@amazon.com, elver@google.com, fan.du@intel.com, foersleo@amazon.de, gthelen@google.com, mgorman@suse.de, minchan@kernel.org, mingo@redhat.com, namhyung@kernel.org, peterz@infradead.org, riel@surriel.com, rientjes@google.com, rostedt@goodmis.org, rppt@kernel.org, shakeelb@google.com, shuah@kernel.org, sj38.park@gmail.com, snu@amazon.de, vbabka@suse.cz, vdavydov.dev@gmail.com, zgf574564920@gmail.com, linux-damon@amazon.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v25 02/13] mm/damon/core: Implement region-based sampling Date: Thu, 18 Mar 2021 10:08:45 +0000 Message-Id: <20210318100856.34715-3-sj38.park@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210318100856.34715-1-sj38.park@gmail.com> References: <20210318100856.34715-1-sj38.park@gmail.com> X-Stat-Signature: weippn6ejuxkhu88ofitikc19isfu9nx X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 4E0022000241 Received-SPF: none (gmail.com>: No applicable sender policy available) receiver=imf18; identity=mailfrom; envelope-from=""; helo=mail-qt1-f173.google.com; client-ip=209.85.160.173 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1616062205-598291 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park To avoid the unbounded increase of the overhead, DAMON groups adjacent pages that are assumed to have the same access frequencies into a region. As long as the assumption (pages in a region have the same access frequencies) is kept, only one page in the region is required to be checked. Thus, for each ``sampling interval``, 1. the 'prepare_access_checks' primitive picks one page in each region, 2. waits for one ``sampling interval``, 3. checks whether the page is accessed meanwhile, and 4. increases the access count of the region if so. Therefore, the monitoring overhead is controllable by adjusting the number of regions. DAMON allows both the underlying primitives and user callbacks to adjust regions for the trade-off. In other words, this commit makes DAMON to use not only time-based sampling but also space-based sampling. This scheme, however, cannot preserve the quality of the output if the assumption is not guaranteed. Next commit will address this problem. Signed-off-by: SeongJae Park Reviewed-by: Leonard Foerster --- include/linux/damon.h | 77 ++++++++++++++++++++++- mm/damon/core.c | 143 ++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 213 insertions(+), 7 deletions(-) diff --git a/include/linux/damon.h b/include/linux/damon.h index 2f652602b1ea..67db309ad61b 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -12,6 +12,48 @@ #include #include +/** + * struct damon_addr_range - Represents an address region of [@start, @end). + * @start: Start address of the region (inclusive). + * @end: End address of the region (exclusive). + */ +struct damon_addr_range { + unsigned long start; + unsigned long end; +}; + +/** + * struct damon_region - Represents a monitoring target region. + * @ar: The address range of the region. + * @sampling_addr: Address of the sample for the next access check. + * @nr_accesses: Access frequency of this region. + * @list: List head for siblings. + */ +struct damon_region { + struct damon_addr_range ar; + unsigned long sampling_addr; + unsigned int nr_accesses; + struct list_head list; +}; + +/** + * struct damon_target - Represents a monitoring target. + * @id: Unique identifier for this target. + * @regions_list: Head of the monitoring target regions of this target. + * @list: List head for siblings. + * + * Each monitoring context could have multiple targets. For example, a context + * for virtual memory address spaces could have multiple target processes. The + * @id of each target should be unique among the targets of the context. For + * example, in the virtual address monitoring context, it could be a pidfd or + * an address of an mm_struct. + */ +struct damon_target { + unsigned long id; + struct list_head regions_list; + struct list_head list; +}; + struct damon_ctx; /** @@ -36,7 +78,7 @@ struct damon_ctx; * * @init should initialize primitive-internal data structures. For example, * this could be used to construct proper monitoring target regions and link - * those to @damon_ctx.target. + * those to @damon_ctx.adaptive_targets. * @update should update the primitive-internal data structures. For example, * this could be used to update monitoring target regions for current status. * @prepare_access_checks should manipulate the monitoring regions to be @@ -130,7 +172,7 @@ struct damon_callback { * @primitive: Set of monitoring primitives for given use cases. * @callback: Set of callbacks for monitoring events notifications. * - * @target: Pointer to the user-defined monitoring target. + * @region_targets: Head of monitoring targets (&damon_target) list. */ struct damon_ctx { unsigned long sample_interval; @@ -149,11 +191,40 @@ struct damon_ctx { struct damon_primitive primitive; struct damon_callback callback; - void *target; + struct list_head region_targets; }; +#define damon_next_region(r) \ + (container_of(r->list.next, struct damon_region, list)) + +#define damon_prev_region(r) \ + (container_of(r->list.prev, struct damon_region, list)) + +#define damon_for_each_region(r, t) \ + list_for_each_entry(r, &t->regions_list, list) + +#define damon_for_each_region_safe(r, next, t) \ + list_for_each_entry_safe(r, next, &t->regions_list, list) + +#define damon_for_each_target(t, ctx) \ + list_for_each_entry(t, &(ctx)->region_targets, list) + +#define damon_for_each_target_safe(t, next, ctx) \ + list_for_each_entry_safe(t, next, &(ctx)->region_targets, list) + #ifdef CONFIG_DAMON +struct damon_region *damon_new_region(unsigned long start, unsigned long end); +inline void damon_insert_region(struct damon_region *r, + struct damon_region *prev, struct damon_region *next); +void damon_add_region(struct damon_region *r, struct damon_target *t); +void damon_destroy_region(struct damon_region *r); + +struct damon_target *damon_new_target(unsigned long id); +void damon_add_target(struct damon_ctx *ctx, struct damon_target *t); +void damon_free_target(struct damon_target *t); +void damon_destroy_target(struct damon_target *t); + struct damon_ctx *damon_new_ctx(void); void damon_destroy_ctx(struct damon_ctx *ctx); int damon_set_attrs(struct damon_ctx *ctx, unsigned long sample_int, diff --git a/mm/damon/core.c b/mm/damon/core.c index 693e51ebc05a..94db494dcf70 100644 --- a/mm/damon/core.c +++ b/mm/damon/core.c @@ -15,6 +15,101 @@ static DEFINE_MUTEX(damon_lock); static int nr_running_ctxs; +/* + * Construct a damon_region struct + * + * Returns the pointer to the new struct if success, or NULL otherwise + */ +struct damon_region *damon_new_region(unsigned long start, unsigned long end) +{ + struct damon_region *region; + + region = kmalloc(sizeof(*region), GFP_KERNEL); + if (!region) + return NULL; + + region->ar.start = start; + region->ar.end = end; + region->nr_accesses = 0; + INIT_LIST_HEAD(®ion->list); + + return region; +} + +/* + * Add a region between two other regions + */ +inline void damon_insert_region(struct damon_region *r, + struct damon_region *prev, struct damon_region *next) +{ + __list_add(&r->list, &prev->list, &next->list); +} + +void damon_add_region(struct damon_region *r, struct damon_target *t) +{ + list_add_tail(&r->list, &t->regions_list); +} + +static void damon_del_region(struct damon_region *r) +{ + list_del(&r->list); +} + +static void damon_free_region(struct damon_region *r) +{ + kfree(r); +} + +void damon_destroy_region(struct damon_region *r) +{ + damon_del_region(r); + damon_free_region(r); +} + +/* + * Construct a damon_target struct + * + * Returns the pointer to the new struct if success, or NULL otherwise + */ +struct damon_target *damon_new_target(unsigned long id) +{ + struct damon_target *t; + + t = kmalloc(sizeof(*t), GFP_KERNEL); + if (!t) + return NULL; + + t->id = id; + INIT_LIST_HEAD(&t->regions_list); + + return t; +} + +void damon_add_target(struct damon_ctx *ctx, struct damon_target *t) +{ + list_add_tail(&t->list, &ctx->region_targets); +} + +static void damon_del_target(struct damon_target *t) +{ + list_del(&t->list); +} + +void damon_free_target(struct damon_target *t) +{ + struct damon_region *r, *next; + + damon_for_each_region_safe(r, next, t) + damon_free_region(r); + kfree(t); +} + +void damon_destroy_target(struct damon_target *t) +{ + damon_del_target(t); + damon_free_target(t); +} + struct damon_ctx *damon_new_ctx(void) { struct damon_ctx *ctx; @@ -32,15 +127,27 @@ struct damon_ctx *damon_new_ctx(void) mutex_init(&ctx->kdamond_lock); - ctx->target = NULL; + INIT_LIST_HEAD(&ctx->region_targets); return ctx; } -void damon_destroy_ctx(struct damon_ctx *ctx) +static void damon_destroy_targets(struct damon_ctx *ctx) { - if (ctx->primitive.cleanup) + struct damon_target *t, *next_t; + + if (ctx->primitive.cleanup) { ctx->primitive.cleanup(ctx); + return; + } + + damon_for_each_target_safe(t, next_t, ctx) + damon_destroy_target(t); +} + +void damon_destroy_ctx(struct damon_ctx *ctx) +{ + damon_destroy_targets(ctx); kfree(ctx); } @@ -217,6 +324,21 @@ static bool kdamond_aggregate_interval_passed(struct damon_ctx *ctx) ctx->aggr_interval); } +/* + * Reset the aggregated monitoring results ('nr_accesses' of each region). + */ +static void kdamond_reset_aggregated(struct damon_ctx *c) +{ + struct damon_target *t; + + damon_for_each_target(t, c) { + struct damon_region *r; + + damon_for_each_region(r, t) + r->nr_accesses = 0; + } +} + /* * Check whether it is time to check and apply the target monitoring regions * @@ -238,6 +360,7 @@ static bool kdamond_need_update_primitive(struct damon_ctx *ctx) */ static bool kdamond_need_stop(struct damon_ctx *ctx) { + struct damon_target *t; bool stop; mutex_lock(&ctx->kdamond_lock); @@ -249,7 +372,12 @@ static bool kdamond_need_stop(struct damon_ctx *ctx) if (!ctx->primitive.target_valid) return false; - return !ctx->primitive.target_valid(ctx->target); + damon_for_each_target(t, ctx) { + if (ctx->primitive.target_valid(t)) + return false; + } + + return true; } static void set_kdamond_stop(struct damon_ctx *ctx) @@ -265,6 +393,8 @@ static void set_kdamond_stop(struct damon_ctx *ctx) static int kdamond_fn(void *data) { struct damon_ctx *ctx = (struct damon_ctx *)data; + struct damon_target *t; + struct damon_region *r, *next; pr_info("kdamond (%d) starts\n", ctx->kdamond->pid); @@ -289,6 +419,7 @@ static int kdamond_fn(void *data) if (ctx->callback.after_aggregation && ctx->callback.after_aggregation(ctx)) set_kdamond_stop(ctx); + kdamond_reset_aggregated(ctx); if (ctx->primitive.reset_aggregated) ctx->primitive.reset_aggregated(ctx); } @@ -298,6 +429,10 @@ static int kdamond_fn(void *data) ctx->primitive.update(ctx); } } + damon_for_each_target(t, ctx) { + damon_for_each_region_safe(r, next, t) + damon_destroy_region(r); + } if (ctx->callback.before_terminate && ctx->callback.before_terminate(ctx)) From patchwork Thu Mar 18 10:08:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 12147795 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B20F9C433E0 for ; Thu, 18 Mar 2021 10:10:09 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2E4E764F40 for ; Thu, 18 Mar 2021 10:10:09 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2E4E764F40 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id BEFD16B0070; Thu, 18 Mar 2021 06:10:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BA0376B0074; Thu, 18 Mar 2021 06:10:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 959AE6B0075; Thu, 18 Mar 2021 06:10:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0061.hostedemail.com [216.40.44.61]) by kanga.kvack.org (Postfix) with ESMTP id 74B7F6B0070 for ; Thu, 18 Mar 2021 06:10:08 -0400 (EDT) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 2C9D0181AF5F7 for ; Thu, 18 Mar 2021 10:10:08 +0000 (UTC) X-FDA: 77932574496.05.A78F9A3 Received: from mail-qt1-f177.google.com (mail-qt1-f177.google.com [209.85.160.177]) by imf28.hostedemail.com (Postfix) with ESMTP id 8C9202000259 for ; Thu, 18 Mar 2021 10:10:07 +0000 (UTC) Received: by mail-qt1-f177.google.com with SMTP id u7so3590576qtq.12 for ; Thu, 18 Mar 2021 03:10:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=b+1Vnr8tb6LbM0SWY6J3IITj0EjR65jYOsFk4fahvOw=; b=PLPFyiEJkJZ7Rxv3qU5Ayb4zEEemc1K6MJBkLg++q6/n+V4syL97mamU4VSISRGAiu LfA/7pNzHEyAF8oPhni8ugRl10y3l+qxkZ065LM1nJSZeV+ObraCsN2ezURua9ulROST HaeF5uMpScMCeWA33r80DubP8ZFbj5gfFDRno4s/Klt7LF6DmniNNUV9PuXUjEEUztvu k1MkSCsnQLbDP8Wz3QUdIqx+Ect1PRKUePCpRsYQSzeY5IBkPaVMoeHN0vcHenliF0WO A8BY9+NE5XbAY3Z6VDPMoeUEmRT2fv2cTyg0xKhj2X2+1vhaXmg5PIhnIOEHMAdLFxqX pD0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=b+1Vnr8tb6LbM0SWY6J3IITj0EjR65jYOsFk4fahvOw=; b=TH4rMr0Yho2vIi2dtdJaOeOIuV/JcjnOjGMmkwqipS5WchVQA98Moy+sxUKjnc65w5 dsY56rDGRQ43omS3pZ3Tl5ClRqHCUH3otea9hmyMapfTS1G/H5rSUhsg/HIxptZXEvMs I9jT87IMHxCboggZ+x2w/qksiLl3KL0sT3r6lPzK5haXMvzhNUtaNTscN2xIs8eY7bQm UxR6nTuRERBhA1Tg9rpeZaVrycERKqd4JzouDlZ9+m6lfMk49TBz0ycBmmDn0TNLKdsf dbZQNYpwBOd0rEps6pTGEQDUGPM8GtflkkkOoxy/EEiGSWkLSE8+HVKgFdyWlTqXtsNG gWEA== X-Gm-Message-State: AOAM53394OnBTkk+27LlsW7HFYrX/ZqWLM0EAb9LOGtf70uAkWIwUYM/ fSmCMTlL2VOdwmT7v16mjOc= X-Google-Smtp-Source: ABdhPJxQPqiQ0ez72BN/OPtodE0O7Qb1AjwbKiFCi0+S/Aq/sldOShTumzfWEOto8cTi/scpsCeKwQ== X-Received: by 2002:ac8:5bd1:: with SMTP id b17mr2964902qtb.53.1616062206866; Thu, 18 Mar 2021 03:10:06 -0700 (PDT) Received: from localhost.localdomain (ec2-35-169-212-159.compute-1.amazonaws.com. [35.169.212.159]) by smtp.gmail.com with ESMTPSA id d16sm1358101qka.43.2021.03.18.03.10.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 18 Mar 2021 03:10:06 -0700 (PDT) From: sj38.park@gmail.com To: akpm@linux-foundation.org Cc: SeongJae Park , Jonathan.Cameron@Huawei.com, acme@kernel.org, alexander.shishkin@linux.intel.com, amit@kernel.org, benh@kernel.crashing.org, brendanhiggins@google.com, corbet@lwn.net, david@redhat.com, dwmw@amazon.com, elver@google.com, fan.du@intel.com, foersleo@amazon.de, gthelen@google.com, mgorman@suse.de, minchan@kernel.org, mingo@redhat.com, namhyung@kernel.org, peterz@infradead.org, riel@surriel.com, rientjes@google.com, rostedt@goodmis.org, rppt@kernel.org, shakeelb@google.com, shuah@kernel.org, sj38.park@gmail.com, snu@amazon.de, vbabka@suse.cz, vdavydov.dev@gmail.com, zgf574564920@gmail.com, linux-damon@amazon.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v25 03/13] mm/damon: Adaptively adjust regions Date: Thu, 18 Mar 2021 10:08:46 +0000 Message-Id: <20210318100856.34715-4-sj38.park@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210318100856.34715-1-sj38.park@gmail.com> References: <20210318100856.34715-1-sj38.park@gmail.com> X-Stat-Signature: 6bi5sq31qua8i477qw8umykcj6eq7h9a X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 8C9202000259 Received-SPF: none (gmail.com>: No applicable sender policy available) receiver=imf28; identity=mailfrom; envelope-from=""; helo=mail-qt1-f177.google.com; client-ip=209.85.160.177 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1616062207-584687 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park Even somehow the initial monitoring target regions are well constructed to fulfill the assumption (pages in same region have similar access frequencies), the data access pattern can be dynamically changed. This will result in low monitoring quality. To keep the assumption as much as possible, DAMON adaptively merges and splits each region based on their access frequency. For each ``aggregation interval``, it compares the access frequencies of adjacent regions and merges those if the frequency difference is small. Then, after it reports and clears the aggregated access frequency of each region, it splits each region into two or three regions if the total number of regions will not exceed the user-specified maximum number of regions after the split. In this way, DAMON provides its best-effort quality and minimal overhead while keeping the upper-bound overhead that users set. Signed-off-by: SeongJae Park Reviewed-by: Leonard Foerster --- include/linux/damon.h | 23 +++-- mm/damon/core.c | 214 +++++++++++++++++++++++++++++++++++++++++- 2 files changed, 227 insertions(+), 10 deletions(-) diff --git a/include/linux/damon.h b/include/linux/damon.h index 67db309ad61b..0bd5d6913a6c 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -12,6 +12,9 @@ #include #include +/* Minimal region size. Every damon_region is aligned by this. */ +#define DAMON_MIN_REGION PAGE_SIZE + /** * struct damon_addr_range - Represents an address region of [@start, @end). * @start: Start address of the region (inclusive). @@ -85,6 +88,8 @@ struct damon_ctx; * prepared for the next access check. * @check_accesses should check the accesses to each region that made after the * last preparation and update the number of observed accesses of each region. + * It should also return max number of observed accesses that made as a result + * of its update. The value will be used for regions adjustment threshold. * @reset_aggregated should reset the access monitoring results that aggregated * by @check_accesses. * @target_valid should check whether the target is still valid for the @@ -95,7 +100,7 @@ struct damon_primitive { void (*init)(struct damon_ctx *context); void (*update)(struct damon_ctx *context); void (*prepare_access_checks)(struct damon_ctx *context); - void (*check_accesses)(struct damon_ctx *context); + unsigned int (*check_accesses)(struct damon_ctx *context); void (*reset_aggregated)(struct damon_ctx *context); bool (*target_valid)(void *target); void (*cleanup)(struct damon_ctx *context); @@ -172,7 +177,9 @@ struct damon_callback { * @primitive: Set of monitoring primitives for given use cases. * @callback: Set of callbacks for monitoring events notifications. * - * @region_targets: Head of monitoring targets (&damon_target) list. + * @min_nr_regions: The minimum number of adaptive monitoring regions. + * @max_nr_regions: The maximum number of adaptive monitoring regions. + * @adaptive_targets: Head of monitoring targets (&damon_target) list. */ struct damon_ctx { unsigned long sample_interval; @@ -191,7 +198,9 @@ struct damon_ctx { struct damon_primitive primitive; struct damon_callback callback; - struct list_head region_targets; + unsigned long min_nr_regions; + unsigned long max_nr_regions; + struct list_head adaptive_targets; }; #define damon_next_region(r) \ @@ -207,10 +216,10 @@ struct damon_ctx { list_for_each_entry_safe(r, next, &t->regions_list, list) #define damon_for_each_target(t, ctx) \ - list_for_each_entry(t, &(ctx)->region_targets, list) + list_for_each_entry(t, &(ctx)->adaptive_targets, list) #define damon_for_each_target_safe(t, next, ctx) \ - list_for_each_entry_safe(t, next, &(ctx)->region_targets, list) + list_for_each_entry_safe(t, next, &(ctx)->adaptive_targets, list) #ifdef CONFIG_DAMON @@ -224,11 +233,13 @@ struct damon_target *damon_new_target(unsigned long id); void damon_add_target(struct damon_ctx *ctx, struct damon_target *t); void damon_free_target(struct damon_target *t); void damon_destroy_target(struct damon_target *t); +unsigned int damon_nr_regions(struct damon_target *t); struct damon_ctx *damon_new_ctx(void); void damon_destroy_ctx(struct damon_ctx *ctx); int damon_set_attrs(struct damon_ctx *ctx, unsigned long sample_int, - unsigned long aggr_int, unsigned long primitive_upd_int); + unsigned long aggr_int, unsigned long primitive_upd_int, + unsigned long min_nr_reg, unsigned long max_nr_reg); int damon_start(struct damon_ctx **ctxs, int nr_ctxs); int damon_stop(struct damon_ctx **ctxs, int nr_ctxs); diff --git a/mm/damon/core.c b/mm/damon/core.c index 94db494dcf70..b36b6bdd94e2 100644 --- a/mm/damon/core.c +++ b/mm/damon/core.c @@ -10,8 +10,12 @@ #include #include #include +#include #include +/* Get a random number in [l, r) */ +#define damon_rand(l, r) (l + prandom_u32_max(r - l)) + static DEFINE_MUTEX(damon_lock); static int nr_running_ctxs; @@ -87,7 +91,7 @@ struct damon_target *damon_new_target(unsigned long id) void damon_add_target(struct damon_ctx *ctx, struct damon_target *t) { - list_add_tail(&t->list, &ctx->region_targets); + list_add_tail(&t->list, &ctx->adaptive_targets); } static void damon_del_target(struct damon_target *t) @@ -110,6 +114,17 @@ void damon_destroy_target(struct damon_target *t) damon_free_target(t); } +unsigned int damon_nr_regions(struct damon_target *t) +{ + struct damon_region *r; + unsigned int nr_regions = 0; + + damon_for_each_region(r, t) + nr_regions++; + + return nr_regions; +} + struct damon_ctx *damon_new_ctx(void) { struct damon_ctx *ctx; @@ -127,7 +142,10 @@ struct damon_ctx *damon_new_ctx(void) mutex_init(&ctx->kdamond_lock); - INIT_LIST_HEAD(&ctx->region_targets); + ctx->min_nr_regions = 10; + ctx->max_nr_regions = 1000; + + INIT_LIST_HEAD(&ctx->adaptive_targets); return ctx; } @@ -157,6 +175,8 @@ void damon_destroy_ctx(struct damon_ctx *ctx) * @sample_int: time interval between samplings * @aggr_int: time interval between aggregations * @primitive_upd_int: time interval between monitoring primitive updates + * @min_nr_reg: minimal number of regions + * @max_nr_reg: maximum number of regions * * This function should not be called while the kdamond is running. * Every time interval is in micro-seconds. @@ -164,15 +184,49 @@ void damon_destroy_ctx(struct damon_ctx *ctx) * Return: 0 on success, negative error code otherwise. */ int damon_set_attrs(struct damon_ctx *ctx, unsigned long sample_int, - unsigned long aggr_int, unsigned long primitive_upd_int) + unsigned long aggr_int, unsigned long primitive_upd_int, + unsigned long min_nr_reg, unsigned long max_nr_reg) { + if (min_nr_reg < 3) { + pr_err("min_nr_regions (%lu) must be at least 3\n", + min_nr_reg); + return -EINVAL; + } + if (min_nr_reg > max_nr_reg) { + pr_err("invalid nr_regions. min (%lu) > max (%lu)\n", + min_nr_reg, max_nr_reg); + return -EINVAL; + } + ctx->sample_interval = sample_int; ctx->aggr_interval = aggr_int; ctx->primitive_update_interval = primitive_upd_int; + ctx->min_nr_regions = min_nr_reg; + ctx->max_nr_regions = max_nr_reg; return 0; } +/* Returns the size upper limit for each monitoring region */ +static unsigned long damon_region_sz_limit(struct damon_ctx *ctx) +{ + struct damon_target *t; + struct damon_region *r; + unsigned long sz = 0; + + damon_for_each_target(t, ctx) { + damon_for_each_region(r, t) + sz += r->ar.end - r->ar.start; + } + + if (ctx->min_nr_regions) + sz /= ctx->min_nr_regions; + if (sz < DAMON_MIN_REGION) + sz = DAMON_MIN_REGION; + + return sz; +} + static bool damon_kdamond_running(struct damon_ctx *ctx) { bool running; @@ -339,6 +393,149 @@ static void kdamond_reset_aggregated(struct damon_ctx *c) } } +#define sz_damon_region(r) (r->ar.end - r->ar.start) + +/* + * Merge two adjacent regions into one region + */ +static void damon_merge_two_regions(struct damon_region *l, + struct damon_region *r) +{ + unsigned long sz_l = sz_damon_region(l), sz_r = sz_damon_region(r); + + l->nr_accesses = (l->nr_accesses * sz_l + r->nr_accesses * sz_r) / + (sz_l + sz_r); + l->ar.end = r->ar.end; + damon_destroy_region(r); +} + +#define diff_of(a, b) (a > b ? a - b : b - a) + +/* + * Merge adjacent regions having similar access frequencies + * + * t target affected by this merge operation + * thres '->nr_accesses' diff threshold for the merge + * sz_limit size upper limit of each region + */ +static void damon_merge_regions_of(struct damon_target *t, unsigned int thres, + unsigned long sz_limit) +{ + struct damon_region *r, *prev = NULL, *next; + + damon_for_each_region_safe(r, next, t) { + if (prev && prev->ar.end == r->ar.start && + diff_of(prev->nr_accesses, r->nr_accesses) <= thres && + sz_damon_region(prev) + sz_damon_region(r) <= sz_limit) + damon_merge_two_regions(prev, r); + else + prev = r; + } +} + +/* + * Merge adjacent regions having similar access frequencies + * + * threshold '->nr_accesses' diff threshold for the merge + * sz_limit size upper limit of each region + * + * This function merges monitoring target regions which are adjacent and their + * access frequencies are similar. This is for minimizing the monitoring + * overhead under the dynamically changeable access pattern. If a merge was + * unnecessarily made, later 'kdamond_split_regions()' will revert it. + */ +static void kdamond_merge_regions(struct damon_ctx *c, unsigned int threshold, + unsigned long sz_limit) +{ + struct damon_target *t; + + damon_for_each_target(t, c) + damon_merge_regions_of(t, threshold, sz_limit); +} + +/* + * Split a region in two + * + * r the region to be split + * sz_r size of the first sub-region that will be made + */ +static void damon_split_region_at(struct damon_ctx *ctx, + struct damon_region *r, unsigned long sz_r) +{ + struct damon_region *new; + + new = damon_new_region(r->ar.start + sz_r, r->ar.end); + if (!new) + return; + + r->ar.end = new->ar.start; + + damon_insert_region(new, r, damon_next_region(r)); +} + +/* Split every region in the given target into 'nr_subs' regions */ +static void damon_split_regions_of(struct damon_ctx *ctx, + struct damon_target *t, int nr_subs) +{ + struct damon_region *r, *next; + unsigned long sz_region, sz_sub = 0; + int i; + + damon_for_each_region_safe(r, next, t) { + sz_region = r->ar.end - r->ar.start; + + for (i = 0; i < nr_subs - 1 && + sz_region > 2 * DAMON_MIN_REGION; i++) { + /* + * Randomly select size of left sub-region to be at + * least 10 percent and at most 90% of original region + */ + sz_sub = ALIGN_DOWN(damon_rand(1, 10) * + sz_region / 10, DAMON_MIN_REGION); + /* Do not allow blank region */ + if (sz_sub == 0 || sz_sub >= sz_region) + continue; + + damon_split_region_at(ctx, r, sz_sub); + sz_region = sz_sub; + } + } +} + +/* + * Split every target region into randomly-sized small regions + * + * This function splits every target region into random-sized small regions if + * current total number of the regions is equal or smaller than half of the + * user-specified maximum number of regions. This is for maximizing the + * monitoring accuracy under the dynamically changeable access patterns. If a + * split was unnecessarily made, later 'kdamond_merge_regions()' will revert + * it. + */ +static void kdamond_split_regions(struct damon_ctx *ctx) +{ + struct damon_target *t; + unsigned int nr_regions = 0; + static unsigned int last_nr_regions; + int nr_subregions = 2; + + damon_for_each_target(t, ctx) + nr_regions += damon_nr_regions(t); + + if (nr_regions > ctx->max_nr_regions / 2) + return; + + /* Maybe the middle of the region has different access frequency */ + if (last_nr_regions == nr_regions && + nr_regions < ctx->max_nr_regions / 3) + nr_subregions = 3; + + damon_for_each_target(t, ctx) + damon_split_regions_of(ctx, t, nr_subregions); + + last_nr_regions = nr_regions; +} + /* * Check whether it is time to check and apply the target monitoring regions * @@ -395,6 +592,8 @@ static int kdamond_fn(void *data) struct damon_ctx *ctx = (struct damon_ctx *)data; struct damon_target *t; struct damon_region *r, *next; + unsigned int max_nr_accesses = 0; + unsigned long sz_limit = 0; pr_info("kdamond (%d) starts\n", ctx->kdamond->pid); @@ -403,6 +602,8 @@ static int kdamond_fn(void *data) if (ctx->callback.before_start && ctx->callback.before_start(ctx)) set_kdamond_stop(ctx); + sz_limit = damon_region_sz_limit(ctx); + while (!kdamond_need_stop(ctx)) { if (ctx->primitive.prepare_access_checks) ctx->primitive.prepare_access_checks(ctx); @@ -413,13 +614,17 @@ static int kdamond_fn(void *data) usleep_range(ctx->sample_interval, ctx->sample_interval + 1); if (ctx->primitive.check_accesses) - ctx->primitive.check_accesses(ctx); + max_nr_accesses = ctx->primitive.check_accesses(ctx); if (kdamond_aggregate_interval_passed(ctx)) { + kdamond_merge_regions(ctx, + max_nr_accesses / 10, + sz_limit); if (ctx->callback.after_aggregation && ctx->callback.after_aggregation(ctx)) set_kdamond_stop(ctx); kdamond_reset_aggregated(ctx); + kdamond_split_regions(ctx); if (ctx->primitive.reset_aggregated) ctx->primitive.reset_aggregated(ctx); } @@ -427,6 +632,7 @@ static int kdamond_fn(void *data) if (kdamond_need_update_primitive(ctx)) { if (ctx->primitive.update) ctx->primitive.update(ctx); + sz_limit = damon_region_sz_limit(ctx); } } damon_for_each_target(t, ctx) { From patchwork Thu Mar 18 10:08:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 12147801 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CBD1BC433E0 for ; Thu, 18 Mar 2021 10:10:20 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5E9AD61580 for ; Thu, 18 Mar 2021 10:10:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5E9AD61580 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B083F6B0075; Thu, 18 Mar 2021 06:10:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id ADDEE6B0078; Thu, 18 Mar 2021 06:10:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8BCCE8D0001; Thu, 18 Mar 2021 06:10:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0096.hostedemail.com [216.40.44.96]) by kanga.kvack.org (Postfix) with ESMTP id 659D46B0075 for ; Thu, 18 Mar 2021 06:10:17 -0400 (EDT) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 24B901812DD06 for ; Thu, 18 Mar 2021 10:10:17 +0000 (UTC) X-FDA: 77932574832.20.2D64C8C Received: from mail-qk1-f169.google.com (mail-qk1-f169.google.com [209.85.222.169]) by imf13.hostedemail.com (Postfix) with ESMTP id 702A1E005F00 for ; Thu, 18 Mar 2021 10:10:09 +0000 (UTC) Received: by mail-qk1-f169.google.com with SMTP id g185so1378811qkf.6 for ; Thu, 18 Mar 2021 03:10:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=Sd0jraoDnqYmxG8x85dNQRB6nR/1nU+ZkXntGFxIbSg=; b=IxUAIOaqQRHYJhfy7QrnRuR894KV8fGI5XY2BIXxsipzydyXMDsElXuDcNNyIhclhS vmLyoIyyjNEsHkxBOL39457K0c96AwVx0ptOZr7OKU85Ps4hxYXAxFmAl501vtYqEQ8j NlqRawgitKJY61wdlxSsnoq2KuV+5dETC1M+Zv/Ybjcvu8qNiFyOXyANumJF8tBCZsQD g7XKaoQ67RMULe0L6+q08dVpv+71gRhGpAz5aAubY+H6HlCm0YesOcQ/1Zxxde9dcWbH k+U6qTmVWsedfYDCBbrz7/2lflvTPJyu4L/NB0KdSlb6hvMUQUR4N4WtP18PLwOUeAjO DbsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Sd0jraoDnqYmxG8x85dNQRB6nR/1nU+ZkXntGFxIbSg=; b=U7TBd6n2snCVdp+dErYwnZUxmdintKz80xCtTncN9yUB8WyhmsZd9XgN/edamYiM2Z UAUID/urg9t5BLxSMjhdMZwchJ0VXD7E3GNUi0ltouShp7VlqvabGPfaB5eNBdNTRgsS QP3fSF7DdXSDAPV5OG9Y85mMxTv/+4u0bDzDai2uBt3OAm7In4zH6Do4LhMYj89WGWk/ GtZSihz2VFIsFKI8uY0JIyJmwnuDLEHFqw5sEVtETZrzlze5rQHR648GGnoGSnnO4ikT znf9aZKqVHKiYYvKia6f/L8DyfSXwRd1w1Epk7+AqW5DrkwliCYjbcJMP7AYshsEZxn8 8Qng== X-Gm-Message-State: AOAM533jqunISV0OPF6JLzIOqCeM8B2O4M8KWox/vSNQNUPxgjspKTtk ME3VpN/zuIf4oVYiTvj6gBY= X-Google-Smtp-Source: ABdhPJzOUjIfdv+Q3Ed8kmE8XilzMh8XwNHnmbSkZ9GnGsNmC36Mp4HqGKLBaYISSNwvCc7v2VHTHA== X-Received: by 2002:a05:620a:993:: with SMTP id x19mr3375030qkx.77.1616062208819; Thu, 18 Mar 2021 03:10:08 -0700 (PDT) Received: from localhost.localdomain (ec2-35-169-212-159.compute-1.amazonaws.com. [35.169.212.159]) by smtp.gmail.com with ESMTPSA id d16sm1358101qka.43.2021.03.18.03.10.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 18 Mar 2021 03:10:08 -0700 (PDT) From: sj38.park@gmail.com To: akpm@linux-foundation.org Cc: SeongJae Park , Jonathan.Cameron@Huawei.com, acme@kernel.org, alexander.shishkin@linux.intel.com, amit@kernel.org, benh@kernel.crashing.org, brendanhiggins@google.com, corbet@lwn.net, david@redhat.com, dwmw@amazon.com, elver@google.com, fan.du@intel.com, foersleo@amazon.de, gthelen@google.com, mgorman@suse.de, minchan@kernel.org, mingo@redhat.com, namhyung@kernel.org, peterz@infradead.org, riel@surriel.com, rientjes@google.com, rostedt@goodmis.org, rppt@kernel.org, shakeelb@google.com, shuah@kernel.org, sj38.park@gmail.com, snu@amazon.de, vbabka@suse.cz, vdavydov.dev@gmail.com, zgf574564920@gmail.com, linux-damon@amazon.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v25 04/13] mm/idle_page_tracking: Make PG_idle reusable Date: Thu, 18 Mar 2021 10:08:47 +0000 Message-Id: <20210318100856.34715-5-sj38.park@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210318100856.34715-1-sj38.park@gmail.com> References: <20210318100856.34715-1-sj38.park@gmail.com> X-Stat-Signature: d5o5176icp5iogpcph1jhwyb166xq9g8 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 702A1E005F00 Received-SPF: none (gmail.com>: No applicable sender policy available) receiver=imf13; identity=mailfrom; envelope-from=""; helo=mail-qk1-f169.google.com; client-ip=209.85.222.169 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1616062209-470172 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park PG_idle and PG_young allow the two PTE Accessed bit users, Idle Page Tracking and the reclaim logic concurrently work while don't interfere each other. That is, when they need to clear the Accessed bit, they set PG_young to represent the previous state of the bit, respectively. And when they need to read the bit, if the bit is cleared, they further read the PG_young to know whether the other has cleared the bit meanwhile or not. We could add another page flag and extend the mechanism to use the flag if we need to add another concurrent PTE Accessed bit user subsystem. However, the space is limited. Meanwhile, if the new subsystem is mutually exclusive with IDLE_PAGE_TRACKING or interfering with it is not a real problem, it would be ok to simply reuse the PG_idle flag. However, it's impossible because the flags are dependent on IDLE_PAGE_TRACKING. To allow such reuse of the flags, this commit separates the PG_young and PG_idle flag logic from IDLE_PAGE_TRACKING and introduces new kernel config, 'PAGE_IDLE_FLAG'. Hence, a new subsystem would be able to reuse PG_idle without depending on IDLE_PAGE_TRACKING. In the next commit, DAMON's reference implementation of the virtual memory address space monitoring primitives will use it. Signed-off-by: SeongJae Park Reviewed-by: Shakeel Butt --- include/linux/page-flags.h | 4 ++-- include/linux/page_ext.h | 2 +- include/linux/page_idle.h | 6 +++--- include/trace/events/mmflags.h | 2 +- mm/Kconfig | 8 ++++++++ mm/page_ext.c | 12 +++++++++++- mm/page_idle.c | 10 ---------- 7 files changed, 26 insertions(+), 18 deletions(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index 04a34c08e0a6..6be2c1e2fb48 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -131,7 +131,7 @@ enum pageflags { #ifdef CONFIG_MEMORY_FAILURE PG_hwpoison, /* hardware poisoned page. Don't touch */ #endif -#if defined(CONFIG_IDLE_PAGE_TRACKING) && defined(CONFIG_64BIT) +#if defined(CONFIG_PAGE_IDLE_FLAG) && defined(CONFIG_64BIT) PG_young, PG_idle, #endif @@ -436,7 +436,7 @@ PAGEFLAG_FALSE(HWPoison) #define __PG_HWPOISON 0 #endif -#if defined(CONFIG_IDLE_PAGE_TRACKING) && defined(CONFIG_64BIT) +#if defined(CONFIG_PAGE_IDLE_FLAG) && defined(CONFIG_64BIT) TESTPAGEFLAG(Young, young, PF_ANY) SETPAGEFLAG(Young, young, PF_ANY) TESTCLEARFLAG(Young, young, PF_ANY) diff --git a/include/linux/page_ext.h b/include/linux/page_ext.h index aff81ba31bd8..fabb2e1e087f 100644 --- a/include/linux/page_ext.h +++ b/include/linux/page_ext.h @@ -19,7 +19,7 @@ struct page_ext_operations { enum page_ext_flags { PAGE_EXT_OWNER, PAGE_EXT_OWNER_ALLOCATED, -#if defined(CONFIG_IDLE_PAGE_TRACKING) && !defined(CONFIG_64BIT) +#if defined(CONFIG_PAGE_IDLE_FLAG) && !defined(CONFIG_64BIT) PAGE_EXT_YOUNG, PAGE_EXT_IDLE, #endif diff --git a/include/linux/page_idle.h b/include/linux/page_idle.h index 1e894d34bdce..d8a6aecf99cb 100644 --- a/include/linux/page_idle.h +++ b/include/linux/page_idle.h @@ -6,7 +6,7 @@ #include #include -#ifdef CONFIG_IDLE_PAGE_TRACKING +#ifdef CONFIG_PAGE_IDLE_FLAG #ifdef CONFIG_64BIT static inline bool page_is_young(struct page *page) @@ -106,7 +106,7 @@ static inline void clear_page_idle(struct page *page) } #endif /* CONFIG_64BIT */ -#else /* !CONFIG_IDLE_PAGE_TRACKING */ +#else /* !CONFIG_PAGE_IDLE_FLAG */ static inline bool page_is_young(struct page *page) { @@ -135,6 +135,6 @@ static inline void clear_page_idle(struct page *page) { } -#endif /* CONFIG_IDLE_PAGE_TRACKING */ +#endif /* CONFIG_PAGE_IDLE_FLAG */ #endif /* _LINUX_MM_PAGE_IDLE_H */ diff --git a/include/trace/events/mmflags.h b/include/trace/events/mmflags.h index 629c7a0eaff2..ea434bbc2d2b 100644 --- a/include/trace/events/mmflags.h +++ b/include/trace/events/mmflags.h @@ -73,7 +73,7 @@ #define IF_HAVE_PG_HWPOISON(flag,string) #endif -#if defined(CONFIG_IDLE_PAGE_TRACKING) && defined(CONFIG_64BIT) +#if defined(CONFIG_PAGE_IDLE_FLAG) && defined(CONFIG_64BIT) #define IF_HAVE_PG_IDLE(flag,string) ,{1UL << flag, string} #else #define IF_HAVE_PG_IDLE(flag,string) diff --git a/mm/Kconfig b/mm/Kconfig index fa8c95a54346..19e8f8284d00 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -765,10 +765,18 @@ config DEFERRED_STRUCT_PAGE_INIT lifetime of the system until these kthreads finish the initialisation. +config PAGE_IDLE_FLAG + bool "Add PG_idle and PG_young flags" + help + This feature adds PG_idle and PG_young flags in 'struct page'. PTE + Accessed bit writers can set the state of the bit in the flags to let + other PTE Accessed bit readers don't disturbed. + config IDLE_PAGE_TRACKING bool "Enable idle page tracking" depends on SYSFS && MMU select PAGE_EXTENSION if !64BIT + select PAGE_IDLE_FLAG help This feature allows to estimate the amount of user pages that have not been touched during a given period of time. This information can diff --git a/mm/page_ext.c b/mm/page_ext.c index df6f74aac8e1..8e59da0f4367 100644 --- a/mm/page_ext.c +++ b/mm/page_ext.c @@ -58,11 +58,21 @@ * can utilize this callback to initialize the state of it correctly. */ +#if defined(CONFIG_PAGE_IDLE_FLAG) && !defined(CONFIG_64BIT) +static bool need_page_idle(void) +{ + return true; +} +struct page_ext_operations page_idle_ops = { + .need = need_page_idle, +}; +#endif + static struct page_ext_operations *page_ext_ops[] = { #ifdef CONFIG_PAGE_OWNER &page_owner_ops, #endif -#if defined(CONFIG_IDLE_PAGE_TRACKING) && !defined(CONFIG_64BIT) +#if defined(CONFIG_PAGE_IDLE_FLAG) && !defined(CONFIG_64BIT) &page_idle_ops, #endif }; diff --git a/mm/page_idle.c b/mm/page_idle.c index 64e5344a992c..edead6a8a5f9 100644 --- a/mm/page_idle.c +++ b/mm/page_idle.c @@ -207,16 +207,6 @@ static const struct attribute_group page_idle_attr_group = { .name = "page_idle", }; -#ifndef CONFIG_64BIT -static bool need_page_idle(void) -{ - return true; -} -struct page_ext_operations page_idle_ops = { - .need = need_page_idle, -}; -#endif - static int __init page_idle_init(void) { int err; From patchwork Thu Mar 18 10:08:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 12147797 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B3567C433DB for ; Thu, 18 Mar 2021 10:10:17 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 12F4764F3B for ; Thu, 18 Mar 2021 10:10:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 12F4764F3B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9A4E06B0071; Thu, 18 Mar 2021 06:10:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 97C196B0074; Thu, 18 Mar 2021 06:10:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 780926B0078; Thu, 18 Mar 2021 06:10:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0242.hostedemail.com [216.40.44.242]) by kanga.kvack.org (Postfix) with ESMTP id 5824A6B0071 for ; Thu, 18 Mar 2021 06:10:16 -0400 (EDT) Received: from smtpin40.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 188AE40E1 for ; Thu, 18 Mar 2021 10:10:16 +0000 (UTC) X-FDA: 77932574832.40.241BE82 Received: from mail-qk1-f181.google.com (mail-qk1-f181.google.com [209.85.222.181]) by imf16.hostedemail.com (Postfix) with ESMTP id DC72B801914A for ; Thu, 18 Mar 2021 10:10:11 +0000 (UTC) Received: by mail-qk1-f181.google.com with SMTP id n79so1392578qke.3 for ; Thu, 18 Mar 2021 03:10:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=KO/E4ogVXyfVzD+Ll4STgUPuR/+p1poNeGEhJzQAkAs=; b=SvlXw1dGXJPiqXEo+MhIk4rV4FcCvoHXc79XVxs3HeD7P8W0xWrGtOszD/H5gIRsOi XmR5XfJ0a6xIUk+wqjCs+Iix4dnzQfYANTXWqCdHPFhh8fy2OwQkWoruUgP73N/rt7J4 j1eI65b8Qg3Q8A/t0aXgD1wCS49mWS73sb2GOaeZIUOo3wQ2MXAI0uHWC3S0ZZr3MpDx HIBztvg0Z9GMQ7Ti5YuvL/TD+SREgiY3nr7C0Wg7H72DaEU6DbUOrnHn8Y2dm5MPnCzj 0NFMiGWnyAwcL4wIrVg1pCcVxZtFbmEXOQ/Ou1sL59uwXpqtpmnMgzEfA+EDGsxVNHvr jvUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=KO/E4ogVXyfVzD+Ll4STgUPuR/+p1poNeGEhJzQAkAs=; b=qeF9mPgKCcJqklGOSN7uS/y3adSLABWMwgyKTPigzf/lJDi8uZKKj/SI++5XiI1G/e T1bxNsB/rVsQDMfzYpbfNYWB0qthL8QmqFq3SMfrTwHYTIQT87rP3H9u7US8DwY11AQn ctJztOPR4TCIyiUoM8xqckRBXsqAvo/VhjbQcr4FUKsrAwS13nvRXFPIHipC+hUNkfP4 +jSKexI5HBQK1WQyOUMmUDEiPsbx5E8HdKpkkzwrcUb6T9f4uwGV4A/9ERgKLMH3QOqe d6cygdPUmDly0IIXEH7BFK5JJJEb8LyCeoKzyfGFjdvkJvGugQCSuGz71SlMqPPkf8Or 9qqQ== X-Gm-Message-State: AOAM531M6X6aU8O5KytNruRM62IENhAm19np8AOnG6NXGrmKvqC0nliD loOIP08kk3mhAn3d1l22OBw= X-Google-Smtp-Source: ABdhPJx2M/5KAOA+W2i9eAc7uR0zDRLesdUhf7u5/4E6CFXVd8Jv4H/wgdom3nKJTXLyYE0IwTN2ag== X-Received: by 2002:a37:40d5:: with SMTP id n204mr3464328qka.79.1616062211076; Thu, 18 Mar 2021 03:10:11 -0700 (PDT) Received: from localhost.localdomain (ec2-35-169-212-159.compute-1.amazonaws.com. [35.169.212.159]) by smtp.gmail.com with ESMTPSA id d16sm1358101qka.43.2021.03.18.03.10.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 18 Mar 2021 03:10:10 -0700 (PDT) From: sj38.park@gmail.com To: akpm@linux-foundation.org Cc: SeongJae Park , Jonathan.Cameron@Huawei.com, acme@kernel.org, alexander.shishkin@linux.intel.com, amit@kernel.org, benh@kernel.crashing.org, brendanhiggins@google.com, corbet@lwn.net, david@redhat.com, dwmw@amazon.com, elver@google.com, fan.du@intel.com, foersleo@amazon.de, gthelen@google.com, mgorman@suse.de, minchan@kernel.org, mingo@redhat.com, namhyung@kernel.org, peterz@infradead.org, riel@surriel.com, rientjes@google.com, rostedt@goodmis.org, rppt@kernel.org, shakeelb@google.com, shuah@kernel.org, sj38.park@gmail.com, snu@amazon.de, vbabka@suse.cz, vdavydov.dev@gmail.com, zgf574564920@gmail.com, linux-damon@amazon.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v25 05/13] mm/damon: Implement primitives for the virtual memory address spaces Date: Thu, 18 Mar 2021 10:08:48 +0000 Message-Id: <20210318100856.34715-6-sj38.park@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210318100856.34715-1-sj38.park@gmail.com> References: <20210318100856.34715-1-sj38.park@gmail.com> X-Stat-Signature: i4xfwnckd6wmujz6nzqwmr9otjchu9h1 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: DC72B801914A Received-SPF: none (gmail.com>: No applicable sender policy available) receiver=imf16; identity=mailfrom; envelope-from=""; helo=mail-qk1-f181.google.com; client-ip=209.85.222.181 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1616062211-972012 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park This commit introduces a reference implementation of the address space specific low level primitives for the virtual address space, so that users of DAMON can easily monitor the data accesses on virtual address spaces of specific processes by simply configuring the implementation to be used by DAMON. The low level primitives for the fundamental access monitoring are defined in two parts: 1. Identification of the monitoring target address range for the address space. 2. Access check of specific address range in the target space. The reference implementation for the virtual address space does the works as below. PTE Accessed-bit Based Access Check ----------------------------------- The implementation uses PTE Accessed-bit for basic access checks. That is, it clears the bit for the next sampling target page and checks whether it is set again after one sampling period. This could disturb the reclaim logic. DAMON uses ``PG_idle`` and ``PG_young`` page flags to solve the conflict, as Idle page tracking does. VMA-based Target Address Range Construction ------------------------------------------- Only small parts in the super-huge virtual address space of the processes are mapped to physical memory and accessed. Thus, tracking the unmapped address regions is just wasteful. However, because DAMON can deal with some level of noise using the adaptive regions adjustment mechanism, tracking every mapping is not strictly required but could even incur a high overhead in some cases. That said, too huge unmapped areas inside the monitoring target should be removed to not take the time for the adaptive mechanism. For the reason, this implementation converts the complex mappings to three distinct regions that cover every mapped area of the address space. Also, the two gaps between the three regions are the two biggest unmapped areas in the given address space. The two biggest unmapped areas would be the gap between the heap and the uppermost mmap()-ed region, and the gap between the lowermost mmap()-ed region and the stack in most of the cases. Because these gaps are exceptionally huge in usual address spaces, excluding these will be sufficient to make a reasonable trade-off. Below shows this in detail:: (small mmap()-ed regions and munmap()-ed regions) Signed-off-by: SeongJae Park Reviewed-by: Leonard Foerster --- include/linux/damon.h | 13 + mm/damon/Kconfig | 9 + mm/damon/Makefile | 1 + mm/damon/vaddr.c | 579 ++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 602 insertions(+) create mode 100644 mm/damon/vaddr.c diff --git a/include/linux/damon.h b/include/linux/damon.h index 0bd5d6913a6c..72cf5ebd35fe 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -246,4 +246,17 @@ int damon_stop(struct damon_ctx **ctxs, int nr_ctxs); #endif /* CONFIG_DAMON */ +#ifdef CONFIG_DAMON_VADDR + +/* Monitoring primitives for virtual memory address spaces */ +void damon_va_init(struct damon_ctx *ctx); +void damon_va_update(struct damon_ctx *ctx); +void damon_va_prepare_access_checks(struct damon_ctx *ctx); +unsigned int damon_va_check_accesses(struct damon_ctx *ctx); +bool damon_va_target_valid(void *t); +void damon_va_cleanup(struct damon_ctx *ctx); +void damon_va_set_primitives(struct damon_ctx *ctx); + +#endif /* CONFIG_DAMON_VADDR */ + #endif /* _DAMON_H */ diff --git a/mm/damon/Kconfig b/mm/damon/Kconfig index d00e99ac1a15..8ae080c52950 100644 --- a/mm/damon/Kconfig +++ b/mm/damon/Kconfig @@ -12,4 +12,13 @@ config DAMON See https://damonitor.github.io/doc/html/latest-damon/index.html for more information. +config DAMON_VADDR + bool "Data access monitoring primitives for virtual address spaces" + depends on DAMON && MMU + select PAGE_EXTENSION if !64BIT + select PAGE_IDLE_FLAG + help + This builds the default data access monitoring primitives for DAMON + that works for virtual address spaces. + endmenu diff --git a/mm/damon/Makefile b/mm/damon/Makefile index 4fd2edb4becf..6ebbd08aed67 100644 --- a/mm/damon/Makefile +++ b/mm/damon/Makefile @@ -1,3 +1,4 @@ # SPDX-License-Identifier: GPL-2.0 obj-$(CONFIG_DAMON) := core.o +obj-$(CONFIG_DAMON_VADDR) += vaddr.o diff --git a/mm/damon/vaddr.c b/mm/damon/vaddr.c new file mode 100644 index 000000000000..e2bf3db31c2c --- /dev/null +++ b/mm/damon/vaddr.c @@ -0,0 +1,579 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * DAMON Primitives for Virtual Address Spaces + * + * Author: SeongJae Park + */ + +#define pr_fmt(fmt) "damon-va: " fmt + +#include +#include +#include +#include +#include +#include +#include + +/* Get a random number in [l, r) */ +#define damon_rand(l, r) (l + prandom_u32_max(r - l)) + +/* + * 't->id' should be the pointer to the relevant 'struct pid' having reference + * count. Caller must put the returned task, unless it is NULL. + */ +#define damon_get_task_struct(t) \ + (get_pid_task((struct pid *)t->id, PIDTYPE_PID)) + +/* + * Get the mm_struct of the given target + * + * Caller _must_ put the mm_struct after use, unless it is NULL. + * + * Returns the mm_struct of the target on success, NULL on failure + */ +static struct mm_struct *damon_get_mm(struct damon_target *t) +{ + struct task_struct *task; + struct mm_struct *mm; + + task = damon_get_task_struct(t); + if (!task) + return NULL; + + mm = get_task_mm(task); + put_task_struct(task); + return mm; +} + +/* + * Functions for the initial monitoring target regions construction + */ + +/* + * Size-evenly split a region into 'nr_pieces' small regions + * + * Returns 0 on success, or negative error code otherwise. + */ +static int damon_va_evenly_split_region(struct damon_ctx *ctx, + struct damon_region *r, unsigned int nr_pieces) +{ + unsigned long sz_orig, sz_piece, orig_end; + struct damon_region *n = NULL, *next; + unsigned long start; + + if (!r || !nr_pieces) + return -EINVAL; + + orig_end = r->ar.end; + sz_orig = r->ar.end - r->ar.start; + sz_piece = ALIGN_DOWN(sz_orig / nr_pieces, DAMON_MIN_REGION); + + if (!sz_piece) + return -EINVAL; + + r->ar.end = r->ar.start + sz_piece; + next = damon_next_region(r); + for (start = r->ar.end; start + sz_piece <= orig_end; + start += sz_piece) { + n = damon_new_region(start, start + sz_piece); + if (!n) + return -ENOMEM; + damon_insert_region(n, r, next); + r = n; + } + /* complement last region for possible rounding error */ + if (n) + n->ar.end = orig_end; + + return 0; +} + +static unsigned long sz_range(struct damon_addr_range *r) +{ + return r->end - r->start; +} + +static void swap_ranges(struct damon_addr_range *r1, + struct damon_addr_range *r2) +{ + struct damon_addr_range tmp; + + tmp = *r1; + *r1 = *r2; + *r2 = tmp; +} + +/* + * Find three regions separated by two biggest unmapped regions + * + * vma the head vma of the target address space + * regions an array of three address ranges that results will be saved + * + * This function receives an address space and finds three regions in it which + * separated by the two biggest unmapped regions in the space. Please refer to + * below comments of '__damon_va_init_regions()' function to know why this is + * necessary. + * + * Returns 0 if success, or negative error code otherwise. + */ +static int __damon_va_three_regions(struct vm_area_struct *vma, + struct damon_addr_range regions[3]) +{ + struct damon_addr_range gap = {0}, first_gap = {0}, second_gap = {0}; + struct vm_area_struct *last_vma = NULL; + unsigned long start = 0; + struct rb_root rbroot; + + /* Find two biggest gaps so that first_gap > second_gap > others */ + for (; vma; vma = vma->vm_next) { + if (!last_vma) { + start = vma->vm_start; + goto next; + } + + if (vma->rb_subtree_gap <= sz_range(&second_gap)) { + rbroot.rb_node = &vma->vm_rb; + vma = rb_entry(rb_last(&rbroot), + struct vm_area_struct, vm_rb); + goto next; + } + + gap.start = last_vma->vm_end; + gap.end = vma->vm_start; + if (sz_range(&gap) > sz_range(&second_gap)) { + swap_ranges(&gap, &second_gap); + if (sz_range(&second_gap) > sz_range(&first_gap)) + swap_ranges(&second_gap, &first_gap); + } +next: + last_vma = vma; + } + + if (!sz_range(&second_gap) || !sz_range(&first_gap)) + return -EINVAL; + + /* Sort the two biggest gaps by address */ + if (first_gap.start > second_gap.start) + swap_ranges(&first_gap, &second_gap); + + /* Store the result */ + regions[0].start = ALIGN(start, DAMON_MIN_REGION); + regions[0].end = ALIGN(first_gap.start, DAMON_MIN_REGION); + regions[1].start = ALIGN(first_gap.end, DAMON_MIN_REGION); + regions[1].end = ALIGN(second_gap.start, DAMON_MIN_REGION); + regions[2].start = ALIGN(second_gap.end, DAMON_MIN_REGION); + regions[2].end = ALIGN(last_vma->vm_end, DAMON_MIN_REGION); + + return 0; +} + +/* + * Get the three regions in the given target (task) + * + * Returns 0 on success, negative error code otherwise. + */ +static int damon_va_three_regions(struct damon_target *t, + struct damon_addr_range regions[3]) +{ + struct mm_struct *mm; + int rc; + + mm = damon_get_mm(t); + if (!mm) + return -EINVAL; + + mmap_read_lock(mm); + rc = __damon_va_three_regions(mm->mmap, regions); + mmap_read_unlock(mm); + + mmput(mm); + return rc; +} + +/* + * Initialize the monitoring target regions for the given target (task) + * + * t the given target + * + * Because only a number of small portions of the entire address space + * is actually mapped to the memory and accessed, monitoring the unmapped + * regions is wasteful. That said, because we can deal with small noises, + * tracking every mapping is not strictly required but could even incur a high + * overhead if the mapping frequently changes or the number of mappings is + * high. The adaptive regions adjustment mechanism will further help to deal + * with the noise by simply identifying the unmapped areas as a region that + * has no access. Moreover, applying the real mappings that would have many + * unmapped areas inside will make the adaptive mechanism quite complex. That + * said, too huge unmapped areas inside the monitoring target should be removed + * to not take the time for the adaptive mechanism. + * + * For the reason, we convert the complex mappings to three distinct regions + * that cover every mapped area of the address space. Also the two gaps + * between the three regions are the two biggest unmapped areas in the given + * address space. In detail, this function first identifies the start and the + * end of the mappings and the two biggest unmapped areas of the address space. + * Then, it constructs the three regions as below: + * + * [mappings[0]->start, big_two_unmapped_areas[0]->start) + * [big_two_unmapped_areas[0]->end, big_two_unmapped_areas[1]->start) + * [big_two_unmapped_areas[1]->end, mappings[nr_mappings - 1]->end) + * + * As usual memory map of processes is as below, the gap between the heap and + * the uppermost mmap()-ed region, and the gap between the lowermost mmap()-ed + * region and the stack will be two biggest unmapped regions. Because these + * gaps are exceptionally huge areas in usual address space, excluding these + * two biggest unmapped regions will be sufficient to make a trade-off. + * + * + * + * + * (other mmap()-ed regions and small unmapped regions) + * + * + * + */ +static void __damon_va_init_regions(struct damon_ctx *c, + struct damon_target *t) +{ + struct damon_region *r; + struct damon_addr_range regions[3]; + unsigned long sz = 0, nr_pieces; + int i; + + if (damon_va_three_regions(t, regions)) { + pr_err("Failed to get three regions of target %lu\n", t->id); + return; + } + + for (i = 0; i < 3; i++) + sz += regions[i].end - regions[i].start; + if (c->min_nr_regions) + sz /= c->min_nr_regions; + if (sz < DAMON_MIN_REGION) + sz = DAMON_MIN_REGION; + + /* Set the initial three regions of the target */ + for (i = 0; i < 3; i++) { + r = damon_new_region(regions[i].start, regions[i].end); + if (!r) { + pr_err("%d'th init region creation failed\n", i); + return; + } + damon_add_region(r, t); + + nr_pieces = (regions[i].end - regions[i].start) / sz; + damon_va_evenly_split_region(c, r, nr_pieces); + } +} + +/* Initialize '->regions_list' of every target (task) */ +void damon_va_init(struct damon_ctx *ctx) +{ + struct damon_target *t; + + damon_for_each_target(t, ctx) { + /* the user may set the target regions as they want */ + if (!damon_nr_regions(t)) + __damon_va_init_regions(ctx, t); + } +} + +/* + * Functions for the dynamic monitoring target regions update + */ + +/* + * Check whether a region is intersecting an address range + * + * Returns true if it is. + */ +static bool damon_intersect(struct damon_region *r, struct damon_addr_range *re) +{ + return !(r->ar.end <= re->start || re->end <= r->ar.start); +} + +/* + * Update damon regions for the three big regions of the given target + * + * t the given target + * bregions the three big regions of the target + */ +static void damon_va_apply_three_regions(struct damon_ctx *ctx, + struct damon_target *t, struct damon_addr_range bregions[3]) +{ + struct damon_region *r, *next; + unsigned int i = 0; + + /* Remove regions which are not in the three big regions now */ + damon_for_each_region_safe(r, next, t) { + for (i = 0; i < 3; i++) { + if (damon_intersect(r, &bregions[i])) + break; + } + if (i == 3) + damon_destroy_region(r); + } + + /* Adjust intersecting regions to fit with the three big regions */ + for (i = 0; i < 3; i++) { + struct damon_region *first = NULL, *last; + struct damon_region *newr; + struct damon_addr_range *br; + + br = &bregions[i]; + /* Get the first and last regions which intersects with br */ + damon_for_each_region(r, t) { + if (damon_intersect(r, br)) { + if (!first) + first = r; + last = r; + } + if (r->ar.start >= br->end) + break; + } + if (!first) { + /* no damon_region intersects with this big region */ + newr = damon_new_region( + ALIGN_DOWN(br->start, + DAMON_MIN_REGION), + ALIGN(br->end, DAMON_MIN_REGION)); + if (!newr) + continue; + damon_insert_region(newr, damon_prev_region(r), r); + } else { + first->ar.start = ALIGN_DOWN(br->start, + DAMON_MIN_REGION); + last->ar.end = ALIGN(br->end, DAMON_MIN_REGION); + } + } +} + +/* + * Update regions for current memory mappings + */ +void damon_va_update(struct damon_ctx *ctx) +{ + struct damon_addr_range three_regions[3]; + struct damon_target *t; + + damon_for_each_target(t, ctx) { + if (damon_va_three_regions(t, three_regions)) + continue; + damon_va_apply_three_regions(ctx, t, three_regions); + } +} + +static void damon_ptep_mkold(pte_t *pte, struct mm_struct *mm, + unsigned long addr) +{ + bool referenced = false; + struct page *page = pte_page(*pte); + + if (pte_young(*pte)) { + referenced = true; + *pte = pte_mkold(*pte); + } + +#ifdef CONFIG_MMU_NOTIFIER + if (mmu_notifier_clear_young(mm, addr, addr + PAGE_SIZE)) + referenced = true; +#endif /* CONFIG_MMU_NOTIFIER */ + + if (referenced) + set_page_young(page); + + set_page_idle(page); +} + +static void damon_pmdp_mkold(pmd_t *pmd, struct mm_struct *mm, + unsigned long addr) +{ +#ifdef CONFIG_TRANSPARENT_HUGEPAGE + bool referenced = false; + struct page *page = pmd_page(*pmd); + + if (pmd_young(*pmd)) { + referenced = true; + *pmd = pmd_mkold(*pmd); + } + +#ifdef CONFIG_MMU_NOTIFIER + if (mmu_notifier_clear_young(mm, addr, + addr + ((1UL) << HPAGE_PMD_SHIFT))) + referenced = true; +#endif /* CONFIG_MMU_NOTIFIER */ + + if (referenced) + set_page_young(page); + + set_page_idle(page); +#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ +} + +static void damon_va_mkold(struct mm_struct *mm, unsigned long addr) +{ + pte_t *pte = NULL; + pmd_t *pmd = NULL; + spinlock_t *ptl; + + if (follow_invalidate_pte(mm, addr, NULL, &pte, &pmd, &ptl)) + return; + + if (pte) { + damon_ptep_mkold(pte, mm, addr); + pte_unmap_unlock(pte, ptl); + } else { + damon_pmdp_mkold(pmd, mm, addr); + spin_unlock(ptl); + } +} + +/* + * Functions for the access checking of the regions + */ + +static void damon_va_prepare_access_check(struct damon_ctx *ctx, + struct mm_struct *mm, struct damon_region *r) +{ + r->sampling_addr = damon_rand(r->ar.start, r->ar.end); + + damon_va_mkold(mm, r->sampling_addr); +} + +void damon_va_prepare_access_checks(struct damon_ctx *ctx) +{ + struct damon_target *t; + struct mm_struct *mm; + struct damon_region *r; + + damon_for_each_target(t, ctx) { + mm = damon_get_mm(t); + if (!mm) + continue; + damon_for_each_region(r, t) + damon_va_prepare_access_check(ctx, mm, r); + mmput(mm); + } +} + +static bool damon_va_young(struct mm_struct *mm, unsigned long addr, + unsigned long *page_sz) +{ + pte_t *pte = NULL; + pmd_t *pmd = NULL; + spinlock_t *ptl; + bool young = false; + + if (follow_invalidate_pte(mm, addr, NULL, &pte, &pmd, &ptl)) + return false; + + *page_sz = PAGE_SIZE; + if (pte) { + if (pte_young(*pte) || !page_is_idle(pte_page(*pte)) || + mmu_notifier_test_young(mm, addr)) + young = true; + pte_unmap_unlock(pte, ptl); + return young; + } + +#ifdef CONFIG_TRANSPARENT_HUGEPAGE + if (pmd_young(*pmd) || !page_is_idle(pmd_page(*pmd)) || + mmu_notifier_test_young(mm, addr)) + young = true; + spin_unlock(ptl); + *page_sz = ((1UL) << HPAGE_PMD_SHIFT); +#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ + + return young; +} + +/* + * Check whether the region was accessed after the last preparation + * + * mm 'mm_struct' for the given virtual address space + * r the region to be checked + */ +static void damon_va_check_access(struct damon_ctx *ctx, + struct mm_struct *mm, struct damon_region *r) +{ + static struct mm_struct *last_mm; + static unsigned long last_addr; + static unsigned long last_page_sz = PAGE_SIZE; + static bool last_accessed; + + /* If the region is in the last checked page, reuse the result */ + if (mm == last_mm && (ALIGN_DOWN(last_addr, last_page_sz) == + ALIGN_DOWN(r->sampling_addr, last_page_sz))) { + if (last_accessed) + r->nr_accesses++; + return; + } + + last_accessed = damon_va_young(mm, r->sampling_addr, &last_page_sz); + if (last_accessed) + r->nr_accesses++; + + last_mm = mm; + last_addr = r->sampling_addr; +} + +unsigned int damon_va_check_accesses(struct damon_ctx *ctx) +{ + struct damon_target *t; + struct mm_struct *mm; + struct damon_region *r; + unsigned int max_nr_accesses = 0; + + damon_for_each_target(t, ctx) { + mm = damon_get_mm(t); + if (!mm) + continue; + damon_for_each_region(r, t) { + damon_va_check_access(ctx, mm, r); + max_nr_accesses = max(r->nr_accesses, max_nr_accesses); + } + mmput(mm); + } + + return max_nr_accesses; +} + +/* + * Functions for the target validity check and cleanup + */ + +bool damon_va_target_valid(void *target) +{ + struct damon_target *t = target; + struct task_struct *task; + + task = damon_get_task_struct(t); + if (task) { + put_task_struct(task); + return true; + } + + return false; +} + +void damon_va_cleanup(struct damon_ctx *ctx) +{ + struct damon_target *t, *next; + + damon_for_each_target_safe(t, next, ctx) { + put_pid((struct pid *)t->id); + damon_destroy_target(t); + } +} + +void damon_va_set_primitives(struct damon_ctx *ctx) +{ + ctx->primitive.init = damon_va_init; + ctx->primitive.update = damon_va_update; + ctx->primitive.prepare_access_checks = damon_va_prepare_access_checks; + ctx->primitive.check_accesses = damon_va_check_accesses; + ctx->primitive.reset_aggregated = NULL; + ctx->primitive.target_valid = damon_va_target_valid; + ctx->primitive.cleanup = damon_va_cleanup; +} From patchwork Thu Mar 18 10:08:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 12147799 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BC4D0C433E9 for ; Thu, 18 Mar 2021 10:10:18 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 641DE64E86 for ; Thu, 18 Mar 2021 10:10:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 641DE64E86 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id BB5F46B0074; Thu, 18 Mar 2021 06:10:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B29EA6B007B; Thu, 18 Mar 2021 06:10:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 843AF6B0075; Thu, 18 Mar 2021 06:10:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0214.hostedemail.com [216.40.44.214]) by kanga.kvack.org (Postfix) with ESMTP id 5FCFF6B0074 for ; Thu, 18 Mar 2021 06:10:16 -0400 (EDT) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 1D56A181AF5F7 for ; Thu, 18 Mar 2021 10:10:16 +0000 (UTC) X-FDA: 77932574832.07.B9F65A4 Received: from mail-qk1-f173.google.com (mail-qk1-f173.google.com [209.85.222.173]) by imf17.hostedemail.com (Postfix) with ESMTP id 9CB4A4080F53 for ; Thu, 18 Mar 2021 10:10:13 +0000 (UTC) Received: by mail-qk1-f173.google.com with SMTP id x10so1375640qkm.8 for ; Thu, 18 Mar 2021 03:10:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=CvzI94r9LiC7RH+dtcaRZqda/Y720VadPnKc/47pvNM=; b=PuZxn3uGlJkbrdBQV2EQcDqKqIG7oPBL7MI8XpTQDyVnCy3rTmqhcG04xOIbBbTjsW 75MlqlkVSZAf95wyHFsoDdF/lPPTyO5q+zt5n2TUlvXeOsYnkO5Sv9UogN7yM8f6kzSm Yt2IE057ESuiBBCFFlfptILYA2UL35R2KRz7yukFM+qfeWu0IbGOzJcEdUMrdInCykRp VQAhaPr/eReOJJDXRfgP2BZaEhl+RrEClDSVCCPFIz0/YoI/7CBIrh3A3KPiB1VCzzUx 865HDpfAxvQCHNiwfeTTLrgfuTScQYL44CS2svog0qt55gs7pmClneKcyFJjVGG7OIRH yCEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=CvzI94r9LiC7RH+dtcaRZqda/Y720VadPnKc/47pvNM=; b=kX4MakYjGzeCbVVk2rqICkJBx/l2kcbgmbJVauov2Dbn63uiqDD5/VpthWaZ067RTA MZ3ZduMywiNhvzWk/yiP+QUI39CzxhYr53q4GEXk9qVQif7uBLNjAcbG0I4WOtwtBcqq MDdS6XTlutvwI39Epgd0szAPlh8qqJB3dAmKGrBDKl2arOfV4vRW+PRDZ9y8OdVHFKmP FrG9H0ZGysMKEfCxEccLP66/O6cAE/8Qat+4PqJ3oCd4ESUHOMPntQIMkk2FsTDsjycD 41/A9vjv6MgXxqER3cxBYgh1uK4UTSnW9lj2TflHzMpDuJYNptj/80l+Tiu9otiZu9qu RNqg== X-Gm-Message-State: AOAM532E3hB0ud47sDAVsRqY3bT+HbhUTfecCVoJO/hHSLsIpPbNiBEY FmuOLTzSDISnArsCHYC5Yj8= X-Google-Smtp-Source: ABdhPJylMfqaO+j8UlaKLOKmMbko+Iu5w9Xv27Gv++d3men301nRra8UmlocjJauixsWVU2kFuMiIA== X-Received: by 2002:a37:270c:: with SMTP id n12mr3496348qkn.277.1616062213034; Thu, 18 Mar 2021 03:10:13 -0700 (PDT) Received: from localhost.localdomain (ec2-35-169-212-159.compute-1.amazonaws.com. [35.169.212.159]) by smtp.gmail.com with ESMTPSA id d16sm1358101qka.43.2021.03.18.03.10.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 18 Mar 2021 03:10:12 -0700 (PDT) From: sj38.park@gmail.com To: akpm@linux-foundation.org Cc: SeongJae Park , Jonathan.Cameron@Huawei.com, acme@kernel.org, alexander.shishkin@linux.intel.com, amit@kernel.org, benh@kernel.crashing.org, brendanhiggins@google.com, corbet@lwn.net, david@redhat.com, dwmw@amazon.com, elver@google.com, fan.du@intel.com, foersleo@amazon.de, gthelen@google.com, mgorman@suse.de, minchan@kernel.org, mingo@redhat.com, namhyung@kernel.org, peterz@infradead.org, riel@surriel.com, rientjes@google.com, rostedt@goodmis.org, rppt@kernel.org, shakeelb@google.com, shuah@kernel.org, sj38.park@gmail.com, snu@amazon.de, vbabka@suse.cz, vdavydov.dev@gmail.com, zgf574564920@gmail.com, linux-damon@amazon.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v25 06/13] mm/damon: Add a tracepoint Date: Thu, 18 Mar 2021 10:08:49 +0000 Message-Id: <20210318100856.34715-7-sj38.park@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210318100856.34715-1-sj38.park@gmail.com> References: <20210318100856.34715-1-sj38.park@gmail.com> X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 9CB4A4080F53 X-Stat-Signature: ofetz4m67akndib8ysr59s5bexeobs6n Received-SPF: none (gmail.com>: No applicable sender policy available) receiver=imf17; identity=mailfrom; envelope-from=""; helo=mail-qk1-f173.google.com; client-ip=209.85.222.173 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1616062213-155438 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park This commit adds a tracepoint for DAMON. It traces the monitoring results of each region for each aggregation interval. Using this, DAMON can easily integrated with tracepoints supporting tools such as perf. Signed-off-by: SeongJae Park Reviewed-by: Leonard Foerster Reviewed-by: Steven Rostedt (VMware) --- include/trace/events/damon.h | 43 ++++++++++++++++++++++++++++++++++++ mm/damon/core.c | 7 +++++- 2 files changed, 49 insertions(+), 1 deletion(-) create mode 100644 include/trace/events/damon.h diff --git a/include/trace/events/damon.h b/include/trace/events/damon.h new file mode 100644 index 000000000000..2f422f4f1fb9 --- /dev/null +++ b/include/trace/events/damon.h @@ -0,0 +1,43 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#undef TRACE_SYSTEM +#define TRACE_SYSTEM damon + +#if !defined(_TRACE_DAMON_H) || defined(TRACE_HEADER_MULTI_READ) +#define _TRACE_DAMON_H + +#include +#include +#include + +TRACE_EVENT(damon_aggregated, + + TP_PROTO(struct damon_target *t, struct damon_region *r, + unsigned int nr_regions), + + TP_ARGS(t, r, nr_regions), + + TP_STRUCT__entry( + __field(unsigned long, target_id) + __field(unsigned int, nr_regions) + __field(unsigned long, start) + __field(unsigned long, end) + __field(unsigned int, nr_accesses) + ), + + TP_fast_assign( + __entry->target_id = t->id; + __entry->nr_regions = nr_regions; + __entry->start = r->ar.start; + __entry->end = r->ar.end; + __entry->nr_accesses = r->nr_accesses; + ), + + TP_printk("target_id=%lu nr_regions=%u %lu-%lu: %u", + __entry->target_id, __entry->nr_regions, + __entry->start, __entry->end, __entry->nr_accesses) +); + +#endif /* _TRACE_DAMON_H */ + +/* This part must be outside protection */ +#include diff --git a/mm/damon/core.c b/mm/damon/core.c index b36b6bdd94e2..912112662d0c 100644 --- a/mm/damon/core.c +++ b/mm/damon/core.c @@ -13,6 +13,9 @@ #include #include +#define CREATE_TRACE_POINTS +#include + /* Get a random number in [l, r) */ #define damon_rand(l, r) (l + prandom_u32_max(r - l)) @@ -388,8 +391,10 @@ static void kdamond_reset_aggregated(struct damon_ctx *c) damon_for_each_target(t, c) { struct damon_region *r; - damon_for_each_region(r, t) + damon_for_each_region(r, t) { + trace_damon_aggregated(t, r, damon_nr_regions(t)); r->nr_accesses = 0; + } } } From patchwork Thu Mar 18 10:08:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 12147805 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5AC02C433E0 for ; Thu, 18 Mar 2021 10:10:25 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id DD46C61580 for ; Thu, 18 Mar 2021 10:10:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DD46C61580 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D789D8D0003; Thu, 18 Mar 2021 06:10:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D00728D0001; Thu, 18 Mar 2021 06:10:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B076E8D0003; Thu, 18 Mar 2021 06:10:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0208.hostedemail.com [216.40.44.208]) by kanga.kvack.org (Postfix) with ESMTP id 8D2A28D0001 for ; Thu, 18 Mar 2021 06:10:21 -0400 (EDT) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 42AAD781C for ; Thu, 18 Mar 2021 10:10:21 +0000 (UTC) X-FDA: 77932575042.13.BECE9B5 Received: from mail-qk1-f175.google.com (mail-qk1-f175.google.com [209.85.222.175]) by imf16.hostedemail.com (Postfix) with ESMTP id 245D4801916F for ; Thu, 18 Mar 2021 10:10:15 +0000 (UTC) Received: by mail-qk1-f175.google.com with SMTP id l132so1377020qke.7 for ; Thu, 18 Mar 2021 03:10:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=kTo/UfhuYk++fPyz4VZwde5Bvi+8OSxfEXLHIf/dOYk=; b=T2f1wqu/63TJLOQcpLGGdfFNDlcgxLfjCfWREavUGR6dd+WuutsaRjwVme4HY1RSWM 41XOG5ZbVsT+ns1TW5HDCzD66S7QfWZUnfxvHIVrqxeAOXud0BpI1z/iqPzqx/8bz0C0 YyUrAYmJgotbYhyzl4JT3I2WDRE7UKZJ6Kmpa1sms9cNIQWZhBM2FxnEHmgEka8vh3ZK Vsbn1XlYQ2o1mo39mW/X9V+q1RmTKVICt8hDI/yJOMXfX29GoBD8mu9bwNou0xFrG+pC LblsXwGPA6mBWON7hal2NwosQjCiYqRF7Q61YaXKlTjjLQZM/NLZYGNBtDCzPBanMa6P 04dQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=kTo/UfhuYk++fPyz4VZwde5Bvi+8OSxfEXLHIf/dOYk=; b=uWz7g85oH9jzLWi/RXprZMs3ESRTcfqDGlT9IST5yZ9bHmUrJ3G70j2y6gXINMSFzm Kp5eDv6HVfjV1J6U6LfEP0IsvvdJjJtfuIJVT6v8MnI04D2no5BxL3l16OKKex6sHYcu pUPCNCqIgjdyAB2zRSmj/9P3j7jck6lc+xHbrqGzA99VW2R2BuehKCZX6sUg9uqSj0Kq XPRUWHk7rra8d8XAYUblIgiBptoDx0fGjkPM9JILrunj4idLtUjB1NvJB6QAXnJtAyMF 9QYfQr3iIOdoqf6NevkX4bQwC2PWDLXhExUtaPBJ0CCMAoJV84mDXiuLPYevtN0UomSk n9TQ== X-Gm-Message-State: AOAM533bm7Sflm8VZ7CF5GfTZUxJyS3EhgoYifYFMIsuwV/irUZlAl+V pRh3fvmGgS9frERLU6B7uD8= X-Google-Smtp-Source: ABdhPJyYLY6oJv/Wr+KuXP24sy9Hg0dIGx6sE2C9G5G1DY1qeJCT8Od56aQle88lTQfHHGSe/ykoeQ== X-Received: by 2002:a05:620a:5b3:: with SMTP id q19mr3561258qkq.98.1616062215365; Thu, 18 Mar 2021 03:10:15 -0700 (PDT) Received: from localhost.localdomain (ec2-35-169-212-159.compute-1.amazonaws.com. [35.169.212.159]) by smtp.gmail.com with ESMTPSA id d16sm1358101qka.43.2021.03.18.03.10.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 18 Mar 2021 03:10:14 -0700 (PDT) From: sj38.park@gmail.com To: akpm@linux-foundation.org Cc: SeongJae Park , Jonathan.Cameron@Huawei.com, acme@kernel.org, alexander.shishkin@linux.intel.com, amit@kernel.org, benh@kernel.crashing.org, brendanhiggins@google.com, corbet@lwn.net, david@redhat.com, dwmw@amazon.com, elver@google.com, fan.du@intel.com, foersleo@amazon.de, gthelen@google.com, mgorman@suse.de, minchan@kernel.org, mingo@redhat.com, namhyung@kernel.org, peterz@infradead.org, riel@surriel.com, rientjes@google.com, rostedt@goodmis.org, rppt@kernel.org, shakeelb@google.com, shuah@kernel.org, sj38.park@gmail.com, snu@amazon.de, vbabka@suse.cz, vdavydov.dev@gmail.com, zgf574564920@gmail.com, linux-damon@amazon.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v25 07/13] mm/damon: Implement a debugfs-based user space interface Date: Thu, 18 Mar 2021 10:08:50 +0000 Message-Id: <20210318100856.34715-8-sj38.park@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210318100856.34715-1-sj38.park@gmail.com> References: <20210318100856.34715-1-sj38.park@gmail.com> X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 245D4801916F X-Stat-Signature: 8o1xtp6ph4whsntx1wiukxzj1qm8nz9w Received-SPF: none (gmail.com>: No applicable sender policy available) receiver=imf16; identity=mailfrom; envelope-from=""; helo=mail-qk1-f175.google.com; client-ip=209.85.222.175 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1616062215-82081 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park DAMON is designed to be used by kernel space code such as the memory management subsystems, and therefore it provides only kernel space API. That said, letting the user space control DAMON could provide some benefits to them. For example, it will allow user space to analyze their specific workloads and make their own special optimizations. For such cases, this commit implements a simple DAMON application kernel module, namely 'damon-dbgfs', which merely wraps the DAMON api and exports those to the user space via the debugfs. 'damon-dbgfs' exports three files, ``attrs``, ``target_ids``, and ``monitor_on`` under its debugfs directory, ``/damon/``. Attributes ---------- Users can read and write the ``sampling interval``, ``aggregation interval``, ``regions update interval``, and min/max number of monitoring target regions by reading from and writing to the ``attrs`` file. For example, below commands set those values to 5 ms, 100 ms, 1,000 ms, 10, 1000 and check it again:: # cd /damon # echo 5000 100000 1000000 10 1000 > attrs # cat attrs 5000 100000 1000000 10 1000 Target IDs ---------- Some types of address spaces supports multiple monitoring target. For example, the virtual memory address spaces monitoring can have multiple processes as the monitoring targets. Users can set the targets by writing relevant id values of the targets to, and get the ids of the current targets by reading from the ``target_ids`` file. In case of the virtual address spaces monitoring, the values should be pids of the monitoring target processes. For example, below commands set processes having pids 42 and 4242 as the monitoring targets and check it again:: # cd /damon # echo 42 4242 > target_ids # cat target_ids 42 4242 Note that setting the target ids doesn't start the monitoring. Turning On/Off -------------- Setting the files as described above doesn't incur effect unless you explicitly start the monitoring. You can start, stop, and check the current status of the monitoring by writing to and reading from the ``monitor_on`` file. Writing ``on`` to the file starts the monitoring of the targets with the attributes. Writing ``off`` to the file stops those. DAMON also stops if every targets are invalidated (in case of the virtual memory monitoring, target processes are invalidated when terminated). Below example commands turn on, off, and check the status of DAMON:: # cd /damon # echo on > monitor_on # echo off > monitor_on # cat monitor_on off Please note that you cannot write to the above-mentioned debugfs files while the monitoring is turned on. If you write to the files while DAMON is running, an error code such as ``-EBUSY`` will be returned. Signed-off-by: SeongJae Park Reviewed-by: Leonard Foerster --- include/linux/damon.h | 3 + mm/damon/Kconfig | 9 + mm/damon/Makefile | 1 + mm/damon/core.c | 47 ++++++ mm/damon/dbgfs.c | 374 ++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 434 insertions(+) create mode 100644 mm/damon/dbgfs.c diff --git a/include/linux/damon.h b/include/linux/damon.h index 72cf5ebd35fe..b17e808a9cae 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -237,9 +237,12 @@ unsigned int damon_nr_regions(struct damon_target *t); struct damon_ctx *damon_new_ctx(void); void damon_destroy_ctx(struct damon_ctx *ctx); +int damon_set_targets(struct damon_ctx *ctx, + unsigned long *ids, ssize_t nr_ids); int damon_set_attrs(struct damon_ctx *ctx, unsigned long sample_int, unsigned long aggr_int, unsigned long primitive_upd_int, unsigned long min_nr_reg, unsigned long max_nr_reg); +int damon_nr_running_ctxs(void); int damon_start(struct damon_ctx **ctxs, int nr_ctxs); int damon_stop(struct damon_ctx **ctxs, int nr_ctxs); diff --git a/mm/damon/Kconfig b/mm/damon/Kconfig index 8ae080c52950..72f1683ba0ee 100644 --- a/mm/damon/Kconfig +++ b/mm/damon/Kconfig @@ -21,4 +21,13 @@ config DAMON_VADDR This builds the default data access monitoring primitives for DAMON that works for virtual address spaces. +config DAMON_DBGFS + bool "DAMON debugfs interface" + depends on DAMON_VADDR && DEBUG_FS + help + This builds the debugfs interface for DAMON. The user space admins + can use the interface for arbitrary data access monitoring. + + If unsure, say N. + endmenu diff --git a/mm/damon/Makefile b/mm/damon/Makefile index 6ebbd08aed67..fed4be3bace3 100644 --- a/mm/damon/Makefile +++ b/mm/damon/Makefile @@ -2,3 +2,4 @@ obj-$(CONFIG_DAMON) := core.o obj-$(CONFIG_DAMON_VADDR) += vaddr.o +obj-$(CONFIG_DAMON_DBGFS) += dbgfs.o diff --git a/mm/damon/core.c b/mm/damon/core.c index 912112662d0c..cad2b4cee39d 100644 --- a/mm/damon/core.c +++ b/mm/damon/core.c @@ -172,6 +172,39 @@ void damon_destroy_ctx(struct damon_ctx *ctx) kfree(ctx); } +/** + * damon_set_targets() - Set monitoring targets. + * @ctx: monitoring context + * @ids: array of target ids + * @nr_ids: number of entries in @ids + * + * This function should not be called while the kdamond is running. + * + * Return: 0 on success, negative error code otherwise. + */ +int damon_set_targets(struct damon_ctx *ctx, + unsigned long *ids, ssize_t nr_ids) +{ + ssize_t i; + struct damon_target *t, *next; + + damon_destroy_targets(ctx); + + for (i = 0; i < nr_ids; i++) { + t = damon_new_target(ids[i]); + if (!t) { + pr_err("Failed to alloc damon_target\n"); + /* The caller should do cleanup of the ids itself */ + damon_for_each_target_safe(t, next, ctx) + damon_destroy_target(t); + return -ENOMEM; + } + damon_add_target(ctx, t); + } + + return 0; +} + /** * damon_set_attrs() - Set attributes for the monitoring. * @ctx: monitoring context @@ -210,6 +243,20 @@ int damon_set_attrs(struct damon_ctx *ctx, unsigned long sample_int, return 0; } +/** + * damon_nr_running_ctxs() - Return number of currently running contexts. + */ +int damon_nr_running_ctxs(void) +{ + int nr_ctxs; + + mutex_lock(&damon_lock); + nr_ctxs = nr_running_ctxs; + mutex_unlock(&damon_lock); + + return nr_ctxs; +} + /* Returns the size upper limit for each monitoring region */ static unsigned long damon_region_sz_limit(struct damon_ctx *ctx) { diff --git a/mm/damon/dbgfs.c b/mm/damon/dbgfs.c new file mode 100644 index 000000000000..02b27be9187a --- /dev/null +++ b/mm/damon/dbgfs.c @@ -0,0 +1,374 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * DAMON Debugfs Interface + * + * Author: SeongJae Park + */ + +#define pr_fmt(fmt) "damon-dbgfs: " fmt + +#include +#include +#include +#include +#include +#include +#include + +static struct damon_ctx **dbgfs_ctxs; +static int dbgfs_nr_ctxs; +static struct dentry **dbgfs_dirs; + +/* + * Returns non-empty string on success, negarive error code otherwise. + */ +static char *user_input_str(const char __user *buf, size_t count, loff_t *ppos) +{ + char *kbuf; + ssize_t ret; + + /* We do not accept continuous write */ + if (*ppos) + return ERR_PTR(-EINVAL); + + kbuf = kmalloc(count + 1, GFP_KERNEL); + if (!kbuf) + return ERR_PTR(-ENOMEM); + + ret = simple_write_to_buffer(kbuf, count + 1, ppos, buf, count); + if (ret != count) { + kfree(kbuf); + return ERR_PTR(-EIO); + } + kbuf[ret] = '\0'; + + return kbuf; +} + +static ssize_t dbgfs_attrs_read(struct file *file, + char __user *buf, size_t count, loff_t *ppos) +{ + struct damon_ctx *ctx = file->private_data; + char kbuf[128]; + int ret; + + mutex_lock(&ctx->kdamond_lock); + ret = scnprintf(kbuf, ARRAY_SIZE(kbuf), "%lu %lu %lu %lu %lu\n", + ctx->sample_interval, ctx->aggr_interval, + ctx->primitive_update_interval, ctx->min_nr_regions, + ctx->max_nr_regions); + mutex_unlock(&ctx->kdamond_lock); + + return simple_read_from_buffer(buf, count, ppos, kbuf, ret); +} + +static ssize_t dbgfs_attrs_write(struct file *file, + const char __user *buf, size_t count, loff_t *ppos) +{ + struct damon_ctx *ctx = file->private_data; + unsigned long s, a, r, minr, maxr; + char *kbuf; + ssize_t ret = count; + int err; + + kbuf = user_input_str(buf, count, ppos); + if (IS_ERR(kbuf)) + return PTR_ERR(kbuf); + + if (sscanf(kbuf, "%lu %lu %lu %lu %lu", + &s, &a, &r, &minr, &maxr) != 5) { + ret = -EINVAL; + goto out; + } + + mutex_lock(&ctx->kdamond_lock); + if (ctx->kdamond) { + ret = -EBUSY; + goto unlock_out; + } + + err = damon_set_attrs(ctx, s, a, r, minr, maxr); + if (err) + ret = err; +unlock_out: + mutex_unlock(&ctx->kdamond_lock); +out: + kfree(kbuf); + return ret; +} + +#define targetid_is_pid(ctx) \ + (ctx->primitive.target_valid == damon_va_target_valid) + +static ssize_t sprint_target_ids(struct damon_ctx *ctx, char *buf, ssize_t len) +{ + struct damon_target *t; + unsigned long id; + int written = 0; + int rc; + + damon_for_each_target(t, ctx) { + id = t->id; + if (targetid_is_pid(ctx)) + /* Show pid numbers to debugfs users */ + id = (unsigned long)pid_vnr((struct pid *)id); + + rc = scnprintf(&buf[written], len - written, "%lu ", id); + if (!rc) + return -ENOMEM; + written += rc; + } + if (written) + written -= 1; + written += scnprintf(&buf[written], len - written, "\n"); + return written; +} + +static ssize_t dbgfs_target_ids_read(struct file *file, + char __user *buf, size_t count, loff_t *ppos) +{ + struct damon_ctx *ctx = file->private_data; + ssize_t len; + char ids_buf[320]; + + mutex_lock(&ctx->kdamond_lock); + len = sprint_target_ids(ctx, ids_buf, 320); + mutex_unlock(&ctx->kdamond_lock); + if (len < 0) + return len; + + return simple_read_from_buffer(buf, count, ppos, ids_buf, len); +} + +/* + * Converts a string into an array of unsigned long integers + * + * Returns an array of unsigned long integers if the conversion success, or + * NULL otherwise. + */ +static unsigned long *str_to_target_ids(const char *str, ssize_t len, + ssize_t *nr_ids) +{ + unsigned long *ids; + const int max_nr_ids = 32; + unsigned long id; + int pos = 0, parsed, ret; + + *nr_ids = 0; + ids = kmalloc_array(max_nr_ids, sizeof(id), GFP_KERNEL); + if (!ids) + return NULL; + while (*nr_ids < max_nr_ids && pos < len) { + ret = sscanf(&str[pos], "%lu%n", &id, &parsed); + pos += parsed; + if (ret != 1) + break; + ids[*nr_ids] = id; + *nr_ids += 1; + } + + return ids; +} + +static void dbgfs_put_pids(unsigned long *ids, int nr_ids) +{ + int i; + + for (i = 0; i < nr_ids; i++) + put_pid((struct pid *)ids[i]); +} + +static ssize_t dbgfs_target_ids_write(struct file *file, + const char __user *buf, size_t count, loff_t *ppos) +{ + struct damon_ctx *ctx = file->private_data; + char *kbuf, *nrs; + unsigned long *targets; + ssize_t nr_targets; + ssize_t ret = count; + int i; + int err; + + kbuf = user_input_str(buf, count, ppos); + if (IS_ERR(kbuf)) + return PTR_ERR(kbuf); + + nrs = kbuf; + + targets = str_to_target_ids(nrs, ret, &nr_targets); + if (!targets) { + ret = -ENOMEM; + goto out; + } + + if (targetid_is_pid(ctx)) { + for (i = 0; i < nr_targets; i++) { + targets[i] = (unsigned long)find_get_pid( + (int)targets[i]); + if (!targets[i]) { + dbgfs_put_pids(targets, i); + ret = -EINVAL; + goto free_targets_out; + } + } + } + + mutex_lock(&ctx->kdamond_lock); + if (ctx->kdamond) { + if (targetid_is_pid(ctx)) + dbgfs_put_pids(targets, nr_targets); + ret = -EBUSY; + goto unlock_out; + } + + err = damon_set_targets(ctx, targets, nr_targets); + if (err) { + if (targetid_is_pid(ctx)) + dbgfs_put_pids(targets, nr_targets); + ret = err; + } + +unlock_out: + mutex_unlock(&ctx->kdamond_lock); +free_targets_out: + kfree(targets); +out: + kfree(kbuf); + return ret; +} + +static int damon_dbgfs_open(struct inode *inode, struct file *file) +{ + file->private_data = inode->i_private; + + return nonseekable_open(inode, file); +} + +static const struct file_operations attrs_fops = { + .owner = THIS_MODULE, + .open = damon_dbgfs_open, + .read = dbgfs_attrs_read, + .write = dbgfs_attrs_write, +}; + +static const struct file_operations target_ids_fops = { + .owner = THIS_MODULE, + .open = damon_dbgfs_open, + .read = dbgfs_target_ids_read, + .write = dbgfs_target_ids_write, +}; + +static int dbgfs_fill_ctx_dir(struct dentry *dir, struct damon_ctx *ctx) +{ + const char * const file_names[] = {"attrs", "target_ids"}; + const struct file_operations *fops[] = {&attrs_fops, &target_ids_fops}; + int i; + + for (i = 0; i < ARRAY_SIZE(file_names); i++) + debugfs_create_file(file_names[i], 0600, dir, ctx, fops[i]); + + return 0; +} + +static struct damon_ctx *dbgfs_new_ctx(void) +{ + struct damon_ctx *ctx; + + ctx = damon_new_ctx(); + if (!ctx) + return NULL; + + damon_va_set_primitives(ctx); + return ctx; +} + +static ssize_t dbgfs_monitor_on_read(struct file *file, + char __user *buf, size_t count, loff_t *ppos) +{ + char monitor_on_buf[5]; + bool monitor_on = damon_nr_running_ctxs() != 0; + int len; + + len = scnprintf(monitor_on_buf, 5, monitor_on ? "on\n" : "off\n"); + + return simple_read_from_buffer(buf, count, ppos, monitor_on_buf, len); +} + +static ssize_t dbgfs_monitor_on_write(struct file *file, + const char __user *buf, size_t count, loff_t *ppos) +{ + ssize_t ret = count; + char *kbuf; + int err; + + kbuf = user_input_str(buf, count, ppos); + if (IS_ERR(kbuf)) + return PTR_ERR(kbuf); + + /* Remove white space */ + if (sscanf(kbuf, "%s", kbuf) != 1) { + kfree(kbuf); + return -EINVAL; + } + + if (!strncmp(kbuf, "on", count)) + err = damon_start(dbgfs_ctxs, dbgfs_nr_ctxs); + else if (!strncmp(kbuf, "off", count)) + err = damon_stop(dbgfs_ctxs, dbgfs_nr_ctxs); + else + err = -EINVAL; + + if (err) + ret = err; + kfree(kbuf); + return ret; +} + +static const struct file_operations monitor_on_fops = { + .owner = THIS_MODULE, + .read = dbgfs_monitor_on_read, + .write = dbgfs_monitor_on_write, +}; + +static int __init __damon_dbgfs_init(void) +{ + struct dentry *dbgfs_root; + const char * const file_names[] = {"monitor_on"}; + const struct file_operations *fops[] = {&monitor_on_fops}; + int i; + + dbgfs_root = debugfs_create_dir("damon", NULL); + + for (i = 0; i < ARRAY_SIZE(file_names); i++) + debugfs_create_file(file_names[i], 0600, dbgfs_root, NULL, + fops[i]); + dbgfs_fill_ctx_dir(dbgfs_root, dbgfs_ctxs[0]); + + dbgfs_dirs = kmalloc_array(1, sizeof(dbgfs_root), GFP_KERNEL); + dbgfs_dirs[0] = dbgfs_root; + + return 0; +} + +/* + * Functions for the initialization + */ + +static int __init damon_dbgfs_init(void) +{ + int rc; + + dbgfs_ctxs = kmalloc(sizeof(*dbgfs_ctxs), GFP_KERNEL); + dbgfs_ctxs[0] = dbgfs_new_ctx(); + if (!dbgfs_ctxs[0]) + return -ENOMEM; + dbgfs_nr_ctxs = 1; + + rc = __damon_dbgfs_init(); + if (rc) + pr_err("%s: dbgfs init failed\n", __func__); + + return rc; +} + +module_init(damon_dbgfs_init); From patchwork Thu Mar 18 10:08:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 12147803 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E20C7C433DB for ; Thu, 18 Mar 2021 10:10:22 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7793564E86 for ; Thu, 18 Mar 2021 10:10:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7793564E86 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2061F8D0002; Thu, 18 Mar 2021 06:10:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1B7578D0001; Thu, 18 Mar 2021 06:10:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EAF328D0002; Thu, 18 Mar 2021 06:10:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0150.hostedemail.com [216.40.44.150]) by kanga.kvack.org (Postfix) with ESMTP id CA45E8D0001 for ; Thu, 18 Mar 2021 06:10:18 -0400 (EDT) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 82C88781C for ; Thu, 18 Mar 2021 10:10:18 +0000 (UTC) X-FDA: 77932574916.16.5CF9CB3 Received: from mail-qk1-f179.google.com (mail-qk1-f179.google.com [209.85.222.179]) by imf24.hostedemail.com (Postfix) with ESMTP id 0F818A0009FE for ; Thu, 18 Mar 2021 10:10:17 +0000 (UTC) Received: by mail-qk1-f179.google.com with SMTP id l4so1417953qkl.0 for ; Thu, 18 Mar 2021 03:10:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=W8+A/g3U4DodIE6llQ+hDjB/HFXop9Ja1bxfrVH/yAg=; b=kT1pBhR3JRIX8rnvf+m/6UkT8V+sxoRM6EP4NLs6YuBOiFd1REtu+fiP9/rVZIT/iR kAqBc+1iKTBqAkqcPRLAaijgDp9PcL+0+KecKQbUQbbMupn7DlgrSM6yDZdDd5E8S7fi OnGeY6vy2K1eyPKbK7XdRhXKLIq0HvkInXI2QoSboDnYGMIyBt1D07nkocIyJL9l96ND yhfO6wbHxXKxxThPSYVVQBv6hCHiigw9TW6J5F8T9LT+j4VhG7fUvhTfEzoe/aVrV+uh VQcO/UUAYWhdYnjAnErE9bUv2O7QBl+2/nHOhRj57QV65RUChXIcVsyjg7jjSgdoBtLr nHug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=W8+A/g3U4DodIE6llQ+hDjB/HFXop9Ja1bxfrVH/yAg=; b=po7fPWLqzj9QnywqJvqOjZJDX4pmq8G7Y9wvqFzguCaGO+swMcqHdp3uVajm4SD4Rl 6HH2878iBVVlMRMQ1TM56rOT49Pc+AklZ33OLaZct8gkAZ/RhTnzs4MNWItj+Qmd03+V CJHTkh8iHLAH0Q1Dyes3+RLAMHorxjaaCDjeT1MhO3kVwUQHdYm4XXV3q53LrXssc73j CIY2AK9hF4TI5O78Lv58yIgx8vGMDZVhNV9wMNhxSXBUnPWS+y0mhXo1tWALYgoQwW5h DIK/Cd72cjJ5/oE+3C31+AJ13liDq1wN/1X7T6vhPTlNAogLPrbQiDB6y+DmxEAKp7y0 3kZA== X-Gm-Message-State: AOAM531m0QLvAZy3dVUQdKurGVaMVJZmnkQ5MKesbWBoefFO+nkP0vkW eYQql2yXl90Hixlb0wHnQJs= X-Google-Smtp-Source: ABdhPJx9r52cWcfoEbyqWv9MA5ztg2rt1UpinGbFCS45+PeKAjitAAjlvY+RVJdVuNlf5TFvPYHD6Q== X-Received: by 2002:a37:a715:: with SMTP id q21mr3363655qke.309.1616062217486; Thu, 18 Mar 2021 03:10:17 -0700 (PDT) Received: from localhost.localdomain (ec2-35-169-212-159.compute-1.amazonaws.com. [35.169.212.159]) by smtp.gmail.com with ESMTPSA id d16sm1358101qka.43.2021.03.18.03.10.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 18 Mar 2021 03:10:17 -0700 (PDT) From: sj38.park@gmail.com To: akpm@linux-foundation.org Cc: SeongJae Park , Jonathan.Cameron@Huawei.com, acme@kernel.org, alexander.shishkin@linux.intel.com, amit@kernel.org, benh@kernel.crashing.org, brendanhiggins@google.com, corbet@lwn.net, david@redhat.com, dwmw@amazon.com, elver@google.com, fan.du@intel.com, foersleo@amazon.de, gthelen@google.com, mgorman@suse.de, minchan@kernel.org, mingo@redhat.com, namhyung@kernel.org, peterz@infradead.org, riel@surriel.com, rientjes@google.com, rostedt@goodmis.org, rppt@kernel.org, shakeelb@google.com, shuah@kernel.org, sj38.park@gmail.com, snu@amazon.de, vbabka@suse.cz, vdavydov.dev@gmail.com, zgf574564920@gmail.com, linux-damon@amazon.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v25 08/13] mm/damon/dbgfs: Export kdamond pid to the user space Date: Thu, 18 Mar 2021 10:08:51 +0000 Message-Id: <20210318100856.34715-9-sj38.park@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210318100856.34715-1-sj38.park@gmail.com> References: <20210318100856.34715-1-sj38.park@gmail.com> X-Stat-Signature: yemn6xqb8mn3kwp5yr6mej7aefdzzuhr X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 0F818A0009FE Received-SPF: none (gmail.com>: No applicable sender policy available) receiver=imf24; identity=mailfrom; envelope-from=""; helo=mail-qk1-f179.google.com; client-ip=209.85.222.179 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1616062217-334117 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park For CPU usage accounting, knowing pid of the monitoring thread could be helpful. For example, users could use cpuaccount cgroups with the pid. This commit therefore exports the pid of currently running monitoring thread to the user space via 'kdamond_pid' file in the debugfs directory. Signed-off-by: SeongJae Park --- mm/damon/dbgfs.c | 38 ++++++++++++++++++++++++++++++++++++-- 1 file changed, 36 insertions(+), 2 deletions(-) diff --git a/mm/damon/dbgfs.c b/mm/damon/dbgfs.c index 02b27be9187a..aec2b7d81809 100644 --- a/mm/damon/dbgfs.c +++ b/mm/damon/dbgfs.c @@ -237,6 +237,32 @@ static ssize_t dbgfs_target_ids_write(struct file *file, return ret; } +static ssize_t dbgfs_kdamond_pid_read(struct file *file, + char __user *buf, size_t count, loff_t *ppos) +{ + struct damon_ctx *ctx = file->private_data; + char *kbuf; + ssize_t len; + + kbuf = kmalloc(count, GFP_KERNEL); + if (!kbuf) + return -ENOMEM; + + mutex_lock(&ctx->kdamond_lock); + if (ctx->kdamond) + len = scnprintf(kbuf, count, "%d\n", ctx->kdamond->pid); + else + len = scnprintf(kbuf, count, "none\n"); + mutex_unlock(&ctx->kdamond_lock); + if (!len) + goto out; + len = simple_read_from_buffer(buf, count, ppos, kbuf, len); + +out: + kfree(kbuf); + return len; +} + static int damon_dbgfs_open(struct inode *inode, struct file *file) { file->private_data = inode->i_private; @@ -258,10 +284,18 @@ static const struct file_operations target_ids_fops = { .write = dbgfs_target_ids_write, }; +static const struct file_operations kdamond_pid_fops = { + .owner = THIS_MODULE, + .open = damon_dbgfs_open, + .read = dbgfs_kdamond_pid_read, +}; + static int dbgfs_fill_ctx_dir(struct dentry *dir, struct damon_ctx *ctx) { - const char * const file_names[] = {"attrs", "target_ids"}; - const struct file_operations *fops[] = {&attrs_fops, &target_ids_fops}; + const char * const file_names[] = {"attrs", "target_ids", + "kdamond_pid"}; + const struct file_operations *fops[] = {&attrs_fops, &target_ids_fops, + &kdamond_pid_fops}; int i; for (i = 0; i < ARRAY_SIZE(file_names); i++) From patchwork Thu Mar 18 10:08:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 12147807 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 26311C433E0 for ; Thu, 18 Mar 2021 10:10:29 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A7A3261580 for ; Thu, 18 Mar 2021 10:10:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A7A3261580 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A91308D0005; Thu, 18 Mar 2021 06:10:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A46578D0001; Thu, 18 Mar 2021 06:10:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 893CE8D0005; Thu, 18 Mar 2021 06:10:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0138.hostedemail.com [216.40.44.138]) by kanga.kvack.org (Postfix) with ESMTP id 671F88D0001 for ; Thu, 18 Mar 2021 06:10:22 -0400 (EDT) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 21121181AF5F7 for ; Thu, 18 Mar 2021 10:10:22 +0000 (UTC) X-FDA: 77932575084.04.DF6D06A Received: from mail-qt1-f175.google.com (mail-qt1-f175.google.com [209.85.160.175]) by imf27.hostedemail.com (Postfix) with ESMTP id 38C5F8019145 for ; Thu, 18 Mar 2021 10:10:20 +0000 (UTC) Received: by mail-qt1-f175.google.com with SMTP id a11so3618860qto.2 for ; Thu, 18 Mar 2021 03:10:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=aKeb3fcwK9W0TVkoKsvlqPl7wq54ptWNz0E2zQWWcJc=; b=ShHum+5ovOm1NJ5iEw76D05I7655PcHacbwu/axeF0GnGmE8sxVFRes9jy3I3K3HnZ HpedKe/ABKMbLwgctO52RI9kdcG/V3nG+uBm/nJlutYEHfzGnk+mz9fteZvA6UvPqPq9 zLZcqsIzUXZe6RGjaySBjPO5qhf14Cbt1dtFIAYHztVnDnDNAmumH4nL3cKC1irLCybu 2nO+WPf49E1rvE7qxJRfO1W+pypXbbt2isZN0Bz4tfRlgIFvAl5YNkpqQ67OmZHLWRIt 7cpSGW1pX+hPFY7xtxUWHBIOTRsZFmMNggeiBBlUs7a5U70e+iJFomw3xKWn43BIbkEb /toQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=aKeb3fcwK9W0TVkoKsvlqPl7wq54ptWNz0E2zQWWcJc=; b=V4Tgxggrr7ADAcEb6B6/d4mBhhoSgW4NQ//Z+pFeWPn27BTxiulx7byGCh9B7lV6Uk gzUBWysEcrRteh8iTd7mvJJcBHmv9atTefaAWVSjiZtNlLA84cQMbzmEFA16z9X5NDBh ZQ0Yww6/FaGtnsEoTebRPSgezqQ75OewXdjyDvbsVS07wOkDjEBvPEjQnOP75BGtBC1S WkkD2dScKWDjd/Xb/OI8253giz8TOc8i0jMCLQFDejEc1aVzPhrLIJ9BS6kuMYlFEpBA futqCwHJToSK+92CrxWoog5CPvPoL4uF8hSi8USlovKjdtKYyb7okS1o5z/aJOtCGYAw 6Kzw== X-Gm-Message-State: AOAM532J+c44pCkzJXHtYqydlvMDmtzUKQONgdn5inzY1HvzcdYKPqoT JukxWfIlXy72O/t1FF/u+9I= X-Google-Smtp-Source: ABdhPJwedHgK+H4l2ncmYruYsBfxcLCyzW+a5kzlNCuxgbraegwwPej5zrsd2zBl+nZeBcT8NLbDuA== X-Received: by 2002:ac8:4314:: with SMTP id z20mr2976539qtm.127.1616062219573; Thu, 18 Mar 2021 03:10:19 -0700 (PDT) Received: from localhost.localdomain (ec2-35-169-212-159.compute-1.amazonaws.com. [35.169.212.159]) by smtp.gmail.com with ESMTPSA id d16sm1358101qka.43.2021.03.18.03.10.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 18 Mar 2021 03:10:19 -0700 (PDT) From: sj38.park@gmail.com To: akpm@linux-foundation.org Cc: SeongJae Park , Jonathan.Cameron@Huawei.com, acme@kernel.org, alexander.shishkin@linux.intel.com, amit@kernel.org, benh@kernel.crashing.org, brendanhiggins@google.com, corbet@lwn.net, david@redhat.com, dwmw@amazon.com, elver@google.com, fan.du@intel.com, foersleo@amazon.de, gthelen@google.com, mgorman@suse.de, minchan@kernel.org, mingo@redhat.com, namhyung@kernel.org, peterz@infradead.org, riel@surriel.com, rientjes@google.com, rostedt@goodmis.org, rppt@kernel.org, shakeelb@google.com, shuah@kernel.org, sj38.park@gmail.com, snu@amazon.de, vbabka@suse.cz, vdavydov.dev@gmail.com, zgf574564920@gmail.com, linux-damon@amazon.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v25 09/13] mm/damon/dbgfs: Support multiple contexts Date: Thu, 18 Mar 2021 10:08:52 +0000 Message-Id: <20210318100856.34715-10-sj38.park@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210318100856.34715-1-sj38.park@gmail.com> References: <20210318100856.34715-1-sj38.park@gmail.com> X-Stat-Signature: zm4zp1rxbkpzieh4cqxyhza1ope39o3a X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 38C5F8019145 Received-SPF: none (gmail.com>: No applicable sender policy available) receiver=imf27; identity=mailfrom; envelope-from=""; helo=mail-qt1-f175.google.com; client-ip=209.85.160.175 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1616062220-41995 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park In some use cases, users would want to run multiple monitoring context. For example, if a user wants a high precision monitoring and dedicating multiple CPUs for the job is ok, because DAMON creates one monitoring thread per one context, the user can split the monitoring target regions into multiple small regions and create one context for each region. Or, someone might want to simultaneously monitor different address spaces, e.g., both virtual address space and physical address space. The DAMON's API allows such usage, but 'damon-dbgfs' does not. Therefore, only kernel space DAMON users can do multiple contexts monitoring. This commit allows the user space DAMON users to use multiple contexts monitoring by introducing two new 'damon-dbgfs' debugfs files, 'mk_context' and 'rm_context'. Users can create a new monitoring context by writing the desired name of the new context to 'mk_context'. Then, a new directory with the name and having the files for setting of the context ('attrs', 'target_ids' and 'record') will be created under the debugfs directory. Writing the name of the context to remove to 'rm_context' will remove the related context and directory. Signed-off-by: SeongJae Park --- mm/damon/dbgfs.c | 203 ++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 201 insertions(+), 2 deletions(-) diff --git a/mm/damon/dbgfs.c b/mm/damon/dbgfs.c index aec2b7d81809..9edef931ed00 100644 --- a/mm/damon/dbgfs.c +++ b/mm/damon/dbgfs.c @@ -18,6 +18,7 @@ static struct damon_ctx **dbgfs_ctxs; static int dbgfs_nr_ctxs; static struct dentry **dbgfs_dirs; +static DEFINE_MUTEX(damon_dbgfs_lock); /* * Returns non-empty string on success, negarive error code otherwise. @@ -316,6 +317,192 @@ static struct damon_ctx *dbgfs_new_ctx(void) return ctx; } +static void dbgfs_destroy_ctx(struct damon_ctx *ctx) +{ + damon_destroy_ctx(ctx); +} + +/* + * Make a context of @name and create a debugfs directory for it. + * + * This function should be called while holding damon_dbgfs_lock. + * + * Returns 0 on success, negative error code otherwise. + */ +static int dbgfs_mk_context(char *name) +{ + struct dentry *root, **new_dirs, *new_dir; + struct damon_ctx **new_ctxs, *new_ctx; + int err; + + if (damon_nr_running_ctxs()) + return -EBUSY; + + new_ctxs = krealloc(dbgfs_ctxs, sizeof(*dbgfs_ctxs) * + (dbgfs_nr_ctxs + 1), GFP_KERNEL); + if (!new_ctxs) + return -ENOMEM; + + new_dirs = krealloc(dbgfs_dirs, sizeof(*dbgfs_dirs) * + (dbgfs_nr_ctxs + 1), GFP_KERNEL); + if (!new_dirs) { + kfree(new_ctxs); + return -ENOMEM; + } + + dbgfs_ctxs = new_ctxs; + dbgfs_dirs = new_dirs; + + root = dbgfs_dirs[0]; + if (!root) + return -ENOENT; + + new_dir = debugfs_create_dir(name, root); + dbgfs_dirs[dbgfs_nr_ctxs] = new_dir; + + new_ctx = dbgfs_new_ctx(); + if (!new_ctx) { + debugfs_remove(new_dir); + dbgfs_dirs[dbgfs_nr_ctxs] = NULL; + return -ENOMEM; + } + dbgfs_ctxs[dbgfs_nr_ctxs] = new_ctx; + + err = dbgfs_fill_ctx_dir(dbgfs_dirs[dbgfs_nr_ctxs], + dbgfs_ctxs[dbgfs_nr_ctxs]); + if (err) + return err; + + dbgfs_nr_ctxs++; + return 0; +} + +static ssize_t dbgfs_mk_context_write(struct file *file, + const char __user *buf, size_t count, loff_t *ppos) +{ + char *kbuf; + char *ctx_name; + ssize_t ret = count; + int err; + + kbuf = user_input_str(buf, count, ppos); + if (IS_ERR(kbuf)) + return PTR_ERR(kbuf); + ctx_name = kmalloc(count + 1, GFP_KERNEL); + if (!ctx_name) { + kfree(kbuf); + return -ENOMEM; + } + + /* Trim white space */ + if (sscanf(kbuf, "%s", ctx_name) != 1) { + ret = -EINVAL; + goto out; + } + + mutex_lock(&damon_dbgfs_lock); + err = dbgfs_mk_context(ctx_name); + if (err) + ret = err; + mutex_unlock(&damon_dbgfs_lock); + +out: + kfree(kbuf); + kfree(ctx_name); + return ret; +} + +/* + * Remove a context of @name and its debugfs directory. + * + * This function should be called while holding damon_dbgfs_lock. + * + * Return 0 on success, negative error code otherwise. + */ +static int dbgfs_rm_context(char *name) +{ + struct dentry *root, *dir, **new_dirs; + struct damon_ctx **new_ctxs; + int i, j; + + if (damon_nr_running_ctxs()) + return -EBUSY; + + root = dbgfs_dirs[0]; + if (!root) + return -ENOENT; + + dir = debugfs_lookup(name, root); + if (!dir) + return -ENOENT; + + new_dirs = kmalloc_array(dbgfs_nr_ctxs - 1, sizeof(*dbgfs_dirs), + GFP_KERNEL); + if (!new_dirs) + return -ENOMEM; + + new_ctxs = kmalloc_array(dbgfs_nr_ctxs - 1, sizeof(*dbgfs_ctxs), + GFP_KERNEL); + if (!new_ctxs) { + kfree(new_dirs); + return -ENOMEM; + } + + for (i = 0, j = 0; i < dbgfs_nr_ctxs; i++) { + if (dbgfs_dirs[i] == dir) { + debugfs_remove(dbgfs_dirs[i]); + dbgfs_destroy_ctx(dbgfs_ctxs[i]); + continue; + } + new_dirs[j] = dbgfs_dirs[i]; + new_ctxs[j++] = dbgfs_ctxs[i]; + } + + kfree(dbgfs_dirs); + kfree(dbgfs_ctxs); + + dbgfs_dirs = new_dirs; + dbgfs_ctxs = new_ctxs; + dbgfs_nr_ctxs--; + + return 0; +} + +static ssize_t dbgfs_rm_context_write(struct file *file, + const char __user *buf, size_t count, loff_t *ppos) +{ + char *kbuf; + ssize_t ret = count; + int err; + char *ctx_name; + + kbuf = user_input_str(buf, count, ppos); + if (IS_ERR(kbuf)) + return PTR_ERR(kbuf); + ctx_name = kmalloc(count + 1, GFP_KERNEL); + if (!ctx_name) { + kfree(kbuf); + return -ENOMEM; + } + + /* Trim white space */ + if (sscanf(kbuf, "%s", ctx_name) != 1) { + ret = -EINVAL; + goto out; + } + + mutex_lock(&damon_dbgfs_lock); + err = dbgfs_rm_context(ctx_name); + if (err) + ret = err; + mutex_unlock(&damon_dbgfs_lock); + +out: + kfree(kbuf); + kfree(ctx_name); + return ret; +} + static ssize_t dbgfs_monitor_on_read(struct file *file, char __user *buf, size_t count, loff_t *ppos) { @@ -358,6 +545,16 @@ static ssize_t dbgfs_monitor_on_write(struct file *file, return ret; } +static const struct file_operations mk_contexts_fops = { + .owner = THIS_MODULE, + .write = dbgfs_mk_context_write, +}; + +static const struct file_operations rm_contexts_fops = { + .owner = THIS_MODULE, + .write = dbgfs_rm_context_write, +}; + static const struct file_operations monitor_on_fops = { .owner = THIS_MODULE, .read = dbgfs_monitor_on_read, @@ -367,8 +564,10 @@ static const struct file_operations monitor_on_fops = { static int __init __damon_dbgfs_init(void) { struct dentry *dbgfs_root; - const char * const file_names[] = {"monitor_on"}; - const struct file_operations *fops[] = {&monitor_on_fops}; + const char * const file_names[] = {"mk_contexts", "rm_contexts", + "monitor_on"}; + const struct file_operations *fops[] = {&mk_contexts_fops, + &rm_contexts_fops, &monitor_on_fops}; int i; dbgfs_root = debugfs_create_dir("damon", NULL); From patchwork Thu Mar 18 10:08:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 12147809 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A7CFDC433E6 for ; Thu, 18 Mar 2021 10:10:30 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1C79964F59 for ; Thu, 18 Mar 2021 10:10:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1C79964F59 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 81F4E8D0006; Thu, 18 Mar 2021 06:10:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7F81E8D0001; Thu, 18 Mar 2021 06:10:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5D4E48D0006; Thu, 18 Mar 2021 06:10:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0239.hostedemail.com [216.40.44.239]) by kanga.kvack.org (Postfix) with ESMTP id 2CB408D0001 for ; Thu, 18 Mar 2021 06:10:25 -0400 (EDT) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id E13C72C32 for ; Thu, 18 Mar 2021 10:10:24 +0000 (UTC) X-FDA: 77932575168.29.18C4DF4 Received: from mail-qk1-f181.google.com (mail-qk1-f181.google.com [209.85.222.181]) by imf16.hostedemail.com (Postfix) with ESMTP id CFC8A8019179 for ; Thu, 18 Mar 2021 10:10:23 +0000 (UTC) Received: by mail-qk1-f181.google.com with SMTP id x10so1375981qkm.8 for ; Thu, 18 Mar 2021 03:10:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=c38wusKiZ9Be/xNlFcrywm9PQXTdOO8T5ykhQZird1w=; b=tPI4arDg32hzz+Oe0rsaUCiRLZq4LWQJDyJZk9xQoX8KD/MbwqZlSww7bNEJawU2yj j+ml/9zm5G9IkPG/Ul1NqiFl3oDfEFV8P3dnGsiAV4bh3f+PKa7iStgjbN+qFt8gKpe4 HHBPGYgfu0OslOx06cehGUU98wJNbZryxlfSBjXYRW+IJiBXIbG9Cr86H2choj+lu/hm KdO1G6XsO/YdP4Bc5vCyDXyerWDOqnrOhxvcBQgwzBa/BSbbUn2SoqNY3u8WW/EZ35ZS XNHGTYVoknGlNiCvWZj9UG2JTqemH02i4EkCLGdhPQ5CuQztFXdq4RQznQfp/qDbl2P0 v5Fg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=c38wusKiZ9Be/xNlFcrywm9PQXTdOO8T5ykhQZird1w=; b=PdVsbePXfI424lMedJf7E4YDSOiCSfwE8dUvOOuqrfY2eG8pB1602giP8a8W2tebsy +3BDiEnXUdtEekAHThz77T36QwICdp2vaDIJ9jEEG7vSCcbEEN0xq5xcGQi0uuPSwuj/ lkqZLijcAOPTmEC5/zz7ngiolIkp+xsLEZOfy5hGrJAyItGU0GngjJb6Mdn+AyRlxC5t Mca5royc+XAvLZBJ/GB/yieIvduwBCjDmf5m4hWNl56xX8Al+9lWbTwB/OzVINT9235Q TZj3sFwmMT3fQ4eJOfQoWPYg6RlhkXdjv0zLNHOGVyfS2MG7anJDJ8uFClk+KptjDCwU iRXg== X-Gm-Message-State: AOAM532y5FN5HpZP6vQxuysmWPFLzsrRKsJ76spyc+gW8dbZsF37OKza o5rPud7MqIUvG6HpuzTZKXs= X-Google-Smtp-Source: ABdhPJyr/ibEGdy0Rf9I+7FhwL2nb7U2fQdIOz/54dCy2bNExq5sTnWHFUw1MiKBUpNcsbAJLmh/fQ== X-Received: by 2002:ae9:df82:: with SMTP id t124mr3345335qkf.267.1616062221834; Thu, 18 Mar 2021 03:10:21 -0700 (PDT) Received: from localhost.localdomain (ec2-35-169-212-159.compute-1.amazonaws.com. [35.169.212.159]) by smtp.gmail.com with ESMTPSA id d16sm1358101qka.43.2021.03.18.03.10.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 18 Mar 2021 03:10:21 -0700 (PDT) From: sj38.park@gmail.com To: akpm@linux-foundation.org Cc: SeongJae Park , Jonathan.Cameron@Huawei.com, acme@kernel.org, alexander.shishkin@linux.intel.com, amit@kernel.org, benh@kernel.crashing.org, brendanhiggins@google.com, corbet@lwn.net, david@redhat.com, dwmw@amazon.com, elver@google.com, fan.du@intel.com, foersleo@amazon.de, gthelen@google.com, mgorman@suse.de, minchan@kernel.org, mingo@redhat.com, namhyung@kernel.org, peterz@infradead.org, riel@surriel.com, rientjes@google.com, rostedt@goodmis.org, rppt@kernel.org, shakeelb@google.com, shuah@kernel.org, sj38.park@gmail.com, snu@amazon.de, vbabka@suse.cz, vdavydov.dev@gmail.com, zgf574564920@gmail.com, linux-damon@amazon.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v25 10/13] Documentation: Add documents for DAMON Date: Thu, 18 Mar 2021 10:08:53 +0000 Message-Id: <20210318100856.34715-11-sj38.park@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210318100856.34715-1-sj38.park@gmail.com> References: <20210318100856.34715-1-sj38.park@gmail.com> MIME-Version: 1.0 X-Stat-Signature: 8jscgeepindxm5bea3oputpkxbs3ysnu X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: CFC8A8019179 Received-SPF: none (gmail.com>: No applicable sender policy available) receiver=imf16; identity=mailfrom; envelope-from=""; helo=mail-qk1-f181.google.com; client-ip=209.85.222.181 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1616062223-294176 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park This commit adds documents for DAMON under `Documentation/admin-guide/mm/damon/` and `Documentation/vm/damon/`. Signed-off-by: SeongJae Park --- Documentation/admin-guide/mm/damon/guide.rst | 159 +++++++++++++ Documentation/admin-guide/mm/damon/index.rst | 15 ++ Documentation/admin-guide/mm/damon/plans.rst | 29 +++ Documentation/admin-guide/mm/damon/start.rst | 97 ++++++++ Documentation/admin-guide/mm/damon/usage.rst | 112 +++++++++ Documentation/admin-guide/mm/index.rst | 1 + Documentation/vm/damon/api.rst | 20 ++ Documentation/vm/damon/design.rst | 166 +++++++++++++ Documentation/vm/damon/eval.rst | 232 +++++++++++++++++++ Documentation/vm/damon/faq.rst | 58 +++++ Documentation/vm/damon/index.rst | 31 +++ Documentation/vm/index.rst | 1 + 12 files changed, 921 insertions(+) create mode 100644 Documentation/admin-guide/mm/damon/guide.rst create mode 100644 Documentation/admin-guide/mm/damon/index.rst create mode 100644 Documentation/admin-guide/mm/damon/plans.rst create mode 100644 Documentation/admin-guide/mm/damon/start.rst create mode 100644 Documentation/admin-guide/mm/damon/usage.rst create mode 100644 Documentation/vm/damon/api.rst create mode 100644 Documentation/vm/damon/design.rst create mode 100644 Documentation/vm/damon/eval.rst create mode 100644 Documentation/vm/damon/faq.rst create mode 100644 Documentation/vm/damon/index.rst diff --git a/Documentation/admin-guide/mm/damon/guide.rst b/Documentation/admin-guide/mm/damon/guide.rst new file mode 100644 index 000000000000..49da40bc4ba9 --- /dev/null +++ b/Documentation/admin-guide/mm/damon/guide.rst @@ -0,0 +1,159 @@ +.. SPDX-License-Identifier: GPL-2.0 + +================== +Optimization Guide +================== + +This document helps you estimating the amount of benefit that you could get +from DAMON-based optimizations, and describes how you could achieve it. You +are assumed to already read :doc:`start`. + + +Check The Signs +=============== + +No optimization can provide same extent of benefit to every case. Therefore +you should first guess how much improvements you could get using DAMON. If +some of below conditions match your situation, you could consider using DAMON. + +- *Low IPC and High Cache Miss Ratios.* Low IPC means most of the CPU time is + spent waiting for the completion of time-consuming operations such as memory + access, while high cache miss ratios mean the caches don't help it well. + DAMON is not for cache level optimization, but DRAM level. However, + improving DRAM management will also help this case by reducing the memory + operation latency. +- *Memory Over-commitment and Unknown Users.* If you are doing memory + overcommitment and you cannot control every user of your system, a memory + bank run could happen at any time. You can estimate when it will happen + based on DAMON's monitoring results and act earlier to avoid or deal better + with the crisis. +- *Frequent Memory Pressure.* Frequent memory pressure means your system has + wrong configurations or memory hogs. DAMON will help you find the right + configuration and/or the criminals. +- *Heterogeneous Memory System.* If your system is utilizing memory devices + that placed between DRAM and traditional hard disks, such as non-volatile + memory or fast SSDs, DAMON could help you utilizing the devices more + efficiently. + + +Profile +======= + +If you found some positive signals, you could start by profiling your workloads +using DAMON. Find major workloads on your systems and analyze their data +access pattern to find something wrong or can be improved. The DAMON user +space tool (``damo``) will be useful for this. You can get ``damo`` from +``tools/damon/`` directory in the DAMON development tree (``damon/master`` +branch of https://github.com/sjp38/linux.git). + +We recommend you to start from working set size distribution check using ``damo +report wss``. If the distribution is ununiform or quite different from what +you estimated, you could consider `Memory Configuration`_ optimization. + +Then, review the overall access pattern in heatmap form using ``damo report +heats``. If it shows a simple pattern consists of a small number of memory +regions having high contrast of access temperature, you could consider manual +`Program Modification`_. + +If you still want to absorb more benefits, you should develop `Personalized +DAMON Application`_ for your special case. + +You don't need to take only one approach among the above plans, but you could +use multiple of the above approaches to maximize the benefit. + + +Optimize +======== + +If the profiling result also says it's worth trying some optimization, you +could consider below approaches. Note that some of the below approaches assume +that your systems are configured with swap devices or other types of auxiliary +memory so that you don't strictly required to accommodate the whole working set +in the main memory. Most of the detailed optimization should be made on your +concrete understanding of your memory devices. + + +Memory Configuration +-------------------- + +No more no less, DRAM should be large enough to accommodate only important +working sets, because DRAM is highly performance critical but expensive and +heavily consumes the power. However, knowing the size of the real important +working sets is difficult. As a consequence, people usually equips +unnecessarily large or too small DRAM. Many problems stem from such wrong +configurations. + +Using the working set size distribution report provided by ``damo report wss``, +you can know the appropriate DRAM size for you. For example, roughly speaking, +if you worry about only 95 percentile latency, you don't need to equip DRAM of +a size larger than 95 percentile working set size. + +Let's see a real example. This `page +`_ +shows the heatmap and the working set size distributions/changes of +``freqmine`` workload in PARSEC3 benchmark suite. The working set size spikes +up to 180 MiB, but keeps smaller than 50 MiB for more than 95% of the time. +Even though you give only 50 MiB of memory space to the workload, it will work +well for 95% of the time. Meanwhile, you can save the 130 MiB of memory space. + + +Program Modification +-------------------- + +If the data access pattern heatmap plotted by ``damo report heats`` is quite +simple so that you can understand how the things are going in the workload with +your human eye, you could manually optimize the memory management. + +For example, suppose that the workload has two big memory object but only one +object is frequently accessed while the other one is only occasionally +accessed. Then, you could modify the program source code to keep the hot +object in the main memory by invoking ``mlock()`` or ``madvise()`` with +``MADV_WILLNEED``. Or, you could proactively evict the cold object using +``madvise()`` with ``MADV_COLD`` or ``MADV_PAGEOUT``. Using both together +would be also worthy. + +A research work [1]_ using the ``mlock()`` achieved up to 2.55x performance +speedup. + +Let's see another realistic example access pattern for this kind of +optimizations. This `page +`_ +shows the visualized access patterns of streamcluster workload in PARSEC3 +benchmark suite. We can easily identify the 100 MiB sized hot object. + + +Personalized DAMON Application +------------------------------ + +Above approaches will work well for many general cases, but would not enough +for some special cases. + +If this is the case, it might be the time to forget the comfortable use of the +user space tool and dive into the debugfs interface (refer to :doc:`usage` for +the detail) of DAMON. Using the interface, you can control the DAMON more +flexibly. Therefore, you can write your personalized DAMON application that +controls the monitoring via the debugfs interface, analyzes the result, and +applies complex optimizations itself. Using this, you can make more creative +and wise optimizations. + +If you are a kernel space programmer, writing kernel space DAMON applications +using the API (refer to the :doc:`/vm/damon/api` for more detail) would be an +option. + + +Reference Practices +=================== + +Referencing previously done successful practices could help you getting the +sense for this kind of optimizations. There is an academic paper [1]_ +reporting the visualized access pattern and manual `Program +Modification`_ results for a number of realistic workloads. You can also get +the visualized access patterns [3]_ [4]_ [5]_ and automated DAMON-based memory +operations results for other realistic workloads that collected with latest +version of DAMON [2]_ . + +.. [1] https://dl.acm.org/doi/10.1145/3366626.3368125 +.. [2] https://damonitor.github.io/test/result/perf/latest/html/ +.. [3] https://damonitor.github.io/test/result/visual/latest/rec.heatmap.1.png.html +.. [4] https://damonitor.github.io/test/result/visual/latest/rec.wss_sz.png.html +.. [5] https://damonitor.github.io/test/result/visual/latest/rec.wss_time.png.html diff --git a/Documentation/admin-guide/mm/damon/index.rst b/Documentation/admin-guide/mm/damon/index.rst new file mode 100644 index 000000000000..0baae7a5402b --- /dev/null +++ b/Documentation/admin-guide/mm/damon/index.rst @@ -0,0 +1,15 @@ +.. SPDX-License-Identifier: GPL-2.0 + +======================== +Monitoring Data Accesses +======================== + +:doc:`DAMON ` allows light-weight data access monitoring. +Using this, users can analyze and optimize their systems. + +.. toctree:: + :maxdepth: 2 + + start + guide + usage diff --git a/Documentation/admin-guide/mm/damon/plans.rst b/Documentation/admin-guide/mm/damon/plans.rst new file mode 100644 index 000000000000..e3aa5ab96c29 --- /dev/null +++ b/Documentation/admin-guide/mm/damon/plans.rst @@ -0,0 +1,29 @@ +.. SPDX-License-Identifier: GPL-2.0 + +============ +Future Plans +============ + +DAMON is still on its first stage. Below plans are still under development. + + +Automate Data Access Monitoring-based Memory Operation Schemes Execution +======================================================================== + +The ultimate goal of DAMON is to be used as a building block for the data +access pattern aware kernel memory management optimization. It will make +system just works efficiently. However, some users having very special +workloads will want to further do their own optimization. DAMON will automate +most of the tasks for such manual optimizations in near future. Users will be +required to only describe what kind of data access pattern-based operation +schemes they want in a simple form. + +By applying a very simple scheme for THP promotion/demotion with a prototype +implementation, DAMON reduced 60% of THP memory footprint overhead while +preserving 50% of the THP performance benefit. The detailed results can be +seen on an external web page [1]_. + +Several RFC patchsets for this plan are available [2]_. + +.. [1] https://damonitor.github.io/test/result/perf/latest/html/ +.. [2] https://lore.kernel.org/linux-mm/20200616073828.16509-1-sjpark@amazon.com/ diff --git a/Documentation/admin-guide/mm/damon/start.rst b/Documentation/admin-guide/mm/damon/start.rst new file mode 100644 index 000000000000..69bac6782624 --- /dev/null +++ b/Documentation/admin-guide/mm/damon/start.rst @@ -0,0 +1,97 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=============== +Getting Started +=============== + +This document briefly describes how you can use DAMON by demonstrating its +default user space tool. Please note that this document describes only a part +of its features for brevity. Please refer to :doc:`usage` for more details. + + +TL; DR +====== + +Follow below 5 commands to monitor and visualize the access pattern of your +workload. :: + + $ git clone https://github.com/sjp38/linux -b damon/master + /* build the kernel with CONFIG_DAMON=y, install, reboot */ + $ mount -t debugfs none /sys/kernel/debug/ + $ cd linux/tools/damon + $ ./damo record $(pidof ) + $ ./damo report heats --heatmap access_pattern.png + + +Prerequisites +============= + +Kernel +------ + +You should first ensure your system is running on a kernel built with +``CONFIG_DAMON=y``. + + +User Space Tool +--------------- + +For the demonstration, we will use the default user space tool for DAMON, +called DAMON Operator (DAMO). It is located at ``tools/damon/damo`` of the +DAMON development kernel source tree (``damon/master`` branch of +https://github.com/sjp38/linux). For brevity, below examples assume you set +``$PATH`` to point it. It's not mandatory, though. + +Because DAMO is using the debugfs interface (refer to :doc:`usage` for the +detail) of DAMON, you should ensure debugfs is mounted. Mount it manually as +below:: + + # mount -t debugfs none /sys/kernel/debug/ + +or append below line to your ``/etc/fstab`` file so that your system can +automatically mount debugfs from next booting:: + + debugfs /sys/kernel/debug debugfs defaults 0 0 + + +Recording Data Access Patterns +============================== + +Below commands record memory access pattern of a program and save the +monitoring results in a file. :: + + $ git clone https://github.com/sjp38/masim + $ cd masim; make; ./masim ./configs/zigzag.cfg & + $ sudo damo record -o damon.data $(pidof masim) + +The first two lines of the commands get an artificial memory access generator +program and runs it in the background. It will repeatedly access two 100 MiB +sized memory regions one by one. You can substitute this with your real +workload. The last line asks ``damo`` to record the access pattern in +``damon.data`` file. + + +Visualizing Recorded Patterns +============================= + +Below three commands visualize the recorded access patterns into three +image files. :: + + $ damo report heats --heatmap access_pattern_heatmap.png + $ damo report wss --range 0 101 1 --plot wss_dist.png + $ damo report wss --range 0 101 1 --sortby time --plot wss_chron_change.png + +- ``access_pattern_heatmap.png`` will show the data access pattern in a + heatmap, which shows when (x-axis) what memory region (y-axis) is how + frequently accessed (color). +- ``wss_dist.png`` will show the distribution of the working set size. +- ``wss_chron_change.png`` will show how the working set size has + chronologically changed. + +You can show the images in a web page [1]_ . Those made with other realistic +workloads are also available [2]_ [3]_ [4]_. + +.. [1] https://damonitor.github.io/doc/html/v17/admin-guide/mm/damon/start.html#visualizing-recorded-patterns +.. [2] https://damonitor.github.io/test/result/visual/latest/rec.heatmap.1.png.html +.. [3] https://damonitor.github.io/test/result/visual/latest/rec.wss_sz.png.html +.. [4] https://damonitor.github.io/test/result/visual/latest/rec.wss_time.png.html diff --git a/Documentation/admin-guide/mm/damon/usage.rst b/Documentation/admin-guide/mm/damon/usage.rst new file mode 100644 index 000000000000..ccf631c3c2c2 --- /dev/null +++ b/Documentation/admin-guide/mm/damon/usage.rst @@ -0,0 +1,112 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=============== +Detailed Usages +=============== + +DAMON provides below three interfaces for different users. + +- *DAMON user space tool.* + This is for privileged people such as system administrators who want a + just-working human-friendly interface. Using this, users can use the DAMON’s + major features in a human-friendly way. It may not be highly tuned for + special cases, though. It supports only virtual address spaces monitoring. +- *debugfs interface.* + This is for privileged user space programmers who want more optimized use of + DAMON. Using this, users can use DAMON’s major features by reading + from and writing to special debugfs files. Therefore, you can write and use + your personalized DAMON debugfs wrapper programs that reads/writes the + debugfs files instead of you. The DAMON user space tool is also a reference + implementation of such programs. It supports only virtual address spaces + monitoring. +- *Kernel Space Programming Interface.* + This is for kernel space programmers. Using this, users can utilize every + feature of DAMON most flexibly and efficiently by writing kernel space + DAMON application programs for you. You can even extend DAMON for various + address spaces. + +This document describes only the debugfs interface because the user space tool +is only in the development tree (``damon/master`` branch of +https://github.com/sjp38/linux) and you could refer to :doc:`/vm/damon/api` for +the kernel space programming interface. + + +debugfs Interface +================= + +DAMON exports three files, ``attrs``, ``target_ids``, and ``monitor_on`` under +its debugfs directory, ``/damon/``. + + +Attributes +---------- + +Users can get and set the ``sampling interval``, ``aggregation interval``, +``regions update interval``, and min/max number of monitoring target regions by +reading from and writing to the ``attrs`` file. To know about the monitoring +attributes in detail, please refer to the :doc:`/vm/damon/design`. For +example, below commands set those values to 5 ms, 100 ms, 1,000 ms, 10 and +1000, and then check it again:: + + # cd /damon + # echo 5000 100000 1000000 10 1000 > attrs + # cat attrs + 5000 100000 1000000 10 1000 + + +Target IDs +---------- + +Some types of address spaces supports multiple monitoring target. For example, +the virtual memory address spaces monitoring can have multiple processes as the +monitoring targets. Users can set the targets by writing relevant id values of +the targets to, and get the ids of the current targets by reading from the +``target_ids`` file. In case of the virtual address spaces monitoring, the +values should be pids of the monitoring target processes. For example, below +commands set processes having pids 42 and 4242 as the monitoring targets and +check it again:: + + # cd /damon + # echo 42 4242 > target_ids + # cat target_ids + 42 4242 + +Note that setting the target ids doesn't start the monitoring. + + +Turning On/Off +-------------- + +Setting the files as described above doesn't incur effect unless you explicitly +start the monitoring. You can start, stop, and check the current status of the +monitoring by writing to and reading from the ``monitor_on`` file. Writing +``on`` to the file starts the monitoring of the targets with the attributes. +Writing ``off`` to the file stops those. DAMON also stops if every target +process is terminated. Below example commands turn on, off, and check the +status of DAMON:: + + # cd /damon + # echo on > monitor_on + # echo off > monitor_on + # cat monitor_on + off + +Please note that you cannot write to the above-mentioned debugfs files while +the monitoring is turned on. If you write to the files while DAMON is running, +an error code such as ``-EBUSY`` will be returned. + + +Tracepoint for Monitoring Results +================================= + +DAMON provides the monitoring results via a tracepoint, +``damon:damon_aggregated``. While the monitoring is turned on, you could +record the tracepoint events and show results using tracepoint supporting tools +like ``perf``. For example:: + + # echo on > monitor_on + # perf record damon:damon_aggregated & + # sleep 5 + # kill 9 $(pidof perf) + # echo off > monitor_on + # perf script diff --git a/Documentation/admin-guide/mm/index.rst b/Documentation/admin-guide/mm/index.rst index 4b14d8b50e9e..cbd19d5e625f 100644 --- a/Documentation/admin-guide/mm/index.rst +++ b/Documentation/admin-guide/mm/index.rst @@ -27,6 +27,7 @@ the Linux memory management. concepts cma_debugfs + damon/index hugetlbpage idle_page_tracking ksm diff --git a/Documentation/vm/damon/api.rst b/Documentation/vm/damon/api.rst new file mode 100644 index 000000000000..08f34df45523 --- /dev/null +++ b/Documentation/vm/damon/api.rst @@ -0,0 +1,20 @@ +.. SPDX-License-Identifier: GPL-2.0 + +============= +API Reference +============= + +Kernel space programs can use every feature of DAMON using below APIs. All you +need to do is including ``damon.h``, which is located in ``include/linux/`` of +the source tree. + +Structures +========== + +.. kernel-doc:: include/linux/damon.h + + +Functions +========= + +.. kernel-doc:: mm/damon/core.c diff --git a/Documentation/vm/damon/design.rst b/Documentation/vm/damon/design.rst new file mode 100644 index 000000000000..727d72093f8f --- /dev/null +++ b/Documentation/vm/damon/design.rst @@ -0,0 +1,166 @@ +.. SPDX-License-Identifier: GPL-2.0 + +====== +Design +====== + +Configurable Layers +=================== + +DAMON provides data access monitoring functionality while making the accuracy +and the overhead controllable. The fundamental access monitorings require +primitives that dependent on and optimized for the target address space. On +the other hand, the accuracy and overhead tradeoff mechanism, which is the core +of DAMON, is in the pure logic space. DAMON separates the two parts in +different layers and defines its interface to allow various low level +primitives implementations configurable with the core logic. + +Due to this separated design and the configurable interface, users can extend +DAMON for any address space by configuring the core logics with appropriate low +level primitive implementations. If appropriate one is not provided, users can +implement the primitives on their own. + +For example, physical memory, virtual memory, swap space, those for specific +processes, NUMA nodes, files, and backing memory devices would be supportable. +Also, if some architectures or devices support special optimized access check +primitives, those will be easily configurable. + + +Reference Implementations of Address Space Specific Primitives +============================================================== + +The low level primitives for the fundamental access monitoring are defined in +two parts: + +1. Identification of the monitoring target address range for the address space. +2. Access check of specific address range in the target space. + +DAMON currently provides the implementation of the primitives for only the +virtual address spaces. Below two subsections describe how it works. + + +PTE Accessed-bit Based Access Check +----------------------------------- + +The implementation for the virtual address space uses PTE Accessed-bit for +basic access checks. It finds the relevant PTE Accessed bit from the address +by walking the page table for the target task of the address. In this way, the +implementation finds and clears the bit for next sampling target address and +checks whether the bit set again after one sampling period. This could disturb +other kernel subsystems using the Accessed bits, namely Idle page tracking and +the reclaim logic. To avoid such disturbances, DAMON makes it mutually +exclusive with Idle page tracking and uses ``PG_idle`` and ``PG_young`` page +flags to solve the conflict with the reclaim logic, as Idle page tracking does. + + +VMA-based Target Address Range Construction +------------------------------------------- + +Only small parts in the super-huge virtual address space of the processes are +mapped to the physical memory and accessed. Thus, tracking the unmapped +address regions is just wasteful. However, because DAMON can deal with some +level of noise using the adaptive regions adjustment mechanism, tracking every +mapping is not strictly required but could even incur a high overhead in some +cases. That said, too huge unmapped areas inside the monitoring target should +be removed to not take the time for the adaptive mechanism. + +For the reason, this implementation converts the complex mappings to three +distinct regions that cover every mapped area of the address space. The two +gaps between the three regions are the two biggest unmapped areas in the given +address space. The two biggest unmapped areas would be the gap between the +heap and the uppermost mmap()-ed region, and the gap between the lowermost +mmap()-ed region and the stack in most of the cases. Because these gaps are +exceptionally huge in usual address spaces, excluding these will be sufficient +to make a reasonable trade-off. Below shows this in detail:: + + + + + (small mmap()-ed regions and munmap()-ed regions) + + + + + +Address Space Independent Core Mechanisms +========================================= + +Below four sections describe each of the DAMON core mechanisms and the five +monitoring attributes, ``sampling interval``, ``aggregation interval``, +``regions update interval``, ``minimum number of regions``, and ``maximum +number of regions``. + + +Access Frequency Monitoring +--------------------------- + +The output of DAMON says what pages are how frequently accessed for a given +duration. The resolution of the access frequency is controlled by setting +``sampling interval`` and ``aggregation interval``. In detail, DAMON checks +access to each page per ``sampling interval`` and aggregates the results. In +other words, counts the number of the accesses to each page. After each +``aggregation interval`` passes, DAMON calls callback functions that previously +registered by users so that users can read the aggregated results and then +clears the results. This can be described in below simple pseudo-code:: + + while monitoring_on: + for page in monitoring_target: + if accessed(page): + nr_accesses[page] += 1 + if time() % aggregation_interval == 0: + for callback in user_registered_callbacks: + callback(monitoring_target, nr_accesses) + for page in monitoring_target: + nr_accesses[page] = 0 + sleep(sampling interval) + +The monitoring overhead of this mechanism will arbitrarily increase as the +size of the target workload grows. + + +Region Based Sampling +--------------------- + +To avoid the unbounded increase of the overhead, DAMON groups adjacent pages +that assumed to have the same access frequencies into a region. As long as the +assumption (pages in a region have the same access frequencies) is kept, only +one page in the region is required to be checked. Thus, for each ``sampling +interval``, DAMON randomly picks one page in each region, waits for one +``sampling interval``, checks whether the page is accessed meanwhile, and +increases the access frequency of the region if so. Therefore, the monitoring +overhead is controllable by setting the number of regions. DAMON allows users +to set the minimum and the maximum number of regions for the trade-off. + +This scheme, however, cannot preserve the quality of the output if the +assumption is not guaranteed. + + +Adaptive Regions Adjustment +--------------------------- + +Even somehow the initial monitoring target regions are well constructed to +fulfill the assumption (pages in same region have similar access frequencies), +the data access pattern can be dynamically changed. This will result in low +monitoring quality. To keep the assumption as much as possible, DAMON +adaptively merges and splits each region based on their access frequency. + +For each ``aggregation interval``, it compares the access frequencies of +adjacent regions and merges those if the frequency difference is small. Then, +after it reports and clears the aggregated access frequency of each region, it +splits each region into two or three regions if the total number of regions +will not exceed the user-specified maximum number of regions after the split. + +In this way, DAMON provides its best-effort quality and minimal overhead while +keeping the bounds users set for their trade-off. + + +Dynamic Target Space Updates Handling +------------------------------------- + +The monitoring target address range could dynamically changed. For example, +virtual memory could be dynamically mapped and unmapped. Physical memory could +be hot-plugged. + +As the changes could be quite frequent in some cases, DAMON checks the dynamic +memory mapping changes and applies it to the abstracted target area only for +each of a user-specified time interval (``regions update interval``). diff --git a/Documentation/vm/damon/eval.rst b/Documentation/vm/damon/eval.rst new file mode 100644 index 000000000000..4ce1a6d86036 --- /dev/null +++ b/Documentation/vm/damon/eval.rst @@ -0,0 +1,232 @@ +.. SPDX-License-Identifier: GPL-2.0 + +========== +Evaluation +========== + +DAMON is lightweight. It increases system memory usage by 0.39% and slows +target workloads down by 1.16%. + +DAMON is accurate and useful for memory management optimizations. An +experimental DAMON-based operation scheme for THP, namely 'ethp', removes +76.15% of THP memory overheads while preserving 51.25% of THP speedup. Another +experimental DAMON-based 'proactive reclamation' implementation, namely 'prcl', +reduces 93.38% of residential sets and 23.63% of system memory footprint while +incurring only 1.22% runtime overhead in the best case (parsec3/freqmine). + + +Setup +===== + +On QEMU/KVM based virtual machines utilizing 130GB of RAM and 36 vCPUs hosted +by AWS EC2 i3.metal instances that running a kernel that v24 DAMON patchset is +applied, I measure runtime and consumed system memory while running various +realistic workloads with several configurations. From each of PARSEC3 [3]_ and +SPLASH-2X [4]_ benchmark suites I pick 12 workloads, so I use 24 workloads in +total. I use another wrapper scripts [5]_ for convenient setup and run of the +workloads. + + +Measurement +----------- + +For the measurement of the amount of consumed memory in system global scope, I +drop caches before starting each of the workloads and monitor 'MemFree' in the +'/proc/meminfo' file. To make results more stable, I repeat the runs 5 times +and average results. + + +Configurations +-------------- + +The configurations I use are as below. + +- orig: Linux v5.10 with 'madvise' THP policy +- rec: 'orig' plus DAMON running with virtual memory access recording +- prec: 'orig' plus DAMON running with physical memory access recording +- thp: same with 'orig', but use 'always' THP policy +- ethp: 'orig' plus a DAMON operation scheme, 'efficient THP' +- prcl: 'orig' plus a DAMON operation scheme, 'proactive reclaim [6]_' + +I use 'rec' for measurement of DAMON overheads to target workloads and system +memory. 'prec' is for physical memory monitroing and recording. It monitors +17GB sized 'System RAM' region. The remaining configs including 'thp', 'ethp', +and 'prcl' are for measurement of DAMON monitoring accuracy. + +'ethp' and 'prcl' are simple DAMON-based operation schemes developed for +proof of concepts of DAMON. 'ethp' reduces memory space waste of THP by using +DAMON for the decision of promotions and demotion for huge pages, while 'prcl' +is as similar as the original work. For example, those can be implemented as +below:: + + # format: + # ethp: Use huge pages if a region shows >=5% access rate, use regular + # pages if a region >=2MB shows 0 access rate for >=7 seconds + min max 5 max min max hugepage + 2M max min min 7s max nohugepage + + # prcl: If a region >=4KB shows 0 access rate for >=5 seconds, page out. + 4K max 0 0 5s max pageout + +Note that these examples are designed with my only straightforward intuition +because those are for only proof of concepts and monitoring accuracy of DAMON. +In other words, those are not for production. For production use, those should +be more tuned. For automation of such tuning, you can use a user space tool +called DAMOOS [8]_. For the evaluation, we use 'ethp' as same to above +example, but we use DAMOOS-tuned 'prcl' for each workload. + +The evaluation is done using the tests package for DAMON, ``damon-tests`` [7]_. +Using it, you can do the evaluation and generate a report on your own. + +.. [1] "Redis latency problems troubleshooting", https://redis.io/topics/latency +.. [2] "Disable Transparent Huge Pages (THP)", + https://docs.mongodb.com/manual/tutorial/transparent-huge-pages/ +.. [3] "The PARSEC Becnhmark Suite", https://parsec.cs.princeton.edu/index.htm +.. [4] "SPLASH-2x", https://parsec.cs.princeton.edu/parsec3-doc.htm#splash2x +.. [5] "parsec3_on_ubuntu", https://github.com/sjp38/parsec3_on_ubuntu +.. [6] "Proactively reclaiming idle memory", https://lwn.net/Articles/787611/ +.. [7] "damon-tests", https://github.com/awslabs/damon-tests +.. [8] "DAMOOS", https://github.com/awslabs/damoos + + +Results +======= + +Below two tables show the measurement results. The runtimes are in seconds +while the memory usages are in KiB. Each configuration except 'orig' shows +its overhead relative to 'orig' in percent within parenthesizes.:: + + runtime orig rec (overhead) prec (overhead) thp (overhead) ethp (overhead) prcl (overhead) + parsec3/blackscholes 139.658 140.168 (0.37) 139.385 (-0.20) 138.367 (-0.92) 139.279 (-0.27) 147.024 (5.27) + parsec3/bodytrack 123.788 124.622 (0.67) 123.636 (-0.12) 125.115 (1.07) 123.840 (0.04) 141.928 (14.65) + parsec3/canneal 207.491 210.318 (1.36) 217.927 (5.03) 174.287 (-16.00) 202.609 (-2.35) 225.483 (8.67) + parsec3/dedup 18.292 18.301 (0.05) 18.378 (0.47) 18.264 (-0.15) 18.298 (0.03) 20.541 (12.30) + parsec3/facesim 343.893 340.286 (-1.05) 338.217 (-1.65) 332.953 (-3.18) 333.840 (-2.92) 365.650 (6.33) + parsec3/fluidanimate 339.959 326.886 (-3.85) 330.286 (-2.85) 331.239 (-2.57) 326.011 (-4.10) 341.684 (0.51) + parsec3/freqmine 445.987 436.332 (-2.16) 435.946 (-2.25) 435.704 (-2.31) 437.595 (-1.88) 451.414 (1.22) + parsec3/raytrace 184.106 182.158 (-1.06) 182.056 (-1.11) 183.180 (-0.50) 183.545 (-0.30) 202.197 (9.83) + parsec3/streamcluster 599.990 674.091 (12.35) 617.314 (2.89) 521.864 (-13.02) 551.971 (-8.00) 696.127 (16.02) + parsec3/swaptions 220.462 222.637 (0.99) 220.449 (-0.01) 219.921 (-0.25) 221.607 (0.52) 223.956 (1.59) + parsec3/vips 87.767 88.700 (1.06) 87.461 (-0.35) 87.466 (-0.34) 87.875 (0.12) 91.768 (4.56) + parsec3/x264 110.843 117.856 (6.33) 113.023 (1.97) 108.665 (-1.97) 115.434 (4.14) 117.811 (6.29) + splash2x/barnes 131.441 129.275 (-1.65) 128.341 (-2.36) 119.317 (-9.22) 126.199 (-3.99) 147.602 (12.30) + splash2x/fft 59.705 58.382 (-2.22) 58.858 (-1.42) 45.949 (-23.04) 59.939 (0.39) 64.548 (8.11) + splash2x/lu_cb 132.552 131.604 (-0.72) 131.846 (-0.53) 132.320 (-0.18) 132.100 (-0.34) 140.289 (5.84) + splash2x/lu_ncb 150.215 149.670 (-0.36) 149.646 (-0.38) 148.823 (-0.93) 149.416 (-0.53) 152.338 (1.41) + splash2x/ocean_cp 84.033 76.405 (-9.08) 75.104 (-10.63) 73.487 (-12.55) 77.789 (-7.43) 77.380 (-7.92) + splash2x/ocean_ncp 153.833 154.247 (0.27) 156.227 (1.56) 106.619 (-30.69) 139.299 (-9.45) 165.030 (7.28) + splash2x/radiosity 143.566 143.654 (0.06) 142.426 (-0.79) 141.193 (-1.65) 141.740 (-1.27) 157.817 (9.93) + splash2x/radix 49.984 49.996 (0.02) 50.519 (1.07) 46.573 (-6.82) 50.724 (1.48) 50.695 (1.42) + splash2x/raytrace 133.238 134.337 (0.83) 134.389 (0.86) 134.833 (1.20) 131.073 (-1.62) 145.541 (9.23) + splash2x/volrend 121.700 120.652 (-0.86) 120.560 (-0.94) 120.629 (-0.88) 119.581 (-1.74) 129.422 (6.35) + splash2x/water_nsquared 370.771 375.236 (1.20) 376.829 (1.63) 355.592 (-4.09) 354.087 (-4.50) 419.606 (13.17) + splash2x/water_spatial 133.295 132.931 (-0.27) 132.762 (-0.40) 133.090 (-0.15) 133.809 (0.39) 153.647 (15.27) + total 4486.580 4538.750 (1.16) 4481.600 (-0.11) 4235.430 (-5.60) 4357.660 (-2.87) 4829.510 (7.64) + + + memused.avg orig rec (overhead) prec (overhead) thp (overhead) ethp (overhead) prcl (overhead) + parsec3/blackscholes 1828693.600 1834084.000 (0.29) 1823839.800 (-0.27) 1819296.600 (-0.51) 1830281.800 (0.09) 1603975.800 (-12.29) + parsec3/bodytrack 1424963.400 1440085.800 (1.06) 1438384.200 (0.94) 1421718.400 (-0.23) 1432834.600 (0.55) 1439283.000 (1.00) + parsec3/canneal 1036782.600 1052828.800 (1.55) 1050148.600 (1.29) 1035104.400 (-0.16) 1051145.400 (1.39) 1050019.400 (1.28) + parsec3/dedup 2511841.400 2507374.000 (-0.18) 2472450.600 (-1.57) 2523557.600 (0.47) 2508912.000 (-0.12) 2493347.200 (-0.74) + parsec3/facesim 537769.800 550740.800 (2.41) 548683.600 (2.03) 543547.800 (1.07) 560556.600 (4.24) 482782.600 (-10.23) + parsec3/fluidanimate 570268.600 585598.000 (2.69) 579837.800 (1.68) 571433.000 (0.20) 582112.800 (2.08) 470073.400 (-17.57) + parsec3/freqmine 982941.400 996253.200 (1.35) 993919.800 (1.12) 990531.800 (0.77) 1000994.400 (1.84) 750685.800 (-23.63) + parsec3/raytrace 1737446.000 1749908.800 (0.72) 1741183.800 (0.22) 1726674.800 (-0.62) 1748530.200 (0.64) 1552275.600 (-10.66) + parsec3/streamcluster 115857.000 155194.400 (33.95) 158272.800 (36.61) 122125.200 (5.41) 134545.600 (16.13) 133448.600 (15.18) + parsec3/swaptions 13694.200 28451.800 (107.76) 28464.600 (107.86) 12797.800 (-6.55) 25328.200 (84.96) 28138.400 (105.48) + parsec3/vips 2976126.400 3002408.600 (0.88) 3008218.800 (1.08) 2978258.600 (0.07) 2995428.600 (0.65) 2936338.600 (-1.34) + parsec3/x264 3233886.200 3258790.200 (0.77) 3248355.000 (0.45) 3232070.000 (-0.06) 3256360.200 (0.69) 3254707.400 (0.64) + splash2x/barnes 1210470.600 1211918.600 (0.12) 1204507.000 (-0.49) 1210892.800 (0.03) 1217414.800 (0.57) 944053.400 (-22.01) + splash2x/fft 9697440.000 9604535.600 (-0.96) 9210571.800 (-5.02) 9867368.000 (1.75) 9637571.800 (-0.62) 9804092.000 (1.10) + splash2x/lu_cb 510680.400 521792.600 (2.18) 517724.600 (1.38) 513500.800 (0.55) 519980.600 (1.82) 351787.000 (-31.11) + splash2x/lu_ncb 512896.200 529353.600 (3.21) 521248.600 (1.63) 513493.200 (0.12) 523793.400 (2.12) 418701.600 (-18.37) + splash2x/ocean_cp 3320800.200 3313688.400 (-0.21) 3225585.000 (-2.87) 3359032.200 (1.15) 3316591.800 (-0.13) 3304702.200 (-0.48) + splash2x/ocean_ncp 3915132.400 3917401.000 (0.06) 3884086.400 (-0.79) 7050398.600 (80.08) 4532528.600 (15.77) 3920395.800 (0.13) + splash2x/radiosity 1456908.200 1467611.800 (0.73) 1453612.600 (-0.23) 1466695.400 (0.67) 1467495.600 (0.73) 421197.600 (-71.09) + splash2x/radix 2345874.600 2318202.200 (-1.18) 2261499.200 (-3.60) 2438228.400 (3.94) 2373697.800 (1.19) 2336605.600 (-0.40) + splash2x/raytrace 43258.800 57624.200 (33.21) 55164.600 (27.52) 46204.400 (6.81) 60475.000 (39.80) 48865.400 (12.96) + splash2x/volrend 149615.000 163809.400 (9.49) 162115.400 (8.36) 149119.600 (-0.33) 162747.800 (8.78) 157734.600 (5.43) + splash2x/water_nsquared 40384.400 54848.600 (35.82) 53796.600 (33.21) 41455.800 (2.65) 53226.400 (31.80) 58260.600 (44.27) + splash2x/water_spatial 670580.200 680444.200 (1.47) 670020.400 (-0.08) 668262.400 (-0.35) 678552.000 (1.19) 372931.000 (-44.39) + total 40844300.000 41002900.000 (0.39) 40311600.000 (-1.30) 44301900.000 (8.47) 41671052.000 (2.02) 38334431.000 (-6.14) + + +DAMON Overheads +--------------- + +In total, DAMON virtual memory access recording feature ('rec') incurs 1.16% +runtime overhead and 0.39% memory space overhead. Even though the size of the +monitoring target region becomes much larger with the physical memory access +recording ('prec'), it still shows only modest amount of overhead (-0.11% for +runtime and -1.30% for memory footprint). + +For a convenient test run of 'rec' and 'prec', I use a Python wrapper. The +wrapper constantly consumes about 10-15MB of memory. This becomes a high +memory overhead if the target workload has a small memory footprint. +Nonetheless, the overheads are not from DAMON, but from the wrapper, and thus +should be ignored. This fake memory overhead continues in 'ethp' and 'prcl', +as those configurations are also using the Python wrapper. + + +Efficient THP +------------- + +THP 'always' enabled policy achieves 5.60% speedup but incurs 8.47% memory +overhead. It achieves 30.69% speedup in the best case, but 80.08% memory +overhead in the worst case. Interestingly, both the best and worst-case are +with 'splash2x/ocean_ncp'). + +The 2-lines implementation of data access monitoring based THP version ('ethp') +shows 2.87% speedup and 2.02% memory overhead. In other words, 'ethp' removes +76.15% of THP memory waste while preserving 51.25% of THP speedup in total. In +the case of the 'splash2x/ocean_ncp', 'ethp' removes 80.30% of THP memory waste +while preserving 30.79% of THP speedup. + + +Proactive Reclamation +--------------------- + +As similar to the original work, I use 4G 'zram' swap device for this +configuration. Also note that we use DAMOOS-tuned ethp schemes for each +workload. + +In total, our 1 line implementation of Proactive Reclamation, 'prcl', incurred +7.64% runtime overhead in total while achieving 6.14% system memory footprint +reduction. Even in the worst case, the runtime overhead was only 16.02%. + +Nonetheless, as the memory usage is calculated with 'MemFree' in +'/proc/meminfo', it contains the SwapCached pages. As the swapcached pages can +be easily evicted, I also measured the residential set size of the workloads:: + + rss.avg orig rec (overhead) prec (overhead) thp (overhead) ethp (overhead) prcl (overhead) + parsec3/blackscholes 587536.800 585720.000 (-0.31) 586233.400 (-0.22) 587045.400 (-0.08) 586753.400 (-0.13) 252207.400 (-57.07) + parsec3/bodytrack 32302.200 32290.600 (-0.04) 32261.800 (-0.13) 32215.800 (-0.27) 32173.000 (-0.40) 6798.800 (-78.95) + parsec3/canneal 842370.600 841443.400 (-0.11) 844012.400 (0.19) 838074.400 (-0.51) 841700.800 (-0.08) 840804.000 (-0.19) + parsec3/dedup 1180414.800 1164634.600 (-1.34) 1188886.200 (0.72) 1207821.000 (2.32) 1193896.200 (1.14) 572359.200 (-51.51) + parsec3/facesim 311848.400 311709.800 (-0.04) 311790.800 (-0.02) 317345.800 (1.76) 315443.400 (1.15) 188488.000 (-39.56) + parsec3/fluidanimate 531868.000 531885.600 (0.00) 531828.800 (-0.01) 532988.000 (0.21) 532959.600 (0.21) 415153.200 (-21.94) + parsec3/freqmine 552491.000 552718.600 (0.04) 552807.200 (0.06) 556574.200 (0.74) 554374.600 (0.34) 36573.400 (-93.38) + parsec3/raytrace 879683.400 880752.200 (0.12) 879907.000 (0.03) 870631.000 (-1.03) 880952.200 (0.14) 293119.200 (-66.68) + parsec3/streamcluster 110991.800 110937.200 (-0.05) 110964.600 (-0.02) 115606.800 (4.16) 116199.000 (4.69) 110108.200 (-0.80) + parsec3/swaptions 5665.000 5718.400 (0.94) 5720.600 (0.98) 5682.200 (0.30) 5628.600 (-0.64) 3613.800 (-36.21) + parsec3/vips 32143.600 31823.200 (-1.00) 31912.200 (-0.72) 33164.200 (3.18) 33925.800 (5.54) 27813.600 (-13.47) + parsec3/x264 81534.000 81811.000 (0.34) 81708.400 (0.21) 83052.400 (1.86) 83758.800 (2.73) 81691.800 (0.19) + splash2x/barnes 1220515.200 1218291.200 (-0.18) 1217699.600 (-0.23) 1228551.600 (0.66) 1220669.800 (0.01) 681096.000 (-44.20) + splash2x/fft 9915850.400 10036461.000 (1.22) 9881242.800 (-0.35) 10334603.600 (4.22) 10006993.200 (0.92) 8975181.200 (-9.49) + splash2x/lu_cb 511327.200 511679.000 (0.07) 511761.600 (0.08) 511971.600 (0.13) 511711.200 (0.08) 338005.000 (-33.90) + splash2x/lu_ncb 511505.000 506816.800 (-0.92) 511392.800 (-0.02) 496623.000 (-2.91) 511410.200 (-0.02) 404734.000 (-20.87) + splash2x/ocean_cp 3398834.000 3405017.800 (0.18) 3415287.800 (0.48) 3443604.600 (1.32) 3416264.200 (0.51) 3387134.000 (-0.34) + splash2x/ocean_ncp 3947092.800 3939805.400 (-0.18) 3952311.600 (0.13) 7165858.800 (81.55) 4610075.000 (16.80) 3944753.400 (-0.06) + splash2x/radiosity 1475024.000 1474053.200 (-0.07) 1475032.400 (0.00) 1483718.800 (0.59) 1475919.600 (0.06) 99637.200 (-93.25) + splash2x/radix 2431302.200 2416928.600 (-0.59) 2455596.800 (1.00) 2568526.400 (5.64) 2479966.800 (2.00) 2437406.600 (0.25) + splash2x/raytrace 23274.400 23278.400 (0.02) 23287.200 (0.05) 28828.000 (23.86) 27800.200 (19.45) 5667.000 (-75.65) + splash2x/volrend 44106.800 44151.400 (0.10) 44186.000 (0.18) 45200.400 (2.48) 44751.200 (1.46) 16912.000 (-61.66) + splash2x/water_nsquared 29427.200 29425.600 (-0.01) 29402.400 (-0.08) 28055.400 (-4.66) 28572.400 (-2.90) 13207.800 (-55.12) + splash2x/water_spatial 664312.200 664095.600 (-0.03) 663025.200 (-0.19) 664100.600 (-0.03) 663597.400 (-0.11) 261214.200 (-60.68) + total 29321300.000 29401500.000 (0.27) 29338300.000 (0.06) 33179900.000 (13.16) 30175600.000 (2.91) 23393600.000 (-20.22) + +In total, 20.22% of residential sets were reduced. + +With parsec3/freqmine, 'prcl' reduced 93.38% of residential sets and 23.63% of +system memory usage while incurring only 1.22% runtime overhead. diff --git a/Documentation/vm/damon/faq.rst b/Documentation/vm/damon/faq.rst new file mode 100644 index 000000000000..088128bbf22b --- /dev/null +++ b/Documentation/vm/damon/faq.rst @@ -0,0 +1,58 @@ +.. SPDX-License-Identifier: GPL-2.0 + +========================== +Frequently Asked Questions +========================== + +Why a new subsystem, instead of extending perf or other user space tools? +========================================================================= + +First, because it needs to be lightweight as much as possible so that it can be +used online, any unnecessary overhead such as kernel - user space context +switching cost should be avoided. Second, DAMON aims to be used by other +programs including the kernel. Therefore, having a dependency on specific +tools like perf is not desirable. These are the two biggest reasons why DAMON +is implemented in the kernel space. + + +Can 'idle pages tracking' or 'perf mem' substitute DAMON? +========================================================= + +Idle page tracking is a low level primitive for access check of the physical +address space. 'perf mem' is similar, though it can use sampling to minimize +the overhead. On the other hand, DAMON is a higher-level framework for the +monitoring of various address spaces. It is focused on memory management +optimization and provides sophisticated accuracy/overhead handling mechanisms. +Therefore, 'idle pages tracking' and 'perf mem' could provide a subset of +DAMON's output, but cannot substitute DAMON. + + +How can I optimize my system's memory management using DAMON? +============================================================= + +Because there are several ways for the DAMON-based optimizations, we wrote a +separate document, :doc:`/admin-guide/mm/damon/guide`. Please refer to that. + + +Does DAMON support virtual memory only? +======================================= + +No. The core of the DAMON is address space independent. The address space +specific low level primitive parts including monitoring target regions +constructions and actual access checks can be implemented and configured on the +DAMON core by the users. In this way, DAMON users can monitor any address +space with any access check technique. + +Nonetheless, DAMON provides vma tracking and PTE Accessed bit check based +implementations of the address space dependent functions for the virtual memory +by default, for a reference and convenient use. In near future, we will +provide those for physical memory address space. + + +Can I simply monitor page granularity? +====================================== + +Yes. You can do so by setting the ``min_nr_regions`` attribute higher than the +working set size divided by the page size. Because the monitoring target +regions size is forced to be ``>=page size``, the region split will make no +effect. diff --git a/Documentation/vm/damon/index.rst b/Documentation/vm/damon/index.rst new file mode 100644 index 000000000000..17dca3c12aad --- /dev/null +++ b/Documentation/vm/damon/index.rst @@ -0,0 +1,31 @@ +.. SPDX-License-Identifier: GPL-2.0 + +========================== +DAMON: Data Access MONitor +========================== + +DAMON is a data access monitoring framework subsystem for the Linux kernel. +The core mechanisms of DAMON (refer to :doc:`design` for the detail) make it + + - *accurate* (the monitoring output is useful enough for DRAM level memory + management; It might not appropriate for CPU Cache levels, though), + - *light-weight* (the monitoring overhead is low enough to be applied online), + and + - *scalable* (the upper-bound of the overhead is in constant range regardless + of the size of target workloads). + +Using this framework, therefore, the kernel's memory management mechanisms can +make advanced decisions. Experimental memory management optimization works +that incurring high data accesses monitoring overhead could implemented again. +In user space, meanwhile, users who have some special workloads can write +personalized applications for better understanding and optimizations of their +workloads and systems. + +.. toctree:: + :maxdepth: 2 + + faq + design + eval + api + plans diff --git a/Documentation/vm/index.rst b/Documentation/vm/index.rst index eff5fbd492d0..b51f0d8992f8 100644 --- a/Documentation/vm/index.rst +++ b/Documentation/vm/index.rst @@ -32,6 +32,7 @@ descriptions of data structures and algorithms. arch_pgtable_helpers balance cleancache + damon/index free_page_reporting frontswap highmem From patchwork Thu Mar 18 10:08:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 12147811 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6F46BC433DB for ; Thu, 18 Mar 2021 10:10:33 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id DA78364F38 for ; Thu, 18 Mar 2021 10:10:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DA78364F38 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8E2C38D0007; Thu, 18 Mar 2021 06:10:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 894868D0001; Thu, 18 Mar 2021 06:10:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 64AD28D0007; Thu, 18 Mar 2021 06:10:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0184.hostedemail.com [216.40.44.184]) by kanga.kvack.org (Postfix) with ESMTP id 3C5808D0001 for ; Thu, 18 Mar 2021 06:10:26 -0400 (EDT) Received: from smtpin02.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id EF5A11812D6D3 for ; Thu, 18 Mar 2021 10:10:25 +0000 (UTC) X-FDA: 77932575210.02.1B000B7 Received: from mail-qk1-f177.google.com (mail-qk1-f177.google.com [209.85.222.177]) by imf24.hostedemail.com (Postfix) with ESMTP id 37CDEA0009F4 for ; Thu, 18 Mar 2021 10:10:25 +0000 (UTC) Received: by mail-qk1-f177.google.com with SMTP id a9so1339421qkn.13 for ; Thu, 18 Mar 2021 03:10:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=vK2glIyejQn7WFd9b5udCRt9IlF5fmyG7DnRcvqscvA=; b=R5jBhANi3HjEb/yMu4S1xbiytdCmeMM4abvYWxg7mSUaezMum7sVg0p+8hRyU2756y spMcOtW3ZGfW5Lc89eZ0oAW2DJfDozLdHhXo/WhW82d7PGz1pb4Yf1Xp8MQuTua+2VVG yNOuD28oWqXsC+vAwLD6eujbvBePvtx5ONYSLHLEZnKW5+o8UENTFw2Xa4ApQErLK0Rs Tyddi6+p5p8+MW9SOHBOC9xa8wQ7OaKWJVq6rcS1jCq7vAPXlPur9UsLRzYjVSoBZgZ2 Yiv7JpFzpqWZmuh6pNVzBksHesq49qd9Oz7CEenG5vWKi+6mG++ofYgFESja62T8gavA euWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=vK2glIyejQn7WFd9b5udCRt9IlF5fmyG7DnRcvqscvA=; b=GOv9gJ8cE7LAD+GJDOHac5cGAkBoZd/c9OfWTYnXCJQMIwy/X/4L6Y4jYaHy2ZxFNV Ofa+4OrAvzIyh4nL306lbnmARa/vD+rOM6zoD8QemztFmHfH6ecZNx6kwXdpSYnyyUCb p/gf2K6ixQVj1v6G9swzOMDFX9KyINlhSgA2Qsl5/eIsqkb7rNUUksiKV8xm11KV1hrz ZHw2FCpeHg/VT9d/lut1LhGJJ8NU8Z2KuIIP4vT99Ut7jBiNxKyHsFE5My/CYT89VaPk x2Bf5q+sSH/X+I3e+Whn9bNGV6dTuCGO/IqE9V07bVkjNDG6yscG3fIXeHSnYEseJNEX yuag== X-Gm-Message-State: AOAM533Fm/NWDAIgj96j0X5EbOGxurEIVHaNMz4gF6T8gMuaGDN3ABTs udW6kkxqX4qRvUxq19Nmf84= X-Google-Smtp-Source: ABdhPJzrpnWSATlN1vELRqGgihwbtz9SEi0dHZWI0LrA0i1Hf5ZmQlWKbbc/QZ4a/J7TUG/Lbf3Jnw== X-Received: by 2002:a37:886:: with SMTP id 128mr3603832qki.430.1616062224276; Thu, 18 Mar 2021 03:10:24 -0700 (PDT) Received: from localhost.localdomain (ec2-35-169-212-159.compute-1.amazonaws.com. [35.169.212.159]) by smtp.gmail.com with ESMTPSA id d16sm1358101qka.43.2021.03.18.03.10.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 18 Mar 2021 03:10:23 -0700 (PDT) From: sj38.park@gmail.com To: akpm@linux-foundation.org Cc: SeongJae Park , Jonathan.Cameron@Huawei.com, acme@kernel.org, alexander.shishkin@linux.intel.com, amit@kernel.org, benh@kernel.crashing.org, brendanhiggins@google.com, corbet@lwn.net, david@redhat.com, dwmw@amazon.com, elver@google.com, fan.du@intel.com, foersleo@amazon.de, gthelen@google.com, mgorman@suse.de, minchan@kernel.org, mingo@redhat.com, namhyung@kernel.org, peterz@infradead.org, riel@surriel.com, rientjes@google.com, rostedt@goodmis.org, rppt@kernel.org, shakeelb@google.com, shuah@kernel.org, sj38.park@gmail.com, snu@amazon.de, vbabka@suse.cz, vdavydov.dev@gmail.com, zgf574564920@gmail.com, linux-damon@amazon.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v25 11/13] mm/damon: Add kunit tests Date: Thu, 18 Mar 2021 10:08:54 +0000 Message-Id: <20210318100856.34715-12-sj38.park@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210318100856.34715-1-sj38.park@gmail.com> References: <20210318100856.34715-1-sj38.park@gmail.com> X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 37CDEA0009F4 X-Stat-Signature: h67rhrc9bp3ud8c6fny1sr3pux8i5ysj Received-SPF: none (gmail.com>: No applicable sender policy available) receiver=imf24; identity=mailfrom; envelope-from=""; helo=mail-qk1-f177.google.com; client-ip=209.85.222.177 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1616062225-831357 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park This commit adds kunit based unit tests for the core and the virtual address spaces monitoring primitives of DAMON. Signed-off-by: SeongJae Park Reviewed-by: Brendan Higgins --- mm/damon/Kconfig | 36 +++++ mm/damon/core-test.h | 253 ++++++++++++++++++++++++++++++++ mm/damon/core.c | 7 + mm/damon/dbgfs-test.h | 214 +++++++++++++++++++++++++++ mm/damon/dbgfs.c | 2 + mm/damon/vaddr-test.h | 328 ++++++++++++++++++++++++++++++++++++++++++ mm/damon/vaddr.c | 7 + 7 files changed, 847 insertions(+) create mode 100644 mm/damon/core-test.h create mode 100644 mm/damon/dbgfs-test.h create mode 100644 mm/damon/vaddr-test.h diff --git a/mm/damon/Kconfig b/mm/damon/Kconfig index 72f1683ba0ee..455995152697 100644 --- a/mm/damon/Kconfig +++ b/mm/damon/Kconfig @@ -12,6 +12,18 @@ config DAMON See https://damonitor.github.io/doc/html/latest-damon/index.html for more information. +config DAMON_KUNIT_TEST + bool "Test for damon" if !KUNIT_ALL_TESTS + depends on DAMON && KUNIT=y + default KUNIT_ALL_TESTS + help + This builds the DAMON Kunit test suite. + + For more information on KUnit and unit tests in general, please refer + to the KUnit documentation. + + If unsure, say N. + config DAMON_VADDR bool "Data access monitoring primitives for virtual address spaces" depends on DAMON && MMU @@ -21,6 +33,18 @@ config DAMON_VADDR This builds the default data access monitoring primitives for DAMON that works for virtual address spaces. +config DAMON_VADDR_KUNIT_TEST + bool "Test for DAMON primitives" if !KUNIT_ALL_TESTS + depends on DAMON_VADDR && KUNIT=y + default KUNIT_ALL_TESTS + help + This builds the DAMON virtual addresses primitives Kunit test suite. + + For more information on KUnit and unit tests in general, please refer + to the KUnit documentation. + + If unsure, say N. + config DAMON_DBGFS bool "DAMON debugfs interface" depends on DAMON_VADDR && DEBUG_FS @@ -30,4 +54,16 @@ config DAMON_DBGFS If unsure, say N. +config DAMON_DBGFS_KUNIT_TEST + bool "Test for damon debugfs interface" if !KUNIT_ALL_TESTS + depends on DAMON_DBGFS && KUNIT=y + default KUNIT_ALL_TESTS + help + This builds the DAMON debugfs interface Kunit test suite. + + For more information on KUnit and unit tests in general, please refer + to the KUnit documentation. + + If unsure, say N. + endmenu diff --git a/mm/damon/core-test.h b/mm/damon/core-test.h new file mode 100644 index 000000000000..b815dfbfb5fd --- /dev/null +++ b/mm/damon/core-test.h @@ -0,0 +1,253 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Data Access Monitor Unit Tests + * + * Copyright 2019 Amazon.com, Inc. or its affiliates. All rights reserved. + * + * Author: SeongJae Park + */ + +#ifdef CONFIG_DAMON_KUNIT_TEST + +#ifndef _DAMON_CORE_TEST_H +#define _DAMON_CORE_TEST_H + +#include + +static void damon_test_regions(struct kunit *test) +{ + struct damon_region *r; + struct damon_target *t; + + r = damon_new_region(1, 2); + KUNIT_EXPECT_EQ(test, 1ul, r->ar.start); + KUNIT_EXPECT_EQ(test, 2ul, r->ar.end); + KUNIT_EXPECT_EQ(test, 0u, r->nr_accesses); + + t = damon_new_target(42); + KUNIT_EXPECT_EQ(test, 0u, damon_nr_regions(t)); + + damon_add_region(r, t); + KUNIT_EXPECT_EQ(test, 1u, damon_nr_regions(t)); + + damon_del_region(r); + KUNIT_EXPECT_EQ(test, 0u, damon_nr_regions(t)); + + damon_free_target(t); +} + +static unsigned int nr_damon_targets(struct damon_ctx *ctx) +{ + struct damon_target *t; + unsigned int nr_targets = 0; + + damon_for_each_target(t, ctx) + nr_targets++; + + return nr_targets; +} + +static void damon_test_target(struct kunit *test) +{ + struct damon_ctx *c = damon_new_ctx(); + struct damon_target *t; + + t = damon_new_target(42); + KUNIT_EXPECT_EQ(test, 42ul, t->id); + KUNIT_EXPECT_EQ(test, 0u, nr_damon_targets(c)); + + damon_add_target(c, t); + KUNIT_EXPECT_EQ(test, 1u, nr_damon_targets(c)); + + damon_destroy_target(t); + KUNIT_EXPECT_EQ(test, 0u, nr_damon_targets(c)); + + damon_destroy_ctx(c); +} + +/* + * Test kdamond_reset_aggregated() + * + * DAMON checks access to each region and aggregates this information as the + * access frequency of each region. In detail, it increases '->nr_accesses' of + * regions that an access has confirmed. 'kdamond_reset_aggregated()' flushes + * the aggregated information ('->nr_accesses' of each regions) to the result + * buffer. As a result of the flushing, the '->nr_accesses' of regions are + * initialized to zero. + */ +static void damon_test_aggregate(struct kunit *test) +{ + struct damon_ctx *ctx = damon_new_ctx(); + unsigned long target_ids[] = {1, 2, 3}; + unsigned long saddr[][3] = {{10, 20, 30}, {5, 42, 49}, {13, 33, 55} }; + unsigned long eaddr[][3] = {{15, 27, 40}, {31, 45, 55}, {23, 44, 66} }; + unsigned long accesses[][3] = {{42, 95, 84}, {10, 20, 30}, {0, 1, 2} }; + struct damon_target *t; + struct damon_region *r; + int it, ir; + + damon_set_targets(ctx, target_ids, 3); + + it = 0; + damon_for_each_target(t, ctx) { + for (ir = 0; ir < 3; ir++) { + r = damon_new_region(saddr[it][ir], eaddr[it][ir]); + r->nr_accesses = accesses[it][ir]; + damon_add_region(r, t); + } + it++; + } + kdamond_reset_aggregated(ctx); + it = 0; + damon_for_each_target(t, ctx) { + ir = 0; + /* '->nr_accesses' should be zeroed */ + damon_for_each_region(r, t) { + KUNIT_EXPECT_EQ(test, 0u, r->nr_accesses); + ir++; + } + /* regions should be preserved */ + KUNIT_EXPECT_EQ(test, 3, ir); + it++; + } + /* targets also should be preserved */ + KUNIT_EXPECT_EQ(test, 3, it); + + damon_destroy_ctx(ctx); +} + +static void damon_test_split_at(struct kunit *test) +{ + struct damon_ctx *c = damon_new_ctx(); + struct damon_target *t; + struct damon_region *r; + + t = damon_new_target(42); + r = damon_new_region(0, 100); + damon_add_region(r, t); + damon_split_region_at(c, r, 25); + KUNIT_EXPECT_EQ(test, r->ar.start, 0ul); + KUNIT_EXPECT_EQ(test, r->ar.end, 25ul); + + r = damon_next_region(r); + KUNIT_EXPECT_EQ(test, r->ar.start, 25ul); + KUNIT_EXPECT_EQ(test, r->ar.end, 100ul); + + damon_free_target(t); + damon_destroy_ctx(c); +} + +static void damon_test_merge_two(struct kunit *test) +{ + struct damon_target *t; + struct damon_region *r, *r2, *r3; + int i; + + t = damon_new_target(42); + r = damon_new_region(0, 100); + r->nr_accesses = 10; + damon_add_region(r, t); + r2 = damon_new_region(100, 300); + r2->nr_accesses = 20; + damon_add_region(r2, t); + + damon_merge_two_regions(r, r2); + KUNIT_EXPECT_EQ(test, r->ar.start, 0ul); + KUNIT_EXPECT_EQ(test, r->ar.end, 300ul); + KUNIT_EXPECT_EQ(test, r->nr_accesses, 16u); + + i = 0; + damon_for_each_region(r3, t) { + KUNIT_EXPECT_PTR_EQ(test, r, r3); + i++; + } + KUNIT_EXPECT_EQ(test, i, 1); + + damon_free_target(t); +} + +static struct damon_region *__nth_region_of(struct damon_target *t, int idx) +{ + struct damon_region *r; + unsigned int i = 0; + + damon_for_each_region(r, t) { + if (i++ == idx) + return r; + } + + return NULL; +} + +static void damon_test_merge_regions_of(struct kunit *test) +{ + struct damon_target *t; + struct damon_region *r; + unsigned long sa[] = {0, 100, 114, 122, 130, 156, 170, 184}; + unsigned long ea[] = {100, 112, 122, 130, 156, 170, 184, 230}; + unsigned int nrs[] = {0, 0, 10, 10, 20, 30, 1, 2}; + + unsigned long saddrs[] = {0, 114, 130, 156, 170}; + unsigned long eaddrs[] = {112, 130, 156, 170, 230}; + int i; + + t = damon_new_target(42); + for (i = 0; i < ARRAY_SIZE(sa); i++) { + r = damon_new_region(sa[i], ea[i]); + r->nr_accesses = nrs[i]; + damon_add_region(r, t); + } + + damon_merge_regions_of(t, 9, 9999); + /* 0-112, 114-130, 130-156, 156-170 */ + KUNIT_EXPECT_EQ(test, damon_nr_regions(t), 5u); + for (i = 0; i < 5; i++) { + r = __nth_region_of(t, i); + KUNIT_EXPECT_EQ(test, r->ar.start, saddrs[i]); + KUNIT_EXPECT_EQ(test, r->ar.end, eaddrs[i]); + } + damon_free_target(t); +} + +static void damon_test_split_regions_of(struct kunit *test) +{ + struct damon_ctx *c = damon_new_ctx(); + struct damon_target *t; + struct damon_region *r; + + t = damon_new_target(42); + r = damon_new_region(0, 22); + damon_add_region(r, t); + damon_split_regions_of(c, t, 2); + KUNIT_EXPECT_EQ(test, damon_nr_regions(t), 2u); + damon_free_target(t); + + t = damon_new_target(42); + r = damon_new_region(0, 220); + damon_add_region(r, t); + damon_split_regions_of(c, t, 4); + KUNIT_EXPECT_EQ(test, damon_nr_regions(t), 4u); + damon_free_target(t); + damon_destroy_ctx(c); +} + +static struct kunit_case damon_test_cases[] = { + KUNIT_CASE(damon_test_target), + KUNIT_CASE(damon_test_regions), + KUNIT_CASE(damon_test_aggregate), + KUNIT_CASE(damon_test_split_at), + KUNIT_CASE(damon_test_merge_two), + KUNIT_CASE(damon_test_merge_regions_of), + KUNIT_CASE(damon_test_split_regions_of), + {}, +}; + +static struct kunit_suite damon_test_suite = { + .name = "damon", + .test_cases = damon_test_cases, +}; +kunit_test_suite(damon_test_suite); + +#endif /* _DAMON_CORE_TEST_H */ + +#endif /* CONFIG_DAMON_KUNIT_TEST */ diff --git a/mm/damon/core.c b/mm/damon/core.c index cad2b4cee39d..6bfb5d7d9142 100644 --- a/mm/damon/core.c +++ b/mm/damon/core.c @@ -16,6 +16,11 @@ #define CREATE_TRACE_POINTS #include +#ifdef CONFIG_DAMON_KUNIT_TEST +#undef DAMON_MIN_REGION +#define DAMON_MIN_REGION 1 +#endif + /* Get a random number in [l, r) */ #define damon_rand(l, r) (l + prandom_u32_max(r - l)) @@ -709,3 +714,5 @@ static int kdamond_fn(void *data) do_exit(0); } + +#include "core-test.h" diff --git a/mm/damon/dbgfs-test.h b/mm/damon/dbgfs-test.h new file mode 100644 index 000000000000..ae6a23a95106 --- /dev/null +++ b/mm/damon/dbgfs-test.h @@ -0,0 +1,214 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * DAMON Debugfs Interface Unit Tests + * + * Author: SeongJae Park + */ + +#ifdef CONFIG_DAMON_DBGFS_KUNIT_TEST + +#ifndef _DAMON_DBGFS_TEST_H +#define _DAMON_DBGFS_TEST_H + +#include + +static void damon_dbgfs_test_str_to_target_ids(struct kunit *test) +{ + char *question; + unsigned long *answers; + unsigned long expected[] = {12, 35, 46}; + ssize_t nr_integers = 0, i; + + question = "123"; + answers = str_to_target_ids(question, strnlen(question, 128), + &nr_integers); + KUNIT_EXPECT_EQ(test, (ssize_t)1, nr_integers); + KUNIT_EXPECT_EQ(test, 123ul, answers[0]); + kfree(answers); + + question = "123abc"; + answers = str_to_target_ids(question, strnlen(question, 128), + &nr_integers); + KUNIT_EXPECT_EQ(test, (ssize_t)1, nr_integers); + KUNIT_EXPECT_EQ(test, 123ul, answers[0]); + kfree(answers); + + question = "a123"; + answers = str_to_target_ids(question, strnlen(question, 128), + &nr_integers); + KUNIT_EXPECT_EQ(test, (ssize_t)0, nr_integers); + kfree(answers); + + question = "12 35"; + answers = str_to_target_ids(question, strnlen(question, 128), + &nr_integers); + KUNIT_EXPECT_EQ(test, (ssize_t)2, nr_integers); + for (i = 0; i < nr_integers; i++) + KUNIT_EXPECT_EQ(test, expected[i], answers[i]); + kfree(answers); + + question = "12 35 46"; + answers = str_to_target_ids(question, strnlen(question, 128), + &nr_integers); + KUNIT_EXPECT_EQ(test, (ssize_t)3, nr_integers); + for (i = 0; i < nr_integers; i++) + KUNIT_EXPECT_EQ(test, expected[i], answers[i]); + kfree(answers); + + question = "12 35 abc 46"; + answers = str_to_target_ids(question, strnlen(question, 128), + &nr_integers); + KUNIT_EXPECT_EQ(test, (ssize_t)2, nr_integers); + for (i = 0; i < 2; i++) + KUNIT_EXPECT_EQ(test, expected[i], answers[i]); + kfree(answers); + + question = ""; + answers = str_to_target_ids(question, strnlen(question, 128), + &nr_integers); + KUNIT_EXPECT_EQ(test, (ssize_t)0, nr_integers); + kfree(answers); + + question = "\n"; + answers = str_to_target_ids(question, strnlen(question, 128), + &nr_integers); + KUNIT_EXPECT_EQ(test, (ssize_t)0, nr_integers); + kfree(answers); +} + +static void damon_dbgfs_test_set_targets(struct kunit *test) +{ + struct damon_ctx *ctx = dbgfs_new_ctx(); + unsigned long ids[] = {1, 2, 3}; + char buf[64]; + + /* Make DAMON consider target id as plain number */ + ctx->primitive.target_valid = NULL; + ctx->primitive.cleanup = NULL; + + damon_set_targets(ctx, ids, 3); + sprint_target_ids(ctx, buf, 64); + KUNIT_EXPECT_STREQ(test, (char *)buf, "1 2 3\n"); + + damon_set_targets(ctx, NULL, 0); + sprint_target_ids(ctx, buf, 64); + KUNIT_EXPECT_STREQ(test, (char *)buf, "\n"); + + damon_set_targets(ctx, (unsigned long []){1, 2}, 2); + sprint_target_ids(ctx, buf, 64); + KUNIT_EXPECT_STREQ(test, (char *)buf, "1 2\n"); + + damon_set_targets(ctx, (unsigned long []){2}, 1); + sprint_target_ids(ctx, buf, 64); + KUNIT_EXPECT_STREQ(test, (char *)buf, "2\n"); + + damon_set_targets(ctx, NULL, 0); + sprint_target_ids(ctx, buf, 64); + KUNIT_EXPECT_STREQ(test, (char *)buf, "\n"); + + dbgfs_destroy_ctx(ctx); +} + +static void damon_dbgfs_test_set_recording(struct kunit *test) +{ + struct damon_ctx *ctx = dbgfs_new_ctx(); + struct dbgfs_recorder *rec = ctx->callback.private; + int err; + + err = dbgfs_set_recording(ctx, 42, "foo"); + KUNIT_EXPECT_EQ(test, err, -EINVAL); + dbgfs_set_recording(ctx, 4242, "foo.bar"); + KUNIT_EXPECT_EQ(test, rec->rbuf_len, 4242u); + KUNIT_EXPECT_STREQ(test, rec->rfile_path, "foo.bar"); + dbgfs_set_recording(ctx, 424242, "foo"); + KUNIT_EXPECT_EQ(test, rec->rbuf_len, 424242u); + KUNIT_EXPECT_STREQ(test, rec->rfile_path, "foo"); + + dbgfs_destroy_ctx(ctx); +} + +static void damon_dbgfs_test_write_rbuf(struct kunit *test) +{ + struct damon_ctx *ctx = dbgfs_new_ctx(); + struct dbgfs_recorder *rec = ctx->callback.private; + char *data; + + dbgfs_set_recording(ctx, 4242, "damon.data"); + + data = "hello"; + dbgfs_write_rbuf(ctx, data, strnlen(data, 256)); + KUNIT_EXPECT_EQ(test, rec->rbuf_offset, 5u); + + dbgfs_write_rbuf(ctx, data, 0); + KUNIT_EXPECT_EQ(test, rec->rbuf_offset, 5u); + + KUNIT_EXPECT_STREQ(test, (char *)rec->rbuf, data); + + dbgfs_destroy_ctx(ctx); +} + +/* + * Test dbgfs_after_aggregation() + * + * dbgfs sets dbgfs_after_aggregation() as aggregate callback. It stores the + * aggregated monitoring information ('->nr_accesses' of each regions) to the + * result buffer. + */ +static void damon_dbgfs_test_aggregate(struct kunit *test) +{ + struct damon_ctx *ctx = dbgfs_new_ctx(); + struct dbgfs_recorder *rec = ctx->callback.private; + unsigned long target_ids[] = {1, 2, 3}; + unsigned long saddr[][3] = {{10, 20, 30}, {5, 42, 49}, {13, 33, 55} }; + unsigned long eaddr[][3] = {{15, 27, 40}, {31, 45, 55}, {23, 44, 66} }; + unsigned long accesses[][3] = {{42, 95, 84}, {10, 20, 30}, {0, 1, 2} }; + struct damon_target *t; + struct damon_region *r; + int it, ir; + ssize_t sz, sr, sp; + + /* Make DAMON consider target id as plain number */ + ctx->primitive.target_valid = NULL; + ctx->primitive.cleanup = NULL; + + dbgfs_set_recording(ctx, 4242, "damon.data"); + damon_set_targets(ctx, target_ids, 3); + + it = 0; + damon_for_each_target(t, ctx) { + for (ir = 0; ir < 3; ir++) { + r = damon_new_region(saddr[it][ir], eaddr[it][ir]); + r->nr_accesses = accesses[it][ir]; + damon_add_region(r, t); + } + it++; + } + dbgfs_after_aggregation(ctx); + + /* The aggregated information should be written in the buffer */ + sr = sizeof(r->ar.start) + sizeof(r->ar.end) + sizeof(r->nr_accesses); + sp = sizeof(t->id) + sizeof(unsigned int) + 3 * sr; + sz = sizeof(struct timespec64) + sizeof(unsigned int) + 3 * sp; + KUNIT_EXPECT_EQ(test, (unsigned int)sz, rec->rbuf_offset); + + damon_destroy_ctx(ctx); +} + +static struct kunit_case damon_test_cases[] = { + KUNIT_CASE(damon_dbgfs_test_str_to_target_ids), + KUNIT_CASE(damon_dbgfs_test_set_targets), + KUNIT_CASE(damon_dbgfs_test_set_recording), + KUNIT_CASE(damon_dbgfs_test_write_rbuf), + KUNIT_CASE(damon_dbgfs_test_aggregate), + {}, +}; + +static struct kunit_suite damon_test_suite = { + .name = "damon-dbgfs", + .test_cases = damon_test_cases, +}; +kunit_test_suite(damon_test_suite); + +#endif /* _DAMON_TEST_H */ + +#endif /* CONFIG_DAMON_KUNIT_TEST */ diff --git a/mm/damon/dbgfs.c b/mm/damon/dbgfs.c index 9edef931ed00..e22ac2bb9356 100644 --- a/mm/damon/dbgfs.c +++ b/mm/damon/dbgfs.c @@ -605,3 +605,5 @@ static int __init damon_dbgfs_init(void) } module_init(damon_dbgfs_init); + +#include "dbgfs-test.h" diff --git a/mm/damon/vaddr-test.h b/mm/damon/vaddr-test.h new file mode 100644 index 000000000000..c9394e1d21d3 --- /dev/null +++ b/mm/damon/vaddr-test.h @@ -0,0 +1,328 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Data Access Monitor Unit Tests + * + * Copyright 2019 Amazon.com, Inc. or its affiliates. All rights reserved. + * + * Author: SeongJae Park + */ + +#ifdef CONFIG_DAMON_VADDR_KUNIT_TEST + +#ifndef _DAMON_VADDR_TEST_H +#define _DAMON_VADDR_TEST_H + +#include + +static void __link_vmas(struct vm_area_struct *vmas, ssize_t nr_vmas) +{ + int i, j; + unsigned long largest_gap, gap; + + if (!nr_vmas) + return; + + for (i = 0; i < nr_vmas - 1; i++) { + vmas[i].vm_next = &vmas[i + 1]; + + vmas[i].vm_rb.rb_left = NULL; + vmas[i].vm_rb.rb_right = &vmas[i + 1].vm_rb; + + largest_gap = 0; + for (j = i; j < nr_vmas; j++) { + if (j == 0) + continue; + gap = vmas[j].vm_start - vmas[j - 1].vm_end; + if (gap > largest_gap) + largest_gap = gap; + } + vmas[i].rb_subtree_gap = largest_gap; + } + vmas[i].vm_next = NULL; + vmas[i].vm_rb.rb_right = NULL; + vmas[i].rb_subtree_gap = 0; +} + +/* + * Test __damon_va_three_regions() function + * + * In case of virtual memory address spaces monitoring, DAMON converts the + * complex and dynamic memory mappings of each target task to three + * discontiguous regions which cover every mapped areas. However, the three + * regions should not include the two biggest unmapped areas in the original + * mapping, because the two biggest areas are normally the areas between 1) + * heap and the mmap()-ed regions, and 2) the mmap()-ed regions and stack. + * Because these two unmapped areas are very huge but obviously never accessed, + * covering the region is just a waste. + * + * '__damon_va_three_regions() receives an address space of a process. It + * first identifies the start of mappings, end of mappings, and the two biggest + * unmapped areas. After that, based on the information, it constructs the + * three regions and returns. For more detail, refer to the comment of + * 'damon_init_regions_of()' function definition in 'mm/damon.c' file. + * + * For example, suppose virtual address ranges of 10-20, 20-25, 200-210, + * 210-220, 300-305, and 307-330 (Other comments represent this mappings in + * more short form: 10-20-25, 200-210-220, 300-305, 307-330) of a process are + * mapped. To cover every mappings, the three regions should start with 10, + * and end with 305. The process also has three unmapped areas, 25-200, + * 220-300, and 305-307. Among those, 25-200 and 220-300 are the biggest two + * unmapped areas, and thus it should be converted to three regions of 10-25, + * 200-220, and 300-330. + */ +static void damon_test_three_regions_in_vmas(struct kunit *test) +{ + struct damon_addr_range regions[3] = {0,}; + /* 10-20-25, 200-210-220, 300-305, 307-330 */ + struct vm_area_struct vmas[] = { + (struct vm_area_struct) {.vm_start = 10, .vm_end = 20}, + (struct vm_area_struct) {.vm_start = 20, .vm_end = 25}, + (struct vm_area_struct) {.vm_start = 200, .vm_end = 210}, + (struct vm_area_struct) {.vm_start = 210, .vm_end = 220}, + (struct vm_area_struct) {.vm_start = 300, .vm_end = 305}, + (struct vm_area_struct) {.vm_start = 307, .vm_end = 330}, + }; + + __link_vmas(vmas, 6); + + __damon_va_three_regions(&vmas[0], regions); + + KUNIT_EXPECT_EQ(test, 10ul, regions[0].start); + KUNIT_EXPECT_EQ(test, 25ul, regions[0].end); + KUNIT_EXPECT_EQ(test, 200ul, regions[1].start); + KUNIT_EXPECT_EQ(test, 220ul, regions[1].end); + KUNIT_EXPECT_EQ(test, 300ul, regions[2].start); + KUNIT_EXPECT_EQ(test, 330ul, regions[2].end); +} + +static struct damon_region *__nth_region_of(struct damon_target *t, int idx) +{ + struct damon_region *r; + unsigned int i = 0; + + damon_for_each_region(r, t) { + if (i++ == idx) + return r; + } + + return NULL; +} + +/* + * Test 'damon_va_apply_three_regions()' + * + * test kunit object + * regions an array containing start/end addresses of current + * monitoring target regions + * nr_regions the number of the addresses in 'regions' + * three_regions The three regions that need to be applied now + * expected start/end addresses of monitoring target regions that + * 'three_regions' are applied + * nr_expected the number of addresses in 'expected' + * + * The memory mapping of the target processes changes dynamically. To follow + * the change, DAMON periodically reads the mappings, simplifies it to the + * three regions, and updates the monitoring target regions to fit in the three + * regions. The update of current target regions is the role of + * 'damon_va_apply_three_regions()'. + * + * This test passes the given target regions and the new three regions that + * need to be applied to the function and check whether it updates the regions + * as expected. + */ +static void damon_do_test_apply_three_regions(struct kunit *test, + unsigned long *regions, int nr_regions, + struct damon_addr_range *three_regions, + unsigned long *expected, int nr_expected) +{ + struct damon_ctx *ctx = damon_new_ctx(); + struct damon_target *t; + struct damon_region *r; + int i; + + t = damon_new_target(42); + for (i = 0; i < nr_regions / 2; i++) { + r = damon_new_region(regions[i * 2], regions[i * 2 + 1]); + damon_add_region(r, t); + } + damon_add_target(ctx, t); + + damon_va_apply_three_regions(ctx, t, three_regions); + + for (i = 0; i < nr_expected / 2; i++) { + r = __nth_region_of(t, i); + KUNIT_EXPECT_EQ(test, r->ar.start, expected[i * 2]); + KUNIT_EXPECT_EQ(test, r->ar.end, expected[i * 2 + 1]); + } + + damon_destroy_ctx(ctx); +} + +/* + * This function test most common case where the three big regions are only + * slightly changed. Target regions should adjust their boundary (10-20-30, + * 50-55, 70-80, 90-100) to fit with the new big regions or remove target + * regions (57-79) that now out of the three regions. + */ +static void damon_test_apply_three_regions1(struct kunit *test) +{ + /* 10-20-30, 50-55-57-59, 70-80-90-100 */ + unsigned long regions[] = {10, 20, 20, 30, 50, 55, 55, 57, 57, 59, + 70, 80, 80, 90, 90, 100}; + /* 5-27, 45-55, 73-104 */ + struct damon_addr_range new_three_regions[3] = { + (struct damon_addr_range){.start = 5, .end = 27}, + (struct damon_addr_range){.start = 45, .end = 55}, + (struct damon_addr_range){.start = 73, .end = 104} }; + /* 5-20-27, 45-55, 73-80-90-104 */ + unsigned long expected[] = {5, 20, 20, 27, 45, 55, + 73, 80, 80, 90, 90, 104}; + + damon_do_test_apply_three_regions(test, regions, ARRAY_SIZE(regions), + new_three_regions, expected, ARRAY_SIZE(expected)); +} + +/* + * Test slightly bigger change. Similar to above, but the second big region + * now require two target regions (50-55, 57-59) to be removed. + */ +static void damon_test_apply_three_regions2(struct kunit *test) +{ + /* 10-20-30, 50-55-57-59, 70-80-90-100 */ + unsigned long regions[] = {10, 20, 20, 30, 50, 55, 55, 57, 57, 59, + 70, 80, 80, 90, 90, 100}; + /* 5-27, 56-57, 65-104 */ + struct damon_addr_range new_three_regions[3] = { + (struct damon_addr_range){.start = 5, .end = 27}, + (struct damon_addr_range){.start = 56, .end = 57}, + (struct damon_addr_range){.start = 65, .end = 104} }; + /* 5-20-27, 56-57, 65-80-90-104 */ + unsigned long expected[] = {5, 20, 20, 27, 56, 57, + 65, 80, 80, 90, 90, 104}; + + damon_do_test_apply_three_regions(test, regions, ARRAY_SIZE(regions), + new_three_regions, expected, ARRAY_SIZE(expected)); +} + +/* + * Test a big change. The second big region has totally freed and mapped to + * different area (50-59 -> 61-63). The target regions which were in the old + * second big region (50-55-57-59) should be removed and new target region + * covering the second big region (61-63) should be created. + */ +static void damon_test_apply_three_regions3(struct kunit *test) +{ + /* 10-20-30, 50-55-57-59, 70-80-90-100 */ + unsigned long regions[] = {10, 20, 20, 30, 50, 55, 55, 57, 57, 59, + 70, 80, 80, 90, 90, 100}; + /* 5-27, 61-63, 65-104 */ + struct damon_addr_range new_three_regions[3] = { + (struct damon_addr_range){.start = 5, .end = 27}, + (struct damon_addr_range){.start = 61, .end = 63}, + (struct damon_addr_range){.start = 65, .end = 104} }; + /* 5-20-27, 61-63, 65-80-90-104 */ + unsigned long expected[] = {5, 20, 20, 27, 61, 63, + 65, 80, 80, 90, 90, 104}; + + damon_do_test_apply_three_regions(test, regions, ARRAY_SIZE(regions), + new_three_regions, expected, ARRAY_SIZE(expected)); +} + +/* + * Test another big change. Both of the second and third big regions (50-59 + * and 70-100) has totally freed and mapped to different area (30-32 and + * 65-68). The target regions which were in the old second and third big + * regions should now be removed and new target regions covering the new second + * and third big regions should be crated. + */ +static void damon_test_apply_three_regions4(struct kunit *test) +{ + /* 10-20-30, 50-55-57-59, 70-80-90-100 */ + unsigned long regions[] = {10, 20, 20, 30, 50, 55, 55, 57, 57, 59, + 70, 80, 80, 90, 90, 100}; + /* 5-7, 30-32, 65-68 */ + struct damon_addr_range new_three_regions[3] = { + (struct damon_addr_range){.start = 5, .end = 7}, + (struct damon_addr_range){.start = 30, .end = 32}, + (struct damon_addr_range){.start = 65, .end = 68} }; + /* expect 5-7, 30-32, 65-68 */ + unsigned long expected[] = {5, 7, 30, 32, 65, 68}; + + damon_do_test_apply_three_regions(test, regions, ARRAY_SIZE(regions), + new_three_regions, expected, ARRAY_SIZE(expected)); +} + +static void damon_test_split_evenly(struct kunit *test) +{ + struct damon_ctx *c = damon_new_ctx(); + struct damon_target *t; + struct damon_region *r; + unsigned long i; + + KUNIT_EXPECT_EQ(test, damon_va_evenly_split_region(c, NULL, 5), -EINVAL); + + t = damon_new_target(42); + r = damon_new_region(0, 100); + KUNIT_EXPECT_EQ(test, damon_va_evenly_split_region(c, r, 0), -EINVAL); + + damon_add_region(r, t); + KUNIT_EXPECT_EQ(test, damon_va_evenly_split_region(c, r, 10), 0); + KUNIT_EXPECT_EQ(test, damon_nr_regions(t), 10u); + + i = 0; + damon_for_each_region(r, t) { + KUNIT_EXPECT_EQ(test, r->ar.start, i++ * 10); + KUNIT_EXPECT_EQ(test, r->ar.end, i * 10); + } + damon_free_target(t); + + t = damon_new_target(42); + r = damon_new_region(5, 59); + damon_add_region(r, t); + KUNIT_EXPECT_EQ(test, damon_va_evenly_split_region(c, r, 5), 0); + KUNIT_EXPECT_EQ(test, damon_nr_regions(t), 5u); + + i = 0; + damon_for_each_region(r, t) { + if (i == 4) + break; + KUNIT_EXPECT_EQ(test, r->ar.start, 5 + 10 * i++); + KUNIT_EXPECT_EQ(test, r->ar.end, 5 + 10 * i); + } + KUNIT_EXPECT_EQ(test, r->ar.start, 5 + 10 * i); + KUNIT_EXPECT_EQ(test, r->ar.end, 59ul); + damon_free_target(t); + + t = damon_new_target(42); + r = damon_new_region(5, 6); + damon_add_region(r, t); + KUNIT_EXPECT_EQ(test, damon_va_evenly_split_region(c, r, 2), -EINVAL); + KUNIT_EXPECT_EQ(test, damon_nr_regions(t), 1u); + + damon_for_each_region(r, t) { + KUNIT_EXPECT_EQ(test, r->ar.start, 5ul); + KUNIT_EXPECT_EQ(test, r->ar.end, 6ul); + } + damon_free_target(t); + damon_destroy_ctx(c); +} + +static struct kunit_case damon_test_cases[] = { + KUNIT_CASE(damon_test_three_regions_in_vmas), + KUNIT_CASE(damon_test_apply_three_regions1), + KUNIT_CASE(damon_test_apply_three_regions2), + KUNIT_CASE(damon_test_apply_three_regions3), + KUNIT_CASE(damon_test_apply_three_regions4), + KUNIT_CASE(damon_test_split_evenly), + {}, +}; + +static struct kunit_suite damon_test_suite = { + .name = "damon-primitives", + .test_cases = damon_test_cases, +}; +kunit_test_suite(damon_test_suite); + +#endif /* _DAMON_VADDR_TEST_H */ + +#endif /* CONFIG_DAMON_VADDR_KUNIT_TEST */ diff --git a/mm/damon/vaddr.c b/mm/damon/vaddr.c index e2bf3db31c2c..95262bdd4757 100644 --- a/mm/damon/vaddr.c +++ b/mm/damon/vaddr.c @@ -15,6 +15,11 @@ #include #include +#ifdef CONFIG_DAMON_VADDR_KUNIT_TEST +#undef DAMON_MIN_REGION +#define DAMON_MIN_REGION 1 +#endif + /* Get a random number in [l, r) */ #define damon_rand(l, r) (l + prandom_u32_max(r - l)) @@ -577,3 +582,5 @@ void damon_va_set_primitives(struct damon_ctx *ctx) ctx->primitive.target_valid = damon_va_target_valid; ctx->primitive.cleanup = damon_va_cleanup; } + +#include "vaddr-test.h" From patchwork Thu Mar 18 10:08:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 12147813 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3D571C433E9 for ; Thu, 18 Mar 2021 10:10:36 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C569261580 for ; Thu, 18 Mar 2021 10:10:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C569261580 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 1B6578D0008; Thu, 18 Mar 2021 06:10:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 145068D0001; Thu, 18 Mar 2021 06:10:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EB1DC8D0008; Thu, 18 Mar 2021 06:10:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0118.hostedemail.com [216.40.44.118]) by kanga.kvack.org (Postfix) with ESMTP id C4F608D0001 for ; Thu, 18 Mar 2021 06:10:27 -0400 (EDT) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 852A58249980 for ; Thu, 18 Mar 2021 10:10:27 +0000 (UTC) X-FDA: 77932575252.20.222BF5B Received: from mail-qk1-f174.google.com (mail-qk1-f174.google.com [209.85.222.174]) by imf10.hostedemail.com (Postfix) with ESMTP id 10F7E4080F52 for ; Thu, 18 Mar 2021 10:10:26 +0000 (UTC) Received: by mail-qk1-f174.google.com with SMTP id l132so1377384qke.7 for ; Thu, 18 Mar 2021 03:10:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=1L8j0imCjqj6Ox0Kf+3S46R4RWttBliVjR+yZC3m/8Y=; b=J/q4BVCGuPL7ibMfGIbmYctpttS7+xSZm6UnyhrEQLwUZMHdAjLFRPOojrB82nJMy3 kpRQXJv+gw66MPAEgdo+d96RRLlYq09AWzOYg19TbSLHbRz8axzAWMMpsl7bZPMqqhb9 mCE8o3Bh7sYjK574O/cgoBd+3QomriGGYq3Wn7d/s9XuPuLU/WViSqcecIETuUwaWLqY rHZkLqAtOEnhrHwSd+Xxr1ju0kkJn7gsKR5ED6F5etrSRlPIHHmtH7LURny9XRhSVZvx iTJRBL7jmSDeDV5rfV7pvX1ZWgPIDGSG+3Ul3LNoMTnmRk2k0XwtM124Y6WipH61pUa/ i0yw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=1L8j0imCjqj6Ox0Kf+3S46R4RWttBliVjR+yZC3m/8Y=; b=I8emGUqVGg38kWx4hMKGrdESktJ58Ng1Sjzhtxv1NIvOMgDYPTdBnE/MFk59PQuQUs rVpHOKMSrEkXfHnXGhok9Fa+nIL+4/spFN6S7jiQFXhPq3fna2V6VAdttV2gY7b6un8Z ymtRy4X/XeUZ7KrzlW90DiSM3DHQKAc43OEqTsO1IbCdEXw8wimcRPyfzgy+NzPfu/EM BnVyRcGkFj1uU6ehnbWtJec6yYSAXj8gXq8MmMiTsQXU5aHH+aKlItoKlqo9xeVCTeg4 LYevTauZA9LOzGPQP2Z+qG2+fNb31fJozx6+yjUybGevKcDgsB1+qxPx0qwsuG/Fz96w CUsQ== X-Gm-Message-State: AOAM532MjScmU6jrvD7YIrvz8ABAai8Wgem6++W3kHqt7EMZOwotrCP1 wi2Ewq4SAaSxujbFVz7V1s0= X-Google-Smtp-Source: ABdhPJz8woB7jquNSUdFa3WRYXwRQeAE4qB3+ESkaVFh2Csedy/kUIN7xvm/EGVk5WZUDQJBARl7gw== X-Received: by 2002:a37:8c42:: with SMTP id o63mr3432983qkd.449.1616062226388; Thu, 18 Mar 2021 03:10:26 -0700 (PDT) Received: from localhost.localdomain (ec2-35-169-212-159.compute-1.amazonaws.com. [35.169.212.159]) by smtp.gmail.com with ESMTPSA id d16sm1358101qka.43.2021.03.18.03.10.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 18 Mar 2021 03:10:26 -0700 (PDT) From: sj38.park@gmail.com To: akpm@linux-foundation.org Cc: SeongJae Park , Jonathan.Cameron@Huawei.com, acme@kernel.org, alexander.shishkin@linux.intel.com, amit@kernel.org, benh@kernel.crashing.org, brendanhiggins@google.com, corbet@lwn.net, david@redhat.com, dwmw@amazon.com, elver@google.com, fan.du@intel.com, foersleo@amazon.de, gthelen@google.com, mgorman@suse.de, minchan@kernel.org, mingo@redhat.com, namhyung@kernel.org, peterz@infradead.org, riel@surriel.com, rientjes@google.com, rostedt@goodmis.org, rppt@kernel.org, shakeelb@google.com, shuah@kernel.org, sj38.park@gmail.com, snu@amazon.de, vbabka@suse.cz, vdavydov.dev@gmail.com, zgf574564920@gmail.com, linux-damon@amazon.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v25 12/13] mm/damon: Add user space selftests Date: Thu, 18 Mar 2021 10:08:55 +0000 Message-Id: <20210318100856.34715-13-sj38.park@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210318100856.34715-1-sj38.park@gmail.com> References: <20210318100856.34715-1-sj38.park@gmail.com> X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 10F7E4080F52 X-Stat-Signature: ufwuc8mg5e8u7phjdbjdghng3nfdzcwj Received-SPF: none (gmail.com>: No applicable sender policy available) receiver=imf10; identity=mailfrom; envelope-from=""; helo=mail-qk1-f174.google.com; client-ip=209.85.222.174 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1616062226-821177 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000005, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park This commit adds a simple user space tests for DAMON. The tests are using kselftest framework. Signed-off-by: SeongJae Park --- tools/testing/selftests/damon/Makefile | 7 + .../selftests/damon/_chk_dependency.sh | 28 +++ tools/testing/selftests/damon/_chk_record.py | 109 ++++++++++++ .../testing/selftests/damon/debugfs_attrs.sh | 161 ++++++++++++++++++ .../testing/selftests/damon/debugfs_record.sh | 50 ++++++ 5 files changed, 355 insertions(+) create mode 100644 tools/testing/selftests/damon/Makefile create mode 100644 tools/testing/selftests/damon/_chk_dependency.sh create mode 100644 tools/testing/selftests/damon/_chk_record.py create mode 100755 tools/testing/selftests/damon/debugfs_attrs.sh create mode 100755 tools/testing/selftests/damon/debugfs_record.sh diff --git a/tools/testing/selftests/damon/Makefile b/tools/testing/selftests/damon/Makefile new file mode 100644 index 000000000000..cfd5393a4639 --- /dev/null +++ b/tools/testing/selftests/damon/Makefile @@ -0,0 +1,7 @@ +# SPDX-License-Identifier: GPL-2.0 +# Makefile for damon selftests + +TEST_FILES = _chk_dependency.sh _chk_record_file.py +TEST_PROGS = debugfs_attrs.sh debugfs_record.sh + +include ../lib.mk diff --git a/tools/testing/selftests/damon/_chk_dependency.sh b/tools/testing/selftests/damon/_chk_dependency.sh new file mode 100644 index 000000000000..b304b7779976 --- /dev/null +++ b/tools/testing/selftests/damon/_chk_dependency.sh @@ -0,0 +1,28 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 + +# Kselftest framework requirement - SKIP code is 4. +ksft_skip=4 + +DBGFS=/sys/kernel/debug/damon + +if [ $EUID -ne 0 ]; +then + echo "Run as root" + exit $ksft_skip +fi + +if [ ! -d $DBGFS ] +then + echo "$DBGFS not found" + exit $ksft_skip +fi + +for f in attrs record target_ids monitor_on +do + if [ ! -f "$DBGFS/$f" ] + then + echo "$f not found" + exit 1 + fi +done diff --git a/tools/testing/selftests/damon/_chk_record.py b/tools/testing/selftests/damon/_chk_record.py new file mode 100644 index 000000000000..73e128904319 --- /dev/null +++ b/tools/testing/selftests/damon/_chk_record.py @@ -0,0 +1,109 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-2.0 + +"Check whether the DAMON record file is valid" + +import argparse +import struct +import sys + +fmt_version = 0 + +def set_fmt_version(f): + global fmt_version + + mark = f.read(16) + if mark == b'damon_recfmt_ver': + fmt_version = struct.unpack('i', f.read(4))[0] + else: + fmt_version = 0 + f.seek(0) + return fmt_version + +def read_pid(f): + if fmt_version == 1: + pid = struct.unpack('i', f.read(4))[0] + else: + pid = struct.unpack('L', f.read(8))[0] + +def err_percent(val, expected): + return abs(val - expected) / expected * 100 + +def chk_task_info(f): + pid = read_pid(f) + nr_regions = struct.unpack('I', f.read(4))[0] + + if nr_regions > max_nr_regions: + print('too many regions: %d > %d' % (nr_regions, max_nr_regions)) + exit(1) + + nr_gaps = 0 + eaddr = 0 + for r in range(nr_regions): + saddr = struct.unpack('L', f.read(8))[0] + if eaddr and saddr != eaddr: + nr_gaps += 1 + eaddr = struct.unpack('L', f.read(8))[0] + nr_accesses = struct.unpack('I', f.read(4))[0] + + if saddr >= eaddr: + print('wrong region [%d,%d)' % (saddr, eaddr)) + exit(1) + + max_nr_accesses = aint / sint + if nr_accesses > max_nr_accesses: + if err_percent(nr_accesses, max_nr_accesses) > 15: + print('too high nr_access: expected %d but %d' % + (max_nr_accesses, nr_accesses)) + exit(1) + if nr_gaps != 2: + print('number of gaps are not two but %d' % nr_gaps) + exit(1) + +def parse_time_us(bindat): + sec = struct.unpack('l', bindat[0:8])[0] + nsec = struct.unpack('l', bindat[8:16])[0] + return (sec * 1000000000 + nsec) / 1000 + +def main(): + global sint + global aint + global min_nr + global max_nr_regions + + parser = argparse.ArgumentParser() + parser.add_argument('file', metavar='', + help='path to the record file') + parser.add_argument('--attrs', metavar='', + default='5000 100000 1000000 10 1000', + help='content of debugfs attrs file') + args = parser.parse_args() + file_path = args.file + attrs = [int(x) for x in args.attrs.split()] + sint, aint, rint, min_nr, max_nr_regions = attrs + + with open(file_path, 'rb') as f: + set_fmt_version(f) + last_aggr_time = None + while True: + timebin = f.read(16) + if len(timebin) != 16: + break + + now = parse_time_us(timebin) + if not last_aggr_time: + last_aggr_time = now + else: + error = err_percent(now - last_aggr_time, aint) + if error > 15: + print('wrong aggr interval: expected %d, but %d' % + (aint, now - last_aggr_time)) + exit(1) + last_aggr_time = now + + nr_tasks = struct.unpack('I', f.read(4))[0] + for t in range(nr_tasks): + chk_task_info(f) + +if __name__ == '__main__': + main() diff --git a/tools/testing/selftests/damon/debugfs_attrs.sh b/tools/testing/selftests/damon/debugfs_attrs.sh new file mode 100755 index 000000000000..c75557e8ba58 --- /dev/null +++ b/tools/testing/selftests/damon/debugfs_attrs.sh @@ -0,0 +1,161 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 + +source ./_chk_dependency.sh + +# Test attrs file +file="$DBGFS/attrs" + +ORIG_CONTENT=$(cat $file) + +echo 1 2 3 4 5 > $file +if [ $? -ne 0 ] +then + echo "$file write failed" + echo $ORIG_CONTENT > $file + exit 1 +fi + +echo 1 2 3 4 > $file +if [ $? -eq 0 ] +then + echo "$file write success (should failed)" + echo $ORIG_CONTENT > $file + exit 1 +fi + +CONTENT=$(cat $file) +if [ "$CONTENT" != "1 2 3 4 5" ] +then + echo "$file not written" + echo $ORIG_CONTENT > $file + exit 1 +fi + +echo $ORIG_CONTENT > $file + +# Test record file +file="$DBGFS/record" + +ORIG_CONTENT=$(cat $file) + +echo "4242 foo.bar" > $file +if [ $? -ne 0 ] +then + echo "$file writing sane input failed" + echo $ORIG_CONTENT > $file + exit 1 +fi + +echo abc 2 3 > $file +if [ $? -eq 0 ] +then + echo "$file writing insane input 1 success (should failed)" + echo $ORIG_CONTENT > $file + exit 1 +fi + +echo 123 > $file +if [ $? -eq 0 ] +then + echo "$file writing insane input 2 success (should failed)" + echo $ORIG_CONTENT > $file + exit 1 +fi + +CONTENT=$(cat $file) +if [ "$CONTENT" != "4242 foo.bar" ] +then + echo "$file not written" + echo $ORIG_CONTENT > $file + exit 1 +fi + +echo "0 null" > $file +if [ $? -ne 0 ] +then + echo "$file disabling write fail" + echo $ORIG_CONTENT > $file + exit 1 +fi + +CONTENT=$(cat $file) +if [ "$CONTENT" != "0 null" ] +then + echo "$file not disabled" + echo $ORIG_CONTENT > $file + exit 1 +fi + +echo "4242 foo.bar" > $file +if [ $? -ne 0 ] +then + echo "$file writing sane data again fail" + echo $ORIG_CONTENT > $file + exit 1 +fi + +echo $ORIG_CONTENT > $file + +# Test target_ids file +file="$DBGFS/target_ids" + +ORIG_CONTENT=$(cat $file) + +echo "1 2 3 4" > $file +if [ $? -ne 0 ] +then + echo "$file write fail" + echo $ORIG_CONTENT > $file + exit 1 +fi + +echo "1 2 abc 4" > $file +if [ $? -ne 0 ] +then + echo "$file write fail" + echo $ORIG_CONTENT > $file + exit 1 +fi + +CONTENT=$(cat $file) +if [ "$CONTENT" != "1 2" ] +then + echo "$file not written" + echo $ORIG_CONTENT > $file + exit 1 +fi + +echo abc 2 3 > $file +if [ $? -ne 0 ] +then + echo "$file wrong value write fail" + echo $ORIG_CONTENT > $file + exit 1 +fi + +if [ ! -z "$(cat $file)" ] +then + echo "$file not cleared" + echo $ORIG_CONTENT > $file + exit 1 +fi + +echo > $file +if [ $? -ne 0 ] +then + echo "$file init fail" + echo $ORIG_CONTENT > $file + exit 1 +fi + +if [ ! -z "$(cat $file)" ] +then + echo "$file not initialized" + echo $ORIG_CONTENT > $file + exit 1 +fi + +echo $ORIG_CONTENT > $file + +echo "PASS" diff --git a/tools/testing/selftests/damon/debugfs_record.sh b/tools/testing/selftests/damon/debugfs_record.sh new file mode 100755 index 000000000000..c0fb8d24dc32 --- /dev/null +++ b/tools/testing/selftests/damon/debugfs_record.sh @@ -0,0 +1,50 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 + +source ./_chk_dependency.sh + +restore_attrs() +{ + echo $ORIG_ATTRS > $DBGFS/attrs + echo $ORIG_TARGET_IDS > $DBGFS/target_ids + echo $ORIG_RECORD > $DBGFS/record +} + +ORIG_ATTRS=$(cat $DBGFS/attrs) +ORIG_TARGET_IDS=$(cat $DBGFS/target_ids) +ORIG_RECORD=$(cat $DBGFS/record) + +rfile=$pwd/damon.data + +rm -f $rfile +ATTRS="5000 100000 1000000 10 1000" +echo $ATTRS > $DBGFS/attrs +echo 4096 $rfile > $DBGFS/record +sleep 5 & +echo $(pidof sleep) > $DBGFS/target_ids +echo on > $DBGFS/monitor_on +sleep 0.5 +killall sleep +echo off > $DBGFS/monitor_on + +sync + +if [ ! -f $rfile ] +then + echo "record file not made" + restore_attrs + + exit 1 +fi + +python3 ./_chk_record.py $rfile --attrs "$ATTRS" +if [ $? -ne 0 ] +then + echo "record file is wrong" + restore_attrs + exit 1 +fi + +rm -f $rfile +restore_attrs +echo "PASS" From patchwork Thu Mar 18 10:08:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 12147815 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F078FC433DB for ; Thu, 18 Mar 2021 10:10:38 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9610761580 for ; Thu, 18 Mar 2021 10:10:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9610761580 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0CC238D0009; Thu, 18 Mar 2021 06:10:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 07D358D0001; Thu, 18 Mar 2021 06:10:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D4E588D0009; Thu, 18 Mar 2021 06:10:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0081.hostedemail.com [216.40.44.81]) by kanga.kvack.org (Postfix) with ESMTP id B4F278D0001 for ; Thu, 18 Mar 2021 06:10:29 -0400 (EDT) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 803F42C32 for ; Thu, 18 Mar 2021 10:10:29 +0000 (UTC) X-FDA: 77932575336.16.7018B28 Received: from mail-qv1-f47.google.com (mail-qv1-f47.google.com [209.85.219.47]) by imf14.hostedemail.com (Postfix) with ESMTP id 0DF46C0007C3 for ; Thu, 18 Mar 2021 10:10:28 +0000 (UTC) Received: by mail-qv1-f47.google.com with SMTP id t16so2866111qvr.12 for ; Thu, 18 Mar 2021 03:10:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=+3m60JeC3jcW717248QSX5WUD949KDlEAqc7aIaeSYg=; b=YeWGsrseemmcW24aWzVapHTFgdzxwtxUdZJ/asMlJ1D7IYUH4gyVxzX3uQ0jL7q6Nr Vu4J58J1wy32zv5yRGdWoiT7Kqm4PbQXxG/zqHYCUmREGM1ky8S5Xo3q7QM5w264rDdo s0ZAjkTShUHTpYjrgcp5EB36aU7Uk2dxCMhAXOu/NazJVc8qeuzzJAfj2KSCNrV9tNmy mZBZLrAAsRedspI712Fq+h8lwoN4PRxQ4JbUd0wQf67pZ2QaL2iIJCOIc7LQx7pKpJbs 7CcMm/2odk5HMha1r5sao4e1GF9t/6R3KrrOCrWANYjHbYC7oh2uWgFVY1j6g8lUsPXp 6p1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=+3m60JeC3jcW717248QSX5WUD949KDlEAqc7aIaeSYg=; b=n4kCrpBDJwd/OJYSfU42K7yxB7cKaOrLdOj5i+LdvScL65cFfnYS4NTgqOsuYzzAfm lo9Mn1CNPPOpTz4qOsHSi3Nuk57HDy/5Bb5rSXQQUZC5ggyE6DgI9X3qmQN/r+nbSuTh V9mR1t+nJjg9BPGjVnzw7rcdgsisaMo0eUgb4HZZoH9PRiXOJrya+b7Wwq90evlDDG9o 9jDHtKsRFjg3jIFi3KeNDUc+aRkHt1Ye693oJBz9iJ6TQQ66LtqapcncCBGgmiSbi488 I/zjduzCdBVZeAV3/zN+qgoL2JYjVjZw7EscB6lHwKBI1G/pSyS7p+MMISWfhKJNVJFz u24g== X-Gm-Message-State: AOAM532q3bZU9gEknOmtxhWqNLs42TJd00SUeQlp1LpM5u8EnSUcDiaS 4c725VVGdCzcazkQ0ONp/0Q= X-Google-Smtp-Source: ABdhPJx3arxomKqg7xQUzyn/nuOtqEJEPH7qfw7lZLsrMV+s/7OJ+HpAL3kZwadzs4tHbutbNcYUYg== X-Received: by 2002:ad4:4c4c:: with SMTP id cs12mr3409196qvb.35.1616062228440; Thu, 18 Mar 2021 03:10:28 -0700 (PDT) Received: from localhost.localdomain (ec2-35-169-212-159.compute-1.amazonaws.com. [35.169.212.159]) by smtp.gmail.com with ESMTPSA id d16sm1358101qka.43.2021.03.18.03.10.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 18 Mar 2021 03:10:28 -0700 (PDT) From: sj38.park@gmail.com To: akpm@linux-foundation.org Cc: SeongJae Park , Jonathan.Cameron@Huawei.com, acme@kernel.org, alexander.shishkin@linux.intel.com, amit@kernel.org, benh@kernel.crashing.org, brendanhiggins@google.com, corbet@lwn.net, david@redhat.com, dwmw@amazon.com, elver@google.com, fan.du@intel.com, foersleo@amazon.de, gthelen@google.com, mgorman@suse.de, minchan@kernel.org, mingo@redhat.com, namhyung@kernel.org, peterz@infradead.org, riel@surriel.com, rientjes@google.com, rostedt@goodmis.org, rppt@kernel.org, shakeelb@google.com, shuah@kernel.org, sj38.park@gmail.com, snu@amazon.de, vbabka@suse.cz, vdavydov.dev@gmail.com, zgf574564920@gmail.com, linux-damon@amazon.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v25 13/13] MAINTAINERS: Update for DAMON Date: Thu, 18 Mar 2021 10:08:56 +0000 Message-Id: <20210318100856.34715-14-sj38.park@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210318100856.34715-1-sj38.park@gmail.com> References: <20210318100856.34715-1-sj38.park@gmail.com> X-Stat-Signature: 7yw4kxot9z3p5p6kxe5dd4ey5zaf6mi9 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 0DF46C0007C3 Received-SPF: none (gmail.com>: No applicable sender policy available) receiver=imf14; identity=mailfrom; envelope-from=""; helo=mail-qv1-f47.google.com; client-ip=209.85.219.47 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1616062228-796301 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park This commit updates MAINTAINERS file for DAMON related files. Signed-off-by: SeongJae Park --- MAINTAINERS | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index b2baeb5e4a68..4665a5699cf6 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -4952,6 +4952,18 @@ F: net/ax25/ax25_out.c F: net/ax25/ax25_timer.c F: net/ax25/sysctl_net_ax25.c +DATA ACCESS MONITOR +M: SeongJae Park +L: linux-mm@kvack.org +S: Maintained +F: Documentation/admin-guide/mm/damon/* +F: Documentation/vm/damon/* +F: include/linux/damon.h +F: include/trace/events/damon.h +F: mm/damon/* +F: tools/damon/* +F: tools/testing/selftests/damon/* + DAVICOM FAST ETHERNET (DMFE) NETWORK DRIVER L: netdev@vger.kernel.org S: Orphan