From patchwork Mon May 11 12:32:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 11540599 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4751792A for ; Mon, 11 May 2020 12:34:23 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 14CB42137B for ; Mon, 11 May 2020 12:34:23 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="raAND24w" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 14CB42137B Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=amazon.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5788190003B; Mon, 11 May 2020 08:34:22 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 54FE1900036; Mon, 11 May 2020 08:34:22 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 48D2F90003B; Mon, 11 May 2020 08:34:22 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0035.hostedemail.com [216.40.44.35]) by kanga.kvack.org (Postfix) with ESMTP id 2F5F7900036 for ; Mon, 11 May 2020 08:34:22 -0400 (EDT) Received: from smtpin02.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id DCDF8181AEF0B for ; Mon, 11 May 2020 12:34:21 +0000 (UTC) X-FDA: 76804381122.02.fifth56_4328ff1206e47 X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,prvs=3933f0155=sjpark@amazon.com,,RULES_HIT:30054:30064,0,RBL:72.21.196.25:@amazon.com:.lbl8.mailshell.net-62.18.0.100 66.10.201.10,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:23,LUA_SUMMARY:none X-HE-Tag: fifth56_4328ff1206e47 X-Filterd-Recvd-Size: 5154 Received: from smtp-fw-2101.amazon.com (smtp-fw-2101.amazon.com [72.21.196.25]) by imf29.hostedemail.com (Postfix) with ESMTP for ; Mon, 11 May 2020 12:34:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1589200462; x=1620736462; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=iUKxsyy8EDqa4LYd6lWomJlLTmy2039pJ+ilYaUf9pk=; b=raAND24w/jqMX7z5rfUmI5qxTXptDGqcOQtHGb0WgEL2+sx4yR5vVOoC rxuhoiJu2BSPEhpl1FP4PW6B5zrcvmgdWxlA4wn9sEcmlXjn8KXryW3Jx sN94J+OMYdiew78oTyc3693MsGRE7A0dlKKaZmcMDX1dvQpIHIN8LEL6O c=; IronPort-SDR: E1P0Scvi08EA1KBg+AP79GvCMQ4zXEhWJj7Kr1t/jNEwZ8qxaUVujEfDLgh8Bts7/n6ARyfyDm 81iceauew21g== X-IronPort-AV: E=Sophos;i="5.73,379,1583193600"; d="scan'208";a="29802095" Received: from iad12-co-svc-p1-lb1-vlan2.amazon.com (HELO email-inbound-relay-1a-16acd5e0.us-east-1.amazon.com) ([10.43.8.2]) by smtp-border-fw-out-2101.iad2.amazon.com with ESMTP; 11 May 2020 12:34:09 +0000 Received: from EX13MTAUEA002.ant.amazon.com (iad55-ws-svc-p15-lb9-vlan2.iad.amazon.com [10.40.159.162]) by email-inbound-relay-1a-16acd5e0.us-east-1.amazon.com (Postfix) with ESMTPS id 7BD62A2462; Mon, 11 May 2020 12:33:57 +0000 (UTC) Received: from EX13D31EUA001.ant.amazon.com (10.43.165.15) by EX13MTAUEA002.ant.amazon.com (10.43.61.77) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 11 May 2020 12:33:56 +0000 Received: from u886c93fd17d25d.ant.amazon.com (10.43.161.253) by EX13D31EUA001.ant.amazon.com (10.43.165.15) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 11 May 2020 12:33:41 +0000 From: SeongJae Park To: CC: SeongJae Park , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v11 01/16] scripts/spelling: Add a few more typos Date: Mon, 11 May 2020 14:32:47 +0200 Message-ID: <20200511123302.12520-2-sjpark@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200511123302.12520-1-sjpark@amazon.com> References: <20200511123302.12520-1-sjpark@amazon.com> MIME-Version: 1.0 X-Originating-IP: [10.43.161.253] X-ClientProxiedBy: EX13D06UWC001.ant.amazon.com (10.43.162.91) To EX13D31EUA001.ant.amazon.com (10.43.165.15) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park This commit adds typos found from DAMON patchset. Signed-off-by: SeongJae Park --- scripts/spelling.txt | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/scripts/spelling.txt b/scripts/spelling.txt index ffa838f3a2b5..a8214d8e8e4b 100644 --- a/scripts/spelling.txt +++ b/scripts/spelling.txt @@ -59,6 +59,7 @@ actualy||actually acumulating||accumulating acumulative||accumulative acumulator||accumulator +acutally||actually adapater||adapter addional||additional additionaly||additionally @@ -245,6 +246,7 @@ calescing||coalescing calle||called callibration||calibration callled||called +callser||caller calucate||calculate calulate||calculate cancelation||cancellation @@ -664,6 +666,7 @@ hanlde||handle hanled||handled happend||happened harware||hardware +havind||having heirarchically||hierarchically helpfull||helpful hexdecimal||hexadecimal @@ -835,6 +838,7 @@ logile||logfile loobpack||loopback loosing||losing losted||lost +maangement||management machinary||machinery maibox||mailbox maintainance||maintenance @@ -999,6 +1003,7 @@ partiton||partition pased||passed passin||passing pathes||paths +pattrns||patterns pecularities||peculiarities peformance||performance peforming||performing @@ -1238,6 +1243,7 @@ shoule||should shrinked||shrunk siginificantly||significantly signabl||signal +significanly||significantly similary||similarly similiar||similar simlar||similar @@ -1352,6 +1358,7 @@ thead||thread therfore||therefore thier||their threds||threads +threee||three threshhold||threshold thresold||threshold throught||through @@ -1391,6 +1398,7 @@ tyep||type udpate||update uesd||used uknown||unknown +usccess||success usupported||unsupported uncommited||uncommitted unconditionaly||unconditionally From patchwork Mon May 11 12:32:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 11540601 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7F225912 for ; Mon, 11 May 2020 12:34:40 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4C0D0207FF for ; Mon, 11 May 2020 12:34:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="t9Vjl+Bg" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4C0D0207FF Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=amazon.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 74CBC90003C; Mon, 11 May 2020 08:34:39 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6FD80900036; Mon, 11 May 2020 08:34:39 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6141590003C; Mon, 11 May 2020 08:34:39 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0018.hostedemail.com [216.40.44.18]) by kanga.kvack.org (Postfix) with ESMTP id 4D3CA900036 for ; Mon, 11 May 2020 08:34:39 -0400 (EDT) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 077BB8248047 for ; Mon, 11 May 2020 12:34:39 +0000 (UTC) X-FDA: 76804381878.30.hole64_45a81c15e104e X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,prvs=3933f0155=sjpark@amazon.com,,RULES_HIT:30054:30064,0,RBL:52.95.49.90:@amazon.com:.lbl8.mailshell.net-64.10.201.10 62.18.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:23,LUA_SUMMARY:none X-HE-Tag: hole64_45a81c15e104e X-Filterd-Recvd-Size: 3868 Received: from smtp-fw-6002.amazon.com (smtp-fw-6002.amazon.com [52.95.49.90]) by imf39.hostedemail.com (Postfix) with ESMTP for ; Mon, 11 May 2020 12:34:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1589200479; x=1620736479; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=G8CneijCkhkmAYrW5RLVj9rtVs0AB1S5RMsQ7jnOqO0=; b=t9Vjl+BgVlC9nQcD/78SQ8QZosuXkuxtNP6PLMuxoTXKHjd+BGbB6YsJ 7b/CMqAJCyaEqJnllh8b2WfHYAl0xRBiMOeJDEDNEJvJjDOyP13PECYdT +HtjGY4HjJM5i6EqMCEN1l5iaOVp5IyEJ9I7RruqfJn7aWQueDTs0UjCv I=; IronPort-SDR: bYq+2GAJ3wjEEJiqjRh1v9dOz7gI/tBXbLR6FE9FYKBblQlbpdWhOayGZnRw0+PL33OSGaRcsH slB314RzfR2w== X-IronPort-AV: E=Sophos;i="5.73,379,1583193600"; d="scan'208";a="29558027" Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO email-inbound-relay-1d-98acfc19.us-east-1.amazon.com) ([10.43.8.6]) by smtp-border-fw-out-6002.iad6.amazon.com with ESMTP; 11 May 2020 12:34:26 +0000 Received: from EX13MTAUEA002.ant.amazon.com (iad55-ws-svc-p15-lb9-vlan3.iad.amazon.com [10.40.159.166]) by email-inbound-relay-1d-98acfc19.us-east-1.amazon.com (Postfix) with ESMTPS id 728BFA2592; Mon, 11 May 2020 12:34:15 +0000 (UTC) Received: from EX13D31EUA001.ant.amazon.com (10.43.165.15) by EX13MTAUEA002.ant.amazon.com (10.43.61.77) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 11 May 2020 12:34:14 +0000 Received: from u886c93fd17d25d.ant.amazon.com (10.43.161.253) by EX13D31EUA001.ant.amazon.com (10.43.165.15) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 11 May 2020 12:33:59 +0000 From: SeongJae Park To: CC: SeongJae Park , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v11 02/16] mm/page_ext: Export lookup_page_ext() to GPL modules Date: Mon, 11 May 2020 14:32:48 +0200 Message-ID: <20200511123302.12520-3-sjpark@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200511123302.12520-1-sjpark@amazon.com> References: <20200511123302.12520-1-sjpark@amazon.com> MIME-Version: 1.0 X-Originating-IP: [10.43.161.253] X-ClientProxiedBy: EX13D06UWC001.ant.amazon.com (10.43.162.91) To EX13D31EUA001.ant.amazon.com (10.43.165.15) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park This commit exports 'lookup_page_ext()' to GPL modules. This will be used by DAMON. Signed-off-by: SeongJae Park --- mm/page_ext.c | 1 + 1 file changed, 1 insertion(+) diff --git a/mm/page_ext.c b/mm/page_ext.c index 4ade843ff588..71169b45bba9 100644 --- a/mm/page_ext.c +++ b/mm/page_ext.c @@ -131,6 +131,7 @@ struct page_ext *lookup_page_ext(const struct page *page) MAX_ORDER_NR_PAGES); return get_entry(base, index); } +EXPORT_SYMBOL_GPL(lookup_page_ext); static int __init alloc_node_page_ext(int nid) { From patchwork Mon May 11 12:32:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 11540603 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D801392A for ; Mon, 11 May 2020 12:35:37 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9761F206D6 for ; Mon, 11 May 2020 12:35:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="P+oc80lf" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9761F206D6 Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=amazon.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B925C90003D; Mon, 11 May 2020 08:35:36 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B6A34900036; Mon, 11 May 2020 08:35:36 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A7F6E90003D; Mon, 11 May 2020 08:35:36 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0179.hostedemail.com [216.40.44.179]) by kanga.kvack.org (Postfix) with ESMTP id 91D51900036 for ; Mon, 11 May 2020 08:35:36 -0400 (EDT) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 475D7181AEF0B for ; Mon, 11 May 2020 12:35:36 +0000 (UTC) X-FDA: 76804384272.24.worm23_4dfd85a5dfd44 X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,prvs=3933f0155=sjpark@amazon.com,,RULES_HIT:30034:30054:30064:30067,0,RBL:72.21.198.25:@amazon.com:.lbl8.mailshell.net-62.18.0.100 66.10.201.10,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: worm23_4dfd85a5dfd44 X-Filterd-Recvd-Size: 9958 Received: from smtp-fw-4101.amazon.com (smtp-fw-4101.amazon.com [72.21.198.25]) by imf02.hostedemail.com (Postfix) with ESMTP for ; Mon, 11 May 2020 12:35:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1589200536; x=1620736536; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=GfwmAepOlGw0jc65vePXQyjyaX4cDEsLZQAUUm5+13E=; b=P+oc80lf3Lx5F8lVKz/XUqkVnWGa6Hp/Rvyycp58yfDUYdiixCMBNqzr HKkUL0HBqVIBoYnkjwzhSRMX1d5aSETU/0xFY/wzBjpCs4Xj5atYQ8ydy uv3h80f9IVFUJ8Y8Vf/egaxRPZNMTY2V4GT7lob3RlfnbjZDdCYEMvfow 0=; IronPort-SDR: h9+H7/56yIOfZ/NHSAmFHYIywlJl9LOrq0URnBSV4VfyRPHlCoJy0XsxkHWsqUKpAmUQGHzw+Z 0o/BsRfKWKfw== X-IronPort-AV: E=Sophos;i="5.73,379,1583193600"; d="scan'208";a="29668255" Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO email-inbound-relay-1a-16acd5e0.us-east-1.amazon.com) ([10.43.8.6]) by smtp-border-fw-out-4101.iad4.amazon.com with ESMTP; 11 May 2020 12:35:19 +0000 Received: from EX13MTAUEA002.ant.amazon.com (iad55-ws-svc-p15-lb9-vlan2.iad.amazon.com [10.40.159.162]) by email-inbound-relay-1a-16acd5e0.us-east-1.amazon.com (Postfix) with ESMTPS id B3FC4A245D; Mon, 11 May 2020 12:35:07 +0000 (UTC) Received: from EX13D31EUA001.ant.amazon.com (10.43.165.15) by EX13MTAUEA002.ant.amazon.com (10.43.61.77) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 11 May 2020 12:35:06 +0000 Received: from u886c93fd17d25d.ant.amazon.com (10.43.161.253) by EX13D31EUA001.ant.amazon.com (10.43.165.15) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 11 May 2020 12:34:50 +0000 From: SeongJae Park To: CC: SeongJae Park , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v11 03/16] mm: Introduce Data Access MONitor (DAMON) Date: Mon, 11 May 2020 14:32:49 +0200 Message-ID: <20200511123302.12520-4-sjpark@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200511123302.12520-1-sjpark@amazon.com> References: <20200511123302.12520-1-sjpark@amazon.com> MIME-Version: 1.0 X-Originating-IP: [10.43.161.253] X-ClientProxiedBy: EX13D06UWC001.ant.amazon.com (10.43.162.91) To EX13D31EUA001.ant.amazon.com (10.43.165.15) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park This commit introduces a kernel module named DAMON. Note that this commit is implementing only the stub for the module load/unload, basic data structures, and simple manipulation functions of the structures to keep the size of commit small. The core mechanisms of DAMON will be implemented one by one by following commits. Signed-off-by: SeongJae Park --- include/linux/damon.h | 36 +++++++++ mm/Kconfig | 12 +++ mm/Makefile | 1 + mm/damon.c | 173 ++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 222 insertions(+) create mode 100644 include/linux/damon.h create mode 100644 mm/damon.c diff --git a/include/linux/damon.h b/include/linux/damon.h new file mode 100644 index 000000000000..19f411d36c0d --- /dev/null +++ b/include/linux/damon.h @@ -0,0 +1,36 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * DAMON api + * + * Copyright 2019-2020 Amazon.com, Inc. or its affiliates. + * + * Author: SeongJae Park + */ + +#ifndef _DAMON_H_ +#define _DAMON_H_ + +#include +#include + +/* Represents a monitoring target region of [vm_start, vm_end) */ +struct damon_region { + unsigned long vm_start; + unsigned long vm_end; + unsigned long sampling_addr; + unsigned int nr_accesses; + struct list_head list; +}; + +/* Represents a monitoring target task */ +struct damon_task { + int pid; + struct list_head regions_list; + struct list_head list; +}; + +struct damon_ctx { + struct list_head tasks_list; /* 'damon_task' objects */ +}; + +#endif diff --git a/mm/Kconfig b/mm/Kconfig index ab80933be65f..9ea49633a6df 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -739,4 +739,16 @@ config ARCH_HAS_HUGEPD config MAPPING_DIRTY_HELPERS bool +config DAMON + tristate "Data Access Monitor" + depends on MMU + help + Provides data access monitoring. + + DAMON is a kernel module that allows users to monitor the actual + memory access pattern of specific user-space processes. It aims to + be 1) accurate enough to be useful for performance-centric domains, + and 2) sufficiently light-weight so that it can be applied online. + If unsure, say N. + endmenu diff --git a/mm/Makefile b/mm/Makefile index 272e66039e70..5346314edee6 100644 --- a/mm/Makefile +++ b/mm/Makefile @@ -110,3 +110,4 @@ obj-$(CONFIG_HMM_MIRROR) += hmm.o obj-$(CONFIG_MEMFD_CREATE) += memfd.o obj-$(CONFIG_MAPPING_DIRTY_HELPERS) += mapping_dirty_helpers.o obj-$(CONFIG_PTDUMP_CORE) += ptdump.o +obj-$(CONFIG_DAMON) += damon.o diff --git a/mm/damon.c b/mm/damon.c new file mode 100644 index 000000000000..e319c1e19f3d --- /dev/null +++ b/mm/damon.c @@ -0,0 +1,173 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Data Access Monitor + * + * Copyright 2019-2020 Amazon.com, Inc. or its affiliates. + * + * Author: SeongJae Park + */ + +#define pr_fmt(fmt) "damon: " fmt + +#include +#include +#include +#include + +#define damon_get_task_struct(t) \ + (get_pid_task(find_vpid(t->pid), PIDTYPE_PID)) + +#define damon_next_region(r) \ + (container_of(r->list.next, struct damon_region, list)) + +#define damon_prev_region(r) \ + (container_of(r->list.prev, struct damon_region, list)) + +#define damon_for_each_region(r, t) \ + list_for_each_entry(r, &t->regions_list, list) + +#define damon_for_each_region_safe(r, next, t) \ + list_for_each_entry_safe(r, next, &t->regions_list, list) + +#define damon_for_each_task(ctx, t) \ + list_for_each_entry(t, &(ctx)->tasks_list, list) + +#define damon_for_each_task_safe(ctx, t, next) \ + list_for_each_entry_safe(t, next, &(ctx)->tasks_list, list) + +/* Get a random number in [l, r) */ +#define damon_rand(l, r) (l + prandom_u32() % (r - l)) + +/* + * Construct a damon_region struct + * + * Returns the pointer to the new struct if success, or NULL otherwise + */ +static struct damon_region *damon_new_region(struct damon_ctx *ctx, + unsigned long vm_start, unsigned long vm_end) +{ + struct damon_region *region; + + region = kmalloc(sizeof(*region), GFP_KERNEL); + if (!region) + return NULL; + + region->vm_start = vm_start; + region->vm_end = vm_end; + region->nr_accesses = 0; + INIT_LIST_HEAD(®ion->list); + + return region; +} + +/* + * Add a region between two other regions + */ +static inline void damon_insert_region(struct damon_region *r, + struct damon_region *prev, struct damon_region *next) +{ + __list_add(&r->list, &prev->list, &next->list); +} + +static void damon_add_region(struct damon_region *r, struct damon_task *t) +{ + list_add_tail(&r->list, &t->regions_list); +} + +static void damon_del_region(struct damon_region *r) +{ + list_del(&r->list); +} + +static void damon_free_region(struct damon_region *r) +{ + kfree(r); +} + +static void damon_destroy_region(struct damon_region *r) +{ + damon_del_region(r); + damon_free_region(r); +} + +/* + * Construct a damon_task struct + * + * Returns the pointer to the new struct if success, or NULL otherwise + */ +static struct damon_task *damon_new_task(int pid) +{ + struct damon_task *t; + + t = kmalloc(sizeof(*t), GFP_KERNEL); + if (!t) + return NULL; + + t->pid = pid; + INIT_LIST_HEAD(&t->regions_list); + + return t; +} + +static void damon_add_task(struct damon_ctx *ctx, struct damon_task *t) +{ + list_add_tail(&t->list, &ctx->tasks_list); +} + +static void damon_del_task(struct damon_task *t) +{ + list_del(&t->list); +} + +static void damon_free_task(struct damon_task *t) +{ + struct damon_region *r, *next; + + damon_for_each_region_safe(r, next, t) + damon_free_region(r); + kfree(t); +} + +static void damon_destroy_task(struct damon_task *t) +{ + damon_del_task(t); + damon_free_task(t); +} + +static unsigned int nr_damon_tasks(struct damon_ctx *ctx) +{ + struct damon_task *t; + unsigned int nr_tasks = 0; + + damon_for_each_task(ctx, t) + nr_tasks++; + + return nr_tasks; +} + +static unsigned int nr_damon_regions(struct damon_task *t) +{ + struct damon_region *r; + unsigned int nr_regions = 0; + + damon_for_each_region(r, t) + nr_regions++; + + return nr_regions; +} + +static int __init damon_init(void) +{ + return 0; +} + +static void __exit damon_exit(void) +{ +} + +module_init(damon_init); +module_exit(damon_exit); + +MODULE_LICENSE("GPL"); +MODULE_AUTHOR("SeongJae Park "); +MODULE_DESCRIPTION("DAMON: Data Access MONitor"); From patchwork Mon May 11 12:32:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 11540605 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C2B1892A for ; Mon, 11 May 2020 12:36:06 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 74FA3206D6 for ; Mon, 11 May 2020 12:36:06 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="bTfCgxfr" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 74FA3206D6 Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=amazon.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9974390003E; Mon, 11 May 2020 08:36:05 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9201B900036; Mon, 11 May 2020 08:36:05 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7BF6F90003E; Mon, 11 May 2020 08:36:05 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0044.hostedemail.com [216.40.44.44]) by kanga.kvack.org (Postfix) with ESMTP id 5A971900036 for ; Mon, 11 May 2020 08:36:05 -0400 (EDT) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 0E24D180AD80F for ; Mon, 11 May 2020 12:36:05 +0000 (UTC) X-FDA: 76804385490.19.table92_5224181261b42 X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,prvs=3933f0155=sjpark@amazon.com,,RULES_HIT:30003:30012:30034:30051:30054:30056:30064:30070:30075:30090,0,RBL:72.21.196.25:@amazon.com:.lbl8.mailshell.net-66.10.201.10 62.18.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:2,LUA_SUMMARY:none X-HE-Tag: table92_5224181261b42 X-Filterd-Recvd-Size: 23396 Received: from smtp-fw-2101.amazon.com (smtp-fw-2101.amazon.com [72.21.196.25]) by imf46.hostedemail.com (Postfix) with ESMTP for ; Mon, 11 May 2020 12:36:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1589200565; x=1620736565; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=40YT47tVy4me3OjP94fEnj3IwnLO50KQj6RU+t9sZqY=; b=bTfCgxfr+TVmOnWIABCfCfSRAAGBuNCVOD0r+12cH8y82NjraZiYgMPF BpnvAxAZmA0TUK5q82lLcaxVF26yzmdJTBNF+cfzcwvaKoEtDNaPuoj9h sBpzXwJV/Ao3Fp/y46W4vJ0TPV6FgNqU0VXCJ4e9VN8tUQuh/mVTGuYH2 E=; IronPort-SDR: 3hs5Khbf+qHTjeE1OU1b5Ayp/HQkU8qplt7vXCuKG4HL3et+lLgr/U7N9NIR8YKzZlMb9vG1Hb Jix53/f63lhA== X-IronPort-AV: E=Sophos;i="5.73,379,1583193600"; d="scan'208";a="29802520" Received: from iad12-co-svc-p1-lb1-vlan2.amazon.com (HELO email-inbound-relay-1a-67b371d8.us-east-1.amazon.com) ([10.43.8.2]) by smtp-border-fw-out-2101.iad2.amazon.com with ESMTP; 11 May 2020 12:36:04 +0000 Received: from EX13MTAUEA002.ant.amazon.com (iad55-ws-svc-p15-lb9-vlan3.iad.amazon.com [10.40.159.166]) by email-inbound-relay-1a-67b371d8.us-east-1.amazon.com (Postfix) with ESMTPS id 5867EA235C; Mon, 11 May 2020 12:35:53 +0000 (UTC) Received: from EX13D31EUA001.ant.amazon.com (10.43.165.15) by EX13MTAUEA002.ant.amazon.com (10.43.61.77) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 11 May 2020 12:35:52 +0000 Received: from u886c93fd17d25d.ant.amazon.com (10.43.161.253) by EX13D31EUA001.ant.amazon.com (10.43.165.15) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 11 May 2020 12:35:36 +0000 From: SeongJae Park To: CC: SeongJae Park , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v11 04/16] mm/damon: Implement region based sampling Date: Mon, 11 May 2020 14:32:50 +0200 Message-ID: <20200511123302.12520-5-sjpark@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200511123302.12520-1-sjpark@amazon.com> References: <20200511123302.12520-1-sjpark@amazon.com> MIME-Version: 1.0 X-Originating-IP: [10.43.161.253] X-ClientProxiedBy: EX13D06UWC001.ant.amazon.com (10.43.162.91) To EX13D31EUA001.ant.amazon.com (10.43.165.15) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park This commit implements DAMON's basic access check and region based sampling mechanisms. This change would seems make no sense, mainly because it is only a part of the DAMON's logics. Following two commits will make more sense. Basic Access Check ------------------ DAMON basically reports what pages are how frequently accessed. Note that the frequency is not an absolute number of accesses, but a relative frequency among the pages of the target workloads. Users can control the resolution of the reports by setting two time intervals, ``sampling interval`` and ``aggregation interval``. In detail, DAMON checks access to each page per ``sampling interval``, aggregates the results (counts the number of the accesses to each page), and reports the aggregated results per ``aggregation interval``. For the access check of each page, DAMON uses the Accessed bits of PTEs. This is thus similar to common periodic access checks based access tracking mechanisms, which overhead is increasing as the size of the target process grows. Region Based Sampling --------------------- To avoid the unbounded increase of the overhead, DAMON groups a number of adjacent pages that assumed to have same access frequencies into a region. As long as the assumption (pages in a region have same access frequencies) is kept, only one page in the region is required to be checked. Thus, for each ``sampling interval``, DAMON randomly picks one page in each region and clears its Accessed bit. After one more ``sampling interval``, DAMON reads the Accessed bit of the page and increases the access frequency of the region if the bit has set meanwhile. Therefore, the monitoring overhead is controllable by setting the number of regions. Nonetheless, this scheme cannot preserve the quality of the output if the assumption is not kept. Following commit will introduce how we can make the guarantee with best effort. Signed-off-by: SeongJae Park --- include/linux/damon.h | 25 ++ mm/damon.c | 597 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 622 insertions(+) diff --git a/include/linux/damon.h b/include/linux/damon.h index 19f411d36c0d..558dd6ae0afa 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -11,6 +11,8 @@ #define _DAMON_H_ #include +#include +#include #include /* Represents a monitoring target region of [vm_start, vm_end) */ @@ -29,8 +31,31 @@ struct damon_task { struct list_head list; }; +/* + * For each 'sample_interval', DAMON checks whether each region is accessed or + * not. It aggregates and keeps the access information (number of accesses to + * each region) for each 'aggr_interval' time. + * + * All time intervals are in micro-seconds. + */ struct damon_ctx { + unsigned long sample_interval; + unsigned long aggr_interval; + unsigned long min_nr_regions; + + struct timespec64 last_aggregation; + + struct task_struct *kdamond; + bool kdamond_stop; + struct mutex kdamond_lock; + struct list_head tasks_list; /* 'damon_task' objects */ }; +int damon_set_pids(struct damon_ctx *ctx, int *pids, ssize_t nr_pids); +int damon_set_attrs(struct damon_ctx *ctx, unsigned long sample_int, + unsigned long aggr_int, unsigned long min_nr_reg); +int damon_start(struct damon_ctx *ctx); +int damon_stop(struct damon_ctx *ctx); + #endif diff --git a/mm/damon.c b/mm/damon.c index e319c1e19f3d..9c9aa18e846a 100644 --- a/mm/damon.c +++ b/mm/damon.c @@ -10,10 +10,19 @@ #define pr_fmt(fmt) "damon: " fmt #include +#include +#include #include #include +#include +#include +#include +#include #include +/* Minimal region size. Every damon_region is aligned by this. */ +#define MIN_REGION PAGE_SIZE + #define damon_get_task_struct(t) \ (get_pid_task(find_vpid(t->pid), PIDTYPE_PID)) @@ -156,6 +165,594 @@ static unsigned int nr_damon_regions(struct damon_task *t) return nr_regions; } +/* + * Get the mm_struct of the given task + * + * Caller _must_ put the mm_struct after use, unless it is NULL. + * + * Returns the mm_struct of the task on success, NULL on failure + */ +static struct mm_struct *damon_get_mm(struct damon_task *t) +{ + struct task_struct *task; + struct mm_struct *mm; + + task = damon_get_task_struct(t); + if (!task) + return NULL; + + mm = get_task_mm(task); + put_task_struct(task); + return mm; +} + +/* + * Size-evenly split a region into 'nr_pieces' small regions + * + * Returns 0 on success, or negative error code otherwise. + */ +static int damon_split_region_evenly(struct damon_ctx *ctx, + struct damon_region *r, unsigned int nr_pieces) +{ + unsigned long sz_orig, sz_piece, orig_end; + struct damon_region *n = NULL, *next; + unsigned long start; + + if (!r || !nr_pieces) + return -EINVAL; + + orig_end = r->vm_end; + sz_orig = r->vm_end - r->vm_start; + sz_piece = ALIGN_DOWN(sz_orig / nr_pieces, MIN_REGION); + + if (!sz_piece) + return -EINVAL; + + r->vm_end = r->vm_start + sz_piece; + next = damon_next_region(r); + for (start = r->vm_end; start + sz_piece <= orig_end; + start += sz_piece) { + n = damon_new_region(ctx, start, start + sz_piece); + if (!n) + return -ENOMEM; + damon_insert_region(n, r, next); + r = n; + } + /* complement last region for possible rounding error */ + if (n) + n->vm_end = orig_end; + + return 0; +} + +struct region { + unsigned long start; + unsigned long end; +}; + +static unsigned long sz_region(struct region *r) +{ + return r->end - r->start; +} + +static void swap_regions(struct region *r1, struct region *r2) +{ + struct region tmp; + + tmp = *r1; + *r1 = *r2; + *r2 = tmp; +} + +/* + * Find three regions separated by two biggest unmapped regions + * + * vma the head vma of the target address space + * regions an array of three 'struct region's that results will be saved + * + * This function receives an address space and finds three regions in it which + * separated by the two biggest unmapped regions in the space. Please refer to + * below comments of 'damon_init_regions_of()' function to know why this is + * necessary. + * + * Returns 0 if success, or negative error code otherwise. + */ +static int damon_three_regions_in_vmas(struct vm_area_struct *vma, + struct region regions[3]) +{ + struct region gap = {0}, first_gap = {0}, second_gap = {0}; + struct vm_area_struct *last_vma = NULL; + unsigned long start = 0; + + /* Find two biggest gaps so that first_gap > second_gap > others */ + for (; vma; vma = vma->vm_next) { + if (!last_vma) { + start = vma->vm_start; + last_vma = vma; + continue; + } + gap.start = last_vma->vm_end; + gap.end = vma->vm_start; + if (sz_region(&gap) > sz_region(&second_gap)) { + swap_regions(&gap, &second_gap); + if (sz_region(&second_gap) > sz_region(&first_gap)) + swap_regions(&second_gap, &first_gap); + } + last_vma = vma; + } + + if (!sz_region(&second_gap) || !sz_region(&first_gap)) + return -EINVAL; + + /* Sort the two biggest gaps by address */ + if (first_gap.start > second_gap.start) + swap_regions(&first_gap, &second_gap); + + /* Store the result */ + regions[0].start = ALIGN(start, MIN_REGION); + regions[0].end = ALIGN(first_gap.start, MIN_REGION); + regions[1].start = ALIGN(first_gap.end, MIN_REGION); + regions[1].end = ALIGN(second_gap.start, MIN_REGION); + regions[2].start = ALIGN(second_gap.end, MIN_REGION); + regions[2].end = ALIGN(last_vma->vm_end, MIN_REGION); + + return 0; +} + +/* + * Get the three regions in the given task + * + * Returns 0 on success, negative error code otherwise. + */ +static int damon_three_regions_of(struct damon_task *t, + struct region regions[3]) +{ + struct mm_struct *mm; + int rc; + + mm = damon_get_mm(t); + if (!mm) + return -EINVAL; + + down_read(&mm->mmap_sem); + rc = damon_three_regions_in_vmas(mm->mmap, regions); + up_read(&mm->mmap_sem); + + mmput(mm); + return rc; +} + +/* + * Initialize the monitoring target regions for the given task + * + * t the given target task + * + * Because only a number of small portions of the entire address space + * is actually mapped to the memory and accessed, monitoring the unmapped + * regions is wasteful. That said, because we can deal with small noises, + * tracking every mapping is not strictly required but could even incur a high + * overhead if the mapping frequently changes or the number of mappings is + * high. Nonetheless, this may seems very weird. DAMON's dynamic regions + * adjustment mechanism, which will be implemented with following commit will + * make this more sense. + * + * For the reason, we convert the complex mappings to three distinct regions + * that cover every mapped area of the address space. Also the two gaps + * between the three regions are the two biggest unmapped areas in the given + * address space. In detail, this function first identifies the start and the + * end of the mappings and the two biggest unmapped areas of the address space. + * Then, it constructs the three regions as below: + * + * [mappings[0]->start, big_two_unmapped_areas[0]->start) + * [big_two_unmapped_areas[0]->end, big_two_unmapped_areas[1]->start) + * [big_two_unmapped_areas[1]->end, mappings[nr_mappings - 1]->end) + * + * As usual memory map of processes is as below, the gap between the heap and + * the uppermost mmap()-ed region, and the gap between the lowermost mmap()-ed + * region and the stack will be two biggest unmapped regions. Because these + * gaps are exceptionally huge areas in usual address space, excluding these + * two biggest unmapped regions will be sufficient to make a trade-off. + * + * + * + * + * (other mmap()-ed regions and small unmapped regions) + * + * + * + */ +static void damon_init_regions_of(struct damon_ctx *c, struct damon_task *t) +{ + struct damon_region *r, *m = NULL; + struct region regions[3]; + int i; + + if (damon_three_regions_of(t, regions)) { + pr_err("Failed to get three regions of task %d\n", t->pid); + return; + } + + /* Set the initial three regions of the task */ + for (i = 0; i < 3; i++) { + r = damon_new_region(c, regions[i].start, regions[i].end); + if (!r) { + pr_err("%d'th init region creation failed\n", i); + return; + } + damon_add_region(r, t); + if (i == 1) + m = r; + } + + /* Split the middle region into 'min_nr_regions - 2' regions */ + if (damon_split_region_evenly(c, m, c->min_nr_regions - 2)) + pr_warn("Init middle region failed to be split\n"); +} + +/* Initialize '->regions_list' of every task */ +static void kdamond_init_regions(struct damon_ctx *ctx) +{ + struct damon_task *t; + + damon_for_each_task(ctx, t) + damon_init_regions_of(ctx, t); +} + +static void damon_mkold(struct mm_struct *mm, unsigned long addr) +{ + pte_t *pte = NULL; + pmd_t *pmd = NULL; + spinlock_t *ptl; + + if (follow_pte_pmd(mm, addr, NULL, &pte, &pmd, &ptl)) + return; + + if (pte) { + if (pte_young(*pte)) { + clear_page_idle(pte_page(*pte)); + set_page_young(pte_page(*pte)); + } + *pte = pte_mkold(*pte); + pte_unmap_unlock(pte, ptl); + return; + } + +#ifdef CONFIG_TRANSPARENT_HUGEPAGE + if (pmd_young(*pmd)) { + clear_page_idle(pmd_page(*pmd)); + set_page_young(pmd_page(*pmd)); + } + *pmd = pmd_mkold(*pmd); + spin_unlock(ptl); +#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ +} + +static void damon_prepare_access_check(struct damon_ctx *ctx, + struct mm_struct *mm, struct damon_region *r) +{ + r->sampling_addr = damon_rand(r->vm_start, r->vm_end); + + damon_mkold(mm, r->sampling_addr); +} + +static void kdamond_prepare_access_checks(struct damon_ctx *ctx) +{ + struct damon_task *t; + struct mm_struct *mm; + struct damon_region *r; + + damon_for_each_task(ctx, t) { + mm = damon_get_mm(t); + if (!mm) + continue; + damon_for_each_region(r, t) + damon_prepare_access_check(ctx, mm, r); + mmput(mm); + } +} + +static bool damon_young(struct mm_struct *mm, unsigned long addr, + unsigned long *page_sz) +{ + pte_t *pte = NULL; + pmd_t *pmd = NULL; + spinlock_t *ptl; + bool young = false; + + if (follow_pte_pmd(mm, addr, NULL, &pte, &pmd, &ptl)) + return false; + + *page_sz = PAGE_SIZE; + if (pte) { + young = pte_young(*pte); + pte_unmap_unlock(pte, ptl); + return young; + } + +#ifdef CONFIG_TRANSPARENT_HUGEPAGE + young = pmd_young(*pmd); + spin_unlock(ptl); + *page_sz = ((1UL) << HPAGE_PMD_SHIFT); +#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ + + return young; +} + +/* + * Check whether the region was accessed and prepare for next check + * + * mm 'mm_struct' for the given virtual address space + * r the region to be checked + */ +static void damon_check_access(struct damon_ctx *ctx, + struct mm_struct *mm, struct damon_region *r) +{ + static struct mm_struct *last_mm; + static unsigned long last_addr; + static unsigned long last_page_sz = PAGE_SIZE; + static bool last_accessed; + + /* If the region is in the last checked page, reuse the result */ + if (mm == last_mm && (ALIGN_DOWN(last_addr, last_page_sz) == + ALIGN_DOWN(r->sampling_addr, last_page_sz))) { + if (last_accessed) + r->nr_accesses++; + return; + } + + last_accessed = damon_young(mm, r->sampling_addr, &last_page_sz); + if (last_accessed) + r->nr_accesses++; + + last_mm = mm; + last_addr = r->sampling_addr; +} + +static void kdamond_check_accesses(struct damon_ctx *ctx) +{ + struct damon_task *t; + struct mm_struct *mm; + struct damon_region *r; + + damon_for_each_task(ctx, t) { + mm = damon_get_mm(t); + if (!mm) + continue; + damon_for_each_region(r, t) + damon_check_access(ctx, mm, r); + mmput(mm); + } +} + +/** + * damon_check_reset_time_interval() - Check if a time interval is elapsed. + * @baseline: the time to check whether the interval has elapsed since + * @interval: the time interval (microseconds) + * + * See whether the given time interval has passed since the given baseline + * time. If so, it also updates the baseline to current time for next check. + * + * Return: true if the time interval has passed, or false otherwise. + */ +static bool damon_check_reset_time_interval(struct timespec64 *baseline, + unsigned long interval) +{ + struct timespec64 now; + + ktime_get_coarse_ts64(&now); + if ((timespec64_to_ns(&now) - timespec64_to_ns(baseline)) < + interval * 1000) + return false; + *baseline = now; + return true; +} + +/* + * Check whether it is time to flush the aggregated information + */ +static bool kdamond_aggregate_interval_passed(struct damon_ctx *ctx) +{ + return damon_check_reset_time_interval(&ctx->last_aggregation, + ctx->aggr_interval); +} + +/* + * Reset the aggregated monitoring results + */ +static void kdamond_reset_aggregated(struct damon_ctx *c) +{ + struct damon_task *t; + struct damon_region *r; + + damon_for_each_task(c, t) { + damon_for_each_region(r, t) + r->nr_accesses = 0; + } +} + +/* + * Check whether current monitoring should be stopped + * + * The monitoring is stopped when either the user requested to stop, or all + * monitoring target tasks are dead. + * + * Returns true if need to stop current monitoring. + */ +static bool kdamond_need_stop(struct damon_ctx *ctx) +{ + struct damon_task *t; + struct task_struct *task; + bool stop; + + mutex_lock(&ctx->kdamond_lock); + stop = ctx->kdamond_stop; + mutex_unlock(&ctx->kdamond_lock); + if (stop) + return true; + + damon_for_each_task(ctx, t) { + task = damon_get_task_struct(t); + if (task) { + put_task_struct(task); + return false; + } + } + + return true; +} + +/* + * The monitoring daemon that runs as a kernel thread + */ +static int kdamond_fn(void *data) +{ + struct damon_ctx *ctx = (struct damon_ctx *)data; + struct damon_task *t; + struct damon_region *r, *next; + + pr_info("kdamond (%d) starts\n", ctx->kdamond->pid); + kdamond_init_regions(ctx); + while (!kdamond_need_stop(ctx)) { + kdamond_prepare_access_checks(ctx); + + usleep_range(ctx->sample_interval, ctx->sample_interval + 1); + + kdamond_check_accesses(ctx); + + if (kdamond_aggregate_interval_passed(ctx)) + kdamond_reset_aggregated(ctx); + + } + damon_for_each_task(ctx, t) { + damon_for_each_region_safe(r, next, t) + damon_destroy_region(r); + } + pr_debug("kdamond (%d) finishes\n", ctx->kdamond->pid); + mutex_lock(&ctx->kdamond_lock); + ctx->kdamond = NULL; + mutex_unlock(&ctx->kdamond_lock); + + do_exit(0); +} + +/* + * Controller functions + */ + +static bool damon_kdamond_running(struct damon_ctx *ctx) +{ + bool running; + + mutex_lock(&ctx->kdamond_lock); + running = ctx->kdamond != NULL; + mutex_unlock(&ctx->kdamond_lock); + + return running; +} + +/** + * damon_start() - Starts monitoring with given context. + * @ctx: monitoring context + * + * Return: 0 on success, negative error code otherwise. + */ +int damon_start(struct damon_ctx *ctx) +{ + int err = -EBUSY; + + mutex_lock(&ctx->kdamond_lock); + if (!ctx->kdamond) { + err = 0; + ctx->kdamond_stop = false; + ctx->kdamond = kthread_run(kdamond_fn, ctx, "kdamond"); + if (IS_ERR(ctx->kdamond)) + err = PTR_ERR(ctx->kdamond); + } + mutex_unlock(&ctx->kdamond_lock); + + return err; +} + +/** + * damon_stop() - Stops monitoring of given context. + * @ctx: monitoring context + * + * Return: 0 on success, negative error code otherwise. + */ +int damon_stop(struct damon_ctx *ctx) +{ + mutex_lock(&ctx->kdamond_lock); + if (ctx->kdamond) { + ctx->kdamond_stop = true; + mutex_unlock(&ctx->kdamond_lock); + while (damon_kdamond_running(ctx)) + usleep_range(ctx->sample_interval, + ctx->sample_interval * 2); + return 0; + } + mutex_unlock(&ctx->kdamond_lock); + + return -EPERM; +} + +/** + * damon_set_pids() - Set monitoring target processes. + * @ctx: monitoring context + * @pids: array of target processes pids + * @nr_pids: number of entries in @pids + * + * This function should not be called while the kdamond is running. + * + * Return: 0 on success, negative error code otherwise. + */ +int damon_set_pids(struct damon_ctx *ctx, int *pids, ssize_t nr_pids) +{ + ssize_t i; + struct damon_task *t, *next; + + damon_for_each_task_safe(ctx, t, next) + damon_destroy_task(t); + + for (i = 0; i < nr_pids; i++) { + t = damon_new_task(pids[i]); + if (!t) { + pr_err("Failed to alloc damon_task\n"); + return -ENOMEM; + } + damon_add_task(ctx, t); + } + + return 0; +} + +/** + * damon_set_attrs() - Set attributes for the monitoring. + * @ctx: monitoring context + * @sample_int: time interval between samplings + * @aggr_int: time interval between aggregations + * @min_nr_reg: minimal number of regions + * + * This function should not be called while the kdamond is running. + * Every time interval is in micro-seconds. + * + * Return: 0 on success, negative error code otherwise. + */ +int damon_set_attrs(struct damon_ctx *ctx, unsigned long sample_int, + unsigned long aggr_int, unsigned long min_nr_reg) +{ + if (min_nr_reg < 3) { + pr_err("min_nr_regions (%lu) must be at least 3\n", + min_nr_reg); + return -EINVAL; + } + + ctx->sample_interval = sample_int; + ctx->aggr_interval = aggr_int; + ctx->min_nr_regions = min_nr_reg; + + return 0; +} + static int __init damon_init(void) { return 0; From patchwork Mon May 11 12:32:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 11540607 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EA22392A for ; Mon, 11 May 2020 12:36:34 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9B9A3207FF for ; Mon, 11 May 2020 12:36:34 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="vrjqUVP7" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9B9A3207FF Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=amazon.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C7CAC90003F; Mon, 11 May 2020 08:36:33 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C2D28900036; Mon, 11 May 2020 08:36:33 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AF54D90003F; Mon, 11 May 2020 08:36:33 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0069.hostedemail.com [216.40.44.69]) by kanga.kvack.org (Postfix) with ESMTP id 90ED6900036 for ; Mon, 11 May 2020 08:36:33 -0400 (EDT) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 57899180AD80F for ; Mon, 11 May 2020 12:36:33 +0000 (UTC) X-FDA: 76804386666.12.hat34_564938b7c5231 X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,prvs=3933f0155=sjpark@amazon.com,,RULES_HIT:30003:30012:30034:30045:30054:30056:30064:30070:30090,0,RBL:52.95.49.90:@amazon.com:.lbl8.mailshell.net-64.10.201.10 62.18.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:1,LUA_SUMMARY:none X-HE-Tag: hat34_564938b7c5231 X-Filterd-Recvd-Size: 13118 Received: from smtp-fw-6002.amazon.com (smtp-fw-6002.amazon.com [52.95.49.90]) by imf49.hostedemail.com (Postfix) with ESMTP for ; Mon, 11 May 2020 12:36:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1589200593; x=1620736593; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=N4V7UhFqV+AFt3/qS/5+jIDCBFo4tCk/bO8L4D+J8kc=; b=vrjqUVP7cXtkp7WyFMJD/FJOwJebvb5iFBRUw7eTLnYUzH+cjYuqFLto xm5YChITajga0iOFxXRwmeeG7Mb239hbWmvlrvWqgV1kyiRDdD12otqO1 nPodfsB3fOD7vkRP2V8lYN4BnU1fXi5ZSY65U/MPN96HsznZZ1I4agPtc 0=; IronPort-SDR: 8PRK515zKS4oCCP7MsC8PRniPnZpJkzYGLR/tRDmecx0/2I70E8NAYyThri22DjdW1E+t/u61S Ap1Md8anePwA== X-IronPort-AV: E=Sophos;i="5.73,379,1583193600"; d="scan'208";a="29558494" Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO email-inbound-relay-1d-9ec21598.us-east-1.amazon.com) ([10.43.8.6]) by smtp-border-fw-out-6002.iad6.amazon.com with ESMTP; 11 May 2020 12:36:32 +0000 Received: from EX13MTAUEA002.ant.amazon.com (iad55-ws-svc-p15-lb9-vlan3.iad.amazon.com [10.40.159.166]) by email-inbound-relay-1d-9ec21598.us-east-1.amazon.com (Postfix) with ESMTPS id E874DA1EE8; Mon, 11 May 2020 12:36:21 +0000 (UTC) Received: from EX13D31EUA001.ant.amazon.com (10.43.165.15) by EX13MTAUEA002.ant.amazon.com (10.43.61.77) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 11 May 2020 12:36:21 +0000 Received: from u886c93fd17d25d.ant.amazon.com (10.43.161.253) by EX13D31EUA001.ant.amazon.com (10.43.165.15) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 11 May 2020 12:36:05 +0000 From: SeongJae Park To: CC: SeongJae Park , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v11 05/16] mm/damon: Adaptively adjust regions Date: Mon, 11 May 2020 14:32:51 +0200 Message-ID: <20200511123302.12520-6-sjpark@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200511123302.12520-1-sjpark@amazon.com> References: <20200511123302.12520-1-sjpark@amazon.com> MIME-Version: 1.0 X-Originating-IP: [10.43.161.253] X-ClientProxiedBy: EX13D06UWC001.ant.amazon.com (10.43.162.91) To EX13D31EUA001.ant.amazon.com (10.43.165.15) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park At the beginning of the monitoring, DAMON constructs the initial regions by evenly splitting the memory mapped address space of the process into the user-specified minimal number of regions. In this initial state, the assumption of the regions (pages in same region have similar access frequencies) is normally not kept and thus the monitoring quality could be low. To keep the assumption as much as possible, DAMON adaptively merges and splits each region. For each ``aggregation interval``, it compares the access frequencies of adjacent regions and merges those if the frequency difference is small. Then, after it reports and clears the aggregated access frequency of each region, it splits each region into two regions if the total number of regions is smaller than the half of the user-specified maximum number of regions. In this way, DAMON provides its best-effort quality and minimal overhead while keeping the bounds users set for their trade-off. Signed-off-by: SeongJae Park --- include/linux/damon.h | 6 +- mm/damon.c | 158 +++++++++++++++++++++++++++++++++++++++--- 2 files changed, 152 insertions(+), 12 deletions(-) diff --git a/include/linux/damon.h b/include/linux/damon.h index 558dd6ae0afa..f1c3f491fc50 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -42,6 +42,7 @@ struct damon_ctx { unsigned long sample_interval; unsigned long aggr_interval; unsigned long min_nr_regions; + unsigned long max_nr_regions; struct timespec64 last_aggregation; @@ -53,8 +54,9 @@ struct damon_ctx { }; int damon_set_pids(struct damon_ctx *ctx, int *pids, ssize_t nr_pids); -int damon_set_attrs(struct damon_ctx *ctx, unsigned long sample_int, - unsigned long aggr_int, unsigned long min_nr_reg); +int damon_set_attrs(struct damon_ctx *ctx, + unsigned long sample_int, unsigned long aggr_int, + unsigned long min_nr_reg, unsigned long max_nr_reg); int damon_start(struct damon_ctx *ctx); int damon_stop(struct damon_ctx *ctx); diff --git a/mm/damon.c b/mm/damon.c index 9c9aa18e846a..cec946197e13 100644 --- a/mm/damon.c +++ b/mm/damon.c @@ -332,9 +332,12 @@ static int damon_three_regions_of(struct damon_task *t, * regions is wasteful. That said, because we can deal with small noises, * tracking every mapping is not strictly required but could even incur a high * overhead if the mapping frequently changes or the number of mappings is - * high. Nonetheless, this may seems very weird. DAMON's dynamic regions - * adjustment mechanism, which will be implemented with following commit will - * make this more sense. + * high. The adaptive regions adjustment mechanism will further help to deal + * with the noise by simply identifying the unmapped areas as a region that + * has no access. Moreover, applying the real mappings that would have many + * unmapped areas inside will make the adaptive mechanism quite complex. That + * said, too huge unmapped areas inside the monitoring target should be removed + * to not take the time for the adaptive mechanism. * * For the reason, we convert the complex mappings to three distinct regions * that cover every mapped area of the address space. Also the two gaps @@ -508,20 +511,25 @@ static void damon_check_access(struct damon_ctx *ctx, last_addr = r->sampling_addr; } -static void kdamond_check_accesses(struct damon_ctx *ctx) +static unsigned int kdamond_check_accesses(struct damon_ctx *ctx) { struct damon_task *t; struct mm_struct *mm; struct damon_region *r; + unsigned int max_nr_accesses = 0; damon_for_each_task(ctx, t) { mm = damon_get_mm(t); if (!mm) continue; - damon_for_each_region(r, t) + damon_for_each_region(r, t) { damon_check_access(ctx, mm, r); + max_nr_accesses = max(r->nr_accesses, max_nr_accesses); + } + mmput(mm); } + return max_nr_accesses; } /** @@ -570,6 +578,125 @@ static void kdamond_reset_aggregated(struct damon_ctx *c) } } +#define sz_damon_region(r) (r->vm_end - r->vm_start) + +/* + * Merge two adjacent regions into one region + */ +static void damon_merge_two_regions(struct damon_region *l, + struct damon_region *r) +{ + l->nr_accesses = (l->nr_accesses * sz_damon_region(l) + + r->nr_accesses * sz_damon_region(r)) / + (sz_damon_region(l) + sz_damon_region(r)); + l->vm_end = r->vm_end; + damon_destroy_region(r); +} + +#define diff_of(a, b) (a > b ? a - b : b - a) + +/* + * Merge adjacent regions having similar access frequencies + * + * t task affected by merge operation + * thres '->nr_accesses' diff threshold for the merge + */ +static void damon_merge_regions_of(struct damon_task *t, unsigned int thres) +{ + struct damon_region *r, *prev = NULL, *next; + + damon_for_each_region_safe(r, next, t) { + if (!prev || prev->vm_end != r->vm_start || + diff_of(prev->nr_accesses, r->nr_accesses) > thres) { + prev = r; + continue; + } + damon_merge_two_regions(prev, r); + } +} + +/* + * Merge adjacent regions having similar access frequencies + * + * threshold merge regions having nr_accesses diff larger than this + * + * This function merges monitoring target regions which are adjacent and their + * access frequencies are similar. This is for minimizing the monitoring + * overhead under the dynamically changeable access pattern. If a merge was + * unnecessarily made, later 'kdamond_split_regions()' will revert it. + */ +static void kdamond_merge_regions(struct damon_ctx *c, unsigned int threshold) +{ + struct damon_task *t; + + damon_for_each_task(c, t) + damon_merge_regions_of(t, threshold); +} + +/* + * Split a region in two + * + * r the region to be split + * sz_r size of the first sub-region that will be made + */ +static void damon_split_region_at(struct damon_ctx *ctx, + struct damon_region *r, unsigned long sz_r) +{ + struct damon_region *new; + + new = damon_new_region(ctx, r->vm_start + sz_r, r->vm_end); + r->vm_end = new->vm_start; + + damon_insert_region(new, r, damon_next_region(r)); +} + +/* Split every region in the given task into two randomly-sized regions */ +static void damon_split_regions_of(struct damon_ctx *ctx, struct damon_task *t) +{ + struct damon_region *r, *next; + unsigned long sz_orig_region, sz_left_region; + + damon_for_each_region_safe(r, next, t) { + sz_orig_region = r->vm_end - r->vm_start; + + /* + * Randomly select size of left sub-region to be at least + * 10 percent and at most 90% of original region + */ + sz_left_region = ALIGN_DOWN(damon_rand(1, 10) * sz_orig_region + / 10, MIN_REGION); + /* Do not allow blank region */ + if (sz_left_region == 0 || sz_left_region >= sz_orig_region) + continue; + + damon_split_region_at(ctx, r, sz_left_region); + } +} + +/* + * splits every target region into two randomly-sized regions + * + * This function splits every target region into two random-sized regions if + * current total number of the regions is equal or smaller than half of the + * user-specified maximum number of regions. This is for maximizing the + * monitoring accuracy under the dynamically changeable access patterns. If a + * split was unnecessarily made, later 'kdamond_merge_regions()' will revert + * it. + */ +static void kdamond_split_regions(struct damon_ctx *ctx) +{ + struct damon_task *t; + unsigned int nr_regions = 0; + + damon_for_each_task(ctx, t) + nr_regions += nr_damon_regions(t); + if (nr_regions > ctx->max_nr_regions / 2) + return; + + damon_for_each_task(ctx, t) + damon_split_regions_of(ctx, t); +} + /* * Check whether current monitoring should be stopped * @@ -609,6 +736,7 @@ static int kdamond_fn(void *data) struct damon_ctx *ctx = (struct damon_ctx *)data; struct damon_task *t; struct damon_region *r, *next; + unsigned int max_nr_accesses = 0; pr_info("kdamond (%d) starts\n", ctx->kdamond->pid); kdamond_init_regions(ctx); @@ -617,11 +745,13 @@ static int kdamond_fn(void *data) usleep_range(ctx->sample_interval, ctx->sample_interval + 1); - kdamond_check_accesses(ctx); + max_nr_accesses = kdamond_check_accesses(ctx); - if (kdamond_aggregate_interval_passed(ctx)) + if (kdamond_aggregate_interval_passed(ctx)) { + kdamond_merge_regions(ctx, max_nr_accesses / 10); kdamond_reset_aggregated(ctx); - + kdamond_split_regions(ctx); + } } damon_for_each_task(ctx, t) { damon_for_each_region_safe(r, next, t) @@ -731,24 +861,32 @@ int damon_set_pids(struct damon_ctx *ctx, int *pids, ssize_t nr_pids) * @sample_int: time interval between samplings * @aggr_int: time interval between aggregations * @min_nr_reg: minimal number of regions + * @max_nr_reg: maximum number of regions * * This function should not be called while the kdamond is running. * Every time interval is in micro-seconds. * * Return: 0 on success, negative error code otherwise. */ -int damon_set_attrs(struct damon_ctx *ctx, unsigned long sample_int, - unsigned long aggr_int, unsigned long min_nr_reg) +int damon_set_attrs(struct damon_ctx *ctx, + unsigned long sample_int, unsigned long aggr_int, + unsigned long min_nr_reg, unsigned long max_nr_reg) { if (min_nr_reg < 3) { pr_err("min_nr_regions (%lu) must be at least 3\n", min_nr_reg); return -EINVAL; } + if (min_nr_reg > max_nr_reg) { + pr_err("invalid nr_regions. min (%lu) > max (%lu)\n", + min_nr_reg, max_nr_reg); + return -EINVAL; + } ctx->sample_interval = sample_int; ctx->aggr_interval = aggr_int; ctx->min_nr_regions = min_nr_reg; + ctx->max_nr_regions = max_nr_reg; return 0; } From patchwork Mon May 11 12:32:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 11540609 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 60469912 for ; Mon, 11 May 2020 12:36:56 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 233962082E for ; Mon, 11 May 2020 12:36:56 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="m5uDmr/5" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 233962082E Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=amazon.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6141D900040; Mon, 11 May 2020 08:36:55 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 59C18900036; Mon, 11 May 2020 08:36:55 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 46531900040; Mon, 11 May 2020 08:36:55 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0243.hostedemail.com [216.40.44.243]) by kanga.kvack.org (Postfix) with ESMTP id 28253900036 for ; Mon, 11 May 2020 08:36:55 -0400 (EDT) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id DC2122480 for ; Mon, 11 May 2020 12:36:54 +0000 (UTC) X-FDA: 76804387548.09.grass60_5966dd886c22c X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,prvs=3933f0155=sjpark@amazon.com,,RULES_HIT:30034:30054:30064:30070,0,RBL:207.171.184.25:@amazon.com:.lbl8.mailshell.net-66.10.201.10 62.18.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:22,LUA_SUMMARY:none X-HE-Tag: grass60_5966dd886c22c X-Filterd-Recvd-Size: 6466 Received: from smtp-fw-9101.amazon.com (smtp-fw-9101.amazon.com [207.171.184.25]) by imf13.hostedemail.com (Postfix) with ESMTP for ; Mon, 11 May 2020 12:36:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1589200615; x=1620736615; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=gm3RkgBnFlsR0KLH3wADEsc5/8hUuBZNui1ctNCxs+I=; b=m5uDmr/53C8vOu/mdZJ6z2nIusCQPSHtaIGeAyzRxIwGTJ5VSZKxBDRq Ft0hlw1urF8I5fXY3/gVrIoTsFP/FPWGwVcKF+Qclronb4Nd1DGniy513 PaopDtMf2De0c/3sKvIEWo78vkaj7x7nt2pqw4UNYcHNRfbcetQQ8+Sei s=; IronPort-SDR: bD3UbNFmyebn4uwq18PQt0i8oiWF+RXahclxjzXJksOdvJkbbH5fFLifCG5di9rzz3zM8c6HQk dpxbU9ZLF78w== X-IronPort-AV: E=Sophos;i="5.73,380,1583193600"; d="scan'208";a="34195023" Received: from sea32-co-svc-lb4-vlan2.sea.corp.amazon.com (HELO email-inbound-relay-1d-474bcd9f.us-east-1.amazon.com) ([10.47.23.34]) by smtp-border-fw-out-9101.sea19.amazon.com with ESMTP; 11 May 2020 12:36:49 +0000 Received: from EX13MTAUEA002.ant.amazon.com (iad55-ws-svc-p15-lb9-vlan2.iad.amazon.com [10.40.159.162]) by email-inbound-relay-1d-474bcd9f.us-east-1.amazon.com (Postfix) with ESMTPS id 5E2E3A2340; Mon, 11 May 2020 12:36:38 +0000 (UTC) Received: from EX13D31EUA001.ant.amazon.com (10.43.165.15) by EX13MTAUEA002.ant.amazon.com (10.43.61.77) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 11 May 2020 12:36:37 +0000 Received: from u886c93fd17d25d.ant.amazon.com (10.43.161.253) by EX13D31EUA001.ant.amazon.com (10.43.165.15) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 11 May 2020 12:36:22 +0000 From: SeongJae Park To: CC: SeongJae Park , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v11 06/16] mm/damon: Split regions into 3 subregions if necessary Date: Mon, 11 May 2020 14:32:52 +0200 Message-ID: <20200511123302.12520-7-sjpark@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200511123302.12520-1-sjpark@amazon.com> References: <20200511123302.12520-1-sjpark@amazon.com> MIME-Version: 1.0 X-Originating-IP: [10.43.161.253] X-ClientProxiedBy: EX13D06UWC001.ant.amazon.com (10.43.162.91) To EX13D31EUA001.ant.amazon.com (10.43.165.15) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park Suppose that there are a very wide and cold region and a hot region, and both regions are identified by DAMON. And then, the middle small region inside the very wide and cold region becomes hot. DAMON will not be able to identify this new region because the adaptive regions adjustment mechanism splits each region to only two subregions. This commit modifies the logic to split each region to 3 subregions if such corner case is detected. Suggested-by: James Cameron Signed-off-by: SeongJae Park --- mm/damon.c | 51 ++++++++++++++++++++++++++++++++++----------------- 1 file changed, 34 insertions(+), 17 deletions(-) diff --git a/mm/damon.c b/mm/damon.c index cec946197e13..fbf37df841ce 100644 --- a/mm/damon.c +++ b/mm/damon.c @@ -650,26 +650,32 @@ static void damon_split_region_at(struct damon_ctx *ctx, damon_insert_region(new, r, damon_next_region(r)); } -/* Split every region in the given task into two randomly-sized regions */ -static void damon_split_regions_of(struct damon_ctx *ctx, struct damon_task *t) +/* Split every region in the given task into 'nr_subs' regions */ +static void damon_split_regions_of(struct damon_ctx *ctx, + struct damon_task *t, int nr_subs) { struct damon_region *r, *next; - unsigned long sz_orig_region, sz_left_region; + unsigned long sz_region, sz_sub = 0; + int i; damon_for_each_region_safe(r, next, t) { - sz_orig_region = r->vm_end - r->vm_start; - - /* - * Randomly select size of left sub-region to be at least - * 10 percent and at most 90% of original region - */ - sz_left_region = ALIGN_DOWN(damon_rand(1, 10) * sz_orig_region - / 10, MIN_REGION); - /* Do not allow blank region */ - if (sz_left_region == 0 || sz_left_region >= sz_orig_region) - continue; - - damon_split_region_at(ctx, r, sz_left_region); + sz_region = r->vm_end - r->vm_start; + + for (i = 0; i < nr_subs - 1 && + sz_region > 2 * MIN_REGION; i++) { + /* + * Randomly select size of left sub-region to be at + * least 10 percent and at most 90% of original region + */ + sz_sub = ALIGN_DOWN(damon_rand(1, 10) * + sz_region / 10, MIN_REGION); + /* Do not allow blank region */ + if (sz_sub == 0 || sz_sub >= sz_region) + continue; + + damon_split_region_at(ctx, r, sz_sub); + sz_region = sz_sub; + } } } @@ -687,14 +693,25 @@ static void kdamond_split_regions(struct damon_ctx *ctx) { struct damon_task *t; unsigned int nr_regions = 0; + static unsigned int last_nr_regions; + int nr_subregions = 2; damon_for_each_task(ctx, t) nr_regions += nr_damon_regions(t); + if (nr_regions > ctx->max_nr_regions / 2) return; + /* If number of regions is not changed, we are maybe in corner case */ + if (last_nr_regions == nr_regions && + nr_regions < ctx->max_nr_regions / 3) + nr_subregions = 3; + damon_for_each_task(ctx, t) - damon_split_regions_of(ctx, t); + damon_split_regions_of(ctx, t, nr_subregions); + + if (!last_nr_regions) + last_nr_regions = nr_regions; } /* From patchwork Mon May 11 12:32:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 11540615 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B414792A for ; Mon, 11 May 2020 12:37:33 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7EC902184D for ; Mon, 11 May 2020 12:37:33 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="n8BW6dSO" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7EC902184D Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=amazon.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8F95C900041; Mon, 11 May 2020 08:37:32 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 8836F900036; Mon, 11 May 2020 08:37:32 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 77143900041; Mon, 11 May 2020 08:37:32 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0149.hostedemail.com [216.40.44.149]) by kanga.kvack.org (Postfix) with ESMTP id 61253900036 for ; Mon, 11 May 2020 08:37:32 -0400 (EDT) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 28592180AD80F for ; Mon, 11 May 2020 12:37:32 +0000 (UTC) X-FDA: 76804389144.09.screw33_5ed02afba4219 X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,prvs=3933f0155=sjpark@amazon.com,,RULES_HIT:30003:30012:30034:30054:30064:30070:30075,0,RBL:207.171.184.29:@amazon.com:.lbl8.mailshell.net-66.10.201.10 62.18.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: screw33_5ed02afba4219 X-Filterd-Recvd-Size: 9727 Received: from smtp-fw-9102.amazon.com (smtp-fw-9102.amazon.com [207.171.184.29]) by imf10.hostedemail.com (Postfix) with ESMTP for ; Mon, 11 May 2020 12:37:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1589200652; x=1620736652; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=ppoTbPGG8Qa0ZJDp/T/jyLvR2piIBoFyJksYX+ogf6k=; b=n8BW6dSO+X1UUSM5lkld37XjH1jGtG5XfTdAYCB9fPWbhN+8gySoX1qf Qx0sEsF932HKFYehj/UL5Ymd1osXLTrO14YcYCBnzHRhsZAI9SOaf2sFq 13rCkeIp4312Jx+rxBSaajHCDQXA/fDdCL711GKDJGA5nO3BlZU3aCgtc k=; IronPort-SDR: 1+3dUDfl9wT23abs+SoWWRF48iFUAwskpxBN4VsnAje0Ghy7OanoJUlcGa9+MEdJYV5DHqfdfW 4B7nMPY4cXUQ== X-IronPort-AV: E=Sophos;i="5.73,379,1583193600"; d="scan'208";a="42510458" Received: from sea32-co-svc-lb4-vlan2.sea.corp.amazon.com (HELO email-inbound-relay-1a-821c648d.us-east-1.amazon.com) ([10.47.23.34]) by smtp-border-fw-out-9102.sea19.amazon.com with ESMTP; 11 May 2020 12:37:27 +0000 Received: from EX13MTAUEA002.ant.amazon.com (iad55-ws-svc-p15-lb9-vlan3.iad.amazon.com [10.40.159.166]) by email-inbound-relay-1a-821c648d.us-east-1.amazon.com (Postfix) with ESMTPS id AA7B2A2390; Mon, 11 May 2020 12:37:15 +0000 (UTC) Received: from EX13D31EUA001.ant.amazon.com (10.43.165.15) by EX13MTAUEA002.ant.amazon.com (10.43.61.77) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 11 May 2020 12:37:14 +0000 Received: from u886c93fd17d25d.ant.amazon.com (10.43.161.253) by EX13D31EUA001.ant.amazon.com (10.43.165.15) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 11 May 2020 12:36:59 +0000 From: SeongJae Park To: CC: SeongJae Park , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v11 07/16] mm/damon: Apply dynamic memory mapping changes Date: Mon, 11 May 2020 14:32:53 +0200 Message-ID: <20200511123302.12520-8-sjpark@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200511123302.12520-1-sjpark@amazon.com> References: <20200511123302.12520-1-sjpark@amazon.com> MIME-Version: 1.0 X-Originating-IP: [10.43.161.253] X-ClientProxiedBy: EX13D06UWC001.ant.amazon.com (10.43.162.91) To EX13D31EUA001.ant.amazon.com (10.43.165.15) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park Only a number of parts in the virtual address space of the processes is mapped to physical memory and accessed. Thus, tracking the unmapped address regions is just wasteful. However, tracking every memory mapping change might incur an overhead. For the reason, DAMON applies the dynamic memory mapping changes to the tracking regions only for each of a user-specified time interval (``regions update interval``). Signed-off-by: SeongJae Park --- include/linux/damon.h | 10 +++-- mm/damon.c | 101 +++++++++++++++++++++++++++++++++++++++++- 2 files changed, 106 insertions(+), 5 deletions(-) diff --git a/include/linux/damon.h b/include/linux/damon.h index f1c3f491fc50..62b9f90ed87b 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -34,17 +34,21 @@ struct damon_task { /* * For each 'sample_interval', DAMON checks whether each region is accessed or * not. It aggregates and keeps the access information (number of accesses to - * each region) for each 'aggr_interval' time. + * each region) for 'aggr_interval' time. DAMON also checks whether the memory + * mapping of the target tasks has changed (e.g., by mmap() calls from the + * application) and applies the changes for each 'regions_update_interval'. * * All time intervals are in micro-seconds. */ struct damon_ctx { unsigned long sample_interval; unsigned long aggr_interval; + unsigned long regions_update_interval; unsigned long min_nr_regions; unsigned long max_nr_regions; struct timespec64 last_aggregation; + struct timespec64 last_regions_update; struct task_struct *kdamond; bool kdamond_stop; @@ -54,8 +58,8 @@ struct damon_ctx { }; int damon_set_pids(struct damon_ctx *ctx, int *pids, ssize_t nr_pids); -int damon_set_attrs(struct damon_ctx *ctx, - unsigned long sample_int, unsigned long aggr_int, +int damon_set_attrs(struct damon_ctx *ctx, unsigned long sample_int, + unsigned long aggr_int, unsigned long regions_update_int, unsigned long min_nr_reg, unsigned long max_nr_reg); int damon_start(struct damon_ctx *ctx); int damon_stop(struct damon_ctx *ctx); diff --git a/mm/damon.c b/mm/damon.c index fbf37df841ce..3d0f9428c925 100644 --- a/mm/damon.c +++ b/mm/damon.c @@ -714,6 +714,98 @@ static void kdamond_split_regions(struct damon_ctx *ctx) last_nr_regions = nr_regions; } +/* + * Check whether it is time to check and apply the dynamic mmap changes + * + * Returns true if it is. + */ +static bool kdamond_need_update_regions(struct damon_ctx *ctx) +{ + return damon_check_reset_time_interval(&ctx->last_regions_update, + ctx->regions_update_interval); +} + +/* + * Check whether regions are intersecting + * + * Note that this function checks 'struct damon_region' and 'struct region'. + * + * Returns true if it is. + */ +static bool damon_intersect(struct damon_region *r, struct region *re) +{ + return !(r->vm_end <= re->start || re->end <= r->vm_start); +} + +/* + * Update damon regions for the three big regions of the given task + * + * t the given task + * bregions the three big regions of the task + */ +static void damon_apply_three_regions(struct damon_ctx *ctx, + struct damon_task *t, struct region bregions[3]) +{ + struct damon_region *r, *next; + unsigned int i = 0; + + /* Remove regions which are not in the three big regions now */ + damon_for_each_region_safe(r, next, t) { + for (i = 0; i < 3; i++) { + if (damon_intersect(r, &bregions[i])) + break; + } + if (i == 3) + damon_destroy_region(r); + } + + /* Adjust intersecting regions to fit with the three big regions */ + for (i = 0; i < 3; i++) { + struct damon_region *first = NULL, *last; + struct damon_region *newr; + struct region *br; + + br = &bregions[i]; + /* Get the first and last regions which intersects with br */ + damon_for_each_region(r, t) { + if (damon_intersect(r, br)) { + if (!first) + first = r; + last = r; + } + if (r->vm_start >= br->end) + break; + } + if (!first) { + /* no damon_region intersects with this big region */ + newr = damon_new_region(ctx, + ALIGN_DOWN(br->start, MIN_REGION), + ALIGN(br->end, MIN_REGION)); + if (!newr) + continue; + damon_insert_region(newr, damon_prev_region(r), r); + } else { + first->vm_start = ALIGN_DOWN(br->start, MIN_REGION); + last->vm_end = ALIGN(br->end, MIN_REGION); + } + } +} + +/* + * Update regions for current memory mappings + */ +static void kdamond_update_regions(struct damon_ctx *ctx) +{ + struct region three_regions[3]; + struct damon_task *t; + + damon_for_each_task(ctx, t) { + if (damon_three_regions_of(t, three_regions)) + continue; + damon_apply_three_regions(ctx, t, three_regions); + } +} + /* * Check whether current monitoring should be stopped * @@ -769,6 +861,9 @@ static int kdamond_fn(void *data) kdamond_reset_aggregated(ctx); kdamond_split_regions(ctx); } + + if (kdamond_need_update_regions(ctx)) + kdamond_update_regions(ctx); } damon_for_each_task(ctx, t) { damon_for_each_region_safe(r, next, t) @@ -876,6 +971,7 @@ int damon_set_pids(struct damon_ctx *ctx, int *pids, ssize_t nr_pids) * damon_set_attrs() - Set attributes for the monitoring. * @ctx: monitoring context * @sample_int: time interval between samplings + * @regions_update_int: time interval between vma update checks * @aggr_int: time interval between aggregations * @min_nr_reg: minimal number of regions * @max_nr_reg: maximum number of regions @@ -885,8 +981,8 @@ int damon_set_pids(struct damon_ctx *ctx, int *pids, ssize_t nr_pids) * * Return: 0 on success, negative error code otherwise. */ -int damon_set_attrs(struct damon_ctx *ctx, - unsigned long sample_int, unsigned long aggr_int, +int damon_set_attrs(struct damon_ctx *ctx, unsigned long sample_int, + unsigned long aggr_int, unsigned long regions_update_int, unsigned long min_nr_reg, unsigned long max_nr_reg) { if (min_nr_reg < 3) { @@ -902,6 +998,7 @@ int damon_set_attrs(struct damon_ctx *ctx, ctx->sample_interval = sample_int; ctx->aggr_interval = aggr_int; + ctx->regions_update_interval = regions_update_int; ctx->min_nr_regions = min_nr_reg; ctx->max_nr_regions = max_nr_reg; From patchwork Mon May 11 12:32:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 11540617 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 16B06912 for ; Mon, 11 May 2020 12:37:52 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D575720746 for ; Mon, 11 May 2020 12:37:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="syp74Bd8" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D575720746 Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=amazon.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 1A511900042; Mon, 11 May 2020 08:37:51 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 1568A900036; Mon, 11 May 2020 08:37:51 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 01F39900042; Mon, 11 May 2020 08:37:50 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0093.hostedemail.com [216.40.44.93]) by kanga.kvack.org (Postfix) with ESMTP id DAD6B900036 for ; Mon, 11 May 2020 08:37:50 -0400 (EDT) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 9AC69180AD80F for ; Mon, 11 May 2020 12:37:50 +0000 (UTC) X-FDA: 76804389900.18.spade53_617f49572dc38 X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,prvs=3933f0155=sjpark@amazon.com,,RULES_HIT:30054:30064,0,RBL:207.171.190.10:@amazon.com:.lbl8.mailshell.net-66.10.201.10 62.18.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:23,LUA_SUMMARY:none X-HE-Tag: spade53_617f49572dc38 X-Filterd-Recvd-Size: 5085 Received: from smtp-fw-33001.amazon.com (smtp-fw-33001.amazon.com [207.171.190.10]) by imf25.hostedemail.com (Postfix) with ESMTP for ; Mon, 11 May 2020 12:37:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1589200670; x=1620736670; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=3qzhEO45OCs2xK7YXoqTIw9W4rei20Zjp8AHQQpGetw=; b=syp74Bd8T5ReoZdWi1ltp2Myfn3FlYxl7d7TmpWmtVLh+auhFD4YhJOz qHRnAkgB87xCtnuRk3F4Rl0MrW3Tlq8A3vcYqA/BylCb9fXKbKb5DTXEp w0th/WvHiREuB6Ev1LjuPuAURkVz2Yz5+VgMD4y3Td1if4jPM6f44Vd1x w=; IronPort-SDR: ijEDsp/2QAQashspAjWy4I7QTeWERsghd33wV575sjyRQpFaalMCuVZTTCmxztYXlWMYowWyGA DFMl5rNxfuRg== X-IronPort-AV: E=Sophos;i="5.73,380,1583193600"; d="scan'208";a="43930682" Received: from sea32-co-svc-lb4-vlan2.sea.corp.amazon.com (HELO email-inbound-relay-1a-af6a10df.us-east-1.amazon.com) ([10.47.23.34]) by smtp-border-fw-out-33001.sea14.amazon.com with ESMTP; 11 May 2020 12:37:47 +0000 Received: from EX13MTAUEA002.ant.amazon.com (iad55-ws-svc-p15-lb9-vlan3.iad.amazon.com [10.40.159.166]) by email-inbound-relay-1a-af6a10df.us-east-1.amazon.com (Postfix) with ESMTPS id D4E0DA1FD2; Mon, 11 May 2020 12:37:35 +0000 (UTC) Received: from EX13D31EUA001.ant.amazon.com (10.43.165.15) by EX13MTAUEA002.ant.amazon.com (10.43.61.77) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 11 May 2020 12:37:35 +0000 Received: from u886c93fd17d25d.ant.amazon.com (10.43.161.253) by EX13D31EUA001.ant.amazon.com (10.43.165.15) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 11 May 2020 12:37:19 +0000 From: SeongJae Park To: CC: SeongJae Park , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v11 08/16] mm/damon: Implement callbacks Date: Mon, 11 May 2020 14:32:54 +0200 Message-ID: <20200511123302.12520-9-sjpark@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200511123302.12520-1-sjpark@amazon.com> References: <20200511123302.12520-1-sjpark@amazon.com> MIME-Version: 1.0 X-Originating-IP: [10.43.161.253] X-ClientProxiedBy: EX13D06UWC001.ant.amazon.com (10.43.162.91) To EX13D31EUA001.ant.amazon.com (10.43.165.15) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park This commit implements callbacks for DAMON. Using this, DAMON users can install their callbacks for each step of the access monitoring so that they can do something interesting with the monitored access patterns online. For example, callbacks can report the monitored patterns to users or do some access pattern based memory management such as proactive reclamations or access pattern based THP promotions/demotions decision makings. Signed-off-by: SeongJae Park --- include/linux/damon.h | 4 ++++ mm/damon.c | 4 ++++ 2 files changed, 8 insertions(+) diff --git a/include/linux/damon.h b/include/linux/damon.h index 62b9f90ed87b..264569b21502 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -55,6 +55,10 @@ struct damon_ctx { struct mutex kdamond_lock; struct list_head tasks_list; /* 'damon_task' objects */ + + /* callbacks */ + void (*sample_cb)(struct damon_ctx *context); + void (*aggregate_cb)(struct damon_ctx *context); }; int damon_set_pids(struct damon_ctx *ctx, int *pids, ssize_t nr_pids); diff --git a/mm/damon.c b/mm/damon.c index 3d0f9428c925..6b01aa641503 100644 --- a/mm/damon.c +++ b/mm/damon.c @@ -851,6 +851,8 @@ static int kdamond_fn(void *data) kdamond_init_regions(ctx); while (!kdamond_need_stop(ctx)) { kdamond_prepare_access_checks(ctx); + if (ctx->sample_cb) + ctx->sample_cb(ctx); usleep_range(ctx->sample_interval, ctx->sample_interval + 1); @@ -858,6 +860,8 @@ static int kdamond_fn(void *data) if (kdamond_aggregate_interval_passed(ctx)) { kdamond_merge_regions(ctx, max_nr_accesses / 10); + if (ctx->aggregate_cb) + ctx->aggregate_cb(ctx); kdamond_reset_aggregated(ctx); kdamond_split_regions(ctx); } From patchwork Mon May 11 12:32:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 11540621 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AD40792A for ; Mon, 11 May 2020 12:38:10 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6CC9A208CA for ; Mon, 11 May 2020 12:38:10 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="Pytc/nAj" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6CC9A208CA Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=amazon.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9E2BE900043; Mon, 11 May 2020 08:38:09 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 96B2F900036; Mon, 11 May 2020 08:38:09 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 834AB900043; Mon, 11 May 2020 08:38:09 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0063.hostedemail.com [216.40.44.63]) by kanga.kvack.org (Postfix) with ESMTP id 68DB8900036 for ; Mon, 11 May 2020 08:38:09 -0400 (EDT) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 13CCF1EE6 for ; Mon, 11 May 2020 12:38:09 +0000 (UTC) X-FDA: 76804390698.05.view58_64320633e004c X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,prvs=3933f0155=sjpark@amazon.com,,RULES_HIT:30003:30012:30034:30051:30054:30064:30070:30075,0,RBL:207.171.184.25:@amazon.com:.lbl8.mailshell.net-62.18.0.100 66.10.201.10,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:23,LUA_SUMMARY:none X-HE-Tag: view58_64320633e004c X-Filterd-Recvd-Size: 10097 Received: from smtp-fw-9101.amazon.com (smtp-fw-9101.amazon.com [207.171.184.25]) by imf36.hostedemail.com (Postfix) with ESMTP for ; Mon, 11 May 2020 12:38:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1589200689; x=1620736689; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=W3k9486TiUCXIiW/oNgYHBamjnpyXjK4jEu9bY7t9VA=; b=Pytc/nAjfHcNoykJNSqjVW+08TKI9c4j3SJgGu3//80ylzapYhu1EjLG DBEnaWeIKTGX/uSUO87mgcFl3Nob2iRDlxtY8Ykwxx5oRa8VivIiNBB1r UK63IAshHSAnDGF97QtI7F+Fk9WXnILk0xeiNiRw+tKY9IKZcB4fo8+VD o=; IronPort-SDR: 4Db7iDOJUcTamZ/BaSNIZdEn4u+dcGHJw/glBhf52nJ0svpAi4IJsBOH4yF8/NAw56gfmBKR6h 0V0BTcHZuBjA== X-IronPort-AV: E=Sophos;i="5.73,380,1583193600"; d="scan'208";a="34195384" Received: from sea32-co-svc-lb4-vlan2.sea.corp.amazon.com (HELO email-inbound-relay-1e-17c49630.us-east-1.amazon.com) ([10.47.23.34]) by smtp-border-fw-out-9101.sea19.amazon.com with ESMTP; 11 May 2020 12:38:05 +0000 Received: from EX13MTAUEA002.ant.amazon.com (iad55-ws-svc-p15-lb9-vlan3.iad.amazon.com [10.40.159.166]) by email-inbound-relay-1e-17c49630.us-east-1.amazon.com (Postfix) with ESMTPS id E30E5A20DC; Mon, 11 May 2020 12:37:53 +0000 (UTC) Received: from EX13D31EUA001.ant.amazon.com (10.43.165.15) by EX13MTAUEA002.ant.amazon.com (10.43.61.77) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 11 May 2020 12:37:53 +0000 Received: from u886c93fd17d25d.ant.amazon.com (10.43.161.253) by EX13D31EUA001.ant.amazon.com (10.43.165.15) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 11 May 2020 12:37:37 +0000 From: SeongJae Park To: CC: SeongJae Park , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v11 09/16] mm/damon: Implement access pattern recording Date: Mon, 11 May 2020 14:32:55 +0200 Message-ID: <20200511123302.12520-10-sjpark@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200511123302.12520-1-sjpark@amazon.com> References: <20200511123302.12520-1-sjpark@amazon.com> MIME-Version: 1.0 X-Originating-IP: [10.43.161.253] X-ClientProxiedBy: EX13D06UWC001.ant.amazon.com (10.43.162.91) To EX13D31EUA001.ant.amazon.com (10.43.165.15) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park This commit implements the recording feature of DAMON. If this feature is enabled, DAMON writes the monitored access patterns in its binary format into a file which specified by the user. This is already able to be implemented by each user using the callbacks. However, as the recording is expected to be used widely, this commit implements the feature in the DAMON, for more convenience and efficiency. Signed-off-by: SeongJae Park --- include/linux/damon.h | 7 +++ mm/damon.c | 131 +++++++++++++++++++++++++++++++++++++++++- 2 files changed, 135 insertions(+), 3 deletions(-) diff --git a/include/linux/damon.h b/include/linux/damon.h index 264569b21502..bc46ea00e9a1 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -50,6 +50,11 @@ struct damon_ctx { struct timespec64 last_aggregation; struct timespec64 last_regions_update; + unsigned char *rbuf; + unsigned int rbuf_len; + unsigned int rbuf_offset; + char *rfile_path; + struct task_struct *kdamond; bool kdamond_stop; struct mutex kdamond_lock; @@ -65,6 +70,8 @@ int damon_set_pids(struct damon_ctx *ctx, int *pids, ssize_t nr_pids); int damon_set_attrs(struct damon_ctx *ctx, unsigned long sample_int, unsigned long aggr_int, unsigned long regions_update_int, unsigned long min_nr_reg, unsigned long max_nr_reg); +int damon_set_recording(struct damon_ctx *ctx, + unsigned int rbuf_len, char *rfile_path); int damon_start(struct damon_ctx *ctx); int damon_stop(struct damon_ctx *ctx); diff --git a/mm/damon.c b/mm/damon.c index 6b01aa641503..306640c72b7d 100644 --- a/mm/damon.c +++ b/mm/damon.c @@ -44,6 +44,9 @@ #define damon_for_each_task_safe(ctx, t, next) \ list_for_each_entry_safe(t, next, &(ctx)->tasks_list, list) +#define MAX_RECORD_BUFFER_LEN (4 * 1024 * 1024) +#define MAX_RFILE_PATH_LEN 256 + /* Get a random number in [l, r) */ #define damon_rand(l, r) (l + prandom_u32() % (r - l)) @@ -565,16 +568,80 @@ static bool kdamond_aggregate_interval_passed(struct damon_ctx *ctx) } /* - * Reset the aggregated monitoring results + * Flush the content in the result buffer to the result file + */ +static void damon_flush_rbuffer(struct damon_ctx *ctx) +{ + ssize_t sz; + loff_t pos = 0; + struct file *rfile; + + rfile = filp_open(ctx->rfile_path, O_CREAT | O_RDWR | O_APPEND, 0644); + if (IS_ERR(rfile)) { + pr_err("Cannot open the result file %s\n", + ctx->rfile_path); + return; + } + + while (ctx->rbuf_offset) { + sz = kernel_write(rfile, ctx->rbuf, ctx->rbuf_offset, &pos); + if (sz < 0) + break; + ctx->rbuf_offset -= sz; + } + filp_close(rfile, NULL); +} + +/* + * Write a data into the result buffer + */ +static void damon_write_rbuf(struct damon_ctx *ctx, void *data, ssize_t size) +{ + if (!ctx->rbuf_len || !ctx->rbuf) + return; + if (ctx->rbuf_offset + size > ctx->rbuf_len) + damon_flush_rbuffer(ctx); + + memcpy(&ctx->rbuf[ctx->rbuf_offset], data, size); + ctx->rbuf_offset += size; +} + +/* + * Flush the aggregated monitoring results to the result buffer + * + * Stores current tracking results to the result buffer and reset 'nr_accesses' + * of each region. The format for the result buffer is as below: + * + *