From patchwork Fri May 13 14:59:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 12848949 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7FD1BC433F5 for ; Fri, 13 May 2022 15:00:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 11E976B0075; Fri, 13 May 2022 11:00:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 078466B0078; Fri, 13 May 2022 11:00:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E60868D0001; Fri, 13 May 2022 11:00:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id D84DA6B0075 for ; Fri, 13 May 2022 11:00:37 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay11.hostedemail.com (Postfix) with ESMTP id 9C10880D68 for ; Fri, 13 May 2022 15:00:37 +0000 (UTC) X-FDA: 79461031314.19.2D2278B Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by imf28.hostedemail.com (Postfix) with ESMTP id EC215C00AF for ; Fri, 13 May 2022 15:00:14 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 27C33B8306C; Fri, 13 May 2022 15:00:35 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5AB7FC34100; Fri, 13 May 2022 15:00:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1652454033; bh=11GGetY1wDUykE5Nyy58CKhxn3If/B0NbPr4dqjZJhc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=JVD2EDqRC51IxA65gs5McXHnMkTqUbN+hLAkSkMsvfpKfy0uENX8P6re5sXBCv5Vn oFpMKegP2kxpOaZ5b++DfDafD2GzJhb0OA0WK/T5gno8MBmL0FUtleA9BeuRWornnB P7yHSCGIkFU7Nzxbu4i62IKOymu7BaVxzOi+y6WXDToTPtLqtiBLWBQsA09K+LAXzK sfFXKRNfF0MUyjurphbZZ5UfQjInipxl2BKx9BSf2KS6HmRE6bXzrwMUZbgF3fC4cQ KODfYyVmTFB1O9V3T0oVTjzkZwvSY6HmR69GzF3ffAbeL3tA3l+8+6hXf644IStnHY fsw74HIIbXkjg== From: SeongJae Park To: Cc: linux-damon@amazon.com, damon@lists.linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, SeongJae Park Subject: [RFC PATCH 1/3] mm/damon/paddr: move DAMOS_PAGEOUT handling to a separate function Date: Fri, 13 May 2022 16:59:58 +0200 Message-Id: <20220513150000.25797-2-sj@kernel.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20220513150000.25797-1-sj@kernel.org> References: <20220513150000.25797-1-sj@kernel.org> X-Rspamd-Queue-Id: EC215C00AF X-Stat-Signature: tracpo6f34rcnmpr1amwxk5nm4hf9333 X-Rspam-User: Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=JVD2EDqR; spf=pass (imf28.hostedemail.com: domain of sj@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=sj@kernel.org; dmarc=pass (policy=none) header.from=kernel.org X-Rspamd-Server: rspam09 X-HE-Tag: 1652454014-488570 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This commit moves code for DAMOS_PAGEOUT handling to a separate function other than damon_pa_apply_scheme() to make damon_pa_apply_scheme() prepared for later additional DAMOS actions support. Signed-off-by: SeongJae Park --- mm/damon/paddr.c | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-) diff --git a/mm/damon/paddr.c b/mm/damon/paddr.c index 21474ae63bc7..08deee12ebfd 100644 --- a/mm/damon/paddr.c +++ b/mm/damon/paddr.c @@ -204,16 +204,11 @@ static unsigned int damon_pa_check_accesses(struct damon_ctx *ctx) return max_nr_accesses; } -static unsigned long damon_pa_apply_scheme(struct damon_ctx *ctx, - struct damon_target *t, struct damon_region *r, - struct damos *scheme) +static unsigned long damon_pa_pageout(struct damon_region *r) { unsigned long addr, applied; LIST_HEAD(page_list); - if (scheme->action != DAMOS_PAGEOUT) - return 0; - for (addr = r->ar.start; addr < r->ar.end; addr += PAGE_SIZE) { struct page *page = damon_get_page(PHYS_PFN(addr)); @@ -238,6 +233,19 @@ static unsigned long damon_pa_apply_scheme(struct damon_ctx *ctx, return applied * PAGE_SIZE; } +static unsigned long damon_pa_apply_scheme(struct damon_ctx *ctx, + struct damon_target *t, struct damon_region *r, + struct damos *scheme) +{ + switch (scheme->action) { + case DAMOS_PAGEOUT: + return damon_pa_pageout(r); + default: + break; + } + return 0; +} + static int damon_pa_scheme_score(struct damon_ctx *context, struct damon_target *t, struct damon_region *r, struct damos *scheme) From patchwork Fri May 13 14:59:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 12848950 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C02A9C433F5 for ; Fri, 13 May 2022 15:00:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 589B76B0078; Fri, 13 May 2022 11:00:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 513098D0002; Fri, 13 May 2022 11:00:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 38B678D0001; Fri, 13 May 2022 11:00:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 22BE56B0078 for ; Fri, 13 May 2022 11:00:40 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id DDB3461847 for ; Fri, 13 May 2022 15:00:39 +0000 (UTC) X-FDA: 79461031398.10.ECC794A Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by imf14.hostedemail.com (Postfix) with ESMTP id C48D51000AE for ; Fri, 13 May 2022 15:00:36 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id B21FDB83071; Fri, 13 May 2022 15:00:37 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0AF7FC34115; Fri, 13 May 2022 15:00:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1652454036; bh=w5mI+m7YFoLXGyrTD97A7aVQJcALsUpoPsjo1Fivhrw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=VZ0VSFfIONCVA1y76eAkXqiBNav2M0U1RT1pz5UliiH0R2Rwhs0OS5Atku3oa5iZe Kuigcj0Nak335/fGN54i8YYqfrQKetOnbSTLTCm9af0C8AVjaIMYsmGERw64g2mM2A V3crrrYRG5g3lCSR7csJrZiuYRfegWS0I6n3GAo+k02usfM21Bdj5kGbx96QYlsIlw LSsvoPfvbdb4TniPKLeZ1ezfqxI/+9F6qdSjCFGTaP8rZ3uI7NUiv05zbnIWTpMT62 luJDOH9SAZQCRzxD5S3DLuhNqnR7ty6E4IfhrWgCvUEcHYaSsDoyTwRL6ayz9Y3Alq ageyVvA548Zuw== From: SeongJae Park To: Cc: linux-damon@amazon.com, damon@lists.linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, SeongJae Park Subject: [RFC PATCH 2/3] mm/damon/schemes: Support 'hot' action Date: Fri, 13 May 2022 16:59:59 +0200 Message-Id: <20220513150000.25797-3-sj@kernel.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20220513150000.25797-1-sj@kernel.org> References: <20220513150000.25797-1-sj@kernel.org> X-Stat-Signature: psoub6tggaomew11o9gg8octhph9zra1 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: C48D51000AE Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=VZ0VSFfI; spf=pass (imf14.hostedemail.com: domain of sj@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=sj@kernel.org; dmarc=pass (policy=none) header.from=kernel.org X-Rspam-User: X-HE-Tag: 1652454036-947111 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This commit adds another DAMOS action called 'hot'. The action marks pages in the memory area of the taregt access pattern to be marked as accessed. Specifically, calls 'mark_page_accessed()'. This is supposed to be used for memory regions having frequent access so that hot pages could be more protected under memory pressure. Signed-off-by: SeongJae Park --- include/linux/damon.h | 2 ++ mm/damon/ops-common.c | 42 ++++++++++++++++++++++++++++++++++++++++++ mm/damon/ops-common.h | 2 ++ mm/damon/paddr.c | 20 ++++++++++++++++++++ mm/damon/sysfs.c | 1 + 5 files changed, 67 insertions(+) diff --git a/include/linux/damon.h b/include/linux/damon.h index 7c62da31ce4b..ed5338c3133d 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -87,6 +87,7 @@ struct damon_target { * @DAMOS_HUGEPAGE: Call ``madvise()`` for the region with MADV_HUGEPAGE. * @DAMOS_NOHUGEPAGE: Call ``madvise()`` for the region with MADV_NOHUGEPAGE. * @DAMOS_STAT: Do nothing but count the stat. + * @DAMOS_HOT: Mark the region as hot. * @NR_DAMOS_ACTIONS: Total number of DAMOS actions */ enum damos_action { @@ -96,6 +97,7 @@ enum damos_action { DAMOS_HUGEPAGE, DAMOS_NOHUGEPAGE, DAMOS_STAT, /* Do nothing but only record the stat */ + DAMOS_HOT, NR_DAMOS_ACTIONS, }; diff --git a/mm/damon/ops-common.c b/mm/damon/ops-common.c index e346cc10d143..fe9288025bae 100644 --- a/mm/damon/ops-common.c +++ b/mm/damon/ops-common.c @@ -131,3 +131,45 @@ int damon_pageout_score(struct damon_ctx *c, struct damon_region *r, /* Return coldness of the region */ return DAMOS_MAX_SCORE - hotness; } + +int damon_hot_score(struct damon_ctx *c, struct damon_region *r, + struct damos *s) +{ + unsigned int max_nr_accesses; + int freq_subscore; + unsigned int age_in_sec; + int age_in_log, age_subscore; + unsigned int freq_weight = s->quota.weight_nr_accesses; + unsigned int age_weight = s->quota.weight_age; + int hotness; + + max_nr_accesses = c->aggr_interval / c->sample_interval; + freq_subscore = r->nr_accesses * DAMON_MAX_SUBSCORE / max_nr_accesses; + + age_in_sec = (unsigned long)r->age * c->aggr_interval / 1000000; + for (age_in_log = 0; age_in_log < DAMON_MAX_AGE_IN_LOG && age_in_sec; + age_in_log++, age_in_sec >>= 1) + ; + + /* If frequency is 0, higher age means it's colder */ + if (freq_subscore == 0) + age_in_log *= -1; + + /* + * Now age_in_log is in [-DAMON_MAX_AGE_IN_LOG, DAMON_MAX_AGE_IN_LOG]. + * Scale it to be in [0, 100] and set it as age subscore. + */ + age_in_log += DAMON_MAX_AGE_IN_LOG; + age_subscore = age_in_log * DAMON_MAX_SUBSCORE / + DAMON_MAX_AGE_IN_LOG / 2; + + hotness = (freq_weight * freq_subscore + age_weight * age_subscore); + if (freq_weight + age_weight) + hotness /= freq_weight + age_weight; + /* + * Transform it to fit in [0, DAMOS_MAX_SCORE] + */ + hotness = hotness * DAMOS_MAX_SCORE / DAMON_MAX_SUBSCORE; + + return hotness; +} diff --git a/mm/damon/ops-common.h b/mm/damon/ops-common.h index e790cb5f8fe0..52329ff361cd 100644 --- a/mm/damon/ops-common.h +++ b/mm/damon/ops-common.h @@ -14,3 +14,5 @@ void damon_pmdp_mkold(pmd_t *pmd, struct mm_struct *mm, unsigned long addr); int damon_pageout_score(struct damon_ctx *c, struct damon_region *r, struct damos *s); +int damon_hot_score(struct damon_ctx *c, struct damon_region *r, + struct damos *s); diff --git a/mm/damon/paddr.c b/mm/damon/paddr.c index 08deee12ebfd..69980b922bf4 100644 --- a/mm/damon/paddr.c +++ b/mm/damon/paddr.c @@ -233,6 +233,22 @@ static unsigned long damon_pa_pageout(struct damon_region *r) return applied * PAGE_SIZE; } +static unsigned long damon_pa_mark_accessed(struct damon_region *r) +{ + unsigned long addr, applied = 0; + + for (addr = r->ar.start; addr < r->ar.end; addr += PAGE_SIZE) { + struct page *page = damon_get_page(PHYS_PFN(addr)); + + if (!page) + continue; + mark_page_accessed(page); + put_page(page); + applied++; + } + return applied * PAGE_SIZE; +} + static unsigned long damon_pa_apply_scheme(struct damon_ctx *ctx, struct damon_target *t, struct damon_region *r, struct damos *scheme) @@ -240,6 +256,8 @@ static unsigned long damon_pa_apply_scheme(struct damon_ctx *ctx, switch (scheme->action) { case DAMOS_PAGEOUT: return damon_pa_pageout(r); + case DAMOS_HOT: + return damon_pa_mark_accessed(r); default: break; } @@ -253,6 +271,8 @@ static int damon_pa_scheme_score(struct damon_ctx *context, switch (scheme->action) { case DAMOS_PAGEOUT: return damon_pageout_score(context, r, scheme); + case DAMOS_HOT: + return damon_hot_score(context, r, scheme); default: break; } diff --git a/mm/damon/sysfs.c b/mm/damon/sysfs.c index 09f9e8ca3d1f..a1de2278f8c2 100644 --- a/mm/damon/sysfs.c +++ b/mm/damon/sysfs.c @@ -763,6 +763,7 @@ static const char * const damon_sysfs_damos_action_strs[] = { "hugepage", "nohugepage", "stat", + "hot", }; static struct damon_sysfs_scheme *damon_sysfs_scheme_alloc( From patchwork Fri May 13 15:00:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 12848951 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 657F0C433EF for ; Fri, 13 May 2022 15:00:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B7F278D0001; Fri, 13 May 2022 11:00:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B05296B007D; Fri, 13 May 2022 11:00:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9CDC88D0001; Fri, 13 May 2022 11:00:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 8D2BC6B007B for ; Fri, 13 May 2022 11:00:42 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 6AA4B330BD for ; Fri, 13 May 2022 15:00:42 +0000 (UTC) X-FDA: 79461031524.10.6A77ABC Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by imf03.hostedemail.com (Postfix) with ESMTP id B7DA5200B8 for ; Fri, 13 May 2022 15:00:32 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 40202B8306C; Fri, 13 May 2022 15:00:40 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 85191C34100; Fri, 13 May 2022 15:00:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1652454038; bh=dvkE+Xle/CL236kYATrTSaFb/CzjqEqgY0TtOELuS3s=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=G8xkcMwft8MJgi5PphUHDXBPYWeRuVAR9cEtuPyM3CN3b3Stzq8L9ooP1Xmc8xPc6 5PPqqjAnCV1UyX2GcnApgnhwTq94CHyWAsTfFHUSlxGkzjNq8SIehWGhdObmVyCSJI dg6QFE3X8lSkpdgqFCieCzbpPGH2nJrpP7BhimaJKGUoigLKieCQkTGNCK9L0IuftZ 0jyXQ3FsdasiWkPdTUqNpq4juzPBY7aWZ5jS70hVOn+avgDIPnQcFFedlc6nNpy9DB EwVsolHZXS/3tLcnqYnIkyh4xwa6N1BtPOLsU1FUi0Y5CCQXGb86NqZ0pNdy4lawcB Rj5bHogHTx8aQ== From: SeongJae Park To: Cc: linux-damon@amazon.com, damon@lists.linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, SeongJae Park Subject: [RFC PATCH 3/3] mm/damon/paddr: Support DAMOS_COLD Date: Fri, 13 May 2022 17:00:00 +0200 Message-Id: <20220513150000.25797-4-sj@kernel.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20220513150000.25797-1-sj@kernel.org> References: <20220513150000.25797-1-sj@kernel.org> X-Stat-Signature: xqkk9kixeyotzjje5nybkz8jufx6rmny Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=G8xkcMwf; spf=pass (imf03.hostedemail.com: domain of sj@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=sj@kernel.org; dmarc=pass (policy=none) header.from=kernel.org X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: B7DA5200B8 X-HE-Tag: 1652454032-429492 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: DAMOS_COLD is currently supported by the virtual address spaces monitoring operations set (vaddr). This commit adds support of the action to the physical address space monitoring operations set (paddr). Using this together with hot DAMOS action, users can proactively sort LRU lists so that performance degradation under memory pressure can be reduced. Signed-off-by: SeongJae Park --- mm/damon/paddr.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/mm/damon/paddr.c b/mm/damon/paddr.c index 69980b922bf4..761b1580271c 100644 --- a/mm/damon/paddr.c +++ b/mm/damon/paddr.c @@ -249,6 +249,22 @@ static unsigned long damon_pa_mark_accessed(struct damon_region *r) return applied * PAGE_SIZE; } +static unsigned long damon_pa_cold(struct damon_region *r) +{ + unsigned long addr, applied = 0; + + for (addr = r->ar.start; addr < r->ar.end; addr += PAGE_SIZE) { + struct page *page = damon_get_page(PHYS_PFN(addr)); + + if (!page) + continue; + deactivate_page(page); + put_page(page); + applied++; + } + return applied * PAGE_SIZE; +} + static unsigned long damon_pa_apply_scheme(struct damon_ctx *ctx, struct damon_target *t, struct damon_region *r, struct damos *scheme) @@ -258,6 +274,8 @@ static unsigned long damon_pa_apply_scheme(struct damon_ctx *ctx, return damon_pa_pageout(r); case DAMOS_HOT: return damon_pa_mark_accessed(r); + case DAMOS_COLD: + return damon_pa_cold(r); default: break; } @@ -273,6 +291,8 @@ static int damon_pa_scheme_score(struct damon_ctx *context, return damon_pageout_score(context, r, scheme); case DAMOS_HOT: return damon_hot_score(context, r, scheme); + case DAMOS_COLD: + return damon_pageout_score(context, r, scheme); default: break; }