From patchwork Fri Sep 15 02:52:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 13386317 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 50187EDE989 for ; Fri, 15 Sep 2023 02:53:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8CE326B030F; Thu, 14 Sep 2023 22:53:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 888566B0312; Thu, 14 Sep 2023 22:53:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7044C6B0313; Thu, 14 Sep 2023 22:53:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 59F096B030F for ; Thu, 14 Sep 2023 22:53:14 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 345701CA1BC for ; Fri, 15 Sep 2023 02:53:14 +0000 (UTC) X-FDA: 81237310308.24.5A78434 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by imf09.hostedemail.com (Postfix) with ESMTP id 621D0140003 for ; Fri, 15 Sep 2023 02:53:12 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="gKRZh/eQ"; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf09.hostedemail.com: domain of sj@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=sj@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1694746392; a=rsa-sha256; cv=none; b=R8BzTlGIL/07gdZO8q7uRUL/hpwkGHQ68lpv7RdToqKD4d5EKCyAcKCGftcxDtnuhQx9ct T+WcYOATcav/tHh+TgGvByM5RQZSu48HbgzCJOAcq7vd3crWbVFZdeNuDbXpf4X2qhPeZk FsPCWiYRT8db+JOmggLAuPnu3cKuYcg= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="gKRZh/eQ"; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf09.hostedemail.com: domain of sj@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=sj@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1694746392; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=yas1krmMytP5jdv3DOjnLBNiWh5GBRJ7GxfCNOjPUeE=; b=ha1wGdckD3GHC5UBxLYheLcp5RvwpwcaIXiagdgJ7wsbDguJntRL+ru+IL5IRDu12k0WS2 9faC9aD/qwk9yRM+SMDkUKcfHc0exxc6lTpS/4LRRrywBQ14oC9XEA5kIzJRhkbUXTUzE2 rf5uwmAM07Z9w62BdKqmJg5UcARfewM= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id B6100B82B79; Fri, 15 Sep 2023 02:53:10 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 22D18C433CA; Fri, 15 Sep 2023 02:53:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1694746389; bh=Nt+EQkpN6g4pK9W8aFPRK0//Zoaw9gTVad64o5lt3eA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=gKRZh/eQUciGBq4zsaSk5NtoLQsupXcxMJ/a50/LGY+nOoGWuUjC6y2orIQp3obtI kXAhaTSvmkPcLcIBo83U43YcT5/rR5+67cCeNTXQlVjlTdinObZwE9rv8twhhAoFQ+ hfODNrii+3froco6CXwiwvl/To4vC87JSvSxXDv/eJWIQZELjCY1rxBMVpzzcvLU93 LKH7YRnIV2jtSEIA2qyPFoj0BJ7KiovZ49aLLrfHP/SjF8OE2hm2oY/XWCe7owdqgw /nEekEFdyI15efQdNOybauZ+1SdgCXF10FOBqA+Hs0IMEHvkEUrNqT1fUkNKZoBVdj COaVPLwQxu3zw== From: SeongJae Park To: Andrew Morton Cc: SeongJae Park , damon@lists.linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 6/8] mm/damon/core: use pseudo-moving sum for nr_accesses_bp Date: Fri, 15 Sep 2023 02:52:49 +0000 Message-Id: <20230915025251.72816-7-sj@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230915025251.72816-1-sj@kernel.org> References: <20230915025251.72816-1-sj@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 621D0140003 X-Stat-Signature: 76tix3yb1zzj3yy7a64bk889bfijsstm X-HE-Tag: 1694746392-923240 X-HE-Meta: U2FsdGVkX1813wIAYKmKDp+IZtOdhd2OSuZGd3N9x/eqhHFWtnf/yE8Cjuwyi4eU1A8KlDTdgop3kvTZkEYgqZ7W5ThgwpJF/aeWTafCOC9KJiFMRNB10cLmizL08hSx5Slybqe99cFvb6+tCFpJUh34QeEAG2nPWpZsGI0kTS24E7ChOEb1K0oh/80k8iOfLhBREY7cEdzqAvPjcN4EWJwrx/tguid5fjFb5NgyCAgYmP1JyfnSghjVJ4zyz8PAZHA8O+MwGjNwcW5oNK5Y0G4xSo56ye6orHaXiRO36FAI1z73LxXmpI7ZNv5ZU/gNJBXaU/Z7qQ0ZqEIjugdWID0Ngctkvy+e7IWt/V9P8EA9PpFHHl2i9gRAAAjVT1XAOMRrdedzI4xNZ3Uth04Tlt3K5G3D09TkdOUgFkRxOkJnk3D6itrp7lm4jAJztCMgO0oBWEiptXO64kao/6qHOj96z3Yf/dD0zK2dVwmyhKj0Q8MN4xhFu2kbuBOVHoZuZWH9arrWKsTMdUkqRYV7WNyphWrNG0phUntUIPjSc7k7CXTvy1zArjdsmRUpKB2p54xzt0Viz6cMmxBrmldhCp2Ul/9z/7tYD7QzVaj7KmzISQpztXdriY4XS3uQya4MEVPwcsRzwcrtTdmy2jRpsqQ1VfKP37bb2DZOpCDkHQrnaB+C04HrfIIznVJwLH6PuLbyoru8jmQYQu9RcALzsUhuxXDJZ7EX52RJFYRhqr9tdIsRTKwiJ2egy7Qca5tKzV3m+WOyx65pH+8uN9YEeOdLTHwFS0mzTnVK4RFZ1sopP2N4sgvc4Dt7N1bmnEEyzexpkOgMnf8d311g7Qe0RKE+4CUTICCyt/Ev4v/70SRJcTf2TZFiggAcqelVNFYoyZA3OiSDQuWNrXc+ujMDrD2IMmCHobGez9mBrjFVKmxw54Oe4YGqA1cD5VV7a/7azM6PwVRDipRfnJV2YPY WuyyQZc+ hsW5+oWd59dPVLXjD+hMZmSU8ZeV1Txki32oC+zh1yo1ntq4FMS5lhafTXdxDzFmmb6+Mvn+F6gSYk5WXBASL1O2oLxtiH1etaqiw2eA3q9DPvpbw+2ligPBTRdFX4Gz+PDbcVvDCJ1ROYruTtbTtsF7QhCDJayRaXYgN9mAYF4nSZYPc5NjTWUQgEd6ilkodoGgCKwuePQ2qebdc2ZzyxBBo21uxzKWqAI+exE7rEOTn8YFjZe8H/zZuoIdKz37+5j9WmEZxpnBcRVhE4S8ovZOXpu31oJepaV3ure/baEZbldA8V6Ra4buvBoVE/7uiQnPsTLwpMLUJDy7KrDien0EG8VCKCiz+j+lbRj/IHquR0cYf/haoDBKl50ZAfbm3nfVM X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Let nr_accesses_bp be calculated as a pseudo-moving sum that updated for every sampling interval, using damon_moving_sum(). This is assumed to be useful for cases that the aggregation interval is set quite huge, but the monivoting results need to be collected earlier than next aggregation interval is passed. Signed-off-by: SeongJae Park --- include/linux/damon.h | 12 +++++++++--- mm/damon/core.c | 16 +++++++++++++++- mm/damon/paddr.c | 9 +++++---- mm/damon/vaddr.c | 12 +++++++----- 4 files changed, 36 insertions(+), 13 deletions(-) diff --git a/include/linux/damon.h b/include/linux/damon.h index 15f24b23c9a0..0fe13482df63 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -40,7 +40,8 @@ struct damon_addr_range { * @ar: The address range of the region. * @sampling_addr: Address of the sample for the next access check. * @nr_accesses: Access frequency of this region. - * @nr_accesses_bp: @nr_accesses in basis point (0.01%). + * @nr_accesses_bp: @nr_accesses in basis point (0.01%) that updated for + * each sampling interval. * @list: List head for siblings. * @age: Age of this region. * @@ -51,7 +52,11 @@ struct damon_addr_range { * damon_update_region_access_rate(). * * @nr_accesses_bp is another representation of @nr_accesses in basis point - * (1 in 10,000) that updated every aggregation interval. + * (1 in 10,000) that updated for every &damon_attrs->sample_interval in a + * manner similar to moving sum. By the algorithm, this value becomes + * @nr_accesses * 10000 for every &struct damon_attrs->aggr_interval. This can + * be used when the aggregation interval is too huge and therefore cannot wait + * for it before getting the access monitoring results. * * @age is initially zero, increased for each aggregation interval, and reset * to zero again if the access frequency is significantly changed. If two @@ -629,7 +634,8 @@ int damon_set_regions(struct damon_target *t, struct damon_addr_range *ranges, unsigned int nr_ranges); unsigned int damon_moving_sum(unsigned int mvsum, unsigned int nomvsum, unsigned int len_window, unsigned int new_value); -void damon_update_region_access_rate(struct damon_region *r, bool accessed); +void damon_update_region_access_rate(struct damon_region *r, bool accessed, + struct damon_attrs *attrs); struct damos_filter *damos_new_filter(enum damos_filter_type type, bool matching); diff --git a/mm/damon/core.c b/mm/damon/core.c index ce85c00b0a4c..29ee1fc18393 100644 --- a/mm/damon/core.c +++ b/mm/damon/core.c @@ -1599,14 +1599,28 @@ unsigned int damon_moving_sum(unsigned int mvsum, unsigned int nomvsum, * damon_update_region_access_rate() - Update the access rate of a region. * @r: The DAMON region to update for its access check result. * @accessed: Whether the region has accessed during last sampling interval. + * @attrs: The damon_attrs of the DAMON context. * * Update the access rate of a region with the region's last sampling interval * access check result. * * Usually this will be called by &damon_operations->check_accesses callback. */ -void damon_update_region_access_rate(struct damon_region *r, bool accessed) +void damon_update_region_access_rate(struct damon_region *r, bool accessed, + struct damon_attrs *attrs) { + unsigned int len_window = 1; + + /* + * sample_interval can be zero, but cannot be larger than + * aggr_interval, owing to validation of damon_set_attrs(). + */ + if (attrs->sample_interval) + len_window = attrs->aggr_interval / attrs->sample_interval; + r->nr_accesses_bp = damon_moving_sum(r->nr_accesses_bp, + r->last_nr_accesses * 10000, len_window, + accessed ? 10000 : 0); + if (accessed) r->nr_accesses++; } diff --git a/mm/damon/paddr.c b/mm/damon/paddr.c index 44f21860b555..081e2a325778 100644 --- a/mm/damon/paddr.c +++ b/mm/damon/paddr.c @@ -148,7 +148,8 @@ static bool damon_pa_young(unsigned long paddr, unsigned long *folio_sz) return accessed; } -static void __damon_pa_check_access(struct damon_region *r) +static void __damon_pa_check_access(struct damon_region *r, + struct damon_attrs *attrs) { static unsigned long last_addr; static unsigned long last_folio_sz = PAGE_SIZE; @@ -157,12 +158,12 @@ static void __damon_pa_check_access(struct damon_region *r) /* If the region is in the last checked page, reuse the result */ if (ALIGN_DOWN(last_addr, last_folio_sz) == ALIGN_DOWN(r->sampling_addr, last_folio_sz)) { - damon_update_region_access_rate(r, last_accessed); + damon_update_region_access_rate(r, last_accessed, attrs); return; } last_accessed = damon_pa_young(r->sampling_addr, &last_folio_sz); - damon_update_region_access_rate(r, last_accessed); + damon_update_region_access_rate(r, last_accessed, attrs); last_addr = r->sampling_addr; } @@ -175,7 +176,7 @@ static unsigned int damon_pa_check_accesses(struct damon_ctx *ctx) damon_for_each_target(t, ctx) { damon_for_each_region(r, t) { - __damon_pa_check_access(r); + __damon_pa_check_access(r, &ctx->attrs); max_nr_accesses = max(r->nr_accesses, max_nr_accesses); } } diff --git a/mm/damon/vaddr.c b/mm/damon/vaddr.c index e36303271f9d..af2cb82e1fad 100644 --- a/mm/damon/vaddr.c +++ b/mm/damon/vaddr.c @@ -557,26 +557,27 @@ static bool damon_va_young(struct mm_struct *mm, unsigned long addr, * r the region to be checked */ static void __damon_va_check_access(struct mm_struct *mm, - struct damon_region *r, bool same_target) + struct damon_region *r, bool same_target, + struct damon_attrs *attrs) { static unsigned long last_addr; static unsigned long last_folio_sz = PAGE_SIZE; static bool last_accessed; if (!mm) { - damon_update_region_access_rate(r, false); + damon_update_region_access_rate(r, false, attrs); return; } /* If the region is in the last checked page, reuse the result */ if (same_target && (ALIGN_DOWN(last_addr, last_folio_sz) == ALIGN_DOWN(r->sampling_addr, last_folio_sz))) { - damon_update_region_access_rate(r, last_accessed); + damon_update_region_access_rate(r, last_accessed, attrs); return; } last_accessed = damon_va_young(mm, r->sampling_addr, &last_folio_sz); - damon_update_region_access_rate(r, last_accessed); + damon_update_region_access_rate(r, last_accessed, attrs); last_addr = r->sampling_addr; } @@ -593,7 +594,8 @@ static unsigned int damon_va_check_accesses(struct damon_ctx *ctx) mm = damon_get_mm(t); same_target = false; damon_for_each_region(r, t) { - __damon_va_check_access(mm, r, same_target); + __damon_va_check_access(mm, r, same_target, + &ctx->attrs); max_nr_accesses = max(r->nr_accesses, max_nr_accesses); same_target = true; }