From patchwork Fri Feb 7 21:20:32 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 13965854 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DA45CC0219C for ; Fri, 7 Feb 2025 21:20:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 568086B008C; Fri, 7 Feb 2025 16:20:43 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 516D6280001; Fri, 7 Feb 2025 16:20:43 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 391976B0095; Fri, 7 Feb 2025 16:20:43 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 199636B008C for ; Fri, 7 Feb 2025 16:20:43 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id C1450B1E7B for ; Fri, 7 Feb 2025 21:20:42 +0000 (UTC) X-FDA: 83094417924.29.4C8443D Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf03.hostedemail.com (Postfix) with ESMTP id 1999220005 for ; Fri, 7 Feb 2025 21:20:40 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=dj4E40C8; spf=pass (imf03.hostedemail.com: domain of sj@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=sj@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738963241; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=q9XlllbjXy8F+/AItPTaZmJdn6pKv5Qs7v7aOPgcjSU=; b=KgA9hTabc9D7rBFNf0fz1uPyMXHOSPmFfrq1JQBzkiHRRqowczzd/VMphCh7YzvYp5nA2X mvyM9HffTYpX04onnhD7rqXHekPBaK7+0X1ZSkNMtkafEFtETeLrbNUcRuOtwP99OjTrml ZhZ9WQwjh7oQdKJzPyNeg97hF+nL67Y= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=dj4E40C8; spf=pass (imf03.hostedemail.com: domain of sj@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=sj@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738963241; a=rsa-sha256; cv=none; b=adQZhXviTNoTm6AUJQ8kf6Fv1GDkE5+n3eOhLswKtt3Su6RsN8OeiJR/YmYy9fUe2AwdlA f7MX5+W/b42asMZiVUDfDNMiN0Qrw5xGBUlZwQESLMmvRmOoXoZfMrPF7vmH3L0ST4wh6u +njAvWVZGlwlqw41OHtOW5SmYPfz3+0= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 8A9755C71CF; Fri, 7 Feb 2025 21:20:00 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B6096C4CED1; Fri, 7 Feb 2025 21:20:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1738963239; bh=5eAb6WCJyd/QZQWOfoLnHQEwZBeSQIp9J24NNYIAJsw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dj4E40C8q0gg3ud86pNgaTm+kbxa71t2XiAGOaMKWerPD4HMKWLzhpR5OcEi1pieM LvSdpdKIOfzfIIiMB/j+G1OcGL+j1k8FEHWrPwhzps3WSQFDeRJ+JRJ3B8ONpNpN2r COTpEwjtAWtj3ET7sA+YfPeQCx4sn7CgXJocQF8NDwo8JB8WGVFcHrHlIfGwg6mGz+ petGKMi5vKt0jKi1NkdjE28Y+2+ohBfFaPRSZGEXe/z937Ej1VA4CvZnOz3JjyAx/c mIi6D+X/jvCNwTiZ/5EqbX7eYmSEquKprB8vy9IVks0wck4efGgRy3FlY6tG03NCs9 jCa5cmMHDpviw== From: SeongJae Park To: Andrew Morton Cc: Usama Arif , SeongJae Park , damon@lists.linux.dev, kernel-team@meta.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, stable@vger.kernel.org Subject: [PATCH 1/2] mm/damon/ops: have damon_get_folio return folio even for tail pages Date: Fri, 7 Feb 2025 13:20:32 -0800 Message-Id: <20250207212033.45269-2-sj@kernel.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250207212033.45269-1-sj@kernel.org> References: <20250207212033.45269-1-sj@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 1999220005 X-Stat-Signature: jet3d5rr7ctzexpajjo3w9qrzxinoiy8 X-Rspam-User: X-Rspamd-Server: rspam01 X-HE-Tag: 1738963240-137795 X-HE-Meta: U2FsdGVkX18R5/EEDj1Pno8Ye8dSwdakyNI70mkthY+6EpNYWWrQyRQV8HOVPH1MSNe2PTIdoJt1dQBuZYQVXe8Tw4Rwz6Z6F4SHFPYLmUUUE1G5iyr8LmzdoZSthy59su/4ADogQE9atUR+xDrQi90gQdpSmsvaAUwgRUchDRjKMCL7O+qUp6yADYQKZJwtTMBvHuA6vIKICHz1SENHjffm4Gq2ONF4A/PvRecwAlyYNQVj3qRXXBS0TJnav6erA7pDqPoRUDl27/W9fjLk/miJqaJluFi//1Syme6V2DT0i0EuuoL+CG2ACwBS3xAgmr4Po3n1yH9vYYZsEk+F2SGH332ttn9rGgd9Ppq2FP6mxaWNR/NM0O8NMQDkew0DlJVLjtapzgKSBy3Nlo59uwDHIhbVu9RO5ltl9u/eu6Haz9PXlz8SoO7oMSz2kcdE9Jle1R0iVZhxyLxD5z1r/bCT3wteKACS8Kn0i951S/dzbzfQMOwAc+0vs3DEmFEwKMTrFJBTRpqRS0y8OgBDN7dEnvgy8kcLNLENmKLOAN5IlOhcoyB87TOqbN/xkmHiSe/LuGBjjujgJqEqT0XrbAXWG1k7uSMnlF6/4QRxZPl82lXPDVXdqRhgvbg0uFVyf6rpUjvxZGPWypysP9YxoMeBHoAr+HwxdYbwXDel8R45IArIh0XVdyppPC+zkXHQf1daqxPabS+N1r7Tbb0RL79Q3uzc7//zdFzt71FK/Anm8YirysDsKzcPtdAdWq34wV0vL3dNMUGaw8AOO9AthEZy0yIdcWfrWqjNOhUvc8NyLgiG1cJwxhxsGy/OFGbTqFDcJBmkCL96QA+/pAUqOagfD9BfkVJ7iyZRONHxYxhmkhQVSO8sQDLIsPwcaB/91R9F7nqi414JKLeprXSevInH+hg11r1tKjV862yPf5MN1/FiJzSMW+/IIN5e39ZB4fsmtOb2LPZy6eY6MqY 9JzmsmdD 83mnLnYD+9MRAz+LXc55SzGOQmOo21bxGz8qD+Vw5fIfKZZhDk48/M8YKJAqELwQ515hHkfzJhTK8YwPdujd9H1xL/TnqWdtHkHNZyhYHuIrVV+gH8BZXdOsT5tPQl6jMvjnVr61zmi/RJL28J8e3WsD4kbWC7Qm3nUWFTt8yIG84wtsS7skXmYtgjlT+My5acNflWc2PTSHZsMXt3XybobMxTQC9C53ng1RysHt53xmWMoUHGah/H8jueoOF/dGzSf4miB5MKN4/k8qpO39vmnlfMhV6nAPU/dPJSANNHT+NFTs= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Usama Arif This effectively adds support for large folios in damon for paddr, as damon_pa_mkold/young won't get a null folio from this function and won't ignore it, hence access will be checked and reported. This also means that larger folios will be considered for different DAMOS actions like pageout, prioritization and migration. As these DAMOS actions will consider larger folios, iterate through the region at folio_size and not PAGE_SIZE intervals. This should not have an affect on vaddr, as damon_young_pmd_entry considers pmd entries. Fixes: a28397beb55b ("mm/damon: implement primitives for physical address space monitoring") Cc: Signed-off-by: Usama Arif Reviewed-by: SeongJae Park Signed-off-by: SeongJae Park --- mm/damon/ops-common.c | 2 +- mm/damon/paddr.c | 24 ++++++++++++++++++------ 2 files changed, 19 insertions(+), 7 deletions(-) diff --git a/mm/damon/ops-common.c b/mm/damon/ops-common.c index d25d99cb5f2b..d511be201c4c 100644 --- a/mm/damon/ops-common.c +++ b/mm/damon/ops-common.c @@ -24,7 +24,7 @@ struct folio *damon_get_folio(unsigned long pfn) struct page *page = pfn_to_online_page(pfn); struct folio *folio; - if (!page || PageTail(page)) + if (!page) return NULL; folio = page_folio(page); diff --git a/mm/damon/paddr.c b/mm/damon/paddr.c index 0f9ae14f884d..0fb61f6ddb8d 100644 --- a/mm/damon/paddr.c +++ b/mm/damon/paddr.c @@ -266,11 +266,14 @@ static unsigned long damon_pa_pageout(struct damon_region *r, struct damos *s, damos_add_filter(s, filter); } - for (addr = r->ar.start; addr < r->ar.end; addr += PAGE_SIZE) { + addr = r->ar.start; + while (addr < r->ar.end) { struct folio *folio = damon_get_folio(PHYS_PFN(addr)); - if (!folio) + if (!folio) { + addr += PAGE_SIZE; continue; + } if (damos_pa_filter_out(s, folio)) goto put_folio; @@ -286,6 +289,7 @@ static unsigned long damon_pa_pageout(struct damon_region *r, struct damos *s, else list_add(&folio->lru, &folio_list); put_folio: + addr += folio_size(folio); folio_put(folio); } if (install_young_filter) @@ -301,11 +305,14 @@ static inline unsigned long damon_pa_mark_accessed_or_deactivate( { unsigned long addr, applied = 0; - for (addr = r->ar.start; addr < r->ar.end; addr += PAGE_SIZE) { + addr = r->ar.start; + while (addr < r->ar.end) { struct folio *folio = damon_get_folio(PHYS_PFN(addr)); - if (!folio) + if (!folio) { + addr += PAGE_SIZE; continue; + } if (damos_pa_filter_out(s, folio)) goto put_folio; @@ -318,6 +325,7 @@ static inline unsigned long damon_pa_mark_accessed_or_deactivate( folio_deactivate(folio); applied += folio_nr_pages(folio); put_folio: + addr += folio_size(folio); folio_put(folio); } return applied * PAGE_SIZE; @@ -464,11 +472,14 @@ static unsigned long damon_pa_migrate(struct damon_region *r, struct damos *s, unsigned long addr, applied; LIST_HEAD(folio_list); - for (addr = r->ar.start; addr < r->ar.end; addr += PAGE_SIZE) { + addr = r->ar.start; + while (addr < r->ar.end) { struct folio *folio = damon_get_folio(PHYS_PFN(addr)); - if (!folio) + if (!folio) { + addr += PAGE_SIZE; continue; + } if (damos_pa_filter_out(s, folio)) goto put_folio; @@ -479,6 +490,7 @@ static unsigned long damon_pa_migrate(struct damon_region *r, struct damos *s, goto put_folio; list_add(&folio->lru, &folio_list); put_folio: + addr += folio_size(folio); folio_put(folio); } applied = damon_pa_migrate_pages(&folio_list, s->target_nid); From patchwork Fri Feb 7 21:20:33 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 13965855 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AEDCCC02196 for ; Fri, 7 Feb 2025 21:20:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A98C5280004; Fri, 7 Feb 2025 16:20:44 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9D520280001; Fri, 7 Feb 2025 16:20:44 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 825C8280004; Fri, 7 Feb 2025 16:20:44 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 619D5280001 for ; Fri, 7 Feb 2025 16:20:44 -0500 (EST) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id BE09AA054F for ; Fri, 7 Feb 2025 21:20:43 +0000 (UTC) X-FDA: 83094417966.12.138630A Received: from nyc.source.kernel.org (nyc.source.kernel.org [147.75.193.91]) by imf17.hostedemail.com (Postfix) with ESMTP id 1C9934000F for ; Fri, 7 Feb 2025 21:20:41 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=fsdU3gjN; spf=pass (imf17.hostedemail.com: domain of sj@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=sj@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738963242; a=rsa-sha256; cv=none; b=cdZ3y5D76sEOrsB6EFsHN7n4VRGp8fDYCzXUWrVvkNMGCDyErfoZ6UpJUgu0EvlWm+8N/Z SWXRmdWuvd9o+aMn/OybTUOFCgSDoL6Dsg/mK2l9pxtKDimvGNKW78fLz7YbZtbPHGQsbB SYnZPEEi4qPZ+uHT+5lvnTxv1NqlU5A= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=fsdU3gjN; spf=pass (imf17.hostedemail.com: domain of sj@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=sj@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738963242; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=wVc7UjchyLnT12fk0LKrqIwV0YXlJpB5P2pQtFIRz2A=; b=xERCQyr27Aj5Et0kopjUxU3Ou57t/oCnRT/Ty8BV3QdijUM+jcC5akDRIGE4oFa8IqqaXR Q78ngLAGGkxRJIRabgzdKPoDWvPO2Jw7MfTCq5IWgSQmKJg0k+Mfzk68ppG+YSVBBn68HF EcqS+Q9DLs67EjrnoaasxUGeGzt/EaI= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id 7412DA43CA1; Fri, 7 Feb 2025 21:18:55 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E4B9DC4CEE4; Fri, 7 Feb 2025 21:20:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1738963241; bh=fyST1HHL6q047gB18uKbWH4029DtMjp2p7R3Opvgd7A=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fsdU3gjNMTuUnmQTkW1xAAT5TFFwXAoonivQvChRLfq+mJd3OPoHqQzNm5fBtIriI iTVfvestb/Xi8dhUqc8z7FMr/r1QkgUzUzSYpu19+ZmlsgvIEYsTExs65tzSFrKZl9 9bjYO2ZMXgaM0pmvOTHYHHVorxI6I5mowQSrhWPp8RacrQNqfWzQJOMmeZ4fpIObjv qKFPX9PYmbBPqAIWdJ0RU/QB4FJ3RYQAEngzGmiIjb3hfd7cljcRFbxhaUj79y2GJN X0U2soxDxFlnIXTIDyZ4KVpuivdhVUkYx8xtwE26faz03XyNC0Q14yKHywgwwbpbiH /HwVAArSqXyTg== From: SeongJae Park To: Andrew Morton Cc: SeongJae Park , Usama Arif , damon@lists.linux.dev, kernel-team@meta.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, stable@vger.kernel.org Subject: [PATCH 2/2] mm/damon: avoid applying DAMOS action to same entity multiple times Date: Fri, 7 Feb 2025 13:20:33 -0800 Message-Id: <20250207212033.45269-3-sj@kernel.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250207212033.45269-1-sj@kernel.org> References: <20250207212033.45269-1-sj@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 1C9934000F X-Stat-Signature: c6ifdxre63pjw34fuyj1ffy65fzx4jc5 X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1738963241-867809 X-HE-Meta: U2FsdGVkX199OiGNCpUk3Lcw2DPYf6hsnwe4WpMB16UNH3+MNbhdAtyhMk/04lJ+MeLtafw4UIazvUkyIDhSvfrRWAKlbxdTuUM7glX8OrqXE0mgsBw5OEtmXyAK4jrmXHvqNZozaNb3PX5K21CIkEHqq8A2w+5XOnls+jxRdUWa+JCS5w2rtIGisd0Z7+sYfuU9sgAsPViQYayBb46f32kRXMohEuaKg+XfxNY7MwFhBJ29+Ff2fsXJt454tYf+rFOgOdUb3u0PLKVgPTLgCPlUlLs7PcOndfYQKBp3YpuAUTVRh0f1MnpbNe6l2aELp90zXszGDmBPsXlAfzVlvDThOauaVo6sc2vennBz46v72cPql3MASOnfqQKspL/06jIdaDIW2GKEiryVzqvR1zvGQ4Bq7xXvmHw6FayVcJj14eeR1D92VddK9wvPbmB4tG1iocHJ/XTXlgJMzKLZS4K6nEemvrCyxJ/LuBZsdkA06vNqRxslXmv0mFwoV3ulrn8KqSSb6eBDF7xKdmMz1qbPuWvtLnv0rw8gkauSn/SXagFC8n96SS5tVz4tlCXC8njKxZl6okGDct3xx0eN4BbaA20ZSpIr2LjwmAGYhFT3eV0FuZIDPO1a4i+beaPhreLLEoGruSy5Fv/be2DiBJT4Wns7pUQGksp6YO/7OrH1KcdytYmpiZVnRoKUbPFAo/hql8x4l34PdH3D6vIPo5Z5aYiGl1U+EMc02ahhmv0GzAt1r0/joX6RSHaVIvDDrIY0TtsixhcF74cfzMTb7PPgYZbhNUUlrcj4Kzohx9Cp1EMBjPCdk2S97cb6exzhy+gZv+7x7ExRS5A7QCccAB8p4tl7iLN6XM2cRSD/kQUSemPqhvyvgeSgPMiUycoIiv6GBXsIZcc7MrTtwM8eSlxZAZGPkEsPdNUR6z8X+APlGarB3lC6l9Oq3Z+Ftc5+Pqvaf4jmHhTLweGXxpc 6XcetS9O LhyyQW9t5o5FaIycGUPeDdJ4sy3elj2tvx+jNweuRF5scKQOPAgovuvgaUqyeN260Gl8x3QosLU/d8bQaTZnqZjHwZ3V2bRfdO4B4sFqpVNhbFW8XDrU3Uv3+3xHx8S9f38MI1uFKTbCt9XdJ2tcfAPeIBnlLa349gYTkbfg4V0hdLX6iOtKb3lp7QNzvE6BiltFsuPpJDpXtJFlOonDfDHZexs49kT0/Mom88M2E4aZXQSn/+VL7jZHTDTVNQc9s3ezD X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: 'paddr' DAMON operations set can apply a DAMOS scheme's action to a large folio multiple times in single DAMOS-regions-walk if the folio is laid on multiple DAMON regions. Add a field for DAMOS scheme object that can be used by the underlying ops to know what was the last entity that the scheme's action has applied. The core layer unsets the field when each DAMOS-regions-walk is done for the given scheme. And update 'paddr' ops to use the infrastructure to avoid the problem. Fixes: 57223ac29584 ("mm/damon/paddr: support the pageout scheme") Cc: Reported-by: Usama Arif Closes: https://lore.kernel.org/20250203225604.44742-3-usamaarif642@gmail.com Signed-off-by: SeongJae Park --- include/linux/damon.h | 11 +++++++++++ mm/damon/core.c | 1 + mm/damon/paddr.c | 39 +++++++++++++++++++++++++++------------ 3 files changed, 39 insertions(+), 12 deletions(-) diff --git a/include/linux/damon.h b/include/linux/damon.h index af525252b853..a390af84cf0f 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -432,6 +432,7 @@ struct damos_access_pattern { * @wmarks: Watermarks for automated (in)activation of this scheme. * @target_nid: Destination node if @action is "migrate_{hot,cold}". * @filters: Additional set of &struct damos_filter for &action. + * @last_applied: Last @action applied ops-managing entity. * @stat: Statistics of this scheme. * @list: List head for siblings. * @@ -454,6 +455,15 @@ struct damos_access_pattern { * implementation could check pages of the region and skip &action to respect * &filters * + * The minimum entity that @action can be applied depends on the underlying + * &struct damon_operations. Since it may not be aligned with the core layer + * abstract, namely &struct damon_region, &struct damon_operations could apply + * @action to same entity multiple times. Large folios that underlying on + * multiple &struct damon region objects could be such examples. The &struct + * damon_operations can use @last_applied to avoid that. DAMOS core logic + * unsets @last_applied when each regions walking for applying the scheme is + * finished. + * * After applying the &action to each region, &stat_count and &stat_sz is * updated to reflect the number of regions and total size of regions that the * &action is applied. @@ -477,6 +487,7 @@ struct damos { int target_nid; }; struct list_head filters; + void *last_applied; struct damos_stat stat; struct list_head list; }; diff --git a/mm/damon/core.c b/mm/damon/core.c index c7b981308862..1a4dd644949b 100644 --- a/mm/damon/core.c +++ b/mm/damon/core.c @@ -1851,6 +1851,7 @@ static void kdamond_apply_schemes(struct damon_ctx *c) s->next_apply_sis = c->passed_sample_intervals + (s->apply_interval_us ? s->apply_interval_us : c->attrs.aggr_interval) / sample_interval; + s->last_applied = NULL; } } diff --git a/mm/damon/paddr.c b/mm/damon/paddr.c index 0fb61f6ddb8d..d64c6fe28667 100644 --- a/mm/damon/paddr.c +++ b/mm/damon/paddr.c @@ -243,6 +243,17 @@ static bool damos_pa_filter_out(struct damos *scheme, struct folio *folio) return false; } +static bool damon_pa_invalid_damos_folio(struct folio *folio, struct damos *s) +{ + if (!folio) + return true; + if (folio == s->last_applied) { + folio_put(folio); + return true; + } + return false; +} + static unsigned long damon_pa_pageout(struct damon_region *r, struct damos *s, unsigned long *sz_filter_passed) { @@ -250,6 +261,7 @@ static unsigned long damon_pa_pageout(struct damon_region *r, struct damos *s, LIST_HEAD(folio_list); bool install_young_filter = true; struct damos_filter *filter; + struct folio *folio; /* check access in page level again by default */ damos_for_each_filter(filter, s) { @@ -268,9 +280,8 @@ static unsigned long damon_pa_pageout(struct damon_region *r, struct damos *s, addr = r->ar.start; while (addr < r->ar.end) { - struct folio *folio = damon_get_folio(PHYS_PFN(addr)); - - if (!folio) { + folio = damon_get_folio(PHYS_PFN(addr)); + if (damon_pa_invalid_damos_folio(folio, s)) { addr += PAGE_SIZE; continue; } @@ -296,6 +307,7 @@ static unsigned long damon_pa_pageout(struct damon_region *r, struct damos *s, damos_destroy_filter(filter); applied = reclaim_pages(&folio_list); cond_resched(); + s->last_applied = folio; return applied * PAGE_SIZE; } @@ -304,12 +316,12 @@ static inline unsigned long damon_pa_mark_accessed_or_deactivate( unsigned long *sz_filter_passed) { unsigned long addr, applied = 0; + struct folio *folio; addr = r->ar.start; while (addr < r->ar.end) { - struct folio *folio = damon_get_folio(PHYS_PFN(addr)); - - if (!folio) { + folio = damon_get_folio(PHYS_PFN(addr)); + if (damon_pa_invalid_damos_folio(folio, s)) { addr += PAGE_SIZE; continue; } @@ -328,6 +340,7 @@ static inline unsigned long damon_pa_mark_accessed_or_deactivate( addr += folio_size(folio); folio_put(folio); } + s->last_applied = folio; return applied * PAGE_SIZE; } @@ -471,12 +484,12 @@ static unsigned long damon_pa_migrate(struct damon_region *r, struct damos *s, { unsigned long addr, applied; LIST_HEAD(folio_list); + struct folio *folio; addr = r->ar.start; while (addr < r->ar.end) { - struct folio *folio = damon_get_folio(PHYS_PFN(addr)); - - if (!folio) { + folio = damon_get_folio(PHYS_PFN(addr)); + if (damon_pa_invalid_damos_folio(folio, s)) { addr += PAGE_SIZE; continue; } @@ -495,6 +508,7 @@ static unsigned long damon_pa_migrate(struct damon_region *r, struct damos *s, } applied = damon_pa_migrate_pages(&folio_list, s->target_nid); cond_resched(); + s->last_applied = folio; return applied * PAGE_SIZE; } @@ -512,15 +526,15 @@ static unsigned long damon_pa_stat(struct damon_region *r, struct damos *s, { unsigned long addr; LIST_HEAD(folio_list); + struct folio *folio; if (!damon_pa_scheme_has_filter(s)) return 0; addr = r->ar.start; while (addr < r->ar.end) { - struct folio *folio = damon_get_folio(PHYS_PFN(addr)); - - if (!folio) { + folio = damon_get_folio(PHYS_PFN(addr)); + if (damon_pa_invalid_damos_folio(folio, s)) { addr += PAGE_SIZE; continue; } @@ -530,6 +544,7 @@ static unsigned long damon_pa_stat(struct damon_region *r, struct damos *s, addr += folio_size(folio); folio_put(folio); } + s->last_applied = folio; return 0; }