From patchwork Mon Mar 10 17:23:10 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 14010442 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 15DB1C282DE for ; Mon, 10 Mar 2025 17:23:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 64EF3280004; Mon, 10 Mar 2025 13:23:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5125C280017; Mon, 10 Mar 2025 13:23:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 22EA7280004; Mon, 10 Mar 2025 13:23:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id EDC7F28000B for ; Mon, 10 Mar 2025 13:23:27 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 241A1C130E for ; Mon, 10 Mar 2025 17:23:28 +0000 (UTC) X-FDA: 83206312896.11.8CC5022 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf18.hostedemail.com (Postfix) with ESMTP id 3E57A1C0021 for ; Mon, 10 Mar 2025 17:23:26 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=e1BK93rR; spf=pass (imf18.hostedemail.com: domain of sj@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=sj@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1741627406; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=v2BgPGEjQCF0GcVfOPHe7F1Yx41Rf6juYV4ECA5muQg=; b=YvzEsLI+0P+QZQe1/iRi4DSgxzvr5j5PXCfiXfDHrLoQds24J4N8j+evE4DqQrbXV7mpgz WpHoKMan9GGk/uJUo/AGZ4v49nO2KzL4/WjNERaf/rrGbnDvDVEsDmvs93QmXhiCuQEJWz KbD+xKZK4PuBVUEmMby0o23lmPqtaaw= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=e1BK93rR; spf=pass (imf18.hostedemail.com: domain of sj@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=sj@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1741627406; a=rsa-sha256; cv=none; b=qZ2o5JLjKR5Onhk/d75rAmE7v6AWlUlgyqQte5LOzkGcJs0xsIcLt87f0mH5gk05UXdiki FH5A33xGRPFVmDiI5M4XxDy3nxfiFg1Wkz8vJ6ZUGncMO5VaQc2iVzGktNrapRkHMi6D+F cSFM4z2rwt1yA5vL1+JLXi4LYlK0DcU= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id A4A655C53D2; Mon, 10 Mar 2025 17:21:08 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id EF349C4CEE5; Mon, 10 Mar 2025 17:23:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1741627405; bh=aKpudoyGMo+z+VXYP9y3gu4MvjeTt1j3Cq/sd3oN+kg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=e1BK93rRkUd3fN3j1E7lnnyaxNRQGTW5BEIjfLCMLxhVvoBz+4jSN85j3VHnm26J6 3ZruHED9gH/KJIR2LwXrwm83oumL2S0dnHxf56U2+HlbZZlVPURpjRX5+KJ0dzTjAY WVGyOAvhEvEh2nwRjdApfUatSmV5bdVGpbVs0cfxEwEuf6SyXlKjjW3gtiKAByCuy7 coXHx5hVZzpuPzyjBbvt+Mgxj6vZqqm30Kxl8DKnklVs7qfC/xDvmHJ6KzSBHe0x4z vtTHOMLgSkugJ/9YYnTS2e0AY38HDjWOgp4TvS5tfLyt9gE/ruVGDyrKVtAbKFX7RO GkthFUtl2sOFQ== From: SeongJae Park To: Andrew Morton Cc: SeongJae Park , "Liam R. Howlett" , David Hildenbrand , Lorenzo Stoakes , Shakeel Butt , Vlastimil Babka , kernel-team@meta.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH 1/9] mm/madvise: use is_memory_failure() from madvise_do_behavior() Date: Mon, 10 Mar 2025 10:23:10 -0700 Message-Id: <20250310172318.653630-2-sj@kernel.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250310172318.653630-1-sj@kernel.org> References: <20250310172318.653630-1-sj@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 3E57A1C0021 X-Stat-Signature: 1kwhqwkrqw6jkfdkc8cex7nkom54s56f X-Rspam-User: X-HE-Tag: 1741627406-486960 X-HE-Meta: U2FsdGVkX18uKXeW4BP3ewS4bpuTLab/A7eq28L4t3kq1aa41kSis7yzEX36louu/3AtMMdoyA8c4/5oOWXlHzJMLQ5FJzgr38geVuvdGH7ra+ZyFbDMygzQdhHOcEVUNMs6rFQxxKCcT02mz/8uBvgNCE+fSLthKnIaqE4hfiv+PA2C7lOWDl1qTL2I3hPRoS9KLMzkvXGnNSa5N2QUaKftFo1oqmiVNCmUmcixJ82V6EWPTfB0zt2vIQqQWiYodNBkRbcSuIF9OH4l9AR3uNRpunZLAqM4/N+ngrmipnr+H3fxb36vvG1fvNlw23q0kqhXF02aRLZqqxlfM5jLeiwvRnbSmO+sXSFmbu5eCKA3mP9uzwro0cKo33zdoL3WGlLYaipgBwuPRi5QNXO5lDrYGkiyVmEsPCToHR9rXjzfT48JDVSS5Dlr8Wcq2Nqoq5xiVRahYxGpDkj8Dv1wwaB0GpOqaNdAfv4DlbRoPy4Rq03ffHcpo1wQURjYy3dCiMksM63oda4w7fKLDHqrpkBxE3Xyxy+0+gacvG8W503J5QHx8hApetHgG+MnUE6o9q0+IoM9Tgy4OZn7MIc56LRB6T0ryXTEdRV2cUH+6q/irVazdDM+ttMrl7fK4BgjGCx1UyCYbJSDtHlidOA+lNufqp5PmzoYP33vuVEbTpKgt/S1wt/J7ODyDqM5YJUIeaqklgXb+4iiz+Xyho2fqQIDwPLNd/A7vkWcs9qCh9Fx89Hs91jVlNepYDYe9Q9VNvbkZSwhf0xxXxMJ+ee6CBvZBCcpDDNRsKLIBEosJn0LjSYddAoovt3G+vVZnrtSHI+uo+FCsPjVkg0VRvKK9vm1nDnvk+S4+JeE+v84Fk3S30FS11/mnVC1OWVtdOtVjPGhztCNajNtSE0/aC0Vyl2qjdpXY/BKiuiEZAJL+h8JuqaP6PPIRhwK67k2oDIHz/8laVGZl/SEJX/AslP 4mZJPT7B A7hjej/xSk2Kv/5t350M/SCJO/xw5uTKc4qzZKDSR5ax4WQADsIrSm/Pp8GWxlnoAg78gfWezaf4tB7GyI5NV+af57T0id/nEOP1yvBjL25wylx5yFQw8TymCCAqy2j32+d1ng41AQ0jvzaC4q0KunISmQksqvqt0t06V/sIqyemzEBUGix3H0JbxynNGLr0Nt0OTaZPA7g2pYxelKhM72Opp3e8d9Z5ELBJ531XDl6uIQXxon+f30flOabDA3pHCC46QEzUxN0hEH9KnOKgdqO7TWQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To reduce redundant open-coded checks of CONFIG_MEMORY_FAILURE and MADV_{HWPOISON,SOFT_OFFLINE} in madvise_[un]lock(), is_memory_failure() has introduced. madvise_do_behavior() is still doing the same open-coded check, though. Use is_memory_failure() instead. To avoid build failure on !CONFIG_MEMORY_FAILURE case, implement an empty madvise_inject_error() under the config. Also move the definition of is_memory_failure() inside #ifdef CONFIG_MEMORY_FAILURE clause for madvise_inject_error() definition, to reduce duplicated ifdef clauses. Signed-off-by: SeongJae Park Reviewed-by: Lorenzo Stoakes --- mm/madvise.c | 49 +++++++++++++++++++++++++++---------------------- 1 file changed, 27 insertions(+), 22 deletions(-) diff --git a/mm/madvise.c b/mm/madvise.c index 388dc289b5d1..c3ab1f283b18 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -1392,7 +1392,32 @@ static int madvise_inject_error(int behavior, return 0; } -#endif + +static bool is_memory_failure(int behavior) +{ + switch (behavior) { + case MADV_HWPOISON: + case MADV_SOFT_OFFLINE: + return true; + default: + return false; + } +} + +#else + +static int madvise_inject_error(int behavior, + unsigned long start, unsigned long end) +{ + return 0; +} + +static bool is_memory_failure(int behavior) +{ + return false; +} + +#endif /* CONFIG_MEMORY_FAILURE */ static bool madvise_behavior_valid(int behavior) @@ -1569,24 +1594,6 @@ int madvise_set_anon_name(struct mm_struct *mm, unsigned long start, } #endif /* CONFIG_ANON_VMA_NAME */ -#ifdef CONFIG_MEMORY_FAILURE -static bool is_memory_failure(int behavior) -{ - switch (behavior) { - case MADV_HWPOISON: - case MADV_SOFT_OFFLINE: - return true; - default: - return false; - } -} -#else -static bool is_memory_failure(int behavior) -{ - return false; -} -#endif - static int madvise_lock(struct mm_struct *mm, int behavior) { if (is_memory_failure(behavior)) @@ -1640,10 +1647,8 @@ static int madvise_do_behavior(struct mm_struct *mm, unsigned long end; int error; -#ifdef CONFIG_MEMORY_FAILURE - if (behavior == MADV_HWPOISON || behavior == MADV_SOFT_OFFLINE) + if (is_memory_failure(behavior)) return madvise_inject_error(behavior, start, start + len_in); -#endif start = untagged_addr_remote(mm, start); end = start + len; From patchwork Mon Mar 10 17:23:11 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 14010443 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7004DC282DE for ; Mon, 10 Mar 2025 17:23:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A8AFC280017; Mon, 10 Mar 2025 13:23:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A11E728000B; Mon, 10 Mar 2025 13:23:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 866D1280017; Mon, 10 Mar 2025 13:23:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 6A1D228000B for ; Mon, 10 Mar 2025 13:23:29 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id DE663C12E8 for ; Mon, 10 Mar 2025 17:23:29 +0000 (UTC) X-FDA: 83206312938.25.E424764 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf19.hostedemail.com (Postfix) with ESMTP id 872741A0005 for ; Mon, 10 Mar 2025 17:23:27 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=QLodKnMG; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf19.hostedemail.com: domain of sj@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=sj@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1741627407; a=rsa-sha256; cv=none; b=cxrJtN6aTwVrCO0LGgfbTXiPslSyRfn2EGnl+K2khRVKSCGGKOGn2O8TPunTowJToYmBds Js7vApn46aKLafY3EPO2TrIbMK5tYWtEdmmgwNaAPaQeOtADmVaVNunmTzpVI1cCUBBshs MR0JJE/LIEz0kiD/tSRHtsJcu7Bgmc8= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=QLodKnMG; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf19.hostedemail.com: domain of sj@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=sj@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1741627407; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=0tixeGrzJIcyGhtY0onfJYjZJKuBi5YtkfsORYDChEQ=; b=VIf784oPETnCQRldrzR3gLlezTSKAZKe7u2bU89kq93qfcIK6ZjhVethqhKnlqxN6umawG SaWKSKYBLRiDWdp8ZyeHR1DX6zU/fWyXLGLhygybNtA9osHj9FE6UTrT6jloLwtGHx4vdR XJAgdRcRo+KrYkWCxqa5YZ6mdE5ufoo= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id CC0D15C5438; Mon, 10 Mar 2025 17:21:09 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 15460C4CEE5; Mon, 10 Mar 2025 17:23:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1741627406; bh=M9/wAyNFr4MzyO45yCxu/y9IRXgOWJVU4DhwCYnIYO8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QLodKnMGunZymOsXhiHwJL3CZN56ph7GyHUBQSnqvbfDZM4FdddyE6UwjyywHRHS1 cQupnW2BYiokTpHo7gTlm1+TlGjayuwf2a+xu80NDLVwzT7QoRMat5a/pBCcLPviUG I2u/9J8exOAV+qOehvNS3kdd/WwNK1e2uTbFksC+1ZA2Dd+i1z2axiZf4RF4O72SYs rRuDT2HOdcp+kQiTqlDhnnFIhMVXzyLGCSCcIu6/KGDv+TdsZi4dur6hETHZOeiWVw q0K7MtgfIqjtRTafPny8uDtlEfc/dSMYN9Pd++aK1UcAwdsXxrPKtRSMlm5C0KuB18 Lwmc/eOHDhANA== From: SeongJae Park To: Andrew Morton Cc: SeongJae Park , "Liam R. Howlett" , David Hildenbrand , Lorenzo Stoakes , Shakeel Butt , Vlastimil Babka , kernel-team@meta.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH 2/9] mm/madvise: split out populate behavior check logic Date: Mon, 10 Mar 2025 10:23:11 -0700 Message-Id: <20250310172318.653630-3-sj@kernel.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250310172318.653630-1-sj@kernel.org> References: <20250310172318.653630-1-sj@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 872741A0005 X-Rspamd-Server: rspam11 X-Stat-Signature: 4ghjz96f1qyaqg34kexzcot4s4ajpog4 X-Rspam-User: X-HE-Tag: 1741627407-522445 X-HE-Meta: U2FsdGVkX1/RNGREavaApH+pXXIFrM+TFAYFmCYJA/ld2d4ONkpjOeKSNxp8Xu92vOeT43Vd2fdecqx7z9bL7EDk7O4Off3TycmNGCVlv1ZmVb9GiME2zLL68Y4hskQ3nqnqGHyZCNZd6sHkUios8K219yCK3oKJpd8ThMe2rL1AUgcNFA+8hxuAUiJV3H81YWZ/CHNouag/MDYVhOB3DyE4yY1yhraaHA0TPao5k7JAt2WRXCdoolmUQscEj0KfG6tv7hN6UwUJgLuuRhwhzPKlsX83fDTbcov0YgicDUigcRG7zsPaal9PBCIs1Q0xRbbeizOHRDIrj3tAYDHLouUt/uv5xp9VQvpWR4EzwJSEUecDxc4HLDeUJJtGxpsF1KQY/ziP+g8Hiuwc8eu4miwD3z67Vu3WXMviUKFrVwWF6B3t0Yh8mncb7JLzRX61sccw0T1VeGSXgPcaSD7BRqFXa/uejlbiAPdOyf6geNiNRHBks1YN1DtnDsnZXph1U09URcc6AIkNswBtJ9DiEwIfAyVErXIirABBV1gStDAqqrBDH8ht2l7LQqfuPHcDnubf+DMWSomWz9QqG1EgGj5syPMqmGkbV7bORILfkBcU++SUWzKxi1lGTgmknyfxe9VYEeKeyOCEQC6/z6kIZQEfRUmrR2Z+MHzxLXT5OoBE0WrQUmgOamcwopwrhkfBcWiRdRcb1z4oBcetOHRsgAIjJ1P8pmxlZK1Acw/7RYfKT+A+bVmbKT4e1833DH24xSf+BZbKlNWavgX96fGrFi4tVBWJZY2nZ5Xj7LoOx6lguI5+sAvdb+870gt0qFraFIoxKvM8ZB/CTqUTquu0slKkOZ4T3pajWwlJ3pvBeDU8iJimQ9ynvFKUfHe/AL+IDSYi0f1earieZvuiGMy5tkqmD6SLAx3fBUL37irsFddVKeqVZirHxZLIadPosdj9BcukRrJqQbHnikEvuVc h4odslYq xuzMQeFDjdnkynVDg5p4riRYWxDARTr8hPBmSA/bNl9yc63BWmms4H1MWqisZixMlwEWoADSmkKkA9+wArikxNitocb9GvGDny4o//zVhZHhD5A8xM4XebBBtTO1Bbtwiub1mCQywHkBDerwjs9EmAKYPAginmRO1lldSOLioeL8OOlbwqfPH5eVfyMpE01nmCV5abYkV0k7o8dRlWcMxkhujgIYtFQlI36nEXb/hBQip3Jg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: madvise_do_behavior() has a long open-coded 'behavior' check for MADV_POPULATE_{READ,WRITE}. It adds multiple layers[1] and make the code arguably take longer time to read. Like is_memory_failure(), split out the check to a separate function. This is not technically removing the additional layer but discourage further extending the switch-case. Also it makes madvise_do_behavior() code shorter and therefore easier to read. [1] https://lore.kernel.org/bd6d0bf1-c79e-46bd-a810-9791efb9ad73@lucifer.local Signed-off-by: SeongJae Park Reviewed-by: Lorenzo Stoakes --- mm/madvise.c | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/mm/madvise.c b/mm/madvise.c index c3ab1f283b18..611db868ae38 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -1640,6 +1640,17 @@ static bool is_valid_madvise(unsigned long start, size_t len_in, int behavior) return true; } +static bool is_madvise_populate(int behavior) +{ + switch (behavior) { + case MADV_POPULATE_READ: + case MADV_POPULATE_WRITE: + return true; + default: + return false; + } +} + static int madvise_do_behavior(struct mm_struct *mm, unsigned long start, size_t len_in, size_t len, int behavior) { @@ -1653,16 +1664,11 @@ static int madvise_do_behavior(struct mm_struct *mm, end = start + len; blk_start_plug(&plug); - switch (behavior) { - case MADV_POPULATE_READ: - case MADV_POPULATE_WRITE: + if (is_madvise_populate(behavior)) error = madvise_populate(mm, start, end, behavior); - break; - default: + else error = madvise_walk_vmas(mm, start, end, behavior, madvise_vma_behavior); - break; - } blk_finish_plug(&plug); return error; } From patchwork Mon Mar 10 17:23:12 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 14010446 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 706ECC282DE for ; Mon, 10 Mar 2025 17:23:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8C72828000B; Mon, 10 Mar 2025 13:23:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8046E280026; Mon, 10 Mar 2025 13:23:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6081D28000B; Mon, 10 Mar 2025 13:23:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 31A4D280026 for ; Mon, 10 Mar 2025 13:23:32 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 593D4C12F1 for ; Mon, 10 Mar 2025 17:23:30 +0000 (UTC) X-FDA: 83206312980.24.5899C8C Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf20.hostedemail.com (Postfix) with ESMTP id 8C0E71C000D for ; Mon, 10 Mar 2025 17:23:28 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=cfK2CND5; spf=pass (imf20.hostedemail.com: domain of sj@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=sj@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1741627408; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=pTtQhCKqvjeYCewdvDnPMtPDfLVqUorUj87AmPqHvZs=; b=WUyk6xLqVpDICpTHCvQEKEiiewsg81fzQUq1Q6C1ABgKGKOdGqHSk2IWcT2Wlujd0GSwQZ CQSiwUsDg32R1pOMF4l7qqTjGdGy3Zfu0S2Vc8daM287kcJnWVtAh9nV5e87o7QLyIAwWF u+6GCsHczcciwmEV2p0tQQfREEgUtdc= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1741627408; a=rsa-sha256; cv=none; b=0/1IIIpcJOSP9MbAWiOGpkgpDup7d9bvKwaOccwbq23GS6pHEsOzLgKM64d9Sboqp1Tqk7 i63z0f70fNdMtLVXVzpRut5GAF7KkRg/VqmUwd5b7q4gl0lA7F/YYZQBzz9khNr9wuTO+x PXDW6mPzA3M1CpvmNUta3sXExP2nG0o= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=cfK2CND5; spf=pass (imf20.hostedemail.com: domain of sj@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=sj@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id BC9B65C5405; Mon, 10 Mar 2025 17:21:10 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3427AC4CEEC; Mon, 10 Mar 2025 17:23:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1741627407; bh=wUeeTxoF6/W5BNmB2wyQPZvoKyORH7hO5pM78LtrSAM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cfK2CND5xWYz9nn8nUacYGS/fBI3F77IP5i/AnejOywNLUUScO7lFdcvtIqseomMj EeJnrxKN9JefLc0Rx5KoOLV0nimE3wn8gMM9f93f+CvKNBenCp2BXokS8fjdiwdmf3 R0qA0M9s5Fe9S+kBRKu5nS7Pzh4vE4wTRWzZe8RA28TE9ujg9vagLeypltyUywXpui DWQjz+kPfXe/Ghjo7OpB2glKV/S/A72l3FZVrrNFJR0QOlQKQAjp7Tz2V6EgPm4ql9 yASG4MgVpytpi5cvrfDIYPvKpJ55rstceXLqloJMGS4JTTgxOFrEeZzirMoDK+wrpc pvGTGCY2URD0g== From: SeongJae Park To: Andrew Morton Cc: SeongJae Park , "Liam R. Howlett" , David Hildenbrand , Lorenzo Stoakes , Shakeel Butt , Vlastimil Babka , kernel-team@meta.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH 3/9] mm/madvise: deduplicate madvise_do_behavior() skip case handlings Date: Mon, 10 Mar 2025 10:23:12 -0700 Message-Id: <20250310172318.653630-4-sj@kernel.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250310172318.653630-1-sj@kernel.org> References: <20250310172318.653630-1-sj@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspam-User: X-Rspamd-Queue-Id: 8C0E71C000D X-Stat-Signature: rgsdbxgh7j1fe43oimoyayqow81jxoan X-HE-Tag: 1741627408-517745 X-HE-Meta: U2FsdGVkX1+Tv6VbEOf740+vN7ZO3QKECv9NvYmEgE2LnnWeeyL4Cx/jikfIdjubOejukE2WkgG2HY1rG3OF4QL7kzizlo3uHCiJoelijOwprDMVfpVvI85wlVkqvm4/U/7XOqyUJDhBdU0YFSi75tvl4EklY5Rmuxsw9jlYnvrKSiJtCMRNLqlTjuTuKD90hpCzknFGBT+XttUK+pLOlsYPVWoxD0OxNRzcdcy5IoyqqC+mzlNAv9cilGq8sMezhw3f2nrfh7peB6EJFb4uQWosTFC7nWbdhlR/0BOU8Ex06V9SFI7o21IuCAcGMYS0Gwox4uY01tbLnNAS7gJURvPwlfqAjfUfLrJk715zudTqwfPa9o9S5J5MPt9xYlZpNoDBo7puZ2TSCc+4jaMl6TusAkLiABdqnlnCttYdS39nkEZuCeDj4Y8mxk7uqM2YTioqbJQQ/2dU641nzdaR3MyNRITW6F241N+3gFOCHBas2uoOJJ48U3D64TGni5T1c47RD+hmf+0sSgEugy5WBakPAuoCxYRfrr48zQXWFhw3luQ+DA1FjzPnUhWEIyzWbPltY+vMg3mHs4xBkMx6vkGtM8KpcZYITcghKVVQSHz2fDvzZ5XJYtAKeP9yvgZEB4Y41Kz1judMzX2Bze5DOXcF3qITpS9qDIcWP2wizFeHvnNQfBki/jQOgY6HOAdOFlOIw7vaYGdXPtADO9uVaRWrkMuCqOhdAKNpSsBc0TApNvgFPMrWVn9NUIk4AyXbEwfvSt9M9hBifBknwlbLuXkW1PAn87hnuyldMGuJj3GOyIymmmuLmQp4swPCvRvigFPcyUnBruncbNtZXgvnGS8jCTInWCE1NWxsml7BEIrOUABNhGNIRPZNq10IsZPV8IiP3AIkv1TRKluLu52mln6iDxukuZtLeyc4TNzzeEJJqWgGKgWvUj5N/scRXE0J0KMf1XYvB6spxH0DONC h+gzhk+Y q/v2BkMRENAaAX2et4QjSpqpvGixNVdi6vFjYChdQ6juHEpUS6Aqv2yCwgLTnNVkNnAaFfDvkBXsMBfcm/5pc8G0vQTqNo9sFLOkk3tMlZ10fST7cjhU/UGG3+Mhj26oJD3KR+HJZPivvYWvlFDD6+wljwIZOx/nR6F4nMQsK9OsdPtQgpeNCvKHAvwldqDGS2cv3uyPusaaK3ZOk+xTYm8lJbpawzVtd2dFmNYtSOuuulfaHOwe2dOspgS6iBTf/zgJN6Uj+tzTuIy86Jrar+qJntw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The logic for checking if a given madvise() request for a single memory range can skip real work, namely madvise_do_behavior(), is duplicated in do_madvise() and vector_madvise(). Split out the logic to a function and resue it. Signed-off-by: SeongJae Park Reviewed-by: Lorenzo Stoakes --- mm/madvise.c | 53 +++++++++++++++++++++++++++++----------------------- 1 file changed, 30 insertions(+), 23 deletions(-) diff --git a/mm/madvise.c b/mm/madvise.c index 611db868ae38..764ec1f2475b 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -1640,6 +1640,27 @@ static bool is_valid_madvise(unsigned long start, size_t len_in, int behavior) return true; } +/* + * madvise_should_skip() - Return if an madivse request can skip real works. + * @start: Start address of madvise-requested address range. + * @len_in: Length of madvise-requested address range. + * @behavior: Requested madvise behavor. + * @err: Pointer to store an error code from the check. + */ +static bool madvise_should_skip(unsigned long start, size_t len_in, + int behavior, int *err) +{ + if (!is_valid_madvise(start, len_in, behavior)) { + *err = -EINVAL; + return true; + } + if (start + PAGE_ALIGN(len_in) == start) { + *err = 0; + return true; + } + return false; +} + static bool is_madvise_populate(int behavior) { switch (behavior) { @@ -1747,23 +1768,15 @@ static int madvise_do_behavior(struct mm_struct *mm, */ int do_madvise(struct mm_struct *mm, unsigned long start, size_t len_in, int behavior) { - unsigned long end; int error; - size_t len; - - if (!is_valid_madvise(start, len_in, behavior)) - return -EINVAL; - - len = PAGE_ALIGN(len_in); - end = start + len; - - if (end == start) - return 0; + if (madvise_should_skip(start, len_in, behavior, &error)) + return error; error = madvise_lock(mm, behavior); if (error) return error; - error = madvise_do_behavior(mm, start, len_in, len, behavior); + error = madvise_do_behavior(mm, start, len_in, PAGE_ALIGN(len_in), + behavior); madvise_unlock(mm, behavior); return error; @@ -1790,19 +1803,13 @@ static ssize_t vector_madvise(struct mm_struct *mm, struct iov_iter *iter, while (iov_iter_count(iter)) { unsigned long start = (unsigned long)iter_iov_addr(iter); size_t len_in = iter_iov_len(iter); - size_t len; - - if (!is_valid_madvise(start, len_in, behavior)) { - ret = -EINVAL; - break; - } + int error; - len = PAGE_ALIGN(len_in); - if (start + len == start) - ret = 0; + if (madvise_should_skip(start, len_in, behavior, &error)) + ret = error; else - ret = madvise_do_behavior(mm, start, len_in, len, - behavior); + ret = madvise_do_behavior(mm, start, len_in, + PAGE_ALIGN(len_in), behavior); /* * An madvise operation is attempting to restart the syscall, * but we cannot proceed as it would not be correct to repeat From patchwork Mon Mar 10 17:23:13 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 14010444 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 15358C28B2E for ; Mon, 10 Mar 2025 17:23:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 21CD7280024; Mon, 10 Mar 2025 13:23:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1D12628000B; Mon, 10 Mar 2025 13:23:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 06B69280024; Mon, 10 Mar 2025 13:23:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id DCAA328000B for ; Mon, 10 Mar 2025 13:23:30 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 711461A1310 for ; Mon, 10 Mar 2025 17:23:31 +0000 (UTC) X-FDA: 83206313022.09.C65FAA6 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf16.hostedemail.com (Postfix) with ESMTP id CF1A8180016 for ; Mon, 10 Mar 2025 17:23:29 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=GElwyusP; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf16.hostedemail.com: domain of sj@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=sj@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1741627409; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=lI3TwUnH54UiiRxQDoJXdOa/94ioR5BB4qZlg/lOUG4=; b=CwPs9pTKXg8DFVD3Qqo557CxZc6GRFx4MdA6i62dqk7tjZah4qv2M0edo99nj2j8K+6kse rmaqE5eVPzX5SZHFtXC7FwObIboohd2jWGXlrS6/NCEHEa5LhGLtoLngqJv+euPbtY7EmW VeyPzwbkzRhmoeRIiUHJW3SiTs/pUus= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1741627409; a=rsa-sha256; cv=none; b=c8wJR8gRLw/qUUVVHPWo93BqWjoNC6w7hMqSlO+bGGOlVjCG28wjVlQLHpsbOX35ph5VmY 7Y60wayhn72BJFzqUeVtZBdnd46XdfhbhUaX4kICcvRs8IP+eQZmN7KzYS5Yb8BrjWHJqt BkkcaWAGLqXIfwGbnFYXMQRDHFfHV3c= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=GElwyusP; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf16.hostedemail.com: domain of sj@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=sj@kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 38B485C550D; Mon, 10 Mar 2025 17:21:12 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6388BC4CEEC; Mon, 10 Mar 2025 17:23:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1741627408; bh=Z3sSqaG1uhkCQqbFP7NKK185LV7FFPBVBGM4l1BKW9I=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=GElwyusP/WtY+XI/onO9LKP/4XGLXdBGbtjzZdwAGOE+F2xfTt6R/VDsNLdlx+0oZ BBKfLvM7iAu/KTC55Szh02Mfi4ajdSBhMaMRPxk7Dju5LCrOFP4yvqRnEraUQNijrT 85tjmyujh2l7Yj4mEk5zlXDgXBw3pseUyDEbTrDQ7NCSP0ohbe/IpYetUZD5quIBFk WRytS3AOLmgHHaThSSqghYVqfm03VCfkdjN9OGQZ0M63Ipvt6NLtyTfhQNsTGzenTU eASBFEQCg+i6eFPetX/VEhwS5Qx/W4Quts1/SEA7O6vOaH/e+/YIqOC6CeMogn+Gmz 0+QB8QKk5/H8Q== From: SeongJae Park To: Andrew Morton Cc: SeongJae Park , "Liam R. Howlett" , David Hildenbrand , Lorenzo Stoakes , Shakeel Butt , Vlastimil Babka , kernel-team@meta.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH 4/9] mm/madvise: remove len parameter of madvise_do_behavior() Date: Mon, 10 Mar 2025 10:23:13 -0700 Message-Id: <20250310172318.653630-5-sj@kernel.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250310172318.653630-1-sj@kernel.org> References: <20250310172318.653630-1-sj@kernel.org> MIME-Version: 1.0 X-Stat-Signature: z84kzdstbm9ii946bh4q3uw7zq67s61j X-Rspamd-Server: rspam05 X-Rspam-User: X-Rspamd-Queue-Id: CF1A8180016 X-HE-Tag: 1741627409-490651 X-HE-Meta: U2FsdGVkX18HtbvKENRVi37YCKBN4+yjL2mLQBnWdQJ4AkqOVLgwx+G7FGA90Pzc6rQkTcMxecAO7V72nzxqkPp34Lmh15zjkqBKJHQ+lBqZjghB9T1f7U9RnhizB1jUixgzCuvkGopbFkR81nKKDYKJPKRa56TIrhCLHt0m7WuDk9HdcU11RMw4syqR1lYX1G2ajbEVCezn1PaXyPKxOyG1YyjzHaANtndrycqb8ZDFSy2IZBtXkKbpUiVxOKIxcst5ZqwOnufJJgjGp2+x5h0ivS/Rtrsm24AtVk3Jczkvxy5PnAaUWfZ23RTvUNnxKAO0c30ij+Tfu8luvzrJr9Z45WkhcT9V2g6qSrakHLyywfNjzYWTsGPjUb3bRqM5LuuCciPoPAO96Pptthq3adCcGwwn0TtqkoRthO7nuCPZOPlG2Yv3vt+4KVhdEcNiI8/gnmLIHSBezLONGxB8cSj+m90gqphwo8vcCJdZ6lXWxHcE/EkTigalxXfGZHHxGUv8Yt8ds/Al7D2U/pVKjKj07OyAIvpSee6FaZu81jsOtr5gQgwu5MXfHepmceVxYD5fGWgSKJEOzRbGv+8at4DUZlq245x3sujY25oOubGOagmRpi/jwqPnHWqicG3OyVHzI2irqdOM/cK5Q73JcoqX9k/I9EZJL7LnVpssSgQROnvqg3AiQkpNgn24DCVHO/M1dH+L8NT6unA4yEyURxk3gbhPI54GwUStXgvSPZMGHHgNypm/dSLqzaTYcbXyNmOnigSqsfJyAHPUsKDNQuhRiyoYYByi7ap8Q+83JYazG/kNqwQf2ZkglkZ8ym0eQWkZx+XMCfuBzpoS+FYVwqiUbnOJvAA1t7QSE5ewg9JGKo+D5qCjlbwVzv575ngkuHyGCJTk43l3sdzaEW1byW+T0M3cUQC1fez5xml8iQC2kXTCPEtyQkKIq1dy0mO6v5PbJAy8rC8kN7eEqcC VOOjr7Gi SvTPUvt4Y0z6B1ETF+oXXllIg/yvtl9EmvUDld/ZMcB05ZuYmWrKS4m1K4t1A5wH+C6BabvvQJP56+yH+Jqxpaiq1RbEmrqPZ1pZmP5WL+O4CXC2D2k7EBvr8FcsD/XlAeOEarUrFWrYaHJkUZAHBqN5gT/M9e5pVnWyKN1+m+0D7Z6oEF0R11jxftXIHr73Ewj57o36YiYRQF/qR7y6UYJV/YA/N9I/C7LbZrWg+zLnUnz+Rt52UXaavSN/yI6hf4+Dql6aUe4E2ysrYwlgi0ivi4g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Because madise_should_skip() logic is factored out, making madvise_do_behavior() calculates 'len' on its own rather then receiving it as a parameter makes code simpler. Remove the parameter. Signed-off-by: SeongJae Park Reviewed-by: Lorenzo Stoakes --- mm/madvise.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/mm/madvise.c b/mm/madvise.c index 764ec1f2475b..469c25690a0e 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -1673,7 +1673,7 @@ static bool is_madvise_populate(int behavior) } static int madvise_do_behavior(struct mm_struct *mm, - unsigned long start, size_t len_in, size_t len, int behavior) + unsigned long start, size_t len_in, int behavior) { struct blk_plug plug; unsigned long end; @@ -1682,7 +1682,7 @@ static int madvise_do_behavior(struct mm_struct *mm, if (is_memory_failure(behavior)) return madvise_inject_error(behavior, start, start + len_in); start = untagged_addr_remote(mm, start); - end = start + len; + end = start + PAGE_ALIGN(len_in); blk_start_plug(&plug); if (is_madvise_populate(behavior)) @@ -1775,8 +1775,7 @@ int do_madvise(struct mm_struct *mm, unsigned long start, size_t len_in, int beh error = madvise_lock(mm, behavior); if (error) return error; - error = madvise_do_behavior(mm, start, len_in, PAGE_ALIGN(len_in), - behavior); + error = madvise_do_behavior(mm, start, len_in, behavior); madvise_unlock(mm, behavior); return error; @@ -1808,8 +1807,7 @@ static ssize_t vector_madvise(struct mm_struct *mm, struct iov_iter *iter, if (madvise_should_skip(start, len_in, behavior, &error)) ret = error; else - ret = madvise_do_behavior(mm, start, len_in, - PAGE_ALIGN(len_in), behavior); + ret = madvise_do_behavior(mm, start, len_in, behavior); /* * An madvise operation is attempting to restart the syscall, * but we cannot proceed as it would not be correct to repeat From patchwork Mon Mar 10 17:23:14 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 14010445 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C4D4EC282DE for ; Mon, 10 Mar 2025 17:23:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 23FA2280025; Mon, 10 Mar 2025 13:23:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1C51228000B; Mon, 10 Mar 2025 13:23:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 03EDE280025; Mon, 10 Mar 2025 13:23:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id D5FE628000B for ; Mon, 10 Mar 2025 13:23:31 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 682D8B8EAF for ; Mon, 10 Mar 2025 17:23:32 +0000 (UTC) X-FDA: 83206313064.25.AB729DD Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf26.hostedemail.com (Postfix) with ESMTP id B5B25140014 for ; Mon, 10 Mar 2025 17:23:30 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=hQaulE8A; spf=pass (imf26.hostedemail.com: domain of sj@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=sj@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1741627410; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=3SF2PSh+LV92puKFbfvOVdj8msgpYK7b4AvM1fkKrao=; b=zbeHx1WxBReMA3TdaNesW+R1GKbQUdDKvnfVME8A7T2UC45rgpfxsHhxfCXBlJ9uyv0Ock wfm0VjMSo9FgTkbYOto/alJNdKUWw6xE7vKNAmGJse5qFvI74077u7eITH2/AC6Xle8nGP 2p+uttgZENg96ubKs0M6nm4ZXi5R7mg= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=hQaulE8A; spf=pass (imf26.hostedemail.com: domain of sj@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=sj@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1741627410; a=rsa-sha256; cv=none; b=nohMF7O+u6WrD8hXHrljmPXAs8K06e5HpOF5Au8vJt6X2bfpSjeK2oszYUowRqeTgb2r1W KU+QkH4QrS6dYaBqvxKhu9zTFOuRsQDhcalzEirbp5nDThNN1mRyoDR6VcmO+/2zgLoZWC 1+Mj7Fy/UYR77W20N10D/oHHMxARPrY= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 0D7215C55B2; Mon, 10 Mar 2025 17:21:13 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 77A85C4CEF7; Mon, 10 Mar 2025 17:23:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1741627409; bh=4dhdwBK3uXi/DICcheMKZstN40gWxuAC0U5kXen1d3Q=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hQaulE8A/QMb9rjKSApl6fMLGmwITsJNjmDE/wylUwceFMbDOjyBzaCG+ymEcYxtB yac+cSxxdxCG0sffnMnnapqPgVDofYfA7hCavPyXj62MObV73yNUVRcBVPpbbJ+pXp Ao1+ptYPErt93XP/gPcnFn4ayqyp1rMQRPBxYKY2tUtmV9FZGignk+VK9guNz7H9B2 Db0uGgEW7OgM/N+Dwnj8hWTSmB634uk7VEeM0J0r06rbD/IUwDR8mmoXV/y82Sgtj+ lO1ipuZJ8GoK4eZQJxxAMeDsw78WXXsY330Uy2c35Zl4OjMo9XCFhWMFlEFss4ZzH7 QeXzSHEmPnwyg== From: SeongJae Park To: Andrew Morton Cc: SeongJae Park , "Liam R. Howlett" , David Hildenbrand , Lorenzo Stoakes , Shakeel Butt , Vlastimil Babka , kernel-team@meta.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH 5/9] mm/madvise: define and use madvise_behavior struct for madvise_do_behavior() Date: Mon, 10 Mar 2025 10:23:14 -0700 Message-Id: <20250310172318.653630-6-sj@kernel.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250310172318.653630-1-sj@kernel.org> References: <20250310172318.653630-1-sj@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: B5B25140014 X-Stat-Signature: rk95616m1ckgxqxp5kcukddybme9rb5r X-Rspam-User: X-HE-Tag: 1741627410-847725 X-HE-Meta: U2FsdGVkX1/6KLJ8Zc+yWNa+t99isQV4zVgb0DFuszlWSY7JhaieEM3AEzZ/b5GXdq3oF4+U+PdMZlLAsGV4iydE2Bbd9TM866SKuCpGNR2M1VIFOiKSaBKBQqhbLZTmAwXqkkpI++1uP+JGMkWugXQnqf2aHCf0ITs62EFtp3caM40NGIfJEOSAd3+f8P5SBIuZIrGCxnjiK/NEsKk1rzGPS3NLqELHJBOhP8TlfitXPp9dT/Grzj8c7rhXPkheSB/aIeiZYMbI+WIafIe8FV3mVYe5Gc+XVNJ4BhXr+drrhY932SrS4qStr+5qLyrSCkKXvzalyOsBW5Aj+yWiwJZdoglrZIxBGp6C7PzzMc8UnaDzfr/cSSdLfTIS/pnbybsz1mHmTCKKvlX+5diV406HraT5B/EILedHHcGKqkjeM54iwQfJymxglwLeZC7PoP2ybAXSFy7lKkqkly1/E6SNeWAfzjZSUjaCN800YzW9e4DLUv8sndZ3PbKPSoPe0IdxSrWDss9b47Rigoc0akKPtGiC/cXq/f8NGxMRdai9bhH4/5/HxO/+EY50EdcdtBdqwVusNnDE6Evi8QJ0gNv19BR9ET2hxqaOsvX3gqCrgjcYELAXczhg+oU5tfEq3H1b0O3EqfyLkcvKwWsEcNL6N9gKuf9y/GeCbT25c8ODerKxrzikYCY9uza5VW7MEv8tWAn/1966KGWdDU6tl7IpBZF63btL+vA47mSSdjNaymhoPdwnn9E5uFjxAYmAzSiJxSEg1GVUbQh3cM+tfw4incE3ds00GrbiJ5YP5mZ2yiknkPlL/hH4s8Q2lLQFUZzy+4/P7v+qB5OP7ag+n1Jc+3i/2wDYG15gpvmOWTyEvSlQJ+9AtcZ8hEfj1l7MUTehkkp0XvwfsfdzxSPYkDVIIhMczHVP3NcTG6x7PbSjYQn0hoG/Fw9+bK0j+vQIcLPe4VoQVn+V1UDRabk nH/ZIuPq 2cssOWann7N6sC/W0IQvWdmHzkh/4KF6ZOIgUWW/9YTuezMOJZVjVxjVNIYJxX6c7/+8k0LDYtYLeNUimyrWFL6k/NShhfWdQbAF9oCTXnPdpDcFwE6qa2oU85B4zL0UOhDQj9+vgUKceGaJbvBpoSIp1rpGQR88FAWpqyKbC8Z1KeyfxgD6G5bhfUtmZUtb09gzSvv0lzmZXBLo3B8qTpq3kovWL3yp2+vTuf9Uletff4QcS6iXRNf4Cez5xGhoHtxTnUvGdH1n0PRSSupzRHQM3UQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To implement batched tlb flushes for MADV_DONTNEED[_LOCKED] and MADV_FREE, an mmu_gather object in addition to the behavior integer need to be passed to the internal logics. Using a struct can make it easy without increasing the number of parameters of all code paths towards the internal logic. Define a struct for the purpose and use it on the code path that starts from madvise_do_behavior() and ends on madvise_dontneed_free(). Signed-off-by: SeongJae Park Reviewed-by: Lorenzo Stoakes --- mm/madvise.c | 36 ++++++++++++++++++++++++------------ 1 file changed, 24 insertions(+), 12 deletions(-) diff --git a/mm/madvise.c b/mm/madvise.c index 469c25690a0e..ba2a78795207 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -890,11 +890,16 @@ static bool madvise_dontneed_free_valid_vma(struct vm_area_struct *vma, return true; } +struct madvise_behavior { + int behavior; +}; + static long madvise_dontneed_free(struct vm_area_struct *vma, struct vm_area_struct **prev, unsigned long start, unsigned long end, - int behavior) + struct madvise_behavior *madv_behavior) { + int behavior = madv_behavior->behavior; struct mm_struct *mm = vma->vm_mm; *prev = vma; @@ -1249,8 +1254,10 @@ static long madvise_guard_remove(struct vm_area_struct *vma, static int madvise_vma_behavior(struct vm_area_struct *vma, struct vm_area_struct **prev, unsigned long start, unsigned long end, - unsigned long behavior) + void *behavior_arg) { + struct madvise_behavior *arg = behavior_arg; + int behavior = arg->behavior; int error; struct anon_vma_name *anon_name; unsigned long new_flags = vma->vm_flags; @@ -1270,7 +1277,7 @@ static int madvise_vma_behavior(struct vm_area_struct *vma, case MADV_FREE: case MADV_DONTNEED: case MADV_DONTNEED_LOCKED: - return madvise_dontneed_free(vma, prev, start, end, behavior); + return madvise_dontneed_free(vma, prev, start, end, arg); case MADV_NORMAL: new_flags = new_flags & ~VM_RAND_READ & ~VM_SEQ_READ; break; @@ -1487,10 +1494,10 @@ static bool process_madvise_remote_valid(int behavior) */ static int madvise_walk_vmas(struct mm_struct *mm, unsigned long start, - unsigned long end, unsigned long arg, + unsigned long end, void *arg, int (*visit)(struct vm_area_struct *vma, struct vm_area_struct **prev, unsigned long start, - unsigned long end, unsigned long arg)) + unsigned long end, void *arg)) { struct vm_area_struct *vma; struct vm_area_struct *prev; @@ -1548,7 +1555,7 @@ int madvise_walk_vmas(struct mm_struct *mm, unsigned long start, static int madvise_vma_anon_name(struct vm_area_struct *vma, struct vm_area_struct **prev, unsigned long start, unsigned long end, - unsigned long anon_name) + void *anon_name) { int error; @@ -1557,7 +1564,7 @@ static int madvise_vma_anon_name(struct vm_area_struct *vma, return -EBADF; error = madvise_update_vma(vma, prev, start, end, vma->vm_flags, - (struct anon_vma_name *)anon_name); + anon_name); /* * madvise() returns EAGAIN if kernel resources, such as @@ -1589,7 +1596,7 @@ int madvise_set_anon_name(struct mm_struct *mm, unsigned long start, if (end == start) return 0; - return madvise_walk_vmas(mm, start, end, (unsigned long)anon_name, + return madvise_walk_vmas(mm, start, end, anon_name, madvise_vma_anon_name); } #endif /* CONFIG_ANON_VMA_NAME */ @@ -1673,8 +1680,10 @@ static bool is_madvise_populate(int behavior) } static int madvise_do_behavior(struct mm_struct *mm, - unsigned long start, size_t len_in, int behavior) + unsigned long start, size_t len_in, + struct madvise_behavior *madv_behavior) { + int behavior = madv_behavior->behavior; struct blk_plug plug; unsigned long end; int error; @@ -1688,7 +1697,7 @@ static int madvise_do_behavior(struct mm_struct *mm, if (is_madvise_populate(behavior)) error = madvise_populate(mm, start, end, behavior); else - error = madvise_walk_vmas(mm, start, end, behavior, + error = madvise_walk_vmas(mm, start, end, madv_behavior, madvise_vma_behavior); blk_finish_plug(&plug); return error; @@ -1769,13 +1778,14 @@ static int madvise_do_behavior(struct mm_struct *mm, int do_madvise(struct mm_struct *mm, unsigned long start, size_t len_in, int behavior) { int error; + struct madvise_behavior madv_behavior = {.behavior = behavior}; if (madvise_should_skip(start, len_in, behavior, &error)) return error; error = madvise_lock(mm, behavior); if (error) return error; - error = madvise_do_behavior(mm, start, len_in, behavior); + error = madvise_do_behavior(mm, start, len_in, &madv_behavior); madvise_unlock(mm, behavior); return error; @@ -1792,6 +1802,7 @@ static ssize_t vector_madvise(struct mm_struct *mm, struct iov_iter *iter, { ssize_t ret = 0; size_t total_len; + struct madvise_behavior madv_behavior = {.behavior = behavior}; total_len = iov_iter_count(iter); @@ -1807,7 +1818,8 @@ static ssize_t vector_madvise(struct mm_struct *mm, struct iov_iter *iter, if (madvise_should_skip(start, len_in, behavior, &error)) ret = error; else - ret = madvise_do_behavior(mm, start, len_in, behavior); + ret = madvise_do_behavior(mm, start, len_in, + &madv_behavior); /* * An madvise operation is attempting to restart the syscall, * but we cannot proceed as it would not be correct to repeat From patchwork Mon Mar 10 17:23:15 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 14010447 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 34447C282EC for ; Mon, 10 Mar 2025 17:23:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2F7E7280027; Mon, 10 Mar 2025 13:23:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2828B280026; Mon, 10 Mar 2025 13:23:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 11E95280027; Mon, 10 Mar 2025 13:23:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id E208B280026 for ; Mon, 10 Mar 2025 13:23:32 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 833191CB33A for ; Mon, 10 Mar 2025 17:23:33 +0000 (UTC) X-FDA: 83206313106.29.C2C8577 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf17.hostedemail.com (Postfix) with ESMTP id B848F40002 for ; Mon, 10 Mar 2025 17:23:31 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="hYzz/5cX"; spf=pass (imf17.hostedemail.com: domain of sj@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=sj@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1741627411; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=+rmRz2JdOMH6tRBprhjyr/GkURpl1h0cVs/0yJLjiIs=; b=o+K4q0VRDno0f8QZcIoe58ytYIKbjPvbVWtDZkauvydHoSWYpiIn/5PYNkO89dD4GIksvK oDQfh7xWRAn2eowuH++CCrZumnxYI5qfp+lipD06SPSgz0K3jYKE4guGR6B1zJqd/0plgk 82ERCXCwCuvRyKA3/+ZVdcfyS/5/tTI= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="hYzz/5cX"; spf=pass (imf17.hostedemail.com: domain of sj@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=sj@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1741627411; a=rsa-sha256; cv=none; b=3x3aRxxECTvr2Seru5jl6VRFdEXkIpVBMPgnsM8bR/mk13nMX+BMZSmDC6mzKWxF0DS8Cu Em7tQ5DSxFAjU3JyuOxT9p/yTSZtIt95xTEUY3pPnhGRwnk+bBbE8FsuXKMxrkCT6MaC3E N48W6c4H6rhgKjflwc7YPO/N5HAVNps= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 218EB5C20FF; Mon, 10 Mar 2025 17:21:14 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8AA1FC4CEF5; Mon, 10 Mar 2025 17:23:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1741627410; bh=zqoCyf4FJzGNezZ0ikot3jkjvp+Q80Kl3ZyQNT+Fx4A=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hYzz/5cXRxoUPQlKUgpos/cWLuZ2KWn45gHyU3U9M75uGnPaM5UcQwf/511qy1Tp2 glCushNsOICZVKgFqNF987iZhCb+LBCWfScQQSy2z8vFAQqxetIkwYZsaB6A7BDeqM tUv73/Id2lgNh4yQOttjQdClVnd7xQMbsU5nybe6xig4aN7Lf/bXIYsmv2PxyGBXfh VLAnoMfZ/TE7CD0N3y+FbIZ+Cx8wkKXwlIWOKgmHD+Jj6vJ35J55zMYCNqUIcHqfZ6 jUqkG1jkGhq0i23slle4gJZMbR5KcqoAVdll1VNfk8SQZre7lOfG2QaUJnROV1GTI6 QX/2LPUg6599g== From: SeongJae Park To: Andrew Morton Cc: SeongJae Park , "Liam R. Howlett" , David Hildenbrand , Lorenzo Stoakes , Shakeel Butt , Vlastimil Babka , kernel-team@meta.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH 6/9] mm/memory: split non-tlb flushing part from zap_page_range_single() Date: Mon, 10 Mar 2025 10:23:15 -0700 Message-Id: <20250310172318.653630-7-sj@kernel.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250310172318.653630-1-sj@kernel.org> References: <20250310172318.653630-1-sj@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: B848F40002 X-Stat-Signature: m5zxd59dbu88skqc94yb8bjoskyu5td8 X-Rspam-User: X-HE-Tag: 1741627411-957450 X-HE-Meta: U2FsdGVkX18vx3MHnr8au9ot8lYTTqaa9Oa+1MUeYrDJBym50BhvRuDxxFgGJHcZPuLIWEbhQc08zO2JfvEC8bUwFvGVnsCIw72ASF9ZhaELi8NYNpWcm5aZzw0N8e64fYGHQIEBahJdXl55eARBnRF2xZ8XwMAgUgfGe5YjUQRTT4DkIxstQuXzbUNfFiYtXgvGyZbma5FhlCcCTBHAFzLAAQ/dZ5UDB5QB14nhsZKBJzpCGhMLOcHNk+qI5uCnR4tNWI05E562YRMNpknjUlnF/dr2V6PEtsCeScOdnAEVHj1zharPptpc0ORNDOLHRxwz7VnWMRKnYm7LS8oBKOgx7fK2G2+L2n05KjmX23E3jtbfLak45RLGZ9AOHRG4lZ+3I64BnZ9mHTt4b/QV/1SeqbI+82XDTVYmUl8i62AduQUTwr2LOoV9pkrcd6BPqFwRTho2b4I3vqHntHBxD7MOmSOUle3THch7UEruAD+JuVi1s/8BSNBDoZsFmSM073Jn/2jyzH5eJvsujQAHLtLjpMwL6iZ4sPuVXYeffMomAeBs27HOuMWkjfDB8ZYCcWOmiE+1neHNF27qML9+EeuAlrL6UOyP931RjrjjOdrxVtY8OpuUYyU1tmX6PyXam25A+Apd1n7aR0nN6NNtYYaTI5OTxmqNJlCl+TpyvXRv51iQ7HX6iNMLEqfB+pTEYdXnBT87FiPVuU/QyLXvWRBZZzL75rMWq2gj0rbYB6jCc7Kf/S+UNVKC/ZgA3X4/Wdpst4tc0oyBTQzJEDMLHkQPAnWkLzL10I+6T9zpInAE23LIG487HBuGTxa7BPs+XUgvc5Goqj3U06yx5oZtBdfOD4mqgkIwyEplhfHr4ge+XSVlrWvEJQMRblvykiCow//4gzAukGtK8S5bAt26NeM5vqf8HGIJBLRgYo7rmWo4DVDu7vmmr/U9apUZzAZpveOP5+jDoQdBFkqr+hQ ZygY1Cjv MbWNX7SoRAM3IzQ5o5mLKJtzrd1GYFaC4rRFXhfZBEAM0yRbUJpx3BR6Xz6VotK4chMzgM57/ePsFlayjD1qrQsAAb7M2gjHciNTj9LR0eCyyfwEBzcJQjuNzYu5P6SDWevk0HIb9WyU1+PlZNrr+qEDkeSiu8Er5ABT1ms80jgtDmLNm3q69l3up2AT4yPgn+uJFJ3eDh+rL1EYzu3dwQvrq/UIGAeytwG/LBzrIK8OQOXQHO2XZnW3S4W2jREhtcjzN0Hg9qpuMF1UIVQpjnQGJ2w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Some of zap_page_range_single() callers such as [process_]madvise() with MADV_DONEED[_LOCKED] cannot batch tlb flushes because zap_page_range_single() does tlb flushing for each invocation. Split out the body of zap_page_range_single() except mmu_gather object initialization and gathered tlb entries flushing parts for such batched tlb flushing usage. Signed-off-by: SeongJae Park --- mm/memory.c | 36 ++++++++++++++++++++++-------------- 1 file changed, 22 insertions(+), 14 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 78c7ee62795e..88c478e2ed1a 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1995,38 +1995,46 @@ void unmap_vmas(struct mmu_gather *tlb, struct ma_state *mas, mmu_notifier_invalidate_range_end(&range); } -/** - * zap_page_range_single - remove user pages in a given range - * @vma: vm_area_struct holding the applicable pages - * @address: starting address of pages to zap - * @size: number of bytes to zap - * @details: details of shared cache invalidation - * - * The range must fit into one VMA. - */ -void zap_page_range_single(struct vm_area_struct *vma, unsigned long address, +static void unmap_vma_single(struct mmu_gather *tlb, + struct vm_area_struct *vma, unsigned long address, unsigned long size, struct zap_details *details) { const unsigned long end = address + size; struct mmu_notifier_range range; - struct mmu_gather tlb; mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, vma->vm_mm, address, end); hugetlb_zap_begin(vma, &range.start, &range.end); - tlb_gather_mmu(&tlb, vma->vm_mm); update_hiwater_rss(vma->vm_mm); mmu_notifier_invalidate_range_start(&range); /* * unmap 'address-end' not 'range.start-range.end' as range * could have been expanded for hugetlb pmd sharing. */ - unmap_single_vma(&tlb, vma, address, end, details, false); + unmap_single_vma(tlb, vma, address, end, details, false); mmu_notifier_invalidate_range_end(&range); - tlb_finish_mmu(&tlb); hugetlb_zap_end(vma, details); } +/** + * zap_page_range_single - remove user pages in a given range + * @vma: vm_area_struct holding the applicable pages + * @address: starting address of pages to zap + * @size: number of bytes to zap + * @details: details of shared cache invalidation + * + * The range must fit into one VMA. + */ +void zap_page_range_single(struct vm_area_struct *vma, unsigned long address, + unsigned long size, struct zap_details *details) +{ + struct mmu_gather tlb; + + tlb_gather_mmu(&tlb, vma->vm_mm); + unmap_vma_single(&tlb, vma, address, size, details); + tlb_finish_mmu(&tlb); +} + /** * zap_vma_ptes - remove ptes mapping the vma * @vma: vm_area_struct holding ptes to be zapped From patchwork Mon Mar 10 17:23:16 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 14010448 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F1B90C282DE for ; Mon, 10 Mar 2025 17:23:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BC70A280028; Mon, 10 Mar 2025 13:23:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B4D92280026; Mon, 10 Mar 2025 13:23:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 97B9C280028; Mon, 10 Mar 2025 13:23:34 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 754C9280026 for ; Mon, 10 Mar 2025 13:23:34 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 03A0DC12B9 for ; Mon, 10 Mar 2025 17:23:35 +0000 (UTC) X-FDA: 83206313190.27.4291E18 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf16.hostedemail.com (Postfix) with ESMTP id 55BD018000E for ; Mon, 10 Mar 2025 17:23:32 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=hSx76SP1; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf16.hostedemail.com: domain of sj@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=sj@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1741627412; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=6bHFtBemQj4wFv7o9DpuwYSyeCUOKe2XoHkTMo5mXUs=; b=c9KRvPh2lxNEXjNpAzn5YVF/h6D3ksjl3Uv4E+mY/BYGJjHg3nD6hDPGrOpaLwIc+L8FHM SU7aeQEgqWaeIS5UhKikpqv41GY3YOjGqZCnXf6yEWs4UZQYo6FLRqwyG+TKs2j7BQmPxd 0zpcerdIhaTmKBXM+oUtloCCTSmX3nE= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1741627412; a=rsa-sha256; cv=none; b=YfWFwAMEAoZ0Cg7IxD8WYCX33+iY5044oNF+NEOp2UZdKMUwCt++8GpbVkhe06JBjyiFqs lvll/d2Xecr8Y4nrBNRUPSJmqTcAmA1SKNv2jMEfrSJFrbwX11KBBWxrI3KHlyRTWMDAWi C9s3MT+JL2JmnhAMU/MjdeGIrW38EZQ= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=hSx76SP1; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf16.hostedemail.com: domain of sj@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=sj@kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 3D9E85C14D9; Mon, 10 Mar 2025 17:21:15 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A5EDEC4CEED; Mon, 10 Mar 2025 17:23:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1741627411; bh=RSFUpifN3HuVWoEj944CoZ0VVGpww5B6SbD8iILWaq8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hSx76SP1aMWn5lYQ40SpWdThFZAVcwsocl3ap/sjfMRB7710mfqiSRU6pLK6zibY+ /JVK/IYLx1GfYEXDCgblSfJbiMNgX96eIfr3qx8F+pOU+KqVLcvH3sqjmx4o8rUgS2 8xu2CsWKaUtKwmy6yCSvJLoHKNOkTkLlB25SLTWdsjZ4GzC0OaJhZtNXn798S5yRAE HHwcXczG6UftPqSdNetvITTqiLoPeMA3WDlTu4ynL3Wkd7HhCMVxVOvpYEGTmTkSf5 Dwo21kDvLQYj+Y2+Ka8/D3qsv0CE7h3+tuFfOS42mpzYM3XRqlPTHO4EyILJQjMfvr oP5hoQkQjgILA== From: SeongJae Park To: Andrew Morton Cc: SeongJae Park , "Liam R. Howlett" , David Hildenbrand , Lorenzo Stoakes , Shakeel Butt , Vlastimil Babka , kernel-team@meta.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH 7/9] mm/madvise: let madvise_{dontneed,free}_single_vma() caller batches tlb flushes Date: Mon, 10 Mar 2025 10:23:16 -0700 Message-Id: <20250310172318.653630-8-sj@kernel.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250310172318.653630-1-sj@kernel.org> References: <20250310172318.653630-1-sj@kernel.org> MIME-Version: 1.0 X-Stat-Signature: 6h84irh3sy7nnt31akupms8hoecsp4m7 X-Rspamd-Server: rspam05 X-Rspam-User: X-Rspamd-Queue-Id: 55BD018000E X-HE-Tag: 1741627412-14690 X-HE-Meta: U2FsdGVkX19hXEBj/lPV7xiK3/UtgZU713t+HL5zfhy5mC8mBAYEMvJfQHG69i/G8nP/bVmZxw+KYVKie0OI1mlWh3jLyIjmvONzsAbraHdVytASTGqNZmSUghWNk37oSQXIvIs1vGaZf8AK+UGerntbvS3hhfF0QtZyEfDM2UZVyL87L2GMpgj191aHBfh5GFs66Vezg2oHbK07MEODz+KGzoawaUoE6hmvB3gUc9KtaSEaJ+VNiGR80oK4paEMdceUQzEKQuIM91TSKbNa41MEP8hGZsigVO4vrgCueIrmOtLW4MJa9/yZUGVX5mgOKsbcWixRiuSSQ9EJHYQvlfn0vZiTKsSfVgPlRfrQKTmFZallWN1IdrGV+PICmtc5S+DnBxG7DzjsS8RnYE02+HFLJOmytWDxSvYuMxwCjG2fQCvnAznI1VZ791fOkOqem93w9YdqcJd91//gBrtWsGrKjzJRMV/cOtbSeUYXixS//LbdOfR68WGV6EBQTSv2uc01mSVrE77YmjB4BTdCk+w/znnSJzoCsu3yG79WCI65FYuf6joHUpIUuvqjfw1qSqD2SSiIVlAfKgwVkd+3qZ7ZsFjpjRLCkE1Dd+XvtgzVc75q5Fro4aMf26IOoyCQ6UdY3vv33vYO4BcpdE1XtHl0Ve8LQMu8IVqIw7Om2LSxJb46JPr0FBgwUxbRR7KtEkPAk7lFCG1496zoc7fe+NtQ1vGz4tQ8ncxnTOBHvzXdBjxKn+fVAjiE+mFT55u648QTiFMb60AFKUJTsspkMRPJ8HxUjprU4YnJ2vP+UXYjrPY/k8vm+fzxJyd2s98dJblfCJNIzf8+QxZ6mXg4UK1y7JnnZBhJAdoxvYQu+ttRFbQZuFDKwSRzOZY/2IPeiQjcze6WR1xdbqjqIopWn/cdTeO04jQzDO+VCVIHIR95s4ZjrP1MLLy3NCKRcMJolNtQvletP/sZkHVv9sn HrU63VBR xCNn7FFvcP5e9m6TLTDD9fQO0gMV8KgoFPFj5xPD2NQryKq4gwccjlLAI9MJZ6Wu9nVj6XmF9nxHtxtUoNVpRM88tfiBh9x0DktlhsNO5RYmYGtRPgoXJon+1s56xzM3J/sWk9urTM656R2iS5oyZWHzBVeq9fbIDt+9dGTllse/rAKlgsvCxepa/fl+EIQ/gLKY6hxC8ZcZa0nx3lFLaRKG+6rJEGuGZFw6gdAx4u8PXy/ymJyj9JQlFCZ4UzmE5OGLZCFmVSiOfqAkhdDZjVnPK2GalZY9x0+FF X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Update madvise_dontneed_single_vma() and madvise_free_single_vma() functions so that the caller can pass an mmu_gather object that should be initialized and will be finished outside, for batched tlb flushes. Also modify their internal code to support such usage by skipping the initialization and finishing of self-allocated mmu_gather object if it received a valid mmu_gather object. Signed-off-by: SeongJae Park --- mm/internal.h | 3 +++ mm/madvise.c | 37 +++++++++++++++++++++++++------------ mm/memory.c | 16 +++++++++++++--- 3 files changed, 41 insertions(+), 15 deletions(-) diff --git a/mm/internal.h b/mm/internal.h index 0caa64dc2cb7..ce7fb2383f65 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -438,6 +438,9 @@ void unmap_page_range(struct mmu_gather *tlb, struct vm_area_struct *vma, unsigned long addr, unsigned long end, struct zap_details *details); +void unmap_vma_single(struct mmu_gather *tlb, struct vm_area_struct *vma, + unsigned long addr, unsigned long size, + struct zap_details *details); int folio_unmap_invalidate(struct address_space *mapping, struct folio *folio, gfp_t gfp); diff --git a/mm/madvise.c b/mm/madvise.c index ba2a78795207..d7ea71c6422c 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -794,12 +794,19 @@ static const struct mm_walk_ops madvise_free_walk_ops = { .walk_lock = PGWALK_RDLOCK, }; -static int madvise_free_single_vma(struct vm_area_struct *vma, - unsigned long start_addr, unsigned long end_addr) +static int madvise_free_single_vma( + struct mmu_gather *caller_tlb, struct vm_area_struct *vma, + unsigned long start_addr, unsigned long end_addr) { struct mm_struct *mm = vma->vm_mm; struct mmu_notifier_range range; - struct mmu_gather tlb; + struct mmu_gather self_tlb; + struct mmu_gather *tlb; + + if (caller_tlb) + tlb = caller_tlb; + else + tlb = &self_tlb; /* MADV_FREE works for only anon vma at the moment */ if (!vma_is_anonymous(vma)) @@ -815,16 +822,18 @@ static int madvise_free_single_vma(struct vm_area_struct *vma, range.start, range.end); lru_add_drain(); - tlb_gather_mmu(&tlb, mm); + if (!caller_tlb) + tlb_gather_mmu(tlb, mm); update_hiwater_rss(mm); mmu_notifier_invalidate_range_start(&range); - tlb_start_vma(&tlb, vma); + tlb_start_vma(tlb, vma); walk_page_range(vma->vm_mm, range.start, range.end, - &madvise_free_walk_ops, &tlb); - tlb_end_vma(&tlb, vma); + &madvise_free_walk_ops, tlb); + tlb_end_vma(tlb, vma); mmu_notifier_invalidate_range_end(&range); - tlb_finish_mmu(&tlb); + if (!caller_tlb) + tlb_finish_mmu(tlb); return 0; } @@ -848,7 +857,8 @@ static int madvise_free_single_vma(struct vm_area_struct *vma, * An interface that causes the system to free clean pages and flush * dirty pages is already available as msync(MS_INVALIDATE). */ -static long madvise_dontneed_single_vma(struct vm_area_struct *vma, +static long madvise_dontneed_single_vma(struct mmu_gather *tlb, + struct vm_area_struct *vma, unsigned long start, unsigned long end) { struct zap_details details = { @@ -856,7 +866,10 @@ static long madvise_dontneed_single_vma(struct vm_area_struct *vma, .even_cows = true, }; - zap_page_range_single(vma, start, end - start, &details); + if (!tlb) + zap_page_range_single(vma, start, end - start, &details); + else + unmap_vma_single(tlb, vma, start, end - start, &details); return 0; } @@ -951,9 +964,9 @@ static long madvise_dontneed_free(struct vm_area_struct *vma, } if (behavior == MADV_DONTNEED || behavior == MADV_DONTNEED_LOCKED) - return madvise_dontneed_single_vma(vma, start, end); + return madvise_dontneed_single_vma(NULL, vma, start, end); else if (behavior == MADV_FREE) - return madvise_free_single_vma(vma, start, end); + return madvise_free_single_vma(NULL, vma, start, end); else return -EINVAL; } diff --git a/mm/memory.c b/mm/memory.c index 88c478e2ed1a..3256b9713cbd 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1995,9 +1995,19 @@ void unmap_vmas(struct mmu_gather *tlb, struct ma_state *mas, mmu_notifier_invalidate_range_end(&range); } -static void unmap_vma_single(struct mmu_gather *tlb, - struct vm_area_struct *vma, unsigned long address, - unsigned long size, struct zap_details *details) +/** + * unmap_vma_single - remove user pages in a given range + * @tlb: pointer to the caller's struct mmu_gather + * @vma: vm_area_struct holding the applicable pages + * @address: starting address of the pages + * @size: number of bytes to remove + * @details: details of shared cache invalidation + * + * @tlb shouldn't be NULL. The range must fit into one VMA. + */ +void unmap_vma_single(struct mmu_gather *tlb, struct vm_area_struct *vma, + unsigned long address, unsigned long size, + struct zap_details *details) { const unsigned long end = address + size; struct mmu_notifier_range range; From patchwork Mon Mar 10 17:23:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 14010449 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C6D77C28B2E for ; Mon, 10 Mar 2025 17:23:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D2375280029; Mon, 10 Mar 2025 13:23:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C7CE1280026; Mon, 10 Mar 2025 13:23:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AA8E4280029; Mon, 10 Mar 2025 13:23:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 85DC2280026 for ; Mon, 10 Mar 2025 13:23:35 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 140BD141343 for ; Mon, 10 Mar 2025 17:23:36 +0000 (UTC) X-FDA: 83206313232.23.26087B1 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf26.hostedemail.com (Postfix) with ESMTP id 67626140004 for ; Mon, 10 Mar 2025 17:23:33 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=krWQwApA; spf=pass (imf26.hostedemail.com: domain of sj@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=sj@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1741627413; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=j/eJqt7TuvyP0SAnStLtA2j7PJ8ZDMjIk9dDwSEf/Lg=; b=ZnZpeErdVC7bGg2u3jHNLzSipXvP5kHhrHnzgfQBwoBqifX+9omEraXXB3HJ0QQ7p95f8U xVrWpOsIIJopAdFK1lnH2T0mE/mrWDYnaIV/0YIbxtBEP0O97At5G7S4ZJC5Mth8fQfMqx VJqKSK4ZJW6k3kLDGoWgz0+MSs0osCU= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=krWQwApA; spf=pass (imf26.hostedemail.com: domain of sj@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=sj@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1741627413; a=rsa-sha256; cv=none; b=NoY+iDIA06N+kD3H5xZXzoYP3HJyRLVpAVm8zg92eEtWU/WjrA8GzJCdOQ4EcucG4VKWTD ZTFh17TvKaQe0qCjUU16ZsNgyqF3mlUfnD+xYzRB51lCnpJyl+P3qmv6jhWCXykf4eTnTo e5G7TPrTfMi00QO3w+8AcI//PKVfYOI= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 5296D5C0608; Mon, 10 Mar 2025 17:21:16 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id BC673C4CEEC; Mon, 10 Mar 2025 17:23:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1741627412; bh=XwNyPWkuo3x8UEQgYUucQN44sSUMxDXOQT6wzHvLom0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=krWQwApAFooJw8RDR+VQklpBSPp/jb0qvzqLu5pCDpb7eW76XF5JyZwxHg4cy4FEU XvMGRSStteW7kyuxgIk/BsxYaqWeRdLQaa6PTij9bEWwE2+rClOI+uMaCmLkdK2M8K 8MzU+FiOthWHwlK/bdFrs0qShtkdkmaAE2UaZYZM8XMrmJtSnawjt/mLWf6uLnDD0d C+kVoxCRMP9beh4TlBf27mnPlUXccx8TLOwatzKscv0CoCZgrpLWWu7zmxnnpVRCyE Liha99fJhrQLYsMoDyQNiPeJZ4C//wNOTCnt8/6+cNVwvl2dr6Jp1ifIZUVfh+ZNIy g/6Q0LWdHYpWA== From: SeongJae Park To: Andrew Morton Cc: SeongJae Park , "Liam R. Howlett" , David Hildenbrand , Lorenzo Stoakes , Shakeel Butt , Vlastimil Babka , kernel-team@meta.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH 8/9] mm/madvise: batch tlb flushes for [process_]madvise(MADV_{DONTNEED[_LOCKED],FREE}) Date: Mon, 10 Mar 2025 10:23:17 -0700 Message-Id: <20250310172318.653630-9-sj@kernel.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250310172318.653630-1-sj@kernel.org> References: <20250310172318.653630-1-sj@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 67626140004 X-Stat-Signature: u5zwcr5kcrwj3a8s7uq9krink6qzqjnh X-Rspam-User: X-HE-Tag: 1741627413-489294 X-HE-Meta: U2FsdGVkX1/vrbaEutRL6rTVo9WziShMBg15ADrC6JcQDLJgSWLTZHBKlZSy7D3KkH63NZv+oPPOg/ABBTdluVgEpaNOIQjEnu2XnGyoZmvxxKbbDktIRNgTZX1+28dXnyhV7O8ESTnELCnzNPn0gClx8E32eqOZbjEtpufBNRGmzdr70tzs84LKUNvDlNlqNP1Xtyp2HlS3z5Svav+sz3E4M6PhtlCS9GepUe+qfXGJEB1qEKCzwgbbsg7PvR2UkKJTJmjytUUEwR6/iYDqoQxcMyGLtYBl0dPjVHBAGx1U2Ggp4BSHvmz6FNXlbN5kxGgoYi2wP7bWxsK980oV2ozS8pYGPekmMpa7OYHvtwcmFyacZVeYhbzFIF6UhxW40TOWdSDB4ffk+sgmrg3TEM6NqM3usHzITnC2OdjXBd9g/JhAsvgnKotkWj79S5n5ZI4eC+KDIWk7T2rV+OLz9rLFDDrU5UrtwA1YaCMrB6c0HE6t+hCxNEuK3+thJX+3mAbFLnrsdqO9t327tDphsQAPqlJf/tBeIOO55QU3WY9EinTHDAejgUTNBWHTWalUNXqXti1eM2uloux2KNMDqjJ4QSd59A8g1lUmjEGkbTzppK0CVKVzP3zEfGx/iGYVerCXzl3SaUEqPB6VRuPOyzFv30s/JifDwUajzoLcivwwgDF2y9xKx5G4bEj53hEZXtFI1dYb4aoHL1YLgVBkNJG5G6XK4dQDz9E2HR0vnbCKXwcEzlJ0sX1DWG+CEYhnit9rwjW788Ne9vOD/yI1ngIWIOrlepkRMPCKS13clwskaEvwFI3j0j5zBa6sYPCpxsAgzQsrf2SB6ia3RlxgoWtWZYN8yajrLAJtiw2kgGMQvrmVS44cUUEhffnThmjxTArykYzLxUnbVUaDt+ubg3RTANjB4iNQ4TwgKey4hY2upCYsqk22aCK4nt3Udlp+A7rfXqSxL2EH5XzCVOK BJLRaJyE V9yiNP4KRKS46zQhzi6AgQeH81WoQ+sZRWYGfaiyeAGoWylruuDppm/MVWQNe/lOmR/6Z/gr2cQcwLC2gvCnnVudqi3R3veVyJD8OtdN+7WXbTWHCDzHuJM4za6L0w/ESaZbU08JD6GMmgtkg+FKEpzEIQr8Ced5Nw6lq3O6GeC37jgbIsM6HJJFLU+PuWoR4cEokuedT73amNyr4+B0iMKjhlgWWpOxUF8at/2nkQGuN+94aEFe/7snvXv8wlDAa0KDabt5SpPGCoX9S/YTigB/UpVqLZNKWuo/v X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: MADV_DONTNEED[_LOCKED] and MADV_FREE internal logics for [process_]madvise() can be invoked with batched tlb flushes. Update vector_madvise() and do_madvise(), which are called for the two system calls respectively, to use those in the efficient way. Initialize an mmu_gather object before starting the internal works, and flush the gathered tlb entries at once after all the internal works are done. Signed-off-by: SeongJae Park --- mm/madvise.c | 51 +++++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 47 insertions(+), 4 deletions(-) diff --git a/mm/madvise.c b/mm/madvise.c index d7ea71c6422c..d5f4ce3041a4 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -905,6 +905,7 @@ static bool madvise_dontneed_free_valid_vma(struct vm_area_struct *vma, struct madvise_behavior { int behavior; + struct mmu_gather *tlb; }; static long madvise_dontneed_free(struct vm_area_struct *vma, @@ -964,9 +965,11 @@ static long madvise_dontneed_free(struct vm_area_struct *vma, } if (behavior == MADV_DONTNEED || behavior == MADV_DONTNEED_LOCKED) - return madvise_dontneed_single_vma(NULL, vma, start, end); + return madvise_dontneed_single_vma( + madv_behavior->tlb, vma, start, end); else if (behavior == MADV_FREE) - return madvise_free_single_vma(NULL, vma, start, end); + return madvise_free_single_vma( + madv_behavior->tlb, vma, start, end); else return -EINVAL; } @@ -1639,6 +1642,32 @@ static void madvise_unlock(struct mm_struct *mm, int behavior) mmap_read_unlock(mm); } +static bool madvise_batch_tlb_flush(int behavior) +{ + switch (behavior) { + case MADV_DONTNEED: + case MADV_DONTNEED_LOCKED: + return true; + default: + return false; + } +} + +static void madvise_init_tlb(struct madvise_behavior *madv_behavior, + struct mm_struct *mm) +{ + if (!madvise_batch_tlb_flush(madv_behavior->behavior)) + return; + tlb_gather_mmu(madv_behavior->tlb, mm); +} + +static void madvise_finish_tlb(struct madvise_behavior *madv_behavior) +{ + if (!madvise_batch_tlb_flush(madv_behavior->behavior)) + return; + tlb_finish_mmu(madv_behavior->tlb); +} + static bool is_valid_madvise(unsigned long start, size_t len_in, int behavior) { size_t len; @@ -1791,14 +1820,20 @@ static int madvise_do_behavior(struct mm_struct *mm, int do_madvise(struct mm_struct *mm, unsigned long start, size_t len_in, int behavior) { int error; - struct madvise_behavior madv_behavior = {.behavior = behavior}; + struct mmu_gather tlb; + struct madvise_behavior madv_behavior = { + .behavior = behavior, + .tlb = &tlb, + }; if (madvise_should_skip(start, len_in, behavior, &error)) return error; error = madvise_lock(mm, behavior); if (error) return error; + madvise_init_tlb(&madv_behavior, mm); error = madvise_do_behavior(mm, start, len_in, &madv_behavior); + madvise_finish_tlb(&madv_behavior); madvise_unlock(mm, behavior); return error; @@ -1815,13 +1850,18 @@ static ssize_t vector_madvise(struct mm_struct *mm, struct iov_iter *iter, { ssize_t ret = 0; size_t total_len; - struct madvise_behavior madv_behavior = {.behavior = behavior}; + struct mmu_gather tlb; + struct madvise_behavior madv_behavior = { + .behavior = behavior, + .tlb = &tlb, + }; total_len = iov_iter_count(iter); ret = madvise_lock(mm, behavior); if (ret) return ret; + madvise_init_tlb(&madv_behavior, mm); while (iov_iter_count(iter)) { unsigned long start = (unsigned long)iter_iov_addr(iter); @@ -1850,14 +1890,17 @@ static ssize_t vector_madvise(struct mm_struct *mm, struct iov_iter *iter, } /* Drop and reacquire lock to unwind race. */ + madvise_finish_tlb(&madv_behavior); madvise_unlock(mm, behavior); madvise_lock(mm, behavior); + madvise_init_tlb(&madv_behavior, mm); continue; } if (ret < 0) break; iov_iter_advance(iter, iter_iov_len(iter)); } + madvise_finish_tlb(&madv_behavior); madvise_unlock(mm, behavior); ret = (total_len - iov_iter_count(iter)) ? : ret; From patchwork Mon Mar 10 17:23:18 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 14010450 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 55D41C282EC for ; Mon, 10 Mar 2025 17:23:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D16A928002A; Mon, 10 Mar 2025 13:23:36 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CC1D7280026; Mon, 10 Mar 2025 13:23:36 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AA1EB28002A; Mon, 10 Mar 2025 13:23:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 87C7B280026 for ; Mon, 10 Mar 2025 13:23:36 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 1619BA9E9C for ; Mon, 10 Mar 2025 17:23:37 +0000 (UTC) X-FDA: 83206313274.21.D481AEB Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf17.hostedemail.com (Postfix) with ESMTP id 768CF4000A for ; Mon, 10 Mar 2025 17:23:34 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Set4HVYi; spf=pass (imf17.hostedemail.com: domain of sj@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=sj@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1741627414; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=le7eqrsuiKcHpEYbmiuI7mTII8OSn/wYe9F5C6bWfsM=; b=CYPNiG5RZOnqechY5SWeU8b0+JEo81R+Iw7/lUXz9qprYCM9wzxYf2iVNnmXd73LSnaMBE HlMwumqM1Co2E6eGR4d3LMLelduQW9U9XtTb7mTbpAy5STvn+wVA0r1C5knSqBHOabot7M s+0ja8rtSjQFEbXCmkoRqHlW3t4hJhI= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Set4HVYi; spf=pass (imf17.hostedemail.com: domain of sj@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=sj@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1741627414; a=rsa-sha256; cv=none; b=ahF5U3huAYDTsAaJ0g4T79+9nP+s59rSfUF1hvhZjOoQlMFRpftLBj2su/pJmIG/P7WwJq deCdYQAFuqJx5rE3D0K+pJr11YN19R0okFwFggJVctQzIwZm0McW93UEU1VJ0qSTcbk+2g yXriQ8SVPjriqcqaP/DQdGzZhY89uqI= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 6688F5C1191; Mon, 10 Mar 2025 17:21:17 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D4A8CC4CEF3; Mon, 10 Mar 2025 17:23:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1741627414; bh=P3FkpaakjLXsflqPzPkcSxDHFwcvtfrO/BuhT1Vlk/c=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Set4HVYiJXagZv6AHBunmJA2YaxUOJNXk/xAV/vhDbFjpU1vNEeI+SBBALuj/M05i KURLdN1pNktQabof22sMSLvUlHkADTNX1hIHgdSslYYSh8X5MurEWENbg0as8amXZ8 3j67efJT7/8ngHcGzDhAqL3JWDpqLWmrNa3eE93RbLgP3ZaGBOJN+U3MkkZUcfemAX j2w0y5eq4tpmZ16QqA+T7gOEJOtMnGL7tyQX/IHV0A4AE+tJiAxejseb4ofZt0hmKP 64a4hJBNiaED6o2r144J+62wh3s3o/d4eb9OVTVKRL8Ck59jNEoJ81sBI1Qo/AZK4L t4BlhlGEzf4NQ== From: SeongJae Park To: Andrew Morton Cc: SeongJae Park , "Liam R. Howlett" , David Hildenbrand , Lorenzo Stoakes , Shakeel Butt , Vlastimil Babka , kernel-team@meta.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH 9/9] mm/madvise: remove !tlb support from madvise_{dontneed,free}_single_vma() Date: Mon, 10 Mar 2025 10:23:18 -0700 Message-Id: <20250310172318.653630-10-sj@kernel.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250310172318.653630-1-sj@kernel.org> References: <20250310172318.653630-1-sj@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 768CF4000A X-Stat-Signature: jhxgaqjqdx87mnnirqrcztyx9akp5g5t X-Rspam-User: X-HE-Tag: 1741627414-240424 X-HE-Meta: U2FsdGVkX194dW4W9sK03Rvxr/poRuzZX2t8htwI3+4D1FkaLuYcqH556w/dP9749mDv/PVBVAehJahR1DZgzKxYiEGEiCqgkCekiXL/SRdPmUPjN/3Rf9l7Br7YMJmwfAtN1lzAqmxZY5oGzL3/JaxQy35Zya7pscvuTYYXComHlQpu/aKJILdwrr/8sgdf51NtxdQL8XIXXJR1otkL3jndj1+K2+MPDXLd376Oblff8d/IJHhwYSCOLiRWlhdOoOqxCweBvp35MaJkJrbOq8E3fVj+Qd4wlaqXfMn3UBk2e8v587yvgZ1HYvTw5sp7jqF5F1l4iJph3kmU0UscjS98bvShWK08/Cc0IdUH7zAkw5COvPV8bnxS8ALlF/BwJXU7TyD5HhwIJTwg0D2OBuxo7HhzMhyXiAKmYME3kWJsFyBgO9n9iQRinKdxuSLFca3cYtPgVmB72YUkKMiquVnISWJFKw4tIm5yI2ti8YKQk+2oRLPfXoau6SioylMP1qIDBGZodjgPjnhq4VW9KtBuSk2JhBm4PpLQDNTit5o1tXBp4QLoPWRrXBIod0C/cAZK1T6uQ7P7LgwI6NgQ+1w0WjEQ+pw/ERreeT89oicKLEgUDpxaNMxkmhlC1zGtJ/mLYo5gmkCxxAv1KOFva7kSfLPLKkLDNsOUpoUSShdW0VINR5OwyvVhGfo+H5MfrXZKT56xPKhITqu1NvKUXp9Jg66NyKTpg8edBNd9spl7YAiR47cPrbIkl2TDT5XCUmT2/CvWYBoOTfUFfO+xZjmjaLORVdtrflEsqrNTTzCwTwuMB4KyAVryPmZmkhfQVVoK5BCtEp82muWP9ka53NI9JQ1vHugzJkpUNRGgiQDK6PqCDSNFIr029xB05F8k5wc1kBjkLpNpVm5vTFLCgQhyOLX3Y24LP+2yI09eDmFTd3o6D08HvIeNsBuAodHKK6s+X2/TWTej/CqWEb/ 3cLvIisT Q6WidoDxcXplFZQz8PjTt3JPLk34VBJNJpYREr/giRaK1OM+SOx36+Ild6fGnZW6paFVfRI7J2FJO/VPYyv+I1OOZO4b7CFJjXIkOB+/h0UZd9Fh9uJDI12SWZSsR70VS+dTSbpVKJJRmHA71i4gRK/x9aLYl3J7gy3QHQTB3nm09vct4plZR0qsMHlbQ+ENlApO8aK56H9o96zyD/IDoOnc5qKgVAaEMt1lZDiR8LGs4356tftsG/KCGkY7W3Zt6NNBh3LX4aScaRSffgHPcswKRWdJm2yDI2epYNQxglLjCBDn4XkaUHrelH438B6JXvWsj X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: madvise_dontneed_single_vma() and madvise_free_single_vma() support both batched tlb flushes and unbatched tlb flushes use cases depending on received tlb parameter's value. The supports were for safe and fine transition of the usages from the unbatched flushes to the batched ones. Now the transition is done, and therefore there is no real unbatched tlb flushes use case. Remove the code for supporting the no more being used cases. Signed-off-by: SeongJae Park Reviewed-by: Lorenzo Stoakes --- mm/madvise.c | 19 ++----------------- 1 file changed, 2 insertions(+), 17 deletions(-) diff --git a/mm/madvise.c b/mm/madvise.c index d5f4ce3041a4..25af0a24c00b 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -795,18 +795,11 @@ static const struct mm_walk_ops madvise_free_walk_ops = { }; static int madvise_free_single_vma( - struct mmu_gather *caller_tlb, struct vm_area_struct *vma, + struct mmu_gather *tlb, struct vm_area_struct *vma, unsigned long start_addr, unsigned long end_addr) { struct mm_struct *mm = vma->vm_mm; struct mmu_notifier_range range; - struct mmu_gather self_tlb; - struct mmu_gather *tlb; - - if (caller_tlb) - tlb = caller_tlb; - else - tlb = &self_tlb; /* MADV_FREE works for only anon vma at the moment */ if (!vma_is_anonymous(vma)) @@ -822,8 +815,6 @@ static int madvise_free_single_vma( range.start, range.end); lru_add_drain(); - if (!caller_tlb) - tlb_gather_mmu(tlb, mm); update_hiwater_rss(mm); mmu_notifier_invalidate_range_start(&range); @@ -832,9 +823,6 @@ static int madvise_free_single_vma( &madvise_free_walk_ops, tlb); tlb_end_vma(tlb, vma); mmu_notifier_invalidate_range_end(&range); - if (!caller_tlb) - tlb_finish_mmu(tlb); - return 0; } @@ -866,10 +854,7 @@ static long madvise_dontneed_single_vma(struct mmu_gather *tlb, .even_cows = true, }; - if (!tlb) - zap_page_range_single(vma, start, end - start, &details); - else - unmap_vma_single(tlb, vma, start, end - start, &details); + unmap_vma_single(tlb, vma, start, end - start, &details); return 0; }