From patchwork Tue Dec 10 21:37:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gregory Price X-Patchwork-Id: 13902102 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8AB8CE7717F for ; Tue, 10 Dec 2024 21:38:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 213368D000B; Tue, 10 Dec 2024 16:38:15 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 199A68D0013; Tue, 10 Dec 2024 16:38:15 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F2CDA8D000B; Tue, 10 Dec 2024 16:38:14 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id CC3A16B0093 for ; Tue, 10 Dec 2024 16:38:14 -0500 (EST) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 8938D1C7BAA for ; Tue, 10 Dec 2024 21:38:14 +0000 (UTC) X-FDA: 82880362488.16.2E4B431 Received: from mail-qt1-f172.google.com (mail-qt1-f172.google.com [209.85.160.172]) by imf25.hostedemail.com (Postfix) with ESMTP id AD4B7A0002 for ; Tue, 10 Dec 2024 21:37:56 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=gourry.net header.s=google header.b=HjjotpAV; spf=pass (imf25.hostedemail.com: domain of gourry@gourry.net designates 209.85.160.172 as permitted sender) smtp.mailfrom=gourry@gourry.net; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733866671; a=rsa-sha256; cv=none; b=58ax8Q/foSN2EDCcD3rQBN1PFKlEOCgVDnu5rFpTzNKlL4TrYsUxP3TOWyM3T3GGKK0V5l 0Snq/A/vvjKHd8Bdmg5Wn2koo8qvC9mfjk4ftHJLTX+hTIVyusSyf9tMKEvX+JSLAPdCRk rQm8859ZUeqrSWueGFMunL+C0IKTxB4= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=gourry.net header.s=google header.b=HjjotpAV; spf=pass (imf25.hostedemail.com: domain of gourry@gourry.net designates 209.85.160.172 as permitted sender) smtp.mailfrom=gourry@gourry.net; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733866671; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=OQZEugtHND2zHmh8pPoKkZgS6JzSbLVFT2/QKBK/z+4=; b=lNqg9nLS4XKn94qrungWNTxXNVA437GgyhTxZVZgxtVpOfzBFzeD13JkP2U3x1CXccik0z LV32XyN/M1Tnfp9oxYGNH/k84kC7L4JCAma6rQmW0GeggS7aVR0S2KWS50i/poWImWPN5P IFYG2K5ppD8noNYZvR6UqwbJ4rRIz4M= Received: by mail-qt1-f172.google.com with SMTP id d75a77b69052e-4676e708aeaso18939911cf.1 for ; Tue, 10 Dec 2024 13:38:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1733866691; x=1734471491; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=OQZEugtHND2zHmh8pPoKkZgS6JzSbLVFT2/QKBK/z+4=; b=HjjotpAVeUovF2tqnmdzoAUnE9pr3jLZz73RyQGtlbnJa2oc6rYaJa7Fo8my2sJ4Of mKp6xcYNQVDYYmzIfKSXy9eGgY7dCq0RbfWhuN3kYRIlWlQBxXTFJzA7/NCy3gRUP49h wLuAo+cKMWa1mbbDvQKFzE6/REcxz9iYKAmxl1SNjd0NnXcSEk4aIrmHnIjLM3EuBZPR DtVbFexQHxt1GeC0jodOidpF76ydttwb40QM9bmUV3xbyGidAHLva1NmWyQ320OtRaJ9 v3xyovO/D1CEQwTF//vTJ8/UNZua1WVzihBry5EHSjE35pgcR2+NFVS2/vByu0flPKPt cgsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733866691; x=1734471491; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=OQZEugtHND2zHmh8pPoKkZgS6JzSbLVFT2/QKBK/z+4=; b=pf5YsyDMtScabxQYnNJ8/aP/pGr5Ua9TnYN7/shoPmBvHywhLZhyV6Y2WATkdSZbRK HrWD+C3OQuP5pKFoAWKUGKlpi8thhZSjq+N9EB63G+PYzyRB1KjJbDXe6KAyWdN7uvQJ AYYXTzpWTxET5Ej5TTCUGOEaNrLmNpnl1TWTXvzx22SZbbytWNrTF1jRNLqkP/omUY31 5jcyo9ib75c/wdy8ygj5oEElqHqNy8h+iL9/YkV9eI49VbEkV20Y+tk66ZH6RAVh2OmF b6CvjWfqldCSURqxjCLMt293X0YbjLdnWGUzI8R+K5ya/qiingRFYT7oFeKMJ3UL9Wdt IQuA== X-Gm-Message-State: AOJu0YyrB7vqsovWkdyGigRT32IyEad+8cSp/qCmLfADKZq+Si7ia+gQ coXnAuXZKP9ByB0NmGrTbm6y2me8qeZtw0rpb/uiJdQWXV9fFrXu/pSeltNTYTX2QQbYDyXHhwd u X-Gm-Gg: ASbGncsKvV7JiKJKuh40HqYG3Z39mYiaS6hTrtSJkXWIPPNzru/mv2/VXiwten+Yj6r YQsnXNu5S3r+MRszVe9QYQ0N6CACsXColLq9OYTmrLU4rpVLFoacSRNLU+f65zNNEn6NJP757Cd 3ZbuTZ7KrlFhw4pMJ0JTDsLsxLEp+ZmYkuk4UN+JWSLcGIQOilZEHQS+uUiyt/mjKzHD4pVTks5 VGQPFoMXSXhn6/E+TDjCdHq//+IwC+zrwyYMXXma7yD3E0N2iHdlzl6MTE4tDIFTN1mYD8U/SUR srBrbR8jFq+xvh3FboSwLtqle/6G6hNWQwnaT7o= X-Google-Smtp-Source: AGHT+IFYbR7aXy36yPVN9o7PP/Nyz3CEWrs6BuemC8qkn+p43Ye5cXgiGx1LfhBNIjaaOLXxBrl2Ig== X-Received: by 2002:ac8:5947:0:b0:466:9197:b503 with SMTP id d75a77b69052e-46789379413mr8570201cf.46.1733866691608; Tue, 10 Dec 2024 13:38:11 -0800 (PST) Received: from PC2K9PVX.TheFacebook.com (pool-173-79-56-208.washdc.fios.verizon.net. [173.79.56.208]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-467296f61f6sm65978991cf.43.2024.12.10.13.38.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Dec 2024 13:38:10 -0800 (PST) From: Gregory Price To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, nehagholkar@meta.com, abhishekd@meta.com, kernel-team@meta.com, david@redhat.com, nphamcs@gmail.com, gourry@gourry.net, akpm@linux-foundation.org, hannes@cmpxchg.org, kbusch@meta.com, ying.huang@linux.alibaba.com Subject: [RFC v2 PATCH 1/5] migrate: Allow migrate_misplaced_folio_prepare() to accept a NULL VMA. Date: Tue, 10 Dec 2024 16:37:40 -0500 Message-ID: <20241210213744.2968-2-gourry@gourry.net> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241210213744.2968-1-gourry@gourry.net> References: <20241210213744.2968-1-gourry@gourry.net> MIME-Version: 1.0 X-Rspamd-Queue-Id: AD4B7A0002 X-Stat-Signature: 3h1y7s5s7fzca8rxgt5mucr4a351354q X-Rspam-User: X-Rspamd-Server: rspam09 X-HE-Tag: 1733866676-212992 X-HE-Meta: U2FsdGVkX1/cF6FzuLfesVHsqGuNMJ9LK309sVTTpr+UdT1/rp2c3WtBXTtPpKKaVydQ3v6ERYg9CfBJSQuHsIH5/YMHNPP4vPfJ7pqz9FjJnQsYFTvvQAPFoXR2guTs5T8vBsRRG+jBBN79qAkt++H/JalydUyxnuKLEBY3Z4EWuI5ShM2GnekpsJDeCOiYxjWDbR21o52tNRMCn5t7pk15OHAujnXcF5fpugvcwadVyY09SlbqBuc47zIuzPzhKvLPuARMzGUxJibIzizunxoDUBEZ0NCGQggKUAARPlpTXrumq1GkK/wXnxszn/R1OGuIkzMny2znYzpfkHIwQZy0tlhxXuHwmQu8haYAalwTMssP5iZekMOxFxGWo7JZqViTxbdmvVsVXJWPdbnvgEP1KuK3TCiioq8CNJGWI42DmGrkRLvMNfbhK7N/r2TXr37OoKwA6i1LXV4TBQyEWV25RK/BAcEhyf91a/2ueSVGTNVQG/Xe0WynuV1FqNOomsHfnhU0N9ACwRipGZUQdCR0rOaFOoSaqtoC/hQZw9qKuEeqlr34sYrfeSryW2NZcMJ2YoH6PvB7S6Ac8qKI9wi4jIF4wrcQLuyPIgi8VkPJfEfEWVbXxjHUzBeiS4LxRjQ6oXWerkO4FlpPYJCArrY+xEKY64oZFf3Jfzy+8kZS+c+j+AHnQQI41FzFOjEsovTmHWQu19K65IDIoVKm3lQ7DFwZPhP61EXzKPu32qF18cC9+KZ6aDHn23lQXiKgQhWACrUP8MXbclSecuSF2PIl738r7HqYb7qgYAZovM/TN+3K0j9YjbjaQnhi8vOJTdWawGlKXI9qBKkI86gCk7RLCuRb1MxqaCgrvyPw0eyC8taUmdSWlf4WY7d18JU9Wymgjg6p/oKEY+xtqeS0/wGW0xoOBGsDbdIa8JNNeWgoRAhGabhDLUhyurUfaUQy4zlU0n5yA54duPKJW9d MqOznwzO dszrBQDidYlgUoz3OgVka8ppUDUdix7DW772YyHYydpG/DEJ3OSyOFJ23J5NMbuiI89yPesUSJ3LlXC/24+Emc4yOTY/Z4z7e7Qq4Siiow/2m9bQ4rQZLQDN1GSDCeeMVuPPjEtziChEeM91ICDTQuBIdG9D9MCHZ+aPfBCSgnr+lkbKCxJo4acFxHx8rBbCIWVlgAnWDCXR9fx0hz7QEJiE7fYsKvmlYY2nT403t6rW3vvyp+H1cv6jIm9ZZ0rEeZx8/I8V9yStn1M32TRwi6UgiSZri4z9YjZ12WfCh+l7ZBUo//E/ufmOnZr4kgUx9E6vimUj1avVBEPm/e30Sd02wJLlYW71IMgJmVMkxS0I5y4oLgjEMPq9bI8RCEvjXbEmzr+R+jxxL9llH3Gb3ZLoX7263JESzFZKy X-Bogosity: Ham, tests=bogofilter, spamicity=0.267604, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: migrate_misplaced_folio_prepare() may be called on a folio without a VMA, and so it must be made to accept a NULL VMA. Suggested-by: Johannes Weiner Signed-off-by: Gregory Price --- mm/migrate.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/migrate.c b/mm/migrate.c index e9e00d1d1d19..af07b399060b 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -2632,7 +2632,7 @@ int migrate_misplaced_folio_prepare(struct folio *folio, * See folio_likely_mapped_shared() on possible imprecision * when we cannot easily detect if a folio is shared. */ - if ((vma->vm_flags & VM_EXEC) && + if (vma && (vma->vm_flags & VM_EXEC) && folio_likely_mapped_shared(folio)) return -EACCES; From patchwork Tue Dec 10 21:37:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gregory Price X-Patchwork-Id: 13902103 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4E393E77182 for ; Tue, 10 Dec 2024 21:38:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CF2C48D0014; Tue, 10 Dec 2024 16:38:16 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C1F788D0013; Tue, 10 Dec 2024 16:38:16 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A9B018D0014; Tue, 10 Dec 2024 16:38:16 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 884B88D0013 for ; Tue, 10 Dec 2024 16:38:16 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 43E42A0F5A for ; Tue, 10 Dec 2024 21:38:16 +0000 (UTC) X-FDA: 82880362488.05.301BAC4 Received: from mail-qk1-f178.google.com (mail-qk1-f178.google.com [209.85.222.178]) by imf27.hostedemail.com (Postfix) with ESMTP id DBA414000D for ; Tue, 10 Dec 2024 21:37:48 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=gourry.net header.s=google header.b=JRhRRfTD; spf=pass (imf27.hostedemail.com: domain of gourry@gourry.net designates 209.85.222.178 as permitted sender) smtp.mailfrom=gourry@gourry.net; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733866670; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7SLO2zP3WSxmf0tbBTHWC3uKgLqzLgdzgwGcqHy1nF8=; b=WBHCEW7Akb5yI814VA6gw5BvW66Kj3xAjadzA5H+QxrA01TVl3czNTzBiIbwwcI8mWQdYv mdCp1SHEo53MfLuUe7TleLYOaO6MhBxj0j9/FMkNMEnpPYHr7wrBNSqqbqPxwlUFt2JfkV YBRhtA0xMitxldiYe1kBBfBwRP2st3Q= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=gourry.net header.s=google header.b=JRhRRfTD; spf=pass (imf27.hostedemail.com: domain of gourry@gourry.net designates 209.85.222.178 as permitted sender) smtp.mailfrom=gourry@gourry.net; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733866670; a=rsa-sha256; cv=none; b=sWF1sqQO3bsFnRCRJelI+a6Cr5VVeDxQW0dbVV2+GvlDtEU6qw7TctMcVMKrOe5ysZGqxS iPr2mwm2ezsgLAohL03VX0uLQt0xB0Y8zDVXkOaocg1vCJadr825Kx6ifpbG1uYY3+y1Vv mxnyDQIIV2TmhaYcv3BSVnUB7qtzlFo= Received: by mail-qk1-f178.google.com with SMTP id af79cd13be357-7b6e9db19c8so43419685a.3 for ; Tue, 10 Dec 2024 13:38:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1733866693; x=1734471493; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7SLO2zP3WSxmf0tbBTHWC3uKgLqzLgdzgwGcqHy1nF8=; b=JRhRRfTDfTpyyAq6F+YE+lGNZ9YNtyIyRB9asGJusL0E8u43CdDDyrbYFzDkI+Whpo /T8nDA7CN0Hm8SGlmtO9TmxiUcj/SmoXtDQVLsh5c172W++2jvQFz+vWKVJoS/s7Wj4u fmJrTNCQOErXET8/BSmkyqxP8TPc4pcEwhawiSRVociFfH9iHj4oQC4Mut24Ln88B1HC wAWlPGUQnAwTUcrayF7fJu9nT4A9DFo/a0eCeqqXpNteY4eAQIdwdgU8NUZsk4Pjl+zE 3CErrRVTvdxvqjLK27DXHSMe/ETDQ29LIhkYvOF8a855PFircHaxRQpoMA48RuMWUTR2 iMNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733866693; x=1734471493; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7SLO2zP3WSxmf0tbBTHWC3uKgLqzLgdzgwGcqHy1nF8=; b=W2jcbDd3jck9sjutjWFypCM1+xfs9YLgTAXHIx+GO1csOJ59ysEF6l7DjG/+g+XlQK oqCCVs4j5H97+IEfuOBLbSDMlh7tujckUaa4y8w8jotSkhI1UUW4foBaMnqLzFqmlHat YwxMUw2WLy/+X7jWfT9iAT4ibmO5Du3LfX6oPnmCt4InM7mcSte/B0y5+XdXhT5rRTL7 d3a7iwpU3DJGRHO9N4RtQA8BD4n31yfHa9+AbHW5TSQOU4idSdqFEkMDNFgchDcJi8iy uQ0V6Xp19QtxFWYDkvyx0XB8cyIKk3gNDhjyrO9RnTrFAXAY4plXOUfb2qbctpdVCYFS JGvw== X-Gm-Message-State: AOJu0YwR9q6c7oEmAn8RQ/7r9FvQ2wkaE+tPqIXgqeCVxuCy5TMCNvzV AC1+lAWdGhifizCLG4cVqHzCbb5fytZqbDew1K+VQI+vuO4xMPKhbmCppB8wHa7TZkkrOUGZord p X-Gm-Gg: ASbGncv/vKYLGQqW27eYvyV4CorGwkhqQ7NdOkAwu2OrtuXiPAChRIaioVXOst93vQ9 F5S2owJDbLpMrMXd6Fn2pZT4KZy7hHeY3KKqysdUt6A53CFLWtsY9Tf6LryHZB9SH+hTwrhfBIg 9aBLD6QlPVagMYXLn3BzVGjPnDlQ+HQSM1SH99mLdKf1zSKga+j3YGatVTqacvCx4PZC+MgXPyO GtZIHX3C0Up1ivK+/USINxfs8wbrUqcnpdC4jtnP4GXCzTq8HhCxkopNCNC9kz+O8B/0hhsM0On eSqMMyErhJfWMF45NSR8ZuxASCUT45ljuljyK90= X-Google-Smtp-Source: AGHT+IGR91VwyQ9Xd3L5CFowo5m6Bq9C98LuxwXRSibAvLZ+jNUO5jNdqjrNPT9txkkOjxePaZAJ4Q== X-Received: by 2002:a05:620a:4014:b0:7b6:d3b3:575e with SMTP id af79cd13be357-7b6eb4d1730mr126961285a.47.1733866693178; Tue, 10 Dec 2024 13:38:13 -0800 (PST) Received: from PC2K9PVX.TheFacebook.com (pool-173-79-56-208.washdc.fios.verizon.net. [173.79.56.208]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-467296f61f6sm65978991cf.43.2024.12.10.13.38.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Dec 2024 13:38:12 -0800 (PST) From: Gregory Price To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, nehagholkar@meta.com, abhishekd@meta.com, kernel-team@meta.com, david@redhat.com, nphamcs@gmail.com, gourry@gourry.net, akpm@linux-foundation.org, hannes@cmpxchg.org, kbusch@meta.com, ying.huang@linux.alibaba.com Subject: [RFC v2 PATCH 2/5] memory: move conditionally defined enums use inside ifdef tags Date: Tue, 10 Dec 2024 16:37:41 -0500 Message-ID: <20241210213744.2968-3-gourry@gourry.net> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241210213744.2968-1-gourry@gourry.net> References: <20241210213744.2968-1-gourry@gourry.net> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: DBA414000D X-Rspam-User: X-Stat-Signature: qd8j3nntjb8e5a383g4oscfjsac8zbej X-HE-Tag: 1733866668-987337 X-HE-Meta: U2FsdGVkX19HNvciWWZc/q3C3G0UPMUzS1LCA4ZwZ4HGDvv8uz7WYy2U9FadeAp7wEbUexhpWHg8rO+n4Hf2LkvGAgJtNxk6cXEwEHjhnz1Hdy9jCpBFop4YIihh4PEkkzOaSPwoiBu3Kq1LSe9RAWpND1dU6ULSaoFxrCZKCoQ7RwP5+NRSehCOcyxS4zcKHQjWslVuURu+z3ZjWQN7G8Cy9Gb4ZXmJc36SQTeTzG3ODZbxZIG6UKZLLhXWTUpwScVaYhTxrhJemQrypwvCHhULDb47NLFIclNiqaESnWEH78s7KCv945i8n96wFXI2MEmR9rjgQk0/jVLiVTdApAc+0I1gpdijcu6PAkurk6UNSDmZk70y8w8DY+YKtzY1Bi1A2F/YkNqWFEBpqT7xtObWa7PwFNimcBdAgytCqVyr9rzkrJeszqsdb5cEiK04K8cWHG/rgRyPHKkGk2lEHrQpQmR9J58HiGWJ2rIYISozYn34Hr9x9b0/sMhUhtoBJ1Om/+Mdmx4HfWAtY1uHGDBFqBhlPZ4RgCMlXKcrQ0H1FSuKlYd0RQibFvJwSx0CVBxZwdVEaTHViNHW3PJe6sghzG3mmf3tA04NBGfGK3Ew1byUDlIKuyQuzEsK5w750qvmRNd4C0SmoONMBy+8bchpiRi2/nFdxQT4z1P2E8r8Sfk/gnwSZjmZdaHGCgmoh5TZup7pWLSCXaZPbIou1b+1PD00/NdPNeLodvM5lhKmazQ2ibxBGzHXcHOPB6LWlXOMQggY2xT+1rw8DXPEdyRbNKfMWR7Rc5aXyE+OQIRGHD9q7MSiqIm87NJ1qje4Zd450nxKFBIOc7Avn0LiHMePrufm+RcAKVu1a5G/Xm8K83Q06D6im290knyl094So0MoIEgjUmh9+PCqQf1tJknh3uvpCjQsi4wK19V8Hqnrxa64SztRwkyaOgzJ1wau6dezZZwiDBB5YeQk+ed LplCuo94 v8keXdaMF49FeYwGX3RerOGRGMdGAIaBp2UrhBbrTmrlX7qzo93Ad41RylKgRJXvkCrn1A4C79P8F+aGrjGQ+xzIUYZUaPT10pCrCpoO97GHuMzDog3AmhPkXbnjw0xrHlJkWSuZEiT97phEFFJ27iI0LISk+nod6amKdFXT5fWw0nwapWmW7WJDWl347fOMTQioRuMyPhMX4phRR5JhR9rYtdgEFnn75+IlSTbyQzvKZKCpK2c2EsCS6uS6fm35Bv4OfaITbohn0OpyyWSAbzIBR26+rthdmgdMP1c7f1O7YX3DOviYi+vjrsg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.140560, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: NUMA_HINT_FAULTS and NUMA_HINT_FAULTS_LOCAL are only defined if CONFIG_NUMA_BALANCING is defined, but are used outside the tags in numa_migrate_check(). Fix this. TNF_SHARED is only used if CONFIG_NUMA_BALANCING is enabled, so moving this line inside the ifdef is also safe - despite use of TNF_* elsewhere in the function. TNF_* are not conditionally defined. Signed-off-by: Gregory Price --- mm/memory.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 83fd35c034d7..6ad7616918c4 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5573,14 +5573,14 @@ int numa_migrate_check(struct folio *folio, struct vm_fault *vmf, /* Record the current PID acceesing VMA */ vma_set_access_pid_bit(vma); - count_vm_numa_event(NUMA_HINT_FAULTS); #ifdef CONFIG_NUMA_BALANCING + count_vm_numa_event(NUMA_HINT_FAULTS); count_memcg_folio_events(folio, NUMA_HINT_FAULTS, 1); -#endif if (folio_nid(folio) == numa_node_id()) { count_vm_numa_event(NUMA_HINT_FAULTS_LOCAL); *flags |= TNF_FAULT_LOCAL; } +#endif return mpol_misplaced(folio, vmf, addr); } From patchwork Tue Dec 10 21:37:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gregory Price X-Patchwork-Id: 13902104 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8196AE7717F for ; Tue, 10 Dec 2024 21:38:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7410A8D0018; Tue, 10 Dec 2024 16:38:18 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6F0BD8D0013; Tue, 10 Dec 2024 16:38:18 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 56AA88D0018; Tue, 10 Dec 2024 16:38:18 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 3C6508D0013 for ; Tue, 10 Dec 2024 16:38:18 -0500 (EST) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id E5A4B120EDD for ; Tue, 10 Dec 2024 21:38:17 +0000 (UTC) X-FDA: 82880362194.13.FC0431B Received: from mail-qt1-f182.google.com (mail-qt1-f182.google.com [209.85.160.182]) by imf19.hostedemail.com (Postfix) with ESMTP id 6EC9B1A000B for ; Tue, 10 Dec 2024 21:37:52 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=gourry.net header.s=google header.b=IZqa84yf; spf=pass (imf19.hostedemail.com: domain of gourry@gourry.net designates 209.85.160.182 as permitted sender) smtp.mailfrom=gourry@gourry.net; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733866675; a=rsa-sha256; cv=none; b=DbBeonwPuObVV3La215GsmwJi7M9spVFQMCkjnUj9hg3bT3GE0+l7FJoVC5fpMSGMdChcU peS4iVLvcE3Ox7PFvcyEW1PgNT0iRmFrS0DU2ZVd8defqzlEsD60a9r5+A+OXut/NR8gXU 7oOEO+0KQFGszMNRLbjevWKqD6VnDRA= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=gourry.net header.s=google header.b=IZqa84yf; spf=pass (imf19.hostedemail.com: domain of gourry@gourry.net designates 209.85.160.182 as permitted sender) smtp.mailfrom=gourry@gourry.net; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733866675; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=rD13RBY30Rrvkd04V+Adk5FJ3Ta4BXdJd5R/HBJ/aEU=; b=Z/kZK5HYtbe+SFJ57mLTKURjk6Juc0wnQ3qGAexolC/30XOy+OG4DXzBPIOw4uso/c4kY+ houDN84CMxdRb4j7UudW2khdLhFaEsWHucowgpojUFo6UQ6PZRhDv7ANRZJ+e9SYqHFGTs eNOfBaMBwAZdzAmY6xVJQovHLcdt/N4= Received: by mail-qt1-f182.google.com with SMTP id d75a77b69052e-46783d44db0so8008511cf.1 for ; Tue, 10 Dec 2024 13:38:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1733866695; x=1734471495; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=rD13RBY30Rrvkd04V+Adk5FJ3Ta4BXdJd5R/HBJ/aEU=; b=IZqa84yfZID2Z1UDLqFVzDxqaGDN4FoPEK3qjZuisbL8zJBvkrwD/ot2B9UqQCpSjE Lc80S/t+/uUlVoGw6R5XRpi7ZVorpAblBAOKdEZcKKJfyvJCUiBWtUCD5zJe92cje7Kt PW0L2zURcHnKZknIjChdeLx3HyHyTUCtMk+5VunFdNDwwFu+ZG3baUszhal1Qxf0ajIu uZ2myilMvlqC8Re2yPjSoKa3JlsZHpXpiRxI5Gf+Sn19d0jbbO00/DXTKzElZ7cfHh2x rBAEEt2IQQFmqijBNjjnXuIEw/3RmtUloMg1Lu3EBC3dxe7NxWceLuqOqiAZDw/f3Vki Agzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733866695; x=1734471495; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rD13RBY30Rrvkd04V+Adk5FJ3Ta4BXdJd5R/HBJ/aEU=; b=YpG/6rQkiKjTjKel3IHyTyXVEE95EW58Zj/eE0msyuT/k/DsfridjCsbI77bSoeL1L 5gIgSzfjrcN4D1+u822z1xPOFLjJeTL655IoJsibvEvaaWpZbn3LnEN9Y3D67i+YzbkB FR6UbKu4WyLB6UEnd1xBP/XK7qBu0MrbhygZrJ7zfOTvaTF0j7F/AC5mIbv4g/uT+NZE NA+CdiQ+0EkYriU1wBbYet8NxySGAcy9JUx0wF2a+pRxPqNYtbjNFXBbrivEGSeFWfmF v4lXGWiuLb+elb1dMDn96B/PT/OzCyQwFVqUmGUuRhW7MNL0CFGBPtoOt3LGcGdeAtpr m0aQ== X-Gm-Message-State: AOJu0Yy3zGa2+JrOoQ72cYFD+K6fI2733FT0GfwcTh7hGQzeIDeLFkqb AajTNsNehhVDnW7tb58iJ7QCujdIL50knq4tHFWx6yNjVRflW/xjOzrrIpb38muZ5oXB7dw4vp1 v X-Gm-Gg: ASbGncuzuVercFfjxl2PyAoQ7S7V7eOadIHQ1vWaAUdXY76QRnXdNsAwGKwmPdg4wfi ZH61hLn/TdD941ajWcdtKxWyXzJY+2onRIOVxJoqjARqPhgCTMkTlrcSrKfzeU4N6mapaHW7jom Np7yIyinVyfMOo03oFp8nWC/u8ZbQt0vnvuGPKVLbOGDq0BzX4ydnPUSzfrKU5wTaMwZkk6D/BY qtwB32ORsG/I7vu7Yp8AtjZsHEr+mHgAlrXQd7zdl+By41DeZxrrEspe85wnfPpgtSIWQ5+Nisu f8Lg0GrYcWXLLALN/6VygTRcNItRSRw/d4T70FI= X-Google-Smtp-Source: AGHT+IFsnNfg3615eI3lmfgEmFA2vm6QMKjB1yR8PQ1ci2ihqRLidFlxPlnZADQfcm/FjEoxGxlntQ== X-Received: by 2002:a05:622a:15c8:b0:466:77d0:5941 with SMTP id d75a77b69052e-4678929f162mr8994041cf.10.1733866695082; Tue, 10 Dec 2024 13:38:15 -0800 (PST) Received: from PC2K9PVX.TheFacebook.com (pool-173-79-56-208.washdc.fios.verizon.net. [173.79.56.208]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-467296f61f6sm65978991cf.43.2024.12.10.13.38.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Dec 2024 13:38:14 -0800 (PST) From: Gregory Price To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, nehagholkar@meta.com, abhishekd@meta.com, kernel-team@meta.com, david@redhat.com, nphamcs@gmail.com, gourry@gourry.net, akpm@linux-foundation.org, hannes@cmpxchg.org, kbusch@meta.com, ying.huang@linux.alibaba.com Subject: [RFC v2 PATCH 3/5] memory: allow non-fault migration in numa_migrate_check path Date: Tue, 10 Dec 2024 16:37:42 -0500 Message-ID: <20241210213744.2968-4-gourry@gourry.net> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241210213744.2968-1-gourry@gourry.net> References: <20241210213744.2968-1-gourry@gourry.net> MIME-Version: 1.0 X-Rspamd-Queue-Id: 6EC9B1A000B X-Stat-Signature: acwjb63g9iirpnwiyxnpcrq9hb5aj1mj X-Rspam-User: X-Rspamd-Server: rspam09 X-HE-Tag: 1733866672-682989 X-HE-Meta: U2FsdGVkX19sIgQLo90flET+3ok1X8uLP5LxHwRE0uyE2Q26umWYSw8kgcIiLDA4MnUQrhMtBYv1D+dpfDkN9oeJn1za20hP0EXzhsTMJv8O6G3DFpB3fqQdDGlQsjW/NhOgmyeEntBTIDPapdyghEFANKNIO1hOp3mQQXYOCd3BomBP6sF8gF3Uj3OjxfPr7wVDVlEqj3byAtprW9y72xbkC7Z56ie2kFMaNG0WalwvnirCN3gBefguft/Mv7ldvW1LRKkvo09rPadgvBGiy3NP1AXr4wnhC2fJG6b4cGJrCmjuxw84C3NAHx5m+nazeiuk1GJVJEnRN8r7dHEKcStSyWm1o4Ny5SiGpbZ8TpUvjF5DPyBEA2/TfHPc4jGudjqnP+xbQ3HxDlf+vwbO/X6YY+3pN2IvK83j4PlleUQMdYs7ia6/aXQfe6kAClPt0N51rrjbYf5I7eb5SUorfzd2BEwQ3fNZkruqsefB8ScUWI64eTcMaSWWuTtX3r4/x/bC0W+DjQxeSS652/KnJ36su4GODL5LnXgQHLJrVcdydA/F2go+0tCPcJa5P5aFRY7E7UVQ4+EfVBGOaSnsJua202+jFXpgj4X0x269V6WbwdePdplXPz3+B6dXOu/7lEVDoplx0z/TmWjkmo1TlAznumtJucHVKpIHdNpnHboxToES7XlqgoTWN+XNZhm+dV/ZeEnj9NvppIKBNMquDqjEERhyBERx04DWdBBBc6reiQ7Z0Kb30y1xEOinGkb4Sfo/8PQisixGCDKyCk5jDKNaHXUttqtNVfp5P36jg5cHmGWrePeQ6lHkVhWW6pfJzzTMzRsQvigIt3ZhRKHYClQnZP6HqTEskUcYj1NZSxJcV5OC4EM7rcaFXH/W4vhWHFET3jfnaG7gw67umxSUixh3494DpSsxPOwCexJMjjG5IigEV1fHF63l9vO0z3ilGbDqheiuGU+ukQ06nm/ 71IMpRbO mZKP6R2xugAqGQoOZLfc8kQIkmLbM2fhLEMzHXnUWG7pSlRW27Xyt79H/fMj3XjGBclkbsZvMMGy+lvryX6drD5i8DDVQaApKzDLv0U7vZ/dJY7fVHdh2BxWejjA/b4w1wzkIMY6oTgj+OoPi6ugRBkeAMiAAcJb3DnIzaidvfHtU05cgpbrmRP1CtnuHtgxhHOXplEfhj399coxiWfAyOnU7htFoClfbuZ/z9bAcU3dz+/AL0qL6gEpNVjhltmYQ2TVBvYcF4Wpiv+4geMp18fMSdiQW7+rzjhatzZHN7A2HZC0G/HYWshW2szayAVRZkD0bgUST81XYcBAZr1LFetNIy939TUNcehvu1mKgdOaigkE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000054, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: numa_migrate_check and mpol_misplaced presume callers are in the fault path with access to a VMA. To enable migrations from page cache, re-using the same logic to handle migration prep is preferable. Mildly refactor numa_migrate_check and mpol_misplaced so that they may be called with (vmf = NULL) from non-faulting paths. Signed-off-by: Gregory Price --- mm/memory.c | 24 ++++++++++++++---------- mm/mempolicy.c | 25 +++++++++++++++++-------- 2 files changed, 31 insertions(+), 18 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 6ad7616918c4..af7ba56a4e1e 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5542,7 +5542,20 @@ int numa_migrate_check(struct folio *folio, struct vm_fault *vmf, unsigned long addr, int *flags, bool writable, int *last_cpupid) { - struct vm_area_struct *vma = vmf->vma; + if (vmf) { + struct vm_area_struct *vma = vmf->vma; + const vm_flags_t vmflags = vma->vm_flags; + + /* + * Flag if the folio is shared between multiple address spaces. + * This used later when determining whether to group tasks. + */ + if (folio_likely_mapped_shared(folio)) + *flags |= vmflags & VM_SHARED ? TNF_SHARED : 0; + + /* Record the current PID acceesing VMA */ + vma_set_access_pid_bit(vma); + } /* * Avoid grouping on RO pages in general. RO pages shouldn't hurt as @@ -5555,12 +5568,6 @@ int numa_migrate_check(struct folio *folio, struct vm_fault *vmf, if (!writable) *flags |= TNF_NO_GROUP; - /* - * Flag if the folio is shared between multiple address spaces. This - * is later used when determining whether to group tasks together - */ - if (folio_likely_mapped_shared(folio) && (vma->vm_flags & VM_SHARED)) - *flags |= TNF_SHARED; /* * For memory tiering mode, cpupid of slow memory page is used * to record page access time. So use default value. @@ -5570,9 +5577,6 @@ int numa_migrate_check(struct folio *folio, struct vm_fault *vmf, else *last_cpupid = folio_last_cpupid(folio); - /* Record the current PID acceesing VMA */ - vma_set_access_pid_bit(vma); - #ifdef CONFIG_NUMA_BALANCING count_vm_numa_event(NUMA_HINT_FAULTS); count_memcg_folio_events(folio, NUMA_HINT_FAULTS, 1); diff --git a/mm/mempolicy.c b/mm/mempolicy.c index 88eef9776bb0..77a123fa71b0 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -2746,12 +2746,16 @@ static void sp_free(struct sp_node *n) * mpol_misplaced - check whether current folio node is valid in policy * * @folio: folio to be checked - * @vmf: structure describing the fault + * @vmf: structure describing the fault (NULL if called outside fault path) * @addr: virtual address in @vma for shared policy lookup and interleave policy + * Ignored if vmf is NULL. * * Lookup current policy node id for vma,addr and "compare to" folio's - * node id. Policy determination "mimics" alloc_page_vma(). - * Called from fault path where we know the vma and faulting address. + * node id - or task's policy node id if vmf is NULL. Policy determination + * "mimics" alloc_page_vma(). + * + * vmf must be non-NULL if called from fault path where we know the vma and + * faulting address. The PTL must be held by caller if vmf is not NULL. * * Return: NUMA_NO_NODE if the page is in a node that is valid for this * policy, or a suitable node ID to allocate a replacement folio from. @@ -2763,7 +2767,6 @@ int mpol_misplaced(struct folio *folio, struct vm_fault *vmf, pgoff_t ilx; struct zoneref *z; int curnid = folio_nid(folio); - struct vm_area_struct *vma = vmf->vma; int thiscpu = raw_smp_processor_id(); int thisnid = numa_node_id(); int polnid = NUMA_NO_NODE; @@ -2773,18 +2776,24 @@ int mpol_misplaced(struct folio *folio, struct vm_fault *vmf, * Make sure ptl is held so that we don't preempt and we * have a stable smp processor id */ - lockdep_assert_held(vmf->ptl); - pol = get_vma_policy(vma, addr, folio_order(folio), &ilx); + if (vmf) { + lockdep_assert_held(vmf->ptl); + pol = get_vma_policy(vmf->vma, addr, folio_order(folio), &ilx); + } else { + pol = get_task_policy(current); + } if (!(pol->flags & MPOL_F_MOF)) goto out; switch (pol->mode) { case MPOL_INTERLEAVE: - polnid = interleave_nid(pol, ilx); + polnid = vmf ? interleave_nid(pol, ilx) : + interleave_nodes(pol); break; case MPOL_WEIGHTED_INTERLEAVE: - polnid = weighted_interleave_nid(pol, ilx); + polnid = vmf ? weighted_interleave_nid(pol, ilx) : + weighted_interleave_nodes(pol); break; case MPOL_PREFERRED: From patchwork Tue Dec 10 21:37:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gregory Price X-Patchwork-Id: 13902105 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3143EE77180 for ; Tue, 10 Dec 2024 21:38:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D35DF8D0019; Tue, 10 Dec 2024 16:38:19 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CBB608D0013; Tue, 10 Dec 2024 16:38:19 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B0F5B8D0019; Tue, 10 Dec 2024 16:38:19 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 91BDE8D0013 for ; Tue, 10 Dec 2024 16:38:19 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 465941605A6 for ; Tue, 10 Dec 2024 21:38:19 +0000 (UTC) X-FDA: 82880362404.10.F156144 Received: from mail-qt1-f176.google.com (mail-qt1-f176.google.com [209.85.160.176]) by imf30.hostedemail.com (Postfix) with ESMTP id ED2E38000D for ; Tue, 10 Dec 2024 21:37:34 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=gourry.net header.s=google header.b="vBSx1/iA"; spf=pass (imf30.hostedemail.com: domain of gourry@gourry.net designates 209.85.160.176 as permitted sender) smtp.mailfrom=gourry@gourry.net; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733866674; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=3Jq0f3dJTpHJXJDokS6aQHyPKu/LFNDC+TZCLMRgrQ0=; b=2Y2Hqa16R0W+sG85KlylaBOW670IQkS1wZMS+R31eLkyMYNfbJrV3Tz+flghVteCHlJRkC OwBde8Gniu/4bFd/gFAnz98TEf9xSZztqZGdtBFvgSQS9IvFeNcOFOxBfhwd6n7bLXP4s8 7KDE696IMH7tvP5CBF4VhJ36X/zBAZM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733866674; a=rsa-sha256; cv=none; b=ySYxKCXzSKovy3aKU3g2I7aWM40LLDEEHMJSx0Q0lFR/KzlLUoAEMdpiO5B+GlysjUYTKC oerfUO8KzJS5CkfWE6OFTQXGkqjd4c0JKDkvAUeDQ/Qkj5qImD+HmSrTGRmPaD74Nuc0oa pG2/EmHGw8at7lD7gEPages2n/Jl1kY= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=gourry.net header.s=google header.b="vBSx1/iA"; spf=pass (imf30.hostedemail.com: domain of gourry@gourry.net designates 209.85.160.176 as permitted sender) smtp.mailfrom=gourry@gourry.net; dmarc=none Received: by mail-qt1-f176.google.com with SMTP id d75a77b69052e-46761d91f7aso25975701cf.1 for ; Tue, 10 Dec 2024 13:38:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1733866696; x=1734471496; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3Jq0f3dJTpHJXJDokS6aQHyPKu/LFNDC+TZCLMRgrQ0=; b=vBSx1/iACy0TeuUBs9Un2DpkChyr3AAu41rJ5cADld2sdKpBzF/HQ2XPPL3c8/FELq f1YlO2R0bA8KVWll0ntX4mRlX2cS7DICeQAG4GifgFGuYm/xh3vQBzQq5rojXqLWqwXx cbWtdrXQLsP/aanVTXY4FvGBKNU5PqKiSSKKXLZ69b9ly+tfSGPJYfGy9s3QIrPs0mQL jl20shk5bUeLJsUUYMxcRWNUnUNAyoPUa2AW89DRJ4o/e5qAh5FMJjtBMeiF0IIHpZ3E wjgOeQIIf6TD/WeJIkb+SIOt5WrfUOZcX+f5QHc0UBCZJRm1Q7lSDD+X07nUVuuknumb cL6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733866696; x=1734471496; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3Jq0f3dJTpHJXJDokS6aQHyPKu/LFNDC+TZCLMRgrQ0=; b=J67aCah+eZrjLiux8M9S7H4i4Yr5KNt2rj545GdxSdse+c+7HPtAQQpizpitkTsIxr iYbjLmr+O0unSa4Srhvth7puOY0Tx/oQMCJA8j19OZWIzUM9/Aisct390V39vxl99qNr BrBf9egSh4eSldyIqLkA973l/DOEFfB0PV4LjGxb8tVIvPvt7Ltnh0a3kXhngSXLbQeW dDkSHuMtG7jmikGLi+hG7Pyy1Xg2a58TtyAJiDyDY9prwmMgqv3qLORPC2yGVhsjwUMl cn+5xGoEtmspr/tJUd0nHVtKGRVWxYKcTxVSA5/z+Da/WAheQXnreKwx20fjHq52+GcK 0Adg== X-Gm-Message-State: AOJu0Yxh3s0kC4Eys9nhtMRpglMb8JQzLaA1xY18YXLwSETq3G/WNRXo nxFZ/LM6R6z8rD/w6G9Sx1UIhS7iRewHNWfhPwoP/55PrInSrLU+lGX3F08sMJBl99eMOi1czMT H X-Gm-Gg: ASbGncv1K6UE3r0Ux4/QjhCz817QTqtlVPgvdu20UZbvHyNF4TAUfYK6LX25DR88CFK DEkcEJl85udFys2PLC4aFu0vbYlIXB1YgO/PK5b3fvGY50P28ZuXgcfHcbnf1jorO8ZgKOO3V/U 1e+wuk1wH+CQXiEpsz7QnGj5QBB8Dsu/4EUsRlMhn0qxG1Zpt1mBGOUKwdKd/7UnXJMQKCUPz2D eVy8fx/esiwHpY9UNBoCWhCdlHR+4ZpWnhV5JYFzTMoek+SgU265O7qShDBPcsu2QK31y45r44N QBmKVS+p94GKSSiBe0BJNUPKbWvHrEgOMyJFolY= X-Google-Smtp-Source: AGHT+IHxInG7lq5H3xVJww7rlvFoTxqlg+CA0d/WG83+KpCA77jzrX+fsIGQc0RjdUSsv/+VccPkKg== X-Received: by 2002:a05:622a:4d45:b0:466:a584:69f8 with SMTP id d75a77b69052e-467893bcc14mr7050891cf.43.1733866696431; Tue, 10 Dec 2024 13:38:16 -0800 (PST) Received: from PC2K9PVX.TheFacebook.com (pool-173-79-56-208.washdc.fios.verizon.net. [173.79.56.208]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-467296f61f6sm65978991cf.43.2024.12.10.13.38.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Dec 2024 13:38:16 -0800 (PST) From: Gregory Price To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, nehagholkar@meta.com, abhishekd@meta.com, kernel-team@meta.com, david@redhat.com, nphamcs@gmail.com, gourry@gourry.net, akpm@linux-foundation.org, hannes@cmpxchg.org, kbusch@meta.com, ying.huang@linux.alibaba.com Subject: [RFC v2 PATCH 4/5] vmstat: add page-cache numa hints Date: Tue, 10 Dec 2024 16:37:43 -0500 Message-ID: <20241210213744.2968-5-gourry@gourry.net> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241210213744.2968-1-gourry@gourry.net> References: <20241210213744.2968-1-gourry@gourry.net> MIME-Version: 1.0 X-Rspamd-Queue-Id: ED2E38000D X-Rspam-User: X-Rspamd-Server: rspam07 X-Stat-Signature: dcfhk6fdphe3knk3fmwddwr9xe9qtn3h X-HE-Tag: 1733866654-825547 X-HE-Meta: U2FsdGVkX1/GpwCM5fYZGlDcugkZ+B7eN3L697OoIbPmoCt2Lib/5EQf5d4Sb1M8AIn/V0nxzEcVAA+ibIG0/9MqVEelPNDo+g3rMU554kPflH1thVYpZqJkNGhKhAvR+X/haJj9AvzyaqXui17iakcIkgPwq3uMgZq2pNzqdxzj3efX+xWMXlhz9skYb1neqx8gwXO+65++TPWidmZKpuQByCtaU73VAADi8kqNpFnRl8rjYGnIg3AVAwHwaUOm7DS8r1PbOcf21sNwBzQDuLcROP0kdjeGCKjY2TrkWhEhimiZs8rKH7PDtwt4H+MzVLJhGOqw71oQ3yD0qDZ08UiPxgQrr5t4RB2BjuZrFYeKTYwMRMR9qRRAPYf73k1tyCaJV4i+u42l+HxSblC08mDIMT8JQpZw1Y0Dl5YEkp7p2y64IWczedn5rxoAkYCDM/usbIM1TwoKrEe+IyhG2HTi6vtGt7dUuMLgIzHHbAYKh9jKh53do7wNSeI0uZhdoOj4LJKYbBK+B0gYK0YdvzQURPVAAoDHdpgT3tDEnU7GGYXAj60IK0/NGpEpj+sAznPEQL9PRphNetsvmpzIyoT3v0uIfmwMNnaPq3z0aOly3YyPIbkd02s0tORxcOJ149lUjZ0h3ul6uDVxGMxDQZOWxofTjRoMaIHOurJseCK6nzgeJKohOdycgT5L+dZsAWcshVPJMTgUO2ID4ykHltYjrhtvFLTAMSGaVx3pyGlGbzj7rbuX7do2LG/yZtcZD6GL0+gHPFvkTPxJg+UkACRo33Mvup1bhQyI2PlUvtNtkPfrfqrOzeTDPqlQdVW7NEpk5b1Hf4XVvoVEOwBctTBbfBjhneBAt7BRntM8IJbOvBBjQYX7syONM1DzY1cXxYmyJeGtKVy66Im6GCGNKKvxJdSZ1fy7BOzzh5ZTp/eDCG4QripO3B2DNfQUKVvangt5/KARt46CKnyJJsu /+ONHn5B fXAsWKH21kde8cHrYxMMMwzotrPi0vcSldTU3BhXTWUYiKWaFk5B437B60LIZlj3+sK7EqLv6/aNOXxlQVL2T1O8FKg4Ikd0EqY89QCh5cGv/wILGNyKCs6/qBPQ20omCPvaryNdvdytE3u/EXuIaZAU+ZOzZnLNZm+drM40jECPbxyi6pSkEALXXueOLdjM6JrTxuLTY24rhILIZxjR4YeguN9UshdxZKtKb3KwxovMY0TgVZbQxtyEO1oEHOjW9+UUIEp64avBYAr0UmjAd5FUF1KgHMRXYmZANkPksFWmmeI11dDqDZBVmgQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000021, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Count non-page-fault events as page-cache numa hints instead of fault hints in vmstat. Add a define to select the hint type to keep the code clean. Signed-off-by: Gregory Price --- include/linux/vm_event_item.h | 8 ++++++++ mm/memory.c | 6 +++--- mm/vmstat.c | 2 ++ 3 files changed, 13 insertions(+), 3 deletions(-) diff --git a/include/linux/vm_event_item.h b/include/linux/vm_event_item.h index f70d0958095c..c5abb0f7cca7 100644 --- a/include/linux/vm_event_item.h +++ b/include/linux/vm_event_item.h @@ -63,6 +63,8 @@ enum vm_event_item { PGPGIN, PGPGOUT, PSWPIN, PSWPOUT, NUMA_HUGE_PTE_UPDATES, NUMA_HINT_FAULTS, NUMA_HINT_FAULTS_LOCAL, + NUMA_HINT_PAGE_CACHE, + NUMA_HINT_PAGE_CACHE_LOCAL, NUMA_PAGE_MIGRATE, #endif #ifdef CONFIG_MIGRATION @@ -185,6 +187,12 @@ enum vm_event_item { PGPGIN, PGPGOUT, PSWPIN, PSWPOUT, NR_VM_EVENT_ITEMS }; +#ifdef CONFIG_NUMA_BALANCING +#define NUMA_HINT_TYPE(vmf) (vmf ? NUMA_HINT_FAULTS : NUMA_HINT_PAGE_CACHE) +#define NUMA_HINT_TYPE_LOCAL(vmf) (vmf ? NUMA_HINT_FAULTS_LOCAL : \ + NUMA_HINT_PAGE_CACHE_LOCAL) +#endif + #ifndef CONFIG_TRANSPARENT_HUGEPAGE #define THP_FILE_ALLOC ({ BUILD_BUG(); 0; }) #define THP_FILE_FALLBACK ({ BUILD_BUG(); 0; }) diff --git a/mm/memory.c b/mm/memory.c index af7ba56a4e1e..620e2045af7b 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5578,10 +5578,10 @@ int numa_migrate_check(struct folio *folio, struct vm_fault *vmf, *last_cpupid = folio_last_cpupid(folio); #ifdef CONFIG_NUMA_BALANCING - count_vm_numa_event(NUMA_HINT_FAULTS); - count_memcg_folio_events(folio, NUMA_HINT_FAULTS, 1); + count_vm_numa_event(NUMA_HINT_TYPE(vmf)); + count_memcg_folio_events(folio, NUMA_HINT_TYPE(vmf), 1); if (folio_nid(folio) == numa_node_id()) { - count_vm_numa_event(NUMA_HINT_FAULTS_LOCAL); + count_vm_numa_event(NUMA_HINT_TYPE_LOCAL(vmf)); *flags |= TNF_FAULT_LOCAL; } #endif diff --git a/mm/vmstat.c b/mm/vmstat.c index 4d016314a56c..bcd9be11e957 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -1338,6 +1338,8 @@ const char * const vmstat_text[] = { "numa_huge_pte_updates", "numa_hint_faults", "numa_hint_faults_local", + "numa_hint_page_cache", + "numa_hint_page_cache_local", "numa_pages_migrated", #endif #ifdef CONFIG_MIGRATION From patchwork Tue Dec 10 21:37:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gregory Price X-Patchwork-Id: 13902106 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AFCBEE77180 for ; Tue, 10 Dec 2024 21:38:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7DE308D001A; Tue, 10 Dec 2024 16:38:21 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 717BC8D0013; Tue, 10 Dec 2024 16:38:21 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 543938D001A; Tue, 10 Dec 2024 16:38:21 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 2F6348D0013 for ; Tue, 10 Dec 2024 16:38:21 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id DCE82C0EB5 for ; Tue, 10 Dec 2024 21:38:20 +0000 (UTC) X-FDA: 82880362950.29.CBB6CED Received: from mail-qt1-f174.google.com (mail-qt1-f174.google.com [209.85.160.174]) by imf29.hostedemail.com (Postfix) with ESMTP id C4A63120016 for ; Tue, 10 Dec 2024 21:37:47 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=gourry.net header.s=google header.b=VaUX+Zyd; dmarc=none; spf=pass (imf29.hostedemail.com: domain of gourry@gourry.net designates 209.85.160.174 as permitted sender) smtp.mailfrom=gourry@gourry.net ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733866689; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=+9y/Pmi/E+qcbGw7qvBfkROEFTzGsGqs1ClRtYT08n0=; b=MTlSCNJderHz/Ja2jMyv5oEkxcf69gifMtym9YKNjRJuS0uEzbHKPInm4R5xN/za99zhnH YJzvPXSjcjRyP8vA/PHWoMIk19/N5QPUoF6t6KwWWw6VGR5KNJn4G29BrA+N+VUxPWY6EU saUI/tiTkDAPC66ClHZDlguiKjasViA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733866689; a=rsa-sha256; cv=none; b=ahI646KdF6XI9IvfYB/osQ9+ZuuIwZo8yhH85NElhpun7EIrrvED+SHMzlfS5Bza8eRoDr BGI3/HGjNA5y69jyPSpXjNQa63ZFUf2m6S6KEnTgvHyITfIFUMR/i4zXjQHfbNz8Xa0Mmd G/9B+h0c0lrzh51Un1pgOYx1EQ5aMXA= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=gourry.net header.s=google header.b=VaUX+Zyd; dmarc=none; spf=pass (imf29.hostedemail.com: domain of gourry@gourry.net designates 209.85.160.174 as permitted sender) smtp.mailfrom=gourry@gourry.net Received: by mail-qt1-f174.google.com with SMTP id d75a77b69052e-46785fbb949so7815461cf.3 for ; Tue, 10 Dec 2024 13:38:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1733866698; x=1734471498; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=+9y/Pmi/E+qcbGw7qvBfkROEFTzGsGqs1ClRtYT08n0=; b=VaUX+Zydk6lw1PBhjXbCpmJaBRn75ywZxSqivoyI+cJrtaHJVvtO6jT/jZuUlhxrBA 5aTtzpdBrFGHTOu8Oos16rGDb1MRMQqQM2+YAeRQRytL+xCuelKdbW0BUedNX+kUZJjc SvSQHQDWXYhzuB0bSwgqEbt6aX87WqWv6jy7bmSHrfG0vgWkZGRU4d48MxUW+lpo+8+Z 7/qVyCqiRgP8cBIrzbtzLpUb92c8M6EFfiMS+VUHjuxiBxCDyyhFUo/bXJ/QEGNhQs3D tIZuW+uKGUn5ToZx5IBRUwKCsYOMcL7lo/Q1XAHexXnGABkitqZNjEMr7tn1IN1FSyVq sPkA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733866698; x=1734471498; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=+9y/Pmi/E+qcbGw7qvBfkROEFTzGsGqs1ClRtYT08n0=; b=hJjYUNlE01CVT/lSRurqI+uNFFEsznkUcD4awn+FpeP9Ykhhl7mZDPDat0lUjCtbYA +z3S6unmRMdadN+k3gGhdO8pvx58cI6/+0dogkhJzq0ZtQqwcyBVEJEQxK4vWn7Jjnyu Zqhk0gt91k0y6V3prNeg6caRGaODPJCCDDb+VEtxw2Q+GKuCvR57u5dQWeUKakjnzhJW 9pHtBMRJOh+FcxpaRj8OhB/EZSdJdBeU/M3zh8vZgODp0gV8S/NahqMy46uypldlKhw1 t+vaBN8/fcoTGi4Koz8LW0oySHPTYz66zWroHPHYhzfEib1Sd1oFQD+xDAJMILtJXUi0 mlxQ== X-Gm-Message-State: AOJu0YywKWAREbQ72kdS7LZDYrLmDmnbfgRPcW2gWFNnudVO6ANwvsPJ 3MdCcoVtu3qOc05V3ZyYOp4VZK7wI9lpqahiXAv/conRTWpm0n905CoYaYnEmeWkgwYacvIV9pk y X-Gm-Gg: ASbGnctcE93K8EijjcCXGDMLOydPZ6ExefWtT7hWayu9ug0G353T6DC0ig4vtaHG3o5 UZuq1vMxuTkvxtiKK2s8uocaP3X2f9r3Sja8s5A01dKePVvE+WB1lFrGQ5JOREfEcsC/OspJACb 9gleIg2T36Iq12FeAka8MUq+Z9bDdGaZNYSZzaj0EItooTQduytqa5xCwAgciIBgx54HhiGp3ut sV2PY7aeK6KFmk63Zu5JJBHNy3lTRV8U/QzmfdlctOUQ6JB2a/PvXYEY3Ly/RDgdR3bVQ1Hkz2s tHCMWo3irewNNOvWKeS1ALCrnTMvKvHAQ6ZDb+k= X-Google-Smtp-Source: AGHT+IFxNZ06/2veak16EjkYa8clNKlgcdi7wLb2OYdhSxCOFJs32hkfuiKlgIR8eXNwOyd59R311Q== X-Received: by 2002:a05:622a:1815:b0:467:7208:8eeb with SMTP id d75a77b69052e-467893769a8mr9097551cf.37.1733866697811; Tue, 10 Dec 2024 13:38:17 -0800 (PST) Received: from PC2K9PVX.TheFacebook.com (pool-173-79-56-208.washdc.fios.verizon.net. [173.79.56.208]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-467296f61f6sm65978991cf.43.2024.12.10.13.38.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Dec 2024 13:38:17 -0800 (PST) From: Gregory Price To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, nehagholkar@meta.com, abhishekd@meta.com, kernel-team@meta.com, david@redhat.com, nphamcs@gmail.com, gourry@gourry.net, akpm@linux-foundation.org, hannes@cmpxchg.org, kbusch@meta.com, ying.huang@linux.alibaba.com Subject: [RFC v2 PATCH 5/5] migrate,sysfs: add pagecache promotion Date: Tue, 10 Dec 2024 16:37:44 -0500 Message-ID: <20241210213744.2968-6-gourry@gourry.net> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241210213744.2968-1-gourry@gourry.net> References: <20241210213744.2968-1-gourry@gourry.net> MIME-Version: 1.0 X-Stat-Signature: 7f14wfa839jtww7cqe5aq7uzn3y5aksc X-Rspamd-Queue-Id: C4A63120016 X-Rspam-User: X-Rspamd-Server: rspam01 X-HE-Tag: 1733866667-888689 X-HE-Meta: U2FsdGVkX1/uINwD4ARUtCM1/B4F+cpV8atxXwaTuctIUseLZVLGMNqIUiCOW5+ukxTGYOCcgauOghPjyR4zKIobVGOsykojrvqDvMdHDcpQP6x5qITkm7cSOB9J9BYkFnmixnjGe3bDAjFidXSWgRWeyGyGnzXYfHZf5lnJxBxSzFps7jRK56HDT/HYPk730bnrNfYqYTM43G5AU6NWoJvD5hFrq5Ydg3Ef6wIGa1uehApzbFVzsAFGv2GhdfSQvfLaAdWLl+enuJ5HNFSNYnRn5tz1ZV6wn8ladPSQ26uKmmHV0GYzMXOJs6jPDtHHLij9htbq6Iw/pUBm3bItZkWFABS3DmrPt33moVD4BDVXagMs5gyYIXmvxf5foI1BteDwBbcm8UzAnUUht/fMaohiaQfr3y13k/pFmKvwZlNNSYYLMfLLpIbziOXuafOwTN69UFtMECM8gyQuK5KXI4VXD7sszU2cE6l6o3hRrvAhvH3l3UVNky4+/u4ezvXzNHrowK26fN2Z5ArY71PL8KiPkoO5iaKbu9XJCx4kP5qS3Q58jp1CTrrthEUjiReYPYhknd0RsHQD75EKUgWf14JrHPMN4ju3AzPnB3VbT4lsdSZ6yuzTPzxiDYdiwIEujwqDoqk/AeIJtIbDboxTiHw4CrenOh3W7eQgFD3rqkAKKvI3xVu6u2TMczRqs1UQ14mHpEeh7Eoggq+lW1Dn7y2utmtjBDT7ly6GG3cLqsLm1nPVdwyG5TlEL6cA2hY9L2zHiKDtpq1qv20hcvE35gfDYlbr6RI0to9/bJq5at6a5HMV9F/80iV4pk9BKaNgbP1ozgqfvFyXHEkAVnsAAOwZInBz2c5hPaZ28hsLXCGAfgX118bvuN33aHTGIXJoAWrcJY2ONFJI7m+rmnoKSz2BIj9RqeIqbE8ypscZ1SkmszIWcM/twtXmBH6nov0C1adqaQyGD+06KlnYwPl jUzEHpJ8 0UsAmLwQ38rNFLjhjCdyCgqzSgvzVS5ROQZlRvQpbaTcc9N1znr7XyPyVa5TvPBRvbodViKDR5zCi12AXWa7jq82QQRVplqTUOzIUrb9IpQGozeQ8Mm5hhF57qE2lToDXJZKI4j69Wf/eCQh8eySM8X6paf4F/5M5d8Jzg13JjjhtqQrTpDZEUk2m5oY+JbPqoKhFUKai36BvmELMFyxZdzVKweidXxuJEE3bJpv9vdXXRh03eWlb6aLaQBxj7TkxfJ0PgRy+fBMeCYobq5S/lxVLZ2VVY70cBfQwN3amQTobPgctFuJ0leC59PYV802pcuxoAIm0bBxrRAk/3bRLkFFHndYCvWfalWvDodqFkUBdTQyv2X9vum6BQ4F+GoWP9sc1kohsuH8JRWXd9If+jbp7pjtXzhONKB6RPDfZ/XUNAX0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: adds /sys/kernel/mm/numa/pagecache_promotion_enabled When page cache lands on lower tiers, there is no way for promotion to occur unless it becomes memory-mapped and exposed to NUMA hint faults. Just adding a mechanism to promote pages unconditionally, however, opens up significant possibility of performance regressions. Similar to the `demotion_enabled` sysfs entry, provide a sysfs toggle to enable and disable page cache promotion. This option will enable opportunistic promotion of unmapped page cache during syscall access. This option is intended for operational conditions where demoted page cache will eventually contain memory which becomes hot - and where said memory likely to cause performance issues due to being trapped on the lower tier of memory. A Page Cache folio is considered a promotion candidates when: 0) tiering and pagecache-promotion are enabled 1) the folio reside on a node not in the top tier 2) the folio is already marked referenced and active. 3) Multiple accesses in (referenced & active) state occur quickly. Since promotion is not safe to execute unconditionally from within folio_mark_accessed, we defer promotion to a new task_work captured in the task_struct. This ensures that the task doing the access has some hand in promoting pages - even among deduplicated read only files. We use numa_hint_fault_latency to help identify when a folio is accessed multiple times in a short period. Along with folio flag checks, this helps us minimize promoting pages on the first few accesses. The promotion node is always the local node of the promoting cpu. Suggested-by: Johannes Weiner Signed-off-by: Gregory Price --- .../ABI/testing/sysfs-kernel-mm-numa | 20 +++++++ include/linux/memory-tiers.h | 2 + include/linux/migrate.h | 2 + include/linux/sched.h | 3 + include/linux/sched/numa_balancing.h | 5 ++ init/init_task.c | 1 + kernel/sched/fair.c | 26 +++++++- mm/memory-tiers.c | 27 +++++++++ mm/migrate.c | 59 +++++++++++++++++++ mm/swap.c | 3 + 10 files changed, 147 insertions(+), 1 deletion(-) diff --git a/Documentation/ABI/testing/sysfs-kernel-mm-numa b/Documentation/ABI/testing/sysfs-kernel-mm-numa index 77e559d4ed80..b846e7d80cba 100644 --- a/Documentation/ABI/testing/sysfs-kernel-mm-numa +++ b/Documentation/ABI/testing/sysfs-kernel-mm-numa @@ -22,3 +22,23 @@ Description: Enable/disable demoting pages during reclaim the guarantees of cpusets. This should not be enabled on systems which need strict cpuset location guarantees. + +What: /sys/kernel/mm/numa/pagecache_promotion_enabled +Date: November 2024 +Contact: Linux memory management mailing list +Description: Enable/disable promoting pages during file access + + Page migration during file access is intended for systems + with tiered memory configurations that have significant + unmapped file cache usage. By default, file cache memory + on slower tiers will not be opportunistically promoted by + normal NUMA hint faults, because the system has no way to + track them. This option enables opportunistic promotion + of pages that are accessed via syscall (e.g. read/write) + if multiple accesses occur in quick succession. + + It may move data to a NUMA node that does not fall into + the cpuset of the allocating process which might be + construed to violate the guarantees of cpusets. This + should not be enabled on systems which need strict cpuset + location guarantees. diff --git a/include/linux/memory-tiers.h b/include/linux/memory-tiers.h index 0dc0cf2863e2..fa96a67b8996 100644 --- a/include/linux/memory-tiers.h +++ b/include/linux/memory-tiers.h @@ -37,6 +37,7 @@ struct access_coordinate; #ifdef CONFIG_NUMA extern bool numa_demotion_enabled; +extern bool numa_pagecache_promotion_enabled; extern struct memory_dev_type *default_dram_type; extern nodemask_t default_dram_nodes; struct memory_dev_type *alloc_memory_type(int adistance); @@ -76,6 +77,7 @@ static inline bool node_is_toptier(int node) #else #define numa_demotion_enabled false +#define numa_pagecache_promotion_enabled false #define default_dram_type NULL #define default_dram_nodes NODE_MASK_NONE /* diff --git a/include/linux/migrate.h b/include/linux/migrate.h index 29919faea2f1..cf58a97d4216 100644 --- a/include/linux/migrate.h +++ b/include/linux/migrate.h @@ -145,6 +145,7 @@ const struct movable_operations *page_movable_ops(struct page *page) int migrate_misplaced_folio_prepare(struct folio *folio, struct vm_area_struct *vma, int node); int migrate_misplaced_folio(struct folio *folio, int node); +void promotion_candidate(struct folio *folio); #else static inline int migrate_misplaced_folio_prepare(struct folio *folio, struct vm_area_struct *vma, int node) @@ -155,6 +156,7 @@ static inline int migrate_misplaced_folio(struct folio *folio, int node) { return -EAGAIN; /* can't migrate now */ } +static inline void promotion_candidate(struct folio *folio) { } #endif /* CONFIG_NUMA_BALANCING */ #ifdef CONFIG_MIGRATION diff --git a/include/linux/sched.h b/include/linux/sched.h index d380bffee2ef..faa84fb7a756 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -1356,6 +1356,9 @@ struct task_struct { unsigned long numa_faults_locality[3]; unsigned long numa_pages_migrated; + + struct callback_head numa_promo_work; + struct list_head promo_list; #endif /* CONFIG_NUMA_BALANCING */ #ifdef CONFIG_RSEQ diff --git a/include/linux/sched/numa_balancing.h b/include/linux/sched/numa_balancing.h index 52b22c5c396d..cc7750d754ff 100644 --- a/include/linux/sched/numa_balancing.h +++ b/include/linux/sched/numa_balancing.h @@ -32,6 +32,7 @@ extern void set_numabalancing_state(bool enabled); extern void task_numa_free(struct task_struct *p, bool final); bool should_numa_migrate_memory(struct task_struct *p, struct folio *folio, int src_nid, int dst_cpu); +int numa_hint_fault_latency(struct folio *folio); #else static inline void task_numa_fault(int last_node, int node, int pages, int flags) @@ -52,6 +53,10 @@ static inline bool should_numa_migrate_memory(struct task_struct *p, { return true; } +static inline int numa_hint_fault_latency(struct folio *folio) +{ + return 0; +} #endif #endif /* _LINUX_SCHED_NUMA_BALANCING_H */ diff --git a/init/init_task.c b/init/init_task.c index e557f622bd90..f831980748c4 100644 --- a/init/init_task.c +++ b/init/init_task.c @@ -187,6 +187,7 @@ struct task_struct init_task __aligned(L1_CACHE_BYTES) = { .numa_preferred_nid = NUMA_NO_NODE, .numa_group = NULL, .numa_faults = NULL, + .promo_list = LIST_HEAD_INIT(init_task.promo_list), #endif #if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) .kasan_depth = 1, diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index a59ae2e23daf..047f02091773 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -42,6 +42,7 @@ #include #include #include +#include #include #include #include @@ -1842,7 +1843,7 @@ static bool pgdat_free_space_enough(struct pglist_data *pgdat) * The smaller the hint page fault latency, the higher the possibility * for the page to be hot. */ -static int numa_hint_fault_latency(struct folio *folio) +int numa_hint_fault_latency(struct folio *folio) { int last_time, time; @@ -3534,6 +3535,27 @@ static void task_numa_work(struct callback_head *work) } } +static void task_numa_promotion_work(struct callback_head *work) +{ + struct task_struct *p = current; + struct list_head *promo_list = &p->promo_list; + struct folio *folio, *tmp; + int nid = numa_node_id(); + + SCHED_WARN_ON(p != container_of(work, struct task_struct, numa_promo_work)); + + work->next = work; + + if (list_empty(promo_list)) + return; + + list_for_each_entry_safe(folio, tmp, promo_list, lru) { + list_del_init(&folio->lru); + migrate_misplaced_folio(folio, nid); + } +} + + void init_numa_balancing(unsigned long clone_flags, struct task_struct *p) { int mm_users = 0; @@ -3558,8 +3580,10 @@ void init_numa_balancing(unsigned long clone_flags, struct task_struct *p) RCU_INIT_POINTER(p->numa_group, NULL); p->last_task_numa_placement = 0; p->last_sum_exec_runtime = 0; + INIT_LIST_HEAD(&p->promo_list); init_task_work(&p->numa_work, task_numa_work); + init_task_work(&p->numa_promo_work, task_numa_promotion_work); /* New address space, reset the preferred nid */ if (!(clone_flags & CLONE_VM)) { diff --git a/mm/memory-tiers.c b/mm/memory-tiers.c index fc14fe53e9b7..4c44598e485e 100644 --- a/mm/memory-tiers.c +++ b/mm/memory-tiers.c @@ -935,6 +935,7 @@ static int __init memory_tier_init(void) subsys_initcall(memory_tier_init); bool numa_demotion_enabled = false; +bool numa_pagecache_promotion_enabled; #ifdef CONFIG_MIGRATION #ifdef CONFIG_SYSFS @@ -957,11 +958,37 @@ static ssize_t demotion_enabled_store(struct kobject *kobj, return count; } +static ssize_t pagecache_promotion_enabled_show(struct kobject *kobj, + struct kobj_attribute *attr, + char *buf) +{ + return sysfs_emit(buf, "%s\n", + numa_pagecache_promotion_enabled ? "true" : "false"); +} + +static ssize_t pagecache_promotion_enabled_store(struct kobject *kobj, + struct kobj_attribute *attr, + const char *buf, size_t count) +{ + ssize_t ret; + + ret = kstrtobool(buf, &numa_pagecache_promotion_enabled); + if (ret) + return ret; + + return count; +} + + static struct kobj_attribute numa_demotion_enabled_attr = __ATTR_RW(demotion_enabled); +static struct kobj_attribute numa_pagecache_promotion_enabled_attr = + __ATTR_RW(pagecache_promotion_enabled); + static struct attribute *numa_attrs[] = { &numa_demotion_enabled_attr.attr, + &numa_pagecache_promotion_enabled_attr.attr, NULL, }; diff --git a/mm/migrate.c b/mm/migrate.c index af07b399060b..320258a1aaba 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -44,6 +44,8 @@ #include #include #include +#include +#include #include @@ -2710,5 +2712,62 @@ int migrate_misplaced_folio(struct folio *folio, int node) BUG_ON(!list_empty(&migratepages)); return nr_remaining ? -EAGAIN : 0; } + +/** + * promotion_candidate() - report a promotion candidate folio + * + * @folio: The folio reported as a candidate + * + * Records folio access time and places the folio on the task promotion list + * if access time is less than the threshold. The folio will be isolated from + * LRU if selected, and task_work will putback the folio on promotion failure. + * + * If selected, takes a folio reference to be released in task work. + */ +void promotion_candidate(struct folio *folio) +{ + struct task_struct *task = current; + struct list_head *promo_list = &task->promo_list; + struct callback_head *work = &task->numa_promo_work; + struct address_space *mapping = folio_mapping(folio); + bool write = mapping ? mapping->gfp_mask & __GFP_WRITE : false; + int nid = folio_nid(folio); + int flags, last_cpupid; + + /* + * Only do this work if: + * 1) tiering and pagecache promotion are enabled + * 2) the page can actually be promoted + * 3) The hint-fault latency is relatively hot + * 4) the folio is not already isolated + * 5) This is not a kernel thread context + */ + if (!(sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING) || + !numa_pagecache_promotion_enabled || + node_is_toptier(nid) || + numa_hint_fault_latency(folio) >= PAGE_ACCESS_TIME_MASK || + folio_test_isolated(folio) || + (current->flags & PF_KTHREAD)) { + return; + } + + nid = numa_migrate_check(folio, NULL, 0, &flags, write, &last_cpupid); + if (nid == NUMA_NO_NODE) + return; + + if (migrate_misplaced_folio_prepare(folio, NULL, nid)) + return; + + /* + * Ensure task can schedule work, otherwise we'll leak folios. + * If the list is not empty, task work has already been scheduled. + */ + if (list_empty(promo_list) && task_work_add(task, work, TWA_RESUME)) { + folio_putback_lru(folio); + return; + } + list_add(&folio->lru, promo_list); +} +EXPORT_SYMBOL(promotion_candidate); #endif /* CONFIG_NUMA_BALANCING */ #endif /* CONFIG_NUMA */ diff --git a/mm/swap.c b/mm/swap.c index 320b959b74c6..57909c349388 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -37,6 +37,7 @@ #include #include #include +#include #include "internal.h" @@ -469,6 +470,8 @@ void folio_mark_accessed(struct folio *folio) __lru_cache_activate_folio(folio); folio_clear_referenced(folio); workingset_activation(folio); + } else { + promotion_candidate(folio); } if (folio_test_idle(folio)) folio_clear_idle(folio);