From patchwork Mon Apr 14 22:24:54 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nico Pache X-Patchwork-Id: 14051095 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AFF97C369B8 for ; Mon, 14 Apr 2025 22:26:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2240A2800AD; Mon, 14 Apr 2025 18:25:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F197C2800A7; Mon, 14 Apr 2025 18:25:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D69BD2800AD; Mon, 14 Apr 2025 18:25:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id B96302800A7 for ; Mon, 14 Apr 2025 18:25:57 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id BB4F9AF74A for ; Mon, 14 Apr 2025 22:25:58 +0000 (UTC) X-FDA: 83334083196.03.66FC5EE Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf04.hostedemail.com (Postfix) with ESMTP id 0373640007 for ; Mon, 14 Apr 2025 22:25:56 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=cxeHqMNC; spf=pass (imf04.hostedemail.com: domain of npache@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=npache@redhat.com; dmarc=pass (policy=quarantine) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744669557; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=CV9IGBx6ODvRJHViy9uocxS1JFT5Dwo7ktLvhKvEEnw=; b=5JSw4cmvPe2cQjYc1oQwduTbm67GV3nIOxuRsoi/AWsZdsZbLo29liVPcX7kZRHxxvV0L2 gzKZofbAb7jy1SpVbeMiCy7XIleUE0wZA0r6rUJcQKqsbqez7QTmdgcuXg+P+dRwUvAB6C JIg9MJrSBo9u+nRgj7O8tAQN3zfLqGQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744669557; a=rsa-sha256; cv=none; b=qHOpSjWRRoeUSeF7CY9C5D9eVUlZKxoRtq2RU5IJ3zBq4dYok1iZWi7yxDwy2/tmbz7e+T ncwjjcZgDA5E67f2xPpMbw5n1AMmYYpeLAv+c89IAIIizXTPU1LTt1w6JCe74GbeHiDxSp MLchtfI26b/AiIHiqm1M3RLKUCuHHcY= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=cxeHqMNC; spf=pass (imf04.hostedemail.com: domain of npache@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=npache@redhat.com; dmarc=pass (policy=quarantine) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744669556; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CV9IGBx6ODvRJHViy9uocxS1JFT5Dwo7ktLvhKvEEnw=; b=cxeHqMNCcrFJOOunmJYDnJL4hEPk4VRUjAZU075SMj4gKa8EY8dmkc2qVakUG2S6A44eSX RaQcBogHkikw3WVEPFfKgILMHQg4cRC8fEK8yUDKPMAY6rDm042NfzBPdQHXv36AltnRe9 L0fbbyxxs8T3QdttQDAiFvtfQzEaAtM= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-10-qe0rroeyNIqL-m90P1mikQ-1; Mon, 14 Apr 2025 18:25:53 -0400 X-MC-Unique: qe0rroeyNIqL-m90P1mikQ-1 X-Mimecast-MFC-AGG-ID: qe0rroeyNIqL-m90P1mikQ_1744669549 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 1EC0F1956087; Mon, 14 Apr 2025 22:25:49 +0000 (UTC) Received: from h1.redhat.com (unknown [10.22.64.91]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 29B63180B486; Mon, 14 Apr 2025 22:25:39 +0000 (UTC) From: Nico Pache To: linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org Cc: akpm@linux-foundation.org, corbet@lwn.net, shuah@kernel.org, david@redhat.com, baohua@kernel.org, baolin.wang@linux.alibaba.com, ryan.roberts@arm.com, willy@infradead.org, peterx@redhat.com, ioworker0@gmail.com, ziy@nvidia.com, wangkefeng.wang@huawei.com, dev.jain@arm.com, mhocko@suse.com, rientjes@google.com, hannes@cmpxchg.org, zokeefe@google.com, surenb@google.com, jglisse@google.com, cl@gentwo.org, jack@suse.cz, dave.hansen@linux.intel.com, will@kernel.org, tiwai@suse.de, catalin.marinas@arm.com, anshuman.khandual@arm.com, raquini@redhat.com, aarcange@redhat.com, kirill.shutemov@linux.intel.com, yang@os.amperecomputing.com, thomas.hellstrom@linux.intel.com, vishal.moola@gmail.com, sunnanyong@huawei.com, usamaarif642@gmail.com, mathieu.desnoyers@efficios.com, mhiramat@kernel.org, rostedt@goodmis.org Subject: [PATCH v3 2/4] mm: document (m)THP defer usage Date: Mon, 14 Apr 2025 16:24:54 -0600 Message-ID: <20250414222456.43212-3-npache@redhat.com> In-Reply-To: <20250414222456.43212-1-npache@redhat.com> References: <20250414222456.43212-1-npache@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 X-Stat-Signature: ra1rxse139juc6f3zux4yn38ze7kb8hw X-Rspam-User: X-Rspamd-Queue-Id: 0373640007 X-Rspamd-Server: rspam08 X-HE-Tag: 1744669556-792124 X-HE-Meta: U2FsdGVkX19IGOIMiMnxXbFzxfMh3By3t5QFoNsq+w5iSl3Q2VBH0TUDAChypL722dFYGeyWrFDjSjCw16KJE2V90t02ZgIopPSNkeRy/pnC+Jgpu81sMuVtYM75Zhbzxpco9xl3f7A2TeTc7Dume0YuvkFMH/QpGAjq85d+aHpaybJoKRUKN6U4HmQweb11sliVsLx9I38KIU6qciSDgWwXCtfKZ8EUiic8RIxuyYwJ4NwFY39/Xjcq1BLug+j3CBk9ZCSMkGLeqiysFwr0Ty1bXnigofyp/q+Vy1yL6KB9lhckP8mB1QJDmeZSm14agG1YKf6YE0SxNfKdFwTQpZ03BPZLV7LoPnxl0QZ8DOTuktOccPbshnGnXqOPBACMNzKFbESREaTccQYBKaQp36ZLTBLcHFDNp6yeLmyWkrBqNmZsbh0jWzdKBLPxUaxcoQ3s583ojMc6+PZOoIb/WnwSlVbjtj8ZTRfsuT2CvzNdsu5XOindAwNeWu5WUKdNqmqDOzl8wVgJzpbBsZBebAY27ZJmasTq5Lw3e0rrJzUOU0aT/4I3XEkErCS0E2mp99TWHOhfw+mwbVF4VsOG7q0NpHyaeIrbDiIRiXQOxaHh6fyju5RBqDKkAfHE1VXTo5n3lnPIznBFz3FLG6J3Re+Xf6wuBTST85zDE28IIXb74bU+yZQEMXbPk6Y+S8U0aRWd9bV6JYrbMRGSw0mBkax9z+HUPrm75YB9kRC60EEKO4ENq7J/JxBPIbCWhUcJGRQhwXREemMUr/A8eFdWIh1QSVX2qs2n92IZaGKc9/U4hb/9qtJsEPJPQCV1XnZGmt1vANiX7kKpVLXdeOUAR9KERT3HeI4NOjKfj8WxlSZfgDY1iIj2WPzXlzb7R4OJByVfp1ysEkatwEB98nDsGS/gBzKgKTCqN2g2HrC8iVow8jUEk5E34VNk/aftBhLUXPk8rvZRkHooKvrjtwO Zvw5Dza8 QyS7nSHPy+Z/Vs1wlfxqtOOi9beq27b2/K/wcFt//XcwdoqurTYP3B3fVYYr9r7FhSos4iEriA6JxdKuHiNIBQSlAU++uE6mhTsUKDI0zYA3uX+zM4lB/+PNYDYiT+bVGEp1ZFmsPK5qAeec+aJsPhyRK0vIc8gvmDRF1NZtU/F1oRYIq7xiTLna4XMF1/rIevzSKHfAuq9PJIyIXQszHhtVQeMBUSwZ2PYKx/L2IJcK0e9KlvHChAqzYFqogqp1aLWEanoaoHecbe0kKgfEYH3KTD7GVCwEeRx0rV8598xaFmKBLJGWyni82aFzcT2Vwd9dSL5/kRIXjGuj5RSOBWyOPuF6Pj6rS7dOURNxVO9YZHDdnCZoIqyXrQROTkT2y/vSHOiRk0Ny1fkDqIuhKF0Jkc8CJsFJ0SuApBwRrARA2j0vIxd1BwZtTUg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The new defer option for (m)THPs allows for a more conservative approach to (m)THPs. Document its usage in the transhuge admin-guide. Signed-off-by: Nico Pache --- Documentation/admin-guide/mm/transhuge.rst | 31 ++++++++++++++++------ 1 file changed, 23 insertions(+), 8 deletions(-) diff --git a/Documentation/admin-guide/mm/transhuge.rst b/Documentation/admin-guide/mm/transhuge.rst index f0d4e78cedaa..d3f072bdd932 100644 --- a/Documentation/admin-guide/mm/transhuge.rst +++ b/Documentation/admin-guide/mm/transhuge.rst @@ -88,8 +88,9 @@ In certain cases when hugepages are enabled system wide, application may end up allocating more memory resources. An application may mmap a large region but only touch 1 byte of it, in that case a 2M page might be allocated instead of a 4k page for no good. This is why it's -possible to disable hugepages system-wide and to only have them inside -MADV_HUGEPAGE madvise regions. +possible to disable hugepages system-wide, only have them inside +MADV_HUGEPAGE madvise regions, or defer them away from the page fault +handler to khugepaged. Embedded systems should enable hugepages only inside madvise regions to eliminate any risk of wasting any precious byte of memory and to @@ -99,6 +100,15 @@ Applications that gets a lot of benefit from hugepages and that don't risk to lose memory by using hugepages, should use madvise(MADV_HUGEPAGE) on their critical mmapped regions. +Applications that would like to benefit from THPs but would still like a +more memory conservative approach can choose 'defer'. This avoids +inserting THPs at the page fault handler unless they are MADV_HUGEPAGE. +Khugepaged will then scan the mappings for potential collapses into (m)THP +pages. Admins using this the 'defer' setting should consider +tweaking khugepaged/max_ptes_none. The current default of 511 may +aggressively collapse your PTEs into PMDs. Lower this value to conserve +more memory (ie. max_ptes_none=64). + .. _thp_sysfs: sysfs @@ -109,11 +119,14 @@ Global THP controls Transparent Hugepage Support for anonymous memory can be entirely disabled (mostly for debugging purposes) or only enabled inside MADV_HUGEPAGE -regions (to avoid the risk of consuming more memory resources) or enabled -system wide. This can be achieved per-supported-THP-size with one of:: +regions (to avoid the risk of consuming more memory resources), defered to +khugepaged, or enabled system wide. + +This can be achieved per-supported-THP-size with one of:: echo always >/sys/kernel/mm/transparent_hugepage/hugepages-kB/enabled echo madvise >/sys/kernel/mm/transparent_hugepage/hugepages-kB/enabled + echo defer >/sys/kernel/mm/transparent_hugepage/hugepages-kB/enabled echo never >/sys/kernel/mm/transparent_hugepage/hugepages-kB/enabled where is the hugepage size being addressed, the available sizes @@ -136,6 +149,7 @@ The top-level setting (for use with "inherit") can be set by issuing one of the following commands:: echo always >/sys/kernel/mm/transparent_hugepage/enabled + echo defer >/sys/kernel/mm/transparent_hugepage/enabled echo madvise >/sys/kernel/mm/transparent_hugepage/enabled echo never >/sys/kernel/mm/transparent_hugepage/enabled @@ -281,7 +295,8 @@ of small pages into one large page:: A higher value leads to use additional memory for programs. A lower value leads to gain less thp performance. Value of max_ptes_none can waste cpu time very little, you can -ignore it. +ignore it. Consider lowering this value when using +``transparent_hugepage=defer`` ``max_ptes_swap`` specifies how many pages can be brought in from swap when collapsing a group of pages into a transparent huge page:: @@ -306,14 +321,14 @@ Boot parameters You can change the sysfs boot time default for the top-level "enabled" control by passing the parameter ``transparent_hugepage=always`` or -``transparent_hugepage=madvise`` or ``transparent_hugepage=never`` to the -kernel command line. +``transparent_hugepage=madvise`` or ``transparent_hugepage=defer`` or +``transparent_hugepage=never`` to the kernel command line. Alternatively, each supported anonymous THP size can be controlled by passing ``thp_anon=[KMG],[KMG]:;[KMG]-[KMG]:``, where ```` is the THP size (must be a power of 2 of PAGE_SIZE and supported anonymous THP) and ```` is one of ``always``, ``madvise``, -``never`` or ``inherit``. +``defer``, ``never`` or ``inherit``. For example, the following will set 16K, 32K, 64K THP to ``always``, set 128K, 512K to ``inherit``, set 256K to ``madvise`` and 1M, 2M