From patchwork Thu Apr 17 00:18:44 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nico Pache X-Patchwork-Id: 14054663 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C781EC369BD for ; Thu, 17 Apr 2025 00:20:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 04E2228013B; Wed, 16 Apr 2025 20:20:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F17CA28001A; Wed, 16 Apr 2025 20:20:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D6ABF28013B; Wed, 16 Apr 2025 20:20:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id B50E128001A for ; Wed, 16 Apr 2025 20:20:08 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id C199DC0FD9 for ; Thu, 17 Apr 2025 00:20:08 +0000 (UTC) X-FDA: 83341628496.10.B049ABE Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf07.hostedemail.com (Postfix) with ESMTP id F17B440005 for ; Thu, 17 Apr 2025 00:20:06 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=X4j02s5F; spf=pass (imf07.hostedemail.com: domain of npache@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=npache@redhat.com; dmarc=pass (policy=quarantine) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744849207; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=T90LCrJZAPUEyLfMWog/+6biSrJXKfE1zcoM8kFo7hQ=; b=KpQdgi2S9gT56RUw2OXFyAhblI5ovKnwXKXmBnNtHwXcbrx6cAA7EDXdWDR1mWIqTmAjel cixZEhpj4ANfMts9S0lsYsbXNjyTRiZXA7wObw2hxCOfYQ3zkxqaiMU1nMXPuVu1XMtPNZ ICx/VNxhPphDN+ChOp+V608THlMvGkc= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=X4j02s5F; spf=pass (imf07.hostedemail.com: domain of npache@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=npache@redhat.com; dmarc=pass (policy=quarantine) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744849207; a=rsa-sha256; cv=none; b=BTgEF6ZxJnRRq7O8P39Eub8K8+aXu0RbxoH32ovS7U4kcGlN1L4axHppbknen9QGVDB5Bg NwdRfgY7To6qWcacLyD1qW8+HL+IJGmhClGgswY/fCwZN0wISY73tQ7fJz/2EhzicddjB9 20Y8DAmRIpAsg4DvkUZjuC1cdVDf5+I= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744849206; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=T90LCrJZAPUEyLfMWog/+6biSrJXKfE1zcoM8kFo7hQ=; b=X4j02s5FUvnrUb60dX8DoTDfWnI9nQ4avwXubD6txVhSSNGNFESClw1YHIkMKLtFsiIbbw lJbT/5QPM044adF+sLMbZB+54iVnZw87+fiJXacpOf47rPbXfjw4fkhGJvTmt+aIUjlhUo 61fySweIntKG+6O4/pFpRrwpI7zW4EY= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-687-wmrC2r89NbChm7BLbFOqFA-1; Wed, 16 Apr 2025 20:20:01 -0400 X-MC-Unique: wmrC2r89NbChm7BLbFOqFA-1 X-Mimecast-MFC-AGG-ID: wmrC2r89NbChm7BLbFOqFA_1744849198 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 6B223195609F; Thu, 17 Apr 2025 00:19:57 +0000 (UTC) Received: from h1.redhat.com (unknown [10.22.88.34]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 06E0C19560A3; Thu, 17 Apr 2025 00:19:47 +0000 (UTC) From: Nico Pache To: linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org Cc: akpm@linux-foundation.org, corbet@lwn.net, rostedt@goodmis.org, mhiramat@kernel.org, mathieu.desnoyers@efficios.com, david@redhat.com, baohua@kernel.org, baolin.wang@linux.alibaba.com, ryan.roberts@arm.com, willy@infradead.org, peterx@redhat.com, shuah@kernel.org, ziy@nvidia.com, wangkefeng.wang@huawei.com, usamaarif642@gmail.com, sunnanyong@huawei.com, vishal.moola@gmail.com, thomas.hellstrom@linux.intel.com, yang@os.amperecomputing.com, kirill.shutemov@linux.intel.com, aarcange@redhat.com, raquini@redhat.com, dev.jain@arm.com, anshuman.khandual@arm.com, catalin.marinas@arm.com, tiwai@suse.de, will@kernel.org, dave.hansen@linux.intel.com, jack@suse.cz, cl@gentwo.org, jglisse@google.com, surenb@google.com, zokeefe@google.com, hannes@cmpxchg.org, rientjes@google.com, mhocko@suse.com, rdunlap@infradead.org Subject: [PATCH v4 2/4] mm: document (m)THP defer usage Date: Wed, 16 Apr 2025 18:18:44 -0600 Message-ID: <20250417001846.81480-3-npache@redhat.com> In-Reply-To: <20250417001846.81480-1-npache@redhat.com> References: <20250417001846.81480-1-npache@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 X-Rspam-User: X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: F17B440005 X-Stat-Signature: 93issfx947g3yu5o69g7rmy93wsdnwe7 X-HE-Tag: 1744849206-583270 X-HE-Meta: U2FsdGVkX1/x3zFo1dRhqU3K+yIMXhGf0bOJT2CjfCfMO+A3XlCnPkVMHzaOvkHzYR7WdSx2z1Q0A3ya7Lp8xZHrdjRwozpyZPu/Mvpw5U2upoc7tmPETENedHt3KLMYOdZ52OHg6XDQfInbIm0DeIor/cFJJE218AqCvAMM4FEbciB5JK3Y326SF7KN2UUOzBR606b4yuIyMdrqGzK5C/TEoBovt6sBE453I0Ahis8Ck/cf87A+hLSyr++pYh/MxCI2P1AvtIaHUWlYShekuR5H+706NSuTL/UZMVmD5qChvblmS7K9bqnRQWkNkD9pvIBgPEVvgax+lSPdVz8x91oWEuClvd8j3xdRFja6MzKnVXCawMA3+RSKrypXhsAQLkempdwtZQ7pgPDpBEdyphl7qXmVQKiDmrVT6SrlLUSE44plxcWgkmOmcpzXmJlqCH91Tye1W6qIDB1F/Z4ka/s4R62AKEXnM8BZCadJ9xT3kI6y5SMxw3+ZXP92uEYwu8Uk7qZ8D5xptPOQvmswSa+rZ17kkPHk94Lk/x5YoS1NCmhnd4CaOsW5XuMsuj39+XEo4KwIYcY+i19fuJL29nwl2rG8cFEfICe7jXCpgL/wQcScc7zwEAHf3Y/miks+dVGJYePTuX1oMp0S9jkpnN/4ilBffrsYH5cWHg8OG+97cIsHiGiu7famt06aetIe6mc+BYmFThoshd82rnEaFKqUuTB3vy0ApcaJIwDNCrW2K/t8NmjV3CeYNDJvdhIluB2F1Ux3coqNCCcdJdrPQst8gPpGW8+uLbgG5t1/XTaamsx1uxX9YV6yMDXMGuaaYY2jg8b0/GRAnXlkNDSGjl2i8boiZw5F2Xqfp3Mtx40JP5EEN4nqbAKnT8Vw6xx8wtjGUz4WzeOexfpY8Mpgxz7VhFFg0yAD+mXCV9Pnx+01jEVa/KBOJ0YwlHD/ciN5mf4lYksSi4tCf4DvFlp XU34N7UU d3c2wTG/OOcr4if7PO6/5Yb/bDr+DJDdzaQGucOLtwfBlEz/1tq20PAvhnE/uxZbB1wjxDKRZA63TItPTPxfvKZiYmm/s5Kb40BRkwKScZnSNjwRvjpJvjihWftF6A0JOMnv46h6CTJjmQ7mLKwpm874347nzCaGLMz8ZkzuX0JRpbUUGWqMLt7kkHkmAJ2+iD7FpOPRBS3WbKCKscAqC7OWQxAQZy8NR8h0lKmTbKXJbFpD6YpCXR4Cw5O1/Fjlps3ffsjSsNd2/0xWcE0lGpSxF1jNuO3DjlQtYRagJ5mrwaZEYRspnpdwQ15FJe3BZb+TWePFhWbdfkX3vq/pTF7JTxawo0Kpn+tPaKAAh870bCGR6+6kZbCd7XN1z8M8UZBXganmAahoaBnQXMDFxzJOz5Qg0wF5VDF7fEr5vREA9vb/PCW9rY1kXwQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The new defer option for (m)THPs allows for a more conservative approach to (m)THPs. Document its usage in the transhuge admin-guide. Signed-off-by: Nico Pache Reviewed-by: Bagas Sanjaya --- Documentation/admin-guide/mm/transhuge.rst | 31 ++++++++++++++++------ 1 file changed, 23 insertions(+), 8 deletions(-) diff --git a/Documentation/admin-guide/mm/transhuge.rst b/Documentation/admin-guide/mm/transhuge.rst index 06814e05e1d5..38e1778d9eaa 100644 --- a/Documentation/admin-guide/mm/transhuge.rst +++ b/Documentation/admin-guide/mm/transhuge.rst @@ -88,8 +88,9 @@ In certain cases when hugepages are enabled system wide, application may end up allocating more memory resources. An application may mmap a large region but only touch 1 byte of it, in that case a 2M page might be allocated instead of a 4k page for no good. This is why it's -possible to disable hugepages system-wide and to only have them inside -MADV_HUGEPAGE madvise regions. +possible to disable hugepages system-wide, only have them inside +MADV_HUGEPAGE madvise regions, or defer them away from the page fault +handler to khugepaged. Embedded systems should enable hugepages only inside madvise regions to eliminate any risk of wasting any precious byte of memory and to @@ -99,6 +100,15 @@ Applications that gets a lot of benefit from hugepages and that don't risk to lose memory by using hugepages, should use madvise(MADV_HUGEPAGE) on their critical mmapped regions. +Applications that would like to benefit from THPs but would still like a +more memory conservative approach can choose 'defer'. This avoids +inserting THPs at the page fault handler unless they are MADV_HUGEPAGE. +Khugepaged will then scan the mappings for potential collapses into (m)THP +pages. Admins using this the 'defer' setting should consider +tweaking khugepaged/max_ptes_none. The current default of 511 may +aggressively collapse your PTEs into PMDs. Lower this value to conserve +more memory (i.e., max_ptes_none=64). + .. _thp_sysfs: sysfs @@ -109,11 +119,14 @@ Global THP controls Transparent Hugepage Support for anonymous memory can be entirely disabled (mostly for debugging purposes) or only enabled inside MADV_HUGEPAGE -regions (to avoid the risk of consuming more memory resources) or enabled -system wide. This can be achieved per-supported-THP-size with one of:: +regions (to avoid the risk of consuming more memory resources), deferred to +khugepaged, or enabled system wide. + +This can be achieved per-supported-THP-size with one of:: echo always >/sys/kernel/mm/transparent_hugepage/hugepages-kB/enabled echo madvise >/sys/kernel/mm/transparent_hugepage/hugepages-kB/enabled + echo defer >/sys/kernel/mm/transparent_hugepage/hugepages-kB/enabled echo never >/sys/kernel/mm/transparent_hugepage/hugepages-kB/enabled where is the hugepage size being addressed, the available sizes @@ -136,6 +149,7 @@ The top-level setting (for use with "inherit") can be set by issuing one of the following commands:: echo always >/sys/kernel/mm/transparent_hugepage/enabled + echo defer >/sys/kernel/mm/transparent_hugepage/enabled echo madvise >/sys/kernel/mm/transparent_hugepage/enabled echo never >/sys/kernel/mm/transparent_hugepage/enabled @@ -282,7 +296,8 @@ of small pages into one large page:: A higher value leads to use additional memory for programs. A lower value leads to gain less thp performance. Value of max_ptes_none can waste cpu time very little, you can -ignore it. +ignore it. Consider lowering this value when using +``transparent_hugepage=defer`` ``max_ptes_swap`` specifies how many pages can be brought in from swap when collapsing a group of pages into a transparent huge page:: @@ -307,14 +322,14 @@ Boot parameters You can change the sysfs boot time default for the top-level "enabled" control by passing the parameter ``transparent_hugepage=always`` or -``transparent_hugepage=madvise`` or ``transparent_hugepage=never`` to the -kernel command line. +``transparent_hugepage=madvise`` or ``transparent_hugepage=defer`` or +``transparent_hugepage=never`` to the kernel command line. Alternatively, each supported anonymous THP size can be controlled by passing ``thp_anon=[KMG],[KMG]:;[KMG]-[KMG]:``, where ```` is the THP size (must be a power of 2 of PAGE_SIZE and supported anonymous THP) and ```` is one of ``always``, ``madvise``, -``never`` or ``inherit``. +``defer``, ``never`` or ``inherit``. For example, the following will set 16K, 32K, 64K THP to ``always``, set 128K, 512K to ``inherit``, set 256K to ``madvise`` and 1M, 2M