From patchwork Fri Nov 1 16:54:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ma=C3=ADra_Canal?= X-Patchwork-Id: 13859618 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E18EEE6F071 for ; Fri, 1 Nov 2024 16:58:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7B83C6B009A; Fri, 1 Nov 2024 12:58:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 741176B009B; Fri, 1 Nov 2024 12:58:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5BB426B009C; Fri, 1 Nov 2024 12:58:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 374E16B009A for ; Fri, 1 Nov 2024 12:58:23 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id CB8DBAAB4F for ; Fri, 1 Nov 2024 16:58:22 +0000 (UTC) X-FDA: 82738133772.05.005467F Received: from fanzine2.igalia.com (fanzine.igalia.com [178.60.130.6]) by imf30.hostedemail.com (Postfix) with ESMTP id 5B68880010 for ; Fri, 1 Nov 2024 16:57:27 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=igalia.com header.s=20170329 header.b=L0ixQzk5; spf=pass (imf30.hostedemail.com: domain of mcanal@igalia.com designates 178.60.130.6 as permitted sender) smtp.mailfrom=mcanal@igalia.com; dmarc=pass (policy=none) header.from=igalia.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1730480168; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Ys/vY/e+8yfzEKIu4LKRRFsTDzyvnthR/qzPM3DoISA=; b=hDFatPhDnZGe3f875z4xMpGH+GTuC2W04DGoLZqMJ1x7IguzA2CX2AoYVHl+tLTuHac+Uf 2wh9M316CNoDbPio9goiSb2t5M2qut9QpgBFDp1ENsP++fTCZaVDQOOEZDPYjYdyrQZD8P kZOPb3GA/31QUwDKh7BLqCkCa48q/RE= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1730480168; a=rsa-sha256; cv=none; b=hBoETOSukswQg4z/RbRxffZgbT99KsJ8LH7AKtpIhT2RcR6d0B7tnT2PNZsdWASVdSy+kc sTbOGZqaxw4qRmySwNTZSuNSNJ/cpG4u9+UOS0AiLEbN5Ep0lQmZtnN9mQyJNs5KhGMrLe Php1vVZo1OXbWusF0wr5JiaC4aCjQKU= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=igalia.com header.s=20170329 header.b=L0ixQzk5; spf=pass (imf30.hostedemail.com: domain of mcanal@igalia.com designates 178.60.130.6 as permitted sender) smtp.mailfrom=mcanal@igalia.com; dmarc=pass (policy=none) header.from=igalia.com DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:Content-Type:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=Ys/vY/e+8yfzEKIu4LKRRFsTDzyvnthR/qzPM3DoISA=; b=L0ixQzk5rq1dwuZsLc85aKbjG6 DAAXXvrSlteNqKUke8PbLf3vBNPrMAOM34JG4VCZVKC6ajdzj5OU2oaaC77BC7JfRpwKnDRcvVwSc +3QYDyAqKYcIm0G/DOXybSkBhzfv/I2klwXRJgts6ftfa9RTM64lgA3BcXW3+anSPMCXZ8IJ7Aass F0+84tV+OUgvxfFcfDK6gUVp2eiyZ+JCYbxOv2uz56Mx5EFbP1ZlKLyiEzNsomfZmc3wqKAnxh2Yg ZdrTRfXJtpPkmDFYe5Rdq2BH/ia1U+T0KujDqpsCN+BrU1aSrtkEW6fsiHvJbY7Ng+jSShxl52vNe 6yR4etKg==; Received: from [187.36.213.55] (helo=morissey..) by fanzine2.igalia.com with esmtpsa (Cipher TLS1.3:ECDHE_X25519__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim) id 1t6uyR-000VPu-59; Fri, 01 Nov 2024 17:58:11 +0100 From: =?utf-8?q?Ma=C3=ADra_Canal?= To: Jonathan Corbet , Andrew Morton , Hugh Dickins , Barry Song , David Hildenbrand , Ryan Roberts , Baolin Wang , Lance Yang Cc: linux-mm@kvack.org, dri-devel@lists.freedesktop.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, kernel-dev@igalia.com, =?utf-8?q?Ma=C3=ADra_Canal?= Subject: [PATCH v5 4/5] mm: shmem: override mTHP shmem default with a kernel parameter Date: Fri, 1 Nov 2024 13:54:08 -0300 Message-ID: <20241101165719.1074234-6-mcanal@igalia.com> X-Mailer: git-send-email 2.46.2 In-Reply-To: <20241101165719.1074234-2-mcanal@igalia.com> References: <20241101165719.1074234-2-mcanal@igalia.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 5B68880010 X-Stat-Signature: i9nn6uqc1q67k7gzph1qx4p4sya39e98 X-HE-Tag: 1730480247-912017 X-HE-Meta: U2FsdGVkX1+jeQ2dKNIMe9WUcyCQF2aMak30rlRLEGYevIjoguBye/dox7iQNJ7Sejt8N5UEeyJz+h9qneCv3pI3w6JIcQ4/PLRkEp0ewOt/N8En3zUmoaVcXEZbuqDqHUv12cQNRbNhM0hKCRLCbLdCDaBDu0Cc4KtbF1QgFewTAk7Fh0sX/3o5YlMf/PJmweyVR+8wfF/IgwESGuQEHFogHGAX+vG5rpC0SN9oFu5pmUgmfocHwY79nc1FfvWld7iYZrpq0Z3UWrb8P2BXt5IwusX5S97JvcEMJjejlbMZgnCpT1spa+TQCyCiBgrhA3c7u/zVv7h5cLSt3LtSMdzVqpT0BF7afhcAONZMy9yJohpdDHhFJJe8cRBx1X+rwORHnaXhiVAT2Z9seOEZAPreqiEcSYMdlVzjOsOV8isdnz0b4Qm6nUNum+HLLgHJ1LtcMwY+OE85eFQxn5+95Sjpb/1WwtqaGicvJJUwHpPx1Ag+SMOiK00kDmKKM/5xQRlX4HouWnCnm0PIGHsOV49FyqYHgKJYv5klHox+GN6x4jPFieS8zdWBcXQP4wcOFhtKtaNFZamrmp19xxxtoDKh/gvXy3SK869DWPLpCVIZ2P2fJtiIXgj/NhLWMlvp3+HFKrWZow43C0B08nA9+8pumzrqd7eEqD+5WklHb/WxyOCqf+vOmDqqozAU4qluZsPWfY9h+D/AS4dS82hjmhcivZYUTyu67nh3WoGZKohqD/gl/jBOnQq7uQWixfwKi1fv2f/V1GTeTbe2BlHomqcmzpBvxt5SGHHQQDiVvjaoL6UQJNnXs3QRDsQoztVQafZkQsdBfOdIJuMZ8E+iiip3jGlsErBeJC6MBhr6ZDmbWgE9YDd7MP4ZJwZf35gwHbXnpYBk89mqTS3pDTOLdrjqEP+FjKtC+L0Mhmi8oUtkwXlbnk8JHqsw1011LKlp+LzkpOy9lQesMAF1/aZ u32iDYuz SKVITR5q9V1F6RHXdVnUwK+m+r3FqIr2iNsOvgZTYN02KbndsM8DoHulFHoJ/GO96jJmjoFU817sfHs1xgk2cTYVqvI6EyZM3soW1W+IawpkMFKr0z8/WnoZCOXnNTGjfaGS2nbTTyfFdomxHj0e1K0pn0FdiLoOQF1tAqvnFDvow3TtZRReQDqu0PMSwuoJ4yYYCDlOqP0S2evToyOHFKa7WeEZA0yyjvrtP4Ys8wOxF3APz/wHfA5cjuKpqBPSU/azaOWKQSqU1ZDHDhAjkTOXpfd6IbdOQ7UKotheaFtbee1vPGBZiXgBtO8UQkaYYlPOGigi/h4rYu+U= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add the ``thp_shmem=`` kernel command line to allow specifying the default policy of each supported shmem hugepage size. The kernel parameter accepts the following format: thp_shmem=[KMG],[KMG]:;[KMG]-[KMG]: For example, thp_shmem=16K-64K:always;128K,512K:inherit;256K:advise;1M-2M:never;4M-8M:within_size Some GPUs may benefit from using huge pages. Since DRM GEM uses shmem to allocate anonymous pageable memory, it’s essential to control the huge page allocation policy for the internal shmem mount. This control can be achieved through the ``transparent_hugepage_shmem=`` parameter. Beyond just setting the allocation policy, it’s crucial to have granular control over the size of huge pages that can be allocated. The GPU may support only specific huge page sizes, and allocating pages larger/smaller than those sizes would be ineffective. Signed-off-by: Maíra Canal Reviewed-by: Baolin Wang --- .../admin-guide/kernel-parameters.txt | 10 ++ Documentation/admin-guide/mm/transhuge.rst | 17 +++ mm/shmem.c | 105 +++++++++++++++++- 3 files changed, 131 insertions(+), 1 deletion(-) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index acabb04d0dd4..b48d744d99b0 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -6700,6 +6700,16 @@ Force threading of all interrupt handlers except those marked explicitly IRQF_NO_THREAD. + thp_shmem= [KNL] + Format: [KMG],[KMG]:;[KMG]-[KMG]: + Control the default policy of each hugepage size for the + internal shmem mount. is one of policies available + for the shmem mount ("always", "inherit", "never", "within_size", + and "advise"). + It can be used multiple times for multiple shmem THP sizes. + See Documentation/admin-guide/mm/transhuge.rst for more + details. + topology= [S390,EARLY] Format: {off | on} Specify if the kernel should make use of the cpu diff --git a/Documentation/admin-guide/mm/transhuge.rst b/Documentation/admin-guide/mm/transhuge.rst index 9c6f6da612c4..5034915f4e8e 100644 --- a/Documentation/admin-guide/mm/transhuge.rst +++ b/Documentation/admin-guide/mm/transhuge.rst @@ -332,6 +332,23 @@ allocation policy for the internal shmem mount by using the kernel parameter seven valid policies for shmem (``always``, ``within_size``, ``advise``, ``never``, ``deny``, and ``force``). +In the same manner as ``thp_anon`` controls each supported anonymous THP +size, ``thp_shmem`` controls each supported shmem THP size. ``thp_shmem`` +has the same format as ``thp_anon``, but also supports the policy +``within_size``. + +``thp_shmem=`` may be specified multiple times to configure all THP sizes +as required. If ``thp_shmem=`` is specified at least once, any shmem THP +sizes not explicitly configured on the command line are implicitly set to +``never``. + +``transparent_hugepage_shmem`` setting only affects the global toggle. If +``thp_shmem`` is not specified, PMD_ORDER hugepage will default to +``inherit``. However, if a valid ``thp_shmem`` setting is provided by the +user, the PMD_ORDER hugepage policy will be overridden. If the policy for +PMD_ORDER is not defined within a valid ``thp_shmem``, its policy will +default to ``never``. + Hugepages in tmpfs/shmem ======================== diff --git a/mm/shmem.c b/mm/shmem.c index dfcc88ec6e34..d2bf98aece40 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -136,6 +136,7 @@ static unsigned long huge_shmem_orders_always __read_mostly; static unsigned long huge_shmem_orders_madvise __read_mostly; static unsigned long huge_shmem_orders_inherit __read_mostly; static unsigned long huge_shmem_orders_within_size __read_mostly; +static bool shmem_orders_configured __initdata; #endif #ifdef CONFIG_TMPFS @@ -5027,7 +5028,8 @@ void __init shmem_init(void) * Default to setting PMD-sized THP to inherit the global setting and * disable all other multi-size THPs. */ - huge_shmem_orders_inherit = BIT(HPAGE_PMD_ORDER); + if (!shmem_orders_configured) + huge_shmem_orders_inherit = BIT(HPAGE_PMD_ORDER); #endif return; @@ -5195,6 +5197,107 @@ static int __init setup_transparent_hugepage_shmem(char *str) } __setup("transparent_hugepage_shmem=", setup_transparent_hugepage_shmem); +static char str_dup[PAGE_SIZE] __initdata; +static int __init setup_thp_shmem(char *str) +{ + char *token, *range, *policy, *subtoken; + unsigned long always, inherit, madvise, within_size; + char *start_size, *end_size; + int start, end, nr; + char *p; + + if (!str || strlen(str) + 1 > PAGE_SIZE) + goto err; + strscpy(str_dup, str); + + always = huge_shmem_orders_always; + inherit = huge_shmem_orders_inherit; + madvise = huge_shmem_orders_madvise; + within_size = huge_shmem_orders_within_size; + p = str_dup; + while ((token = strsep(&p, ";")) != NULL) { + range = strsep(&token, ":"); + policy = token; + + if (!policy) + goto err; + + while ((subtoken = strsep(&range, ",")) != NULL) { + if (strchr(subtoken, '-')) { + start_size = strsep(&subtoken, "-"); + end_size = subtoken; + + start = get_order_from_str(start_size, + THP_ORDERS_ALL_FILE_DEFAULT); + end = get_order_from_str(end_size, + THP_ORDERS_ALL_FILE_DEFAULT); + } else { + start_size = end_size = subtoken; + start = end = get_order_from_str(subtoken, + THP_ORDERS_ALL_FILE_DEFAULT); + } + + if (start == -EINVAL) { + pr_err("invalid size %s in thp_shmem boot parameter\n", + start_size); + goto err; + } + + if (end == -EINVAL) { + pr_err("invalid size %s in thp_shmem boot parameter\n", + end_size); + goto err; + } + + if (start < 0 || end < 0 || start > end) + goto err; + + nr = end - start + 1; + if (!strcmp(policy, "always")) { + bitmap_set(&always, start, nr); + bitmap_clear(&inherit, start, nr); + bitmap_clear(&madvise, start, nr); + bitmap_clear(&within_size, start, nr); + } else if (!strcmp(policy, "advise")) { + bitmap_set(&madvise, start, nr); + bitmap_clear(&inherit, start, nr); + bitmap_clear(&always, start, nr); + bitmap_clear(&within_size, start, nr); + } else if (!strcmp(policy, "inherit")) { + bitmap_set(&inherit, start, nr); + bitmap_clear(&madvise, start, nr); + bitmap_clear(&always, start, nr); + bitmap_clear(&within_size, start, nr); + } else if (!strcmp(policy, "within_size")) { + bitmap_set(&within_size, start, nr); + bitmap_clear(&inherit, start, nr); + bitmap_clear(&madvise, start, nr); + bitmap_clear(&always, start, nr); + } else if (!strcmp(policy, "never")) { + bitmap_clear(&inherit, start, nr); + bitmap_clear(&madvise, start, nr); + bitmap_clear(&always, start, nr); + bitmap_clear(&within_size, start, nr); + } else { + pr_err("invalid policy %s in thp_shmem boot parameter\n", policy); + goto err; + } + } + } + + huge_shmem_orders_always = always; + huge_shmem_orders_madvise = madvise; + huge_shmem_orders_inherit = inherit; + huge_shmem_orders_within_size = within_size; + shmem_orders_configured = true; + return 1; + +err: + pr_warn("thp_shmem=%s: error parsing string, ignoring setting\n", str); + return 0; +} +__setup("thp_shmem=", setup_thp_shmem); + #endif /* CONFIG_TRANSPARENT_HUGEPAGE */ #else /* !CONFIG_SHMEM */