From patchwork Fri Nov 15 21:52:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kalesh Singh X-Patchwork-Id: 13877363 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 685E2D68BDA for ; Fri, 15 Nov 2024 21:53:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CEA5C6B00C9; Fri, 15 Nov 2024 16:53:02 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C98076B00CC; Fri, 15 Nov 2024 16:53:02 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B395E6B00CD; Fri, 15 Nov 2024 16:53:02 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 923626B00C9 for ; Fri, 15 Nov 2024 16:53:02 -0500 (EST) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 22B8BC0486 for ; Fri, 15 Nov 2024 21:53:02 +0000 (UTC) X-FDA: 82789678440.06.87BE94B Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) by imf16.hostedemail.com (Postfix) with ESMTP id DAB80180013 for ; Fri, 15 Nov 2024 21:52:14 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=ikhzQfF0; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf16.hostedemail.com: domain of 3u8I3ZwsKCGAI8JCQFQGLEFEMMEJC.AMKJGLSV-KKIT8AI.MPE@flex--kaleshsingh.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3u8I3ZwsKCGAI8JCQFQGLEFEMMEJC.AMKJGLSV-KKIT8AI.MPE@flex--kaleshsingh.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731707401; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=F3Lr4WdOKy4VbAqfpb23fqAMFS0TE/YrDm6ZBGH4WtQ=; b=FOnbg1TiSdRxd95lp4A5yGQsF0ereItzR+fjBv6MtRTJKjlX0fu+3rEioseHapP4/5vB98 VYyC4CIiQ4e7zrMTVXM8FKFe3S53TAhyUSP8VRspAv/rA9tkTbXYTpYj9B2d0C8tglEIiY jxan9kj/9VtzLVPfZfn9HCc8SbEttPw= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=ikhzQfF0; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf16.hostedemail.com: domain of 3u8I3ZwsKCGAI8JCQFQGLEFEMMEJC.AMKJGLSV-KKIT8AI.MPE@flex--kaleshsingh.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3u8I3ZwsKCGAI8JCQFQGLEFEMMEJC.AMKJGLSV-KKIT8AI.MPE@flex--kaleshsingh.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731707401; a=rsa-sha256; cv=none; b=uh4zKRDTfzgql2QV1BDFKUDD55RArcwpyw03KBR0dDRT237ddcHZBYik0Z9xK8dggYNqsG rzvZk6Z+jwNvuT6aJ7JhX8AyzNVksKd2mcqGaidrwgCTaV4Pmu27MFDhctqBKyXv+pfGrV 2a+8JY00PA/U9/ApH6I0r2Lo7ap46/A= Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6ee484ec18aso37706027b3.2 for ; Fri, 15 Nov 2024 13:53:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1731707579; x=1732312379; darn=kvack.org; h=cc:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=F3Lr4WdOKy4VbAqfpb23fqAMFS0TE/YrDm6ZBGH4WtQ=; b=ikhzQfF0k9wQAEgtQ1vcLK7bxFx85GraWDeJYWXpUKteVXpGw2AiygwOgs6po9o4Mb jsTQBLdsqomR7/F2ZOEbZMUQ5k1HaLwdRD4UdRNIISdvnZJP9qrX+t4BBUcDg/cpHCTc ybtGb7B+ZYvtfiBNwlLEen52r77/qssQmk4j1cqX3ohrsv1gY2EAYL511srvUNQkBnG7 PCzS61pC+Gv5TTMyS9+Nbp/7bsXyDeuOGQud5iqIs28lKVsD/mlV8QOwbTl/VH1aynJ3 r52bYlqt4L6zgXI5rroQMVN73pRWga0dz7O5Fg8+6adBbpFBmcGVts0egtGAXUjsVKa8 yTQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731707579; x=1732312379; h=cc:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=F3Lr4WdOKy4VbAqfpb23fqAMFS0TE/YrDm6ZBGH4WtQ=; b=BhdcLkzqgA8j/V9qksY6VZ/mHCx3kNPQkd49gmzfWU7wLHm9JJQWFTg3S6yrUBO3qD rLaOvyCoaiAl/12m2D1FeUX1tRxppt3Cxs5Qkc6okX8lgdVM4xeKbRdDNiGUkND9fTsp NAxZJyQvIE3HaTOs4NxHSd/4YI6oU0LfPLprSaf7AK59LWZdAGl1B7BUXWYXGss0CQO/ lyki6DDmIqhJtuXHIi9dZVUL8BJEVZAucnl/svuL4F2GbQMBnxPzzqNzRzQm6GQxqXx1 V7NF1yrhF4Ta+7aqmhM7WyX6MtX5LvODr1X4wFWkul+yRmHoSDJIxY/AHPM4YjBVcKiW eSNQ== X-Forwarded-Encrypted: i=1; AJvYcCX3QA6dagOynZUo+mpTU4o5v+TNqNgoIOtxNK4ZSVSVh2ASAvSBVzZszfHY+wbxaXtnzK98WmUzUA==@kvack.org X-Gm-Message-State: AOJu0YyQcQxN/jeRg7V9FkllL70g2du+x2sBs+0RP+VhYqMwZDSd4H1h wAEEjDriFB7DxplTDO/x30ONIAz3RaaG/fsz8usUJVEQCNGW7Du63A+rOh3S7060g7jlwHSHr4U ACr5cmmn6ozyG+QMBfLKCEg== X-Google-Smtp-Source: AGHT+IH2lTnSTmkXeTH4jQY5dTMIBM7aAaY2KmefhKNRgP76mCWxBocb32PIi9hyqSL27HkXkJlJGHgg1gmJMiKP0A== X-Received: from kalesh.mtv.corp.google.com ([2a00:79e0:2e3f:8:fabe:251a:db8:7fc4]) (user=kaleshsingh job=sendgmr) by 2002:a05:690c:8f07:b0:6ea:34c3:742b with SMTP id 00721157ae682-6ee55c7a403mr497207b3.6.1731707579432; Fri, 15 Nov 2024 13:52:59 -0800 (PST) Date: Fri, 15 Nov 2024 13:52:53 -0800 Mime-Version: 1.0 X-Mailer: git-send-email 2.47.0.338.g60cca15819-goog Message-ID: <20241115215256.578125-1-kaleshsingh@google.com> Subject: [PATCH] mm: Respect mmap hint address when aligning for THP From: Kalesh Singh Cc: kernel-team@android.com, android-mm@google.com, Kalesh Singh , Andrew Morton , Vlastimil Babka , Yang Shi , Rik van Riel , Ryan Roberts , Suren Baghdasaryan , Minchan Kim , Hans Boehm , Lokesh Gidra , stable@vger.kernel.org, "Liam R. Howlett" , Lorenzo Stoakes , Jann Horn , Yang Shi , linux-mm@kvack.org, linux-kernel@vger.kernel.org X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: DAB80180013 X-Stat-Signature: g4q1krh1zuoh369iwn1jp7dow7okn9ad X-Rspam-User: X-HE-Tag: 1731707534-761584 X-HE-Meta: U2FsdGVkX1+bD/m9h1mOvkE4dhkehCQct9yJ0LFqgjQxGxU/MSF5p+zjoW5wQ/tcOcnkG7gPNeBaxZlZa3EDF42w/2Etmp4iPBI3YZJ+/ldCKXswNO6wdfkmwclrMVZyhsE/Yjfu9Kdnc9/wqpbS5n5e/spYobRM7UmfIP1foH739y90cj0vnPx1TDxL1W5MVy3J3Ocg3AvTfCING5Kpp3sJgoLtMa6EYsHabFR7zZhTwePKWOLn8+bmqClyRoZ8zCdLZx5vkgOxzFzZhZv/2jrH7XRk5XqvVgWlK3/Wh7s7JmTiJVVSpt6F8e1all2pHszEH3eTrsRz5WdnLX9Bkl5ZEhBT1JvhcijI61f+nVWRxmRgW+0qvI+OTAa5j6pPS6ioUmE/d07NP8UgzbrqPCUds+86tZ+4kEG+DlJQveWN/af4QcPoVnrSYMHAdkwdfTvK6VR7GDJZ6a3+k4UK01A65o+Bct7j8k/jPmzQ2a0y+nbRDw/j5nI/YKVcr3A+sSfQOWwicLdniUVdpRP9otn3dtdyQdwhwH2IelqCreKA9VKMft1fpjdX8G5bsbPLggF4CnrEyneM6sOr5LDsrmJRR29hNV80pwiweVa9qRna5WwDF9X1F4PhiDSzaFP46Wv79RssErRr0e7NZG9bd87nRwSYWgWHM231jAYl6wKtVgYTVZx1aQkKobdYy5nco8rU1Aye7Ac1LDA6RrIuUvK9BNTnQhPWhVXJUOFNggzeLceqyWBJiVyiTZp3c3/GU7jVOp93TEQPCZ7HHlNgrtrZUG2SSFdxMvg9EEpvLOWCeZlVFuOjyg8jug9BQMmCYiH7ktSuq/Zm1FKq2wOun2btf0b7OaEH66beN/i4oyLqIzcMsKz7B6IyhEjvAoVyBHurScGFz0BqLaoxx4uPJgVLgjZitGwmhFovHnf5mUvVXduqaAwRUzod1CIhibwtlOlept7DveEKAvw8pj0 hxdDTv1T kWVOUYCpE9gaI3MqPOQpzyEdYJKzuZHTNKjdggrJ3rZL3pLLSduzOEnr/NpFF5wiEmKI0Bsj7RZvH8oNIQR8fGYfOio2T99/X1583fvhem/eXu3LXsqq/lIi2d/tQ3DzEbJz6h8xqfN7qwv3q2U1zkZrIp7j+qeZdQW+IDki6jaYhnRQjQSumsG7Eia+pF/2y8NzWrtnxKBC3GjDXW5b/7HosVLr5WVHZZ3DNARBz2/pnmyz4Vs6t//eP9HoZwzp2tHy7v6SYDR3uzRItSAM2HLUtMx9HDpDv9B7HPz9Yg94Y1cKf5MEx0a7NqpiACLnZiOdZjAbYAPcNmY2VBacHwuQPUlUEfoOiYlO4tBkeFSAJu9MJJx7LhG4eE4pJ+wwgDDQlRz+8q0IPbpqKp2BjoPzA3O22F6aBCyixMDgUCRLcJa6yFr84Cx14Ysc024r6+744GYxs+8sEqZT9aDIk3E5atNGt1eA1LDochwDWnNMgUEXdjoGrdLOWp8bgyDRvvRiiOZt9rJnv3BYVqAD8EtyhIEf2vYZJb8zHr4uL4B7L/ufprknCplkYmO9/g/sEUDkA4t48psrst1ItE+ujtUXVlhu4RLmnHZojJTt5bDIuyWyyuofXoPO68es7N5NxpHG/t9rsF+1m5H2v5nshecIB/fPqg6NolQlhxjCpO8WW501+oj9zI2hw2yl/2zrWOZLL8Hi2vM97U8zCuAqVmthOiWDgT0Fap0sdf4djcz5AJl6rTfSbHeaBa3STye4j6eFLjhrghWeF7mQGHXZNHU41yIAcCB/RgdBGeetUQSl0iVqX08+b6MtLjDuHEW3i9N1c0qPm8EDwlBil9Rb1Qlp+t8smtmxfmq1kjgLzGkl073iktC5ajAglG+9D1eOkygn9ub1xf8JTeps= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Commit efa7df3e3bb5 ("mm: align larger anonymous mappings on THP boundaries") updated __get_unmapped_area() to align the start address for the VMA to a PMD boundary if CONFIG_TRANSPARENT_HUGEPAGE=y. It does this by effectively looking up a region that is of size, request_size + PMD_SIZE, and aligning up the start to a PMD boundary. Commit 4ef9ad19e176 ("mm: huge_memory: don't force huge page alignment on 32 bit") opted out of this for 32bit due to regressions in mmap base randomization. Commit d4148aeab412 ("mm, mmap: limit THP alignment of anonymous mappings to PMD-aligned sizes") restricted this to only mmap sizes that are multiples of the PMD_SIZE due to reported regressions in some performance benchmarks -- which seemed mostly due to the reduced spatial locality of related mappings due to the forced PMD-alignment. Another unintended side effect has emerged: When a user specifies an mmap hint address, the THP alignment logic modifies the behavior, potentially ignoring the hint even if a sufficiently large gap exists at the requested hint location. Example Scenario: Consider the following simplified virtual address (VA) space: ... 0x200000-0x400000 --- VMA A 0x400000-0x600000 --- Hole 0x600000-0x800000 --- VMA B ... A call to mmap() with hint=0x400000 and len=0x200000 behaves differently: - Before THP alignment: The requested region (size 0x200000) fits into the gap at 0x400000, so the hint is respected. - After alignment: The logic searches for a region of size 0x400000 (len + PMD_SIZE) starting at 0x400000. This search fails due to the mapping at 0x600000 (VMA B), and the hint is ignored, falling back to arch_get_unmapped_area[_topdown](). In general the hint is effectively ignored, if there is any existing mapping in the below range: [mmap_hint + mmap_size, mmap_hint + mmap_size + PMD_SIZE) This changes the semantics of mmap hint; from ""Respect the hint if a sufficiently large gap exists at the requested location" to "Respect the hint only if an additional PMD-sized gap exists beyond the requested size". This has performance implications for allocators that allocate their heap using mmap but try to keep it "as contiguous as possible" by using the end of the exisiting heap as the address hint. With the new behavior it's more likely to get a much less contiguous heap, adding extra fragmentation and performance overhead. To restore the expected behavior; don't use thp_get_unmapped_area_vmflags() when the user provided a hint address. Cc: Andrew Morton Cc: Vlastimil Babka Cc: Yang Shi Cc: Rik van Riel Cc: Ryan Roberts Cc: Suren Baghdasaryan Cc: Minchan Kim Cc: Hans Boehm Cc: Lokesh Gidra Cc: Fixes: efa7df3e3bb5 ("mm: align larger anonymous mappings on THP boundaries") Signed-off-by: Kalesh Singh Reviewed-by: Rik van Riel Reviewed-by: Vlastimil Babka --- mm/mmap.c | 1 + 1 file changed, 1 insertion(+) base-commit: 2d5404caa8c7bb5c4e0435f94b28834ae5456623 diff --git a/mm/mmap.c b/mm/mmap.c index 79d541f1502b..2f01f1a8e304 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -901,6 +901,7 @@ __get_unmapped_area(struct file *file, unsigned long addr, unsigned long len, if (get_area) { addr = get_area(file, addr, len, pgoff, flags); } else if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) + && !addr /* no hint */ && IS_ALIGNED(len, PMD_SIZE)) { /* Ensures that larger anonymous mappings are THP aligned. */ addr = thp_get_unmapped_area_vmflags(file, addr, len,