From patchwork Mon Sep 25 20:01:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zach O'Keefe X-Patchwork-Id: 13398384 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5B7F8CE79A1 for ; Mon, 25 Sep 2023 20:01:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5ACEB6B00DE; Mon, 25 Sep 2023 16:01:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 55B386B00E5; Mon, 25 Sep 2023 16:01:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3FC4D6B00E8; Mon, 25 Sep 2023 16:01:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 2AF686B00DE for ; Mon, 25 Sep 2023 16:01:22 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id DBFCF8078C for ; Mon, 25 Sep 2023 20:01:21 +0000 (UTC) X-FDA: 81276189162.01.3E39D4A Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) by imf27.hostedemail.com (Postfix) with ESMTP id 1D5E84002F for ; Mon, 25 Sep 2023 20:01:18 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=StnatUfl; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf27.hostedemail.com: domain of 3DecRZQcKCFIJ84yyzy08805y.w86527EH-664Fuw4.8B0@flex--zokeefe.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3DecRZQcKCFIJ84yyzy08805y.w86527EH-664Fuw4.8B0@flex--zokeefe.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1695672079; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=zmn8UWXrJoykttF2WL0ItLwCoiyMGCPwIkseCGByc8Y=; b=KTTPrMLTFERHEkEkN+4vzZzij4XP6ofCIztnVllAgnW84uUDRjM82uqsfkiz1aK2AjfyO7 KzTttfQ9UxE/ToXU0pYERXRNOLvYpDkz+6xSm9DQA7+wI6BjdJntTk8//64yj6nI1KiyyQ pPBeYLqXi5mqrBfU3DgfQeL5ME0gjlg= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=StnatUfl; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf27.hostedemail.com: domain of 3DecRZQcKCFIJ84yyzy08805y.w86527EH-664Fuw4.8B0@flex--zokeefe.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3DecRZQcKCFIJ84yyzy08805y.w86527EH-664Fuw4.8B0@flex--zokeefe.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1695672079; a=rsa-sha256; cv=none; b=TeXxjHXyjd2cvLLgZYrnYMBLru42EkKDOvqOIyv2vFMEoIjcCklj/fUH/z/jepfL4RwuUH HsOFR61AOtC+vUZWyt5xXnYkJz6xopzC7IdSgG/sw9gymcK8d0RwMGJlrkzy7UqIJwm67g TJ7eTnh89NIxvMEVmTAYQuk8lUVMk+U= Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-59f67676065so65768527b3.0 for ; Mon, 25 Sep 2023 13:01:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1695672078; x=1696276878; darn=kvack.org; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=zmn8UWXrJoykttF2WL0ItLwCoiyMGCPwIkseCGByc8Y=; b=StnatUflNQ+5b1zLM7qXFF9DNGmt4E5MDxNd71wA39rCl34PfMrrj5w0ClRTuN4Yve yXFmEqwbSCcwKQkg0z0YyaajEUvchhEkIYs+rrN3Z74LHfhTmbeOXCrkwb8wCNAtl+HS 5dDof0USPWxcB5AnYd/YWcADIgBLzWmTRxUSCv3bNzyW+qF/c15cha3UnI64T5PXgSsl lGv9tDTWtZNch00s7V0MMC/q+Wu9oKP/FrrZUwj5fEz55SUNIYg9OmYYUEy68qEvc8xv 1fDdndghY+x5NFWiaz4r+zUoECHMPeNoPcAzGs3EFoi+gK4WTIMn6QAOH92SXIKjBUhJ txoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695672078; x=1696276878; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=zmn8UWXrJoykttF2WL0ItLwCoiyMGCPwIkseCGByc8Y=; b=PPsGIBClsEolPgrn1ochMthz3fVGEczzjzKtjeJmW6RvZyXYXWRaco8M2HaxVNZSSB o+fcA5ip5BjUK8EqKRc6KxZLQCQ9hj572A1V8NoQePEAzfjJReNux4PJRRLfC+abOjlr RUWgXQMd54wt7jZPle3ZyjHHhnpQ7KLmHZXW73JZHeo+A3UqklaMizP0BfdR60FUKP++ iOK3XSwtRzv9eh6JhJdgd0iQhwOotxzR0Yv+mXxBTNZB4/9qQX2sqbStlEnc36Vk4ySH HlpiZZLEQl/1oAYSNjlqh6184OuGn9CuwAEf4rtHkZV8Nh6RSjfWpXZW72Gi9C75oENu XtJQ== X-Gm-Message-State: AOJu0YzDfw6R2mK4JVajaw9sAEle/F2TUU2/ildVUqg0c2F7MMxA1Lbv eRQ2aiLRtIxpj8hFmTZpFjJ1TB+3aN5x07fTgiwtiPO2x/EDkxVs4Wn9/sYh5mBGzcaGQAwrFVz FiGVQEJwNayPg1OwEHBMRYtRsTxPm7PPI3WwMj+F7knUVO0rhYnn1ZI7qbeE= X-Google-Smtp-Source: AGHT+IHynGz8uTuhPCcteQ9fRpzZKQn4SfmpahOj3xEXuSYuZ5PekHFLjW28hJJbizPAzZP6dnw3loQr/g5+ X-Received: from zokeefe3.c.googlers.com ([fda3:e722:ac3:cc00:20:ed76:c0a8:2971]) (user=zokeefe job=sendgmr) by 2002:a81:e503:0:b0:59b:e81f:62ab with SMTP id s3-20020a81e503000000b0059be81f62abmr92301ywl.7.1695672077978; Mon, 25 Sep 2023 13:01:17 -0700 (PDT) Date: Mon, 25 Sep 2023 13:01:10 -0700 Mime-Version: 1.0 X-Mailer: git-send-email 2.42.0.515.g380fc7ccd1-goog Message-ID: <20230925200110.1979606-1-zokeefe@google.com> Subject: [PATCH v4] mm/thp: fix "mm: thp: kill __transhuge_page_enabled()" From: "Zach O'Keefe" To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, "Zach O'Keefe" , Saurabh Singh Sengar , Yang Shi , Matthew Wilcox , David Hildenbrand X-Rspamd-Queue-Id: 1D5E84002F X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: 1enyy9e5w3i1txy1ey4o73i5hjzcsxpj X-HE-Tag: 1695672078-656349 X-HE-Meta: U2FsdGVkX1+DRqvTrhDcK3KYV7ZkZWhdcVhpsHmT6wYiCAhwfwH7EIICCsTJTYxMjuIR4lfZzIbDO90zf2I/cW8MBiWihx3OGwM0fDj49hT7/gOC6qEX7hJzXl2AJZnUI7QE40iKZStLJ2oGiKWPHorG4ajj3Y+Jj0PfZHUXMcTT70cXRjcX5Jzk5FKKcLy8vsXnWHty3Jk9gVtb6tdCRWR1yMa8G9mlsRHpr0e/Mmj0xellGTVZgou193xqOkPb3BRlnXQjcTSlEwBqSbMUYg/9GCmQu3uGRaZeamIyPJMtn7UBMWDuQfrLoaXb4hKj8EEcddYPHA8L0igt5wNNhHeyf5INgUbQL72Y2Sn7iNWqU+AtxDhhg+svoO//iWgXxesnde7GerSMKlQIRVkSNqMpoZVI5SyPPQVCxh1tPA6XO1BSwJzLvdqbcMlDyb2eecXg6bRryOZ9+fVrVgNAJ8ycqJh99c2VXjW0oPAhfwU/wp4ltiw7x00VZ0HoAXvyIkLFPQeEMjqYGlucb9B4Ic32xmOTkWsRLmD1vztG4CA8/Ip8snCnGvxuc7fXcGF4WUxTjafUrlc/3xY+WuY3LhWkVQjasg9vtPFwTmyH2sxaF9rMjwNvt68sZo83tIPCiwnGWxp2b3S3fI9/wISx5baf7bG5/6f9jQWrsBOscCqrxBa6sqgIZd718alO5DMn01/PWMZ1tbwnpDEzL1pBEW1ybHIn7KMvXlfP8bPfr4ijpJ9dNqLaTbD2VSpaWE90YVrpM8vcCyMUk3pkUHdcmNmchuO2TZSOWXlKQ2wTHBzd2n1H8zmQsQKpVYt/ABSX1Eq39qO49jfLxgc6DnPHfdvUFiBiwwqcFyq2J8qI1ShKmOCItS8CcQ9oD2s5hbR2OTT5lVFQRj7xrKlEoOYhc655j/QBBbr9oRtjv8OIRKcHs1B8xBgIDeLA9t6DQowONYaZw1rZzNRJcDPQYsp 5ktLrrSQ ECIi6YejHUeCPlf/O3hKUizv/nJ7/wFQZir8xcjLZlNPOLtuSVS6KgGy/tBeIty2TVTUIFal+rmN5qUx/QlQidqOt2ZzPd7rmZfPJvGjDwhLieGEvHNwwtKjI0dU04yPMZS1kDtBz7vmbAxByROlZxCVs9ioLuYFvkGibMs9C61MmleOJYsRkQCjMgHr1gbhGmMj10dntfB2U5lzw/0ax8TZLXadihj7p1+PlRA5ZKURLDSxdfJ0+Men1l7btEZNWPkAKW2GZ+K1NSjHiYS8+qq/w/BUgyPCfX3PcwcgLuNNxXMrEHHBl4wlj32CniJhg/BZ+H99pSb+F5B7b9k+EcRvoYpyb7nDNibXW5QMxWvME9hctNsyd/HRw5qBcf01HZmWrtufGuQla1w/u/QEnLLIAohDEoV2CGitvlzMUKERhGH1I7oR0NQ75jeFYQV+w8W/QznzwgOoKV9YfAPI8Ew0nl3ezRMJIAbsA1a5pItEQydN1hJVOFlrCtyxRM3cgBJ+/r0XVIDz8zenjvUeSJI/qu0FBr4LCvLQUnrmRIPEX+W1O7cT+g9aqimwygtdA/Tft7yj/D23eNiyGEktHyNcs7CE5l8fzeuNyMJHbl0vHi6j7YL/RXWulIBMx8ejFteIDMz6nc5aqcSRi+4BbD6KzfWZEBI1ji3+91n+WTaqSyvPqMmWMoa6mIYmCymNVapDN5Jtd3wZDNqjgiuoStByEcVVACAh0S9DZ81FeE9ZpXzFXuLDShDh6o+HqsOnFR7Ox3+hJBzaO3xe7hyhazwb5nOtx+wuq6PYf1H1ys+imQREN8jY3t1g751GG/NMjpDtqA0TmytHVexzCA9SmtRrfuCFvS1yikLKB X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The 6.0 commits: commit 9fec51689ff6 ("mm: thp: kill transparent_hugepage_active()") commit 7da4e2cb8b1f ("mm: thp: kill __transhuge_page_enabled()") merged "can we have THPs in this VMA?" logic that was previously done separately by fault-path, khugepaged, and smaps "THPeligible" checks. During the process, the semantics of the fault path check changed in two ways: 1) A VM_NO_KHUGEPAGED check was introduced (also added to smaps path). 2) We no longer checked if non-anonymous memory had a vm_ops->huge_fault handler that could satisfy the fault. Previously, this check had been done in create_huge_pud() and create_huge_pmd() routines, but after the changes, we never reach those routines. During the review of the above commits, it was determined that in-tree users weren't affected by the change; most notably, since the only relevant user (in terms of THP) of VM_MIXEDMAP or ->huge_fault is DAX, which is explicitly approved early in approval logic. However, this was a bad assumption to make as it assumes the only reason to support ->huge_fault was for DAX (which is not true in general). Remove the VM_NO_KHUGEPAGED check when not in collapse path and give any ->huge_fault handler a chance to handle the fault. Note that we don't validate the file mode or mapping alignment, which is consistent with the behavior before the aforementioned commits. Fixes: 7da4e2cb8b1f ("mm: thp: kill __transhuge_page_enabled()") Reported-by: Saurabh Singh Sengar Signed-off-by: Zach O'Keefe Cc: Yang Shi Cc: Matthew Wilcox Cc: David Hildenbrand Acked-by: David Hildenbrand --- I've updated the changelog to reflect discussions in [1] -- leaving ack to David / Matthew on whether to take the patch. Changed from v3[1]: - [akpm / David H. / M. Wilcox] Updated log to capture email discussion Changed from v2[2]: - Fixed false negative in smaps check when !dax && ->huge_fault Changed from v1[3]: - [Saurabhi] Allow ->huge_fault handler to handle fault, if it exists [1] https://lore.kernel.org/linux-mm/20230821234844.699818-1-zokeefe@google.com/ [2] https://lore.kernel.org/linux-mm/20230818211533.2523697-1-zokeefe@google.com/ [3] https://lore.kernel.org/linux-mm/CAAa6QmQw+F=o6htOn=6ADD6mwvMO=Ow_67f3ifBv3GpXx9Xg_g@mail.gmail.com/ --- mm/huge_memory.c | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 0f93a73115f73..797fe617e51ab 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -100,11 +100,11 @@ bool hugepage_vma_check(struct vm_area_struct *vma, unsigned long vm_flags, return in_pf; /* - * Special VMA and hugetlb VMA. + * khugepaged special VMA and hugetlb VMA. * Must be checked after dax since some dax mappings may have * VM_MIXEDMAP set. */ - if (vm_flags & VM_NO_KHUGEPAGED) + if (!in_pf && !smaps && (vm_flags & VM_NO_KHUGEPAGED)) return false; /* @@ -132,12 +132,18 @@ bool hugepage_vma_check(struct vm_area_struct *vma, unsigned long vm_flags, !hugepage_flags_always()))) return false; - /* Only regular file is valid */ - if (!in_pf && file_thp_enabled(vma)) - return true; - - if (!vma_is_anonymous(vma)) + if (!vma_is_anonymous(vma)) { + /* + * Trust that ->huge_fault() handlers know what they are doing + * in fault path. + */ + if (((in_pf || smaps)) && vma->vm_ops->huge_fault) + return true; + /* Only regular file is valid in collapse path */ + if (((!in_pf || smaps)) && file_thp_enabled(vma)) + return true; return false; + } if (vma_is_temporary_stack(vma)) return false;