From patchwork Sat Feb 18 00:27:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Houghton X-Patchwork-Id: 13145374 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E8581C05027 for ; Sat, 18 Feb 2023 00:29:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 853FB280004; Fri, 17 Feb 2023 19:28:54 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7B37D280002; Fri, 17 Feb 2023 19:28:54 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 67B64280004; Fri, 17 Feb 2023 19:28:54 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 5A0B3280002 for ; Fri, 17 Feb 2023 19:28:54 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 375231C5DF5 for ; Sat, 18 Feb 2023 00:28:54 +0000 (UTC) X-FDA: 80478527388.29.D5474B0 Received: from mail-vk1-f202.google.com (mail-vk1-f202.google.com [209.85.221.202]) by imf29.hostedemail.com (Postfix) with ESMTP id 6656B120013 for ; Sat, 18 Feb 2023 00:28:52 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b="f9I/sF01"; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf29.hostedemail.com: domain of 3wxvwYwoKCOALVJQWIJVQPIQQING.EQONKPWZ-OOMXCEM.QTI@flex--jthoughton.bounces.google.com designates 209.85.221.202 as permitted sender) smtp.mailfrom=3wxvwYwoKCOALVJQWIJVQPIQQING.EQONKPWZ-OOMXCEM.QTI@flex--jthoughton.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676680132; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=EVR7nxu9NqFl8iop5UBI5tIYqVwEs8TwTl7GvIiAmDc=; b=27lrWlb58LbbO13Y7z9rtr8Nu0APsF+fL0rkV9+yMY8Qw6JazZqaPrqBWUI5PlQ03ad6OH 1jxUpkxu3odKLAvownzUoEF/HANRH6EvE4vg3jsPae23fHIYtP95CAbf13nnlrohGrY+3b 7FhWYrxJPIho36mcQeFKVhTYB+sH+co= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b="f9I/sF01"; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf29.hostedemail.com: domain of 3wxvwYwoKCOALVJQWIJVQPIQQING.EQONKPWZ-OOMXCEM.QTI@flex--jthoughton.bounces.google.com designates 209.85.221.202 as permitted sender) smtp.mailfrom=3wxvwYwoKCOALVJQWIJVQPIQQING.EQONKPWZ-OOMXCEM.QTI@flex--jthoughton.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676680132; a=rsa-sha256; cv=none; b=ZGq+xQtUwMVfTzBbOZ7F1mD91UPfGuTkVbs0ZKaeY71Mbj3ZjUfPxAm30m1kG3m4JdMCTa Him0bT7fldAb3fYSzUO0xg9kX7oNQ1lTdkcNeRZUJrWBDP/FE37a25yaIwv3tUWfffGS/m 7wURPl5T10VTAhbVQZNstJ8iKzbe8cM= Received: by mail-vk1-f202.google.com with SMTP id o73-20020a1f414c000000b0040163d749ecso646417vka.11 for ; Fri, 17 Feb 2023 16:28:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=EVR7nxu9NqFl8iop5UBI5tIYqVwEs8TwTl7GvIiAmDc=; b=f9I/sF017TCFQ06v8JVGvP2Gz6zjUBu/YI/6tPRqHUhBratYAP73U723CvkiNRmX+2 Xq1dOa8TR7SCr3LVAGWBGIMiSfJg1XkFZM5gm/al2A53lbtwLqPsmpTxwC6EJWviOgyV 6ZoThnT1qOoxqJ91hnxCH3KMvw34Z4n+qBZp+x8ogItt4N49rc+X+Kia1yaFeZctlPrt dHqaxn4yVk5u2oBjIzFDTnhm8t11FS1zIMAY2JsmW8Lyk6cGaRH4PRACzyyvtyhEDhjv 2AMBPlp0rxu60Rp9Xuc47FP4zxAkrddmeJ3HXqOflP02EXV117YxiSCeFjnJ8DA0bJ1V pWMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=EVR7nxu9NqFl8iop5UBI5tIYqVwEs8TwTl7GvIiAmDc=; b=LAptAKojHLHIdgm5LPD6bg7ps6Nhy6w5t6H00e9J/Gu2GlEavxXL/MbE2JanMWCYuw Iv6rXdPPlPWrQaeMrtokme5KA5rz7DZTt/sYKmhaXkruIZKkOEvD4oS/LGdecoF5/H6G lC4hZ6fWX+HS2obMIJCgx3uvQVpelN6KSxX6+69otMBsvxmwb1trZKNZcwyz56EE2Vrx Fu9Jsbzd91uqdpt9AYswcJ8zqTdH7XXaZF1VohAMG2CqDEOyGExLEa42F4stD6pGoh6c iveqq8w6X3lG7gt0bEo4S9+Rt4mxd2m1Him4BstBpwqG/oFn6VEqFY4+DhfTznG5N5Pu N7Uw== X-Gm-Message-State: AO0yUKV8zbTh2iQVzWTsGQ4Rrus1lNxyZgOWLrmEak1FIXlWufZiGg+f B/5WH/zkb68yyeIqFKHRPm0iHwEYUrRIXF9A X-Google-Smtp-Source: AK7set+7kve29zHBIESSAgNBn7bVNkTyjU2Wm0RcYZpG6xt7UjvZapCdpaj7ibFEOjWwLks8K3N9f134pdwQEDVw X-Received: from jthoughton.c.googlers.com ([fda3:e722:ac3:cc00:14:4d90:c0a8:2a4f]) (user=jthoughton job=sendgmr) by 2002:a1f:21c4:0:b0:406:983c:e6de with SMTP id h187-20020a1f21c4000000b00406983ce6demr657672vkh.1.1676680131643; Fri, 17 Feb 2023 16:28:51 -0800 (PST) Date: Sat, 18 Feb 2023 00:27:41 +0000 In-Reply-To: <20230218002819.1486479-1-jthoughton@google.com> Mime-Version: 1.0 References: <20230218002819.1486479-1-jthoughton@google.com> X-Mailer: git-send-email 2.39.2.637.g21b0678d19-goog Message-ID: <20230218002819.1486479-9-jthoughton@google.com> Subject: [PATCH v2 08/46] hugetlb: add HugeTLB HGM enablement helpers From: James Houghton To: Mike Kravetz , Muchun Song , Peter Xu , Andrew Morton Cc: David Hildenbrand , David Rientjes , Axel Rasmussen , Mina Almasry , "Zach O'Keefe" , Manish Mishra , Naoya Horiguchi , "Dr . David Alan Gilbert" , "Matthew Wilcox (Oracle)" , Vlastimil Babka , Baolin Wang , Miaohe Lin , Yang Shi , Frank van der Linden , Jiaqi Yan , linux-mm@kvack.org, linux-kernel@vger.kernel.org, James Houghton X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 6656B120013 X-Stat-Signature: tq3fofgj1sahdn9ii1qtok6ikmgdwutk X-HE-Tag: 1676680132-730096 X-HE-Meta: U2FsdGVkX19DCOR6D25v5qRclyTwsPlLH5SFjaPyR9u0i0AgCUfsfAUI08Ns9fG7HMDMmqEzgV9/CWqm5csQoDAGwnRp4/9tFBoJ3IFGL94Z+94Y1h9+n7Twlez6bQ+TwIxsARIxcThLqTZbjbFKB6muStaOBgNeEcQ0uT0kQfGEBZJXCcvwgLBSRvyLJlupG0tAV9/S7PZeVQlTM+oyvDyrENNnVSO2lUD0COmcHP9xonS8CZ8SLa/pXpLT3IH+seW306h6ycY5C8/SCNJZLu5q5RNLh+b9fXi6Md+LJKVcPnQ9hYOmMkoIHTAPFugG9m5iXK8DplLuq0Kge3nTJgwnAItwEzBSl43lUG73/WDkiK+Hr1Pxb5TEKww5gyGd50bVttId5oJEHsOdBpxdk+ru1qEe/Emv+lwugfymmgdjBXXIlTqQAlxUp8lKE2xiX9m6yWCy4w7ejq+1aFRHtFPn0UtEz/ZN3/oY9Y4CDLc0xNzHjyk0h36Xea2z2AZqUlbANSShLRjn3rBTf6oeU4pOox2ydppdzc7igVpvQ+7fR4nhSaRrlZbPMUcb+NsZ9Wat8A1bAQbcZcSt2IYYB0rzhu8dAKSj2S6qaqzw8eXpRmp2LnJdPCdF34JIxbufv10v4IYmFInx6dLkr9Cmo0Pn1bgec7MHZ02gjf8bq9Tx3hEFAL6MEN2F8+HCi5wb0uiDMF4VkLKajHYyDvmElNAJ25LR1PKg0gZtOfK8cUsXghCLiLYQGKsc24iOjoCJ66AbL0wU3kSLcN+JV3t+lVm0RPNslUQovSPomxMVJaOhj8VsHWkFT8UBzyjPj5MiE8Bar0Sst7cf2VKefIZC9ZWdsiH4w4a1E5J9Cucvr9fhg3lPzxgRLC6RW1qj9rK94XxlKCNsiBXi9ECweUfjNNXQaBNwKITrtF4RArFb/7onZ6PNoIwSA+Ah/kZOBfrf/p3i/EPJ+dMsoKNty/R xbAMoqAh HlFXL5d963QztHN1RQui3Jn7r81F8S3DWB5n0ivh/HTSobAnq8cQYT+97kYxjvdOC4OMjFcW47GcTZGumMTUb//izVRHyBG0n5TBuruyX+64k953lA9QCunXVwXOgB1XrmKZxP+9jIuwqcNDEfZjduhwpzql4XyY6gv1KKmKbjy4iMsweOITX9GHqUnXWl71PwtiUfdcNI7meSCLdxpoZo2j7HOG7vIU34vwdZPjgV+GE6gCQLwLu552JqAUh+Q/nBMCoxu7Q4szSDFMy8d7B/dVEr9zZLCFTovJwmDD2K+im7Lwi+rJnq2J3imUFViKAy/Gypb3HIpdm+vvKuaduFI/0vmKSdPfU6tJam3M6sKOF6q0II/t9L96YjG219d+DMgX6auPUlBxsmFcIachHnO3EYacVZGhZrG/tze/NJG0ZQ4Drv3Bf7Y8tEt1pNFUOtRps45UgA2mEmXkrVKAElF/VfN5HzIZKi44JbCytlEn0jWX67vT+hHOGSYM/eUZzjKsVdN+GC+tI9IiGz+2MFFqALyLcNEcS8jnYDwgIDHAFoA/TXsJPQYKssbp11DUPa2mlsPicnqufFKdm22QDmMPx9400qmJblw54YaBwvxBUm7d81MEYaTpuvsLAJ2t2L6x+SBG/htzTZ/t7YSNaAwejQg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: hugetlb_hgm_eligible indicates that a VMA is eligible to have HGM explicitly enabled via MADV_SPLIT, and hugetlb_hgm_enabled indicates that HGM has been enabled. Signed-off-by: James Houghton Reviewed-by: Mina Almasry diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 7c977d234aba..efd2635a87f5 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -1211,6 +1211,20 @@ static inline void hugetlb_unregister_node(struct node *node) } #endif /* CONFIG_HUGETLB_PAGE */ +#ifdef CONFIG_HUGETLB_HIGH_GRANULARITY_MAPPING +bool hugetlb_hgm_enabled(struct vm_area_struct *vma); +bool hugetlb_hgm_eligible(struct vm_area_struct *vma); +#else +static inline bool hugetlb_hgm_enabled(struct vm_area_struct *vma) +{ + return false; +} +static inline bool hugetlb_hgm_eligible(struct vm_area_struct *vma) +{ + return false; +} +#endif + static inline spinlock_t *huge_pte_lock(struct hstate *h, struct mm_struct *mm, pte_t *pte) { diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 6c008c9de80e..0576dcc98044 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -7004,6 +7004,10 @@ static bool pmd_sharing_possible(struct vm_area_struct *vma) #ifdef CONFIG_USERFAULTFD if (uffd_disable_huge_pmd_share(vma)) return false; +#endif +#ifdef CONFIG_HUGETLB_HIGH_GRANULARITY_MAPPING + if (hugetlb_hgm_enabled(vma)) + return false; #endif /* * Only shared VMAs can share PMDs. @@ -7267,6 +7271,18 @@ __weak unsigned long hugetlb_mask_last_page(struct hstate *h) #endif /* CONFIG_ARCH_WANT_GENERAL_HUGETLB */ +#ifdef CONFIG_HUGETLB_HIGH_GRANULARITY_MAPPING +bool hugetlb_hgm_eligible(struct vm_area_struct *vma) +{ + /* All shared VMAs may have HGM. */ + return vma && (vma->vm_flags & VM_MAYSHARE); +} +bool hugetlb_hgm_enabled(struct vm_area_struct *vma) +{ + return vma && (vma->vm_flags & VM_HUGETLB_HGM); +} +#endif /* CONFIG_HUGETLB_HIGH_GRANULARITY_MAPPING */ + /* * These functions are overwritable if your architecture needs its own * behavior.