From patchwork Sat Feb 18 00:27:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Houghton X-Patchwork-Id: 13145370 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CB0C1C64ED6 for ; Sat, 18 Feb 2023 00:28:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3E6BC6B0078; Fri, 17 Feb 2023 19:28:50 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 34710280001; Fri, 17 Feb 2023 19:28:50 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 173256B007D; Fri, 17 Feb 2023 19:28:50 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 032A76B0078 for ; Fri, 17 Feb 2023 19:28:50 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id CCFCD1A073D for ; Sat, 18 Feb 2023 00:28:49 +0000 (UTC) X-FDA: 80478527178.29.188758E Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) by imf07.hostedemail.com (Postfix) with ESMTP id 17A7840004 for ; Sat, 18 Feb 2023 00:28:47 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b="n/AuNA+w"; spf=pass (imf07.hostedemail.com: domain of 3vxvwYwoKCNwHRFMSEFRMLEMMEJC.AMKJGLSV-KKIT8AI.MPE@flex--jthoughton.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=3vxvwYwoKCNwHRFMSEFRMLEMMEJC.AMKJGLSV-KKIT8AI.MPE@flex--jthoughton.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676680128; a=rsa-sha256; cv=none; b=FNVG0DH52cwFJAcJ39yF6FuerYuw3DPAVq1ou8ECyJBDAcPRqVLo7GuflbeHkmHapWhXbu ZvDfoC9tdc/RP4VZFSSvDxbxTdU779LFLSdWnWgmzN8M/QUwhweRq+rJa0SpQqeml0pJG9 PcSi64ZkIipz+7HvoIhwWtMHjMHY/U4= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b="n/AuNA+w"; spf=pass (imf07.hostedemail.com: domain of 3vxvwYwoKCNwHRFMSEFRMLEMMEJC.AMKJGLSV-KKIT8AI.MPE@flex--jthoughton.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=3vxvwYwoKCNwHRFMSEFRMLEMMEJC.AMKJGLSV-KKIT8AI.MPE@flex--jthoughton.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676680128; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ywqgi26Brd11uCoWP0sbMHtbYsofzUY9ZAimSzTW2NE=; b=rxfu1FKYDfvb5V0KneM1vGmkLC1UeML34HUUzpn7FWDaH3bUveHZwAdXX1nMjbCSNhH4CM zFEwW6iRJNH6RWnGHIl0fJDOQ7ZlMAx9GoaYrch9rqhzciM50b0SFdZZzDuPMC7J1otyrH lM+fFeKUbgetBztsF0ddwHBg7BIc9Ng= Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-536566339d6so26924587b3.11 for ; Fri, 17 Feb 2023 16:28:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=ywqgi26Brd11uCoWP0sbMHtbYsofzUY9ZAimSzTW2NE=; b=n/AuNA+wao69AJqH2jxvia/RDl00z2MfIhcsqjJucnXsT0cN+rnMxxAJPhBDLk5ZSi MCiuvlZOLO6F7xC0qObRnm2Fw7Kk4xKcB8RUyTN9WchfsVg6IiD35+jkjOxG7TArgt04 nthdLjbi+YqMB//69CKeg7ZPl8QIrBQY7Ll3FExxGFCSIT1L0xNOUp19y9PDuIe+SxhZ B4Zyh5fsXW/P04RT85l9L0piLa0E//fpZzktUrEB+JJqehCEtNNVOMSTn6SN8kGNhUKz UMxeGr/Oj6oj5yYpspoF7jsafZJJiQT+dGrZV68rl4q3U6mgeh9r3mvqGZ+fe/wLSD9M jzxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=ywqgi26Brd11uCoWP0sbMHtbYsofzUY9ZAimSzTW2NE=; b=VgcNFikIlhQG2aDAJotu6eOnuN+iLj1vOZarhEbL1JOUnZQ4NFfrSoRe0Q0b4NupA0 whAXYu6/j3el8bjWDaS1TVZdnZYhk840zqvHuU8BuwZXvbST5vHPRtMQRffoY4B8IpBq 8W1ieUH9cDz+eCRg0WesKb/zvSKrS68ZSyzbl1CQFtPfwdPdh15RX1h85n3dsQLS6raH Wkpu0bGvz0sp9PfopOOUvD7wknuBOIRSzxmkJdsCbWPCQY/EsE1RVEoBrAdHaawEuMEr W1gFpOoYVcb0UXip6e2mpRAWuUBeHs53qTR+2t9YOllLz0BqYcomjQ+4pBkRXqAysZAj Spqw== X-Gm-Message-State: AO0yUKW1jTsnhNNc8fgvFc6dYkhZq0EUzz0rTud/YyIhA0y1CgRDW6kl iDorT/VRTkBDvYf1fIn5HL7JpR1f1CxD7NzE X-Google-Smtp-Source: AK7set91EOlgY73vi4WtH0/wZAAYWGqc3EHj1NmJd/UToA1QLT2ebGJE6OT31llBzsdBdICY2Qgx1JjnImasJfkA X-Received: from jthoughton.c.googlers.com ([fda3:e722:ac3:cc00:14:4d90:c0a8:2a4f]) (user=jthoughton job=sendgmr) by 2002:a5b:84:0:b0:902:5b5c:73f7 with SMTP id b4-20020a5b0084000000b009025b5c73f7mr14406ybp.12.1676680127282; Fri, 17 Feb 2023 16:28:47 -0800 (PST) Date: Sat, 18 Feb 2023 00:27:37 +0000 In-Reply-To: <20230218002819.1486479-1-jthoughton@google.com> Mime-Version: 1.0 References: <20230218002819.1486479-1-jthoughton@google.com> X-Mailer: git-send-email 2.39.2.637.g21b0678d19-goog Message-ID: <20230218002819.1486479-5-jthoughton@google.com> Subject: [PATCH v2 04/46] hugetlb: only adjust address ranges when VMAs want PMD sharing From: James Houghton To: Mike Kravetz , Muchun Song , Peter Xu , Andrew Morton Cc: David Hildenbrand , David Rientjes , Axel Rasmussen , Mina Almasry , "Zach O'Keefe" , Manish Mishra , Naoya Horiguchi , "Dr . David Alan Gilbert" , "Matthew Wilcox (Oracle)" , Vlastimil Babka , Baolin Wang , Miaohe Lin , Yang Shi , Frank van der Linden , Jiaqi Yan , linux-mm@kvack.org, linux-kernel@vger.kernel.org, James Houghton X-Rspam-User: X-Rspamd-Queue-Id: 17A7840004 X-Rspamd-Server: rspam01 X-Stat-Signature: g73pgms4b1zbt8b6dc1egrqb3yh8bk1c X-HE-Tag: 1676680127-752798 X-HE-Meta: U2FsdGVkX1+TRGBPOPUC4RQ78C63zdqGdLpeb+QU/ijvbjCAuRB+RuVq2LvWzzxlCaiaj0HcScwocldxxz9b16rvwk/4oBrHCTp2g/IZfcdluEeN3gPvUqPJ4PFQQL6yPiahL7aVAXOfcO0r0at8tsH22chR9JWLInLdQlIa4SPVuGGx7/+ebfwx7PfY2qTPy+w1PUmJ0PMtOO0/dqsvvg9tzpypS8LZHbNVptL7GxmpzCcr3EG98cjGZmdhkNDbXYp445nEQgJkcFHuw3WIF/8eSJ0JAGEtZjzZZqvSmQF9X83Ur88SvZgEIhU2JWI1h7ssvZ4LC+d1p5dC1+JZQW7/3QDT0TP1A/T49l6O2+nimLd+4UuKhp62FM4defXpl2Pv5OJH9Z4FEt3OaaY9MjBD84qDz4++YfaE5vRgCBaCOes+YT25KHlFZO45FAr54+eGVVHP9W+Ec0EhKP/VcVrwNO9yOyjtfksrAGEUhVcJ53E7u70M3sq2c8mv3EspqcyiyZBAmMUWk4ls046cM1/bYGHkWFCRaQrylFYwXYrkYzlDOHrpDZyFGa68rV6D0Y/svRll9ndivjZrGub3NPtVdkH1Th8S9mchBSybUJ30Fo1G3Q7yHsclVPa217eijZ8vy5CDiRGaxS4mCb9juG8hU0hKe7r56v99zlBnS79Pf7Kc3rRr9a7+mXWBFaWhkfqtedL7Yuqod0uIgYFfI4F167VdTCPir4Zi8HIYsZ48tb0UUCYNkXIBZ8FzH9zGgaxeIsitjt/PLymq1gpciDuB6o7dSEwclXOKn/XyjijjMAcLx4JnrX7xk9tB9W4FZp9qEUJrQ4sRDhsrAzf2YxPsrHUOHyxJW1kw2E+5nz6S6AO2iSwjsMY26vbZdjuraCXqzNs+CVUR3M2lYAuAeHRbgseCyICNuhKdeVt6QXi8OuUi7Ua0AtCVNo9ofnKIcYz5/ooVi6z1EWckZ+1 +fmu6He+ iLD3ZhbvoRu1NLOMnNg0s0VGK/2HkJ3mFN3tTQVMQyv3flmb0gtPI40sYWPAmgtE5dzpRnKQBlURB7BY6rQsBcy5PF3QP+9IbngX71QLmPQKdYUSMoruujQWj728QcpBYRL+kCHyuXap2+nNrg+E5duw31mWc73V4h09Kg4cjsS9evSgUx95/oqQqOdy40m7aq7JjK8VqNmWSzrugo+jZ14zlbaBD4Y7AntfRGkqogSP1OGfDQhmTlt+gl1w+0LK2+T39AT4BBVQvJ6fKJTv7rUgdi8Jp+waLtgtguBtZFqEAfMIHXnyfiAiZ9xK7Ht3IjB9XnDXFO1sGhUbKMx+Ydqdg4f9QTylqUBn8IMwKe3bK8g6gQdO6afBLUqq6vVRsjyFXJYoS4rHT0lacjzdW+ItIqQcMTWzohLaUU3AKox+bk4up5yz16/arWxhLFcLD3pkgbCuw8YX2E0Vaq2LPr2SIBCxJulUqa0XHPLbjNvI4B8cQnSLfR9GsVUlCA+ZCh5NRW/dHA23GkWxnQ4LehaP4a4HKUKFdBksC5w99/CMpxKs3ezVJlt4vt2Yuuw8xZjrzyi4ceioPuGsu7Je9c/anheVX5KaePVZicbEUID6YQG0hqkpva8EN609gsJr9E0pfqc1/vcQG9Bd7sSacbD9TGGct6Wvi0pN6E2UGwJEGGSJuAYupMVwWm9+X+6c4yftmEjEK3gI2e5E= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently this check is overly aggressive. For some userfaultfd VMAs, VMA sharing is disabled, yet we still widen the address range, which is used for flushing TLBs and sending MMU notifiers. This is done now, as HGM VMAs also have sharing disabled, yet would still have flush ranges adjusted. Overaggressively flushing TLBs and triggering MMU notifiers is particularly harmful with lots of high-granularity operations. Acked-by: Peter Xu Reviewed-by: Mike Kravetz Signed-off-by: James Houghton Acked-by: Mina Almasry diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 540cdf9570d3..08004371cfed 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -6999,22 +6999,31 @@ static unsigned long page_table_shareable(struct vm_area_struct *svma, return saddr; } -bool want_pmd_share(struct vm_area_struct *vma, unsigned long addr) +static bool pmd_sharing_possible(struct vm_area_struct *vma) { - unsigned long start = addr & PUD_MASK; - unsigned long end = start + PUD_SIZE; - #ifdef CONFIG_USERFAULTFD if (uffd_disable_huge_pmd_share(vma)) return false; #endif /* - * check on proper vm_flags and page table alignment + * Only shared VMAs can share PMDs. */ if (!(vma->vm_flags & VM_MAYSHARE)) return false; if (!vma->vm_private_data) /* vma lock required for sharing */ return false; + return true; +} + +bool want_pmd_share(struct vm_area_struct *vma, unsigned long addr) +{ + unsigned long start = addr & PUD_MASK; + unsigned long end = start + PUD_SIZE; + /* + * check on proper vm_flags and page table alignment + */ + if (!pmd_sharing_possible(vma)) + return false; if (!range_in_vma(vma, start, end)) return false; return true; @@ -7035,7 +7044,7 @@ void adjust_range_if_pmd_sharing_possible(struct vm_area_struct *vma, * vma needs to span at least one aligned PUD size, and the range * must be at least partially within in. */ - if (!(vma->vm_flags & VM_MAYSHARE) || !(v_end > v_start) || + if (!pmd_sharing_possible(vma) || !(v_end > v_start) || (*end <= v_start) || (*start >= v_end)) return;