From patchwork Fri Oct 21 16:36:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Houghton X-Patchwork-Id: 13015073 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 421F6FA373F for ; Fri, 21 Oct 2022 16:37:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D67218E0007; Fri, 21 Oct 2022 12:37:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C2C488E0001; Fri, 21 Oct 2022 12:37:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9E2D28E0007; Fri, 21 Oct 2022 12:37:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 8C4EE8E0001 for ; Fri, 21 Oct 2022 12:37:21 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 5A105120504 for ; Fri, 21 Oct 2022 16:37:21 +0000 (UTC) X-FDA: 80045511882.01.1ED97D9 Received: from mail-vs1-f73.google.com (mail-vs1-f73.google.com [209.85.217.73]) by imf09.hostedemail.com (Postfix) with ESMTP id E27FC14003A for ; Fri, 21 Oct 2022 16:37:20 +0000 (UTC) Received: by mail-vs1-f73.google.com with SMTP id p10-20020a056102200a00b003a786b572f3so1067167vsr.20 for ; Fri, 21 Oct 2022 09:37:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=3+WerD0u1o5rdjYYXB/kVnQa0VLhkM9eeGW4etvGwB4=; b=tdKQZAo0gYs0G2kdztjg1TX/oj/RnJXrve82tS8xBK6mSdOdMhGNTvkXAs8ZVxrK7z c8Kzgsu4FgaGzd7K4NG6nHrY9Icl4OgkNHSsL2+fveGeW7tPGAvA7jOfo/RrP3+xhVsj hj3HZu2VVZPzkXEDKmJLfia5sRbAnjgM+kgr5WU35yWKtsbyjTr1njgKfeGFoYirHR50 cSTvm6iAX5EBkULkMrNaZyqGJUKF7r8UPQRlxf5X/7PYgCLrcqJENsCKtKyvvJj2Wm1K 9YyCpBMTJx1mwbB+Hpr2b1STLxNq4+6UD5uPyGqMcDZTJDNCR2PlND0UA0Ab0EPHpo7w JnYg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=3+WerD0u1o5rdjYYXB/kVnQa0VLhkM9eeGW4etvGwB4=; b=SrZI1aN/ZWtCyuUUSQkMFpp3rTJH7nHC6VqP4G+IjgKUMY3p4dbsrkVpPZ9UzdqN5M PGRkx+7JBXIsSCUB4y/s5QKrPggw1HxAxHUoFsPZElv4M6Vrz0Z99EA3VT8rKe9AVJJ+ VfzB8144fHr6jOUz95pzxkAMQXxWf9onPt4omtfaWAwpeSb5PzQlePa1HKXBT0NMc5bf RZ8ZW60+X1FaCi4G4b7Sd410Tt6o7IZRj5E+OpfY0QLUUnbCFW2HyWdhkmJYXLP1ntXh VZupbao9ExIj/cBCxQncDQUFXEJicvy3dXbJeeaVcxYqg67PwUUj437y91O0REjutVz5 KOGg== X-Gm-Message-State: ACrzQf3rhpCsZovldeOLVm52wH4ZRhgEeJUi2xJoHB0dyvOZQSDvTRQq PUwDxYtApCDb3Ojpw2FkoZawTGvvlY/lzK+d X-Google-Smtp-Source: AMsMyM6gtOTS1JtxBT054d/UW8WqhzjYFLQgqbNi7vzatLh9x6zB+jmUsXXfe5Kdu0BRGsbHY5icbADcYMj/NEsT X-Received: from jthoughton.c.googlers.com ([fda3:e722:ac3:cc00:14:4d90:c0a8:2a4f]) (user=jthoughton job=sendgmr) by 2002:a1f:aa42:0:b0:3ab:81ee:8fa9 with SMTP id t63-20020a1faa42000000b003ab81ee8fa9mr12468300vke.17.1666370240102; Fri, 21 Oct 2022 09:37:20 -0700 (PDT) Date: Fri, 21 Oct 2022 16:36:20 +0000 In-Reply-To: <20221021163703.3218176-1-jthoughton@google.com> Mime-Version: 1.0 References: <20221021163703.3218176-1-jthoughton@google.com> X-Mailer: git-send-email 2.38.0.135.g90850a2211-goog Message-ID: <20221021163703.3218176-5-jthoughton@google.com> Subject: [RFC PATCH v2 04/47] hugetlb: only adjust address ranges when VMAs want PMD sharing From: James Houghton To: Mike Kravetz , Muchun Song , Peter Xu Cc: David Hildenbrand , David Rientjes , Axel Rasmussen , Mina Almasry , "Zach O'Keefe" , Manish Mishra , Naoya Horiguchi , "Dr . David Alan Gilbert" , "Matthew Wilcox (Oracle)" , Vlastimil Babka , Baolin Wang , Miaohe Lin , Yang Shi , Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, James Houghton ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=tdKQZAo0; spf=pass (imf09.hostedemail.com: domain of 3wMpSYwoKCMEq0ov1no0vunvvnsl.jvtspu14-ttr2hjr.vyn@flex--jthoughton.bounces.google.com designates 209.85.217.73 as permitted sender) smtp.mailfrom=3wMpSYwoKCMEq0ov1no0vunvvnsl.jvtspu14-ttr2hjr.vyn@flex--jthoughton.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1666370240; a=rsa-sha256; cv=none; b=1LlCC/WfD3+O8xGzC27xakpt7AxoNPvZv6i9yT3jMiDRnfMqLEuvlSeV/dBdn2bZUObktn lrMNXVWxEymkGKAGXFNknPmfd0A37QgQtVrHUo4uTWCMw3HecCXiGM+cd0xhiE4DZGobdy uqye/HFRVFBFWOsP7sKVsaTj/NG1Iik= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1666370240; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=3+WerD0u1o5rdjYYXB/kVnQa0VLhkM9eeGW4etvGwB4=; b=ftKwbWyaCvfXrkGslCnnomB6S/Rf/7DqZ9Mw4osBkFxayNa0pu1ePCQrQI5/2QbAi4Tc46 mT0hr+KBy0cKozY4kMNExGsTB5eNVbC926Ka0YC/JxQszbU0qsxP8E2Xs3eRQYFoeKbli7 rJAruePNEZjoe3nLtJPXO0C8p+GzTzw= X-Rspamd-Queue-Id: E27FC14003A Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=tdKQZAo0; spf=pass (imf09.hostedemail.com: domain of 3wMpSYwoKCMEq0ov1no0vunvvnsl.jvtspu14-ttr2hjr.vyn@flex--jthoughton.bounces.google.com designates 209.85.217.73 as permitted sender) smtp.mailfrom=3wMpSYwoKCMEq0ov1no0vunvvnsl.jvtspu14-ttr2hjr.vyn@flex--jthoughton.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com X-Rspam-User: X-Rspamd-Server: rspam09 X-Stat-Signature: 7j973pq5x6j5i3nup9tein39qocsy7jm X-HE-Tag: 1666370240-677980 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently this check is overly aggressive. For some userfaultfd VMAs, VMA sharing is disabled, yet we still widen the address range, which is used for flushing TLBs and sending MMU notifiers. This is done now, as HGM VMAs also have sharing disabled, yet would still have flush ranges adjusted. Overaggressively flushing TLBs and triggering MMU notifiers is particularly harmful with lots of high-granularity operations. Signed-off-by: James Houghton Acked-by: Peter Xu Reviewed-by: Mike Kravetz --- mm/hugetlb.c | 21 +++++++++++++++------ 1 file changed, 15 insertions(+), 6 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 20a111b532aa..52cec5b0789e 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -6835,22 +6835,31 @@ static unsigned long page_table_shareable(struct vm_area_struct *svma, return saddr; } -bool want_pmd_share(struct vm_area_struct *vma, unsigned long addr) +static bool pmd_sharing_possible(struct vm_area_struct *vma) { - unsigned long start = addr & PUD_MASK; - unsigned long end = start + PUD_SIZE; - #ifdef CONFIG_USERFAULTFD if (uffd_disable_huge_pmd_share(vma)) return false; #endif /* - * check on proper vm_flags and page table alignment + * Only shared VMAs can share PMDs. */ if (!(vma->vm_flags & VM_MAYSHARE)) return false; if (!vma->vm_private_data) /* vma lock required for sharing */ return false; + return true; +} + +bool want_pmd_share(struct vm_area_struct *vma, unsigned long addr) +{ + unsigned long start = addr & PUD_MASK; + unsigned long end = start + PUD_SIZE; + /* + * check on proper vm_flags and page table alignment + */ + if (!pmd_sharing_possible(vma)) + return false; if (!range_in_vma(vma, start, end)) return false; return true; @@ -6871,7 +6880,7 @@ void adjust_range_if_pmd_sharing_possible(struct vm_area_struct *vma, * vma needs to span at least one aligned PUD size, and the range * must be at least partially within in. */ - if (!(vma->vm_flags & VM_MAYSHARE) || !(v_end > v_start) || + if (!pmd_sharing_possible(vma) || !(v_end > v_start) || (*end <= v_start) || (*start >= v_end)) return;