From patchwork Sun Feb 6 21:32:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hugh Dickins X-Patchwork-Id: 12736735 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7011C433EF for ; Sun, 6 Feb 2022 21:32:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5F9CC6B0072; Sun, 6 Feb 2022 16:32:20 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5A84B6B0073; Sun, 6 Feb 2022 16:32:20 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 423226B0074; Sun, 6 Feb 2022 16:32:20 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.a.hostedemail.com [64.99.140.24]) by kanga.kvack.org (Postfix) with ESMTP id 342A86B0072 for ; Sun, 6 Feb 2022 16:32:20 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id EA594208C8 for ; Sun, 6 Feb 2022 21:32:19 +0000 (UTC) X-FDA: 79113653598.02.B5E7427 Received: from mail-qk1-f180.google.com (mail-qk1-f180.google.com [209.85.222.180]) by imf10.hostedemail.com (Postfix) with ESMTP id 7FA57C0002 for ; Sun, 6 Feb 2022 21:32:19 +0000 (UTC) Received: by mail-qk1-f180.google.com with SMTP id c189so9492211qkg.11 for ; Sun, 06 Feb 2022 13:32:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:from:to:cc:subject:in-reply-to:message-id:references :mime-version; bh=zQkpuLA9DZJBUcE3jcYKmscDjcpqSqOLAzrvayz8ZnU=; b=GR8TM5vXeQKiwsAkIe1lxgAeTCAhUmGm16fCx+tfDblMF1EMkt4QZ4Hy/vppBd/MN0 3UzX65hlcREiX7l/hWchOeXkrk7bmgMPSJT+V6pYw0zBYf3UjwiwzTE94NidxzhmA/z4 JXkN9JTi/BRohA6w7XEHQMiAYzehy5Q4fMCx2vIQNzs1SKF+gfc+1DtQm76W0l8ErE9+ wQvZ5eDJafIatsgNYCocYBFZd1leAMWZIt0e2816Pq3GAn8SpZr9GRn3lsGcMFsOIzyr FGKMVW302u+8DJp5iuQRFQfQD9sAV2fGeKibi+QKzHSugVsQmj4soDsg0sIXK86OuhDI B7ZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:in-reply-to:message-id :references:mime-version; bh=zQkpuLA9DZJBUcE3jcYKmscDjcpqSqOLAzrvayz8ZnU=; b=vojQRDMuuYfAtnOE27ivTA7jKDC150mx3Qq13DXR2Dj6G1m60ryQQrcCojDUgJGxz/ B6xFRMu14+Jvih8B0woNAKIcoGxeKl6W4+x22y7x7L8ijeyoq6Zl0TsINyoixcEKQLt9 XzgeoHcBHalTj/G3TxELuhnrvsepRsK+zgtEaaJqMSW6NHpdvsALay8tdpP3tF5cLqSw dX/LrBtKt7BckWFlNhEL/RoU3SLa/0dJA2S63Bmh4BLsA+uztsmKuQwi44P1OSUAxTnw 8NfWMUvt7AH6ufkpU2j4SaMss02YiW6BamqwMG3AeDE5ZiNWiuFmEagBYbhUbyewq0bF iZmw== X-Gm-Message-State: AOAM533a1BVKjCE6G7xdK0KOmJ+StAYA4WBVHukh+3JX4oe89zhegr54 s6wr7ZBuetGjYYk+sm6qlCJZhQ== X-Google-Smtp-Source: ABdhPJwjY0WNGaOWMQXMYnAUM4AdKudIMQlTqPQspgi9cVygJ/L+ge74DGirQVBQMsWFL+2MhlIGzw== X-Received: by 2002:a05:620a:1914:: with SMTP id bj20mr4920106qkb.56.1644183138671; Sun, 06 Feb 2022 13:32:18 -0800 (PST) Received: from ripple.attlocal.net (172-10-233-147.lightspeed.sntcca.sbcglobal.net. [172.10.233.147]) by smtp.gmail.com with ESMTPSA id w10sm5068516qtj.73.2022.02.06.13.32.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 06 Feb 2022 13:32:18 -0800 (PST) Date: Sun, 6 Feb 2022 13:32:15 -0800 (PST) From: Hugh Dickins X-X-Sender: hugh@ripple.anvils To: Andrew Morton cc: Michal Hocko , Vlastimil Babka , "Kirill A. Shutemov" , Matthew Wilcox , David Hildenbrand , Alistair Popple , Johannes Weiner , Rik van Riel , Suren Baghdasaryan , Yu Zhao , Greg Thelen , Shakeel Butt , linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH 02/13] mm/munlock: delete FOLL_MLOCK and FOLL_POPULATE In-Reply-To: <8e4356d-9622-a7f0-b2c-f116b5f2efea@google.com> Message-ID: <2b5eee76-183f-bd97-2e9d-f5ff8df63db@google.com> References: <8e4356d-9622-a7f0-b2c-f116b5f2efea@google.com> MIME-Version: 1.0 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 7FA57C0002 X-Stat-Signature: iazprji3bouxr3nh5qunuxkaimwp89q4 X-Rspam-User: nil Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=GR8TM5vX; spf=pass (imf10.hostedemail.com: domain of hughd@google.com designates 209.85.222.180 as permitted sender) smtp.mailfrom=hughd@google.com; dmarc=pass (policy=reject) header.from=google.com X-HE-Tag: 1644183139-318783 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: If counting page mlocks, we must not double-count: follow_page_pte() can tell if a page has already been Mlocked or not, but cannot tell if a pte has already been counted or not: that will have to be done when the pte is mapped in (which lru_cache_add_inactive_or_unevictable() already tracks for new anon pages, but there's no such tracking yet for others). Delete all the FOLL_MLOCK code - faulting in the missing pages will do all that is necessary, without special mlock_vma_page() calls from here. But then FOLL_POPULATE turns out to serve no purpose - it was there so that its absence would tell faultin_page() not to faultin page when setting up VM_LOCKONFAULT areas; but if there's no special work needed here for mlock, then there's no work at all here for VM_LOCKONFAULT. Have I got that right? I've not looked into the history, but see that FOLL_POPULATE goes back before VM_LOCKONFAULT: did it serve a different purpose before? Ah, yes, it was used to skip the old stack guard page. And is it intentional that COW is not broken on existing pages when setting up a VM_LOCKONFAULT area? I can see that being argued either way, and have no reason to disagree with current behaviour. Signed-off-by: Hugh Dickins Acked-by: Vlastimil Babka --- include/linux/mm.h | 2 -- mm/gup.c | 43 ++++++++----------------------------------- mm/huge_memory.c | 33 --------------------------------- 3 files changed, 8 insertions(+), 70 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 213cc569b192..74ee50c2033b 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2925,13 +2925,11 @@ struct page *follow_page(struct vm_area_struct *vma, unsigned long address, #define FOLL_FORCE 0x10 /* get_user_pages read/write w/o permission */ #define FOLL_NOWAIT 0x20 /* if a disk transfer is needed, start the IO * and return without waiting upon it */ -#define FOLL_POPULATE 0x40 /* fault in pages (with FOLL_MLOCK) */ #define FOLL_NOFAULT 0x80 /* do not fault in pages */ #define FOLL_HWPOISON 0x100 /* check page is hwpoisoned */ #define FOLL_NUMA 0x200 /* force NUMA hinting page fault */ #define FOLL_MIGRATION 0x400 /* wait for page to replace migration entry */ #define FOLL_TRIED 0x800 /* a retry, previous pass started an IO */ -#define FOLL_MLOCK 0x1000 /* lock present pages */ #define FOLL_REMOTE 0x2000 /* we are working on non-current tsk/mm */ #define FOLL_COW 0x4000 /* internal GUP flag */ #define FOLL_ANON 0x8000 /* don't do file mappings */ diff --git a/mm/gup.c b/mm/gup.c index f0af462ac1e2..2076902344d8 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -572,32 +572,6 @@ static struct page *follow_page_pte(struct vm_area_struct *vma, */ mark_page_accessed(page); } - if ((flags & FOLL_MLOCK) && (vma->vm_flags & VM_LOCKED)) { - /* Do not mlock pte-mapped THP */ - if (PageTransCompound(page)) - goto out; - - /* - * The preliminary mapping check is mainly to avoid the - * pointless overhead of lock_page on the ZERO_PAGE - * which might bounce very badly if there is contention. - * - * If the page is already locked, we don't need to - * handle it now - vmscan will handle it later if and - * when it attempts to reclaim the page. - */ - if (page->mapping && trylock_page(page)) { - lru_add_drain(); /* push cached pages to LRU */ - /* - * Because we lock page here, and migration is - * blocked by the pte's page reference, and we - * know the page is still mapped, we don't even - * need to check for file-cache page truncation. - */ - mlock_vma_page(page); - unlock_page(page); - } - } out: pte_unmap_unlock(ptep, ptl); return page; @@ -920,9 +894,6 @@ static int faultin_page(struct vm_area_struct *vma, unsigned int fault_flags = 0; vm_fault_t ret; - /* mlock all present pages, but do not fault in new pages */ - if ((*flags & (FOLL_POPULATE | FOLL_MLOCK)) == FOLL_MLOCK) - return -ENOENT; if (*flags & FOLL_NOFAULT) return -EFAULT; if (*flags & FOLL_WRITE) @@ -1173,8 +1144,6 @@ static long __get_user_pages(struct mm_struct *mm, case -ENOMEM: case -EHWPOISON: goto out; - case -ENOENT: - goto next_page; } BUG(); } else if (PTR_ERR(page) == -EEXIST) { @@ -1472,9 +1441,14 @@ long populate_vma_page_range(struct vm_area_struct *vma, VM_BUG_ON_VMA(end > vma->vm_end, vma); mmap_assert_locked(mm); - gup_flags = FOLL_TOUCH | FOLL_POPULATE | FOLL_MLOCK; + /* + * Rightly or wrongly, the VM_LOCKONFAULT case has never used + * faultin_page() to break COW, so it has no work to do here. + */ if (vma->vm_flags & VM_LOCKONFAULT) - gup_flags &= ~FOLL_POPULATE; + return nr_pages; + + gup_flags = FOLL_TOUCH; /* * We want to touch writable mappings with a write fault in order * to break COW, except for shared mappings because these don't COW @@ -1541,10 +1515,9 @@ long faultin_vma_page_range(struct vm_area_struct *vma, unsigned long start, * in the page table. * FOLL_HWPOISON: Return -EHWPOISON instead of -EFAULT when we hit * a poisoned page. - * FOLL_POPULATE: Always populate memory with VM_LOCKONFAULT. * !FOLL_FORCE: Require proper access permissions. */ - gup_flags = FOLL_TOUCH | FOLL_POPULATE | FOLL_MLOCK | FOLL_HWPOISON; + gup_flags = FOLL_TOUCH | FOLL_HWPOISON; if (write) gup_flags |= FOLL_WRITE; diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 406a3c28c026..9a34b85ebcf8 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1380,39 +1380,6 @@ struct page *follow_trans_huge_pmd(struct vm_area_struct *vma, if (flags & FOLL_TOUCH) touch_pmd(vma, addr, pmd, flags); - if ((flags & FOLL_MLOCK) && (vma->vm_flags & VM_LOCKED)) { - /* - * We don't mlock() pte-mapped THPs. This way we can avoid - * leaking mlocked pages into non-VM_LOCKED VMAs. - * - * For anon THP: - * - * In most cases the pmd is the only mapping of the page as we - * break COW for the mlock() -- see gup_flags |= FOLL_WRITE for - * writable private mappings in populate_vma_page_range(). - * - * The only scenario when we have the page shared here is if we - * mlocking read-only mapping shared over fork(). We skip - * mlocking such pages. - * - * For file THP: - * - * We can expect PageDoubleMap() to be stable under page lock: - * for file pages we set it in page_add_file_rmap(), which - * requires page to be locked. - */ - - if (PageAnon(page) && compound_mapcount(page) != 1) - goto skip_mlock; - if (PageDoubleMap(page) || !page->mapping) - goto skip_mlock; - if (!trylock_page(page)) - goto skip_mlock; - if (page->mapping && !PageDoubleMap(page)) - mlock_vma_page(page); - unlock_page(page); - } -skip_mlock: page += (addr & ~HPAGE_PMD_MASK) >> PAGE_SHIFT; VM_BUG_ON_PAGE(!PageCompound(page) && !is_zone_device_page(page), page);