From patchwork Tue Feb 15 02:18:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hugh Dickins X-Patchwork-Id: 12746438 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A70AFC433F5 for ; Tue, 15 Feb 2022 02:18:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0A1C16B0078; Mon, 14 Feb 2022 21:18:48 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0520C6B007B; Mon, 14 Feb 2022 21:18:48 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E34416B007D; Mon, 14 Feb 2022 21:18:47 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0195.hostedemail.com [216.40.44.195]) by kanga.kvack.org (Postfix) with ESMTP id D44646B0078 for ; Mon, 14 Feb 2022 21:18:47 -0500 (EST) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 9BFA6901BD for ; Tue, 15 Feb 2022 02:18:47 +0000 (UTC) X-FDA: 79143405894.17.C788174 Received: from mail-qt1-f177.google.com (mail-qt1-f177.google.com [209.85.160.177]) by imf15.hostedemail.com (Postfix) with ESMTP id 1AA9AA0004 for ; Tue, 15 Feb 2022 02:18:46 +0000 (UTC) Received: by mail-qt1-f177.google.com with SMTP id t1so17231324qtq.13 for ; Mon, 14 Feb 2022 18:18:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:from:to:cc:subject:message-id:mime-version; bh=Lxapmp7hvxMfhcEgIR9IOKw4+JdJYHsOl4yuqp6IrFE=; b=KammbQQB7R7dcpVFa/OeFRwAi3DqIw3osyJ9PcJUn3GRd6kE34NSu+P4nFNuG4pfDY iGaFmYZFOJwOJcOvYp+Vp8DYMJ5hz/ipx5Up784xHTLb7jYSo2zMOZVB2KIkdZThHZhx m2Vnvd3Mwvm5P5Gnu8IT46eBeHfiMRey+r0gjyxDehWhxjvcGZqx60ET5YpD3VYT0ehS RzTtTEhSAUOh/gVTpmXBGiIsa9ylrILctpkm0Ymmrzuzlp0ImepqEecpEMny6ZvmoQQ2 KapyAO9cotXHvrwmDb01DlhSQ9ya43UfZ+Vk68sKX7bC/4HpIeXTn+emDiKneuo2LLkJ 2oYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:mime-version; bh=Lxapmp7hvxMfhcEgIR9IOKw4+JdJYHsOl4yuqp6IrFE=; b=FInDRRF/J+9zy/3LNfk0vK1Il1/o3BSxtN6KX9fKEBXg095cg5hwu4MOXg+xVW/Psq Vsx3DnJWM6PXfAMKUcqFIMg5Zwf9GlW6UYXKx7gmaL2Iw+NByRhl/OOsv6c/o4QpMHIT nXi12sI2wCyjOapoPfQBU6G/l1uXMC+rTGaxmcE7AQTbt+HncUcx5CwZJZo2ml1up0D2 KYFi64D0GFvaMDs9VAlpvTThBcaQTflrqHRl6XOTgwPbRcViWemf48mjfpKLLFhq6kW4 xDcIOJWjdzu/OqdQFVgP2TWttKFCs8iYSpt1acfqp/8dNqUePvKAmPE4bA9Fso4Z0bDA IXXA== X-Gm-Message-State: AOAM531Na2PjUH/l+nfL1CSysajpnZyBP8HyXmDYb5jcTN0247KUvX/y rnEKkYcH/cTDcTTRvWvUK2ohoQ== X-Google-Smtp-Source: ABdhPJzmILMNMpour4YK5+aHp3FRrJ3sl/MFwG0q57Lm5bb6veWgY8T0mK5xdiGd6mw7JDnG+bW1ow== X-Received: by 2002:a05:622a:487:: with SMTP id p7mr1323742qtx.426.1644891526299; Mon, 14 Feb 2022 18:18:46 -0800 (PST) Received: from ripple.attlocal.net (172-10-233-147.lightspeed.sntcca.sbcglobal.net. [172.10.233.147]) by smtp.gmail.com with ESMTPSA id c20sm19128387qtb.58.2022.02.14.18.18.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Feb 2022 18:18:45 -0800 (PST) Date: Mon, 14 Feb 2022 18:18:34 -0800 (PST) From: Hugh Dickins X-X-Sender: hugh@ripple.anvils To: Andrew Morton cc: Hugh Dickins , Michal Hocko , Vlastimil Babka , "Kirill A. Shutemov" , Matthew Wilcox , David Hildenbrand , Alistair Popple , Johannes Weiner , Rik van Riel , Suren Baghdasaryan , Yu Zhao , Greg Thelen , Shakeel Butt , linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH v2 00/13] mm/munlock: rework of mlock+munlock page handling Message-ID: <55a49083-37f9-3766-1de9-9feea7428ac@google.com> MIME-Version: 1.0 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 1AA9AA0004 X-Stat-Signature: 6e8d5dmdhukmr8ybhqq3ro5qgugnjrde X-Rspam-User: Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=KammbQQB; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf15.hostedemail.com: domain of hughd@google.com designates 209.85.160.177 as permitted sender) smtp.mailfrom=hughd@google.com X-HE-Tag: 1644891526-110296 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: I wondered whether to post this munlocking rework in https://lore.kernel.org/linux-mm/35c340a6-96f-28a0-2b7b-2f9fbddc01f@google.com/ There the discussion was OOM reaping, but the main reason for the rework has been catastrophic contention on i_mmap_rwsem when exiting from multiply mlocked files; and frustration with how contorted munlocking is. tl;dr mm/mlock.c | 637 +++++++++++++++----------------------- 23 files changed, 514 insertions(+), 780 deletions(-) v1 of the series was posted on 6 Feb 2022: https://lore.kernel.org/linux-mm/8e4356d-9622-a7f0-b2c-f116b5f2efea@google.com/ Preview of v2 changed patches 01 04 07 10 11 were posted on 13 Feb 2022. Here is the full v2 series, in case this is easier to manage: based again on 5.17-rc2, applies also to -rc3 and -rc4. Andrew, many thanks for including v1 and fixes in mmotm: please now replace mm-munlock-delete-page_mlock-and-all-its-works.patch mm-munlock-delete-foll_mlock-and-foll_populate.patch mm-munlock-delete-munlock_vma_pages_all-allow-oomreap.patch mm-munlock-rmap-call-mlock_vma_page-munlock_vma_page.patch mm-munlock-replace-clear_page_mlock-by-final-clearance.patch mm-munlock-maintain-page-mlock_count-while-unevictable.patch mm-munlock-mlock_pte_range-when-mlocking-or-munlocking.patch mm-migrate-__unmap_and_move-push-good-newpage-to-lru.patch mm-munlock-delete-smp_mb-from-__pagevec_lru_add_fn.patch mm-munlock-mlock_page-munlock_page-batch-by-pagevec.patch mm-munlock-mlock_page-munlock_page-batch-by-pagevec-fix.patch mm-munlock-mlock_page-munlock_page-batch-by-pagevec-fix-2.patch mm-munlock-page-migration-needs-mlock-pagevec-drained.patch mm-thp-collapse_file-do-try_to_unmapttu_batch_flush.patch mm-thp-shrink_page_list-avoid-splitting-vm_locked-thp.patch by the following thirteen of v2. As before, some easy fixups will be needed to apply in mm/huge_memory.c, but otherwise expected to be clean. Many thanks to Vlastimil Babka for his review of 01 through 11, and to Matthew Wilcox for graciously volunteering to rebase his over these. At present there's no update to Documentation/vm/unevictable-lru.rst: that always needs a different mindset, can follow later, please refer to commit messages for now. There are two half-related mm/thp patches at the end: enhancements we've had for a long time, but needed more after the mlock changes. 01/13 mm/munlock: delete page_mlock() and all its works 02/13 mm/munlock: delete FOLL_MLOCK and FOLL_POPULATE 03/13 mm/munlock: delete munlock_vma_pages_all(), allow oomreap 04/13 mm/munlock: rmap call mlock_vma_page() munlock_vma_page() 05/13 mm/munlock: replace clear_page_mlock() by final clearance 06/13 mm/munlock: maintain page->mlock_count while unevictable 07/13 mm/munlock: mlock_pte_range() when mlocking or munlocking 08/13 mm/migrate: __unmap_and_move() push good newpage to LRU 09/13 mm/munlock: delete smp_mb() from __pagevec_lru_add_fn() 10/13 mm/munlock: mlock_page() munlock_page() batch by pagevec 11/13 mm/munlock: page migration needs mlock pagevec drained 12/13 mm/thp: collapse_file() do try_to_unmap(TTU_BATCH_FLUSH) 13/13 mm/thp: shrink_page_list() avoid splitting VM_LOCKED THP include/linux/mm.h | 2 include/linux/mm_inline.h | 11 include/linux/mm_types.h | 19 + include/linux/rmap.h | 23 - kernel/events/uprobes.c | 7 mm/gup.c | 43 -- mm/huge_memory.c | 55 --- mm/hugetlb.c | 4 mm/internal.h | 66 ++- mm/khugepaged.c | 14 mm/ksm.c | 12 mm/madvise.c | 5 mm/memcontrol.c | 3 mm/memory.c | 45 -- mm/migrate.c | 42 +- mm/mlock.c | 637 +++++++++++++++----------------------- mm/mmap.c | 32 - mm/mmzone.c | 7 mm/oom_kill.c | 2 mm/rmap.c | 156 ++------- mm/swap.c | 89 ++--- mm/userfaultfd.c | 14 mm/vmscan.c | 6 23 files changed, 514 insertions(+), 780 deletions(-) Hugh