From patchwork Mon Sep 18 07:33:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yin Fengwei X-Patchwork-Id: 13388983 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9BFCACD13DF for ; Mon, 18 Sep 2023 07:33:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3E4E66B0297; Mon, 18 Sep 2023 03:33:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 394726B0299; Mon, 18 Sep 2023 03:33:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 25BEA6B029A; Mon, 18 Sep 2023 03:33:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 12A7C6B0297 for ; Mon, 18 Sep 2023 03:33:28 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id EBE6FA0C41 for ; Mon, 18 Sep 2023 07:33:27 +0000 (UTC) X-FDA: 81248902854.23.E450212 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.31]) by imf23.hostedemail.com (Postfix) with ESMTP id A0520140002 for ; Mon, 18 Sep 2023 07:33:25 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=Yu7H9J9O; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf23.hostedemail.com: domain of fengwei.yin@intel.com designates 134.134.136.31 as permitted sender) smtp.mailfrom=fengwei.yin@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1695022406; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=lT6UF5ILleu9h6Vq9PriQChkKcgzw5T/lmPdaSatTgA=; b=YKBoso+AyH/MD4tVH3dxiy4wyPJBdzToawLjqQW/ouex6Y7VJAqwkvzGerXAEuwkRmS6B+ UwvnSFldUI7N099Z/HdW65utSgf9LhBRPHXJlC7jgBOKa+c9MJ88k0FvR0vD9895y/Gmcd IgU13Id8Jd7+y4Y8/l1KlIUoYQ0voSw= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=Yu7H9J9O; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf23.hostedemail.com: domain of fengwei.yin@intel.com designates 134.134.136.31 as permitted sender) smtp.mailfrom=fengwei.yin@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1695022406; a=rsa-sha256; cv=none; b=esVBXPWtE8N8tADJ70tS+kkWN+w8DjEWI4x8sxoXIgwVaT9fGu+s4tt4D6iTr7TZ+ABfZW V7rQ+rAu9CbuCB0xRto9WG8mopt+KyPXXuvqyaC0xjwJ/KpTj/HyU96f5fNttn6iDHseEk fQlAhpafjjUeA5Ed1drA9UtwSogsfmg= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1695022404; x=1726558404; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=LismrfeNewiy95Gm2WxPWMqPvAHx57HKqPCYg8Zeheg=; b=Yu7H9J9OXNmOVWKfAZxPso1bFM6racH9/vJW5USG1m2kWF6Kd71/dci/ 6Hpg75++5mIkL/UQv5DQto5l8snHafJqZ/3YJU4FM5k7/9ldpoAnpOxCf WJx0x5Shw55RyatZL297EmgDo5TVhMVBIcNrRqBMzHk9EJ911hFynjH4X bYT33e9cJ/FI/zDDQcFO6AEVi/aPK9MyR4tyqXEbs+x8kaZajjXgw36+b Hk80Ta+gDTAUSVfv9TggdCnPja+gkqKXGyY4quQ8JfNhhWZhV3Mgqpr8+ kQL22nsxBL3JQCT5FSo2Fbp+JQ3vrg7FyGBuTK3mMOTaCUnBdpBDrWT6K g==; X-IronPort-AV: E=McAfee;i="6600,9927,10836"; a="443664570" X-IronPort-AV: E=Sophos;i="6.02,156,1688454000"; d="scan'208";a="443664570" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Sep 2023 00:33:22 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10836"; a="775030619" X-IronPort-AV: E=Sophos;i="6.02,156,1688454000"; d="scan'208";a="775030619" Received: from fyin-dev.sh.intel.com ([10.239.159.24]) by orsmga008.jf.intel.com with ESMTP; 18 Sep 2023 00:33:19 -0700 From: Yin Fengwei To: linux-mm@kvack.org, linux-kernel@vger.kernel.org, akpm@linux-foundation.org, yuzhao@google.com, willy@infradead.org, hughd@google.com, yosryahmed@google.com, ryan.roberts@arm.com, david@redhat.com, shy828301@gmail.com Cc: fengwei.yin@intel.com Subject: [PATCH v3 0/3] support large folio for mlock Date: Mon, 18 Sep 2023 15:33:15 +0800 Message-Id: <20230918073318.1181104-1-fengwei.yin@intel.com> X-Mailer: git-send-email 2.39.2 MIME-Version: 1.0 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: A0520140002 X-Stat-Signature: b3t75h1yzof4igcg1i4yiobg9bp91t7d X-Rspam-User: X-HE-Tag: 1695022405-45868 X-HE-Meta: U2FsdGVkX1+aMTSdzWwty0ZK8KIOdhSkP0GJpA3T+ERXiAjb6MUpi4IteZ9Szbc3drwU2mgacMDhU+L86s0nTTiDgZDUa0UspZp1/VJgil6QRJE9WtHsb+IulhaF7QQXvofHamqfaqKpE3kOThtilvJoX/3w35VBuR5ZSCogZoFR9hXIni75lxQsNeeVVALIdvHy9tZRfb9yVHPNP5+bV8yXhhx/qN4g9vsBr1GNhE1yxHaaFdRBp/D2gOtTMaz69aWMSvOHyOH4dXYFX8pAbgVK9i5opR1PgNsTFyDjFt7a7rJexGHhA142BePO7BlYuUGOYlgwACun0RFrwhfGuNQ5k/qvOzq1bpJSRW9SGtcEsJmOUr7y94lfP6/wVCZZbPJg3r74INaBqvLtQliHog5vXg9dcIKOL5AlleN6B/HNjwv8wAjoitKQoC2AbclxhMds/c4H0f2QpwqoWtzOquLcQrtNOABslrORh+Dr2xsvFDHizzyhofJPUsrLJDiuk5LtX2LeHCMZ64UhCMAoDLRVP2jPmIUVPkUWkhUeX2R/Nm8WV+A69rosXNEvS00NtdrzLxD/kFh+XzY3efB4VQ0Pz1I4E3erBGLTqmXymsVIQrQd5CPhd/Jbi4lyrS8n5QnDoAimoqHWB/EKJrwWb3aOohnLNJ1gFT7rf4isd2fjtYFRjLmEIwbfMOO+2CIIRA6JtQNz1cDK0qLST+QOQf34Qqz3roJ8whz1A2BHWhmFeM0HqpwJ+Sjci/KZL369OL3O2yviwOohP2Z413STcke2Bi09O9Pd9NTxtEawLcPRs4AZX7v+5rk4ylWTN8fKlhGXxDjo4LwClxQabR9FB+OX3TWvxfm1rQceYG6ceN+9+8nDh561D2RT1zAoJoN/uqUr02/oMl2/zKIUnagQ0HVmK613il9ZcK0FlB20OPTpn2SEVkaqOZMUJpUQg7NfBMB5s7M4ARYeh8dBH/u BAHtdm1h SMHYGSE4pltAhUaf++5EfLbIuSbQikGQfH+0C0V6zUKQlECpXrimSEpXRPgP24TSFkI6vJO8cN+pBUokvQ48YxfVY4/SetkMaxppuDzHZSNqsHiU+dEnlyqdyaZ8aTBnL+OxLRIMn0/MMykGm2m5toN6MpFuIVIvbE+RpcfbowwUdS6XROvG4CTDorCbY9hXvVSR6AFPhKlxDhJPUqZ0Sgh5KRBtBUYd00Aeqd1W2MWcYATI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Yu mentioned at [1] about the mlock() can't be applied to large folio. I leant the related code and here is my understanding: - For RLIMIT_MEMLOCK related, there is no problem. Because the RLIMIT_MEMLOCK statistics is not related underneath page. That means underneath page mlock or munlock doesn't impact the RLIMIT_MEMLOCK statistics collection which is always correct. - For keeping the page in RAM, there is no problem either. At least, during try_to_unmap_one(), once detect the VMA has VM_LOCKED bit set in vm_flags, the folio will be kept whatever the folio is mlocked or not. So the function of mlock for large folio works. But it's not optimized because the page reclaim needs scan these large folio and may split them. This series identified the large folio for mlock to four types: - The large folio is in VM_LOCKED range and fully mapped to the range - The large folio is in the VM_LOCKED range but not fully mapped to the range - The large folio cross VM_LOCKED VMA boundary - The large folio cross last level page table boundary For the first type, we mlock large folio so page reclaim will skip it. For the second/third type, we don't mlock large folio. As the pages not mapped to VM_LOACKED range are mapped to none VM_LOCKED range, if system is in memory pressure situation, the large folio can be picked by page reclaim and split. Then the pages not mapped to VM_LOCKED range can be reclaimed. For the fourth type, we don't mlock large folio because locking one page table lock can't prevent the part in another last level page table being unmapped. Thanks to Ryan for pointing this out. To check whether the folio is fully mapped to the range, PTEs needs be checked to see whether the page of folio is associated. Which needs take page table lock and is heavy operation. So far, the only place needs this check is madvise and page reclaim. These functions already have their own PTE iterator. patch1 introduce API to check whether large folio is in VMA range. patch2 make page reclaim/mlock_vma_folio/munlock_vma_folio support large folio mlock/munlock. patch3 make mlock/munlock syscall support large folio. testing done: - kernel selftest. No extra failure introduced v2 was post here [2]. Yu also mentioned a race which can make folio unevictable after munlock during RFC v2 discussion [3]: We decided that race issue didn't block this series based on: - That race issue was not introduced by this series - We had a looks-ok fix for that race issue. Need to wait for mlock_count fixing patch as Yosry Ahmed suggested [4] ChangeLog from V2: - Rebase to latest mm-unstable branch - Add comment to function folio_within_range() per Ryan's suggestions - Change function name folio_in_range() to folio_within_range() per Ryan's suggestions - No real code change ChangeLog from V1: - Remove the PTE check from folio_in_range() and reuse the page table iterator (in madvise and folio_referenced_one) to check whether fully mapped or not in callers - Avoid mlock the folio which cross last level page table. Thanks to Ryan for pointing this out. - Drop pte_none() check when iterate page table because we only care pte_present() case. - move folio_test_large() out of m(un)lock_vma_folio() ChangeLog from RFC v2: - Removed RFC - dropped folio_is_large() check as suggested by both Yu and Huge - Besides the address/pgoff check, also check the page table entry when check whether the folio is in the range. This is to handle mremap case that address/pgoff is in range, but folio can't be identified as in range. - Fixed one issue in page_add_anon_rmap() and page_add_anon_rmap() introdued by RFC v2. As these two functions can be called multiple times against one folio. And remove_rmap() may not be called same times. Which can bring imbalanced mlock_count. Fix it by skip mlock large folio in these two functions. [1] https://lore.kernel.org/linux-mm/CAOUHufbtNPkdktjt_5qM45GegVO-rCFOMkSh0HQminQ12zsV8Q@mail.gmail.com/ [2] https://lore.kernel.org/linux-mm/20230809061105.3369958-1-fengwei.yin@intel.com/ [3] https://lore.kernel.org/linux-mm/CAOUHufZ6=9P_=CAOQyw0xw-3q707q-1FVV09dBNDC-hpcpj2Pg@mail.gmail.com/ [4] https://lore.kernel.org/linux-mm/CAJD7tkZJFG=7xs=9otc5CKs6odWu48daUuZP9Wd9Z-sZF07hXg@mail.gmail.com/ Yin Fengwei (3): mm: add functions folio_in_range() and folio_within_vma() mm: handle large folio when large folio in VM_LOCKED VMA range mm: mlock: update mlock_pte_range to handle large folio mm/internal.h | 73 ++++++++++++++++++++++++++++++++++++++++++++------- mm/mlock.c | 66 ++++++++++++++++++++++++++++++++++++++++++++-- mm/rmap.c | 66 ++++++++++++++++++++++++++++++++++++++-------- 3 files changed, 182 insertions(+), 23 deletions(-)