From patchwork Mon Dec 16 16:51:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dev Jain X-Patchwork-Id: 13910084 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C4B1DE7717F for ; Mon, 16 Dec 2024 16:53:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5396D6B0093; Mon, 16 Dec 2024 11:53:30 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4E6BF6B009C; Mon, 16 Dec 2024 11:53:30 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 338DB6B0096; Mon, 16 Dec 2024 11:53:30 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 13A118D0001 for ; Mon, 16 Dec 2024 11:53:30 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 88ADB1405C4 for ; Mon, 16 Dec 2024 16:53:29 +0000 (UTC) X-FDA: 82901417886.29.A24B850 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf17.hostedemail.com (Postfix) with ESMTP id 5AED740003 for ; Mon, 16 Dec 2024 16:53:05 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf17.hostedemail.com: domain of dev.jain@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=dev.jain@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1734367994; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=39qRkFxeQ7cExo69lLQfkW13STIJ4xIJ5DI6eTVjZ7A=; b=W1YM5oqpiCYN3qBHEaDn95PKSCxrrTnlUEWcOXa4FmT+/ChwiayUm9b0ujCYeeWCyZHCxM MjI4kjJVZZuQumcpNcSNgtE8Q8RfT7KMdAZcfo+3SqJ86Qx/qTaf01iIx3ItJwFapIseR1 ta1JWbenOvVyRpJHJ4EdN3+URBGJN8I= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf17.hostedemail.com: domain of dev.jain@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=dev.jain@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1734367994; a=rsa-sha256; cv=none; b=7PiS/1r+7wHavmrY/4OmRVW3CHkigu+pBGKKgkfqBqyoShfvNSQDvT1jlpuL/GI00zrGia 9E24HY2uD52FCgg9U184vQ7JpFN0rHRgEt4kX0KIHcog0qFTtdUIifSfS4+pEmTbAy2AH2 CbZo4zwXXHKBEUCEJ7dm/TKyFXhUqc0= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 32FFE16F8; Mon, 16 Dec 2024 08:53:55 -0800 (PST) Received: from K4MQJ0H1H2.arm.com (unknown [10.163.78.212]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 552383F528; Mon, 16 Dec 2024 08:53:16 -0800 (PST) From: Dev Jain To: akpm@linux-foundation.org, david@redhat.com, willy@infradead.org, kirill.shutemov@linux.intel.com Cc: ryan.roberts@arm.com, anshuman.khandual@arm.com, catalin.marinas@arm.com, cl@gentwo.org, vbabka@suse.cz, mhocko@suse.com, apopple@nvidia.com, dave.hansen@linux.intel.com, will@kernel.org, baohua@kernel.org, jack@suse.cz, srivatsa@csail.mit.edu, haowenchao22@gmail.com, hughd@google.com, aneesh.kumar@kernel.org, yang@os.amperecomputing.com, peterx@redhat.com, ioworker0@gmail.com, wangkefeng.wang@huawei.com, ziy@nvidia.com, jglisse@google.com, surenb@google.com, vishal.moola@gmail.com, zokeefe@google.com, zhengqi.arch@bytedance.com, jhubbard@nvidia.com, 21cnbao@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Dev Jain Subject: [RFC PATCH 10/12] khugepaged: Skip PTE range if a larger mTHP is already mapped Date: Mon, 16 Dec 2024 22:21:03 +0530 Message-Id: <20241216165105.56185-11-dev.jain@arm.com> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: <20241216165105.56185-1-dev.jain@arm.com> References: <20241216165105.56185-1-dev.jain@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 5AED740003 X-Stat-Signature: 41k5utfpmte69w19cmw4k1korqdjcezg X-Rspam-User: X-Rspamd-Server: rspam11 X-HE-Tag: 1734367985-221203 X-HE-Meta: U2FsdGVkX1/VeAx9Jnpzwmp7IZEtXqWoQjsexgQOAgaJYtQngzw7f+nMXGZEb8hmcFtSW939yop9CanUWBmtLteTolVTLTBcVN7ow41pyAJdI6EfUcd1SAcMwLUha0V2qW7xi0dYQiuXGc744TBS2XjTYWciHcU4FxMFXkD5J7DkeWhgAIqxnN93AC3Pj+ObC2CRnwh5yx+WPI3jC5zLEnBoiM+J0qaqFX0aTpwQ2n/fcjXlG6H/SrCdItgLwezcrn9arLWXFTQWrEdcScvWCwAm5tKmIMIikuCqicjj4I9vP2gUfkdHgIbbf3K8fSusFbVyVrMAKrZ9jKKYfoSREyTT1GAd+c1rqwrtqTtCqLPw5jtPkHeWWRMADhPNZT9dnebEYuA082BmITHuFriahkuWAo/O3+sxYr2u/FpKOIeL3hAGPUcr1oYzJA3e0whrYvNQtLuVI3cV1zqvZq65OTey25XzJmC8YYBxMr9lUhQMIMxtMlAkm9jALRQnhWr09vYwhW0os15WIQBgIYkJkZJAy7qXVDAl8G1CWORVDB318PhhWGSNj3j7TgEHRfks4Zozc8WZd8kVa1PMkwjaJHMMLAMRv9lBBKnCzsiXZiTePdSP7A//xdJQALd3X6hwxj3JCfRV7wI/1Nnyd5nYTwOFlxzW9xfoa0QLr6z0GUEby2/lgoPI+sbF2wyGFZPBSBVKJJwyw0TZ1ZIDd2iX7URDwVplUmJvUglkbcoCpLgUiidzduE30OnhRxMf+YTHUVkRObteIBAndzRgIt3bJCzp8TgDkB8cGOUb9kCC5JxbFLHjpNC7RYVnrwTcwdNqN9k7iihGa5Xd6eajF1/Dh9xMGLbDBSp3kVz3pJqVOUMDQKxR4VKPjb7wqcpDv8+msmL6b+Nw8auibJaHGHWquacbmmnxQ8kDPDZ7TjToRj1JajFNVRdZ8Edr0BS6G1ZxAITUmePTKEMRhdVVq4o w1eLHRVO PTBwEVR9C5GqBGSpjG4vOn72bQkpbFMr7Aj5ApUzwyT/YXYJYUfpzccIxkTU6ZjmdSbGw32Yo/X0iZAe4QF3M8Ug8DD5O/1aD1jNViJ//7P7knbNCHEnz8eSlSbI0RrN8PG7C3bTKWp40cbNSbD2heePE2CtdICg08xA98NSPKUyldffdc86Zv142GK5kq/DYvz2Lkjd3+Py8cEk7UUWnPnviPV5/QIFiWDnnJKgU23ngx4tFcEbWa0SOLucVktwTsJ8H/4i96g8TUtfUejIEw7w5KMFwslSX7GC68VbaXxgIotz6k0r8sPbRLF8ix2qy0g7C X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: We may hit a situation wherein we have a larger folio mapped. It is incorrect to go ahead with the collapse since some pages will be unmapped, leading to the entire folio getting unmapped. Therefore, skip the corresponding range. Signed-off-by: Dev Jain --- In the future, if at all it is required that at some point we want all the folios in the system to be of a specific order, we may split these larger folios. mm/khugepaged.c | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 8040b130e677..47e7c476b893 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -33,6 +33,7 @@ enum scan_result { SCAN_PMD_NULL, SCAN_PMD_NONE, SCAN_PMD_MAPPED, + SCAN_PTE_MAPPED, SCAN_EXCEED_NONE_PTE, SCAN_EXCEED_SWAP_PTE, SCAN_EXCEED_SHARED_PTE, @@ -609,6 +610,11 @@ static int __collapse_huge_page_isolate(struct vm_area_struct *vma, folio = page_folio(page); VM_BUG_ON_FOLIO(!folio_test_anon(folio), folio); + if (order !=HPAGE_PMD_ORDER && folio_order(folio) >= order) { + result = SCAN_PTE_MAPPED; + goto out; + } + /* See hpage_collapse_scan_ptes(). */ if (folio_likely_mapped_shared(folio)) { ++shared; @@ -1369,6 +1375,7 @@ static int hpage_collapse_scan_ptes(struct mm_struct *mm, unsigned long orders; pte_t *pte, *_pte; spinlock_t *ptl; + int found_order; pmd_t *pmd; int order; @@ -1467,6 +1474,24 @@ static int hpage_collapse_scan_ptes(struct mm_struct *mm, goto out_unmap; } + found_order = folio_order(folio); + + /* + * No point of scanning. Two options: if this folio was hit + * somewhere in the middle of the scan, then drop down the + * order. Or, completely skip till the end of this folio. The + * latter gives us a higher order to start with, with atmost + * 1 << order PTEs not collapsed; the former may force us + * to end up going below order 2 and exiting. + */ + if (order != HPAGE_PMD_ORDER && found_order >= order) { + result = SCAN_PTE_MAPPED; + _address += (PAGE_SIZE << found_order); + _pte += (1UL << found_order); + pte_unmap_unlock(pte, ptl); + goto decide_order; + } + /* * We treat a single page as shared if any part of the THP * is shared. "False negatives" from @@ -1550,6 +1575,10 @@ static int hpage_collapse_scan_ptes(struct mm_struct *mm, if (_address == org_address + (PAGE_SIZE << HPAGE_PMD_ORDER)) goto out; } + /* A larger folio was mapped; it will be skipped in next iteration */ + if (result == SCAN_PTE_MAPPED) + goto decide_order; + if (result != SCAN_SUCCEED) { /* Go to the next order. */ @@ -1558,6 +1587,8 @@ static int hpage_collapse_scan_ptes(struct mm_struct *mm, goto out; goto maybe_mmap_lock; } else { + +decide_order: address = _address; pte = _pte;