From patchwork Tue Mar 26 20:28:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13605070 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B1D14C54E67 for ; Tue, 26 Mar 2024 20:28:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C45346B0099; Tue, 26 Mar 2024 16:28:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 89FBA6B0093; Tue, 26 Mar 2024 16:28:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 679C26B0095; Tue, 26 Mar 2024 16:28:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 0E8D56B0098 for ; Tue, 26 Mar 2024 16:28:40 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id C83B41406DE for ; Tue, 26 Mar 2024 20:28:39 +0000 (UTC) X-FDA: 81940328358.26.07429D7 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf26.hostedemail.com (Postfix) with ESMTP id CC3B1140015 for ; Tue, 26 Mar 2024 20:28:37 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=RcI9e6JX; dmarc=none; spf=none (imf26.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711484918; a=rsa-sha256; cv=none; b=AqSWUIyi9dk7aj4jqVjThuAmGjzcW2iHhoyv92kBT06oR6+OGkSfIXJ65tI0UrNRQhqsM9 c5NtxB/RiuxVB3t4Tymdryrdqyfb3gUhr2BV1Xdk7wWae844UgPl+KQetQwQEpJ8ujHgLO tyj0450oUkAufLodjcNqrkzFvGxi1LM= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=RcI9e6JX; dmarc=none; spf=none (imf26.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711484918; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=tqTrfCsLVUpO7BugEdTxqQoWqJdTsnkKNlvGkBxqLaA=; b=o4s9DfnPDEfimCy9VPH/uMSGpusma7VPHQEUq1yjmBoSxybsxM/yTsSd4lISPcXvQ7FeHl 5vRul/eYdW9c+eWmUzRznY4lv/5CBpSqUx8T1vWSpFYsD9pdc1+DxtvOtfhkSTGv5lv71X JoPlntEgjMxujHvOnZ7bMRy/V4U9Kf8= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=tqTrfCsLVUpO7BugEdTxqQoWqJdTsnkKNlvGkBxqLaA=; b=RcI9e6JXAb+jplwu/Gg/0XJJHT 0oqdMzS3blalS+NwfNKbV0W+Zc9lUzw9RdXZW2oTZsmMnF2rsMcaCP43nuMz4BBZwb5BUWnSvYXjK fQ6X1bTkZs0OFtdn1jBJEgFTI96snnI0l/cTfUGhtsHx+avXhPCXFj23IX1K3CqUg8L/apZievtwR P4iO21g7WSnKVAuIrrKqqN0Ae1PEljWSlT/Zp9BewP4yUWIlZ5aT5DM8DKEdZOUjBptyiPO2KCWTI TMervibh8yT+3Rae1asYZNK03ge859rhp1ql4308BeLqUwcQKDTgyWXEUs22lyqF6eloR2uf7wacb cLedaLxg==; Received: from willy by casper.infradead.org with local (Exim 4.97.1 #2 (Red Hat Linux)) id 1rpDPQ-00000002CGF-1EIZ; Tue, 26 Mar 2024 20:28:36 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org Subject: [PATCH 1/8] sparc: Use is_huge_zero_pmd() Date: Tue, 26 Mar 2024 20:28:21 +0000 Message-ID: <20240326202833.523759-2-willy@infradead.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240326202833.523759-1-willy@infradead.org> References: <20240326202833.523759-1-willy@infradead.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: CC3B1140015 X-Stat-Signature: 5pxqca6dijbjk79zy7b34j461w4qwt6j X-HE-Tag: 1711484917-460570 X-HE-Meta: U2FsdGVkX18WtX2qSjtdrj8lCl+xnFG2FXN4nXPfBIwNKTEQMB1RSL71xr/scTWyEKuY4ZTSoGhaoY/Hxhc1GchUQlG4ry/bpG9Yu5t3hOBVISiE8SCUPbCHyICLdn8QR/z1GrU17tRHNfm9NgvJz5PsytPW3VXCFiU7nXluNIBCnsHkgDU6MAKfCuwggBYF0ypKQr9kf+nz0why7k0MYkZ3zBCZlHoVfwRxOkFZkgBfFQksJeqpSE3pQ8QkvCTMhsrRa6OhQ88/1AeCykr4HhFVZAqCykH12c0z3i0DWSghUMYbv/ZXEon2YcfYOwKSyEhx63ttN5FauEX02087uT5qLH2xj7w0SGYdLmDBmUFERP3XREZVcq4jcpnyc7Pp2FSiaBPsXuyRvnKe7Tr9g6W/8sqs4SHE5iQpO44KdDT8vpWsQ52ns5HByu2T5OhU89ACBiPf09mWc2LBMKn8X8Np0C8ftxydv+ywOfLoD9VpnzlcH09xl4yIygTDV9NOHOYkGtS8e3EIq+l+hB/t9L/VlrT2ORpnmiNPYwXyvEOUAhYo+DwUq4yc2lf0gHWE6C9PmXJMpCdhdjUIkPcPWWpGfbqm3DJTHglw4Rzs8eJ1DESQKEZFAvQDdY9+IKJUuPITPgcXTg0jtE0hIhkqI6xu3I9lzFAC7z2BgFIC0I8rgOZqK+YzRJP5RspadSo4XdBe46Lif9MeoKnYwOJo6woBoqd+AB5jTOA6DRNq9p/XgMZCtoJiGSXJ/cbunzYRaL7q1qysiW+TpDctycQeaT0/IhrKDI7cCUjS/4UiRw1WQ5H+oA3C/DhdbINDI6kwYDxyrGP7aCRHg8uHHezhTUgKFEQU64YysCRYC6eGVS8/DJYPKahe9ZUDXCZ3Xur3qIOExIK4fGekceFQWEYo89LN8KxjPopziMsOpsn1MSxpKxUpFa/KRKEdBA8OelZy/COP4/wo2/uoWWSJKnO IjxgKfYs Buwo8L236W8UvILogALJC5GSsPTvlx4Rql4ILSs/qyzXkYb21p1uhlZU8Zecgz8sqW8ZMz8B9GcB4QPkvUdqwDpV5HyO8tVIfY/0xcZhn0EaULUG9Va1US/yorv1Td5tA0TE47ihx2bYtvxUzV4u45elukEPfrZHHtJi7PX/zi01p4y843kWSPtF1cm1LJK2b6zMVwQelNkeA9z5wVKM/+1xdonI1sFs+EIGg X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: There's no need to convert to a page, much less a folio. We can tell from the pmd whether it is a huge zero page or not. Saves 60 bytes of text. Signed-off-by: Matthew Wilcox (Oracle) --- arch/sparc/mm/tlb.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/arch/sparc/mm/tlb.c b/arch/sparc/mm/tlb.c index b44d79d778c7..19642f7ffb52 100644 --- a/arch/sparc/mm/tlb.c +++ b/arch/sparc/mm/tlb.c @@ -183,12 +183,12 @@ static void __set_pmd_acct(struct mm_struct *mm, unsigned long addr, * hugetlb_pte_count. */ if (pmd_val(pmd) & _PAGE_PMD_HUGE) { - if (is_huge_zero_page(pmd_page(pmd))) + if (is_huge_zero_pmd(pmd)) mm->context.hugetlb_pte_count++; else mm->context.thp_pte_count++; } else { - if (is_huge_zero_page(pmd_page(orig))) + if (is_huge_zero_pmd(orig)) mm->context.hugetlb_pte_count--; else mm->context.thp_pte_count--; @@ -259,7 +259,7 @@ pmd_t pmdp_invalidate(struct vm_area_struct *vma, unsigned long address, * Sanity check pmd before doing the actual decrement. */ if ((pmd_val(entry) & _PAGE_PMD_HUGE) && - !is_huge_zero_page(pmd_page(entry))) + !is_huge_zero_pmd(entry)) (vma->vm_mm)->context.thp_pte_count--; return old; From patchwork Tue Mar 26 20:28:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13605067 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4C775CD1283 for ; Tue, 26 Mar 2024 20:28:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 34B126B009C; Tue, 26 Mar 2024 16:28:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2AAE16B0095; Tue, 26 Mar 2024 16:28:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 123296B009A; Tue, 26 Mar 2024 16:28:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id EC02B6B0095 for ; Tue, 26 Mar 2024 16:28:39 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id C7955160165 for ; Tue, 26 Mar 2024 20:28:39 +0000 (UTC) X-FDA: 81940328358.24.5BA9C23 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf05.hostedemail.com (Postfix) with ESMTP id B59BE10000B for ; Tue, 26 Mar 2024 20:28:37 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=ST1EguAN; dmarc=none; spf=none (imf05.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711484918; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=rtFLGbGqXpsjSue0ZqqRPXjXFuO8I+KTByqez1b7bEw=; b=ODPpPoPTz54DAyaPfhxxCxO6aLgsKhWqsVKfj3iEzWlexacbUoFSFW2HDDxPq1eDsdDgog oLNvhpd6a5JekqVPiXqRBee4JAiu5+09MTEfw3MtJdnkj2YbGo65pNxCF/+e1XxQMvEHqN FZS85XiYE3vMocQcRADqBKfWEGMXx2M= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=ST1EguAN; dmarc=none; spf=none (imf05.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711484918; a=rsa-sha256; cv=none; b=lZDylh+o6WgqbEdmIXux/Mr+lzZDXtr2u8lRmkgRklpncxx4ZVwJt5wkV2qZ5OFNnPwgnS jOl23U6CD7ORQi2chVgWvi/8C2zDNN3LeD9Lud8em2aJ2T0OJEefJC68DhCc3PpyHbw8DU Y+r9VcPDpHerd0ks0CRpiEvz6W2eu3c= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=rtFLGbGqXpsjSue0ZqqRPXjXFuO8I+KTByqez1b7bEw=; b=ST1EguANN2O3F1yfmR/V1z/Y6k CZdv5TxWFfImDgLBx+j2bO6GScpjQgXpRX/DbdZS2SByG7B/PLFS+2YdLZ2eznTf0Snf6VCnrS8RK yIIxrTN3GQd+KqFLfg8yY01elwA3jkHN3q5KPOIbQIwVzp3bjPHky2XLv9g6sI9Fkaxqt+MG+EWqk MdgpJGrfxRTu70wp4Wg0q3EZHebekaYYI9+bqXu5KZDxAsdPYv9vQkNPCNbUHI3+kfyGY7RfCggQe /GK8P7roppJTk3HGS6/4bT5ruhRPgzVA2FOw7E33xTPLTJQfH2B02vGBHEJoK8dlqZVeTPuVJjfAW b647SMPA==; Received: from willy by casper.infradead.org with local (Exim 4.97.1 #2 (Red Hat Linux)) id 1rpDPQ-00000002CGL-1pEM; Tue, 26 Mar 2024 20:28:36 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org Subject: [PATCH 2/8] mm: Add is_huge_zero_folio() Date: Tue, 26 Mar 2024 20:28:22 +0000 Message-ID: <20240326202833.523759-3-willy@infradead.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240326202833.523759-1-willy@infradead.org> References: <20240326202833.523759-1-willy@infradead.org> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: 1hgcmx6h74ejtca89ga5jmt468saidjo X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: B59BE10000B X-HE-Tag: 1711484917-653137 X-HE-Meta: U2FsdGVkX18IXcVytZz6WKRhbBzKL55yuTXnjtt/lWAGwV9CLzmrbCBXenubtSvuzy4X/bNNeHAnzxstKq9QFHTGvIkNW3mLpc7NkGcCCbVW7+7U0ME374pjAeGf/veKBkFc6o3rPcAuINHe+WNb9+s9XnzriDJs788+2Xj55vO1oCRyBexpSUtRwE3tSE401we1inhqor4FJfMRHsnc30mIwSBkAr3otBO8dUmBNtVW0rWps1zIU7E9zgKTjxCyy8EPrl7o75tM/HBFtd/Vwl9MO/fFbY18VTVxSt5+LguVg6oVnaawOHU2RZksX0NY5nJupY1Zc+m4iI3Ygmj5QFn/9Xm8fCOu7hWx0abJhR6h/gbHIgGDc9VjIIS14kjSOSOyGRhMwIQeNhhlldhRGe3i7rQDPAbOKF7IjN0V3nL5oJwICBEw0Ltc/hunzEyKYXuTyMK2OQoytzS1kJPkEXdAOTanPCle54G3LeBkxJrR/umqWED/K6Rv7uD11QUZTh3mVcpPSzcC+RTm+z1E9xYYAGWQkIhHpggy/6C5swS3s7xvcITwdAsUjYnG7vQ7HjRlP1I91RkysiuDhTUgyIEgunCGoZwOrCXTuzBtWu7sIouZCCFga9LWkOddAc+XcaVL3rzfASVKhLsbyTcoLnMlBJWb4WnBhh585EcdHRrg7xGUJF+thpmYspWHqyJOfmjedfIgBSN38Lhjxv2LQdbz11QB8KHLWtKeIbBGdzwCHj3qeYuUpJrrog8RNCcWWxEdq1k6qRFo2V3w5FfDnlNYOpOKC7AmlY8+xGDP6gCbLM9a7ZrMGy+CsJrJkVPBWGii6j5v2MgMRvjgurcOeBTmGseXZeD4TIZUQbBmaWOAxFFIReFXgTT4/hyoQ/1Uw0IXFBV6CnYsgdljXFO54/W5CBtVb52aWMFJid5YQW0gclRg/uWJ7NPpzGL4NU7Vsv2s0CsX8nPzPVrshN4 V5LVWine kx7CYtCYySiPbEL575I2y4iKWTEQtQHn9bT8Ubl6CY68He4YWnBbOvxr+Z7OOrAy7/5T2ENBrdIvwUlHO2svg4xGNQpvIZqQe1Ss72gygums0433xjoDj0EV1FaymB4SmX4F372Bq8EtJT1i2JdWcZX3Q0PXb7OZlFJUckj8SeYVVQ0xQ7jzXFxnaCNBaZEnue7Zq5oV38bgV3Xqh/t3rFc/RNhfM3YYc8DS3 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This is the folio equivalent of is_huge_zero_page(). It doesn't add any efficiency, but it does prevent the caller from passing a tail page and getting confused when the predicate returns false. Signed-off-by: Matthew Wilcox (Oracle) --- fs/proc/page.c | 2 +- include/linux/huge_mm.h | 10 ++++++++++ mm/huge_memory.c | 6 +++--- mm/mempolicy.c | 2 +- mm/swap.c | 2 +- mm/swap_state.c | 2 +- mm/userfaultfd.c | 2 +- 7 files changed, 18 insertions(+), 8 deletions(-) diff --git a/fs/proc/page.c b/fs/proc/page.c index d6953f95e3b4..5bc82828c6aa 100644 --- a/fs/proc/page.c +++ b/fs/proc/page.c @@ -155,7 +155,7 @@ u64 stable_page_flags(const struct page *page) else if (folio_test_large(folio)) { if ((k & PG_lru) || is_anon) u |= 1 << KPF_THP; - else if (is_huge_zero_page(&folio->page)) { + else if (is_huge_zero_folio(folio)) { u |= 1 << KPF_ZERO_PAGE; u |= 1 << KPF_THP; } diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index 1540a1481daf..600c6008262b 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -356,6 +356,11 @@ static inline bool is_huge_zero_page(const struct page *page) return READ_ONCE(huge_zero_page) == page; } +static inline bool is_huge_zero_folio(const struct folio *folio) +{ + return READ_ONCE(huge_zero_page) == &folio->page; +} + static inline bool is_huge_zero_pmd(pmd_t pmd) { return pmd_present(pmd) && READ_ONCE(huge_zero_pfn) == pmd_pfn(pmd); @@ -485,6 +490,11 @@ static inline bool is_huge_zero_page(const struct page *page) return false; } +static inline bool is_huge_zero_folio(const struct folio *folio) +{ + return false; +} + static inline bool is_huge_zero_pmd(pmd_t pmd) { return false; diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 4556144cff09..78f9132daa52 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -789,12 +789,12 @@ struct deferred_split *get_deferred_split_queue(struct folio *folio) } #endif -static inline bool is_transparent_hugepage(struct folio *folio) +static inline bool is_transparent_hugepage(const struct folio *folio) { if (!folio_test_large(folio)) return false; - return is_huge_zero_page(&folio->page) || + return is_huge_zero_folio(folio) || folio_test_large_rmappable(folio); } @@ -3085,7 +3085,7 @@ int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, } - is_hzp = is_huge_zero_page(&folio->page); + is_hzp = is_huge_zero_folio(folio); if (is_hzp) { pr_warn_ratelimited("Called split_huge_page for huge zero page\n"); return -EBUSY; diff --git a/mm/mempolicy.c b/mm/mempolicy.c index 3fae74b62c51..6e7069ecf713 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -510,7 +510,7 @@ static void queue_folios_pmd(pmd_t *pmd, struct mm_walk *walk) return; } folio = pfn_folio(pmd_pfn(*pmd)); - if (is_huge_zero_page(&folio->page)) { + if (is_huge_zero_folio(folio)) { walk->action = ACTION_CONTINUE; return; } diff --git a/mm/swap.c b/mm/swap.c index 500a09a48dfd..f72364e92d5f 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -985,7 +985,7 @@ void folios_put_refs(struct folio_batch *folios, unsigned int *refs) struct folio *folio = folios->folios[i]; unsigned int nr_refs = refs ? refs[i] : 1; - if (is_huge_zero_page(&folio->page)) + if (is_huge_zero_folio(folio)) continue; if (folio_is_zone_device(folio)) { diff --git a/mm/swap_state.c b/mm/swap_state.c index bfc7e8c58a6d..2deac23633cd 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -301,7 +301,7 @@ void free_page_and_swap_cache(struct page *page) struct folio *folio = page_folio(page); free_swap_cache(folio); - if (!is_huge_zero_page(page)) + if (!is_huge_zero_folio(folio)) folio_put(folio); } diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index 163511ab5896..4f1a392fe84f 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -1699,7 +1699,7 @@ ssize_t move_pages(struct userfaultfd_ctx *ctx, unsigned long dst_start, !pmd_none(dst_pmdval)) { struct folio *folio = pfn_folio(pmd_pfn(*src_pmd)); - if (!folio || (!is_huge_zero_page(&folio->page) && + if (!folio || (!is_huge_zero_folio(folio) && !PageAnonExclusive(&folio->page))) { spin_unlock(ptl); err = -EBUSY; From patchwork Tue Mar 26 20:28:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13605066 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4FCF1C54E67 for ; Tue, 26 Mar 2024 20:28:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DA7006B0089; Tue, 26 Mar 2024 16:28:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D572B6B0093; Tue, 26 Mar 2024 16:28:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C45696B0095; Tue, 26 Mar 2024 16:28:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id B34836B0089 for ; Tue, 26 Mar 2024 16:28:39 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 7C38080472 for ; Tue, 26 Mar 2024 20:28:39 +0000 (UTC) X-FDA: 81940328358.04.F8B763B Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf11.hostedemail.com (Postfix) with ESMTP id D58F240011 for ; Tue, 26 Mar 2024 20:28:37 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b="vYkf3/Ev"; dmarc=none; spf=none (imf11.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711484918; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Fk/PBeTwF3EdydrfntGNeSILbnkq3z7D7Gx9zai2XuQ=; b=lJiJ4MAy6sUPPNWZgpRx93i89yMm9mp0qHQF1FgxCYFqXkXZ/jV/1CRst748r1wKyP10Kp lcqhO+YYUqsIu55Fj8qOVrzgI+6MUPwqOabWX+vM7FzGCe6THGhSkCiWFPVLyF1AHCDwFr 8Zgha6zk2GYoPO2Y6yHnAq+d7jMiEc0= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b="vYkf3/Ev"; dmarc=none; spf=none (imf11.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711484918; a=rsa-sha256; cv=none; b=HOdkCD41n8xz4paEVvGV1fRlT0ZhBslplICLYyk0PEtadO0CttuH0mi9ab9OgjfaRaUBJ5 I5kSK/W4xx5V6uPQ6Z+TNsb9yOYP8MHF+P7QCc9fIqJ3TZwiLb9H1tUv+sL6cUSyJsuDIw 3eg2+SuJOsFXiW8Aj/Og3XRcnNJXEk8= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=Fk/PBeTwF3EdydrfntGNeSILbnkq3z7D7Gx9zai2XuQ=; b=vYkf3/EvK6jT6LVS2H5DMniySH VVyyFJrN+99AqItf5QcWoI+UovVDIOl/DMDbH0zASNYm4RnlWe5gR3Nv2SknRhUZkeB0ObAIdF/yX ac1ybl2KG58aKlM1yDag3/CO2X198uhBj4NWcAJyzKZfbNu6nH8vKlC5EsDd8O3QyxJh80o3O6UFe lD0vz6Vtu0pi8WYUd1H5OsV9JiZmBbEHCKqb70nRhqa8cx6J7G67iiegfkJ3dM3OGi4KdXoOaHNfG MRgSbuuWCCKWUkJgasH6buy4knv3x3DpaXmj84ygqyEDWs1RfX1520635fXf4g47Hs4v6LtC0mqzu /F45iItQ==; Received: from willy by casper.infradead.org with local (Exim 4.97.1 #2 (Red Hat Linux)) id 1rpDPQ-00000002CGR-2UDq; Tue, 26 Mar 2024 20:28:36 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org Subject: [PATCH 3/8] mm: Add pmd_folio() Date: Tue, 26 Mar 2024 20:28:23 +0000 Message-ID: <20240326202833.523759-4-willy@infradead.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240326202833.523759-1-willy@infradead.org> References: <20240326202833.523759-1-willy@infradead.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: D58F240011 X-Stat-Signature: edhffci4b7y4pdopgd47f38cyk6651wz X-HE-Tag: 1711484917-727475 X-HE-Meta: U2FsdGVkX19XQtPdsZDsQDYxaqMkzTJIpY0d6vFBkp9LyPCPhRL80OJkB2lq6g1wfOE/CDNKMnuH+cJat8HeQSjw1jymohYqnCBCjIdn8ikZqRKfcYXXH5BmhqUfpJm0kFSxXtCrfYNLYBbwzPkM5FQ60+ZdozSVoXEmWh+4460KCQhzOnQN7Xz8E3kTWkynbW/EeeTXNX3iEYtUSyRawteo2tekKBzQFCUdqilkRYPlYosXvUKcbBNpZ+byNfeDd69jdhYNhaKl5kDpZBBvvHPJWBfVGsm0W1OI+zB0TOyebOQVyDUo0ii1P8YQicF3Md42pqKeb+O+uTKCpibcPJMdQ9McxGtMUYLO+9AN6Wmv6N0CSPb1lK+JgVTJP1p5WQdQi4X2h++sdCVkONvfskfkL5KQcs+4kg/wGzjDERjtF12QTaJe168+dHnwKAjBfGgS/Df1vlef/nZlGXmLRbiF4O6JnUAUtWfIpVDHMrrsQank872wbny1lFh69p/JblqEInqPe6F44ZsuUPcH4w3yeRAd7d22DpUeyFLe+cjkVhalOhv6WXzuYuIg/5wdkMukM1wPlkWQnxz5lNLqWxMTFaXErZjUwffq+i3tpnHlGsPZmtZrTMo3ATxt6TwmDQnlj5OcOdHxpN+5+9eGhM10q/Q5ZU36J1jNSbW/XLDUgXxcMOzPJi+hBkTW83aJuefVEDahgTncrhPkOYTSTb+vwGqQyYNZ5ouHtGicbYktQXw7ppXP1aWDQLqoApA7C2T7M0lguRUPJ1VenWv7CZ4s6ghAK8gQgdyEubCFO4rRWTjFZtVCyrFgivWLOqQOq5g3NsKiYh+Twv4SpYqqGyXvdk2RA2g5JiTo1AjqLePsUGqTn0G39+D7okLaogyWEeP/Xfof3EuKAFuZa7/CJnrMEkivuqeP X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Convert directly from a pmd to a folio without going through another representation first. For now this is just a slightly shorter way to write it, but it might end up being more efficient later. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: David Hildenbrand --- include/linux/pgtable.h | 2 ++ mm/huge_memory.c | 6 +++--- mm/madvise.c | 2 +- mm/mempolicy.c | 2 +- mm/mlock.c | 2 +- mm/userfaultfd.c | 2 +- 6 files changed, 9 insertions(+), 7 deletions(-) diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h index 600e17d03659..09c85c7bf9c2 100644 --- a/include/linux/pgtable.h +++ b/include/linux/pgtable.h @@ -50,6 +50,8 @@ #define pmd_pgtable(pmd) pmd_page(pmd) #endif +#define pmd_folio(pmd) page_folio(pmd_page(pmd)) + /* * A page table page can be thought of an array like this: pXd_t[PTRS_PER_PxD] * diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 78f9132daa52..8ee09bfdfdb7 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1816,7 +1816,7 @@ bool madvise_free_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, goto out; } - folio = pfn_folio(pmd_pfn(orig_pmd)); + folio = pmd_folio(orig_pmd); /* * If other processes are mapping this folio, we couldn't discard * the folio unless they all do MADV_FREE so let's skip the folio. @@ -2086,7 +2086,7 @@ int change_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, if (pmd_protnone(*pmd)) goto unlock; - folio = page_folio(pmd_page(*pmd)); + folio = pmd_folio(*pmd); toptier = node_is_toptier(folio_nid(folio)); /* * Skip scanning top tier node if normal numa @@ -2663,7 +2663,7 @@ void __split_huge_pmd(struct vm_area_struct *vma, pmd_t *pmd, * It's safe to call pmd_page when folio is set because it's * guaranteed that pmd is present. */ - if (folio && folio != page_folio(pmd_page(*pmd))) + if (folio && folio != pmd_folio(*pmd)) goto out; __split_huge_pmd_locked(vma, pmd, range.start, freeze); } diff --git a/mm/madvise.c b/mm/madvise.c index 7625830d6ae9..1f77a51baaac 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -363,7 +363,7 @@ static int madvise_cold_or_pageout_pte_range(pmd_t *pmd, goto huge_unlock; } - folio = pfn_folio(pmd_pfn(orig_pmd)); + folio = pmd_folio(orig_pmd); /* Do not interfere with other mappings of this folio */ if (folio_likely_mapped_shared(folio)) diff --git a/mm/mempolicy.c b/mm/mempolicy.c index 6e7069ecf713..4f5d0923af8f 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -509,7 +509,7 @@ static void queue_folios_pmd(pmd_t *pmd, struct mm_walk *walk) qp->nr_failed++; return; } - folio = pfn_folio(pmd_pfn(*pmd)); + folio = pmd_folio(*pmd); if (is_huge_zero_folio(folio)) { walk->action = ACTION_CONTINUE; return; diff --git a/mm/mlock.c b/mm/mlock.c index 1ed2f2ab37cd..30b51cdea89d 100644 --- a/mm/mlock.c +++ b/mm/mlock.c @@ -378,7 +378,7 @@ static int mlock_pte_range(pmd_t *pmd, unsigned long addr, goto out; if (is_huge_zero_pmd(*pmd)) goto out; - folio = page_folio(pmd_page(*pmd)); + folio = pmd_folio(*pmd); if (vma->vm_flags & VM_LOCKED) mlock_folio(folio); else diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index 4f1a392fe84f..f6267afe65d1 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -1697,7 +1697,7 @@ ssize_t move_pages(struct userfaultfd_ctx *ctx, unsigned long dst_start, /* Check if we can move the pmd without splitting it. */ if (move_splits_huge_pmd(dst_addr, src_addr, src_start + len) || !pmd_none(dst_pmdval)) { - struct folio *folio = pfn_folio(pmd_pfn(*src_pmd)); + struct folio *folio = pmd_folio(*src_pmd); if (!folio || (!is_huge_zero_folio(folio) && !PageAnonExclusive(&folio->page))) { From patchwork Tue Mar 26 20:28:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13605073 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4DB61C54E67 for ; Tue, 26 Mar 2024 20:28:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9FE326B009B; Tue, 26 Mar 2024 16:28:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 939E26B0096; Tue, 26 Mar 2024 16:28:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7B3196B009E; Tue, 26 Mar 2024 16:28:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 63BC66B0096 for ; Tue, 26 Mar 2024 16:28:41 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 300DC140431 for ; Tue, 26 Mar 2024 20:28:41 +0000 (UTC) X-FDA: 81940328442.22.A1633CC Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf11.hostedemail.com (Postfix) with ESMTP id A74F14000C for ; Tue, 26 Mar 2024 20:28:39 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=ZAeDBXor; dmarc=none; spf=none (imf11.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711484919; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=kpw//YUN/Z8trrNfMYgvIjdADXKrm2X8DtHY91ODnyE=; b=veQVYUohgBmRSRK4oWPB8m/ZQ0WvmOULB48RM1Tzf9FVaC+s6zbW37mYSKts+8gpNPutUU nuw+aQgW0OtgKMH57gAotgvxwGtdPESKPTnOtKuz4OKZeVFJabfziAFHxhRBqoo6Jczdvs 1nQFbkSM+oSTZcORsHpi+7JcSLSxKWc= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=ZAeDBXor; dmarc=none; spf=none (imf11.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711484919; a=rsa-sha256; cv=none; b=7+AYrV5P2TOMNFS24q5g6zimBEd28NzhaCAY5Mhmu/zP4gkQwDci9AkDQ1tkraad4DeSSC 6Fuf0TL+MkCPxt1Mh0uWCwS+/BurUhkBLXPfMaXQ1W1D79Le31CGwA3XBqWZRmTkm6mY4k oMnGTphdeXYAzFDSKLAbEQW/4CqWQXQ= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=kpw//YUN/Z8trrNfMYgvIjdADXKrm2X8DtHY91ODnyE=; b=ZAeDBXorkmz6TIILRtABrSJ9sy A2mpPilG74gqWzMeJkTQQTa+KXY7hqpIwViWebt1fE+Iae7Scku4yKWlJBc7fMMve5UCnSLzRaIuL zLLxZOUYdNfTB7TWJlRckMExgUHODfCUjpODqSNx3SHgaeI+fgF3PAyDQ6U+uji0Gol6kWCLVH2gK HmRMLpJ3q1G3xv/t81Dx7z6hGYwHPA/w/YUPbeDgmGiXkGgNqIg+LD+xW5yUbZIIu6789VwDmvWjP udIjUBcUAzKkHVXGM5l9+pA+gTXM8fSRZ3ZdN//MkCxdEd8TVU4rIuaj9ZrZS9ZnFGtc+nnMcVG+e a2V/zLiA==; Received: from willy by casper.infradead.org with local (Exim 4.97.1 #2 (Red Hat Linux)) id 1rpDPQ-00000002CGX-317V; Tue, 26 Mar 2024 20:28:36 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org Subject: [PATCH 4/8] mm: Convert migrate_vma_collect_pmd to use a folio Date: Tue, 26 Mar 2024 20:28:24 +0000 Message-ID: <20240326202833.523759-5-willy@infradead.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240326202833.523759-1-willy@infradead.org> References: <20240326202833.523759-1-willy@infradead.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: A74F14000C X-Stat-Signature: kh9b5pbcq7buj55asw555e7wmi6bjm8f X-HE-Tag: 1711484919-874469 X-HE-Meta: U2FsdGVkX1/gIFI+H0zp2n/DV8i8FIKVT6yJD86y5VsL5A6DApX0lWrKSYAaxKljtJP8EBSJEMjqCWBhWaKwCI90pWQFdYcihj4DgQGhImZEAZSOYPfLXotQg/DIcMi90bMQcV3oV+1oBa8+9d0EYUoz103bJNyAnn9qTlNv/mHKfd/kB29jLguxdgb5ZDKVWDZomDW+NLUcO8GsbGIbe6RtnDDaipO7lySy75SBVrL378vINTVCyXMbk+dBdLCUX8blsEFCVwd6dqZP9MgcZyT7rUe/p7QmdHY4VCBBoM8mU4+OA5xtiv2AFMtYIgfplsxaKCC4hhve5InnGDXzSrSx8qWvr8H0IXZQsYvOre8n1VspigXtSM23XRAr1LRrf5Jvfy2S8UnuUpOAu/PYH0EkcqNVJcElhEw85Qw+hj2yRowKl46bBU0SKbg/svIx9NFz9WXqtq7YLfhGz7vzySxiRdzB2pm2jXea+HmOf8ChwPPmMywprqF4XzAzwWA6cWEluisJpwweciC54Hlvgh1boprHiESe7Ar2Gjf4F9EuAG0v0HlT22r7EV8TroV+Q+qpyO9qxVkIjPXBlOvlPqs9YFpaKdL97lzRgX50OVWaujOBiaAdxkz0vsrwWh6L6R0F2Ef0iDqc8D5TipUgFCisSxsDRhODM15ELDsX+zCpUKkrNtSAOIREBB2CCNhGW5f83lp5zxvgIJyspowxcDZ6AF/nr/QFoee2v2H4thTurdWoTymtRdFDkY/KAl6F5usmbSTIIeO0rhEaU2NA6UmqajUgGP/TifCFzRYkZQBZokp7JUw/KqJpfTkfrijQTSZhaZtAmorycUbPFjocPDnnq7sFQ/+qhUzJpefwboCK487B6z3Ocn+XU5fx0dLwrOjYyFmtfpxbNpHKljUDzek2nc1MagI8 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Convert the pmd directly to a folio and use it. Turns four calls to compound_head() into one. Signed-off-by: Matthew Wilcox (Oracle) --- mm/migrate_device.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/mm/migrate_device.c b/mm/migrate_device.c index b6c27c76e1a0..d40b46ae9d65 100644 --- a/mm/migrate_device.c +++ b/mm/migrate_device.c @@ -71,7 +71,7 @@ static int migrate_vma_collect_pmd(pmd_t *pmdp, return migrate_vma_collect_hole(start, end, -1, walk); if (pmd_trans_huge(*pmdp)) { - struct page *page; + struct folio *folio; ptl = pmd_lock(mm, pmdp); if (unlikely(!pmd_trans_huge(*pmdp))) { @@ -79,21 +79,21 @@ static int migrate_vma_collect_pmd(pmd_t *pmdp, goto again; } - page = pmd_page(*pmdp); - if (is_huge_zero_page(page)) { + folio = pmd_folio(*pmdp); + if (is_huge_zero_folio(folio)) { spin_unlock(ptl); split_huge_pmd(vma, pmdp, addr); } else { int ret; - get_page(page); + folio_get(folio); spin_unlock(ptl); - if (unlikely(!trylock_page(page))) + if (unlikely(!folio_trylock(folio))) return migrate_vma_collect_skip(start, end, walk); - ret = split_huge_page(page); - unlock_page(page); - put_page(page); + ret = split_folio(folio); + folio_unlock(folio); + folio_put(folio); if (ret) return migrate_vma_collect_skip(start, end, walk); From patchwork Tue Mar 26 20:28:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13605068 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 873A5C54E67 for ; Tue, 26 Mar 2024 20:28:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6849E6B0098; Tue, 26 Mar 2024 16:28:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3E4716B009B; Tue, 26 Mar 2024 16:28:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 19C546B0099; Tue, 26 Mar 2024 16:28:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id EDB0F6B0093 for ; Tue, 26 Mar 2024 16:28:39 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 93123160339 for ; Tue, 26 Mar 2024 20:28:39 +0000 (UTC) X-FDA: 81940328358.22.0A36C74 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf13.hostedemail.com (Postfix) with ESMTP id 2575620015 for ; Tue, 26 Mar 2024 20:28:37 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=r69DxK8f; dmarc=none; spf=none (imf13.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711484918; a=rsa-sha256; cv=none; b=54xxrx2BVUTWjcgYQ1XwJMMrRstSu37HcM51qxfSiuAh1dVkUCey/jxmOL/Sy/FuTSSIbM ZCh6bvSv4LGKO3zRrIHq107++Z9XNeRMZp25hiSYTebZcMc20WOsOzXpbe25rx80gE87eR M3Bq7ZvUc4aT8fUh8Y7iiXRLNtobPeY= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=r69DxK8f; dmarc=none; spf=none (imf13.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711484918; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=YM0bEoV78naax0Yzl0xLH8KstlzK88rkA2GPtu5jU1g=; b=GE3qeY8/AX4XC2sIJGhntu1CheNsxYUqegc9bhNyWiGGAs4mv5AFA+IYxDDsx28y3wmyK8 Miq2aN+whQkXsTr3X0LIoYu+RDq+NQ8K5p7Efz40VEjdH3X7liiedcNeVD4Sva1q4YOrhb ii6OjGi6hSmaSkfeBExOON7U5MDJ/38= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=YM0bEoV78naax0Yzl0xLH8KstlzK88rkA2GPtu5jU1g=; b=r69DxK8foYzxXQxJL5nJIUr3hM xXOJcylzV7RBHeNvVxI4/68n+Flcs9GtBAYP78lxMa2Q7HiU1NmZiG0qc3p2qlpwr9gplxIEwFs7y b0kJUryfKx7NGjLklYOhwTCnbFH0ZzwVhp+E4IH53fqXcaA9h8/eS/94wD6mSietY5qw5mu8Ea1Rm GFuyK/UlVHDbtzYbvRCfNj3pMwnJTabXpC1uEtcfxKzPv3zwW5sp80nP09DhrkQmf22JPc3zuYS7N RovBUNHnir1FUj64AjGDp1UuSE9j8rnoUMHAKp+wZoTregFk6mD7Tr/7IxwIWS8fgJ9Z/VoQilWbr HOW/c6lg==; Received: from willy by casper.infradead.org with local (Exim 4.97.1 #2 (Red Hat Linux)) id 1rpDPQ-00000002CGd-3TpQ; Tue, 26 Mar 2024 20:28:36 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org Subject: [PATCH 5/8] mm: Convert huge_zero_page to huge_zero_folio Date: Tue, 26 Mar 2024 20:28:25 +0000 Message-ID: <20240326202833.523759-6-willy@infradead.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240326202833.523759-1-willy@infradead.org> References: <20240326202833.523759-1-willy@infradead.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 2575620015 X-Stat-Signature: qeafea8mkb8kqaqd4k35wm9boic4x1hd X-HE-Tag: 1711484917-960018 X-HE-Meta: U2FsdGVkX1/C/bAqbYtBqQ1qpuWKZWz7Fal/zpIHm9p5JvpcQMXi4h4GSXa4/oUSOXFCHx7FpzzQ5vCVEBUwGgOapxD7RgigdDiKGzrwsncF10j8H97WNy7eAnWUvp1PwUsRLBhk1ueNrRxkcThCDbn9hDWPJhW16qwx9AIKzJgCPovyjZsnZ+NfV/yDfP8OBRfsGHWTT+zPoqxgBnjXxw+7n0NMMGglGEHERtylzPMJY7CjIplF7Iz0SYBeqViF/QlauPbAnduEE6f7f4oWBInVAJbfYtyioe9foiqzgHrDRI/WdGU2mNYhLiYdMFsg6d5Snlc6HoTfVaAAFwYvRHVkpkm5r8oKMnnOMwj7SUHgjxCrWl4rPjFBA0Dsyy8utj+UUXGFruPKVb0uvhTxMBuX9Puk2RcPWnj4Pylzb71RbQ6Z/k6He78xJ1sMFITPM+gTecpo0QcoZcZrHUOrW6NhHZDmgtHawLeYDN61uJ/ZtYVTDxGdbGEJS5W1FX0McWY4smvkGloT6dcpVUUrY8CJkVWui92usg+7DRLdBlHsXkwNBFtnaBaH9rtL1RRzRlY5O/9IpJl+ju9TdAlGoUcFnOQeKLSq5a/kAppS1eZrWBcAV+1Uy0mfdmjUB4tpU9p2KlcQ7ajaM9HS8Q17IltceBvMfLm5x2H1jEJ80GTSuuKDFiHSU6n/vF0lmeNAeoymXTwj19xQq8yhWhHKJ61rx7FsINIj7mD0BOx6k3BDazTs/8ip/2FPL0r3x633qRYy3WgOORqKHNNmQxsNncHCYGfXq7sa/d+NL06pmYdnoLhsO6MEhIsXrXH8jJSP6pJ5FBl3dp5uKqw/HnuMe7GcYUit24k03tkDxZnOJciAtReEQ7Nlq3SPo4jnizSA0mMBufc262EnI3wYkJ4OjxHF1SBX7GNZcnSqm8PN4jA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: With all callers of is_huge_zero_page() converted, we can now switch the huge_zero_page itself from being a compound page to a folio. Signed-off-by: Matthew Wilcox (Oracle) --- include/linux/huge_mm.h | 21 ++++++++------------- mm/huge_memory.c | 28 ++++++++++++++-------------- 2 files changed, 22 insertions(+), 27 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index 600c6008262b..7ba59ba36354 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -348,17 +348,12 @@ struct page *follow_devmap_pud(struct vm_area_struct *vma, unsigned long addr, vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf); -extern struct page *huge_zero_page; +extern struct folio *huge_zero_folio; extern unsigned long huge_zero_pfn; -static inline bool is_huge_zero_page(const struct page *page) -{ - return READ_ONCE(huge_zero_page) == page; -} - static inline bool is_huge_zero_folio(const struct folio *folio) { - return READ_ONCE(huge_zero_page) == &folio->page; + return READ_ONCE(huge_zero_folio) == folio; } static inline bool is_huge_zero_pmd(pmd_t pmd) @@ -371,9 +366,14 @@ static inline bool is_huge_zero_pud(pud_t pud) return false; } -struct page *mm_get_huge_zero_page(struct mm_struct *mm); +struct folio *mm_get_huge_zero_folio(struct mm_struct *mm); void mm_put_huge_zero_page(struct mm_struct *mm); +static inline struct page *mm_get_huge_zero_page(struct mm_struct *mm) +{ + return &mm_get_huge_zero_folio(mm)->page; +} + #define mk_huge_pmd(page, prot) pmd_mkhuge(mk_pmd(page, prot)) static inline bool thp_migration_supported(void) @@ -485,11 +485,6 @@ static inline vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf) return 0; } -static inline bool is_huge_zero_page(const struct page *page) -{ - return false; -} - static inline bool is_huge_zero_folio(const struct folio *folio) { return false; diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 8ee09bfdfdb7..91eb5de3c728 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -74,7 +74,7 @@ static unsigned long deferred_split_scan(struct shrinker *shrink, struct shrink_control *sc); static atomic_t huge_zero_refcount; -struct page *huge_zero_page __read_mostly; +struct folio *huge_zero_folio __read_mostly; unsigned long huge_zero_pfn __read_mostly = ~0UL; unsigned long huge_anon_orders_always __read_mostly; unsigned long huge_anon_orders_madvise __read_mostly; @@ -192,24 +192,24 @@ unsigned long __thp_vma_allowable_orders(struct vm_area_struct *vma, static bool get_huge_zero_page(void) { - struct page *zero_page; + struct folio *zero_folio; retry: if (likely(atomic_inc_not_zero(&huge_zero_refcount))) return true; - zero_page = alloc_pages((GFP_TRANSHUGE | __GFP_ZERO) & ~__GFP_MOVABLE, + zero_folio = folio_alloc((GFP_TRANSHUGE | __GFP_ZERO) & ~__GFP_MOVABLE, HPAGE_PMD_ORDER); - if (!zero_page) { + if (!zero_folio) { count_vm_event(THP_ZERO_PAGE_ALLOC_FAILED); return false; } preempt_disable(); - if (cmpxchg(&huge_zero_page, NULL, zero_page)) { + if (cmpxchg(&huge_zero_folio, NULL, zero_folio)) { preempt_enable(); - __free_pages(zero_page, compound_order(zero_page)); + folio_put(zero_folio); goto retry; } - WRITE_ONCE(huge_zero_pfn, page_to_pfn(zero_page)); + WRITE_ONCE(huge_zero_pfn, folio_pfn(zero_folio)); /* We take additional reference here. It will be put back by shrinker */ atomic_set(&huge_zero_refcount, 2); @@ -227,10 +227,10 @@ static void put_huge_zero_page(void) BUG_ON(atomic_dec_and_test(&huge_zero_refcount)); } -struct page *mm_get_huge_zero_page(struct mm_struct *mm) +struct folio *mm_get_huge_zero_folio(struct mm_struct *mm) { if (test_bit(MMF_HUGE_ZERO_PAGE, &mm->flags)) - return READ_ONCE(huge_zero_page); + return READ_ONCE(huge_zero_folio); if (!get_huge_zero_page()) return NULL; @@ -238,7 +238,7 @@ struct page *mm_get_huge_zero_page(struct mm_struct *mm) if (test_and_set_bit(MMF_HUGE_ZERO_PAGE, &mm->flags)) put_huge_zero_page(); - return READ_ONCE(huge_zero_page); + return READ_ONCE(huge_zero_folio); } void mm_put_huge_zero_page(struct mm_struct *mm) @@ -258,10 +258,10 @@ static unsigned long shrink_huge_zero_page_scan(struct shrinker *shrink, struct shrink_control *sc) { if (atomic_cmpxchg(&huge_zero_refcount, 1, 0) == 1) { - struct page *zero_page = xchg(&huge_zero_page, NULL); - BUG_ON(zero_page == NULL); + struct folio *zero_folio = xchg(&huge_zero_folio, NULL); + BUG_ON(zero_folio == NULL); WRITE_ONCE(huge_zero_pfn, ~0UL); - __free_pages(zero_page, compound_order(zero_page)); + folio_put(zero_folio); return HPAGE_PMD_NR; } @@ -1340,7 +1340,7 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm, * since we already have a zero page to copy. It just takes a * reference. */ - mm_get_huge_zero_page(dst_mm); + mm_get_huge_zero_folio(dst_mm); goto out_zero_page; } From patchwork Tue Mar 26 20:28:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13605071 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B5B07CD1280 for ; Tue, 26 Mar 2024 20:28:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F0EBB6B0093; Tue, 26 Mar 2024 16:28:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BD2AA6B009B; Tue, 26 Mar 2024 16:28:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 716056B0099; Tue, 26 Mar 2024 16:28:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 2F4F36B009A for ; Tue, 26 Mar 2024 16:28:40 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 07C26160931 for ; Tue, 26 Mar 2024 20:28:40 +0000 (UTC) X-FDA: 81940328400.02.C3A6099 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf24.hostedemail.com (Postfix) with ESMTP id 613EC180016 for ; Tue, 26 Mar 2024 20:28:38 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=tPHk1xLT; dmarc=none; spf=none (imf24.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711484918; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=VzuGfQEJVf39ZiDAt09dMPTvgwh2rkzi1XBtXHq60Dk=; b=Rqf+EiMTZJLwmXN7E78TDI1HMKgG3UBMDym/8E6l6MLGBKhEabLcwhXAwDA2zRhbs7rrXL vaTyjKlAfmqsXXz8HvVgKeJPYYeScV+rSAZ5wU2tH6/0mcq2Sq3BlqZshX5025smvXzlcI jtWPAgmG5Y7bW52t9NLGn8rxGmlnFN4= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=tPHk1xLT; dmarc=none; spf=none (imf24.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711484918; a=rsa-sha256; cv=none; b=Y1B+nOKunoVDHodN+tPDC2vHs/tYSzI5dqA2ZqG2a9X/1WsRObzQgwykRLbqYMM1tB+5+T yRY+rFyezj+Q223zyx/Gpfy0dV1yyDu0YFUklRN4CA1eTxVnDHOk7ewArDhoT2o7eF17Lw +gRUPKweJ3zGhLRp0FAZqovWK5D4Mqo= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=VzuGfQEJVf39ZiDAt09dMPTvgwh2rkzi1XBtXHq60Dk=; b=tPHk1xLTANF41P+YvclPaThq2C IybrgYAI67DiC1EXO42hWERlIGbBnNSeJ1loDjRAXs6qXs9HtGOSidn5ppwV7c1H16E0L3Fh/Ygwo bxVPD5MRp0DiA3mjzY6fgE8GCloBf51YVkI7ivA76zj1IVvOjxcGOd09E6a0zl0gW7rL8TQgC0xUV GOOAdXseMBU7tdB54A1V2+G0+OMWL+BU3Jau8iBKy7pT89ilQeeem/ozcYrLk7imdrMsVg7NVL6SR dsLtG6O1NPWHzLV/8XnGwNJhlT6C1vM4zF/Tu7XSHZr7zh1R3G4cLzBGEDdMTtJt7sHzEf7aX58VA Hk7k7FKw==; Received: from willy by casper.infradead.org with local (Exim 4.97.1 #2 (Red Hat Linux)) id 1rpDPQ-00000002CGj-42Ng; Tue, 26 Mar 2024 20:28:36 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org Subject: [PATCH 6/8] mm: Convert do_huge_pmd_anonymous_page to huge_zero_folio Date: Tue, 26 Mar 2024 20:28:26 +0000 Message-ID: <20240326202833.523759-7-willy@infradead.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240326202833.523759-1-willy@infradead.org> References: <20240326202833.523759-1-willy@infradead.org> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: hudwqxggeecjqnfx7xmyoiyhkeuuowb1 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 613EC180016 X-HE-Tag: 1711484918-201223 X-HE-Meta: U2FsdGVkX19sheus0VUWuOp+OGJonVUONr4XS/4BdAKUkn+mSLPcxWdo06f/g4uCSro4QF87NEeittwwn3dVRxYusWntr+eWY3nqOXFP2PW1+woPM0U4UPIXced9r2fXeZjDIPGsPLenIZK8ajAhfFo3awRrzdLn+l+z92TTF44TLYOdKDm3Klk2hRbp6OKHLpf1oKKPOwQJSyejpSADmI4/Quh7wWaUeg0MfMylZnGdtDX/go/Vy13mh70iEqfzdve975ZoTSvW9Eya5K4i1Ndjk9R/UTpk5HSb/2UUf5H+eOOvpIYHd+occF+Xm32LjngdcRiiwNfpAmAcyVVPC7leUbAx3Nn5nBSVqiJlEf2KRObmmb/NQUJZPn+x9PjfLphSfNjL0JRicKhKvuZhaQUVfZeUxiWQJFBnR9YeTP0N7x3qrg/Ux5UMCx+EbS2C46KLlFwN7iP5nnhKsoDlx2MvUo/v6Hn9MelBcDKn2N6VC5LPic7KPai3jr+qI7joJ6h3HydodyXtkWoEjM7OzR3s15h62QWjG49yrgpWCRCZgHJLKXMwPbIEeDMUQ1QcqNks8P3+Q9kdXmqbugOYsdQDIrq5kkN0TcRnuTuGxUW61WswF9gPvP917ymxMqgQ0gyDuIfxnRRqcjTD9AXL6yz/x1FRjFvPj1iGVXOYOjWFKvneMEC0LwNiEMxMAZa3jKMmXtoqizzs1D9EWoG9egHlu3QJtihwqC5Wj09P3a0cCCfe9/hvkicT4m8zcR4e4/8tsH+2NCbU9k2ZhpGBqM2m4Ru4pkptjEFAqeLJ7ZpYXsCbz2Pused5PIGW2rPH301700mOeLcum6QVW+lr+2p7K6dCem1s0CRgWheIYv3FnZGPF2BGWU8xDvrRcd3cLRBLqVo6+RNZTXhpdUHIYwzZ5WpUscxZQ8GzNGKl0jGUJQ/cvlUb1T2yX+hiWpHGxTcEHPm2V6eYD8O9RKy DpYt4Hy2 kqwwAenl8oUiW+YgSh3wwNj1olMrIm+nTYrdgdD7Gt+8idADZYXIs0ZUabVY6e3Ewd/9IgDdUKzsQu4N/SeznceS9YKELAomYUC6tSnJakI9qlA4RQIYfvumt61JmpCACUHs4kUgspDlGaJG4xDjrN8rMB6kJuw9KJ6fXJurDSz51+QI/P1XiW4KH+uIU7wdeEEhUelJcrt9zlfkw/iU4vTK8v590u/GDuCjz X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Use folios more widely. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: David Hildenbrand --- mm/huge_memory.c | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 91eb5de3c728..ff282317e71f 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -971,14 +971,14 @@ gfp_t vma_thp_gfp_mask(struct vm_area_struct *vma) } /* Caller must hold page table lock. */ -static void set_huge_zero_page(pgtable_t pgtable, struct mm_struct *mm, +static void set_huge_zero_folio(pgtable_t pgtable, struct mm_struct *mm, struct vm_area_struct *vma, unsigned long haddr, pmd_t *pmd, - struct page *zero_page) + struct folio *zero_folio) { pmd_t entry; if (!pmd_none(*pmd)) return; - entry = mk_pmd(zero_page, vma->vm_page_prot); + entry = mk_pmd(&zero_folio->page, vma->vm_page_prot); entry = pmd_mkhuge(entry); pgtable_trans_huge_deposit(mm, pmd, pgtable); set_pmd_at(mm, haddr, pmd, entry); @@ -1002,13 +1002,14 @@ vm_fault_t do_huge_pmd_anonymous_page(struct vm_fault *vmf) !mm_forbids_zeropage(vma->vm_mm) && transparent_hugepage_use_zero_page()) { pgtable_t pgtable; - struct page *zero_page; + struct folio *zero_folio; vm_fault_t ret; + pgtable = pte_alloc_one(vma->vm_mm); if (unlikely(!pgtable)) return VM_FAULT_OOM; - zero_page = mm_get_huge_zero_page(vma->vm_mm); - if (unlikely(!zero_page)) { + zero_folio = mm_get_huge_zero_folio(vma->vm_mm); + if (unlikely(!zero_folio)) { pte_free(vma->vm_mm, pgtable); count_vm_event(THP_FAULT_FALLBACK); return VM_FAULT_FALLBACK; @@ -1026,8 +1027,8 @@ vm_fault_t do_huge_pmd_anonymous_page(struct vm_fault *vmf) ret = handle_userfault(vmf, VM_UFFD_MISSING); VM_BUG_ON(ret & VM_FAULT_FALLBACK); } else { - set_huge_zero_page(pgtable, vma->vm_mm, vma, - haddr, vmf->pmd, zero_page); + set_huge_zero_folio(pgtable, vma->vm_mm, vma, + haddr, vmf->pmd, zero_folio); update_mmu_cache_pmd(vma, vmf->address, vmf->pmd); spin_unlock(vmf->ptl); } @@ -1336,9 +1337,9 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm, */ if (is_huge_zero_pmd(pmd)) { /* - * get_huge_zero_page() will never allocate a new page here, - * since we already have a zero page to copy. It just takes a - * reference. + * mm_get_huge_zero_folio() will never allocate a new + * folio here, since we already have a zero page to + * copy. It just takes a reference. */ mm_get_huge_zero_folio(dst_mm); goto out_zero_page; From patchwork Tue Mar 26 20:28:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13605072 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0BB11CD1283 for ; Tue, 26 Mar 2024 20:28:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2269A6B0095; Tue, 26 Mar 2024 16:28:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E69506B0096; Tue, 26 Mar 2024 16:28:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9884B6B0095; Tue, 26 Mar 2024 16:28:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 49FF06B0096 for ; Tue, 26 Mar 2024 16:28:40 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 23A941A036E for ; Tue, 26 Mar 2024 20:28:40 +0000 (UTC) X-FDA: 81940328400.20.51A9321 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf17.hostedemail.com (Postfix) with ESMTP id 7463D4000B for ; Tue, 26 Mar 2024 20:28:38 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=TfaZALBy; spf=none (imf17.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711484918; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=sq9t5ZRvbstKnwCVhJax9lK120QpoDyuWhQ2hhEZpOI=; b=y6ysO5jKp6PemXb2z6cYez5X6GynvpuCbi8ij9nVp2RRrJ3M8BS6MLCnr/1W1xh2gwFTaV lkfkHZon6cePrqX6kfYgAgaI9btQN15q1hUzq7PHEKeRpR4cqNsOyBkXJPB8EviurhFV6X a689uDJhFQ96c9CvtXsBzFQxWHVTToU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711484918; a=rsa-sha256; cv=none; b=oZzlkmXh57IppTY6lzTtuLIrsGYiqAWgeHyhVljexytgtDjKoHayr3DI9gjU2vNWkoSR/K NKJJ/q3zbMURjSsigArukGipLYHJuUUJak7NcAj5mPn1949h+9cHk9UBzPyzdEvIfNcs7O GrjmYLCRA/rZdMRdJLhJnuZsW4plWsQ= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=TfaZALBy; spf=none (imf17.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=sq9t5ZRvbstKnwCVhJax9lK120QpoDyuWhQ2hhEZpOI=; b=TfaZALByTDUioW1NWYEi7Z5MnM WtGtgg3wa3J2iKinVs4P/po9H4jlJeZEVee8tBzsARkr+ZaPbu7DUYFYFjCuZ+NEUCLSmO+9XL7ez /drg9Y5wGoeldAAc1mHNz0j4dCARsuYO7QLHNlMm+ZmyR1tK9A8OXsbZVhr17QIqeamawjf8yyQDV KUIHZAA8rq561Ce3FTKga46lDsOoPLdrmO17AvE2Q4Qz9KSGEGN496oKh+svatmuANIvxoE06yiZL d1DymReEBQexBgB/PcBo5y86qCfcO1mw2HLoEQscbPkD7fWSepZLsLZdxFiqOcCRqEc4kTCQDlvT5 Ze6/hPDQ==; Received: from willy by casper.infradead.org with local (Exim 4.97.1 #2 (Red Hat Linux)) id 1rpDPR-00000002CGr-0S40; Tue, 26 Mar 2024 20:28:37 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org Subject: [PATCH 7/8] dax: Use huge_zero_folio Date: Tue, 26 Mar 2024 20:28:27 +0000 Message-ID: <20240326202833.523759-8-willy@infradead.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240326202833.523759-1-willy@infradead.org> References: <20240326202833.523759-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 7463D4000B X-Rspam-User: X-Stat-Signature: 6bchqdq4ecyia9huz1s8f49yn8ef9gec X-Rspamd-Server: rspam03 X-HE-Tag: 1711484918-602895 X-HE-Meta: U2FsdGVkX1+smH49bHtwkvWx0MJPFG4OCSvY3KsuYs3G3hn2n9kJtulvQQXzpPXmYfKrZjtlVWtVfXgXFkFaqY+NqNEYyLgBQGl8AOrKpSgpJ7roiejFHEroZjowtiTXXwsauhzl+ekz1iwY7JcTa695O3/6CzVjQpxm01JS3qbFC40udrlL389P1tRQjHLTXvwjPFubNeygZTdCRP6+XVrJL3KpRp3STiXxRKeFFf+dtzkfy1ITC8W6Y80ni5XWGKibNkxKQzcPHyvohfalXR3d7Y64Fje3NIygY4NbR8PRCopkDU3Naa0h6mVns+FntD3r6P89+Y7F7Qk984WH3ETHga+eKg2vgJ/DYPtcJL/A0lQG5EM8B57yB6cG72ZTMptRIciOtP6jIzs9sqheunOXN0VFpbioe4EUGTqsa71U1RT9VRtWEBuVETttkFeymJrBPvG9oGClno898YXVwXA4shgkJUVhF/FAFmg7haEUVnfbY201E/IWT9V+cDy0nsogCkGHhynrL/JCfHiBLWRCxPIo186789GM+7r3v+bqq+rq8TpkYTqXv1M/9xeJ86zE/BKetEzQZwp/BMr6nLpGXaNZyEnlF5G5a+D/YngOIx8ZBEl5fSqeIG9zWugxL8pn4Jszs6cBdh+ULXfWKIG1gZxmdx85H4uzXBXCcwl0ry6vziFjTVEEAsgJfU0GoD0HWCTkHRIuOXhdpwyoJPUOfsZioYvu0TzGLysWr356PBYepr4No2bs4gAa75rmqmjJNmP4WjcnwRCLpnkC+e8djC+CqZwe6czC0KK9CK6nbHUK8kMtj+iDMwPjG/5SYX5BQEUBBl4sLl8Gpy//0Yj5l0N3JFu+69/kWatoV5TmH3QSuLhOk8jRTkBiVeWUx/9o/TX1I6YJcPbvPDdXjkkUEe5s9Uss8RBQrUkhugmVhqGUTt9oQvI8ejYsguJAGvY7xeY7xnEywkFoS1f 2ZjFyZLg RwgBe9mZ/Kvrc4FQmDYI2EzYdUbYV4GKPKYsQVbUpK5GCmgcXulcEPRm+Ir1kiczOG8QHRupvULEoVNJkXOMnq23iaRQIB3XOGNN84Ba0XiEkn1gkS6t+7rW7qtpIp3+E4HYgtvjVfYcyHdXoaRSyY6u9ZMEM/voJ8EcbFw2PZcTWtGQ94Vga02+0qj/kGuA9NyvgZKJoV7rZ9ABVtZjwigI7LfgLHeyzV+LR X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Convert from huge_zero_page to huge_zero_folio. Signed-off-by: Matthew Wilcox (Oracle) --- fs/dax.c | 14 +++++++------- include/trace/events/fs_dax.h | 16 ++++++++-------- 2 files changed, 15 insertions(+), 15 deletions(-) diff --git a/fs/dax.c b/fs/dax.c index 423fc1607dfa..becb4a6920c6 100644 --- a/fs/dax.c +++ b/fs/dax.c @@ -1207,17 +1207,17 @@ static vm_fault_t dax_pmd_load_hole(struct xa_state *xas, struct vm_fault *vmf, struct vm_area_struct *vma = vmf->vma; struct inode *inode = mapping->host; pgtable_t pgtable = NULL; - struct page *zero_page; + struct folio *zero_folio; spinlock_t *ptl; pmd_t pmd_entry; pfn_t pfn; - zero_page = mm_get_huge_zero_page(vmf->vma->vm_mm); + zero_folio = mm_get_huge_zero_folio(vmf->vma->vm_mm); - if (unlikely(!zero_page)) + if (unlikely(!zero_folio)) goto fallback; - pfn = page_to_pfn_t(zero_page); + pfn = page_to_pfn_t(&zero_folio->page); *entry = dax_insert_entry(xas, vmf, iter, *entry, pfn, DAX_PMD | DAX_ZERO_PAGE); @@ -1237,17 +1237,17 @@ static vm_fault_t dax_pmd_load_hole(struct xa_state *xas, struct vm_fault *vmf, pgtable_trans_huge_deposit(vma->vm_mm, vmf->pmd, pgtable); mm_inc_nr_ptes(vma->vm_mm); } - pmd_entry = mk_pmd(zero_page, vmf->vma->vm_page_prot); + pmd_entry = mk_pmd(&zero_folio->page, vmf->vma->vm_page_prot); pmd_entry = pmd_mkhuge(pmd_entry); set_pmd_at(vmf->vma->vm_mm, pmd_addr, vmf->pmd, pmd_entry); spin_unlock(ptl); - trace_dax_pmd_load_hole(inode, vmf, zero_page, *entry); + trace_dax_pmd_load_hole(inode, vmf, zero_folio, *entry); return VM_FAULT_NOPAGE; fallback: if (pgtable) pte_free(vma->vm_mm, pgtable); - trace_dax_pmd_load_hole_fallback(inode, vmf, zero_page, *entry); + trace_dax_pmd_load_hole_fallback(inode, vmf, zero_folio, *entry); return VM_FAULT_FALLBACK; } #else diff --git a/include/trace/events/fs_dax.h b/include/trace/events/fs_dax.h index 97b09fcf7e52..86fe6aecff1e 100644 --- a/include/trace/events/fs_dax.h +++ b/include/trace/events/fs_dax.h @@ -62,14 +62,14 @@ DEFINE_PMD_FAULT_EVENT(dax_pmd_fault_done); DECLARE_EVENT_CLASS(dax_pmd_load_hole_class, TP_PROTO(struct inode *inode, struct vm_fault *vmf, - struct page *zero_page, + struct folio *zero_folio, void *radix_entry), - TP_ARGS(inode, vmf, zero_page, radix_entry), + TP_ARGS(inode, vmf, zero_folio, radix_entry), TP_STRUCT__entry( __field(unsigned long, ino) __field(unsigned long, vm_flags) __field(unsigned long, address) - __field(struct page *, zero_page) + __field(struct folio *, zero_folio) __field(void *, radix_entry) __field(dev_t, dev) ), @@ -78,17 +78,17 @@ DECLARE_EVENT_CLASS(dax_pmd_load_hole_class, __entry->ino = inode->i_ino; __entry->vm_flags = vmf->vma->vm_flags; __entry->address = vmf->address; - __entry->zero_page = zero_page; + __entry->zero_folio = zero_folio; __entry->radix_entry = radix_entry; ), - TP_printk("dev %d:%d ino %#lx %s address %#lx zero_page %p " + TP_printk("dev %d:%d ino %#lx %s address %#lx zero_folio %p " "radix_entry %#lx", MAJOR(__entry->dev), MINOR(__entry->dev), __entry->ino, __entry->vm_flags & VM_SHARED ? "shared" : "private", __entry->address, - __entry->zero_page, + __entry->zero_folio, (unsigned long)__entry->radix_entry ) ) @@ -96,8 +96,8 @@ DECLARE_EVENT_CLASS(dax_pmd_load_hole_class, #define DEFINE_PMD_LOAD_HOLE_EVENT(name) \ DEFINE_EVENT(dax_pmd_load_hole_class, name, \ TP_PROTO(struct inode *inode, struct vm_fault *vmf, \ - struct page *zero_page, void *radix_entry), \ - TP_ARGS(inode, vmf, zero_page, radix_entry)) + struct folio *zero_folio, void *radix_entry), \ + TP_ARGS(inode, vmf, zero_folio, radix_entry)) DEFINE_PMD_LOAD_HOLE_EVENT(dax_pmd_load_hole); DEFINE_PMD_LOAD_HOLE_EVENT(dax_pmd_load_hole_fallback); From patchwork Tue Mar 26 20:28:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13605074 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9312ECD1283 for ; Tue, 26 Mar 2024 20:28:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CAF3B6B0096; Tue, 26 Mar 2024 16:28:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A4A166B009F; Tue, 26 Mar 2024 16:28:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 84F126B009D; Tue, 26 Mar 2024 16:28:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 67D286B009B for ; Tue, 26 Mar 2024 16:28:41 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 43508C032F for ; Tue, 26 Mar 2024 20:28:41 +0000 (UTC) X-FDA: 81940328442.12.CE665DB Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf23.hostedemail.com (Postfix) with ESMTP id A3289140012 for ; Tue, 26 Mar 2024 20:28:39 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=VE5dn9fo; dmarc=none; spf=none (imf23.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711484919; a=rsa-sha256; cv=none; b=UbiqWU4/bghFTjfqJPBtymdW6Ygdbib7FaCutEq32zNTT3ALUWhAiEiU+afPHVyVdCWEMK yJzR8AF3CC8sLKzXXN0WlKusVfjLoHmoW5DYj8p5kAOQk9zSmJtj7cyYdepcXSZEk7QK95 GthehU/W+9cHLDrW+gE9cnM0BNmba6Q= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=VE5dn9fo; dmarc=none; spf=none (imf23.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711484919; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=M3F/xDzH/g+Ev/ro5eMn/tEXDoRXLGDPOJTAfCblJyY=; b=f6nvU7h/gGhAaLH+3QF/+bd3RQW/8nZIbi0LK644CPgPeBJeJcZPbB6EQA+3CN6YDjNqMK dcXODXZ6yz0XbjSjDVpRDvxiU7asie2HGqTZb87IxCILe/ADckr2CsglUP5wIgn6TDC20c gb+CbvOc/+U/681BGjUGXlmAJ0hKe5o= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=M3F/xDzH/g+Ev/ro5eMn/tEXDoRXLGDPOJTAfCblJyY=; b=VE5dn9fos0QpPqCmTp1yoHpjGA 7Ci25tpL4uHtvTdEnhW6XmbGMd/EIbycJP7rJql/xWUugyq1i+AzqlR3YXK1C17C1o/Yen/CcZjEr BzPU3VkPwbLfOHVuhbez4BMMzBCMF5bHUYLwZWRBs3cdunpgtCEz2PQqdHdawiDD2kGkUuMAsr2ln msdHKxyMRxsAZNtITDdigTy6e29Y3b2LbV1vNTKxKpRfBh767JjZJ/Sgkm/2JlM9pu/UrPv4ZSE+h PVkTyH9d+PS7xmAXKaXsXjOK5XCUBpRi4yKN3KBN06LDbK9qevj7ln9VhCob9/Dkn9RdjDhdkmM8u pMtcDb+A==; Received: from willy by casper.infradead.org with local (Exim 4.97.1 #2 (Red Hat Linux)) id 1rpDPR-00000002CGx-11N3; Tue, 26 Mar 2024 20:28:37 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org Subject: [PATCH 8/8] mm: Rename mm_put_huge_zero_page to mm_put_huge_zero_folio Date: Tue, 26 Mar 2024 20:28:28 +0000 Message-ID: <20240326202833.523759-9-willy@infradead.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240326202833.523759-1-willy@infradead.org> References: <20240326202833.523759-1-willy@infradead.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: A3289140012 X-Stat-Signature: 85or6sggpek75ra4b11qr1qixbzy83cg X-HE-Tag: 1711484919-694818 X-HE-Meta: U2FsdGVkX1+4w+HW6w7wOLWEuQlw/mapxHJ7a/yv3mCLVGKqKFlKs2gDHPDpsHPjiNRu9FzjsH85IccS0sZ9uRG3585f18KOLaeqHu6/7s2By0iqnbq95soy/VLrrfzP2sRcW90mfe+kD2HTW7pEd8F72ytR4j4f1utr515q4XUi9oKKq0kj1++nl3Pwb7nYIN3vLRU/T1cOQpwuYOCVHGZRpNmsx7D0vjttJzahla5l60CG5vtrQK0IJ5aNJ3cUu7Mpg9UUmVaSY5uDCIO7YhhIl+FnAvn4gyev6/OSw30j/Kq4qXfKrc9mfaiUhE+hLvsykZ29sxi2Y7bCM6uvNiwYs834hf2D7bTtrx9Cf8Par/2CXDkW3fpI2pRd3kbw88QitbIpv68oWMHjLvW2tyd0MpoDnr0KzCNxPCehWnczvYoIxY/WZdHIvKWm+AVD361yZYgjMEB3Z1TpYv0qk2LR3D3IusCfMACeZS172xqk96DwLNdBObUNbBmCKylpSCU4q2xRdk2/+BRTr6guN5W/DAlYGk6FLWBdTPyjXjm53j/BEZuC2NurHxpE3sevarGfp3fuPY5KMeWasD+UhtwoiQH7ku0Pim0h9aikmAhzYrpvau4TSDaomkCBlH5aDP/zrA8vIZRgnhMDD0NqXuYWqs4jYunR3AfnMt2k2iV0BLcUS8k8hpEl8HbkSG2pYFgCAzeCMB6YY5oXF92G2a4FP4Iv26staQzVFrTYBpBxZAcrARklLtlgZwuNhw7ERvINnVA0te76aF4aA5EmxuMs0c+lQfrPivEfp6m5X3gIU4IgboMm+eCsyJYCpbHQ6y0mReWJK2DMV13f9k9fblYdEC2kU82TPMjUwrdtdx9ucst52jE7qoPqQ7KttUalR5+9OnlWLWL16fTYD/UMO9S/0xxUKXZ+VWDvb5LBmADB459AjkuRyNd7xf8hDLkRm+4gwQI/C6cqYiGEsIu bD2EdcGf ps4b5SARVWqLrc9H6eXktWWCnIBFXscL4U8aIufo/KMNCkbOVr3a5jgMXxzr3FkV5L6n0ayz+duZH0STgpFv1noO7EgckfT/jdPlnIQXtdyehkxJQ2GJOThU/mJ1l+j0vBFfU64v2Ip+EcsIRcr9vQJ/kQvsbY0uy7UfNhmnLB0XSrCxSQNbe/hwP18gCjY8bqdpY7Wc2gN7R4KeLKRzzL9k1yUJAhQ28BLMP X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Also remove mm_get_huge_zero_page() now it has no users. Signed-off-by: Matthew Wilcox (Oracle) --- include/linux/huge_mm.h | 9 ++------- kernel/fork.c | 2 +- mm/huge_memory.c | 2 +- 3 files changed, 4 insertions(+), 9 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index 7ba59ba36354..9d11e886aa9a 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -367,12 +367,7 @@ static inline bool is_huge_zero_pud(pud_t pud) } struct folio *mm_get_huge_zero_folio(struct mm_struct *mm); -void mm_put_huge_zero_page(struct mm_struct *mm); - -static inline struct page *mm_get_huge_zero_page(struct mm_struct *mm) -{ - return &mm_get_huge_zero_folio(mm)->page; -} +void mm_put_huge_zero_folio(struct mm_struct *mm); #define mk_huge_pmd(page, prot) pmd_mkhuge(mk_pmd(page, prot)) @@ -500,7 +495,7 @@ static inline bool is_huge_zero_pud(pud_t pud) return false; } -static inline void mm_put_huge_zero_page(struct mm_struct *mm) +static inline void mm_put_huge_zero_folio(struct mm_struct *mm) { return; } diff --git a/kernel/fork.c b/kernel/fork.c index 39a5046c2f0b..84de5faa8c9a 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -1343,7 +1343,7 @@ static inline void __mmput(struct mm_struct *mm) ksm_exit(mm); khugepaged_exit(mm); /* must run before exit_mmap */ exit_mmap(mm); - mm_put_huge_zero_page(mm); + mm_put_huge_zero_folio(mm); set_mm_exe_file(mm, NULL); if (!list_empty(&mm->mmlist)) { spin_lock(&mmlist_lock); diff --git a/mm/huge_memory.c b/mm/huge_memory.c index ff282317e71f..b49fcb8a16cc 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -241,7 +241,7 @@ struct folio *mm_get_huge_zero_folio(struct mm_struct *mm) return READ_ONCE(huge_zero_folio); } -void mm_put_huge_zero_page(struct mm_struct *mm) +void mm_put_huge_zero_folio(struct mm_struct *mm) { if (test_bit(MMF_HUGE_ZERO_PAGE, &mm->flags)) put_huge_zero_page();