From patchwork Tue Mar 26 20:28:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13605071 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B5B07CD1280 for ; Tue, 26 Mar 2024 20:28:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F0EBB6B0093; Tue, 26 Mar 2024 16:28:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BD2AA6B009B; Tue, 26 Mar 2024 16:28:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 716056B0099; Tue, 26 Mar 2024 16:28:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 2F4F36B009A for ; Tue, 26 Mar 2024 16:28:40 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 07C26160931 for ; Tue, 26 Mar 2024 20:28:40 +0000 (UTC) X-FDA: 81940328400.02.C3A6099 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf24.hostedemail.com (Postfix) with ESMTP id 613EC180016 for ; Tue, 26 Mar 2024 20:28:38 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=tPHk1xLT; dmarc=none; spf=none (imf24.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711484918; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=VzuGfQEJVf39ZiDAt09dMPTvgwh2rkzi1XBtXHq60Dk=; b=Rqf+EiMTZJLwmXN7E78TDI1HMKgG3UBMDym/8E6l6MLGBKhEabLcwhXAwDA2zRhbs7rrXL vaTyjKlAfmqsXXz8HvVgKeJPYYeScV+rSAZ5wU2tH6/0mcq2Sq3BlqZshX5025smvXzlcI jtWPAgmG5Y7bW52t9NLGn8rxGmlnFN4= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=tPHk1xLT; dmarc=none; spf=none (imf24.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711484918; a=rsa-sha256; cv=none; b=Y1B+nOKunoVDHodN+tPDC2vHs/tYSzI5dqA2ZqG2a9X/1WsRObzQgwykRLbqYMM1tB+5+T yRY+rFyezj+Q223zyx/Gpfy0dV1yyDu0YFUklRN4CA1eTxVnDHOk7ewArDhoT2o7eF17Lw +gRUPKweJ3zGhLRp0FAZqovWK5D4Mqo= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=VzuGfQEJVf39ZiDAt09dMPTvgwh2rkzi1XBtXHq60Dk=; b=tPHk1xLTANF41P+YvclPaThq2C IybrgYAI67DiC1EXO42hWERlIGbBnNSeJ1loDjRAXs6qXs9HtGOSidn5ppwV7c1H16E0L3Fh/Ygwo bxVPD5MRp0DiA3mjzY6fgE8GCloBf51YVkI7ivA76zj1IVvOjxcGOd09E6a0zl0gW7rL8TQgC0xUV GOOAdXseMBU7tdB54A1V2+G0+OMWL+BU3Jau8iBKy7pT89ilQeeem/ozcYrLk7imdrMsVg7NVL6SR dsLtG6O1NPWHzLV/8XnGwNJhlT6C1vM4zF/Tu7XSHZr7zh1R3G4cLzBGEDdMTtJt7sHzEf7aX58VA Hk7k7FKw==; Received: from willy by casper.infradead.org with local (Exim 4.97.1 #2 (Red Hat Linux)) id 1rpDPQ-00000002CGj-42Ng; Tue, 26 Mar 2024 20:28:36 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org Subject: [PATCH 6/8] mm: Convert do_huge_pmd_anonymous_page to huge_zero_folio Date: Tue, 26 Mar 2024 20:28:26 +0000 Message-ID: <20240326202833.523759-7-willy@infradead.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240326202833.523759-1-willy@infradead.org> References: <20240326202833.523759-1-willy@infradead.org> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: hudwqxggeecjqnfx7xmyoiyhkeuuowb1 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 613EC180016 X-HE-Tag: 1711484918-201223 X-HE-Meta: U2FsdGVkX19sheus0VUWuOp+OGJonVUONr4XS/4BdAKUkn+mSLPcxWdo06f/g4uCSro4QF87NEeittwwn3dVRxYusWntr+eWY3nqOXFP2PW1+woPM0U4UPIXced9r2fXeZjDIPGsPLenIZK8ajAhfFo3awRrzdLn+l+z92TTF44TLYOdKDm3Klk2hRbp6OKHLpf1oKKPOwQJSyejpSADmI4/Quh7wWaUeg0MfMylZnGdtDX/go/Vy13mh70iEqfzdve975ZoTSvW9Eya5K4i1Ndjk9R/UTpk5HSb/2UUf5H+eOOvpIYHd+occF+Xm32LjngdcRiiwNfpAmAcyVVPC7leUbAx3Nn5nBSVqiJlEf2KRObmmb/NQUJZPn+x9PjfLphSfNjL0JRicKhKvuZhaQUVfZeUxiWQJFBnR9YeTP0N7x3qrg/Ux5UMCx+EbS2C46KLlFwN7iP5nnhKsoDlx2MvUo/v6Hn9MelBcDKn2N6VC5LPic7KPai3jr+qI7joJ6h3HydodyXtkWoEjM7OzR3s15h62QWjG49yrgpWCRCZgHJLKXMwPbIEeDMUQ1QcqNks8P3+Q9kdXmqbugOYsdQDIrq5kkN0TcRnuTuGxUW61WswF9gPvP917ymxMqgQ0gyDuIfxnRRqcjTD9AXL6yz/x1FRjFvPj1iGVXOYOjWFKvneMEC0LwNiEMxMAZa3jKMmXtoqizzs1D9EWoG9egHlu3QJtihwqC5Wj09P3a0cCCfe9/hvkicT4m8zcR4e4/8tsH+2NCbU9k2ZhpGBqM2m4Ru4pkptjEFAqeLJ7ZpYXsCbz2Pused5PIGW2rPH301700mOeLcum6QVW+lr+2p7K6dCem1s0CRgWheIYv3FnZGPF2BGWU8xDvrRcd3cLRBLqVo6+RNZTXhpdUHIYwzZ5WpUscxZQ8GzNGKl0jGUJQ/cvlUb1T2yX+hiWpHGxTcEHPm2V6eYD8O9RKy DpYt4Hy2 kqwwAenl8oUiW+YgSh3wwNj1olMrIm+nTYrdgdD7Gt+8idADZYXIs0ZUabVY6e3Ewd/9IgDdUKzsQu4N/SeznceS9YKELAomYUC6tSnJakI9qlA4RQIYfvumt61JmpCACUHs4kUgspDlGaJG4xDjrN8rMB6kJuw9KJ6fXJurDSz51+QI/P1XiW4KH+uIU7wdeEEhUelJcrt9zlfkw/iU4vTK8v590u/GDuCjz X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Use folios more widely. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: David Hildenbrand --- mm/huge_memory.c | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 91eb5de3c728..ff282317e71f 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -971,14 +971,14 @@ gfp_t vma_thp_gfp_mask(struct vm_area_struct *vma) } /* Caller must hold page table lock. */ -static void set_huge_zero_page(pgtable_t pgtable, struct mm_struct *mm, +static void set_huge_zero_folio(pgtable_t pgtable, struct mm_struct *mm, struct vm_area_struct *vma, unsigned long haddr, pmd_t *pmd, - struct page *zero_page) + struct folio *zero_folio) { pmd_t entry; if (!pmd_none(*pmd)) return; - entry = mk_pmd(zero_page, vma->vm_page_prot); + entry = mk_pmd(&zero_folio->page, vma->vm_page_prot); entry = pmd_mkhuge(entry); pgtable_trans_huge_deposit(mm, pmd, pgtable); set_pmd_at(mm, haddr, pmd, entry); @@ -1002,13 +1002,14 @@ vm_fault_t do_huge_pmd_anonymous_page(struct vm_fault *vmf) !mm_forbids_zeropage(vma->vm_mm) && transparent_hugepage_use_zero_page()) { pgtable_t pgtable; - struct page *zero_page; + struct folio *zero_folio; vm_fault_t ret; + pgtable = pte_alloc_one(vma->vm_mm); if (unlikely(!pgtable)) return VM_FAULT_OOM; - zero_page = mm_get_huge_zero_page(vma->vm_mm); - if (unlikely(!zero_page)) { + zero_folio = mm_get_huge_zero_folio(vma->vm_mm); + if (unlikely(!zero_folio)) { pte_free(vma->vm_mm, pgtable); count_vm_event(THP_FAULT_FALLBACK); return VM_FAULT_FALLBACK; @@ -1026,8 +1027,8 @@ vm_fault_t do_huge_pmd_anonymous_page(struct vm_fault *vmf) ret = handle_userfault(vmf, VM_UFFD_MISSING); VM_BUG_ON(ret & VM_FAULT_FALLBACK); } else { - set_huge_zero_page(pgtable, vma->vm_mm, vma, - haddr, vmf->pmd, zero_page); + set_huge_zero_folio(pgtable, vma->vm_mm, vma, + haddr, vmf->pmd, zero_folio); update_mmu_cache_pmd(vma, vmf->address, vmf->pmd); spin_unlock(vmf->ptl); } @@ -1336,9 +1337,9 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm, */ if (is_huge_zero_pmd(pmd)) { /* - * get_huge_zero_page() will never allocate a new page here, - * since we already have a zero page to copy. It just takes a - * reference. + * mm_get_huge_zero_folio() will never allocate a new + * folio here, since we already have a zero page to + * copy. It just takes a reference. */ mm_get_huge_zero_folio(dst_mm); goto out_zero_page;