From patchwork Wed Oct 18 20:32:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vishal Moola X-Patchwork-Id: 13427840 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9F92BCDB483 for ; Wed, 18 Oct 2023 20:33:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 06BD980019; Wed, 18 Oct 2023 16:33:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F38198D0016; Wed, 18 Oct 2023 16:33:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D3B9080019; Wed, 18 Oct 2023 16:33:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id BFEDD8D0016 for ; Wed, 18 Oct 2023 16:33:10 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 7A88140527 for ; Wed, 18 Oct 2023 20:33:10 +0000 (UTC) X-FDA: 81359731740.28.30B3941 Received: from mail-pg1-f171.google.com (mail-pg1-f171.google.com [209.85.215.171]) by imf25.hostedemail.com (Postfix) with ESMTP id 6F966A000E for ; Wed, 18 Oct 2023 20:33:08 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="Jc/i759g"; spf=pass (imf25.hostedemail.com: domain of vishal.moola@gmail.com designates 209.85.215.171 as permitted sender) smtp.mailfrom=vishal.moola@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1697661188; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=VN0zp0snARiXWQz31GgzZoIBHJFk7ynRrvZrhH9BDQE=; b=Lv0HmBzCXjGESbbhRY5OINN42Rt/VTdezSY6YDesxJzhz+LFvPAUNg2Eiwmw4JyenfuzJB 81Uk4waX1FYrJHOlYr8GkT5W0pUsfaOwfd5vKC2325ZZO/3TZq+TQAGfWvPVr8+aDuSAdN JJ2QDW1zStIq4czUkt9Cmcju+LuEvN8= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1697661188; a=rsa-sha256; cv=none; b=vma5xMdsXIXDmDpfUNb+06iq6fUs4IlP7/ry+gWNAt7rrJvngbXcZBzEHlYxYHXFqKF35J Wkhb/VX4YfIT7oqyHSQ2Pf41aLRWJX6GyIhuirIvorLj702Djd9HILU2xchHEP1Eib9KAo g40xwyhakC+DGmHFYzFM1B4r1rrChc4= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="Jc/i759g"; spf=pass (imf25.hostedemail.com: domain of vishal.moola@gmail.com designates 209.85.215.171 as permitted sender) smtp.mailfrom=vishal.moola@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-pg1-f171.google.com with SMTP id 41be03b00d2f7-5ad5178d1bfso4044861a12.2 for ; Wed, 18 Oct 2023 13:33:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1697661187; x=1698265987; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=VN0zp0snARiXWQz31GgzZoIBHJFk7ynRrvZrhH9BDQE=; b=Jc/i759gjoJZcIy9K00tPPGX2kUaHWfNvMwQb/YIWjPg5ER/02tbIqB5dawo8s+i+/ /ZzFfyWmxMU6Toa3seEFRtznJkMFBTirNwnk4wrvYgU7apCvYi55e0UPXZSllv9Gmmsq eeLKS2yHLbomR/wkr/H4JNX1wEg/Yy8N3/lesorKxikUpHBdaCGUIbNGliCHxolBiWPM ya0V78zyrvCUrl/AOzZx/D4kXAgzW1gRO0Uk1bFoJVzlLFhSxYmEbZkmG832Hqtg+WH3 E1FmJGjIYuSWiJ5ks5pL+o3vroQHzou4kk/kkoQD08HWdbysEAb5gSBKJeAevJIBeFag p0hQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697661187; x=1698265987; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VN0zp0snARiXWQz31GgzZoIBHJFk7ynRrvZrhH9BDQE=; b=QE1j9YFIVGPENpxdTzwG8iwsmhoPFJtHtpdayXYEw0oCg4nqhfjAB2xdS5maDax8Zn YCJh1jai5t65/XtvzmQGoEp2vrClo2ticJeVWEQKA6b3mVgU3P/jjIcqw7lyP0aWclhV 7dBcTndsV2XgKEQFsFRZ3YnEkKSV8lmL3Y/bfQexsZK+3tK1NiTsMB/kBQ4XwtWK56nx apafx5nd6gg3VRXpnAAKqQAkcPAzcMkV6B85nvwzfsu2Ol2JE3X21kRfIA1vnX8OK1UY 7LZ+RNntc1QrkPFn3UW2l5CUhjvmm5spRqZiXwCZdAuPAscf9dwKavCJCYA/YiWgwhVJ D94w== X-Gm-Message-State: AOJu0YxfUFXZFu1SG21d4/WFXpF93LlhefIZC+BuA51jipGlgmhXU9V2 ciF1l4YMiwkK8U+SVSQlgVLrrD+c+5E= X-Google-Smtp-Source: AGHT+IE6UrwDxrBBS7hjFHieWc31jxdejoBL6wMFdYPOtBiCq+4eALL+xiIct2VZuc3ylwqsKZTYgw== X-Received: by 2002:a05:6a21:6da0:b0:14c:d494:77c5 with SMTP id wl32-20020a056a216da000b0014cd49477c5mr307966pzb.13.1697661187061; Wed, 18 Oct 2023 13:33:07 -0700 (PDT) Received: from fedora.. (c-73-170-51-167.hsd1.ca.comcast.net. [73.170.51.167]) by smtp.googlemail.com with ESMTPSA id w12-20020a170902d3cc00b001c61e628e9dsm354412plb.77.2023.10.18.13.33.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Oct 2023 13:33:06 -0700 (PDT) From: "Vishal Moola (Oracle)" To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, akpm@linux-foundation.org, shy828301@gmail.com, "Vishal Moola (Oracle)" , Matthew Wilcox Subject: [PATCH v2 1/5] mm/khugepaged: Convert __collapse_huge_page_isolate() to use folios Date: Wed, 18 Oct 2023 13:32:09 -0700 Message-Id: <20231018203213.50224-2-vishal.moola@gmail.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231018203213.50224-1-vishal.moola@gmail.com> References: <20231018203213.50224-1-vishal.moola@gmail.com> MIME-Version: 1.0 X-Stat-Signature: oeepbi58erywxybhkcck4gm38hahr7to X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 6F966A000E X-Rspam-User: X-HE-Tag: 1697661188-73539 X-HE-Meta: U2FsdGVkX1/FXG6HUQ5/uT2WiSKXFkz/H7/wIkdgL3L7blSws0L6ne/CGENL+E6lo1GS4+4jQNqoubRooQLdpxKwaOquGtQANXuUmBeS3D5Hxjvhmq/fhUvw/3CzGjH5FJ+WkDwVg/EUGcGlShdhyCmR/CAkArgfKoHZDER2gfWLG0SS52cL46N+tt5BtKvABVzAhUro6u25s1AKbo6tRdO/NAzzIG83+1RsRA4oWUooVPLDsdf6ij0AXvhjHatfJiR+VBIBfj6rmuQUZT/48f7930YLTy15lq7oSjpYnOrHO1bL/djlLpfrNATsgSozsfLHPd/1nyynHCqCVuJlyleU0sudiWA1XwQfNCLx4N2LVRmI2wn7nx8eah9gxAWw2D3eXeKUCs9F6fj//t2TmqgolwyX+5chN2Gdc5Ssdhq6Ay39ufr1V/Z89goN2RHZC4iy0CN8l60QZllqMLT3XEX5Wv8kq/fPsG83dsM3mHkUGWPIH/TmgDv/KbxDsY9qz0IkfOK19FcwM23MqRbL4TCTgyWt1TbWC7sME607agZ7Dkv+NWdvJXBzQKrBhVXXlWfTUxtXfltsPxME77DPPAlu1zTVK43A9YnXv9GB8ekSFp88HcY3UBrZxoZ8fY8JBbUuyNXgpukv8l8VYAogVLvpT5bTJK9QGMzSML5GMbJoJl33PHlv0Q8mgWYFpISW/yVeJPRBwP9rZ8hNrExzT6McVs/6L79FQ4OhsxwSJ8mkS1JQ4GWdLNb6f5dWso3bToqv/6zk514g9pSkGhV5jnt/+qgjt2bnM95T7ldX1ah5/I6xPMP7iGn3CiQmattBsLb6XSWs94ZeNW1ejFfKOS3YJ266Pz/swhaLurGEt3jjt8Gl5z2bKxp7jJyymWmh9JQww8WdsHS4LClWXVtzvhkx3iPp7o+/Vva8DyHUoYHN7ndnG87/XgDaxRU3EQ/vlPLMLc8kKL0Wxg1OGiJ rQsC9Q2K 4KSE+3Yj1Cb1loDZjBFwGYWTi6/FhKAzL/WBqRnzOYwWGvFXx1zujhpEeHBke4Y3DIzb/gipnrfBWSU/LGC7th9BvtkEwuXdfunhj8PBjEAkCATblp5N5zgREQhmsWIQc6vNBz1gn2hy29z7B31m7qry6xJu7zxVrgKr05kzOJeHs+KnlMz3nyLJqefhElDakAHIxzJjFil6lq3EQ42njCglJaiqljEnMxg/rj/+tMogMZUuD8BCYhb2u6MSmjQqq9xO2cqSD4mZZVeOVlEyEY5FGa4kDJxKTtyrBA0XAKSDboIja9Aku5IGa8fKLnK/Y1LP7dBahd9YRiuntdtpNT4tBCwSddJz/4a4uAEz3Y8R52A8nqrWnUqcfKvzRjXaCzr8mjPKgyOYcSZyhnAKQaNJFTluU1KqMTrCpd+RecQfFJUFKb+yLgdpovck9iMLjNJAg+efBTDbMkGUY50jdcCM3p299WbKHCdgbcoxfbp/jHP3uLd4//vItMHn8GqBirWv4GoCPIJgfCW6mu+jtj1t9XUrAqrCjwU8/w0mm4nCOIASrnwh27683Gqo9mtCgjQUKw01dHMdKqj85YEVNAglcPaFHB8N8F67cS85nHEX3f3eyXzZKInN7Hrq6PZsXf3vSvRt9A3vSyNDrIJ7Cpr+2Eg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Replaces 11 calls to compound_head() with 1, and removes 1348 bytes of kernel text. Signed-off-by: Vishal Moola (Oracle) Reviewed-by: Matthew Wilcox (Oracle) --- mm/khugepaged.c | 45 +++++++++++++++++++++++---------------------- 1 file changed, 23 insertions(+), 22 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 88433cc25d8a..500756604488 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -542,6 +542,7 @@ static int __collapse_huge_page_isolate(struct vm_area_struct *vma, struct list_head *compound_pagelist) { struct page *page = NULL; + struct folio *folio = NULL; pte_t *_pte; int none_or_zero = 0, shared = 0, result = SCAN_FAIL, referenced = 0; bool writable = false; @@ -576,7 +577,8 @@ static int __collapse_huge_page_isolate(struct vm_area_struct *vma, goto out; } - VM_BUG_ON_PAGE(!PageAnon(page), page); + folio = page_folio(page); + VM_BUG_ON_FOLIO(!folio_test_anon(folio), folio); if (page_mapcount(page) > 1) { ++shared; @@ -588,16 +590,15 @@ static int __collapse_huge_page_isolate(struct vm_area_struct *vma, } } - if (PageCompound(page)) { - struct page *p; - page = compound_head(page); + if (folio_test_large(folio)) { + struct folio *f; /* * Check if we have dealt with the compound page * already */ - list_for_each_entry(p, compound_pagelist, lru) { - if (page == p) + list_for_each_entry(f, compound_pagelist, lru) { + if (folio == f) goto next; } } @@ -608,7 +609,7 @@ static int __collapse_huge_page_isolate(struct vm_area_struct *vma, * is needed to serialize against split_huge_page * when invoked from the VM. */ - if (!trylock_page(page)) { + if (!folio_trylock(folio)) { result = SCAN_PAGE_LOCK; goto out; } @@ -624,8 +625,8 @@ static int __collapse_huge_page_isolate(struct vm_area_struct *vma, * but not from this process. The other process cannot write to * the page, only trigger CoW. */ - if (!is_refcount_suitable(page)) { - unlock_page(page); + if (!is_refcount_suitable(&folio->page)) { + folio_unlock(folio); result = SCAN_PAGE_COUNT; goto out; } @@ -634,27 +635,27 @@ static int __collapse_huge_page_isolate(struct vm_area_struct *vma, * Isolate the page to avoid collapsing an hugepage * currently in use by the VM. */ - if (!isolate_lru_page(page)) { - unlock_page(page); + if (!folio_isolate_lru(folio)) { + folio_unlock(folio); result = SCAN_DEL_PAGE_LRU; goto out; } - mod_node_page_state(page_pgdat(page), - NR_ISOLATED_ANON + page_is_file_lru(page), - compound_nr(page)); - VM_BUG_ON_PAGE(!PageLocked(page), page); - VM_BUG_ON_PAGE(PageLRU(page), page); + node_stat_mod_folio(folio, + NR_ISOLATED_ANON + folio_is_file_lru(folio), + folio_nr_pages(folio)); + VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio); + VM_BUG_ON_FOLIO(folio_test_lru(folio), folio); - if (PageCompound(page)) - list_add_tail(&page->lru, compound_pagelist); + if (folio_test_large(folio)) + list_add_tail(&folio->lru, compound_pagelist); next: /* * If collapse was initiated by khugepaged, check that there is * enough young pte to justify collapsing the page */ if (cc->is_khugepaged && - (pte_young(pteval) || page_is_young(page) || - PageReferenced(page) || mmu_notifier_test_young(vma->vm_mm, + (pte_young(pteval) || folio_test_young(folio) || + folio_test_referenced(folio) || mmu_notifier_test_young(vma->vm_mm, address))) referenced++; @@ -668,13 +669,13 @@ static int __collapse_huge_page_isolate(struct vm_area_struct *vma, result = SCAN_LACK_REFERENCED_PAGE; } else { result = SCAN_SUCCEED; - trace_mm_collapse_huge_page_isolate(page, none_or_zero, + trace_mm_collapse_huge_page_isolate(&folio->page, none_or_zero, referenced, writable, result); return result; } out: release_pte_pages(pte, _pte, compound_pagelist); - trace_mm_collapse_huge_page_isolate(page, none_or_zero, + trace_mm_collapse_huge_page_isolate(&folio->page, none_or_zero, referenced, writable, result); return result; } From patchwork Wed Oct 18 20:32:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vishal Moola X-Patchwork-Id: 13427841 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 19293CDB47E for ; Wed, 18 Oct 2023 20:33:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 243338001D; Wed, 18 Oct 2023 16:33:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1A57C8D0016; Wed, 18 Oct 2023 16:33:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E75798001D; Wed, 18 Oct 2023 16:33:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id C9EF58D0016 for ; Wed, 18 Oct 2023 16:33:11 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 9F5891A0393 for ; Wed, 18 Oct 2023 20:33:11 +0000 (UTC) X-FDA: 81359731782.08.5057E31 Received: from mail-pf1-f180.google.com (mail-pf1-f180.google.com [209.85.210.180]) by imf27.hostedemail.com (Postfix) with ESMTP id A7F2640027 for ; Wed, 18 Oct 2023 20:33:09 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=JUTVa4Ry; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf27.hostedemail.com: domain of vishal.moola@gmail.com designates 209.85.210.180 as permitted sender) smtp.mailfrom=vishal.moola@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1697661189; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=kOxvYaksrDFINgvTHvg7i3BLH6nikNi8tBwFFhT4+xI=; b=ZHDPDftTEQo9Y69vRL/FtBiIRsSHxpRY5W6+bGGWNnJaISMtMlriAmwQ2HJYuwmzMqPvwb 77Xzgl3vrIBe54xbVxrKspt8uNtyUeZzVrj0OskNUt4GEmCSJGrUB76vp3aJ6ri0njHNZg /pV5e4kxCkn2NiWAJICY7pbzGnX0jW0= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=JUTVa4Ry; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf27.hostedemail.com: domain of vishal.moola@gmail.com designates 209.85.210.180 as permitted sender) smtp.mailfrom=vishal.moola@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1697661189; a=rsa-sha256; cv=none; b=XxFgV3cMls5ULSLVy8FAjE5Gc51r006piM0RvzeVo2OZEWDyrUm4GlAOpi/j19fBTT7f3i PYpPHSYL3cIh27DnpcChqWwkV3RUKl/10yk+3KQtWHkOSQAbs/UbOvOpUN9b7Ig8bMG5+P ROP6KmYd68cF/VWU/I0rLy5K48ZuwM0= Received: by mail-pf1-f180.google.com with SMTP id d2e1a72fcca58-6b9af7d41d2so3913528b3a.0 for ; Wed, 18 Oct 2023 13:33:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1697661188; x=1698265988; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=kOxvYaksrDFINgvTHvg7i3BLH6nikNi8tBwFFhT4+xI=; b=JUTVa4RybRF/6beipfKFSMgzntSc+ea1docrmpP7g8H6z07i7WMNudFG/P8tbtufDt RYC8vFctryn3WD0T5FHxxLIh6r2CFzsgqbXB5jqiWWg7itny4Xq3wY7dlxfv69S7FzLM Wz78wTuDWjsB25POzSJGrkXI83eVidYngHqt7YcdarjRVg4f6QUluMjA4jqcsC3hUt/Z Gks2ZjwrC7+HL8TWy1ujvdR7SldZD1p1/qlxFCNnsbPJ7n4cHEF2Pdj95/lfDBGmAPNc JFVtHnqGfH6NKrl4FikX1dA6NE9SVnRC1t4+Dxhy6oPFfY9QH4wrbvLuUjf2ChjBKIYC KtTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697661188; x=1698265988; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kOxvYaksrDFINgvTHvg7i3BLH6nikNi8tBwFFhT4+xI=; b=tzZT4Vg4oq8GG7xn8Qfi+ECHihlFlNuUXHiAs26iznN3SuwWgxoGJzeyR3YI/CNk4j cHOZBlCmuPQqYOLPKYyaJulc9py8CPB/NOjUYDbaTf2xBRatD8o/s2CQ+Yr8KcPHtTFn hfcgcDk8xinLZwaST8Q77ZLPc3rc63XRkbrlLvHwgvSmgtKHIW4yK4r4qrvxNqQMRZli HRrtJGgOskcDkNJyjX73Ipp2zNKPunr/pFPaWvTQvc6l/q4g/s/oRYPJsYb84FVhT4/H 1nMAJSQZBaJkfGpOfI//c/GwVe+134DzAaJXx02NEcQSez4vXQ6+QY9OhfmOmLOOelXG xVnw== X-Gm-Message-State: AOJu0YxM/8BInDeNjizWsCeY1GuV1S+IeXpfuFWkjTJlKHnu89QWxZBn N4M9wm0fnlvU1iBrP3JbCLWHeLULqHk= X-Google-Smtp-Source: AGHT+IGSy+gf+h7WIcfgpR19k4aoWiTB0tv72CZNC2cmYBpamwb6VVIlhQcsy15b27bnd2RjKzzbdg== X-Received: by 2002:a05:6a21:6d99:b0:14c:d494:77d1 with SMTP id wl25-20020a056a216d9900b0014cd49477d1mr327396pzb.10.1697661188209; Wed, 18 Oct 2023 13:33:08 -0700 (PDT) Received: from fedora.. (c-73-170-51-167.hsd1.ca.comcast.net. [73.170.51.167]) by smtp.googlemail.com with ESMTPSA id w12-20020a170902d3cc00b001c61e628e9dsm354412plb.77.2023.10.18.13.33.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Oct 2023 13:33:07 -0700 (PDT) From: "Vishal Moola (Oracle)" To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, akpm@linux-foundation.org, shy828301@gmail.com, "Vishal Moola (Oracle)" Subject: [PATCH v2 2/5] mm/khugepaged: Convert hpage_collapse_scan_pmd() to use folios Date: Wed, 18 Oct 2023 13:32:10 -0700 Message-Id: <20231018203213.50224-3-vishal.moola@gmail.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231018203213.50224-1-vishal.moola@gmail.com> References: <20231018203213.50224-1-vishal.moola@gmail.com> MIME-Version: 1.0 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: A7F2640027 X-Stat-Signature: xthyfektzzhym59jx69zt1gyif5bd5hz X-Rspam-User: X-HE-Tag: 1697661189-655181 X-HE-Meta: U2FsdGVkX1+VpDOE36H//rQOn6lgrLiQrr+uPxkKceE+a61gKu5QJ878zl2cP5OI44R7Ic0VN5B/EVkalJwix7vBE4TOu/SSgr1DqtuWPf+Q5ct/xmpNbN7PnmKCA7c8U1NiMcrU4AekwxOGT59pcQgQoyg8SfEMCNBHJVIq5V4fHq542W86MHYExyFafOw9jjXfgzO8AjQwSgfl3vVCgMvO6x5IQgg07xh8k+tRamZc5Vjw5I8f/6HZoWJZ2J5ifgdqYPxnytmu/YOK6ZAh8lL7TbG47Jt1ljxCyMtlK0qMDEjw5mwULUSmfH0/1QB0pfha2sXxQuB7UWxMngPKbHZlXURY/5oGsbJOYUtZNID37KT2F5DWzJOyiZ8aYjpD/V71h8kaxRyGji8pjA6EORaiRmy48fLLqBKU1DkXa34aHZDZdAt7A6W0xIoMXSuR1Y0rYzxj6BImoyYC5g6GW9p99XMNyOlZV1agOt4b3hWzjJMcS2eyX/dz9+st97UObiyT1hxRL+Nh2sYdX41j+THZT3y+eJXj8nssjwhMGRj95jug7hOoQgkhC4ixXf1KZtOd8dvewU+uKMUlghWhtxoEhmJzox4MDFINgmvq/OLnmhgadQRoW8o/80bHAToEl6oqjN9ECWgBFAzBqLFfnLJ8vw9QuOqZL2ySSq5kNtUgFdRy5E9pe/DGL1bZhzOcoQD7YwuMk+QJXGCms7t895UsoREg643j68xpw9tDSRZKKCkIYn1e9cekz9Yoh+z/mk50jvFld1PRTRiYiAmZ61nNah9N/DX1BlriJFumPss8VdV0BHAs/n+scho36LW1+Nc9jwYKA+RPmphKVqqDLrw2vvtoOxAefhQl8YoYrdvzKMQxRbslG8OX7PAWgEy7R/MXEteRj2JS5EYVILk2oEFeTh77byLPSPxaoNX260Da9ZFL0HBvhQTX5mUA+mdKmgY31JRHemE8zuF4tNu BWWJwV89 k0VOJ3djNAdsFH9uFpwDC8Ytz0jce1juVawm+fK8ZTyiYJPKbWUa7uiIcJzP07WkZ9wJ14QEojfezoRVsMCzMT4/0e896yN39lpgfhuR9XuVZKgZ/48NJV3gfMtOJuHWIp548bkWFmlQDNQqBEBZJO2b+xXyoiETSDI6Jz/h4STlxuLKVW0n8Sso/p5MmPofWVsseqjgiOOQN73hD28lGaeneNEnvDHaRuVWQTTDnSRLlCGHy9tcUGfOErkdGCdm9h0SknQJH97nQxsL2VaD9DzatIXuBbf38DJ2xOSjqt+hmdw63cekKsvA0d3SN8Rl20HHV3vBFiMqxRJgN1H2fDZ8zpfQ1pSsxq+tD8UqNcHp3/4H/xKH3OM4MxDWo/+yOh5waNzQqDZDOsI4fr6UvaHkoIu2NBi4WMDe5qp3XGAAqDsxlN7GkA6Z6pHauljwg3j/96XTxvTuCVrsgZhWJheUe/R80W5nuspWAyNcFZToH4ElYjgsK46h+cF/8rTlgXIQOEf+GcPPIMhshvx/YQGRizzVgVgO6va89+5rL9QfhoKLUPYzapJvWD2wh8kYyRS+UytVubLolDmHnHYvvmSV/lg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Replaces 5 calls to compound_head(), and removes 1385 bytes of kernel text. Signed-off-by: Vishal Moola (Oracle) --- mm/khugepaged.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 500756604488..6c4b5af43371 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1248,6 +1248,7 @@ static int hpage_collapse_scan_pmd(struct mm_struct *mm, int result = SCAN_FAIL, referenced = 0; int none_or_zero = 0, shared = 0; struct page *page = NULL; + struct folio *folio = NULL; unsigned long _address; spinlock_t *ptl; int node = NUMA_NO_NODE, unmapped = 0; @@ -1334,29 +1335,28 @@ static int hpage_collapse_scan_pmd(struct mm_struct *mm, } } - page = compound_head(page); - + folio = page_folio(page); /* * Record which node the original page is from and save this * information to cc->node_load[]. * Khugepaged will allocate hugepage from the node has the max * hit record. */ - node = page_to_nid(page); + node = folio_nid(folio); if (hpage_collapse_scan_abort(node, cc)) { result = SCAN_SCAN_ABORT; goto out_unmap; } cc->node_load[node]++; - if (!PageLRU(page)) { + if (!folio_test_lru(folio)) { result = SCAN_PAGE_LRU; goto out_unmap; } - if (PageLocked(page)) { + if (folio_test_locked(folio)) { result = SCAN_PAGE_LOCK; goto out_unmap; } - if (!PageAnon(page)) { + if (!folio_test_anon(folio)) { result = SCAN_PAGE_ANON; goto out_unmap; } @@ -1371,7 +1371,7 @@ static int hpage_collapse_scan_pmd(struct mm_struct *mm, * has excessive GUP pins (i.e. 512). Anyway the same check * will be done again later the risk seems low. */ - if (!is_refcount_suitable(page)) { + if (!is_refcount_suitable(&folio->page)) { result = SCAN_PAGE_COUNT; goto out_unmap; } @@ -1381,8 +1381,8 @@ static int hpage_collapse_scan_pmd(struct mm_struct *mm, * enough young pte to justify collapsing the page */ if (cc->is_khugepaged && - (pte_young(pteval) || page_is_young(page) || - PageReferenced(page) || mmu_notifier_test_young(vma->vm_mm, + (pte_young(pteval) || folio_test_young(folio) || + folio_test_referenced(folio) || mmu_notifier_test_young(vma->vm_mm, address))) referenced++; } @@ -1404,7 +1404,7 @@ static int hpage_collapse_scan_pmd(struct mm_struct *mm, *mmap_locked = false; } out: - trace_mm_khugepaged_scan_pmd(mm, page, writable, referenced, + trace_mm_khugepaged_scan_pmd(mm, &folio->page, writable, referenced, none_or_zero, result, unmapped); return result; } From patchwork Wed Oct 18 20:32:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vishal Moola X-Patchwork-Id: 13427842 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 951ADCDB482 for ; Wed, 18 Oct 2023 20:33:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B69BA8001E; Wed, 18 Oct 2023 16:33:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AF43E8D0016; Wed, 18 Oct 2023 16:33:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 91BEF8001E; Wed, 18 Oct 2023 16:33:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 7CA0A8D0016 for ; Wed, 18 Oct 2023 16:33:12 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 5265CA0F06 for ; Wed, 18 Oct 2023 20:33:12 +0000 (UTC) X-FDA: 81359731824.01.AC54827 Received: from mail-pg1-f181.google.com (mail-pg1-f181.google.com [209.85.215.181]) by imf29.hostedemail.com (Postfix) with ESMTP id 7E4AB12001F for ; Wed, 18 Oct 2023 20:33:10 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=i5wRpO3u; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf29.hostedemail.com: domain of vishal.moola@gmail.com designates 209.85.215.181 as permitted sender) smtp.mailfrom=vishal.moola@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1697661190; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Q/rAUjlYbudj5wnw8lHkufWutUwYDRgsClPJWjawwgo=; b=tu7fpALvfgzBFXFAYT0hhcOd1VpJHdMCqfJW3NmIfecPH+Jll91yo42tmSsi5y9RQDOkA9 mx6s9SROXldCTEWdC9K+DsQdNNmNMwSfqqhBf8GXPuDzo3LjtSKduxN1JLxH9WRKgqzh03 SnTVATujP0msbOR2cx2Ym8HlvXlVETs= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=i5wRpO3u; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf29.hostedemail.com: domain of vishal.moola@gmail.com designates 209.85.215.181 as permitted sender) smtp.mailfrom=vishal.moola@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1697661190; a=rsa-sha256; cv=none; b=GVPwoQEQVD52aw+FoBHSkBliGyTJj2VWyyo4udseRH5upiYKvFYMxT6GAXPKGioYN5WB1c QQX/+A6OepkMW22/dZWoavELecd2i/iY1brNkow7nBYlTjSXvZ1WOtfJ374bBwNDzTakd6 2KpG49a5azLQXx1/W7COTP0DlGMZtiE= Received: by mail-pg1-f181.google.com with SMTP id 41be03b00d2f7-5aaebfac4b0so3660734a12.2 for ; Wed, 18 Oct 2023 13:33:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1697661189; x=1698265989; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Q/rAUjlYbudj5wnw8lHkufWutUwYDRgsClPJWjawwgo=; b=i5wRpO3u4HGYoSnpMdQgf6rXFmJ11otYFtqlbXH7IAzpq0WsgVpKYkRCyZdexR+1zJ CAMKyaNJpOVyzC+f0CW8toEeuJqbYykMpGbq2wpUTPQ4v+VbfWNchRGZucErSxZuaEMp WaZivVb3koitzW5gti1sKBdqXPNO5MG8yqQGXAy+0NE2nTdgSYhOo25NTc1DrQgcOxjh ZYnvqg8EgkJmpM2CEuK3vyPkMMtjVm6xp9ys8vswz7k1g4GWpf3B4fx4pviJhMM5dqR6 1tAcY+DF7ZlcMu6YCpAtOt61mkGPWL/mTy+ga6K/Sma8+5fapmBCSKflNUU7GpEa78vo W/8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697661189; x=1698265989; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Q/rAUjlYbudj5wnw8lHkufWutUwYDRgsClPJWjawwgo=; b=RDF4C+dLI5e63vM8ljdx+LpzIw+DQT0EZaF26N3T20rP9RPN4VhWcFEWURclews9yu Fxf40e7VD1WsRt9df/+xIMJlaUTfxSd2VlBwThpcIPls+QnYcfgrJeYLiuCRdHuLL8Vh 2avJbioB9hpeeNQ/vHBYuqDA7jRjiDgREebUuXlsAD8kk2h6+ozkPg/+jZHPqrldFWv/ 0hb2bonipUArTxRFmJ/xJFDxd+ZgtYPGDc+uMP6OMiKL986667iQVAX+WWhzByJ5XHr1 JV871VF7nKIs4l1338gfeK+BHGtKCVz/GVr1tVuDONHMlUVU8a0tzdUuzp6iHYfyAI8M wfeQ== X-Gm-Message-State: AOJu0Yw9uTcEPmOg94NB0484If18S63wEKNUqhumSXv1lKcsXrjo1SqG YdKNN7V/DkhM7PxQpoi8HVicrz9nphE= X-Google-Smtp-Source: AGHT+IHHKufGkhuDo4UJftnY7SwsN1FFsHrItzDWiS842QRY9mPhF1ZSrMfqcuvk3t7Vnjm0ZSWi9g== X-Received: by 2002:a05:6a21:3e0b:b0:171:c88a:890c with SMTP id bk11-20020a056a213e0b00b00171c88a890cmr269714pzc.25.1697661189243; Wed, 18 Oct 2023 13:33:09 -0700 (PDT) Received: from fedora.. (c-73-170-51-167.hsd1.ca.comcast.net. [73.170.51.167]) by smtp.googlemail.com with ESMTPSA id w12-20020a170902d3cc00b001c61e628e9dsm354412plb.77.2023.10.18.13.33.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Oct 2023 13:33:08 -0700 (PDT) From: "Vishal Moola (Oracle)" To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, akpm@linux-foundation.org, shy828301@gmail.com, "Vishal Moola (Oracle)" Subject: [PATCH v2 3/5] mm/khugepaged: Convert is_refcount_suitable() to use folios Date: Wed, 18 Oct 2023 13:32:11 -0700 Message-Id: <20231018203213.50224-4-vishal.moola@gmail.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231018203213.50224-1-vishal.moola@gmail.com> References: <20231018203213.50224-1-vishal.moola@gmail.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 7E4AB12001F X-Stat-Signature: ip38s8p616enqztn9eqz4jndz7bfihm9 X-HE-Tag: 1697661190-372906 X-HE-Meta: U2FsdGVkX1+Gqm2YXT+a4P99mOyS3gc39ZVP2CH7SfJc42xSf3w4f6Q0purvKg0ZqlUL2NRaWzwi7hC4GpyRyCXmC3KtZX2wLyEsQscnK/Vl2iIhynuYy3BKWm84uUcPelfzr3vmELmpKv81pzHEdnMX+T8wmdO8XqiVHmjQwcv2qy0v60t2YD2ZKDfP4oog1PnWDb/ucJ+jc8FD6i0DC1xzlMPQcNCTLMFPpbl06OXNhhyagZkGq1WWPsPx97ebM6UPlN741M/gc0NpAnPljVycLA+aJ+I77c9h6FtYbWWNpOHzrXhu1SrBXY8VE0zj4eOCxVM3YRmRnLqjQtx12JgV4O5Fbf6TtvTUWTgxx2+ahQppPHxjnjSk2aFDrLMso5VAyQe3GR8YiQhjM8mtdfw83la+SlOTJTrr6kGkemAFIZ8n/xIpcdGzBw9uX7UKtuYE3wkfOPetOzmxhcS/cVd9o6/OPeIeal+kcx+w1zKareTjsIncfBaSQTx5LCpsxBRgQdzhb3RVfsccvBfKB/RyiYVGwQjkfnGLIISYQGPeaOO6D/CcpUzIxVItQTh2hpNSn5rA9r5achooVS58py1jQfsVyGETkQi+xf5m84DcMGfvE7KTZ4fDLcHZdya1nfLvxgMZLgnxCPvImnyVJ7n07IUDltVbr2oZG5Aj++0t7R8qrRlN/gxGn6fIwrWD6k+f3elNriZd0opBzwnsXJHTVbm4lpetlKeyqTalyXCNc7ySrAeV9DW71TrKC0i04aR6lh4sjcOPUaENjHePTKKsdm8tbID6LxOWnBHTsOg8tn39HzqU71atKUhyd7znRzu8KkW9oC7MXPPddshFgtevbO7EitXR5pAcK11vFhnOYFJTpIWZCT1UHo9/g68qFGypxZBqF7tXCM0J6eJr/f6aeREXqBg4B08Jc4ZRLNpvfoj5yXz8F2m0pjEQMlp7LVfbDr8Autr8ttjdKyu xAg89y6Y pA03MRQjoZYhYhB791MDBc9K3OaXs3bP4Nn6udQn5L4GVXyc64BO9brFsPcGtU0feFZpbUbAoodzWEwKwY4yzzX2K2ZXPjH9jA3QJ2ZbWynzY0xWaztP/lc6IniC791uk4k7VTiQ74bJTjEIW22/XpJgTA8dGY63PIlfLgvJs/h/2zintXiRrYrqIJwFGfy5/5n8PNjVOoVTrJGTVrHuWbaIr4+YgGwz5JJeHr9ERto6R+cNAFIIvtebEfaWckG0A+ipsA9GFeO/4kGDumUXNAIdhk3/JXCF5p5bMtEnWJmdRsGEfOKvbdWGWgTZIW6EQT/QBMVriuxL2BDD8/AlOBTzs94TCd8Qjfr7ySGLk60hYEoHaIxQB+Ord6HIXFI0DcNRkDiSzxoF7/6BOZj2rac3UZgfWzzCEoEMwJz4OG55sj4d0TQTWpoAtLy6BgMhKh3H8UW/bwwam3QTdFCqXox19NUbpM7vQ3643tDlyKHn03nC0NDHdYQQmEafFoVnYxdjQPLZOhU5RexDPfBbC49Wjfg1MFVPKOnUT9sMfcWrbDjn5kNbMNmt1cE4/G+0cDM6zb6thsn37BYWrNEYE2rNPjA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000007, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Both callers of is_refcount_suitable() have been converted to use folios, so convert it to take in a folio. Both callers only operate on head pages of folios so mapcount/refcount conversions here are trivial. Removes 3 calls to compound head, and removes 315 bytes of kernel text. Signed-off-by: Vishal Moola (Oracle) --- mm/khugepaged.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 6c4b5af43371..9efd8ff68f06 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -524,15 +524,15 @@ static void release_pte_pages(pte_t *pte, pte_t *_pte, } } -static bool is_refcount_suitable(struct page *page) +static bool is_refcount_suitable(struct folio *folio) { int expected_refcount; - expected_refcount = total_mapcount(page); - if (PageSwapCache(page)) - expected_refcount += compound_nr(page); + expected_refcount = folio_mapcount(folio); + if (folio_test_swapcache(folio)) + expected_refcount += folio_nr_pages(folio); - return page_count(page) == expected_refcount; + return folio_ref_count(folio) == expected_refcount; } static int __collapse_huge_page_isolate(struct vm_area_struct *vma, @@ -625,7 +625,7 @@ static int __collapse_huge_page_isolate(struct vm_area_struct *vma, * but not from this process. The other process cannot write to * the page, only trigger CoW. */ - if (!is_refcount_suitable(&folio->page)) { + if (!is_refcount_suitable(folio)) { folio_unlock(folio); result = SCAN_PAGE_COUNT; goto out; @@ -1371,7 +1371,7 @@ static int hpage_collapse_scan_pmd(struct mm_struct *mm, * has excessive GUP pins (i.e. 512). Anyway the same check * will be done again later the risk seems low. */ - if (!is_refcount_suitable(&folio->page)) { + if (!is_refcount_suitable(folio)) { result = SCAN_PAGE_COUNT; goto out_unmap; } From patchwork Wed Oct 18 20:32:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vishal Moola X-Patchwork-Id: 13427843 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 49E48CDB483 for ; Wed, 18 Oct 2023 20:33:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2B6B38001F; Wed, 18 Oct 2023 16:33:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 23E8B8D0016; Wed, 18 Oct 2023 16:33:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0B90E8001F; Wed, 18 Oct 2023 16:33:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id ECECC8D0016 for ; Wed, 18 Oct 2023 16:33:13 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id BC80580559 for ; Wed, 18 Oct 2023 20:33:13 +0000 (UTC) X-FDA: 81359731866.06.B93AA42 Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) by imf22.hostedemail.com (Postfix) with ESMTP id 98483C0032 for ; Wed, 18 Oct 2023 20:33:11 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=RHkQzeVY; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf22.hostedemail.com: domain of vishal.moola@gmail.com designates 209.85.214.176 as permitted sender) smtp.mailfrom=vishal.moola@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1697661191; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=rvpcd99NjN7SPkeCiS8fl8F++V5hN5b5iiRnzQcVK8o=; b=Yk5rujcACCqKBx99tg/DS13a1oMxYNb7ZiCBnNA/kMxqAtVZ3QBOWZ3Fm8gt+yoha9StC+ 8pJJ9re98gYOmYR8A/kc6WWTtQpOjbu88nLEcnzH4yv/iJau3gMDYLWpZBzBo69DxwbjSj aUYh1bCEufjDA6ucYBc8KbSjg+iesUM= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=RHkQzeVY; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf22.hostedemail.com: domain of vishal.moola@gmail.com designates 209.85.214.176 as permitted sender) smtp.mailfrom=vishal.moola@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1697661191; a=rsa-sha256; cv=none; b=aFT52FOvNNOpcPQJdVqYyywLk/TJY6Yy/oO6BPip6NxVJlscp1koVTd78K3ySFhwspmrO4 NSwFDphR0C3IOd6EmBspqmlwCrbXqqOYumMiHWou7mY/Jex88pc1mi3CiH3Plz3+sfWjk6 NeVSMbputDyMc4n/Bos0UCK+WTQpNKU= Received: by mail-pl1-f176.google.com with SMTP id d9443c01a7336-1c9e06f058bso748415ad.0 for ; Wed, 18 Oct 2023 13:33:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1697661190; x=1698265990; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=rvpcd99NjN7SPkeCiS8fl8F++V5hN5b5iiRnzQcVK8o=; b=RHkQzeVYswahmrL8WkNn/fgVjqsWhgJfTNqvVtYRs6Tk5zGhPcihicpvtMD3g5pUpz vvCLdcX75PkibI6SujbrZkdXusERD/idgyKS0GmTxxqgSvBmT8W2YCCzgKGq3zy85lTL /k3/a86vcrgwRx74jN/MwBKN2NmrevHG/G+jTf8oZeJ7yfSfS5Dk07ywkEA+jYKHZSiU 4wgnts0JGAmXyMrQx6yA+yAeagt7JcvE458nyc58H/+/FaJXmKmpNnEWF+Aa8SqxJaSG 0XMma9vCAgy76kEb+g8vMoSNCraem09Gk3vcGyqN1dAEfI033GZeLZg+xUgUhgBDb8kV +vNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697661190; x=1698265990; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rvpcd99NjN7SPkeCiS8fl8F++V5hN5b5iiRnzQcVK8o=; b=mgfi60GcYwcACxJ21VLoEH0lylbhR14f6cUnZkOyartBPzyLxOUPc+XKtvCtzwyjdi kZLjqIeLeKlI+c1b6j9/qPAZsg3JncOI3+HlC2U43CXxR5+DPmE1QkOwzvFNK4Y+9PIS hcYKafVV5MU5X91H/bFc2KR4kW8YwwbHFW2UEPlazSXdm2PHGWqpDtV8UVxH3jQ36kXg Bp3gI90nbuN0QEXMqjZFL+X6yqD2GUkhZI3RaRgbJUkehK1gMa+FKiKk4BO7zUsyflFj +Mw/f9rjkanh+Gx5YWAiSdd7Wb+CuGlqVOHqwOo91Ok04wIsl0Az6GcVqjI9xOqeV4oK XBiQ== X-Gm-Message-State: AOJu0YyHt6LgAl9MSZQdTKYto4UJzJCG6GUtw5YRQfdHGKcCD0w5duzL c4X9RAlEEvo86nUt/R+exhqS9o6KyN8= X-Google-Smtp-Source: AGHT+IEX5D/uPbXU1tWc0zw0n/3MIN2cV8mRj2GKuowCEB+zQiYHmm8zVTMDrPAIhqcUNoWkYM/oPw== X-Received: by 2002:a17:902:d2ca:b0:1c5:7d49:570e with SMTP id n10-20020a170902d2ca00b001c57d49570emr615553plc.29.1697661190159; Wed, 18 Oct 2023 13:33:10 -0700 (PDT) Received: from fedora.. (c-73-170-51-167.hsd1.ca.comcast.net. [73.170.51.167]) by smtp.googlemail.com with ESMTPSA id w12-20020a170902d3cc00b001c61e628e9dsm354412plb.77.2023.10.18.13.33.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Oct 2023 13:33:09 -0700 (PDT) From: "Vishal Moola (Oracle)" To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, akpm@linux-foundation.org, shy828301@gmail.com, "Vishal Moola (Oracle)" Subject: [PATCH v2 4/5] mm/khugepaged: Convert alloc_charge_hpage() to use folios Date: Wed, 18 Oct 2023 13:32:12 -0700 Message-Id: <20231018203213.50224-5-vishal.moola@gmail.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231018203213.50224-1-vishal.moola@gmail.com> References: <20231018203213.50224-1-vishal.moola@gmail.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 98483C0032 X-Stat-Signature: 68koowuguxobw3cugs67n16xeyxnh3it X-HE-Tag: 1697661191-440016 X-HE-Meta: U2FsdGVkX1+M1JdZKVk5mMca2ZkLJPuuxQiceMPVSVawrgNXU3XR/BfJCnZ1e5Qo7gf43y9agMJC5zth2KElPYVuxMD5rNzOORJC4gRBU6mNWZrzp+VXgsxz1rfNexlaCFYFu29KnWQp3ggOKHIia2OVsqqV85Tus8kLE52uaNG8j3OOBu5tjxqbmKGjwGJl7tH6F4JjOm5aB5iT/GoCHIAq6Mp5uAcESpZ7OLNX0j4rlCCvjdds2lSS/N2MuXgZbsE4vrh5n3ZYZdby9UOnVY9MbhB18SNTibfdmiaOVH+Kx/B/rxKac6AOCOQN3nd9566FG8a8GNReRhiGlKvzUvB86fDLikkteCUvVuuY5uKC0tNw3gLhKaz8Zvf9diO4V6ZTLcAxq7e0p84ybOg069YFt3O2n+CgJnOaWaOsZ/RrpFbcmrz0Vy+poWwt2N21Uxu5X7FXgp++y+b3CPIFDcpU6iXQTiqF1rmpEuBMr0rzjw+JxsTp/JDvtjDcl4PPf2y6yu3bgVBV73Wzh2VzTtpLsk2ATaD1AG/w3Cf9GNC2LGHujUml9R7h9dJKSVXHJYo0BCmEsujotK4rctzyyp2xqduc0P2/7e2RfudkJ16szh1uWs5YJndnPour9j8QmRPNlOlXNE/dBkLzFwfVqjo65rfOLnF6qz1Qv0cMVyubzYC5yeiTspBtnkd3XHP8o22ZYmNeqEinb/mkpxTm81zrTmQeq9fSdES9NrBkwR0RYAG/du2OzPcjNj/jTbDyB7aEUiauBuG2GEK2aKnbrXsXyan6uQIO4pU0ZIgQpD0hTutj5CpVSCkMvElsw764KFZnNyJ9vTobKQ6F5KeEfCFDoZ1h3bs6Yrfnw+UaDPwOa/JdH3qD2TKWSV4F9Rj6gKsGuL5FflKk2a0UyExlBSU6QXtSnXpDq/2RGyIFpcMXslS2YYUrp1l9pTV9HTwXg75Q2pJq0JXDD1Lnd0W fIBGsZee ICUvhF127aE3olwufnEyI0I0pkvjrdtRE//Y8MQbTFxW/Y/g3FgT0iBqyNM+I7456LqULswjSJmPRxvz2mwy8RCAcI6/os/BS9X9xBcA5oigBhCn4SEtTnKrtr7GwVgHQjsBlYVgG9XjvS8i290Uhw/w++geW3JDhaKw8tdKWzK+BPCswIA1vugILYbjaieCI2Xr2NqKr1qMGvh8pDlzJmCS8JPyDS8wHgNpu254UlmBqspgGr0yOyMotaOFTv3zq7D362TzCokhO9crS/4Bt6wRc3iOehaCJo5PTnueZiN9HtSO6dSs2nB1plkYuhyXN83cwvVd0OCiBGHRGQPN8PEGU1WHwBUsofuf8KNWvOdLCGknBBp+LbsQPI4gsh3vuOH6mvdZhvxTMSQV7St0+z1+LduKU+6UYQkzMGZ7MCBCpMCTylGfFDUCoOmOu2R9qE+8wXbvGMsgWIaNa2ZQuS5sQv1MbjnKRVcqSNPMfW9aA/EsDMHE4QcDDEfnoSqJNCKxla0MWWF0u0vEVv5mAIImUgwNLZRdSVPLBjFvFTcqZN7EVSChX+HrgX2WnoR9mHRyekaRqYIRopamgu8Vx/gLm8g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Also remove count_memcg_page_event now that its last caller no longer uses it, and convert hpage_collapse_alloc_page() to hpage_collapse_alloc_folio(), This removes 1 call to compound_head() and helps convert khugepaged to use folios throughout. Signed-off-by: Vishal Moola (Oracle) --- include/linux/memcontrol.h | 14 -------------- mm/khugepaged.c | 15 ++++++++------- 2 files changed, 8 insertions(+), 21 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index ab94ad4597d0..3126bde982e8 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -1080,15 +1080,6 @@ static inline void count_memcg_events(struct mem_cgroup *memcg, local_irq_restore(flags); } -static inline void count_memcg_page_event(struct page *page, - enum vm_event_item idx) -{ - struct mem_cgroup *memcg = page_memcg(page); - - if (memcg) - count_memcg_events(memcg, idx, 1); -} - static inline void count_memcg_folio_events(struct folio *folio, enum vm_event_item idx, unsigned long nr) { @@ -1565,11 +1556,6 @@ static inline void __count_memcg_events(struct mem_cgroup *memcg, { } -static inline void count_memcg_page_event(struct page *page, - int idx) -{ -} - static inline void count_memcg_folio_events(struct folio *folio, enum vm_event_item idx, unsigned long nr) { diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 9efd8ff68f06..d49aa22d99c9 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -888,16 +888,16 @@ static int hpage_collapse_find_target_node(struct collapse_control *cc) } #endif -static bool hpage_collapse_alloc_page(struct page **hpage, gfp_t gfp, int node, +static bool hpage_collapse_alloc_folio(struct folio **folio, gfp_t gfp, int node, nodemask_t *nmask) { - *hpage = __alloc_pages(gfp, HPAGE_PMD_ORDER, node, nmask); - if (unlikely(!*hpage)) { + *folio = __folio_alloc(gfp, HPAGE_PMD_ORDER, node, nmask); + + if (unlikely(!*folio)) { count_vm_event(THP_COLLAPSE_ALLOC_FAILED); return false; } - folio_prep_large_rmappable((struct folio *)*hpage); count_vm_event(THP_COLLAPSE_ALLOC); return true; } @@ -1064,17 +1064,18 @@ static int alloc_charge_hpage(struct page **hpage, struct mm_struct *mm, int node = hpage_collapse_find_target_node(cc); struct folio *folio; - if (!hpage_collapse_alloc_page(hpage, gfp, node, &cc->alloc_nmask)) + if (!hpage_collapse_alloc_folio(&folio, gfp, node, &cc->alloc_nmask)) return SCAN_ALLOC_HUGE_PAGE_FAIL; - folio = page_folio(*hpage); if (unlikely(mem_cgroup_charge(folio, mm, gfp))) { folio_put(folio); *hpage = NULL; return SCAN_CGROUP_CHARGE_FAIL; } - count_memcg_page_event(*hpage, THP_COLLAPSE_ALLOC); + count_memcg_folio_events(folio, THP_COLLAPSE_ALLOC, 1); + + *hpage = folio_page(folio, 0); return SCAN_SUCCEED; } From patchwork Wed Oct 18 20:32:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vishal Moola X-Patchwork-Id: 13427844 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 149C9CDB482 for ; Wed, 18 Oct 2023 20:33:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B921B80020; Wed, 18 Oct 2023 16:33:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AF1528D0016; Wed, 18 Oct 2023 16:33:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 942EE80020; Wed, 18 Oct 2023 16:33:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 7F00F8D0016 for ; Wed, 18 Oct 2023 16:33:14 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 59EA4A0D92 for ; Wed, 18 Oct 2023 20:33:14 +0000 (UTC) X-FDA: 81359731908.23.95CD5D2 Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) by imf01.hostedemail.com (Postfix) with ESMTP id 79FB540024 for ; Wed, 18 Oct 2023 20:33:12 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=KUqsdhvw; spf=pass (imf01.hostedemail.com: domain of vishal.moola@gmail.com designates 209.85.214.173 as permitted sender) smtp.mailfrom=vishal.moola@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1697661192; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=AcNlnHmZuXPr4rVPOwy7uUYyAYPeSLuYyJXFxeMfebs=; b=g1Dd3eGBi/A82pFyTMQJ7v6Jpwbs4XavVaVmAKvBLZR0bmHHg5Dj7OyOPt+H8GtQk1o7Dy Xx6gr4T2sVq7mbVL+IiFeJqKv6vt+f+X7vieeLYYEAOuFcTcjNeiLI9oV413U62wEMVmbt lbOftACnTH5ebozSIL2LHZuT8/QpoMo= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=KUqsdhvw; spf=pass (imf01.hostedemail.com: domain of vishal.moola@gmail.com designates 209.85.214.173 as permitted sender) smtp.mailfrom=vishal.moola@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1697661192; a=rsa-sha256; cv=none; b=asloCwf4xB/+z5I85aNnCbvPT1UNtCZNQB5ap2Ds/D6UceJ1OaVO+yB5QWxOzbEmf69hz0 wrVTrGmVNARaKM3Fn833y40IyTLJ05IK4U13bC8vN0MGHns7xmEPIMQ8nKHJ+G/UsTuLEt /COhXZue/2VKKS2MCYMEHLUMMAS+QEs= Received: by mail-pl1-f173.google.com with SMTP id d9443c01a7336-1c9e95aa02dso776355ad.0 for ; Wed, 18 Oct 2023 13:33:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1697661191; x=1698265991; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=AcNlnHmZuXPr4rVPOwy7uUYyAYPeSLuYyJXFxeMfebs=; b=KUqsdhvwlbZNq4yI1oN1btXYWwSgVSA2HdtpmPYsTqtMP8NclIpzB4OEQiMPE8deNf HEiTRCwgTYGdcjTU0rJESrYQPYAeKFkZlYSxO/Gb8r300HPHAeTt2t/ouWA9o9v3b/Cu 2PrkGA6c9Z9hyFZeAIRRXnPToFeqgSpBZz0/rukjnBIcRH36uIxBOyD80IpdXHalIEhJ ZGzQ/rpt6d/gnp3mHqMSZOduOpXSow8/bVAvEEOrmLvQ2BCQnzNyN8hsdox3Zj4DoPpN HXIsrwQrj6mCJ4LFETkDDRX/Z3rri9S7VEwPc5LttmC8GWaIWBPDbNQVZg676eSyY/ku 8nXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697661191; x=1698265991; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AcNlnHmZuXPr4rVPOwy7uUYyAYPeSLuYyJXFxeMfebs=; b=n9nwCAOcarHNtk3PuO3mJjftA8l8cuVtJxBwYOp6R4xU75XweDLRKczy4uhuj8JP6P dn6d113GhgP/FeOlBEQ22wCS8Ln8Qgm2bwiEopYoaHmo16gJ0sd57b6Nv9G20G5U1Bf2 7KnfPk4HI9cYaF8X8jCPbSP/9SKb9jv0/2ImmZ/zlBwll4iWSwgv0qI04U5tuics7JvE /+nVlNXZC2h9yYa8ZKvLlFWC5IAg52fZMp3P/BdBGBX2V2sG2loDNqsN/JiigIWiMvOT B1DPYsaw7yNUXtwuhjOWATUAVaByELZqdGLks0UlhGn3neYlIPCJIEAyoAZEqj7G3yh0 8MlA== X-Gm-Message-State: AOJu0YzWY1ZmfCgg+faUKjmDMmALgRY51Hh17J5iZzAFtCGa03suVDtq Zp5ENGC8+gfp6r7x0fwj7tN+qTOUpJ0= X-Google-Smtp-Source: AGHT+IEf7Gnz9O9WMz3CQOd+A4MjLM6TTZK4jiZl9DPWEVFFAHxMsHTxdrbBpGMTCyZdudHzKKxBQw== X-Received: by 2002:a17:902:e5cb:b0:1c6:e1d:8be0 with SMTP id u11-20020a170902e5cb00b001c60e1d8be0mr677167plf.2.1697661191282; Wed, 18 Oct 2023 13:33:11 -0700 (PDT) Received: from fedora.. (c-73-170-51-167.hsd1.ca.comcast.net. [73.170.51.167]) by smtp.googlemail.com with ESMTPSA id w12-20020a170902d3cc00b001c61e628e9dsm354412plb.77.2023.10.18.13.33.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Oct 2023 13:33:10 -0700 (PDT) From: "Vishal Moola (Oracle)" To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, akpm@linux-foundation.org, shy828301@gmail.com, "Vishal Moola (Oracle)" Subject: [PATCH v2 5/5] mm/khugepaged: Convert collapse_pte_mapped_thp() to use folios Date: Wed, 18 Oct 2023 13:32:13 -0700 Message-Id: <20231018203213.50224-6-vishal.moola@gmail.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231018203213.50224-1-vishal.moola@gmail.com> References: <20231018203213.50224-1-vishal.moola@gmail.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 79FB540024 X-Rspam-User: X-Stat-Signature: a3duebz4rfgfeb9zm13fwo8qpt8q181h X-Rspamd-Server: rspam01 X-HE-Tag: 1697661192-363886 X-HE-Meta: U2FsdGVkX1/dnQX7kl9NwNCvX45Ho+vtCI8vzEXGkk4uBE/KXV32wGnMgsCv0rFo0EtxDYlswziN0rTJIm8YCgW68WOTPB6fwcqT1w78DAck4YmeQwEgjkY4+91uqTEP61uLBq4n/Uw2sLRKupfA1xwlTVKi2xke6mm13ZLeVtBLb7EZZf6SfXfytj4fSmU+7bd37QfV2KIgsQCevtEwomYEVrh80oCjnlPHkoB5OGsIUzgtPhtS4qbLxRHwU9YVU8FrVamtkWXQZnzT/sAUF7ceASXtbz7oK5Z0EWxjhbHt3eH0tLQcovNcg8Yi0wFlO+J8c1r81VaipZ+1tlNQAPJ226cq5EXq20PaWDgm8S/DI6mW4usy9pso92hDFmbc40RfCIlQnuI5PJFVvsf35ey5ocumHo7ITYZ2W0CY21vUvAJasbH//E0Q5dkI5+8NZGXzkVsaPqxsmwtZo0tKw5+VOJl7XkqU1vLCeqVEs+SGgByH5joTlRl+nNucmJDTZ/X5qmW/GTIiCPbaeHokKAvvq0J8cRSXP5XI+59dy9qaW3FZMkLCApP1PoCD1PCxYv3IeCaeVVl68l8Ejnz72uXEd/sHM70KjtZDZ/a1ct3Fz4GfvYpRp3VXFjBqxthV5k4/dBzUxJ6D138j82GiuiNuQB33IyTdtZ2BIRKs51RFfdgaiCMo2Y8Nonp5enRBDy4Z9QjnAqM830FGJxxqo7H0JFfaz/iGDKyi77jrJOMFzlqT6ZCz+LO6kg9uK6oDUZE6q25bS+6pMCaVJzqUngTEUWTw35n1VCDzCA2Ms04oXZSf+iFpWbyij+Z8CQL98Z/TUEEm3zEUsloQPXi357S0SorBoN+1Qvu9j4tPgRLuV0Cxu0bRtfiQrw6ZH+ZBIltTfPvqBbz8W0zAMP6wj9UOgTs7UaG59C7s6kz95dpruZpJCXilaTlu+peAyLGf7RN/Dvv7JstA3kJxOhJ m7QTpeTu RB3R0EkIqtQIPme3GE5QiTWh6b751ymPC3DHqAcw/gUWAMR359/Ewus3g8g4/iRb0jbkfvycrvudAP/7Zhfi/dPTda6d4YkG4BUwJD5xH2Eje3GHRgyg5U1kY8+gYxDzm5OAsYRKijIS3939zA8Av+PM5B5eUf5eqSr8xiE863u0lFu/op8gM/hQQY6iIJhilomyccrfwQmUyoVs90LnqrHL0+nfb5Geozoezc2NveUvSzLBzbc925l+/MI1CTj0jRNZrPgFnnz3Vnk7Wxz1UgWpQs9Pry5VQDBz7sRjr2gFyBpJFOaGJIufndln2q5tQOynXOWeXXyHohlOnJnEZhWrqSxyD22zDl3MtFGbDAiZE2b+pDhhr51ov+3F+Dxzd/HNC7G0dQjqmqIC+MGm+FCPAVCRAySgHj7K7ZvGlBUZFQB6JIfVlNrqVmkzNU1dOS9iqY0mQGeolvxvRTfmunawKT6jT3sIadDB5OW5WV0MD19grbxPLMlIbYy/uX5V+eLeYAGE+TYDBmQpJJdMYYtq50ygyMoSUMAJRsLF1QMWJH6wA5VNgJng2inaaE3brX8wpPjwjaXlN5+kq1Cvjilt2gA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This removes 2 calls to compound_head() and helps convert khugepaged to use folios throughout. Previously, if the address passed to collapse_pte_mapped_thp() corresponded to a tail page, the scan would fail immediately. Using filemap_lock_folio() we can get the corresponding folio back and try to operate on the folio instead. Signed-off-by: Vishal Moola (Oracle) Signed-off-by: Vishal Moola (Oracle) --- mm/khugepaged.c | 45 ++++++++++++++++++++------------------------- 1 file changed, 20 insertions(+), 25 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index d49aa22d99c9..94c1dd09a8a6 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1475,7 +1475,7 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, bool notified = false; unsigned long haddr = addr & HPAGE_PMD_MASK; struct vm_area_struct *vma = vma_lookup(mm, haddr); - struct page *hpage; + struct folio *folio; pte_t *start_pte, *pte; pmd_t *pmd, pgt_pmd; spinlock_t *pml = NULL, *ptl; @@ -1508,19 +1508,14 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, if (userfaultfd_wp(vma)) return SCAN_PTE_UFFD_WP; - hpage = find_lock_page(vma->vm_file->f_mapping, + folio = filemap_lock_folio(vma->vm_file->f_mapping, linear_page_index(vma, haddr)); - if (!hpage) + if (!folio) return SCAN_PAGE_NULL; - if (!PageHead(hpage)) { - result = SCAN_FAIL; - goto drop_hpage; - } - - if (compound_order(hpage) != HPAGE_PMD_ORDER) { + if (folio_order(folio) != HPAGE_PMD_ORDER) { result = SCAN_PAGE_COMPOUND; - goto drop_hpage; + goto drop_folio; } result = find_pmd_or_thp_or_none(mm, haddr, &pmd); @@ -1534,13 +1529,13 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, */ goto maybe_install_pmd; default: - goto drop_hpage; + goto drop_folio; } result = SCAN_FAIL; start_pte = pte_offset_map_lock(mm, pmd, haddr, &ptl); if (!start_pte) /* mmap_lock + page lock should prevent this */ - goto drop_hpage; + goto drop_folio; /* step 1: check all mapped PTEs are to the right huge page */ for (i = 0, addr = haddr, pte = start_pte; @@ -1565,7 +1560,7 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, * Note that uprobe, debugger, or MAP_PRIVATE may change the * page table, but the new page will not be a subpage of hpage. */ - if (hpage + i != page) + if (folio_page(folio, i) != page) goto abort; } @@ -1580,7 +1575,7 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, * page_table_lock) ptl nests inside pml. The less time we hold pml, * the better; but userfaultfd's mfill_atomic_pte() on a private VMA * inserts a valid as-if-COWed PTE without even looking up page cache. - * So page lock of hpage does not protect from it, so we must not drop + * So page lock of folio does not protect from it, so we must not drop * ptl before pgt_pmd is removed, so uffd private needs pml taken now. */ if (userfaultfd_armed(vma) && !(vma->vm_flags & VM_SHARED)) @@ -1604,7 +1599,7 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, continue; /* * We dropped ptl after the first scan, to do the mmu_notifier: - * page lock stops more PTEs of the hpage being faulted in, but + * page lock stops more PTEs of the folio being faulted in, but * does not stop write faults COWing anon copies from existing * PTEs; and does not stop those being swapped out or migrated. */ @@ -1613,7 +1608,7 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, goto abort; } page = vm_normal_page(vma, addr, ptent); - if (hpage + i != page) + if (folio_page(folio, i) != page) goto abort; /* @@ -1632,8 +1627,8 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, /* step 3: set proper refcount and mm_counters. */ if (nr_ptes) { - page_ref_sub(hpage, nr_ptes); - add_mm_counter(mm, mm_counter_file(hpage), -nr_ptes); + folio_ref_sub(folio, nr_ptes); + add_mm_counter(mm, mm_counter_file(&folio->page), -nr_ptes); } /* step 4: remove empty page table */ @@ -1657,14 +1652,14 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, maybe_install_pmd: /* step 5: install pmd entry */ result = install_pmd - ? set_huge_pmd(vma, haddr, pmd, hpage) + ? set_huge_pmd(vma, haddr, pmd, &folio->page) : SCAN_SUCCEED; - goto drop_hpage; + goto drop_folio; abort: if (nr_ptes) { flush_tlb_mm(mm); - page_ref_sub(hpage, nr_ptes); - add_mm_counter(mm, mm_counter_file(hpage), -nr_ptes); + folio_ref_sub(folio, nr_ptes); + add_mm_counter(mm, mm_counter_file(&folio->page), -nr_ptes); } if (start_pte) pte_unmap_unlock(start_pte, ptl); @@ -1672,9 +1667,9 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, spin_unlock(pml); if (notified) mmu_notifier_invalidate_range_end(&range); -drop_hpage: - unlock_page(hpage); - put_page(hpage); +drop_folio: + folio_unlock(folio); + folio_put(folio); return result; }