From patchwork Tue Aug 22 16:28:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13361201 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9BD98EE4996 for ; Tue, 22 Aug 2023 16:28:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9C70194001B; Tue, 22 Aug 2023 12:28:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 97735280049; Tue, 22 Aug 2023 12:28:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 83E8794002D; Tue, 22 Aug 2023 12:28:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 76BB694001B for ; Tue, 22 Aug 2023 12:28:22 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 593C11C91F1 for ; Tue, 22 Aug 2023 16:28:22 +0000 (UTC) X-FDA: 81152273244.14.C8857A1 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf05.hostedemail.com (Postfix) with ESMTP id BAC9F10001B for ; Tue, 22 Aug 2023 16:28:20 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=upHYpxGi; spf=none (imf05.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1692721700; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=fikYGvhW/M2pb63cjdLz202Dx7trTh2hg6lPyaOt2gA=; b=pPyQPp5464ckr5Sw0TmH7QfKWG9g3QUM4otFnBBll1ftYWvBfW/zNNqc+TqfxSdHQ6fWhz Wt7ZiQYD1nsSBcG6dSjWVXWfcHUqGLh/rFoRX9Elg6zB18PvKpTvoNxHEG6RvRY/k07ji7 9UujcAqgf//3bUNzbOWWKL6OJoVyL1o= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=upHYpxGi; spf=none (imf05.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1692721700; a=rsa-sha256; cv=none; b=J6AK1Bkd2tIaMT1FQeI5op8Ko3Xibui4K6Tr3r3LRevGKdYZSgIL3KtmQEV/rz+WV2agrM qkJ/Ffz23z5E6QvQQmqGfsnbih5Be5fDFPpym6YGUj+IfHyJa9hZfTVcj7XETADCv0GWUQ LhdKMZCr+7xrXGJzvcX40ovp9NVl8Z8= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=fikYGvhW/M2pb63cjdLz202Dx7trTh2hg6lPyaOt2gA=; b=upHYpxGiQZeXbrprfEOdwG+T0s /wOpvkhUj+N050c8r3jimHnSTjzGHFLD1sGHtT0VjOsfCDf4sukp3pNcvzoAG0+lXYMcM/iFCTZIc iekm5/Z/cO4RY2+i7+asZJxExDF/XwbTJeQZw32HuE5hTEyWW85ir+xMkdHWPYfDR7USk4JXF1mVH p/OKEDOl2QVy8Kh5jDQ0UMMWql0RACAChyuOTEsluM1JFfwtfoUAuJY3lTBspK8/CqUpujtCYWJAW 667xJVK1O4QDItLgQE5xm3hPw9dZCUVtIyQEqkdXDDTeKXfjy8XjGsHIycDU33+QdyX/cKB5NRHMA a1HHvS+w==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1qYUEk-00HKlh-LF; Tue, 22 Aug 2023 16:28:10 +0000 From: "Matthew Wilcox (Oracle)" To: Mike Kravetz Cc: "Matthew Wilcox (Oracle)" , Muchun Song , linux-mm@kvack.org, Sidhartha Kumar Subject: [PATCH 2/3] hugetlb: Remove a few calls to page_folio() Date: Tue, 22 Aug 2023 17:28:06 +0100 Message-Id: <20230822162808.4131399-2-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20230822162808.4131399-1-willy@infradead.org> References: <20230822162808.4131399-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: BAC9F10001B X-Rspam-User: X-Stat-Signature: j4jifpxmt3hwtr8r4aw5i61775h3a868 X-Rspamd-Server: rspam01 X-HE-Tag: 1692721700-704991 X-HE-Meta: U2FsdGVkX1/x30xmuNA+VelV5u6OOl758WC1FR/WHAJBRxzc+IjHmlif/rCeK3UvufRAZhOzN/cl9mC+nOYMoS/OBZ3/Hd984Ok8c/pcckFZXezxcnEoe0Q6tKgj5kHFc8LhOA+IiG/xjLFZUUiG9Opt4DGvRc8d8kbYft1UBZQWYtjb/2Ec5BmIm01ouiaCvEP3jIJwx2N9hCyzll07XGpOXY4zGoJYtHJKdRy28fxslRE+pZj5BvNpxUaxVlqkBMeqk434AAEP83mTywEbsQIZoaa/ZX2RlMCQVh/HN8W7f9EOuMIawzgro1uyzgxByrM+Z/xSX4HHDmHRq9pRGT/w9pgZ9PsN/a67N0XkcCGhwxEkBldNH+LFQmqne/GghmYNZZFZqt4gz2wYlAdB53QHkql8q/uNuCAgkUKqrDpWLYv8BlIubcFBNj8HXVFxLd186cLQ9U/ju+9HP+7+n4BL+k8qpckZKYr5qCB7+frZSgvv67XSyY5QB06fdl/jxkFcTTe/aDlwobn38vhwtTjOSnTn46BIa02BzBY9a03PlR3ns9g23bkKgPPJ9jrhOb4qKE7GHdhYvnVavjXLbTfgV8nQW3uBUtyx60rwoJrE/i/+K8PM+/2F14v3lyNq/NXelzJkvfjDDilQp8RlVVBFl4qiDq+7dSqIiilfQSByJkT7LCcTxjP8MfrHvVEN446Q0JO/Si75h0/0EHvd3e7gGS+9bP6J3JNrx6RL3cETZi15x551ZWvggOrjgKW1UXj3iFP41kLBzF4j+bNL3bY6UratooAnNXSgd4t3c/q9KelsPV4LVX6C/ycEBq7dPwd/j4KwnrCgm6U/RsRi6dyYor1boq5W+yIlBJ+t5fxqLyCAaKf4mRG/0w9OEY8nb4uZLHJqjaiJpDQ0yFas2jkQQJIF6iwDsxuMoset0zdFWhyU9ew2V+cZKcA6VjvtwvvFXHw3+23ghIwC+kN Vc2Vg1hn PM3PRyrMWxcDi7jzsVYiC7/2V1z/1nzhddCsX+jsXhQEy9TCnqov+KeFGsguGn28QHcsK7eMv6JM2aKA3T3YmUkvD8Up35tJya6XjJqrtt/1iFEE5eAYBSIbs+IsVhbq7NVewuEC/MMEbB14HDju61JYDRkSbh6kB5LNo87Q8mTcjqYzIrjU5P8NQbpHeDsfgBBwSxgeXLsi2crjEhfoZU1G9SkCGRzITGBKAmLJaZhjI2jTaTywWxRn/9Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Anything found on a linked list threaded through ->lru is guaranteed to be a folio as the compound_head found in a tail page overlaps the ->lru member of struct page. So we can pull folios directly off these lists no matter whether pages or folios were added to the list. Signed-off-by: Matthew Wilcox (Oracle) Cc: Sidhartha Kumar Reviewed-by: Mike Kravetz Reviewed-by: Muchun Song --- mm/hugetlb.c | 24 ++++++++++-------------- 1 file changed, 10 insertions(+), 14 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index d1c856628bac..d6309edb59e5 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -1835,11 +1835,9 @@ static void update_and_free_hugetlb_folio(struct hstate *h, struct folio *folio, static void update_and_free_pages_bulk(struct hstate *h, struct list_head *list) { - struct page *page, *t_page; - struct folio *folio; + struct folio *folio, *t_folio; - list_for_each_entry_safe(page, t_page, list, lru) { - folio = page_folio(page); + list_for_each_entry_safe(folio, t_folio, list, lru) { update_and_free_hugetlb_folio(h, folio, false); cond_resched(); } @@ -2228,7 +2226,6 @@ static struct page *remove_pool_huge_page(struct hstate *h, bool acct_surplus) { int nr_nodes, node; - struct page *page = NULL; struct folio *folio; lockdep_assert_held(&hugetlb_lock); @@ -2239,15 +2236,14 @@ static struct page *remove_pool_huge_page(struct hstate *h, */ if ((!acct_surplus || h->surplus_huge_pages_node[node]) && !list_empty(&h->hugepage_freelists[node])) { - page = list_entry(h->hugepage_freelists[node].next, - struct page, lru); - folio = page_folio(page); + folio = list_entry(h->hugepage_freelists[node].next, + struct folio, lru); remove_hugetlb_folio(h, folio, acct_surplus); break; } } - return page; + return &folio->page; } /* @@ -3363,15 +3359,15 @@ static void try_to_free_low(struct hstate *h, unsigned long count, * Collect pages to be freed on a list, and free after dropping lock */ for_each_node_mask(i, *nodes_allowed) { - struct page *page, *next; + struct folio *folio, *next; struct list_head *freel = &h->hugepage_freelists[i]; - list_for_each_entry_safe(page, next, freel, lru) { + list_for_each_entry_safe(folio, next, freel, lru) { if (count >= h->nr_huge_pages) goto out; - if (PageHighMem(page)) + if (folio_test_highmem(folio)) continue; - remove_hugetlb_folio(h, page_folio(page), false); - list_add(&page->lru, &page_list); + remove_hugetlb_folio(h, folio, false); + list_add(&folio->lru, &page_list); } }