From patchwork Thu Aug 29 16:56:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 13783455 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 208D3C87FC3 for ; Thu, 29 Aug 2024 16:57:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9D5716B008C; Thu, 29 Aug 2024 12:57:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9844B6B0092; Thu, 29 Aug 2024 12:57:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 84C3F6B0093; Thu, 29 Aug 2024 12:57:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 659E26B008C for ; Thu, 29 Aug 2024 12:57:15 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id E8B821C27BE for ; Thu, 29 Aug 2024 16:57:14 +0000 (UTC) X-FDA: 82505888388.06.DCF36F5 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf21.hostedemail.com (Postfix) with ESMTP id 4B7751C0006 for ; Thu, 29 Aug 2024 16:57:13 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=adc3hfGV; spf=pass (imf21.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724950588; a=rsa-sha256; cv=none; b=jwlcU0xqKLhaaro4lEyryt2ouyh6YBQf9G4XZjV0ONLOxDcoIZrVB/3s8Ljzn6b90DEuCQ //i4mV1czFGrto/ZADAoqrYF5mFcG1Vq00ltzK+tsn6bBzmAQrV9POSXVIR6yzNOReVP6P Ct/gDDWDkao13YE1arY0JrYI0W1NfqA= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=adc3hfGV; spf=pass (imf21.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724950588; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=lJMXjWc4+IBsQZT+SpDHJ06KBG+eQaVO9mpec+pSrAQ=; b=xnmdcPoto62rUvA35CxkP+oX3hu8obYtLcTgGoUECMcat6XYQpUZcQU9TOlJtuV3OQ5qSN mJsCuDsNMXz3+VKjyQI0QyvYtyGWCKj5ySiFVZrQS4mjRxJ3kEuJ9SyM0dAxHCZAocHwLF 8Doc+va6t+/9pWF4J1hevtJl2rheHYs= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1724950632; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lJMXjWc4+IBsQZT+SpDHJ06KBG+eQaVO9mpec+pSrAQ=; b=adc3hfGVjc3+OrxV9LbFw+4c1f8iRsc0fwHqBqpYjzNh2H1/Dp1o1z9QogEgj+hP9/hcbG k8CNZB9QnOtOgJ837KzpnjUGHKbnOkNcpf9dw2uO0xkAM97CArs2Q7G9v+CnN0e6uVBe8e LvMAiwCkPRFMCr99KR2/L1xs4xymH0o= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-688-deQlVP0ONhqTZhWhlLdllw-1; Thu, 29 Aug 2024 12:57:11 -0400 X-MC-Unique: deQlVP0ONhqTZhWhlLdllw-1 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 36A261954B00; Thu, 29 Aug 2024 16:57:08 +0000 (UTC) Received: from t14s.redhat.com (unknown [10.39.193.245]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id E03F91955F66; Thu, 29 Aug 2024 16:56:59 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, cgroups@vger.kernel.org, x86@kernel.org, linux-fsdevel@vger.kernel.org, David Hildenbrand , Andrew Morton , "Matthew Wilcox (Oracle)" , Tejun Heo , Zefan Li , Johannes Weiner , =?utf-8?q?Michal_Koutn=C3=BD?= , Jonathan Corbet , Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen Subject: [PATCH v1 02/17] mm: factor out large folio handling from folio_nr_pages() into folio_large_nr_pages() Date: Thu, 29 Aug 2024 18:56:05 +0200 Message-ID: <20240829165627.2256514-3-david@redhat.com> In-Reply-To: <20240829165627.2256514-1-david@redhat.com> References: <20240829165627.2256514-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 X-Stat-Signature: kr395j9zez3sby9pzczu34u3dk6umnim X-Rspamd-Queue-Id: 4B7751C0006 X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1724950633-649292 X-HE-Meta: U2FsdGVkX1+FASnFd/NC9HDyV9cC1zJRmf9dTa7xTplHxOec2HI5eI8RuCB9TUbO58EVpqzXPuw8yTh48qkP645EvwpvWIFcbiqOzZaF9DJi164jL2eJtUuOHDKaFrTNonQPMcEKqSB6hpi/OZTq95oqKxEKVYHR3eriDXPmZ+aeh1Vg8RGCYufFezwE0pz9RJBXZsrNI+szgf1rXSGd60B1CEuqI9BzTaES+eMG5QQUGIpHUW5W9sxd5I68P1BSN9tZ23h7Ik9KVlWmcRWaGurk9J8hzcdsQB/IFcHUoE0lkLaafhzOt1yD7FDGE9Vzoas96tyybO+uZClCftkScxF+n/t2uIlzHvixzgUn/FDAmxC1sFzzcFvuN7Gxt0/+enTddRADZ4mGsWebrPPeXmO/haEwerFKbREVDu25ZZnPGVmpGhPgV7YzIEN3H7lyEctY4egpwADoueMcG0jzONeLrY+9LUMgPoxDhKpS51hE8JYwKZFTaq2Tyala+gyOW9dd3gajxSFFJyappuB0uE27OrjO39SLUrmYBIMNP0mlrl61BWs8KKctGLF8SBgr/j5nLsrMTv9l1LZ22vqj+4UJT3wWJNHhGWnY048eyM7QM02OMnzH9U+vtcDALjDRiJsoatLVsi7vs2MF/o3WqWo1SQ94NgfyBy6JmQyryEe+fj2F9ykTJ20QT+3v5AFNK59n5FCkp4IghA3MPeSl93jx9ByP660h+effrMXQeTwfVOCwuWz9Vbdfq0hH3H1igGnHNgifHl/Qu01eL5burFEzsGmJ1RWEFVSJSaFfXZ/ov9KKiiJQ/ynMBq1nqz0sUfwiVa8XuysfO9sf+ocdW3GcbN5k57vb60/5nST61E0BaPLwy4jHedo/TvUoSyWUgtbU3q7l8PRz98qsGszhK/SGjejbxcn5ikYychM5vSsK82k3oFS6LSPyJOpO1Hl993zzmBxYkkdoGkyULF7 PN8kSK0z Ax0xi9SWMFxzKNqFkWQ1rXWmJ/nt3/D/zSwWnCQms0vwadT5pPHm7YhaR4+Qk04MgSaefAzxpBW6hCuzFNscqWIl3sq8EObxXoBQ5rM8cNJ5jVl18kVarIEbMqtYkbtloKkvUIDkhtHpQmN15zIeJtdsBR3gwZ7A8xtjdBntrHw3BlckzUphakE/R9N+fNkPF6Fn4PWQThRzFXD5d6cEePVQMtas5kbNyv8/AD5C1im0IRhghsyBbwOPXMcthukOOQRm4D7O1KJPwUQaXR594iU3nj/ElvWrtC9z/ZFlAewRTk2FrT8CZZ8Kj2OxAD0PPvFHT4AJPx61HwWJLUNvLOIO3APEZuwjYXeh1gzRf4UpvJ2ssYshHstRjGg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Let's factor it out into a simple helper function. This helper will also come in handy when working with code where we know that our folio is large. Make use of it in internal.h and mm.h, where applicable. While at it, let's consistently return a "long" value from all these similar functions. Note that we cannot use "unsigned int" (even though _folio_nr_pages is of that type), because it would break some callers that do stuff like "-folio_nr_pages()". Both "int" or "unsigned long" would work as well. Signed-off-by: David Hildenbrand Reviewed-by: Kirill A. Shutemov --- include/linux/mm.h | 27 ++++++++++++++------------- mm/internal.h | 2 +- 2 files changed, 15 insertions(+), 14 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 3c6270f87bdc3..fa8b6ce54235c 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1076,6 +1076,15 @@ static inline unsigned int folio_large_order(const struct folio *folio) return folio->_flags_1 & 0xff; } +static inline long folio_large_nr_pages(const struct folio *folio) +{ +#ifdef CONFIG_64BIT + return folio->_folio_nr_pages; +#else + return 1L << folio_large_order(folio); +#endif +} + /* * compound_order() can be called without holding a reference, which means * that niceties like page_folio() don't work. These callers should be @@ -2037,11 +2046,7 @@ static inline long folio_nr_pages(const struct folio *folio) { if (!folio_test_large(folio)) return 1; -#ifdef CONFIG_64BIT - return folio->_folio_nr_pages; -#else - return 1L << folio_large_order(folio); -#endif + return folio_large_nr_pages(folio); } /* Only hugetlbfs can allocate folios larger than MAX_ORDER */ @@ -2056,24 +2061,20 @@ static inline long folio_nr_pages(const struct folio *folio) * page. compound_nr() can be called on a tail page, and is defined to * return 1 in that case. */ -static inline unsigned long compound_nr(struct page *page) +static inline long compound_nr(struct page *page) { struct folio *folio = (struct folio *)page; if (!test_bit(PG_head, &folio->flags)) return 1; -#ifdef CONFIG_64BIT - return folio->_folio_nr_pages; -#else - return 1L << folio_large_order(folio); -#endif + return folio_large_nr_pages(folio); } /** * thp_nr_pages - The number of regular pages in this huge page. * @page: The head page of a huge page. */ -static inline int thp_nr_pages(struct page *page) +static inline long thp_nr_pages(struct page *page) { return folio_nr_pages((struct folio *)page); } @@ -2183,7 +2184,7 @@ static inline bool folio_likely_mapped_shared(struct folio *folio) return false; /* If any page is mapped more than once we treat it "mapped shared". */ - if (folio_entire_mapcount(folio) || mapcount > folio_nr_pages(folio)) + if (folio_entire_mapcount(folio) || mapcount > folio_large_nr_pages(folio)) return true; /* Let's guess based on the first subpage. */ diff --git a/mm/internal.h b/mm/internal.h index 44c8dec1f0d75..97d6b94429ebd 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -159,7 +159,7 @@ static inline int folio_pte_batch(struct folio *folio, unsigned long addr, pte_t *start_ptep, pte_t pte, int max_nr, fpb_t flags, bool *any_writable, bool *any_young, bool *any_dirty) { - unsigned long folio_end_pfn = folio_pfn(folio) + folio_nr_pages(folio); + unsigned long folio_end_pfn = folio_pfn(folio) + folio_large_nr_pages(folio); const pte_t *end_ptep = start_ptep + max_nr; pte_t expected_pte, *ptep; bool writable, young, dirty;