From patchwork Thu Mar 21 14:24:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Matthew Wilcox (Oracle)" X-Patchwork-Id: 13598778 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 75DD4C54E58 for ; Thu, 21 Mar 2024 14:25:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EFB2E6B0098; Thu, 21 Mar 2024 10:25:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E83E06B0099; Thu, 21 Mar 2024 10:25:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CB11F6B009A; Thu, 21 Mar 2024 10:25:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id B35A66B0098 for ; Thu, 21 Mar 2024 10:25:14 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 78ABB804BB for ; Thu, 21 Mar 2024 14:25:14 +0000 (UTC) X-FDA: 81921268548.01.E9470AC Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf15.hostedemail.com (Postfix) with ESMTP id D4350A001B for ; Thu, 21 Mar 2024 14:25:12 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=J4T7BUhH; spf=none (imf15.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711031112; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=H59Aoy3vN+vh0NMsD3DjW5sbwztCBqX+xLBKaHRsFkw=; b=nsrmAufzrZMgmfjdTGUvQSFFLEMrFsqPMoIjX0uWbNWIL/Nrm2uFdHELO8D3n2R5K5wxUx ZRKUrfvzFIjF4DStS7gw8aD/iRkRjyvfSviJH1EMWC8u69D3vHXPquEjxjVoSgmjeWOj1C X3XPCqY9Yx/q5j5QbRCYXFGSfabDfFI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711031112; a=rsa-sha256; cv=none; b=oJkquw92LIBzYx+yF2WXmVsgq26QAK8/SkzRRgLaWcZwmVj+lmzWTGNfWlRt5SI62tkFPX jghONjTYZIo/ozCWKxl6/zFQ7P9ijS3l23yxJFnPuCuVZvmF57ckJDEkHfSavHQW8edHkQ G8Dy811oPaBAp96w93HOzhfsmc01wPk= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=J4T7BUhH; spf=none (imf15.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=H59Aoy3vN+vh0NMsD3DjW5sbwztCBqX+xLBKaHRsFkw=; b=J4T7BUhH0G+A9Dl7iHQbYUR64L xfcfXa6qDJNyj5/LEDXd3O7dQAULqO7j0j76FCHQpZOECwB7wS87KlVcrAALEWaxKV6TxSNHEWgey 4xKuQ9o0LYynbztzkFbschADJTF9mO87MozrdmQTtMNsWWwxfKS81KyjFmIrnjXexymUXUvhU/gxX CrVSpVJkKXDsAjSbYyb4B1r7t63Q5THlU3GHGsOrLhetDeG7ICph3PItO6HzWcHnFmr/VhXeA76Bd eFjum8m8Fa7UB3ACu4h4D6xMsQFUFSihvSdJ/nSSbnFW0DPQD0UOKrbIzz8HMahjDOR5je98RMadj JGC0GwHg==; Received: from willy by casper.infradead.org with local (Exim 4.97.1 #2 (Red Hat Linux)) id 1rnJLf-00000006u46-3Xaa; Thu, 21 Mar 2024 14:24:51 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, David Hildenbrand , Vlastimil Babka , Miaohe Lin , Muchun Song , Oscar Salvador Subject: [PATCH 7/9] mm: Free up PG_slab Date: Thu, 21 Mar 2024 14:24:45 +0000 Message-ID: <20240321142448.1645400-8-willy@infradead.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240321142448.1645400-1-willy@infradead.org> References: <20240321142448.1645400-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: D4350A001B X-Rspam-User: X-Stat-Signature: p3gstd9praeykptnpko4h8ya7f936fgz X-Rspamd-Server: rspam03 X-HE-Tag: 1711031112-727641 X-HE-Meta: U2FsdGVkX1+XY8/PThLnZLUmaN1+2M67nrSF8EULcyQuni9jHCya0DcvXf23IrbAabvjgJTUUlNU1/6qm9gBKcON+O/eLlrxyRyIe/Hsq0gKZHjIwrQUForj4qa4BZPwW/IE1NkEc6f+igt+64WsNcX3uM2+BPAAU9rhr9kZIA1khpxki5fYUdTEGAFjPv3L3Km/vQYiE7V4E9L10Gsbtyl4nHypqS9LIT1HkLleRTHtQxRiUML/kXvCGenVBgOp17fLbeX3OI/sReUQoKQlSM9hqmVMcqIINvDC0NsdGv137IO1ewF8l7Y14rkMJqPvQsi48GmGbsVbUhxwHzQMHY02QFIeymBafojXS89uFCTYO4hg9Fi6RzFuTknGSnmkKgpw+Rls5urgx6PdvmuP5Qby5++Yyil+e1IW/BEOvAlohDN5TbjL5Z1z8FffYh0KfbqUpfuItSQ5A5OitXHR63BKJJLl7SN06m6Ic0MB+8TEAjxYOAN1lXurtsnH9IsWmhuzA71znQDUp0AwG32Dr7zAp7hObS1xjtq+J5q7qg8dHIs1adnZ0rLW4WzM+e/Uv494h+qiGZFgwqoH2GSscrCZ+W2Y5fVIZR54GnnJs5HYpZ3aOspJG2L5zmZ5/hJqIO8eMDV0FkVsBx0kE0Y3jeqo+LcahKhcdrHR2xWnJr9TmtW3rKwzezYCO+OnNKiqone+T7+Ea8KTrcHTV8NvGmyFSFZNeZwB4drH6COLvv/6ASv1RLe7GoUGauYR6/oujjlxIrXUvcFk+sS7Xsi/edfk7pX+nuI68K9/2kqPM6qN0p1utFsdK12S34VnPjcDxWjNs1cEWwVLOM5CwativVCSC0FK22XC4Pmf86K0BN3UrBiraizBiylRePWAUQE73iIf8aXSflaTsFHI3j83TchvuKs9aIOE1r0ULwkx5W/hTmQi9DmL6dYAvIwwDXmKSRtH1qNEVfsHYYTlZ2O v2BGg7ES N6i6yrPQKJGHDdcaOycM2j8vHLhwJ89lJGcqegpBeghkJu8OS7yzhr5kTva96VYnJ4178un8zQm0gpFIM1y23B6mcNWWx4UWT8ByUUH1Rwh1aaOs6AQa/t9/DJfDpbhJnpTXBuvGgHekPMuE8iVpkS3Pw+d63U03Gc9MJ+1/q+emYYXtJ4DwCDlLeiB+ktctUxp7qJ/F9Y7LVrOf05UHOlW3gcbIRFE2kTtl/ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Reclaim the Slab page flag by using a spare bit in PageType. We are perennially short of page flags for various purposes, and now that the original SLAB allocator has been retired, SLUB does not use the mapcount/page_type field. This lets us remove a number of special cases for ignoring mapcount on Slab pages. Signed-off-by: Matthew Wilcox (Oracle) Acked-by: Vlastimil Babka Acked-by: David Hildenbrand --- include/linux/page-flags.h | 21 +++++++++++++++++---- include/trace/events/mmflags.h | 2 +- mm/memory-failure.c | 9 --------- mm/slab.h | 2 +- 4 files changed, 19 insertions(+), 15 deletions(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index 94eb8a11a321..73e0b17c7728 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -109,7 +109,6 @@ enum pageflags { PG_active, PG_workingset, PG_error, - PG_slab, PG_owner_priv_1, /* Owner use. If pagecache, fs may use*/ PG_arch_1, PG_reserved, @@ -524,7 +523,6 @@ PAGEFLAG(Active, active, PF_HEAD) __CLEARPAGEFLAG(Active, active, PF_HEAD) TESTCLEARFLAG(Active, active, PF_HEAD) PAGEFLAG(Workingset, workingset, PF_HEAD) TESTCLEARFLAG(Workingset, workingset, PF_HEAD) -__PAGEFLAG(Slab, slab, PF_NO_TAIL) PAGEFLAG(Checked, checked, PF_NO_COMPOUND) /* Used by some filesystems */ /* Xen */ @@ -931,7 +929,7 @@ PAGEFLAG_FALSE(HasHWPoisoned, has_hwpoisoned) #endif /* - * For pages that are never mapped to userspace (and aren't PageSlab), + * For pages that are never mapped to userspace, * page_type may be used. Because it is initialised to -1, we invert the * sense of the bit, so __SetPageFoo *clears* the bit used for PageFoo, and * __ClearPageFoo *sets* the bit used for PageFoo. We reserve a few high and @@ -947,6 +945,7 @@ PAGEFLAG_FALSE(HasHWPoisoned, has_hwpoisoned) #define PG_table 0x00000200 #define PG_guard 0x00000400 #define PG_hugetlb 0x00000800 +#define PG_slab 0x00001000 #define PageType(page, flag) \ ((page->page_type & (PAGE_TYPE_BASE | flag)) == PAGE_TYPE_BASE) @@ -1041,6 +1040,20 @@ PAGE_TYPE_OPS(Table, table, pgtable) */ PAGE_TYPE_OPS(Guard, guard, guard) +FOLIO_TYPE_OPS(slab, slab) + +/** + * PageSlab - Determine if the page belongs to the slab allocator + * @page: The page to test. + * + * Context: Any context. + * Return: True for slab pages, false for any other kind of page. + */ +static inline bool PageSlab(const struct page *page) +{ + return folio_test_slab(page_folio(page)); +} + #ifdef CONFIG_HUGETLB_PAGE FOLIO_TYPE_OPS(hugetlb, hugetlb) #else @@ -1121,7 +1134,7 @@ static __always_inline void __ClearPageAnonExclusive(struct page *page) (1UL << PG_lru | 1UL << PG_locked | \ 1UL << PG_private | 1UL << PG_private_2 | \ 1UL << PG_writeback | 1UL << PG_reserved | \ - 1UL << PG_slab | 1UL << PG_active | \ + 1UL << PG_active | \ 1UL << PG_unevictable | __PG_MLOCKED | LRU_GEN_MASK) /* diff --git a/include/trace/events/mmflags.h b/include/trace/events/mmflags.h index d55e53ac91bd..e46d6e82765e 100644 --- a/include/trace/events/mmflags.h +++ b/include/trace/events/mmflags.h @@ -107,7 +107,6 @@ DEF_PAGEFLAG_NAME(lru), \ DEF_PAGEFLAG_NAME(active), \ DEF_PAGEFLAG_NAME(workingset), \ - DEF_PAGEFLAG_NAME(slab), \ DEF_PAGEFLAG_NAME(owner_priv_1), \ DEF_PAGEFLAG_NAME(arch_1), \ DEF_PAGEFLAG_NAME(reserved), \ @@ -135,6 +134,7 @@ IF_HAVE_PG_ARCH_X(arch_3) #define DEF_PAGETYPE_NAME(_name) { PG_##_name, __stringify(_name) } #define __def_pagetype_names \ + DEF_PAGETYPE_NAME(slab), \ DEF_PAGETYPE_NAME(hugetlb), \ DEF_PAGETYPE_NAME(offline), \ DEF_PAGETYPE_NAME(guard), \ diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 9349948f1abf..1cb41ba7870c 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -1239,7 +1239,6 @@ static int me_huge_page(struct page_state *ps, struct page *p) #define mlock (1UL << PG_mlocked) #define lru (1UL << PG_lru) #define head (1UL << PG_head) -#define slab (1UL << PG_slab) #define reserved (1UL << PG_reserved) static struct page_state error_states[] = { @@ -1249,13 +1248,6 @@ static struct page_state error_states[] = { * PG_buddy pages only make a small fraction of all free pages. */ - /* - * Could in theory check if slab page is free or if we can drop - * currently unused objects without touching them. But just - * treat it as standard kernel for now. - */ - { slab, slab, MF_MSG_SLAB, me_kernel }, - { head, head, MF_MSG_HUGE, me_huge_page }, { sc|dirty, sc|dirty, MF_MSG_DIRTY_SWAPCACHE, me_swapcache_dirty }, @@ -1282,7 +1274,6 @@ static struct page_state error_states[] = { #undef mlock #undef lru #undef head -#undef slab #undef reserved static void update_per_node_mf_stats(unsigned long pfn, diff --git a/mm/slab.h b/mm/slab.h index d2bc9b191222..457b15da2a6b 100644 --- a/mm/slab.h +++ b/mm/slab.h @@ -84,8 +84,8 @@ struct slab { }; struct rcu_head rcu_head; }; - unsigned int __unused; + unsigned int __page_type; atomic_t __page_refcount; #ifdef CONFIG_MEMCG unsigned long memcg_data;