From patchwork Mon Nov 25 21:01:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13885289 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9DA99D59D71 for ; Mon, 25 Nov 2024 21:18:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E8C826B0088; Mon, 25 Nov 2024 16:18:21 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E3CBA6B0089; Mon, 25 Nov 2024 16:18:21 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D2B466B008C; Mon, 25 Nov 2024 16:18:21 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id B57916B0088 for ; Mon, 25 Nov 2024 16:18:21 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 6A83EADBAA for ; Mon, 25 Nov 2024 21:18:21 +0000 (UTC) X-FDA: 82825880550.17.EE96E79 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf16.hostedemail.com (Postfix) with ESMTP id 9E30818000E for ; Mon, 25 Nov 2024 21:18:15 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=GquQtnfs; dmarc=none; spf=none (imf16.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1732569496; a=rsa-sha256; cv=none; b=uqYLA0BUafgNYNFXVuVmL5G4JiYppG8+fMNyRFKGCjmJjVgrVjmWZN7vnfwNqb2Gn/gFld SdUsPuxXzblY+RYaRZOQ4O+LAbf+B/CUTqOyK3X3EOUXhV659g/PXe/yRpAQQsuda/e4xD we/gjLLbGtu3ymgZJstSn2UTHgbj0MI= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=GquQtnfs; dmarc=none; spf=none (imf16.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1732569496; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=D7AjNZcJ7LYPBNAHKmq7G2riCYIe6jFfSqMnQ3ICvDw=; b=FlmLLUvI6qNXSdi99fluvccOgds5cDg+OumsRhVEmAekZwxmX7bgbriCT7Q9RGlGK9+Lww qpG+sE4IATYb/B7K2VZL4ExPcvhNbbipV4AkBdTVegjq9tsaMJ6cA632SugXT7DrZeT+7+ V9ltacBmpAdlpkuS3pip62UUnez1jgI= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=D7AjNZcJ7LYPBNAHKmq7G2riCYIe6jFfSqMnQ3ICvDw=; b=GquQtnfsOpt/2hGkvqfdBsClva aXlHaCBMK0+g1L33DmzhGNi8QjRtT5wh9O4m6016Qy1Usqgf+pdOikJ/vKJ25KXTLFBDWnsboc7mo PNOumX53WI10RU6yUIEZj5yeg2aYfKWes2P0wiFQbw5wdh9HH1Wt1p9YTPbtgiRmOXeo3s6UyL8j7 NuVO6Y9FdQkR85JG1djqr1X+RzTjLwxkMKAj/r+s01xpmQNh3fJ5C8pKVUWdKYeglLC4pbZUguaoG hkwEU81yCb+a9sZcnKKFPFBbnRGGaTZyoI28xSR1kiF0Re1RuxSmQzxPgw+y5rB/J2t5s7MCV8gH4 HTKcdROg==; Received: from willy by casper.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tFgDQ-0000000CUEz-2Pwq; Mon, 25 Nov 2024 21:01:52 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, David Hildenbrand , William Kucharski Subject: [PATCH v3 15/15] slab: Allocate frozen pages Date: Mon, 25 Nov 2024 21:01:47 +0000 Message-ID: <20241125210149.2976098-16-willy@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241125210149.2976098-1-willy@infradead.org> References: <20241125210149.2976098-1-willy@infradead.org> MIME-Version: 1.0 X-Stat-Signature: 3khr9ubehk36dy74hugerc6to5agpy9w X-Rspamd-Queue-Id: 9E30818000E X-Rspamd-Server: rspam08 X-Rspam-User: X-HE-Tag: 1732569495-17895 X-HE-Meta: U2FsdGVkX1+GIuxhV0sbd5ND7i/UOhtKAQs4W5eg/gqt8r66pcN41bXMQSqXt7pqFNniqf2+IDilqLYCeH0p1xrwqyTIx78dBrg3LAbISGxtzCAzryYDy1VYb4NMN2dc3Z88tW5xfpoqMfpl930/j1Y0NSoAxDGRzcBrqT2V47she57VBjr0I9JjgLD0QGnBGekQxNoQRFpMQrDe6n9eqN37dka3ofkKLqYEEzoNbt0TRLrVK9BpRYrm1x3XsjYWN02bjoAOoFHKDjMN4LpzQyLAXRZRN8Bm9tF/c4CXgogy0m20TQy/mqGnhOySeU5+RKfekBdSFT7X5iUb8DwHbnor4uCxeqrMNysvl3PRz3afkvmElviJxHEmsMD6ExKaxUvbv3LL9+Tu+NolAoWembUZrOkoSNKbLGl5WwLoRIjeX1oIYD/GPX7TOUW6fidklykOaIQ5QOokB84etCbghNC0VRv5rBrVyrdNt2p+Z6XoZAm3IKeacbbC4f/XbQZKHPFCl2Mj4d8U3pBkddAt0Lp45l0U4dlwSTDKJP4SNl2b4duBMZMGJVYcLbYLQxTX0BZfX7M11s62tALcyLJryMN+CESsKwSrtKBcJbpcIEXX3/9B0v0iAzjs6ed6K2334ycaaTgTb3/eTjGPWNfWIMQade1vTlHMfnPGLesU3czxWp/I1HuhmWCisvqqtxOF0GGf2A3cmsVmjFIfQMT5+3whzsRi57y3Np3g9ViPTVC3gJO2Gm3bQlwh/XVszSlcPkuTDPewZQ5vOrODFfj0L4XZg4fQVsP/0f15GNjbuKdNDawHo7fdgXQjf2RoU3MHu4y9Ud5WR1GzosGR1Bx13pYzkmEr3fDM47Pw/YkT4jiVyIf2wtY/WKs/kraLs97z4vh+lqFrpzdUXafYKSWAuXjLZBAvnQAjE3ehpehs+w5Bs2qxwXdS9B0enRcncCs5Bqw68duFLOz/zO+eVu4 uLJPC53H /peBPQp+xWJRW0W2hY60YFW+sI4U8769o3eUdZlUEe2gOhvPJjWQprbHJs5js8tgyU0ilgVzMbN7b6amvLay4dTKjHv4uS/hiPNj3axUgqLcqpbZUffaR3OIWsLhLK/kCXe7GMf0oyOJjQILNChfEsizB2ctU7KSqGyFs16f50MujRlWEHX2OAkywyNro0m31O3MZIzvr3Z8vpGFpkANj9exXFqwoAMtkYQ+R8FLD9p4IP8vchBAI65hTnkDKXs7iwhke X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Since slab does not use the page refcount, it can allocate and free frozen pages, saving one atomic operation per free. Reviewed-by: William Kucharski Signed-off-by: Matthew Wilcox (Oracle) --- mm/slub.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/mm/slub.c b/mm/slub.c index 19980419b176..3d0c1cd6bdf5 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -2405,9 +2405,9 @@ static inline struct slab *alloc_slab_page(gfp_t flags, int node, unsigned int order = oo_order(oo); if (node == NUMA_NO_NODE) - folio = (struct folio *)alloc_pages(flags, order); + folio = (struct folio *)alloc_frozen_pages(flags, order); else - folio = (struct folio *)__alloc_pages_node(node, flags, order); + folio = (struct folio *)__alloc_frozen_pages(flags, order, node, NULL); if (!folio) return NULL; @@ -2641,7 +2641,7 @@ static void __free_slab(struct kmem_cache *s, struct slab *slab) __folio_clear_slab(folio); mm_account_reclaimed_pages(pages); unaccount_slab(slab, order, s); - __free_pages(&folio->page, order); + free_frozen_pages(&folio->page, order); } static void rcu_free_slab(struct rcu_head *h)