From patchwork Tue Oct 1 07:58:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yunsheng Lin X-Patchwork-Id: 13895549 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6DA75E7716C for ; Thu, 5 Dec 2024 15:22:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3B2D16B0113; Thu, 5 Dec 2024 10:19:19 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id F417C6B012A; Thu, 5 Dec 2024 10:19:14 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DE1536B00B6; Thu, 5 Dec 2024 10:19:09 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 872E7280036 for ; Tue, 1 Oct 2024 03:59:52 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 0F2C6A0E7E for ; Tue, 1 Oct 2024 07:59:52 +0000 (UTC) X-FDA: 82624284624.30.6DF801E Received: from mail-pg1-f196.google.com (mail-pg1-f196.google.com [209.85.215.196]) by imf11.hostedemail.com (Postfix) with ESMTP id 285CA40016 for ; Tue, 1 Oct 2024 07:59:49 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=j+S2jJQa; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf11.hostedemail.com: domain of yunshenglin0825@gmail.com designates 209.85.215.196 as permitted sender) smtp.mailfrom=yunshenglin0825@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727769569; a=rsa-sha256; cv=none; b=qeL6Yt45TgRNgquSQZ6sYfoTp8YzNNcfC3GEshopVjA7UMR7aLxYHK2JX04f/RXQr0LnZZ oacqjubVPC+6oH+9/g2RWZ83jKeI6gsCEw4iq7p9sElhgaKb7azNTYLLQY/C5W8avxgWmn iNLmOzmlECXSkNblnsZHITrV8ZnkgL8= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=j+S2jJQa; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf11.hostedemail.com: domain of yunshenglin0825@gmail.com designates 209.85.215.196 as permitted sender) smtp.mailfrom=yunshenglin0825@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727769569; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=gd1RgN1SUA8afyUXxZzVZIDkcTWdN6ZwdQZ6zZRg9I0=; b=xKEmRsqZZzKsjLxbjR8SUJg6dZzmYMck9yOA7Xo25WO3nMuMXzlbzxNLRIGcE6Aou0g9Qp st8xPud72cP/KMF2/cxvuVWhRfwxuIgMWiHAillJPzxAz/XIPUa14R4kKjZ1e/orzhfpxW lERrFcgt+f0tZSoaYWyzG4hOc6PgXVE= Received: by mail-pg1-f196.google.com with SMTP id 41be03b00d2f7-7d50e7a3652so3502790a12.3 for ; Tue, 01 Oct 2024 00:59:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1727769589; x=1728374389; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=gd1RgN1SUA8afyUXxZzVZIDkcTWdN6ZwdQZ6zZRg9I0=; b=j+S2jJQaQyMwYxWDr8OfrPYXk/Wo2JpjVhxLZ4WV+GoP6UPnNuoCQ29huf9s4P39DI bwDEVGcTiTYdyzJKYnXVjc8zaNXpEXC9JUZjtQbU7G9qDwCB/WYI5/hTu8yFUkR7ToT/ JM1mfgtHF+kXrQYBa8a1mN+uRFv7sLhozIqKznuP18/JuCOWOQUdqakWhuCLjIfhGX1E /82PMbzEU2WidT7WtwkeFHm0AbClSJnUarwJuyMIJs5n3dT9dQgJlib097ZvBRnmcHY4 w6Ixq9j8rPcsywCBWSHAMKYDM1vybQRt+jW94bKHpXu19D/7MLE8rJ+hCfPL5DvLS+MY sCJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727769589; x=1728374389; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=gd1RgN1SUA8afyUXxZzVZIDkcTWdN6ZwdQZ6zZRg9I0=; b=OFS3TF9orNNlqWyb59xzIUlu4Wx8qkTAYNpeZq9GGPdArqO2miY4NhVTaczKJN+ACt usIaIitu22QBF4Ap+zZuoGkEyC1XRFgcumk4fdb3mW4TRih0KPOnqdb4Y7zUTkFfHGW0 OUAo/e6L7TVooIiVgmMX78OVDLyCWgNVnxQXkJ+9pnAugf2csiekDaaRirdCzKFBnp4o jAiy53VEU5/j2Z7qSlXLVcn12wKwuS7gTgp6zfHGeLE0XReqHG7xNuvmgkVymnHahjps a/55GF3wfNLC7aZrKrhvG58s4xLDE39p/Rc7lmLJQTJDjcoh0YGonlqfCBxF1smuLUEu Q44A== X-Forwarded-Encrypted: i=1; AJvYcCUaPP7elz7JkAKs64euZrKyMm2u1jMzhk6yNmMFB3tvJ0WBdU5UAjIEIMZ4jKKW97lVHO2NQnthOg==@kvack.org X-Gm-Message-State: AOJu0YwOx9zGbi/BGg67LpeYjbvuo6ltFPHiiULyKAS17//PPHb8K8Ig NvcXKXbQEJA1EQ2qQrG1r8K7lkTJIcJp86MileO8xGXB9YKhVXrF X-Google-Smtp-Source: AGHT+IFZp+Gi8vE0ox+pNEB55COsRo/wNQjqM5glSXrin4ALoGI9vxZDRJ2kDHOmSEGDlwKVj4MDzw== X-Received: by 2002:a05:6a20:6f0a:b0:1cf:2843:f798 with SMTP id adf61e73a8af0-1d4fa7ecda0mr22889063637.47.1727769588834; Tue, 01 Oct 2024 00:59:48 -0700 (PDT) Received: from yunshenglin-MS-7549.. ([2409:8a55:301b:e120:88bd:a0fb:c6d6:c4a2]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2e06e16d6d2sm13168168a91.2.2024.10.01.00.59.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 01 Oct 2024 00:59:48 -0700 (PDT) From: Yunsheng Lin X-Google-Original-From: Yunsheng Lin To: davem@davemloft.net, kuba@kernel.org, pabeni@redhat.com Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Yunsheng Lin , Alexander Duyck , Andrew Morton , linux-mm@kvack.org Subject: [PATCH net-next v19 07/14] mm: page_frag: some minor refactoring before adding new API Date: Tue, 1 Oct 2024 15:58:50 +0800 Message-Id: <20241001075858.48936-8-linyunsheng@huawei.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241001075858.48936-1-linyunsheng@huawei.com> References: <20241001075858.48936-1-linyunsheng@huawei.com> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: n3sszgo14ji6fqmenqk8czxdmj3k1dnz X-Rspamd-Queue-Id: 285CA40016 X-Rspamd-Server: rspam02 X-HE-Tag: 1727769589-227439 X-HE-Meta: U2FsdGVkX19VlMnXaxR6u1VsI1PCyFqSqZnwPq67FJbSV5vp82Hv7WMFRUYnvodyHEDPBQJxHBR9+FBIKy1NNIMcIwl9mxmtMbb3CqEre/PfQ6gUdu61vELRp3ET9PtqMTXclo42lKHayCWN2X0f4sbEyQ0XfrjskgS2dHEXOrI6O4u1c8AdP8F0sK6rjjCt8AtHFEAMknDRyg0wYWUbX8BlXYEd2c4/yOHwzgbz66OL0VMvAp/ZAJ29cCpFYPac2GzgUO4WhIzsDw1lqVl5Yx0yrgUMocF3KHDsRO4cUXOKs6+PwI5eG/z3JkzMNENKINhqQP3zUI8XZP51OWJllDFARqKeG+/56Bs8rGDkTNS07GwpoLw7Y6s7c4x0GVGS4qowdLUqqmsM1INinqpPMvRu2LJv8c+9Br74fVP2Dp+kdvGtMa+RpvxJr9Gal7jJTrdxWIOgB6GFg9DJis8r3FOrnkLznaBbgYVCb9uffhjxlwCwhQr2YAdfDCXgXZaDnXvcoc9UIqsc/JMsgFX6QjZUYEPcph7kk85yg20DW4aJqcZmC9XigqdfLe8SY8iZk0bIytz1T0LUeNmEavhuUN3ARMlG6vZM6ETgny74kRermy8Xahj78pky1G4fEdpYXLvizRxvWvdBjvYAPhOEDtNVA7i5iYd5dXu49teXZ+jM2Ve5TUaN9RU1YnFaOGbblUqy1tJPnIxM5LZT8KwgwF9RJhC70Mow7nrdEL0eZ0sKc6EDpQ9DkUvLlT2KBub4GSPu2xr9VSpXyCncCOtMhuTGg4wRftATLc/LcUcDD8ckOrESXacz02N6L0gKbSljLNaitgROT5LcIEvzSi8AR0tox6P+sM+qrfbFLl/ZWNxG/nrMR1RNnoI97EmIeaPENqwREmxq8vSFdXfsIqkl3xnzKlEMbfoFB6G4v2AX7TtEn9QJTH9XKupgw6SKrUiMVvxTjxXpPDG9GssDkDx cz8cXeLa JWKAE/gMBuzjx3EtTSAIBJVdnqd1lAteAtiaTDw6Y/zoL+W482meK6aJtvHXCG8FRF/9S1gCnNciSC6UGi/bhAI+EFoEByJmYoKr387arkOPk66vmakXF9UAC3fAVC7RAkC2Cic2/tjnJ4ovzdYIbDdx2A3FeCqm01Gyfe3d5JWH5zs9VPTbBXs6qGn7EAo7joZPZbBCEjFjcC53ijAYUL9ly89j+uGSJrHdw3fLvm5e03ZGLNFxkTkMX/3FNgSiDHEeI9AtShfGAr0/rlws8AVs8n7joPO6dHq4lAKVhSNZkVOYS5upZO3PWKlrrirpu/K65W6a8+82W3EUZBOSnkGSPney6Ml+ILlQOO4buS59+SYMgmHjXd/zsbJyEnv0VOw+g8VKdUNacFe1SSCWisB+0mPr+YiNuySOgPQcw99GtwATdk/IVmW7pb8fvpoBzudStLphP1uy+s09YHhXqugt5CHfPw+Tdd+LWcBSK/BOTlTJgYEdmgUdpwXfcrN2GR4A4rOB25+yfS0RSBTwJjVgf1hxs562cWep19qLJ2nKwSwlUEh7ut+zqFWJ377dE366xL4Zz+1fcKhZXJqyV+RSqezZ9fjdTyjut X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Refactor common codes from __page_frag_alloc_va_align() to __page_frag_cache_prepare() and __page_frag_cache_commit(), so that the new API can make use of them. CC: Alexander Duyck Signed-off-by: Yunsheng Lin --- include/linux/page_frag_cache.h | 36 +++++++++++++++++++++++++++-- mm/page_frag_cache.c | 40 ++++++++++++++++++++++++++------- 2 files changed, 66 insertions(+), 10 deletions(-) diff --git a/include/linux/page_frag_cache.h b/include/linux/page_frag_cache.h index 75aaad6eaea2..b634e1338741 100644 --- a/include/linux/page_frag_cache.h +++ b/include/linux/page_frag_cache.h @@ -5,6 +5,7 @@ #include #include +#include #include #include @@ -41,8 +42,39 @@ static inline bool page_frag_cache_is_pfmemalloc(struct page_frag_cache *nc) void page_frag_cache_drain(struct page_frag_cache *nc); void __page_frag_cache_drain(struct page *page, unsigned int count); -void *__page_frag_alloc_align(struct page_frag_cache *nc, unsigned int fragsz, - gfp_t gfp_mask, unsigned int align_mask); +void *__page_frag_cache_prepare(struct page_frag_cache *nc, unsigned int fragsz, + struct page_frag *pfrag, gfp_t gfp_mask, + unsigned int align_mask); +unsigned int __page_frag_cache_commit_noref(struct page_frag_cache *nc, + struct page_frag *pfrag, + unsigned int used_sz); + +static inline unsigned int __page_frag_cache_commit(struct page_frag_cache *nc, + struct page_frag *pfrag, + unsigned int used_sz) +{ + VM_BUG_ON(!nc->pagecnt_bias); + nc->pagecnt_bias--; + + return __page_frag_cache_commit_noref(nc, pfrag, used_sz); +} + +static inline void *__page_frag_alloc_align(struct page_frag_cache *nc, + unsigned int fragsz, gfp_t gfp_mask, + unsigned int align_mask) +{ + struct page_frag page_frag; + void *va; + + va = __page_frag_cache_prepare(nc, fragsz, &page_frag, gfp_mask, + align_mask); + if (unlikely(!va)) + return NULL; + + __page_frag_cache_commit(nc, &page_frag, fragsz); + + return va; +} static inline void *page_frag_alloc_align(struct page_frag_cache *nc, unsigned int fragsz, gfp_t gfp_mask, diff --git a/mm/page_frag_cache.c b/mm/page_frag_cache.c index cf9375a81a64..6f6e47bbdc8d 100644 --- a/mm/page_frag_cache.c +++ b/mm/page_frag_cache.c @@ -95,9 +95,31 @@ void __page_frag_cache_drain(struct page *page, unsigned int count) } EXPORT_SYMBOL(__page_frag_cache_drain); -void *__page_frag_alloc_align(struct page_frag_cache *nc, - unsigned int fragsz, gfp_t gfp_mask, - unsigned int align_mask) +unsigned int __page_frag_cache_commit_noref(struct page_frag_cache *nc, + struct page_frag *pfrag, + unsigned int used_sz) +{ + unsigned int orig_offset; + + VM_BUG_ON(used_sz > pfrag->size); + VM_BUG_ON(pfrag->page != page_frag_encoded_page_ptr(nc->encoded_page)); + VM_BUG_ON(pfrag->offset + pfrag->size > + page_frag_cache_page_size(nc->encoded_page)); + + /* pfrag->offset might be bigger than the nc->offset due to alignment */ + VM_BUG_ON(nc->offset > pfrag->offset); + + orig_offset = nc->offset; + nc->offset = pfrag->offset + used_sz; + + /* Return true size back to caller considering the offset alignment */ + return nc->offset - orig_offset; +} +EXPORT_SYMBOL(__page_frag_cache_commit_noref); + +void *__page_frag_cache_prepare(struct page_frag_cache *nc, unsigned int fragsz, + struct page_frag *pfrag, gfp_t gfp_mask, + unsigned int align_mask) { unsigned long encoded_page = nc->encoded_page; unsigned int size, offset; @@ -119,6 +141,8 @@ void *__page_frag_alloc_align(struct page_frag_cache *nc, /* reset page count bias and offset to start of new frag */ nc->pagecnt_bias = PAGE_FRAG_CACHE_MAX_SIZE + 1; nc->offset = 0; + } else { + page = page_frag_encoded_page_ptr(encoded_page); } size = page_frag_cache_page_size(encoded_page); @@ -137,8 +161,6 @@ void *__page_frag_alloc_align(struct page_frag_cache *nc, return NULL; } - page = page_frag_encoded_page_ptr(encoded_page); - if (!page_ref_sub_and_test(page, nc->pagecnt_bias)) goto refill; @@ -153,15 +175,17 @@ void *__page_frag_alloc_align(struct page_frag_cache *nc, /* reset page count bias and offset to start of new frag */ nc->pagecnt_bias = PAGE_FRAG_CACHE_MAX_SIZE + 1; + nc->offset = 0; offset = 0; } - nc->pagecnt_bias--; - nc->offset = offset + fragsz; + pfrag->page = page; + pfrag->offset = offset; + pfrag->size = size - offset; return page_frag_encoded_page_address(encoded_page) + offset; } -EXPORT_SYMBOL(__page_frag_alloc_align); +EXPORT_SYMBOL(__page_frag_cache_prepare); /* * Frees a page fragment allocated out of either a compound or order 0 page.