From patchwork Thu Mar 28 13:38:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yunsheng Lin X-Patchwork-Id: 13608548 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 97BC5C54E64 for ; Thu, 28 Mar 2024 13:41:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F11986B0099; Thu, 28 Mar 2024 09:41:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EC2556B009A; Thu, 28 Mar 2024 09:41:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D141C6B009B; Thu, 28 Mar 2024 09:41:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id B2B296B0099 for ; Thu, 28 Mar 2024 09:41:06 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 5CD46C0A9D for ; Thu, 28 Mar 2024 13:41:06 +0000 (UTC) X-FDA: 81946558932.11.2F93D93 Received: from szxga04-in.huawei.com (szxga04-in.huawei.com [45.249.212.190]) by imf10.hostedemail.com (Postfix) with ESMTP id 10FAFC001F for ; Thu, 28 Mar 2024 13:41:03 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf10.hostedemail.com: domain of linyunsheng@huawei.com designates 45.249.212.190 as permitted sender) smtp.mailfrom=linyunsheng@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711633264; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yCYDp/XbyUd0jUC8G9YQwMvOnUflWEyDFLqBUw4b2iM=; b=LYnY+kaNfKMGY5WtXrqMqF8qSS58SakwEb+w4UeMCYzXInm7Jj/HrAtYuxe6KqjTqJO6yV O2ut6PbpM31zQlx8h5zrlULn4GmYIRcr2REcMvWAAFlY8Bst445rm6GeZdWEhOQ13IVmSn dfq+exADDxhGEdRJD2M+MAAVkYAAk90= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf10.hostedemail.com: domain of linyunsheng@huawei.com designates 45.249.212.190 as permitted sender) smtp.mailfrom=linyunsheng@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711633264; a=rsa-sha256; cv=none; b=Wd61ZGdcn96Z8KaMEZZ19hc0vFpNZ4L0P8c7uyVHofmwdec8FtW0Nb/6NlqHpnTD75dTcd h17MKr1Kb32hCqXN7dq0y7t26deiED3P9C+0EDOS+QL1aUzLn8hsPOZSoZSh2TrZ4zausc F5etKIjL16txQWNdioA75vmBxcR/L1o= Received: from mail.maildlp.com (unknown [172.19.163.44]) by szxga04-in.huawei.com (SkyGuard) with ESMTP id 4V54Pv4MbXz1xrhm; Thu, 28 Mar 2024 21:38:59 +0800 (CST) Received: from dggpemm500005.china.huawei.com (unknown [7.185.36.74]) by mail.maildlp.com (Postfix) with ESMTPS id 4BF871402CA; Thu, 28 Mar 2024 21:41:01 +0800 (CST) Received: from localhost.localdomain (10.69.192.56) by dggpemm500005.china.huawei.com (7.185.36.74) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.35; Thu, 28 Mar 2024 21:41:01 +0800 From: Yunsheng Lin To: , , CC: , , Yunsheng Lin , Andrew Morton , Subject: [PATCH RFC 09/10] mm: page_frag: introduce prepare/commit API for page_frag Date: Thu, 28 Mar 2024 21:38:38 +0800 Message-ID: <20240328133839.13620-10-linyunsheng@huawei.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20240328133839.13620-1-linyunsheng@huawei.com> References: <20240328133839.13620-1-linyunsheng@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.69.192.56] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm500005.china.huawei.com (7.185.36.74) X-Rspamd-Queue-Id: 10FAFC001F X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: m9jqfcd4qioof4cqdtoesqymjirp1d1e X-HE-Tag: 1711633263-319468 X-HE-Meta: U2FsdGVkX18ixDNt4KB0+U1xGsBilSHk4HLLDqpCNdwC6zn0KJ7vdoVuTsN7S39OupJnbzV/4aHAOoJA7GNJwbxZ4wnQzp3wR4fQtrlrrgQEoFW55f4GRwgBCSN8mMyeJEKD+0qRnDzmRTzSo6XfdaMlALCAiF5uZKJY7y2ynLcSWVHkqUm9vSsy27LP+kBZVYmPSjPf7l8cijsYIWFviaOL9uZ/NcZcf2mNcWRaO366uriDuh9EjKJc/OHn2yoX3aFKaTdFeNSy+FxwdYucWY+9ePIEJqFb1MNu1hm7shtu5L4hiD9iopErPVpj5yhPMmq1TE9kBiDM5ZzWufCMtk07eT7+eoa8/XeGOSCgJKWFBbCEaAX+689j9/rmpvZn8y2cjcEwo0O3DS7HWWvA46e+3Wb+ZfKRgFnwSMXDj+YdsG2qx6kr8VzRR43vB5PvUu2bMme0//iIV+xuNzAYvqqRvZzDx97sc6ku2O1IiTOjBaogMMAT3f6f9kgzWAkcA7Cui/4F8TO+RkV551suaqVv61dGKIcfeYh8Xu9GVMZ/q7e3MI40hsrHtCEa84QXcvcL9sTRwuP3/knvWqOUo5XkBaYBRNUdcKvE8cPdgRWveaJimQTdwQx1eiX2cwNgXkObppWt1prEdyKsAk4uow/DU9wib4EGzinDUkOV2EVtkAOr/Zy9SYAtWR1KiPEirzSSL19DhCYaPQYSg66eovbjjHhSAvqZ7sfKifAyfRKMcAgSpRDE7BgEJ0eDqet/4JUx9BiEdr0l/7r7i5JNIdEJ6HS4Y8hbvUVKNhz0fttg9t9K5Pi3sVWrzqoGbkWz9LGi/li6kDIC8EW4TOr9SJva41uMjYmjy8R3JmEEMGzKkUaisok7auXlNQVvm7Y2IThFxbNeAPg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: There are many use cases that need minimum memory in order for forward progressing, but can do better if there is more memory available. Currently skb_page_frag_refill() API is used to solve the above usecases, as mentioned in [1], its implementation is similar to the one in mm subsystem. To unify those two page_frag implementations, introduce a prepare API to ensure minimum memory is satisfied and return how much the actual memory is available to the caller. And the caller can decide how much memory to use by calling commit API, or not calling the commit API if deciding to not use any memory. 1. https://lore.kernel.org/all/20240228093013.8263-1-linyunsheng@huawei.com/ Signed-off-by: Yunsheng Lin --- include/linux/page_frag_cache.h | 72 ++++++++++++++++++++++++++++++++- mm/page_frag_alloc.c | 13 +++--- 2 files changed, 75 insertions(+), 10 deletions(-) diff --git a/include/linux/page_frag_cache.h b/include/linux/page_frag_cache.h index a97a1ac017d6..3f17c0eba7fa 100644 --- a/include/linux/page_frag_cache.h +++ b/include/linux/page_frag_cache.h @@ -43,8 +43,76 @@ static inline bool page_frag_cache_is_pfmemalloc(struct page_frag_cache *nc) void page_frag_cache_drain(struct page_frag_cache *nc); void __page_frag_cache_drain(struct page *page, unsigned int count); -void *page_frag_alloc_va(struct page_frag_cache *nc, unsigned int fragsz, - gfp_t gfp_mask); +void *__page_frag_alloc_prepare(struct page_frag_cache *nc, unsigned int fragsz, + gfp_t gfp_mask); + +static inline void *page_frag_alloc_va(struct page_frag_cache *nc, + unsigned int fragsz, gfp_t gfp_mask) +{ + void *va; + + va = __page_frag_alloc_prepare(nc, fragsz, gfp_mask); + if (unlikely(!va)) + return NULL; + + va += nc->offset; + nc->pagecnt_bias--; + nc->offset = nc->offset + fragsz; + + return va; +} + +static inline void *page_frag_alloc_prepare(struct page_frag_cache *nc, + unsigned int *offset, + unsigned int *size, + gfp_t gfp_mask) +{ + void *va; + + va = __page_frag_alloc_prepare(nc, *size, gfp_mask); + if (unlikely(!va)) + return NULL; + + *offset = nc->offset; + +#if (PAGE_SIZE < PAGE_FRAG_CACHE_MAX_SIZE) + *size = nc->size_mask - *offset + 1; +#else + *size = PAGE_SIZE - *offset; +#endif + + return va; +} + +static inline void *page_frag_alloc_prepare_align(struct page_frag_cache *nc, + unsigned int *offset, + unsigned int *size, + unsigned int align, + gfp_t gfp_mask) +{ + unsigned int old_offset = nc->offset; + + WARN_ON_ONCE(!is_power_of_2(align) || align >= PAGE_SIZE || + *size < sizeof(unsigned int)); + + nc->offset = ALIGN(old_offset, align); + return page_frag_alloc_prepare(nc, offset, size, gfp_mask); +} + +static inline void page_frag_alloc_commit(struct page_frag_cache *nc, + unsigned int offset, + unsigned int size) +{ + nc->pagecnt_bias--; + nc->offset = offset + size; +} + +static inline void page_frag_alloc_commit_noref(struct page_frag_cache *nc, + unsigned int offset, + unsigned int size) +{ + nc->offset = offset + size; +} static inline void *__page_frag_alloc_va_align(struct page_frag_cache *nc, unsigned int fragsz, diff --git a/mm/page_frag_alloc.c b/mm/page_frag_alloc.c index ae1393d0619a..c4d4fc10a850 100644 --- a/mm/page_frag_alloc.c +++ b/mm/page_frag_alloc.c @@ -81,8 +81,8 @@ void __page_frag_cache_drain(struct page *page, unsigned int count) } EXPORT_SYMBOL(__page_frag_cache_drain); -void *page_frag_alloc_va(struct page_frag_cache *nc, unsigned int fragsz, - gfp_t gfp_mask) +void *__page_frag_alloc_prepare(struct page_frag_cache *nc, unsigned int fragsz, + gfp_t gfp_mask) { unsigned long size_mask; unsigned int offset; @@ -120,7 +120,7 @@ void *page_frag_alloc_va(struct page_frag_cache *nc, unsigned int fragsz, set_page_count(page, size_mask); nc->pagecnt_bias |= size_mask; - offset = 0; + nc->offset = 0; if (unlikely(fragsz > (size_mask + 1))) { /* * The caller is trying to allocate a fragment @@ -135,12 +135,9 @@ void *page_frag_alloc_va(struct page_frag_cache *nc, unsigned int fragsz, } } - nc->pagecnt_bias--; - nc->offset = offset + fragsz; - - return va + offset; + return va; } -EXPORT_SYMBOL(page_frag_alloc_va); +EXPORT_SYMBOL(__page_frag_alloc_prepare); /* * Frees a page fragment allocated out of either a compound or order 0 page.