From patchwork Fri Dec 6 12:25:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yunsheng Lin X-Patchwork-Id: 13897123 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F136DE77173 for ; Fri, 6 Dec 2024 12:32:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6E16F6B026B; Fri, 6 Dec 2024 07:32:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6925D6B026D; Fri, 6 Dec 2024 07:32:46 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 559B46B026E; Fri, 6 Dec 2024 07:32:46 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 361386B026B for ; Fri, 6 Dec 2024 07:32:46 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id D54D142800 for ; Fri, 6 Dec 2024 12:32:35 +0000 (UTC) X-FDA: 82864471914.19.302CB59 Received: from szxga04-in.huawei.com (szxga04-in.huawei.com [45.249.212.190]) by imf06.hostedemail.com (Postfix) with ESMTP id 832F9180003 for ; Fri, 6 Dec 2024 12:32:21 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf06.hostedemail.com: domain of linyunsheng@huawei.com designates 45.249.212.190 as permitted sender) smtp.mailfrom=linyunsheng@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733488346; a=rsa-sha256; cv=none; b=XnOT7zw+8cSV22YsYWvRaY0ZWgeem8hx8J/13LEGCJSqvSTqmK/f9++41kdtVW7Fr7R3G1 eJBd2VC0kDJNpM9BjSCkDHPi/i8Pe10fnAcpHXeZ705/cfc52S6CtdD0oStcVyjNZXctJi FAmYi0K5OaWeD8GLK4XzX0fJ3tC3h30= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf06.hostedemail.com: domain of linyunsheng@huawei.com designates 45.249.212.190 as permitted sender) smtp.mailfrom=linyunsheng@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733488346; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SU9hFE/IxapLxlkNs+ckNoGZnzqsXf/PqdlweufhXC8=; b=40+uaOTUlDpdp4HicX/14+jV2tLhJG/SzXo10+S0d8/+OivM5YmJ7YQB0RT+/hbcejbjdd Mu5Q5JkXqkBVLmlJUiqhOnMhRrZPtfbDDjw8VC6G4TOS0d9wIrPqWjvjC4ThvMxwxb3KUH Mfr2mbEmBwv7dSpGWbbP3tPGjCJuPlw= Received: from mail.maildlp.com (unknown [172.19.162.112]) by szxga04-in.huawei.com (SkyGuard) with ESMTP id 4Y4VwV3k1zz21mgp; Fri, 6 Dec 2024 20:30:50 +0800 (CST) Received: from dggpemf200006.china.huawei.com (unknown [7.185.36.61]) by mail.maildlp.com (Postfix) with ESMTPS id 1577A1401DC; Fri, 6 Dec 2024 20:32:30 +0800 (CST) Received: from localhost.localdomain (10.90.30.45) by dggpemf200006.china.huawei.com (7.185.36.61) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Fri, 6 Dec 2024 20:32:29 +0800 From: Yunsheng Lin To: , , CC: , , Yunsheng Lin , Alexander Duyck , Andrew Morton , Linux-MM , Jonathan Corbet , Subject: [PATCH net-next v2 05/10] mm: page_frag: introduce refill prepare & commit API Date: Fri, 6 Dec 2024 20:25:28 +0800 Message-ID: <20241206122533.3589947-6-linyunsheng@huawei.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20241206122533.3589947-1-linyunsheng@huawei.com> References: <20241206122533.3589947-1-linyunsheng@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.90.30.45] X-ClientProxiedBy: dggems702-chm.china.huawei.com (10.3.19.179) To dggpemf200006.china.huawei.com (7.185.36.61) X-Rspam-User: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 832F9180003 X-Stat-Signature: rkuhf1rtj3fazxfp3ad5zrrut56tjwah X-HE-Tag: 1733488341-920135 X-HE-Meta: U2FsdGVkX1/GbohKg8pGXx8/nyYV21SyWOP8KQ4siCal8DK8b6KK4g4wQo3hf2b2bhencufzTTjZwvnkGGXVL1a9G9V6p9j+wBRfufdkxV9Y+qBbrrcTC9LqkMDFv/HLoRn/CCwZm7u60Refe5f67ugArDViwo44e8n3Ccjw2rHfjQfvsnFyDOR158a3dFtZBbVIAFUP7VG+st/EC2Lswrb+Tw1AJ0OYQg4xWyJ85Upx5hVq34TFleaRwvrPl8WQac8FMImPDYgiiInG2A9U2YdPr8EKqQhaop0fbs97QqFzISK2R84zGuen6u27tlBcrkLCrPmgxpFcZBLKzGh/wzSChBrWDJkSkPpGx5sXxSci1U/uuMUi9XTDlX/3+bE4GsLfA2Fyis7a5bO8lyd4nuLsKkUrejQKSIOwKMdccsL2JyI4qnHc66Gc5uZ75bNNCs9RG/f28/YeGNxDqTO33dMsGh+MR6AQHt2J8wkDwtvkYI69evmAwoktBHlpxtNeMnOIwySnPvOOAdLczots2J3PR9IKJBdLWn8Sk/4VcIJUAyb0wf10gIJ8EtKyyGbsGdE5llafVlIruFG8JN6sM+l2J9LrxAWQNSuTjz6/p3Tnlg5zKeXfeecEqI+hxMuhAq2kxiHInr4KzkGJ4viQIiJZ1jJwhpWt97vbhebjhNLPMpSxZTtVLL/LdOqpTbIo69mlMrmIcnr+WPgrt9AgMn7COHCp7t1EGTLoUAlfgCaCV3c4RjRf4HMgoF2VGMBPpE95+rqkJ/KSjhGBFmaSH+KppX9Nf8hvRhMnTxnSC4xz4JMPckQEUZVTUBQEvlG7z7Yg/N+64xG15uKc3wfg/mk1KX6BHgAOZoyOWuTyPRtOob/uLggRzoqqJ6fKVH2p8JevHuKI4OoW4aJxdoMmeQqLLYL950Og/F8nYOdqeXqfEOJxwLgsriY+s4WPxmwPW3LzRBvb3p0GcjKYHOl Z5zAaU9m 9zDuugH8BoeBoidST7miHkS1u/T+YkQI/tQ86KHJ5VBOlfVG1mL5o0200KAquvywMa8nPKJAJTlFh5U0QIsNQLI1a/kU9TE4d6uIbitV8AG7vIxaL627fMw2ADewfz2SpoGIywrjy/3C9B7cQMgsfu46vN9G6jfUYXIpVu0MMMdyFVh2oh+tC0OpiLtqTkX78Q0KMJdgTzNzT+9B8tCtgFhI1VfB/4J5fDJH1oHmFARo/tZUwTMTZH1Y5Fh3pOJd4nRDRZHiXwow6eG/VYMTOk9xZhu4ErOUaRvP9fyQ7Du8IVIgJmJKPl/aNZIVjhi8JcrvgVylCTttsBQUV9d/1P2W0pPZr/EoKf3bbNy2Dz60Yt63xJerYnxCrodQFvL6r1wg9UJx5j5KVtK3KCbFM9Vq98Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Currently page_frag only have a alloc API which returns the virtual address of a fragment by a specific size. There are many use cases that need minimum memory in order for forward progress, but more performant if more memory is available, and expect to use the 'struct page' of the allocated fragment directly instead of the virtual address. Currently skb_page_frag_refill() API is used to solve the above use cases, but caller needs to know about the internal detail and access the data field of 'struct page_frag' to meet the requirement of the above use cases and its implementation is similar to the one in mm subsystem. To unify those two page_frag implementations, introduce a prepare API to ensure minimum memory is satisfied and return how much the actual memory is available to the caller. The caller needs to either call the commit API to report how much memory it actually uses, or not do so if deciding to not use any memory. CC: Alexander Duyck CC: Andrew Morton CC: Linux-MM Signed-off-by: Yunsheng Lin --- Documentation/mm/page_frags.rst | 43 ++++++++++++- include/linux/page_frag_cache.h | 110 ++++++++++++++++++++++++++++++++ 2 files changed, 152 insertions(+), 1 deletion(-) diff --git a/Documentation/mm/page_frags.rst b/Documentation/mm/page_frags.rst index 339e641beb53..4cfdbe7db55a 100644 --- a/Documentation/mm/page_frags.rst +++ b/Documentation/mm/page_frags.rst @@ -111,10 +111,18 @@ page is aligned according to the 'align/alignment' parameter. Note the size of the allocated fragment is not aligned, the caller needs to provide an aligned fragsz if there is an alignment requirement for the size of the fragment. +There is a use case that needs minimum memory in order for forward progress, but +more performant if more memory is available. By using the prepare and commit +related API, the caller calls prepare API to requests the minimum memory it +needs and prepare API will return the maximum size of the fragment returned. The +caller needs to either call the commit API to report how much memory it actually +uses, or not do so if deciding to not use any memory. + .. kernel-doc:: include/linux/page_frag_cache.h :identifiers: page_frag_cache_init page_frag_cache_is_pfmemalloc __page_frag_alloc_align page_frag_alloc_align page_frag_alloc - page_frag_alloc_abort + page_frag_alloc_abort __page_frag_refill_prepare_align + page_frag_refill_prepare_align page_frag_refill_prepare .. kernel-doc:: mm/page_frag_cache.c :identifiers: page_frag_cache_drain page_frag_free page_frag_alloc_abort_ref @@ -152,3 +160,36 @@ Allocation & freeing API ... page_frag_free(va); + + +Refill Preparation & committing API +----------------------------------- + +.. code-block:: c + + struct page_frag page_frag, *pfrag; + bool merge = true; + + pfrag = &page_frag; + if (!page_frag_refill_prepare(nc, 32U, pfrag, GFP_KERNEL)) + goto wait_for_space; + + copy = min_t(unsigned int, copy, pfrag->size); + if (!skb_can_coalesce(skb, i, pfrag->page, pfrag->offset)) { + if (i >= max_skb_frags) + goto new_segment; + + merge = false; + } + + copy = mem_schedule(copy); + if (!copy) + goto wait_for_space; + + if (merge) { + skb_frag_size_add(&skb_shinfo(skb)->frags[i - 1], copy); + page_frag_refill_commit_noref(nc, pfrag, copy); + } else { + skb_fill_page_desc(skb, i, pfrag->page, pfrag->offset, copy); + page_frag_refill_commit(nc, pfrag, copy); + } diff --git a/include/linux/page_frag_cache.h b/include/linux/page_frag_cache.h index c3347c97522c..1e699334646a 100644 --- a/include/linux/page_frag_cache.h +++ b/include/linux/page_frag_cache.h @@ -140,6 +140,116 @@ static inline void *page_frag_alloc(struct page_frag_cache *nc, return __page_frag_alloc_align(nc, fragsz, gfp_mask, ~0u); } +/** + * __page_frag_refill_prepare_align() - Prepare refilling a page_frag with + * aligning requirement. + * @nc: page_frag cache from which to refill + * @fragsz: the requested fragment size + * @pfrag: the page_frag to be refilled. + * @gfp_mask: the allocation gfp to use when cache need to be refilled + * @align_mask: the requested aligning requirement for the fragment + * + * Prepare refilling a page_frag from page_frag cache with aligning requirement. + * + * Return: + * True if prepare refilling succeeds, otherwise return false. + */ +static inline bool __page_frag_refill_prepare_align(struct page_frag_cache *nc, + unsigned int fragsz, + struct page_frag *pfrag, + gfp_t gfp_mask, + unsigned int align_mask) +{ + return !!__page_frag_cache_prepare(nc, fragsz, pfrag, gfp_mask, + align_mask); +} + +/** + * page_frag_refill_prepare_align() - Prepare refilling a page_frag with + * aligning requirement. + * @nc: page_frag cache from which to refill + * @fragsz: the requested fragment size + * @pfrag: the page_frag to be refilled. + * @gfp_mask: the allocation gfp to use when cache needs to be refilled + * @align: the requested aligning requirement for the fragment + * + * WARN_ON_ONCE() checking for @align before prepare refilling a page_frag from + * page_frag cache with aligning requirement. + * + * Return: + * True if prepare refilling succeeds, otherwise return false. + */ +static inline bool page_frag_refill_prepare_align(struct page_frag_cache *nc, + unsigned int fragsz, + struct page_frag *pfrag, + gfp_t gfp_mask, + unsigned int align) +{ + WARN_ON_ONCE(!is_power_of_2(align)); + return __page_frag_refill_prepare_align(nc, fragsz, pfrag, gfp_mask, + -align); +} + +/** + * page_frag_refill_prepare() - Prepare refilling a page_frag. + * @nc: page_frag cache from which to refill + * @fragsz: the requested fragment size + * @pfrag: the page_frag to be refilled. + * @gfp_mask: the allocation gfp to use when cache need to be refilled + * + * Prepare refilling a page_frag from page_frag cache. + * + * Return: + * True if refill succeeds, otherwise return false. + */ +static inline bool page_frag_refill_prepare(struct page_frag_cache *nc, + unsigned int fragsz, + struct page_frag *pfrag, + gfp_t gfp_mask) +{ + return __page_frag_refill_prepare_align(nc, fragsz, pfrag, gfp_mask, + ~0u); +} + +/** + * page_frag_refill_commit - Commit a prepare refilling. + * @nc: page_frag cache from which to commit + * @pfrag: the page_frag to be committed + * @used_sz: size of the page fragment has been used + * + * Commit the actual used size for the refill that was prepared. + * + * Return: + * The true size of the fragment considering the offset alignment. + */ +static inline unsigned int page_frag_refill_commit(struct page_frag_cache *nc, + struct page_frag *pfrag, + unsigned int used_sz) +{ + return __page_frag_cache_commit(nc, pfrag, used_sz); +} + +/** + * page_frag_refill_commit_noref - Commit a prepare refilling without taking + * refcount. + * @nc: page_frag cache from which to commit + * @pfrag: the page_frag to be committed + * @used_sz: size of the page fragment has been used + * + * Commit the prepare refilling by passing the actual used size, but not taking + * refcount. Mostly used for fragmemt coalescing case when the current fragment + * can share the same refcount with previous fragment. + * + * Return: + * The true size of the fragment considering the offset alignment. + */ +static inline unsigned int +page_frag_refill_commit_noref(struct page_frag_cache *nc, + struct page_frag *pfrag, unsigned int used_sz) +{ + return __page_frag_cache_commit_noref(nc, pfrag, used_sz); +} + void page_frag_free(void *addr); void page_frag_alloc_abort_ref(struct page_frag_cache *nc, void *va, unsigned int fragsz);