From patchwork Mon Oct 28 11:58:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yunsheng Lin X-Patchwork-Id: 13853385 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 16F95D1359C for ; Mon, 28 Oct 2024 12:05:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4D8C46B00AA; Mon, 28 Oct 2024 08:05:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 460C26B00AB; Mon, 28 Oct 2024 08:05:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3289E6B00AC; Mon, 28 Oct 2024 08:05:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 12EDC6B00AA for ; Mon, 28 Oct 2024 08:05:56 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id A4CB01C507B for ; Mon, 28 Oct 2024 12:05:29 +0000 (UTC) X-FDA: 82722881262.08.3C51E84 Received: from szxga08-in.huawei.com (szxga08-in.huawei.com [45.249.212.255]) by imf27.hostedemail.com (Postfix) with ESMTP id 8AFA94000B for ; Mon, 28 Oct 2024 12:05:29 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf27.hostedemail.com: domain of linyunsheng@huawei.com designates 45.249.212.255 as permitted sender) smtp.mailfrom=linyunsheng@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1730117111; a=rsa-sha256; cv=none; b=aD48Bb4riBOzkrTlIl0jxB71B5uy68vNb9qblcsh+qTUomCz0phKkPruKMWmnX36ENf2Ov AWXqYCl59Prop4L8dTQtbgF+SfPUeRrgNpLC8NVzRN6Z/6bAeG9pJWJXzl8eePONe7wIvz bEvOahQVSi1ZA0u3JubN2UJb/k+InwI= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf27.hostedemail.com: domain of linyunsheng@huawei.com designates 45.249.212.255 as permitted sender) smtp.mailfrom=linyunsheng@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1730117111; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SU9hFE/IxapLxlkNs+ckNoGZnzqsXf/PqdlweufhXC8=; b=Awjigq/4GuJsVQF1H5u2ybKl3KZWJJ1G8Ke87yoiVSYcYB7fnW4Hh3sL34SfVbcu7SjtLP LAW8ob8vMeDl7/Xm5szUsuZ0wwxLjortoDIK/5Z3VwntXmR4fYH6k8uADffXGySI2CU+Bt K+0Ky/TfgdgeRTg3ja63Tux0SMgJcTs= Received: from mail.maildlp.com (unknown [172.19.163.252]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4XcX8Y0fJdz1T9CN; Mon, 28 Oct 2024 20:03:09 +0800 (CST) Received: from dggpemf200006.china.huawei.com (unknown [7.185.36.61]) by mail.maildlp.com (Postfix) with ESMTPS id 5C07D1800CF; Mon, 28 Oct 2024 20:05:17 +0800 (CST) Received: from localhost.localdomain (10.90.30.45) by dggpemf200006.china.huawei.com (7.185.36.61) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Mon, 28 Oct 2024 20:05:17 +0800 From: Yunsheng Lin To: , , CC: , , Yunsheng Lin , Alexander Duyck , Andrew Morton , Linux-MM , Jonathan Corbet , Subject: [PATCH RFC 05/10] mm: page_frag: introduce refill prepare & commit API Date: Mon, 28 Oct 2024 19:58:45 +0800 Message-ID: <20241028115850.3409893-6-linyunsheng@huawei.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20241028115850.3409893-1-linyunsheng@huawei.com> References: <20241028115850.3409893-1-linyunsheng@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.90.30.45] X-ClientProxiedBy: dggems702-chm.china.huawei.com (10.3.19.179) To dggpemf200006.china.huawei.com (7.185.36.61) X-Rspam-User: X-Stat-Signature: w8y7ornb6ce54edf76b87dh5pigxtoeu X-Rspamd-Queue-Id: 8AFA94000B X-Rspamd-Server: rspam02 X-HE-Tag: 1730117129-616811 X-HE-Meta: U2FsdGVkX1+fAk6vR/gpa9IrkN9Tt6bDisnMTv7Rk9gHbYnUpHzXccxqmsb+GlBQyAFKvAVIzv/XrG6eEUHaX4vH7x2plFUHzONo8s0MR6Qru8xtNdmYSuSRYIDGf+hPhCssLNe64qqc3BN3fYWSiM4ifpJ7JMIZBv7e4541g+pthd6qQqHHoUqoFLcJSUHLudBKVEFOWbJhrDQg6idcCNNPFMeBzQGjZ6BN4krTJUYqJlPyvo155EOEwHoLCAgG6tvaYIf5t/F9wzyGdyZpcEb6h81i7ueDu2nEQYlkU8ImpeSWuqPjPrceZRtqWevX910DzSVqAok6r93k7Ys+mDSQubfK0w5kD4y2uYwFI0z0Y70hZzDGDUjI9h4FXx+wIsDiexypUUfONjhevtILzmJyHEQtMF5krGziyci1xbpfUE83vLTNca3Jg0/L1RUnfFiVyYaydJ1UE4YjRl6yQdNvgcqOIb36mRbQTdW7LbwPBSkFcIj5ti4ZnhuHX4cUMg30jy0RKNcyJ4uj/KxgBO3d4VKxW5EURmacDHgB8o8xE7YUWvFS0DzbPjvJpgZtOFljZ2vdSwTB7nJScCJVBN++erOQtk3dMfu1XpHg/6Vuz6koIF5QtjeSvSBtWD0CNnRcop3kx6MwEq7xhDEiaeH7b1QIWIs3c6Qo26EHZZPBjboB+pdB1MBmnex9Qg5KDj09EMWPldemm7dZzB2arTO1cVLguPlIuZF5/UH5pCM8tKnX+rJ4gW2jXWO2tQDy27rQRBwGYFkoM88O3ZKP7029J9WnXpfCgwh4eNnF27nl50Odn5Y3AVkBzSIfcVhpAC2jGC7UyMfPilCPhuGM1Bt/1J/Egwp+IYDwu+cwENMjRNqzHZ7lxzNcxP53CnZBj8ucMVHmyHYrpzryO4iE8H4+IFO39xF9jQ99BzPtDDbqACyatfYlzM5QCpbh+MnohX2e+LCSWLAdtBlb4+2 HkJjlCKN /YRQ5+XturyQw9Dgx8H/MfML3/DoALzZFfSaLZSA0FxiY3jEpfCo6jiuBHmIgQvxNeHwhCHkyr/drUr2vuUz2842Tcp7i2hFtWjlIW9sM9ub0Jc26iXvCTMiJIPaTsiBcIEp4P8+kNczplAE+RO9ZaQO/88JIanmuehHGzdlKpm3TCyc5egvz7XDIkmmCJYjroon/VjPpfaAmdoxk4dA5WrRX698dvEtjjGtVUaqROzHP1jzq7PSxjO+7NKPGa6MBdXERUanqMVJuaMNx8SWiRKVia4V51Mzjo374yxyvlAiXBlQglzhhObk74iKgd/e95uVs9VJ4rlUoIA5i5xEwwOqYhvUML6f67gthGUqaKeqqKuVVQYgp6IvZ1iXFRM+B+G2AM1vwDtX0aDuKfT56u6WXZg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Currently page_frag only have a alloc API which returns the virtual address of a fragment by a specific size. There are many use cases that need minimum memory in order for forward progress, but more performant if more memory is available, and expect to use the 'struct page' of the allocated fragment directly instead of the virtual address. Currently skb_page_frag_refill() API is used to solve the above use cases, but caller needs to know about the internal detail and access the data field of 'struct page_frag' to meet the requirement of the above use cases and its implementation is similar to the one in mm subsystem. To unify those two page_frag implementations, introduce a prepare API to ensure minimum memory is satisfied and return how much the actual memory is available to the caller. The caller needs to either call the commit API to report how much memory it actually uses, or not do so if deciding to not use any memory. CC: Alexander Duyck CC: Andrew Morton CC: Linux-MM Signed-off-by: Yunsheng Lin --- Documentation/mm/page_frags.rst | 43 ++++++++++++- include/linux/page_frag_cache.h | 110 ++++++++++++++++++++++++++++++++ 2 files changed, 152 insertions(+), 1 deletion(-) diff --git a/Documentation/mm/page_frags.rst b/Documentation/mm/page_frags.rst index 339e641beb53..4cfdbe7db55a 100644 --- a/Documentation/mm/page_frags.rst +++ b/Documentation/mm/page_frags.rst @@ -111,10 +111,18 @@ page is aligned according to the 'align/alignment' parameter. Note the size of the allocated fragment is not aligned, the caller needs to provide an aligned fragsz if there is an alignment requirement for the size of the fragment. +There is a use case that needs minimum memory in order for forward progress, but +more performant if more memory is available. By using the prepare and commit +related API, the caller calls prepare API to requests the minimum memory it +needs and prepare API will return the maximum size of the fragment returned. The +caller needs to either call the commit API to report how much memory it actually +uses, or not do so if deciding to not use any memory. + .. kernel-doc:: include/linux/page_frag_cache.h :identifiers: page_frag_cache_init page_frag_cache_is_pfmemalloc __page_frag_alloc_align page_frag_alloc_align page_frag_alloc - page_frag_alloc_abort + page_frag_alloc_abort __page_frag_refill_prepare_align + page_frag_refill_prepare_align page_frag_refill_prepare .. kernel-doc:: mm/page_frag_cache.c :identifiers: page_frag_cache_drain page_frag_free page_frag_alloc_abort_ref @@ -152,3 +160,36 @@ Allocation & freeing API ... page_frag_free(va); + + +Refill Preparation & committing API +----------------------------------- + +.. code-block:: c + + struct page_frag page_frag, *pfrag; + bool merge = true; + + pfrag = &page_frag; + if (!page_frag_refill_prepare(nc, 32U, pfrag, GFP_KERNEL)) + goto wait_for_space; + + copy = min_t(unsigned int, copy, pfrag->size); + if (!skb_can_coalesce(skb, i, pfrag->page, pfrag->offset)) { + if (i >= max_skb_frags) + goto new_segment; + + merge = false; + } + + copy = mem_schedule(copy); + if (!copy) + goto wait_for_space; + + if (merge) { + skb_frag_size_add(&skb_shinfo(skb)->frags[i - 1], copy); + page_frag_refill_commit_noref(nc, pfrag, copy); + } else { + skb_fill_page_desc(skb, i, pfrag->page, pfrag->offset, copy); + page_frag_refill_commit(nc, pfrag, copy); + } diff --git a/include/linux/page_frag_cache.h b/include/linux/page_frag_cache.h index c3347c97522c..1e699334646a 100644 --- a/include/linux/page_frag_cache.h +++ b/include/linux/page_frag_cache.h @@ -140,6 +140,116 @@ static inline void *page_frag_alloc(struct page_frag_cache *nc, return __page_frag_alloc_align(nc, fragsz, gfp_mask, ~0u); } +/** + * __page_frag_refill_prepare_align() - Prepare refilling a page_frag with + * aligning requirement. + * @nc: page_frag cache from which to refill + * @fragsz: the requested fragment size + * @pfrag: the page_frag to be refilled. + * @gfp_mask: the allocation gfp to use when cache need to be refilled + * @align_mask: the requested aligning requirement for the fragment + * + * Prepare refilling a page_frag from page_frag cache with aligning requirement. + * + * Return: + * True if prepare refilling succeeds, otherwise return false. + */ +static inline bool __page_frag_refill_prepare_align(struct page_frag_cache *nc, + unsigned int fragsz, + struct page_frag *pfrag, + gfp_t gfp_mask, + unsigned int align_mask) +{ + return !!__page_frag_cache_prepare(nc, fragsz, pfrag, gfp_mask, + align_mask); +} + +/** + * page_frag_refill_prepare_align() - Prepare refilling a page_frag with + * aligning requirement. + * @nc: page_frag cache from which to refill + * @fragsz: the requested fragment size + * @pfrag: the page_frag to be refilled. + * @gfp_mask: the allocation gfp to use when cache needs to be refilled + * @align: the requested aligning requirement for the fragment + * + * WARN_ON_ONCE() checking for @align before prepare refilling a page_frag from + * page_frag cache with aligning requirement. + * + * Return: + * True if prepare refilling succeeds, otherwise return false. + */ +static inline bool page_frag_refill_prepare_align(struct page_frag_cache *nc, + unsigned int fragsz, + struct page_frag *pfrag, + gfp_t gfp_mask, + unsigned int align) +{ + WARN_ON_ONCE(!is_power_of_2(align)); + return __page_frag_refill_prepare_align(nc, fragsz, pfrag, gfp_mask, + -align); +} + +/** + * page_frag_refill_prepare() - Prepare refilling a page_frag. + * @nc: page_frag cache from which to refill + * @fragsz: the requested fragment size + * @pfrag: the page_frag to be refilled. + * @gfp_mask: the allocation gfp to use when cache need to be refilled + * + * Prepare refilling a page_frag from page_frag cache. + * + * Return: + * True if refill succeeds, otherwise return false. + */ +static inline bool page_frag_refill_prepare(struct page_frag_cache *nc, + unsigned int fragsz, + struct page_frag *pfrag, + gfp_t gfp_mask) +{ + return __page_frag_refill_prepare_align(nc, fragsz, pfrag, gfp_mask, + ~0u); +} + +/** + * page_frag_refill_commit - Commit a prepare refilling. + * @nc: page_frag cache from which to commit + * @pfrag: the page_frag to be committed + * @used_sz: size of the page fragment has been used + * + * Commit the actual used size for the refill that was prepared. + * + * Return: + * The true size of the fragment considering the offset alignment. + */ +static inline unsigned int page_frag_refill_commit(struct page_frag_cache *nc, + struct page_frag *pfrag, + unsigned int used_sz) +{ + return __page_frag_cache_commit(nc, pfrag, used_sz); +} + +/** + * page_frag_refill_commit_noref - Commit a prepare refilling without taking + * refcount. + * @nc: page_frag cache from which to commit + * @pfrag: the page_frag to be committed + * @used_sz: size of the page fragment has been used + * + * Commit the prepare refilling by passing the actual used size, but not taking + * refcount. Mostly used for fragmemt coalescing case when the current fragment + * can share the same refcount with previous fragment. + * + * Return: + * The true size of the fragment considering the offset alignment. + */ +static inline unsigned int +page_frag_refill_commit_noref(struct page_frag_cache *nc, + struct page_frag *pfrag, unsigned int used_sz) +{ + return __page_frag_cache_commit_noref(nc, pfrag, used_sz); +} + void page_frag_free(void *addr); void page_frag_alloc_abort_ref(struct page_frag_cache *nc, void *va, unsigned int fragsz);