From patchwork Tue Dec 19 22:28:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 13499309 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0681AC41535 for ; Tue, 19 Dec 2023 22:31:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CB4AD8D0012; Tue, 19 Dec 2023 17:31:28 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C66C58D000D; Tue, 19 Dec 2023 17:31:28 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A8FC08D0012; Tue, 19 Dec 2023 17:31:28 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 985B28D000D for ; Tue, 19 Dec 2023 17:31:28 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 745DFA117A for ; Tue, 19 Dec 2023 22:31:28 +0000 (UTC) X-FDA: 81585015456.27.0089433 Received: from out-183.mta0.migadu.com (out-183.mta0.migadu.com [91.218.175.183]) by imf21.hostedemail.com (Postfix) with ESMTP id A3B5D1C0012 for ; Tue, 19 Dec 2023 22:31:26 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=CZpopGBZ; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf21.hostedemail.com: domain of andrey.konovalov@linux.dev designates 91.218.175.183 as permitted sender) smtp.mailfrom=andrey.konovalov@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1703025086; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Vw9Ju+XsuH7jTh0rV53QjnTfccLPGDaCnQHzua8vht8=; b=6QVnKcm0dkHN7AHZsnPR7O6PqHMjop2CGeoelNezxDQJ6bZSpgEVmOFTvR9XrNizVkM0KM Ld3OWOXJh+M1iDypg5fkR0+kqLFlloq1zsTaiR4wpQjrxqVEZl3jVnljJbBduFR7i3I8be hxudUZ7DoNt8ZrT3WG/wxF0Z6d9iFVM= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=CZpopGBZ; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf21.hostedemail.com: domain of andrey.konovalov@linux.dev designates 91.218.175.183 as permitted sender) smtp.mailfrom=andrey.konovalov@linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1703025086; a=rsa-sha256; cv=none; b=tk58b+xci+xZ0IKSl7f1QoaKJo1FpV6CRB39ID2Xi1FnQyfz0cVccX+cUll5zPty4pzBWX 2LmbLuwu9QcWyvaraNBetx06teCF7sKx6exD5FqONn6luH6GVKsynjiWjLaJklj1nJwymD +IhzvK/OahIwtiXIXRN/7WzJ6Wyn+O8= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1703025085; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Vw9Ju+XsuH7jTh0rV53QjnTfccLPGDaCnQHzua8vht8=; b=CZpopGBZsYX+uocBYMIAfDP7PDv7NGAgHL88CMPaHeNmC1Jd3UkHAbZqq1SXvnp8qR3PRk ko3vkaTQDOJV582w6cY6Jpq7gGMvDm6ekS2hgFsxwAQMhisoeEHQlLiOfoxsL6ydfk5Smi uW9JPOVYPoMNbIRTQbjw3x7wa9H6dN4= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko Cc: Andrey Konovalov , Dmitry Vyukov , Andrey Ryabinin , kasan-dev@googlegroups.com, Evgenii Stepanov , Breno Leitao , Alexander Lobakin , Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH mm 15/21] mempool: introduce mempool_use_prealloc_only Date: Tue, 19 Dec 2023 23:28:59 +0100 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Rspamd-Queue-Id: A3B5D1C0012 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: c6wgoxca6bcih8mwas1mfew39px19kmd X-HE-Tag: 1703025086-319171 X-HE-Meta: U2FsdGVkX1/jd1uLNokdP5o3EzQAjhj1+XGStuK0iWY4+q/DABImAhBkMgxijTRQTQCatw/d+GNMndq1i3+g4/wTiDhK9NXD9tPnMldCzIcuT02xhC7LNhykUd/Zw/nWBfqAWlBQzh1wG2fWWQmBiNU4TZSR8HhW6EeukyDAJMp7miE8B9VRGoaHMbflIHHwKav5XcoIdgq4B0Xj8Fqbq32u0/5LQou035cD3dufweZtaNDfhYzpLSqJ0c6Zwwm8d3y1ulnMQV7LRiWLMcgLuZ9FDl88VAVGhfkxMkbQ1uAHNf8cO0n6DcGbbwU9SKYS+DLElEa7Fe6JF0zO0PvFJwTX8prmeVPKr+8R1dCp8oFJvVnr678oYQNS9N6YO9yNrHkK/7/UbFKj3lKTbPioFIuSDLBIeMra2tjvkBJMbENha6iOUvbBSf3qgptAi8athB9EByV+997MdS79WYjRNitINalAPsKMNQ0Z45oTs2lpKrHynfDWAYMjMfWPHD5V3YCq1YGdrd5DqdzsT8gTYmA78yPGxrgKt3qKMnshtcxg/mx6WFIGtxoCM7jIDOTz7FkJjHO+hvvH5DdhNXDTNu6U8lFf673osGPbq4xnCCT+en8KKMoQOqclwqFZN+GvhTiStmoVQ/fziGWaO2X8jntwTwLn/7h5aNrVkAM5zGKvP34FjDEh2JSv1KXRr4AFHtlhFhfbWpNWbncEtuHP0VDb+8evXQkKxdMy5+vjkvsYGWDGMocjtLY/35zcld5+XwDrqlTpenZOM4PkPFpra1pTdXqzI6g01auGOgQoVPw1H48CHQD0vAlrFAOMSPGdQTTi4lT3EFnCBt2ZBurPjSA1Rk067HHcOAArQqjk7bFMV5wDRs4ZPitFlaE+fjJq/baK15hdyIVWLAyxNWnIYs9Yq4nktBhJrdJ/+t2TBREa2Bto1Sa2/dJHwKied3dR3wwJJMXEYBaoPRzhlPk SgUlysWn brbF+J5UJ9LNWXvjNFJDvnFakSzGZjVzPH+wqCF8HE3ns3AaBW9nOXr+EB+45k71tSRiMYbjJ4IA8Rs/bTECa5BlIoFkkjV3JsLIivkL1aLHiuI3JJclmlKBtwselpypbhLHsI8ngDFUCxn/NHQkOPDFDEjnAAFeBUhN8X+FwESs9vD/LmmXojZk59kp39tSAIV3tGekDBxUEqRlPlcoI9NZDVNqx/0gVzJ0Y X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Andrey Konovalov Introduce a new mempool_alloc_preallocated API that asks the mempool to only use the elements preallocated during the mempool's creation when allocating and to not attempt allocating new ones from the underlying allocator. This API is required to test the KASAN poisoning/unpoisoning functionality in KASAN tests, but it might be also useful on its own. Signed-off-by: Andrey Konovalov --- Changes RFC->v1: - Introduce a new mempool_alloc_preallocated API instead of adding a flag into mempool_t. --- include/linux/mempool.h | 1 + mm/mempool.c | 37 +++++++++++++++++++++++++++++++++++++ 2 files changed, 38 insertions(+) diff --git a/include/linux/mempool.h b/include/linux/mempool.h index 4aae6c06c5f2..7be1e32e6d42 100644 --- a/include/linux/mempool.h +++ b/include/linux/mempool.h @@ -51,6 +51,7 @@ extern mempool_t *mempool_create_node(int min_nr, mempool_alloc_t *alloc_fn, extern int mempool_resize(mempool_t *pool, int new_min_nr); extern void mempool_destroy(mempool_t *pool); extern void *mempool_alloc(mempool_t *pool, gfp_t gfp_mask) __malloc; +extern void *mempool_alloc_preallocated(mempool_t *pool) __malloc; extern void mempool_free(void *element, mempool_t *pool); /* diff --git a/mm/mempool.c b/mm/mempool.c index 103dc4770cfb..cb7b4b56cec1 100644 --- a/mm/mempool.c +++ b/mm/mempool.c @@ -456,6 +456,43 @@ void *mempool_alloc(mempool_t *pool, gfp_t gfp_mask) } EXPORT_SYMBOL(mempool_alloc); +/** + * mempool_alloc_preallocated - allocate an element from preallocated elements + * belonging to a specific memory pool + * @pool: pointer to the memory pool which was allocated via + * mempool_create(). + * + * This function is similar to mempool_alloc, but it only attempts allocating + * an element from the preallocated elements. It does not sleep and immediately + * returns if no preallocated elements are available. + * + * Return: pointer to the allocated element or %NULL if no elements are + * available. + */ +void *mempool_alloc_preallocated(mempool_t *pool) +{ + void *element; + unsigned long flags; + + spin_lock_irqsave(&pool->lock, flags); + if (likely(pool->curr_nr)) { + element = remove_element(pool); + spin_unlock_irqrestore(&pool->lock, flags); + /* paired with rmb in mempool_free(), read comment there */ + smp_wmb(); + /* + * Update the allocation stack trace as this is more useful + * for debugging. + */ + kmemleak_update_trace(element); + return element; + } + spin_unlock_irqrestore(&pool->lock, flags); + + return NULL; +} +EXPORT_SYMBOL(mempool_alloc_preallocated); + /** * mempool_free - return an element to the pool. * @element: pool element pointer.