From patchwork Tue Aug 8 09:53:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vlastimil Babka X-Patchwork-Id: 13345913 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 08D12C001B0 for ; Tue, 8 Aug 2023 09:54:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 867278D0002; Tue, 8 Aug 2023 05:53:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 806E06B0080; Tue, 8 Aug 2023 05:53:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4867F6B007D; Tue, 8 Aug 2023 05:53:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 0BA438D0002 for ; Tue, 8 Aug 2023 05:53:57 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id BD4F11C9275 for ; Tue, 8 Aug 2023 09:53:56 +0000 (UTC) X-FDA: 81100476072.30.CB8487E Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by imf09.hostedemail.com (Postfix) with ESMTP id C70F7140014 for ; Tue, 8 Aug 2023 09:53:54 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b="0ypz/M4h"; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=gHwsy0hi; dmarc=none; spf=pass (imf09.hostedemail.com: domain of vbabka@suse.cz designates 195.135.220.28 as permitted sender) smtp.mailfrom=vbabka@suse.cz ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1691488435; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=19m9+YAryd9zQqOJKLPS0ScJg55yhuc9Zn7O2SmAkY8=; b=yy8ZngryVYEUGSTyyUMEY1gtA2Vq/9Fhl/NKDLMCz2hUdSzv9okb1vY+ByCtKYMng0wOUz GidXMG0IT+flc6m+AbGwz5QBLUYiORetfiGmyh+OCVJ0SE4PjvNRux2K2+6mrnNd2ZphYn gVsFICyUIXRX1Uma6nkjh0uZKVZhH7k= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b="0ypz/M4h"; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=gHwsy0hi; dmarc=none; spf=pass (imf09.hostedemail.com: domain of vbabka@suse.cz designates 195.135.220.28 as permitted sender) smtp.mailfrom=vbabka@suse.cz ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1691488435; a=rsa-sha256; cv=none; b=yZl12evcd9c9IxzWG5nDYBF5hkuGTnHuAWRo8S9oZLqhlI2PbHAfsLYK0FEsZR/PXyJ3IR 1XOAgGpsu559/RL8Q5Af//xF1hr7Hprh4UL2r/49gS1ZXvzUel1WuiH2I5Q3nfB0y0XFUT Rm0R1hzpv8XObxHj+Cb8hH9bnbf1bt8= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 7C5062248B; Tue, 8 Aug 2023 09:53:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1691488433; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=19m9+YAryd9zQqOJKLPS0ScJg55yhuc9Zn7O2SmAkY8=; b=0ypz/M4hFEYVg3RIkyKTZpGOtO0OELdvePk4KsvNp0QXKtSQ1NTkwvJ6x9bkVpsk5ACEo+ VdhEFCaw7TRHrCXiaOhP4fR0hWTfeBY2vexIA+Fg3uiY/xr+RGb+kCzEL8ZCXD6x0fD/My 6dCsKBwvmBeEB1+gOW5PonTGZTXEndk= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1691488433; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=19m9+YAryd9zQqOJKLPS0ScJg55yhuc9Zn7O2SmAkY8=; b=gHwsy0hibpgfl2awLPIMsMc4dKqncic8vqqIN/UioyCepHOo4bgnZRyoxdFoRXKIOTGPXa oVyI5GXsppN4AdBQ== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 53972139E9; Tue, 8 Aug 2023 09:53:53 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id eKi8E7EQ0mSBJQAAMHmgww (envelope-from ); Tue, 08 Aug 2023 09:53:53 +0000 From: Vlastimil Babka To: "Liam R. Howlett" , Matthew Wilcox , Christoph Lameter , David Rientjes , Pekka Enberg , Joonsoo Kim Cc: Hyeonggon Yoo <42.hyeyoo@gmail.com>, Roman Gushchin , linux-mm@kvack.org, linux-kernel@vger.kernel.org, patches@lists.linux.dev, Vlastimil Babka Subject: [RFC v1 5/5] maple_tree: replace preallocation with slub percpu array prefill Date: Tue, 8 Aug 2023 11:53:48 +0200 Message-ID: <20230808095342.12637-12-vbabka@suse.cz> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230808095342.12637-7-vbabka@suse.cz> References: <20230808095342.12637-7-vbabka@suse.cz> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: 7qpd5hg1is8zen77m34iismec5bku833 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: C70F7140014 X-HE-Tag: 1691488434-66351 X-HE-Meta: U2FsdGVkX1/+6U+Rqh7IE25Fg0EIVd3dGEisW3Y9098cUQC0OxKyML7qlT7o3c/uhZ40Bh72M2Sj2y93tubT2elfowju4kI6EdPa7t69aVqSlX7PjpvEyO8b2pVESkacVgVuBBNgnVLvL9RIjZzsJf+rzd8wlIPiYGYGCUUGM6XTNTSGKsm8AdVZRCiuvwzv1YZsGszD8ELAefo0k2sku4dNdM6DzYo3YxiyXCu8AxODTk24UWCtJG965WK2DkQJHlX6ES/8rmQgDUC9wXhLOJBW19HLMLU7Qw0xVDhBDNlDxNJauad6nCSyGDPahHJx7oJKfA/VRfu/5EfzMpKQGni6m2UB8Po2If3U2eIy6pbw/DhQurLREeNZUeAKo2HIBvrTdwt4p1Ngwp2j/v9sX5GeRYkoxsqB2eIwgMccFfm2MlPKteoQrucUqyqLc44J9RY/4MvF9sie49MqWkghV7yKAVf9O8EpO7GiWqTXpvjDgqzkviZypemjGY7yw8PX82Jls3NW5/V5piPxYlKNf18vFK/5QpsFjKqQQiC8pNqGFcFFAir+/bYaIzLDp4GfTTSMfaJywmuHMlm0DLJlyD2YkS167Etz+uXU4iRByoxNvQT8nZrAB8UcZebXr5zvKpK4UBwtt/4EoHWv5J0+YAmeBJwhcqV5/v9EnDLAKoLYIP2sIvVcvw4yPXBhfJLWjLggHnWeN4RA0CoyYRCBFg0XupoOYed/nQzT58Gr0Cu54hkqrqw5CJHl62+kRb+tJ4vik0jfEm/q0sq5Vpf1AO/VilRbpazCFIsVoIUA3dWHPojwIqNWR2BdbUJxXVxMy/LHK9EbJ4qfd/kfSpncbE8Rvd70ho1O+d2biX9chchygHRp3Es7y6Hj7zsAGKVTmytN+3JZ5nMoAVYQBXbPhdjyN5lxigt1NwKfJoqDPd3BW6NDJkC5JdFreXHqqfuLWFLhzj5oyAPFi8CGY8g 5p7/XA7r aLs1QY1i5w09R5XpX3E9WWutwSL5SRIc4b9C5VQrI44YbFgSCxURSAfaS9uWF3ZAAir+CqzcttJV5AqyftIgjvehOb0fYhFGskTwNaWLrozNJO1s3DNHRp7iAGhq/9eJgAcj7bj8mxhNuym0f+VAZNgEEOPJogf389GMvSRWfD/05c6ZF+hxO/cJxquxzUdp4xENcQR4x8NkXwJLW/q0BbMWWiwcCAno4sCDo4SfgY4eSNLSzLS1/ZTk1mE52EEMFwPrDtMX/K7WI/h0CG/guj9lWoRAWJ5/5JFVCQVwYd+lgtXdcvWwhHeCAfA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: With the percpu array we can try not doing the preallocations in maple tree, and instead make sure the percpu array is prefilled, and using GFP_ATOMIC in places that relied on the preallocation (in case we miss or fail trylock on the array), i.e. mas_store_prealloc(). For now simply add __GFP_NOFAIL there as well. First I tried to change mas_node_count_gfp() to not preallocate anything anywhere, but that lead to warns and panics, even though the other caller mas_node_count() uses GFP_NOWAIT | __GFP_NOWARN so it has no guarantees... So I changed just mas_preallocate(). I let it still to truly preallocate a single node, but maybe it's not necessary? --- lib/maple_tree.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/lib/maple_tree.c b/lib/maple_tree.c index 7a8e7c467d7c..5a209d88c318 100644 --- a/lib/maple_tree.c +++ b/lib/maple_tree.c @@ -5534,7 +5534,12 @@ void mas_store_prealloc(struct ma_state *mas, void *entry) mas_wr_store_setup(&wr_mas); trace_ma_write(__func__, mas, 0, entry); + +retry: mas_wr_store_entry(&wr_mas); + if (unlikely(mas_nomem(mas, GFP_ATOMIC | __GFP_NOFAIL))) + goto retry; + MAS_WR_BUG_ON(&wr_mas, mas_is_err(mas)); mas_destroy(mas); } @@ -5550,9 +5555,10 @@ EXPORT_SYMBOL_GPL(mas_store_prealloc); int mas_preallocate(struct ma_state *mas, gfp_t gfp) { int ret; + int count = 1 + mas_mt_height(mas) * 3; - mas_node_count_gfp(mas, 1 + mas_mt_height(mas) * 3, gfp); - mas->mas_flags |= MA_STATE_PREALLOC; + mas_node_count_gfp(mas, 1, gfp); + kmem_cache_prefill_percpu_array(maple_node_cache, count, gfp); if (likely(!mas_is_err(mas))) return 0;