From patchwork Thu Jul 11 16:35:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vlastimil Babka X-Patchwork-Id: 13730859 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E91F2C3DA45 for ; Thu, 11 Jul 2024 16:35:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5868B6B007B; Thu, 11 Jul 2024 12:35:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 535F26B0093; Thu, 11 Jul 2024 12:35:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3D72D6B0099; Thu, 11 Jul 2024 12:35:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 1CFE46B007B for ; Thu, 11 Jul 2024 12:35:52 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id C3CFF140719 for ; Thu, 11 Jul 2024 16:35:51 +0000 (UTC) X-FDA: 82328023302.11.C50E9F0 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf26.hostedemail.com (Postfix) with ESMTP id 75D3614002A for ; Thu, 11 Jul 2024 16:35:49 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=none; spf=pass (imf26.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.130 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720715716; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SBVsOcQMGJwDCakmLVBUTWka6/w6YHLNVOwWDvYpTdI=; b=pZGZA9s7SILUB4sQGpPBctdHqeibUtcEJC7cZhCsJ7qz8uMRMaW+uM3x91Zx9eovOxINLW cm4xLjz+jQggrZg7o7NOeyTAYE99V+JfjtOvVy+2tmUZzxmY3/VKqEHNcipZ2wI7VzP8yd lyfSE8iA6kEuzp+YLrwmJ+G7UvjAzPY= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720715716; a=rsa-sha256; cv=none; b=Dw6QP045P2PdGjCdoceHhChK2ajFuMXlHS6xrMgYwyqAnDoroJ1XzIFx6vizxBm6Im/KFL 4UnULtIzYNYbS3eiph+XsgFjDxxYE3yON6fZa7eiWFJwb6OQJ5CyMGrIiKiDMPLzhDBjoc vTlNwV/j0LP5AXhr63f5TV5ZhxeG1Vw= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=none; spf=pass (imf26.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.130 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id B66EA21BDB; Thu, 11 Jul 2024 16:35:47 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 86C80139E7; Thu, 11 Jul 2024 16:35:47 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id ID2lIOMJkGa0NwAAD6G6ig (envelope-from ); Thu, 11 Jul 2024 16:35:47 +0000 From: Vlastimil Babka Date: Thu, 11 Jul 2024 18:35:30 +0200 Subject: [PATCH 1/2] mm, slab: put should_failslab() back behind CONFIG_SHOULD_FAILSLAB MIME-Version: 1.0 Message-Id: <20240711-b4-fault-injection-reverts-v1-1-9e2651945d68@suse.cz> References: <20240711-b4-fault-injection-reverts-v1-0-9e2651945d68@suse.cz> In-Reply-To: <20240711-b4-fault-injection-reverts-v1-0-9e2651945d68@suse.cz> To: Andrew Morton Cc: Mateusz Guzik , Akinobu Mita , Alexei Starovoitov , Daniel Borkmann , John Fastabend , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Christoph Lameter , David Rientjes , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-kernel@vger.kernel.org, bpf@vger.kernel.org, linux-mm@kvack.org, Vlastimil Babka X-Mailer: b4 0.14.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=4644; i=vbabka@suse.cz; h=from:subject:message-id; bh=zbEHVRkDsvgXvgy7ePG+HZP/L/SCK/mB/uExSVUIwQ4=; b=owEBbQGS/pANAwAIAbvgsHXSRYiaAcsmYgBmkAne2MB78FAzF9cge2o8lX8PRgTFUuIDeWIGI vdS6xc0p1CJATMEAAEIAB0WIQR7u8hBFZkjSJZITfG74LB10kWImgUCZpAJ3gAKCRC74LB10kWI mpD+CACkvSZCKzbtwIQGbMaxAGE6arvHPzZDGEKnPm90OsjQ/svV+W/kmwFLbyfc9FYzxoK7Q3+ XeRX2uxBj6CFYm2iYuBMkUlG9xYT/UHDhBfzRXSF503fcD0Px26q9Nt9FlPJu9/5MtWy3kQpTZr 6yO3l/UZ0qWsLBv56Mwncrimgdlg+heHDFAVYpK5OCkwWwoUMyn/7Xd+JdZWBRTnqL2xRTcpGM+ sE/rGJwqp/2ZX1qRjeW/brP+D7fFM/dAmT+hQ8WBkvOF3mLabth3TiaI/mDt/nn5scYFegTcj1+ ZwtvW3N3vnLh8kRVJ2NmmQMdl6Lf1XhMqrKOt4CvjLYnWfwt X-Developer-Key: i=vbabka@suse.cz; a=openpgp; fpr=A940D434992C2E8E99103D50224FA7E7CC82A664 X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Stat-Signature: 634sin3pnfqf94xw6n474gtp33rh76j9 X-Rspamd-Queue-Id: 75D3614002A X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1720715749-372882 X-HE-Meta: U2FsdGVkX1/vszT/r7VS3wJM6b4wX9daxC7g7ABYfLakjnS2IlqIa5mR9TznNBZqk7eP0IwtITq+WyZp3Qmhs4Fi++sCRbKvCmUGNSQYnFaJn+OzbCRYnK8Q5hQzhcgVZQVLEc0xQERYZLkXVp1IpDf+Lgh8ObzNOauG8rEIj3UAjn7fcsObiLmbQ3TYemAWQLEG0BMTrkMeYeREfNJT2+NfiinJU3hmCcfSKDCCPc4ws71n/VjEXsrbuOkoZJ76X0eQ1c4ZGNbfMy2cHHQvpNqtu+Q9YqdiRbKXiQ1N+qbKSUJAVf59UOl//AWa4YRzJA5e7gE8QbwLycO3aCyMF/Cvqw/yAGDdWblVa9f76SQv5ehrhfW+ezmgaeWXo/RArJOBe3KTDUT4cpa3WDNuiDDDOqToE7RWj7ceJrJhqiHJV7Z3LcBEG1ANhUwGTiEOLK82KpWHMhgk0mI/5sRZCkihUkfvj/YoAIng0el4DozzV6SiBXIH9aT6SDV9t2lyBIPqS61Evj4KlAs2qkgtgO4Va//CD32AUNCiZju0ZJz0ThT8GYEBjiYs1o1bvS5/LOzrr8eIEWKA2P3AMUV98iIvsgcqz2+bx5T4yUWeLyZTJ42IyujMYoMs1/oE3VToU4F/emSttyTia07eBzfnn2hYtVkc1IR8oiCIsIG2MCmqAyzhmzVev6kooHi+AteOXZ5PUUs9YpqJqywSSfkTZkXBOFwNGBohTlkaAYCmIxMKz6XR0vrbKwb/2oq9ivzZS/kWpJioYlhtWlnZlZK9k+lnuVFaJc4S7QwMcbF9/VUEKd6SW4+yLxdtpfQs30pGIlmnR5vsJ8HX4tcgZ9NG6/oH1cYc3F5RI2NUCsX19WB7TVAeS1pIKoQ8CiLbXOvkjWIA4UBvO60P0o/ltmHoRBdV+MILsmoEF2mJHwE5ixPim5El0BEb21+OuPgnfAeBrZqQ5GnJCumi6FFCPMm hYDmSz6W v/QYBNPdSwNxdyzSN2dlmgyGji3zvZvhBJ1WihiXVMYPsqmwe4y+/VMEhDJaasrmT0vH3PleUma8yfGW1C0L9qF8nL9LDaMK/qaeWfpdOvHNV9o2WOdCvJ+h+J1Sq1DtLgqPxGkbgddsOpGQqeJc2v4rJKWTEoSN+MDt0khaMPiPqFqRnLCTdYvzoL+O8rHIqquSI9yItlydsljjTGf3BSimhOLPlQRCmtyUmXs3W1Y2gVtiO7A1JkNQxoP0ciC88AAbZ7oC+5Pnuq6zpBP6/l9aGNm2BRmYPBKpE9SNioeFDMAU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This mostly reverts commit 4f6923fbb352 ("mm: make should_failslab always available for fault injection"). The commit made should_failslab() a noinline function that's always called from the slab allocation hotpath, even if it's empty because CONFIG_SHOULD_FAILSLAB is not enabled, and there is no option to disable that call. This is visible in profiles and the function call overhead can be noticeable especially with cpu mitigations. Meanwhile the bpftrace program example in the commit silently does not work without CONFIG_SHOULD_FAILSLAB anyway with a recent gcc, because the empty function gets a .constprop clone that is actually being called (uselessly) from the slab hotpath, while the error injection is hooked to the original function that's not being called at all [1]. Thus put the whole should_failslab() function back behind CONFIG_SHOULD_FAILSLAB. It's not a complete revert of 4f6923fbb352 - the int return type that returns -ENOMEM on failure is preserved, as well ALLOW_ERROR_INJECTION annotation. The BTF_ID() record that was meanwhile added is also guarded by CONFIG_SHOULD_FAILSLAB. [1] https://github.com/bpftrace/bpftrace/issues/3258 Signed-off-by: Vlastimil Babka --- include/linux/fault-inject.h | 5 ++--- kernel/bpf/verifier.c | 2 ++ mm/failslab.c | 14 ++++++++------ mm/slub.c | 8 -------- 4 files changed, 12 insertions(+), 17 deletions(-) diff --git a/include/linux/fault-inject.h b/include/linux/fault-inject.h index 6d5edef09d45..be6d0bc111ad 100644 --- a/include/linux/fault-inject.h +++ b/include/linux/fault-inject.h @@ -102,11 +102,10 @@ static inline bool __should_fail_alloc_page(gfp_t gfp_mask, unsigned int order) } #endif /* CONFIG_FAIL_PAGE_ALLOC */ -int should_failslab(struct kmem_cache *s, gfp_t gfpflags); #ifdef CONFIG_FAILSLAB -extern bool __should_failslab(struct kmem_cache *s, gfp_t gfpflags); +int should_failslab(struct kmem_cache *s, gfp_t gfpflags); #else -static inline bool __should_failslab(struct kmem_cache *s, gfp_t gfpflags) +static inline int should_failslab(struct kmem_cache *s, gfp_t gfpflags) { return false; } diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 214a9fa8c6fb..e455654f3b91 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -21123,7 +21123,9 @@ BTF_SET_START(btf_non_sleepable_error_inject) */ BTF_ID(func, __filemap_add_folio) BTF_ID(func, should_fail_alloc_page) +#ifdef CONFIG_FAILSLAB BTF_ID(func, should_failslab) +#endif BTF_SET_END(btf_non_sleepable_error_inject) static int check_non_sleepable_error_inject(u32 btf_id) diff --git a/mm/failslab.c b/mm/failslab.c index ffc420c0e767..af16c2ed578f 100644 --- a/mm/failslab.c +++ b/mm/failslab.c @@ -1,5 +1,6 @@ // SPDX-License-Identifier: GPL-2.0 #include +#include #include #include #include "slab.h" @@ -14,23 +15,23 @@ static struct { .cache_filter = false, }; -bool __should_failslab(struct kmem_cache *s, gfp_t gfpflags) +int should_failslab(struct kmem_cache *s, gfp_t gfpflags) { int flags = 0; /* No fault-injection for bootstrap cache */ if (unlikely(s == kmem_cache)) - return false; + return 0; if (gfpflags & __GFP_NOFAIL) - return false; + return 0; if (failslab.ignore_gfp_reclaim && (gfpflags & __GFP_DIRECT_RECLAIM)) - return false; + return 0; if (failslab.cache_filter && !(s->flags & SLAB_FAILSLAB)) - return false; + return 0; /* * In some cases, it expects to specify __GFP_NOWARN @@ -41,8 +42,9 @@ bool __should_failslab(struct kmem_cache *s, gfp_t gfpflags) if (gfpflags & __GFP_NOWARN) flags |= FAULT_NOWARN; - return should_fail_ex(&failslab.attr, s->object_size, flags); + return should_fail_ex(&failslab.attr, s->object_size, flags) ? -ENOMEM : 0; } +ALLOW_ERROR_INJECTION(should_failslab, ERRNO); static int __init setup_failslab(char *str) { diff --git a/mm/slub.c b/mm/slub.c index 4927edec6a8c..2e7c7289de91 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -3875,14 +3875,6 @@ static __always_inline void maybe_wipe_obj_freeptr(struct kmem_cache *s, 0, sizeof(void *)); } -noinline int should_failslab(struct kmem_cache *s, gfp_t gfpflags) -{ - if (__should_failslab(s, gfpflags)) - return -ENOMEM; - return 0; -} -ALLOW_ERROR_INJECTION(should_failslab, ERRNO); - static __fastpath_inline struct kmem_cache *slab_pre_alloc_hook(struct kmem_cache *s, gfp_t flags) {