From patchwork Sun Dec 22 07:24:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13918019 Received: from fout-b1-smtp.messagingengine.com (fout-b1-smtp.messagingengine.com [202.12.124.144]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 50CCE8472 for ; Sun, 22 Dec 2024 07:24:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=202.12.124.144 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734852293; cv=none; b=U/5S0LB3kiOKNmHFauTethAmDmdHC/atuvwGS5qh6zfaynt1vM/R+OUXseKE8ICipT/TTw0Sd0NafH12PvbDyBSFeD+BDJ3TeDxmfiHDq2P+rLjwXc0L0IAl7UYBt946e+gekp82H5OiUr/xgbRNPagw3Zk0cTAFrbofyYdRZHw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734852293; c=relaxed/simple; bh=c8MaOICLpqY6eV1Un5D2TVh0n76bgKGymqF8tvvqhv4=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=CdgxtNVsfdaL1QIzM3Am8Qhaivy/qJBNntRV9aqmywKT5zRqIZTCM/9gQGIh+2Ym8nzqQH4C7WT706JaJEXHws5jbE4JoNdPFXgFZb7P+WHtm1eUX6zEoWH+YhtycmouunQxZzTbQg456bcJQJc6OQ+tvY0wC3MmHvMbtyxeIME= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=nPb9/C2u; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=xVYiDOAW; arc=none smtp.client-ip=202.12.124.144 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="nPb9/C2u"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="xVYiDOAW" Received: from phl-compute-12.internal (phl-compute-12.phl.internal [10.202.2.52]) by mailfout.stl.internal (Postfix) with ESMTP id 5EC661140072; Sun, 22 Dec 2024 02:24:51 -0500 (EST) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-12.internal (MEProxy); Sun, 22 Dec 2024 02:24:51 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm2; t=1734852291; x=1734938691; bh=jnPp2/o2ZxHQ1n3s/c/vQTCvNOw1NHKySaFamODYs18=; b= nPb9/C2u/zEzAJDTabEsWkHYA0JzvDsorgeH5P9ASYXXgOh72eBipEqKi2BFqZRf ytY12+OwWjCeoaToOYV/XiG69EIZc7Vqd9AqwyXHiMsNsNv8iHI46G1RvDAXFlup yyDxaL5WLZyF/ze4q9FGmlkguqBXQdJvM9H20fYfIKkVaYLtP2SLmd3y3RhVJGzY seaySeMb8AW4BE2Kp0ovse3FmJcwSHTf9efDQjRkP52BR5OSVFqjfcj6xaP0Z+RP IWRNN1f/D7IydGlSU5BFnzp9NxdSIGuxEqXh7ljz2BvCLRVfsfk8M5ErcKo6K/B+ 8SWL0lTwpMYNXmZ+EyU/lg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1734852291; x= 1734938691; bh=jnPp2/o2ZxHQ1n3s/c/vQTCvNOw1NHKySaFamODYs18=; b=x VYiDOAWRmTUtRB5U5LM3hD0wzi8RojY6SKmFxft5lqRWSunaXI/l3WAIHenftvfH 8YPqQOpCnv2BVZnF8cAkrjOGTQa38MzttD/jE+o2y6XbkHoGdjqoTc8ovIc4tPMV aerEaNnDRCTzoImiW8R2uV46flgwOvXylk/u0lsQ/9AGRrYemOtDkFd4+AUw12ch NyauZG1AyKmlNxf+x4xI6xQSN4ieiED6KRy7IEk4aSK71hS9r/Tl5hqLLNBLzwzo NUHw7By73SdFByDbid2zsM4iKnZ/QKo75NZc5ErXhsmmcr2L5UBK9X5rVpNPa84R HITVCX6y5z/7bkANXBARg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefuddruddtjedgtddvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggvpdfu rfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnh htshculddquddttddmnecujfgurhephfffufggtgfgkfhfjgfvvefosehtjeertdertdej necuhfhrohhmpefrrghtrhhitghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrih hmqeenucggtffrrghtthgvrhhnpeffueeiudejvdekheeuvdekfeffiedvueelteekudeh jeetkeegvddugfdtgfeileenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmh grihhlfhhrohhmpehpshesphhkshdrihhmpdhnsggprhgtphhtthhopeegpdhmohguvgep shhmthhpohhuthdprhgtphhtthhopehgihhtshhtvghrsehpohgsohigrdgtohhmpdhrtg hpthhtohepkhhrihhsthhofhhfvghrhhgruhhgshgsrghkkhesfhgrshhtmhgrihhlrdgt ohhmpdhrtghpthhtohepghhithesvhhgvghrrdhkvghrnhgvlhdrohhrghdprhgtphhtth hopehrrghnuggrlhhlrdgsvggtkhgvrhesnhgvgigsrhhiughgvgdrtggr X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sun, 22 Dec 2024 02:24:50 -0500 (EST) Received: by vm-mail (OpenSMTPD) with ESMTPSA id e2225c99 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Sun, 22 Dec 2024 07:22:51 +0000 (UTC) From: Patrick Steinhardt Date: Sun, 22 Dec 2024 08:24:28 +0100 Subject: [PATCH v2 1/4] reftable/stack: don't perform auto-compaction with less than two tables Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241222-b4-pks-reftable-oom-fix-without-readers-v2-1-19550090d15a@pks.im> References: <20241222-b4-pks-reftable-oom-fix-without-readers-v2-0-19550090d15a@pks.im> In-Reply-To: <20241222-b4-pks-reftable-oom-fix-without-readers-v2-0-19550090d15a@pks.im> To: git@vger.kernel.org Cc: Junio C Hamano , Kristoffer Haugsbakk , "Randall S. Becker" X-Mailer: b4 0.14.2 In order to compact tables we need at least two tables. Bail out early from `reftable_stack_auto_compact()` in case we have less than two tables. In the original, `stack_table_sizes_for_compaction()` yields an array that has the same length as the number of tables. This array is then passed on to `suggest_compaction_segment()`, which returns an empty segment in case we have less than two tables. The segment is then passed to `segment_size()`, which will return `0` because both start and end of the segment are `0`. And because we only call `stack_compact_range()` in case we have a positive segment size we don't perform auto-compaction at all. Consequently, this change does not result in a user-visible change in behaviour when called with a single table. But when called with no tables this protects us against a potential out-of-memory error: `stack_table_sizes_for_compaction()` would try to allocate a zero-byte object when there aren't any tables, and that may lead to a `NULL` pointer on some platforms like NonStop which causes us to bail out with an out-of-memory error. Signed-off-by: Patrick Steinhardt --- reftable/stack.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/reftable/stack.c b/reftable/stack.c index 59fd695a12c2033ed589a21ef1c9155eeecc4641..6ca21965d8e1135d986043113d465abd14cd532c 100644 --- a/reftable/stack.c +++ b/reftable/stack.c @@ -1627,6 +1627,9 @@ int reftable_stack_auto_compact(struct reftable_stack *st) struct segment seg; uint64_t *sizes; + if (st->merged->readers_len < 2) + return 0; + sizes = stack_table_sizes_for_compaction(st); if (!sizes) return REFTABLE_OUT_OF_MEMORY_ERROR; From patchwork Sun Dec 22 07:24:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13918020 Received: from fout-b1-smtp.messagingengine.com (fout-b1-smtp.messagingengine.com [202.12.124.144]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E3ACE15B554 for ; Sun, 22 Dec 2024 07:24:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=202.12.124.144 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734852295; cv=none; b=T8OVQVNz8PTKCcwuqPNg5Y0dB/eMD16UN2Y9/gf9WybkUgFq33XxQ/D71VtjJEFcmDNJ5TJISvgdyEql4aUqvyQfi1LGt5NcBEccq0B2+RIJDL4S1V4p9ZM531/efe3bTR8WEIdTAG0eNFylZoJ3uemA4Ae/kJYulfSWbkgJQxw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734852295; c=relaxed/simple; bh=wzixTvMRTYVAde8xzENZMNKqQ4ftU01bPEFhhl1TM/s=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Iuxup+eYFssCXcNb1P02QjQaVtxCeXz2aqIa6BP8v8txhwtAi7FXhEMspC4GUkpUfDuMBaVkh7l7pIZrWVG6tN3we329lIhZ+anegJVsKTKYqh+w6aKtCUJpESidVUFQEAXkP4pfVMVk412288MILl7GpD25SpbKmFqIg6UY6IE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=ZPlC2OOb; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=hcosWlXe; arc=none smtp.client-ip=202.12.124.144 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="ZPlC2OOb"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="hcosWlXe" Received: from phl-compute-08.internal (phl-compute-08.phl.internal [10.202.2.48]) by mailfout.stl.internal (Postfix) with ESMTP id E7A78114017B; Sun, 22 Dec 2024 02:24:52 -0500 (EST) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-08.internal (MEProxy); Sun, 22 Dec 2024 02:24:53 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm2; t=1734852292; x=1734938692; bh=bWOmNsq890tCQ1Ux3Gv89ne0GrtNjyo/ZZyuT1rkEVs=; b= ZPlC2OObj8FoOlUqA3RZVzyD9BgL6pEHvtlEJwgc4PAZXe5/lsObTMT97wOb5d8B PhqIdefkjT6qTo7QtoeluAMkuIRozOW9IXlh8x28S0gyDnuV/eHd2sBBbq3OkCY0 YFbDwMCvNf74bcoJzq14sH08tvXPzKXrwk6O2PP/UmaasOM9P07WPtw1WsNxBID4 6Gsyy5feJpJSmncIjx3hVgLbN3DVcS4LN8/xwh3MKIQEya/WYc81/+UIvWk14UJE fFDLC9WIfKaJcg2KVUQFSxafQRXgKMozNp+wcV1Ng/ZVSotmT1x7uHe/f/l18OJH hOS43w+7NsRjzat9YnYfhQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1734852292; x= 1734938692; bh=bWOmNsq890tCQ1Ux3Gv89ne0GrtNjyo/ZZyuT1rkEVs=; b=h cosWlXeBkxGSnot1vqRRD2zHeFTV+YT6rQofkOibdO1uBiZA6CCWEgzevAPHBcdc QhmTwKCNlknhipf12+SItmcUB/Sfx4iveUkJN8iB3x8/brMIbwEi5+npZxnM6AAI QInBXeSsz1c9OCzJqcwGA72FEaMyvDzp4+B0iKvDcen+GUxcfqYcppKCQEUPLJGg mPxeJhcXdxyCUksX9mp2wSFus+eWwik2qOD9/tI372FndvhV+yzmq6bh3aMlNRua jiMnjVBDELaRNMZkmGQA1WlrM8Xn9US0Ok/WHf+ijSe0HALwE56opvrw5ctPJEdp /QXYhpJf/UHt8sQ4FIdYw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefuddruddtjedgtddvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggvpdfu rfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnh htshculddquddttddmnecujfgurhephfffufggtgfgkfhfjgfvvefosehtjeertdertdej necuhfhrohhmpefrrghtrhhitghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrih hmqeenucggtffrrghtthgvrhhnpeffueeiudejvdekheeuvdekfeffiedvueelteekudeh jeetkeegvddugfdtgfeileenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmh grihhlfhhrohhmpehpshesphhkshdrihhmpdhnsggprhgtphhtthhopeegpdhmohguvgep shhmthhpohhuthdprhgtphhtthhopehgihhtsehvghgvrhdrkhgvrhhnvghlrdhorhhgpd hrtghpthhtohepghhithhsthgvrhesphhosghogidrtghomhdprhgtphhtthhopehrrghn uggrlhhlrdgsvggtkhgvrhesnhgvgigsrhhiughgvgdrtggrpdhrtghpthhtohepkhhrih hsthhofhhfvghrhhgruhhgshgsrghkkhesfhgrshhtmhgrihhlrdgtohhm X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sun, 22 Dec 2024 02:24:51 -0500 (EST) Received: by vm-mail (OpenSMTPD) with ESMTPSA id 2578ebd7 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Sun, 22 Dec 2024 07:22:52 +0000 (UTC) From: Patrick Steinhardt Date: Sun, 22 Dec 2024 08:24:29 +0100 Subject: [PATCH v2 2/4] reftable/merged: fix zero-sized allocation when there are no readers Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241222-b4-pks-reftable-oom-fix-without-readers-v2-2-19550090d15a@pks.im> References: <20241222-b4-pks-reftable-oom-fix-without-readers-v2-0-19550090d15a@pks.im> In-Reply-To: <20241222-b4-pks-reftable-oom-fix-without-readers-v2-0-19550090d15a@pks.im> To: git@vger.kernel.org Cc: Junio C Hamano , Kristoffer Haugsbakk , "Randall S. Becker" X-Mailer: b4 0.14.2 It was reported [1] that Git started to fail with an out-of-memory error when initializing repositories with the reftable backend on NonStop platforms. A bisect led to 802c0646ac (reftable/merged: handle allocation failures in `merged_table_init_iter()`, 2024-10-02), which changed how we allocate memory when initializing a merged table. The root cause of this seems to be that NonStop returns a `NULL` pointer when doing a zero-sized allocation. This would've already happened before the above change, but we never noticed because we did not check the result. Now we do notice and thus return an out-of-memory error to the caller. Fix the issue by skipping the allocation altogether in case there are no readers. [1]: <00ad01db5017$aa9ce340$ffd6a9c0$@nexbridge.com> Reported-by: Randall S. Becker Signed-off-by: Patrick Steinhardt --- reftable/merged.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/reftable/merged.c b/reftable/merged.c index bb0836e3443271f9c0d5ba5582c78694d437ddc2..e72b39e178d4dec6ddfca970b5af71b71431397a 100644 --- a/reftable/merged.c +++ b/reftable/merged.c @@ -240,14 +240,16 @@ int merged_table_init_iter(struct reftable_merged_table *mt, struct reftable_iterator *it, uint8_t typ) { - struct merged_subiter *subiters; + struct merged_subiter *subiters = NULL; struct merged_iter *mi = NULL; int ret; - REFTABLE_CALLOC_ARRAY(subiters, mt->readers_len); - if (!subiters) { - ret = REFTABLE_OUT_OF_MEMORY_ERROR; - goto out; + if (mt->readers_len) { + REFTABLE_CALLOC_ARRAY(subiters, mt->readers_len); + if (!subiters) { + ret = REFTABLE_OUT_OF_MEMORY_ERROR; + goto out; + } } for (size_t i = 0; i < mt->readers_len; i++) { From patchwork Sun Dec 22 07:24:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13918021 Received: from fhigh-b2-smtp.messagingengine.com (fhigh-b2-smtp.messagingengine.com [202.12.124.153]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B5850171658 for ; Sun, 22 Dec 2024 07:24:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=202.12.124.153 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734852296; cv=none; b=oO2o0foK+ocPxcnlqzkasba2T6S6YJIvhlopKGvFBzwQkw4qCF60hvvGEl7g1jDCkIrQg8qMYy8hz6c43XiflfBCpiC03CqwdU5YzZMvQ5Zkm4HvtsVrDWch4l7ozxDfS8s29cyJbURoLJgXjRLtZFNmEMFLH90CIIqBzzMV6eg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734852296; c=relaxed/simple; bh=jUN4j4x4e8Vy3aqnelpWQvOGVcV/vJpZOw9tIZzNMY8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Gjd9FTFKyrl+CvfW97yGX6k2dZCCmaJ8AYXwbw/vnDPqaePizujt54BqwfzrwthU7g2GDD+Vl6yVA/ylU4nDa+zyhAIF8MWcCvMe5SkQ7EY/Wrr/ra1zq4b9Tu5NpmHw62Ou0T8nyYth/086d7uhHVFiry0vdDw4crIqA1nMoZY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=iiYguMqH; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=BZSgiPWu; arc=none smtp.client-ip=202.12.124.153 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="iiYguMqH"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="BZSgiPWu" Received: from phl-compute-11.internal (phl-compute-11.phl.internal [10.202.2.51]) by mailfhigh.stl.internal (Postfix) with ESMTP id 89F1D2540189; Sun, 22 Dec 2024 02:24:53 -0500 (EST) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-11.internal (MEProxy); Sun, 22 Dec 2024 02:24:53 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm2; t=1734852293; x=1734938693; bh=DxeckagWI7MDIzubYk7GkEXi4fHJZ692S6m5GYLFQfs=; b= iiYguMqHlwg/yma2D4swoH4FgEs7vHPeBA/KRlzYW+wKIltEKSgNQ32aDpiFGd+1 jIkVlzjGbOVGDiby2C08EboCsEjCnL7827XGZgZLkkiXluD4Jw4aTCnP0go0RUdm GZxk4IUDne88o7pzyHMetYystIUnQlzA7/iWbJUEpKvV2wON0VJFm0sOQRuvUrLr GjssQeUyHk6tiNcA0TrCBWpq6ZBXuXP7f+y3VDO1AAZmisWG20hfXBpU3vyfkJ8v q9sWdHQKwzJOt0urIJZPBBW6vSGwmWA5oo5lClsajb/PCSaMhkVKIUaHn+U+qVY7 aTb4RyS6vnUgi4GOSI2rrQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1734852293; x= 1734938693; bh=DxeckagWI7MDIzubYk7GkEXi4fHJZ692S6m5GYLFQfs=; b=B ZSgiPWu7GXgJo3e4IEGabnsnrhGnoopDybhr20NqR3Jm/jrp9aQ5IRKjI80o7+y8 sZWmv0Ajp4O0+hCfWiqDch+z6cLG5nVqbe7UpxohgC8EYe23AeN5bRGpSkbJ/yXw kijjc0Svue3rxvDclD6WQ1z586sMVbIsUIeEM9bbSIrt+W9i5XEbXh3bHNtxRj8y 9KhMIgydwRulmkOWqVNqhp3E40YHAsEd2K4t8Sc+vOqB3w/JhLOO1hbTtQIUHN6P 430FuNKM/BzeeJi+TdTE9ArmZaLhUEjbGvo5GZxIuUhMA5u1cELKKGeR/8ttymzF PfRbOygtbybrQ0/J0jDVw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefuddruddtjedgtddvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggvpdfu rfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnh htshculddquddttddmnecujfgurhephfffufggtgfgkfhfjgfvvefosehtjeertdertdej necuhfhrohhmpefrrghtrhhitghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrih hmqeenucggtffrrghtthgvrhhnpeffueeiudejvdekheeuvdekfeffiedvueelteekudeh jeetkeegvddugfdtgfeileenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmh grihhlfhhrohhmpehpshesphhkshdrihhmpdhnsggprhgtphhtthhopeegpdhmohguvgep shhmthhpohhuthdprhgtphhtthhopehkrhhishhtohhffhgvrhhhrghughhssggrkhhkse hfrghsthhmrghilhdrtghomhdprhgtphhtthhopehrrghnuggrlhhlrdgsvggtkhgvrhes nhgvgigsrhhiughgvgdrtggrpdhrtghpthhtohepghhithesvhhgvghrrdhkvghrnhgvlh drohhrghdprhgtphhtthhopehgihhtshhtvghrsehpohgsohigrdgtohhm X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sun, 22 Dec 2024 02:24:52 -0500 (EST) Received: by vm-mail (OpenSMTPD) with ESMTPSA id 553e6da4 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Sun, 22 Dec 2024 07:22:53 +0000 (UTC) From: Patrick Steinhardt Date: Sun, 22 Dec 2024 08:24:30 +0100 Subject: [PATCH v2 3/4] reftable/stack: fix zero-sized allocation when there are no readers Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241222-b4-pks-reftable-oom-fix-without-readers-v2-3-19550090d15a@pks.im> References: <20241222-b4-pks-reftable-oom-fix-without-readers-v2-0-19550090d15a@pks.im> In-Reply-To: <20241222-b4-pks-reftable-oom-fix-without-readers-v2-0-19550090d15a@pks.im> To: git@vger.kernel.org Cc: Junio C Hamano , Kristoffer Haugsbakk , "Randall S. Becker" X-Mailer: b4 0.14.2 Similar as the preceding commit, we may try to do a zero-sized allocation when reloading a reftable stack that ain't got any tables. It is implementation-defined whether malloc(3p) returns a NULL pointer in that case or a zero-sized object. In case it does return a NULL pointer though it causes us to think we have run into an out-of-memory situation, and thus we return an error. Fix this by only allocating arrays when they have at least one entry. Reported-by: Randall S. Becker Signed-off-by: Patrick Steinhardt --- reftable/stack.c | 24 ++++++++++++++---------- 1 file changed, 14 insertions(+), 10 deletions(-) diff --git a/reftable/stack.c b/reftable/stack.c index 6ca21965d8e1135d986043113d465abd14cd532c..634f0c54251b3581ca73250aca9f653f4645a569 100644 --- a/reftable/stack.c +++ b/reftable/stack.c @@ -270,9 +270,9 @@ static int reftable_stack_reload_once(struct reftable_stack *st, int reuse_open) { size_t cur_len = !st->merged ? 0 : st->merged->readers_len; - struct reftable_reader **cur; + struct reftable_reader **cur = NULL; struct reftable_reader **reused = NULL; - struct reftable_reader **new_readers; + struct reftable_reader **new_readers = NULL; size_t reused_len = 0, reused_alloc = 0, names_len; size_t new_readers_len = 0; struct reftable_merged_table *new_merged = NULL; @@ -280,18 +280,22 @@ static int reftable_stack_reload_once(struct reftable_stack *st, int err = 0; size_t i; - cur = stack_copy_readers(st, cur_len); - if (!cur) { - err = REFTABLE_OUT_OF_MEMORY_ERROR; - goto done; + if (cur_len) { + cur = stack_copy_readers(st, cur_len); + if (!cur) { + err = REFTABLE_OUT_OF_MEMORY_ERROR; + goto done; + } } names_len = names_length(names); - new_readers = reftable_calloc(names_len, sizeof(*new_readers)); - if (!new_readers) { - err = REFTABLE_OUT_OF_MEMORY_ERROR; - goto done; + if (names_len) { + new_readers = reftable_calloc(names_len, sizeof(*new_readers)); + if (!new_readers) { + err = REFTABLE_OUT_OF_MEMORY_ERROR; + goto done; + } } while (*names) { From patchwork Sun Dec 22 07:24:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13918022 Received: from fhigh-b2-smtp.messagingengine.com (fhigh-b2-smtp.messagingengine.com [202.12.124.153]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7AAED17B421 for ; Sun, 22 Dec 2024 07:24:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=202.12.124.153 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734852297; cv=none; b=SspgOivT5hL49csqE1dShMqQWLq1GhpySCUoAEcyPxG4YJMFuK8WjU+s8Z2YOp7ySbdiQXc702dL0B9ikhu7jq3mz/iiFKDnl7xOqjci+0WFmbbO12j2cs7kGksTDpCtYZH5XrB8AIoWL/mbcq/vbBDn9Qcb6CCeOz8dRBTkEGc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734852297; c=relaxed/simple; bh=B72wCzzjAMScPASrFs+lO/j/sySeBigjXsrrfoNVCSE=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Ptkh/M89kB+1zvLuv0GpY845DZjvBNLwCayVyIrAHZsrLj0XOdSy0SzN8lU0xTIxit3q1VMu0UVDOPHI/penfi+0ZVZ1Ue1z6ptDUfWv0fQ0a9kwYqLsv2xw9SPTosn5bYKziozlI3xKphcTwvul2d6tcmVW5Wez+j+TvJQw4aw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=O2zSzp2m; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=Ov58S4Bt; arc=none smtp.client-ip=202.12.124.153 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="O2zSzp2m"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="Ov58S4Bt" Received: from phl-compute-04.internal (phl-compute-04.phl.internal [10.202.2.44]) by mailfhigh.stl.internal (Postfix) with ESMTP id 79C56254018A; Sun, 22 Dec 2024 02:24:54 -0500 (EST) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-04.internal (MEProxy); Sun, 22 Dec 2024 02:24:54 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm2; t=1734852294; x=1734938694; bh=vM4VanXwFujuK2w2Psv62FAZc8cPmTAe1U8IwEHjapI=; b= O2zSzp2mZZYFqPc6iTaEyFgyOt5u8zh2Q5Nyng1EDmAvJwne5OXu/oUiasFDTtyH MySlYap+aNNU7veRAJdUOD4lGmwDFtzvvFa+i2ppaS7FMjqFXm4ZUIHg9LeYLUh5 DUuzXv935+duwalXdmFDY/mkDms7LlwQ9MwKai1T7a4yOnVQNhzMUpX6mfDmMdBL 7mtOJqNP66vrKjEzvWcFuLTmhwzNw98VTrbjRYK0GRbBifDsWbuJGSEkXG8NAIRS KEBX8ndbffDpw4H+yDa/y2wQNbZyDVUMfAJscGzCG8xjSHtm7MCe0Eez3QK/RoyM gCHiQWpABvWZ6auh0dndGQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1734852294; x= 1734938694; bh=vM4VanXwFujuK2w2Psv62FAZc8cPmTAe1U8IwEHjapI=; b=O v58S4BttPCjBPjA4A48P4zKlLMA3TjtGTI8gBEtoEW7r4heVXSBkxho9ZdwtcfRr 7Pald6bDmWdlUtixRITeEiGWwfwbtYPUj4dbAVA7h97iBFOwQN93gusO85Su59rp hHAKiiqGzm3dZyktHLLuELwrW8xwWGWJmPvkF37jqOjXzHpO3raEWCz9xdSIq3S8 BFTSCYoNL2tai8OWVOpKskZwiaIfUMAkSJwHvbgC50JwJ55ofyhFy124RGdT+l53 qyXdExDqlRgESZEghMTUdUO8K8auC5gYUXf5cdfEYVSnjv0mgKtW061Q7bVZDHL4 qQ4yngBWEBzftNkabq6mQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefuddruddtjedgtddvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggvpdfu rfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnh htshculddquddttddmnecujfgurhephfffufggtgfgkfhfjgfvvefosehtjeertdertdej necuhfhrohhmpefrrghtrhhitghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrih hmqeenucggtffrrghtthgvrhhnpeffueeiudejvdekheeuvdekfeffiedvueelteekudeh jeetkeegvddugfdtgfeileenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmh grihhlfhhrohhmpehpshesphhkshdrihhmpdhnsggprhgtphhtthhopeegpdhmohguvgep shhmthhpohhuthdprhgtphhtthhopehkrhhishhtohhffhgvrhhhrghughhssggrkhhkse hfrghsthhmrghilhdrtghomhdprhgtphhtthhopehrrghnuggrlhhlrdgsvggtkhgvrhes nhgvgigsrhhiughgvgdrtggrpdhrtghpthhtohepghhithesvhhgvghrrdhkvghrnhgvlh drohhrghdprhgtphhtthhopehgihhtshhtvghrsehpohgsohigrdgtohhm X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sun, 22 Dec 2024 02:24:53 -0500 (EST) Received: by vm-mail (OpenSMTPD) with ESMTPSA id 00e48c19 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Sun, 22 Dec 2024 07:22:54 +0000 (UTC) From: Patrick Steinhardt Date: Sun, 22 Dec 2024 08:24:31 +0100 Subject: [PATCH v2 4/4] reftable/basics: return NULL on zero-sized allocations Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241222-b4-pks-reftable-oom-fix-without-readers-v2-4-19550090d15a@pks.im> References: <20241222-b4-pks-reftable-oom-fix-without-readers-v2-0-19550090d15a@pks.im> In-Reply-To: <20241222-b4-pks-reftable-oom-fix-without-readers-v2-0-19550090d15a@pks.im> To: git@vger.kernel.org Cc: Junio C Hamano , Kristoffer Haugsbakk , "Randall S. Becker" X-Mailer: b4 0.14.2 In the preceding commits we have fixed a couple of issues when allocating zero-sized objects. These issues were masked by implementation-defined behaviour. Quoting malloc(3p): If size is 0, either: * A null pointer shall be returned and errno may be set to an implementation-defined value, or * A pointer to the allocated space shall be returned. The application shall ensure that the pointer is not used to access an object. So it is perfectly valid that implementations of this function may or may not return a NULL pointer in such a case. Adapt both `reftable_malloc()` and `reftable_realloc()` so that they return NULL pointers on zero-sized allocations. This should remove any implementation-defined behaviour in our allocators and thus allows us to detect such platform-specific issues more easily going forward. Signed-off-by: Patrick Steinhardt --- reftable/basics.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/reftable/basics.c b/reftable/basics.c index 7d84a5d62dead1cf1a60698b1bb12fe6ac41c090..70b1091d1495bb5b4c8aae63bd9213dc704aecde 100644 --- a/reftable/basics.c +++ b/reftable/basics.c @@ -17,6 +17,8 @@ static void (*reftable_free_ptr)(void *); void *reftable_malloc(size_t sz) { + if (!sz) + return NULL; if (reftable_malloc_ptr) return (*reftable_malloc_ptr)(sz); return malloc(sz); @@ -24,6 +26,11 @@ void *reftable_malloc(size_t sz) void *reftable_realloc(void *p, size_t sz) { + if (!sz) { + reftable_free(p); + return NULL; + } + if (reftable_realloc_ptr) return (*reftable_realloc_ptr)(p, sz); return realloc(p, sz);