From patchwork Wed Jan 8 19:14:32 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13931493 Received: from mail-yb1-f180.google.com (mail-yb1-f180.google.com [209.85.219.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9CDC2202C25 for ; Wed, 8 Jan 2025 19:14:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736363677; cv=none; b=cqEpY+u1+Am0XGMdBLToT9btDA6LpeS1wN7I6O8WV9H/PZ5CcCgj90tQe+0tftxLtAbydBX9sMTUbbcSjsuSMgQNGpdc6yC1hEY4AnIFEabMVb0lI2dB2ZIXg9H7iLNNe4T7q5bdiMN/XM+QUlHFMJQi2kJl2c1c9l7/UKcQR7I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736363677; c=relaxed/simple; bh=/hI/VO+IIDfdkwDz/wVXehrZAu9rGcjhNDYmybrUDgk=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=pz3DZQ5fX34u2lasillsKiQnXWaV/BKM1b/Ufr3ywbjIt0phtq52rH4+OOQAOwnLyyuWLtqCq0AStMDqEaZ1tIr0YyOIJKwLJATyF0nRWIRA1JH/QBXQsscRgb5YmivcTxzEUmS4PVSXZbZapL/Q9KCQinIcdZu9jK3+7KpgFtg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com; spf=pass smtp.mailfrom=ttaylorr.com; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b=02L+rK9r; arc=none smtp.client-ip=209.85.219.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b="02L+rK9r" Received: by mail-yb1-f180.google.com with SMTP id 3f1490d57ef6-e46ac799015so100285276.0 for ; Wed, 08 Jan 2025 11:14:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1736363674; x=1736968474; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=BETTMx/ufgLN7/xuQO/U961kXk4u6htxwTmP1PEfcJg=; b=02L+rK9r75L4mkraeAmsh4bV1t/2haY6t7wCSiiiLWS1S+geT0aPfFGDewDy2P231g z/Df0cw7s8NfyOypUnnBBtOfAKLM39aqSTJOxpRcu65qZel1+PDX+J421KUFuW21bUSt WFP7r62/UKjuLw4l8AmlCkUNA5GFLbbyAWymXi9keDcuOjBbr/l9Fbynk5akXIP50gES Jr7cuEQkCRRtAbeX51h0cpesZ0OSIqE0yHeuR8tZmUGqt6uf2sL9jkpjYaph71LEw6DA 2it9X6rWdH1nSuLsu39UA1QtHTY+lMGgZkr6f7QnmltZPEhI2ykBSj1XbyN1BFsHREvx kLuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736363674; x=1736968474; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=BETTMx/ufgLN7/xuQO/U961kXk4u6htxwTmP1PEfcJg=; b=f8RO8+UwNlbBipTzFAj+EJjVO+mgnRbo2t6y0AoiFK1CNBTzR51OJrY2L16eba9oiK dAFXymB3nJFguds5KUojLA488AMA14gx/nAJOPA+w8apl44vUVaIw9ui/PX7oABoL/Gr ZfeldXJ3gLeJGRW7VlbrH5afSWveasukrqlOQJ/DKJalXf5rOpLSNc4B6pB3R6dU3Hq2 nrY69GJPXXFWvq6PVy9cbQzHpjlb9nHZBEVgeetzoQNHnJ0Kt7VME13ogvBnOoXH4aXK fd6jEbqv0OG3PlhOBiBZuU4nbZ4L4kt69nrC5x11HEfFoH2tZJs0xk0kpWjM2UleAyKn vMcg== X-Gm-Message-State: AOJu0Ywy/5ByhBM0Dbj/DYW1vZVydBMRy6Mtxye2+OFNivOPGVPjyDfa JBSfzi0jPORbMA48TkdAXYOYLiUvjFWvkCT2oevuEM8BypEviLFfshJed+AuyoWDKwmKKgmqgP/ 0M7Q= X-Gm-Gg: ASbGncvmSdJY5eyedSICTjMyZsPrZAUbPUAgRpH6xhwR1WZ7mGf8mcvds8pG/RaYz9R UwEak41KoCT3BMpklxN7XtKTLZUKGcuX2bpbmxrpTjPSGFgw7Cv95vpZC+X/eMTSu3klvCFYmtc UasHshY6YODTtROALiCPjcO18vAnHQ5vLxES5mUNDwToPkJzXJz1W1cnkKrHxIDgGGs6YLcDDMG PSI69lJmndKGgS/OthO8wkJ7FLjvN+9YlwNAHoDeYHL3WPFOBEjUmDbLMG6Vh44WXoDk5Qx2/hV e8e3SkFoBQpf/2Xw0JLsU4CuDP9k X-Google-Smtp-Source: AGHT+IHQz7xU527gj7XQPOFHbmWSaik7AdpjI1hz4I4+KfPRQfr6XSOijG9ephbNXyN6BUVajWewtw== X-Received: by 2002:a05:690c:61c1:b0:6f4:2794:6626 with SMTP id 00721157ae682-6f531203891mr42233797b3.2.1736363674223; Wed, 08 Jan 2025 11:14:34 -0800 (PST) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id 00721157ae682-6f3e77ed1f2sm94698547b3.72.2025.01.08.11.14.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jan 2025 11:14:33 -0800 (PST) Date: Wed, 8 Jan 2025 14:14:32 -0500 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Junio C Hamano , Elijah Newren , Patrick Steinhardt Subject: [PATCH v2 1/8] t/helper/test-tool: implement sha1-unsafe helper Message-ID: <4c1523a04f1b4750da38f5e2149109753d84022b.1736363652.git.me@ttaylorr.com> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: With the new "unsafe" SHA-1 build knob, it is convenient to have a test-tool that can exercise Git's unsafe SHA-1 wrappers for testing, similar to 't/helper/test-tool sha1'. Implement that helper by altering the implementation of that test-tool (in cmd_hash_impl(), which is generic and parameterized over different hash functions) to conditionally run the unsafe variants of the chosen hash function, and expose the new behavior via a new 'sha1-unsafe' test helper. Signed-off-by: Taylor Blau --- t/helper/test-hash.c | 17 +++++++++++++---- t/helper/test-sha1.c | 7 ++++++- t/helper/test-sha1.sh | 38 ++++++++++++++++++++++---------------- t/helper/test-sha256.c | 2 +- t/helper/test-tool.c | 1 + t/helper/test-tool.h | 3 ++- 6 files changed, 45 insertions(+), 23 deletions(-) diff --git a/t/helper/test-hash.c b/t/helper/test-hash.c index 45d829c908f..d0ee668df95 100644 --- a/t/helper/test-hash.c +++ b/t/helper/test-hash.c @@ -1,7 +1,7 @@ #include "test-tool.h" #include "hex.h" -int cmd_hash_impl(int ac, const char **av, int algo) +int cmd_hash_impl(int ac, const char **av, int algo, int unsafe) { git_hash_ctx ctx; unsigned char hash[GIT_MAX_HEXSZ]; @@ -27,7 +27,10 @@ int cmd_hash_impl(int ac, const char **av, int algo) die("OOPS"); } - algop->init_fn(&ctx); + if (unsafe) + algop->unsafe_init_fn(&ctx); + else + algop->init_fn(&ctx); while (1) { ssize_t sz, this_sz; @@ -46,9 +49,15 @@ int cmd_hash_impl(int ac, const char **av, int algo) } if (this_sz == 0) break; - algop->update_fn(&ctx, buffer, this_sz); + if (unsafe) + algop->unsafe_update_fn(&ctx, buffer, this_sz); + else + algop->update_fn(&ctx, buffer, this_sz); } - algop->final_fn(hash, &ctx); + if (unsafe) + algop->unsafe_final_fn(hash, &ctx); + else + algop->final_fn(hash, &ctx); if (binary) fwrite(hash, 1, algop->rawsz, stdout); diff --git a/t/helper/test-sha1.c b/t/helper/test-sha1.c index e60d000c039..349540c4df8 100644 --- a/t/helper/test-sha1.c +++ b/t/helper/test-sha1.c @@ -3,7 +3,7 @@ int cmd__sha1(int ac, const char **av) { - return cmd_hash_impl(ac, av, GIT_HASH_SHA1); + return cmd_hash_impl(ac, av, GIT_HASH_SHA1, 0); } int cmd__sha1_is_sha1dc(int argc UNUSED, const char **argv UNUSED) @@ -13,3 +13,8 @@ int cmd__sha1_is_sha1dc(int argc UNUSED, const char **argv UNUSED) #endif return 1; } + +int cmd__sha1_unsafe(int ac, const char **av) +{ + return cmd_hash_impl(ac, av, GIT_HASH_SHA1, 1); +} diff --git a/t/helper/test-sha1.sh b/t/helper/test-sha1.sh index 84594885c70..bf387d3db14 100755 --- a/t/helper/test-sha1.sh +++ b/t/helper/test-sha1.sh @@ -3,25 +3,31 @@ dd if=/dev/zero bs=1048576 count=100 2>/dev/null | /usr/bin/time t/helper/test-tool sha1 >/dev/null +dd if=/dev/zero bs=1048576 count=100 2>/dev/null | +/usr/bin/time t/helper/test-tool sha1-unsafe >/dev/null + while read expect cnt pfx do case "$expect" in '#'*) continue ;; esac - actual=$( - { - test -z "$pfx" || echo "$pfx" - dd if=/dev/zero bs=1048576 count=$cnt 2>/dev/null | - perl -pe 'y/\000/g/' - } | ./t/helper/test-tool sha1 $cnt - ) - if test "$expect" = "$actual" - then - echo "OK: $expect $cnt $pfx" - else - echo >&2 "OOPS: $cnt" - echo >&2 "expect: $expect" - echo >&2 "actual: $actual" - exit 1 - fi + for sha1 in sha1 sha1-unsafe + do + actual=$( + { + test -z "$pfx" || echo "$pfx" + dd if=/dev/zero bs=1048576 count=$cnt 2>/dev/null | + perl -pe 'y/\000/g/' + } | ./t/helper/test-tool $sha1 $cnt + ) + if test "$expect" = "$actual" + then + echo "OK ($sha1): $expect $cnt $pfx" + else + echo >&2 "OOPS ($sha1): $cnt" + echo >&2 "expect ($sha1): $expect" + echo >&2 "actual ($sha1): $actual" + exit 1 + fi + done done < X-Patchwork-Id: 13931494 Received: from mail-yb1-f179.google.com (mail-yb1-f179.google.com [209.85.219.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B183B202F61 for ; Wed, 8 Jan 2025 19:14:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736363680; cv=none; b=cOw8ksaendubvkRVGbb921NWexNG3aJwHQGSAa9SrKv0xRwRahW5mJhfuJO3e/kJtd/FHBdUZ6kIQnA5RnkVGPObqI+PxT7lKvsHu9xeSWxNAohKbtdoAuKpbxFlkLYoZ2JqQqU46u91il4USYC/tf1raprh4axkaUxYOzdRQXI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736363680; c=relaxed/simple; bh=UK7RePgewRyvzvEnssRzE9utl09yZotL7Ouu9D4hgZ4=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=W9QiPGrlVPPva3qFo6Ep9EhVrajhd1TTvTVC+QD2UaBulZv/ssFsqhOTe1MiXq3GQ72Vo4tT58tbBrv4lEQlu3saI6ZDGN5erh/tpgykha+/SaT6VTkoTYacT2gAeF5DxbDWFPg4KjXPfgpCg/2OODCTaOZdamBdQW2WvuWW6hQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com; spf=pass smtp.mailfrom=ttaylorr.com; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b=IMnaievt; arc=none smtp.client-ip=209.85.219.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b="IMnaievt" Received: by mail-yb1-f179.google.com with SMTP id 3f1490d57ef6-e39f43344c5so96558276.1 for ; Wed, 08 Jan 2025 11:14:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1736363677; x=1736968477; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=tgwDwIjUXMZMpDhITbnkdObRyaeVei/v3zDkEhXmicI=; b=IMnaievtYaX/00vmaSbPsl/fNJElb7uR+WF8ZqN8O5cmk4YvwqZ9hMNSnhJWDF3UAm 59klhpa0Q7z/ABK4UzTt8imbbALk14rGnYrQ1f2EyetqgctQyxbsgz+1FEWFeYCu/fkj F5lKjKvBwTKqjLyTtvuB1r49JqnXl1aGxWjhiUMriMqWcIeAS6ZWK5TUdkUHE7prDPZo 7HsB18YycrnuJNK64InDX5+U8YzxLZeOPyKxoYR3qJRbff0gPZwXwu1PzmhVwby82r4a iOiMmC+GkILv24GFX6qgcJ6qh83GXNFFAV3p2N48/InZ81R4f6BuIFEMbvSmJl7uZ0xU XfLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736363677; x=1736968477; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=tgwDwIjUXMZMpDhITbnkdObRyaeVei/v3zDkEhXmicI=; b=CdX4xCOMVykY3knpkGdOiKUWMPqSof8t8i75d6IfZ8irG6CABvCWYkSQAFw5/D5k9i wvWKK8z++An4z2K2HuzRQbP2szIFHshYnlV8bEsKYP8huj9gntl/7Ur7GhPMdsVs/rPH uWHW8/u1Eyk0dro/1CL4ztYzqWhhzRzh6cPSxKw4f6WX7hGb4zUorMaScPxwOWbJL62G yLnAsfpq+YyDaGv067dxGl5PTHiQ7Mo22ONN3jDUkf1GDGmBt2a5a8mtJONUAxoZTpFo GpZrz1dMOKkf7XLWwMCskzre2/v8zC6v6j4W6XHjtSD8DF4bpw0Qx/nsuOn/+AWiLB8B WcPw== X-Gm-Message-State: AOJu0YwSgzUOj6LPcB64F1DBUM7vVnNmrffDpJjgHkb4OZC7Cv0MAQ4d 8SJFW5UwKfsvI+rso0VjoQ89FH3+yGc/Lu1GYn8XE8LFrssuV6n6RnLiHtJ5FewFxRmFy72ZwGR X0uA= X-Gm-Gg: ASbGncu4pCnS22qZJX4hlG6zZLvzD0mik0YRTrT4xBFb0uWgrzNWbZxXyqo+82oxqWM 1CE3AwP0ArbCon3Y908sqRYu0E2BrUU0nBr64xf6g62k8dOs2u5n+ETHX+GafM8aZJCL4yZJ6S5 D3cxtFbRe9iXBOydafbJw1cY24LHdqbA3x9XgLrbjk8Mh55Kgno9lW/HpGbB3zNzGD/+otKBKMy D+kHUKpOf8a3QsIBN58/d7HYfjxfm/5lZ6tTdMTdlfWMCMkgfCLSEwpudExty+uQbpWIca+uQmN MNVnJTyIt2KLpZzfows6QChaSd86 X-Google-Smtp-Source: AGHT+IHANNEspklJhCEQNb+yH/4vkYa23djjx6/kO42M6s+y5oR9x7dDYY5dqtOa1KAgJRXvwbaQAA== X-Received: by 2002:a05:690c:740a:b0:6ef:90a7:16ec with SMTP id 00721157ae682-6f5312d3be8mr36944667b3.28.1736363677479; Wed, 08 Jan 2025 11:14:37 -0800 (PST) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id 00721157ae682-6f3e77f10dasm92788737b3.89.2025.01.08.11.14.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jan 2025 11:14:36 -0800 (PST) Date: Wed, 8 Jan 2025 14:14:35 -0500 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Junio C Hamano , Elijah Newren , Patrick Steinhardt Subject: [PATCH v2 2/8] csum-file: store the hash algorithm as a struct field Message-ID: <99cc44895b57cc75fc8f447000817b3595368e4c.1736363652.git.me@ttaylorr.com> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Throughout the hashfile API, we rely on a reference to 'the_hash_algo', and call its _usnafe function variants directly. Prepare for a future change where we may use a different 'git_hash_algo' pointer (instead of just relying on 'the_hash_algo' throughout) by making the 'git_hash_algo' pointer a member of the 'hashfile' structure itself. Signed-off-by: Taylor Blau --- csum-file.c | 20 +++++++++++--------- csum-file.h | 1 + 2 files changed, 12 insertions(+), 9 deletions(-) diff --git a/csum-file.c b/csum-file.c index 5716016e12e..b28cd047e3f 100644 --- a/csum-file.c +++ b/csum-file.c @@ -50,7 +50,7 @@ void hashflush(struct hashfile *f) if (offset) { if (!f->skip_hash) - the_hash_algo->unsafe_update_fn(&f->ctx, f->buffer, offset); + f->algop->unsafe_update_fn(&f->ctx, f->buffer, offset); flush(f, f->buffer, offset); f->offset = 0; } @@ -71,14 +71,14 @@ int finalize_hashfile(struct hashfile *f, unsigned char *result, hashflush(f); if (f->skip_hash) - hashclr(f->buffer, the_repository->hash_algo); + hashclr(f->buffer, f->algop); else - the_hash_algo->unsafe_final_fn(f->buffer, &f->ctx); + f->algop->unsafe_final_fn(f->buffer, &f->ctx); if (result) - hashcpy(result, f->buffer, the_repository->hash_algo); + hashcpy(result, f->buffer, f->algop); if (flags & CSUM_HASH_IN_STREAM) - flush(f, f->buffer, the_hash_algo->rawsz); + flush(f, f->buffer, f->algop->rawsz); if (flags & CSUM_FSYNC) fsync_component_or_die(component, f->fd, f->name); if (flags & CSUM_CLOSE) { @@ -128,7 +128,7 @@ void hashwrite(struct hashfile *f, const void *buf, unsigned int count) * f->offset is necessarily zero. */ if (!f->skip_hash) - the_hash_algo->unsafe_update_fn(&f->ctx, buf, nr); + f->algop->unsafe_update_fn(&f->ctx, buf, nr); flush(f, buf, nr); } else { /* @@ -174,7 +174,9 @@ static struct hashfile *hashfd_internal(int fd, const char *name, f->name = name; f->do_crc = 0; f->skip_hash = 0; - the_hash_algo->unsafe_init_fn(&f->ctx); + + f->algop = the_hash_algo; + f->algop->unsafe_init_fn(&f->ctx); f->buffer_len = buffer_len; f->buffer = xmalloc(buffer_len); @@ -208,7 +210,7 @@ void hashfile_checkpoint(struct hashfile *f, struct hashfile_checkpoint *checkpo { hashflush(f); checkpoint->offset = f->total; - the_hash_algo->unsafe_clone_fn(&checkpoint->ctx, &f->ctx); + f->algop->unsafe_clone_fn(&checkpoint->ctx, &f->ctx); } int hashfile_truncate(struct hashfile *f, struct hashfile_checkpoint *checkpoint) @@ -219,7 +221,7 @@ int hashfile_truncate(struct hashfile *f, struct hashfile_checkpoint *checkpoint lseek(f->fd, offset, SEEK_SET) != offset) return -1; f->total = offset; - the_hash_algo->unsafe_clone_fn(&f->ctx, &checkpoint->ctx); + f->algop->unsafe_clone_fn(&f->ctx, &checkpoint->ctx); f->offset = 0; /* hashflush() was called in checkpoint */ return 0; } diff --git a/csum-file.h b/csum-file.h index 7c73da0a40a..2b45f4673a2 100644 --- a/csum-file.h +++ b/csum-file.h @@ -20,6 +20,7 @@ struct hashfile { size_t buffer_len; unsigned char *buffer; unsigned char *check_buffer; + const struct git_hash_algo *algop; /** * If non-zero, skip_hash indicates that we should From patchwork Wed Jan 8 19:14:39 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13931495 Received: from mail-yb1-f182.google.com (mail-yb1-f182.google.com [209.85.219.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E5871202F61 for ; Wed, 8 Jan 2025 19:14:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736363683; cv=none; b=CnUuisAuYlQ3y7Mtzc8oMjU7TN2xcPwRc1AxRPlPufmj1v8eIhPqrF1qoH5zGmg7hnke6tTf57Z0sHZz/NIBn8i9E1DfsnbKmSFM8IDFXleavc4JnWSybqQ/tEubY8Mx8Dpkf1RJJobTXmkA83C2E8IXj+C0+avgKaVHYH7K1AY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736363683; c=relaxed/simple; bh=joMZg4qmYz2Lz0QTlg6bh5rrHnJFa9ko+IlIwy68Qiw=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=GXYmiC/SeIVu30lRjJ2eJ6YkYpkcFILXpS80TofS0oFx7780D8PeubqYBkyUhV1BRzOWAy2cndmlX5rYP0C+H90bqGNLi9Z/7QF01Mb802L65hinoIUqSJyO4pxm9LO/8Lks+LcPEKhm0JfKOsTf6OyXm5zf6h/auOuu1j+EE1E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com; spf=pass smtp.mailfrom=ttaylorr.com; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b=ji/X1Nnw; arc=none smtp.client-ip=209.85.219.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b="ji/X1Nnw" Received: by mail-yb1-f182.google.com with SMTP id 3f1490d57ef6-e4930eca0d4so73264276.3 for ; Wed, 08 Jan 2025 11:14:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1736363680; x=1736968480; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=rxjyTX8VFudYaSaxpA1JZ7mGErbYWSHzZV0u9GWv0sk=; b=ji/X1NnwUSj8Ws/JKdgKdjccR2Y8xmXlAZXXK43WHTkgxfIpWT99cQOndpbIiTwPBP F7JW2qMIBJtY7DPmPmSfSkW23nK9eWde8JHx6CQiTc/hs7/wQADIegL8xPGhdHHfLkHe zDGPkmBxdrrgqmwHrfdpoH8auUWwORyko6VfKGL6bk+4YcOGDpFSSzrHqrCQT+7DLq+z DNCBJ3e9gTgmOAuyovPajeJxlRC19C5YtUYVA53wpxcFDXFREjhYur8l9bhcJ+eqkTdA zjRQaV9Ko4wOSHQ5YyBCRb1qo+o1H62AU5a7dAHUG6FbGzWJ7JHwh0dU6ZTNRmrdhGik ZoGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736363680; x=1736968480; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=rxjyTX8VFudYaSaxpA1JZ7mGErbYWSHzZV0u9GWv0sk=; b=IlXoI5wp9+ojUfredAY3QiZPWTu4iY4VtpRhHWNWaM48nCR4BMGn8hqYDrVAd2f1J+ 1Qjc4DDzluq+70c1kHV5wWG4W30DwGU+KvPp16tiSHMm0X6LaBkbJkjpiQftN+/XIFyx yxiAqyppNSjUUQc6XFPQPA5xMRNNZWyjJ0+4RZEBfEPu5fRwkaJG7g2/9sx8N1uvd7ha 0Bedu1MLhmHnkYqnugUrxFXBUz1G8lOEb8QvSc3+sBbWm+7mwQUmFT1DXuDEGOZipqrt lAJka7Sv7VU43mXb2mgevaqvw5l3/bvNdsrVuG82WO/sQ7HViz1rGEjSuhJV7rEb1D13 LwiA== X-Gm-Message-State: AOJu0YwqR7c7ZyY8ff8zst4Ida/JBOI6PNPD5EEl5v9GCDKhlnYnlJcu VMSRA5vho46ZNKVqKN8Ru+ks7D117ySlVqhID9EvbzrmgSCmprHK2nHlwdq/Vfi96IFPYBrUN+n AyGw= X-Gm-Gg: ASbGncu/Wfsr3Sv0oL/E6WvvFn6iwNFzDrW/SGilZ1FpQlk6zQ6XMmLGzJh97Z7Qfm/ D1ctkhZCaur1kunbCy7QLdKB7gNwfkrHESDLPDz4j4JbbY0t75lrBSWzf97dUnfoMz4htJjLaQa dmkCMqIND6LBeHrYF0ku+3O5dx/Dzj15HL/29x/2pJdWLFd9ly0S2aq1tkA6N7xoFxvTM8f1CyW DBWciwPi/xT/J6rPI1ds05AY0qKoscqJiC9iV5XuyeJWD4x3ivPZRDuJm4a27CA/AqbzbwMnB7K 6NbnF399GbhYAKUBsKLHprNZXTX9 X-Google-Smtp-Source: AGHT+IHNtjwRX5y/Xf5rD2QTHTdgf9qpV+YdrY1Geq7WtnC0Dh5DGflaUJKDqVQDkoTVUEhw9QMmyA== X-Received: by 2002:a05:690c:6a85:b0:6ef:6e34:6561 with SMTP id 00721157ae682-6f5312dd8c2mr35876127b3.32.1736363680574; Wed, 08 Jan 2025 11:14:40 -0800 (PST) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id 00721157ae682-6f3e783ac8csm92959367b3.116.2025.01.08.11.14.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jan 2025 11:14:40 -0800 (PST) Date: Wed, 8 Jan 2025 14:14:39 -0500 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Junio C Hamano , Elijah Newren , Patrick Steinhardt Subject: [PATCH v2 3/8] csum-file.c: extract algop from hashfile_checksum_valid() Message-ID: <1ffab2f82891af8d8644443d75bd6f29bcbd6906.1736363652.git.me@ttaylorr.com> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Perform a similar transformation as in the previous commit, but focused instead on hashfile_checksum_valid(). This function does not work with a hashfile structure itself, and instead validates the raw contents of a file written using the hashfile API. We'll want to be prepared for a similar change to this function in the future, so prepare ourselves for that by extracting 'the_hash_algo' into its own field for use within this function. Signed-off-by: Taylor Blau --- csum-file.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/csum-file.c b/csum-file.c index b28cd047e3f..7a71121e340 100644 --- a/csum-file.c +++ b/csum-file.c @@ -242,14 +242,15 @@ int hashfile_checksum_valid(const unsigned char *data, size_t total_len) { unsigned char got[GIT_MAX_RAWSZ]; git_hash_ctx ctx; - size_t data_len = total_len - the_hash_algo->rawsz; + const struct git_hash_algo *algop = the_hash_algo; + size_t data_len = total_len - algop->rawsz; - if (total_len < the_hash_algo->rawsz) + if (total_len < algop->rawsz) return 0; /* say "too short"? */ - the_hash_algo->unsafe_init_fn(&ctx); - the_hash_algo->unsafe_update_fn(&ctx, data, data_len); - the_hash_algo->unsafe_final_fn(got, &ctx); + algop->unsafe_init_fn(&ctx); + algop->unsafe_update_fn(&ctx, data, data_len); + algop->unsafe_final_fn(got, &ctx); - return hasheq(got, data + data_len, the_repository->hash_algo); + return hasheq(got, data + data_len, algop); } From patchwork Wed Jan 8 19:14:42 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13931496 Received: from mail-yb1-f176.google.com (mail-yb1-f176.google.com [209.85.219.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DA2641E515 for ; Wed, 8 Jan 2025 19:14:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736363686; cv=none; b=KVCPaQ7Q9aH3NpEtzfzSQbIPqs+TKCpKQ24ddK1u1lQMRjYw+IL6one+HoMUe9NcWvEmsQaBbnvdo9IZE+OCj+DTU0xeY+Igq4QsAOXeUvu4f/1ymiZ1f3jG4sRnZnXqa6CGrRZziu++Ca3y32Y1AhLP9HWiyFzFITzyjj6JmCY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736363686; c=relaxed/simple; bh=hWJ5m+6JIV2StaPUhvhWkY/RO0LOEQwoY3LStCgaWbU=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=E0dagPtQfn5+ggpgKH7o9o/vvMgwNydtGeWH00wt9xEf/jYm44N3ZexwJchuDq5hh5ba5KqzEKPgsDtqpfVtT+opXdlxqPjDvhk4VCUBpxwzWyoIsGxzNrgZmO+1HaMZ8A2B09B24sv4EM0ZZKF9DKO+e7slX/WRUlGyAAm78/4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com; spf=pass smtp.mailfrom=ttaylorr.com; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b=ipwbnmrn; arc=none smtp.client-ip=209.85.219.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b="ipwbnmrn" Received: by mail-yb1-f176.google.com with SMTP id 3f1490d57ef6-e46c6547266so72881276.3 for ; Wed, 08 Jan 2025 11:14:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1736363684; x=1736968484; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=Wvoe2VC0QAEm2MWq7n2gKbgG8Tdc7EQmwXMOqmfT8dY=; b=ipwbnmrnCHvTFeKJ8c9RrZBmPaWF9aPi/VUAV4EBR2Tv86gl6PVdeqzttljkf3wa0i eF5X/MD0FM1yHI6xz+CDYnI1Maj4kAehjlUyXu3M4AEewjoWeOiNGKbpGbh98aZGf+hj jhkqUvyhur11A/jP50xqw8iL5aGKscwBi9GNPUA90JeRh5RKYvRorKXoxHm2aicdp8pT o3C/adWMC8faqh1QHko4EKzI+F/YmPOSII7oF8HNZd0LQ5mm9G52KnVexdHMIoAeq17i 80r9BX5cphGEsrz5cWPhR1CRqZLnhiAlLkWP1i/NA5oCy4mXXcM+cZCBf6vWCVffv9x7 n17A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736363684; x=1736968484; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=Wvoe2VC0QAEm2MWq7n2gKbgG8Tdc7EQmwXMOqmfT8dY=; b=ZbCfnzk0oXg+75siSSwb/SbGMIdS4Ufd4SoBKdt3slORqw+YpoahyTlbyGj97/L80o yz9/CflbhscWAEIAmJLBf/w7JPQlrxf1JteuJMDNw6yZizGm64lqfHqG0lTCXo932flm eNC1CfBzGCUpV/CAFjihsLpX0o0T8mXF9atS96UGiC4U3t0z38HTQt82hzQ61iZyJFeq jvoCiDpjjClH8LvPpBMr/vN9NyX/kJ7FMcOVrGuj20tAE4b+ilAqR8JPeGlloTLevDpZ nYqcierEQTvPeaVXcxmKmCDuBw6NRqmcuAp+4w//YQXwCizOxCi2IO0LoEsx7pwTV+Qi O5rw== X-Gm-Message-State: AOJu0YwSR7gIKAUJvZuSY6oZbqJ4jXCUdbZvZKCzsmmHyzHVeo0g3Zz2 lNmUwB8qxRQ3wLaFUlKc2XuIT7yZaH9Wp4xMapSvQFqaeviZRF5WxeW5M6TNYEZKKDmyaPzORf5 bVc0= X-Gm-Gg: ASbGnctvZez1O+R7qxigC0LexrcKoD39ctFTpsIXXatI3S18obZcSVd8CGPcEehC04R Fn1al7mf5M0lifTgxSQ0qRHzJ01cN/oGMmD2OV99aWgsJYpFFGk287FLgiBPdSUyKs82MA1K9k4 NZeqsRgZ9nfOTN9aC3kXaEYl0Bfeq20x/wqThdVFzuVEokgUUMF+zWRuJeIgAWBAtNbSYcqQ/3t AbG5ULQTLw2WcOFuWJL4pqu/xMa+WZK9zj3pSt2Y2mpLAGySaib2BMqeyT7quNAQ/7LxHFlxx1q UBp5aKRUQZRRj+v3VYK1gVCnkrXn X-Google-Smtp-Source: AGHT+IFOkGuQufZ/xE3B64qIHfBuHR1MgB9FNG6vMjiaOP5nEwyCZqi8/d61E83ZCrRunhHuFycC0Q== X-Received: by 2002:a05:690c:7348:b0:6ef:4c6c:3049 with SMTP id 00721157ae682-6f531220b04mr33898297b3.14.1736363683735; Wed, 08 Jan 2025 11:14:43 -0800 (PST) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id 00721157ae682-6f3e77ed520sm95206387b3.97.2025.01.08.11.14.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jan 2025 11:14:43 -0800 (PST) Date: Wed, 8 Jan 2025 14:14:42 -0500 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Junio C Hamano , Elijah Newren , Patrick Steinhardt Subject: [PATCH v2 4/8] hash.h: introduce `unsafe_hash_algo()` Message-ID: <99dcbe2e7165d96e3a9c025540995a75f74b2489.1736363652.git.me@ttaylorr.com> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: In 253ed9ecff (hash.h: scaffolding for _unsafe hashing variants, 2024-09-26), we introduced "unsafe" variants of the SHA-1 hashing functions by introducing new functions like "unsafe_init_fn()" and so on. This approach has a major shortcoming that callers must remember to consistently use one variant or the other. Failing to consistently use (or not use) the unsafe variants can lead to crashes at best, or subtle memory corruption issues at worst. In the hashfile API, this isn't difficult to achieve, but verifying that all callers consistently use the unsafe variants is somewhat of a chore given how spread out all of the callers are. In the sha1 and sha1-unsafe test helpers, all of the calls to various hash functions are guarded by an "if (unsafe)" conditional, which is repetitive and cumbersome. Address these issues by introducing a new pattern whereby one 'git_hash_algo' can return a pointer to another 'git_hash_algo' that represents the unsafe version of itself. So instead of having something like: if (unsafe) the_hash_algo->init_fn(...); the_hash_algo->update_fn(...); the_hash_algo->final_fn(...); else the_hash_algo->unsafe_init_fn(...); the_hash_algo->unsafe_update_fn(...); the_hash_algo->unsafe_final_fn(...); we can instead write: struct git_hash_algo *algop = the_hash_algo; if (unsafe) algop = unsafe_hash_algo(algop); the_hash_algo->init_fn(...); the_hash_algo->update_fn(...); the_hash_algo->final_fn(...); This removes the existing shortcoming by no longer forcing the caller to "remember" which variant of the hash functions it wants to call, only to hold onto a 'struct git_hash_algo' pointer that is initialized once. Similarly, while there currently is still a way to "mix" safe and unsafe functions, this too will go away after subsequent commits remove all direct calls to the unsafe_ variants. Suggested-by: Jeff King Signed-off-by: Taylor Blau --- hash.h | 5 +++++ object-file.c | 26 ++++++++++++++++++++++++++ 2 files changed, 31 insertions(+) diff --git a/hash.h b/hash.h index 756166ce5e8..23cf6876e50 100644 --- a/hash.h +++ b/hash.h @@ -305,6 +305,9 @@ struct git_hash_algo { /* The all-zeros OID. */ const struct object_id *null_oid; + + /* The unsafe variant of this hash function, if one exists. */ + const struct git_hash_algo *unsafe; }; extern const struct git_hash_algo hash_algos[GIT_HASH_NALGOS]; @@ -323,6 +326,8 @@ static inline int hash_algo_by_ptr(const struct git_hash_algo *p) return p - hash_algos; } +const struct git_hash_algo *unsafe_hash_algo(const struct git_hash_algo *algop); + const struct object_id *null_oid(void); static inline int hashcmp(const unsigned char *sha1, const unsigned char *sha2, const struct git_hash_algo *algop) diff --git a/object-file.c b/object-file.c index 5b792b3dd42..43efa4ca361 100644 --- a/object-file.c +++ b/object-file.c @@ -202,6 +202,22 @@ static void git_hash_unknown_final_oid(struct object_id *oid UNUSED, BUG("trying to finalize unknown hash"); } +static const struct git_hash_algo sha1_unsafe_algo = { + .name = "sha1", + .format_id = GIT_SHA1_FORMAT_ID, + .rawsz = GIT_SHA1_RAWSZ, + .hexsz = GIT_SHA1_HEXSZ, + .blksz = GIT_SHA1_BLKSZ, + .init_fn = git_hash_sha1_init_unsafe, + .clone_fn = git_hash_sha1_clone_unsafe, + .update_fn = git_hash_sha1_update_unsafe, + .final_fn = git_hash_sha1_final_unsafe, + .final_oid_fn = git_hash_sha1_final_oid_unsafe, + .empty_tree = &empty_tree_oid, + .empty_blob = &empty_blob_oid, + .null_oid = &null_oid_sha1, +}; + const struct git_hash_algo hash_algos[GIT_HASH_NALGOS] = { { .name = NULL, @@ -239,6 +255,7 @@ const struct git_hash_algo hash_algos[GIT_HASH_NALGOS] = { .unsafe_update_fn = git_hash_sha1_update_unsafe, .unsafe_final_fn = git_hash_sha1_final_unsafe, .unsafe_final_oid_fn = git_hash_sha1_final_oid_unsafe, + .unsafe = &sha1_unsafe_algo, .empty_tree = &empty_tree_oid, .empty_blob = &empty_blob_oid, .null_oid = &null_oid_sha1, @@ -305,6 +322,15 @@ int hash_algo_by_length(int len) return GIT_HASH_UNKNOWN; } +const struct git_hash_algo *unsafe_hash_algo(const struct git_hash_algo *algop) +{ + /* If we have a faster "unsafe" implementation, use that. */ + if (algop->unsafe) + return algop->unsafe; + /* Otherwise use the default one. */ + return algop; +} + /* * This is meant to hold a *small* number of objects that you would * want repo_read_object_file() to be able to return, but yet you do not want From patchwork Wed Jan 8 19:14:45 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13931497 Received: from mail-yb1-f172.google.com (mail-yb1-f172.google.com [209.85.219.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 47E38202C51 for ; Wed, 8 Jan 2025 19:14:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736363689; cv=none; b=Oj4lbJzzmubL8DkxRGWuVbZSXAzWea70k3YNnMuRh62ojdOvmIsVay7RS5SN6VKLlsyMHTUo+fO7Rn7rY2P1G336WgmClam7Fm4Oywd9MtZiGh27pbEFXPsjYbFCUeNHxfG/dRN77r24jwFrHxW3q3bVR+gLwamPe5VXGd3yrsY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736363689; c=relaxed/simple; bh=tOa+WdgCtIkciRv1obBIhUD2Qovp2K+ZJmUkBPOtpGk=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=sS7Qsvo/hohVHJXLOi0223w6/L3mfHOobMPsC8ZFE/mvAFCch94I+LZvXSjNN4PJwhNmGtLzpPnu6swfwL4y2cQJeuV5c3qi+bKwv2cYfQHn5s8i91k4t1DhVaTzP/j59feCch2ePWJ134L+07nzXrzBi8ZZAIAPosCLchdP+EE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com; spf=pass smtp.mailfrom=ttaylorr.com; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b=HPHo7/zg; arc=none smtp.client-ip=209.85.219.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b="HPHo7/zg" Received: by mail-yb1-f172.google.com with SMTP id 3f1490d57ef6-e5372a2fbddso95309276.3 for ; Wed, 08 Jan 2025 11:14:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1736363687; x=1736968487; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=IOhIYN5HqyIukCSx0+tJxmlft7yPAcOE9uzVrjzlCIY=; b=HPHo7/zghscGZuVUoItuSM+KFGZIXurS3ok9rUK0cE53mV1QpyM0kvCwfzCbNxJURd MuW/Ra8I16SxWxAQzzCMR2MO7XXjvzT0IZM0JKusJypqXfKCyaUYcvMAbb/pCGzwkImd nThg+XnRiAul163H1kB26WRnpIhql/0G+z5aBdknrvFyhPCpblI+B4gCJncqxrmlkITV 7qsvIh+1rlDlkqlsndRzp/DZj/YQVWFtywnmnB9O8hK8h1AcAHR4qVUVHMhxn8nh7oAF 13raCg52YQL53pEkMrfzQE/Hsss1+5tosFS6Vr021PZsLqayrwwQUcNtTfHh/ON8FkbK RiPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736363687; x=1736968487; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=IOhIYN5HqyIukCSx0+tJxmlft7yPAcOE9uzVrjzlCIY=; b=CWzKvaRds8xCqEAQrfG33e0cnakXW51C2rLazUUEot62pwlmRv9C26BZeOwyFmhryM RDSjt2ltcZiHPFVuWQ0jnCaNDqgwPJ/dAcVDxAx/r+UagatdjGmPk+7VdkTlMgsW5HC3 grcnzWzoJ8Lk11cANH/3NaR6J6zDYtgqFBQO9EJNyfSlg+yxKUBsPlAl2DgoKoqR75ZY QI1O7LNJjfq9LrGxOqLRFxiC1AEZmHlt8W6l7SmZIlHOwACsKESfaUWv+/eOZlxyIch5 7INmQUyTERvSzZvOi7s99nE866s8npRByvRXqZPuL7xkWLMCFyeiW5ra/zyTqkNVynfJ Nj2A== X-Gm-Message-State: AOJu0YwQJDHUtNUGAfW/CRAJz5xezxLpO2qf72xhCuFQ16UNQCG9ZgKl eJ4N6ioXT3CiizVOupTte6CfnACzBRmCiXPRxmajqGn3vduiP38DA+kj4o+AIqt6J+h1RpYP8tl UUEs= X-Gm-Gg: ASbGncsh50h258cazc8D0OGFi4nHuqpSkQ5bDgH39VNA8tW+DWDAtHPvVQpdFgIF8He fnaW4SZpPkC/MbVflBoaZZsImNY4IKekpDyW9GxJyFNOpA0wObr0IP7Y/qjVFTew2rnmGoFLmeD pu/PYffXtyVzDWgGdo2w+X4NU11Z4dFesW85oWZ68z7AsH1w2ELvtGhN6uMjpvCj0WjeST0Sntc +joGMZzXhbd0G1Ljho6PfI1za6+HXqNfeLMk41jhE5uG5whPsmkztKNRmoqy/8o3dbrune7ZbHt HUOVCLpafsByr3UF0IKG7u/hMoAB X-Google-Smtp-Source: AGHT+IETTZ71r3nf3Hx7GRjUhxcq0fsmR+8vB1hVKTp9Uki4zS1V6/Da5tLZKrvx8Tpar2v6yFL7vQ== X-Received: by 2002:a05:690c:74c3:b0:6ef:4e42:64d0 with SMTP id 00721157ae682-6f531222ad4mr29783457b3.11.1736363686803; Wed, 08 Jan 2025 11:14:46 -0800 (PST) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id 00721157ae682-6f3e7883a20sm91161747b3.131.2025.01.08.11.14.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jan 2025 11:14:46 -0800 (PST) Date: Wed, 8 Jan 2025 14:14:45 -0500 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Junio C Hamano , Elijah Newren , Patrick Steinhardt Subject: [PATCH v2 5/8] csum-file.c: use unsafe_hash_algo() Message-ID: <2dcc2aa6803cdaad762ec82cd31fc14dd6aca414.1736363652.git.me@ttaylorr.com> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Instead of calling the unsafe_ hash function variants directly, make use of the shared 'algop' pointer by initializing it to: f->algop = unsafe_hash_algo(the_hash_algo); , thus making all calls use the unsafe variants directly. Signed-off-by: Taylor Blau --- csum-file.c | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/csum-file.c b/csum-file.c index 7a71121e340..ebffc80ef71 100644 --- a/csum-file.c +++ b/csum-file.c @@ -50,7 +50,7 @@ void hashflush(struct hashfile *f) if (offset) { if (!f->skip_hash) - f->algop->unsafe_update_fn(&f->ctx, f->buffer, offset); + f->algop->update_fn(&f->ctx, f->buffer, offset); flush(f, f->buffer, offset); f->offset = 0; } @@ -73,7 +73,7 @@ int finalize_hashfile(struct hashfile *f, unsigned char *result, if (f->skip_hash) hashclr(f->buffer, f->algop); else - f->algop->unsafe_final_fn(f->buffer, &f->ctx); + f->algop->final_fn(f->buffer, &f->ctx); if (result) hashcpy(result, f->buffer, f->algop); @@ -128,7 +128,7 @@ void hashwrite(struct hashfile *f, const void *buf, unsigned int count) * f->offset is necessarily zero. */ if (!f->skip_hash) - f->algop->unsafe_update_fn(&f->ctx, buf, nr); + f->algop->update_fn(&f->ctx, buf, nr); flush(f, buf, nr); } else { /* @@ -175,8 +175,8 @@ static struct hashfile *hashfd_internal(int fd, const char *name, f->do_crc = 0; f->skip_hash = 0; - f->algop = the_hash_algo; - f->algop->unsafe_init_fn(&f->ctx); + f->algop = unsafe_hash_algo(the_hash_algo); + f->algop->init_fn(&f->ctx); f->buffer_len = buffer_len; f->buffer = xmalloc(buffer_len); @@ -210,7 +210,7 @@ void hashfile_checkpoint(struct hashfile *f, struct hashfile_checkpoint *checkpo { hashflush(f); checkpoint->offset = f->total; - f->algop->unsafe_clone_fn(&checkpoint->ctx, &f->ctx); + f->algop->clone_fn(&checkpoint->ctx, &f->ctx); } int hashfile_truncate(struct hashfile *f, struct hashfile_checkpoint *checkpoint) @@ -221,7 +221,7 @@ int hashfile_truncate(struct hashfile *f, struct hashfile_checkpoint *checkpoint lseek(f->fd, offset, SEEK_SET) != offset) return -1; f->total = offset; - f->algop->unsafe_clone_fn(&f->ctx, &checkpoint->ctx); + f->algop->clone_fn(&f->ctx, &checkpoint->ctx); f->offset = 0; /* hashflush() was called in checkpoint */ return 0; } @@ -242,15 +242,15 @@ int hashfile_checksum_valid(const unsigned char *data, size_t total_len) { unsigned char got[GIT_MAX_RAWSZ]; git_hash_ctx ctx; - const struct git_hash_algo *algop = the_hash_algo; + const struct git_hash_algo *algop = unsafe_hash_algo(the_hash_algo); size_t data_len = total_len - algop->rawsz; if (total_len < algop->rawsz) return 0; /* say "too short"? */ - algop->unsafe_init_fn(&ctx); - algop->unsafe_update_fn(&ctx, data, data_len); - algop->unsafe_final_fn(got, &ctx); + algop->init_fn(&ctx); + algop->update_fn(&ctx, data, data_len); + algop->final_fn(got, &ctx); return hasheq(got, data + data_len, algop); } From patchwork Wed Jan 8 19:14:48 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13931498 Received: from mail-yb1-f182.google.com (mail-yb1-f182.google.com [209.85.219.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 110A2202F9A for ; Wed, 8 Jan 2025 19:14:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736363692; cv=none; b=DHd5RoKTmN2AP4GPL0PjIm7TjcShmuNb+s7gXr0/E6SP9ex7yhBD35mZ3arwOm5Xzb2PXTQn95X68gb0Q3EorJ8fx0X+5Ou6apHpu8jQmf5GjM7K8U1eoWUjeTpq+9jbfcUWXwkZH/RV3aHNHWIHGFWpDbz+LCCfPspm8ej5SoM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736363692; c=relaxed/simple; bh=I8b+brVSHXda+TWSVUyYNES/AqHV9Opc0j4jgRI3bdg=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=jEUYU4OYxNOcXtzgXph1fpYG3naumVus97qqxHIzijZrwo1ur5zq5DEnZwohpP2813mSpk0IQl2h9oDagj2gX7LR+jfFon7DDH+9TLvhiZ6HvErKGgYGQWSU+oZP3YvUI3eqLk6SsnpTGBcfDZfjyfy4b/dwjW1rDiSsbdwOMjs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com; spf=pass smtp.mailfrom=ttaylorr.com; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b=nZwpmIka; arc=none smtp.client-ip=209.85.219.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b="nZwpmIka" Received: by mail-yb1-f182.google.com with SMTP id 3f1490d57ef6-e5447fae695so103183276.2 for ; Wed, 08 Jan 2025 11:14:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1736363690; x=1736968490; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=sc62gKTFY5X8YNEhTN/LBtFdmPFQi1T0wqUTSv1smlA=; b=nZwpmIkaOxxJs7LZmeq9IvA7Pg1gGJru6iSb1dW5emhM80wXJzT8AUORCjD0SNwIbe J6Ugp2Aj9gyN/399BzjpeQHuugMnC6EvZkPFyOBVriZ8gx5TqbzQ+7mdOcT5KkEEF0hX jqenDVD/diyKKrzSF/OjlaSQ/91+QOE0RyIKMLmDYgRUPaZzR5M8WErxHBd/snxlOm4j +R/iTbBuuiakD7fzghVkXZsGPpnH9RvwCAx3a3+RWxpmqIdX24IdRzt6DTYZwzR/05zw pXUsCs3S46dOAtBV7Nfnic5dYuwZPmPJFzNQqiXQxhFp0I0yzSMiPxmLJmJC3PJaNWIK x67w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736363690; x=1736968490; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=sc62gKTFY5X8YNEhTN/LBtFdmPFQi1T0wqUTSv1smlA=; b=Isz6QPe/RLhC/2UZxnb1Mowpy0rSXYTsEmHB1I1Xv28i94FeYrzvraIBFbXJwBjyUI PF9ikFVUBWPk2+DUuC2n9bcSiD+DXL9ckcTESm82ZdorbdSIc28kAcGKshQ+XcYoaZ10 +NQu4+1kYiSUTDjn5F/g++c73+CWod4oXFdEDL9pD7GeVT+crDmQWvpgugWqKO4BQ/6S RN4sN5g7nvP6T1PbNpHxGJwtLITkZeQnxBfoi4XobVcQZN+gi5bDrhZiJB9bLTAWjQW4 /Pg1hJQMZ7XK07Yvsc69W3usyF7hD7tCx80ipdtPTdi8AV4HW62aNSWtqM/yIzlQd9XG wH5w== X-Gm-Message-State: AOJu0YzOEeoPU6Z9eeRX+KaelZXt9hAnv/nG6tLUWtNRg9pPYRsvYmTK iY2pxN2Yc750rXuYzqyPNbPlS5kDbaU+Ji1pk9iQsfJvfUX2O60uySBysZPHdaliPAwqkp1Lah5 200c= X-Gm-Gg: ASbGncuIjd5WzIf96tPqCCMtlRe1nSmc7T6D/FY+JBsTpM/g5MgXnSHMuGg8tBkZtj/ ghDqW8/5RspR9HF/rZcaAsJ0f4ozaQ8hA3nXL+fg/hks0TkpvNBXCM8UaYRzOKdOYshh/GKSjVT GnijMito6iA0W5+SWu530Px9G9JUCDjT3xA8AKQ1AlmNpPWKoJ3S/S3NTc5kk3mjyyfgiKRvwf6 ZX1PlfQ+TR4HuXEt/dIhEwzxrnLKpU5HEZa6UUQe9Li1UJebjNNYylbkObqSIJsgzXmiYh2pAFw dFxg8tSu/tfXMxy/Oijrobk4EU9J X-Google-Smtp-Source: AGHT+IEAMw5Fm9r5jT371FNuv1YLqVpK9bXSREdahPNPFXbSg8AU4KMglmtJEo/pIV9/RwVfi8wEvw== X-Received: by 2002:a05:690c:b17:b0:6ee:66d2:e738 with SMTP id 00721157ae682-6f5311e4ab2mr39751387b3.2.1736363689865; Wed, 08 Jan 2025 11:14:49 -0800 (PST) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id 00721157ae682-6f3e7837fc1sm93268697b3.108.2025.01.08.11.14.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jan 2025 11:14:49 -0800 (PST) Date: Wed, 8 Jan 2025 14:14:48 -0500 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Junio C Hamano , Elijah Newren , Patrick Steinhardt Subject: [PATCH v2 6/8] t/helper/test-hash.c: use unsafe_hash_algo() Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Remove a series of conditionals within the shared cmd_hash_impl() helper that powers the 'sha1' and 'sha1-unsafe' helpers. Instead, replace them with a single conditional that transforms the specified hash algorithm into its unsafe variant. Then all subsequent calls can directly use whatever function it wants to call without having to decide between the safe and unsafe variants. Signed-off-by: Taylor Blau --- t/helper/test-hash.c | 17 +++++------------ 1 file changed, 5 insertions(+), 12 deletions(-) diff --git a/t/helper/test-hash.c b/t/helper/test-hash.c index d0ee668df95..aa82638c621 100644 --- a/t/helper/test-hash.c +++ b/t/helper/test-hash.c @@ -9,6 +9,8 @@ int cmd_hash_impl(int ac, const char **av, int algo, int unsafe) int binary = 0; char *buffer; const struct git_hash_algo *algop = &hash_algos[algo]; + if (unsafe) + algop = unsafe_hash_algo(algop); if (ac == 2) { if (!strcmp(av[1], "-b")) @@ -27,10 +29,7 @@ int cmd_hash_impl(int ac, const char **av, int algo, int unsafe) die("OOPS"); } - if (unsafe) - algop->unsafe_init_fn(&ctx); - else - algop->init_fn(&ctx); + algop->init_fn(&ctx); while (1) { ssize_t sz, this_sz; @@ -49,15 +48,9 @@ int cmd_hash_impl(int ac, const char **av, int algo, int unsafe) } if (this_sz == 0) break; - if (unsafe) - algop->unsafe_update_fn(&ctx, buffer, this_sz); - else - algop->update_fn(&ctx, buffer, this_sz); + algop->update_fn(&ctx, buffer, this_sz); } - if (unsafe) - algop->unsafe_final_fn(hash, &ctx); - else - algop->final_fn(hash, &ctx); + algop->final_fn(hash, &ctx); if (binary) fwrite(hash, 1, algop->rawsz, stdout); From patchwork Wed Jan 8 19:14:51 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13931499 Received: from mail-yb1-f172.google.com (mail-yb1-f172.google.com [209.85.219.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 40554202F76 for ; Wed, 8 Jan 2025 19:14:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736363695; cv=none; b=ChuSLwFG5SffjmfxvQGyl0BzlIFxAJ6OdHrtG6X0h9tRY3fJGpEKoCicedc+ekX+RRcNCnmEkANMwA9NgO1lqtLdWt6MvcOblUY8dw9FoUDy99tmf7Wk4fHusItkJoAckoeGgo+Tn8P80MNx3eC7Z3HQ/jdf6bqv81gbXolTY4k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736363695; c=relaxed/simple; bh=cp8xYHOHc0/JJdJvKjPucl9ZjETWFmLZVvnsMlIZMco=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=rDDqZKxfz/TmqL5pkp0/Ctbjx01THbCn8P24kl7vNr3djWHMamz3h7ID03GTLNbdxefrIDk0+1VlXzvlnEtuH1qIiz9bXBU2lKLDgpKeZkvCT/j01JMpRpng0Dlgudse6de2OKMkF7IA9GlD5d2KnTMeZEU/1GmOIuVDUfRdJ1Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com; spf=pass smtp.mailfrom=ttaylorr.com; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b=jbINAZan; arc=none smtp.client-ip=209.85.219.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b="jbINAZan" Received: by mail-yb1-f172.google.com with SMTP id 3f1490d57ef6-e5372a2fbddso95502276.3 for ; Wed, 08 Jan 2025 11:14:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1736363693; x=1736968493; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=QeqphRlzbXNfsgl4zzUbnDr49e2gHpY5J6nEWyeZhNQ=; b=jbINAZanubRs4koJYL3Kq1zk4V5P1X8alSEUOjpgpqPXdxE6XIigRLb84s4xQDmLv2 Nk0ohoGUrX7zhOzU4RhzWnEC4zFjl6le/CAj3ktaSXyKEmx5Sj1o/qd9AqcQnVz1AgBm tFa9aapoiZEijI9mbxYnUyfXi4vue4nT5x62U71LLZRHFXaM7DQL2roH+aEZmhMdV2tI BzKOZ7E4TG2cst2of/vC4IH2fB8z5HWLk9g9r5BT9RtVJ5rNYOFHqb1u2DCsdZgMF04m sapslNr/XI7pYVr3FEKfkEIr2sCErZ1Q6+cTpNPaGYfio3D9ahyiXVklhXWrxuU3A3oZ ktdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736363693; x=1736968493; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=QeqphRlzbXNfsgl4zzUbnDr49e2gHpY5J6nEWyeZhNQ=; b=KWe6q6J90gdzmFh+mFs2eJ/JNB0+zjj0u7SZ31UTZldhonTwJTMDJqA2HM/gIzQbFz /cHJvkJM6DSeFGBqQaJUbwRieYV+fVHV7qzmyG712hwg2GNISVAshoXJdL81Cje+Whfu D9JPFFSUff9tiRt/uhHS0p/e1QC7wgg739c2Hn/6ElQRqazpYMtAZ9TmRN+ByuL2VRo4 jALBJ2cOtik0Bj1zwG4x3BOvRS8LoXtxPjkYF+wToTYZRmgsx9aoHlGX8e8ExGzCtoqP kZV+x/gkRlss7xNDvNP8/034AMnfPTB5YkQEa/2Nd4j+DMW5s+Qz8XxsbFdaq/SnifYp uKhw== X-Gm-Message-State: AOJu0YxXKTBJ+sO1Z8VPW9fTEbbRnTQSAYD6tSwsDNrpmLVplry2Fles kZZnwriA4h+eLE0lkKqCNBUsc+5pldkkt+a4JGbL5zAGbCy+fGeYPs94Bhpbc8VEBY5MN1lczIK Zuzw= X-Gm-Gg: ASbGncvsK0nalPVOJNMWIOCqjSM2NlrzdvDXoYF1lW5qzUqij14WJQUOqgAFn1Cs7yA RViI3PvpV7Xtk8cWxnyb5BsiBwvMJ73aRBxvGC6mpKzZ+PZ+g9fy5Ka9z2IE+xDq/kh+1wL5Hd/ hCUzOAbxfVHjYITEVVCVJJZ5ITAgdWBD8DsZhvEUBz6h4ELvniIz3Ds+refPTqJw5TD8ERH9K+X Tx1IypHk/HrMnCcoX0hhL0OcCZlGfXyjGS+GI/t1Jk5S8sZV2TIDcxj3Uiu+0bElKd3nyRVrFRM /s4liHDyEBKFbpBU2cgZGiUTeieP X-Google-Smtp-Source: AGHT+IEzDEaBbK29p2NvXHMXltxeRbqghW3rmaLX0FcTvFZNP0ISZ6JRN7l4GXsxnKm8rW74wZmqzw== X-Received: by 2002:a05:690c:23c6:b0:6d4:4a0c:fcf0 with SMTP id 00721157ae682-6f531241ec8mr35553287b3.20.1736363693009; Wed, 08 Jan 2025 11:14:53 -0800 (PST) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id 00721157ae682-6f3e73e8b24sm94877357b3.4.2025.01.08.11.14.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jan 2025 11:14:52 -0800 (PST) Date: Wed, 8 Jan 2025 14:14:51 -0500 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Junio C Hamano , Elijah Newren , Patrick Steinhardt Subject: [PATCH v2 7/8] csum-file: introduce hashfile_checkpoint_init() Message-ID: <94c07fd8a557c569fdc83015d5f3902094f21994.1736363652.git.me@ttaylorr.com> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: In 106140a99f (builtin/fast-import: fix segfault with unsafe SHA1 backend, 2024-12-30) and 9218c0bfe1 (bulk-checkin: fix segfault with unsafe SHA1 backend, 2024-12-30), we observed the effects of failing to initialize a hashfile_checkpoint with the same hash function implementation as is used by the hashfile it is used to checkpoint. While both 106140a99f and 9218c0bfe1 work around the immediate crash, changing the hash function implementation within the hashfile API to, for example, the non-unsafe variant would re-introduce the crash. This is a result of the tight coupling between initializing hashfiles and hashfile_checkpoints. Introduce and use a new function which ensures that both parts of a hashfile and hashfile_checkpoint pair use the same hash function implementation to avoid such crashes. A few things worth noting: - In the change to builtin/fast-import.c::stream_blob(), we can see that by removing the explicit reference to 'the_hash_algo->unsafe_init_fn()', we are hardened against the hashfile API changing away from the_hash_algo (or its unsafe variant) in the future. - The bulk-checkin code no longer needs to explicitly zero-initialize the hashfile_checkpoint, since it is now done as a result of calling 'hashfile_checkpoint_init()'. - Also in the bulk-checkin code, we add an additional call to prepare_to_stream() outside of the main loop in order to initialize 'state->f' so we know which hash function implementation to use when calling 'hashfile_checkpoint_init()'. This is OK, since subsequent 'prepare_to_stream()' calls are noops. However, we only need to call 'prepare_to_stream()' when we have the HASH_WRITE_OBJECT bit set in our flags. Without that bit, calling 'prepare_to_stream()' does not assign 'state->f', so we have nothing to initialize. - Other uses of the 'checkpoint' in 'deflate_blob_to_pack()' are appropriately guarded. Helped-by: Patrick Steinhardt Signed-off-by: Taylor Blau --- builtin/fast-import.c | 2 +- bulk-checkin.c | 9 ++++++--- csum-file.c | 7 +++++++ csum-file.h | 1 + 4 files changed, 15 insertions(+), 4 deletions(-) diff --git a/builtin/fast-import.c b/builtin/fast-import.c index 0f86392761a..4a6c7ab52ac 100644 --- a/builtin/fast-import.c +++ b/builtin/fast-import.c @@ -1106,7 +1106,7 @@ static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark) || (pack_size + PACK_SIZE_THRESHOLD + len) < pack_size) cycle_packfile(); - the_hash_algo->unsafe_init_fn(&checkpoint.ctx); + hashfile_checkpoint_init(pack_file, &checkpoint); hashfile_checkpoint(pack_file, &checkpoint); offset = checkpoint.offset; diff --git a/bulk-checkin.c b/bulk-checkin.c index 433070a3bda..892176d23d2 100644 --- a/bulk-checkin.c +++ b/bulk-checkin.c @@ -261,7 +261,7 @@ static int deflate_blob_to_pack(struct bulk_checkin_packfile *state, git_hash_ctx ctx; unsigned char obuf[16384]; unsigned header_len; - struct hashfile_checkpoint checkpoint = {0}; + struct hashfile_checkpoint checkpoint; struct pack_idx_entry *idx = NULL; seekback = lseek(fd, 0, SEEK_CUR); @@ -272,12 +272,15 @@ static int deflate_blob_to_pack(struct bulk_checkin_packfile *state, OBJ_BLOB, size); the_hash_algo->init_fn(&ctx); the_hash_algo->update_fn(&ctx, obuf, header_len); - the_hash_algo->unsafe_init_fn(&checkpoint.ctx); /* Note: idx is non-NULL when we are writing */ - if ((flags & HASH_WRITE_OBJECT) != 0) + if ((flags & HASH_WRITE_OBJECT) != 0) { CALLOC_ARRAY(idx, 1); + prepare_to_stream(state, flags); + hashfile_checkpoint_init(state->f, &checkpoint); + } + already_hashed_to = 0; while (1) { diff --git a/csum-file.c b/csum-file.c index ebffc80ef71..232121f415f 100644 --- a/csum-file.c +++ b/csum-file.c @@ -206,6 +206,13 @@ struct hashfile *hashfd_throughput(int fd, const char *name, struct progress *tp return hashfd_internal(fd, name, tp, 8 * 1024); } +void hashfile_checkpoint_init(struct hashfile *f, + struct hashfile_checkpoint *checkpoint) +{ + memset(checkpoint, 0, sizeof(*checkpoint)); + f->algop->init_fn(&checkpoint->ctx); +} + void hashfile_checkpoint(struct hashfile *f, struct hashfile_checkpoint *checkpoint) { hashflush(f); diff --git a/csum-file.h b/csum-file.h index 2b45f4673a2..b7475f16c20 100644 --- a/csum-file.h +++ b/csum-file.h @@ -36,6 +36,7 @@ struct hashfile_checkpoint { git_hash_ctx ctx; }; +void hashfile_checkpoint_init(struct hashfile *, struct hashfile_checkpoint *); void hashfile_checkpoint(struct hashfile *, struct hashfile_checkpoint *); int hashfile_truncate(struct hashfile *, struct hashfile_checkpoint *); From patchwork Wed Jan 8 19:14:54 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13931500 Received: from mail-yb1-f174.google.com (mail-yb1-f174.google.com [209.85.219.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 63959202C43 for ; Wed, 8 Jan 2025 19:14:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736363699; cv=none; b=uyJ4isC0RmfAJMNvOBSVb8xn9NW14vrBO/lkSW3XsnFoEwXiocK9Ucby93ijBZVIdzwWDZLpb5S/r2OCAC3JbnKj+krk2X35qx4GixQUCwzcDevtoev8YHXOP5xVU6wWbe903uR05soRDAS3O1EE4TL3+6XBOtQP2tBmou8SC7k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736363699; c=relaxed/simple; bh=0R8CMZ2t8B2qiNxcowckwjvrCL6YH9uzs7njIirFzNM=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=Y0G6JJBJQ0BT81+EfPLPwUnvb+UAYHHlKCUtqPVSSKS8cTsw+wlRJznpuumSkIpjqJsunzH6QAdWkglemOt6AqA7kapSD1xpN4Hto0hPfkIWeSSDZs+Jzm0LgNzardDGZOMr3LvxipH66sqHMZzq9+OH7BUJZuYUv1gt0KjhRYo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com; spf=pass smtp.mailfrom=ttaylorr.com; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b=cPFBAM9X; arc=none smtp.client-ip=209.85.219.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b="cPFBAM9X" Received: by mail-yb1-f174.google.com with SMTP id 3f1490d57ef6-e4930eca0d4so73663276.3 for ; Wed, 08 Jan 2025 11:14:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1736363696; x=1736968496; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=NL2kpIkDtmrtdsHAuH7/yDLd5xKXPl1QWSgXxmyWk4w=; b=cPFBAM9XhlQDne8M9bmlDVQqjU+T47A0DvsFDH06RIKf6iBFn7q7rHZrclsAsxTmeP zNl3Cf5PmiLTYK1ghN0eDvNNfcJz28yPI6MAIEPCpbExJbdkp9ibUmRVUfo9PoKVWK3+ o/YgT3nEp3fUonk87aXRhLpt2AY+q5qym1dXhpkwD47BttUNdlgDOeVV1q6OCPeYA/gC 8Df2N+ruzBqMGjyAY93vdaRDonPUB3TzJmklMeGAaPdODHtgolSHrjmixoZVSfvKY9y+ OgCdS0Drng37bgxFgRi/yU7+vBwJZszHkDORMcg4VPKlT7t8XxCvan+gT8EJ/fTzb1L5 S24A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736363696; x=1736968496; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=NL2kpIkDtmrtdsHAuH7/yDLd5xKXPl1QWSgXxmyWk4w=; b=eVbmaHNlR93GIhKxJLX8UNq29Mi0L2xUhSDYooEjCrLE/E92w3Bsb6sgF3EOsry4gh 3pYF7lZ3tNwBur/KA1humX1p5CMMhw3pZ0fs9GHVBAi7xVy2DTlM8521DGx8ZOtnBUpX bpDS8M6uayICRMY/yLANPdHGoAEWzuWzY2CzshLmdGpvwczBvuKs4IoYITtyj72p8MFw tuuaJ2yMVhMhLyDdAQnDwq+D3z7MjRx3r+OvD1/4wqmZKKTo80i8G3TXNoq3QUqjhg5L HscefnwWe0TREYcvoHyUzW1vEUMaJfETfxRivyDF0iHPaCwDsIl5JnvCggOc9i5JPKtp Kwjg== X-Gm-Message-State: AOJu0YzS4w+t3oIVs0CXTF/uEtKlCw2lfSOdBmRdgUyIFkshs1bXJwtx lERtTFidiDgp6XcZJe8zAQ5pDSO2VLbVSOQl5rTNHBgFf4ojBqwNP4llzoIAgV9W/1sBRC3VhPW GLSU= X-Gm-Gg: ASbGncvI/A6N0NFpbfYRF+D7EZS0d5PDdvEyGLKo8Ua9a1sudSQgfjrenCuTLtCCqEB LtNBBfi2oY/CSUDYKBL1WbxIwO1OqjVzCgNbxYzj4QE2o0FoHLpPk2bCr2TqK7eS4WIcJ56B/K3 RcQmCev/B+eoL2dnntY0NPiXxQZ3eON7ycyRV4URiT5EHHmfvHaDfh4dfD8hLACBplC3QJ7/D5y qM4aOOHontGgto1CIAtQ1KzQ+FS66bkzF7CXpJdtrfWbnUWTOkCcJBxITfIHYPWs/Thq7bkl7wj h1t9mGJiXv+Ms2eJl8aQ1oc6ihas X-Google-Smtp-Source: AGHT+IFoKNCv2VzIs032DzXn2deOaFDtYS1MenRmTCGBl8vPiKXXfiS/Fm4Rxea3NjNvnVwa+rTmEQ== X-Received: by 2002:a05:690c:b1d:b0:6ef:4b7a:eb07 with SMTP id 00721157ae682-6f5313254c8mr35721487b3.36.1736363696073; Wed, 08 Jan 2025 11:14:56 -0800 (PST) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id 00721157ae682-6f3e73e8939sm95062557b3.19.2025.01.08.11.14.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jan 2025 11:14:55 -0800 (PST) Date: Wed, 8 Jan 2025 14:14:54 -0500 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Junio C Hamano , Elijah Newren , Patrick Steinhardt Subject: [PATCH v2 8/8] hash.h: drop unsafe_ function variants Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Now that all callers have been converted from: the_hash_algo->unsafe_init_fn(); to unsafe_hash_algo(the_hash_algo)->unsafe_init_fn(); and similar, we can remove the scaffolding for the unsafe_ function variants and force callers to use the new unsafe_hash_algo() mechanic instead. Signed-off-by: Taylor Blau --- hash.h | 15 --------------- object-file.c | 15 --------------- 2 files changed, 30 deletions(-) diff --git a/hash.h b/hash.h index 23cf6876e50..6dcbf6ab835 100644 --- a/hash.h +++ b/hash.h @@ -282,21 +282,6 @@ struct git_hash_algo { /* The hash finalization function for object IDs. */ git_hash_final_oid_fn final_oid_fn; - /* The non-cryptographic hash initialization function. */ - git_hash_init_fn unsafe_init_fn; - - /* The non-cryptographic hash context cloning function. */ - git_hash_clone_fn unsafe_clone_fn; - - /* The non-cryptographic hash update function. */ - git_hash_update_fn unsafe_update_fn; - - /* The non-cryptographic hash finalization function. */ - git_hash_final_fn unsafe_final_fn; - - /* The non-cryptographic hash finalization function. */ - git_hash_final_oid_fn unsafe_final_oid_fn; - /* The OID of the empty tree. */ const struct object_id *empty_tree; diff --git a/object-file.c b/object-file.c index 43efa4ca361..c4b42dd4be9 100644 --- a/object-file.c +++ b/object-file.c @@ -230,11 +230,6 @@ const struct git_hash_algo hash_algos[GIT_HASH_NALGOS] = { .update_fn = git_hash_unknown_update, .final_fn = git_hash_unknown_final, .final_oid_fn = git_hash_unknown_final_oid, - .unsafe_init_fn = git_hash_unknown_init, - .unsafe_clone_fn = git_hash_unknown_clone, - .unsafe_update_fn = git_hash_unknown_update, - .unsafe_final_fn = git_hash_unknown_final, - .unsafe_final_oid_fn = git_hash_unknown_final_oid, .empty_tree = NULL, .empty_blob = NULL, .null_oid = NULL, @@ -250,11 +245,6 @@ const struct git_hash_algo hash_algos[GIT_HASH_NALGOS] = { .update_fn = git_hash_sha1_update, .final_fn = git_hash_sha1_final, .final_oid_fn = git_hash_sha1_final_oid, - .unsafe_init_fn = git_hash_sha1_init_unsafe, - .unsafe_clone_fn = git_hash_sha1_clone_unsafe, - .unsafe_update_fn = git_hash_sha1_update_unsafe, - .unsafe_final_fn = git_hash_sha1_final_unsafe, - .unsafe_final_oid_fn = git_hash_sha1_final_oid_unsafe, .unsafe = &sha1_unsafe_algo, .empty_tree = &empty_tree_oid, .empty_blob = &empty_blob_oid, @@ -271,11 +261,6 @@ const struct git_hash_algo hash_algos[GIT_HASH_NALGOS] = { .update_fn = git_hash_sha256_update, .final_fn = git_hash_sha256_final, .final_oid_fn = git_hash_sha256_final_oid, - .unsafe_init_fn = git_hash_sha256_init, - .unsafe_clone_fn = git_hash_sha256_clone, - .unsafe_update_fn = git_hash_sha256_update, - .unsafe_final_fn = git_hash_sha256_final, - .unsafe_final_oid_fn = git_hash_sha256_final_oid, .empty_tree = &empty_tree_oid_sha256, .empty_blob = &empty_blob_oid_sha256, .null_oid = &null_oid_sha256,