From patchwork Mon Oct 2 02:40:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13405521 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4812AE784A7 for ; Mon, 2 Oct 2023 02:40:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234028AbjJBCko (ORCPT ); Sun, 1 Oct 2023 22:40:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39232 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229930AbjJBCkn (ORCPT ); Sun, 1 Oct 2023 22:40:43 -0400 Received: from mail-io1-xd30.google.com (mail-io1-xd30.google.com [IPv6:2607:f8b0:4864:20::d30]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 88A2FA7 for ; Sun, 1 Oct 2023 19:40:41 -0700 (PDT) Received: by mail-io1-xd30.google.com with SMTP id ca18e2360f4ac-79fb78e297bso459760539f.0 for ; Sun, 01 Oct 2023 19:40:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696214441; x=1696819241; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2bBvs7TmTv1cZQ6idsEoGl4pr/Ceq6rO09b32kcUBoA=; b=cDeuT86x7svWrYywr2ZEV3HUrNm0LZ7Xh4FWnOcv8q9kGN5/pytdEr3QrK4841vRaR UjzwE6HuUyJgSpoNA2ts9MAOMiGj2dZAueFMW2h+j8MNoESfYj+t5dnPKigQL0lDcxZz BIWQJQrztLdrO9UQS5HaSAUINxJ8b0oTKZtfsoYc3ys6Z8vUS69gn96wGmLp4+Bdr/9U gHNBIHYsbFGnx7SsJw55htwjgnc1N8ZILxTXc0bxwp8cOJ3R2iOfCUNYLlLbv50CYNXC XLywIfXlioRSZAXAnotoaCCgTs6KN4JmgGxJDT2av/wpYAnfhSHtoQXGn9BPiZHZbbxR EROw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696214441; x=1696819241; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2bBvs7TmTv1cZQ6idsEoGl4pr/Ceq6rO09b32kcUBoA=; b=A2245FV8drSPBG7ygkAT2vVNUrV6i74rjo+sA8duFBZUsD8aM6tI+zMRcIGX+WXONP UCXrofPdAfXaij9LOVtMw00cKzAL2au1T5hYqGEIVvFVuZBaNaN02XR/9uSguW8rPrra efi2yX6lvIvQpZqAEUeyT7CG4EyczCgA/J0Z9oCyfZAkGIWymN6DerxCSkMw9Sfln5wk UoWzE+DHd5rjPNq3mS3ZlLDkVEPMx7CJXHWM1eYtJ1E/5Xydw0WutBLHCGWOUB73HGAT p/2RXKXDXgLXW2HKvxi2VBZXP0zAxLHXHl8j3KriLzfnKP2SDeMDo9Mr9jvIfsNl+v0U inFw== X-Gm-Message-State: AOJu0YxyeJ6fiWr5TBS0dUFspp7RASl3LNgUsvDaF/A3qlT8n5odzscS Q4vtL3gFBc7xAwtVRRPurbM= X-Google-Smtp-Source: AGHT+IGfFgc3Y05d1ydru92uqd4h/vIqckns3iJq3DqjLcBMc44aJtfuLDIJ2xAtE3ktcWHPgwlUFw== X-Received: by 2002:a6b:e30d:0:b0:792:96e5:962 with SMTP id u13-20020a6be30d000000b0079296e50962mr10931833ioc.6.1696214440870; Sun, 01 Oct 2023 19:40:40 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id v6-20020a056602058600b0079f9f7a2565sm6346113iox.38.2023.10.01.19.40.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Oct 2023 19:40:40 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , Eric Sunshine , "Eric W. Biederman" Subject: [PATCH v2 01/30] object-file-convert: stubs for converting from one object format to another Date: Sun, 1 Oct 2023 21:40:05 -0500 Message-Id: <20231002024034.2611-1-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> References: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "Eric W. Biederman" Two basic functions are provided: - convert_object_file Takes an object file it's type and hash algorithm and converts it into the equivalent object file that would have been generated with hash algorithm "to". For blob objects there is no conversation to be done and it is an error to use this function on them. For commit, tree, and tag objects embedded oids are replaced by the oids of the objects they refer to with those objects and their object ids reencoded in with the hash algorithm "to". Signatures are rearranged so that they remain valid after the object has been reencoded. - repo_oid_to_algop which takes an oid that refers to an object file and returns the oid of the equivalent object file generated with the target hash algorithm. The pair of files object-file-convert.c and object-file-convert.h are introduced to hold as much of this logic as possible to keep this conversion logic cleanly separated from everything else and in the hopes that someday the code will be clean enough git can support compiling out support for sha1 and the various conversion functions. Signed-off-by: "Eric W. Biederman" --- Makefile | 1 + object-file-convert.c | 57 +++++++++++++++++++++++++++++++++++++++++++ object-file-convert.h | 24 ++++++++++++++++++ 3 files changed, 82 insertions(+) create mode 100644 object-file-convert.c create mode 100644 object-file-convert.h diff --git a/Makefile b/Makefile index 577630936535..f7e824f25cda 100644 --- a/Makefile +++ b/Makefile @@ -1073,6 +1073,7 @@ LIB_OBJS += notes-cache.o LIB_OBJS += notes-merge.o LIB_OBJS += notes-utils.o LIB_OBJS += notes.o +LIB_OBJS += object-file-convert.o LIB_OBJS += object-file.o LIB_OBJS += object-name.o LIB_OBJS += object.o diff --git a/object-file-convert.c b/object-file-convert.c new file mode 100644 index 000000000000..4777aba83636 --- /dev/null +++ b/object-file-convert.c @@ -0,0 +1,57 @@ +#include "git-compat-util.h" +#include "gettext.h" +#include "strbuf.h" +#include "repository.h" +#include "hash-ll.h" +#include "object.h" +#include "object-file-convert.h" + +int repo_oid_to_algop(struct repository *repo, const struct object_id *src, + const struct git_hash_algo *to, struct object_id *dest) +{ + /* + * If the source algorithm is not set, then we're using the + * default hash algorithm for that object. + */ + const struct git_hash_algo *from = + src->algo ? &hash_algos[src->algo] : repo->hash_algo; + + if (from == to) { + if (src != dest) + oidcpy(dest, src); + return 0; + } + return -1; +} + +int convert_object_file(struct strbuf *outbuf, + const struct git_hash_algo *from, + const struct git_hash_algo *to, + const void *buf, size_t len, + enum object_type type, + int gentle) +{ + int ret; + + /* Don't call this function when no conversion is necessary */ + if ((from == to) || (type == OBJ_BLOB)) + BUG("Refusing noop object file conversion"); + + switch (type) { + case OBJ_COMMIT: + case OBJ_TREE: + case OBJ_TAG: + default: + /* Not implemented yet, so fail. */ + ret = -1; + break; + } + if (!ret) + return 0; + if (gentle) { + strbuf_release(outbuf); + return ret; + } + die(_("Failed to convert object from %s to %s"), + from->name, to->name); +} diff --git a/object-file-convert.h b/object-file-convert.h new file mode 100644 index 000000000000..a4f802aa8eea --- /dev/null +++ b/object-file-convert.h @@ -0,0 +1,24 @@ +#ifndef OBJECT_CONVERT_H +#define OBJECT_CONVERT_H + +struct repository; +struct object_id; +struct git_hash_algo; +struct strbuf; +#include "object.h" + +int repo_oid_to_algop(struct repository *repo, const struct object_id *src, + const struct git_hash_algo *to, struct object_id *dest); + +/* + * Convert an object file from one hash algorithm to another algorithm. + * Return -1 on failure, 0 on success. + */ +int convert_object_file(struct strbuf *outbuf, + const struct git_hash_algo *from, + const struct git_hash_algo *to, + const void *buf, size_t len, + enum object_type type, + int gentle); + +#endif /* OBJECT_CONVERT_H */ From patchwork Mon Oct 2 02:40:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13405522 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3335EE784A6 for ; Mon, 2 Oct 2023 02:40:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235120AbjJBCkq (ORCPT ); Sun, 1 Oct 2023 22:40:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39238 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229930AbjJBCko (ORCPT ); Sun, 1 Oct 2023 22:40:44 -0400 Received: from mail-io1-xd2d.google.com (mail-io1-xd2d.google.com [IPv6:2607:f8b0:4864:20::d2d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 55FE8C9 for ; Sun, 1 Oct 2023 19:40:42 -0700 (PDT) Received: by mail-io1-xd2d.google.com with SMTP id ca18e2360f4ac-79fd60f40ebso60581639f.1 for ; Sun, 01 Oct 2023 19:40:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696214441; x=1696819241; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=OXaIo4l7bppGgbiSoswt2zYWU1XZwpLqCFOmAhHkfkE=; b=dC3eoVOVnYExj691ZTUbtZKxoLWJvuYlx0nz//7FwGhiscEeZdHWdqJRtildB695uf LFima/IPKSABtp97OcQAkgs47OZjsP9TnWW1J4HrJVkRwDB8UifCyDKg4n2Ve0g4+Xyk cWv19IVezS356WSi09jR8NSUCk3MzhqWL9f/u99arVwgwhaBd/YeRBAA1Ywv6AAPm/d4 Q2YqzhSTy6QbfCwt1VEU6ymwViYJ+jxYLeIbCom0bw+qGbX15GTyLHy34Q5VL6EtZgYg OuPUCz58ZgHD6/TkAyZ4YX9T4KrnJKPR39qIOF2VXxEv62QNiE4JtInPMXrqX2BGuLLz WT5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696214441; x=1696819241; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=OXaIo4l7bppGgbiSoswt2zYWU1XZwpLqCFOmAhHkfkE=; b=BGzVo/KkdWFkHKzLwOxmHXc2mBYBkwkg9JHSwtx+wvgQNvfsZiXq+gojicqwqG6q5+ fjKCbHpL5NKt5nR57XfsLMyyemguXnaNwqPcujaFYB4Lvn2LgHmwie7O3pSyfz8sK6N7 XCFAwBBXAF4HKo5C1vkiZaGa+WNUV6eZxt+7C8VFsaV+Uf2CM7ktptewX++o59OKgM+e 4CY9iUi+7SEb27NlSjbVuz6FkTilc9yaiXLw7uV0YPTp8RsZmPsskzSNQyOgmgw5s7MO Pwzym9LTHcf2Oy0XLCQ7wswOO1VHJAQtk1lwUsbZ24rVzysxy2y2rqOMN0GySNuf62q1 Jv7w== X-Gm-Message-State: AOJu0YwyvCDtZVM1VdtkzoQjAts7ru2fjCNnUeJOcr3Rn23d5qlBFxCb xVNuBqpqQouzQfA5JzaRC8I= X-Google-Smtp-Source: AGHT+IH9TTUyjJQ6oB2DqUv9fUwlVmkzqpAi8e7x31M2r3J8q2s+SnMNuYEYv1mdiBzr7HH8BYUfuQ== X-Received: by 2002:a05:6602:2577:b0:790:b44f:b9ee with SMTP id dj23-20020a056602257700b00790b44fb9eemr8016568iob.10.1696214441615; Sun, 01 Oct 2023 19:40:41 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id v6-20020a056602058600b0079f9f7a2565sm6346113iox.38.2023.10.01.19.40.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Oct 2023 19:40:41 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , Eric Sunshine , "Eric W. Biederman" Subject: [PATCH v2 02/30] oid-array: teach oid-array to handle multiple kinds of oids Date: Sun, 1 Oct 2023 21:40:06 -0500 Message-Id: <20231002024034.2611-2-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> References: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "Eric W. Biederman" While looking at how to handle input of both SHA-1 and SHA-256 oids in get_oid_with_context, I realized that the oid_array in repo_for_each_abbrev might have more than one kind of oid stored in it simultaneously. Update to oid_array_append to ensure that oids added to an oid array always have an algorithm set. Update void_hashcmp to first verify two oids use the same hash algorithm before comparing them to each other. With that oid-array should be safe to use with different kinds of oids simultaneously. Signed-off-by: "Eric W. Biederman" --- oid-array.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/oid-array.c b/oid-array.c index 8e4717746c31..1f36651754ed 100644 --- a/oid-array.c +++ b/oid-array.c @@ -6,12 +6,20 @@ void oid_array_append(struct oid_array *array, const struct object_id *oid) { ALLOC_GROW(array->oid, array->nr + 1, array->alloc); oidcpy(&array->oid[array->nr++], oid); + if (!oid->algo) + oid_set_algo(&array->oid[array->nr - 1], the_hash_algo); array->sorted = 0; } -static int void_hashcmp(const void *a, const void *b) +static int void_hashcmp(const void *va, const void *vb) { - return oidcmp(a, b); + const struct object_id *a = va, *b = vb; + int ret; + if (a->algo == b->algo) + ret = oidcmp(a, b); + else + ret = a->algo > b->algo ? 1 : -1; + return ret; } void oid_array_sort(struct oid_array *array) From patchwork Mon Oct 2 02:40:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13405523 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 70B94E784A7 for ; Mon, 2 Oct 2023 02:40:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235342AbjJBCkr (ORCPT ); Sun, 1 Oct 2023 22:40:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39240 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235137AbjJBCkq (ORCPT ); Sun, 1 Oct 2023 22:40:46 -0400 Received: from mail-io1-xd2b.google.com (mail-io1-xd2b.google.com [IPv6:2607:f8b0:4864:20::d2b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 38F9EA7 for ; Sun, 1 Oct 2023 19:40:43 -0700 (PDT) Received: by mail-io1-xd2b.google.com with SMTP id ca18e2360f4ac-79fa2125e19so507466239f.0 for ; Sun, 01 Oct 2023 19:40:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696214442; x=1696819242; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=YJjdapxWuM0wr8A2uYOG1s6AXd+bHvGi5oBm5BjCs8k=; b=QMIWP8hqEFGIWktoe06gN0ewDkVUAFdhFxLBUZNv5dwZTAJor2jVcA2gYJyml097co /OJzGaLf//mdnU7oiWCDsll58y0ouvx+AyaYYGcW8ca2DSb22GY+0IzSRavxTT6mUB3J 6HuSyxap07QXzBtIaQ6/x+bcKloka3leW7qdV7ZkTgRDWZU3PxDbObDxAxD843bPtJCX lIoAV8rnqmUnCPElSOk+GiKnfCzgnv/MOVQZ8FB1N1yUZOVuz79UrFYPlktRsGi8zedi PoIH6eT4LVxZ5MBeT3D6W2QwOpXbKaY3Ax0/7FCP0K2+SvapiSSMaBu7XNFSF9BAOApk +tvQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696214442; x=1696819242; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YJjdapxWuM0wr8A2uYOG1s6AXd+bHvGi5oBm5BjCs8k=; b=YkkrsLeuB/rdAurtROYR7ADAXURZlm80eUu9QjKwJEknRsazDtzdy87EVYd31IS2af 3CxsjCE0QI/bTLaemdmoBVsBmsgPGbg/gKIogsaiNc3YK1givKkXAayIZfD90+AsIdIk f3/vyQoedNIny7Zuwgvf/8MD61mC17W2a15jNC9QiyGslBYyO5FjorITRMFZfjXWe/E2 QrlU5W0rE3KiIUJZf6j/nxxxxyVrpoRsnB3cT+QOaJBYBBEtviY3IjexQVRB+7VhACZw giib4etkfT5ZOl/ldMWJ8JUW/SCqIq3edv/Q+RJu7Od3GwQmlssUceE43B+q9QtrJ/3q rYdg== X-Gm-Message-State: AOJu0Yxc7iTlgfKqOmMZUiumr5ybVXTfz+JqgJAD1kJn/UPO8mjcfhID BLnQD0sRz63nPxAljkLVepshkfD0Shnuyg== X-Google-Smtp-Source: AGHT+IHtTey+Jo8t6L+/X45lga4O8DXaTKiyL9MU6b4Bl9hkoS9odM2RvdAsGdTe1ElMarWfarZkDA== X-Received: by 2002:a05:6602:21d1:b0:79a:bbd2:8f5a with SMTP id c17-20020a05660221d100b0079abbd28f5amr11799381ioc.12.1696214442505; Sun, 01 Oct 2023 19:40:42 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id v6-20020a056602058600b0079f9f7a2565sm6346113iox.38.2023.10.01.19.40.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Oct 2023 19:40:42 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , Eric Sunshine , "Eric W. Biederman" Subject: [PATCH v2 03/30] object-names: support input of oids in any supported hash Date: Sun, 1 Oct 2023 21:40:07 -0500 Message-Id: <20231002024034.2611-3-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> References: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "Eric W. Biederman" Support short oids encoded in any algorithm, while ensuring enough of the oid is specified to disambiguate between all of the oids in the repository encoded in any algorithm. By default have the code continue to only accept oids specified in the storage hash algorithm of the repository, but when something is ambiguous display all of the possible oids from any accepted oid encoding. A new flag is added GET_OID_HASH_ANY that when supplied causes the code to accept oids specified in any hash algorithm, and to return the oids that were resolved. This implements the functionality that allows both SHA-1 and SHA-256 object names, from the "Object names on the command line" section of the hash function transition document. Care is taken in get_short_oid so that when the result is ambiguous the output remains the same if GIT_OID_HASH_ANY was not supplied. If GET_OID_HASH_ANY was supplied objects of any hash algorithm that match the prefix are displayed. This required updating repo_for_each_abbrev to give it a parameter so that it knows to look at all hash algorithms. Signed-off-by: "Eric W. Biederman" --- builtin/rev-parse.c | 2 +- hash-ll.h | 1 + object-name.c | 46 ++++++++++++++++++++++++++++++++++----------- object-name.h | 3 ++- 4 files changed, 39 insertions(+), 13 deletions(-) diff --git a/builtin/rev-parse.c b/builtin/rev-parse.c index fde8861ca4e0..43e96765400c 100644 --- a/builtin/rev-parse.c +++ b/builtin/rev-parse.c @@ -882,7 +882,7 @@ int cmd_rev_parse(int argc, const char **argv, const char *prefix) continue; } if (skip_prefix(arg, "--disambiguate=", &arg)) { - repo_for_each_abbrev(the_repository, arg, + repo_for_each_abbrev(the_repository, arg, the_hash_algo, show_abbrev, NULL); continue; } diff --git a/hash-ll.h b/hash-ll.h index 10d84cc20888..2cfde63ae1cf 100644 --- a/hash-ll.h +++ b/hash-ll.h @@ -145,6 +145,7 @@ struct object_id { #define GET_OID_RECORD_PATH 0200 #define GET_OID_ONLY_TO_DIE 04000 #define GET_OID_REQUIRE_PATH 010000 +#define GET_OID_HASH_ANY 020000 #define GET_OID_DISAMBIGUATORS \ (GET_OID_COMMIT | GET_OID_COMMITTISH | \ diff --git a/object-name.c b/object-name.c index 0bfa29dbbfe9..7dd6e5e47566 100644 --- a/object-name.c +++ b/object-name.c @@ -25,6 +25,7 @@ #include "midx.h" #include "commit-reach.h" #include "date.h" +#include "object-file-convert.h" static int get_oid_oneline(struct repository *r, const char *, struct object_id *, struct commit_list *); @@ -49,6 +50,7 @@ struct disambiguate_state { static void update_candidates(struct disambiguate_state *ds, const struct object_id *current) { + /* The hash algorithm of current has already been filtered */ if (ds->always_call_fn) { ds->ambiguous = ds->fn(ds->repo, current, ds->cb_data) ? 1 : 0; return; @@ -134,6 +136,8 @@ static void unique_in_midx(struct multi_pack_index *m, { uint32_t num, i, first = 0; const struct object_id *current = NULL; + int len = ds->len > ds->repo->hash_algo->hexsz ? + ds->repo->hash_algo->hexsz : ds->len; num = m->num_objects; if (!num) @@ -149,7 +153,7 @@ static void unique_in_midx(struct multi_pack_index *m, for (i = first; i < num && !ds->ambiguous; i++) { struct object_id oid; current = nth_midxed_object_oid(&oid, m, i); - if (!match_hash(ds->len, ds->bin_pfx.hash, current->hash)) + if (!match_hash(len, ds->bin_pfx.hash, current->hash)) break; update_candidates(ds, current); } @@ -159,6 +163,8 @@ static void unique_in_pack(struct packed_git *p, struct disambiguate_state *ds) { uint32_t num, i, first = 0; + int len = ds->len > ds->repo->hash_algo->hexsz ? + ds->repo->hash_algo->hexsz : ds->len; if (p->multi_pack_index) return; @@ -177,7 +183,7 @@ static void unique_in_pack(struct packed_git *p, for (i = first; i < num && !ds->ambiguous; i++) { struct object_id oid; nth_packed_object_id(&oid, p, i); - if (!match_hash(ds->len, ds->bin_pfx.hash, oid.hash)) + if (!match_hash(len, ds->bin_pfx.hash, oid.hash)) break; update_candidates(ds, &oid); } @@ -188,6 +194,10 @@ static void find_short_packed_object(struct disambiguate_state *ds) struct multi_pack_index *m; struct packed_git *p; + /* Skip, unless oids from the storage hash algorithm are wanted */ + if (ds->bin_pfx.algo && (&hash_algos[ds->bin_pfx.algo] != ds->repo->hash_algo)) + return; + for (m = get_multi_pack_index(ds->repo); m && !ds->ambiguous; m = m->next) unique_in_midx(m, ds); @@ -326,11 +336,12 @@ int set_disambiguate_hint_config(const char *var, const char *value) static int init_object_disambiguation(struct repository *r, const char *name, int len, + const struct git_hash_algo *algo, struct disambiguate_state *ds) { int i; - if (len < MINIMUM_ABBREV || len > the_hash_algo->hexsz) + if (len < MINIMUM_ABBREV || len > GIT_MAX_HEXSZ) return -1; memset(ds, 0, sizeof(*ds)); @@ -357,6 +368,7 @@ static int init_object_disambiguation(struct repository *r, ds->len = len; ds->hex_pfx[len] = '\0'; ds->repo = r; + ds->bin_pfx.algo = algo ? hash_algo_by_ptr(algo) : GIT_HASH_UNKNOWN; prepare_alt_odb(r); return 0; } @@ -491,9 +503,10 @@ static int repo_collect_ambiguous(struct repository *r UNUSED, return collect_ambiguous(oid, data); } -static int sort_ambiguous(const void *a, const void *b, void *ctx) +static int sort_ambiguous(const void *va, const void *vb, void *ctx) { struct repository *sort_ambiguous_repo = ctx; + const struct object_id *a = va, *b = vb; int a_type = oid_object_info(sort_ambiguous_repo, a, NULL); int b_type = oid_object_info(sort_ambiguous_repo, b, NULL); int a_type_sort; @@ -503,8 +516,12 @@ static int sort_ambiguous(const void *a, const void *b, void *ctx) * Sorts by hash within the same object type, just as * oid_array_for_each_unique() would do. */ - if (a_type == b_type) - return oidcmp(a, b); + if (a_type == b_type) { + if (a->algo == b->algo) + return oidcmp(a, b); + else + return a->algo > b->algo ? 1 : -1; + } /* * Between object types show tags, then commits, and finally @@ -533,8 +550,12 @@ static enum get_oid_result get_short_oid(struct repository *r, int status; struct disambiguate_state ds; int quietly = !!(flags & GET_OID_QUIETLY); + const struct git_hash_algo *algo = r->hash_algo; + + if (flags & GET_OID_HASH_ANY) + algo = NULL; - if (init_object_disambiguation(r, name, len, &ds) < 0) + if (init_object_disambiguation(r, name, len, algo, &ds) < 0) return -1; if (HAS_MULTI_BITS(flags & GET_OID_DISAMBIGUATORS)) @@ -588,7 +609,7 @@ static enum get_oid_result get_short_oid(struct repository *r, if (!ds.ambiguous) ds.fn = NULL; - repo_for_each_abbrev(r, ds.hex_pfx, collect_ambiguous, &collect); + repo_for_each_abbrev(r, ds.hex_pfx, algo, collect_ambiguous, &collect); sort_ambiguous_oid_array(r, &collect); if (oid_array_for_each(&collect, show_ambiguous_object, &out)) @@ -610,13 +631,14 @@ static enum get_oid_result get_short_oid(struct repository *r, } int repo_for_each_abbrev(struct repository *r, const char *prefix, + const struct git_hash_algo *algo, each_abbrev_fn fn, void *cb_data) { struct oid_array collect = OID_ARRAY_INIT; struct disambiguate_state ds; int ret; - if (init_object_disambiguation(r, prefix, strlen(prefix), &ds) < 0) + if (init_object_disambiguation(r, prefix, strlen(prefix), algo, &ds) < 0) return -1; ds.always_call_fn = 1; @@ -787,10 +809,12 @@ void strbuf_add_unique_abbrev(struct strbuf *sb, const struct object_id *oid, int repo_find_unique_abbrev_r(struct repository *r, char *hex, const struct object_id *oid, int len) { + const struct git_hash_algo *algo = + oid->algo ? &hash_algos[oid->algo] : r->hash_algo; struct disambiguate_state ds; struct min_abbrev_data mad; struct object_id oid_ret; - const unsigned hexsz = r->hash_algo->hexsz; + const unsigned hexsz = algo->hexsz; if (len < 0) { unsigned long count = repo_approximate_object_count(r); @@ -826,7 +850,7 @@ int repo_find_unique_abbrev_r(struct repository *r, char *hex, find_abbrev_len_packed(&mad); - if (init_object_disambiguation(r, hex, mad.cur_len, &ds) < 0) + if (init_object_disambiguation(r, hex, mad.cur_len, algo, &ds) < 0) return -1; ds.fn = repo_extend_abbrev_len; diff --git a/object-name.h b/object-name.h index 9ae522307148..064ddc97d1fe 100644 --- a/object-name.h +++ b/object-name.h @@ -67,7 +67,8 @@ enum get_oid_result get_oid_with_context(struct repository *repo, const char *st typedef int each_abbrev_fn(const struct object_id *oid, void *); -int repo_for_each_abbrev(struct repository *r, const char *prefix, each_abbrev_fn, void *); +int repo_for_each_abbrev(struct repository *r, const char *prefix, + const struct git_hash_algo *algo, each_abbrev_fn, void *); int set_disambiguate_hint_config(const char *var, const char *value); From patchwork Mon Oct 2 02:40:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13405524 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 22AEDE784A7 for ; Mon, 2 Oct 2023 02:40:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235396AbjJBCkv (ORCPT ); Sun, 1 Oct 2023 22:40:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39242 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229930AbjJBCkq (ORCPT ); Sun, 1 Oct 2023 22:40:46 -0400 Received: from mail-io1-xd32.google.com (mail-io1-xd32.google.com [IPv6:2607:f8b0:4864:20::d32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DB6C1C9 for ; Sun, 1 Oct 2023 19:40:43 -0700 (PDT) Received: by mail-io1-xd32.google.com with SMTP id ca18e2360f4ac-7a2a3fb0713so16445539f.1 for ; Sun, 01 Oct 2023 19:40:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696214443; x=1696819243; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=VoAjroQO1iVmfkizqGV2J0Hl4Oax5kmQ5faEhmWaBNI=; b=Tw2Nt6pKJaxNKvJiv7KoTfJaLuIjoWskYGERpeO7jSst7FZuSHxSKCPd+HlDB9nQ5v yd5schl4kEcr554UMPsOqX1mGva5/laCmP4tYhgb/OZRZXF8a+m4EQzjH/t0PmzPUxGt VC8U42bYbeyAdkOejAB7pVFPS1Zj6Wl1nk05Y2PbLtFevS5ZCQY0SZMEXAHBdrEXEczU zH0RIXR6w2zn9juP303nnGMpaDsAQkEPmIxKxpU4JHhLfYYCxSk8GuYLuMtUcn/2er/u s/g5WtZVTmulrpogXlmsrragCrf33SAEq7hY6T6DCTwZE2JFygsBvsHMwvKThrXoseA+ 3FoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696214443; x=1696819243; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VoAjroQO1iVmfkizqGV2J0Hl4Oax5kmQ5faEhmWaBNI=; b=aoD7bn+Ile8TpeYKkIYuOhpfljY5e2BpD+HEhtmQwpEfUZ0c2Vvo9uRBhiaOhK6v3k KsCkzw1pM1PrDp7bZyOIyK8D6A4jSalaIgIXeFBc6LqP+cy/bYeQtLJwHczUuitcGo7C XTx7gHFxGZL5LFxJO26D6B5gJ6Xr2t0pRwlKKsOF4c8pmZ1AQ2svS4rrw2Oye/WqPssh BRruUdfeYc5Y/oBigbw/kNsKqLo1u/np68BT+h9ihne/MO7tuw0Ikhj7b4ZX7NVJviy+ 7ImjhQcoC3QKT0ogJmrlimq9woQNPstn/4rh5V7xIQQDaGcKDU0ZRR+zO7SlzK5axnjk HlBg== X-Gm-Message-State: AOJu0YyeKrGJ1MgDigNNaXPfsuXqwzZgazz0zZ2SBJ4+ArjGabsBfQhI lS++I+c4R7BR8gcJwBcKazy19cB71U9UDQ== X-Google-Smtp-Source: AGHT+IFjaVdBIAyPY5SVk/2vzddHvW+CfyjaiTh7IEiCzdhbq6xHMqk5OZym+rOSMxA0PUp+hwhVcg== X-Received: by 2002:a6b:e211:0:b0:795:1c40:a24e with SMTP id z17-20020a6be211000000b007951c40a24emr11655394ioc.12.1696214443271; Sun, 01 Oct 2023 19:40:43 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id v6-20020a056602058600b0079f9f7a2565sm6346113iox.38.2023.10.01.19.40.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Oct 2023 19:40:42 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , Eric Sunshine , "Eric W. Biederman" Subject: [PATCH v2 04/30] repository: add a compatibility hash algorithm Date: Sun, 1 Oct 2023 21:40:08 -0500 Message-Id: <20231002024034.2611-4-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> References: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "Eric W. Biederman" We currently have support for using a full stage 4 SHA-256 implementation. However, we'd like to support interoperability with SHA-1 repositories as well. The transition plan anticipates a compatibility hash algorithm configuration option that we can use to implement support for this. Let's add an element to the repository structure that indicates the compatibility hash algorithm so we can use it when we need to consider interoperability between algorithms. Add a helper function repo_set_compat_hash_algo that takes a compatibility hash algorithm and sets "repo->compat_hash_algo". If GIT_HASH_UNKNOWN is passed as the compatibility hash algorithm "repo->compat_hash_algo" is set to NULL. For now, the code results in "repo->compat_hash_algo" always being set to NULL, but that will change once a configuration option is added. Inspired-by: brian m. carlson Signed-off-by: Eric W. Biederman --- repository.c | 8 ++++++++ repository.h | 4 ++++ setup.c | 3 +++ 3 files changed, 15 insertions(+) diff --git a/repository.c b/repository.c index a7679ceeaa45..80252b79e93e 100644 --- a/repository.c +++ b/repository.c @@ -104,6 +104,13 @@ void repo_set_hash_algo(struct repository *repo, int hash_algo) repo->hash_algo = &hash_algos[hash_algo]; } +void repo_set_compat_hash_algo(struct repository *repo, int algo) +{ + if (hash_algo_by_ptr(repo->hash_algo) == algo) + BUG("hash_algo and compat_hash_algo match"); + repo->compat_hash_algo = algo ? &hash_algos[algo] : NULL; +} + /* * Attempt to resolve and set the provided 'gitdir' for repository 'repo'. * Return 0 upon success and a non-zero value upon failure. @@ -184,6 +191,7 @@ int repo_init(struct repository *repo, goto error; repo_set_hash_algo(repo, format.hash_algo); + repo_set_compat_hash_algo(repo, GIT_HASH_UNKNOWN); repo->repository_format_worktree_config = format.worktree_config; /* take ownership of format.partial_clone */ diff --git a/repository.h b/repository.h index 5f18486f6465..bf3fc601cc53 100644 --- a/repository.h +++ b/repository.h @@ -160,6 +160,9 @@ struct repository { /* Repository's current hash algorithm, as serialized on disk. */ const struct git_hash_algo *hash_algo; + /* Repository's compatibility hash algorithm. */ + const struct git_hash_algo *compat_hash_algo; + /* A unique-id for tracing purposes. */ int trace2_repo_id; @@ -199,6 +202,7 @@ void repo_set_gitdir(struct repository *repo, const char *root, const struct set_gitdir_args *extra_args); void repo_set_worktree(struct repository *repo, const char *path); void repo_set_hash_algo(struct repository *repo, int algo); +void repo_set_compat_hash_algo(struct repository *repo, int compat_algo); void initialize_the_repository(void); RESULT_MUST_BE_USED int repo_init(struct repository *r, const char *gitdir, const char *worktree); diff --git a/setup.c b/setup.c index 18927a847b86..aa8bf5da5226 100644 --- a/setup.c +++ b/setup.c @@ -1564,6 +1564,8 @@ const char *setup_git_directory_gently(int *nongit_ok) } if (startup_info->have_repository) { repo_set_hash_algo(the_repository, repo_fmt.hash_algo); + repo_set_compat_hash_algo(the_repository, + GIT_HASH_UNKNOWN); the_repository->repository_format_worktree_config = repo_fmt.worktree_config; /* take ownership of repo_fmt.partial_clone */ @@ -1657,6 +1659,7 @@ void check_repository_format(struct repository_format *fmt) check_repository_format_gently(get_git_dir(), fmt, NULL); startup_info->have_repository = 1; repo_set_hash_algo(the_repository, fmt->hash_algo); + repo_set_compat_hash_algo(the_repository, GIT_HASH_UNKNOWN); the_repository->repository_format_worktree_config = fmt->worktree_config; the_repository->repository_format_partial_clone = From patchwork Mon Oct 2 02:40:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13405526 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F22FAE784A7 for ; Mon, 2 Oct 2023 02:40:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235428AbjJBCky (ORCPT ); Sun, 1 Oct 2023 22:40:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41222 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235363AbjJBCks (ORCPT ); Sun, 1 Oct 2023 22:40:48 -0400 Received: from mail-io1-xd2d.google.com (mail-io1-xd2d.google.com [IPv6:2607:f8b0:4864:20::d2d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CAD7BD3 for ; Sun, 1 Oct 2023 19:40:44 -0700 (PDT) Received: by mail-io1-xd2d.google.com with SMTP id ca18e2360f4ac-79fa2dbd793so514409139f.2 for ; Sun, 01 Oct 2023 19:40:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696214444; x=1696819244; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/0I9pZq6uNndUtKlD223KO9nnA46VNsfxekaStGQ6U8=; b=eYS9YHnONoYhPZ4rzNpNurBNR9Er808z3CLyf32+mkUIFSE1uKemTVOiArW9YNOVdw /yPjf50IZ8ot5Rdm9DAEXw/jAmU7Vq/joy3gJG0tNJmkU4A4HUkvjdHxqjF1KAOh9l3x lzN93CEAV6Vg942ylam7uSBa7tJPVXlgUV4opGcAIW1lb400DqFkN5DmAFSPYpqgqlzV dbcyFY1O+JOHjq+dM0M9V7p4wiaRSPs/l71GDxGP491bBbUSDmHgQyJ0JlvhyPwL3bmc ugpJYxYsYketkznYCACHxvQ94dSVhWVnsKLaQ3hDfVXJuCxBzA5ttjlkgiKSiI+GRiJ4 ivxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696214444; x=1696819244; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/0I9pZq6uNndUtKlD223KO9nnA46VNsfxekaStGQ6U8=; b=DiNpHVeCaNiz5J/fPZJzuIpq79SzU8DzpOV7SMyZvSEyZYBrq9KGz/ETSu7b+iuHqr hFQrLGOWOc6p4dTdqB6etOOvW/M/fxJy62KFqU8kXdaaLK0MYTok/19l58sOHI7ppp4r j7POoI8aXN82qyKf7fZG+PQ09ZCCOm3iSy3Rqf76MtlTbL/VVIY26wJGGrzmzQEFTuRY traoZGnl+15flwJ5iw+qrgA9y5BhexIq0oGcNHyN0rWpPFEDyE7pOm6sDfNmLUOGuPPt QhvFf2gXbkZZwwaRBAmsCAZWhkUN+t23J4APKlTJ5ig4I419YrigZihbQU5UCooWT26I lSUA== X-Gm-Message-State: AOJu0YyLfOfSUOpc+j0Y167u92svscziGH7Y9KYfSpKpLraSjZ7R3jXT wHFsO96AsI1DEvMdPjJHw5U= X-Google-Smtp-Source: AGHT+IFxgyPG3s9M6sCdwMiDl+Ul0qn3lWLNF95qQd+RiBENHpFbwIEgPa0VmJI3NGxXKFv3sKCxUQ== X-Received: by 2002:a6b:e211:0:b0:790:fab3:2052 with SMTP id z17-20020a6be211000000b00790fab32052mr11097802ioc.5.1696214444073; Sun, 01 Oct 2023 19:40:44 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id v6-20020a056602058600b0079f9f7a2565sm6346113iox.38.2023.10.01.19.40.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Oct 2023 19:40:43 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , Eric Sunshine , "Eric W. Biederman" Subject: [PATCH v2 05/30] loose: add a mapping between SHA-1 and SHA-256 for loose objects Date: Sun, 1 Oct 2023 21:40:09 -0500 Message-Id: <20231002024034.2611-5-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> References: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "brian m. carlson" As part of the transition plan, we'd like to add a file in the .git directory that maps loose objects between SHA-1 and SHA-256. Let's implement the specification in the transition plan and store this data on a per-repository basis in struct repository. Signed-off-by: brian m. carlson Signed-off-by: Eric W. Biederman --- Makefile | 1 + loose.c | 246 ++++++++++++++++++++++++++++++++++++++++++ loose.h | 22 ++++ object-file-convert.c | 14 ++- object-store-ll.h | 3 + object.c | 2 + repository.c | 6 ++ 7 files changed, 293 insertions(+), 1 deletion(-) create mode 100644 loose.c create mode 100644 loose.h diff --git a/Makefile b/Makefile index f7e824f25cda..3c18664def9a 100644 --- a/Makefile +++ b/Makefile @@ -1053,6 +1053,7 @@ LIB_OBJS += list-objects-filter.o LIB_OBJS += list-objects.o LIB_OBJS += lockfile.o LIB_OBJS += log-tree.o +LIB_OBJS += loose.o LIB_OBJS += ls-refs.o LIB_OBJS += mailinfo.o LIB_OBJS += mailmap.o diff --git a/loose.c b/loose.c new file mode 100644 index 000000000000..6ba73cc84dca --- /dev/null +++ b/loose.c @@ -0,0 +1,246 @@ +#include "git-compat-util.h" +#include "hash.h" +#include "path.h" +#include "object-store.h" +#include "hex.h" +#include "wrapper.h" +#include "gettext.h" +#include "loose.h" +#include "lockfile.h" + +static const char *loose_object_header = "# loose-object-idx\n"; + +static inline int should_use_loose_object_map(struct repository *repo) +{ + return repo->compat_hash_algo && repo->gitdir; +} + +void loose_object_map_init(struct loose_object_map **map) +{ + struct loose_object_map *m; + m = xmalloc(sizeof(**map)); + m->to_compat = kh_init_oid_map(); + m->to_storage = kh_init_oid_map(); + *map = m; +} + +static int insert_oid_pair(kh_oid_map_t *map, const struct object_id *key, const struct object_id *value) +{ + khiter_t pos; + int ret; + struct object_id *stored; + + pos = kh_put_oid_map(map, *key, &ret); + + /* This item already exists in the map. */ + if (ret == 0) + return 0; + + stored = xmalloc(sizeof(*stored)); + oidcpy(stored, value); + kh_value(map, pos) = stored; + return 1; +} + +static int load_one_loose_object_map(struct repository *repo, struct object_directory *dir) +{ + struct strbuf buf = STRBUF_INIT, path = STRBUF_INIT; + FILE *fp; + + if (!dir->loose_map) + loose_object_map_init(&dir->loose_map); + + insert_oid_pair(dir->loose_map->to_compat, repo->hash_algo->empty_tree, repo->compat_hash_algo->empty_tree); + insert_oid_pair(dir->loose_map->to_storage, repo->compat_hash_algo->empty_tree, repo->hash_algo->empty_tree); + + insert_oid_pair(dir->loose_map->to_compat, repo->hash_algo->empty_blob, repo->compat_hash_algo->empty_blob); + insert_oid_pair(dir->loose_map->to_storage, repo->compat_hash_algo->empty_blob, repo->hash_algo->empty_blob); + + insert_oid_pair(dir->loose_map->to_compat, repo->hash_algo->null_oid, repo->compat_hash_algo->null_oid); + insert_oid_pair(dir->loose_map->to_storage, repo->compat_hash_algo->null_oid, repo->hash_algo->null_oid); + + strbuf_git_common_path(&path, repo, "objects/loose-object-idx"); + fp = fopen(path.buf, "rb"); + if (!fp) { + strbuf_release(&path); + return 0; + } + + errno = 0; + if (strbuf_getwholeline(&buf, fp, '\n') || strcmp(buf.buf, loose_object_header)) + goto err; + while (!strbuf_getline_lf(&buf, fp)) { + const char *p; + struct object_id oid, compat_oid; + if (parse_oid_hex_algop(buf.buf, &oid, &p, repo->hash_algo) || + *p++ != ' ' || + parse_oid_hex_algop(p, &compat_oid, &p, repo->compat_hash_algo) || + p != buf.buf + buf.len) + goto err; + insert_oid_pair(dir->loose_map->to_compat, &oid, &compat_oid); + insert_oid_pair(dir->loose_map->to_storage, &compat_oid, &oid); + } + + strbuf_release(&buf); + strbuf_release(&path); + return errno ? -1 : 0; +err: + strbuf_release(&buf); + strbuf_release(&path); + return -1; +} + +int repo_read_loose_object_map(struct repository *repo) +{ + struct object_directory *dir; + + if (!should_use_loose_object_map(repo)) + return 0; + + prepare_alt_odb(repo); + + for (dir = repo->objects->odb; dir; dir = dir->next) { + if (load_one_loose_object_map(repo, dir) < 0) { + return -1; + } + } + return 0; +} + +int repo_write_loose_object_map(struct repository *repo) +{ + kh_oid_map_t *map = repo->objects->odb->loose_map->to_compat; + struct lock_file lock; + int fd; + khiter_t iter; + struct strbuf buf = STRBUF_INIT, path = STRBUF_INIT; + + if (!should_use_loose_object_map(repo)) + return 0; + + strbuf_git_common_path(&path, repo, "objects/loose-object-idx"); + fd = hold_lock_file_for_update_timeout(&lock, path.buf, LOCK_DIE_ON_ERROR, -1); + iter = kh_begin(map); + if (write_in_full(fd, loose_object_header, strlen(loose_object_header)) < 0) + goto errout; + + for (; iter != kh_end(map); iter++) { + if (kh_exist(map, iter)) { + if (oideq(&kh_key(map, iter), the_hash_algo->empty_tree) || + oideq(&kh_key(map, iter), the_hash_algo->empty_blob)) + continue; + strbuf_addf(&buf, "%s %s\n", oid_to_hex(&kh_key(map, iter)), oid_to_hex(kh_value(map, iter))); + if (write_in_full(fd, buf.buf, buf.len) < 0) + goto errout; + strbuf_reset(&buf); + } + } + strbuf_release(&buf); + if (commit_lock_file(&lock) < 0) { + error_errno(_("could not write loose object index %s"), path.buf); + strbuf_release(&path); + return -1; + } + strbuf_release(&path); + return 0; +errout: + rollback_lock_file(&lock); + strbuf_release(&buf); + error_errno(_("failed to write loose object index %s\n"), path.buf); + strbuf_release(&path); + return -1; +} + +static int write_one_object(struct repository *repo, const struct object_id *oid, + const struct object_id *compat_oid) +{ + struct lock_file lock; + int fd; + struct stat st; + struct strbuf buf = STRBUF_INIT, path = STRBUF_INIT; + + strbuf_git_common_path(&path, repo, "objects/loose-object-idx"); + hold_lock_file_for_update_timeout(&lock, path.buf, LOCK_DIE_ON_ERROR, -1); + + fd = open(path.buf, O_WRONLY | O_CREAT | O_APPEND, 0666); + if (fd < 0) + goto errout; + if (fstat(fd, &st) < 0) + goto errout; + if (!st.st_size && write_in_full(fd, loose_object_header, strlen(loose_object_header)) < 0) + goto errout; + + strbuf_addf(&buf, "%s %s\n", oid_to_hex(oid), oid_to_hex(compat_oid)); + if (write_in_full(fd, buf.buf, buf.len) < 0) + goto errout; + if (close(fd)) + goto errout; + adjust_shared_perm(path.buf); + rollback_lock_file(&lock); + strbuf_release(&buf); + strbuf_release(&path); + return 0; +errout: + error_errno(_("failed to write loose object index %s\n"), path.buf); + close(fd); + rollback_lock_file(&lock); + strbuf_release(&buf); + strbuf_release(&path); + return -1; +} + +int repo_add_loose_object_map(struct repository *repo, const struct object_id *oid, + const struct object_id *compat_oid) +{ + int inserted = 0; + + if (!should_use_loose_object_map(repo)) + return 0; + + inserted |= insert_oid_pair(repo->objects->odb->loose_map->to_compat, oid, compat_oid); + inserted |= insert_oid_pair(repo->objects->odb->loose_map->to_storage, compat_oid, oid); + if (inserted) + return write_one_object(repo, oid, compat_oid); + return 0; +} + +int repo_loose_object_map_oid(struct repository *repo, + const struct object_id *src, + const struct git_hash_algo *to, + struct object_id *dest) +{ + struct object_directory *dir; + kh_oid_map_t *map; + khiter_t pos; + + for (dir = repo->objects->odb; dir; dir = dir->next) { + struct loose_object_map *loose_map = dir->loose_map; + if (!loose_map) + continue; + map = (to == repo->compat_hash_algo) ? + loose_map->to_compat : + loose_map->to_storage; + pos = kh_get_oid_map(map, *src); + if (pos < kh_end(map)) { + oidcpy(dest, kh_value(map, pos)); + return 0; + } + } + return -1; +} + +void loose_object_map_clear(struct loose_object_map **map) +{ + struct loose_object_map *m = *map; + struct object_id *oid; + + if (!m) + return; + + kh_foreach_value(m->to_compat, oid, free(oid)); + kh_foreach_value(m->to_storage, oid, free(oid)); + kh_destroy_oid_map(m->to_compat); + kh_destroy_oid_map(m->to_storage); + free(m); + *map = NULL; +} diff --git a/loose.h b/loose.h new file mode 100644 index 000000000000..2c2957072c5f --- /dev/null +++ b/loose.h @@ -0,0 +1,22 @@ +#ifndef LOOSE_H +#define LOOSE_H + +#include "khash.h" + +struct loose_object_map { + kh_oid_map_t *to_compat; + kh_oid_map_t *to_storage; +}; + +void loose_object_map_init(struct loose_object_map **map); +void loose_object_map_clear(struct loose_object_map **map); +int repo_loose_object_map_oid(struct repository *repo, + const struct object_id *src, + const struct git_hash_algo *dest_algo, + struct object_id *dest); +int repo_add_loose_object_map(struct repository *repo, const struct object_id *oid, + const struct object_id *compat_oid); +int repo_read_loose_object_map(struct repository *repo); +int repo_write_loose_object_map(struct repository *repo); + +#endif diff --git a/object-file-convert.c b/object-file-convert.c index 4777aba83636..1ec945eaa17f 100644 --- a/object-file-convert.c +++ b/object-file-convert.c @@ -4,6 +4,7 @@ #include "repository.h" #include "hash-ll.h" #include "object.h" +#include "loose.h" #include "object-file-convert.h" int repo_oid_to_algop(struct repository *repo, const struct object_id *src, @@ -21,7 +22,18 @@ int repo_oid_to_algop(struct repository *repo, const struct object_id *src, oidcpy(dest, src); return 0; } - return -1; + if (repo_loose_object_map_oid(repo, src, to, dest)) { + /* + * We may have loaded the object map at repo initialization but + * another process (perhaps upstream of a pipe from us) may have + * written a new object into the map. If the object is missing, + * let's reload the map to see if the object has appeared. + */ + repo_read_loose_object_map(repo); + if (repo_loose_object_map_oid(repo, src, to, dest)) + return -1; + } + return 0; } int convert_object_file(struct strbuf *outbuf, diff --git a/object-store-ll.h b/object-store-ll.h index 26a3895c821c..bc76d6bec80d 100644 --- a/object-store-ll.h +++ b/object-store-ll.h @@ -26,6 +26,9 @@ struct object_directory { uint32_t loose_objects_subdir_seen[8]; /* 256 bits */ struct oidtree *loose_objects_cache; + /* Map between object IDs for loose objects. */ + struct loose_object_map *loose_map; + /* * This is a temporary object store created by the tmp_objdir * facility. Disable ref updates since the objects in the store diff --git a/object.c b/object.c index 2c61e4c86217..186a0a47c0fb 100644 --- a/object.c +++ b/object.c @@ -13,6 +13,7 @@ #include "alloc.h" #include "packfile.h" #include "commit-graph.h" +#include "loose.h" unsigned int get_max_object_index(void) { @@ -540,6 +541,7 @@ void free_object_directory(struct object_directory *odb) { free(odb->path); odb_clear_loose_cache(odb); + loose_object_map_clear(&odb->loose_map); free(odb); } diff --git a/repository.c b/repository.c index 80252b79e93e..6214f61cf4e7 100644 --- a/repository.c +++ b/repository.c @@ -14,6 +14,7 @@ #include "read-cache-ll.h" #include "remote.h" #include "setup.h" +#include "loose.h" #include "submodule-config.h" #include "sparse-index.h" #include "trace2.h" @@ -109,6 +110,8 @@ void repo_set_compat_hash_algo(struct repository *repo, int algo) if (hash_algo_by_ptr(repo->hash_algo) == algo) BUG("hash_algo and compat_hash_algo match"); repo->compat_hash_algo = algo ? &hash_algos[algo] : NULL; + if (repo->compat_hash_algo) + repo_read_loose_object_map(repo); } /* @@ -201,6 +204,9 @@ int repo_init(struct repository *repo, if (worktree) repo_set_worktree(repo, worktree); + if (repo->compat_hash_algo) + repo_read_loose_object_map(repo); + clear_repository_format(&format); return 0; From patchwork Mon Oct 2 02:40:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13405525 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C6E30E784A6 for ; Mon, 2 Oct 2023 02:40:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229930AbjJBCkw (ORCPT ); Sun, 1 Oct 2023 22:40:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41238 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235137AbjJBCks (ORCPT ); Sun, 1 Oct 2023 22:40:48 -0400 Received: from mail-io1-xd2f.google.com (mail-io1-xd2f.google.com [IPv6:2607:f8b0:4864:20::d2f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 89773A7 for ; Sun, 1 Oct 2023 19:40:45 -0700 (PDT) Received: by mail-io1-xd2f.google.com with SMTP id ca18e2360f4ac-7a26fd82847so105002139f.3 for ; Sun, 01 Oct 2023 19:40:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696214445; x=1696819245; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Rg4RBplxhFRN7ZVX9/Aa8dakqT0wYQPCYrnaA581JuA=; b=cV+ZhgKZ1913/dkvVmrVt9aG49rexnRbk5M6xRhA4DqFOgKf40AweqhbTxUPmpeCrc zZDvqZy3WOLNagIo2iPN/YfZlFDpOEE7kVZak8Es5xf6sucbeifua7pE/yXUc1lXm+BQ /MnCRtpwY0UbadgYBUKsDx7iXnuyZXFaIg99Z5VLrkX5SQEf+ZN8F3tASml354lnnQN1 WpZyW6aT12Z+JRf2gkUyzdvD3xTqA7LXcnlyiW+pdYzQon6Lpo+WnxwCNBuQ/B0fhQgi 8gGxMlIgg/siCRIkT7H6/758GdGksUDp6wqOCfeIWtv+V8eKshdoIj9vmiGt6V91lSjD 0+mw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696214445; x=1696819245; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Rg4RBplxhFRN7ZVX9/Aa8dakqT0wYQPCYrnaA581JuA=; b=vdkFWsU+s+362kbnG6jw27PEg5sROr5tDRWS+S4FhBp3mrIiQWvavgY1U3iNgEG+sZ UX/ihvU0QHvG1XAZkOC6Kjh1H83O7mloWuz/x+6NNbZRn1/tM8qnZschunUep3CUI7Xz AE4B3WH//HOIq7Ua6Ub3cZ4HTDGwBYAxpPomWAwYWQD/gpDdZGk9WBbNp6ICPqiyjmIw hvQROQW6uSpMcBHzKzKpRkSOy0JHsGi81q/yWU2cQpdWpzx1KUUKedHAMws5gp7IagoQ aP0lzNfBEX9iXxZ4ZMTBEWF0TW7OLNc7fHgNwrw07g1mqe/fBN2x2Bfp+9/H8AvC2vs4 /Eug== X-Gm-Message-State: AOJu0YyMMMC/LPY4zUMF3dM2BnE4QiveywMUjNmVMa3+uL2uRlqhJ4J7 bBKYXfWFNMpSi5+LSs4vRDI= X-Google-Smtp-Source: AGHT+IEU2RDraFBJc6qDPjAtxyB8TM/J9th2188B/Ka+jtKiJoYdn5X2FiGxGAN0QESfZiH4GRsi6Q== X-Received: by 2002:a05:6602:2288:b0:794:efb0:83d6 with SMTP id d8-20020a056602228800b00794efb083d6mr11600507iod.12.1696214444931; Sun, 01 Oct 2023 19:40:44 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id v6-20020a056602058600b0079f9f7a2565sm6346113iox.38.2023.10.01.19.40.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Oct 2023 19:40:44 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , Eric Sunshine , "Eric W. Biederman" Subject: [PATCH v2 06/30] loose: compatibilty short name support Date: Sun, 1 Oct 2023 21:40:10 -0500 Message-Id: <20231002024034.2611-6-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> References: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "Eric W. Biederman" Update loose_objects_cache when udpating the loose objects map. This oidtree is used to discover which oids are possibilities when resolving short names, and it can support a mixture of sha1 and sha256 oids. With this any oid recorded objects/loose-objects-idx is usable for resolving an oid to an object. To make this maintainable a helper insert_loose_map is factored out of load_one_loose_object_map and repo_add_loose_object_map, and then modified to also update the loose_objects_cache. Signed-off-by: "Eric W. Biederman" --- loose.c | 37 +++++++++++++++++++++++++------------ 1 file changed, 25 insertions(+), 12 deletions(-) diff --git a/loose.c b/loose.c index 6ba73cc84dca..f6faa6216a08 100644 --- a/loose.c +++ b/loose.c @@ -7,6 +7,7 @@ #include "gettext.h" #include "loose.h" #include "lockfile.h" +#include "oidtree.h" static const char *loose_object_header = "# loose-object-idx\n"; @@ -42,6 +43,21 @@ static int insert_oid_pair(kh_oid_map_t *map, const struct object_id *key, const return 1; } +static int insert_loose_map(struct object_directory *odb, + const struct object_id *oid, + const struct object_id *compat_oid) +{ + struct loose_object_map *map = odb->loose_map; + int inserted = 0; + + inserted |= insert_oid_pair(map->to_compat, oid, compat_oid); + inserted |= insert_oid_pair(map->to_storage, compat_oid, oid); + if (inserted) + oidtree_insert(odb->loose_objects_cache, compat_oid); + + return inserted; +} + static int load_one_loose_object_map(struct repository *repo, struct object_directory *dir) { struct strbuf buf = STRBUF_INIT, path = STRBUF_INIT; @@ -49,15 +65,14 @@ static int load_one_loose_object_map(struct repository *repo, struct object_dire if (!dir->loose_map) loose_object_map_init(&dir->loose_map); + if (!dir->loose_objects_cache) { + ALLOC_ARRAY(dir->loose_objects_cache, 1); + oidtree_init(dir->loose_objects_cache); + } - insert_oid_pair(dir->loose_map->to_compat, repo->hash_algo->empty_tree, repo->compat_hash_algo->empty_tree); - insert_oid_pair(dir->loose_map->to_storage, repo->compat_hash_algo->empty_tree, repo->hash_algo->empty_tree); - - insert_oid_pair(dir->loose_map->to_compat, repo->hash_algo->empty_blob, repo->compat_hash_algo->empty_blob); - insert_oid_pair(dir->loose_map->to_storage, repo->compat_hash_algo->empty_blob, repo->hash_algo->empty_blob); - - insert_oid_pair(dir->loose_map->to_compat, repo->hash_algo->null_oid, repo->compat_hash_algo->null_oid); - insert_oid_pair(dir->loose_map->to_storage, repo->compat_hash_algo->null_oid, repo->hash_algo->null_oid); + insert_loose_map(dir, repo->hash_algo->empty_tree, repo->compat_hash_algo->empty_tree); + insert_loose_map(dir, repo->hash_algo->empty_blob, repo->compat_hash_algo->empty_blob); + insert_loose_map(dir, repo->hash_algo->null_oid, repo->compat_hash_algo->null_oid); strbuf_git_common_path(&path, repo, "objects/loose-object-idx"); fp = fopen(path.buf, "rb"); @@ -77,8 +92,7 @@ static int load_one_loose_object_map(struct repository *repo, struct object_dire parse_oid_hex_algop(p, &compat_oid, &p, repo->compat_hash_algo) || p != buf.buf + buf.len) goto err; - insert_oid_pair(dir->loose_map->to_compat, &oid, &compat_oid); - insert_oid_pair(dir->loose_map->to_storage, &compat_oid, &oid); + insert_loose_map(dir, &oid, &compat_oid); } strbuf_release(&buf); @@ -197,8 +211,7 @@ int repo_add_loose_object_map(struct repository *repo, const struct object_id *o if (!should_use_loose_object_map(repo)) return 0; - inserted |= insert_oid_pair(repo->objects->odb->loose_map->to_compat, oid, compat_oid); - inserted |= insert_oid_pair(repo->objects->odb->loose_map->to_storage, compat_oid, oid); + inserted = insert_loose_map(repo->objects->odb, oid, compat_oid); if (inserted) return write_one_object(repo, oid, compat_oid); return 0; From patchwork Mon Oct 2 02:40:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13405527 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5B19CE784A9 for ; Mon, 2 Oct 2023 02:40:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235430AbjJBCkz (ORCPT ); Sun, 1 Oct 2023 22:40:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41306 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235419AbjJBCkv (ORCPT ); Sun, 1 Oct 2023 22:40:51 -0400 Received: from mail-io1-xd2d.google.com (mail-io1-xd2d.google.com [IPv6:2607:f8b0:4864:20::d2d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 71A45DC for ; Sun, 1 Oct 2023 19:40:46 -0700 (PDT) Received: by mail-io1-xd2d.google.com with SMTP id ca18e2360f4ac-79faf4210b2so459442439f.2 for ; Sun, 01 Oct 2023 19:40:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696214446; x=1696819246; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Tme4B5GXuP7vVYZM1AIaqIm9NSjd6cAhrFk9ymPETtY=; b=QGiaFRG8rcFY9E7GWLSNy55pdOHCsBcG9squHqRGm7aRpXZw0sgAX0QNytLOjP9wOs +rhc06F2hTwk91VUAI4UuAqLqwPfY+zDElVPuPo3SbZK0iexN+lKdbCmfqUC/BjSeM6K V35EbJqKGC5ULCC/REZTAUKcxz9sbBz+/oC+FLix7+AES7hSWcmJrFXexJ0Xx0MEVCUr I7f8BXMpu+4jQRrzZqMBCorqErQWCjp+H1xm/EaZRHjqGv/dPmMHhzdN8njzreNVJTti 4g4PyDlkaTMjAqk6C1Z9iWodfYirf1/RhqsiaxcT80ws9TkCqmw4JUBrFHbV5swoX/lU qsXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696214446; x=1696819246; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Tme4B5GXuP7vVYZM1AIaqIm9NSjd6cAhrFk9ymPETtY=; b=HrMrcKpR1kd/U+5u0DB7OwjPexDLTC2UVmueYKHqzjbWqe7C/okgrr9vN/45/OkQuV 63MCKHDEx/Aj0u2TY6mM513Js/bX1UTWczLLi86V5akqaeghg34k7ykbGkL0HKpf2VCZ i/ZnMs+yqK+NJFlPqgCO7vm+gDIiAS7Thx9st2IKM2tgKXVCfmwOa8oFbAAc0VzPuVSA 2pl/WcyIHmJRNxdhNJcukDLVTXseEhZrHlbNyatBjUz/jDRX6I9Ze+aLK3Gnw4sAmHaN yVJ0/yZEXP9Qy/fRe8oUkV3FXkXYzymh8L8CaRdVd9ilkXkWU2RvA+0KQdG30/63yVni GggQ== X-Gm-Message-State: AOJu0YzrgDdlleUedXcZ+elSMjDy8TCf52Ma/uSnVjrSnsuICSfP9m/2 d8z+vqFYffXdiRlWo3iJFKk= X-Google-Smtp-Source: AGHT+IG5f8M883SaXh2uQzekVk43IQ3MZcb+CCiU0LaEBjM1XeGeyy8QhjjYSiQCkaYR0J9sDbWFVQ== X-Received: by 2002:a6b:f319:0:b0:786:cc36:360c with SMTP id m25-20020a6bf319000000b00786cc36360cmr12338025ioh.8.1696214445831; Sun, 01 Oct 2023 19:40:45 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id v6-20020a056602058600b0079f9f7a2565sm6346113iox.38.2023.10.01.19.40.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Oct 2023 19:40:45 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , Eric Sunshine , "Eric W. Biederman" Subject: [PATCH v2 07/30] object-file: update the loose object map when writing loose objects Date: Sun, 1 Oct 2023 21:40:11 -0500 Message-Id: <20231002024034.2611-7-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> References: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "Eric W. Biederman" To implement SHA1 compatibility on SHA256 repositories the loose object map needs to be updated whenver a loose object is written. Updating the loose object map this way allows git to support the old hash algorithm in constant time. The functions write_loose_object, and stream_loose_object are the only two functions that write to the loose object store. Update stream_loose_object to compute the compatibiilty hash, update the loose object, and then call repo_add_loose_object_map to update the loose object map. Update write_object_file_flags to convert the object into it's compatibility encoding, hash the compatibility encoding, write the object, and then update the loose object map. Update force_object_loose to lookup the hash of the compatibility encoding, write the loose object, and then update the loose object map. Update write_object_file_literally to convert the object into it's compatibility hash encoding, hash the compatibility enconding, write the object, and then update the loose object map, when the type string is a known type. For objects with an unknown type this results in a partially broken repository, as the objects are not mapped. The point of write_object_file_literally is to generate a partially broken repository for testing. For testing skipping writing the loose object map is much more useful than refusing to write the broken object at all. Except that the loose objects are updated before the loose object map I have not done any analysis to see how robust this scheme is in the event of failure. Signed-off-by: "Eric W. Biederman" --- object-file.c | 113 ++++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 95 insertions(+), 18 deletions(-) diff --git a/object-file.c b/object-file.c index 7dc0c4bfbba8..4e55f475b3b4 100644 --- a/object-file.c +++ b/object-file.c @@ -43,6 +43,8 @@ #include "setup.h" #include "submodule.h" #include "fsck.h" +#include "loose.h" +#include "object-file-convert.h" /* The maximum size for an object header. */ #define MAX_HEADER_LEN 32 @@ -1952,9 +1954,12 @@ static int start_loose_object_common(struct strbuf *tmp_file, const char *filename, unsigned flags, git_zstream *stream, unsigned char *buf, size_t buflen, - git_hash_ctx *c, + git_hash_ctx *c, git_hash_ctx *compat_c, char *hdr, int hdrlen) { + struct repository *repo = the_repository; + const struct git_hash_algo *algo = repo->hash_algo; + const struct git_hash_algo *compat = repo->compat_hash_algo; int fd; fd = create_tmpfile(tmp_file, filename); @@ -1974,14 +1979,18 @@ static int start_loose_object_common(struct strbuf *tmp_file, git_deflate_init(stream, zlib_compression_level); stream->next_out = buf; stream->avail_out = buflen; - the_hash_algo->init_fn(c); + algo->init_fn(c); + if (compat && compat_c) + compat->init_fn(compat_c); /* Start to feed header to zlib stream */ stream->next_in = (unsigned char *)hdr; stream->avail_in = hdrlen; while (git_deflate(stream, 0) == Z_OK) ; /* nothing */ - the_hash_algo->update_fn(c, hdr, hdrlen); + algo->update_fn(c, hdr, hdrlen); + if (compat && compat_c) + compat->update_fn(compat_c, hdr, hdrlen); return fd; } @@ -1990,16 +1999,21 @@ static int start_loose_object_common(struct strbuf *tmp_file, * Common steps for the inner git_deflate() loop for writing loose * objects. Returns what git_deflate() returns. */ -static int write_loose_object_common(git_hash_ctx *c, +static int write_loose_object_common(git_hash_ctx *c, git_hash_ctx *compat_c, git_zstream *stream, const int flush, unsigned char *in0, const int fd, unsigned char *compressed, const size_t compressed_len) { + struct repository *repo = the_repository; + const struct git_hash_algo *algo = repo->hash_algo; + const struct git_hash_algo *compat = repo->compat_hash_algo; int ret; ret = git_deflate(stream, flush ? Z_FINISH : 0); - the_hash_algo->update_fn(c, in0, stream->next_in - in0); + algo->update_fn(c, in0, stream->next_in - in0); + if (compat && compat_c) + compat->update_fn(compat_c, in0, stream->next_in - in0); if (write_in_full(fd, compressed, stream->next_out - compressed) < 0) die_errno(_("unable to write loose object file")); stream->next_out = compressed; @@ -2014,15 +2028,21 @@ static int write_loose_object_common(git_hash_ctx *c, * - End the compression of zlib stream. * - Get the calculated oid to "oid". */ -static int end_loose_object_common(git_hash_ctx *c, git_zstream *stream, - struct object_id *oid) +static int end_loose_object_common(git_hash_ctx *c, git_hash_ctx *compat_c, + git_zstream *stream, struct object_id *oid, + struct object_id *compat_oid) { + struct repository *repo = the_repository; + const struct git_hash_algo *algo = repo->hash_algo; + const struct git_hash_algo *compat = repo->compat_hash_algo; int ret; ret = git_deflate_end_gently(stream); if (ret != Z_OK) return ret; - the_hash_algo->final_oid_fn(oid, c); + algo->final_oid_fn(oid, c); + if (compat && compat_c) + compat->final_oid_fn(compat_oid, compat_c); return Z_OK; } @@ -2046,7 +2066,7 @@ static int write_loose_object(const struct object_id *oid, char *hdr, fd = start_loose_object_common(&tmp_file, filename.buf, flags, &stream, compressed, sizeof(compressed), - &c, hdr, hdrlen); + &c, NULL, hdr, hdrlen); if (fd < 0) return -1; @@ -2056,14 +2076,14 @@ static int write_loose_object(const struct object_id *oid, char *hdr, do { unsigned char *in0 = stream.next_in; - ret = write_loose_object_common(&c, &stream, 1, in0, fd, + ret = write_loose_object_common(&c, NULL, &stream, 1, in0, fd, compressed, sizeof(compressed)); } while (ret == Z_OK); if (ret != Z_STREAM_END) die(_("unable to deflate new object %s (%d)"), oid_to_hex(oid), ret); - ret = end_loose_object_common(&c, &stream, ¶no_oid); + ret = end_loose_object_common(&c, NULL, &stream, ¶no_oid, NULL); if (ret != Z_OK) die(_("deflateEnd on object %s failed (%d)"), oid_to_hex(oid), ret); @@ -2108,10 +2128,12 @@ static int freshen_packed_object(const struct object_id *oid) int stream_loose_object(struct input_stream *in_stream, size_t len, struct object_id *oid) { + const struct git_hash_algo *compat = the_repository->compat_hash_algo; + struct object_id compat_oid; int fd, ret, err = 0, flush = 0; unsigned char compressed[4096]; git_zstream stream; - git_hash_ctx c; + git_hash_ctx c, compat_c; struct strbuf tmp_file = STRBUF_INIT; struct strbuf filename = STRBUF_INIT; int dirlen; @@ -2135,7 +2157,7 @@ int stream_loose_object(struct input_stream *in_stream, size_t len, */ fd = start_loose_object_common(&tmp_file, filename.buf, 0, &stream, compressed, sizeof(compressed), - &c, hdr, hdrlen); + &c, &compat_c, hdr, hdrlen); if (fd < 0) { err = -1; goto cleanup; @@ -2153,7 +2175,7 @@ int stream_loose_object(struct input_stream *in_stream, size_t len, if (in_stream->is_finished) flush = 1; } - ret = write_loose_object_common(&c, &stream, flush, in0, fd, + ret = write_loose_object_common(&c, &compat_c, &stream, flush, in0, fd, compressed, sizeof(compressed)); /* * Unlike write_loose_object(), we do not have the entire @@ -2176,7 +2198,7 @@ int stream_loose_object(struct input_stream *in_stream, size_t len, */ if (ret != Z_STREAM_END) die(_("unable to stream deflate new object (%d)"), ret); - ret = end_loose_object_common(&c, &stream, oid); + ret = end_loose_object_common(&c, &compat_c, &stream, oid, &compat_oid); if (ret != Z_OK) die(_("deflateEnd on stream object failed (%d)"), ret); close_loose_object(fd, tmp_file.buf); @@ -2203,6 +2225,8 @@ int stream_loose_object(struct input_stream *in_stream, size_t len, } err = finalize_object_file(tmp_file.buf, filename.buf); + if (!err && compat) + err = repo_add_loose_object_map(the_repository, oid, &compat_oid); cleanup: strbuf_release(&tmp_file); strbuf_release(&filename); @@ -2213,17 +2237,38 @@ int write_object_file_flags(const void *buf, unsigned long len, enum object_type type, struct object_id *oid, unsigned flags) { + struct repository *repo = the_repository; + const struct git_hash_algo *algo = repo->hash_algo; + const struct git_hash_algo *compat = repo->compat_hash_algo; + struct object_id compat_oid; char hdr[MAX_HEADER_LEN]; int hdrlen = sizeof(hdr); + /* Generate compat_oid */ + if (compat) { + if (type == OBJ_BLOB) + hash_object_file(compat, buf, len, type, &compat_oid); + else { + struct strbuf converted = STRBUF_INIT; + convert_object_file(&converted, algo, compat, + buf, len, type, 0); + hash_object_file(compat, converted.buf, converted.len, + type, &compat_oid); + strbuf_release(&converted); + } + } + /* Normally if we have it in the pack then we do not bother writing * it out into .git/objects/??/?{38} file. */ - write_object_file_prepare(the_hash_algo, buf, len, type, oid, hdr, - &hdrlen); + write_object_file_prepare(algo, buf, len, type, oid, hdr, &hdrlen); if (freshen_packed_object(oid) || freshen_loose_object(oid)) return 0; - return write_loose_object(oid, hdr, hdrlen, buf, len, 0, flags); + if (write_loose_object(oid, hdr, hdrlen, buf, len, 0, flags)) + return -1; + if (compat) + return repo_add_loose_object_map(repo, oid, &compat_oid); + return 0; } int write_object_file_literally(const void *buf, unsigned long len, @@ -2231,7 +2276,27 @@ int write_object_file_literally(const void *buf, unsigned long len, unsigned flags) { char *header; + struct repository *repo = the_repository; + const struct git_hash_algo *algo = repo->hash_algo; + const struct git_hash_algo *compat = repo->compat_hash_algo; + struct object_id compat_oid; int hdrlen, status = 0; + int compat_type = -1; + + if (compat) { + compat_type = type_from_string_gently(type, -1, 1); + if (compat_type == OBJ_BLOB) + hash_object_file(compat, buf, len, compat_type, + &compat_oid); + else if (compat_type != -1) { + struct strbuf converted = STRBUF_INIT; + convert_object_file(&converted, algo, compat, + buf, len, compat_type, 0); + hash_object_file(compat, converted.buf, converted.len, + compat_type, &compat_oid); + strbuf_release(&converted); + } + } /* type string, SP, %lu of the length plus NUL must fit this */ hdrlen = strlen(type) + MAX_HEADER_LEN; @@ -2244,6 +2309,8 @@ int write_object_file_literally(const void *buf, unsigned long len, if (freshen_packed_object(oid) || freshen_loose_object(oid)) goto cleanup; status = write_loose_object(oid, header, hdrlen, buf, len, 0, 0); + if (compat_type != -1) + return repo_add_loose_object_map(repo, oid, &compat_oid); cleanup: free(header); @@ -2252,9 +2319,12 @@ int write_object_file_literally(const void *buf, unsigned long len, int force_object_loose(const struct object_id *oid, time_t mtime) { + struct repository *repo = the_repository; + const struct git_hash_algo *compat = repo->compat_hash_algo; void *buf; unsigned long len; struct object_info oi = OBJECT_INFO_INIT; + struct object_id compat_oid; enum object_type type; char hdr[MAX_HEADER_LEN]; int hdrlen; @@ -2267,8 +2337,15 @@ int force_object_loose(const struct object_id *oid, time_t mtime) oi.contentp = &buf; if (oid_object_info_extended(the_repository, oid, &oi, 0)) return error(_("cannot read object for %s"), oid_to_hex(oid)); + if (compat) { + if (repo_oid_to_algop(repo, oid, compat, &compat_oid)) + return error(_("cannot map object %s to %s"), + oid_to_hex(oid), compat->name); + } hdrlen = format_object_header(hdr, sizeof(hdr), type, len); ret = write_loose_object(oid, hdr, hdrlen, buf, len, mtime, 0); + if (!ret && compat) + ret = repo_add_loose_object_map(the_repository, oid, &compat_oid); free(buf); return ret; From patchwork Mon Oct 2 02:40:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13405528 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0687DE784A6 for ; Mon, 2 Oct 2023 02:40:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235432AbjJBCk5 (ORCPT ); Sun, 1 Oct 2023 22:40:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41302 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235418AbjJBCkv (ORCPT ); Sun, 1 Oct 2023 22:40:51 -0400 Received: from mail-io1-xd36.google.com (mail-io1-xd36.google.com [IPv6:2607:f8b0:4864:20::d36]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C5B25E3 for ; Sun, 1 Oct 2023 19:40:47 -0700 (PDT) Received: by mail-io1-xd36.google.com with SMTP id ca18e2360f4ac-79fa416b7ffso519952239f.2 for ; Sun, 01 Oct 2023 19:40:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696214447; x=1696819247; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Dt5KEv9KVtgQEbQQg9gC/QBXXLffLz2k9RehcgzxHR8=; b=XkUlqTYJTwypwybwm2GSJ895fYSCifPLCrRUK0/6/n3yAVtDtSqMZq5PU6UVIHuPpu xw0Fu0YrWNlpX3/JxEAHTLUa+ibnSM8RFUAAc++96b9l4/S+jl3GT9+rfUQX1Oj6IAQI HfJ2oyAdCGyKnqgvNvtBEP2HbRnso7121dCU9J87t50GCncarkyFrvcb+9Rkll4Zc/Pj z0EFtFkOKa6m0ulLJpXIpvnkq8qUv1jtP4qUhM8M4lP7kVYYlIciWHqAFVddx52VRg3z mDo7zlvoQBtucvlWZFIxyr3z1Pzp/PGG1tzYXKsNBZ+D+vXsW1gmH1B1xfbH+LwBe5Zp 2m2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696214447; x=1696819247; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Dt5KEv9KVtgQEbQQg9gC/QBXXLffLz2k9RehcgzxHR8=; b=Mdu3/nYhdc1pAChpxDMYfSrdzPWbbiCgvRzuOzm+3TfXsuxa/eQt8asA089+WIXyUI Z/6EMWlTylkdV+TZTGtrnbmA6lQOJgAzcx6OXg6lE2pGSc+zYigvpCk/GRyVQ21R5mF/ nJnoIbtiLfoa/yjIx3Azgtz2iE8WAtg/dSmWtOpvhfrY6+iAzEK2fRGWiE4HinbFki2E +LNmN3bBjrffDXlSSA2qqeHVOetXXRDa/oegdHK46dRcttwUlr8ciEi641ZoRZWh/Pht Ui55loy1Y9/h9lC5rpD+MhGVQgdtH4FbB53WqByCuoIt5sy9/eNev4jp0RgBmMqgfR4g ViDw== X-Gm-Message-State: AOJu0Yx5ZUkDVDnKlBfQoDoKFaWLAqHAtfRS9A+mN0dHOJjezoHgQnGj 3SSHcE68c94qM8uLO8Etbo0= X-Google-Smtp-Source: AGHT+IEAUz9L+2YaVdcjE8wC3JlKClYb+PmL3chWUcqlFuZdNWE6lcdbpEA1TfipbxFYdAxVpkcdcw== X-Received: by 2002:a6b:6618:0:b0:790:a010:4c42 with SMTP id a24-20020a6b6618000000b00790a0104c42mr10485046ioc.13.1696214446716; Sun, 01 Oct 2023 19:40:46 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id v6-20020a056602058600b0079f9f7a2565sm6346113iox.38.2023.10.01.19.40.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Oct 2023 19:40:46 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , Eric Sunshine , "Eric W. Biederman" Subject: [PATCH v2 08/30] object-file: add a compat_oid_in parameter to write_object_file_flags Date: Sun, 1 Oct 2023 21:40:12 -0500 Message-Id: <20231002024034.2611-8-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> References: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "Eric W. Biederman" To create the proper signatures for commit objects both versions of the commit object need to be generated and signed. After that it is a waste to throw away the work of generating the compatibility hash so update write_object_file_flags to take a compatibility hash input parameter that it can use to skip the work of generating the compatability hash. Update the places that don't generate the compatability hash to pass NULL so it is easy to tell write_object_file_flags should not attempt to use their compatability hash. Signed-off-by: "Eric W. Biederman" --- cache-tree.c | 2 +- object-file.c | 6 ++++-- object-store-ll.h | 4 ++-- 3 files changed, 7 insertions(+), 5 deletions(-) diff --git a/cache-tree.c b/cache-tree.c index 641427ed410a..ddc7d3d86959 100644 --- a/cache-tree.c +++ b/cache-tree.c @@ -448,7 +448,7 @@ static int update_one(struct cache_tree *it, hash_object_file(the_hash_algo, buffer.buf, buffer.len, OBJ_TREE, &it->oid); } else if (write_object_file_flags(buffer.buf, buffer.len, OBJ_TREE, - &it->oid, flags & WRITE_TREE_SILENT + &it->oid, NULL, flags & WRITE_TREE_SILENT ? HASH_SILENT : 0)) { strbuf_release(&buffer); return -1; diff --git a/object-file.c b/object-file.c index 4e55f475b3b4..820810a5f4b3 100644 --- a/object-file.c +++ b/object-file.c @@ -2235,7 +2235,7 @@ int stream_loose_object(struct input_stream *in_stream, size_t len, int write_object_file_flags(const void *buf, unsigned long len, enum object_type type, struct object_id *oid, - unsigned flags) + struct object_id *compat_oid_in, unsigned flags) { struct repository *repo = the_repository; const struct git_hash_algo *algo = repo->hash_algo; @@ -2246,7 +2246,9 @@ int write_object_file_flags(const void *buf, unsigned long len, /* Generate compat_oid */ if (compat) { - if (type == OBJ_BLOB) + if (compat_oid_in) + oidcpy(&compat_oid, compat_oid_in); + else if (type == OBJ_BLOB) hash_object_file(compat, buf, len, type, &compat_oid); else { struct strbuf converted = STRBUF_INIT; diff --git a/object-store-ll.h b/object-store-ll.h index bc76d6bec80d..c5f2bb2fc2fe 100644 --- a/object-store-ll.h +++ b/object-store-ll.h @@ -255,11 +255,11 @@ void hash_object_file(const struct git_hash_algo *algo, const void *buf, int write_object_file_flags(const void *buf, unsigned long len, enum object_type type, struct object_id *oid, - unsigned flags); + struct object_id *comapt_oid_in, unsigned flags); static inline int write_object_file(const void *buf, unsigned long len, enum object_type type, struct object_id *oid) { - return write_object_file_flags(buf, len, type, oid, 0); + return write_object_file_flags(buf, len, type, oid, NULL, 0); } int write_object_file_literally(const void *buf, unsigned long len, From patchwork Mon Oct 2 02:40:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13405530 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8DAF3E784A7 for ; Mon, 2 Oct 2023 02:41:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235451AbjJBClB (ORCPT ); Sun, 1 Oct 2023 22:41:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41364 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235377AbjJBCkw (ORCPT ); Sun, 1 Oct 2023 22:40:52 -0400 Received: from mail-io1-xd31.google.com (mail-io1-xd31.google.com [IPv6:2607:f8b0:4864:20::d31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 65B0BEA for ; Sun, 1 Oct 2023 19:40:48 -0700 (PDT) Received: by mail-io1-xd31.google.com with SMTP id ca18e2360f4ac-79f82b26abfso562690339f.1 for ; Sun, 01 Oct 2023 19:40:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696214447; x=1696819247; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=v2VdQLCgD1g/wvu2D1A7qPZvOJZIRNkq1Ece2NrTAfo=; b=dN0K5QOwbV9/DJJiYGdZFKiIvxKc/aidx9Leva7VS3vkyvYXLSM6bcZbe6XS57ErHO mxPfvMzDy74ei35f8zG96j4y+pC9rLGYr01b8bbjoxJbdCNlEDDGxoD6he74a+shRj7T M/MhFi53k7M4jiQeo6WB35Sid3Xni7zUYw9vSnvWbD3QagF1Lq4NSiBOY53XdAW252gm 3MXXUT0NqHJgNe2sXY5AaU7WyQtTPQFPa6LY1wsr1eP+0LZ3kxrQsziqV0Hf1JPS5oQv wlle8y72SRpsxhhTtKS3gwg/WLBfkFHO240yY1GCUft0AUIJ5r9Ypk2h1T6fk4ViY1Qn 4jrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696214447; x=1696819247; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=v2VdQLCgD1g/wvu2D1A7qPZvOJZIRNkq1Ece2NrTAfo=; b=K9zYdacgdFGGcHnxiPQBL7XmnKdYh1x4eJeT7tkggj6CDEZCmFhBwzxiPXD0Iy0+kB 13lZiEZAHl7zx+4AvB0D2KU4TSR1h51kbHLz1jYgcFrkLD5G2sn8MVAnxx816WN4BFOy mTMLdVA5J5ahBVLcWkxnelNaghtKBlAAJljEPkT3bhTFKZ8HTay3FlzA4YsnF1NnUhGT eGlDWZo4aAJT0McwwbX4YufAxq3LbHBh/Fhhy5u1hZZHja4nNYQBQO54sbr4yWPm7Q3v B8YaW2rCygT6z8XRumMo32StIlc9DleGClfDe4oJTggAb6u7Rvr0fjJAnNfMfHrxFiSE XRyw== X-Gm-Message-State: AOJu0YwJHxmWuUUa7msdH0rRHMgfvqUH6gN8JnOJa+fIsPbzFDSg9Omd JTDcYQlzty61xF7/3qpTbPMsvvlRYGBwpg== X-Google-Smtp-Source: AGHT+IEBv/rcTUED8hHzml1oxYGtqJIHj9rgEpZKf+ohDzkQ0+8Fmlx4t+i+wBxixSODTxRtYEzhoA== X-Received: by 2002:a6b:651a:0:b0:798:26fb:ff2 with SMTP id z26-20020a6b651a000000b0079826fb0ff2mr12088341iob.10.1696214447603; Sun, 01 Oct 2023 19:40:47 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id v6-20020a056602058600b0079f9f7a2565sm6346113iox.38.2023.10.01.19.40.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Oct 2023 19:40:47 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , Eric Sunshine , "Eric W. Biederman" Subject: [PATCH v2 09/30] commit: write commits for both hashes Date: Sun, 1 Oct 2023 21:40:13 -0500 Message-Id: <20231002024034.2611-9-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> References: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "brian m. carlson" When we write a commit, we include data that is specific to the hash algorithm, such as parents and the root tree. In order to write both a SHA-1 commit and a SHA-256 version, we need to convert between them. However, a straightforward conversion isn't necessarily what we want. When we sign a commit, we sign its data, so if we create a commit for SHA-256 and then write a SHA-1 version, we'll still have only signed the SHA-256 data. While this is valid, it would be better to sign both forms of data so people using SHA-1 can verify the signatures as well. Consequently, we don't want to use the standard mapping that occurs when we write an object. Instead, let's move most of the writing of the commit into a separate function which is agnostic of the hash algorithm and which simply writes into a buffer and specify both versions of the object ourselves. We can then call this function twice: once with the SHA-256 contents, and if SHA-1 is enabled, once with the SHA-1 contents. If we're signing the commit, we then sign both versions and append both signatures to both buffers. To produce a consistent hash, we always append the signatures in the order in which Git implemented them: first SHA-1, then SHA-256. In order to make this signing code work, we split the commit signing code into two functions, one which signs the buffer, and one which appends the signature. Signed-off-by: brian m. carlson Signed-off-by: Eric W. Biederman --- commit.c | 181 +++++++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 136 insertions(+), 45 deletions(-) diff --git a/commit.c b/commit.c index b3223478bc2a..6765f3a82b9d 100644 --- a/commit.c +++ b/commit.c @@ -28,6 +28,7 @@ #include "shallow.h" #include "tree.h" #include "hook.h" +#include "object-file-convert.h" static struct commit_extra_header *read_commit_extra_header_lines(const char *buf, size_t len, const char **); @@ -1100,12 +1101,11 @@ static const char *gpg_sig_headers[] = { "gpgsig-sha256", }; -int sign_with_header(struct strbuf *buf, const char *keyid) +static int add_commit_signature(struct strbuf *buf, struct strbuf *sig, const struct git_hash_algo *algo) { - struct strbuf sig = STRBUF_INIT; int inspos, copypos; const char *eoh; - const char *gpg_sig_header = gpg_sig_headers[hash_algo_by_ptr(the_hash_algo)]; + const char *gpg_sig_header = gpg_sig_headers[hash_algo_by_ptr(algo)]; int gpg_sig_header_len = strlen(gpg_sig_header); /* find the end of the header */ @@ -1115,15 +1115,8 @@ int sign_with_header(struct strbuf *buf, const char *keyid) else inspos = eoh - buf->buf + 1; - if (!keyid || !*keyid) - keyid = get_signing_key(); - if (sign_buffer(buf, &sig, keyid)) { - strbuf_release(&sig); - return -1; - } - - for (copypos = 0; sig.buf[copypos]; ) { - const char *bol = sig.buf + copypos; + for (copypos = 0; sig->buf[copypos]; ) { + const char *bol = sig->buf + copypos; const char *eol = strchrnul(bol, '\n'); int len = (eol - bol) + !!*eol; @@ -1136,11 +1129,17 @@ int sign_with_header(struct strbuf *buf, const char *keyid) inspos += len; copypos += len; } - strbuf_release(&sig); return 0; } - +static int sign_commit_to_strbuf(struct strbuf *sig, struct strbuf *buf, const char *keyid) +{ + if (!keyid || !*keyid) + keyid = get_signing_key(); + if (sign_buffer(buf, sig, keyid)) + return -1; + return 0; +} int parse_signed_commit(const struct commit *commit, struct strbuf *payload, struct strbuf *signature, @@ -1599,70 +1598,162 @@ N_("Warning: commit message did not conform to UTF-8.\n" "You may want to amend it after fixing the message, or set the config\n" "variable i18n.commitEncoding to the encoding your project uses.\n"); -int commit_tree_extended(const char *msg, size_t msg_len, - const struct object_id *tree, - struct commit_list *parents, struct object_id *ret, - const char *author, const char *committer, - const char *sign_commit, - struct commit_extra_header *extra) +static void write_commit_tree(struct strbuf *buffer, const char *msg, size_t msg_len, + const struct object_id *tree, + const struct object_id *parents, size_t parents_len, + const char *author, const char *committer, + struct commit_extra_header *extra) { - int result; int encoding_is_utf8; - struct strbuf buffer; - - assert_oid_type(tree, OBJ_TREE); - - if (memchr(msg, '\0', msg_len)) - return error("a NUL byte in commit log message not allowed."); + size_t i; /* Not having i18n.commitencoding is the same as having utf-8 */ encoding_is_utf8 = is_encoding_utf8(git_commit_encoding); - strbuf_init(&buffer, 8192); /* should avoid reallocs for the headers */ - strbuf_addf(&buffer, "tree %s\n", oid_to_hex(tree)); + strbuf_grow(buffer, 8192); /* should avoid reallocs for the headers */ + strbuf_addf(buffer, "tree %s\n", oid_to_hex(tree)); /* * NOTE! This ordering means that the same exact tree merged with a * different order of parents will be a _different_ changeset even * if everything else stays the same. */ - while (parents) { - struct commit *parent = pop_commit(&parents); - strbuf_addf(&buffer, "parent %s\n", - oid_to_hex(&parent->object.oid)); - } + for (i = 0; i < parents_len; i++) + strbuf_addf(buffer, "parent %s\n", oid_to_hex(&parents[i])); /* Person/date information */ if (!author) author = git_author_info(IDENT_STRICT); - strbuf_addf(&buffer, "author %s\n", author); + strbuf_addf(buffer, "author %s\n", author); if (!committer) committer = git_committer_info(IDENT_STRICT); - strbuf_addf(&buffer, "committer %s\n", committer); + strbuf_addf(buffer, "committer %s\n", committer); if (!encoding_is_utf8) - strbuf_addf(&buffer, "encoding %s\n", git_commit_encoding); + strbuf_addf(buffer, "encoding %s\n", git_commit_encoding); while (extra) { - add_extra_header(&buffer, extra); + add_extra_header(buffer, extra); extra = extra->next; } - strbuf_addch(&buffer, '\n'); + strbuf_addch(buffer, '\n'); /* And add the comment */ - strbuf_add(&buffer, msg, msg_len); + strbuf_add(buffer, msg, msg_len); +} - /* And check the encoding */ - if (encoding_is_utf8 && !verify_utf8(&buffer)) - fprintf(stderr, _(commit_utf8_warn)); +int commit_tree_extended(const char *msg, size_t msg_len, + const struct object_id *tree, + struct commit_list *parents, struct object_id *ret, + const char *author, const char *committer, + const char *sign_commit, + struct commit_extra_header *extra) +{ + struct repository *r = the_repository; + int result = 0; + int encoding_is_utf8; + struct strbuf buffer = STRBUF_INIT, compat_buffer = STRBUF_INIT; + struct strbuf sig = STRBUF_INIT, compat_sig = STRBUF_INIT; + struct object_id *parent_buf = NULL, *compat_oid = NULL; + struct object_id compat_oid_buf; + size_t i, nparents; + + /* Not having i18n.commitencoding is the same as having utf-8 */ + encoding_is_utf8 = is_encoding_utf8(git_commit_encoding); + + assert_oid_type(tree, OBJ_TREE); + + if (memchr(msg, '\0', msg_len)) + return error("a NUL byte in commit log message not allowed."); - if (sign_commit && sign_with_header(&buffer, sign_commit)) { + nparents = commit_list_count(parents); + CALLOC_ARRAY(parent_buf, nparents); + i = 0; + while (parents) { + struct commit *parent = pop_commit(&parents); + oidcpy(&parent_buf[i++], &parent->object.oid); + } + + write_commit_tree(&buffer, msg, msg_len, tree, parent_buf, nparents, author, committer, extra); + if (sign_commit && sign_commit_to_strbuf(&sig, &buffer, sign_commit)) { result = -1; goto out; } + if (r->compat_hash_algo) { + struct object_id mapped_tree; + struct object_id *mapped_parents; + + CALLOC_ARRAY(mapped_parents, nparents); + + if (repo_oid_to_algop(r, tree, r->compat_hash_algo, &mapped_tree)) { + result = -1; + free(mapped_parents); + goto out; + } + for (i = 0; i < nparents; i++) + if (repo_oid_to_algop(r, &parent_buf[i], r->compat_hash_algo, &mapped_parents[i])) { + result = -1; + free(mapped_parents); + goto out; + } + write_commit_tree(&compat_buffer, msg, msg_len, &mapped_tree, + mapped_parents, nparents, author, committer, extra); + free(mapped_parents); + + if (sign_commit && sign_commit_to_strbuf(&compat_sig, &compat_buffer, sign_commit)) { + result = -1; + goto out; + } + } + + if (sign_commit) { + struct sig_pairs { + struct strbuf *sig; + const struct git_hash_algo *algo; + } bufs [2] = { + { &compat_sig, r->compat_hash_algo }, + { &sig, r->hash_algo }, + }; + int i; + + /* + * We write algorithms in the order they were implemented in + * Git to produce a stable hash when multiple algorithms are + * used. + */ + if (r->compat_hash_algo && hash_algo_by_ptr(bufs[0].algo) > hash_algo_by_ptr(bufs[1].algo)) + SWAP(bufs[0], bufs[1]); + + /* + * We traverse each algorithm in order, and apply the signature + * to each buffer. + */ + for (i = 0; i < ARRAY_SIZE(bufs); i++) { + if (!bufs[i].algo) + continue; + add_commit_signature(&buffer, bufs[i].sig, bufs[i].algo); + if (r->compat_hash_algo) + add_commit_signature(&compat_buffer, bufs[i].sig, bufs[i].algo); + } + } - result = write_object_file(buffer.buf, buffer.len, OBJ_COMMIT, ret); + /* And check the encoding. */ + if (encoding_is_utf8 && (!verify_utf8(&buffer) || !verify_utf8(&compat_buffer))) + fprintf(stderr, _(commit_utf8_warn)); + + if (r->compat_hash_algo) { + hash_object_file(r->compat_hash_algo, compat_buffer.buf, compat_buffer.len, + OBJ_COMMIT, &compat_oid_buf); + compat_oid = &compat_oid_buf; + } + + result = write_object_file_flags(buffer.buf, buffer.len, OBJ_COMMIT, + ret, compat_oid, 0); out: + free(parent_buf); strbuf_release(&buffer); + strbuf_release(&compat_buffer); + strbuf_release(&sig); + strbuf_release(&compat_sig); return result; } From patchwork Mon Oct 2 02:40:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13405529 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 72B06E784A8 for ; Mon, 2 Oct 2023 02:40:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235438AbjJBCk7 (ORCPT ); Sun, 1 Oct 2023 22:40:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41336 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235394AbjJBCkv (ORCPT ); Sun, 1 Oct 2023 22:40:51 -0400 Received: from mail-io1-xd31.google.com (mail-io1-xd31.google.com [IPv6:2607:f8b0:4864:20::d31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 165F8C9 for ; Sun, 1 Oct 2023 19:40:49 -0700 (PDT) Received: by mail-io1-xd31.google.com with SMTP id ca18e2360f4ac-77acb04309dso577447939f.2 for ; Sun, 01 Oct 2023 19:40:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696214448; x=1696819248; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=xfjtdZNXP497JCb5qMkab/9O6Ztt2eBtpACMqwRx5iQ=; b=CdCYAb5L/xGYa1ZnfdEvTavafRAMNT+1WvOIlVnIvg8iiP0LG8oJFYydwVuVTzRaQL xNKUmQPkKnrHgua7OwFzE5qdxKTKezOY5I6LjhVDtR3q3MJSYCZ1w/T6jXluX8dyteMi 20ulhm0DiDowxxdue5qvqQru4mCf5LlQ33b8p0jp/3xjtjuIdSmymhzfGWlEhH8M6OOo SVR5Rpt8XEhOA+4/5Gplxok4rrXlm0Fdd9nSyUr9VverJTrPJq+vSKEjf3GStqS6xmLP wvpWNrKuoFNL7DX7SywmC1YSsRDpnmvF4x4P+do5z4PL/7ESlbCTW6ljbaRJtG+kILUu nFEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696214448; x=1696819248; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xfjtdZNXP497JCb5qMkab/9O6Ztt2eBtpACMqwRx5iQ=; b=fD695Xkk44q4xCzBe6hFZsl9MfkTLW1yjXmLJVIhXt+wWq5s1SIM9393usHLYObP/J 6XVYPFVwO6bYmfvFJyxZnOcDmgwHCPVDXElAqfUaK3nM8eriTM/BV/sNRNeQvkTpw8Y6 IqDYsQ7PIe0xCAOrcd3bl5ON5InjKzHT9kUJt11Ch6nvauzy5sRnZLVUr0w6YDZ1ERBL SdrZE9oowyCEehsWZb7cbQ5oV3L1GnAqvRkDvlASZdFxr0U8GDdQD/WRPzaZAbOubTuN 8qxIEAlaP24UXp3MkwPRBXcdZy55CwoOR1ojiZQavjoGx0jkG6XTOsI9EXbbaMm51DYE P3wg== X-Gm-Message-State: AOJu0YyJxOPsr7Pl83TwF5UR/eA5sh/qsP+vcBpd13dgLKLazEabWlqG XM/2cYaMdbnTuQfc0yNrRhg= X-Google-Smtp-Source: AGHT+IEHxOF8b6LjGyAo7DZGKmTz/fE1dDy3tyUzJaSLnGTQwjT+j3ihlTYsW0/v2QUkHKC/fYNUWg== X-Received: by 2002:a5e:c019:0:b0:79f:9edb:6e2d with SMTP id u25-20020a5ec019000000b0079f9edb6e2dmr11353695iol.18.1696214448406; Sun, 01 Oct 2023 19:40:48 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id v6-20020a056602058600b0079f9f7a2565sm6346113iox.38.2023.10.01.19.40.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Oct 2023 19:40:48 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , Eric Sunshine , "Eric W. Biederman" Subject: [PATCH v2 10/30] commit: convert mergetag before computing the signature of a commit Date: Sun, 1 Oct 2023 21:40:14 -0500 Message-Id: <20231002024034.2611-10-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> References: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "Eric W. Biederman" It so happens that commit mergetag lines embed a tag object. So to compute the compatible signature of a commit object that has mergetag lines the compatible embedded tag must be computed first. Implement this by duplicating and converting the commit extra headers into the compatible version of the commit extra headers, that need to be passed to commit_tree_extended. To handle merge tags only the compatible extra headers need to be computed. Signed-off-by: Eric W. Biederman --- commit.c | 42 +++++++++++++++++++++++++++++++++++++++++- 1 file changed, 41 insertions(+), 1 deletion(-) diff --git a/commit.c b/commit.c index 6765f3a82b9d..913e015966b4 100644 --- a/commit.c +++ b/commit.c @@ -1355,6 +1355,39 @@ void append_merge_tag_headers(struct commit_list *parents, } } +static int convert_commit_extra_headers(struct commit_extra_header *orig, + struct commit_extra_header **result) +{ + const struct git_hash_algo *compat = the_repository->compat_hash_algo; + const struct git_hash_algo *algo = the_repository->hash_algo; + struct commit_extra_header *extra = NULL, **tail = &extra; + struct strbuf out = STRBUF_INIT; + while (orig) { + struct commit_extra_header *new; + CALLOC_ARRAY(new, 1); + if (!strcmp(orig->key, "mergetag")) { + if (convert_object_file(&out, algo, compat, + orig->value, orig->len, + OBJ_TAG, 1)) { + free(new); + free_commit_extra_headers(extra); + return -1; + } + new->key = xstrdup("mergetag"); + new->value = strbuf_detach(&out, &new->len); + } else { + new->key = xstrdup(orig->key); + new->len = orig->len; + new->value = xmemdupz(orig->value, orig->len); + } + *tail = new; + tail = &new->next; + orig = orig->next; + } + *result = extra; + return 0; +} + static void add_extra_header(struct strbuf *buffer, struct commit_extra_header *extra) { @@ -1679,6 +1712,7 @@ int commit_tree_extended(const char *msg, size_t msg_len, goto out; } if (r->compat_hash_algo) { + struct commit_extra_header *compat_extra = NULL; struct object_id mapped_tree; struct object_id *mapped_parents; @@ -1695,8 +1729,14 @@ int commit_tree_extended(const char *msg, size_t msg_len, free(mapped_parents); goto out; } + if (convert_commit_extra_headers(extra, &compat_extra)) { + result = -1; + free(mapped_parents); + goto out; + } write_commit_tree(&compat_buffer, msg, msg_len, &mapped_tree, - mapped_parents, nparents, author, committer, extra); + mapped_parents, nparents, author, committer, compat_extra); + free_commit_extra_headers(compat_extra); free(mapped_parents); if (sign_commit && sign_commit_to_strbuf(&compat_sig, &compat_buffer, sign_commit)) { From patchwork Mon Oct 2 02:40:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13405531 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0AB94E784A6 for ; Mon, 2 Oct 2023 02:41:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235394AbjJBClD (ORCPT ); Sun, 1 Oct 2023 22:41:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41370 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235414AbjJBCkw (ORCPT ); Sun, 1 Oct 2023 22:40:52 -0400 Received: from mail-io1-xd2b.google.com (mail-io1-xd2b.google.com [IPv6:2607:f8b0:4864:20::d2b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 05C24CE for ; Sun, 1 Oct 2023 19:40:50 -0700 (PDT) Received: by mail-io1-xd2b.google.com with SMTP id ca18e2360f4ac-79fe6da0095so276789039f.0 for ; Sun, 01 Oct 2023 19:40:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696214449; x=1696819249; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6KAF/A/gf2AQ43Th7UFJ8qLbpspkX1qh5IF2KzZ6tTo=; b=Y/A+DBq35rgRvGdH6fwiJGyqaSU7OWwdulh7UCppQAcVY19jxTb9+jnjVzO7hzHhK5 wSo07Uq6NgVCziryEwEk9QA3AVfQq4IEm7ELnK2f7Iz4e9JATX5AQgpSHwyMonrt6zKP R5XYA8d3tBzxmWc0uAf8iy4VmVSOtpq51bW26PMvIu6wG+1TxJ+ceO+L5JB5FNChj3Ld BRXNEZpDxozCFeKu314rO4kvlRvIWm8OO0gQlfEdHVsBP3/+TriPGz5BL5TOHTmcuTlt vwFnSmiCMsVbl/Z2XZKuJxzOe0RsAesnU2FSmAVbFv6njEqfkiRmT9JUWtNQ8eIKrtVC IKQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696214449; x=1696819249; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6KAF/A/gf2AQ43Th7UFJ8qLbpspkX1qh5IF2KzZ6tTo=; b=B0bqYEOd9bJW0SpICY4HBNpRFI5LOiU9qciYgCTwAOqluDnJ/dtGQxTGksqDr5kWvx +FExvKxc6fR8Qy+rkeJabs+oJN11YTN1dZovKQ/NQq4QWF+0y3VBlTE7mutTAgCiNr7B CeugAAC0+nQkFxrIEAoyQscHM3SqKRm/qDAeKAKpnnGkkCg6pekkPiw4rRxqvD9yk0nc 9ZjD+bhN3aj06N1r8t2s3O1f/yAjaG/9J0GIC7JHs/phj3HpIOSfLY/iMJyZTZCV5ZpH ItSKLuveak/8yGObZXvhZtZjx1N7xaL5UnUh4ZE+Eicgx9x/Vbx4kzq59eJT8zv6Y756 TNUg== X-Gm-Message-State: AOJu0YzVbGk4+gdferr0bIiijsBzFGn9C95rly7T8izED67s01R2WplB S/CioPKY7JpV8ar4QO7i/HQ= X-Google-Smtp-Source: AGHT+IFVaFt7oOepXpth1VQLNY+3YZlMZrENZ5ISdB5vUIRo5G3aCfW1fp7roDfHxK5x8Ezz1NyupA== X-Received: by 2002:a5d:8a14:0:b0:792:884b:209b with SMTP id w20-20020a5d8a14000000b00792884b209bmr11816162iod.9.1696214449271; Sun, 01 Oct 2023 19:40:49 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id v6-20020a056602058600b0079f9f7a2565sm6346113iox.38.2023.10.01.19.40.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Oct 2023 19:40:48 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , Eric Sunshine , "Eric W. Biederman" Subject: [PATCH v2 11/30] commit: export add_header_signature to support handling signatures on tags Date: Sun, 1 Oct 2023 21:40:15 -0500 Message-Id: <20231002024034.2611-11-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> References: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "Eric W. Biederman" Rename add_commit_signature as add_header_signature, and expose it so that it can be used for converting tags from one object format to another. Inspired-by: brian m. carlson Signed-off-by: "Eric W. Biederman" --- commit.c | 6 +++--- commit.h | 1 + 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/commit.c b/commit.c index 913e015966b4..2b61a4d0aa11 100644 --- a/commit.c +++ b/commit.c @@ -1101,7 +1101,7 @@ static const char *gpg_sig_headers[] = { "gpgsig-sha256", }; -static int add_commit_signature(struct strbuf *buf, struct strbuf *sig, const struct git_hash_algo *algo) +int add_header_signature(struct strbuf *buf, struct strbuf *sig, const struct git_hash_algo *algo) { int inspos, copypos; const char *eoh; @@ -1770,9 +1770,9 @@ int commit_tree_extended(const char *msg, size_t msg_len, for (i = 0; i < ARRAY_SIZE(bufs); i++) { if (!bufs[i].algo) continue; - add_commit_signature(&buffer, bufs[i].sig, bufs[i].algo); + add_header_signature(&buffer, bufs[i].sig, bufs[i].algo); if (r->compat_hash_algo) - add_commit_signature(&compat_buffer, bufs[i].sig, bufs[i].algo); + add_header_signature(&compat_buffer, bufs[i].sig, bufs[i].algo); } } diff --git a/commit.h b/commit.h index 28928833c544..03edcec0129f 100644 --- a/commit.h +++ b/commit.h @@ -370,5 +370,6 @@ int parse_buffer_signed_by_header(const char *buffer, struct strbuf *payload, struct strbuf *signature, const struct git_hash_algo *algop); +int add_header_signature(struct strbuf *buf, struct strbuf *sig, const struct git_hash_algo *algo); #endif /* COMMIT_H */ From patchwork Mon Oct 2 02:40:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13405532 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 92654E784A9 for ; Mon, 2 Oct 2023 02:41:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235441AbjJBClF (ORCPT ); Sun, 1 Oct 2023 22:41:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41412 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235417AbjJBCkx (ORCPT ); Sun, 1 Oct 2023 22:40:53 -0400 Received: from mail-io1-xd2d.google.com (mail-io1-xd2d.google.com [IPv6:2607:f8b0:4864:20::d2d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C68ECD9 for ; Sun, 1 Oct 2023 19:40:50 -0700 (PDT) Received: by mail-io1-xd2d.google.com with SMTP id ca18e2360f4ac-79f8e4108c3so558932339f.3 for ; Sun, 01 Oct 2023 19:40:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696214450; x=1696819250; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=WkhLsqo78Srsesp5Qst5wXtDzgDVzwUYy+O/4Wvj/rI=; b=MA3Awkj5GlwEqu/hEax+5SAB/TNk+Blh/yVSaEKVUcejovMJIsjuE/8y4gBrIPbRrA wmx8C7NuCOTD4OSKoilR/eG+Hs2qLjzmD3lYJt5kfh7187/J2jvulr3v78khXpe1m6fR SYGt5VQwxlpMS2blPwY60LDb0dUDEtvpcclJqv30afm5lcZPqij0mxht7LT8Z5bOnyd3 ee8wgvOk+tEv3tIopvuY735iCoro42VpQm5kjlGrc5jSoTMaSaf0+qBhHHTvoMp3Q79Y WHNUSLD6Kewg2vhb9x+sj97XIi2qEAmKQLivqKVrjymDKo81JSkKob/FeMymGCmRXIEp S9bA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696214450; x=1696819250; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WkhLsqo78Srsesp5Qst5wXtDzgDVzwUYy+O/4Wvj/rI=; b=i/i/yPSNpjB6jOK5FR0dJJeCVhEdGKCXdtL6Dq7gyA3WKqF5GWWnDhlIkgvVSkcp66 nu3bHxH9fq84UAPokKMzX7j4qFdVvUDpBFLK9oMtKb0W/pjrgKoqn/KCD61sZReTFnUs qtibsAViFgcc9YCJ8WthCbUzvM4asbvLXgG7EZI14ul/4cu/S3HGWy+ZHNKL2n/y3X6p Lqdksi68wejLrU7So9Tyoynx/44bt4pxmalWnh0RWniNuE+kdl9/a7xC/Ej3dA4gVwjb s63AnTRyG64tDzhwpvFgQOpPf0uf79AyUob7h3oQIjCqX7q9K2UC8CCs1gXjJVgtBK28 Jk3Q== X-Gm-Message-State: AOJu0Yyk2PDEzV2JvVJuDRRYUpgAiQQlWt58ffjNo/52op9J3cK3Qczw ZPtjfQJsF1XNP/qgWWUsbck= X-Google-Smtp-Source: AGHT+IGsaH8Yk13ecAaHTTUvBqnFjBZdHiFY+EpPOll0U8/AI9HGpvU7a4LT8uu25vzfg6bSOo4m/g== X-Received: by 2002:a6b:db08:0:b0:791:385c:f8b0 with SMTP id t8-20020a6bdb08000000b00791385cf8b0mr11298595ioc.3.1696214450098; Sun, 01 Oct 2023 19:40:50 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id v6-20020a056602058600b0079f9f7a2565sm6346113iox.38.2023.10.01.19.40.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Oct 2023 19:40:49 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , Eric Sunshine , "Eric W. Biederman" Subject: [PATCH v2 12/30] tag: sign both hashes Date: Sun, 1 Oct 2023 21:40:16 -0500 Message-Id: <20231002024034.2611-12-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> References: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "Eric W. Biederman" When we write a tag the object oid is specific to the hash algorithm. This matters when a tag is signed. The hash transition plan calls for signatures on both the sha1 form and the sha256 form of the object, and for both of those signatures to live in the tag object. To generate tag object with multiple signatures, first compute the unsigned form of the tag, and then if the tag is being signed compute the unsigned form of the tag with the compatibilityr hash. Then compute compute the signatures of both buffers. Once the signatures are computed add them to both buffers. This allows computing the compatibility hash in do_sign, saving write_object_file the expense of recomputing the compatibility tag just to compute it's hash. Signed-off-by: Eric W. Biederman --- builtin/tag.c | 45 +++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 41 insertions(+), 4 deletions(-) diff --git a/builtin/tag.c b/builtin/tag.c index 3918eacbb57b..8c4bc28952c2 100644 --- a/builtin/tag.c +++ b/builtin/tag.c @@ -28,6 +28,7 @@ #include "ref-filter.h" #include "date.h" #include "write-or-die.h" +#include "object-file-convert.h" static const char * const git_tag_usage[] = { N_("git tag [-a | -s | -u ] [-f] [-m | -F ] [-e]\n" @@ -174,9 +175,43 @@ static int verify_tag(const char *name, const char *ref UNUSED, return 0; } -static int do_sign(struct strbuf *buffer) +static int do_sign(struct strbuf *buffer, struct object_id **compat_oid, + struct object_id *compat_oid_buf) { - return sign_buffer(buffer, buffer, get_signing_key()); + const struct git_hash_algo *compat = the_repository->compat_hash_algo; + struct strbuf sig = STRBUF_INIT, compat_sig = STRBUF_INIT; + struct strbuf compat_buf = STRBUF_INIT; + const char *keyid = get_signing_key(); + int ret = -1; + + if (sign_buffer(buffer, &sig, keyid)) + return -1; + + if (compat) { + const struct git_hash_algo *algo = the_repository->hash_algo; + + if (convert_object_file(&compat_buf, algo, compat, + buffer->buf, buffer->len, OBJ_TAG, 1)) + goto out; + if (sign_buffer(&compat_buf, &compat_sig, keyid)) + goto out; + add_header_signature(&compat_buf, &sig, algo); + strbuf_addbuf(&compat_buf, &compat_sig); + hash_object_file(compat, compat_buf.buf, compat_buf.len, + OBJ_TAG, compat_oid_buf); + *compat_oid = compat_oid_buf; + } + + if (compat_sig.len) + add_header_signature(buffer, &compat_sig, compat); + + strbuf_addbuf(buffer, &sig); + ret = 0; +out: + strbuf_release(&sig); + strbuf_release(&compat_sig); + strbuf_release(&compat_buf); + return ret; } static const char tag_template[] = @@ -249,9 +284,11 @@ static void write_tag_body(int fd, const struct object_id *oid) static int build_tag_object(struct strbuf *buf, int sign, struct object_id *result) { - if (sign && do_sign(buf) < 0) + struct object_id *compat_oid = NULL, compat_oid_buf; + if (sign && do_sign(buf, &compat_oid, &compat_oid_buf) < 0) return error(_("unable to sign the tag")); - if (write_object_file(buf->buf, buf->len, OBJ_TAG, result) < 0) + if (write_object_file_flags(buf->buf, buf->len, OBJ_TAG, result, + compat_oid, 0) < 0) return error(_("unable to write tag file")); return 0; } From patchwork Mon Oct 2 02:40:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13405533 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 22769E784A7 for ; Mon, 2 Oct 2023 02:41:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235477AbjJBClO (ORCPT ); Sun, 1 Oct 2023 22:41:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38940 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235443AbjJBClA (ORCPT ); Sun, 1 Oct 2023 22:41:00 -0400 Received: from mail-io1-xd2f.google.com (mail-io1-xd2f.google.com [IPv6:2607:f8b0:4864:20::d2f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B5843E0 for ; Sun, 1 Oct 2023 19:40:51 -0700 (PDT) Received: by mail-io1-xd2f.google.com with SMTP id ca18e2360f4ac-79fce245bf6so380643939f.1 for ; Sun, 01 Oct 2023 19:40:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696214451; x=1696819251; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=rjIQESPk0o8Npu7i1ELfD9lAS93ZgVT64m2daSkMins=; b=OReJxIyzCPNkjsRnMFNDqbVvEFlmLzenqwa+8yYCCzz8LbR/LXvmozOXn1+OMbHqsI 851Vkug8tC/+y/TldwGynABbwRO5bqs1fwBGcKF39oE8/rk6aLrr5UQHmL34PK/c5+KO 4E+4wR5gxwBUctu1qnE8giIBEHJH7TomsoLBztwy1lkUz/A1I85zdQtQlFKvswshNM7q Dx2KY0U0LkRB8ycoihh7uPds8I2lnVUO6MF6TbAKQnNOiqzbha45vkSPc1PKP+0CYosi MfRQvTLqDbYPrXXOo6AYjhk7AoQMaAyPTAlOmZszQkxpLGhJoy4rnTmEDnBTc3EFDQL5 Z1WA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696214451; x=1696819251; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rjIQESPk0o8Npu7i1ELfD9lAS93ZgVT64m2daSkMins=; b=kykPSYi6tM8ZoU8YfI1NmdSI5xcscv3KywyxGq5Cup5jwc8kq+fFlnDglSYHIFV6Bn O1XpRx+ufqMXJ/oMZSxqq0JcYGSuni9hEhr6cuH7GZbCQxSobMHSGxr3h/HwbcunDGyf PLChmHwQOLQ1kxQDeKz9p5fC/ks3lBAko6zdyKdFq7XINrN0T8/G61NRqYOMOwo3Xenn Hj6ShwVgSjVuwZipV7bEyPMXd3cC/aQgKkm2uxKLW6TUYZfDH6vMDABMu/d3tZHNitDH wdB67g7cCLrDhbgy1LlfLSGLiSl6JJxq9QlyNBblBQTdoo7kXHJHBMw5/q97GpnexU8A m3lg== X-Gm-Message-State: AOJu0Yy+i//TKfNPYM52cM+Cy7L8fkVAUaXKWZWTQ7WTMOVZf1yAjQqO euMdGtQsFjy1LygszQezfWQ= X-Google-Smtp-Source: AGHT+IGmpC38Orvr7CXdkRbi0sDovesiqpiQ826ihm/INmwugN84/5ZCO9as5zlvnjbObjXCwoLIEA== X-Received: by 2002:a6b:5c02:0:b0:79a:b526:2f2a with SMTP id z2-20020a6b5c02000000b0079ab5262f2amr11175034ioh.5.1696214450919; Sun, 01 Oct 2023 19:40:50 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id v6-20020a056602058600b0079f9f7a2565sm6346113iox.38.2023.10.01.19.40.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Oct 2023 19:40:50 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , Eric Sunshine , "Eric W. Biederman" Subject: [PATCH v2 13/30] cache: add a function to read an OID of a specific algorithm Date: Sun, 1 Oct 2023 21:40:17 -0500 Message-Id: <20231002024034.2611-13-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> References: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "brian m. carlson" Currently, we always read a object ID of the current algorithm with oidread. However, once we start converting objects, we'll need to consider what happens when we want to read an object ID of a specific algorithm, such as the compatibility algorithm. To make this easier, let's define oidread_algop, which specifies which algorithm we should use for our object ID, and define oidread in terms of it. Signed-off-by: brian m. carlson Signed-off-by: "Eric W. Biederman" --- hash.h | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/hash.h b/hash.h index 615ae0691d07..e064807c1733 100644 --- a/hash.h +++ b/hash.h @@ -73,10 +73,15 @@ static inline void oidclr(struct object_id *oid) oid->algo = hash_algo_by_ptr(the_hash_algo); } +static inline void oidread_algop(struct object_id *oid, const unsigned char *hash, const struct git_hash_algo *algop) +{ + memcpy(oid->hash, hash, algop->rawsz); + oid->algo = hash_algo_by_ptr(algop); +} + static inline void oidread(struct object_id *oid, const unsigned char *hash) { - memcpy(oid->hash, hash, the_hash_algo->rawsz); - oid->algo = hash_algo_by_ptr(the_hash_algo); + oidread_algop(oid, hash, the_hash_algo); } static inline int is_empty_blob_sha1(const unsigned char *sha1) From patchwork Mon Oct 2 02:40:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13405534 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 700DCE784A8 for ; Mon, 2 Oct 2023 02:41:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235501AbjJBClQ (ORCPT ); Sun, 1 Oct 2023 22:41:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38896 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235450AbjJBClA (ORCPT ); Sun, 1 Oct 2023 22:41:00 -0400 Received: from mail-io1-xd31.google.com (mail-io1-xd31.google.com [IPv6:2607:f8b0:4864:20::d31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 59D1FE9 for ; Sun, 1 Oct 2023 19:40:52 -0700 (PDT) Received: by mail-io1-xd31.google.com with SMTP id ca18e2360f4ac-7a26fd82847so105004739f.3 for ; Sun, 01 Oct 2023 19:40:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696214451; x=1696819251; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FkazeKVM1RtP9Z70mx3eJTcTbac/plstEPcnoAezcv4=; b=Yq8wFHiq4YgZ54tZM4WEvjxhvZs8wI9D7O4kY4sQ8VZNJQuLHkNgfmqc7ViU4o8LLq 1cCi+OGCVbACYwq23P9pTMZt8T1eadH0adMGibXs0/NYzC3+P/0GUpr6nBri6qvydYLw 8xxXuZAsXXRsaMDhB/6fI04nMrYSnOauuC2PKLTV0vV8IEc/UN55Tk8e9af+ICOSdfz1 eAxDYtNAbAcMnOqpiXMD24u/pdQGezLNh9M+9BFW6WB13ZDEhWRjZgy32IX7ONfMijNK Hn2yYs0d8Jw12Q6GnpM2RfbGqlOwQ+aaKluhfUYk3l9KcFK/wwjCxomEshn//8bXO1Fr rMJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696214451; x=1696819251; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FkazeKVM1RtP9Z70mx3eJTcTbac/plstEPcnoAezcv4=; b=RI6qKaXNbu4VrwLyCWkx0NzZAanliby/UCGdFdydjYVKrEIZkA5ZfCYe/HPLYieFNr S3vEdJo3rlYd65y0Pxq4S3NEuNVY86g7b488FHONL29s9PhRLhoDZw/7yPxIcgQcCe+Q UyQmgFji7Z+YnsLl+IgpW9rItxmgkQ3kq/ccOo+LP4K0lDl+2ZJiEMuppqtxpnH9eDWO lQ6qayL7mmS680fHM864MORjF1cwpcdkAL3PQRqB0EcYFwz4PPdPykLR79IsiLX3CbIj 4i4b9x+zp4r1hV5Hit1EG1JJOW7D9iRMI0GR9HeBnBFr8MufBXQa7JzxVFxsQji/J6vS OJyA== X-Gm-Message-State: AOJu0Yycrc8VRJbrKx8KQnN4ar5gT5GV3PxUaOqksFoUgAsp2MTL6cwc Vnsg7NiUL86LyegHN1dweSE= X-Google-Smtp-Source: AGHT+IHwODp3nH5RRGByNLKK2zKRZqAeWSF2Nbx3539Mm5NOnMsIFl7rvBNWUHsR81fPArbRo0MLpQ== X-Received: by 2002:a05:6602:420d:b0:798:312a:5403 with SMTP id cb13-20020a056602420d00b00798312a5403mr12903850iob.19.1696214451708; Sun, 01 Oct 2023 19:40:51 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id v6-20020a056602058600b0079f9f7a2565sm6346113iox.38.2023.10.01.19.40.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Oct 2023 19:40:51 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , Eric Sunshine , "Eric W. Biederman" Subject: [PATCH v2 14/30] object: factor out parse_mode out of fast-import and tree-walk into in object.h Date: Sun, 1 Oct 2023 21:40:18 -0500 Message-Id: <20231002024034.2611-14-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> References: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "Eric W. Biederman" builtin/fast-import.c and tree-walk.c have almost identical version of get_mode. The two functions started out the same but have diverged slightly. The version in fast-import changed mode to a uint16_t to save memory. The version in tree-walk started erroring if no mode was present. As far as I can tell both of these changes are valid for both of the callers, so add the both changes and place the common parsing helper in object.h Rename the helper from get_mode to parse_mode so it does not conflict with another helper named get_mode in diff-no-index.c This will be used shortly in a new helper decode_tree_entry_raw which is used to compute cmpatibility objects as part of the sha256 transition. Signed-off-by: "Eric W. Biederman" --- builtin/fast-import.c | 18 ++---------------- object.h | 18 ++++++++++++++++++ tree-walk.c | 22 +++------------------- 3 files changed, 23 insertions(+), 35 deletions(-) diff --git a/builtin/fast-import.c b/builtin/fast-import.c index 4dbb10aff3da..2c645fcfbe3f 100644 --- a/builtin/fast-import.c +++ b/builtin/fast-import.c @@ -1235,20 +1235,6 @@ static void *gfi_unpack_entry( return unpack_entry(the_repository, p, oe->idx.offset, &type, sizep); } -static const char *get_mode(const char *str, uint16_t *modep) -{ - unsigned char c; - uint16_t mode = 0; - - while ((c = *str++) != ' ') { - if (c < '0' || c > '7') - return NULL; - mode = (mode << 3) + (c - '0'); - } - *modep = mode; - return str; -} - static void load_tree(struct tree_entry *root) { struct object_id *oid = &root->versions[1].oid; @@ -1286,7 +1272,7 @@ static void load_tree(struct tree_entry *root) t->entries[t->entry_count++] = e; e->tree = NULL; - c = get_mode(c, &e->versions[1].mode); + c = parse_mode(c, &e->versions[1].mode); if (!c) die("Corrupt mode in %s", oid_to_hex(oid)); e->versions[0].mode = e->versions[1].mode; @@ -2275,7 +2261,7 @@ static void file_change_m(const char *p, struct branch *b) struct object_id oid; uint16_t mode, inline_data = 0; - p = get_mode(p, &mode); + p = parse_mode(p, &mode); if (!p) die("Corrupt mode: %s", command_buf.buf); switch (mode) { diff --git a/object.h b/object.h index 114d45954d08..70c8d4ae63dc 100644 --- a/object.h +++ b/object.h @@ -190,6 +190,24 @@ void *create_object(struct repository *r, const struct object_id *oid, void *obj void *object_as_type(struct object *obj, enum object_type type, int quiet); + +static inline const char *parse_mode(const char *str, uint16_t *modep) +{ + unsigned char c; + unsigned int mode = 0; + + if (*str == ' ') + return NULL; + + while ((c = *str++) != ' ') { + if (c < '0' || c > '7') + return NULL; + mode = (mode << 3) + (c - '0'); + } + *modep = mode; + return str; +} + /* * Returns the object, having parsed it to find out what it is. * diff --git a/tree-walk.c b/tree-walk.c index 29ead71be173..3af50a01c2c7 100644 --- a/tree-walk.c +++ b/tree-walk.c @@ -10,27 +10,11 @@ #include "pathspec.h" #include "json-writer.h" -static const char *get_mode(const char *str, unsigned int *modep) -{ - unsigned char c; - unsigned int mode = 0; - - if (*str == ' ') - return NULL; - - while ((c = *str++) != ' ') { - if (c < '0' || c > '7') - return NULL; - mode = (mode << 3) + (c - '0'); - } - *modep = mode; - return str; -} - static int decode_tree_entry(struct tree_desc *desc, const char *buf, unsigned long size, struct strbuf *err) { const char *path; - unsigned int mode, len; + unsigned int len; + uint16_t mode; const unsigned hashsz = the_hash_algo->rawsz; if (size < hashsz + 3 || buf[size - (hashsz + 1)]) { @@ -38,7 +22,7 @@ static int decode_tree_entry(struct tree_desc *desc, const char *buf, unsigned l return -1; } - path = get_mode(buf, &mode); + path = parse_mode(buf, &mode); if (!path) { strbuf_addstr(err, _("malformed mode in tree entry")); return -1; From patchwork Mon Oct 2 02:40:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13405535 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 25755E784A7 for ; Mon, 2 Oct 2023 02:41:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235417AbjJBClS (ORCPT ); Sun, 1 Oct 2023 22:41:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38982 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235437AbjJBClB (ORCPT ); Sun, 1 Oct 2023 22:41:01 -0400 Received: from mail-io1-xd35.google.com (mail-io1-xd35.google.com [IPv6:2607:f8b0:4864:20::d35]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6B04EFD for ; Sun, 1 Oct 2023 19:40:53 -0700 (PDT) Received: by mail-io1-xd35.google.com with SMTP id ca18e2360f4ac-7a26fbfcf8eso91522839f.2 for ; Sun, 01 Oct 2023 19:40:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696214452; x=1696819252; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=+UQ12IiVRZf6rc/H6QZj+3H9gwwx4IxYNbtpQetqEnY=; b=dYt2y+e4WtTCOc0TUfiwvPbaDY0DUJyiFHHIE5oXatOOsYZetTNa3vMZhVKREbpTk9 tRRODjrzo7hZMW1dkg6H0SmRHFoKqMUtepYzdpJQkXAlFISnyDNNXcdiWUHP4YHZJjMS gpsd7oDsChiY2OwIIVyjuHtATKi+WAPGHjBneBAI4LJnmIlyVDIZFgVVs07/Y/7vGo31 KxxwDL6UKHklK9hLXjMsX5sl6odZndCnwLN5sip6y/kZ4EuLl9muvRAz3zNMjUSRPo3y dt7QeIgvgotKFC7ZCCDqBhw1gqYxsTFfdUJzmxld2Uh4YPynPVrPC0hg1R/rt8qWQcLK MpSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696214452; x=1696819252; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=+UQ12IiVRZf6rc/H6QZj+3H9gwwx4IxYNbtpQetqEnY=; b=U7ZwHadaJF18fr+AGMSKzD2YLuE1QKKCA+mNCrZYH5jdH1PN2jqf45eBWQlkReGgKq sa99G1NUvUEDdynDwvetMDEvWqJUn4k/LvUVkqnSYsSQI30Qwegx2Zyud/ZKXfb5KWLA joyuSDtvqBU9CbINtWThT3h+KsqeykAu+g97pi/Apt4xVmTIOngREvZ8VgSzcJCPqgjz P23gVsHZf8EavnCHAy9fyLT8DsAJdeOzcBw905vJ4vhQ2c/gajV0LRrH+/WXbmuyypRj ltf9Om1VoOKAJvETVwb130Oz2oubh1teuoSb/SjGQrOnlHtmyqlhDai+gmdGj9cQSyYt ymww== X-Gm-Message-State: AOJu0Yx6aZQPvzqY0imyMXFWgP4K9iDkZ9XpDbUlRjz0IezrYEYSqg0D Nw6DkiUjAhW5H4+yLkfBgQ4= X-Google-Smtp-Source: AGHT+IENIjd23Cg+lLyZl1mC/Lm1x7gl5iRu9+/zafEjvsjuCfycmY7ZGCFdlFI/NbQ0XA48s7lAkA== X-Received: by 2002:a6b:dc02:0:b0:79f:ce11:c1b0 with SMTP id s2-20020a6bdc02000000b0079fce11c1b0mr11878279ioc.6.1696214452591; Sun, 01 Oct 2023 19:40:52 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id v6-20020a056602058600b0079f9f7a2565sm6346113iox.38.2023.10.01.19.40.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Oct 2023 19:40:52 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , Eric Sunshine , "Eric W. Biederman" Subject: [PATCH v2 15/30] object-file-convert: add a function to convert trees between algorithms Date: Sun, 1 Oct 2023 21:40:19 -0500 Message-Id: <20231002024034.2611-15-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> References: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "brian m. carlson" In the future, we're going to want to provide SHA-256 repositories that have compatibility support for SHA-1 as well. In order to do so, we'll need to be able to convert tree objects from SHA-256 to SHA-1 by writing a tree with each SHA-256 object ID mapped to a SHA-1 object ID. We implement a function, convert_tree_object, that takes an existing tree buffer and writes it to a new strbuf, converting between algorithms. Let's make this function generic, because while we only need it to convert from the main algorithm to the compatibility algorithm now, we may need to do the other way around in the future, such as for transport. We avoid reusing the code in decode_tree_entry because that code normalizes data, and we don't want that here. We want to produce a complete round trip of data, so if, for example, the old entry had a wrongly zero-padded mode, we'd want to preserve that when converting to ensure a stable hash value. Signed-off-by: brian m. carlson Signed-off-by: Eric W. Biederman --- object-file-convert.c | 51 ++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 50 insertions(+), 1 deletion(-) diff --git a/object-file-convert.c b/object-file-convert.c index 1ec945eaa17f..70b80fb61e54 100644 --- a/object-file-convert.c +++ b/object-file-convert.c @@ -1,8 +1,10 @@ #include "git-compat-util.h" #include "gettext.h" #include "strbuf.h" +#include "hex.h" #include "repository.h" #include "hash-ll.h" +#include "hash.h" #include "object.h" #include "loose.h" #include "object-file-convert.h" @@ -36,6 +38,51 @@ int repo_oid_to_algop(struct repository *repo, const struct object_id *src, return 0; } +static int decode_tree_entry_raw(struct object_id *oid, const char **path, + size_t *len, const struct git_hash_algo *algo, + const char *buf, unsigned long size) +{ + uint16_t mode; + const unsigned hashsz = algo->rawsz; + + if (size < hashsz + 3 || buf[size - (hashsz + 1)]) { + return -1; + } + + *path = parse_mode(buf, &mode); + if (!*path || !**path) + return -1; + *len = strlen(*path) + 1; + + oidread_algop(oid, (const unsigned char *)*path + *len, algo); + return 0; +} + +static int convert_tree_object(struct strbuf *out, + const struct git_hash_algo *from, + const struct git_hash_algo *to, + const char *buffer, size_t size) +{ + const char *p = buffer, *end = buffer + size; + + while (p < end) { + struct object_id entry_oid, mapped_oid; + const char *path = NULL; + size_t pathlen; + + if (decode_tree_entry_raw(&entry_oid, &path, &pathlen, from, p, + end - p)) + return error(_("failed to decode tree entry")); + if (repo_oid_to_algop(the_repository, &entry_oid, to, &mapped_oid)) + return error(_("failed to map tree entry for %s"), oid_to_hex(&entry_oid)); + strbuf_add(out, p, path - p); + strbuf_add(out, path, pathlen); + strbuf_add(out, mapped_oid.hash, to->rawsz); + p = path + pathlen + from->rawsz; + } + return 0; +} + int convert_object_file(struct strbuf *outbuf, const struct git_hash_algo *from, const struct git_hash_algo *to, @@ -50,8 +97,10 @@ int convert_object_file(struct strbuf *outbuf, BUG("Refusing noop object file conversion"); switch (type) { - case OBJ_COMMIT: case OBJ_TREE: + ret = convert_tree_object(outbuf, from, to, buf, len); + break; + case OBJ_COMMIT: case OBJ_TAG: default: /* Not implemented yet, so fail. */ From patchwork Mon Oct 2 02:40:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13405536 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 601CEE784A6 for ; Mon, 2 Oct 2023 02:41:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235450AbjJBClT (ORCPT ); Sun, 1 Oct 2023 22:41:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38984 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235440AbjJBClB (ORCPT ); Sun, 1 Oct 2023 22:41:01 -0400 Received: from mail-io1-xd30.google.com (mail-io1-xd30.google.com [IPv6:2607:f8b0:4864:20::d30]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1D098106 for ; Sun, 1 Oct 2023 19:40:54 -0700 (PDT) Received: by mail-io1-xd30.google.com with SMTP id ca18e2360f4ac-79fa2dbd793so514412039f.2 for ; Sun, 01 Oct 2023 19:40:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696214453; x=1696819253; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=D6Dj1s4v7nOhcXIwavrd7RvfOcT4mCe2pnrFxKbopgY=; b=dGKJopLXzAgawtfe/WbfMUJDO0AF/A77FcRRBvxO57IrRKj4rfJsnVnn4+jOGiIRPL Vcn+c416zyLEd39HKsrFGxYhKfIfFHxrTli7uKthOizBKNUTPSLXgWqoIQCTO67RYIBn KGLf6vdHH6JlGL2v8W9Ng6on2+N7xXer0QwsN3c9on1E/FGYhAyGkQfbK8npJYdEju4N aVg6pJsOkyDKN0sEhzPKyd8Zc8BZ6jndzoIXjtjrBUpfB1Iu8m1Ub9yCcbrp+B5/uw8B IqPJlyhklhUw5P69FmGEoLl3IM8ieGGDZLZPP1QtznK5MoKMkcSVxFPeVmdOz+xuFZx0 SmTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696214453; x=1696819253; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=D6Dj1s4v7nOhcXIwavrd7RvfOcT4mCe2pnrFxKbopgY=; b=CxDwWr5xIsGh9AnGPuvYMnCCsMtjL0TmYHJbpOktJ3Xpu/tmpa1OFxeJWZ2B7HEIHL goNlRbsVm2KRcKTYjWcR6Jk1yuShI8ANo3JGmWBZpV29HONVZTlJBwZBPSVEUXHJowM/ OBKhd7s3u/G+dakuDNVokU8BDAxI0Wwv0dfqMi5RPQLwEsUIaH39ffCnYdd/+TDgjtfs taIB0UWJjbiTHPRILsFWWOev2z7T+MOvmpd+NY6FtQzmmZfMoyVw3sf+nJhKIBeKCqez /QbETe7EDtTSR8gg0gbPXYf3emLjMpFCnKxmbqZrhVAMuL2ADLu0q5J0PqykXlu+lRlE pVBQ== X-Gm-Message-State: AOJu0Yx7rgX32eT+2krMDGQ/FHEWZR/72FpIQxpwboLeCcO57C/Dop/0 MC0+Tf/PbkK2xJXZVfGKkAo= X-Google-Smtp-Source: AGHT+IHx1W18fdnTYWWkDw+HLOZIkq7RSQzIKnRjQnOboCp3WlfZOC9jvIBUR2MTyebtxwrnQfDLvQ== X-Received: by 2002:a6b:e005:0:b0:794:eaef:f43e with SMTP id z5-20020a6be005000000b00794eaeff43emr11436089iog.1.1696214453390; Sun, 01 Oct 2023 19:40:53 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id v6-20020a056602058600b0079f9f7a2565sm6346113iox.38.2023.10.01.19.40.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Oct 2023 19:40:53 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , Eric Sunshine , "Eric W. Biederman" Subject: [PATCH v2 16/30] object-file-convert: convert tag objects when writing Date: Sun, 1 Oct 2023 21:40:20 -0500 Message-Id: <20231002024034.2611-16-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> References: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "brian m. carlson" When writing a tag object in a repository with both SHA-1 and SHA-256, we'll need to convert our commit objects so that we can write the hash values for both into the repository. To do so, let's add a function to convert tag objects. Note that signatures for tag objects in the current algorithm trail the message, and those for the alternate algorithm are in headers. Therefore, we parse the tag object for both a trailing signature and a header and then, when writing the other format, swap the two around. Signed-off-by: brian m. carlson Signed-off-by: Eric W. Biederman --- object-file-convert.c | 52 ++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 51 insertions(+), 1 deletion(-) diff --git a/object-file-convert.c b/object-file-convert.c index 70b80fb61e54..089b68442de8 100644 --- a/object-file-convert.c +++ b/object-file-convert.c @@ -7,6 +7,8 @@ #include "hash.h" #include "object.h" #include "loose.h" +#include "commit.h" +#include "gpg-interface.h" #include "object-file-convert.h" int repo_oid_to_algop(struct repository *repo, const struct object_id *src, @@ -83,6 +85,52 @@ static int convert_tree_object(struct strbuf *out, return 0; } +static int convert_tag_object(struct strbuf *out, + const struct git_hash_algo *from, + const struct git_hash_algo *to, + const char *buffer, size_t size) +{ + struct strbuf payload = STRBUF_INIT, temp = STRBUF_INIT, oursig = STRBUF_INIT, othersig = STRBUF_INIT; + size_t payload_size; + struct object_id oid, mapped_oid; + const char *p; + + /* Add some slop for longer signature header in the new algorithm. */ + strbuf_grow(out, size + 7); + + /* Is there a signature for our algorithm? */ + payload_size = parse_signed_buffer(buffer, size); + strbuf_add(&payload, buffer, payload_size); + if (payload_size != size) { + /* Yes, there is. */ + strbuf_add(&oursig, buffer + payload_size, size - payload_size); + } + /* Now, is there a signature for the other algorithm? */ + if (parse_buffer_signed_by_header(payload.buf, payload.len, &temp, &othersig, to)) { + /* Yes, there is. */ + strbuf_swap(&payload, &temp); + strbuf_release(&temp); + } + + /* + * Our payload is now in payload and we may have up to two signatrures + * in oursig and othersig. + */ + if (strncmp(payload.buf, "object ", 7) || payload.buf[from->hexsz + 7] != '\n') + return error("bogus tag object"); + if (parse_oid_hex_algop(payload.buf + 7, &oid, &p, from) < 0) + return error("bad tag object ID"); + if (repo_oid_to_algop(the_repository, &oid, to, &mapped_oid)) + return error("unable to map tree %s in tag object", + oid_to_hex(&oid)); + strbuf_addf(out, "object %s", oid_to_hex(&mapped_oid)); + strbuf_add(out, p, payload.len - (p - payload.buf)); + strbuf_addbuf(out, &othersig); + if (oursig.len) + add_header_signature(out, &oursig, from); + return 0; +} + int convert_object_file(struct strbuf *outbuf, const struct git_hash_algo *from, const struct git_hash_algo *to, @@ -100,8 +148,10 @@ int convert_object_file(struct strbuf *outbuf, case OBJ_TREE: ret = convert_tree_object(outbuf, from, to, buf, len); break; - case OBJ_COMMIT: case OBJ_TAG: + ret = convert_tag_object(outbuf, from, to, buf, len); + break; + case OBJ_COMMIT: default: /* Not implemented yet, so fail. */ ret = -1; From patchwork Mon Oct 2 02:40:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13405537 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A9E8AE784A9 for ; Mon, 2 Oct 2023 02:41:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235457AbjJBClV (ORCPT ); Sun, 1 Oct 2023 22:41:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38896 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235444AbjJBClN (ORCPT ); Sun, 1 Oct 2023 22:41:13 -0400 Received: from mail-io1-xd2f.google.com (mail-io1-xd2f.google.com [IPv6:2607:f8b0:4864:20::d2f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 10C2911B for ; Sun, 1 Oct 2023 19:40:55 -0700 (PDT) Received: by mail-io1-xd2f.google.com with SMTP id ca18e2360f4ac-79fd60f40ebso60583739f.1 for ; Sun, 01 Oct 2023 19:40:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696214454; x=1696819254; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2kiMuJR1HWDeYcrEqgyukg8c+vv+20BE/y5lrSGyOzc=; b=hqI+EkHl6MU/910Phgthn+fFSLXlhmeQr2rUSkswgLI1zMvtZOejZiXePEAT4jV8Hk qFUyGnQC4DD8TVP+bs4AsTlynxMwjToad3rZa7bmZ3fMMy3v48UwBtGijm1EYnQGPMhw 7NDvV49bt1371O0L0Q8hQ+/A74//In8wi2vnb7HfkkX4zZKYGT+Z6mKqdLX9Y7lO0qtv VJzkCvr4zTqSZzagZ20Q/Kd1ynqr3ZnnPReFzTBexYsvrbKp3PRrjW16Qdcv1Aa7phWr bV6lcFfsUSD0jEmHtJhY4gbbfshSykSgTEYZAHC7hEJXde0JoFN3S97Sx3FB2jcTeiRB qMpQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696214454; x=1696819254; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2kiMuJR1HWDeYcrEqgyukg8c+vv+20BE/y5lrSGyOzc=; b=pTVWhvlwajIsWi3AlHzXLaNk0f1oADzrgOuoVFAc6Fcmp01XQL2hBHR6+BxYcQ4nAA euJMdu9trS+wPnaS76G6nIOvkncvErKGkJ+M0BZHUNpRncefHmoc8W4psJZK3BtJN/Ob ZuBzfdiJJGyzuCf74AsnYLRNdU+wG7aI1WmQKCUASjMUC4yaADh/G5SvTp6r4FDfz7cn qrermRlv0hjls+mOr8lsGCBgCNfP7rV0y7GgVT2xGjhOmXLb0BMN1IGWAC4jqJ/H08DV qR9DE6mwH85R0Ko3ub7RuKCHO8EIFpDuRgF20cPyJ78kZffj0Ywey3AhFfd83wHxltLb xn+Q== X-Gm-Message-State: AOJu0YweomE050OMHB4lX7EtjBXmn+kvNi3JkT5mEodUiqWcxhCgG7BE bCjPSGyGnOaMEgpvkOryqOTctImU1noB8g== X-Google-Smtp-Source: AGHT+IHSAnYDQOxoTJj8svwXvzGU7z3LwCjSF4UbN+5MtzI3Slv0syq/D4nEotb8asvJtWtwa2scAA== X-Received: by 2002:a05:6602:1853:b0:787:8cf:fe82 with SMTP id d19-20020a056602185300b0078708cffe82mr8184374ioi.8.1696214454273; Sun, 01 Oct 2023 19:40:54 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id v6-20020a056602058600b0079f9f7a2565sm6346113iox.38.2023.10.01.19.40.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Oct 2023 19:40:53 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , Eric Sunshine , "Eric W. Biederman" Subject: [PATCH v2 17/30] object-file-convert: don't leak when converting tag objects Date: Sun, 1 Oct 2023 21:40:21 -0500 Message-Id: <20231002024034.2611-17-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> References: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "Eric W. Biederman" Upon close examination I discovered that while brian's code to convert tag objects was functionally correct, it leaked memory. Rearrange the code so that all error checking happens before any memory is allocated. Add code to release the temporary strbufs the code uses. The code pretty much assumes the tag object ends with a newline, so add an explict test to verify that is the case. Signed-off-by: Eric W. Biederman --- object-file-convert.c | 45 ++++++++++++++++++++++++------------------- 1 file changed, 25 insertions(+), 20 deletions(-) diff --git a/object-file-convert.c b/object-file-convert.c index 089b68442de8..79e8e211ff95 100644 --- a/object-file-convert.c +++ b/object-file-convert.c @@ -90,44 +90,49 @@ static int convert_tag_object(struct strbuf *out, const struct git_hash_algo *to, const char *buffer, size_t size) { - struct strbuf payload = STRBUF_INIT, temp = STRBUF_INIT, oursig = STRBUF_INIT, othersig = STRBUF_INIT; + struct strbuf payload = STRBUF_INIT, oursig = STRBUF_INIT, othersig = STRBUF_INIT; + const int entry_len = from->hexsz + 7; size_t payload_size; struct object_id oid, mapped_oid; const char *p; - /* Add some slop for longer signature header in the new algorithm. */ - strbuf_grow(out, size + 7); + /* Consume the object line */ + if ((entry_len >= size) || + memcmp(buffer, "object ", 7) || buffer[entry_len] != '\n') + return error("bogus tag object"); + if (parse_oid_hex_algop(buffer + 7, &oid, &p, from) < 0) + return error("bad tag object ID"); + if (repo_oid_to_algop(the_repository, &oid, to, &mapped_oid)) + return error("unable to map tree %s in tag object", + oid_to_hex(&oid)); + size -= ((p + 1) - buffer); + buffer = p + 1; /* Is there a signature for our algorithm? */ payload_size = parse_signed_buffer(buffer, size); - strbuf_add(&payload, buffer, payload_size); if (payload_size != size) { /* Yes, there is. */ strbuf_add(&oursig, buffer + payload_size, size - payload_size); } - /* Now, is there a signature for the other algorithm? */ - if (parse_buffer_signed_by_header(payload.buf, payload.len, &temp, &othersig, to)) { - /* Yes, there is. */ - strbuf_swap(&payload, &temp); - strbuf_release(&temp); - } + /* Now, is there a signature for the other algorithm? */ + parse_buffer_signed_by_header(buffer, payload_size, &payload, &othersig, to); /* * Our payload is now in payload and we may have up to two signatrures * in oursig and othersig. */ - if (strncmp(payload.buf, "object ", 7) || payload.buf[from->hexsz + 7] != '\n') - return error("bogus tag object"); - if (parse_oid_hex_algop(payload.buf + 7, &oid, &p, from) < 0) - return error("bad tag object ID"); - if (repo_oid_to_algop(the_repository, &oid, to, &mapped_oid)) - return error("unable to map tree %s in tag object", - oid_to_hex(&oid)); - strbuf_addf(out, "object %s", oid_to_hex(&mapped_oid)); - strbuf_add(out, p, payload.len - (p - payload.buf)); - strbuf_addbuf(out, &othersig); + + /* Add some slop for longer signature header in the new algorithm. */ + strbuf_grow(out, (7 + to->hexsz + 1) + size + 7); + strbuf_addf(out, "object %s\n", oid_to_hex(&mapped_oid)); + strbuf_addbuf(out, &payload); if (oursig.len) add_header_signature(out, &oursig, from); + strbuf_addbuf(out, &othersig); + + strbuf_release(&payload); + strbuf_release(&othersig); + strbuf_release(&oursig); return 0; } From patchwork Mon Oct 2 02:40:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13405539 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5B22BE784A8 for ; Mon, 2 Oct 2023 02:41:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235556AbjJBClc (ORCPT ); Sun, 1 Oct 2023 22:41:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38984 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235473AbjJBClO (ORCPT ); Sun, 1 Oct 2023 22:41:14 -0400 Received: from mail-io1-xd29.google.com (mail-io1-xd29.google.com [IPv6:2607:f8b0:4864:20::d29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B9AB9195 for ; Sun, 1 Oct 2023 19:40:55 -0700 (PDT) Received: by mail-io1-xd29.google.com with SMTP id ca18e2360f4ac-79fca042ec0so387508439f.3 for ; Sun, 01 Oct 2023 19:40:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696214455; x=1696819255; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=SEXEDOM77zShgkhxlSm3FRtPftyWOh24/z/5gLCNmyk=; b=BSZxSLojKHwuQOyv2ifQv/q6F+fZ5UmnA2OibVhrUxQ+hsyuz9TthmAhSjTw3a5nB1 nF5akWG0fg6bCHp6gOk9YjeFxZg6JxkAVciyT/iYlwbIxR/ZS0pJreunlsCqDypSVRS2 3q3FJtFvhCDBED+u+yz+WFV1ExKnSkZHv9CrAeamLydNk2hlBsAyanR9vbkxYFJ+ut8v c0G6eEOaRFZoMz8q/Ck45CoEJnks6ERcEUxxCH9MwMWehRMrI1Gwmbu84JV2G4DcC7VJ IOeDl8faL9M06KO904FuQWjEtr6/5PNBbf7PKc4uILKmv54z4eZb7QGjDK+3jP2Dw7wu 9aHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696214455; x=1696819255; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=SEXEDOM77zShgkhxlSm3FRtPftyWOh24/z/5gLCNmyk=; b=X2ZbVm1rUrDjX7bLBzEno8hxJfRuH3ac0qsd7yBc0kPup+GcKHjHrioDyI5OlFGTlt FZ07eXof42Oa71KHLyzSuBAiCe0RCZrWQLQYGi8160jHkzzJ8P6T7E7CoQMDFCPWD40o ETIQ4iENsF1um3rxgAdCqoRN37JMYVyIotHsV0kN/DnJ2DCOwmmagRrevXOG0eX/2nkp xuOlZwJnPbWjz1mOtD6IRzI2xZVPIJrs2Gqj1S3QfiDIj5MtbYfSbnHIEyLdjwIFIBRq LPbrUv1yxcFTNiYDFzXTblSuG+pl5NhBGpgRx0hXjj5R0foJEhr9iQjQKmFPk/Tuht5Z Lqrg== X-Gm-Message-State: AOJu0YwyovtmcYH9/AQ9gzdHERBzb5eho2bkoXJbkoncTtRZLkIpDMsC uHw9/rvmhRrSb/0DG/w03DE= X-Google-Smtp-Source: AGHT+IGuQUgPD47R1119U48c4WL56770732I7sov1FV8i0/SWhIfy3C7uHap3uhTpEg+F/A5UPzVIw== X-Received: by 2002:a05:6602:91:b0:794:da97:d194 with SMTP id h17-20020a056602009100b00794da97d194mr12880408iob.19.1696214455060; Sun, 01 Oct 2023 19:40:55 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id v6-20020a056602058600b0079f9f7a2565sm6346113iox.38.2023.10.01.19.40.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Oct 2023 19:40:54 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , Eric Sunshine , "Eric W. Biederman" Subject: [PATCH v2 18/30] object-file-convert: convert commit objects when writing Date: Sun, 1 Oct 2023 21:40:22 -0500 Message-Id: <20231002024034.2611-18-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> References: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "brian m. carlson" When writing a commit object in a repository with both SHA-1 and SHA-256, we'll need to convert our commit objects so that we can write the hash values for both into the repository. To do so, let's add a function to convert commit objects. Read the commit object and map the tree value and any of the parent values, and copy the rest of the commit through unmodified. Note that we don't need to modify the signature headers, because they are the same under both algorithms. Signed-off-by: brian m. carlson Signed-off-by: Eric W. Biederman --- object-file-convert.c | 46 ++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 45 insertions(+), 1 deletion(-) diff --git a/object-file-convert.c b/object-file-convert.c index 79e8e211ff95..0da081104ed4 100644 --- a/object-file-convert.c +++ b/object-file-convert.c @@ -136,6 +136,48 @@ static int convert_tag_object(struct strbuf *out, return 0; } +static int convert_commit_object(struct strbuf *out, + const struct git_hash_algo *from, + const struct git_hash_algo *to, + const char *buffer, size_t size) +{ + const char *tail = buffer; + const char *bufptr = buffer; + const int tree_entry_len = from->hexsz + 5; + const int parent_entry_len = from->hexsz + 7; + struct object_id oid, mapped_oid; + const char *p; + + tail += size; + if (tail <= bufptr + tree_entry_len + 1 || memcmp(bufptr, "tree ", 5) || + bufptr[tree_entry_len] != '\n') + return error("bogus commit object"); + if (parse_oid_hex_algop(bufptr + 5, &oid, &p, from) < 0) + return error("bad tree pointer"); + + if (repo_oid_to_algop(the_repository, &oid, to, &mapped_oid)) + return error("unable to map tree %s in commit object", + oid_to_hex(&oid)); + strbuf_addf(out, "tree %s\n", oid_to_hex(&mapped_oid)); + bufptr = p + 1; + + while (bufptr + parent_entry_len < tail && !memcmp(bufptr, "parent ", 7)) { + if (tail <= bufptr + parent_entry_len + 1 || + parse_oid_hex_algop(bufptr + 7, &oid, &p, from) || + *p != '\n') + return error("bad parents in commit"); + + if (repo_oid_to_algop(the_repository, &oid, to, &mapped_oid)) + return error("unable to map parent %s in commit object", + oid_to_hex(&oid)); + + strbuf_addf(out, "parent %s\n", oid_to_hex(&mapped_oid)); + bufptr = p + 1; + } + strbuf_add(out, bufptr, tail - bufptr); + return 0; +} + int convert_object_file(struct strbuf *outbuf, const struct git_hash_algo *from, const struct git_hash_algo *to, @@ -150,13 +192,15 @@ int convert_object_file(struct strbuf *outbuf, BUG("Refusing noop object file conversion"); switch (type) { + case OBJ_COMMIT: + ret = convert_commit_object(outbuf, from, to, buf, len); + break; case OBJ_TREE: ret = convert_tree_object(outbuf, from, to, buf, len); break; case OBJ_TAG: ret = convert_tag_object(outbuf, from, to, buf, len); break; - case OBJ_COMMIT: default: /* Not implemented yet, so fail. */ ret = -1; From patchwork Mon Oct 2 02:40:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13405538 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E63FAE784A6 for ; Mon, 2 Oct 2023 02:41:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235533AbjJBClb (ORCPT ); Sun, 1 Oct 2023 22:41:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38976 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235425AbjJBClO (ORCPT ); Sun, 1 Oct 2023 22:41:14 -0400 Received: from mail-io1-xd30.google.com (mail-io1-xd30.google.com [IPv6:2607:f8b0:4864:20::d30]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 948CF1AE for ; Sun, 1 Oct 2023 19:40:56 -0700 (PDT) Received: by mail-io1-xd30.google.com with SMTP id ca18e2360f4ac-79fd60f40ebso60584139f.1 for ; Sun, 01 Oct 2023 19:40:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696214456; x=1696819256; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=AK/zLNASM4NMsxXUNo/o2FZi7vdcD789Xby5LHgAm5U=; b=P33xejOGwnr30LxIuS0Qccz+r8b8AhWbZ57QBzaAYjOZ2SuC856ZXFgmlDiE1ltg3D gFn6YTMebvXg45JJDtYdN+XiMwtbMWgbf7w4Os1Hp3jdWVElRbgB8gU3CIM8tP93/X7B j39xpYerJj7ON+Zq8BvbAyJz6AF6+zSycn71yND0DgttrZnsBlaXYQHARCycLtn9efGO CQLY8Iw3veeGvHEHQy6far4pn4aCBG2mW+yRtYXIAtli6oQmNR5juWZMn6iaNSTWE/o1 acfSJw6nPTmGGgB9x6jeBTfN+SvF58XKPVX1xwo2mw+cvjR3+I9/Q+RqEOeJ6fBhPO04 h/dw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696214456; x=1696819256; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AK/zLNASM4NMsxXUNo/o2FZi7vdcD789Xby5LHgAm5U=; b=Z5TojZ/xZzDpG/S2nCwuP6aNoxPvPcLBxdlJql9DgOnAOt0e+q/7ZYdb6IR6nKNQZe mCTEwTs0z8fOQ/tuke31YC1uYBJi9zqzTfzf98evWl7XOK6gEXa8CHx0jeMzco+ThG49 2hsLyAtmKbRfpf9uunnQSGtmE3b0jSoe7LFJGk9yxxM4+5lBtvfjXpSoCEhCN/PAtmxr Y4IoqtQAu9U5cWUL+EpxXgN0jf/FyNPo2opBf0LyNbfmtySkOGr/tl2yCxKmgn/8y9Ba GgKHS+R/IQ0M3D9lWd2YPRdYjFT4dpRi+yFPLneOGo+RUGCQvfF8TatNKZybmPUyXfCp SI8A== X-Gm-Message-State: AOJu0YxMvGl9HgCnVZmq4mohGbOr85w0VlTjIdWrePsd/9cVf78ePmzQ IiuNjE/M3/k+pl72kENTVjg= X-Google-Smtp-Source: AGHT+IFPRUsi71V0djRGPFXsumG2NBynXmv7GJ1DawCQGQvFaUmO/mjF620UlBXpgxI4/4T6KtESsg== X-Received: by 2002:a5e:a519:0:b0:79f:b4f1:186a with SMTP id 25-20020a5ea519000000b0079fb4f1186amr8714929iog.1.1696214455890; Sun, 01 Oct 2023 19:40:55 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id v6-20020a056602058600b0079f9f7a2565sm6346113iox.38.2023.10.01.19.40.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Oct 2023 19:40:55 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , Eric Sunshine , "Eric W. Biederman" Subject: [PATCH v2 19/30] object-file-convert: convert commits that embed signed tags Date: Sun, 1 Oct 2023 21:40:23 -0500 Message-Id: <20231002024034.2611-19-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> References: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "Eric W. Biederman" As mentioned in the hash function transition plan commit mergetag lines need to be handled. The commit mergetag lines embed an entire tag object in a commit object. Keep the implementation sane if not fast by unembedding the tag object, converting the tag object, and embedding the new tag object, in the new commit object. In the long run I don't expect any other approach is maintainable, as tag objects may be extended in ways that require additional translation. To keep the implementation of convert_commit_object maintainable I have modified convert_commit_object to process the lines in any order, and to fail on unknown lines. We can't know ahead of time if a new line might embed something that needs translation or not so it is better to fail and require the code to be updated instead of silently mistranslating objects. Signed-off-by: Eric W. Biederman --- object-file-convert.c | 104 +++++++++++++++++++++++++++++++++--------- 1 file changed, 82 insertions(+), 22 deletions(-) diff --git a/object-file-convert.c b/object-file-convert.c index 0da081104ed4..4f6189095be8 100644 --- a/object-file-convert.c +++ b/object-file-convert.c @@ -146,35 +146,95 @@ static int convert_commit_object(struct strbuf *out, const int tree_entry_len = from->hexsz + 5; const int parent_entry_len = from->hexsz + 7; struct object_id oid, mapped_oid; - const char *p; + const char *p, *eol; tail += size; - if (tail <= bufptr + tree_entry_len + 1 || memcmp(bufptr, "tree ", 5) || - bufptr[tree_entry_len] != '\n') - return error("bogus commit object"); - if (parse_oid_hex_algop(bufptr + 5, &oid, &p, from) < 0) - return error("bad tree pointer"); - if (repo_oid_to_algop(the_repository, &oid, to, &mapped_oid)) - return error("unable to map tree %s in commit object", - oid_to_hex(&oid)); - strbuf_addf(out, "tree %s\n", oid_to_hex(&mapped_oid)); - bufptr = p + 1; + while ((bufptr < tail) && (*bufptr != '\n')) { + eol = memchr(bufptr, '\n', tail - bufptr); + if (!eol) + return error(_("bad %s in commit"), "line"); + + if (((bufptr + 5) < eol) && !memcmp(bufptr, "tree ", 5)) + { + if (((bufptr + tree_entry_len) != eol) || + parse_oid_hex_algop(bufptr + 5, &oid, &p, from) || + (p != eol)) + return error(_("bad %s in commit"), "tree"); + + if (repo_oid_to_algop(the_repository, &oid, to, &mapped_oid)) + return error(_("unable to map %s %s in commit object"), + "tree", oid_to_hex(&oid)); + strbuf_addf(out, "tree %s\n", oid_to_hex(&mapped_oid)); + } + else if (((bufptr + 7) < eol) && !memcmp(bufptr, "parent ", 7)) + { + if (((bufptr + parent_entry_len) != eol) || + parse_oid_hex_algop(bufptr + 7, &oid, &p, from) || + (p != eol)) + return error(_("bad %s in commit"), "parent"); - while (bufptr + parent_entry_len < tail && !memcmp(bufptr, "parent ", 7)) { - if (tail <= bufptr + parent_entry_len + 1 || - parse_oid_hex_algop(bufptr + 7, &oid, &p, from) || - *p != '\n') - return error("bad parents in commit"); + if (repo_oid_to_algop(the_repository, &oid, to, &mapped_oid)) + return error(_("unable to map %s %s in commit object"), + "parent", oid_to_hex(&oid)); - if (repo_oid_to_algop(the_repository, &oid, to, &mapped_oid)) - return error("unable to map parent %s in commit object", - oid_to_hex(&oid)); + strbuf_addf(out, "parent %s\n", oid_to_hex(&mapped_oid)); + } + else if (((bufptr + 9) < eol) && !memcmp(bufptr, "mergetag ", 9)) + { + struct strbuf tag = STRBUF_INIT, new_tag = STRBUF_INIT; - strbuf_addf(out, "parent %s\n", oid_to_hex(&mapped_oid)); - bufptr = p + 1; + /* Recover the tag object from the mergetag */ + strbuf_add(&tag, bufptr + 9, (eol - (bufptr + 9)) + 1); + + bufptr = eol + 1; + while ((bufptr < tail) && (*bufptr == ' ')) { + eol = memchr(bufptr, '\n', tail - bufptr); + if (!eol) { + strbuf_release(&tag); + return error(_("bad %s in commit"), "mergetag continuation"); + } + strbuf_add(&tag, bufptr + 1, (eol - (bufptr + 1)) + 1); + bufptr = eol + 1; + } + + /* Compute the new tag object */ + if (convert_tag_object(&new_tag, from, to, tag.buf, tag.len)) { + strbuf_release(&tag); + strbuf_release(&new_tag); + return -1; + } + + /* Write the new mergetag */ + strbuf_addstr(out, "mergetag"); + strbuf_add_lines(out, " ", new_tag.buf, new_tag.len); + strbuf_release(&tag); + strbuf_release(&new_tag); + } + else if (((bufptr + 7) < tail) && !memcmp(bufptr, "author ", 7)) + strbuf_add(out, bufptr, (eol - bufptr) + 1); + else if (((bufptr + 10) < tail) && !memcmp(bufptr, "committer ", 10)) + strbuf_add(out, bufptr, (eol - bufptr) + 1); + else if (((bufptr + 9) < tail) && !memcmp(bufptr, "encoding ", 9)) + strbuf_add(out, bufptr, (eol - bufptr) + 1); + else if (((bufptr + 6) < tail) && !memcmp(bufptr, "gpgsig", 6)) + strbuf_add(out, bufptr, (eol - bufptr) + 1); + else { + /* Unknown line fail it might embed an oid */ + return -1; + } + /* Consume any trailing continuation lines */ + bufptr = eol + 1; + while ((bufptr < tail) && (*bufptr == ' ')) { + eol = memchr(bufptr, '\n', tail - bufptr); + if (!eol) + return error(_("bad %s in commit"), "continuation"); + strbuf_add(out, bufptr, (eol - bufptr) + 1); + bufptr = eol + 1; + } } - strbuf_add(out, bufptr, tail - bufptr); + if (bufptr < tail) + strbuf_add(out, bufptr, tail - bufptr); return 0; } From patchwork Mon Oct 2 02:40:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13405540 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EA6ABE784A7 for ; Mon, 2 Oct 2023 02:41:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235460AbjJBCle (ORCPT ); Sun, 1 Oct 2023 22:41:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58298 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235479AbjJBClP (ORCPT ); Sun, 1 Oct 2023 22:41:15 -0400 Received: from mail-io1-xd32.google.com (mail-io1-xd32.google.com [IPv6:2607:f8b0:4864:20::d32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A56E31B4 for ; Sun, 1 Oct 2023 19:40:57 -0700 (PDT) Received: by mail-io1-xd32.google.com with SMTP id ca18e2360f4ac-79f82b26abfso562693039f.1 for ; Sun, 01 Oct 2023 19:40:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696214457; x=1696819257; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=i2Tpg3bVdy9YaIiHJYFcCNoaRuC55AkyHBlJKIbRy1I=; b=Hk4ptNPGn/HdGrowT70PuVWTcltRGQp3mGLyFIXDSLq/crU8nuMp0GgRFw/sDtgEr2 TO9b97IwRP1nqgYFOUINiqcXG5kMhd8XAWuo63IuZDCDfaGEX77bwZludMF/SJ78TS+m qfq3wMktEJd1uBW44uStF4JJPIVVIITQ3VjQsZdWTibeixqH8TPG5Yw043B91VtP8ZCX zFeUOrkunzu/vYR9RdF+KBlArD+SjRRQnOzA4FdlXFbfOofeP0VkBnueixFKaUwOWYuE /2utLFdf3KC4+nLh2XNn4F2tTyI3LLctnEaYFdt5c9zOZtzg7HIUeMD9ixyLWKZ0wOjO hPag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696214457; x=1696819257; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=i2Tpg3bVdy9YaIiHJYFcCNoaRuC55AkyHBlJKIbRy1I=; b=osn7orJXYsFdYcGcWPnrwEeuDedTAhU1XOFhepZ5IJ8gcipenlpwhTzmV7q7b2Nd/0 j++OYaoom2YkybXmehFXCEr4rzdqFcFzicRC+4keejBnM6rovdb/QyqjI1fg7I+ylSAb Nzd5mhEPrmCCacqm4cr8V0oNz0r2nt0xF4NIyWNoUVa1eLKDKLbTNMzNUGXVfaY9zlxe D3rsmB/HWZZC+jQy45EHFBrngY7n+r2ZT8McAR80R8SWPi/YO4a8vNk2Dlt3ZjIragQB EVogm0Exry5/7W6Yoc5QclwKg4eHzjPqY4NOx31S0vn9ADf7XVSW2QvHrF5lBhvjVEhb 5Yzw== X-Gm-Message-State: AOJu0YyPQj4CftEJfHnoe5VMacxdFEjoKTytWX1ZSaLGX3ihmV6yB2qB sfpGqpYnXiEw/MwHl9CyVvY= X-Google-Smtp-Source: AGHT+IFta2hh8MA0cRZLqHhmemGLv8oENMUpv3MOWNzVo5jWOYW+MAvyjs/cfIEw5tDsGpH/VmLTVA== X-Received: by 2002:a5e:a903:0:b0:79f:97b6:76de with SMTP id c3-20020a5ea903000000b0079f97b676demr11495924iod.3.1696214456806; Sun, 01 Oct 2023 19:40:56 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id v6-20020a056602058600b0079f9f7a2565sm6346113iox.38.2023.10.01.19.40.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Oct 2023 19:40:56 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , Eric Sunshine , "Eric W. Biederman" Subject: [PATCH v2 20/30] object-file: update object_info_extended to reencode objects Date: Sun, 1 Oct 2023 21:40:24 -0500 Message-Id: <20231002024034.2611-20-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> References: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "Eric W. Biederman" oid_object_info_extended is updated to detect an oid encoding that does not match the current repository, use repo_oid_to_algop to find the correspoding oid in the current repository and to return the data for the oid. Signed-off-by: "Eric W. Biederman" --- object-file.c | 91 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 91 insertions(+) diff --git a/object-file.c b/object-file.c index 820810a5f4b3..b2d43d009898 100644 --- a/object-file.c +++ b/object-file.c @@ -1662,10 +1662,101 @@ static int do_oid_object_info_extended(struct repository *r, return 0; } +static int oid_object_info_convert(struct repository *r, + const struct object_id *input_oid, + struct object_info *input_oi, unsigned flags) +{ + const struct git_hash_algo *input_algo = &hash_algos[input_oid->algo]; + int do_die = flags & OBJECT_INFO_DIE_IF_CORRUPT; + struct strbuf type_name = STRBUF_INIT; + struct object_id oid, delta_base_oid; + struct object_info new_oi, *oi; + unsigned long size; + void *content; + int ret; + + if (repo_oid_to_algop(r, input_oid, the_hash_algo, &oid)) { + if (do_die) + die(_("missing mapping of %s to %s"), + oid_to_hex(input_oid), the_hash_algo->name); + return -1; + } + + /* Is new_oi needed? */ + oi = input_oi; + if (input_oi && (input_oi->delta_base_oid || input_oi->sizep || + input_oi->contentp)) { + new_oi = *input_oi; + /* Does delta_base_oid need to be converted? */ + if (input_oi->delta_base_oid) + new_oi.delta_base_oid = &delta_base_oid; + /* Will the attributes differ when converted? */ + if (input_oi->sizep || input_oi->contentp) { + new_oi.contentp = &content; + new_oi.sizep = &size; + new_oi.type_name = &type_name; + } + oi = &new_oi; + } + + ret = oid_object_info_extended(r, &oid, oi, flags); + if (ret) + return -1; + if (oi == input_oi) + return ret; + + if (new_oi.contentp) { + struct strbuf outbuf = STRBUF_INIT; + enum object_type type; + + type = type_from_string_gently(type_name.buf, type_name.len, + !do_die); + if (type == -1) + return -1; + if (type != OBJ_BLOB) { + ret = convert_object_file(&outbuf, + the_hash_algo, input_algo, + content, size, type, !do_die); + if (ret == -1) + return -1; + free(content); + size = outbuf.len; + content = strbuf_detach(&outbuf, NULL); + } + if (input_oi->sizep) + *input_oi->sizep = size; + if (input_oi->contentp) + *input_oi->contentp = content; + else + free(content); + if (input_oi->type_name) + *input_oi->type_name = type_name; + else + strbuf_release(&type_name); + } + if (new_oi.delta_base_oid == &delta_base_oid) { + if (repo_oid_to_algop(r, &delta_base_oid, input_algo, + input_oi->delta_base_oid)) { + if (do_die) + die(_("missing mapping of %s to %s"), + oid_to_hex(&delta_base_oid), + input_algo->name); + return -1; + } + } + input_oi->whence = new_oi.whence; + input_oi->u = new_oi.u; + return ret; +} + int oid_object_info_extended(struct repository *r, const struct object_id *oid, struct object_info *oi, unsigned flags) { int ret; + + if (oid->algo && (hash_algo_by_ptr(r->hash_algo) != oid->algo)) + return oid_object_info_convert(r, oid, oi, flags); + obj_read_lock(); ret = do_oid_object_info_extended(r, oid, oi, flags); obj_read_unlock(); From patchwork Mon Oct 2 02:40:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13405541 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4D7E9E784A6 for ; Mon, 2 Oct 2023 02:41:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235495AbjJBClf (ORCPT ); Sun, 1 Oct 2023 22:41:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58468 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235486AbjJBClP (ORCPT ); Sun, 1 Oct 2023 22:41:15 -0400 Received: from mail-io1-xd33.google.com (mail-io1-xd33.google.com [IPv6:2607:f8b0:4864:20::d33]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6C572D9 for ; Sun, 1 Oct 2023 19:40:58 -0700 (PDT) Received: by mail-io1-xd33.google.com with SMTP id ca18e2360f4ac-79f82b26abfso562693439f.1 for ; Sun, 01 Oct 2023 19:40:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696214457; x=1696819257; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=AF5mnlyuAtBynDxzlMk6AzIHona7RoAHcXNCm+fnaGE=; b=NxJPIa1sgXrvJ3wAcu7086OtmwgUkdbFRKzqV8WIEd2ZbzrPkXD2js+3jBorp2RfUy zl/UP43ENGKzbpBT0WXUGx/8Hg02GMsd2RqH8pMh5Ooai0eOCd/D/Jz9FI7XK8O4XTLa neLyMJwAxrRXAE4SooJKhRhAellkBbihkE+N+OuIRa0T0naDRDyuYZGRA6rDKGptPtwq v+zffUzw0yKpHKv10K7gtkK0P1Q/bXhkcnWb1dPecMZkt3ElAz2rO68wlhO1jKhAUnB/ LXdJlBG76LBS+UyBGAKB4M/lpf3UFO41YYR8ArcP8+yJMteieuHg3fCeoa1/piSKu5uh W1RQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696214457; x=1696819257; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AF5mnlyuAtBynDxzlMk6AzIHona7RoAHcXNCm+fnaGE=; b=seRz0NBfKQy2J2nI1gmgSKA7bdbsUwFYP43f+ySK6nS6MfX3WWLd0EKixXybSa3c4X +z+caOOrvRymcv5JMaDlhdO8eGBdjGsVG2Zz0yzIrzij/Q2y17h/MkFwSpxr4pfub4H/ aPQXMuLglmkYu9CkCATF2fqIvQ5qcLvTQryJP4TAKCBiwgc7DL5XKcB2yNedZ+MpGOKw 2i/xV0eYnRuOeYgn5fLD/GxdRJAOn5rtzRJwBR7cevcB5d5VjYk5PQKyTg0tpu+PZLPU /xQc9kBHeL7pRYh8BSHP/CZviUghUkU6uUMINJgOdmmHf1Ev/frNAUI+Y28FAqWY/Zsh v1Ag== X-Gm-Message-State: AOJu0Yz2rXYxwRQ2IlSCJHfOSVBb7HnGUkGE0j7tpnrE+nIbIwy413el tZjmP+a6ER04XUkyiKXZjlg= X-Google-Smtp-Source: AGHT+IGQzZyjTlBo8/FsPg+1MHLlov9FIWJLQNeJXXEZ8rOhm+FNKnbkrE3ZkFWNObM0Jab/VY4/+A== X-Received: by 2002:a6b:a12:0:b0:792:792e:6619 with SMTP id z18-20020a6b0a12000000b00792792e6619mr11543582ioi.2.1696214457687; Sun, 01 Oct 2023 19:40:57 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id v6-20020a056602058600b0079f9f7a2565sm6346113iox.38.2023.10.01.19.40.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Oct 2023 19:40:57 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , Eric Sunshine , "Eric W. Biederman" Subject: [PATCH v2 21/30] repository: implement extensions.compatObjectFormat Date: Sun, 1 Oct 2023 21:40:25 -0500 Message-Id: <20231002024034.2611-21-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> References: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "brian m. carlson" Add a configuration option to enable updating and reading from compatibility hash maps when git accesses the reposotiry. Call the helper function repo_set_compat_hash_algo with the value that compatObjectFormat is set to. Signed-off-by: "Eric W. Biederman" --- Documentation/config/extensions.txt | 12 ++++++++++++ repository.c | 2 +- setup.c | 23 +++++++++++++++++++++-- setup.h | 1 + 4 files changed, 35 insertions(+), 3 deletions(-) diff --git a/Documentation/config/extensions.txt b/Documentation/config/extensions.txt index bccaec7a9636..9f72e6d9f4f1 100644 --- a/Documentation/config/extensions.txt +++ b/Documentation/config/extensions.txt @@ -7,6 +7,18 @@ Note that this setting should only be set by linkgit:git-init[1] or linkgit:git-clone[1]. Trying to change it after initialization will not work and will produce hard-to-diagnose issues. +extensions.compatObjectFormat:: + + Specify a compatitbility hash algorithm to use. The acceptable values + are `sha1` and `sha256`. The value specified must be different from the + value of extensions.objectFormat. This allows client level + interoperability between git repositories whose objectFormat matches + this compatObjectFormat. In particular when fully implemented the + pushes and pulls from a repository in whose objectFormat matches + compatObjectFormat. As well as being able to use oids encoded in + compatObjectFormat in addition to oids encoded with objectFormat to + locally specify objects. + extensions.worktreeConfig:: If enabled, then worktrees will load config settings from the `$GIT_DIR/config.worktree` file in addition to the diff --git a/repository.c b/repository.c index 6214f61cf4e7..9d91536b613b 100644 --- a/repository.c +++ b/repository.c @@ -194,7 +194,7 @@ int repo_init(struct repository *repo, goto error; repo_set_hash_algo(repo, format.hash_algo); - repo_set_compat_hash_algo(repo, GIT_HASH_UNKNOWN); + repo_set_compat_hash_algo(repo, format.compat_hash_algo); repo->repository_format_worktree_config = format.worktree_config; /* take ownership of format.partial_clone */ diff --git a/setup.c b/setup.c index aa8bf5da5226..85259a259be3 100644 --- a/setup.c +++ b/setup.c @@ -590,6 +590,25 @@ static enum extension_result handle_extension(const char *var, "extensions.objectformat", value); data->hash_algo = format; return EXTENSION_OK; + } else if (!strcmp(ext, "compatobjectformat")) { + struct string_list_item *item; + int format; + + if (!value) + return config_error_nonbool(var); + format = hash_algo_by_name(value); + if (format == GIT_HASH_UNKNOWN) + return error(_("invalid value for '%s': '%s'"), + "extensions.compatobjectformat", value); + /* For now only support compatObjectFormat being specified once. */ + for_each_string_list_item(item, &data->v1_only_extensions) { + if (!strcmp(item->string, "compatobjectformat")) + return error(_("'%s' already specified as '%s'"), + "extensions.compatobjectformat", + hash_algos[data->compat_hash_algo].name); + } + data->compat_hash_algo = format; + return EXTENSION_OK; } return EXTENSION_UNKNOWN; } @@ -1565,7 +1584,7 @@ const char *setup_git_directory_gently(int *nongit_ok) if (startup_info->have_repository) { repo_set_hash_algo(the_repository, repo_fmt.hash_algo); repo_set_compat_hash_algo(the_repository, - GIT_HASH_UNKNOWN); + repo_fmt.compat_hash_algo); the_repository->repository_format_worktree_config = repo_fmt.worktree_config; /* take ownership of repo_fmt.partial_clone */ @@ -1659,7 +1678,7 @@ void check_repository_format(struct repository_format *fmt) check_repository_format_gently(get_git_dir(), fmt, NULL); startup_info->have_repository = 1; repo_set_hash_algo(the_repository, fmt->hash_algo); - repo_set_compat_hash_algo(the_repository, GIT_HASH_UNKNOWN); + repo_set_compat_hash_algo(the_repository, fmt->compat_hash_algo); the_repository->repository_format_worktree_config = fmt->worktree_config; the_repository->repository_format_partial_clone = diff --git a/setup.h b/setup.h index 58fd2605dd26..5d678ceb8caa 100644 --- a/setup.h +++ b/setup.h @@ -86,6 +86,7 @@ struct repository_format { int worktree_config; int is_bare; int hash_algo; + int compat_hash_algo; int sparse_index; char *work_tree; struct string_list unknown_extensions; From patchwork Mon Oct 2 02:40:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13405542 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC591E784A7 for ; Mon, 2 Oct 2023 02:41:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235496AbjJBClh (ORCPT ); Sun, 1 Oct 2023 22:41:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58474 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235490AbjJBClP (ORCPT ); Sun, 1 Oct 2023 22:41:15 -0400 Received: from mail-io1-xd43.google.com (mail-io1-xd43.google.com [IPv6:2607:f8b0:4864:20::d43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 47A53CC4 for ; Sun, 1 Oct 2023 19:40:59 -0700 (PDT) Received: by mail-io1-xd43.google.com with SMTP id ca18e2360f4ac-79faf4210b2so459447739f.2 for ; Sun, 01 Oct 2023 19:40:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696214458; x=1696819258; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Ah9lMgvMVvnNRn//l/9eCmgPadLsslMM4KL7LWB14S4=; b=XoFr9vzMvmHovO4JwTJs637algt7T1k5JYWys4uPIgr2h7IzCTTeve6aJIkuVSI5ed hXAqnplg2skbQbsgnCBcGGWQ2YbiMSg6krbcRgoreZ2DPe8elJhvYYmf0RzbTO5p4k0j owmnYO2rZcszaUT7V/whO2WvZk2BGgrG3lBIhsUFABxgrJ1RZp7ZCByBah5Wb77xy/Z6 X+bzS9nlGJgAFY+Y3Cxj5geOAsnSrpxFpIr7yMzjV7cljJBctSOCe1x8H2zC0WzLf6lx LCclrjQJTVW9hOgKfbUIUD7GazTMzNSQaox5P+hWf/Z7v6/X1p4n9c2gzPQ+m9qO6RTV wGIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696214458; x=1696819258; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Ah9lMgvMVvnNRn//l/9eCmgPadLsslMM4KL7LWB14S4=; b=rnSw89CKsen4LqE0z9Iodg+qSSuvGC8bf+ZuuSyxAU29aN7+tXvPrs+/G7oTxUMbDu oPINYorlQ3ZmohJkmcz4ZaHEiMRbv7pR5eQurmfIyHcl5u+zagh1S2OWshb5LlZbniSE XbNUVzadbCC7dOr7ZCEL0wFOmhnrgC3iwlBdNqHJ91v62dBaLJKOvmHg9oYnEn53e9QK bFvM7SdMafmDDQ5AFs7+EAaB0lZX8AZyJbhLdfagZBKF2SLdNlyZTrA8Iu5lgdkZRRuo xtQHSWsyimBR4cf1pH5/4cCHFDJDUo39GRio1LtEYL1oIkk+kKsWjw4JpYeu/wKYLUvz IrEA== X-Gm-Message-State: AOJu0YwpBCQuaA9ohJ9tGJkIgehUejkJTb28MQREdBuTB2L+gWqSRjUu jm3DschE0Qp7GlfowJTohfFXI01rt5v5wxJM X-Google-Smtp-Source: AGHT+IHp/NiOmqU6UF4EK3Kmu2gk+VGwaa6BzcqBIOUS9NbPL2rRHfTdGw45X/z240iG5FBiDXUsSA== X-Received: by 2002:a5e:aa09:0:b0:783:4bc6:636e with SMTP id s9-20020a5eaa09000000b007834bc6636emr10314707ioe.21.1696214458611; Sun, 01 Oct 2023 19:40:58 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id v6-20020a056602058600b0079f9f7a2565sm6346113iox.38.2023.10.01.19.40.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Oct 2023 19:40:58 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , Eric Sunshine , "Eric W. Biederman" Subject: [PATCH v2 22/30] rev-parse: add an --output-object-format parameter Date: Sun, 1 Oct 2023 21:40:26 -0500 Message-Id: <20231002024034.2611-22-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> References: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "Eric W. Biederman" The new --output-object-format parameter returns the oid in the specified format. This is a generally useful plumbing facility. It is useful for writing test cases and for directly querying the translation maps. Signed-off-by: "Eric W. Biederman" --- Documentation/git-rev-parse.txt | 12 ++++++++++++ builtin/rev-parse.c | 23 +++++++++++++++++++++++ 2 files changed, 35 insertions(+) diff --git a/Documentation/git-rev-parse.txt b/Documentation/git-rev-parse.txt index f26a7591e373..f0f9021f2a5a 100644 --- a/Documentation/git-rev-parse.txt +++ b/Documentation/git-rev-parse.txt @@ -159,6 +159,18 @@ for another option. unfortunately named tag "master"), and show them as full refnames (e.g. "refs/heads/master"). +--output-object-format=(sha1|sha256|storage):: + + Allow oids to be input from any object format that the current + repository supports. + + Specifying "sha1" translates if necessary and returns a sha1 oid. + + Specifying "sha256" translates if necessary and returns a sha256 oid. + + Specifying "storage" translates if necessary and returns an oid in + encoded in the storage hash algorithm. + Options for Objects ~~~~~~~~~~~~~~~~~~~ diff --git a/builtin/rev-parse.c b/builtin/rev-parse.c index 43e96765400c..0ef3e658cc5b 100644 --- a/builtin/rev-parse.c +++ b/builtin/rev-parse.c @@ -25,6 +25,7 @@ #include "submodule.h" #include "commit-reach.h" #include "shallow.h" +#include "object-file-convert.h" #define DO_REVS 1 #define DO_NOREV 2 @@ -675,6 +676,8 @@ static void print_path(const char *path, const char *prefix, enum format_type fo int cmd_rev_parse(int argc, const char **argv, const char *prefix) { int i, as_is = 0, verify = 0, quiet = 0, revs_count = 0, type = 0; + const struct git_hash_algo *output_algo = NULL; + const struct git_hash_algo *compat = NULL; int did_repo_setup = 0; int has_dashdash = 0; int output_prefix = 0; @@ -746,6 +749,7 @@ int cmd_rev_parse(int argc, const char **argv, const char *prefix) prepare_repo_settings(the_repository); the_repository->settings.command_requires_full_index = 0; + compat = the_repository->compat_hash_algo; } if (!strcmp(arg, "--")) { @@ -833,6 +837,22 @@ int cmd_rev_parse(int argc, const char **argv, const char *prefix) flags |= GET_OID_QUIETLY; continue; } + if (opt_with_value(arg, "--output-object-format", &arg)) { + if (!arg) + die(_("no object format specified")); + if (!strcmp(arg, the_hash_algo->name) || + !strcmp(arg, "storage")) { + flags |= GET_OID_HASH_ANY; + output_algo = the_hash_algo; + continue; + } + else if (compat && !strcmp(arg, compat->name)) { + flags |= GET_OID_HASH_ANY; + output_algo = compat; + continue; + } + else die(_("unsupported object format: %s"), arg); + } if (opt_with_value(arg, "--short", &arg)) { filter &= ~(DO_FLAGS|DO_NOREV); verify = 1; @@ -1083,6 +1103,9 @@ int cmd_rev_parse(int argc, const char **argv, const char *prefix) } if (!get_oid_with_context(the_repository, name, flags, &oid, &unused)) { + if (output_algo) + repo_oid_to_algop(the_repository, &oid, + output_algo, &oid); if (verify) revs_count++; else From patchwork Mon Oct 2 02:40:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13405543 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E9D11E784A6 for ; Mon, 2 Oct 2023 02:41:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235498AbjJBCli (ORCPT ); Sun, 1 Oct 2023 22:41:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58498 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235493AbjJBClP (ORCPT ); Sun, 1 Oct 2023 22:41:15 -0400 Received: from mail-io1-xd29.google.com (mail-io1-xd29.google.com [IPv6:2607:f8b0:4864:20::d29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 22BD2DD for ; Sun, 1 Oct 2023 19:41:00 -0700 (PDT) Received: by mail-io1-xd29.google.com with SMTP id ca18e2360f4ac-7a25040faffso218162039f.1 for ; Sun, 01 Oct 2023 19:41:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696214459; x=1696819259; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=qNlEB4NzHehHwb5iJrKXvrOJbxoQMMja8pt5tTtXO1Q=; b=YbDYTSJogYvNx1oRdDY88y98ldblduoOUXnJwtMPLKhBRucyYJeQFq4aKIUSxEjqai My9yMR7DLqLj2vaQr0D3PmqJ3kJl+uDe7HHV/2JJhGy6PbHp/e/oA0P7ujirX98kbPrB HJjWI0dtftjY3kqhbBcH0wIeT+Lm/HXILpKRoig4AWjnLyxlv/IHFsbYm9YhE2dKApFE FR3tC0P5Ii9auCKuZrqQBGKymG/3nU0d/p9o9l4FztYUhYVEB2NCP8leslEQtkLmwIOi O+8nOxBKiPFGnLsftk73KbOZLIYDHu6bDgUFeriv4Mj3gVmXgbcx9aWdrLF6iH/5/AnU nqXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696214459; x=1696819259; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qNlEB4NzHehHwb5iJrKXvrOJbxoQMMja8pt5tTtXO1Q=; b=lI0Up3w2Gawi8IGwBoSj5XSWNQWlmWfstF4yXgB9ABeIDqihdqexQ1/beN4lgLFWos bXMoFaf7kewU8NdCKRAqc1sZGRNa0x20vWVol+IpE9CZkPfkoJY6h0v2EHafD2Tp+Bc2 zPI3O4Ez0Hi83HUA7je43GLKrNIWMLAt9rkVH5RCuP6Zluw7uL5UmQi/VhH6kjA23upf KVqU/56pI8G7rirMwK3yfG1Mmk6zl564KAg+/Y9YG+I+3BeCV5VQm6d1Ua0xAFLI0jLE r8EZeG5MNXMHVr6X6YSFNpQhWJK7hJCfL+H7KrWD0lefAlVhzJp2xzhbKpDZCb/Zlk7Z UWLw== X-Gm-Message-State: AOJu0YxOIMfNEERrecLin6VBH7bWdy11uPhfEB8y3xGg4UcJTQFXIPDl VyDdfDWt2ngU40yj/PFad1M= X-Google-Smtp-Source: AGHT+IEt6LFJRKC+RVrPfndFll2ron/E6h1d7stLk3XgZhlhpcV7qwSEH+TIG/BoKpL2sCASGT+qcw== X-Received: by 2002:a6b:f108:0:b0:79f:d195:5384 with SMTP id e8-20020a6bf108000000b0079fd1955384mr12997043iog.17.1696214459420; Sun, 01 Oct 2023 19:40:59 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id v6-20020a056602058600b0079f9f7a2565sm6346113iox.38.2023.10.01.19.40.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Oct 2023 19:40:59 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , Eric Sunshine , "Eric W. Biederman" Subject: [PATCH v2 23/30] builtin/cat-file: let the oid determine the output algorithm Date: Sun, 1 Oct 2023 21:40:27 -0500 Message-Id: <20231002024034.2611-23-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> References: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "Eric W. Biederman" Use GET_OID_HASH_ANY when calling get_oid_with_context. This implements the semi-obvious behaviour that specifying a sha1 oid shows the output for a sha1 encoded object, and specifying a sha256 oid shows the output for a sha256 encoded object. This is useful for testing the the conversion of an object to an equivalent object encoded with a different hash function. Signed-off-by: "Eric W. Biederman" --- builtin/cat-file.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/builtin/cat-file.c b/builtin/cat-file.c index 694c8538df2f..e615d1f8e0da 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -107,7 +107,10 @@ static int cat_one_file(int opt, const char *exp_type, const char *obj_name, struct object_info oi = OBJECT_INFO_INIT; struct strbuf sb = STRBUF_INIT; unsigned flags = OBJECT_INFO_LOOKUP_REPLACE; - unsigned get_oid_flags = GET_OID_RECORD_PATH | GET_OID_ONLY_TO_DIE; + unsigned get_oid_flags = + GET_OID_RECORD_PATH | + GET_OID_ONLY_TO_DIE | + GET_OID_HASH_ANY; const char *path = force_path; const int opt_cw = (opt == 'c' || opt == 'w'); if (!path && opt_cw) @@ -223,7 +226,8 @@ static int cat_one_file(int opt, const char *exp_type, const char *obj_name, &size); const char *target; if (!skip_prefix(buffer, "object ", &target) || - get_oid_hex(target, &blob_oid)) + get_oid_hex_algop(target, &blob_oid, + &hash_algos[oid.algo])) die("%s not a valid tag", oid_to_hex(&oid)); free(buffer); } else @@ -512,7 +516,9 @@ static void batch_one_object(const char *obj_name, struct expand_data *data) { struct object_context ctx; - int flags = opt->follow_symlinks ? GET_OID_FOLLOW_SYMLINKS : 0; + int flags = + GET_OID_HASH_ANY | + (opt->follow_symlinks ? GET_OID_FOLLOW_SYMLINKS : 0); enum get_oid_result result; result = get_oid_with_context(the_repository, obj_name, From patchwork Mon Oct 2 02:40:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13405546 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2768AE784A7 for ; Mon, 2 Oct 2023 02:41:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235472AbjJBClx (ORCPT ); Sun, 1 Oct 2023 22:41:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35568 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235503AbjJBCl3 (ORCPT ); Sun, 1 Oct 2023 22:41:29 -0400 Received: from mail-io1-xd33.google.com (mail-io1-xd33.google.com [IPv6:2607:f8b0:4864:20::d33]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2096ACC8 for ; Sun, 1 Oct 2023 19:41:01 -0700 (PDT) Received: by mail-io1-xd33.google.com with SMTP id ca18e2360f4ac-79f9acc857cso518798439f.2 for ; Sun, 01 Oct 2023 19:41:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696214460; x=1696819260; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ctOzb5UNmN2By2/XBixPoWkARfwfWfm6uJO9oRBQccg=; b=VmdsiGArT01LgkjHeahwuSgw9qK7PwfrCT4+uScyba02itzpGEsVVKFDmKN+791YUa Oe6+mBpoXwE7JuzkrbV9MRhgRPuJTfxdqkNRxayRn5BC32bSDg3q8gmZckpAEXplYAHB tvWdJEtpHnkfOzQApcR6o4RJmW1CL7zidCys+1RWnnMn2vrZFYh1xeogFL1BiOm3Tn4g O75lmqSS6kCZ4lPoCiy3xE/kxeM45Ui73yszMJ4gjsNHNKuZ8LpdvGnZoWZ7MdCe6pfp S3gOYHu6hK8lRjS5HyfAtm3zWQi/KBd1RvX8A5VtCuyRyr53wXG0sLtZt4zcuOYth2Sk SB6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696214460; x=1696819260; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ctOzb5UNmN2By2/XBixPoWkARfwfWfm6uJO9oRBQccg=; b=GWf53ayAmyYHsg5d6Kk4OaIn5fODy2Begl2Bqm1IZ8eQRL9w+bRzu5jwltayu2Dehb /iwEq5V+TSlmsr5qSkm2Ow1qhLgpQF1IGTNEf0wV4zFpWL92I50seIvv35SzQ52hBnh8 aYy1OdGTvv0j1vqv4yXA4Ra2lHd7JJX8eZOiqPYadWBqKgsl18JwW8hfBGivO0sfQuzt IGnOKrqRixx4vsNSvE3IbXUXSin/UCUg4LE6e3ANRrR6K8OxZ8PApmObqPiTLeen1Ds6 ervSg/7mwzIC8cocVPmcXwZ0eur+8NIQHEoTZR4VFSlJcTn0epU8GH3AN5NeHW995aV8 GTVA== X-Gm-Message-State: AOJu0YyBsrzpQl9+H/pr9YqHExJ+yBbM+SFlojy4yr3EuHjkFmC+WY1B 82W7lkiRKz9fCJcKI9EWq3ggPo1v+WGcKA== X-Google-Smtp-Source: AGHT+IHkMKLg02G14omqTx0bmSq7zioM0mk2Xp2n2OaBDS4qXe5zXnBLcjYAg4Dxt91aGQypWOGIeg== X-Received: by 2002:a6b:7f42:0:b0:795:16b8:8601 with SMTP id m2-20020a6b7f42000000b0079516b88601mr10674845ioq.0.1696214460204; Sun, 01 Oct 2023 19:41:00 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id v6-20020a056602058600b0079f9f7a2565sm6346113iox.38.2023.10.01.19.40.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Oct 2023 19:40:59 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , Eric Sunshine , "Eric W. Biederman" Subject: [PATCH v2 24/30] tree-walk: init_tree_desc take an oid to get the hash algorithm Date: Sun, 1 Oct 2023 21:40:28 -0500 Message-Id: <20231002024034.2611-24-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> References: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "Eric W. Biederman" To make it possible for git ls-tree to display the tree encoded in the hash algorithm of the oid specified to git ls-tree, update init_tree_desc to take as a parameter the oid of the tree object. Update all callers of init_tree_desc and init_tree_desc_gently to pass the oid of the tree object. Use the oid of the tree object to discover the hash algorithm of the oid and store that hash algorithm in struct tree_desc. Use the hash algorithm in decode_tree_entry and update_tree_entry_internal to handle reading a tree object encoded in a hash algorithm that differs from the repositories hash algorithm. Signed-off-by: "Eric W. Biederman" --- archive.c | 3 ++- builtin/am.c | 6 +++--- builtin/checkout.c | 8 +++++--- builtin/clone.c | 2 +- builtin/commit.c | 2 +- builtin/grep.c | 8 ++++---- builtin/merge.c | 3 ++- builtin/pack-objects.c | 6 ++++-- builtin/read-tree.c | 2 +- builtin/stash.c | 5 +++-- cache-tree.c | 2 +- delta-islands.c | 2 +- diff-lib.c | 2 +- fsck.c | 6 ++++-- http-push.c | 2 +- list-objects.c | 2 +- match-trees.c | 4 ++-- merge-ort.c | 11 ++++++----- merge-recursive.c | 2 +- merge.c | 3 ++- pack-bitmap-write.c | 2 +- packfile.c | 3 ++- reflog.c | 2 +- revision.c | 4 ++-- tree-walk.c | 36 +++++++++++++++++++++--------------- tree-walk.h | 7 +++++-- tree.c | 2 +- walker.c | 2 +- 28 files changed, 80 insertions(+), 59 deletions(-) diff --git a/archive.c b/archive.c index ca11db185b15..b10269aee7be 100644 --- a/archive.c +++ b/archive.c @@ -339,7 +339,8 @@ int write_archive_entries(struct archiver_args *args, opts.src_index = args->repo->index; opts.dst_index = args->repo->index; opts.fn = oneway_merge; - init_tree_desc(&t, args->tree->buffer, args->tree->size); + init_tree_desc(&t, &args->tree->object.oid, + args->tree->buffer, args->tree->size); if (unpack_trees(1, &t, &opts)) return -1; git_attr_set_direction(GIT_ATTR_INDEX); diff --git a/builtin/am.c b/builtin/am.c index 8bde034fae68..4dfd714b910e 100644 --- a/builtin/am.c +++ b/builtin/am.c @@ -1991,8 +1991,8 @@ static int fast_forward_to(struct tree *head, struct tree *remote, int reset) opts.reset = reset ? UNPACK_RESET_PROTECT_UNTRACKED : 0; opts.preserve_ignored = 0; /* FIXME: !overwrite_ignore */ opts.fn = twoway_merge; - init_tree_desc(&t[0], head->buffer, head->size); - init_tree_desc(&t[1], remote->buffer, remote->size); + init_tree_desc(&t[0], &head->object.oid, head->buffer, head->size); + init_tree_desc(&t[1], &remote->object.oid, remote->buffer, remote->size); if (unpack_trees(2, t, &opts)) { rollback_lock_file(&lock_file); @@ -2026,7 +2026,7 @@ static int merge_tree(struct tree *tree) opts.dst_index = &the_index; opts.merge = 1; opts.fn = oneway_merge; - init_tree_desc(&t[0], tree->buffer, tree->size); + init_tree_desc(&t[0], &tree->object.oid, tree->buffer, tree->size); if (unpack_trees(1, t, &opts)) { rollback_lock_file(&lock_file); diff --git a/builtin/checkout.c b/builtin/checkout.c index f53612f46870..03eff73fd031 100644 --- a/builtin/checkout.c +++ b/builtin/checkout.c @@ -701,7 +701,7 @@ static int reset_tree(struct tree *tree, const struct checkout_opts *o, info->commit ? &info->commit->object.oid : null_oid(), NULL); parse_tree(tree); - init_tree_desc(&tree_desc, tree->buffer, tree->size); + init_tree_desc(&tree_desc, &tree->object.oid, tree->buffer, tree->size); switch (unpack_trees(1, &tree_desc, &opts)) { case -2: *writeout_error = 1; @@ -815,10 +815,12 @@ static int merge_working_tree(const struct checkout_opts *opts, die(_("unable to parse commit %s"), oid_to_hex(old_commit_oid)); - init_tree_desc(&trees[0], tree->buffer, tree->size); + init_tree_desc(&trees[0], &tree->object.oid, + tree->buffer, tree->size); parse_tree(new_tree); tree = new_tree; - init_tree_desc(&trees[1], tree->buffer, tree->size); + init_tree_desc(&trees[1], &tree->object.oid, + tree->buffer, tree->size); ret = unpack_trees(2, trees, &topts); clear_unpack_trees_porcelain(&topts); diff --git a/builtin/clone.c b/builtin/clone.c index c6357af94989..79ceefb93995 100644 --- a/builtin/clone.c +++ b/builtin/clone.c @@ -737,7 +737,7 @@ static int checkout(int submodule_progress, int filter_submodules) if (!tree) die(_("unable to parse commit %s"), oid_to_hex(&oid)); parse_tree(tree); - init_tree_desc(&t, tree->buffer, tree->size); + init_tree_desc(&t, &tree->object.oid, tree->buffer, tree->size); if (unpack_trees(1, &t, &opts) < 0) die(_("unable to checkout working tree")); diff --git a/builtin/commit.c b/builtin/commit.c index 7da5f924484d..537319932b65 100644 --- a/builtin/commit.c +++ b/builtin/commit.c @@ -340,7 +340,7 @@ static void create_base_index(const struct commit *current_head) if (!tree) die(_("failed to unpack HEAD tree object")); parse_tree(tree); - init_tree_desc(&t, tree->buffer, tree->size); + init_tree_desc(&t, &tree->object.oid, tree->buffer, tree->size); if (unpack_trees(1, &t, &opts)) exit(128); /* We've already reported the error, finish dying */ } diff --git a/builtin/grep.c b/builtin/grep.c index 50e712a18479..0c2b8a376f8e 100644 --- a/builtin/grep.c +++ b/builtin/grep.c @@ -530,7 +530,7 @@ static int grep_submodule(struct grep_opt *opt, strbuf_addstr(&base, filename); strbuf_addch(&base, '/'); - init_tree_desc(&tree, data, size); + init_tree_desc(&tree, oid, data, size); hit = grep_tree(&subopt, pathspec, &tree, &base, base.len, object_type == OBJ_COMMIT); strbuf_release(&base); @@ -574,7 +574,7 @@ static int grep_cache(struct grep_opt *opt, data = repo_read_object_file(the_repository, &ce->oid, &type, &size); - init_tree_desc(&tree, data, size); + init_tree_desc(&tree, &ce->oid, data, size); hit |= grep_tree(opt, pathspec, &tree, &name, 0, 0); strbuf_setlen(&name, name_base_len); @@ -670,7 +670,7 @@ static int grep_tree(struct grep_opt *opt, const struct pathspec *pathspec, oid_to_hex(&entry.oid)); strbuf_addch(base, '/'); - init_tree_desc(&sub, data, size); + init_tree_desc(&sub, &entry.oid, data, size); hit |= grep_tree(opt, pathspec, &sub, base, tn_len, check_attr); free(data); @@ -714,7 +714,7 @@ static int grep_object(struct grep_opt *opt, const struct pathspec *pathspec, strbuf_add(&base, name, len); strbuf_addch(&base, ':'); } - init_tree_desc(&tree, data, size); + init_tree_desc(&tree, &obj->oid, data, size); hit = grep_tree(opt, pathspec, &tree, &base, base.len, obj->type == OBJ_COMMIT); strbuf_release(&base); diff --git a/builtin/merge.c b/builtin/merge.c index de68910177fb..718165d45917 100644 --- a/builtin/merge.c +++ b/builtin/merge.c @@ -704,7 +704,8 @@ static int read_tree_trivial(struct object_id *common, struct object_id *head, cache_tree_free(&the_index.cache_tree); for (i = 0; i < nr_trees; i++) { parse_tree(trees[i]); - init_tree_desc(t+i, trees[i]->buffer, trees[i]->size); + init_tree_desc(t+i, &trees[i]->object.oid, + trees[i]->buffer, trees[i]->size); } if (unpack_trees(nr_trees, t, &opts)) return -1; diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c index d2a162d52804..d34902002656 100644 --- a/builtin/pack-objects.c +++ b/builtin/pack-objects.c @@ -1756,7 +1756,8 @@ static void add_pbase_object(struct tree_desc *tree, tree = pbase_tree_get(&entry.oid); if (!tree) return; - init_tree_desc(&sub, tree->tree_data, tree->tree_size); + init_tree_desc(&sub, &tree->oid, + tree->tree_data, tree->tree_size); add_pbase_object(&sub, down, downlen, fullname); pbase_tree_put(tree); @@ -1816,7 +1817,8 @@ static void add_preferred_base_object(const char *name) } else { struct tree_desc tree; - init_tree_desc(&tree, it->pcache.tree_data, it->pcache.tree_size); + init_tree_desc(&tree, &it->pcache.oid, + it->pcache.tree_data, it->pcache.tree_size); add_pbase_object(&tree, name, cmplen, name); } } diff --git a/builtin/read-tree.c b/builtin/read-tree.c index 1fec702a04fa..24d6d156d3a2 100644 --- a/builtin/read-tree.c +++ b/builtin/read-tree.c @@ -264,7 +264,7 @@ int cmd_read_tree(int argc, const char **argv, const char *cmd_prefix) for (i = 0; i < nr_trees; i++) { struct tree *tree = trees[i]; parse_tree(tree); - init_tree_desc(t+i, tree->buffer, tree->size); + init_tree_desc(t+i, &tree->object.oid, tree->buffer, tree->size); } if (unpack_trees(nr_trees, t, &opts)) return 128; diff --git a/builtin/stash.c b/builtin/stash.c index fe64cde9ce30..9ee52af4d28e 100644 --- a/builtin/stash.c +++ b/builtin/stash.c @@ -285,7 +285,7 @@ static int reset_tree(struct object_id *i_tree, int update, int reset) if (parse_tree(tree)) return -1; - init_tree_desc(t, tree->buffer, tree->size); + init_tree_desc(t, &tree->object.oid, tree->buffer, tree->size); opts.head_idx = 1; opts.src_index = &the_index; @@ -871,7 +871,8 @@ static void diff_include_untracked(const struct stash_info *info, struct diff_op tree[i] = parse_tree_indirect(oid[i]); if (parse_tree(tree[i]) < 0) die(_("failed to parse tree")); - init_tree_desc(&tree_desc[i], tree[i]->buffer, tree[i]->size); + init_tree_desc(&tree_desc[i], &tree[i]->object.oid, + tree[i]->buffer, tree[i]->size); } unpack_tree_opt.head_idx = -1; diff --git a/cache-tree.c b/cache-tree.c index ddc7d3d86959..334973a01cee 100644 --- a/cache-tree.c +++ b/cache-tree.c @@ -770,7 +770,7 @@ static void prime_cache_tree_rec(struct repository *r, oidcpy(&it->oid, &tree->object.oid); - init_tree_desc(&desc, tree->buffer, tree->size); + init_tree_desc(&desc, &tree->object.oid, tree->buffer, tree->size); cnt = 0; while (tree_entry(&desc, &entry)) { if (!S_ISDIR(entry.mode)) diff --git a/delta-islands.c b/delta-islands.c index 5de5759f3f13..1ff3506b10f2 100644 --- a/delta-islands.c +++ b/delta-islands.c @@ -289,7 +289,7 @@ void resolve_tree_islands(struct repository *r, if (!tree || parse_tree(tree) < 0) die(_("bad tree object %s"), oid_to_hex(&ent->idx.oid)); - init_tree_desc(&desc, tree->buffer, tree->size); + init_tree_desc(&desc, &tree->object.oid, tree->buffer, tree->size); while (tree_entry(&desc, &entry)) { struct object *obj; diff --git a/diff-lib.c b/diff-lib.c index 6b0c6a7180cc..add323f5628d 100644 --- a/diff-lib.c +++ b/diff-lib.c @@ -558,7 +558,7 @@ static int diff_cache(struct rev_info *revs, opts.pathspec = &revs->diffopt.pathspec; opts.pathspec->recursive = 1; - init_tree_desc(&t, tree->buffer, tree->size); + init_tree_desc(&t, &tree->object.oid, tree->buffer, tree->size); return unpack_trees(1, &t, &opts); } diff --git a/fsck.c b/fsck.c index 2b1e348005b7..6b492a48da82 100644 --- a/fsck.c +++ b/fsck.c @@ -313,7 +313,8 @@ static int fsck_walk_tree(struct tree *tree, void *data, struct fsck_options *op return -1; name = fsck_get_object_name(options, &tree->object.oid); - if (init_tree_desc_gently(&desc, tree->buffer, tree->size, 0)) + if (init_tree_desc_gently(&desc, &tree->object.oid, + tree->buffer, tree->size, 0)) return -1; while (tree_entry_gently(&desc, &entry)) { struct object *obj; @@ -583,7 +584,8 @@ static int fsck_tree(const struct object_id *tree_oid, const char *o_name; struct name_stack df_dup_candidates = { NULL }; - if (init_tree_desc_gently(&desc, buffer, size, TREE_DESC_RAW_MODES)) { + if (init_tree_desc_gently(&desc, tree_oid, buffer, size, + TREE_DESC_RAW_MODES)) { retval += report(options, tree_oid, OBJ_TREE, FSCK_MSG_BAD_TREE, "cannot be parsed as a tree"); diff --git a/http-push.c b/http-push.c index a704f490fdb2..81c35b5e96f7 100644 --- a/http-push.c +++ b/http-push.c @@ -1308,7 +1308,7 @@ static struct object_list **process_tree(struct tree *tree, obj->flags |= SEEN; p = add_one_object(obj, p); - init_tree_desc(&desc, tree->buffer, tree->size); + init_tree_desc(&desc, &tree->object.oid, tree->buffer, tree->size); while (tree_entry(&desc, &entry)) switch (object_type(entry.mode)) { diff --git a/list-objects.c b/list-objects.c index e60a6cd5b46e..312335c8a7f2 100644 --- a/list-objects.c +++ b/list-objects.c @@ -97,7 +97,7 @@ static void process_tree_contents(struct traversal_context *ctx, enum interesting match = ctx->revs->diffopt.pathspec.nr == 0 ? all_entries_interesting : entry_not_interesting; - init_tree_desc(&desc, tree->buffer, tree->size); + init_tree_desc(&desc, &tree->object.oid, tree->buffer, tree->size); while (tree_entry(&desc, &entry)) { if (match != all_entries_interesting) { diff --git a/match-trees.c b/match-trees.c index 0885ac681cd5..3412b6a1401d 100644 --- a/match-trees.c +++ b/match-trees.c @@ -63,7 +63,7 @@ static void *fill_tree_desc_strict(struct tree_desc *desc, die("unable to read tree (%s)", oid_to_hex(hash)); if (type != OBJ_TREE) die("%s is not a tree", oid_to_hex(hash)); - init_tree_desc(desc, buffer, size); + init_tree_desc(desc, hash, buffer, size); return buffer; } @@ -194,7 +194,7 @@ static int splice_tree(const struct object_id *oid1, const char *prefix, buf = repo_read_object_file(the_repository, oid1, &type, &sz); if (!buf) die("cannot read tree %s", oid_to_hex(oid1)); - init_tree_desc(&desc, buf, sz); + init_tree_desc(&desc, oid1, buf, sz); rewrite_here = NULL; while (desc.size) { diff --git a/merge-ort.c b/merge-ort.c index 8631c997002d..3a5729c91e48 100644 --- a/merge-ort.c +++ b/merge-ort.c @@ -1679,9 +1679,10 @@ static int collect_merge_info(struct merge_options *opt, parse_tree(merge_base); parse_tree(side1); parse_tree(side2); - init_tree_desc(t + 0, merge_base->buffer, merge_base->size); - init_tree_desc(t + 1, side1->buffer, side1->size); - init_tree_desc(t + 2, side2->buffer, side2->size); + init_tree_desc(t + 0, &merge_base->object.oid, + merge_base->buffer, merge_base->size); + init_tree_desc(t + 1, &side1->object.oid, side1->buffer, side1->size); + init_tree_desc(t + 2, &side2->object.oid, side2->buffer, side2->size); trace2_region_enter("merge", "traverse_trees", opt->repo); ret = traverse_trees(NULL, 3, t, &info); @@ -4400,9 +4401,9 @@ static int checkout(struct merge_options *opt, unpack_opts.fn = twoway_merge; unpack_opts.preserve_ignored = 0; /* FIXME: !opts->overwrite_ignore */ parse_tree(prev); - init_tree_desc(&trees[0], prev->buffer, prev->size); + init_tree_desc(&trees[0], &prev->object.oid, prev->buffer, prev->size); parse_tree(next); - init_tree_desc(&trees[1], next->buffer, next->size); + init_tree_desc(&trees[1], &next->object.oid, next->buffer, next->size); ret = unpack_trees(2, trees, &unpack_opts); clear_unpack_trees_porcelain(&unpack_opts); diff --git a/merge-recursive.c b/merge-recursive.c index 6a4081bb0f52..93df9eecdd95 100644 --- a/merge-recursive.c +++ b/merge-recursive.c @@ -411,7 +411,7 @@ static inline int merge_detect_rename(struct merge_options *opt) static void init_tree_desc_from_tree(struct tree_desc *desc, struct tree *tree) { parse_tree(tree); - init_tree_desc(desc, tree->buffer, tree->size); + init_tree_desc(desc, &tree->object.oid, tree->buffer, tree->size); } static int unpack_trees_start(struct merge_options *opt, diff --git a/merge.c b/merge.c index b60925459c29..86179c34102d 100644 --- a/merge.c +++ b/merge.c @@ -81,7 +81,8 @@ int checkout_fast_forward(struct repository *r, } for (i = 0; i < nr_trees; i++) { parse_tree(trees[i]); - init_tree_desc(t+i, trees[i]->buffer, trees[i]->size); + init_tree_desc(t+i, &trees[i]->object.oid, + trees[i]->buffer, trees[i]->size); } memset(&opts, 0, sizeof(opts)); diff --git a/pack-bitmap-write.c b/pack-bitmap-write.c index f6757c3cbf20..9211e08f0127 100644 --- a/pack-bitmap-write.c +++ b/pack-bitmap-write.c @@ -366,7 +366,7 @@ static int fill_bitmap_tree(struct bitmap *bitmap, if (parse_tree(tree) < 0) die("unable to load tree object %s", oid_to_hex(&tree->object.oid)); - init_tree_desc(&desc, tree->buffer, tree->size); + init_tree_desc(&desc, &tree->object.oid, tree->buffer, tree->size); while (tree_entry(&desc, &entry)) { switch (object_type(entry.mode)) { diff --git a/packfile.c b/packfile.c index 9cc0a2e37a83..1fae0fcdd9e7 100644 --- a/packfile.c +++ b/packfile.c @@ -2250,7 +2250,8 @@ static int add_promisor_object(const struct object_id *oid, struct tree *tree = (struct tree *)obj; struct tree_desc desc; struct name_entry entry; - if (init_tree_desc_gently(&desc, tree->buffer, tree->size, 0)) + if (init_tree_desc_gently(&desc, &tree->object.oid, + tree->buffer, tree->size, 0)) /* * Error messages are given when packs are * verified, so do not print any here. diff --git a/reflog.c b/reflog.c index 9ad50e7d93e4..c6992a19268f 100644 --- a/reflog.c +++ b/reflog.c @@ -40,7 +40,7 @@ static int tree_is_complete(const struct object_id *oid) tree->buffer = data; tree->size = size; } - init_tree_desc(&desc, tree->buffer, tree->size); + init_tree_desc(&desc, &tree->object.oid, tree->buffer, tree->size); complete = 1; while (tree_entry(&desc, &entry)) { if (!repo_has_object_file(the_repository, &entry.oid) || diff --git a/revision.c b/revision.c index 2f4c53ea207b..a60dfc23a2a5 100644 --- a/revision.c +++ b/revision.c @@ -82,7 +82,7 @@ static void mark_tree_contents_uninteresting(struct repository *r, if (parse_tree_gently(tree, 1) < 0) return; - init_tree_desc(&desc, tree->buffer, tree->size); + init_tree_desc(&desc, &tree->object.oid, tree->buffer, tree->size); while (tree_entry(&desc, &entry)) { switch (object_type(entry.mode)) { case OBJ_TREE: @@ -189,7 +189,7 @@ static void add_children_by_path(struct repository *r, if (parse_tree_gently(tree, 1) < 0) return; - init_tree_desc(&desc, tree->buffer, tree->size); + init_tree_desc(&desc, &tree->object.oid, tree->buffer, tree->size); while (tree_entry(&desc, &entry)) { switch (object_type(entry.mode)) { case OBJ_TREE: diff --git a/tree-walk.c b/tree-walk.c index 3af50a01c2c7..0b44ec7c75ff 100644 --- a/tree-walk.c +++ b/tree-walk.c @@ -15,7 +15,7 @@ static int decode_tree_entry(struct tree_desc *desc, const char *buf, unsigned l const char *path; unsigned int len; uint16_t mode; - const unsigned hashsz = the_hash_algo->rawsz; + const unsigned hashsz = desc->algo->rawsz; if (size < hashsz + 3 || buf[size - (hashsz + 1)]) { strbuf_addstr(err, _("too-short tree object")); @@ -37,15 +37,19 @@ static int decode_tree_entry(struct tree_desc *desc, const char *buf, unsigned l desc->entry.path = path; desc->entry.mode = (desc->flags & TREE_DESC_RAW_MODES) ? mode : canon_mode(mode); desc->entry.pathlen = len - 1; - oidread(&desc->entry.oid, (const unsigned char *)path + len); + oidread_algop(&desc->entry.oid, (const unsigned char *)path + len, + desc->algo); return 0; } -static int init_tree_desc_internal(struct tree_desc *desc, const void *buffer, - unsigned long size, struct strbuf *err, +static int init_tree_desc_internal(struct tree_desc *desc, + const struct object_id *oid, + const void *buffer, unsigned long size, + struct strbuf *err, enum tree_desc_flags flags) { + desc->algo = (oid && oid->algo) ? &hash_algos[oid->algo] : the_hash_algo; desc->buffer = buffer; desc->size = size; desc->flags = flags; @@ -54,19 +58,21 @@ static int init_tree_desc_internal(struct tree_desc *desc, const void *buffer, return 0; } -void init_tree_desc(struct tree_desc *desc, const void *buffer, unsigned long size) +void init_tree_desc(struct tree_desc *desc, const struct object_id *tree_oid, + const void *buffer, unsigned long size) { struct strbuf err = STRBUF_INIT; - if (init_tree_desc_internal(desc, buffer, size, &err, 0)) + if (init_tree_desc_internal(desc, tree_oid, buffer, size, &err, 0)) die("%s", err.buf); strbuf_release(&err); } -int init_tree_desc_gently(struct tree_desc *desc, const void *buffer, unsigned long size, +int init_tree_desc_gently(struct tree_desc *desc, const struct object_id *oid, + const void *buffer, unsigned long size, enum tree_desc_flags flags) { struct strbuf err = STRBUF_INIT; - int result = init_tree_desc_internal(desc, buffer, size, &err, flags); + int result = init_tree_desc_internal(desc, oid, buffer, size, &err, flags); if (result) error("%s", err.buf); strbuf_release(&err); @@ -85,7 +91,7 @@ void *fill_tree_descriptor(struct repository *r, if (!buf) die("unable to read tree %s", oid_to_hex(oid)); } - init_tree_desc(desc, buf, size); + init_tree_desc(desc, oid, buf, size); return buf; } @@ -102,7 +108,7 @@ static void entry_extract(struct tree_desc *t, struct name_entry *a) static int update_tree_entry_internal(struct tree_desc *desc, struct strbuf *err) { const void *buf = desc->buffer; - const unsigned char *end = (const unsigned char *)desc->entry.path + desc->entry.pathlen + 1 + the_hash_algo->rawsz; + const unsigned char *end = (const unsigned char *)desc->entry.path + desc->entry.pathlen + 1 + desc->algo->rawsz; unsigned long size = desc->size; unsigned long len = end - (const unsigned char *)buf; @@ -611,7 +617,7 @@ int get_tree_entry(struct repository *r, retval = -1; } else { struct tree_desc t; - init_tree_desc(&t, tree, size); + init_tree_desc(&t, tree_oid, tree, size); retval = find_tree_entry(r, &t, name, oid, mode); } free(tree); @@ -654,7 +660,7 @@ enum get_oid_result get_tree_entry_follow_symlinks(struct repository *r, struct tree_desc t; int follows_remaining = GET_TREE_ENTRY_FOLLOW_SYMLINKS_MAX_LINKS; - init_tree_desc(&t, NULL, 0UL); + init_tree_desc(&t, NULL, NULL, 0UL); strbuf_addstr(&namebuf, name); oidcpy(¤t_tree_oid, tree_oid); @@ -690,7 +696,7 @@ enum get_oid_result get_tree_entry_follow_symlinks(struct repository *r, goto done; /* descend */ - init_tree_desc(&t, tree, size); + init_tree_desc(&t, ¤t_tree_oid, tree, size); } /* Handle symlinks to e.g. a//b by removing leading slashes */ @@ -724,7 +730,7 @@ enum get_oid_result get_tree_entry_follow_symlinks(struct repository *r, free(parent->tree); parents_nr--; parent = &parents[parents_nr - 1]; - init_tree_desc(&t, parent->tree, parent->size); + init_tree_desc(&t, &parent->oid, parent->tree, parent->size); strbuf_remove(&namebuf, 0, remainder ? 3 : 2); continue; } @@ -804,7 +810,7 @@ enum get_oid_result get_tree_entry_follow_symlinks(struct repository *r, contents_start = contents; parent = &parents[parents_nr - 1]; - init_tree_desc(&t, parent->tree, parent->size); + init_tree_desc(&t, &parent->oid, parent->tree, parent->size); strbuf_splice(&namebuf, 0, len, contents_start, link_len); if (remainder) diff --git a/tree-walk.h b/tree-walk.h index 74cdceb3fed2..cf54d01019e9 100644 --- a/tree-walk.h +++ b/tree-walk.h @@ -26,6 +26,7 @@ struct name_entry { * A semi-opaque data structure used to maintain the current state of the walk. */ struct tree_desc { + const struct git_hash_algo *algo; /* * pointer into the memory representation of the tree. It always * points at the current entry being visited. @@ -85,9 +86,11 @@ int update_tree_entry_gently(struct tree_desc *); * size parameters are assumed to be the same as the buffer and size * members of `struct tree`. */ -void init_tree_desc(struct tree_desc *desc, const void *buf, unsigned long size); +void init_tree_desc(struct tree_desc *desc, const struct object_id *tree_oid, + const void *buf, unsigned long size); -int init_tree_desc_gently(struct tree_desc *desc, const void *buf, unsigned long size, +int init_tree_desc_gently(struct tree_desc *desc, const struct object_id *oid, + const void *buf, unsigned long size, enum tree_desc_flags flags); /* diff --git a/tree.c b/tree.c index c745462f968e..44bcf728f10a 100644 --- a/tree.c +++ b/tree.c @@ -27,7 +27,7 @@ int read_tree_at(struct repository *r, if (parse_tree(tree)) return -1; - init_tree_desc(&desc, tree->buffer, tree->size); + init_tree_desc(&desc, &tree->object.oid, tree->buffer, tree->size); while (tree_entry(&desc, &entry)) { if (retval != all_entries_interesting) { diff --git a/walker.c b/walker.c index 65002a7220ad..c0fd632d921c 100644 --- a/walker.c +++ b/walker.c @@ -45,7 +45,7 @@ static int process_tree(struct walker *walker, struct tree *tree) if (parse_tree(tree)) return -1; - init_tree_desc(&desc, tree->buffer, tree->size); + init_tree_desc(&desc, &tree->object.oid, tree->buffer, tree->size); while (tree_entry(&desc, &entry)) { struct object *obj = NULL; From patchwork Mon Oct 2 02:40:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13405544 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5D247E784A6 for ; Mon, 2 Oct 2023 02:41:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235582AbjJBClt (ORCPT ); Sun, 1 Oct 2023 22:41:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58598 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235466AbjJBClQ (ORCPT ); Sun, 1 Oct 2023 22:41:16 -0400 Received: from mail-io1-xd2a.google.com (mail-io1-xd2a.google.com [IPv6:2607:f8b0:4864:20::d2a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E2C67CD3 for ; Sun, 1 Oct 2023 19:41:01 -0700 (PDT) Received: by mail-io1-xd2a.google.com with SMTP id ca18e2360f4ac-7a25040faffso218163039f.1 for ; Sun, 01 Oct 2023 19:41:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696214461; x=1696819261; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=trUwtva8DMZBzugwmwLMA6mftWeVdjTvb5jeyEQmyuY=; b=WxoVLWjh4e/PjtSkvyvPbomK32TDMUjCk7EOuKSdMq/DoQ01iIDk5/NeO9SnvXR7ZU Xa3J4aPlX4UE6+Sd3JrExlPkmwel1MesrMq0OP8t8aXx8OS3GtdvnOfE4IECyS8JXg0R FnEEzH2Kw9MDrCNR9TxPpelawGsySJPct7PTH/2fv5+GV0F7I5p2R6GC9yU7oL3M609D jWj2P8g/sb/g2AZxsUy58IL5/vQf0CCU6rS4O3uXc+ikpP5Zw2TqLqlbYF0jy7CgLwx0 v5aVN6C6QgQXCmuH7d8C3CKddmbvyO0vGxG8URU1VKAkD6J+vvgH8au7NjD3bnH30B72 Jx8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696214461; x=1696819261; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=trUwtva8DMZBzugwmwLMA6mftWeVdjTvb5jeyEQmyuY=; b=WvAr8wyCtzJJFW87Fd2uKBuIpwWzMoZy+VV3BfBuEmhiD+W5lPdRE9LLIZBxMyUW2f 0rCe5zGO3+CuUF3aepFXG1qdKrOnwr0vdIPFh3DrnipW8b1Z6aFbapGFQyYXppPzg1QE U47QIxIs7K7xKdEnq3nkCEYxCNRZ0JJFILp6uRyhBucIjEjWC7xOgy8dJdiiBynuANGn w4sQuBA1TvRmVhYoj12rEVwlGbJJscHYuYzfJRiqkOA1GasJ/iX+RT6yFhSNPZcHtWgN taLqdxiTRTH36ExGG0Md+d9xv2HZ0S0dZOyDlyo6EvwDB38wyt51rcyPlh08NJPk7lID ujqA== X-Gm-Message-State: AOJu0YwfEp3Kyu8I+hJsg5yifXL26JfQBK34N+FWA1rTsfSXRgzrpQX+ 0XSP0DesY0n+XdXutNKyX/U= X-Google-Smtp-Source: AGHT+IHwfAdZdPXr5Nxxh6qNndh5pzsr+NItSFjxe4jDbO7IRY5c/Z8h2YQlV7ct4aKa0opan7r/VA== X-Received: by 2002:a6b:610a:0:b0:79f:a0c2:4b85 with SMTP id v10-20020a6b610a000000b0079fa0c24b85mr10368357iob.20.1696214461195; Sun, 01 Oct 2023 19:41:01 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id v6-20020a056602058600b0079f9f7a2565sm6346113iox.38.2023.10.01.19.41.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Oct 2023 19:41:00 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , Eric Sunshine , "Eric W. Biederman" Subject: [PATCH v2 25/30] object-file: handle compat objects in check_object_signature Date: Sun, 1 Oct 2023 21:40:29 -0500 Message-Id: <20231002024034.2611-25-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> References: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "Eric W. Biederman" Update check_object_signature to find the hash algorithm the exising signature uses, and to use the same hash algorithm when recomputing it to check the signature is valid. This will be useful when teaching git ls-tree to display objects encoded with the compat hash algorithm. Signed-off-by: "Eric W. Biederman" --- object-file.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/object-file.c b/object-file.c index b2d43d009898..5fa4b14baee0 100644 --- a/object-file.c +++ b/object-file.c @@ -1094,9 +1094,11 @@ int check_object_signature(struct repository *r, const struct object_id *oid, void *buf, unsigned long size, enum object_type type) { + const struct git_hash_algo *algo = + oid->algo ? &hash_algos[oid->algo] : r->hash_algo; struct object_id real_oid; - hash_object_file(r->hash_algo, buf, size, type, &real_oid); + hash_object_file(algo, buf, size, type, &real_oid); return !oideq(oid, &real_oid) ? -1 : 0; } From patchwork Mon Oct 2 02:40:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13405545 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 197FBE784A6 for ; Mon, 2 Oct 2023 02:41:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235426AbjJBClv (ORCPT ); Sun, 1 Oct 2023 22:41:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35564 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235459AbjJBCl3 (ORCPT ); Sun, 1 Oct 2023 22:41:29 -0400 Received: from mail-io1-xd2e.google.com (mail-io1-xd2e.google.com [IPv6:2607:f8b0:4864:20::d2e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E8BD3CD9 for ; Sun, 1 Oct 2023 19:41:02 -0700 (PDT) Received: by mail-io1-xd2e.google.com with SMTP id ca18e2360f4ac-79fe612beabso285194339f.3 for ; Sun, 01 Oct 2023 19:41:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696214462; x=1696819262; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bH2hzoWai8vH7Zjh0kf9Emav2ZMVmyeQiZtNTadsv4M=; b=lASEcsrNuT4hGZb3nIUzcE4YEcwJxX9o6ceBa89bpPNCBX7C8Mge/HNFFJxJn8wUsO RK4jelrF4vFKOgutPbxn5UkUuJPPx7PooeFr21XO3Kwn8JjnTQY4y/bsQ75dGUlzZ76s 27basMjKCFH34+fDgb17OqJxqKXHSNvcHZ6YOpOfI6QAk7wijK9MlndH4v0UZ3HI2S0z twy6pr1Kh1JEYyXy9tF8RrEatHYmk0to6J/Tqp8tp1hs0tfeLif0o229teT/NrcctSoD rVRxlcHElilVoRQt2kWVFpuCEqibvlLOENAJgKslknj57FiBTHNvD5g6XTylZvQTvbI6 /nmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696214462; x=1696819262; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bH2hzoWai8vH7Zjh0kf9Emav2ZMVmyeQiZtNTadsv4M=; b=I3FdlKyXn8vGTVRLNLhF8RipsanfD0OUnn3xlDog0ZVCbT1m/XIy95kDp00V0YkdF3 AthjdO7pfEdC/5UoV59Rp5D4Zprm8taXF8AjKVH2fdBVZrkD2S1jXrNZcMQXZbFA3hYk tDxUtwufHfrS2iedg6BAs4mS6xKW5Rxvcptt+x9c80TSW5vcy4vliNwYNFjDCwmLGuiu 9vc3kM8eYDjpJPWXn6eO5LDxjWJqwFGrcLOEgNBCGldYNIlGCLxqf1zKa3C1DpKz8zES YQN6GYcpJyc4+UxhK/RFa4NzzctIDbWF+3RjTArchulCBHCarDX6Be5cALRgoJOsfD6K 20fg== X-Gm-Message-State: AOJu0YwRl8xE9b2Do791wY7euAy5iXrMB8CYNko8G4MELMzYCMHnOgX/ VacYiGU2BAOfrM0vXoOkzoY= X-Google-Smtp-Source: AGHT+IEwJdTV/uujbwpgJGjS9gG1Zq6HGmBjhfqI7Pj7Z9XYdRfcCKUwKGNAP7cH1OTRMv8u8cBgKw== X-Received: by 2002:a5e:891a:0:b0:794:e11c:d8c3 with SMTP id k26-20020a5e891a000000b00794e11cd8c3mr11482632ioj.16.1696214462207; Sun, 01 Oct 2023 19:41:02 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id v6-20020a056602058600b0079f9f7a2565sm6346113iox.38.2023.10.01.19.41.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Oct 2023 19:41:01 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , Eric Sunshine , "Eric W. Biederman" Subject: [PATCH v2 26/30] builtin/ls-tree: let the oid determine the output algorithm Date: Sun, 1 Oct 2023 21:40:30 -0500 Message-Id: <20231002024034.2611-26-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> References: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "Eric W. Biederman" Update cmd_ls_tree to call get_oid_with_context and pass GET_OID_HASH_ANY instead of calling the simpler repo_get_oid. This implments in ls-tree the behavior that asking to display a sha1 hash displays the corrresponding sha1 encoded object and asking to display a sha256 hash displayes the corresponding sha256 encoded object. This is useful for testing the conversion of an object to an equivlanet object encoded with a different hash function. Signed-off-by: "Eric W. Biederman" --- builtin/ls-tree.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/builtin/ls-tree.c b/builtin/ls-tree.c index f558db5f3b80..71281ab705b6 100644 --- a/builtin/ls-tree.c +++ b/builtin/ls-tree.c @@ -376,6 +376,7 @@ int cmd_ls_tree(int argc, const char **argv, const char *prefix) OPT_END() }; struct ls_tree_cmdmode_to_fmt *m2f = ls_tree_cmdmode_format; + struct object_context obj_context; int ret; git_config(git_default_config, NULL); @@ -407,7 +408,9 @@ int cmd_ls_tree(int argc, const char **argv, const char *prefix) ls_tree_usage, ls_tree_options); if (argc < 1) usage_with_options(ls_tree_usage, ls_tree_options); - if (repo_get_oid(the_repository, argv[0], &oid)) + if (get_oid_with_context(the_repository, argv[0], + GET_OID_HASH_ANY, &oid, + &obj_context)) die("Not a valid object name %s", argv[0]); /* From patchwork Mon Oct 2 02:40:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13405547 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B2C67E784A6 for ; Mon, 2 Oct 2023 02:41:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235513AbjJBClz (ORCPT ); Sun, 1 Oct 2023 22:41:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38976 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235506AbjJBCl3 (ORCPT ); Sun, 1 Oct 2023 22:41:29 -0400 Received: from mail-io1-xd2d.google.com (mail-io1-xd2d.google.com [IPv6:2607:f8b0:4864:20::d2d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A2F92CDB for ; Sun, 1 Oct 2023 19:41:03 -0700 (PDT) Received: by mail-io1-xd2d.google.com with SMTP id ca18e2360f4ac-79fb70402a7so456212739f.0 for ; Sun, 01 Oct 2023 19:41:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696214463; x=1696819263; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/lEoBpkJLVvzE2xIITImYwnRjfOno4E1sdAiC3yAWps=; b=Xf4JhLAQEDaKLejG95nUHX2Ph6AHleR5XX4UaQfwKnanmaRf7aFro/wItHFGgDsomj DCXPHKeWmKwpYE8kkcmKSA6ncCThf7tmMMM9iunOaRCH4SpgTPkjUpjSb8lW/uMAVupK Y6wnv3YK8P3v1C7Cf7MIxfnDDPDu14yI+MjmnMMnOJK894HJ+PbG/SEYQ/041lWioOtE AU/IpO/9XZCAytYxjDz1Fshb4L6CuQPCViYW6OpWKjH5iq/hEIl02j/lzAKaZOlkwIwm lkIEilcocF6DqDCTgypql/O+jSX2XSXVgj7sbLf8yZnIpP7jYeJDHFs4HtoaxQNl7Sbv QoCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696214463; x=1696819263; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/lEoBpkJLVvzE2xIITImYwnRjfOno4E1sdAiC3yAWps=; b=USlTxSpJpD7+nVNCnry9F7pP0YBkoBvpg+dKqInWRpWSzyjdgIiWlQ9LsdB2a9JbT3 jfkigpsc5wZzx1LRduTnSYc9Vnc9B6aMH1OAKpRHY8/NZW2SXYeyYrCenhiJAUTsKyHH 0CSqQcXOvpU622SFZRIRYLJ0ZAoocUYQYDnSeorJh5nhfGC7RTI4kfHgf5bgcEUrO0Ja qkSxargEETj6Ok3Gbc71obEMEXLiF4sQvi+MFxdqEIdf6RnuCpRH/gjnkCpOsxG0WU2S zxJ0WqExB/4gBMN0ZWVUMp1L/2ryvyGeV4sn0XrX0xz6ceBH6gyNCS6fJTHMOx77xlyk 0H5A== X-Gm-Message-State: AOJu0Yz+2fFxUHjAfXA+t2Nvc6Zioj3CBAb3lopBj+E9yyxntkZTfyLa sGJ+iaywvcUIrlone+A1F4yq+1CvUTqjCw== X-Google-Smtp-Source: AGHT+IHt1Epru5K4RxLCwCNwGp9vNUKcmVwLshPgecsa7VZveE0BiWUGp8yXKK0uLCONVbnFSGbKKg== X-Received: by 2002:a6b:5b0d:0:b0:790:c3d0:8f87 with SMTP id v13-20020a6b5b0d000000b00790c3d08f87mr11777546ioh.19.1696214463018; Sun, 01 Oct 2023 19:41:03 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id v6-20020a056602058600b0079f9f7a2565sm6346113iox.38.2023.10.01.19.41.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Oct 2023 19:41:02 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , Eric Sunshine , "Eric W. Biederman" Subject: [PATCH v2 27/30] test-lib: compute the compatibility hash so tests may use it Date: Sun, 1 Oct 2023 21:40:31 -0500 Message-Id: <20231002024034.2611-27-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> References: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "Eric W. Biederman" Inspired-by: brian m. carlson Signed-off-by: "Eric W. Biederman" --- t/test-lib-functions.sh | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh index 2f8868caa171..92b462e2e711 100644 --- a/t/test-lib-functions.sh +++ b/t/test-lib-functions.sh @@ -1599,7 +1599,16 @@ test_set_hash () { # Detect the hash algorithm in use. test_detect_hash () { - test_hash_algo="${GIT_TEST_DEFAULT_HASH:-sha1}" + case "$GIT_TEST_DEFAULT_HASH" in + "sha256") + test_hash_algo=sha256 + test_compat_hash_algo=sha1 + ;; + *) + test_hash_algo=sha1 + test_compat_hash_algo=sha256 + ;; + esac } # Load common hash metadata and common placeholder object IDs for use with @@ -1651,6 +1660,12 @@ test_oid () { local algo="${test_hash_algo}" && case "$1" in + --hash=storage) + algo="$test_hash_algo" && + shift;; + --hash=compat) + algo="$test_compat_hash_algo" && + shift;; --hash=*) algo="${1#--hash=}" && shift;; From patchwork Mon Oct 2 02:40:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13405549 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5778AE784A7 for ; Mon, 2 Oct 2023 02:41:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235523AbjJBCl7 (ORCPT ); Sun, 1 Oct 2023 22:41:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58430 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235434AbjJBCla (ORCPT ); Sun, 1 Oct 2023 22:41:30 -0400 Received: from mail-io1-xd2e.google.com (mail-io1-xd2e.google.com [IPv6:2607:f8b0:4864:20::d2e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DD97FCE3 for ; Sun, 1 Oct 2023 19:41:04 -0700 (PDT) Received: by mail-io1-xd2e.google.com with SMTP id ca18e2360f4ac-7a25040faffso218164539f.1 for ; Sun, 01 Oct 2023 19:41:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696214464; x=1696819264; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Ku1SPtYJYS0jAS4sy8h0S6jPalVmdQPKuhDb6UU5UX8=; b=hypxAKFrs180GHkVaIyRBAeUJZiEf2hRUBm1hEAPFnAERLOcWTikVT+7pOg6JNO9rM p08HSZxZzE8je43mOpXsqcSMkL2de1FG/NnCGR9Hvb4jrLFVrXQQpLwEHg71bRVbJqLL eftNVw8vEBfJX7zTQnhcP451A49w+qLKvvF6ADYCazBNMHVq/OOkhaUhcLnDIKjREHJs fFBxSIhjzTxQ8AAbJvW5AuChMTl6HjcraX4WAk0VogPR9bSWETtFiN2cQstCKb1fJmY6 Ub84NidJRSNwLAjMiI88ASJsENdRV1OL+oKkX28k5CLJrH/DRAiG3CQnsuX7i9NptmyM h6sg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696214464; x=1696819264; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Ku1SPtYJYS0jAS4sy8h0S6jPalVmdQPKuhDb6UU5UX8=; b=A5WBW5X+RekCFQGRcU6T0metSfvxr7Y3bqdbcufDhUp6IrGHvZE8JIbKPOe0EEeLD2 0jrLmiZPnmkSg4ciwAIcyJAgM37MWY1Yio8Js+wjCMYqxKJASloImbxochtIcIC5c4ng YS2WsMLZctUKjt3slpjFfLn/R/8o+UwEAcdRYfYMBJeiSs6jzc4v94jQQfRv6pQ9SCIg 9LaWpIR9Lq+baTa2jn5dG0C4aNuFtNNR6V4PTvBpfjEeaPUHcx5kS8DfypuK2lAxLp80 MGU/vQLi4oLFTD2tW4IBhZmhAGFpIVOb95K20NdlrsYOavWwju+fWnjYLNb9roNqpHXy sBWw== X-Gm-Message-State: AOJu0Yy6fw+F0v2vqnK55PJRL190ejhje97PYf2ZtZqjxptUcZc0FCzg YTsh7eg5NUvqdut0WUEgKkU= X-Google-Smtp-Source: AGHT+IFCsrMyCFqFwe25n7kvrf+iP0HGGTrmUx+t9Zx9BBVvsLMri1zQOrCksykEKc6KqbXdL6AHXw== X-Received: by 2002:a05:6602:2215:b0:787:1697:1b3b with SMTP id n21-20020a056602221500b0078716971b3bmr11574696ion.8.1696214463861; Sun, 01 Oct 2023 19:41:03 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id v6-20020a056602058600b0079f9f7a2565sm6346113iox.38.2023.10.01.19.41.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Oct 2023 19:41:03 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , Eric Sunshine , "Eric W. Biederman" Subject: [PATCH v2 28/30] t1006: rename sha1 to oid Date: Sun, 1 Oct 2023 21:40:32 -0500 Message-Id: <20231002024034.2611-28-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> References: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "Eric W. Biederman" Before I extend this test, changing the naming of the relevant hash from sha1 to oid. Calling the hash sha1 is incorrect today as it can be either sha1 or sha256 depending on the value of GIT_DEFAULT_HASH_FUNCTION when the test is called. I plan to test sha1 and sha256 simultaneously in the same repository. Having a name like sha1 will be even more confusing. Signed-off-by: "Eric W. Biederman" --- t/t1006-cat-file.sh | 220 ++++++++++++++++++++++---------------------- 1 file changed, 110 insertions(+), 110 deletions(-) diff --git a/t/t1006-cat-file.sh b/t/t1006-cat-file.sh index d73a0be1b9d1..9b018b538950 100755 --- a/t/t1006-cat-file.sh +++ b/t/t1006-cat-file.sh @@ -112,65 +112,65 @@ strlen () { run_tests () { type=$1 - sha1=$2 + oid=$2 size=$3 content=$4 pretty_content=$5 - batch_output="$sha1 $type $size + batch_output="$oid $type $size $content" test_expect_success "$type exists" ' - git cat-file -e $sha1 + git cat-file -e $oid ' test_expect_success "Type of $type is correct" ' echo $type >expect && - git cat-file -t $sha1 >actual && + git cat-file -t $oid >actual && test_cmp expect actual ' test_expect_success "Size of $type is correct" ' echo $size >expect && - git cat-file -s $sha1 >actual && + git cat-file -s $oid >actual && test_cmp expect actual ' test_expect_success "Type of $type is correct using --allow-unknown-type" ' echo $type >expect && - git cat-file -t --allow-unknown-type $sha1 >actual && + git cat-file -t --allow-unknown-type $oid >actual && test_cmp expect actual ' test_expect_success "Size of $type is correct using --allow-unknown-type" ' echo $size >expect && - git cat-file -s --allow-unknown-type $sha1 >actual && + git cat-file -s --allow-unknown-type $oid >actual && test_cmp expect actual ' test -z "$content" || test_expect_success "Content of $type is correct" ' echo_without_newline "$content" >expect && - git cat-file $type $sha1 >actual && + git cat-file $type $oid >actual && test_cmp expect actual ' test_expect_success "Pretty content of $type is correct" ' echo_without_newline "$pretty_content" >expect && - git cat-file -p $sha1 >actual && + git cat-file -p $oid >actual && test_cmp expect actual ' test -z "$content" || test_expect_success "--batch output of $type is correct" ' echo "$batch_output" >expect && - echo $sha1 | git cat-file --batch >actual && + echo $oid | git cat-file --batch >actual && test_cmp expect actual ' test_expect_success "--batch-check output of $type is correct" ' - echo "$sha1 $type $size" >expect && - echo_without_newline $sha1 | git cat-file --batch-check >actual && + echo "$oid $type $size" >expect && + echo_without_newline $oid | git cat-file --batch-check >actual && test_cmp expect actual ' @@ -179,33 +179,33 @@ $content" test -z "$content" || test_expect_success "--batch-command $opt output of $type content is correct" ' echo "$batch_output" >expect && - test_write_lines "contents $sha1" | git cat-file --batch-command $opt >actual && + test_write_lines "contents $oid" | git cat-file --batch-command $opt >actual && test_cmp expect actual ' test_expect_success "--batch-command $opt output of $type info is correct" ' - echo "$sha1 $type $size" >expect && - test_write_lines "info $sha1" | + echo "$oid $type $size" >expect && + test_write_lines "info $oid" | git cat-file --batch-command $opt >actual && test_cmp expect actual ' done test_expect_success "custom --batch-check format" ' - echo "$type $sha1" >expect && - echo $sha1 | git cat-file --batch-check="%(objecttype) %(objectname)" >actual && + echo "$type $oid" >expect && + echo $oid | git cat-file --batch-check="%(objecttype) %(objectname)" >actual && test_cmp expect actual ' test_expect_success "custom --batch-command format" ' - echo "$type $sha1" >expect && - echo "info $sha1" | git cat-file --batch-command="%(objecttype) %(objectname)" >actual && + echo "$type $oid" >expect && + echo "info $oid" | git cat-file --batch-command="%(objecttype) %(objectname)" >actual && test_cmp expect actual ' test_expect_success '--batch-check with %(rest)' ' echo "$type this is some extra content" >expect && - echo "$sha1 this is some extra content" | + echo "$oid this is some extra content" | git cat-file --batch-check="%(objecttype) %(rest)" >actual && test_cmp expect actual ' @@ -216,7 +216,7 @@ $content" echo "$size" && echo "$content" } >expect && - echo $sha1 | git cat-file --batch="%(objectsize)" >actual && + echo $oid | git cat-file --batch="%(objectsize)" >actual && test_cmp expect actual ' @@ -226,25 +226,25 @@ $content" echo "$type" && echo "$content" } >expect && - echo $sha1 | git cat-file --batch="%(objecttype)" >actual && + echo $oid | git cat-file --batch="%(objecttype)" >actual && test_cmp expect actual ' } hello_content="Hello World" hello_size=$(strlen "$hello_content") -hello_sha1=$(echo_without_newline "$hello_content" | git hash-object --stdin) +hello_oid=$(echo_without_newline "$hello_content" | git hash-object --stdin) test_expect_success "setup" ' echo_without_newline "$hello_content" > hello && git update-index --add hello ' -run_tests 'blob' $hello_sha1 $hello_size "$hello_content" "$hello_content" +run_tests 'blob' $hello_oid $hello_size "$hello_content" "$hello_content" test_expect_success '--batch-command --buffer with flush for blob info' ' - echo "$hello_sha1 blob $hello_size" >expect && - test_write_lines "info $hello_sha1" "flush" | + echo "$hello_oid blob $hello_size" >expect && + test_write_lines "info $hello_oid" "flush" | GIT_TEST_CAT_FILE_NO_FLUSH_ON_EXIT=1 \ git cat-file --batch-command --buffer >actual && test_cmp expect actual @@ -252,38 +252,38 @@ test_expect_success '--batch-command --buffer with flush for blob info' ' test_expect_success '--batch-command --buffer without flush for blob info' ' touch output && - test_write_lines "info $hello_sha1" | + test_write_lines "info $hello_oid" | GIT_TEST_CAT_FILE_NO_FLUSH_ON_EXIT=1 \ git cat-file --batch-command --buffer >>output && test_must_be_empty output ' test_expect_success '--batch-check without %(rest) considers whole line' ' - echo "$hello_sha1 blob $hello_size" >expect && - git update-index --add --cacheinfo 100644 $hello_sha1 "white space" && + echo "$hello_oid blob $hello_size" >expect && + git update-index --add --cacheinfo 100644 $hello_oid "white space" && test_when_finished "git update-index --remove \"white space\"" && echo ":white space" | git cat-file --batch-check >actual && test_cmp expect actual ' -tree_sha1=$(git write-tree) +tree_oid=$(git write-tree) tree_size=$(($(test_oid rawsz) + 13)) -tree_pretty_content="100644 blob $hello_sha1 hello${LF}" +tree_pretty_content="100644 blob $hello_oid hello${LF}" -run_tests 'tree' $tree_sha1 $tree_size "" "$tree_pretty_content" +run_tests 'tree' $tree_oid $tree_size "" "$tree_pretty_content" commit_message="Initial commit" -commit_sha1=$(echo_without_newline "$commit_message" | git commit-tree $tree_sha1) +commit_oid=$(echo_without_newline "$commit_message" | git commit-tree $tree_oid) commit_size=$(($(test_oid hexsz) + 137)) -commit_content="tree $tree_sha1 +commit_content="tree $tree_oid author $GIT_AUTHOR_NAME <$GIT_AUTHOR_EMAIL> $GIT_AUTHOR_DATE committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE $commit_message" -run_tests 'commit' $commit_sha1 $commit_size "$commit_content" "$commit_content" +run_tests 'commit' $commit_oid $commit_size "$commit_content" "$commit_content" -tag_header_without_timestamp="object $hello_sha1 +tag_header_without_timestamp="object $hello_oid type blob tag hellotag tagger $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL>" @@ -292,14 +292,14 @@ tag_content="$tag_header_without_timestamp 0 +0000 $tag_description" -tag_sha1=$(echo_without_newline "$tag_content" | git hash-object -t tag --stdin -w) +tag_oid=$(echo_without_newline "$tag_content" | git hash-object -t tag --stdin -w) tag_size=$(strlen "$tag_content") -run_tests 'tag' $tag_sha1 $tag_size "$tag_content" "$tag_content" +run_tests 'tag' $tag_oid $tag_size "$tag_content" "$tag_content" test_expect_success "Reach a blob from a tag pointing to it" ' echo_without_newline "$hello_content" >expect && - git cat-file blob $tag_sha1 >actual && + git cat-file blob $tag_oid >actual && test_cmp expect actual ' @@ -308,31 +308,31 @@ do for opt in t s e p do test_expect_success "Passing -$opt with --$batch fails" ' - test_must_fail git cat-file --$batch -$opt $hello_sha1 + test_must_fail git cat-file --$batch -$opt $hello_oid ' test_expect_success "Passing --$batch with -$opt fails" ' - test_must_fail git cat-file -$opt --$batch $hello_sha1 + test_must_fail git cat-file -$opt --$batch $hello_oid ' done test_expect_success "Passing with --$batch fails" ' - test_must_fail git cat-file --$batch blob $hello_sha1 + test_must_fail git cat-file --$batch blob $hello_oid ' test_expect_success "Passing --$batch with fails" ' - test_must_fail git cat-file blob --$batch $hello_sha1 + test_must_fail git cat-file blob --$batch $hello_oid ' - test_expect_success "Passing sha1 with --$batch fails" ' - test_must_fail git cat-file --$batch $hello_sha1 + test_expect_success "Passing oid with --$batch fails" ' + test_must_fail git cat-file --$batch $hello_oid ' done for opt in t s e p do test_expect_success "Passing -$opt with --follow-symlinks fails" ' - test_must_fail git cat-file --follow-symlinks -$opt $hello_sha1 + test_must_fail git cat-file --follow-symlinks -$opt $hello_oid ' done @@ -360,12 +360,12 @@ test_expect_success "--batch-check for a non-existent hash" ' test_expect_success "--batch for an existent and a non-existent hash" ' cat >expect <<-EOF && - $tag_sha1 tag $tag_size + $tag_oid tag $tag_size $tag_content 0000000000000000000000000000000000000000 missing EOF - printf "$tag_sha1\n0000000000000000000000000000000000000000" >in && + printf "$tag_oid\n0000000000000000000000000000000000000000" >in && git cat-file --batch actual && test_cmp expect actual ' @@ -386,74 +386,74 @@ test_expect_success 'empty --batch-check notices missing object' ' test_cmp expect actual ' -batch_input="$hello_sha1 -$commit_sha1 -$tag_sha1 +batch_input="$hello_oid +$commit_oid +$tag_oid deadbeef " printf "%s\0" \ - "$hello_sha1 blob $hello_size" \ + "$hello_oid blob $hello_size" \ "$hello_content" \ - "$commit_sha1 commit $commit_size" \ + "$commit_oid commit $commit_size" \ "$commit_content" \ - "$tag_sha1 tag $tag_size" \ + "$tag_oid tag $tag_size" \ "$tag_content" \ "deadbeef missing" \ " missing" >batch_output -test_expect_success '--batch with multiple sha1s gives correct format' ' +test_expect_success '--batch with multiple oids gives correct format' ' tr "\0" "\n" expect && echo_without_newline "$batch_input" >in && git cat-file --batch actual && test_cmp expect actual ' -test_expect_success '--batch, -z with multiple sha1s gives correct format' ' +test_expect_success '--batch, -z with multiple oids gives correct format' ' echo_without_newline_nul "$batch_input" >in && tr "\0" "\n" expect && git cat-file --batch -z actual && test_cmp expect actual ' -test_expect_success '--batch, -Z with multiple sha1s gives correct format' ' +test_expect_success '--batch, -Z with multiple oids gives correct format' ' echo_without_newline_nul "$batch_input" >in && git cat-file --batch -Z actual && test_cmp batch_output actual ' -batch_check_input="$hello_sha1 -$tree_sha1 -$commit_sha1 -$tag_sha1 +batch_check_input="$hello_oid +$tree_oid +$commit_oid +$tag_oid deadbeef " printf "%s\0" \ - "$hello_sha1 blob $hello_size" \ - "$tree_sha1 tree $tree_size" \ - "$commit_sha1 commit $commit_size" \ - "$tag_sha1 tag $tag_size" \ + "$hello_oid blob $hello_size" \ + "$tree_oid tree $tree_size" \ + "$commit_oid commit $commit_size" \ + "$tag_oid tag $tag_size" \ "deadbeef missing" \ " missing" >batch_check_output -test_expect_success "--batch-check with multiple sha1s gives correct format" ' +test_expect_success "--batch-check with multiple oids gives correct format" ' tr "\0" "\n" expect && echo_without_newline "$batch_check_input" >in && git cat-file --batch-check actual && test_cmp expect actual ' -test_expect_success "--batch-check, -z with multiple sha1s gives correct format" ' +test_expect_success "--batch-check, -z with multiple oids gives correct format" ' tr "\0" "\n" expect && echo_without_newline_nul "$batch_check_input" >in && git cat-file --batch-check -z actual && test_cmp expect actual ' -test_expect_success "--batch-check, -Z with multiple sha1s gives correct format" ' +test_expect_success "--batch-check, -Z with multiple oids gives correct format" ' echo_without_newline_nul "$batch_check_input" >in && git cat-file --batch-check -Z actual && test_cmp batch_check_output actual @@ -480,18 +480,18 @@ test_expect_success FUNNYNAMES '--batch-check, -Z with newline in input' ' test_cmp expect actual ' -batch_command_multiple_info="info $hello_sha1 -info $tree_sha1 -info $commit_sha1 -info $tag_sha1 +batch_command_multiple_info="info $hello_oid +info $tree_oid +info $commit_oid +info $tag_oid info deadbeef" test_expect_success '--batch-command with multiple info calls gives correct format' ' cat >expect <<-EOF && - $hello_sha1 blob $hello_size - $tree_sha1 tree $tree_size - $commit_sha1 commit $commit_size - $tag_sha1 tag $tag_size + $hello_oid blob $hello_size + $tree_oid tree $tree_size + $commit_oid commit $commit_size + $tag_oid tag $tag_size deadbeef missing EOF @@ -512,19 +512,19 @@ test_expect_success '--batch-command with multiple info calls gives correct form test_cmp expect_nul actual ' -batch_command_multiple_contents="contents $hello_sha1 -contents $commit_sha1 -contents $tag_sha1 +batch_command_multiple_contents="contents $hello_oid +contents $commit_oid +contents $tag_oid contents deadbeef flush" test_expect_success '--batch-command with multiple command calls gives correct format' ' printf "%s\0" \ - "$hello_sha1 blob $hello_size" \ + "$hello_oid blob $hello_size" \ "$hello_content" \ - "$commit_sha1 commit $commit_size" \ + "$commit_oid commit $commit_size" \ "$commit_content" \ - "$tag_sha1 tag $tag_size" \ + "$tag_oid tag $tag_size" \ "$tag_content" \ "deadbeef missing" >expect_nul && tr "\0" "\n" expect && @@ -569,7 +569,7 @@ test_expect_success 'confirm that neither loose blob is a delta' ' # we will check only that one of the two objects is a delta # against the other, but not the order. We can do so by just # asking for the base of both, and checking whether either -# sha1 appears in the output. +# oid appears in the output. test_expect_success '%(deltabase) reports packed delta bases' ' git repack -ad && git cat-file --batch-check="%(deltabase)" actual && @@ -583,12 +583,12 @@ test_expect_success 'setup bogus data' ' bogus_short_type="bogus" && bogus_short_content="bogus" && bogus_short_size=$(strlen "$bogus_short_content") && - bogus_short_sha1=$(echo_without_newline "$bogus_short_content" | git hash-object -t $bogus_short_type --literally -w --stdin) && + bogus_short_oid=$(echo_without_newline "$bogus_short_content" | git hash-object -t $bogus_short_type --literally -w --stdin) && bogus_long_type="abcdefghijklmnopqrstuvwxyz1234679" && bogus_long_content="bogus" && bogus_long_size=$(strlen "$bogus_long_content") && - bogus_long_sha1=$(echo_without_newline "$bogus_long_content" | git hash-object -t $bogus_long_type --literally -w --stdin) + bogus_long_oid=$(echo_without_newline "$bogus_long_content" | git hash-object -t $bogus_long_type --literally -w --stdin) ' for arg1 in '' --allow-unknown-type @@ -608,9 +608,9 @@ do if test "$arg1" = "--allow-unknown-type" then - git cat-file $arg1 $arg2 $bogus_short_sha1 + git cat-file $arg1 $arg2 $bogus_short_oid else - test_must_fail git cat-file $arg1 $arg2 $bogus_short_sha1 >out 2>actual && + test_must_fail git cat-file $arg1 $arg2 $bogus_short_oid >out 2>actual && test_must_be_empty out && test_cmp expect actual fi @@ -620,21 +620,21 @@ do if test "$arg2" = "-p" then cat >expect <<-EOF - error: header for $bogus_long_sha1 too long, exceeds 32 bytes - fatal: Not a valid object name $bogus_long_sha1 + error: header for $bogus_long_oid too long, exceeds 32 bytes + fatal: Not a valid object name $bogus_long_oid EOF else cat >expect <<-EOF - error: header for $bogus_long_sha1 too long, exceeds 32 bytes + error: header for $bogus_long_oid too long, exceeds 32 bytes fatal: git cat-file: could not get object info EOF fi && if test "$arg1" = "--allow-unknown-type" then - git cat-file $arg1 $arg2 $bogus_short_sha1 + git cat-file $arg1 $arg2 $bogus_short_oid else - test_must_fail git cat-file $arg1 $arg2 $bogus_long_sha1 >out 2>actual && + test_must_fail git cat-file $arg1 $arg2 $bogus_long_oid >out 2>actual && test_must_be_empty out && test_cmp expect actual fi @@ -668,28 +668,28 @@ do done test_expect_success '-e is OK with a broken object without --allow-unknown-type' ' - git cat-file -e $bogus_short_sha1 + git cat-file -e $bogus_short_oid ' test_expect_success '-e can not be combined with --allow-unknown-type' ' - test_expect_code 128 git cat-file -e --allow-unknown-type $bogus_short_sha1 + test_expect_code 128 git cat-file -e --allow-unknown-type $bogus_short_oid ' test_expect_success '-p cannot print a broken object even with --allow-unknown-type' ' - test_must_fail git cat-file -p $bogus_short_sha1 && - test_expect_code 128 git cat-file -p --allow-unknown-type $bogus_short_sha1 + test_must_fail git cat-file -p $bogus_short_oid && + test_expect_code 128 git cat-file -p --allow-unknown-type $bogus_short_oid ' test_expect_success ' does not work with objects of broken types' ' cat >err.expect <<-\EOF && fatal: invalid object type "bogus" EOF - test_must_fail git cat-file $bogus_short_type $bogus_short_sha1 2>err.actual && + test_must_fail git cat-file $bogus_short_type $bogus_short_oid 2>err.actual && test_cmp err.expect err.actual ' test_expect_success 'broken types combined with --batch and --batch-check' ' - echo $bogus_short_sha1 >bogus-oid && + echo $bogus_short_oid >bogus-oid && cat >err.expect <<-\EOF && fatal: invalid object type @@ -711,52 +711,52 @@ test_expect_success 'the --allow-unknown-type option does not consider replaceme cat >expect <<-EOF && $bogus_short_type EOF - git cat-file -t --allow-unknown-type $bogus_short_sha1 >actual && + git cat-file -t --allow-unknown-type $bogus_short_oid >actual && test_cmp expect actual && # Create it manually, as "git replace" will die on bogus # types. head=$(git rev-parse --verify HEAD) && - test_when_finished "test-tool ref-store main delete-refs 0 msg refs/replace/$bogus_short_sha1" && - test-tool ref-store main update-ref msg "refs/replace/$bogus_short_sha1" $head $ZERO_OID REF_SKIP_OID_VERIFICATION && + test_when_finished "test-tool ref-store main delete-refs 0 msg refs/replace/$bogus_short_oid" && + test-tool ref-store main update-ref msg "refs/replace/$bogus_short_oid" $head $ZERO_OID REF_SKIP_OID_VERIFICATION && cat >expect <<-EOF && commit EOF - git cat-file -t --allow-unknown-type $bogus_short_sha1 >actual && + git cat-file -t --allow-unknown-type $bogus_short_oid >actual && test_cmp expect actual ' test_expect_success "Type of broken object is correct" ' echo $bogus_short_type >expect && - git cat-file -t --allow-unknown-type $bogus_short_sha1 >actual && + git cat-file -t --allow-unknown-type $bogus_short_oid >actual && test_cmp expect actual ' test_expect_success "Size of broken object is correct" ' echo $bogus_short_size >expect && - git cat-file -s --allow-unknown-type $bogus_short_sha1 >actual && + git cat-file -s --allow-unknown-type $bogus_short_oid >actual && test_cmp expect actual ' test_expect_success 'clean up broken object' ' - rm .git/objects/$(test_oid_to_path $bogus_short_sha1) + rm .git/objects/$(test_oid_to_path $bogus_short_oid) ' test_expect_success "Type of broken object is correct when type is large" ' echo $bogus_long_type >expect && - git cat-file -t --allow-unknown-type $bogus_long_sha1 >actual && + git cat-file -t --allow-unknown-type $bogus_long_oid >actual && test_cmp expect actual ' test_expect_success "Size of large broken object is correct when type is large" ' echo $bogus_long_size >expect && - git cat-file -s --allow-unknown-type $bogus_long_sha1 >actual && + git cat-file -s --allow-unknown-type $bogus_long_oid >actual && test_cmp expect actual ' test_expect_success 'clean up broken object' ' - rm .git/objects/$(test_oid_to_path $bogus_long_sha1) + rm .git/objects/$(test_oid_to_path $bogus_long_oid) ' test_expect_success 'cat-file -t and -s on corrupt loose object' ' @@ -853,7 +853,7 @@ test_expect_success 'prep for symlink tests' ' test_ln_s_add loop2 loop1 && git add morx dir/subdir/ind2 dir/ind1 && git commit -am "test" && - echo $hello_sha1 blob $hello_size >found + echo $hello_oid blob $hello_size >found ' test_expect_success 'git cat-file --batch-check --follow-symlinks works for non-links' ' @@ -941,7 +941,7 @@ test_expect_success 'git cat-file --batch-check --follow-symlinks works for dir/ echo HEAD:dirlink/morx >>expect && echo HEAD:dirlink/morx | git cat-file --batch-check --follow-symlinks >actual && test_cmp expect actual && - echo $hello_sha1 blob $hello_size >expect && + echo $hello_oid blob $hello_size >expect && echo HEAD:dirlink/ind1 | git cat-file --batch-check --follow-symlinks >actual && test_cmp expect actual ' From patchwork Mon Oct 2 02:40:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13405548 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 60FB6E784A6 for ; Mon, 2 Oct 2023 02:41:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235519AbjJBCl5 (ORCPT ); Sun, 1 Oct 2023 22:41:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38944 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235475AbjJBCla (ORCPT ); Sun, 1 Oct 2023 22:41:30 -0400 Received: from mail-io1-xd32.google.com (mail-io1-xd32.google.com [IPv6:2607:f8b0:4864:20::d32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5CA67CE7 for ; Sun, 1 Oct 2023 19:41:05 -0700 (PDT) Received: by mail-io1-xd32.google.com with SMTP id ca18e2360f4ac-79faba5fe12so473922239f.3 for ; Sun, 01 Oct 2023 19:41:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696214464; x=1696819264; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=iIp5xN1/FiLxLp7U+weLwdU6T91ZE64ygxLvw8ASvC4=; b=Pbr/hdjvriikTKIcVB4S5gCPnqCYwu7RIEYqlZyzG9CYJ7WnrGqR+zE8Ej8nxQBxW2 AOQOJ5oBsTzm6+L7WSeddKfOmzqlMLIWOkmCZCwodoOLE+woo63hu+fxwtUTfYLVzvyk KND/rN+H5jGQCnH7LHvFvpB+JjWJ6p6+YH18m3i/UkUR+JYh7qTR/uJRN1AGbqFr2sBe wKe190w0wKj3kZ69EOwG+amZl9xhKeyicJM40Mtmj2BAJ3Ogxc8AebgR2wNFES8tpEiy z7moJx3cLCQf3evk0G4N6bSFzDdn2BxfRTMZR9WDAVfxT+c22wh5Aa27q5Udmvgggv8Y Wtzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696214464; x=1696819264; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=iIp5xN1/FiLxLp7U+weLwdU6T91ZE64ygxLvw8ASvC4=; b=k2SGqgbjBa/nMnDSnvNw/USmFguisVYqg3R8wr1rZIjSZ3Disx4n3xF9jU7slo6lOK IPb1xrCyqY6omZoJwqF/oOAjI9Fy8eXorZET1Ci/lPxutQ3HtXDX820k4ratGGMIYHO2 pOdKNQ7/0ydUACI9ckZBGcxLf58XnKq4D3FftVJjI6+IDf2luOP3v7bGAUl6pPQIUWbY La9hn9zRROMO4swA1n2WfFOc8fH4NJ4AJo5mLmYjGo+4d71GOLqxPE9uev0lBcUF4DNl td9jGbSJi7oZbficWIYuhii+wd6c5fH8ZcHLqMF7NqE7OjlC5K8U9uwU+0M3n0qSnyxd oiDw== X-Gm-Message-State: AOJu0YwXgb+LSteN3/YdPPGnYIt9fsjl6AE7rQnjUu2yWut2XIZoXCEe HX0BlqaWDFYrvM8K/EEN8Ak= X-Google-Smtp-Source: AGHT+IFfwhSGB/hE/r/F8bIeENRSx3HRHVPt8cEmHUtLkjsmruh6HTobQtsq4fB4k5ksfg05zsTVUQ== X-Received: by 2002:a6b:4405:0:b0:792:4cfc:178f with SMTP id r5-20020a6b4405000000b007924cfc178fmr12248304ioa.16.1696214464560; Sun, 01 Oct 2023 19:41:04 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id v6-20020a056602058600b0079f9f7a2565sm6346113iox.38.2023.10.01.19.41.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Oct 2023 19:41:04 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , Eric Sunshine , "Eric W. Biederman" Subject: [PATCH v2 29/30] t1006: test oid compatibility with cat-file Date: Sun, 1 Oct 2023 21:40:33 -0500 Message-Id: <20231002024034.2611-29-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> References: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "Eric W. Biederman" Update the existing tests that are oid based to test that cat-file works correctly with the normal oid and the compat_oid. Signed-off-by: "Eric W. Biederman" --- t/t1006-cat-file.sh | 251 +++++++++++++++++++++++++++----------------- 1 file changed, 154 insertions(+), 97 deletions(-) diff --git a/t/t1006-cat-file.sh b/t/t1006-cat-file.sh index 9b018b538950..23d3d37283bb 100755 --- a/t/t1006-cat-file.sh +++ b/t/t1006-cat-file.sh @@ -236,27 +236,38 @@ hello_size=$(strlen "$hello_content") hello_oid=$(echo_without_newline "$hello_content" | git hash-object --stdin) test_expect_success "setup" ' + git config core.repositoryformatversion 1 && + git config extensions.objectformat $test_hash_algo && + git config extensions.compatobjectformat $test_compat_hash_algo && echo_without_newline "$hello_content" > hello && git update-index --add hello ' -run_tests 'blob' $hello_oid $hello_size "$hello_content" "$hello_content" +run_blob_tests () { + oid=$1 -test_expect_success '--batch-command --buffer with flush for blob info' ' - echo "$hello_oid blob $hello_size" >expect && - test_write_lines "info $hello_oid" "flush" | + run_tests 'blob' $oid $hello_size "$hello_content" "$hello_content" + + test_expect_success '--batch-command --buffer with flush for blob info' ' + echo "$oid blob $hello_size" >expect && + test_write_lines "info $oid" "flush" | GIT_TEST_CAT_FILE_NO_FLUSH_ON_EXIT=1 \ git cat-file --batch-command --buffer >actual && test_cmp expect actual -' + ' -test_expect_success '--batch-command --buffer without flush for blob info' ' + test_expect_success '--batch-command --buffer without flush for blob info' ' touch output && - test_write_lines "info $hello_oid" | + test_write_lines "info $oid" | GIT_TEST_CAT_FILE_NO_FLUSH_ON_EXIT=1 \ git cat-file --batch-command --buffer >>output && test_must_be_empty output -' + ' +} + +hello_compat_oid=$(git rev-parse --output-object-format=$test_compat_hash_algo $hello_oid) +run_blob_tests $hello_oid +run_blob_tests $hello_compat_oid test_expect_success '--batch-check without %(rest) considers whole line' ' echo "$hello_oid blob $hello_size" >expect && @@ -267,35 +278,58 @@ test_expect_success '--batch-check without %(rest) considers whole line' ' ' tree_oid=$(git write-tree) +tree_compat_oid=$(git rev-parse --output-object-format=$test_compat_hash_algo $tree_oid) tree_size=$(($(test_oid rawsz) + 13)) +tree_compat_size=$(($(test_oid --hash=compat rawsz) + 13)) tree_pretty_content="100644 blob $hello_oid hello${LF}" +tree_compat_pretty_content="100644 blob $hello_compat_oid hello${LF}" run_tests 'tree' $tree_oid $tree_size "" "$tree_pretty_content" +run_tests 'tree' $tree_compat_oid $tree_compat_size "" "$tree_compat_pretty_content" commit_message="Initial commit" commit_oid=$(echo_without_newline "$commit_message" | git commit-tree $tree_oid) +commit_compat_oid=$(git rev-parse --output-object-format=$test_compat_hash_algo $commit_oid) commit_size=$(($(test_oid hexsz) + 137)) +commit_compat_size=$(($(test_oid --hash=compat hexsz) + 137)) commit_content="tree $tree_oid author $GIT_AUTHOR_NAME <$GIT_AUTHOR_EMAIL> $GIT_AUTHOR_DATE committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE $commit_message" +commit_compat_content="tree $tree_compat_oid +author $GIT_AUTHOR_NAME <$GIT_AUTHOR_EMAIL> $GIT_AUTHOR_DATE +committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE + +$commit_message" + run_tests 'commit' $commit_oid $commit_size "$commit_content" "$commit_content" +run_tests 'commit' $commit_compat_oid $commit_compat_size "$commit_compat_content" "$commit_compat_content" -tag_header_without_timestamp="object $hello_oid -type blob +tag_header_without_oid="type blob tag hellotag tagger $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL>" +tag_header_without_timestamp="object $hello_oid +$tag_header_without_oid" +tag_compat_header_without_timestamp="object $hello_compat_oid +$tag_header_without_oid" tag_description="This is a tag" tag_content="$tag_header_without_timestamp 0 +0000 +$tag_description" +tag_compat_content="$tag_compat_header_without_timestamp 0 +0000 + $tag_description" tag_oid=$(echo_without_newline "$tag_content" | git hash-object -t tag --stdin -w) tag_size=$(strlen "$tag_content") +tag_compat_oid=$(git rev-parse --output-object-format=$test_compat_hash_algo $tag_oid) +tag_compat_size=$(strlen "$tag_compat_content") + run_tests 'tag' $tag_oid $tag_size "$tag_content" "$tag_content" +run_tests 'tag' $tag_compat_oid $tag_compat_size "$tag_compat_content" "$tag_compat_content" test_expect_success "Reach a blob from a tag pointing to it" ' echo_without_newline "$hello_content" >expect && @@ -303,37 +337,43 @@ test_expect_success "Reach a blob from a tag pointing to it" ' test_cmp expect actual ' -for batch in batch batch-check batch-command +for oid in $hello_oid $hello_compat_oid do - for opt in t s e p + for batch in batch batch-check batch-command do + for opt in t s e p + do test_expect_success "Passing -$opt with --$batch fails" ' - test_must_fail git cat-file --$batch -$opt $hello_oid + test_must_fail git cat-file --$batch -$opt $oid ' test_expect_success "Passing --$batch with -$opt fails" ' - test_must_fail git cat-file -$opt --$batch $hello_oid + test_must_fail git cat-file -$opt --$batch $oid ' - done + done - test_expect_success "Passing with --$batch fails" ' - test_must_fail git cat-file --$batch blob $hello_oid - ' + test_expect_success "Passing with --$batch fails" ' + test_must_fail git cat-file --$batch blob $oid + ' - test_expect_success "Passing --$batch with fails" ' - test_must_fail git cat-file blob --$batch $hello_oid - ' + test_expect_success "Passing --$batch with fails" ' + test_must_fail git cat-file blob --$batch $oid + ' - test_expect_success "Passing oid with --$batch fails" ' - test_must_fail git cat-file --$batch $hello_oid - ' + test_expect_success "Passing oid with --$batch fails" ' + test_must_fail git cat-file --$batch $oid + ' + done done -for opt in t s e p +for oid in $hello_oid $hello_compat_oid do - test_expect_success "Passing -$opt with --follow-symlinks fails" ' - test_must_fail git cat-file --follow-symlinks -$opt $hello_oid + for opt in t s e p + do + test_expect_success "Passing -$opt with --follow-symlinks fails" ' + test_must_fail git cat-file --follow-symlinks -$opt $oid ' + done done test_expect_success "--batch-check for a non-existent named object" ' @@ -386,112 +426,102 @@ test_expect_success 'empty --batch-check notices missing object' ' test_cmp expect actual ' -batch_input="$hello_oid -$commit_oid -$tag_oid +batch_tests () { + boid=$1 + loid=$2 + lsize=$3 + coid=$4 + csize=$5 + ccontent=$6 + toid=$7 + tsize=$8 + tcontent=$9 + + batch_input="$boid +$coid +$toid deadbeef " -printf "%s\0" \ - "$hello_oid blob $hello_size" \ + printf "%s\0" \ + "$boid blob $hello_size" \ "$hello_content" \ - "$commit_oid commit $commit_size" \ - "$commit_content" \ - "$tag_oid tag $tag_size" \ - "$tag_content" \ + "$coid commit $csize" \ + "$ccontent" \ + "$toid tag $tsize" \ + "$tcontent" \ "deadbeef missing" \ " missing" >batch_output -test_expect_success '--batch with multiple oids gives correct format' ' + test_expect_success '--batch with multiple oids gives correct format' ' tr "\0" "\n" expect && echo_without_newline "$batch_input" >in && git cat-file --batch actual && test_cmp expect actual -' + ' -test_expect_success '--batch, -z with multiple oids gives correct format' ' + test_expect_success '--batch, -z with multiple oids gives correct format' ' echo_without_newline_nul "$batch_input" >in && tr "\0" "\n" expect && git cat-file --batch -z actual && test_cmp expect actual -' + ' -test_expect_success '--batch, -Z with multiple oids gives correct format' ' + test_expect_success '--batch, -Z with multiple oids gives correct format' ' echo_without_newline_nul "$batch_input" >in && git cat-file --batch -Z actual && test_cmp batch_output actual -' + ' -batch_check_input="$hello_oid -$tree_oid -$commit_oid -$tag_oid +batch_check_input="$boid +$loid +$coid +$toid deadbeef " -printf "%s\0" \ - "$hello_oid blob $hello_size" \ - "$tree_oid tree $tree_size" \ - "$commit_oid commit $commit_size" \ - "$tag_oid tag $tag_size" \ + printf "%s\0" \ + "$boid blob $hello_size" \ + "$loid tree $lsize" \ + "$coid commit $csize" \ + "$toid tag $tsize" \ "deadbeef missing" \ " missing" >batch_check_output -test_expect_success "--batch-check with multiple oids gives correct format" ' + test_expect_success "--batch-check with multiple oids gives correct format" ' tr "\0" "\n" expect && echo_without_newline "$batch_check_input" >in && git cat-file --batch-check actual && test_cmp expect actual -' + ' -test_expect_success "--batch-check, -z with multiple oids gives correct format" ' + test_expect_success "--batch-check, -z with multiple oids gives correct format" ' tr "\0" "\n" expect && echo_without_newline_nul "$batch_check_input" >in && git cat-file --batch-check -z actual && test_cmp expect actual -' + ' -test_expect_success "--batch-check, -Z with multiple oids gives correct format" ' + test_expect_success "--batch-check, -Z with multiple oids gives correct format" ' echo_without_newline_nul "$batch_check_input" >in && git cat-file --batch-check -Z actual && test_cmp batch_check_output actual -' - -test_expect_success FUNNYNAMES 'setup with newline in input' ' - touch -- "newline${LF}embedded" && - git add -- "newline${LF}embedded" && - git commit -m "file with newline embedded" && - test_tick && - - printf "HEAD:newline${LF}embedded" >in -' - -test_expect_success FUNNYNAMES '--batch-check, -z with newline in input' ' - git cat-file --batch-check -z actual && - echo "$(git rev-parse "HEAD:newline${LF}embedded") blob 0" >expect && - test_cmp expect actual -' - -test_expect_success FUNNYNAMES '--batch-check, -Z with newline in input' ' - git cat-file --batch-check -Z actual && - printf "%s\0" "$(git rev-parse "HEAD:newline${LF}embedded") blob 0" >expect && - test_cmp expect actual -' + ' -batch_command_multiple_info="info $hello_oid -info $tree_oid -info $commit_oid -info $tag_oid +batch_command_multiple_info="info $boid +info $loid +info $coid +info $toid info deadbeef" -test_expect_success '--batch-command with multiple info calls gives correct format' ' + test_expect_success '--batch-command with multiple info calls gives correct format' ' cat >expect <<-EOF && - $hello_oid blob $hello_size - $tree_oid tree $tree_size - $commit_oid commit $commit_size - $tag_oid tag $tag_size + $boid blob $hello_size + $loid tree $lsize + $coid commit $csize + $toid tag $tsize deadbeef missing EOF @@ -510,22 +540,22 @@ test_expect_success '--batch-command with multiple info calls gives correct form git cat-file --batch-command --buffer -Z actual && test_cmp expect_nul actual -' + ' -batch_command_multiple_contents="contents $hello_oid -contents $commit_oid -contents $tag_oid +batch_command_multiple_contents="contents $boid +contents $coid +contents $toid contents deadbeef flush" -test_expect_success '--batch-command with multiple command calls gives correct format' ' + test_expect_success '--batch-command with multiple command calls gives correct format' ' printf "%s\0" \ - "$hello_oid blob $hello_size" \ + "$boid blob $hello_size" \ "$hello_content" \ - "$commit_oid commit $commit_size" \ - "$commit_content" \ - "$tag_oid tag $tag_size" \ - "$tag_content" \ + "$coid commit $csize" \ + "$ccontent" \ + "$toid tag $tsize" \ + "$tcontent" \ "deadbeef missing" >expect_nul && tr "\0" "\n" expect && @@ -543,6 +573,33 @@ test_expect_success '--batch-command with multiple command calls gives correct f git cat-file --batch-command --buffer -Z actual && test_cmp expect_nul actual + ' + +} + +batch_tests $hello_oid $tree_oid $tree_size $commit_oid $commit_size "$commit_content" $tag_oid $tag_size "$tag_content" +batch_tests $hello_compat_oid $tree_compat_oid $tree_compat_size $commit_compat_oid $commit_compat_size "$commit_compat_content" $tag_compat_oid $tag_compat_size "$tag_compat_content" + + +test_expect_success FUNNYNAMES 'setup with newline in input' ' + touch -- "newline${LF}embedded" && + git add -- "newline${LF}embedded" && + git commit -m "file with newline embedded" && + test_tick && + + printf "HEAD:newline${LF}embedded" >in +' + +test_expect_success FUNNYNAMES '--batch-check, -z with newline in input' ' + git cat-file --batch-check -z actual && + echo "$(git rev-parse "HEAD:newline${LF}embedded") blob 0" >expect && + test_cmp expect actual +' + +test_expect_success FUNNYNAMES '--batch-check, -Z with newline in input' ' + git cat-file --batch-check -Z actual && + printf "%s\0" "$(git rev-parse "HEAD:newline${LF}embedded") blob 0" >expect && + test_cmp expect actual ' test_expect_success 'setup blobs which are likely to delta' ' From patchwork Mon Oct 2 02:40:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13405550 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C9742E784A7 for ; Mon, 2 Oct 2023 02:42:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235483AbjJBCmI (ORCPT ); Sun, 1 Oct 2023 22:42:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35724 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235529AbjJBCla (ORCPT ); Sun, 1 Oct 2023 22:41:30 -0400 Received: from mail-io1-xd2e.google.com (mail-io1-xd2e.google.com [IPv6:2607:f8b0:4864:20::d2e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 14AABCEC for ; Sun, 1 Oct 2023 19:41:06 -0700 (PDT) Received: by mail-io1-xd2e.google.com with SMTP id ca18e2360f4ac-79f909071c3so570339339f.0 for ; Sun, 01 Oct 2023 19:41:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696214465; x=1696819265; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Uc1k82xvBrnb7Z3un/5fTBvAnqrfiI/m0uwTG7KGIlE=; b=Uz5FQne7QoTdHcpIwtvavJ19BYp6/h2dD7mwnJs4P2oedmT2smKh1zK5Y2r4HVpPw7 BdXRc2yWjE7KvwFe2wDZExlIHQ1n8g+SpTV/Np1oZsvfQMpGhVhCJ6ghWLmd8JHIHfsw mDONutSnMu24uFIAvq+ZwbiC1hynNs6qIVicmGfffomcaaRFyh/3QScioNmScm5hPz+8 rLr86A5WOxJWfXxlaIPEcV+ifkHxujKcpTswnSgGJ/qZs79QGkahPz89ogcUytgX7MTU r65FPImVG7XTKEo7MNXGkL/XKKj9rhByEqktx/cmtVYjQWGPTjWc4AKV0k8qdn7Qt9f2 mGlw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696214465; x=1696819265; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Uc1k82xvBrnb7Z3un/5fTBvAnqrfiI/m0uwTG7KGIlE=; b=j2bfecgYlqwh1rG3QJIAINlE4hhNIYjtLUlZ1J/bd3hondULv56fkCNa2LGNLdkAxk cTNUXnh+TCJqrM2++b86V9G/aM1qS1q00V99c18cfEosxa7M0ibSrJgyGRG1FHfpqUHo /Tthg0Fo2rd0f0BAEp0E3/lHfe36UCLtPkmbdjL9qsTeUADPaqMOx8nk2XH0cysguxA3 /n8QGw1qmtUTDLJIIpPWKPBzojpey5LdH0g/60T1wS40sJfg14L/tF7MhZEB/O3ZDX4i a4rpLamJA0HBCwmbaEYoIQeaBI8cce0PgirfdVwvXrY8E0EyJLpxRWCO2WvjFIrHhrvM KREA== X-Gm-Message-State: AOJu0YxZmyju2rzMwarAeHhNHP5TsE2DCH818EngwjkFJirQ1Ykh+kpJ YB78MuGvbGY8D7O1zuD9CZI= X-Google-Smtp-Source: AGHT+IGJZLtkpDBvveaUJfGfF4mx84Heyrm5ngtpa/40IEpANYrGrM4RKRGfSLSwdGmCgN3fwL+ScQ== X-Received: by 2002:a6b:4405:0:b0:785:5917:a35f with SMTP id r5-20020a6b4405000000b007855917a35fmr12552355ioa.8.1696214465380; Sun, 01 Oct 2023 19:41:05 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id v6-20020a056602058600b0079f9f7a2565sm6346113iox.38.2023.10.01.19.41.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Oct 2023 19:41:04 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , Eric Sunshine , "Eric W. Biederman" Subject: [PATCH v2 30/30] t1016-compatObjectFormat: add tests to verify the conversion between objects Date: Sun, 1 Oct 2023 21:40:34 -0500 Message-Id: <20231002024034.2611-30-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> References: <878r8l929e.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "Eric W. Biederman" For now my strategy is simple. Create two identical repositories one in each format. Use fixed timestamps. Verify the dynamically computed compatibility objects from one repository match the objects stored in the other repository. A general limitation of this strategy is that the git when generating signed tags and commits with compatObjectFormat enabled will generate a signature for both formats. To overcome this limitation I have added "test-tool delete-gpgsig" that when fed an signed commit or tag with two signatures deletes one of the signatures. With that in place I can have "git commit" and "git tag" generate signed objects, have my tool delete one, and feed the new object into "git hash-object" to create the kinds of commits and tags git without compatObjectFormat enabled will generate. Signed-off-by: "Eric W. Biederman" --- Makefile | 1 + t/helper/test-delete-gpgsig.c | 62 ++++++++ t/helper/test-tool.c | 1 + t/helper/test-tool.h | 1 + t/t1016-compatObjectFormat.sh | 281 ++++++++++++++++++++++++++++++++++ t/t1016/gpg | 2 + 6 files changed, 348 insertions(+) create mode 100644 t/helper/test-delete-gpgsig.c create mode 100755 t/t1016-compatObjectFormat.sh create mode 100755 t/t1016/gpg diff --git a/Makefile b/Makefile index 3c18664def9a..3e4444fb9ab2 100644 --- a/Makefile +++ b/Makefile @@ -790,6 +790,7 @@ TEST_BUILTINS_OBJS += test-crontab.o TEST_BUILTINS_OBJS += test-csprng.o TEST_BUILTINS_OBJS += test-ctype.o TEST_BUILTINS_OBJS += test-date.o +TEST_BUILTINS_OBJS += test-delete-gpgsig.o TEST_BUILTINS_OBJS += test-delta.o TEST_BUILTINS_OBJS += test-dir-iterator.o TEST_BUILTINS_OBJS += test-drop-caches.o diff --git a/t/helper/test-delete-gpgsig.c b/t/helper/test-delete-gpgsig.c new file mode 100644 index 000000000000..e36831af03f6 --- /dev/null +++ b/t/helper/test-delete-gpgsig.c @@ -0,0 +1,62 @@ +#include "test-tool.h" +#include "gpg-interface.h" +#include "strbuf.h" + + +int cmd__delete_gpgsig(int argc, const char **argv) +{ + struct strbuf buf = STRBUF_INIT; + const char *pattern = "gpgsig"; + const char *bufptr, *tail, *eol; + int deleting = 0; + size_t plen; + + if (argc >= 2) { + pattern = argv[1]; + argv++; + argc--; + } + + plen = strlen(pattern); + strbuf_read(&buf, 0, 0); + + if (!strcmp(pattern, "trailer")) { + size_t payload_size = parse_signed_buffer(buf.buf, buf.len); + fwrite(buf.buf, 1, payload_size, stdout); + fflush(stdout); + return 0; + } + + bufptr = buf.buf; + tail = bufptr + buf.len; + + while (bufptr < tail) { + /* Find the end of the line */ + eol = memchr(bufptr, '\n', tail - bufptr); + if (!eol) + eol = tail; + + /* Drop continuation lines */ + if (deleting && (bufptr < eol) && (bufptr[0] == ' ')) { + bufptr = eol + 1; + continue; + } + deleting = 0; + + /* Does the line match the prefix? */ + if (((bufptr + plen) < eol) && + !memcmp(bufptr, pattern, plen) && + (bufptr[plen] == ' ')) { + deleting = 1; + bufptr = eol + 1; + continue; + } + + /* Print all other lines */ + fwrite(bufptr, 1, (eol - bufptr) + 1, stdout); + bufptr = eol + 1; + } + fflush(stdout); + + return 0; +} diff --git a/t/helper/test-tool.c b/t/helper/test-tool.c index abe8a785eb65..8b6c84f202d6 100644 --- a/t/helper/test-tool.c +++ b/t/helper/test-tool.c @@ -21,6 +21,7 @@ static struct test_cmd cmds[] = { { "csprng", cmd__csprng }, { "ctype", cmd__ctype }, { "date", cmd__date }, + { "delete-gpgsig", cmd__delete_gpgsig }, { "delta", cmd__delta }, { "dir-iterator", cmd__dir_iterator }, { "drop-caches", cmd__drop_caches }, diff --git a/t/helper/test-tool.h b/t/helper/test-tool.h index ea2672436c9a..76baaece35b9 100644 --- a/t/helper/test-tool.h +++ b/t/helper/test-tool.h @@ -15,6 +15,7 @@ int cmd__csprng(int argc, const char **argv); int cmd__ctype(int argc, const char **argv); int cmd__date(int argc, const char **argv); int cmd__delta(int argc, const char **argv); +int cmd__delete_gpgsig(int argc, const char **argv); int cmd__dir_iterator(int argc, const char **argv); int cmd__drop_caches(int argc, const char **argv); int cmd__dump_cache_tree(int argc, const char **argv); diff --git a/t/t1016-compatObjectFormat.sh b/t/t1016-compatObjectFormat.sh new file mode 100755 index 000000000000..8132cd37b8c8 --- /dev/null +++ b/t/t1016-compatObjectFormat.sh @@ -0,0 +1,281 @@ +#!/bin/sh +# +# Copyright (c) 2023 Eric Biederman +# + +test_description='Test how well compatObjectFormat works' + +TEST_PASSES_SANITIZE_LEAK=true +. ./test-lib.sh +. "$TEST_DIRECTORY"/lib-gpg.sh + +# All of the follow variables must be defined in the environment: +# GIT_AUTHOR_NAME +# GIT_AUTHOR_EMAIL +# GIT_AUTHOR_DATE +# GIT_COMMITTER_NAME +# GIT_COMMITTER_EMAIL +# GIT_COMMITTER_DATE +# +# The test relies on these variables being set so that the two +# different commits in two different repositories encoded with two +# different hash functions result in the same content in the commits. +# This means that when the commit is translated between hash functions +# the commit is identical to the commit in the other repository. + +compat_hash () { + case "$1" in + "sha1") + echo "sha256" + ;; + "sha256") + echo "sha1" + ;; + esac +} + +hello_oid () { + case "$1" in + "sha1") + echo "$hello_sha1_oid" + ;; + "sha256") + echo "$hello_sha256_oid" + ;; + esac +} + +tree_oid () { + case "$1" in + "sha1") + echo "$tree_sha1_oid" + ;; + "sha256") + echo "$tree_sha256_oid" + ;; + esac +} + +commit_oid () { + case "$1" in + "sha1") + echo "$commit_sha1_oid" + ;; + "sha256") + echo "$commit_sha256_oid" + ;; + esac +} + +commit2_oid () { + case "$1" in + "sha1") + echo "$commit2_sha1_oid" + ;; + "sha256") + echo "$commit2_sha256_oid" + ;; + esac +} + +del_sigcommit () { + local delete=$1 + + if test "$delete" = "sha256" ; then + local pattern="gpgsig-sha256" + else + local pattern="gpgsig" + fi + test-tool delete-gpgsig "$pattern" +} + + +del_sigtag () { + local storage=$1 + local delete=$2 + + if test "$storage" = "$delete" ; then + local pattern="trailer" + elif test "$storage" = "sha256" ; then + local pattern="gpgsig" + else + local pattern="gpgsig-sha256" + fi + test-tool delete-gpgsig "$pattern" +} + +base=$(pwd) +for hash in sha1 sha256 +do + cd "$base" + mkdir -p repo-$hash + cd repo-$hash + + test_expect_success "setup $hash repository" ' + git init --object-format=$hash && + git config core.repositoryformatversion 1 && + git config extensions.objectformat $hash && + git config extensions.compatobjectformat $(compat_hash $hash) && + git config gpg.program $TEST_DIRECTORY/t1016/gpg && + echo "Hellow World!" > hello && + eval hello_${hash}_oid=$(git hash-object hello) && + git update-index --add hello && + git commit -m "Initial commit" && + eval commit_${hash}_oid=$(git rev-parse HEAD) && + eval tree_${hash}_oid=$(git rev-parse HEAD^{tree}) + ' + test_expect_success "create a $hash tagged blob" ' + git tag --no-sign -m "This is a tag" hellotag $(hello_oid $hash) && + eval hellotag_${hash}_oid=$(git rev-parse hellotag) + ' + test_expect_success "create a $hash tagged tree" ' + git tag --no-sign -m "This is a tag" treetag $(tree_oid $hash) && + eval treetag_${hash}_oid=$(git rev-parse treetag) + ' + test_expect_success "create a $hash tagged commit" ' + git tag --no-sign -m "This is a tag" committag $(commit_oid $hash) && + eval committag_${hash}_oid=$(git rev-parse committag) + ' + test_expect_success GPG2 "create a $hash signed commit" ' + git commit --gpg-sign --allow-empty -m "This is a signed commit" && + eval signedcommit_${hash}_oid=$(git rev-parse HEAD) + ' + test_expect_success GPG2 "create a $hash signed tag" ' + git tag -s -m "This is a signed tag" signedtag HEAD && + eval signedtag_${hash}_oid=$(git rev-parse signedtag) + ' + test_expect_success "create a $hash branch" ' + git checkout -b branch $(commit_oid $hash) && + echo "More more more give me more!" > more && + eval more_${hash}_oid=$(git hash-object more) && + echo "Another and another and another" > another && + eval another_${hash}_oid=$(git hash-object another) && + git update-index --add more another && + git commit -m "Add more files!" && + eval commit2_${hash}_oid=$(git rev-parse HEAD) && + eval tree2_${hash}_oid=$(git rev-parse HEAD^{tree}) + ' + test_expect_success GPG2 "create another $hash signed tag" ' + git tag -s -m "This is another signed tag" signedtag2 $(commit2_oid $hash) && + eval signedtag2_${hash}_oid=$(git rev-parse signedtag2) + ' + test_expect_success GPG2 "merge the $hash branches together" ' + git merge -S -m "merge some signed tags together" signedtag signedtag2 && + eval signedcommit2_${hash}_oid=$(git rev-parse HEAD) + ' + test_expect_success GPG2 "create additional $hash signed commits" ' + git commit --gpg-sign --allow-empty -m "This is an additional signed commit" && + git cat-file commit HEAD | del_sigcommit sha256 > "../${hash}_signedcommit3" && + git cat-file commit HEAD | del_sigcommit sha1 > "../${hash}_signedcommit4" && + eval signedcommit3_${hash}_oid=$(git hash-object -t commit -w ../${hash}_signedcommit3) && + eval signedcommit4_${hash}_oid=$(git hash-object -t commit -w ../${hash}_signedcommit4) + ' + test_expect_success GPG2 "create additional $hash signed tags" ' + git tag -s -m "This is an additional signed tag" signedtag34 HEAD && + git cat-file tag signedtag34 | del_sigtag "${hash}" sha256 > ../${hash}_signedtag3 && + git cat-file tag signedtag34 | del_sigtag "${hash}" sha1 > ../${hash}_signedtag4 && + eval signedtag3_${hash}_oid=$(git hash-object -t tag -w ../${hash}_signedtag3) && + eval signedtag4_${hash}_oid=$(git hash-object -t tag -w ../${hash}_signedtag4) + ' +done +cd "$base" + +compare_oids () { + test "$#" = 5 && { local PREREQ=$1; shift; } || PREREQ= + local type="$1" + local name="$2" + local sha1_oid="$3" + local sha256_oid="$4" + + echo ${sha1_oid} > ${name}_sha1_expected + echo ${sha256_oid} > ${name}_sha256_expected + echo ${type} > ${name}_type_expected + + git --git-dir=repo-sha1/.git rev-parse --output-object-format=sha256 ${sha1_oid} > ${name}_sha1_sha256_found + git --git-dir=repo-sha256/.git rev-parse --output-object-format=sha1 ${sha256_oid} > ${name}_sha256_sha1_found + local sha1_sha256_oid=$(cat ${name}_sha1_sha256_found) + local sha256_sha1_oid=$(cat ${name}_sha256_sha1_found) + + test_expect_success $PREREQ "Verify ${type} ${name}'s sha1 oid" ' + git --git-dir=repo-sha256/.git rev-parse --output-object-format=sha1 ${sha256_oid} > ${name}_sha1 && + test_cmp ${name}_sha1 ${name}_sha1_expected +' + + test_expect_success $PREREQ "Verify ${type} ${name}'s sha256 oid" ' + git --git-dir=repo-sha1/.git rev-parse --output-object-format=sha256 ${sha1_oid} > ${name}_sha256 && + test_cmp ${name}_sha256 ${name}_sha256_expected +' + + test_expect_success $PREREQ "Verify ${name}'s sha1 type" ' + git --git-dir=repo-sha1/.git cat-file -t ${sha1_oid} > ${name}_type1 && + git --git-dir=repo-sha256/.git cat-file -t ${sha256_sha1_oid} > ${name}_type2 && + test_cmp ${name}_type1 ${name}_type2 && + test_cmp ${name}_type1 ${name}_type_expected +' + + test_expect_success $PREREQ "Verify ${name}'s sha256 type" ' + git --git-dir=repo-sha256/.git cat-file -t ${sha256_oid} > ${name}_type3 && + git --git-dir=repo-sha1/.git cat-file -t ${sha1_sha256_oid} > ${name}_type4 && + test_cmp ${name}_type3 ${name}_type4 && + test_cmp ${name}_type3 ${name}_type_expected +' + + test_expect_success $PREREQ "Verify ${name}'s sha1 size" ' + git --git-dir=repo-sha1/.git cat-file -s ${sha1_oid} > ${name}_size1 && + git --git-dir=repo-sha256/.git cat-file -s ${sha256_sha1_oid} > ${name}_size2 && + test_cmp ${name}_size1 ${name}_size2 +' + + test_expect_success $PREREQ "Verify ${name}'s sha256 size" ' + git --git-dir=repo-sha256/.git cat-file -s ${sha256_oid} > ${name}_size3 && + git --git-dir=repo-sha1/.git cat-file -s ${sha1_sha256_oid} > ${name}_size4 && + test_cmp ${name}_size3 ${name}_size4 +' + + test_expect_success $PREREQ "Verify ${name}'s sha1 pretty content" ' + git --git-dir=repo-sha1/.git cat-file -p ${sha1_oid} > ${name}_content1 && + git --git-dir=repo-sha256/.git cat-file -p ${sha256_sha1_oid} > ${name}_content2 && + test_cmp ${name}_content1 ${name}_content2 +' + + test_expect_success $PREREQ "Verify ${name}'s sha256 pretty content" ' + git --git-dir=repo-sha256/.git cat-file -p ${sha256_oid} > ${name}_content3 && + git --git-dir=repo-sha1/.git cat-file -p ${sha1_sha256_oid} > ${name}_content4 && + test_cmp ${name}_content3 ${name}_content4 +' + + test_expect_success $PREREQ "Verify ${name}'s sha1 content" ' + git --git-dir=repo-sha1/.git cat-file ${type} ${sha1_oid} > ${name}_content5 && + git --git-dir=repo-sha256/.git cat-file ${type} ${sha256_sha1_oid} > ${name}_content6 && + test_cmp ${name}_content5 ${name}_content6 +' + + test_expect_success $PREREQ "Verify ${name}'s sha256 content" ' + git --git-dir=repo-sha256/.git cat-file ${type} ${sha256_oid} > ${name}_content7 && + git --git-dir=repo-sha1/.git cat-file ${type} ${sha1_sha256_oid} > ${name}_content8 && + test_cmp ${name}_content7 ${name}_content8 +' + +} + +compare_oids 'blob' hello "$hello_sha1_oid" "$hello_sha256_oid" +compare_oids 'tree' tree "$tree_sha1_oid" "$tree_sha256_oid" +compare_oids 'commit' commit "$commit_sha1_oid" "$commit_sha256_oid" +compare_oids GPG2 'commit' signedcommit "$signedcommit_sha1_oid" "$signedcommit_sha256_oid" +compare_oids 'tag' hellotag "$hellotag_sha1_oid" "$hellotag_sha256_oid" +compare_oids 'tag' treetag "$treetag_sha1_oid" "$treetag_sha256_oid" +compare_oids 'tag' committag "$committag_sha1_oid" "$committag_sha256_oid" +compare_oids GPG2 'tag' signedtag "$signedtag_sha1_oid" "$signedtag_sha256_oid" + +compare_oids 'blob' more "$more_sha1_oid" "$more_sha256_oid" +compare_oids 'blob' another "$another_sha1_oid" "$another_sha256_oid" +compare_oids 'tree' tree2 "$tree2_sha1_oid" "$tree2_sha256_oid" +compare_oids 'commit' commit2 "$commit2_sha1_oid" "$commit2_sha256_oid" +compare_oids GPG2 'tag' signedtag2 "$signedtag2_sha1_oid" "$signedtag2_sha256_oid" +compare_oids GPG2 'commit' signedcommit2 "$signedcommit2_sha1_oid" "$signedcommit2_sha256_oid" +compare_oids GPG2 'commit' signedcommit3 "$signedcommit3_sha1_oid" "$signedcommit3_sha256_oid" +compare_oids GPG2 'commit' signedcommit4 "$signedcommit4_sha1_oid" "$signedcommit4_sha256_oid" +compare_oids GPG2 'tag' signedtag3 "$signedtag3_sha1_oid" "$signedtag3_sha256_oid" +compare_oids GPG2 'tag' signedtag4 "$signedtag4_sha1_oid" "$signedtag4_sha256_oid" + +test_done diff --git a/t/t1016/gpg b/t/t1016/gpg new file mode 100755 index 000000000000..2601cb18a5b3 --- /dev/null +++ b/t/t1016/gpg @@ -0,0 +1,2 @@ +#!/bin/sh +exec gpg --faked-system-time "20230918T154812" "$@"