From patchwork Sat Sep 11 07:59:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ren=C3=A9_Scharfe?= X-Patchwork-Id: 12486191 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.3 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 09EEFC433F5 for ; Sat, 11 Sep 2021 07:59:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CD1B9611C0 for ; Sat, 11 Sep 2021 07:59:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235372AbhIKIAd (ORCPT ); Sat, 11 Sep 2021 04:00:33 -0400 Received: from mout.web.de ([212.227.15.3]:57443 "EHLO mout.web.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235331AbhIKIAc (ORCPT ); Sat, 11 Sep 2021 04:00:32 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=web.de; s=dbaedf251592; t=1631347156; bh=1ybyE/TefLvk4oGIRd+lwdl6BRhAhvdxk/TeEKwPKCI=; h=X-UI-Sender-Class:Subject:From:To:Cc:References:Date:In-Reply-To; b=qSlWuXNesm9sWn16p6HtFkFe6eg4NVyKlasKgnHFBJ7fcZcP9gu3jmWJHd172zqtZ qSxOtbKyr0TllUXSgv2RTaT28C8EIj6YyPoWWQEbvOhOWqAlLBX1MqEoQX3kuaZj39 9LgUUUamf/m1nv7GSuK8XhCmvdTLhy3K4B2vbBNs= X-UI-Sender-Class: c548c8c5-30a9-4db5-a2e7-cb6cb037b8f9 Received: from Mini-von-Rene.fritz.box ([79.203.20.171]) by smtp.web.de (mrweb002 [213.165.67.108]) with ESMTPSA (Nemesis) id 0LkyTl-1mxTij2Yrt-00als6; Sat, 11 Sep 2021 09:59:16 +0200 Subject: [PATCH 1/3] packfile: convert mark_bad_packed_object() to object_id From: =?utf-8?q?Ren=C3=A9_Scharfe?= To: Git List Cc: Junio C Hamano References: <4a702bfe-afd0-669a-c893-0262289c24b8@web.de> Message-ID: Date: Sat, 11 Sep 2021 09:59:16 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Thunderbird/78.14.0 MIME-Version: 1.0 In-Reply-To: <4a702bfe-afd0-669a-c893-0262289c24b8@web.de> Content-Language: en-US X-Provags-ID: V03:K1:MV5HI08Ukn+mNF2zaiyylLep/WQ+yDMya4babxOhLVGcydAZOts LijpjWPpmegad/MfNSVW6RKsfCPIbB5pw+9vkeYBWU2/0Yj3T7fFFgwlgznC/+l2i/8iPae n+wYhPq6V6n4wqFMM3RoGG2qdHaBPCNDAFUK4IRzv6Hc6/OtNMeXvf5HgyzCWnFbGJCi2+D qWjbiLcuMoKiyXRKW1piA== X-UI-Out-Filterresults: notjunk:1;V03:K0:3QD4rIegpps=:3NZrSYdLTVpwbkiaxmV2On n3NA03j3cm2jmuk+whKySiJ1bCXdiSnOlAqoak4gQG+8msLUija9Imqvi16oVZGoJcNEjG5Us P7UgvJP9WJfqsej3mOmFG1WX8pdTr6OVa6+nlRfzL5bO9ND5IshoQKs7mYOmPlspIy5qNGNhI Y14IBpjo0mWRhu2n7Zuc/wwfs9IdzUZtDX3S8LlYysjT8bXSZr3ro9zwJ8iXEK6wfHfw4g8CJ 1mBXd/5WClU4K1AyQ9NNAoVjxcHik1/wkH5INxf0uwUWp3EoYl5UYrto67FOxsiE5qsqoxGl4 86ARG3c3CiP4KFhFCZPmsIEYtLTxdcQnhnOUG9a1TCVTYVZlOyDT+dDvdGXLarifmNk0uzyPX 7gFj0BmKGK3K5f/kJXTaLWdi93MiMmHz0IItQi9Ixoi73tjs7Tor9KMEXOEP21EIeLtjrrPDp ts567BfcgHM1s3r9JZslrg34kBaGO09wUt/KNdZyPEjPsznEPE8wfMqqqnscG8vFDgQ2zflic aT7SkARSWPNOTBxFIGHg4jqYUXEhVtvMhMyUMbX8KH2lceVJjf2507L4hxIiNxlxFHI0xRIof X6s6C85XWAK3rzmql02rL/Lfgz6RT5bhDFFbR+SR7lNLarh5uViGt612zy50JclaIxoMApAYO 4kQNLf5H77b0E7ebteAAXVA8e7uLo0u+T3uUEtL9ITw94pyk/eWT0pJD0EMJ8mCYqtx3ewh+n B4m6RbWMM8qiC1FeBpEUQmBkDn4zknm/jzlPN77M5eDguNkBVBDiL1RXI3YdPP5uxERLJupa9 5fuHM2W22sr+AFD4OUyVwVp6Xlbse36JD1PWBpxOVrF6h0D2cIddi7zi+5ENbPI9VERI8+y/z Rm6PypEurWfMeYeiEyaqx55d268VMAKZp7Y5W4PclIXT15j7sHbKkjmJJosIP5ibuhsE73adb Qec7rWvCFCcVd4EgzJhpTo8AvPS32xVWj4cwe+8XQixacB7j9TKNENHTmRXLHbFaVsrWN7b8X lb9RSxt7yWtFr4v35kx2gGMPt7OwZIPdmV7UZbryWqfPpq4Sr1I2VM5GJEbUPPd1wA== Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org All callers have full object IDs, so pass them on instead of just their hash member. Signed-off-by: René Scharfe --- object-file.c | 2 +- packfile.c | 12 ++++++------ packfile.h | 2 +- 3 files changed, 8 insertions(+), 8 deletions(-) -- 2.33.0 diff --git a/object-file.c b/object-file.c index a8be899481..fb5a385a06 100644 --- a/object-file.c +++ b/object-file.c @@ -1616,7 +1616,7 @@ static int do_oid_object_info_extended(struct repository *r, return 0; rtype = packed_object_info(r, e.p, e.offset, oi); if (rtype < 0) { - mark_bad_packed_object(e.p, real->hash); + mark_bad_packed_object(e.p, real); return do_oid_object_info_extended(r, real, oi, 0); } else if (oi->whence == OI_PACKED) { oi->u.packed.offset = e.offset; diff --git a/packfile.c b/packfile.c index 4d0d625238..fb15fc5b49 100644 --- a/packfile.c +++ b/packfile.c @@ -1161,17 +1161,17 @@ int unpack_object_header(struct packed_git *p, return type; } -void mark_bad_packed_object(struct packed_git *p, const unsigned char *sha1) +void mark_bad_packed_object(struct packed_git *p, const struct object_id *oid) { unsigned i; const unsigned hashsz = the_hash_algo->rawsz; for (i = 0; i < p->num_bad_objects; i++) - if (hasheq(sha1, p->bad_object_sha1 + hashsz * i)) + if (hasheq(oid->hash, p->bad_object_sha1 + hashsz * i)) return; p->bad_object_sha1 = xrealloc(p->bad_object_sha1, st_mult(GIT_MAX_RAWSZ, st_add(p->num_bad_objects, 1))); - hashcpy(p->bad_object_sha1 + hashsz * p->num_bad_objects, sha1); + hashcpy(p->bad_object_sha1 + hashsz * p->num_bad_objects, oid->hash); p->num_bad_objects++; } @@ -1272,7 +1272,7 @@ static int retry_bad_packed_offset(struct repository *r, if (offset_to_pack_pos(p, obj_offset, &pos) < 0) return OBJ_BAD; nth_packed_object_id(&oid, p, pack_pos_to_index(p, pos)); - mark_bad_packed_object(p, oid.hash); + mark_bad_packed_object(p, &oid); type = oid_object_info(r, &oid, NULL); if (type <= OBJ_NONE) return OBJ_BAD; @@ -1722,7 +1722,7 @@ void *unpack_entry(struct repository *r, struct packed_git *p, off_t obj_offset, nth_packed_object_id(&oid, p, index_pos); error("bad packed object CRC for %s", oid_to_hex(&oid)); - mark_bad_packed_object(p, oid.hash); + mark_bad_packed_object(p, &oid); data = NULL; goto out; } @@ -1811,7 +1811,7 @@ void *unpack_entry(struct repository *r, struct packed_git *p, off_t obj_offset, " at offset %"PRIuMAX" from %s", oid_to_hex(&base_oid), (uintmax_t)obj_offset, p->pack_name); - mark_bad_packed_object(p, base_oid.hash); + mark_bad_packed_object(p, &base_oid); base = read_object(r, &base_oid, &type, &base_size); external_base = base; } diff --git a/packfile.h b/packfile.h index 3ae117a8ae..a982ed9994 100644 --- a/packfile.h +++ b/packfile.h @@ -159,7 +159,7 @@ int packed_object_info(struct repository *r, struct packed_git *pack, off_t offset, struct object_info *); -void mark_bad_packed_object(struct packed_git *p, const unsigned char *sha1); +void mark_bad_packed_object(struct packed_git *, const struct object_id *); const struct packed_git *has_packed_and_bad(struct repository *r, const unsigned char *sha1); #define ON_DISK_KEEP_PACKS 1 From patchwork Sat Sep 11 08:00:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ren=C3=A9_Scharfe?= X-Patchwork-Id: 12486193 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.3 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9479C433F5 for ; Sat, 11 Sep 2021 08:00:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 959DA611B0 for ; Sat, 11 Sep 2021 08:00:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235391AbhIKIBz (ORCPT ); Sat, 11 Sep 2021 04:01:55 -0400 Received: from mout.web.de ([212.227.15.4]:40933 "EHLO mout.web.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235331AbhIKIBz (ORCPT ); Sat, 11 Sep 2021 04:01:55 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=web.de; s=dbaedf251592; t=1631347238; bh=xJZnghI+6gjHEugvJNlOxXX5Xoz8chOUFnGa2sT0nRI=; h=X-UI-Sender-Class:Subject:From:To:Cc:References:Date:In-Reply-To; b=jwp8HFHu+3DKuOPVWthZS9K1iGX8t9Jtu2k/jKk6jLqS6E7AtDr9KC23bVnP/3sWi ZbylMiZNhZJaEKZ8UQC1z5+UHVS8m/QmS26KFYjKyFNmEq4VjBaZCclApctzLo1Eru KQ1nPfHbXBg+g+pL3+z0UJ7lmk3JanIYDbEzP2dg= X-UI-Sender-Class: c548c8c5-30a9-4db5-a2e7-cb6cb037b8f9 Received: from Mini-von-Rene.fritz.box ([79.203.20.171]) by smtp.web.de (mrweb002 [213.165.67.108]) with ESMTPSA (Nemesis) id 0M2dt7-1nF0pH0NpF-00sRHK; Sat, 11 Sep 2021 10:00:38 +0200 Subject: [PATCH 2/3] packfile: convert has_packed_and_bad() to object_id From: =?utf-8?q?Ren=C3=A9_Scharfe?= To: Git List Cc: Junio C Hamano References: <4a702bfe-afd0-669a-c893-0262289c24b8@web.de> Message-ID: <8d7328ee-57b5-247c-7b28-8d28ece8a4f2@web.de> Date: Sat, 11 Sep 2021 10:00:37 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Thunderbird/78.14.0 MIME-Version: 1.0 In-Reply-To: <4a702bfe-afd0-669a-c893-0262289c24b8@web.de> Content-Language: en-US X-Provags-ID: V03:K1:ltukPuk+plQHcckXVbDEsjxjax5BO8K3EDAlLp+NWVAOGveNpzc wdTNVgrZpA0Gs2/N11OcnUIyx+m8CwDakkIT8dsrDNTAbEN9ZIkY+bf5Ds2xE14cx+oaiwl pOOswR8no1/2V5FQK9M9hqk5wg3E6pZs4F52bWuMmRlkFiVh2LN8H8Z5TticjYscM5jeAUy a0egYKawR1RahSC3FuHTg== X-UI-Out-Filterresults: notjunk:1;V03:K0:e7xHWMxosGg=:wioYvQV22WDFcyzg3np29w YhFiO3u2GSoNKisPvPzN+kF/Kkejku87y52eKrjufvDrxkC8/Uu630Im0SyDH5rYObotrUd0q AW1twU8Lho+FWfQgvS1KHO4Aqbk5CSHj94b076bv7t3ZQNq0QqcRIEaYdMQaS+YbN4ZUzhvQS nyjsgcqpAVQ8XklLsb2Uf7a62UAs6S5gMT8l7PNSWAJVg/Mo7Kxf1NPvIjof+ZeG1P6lDnaXe R3a5zaT/MAhaWx1hXe7bhzONg+/bybS6lvgOiQfppcgf+sREgBKIhasRuFwyWxFY66Q2LkZwL yhP+2loY+m1t+2HNQl544YBj7SD879qev92H9JOyQq4+svxYpGvcE3CQq+QY/7YjVjkeFhYBi p/EXNwE7EfaKBn2MHQoUWUZHZYGJoxf7iIzYnMizy0+bFEzLB9kEbGegdhX/zTz9DaqNP/ibQ 3LTL/1ks1byC7ouRjUefnWyxyfrM/BBRzjntSBJ4muVywEjBT1EtzSxQFyB9PDMCWsHX8WkZl TGQzf2NozPCdl6GhSRA1Rs4Ww3ZLnvqJxNwodMiiCbMiFGBcliCfqTDlFjkc1YXYFqHLZnCT/ H4lGE9vFv+vdv8Y7AvlJZwSjXVSuZ/YBV/XNUiTsKcGr3YdvVswwl59OwVJJ1e+nN+uTAjRGy QTrPCaMMkVSb2TtZS9b1ViG1IeIZtQoOhOsQL3KtSGafjUHnVTLcBEzWNwq/eVJzrsTIIJY9M rV5IxbERDv+C96kLNPhsQYdW5pPVpKsrZtGsALZyqSrKY1TsWFbAybwxNgjQ9WTf6AuMHPZL7 5X1p7LUJKYzx5x31h7kPppzw/3KncN3CkM7M0SQZFnEe4eazcJY0dPSbEqDDQ8q/wV34uDHPf eoWGYoiUlRuOS3UrpcJtO5/GB5jFNqFHv5aw5hpT17y9mTZpFLjrikB+Yy0KaMe0UoqrXol5B UkUtZPpgrbTKTCIoWVWQ/3cFC6Z+ohySjS5mv4wZDfgqlM13DMKMr5V8uQ4RwItCpGLibRWP5 eLHuYCqwubvz+9Fx3MDWpD32c5VoTTIbnc05fzfJZ+yVg+9zGrhTYFGj/P3ExIvEpw== Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org The single caller has a full object ID, so pass it on instead of just its hash member. Signed-off-by: René Scharfe --- object-file.c | 2 +- packfile.c | 4 ++-- packfile.h | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) -- 2.33.0 diff --git a/object-file.c b/object-file.c index fb5a385a06..01e7058b4e 100644 --- a/object-file.c +++ b/object-file.c @@ -1725,7 +1725,7 @@ void *read_object_file_extended(struct repository *r, die(_("loose object %s (stored in %s) is corrupt"), oid_to_hex(repl), path); - if ((p = has_packed_and_bad(r, repl->hash)) != NULL) + if ((p = has_packed_and_bad(r, repl)) != NULL) die(_("packed object %s (stored in %s) is corrupt"), oid_to_hex(repl), p->pack_name); obj_read_unlock(); diff --git a/packfile.c b/packfile.c index fb15fc5b49..04080a558b 100644 --- a/packfile.c +++ b/packfile.c @@ -1176,14 +1176,14 @@ void mark_bad_packed_object(struct packed_git *p, const struct object_id *oid) } const struct packed_git *has_packed_and_bad(struct repository *r, - const unsigned char *sha1) + const struct object_id *oid) { struct packed_git *p; unsigned i; for (p = r->objects->packed_git; p; p = p->next) for (i = 0; i < p->num_bad_objects; i++) - if (hasheq(sha1, + if (hasheq(oid->hash, p->bad_object_sha1 + the_hash_algo->rawsz * i)) return p; return NULL; diff --git a/packfile.h b/packfile.h index a982ed9994..186146779d 100644 --- a/packfile.h +++ b/packfile.h @@ -160,7 +160,7 @@ int packed_object_info(struct repository *r, off_t offset, struct object_info *); void mark_bad_packed_object(struct packed_git *, const struct object_id *); -const struct packed_git *has_packed_and_bad(struct repository *r, const unsigned char *sha1); +const struct packed_git *has_packed_and_bad(struct repository *, const struct object_id *); #define ON_DISK_KEEP_PACKS 1 #define IN_CORE_KEEP_PACKS 2 From patchwork Sat Sep 11 08:01:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ren=C3=A9_Scharfe?= X-Patchwork-Id: 12486195 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.3 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E952C433EF for ; Sat, 11 Sep 2021 08:01:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7BB3E60F35 for ; Sat, 11 Sep 2021 08:01:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235406AbhIKIDG (ORCPT ); Sat, 11 Sep 2021 04:03:06 -0400 Received: from mout.web.de ([212.227.15.14]:41631 "EHLO mout.web.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235334AbhIKIDF (ORCPT ); Sat, 11 Sep 2021 04:03:05 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=web.de; s=dbaedf251592; t=1631347301; bh=w7snnXce2LM+ZPD4VOQ+Mof2r/y6sxTsKSwJhVS4Grs=; h=X-UI-Sender-Class:Subject:From:To:Cc:References:Date:In-Reply-To; b=LdJPhv1kT4I+JgEyQC+x9FGw0G3bw3hgwD+3yWaSfyUlLcdTrUNXUrj8Ry+hI7a58 K+VwGRCgtd2K82k66c+TNg11/QqQ943rkskeixMYUcq6jcqf8RVmp8y67R40qVmKtI ZToyEVn21Sa/ti5C8a5hIn9TYKywYWhlhxEqcPeI= X-UI-Sender-Class: c548c8c5-30a9-4db5-a2e7-cb6cb037b8f9 Received: from Mini-von-Rene.fritz.box ([79.203.20.171]) by smtp.web.de (mrweb001 [213.165.67.108]) with ESMTPSA (Nemesis) id 0LxNyU-1n0yJb18eo-016xZI; Sat, 11 Sep 2021 10:01:41 +0200 Subject: [PATCH 3/3] packfile: use oidset for bad objects From: =?utf-8?q?Ren=C3=A9_Scharfe?= To: Git List Cc: Junio C Hamano References: <4a702bfe-afd0-669a-c893-0262289c24b8@web.de> Message-ID: <14d48124-d8bb-aa34-aad0-4203d699e17e@web.de> Date: Sat, 11 Sep 2021 10:01:40 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Thunderbird/78.14.0 MIME-Version: 1.0 In-Reply-To: <4a702bfe-afd0-669a-c893-0262289c24b8@web.de> Content-Language: en-US X-Provags-ID: V03:K1:Eeh0VQEvLSR8qJq3uGQB6DgfdT40Iha9+V+3hhogfSbXz/j9PYO 6koxjub2pn9Db9u8tiH7y1FpFmHvhLcY6mTSLiUem1ehnbryf8etSNX4PP+YYsdIwRetQ/+ K5j499Xx3SqaQHumBRsHGSLOaa4CIlzvEs4IsLJedQD7WtArXx3zELkYW+NM3axEcJhGuUO 25CTpB/xtoy5UoxK2RHtA== X-UI-Out-Filterresults: notjunk:1;V03:K0:/YyZCnvPZUI=:HrfR0PJx4MxzT5eO2nVpBV CZRe0eylZnR9rPIbSnIv6mzlg3cT66hXh7ygsPj+55NoY648eor/65S+Y8Eek+L/pKnewefCC s3rCiRzvAts/Sovgl4N/drPTlYkaNYvmjTLx7kESMHBrdWZ67t4hZmKy391hWKx12n5sJbTXX dKLODblU1921sGNJuf4QkwJTNFmsMrO1jCu7Zg3SzOqHawyYXZgmLXLIpf7ltAJtcjVLYW7pw rsRppYtQDlR4D0A4KuWyY+o9OBBgp7uA1U3hJaqW+w8rZXrtrGaxcP2k9s4mw27iYqhpmFEgi tt1vi+Oqx1GbMmFH4oCVgoSkIrVvWRlxEy/kCA7wdhAOyTDUJqjx4ycxZWr8ah+zvTY6bgAXJ wcZ6qJqFcAFbVMY7KYJYHiY+ZPVgf7xk9b3L4RpDmIpucpQ14NRQJE64faU0ispt1jn/SLBzH Vg0dPT23Hm9i/32hgcNeL4rR7x2X9arCc7qGn9a9YoEfVpGvD4RiLrfUI1IK5j81P7fo4UYB6 FoJPZ8WlyffWhIr2PIs1zxNhiPy0qZuyp/Fzf3uSxbXo70oIsE/3Iphaja1wefd20tRzfPpp8 FrEyGJoAqSmQp46kPlDafQ+VHuyKHd8GzUmDW0clQmm4dxe4x5TkfP5wRQ7wfyQrJgJlgecw1 +t51qI+pJOGZ19nmdYC5cOfXvZyqgr3KGBrMoGtwE56MrTHZGPbC6bLKJWN+C01RODTOlTpi8 bTlsggTppT5pffHObKoXCj8GRGly3Cg9o0JeyiqjBYJDZUw5yFyZuGvledc7sKVOSDLWNgZYM nYfRlQdguXvtjwHBSTLaWq8aESW3v9hsybNkZyxKf37GsZIucLhPeChxMacgfzxcDEOJh2YMn JTy2pscTGOEbVk/Q5BJ8GRDVyhEHgV8T0vKzAe6ZcLcmbd5PUdQh+74ZLYjt6OfNXbZMCCAU2 hgbx5lPbEpcUtGVfziZz0od4/wY8+HM9gAXx7N1dZI6PWdeQY+yCzhikT30yLcbOsmzHO9JjV pq3p2wqgpept1e6tvsZCTRhNcmZNLT5zJTYot5LVEU7ytvFjSox7s07oFkB51iNGEA== Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Store the object ID of broken pack entries in an oidset instead of keeping only their hashes in an unsorted array. The resulting code is shorter and easier to read. It also handles the (hopefully) very rare case of having a high number of bad objects better. Signed-off-by: René Scharfe --- midx.c | 13 ++++--------- object-store.h | 4 ++-- packfile.c | 27 +++++---------------------- 3 files changed, 11 insertions(+), 33 deletions(-) -- 2.33.0 diff --git a/midx.c b/midx.c index 321c6fdd2f..01623fb339 100644 --- a/midx.c +++ b/midx.c @@ -283,6 +283,7 @@ static int nth_midxed_pack_entry(struct repository *r, { uint32_t pack_int_id; struct packed_git *p; + struct object_id oid; if (pos >= m->num_objects) return 0; @@ -303,15 +304,9 @@ static int nth_midxed_pack_entry(struct repository *r, if (!is_pack_valid(p)) return 0; - if (p->num_bad_objects) { - uint32_t i; - struct object_id oid; - nth_midxed_object_oid(&oid, m, pos); - for (i = 0; i < p->num_bad_objects; i++) - if (hasheq(oid.hash, - p->bad_object_sha1 + the_hash_algo->rawsz * i)) - return 0; - } + nth_midxed_object_oid(&oid, m, pos); + if (oidset_contains(&p->bad_objects, &oid)) + return 0; e->offset = nth_midxed_offset(m, pos); e->p = p; diff --git a/object-store.h b/object-store.h index b4dc6668aa..c7bead66f6 100644 --- a/object-store.h +++ b/object-store.h @@ -10,6 +10,7 @@ #include "khash.h" #include "dir.h" #include "oidtree.h" +#include "oidset.h" struct object_directory { struct object_directory *next; @@ -75,9 +76,8 @@ struct packed_git { const void *index_data; size_t index_size; uint32_t num_objects; - uint32_t num_bad_objects; uint32_t crc_offset; - unsigned char *bad_object_sha1; + struct oidset bad_objects; int index_version; time_t mtime; int pack_fd; diff --git a/packfile.c b/packfile.c index 04080a558b..8f6d1d6328 100644 --- a/packfile.c +++ b/packfile.c @@ -1163,29 +1163,17 @@ int unpack_object_header(struct packed_git *p, void mark_bad_packed_object(struct packed_git *p, const struct object_id *oid) { - unsigned i; - const unsigned hashsz = the_hash_algo->rawsz; - for (i = 0; i < p->num_bad_objects; i++) - if (hasheq(oid->hash, p->bad_object_sha1 + hashsz * i)) - return; - p->bad_object_sha1 = xrealloc(p->bad_object_sha1, - st_mult(GIT_MAX_RAWSZ, - st_add(p->num_bad_objects, 1))); - hashcpy(p->bad_object_sha1 + hashsz * p->num_bad_objects, oid->hash); - p->num_bad_objects++; + oidset_insert(&p->bad_objects, oid); } const struct packed_git *has_packed_and_bad(struct repository *r, const struct object_id *oid) { struct packed_git *p; - unsigned i; for (p = r->objects->packed_git; p; p = p->next) - for (i = 0; i < p->num_bad_objects; i++) - if (hasheq(oid->hash, - p->bad_object_sha1 + the_hash_algo->rawsz * i)) - return p; + if (oidset_contains(&p->bad_objects, oid)) + return p; return NULL; } @@ -2016,13 +2004,8 @@ static int fill_pack_entry(const struct object_id *oid, { off_t offset; - if (p->num_bad_objects) { - unsigned i; - for (i = 0; i < p->num_bad_objects; i++) - if (hasheq(oid->hash, - p->bad_object_sha1 + the_hash_algo->rawsz * i)) - return 0; - } + if (oidset_contains(&p->bad_objects, oid)) + return 0; offset = find_pack_entry_one(oid->hash, p); if (!offset) From patchwork Sat Sep 11 16:08:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ren=C3=A9_Scharfe?= X-Patchwork-Id: 12486481 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.3 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E15DC433EF for ; Sat, 11 Sep 2021 16:08:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E8B8461208 for ; Sat, 11 Sep 2021 16:08:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229680AbhIKQKC (ORCPT ); Sat, 11 Sep 2021 12:10:02 -0400 Received: from mout.web.de ([212.227.15.4]:36643 "EHLO mout.web.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229519AbhIKQKC (ORCPT ); Sat, 11 Sep 2021 12:10:02 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=web.de; s=dbaedf251592; t=1631376523; bh=05Z4jkKLJpZoidHD9mjl4lxCXgQUSFmwwaTirFUuHNE=; h=X-UI-Sender-Class:Subject:From:To:Cc:References:Date:In-Reply-To; b=EBcG74LhN4ifEQciniLl/5eCzas84yjh0TL/6ey4TIg07M5/NwuCIFvnwJ29LnCIY 5lDh8CuNB/nUQWEZLOaziz0C0ApjtdWYgr0wYRbt2At2TiRZyoHIqp4fZdhXs/sMBt iCS2um8toeTLBSH8qF8dEc5Kk7hZYx8cQs9ejyTI= X-UI-Sender-Class: c548c8c5-30a9-4db5-a2e7-cb6cb037b8f9 Received: from Mini-von-Rene.fritz.box ([79.203.20.171]) by smtp.web.de (mrweb001 [213.165.67.108]) with ESMTPSA (Nemesis) id 0LbZpT-1mo1Jo06pv-00lBcz; Sat, 11 Sep 2021 18:08:43 +0200 Subject: [PATCH 4/3] midx: inline nth_midxed_pack_entry() From: =?utf-8?q?Ren=C3=A9_Scharfe?= To: Git List Cc: Junio C Hamano , Jeff King References: <4a702bfe-afd0-669a-c893-0262289c24b8@web.de> Message-ID: <7d9e67bf-e057-694c-c976-ba19e9521882@web.de> Date: Sat, 11 Sep 2021 18:08:42 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Thunderbird/78.14.0 MIME-Version: 1.0 In-Reply-To: <4a702bfe-afd0-669a-c893-0262289c24b8@web.de> Content-Language: en-US X-Provags-ID: V03:K1:V1cy5VYXHao0oNnhHCNdG0Arne/BIAQLjf2vRR9JS0SSJD1pH7q ju6lHa7ersCCjDATANdLJZQIyV0kk5xWmke2eJ3m+ZIA0Vv1oeeCf/g7VdoevEmhF9/Ejvy UYYNMNGUmX5+4vC0150L7frsQpEDYN21ZBRRAqI+w9h+Ggm+E2GSvvaFhubIlZLF3bjQ/m5 tP9xrJsUbvFzKxMAHEtfQ== X-UI-Out-Filterresults: notjunk:1;V03:K0:rwiETTILFiM=:4SCj4ZvBeWSptCZZr4x5xI nKUOAgf29IWjbyf0iqwVy1oeZ6qe+cjKx5hwVimnU3yO9eP8SHu70CDIJ51e1XtMZWKU22CKg W8n8N/GNupGLC1fVHjp2brOvY4as8PBpE8QE5qEnF3O8ek89Y6MI7TRMyl+imFrq3CNgEDXid rwhxZ2vgK8as37oWyDjgbTRNGGH6I2wCvA64/ChTBttDO/R9EanF6aTTm2QgFB/Jsq5EWd9t8 LxqyylImW2ifM7RRzX+YNW2GQPmsMmHN61+ocfaaYr9y+XgwA9dQYvTSovbzxPYpbe2SimuD5 3xJYzck8Rp4yh7Vete+yzSavirkhjUp+i85q+zyXAAIu14yoRFICsREZ0h6ds8hgMjrn44zxe vajyNUhOct2scOl7dCKlUlPbDNuliyf/VylHiJWb4VWkzbgsUnKrDLnVDxhh8So6cx/Z4n8zt 7Lq31i/bp/08BaQkWrP3mTydi0vu23uRF1VVyUAAM0wpOsV53XeuBoBSlNRhvIHtoBEQsJwLp qWc/vOQPdE4YQSPtUkjapjCnkPlQlErrlDoLfzV9jdcsdV3rzT4yv97VqmjcTMDyYjz7xDEV1 NSrwKW4chxw+HlCBQasPLO3jDCYrtS7sNg2DVYUXL93fnZBaZq3aaqgt5QPryc5NjMlvlLmxF 3WtqBnlAKIrAr3IsoVccyPGHusbkL3Aoz9GkaATeegk4yfJ5Z75PnJYe7QsbX3yq3VKIFQnAs lFS3WK2sizPpjfTyvijMwuzslJlppgLumhnKlA1UcQzbdwbx2eXJUGlCcu6WcOkpv0c2AzNvA Ks94yQpLIw5lHBXUMQaXSICuipOgmIWxjBtdRzBNNkeGK6pi4btdnuQAU83gLU/E+G7p+QGeL yzWnET66qKkfXC/Ce2YO9oyU4TH9iNa9ASb3KL7OpXMJrCJgjVzB6nnwqwYBvSaQFPvbxYuxx d6BBqLLYwokVHixpPwzMzPWE3ovcQr+nsMndko8zv2jTiAAG9wQ6q9KFdghqtRVMMvsgDS4D/ w2QwCPpppR9W06ZUnD1OilZUfQen1ay1lXAI09h9rEyAT0FcM3EnB64Wdm5bTWEMjaodiHXL3 ZWTyF6b/liu+IQ= Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org fill_midx_entry() finds the position of an object ID and passes it to nth_midxed_pack_entry(), which uses the position to look up the object ID for its own purposes. Inline the latter into the former to avoid that lookup. Signed-off-by: René Scharfe Reported-by: Jeff King --- midx.c | 29 +++++++++-------------------- 1 file changed, 9 insertions(+), 20 deletions(-) -- 2.33.0 diff --git a/midx.c b/midx.c index 01623fb339..59517938a8 100644 --- a/midx.c +++ b/midx.c @@ -276,14 +276,17 @@ uint32_t nth_midxed_pack_int_id(struct multi_pack_index *m, uint32_t pos) (off_t)pos * MIDX_CHUNK_OFFSET_WIDTH); } -static int nth_midxed_pack_entry(struct repository *r, - struct multi_pack_index *m, - struct pack_entry *e, - uint32_t pos) +int fill_midx_entry(struct repository * r, + const struct object_id *oid, + struct pack_entry *e, + struct multi_pack_index *m) { + uint32_t pos; uint32_t pack_int_id; struct packed_git *p; - struct object_id oid; + + if (!bsearch_midx(oid, m, &pos)) + return 0; if (pos >= m->num_objects) return 0; @@ -304,8 +307,7 @@ static int nth_midxed_pack_entry(struct repository *r, if (!is_pack_valid(p)) return 0; - nth_midxed_object_oid(&oid, m, pos); - if (oidset_contains(&p->bad_objects, &oid)) + if (oidset_contains(&p->bad_objects, oid)) return 0; e->offset = nth_midxed_offset(m, pos); @@ -314,19 +316,6 @@ static int nth_midxed_pack_entry(struct repository *r, return 1; } -int fill_midx_entry(struct repository * r, - const struct object_id *oid, - struct pack_entry *e, - struct multi_pack_index *m) -{ - uint32_t pos; - - if (!bsearch_midx(oid, m, &pos)) - return 0; - - return nth_midxed_pack_entry(r, m, e, pos); -} - /* Match "foo.idx" against either "foo.pack" _or_ "foo.idx". */ static int cmp_idx_or_pack_name(const char *idx_or_pack_name, const char *idx_name)