From patchwork Thu Sep 13 18:02:13 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Philippe Blain via GitGitGadget X-Patchwork-Id: 10599897 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A505415A7 for ; Thu, 13 Sep 2018 18:02:39 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 936662B415 for ; Thu, 13 Sep 2018 18:02:39 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 87D652B418; Thu, 13 Sep 2018 18:02:39 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EC9122B415 for ; Thu, 13 Sep 2018 18:02:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727847AbeIMXMu (ORCPT ); Thu, 13 Sep 2018 19:12:50 -0400 Received: from mail-pl1-f195.google.com ([209.85.214.195]:40769 "EHLO mail-pl1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727413AbeIMXMt (ORCPT ); Thu, 13 Sep 2018 19:12:49 -0400 Received: by mail-pl1-f195.google.com with SMTP id s17-v6so2938897plp.7 for ; Thu, 13 Sep 2018 11:02:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=Lc4TLkm1hGb1dmiFnB5R9pnfNtsNlsha0zbwJfhV358=; b=SrnGJCZizY0icWFhdot0D6t9POLNafWrXgmdTqmUTb3LryJ/aHOJ12QOEQHraa32cX uJ3FQow2/e2yuvY0NAvmJvAOOk8H755kg6BXI1s1RTYqiWm0QkfVOfdIZi8TPudLzD8M O0MOuh45/wDioKrYsYnf0DbmZmuphdKu9DDaeL1wrYYp3nDGWp4IjAGXeC6rvxGrnTv4 3naYISxxnhf9UnEtWzA1NeNprTPRstgcG4IhXrj42IEYsEeSU8p/cLyeYZ1MEP0QnLYs HZaA3yjlvMCt6uhOKa56bE5IAn+/JT4sXeRvVWuq3wePKPPbvrhikp7ak+N3U5QdfzJ8 ZY/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=Lc4TLkm1hGb1dmiFnB5R9pnfNtsNlsha0zbwJfhV358=; b=WP4Wjruz6EKiWRxCTYF2jdsULtN4FjI21qjoltr3j16QvrPeB7k7Gyty9FRxIqHgUA vHwxnT5XW5xy+yhvhZIGYHyqQK3qcV0R0GlAkb8NiftQTi/vh71hCPAibEbJeazQNniC 6r13Zn/wISMObWd2YamYsP8yrLWw3ahJor7C182CKFcli/VgFjMCfHoU8Hp/GiHdLrOT sDO4DlseHL7COOtE0enWv8xRt99BblEhOSHRbqe3fF1Uxaf4kemMjZNJC7RuH4nst5uP CejZZm5N8w5i3UZTecawFJRF8FbFhewXJTbcu0NEcHUcZLLN+wMMCUl/2SgUU080wjxV WFng== X-Gm-Message-State: APzg51Aglt4xeTP1U6iRqoAJceF721VrZuLbmHJ5u/KpZixarMKWiHQ3 MCVF2arNFIdPRjAyRQUf1N6HQGK9 X-Google-Smtp-Source: ANB0VdarPbhvFruhyy57RwQaqFBCNy8hNKYFzSh/TxBTvXVCU669Q9vkdZI0KZDVb0pqNX51R83ynQ== X-Received: by 2002:a17:902:274a:: with SMTP id j10-v6mr8492156plg.152.1536861734698; Thu, 13 Sep 2018 11:02:14 -0700 (PDT) Received: from [127.0.0.1] ([40.112.142.204]) by smtp.gmail.com with ESMTPSA id w81-v6sm9276466pfk.92.2018.09.13.11.02.13 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 13 Sep 2018 11:02:13 -0700 (PDT) Date: Thu, 13 Sep 2018 11:02:13 -0700 (PDT) X-Google-Original-Date: Thu, 13 Sep 2018 18:02:00 GMT Message-Id: In-Reply-To: References: From: "Derrick Stolee via GitGitGadget" Subject: [PATCH v2 01/11] multi-pack-index: add 'verify' verb Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: avarab@gmail.com, Junio C Hamano , Derrick Stolee Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Derrick Stolee The multi-pack-index builtin writes multi-pack-index files, and uses a 'write' verb to do so. Add a 'verify' verb that checks this file matches the contents of the pack-indexes it replaces. The current implementation is a no-op, but will be extended in small increments in later commits. Signed-off-by: Derrick Stolee --- Documentation/git-multi-pack-index.txt | 10 ++++++++++ builtin/multi-pack-index.c | 4 +++- midx.c | 13 +++++++++++++ midx.h | 1 + t/t5319-multi-pack-index.sh | 8 ++++++++ 5 files changed, 35 insertions(+), 1 deletion(-) diff --git a/Documentation/git-multi-pack-index.txt b/Documentation/git-multi-pack-index.txt index 1f97e79912..f7778a2c85 100644 --- a/Documentation/git-multi-pack-index.txt +++ b/Documentation/git-multi-pack-index.txt @@ -27,6 +27,10 @@ write:: When given as the verb, write a new MIDX file to `/packs/multi-pack-index`. +verify:: + When given as the verb, verify the contents of the MIDX file + at `/packs/multi-pack-index`. + EXAMPLES -------- @@ -43,6 +47,12 @@ $ git multi-pack-index write $ git multi-pack-index --object-dir write ----------------------------------------------- +* Verify the MIDX file for the packfiles in the current .git folder. ++ +----------------------------------------------- +$ git multi-pack-index verify +----------------------------------------------- + SEE ALSO -------- diff --git a/builtin/multi-pack-index.c b/builtin/multi-pack-index.c index 2633efd95d..fca70f8e4f 100644 --- a/builtin/multi-pack-index.c +++ b/builtin/multi-pack-index.c @@ -5,7 +5,7 @@ #include "midx.h" static char const * const builtin_multi_pack_index_usage[] = { - N_("git multi-pack-index [--object-dir=] write"), + N_("git multi-pack-index [--object-dir=] (write|verify)"), NULL }; @@ -42,6 +42,8 @@ int cmd_multi_pack_index(int argc, const char **argv, if (!strcmp(argv[0], "write")) return write_midx_file(opts.object_dir); + if (!strcmp(argv[0], "verify")) + return verify_midx_file(opts.object_dir); die(_("unrecognized verb: %s"), argv[0]); } diff --git a/midx.c b/midx.c index f3e8dbc108..b253bed517 100644 --- a/midx.c +++ b/midx.c @@ -928,3 +928,16 @@ void clear_midx_file(const char *object_dir) free(midx); } + +int verify_midx_error; + +int verify_midx_file(const char *object_dir) +{ + struct multi_pack_index *m = load_multi_pack_index(object_dir, 1); + verify_midx_error = 0; + + if (!m) + return 0; + + return verify_midx_error; +} diff --git a/midx.h b/midx.h index a210f1af2a..ce80b91c68 100644 --- a/midx.h +++ b/midx.h @@ -43,5 +43,6 @@ int prepare_multi_pack_index_one(struct repository *r, const char *object_dir, i int write_midx_file(const char *object_dir); void clear_midx_file(const char *object_dir); +int verify_midx_file(const char *object_dir); #endif diff --git a/t/t5319-multi-pack-index.sh b/t/t5319-multi-pack-index.sh index 6f56b38674..1c4e0e6d31 100755 --- a/t/t5319-multi-pack-index.sh +++ b/t/t5319-multi-pack-index.sh @@ -150,6 +150,10 @@ test_expect_success 'write midx with twelve packs' ' compare_results_with_midx "twelve packs" +test_expect_success 'verify multi-pack-index success' ' + git multi-pack-index verify --object-dir=$objdir +' + test_expect_success 'repack removes multi-pack-index' ' test_path_is_file $objdir/pack/multi-pack-index && git repack -adf && @@ -214,4 +218,8 @@ test_expect_success 'force some 64-bit offsets with pack-objects' ' midx_read_expect 1 63 5 objects64 " large-offsets" ' +test_expect_success 'verify multi-pack-index with 64-bit offsets' ' + git multi-pack-index verify --object-dir=objects64 +' + test_done From patchwork Thu Sep 13 18:02:15 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Philippe Blain via GitGitGadget X-Patchwork-Id: 10599877 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9410B13B6 for ; Thu, 13 Sep 2018 18:02:20 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 827A52B410 for ; Thu, 13 Sep 2018 18:02:20 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 762D42B415; Thu, 13 Sep 2018 18:02:20 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DF5A22B410 for ; Thu, 13 Sep 2018 18:02:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728166AbeIMXMw (ORCPT ); Thu, 13 Sep 2018 19:12:52 -0400 Received: from mail-pf1-f193.google.com ([209.85.210.193]:46159 "EHLO mail-pf1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727598AbeIMXMv (ORCPT ); Thu, 13 Sep 2018 19:12:51 -0400 Received: by mail-pf1-f193.google.com with SMTP id u24-v6so3008170pfn.13 for ; Thu, 13 Sep 2018 11:02:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=ljO1hsjxgc32SQ3sWj6ks61ft/Aywvqn64928Eh70gc=; b=ggrtWkLSp3ok2jVIbkY10GDsukV3WPBAV+9FK/6iEGg1b7FTqKIM/P1kH/FUnytT4l 5znx2MAb+CKnhQ9nsEH1pV+a+eMu78XpZAH0saguQg/OWjuVHjmV1j4sOcyd1eRj+2pk d5F1P4v3UiqTRgZSxJlVv03b/QReQCFUOiUq6HwBDT/31xanmeikxQN7p5LPR0flk4EK 28ZP13iBMcqdO2FVrHDjC9J/hZ1DIlsKqJiZ11l53/2RFmedGeCj85L9C3jxibo1hXXX OsZhyAJ0oxBBUh37TKr/3ucCJbc/neYDDt/7jM8SiGs+R8zS4qjJPzuv8ZmxMJtbgKpq Ec2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=ljO1hsjxgc32SQ3sWj6ks61ft/Aywvqn64928Eh70gc=; b=MsGM8MfDcb5iOgknTapCySgy6YgtqUSV7JGJvL+JbPuEWaAHt5NzB+O9uvHiPiK+zU dkCmChmEhwwJI7gjNReD7g9Q9Vz5Ey8yXVYBZk+J2aMMpHy8rzR7jxE0LhxXUAAM6W5m p/TroGl14dj24BBEaGjC7ExpRGVv9lNCOLzeHf5fF8MKofguz0wBcXR9ahINGWBckqDz kRX6rydmL8AJBxPKx1Qepadb3clydjohbs6qvWHH3YqXCDO33uAv+2AwZZFrmxommS45 sH4VZ2WK/p0AqBNEfHzI5Ye6rBn/bbbRkMlnhvZW30xZLfrAa3I7ZAVqSRqpodN25iE4 8zkA== X-Gm-Message-State: APzg51C7tzDQOtTf35cJs44Np/ReVoRbRz7P5EJc3mIGtvuNM4GDR4qI mTFFTWzfbsk37BBnnHttf7rrRsqL X-Google-Smtp-Source: ANB0VdahQzMqz09dVBIL2hpwnczYrhdUKi+VEnowANQloswo7KDQb9diAlKsaKNfktl8QYhtfqUlSA== X-Received: by 2002:a65:520d:: with SMTP id o13-v6mr6531864pgp.282.1536861736129; Thu, 13 Sep 2018 11:02:16 -0700 (PDT) Received: from [127.0.0.1] ([40.112.142.204]) by smtp.gmail.com with ESMTPSA id y18-v6sm5520986pfl.90.2018.09.13.11.02.15 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 13 Sep 2018 11:02:15 -0700 (PDT) Date: Thu, 13 Sep 2018 11:02:15 -0700 (PDT) X-Google-Original-Date: Thu, 13 Sep 2018 18:02:01 GMT Message-Id: <95908958302a4a27d8ba444ed7bfd867cb43cd20.1536861730.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Derrick Stolee via GitGitGadget" Subject: [PATCH v2 02/11] multi-pack-index: verify bad header Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: avarab@gmail.com, Junio C Hamano , Derrick Stolee Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Derrick Stolee When verifying if a multi-pack-index file is valid, we want the command to fail to signal an invalid file. Previously, we wrote an error to stderr and continued as if we had no multi-pack-index. Now, die() instead of error(). Add tests that check corrupted headers in a few ways: * Bad signature * Bad file version * Bad hash version * Truncated hash count * Extended hash count Signed-off-by: Derrick Stolee --- midx.c | 18 +++++---------- t/t5319-multi-pack-index.sh | 46 ++++++++++++++++++++++++++++++++++++- 2 files changed, 51 insertions(+), 13 deletions(-) diff --git a/midx.c b/midx.c index b253bed517..ec78254bb6 100644 --- a/midx.c +++ b/midx.c @@ -76,24 +76,18 @@ struct multi_pack_index *load_multi_pack_index(const char *object_dir, int local m->local = local; m->signature = get_be32(m->data); - if (m->signature != MIDX_SIGNATURE) { - error(_("multi-pack-index signature 0x%08x does not match signature 0x%08x"), + if (m->signature != MIDX_SIGNATURE) + die(_("multi-pack-index signature 0x%08x does not match signature 0x%08x"), m->signature, MIDX_SIGNATURE); - goto cleanup_fail; - } m->version = m->data[MIDX_BYTE_FILE_VERSION]; - if (m->version != MIDX_VERSION) { - error(_("multi-pack-index version %d not recognized"), + if (m->version != MIDX_VERSION) + die(_("multi-pack-index version %d not recognized"), m->version); - goto cleanup_fail; - } hash_version = m->data[MIDX_BYTE_HASH_VERSION]; - if (hash_version != MIDX_HASH_VERSION) { - error(_("hash version %u does not match"), hash_version); - goto cleanup_fail; - } + if (hash_version != MIDX_HASH_VERSION) + die(_("hash version %u does not match"), hash_version); m->hash_len = MIDX_HASH_LEN; m->num_chunks = m->data[MIDX_BYTE_NUM_CHUNKS]; diff --git a/t/t5319-multi-pack-index.sh b/t/t5319-multi-pack-index.sh index 1c4e0e6d31..e04b5f43a2 100755 --- a/t/t5319-multi-pack-index.sh +++ b/t/t5319-multi-pack-index.sh @@ -154,6 +154,51 @@ test_expect_success 'verify multi-pack-index success' ' git multi-pack-index verify --object-dir=$objdir ' +# usage: corrupt_midx_and_verify +corrupt_midx_and_verify() { + POS=$1 && + DATA="${2:-\0}" && + OBJDIR=$3 && + GREPSTR="$4" && + FILE=$OBJDIR/pack/multi-pack-index && + chmod a+w $FILE && + test_when_finished mv midx-backup $FILE && + cp $FILE midx-backup && + printf "$DATA" | dd of="$FILE" bs=1 seek="$POS" conv=notrunc && + test_must_fail git multi-pack-index verify --object-dir=$OBJDIR 2>test_err && + grep -v "^+" test_err >err && + test_i18ngrep "$GREPSTR" err +} + +test_expect_success 'verify bad signature' ' + corrupt_midx_and_verify 0 "\00" $objdir \ + "multi-pack-index signature" +' + +MIDX_BYTE_VERSION=4 +MIDX_BYTE_OID_VERSION=5 +MIDX_BYTE_CHUNK_COUNT=6 + +test_expect_success 'verify bad version' ' + corrupt_midx_and_verify $MIDX_BYTE_VERSION "\00" $objdir \ + "multi-pack-index version" +' + +test_expect_success 'verify bad OID version' ' + corrupt_midx_and_verify $MIDX_BYTE_OID_VERSION "\02" $objdir \ + "hash version" +' + +test_expect_success 'verify truncated chunk count' ' + corrupt_midx_and_verify $MIDX_BYTE_CHUNK_COUNT "\01" $objdir \ + "missing required" +' + +test_expect_success 'verify extended chunk count' ' + corrupt_midx_and_verify $MIDX_BYTE_CHUNK_COUNT "\07" $objdir \ + "terminating multi-pack-index chunk id appears earlier than expected" +' + test_expect_success 'repack removes multi-pack-index' ' test_path_is_file $objdir/pack/multi-pack-index && git repack -adf && @@ -191,7 +236,6 @@ test_expect_success 'multi-pack-index in an alternate' ' compare_results_with_midx "with alternate (remote midx)" - # usage: corrupt_data [] corrupt_data () { file=$1 From patchwork Thu Sep 13 18:02:16 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Philippe Blain via GitGitGadget X-Patchwork-Id: 10599879 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1E4A817DF for ; Thu, 13 Sep 2018 18:02:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0D06E2B410 for ; Thu, 13 Sep 2018 18:02:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 010302B415; Thu, 13 Sep 2018 18:02:20 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A35602B410 for ; Thu, 13 Sep 2018 18:02:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728171AbeIMXMx (ORCPT ); Thu, 13 Sep 2018 19:12:53 -0400 Received: from mail-pl1-f196.google.com ([209.85.214.196]:46336 "EHLO mail-pl1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726914AbeIMXMw (ORCPT ); Thu, 13 Sep 2018 19:12:52 -0400 Received: by mail-pl1-f196.google.com with SMTP id t19-v6so2929261ply.13 for ; Thu, 13 Sep 2018 11:02:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=o3OAjBe90g2czbSgAa3sycpVXfAWdcSWAqf+wUf4GBo=; b=ILyacLPdLw44UeDzIprXBSt60HgFjRWyu0BxtsjqJ3VNHnNRBilscGfTW0+MJdC6Yr NyRGXMGZhbx++/LzY0VSq1XQFtUcPnIj5oeqIWrrisDNP/pVUMkXZX/c0FluZ9vqYsQU RBcw7ZSri6pncFMLgbmTEF0PHe1psyFwVR23hLv1iO2hsmXScKDwtY6XQh4+UurY78IB RpVp4yll6kOicD0oo2mhRuUldXr9BeW8nSkacclov+vBOeDz0MEN8iZXCYV9i++7+qsw sfteSrU40ITsWG0OyqLM906ZWwstgptXGLKqpxX4iCAlEAucLbq3wFFtmF8j0WV5RSDV ZptA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=o3OAjBe90g2czbSgAa3sycpVXfAWdcSWAqf+wUf4GBo=; b=fl33ARt5SHF2WTEPsMc+OUaCgZbE+wMWTlkYXJ89hA+uM4809/HMC/RhB8NwcBYm8V pj4N75D65ZJFO355YPuI59ga0WbmhOaFZmgvjmVM3D5zMw09ALM8nM3UTIiFrjwKZjHA VT5Yhg+2jiXC3hGkeSFPmwSIoHgRwOxTizq6yN3s78TJ7WGYMxl1uj/+cRoduXVRa+Xy ncwiAKdqwtlovD363sgd8TdljZtLwnuPvzMjuqTR12QfMSwnezAhRIkGnU4Kr53Yao31 zK8lyHqGJjzD3uEahFNiVIeUDNbs2ycBiBJahJ8RuOCDZoNh9nnGnX6q0iYdCw0dcsJo oe9w== X-Gm-Message-State: APzg51DoUtn6BMolBST0twCKpdkVLRQ7nziXaGhfUxHGmKWjCknVmYpU WlxRy1Bv1H66QdsbKZO+agbpPUf/ X-Google-Smtp-Source: ANB0VdYuxXuTx/JqhFx5281GQW8X9bR9tSWibLn1Pkm7At0TPvty2vhY/9EMQxZ3UqvdCNNwj8gyrA== X-Received: by 2002:a17:902:c6:: with SMTP id a64-v6mr8386673pla.180.1536861737507; Thu, 13 Sep 2018 11:02:17 -0700 (PDT) Received: from [127.0.0.1] ([40.112.142.204]) by smtp.gmail.com with ESMTPSA id g11-v6sm6492316pgi.90.2018.09.13.11.02.16 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 13 Sep 2018 11:02:16 -0700 (PDT) Date: Thu, 13 Sep 2018 11:02:16 -0700 (PDT) X-Google-Original-Date: Thu, 13 Sep 2018 18:02:02 GMT Message-Id: <24481738444bf7da90dddff700dc69d653c440fb.1536861730.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Derrick Stolee via GitGitGadget" Subject: [PATCH v2 03/11] multi-pack-index: verify corrupt chunk lookup table Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: avarab@gmail.com, Junio C Hamano , Derrick Stolee Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Derrick Stolee Signed-off-by: Derrick Stolee --- midx.c | 3 +++ t/t5319-multi-pack-index.sh | 13 +++++++++++++ 2 files changed, 16 insertions(+) diff --git a/midx.c b/midx.c index ec78254bb6..8b054b39ab 100644 --- a/midx.c +++ b/midx.c @@ -100,6 +100,9 @@ struct multi_pack_index *load_multi_pack_index(const char *object_dir, int local uint64_t chunk_offset = get_be64(m->data + MIDX_HEADER_SIZE + 4 + MIDX_CHUNKLOOKUP_WIDTH * i); + if (chunk_offset >= m->data_len) + die(_("invalid chunk offset (too large)")); + switch (chunk_id) { case MIDX_CHUNKID_PACKNAMES: m->chunk_pack_names = m->data + chunk_offset; diff --git a/t/t5319-multi-pack-index.sh b/t/t5319-multi-pack-index.sh index e04b5f43a2..c54b6e7188 100755 --- a/t/t5319-multi-pack-index.sh +++ b/t/t5319-multi-pack-index.sh @@ -178,6 +178,9 @@ test_expect_success 'verify bad signature' ' MIDX_BYTE_VERSION=4 MIDX_BYTE_OID_VERSION=5 MIDX_BYTE_CHUNK_COUNT=6 +MIDX_HEADER_SIZE=12 +MIDX_BYTE_CHUNK_ID=$MIDX_HEADER_SIZE +MIDX_BYTE_CHUNK_OFFSET=$(($MIDX_HEADER_SIZE + 4)) test_expect_success 'verify bad version' ' corrupt_midx_and_verify $MIDX_BYTE_VERSION "\00" $objdir \ @@ -199,6 +202,16 @@ test_expect_success 'verify extended chunk count' ' "terminating multi-pack-index chunk id appears earlier than expected" ' +test_expect_success 'verify missing required chunk' ' + corrupt_midx_and_verify $MIDX_BYTE_CHUNK_ID "\01" $objdir \ + "missing required" +' + +test_expect_success 'verify invalid chunk offset' ' + corrupt_midx_and_verify $MIDX_BYTE_CHUNK_OFFSET "\01" $objdir \ + "invalid chunk offset (too large)" +' + test_expect_success 'repack removes multi-pack-index' ' test_path_is_file $objdir/pack/multi-pack-index && git repack -adf && From patchwork Thu Sep 13 18:02:18 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Philippe Blain via GitGitGadget X-Patchwork-Id: 10599881 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3171F15A7 for ; Thu, 13 Sep 2018 18:02:22 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1F4522B410 for ; Thu, 13 Sep 2018 18:02:22 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 139532B415; Thu, 13 Sep 2018 18:02:22 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AF4962B410 for ; Thu, 13 Sep 2018 18:02:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728176AbeIMXMy (ORCPT ); Thu, 13 Sep 2018 19:12:54 -0400 Received: from mail-pl1-f195.google.com ([209.85.214.195]:41219 "EHLO mail-pl1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726914AbeIMXMy (ORCPT ); Thu, 13 Sep 2018 19:12:54 -0400 Received: by mail-pl1-f195.google.com with SMTP id b12-v6so2944074plr.8 for ; Thu, 13 Sep 2018 11:02:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=YeLw3biOgOq52i3OgPpQIGpkbspl7dTR0sgQ8lB65nY=; b=RR2s1zY/uUh3JaUzQ+l9wrb8mHsNty28ag4jfOyEfhwnxkr/JHjyuK+9t+7We/tePA HUziFGxwFfvief/Se0nbaI0aoqrrr6pI/ftFxPVTPELuWd7jyG4fTMcbK2+fWwEd2syZ bV9qtVHZ5bYoDJK3P3l22402bNHhp0ywQY8/w7fwwuykDEQ28rUFSwPape/Cj7h7l9Xs dOUVVblTuc4MmIB7CeIDKCzymZWoaak11a2xbv1zCpTHwsnWDUvLdKypFpLdyErJVES7 4Ez23TqEMzeNOySgwkF+OkDjFEVJ7VC1t2gf28R5c2nPx0IkA368hLCkq2nUnPprxW5p WUdA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=YeLw3biOgOq52i3OgPpQIGpkbspl7dTR0sgQ8lB65nY=; b=W+vKXYOL8jwAK9DP05yjJFJyORzw0/kNaPUM1vONpdMr5BV8PIQot6oqjEgtJGiuhI 9gA4c7PJA2VkW42lV3q01p+0Catv9eFn/onaPMuLnO5oCehXx7bCR0MRSbO+gRQPxcID dJ5fekt3+V4FBybZ2UfPMA3xNgLs6vbVij1oY6Tn1I6dV+1fyyfoWHnkRj3L6jumwDt0 7pftrBeLjI/sfg5RO4yK+N8VrENsq3umouSVNMwooESs1iI4Afhi29szFS6lhIh/Wudy fVEXVAIpI690bjQLnkHKGjolA/qiZN3whllTsCqKMG8Unv+5cJtqUkZqekfhB/ZuuJxS k3AQ== X-Gm-Message-State: APzg51B6Mgh7EpG0ng7tC1sPz9OUJyLFdknuQjCu3eN/DltiGyTKGt/C eOFyTd/5IqM3yxxv2mzK2LbypKWo X-Google-Smtp-Source: ANB0VdYrbAYs9aMflpI4lAlFQZ2FlaNhJMAFCoXQlMY+ejVP7X3IXchTXvUDoPGeoB5HIYTk2vvKXQ== X-Received: by 2002:a17:902:bccb:: with SMTP id o11-v6mr8268646pls.167.1536861738922; Thu, 13 Sep 2018 11:02:18 -0700 (PDT) Received: from [127.0.0.1] ([40.112.142.204]) by smtp.gmail.com with ESMTPSA id 9-v6sm9366879pfc.20.2018.09.13.11.02.17 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 13 Sep 2018 11:02:18 -0700 (PDT) Date: Thu, 13 Sep 2018 11:02:18 -0700 (PDT) X-Google-Original-Date: Thu, 13 Sep 2018 18:02:03 GMT Message-Id: <947241bfdcc93223ea50a6721cbac2b435ca6b2e.1536861730.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Derrick Stolee via GitGitGadget" Subject: [PATCH v2 04/11] multi-pack-index: verify packname order Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: avarab@gmail.com, Junio C Hamano , Derrick Stolee Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Derrick Stolee The final check we make while loading a multi-pack-index is that the packfile names are in lexicographical order. Make this error be a die() instead. In order to test this condition, we need multiple packfiles. Earlier in t5319-multi-pack-index.sh, we tested the interaction with 'git repack' but this limits us to one packfile in our object dir. Move these repack tests until after the 'verify' tests. Signed-off-by: Derrick Stolee --- midx.c | 6 ++---- t/t5319-multi-pack-index.sh | 10 ++++++++++ 2 files changed, 12 insertions(+), 4 deletions(-) diff --git a/midx.c b/midx.c index 8b054b39ab..e655a15aed 100644 --- a/midx.c +++ b/midx.c @@ -157,12 +157,10 @@ struct multi_pack_index *load_multi_pack_index(const char *object_dir, int local cur_pack_name += strlen(cur_pack_name) + 1; - if (i && strcmp(m->pack_names[i], m->pack_names[i - 1]) <= 0) { - error(_("multi-pack-index pack names out of order: '%s' before '%s'"), + if (i && strcmp(m->pack_names[i], m->pack_names[i - 1]) <= 0) + die(_("multi-pack-index pack names out of order: '%s' before '%s'"), m->pack_names[i - 1], m->pack_names[i]); - goto cleanup_fail; - } } return m; diff --git a/t/t5319-multi-pack-index.sh b/t/t5319-multi-pack-index.sh index c54b6e7188..01a3cd6b00 100755 --- a/t/t5319-multi-pack-index.sh +++ b/t/t5319-multi-pack-index.sh @@ -181,6 +181,11 @@ MIDX_BYTE_CHUNK_COUNT=6 MIDX_HEADER_SIZE=12 MIDX_BYTE_CHUNK_ID=$MIDX_HEADER_SIZE MIDX_BYTE_CHUNK_OFFSET=$(($MIDX_HEADER_SIZE + 4)) +MIDX_NUM_CHUNKS=5 +MIDX_CHUNK_LOOKUP_WIDTH=12 +MIDX_OFFSET_PACKNAMES=$(($MIDX_HEADER_SIZE + \ + $MIDX_NUM_CHUNKS * $MIDX_CHUNK_LOOKUP_WIDTH)) +MIDX_BYTE_PACKNAME_ORDER=$(($MIDX_OFFSET_PACKNAMES + 2)) test_expect_success 'verify bad version' ' corrupt_midx_and_verify $MIDX_BYTE_VERSION "\00" $objdir \ @@ -212,6 +217,11 @@ test_expect_success 'verify invalid chunk offset' ' "invalid chunk offset (too large)" ' +test_expect_success 'verify packnames out of order' ' + corrupt_midx_and_verify $MIDX_BYTE_PACKNAME_ORDER "z" $objdir \ + "pack names out of order" +' + test_expect_success 'repack removes multi-pack-index' ' test_path_is_file $objdir/pack/multi-pack-index && git repack -adf && From patchwork Thu Sep 13 18:02:19 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Philippe Blain via GitGitGadget X-Patchwork-Id: 10599883 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id ED44C15A7 for ; Thu, 13 Sep 2018 18:02:23 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DABC42B410 for ; Thu, 13 Sep 2018 18:02:23 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CF0F62B415; Thu, 13 Sep 2018 18:02:23 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 742AE2B410 for ; Thu, 13 Sep 2018 18:02:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728183AbeIMXM4 (ORCPT ); Thu, 13 Sep 2018 19:12:56 -0400 Received: from mail-pf1-f194.google.com ([209.85.210.194]:46165 "EHLO mail-pf1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727598AbeIMXMz (ORCPT ); Thu, 13 Sep 2018 19:12:55 -0400 Received: by mail-pf1-f194.google.com with SMTP id u24-v6so3008242pfn.13 for ; Thu, 13 Sep 2018 11:02:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=fKSvQdjUtWWxX7coJ2C4/0Pu3cG+W/VEg3VG0pvoWYE=; b=c6h9bMACJWN0/l6X9gurdZcnqvDJ79nOoK9kK+Z7q04yc8SnY0cjvfe7fLtKXl8gyW cwhKbHryBzBVUNVaRqb1UVS8wniOeyJcGVrYomYYQU4FntPuaYAApfnw3k79uylQVRxa Sn0Ln4R+YZYss2kutoaHfP200aQFKcx6dOKI82oDw/Ppg4sXdNNRWl3CfuzsC+EuQKKy iJ7EFdXoyJg38tluffP0nMUVUzQSWIKblRLX83OyzB6OfdMcB6UW1EoZ1kfDCgYspMzY xk2Yfr58owADoH4DFNBfX8d51J+sHP7nurZH36qdnGBFnLeD/itiPgAKMh5jYM3S14vS MGtA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=fKSvQdjUtWWxX7coJ2C4/0Pu3cG+W/VEg3VG0pvoWYE=; b=hbb34G/lheAim8gAOOk7aesu0iKFNBGP0jRRGuJWNao3TRXDkVcY4T58P5Lo5hTq7Q xrPmhk3NjN3P6Og7qll3ZM5I3i6j76geypgiJYAxNU5s3QHVQX5+rmFn5sllMNZ6XlYD 8HZYvhZ88jcSrUmxDK2BGDTgnWSpldHu9fuf1baYi/3FJFTGa6eYtqLBQcZowI8y1Du3 yJa4T0tbxqNXSHHbSqi4DRKl7vBYYQ5Ycd8yJ8VEmH8g3GRmZ2NMZPMchpjF8BmqOKPB 5CwgbO7uaZRoZc2ZkdXIhoy6pXlV8qAUn1xW77nL7ysZSdieVUkw9VwpDbfdPv+399p6 ipdw== X-Gm-Message-State: APzg51AFjZqT50i1qJosOb5cfUuSYrwswxc+6kR4BazyNm5A08rCzOhc +JdbJe4xBJxz1N3v7M5uO6/6YBca X-Google-Smtp-Source: ANB0VdZe8w91swTYWncHvRWApos0Ud6zJvl4Exi+VUa0qnoESzLvvND4zKJ5zvSsadkyUscWcvgeYg== X-Received: by 2002:a63:df4e:: with SMTP id h14-v6mr8118472pgj.300.1536861740297; Thu, 13 Sep 2018 11:02:20 -0700 (PDT) Received: from [127.0.0.1] ([40.112.142.204]) by smtp.gmail.com with ESMTPSA id o20-v6sm13844574pfj.35.2018.09.13.11.02.19 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 13 Sep 2018 11:02:19 -0700 (PDT) Date: Thu, 13 Sep 2018 11:02:19 -0700 (PDT) X-Google-Original-Date: Thu, 13 Sep 2018 18:02:04 GMT Message-Id: <405886738002e60b9f72d6edb10e4929c9493374.1536861730.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Derrick Stolee via GitGitGadget" Subject: [PATCH v2 05/11] multi-pack-index: verify missing pack Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: avarab@gmail.com, Junio C Hamano , Derrick Stolee Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Derrick Stolee Signed-off-by: Derrick Stolee --- midx.c | 16 ++++++++++++++++ t/t5319-multi-pack-index.sh | 5 +++++ 2 files changed, 21 insertions(+) diff --git a/midx.c b/midx.c index e655a15aed..a02b19efc1 100644 --- a/midx.c +++ b/midx.c @@ -926,13 +926,29 @@ void clear_midx_file(const char *object_dir) int verify_midx_error; +static void midx_report(const char *fmt, ...) +{ + va_list ap; + verify_midx_error = 1; + va_start(ap, fmt); + vfprintf(stderr, fmt, ap); + fprintf(stderr, "\n"); + va_end(ap); +} + int verify_midx_file(const char *object_dir) { + uint32_t i; struct multi_pack_index *m = load_multi_pack_index(object_dir, 1); verify_midx_error = 0; if (!m) return 0; + for (i = 0; i < m->num_packs; i++) { + if (prepare_midx_pack(m, i)) + midx_report("failed to load pack in position %d", i); + } + return verify_midx_error; } diff --git a/t/t5319-multi-pack-index.sh b/t/t5319-multi-pack-index.sh index 01a3cd6b00..0a566afb05 100755 --- a/t/t5319-multi-pack-index.sh +++ b/t/t5319-multi-pack-index.sh @@ -222,6 +222,11 @@ test_expect_success 'verify packnames out of order' ' "pack names out of order" ' +test_expect_success 'verify packnames out of order' ' + corrupt_midx_and_verify $MIDX_BYTE_PACKNAME_ORDER "a" $objdir \ + "failed to load pack" +' + test_expect_success 'repack removes multi-pack-index' ' test_path_is_file $objdir/pack/multi-pack-index && git repack -adf && From patchwork Thu Sep 13 18:02:20 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Philippe Blain via GitGitGadget X-Patchwork-Id: 10599885 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8990313B6 for ; Thu, 13 Sep 2018 18:02:25 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 789C02B414 for ; Thu, 13 Sep 2018 18:02:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6CFA62B415; Thu, 13 Sep 2018 18:02:25 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 11F5F2B410 for ; Thu, 13 Sep 2018 18:02:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728187AbeIMXM5 (ORCPT ); Thu, 13 Sep 2018 19:12:57 -0400 Received: from mail-pg1-f195.google.com ([209.85.215.195]:37970 "EHLO mail-pg1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727598AbeIMXM4 (ORCPT ); Thu, 13 Sep 2018 19:12:56 -0400 Received: by mail-pg1-f195.google.com with SMTP id t84-v6so3107797pgb.5 for ; Thu, 13 Sep 2018 11:02:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=eeQI1Q4p3rVCBb3hKEdqdhHNQAtcxpOxE5nRJZ8n2jc=; b=HKQXPuduWmsb5o+ta8K+V7qdpLLkU5z/hSk7mR+NrbCFeCaBLn+ij5FdUKE2656XgK 9tG0K8Znx0HD0GT7mUz+Di/hJWGF1j5kNuQrCnRW/GqCDB9OE5ip3e+xi1VNw6IrYynq tVEpPpZOm8YbOwgv5t/S+gb+S3+6hdVLhxLbqeRCk7GKEBC1ov/kRLTYdmQrfJtZEjgK W7V02FlLTZ2XF3IcOsD/pKZwaK0knjcAfLO5IfXEn1M40I6ggVz+AGZrq4Bufptas/nW c7YgJm56ZDU2yWpbFjixNQeoFDA0bX9CvqMV4khiwwgQu9AqlORGGedIIVsBS5/pD4Jj cjxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=eeQI1Q4p3rVCBb3hKEdqdhHNQAtcxpOxE5nRJZ8n2jc=; b=o17O7ksGoWpbokUaL/CpWx44u+9oSzIOEMkMmnxJMX/SmEL2BiiNykfA+fElhxOQXi 4AjGvkFHC6+vnVkYMGkRYPgnj0FFQNCV0eLnsYhtXiUH3wMknTTEMvBW4ubBmRnc0upC kn7GAs3vpcGziJrBB6llVT2UDc365Q2OlaNAUMnFGz8cyzuFCsCHt599smJ0npSj3auO 0iO9g8zT54ufEO4PQOGfLikxBPe6IL+pMZqjluf1TXrNP8KSuYG3+AiD7rVSMpbkIxDC eK0sKjYEZnSDKgsOKq1SVfNKwioIoyffRrlxxSdPhWmfsxJgG7QI6ccdGgt5Mi3qU3yC 6BXQ== X-Gm-Message-State: APzg51D1KNjYJTuMGSmkRAwKOowD40qwrBTbI//iXiTu0s5mbyU2od4n UBrAldZyFwkGXM+1vrgOJsSgVhsY X-Google-Smtp-Source: ANB0VdYyXFGWnxCPPAh5RZh4gj4jU2Qvs5PfugNe3XpMKSiu30UxP9igx2tw4AD7TmRpm2/7r4j3Tg== X-Received: by 2002:a62:4808:: with SMTP id v8-v6mr8571205pfa.89.1536861741713; Thu, 13 Sep 2018 11:02:21 -0700 (PDT) Received: from [127.0.0.1] ([40.112.142.204]) by smtp.gmail.com with ESMTPSA id p73-v6sm7504912pfk.186.2018.09.13.11.02.20 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 13 Sep 2018 11:02:20 -0700 (PDT) Date: Thu, 13 Sep 2018 11:02:20 -0700 (PDT) X-Google-Original-Date: Thu, 13 Sep 2018 18:02:05 GMT Message-Id: In-Reply-To: References: From: "Derrick Stolee via GitGitGadget" Subject: [PATCH v2 06/11] multi-pack-index: verify oid fanout order Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: avarab@gmail.com, Junio C Hamano , Derrick Stolee Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Derrick Stolee Signed-off-by: Derrick Stolee --- midx.c | 9 +++++++++ t/t5319-multi-pack-index.sh | 8 ++++++++ 2 files changed, 17 insertions(+) diff --git a/midx.c b/midx.c index a02b19efc1..dfd26b4d74 100644 --- a/midx.c +++ b/midx.c @@ -950,5 +950,14 @@ int verify_midx_file(const char *object_dir) midx_report("failed to load pack in position %d", i); } + for (i = 0; i < 255; i++) { + uint32_t oid_fanout1 = ntohl(m->chunk_oid_fanout[i]); + uint32_t oid_fanout2 = ntohl(m->chunk_oid_fanout[i + 1]); + + if (oid_fanout1 > oid_fanout2) + midx_report(_("oid fanout out of order: fanout[%d] = %"PRIx32" > %"PRIx32" = fanout[%d]"), + i, oid_fanout1, oid_fanout2, i + 1); + } + return verify_midx_error; } diff --git a/t/t5319-multi-pack-index.sh b/t/t5319-multi-pack-index.sh index 0a566afb05..47a54e138d 100755 --- a/t/t5319-multi-pack-index.sh +++ b/t/t5319-multi-pack-index.sh @@ -186,6 +186,9 @@ MIDX_CHUNK_LOOKUP_WIDTH=12 MIDX_OFFSET_PACKNAMES=$(($MIDX_HEADER_SIZE + \ $MIDX_NUM_CHUNKS * $MIDX_CHUNK_LOOKUP_WIDTH)) MIDX_BYTE_PACKNAME_ORDER=$(($MIDX_OFFSET_PACKNAMES + 2)) +MIDX_OFFSET_OID_FANOUT=$(($MIDX_OFFSET_PACKNAMES + 652)) +MIDX_OID_FANOUT_WIDTH=4 +MIDX_BYTE_OID_FANOUT_ORDER=$((MIDX_OFFSET_OID_FANOUT + 250 * $MIDX_OID_FANOUT_WIDTH + 1)) test_expect_success 'verify bad version' ' corrupt_midx_and_verify $MIDX_BYTE_VERSION "\00" $objdir \ @@ -227,6 +230,11 @@ test_expect_success 'verify packnames out of order' ' "failed to load pack" ' +test_expect_success 'verify oid fanout out of order' ' + corrupt_midx_and_verify $MIDX_BYTE_OID_FANOUT_ORDER "\01" $objdir \ + "oid fanout out of order" +' + test_expect_success 'repack removes multi-pack-index' ' test_path_is_file $objdir/pack/multi-pack-index && git repack -adf && From patchwork Thu Sep 13 18:02:22 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Philippe Blain via GitGitGadget X-Patchwork-Id: 10599887 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E82A417DF for ; Thu, 13 Sep 2018 18:02:25 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D5E2D2B414 for ; Thu, 13 Sep 2018 18:02:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C9E4D2B410; Thu, 13 Sep 2018 18:02:25 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 705782B410 for ; Thu, 13 Sep 2018 18:02:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728188AbeIMXM6 (ORCPT ); Thu, 13 Sep 2018 19:12:58 -0400 Received: from mail-pl1-f195.google.com ([209.85.214.195]:42844 "EHLO mail-pl1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726914AbeIMXM6 (ORCPT ); Thu, 13 Sep 2018 19:12:58 -0400 Received: by mail-pl1-f195.google.com with SMTP id g23-v6so2937475plq.9 for ; Thu, 13 Sep 2018 11:02:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=vHUkBRtBqXGfNO9a1LGkAuldc5o3Moxr27e8xRcaDms=; b=mmRT8jTpvypkhq7ptwq2K7n0gq6tyg5T0he1hhCOqIIhcrMhPWXJ11raM8qgr9U+fX e1Li0fyNAN3l5bw1Tv9Opzh+6G7LZCEaHHSrPuIJdmm5zbrDVPL24PXAk9cwLJBijOXY 3KRmj88Ax9v0Kf9yn+Br0U8MlhoP+XxHo5obyZmrYBq0F53z5F4OovmoeWprHoEdJpve qFGiPrpkqpdS4HCYu31g8PJ7XVJ1n8wzbBQKgl9J5mcEl3VSRQmnTTW3nXnuH9zvoGlx bFhPtMfq4bplE3ygB6qhAzDJnPQ+DAm0BivJtw55T/F6yZk0DTy9gysiX3AYPMkiPXKX XfSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=vHUkBRtBqXGfNO9a1LGkAuldc5o3Moxr27e8xRcaDms=; b=DMaXoXmHmyGe9Qq6ecb/nQ/2Z6iptpeToL+BI1iomsPX98plUMjpcEd1HNO2gS3p3b /K2SJrejvHEu/CtzUSr20J9mJtu4It9JjxJ4AV/q8WRDz6uMQPFELiJJmCx5wEEHeGE6 ghiG5KjOHM2LBAY1rGTReRStsmvcWpnej+91xIOi+3WDYwhc8XUQZ4XuDRSm6/5qj3TC SjgGDKJvUcgoxN/kheyPZqXmFgPCvDiPtvKFe8KiTCWOoLzm8ENYNfX4YspCwdrSyKW4 mGCiolz1F42OeHWYfgPALqrNp+iP0JhBe1NlFv/PSB8JekCVAR+df1GGGY6IYapr9cS6 laIQ== X-Gm-Message-State: APzg51AxpxfvkcCdOAaicxxOcvfg28/F3dWnxcSgHmFZPACjw5MFdq7n 2QhYSZ3fjLneCvWpOzdp+CKnu9B9 X-Google-Smtp-Source: ANB0Vdbi3eJqYniD4X8DGR6s9zoDAZMdFmo2MOqW0j2lkYuOO3n8T76U+57zSiH7jKUBlufVc6SZxw== X-Received: by 2002:a17:902:8ec8:: with SMTP id x8-v6mr8379579plo.308.1536861743103; Thu, 13 Sep 2018 11:02:23 -0700 (PDT) Received: from [127.0.0.1] ([40.112.142.204]) by smtp.gmail.com with ESMTPSA id d132-v6sm9206879pgc.93.2018.09.13.11.02.22 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 13 Sep 2018 11:02:22 -0700 (PDT) Date: Thu, 13 Sep 2018 11:02:22 -0700 (PDT) X-Google-Original-Date: Thu, 13 Sep 2018 18:02:06 GMT Message-Id: <511791de91115e0d51c1635f8ec3d337ca009971.1536861730.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Derrick Stolee via GitGitGadget" Subject: [PATCH v2 07/11] multi-pack-index: verify oid lookup order Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: avarab@gmail.com, Junio C Hamano , Derrick Stolee Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Derrick Stolee Signed-off-by: Derrick Stolee --- midx.c | 11 +++++++++++ t/t5319-multi-pack-index.sh | 8 ++++++++ 2 files changed, 19 insertions(+) diff --git a/midx.c b/midx.c index dfd26b4d74..06d5cfc826 100644 --- a/midx.c +++ b/midx.c @@ -959,5 +959,16 @@ int verify_midx_file(const char *object_dir) i, oid_fanout1, oid_fanout2, i + 1); } + for (i = 0; i < m->num_objects - 1; i++) { + struct object_id oid1, oid2; + + nth_midxed_object_oid(&oid1, m, i); + nth_midxed_object_oid(&oid2, m, i + 1); + + if (oidcmp(&oid1, &oid2) >= 0) + midx_report(_("oid lookup out of order: oid[%d] = %s >= %s = oid[%d]"), + i, oid_to_hex(&oid1), oid_to_hex(&oid2), i + 1); + } + return verify_midx_error; } diff --git a/t/t5319-multi-pack-index.sh b/t/t5319-multi-pack-index.sh index 47a54e138d..a968b9a959 100755 --- a/t/t5319-multi-pack-index.sh +++ b/t/t5319-multi-pack-index.sh @@ -175,6 +175,7 @@ test_expect_success 'verify bad signature' ' "multi-pack-index signature" ' +HASH_LEN=20 MIDX_BYTE_VERSION=4 MIDX_BYTE_OID_VERSION=5 MIDX_BYTE_CHUNK_COUNT=6 @@ -189,6 +190,8 @@ MIDX_BYTE_PACKNAME_ORDER=$(($MIDX_OFFSET_PACKNAMES + 2)) MIDX_OFFSET_OID_FANOUT=$(($MIDX_OFFSET_PACKNAMES + 652)) MIDX_OID_FANOUT_WIDTH=4 MIDX_BYTE_OID_FANOUT_ORDER=$((MIDX_OFFSET_OID_FANOUT + 250 * $MIDX_OID_FANOUT_WIDTH + 1)) +MIDX_OFFSET_OID_LOOKUP=$(($MIDX_OFFSET_OID_FANOUT + 256 * $MIDX_OID_FANOUT_WIDTH)) +MIDX_BYTE_OID_LOOKUP=$(($MIDX_OFFSET_OID_LOOKUP + 16 * $HASH_LEN)) test_expect_success 'verify bad version' ' corrupt_midx_and_verify $MIDX_BYTE_VERSION "\00" $objdir \ @@ -235,6 +238,11 @@ test_expect_success 'verify oid fanout out of order' ' "oid fanout out of order" ' +test_expect_success 'verify oid lookup out of order' ' + corrupt_midx_and_verify $MIDX_BYTE_OID_LOOKUP "\00" $objdir \ + "oid lookup out of order" +' + test_expect_success 'repack removes multi-pack-index' ' test_path_is_file $objdir/pack/multi-pack-index && git repack -adf && From patchwork Thu Sep 13 18:02:23 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Philippe Blain via GitGitGadget X-Patchwork-Id: 10599889 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0EB9615A7 for ; Thu, 13 Sep 2018 18:02:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F0A6C2B414 for ; Thu, 13 Sep 2018 18:02:27 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E50B82B416; Thu, 13 Sep 2018 18:02:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 96D5A2B414 for ; Thu, 13 Sep 2018 18:02:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728210AbeIMXNA (ORCPT ); Thu, 13 Sep 2018 19:13:00 -0400 Received: from mail-pg1-f196.google.com ([209.85.215.196]:38977 "EHLO mail-pg1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728193AbeIMXM7 (ORCPT ); Thu, 13 Sep 2018 19:12:59 -0400 Received: by mail-pg1-f196.google.com with SMTP id i190-v6so3105671pgc.6 for ; Thu, 13 Sep 2018 11:02:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=lgN9+HaKNLJe4NXAvjUb52XkOhnN7utX07voe/Kf4ug=; b=pdS4COBH4WyZzgSc8xWgExvVl/w5AGWJv1wRVGTtPZ5/fspOJ6LS+EeIQGS9Z+ipbV vmBXUY0MkxzdyEP4Kjp6uq2+lU41LjsfXRGx3b64eqF0ovrosold7Fu72wm8r9EyMZys RVdx2t404/StUiREt8JPUKq8QbwP9TrZxRnT8EqOU0fBOsDLOZ9ZatwCDT2qDXQrt5Z1 ZOYC7L93HWqOwQEdcD3r+sbi0OqE46cCzBNApcSTQu8yIlXmK2iwPnLJnNnLtb20OKFo zeSkwAbICSFrmJ9woa/VtJ6tXukVd5qKBvMOgfALmBIZ0UuvJjI1Jz3IUjEhGzrUWi+H gDaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=lgN9+HaKNLJe4NXAvjUb52XkOhnN7utX07voe/Kf4ug=; b=ZDRrQF4veuU5OU/BeDPrCO6PRC5+yRdWagua/9RA1rbnysPMtxyz+vmw0/pBfCfnoc RPx5Zv1K8wWxkxVbVm/nYWIWiYA32Ws/sOfU6eq1dUZLF79xQeLCP1dzOc3+OoTdJAiO m2zdBjqXEPG6ZALQz3nmEIzPqYxwp637y8L+0cgoadeBgilv3vZLuZVRf+h/IiicxfAI znxZJXSIFNBZTg1v4KNznA2ljnlGZOy3qzGSWF88MnI+rCgx55FOCBoVRAg6BtlwJSHI BSUMlSjA8vPqBnSSz32jnq9tjq/2VNXNLvS/gnjs/YjhDYeqfffsuVxWHLI0NWh0cdSa wtoA== X-Gm-Message-State: APzg51DRYbzbmtbOwCS3I5dwGo6zec+BhZzogtfJxev0luFWqHHqPjTj /U+4ym7W2//BCz1J5S4KrGRBRjCb X-Google-Smtp-Source: ANB0Vda3kzybL0pQ1eN4c0EWabQLugFEUZpfO9TBctAzJT+TX0SmMaInSPDHmYThIU3RoCQmu6iCcg== X-Received: by 2002:a63:e756:: with SMTP id j22-v6mr8206974pgk.185.1536861744547; Thu, 13 Sep 2018 11:02:24 -0700 (PDT) Received: from [127.0.0.1] ([40.112.139.85]) by smtp.gmail.com with ESMTPSA id v140-v6sm6357927pgb.45.2018.09.13.11.02.23 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 13 Sep 2018 11:02:23 -0700 (PDT) Date: Thu, 13 Sep 2018 11:02:23 -0700 (PDT) X-Google-Original-Date: Thu, 13 Sep 2018 18:02:07 GMT Message-Id: <210649bf838bf0dc338671c691cbd015bef41240.1536861730.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Derrick Stolee via GitGitGadget" Subject: [PATCH v2 08/11] multi-pack-index: fix 32-bit vs 64-bit size check Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: avarab@gmail.com, Junio C Hamano , Derrick Stolee Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Derrick Stolee When loading a 64-bit offset, we intend to check that off_t can store the resulting offset. However, the condition accidentally checks the 32-bit offset to see if it is smaller than a 64-bit value. Fix it, and this will be covered by a test in the 'git multi-pack-index verify' command in a later commit. Signed-off-by: Derrick Stolee --- midx.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/midx.c b/midx.c index 06d5cfc826..80094c02a7 100644 --- a/midx.c +++ b/midx.c @@ -236,7 +236,7 @@ static off_t nth_midxed_offset(struct multi_pack_index *m, uint32_t pos) offset32 = get_be32(offset_data + sizeof(uint32_t)); if (m->chunk_large_offsets && offset32 & MIDX_LARGE_OFFSET_NEEDED) { - if (sizeof(offset32) < sizeof(uint64_t)) + if (sizeof(off_t) < sizeof(uint64_t)) die(_("multi-pack-index stores a 64-bit offset, but off_t is too small")); offset32 ^= MIDX_LARGE_OFFSET_NEEDED; From patchwork Thu Sep 13 18:02:25 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Philippe Blain via GitGitGadget X-Patchwork-Id: 10599891 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3B3EC15A7 for ; Thu, 13 Sep 2018 18:02:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 292F52B414 for ; Thu, 13 Sep 2018 18:02:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1DD552B416; Thu, 13 Sep 2018 18:02:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A504A2B414 for ; Thu, 13 Sep 2018 18:02:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728215AbeIMXNB (ORCPT ); Thu, 13 Sep 2018 19:13:01 -0400 Received: from mail-pf1-f174.google.com ([209.85.210.174]:42127 "EHLO mail-pf1-f174.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728193AbeIMXNB (ORCPT ); Thu, 13 Sep 2018 19:13:01 -0400 Received: by mail-pf1-f174.google.com with SMTP id l9-v6so3012497pff.9 for ; Thu, 13 Sep 2018 11:02:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=1tm27vDdr+OSR1KPv2/mxLVXxHkgJHvoYoKzG0Da0dk=; b=pPUve9i5Nd2SxcKym7pnKv58k0d9ng6+MB3+JArXHnjwh9acfKrqUNsZW5ihLlpz7M m5IUXKCSkJ40euoOfFMHBAJ5Uf9ad68u6zcrz9B8sG6+wF0wYS2w0HtjRBdz3ctCMGhp DtEsbWrQ3gdROvOMndDYixtRyDBwabzl9ZPVX8369oOnPOZMBVxrgaEmoFg0O6MhAhkF 9EpWFSO79AW9qErF289LUUoUTuXrai3eWf5ztXDl+oALqD8qxUN27GJkfp3OU47bf1Q4 e/1PFjO9jlaP2uWNwbWrZuZWo/n1PQoHNHEl3SYySz4H+UFwiKIDt/Z3Zwupt4h8+HHQ jqsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=1tm27vDdr+OSR1KPv2/mxLVXxHkgJHvoYoKzG0Da0dk=; b=EMXUertTDluCcgdYDKPXvwdX4S8ugfDEOCUIbH7e946KDPBb3OdnuES0rR++70SCK7 RK9bthZeyAY8S6aMzkYGpA+mZ28MddZnCorNT2aV5YzSGvB9QJsveIUa3DCutSz2ywDz aLKdhpcDIf3BVVjiNpfNbSjiCZyVVtpBlk8gKEHQzQdkys0JjEMOpEGi9wLbAPFJnkNW BfAj5EL6+ImJqkR71uvWRMPXsfCzNMr9TGb0Fn9TzJ9c5q82daFNL+5ifwlxQzasTOfp mOyKZ40jsrTXMrDj2UGm3DY+wo8ddReksSvxb/B5lGOaNhgzzpO7OvoPYKsS0XxGy6qr 7xRg== X-Gm-Message-State: APzg51CnXdqsN6NUPBSQCFBLErfAYvf4ze3JLfuwa8NSOAdv/6g7tiLR F+J9hiUHp2pWGV13wKxLe0KqVg1h X-Google-Smtp-Source: ANB0VdbU6r7k9hR3OWV9JvpkoNmENo6A5Fsx4Ezvfrj22wcpDE9MSeuk3W9OsfWjqcfS2XvlLY3AvQ== X-Received: by 2002:a62:23c2:: with SMTP id q63-v6mr8526257pfj.116.1536861745945; Thu, 13 Sep 2018 11:02:25 -0700 (PDT) Received: from [127.0.0.1] ([40.112.139.85]) by smtp.gmail.com with ESMTPSA id b64-v6sm5964267pfg.66.2018.09.13.11.02.24 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 13 Sep 2018 11:02:25 -0700 (PDT) Date: Thu, 13 Sep 2018 11:02:25 -0700 (PDT) X-Google-Original-Date: Thu, 13 Sep 2018 18:02:08 GMT Message-Id: In-Reply-To: References: From: "Derrick Stolee via GitGitGadget" Subject: [PATCH v2 09/11] multi-pack-index: verify object offsets Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: avarab@gmail.com, Junio C Hamano , Derrick Stolee Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Derrick Stolee The 'git multi-pack-index verify' command must verify the object offsets stored in the multi-pack-index are correct. There are two ways the offset chunk can be incorrect: the pack-int-id and the object offset. Replace the BUG() statement with a die() statement, now that we may hit a bad pack-int-id during a 'verify' command on a corrupt multi-pack-index, and it is covered by a test. Signed-off-by: Derrick Stolee --- midx.c | 29 ++++++++++++++++++++++++++++- t/t5319-multi-pack-index.sh | 27 +++++++++++++++++++++++++++ 2 files changed, 55 insertions(+), 1 deletion(-) diff --git a/midx.c b/midx.c index 80094c02a7..47e7e6113a 100644 --- a/midx.c +++ b/midx.c @@ -197,7 +197,8 @@ int prepare_midx_pack(struct multi_pack_index *m, uint32_t pack_int_id) struct strbuf pack_name = STRBUF_INIT; if (pack_int_id >= m->num_packs) - BUG("bad pack-int-id"); + die(_("bad pack-int-id: %u (%u total packs"), + pack_int_id, m->num_packs); if (m->packs[pack_int_id]) return 0; @@ -970,5 +971,31 @@ int verify_midx_file(const char *object_dir) i, oid_to_hex(&oid1), oid_to_hex(&oid2), i + 1); } + for (i = 0; i < m->num_objects; i++) { + struct object_id oid; + struct pack_entry e; + off_t m_offset, p_offset; + + nth_midxed_object_oid(&oid, m, i); + if (!fill_midx_entry(&oid, &e, m)) { + midx_report(_("failed to load pack entry for oid[%d] = %s"), + i, oid_to_hex(&oid)); + continue; + } + + if (open_pack_index(e.p)) { + midx_report(_("failed to load pack-index for packfile %s"), + e.p->pack_name); + break; + } + + m_offset = e.offset; + p_offset = find_pack_entry_one(oid.hash, e.p); + + if (m_offset != p_offset) + midx_report(_("incorrect object offset for oid[%d] = %s: %"PRIx64" != %"PRIx64), + i, oid_to_hex(&oid), m_offset, p_offset); + } + return verify_midx_error; } diff --git a/t/t5319-multi-pack-index.sh b/t/t5319-multi-pack-index.sh index a968b9a959..828c240389 100755 --- a/t/t5319-multi-pack-index.sh +++ b/t/t5319-multi-pack-index.sh @@ -176,6 +176,7 @@ test_expect_success 'verify bad signature' ' ' HASH_LEN=20 +NUM_OBJECTS=74 MIDX_BYTE_VERSION=4 MIDX_BYTE_OID_VERSION=5 MIDX_BYTE_CHUNK_COUNT=6 @@ -192,6 +193,10 @@ MIDX_OID_FANOUT_WIDTH=4 MIDX_BYTE_OID_FANOUT_ORDER=$((MIDX_OFFSET_OID_FANOUT + 250 * $MIDX_OID_FANOUT_WIDTH + 1)) MIDX_OFFSET_OID_LOOKUP=$(($MIDX_OFFSET_OID_FANOUT + 256 * $MIDX_OID_FANOUT_WIDTH)) MIDX_BYTE_OID_LOOKUP=$(($MIDX_OFFSET_OID_LOOKUP + 16 * $HASH_LEN)) +MIDX_OFFSET_OBJECT_OFFSETS=$(($MIDX_OFFSET_OID_LOOKUP + $NUM_OBJECTS * $HASH_LEN)) +MIDX_OFFSET_WIDTH=8 +MIDX_BYTE_PACK_INT_ID=$(($MIDX_OFFSET_OBJECT_OFFSETS + 16 * $MIDX_OFFSET_WIDTH + 2)) +MIDX_BYTE_OFFSET=$(($MIDX_OFFSET_OBJECT_OFFSETS + 16 * $MIDX_OFFSET_WIDTH + 6)) test_expect_success 'verify bad version' ' corrupt_midx_and_verify $MIDX_BYTE_VERSION "\00" $objdir \ @@ -243,6 +248,16 @@ test_expect_success 'verify oid lookup out of order' ' "oid lookup out of order" ' +test_expect_success 'verify incorrect pack-int-id' ' + corrupt_midx_and_verify $MIDX_BYTE_PACK_INT_ID "\07" $objdir \ + "bad pack-int-id" +' + +test_expect_success 'verify incorrect offset' ' + corrupt_midx_and_verify $MIDX_BYTE_OFFSET "\07" $objdir \ + "incorrect object offset" +' + test_expect_success 'repack removes multi-pack-index' ' test_path_is_file $objdir/pack/multi-pack-index && git repack -adf && @@ -310,4 +325,16 @@ test_expect_success 'verify multi-pack-index with 64-bit offsets' ' git multi-pack-index verify --object-dir=objects64 ' +NUM_OBJECTS=63 +MIDX_OFFSET_OID_FANOUT=$((MIDX_OFFSET_PACKNAMES + 54)) +MIDX_OFFSET_OID_LOOKUP=$((MIDX_OFFSET_OID_FANOUT + 256 * $MIDX_OID_FANOUT_WIDTH)) +MIDX_OFFSET_OBJECT_OFFSETS=$(($MIDX_OFFSET_OID_LOOKUP + $NUM_OBJECTS * $HASH_LEN)) +MIDX_OFFSET_LARGE_OFFSETS=$(($MIDX_OFFSET_OBJECT_OFFSETS + $NUM_OBJECTS * $MIDX_OFFSET_WIDTH)) +MIDX_BYTE_LARGE_OFFSET=$(($MIDX_OFFSET_LARGE_OFFSETS + 3)) + +test_expect_success 'verify incorrect 64-bit offset' ' + corrupt_midx_and_verify $MIDX_BYTE_LARGE_OFFSET "\07" objects64 \ + "incorrect object offset" +' + test_done From patchwork Thu Sep 13 18:02:26 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Philippe Blain via GitGitGadget X-Patchwork-Id: 10599893 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B28A513B6 for ; Thu, 13 Sep 2018 18:02:30 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A02FC2B414 for ; Thu, 13 Sep 2018 18:02:30 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 948162B416; Thu, 13 Sep 2018 18:02:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 377F32B414 for ; Thu, 13 Sep 2018 18:02:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728221AbeIMXNC (ORCPT ); Thu, 13 Sep 2018 19:13:02 -0400 Received: from mail-pl1-f193.google.com ([209.85.214.193]:42848 "EHLO mail-pl1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728193AbeIMXNC (ORCPT ); Thu, 13 Sep 2018 19:13:02 -0400 Received: by mail-pl1-f193.google.com with SMTP id g23-v6so2937573plq.9 for ; Thu, 13 Sep 2018 11:02:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=kDbI8PetvqkYBZ+33UKq7bQUaD3Bl3dwcBq+W7V66oo=; b=CTFvjng6KbH6V18EpLOvUiZFNzNUIz+e9Ksvx2TXNmpvKAx9wx9Lk5IL4kQZoy7wkF iTidefp2toOrx9awLCxMsbcbpOjMZlrhzpHOJ2yj207yxPZ7Ji1oRn0BOsyH9fOnY0c7 hiMQ/vJB+aL9OMrGNd66WAY/rquqrKitHNDIr4+3aX84zUAmbi0J/0xHfXiI08BtkCgr oYztnQlPtuU2ty83IIbFusk5N8yGHKgNKyxu8f1k/kjVLBJrATeaZUqBzj20tzCxzy/d 7lzR0l+1n+c3ho6/tv2mqgLddvcRKYFZdo793juoo78E3dOJXdhItn6McvjhTS3Ahd/j Er0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=kDbI8PetvqkYBZ+33UKq7bQUaD3Bl3dwcBq+W7V66oo=; b=OBHKLHu7egmeoJy293yqbnOIoDRgCI8bETtBdxAkgxBoQPzpaW6DA/uHfiwQIWAXWU Xaw3YrAHEG9BVnG1RYtoSFzGxb1NV81bS7v7z+MHKN8EL/L695X8mc8CbxzwS3q1HGMG AZy57uy2UtkRauTUgI9E0Dtn2sW/dEjn19LQr0SYZFUkW05bOGnDS6WFboeH3R9dTf31 tMM/2DAjK5676PN98v5E71qBL81ZlCKXwF+opr2NNFUUdrgD//AhtcdkWS+Fa0N/Rm9N 1t2gigwFWz8YELEmROny2BFZqGbX1dawnlDYamue8Moe9jbjNLi/TvL2Td5fNzestr6T rRCA== X-Gm-Message-State: APzg51D9cWgTXOVLRE0ay2yf1rHnLhH/laxEnX27t8CUhoveR3yOKsJT 3yAcFW6Oz99747O0BdKI5/GrXAdg X-Google-Smtp-Source: ANB0VdaCNsj4vvxyQ8Nk6ed/0064xX1gwnHXbhsdMrqiFFnT5tAT9MQrIq0rgzw0XvpNFfNynT5rqA== X-Received: by 2002:a17:902:82c5:: with SMTP id u5-v6mr8275594plz.83.1536861747340; Thu, 13 Sep 2018 11:02:27 -0700 (PDT) Received: from [127.0.0.1] ([40.112.139.85]) by smtp.gmail.com with ESMTPSA id n29-v6sm5923861pgl.30.2018.09.13.11.02.26 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 13 Sep 2018 11:02:26 -0700 (PDT) Date: Thu, 13 Sep 2018 11:02:26 -0700 (PDT) X-Google-Original-Date: Thu, 13 Sep 2018 18:02:09 GMT Message-Id: <29ebc1716123a953663c43064d73b98cf2fe0cbd.1536861730.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Derrick Stolee via GitGitGadget" Subject: [PATCH v2 10/11] multi-pack-index: report progress during 'verify' Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: avarab@gmail.com, Junio C Hamano , Derrick Stolee Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Derrick Stolee When verifying a multi-pack-index, the only action that takes significant time is checking the object offsets. For example, to verify a multi-pack-index containing 6.2 million objects in the Linux kernel repository takes 1.3 seconds on my machine. 99% of that time is spent looking up object offsets in each of the packfiles and comparing them to the multi-pack-index offset. Add a progress indicator for that section of the 'verify' verb. Signed-off-by: Derrick Stolee --- midx.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/midx.c b/midx.c index 47e7e6113a..4d4c930522 100644 --- a/midx.c +++ b/midx.c @@ -7,6 +7,7 @@ #include "object-store.h" #include "sha1-lookup.h" #include "midx.h" +#include "progress.h" #define MIDX_SIGNATURE 0x4d494458 /* "MIDX" */ #define MIDX_VERSION 1 @@ -940,6 +941,7 @@ static void midx_report(const char *fmt, ...) int verify_midx_file(const char *object_dir) { uint32_t i; + struct progress *progress = NULL; struct multi_pack_index *m = load_multi_pack_index(object_dir, 1); verify_midx_error = 0; @@ -971,6 +973,7 @@ int verify_midx_file(const char *object_dir) i, oid_to_hex(&oid1), oid_to_hex(&oid2), i + 1); } + progress = start_progress(_("Verifying object offsets"), m->num_objects); for (i = 0; i < m->num_objects; i++) { struct object_id oid; struct pack_entry e; @@ -995,7 +998,10 @@ int verify_midx_file(const char *object_dir) if (m_offset != p_offset) midx_report(_("incorrect object offset for oid[%d] = %s: %"PRIx64" != %"PRIx64), i, oid_to_hex(&oid), m_offset, p_offset); + + display_progress(progress, i + 1); } + stop_progress(&progress); return verify_midx_error; } From patchwork Thu Sep 13 18:02:27 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Philippe Blain via GitGitGadget X-Patchwork-Id: 10599895 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6EEC415A7 for ; Thu, 13 Sep 2018 18:02:31 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5DFE82B414 for ; Thu, 13 Sep 2018 18:02:31 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 51DD02B416; Thu, 13 Sep 2018 18:02:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E7EF92B414 for ; Thu, 13 Sep 2018 18:02:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728226AbeIMXND (ORCPT ); Thu, 13 Sep 2018 19:13:03 -0400 Received: from mail-pl1-f169.google.com ([209.85.214.169]:36387 "EHLO mail-pl1-f169.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728193AbeIMXND (ORCPT ); Thu, 13 Sep 2018 19:13:03 -0400 Received: by mail-pl1-f169.google.com with SMTP id p5-v6so2949121plk.3 for ; Thu, 13 Sep 2018 11:02:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=7pOMN3l/b4ziuxVezIX6mLt4cpjx8qscyj3hQtH8GgY=; b=egY0RfxXm2H94Oog+KMDSILoJFRGFR1qaLjFbVScdUcQCexOIc35nzmQyTYiLM3diz +dvHXETHA7Uf0DzFNtqkR3kkTnlqoZHwtieMLVkxyLIksR9iXHfMOWpRurihE6EVjynt RnjNT719mMW2AzMaD397G55DV0MIL8U2dsStfOUCq2bm+EPiTpIcBScP+Rxwl7eRyk1S Z/u0OmIYcyrTs/CQ74wbNKv8zc0Sko1M1YPjdYBwzZug1IVBYxFFBew6g1serWLGWHJq +La6OgC138K1n3LYBKWQ7hOd/Klr48JFgnP1JhT+Kktq2nRVv1R7ZR0uHKx+fPYeLXdB YFAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=7pOMN3l/b4ziuxVezIX6mLt4cpjx8qscyj3hQtH8GgY=; b=pgr0/H/qoGCJJdzT3rNXSbHce44ODcCGHMwwPf6m6uZ59PLJbfSNySgV14GE0ei9mR QI3w+4+F58cQOLBDuNTDoaXoVNkNfO6s06ydklLZB3aSiEHbGQKbGpnwwI1k7Fxr2ROA lIojWK5OvUafXgFvoHFynw3TetJsGlZkrwZ20Suzh6RTL7XWj5jCWIpzYMNLAI9WfCRV j7hNIaa3evED8h+TRG4zhAdehbz1IPxgbgvprQO8BQXLm13rdUcYY66SQ06mAzEbY2BZ Ld++i033lragPcLPjtCvUqjgTHmqCEuaqOye9maYzuZXz17hNkkpGThUvyqlMgHsmvoX xEXA== X-Gm-Message-State: APzg51Bvj7pbTqYr10bmiEEXWmKtHIW4P2B82hEPSUfPZ3oEBuZfpg7B wAjypOD5BjkW9M8KioAg0xWY6OcX X-Google-Smtp-Source: ANB0VdYJ4OO+Jclms+S9MRhXX2QbjTfIlkyd9fK1/5tvxsClFHD/biyyUb5VbX6Gvc6Br3UTihxHow== X-Received: by 2002:a17:902:3a2:: with SMTP id d31-v6mr8281986pld.287.1536861748717; Thu, 13 Sep 2018 11:02:28 -0700 (PDT) Received: from [127.0.0.1] ([40.112.139.85]) by smtp.gmail.com with ESMTPSA id a2-v6sm4755627pgc.68.2018.09.13.11.02.27 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 13 Sep 2018 11:02:27 -0700 (PDT) Date: Thu, 13 Sep 2018 11:02:27 -0700 (PDT) X-Google-Original-Date: Thu, 13 Sep 2018 18:02:10 GMT Message-Id: <406c88b456c06bc42a9bb23bda48b6b3be461435.1536861730.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Derrick Stolee via GitGitGadget" Subject: [PATCH v2 11/11] fsck: verify multi-pack-index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: avarab@gmail.com, Junio C Hamano , Derrick Stolee Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Derrick Stolee When core.multiPackIndex is true, we may have a multi-pack-index in our object directory. Add calls to 'git multi-pack-index verify' at the end of 'git fsck' if so. Signed-off-by: Derrick Stolee --- builtin/fsck.c | 18 ++++++++++++++++++ t/t5319-multi-pack-index.sh | 13 ++++++++++++- 2 files changed, 30 insertions(+), 1 deletion(-) diff --git a/builtin/fsck.c b/builtin/fsck.c index 63c8578cc1..06eb421720 100644 --- a/builtin/fsck.c +++ b/builtin/fsck.c @@ -848,5 +848,23 @@ int cmd_fsck(int argc, const char **argv, const char *prefix) } } + if (!git_config_get_bool("core.multipackindex", &i) && i) { + struct child_process midx_verify = CHILD_PROCESS_INIT; + const char *midx_argv[] = { "multi-pack-index", "verify", NULL, NULL, NULL }; + + midx_verify.argv = midx_argv; + midx_verify.git_cmd = 1; + if (run_command(&midx_verify)) + errors_found |= ERROR_COMMIT_GRAPH; + + prepare_alt_odb(the_repository); + for (alt = the_repository->objects->alt_odb_list; alt; alt = alt->next) { + midx_argv[2] = "--object-dir"; + midx_argv[3] = alt->path; + if (run_command(&midx_verify)) + errors_found |= ERROR_COMMIT_GRAPH; + } + } + return errors_found; } diff --git a/t/t5319-multi-pack-index.sh b/t/t5319-multi-pack-index.sh index 828c240389..bd8e841b81 100755 --- a/t/t5319-multi-pack-index.sh +++ b/t/t5319-multi-pack-index.sh @@ -160,12 +160,17 @@ corrupt_midx_and_verify() { DATA="${2:-\0}" && OBJDIR=$3 && GREPSTR="$4" && + COMMAND="$5" && + if test -z "$COMMAND" + then + COMMAND="git multi-pack-index verify --object-dir=$OBJDIR" + fi && FILE=$OBJDIR/pack/multi-pack-index && chmod a+w $FILE && test_when_finished mv midx-backup $FILE && cp $FILE midx-backup && printf "$DATA" | dd of="$FILE" bs=1 seek="$POS" conv=notrunc && - test_must_fail git multi-pack-index verify --object-dir=$OBJDIR 2>test_err && + test_must_fail $COMMAND 2>test_err && grep -v "^+" test_err >err && test_i18ngrep "$GREPSTR" err } @@ -258,6 +263,12 @@ test_expect_success 'verify incorrect offset' ' "incorrect object offset" ' +test_expect_success 'git-fsck incorrect offset' ' + corrupt_midx_and_verify $MIDX_BYTE_OFFSET "\07" $objdir \ + "incorrect object offset" \ + "git -c core.multipackindex=true fsck" +' + test_expect_success 'repack removes multi-pack-index' ' test_path_is_file $objdir/pack/multi-pack-index && git repack -adf &&