From patchwork Tue Aug 25 16:04:36 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 11735941 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5D82314F6 for ; Tue, 25 Aug 2020 16:04:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 39C8E20782 for ; Tue, 25 Aug 2020 16:04:46 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20150623.gappssmtp.com header.i=@ttaylorr-com.20150623.gappssmtp.com header.b="rWHkgmWD" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726988AbgHYQEp (ORCPT ); Tue, 25 Aug 2020 12:04:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48400 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726934AbgHYQEm (ORCPT ); Tue, 25 Aug 2020 12:04:42 -0400 Received: from mail-qt1-x834.google.com (mail-qt1-x834.google.com [IPv6:2607:f8b0:4864:20::834]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 01F9AC061574 for ; Tue, 25 Aug 2020 09:04:42 -0700 (PDT) Received: by mail-qt1-x834.google.com with SMTP id k20so5764436qtq.11 for ; Tue, 25 Aug 2020 09:04:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=h4gSGXt6cvZR8feRVQV9O3+vgKzo//dCQb8fIe+AvyQ=; b=rWHkgmWDY0O89Qu/vYlDkeohH9X9g/zcRfiSwMjPm1UxMHNPX2cYIlrCImkGfhLCef /M08rUXWM8dJMRaDkl6ZZ46iwA8KbWTNA5sKaCV7GSa8j9TZenNmUq7l3LP+BwSikby2 D5jA4HBABmSONlFQi+js814pI9LELw/By1CoPPuCQq5bp+paL4qZg7qDGHPUnkOPWcqY x9gpRHQrQMxsABDVOBybcX66V4IPVYtuVIcYIhdYFE8UGtgSQWIGGo+kc2jkplR8Dpan +8GnR8mMhGFfS/26weFeudKeZ+d7uX1A+yv0d1s12fxlGIOkPSSOj8ifySpEUEHlzshn m5Rg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=h4gSGXt6cvZR8feRVQV9O3+vgKzo//dCQb8fIe+AvyQ=; b=qOoX+m3boRJlIO5ROH0Q904lwFuHNK/Ik03YwmhhgklrpwlRozPVM3wqC5Q01rT/fO 3CFuBfeAXo6fUV+G1hrazbLJifc/ZllcwbpLp+at7cPaGUES9tG7uRzfQkZQiFsIe+mZ dtrQwSc2C9LEUPAyRCchUgVbYP+sUrhlGAkBHxYnpr5+gBYXl6OgwrtQJBy8urcD7n0v uF/TF+xQlD4o0JqKEMXivWW1Qw+HolYF0YKKgdvpWo79bJTfMCToKByxSs/BiZSzTKLI HTUbvfVFSKvd12GSs27zyeVZ8mfzWBlknxpjMa9F2B+twrTsMVsHjR3laXzvohAws7zt 5dTg== X-Gm-Message-State: AOAM530oL9XI5AhwGXPR6wiTcgPN499NdrhGchrAwy+38RoUGso0sY3z ruhzX9u4nQfC8KoBLycstnnUXgCxL6L1J306 X-Google-Smtp-Source: ABdhPJwbJ45tyCFk5N2oBLRT2fijjY4shCIZJ0J4pIgTRp0NOJjT2SFSj3ikodDrLLiZQSvsc9NNDw== X-Received: by 2002:ac8:480c:: with SMTP id g12mr9869791qtq.257.1598371479238; Tue, 25 Aug 2020 09:04:39 -0700 (PDT) Received: from localhost ([2605:9480:22e:ff10:1558:8dc0:5209:36ba]) by smtp.gmail.com with ESMTPSA id n85sm12108056qkn.80.2020.08.25.09.04.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Aug 2020 09:04:37 -0700 (PDT) Date: Tue, 25 Aug 2020 12:04:36 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: dstolee@microsoft.com, peff@peff.net, sluongng@gmail.com Subject: [PATCH v2] builtin/repack.c: invalidate MIDX only when necessary Message-ID: <87a3b7a5a2f091e2a23c163a7d86effbbbedfa3a.1598371475.git.me@ttaylorr.com> References: <20200825144515.GB7671@syl.lan> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20200825144515.GB7671@syl.lan> Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In 525e18c04b (midx: clear midx on repack, 2018-07-12), 'git repack' learned to remove a multi-pack-index file if it added or removed a pack from the object store. This mechanism is a little over-eager, since it is only necessary to drop a MIDX if 'git repack' removes a pack that the MIDX references. Adding a pack outside of the MIDX does not require invalidating the MIDX, and likewise for removing a pack the MIDX does not know about. Teach 'git repack' to check for this by loading the MIDX, and checking whether the to-be-removed pack is known to the MIDX. This requires a slightly odd alternation to a test in t5319, which is explained with a comment. A new test is added to show that the MIDX is left alone when both packs known to it are marked as .keep, but two packs unknown to it are removed and combined into one new pack. Helped-by: Derrick Stolee Signed-off-by: Taylor Blau --- Range-diff against v1: 1: ef9186a8df ! 1: 87a3b7a5a2 builtin/repack.c: invalidate MIDX only when necessary @@ Commit message Teach 'git repack' to check for this by loading the MIDX, and checking whether the to-be-removed pack is known to the MIDX. This requires a slightly odd alternation to a test in t5319, which is explained with a - comment. + comment. A new test is added to show that the MIDX is left alone when + both packs known to it are marked as .keep, but two packs unknown to it + are removed and combined into one new pack. + Helped-by: Derrick Stolee Signed-off-by: Taylor Blau ## builtin/repack.c ## @@ t/t5319-multi-pack-index.sh: test_expect_success 'repack with the --no-progress test_path_is_missing $objdir/pack/multi-pack-index ' -+test_expect_success 'repack preserves multi-pack-index when deleting unknown packs' ' -+ git multi-pack-index write && -+ cp $objdir/pack/multi-pack-index $objdir/pack/multi-pack-index.bak && -+ test_when_finished "rm -f $objdir/pack/multi-pack-index.bak" && -+ -+ # Write a new pack that is unknown to the multi-pack-index. -+ git hash-object -w blob && -+ git pack-objects $objdir/pack/pack pack-input <<-EOF && ++ HEAD ++ ^HEAD~1 ++ EOF ++ test_commit 3 && ++ pack3=$(git pack-objects --revs $packdir/pack string, exts[ext].name); if (!file_exists(fname)) { diff --git a/t/t5319-multi-pack-index.sh b/t/t5319-multi-pack-index.sh index 43b1b5b2af..f340b376bc 100755 --- a/t/t5319-multi-pack-index.sh +++ b/t/t5319-multi-pack-index.sh @@ -382,12 +382,52 @@ test_expect_success 'repack with the --no-progress option' ' test_line_count = 0 err ' -test_expect_success 'repack removes multi-pack-index' ' +test_expect_success 'repack removes multi-pack-index when deleting packs' ' test_path_is_file $objdir/pack/multi-pack-index && - GIT_TEST_MULTI_PACK_INDEX=0 git repack -adf && + # Set GIT_TEST_MULTI_PACK_INDEX to 0 to avoid writing a new + # multi-pack-index after repacking, but set "core.multiPackIndex" to + # true so that "git repack" can read the existing MIDX. + GIT_TEST_MULTI_PACK_INDEX=0 git -c core.multiPackIndex repack -adf && test_path_is_missing $objdir/pack/multi-pack-index ' +test_expect_success 'repack preserves multi-pack-index when creating packs' ' + git init preserve && + test_when_finished "rm -fr preserve" && + ( + cd preserve && + packdir=.git/objects/pack && + midx=$packdir/multi-pack-index && + + test_commit 1 && + pack1=$(git pack-objects --all $packdir/pack) && + touch $packdir/pack-$pack1.keep && + test_commit 2 && + pack2=$(git pack-objects --revs $packdir/pack) && + touch $packdir/pack-$pack2.keep && + + git multi-pack-index write && + cp $midx $midx.bak && + + cat >pack-input <<-EOF && + HEAD + ^HEAD~1 + EOF + test_commit 3 && + pack3=$(git pack-objects --revs $packdir/pack