From patchwork Wed Mar 19 22:52:45 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 14023264 Received: from mail-qv1-f51.google.com (mail-qv1-f51.google.com [209.85.219.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A9818221DA4 for ; Wed, 19 Mar 2025 22:52:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742424770; cv=none; b=DficEzBTftFsITQGBWu/cXINn1e4+dEhydZWxyfehAcI1JCIACp+PM6VGaIA0yG8r2abOIW5A7RymFrFwiEnRQU91P+Zk8ydb9Nkz8EJV1WJd34n4ettCqCqR2K60BLBw0jvgHKIrrt6IO7LobgVf9kpDuNCSFtTvsmz/d5weP8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742424770; c=relaxed/simple; bh=M8lMyryXTlKntXdKRpG8m/DoLQCBXa13XkyUMZkHiug=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=uyTPdrB1zUGpNs3tNB6BG4/00+T16T/rOnfmqb2qgUT0PIwDCLXRtsCeLRIfvxqKQrzrWTL8tCmIEzf9EYxBmkiNu9Q3I3VJewXSUoMvsRRVTyCOveQQ45Su1ArF71rbgjEXaf0Ta7jcVU6k0uvMXOc51kYlGTL4YCbf8cMERSA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com; spf=pass smtp.mailfrom=ttaylorr.com; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b=U8jE4sMW; arc=none smtp.client-ip=209.85.219.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b="U8jE4sMW" Received: by mail-qv1-f51.google.com with SMTP id 6a1803df08f44-6e8f4c50a8fso1264666d6.1 for ; Wed, 19 Mar 2025 15:52:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1742424767; x=1743029567; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=f0yqSXE75GzGiSV3FC35bObeuwJ2MJyt6g58ImoIqSM=; b=U8jE4sMWVDS03LzgDfZSnZEFVy0RcLBQtRLDTfwcbQxWCHKEP0A0QKgl8nYiEp01wR +30nicss55pADjeOQGIcGbD9hpy9TstHpUEI0OcY5kTlocszfHVqz6PLawGRDaWJe5cZ wzTGqCmYZNdvfw1EHqOqmecPWJEmx2626N3Q/qVLdR3zTd1Nc2TfThzDOUtTsWUGzjGR b+fLkktqNeSafJTHnyhbzAOPdAe6dsKZ2iwupMyO9EHs9+9OXOgW+ZFpQlMHhcsrLa1o 0SdUVEA8VgZtXbLT0AtCOuWFBJwb5io9irVOq9X4M37EQCp/Dq6SyDRH9WKr69m4+9+h seeQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742424767; x=1743029567; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=f0yqSXE75GzGiSV3FC35bObeuwJ2MJyt6g58ImoIqSM=; b=HaKI3POF6tz2Zp4qIv1Ijxp0fGtkGWRbmZ4QoaJchw0QruSz6Lmlc42VU/Jd18RLmw OTTk9OnwJ067wdlRLa73coIN6l9pw6qUV+ZhnNfYz07Z/wV6Nf02yGUK+bvI+yDTaAbR 5Kah6Kfbi2qJO3FWABpWTViGrKCsFgudPxzN9FmJZwsFuY6GSnYuh402XseS02oRWEUE SEQ6wK9lBZUXQG7nR/94YsuV+7QTsBqQ3hGuJ/ILmqJ+f336EBcAEgDeAJHGaGvqsOnJ d8vLY40aUMvpzsX4CjkIu10sdBy/rpmWMQsY+QHzwR5Fs2oU1sv+7Uxyo54IGjKbYtVZ pC8A== X-Gm-Message-State: AOJu0Yw1ZBjdRwh54JQp90AQbS/ZcCrWGdInzqnUrY0FF3rMjE+X7PSZ H3KT9+xiR59B61anEKwZb6PPGjOzmAmkjROnQYXOxf1nEEs2HRN4DrNR9NSu01LLZA32y+OIwst kfbU= X-Gm-Gg: ASbGncs7L4FN7BDdwJDpYX2jtVcAe9OrfdSqk2/po+A5RzMPvuGz8vd5EUKu4BKN6nI o/Q1xNxkNoxPQbvV1iy4Gy2/v4S0qjEs9r1C5uhK0GlL5cBnYMkOYI3uRJAweltQqFpCckkGadX 000auLdDLQ6zSALl7UmvX6RZ5DwK9RQmuCFSPEYRogBx96YX8B2mkl8kVsLdT0yfYs+psUFrsiA 3RWhHqziKPHI0uifGlpQlu/TO3NbC8xl6UoLRmmv/UP3k9DiuxThTTaKH0aRI4wwN2f3T979BKJ AoaXIy44niSc6vSQPq5Z1N22DDJbBXMYoyse/5bbW47fARD0Imn/ezoJ6FPBkOIVh+TONvxDY0W FQer0uaMnKiiNGB3/ X-Google-Smtp-Source: AGHT+IHRhPBBhAkccH2YqkBWuzARj7wWYqf+ZI+irUOKJs23ExNDt6kLGTnu9Ux1GNGFUeqghdy1Cg== X-Received: by 2002:ad4:5cee:0:b0:6e6:9ce0:da9d with SMTP id 6a1803df08f44-6eb352d586dmr11696226d6.27.1742424767121; Wed, 19 Mar 2025 15:52:47 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with UTF8SMTPSA id 6a1803df08f44-6eade231bcbsm84982686d6.36.2025.03.19.15.52.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Mar 2025 15:52:46 -0700 (PDT) Date: Wed, 19 Mar 2025 18:52:45 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Junio C Hamano , Elijah Newren Subject: [PATCH v2 1/5] t/t5329-pack-objects-cruft.sh: evict 'repack'-related tests Message-ID: <0aa8aa65c130fd62d38b4944c1ce2a97451b0064.1742424671.git.me@ttaylorr.com> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: The cruft pack feature has two primary test scripts which exercise various parts of it, which are: - t5329-pack-objects-cruft.sh - t7704-repack-cruft.sh The former is designed to test low-level pack generation mechanics at the 'git pack-objects --cruft'-level, which is plumbing. The latter, on the other hand, is designed to test the user-facing behavior through 'git repack --cruft', which is porcelain (under the "ancillary manipulators" sub-section). At some point a handful of tests which should have been added to the latter script were instead written to the former. This isn't a huge deal, but rectifying it is straightforward. Move a handful of 'repack'-related tests out of t5329 and into their rightful home in t7704. Signed-off-by: Taylor Blau --- t/t5329-pack-objects-cruft.sh | 250 ---------------------------------- t/t7704-repack-cruft.sh | 250 ++++++++++++++++++++++++++++++++++ 2 files changed, 250 insertions(+), 250 deletions(-) diff --git a/t/t5329-pack-objects-cruft.sh b/t/t5329-pack-objects-cruft.sh index b71a0aef40..60dac8312d 100755 --- a/t/t5329-pack-objects-cruft.sh +++ b/t/t5329-pack-objects-cruft.sh @@ -360,43 +360,6 @@ test_expect_success 'expired objects are pruned' ' ) ' -test_expect_success 'repack --cruft generates a cruft pack' ' - git init repo && - test_when_finished "rm -fr repo" && - ( - cd repo && - - test_commit reachable && - git branch -M main && - git checkout --orphan other && - test_commit unreachable && - - git checkout main && - git branch -D other && - git tag -d unreachable && - # objects are not cruft if they are contained in the reflogs - git reflog expire --all --expire=all && - - git rev-list --objects --all --no-object-names >reachable.raw && - git cat-file --batch-all-objects --batch-check="%(objectname)" >objects && - sort reachable && - comm -13 reachable objects >unreachable && - - git repack --cruft -d && - - cruft=$(basename $(ls $packdir/pack-*.mtimes) .mtimes) && - pack=$(basename $(ls $packdir/pack-*.pack | grep -v $cruft) .pack) && - - git show-index <$packdir/$pack.idx >actual.raw && - cut -f2 -d" " actual.raw | sort >actual && - test_cmp reachable actual && - - git show-index <$packdir/$cruft.idx >actual.raw && - cut -f2 -d" " actual.raw | sort >actual && - test_cmp unreachable actual - ) -' - test_expect_success 'loose objects mtimes upsert others' ' git init repo && test_when_finished "rm -fr repo" && @@ -470,219 +433,6 @@ test_expect_success 'expiring cruft objects with git gc' ' ) ' -test_expect_success 'cruft packs are not included in geometric repack' ' - git init repo && - test_when_finished "rm -fr repo" && - ( - cd repo && - - test_commit reachable && - git repack -Ad && - git branch -M main && - - git checkout --orphan other && - test_commit cruft && - git repack -d && - - git checkout main && - git branch -D other && - git tag -d cruft && - git reflog expire --all --expire=all && - - git repack --cruft && - - find $packdir -type f | sort >before && - git repack --geometric=2 -d && - find $packdir -type f | sort >after && - - test_cmp before after - ) -' - -test_expect_success 'repack --geometric collects once-cruft objects' ' - git init repo && - test_when_finished "rm -fr repo" && - ( - cd repo && - - test_commit reachable && - git repack -Ad && - git branch -M main && - - git checkout --orphan other && - git rm -rf . && - test_commit --no-tag cruft && - cruft="$(git rev-parse HEAD)" && - - git checkout main && - git branch -D other && - git reflog expire --all --expire=all && - - # Pack the objects created in the previous step into a cruft - # pack. Intentionally leave loose copies of those objects - # around so we can pick them up in a subsequent --geometric - # reapack. - git repack --cruft && - - # Now make those objects reachable, and ensure that they are - # packed into the new pack created via a --geometric repack. - git update-ref refs/heads/other $cruft && - - # Without this object, the set of unpacked objects is exactly - # the set of objects already in the cruft pack. Tweak that set - # to ensure we do not overwrite the cruft pack entirely. - test_commit reachable2 && - - find $packdir -name "pack-*.idx" | sort >before && - git repack --geometric=2 -d && - find $packdir -name "pack-*.idx" | sort >after && - - { - git rev-list --objects --no-object-names $cruft && - git rev-list --objects --no-object-names reachable..reachable2 - } >want.raw && - sort want.raw >want && - - pack=$(comm -13 before after) && - git show-index <$pack >objects.raw && - - cut -d" " -f2 objects.raw | sort >got && - - test_cmp want got - ) -' - -test_expect_success 'cruft repack with no reachable objects' ' - git init repo && - test_when_finished "rm -fr repo" && - ( - cd repo && - - test_commit base && - git repack -ad && - - base="$(git rev-parse base)" && - - git for-each-ref --format="delete %(refname)" >in && - git update-ref --stdin in && - git hash-object -w -t blob in -} - -find_pack () { - for idx in $(ls $packdir/pack-*.idx) - do - git show-index <$idx >out && - if grep -q "$1" out - then - echo $idx - fi || return 1 - done -} - -test_expect_success 'cruft repack with --max-pack-size' ' - git init max-pack-size && - ( - cd max-pack-size && - test_commit base && - - # two cruft objects which exceed the maximum pack size - foo=$(write_blob foo 1048576) && - bar=$(write_blob bar 1048576) && - test-tool chmtime --get -1000 \ - "$objdir/$(test_oid_to_path $foo)" >foo.mtime && - test-tool chmtime --get -2000 \ - "$objdir/$(test_oid_to_path $bar)" >bar.mtime && - git repack --cruft --max-pack-size=1M && - find $packdir -name "*.mtimes" >cruft && - test_line_count = 2 cruft && - - foo_mtimes="$(basename $(find_pack $foo) .idx).mtimes" && - bar_mtimes="$(basename $(find_pack $bar) .idx).mtimes" && - test-tool pack-mtimes $foo_mtimes >foo.actual && - test-tool pack-mtimes $bar_mtimes >bar.actual && - - echo "$foo $(cat foo.mtime)" >foo.expect && - echo "$bar $(cat bar.mtime)" >bar.expect && - - test_cmp foo.expect foo.actual && - test_cmp bar.expect bar.actual && - test "$foo_mtimes" != "$bar_mtimes" - ) -' - -test_expect_success 'cruft repack with pack.packSizeLimit' ' - ( - cd max-pack-size && - # repack everything back together to remove the existing cruft - # pack (but to keep its objects) - git repack -adk && - git -c pack.packSizeLimit=1M repack --cruft && - # ensure the same post condition is met when --max-pack-size - # would otherwise be inferred from the configuration - find $packdir -name "*.mtimes" >cruft && - test_line_count = 2 cruft && - for pack in $(cat cruft) - do - test-tool pack-mtimes "$(basename $pack)" >objects && - test_line_count = 1 objects || return 1 - done - ) -' - -test_expect_success 'cruft repack respects repack.cruftWindow' ' - git init repo && - test_when_finished "rm -fr repo" && - ( - cd repo && - - test_commit base && - - GIT_TRACE2_EVENT=$(pwd)/event.trace \ - git -c pack.window=1 -c repack.cruftWindow=2 repack \ - --cruft --window=3 && - - grep "pack-objects.*--window=2.*--cruft" event.trace - ) -' - -test_expect_success 'cruft repack respects --window by default' ' - git init repo && - test_when_finished "rm -fr repo" && - ( - cd repo && - - test_commit base && - - GIT_TRACE2_EVENT=$(pwd)/event.trace \ - git -c pack.window=2 repack --cruft --window=3 && - - grep "pack-objects.*--window=3.*--cruft" event.trace - ) -' - -test_expect_success 'cruft repack respects --quiet' ' - git init repo && - test_when_finished "rm -fr repo" && - ( - cd repo && - - test_commit base && - GIT_PROGRESS_DELAY=0 git repack --cruft --quiet 2>err && - test_must_be_empty err - ) -' - test_expect_success 'cruft --local drops unreachable objects' ' git init alternate && git init repo && diff --git a/t/t7704-repack-cruft.sh b/t/t7704-repack-cruft.sh index 43d2947d28..cd452040ea 100755 --- a/t/t7704-repack-cruft.sh +++ b/t/t7704-repack-cruft.sh @@ -477,4 +477,254 @@ test_expect_success 'reachable packs are preferred over cruft ones' ' ) ' +test_expect_success 'repack --cruft generates a cruft pack' ' + git init repo && + test_when_finished "rm -fr repo" && + ( + cd repo && + + test_commit reachable && + git branch -M main && + git checkout --orphan other && + test_commit unreachable && + + git checkout main && + git branch -D other && + git tag -d unreachable && + # objects are not cruft if they are contained in the reflogs + git reflog expire --all --expire=all && + + git rev-list --objects --all --no-object-names >reachable.raw && + git cat-file --batch-all-objects --batch-check="%(objectname)" >objects && + sort reachable && + comm -13 reachable objects >unreachable && + + git repack --cruft -d && + + cruft=$(basename $(ls $packdir/pack-*.mtimes) .mtimes) && + pack=$(basename $(ls $packdir/pack-*.pack | grep -v $cruft) .pack) && + + git show-index <$packdir/$pack.idx >actual.raw && + cut -f2 -d" " actual.raw | sort >actual && + test_cmp reachable actual && + + git show-index <$packdir/$cruft.idx >actual.raw && + cut -f2 -d" " actual.raw | sort >actual && + test_cmp unreachable actual + ) +' + +test_expect_success 'cruft packs are not included in geometric repack' ' + git init repo && + test_when_finished "rm -fr repo" && + ( + cd repo && + + test_commit reachable && + git repack -Ad && + git branch -M main && + + git checkout --orphan other && + test_commit cruft && + git repack -d && + + git checkout main && + git branch -D other && + git tag -d cruft && + git reflog expire --all --expire=all && + + git repack --cruft && + + find $packdir -type f | sort >before && + git repack --geometric=2 -d && + find $packdir -type f | sort >after && + + test_cmp before after + ) +' + +test_expect_success 'repack --geometric collects once-cruft objects' ' + git init repo && + test_when_finished "rm -fr repo" && + ( + cd repo && + + test_commit reachable && + git repack -Ad && + git branch -M main && + + git checkout --orphan other && + git rm -rf . && + test_commit --no-tag cruft && + cruft="$(git rev-parse HEAD)" && + + git checkout main && + git branch -D other && + git reflog expire --all --expire=all && + + # Pack the objects created in the previous step into a cruft + # pack. Intentionally leave loose copies of those objects + # around so we can pick them up in a subsequent --geometric + # reapack. + git repack --cruft && + + # Now make those objects reachable, and ensure that they are + # packed into the new pack created via a --geometric repack. + git update-ref refs/heads/other $cruft && + + # Without this object, the set of unpacked objects is exactly + # the set of objects already in the cruft pack. Tweak that set + # to ensure we do not overwrite the cruft pack entirely. + test_commit reachable2 && + + find $packdir -name "pack-*.idx" | sort >before && + git repack --geometric=2 -d && + find $packdir -name "pack-*.idx" | sort >after && + + { + git rev-list --objects --no-object-names $cruft && + git rev-list --objects --no-object-names reachable..reachable2 + } >want.raw && + sort want.raw >want && + + pack=$(comm -13 before after) && + git show-index <$pack >objects.raw && + + cut -d" " -f2 objects.raw | sort >got && + + test_cmp want got + ) +' + +test_expect_success 'cruft repack with no reachable objects' ' + git init repo && + test_when_finished "rm -fr repo" && + ( + cd repo && + + test_commit base && + git repack -ad && + + base="$(git rev-parse base)" && + + git for-each-ref --format="delete %(refname)" >in && + git update-ref --stdin in && + git hash-object -w -t blob in +} + +find_pack () { + for idx in $(ls $packdir/pack-*.idx) + do + git show-index <$idx >out && + if grep -q "$1" out + then + echo $idx + fi || return 1 + done +} + +test_expect_success 'cruft repack with --max-pack-size' ' + git init max-pack-size && + ( + cd max-pack-size && + test_commit base && + + # two cruft objects which exceed the maximum pack size + foo=$(write_blob foo 1048576) && + bar=$(write_blob bar 1048576) && + test-tool chmtime --get -1000 \ + "$objdir/$(test_oid_to_path $foo)" >foo.mtime && + test-tool chmtime --get -2000 \ + "$objdir/$(test_oid_to_path $bar)" >bar.mtime && + git repack --cruft --max-pack-size=1M && + find $packdir -name "*.mtimes" >cruft && + test_line_count = 2 cruft && + + foo_mtimes="$(basename $(find_pack $foo) .idx).mtimes" && + bar_mtimes="$(basename $(find_pack $bar) .idx).mtimes" && + test-tool pack-mtimes $foo_mtimes >foo.actual && + test-tool pack-mtimes $bar_mtimes >bar.actual && + + echo "$foo $(cat foo.mtime)" >foo.expect && + echo "$bar $(cat bar.mtime)" >bar.expect && + + test_cmp foo.expect foo.actual && + test_cmp bar.expect bar.actual && + test "$foo_mtimes" != "$bar_mtimes" + ) +' + +test_expect_success 'cruft repack with pack.packSizeLimit' ' + ( + cd max-pack-size && + # repack everything back together to remove the existing cruft + # pack (but to keep its objects) + git repack -adk && + git -c pack.packSizeLimit=1M repack --cruft && + # ensure the same post condition is met when --max-pack-size + # would otherwise be inferred from the configuration + find $packdir -name "*.mtimes" >cruft && + test_line_count = 2 cruft && + for pack in $(cat cruft) + do + test-tool pack-mtimes "$(basename $pack)" >objects && + test_line_count = 1 objects || return 1 + done + ) +' + +test_expect_success 'cruft repack respects repack.cruftWindow' ' + git init repo && + test_when_finished "rm -fr repo" && + ( + cd repo && + + test_commit base && + + GIT_TRACE2_EVENT=$(pwd)/event.trace \ + git -c pack.window=1 -c repack.cruftWindow=2 repack \ + --cruft --window=3 && + + grep "pack-objects.*--window=2.*--cruft" event.trace + ) +' + +test_expect_success 'cruft repack respects --window by default' ' + git init repo && + test_when_finished "rm -fr repo" && + ( + cd repo && + + test_commit base && + + GIT_TRACE2_EVENT=$(pwd)/event.trace \ + git -c pack.window=2 repack --cruft --window=3 && + + grep "pack-objects.*--window=3.*--cruft" event.trace + ) +' + +test_expect_success 'cruft repack respects --quiet' ' + git init repo && + test_when_finished "rm -fr repo" && + ( + cd repo && + + test_commit base && + GIT_PROGRESS_DELAY=0 git repack --cruft --quiet 2>err && + test_must_be_empty err + ) +' + test_done From patchwork Wed Mar 19 22:52:48 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 14023265 Received: from mail-qt1-f180.google.com (mail-qt1-f180.google.com [209.85.160.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6D078221DB6 for ; Wed, 19 Mar 2025 22:52:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.160.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742424773; cv=none; b=ITiRmcjQ30saSdSU3B6CwJVMy3Prf82PbqKhreRHaZlqB+ac73UZgLLf/CpT130EbvEiaAX0nxIF20rpmugartM+iMNncV5ptG+zXulbTn/V8VmPxFoYKcflL8iywYLfBUHUjVt0YiXfnapzdNdW9aRrJOEAMjZD3wcG32uhcYM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742424773; c=relaxed/simple; bh=Sf7sUzDrfgaX4D0MYTRigJASUSWSFcyRXD+uI/TLUm0=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=MDCYMCdc2CSR1TwH27oUt90o1lPL5nBWRUhZEk/hhW+V2JYPdzcBP5rWLYi0LYXpfh99SFx84PyuheXOvVGLXaEWTnk0KmKS49qIM5VuHANd94H9PNnlbuPmQ1ynZXmGe+xDTvH923O3MW7uhXnk5yh+5daeHSFcZnsNtO3gRfo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com; spf=pass smtp.mailfrom=ttaylorr.com; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b=gmweT2ia; arc=none smtp.client-ip=209.85.160.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b="gmweT2ia" Received: by mail-qt1-f180.google.com with SMTP id d75a77b69052e-46fcbb96ba9so3712781cf.0 for ; Wed, 19 Mar 2025 15:52:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1742424770; x=1743029570; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=hkadr0ec63dWV1Qj07O2dbQWEVCNoVPFLDaLtNcdHaU=; b=gmweT2iaDmoLyKUBYn+GO/cinbL4RpCOLLSE/JP+B4ZdvWTBTYlSaLCIE9l2wzX7Hx b4M+DEEMU9TcuG9qycToFL+7p2EcIZuVGUAAxKbkU9ZlsYXjiT8UrqiSQVXXgjcWNfiN Nz1CG77IiEDNDOSS5mmwChSrbq/DuAHntuj/NolYnE4vDyVA+xMJJoF9P6jy1gElZ9IQ 7tWgg8Vweu1DD39P5iEa07CZeQtcraSv3CUMt0hXA9kGtdrHQddXPu4SgYjTzgvQoQrb cie1RE/ogR4WwSUQcll/Yh3W+8zPqtEJ+S+UeftjeVhRNtnxB1n1XUH3whzySTFLwKxc duNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742424770; x=1743029570; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=hkadr0ec63dWV1Qj07O2dbQWEVCNoVPFLDaLtNcdHaU=; b=RAU5/46HGfeWDAJbJEI9U+UpCpgsvNQypb7kvreZfb/b0fj6yD98cYq8yrHmnRd5Wc QTAvsT7RX82skAEV99LQmGizeI4Jm9tn2sQ4D7vC1W37VT2gYnICyIuXmkIWx/FP/1Xr 4XNyJHLd2AqjKZhRJRJyb8rGGGVoMc5GR+1X0t/4WBKLm7xDFVnwZ1jklqbfmVlUtM2T TP0mkzkMP4UJ46xotN4P/QTbsmgLHnzaIw2jMU3uBrYCUr0pU46250zaokUhy1VX4UHA GyecSRavmG5u3d+mYI+rVoTIS/TLqs45AnrO2532fgfs9RxXE+JcTRRs5e44QWK5whth OZtg== X-Gm-Message-State: AOJu0YwYUIOIhSv4vY8VAkq0YL6qVeUcQK8f1ouA5e1bG/6PBjFpCB3c pYeU2XwOZe+0hjM27elftgvhZSG7O4/iPtBA6LUlKQ+fI/B5gihIm4CSvbvICWovs21nkRwX4Jo 5S+4= X-Gm-Gg: ASbGncvIy7MDnDis7kgdbvrg5w0PIcDHjiq3npufWHfzvKbTP9CkKJAmi0aJppgUyoR pvm7MG1YkViGmahrsuDnl33Qdlvo3NxGWamJ/Sk8ezJrdcVLyzZhy67CbaZQZH0Spyz0Md9f/Zw f7UOFca4TVbNajbQ1voG0qHp6FNiSsIaPwdLvihiPkBFNlhDsMWEiZMEL85CaEqIpMJjsaQXrJT PEoHDl18OZ3zIgTBVLcq4Lmqy9Po8CuPOkgFDorQBh9vLpaIBIgxkvC6fOrKeSJas2nvpA1gC5F PoorkOIIOIncIRcA/k8s8VtsQHHVQaBOBbkYhGI2eG88R9V6BwMONCCmcmmvZAu06PR4HzYEN/v tBPg1k0sJVGqmhHJ6 X-Google-Smtp-Source: AGHT+IFwq7MIQQGGJRsx3SObmSnKK6knM+1QiMZ/d22VVL/dc0xnhrBx5ViTATBCqXFxDyQ2/yPqUg== X-Received: by 2002:ad4:5c44:0:b0:6e8:f17e:e00d with SMTP id 6a1803df08f44-6eb2938e55fmr84889736d6.14.1742424770114; Wed, 19 Mar 2025 15:52:50 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with UTF8SMTPSA id 6a1803df08f44-6eade231bb5sm85516446d6.34.2025.03.19.15.52.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Mar 2025 15:52:49 -0700 (PDT) Date: Wed, 19 Mar 2025 18:52:48 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Junio C Hamano , Elijah Newren Subject: [PATCH v2 2/5] t/t7704-repack-cruft.sh: clarify wording in --max-cruft-size tests Message-ID: <5e8bd3e66e304f42d4e84ec2aa9f06b82321d64b.1742424671.git.me@ttaylorr.com> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Now that a number of new tests have landed in t7704, make sure that they all make sense and are testing the things they say they are. Things are mostly OK, but a handful of tests needed tweaks. Those tweaks are as follows: - Use the terms "too large" or "too small" in tests that exercise the '--max-cruft-size' behavior. This has historically been treated as a threshold beneath which to combine cruft packs, but that will change in a subsequent commit. Prepare for that by using a more generic term. - Remove references to "--max-cruft-size" in the freshening tests. These tests provide coverage of our ability to record updated mtimes for objects already in cruft packs whose mtimes are upserted from various sources (loose objects, finding that object in a new pack, another cruft pack, etc.). These have nothing to do with the '--max-cruft-size' feature, and in fact none of the tests even *use* '--max-cruft-size'. Name them appropriately to make it clear that these tests exercise freshening behavior, not '--max-cruft-size' behavior. Signed-off-by: Taylor Blau --- t/t7704-repack-cruft.sh | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/t/t7704-repack-cruft.sh b/t/t7704-repack-cruft.sh index cd452040ea..e6e4c2fad8 100755 --- a/t/t7704-repack-cruft.sh +++ b/t/t7704-repack-cruft.sh @@ -149,7 +149,7 @@ generate_cruft_pack () { echo "$packdir/pack-$pack.mtimes" } -test_expect_success '--max-cruft-size creates new packs when above threshold' ' +test_expect_success '--max-cruft-size creates new packs when too large' ' git init max-cruft-size-large && ( cd max-cruft-size-large && @@ -173,7 +173,7 @@ test_expect_success '--max-cruft-size creates new packs when above threshold' ' ) ' -test_expect_success '--max-cruft-size combines existing packs when below threshold' ' +test_expect_success '--max-cruft-size combines existing packs when not too large' ' git init max-cruft-size-small && ( cd max-cruft-size-small && @@ -236,10 +236,10 @@ test_expect_success '--max-cruft-size combines smaller packs first' ' ) ' -test_expect_success 'setup --max-cruft-size with freshened objects' ' - git init max-cruft-size-freshen && +test_expect_success 'setup cruft with freshened objects' ' + git init cruft-freshen && ( - cd max-cruft-size-freshen && + cd cruft-freshen && test_commit base && git repack -ad && @@ -257,9 +257,9 @@ test_expect_success 'setup --max-cruft-size with freshened objects' ' ) ' -test_expect_success '--max-cruft-size with freshened objects (loose)' ' +test_expect_success 'cruft with freshened objects (loose)' ' ( - cd max-cruft-size-freshen && + cd cruft-freshen && # regenerate the object, setting its mtime to be more recent foo="$(generate_random_blob foo 64)" && @@ -275,9 +275,9 @@ test_expect_success '--max-cruft-size with freshened objects (loose)' ' ) ' -test_expect_success '--max-cruft-size with freshened objects (packed)' ' +test_expect_success 'cruft with freshened objects (packed)' ' ( - cd max-cruft-size-freshen && + cd cruft-freshen && # regenerate the object and store it in a packfile, # setting its mtime to be more recent @@ -304,7 +304,7 @@ test_expect_success '--max-cruft-size with freshened objects (packed)' ' ) ' -test_expect_success '--max-cruft-size with freshened objects (previously cruft)' ' +test_expect_success 'multi-cruft with freshened objects (previously cruft)' ' repo="max-cruft-size-threshold" && test_when_finished "rm -fr $repo" && From patchwork Wed Mar 19 22:52:51 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 14023266 Received: from mail-qt1-f169.google.com (mail-qt1-f169.google.com [209.85.160.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 51186221F16 for ; Wed, 19 Mar 2025 22:52:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.160.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742424775; cv=none; b=cJmok9pVGV/KDUQw+lMZkzXB2SlC4afQabLTqdTuY4cd9m7cm7ijnNG1ELXoK6t7FGM6YDyEuriHO9Phjis7hRb2mNa2uURpu8Mc4P2MPqdKh0HhmOp6BUuc24ZI0TZTCRZ3HuvJBUNNH3OAT5vd1KabfuNjOZhmg2ggN71EPiM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742424775; c=relaxed/simple; bh=0nCxrL/G8M/O+WhyBIKBTCT8LyXBtBEchykTSXy5Ueg=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=TQ+XKcGxliXm5DZOi9W9c1SQvmEuPjgmqtnyVLWwLBZSb818oveUvt09vmlw/QhR7mH+A8cFI1taFim6WDghpSCsqTcSkdy1EmDAaebxFhzPOgn6F37HTa2mMYbxSZTM67IhXLukw7z5KQSy+pxV0t/EV0L/nwWYXeMz4fq9O6c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com; spf=pass smtp.mailfrom=ttaylorr.com; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b=jl1iiw1L; arc=none smtp.client-ip=209.85.160.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b="jl1iiw1L" Received: by mail-qt1-f169.google.com with SMTP id d75a77b69052e-476af5479feso1969221cf.2 for ; Wed, 19 Mar 2025 15:52:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1742424773; x=1743029573; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=MIMKyCSylCTCD/pYdQEqf/gszAmYw6lsJBhriIG2fAE=; b=jl1iiw1LVAHHS/J6zYmZ3hMsSojRog7hxygo/i9QCsWHWvrryvua9yYtZfcux+DtlJ 8MBhISCoENEv+R9mR1jPkSNgghcuqGhaIrnMEtgOUg3G8BmA4zYmHFv2LjBu+M5Kex12 Nr1NcXbaGJ9mhhf5dD+mCX2w7PPOvmaWALFaUzXJ6ZeKldX92Qh75zgI9bo0YQ5Jx5oT xshR/zNSe0VwqPoucItXRC9FgC2xuKMgeCJCrbUr6NFNt6LrymEzXttThZAQcvRKWNUa GNdPOmyR36JO6FVlqYPPsJHhfnNWWclh0GPDgX059a9R3w0gZM1/AnduJ5QtvgHRYobv OAFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742424773; x=1743029573; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=MIMKyCSylCTCD/pYdQEqf/gszAmYw6lsJBhriIG2fAE=; b=Z6pIfwjioVgEchKCoTvfXl43Fmr5qhcWBCnghKFnjz5bwkGELPIvYKWy/rB50a0BLO ZpVQPcyJx2eVpP20tWrz79m1ADlfNnd5lZ97Zrt6mTQnbVR2UrBRXk0PwNPF7r9Fo54Q m5Ovvg7SYv8EykI+e5eaQFzpCzEDvD+l/6jYfP7iqltEHSPGEMpxifM1ArBAzj7pl0ZT caRhlMeY8pgjzLghAjc/gkvGeFqweJ/bHgLUopm5VpmMvZ7YxtPHtPbHEYIClN9gqAhZ meOKKs8X0rVA4oBXuo2KgduD4ER8YrSksQCz8i/o9+T/+behr+kv8mKNlLYGeR3xzSS5 2NDA== X-Gm-Message-State: AOJu0Yx28fqJnGFKjRCkKvqcNAp5aWiznwbcu+BrSQi1m2VAETD8Pxwr zrMaj1qwKnTUfgQKCXZgfpwrypfh1VuUXGTJH7EY5GREvSJL9sQFHJsrEN6TIUeExkHmjlea9BU aEFE= X-Gm-Gg: ASbGncvycSbQYW+5W+zAxEh+ZrBTRcHWoATSi1ExHuMHifJMO2B8DJb+AqjF1BnchSt lNBuNXI7yXh77YMgLvrYdL8IsyrpkQ2zIyFxN/xJis4GEhN2uDESgucAU+sZkPm/iwzQ7Wff7KF NiRoioCvfcUbgk0yLyymyiJgKa32uj398TM/AtGAqgKYUV02vW1EeYp9NGRb1OY+1HRGTrFcb1N 6TUHIZUb5tFt9umNJ5CtvzRqQZtqqJPeh8wHXL8n2nu2diYSTugvetlaLoCCVla3Bawwrt99QJO vkZvHReG3kPAeD/QHFVzSyMEv747h4XJQqEWcyWzvD6dK0Qw9ZxvwDr/aHslwBnI8hExAfpg9lR zSc/9U/CsnhLa6mek X-Google-Smtp-Source: AGHT+IFrbkypvKRsYsL3n9R6B55J1KB7q8NGMcBJPbDh6X2MmO/l2+uNekCqGl2giJNjI/gjL2Ht/A== X-Received: by 2002:a05:622a:1c0c:b0:476:9cf2:d907 with SMTP id d75a77b69052e-4770841aba3mr80664051cf.52.1742424773096; Wed, 19 Mar 2025 15:52:53 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with UTF8SMTPSA id d75a77b69052e-476bb60aac8sm84612541cf.15.2025.03.19.15.52.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Mar 2025 15:52:52 -0700 (PDT) Date: Wed, 19 Mar 2025 18:52:51 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Junio C Hamano , Elijah Newren Subject: [PATCH v2 3/5] t/t7704-repack-cruft.sh: consolidate `write_blob()` Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: A previous commit moved a handful of tests from a different script into t7704, including one that relies on generating random blobs. Incidentally, the original home of this test defined its own helper "write_blob" for doing so, which is identical in function to our "generate_random_blob" (and is slightly inferior to the latter, which cleans up after itself). Rewrite the test that uses "write_blob" to no longer do so and then remove the function. Signed-off-by: Taylor Blau --- t/t7704-repack-cruft.sh | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/t/t7704-repack-cruft.sh b/t/t7704-repack-cruft.sh index e6e4c2fad8..3fd5aa6089 100755 --- a/t/t7704-repack-cruft.sh +++ b/t/t7704-repack-cruft.sh @@ -618,11 +618,6 @@ test_expect_success 'cruft repack with no reachable objects' ' ) ' -write_blob () { - test-tool genrandom "$@" >in && - git hash-object -w -t blob in -} - find_pack () { for idx in $(ls $packdir/pack-*.idx) do @@ -641,8 +636,8 @@ test_expect_success 'cruft repack with --max-pack-size' ' test_commit base && # two cruft objects which exceed the maximum pack size - foo=$(write_blob foo 1048576) && - bar=$(write_blob bar 1048576) && + foo=$(generate_random_blob foo 1048576) && + bar=$(generate_random_blob bar 1048576) && test-tool chmtime --get -1000 \ "$objdir/$(test_oid_to_path $foo)" >foo.mtime && test-tool chmtime --get -2000 \ From patchwork Wed Mar 19 22:52:54 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 14023267 Received: from mail-qk1-f170.google.com (mail-qk1-f170.google.com [209.85.222.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5E097221F16 for ; Wed, 19 Mar 2025 22:52:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742424779; cv=none; b=a9DOH57Gz4F8OwpA1ayzgGusM1XAC4Ogs9jDVuF789kVJKEJVIEibkyVXIAWjVlxhWfBCv871+a+VSCt1jlSsCaz7RM/WSu4N5e3vWUCtU0h0Np/kBrIx/btxA4AHtGucT3UIgRtIwHXVlXQM5/ATaV672AQjdu3OKs8Y8BHy10= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742424779; c=relaxed/simple; bh=1FRAxN/JPTq0di3fx3dGlCEE8mrAN47c6Q9ippxG6x8=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=M711nLsnCpnsULQF48c4ou79//qL8PVPxpcJDUgy8fTCdqOxRCOKmyX86kDvHlzCm+bAwbe0VvrqEmA23PTjD9QMhgD7ITZgH3qqQlPrFJQxVHngioiKAj7MZGVQ6fFetFhRsv02Xm35nkSM1hKNSpfi9bUzxjHqsgdzhaAFaDc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com; spf=pass smtp.mailfrom=ttaylorr.com; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b=2WtHIB2Y; arc=none smtp.client-ip=209.85.222.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b="2WtHIB2Y" Received: by mail-qk1-f170.google.com with SMTP id af79cd13be357-7c3bf231660so29627685a.0 for ; Wed, 19 Mar 2025 15:52:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1742424776; x=1743029576; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=xynV+fPJaMlXu7n2AQVwSzn2g5j7MJeIHIFu/pT3ZX4=; b=2WtHIB2YxMOMSLAZDOlKGtP/jnWBH4XUJke1L4CvB+SnYuG664wSeBPKK8LJIMbIp3 TlKrCokYUJQYZjar2CaojsgKVMhtjn6CuPNdxbzfs9sBm5nXnRcOsRzcPk0b/2xDtWF8 aMknc+Qdxt0XUBjE8MXJPYUHi7mvBspAByDK0U+2KQmSeyCejtdDmqI17BBkzmBXDw/w D2Opf2OYS97Yk1kLR1szUo0RQNFMoWjbWW0kLlq5jT4mT2JxTkQIujouP+7mF6uNOMNF u/hO3XQE8vuKn92vYjhmAZn7CWGKFctd7QNgNWKd5/JDu0wUz7dGlKzRj12d7NOmJ6vP yjcQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742424776; x=1743029576; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=xynV+fPJaMlXu7n2AQVwSzn2g5j7MJeIHIFu/pT3ZX4=; b=PsB7p+5MRhfIVT3HTCuq9XNQtfI9cuFRKme27MJA93KfD9JDx9wXTvUb5ZYxhLO2GC RfyHMg8KTUW6FxsZTAKnYzio49F++Ht27JqV5CQjJjHj5s4bHojJUPfQXSQRCg0X16L+ Zrh3uODVEbvJbj8uBkvl5G+4zmJPXb+XalcY+JDdUutPevOUPjDZIMxB03U833JWpynT HIjauADdfJT5evWOf6supkkUsIShYpQnKhlq3ARJqAIu2DdySQqhgoupoxdJ68gVCVVp cQp+zKIinFkjYfsb3KcqnaZozeeLRGSyKIr+BsIbqamFlFlsLK/dTDkLgxXp8nw6qk3i uoPA== X-Gm-Message-State: AOJu0Yzw2x5WAR1Q4FOnqrUsdkaWBvsfmMJiGErncxqqkMXVYkeXeNKp aOHz5BTAEraAhXrbCL0O3bPiSyOMnKHFVnagumZiEFum+cX0o43SQUF0iUb/x95b/36UwFKc99d sTj8= X-Gm-Gg: ASbGnctvBOvFH6hnSFwf8TH+sAB/xw8FUuOrCshzVd+kiUkcKZ//mk8zXQ6H5dNpTdp g7Sb3MZgscbWqPxkVRrMqPDS9XK4uBXkG73qmc5kXbNRZTyneGhpkwO0Lzz6AtC9J8p9YEwPju3 ZU2QoUK3c8AA+hCRthIrDuStl8HU9UlfSN5FQr4TlizTwhJscC8TXARiGl6O0pBmojxBRevLJRv pXm8bLX8ZXtIGxd7LcTRKeNvq8bx1A4tJGznZIQ55Q44i5LvTwkibeC0c5DORoJKX0vNN8v+Ll3 IzINpQ1ZGW/sj80Hmmfmd2E6FgVQYHTdEPcRdMatuKOOJNeNI73FXOOdwRE0GblRdAmNVcGjNb8 VkDWQ9rlM2QsSB4Mh X-Google-Smtp-Source: AGHT+IHIeUlc3ZFxfK1Y7EkthqRYFE6bZxrtax5O+ZNnpLlJRSL5TUsID50LKDpqRsvMOvdqLumG1g== X-Received: by 2002:a05:620a:4593:b0:7c5:5d4b:e63c with SMTP id af79cd13be357-7c5a84ae82cmr761183585a.47.1742424776020; Wed, 19 Mar 2025 15:52:56 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with UTF8SMTPSA id af79cd13be357-7c573c4db57sm920934985a.8.2025.03.19.15.52.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Mar 2025 15:52:55 -0700 (PDT) Date: Wed, 19 Mar 2025 18:52:54 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Junio C Hamano , Elijah Newren Subject: [PATCH v2 4/5] repack: avoid combining cruft packs with `--max-cruft-size` Message-ID: <7941997e332828f0005acf97f925efc1e7251a33.1742424671.git.me@ttaylorr.com> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: In 37dc6d8104 (builtin/repack.c: implement support for `--max-cruft-size`, 2023-10-02), we exposed new functionality that allowed repositories to specify the behavior of when we should combine multiple cruft packs together. This feature was designed to ensure that we never repacked cruft packs which were larger than the given threshold in order to provide tighter I/O bounds for repositories that have many unreachable objects. In essence, specifying '--max-cruft-size=N' instructed 'repack' to aggregate cruft packs together (in order of ascending size) until the combine size grows past 'N', and then make a new cruft pack whose contents includes the packs we rolled up. But this isn't quite how it works in practice. Suppose for example that we have two cruft packs which are each 100MiB in size. One might expect specifying "--max-cruft-size=200M" would combine these two packs together, and then avoid repacking them until a pruning GC takes place. In reality, 'repack' would try and aggregate these together, but writing a pack that is strictly smaller than 200 MiB (since pack-objects' "--max-pack-size" provides a strict bound for packs containing more than one object). So instead we'll write out a pack that is, say, 199 MiB in size, and then another 1 MiB pack containing the balance. If we later repack the repository without adding any new unreachable objects, we'll repeat the same exercise again, making the same 199 MiB and 1 MiB packs each time. This happens because of a poor choice to bolt the '--max-cruft-size' functionality onto pack-objects' '--max-pack-size', forcing us to generate packs which are always smaller than the provided threshold and thus subject to repacking. The following commit will introduce a new flag that implements something similar to the behavior above. Let's prepare for that by making repack's '--max-cruft-size' flag behave as an cruft pack-specific override for '--max-pack-size'. Do so by temporarily repurposing the 'collapse_small_cruft_packs()' function to instead generate a cruft pack using the same instructions as if we didn't specify any maximum pack size. The calling code looks something like: if (args->max_pack_size && !cruft_expiration) { collapse_small_cruft_packs(in, args->max_pack_size, existing); } else { for_each_string_list_item(item, &existing->non_kept_packs) fprintf(in, "-%s.pack\n", item->string); for_each_string_list_item(item, &existing->cruft_packs) fprintf(in, "-%s.pack\n", item->string); } This patch makes collapse_small_cruft_packs() behave identically to the 'else' arm of the conditional above. This repurposing of 'collapse_small_cruft_packs()' is intentional, since it will set us up nicely to introduce the new behavior in the following commit. Naturally, there is some test fallout in the test which exercises the old meaning of '--max-cruft-size'. Mark that test as failing for now to be dealt with in the following commit. Likewise, add a new test which explicitly tests the behavior of '--max-cruft-size' to place a hard limit on the size of any generated cruft pack(s). Note that this is a breaking change, as it alters the user-visible behavior of '--max-cruft-size'. But I'm OK changing this behavior in this instance, since the behavior wasn't accurate to begin with. Signed-off-by: Taylor Blau --- Documentation/git-repack.adoc | 12 ++------ builtin/repack.c | 50 ++++++--------------------------- t/t5329-pack-objects-cruft.sh | 52 +++++++++++++++++++++++++++++++++++ t/t7704-repack-cruft.sh | 8 ++---- 4 files changed, 67 insertions(+), 55 deletions(-) diff --git a/Documentation/git-repack.adoc b/Documentation/git-repack.adoc index 5852a5c973..11db43b1c5 100644 --- a/Documentation/git-repack.adoc +++ b/Documentation/git-repack.adoc @@ -77,15 +77,9 @@ to the new separate pack will be written. Only useful with `--cruft -d`. --max-cruft-size=:: - Repack cruft objects into packs as large as `` bytes before - creating new packs. As long as there are enough cruft packs - smaller than ``, repacking will cause a new cruft pack to - be created containing objects from any combined cruft packs, - along with any new unreachable objects. Cruft packs larger than - `` will not be modified. When the new cruft pack is larger - than `` bytes, it will be split into multiple packs, all of - which are guaranteed to be at most `` bytes in size. Only - useful with `--cruft -d`. + Overrides `--max-pack-size` for cruft packs. Inherits the value of + `--max-pack-size` (if any) by default. See the documentation for + `--max-pack-size` for more details. --expire-to=:: Write a cruft pack containing pruned objects (if any) to the diff --git a/builtin/repack.c b/builtin/repack.c index 75e3752353..9658f6b354 100644 --- a/builtin/repack.c +++ b/builtin/repack.c @@ -1022,28 +1022,19 @@ static int write_filtered_pack(const struct pack_objects_args *args, return finish_pack_objects_cmd(&cmd, names, local); } -static int existing_cruft_pack_cmp(const void *va, const void *vb) -{ - struct packed_git *a = *(struct packed_git **)va; - struct packed_git *b = *(struct packed_git **)vb; - - if (a->pack_size < b->pack_size) - return -1; - if (a->pack_size > b->pack_size) - return 1; - return 0; -} - -static void collapse_small_cruft_packs(FILE *in, size_t max_size, +static void collapse_small_cruft_packs(FILE *in, size_t max_size UNUSED, struct existing_packs *existing) { - struct packed_git **existing_cruft, *p; + struct packed_git *p; struct strbuf buf = STRBUF_INIT; - size_t total_size = 0; - size_t existing_cruft_nr = 0; size_t i; - ALLOC_ARRAY(existing_cruft, existing->cruft_packs.nr); + /* + * Squelch a -Wunused-function warning while we rationalize + * the behavior of --max-cruft-size. This function will become + * used again in a future commit. + */ + (void)retain_cruft_pack; for (p = get_all_packs(the_repository); p; p = p->next) { if (!(p->is_cruft && p->pack_local)) @@ -1056,29 +1047,7 @@ static void collapse_small_cruft_packs(FILE *in, size_t max_size, if (!string_list_has_string(&existing->cruft_packs, buf.buf)) continue; - if (existing_cruft_nr >= existing->cruft_packs.nr) - BUG("too many cruft packs (found %"PRIuMAX", but knew " - "of %"PRIuMAX")", - (uintmax_t)existing_cruft_nr + 1, - (uintmax_t)existing->cruft_packs.nr); - existing_cruft[existing_cruft_nr++] = p; - } - - QSORT(existing_cruft, existing_cruft_nr, existing_cruft_pack_cmp); - - for (i = 0; i < existing_cruft_nr; i++) { - size_t proposed; - - p = existing_cruft[i]; - proposed = st_add(total_size, p->pack_size); - - if (proposed <= max_size) { - total_size = proposed; - fprintf(in, "-%s\n", pack_basename(p)); - } else { - retain_cruft_pack(existing, p); - fprintf(in, "%s\n", pack_basename(p)); - } + fprintf(in, "-%s.pack\n", buf.buf); } for (i = 0; i < existing->non_kept_packs.nr; i++) @@ -1086,7 +1055,6 @@ static void collapse_small_cruft_packs(FILE *in, size_t max_size, existing->non_kept_packs.items[i].string); strbuf_release(&buf); - free(existing_cruft); } static int write_cruft_pack(const struct pack_objects_args *args, diff --git a/t/t5329-pack-objects-cruft.sh b/t/t5329-pack-objects-cruft.sh index 60dac8312d..25ddda5cf3 100755 --- a/t/t5329-pack-objects-cruft.sh +++ b/t/t5329-pack-objects-cruft.sh @@ -695,4 +695,56 @@ test_expect_success 'additional cruft blobs via gc.recentObjectsHook' ' ) ' +test_expect_success 'split cruft packs with --max-cruft-size' ' + repo=cruft-with--max-cruft-size && + test_when_finished "rm -fr $repo" && + + git init "$repo" && + + ( + cd "$repo" && + + git config core.compression 0 && + + sz=$((1024 * 1024)) && # 1MiB + test-tool genrandom foo $sz >foo && + test-tool genrandom bar $sz >bar && + foo="$(git hash-object -w -t blob foo)" && + bar="$(git hash-object -w -t blob bar)" && + + to=$packdir/pack && + # Pack together foo and bar into a single 2MiB pack. + pack="$(git pack-objects $to <<-EOF + $foo + $bar + EOF + )" && + + # Then generate a cruft pack containing foo and bar. + # + # Generate the pack with --max-pack-size equal to the + # size of one object, forcing us to write two cruft + # packs. + git pack-objects --cruft --max-pack-size=$sz $to <<-EOF && + -pack-$pack.pack + EOF + + ls $packdir/pack-*.mtimes >crufts && + test_line_count = 2 crufts && + + for cruft in $(cat crufts) + do + test-tool pack-mtimes "$(basename "$cruft")" || return 1 + done >actual.raw && + + cut -d" " -f1 actual && + sort >expect <<-EOF && + $foo + $bar + EOF + + test_cmp expect actual + ) +' + test_done diff --git a/t/t7704-repack-cruft.sh b/t/t7704-repack-cruft.sh index 3fd5aa6089..6debad368d 100755 --- a/t/t7704-repack-cruft.sh +++ b/t/t7704-repack-cruft.sh @@ -194,7 +194,7 @@ test_expect_success '--max-cruft-size combines existing packs when not too large ) ' -test_expect_success '--max-cruft-size combines smaller packs first' ' +test_expect_failure '--max-cruft-size combines smaller packs first' ' git init max-cruft-size-consume-small && ( cd max-cruft-size-consume-small && @@ -354,13 +354,11 @@ test_expect_success 'multi-cruft with freshened objects (previously cruft)' ' done >actual.raw && sort actual.raw >actual && - # Among the set of all cruft packs, we should see both - # mtimes for object $foo and $bar, as well as the + # Among the set of all cruft packs, we should see the + # new mtimes for object $foo and $bar, as well as the # single new copy of $baz. sort >expect <<-EOF && - $foo $(cat foo.old) $foo $(cat foo.new) - $bar $(cat bar.old) $bar $(cat bar.new) $baz $(cat baz.old) $quux $(cat quux.new) From patchwork Wed Mar 19 22:52:58 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 14023268 Received: from mail-qk1-f175.google.com (mail-qk1-f175.google.com [209.85.222.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 96008221DBA for ; Wed, 19 Mar 2025 22:53:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742424782; cv=none; b=F6G3c80hmg9pUK/7YXHgQUo1i8SaaaSNilO2Qq833XypHJkA8WW1qoUggVRH31frMt3Ps31JzKuzSsIwpILqOXn2ZMEJACZNIY3X4/XMawI/fE/I1DcseQt1dUVzolxmdgYDs6BmYjvhwuMQ57QTlTCGmoh61K4vN1TJFq6Tv9E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742424782; c=relaxed/simple; bh=oncs16QJNVjxqFC8yqb1LGD57A6EfAaK+dw0P5bqdiw=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=onw3loj45EDpc89dKM8KHmhoUqDTSvsNDb335kY4dTu0GZUvmVQlERSm9u7/1oLEpfni/bzGcHA3B4jX/qn4txpVuvsuhSdBBTIgXgxjHkmwhMTSsjy9SjccUhLoHQO2W/8XvO0Q2FPDYJ36eEoHGhAbPY5fFT+uJReeYpdLmP8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com; spf=pass smtp.mailfrom=ttaylorr.com; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b=wxNRfJsu; arc=none smtp.client-ip=209.85.222.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b="wxNRfJsu" Received: by mail-qk1-f175.google.com with SMTP id af79cd13be357-7c08fc20194so40960685a.2 for ; Wed, 19 Mar 2025 15:53:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1742424779; x=1743029579; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=0SxAoQyp+NsqsirMIQg/eDXxSEKM4dYidALpDkFfYwA=; b=wxNRfJsu67DD7GOCzyUgKcn+dAVoQaJkVzTQ9kKPQJpD70zV9+ECps0oGPTqGo70fU N7ALuesPhfubT4jVJPRPAG+5LTU5UqAShDxEPV5GiZ+Az35OaQIJaMmaHNVvER7DxMvI lm8iv/1o/ZcM6ki7jiH11qipKbpxg4WygeVMOIc44K5D7PkNEozNAUNLmg9T9vC5AXD8 D0OU8yqG9b93A0h/g1E0UM0RsOMz1ZVhdD3MfpxYl9AAQsyYrEHVxHlEGFTh2KUiB2na PSEtyQ5D5KwywViLEuj4Na501C9ol0jnAAgYF/6UdYDgYzKVq1BqbGPuJXM9D2xUoo60 2FLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742424779; x=1743029579; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=0SxAoQyp+NsqsirMIQg/eDXxSEKM4dYidALpDkFfYwA=; b=gmFHEgPrf4YcadDo035uMW2SxjT1mrX7Swocs8hg/8+/Fmi3yPGHbuXnkA69kQSjOC gw1ZyImCA9JkMgGQnR0NSKq2xvejnolW1ZjBCMgb3zPrbml1POFXCpEHj1eKCBK3iQvH ltLpuk1s9w+f1KT+BR+e8xiYC3TDFUvjO0B/R+SFd+Y+DXQqK5XrX69oLC1DmEJMBc8i 7uoychmjSxMUWZFIF+VGSxrq+c79CZUWqf/rqaVc75k4/UenY5ECPl9ZWJROz//gckC0 sxQMVB7vpsID5VkwfT5tU9UIuHZ5q9J48x32qVjpEMuV8KvMFXsmO/w4cstwSIGSXTs2 /1RQ== X-Gm-Message-State: AOJu0Yyz5k8mylCCDHIdEM+FhphqPKAPzMsSbWoSPfJHe+ZLAamjKgM/ /lDjTae9q8VlLrussaZ7Ezx6Gpd7Yie/0I4zeocm0QLiW3Bzvg2NdlJ/8xbriwpWaTmPRS/htg6 ATuo= X-Gm-Gg: ASbGncv7vDhCYU8Y2QwoJGmvommFwTsA522qxHIJ53WbczRlxi1Af2GPlaa2710MtLb /KrJ4X6PEP638WAsZV2fiuyU8H6OSlQA0TnuEvEiF6z/blsqYIbwKZVbyy65g1G8j08S6Y0XVQx FckzHSP9+u0dd61mcCm6ygrJNPwumckgg0oOnmfLsqKifmFG0Qz1OPiVKFScdIIWjbPRM3KT/y8 YRg2gvpvI40y3MjGoTFU7cBeB20cmbZskA7HnhK28nzNCGKASD5HUiDLfr8CXip5LHifGsLS/QM RbxsYZSZxSg0J161JLpP+UKu7Fcp7HAkE3fkMowkG5YSrVcDpkz3QoEz41jwbKHXzljTsxC2oZM vSQdeoO/50aVExjVO X-Google-Smtp-Source: AGHT+IGz60a32bMTk67nOJ/J/TXpPpKnO8C7Wt/w9EOl8rgwn1rA4o0kvVOVI6OyqNi7+Ag8LdYB2w== X-Received: by 2002:a05:620a:2616:b0:7c5:5cd6:5cea with SMTP id af79cd13be357-7c5a83969aamr672406985a.15.1742424779373; Wed, 19 Mar 2025 15:52:59 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with UTF8SMTPSA id af79cd13be357-7c573da455dsm911621985a.115.2025.03.19.15.52.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Mar 2025 15:52:59 -0700 (PDT) Date: Wed, 19 Mar 2025 18:52:58 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Junio C Hamano , Elijah Newren Subject: [PATCH v2 5/5] repack: begin combining cruft packs with `--combine-cruft-below-size` Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: The previous commit changed the behavior of repack's '--max-cruft-size' to specify a cruft pack-specific override for '--max-pack-size'. Introduce a new flag, '--combine-cruft-below-size' which is a replacement for the old behavior of '--max-cruft-size'. This new flag does explicitly what it says: it combines together cruft packs which are smaller than a given threshold, and leaves alone ones which are larger. This accomplishes the original intent of '--max-cruft-size', which was to avoid repacking cruft packs larger than the given threshold. The new behavior is slightly different. Instead of building up small packs together until the threshold is met, '--combine-cruft-below-size' packs up *all* cruft packs smaller than the threshold. This means that we may make a pack much larger than the given threshold (e.g., if you aggregate 5 packs which are each 99 MiB in size with a threshold of 100 MiB). But that's OK: the point isn't to restrict the size of the cruft packs we generate, it's to avoid working with ones that have already grown too large. If repositories still want to limit the size of the generated cruft pack(s), they may use '--max-cruft-size'. There's some minor test fallout as a result of the slight differences in behavior between the old meaning of '--max-cruft-size' and the behavior of '--combine-cruft-below-size'. In the test which is now called "--combine-cruft-below-size combines packs", we need to use the new flag over the old one to exercise that test's intended behavior. The remainder of the changes there are to improve the clarity of the comments. Suggested-by: Elijah Newren Signed-off-by: Taylor Blau --- Documentation/git-repack.adoc | 9 +++++++++ builtin/repack.c | 38 +++++++++++++++++++++++------------ t/t7704-repack-cruft.sh | 22 +++++++++++--------- 3 files changed, 47 insertions(+), 22 deletions(-) diff --git a/Documentation/git-repack.adoc b/Documentation/git-repack.adoc index 11db43b1c5..e1cd75eebe 100644 --- a/Documentation/git-repack.adoc +++ b/Documentation/git-repack.adoc @@ -81,6 +81,15 @@ to the new separate pack will be written. `--max-pack-size` (if any) by default. See the documentation for `--max-pack-size` for more details. +--combine-cruft-below-size=:: + When generating cruft packs without pruning, only repack + existing cruft packs whose size is strictly less than ``, + where `` represents a number of bytes, which can optionally + be suffixed with "k", "m", or "g". Cruft packs whose size is + greater than or equal to `` are left as-is and not repacked. + Useful when you want to avoid repacking large cruft pack(s) in + repositories that have many and/or large unreachable objects. + --expire-to=:: Write a cruft pack containing pruned objects (if any) to the directory ``. This option is useful for keeping a copy of diff --git a/builtin/repack.c b/builtin/repack.c index 9658f6b354..f3330ade7b 100644 --- a/builtin/repack.c +++ b/builtin/repack.c @@ -1022,20 +1022,13 @@ static int write_filtered_pack(const struct pack_objects_args *args, return finish_pack_objects_cmd(&cmd, names, local); } -static void collapse_small_cruft_packs(FILE *in, size_t max_size UNUSED, - struct existing_packs *existing) +static void combine_small_cruft_packs(FILE *in, size_t combine_cruft_below_size, + struct existing_packs *existing) { struct packed_git *p; struct strbuf buf = STRBUF_INIT; size_t i; - /* - * Squelch a -Wunused-function warning while we rationalize - * the behavior of --max-cruft-size. This function will become - * used again in a future commit. - */ - (void)retain_cruft_pack; - for (p = get_all_packs(the_repository); p; p = p->next) { if (!(p->is_cruft && p->pack_local)) continue; @@ -1047,7 +1040,12 @@ static void collapse_small_cruft_packs(FILE *in, size_t max_size UNUSED, if (!string_list_has_string(&existing->cruft_packs, buf.buf)) continue; - fprintf(in, "-%s.pack\n", buf.buf); + if (p->pack_size < combine_cruft_below_size) { + fprintf(in, "-%s\n", pack_basename(p)); + } else { + retain_cruft_pack(existing, p); + fprintf(in, "%s\n", pack_basename(p)); + } } for (i = 0; i < existing->non_kept_packs.nr; i++) @@ -1061,6 +1059,7 @@ static int write_cruft_pack(const struct pack_objects_args *args, const char *destination, const char *pack_prefix, const char *cruft_expiration, + unsigned long combine_cruft_below_size, struct string_list *names, struct existing_packs *existing) { @@ -1103,8 +1102,9 @@ static int write_cruft_pack(const struct pack_objects_args *args, in = xfdopen(cmd.in, "w"); for_each_string_list_item(item, names) fprintf(in, "%s-%s.pack\n", pack_prefix, item->string); - if (args->max_pack_size && !cruft_expiration) { - collapse_small_cruft_packs(in, args->max_pack_size, existing); + if (combine_cruft_below_size && !cruft_expiration) { + combine_small_cruft_packs(in, combine_cruft_below_size, + existing); } else { for_each_string_list_item(item, &existing->non_kept_packs) fprintf(in, "-%s.pack\n", item->string); @@ -1158,6 +1158,7 @@ int cmd_repack(int argc, const char *opt_window_memory = NULL; const char *opt_depth = NULL; const char *opt_threads = NULL; + unsigned long combine_cruft_below_size = 0ul; struct option builtin_repack_options[] = { OPT_BIT('a', NULL, &pack_everything, @@ -1170,6 +1171,9 @@ int cmd_repack(int argc, PACK_CRUFT), OPT_STRING(0, "cruft-expiration", &cruft_expiration, N_("approxidate"), N_("with --cruft, expire objects older than this")), + OPT_MAGNITUDE(0, "combine-cruft-below-size", + &combine_cruft_below_size, + N_("with --cruft, only repack cruft packs smaller than this")), OPT_MAGNITUDE(0, "max-cruft-size", &cruft_po_args.max_pack_size, N_("with --cruft, limit the size of new cruft packs")), OPT_BOOL('d', NULL, &delete_redundant, @@ -1413,7 +1417,8 @@ int cmd_repack(int argc, cruft_po_args.quiet = po_args.quiet; ret = write_cruft_pack(&cruft_po_args, packtmp, pack_prefix, - cruft_expiration, &names, + cruft_expiration, + combine_cruft_below_size, &names, &existing); if (ret) goto cleanup; @@ -1440,10 +1445,17 @@ int cmd_repack(int argc, * generate an empty pack (since every object not in the * cruft pack generated above will have an mtime older * than the expiration). + * + * Pretend we don't have a `--combine-cruft-below-size` + * argument, since we're not selectively combining + * anything based on size to generate the limbo cruft + * pack, but rather removing all cruft packs from the + * main repository regardless of size. */ ret = write_cruft_pack(&cruft_po_args, expire_to, pack_prefix, NULL, + 0ul, &names, &existing); if (ret) diff --git a/t/t7704-repack-cruft.sh b/t/t7704-repack-cruft.sh index 6debad368d..8aebfb45f5 100755 --- a/t/t7704-repack-cruft.sh +++ b/t/t7704-repack-cruft.sh @@ -194,10 +194,13 @@ test_expect_success '--max-cruft-size combines existing packs when not too large ) ' -test_expect_failure '--max-cruft-size combines smaller packs first' ' - git init max-cruft-size-consume-small && +test_expect_success '--combine-cruft-below-size combines packs' ' + repo=combine-cruft-below-size && + test_when_finished "rm -fr $repo" && + + git init "$repo" && ( - cd max-cruft-size-consume-small && + cd "$repo" && test_commit base && git repack -ad && @@ -211,11 +214,11 @@ test_expect_failure '--max-cruft-size combines smaller packs first' ' test-tool pack-mtimes "$(basename $cruft_bar)" >>expect.raw && sort expect.raw >expect.objects && - # repacking with `--max-cruft-size=2M` should combine - # both 0.5 MiB packs together, instead of, say, one of - # the 0.5 MiB packs with the 1.0 MiB pack + # Repacking with `--combine-cruft-below-size=1M` + # should combine both 0.5 MiB packs together, but + # ignore the two packs which are >= 1.0 MiB. ls $packdir/pack-*.mtimes | sort >cruft.before && - git repack -d --cruft --max-cruft-size=2M && + git repack -d --cruft --combine-cruft-below-size=1M && ls $packdir/pack-*.mtimes | sort >cruft.after && comm -13 cruft.before cruft.after >cruft.new && @@ -224,11 +227,12 @@ test_expect_failure '--max-cruft-size combines smaller packs first' ' test_line_count = 1 cruft.new && test_line_count = 2 cruft.removed && - # the two smaller packs should be rolled up first + # The two packs smaller than 1.0MiB should be repacked + # together. printf "%s\n" $cruft_foo $cruft_bar | sort >expect.removed && test_cmp expect.removed cruft.removed && - # ...and contain the set of objects rolled up + # ...and contain the set of objects rolled up. test-tool pack-mtimes "$(basename $(cat cruft.new))" >actual.raw && sort actual.raw >actual.objects &&