From patchwork Mon Aug 28 22:49:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13368391 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9603BC71153 for ; Mon, 28 Aug 2023 22:49:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234257AbjH1WtX (ORCPT ); Mon, 28 Aug 2023 18:49:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41876 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234262AbjH1WtJ (ORCPT ); Mon, 28 Aug 2023 18:49:09 -0400 Received: from mail-yw1-x1134.google.com (mail-yw1-x1134.google.com [IPv6:2607:f8b0:4864:20::1134]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4103B11A for ; Mon, 28 Aug 2023 15:49:07 -0700 (PDT) Received: by mail-yw1-x1134.google.com with SMTP id 00721157ae682-591ba8bd094so44269427b3.3 for ; Mon, 28 Aug 2023 15:49:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1693262945; x=1693867745; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=uSdw4I7fJJKJgYsOxrr5RSYCOCmBu47Cyk4PipGiC34=; b=mid77yEexKir27PkJZJV+4qYm8yxJoYUf+Zf5XtVRjWjUPFPXK/S9Bns0QMaVGWG5U ebLS5XoLNiUk9pGIpc0/UbviPKBZ9Phny1J1Gx8OFQGoO0k6IM6nRbhMqYC0rDflmULe NpKXjtykz/wKn25GmEsHIHo5XazPIJWs3rrou6/GR1T4/7/BPbpZgoMA3m3w5u/urtNd e8D50lQzDxgsB+HoE/ACXe1a31tBzPqWnJ+dNfAUwgV/rZVSPl7N5ZGtMZ8ufBXTBHKP nIpinhZAQ1shRH08qiw3MYzpA6hvvrzxopi87k3O3MobVwkamup5rduLUfN902I0P5fo 6gwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693262945; x=1693867745; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=uSdw4I7fJJKJgYsOxrr5RSYCOCmBu47Cyk4PipGiC34=; b=VYIWGOHkuQdSbOAEQEEyRvuEUrL9bODQk9OKw8d+ak1ORFVCnujIL1eti79IKz/TCc Y7BdqNj0f5QNn0s9a0RwlMSTdNHyE909O5MvgFDryd+SvpOcOqHjML4InQolwlm1+l9P pX4b14S9KaXJzd2995876Y/ZCvrc3zaAf36+Ejw48mssLdmVdLMy9IpBQ9qB6JIyD8yL FHqXTN1urDvpr6Lc2ehFvEFSOBs4kDq6uR1JxJmLic3BdM4dH0x8lFXr3M+TqUgaKrcQ hniNi80s20ZlsQVvltBjw13Gmu+yzn6ekia0n3MSatJ134K3m/xn5UVdJvY4jS4d2oxt ucIw== X-Gm-Message-State: AOJu0Yz4+f5WDzOJY702n0JTI2QEjFivm8dqhLLrAySqy/u/GQzmGOzo S4iw+9cm9QMhwmapzB43WVyOYiHqkyph/+COPjmjIQ== X-Google-Smtp-Source: AGHT+IGUP2yJMuJtSzw9VAtZ0xgL56nl2Ayy1wEELm6YGZ5hL9lk2FYM3CNjKskPYka4aCtZlKfpYg== X-Received: by 2002:a0d:eb10:0:b0:586:c0bc:77a2 with SMTP id u16-20020a0deb10000000b00586c0bc77a2mr28500663ywe.0.1693262945695; Mon, 28 Aug 2023 15:49:05 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id j125-20020a0df983000000b005772abf6234sm2430356ywf.11.2023.08.28.15.49.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Aug 2023 15:49:05 -0700 (PDT) Date: Mon, 28 Aug 2023 18:49:04 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King Subject: [PATCH 1/4] builtin/pack-objects.c: remove unnecessary strbuf_reset() Message-ID: <921a79c8bd6ab688fb9f403a59eeaed3176b630e.1693262936.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When reading input with the `--cruft` option, `git pack-objects` reads each line into a strbuf, and then moves it to either the list of discarded or fresh packs, depending on whether or not the input line starts with a '-' character. At the beginning of each loop iteration, the next line of input is read with `strbuf_getline()`, which calls `strbuf_reset()` (as a part of `strbuf_getwholeline()`) before reading the next line of input. Thus, the call to `strbuf_reset()` (added back in b757353676 (builtin/pack-objects.c: --cruft without expiration, 2022-05-20)) at the end of the loop is unnecessary, so let's remove it accordingly. Signed-off-by: Taylor Blau --- builtin/pack-objects.c | 1 - 1 file changed, 1 deletion(-) diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c index d2a162d528..868efe7e0f 100644 --- a/builtin/pack-objects.c +++ b/builtin/pack-objects.c @@ -3603,7 +3603,6 @@ static void read_cruft_objects(void) string_list_append(&discard_packs, buf.buf + 1); else string_list_append(&fresh_packs, buf.buf); - strbuf_reset(&buf); } string_list_sort(&discard_packs); From patchwork Mon Aug 28 22:49:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13368388 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 84ECBC83F14 for ; Mon, 28 Aug 2023 22:49:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234262AbjH1WtY (ORCPT ); Mon, 28 Aug 2023 18:49:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41400 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234264AbjH1WtM (ORCPT ); Mon, 28 Aug 2023 18:49:12 -0400 Received: from mail-oo1-xc29.google.com (mail-oo1-xc29.google.com [IPv6:2607:f8b0:4864:20::c29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 724A511A for ; Mon, 28 Aug 2023 15:49:09 -0700 (PDT) Received: by mail-oo1-xc29.google.com with SMTP id 006d021491bc7-5735282d713so1957806eaf.2 for ; Mon, 28 Aug 2023 15:49:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1693262948; x=1693867748; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=olN3KiS+MoHMfO0pFBF5q9zPvvk/HyiT/qH9c9t+GTQ=; b=wPOaju1Uf0gqmTdytFZ9FKiLxXaUz5dtJvL8OBeFZZyQhiIWTRgEmYrWvWuzrZnSyg 35m3d4muivjkqEznf1h8NBSh0W7AzXFuUhPtLZ1WNb7F87DMYxvkvrvlLyO9+Mv9NGSJ OurwRxkHPlMeUrj/vpd1gET9k0mDUPri2NrD64Cqclh5/44LnyLkAIjsiPzzumwmZk+Z cw3wDV5urJVgHKIFQp4ZFK6XVlTxnnyeir/l1Oq8VhyXOcTOsfyDLfwXUgmNNkLagv/v m07WhYqqi77mDWaajUVkBLYr+NDiQ0gX2VEBonm0cAvxa5L+ZgV6cYuifhspFltVLpiO oryw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693262948; x=1693867748; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=olN3KiS+MoHMfO0pFBF5q9zPvvk/HyiT/qH9c9t+GTQ=; b=Ac8tjadaqWbb8Fr57X3c7OkFNkJe3QuTyi+DfsIkM9QsK3yB2QA90TvmCX4FM4fk1e Q/fzMItqu7nbrFW+11zJPZKpK/NeQ0RC5eNWfaEV43ndCGDicFOHYKy3oTZZE7owWH+q /aQbxzk3NB1BjybZNqg9qBFMtqaC0PNaCtIIOXvLt2bZb5Z+vH1jYhlRsbI/uSeaBuwp SkKGAvWw8V5uMRCJNvfevSl5rwbUhDkwLCvHsyFq8gWgjIdukEDFf9AY9efolRDeyhdP hCUx8A7CtYKS8BfpaVnDwS+TKtHVuqiFSgBRSYNv3hNU73Y4r6PUNYnJjs8YDuQXTetR OV3g== X-Gm-Message-State: AOJu0YzYwMGxHoMth8ps2eeAXk7G747BAZqY71mUDLHFzI9HDLavuCBV 2I92Z4snGpNTLhPdxsiO/syg5AHV6WEkTHzQ2jbYgQ== X-Google-Smtp-Source: AGHT+IGEvkTYvjAzgSoQ9Z8MrqHBPQsIJXa7VnUOf4BRLbPlR527zgpTxwKc9jCYDHq+Am7Y0d7/Tw== X-Received: by 2002:a05:6358:787:b0:13e:b477:8015 with SMTP id n7-20020a056358078700b0013eb4778015mr299226rwj.9.1693262948408; Mon, 28 Aug 2023 15:49:08 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id x204-20020a0dd5d5000000b005897fd75c80sm2429073ywd.78.2023.08.28.15.49.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Aug 2023 15:49:08 -0700 (PDT) Date: Mon, 28 Aug 2023 18:49:07 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King Subject: [PATCH 2/4] builtin/pack-objects.c: support `--max-pack-size` with `--cruft` Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When pack-objects learned the `--cruft` option back in b757353676 (builtin/pack-objects.c: --cruft without expiration, 2022-05-20), we explicitly forbade `--cruft` with `--max-pack-size`. At the time, there was no specific rationale given in the patch for not supporting the `--max-pack-size` option with `--cruft`. (As best I can remember, it's because we were trying to push users towards only ever having a single cruft pack, but I cannot be sure). However, `--max-pack-size` is flexible enough that it already works with `--cruft` and can shard unreachable objects across multiple cruft packs, creating separate ".mtimes" files as appropriate. In fact, the `--max-pack-size` option worked with `--cruft` as far back as b757353676! This is because we overwrite the `written_list`, and pass down the appropriate length, i.e. the number of objects written in each pack shard. Signed-off-by: Taylor Blau --- Documentation/git-pack-objects.txt | 4 +-- builtin/pack-objects.c | 7 ++-- builtin/repack.c | 3 +- t/t5329-pack-objects-cruft.sh | 54 ++++++++++++++++++++++++------ 4 files changed, 49 insertions(+), 19 deletions(-) diff --git a/Documentation/git-pack-objects.txt b/Documentation/git-pack-objects.txt index a9995a932c..dea7eacb0f 100644 --- a/Documentation/git-pack-objects.txt +++ b/Documentation/git-pack-objects.txt @@ -116,9 +116,7 @@ unreachable object whose mtime is newer than the `--cruft-expiration`). + Incompatible with `--unpack-unreachable`, `--keep-unreachable`, `--pack-loose-unreachable`, `--stdin-packs`, as well as any other -options which imply `--revs`. Also incompatible with `--max-pack-size`; -when this option is set, the maximum pack size is not inferred from -`pack.packSizeLimit`. +options which imply `--revs`. --cruft-expiration=:: If specified, objects are eliminated from the cruft pack if they diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c index 868efe7e0f..a046994a43 100644 --- a/builtin/pack-objects.c +++ b/builtin/pack-objects.c @@ -1190,8 +1190,7 @@ static void write_pack_file(void) unsigned char hash[GIT_MAX_RAWSZ]; char *pack_tmp_name = NULL; - if (pack_to_stdout) - f = hashfd_throughput(1, "", progress_state); + if (pack_to_stdout) f = hashfd_throughput(1, "", progress_state); else f = create_tmp_packfile(&pack_tmp_name); @@ -4382,7 +4381,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix) if (!HAVE_THREADS && delta_search_threads != 1) warning(_("no threads support, ignoring --threads")); - if (!pack_to_stdout && !pack_size_limit && !cruft) + if (!pack_to_stdout && !pack_size_limit) pack_size_limit = pack_size_limit_cfg; if (pack_to_stdout && pack_size_limit) die(_("--max-pack-size cannot be used to build a pack for transfer")); @@ -4414,8 +4413,6 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix) die(_("cannot use internal rev list with --cruft")); if (stdin_packs) die(_("cannot use --stdin-packs with --cruft")); - if (pack_size_limit) - die(_("cannot use --max-pack-size with --cruft")); } /* diff --git a/builtin/repack.c b/builtin/repack.c index 2b43a5be08..6943c5ba11 100644 --- a/builtin/repack.c +++ b/builtin/repack.c @@ -720,7 +720,6 @@ static int write_cruft_pack(const struct pack_objects_args *args, strvec_push(&cmd.args, "--honor-pack-keep"); strvec_push(&cmd.args, "--non-empty"); - strvec_push(&cmd.args, "--max-pack-size=0"); cmd.in = -1; @@ -1048,6 +1047,8 @@ int cmd_repack(int argc, const char **argv, const char *prefix) cruft_po_args.depth = po_args.depth; if (!cruft_po_args.threads) cruft_po_args.threads = po_args.threads; + if (!cruft_po_args.max_pack_size) + cruft_po_args.max_pack_size = po_args.max_pack_size; cruft_po_args.local = po_args.local; cruft_po_args.quiet = po_args.quiet; diff --git a/t/t5329-pack-objects-cruft.sh b/t/t5329-pack-objects-cruft.sh index 45667d4999..fc5fedbe9b 100755 --- a/t/t5329-pack-objects-cruft.sh +++ b/t/t5329-pack-objects-cruft.sh @@ -573,23 +573,54 @@ test_expect_success 'cruft repack with no reachable objects' ' ) ' -test_expect_success 'cruft repack ignores --max-pack-size' ' +write_blob () { + test-tool genrandom "$@" >in && + git hash-object -w -t blob in +} + +find_pack () { + for idx in $(ls $packdir/pack-*.idx) + do + git show-index <$idx >out && + if grep -q "$1" out + then + echo $idx + fi || return 1 + done +} + +test_expect_success 'cruft repack with --max-pack-size' ' git init max-pack-size && ( cd max-pack-size && test_commit base && + # two cruft objects which exceed the maximum pack size - test-tool genrandom foo 1048576 | git hash-object --stdin -w && - test-tool genrandom bar 1048576 | git hash-object --stdin -w && + foo=$(write_blob foo 1048576) && + bar=$(write_blob bar 1048576) && + test-tool chmtime --get -1000 \ + "$objdir/$(test_oid_to_path $foo)" >foo.mtime && + test-tool chmtime --get -2000 \ + "$objdir/$(test_oid_to_path $bar)" >bar.mtime && git repack --cruft --max-pack-size=1M && find $packdir -name "*.mtimes" >cruft && - test_line_count = 1 cruft && - test-tool pack-mtimes "$(basename "$(cat cruft)")" >objects && - test_line_count = 2 objects + test_line_count = 2 cruft && + + foo_mtimes="$(basename $(find_pack $foo) .idx).mtimes" && + bar_mtimes="$(basename $(find_pack $bar) .idx).mtimes" && + test-tool pack-mtimes $foo_mtimes >foo.actual && + test-tool pack-mtimes $bar_mtimes >bar.actual && + + echo "$foo $(cat foo.mtime)" >foo.expect && + echo "$bar $(cat bar.mtime)" >bar.expect && + + test_cmp foo.expect foo.actual && + test_cmp bar.expect bar.actual && + test "$foo_mtimes" != "$bar_mtimes" ) ' -test_expect_success 'cruft repack ignores pack.packSizeLimit' ' +test_expect_success 'cruft repack with pack.packSizeLimit' ' ( cd max-pack-size && # repack everything back together to remove the existing cruft @@ -599,9 +630,12 @@ test_expect_success 'cruft repack ignores pack.packSizeLimit' ' # ensure the same post condition is met when --max-pack-size # would otherwise be inferred from the configuration find $packdir -name "*.mtimes" >cruft && - test_line_count = 1 cruft && - test-tool pack-mtimes "$(basename "$(cat cruft)")" >objects && - test_line_count = 2 objects + test_line_count = 2 cruft && + for pack in $(cat cruft) + do + test-tool pack-mtimes "$(basename $pack)" >objects && + test_line_count = 1 objects || return 1 + done ) ' From patchwork Mon Aug 28 22:49:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13368389 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 05EDAC83F15 for ; Mon, 28 Aug 2023 22:49:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234269AbjH1WtY (ORCPT ); Mon, 28 Aug 2023 18:49:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41406 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234265AbjH1WtO (ORCPT ); Mon, 28 Aug 2023 18:49:14 -0400 Received: from mail-yb1-xb2d.google.com (mail-yb1-xb2d.google.com [IPv6:2607:f8b0:4864:20::b2d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1B9A311A for ; Mon, 28 Aug 2023 15:49:12 -0700 (PDT) Received: by mail-yb1-xb2d.google.com with SMTP id 3f1490d57ef6-d35a9d7a5bdso3388008276.0 for ; Mon, 28 Aug 2023 15:49:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1693262951; x=1693867751; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=XOikyeoGucEsamf00Blvl897MMF/3uizJb60cZn2xCU=; b=kzf85jdSjj79TBZkCfGpEP+l838eapC21xt37wX78VTGSosgX/Z3euo+Uv4mtoeKGO TSN6P9KrlQoHxXgDr3ELl+CsfZRkfbXDliqzbK2c7/LsbQNnSk3zFCfZYgib8jgJMDpj ZfFU2q4yQgfZuwYV5o707bZaz9+Pioq2Twa+PFNSw1GOFbMC/GS2cTvwJZjektTe3otT Rfup3aIfUcSQ087MHTwVgleNm5mJOp+gzOI3EwplLo6YP+rz8vw310o/v280NPQ3qZY9 baqJt4Ca+7Cuf7qocbPAgHfdwciDzsq0E77ENIsC9ZPkJFo/nrAd/6JKwhjPOas2CA29 uisg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693262951; x=1693867751; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=XOikyeoGucEsamf00Blvl897MMF/3uizJb60cZn2xCU=; b=L6QTwwrqqdGJD/j8YxMLH4G+GVz0TkXFZeTXxlPZOLjWHt36ovaQd/bTZtaPP2UOnA fFI0t0LKxOU/H7z2vyawZVUwkw8TFMf0UKZn3knPWgX9fqm/n+SXiCeFlIGVOFQGMrwj WKW9VoiP5jo3Gq0Cfm+L+Mphi/zsy5IW0B4HBjMupjcL90KPTmhxf/se75XvU3Z/4nS6 MTysPTmZi+mxMPHKARcE6gygwq150nrES00j6Umqs6xbkbMdxoOtZD2E5H0tYfTQFoQk AZiUAjCw7imvEyyQSTaaI2y5xVGLjJUnOJQJX8admSagoaD8nsIifWreH5nipi3wY2N7 8dgA== X-Gm-Message-State: AOJu0Yzs94Yw2IjKvJSoEAsVbVfZzoJHPn8Wm525xvI+gsvumb2gNUfX 0hjGzosgM3LLnNwIF3DyP2fC7bzWK3mg7ZXfFjkDng== X-Google-Smtp-Source: AGHT+IFbKYuigfxT4zQ6526EvPnOIXFP2Aj9Eh9JSd/Vul6hFEagCPD6ocEDlkSnKj0Zdh3cS1VO7w== X-Received: by 2002:a5b:b4a:0:b0:d71:6b6e:1071 with SMTP id b10-20020a5b0b4a000000b00d716b6e1071mr25583339ybr.32.1693262951085; Mon, 28 Aug 2023 15:49:11 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id v124-20020a252f82000000b00d7465a90f0csm1908662ybv.22.2023.08.28.15.49.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Aug 2023 15:49:10 -0700 (PDT) Date: Mon, 28 Aug 2023 18:49:10 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King Subject: [PATCH 3/4] Documentation/gitformat-pack.txt: remove multi-cruft packs alternative Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org This text, originally from 3d89a8c118 (Documentation/technical: add cruft-packs.txt, 2022-05-20) lists multiple cruft packs as a potential alternative to the design of cruft packs. We have always supported multiple cruft packs (i.e. we use the most recent mtime for a given object among all cruft packs which contain it, etc.), but haven't encouraged its use. We still aren't encouraging users to go out and generate multiple cruft packs, but let's take a step in that direction by dropping language that suggests we aren't capable of working with multiple cruft packs. Signed-off-by: Taylor Blau --- Documentation/gitformat-pack.txt | 10 +--------- 1 file changed, 1 insertion(+), 9 deletions(-) diff --git a/Documentation/gitformat-pack.txt b/Documentation/gitformat-pack.txt index 0c1be2dbe8..49bb09d7df 100644 --- a/Documentation/gitformat-pack.txt +++ b/Documentation/gitformat-pack.txt @@ -618,21 +618,13 @@ understand cruft packs. Notable alternatives to this design include: - - The location of the per-object mtime data, and - - Storing unreachable objects in multiple cruft packs. + - The location of the per-object mtime data. On the location of mtime data, a new auxiliary file tied to the pack was chosen to avoid complicating the `.idx` format. If the `.idx` format were ever to gain support for optional chunks of data, it may make sense to consolidate the `.mtimes` format into the `.idx` itself. -Storing unreachable objects among multiple cruft packs (e.g., creating a new -cruft pack during each repacking operation including only unreachable objects -which aren't already stored in an earlier cruft pack) is significantly more -complicated to construct, and so aren't pursued here. The obvious drawback to -the current implementation is that the entire cruft pack must be re-written from -scratch. - GIT --- Part of the linkgit:git[1] suite From patchwork Mon Aug 28 22:49:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13368390 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A73D0C83F17 for ; Mon, 28 Aug 2023 22:49:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234271AbjH1WtZ (ORCPT ); Mon, 28 Aug 2023 18:49:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41420 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234267AbjH1WtR (ORCPT ); Mon, 28 Aug 2023 18:49:17 -0400 Received: from mail-yw1-x1131.google.com (mail-yw1-x1131.google.com [IPv6:2607:f8b0:4864:20::1131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9E2EF11A for ; Mon, 28 Aug 2023 15:49:14 -0700 (PDT) Received: by mail-yw1-x1131.google.com with SMTP id 00721157ae682-579de633419so44186267b3.3 for ; Mon, 28 Aug 2023 15:49:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1693262953; x=1693867753; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=6cF7NzjRgrAHsTCdhX+DXEkLPa2jkZhRp7bQ1m1SDus=; b=w3FJgbDxkGwqXdPQk+9Zo8gNGP6GUN4iPLej0nLoTnZYsQ2nk5TSKsDSCDuM0+QN1J 5NuUFY7VyOoASDTXL58Lnalnq3f/v8BrvuexDgyihu55QZ8f/BnEBtV8rBNr8GctU8xP IlTe7dXGKSZuEeUQbYqN8W69CBdOOOmNbrqTSF6J9sO1JptyNQnB5dJIl+7mfXIOru4p VobRaz41JXKX+zDSEOWayIeNKKkwt2yGdfM5ORRKdN8rHsh2fG6XYctP+gi1Gl519jNo /e1STONFnboDYimYh22dVYjAAfm1XGXi3TDmjq/8d4nu87VTqtvvULmi+e0ceWYiLZLc RGQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693262953; x=1693867753; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=6cF7NzjRgrAHsTCdhX+DXEkLPa2jkZhRp7bQ1m1SDus=; b=OAPaQZnIfjKkM35PnkxNlUJhxN6CktuI8Z0ksJUz/D8YXCEB5vakU48TfdkQGga9Bb B5O1Y8yrb8WJPSVkKBP659Ahx8LFtz09XgOnpfBxRUJbf964Dxjf6Dkm8I2jH45fZ1qz QJ4+Bm3Ak9oPR4uGRgvo2ZMlNoe7wzs8RNt1eNvFQf8Ol7w8fYY1RKiX9RpRLcgFsSwy Nl+NFENe+1zESOC9rS7E2OFRwEoGAmjI21ehfFQ8WfwhEtO0dWKY8DQS11RVSdjRxbB8 eYGKvqr35DDYimhtiIVS0eSkxi4ixqjqCDlJCy+VlmgycdsvndWKmClujbDld+A//GJb 1rdg== X-Gm-Message-State: AOJu0YztNV1A5T5IAzyw6BzB1XsXRay6Ynh0egB67+797GgRg7XT/fYD SIYH04ovGE02vTquukmcJkrFHVOXWiEhAlEoZUjLzA== X-Google-Smtp-Source: AGHT+IGQfo3O3b3PB7Hrv12Uxsj56lQB1twQI2xz7bVF5k3KiIgz2ir+nU8BdHp7tqZDB6iV4BlMmg== X-Received: by 2002:a81:8782:0:b0:589:a9fc:ffcd with SMTP id x124-20020a818782000000b00589a9fcffcdmr27624036ywf.20.1693262953657; Mon, 28 Aug 2023 15:49:13 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id t130-20020a0dea88000000b005839a780c29sm2406323ywe.102.2023.08.28.15.49.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Aug 2023 15:49:13 -0700 (PDT) Date: Mon, 28 Aug 2023 18:49:12 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King Subject: [PATCH 4/4] Documentation/gitformat-pack.txt: drop mixed version section Message-ID: <03c54e21ee48028cd7c0d136671eb9d1e8b9546d.1693262936.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org This section was added in 3d89a8c118 (Documentation/technical: add cruft-packs.txt, 2022-05-20) to highlight a potential pitfall when deploying cruft packs in an environment where multiple versions of Git are GC-ing the same repository. Now that it has been more than a year since 3d89a8c118 was written, let's drop this section as it is no longer relevant. Signed-off-by: Taylor Blau --- Documentation/gitformat-pack.txt | 26 -------------------------- 1 file changed, 26 deletions(-) diff --git a/Documentation/gitformat-pack.txt b/Documentation/gitformat-pack.txt index 49bb09d7df..870e00f298 100644 --- a/Documentation/gitformat-pack.txt +++ b/Documentation/gitformat-pack.txt @@ -588,32 +588,6 @@ later on. It is linkgit:git-gc[1] that is typically responsible for removing expired unreachable objects. -=== Caution for mixed-version environments - -Repositories that have cruft packs in them will continue to work with any older -version of Git. Note, however, that previous versions of Git which do not -understand the `.mtimes` file will use the cruft pack's mtime as the mtime for -all of the objects in it. In other words, do not expect older (pre-cruft pack) -versions of Git to interpret or even read the contents of the `.mtimes` file. - -Note that having mixed versions of Git GC-ing the same repository can lead to -unreachable objects never being completely pruned. This can happen under the -following circumstances: - - - An older version of Git running GC explodes the contents of an existing - cruft pack loose, using the cruft pack's mtime. - - A newer version running GC collects those loose objects into a cruft pack, - where the .mtime file reflects the loose object's actual mtimes, but the - cruft pack mtime is "now". - -Repeating this process will lead to unreachable objects not getting pruned as a -result of repeatedly resetting the objects' mtimes to the present time. - -If you are GC-ing repositories in a mixed version environment, consider omitting -the `--cruft` option when using linkgit:git-repack[1] and linkgit:git-gc[1], and -setting the `gc.cruftPacks` configuration to "false" until all writers -understand cruft packs. - === Alternatives Notable alternatives to this design include: