From patchwork Wed Aug 3 20:57:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 12935909 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9765AC19F2D for ; Wed, 3 Aug 2022 20:57:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238454AbiHCU5g (ORCPT ); Wed, 3 Aug 2022 16:57:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59200 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238367AbiHCU5d (ORCPT ); Wed, 3 Aug 2022 16:57:33 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AF5131F2F2 for ; Wed, 3 Aug 2022 13:57:32 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id s6-20020a25c206000000b0066ebb148de6so14253180ybf.15 for ; Wed, 03 Aug 2022 13:57:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:references:mime-version:message-id:in-reply-to :date:from:to:cc; bh=Ay6KooD/d7Pw4Wgt1Dk883STxhxjgAuHCg+rloqNWi0=; b=jB9uNPmGXyp3jaT/y+/9OaSlbJHAnOUgjI6mMU8bMoEJ5co1sX7+TrB5kSCv7Qlz0Z AXhoEkIuqz835XrIeakYYrbpnGRxTXZYyImOFu6GNyUIxTXEkMqPLdarHEpORNU6CrF2 Jn4tf2uxgzMjTJ/Ypen/5JyKbRkbHOUrd6XBQ9BDTyVhy6LUI7TRMJpUqZLzq3+/Pqb/ ZjBfC4F9bUtJgeY1uAi+fKkt5oOr6Uxnag+TrmO9vm9rlVp3FF3bqB2HealMAhNxjKcN HPutGE9JjFEE9djsF7UYQyp4kPFVYJAZorGFAJGfFc80gcnNloQm/xzaokFSZw7EnoXt iGkw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:references:mime-version:message-id:in-reply-to :date:x-gm-message-state:from:to:cc; bh=Ay6KooD/d7Pw4Wgt1Dk883STxhxjgAuHCg+rloqNWi0=; b=D/exCRWo0mDIMYafJB/V/q7ZsdhvcvNKPOWMQphePyjChdd4acx9oQpbSBIdRfHuYB hihBxuXvZq2b4dX3lMGv0yA7RgXbUsF4qFXDUE1VHNRESga50PBdoVCW0ez1vi3MU7KV SVTlo8ltpuqHLRRxsVhEEA6CFn2x6jpxaZ6F1hxI3LYYiBntBCHDtYZdFCkA6dGf/OUd kvfZPcjjcawp0bCTtqcN+iOd6gcD8PEucgltM5lTe3F3rFMhZGUdZPufoIJ46taGUL6u 93T7SzWcmCk5pJEuZcJ57mjri3I4iVbNQti0aXAzPDROSBjTSjhYEGkh7BKNVCTOa0Pi NYYQ== X-Gm-Message-State: ACgBeo2eloT9QOuj2WAGoHsZ1fQ/S+cgX824xhyBeZu9XFv7kIRyPh+E vuqcHBXzdrjsibwbqB+bAtdHQKFuthP/sjIrHjYl7VF4sIIDdnmdr3FJ/awc+0y6vYP7BBMXbaF 6kpPuyUhmtC1k1fq52WRbX4N40IPmLLNd56gGBlCdiCMx03uKNrS12kVtaIk5Nb0Wb/UbfMAqkw == X-Google-Smtp-Source: AA6agR66Jp9REb59k4beye/AUY9QTZF5P1bYDNQBuBX7qxDFPdcvO+feskWfm5E/bM2PsU8zVfE64c9Wvuj19K0uXDo= X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:200:b3d5:c5f6:e39a:a9e2]) (user=emilyshaffer job=sendgmr) by 2002:a0d:e2ca:0:b0:323:1ac3:8a38 with SMTP id l193-20020a0de2ca000000b003231ac38a38mr25954115ywe.89.1659560252459; Wed, 03 Aug 2022 13:57:32 -0700 (PDT) Date: Wed, 3 Aug 2022 13:57:20 -0700 In-Reply-To: <20220803205721.3686361-1-emilyshaffer@google.com> Message-Id: <20220803205721.3686361-2-emilyshaffer@google.com> Mime-Version: 1.0 References: <20220803205721.3686361-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.37.1.455.g008518b4e5-goog Subject: [PATCH 1/2] gc: add tests for --cruft and friends From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In 5b92477f89 (builtin/gc.c: conditionally avoid pruning objects via loose, 2022-05-20) gc learned to respect '--cruft' and 'gc.cruftPacks'. '--cruft' is exercised in t5329-pack-objects-cruft.sh, but in a way that doesn't check whether a lone gc run generates these cruft packs. 'gc.cruftPacks' is never exercised. Add some tests to exercise these options to gc in the gc test suite. Signed-off-by: Emily Shaffer --- t/t6500-gc.sh | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) diff --git a/t/t6500-gc.sh b/t/t6500-gc.sh index cd6c53360d..e4c2c3583d 100755 --- a/t/t6500-gc.sh +++ b/t/t6500-gc.sh @@ -202,6 +202,42 @@ test_expect_success 'one of gc.reflogExpire{Unreachable,}=never does not skip "e grep -E "^trace: (built-in|exec|run_command): git reflog expire --" trace.out ' +test_expect_success 'gc --cruft generates a cruft pack' ' + git init crufts && + test_when_finished "rm -fr crufts" && + ( + cd crufts && + test_commit base && + + test_commit --no-tag foo && + test_commit --no-tag bar && + git reset HEAD^^ && + + git gc --cruft && + + cruft=$(basename $(ls .git/objects/pack/pack-*.mtimes) .mtimes) && + test_path_is_file .git/objects/pack/$cruft.pack + ) +' + +test_expect_success 'gc.cruftPacks=true generates a cruft pack' ' + git init crufts && + test_when_finished "rm -fr crufts" && + ( + cd crufts && + test_commit base && + + test_commit --no-tag foo && + test_commit --no-tag bar && + git reset HEAD^^ && + + git -c gc.cruftPacks=true gc && + + cruft=$(basename $(ls .git/objects/pack/pack-*.mtimes) .mtimes) && + test_path_is_file .git/objects/pack/$cruft.pack + ) +' + run_and_wait_for_auto_gc () { # We read stdout from gc for the side effect of waiting until the # background gc process exits, closing its fd 9. Furthermore, the From patchwork Wed Aug 3 20:57:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 12935910 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0784DC19F2B for ; Wed, 3 Aug 2022 20:57:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238619AbiHCU5i (ORCPT ); Wed, 3 Aug 2022 16:57:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59254 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238367AbiHCU5g (ORCPT ); Wed, 3 Aug 2022 16:57:36 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8C664201AC for ; Wed, 3 Aug 2022 13:57:35 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id s186-20020a255ec3000000b0067162ed1bd3so14186795ybb.8 for ; Wed, 03 Aug 2022 13:57:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:references:mime-version:message-id:in-reply-to :date:from:to:cc; bh=7izbWjn+uPW+NFfolN43cdTPVVisHDTVwliVcGQdWcA=; b=HEDdt2B7PrPe8XK+RqsBC3kHN9bjkYVQZNy1B9yaOryrDozNbWgpaRR1uBaPlPi6Ix KfrPr649BYE5J7hj+jKCMB+0LWINYQkYkSOJWvv2BI7ntAY56cFx621Qo+UY2ZnEoA+O 66ul193byWlg9l8b0S/yseWang1FIMwv+mn22GUUuaEjQNY/2liGifIosZWexB8y2xjc SA9PiX20c/3wnKfI93lzyeke/n7aPO1l37Pho5rkqOBBNDeZ30B5JpzVxlxrswa+l8bZ qtXS+DweFNxNXjOhQXg7GAI6PPRtDMdENoGGtCuaElISg6PoTvdK5cMIgEr3Dmptssut PLBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:references:mime-version:message-id:in-reply-to :date:x-gm-message-state:from:to:cc; bh=7izbWjn+uPW+NFfolN43cdTPVVisHDTVwliVcGQdWcA=; b=PREGg7yF5UH7+HTqZU7W1lFE8Od8++M/kMNADB3ReSxxaGnEQX9Q0JyxF3irbGE3kW H6qsBdy/6v5xlSWmAuDUtf4slk/zhBiF87vLjKA3kxGHhncRYRp7fF/ItkQYv+5GD13Y pMFhYvFE7pxY5d7DDKbvW7Y1bLQlscxesOva/giQgIPR5W3GQKMI3lLbiIXwxArh2NuU Q7hJnZ+MDrIpEamh+I+4n5qxSRFKoYVNPM6I1sLPyBVqlPMTtAtkk20jATeB+dlJvMCg nncgbPVvrDgA5uK5QcvhiVhBaAQNnLv+h8rdsc/Y89PiLWm3Snfmez618GTqE3s494wU GYAg== X-Gm-Message-State: ACgBeo1htawx5AfBDX55386zC+OAXTz9oz6CNrY40YpZIBacdkLZMyvb WYwal/rgohBDIPLAhB1WyV7LFzjM5Sxbg9bPouvdwXBTV+qXXIkwq/9Ixvk/KqnH8niWHqxxM+F G4X+6bT3+DBtXxSX1KitVk5NiP1KrSrvutRBKa8Zm8RpsxDDeC8lzEl+eZFa2JUMZiRzNIH6hig == X-Google-Smtp-Source: AA6agR6MlMJLKs8jk7JBObbY88kS7c8rBjljc5LU67kuMie70/Ov2Aa/iL+LpCdQ6GMCjB656GfzZX89XU03Itvhr4U= X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:200:b3d5:c5f6:e39a:a9e2]) (user=emilyshaffer job=sendgmr) by 2002:a25:e212:0:b0:670:80bb:ce9e with SMTP id h18-20020a25e212000000b0067080bbce9emr19661967ybe.603.1659560254876; Wed, 03 Aug 2022 13:57:34 -0700 (PDT) Date: Wed, 3 Aug 2022 13:57:21 -0700 In-Reply-To: <20220803205721.3686361-1-emilyshaffer@google.com> Message-Id: <20220803205721.3686361-3-emilyshaffer@google.com> Mime-Version: 1.0 References: <20220803205721.3686361-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.37.1.455.g008518b4e5-goog Subject: [PATCH 2/2] config: let feature.experimental imply gc.cruftPacks=true From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org We are interested in exploring whether gc.cruftPacks=true should become the default value; to determine whether it is safe to do so, let's encourage more users to try it out. Users who have set feature.experimental=true have already volunteered to try new and possibly-breaking config changes, so let's try this new default with that set of users. Signed-off-by: Emily Shaffer --- Documentation/config/feature.txt | 2 ++ builtin/gc.c | 6 ++++++ t/t6500-gc.sh | 35 ++++++++++++++++++++++++++++++++ 3 files changed, 43 insertions(+) diff --git a/Documentation/config/feature.txt b/Documentation/config/feature.txt index cdecd04e5b..f029c422be 100644 --- a/Documentation/config/feature.txt +++ b/Documentation/config/feature.txt @@ -14,6 +14,8 @@ feature.experimental:: + * `fetch.negotiationAlgorithm=skipping` may improve fetch negotiation times by skipping more commits at a time, reducing the number of round trips. +* `gc.cruftPacks=true` reduces disk space used by unreachable objects during +garbage collection. feature.manyFiles:: Enable config options that optimize for repos with many files in the diff --git a/builtin/gc.c b/builtin/gc.c index eeff2b760e..919cc508c5 100644 --- a/builtin/gc.c +++ b/builtin/gc.c @@ -136,6 +136,7 @@ static int gc_config_is_timestamp_never(const char *var) static void gc_config(void) { const char *value; + int experimental = 0; if (!git_config_get_value("gc.packrefs", &value)) { if (value && !strcmp(value, "notbare")) @@ -148,6 +149,11 @@ static void gc_config(void) gc_config_is_timestamp_never("gc.reflogexpireunreachable")) prune_reflogs = 0; + /* feature.experimental implies gc.cruftPacks=true */ + git_config_get_bool("feature.experimental", &experimental); + if (experimental) + cruft_packs = 1; + git_config_get_int("gc.aggressivewindow", &aggressive_window); git_config_get_int("gc.aggressivedepth", &aggressive_depth); git_config_get_int("gc.auto", &gc_auto_threshold); diff --git a/t/t6500-gc.sh b/t/t6500-gc.sh index e4c2c3583d..4ab6750111 100755 --- a/t/t6500-gc.sh +++ b/t/t6500-gc.sh @@ -238,6 +238,41 @@ test_expect_success 'gc.cruftPacks=true generates a cruft pack' ' ) ' +test_expect_success 'feature.experimental=true generates a cruft pack' ' + git init crufts && + test_when_finished "rm -fr crufts" && + ( + cd crufts && + test_commit base && + + test_commit --no-tag foo && + test_commit --no-tag bar && + git reset HEAD^^ && + + git -c feature.experimental=true gc && + + cruft=$(basename $(ls .git/objects/pack/pack-*.mtimes) .mtimes) && + test_path_is_file .git/objects/pack/$cruft.pack + ) +' + +test_expect_success 'feature.experimental=false allows explicit cruft packs' ' + git init crufts && + test_when_finished "rm -fr crufts" && + ( + cd crufts && + test_commit base && + + test_commit --no-tag foo && + test_commit --no-tag bar && + git reset HEAD^^ && + + git -c gc.cruftPacks=true -c feature.experimental=false gc && + cruft=$(basename $(ls .git/objects/pack/pack-*.mtimes) .mtimes) && + test_path_is_file .git/objects/pack/$cruft.pack + ) +' + run_and_wait_for_auto_gc () { # We read stdout from gc for the side effect of waiting until the # background gc process exits, closing its fd 9. Furthermore, the