From patchwork Mon Dec 20 14:48:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12688047 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3BB1BC433FE for ; Mon, 20 Dec 2021 14:57:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236918AbhLTO5d (ORCPT ); Mon, 20 Dec 2021 09:57:33 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34562 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238918AbhLTOyr (ORCPT ); Mon, 20 Dec 2021 09:54:47 -0500 Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com [IPv6:2a00:1450:4864:20::430]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 51CCCC08EC32 for ; Mon, 20 Dec 2021 06:48:15 -0800 (PST) Received: by mail-wr1-x430.google.com with SMTP id e5so20588345wrc.5 for ; Mon, 20 Dec 2021 06:48:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=Lisa/RicwXWE0t+E5583E7BBfl3r4+pUbqm6E10qN8U=; b=Ia399P1Ln4W9McX0dwGXp/zxmmvdfX8/4fgRsbGXrv/gx7Ssos7IrazLxI1eGIIKBB OJo6mWgPAr/6ra+hXDif91IjMQApUWIBy6UKKPapQKy8sRGO4zkAdDMYONi63fuz/DFZ M3DTkxxBfTMIqkREkkJ2Bc7bctpUF92GwPp3/EouZH1qect0eB2x1zPw92T7ts0JpU8X TjECr5Gdw+6tyF7r3XaTVqsVWzyK4dRS767Ps032qPpHQITOh8uKFmB7OkaK4/omPHas 2mkzcC01Z9cxtopLQ6Q2VyxVhHw3tIMWhvetYSw74hSi2yCKezkTuG0P1PseF4XFJMes OrlA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=Lisa/RicwXWE0t+E5583E7BBfl3r4+pUbqm6E10qN8U=; b=1L11uTWrjtEKhbhKuO5Ekl5qtLJjDFxvlNm0qO+4gNFw6ziisz0ZVC1l2bElmlcaDr tM6HY6XTSLkCABSuLByGiQBY57h4RLWeSw0Y3uI9RXKtjMHzsXfLgWBpwkjrrsZNDGB4 BMywpjHAMz9uqhCRhs1y5z0sCufyVmFyMpxcuVpIak0KuHVgNMgxygtuxD97HEaGSeb7 QNYTCpQb7LfINr8RDtQWXVPNuLcBmW6TNv/BFvvlrbgCs+fG2XZfJGhT3hci4RrUhUwR yLoqvR3I2WuYfyxIyN4n7VJNpvigrP9STooei6s2LG0KyUq42OpGP/MAXheemxepqKSB hDkQ== X-Gm-Message-State: AOAM5316p7vKVC2gA1YhvMZtIeC705kUpoh/Qrhdyz+z6YCj5s76Dy2A TQTxl4SBj/tEGboMmmr5HUc1MrlbvqY= X-Google-Smtp-Source: ABdhPJxAj/3VQVl3/XhZIM4VA4INisht4AuLRBHeJcdHIY6jiV6h/XdxeEYLuIWMcZ3unPK9byKPzw== X-Received: by 2002:a5d:53cf:: with SMTP id a15mr12872482wrw.395.1640011693751; Mon, 20 Dec 2021 06:48:13 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id g3sm12333448wrp.79.2021.12.20.06.48.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Dec 2021 06:48:13 -0800 (PST) Message-Id: <747328a4dd69f2325892b4d89e3e62a0bcfbba59.1640011691.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 20 Dec 2021 14:48:10 +0000 Subject: [PATCH v2 1/2] repack: respect kept objects with '--write-midx -b' Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: me@ttaylorr.com, gitster@pobox.com, Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Derrick Stolee , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Historically, we needed a single packfile in order to have reachability bitmaps. This introduced logic that when 'git repack' had a '-b' option that we should stop sending the '--honor-pack-keep' option to the 'git pack-objects' child process, ensuring that we create a packfile containing all reachable objects. In the world of multi-pack-index bitmaps, we no longer need to repack all objects into a single pack to have valid bitmaps. Thus, we should continue sending the '--honor-pack-keep' flag to 'git pack-objects'. The fix is very simple: only disable the flag when writing bitmaps but also _not_ writing the multi-pack-index. This opens the door to new repacking strategies that might want to keep some historical set of objects in a stable pack-file while only repacking more recent objects. To test, create a new 'test_subcommand_inexact' helper that is more flexible than 'test_subcommand'. This allows us to look for the --honor-pack-keep flag without over-indexing on the exact set of arguments. Signed-off-by: Derrick Stolee --- builtin/repack.c | 2 +- t/t7700-repack.sh | 6 ++++++ t/test-lib-functions.sh | 34 ++++++++++++++++++++++++++++++++++ 3 files changed, 41 insertions(+), 1 deletion(-) diff --git a/builtin/repack.c b/builtin/repack.c index 9b0be6a6ab3..1f128b7c90b 100644 --- a/builtin/repack.c +++ b/builtin/repack.c @@ -693,7 +693,7 @@ int cmd_repack(int argc, const char **argv, const char *prefix) write_bitmaps = 0; } if (pack_kept_objects < 0) - pack_kept_objects = write_bitmaps > 0; + pack_kept_objects = write_bitmaps > 0 && !write_midx; if (write_bitmaps && !(pack_everything & ALL_INTO_ONE) && !write_midx) die(_(incremental_bitmap_conflict_error)); diff --git a/t/t7700-repack.sh b/t/t7700-repack.sh index 0260ad6f0e0..63c9a247f57 100755 --- a/t/t7700-repack.sh +++ b/t/t7700-repack.sh @@ -372,4 +372,10 @@ test_expect_success '--write-midx with preferred bitmap tips' ' ) ' +test_expect_success '--write-midx -b packs non-kept objects' ' + GIT_TRACE2_EVENT="$(pwd)/trace.txt" \ + git repack --write-midx -a -b && + test_subcommand_inexact git pack-objects --honor-pack-keep ... < +# +# For example, to look for an invocation of "git pack-objects" +# with the "--honor-pack-keep" argument, use +# +# GIT_TRACE2_EVENT=event.log git repack ... && +# test_subcommand git pack-objects --honor-pack-keep X-Patchwork-Id: 12688045 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E0A6DC433F5 for ; Mon, 20 Dec 2021 14:57:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235146AbhLTO50 (ORCPT ); Mon, 20 Dec 2021 09:57:26 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35638 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239395AbhLTOz2 (ORCPT ); Mon, 20 Dec 2021 09:55:28 -0500 Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 66804C08EC6F for ; Mon, 20 Dec 2021 06:48:21 -0800 (PST) Received: by mail-wm1-x334.google.com with SMTP id g132so4206357wmg.2 for ; Mon, 20 Dec 2021 06:48:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=viV05ECn3yXTQJKvdp3nVCqxOIRQk+2LXBWPBXwsFus=; b=T8Ksi7sS+tFR6l2MSFht/faLe4rK7YfUF5Fx3W6vpmhS6Ja/eQogasl7XIVeeP+6ZU DqoGNSkSBfPhZfvugsmzBDm7VWNlWktpMUf6XvrNB6ige7rjoi22zGNxSzIGJtiTCN/T yPqNjd8g1GkRrQgA5UyWKKg1voVLsE+MiqLjGXYvEcxBcf0FgPsfpa3MXrbpQxdJaYA0 JxPJCtNyUZpFl9Wo0ZSCgaUKp4FUzOO2A/DKd4XOrZ4hnWdftKYzg/51XQRMKAj7b+xD Ajxb9SmdhIJOs1HVmm1b2FS8qdDB8G9fSDJd8RClpYTxx6RqFw83qzGtLS6/nDMUtNZ3 /uIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=viV05ECn3yXTQJKvdp3nVCqxOIRQk+2LXBWPBXwsFus=; b=xaIUePPHQAO48J4WBxoW0d0+tU2Wr/k8rvdJy7BbXywhLdneabkK2Gi1yAeH9VGGHs fmim/CXKs26hOC77wtz3FsgoH++raaFYmdSJQgRNuBUYOf7SVmIvhXoy2m1vJ5I3FivE 4GsNYjpjEHVFPLcKnOE42IcvcJX4Qj2wb3tZyorIZvJjMfQ3wGHegQgnf9qjHquJ1NOh ZONdEiEBggALOUAGB/lZyreD7ZdoU8aLqwiR1pL1txVcXuXoG2xOBa2HyKkw7xmA1fRP 8OeJxAuvUonZPXFKD4wfx2ZxVU8azBtjkzXa2icV2G/BS2CGYE0a83YPXCwIeK8PrnoQ uGww== X-Gm-Message-State: AOAM530NwLDo4yiQQDK2pMc5FoPjqdLEvMEzcS8Qn/FzXEjZBdEymmQO oNHCgLBvBul0RWxhZsYrGb5a8SzqUrk= X-Google-Smtp-Source: ABdhPJxlHXMFjas2PYW6fVoNGo0vwAuXr7eeB5QF/x9Uto5OZUhRNWZW9jokugM+KHQxoA+Ov79MOg== X-Received: by 2002:a05:600c:1548:: with SMTP id f8mr14607677wmg.20.1640011694505; Mon, 20 Dec 2021 06:48:14 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id u9sm14403589wmm.7.2021.12.20.06.48.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Dec 2021 06:48:14 -0800 (PST) Message-Id: <41260bf0829970eeb27bec2cb41cabeffa54dabb.1640011691.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 20 Dec 2021 14:48:11 +0000 Subject: [PATCH v2 2/2] repack: make '--quiet' disable progress Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: me@ttaylorr.com, gitster@pobox.com, Jeff King , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Derrick Stolee , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee While testing some ideas in 'git repack', I ran it with '--quiet' and discovered that some progress output was still shown. Specifically, the output for writing the multi-pack-index showed the progress. The 'show_progress' variable in cmd_repack() is initialized with isatty(2) and is not modified at all by the '--quiet' flag. The '--quiet' flag modifies the po_args.quiet option which is translated into a '--quiet' flag for the 'git pack-objects' child process. However, 'show_progress' is used to directly send progress information to the multi-pack-index writing logic which does not use a child process. The fix here is to modify 'show_progress' to be false if po_opts.quiet is true, and isatty(2) otherwise. This new expectation simplifies a later condition that checks both. Update the documentation to make it clear that '-q' will disable all progress in addition to ensuring the 'git pack-objects' child process will receive the flag. Use 'test_terminal' to check that this works to get around the isatty(2) check. Helped-by: Jeff King Signed-off-by: Derrick Stolee --- Documentation/git-repack.txt | 5 +++-- builtin/repack.c | 6 ++++-- t/t7700-repack.sh | 7 +++++++ 3 files changed, 14 insertions(+), 4 deletions(-) diff --git a/Documentation/git-repack.txt b/Documentation/git-repack.txt index 7183fb498f4..ee30edc178a 100644 --- a/Documentation/git-repack.txt +++ b/Documentation/git-repack.txt @@ -76,8 +76,9 @@ to the new separate pack will be written. linkgit:git-pack-objects[1]. -q:: - Pass the `-q` option to 'git pack-objects'. See - linkgit:git-pack-objects[1]. +--quiet:: + Show no progress over the standard error stream and pass the `-q` + option to 'git pack-objects'. See linkgit:git-pack-objects[1]. -n:: Do not update the server information with diff --git a/builtin/repack.c b/builtin/repack.c index 1f128b7c90b..c393a5db774 100644 --- a/builtin/repack.c +++ b/builtin/repack.c @@ -612,7 +612,7 @@ int cmd_repack(int argc, const char **argv, const char *prefix) struct tempfile *refs_snapshot = NULL; int i, ext, ret; FILE *out; - int show_progress = isatty(2); + int show_progress; /* variables to be filled by option parsing */ int pack_everything = 0; @@ -725,6 +725,8 @@ int cmd_repack(int argc, const char **argv, const char *prefix) prepare_pack_objects(&cmd, &po_args); + show_progress = !po_args.quiet && isatty(2); + strvec_push(&cmd.args, "--keep-true-parents"); if (!pack_kept_objects) strvec_push(&cmd.args, "--honor-pack-keep"); @@ -926,7 +928,7 @@ int cmd_repack(int argc, const char **argv, const char *prefix) } strbuf_release(&buf); } - if (!po_args.quiet && show_progress) + if (show_progress) opts |= PRUNE_PACKED_VERBOSE; prune_packed_objects(opts); diff --git a/t/t7700-repack.sh b/t/t7700-repack.sh index 63c9a247f57..fe2b493d0ee 100755 --- a/t/t7700-repack.sh +++ b/t/t7700-repack.sh @@ -5,6 +5,7 @@ test_description='git repack works correctly' . ./test-lib.sh . "${TEST_DIRECTORY}/lib-bitmap.sh" . "${TEST_DIRECTORY}/lib-midx.sh" +. "${TEST_DIRECTORY}/lib-terminal.sh" commit_and_pack () { test_commit "$@" 1>&2 && @@ -378,4 +379,10 @@ test_expect_success '--write-midx -b packs non-kept objects' ' test_subcommand_inexact git pack-objects --honor-pack-keep stderr && + test_must_be_empty stderr +' + test_done