From patchwork Tue Dec 31 02:18:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ZheNing Hu X-Patchwork-Id: 13923590 Received: from mail-wm1-f45.google.com (mail-wm1-f45.google.com [209.85.128.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2C49B1F94C for ; Tue, 31 Dec 2024 02:18:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.45 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735611520; cv=none; b=b2ZxvBKB142GEJIwMgLUej2n4iRkwjAPNagMdqsnH1Wynu/eyhDnFqx4bR8sdiETk1WW8VPCjgzSc43JiXNysVOycqBGiSv6S4S0gEfo2kVl7rQf92QVdmso7OWDaz24szsjzsu2TDRVwoYSQbHuFOvDlMy3UFKktIi5leDUJQ0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735611520; c=relaxed/simple; bh=Y1/6q3uZ4YV8INkRWuZes4zDXjMkBukkYQ8+cTN0X/E=; h=Message-Id:In-Reply-To:References:From:Date:Subject:Content-Type: MIME-Version:To:Cc; b=Yj2cstlAVR6bp7GnwVyafY9q8bow4uDkpT0AaaEqYdjvn64aO5IpSmaV4QErIuFVoUkSFDgE1cyZKqsEv7XkTswYCfJ1HBDeORaFmfs8DE47GMZTnPK4vxhNNaxI2OJU6Eq3elPS/V9FUpV4a4DvWusmWQwiQHXV6tf5aWlkf18= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=nFohmI7k; arc=none smtp.client-ip=209.85.128.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="nFohmI7k" Received: by mail-wm1-f45.google.com with SMTP id 5b1f17b1804b1-4368a293339so56602115e9.3 for ; Mon, 30 Dec 2024 18:18:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1735611517; x=1736216317; darn=vger.kernel.org; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=burQHKoOYZby68jA4umEYDoh5VxoEqrj5exShMvyTQI=; b=nFohmI7kK6Y1Hd22UbNqSMrcGNSrni9oGeigdHNVUImCuQQcXjwcL2Vq/eoVbRs1qf YEeNnmIzCIS8nwuPIi4FXZaQhKSfTmBnKL11Uz4UcgkVx2WioPlgeB/nn/kAEj+fQbN8 1iXPN/t7JewkweaHa074zWrwQCcVsh8DMGBQ3jTkmExioSCqHu2KskMOOJLjjmwa8DU3 0vF2B0g6mQpESPNzl4Ow0jjQC0DpJV1A1nTpXiviLxNorJpUOSpmWsCC9dxzVKdu+hVZ /KhggqKDCRjcC0UyvfN0buLv+Nlyl4fzzNqqgp/QeizSQB914mjKu+r1y8R94apzqDIC 8xIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735611517; x=1736216317; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=burQHKoOYZby68jA4umEYDoh5VxoEqrj5exShMvyTQI=; b=Rgp1j3QHK4W8h1E5T9HmatE4qw/nG/x9V0HmOBOJLffb8THfkDDcEDKSQQ0BENzhbG 3dvP/GMYq9QUvD2tyIX1Nh3dAmMSU3Xpfab9Wh8pKGMcmPRbKi14ovtXyvQXmXM/oFoN mPlWMpEBc682wM7QlzcBNYqv/xFvfPy8ob+xdM0/Pb30kVv2U9N9cV3ys5zNJavewsXa tytblgh332mIQ5oKaQm3Zm8Udr0UYdpoVo8049BaSdYNYWZSgNWiVKvo9m3G16aZ40Uk 5VJP5myU+/C/kUVzPeWMb69m6nLp2j+X5ycX9hjSkRJSNPczzXli9znta2nsw8kdu6N2 mKyg== X-Gm-Message-State: AOJu0YzQ4zba4g6FzLB7ArigChOVHtEt3tzWUsyTXag/bu5Pt/do0mTY 3P7SdJzgi+23nYL7VigWWcs4GpokcYIWujOsmGch0yHMW1tfnrDz0T+qQw== X-Gm-Gg: ASbGncsEtJxb2ztlMCoIPbZzaVBZFcO5FnSrFLQYXFG/QfqGz0FXumpmCDfLYFOYb5N a8I8JYiA9Xs79RgBK+XrL7HysgqSR9zZB4b6SXB1tuo+c/ExHU7BAtjwN0p5uYoy8Ih1GCJO1K2 lsy4Ci02qQtq/VHsRw7qtALBgtzBIlcvd4GR1RGj9tp3NNIdKzBuC0pdsZnntDgpekoo679LRg2 HEt76Inx3TNeQgL/1m9JrJk2EV1O8mRER2B3MDxezmVw9Vn8QLUvKzrWQ== X-Google-Smtp-Source: AGHT+IErgA18uUIxxW4K5aZuD2SUvZ+pt4d9c7jqDlGTdHWLv5A8ZYOeq3OSVV7zONawLft8ce6foA== X-Received: by 2002:a05:6000:144b:b0:385:e37a:2a56 with SMTP id ffacd0b85a97d-38a223fd302mr30004874f8f.52.1735611516941; Mon, 30 Dec 2024 18:18:36 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-38a1c828f8fsm31077750f8f.12.2024.12.30.18.18.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 30 Dec 2024 18:18:36 -0800 (PST) Message-Id: <14e94bf04e5ae3895eb61253be9c6d0a0fe56328.1735611513.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 31 Dec 2024 02:18:32 +0000 Subject: [PATCH v2 1/2] gc: add `--expire-to` option Fcc: Sent Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, me@ttaylorr.com, ZheNing Hu , ZheNing Hu From: ZheNing Hu From: ZheNing Hu This commit extends the functionality of `git gc` by adding a new option, `--expire-to=`. Previously, this feature was implemented in `git repack` (see 91badeb), allowing users to specify a directory where unreachable and expired cruft packs are stored during garbage collection. However, users had to run `git repack --cruft --expire-to=` followed by `git prune` to achieve similar results within `git gc`. By introducing `--expire-to=` directly into `git gc`, we simplify the process for users who wish to manage their repository's cleanup more efficiently. This change involves passing the `--expire-to=` parameter through to `git repack`, making it easier for users to set up a backup location for cruft packs that will be pruned. Signed-off-by: ZheNing Hu --- Documentation/git-gc.txt | 6 ++++++ builtin/gc.c | 6 +++++- t/t6500-gc.sh | 6 ++++++ 3 files changed, 17 insertions(+), 1 deletion(-) diff --git a/Documentation/git-gc.txt b/Documentation/git-gc.txt index 370e22faaeb..b4c0cf02972 100644 --- a/Documentation/git-gc.txt +++ b/Documentation/git-gc.txt @@ -69,6 +69,12 @@ be performed as well. the `--max-cruft-size` option of linkgit:git-repack[1] for more. +--expire-to=:: + When packing unreachable objects into a cruft pack, write a cruft + pack containing pruned objects (if any) to the directory ``. + See the `--expire-to` option of linkgit:git-repack[1] for + more. + --prune=:: Prune loose objects older than date (default is 2 weeks ago, overridable by the config variable `gc.pruneExpire`). diff --git a/builtin/gc.c b/builtin/gc.c index d52735354c9..77904694c9f 100644 --- a/builtin/gc.c +++ b/builtin/gc.c @@ -136,6 +136,7 @@ struct gc_config { char *prune_worktrees_expire; char *repack_filter; char *repack_filter_to; + char *repack_expire_to; unsigned long big_pack_threshold; unsigned long max_delta_cache_size; }; @@ -441,6 +442,8 @@ static void add_repack_all_option(struct gc_config *cfg, if (cfg->max_cruft_size) strvec_pushf(&repack, "--max-cruft-size=%lu", cfg->max_cruft_size); + if (cfg->repack_expire_to) + strvec_pushf(&repack, "--expire-to=%s", cfg->repack_expire_to); } else { strvec_push(&repack, "-A"); if (cfg->prune_expire) @@ -675,7 +678,6 @@ struct repository *repo UNUSED) const char *prune_expire_sentinel = "sentinel"; const char *prune_expire_arg = prune_expire_sentinel; int ret; - struct option builtin_gc_options[] = { OPT__QUIET(&quiet, N_("suppress progress reporting")), { OPTION_STRING, 0, "prune", &prune_expire_arg, N_("date"), @@ -694,6 +696,8 @@ struct repository *repo UNUSED) PARSE_OPT_NOCOMPLETE), OPT_BOOL(0, "keep-largest-pack", &keep_largest_pack, N_("repack all other packs except the largest pack")), + OPT_STRING(0, "expire-to", &cfg.repack_expire_to, N_("dir"), + N_("pack prefix to store a pack containing pruned objects")), OPT_END() }; diff --git a/t/t6500-gc.sh b/t/t6500-gc.sh index ee074b99b70..d4b0653a9b7 100755 --- a/t/t6500-gc.sh +++ b/t/t6500-gc.sh @@ -339,6 +339,12 @@ test_expect_success 'gc.maxCruftSize sets appropriate repack options' ' test_subcommand $cruft_max_size_opts --max-cruft-size=3145728 X-Patchwork-Id: 13923591 Received: from mail-wm1-f52.google.com (mail-wm1-f52.google.com [209.85.128.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 391FB13B2B8 for ; Tue, 31 Dec 2024 02:18:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735611521; cv=none; b=DvEyPWC4sfYKRdStzVAU9OXztKAhbxByChQ53v5LdWtlwBuopSXc3byXQvDZmShim7cxIvmQLmTcOyvV9D3KScGZhxXKlskqpn/jsZwbeL+7ftP4jBnJjYWWEpFG6I4gSZaoWDPzzL//3Uy9yC8CYTvymcz22a/ppHAnBVewZdM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735611521; c=relaxed/simple; bh=oV2K3RMAEeTgYxf1BJ0PaJ4cc7u/M5GQ67eKQZgqkW8=; h=Message-Id:In-Reply-To:References:From:Date:Subject:Content-Type: MIME-Version:To:Cc; b=OXA5RPEM7CHbe0hRAQMlq23fxc2bxHV4jFuOXCTwS1cTWtfWHda4JuXfJK8rpMdpyj+YSIntIAqIsUXQLnYr54kD1zie8fJ73zBRX3IFR3LoMnYow7TKK9WaJSnb3odulnAK9rNX4ww65IHOt/VHC6AZ+Q3Rf0Ufn1g7Lokg574= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=i1pEFJsp; arc=none smtp.client-ip=209.85.128.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="i1pEFJsp" Received: by mail-wm1-f52.google.com with SMTP id 5b1f17b1804b1-436281c8a38so69539735e9.3 for ; Mon, 30 Dec 2024 18:18:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1735611518; x=1736216318; darn=vger.kernel.org; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=Q26BvtnsuSsvW6l1iFQ2+lkiaTIpu/8kYVIAVV3ESWo=; b=i1pEFJspZa9vbeG/yVzU2NRn0DnqlIu/OXRVIQ68/bZfVItNB7h2Dr97Wp48+ORIY5 vhcTF2iTcXmbVTRH4KvUs5Pg3U9pd1PvK092Y0nMxHEqImy54uA8H2rCPgbdwVGWZoi3 O3hGDhkesuUGHdsmi31K0ZHaF8319jbfC4adzBbDkfosPLClpqz2J1T7Cn0LjBuwA3Z9 K9pK8/iQtmfPFVHuRnPG4xI6E0DBynuvbRKm8BLmth+ZFs8UmQmH5YKXPokqjgBoN74u 2+DtmvEl3PeqUsENEAKOc+qyhFsEsHlIObdywxuSVKislwaotlMYmmWuTGtih0nBo2iC Zenw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735611518; x=1736216318; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Q26BvtnsuSsvW6l1iFQ2+lkiaTIpu/8kYVIAVV3ESWo=; b=Kfh8VxVGgjI1wuK3m18WMXXcwITIlzGllLFAHPxk1/x6s9XKIB/FZTlqS+rcMEH2l7 QlWq1/Ua6dP3/ftZ9RZzvGAmVf597Lta6UPR3GWYKDGJwTW5wImZYCckDDTOhLqiMSAF GedspdpfZXX+S879vmRWsdaJNg6Oj39Vyzi445LHjhhvxPYP1MK3LFEHMr+5Qtd9Sy8/ CiK87XUzMBO7zbBJrKFLYeFPnxCL/0bIe0m1m15Vfh3FHNu/st9WsPrz14pUMWtXtiGS 8ncZ5vWwU8AyXWe9N5uaDNwZjXnATQt53LV9cpJBHxYhdECG0oF5S1jPd+Sd3Btz049V ddZg== X-Gm-Message-State: AOJu0Yw0gM+MiU7qJlHmGqvpwUH5NkYfv3FbrunDH1cDcSH8bmTeayww hbbIJoplHFQMcwTf4d/O5Gio1haCXnTqUIzhAYAEQCEtTEL+/wPHAqVBQQ== X-Gm-Gg: ASbGncuyNm16fwyb7Da7Bc9QitgDP+gsn5QqU5JCE6s4tKKBX8EmyJxLiIWKf3fbNvO SP4Q6kbxzz0e5x68QVk7Qivn1DoxUrXOk9cupcM76Wk6Nv1rxwQlHOlKUCRWAZMIImfDcO7B9j8 LJa7xf9TDgI9Y3/+H57wSg5zS+OrM/Scj4z2PnZuX63dE1gxjitXt5flA37nBdwq1OzN8uDJnoJ dtNArmSgkWg1mSXsSMma/RicPwz+ne5dRpsLuPXCDKy8QUJZLd+NPW2zg== X-Google-Smtp-Source: AGHT+IGw8rXP8ip6aa7iJ52uAJ9Z+icd1FI7yOkaNty/P7qsbllnEpWvjObxN19MYzUJSclsbE2IaA== X-Received: by 2002:a05:600c:45cd:b0:434:effb:9f8a with SMTP id 5b1f17b1804b1-4366864619dmr341031415e9.15.1735611518022; Mon, 30 Dec 2024 18:18:38 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-43656af6c66sm415788365e9.5.2024.12.30.18.18.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 30 Dec 2024 18:18:37 -0800 (PST) Message-Id: <579757957d21faaa8dd9228a191d82f663e93c03.1735611513.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 31 Dec 2024 02:18:33 +0000 Subject: [PATCH v2 2/2] fix(gc): make --prune=now compatible with --expire-to Fcc: Sent Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, me@ttaylorr.com, ZheNing Hu , ZheNing Hu From: ZheNing Hu From: ZheNing Hu The original `git gc --prune=now` attempted to delete all unreachable objects. However, after the introduction of `--cruft` and `--expire-to=` in git gc, `--prune=now` can now compress unreachable objects into a cruft pack and store them in the specified instead of deleting them directly. This is beneficial for recovery in case of data corruption during repository GC. Therefore, update the handling logic of `--prune=now` in gc so that `-a` parameter is only passed to the repack command when neither `--cruft` nor `--expire-to` are used. Signed-off-by: ZheNing Hu --- builtin/gc.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/builtin/gc.c b/builtin/gc.c index 77904694c9f..8656e1caff0 100644 --- a/builtin/gc.c +++ b/builtin/gc.c @@ -433,7 +433,8 @@ static int keep_one_pack(struct string_list_item *item, void *data UNUSED) static void add_repack_all_option(struct gc_config *cfg, struct string_list *keep_pack) { - if (cfg->prune_expire && !strcmp(cfg->prune_expire, "now")) + if (cfg->prune_expire && !strcmp(cfg->prune_expire, "now") + && !(cfg->cruft_packs && cfg->repack_expire_to)) strvec_push(&repack, "-a"); else if (cfg->cruft_packs) { strvec_push(&repack, "--cruft");