[0/4] repack: implement `--expire-to` option

Message

Taylor Blau Oct. 24, 2022, 6:42 p.m. UTC

This series polishes off the RFC [1] I sent over the summer teaching
`git repack` how to move pruned objects into a separate cruft pack in a
different repository.

We've been using this series at GitHub for some time now as an
escape-hatch mechanism for quickly recovering pruned objects that run
afoul of the "advertise-then-prune" race described in [2].

The general idea is that we write out a second cruft pack to capture the
pruned objects, with two minor changes:

  - The second cruft pack counts the first cruft pack as "saved",
    meaning that we'll exclude its objects (thus ignoring both reachable
    and unreachable-but-recent objects from the second cruft pack).

  - The second cruft pack does *not* exclude objects based on age (i.e.,
    it acts as if `--cruft-expiration=never` were given) so as to avoid
    excluding objects which are being pruned due to their age.

The first few patches are preparatory, and the substantive change is
implemented in the fourth patch. I would suggest that reviewers start
there, and then read the first three patches.

Since this series basically boils down to adding one new conditional,
like:

    if (delete_redundant && expire_to) {
        ret = write_cruft_pack(...):
        if (ret)
            return ret;
    }

The (second) call to `write_cruft_pack()` comes with a rather extensive
inline comment explaining what is going on ;-).

Thanks in advance for your review.

[1]: https://lore.kernel.org/git/[e-mail]/
[2]: https://lore.kernel.org/git/[e-mail]/

Taylor Blau (4):
  builtin/repack.c: pass "out" to `prepare_pack_objects`
  builtin/repack.c: pass "cruft_expiration" to `write_cruft_pack`
  builtin/repack.c: write cruft packs to arbitrary locations
  builtin/repack.c: implement `--expire-to` for storing pruned objects

 Documentation/git-repack.txt |   6 ++
 builtin/repack.c             |  67 ++++++++++++++++---
 t/t7700-repack.sh            | 121 +++++++++++++++++++++++++++++++++++
 3 files changed, 186 insertions(+), 8 deletions(-)