From patchwork Tue Apr 14 04:04:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 11486693 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7679F912 for ; Tue, 14 Apr 2020 04:04:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5D11F206E9 for ; Tue, 14 Apr 2020 04:04:09 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20150623.gappssmtp.com header.i=@ttaylorr-com.20150623.gappssmtp.com header.b="VaOOsS1t" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2405063AbgDNEEI (ORCPT ); Tue, 14 Apr 2020 00:04:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54462 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1726058AbgDNEEH (ORCPT ); Tue, 14 Apr 2020 00:04:07 -0400 Received: from mail-pf1-x442.google.com (mail-pf1-x442.google.com [IPv6:2607:f8b0:4864:20::442]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 111F1C0A3BDC for ; Mon, 13 Apr 2020 21:04:07 -0700 (PDT) Received: by mail-pf1-x442.google.com with SMTP id c138so5517603pfc.0 for ; Mon, 13 Apr 2020 21:04:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=Ud+kuMoXbePDAxE/+dZegJ8NGHZwbj/msbUcBnU5f6g=; b=VaOOsS1t1WXh2CusZ7FopSgOkRqyiEcANmD5TFvl9FyhMtBFQDT0T1k0l6bCohltLl y9rGOAxqNWDHxzsv/vptawfeCi1Oi64BUII1qBMoo3ZQPe9RxJHlhS900reyV39uNSyh skG73qHx92JU945A31Ldk636SvI5ZpGh6HLYu1gV3at8wZgkPeZj+gDxtCG/M1AbvPWz RbKIKrhAHKQCJ+mf/whM1vdii7zHrzWMi1x5JTHbfboRPYCRy91YGR6BekTJVNwYLD7q ftk5TwoHSv3RbZ8iOXgU1GV8wnZte2x+ketcL1WtLkjZWp3drscl6a8vyvysi2YdSYrS uNjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=Ud+kuMoXbePDAxE/+dZegJ8NGHZwbj/msbUcBnU5f6g=; b=FF4R2xe9pmND5m1ySWW854sJDUMe2UMH9f4rWr2vJoYZVpObsI06dbNSejKkSTmiTs dkjAptVl0q6T090k2ezEEslXDGnZxjR//3EZuO4MxUIrmtfkVgluLWH96KzwADEJulG8 wYblTl6FvKIUOJfZHodkCqG8o3os8+VUwKWbNgKhxzsIe2PEDsuqHzW9dXZsHwKOU8GO hDkpkuOwzHtAeiM30rSHFzkgZ3oWbrPIMGdroOFLjPFHe6pXlxPWHMJXJwl7fLXjv2YT uglpdQApEuEq6vypNSE/SoWT0TvWoEVSLnagy/56YhrZYwjfJpjJoMpfDVZpu7Nub9wq UzTQ== X-Gm-Message-State: AGi0PuYdEHZCNKu6RQxew1LAXqfuhm+Lqj8lwLiwGhedP0vJmD8Glh9J EwWRKOA6Ds4iwOM65Hetzx6/L5+DV1d6rHcS X-Google-Smtp-Source: APiQypLndCWikIIhAOMDaYnzKfExvjBoCMl7uLgJiJ/7V5P/I5Jqh3PU2ppDWiq6wsbAyQDkV7z7lw== X-Received: by 2002:a63:6a84:: with SMTP id f126mr17748445pgc.14.1586837046211; Mon, 13 Apr 2020 21:04:06 -0700 (PDT) Received: from localhost ([8.44.146.30]) by smtp.gmail.com with ESMTPSA id y126sm4670468pgy.91.2020.04.13.21.04.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 13 Apr 2020 21:04:05 -0700 (PDT) Date: Mon, 13 Apr 2020 22:04:04 -0600 From: Taylor Blau To: git@vger.kernel.org Cc: dstolee@microsoft.com, gitster@pobox.com, martin.agren@gmail.com, peff@peff.net, szeder.dev@gmail.com Subject: [PATCH 1/7] t/helper/test-read-graph.c: support commit-graph chains Message-ID: <0eb2c610283689675450b1e4e2c66147f1500cf1.1586836700.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In 61df89c8e5 (commit-graph: don't early exit(1) on e.g. "git status", 2019-03-25), the former 'load_commit_graph_one' was refactored into 'open_commit_graph' and 'load_commit_graph_one_fd_st' as a means of avoiding an early-exit from non-library code. However, 'load_commit_graph_one' does not support commit-graph chains, and hence the 'read-graph' test tool does not work with them. Replace 'load_commit_graph_one' with 'read_commit_graph_one' in order to support commit-graph chains. In the spirit of 61df89c8e5, 'read_commit_graph_one' does not ever 'die()', making it a suitable replacement here. Signed-off-by: Taylor Blau --- t/helper/test-read-graph.c | 13 +------------ 1 file changed, 1 insertion(+), 12 deletions(-) diff --git a/t/helper/test-read-graph.c b/t/helper/test-read-graph.c index f8a461767c..4846040363 100644 --- a/t/helper/test-read-graph.c +++ b/t/helper/test-read-graph.c @@ -7,26 +7,15 @@ int cmd__read_graph(int argc, const char **argv) { struct commit_graph *graph = NULL; - char *graph_name; - int open_ok; - int fd; - struct stat st; struct object_directory *odb; setup_git_directory(); odb = the_repository->objects->odb; - graph_name = get_commit_graph_filename(odb); - - open_ok = open_commit_graph(graph_name, &fd, &st); - if (!open_ok) - die_errno(_("Could not open commit-graph '%s'"), graph_name); - - graph = load_commit_graph_one_fd_st(fd, &st, odb); + graph = read_commit_graph_one(the_repository, odb); if (!graph) return 1; - FREE_AND_NULL(graph_name); printf("header: %08x %d %d %d %d\n", ntohl(*(uint32_t*)graph->data), From patchwork Tue Apr 14 04:04:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 11486695 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0DECD81 for ; Tue, 14 Apr 2020 04:04:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E5F3B20737 for ; Tue, 14 Apr 2020 04:04:16 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20150623.gappssmtp.com header.i=@ttaylorr-com.20150623.gappssmtp.com header.b="WsH7/9Z4" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2405067AbgDNEEP (ORCPT ); Tue, 14 Apr 2020 00:04:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54474 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1726058AbgDNEEN (ORCPT ); Tue, 14 Apr 2020 00:04:13 -0400 Received: from mail-pj1-x1043.google.com (mail-pj1-x1043.google.com [IPv6:2607:f8b0:4864:20::1043]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D575BC0A3BDC for ; Mon, 13 Apr 2020 21:04:11 -0700 (PDT) Received: by mail-pj1-x1043.google.com with SMTP id z9so4684180pjd.2 for ; Mon, 13 Apr 2020 21:04:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=pnmstrq41vvkwasQvJo9GppKN9vte6zAsQ24x/vm4nQ=; b=WsH7/9Z4i1vVES0OsDVOmvJ3FpYioR0Ji/nt2PKMKqYQdyWIImkpq9AaoIeBhY2j61 JS8RIBKiJmM66a45gdCWwxtGxZZ9i+JSEb2m4fd+/Q1D/yjjIxDjo4pBVLc62MmKKPGb UBVq0IsL/HoLFnN1v1SdPo0NHkXjgxdENIuU/RLIZaieNGr5jF79p4efQSjsL8xI5d6j bMvSKzmxjhZy6hImT6e8y8CCSJUy9SVU2SGrLUuVPJdrqPy+kHOHrG7UUXnz0tkhZDib TSFxxk2I1Gn5bSQJlHOLfE3QIGDs32FzhRNGU7hQHWnwMwTMoMBJIJPnUNmw11hLnXYL WgBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=pnmstrq41vvkwasQvJo9GppKN9vte6zAsQ24x/vm4nQ=; b=GSiQIMCcX9N2mgJlSBRMQ59ZMizhPQ7nX+ekRBpMJu7ddNmYXwKTbAvQ5L8H0ufKTC pESQ8XlMC55bo96pZe0vWGqspSXpnX7X8HpFviHhQLwnWuZZ2HRtNBgUI7uGDD/+Nl5X /B2y6gCcj+nqBfW7+3vCLNhW9Y4LC3+/z4pGZ3GqbXuWoAftR3MS3Qcn0GLU/Gm+9wxC YIo2GxaFr816RjWv6TPbKJLOhG8nfGpo4UfghWI1qk0ZuekmV3aHrsRgXSzO1MfmDGh8 gda2Yo1fN6cqdBg+xzjQMse8g6JYJ6xmxFl4zgdaDASBGvGjJv/3E2be1DIQY+A/vJ4w nZqA== X-Gm-Message-State: AGi0PuauAbLehkQNQmSafzgvFIq1eg6Io5lM9Q2zBkQOKeIHw5Q37DB8 c/x2twXdEY+kYT6YpNs/tXMhHOiV88eD0Cuf X-Google-Smtp-Source: APiQypL9rZUrm5DCWIT/4pF3rZFUboK16xSZClu2S1WwwGziqMQvuS9Tj5bxeIQjENE07PpTH7xr0w== X-Received: by 2002:a17:90a:30e7:: with SMTP id h94mr23583661pjb.186.1586837050487; Mon, 13 Apr 2020 21:04:10 -0700 (PDT) Received: from localhost ([8.44.146.30]) by smtp.gmail.com with ESMTPSA id x4sm9971526pfa.191.2020.04.13.21.04.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 13 Apr 2020 21:04:09 -0700 (PDT) Date: Mon, 13 Apr 2020 22:04:08 -0600 From: Taylor Blau To: git@vger.kernel.org Cc: dstolee@microsoft.com, gitster@pobox.com, martin.agren@gmail.com, peff@peff.net, szeder.dev@gmail.com Subject: [PATCH 2/7] builtin/commit-graph.c: support for '--split[=]' Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org With '--split', the commit-graph machinery writes new commits in another incremental commit-graph which is part of the existing chain, and optionally decides to condense the chain into a single commit-graph. This is done to ensure that the asymptotic behavior of looking up a commit in an incremental chain is not dominated by the number of incrementals in that chain. It can be controlled by the '--max-commits' and '--size-multiple' options. In the next two commits, we will introduce additional splitting strategies that can exert additional control over: - when a split commit-graph is and isn't written, and - when the existing commit-graph chain is discarded completely and replaced with another graph To prepare for this, make '--split' take an optional strategy (as in '--split[=]'), and add a new enum to describe which strategy is being used. For now, no strategies are given, and the only enumerated value is 'COMMIT_GRAPH_SPLIT_UNSPECIFIED', indicating the absence of a strategy. Signed-off-by: Taylor Blau --- Documentation/git-commit-graph.txt | 11 ++++++----- builtin/commit-graph.c | 26 ++++++++++++++++++++++---- commit-graph.h | 5 +++++ 3 files changed, 33 insertions(+), 9 deletions(-) diff --git a/Documentation/git-commit-graph.txt b/Documentation/git-commit-graph.txt index 28d1fee505..10d757c5cc 100644 --- a/Documentation/git-commit-graph.txt +++ b/Documentation/git-commit-graph.txt @@ -57,11 +57,12 @@ or `--stdin-packs`.) With the `--append` option, include all commits that are present in the existing commit-graph file. + -With the `--split` option, write the commit-graph as a chain of multiple -commit-graph files stored in `/info/commit-graphs`. The new commits -not already in the commit-graph are added in a new "tip" file. This file -is merged with the existing file if the following merge conditions are -met: +With the `--split[=]` option, write the commit-graph as a +chain of multiple commit-graph files stored in +`/info/commit-graphs`. Commit-graph layers are merged based on the +strategy and other splitting options. The new commits not already in the +commit-graph are added in a new "tip" file. This file is merged with the +existing file if the following merge conditions are met: + * If `--size-multiple=` is not specified, let `X` equal 2. If the new tip file would have `N` commits and the previous tip has `M` commits and diff --git a/builtin/commit-graph.c b/builtin/commit-graph.c index d1ab6625f6..342094fc77 100644 --- a/builtin/commit-graph.c +++ b/builtin/commit-graph.c @@ -9,7 +9,9 @@ static char const * const builtin_commit_graph_usage[] = { N_("git commit-graph verify [--object-dir ] [--shallow] [--[no-]progress]"), - N_("git commit-graph write [--object-dir ] [--append|--split] [--reachable|--stdin-packs|--stdin-commits] [--[no-]progress] "), + N_("git commit-graph write [--object-dir ] [--append] " + "[--split[=]] [--reachable|--stdin-packs|--stdin-commits] " + "[--[no-]progress] "), NULL }; @@ -19,7 +21,9 @@ static const char * const builtin_commit_graph_verify_usage[] = { }; static const char * const builtin_commit_graph_write_usage[] = { - N_("git commit-graph write [--object-dir ] [--append|--split] [--reachable|--stdin-packs|--stdin-commits] [--[no-]progress] "), + N_("git commit-graph write [--object-dir ] [--append] " + "[--split[=]] [--reachable|--stdin-packs|--stdin-commits] " + "[--[no-]progress] "), NULL }; @@ -114,6 +118,18 @@ static int graph_verify(int argc, const char **argv) extern int read_replace_refs; static struct split_commit_graph_opts split_opts; +static int write_option_parse_split(const struct option *opt, const char *arg, + int unset) +{ + opts.split = 1; + if (!arg) + return 0; + + die(_("unrecognized --split argument, %s"), arg); + + return 0; +} + static int graph_write(int argc, const char **argv) { struct string_list *pack_indexes = NULL; @@ -136,8 +152,10 @@ static int graph_write(int argc, const char **argv) OPT_BOOL(0, "append", &opts.append, N_("include all commits already in the commit-graph file")), OPT_BOOL(0, "progress", &opts.progress, N_("force progress reporting")), - OPT_BOOL(0, "split", &opts.split, - N_("allow writing an incremental commit-graph file")), + OPT_CALLBACK_F(0, "split", &split_opts.flags, NULL, + N_("allow writing an incremental commit-graph file"), + PARSE_OPT_OPTARG | PARSE_OPT_NONEG, + write_option_parse_split), OPT_INTEGER(0, "max-commits", &split_opts.max_commits, N_("maximum number of commits in a non-base split commit-graph")), OPT_INTEGER(0, "size-multiple", &split_opts.size_multiple, diff --git a/commit-graph.h b/commit-graph.h index e87a6f6360..e799008ff4 100644 --- a/commit-graph.h +++ b/commit-graph.h @@ -82,10 +82,15 @@ enum commit_graph_write_flags { COMMIT_GRAPH_WRITE_CHECK_OIDS = (1 << 3) }; +enum commit_graph_split_flags { + COMMIT_GRAPH_SPLIT_UNSPECIFIED = 0 +}; + struct split_commit_graph_opts { int size_multiple; int max_commits; timestamp_t expire_time; + enum commit_graph_split_flags flags; }; /* From patchwork Tue Apr 14 04:04:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 11486697 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E1F6F912 for ; Tue, 14 Apr 2020 04:04:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C4106206D5 for ; Tue, 14 Apr 2020 04:04:18 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20150623.gappssmtp.com header.i=@ttaylorr-com.20150623.gappssmtp.com header.b="w9vNubAW" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2405069AbgDNEER (ORCPT ); Tue, 14 Apr 2020 00:04:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54486 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S2405066AbgDNEEP (ORCPT ); Tue, 14 Apr 2020 00:04:15 -0400 Received: from mail-pl1-x644.google.com (mail-pl1-x644.google.com [IPv6:2607:f8b0:4864:20::644]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1465DC0A3BDC for ; Mon, 13 Apr 2020 21:04:15 -0700 (PDT) Received: by mail-pl1-x644.google.com with SMTP id v2so4186323plp.9 for ; Mon, 13 Apr 2020 21:04:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=ar0e0n4L/LwFcRhj8LwES+O7fGOMSRI95yDZn/Cd+Os=; b=w9vNubAW+BoiAtG2/qjdMemnX8xfcPfrnJ0PLFEA60/fjzBnxQ1qHpaItq+PFPF7nC HeuvXi2TizmZNTz8TthCrnncbXuwxUfJacwFX062aZ7VcRgFYdXy1O5cxDFfpNDCZvdN z144IX1QQj1e9JX7x0Y9D9+TNgLFedakK9pCWz6wbPjQ5hUNczLzQoE5amdksT/XfG9o FlKXsEX4I6rsrtmrBbC4/zP9MpQj+4eERb63INd0t2wt4+vN6lW5cdPa8qgVsEcFAoLK gGbL2oQI/ysfo5eAAcFjTLrJY2o5m6wb9Z4LHzGl0ETEAnNiJqsnzeezAlxq3GAfOBql dF8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=ar0e0n4L/LwFcRhj8LwES+O7fGOMSRI95yDZn/Cd+Os=; b=QLCNTtCxJ6EcSXUZZ+AKU9miePRVq9DKMFUDJqbHveM5MB389ilxRUxTctPm5ronSL c+yvvJyocjbwO1ihwo38Nx6lAOxo5ynbBVcVoqa/5oJ1z9oBYfWNUM46kGxhNsfj7/b7 9/NS9zH+koCzTiCNf/8by3fstCAEqdUjzDVonjs9bKjZXKOuyvo5IP0x48VsmqEIO6Tk ggJbzN+cnZFLHkpk5/S/K5xGJ+7D2LeARz9t0NuacWWsZUQN0SxYIfB7IGpydoQbGebB gm77TQSNx5RQ+Rq41DyKlOpQYMCo+is0/WQVpQlm8WP6re2o27iHGlCsI2gp+B3KPxq8 6TDg== X-Gm-Message-State: AGi0PuaxtbjSo/2abXwNccrZFztwd6MOtpWixG2U9dccKPRSHrx/fFJA 3Zhm/f9G5K1GhXm14pBEqNS1sAmUnXRcLX3a X-Google-Smtp-Source: APiQypLRh5PE5JBPpcJDCuec9F7g60w0xyT0NMABNk68955N4yauB3DBqhih41L5ZyQsoHIEMJMWtw== X-Received: by 2002:a17:902:eb13:: with SMTP id l19mr20397898plb.156.1586837054180; Mon, 13 Apr 2020 21:04:14 -0700 (PDT) Received: from localhost ([8.44.146.30]) by smtp.gmail.com with ESMTPSA id x26sm1485094pfo.218.2020.04.13.21.04.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 13 Apr 2020 21:04:13 -0700 (PDT) Date: Mon, 13 Apr 2020 22:04:12 -0600 From: Taylor Blau To: git@vger.kernel.org Cc: dstolee@microsoft.com, gitster@pobox.com, martin.agren@gmail.com, peff@peff.net, szeder.dev@gmail.com Subject: [PATCH 3/7] builtin/commit-graph.c: introduce split strategy 'no-merge' Message-ID: <2741858596ce9cd48aa0e3d126310435166d37ab.1586836700.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In the previous commit, we laid the groundwork for supporting different splitting strategies. In this commit, we introduce the first splitting strategy: 'no-merge'. Passing '--split=no-merge' is useful for callers which wish to write a new incremental commit-graph, but do not want to spend effort condensing the incremental chain [1]. Previously, this was possible by passing '--size-multiple=0', but this no longer the case following 63020f175f (commit-graph: prefer default size_mult when given zero, 2020-01-02). When '--split=no-merge' is given, the commit-graph machinery will never condense an existing chain, and it will always write a new incremental. [1]: This might occur when, for example, a server administrator running some program after each push may want to ensure that each job runs proportional in time to the size of the push, and does not "jump" when the commit-graph machinery decides to trigger a merge. Signed-off-by: Taylor Blau --- Documentation/git-commit-graph.txt | 5 +++++ builtin/commit-graph.c | 7 ++++++- commit-graph.c | 19 ++++++++++++------- commit-graph.h | 3 ++- t/t5324-split-commit-graph.sh | 11 +++++++++++ 5 files changed, 36 insertions(+), 9 deletions(-) diff --git a/Documentation/git-commit-graph.txt b/Documentation/git-commit-graph.txt index 10d757c5cc..a4c4a641e5 100644 --- a/Documentation/git-commit-graph.txt +++ b/Documentation/git-commit-graph.txt @@ -63,6 +63,11 @@ chain of multiple commit-graph files stored in strategy and other splitting options. The new commits not already in the commit-graph are added in a new "tip" file. This file is merged with the existing file if the following merge conditions are met: +* If `--split=no-merge` is specified, a merge is never performed, and +the remaining options are ignored. A bare `--split` defers to the +remaining options. (Note that merging a chain of commit graphs replaces +the existing chain with a length-1 chain where the first and only +incremental holds the entire graph). + * If `--size-multiple=` is not specified, let `X` equal 2. If the new tip file would have `N` commits and the previous tip has `M` commits and diff --git a/builtin/commit-graph.c b/builtin/commit-graph.c index 342094fc77..42182ed71c 100644 --- a/builtin/commit-graph.c +++ b/builtin/commit-graph.c @@ -121,11 +121,16 @@ static struct split_commit_graph_opts split_opts; static int write_option_parse_split(const struct option *opt, const char *arg, int unset) { + enum commit_graph_split_flags *flags = opt->value; + opts.split = 1; if (!arg) return 0; - die(_("unrecognized --split argument, %s"), arg); + if (!strcmp(arg, "no-merge")) + *flags = COMMIT_GRAPH_SPLIT_MERGE_PROHIBITED; + else + die(_("unrecognized --split argument, %s"), arg); return 0; } diff --git a/commit-graph.c b/commit-graph.c index f013a84e29..af3fe20bb5 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -1529,6 +1529,7 @@ static void split_graph_merge_strategy(struct write_commit_graph_context *ctx) { struct commit_graph *g; uint32_t num_commits; + enum commit_graph_split_flags flags = COMMIT_GRAPH_SPLIT_UNSPECIFIED; uint32_t i; int max_commits = 0; @@ -1539,21 +1540,25 @@ static void split_graph_merge_strategy(struct write_commit_graph_context *ctx) if (ctx->split_opts->size_multiple) size_mult = ctx->split_opts->size_multiple; + + flags = ctx->split_opts->flags; } g = ctx->r->objects->commit_graph; num_commits = ctx->commits.nr; ctx->num_commit_graphs_after = ctx->num_commit_graphs_before + 1; - while (g && (g->num_commits <= size_mult * num_commits || - (max_commits && num_commits > max_commits))) { - if (g->odb != ctx->odb) - break; + if (flags != COMMIT_GRAPH_SPLIT_MERGE_PROHIBITED) { + while (g && (g->num_commits <= size_mult * num_commits || + (max_commits && num_commits > max_commits))) { + if (g->odb != ctx->odb) + break; - num_commits += g->num_commits; - g = g->base_graph; + num_commits += g->num_commits; + g = g->base_graph; - ctx->num_commit_graphs_after--; + ctx->num_commit_graphs_after--; + } } ctx->new_base_graph = g; diff --git a/commit-graph.h b/commit-graph.h index e799008ff4..8752afb88d 100644 --- a/commit-graph.h +++ b/commit-graph.h @@ -83,7 +83,8 @@ enum commit_graph_write_flags { }; enum commit_graph_split_flags { - COMMIT_GRAPH_SPLIT_UNSPECIFIED = 0 + COMMIT_GRAPH_SPLIT_UNSPECIFIED = 0, + COMMIT_GRAPH_SPLIT_MERGE_PROHIBITED = 1 }; struct split_commit_graph_opts { diff --git a/t/t5324-split-commit-graph.sh b/t/t5324-split-commit-graph.sh index 53b2e6b455..1438d63f24 100755 --- a/t/t5324-split-commit-graph.sh +++ b/t/t5324-split-commit-graph.sh @@ -351,4 +351,15 @@ test_expect_success 'split across alternate where alternate is not split' ' test_cmp commit-graph .git/objects/info/commit-graph ' +test_expect_success '--split=no-merge always writes an incremental' ' + test_when_finished rm -rf a b && + rm -rf $graphdir $infodir/commit-graph && + git reset --hard commits/2 && + git rev-list HEAD~1 >a && + git rev-list HEAD >b && + git commit-graph write --split --stdin-commits X-Patchwork-Id: 11486699 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 409C6912 for ; Tue, 14 Apr 2020 04:04:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1FE0F20737 for ; Tue, 14 Apr 2020 04:04:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20150623.gappssmtp.com header.i=@ttaylorr-com.20150623.gappssmtp.com header.b="Z8VogUl/" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2405072AbgDNEEY (ORCPT ); Tue, 14 Apr 2020 00:04:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54502 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S2405066AbgDNEEV (ORCPT ); Tue, 14 Apr 2020 00:04:21 -0400 Received: from mail-pg1-x541.google.com (mail-pg1-x541.google.com [IPv6:2607:f8b0:4864:20::541]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DEA3FC0A3BDC for ; Mon, 13 Apr 2020 21:04:20 -0700 (PDT) Received: by mail-pg1-x541.google.com with SMTP id c5so5440958pgi.7 for ; Mon, 13 Apr 2020 21:04:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=Zduc4neDurObiXp22bggDoN5FRJTrfoEwdnk8oyGWBQ=; b=Z8VogUl/7KnPhtr45NxYvrAVmqB5hFZctvG6SWFzP9HJcm6BBvYNHlUslP0MJdIo/b oAuVvgEEQ7molyu2s3eIdLWNT5tqUOrnS54O/hev6PNFJf+kfns+WieMFBJ+zYhOHf4G Taku5zsc1+VvV97pZhfHqGFtwKnJOqyP2WvPLj+JZJgxXTETEilThTIBiWEO+SQYLjxZ ZzJpPoyq1Rbksl+7YVJqYazQBuYoENgOKMjM3/UJF7xb/8mI/jeMAUrEJu+8B9J7/yyE CDELfdvgkNGDdOBZttsX4z7Hc98UkmzqvZZnqjwbRFWTuOBEklgsZ0dqzziJ+A8QcMwY mwjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=Zduc4neDurObiXp22bggDoN5FRJTrfoEwdnk8oyGWBQ=; b=HsTLIstpfHGTC6sKLmGzZNH6wg2zDB3y1ag4533HPD15wdxNBBxSY2ZskiIPwYVFj2 LaEMY9vTqIDXVWVxjykL8Vo8pDhCxqt4obAyCxewNt5bnbgdRCgiXLqDZnz1t1WZjGVp R5x3EyTtt9cC+Hxkbqa7Gkf7XWvDVfXX/XLU178lZp1Njm+4f5Huaoz+flbUcl5bhZVl Oi0L2hsz0zAc9NKw6VbZZpF7KfAjUwfVpc/sZO5Xg5wgt7WrjICLQbYlNzEaTRcxrrIK cFhNot3MH0KiOHt+laOHSj3zdi2mNjgQNYilVBcDhaZorReJFKumzKXWIviDcvSmgYM0 sa0Q== X-Gm-Message-State: AGi0PuZydfdEdIhpupLgAXTrG4+QsdEwfOU14xt7B0nOK1kOO6TPnihX /Rluc+cudO5NfPPHZpjUzRb6JMqXJQqN3MFn X-Google-Smtp-Source: APiQypL5eKZ8EfDVd4oQFPLktDH98B35PTNzWqCEnRCuZqp7Siw0aW6fGGPm7cICK152zl5JQo2MXw== X-Received: by 2002:a63:6c83:: with SMTP id h125mr9484153pgc.168.1586837059667; Mon, 13 Apr 2020 21:04:19 -0700 (PDT) Received: from localhost ([8.44.146.30]) by smtp.gmail.com with ESMTPSA id h2sm10024287pfr.220.2020.04.13.21.04.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 13 Apr 2020 21:04:18 -0700 (PDT) Date: Mon, 13 Apr 2020 22:04:17 -0600 From: Taylor Blau To: git@vger.kernel.org Cc: dstolee@microsoft.com, gitster@pobox.com, martin.agren@gmail.com, peff@peff.net, szeder.dev@gmail.com Subject: [PATCH 4/7] builtin/commit-graph.c: introduce split strategy 'replace' Message-ID: <50d2e005ac685ad771e2450222b52474fd3cbde4.1586836700.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When using split commit-graphs, it is sometimes useful to completely replace the commit-graph chain with a new base. For example, consider a scenario in which a repository builds a new commit-graph incremental for each push. Occasionally (say, after some fixed number of pushes), they may wish to rebuild the commit-graph chain with all reachable commits. They can do so with $ git commit-graph write --reachable but this removes the chain entirely and replaces it with a single commit-graph in 'objects/info/commit-graph'. Unfortunately, this means that the next push will have to move this commit-graph into the first layer of a new chain, and then write its new commits on top. Avoid such copying entirely by allowing the caller to specify that they wish to replace the entirety of their commit-graph chain, while also specifying that the new commit-graph should become the basis of a fresh, length-one chain. This addresses the above situation by making it possible for the caller to instead write: $ git commit-graph write --reachable --split=replace which writes a new length-one chain to 'objects/info/commit-graphs', making the commit-graph incremental generated by the subsequent push relatively cheap by avoiding the aforementioned copy. In order to do this, remove an assumption in 'write_commit_graph_file' that chains are always at least two incrementals long. Signed-off-by: Taylor Blau --- Documentation/git-commit-graph.txt | 7 ++-- builtin/commit-graph.c | 2 ++ commit-graph.c | 53 ++++++++++++++++++++++-------- commit-graph.h | 3 +- t/t5324-split-commit-graph.sh | 19 +++++++++++ 5 files changed, 66 insertions(+), 18 deletions(-) diff --git a/Documentation/git-commit-graph.txt b/Documentation/git-commit-graph.txt index a4c4a641e5..46f7f7c573 100644 --- a/Documentation/git-commit-graph.txt +++ b/Documentation/git-commit-graph.txt @@ -64,9 +64,10 @@ strategy and other splitting options. The new commits not already in the commit-graph are added in a new "tip" file. This file is merged with the existing file if the following merge conditions are met: * If `--split=no-merge` is specified, a merge is never performed, and -the remaining options are ignored. A bare `--split` defers to the -remaining options. (Note that merging a chain of commit graphs replaces -the existing chain with a length-1 chain where the first and only +the remaining options are ignored. `--split=replace` overwrites the +existing chain with a new one. A bare `--split` defers to the remaining +options. (Note that merging a chain of commit graphs replaces the +existing chain with a length-1 chain where the first and only incremental holds the entire graph). + * If `--size-multiple=` is not specified, let `X` equal 2. If the new diff --git a/builtin/commit-graph.c b/builtin/commit-graph.c index 42182ed71c..922f876bfa 100644 --- a/builtin/commit-graph.c +++ b/builtin/commit-graph.c @@ -129,6 +129,8 @@ static int write_option_parse_split(const struct option *opt, const char *arg, if (!strcmp(arg, "no-merge")) *flags = COMMIT_GRAPH_SPLIT_MERGE_PROHIBITED; + else if (!strcmp(arg, "replace")) + *flags = COMMIT_GRAPH_SPLIT_REPLACE; else die(_("unrecognized --split argument, %s"), arg); diff --git a/commit-graph.c b/commit-graph.c index af3fe20bb5..c598508d7f 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -866,7 +866,7 @@ static void write_graph_chunk_data(struct hashfile *f, int hash_len, if (edge_value >= 0) edge_value += ctx->new_num_commits_in_base; - else { + else if (ctx->new_base_graph) { uint32_t pos; if (find_commit_in_graph(parent->item, ctx->new_base_graph, @@ -897,7 +897,7 @@ static void write_graph_chunk_data(struct hashfile *f, int hash_len, if (edge_value >= 0) edge_value += ctx->new_num_commits_in_base; - else { + else if (ctx->new_base_graph) { uint32_t pos; if (find_commit_in_graph(parent->item, ctx->new_base_graph, @@ -964,7 +964,7 @@ static void write_graph_chunk_extra_edges(struct hashfile *f, if (edge_value >= 0) edge_value += ctx->new_num_commits_in_base; - else { + else if (ctx->new_base_graph) { uint32_t pos; if (find_commit_in_graph(parent->item, ctx->new_base_graph, @@ -1037,6 +1037,8 @@ static void close_reachable(struct write_commit_graph_context *ctx) { int i; struct commit *commit; + enum commit_graph_split_flags flags = ctx->split_opts ? + ctx->split_opts->flags : COMMIT_GRAPH_SPLIT_UNSPECIFIED; if (ctx->report_progress) ctx->progress = start_delayed_progress( @@ -1066,8 +1068,9 @@ static void close_reachable(struct write_commit_graph_context *ctx) if (!commit) continue; if (ctx->split) { - if (!parse_commit(commit) && - commit->graph_pos == COMMIT_NOT_FROM_GRAPH) + if ((!parse_commit(commit) && + commit->graph_pos == COMMIT_NOT_FROM_GRAPH) || + flags == COMMIT_GRAPH_SPLIT_REPLACE) add_missing_parents(ctx, commit); } else if (!parse_commit_no_graph(commit)) add_missing_parents(ctx, commit); @@ -1287,6 +1290,8 @@ static uint32_t count_distinct_commits(struct write_commit_graph_context *ctx) static void copy_oids_to_commits(struct write_commit_graph_context *ctx) { uint32_t i; + enum commit_graph_split_flags flags = ctx->split_opts ? + ctx->split_opts->flags : COMMIT_GRAPH_SPLIT_UNSPECIFIED; ctx->num_extra_edges = 0; if (ctx->report_progress) @@ -1303,11 +1308,14 @@ static void copy_oids_to_commits(struct write_commit_graph_context *ctx) ALLOC_GROW(ctx->commits.list, ctx->commits.nr + 1, ctx->commits.alloc); ctx->commits.list[ctx->commits.nr] = lookup_commit(ctx->r, &ctx->oids.list[i]); - if (ctx->split && + if (ctx->split && flags != COMMIT_GRAPH_SPLIT_REPLACE && ctx->commits.list[ctx->commits.nr]->graph_pos != COMMIT_NOT_FROM_GRAPH) continue; - parse_commit_no_graph(ctx->commits.list[ctx->commits.nr]); + if (ctx->split && flags == COMMIT_GRAPH_SPLIT_REPLACE) + parse_commit(ctx->commits.list[ctx->commits.nr]); + else + parse_commit_no_graph(ctx->commits.list[ctx->commits.nr]); num_parents = commit_list_count(ctx->commits.list[ctx->commits.nr]->parents); if (num_parents > 2) @@ -1488,8 +1496,12 @@ static int write_commit_graph_file(struct write_commit_graph_context *ctx) } if (ctx->base_graph_name) { - const char *dest = ctx->commit_graph_filenames_after[ - ctx->num_commit_graphs_after - 2]; + const char *dest; + int idx = ctx->num_commit_graphs_after - 1; + if (ctx->num_commit_graphs_after > 1) + idx--; + + dest = ctx->commit_graph_filenames_after[idx]; if (strcmp(ctx->base_graph_name, dest)) { result = rename(ctx->base_graph_name, dest); @@ -1546,9 +1558,13 @@ static void split_graph_merge_strategy(struct write_commit_graph_context *ctx) g = ctx->r->objects->commit_graph; num_commits = ctx->commits.nr; - ctx->num_commit_graphs_after = ctx->num_commit_graphs_before + 1; + if (flags == COMMIT_GRAPH_SPLIT_REPLACE) + ctx->num_commit_graphs_after = 1; + else + ctx->num_commit_graphs_after = ctx->num_commit_graphs_before + 1; - if (flags != COMMIT_GRAPH_SPLIT_MERGE_PROHIBITED) { + if (flags != COMMIT_GRAPH_SPLIT_MERGE_PROHIBITED && + flags != COMMIT_GRAPH_SPLIT_REPLACE) { while (g && (g->num_commits <= size_mult * num_commits || (max_commits && num_commits > max_commits))) { if (g->odb != ctx->odb) @@ -1561,7 +1577,11 @@ static void split_graph_merge_strategy(struct write_commit_graph_context *ctx) } } - ctx->new_base_graph = g; + if (flags != COMMIT_GRAPH_SPLIT_REPLACE) + ctx->new_base_graph = g; + else if (ctx->num_commit_graphs_after != 1) + BUG("split_graph_merge_strategy: num_commit_graphs_after " + "should be 1 with --split=replace"); if (ctx->num_commit_graphs_after == 2) { char *old_graph_name = get_commit_graph_filename(g->odb); @@ -1768,6 +1788,7 @@ int write_commit_graph(struct object_directory *odb, struct write_commit_graph_context *ctx; uint32_t i, count_distinct = 0; int res = 0; + int replace = 0; if (!commit_graph_compatible(the_repository)) return 0; @@ -1802,6 +1823,9 @@ int write_commit_graph(struct object_directory *odb, g = g->base_graph; } } + + if (ctx->split_opts) + replace = ctx->split_opts->flags & COMMIT_GRAPH_SPLIT_REPLACE; } ctx->approx_nr_objects = approximate_object_count(); @@ -1862,13 +1886,14 @@ int write_commit_graph(struct object_directory *odb, goto cleanup; } - if (!ctx->commits.nr) + if (!ctx->commits.nr && !replace) goto cleanup; if (ctx->split) { split_graph_merge_strategy(ctx); - merge_commit_graphs(ctx); + if (!replace) + merge_commit_graphs(ctx); } else ctx->num_commit_graphs_after = 1; diff --git a/commit-graph.h b/commit-graph.h index 8752afb88d..718433d79b 100644 --- a/commit-graph.h +++ b/commit-graph.h @@ -84,7 +84,8 @@ enum commit_graph_write_flags { enum commit_graph_split_flags { COMMIT_GRAPH_SPLIT_UNSPECIFIED = 0, - COMMIT_GRAPH_SPLIT_MERGE_PROHIBITED = 1 + COMMIT_GRAPH_SPLIT_MERGE_PROHIBITED = 1, + COMMIT_GRAPH_SPLIT_REPLACE = 2 }; struct split_commit_graph_opts { diff --git a/t/t5324-split-commit-graph.sh b/t/t5324-split-commit-graph.sh index 1438d63f24..e5d8d64170 100755 --- a/t/t5324-split-commit-graph.sh +++ b/t/t5324-split-commit-graph.sh @@ -362,4 +362,23 @@ test_expect_success '--split=no-merge always writes an incremental' ' test_line_count = 2 $graphdir/commit-graph-chain ' +test_expect_success '--split=replace replaces the chain' ' + rm -rf $graphdir $infodir/commit-graph && + git reset --hard commits/3 && + git rev-list -1 HEAD~2 >a && + git rev-list -1 HEAD~1 >b && + git rev-list -1 HEAD >c && + git commit-graph write --split=no-merge --stdin-commits graph-files && + test_line_count = 1 graph-files && + verify_chain_files_exist $graphdir && + graph_read_expect 2 +' + test_done From patchwork Tue Apr 14 04:04:22 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 11486701 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B0AE081 for ; Tue, 14 Apr 2020 04:04:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 98A4C206D5 for ; Tue, 14 Apr 2020 04:04:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20150623.gappssmtp.com header.i=@ttaylorr-com.20150623.gappssmtp.com header.b="vnNWnOLA" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2405075AbgDNEE0 (ORCPT ); Tue, 14 Apr 2020 00:04:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54514 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S2405073AbgDNEEZ (ORCPT ); Tue, 14 Apr 2020 00:04:25 -0400 Received: from mail-pl1-x643.google.com (mail-pl1-x643.google.com [IPv6:2607:f8b0:4864:20::643]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ABC25C0A3BDC for ; Mon, 13 Apr 2020 21:04:24 -0700 (PDT) Received: by mail-pl1-x643.google.com with SMTP id h11so4182452plr.11 for ; Mon, 13 Apr 2020 21:04:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=p04prpl1Jzam3H9T+aaVfeoigWnxDl1R7+i/o5tR4aA=; b=vnNWnOLA0b/k/TuJeSzCZ6MCEFR8aW22B8YzMnyrD2EH4nWLGmuFg8Dgn1s+rSlgeN Z6HMSbKbexnMEwZgaDqlee1/mnitutEvWeebef416518kVkOQYsD7H+mDZK3PruF3Dxn XteyRapF8zsCHr1KfJ4b5/SPdH1WRalanvod+thrJtTqZ7ewHCyp6r8o4cM6Q2FVbjlX MP0jmGSq/w9+SbM3Zes5DLAfYInv6fypiqOEEtCL6+3medoux17FHea897g+ToAf1Qb6 aqu88y4xxlh5l4a2OrjpOTvA7zngabF1WpaUGyLbqcHo8jhOeJUzmaxf+p+XJVAWf1nW C3zQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=p04prpl1Jzam3H9T+aaVfeoigWnxDl1R7+i/o5tR4aA=; b=pDfU4y2HSCgz0g5UzbXllLHqwHa5xC/nFxYe615xL+emE8TZRxDsvEXs6/+9zl+2SE kHJTKjttdmnPxuIyooIDbPTLYmw2E6hBTOGYCpO3fFN3O1I+jGlbwcm6qcKswItF4w1k vo+cWwowfl17Sar7+6DZrrUkPfqLIivKkEHvvjBFGQuo00FFpy+SQtpGqry2iuBhZRwd mcFOVbpmfunuvPFC3TwU2vQ1OWITKveV/NUSsAc5IpEjIP7Fvm7gLCyOprvx7c1WJ0fj a+kUsqb4jF8uqpIYk/fpF/NAtgKBV569dZvQbCJCJCCksai/U0nEtkR5+9pHUq9/7DvM hx+Q== X-Gm-Message-State: AGi0PuatX6mrT/7tpV9E0N//f7QNILHK7G/kJjivCSGfmwUbtXRs69ca 7T/W+JSsTqnU8qsBvR5iWwn4r+BU1ojTKKcu X-Google-Smtp-Source: APiQypL6BjudZtA4n7Ud2hOfprhemdfrSeMGp2YuK/qMWjjzeu2Sn5lQ/w62ApPLd/Jz5WvAH8VeCA== X-Received: by 2002:a17:90a:8c10:: with SMTP id a16mr16478567pjo.78.1586837063843; Mon, 13 Apr 2020 21:04:23 -0700 (PDT) Received: from localhost ([8.44.146.30]) by smtp.gmail.com with ESMTPSA id q201sm10063759pfq.32.2020.04.13.21.04.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 13 Apr 2020 21:04:23 -0700 (PDT) Date: Mon, 13 Apr 2020 22:04:22 -0600 From: Taylor Blau To: git@vger.kernel.org Cc: dstolee@microsoft.com, gitster@pobox.com, martin.agren@gmail.com, peff@peff.net, szeder.dev@gmail.com Subject: [PATCH 5/7] oidset: introduce 'oidset_size' Message-ID: <4a9798ba6f3aab081dff5e40e1d91568cf4ba23b.1586836700.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Occasionally, it may be useful for callers to know the number of object IDs in an oidset. Right now, the only way to compute this is to call 'kh_size' on the internal 'kh_set_oid_t'. Similar to how we wrap other 'kh_*' functions over the 'oidset' type, let's allow callers to compute this value by introducing 'oidset_size'. We will add its first caller in the subsequent commit. Signed-off-by: Taylor Blau --- oidset.c | 5 +++++ oidset.h | 5 +++++ 2 files changed, 10 insertions(+) diff --git a/oidset.c b/oidset.c index f63ce818f6..15d4e18c37 100644 --- a/oidset.c +++ b/oidset.c @@ -36,6 +36,11 @@ void oidset_clear(struct oidset *set) oidset_init(set, 0); } +int oidset_size(struct oidset *set) +{ + return kh_size(&set->set); +} + void oidset_parse_file(struct oidset *set, const char *path) { FILE *fp; diff --git a/oidset.h b/oidset.h index d8a106b127..8261c36b91 100644 --- a/oidset.h +++ b/oidset.h @@ -54,6 +54,11 @@ int oidset_insert(struct oidset *set, const struct object_id *oid); */ int oidset_remove(struct oidset *set, const struct object_id *oid); +/** + * Returns the number of oids in the set. + */ +int oidset_size(struct oidset *set); + /** * Remove all entries from the oidset, freeing any resources associated with * it. From patchwork Tue Apr 14 04:04:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 11486703 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DF41C912 for ; Tue, 14 Apr 2020 04:04:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C233B206D5 for ; Tue, 14 Apr 2020 04:04:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20150623.gappssmtp.com header.i=@ttaylorr-com.20150623.gappssmtp.com header.b="0cM3m0WB" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2405078AbgDNEE3 (ORCPT ); Tue, 14 Apr 2020 00:04:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54526 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S2405073AbgDNEE2 (ORCPT ); Tue, 14 Apr 2020 00:04:28 -0400 Received: from mail-pl1-x641.google.com (mail-pl1-x641.google.com [IPv6:2607:f8b0:4864:20::641]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5F098C0A3BDC for ; Mon, 13 Apr 2020 21:04:28 -0700 (PDT) Received: by mail-pl1-x641.google.com with SMTP id t4so4178959plq.12 for ; Mon, 13 Apr 2020 21:04:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=nfIRM5MvACeZd3GmE3VOfNjO9AsWS/C+JtoP8T4WkgQ=; b=0cM3m0WBKlK7F7OVC1H73NmLJYK+jkn25c7oMUApdlxx5hrFBFBmpdx6WBjdlaydPS 3lzuvN0UmEp5qyBAc4Szbw2P1U1QqefeGHBPvzfw9LGvoU/D7/DUalU+/DpjMwzh5HWn wcQfW5X5om7QV4M+6+QpswfIEItzD3vWMxOpQaJSTJjM/AKXktrh/I/cVGRRxPx16u7S O6u0KhBoWZk81yaCM/USpGaS65jV+haYemDKhM+2woSlwbD+IdMOfzV3nUbkG/8nMnJw cf3kozxtxfGTEmqXIEuY0m4KHAXh0b2rf5rEOqTq8vWNMUktiiE1qJs6NSBNvg9VWYW2 nPhQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=nfIRM5MvACeZd3GmE3VOfNjO9AsWS/C+JtoP8T4WkgQ=; b=b2Pkc9zekPcMPjJ+bKP+lGLLwhnzPT5LeisLEXpD+Lrhtwcby2eqNXQ9hB5twbWodk 0MP8/q271F41k17fWbhOb06lfoabNQBJTQaFqe1Nfl/MsAnzzHqI8EJANBElsZOeKf/E Tnn/+S1LEa3xeMav9Q0wmhC9okhrVedKgJiNgXEbeKntg3LoEsmsKM5ndtKBKc6FIFjr lJyh3OPy6kJEaimWNfdbPhLLdeIP4Mo8oYjl7kn7lnoLUDRXTjg/iMBQzxgrpJu7VRVA m/jkyr+4PXqF29ibvBjLUSaXYXhc/eQYXCQTFZ8bXgqZUmmdcLnebU4g9yZKOaTrfa1r 6htg== X-Gm-Message-State: AGi0PuYj58ARiRwrEYdeyAyXq0N3EacRjJxX5RTh1h4n7qj8ZjFQL0M5 R2Imrlw23gpOQ2JY4q0RWejk2NgV0GD6DNXG X-Google-Smtp-Source: APiQypK1kmxaBgtUmfeaZTfB/sqL62n4g9iC7SRlKLkXlPjbeNvhqfplHxjG60Aa4XugtnpE5VQQug== X-Received: by 2002:a17:90a:868b:: with SMTP id p11mr26610718pjn.34.1586837067396; Mon, 13 Apr 2020 21:04:27 -0700 (PDT) Received: from localhost ([8.44.146.30]) by smtp.gmail.com with ESMTPSA id b189sm1207755pfb.163.2020.04.13.21.04.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 13 Apr 2020 21:04:26 -0700 (PDT) Date: Mon, 13 Apr 2020 22:04:25 -0600 From: Taylor Blau To: git@vger.kernel.org Cc: dstolee@microsoft.com, gitster@pobox.com, martin.agren@gmail.com, peff@peff.net, szeder.dev@gmail.com Subject: [PATCH 6/7] commit-graph.h: replace 'commit_hex' with 'commits' Message-ID: <811a2ece05f2d576bf2b3d6429c537800649b27b.1586836700.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org The 'write_commit_graph()' function takes in either a string list of pack indices, or a string list of hexadecimal commit OIDs. These correspond to the '--stdin-packs' and '--stdin-commits' mode(s) from 'git commit-graph write'. Using a string_list of hexadecimal commit IDs is not the most efficient use of memory, since we can instead use the 'struct oidset', which is more well-suited for this case. This has another benefit which will become apparent in the following commit. This is that we are about to disambiguate the kinds of errors we produce with '--stdin-commits' into "non-hex input" and "hex-input, but referring to a non-commit object". By having 'write_commit_graph' take in a 'struct oidset *' of commits, we place the burden on the caller (in this case, the builtin) to handle the first case, and the commit-graph machinery can handle the second case. Suggested-by: Jeff King Signed-off-by: Taylor Blau --- builtin/commit-graph.c | 19 ++++++++++--- commit-graph.c | 59 +++++++++++++++++++++++------------------ commit-graph.h | 3 ++- t/t5318-commit-graph.sh | 2 +- 4 files changed, 52 insertions(+), 31 deletions(-) diff --git a/builtin/commit-graph.c b/builtin/commit-graph.c index 922f876bfa..c69716aa7e 100644 --- a/builtin/commit-graph.c +++ b/builtin/commit-graph.c @@ -140,7 +140,7 @@ static int write_option_parse_split(const struct option *opt, const char *arg, static int graph_write(int argc, const char **argv) { struct string_list *pack_indexes = NULL; - struct string_list *commit_hex = NULL; + struct oidset commits = OIDSET_INIT; struct object_directory *odb = NULL; struct string_list lines; int result = 0; @@ -213,7 +213,20 @@ static int graph_write(int argc, const char **argv) if (opts.stdin_packs) pack_indexes = &lines; if (opts.stdin_commits) { - commit_hex = &lines; + struct string_list_item *item; + oidset_init(&commits, lines.nr); + for_each_string_list_item(item, &lines) { + struct object_id oid; + const char *end; + + if (parse_oid_hex(item->string, &oid, &end)) { + error(_("unexpected non-hex object ID: " + "%s"), item->string); + return 1; + } + + oidset_insert(&commits, &oid); + } flags |= COMMIT_GRAPH_WRITE_CHECK_OIDS; } @@ -222,7 +235,7 @@ static int graph_write(int argc, const char **argv) if (write_commit_graph(odb, pack_indexes, - commit_hex, + opts.stdin_commits ? &commits : NULL, flags, &split_opts)) result = 1; diff --git a/commit-graph.c b/commit-graph.c index c598508d7f..f60346baee 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -1136,13 +1136,13 @@ static void compute_generation_numbers(struct write_commit_graph_context *ctx) stop_progress(&ctx->progress); } -static int add_ref_to_list(const char *refname, - const struct object_id *oid, - int flags, void *cb_data) +static int add_ref_to_set(const char *refname, + const struct object_id *oid, + int flags, void *cb_data) { - struct string_list *list = (struct string_list *)cb_data; + struct oidset *commits = (struct oidset *)cb_data; - string_list_append(list, oid_to_hex(oid)); + oidset_insert(commits, oid); return 0; } @@ -1150,14 +1150,14 @@ int write_commit_graph_reachable(struct object_directory *odb, enum commit_graph_write_flags flags, const struct split_commit_graph_opts *split_opts) { - struct string_list list = STRING_LIST_INIT_DUP; + struct oidset commits = OIDSET_INIT; int result; - for_each_ref(add_ref_to_list, &list); - result = write_commit_graph(odb, NULL, &list, + for_each_ref(add_ref_to_set, &commits); + result = write_commit_graph(odb, NULL, &commits, flags, split_opts); - string_list_clear(&list, 0); + oidset_clear(&commits); return result; } @@ -1206,39 +1206,46 @@ static int fill_oids_from_packs(struct write_commit_graph_context *ctx, return 0; } -static int fill_oids_from_commit_hex(struct write_commit_graph_context *ctx, - struct string_list *commit_hex) +static int fill_oids_from_commits(struct write_commit_graph_context *ctx, + struct oidset *commits) { - uint32_t i; + uint32_t i = 0; struct strbuf progress_title = STRBUF_INIT; + struct oidset_iter iter; + struct object_id *oid; + + if (!oidset_size(commits)) + return 0; if (ctx->report_progress) { strbuf_addf(&progress_title, Q_("Finding commits for commit graph from %d ref", "Finding commits for commit graph from %d refs", - commit_hex->nr), - commit_hex->nr); + oidset_size(commits)), + oidset_size(commits)); ctx->progress = start_delayed_progress( progress_title.buf, - commit_hex->nr); + oidset_size(commits)); } - for (i = 0; i < commit_hex->nr; i++) { - const char *end; - struct object_id oid; + + oidset_iter_init(commits, &iter); + while ((oid = oidset_iter_next(&iter))) { struct commit *result; - display_progress(ctx->progress, i + 1); - if (!parse_oid_hex(commit_hex->items[i].string, &oid, &end) && - (result = lookup_commit_reference_gently(ctx->r, &oid, 1))) { + display_progress(ctx->progress, ++i); + + result = lookup_commit_reference_gently(ctx->r, oid, 1); + if (result) { ALLOC_GROW(ctx->oids.list, ctx->oids.nr + 1, ctx->oids.alloc); oidcpy(&ctx->oids.list[ctx->oids.nr], &(result->object.oid)); ctx->oids.nr++; } else if (ctx->check_oids) { error(_("invalid commit object id: %s"), - commit_hex->items[i].string); + oid_to_hex(oid)); return -1; } } + stop_progress(&ctx->progress); strbuf_release(&progress_title); @@ -1781,7 +1788,7 @@ static void expire_commit_graphs(struct write_commit_graph_context *ctx) int write_commit_graph(struct object_directory *odb, struct string_list *pack_indexes, - struct string_list *commit_hex, + struct oidset *commits, enum commit_graph_write_flags flags, const struct split_commit_graph_opts *split_opts) { @@ -1857,12 +1864,12 @@ int write_commit_graph(struct object_directory *odb, goto cleanup; } - if (commit_hex) { - if ((res = fill_oids_from_commit_hex(ctx, commit_hex))) + if (commits) { + if ((res = fill_oids_from_commits(ctx, commits))) goto cleanup; } - if (!pack_indexes && !commit_hex) + if (!pack_indexes && !commits) fill_oids_from_all_packs(ctx); close_reachable(ctx); diff --git a/commit-graph.h b/commit-graph.h index 718433d79b..98ef121924 100644 --- a/commit-graph.h +++ b/commit-graph.h @@ -6,6 +6,7 @@ #include "string-list.h" #include "cache.h" #include "object-store.h" +#include "oidset.h" #define GIT_TEST_COMMIT_GRAPH "GIT_TEST_COMMIT_GRAPH" #define GIT_TEST_COMMIT_GRAPH_DIE_ON_LOAD "GIT_TEST_COMMIT_GRAPH_DIE_ON_LOAD" @@ -106,7 +107,7 @@ int write_commit_graph_reachable(struct object_directory *odb, const struct split_commit_graph_opts *split_opts); int write_commit_graph(struct object_directory *odb, struct string_list *pack_indexes, - struct string_list *commit_hex, + struct oidset *commits, enum commit_graph_write_flags flags, const struct split_commit_graph_opts *split_opts); diff --git a/t/t5318-commit-graph.sh b/t/t5318-commit-graph.sh index 9bf920ae17..e874a12696 100755 --- a/t/t5318-commit-graph.sh +++ b/t/t5318-commit-graph.sh @@ -43,7 +43,7 @@ test_expect_success 'create commits and repack' ' test_expect_success 'exit with correct error on bad input to --stdin-commits' ' cd "$TRASH_DIRECTORY/full" && echo HEAD | test_expect_code 1 git commit-graph write --stdin-commits 2>stderr && - test_i18ngrep "invalid commit object id" stderr && + test_i18ngrep "unexpected non-hex object ID: HEAD" stderr && # valid tree OID, but not a commit OID git rev-parse HEAD^{tree} | test_expect_code 1 git commit-graph write --stdin-commits 2>stderr && test_i18ngrep "invalid commit object id" stderr From patchwork Tue Apr 14 04:04:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 11486705 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9A0A3912 for ; Tue, 14 Apr 2020 04:04:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7CDF7206D5 for ; Tue, 14 Apr 2020 04:04:34 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20150623.gappssmtp.com header.i=@ttaylorr-com.20150623.gappssmtp.com header.b="eXRQ6LoS" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2405081AbgDNEEd (ORCPT ); Tue, 14 Apr 2020 00:04:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54538 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S2405073AbgDNEEc (ORCPT ); Tue, 14 Apr 2020 00:04:32 -0400 Received: from mail-pf1-x443.google.com (mail-pf1-x443.google.com [IPv6:2607:f8b0:4864:20::443]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 55F96C0A3BDC for ; Mon, 13 Apr 2020 21:04:32 -0700 (PDT) Received: by mail-pf1-x443.google.com with SMTP id k15so5503904pfh.6 for ; Mon, 13 Apr 2020 21:04:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=Aut8rMJV0jp/BNl6D8qmcJnjFg6A4MxBxy2OkAM1n24=; b=eXRQ6LoSL1SPK/Job0BfTE+nOHijKwRZpIinxYdRbhzR5CApydH++6rdcr/fNedsxJ 8Cx+YEmH+adhewC/0uCdPVU/m9C63FMxR1J3ilKT+KEkmBFN321xsTWctKnLIgSu8cEA wwVkEfGAX6RLrCBtJR3H/QYZCSlZguZZsTZzdczMks3REjeak4p+sC/xRk7FEZbf70Jc XoQfYPCgjvw7n6wPpQmmzMBLI3hiG2zWTt+SI2UqVplPtAHUT6PJe6NWvyBZbtaO79CR uoOF9Ba3mlQvsdgWXigOwdqe/XvoldwlrT8QAsXrlcp8RAsv44AP1BGxHG+fpeelzwcC MDsA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=Aut8rMJV0jp/BNl6D8qmcJnjFg6A4MxBxy2OkAM1n24=; b=d0oa0VurlvHPViizjPeeQxEorbgo5MyfjuTuZyFZDjqpnx3tGF9ilob+7eTkrrJBCf s8iF0Puuyn8jtPsmM1L35y/AuxwzgNgp6V4L5DMNA1LZa9ZAsld/6b41sDep04HOaJf2 ZTkSzvZE4+AZapbHiH2qz6rFRvWbm13fsEGi3DG7KrCZArBuxMXQBezg+nUhJFYPntHT bw7uYAC7fFjcrPROmZaP3VygXXJ8YRU23o0huvoOJNbB47ECvruPbT0YmCOkzXwqAD82 TgT51Wm2iLKMJ0Mc3doVd0yTbmlxEz7bEsLmGlG1Lw3ZyD9EjlvuzW8+9moVmgA8qzFX bW+g== X-Gm-Message-State: AGi0PuZ8rJbINqgzBIO7kASUU5YK11rhgPiUrCh3B2QMSZZsPXLiecvX +JbA+56uq2GP7fglB80s4+rdff6jBJz+KPIu X-Google-Smtp-Source: APiQypKjIuKXEmwHPjMRriH+6usqf+AEtZHanj3o74zdrsn90nMI89G/RnwQvVJsX6Z1VDg/gLvNEg== X-Received: by 2002:a63:296:: with SMTP id 144mr20017590pgc.110.1586837071373; Mon, 13 Apr 2020 21:04:31 -0700 (PDT) Received: from localhost ([8.44.146.30]) by smtp.gmail.com with ESMTPSA id x186sm9872502pfb.151.2020.04.13.21.04.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 13 Apr 2020 21:04:30 -0700 (PDT) Date: Mon, 13 Apr 2020 22:04:29 -0600 From: Taylor Blau To: git@vger.kernel.org Cc: dstolee@microsoft.com, gitster@pobox.com, martin.agren@gmail.com, peff@peff.net, szeder.dev@gmail.com Subject: [PATCH 7/7] commit-graph.c: introduce '--[no-]check-oids' Message-ID: <1ff42f4c3d568dd25889d2808cda3edf38a36cb9.1586836700.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When operating on a stream of commit OIDs on stdin, 'git commit-graph write' checks that each OID refers to an object that is indeed a commit. This is convenient to make sure that the given input is well-formed, but can sometimes be undesirable. For example, server operators may wish to feed the refnames that were updated during a push to 'git commit-graph write --input=stdin-commits', and silently discard refs that don't point at commits. This can be done by combing the output of 'git for-each-ref' with '--format %(*objecttype)', but this requires opening up a potentially large number of objects. Instead, it is more convenient to feed the updated refs to the commit-graph machinery, and let it throw out refs that don't point to commits. Introduce '--[no-]check-oids' to make such a behavior possible. With '--check-oids' (the default behavior to retain backwards compatibility), 'git commit-graph write' will barf on a non-commit line in its input. With 'no-check-oids', such lines will be silently ignored, making the above possible by specifying this option. No matter which is supplied, 'git commit-graph write' retains the behavior from the previous commit of rejecting non-OID inputs like "HEAD" and "refs/heads/foo" as before. Signed-off-by: Taylor Blau Signed-off-by: Taylor Blau Signed-off-by: Taylor Blau Signed-off-by: Taylor Blau --- Documentation/git-commit-graph.txt | 5 +++++ builtin/commit-graph.c | 11 ++++++++--- t/t5318-commit-graph.sh | 28 ++++++++++++++++++++++++++++ 3 files changed, 41 insertions(+), 3 deletions(-) diff --git a/Documentation/git-commit-graph.txt b/Documentation/git-commit-graph.txt index 46f7f7c573..91e8027b86 100644 --- a/Documentation/git-commit-graph.txt +++ b/Documentation/git-commit-graph.txt @@ -82,6 +82,11 @@ tip with the previous tip. Finally, if `--expire-time=` is not specified, let `datetime` be the current time. After writing the split commit-graph, delete all unused commit-graph whose modified times are older than `datetime`. ++ +The `--[no-]check-oids` option decides whether or not OIDs are required +to be commits. By default, `--check-oids` is implied, generating an +error on non-commit objects. If `--no-check-oids` is given, non-commits +are silently discarded. 'verify':: diff --git a/builtin/commit-graph.c b/builtin/commit-graph.c index c69716aa7e..2d0a8e822a 100644 --- a/builtin/commit-graph.c +++ b/builtin/commit-graph.c @@ -11,7 +11,7 @@ static char const * const builtin_commit_graph_usage[] = { N_("git commit-graph verify [--object-dir ] [--shallow] [--[no-]progress]"), N_("git commit-graph write [--object-dir ] [--append] " "[--split[=]] [--reachable|--stdin-packs|--stdin-commits] " - "[--[no-]progress] "), + "[--[no-]progress] [--[no-]check-oids] "), NULL }; @@ -23,7 +23,7 @@ static const char * const builtin_commit_graph_verify_usage[] = { static const char * const builtin_commit_graph_write_usage[] = { N_("git commit-graph write [--object-dir ] [--append] " "[--split[=]] [--reachable|--stdin-packs|--stdin-commits] " - "[--[no-]progress] "), + "[--[no-]progress] [--[no-]check-oids] "), NULL }; @@ -36,6 +36,7 @@ static struct opts_commit_graph { int split; int shallow; int progress; + int check_oids; } opts; static struct object_directory *find_odb(struct repository *r, @@ -163,6 +164,8 @@ static int graph_write(int argc, const char **argv) N_("allow writing an incremental commit-graph file"), PARSE_OPT_OPTARG | PARSE_OPT_NONEG, write_option_parse_split), + OPT_BOOL(0, "check-oids", &opts.check_oids, + N_("require OIDs to be commits")), OPT_INTEGER(0, "max-commits", &split_opts.max_commits, N_("maximum number of commits in a non-base split commit-graph")), OPT_INTEGER(0, "size-multiple", &split_opts.size_multiple, @@ -173,6 +176,7 @@ static int graph_write(int argc, const char **argv) }; opts.progress = isatty(2); + opts.check_oids = 1; split_opts.size_multiple = 2; split_opts.max_commits = 0; split_opts.expire_time = 0; @@ -227,7 +231,8 @@ static int graph_write(int argc, const char **argv) oidset_insert(&commits, &oid); } - flags |= COMMIT_GRAPH_WRITE_CHECK_OIDS; + if (opts.check_oids) + flags |= COMMIT_GRAPH_WRITE_CHECK_OIDS; } UNLEAK(buf); diff --git a/t/t5318-commit-graph.sh b/t/t5318-commit-graph.sh index e874a12696..2cbd301abe 100755 --- a/t/t5318-commit-graph.sh +++ b/t/t5318-commit-graph.sh @@ -49,6 +49,34 @@ test_expect_success 'exit with correct error on bad input to --stdin-commits' ' test_i18ngrep "invalid commit object id" stderr ' +graph_expect_commits() { + test-tool read-graph >got + if ! grep "num_commits: $1" got + then + echo "graph_expect_commits: expected $1 commit(s), got:" + cat got + false + fi +} + +test_expect_success 'ignores non-commit OIDs to --input=stdin-commits with --no-check-oids' ' + test_when_finished rm -rf "$objdir/info/commit-graph" && + cd "$TRASH_DIRECTORY/full" && + # write a graph to ensure layers are/are not added appropriately + git rev-parse HEAD~1 >base && + git commit-graph write --stdin-commits bad && + test_expect_code 1 git commit-graph write --stdin-commits err && + test_i18ngrep "invalid commit object id" err && + graph_expect_commits 2 && + # update with valid commit OID, ignore tree OID + git rev-parse HEAD HEAD^{tree} >in && + git commit-graph write --stdin-commits --no-check-oids output git -c core.commitGraph=false $1 >expect