From patchwork Sat Apr 10 02:03:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12195299 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9C48AC43462 for ; Sat, 10 Apr 2021 02:04:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 77D5961105 for ; Sat, 10 Apr 2021 02:04:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233827AbhDJCEO (ORCPT ); Fri, 9 Apr 2021 22:04:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38406 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232796AbhDJCEL (ORCPT ); Fri, 9 Apr 2021 22:04:11 -0400 Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 86E83C061765 for ; Fri, 9 Apr 2021 19:03:57 -0700 (PDT) Received: by mail-wm1-x334.google.com with SMTP id n11-20020a05600c4f8bb029010e5cf86347so6085372wmq.1 for ; Fri, 09 Apr 2021 19:03:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=Llo0Of7xrxXb6L4qe3wjXKvb7OepjL9FntDDDajZ5Mw=; b=bfSy4e8C7QUX2KgsMixc2BGMKIsKmd0bn3Ff6HL4h725zawcxvVNPPn2l9lPwXfP0s vOO5FHrXEwZcUr+5fFgOt3U27zZyqq4CwwT4tqNYs7jLFMazwgyMJ6yKzHRKL2i9abY1 HDA6HY9CkMV3guZU/lrLPGD89hmvj1FtMg4CWdDpxKAJF++OECBeTAzq2CqBZybrE3tr +kEieXf9+JLxdcBbuMkkhpQizho9yc53UyNFxw2YUo03kPyC8aCkLJWtRkTO2ZGNsBYH rqqwlfx42Y71EXDvt/Pt9Qz9lMxqnH4c5wRNXTlCPKlPvY9a1oWaFUTFi5CDQJdVwGAu BFuA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=Llo0Of7xrxXb6L4qe3wjXKvb7OepjL9FntDDDajZ5Mw=; b=cAHCBqk/zAyt7yMmTxwBAHAzCuD0sVCgVLvPDvkS6/HWPrrd0/MDzd5VcNxbmu8csI pMLy+2xv9w4xaTt8SvvDMfzHj1mDx9r6zn8ZI9nm0GWnyH6ngUu2zZfJJ/SmFchIuSdz ZpWXPImDMkx5xYskJ0M0jVrvkO8ARAqeQVRRqbxlVWJz31KyGZlFcsOGFXA4ujHLhHO7 oky5mILgA8aETzqgEiyS2Ghd0pq2pPJX6irWwbDku/0rWFGE8BxXP8qeI9nnsYS2yjSe /CqqmivX8SP8bh83xXZcMn83dK3Y7/KhH1I14GdFGWHUgk4cUtBIqOeiogsxNsCq8QzY qfLA== X-Gm-Message-State: AOAM531V9bf8ZFu3b9fHEH0yxJr5KWHlbymXW8E+JEt9e+oNuylDaVLD GCJv748lLp4j9sfF2S0G+oTOV6vY8Go= X-Google-Smtp-Source: ABdhPJwXDpSyP3G4UDySc1bMqM5rCD8qOddf3G/sA2nvzS57uqEcBIViqn0/uYSPKXF0E+pHWJJOJQ== X-Received: by 2002:a05:600c:21d1:: with SMTP id x17mr16081963wmj.3.1618020227680; Fri, 09 Apr 2021 19:03:47 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id v14sm6883348wrd.48.2021.04.09.19.03.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 09 Apr 2021 19:03:47 -0700 (PDT) Message-Id: <4c0e983ba56f030851370ef71887117db45e8479.1618020225.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Sat, 10 Apr 2021 02:03:43 +0000 Subject: [PATCH v3 1/3] maintenance: simplify prefetch logic Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: tom.saeger@oracle.com, gitster@pobox.com, sunshine@sunshineco.com, Derrick Stolee , Josh Steadmon , Emily Shaffer , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee The previous logic filled a string list with the names of each remote, but instead we could simply run the appropriate 'git fetch' data directly in the remote iterator. Do this for reduced code size, but also because it sets up an upcoming change to use the remote's refspec. This data is accessible from the 'struct remote' data that is now accessible in fetch_remote(). Signed-off-by: Derrick Stolee --- builtin/gc.c | 33 ++++++++------------------------- 1 file changed, 8 insertions(+), 25 deletions(-) diff --git a/builtin/gc.c b/builtin/gc.c index ef7226d7bca4..fa8128de9ae1 100644 --- a/builtin/gc.c +++ b/builtin/gc.c @@ -873,55 +873,38 @@ static int maintenance_task_commit_graph(struct maintenance_run_opts *opts) return 0; } -static int fetch_remote(const char *remote, struct maintenance_run_opts *opts) +static int fetch_remote(struct remote *remote, void *cbdata) { + struct maintenance_run_opts *opts = cbdata; struct child_process child = CHILD_PROCESS_INIT; child.git_cmd = 1; - strvec_pushl(&child.args, "fetch", remote, "--prune", "--no-tags", + strvec_pushl(&child.args, "fetch", remote->name, "--prune", "--no-tags", "--no-write-fetch-head", "--recurse-submodules=no", "--refmap=", NULL); if (opts->quiet) strvec_push(&child.args, "--quiet"); - strvec_pushf(&child.args, "+refs/heads/*:refs/prefetch/%s/*", remote); + strvec_pushf(&child.args, "+refs/heads/*:refs/prefetch/%s/*", remote->name); return !!run_command(&child); } -static int append_remote(struct remote *remote, void *cbdata) -{ - struct string_list *remotes = (struct string_list *)cbdata; - - string_list_append(remotes, remote->name); - return 0; -} - static int maintenance_task_prefetch(struct maintenance_run_opts *opts) { - int result = 0; - struct string_list_item *item; - struct string_list remotes = STRING_LIST_INIT_DUP; - git_config_set_multivar_gently("log.excludedecoration", "refs/prefetch/", "refs/prefetch/", CONFIG_FLAGS_FIXED_VALUE | CONFIG_FLAGS_MULTI_REPLACE); - if (for_each_remote(append_remote, &remotes)) { - error(_("failed to fill remotes")); - result = 1; - goto cleanup; + if (for_each_remote(fetch_remote, opts)) { + error(_("failed to prefetch remotes")); + return 1; } - for_each_string_list_item(item, &remotes) - result |= fetch_remote(item->string, opts); - -cleanup: - string_list_clear(&remotes, 0); - return result; + return 0; } static int maintenance_task_gc(struct maintenance_run_opts *opts) From patchwork Sat Apr 10 02:03:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12195295 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1D7A7C433B4 for ; Sat, 10 Apr 2021 02:03:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DECA861029 for ; Sat, 10 Apr 2021 02:03:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233527AbhDJCEL (ORCPT ); Fri, 9 Apr 2021 22:04:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38394 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229665AbhDJCEK (ORCPT ); Fri, 9 Apr 2021 22:04:10 -0400 Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A5A5BC061762 for ; Fri, 9 Apr 2021 19:03:56 -0700 (PDT) Received: by mail-wr1-x434.google.com with SMTP id p6so588428wrn.9 for ; Fri, 09 Apr 2021 19:03:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=4wGUux8KYG7o0rKSd3ApIAUDbMevYzDylGxVWjE8Ips=; b=JLVhPrkoLE65GlIvEgbSlBnIP1EPJStLuSpOH8mkrJammVa1rYXXGnxJb7ngmWZ4cN mfLWt+hKTDT8GmtCUDLRH+sgAdJvmY3ks2XP5bFfqkPSf0qT73QdmN4YNgupiTEAFZnU obsUypUQgQEBxRxo6So8t5OR8ZUiLv8n64J4rl2d96fYlCh53i5Wx7Rpn7iGs7KcHAiM /cyz3V7pQMz8uQ1xs4AqTjt3CZe9d72M6bUPJdYcJmFAY6EqGw3DBOqICOWJjzmh2pD+ AvlI+cxCmhtaClKRCjjoHUAaymJi9JFTE9d8mU+V2Yy5Mq90JLbWn47xv0khyhdMRlDn Z4Ug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=4wGUux8KYG7o0rKSd3ApIAUDbMevYzDylGxVWjE8Ips=; b=GQbKmQaTW9daZxhWFfDjkde1bYMyXKH2zPcyAkQneKSZCf0BRbCjXFEKRiaH7FIqSi ncCSsA60UQidVb0iwKQLaamIFRDyoBsTkx9M62OSG7eXoobaDTMYYpSM+rjnqsLCi+bq GB6TUvjU4maKJWxkYNZRUo7M8CJakKIBEEk19wuZQhupzVB4pRWGAHU6kKzo8vO1Qpnv DccKiu4H/zvpI9cIPhJIeIrvNuhRgWyqW/e83L/zQdoLkXUAgOzMrVLGw66j9cWhnXMP ONdUlzkFqLvJu8cxFr+ecYmp0Fw0Ly7TTyXoDbV8xU/jG3W6YI1XZsEhzorHt526v8Pm eP9g== X-Gm-Message-State: AOAM533EZKeRjLvw13cuMV33jzgL4EjwJvNXmxY+otYUBWSfZsL8gomH bMQ9H3fT6LdmITLE7HCG/YCC4flHy/4= X-Google-Smtp-Source: ABdhPJya+QanWW1plxnKnA71gNoZAJSpDIZWH2CZ8n/VtURitEK1dG8F+l5n60rqJSnW5+ti0uMtiQ== X-Received: by 2002:adf:e108:: with SMTP id t8mr19714739wrz.371.1618020228224; Fri, 09 Apr 2021 19:03:48 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id k16sm6916792wrl.47.2021.04.09.19.03.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 09 Apr 2021 19:03:47 -0700 (PDT) Message-Id: <7f488eea6dbdd93ff1a97a2572de7ddb0d65b7e0.1618020225.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Sat, 10 Apr 2021 02:03:44 +0000 Subject: [PATCH v3 2/3] fetch: add --prefetch option Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: tom.saeger@oracle.com, gitster@pobox.com, sunshine@sunshineco.com, Derrick Stolee , Josh Steadmon , Emily Shaffer , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee The --prefetch option will be used by the 'prefetch' maintenance task instead of sending refspecs explicitly across the command-line. The intention is to modify the refspec to place all results in refs/prefetch/ instead of anywhere else. Create helper method filter_prefetch_refspec() to modify a given refspec to fit the rules expected of the prefetch task: * Negative refspecs are preserved. * Refspecs without a destination are removed. * Refspecs whose source starts with "refs/tags/" are removed. * Other refspecs are placed within "refs/prefetch/". Finally, we add the 'force' option to ensure that prefetch refs are replaced as necessary. Signed-off-by: Derrick Stolee --- Documentation/fetch-options.txt | 5 +++ builtin/fetch.c | 56 +++++++++++++++++++++++++++++++ t/t5582-fetch-negative-refspec.sh | 30 +++++++++++++++++ 3 files changed, 91 insertions(+) diff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt index 07783deee309..9e7b4e189ce0 100644 --- a/Documentation/fetch-options.txt +++ b/Documentation/fetch-options.txt @@ -110,6 +110,11 @@ ifndef::git-pull[] setting `fetch.writeCommitGraph`. endif::git-pull[] +--prefetch:: + Modify the configured refspec to place all refs into the + `refs/prefetch/` namespace. See the `prefetch` task in + linkgit:git-maintenance[1]. + -p:: --prune:: Before fetching, remove any remote-tracking references that no diff --git a/builtin/fetch.c b/builtin/fetch.c index 0b90de87c7a2..30856b442b79 100644 --- a/builtin/fetch.c +++ b/builtin/fetch.c @@ -48,6 +48,7 @@ enum { static int fetch_prune_config = -1; /* unspecified */ static int fetch_show_forced_updates = 1; static uint64_t forced_updates_ms = 0; +static int prefetch = 0; static int prune = -1; /* unspecified */ #define PRUNE_BY_DEFAULT 0 /* do we prune by default? */ @@ -158,6 +159,8 @@ static struct option builtin_fetch_options[] = { N_("do not fetch all tags (--no-tags)"), TAGS_UNSET), OPT_INTEGER('j', "jobs", &max_jobs, N_("number of submodules fetched in parallel")), + OPT_BOOL(0, "prefetch", &prefetch, + N_("modify the refspec to place all refs within refs/prefetch/")), OPT_BOOL('p', "prune", &prune, N_("prune remote-tracking branches no longer on remote")), OPT_BOOL('P', "prune-tags", &prune_tags, @@ -436,6 +439,55 @@ static void find_non_local_tags(const struct ref *refs, oidset_clear(&fetch_oids); } +static void filter_prefetch_refspec(struct refspec *rs) +{ + int i; + + if (!prefetch) + return; + + for (i = 0; i < rs->nr; i++) { + struct strbuf new_dst = STRBUF_INIT; + char *old_dst; + const char *sub = NULL; + + if (rs->items[i].negative) + continue; + if (!rs->items[i].dst || + (rs->items[i].src && + !strncmp(rs->items[i].src, "refs/tags/", 10))) { + int j; + + free(rs->items[i].src); + free(rs->items[i].dst); + + for (j = i + 1; j < rs->nr; j++) { + rs->items[j - 1] = rs->items[j]; + rs->raw[j - 1] = rs->raw[j]; + } + rs->nr--; + continue; + } + + old_dst = rs->items[i].dst; + strbuf_addstr(&new_dst, "refs/prefetch/"); + + /* + * If old_dst starts with "refs/", then place + * sub after that prefix. Otherwise, start at + * the beginning of the string. + */ + if (!skip_prefix(old_dst, "refs/", &sub)) + sub = old_dst; + strbuf_addstr(&new_dst, sub); + + rs->items[i].dst = strbuf_detach(&new_dst, NULL); + rs->items[i].force = 1; + + free(old_dst); + } +} + static struct ref *get_ref_map(struct remote *remote, const struct ref *remote_refs, struct refspec *rs, @@ -452,6 +504,10 @@ static struct ref *get_ref_map(struct remote *remote, struct hashmap existing_refs; int existing_refs_populated = 0; + filter_prefetch_refspec(rs); + if (remote) + filter_prefetch_refspec(&remote->fetch); + if (rs->nr) { struct refspec *fetch_refspec; diff --git a/t/t5582-fetch-negative-refspec.sh b/t/t5582-fetch-negative-refspec.sh index f34509727702..030e6f978c4e 100755 --- a/t/t5582-fetch-negative-refspec.sh +++ b/t/t5582-fetch-negative-refspec.sh @@ -240,4 +240,34 @@ test_expect_success "push with matching +: and negative refspec" ' git -C two push -v one ' +test_expect_success '--prefetch correctly modifies refspecs' ' + git -C one config --unset-all remote.origin.fetch && + git -C one config --add remote.origin.fetch "refs/tags/*:refs/tags/*" && + git -C one config --add remote.origin.fetch ^refs/heads/bogus/ignore && + git -C one config --add remote.origin.fetch "refs/heads/bogus/*:bogus/*" && + + git tag -a -m never never-fetch-tag HEAD && + + git branch bogus/fetched HEAD~1 && + git branch bogus/ignore HEAD && + + git -C one fetch --prefetch --no-tags && + test_must_fail git -C one rev-parse never-fetch-tag && + git -C one rev-parse refs/prefetch/bogus/fetched && + test_must_fail git -C one rev-parse refs/prefetch/bogus/ignore && + + # correctly handle when refspec set becomes empty + # after removing the refs/tags/* refspec. + git -C one config --unset-all remote.origin.fetch && + git -C one config --add remote.origin.fetch "refs/tags/*:refs/tags/*" && + + git -C one fetch --prefetch --no-tags && + test_must_fail git -C one rev-parse never-fetch-tag && + + # The refspec for refs that are not fully qualified + # are filtered multiple times. + git -C one rev-parse refs/prefetch/bogus/fetched && + test_must_fail git -C one rev-parse refs/prefetch/bogus/ignore +' + test_done From patchwork Sat Apr 10 02:03:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12195297 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B7027C43460 for ; Sat, 10 Apr 2021 02:03:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8AF0A61105 for ; Sat, 10 Apr 2021 02:03:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233577AbhDJCEM (ORCPT ); Fri, 9 Apr 2021 22:04:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38404 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231946AbhDJCEL (ORCPT ); Fri, 9 Apr 2021 22:04:11 -0400 Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5F83EC061764 for ; Fri, 9 Apr 2021 19:03:57 -0700 (PDT) Received: by mail-wr1-x42b.google.com with SMTP id w4so3560024wrt.5 for ; Fri, 09 Apr 2021 19:03:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=BKZMXuN/+p8qrPdSacUofbQM5WONxi8AP2yKhlLmgCo=; b=eXJlHXWpZfiGYGIc5qd1jevwlTcm/y5MvhuD3ZcREvhmbTm76mJLzv7v/G2tYBjafs Xk/7AD9lADUqchhWOcbrlGR/UY/wYYp7p3jKsWuHk2ybRpY8CAhIxF4yV4mw7WoxStjm 0lMbLD3oChBDsQQUInzYxfA7GEDsL91HkVixqScF/0oQPq/D2rXE7i1jZY/r3oLB0DkS jEaafPzSvZyk77M5kuP42880ZbX6lRqoS6emF67/rRVKiH4dd69cEoihIsl9bx+2/iPm E9WVIh/dqeX3NPiz1DBNhYkQdIRp7Rrv8WRntQ/QkIcX5kLj2ZIisg8lTgqZ0OspSklK ZBjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=BKZMXuN/+p8qrPdSacUofbQM5WONxi8AP2yKhlLmgCo=; b=KwFktYyKpSdgcV6jND7FehaA/myfWpwGoUUoznXvG95v/ShFugT82k3t6sPLhY16FC SYRqHLJWMsLVOMfJN6C0/AJo+/5BpFQfY4fBn0DvS+cn44vyuuF1Jg9CfDzeQqfxG/NS SRXGPszBeLaWkAVunMNpN0k0qvaVPwWzrWLFRQcPAggZMku4ltz7GunGcuZmtVwV/ZyN Ug61OXS/hs77xqvI9bE3hxCAO1GlJce4HkfwPCu3uqyweZUEo7Gq6bm67/ZkG9OxCZcc jkpLwdLnilTSEPp58Anq7G4A1XP1ZI6t+H+dB27hNU7MvBvj6W5UmF1KgGVqj/01yuo8 Wkpw== X-Gm-Message-State: AOAM530wImj5M5epctnFZ6MS/McjSgPpVeDgPAtV4mbU+qSkWNG1cBhN ehTyAdudvEG/iNSQfjfeqe+ZO4HtP/Y= X-Google-Smtp-Source: ABdhPJxn0sGfT1hVTmT0GKA6Q8bMN/453OXD7nvNo105or+7hz02l/ob13GxsM93rhkFNmrbP5K6/Q== X-Received: by 2002:a5d:5681:: with SMTP id f1mr16392855wrv.32.1618020228713; Fri, 09 Apr 2021 19:03:48 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id e18sm3151179wrc.85.2021.04.09.19.03.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 09 Apr 2021 19:03:48 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Sat, 10 Apr 2021 02:03:45 +0000 Subject: [PATCH v3 3/3] maintenance: use 'git fetch --prefetch' Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: tom.saeger@oracle.com, gitster@pobox.com, sunshine@sunshineco.com, Derrick Stolee , Josh Steadmon , Emily Shaffer , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee The 'prefetch' maintenance task previously forced the following refspec for each remote: +refs/heads/*:refs/prefetch//* If a user has specified a more strict refspec for the remote, then this prefetch task downloads more objects than necessary. The previous change introduced the '--prefetch' option to 'git fetch' which manipulates the remote's refspec to place all resulting refs into refs/prefetch/, with further partitioning based on the destinations of those refspecs. Update the documentation to be more generic about the destination refs. Do not mention custom refspecs explicitly, as that does not need to be highlighted in this documentation. The important part of placing refs in refs/prefetch/ remains. Reported-by: Tom Saeger Signed-off-by: Derrick Stolee --- Documentation/git-maintenance.txt | 6 ++---- builtin/gc.c | 7 +++---- t/t7900-maintenance.sh | 14 +++++++------- 3 files changed, 12 insertions(+), 15 deletions(-) diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt index 80ddd33ceba0..1e738ad39832 100644 --- a/Documentation/git-maintenance.txt +++ b/Documentation/git-maintenance.txt @@ -92,10 +92,8 @@ commit-graph:: prefetch:: The `prefetch` task updates the object directory with the latest objects from all registered remotes. For each remote, a `git fetch` - command is run. The refmap is custom to avoid updating local or remote - branches (those in `refs/heads` or `refs/remotes`). Instead, the - remote refs are stored in `refs/prefetch//`. Also, tags are - not updated. + command is run. The configured refspec is modified to place all + requested refs within `refs/prefetch/`. Also, tags are not updated. + This is done to avoid disrupting the remote-tracking branches. The end users expect these refs to stay unmoved unless they initiate a fetch. With prefetch diff --git a/builtin/gc.c b/builtin/gc.c index fa8128de9ae1..9d35f7da50d8 100644 --- a/builtin/gc.c +++ b/builtin/gc.c @@ -879,15 +879,14 @@ static int fetch_remote(struct remote *remote, void *cbdata) struct child_process child = CHILD_PROCESS_INIT; child.git_cmd = 1; - strvec_pushl(&child.args, "fetch", remote->name, "--prune", "--no-tags", + strvec_pushl(&child.args, "fetch", remote->name, + "--prefetch", "--prune", "--no-tags", "--no-write-fetch-head", "--recurse-submodules=no", - "--refmap=", NULL); + NULL); if (opts->quiet) strvec_push(&child.args, "--quiet"); - strvec_pushf(&child.args, "+refs/heads/*:refs/prefetch/%s/*", remote->name); - return !!run_command(&child); } diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh index 2412d8c5c006..eadb800c08cc 100755 --- a/t/t7900-maintenance.sh +++ b/t/t7900-maintenance.sh @@ -141,15 +141,15 @@ test_expect_success 'prefetch multiple remotes' ' test_commit -C clone1 one && test_commit -C clone2 two && GIT_TRACE2_EVENT="$(pwd)/run-prefetch.txt" git maintenance run --task=prefetch 2>/dev/null && - fetchargs="--prune --no-tags --no-write-fetch-head --recurse-submodules=no --refmap= --quiet" && - test_subcommand git fetch remote1 $fetchargs +refs/heads/\\*:refs/prefetch/remote1/\\* log &&