From patchwork Thu Jun 17 17:13:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Tan X-Patchwork-Id: 12328953 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.2 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,GAPPY_SUBJECT, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2247BC2B9F4 for ; Thu, 17 Jun 2021 17:13:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EE944608FC for ; Thu, 17 Jun 2021 17:13:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232012AbhFQRPp (ORCPT ); Thu, 17 Jun 2021 13:15:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45072 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231250AbhFQRPl (ORCPT ); Thu, 17 Jun 2021 13:15:41 -0400 Received: from mail-qk1-x749.google.com (mail-qk1-x749.google.com [IPv6:2607:f8b0:4864:20::749]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BBAF8C061574 for ; Thu, 17 Jun 2021 10:13:32 -0700 (PDT) Received: by mail-qk1-x749.google.com with SMTP id c3-20020a37b3030000b02903ad0001a2e8so2578396qkf.3 for ; Thu, 17 Jun 2021 10:13:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=FP14PzBmaijxsgz7jsoEwQZI00PgyioXjq7JZoiyMhI=; b=ce3Whd9djqnswt38Q6qRgbJCNP8EQmln0e/FqH5whsT/pcLzXdXOCY840HHl8zNjZ4 IFW4U3OsfNgoPkhacJ/b+Tx4oSBPfkLZtAPo9f3ZtDgqe2JA5wQZGbzsBvdoz7O03Xj1 HJVyoxStxA6gEZRWC6cFPlCWa8DxZqcu8cLs1mh2MQ7LlooFijnnJHbYW5FVsbaXhZS5 MpNlGu1N19GUQK2CXVD1YuVLgmvjM0zcGnp/ptpcIijJMlEmX+vPNrxJRsh9KrqKvCxV KihlTjsVi6wmDconuiZ6Xc/SaqS4uf2KCMrDx35idjkjLViwAofjLzoQVciAHMoqs0wp ksSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=FP14PzBmaijxsgz7jsoEwQZI00PgyioXjq7JZoiyMhI=; b=uHIyglen43obxapaIb+tRWbER/mhb2ests4TTceaQPc+GZeV1igJmY03dfRlwZn2wS 4PDKA2HhpcjW7uHNe5uS+r2ZXue8+7sMyQqPg6o4v6AOVQr2cX+toldfpE7BaMnJD+B1 IZXetRHcMQwlXk0ur+4YOhvsRAIR6NiFwCv6SBzOZN3OOL4a0aPK07cUZDM6oDnGeeLY m+NhmIpKfwKvFnkeEcaoVX7umHeQW7XsjnJz9SwnibgT0w761wdHECEi4fxjavkYwg/7 ldb+HCERznuJdIMY+uywhqtjoZgK13SqpyUqDK0PGtwqWoOjK1mnAVPdAYELWRgcZy8e NXMQ== X-Gm-Message-State: AOAM5321VDt/o3jEFy4U4B0PpD3VmsZdZF+ocUnqsGH2HQIdTVsCp1fE Ur0INhea7p5XbfTwmm+rnn7nYky2FlYpW4KQLCBJYJzo9UwzQMk/PYSDitadVxXGQC0xpepnnA8 I12nNGOtTmoOouG6Rdgm7SU8iEKOhnEH7uTpYXbhKozwQxdt85R3hD7eHr2erVdnBUy3WXbW56w D/ X-Google-Smtp-Source: ABdhPJwvXUTmFcakmaF9MczCp386nQbOL4+U0PuAqsk2Url5rcMhmwsGmOYTP8odpizKkM1GCBcj0jOCoxAsxYyasjNj X-Received: from twelve4.c.googlers.com ([fda3:e722:ac3:10:24:72f4:c0a8:437a]) (user=jonathantanmy job=sendgmr) by 2002:a25:824f:: with SMTP id d15mr8145723ybn.96.1623950011816; Thu, 17 Jun 2021 10:13:31 -0700 (PDT) Date: Thu, 17 Jun 2021 10:13:22 -0700 In-Reply-To: Message-Id: <0bd009597ddccddba15e1e30bf0965c5d492fd44.1623949899.git.jonathantanmy@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.32.0.272.g935e593368-goog Subject: [PATCH v4 1/5] repository: move global r_f_p_c to repo struct From: Jonathan Tan To: git@vger.kernel.org Cc: Jonathan Tan , newren@gmail.com, Junio C Hamano Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Move repository_format_partial_clone, which is currently a global variable, into struct repository. (Full support for per-repository partial clone config will be done in a subsequent commit - this is split into its own commit because of the extent of the changes needed.) The new repo-specific variable cannot be set in check_repository_format_gently() (as is currently), because that function does not know which repo it is operating on (or even whether the value is important); therefore this responsibility is delegated to the outermost caller that knows. Of all the outermost callers that know (found by looking at all functions that call clear_repository_format()), I looked at those that either read from the main Git directory or write into a struct repository. These callers have been modified accordingly (write to the_repository in the former case and write to the given struct repository in the latter case). Signed-off-by: Jonathan Tan Signed-off-by: Junio C Hamano --- promisor-remote.c | 13 +++---------- promisor-remote.h | 6 ------ repository.c | 4 ++++ repository.h | 3 +++ setup.c | 17 +++++++++++++---- 5 files changed, 23 insertions(+), 20 deletions(-) diff --git a/promisor-remote.c b/promisor-remote.c index da3f2ca261..d24081dc21 100644 --- a/promisor-remote.c +++ b/promisor-remote.c @@ -5,13 +5,6 @@ #include "transport.h" #include "strvec.h" -static char *repository_format_partial_clone; - -void set_repository_format_partial_clone(char *partial_clone) -{ - repository_format_partial_clone = xstrdup_or_null(partial_clone); -} - static int fetch_objects(const char *remote_name, const struct object_id *oids, int oid_nr) @@ -145,15 +138,15 @@ static void promisor_remote_init(void) git_config(promisor_remote_config, NULL); - if (repository_format_partial_clone) { + if (the_repository->repository_format_partial_clone) { struct promisor_remote *o, *previous; - o = promisor_remote_lookup(repository_format_partial_clone, + o = promisor_remote_lookup(the_repository->repository_format_partial_clone, &previous); if (o) promisor_remote_move_to_tail(o, previous); else - promisor_remote_new(repository_format_partial_clone); + promisor_remote_new(the_repository->repository_format_partial_clone); } } diff --git a/promisor-remote.h b/promisor-remote.h index c7a14063c5..687210ab87 100644 --- a/promisor-remote.h +++ b/promisor-remote.h @@ -32,10 +32,4 @@ int promisor_remote_get_direct(struct repository *repo, const struct object_id *oids, int oid_nr); -/* - * This should be used only once from setup.c to set the value we got - * from the extensions.partialclone config option. - */ -void set_repository_format_partial_clone(char *partial_clone); - #endif /* PROMISOR_REMOTE_H */ diff --git a/repository.c b/repository.c index 448cd557d4..057a4748a0 100644 --- a/repository.c +++ b/repository.c @@ -172,6 +172,10 @@ int repo_init(struct repository *repo, repo_set_hash_algo(repo, format.hash_algo); + /* take ownership of format.partial_clone */ + repo->repository_format_partial_clone = format.partial_clone; + format.partial_clone = NULL; + if (worktree) repo_set_worktree(repo, worktree); diff --git a/repository.h b/repository.h index a45f7520fd..6fb16ed336 100644 --- a/repository.h +++ b/repository.h @@ -139,6 +139,9 @@ struct repository { /* True if commit-graph has been disabled within this process. */ int commit_graph_disabled; + /* Configurations related to promisor remotes. */ + char *repository_format_partial_clone; + /* Configurations */ /* Indicate if a repository has a different 'commondir' from 'gitdir' */ diff --git a/setup.c b/setup.c index ead2f80cd8..eb9367ca5c 100644 --- a/setup.c +++ b/setup.c @@ -468,8 +468,6 @@ static enum extension_result handle_extension_v0(const char *var, data->precious_objects = git_config_bool(var, value); return EXTENSION_OK; } else if (!strcmp(ext, "partialclone")) { - if (!value) - return config_error_nonbool(var); data->partial_clone = xstrdup(value); return EXTENSION_OK; } else if (!strcmp(ext, "worktreeconfig")) { @@ -566,7 +564,6 @@ static int check_repository_format_gently(const char *gitdir, struct repository_ } repository_format_precious_objects = candidate->precious_objects; - set_repository_format_partial_clone(candidate->partial_clone); repository_format_worktree_config = candidate->worktree_config; string_list_clear(&candidate->unknown_extensions, 0); string_list_clear(&candidate->v1_only_extensions, 0); @@ -1197,6 +1194,11 @@ int discover_git_directory(struct strbuf *commondir, return -1; } + /* take ownership of candidate.partial_clone */ + the_repository->repository_format_partial_clone = + candidate.partial_clone; + candidate.partial_clone = NULL; + clear_repository_format(&candidate); return 0; } @@ -1304,8 +1306,13 @@ const char *setup_git_directory_gently(int *nongit_ok) gitdir = DEFAULT_GIT_DIR_ENVIRONMENT; setup_git_env(gitdir); } - if (startup_info->have_repository) + if (startup_info->have_repository) { repo_set_hash_algo(the_repository, repo_fmt.hash_algo); + /* take ownership of repo_fmt.partial_clone */ + the_repository->repository_format_partial_clone = + repo_fmt.partial_clone; + repo_fmt.partial_clone = NULL; + } } /* * Since precompose_string_if_needed() needs to look at @@ -1390,6 +1397,8 @@ void check_repository_format(struct repository_format *fmt) check_repository_format_gently(get_git_dir(), fmt, NULL); startup_info->have_repository = 1; repo_set_hash_algo(the_repository, fmt->hash_algo); + the_repository->repository_format_partial_clone = + xstrdup_or_null(fmt->partial_clone); clear_repository_format(&repo_fmt); } From patchwork Thu Jun 17 17:13:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Tan X-Patchwork-Id: 12328955 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 03488C48BE5 for ; Thu, 17 Jun 2021 17:13:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DBC54613BF for ; Thu, 17 Jun 2021 17:13:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231889AbhFQRPq (ORCPT ); Thu, 17 Jun 2021 13:15:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45078 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231960AbhFQRPn (ORCPT ); Thu, 17 Jun 2021 13:15:43 -0400 Received: from mail-pf1-x44a.google.com (mail-pf1-x44a.google.com [IPv6:2607:f8b0:4864:20::44a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2667AC06175F for ; Thu, 17 Jun 2021 10:13:34 -0700 (PDT) Received: by mail-pf1-x44a.google.com with SMTP id q3-20020aa784230000b02902ea311f25e2so1371070pfn.1 for ; Thu, 17 Jun 2021 10:13:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=+xN1Y02X2Xt08NYcKUmv/3ZIjFO1TysBeVC64u2e0t4=; b=DWnUO7Uav6J4rBENqv8qDYO900EljIJP3sgRj0V0v6B/me6dI8HIsgAhulbCKIHjId czXWO7d2qQYVTzRX4rse8Hkc1GW3mdMJ9hTJ90CaTtTllRWJV8KCmAIl5EIFISZ/lArs 1DCrSltrScth1xR72go/E1DXUPM1oPtg4hzi7RQeiwtgUlR8sgFNRmINKQrFyPUr1wxm NftZ2SBnTJS3ojg5i1Jh7WfnDwrXRwiobYelnR2r8c4axJ4d+j/yNk6fVNNyB5SF43UT nKxODJSAzbLHfu7yXTvs1y5RTK+H6XXC+N6S/p08cveEN6BI7Li8zCEMSUNLsTBD7CXo cbBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=+xN1Y02X2Xt08NYcKUmv/3ZIjFO1TysBeVC64u2e0t4=; b=g1V1QUZj5AdvpFa+OzQrmafxcZKlqQXNo7A6q2f3nRCw7yvxxXGp68uixw5OpPdEn8 EV6KAKqxO0AjoWdp7Qocg7R8a6CbaqDZZlo2qcGiLfe2JWzjvrYKDCpt9QvMy5xGSV1i otqOKQhuXENvt9yoLOd5/cQqwwqYEGaAzJPcVoTnT3Q3cV+XwZ1V62WkrhZTZfB2VAD2 Z5eyv7Pm8/162RKauWSuHrdnUVQwUvIADO5lFOzxbTIXHqc9dbclEQai4/jqMzK58MZh xXPRTh8/ommG8k9IswYiW6zM2nRJlp7yVNYrU9pJQDweJcrr2o3mTojy4UmKChnA8sxn P+8w== X-Gm-Message-State: AOAM530oMVd+80hH0Bf1wcrPcPHVk38Q9IadyQxSZw8U4ApVi5FgkYov t7JhavIC8WVXvk8b8zyh3/v2m8ZVlUV29V06eYdmPjeWLlvdmBwwuFFpVOlBUMDaHE1JtY94BNA X6qIHRmSCO1tZ8fD0wmi9sIi01QBfDvtOdZHwX7b+lMc1SMc24avLoSYroNS42DpEachSeX0fUQ fi X-Google-Smtp-Source: ABdhPJyW/BZDsvEZ7FMqejnYqV85bFumhT6hz1YFttAJ28cCf1Clj5kmONRPgFe7d7OHqd3dUABjaMKLPxKV9gXn+w5r X-Received: from twelve4.c.googlers.com ([fda3:e722:ac3:10:24:72f4:c0a8:437a]) (user=jonathantanmy job=sendgmr) by 2002:a17:90a:1d0e:: with SMTP id c14mr5560281pjd.171.1623950013451; Thu, 17 Jun 2021 10:13:33 -0700 (PDT) Date: Thu, 17 Jun 2021 10:13:23 -0700 In-Reply-To: Message-Id: <8a478b46bffc20e34aed2251e812ec44314106b8.1623949899.git.jonathantanmy@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.32.0.272.g935e593368-goog Subject: [PATCH v4 2/5] promisor-remote: support per-repository config From: Jonathan Tan To: git@vger.kernel.org Cc: Jonathan Tan , newren@gmail.com, Junio C Hamano Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Instead of using global variables to store promisor remote information, store this config in struct repository instead, and add repository-agnostic non-static functions corresponding to the existing non-static functions that only work on the_repository. The actual lazy-fetching of missing objects currently does not work on repositories other than the_repository, and will still not work after this commit, so add a BUG message explaining this. A subsequent commit will remove this limitation. Signed-off-by: Jonathan Tan Signed-off-by: Junio C Hamano --- promisor-remote.c | 98 ++++++++++++++++++++++++++--------------------- promisor-remote.h | 22 +++++++++-- repository.c | 6 +++ repository.h | 2 + 4 files changed, 82 insertions(+), 46 deletions(-) diff --git a/promisor-remote.c b/promisor-remote.c index d24081dc21..1e00e16b0f 100644 --- a/promisor-remote.c +++ b/promisor-remote.c @@ -5,6 +5,11 @@ #include "transport.h" #include "strvec.h" +struct promisor_remote_config { + struct promisor_remote *promisors; + struct promisor_remote **promisors_tail; +}; + static int fetch_objects(const char *remote_name, const struct object_id *oids, int oid_nr) @@ -35,10 +40,8 @@ static int fetch_objects(const char *remote_name, return finish_command(&child) ? -1 : 0; } -static struct promisor_remote *promisors; -static struct promisor_remote **promisors_tail = &promisors; - -static struct promisor_remote *promisor_remote_new(const char *remote_name) +static struct promisor_remote *promisor_remote_new(struct promisor_remote_config *config, + const char *remote_name) { struct promisor_remote *r; @@ -50,18 +53,19 @@ static struct promisor_remote *promisor_remote_new(const char *remote_name) FLEX_ALLOC_STR(r, name, remote_name); - *promisors_tail = r; - promisors_tail = &r->next; + *config->promisors_tail = r; + config->promisors_tail = &r->next; return r; } -static struct promisor_remote *promisor_remote_lookup(const char *remote_name, +static struct promisor_remote *promisor_remote_lookup(struct promisor_remote_config *config, + const char *remote_name, struct promisor_remote **previous) { struct promisor_remote *r, *p; - for (p = NULL, r = promisors; r; p = r, r = r->next) + for (p = NULL, r = config->promisors; r; p = r, r = r->next) if (!strcmp(r->name, remote_name)) { if (previous) *previous = p; @@ -71,7 +75,8 @@ static struct promisor_remote *promisor_remote_lookup(const char *remote_name, return NULL; } -static void promisor_remote_move_to_tail(struct promisor_remote *r, +static void promisor_remote_move_to_tail(struct promisor_remote_config *config, + struct promisor_remote *r, struct promisor_remote *previous) { if (r->next == NULL) @@ -80,14 +85,15 @@ static void promisor_remote_move_to_tail(struct promisor_remote *r, if (previous) previous->next = r->next; else - promisors = r->next ? r->next : r; + config->promisors = r->next ? r->next : r; r->next = NULL; - *promisors_tail = r; - promisors_tail = &r->next; + *config->promisors_tail = r; + config->promisors_tail = &r->next; } static int promisor_remote_config(const char *var, const char *value, void *data) { + struct promisor_remote_config *config = data; const char *name; size_t namelen; const char *subkey; @@ -103,8 +109,8 @@ static int promisor_remote_config(const char *var, const char *value, void *data remote_name = xmemdupz(name, namelen); - if (!promisor_remote_lookup(remote_name, NULL)) - promisor_remote_new(remote_name); + if (!promisor_remote_lookup(config, remote_name, NULL)) + promisor_remote_new(config, remote_name); free(remote_name); return 0; @@ -113,9 +119,9 @@ static int promisor_remote_config(const char *var, const char *value, void *data struct promisor_remote *r; char *remote_name = xmemdupz(name, namelen); - r = promisor_remote_lookup(remote_name, NULL); + r = promisor_remote_lookup(config, remote_name, NULL); if (!r) - r = promisor_remote_new(remote_name); + r = promisor_remote_new(config, remote_name); free(remote_name); @@ -128,59 +134,63 @@ static int promisor_remote_config(const char *var, const char *value, void *data return 0; } -static int initialized; - -static void promisor_remote_init(void) +static void promisor_remote_init(struct repository *r) { - if (initialized) + struct promisor_remote_config *config; + + if (r->promisor_remote_config) return; - initialized = 1; + config = r->promisor_remote_config = + xcalloc(sizeof(*r->promisor_remote_config), 1); + config->promisors_tail = &config->promisors; - git_config(promisor_remote_config, NULL); + repo_config(r, promisor_remote_config, config); - if (the_repository->repository_format_partial_clone) { + if (r->repository_format_partial_clone) { struct promisor_remote *o, *previous; - o = promisor_remote_lookup(the_repository->repository_format_partial_clone, + o = promisor_remote_lookup(config, + r->repository_format_partial_clone, &previous); if (o) - promisor_remote_move_to_tail(o, previous); + promisor_remote_move_to_tail(config, o, previous); else - promisor_remote_new(the_repository->repository_format_partial_clone); + promisor_remote_new(config, r->repository_format_partial_clone); } } -static void promisor_remote_clear(void) +void promisor_remote_clear(struct promisor_remote_config *config) { - while (promisors) { - struct promisor_remote *r = promisors; - promisors = promisors->next; + while (config->promisors) { + struct promisor_remote *r = config->promisors; + config->promisors = config->promisors->next; free(r); } - promisors_tail = &promisors; + config->promisors_tail = &config->promisors; } -void promisor_remote_reinit(void) +void repo_promisor_remote_reinit(struct repository *r) { - initialized = 0; - promisor_remote_clear(); - promisor_remote_init(); + promisor_remote_clear(r->promisor_remote_config); + FREE_AND_NULL(r->promisor_remote_config); + promisor_remote_init(r); } -struct promisor_remote *promisor_remote_find(const char *remote_name) +struct promisor_remote *repo_promisor_remote_find(struct repository *r, + const char *remote_name) { - promisor_remote_init(); + promisor_remote_init(r); if (!remote_name) - return promisors; + return r->promisor_remote_config->promisors; - return promisor_remote_lookup(remote_name, NULL); + return promisor_remote_lookup(r->promisor_remote_config, remote_name, NULL); } -int has_promisor_remote(void) +int repo_has_promisor_remote(struct repository *r) { - return !!promisor_remote_find(NULL); + return !!repo_promisor_remote_find(r, NULL); } static int remove_fetched_oids(struct repository *repo, @@ -228,9 +238,11 @@ int promisor_remote_get_direct(struct repository *repo, if (oid_nr == 0) return 0; - promisor_remote_init(); + promisor_remote_init(repo); - for (r = promisors; r; r = r->next) { + if (repo != the_repository) + BUG("only the_repository is supported for now"); + for (r = repo->promisor_remote_config->promisors; r; r = r->next) { if (fetch_objects(r->name, remaining_oids, remaining_nr) < 0) { if (remaining_nr == 1) continue; diff --git a/promisor-remote.h b/promisor-remote.h index 687210ab87..edc45ab0f5 100644 --- a/promisor-remote.h +++ b/promisor-remote.h @@ -17,9 +17,25 @@ struct promisor_remote { const char name[FLEX_ARRAY]; }; -void promisor_remote_reinit(void); -struct promisor_remote *promisor_remote_find(const char *remote_name); -int has_promisor_remote(void); +void repo_promisor_remote_reinit(struct repository *r); +static inline void promisor_remote_reinit(void) +{ + repo_promisor_remote_reinit(the_repository); +} + +void promisor_remote_clear(struct promisor_remote_config *config); + +struct promisor_remote *repo_promisor_remote_find(struct repository *r, const char *remote_name); +static inline struct promisor_remote *promisor_remote_find(const char *remote_name) +{ + return repo_promisor_remote_find(the_repository, remote_name); +} + +int repo_has_promisor_remote(struct repository *r); +static inline int has_promisor_remote(void) +{ + return repo_has_promisor_remote(the_repository); +} /* * Fetches all requested objects from all promisor remotes, trying them one at diff --git a/repository.c b/repository.c index 057a4748a0..b2bf44c6fa 100644 --- a/repository.c +++ b/repository.c @@ -11,6 +11,7 @@ #include "lockfile.h" #include "submodule-config.h" #include "sparse-index.h" +#include "promisor-remote.h" /* The main repository */ static struct repository the_repo; @@ -262,6 +263,11 @@ void repo_clear(struct repository *repo) if (repo->index != &the_index) FREE_AND_NULL(repo->index); } + + if (repo->promisor_remote_config) { + promisor_remote_clear(repo->promisor_remote_config); + FREE_AND_NULL(repo->promisor_remote_config); + } } int repo_read_index(struct repository *repo) diff --git a/repository.h b/repository.h index 6fb16ed336..3740c93bc0 100644 --- a/repository.h +++ b/repository.h @@ -10,6 +10,7 @@ struct lock_file; struct pathspec; struct raw_object_store; struct submodule_cache; +struct promisor_remote_config; enum untracked_cache_setting { UNTRACKED_CACHE_UNSET = -1, @@ -141,6 +142,7 @@ struct repository { /* Configurations related to promisor remotes. */ char *repository_format_partial_clone; + struct promisor_remote_config *promisor_remote_config; /* Configurations */ From patchwork Thu Jun 17 17:13:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Tan X-Patchwork-Id: 12328957 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E26AC2B9F4 for ; Thu, 17 Jun 2021 17:13:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 879E6613BF for ; Thu, 17 Jun 2021 17:13:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232059AbhFQRPs (ORCPT ); Thu, 17 Jun 2021 13:15:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45090 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232008AbhFQRPn (ORCPT ); Thu, 17 Jun 2021 13:15:43 -0400 Received: from mail-pl1-x64a.google.com (mail-pl1-x64a.google.com [IPv6:2607:f8b0:4864:20::64a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EC3CDC061760 for ; Thu, 17 Jun 2021 10:13:35 -0700 (PDT) Received: by mail-pl1-x64a.google.com with SMTP id e14-20020a170902784eb0290102b64712f9so2060196pln.10 for ; Thu, 17 Jun 2021 10:13:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=Wezj+RNfRGs1nzz2/Vtw8tnsSdlZvdY/E3su3aS8IPY=; b=qFscNkddpXgeXs7xdfGVHCfB5DB9eHq7wHDYyA/IAKiGYxVooqAmxIKGKXQe4ofsf0 n8a2UH05IdpmZZLa1r+spPdKuvnwNF/NVj1CLwKhYqpQu9VYrMIRTqQ4c7bJO8cQbr2a XH2k8YlMEoQQOFw8Wp/hwDOrzC48NyATTga2GzPJVjPdO9VSHMkdBOS0jy9vjU67aLuM HcUDGKqx1wjJmq4GMBueoWvnDm94pCWrsfRHJNk6FJGrpIrPJcWmIcA0FmHuWN6dOaR5 3jpNJBdMGpnOPKZCS18uk6PldGAEZk/2HmSjpDLvZtgRKiqEo8i5ZfAjPwm78LeOlA9Z Z/xg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=Wezj+RNfRGs1nzz2/Vtw8tnsSdlZvdY/E3su3aS8IPY=; b=bFGumk5+KVXwAJhZQoNlZUB3FBXe2qKw4FRbTftoFHCMcw8nCGxIHLOut8jpmftbn9 6KKQt8DWmAdAb4XnRA4Cc9t3WglEzwUbU6zboI3FMAmHuD2hHz5UamWEfHqBwzY/D5Ab 3LsYN6HWQZsdZFBrgAZ+zQlQGSQVPw/EL5+Z3ovUh65pIckFBwCXaH2DqGEQcyCEl6/p IrbyJ6d4/NpLinphp/u9hLFCO7fzIT8tOm2wv9zn+5kKO4Rw8UoOQkJhKlDzrzy9Wt5K nO26lyVO1MHYuzdPL2cJ3+D80uQqaqFPM7g6mqDV0FJlULK4+Q/S8+Tx9u5OqBxvJUgX GCtQ== X-Gm-Message-State: AOAM530SJexx09eavjqlOF5GvXDgFyL3E3DAPVky/Si+JKndxZbJwxo9 eOJ7rf9b9zjgiJF6Kbf0xG3svIA+7QdE5RxpZbZFmWvQ1VSAUGQWfleCPw+dihEu0zRCb++RG9k tfl5TmFgLnv57uYpuCwiO4YWMPNMpB39Zc3zZjxuXTydfC9smGuvitD0mY0UJlWzLlFnTwjM36V ad X-Google-Smtp-Source: ABdhPJyec1kIe6DrzbFiW1zjAvBeLNE5byJga3RdMUYbwlQt5DuCyIJeLjf0RFS3QWTV7Be2Xg6Fg5a0XPE4U1326oES X-Received: from twelve4.c.googlers.com ([fda3:e722:ac3:10:24:72f4:c0a8:437a]) (user=jonathantanmy job=sendgmr) by 2002:a62:b618:0:b029:2f9:f3b1:8afd with SMTP id j24-20020a62b6180000b02902f9f3b18afdmr767153pff.81.1623950015197; Thu, 17 Jun 2021 10:13:35 -0700 (PDT) Date: Thu, 17 Jun 2021 10:13:24 -0700 In-Reply-To: Message-Id: <78b4108ae17f8ebd0113bffc736a7ac4c62c3adf.1623949899.git.jonathantanmy@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.32.0.272.g935e593368-goog Subject: [PATCH v4 3/5] submodule: refrain from filtering GIT_CONFIG_COUNT From: Jonathan Tan To: git@vger.kernel.org Cc: Jonathan Tan , newren@gmail.com, Junio C Hamano Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org 14111fc492 ("git: submodule honor -c credential.* from command line", 2016-03-01) taught Git to pass through the GIT_CONFIG_PARAMETERS environment variable when invoking a subprocess on behalf of a submodule. But when d8d77153ea ("config: allow specifying config entries via envvar pairs", 2021-01-15) introduced support for GIT_CONFIG_COUNT (and its associated GIT_CONFIG_KEY_? and GIT_CONFIG_VALUE_?), the subprocess mechanism wasn't updated to also pass through these variables. Since they are conceptually the same (d8d77153ea was written to address a shortcoming of GIT_CONFIG_PARAMETERS), update the submodule subprocess mechanism to also pass through GIT_CONFIG_COUNT. Signed-off-by: Jonathan Tan Signed-off-by: Junio C Hamano --- submodule.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/submodule.c b/submodule.c index 0b1d9c1dde..f09031e397 100644 --- a/submodule.c +++ b/submodule.c @@ -489,7 +489,8 @@ static void prepare_submodule_repo_env_no_git_dir(struct strvec *out) const char * const *var; for (var = local_repo_env; *var; var++) { - if (strcmp(*var, CONFIG_DATA_ENVIRONMENT)) + if (strcmp(*var, CONFIG_DATA_ENVIRONMENT) && + strcmp(*var, CONFIG_COUNT_ENVIRONMENT)) strvec_push(out, *var); } } From patchwork Thu Jun 17 17:13:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Tan X-Patchwork-Id: 12328959 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5DC82C49EA2 for ; Thu, 17 Jun 2021 17:13:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 484F8613D6 for ; Thu, 17 Jun 2021 17:13:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231966AbhFQRPu (ORCPT ); Thu, 17 Jun 2021 13:15:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45100 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232021AbhFQRPp (ORCPT ); Thu, 17 Jun 2021 13:15:45 -0400 Received: from mail-qk1-x74a.google.com (mail-qk1-x74a.google.com [IPv6:2607:f8b0:4864:20::74a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A0159C06175F for ; Thu, 17 Jun 2021 10:13:37 -0700 (PDT) Received: by mail-qk1-x74a.google.com with SMTP id c15-20020ae9e20f0000b02903aafa8c83e7so1773907qkc.21 for ; Thu, 17 Jun 2021 10:13:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=FE4RwdAK1219mLrHfLfE5RXM90DqsMMKszEJR7HPT50=; b=mLvpl6smT/GnxEQ4it5hNTeEqGhj1ONFfo5M4FWiweQtTInjq7mcvHBuC3fKdgHOem dewJdUMYNV/4yNXe8h133Wc87H9ZCLAu3oc1RP6a36BJoVJ2yzgOl+giueoT2hhwCGXG NpJIX8J1puUzxUAmL8roWgEscBTh+HWU2bAQhP61nlHg3h/uVZbBiYnviXL9E9byTD2L PVJXg7wA3CSP10qmPSKb9VBngJh64sHsTV6qewpzh/+Jt033AqKUWjknrkzQ+53wVlRg fBSU4OFpcrRs2gB/FSbzoP13pdzB/xpRlaounpEFBj/cCPKEickQYD9mr1PvsqJoPWs+ rhAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=FE4RwdAK1219mLrHfLfE5RXM90DqsMMKszEJR7HPT50=; b=egDg1XfzoUxmrdSfjFI12vaBBrbww5LUMxvrc7zEdWqzCcwLySgVxHr6r/CCj+13FK WDOsyPJ38rfUg0o5u7bk5Tm4LtgwlZ9/c8lCJngZmiAeIJYX6I7sHt48rQm8oa2RTS+U 67pCHW+T0OPTnzytBrVuxWLcv3iPG3x51YpVkzmMCRayK3Eqdsy27yDOUGFfJkOXRbUh M/1upYWQ219+QbdmCqSV0EOrp3t0uSilQdcN/laK9gDhT1/zMhr6uf/mcSAtFW1SItrf pXG/Y8J/pm1T6kdb4tcaAp+bqW7her8I9UnNQOFKI13/4apyBvJXmi0/BlN68ymvsIOZ imbg== X-Gm-Message-State: AOAM53331d7Ly0bWXAQt4B7R+sKFqgU0EdPtTUhNKoSa619r4225qm56 YzRlqf4yP1BQNipSomYjafc0oCSTQesbbKu+RGEgZD6ws6qm0p1qcfsPxN4jTvo7KwAEZeFAb3V 8/U4vVEiO1u/CHXUzP0fcYiMj4QinYElAmh2CaeaqFFxFpw33Hnlu/AQv7uDnhFPI7Wc5mCso4i 2X X-Google-Smtp-Source: ABdhPJx7XilBKW842hlniSugLRkX8I7LxY1baBbJAcvV9WUpcIiBjjdd2RObQrwZiVqLo9QYfJgaR3V7HKliOLPamiDm X-Received: from twelve4.c.googlers.com ([fda3:e722:ac3:10:24:72f4:c0a8:437a]) (user=jonathantanmy job=sendgmr) by 2002:a0c:ba05:: with SMTP id w5mr861304qvf.60.1623950016706; Thu, 17 Jun 2021 10:13:36 -0700 (PDT) Date: Thu, 17 Jun 2021 10:13:25 -0700 In-Reply-To: Message-Id: <1778cbf878b992c9e00e0d0f54301426467977ab.1623949899.git.jonathantanmy@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.32.0.272.g935e593368-goog Subject: [PATCH v4 4/5] run-command: refactor subprocess env preparation From: Jonathan Tan To: git@vger.kernel.org Cc: Jonathan Tan , newren@gmail.com, Junio C Hamano Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org submodule.c has functionality that prepares the environment for running a subprocess in a new repo. The lazy-fetching code (used in partial clones) will need this in a subsequent commit, so move it to a more central location. Signed-off-by: Jonathan Tan Signed-off-by: Junio C Hamano --- run-command.c | 12 ++++++++++++ run-command.h | 10 ++++++++++ submodule.c | 18 ++---------------- 3 files changed, 24 insertions(+), 16 deletions(-) diff --git a/run-command.c b/run-command.c index be6bc128cd..549a94a6a4 100644 --- a/run-command.c +++ b/run-command.c @@ -1892,3 +1892,15 @@ int run_auto_maintenance(int quiet) return run_command(&maint); } + +void prepare_other_repo_env(struct strvec *env_array, const char *new_git_dir) +{ + const char * const *var; + + for (var = local_repo_env; *var; var++) { + if (strcmp(*var, CONFIG_DATA_ENVIRONMENT) && + strcmp(*var, CONFIG_COUNT_ENVIRONMENT)) + strvec_push(env_array, *var); + } + strvec_pushf(env_array, "%s=%s", GIT_DIR_ENVIRONMENT, new_git_dir); +} diff --git a/run-command.h b/run-command.h index d08414a92e..22132536a9 100644 --- a/run-command.h +++ b/run-command.h @@ -483,4 +483,14 @@ int run_processes_parallel_tr2(int n, get_next_task_fn, start_failure_fn, task_finished_fn, void *pp_cb, const char *tr2_category, const char *tr2_label); +/** + * Convenience function which prepares env_array for a command to be run in a + * new repo. This adds all GIT_* environment variables to env_array with the + * exception of GIT_CONFIG_PARAMETERS and GIT_CONFIG_COUNT (which cause the + * corresponding environment variables to be unset in the subprocess) and adds + * an environment variable pointing to new_git_dir. See local_repo_env in + * cache.h for more information. + */ +void prepare_other_repo_env(struct strvec *env_array, const char *new_git_dir); + #endif diff --git a/submodule.c b/submodule.c index f09031e397..8e611fe1db 100644 --- a/submodule.c +++ b/submodule.c @@ -484,28 +484,14 @@ static void print_submodule_diff_summary(struct repository *r, struct rev_info * strbuf_release(&sb); } -static void prepare_submodule_repo_env_no_git_dir(struct strvec *out) -{ - const char * const *var; - - for (var = local_repo_env; *var; var++) { - if (strcmp(*var, CONFIG_DATA_ENVIRONMENT) && - strcmp(*var, CONFIG_COUNT_ENVIRONMENT)) - strvec_push(out, *var); - } -} - void prepare_submodule_repo_env(struct strvec *out) { - prepare_submodule_repo_env_no_git_dir(out); - strvec_pushf(out, "%s=%s", GIT_DIR_ENVIRONMENT, - DEFAULT_GIT_DIR_ENVIRONMENT); + prepare_other_repo_env(out, DEFAULT_GIT_DIR_ENVIRONMENT); } static void prepare_submodule_repo_env_in_gitdir(struct strvec *out) { - prepare_submodule_repo_env_no_git_dir(out); - strvec_pushf(out, "%s=.", GIT_DIR_ENVIRONMENT); + prepare_other_repo_env(out, "."); } /* From patchwork Thu Jun 17 17:13:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Tan X-Patchwork-Id: 12328961 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BA1E9C2B9F4 for ; Thu, 17 Jun 2021 17:13:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 98E8B613CB for ; Thu, 17 Jun 2021 17:13:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232046AbhFQRPv (ORCPT ); Thu, 17 Jun 2021 13:15:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45104 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232041AbhFQRPs (ORCPT ); Thu, 17 Jun 2021 13:15:48 -0400 Received: from mail-qk1-x74a.google.com (mail-qk1-x74a.google.com [IPv6:2607:f8b0:4864:20::74a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 46482C061574 for ; Thu, 17 Jun 2021 10:13:39 -0700 (PDT) Received: by mail-qk1-x74a.google.com with SMTP id d194-20020a3768cb0000b02903ad9d001bb6so2598811qkc.7 for ; Thu, 17 Jun 2021 10:13:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=6hk4zM3kNAj5NMgi0ayc8vw+3/YdS03ZN5V+i3iB+nQ=; b=mOMTrg6hqRAb3eyhtzgAnGI7EbtbMKAYnIo3Xzo8bZ9MmZaG11xhECucC59Cy8NsyA tFCRAiG/+bCDMSlwfCeQldJmK9amvU4uOFRANtysqRvpgaVz1qhscX6zvvBQGn4Hi+nt +9opQF8/915M7TogNigYjrneR5ao7Hro7nWJ0kKeeEXMP0pEoxIFogux/Tlcp4neFgOb E09+iNu0+P1E9V1deH9MzQwb3xZYkZvlcnFatq3SiTomjJR1nrKaQY73ameMa5yzrVm8 wfov97dqdmW+PcXWTwH+ywQ2GUo+gRPRHj9lw+Jlm3kZ9gOf1yhCJMHIJW5NnwsGV0xF sl+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=6hk4zM3kNAj5NMgi0ayc8vw+3/YdS03ZN5V+i3iB+nQ=; b=K9RsbTN03isP+gSUs7TG1Ta86eHnWwNRQJvEC2e09tYZ4qxAZMOwD8agBzcrjAn24Q ED2XJNzM0LyVXNH4jXV+k+VorKfGlZAeAadxkubTw7ApMpkPfNxYy/N/oUMM5KwmnFqj Mf7kXQcZ534cVchwxx/TZbFr9+lDzUIlp7n/Sej308L6wb4YJwxNnynlLn/yDwrcaMtS BYYZTWkmKAT0ceH+eo0hPiB+XoOObu64BDaRSDAMfZV5LDTETde/4sXcChlCvxWevTce JNdhhWG4RaCAdvFQWgEExq03GwrodxeJyrWPepi+uYsbZhuYssvXofPwowdF5gppild+ 35XQ== X-Gm-Message-State: AOAM533FU+AElUPCOm+c+Bu0WLF/01MTsnHQ/KY5NSOvK5T+ucIa9Hq5 9KnS5ahNHdXBPIY7axoEddFDiO0cj3s09CKYlqAzdKKCjOkdsXZgoHkgNMRtNsI7ryMaDx4ohzx 5yvNOFuu1U0qBoLzOTH8hDVcMZi0vv3+pN7rr/+3Y6i+EF9+dildRDWexHnNkepCA6o1+jVqQmd Sk X-Google-Smtp-Source: ABdhPJzZsYf/iPw8x916TF1B0eAtBgZX+DuszZxHfkCTbC+Z0JcH3x5OgW/q3/cNf9xcwmlyUIuHPI20F7fgjxdDxaMm X-Received: from twelve4.c.googlers.com ([fda3:e722:ac3:10:24:72f4:c0a8:437a]) (user=jonathantanmy job=sendgmr) by 2002:ad4:5426:: with SMTP id g6mr962139qvt.47.1623950018342; Thu, 17 Jun 2021 10:13:38 -0700 (PDT) Date: Thu, 17 Jun 2021 10:13:26 -0700 In-Reply-To: Message-Id: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.32.0.272.g935e593368-goog Subject: [PATCH v4 5/5] promisor-remote: teach lazy-fetch in any repo From: Jonathan Tan To: git@vger.kernel.org Cc: Jonathan Tan , newren@gmail.com, Junio C Hamano Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org This is one step towards supporting partial clone submodules. Even after this patch, we will still lack partial clone submodules support, primarily because a lot of Git code that accesses submodule objects does so by adding their object stores as alternates, meaning that any lazy fetches that would occur in the submodule would be done based on the config of the superproject, not of the submodule. This also prevents testing of the functionality in this patch by user-facing commands. So for now, test this mechanism using a test helper. Besides that, there is some code that uses the wrapper functions like has_promisor_remote(). Those will need to be checked to see if they could support the non-wrapper functions instead (and thus support any repository, not just the_repository). Signed-off-by: Jonathan Tan Signed-off-by: Junio C Hamano --- Makefile | 1 + object-file.c | 7 ++---- promisor-remote.c | 9 ++++---- t/helper/test-partial-clone.c | 43 +++++++++++++++++++++++++++++++++++ t/helper/test-tool.c | 1 + t/helper/test-tool.h | 1 + t/t0410-partial-clone.sh | 23 +++++++++++++++++++ 7 files changed, 76 insertions(+), 9 deletions(-) create mode 100644 t/helper/test-partial-clone.c diff --git a/Makefile b/Makefile index c3565fc0f8..f6653bcd5e 100644 --- a/Makefile +++ b/Makefile @@ -725,6 +725,7 @@ TEST_BUILTINS_OBJS += test-oidmap.o TEST_BUILTINS_OBJS += test-online-cpus.o TEST_BUILTINS_OBJS += test-parse-options.o TEST_BUILTINS_OBJS += test-parse-pathspec-file.o +TEST_BUILTINS_OBJS += test-partial-clone.o TEST_BUILTINS_OBJS += test-path-utils.o TEST_BUILTINS_OBJS += test-pcre2-config.o TEST_BUILTINS_OBJS += test-pkt-line.o diff --git a/object-file.c b/object-file.c index f233b440b2..ebf273e9e7 100644 --- a/object-file.c +++ b/object-file.c @@ -1570,15 +1570,12 @@ static int do_oid_object_info_extended(struct repository *r, } /* Check if it is a missing object */ - if (fetch_if_missing && has_promisor_remote() && - !already_retried && r == the_repository && + if (fetch_if_missing && repo_has_promisor_remote(r) && + !already_retried && !(flags & OBJECT_INFO_SKIP_FETCH_OBJECT)) { /* * TODO Investigate checking promisor_remote_get_direct() * TODO return value and stopping on error here. - * TODO Pass a repository struct through - * promisor_remote_get_direct(), such that arbitrary - * repositories work. */ promisor_remote_get_direct(r, real, 1); already_retried = 1; diff --git a/promisor-remote.c b/promisor-remote.c index 1e00e16b0f..c088dcbff3 100644 --- a/promisor-remote.c +++ b/promisor-remote.c @@ -10,7 +10,8 @@ struct promisor_remote_config { struct promisor_remote **promisors_tail; }; -static int fetch_objects(const char *remote_name, +static int fetch_objects(struct repository *repo, + const char *remote_name, const struct object_id *oids, int oid_nr) { @@ -20,6 +21,8 @@ static int fetch_objects(const char *remote_name, child.git_cmd = 1; child.in = -1; + if (repo != the_repository) + prepare_other_repo_env(&child.env_array, repo->gitdir); strvec_pushl(&child.args, "-c", "fetch.negotiationAlgorithm=noop", "fetch", remote_name, "--no-tags", "--no-write-fetch-head", "--recurse-submodules=no", @@ -240,10 +243,8 @@ int promisor_remote_get_direct(struct repository *repo, promisor_remote_init(repo); - if (repo != the_repository) - BUG("only the_repository is supported for now"); for (r = repo->promisor_remote_config->promisors; r; r = r->next) { - if (fetch_objects(r->name, remaining_oids, remaining_nr) < 0) { + if (fetch_objects(repo, r->name, remaining_oids, remaining_nr) < 0) { if (remaining_nr == 1) continue; remaining_nr = remove_fetched_oids(repo, &remaining_oids, diff --git a/t/helper/test-partial-clone.c b/t/helper/test-partial-clone.c new file mode 100644 index 0000000000..3f102cfddd --- /dev/null +++ b/t/helper/test-partial-clone.c @@ -0,0 +1,43 @@ +#include "cache.h" +#include "test-tool.h" +#include "repository.h" +#include "object-store.h" + +/* + * Prints the size of the object corresponding to the given hash in a specific + * gitdir. This is similar to "git -C gitdir cat-file -s", except that this + * exercises the code that accesses the object of an arbitrary repository that + * is not the_repository. ("git -C gitdir" makes it so that the_repository is + * the one in gitdir.) + */ +static void object_info(const char *gitdir, const char *oid_hex) +{ + struct repository r; + struct object_id oid; + unsigned long size; + struct object_info oi = {.sizep = &size}; + const char *p; + + if (repo_init(&r, gitdir, NULL)) + die("could not init repo"); + if (parse_oid_hex(oid_hex, &oid, &p)) + die("could not parse oid"); + if (oid_object_info_extended(&r, &oid, &oi, 0)) + die("could not obtain object info"); + printf("%d\n", (int) size); +} + +int cmd__partial_clone(int argc, const char **argv) +{ + setup_git_directory(); + + if (argc < 4) + die("too few arguments"); + + if (!strcmp(argv[1], "object-info")) + object_info(argv[2], argv[3]); + else + die("invalid argument '%s'", argv[1]); + + return 0; +} diff --git a/t/helper/test-tool.c b/t/helper/test-tool.c index c5bd0c6d4c..b21e8f1519 100644 --- a/t/helper/test-tool.c +++ b/t/helper/test-tool.c @@ -46,6 +46,7 @@ static struct test_cmd cmds[] = { { "online-cpus", cmd__online_cpus }, { "parse-options", cmd__parse_options }, { "parse-pathspec-file", cmd__parse_pathspec_file }, + { "partial-clone", cmd__partial_clone }, { "path-utils", cmd__path_utils }, { "pcre2-config", cmd__pcre2_config }, { "pkt-line", cmd__pkt_line }, diff --git a/t/helper/test-tool.h b/t/helper/test-tool.h index e8069a3b22..f845ced4b3 100644 --- a/t/helper/test-tool.h +++ b/t/helper/test-tool.h @@ -35,6 +35,7 @@ int cmd__oidmap(int argc, const char **argv); int cmd__online_cpus(int argc, const char **argv); int cmd__parse_options(int argc, const char **argv); int cmd__parse_pathspec_file(int argc, const char** argv); +int cmd__partial_clone(int argc, const char **argv); int cmd__path_utils(int argc, const char **argv); int cmd__pcre2_config(int argc, const char **argv); int cmd__pkt_line(int argc, const char **argv); diff --git a/t/t0410-partial-clone.sh b/t/t0410-partial-clone.sh index 584a039b85..a211a66c67 100755 --- a/t/t0410-partial-clone.sh +++ b/t/t0410-partial-clone.sh @@ -604,6 +604,29 @@ test_expect_success 'do not fetch when checking existence of tree we construct o git -C repo cherry-pick side1 ' +test_expect_success 'lazy-fetch when accessing object not in the_repository' ' + rm -rf full partial.git && + test_create_repo full && + test_commit -C full create-a-file file.txt && + + test_config -C full uploadpack.allowfilter 1 && + test_config -C full uploadpack.allowanysha1inwant 1 && + git clone --filter=blob:none --bare "file://$(pwd)/full" partial.git && + FILE_HASH=$(git -C full rev-parse HEAD:file.txt) && + + # Sanity check that the file is missing + git -C partial.git rev-list --objects --missing=print HEAD >out && + grep "[?]$FILE_HASH" out && + + git -C full cat-file -s "$FILE_HASH" >expect && + test-tool partial-clone object-info partial.git "$FILE_HASH" >actual && + test_cmp expect actual && + + # Sanity check that the file is now present + git -C partial.git rev-list --objects --missing=print HEAD >out && + ! grep "[?]$FILE_HASH" out +' + . "$TEST_DIRECTORY"/lib-httpd.sh start_httpd