[RFC,v2,4/4] fetch: do not list refs if fetching only hashes

Commit Message

Jonathan Tan Sept. 27, 2018, 7:24 p.m. UTC

If only hash literals are given on a "git fetch" command-line, tag
following is not requested, and the fetch is done using protocol v2, a
list of refs is not required from the remote. Therefore, optimize by
invoking transport_get_remote_refs() only if we need the refs.

Signed-off-by: Jonathan Tan <[e-mail]>
---
 builtin/fetch.c             | 32 ++++++++++++++++++++++++++------
 t/t5551-http-fetch-smart.sh | 15 +++++++++++++++
 t/t5702-protocol-v2.sh      | 13 +++++++++++++
 3 files changed, 54 insertions(+), 6 deletions(-)

Comments

Stefan Beller Sept. 27, 2018, 9:43 p.m. UTC | #1

On Thu, Sep 27, 2018 at 12:24 PM Jonathan Tan <[e-mail]> wrote:
>
> If only hash literals are given on a "git fetch" command-line, tag
> following is not requested, and the fetch is done using protocol v2, a
> list of refs is not required from the remote. Therefore, optimize by
> invoking transport_get_remote_refs() only if we need the refs.
>

Makes sense

> +
> +               /*
> +                * We can avoid listing refs if all of them are exact
> +                * OIDs
> +                */
> +               must_list_refs = 0;
> +               for (i = 0; i < rs->nr; i++) {
> +                       if (!rs->items[i].exact_sha1) {
> +                               must_list_refs = 1;
> +                               break;
> +                       }
> +               }

This seems to be a repeat pattern, Is it worth it to encapsulate it
as a function in transport or refs?

  int must_list_refs(struct ref **to_fetch)
  {
    // body as the loop above
  }

Jonathan Tan Sept. 28, 2018, 5:50 p.m. UTC | #2

> > +               /*
> > +                * We can avoid listing refs if all of them are exact
> > +                * OIDs
> > +                */
> > +               must_list_refs = 0;
> > +               for (i = 0; i < rs->nr; i++) {
> > +                       if (!rs->items[i].exact_sha1) {
> > +                               must_list_refs = 1;
> > +                               break;
> > +                       }
> > +               }
> 
> This seems to be a repeat pattern, Is it worth it to encapsulate it
> as a function in transport or refs?
> 
>   int must_list_refs(struct ref **to_fetch)
>   {
>     // body as the loop above
>   }

The repetition is unfortunate - I tried to think of a better way to do
it but couldn't. We can't do what you suggest because this one loops
over refspecs but the other one loops over refs.

Patch

diff --git a/builtin/fetch.c b/builtin/fetch.c
index 0696abfc2a..4c4f8fa194 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1175,6 +1175,7 @@  static int do_fetch(struct transport *transport,
 	int retcode = 0;
 	const struct ref *remote_refs;
 	struct argv_array ref_prefixes = ARGV_ARRAY_INIT;
+	int must_list_refs = 1;
 
 	if (tags == TAGS_DEFAULT) {
 		if (transport->remote->fetch_tags == 2)
@@ -1190,17 +1191,36 @@  static int do_fetch(struct transport *transport,
 			goto cleanup;
 	}
 
-	if (rs->nr)
+	if (rs->nr) {
+		int i;
+
 		refspec_ref_prefixes(rs, &ref_prefixes);
-	else if (transport->remote && transport->remote->fetch.nr)
+
+		/*
+		 * We can avoid listing refs if all of them are exact
+		 * OIDs
+		 */
+		must_list_refs = 0;
+		for (i = 0; i < rs->nr; i++) {
+			if (!rs->items[i].exact_sha1) {
+				must_list_refs = 1;
+				break;
+			}
+		}
+	} else if (transport->remote && transport->remote->fetch.nr)
 		refspec_ref_prefixes(&transport->remote->fetch, &ref_prefixes);
 
-	if (ref_prefixes.argc &&
-	    (tags == TAGS_SET || (tags == TAGS_DEFAULT))) {
-		argv_array_push(&ref_prefixes, "refs/tags/");
+	if (tags == TAGS_SET || tags == TAGS_DEFAULT) {
+		must_list_refs = 1;
+		if (ref_prefixes.argc)
+			argv_array_push(&ref_prefixes, "refs/tags/");
 	}
 
-	remote_refs = transport_get_remote_refs(transport, &ref_prefixes);
+	if (must_list_refs)
+		remote_refs = transport_get_remote_refs(transport, &ref_prefixes);
+	else
+		remote_refs = NULL;
+
 	argv_array_clear(&ref_prefixes);
 
 	ref_map = get_ref_map(transport->remote, remote_refs, rs,
diff --git a/t/t5551-http-fetch-smart.sh b/t/t5551-http-fetch-smart.sh
index 771f36f9ff..12b339d239 100755
--- a/t/t5551-http-fetch-smart.sh
+++ b/t/t5551-http-fetch-smart.sh
@@ -381,6 +381,21 @@  test_expect_success 'using fetch command in remote-curl updates refs' '
 	test_cmp expect actual
 '
 
+test_expect_success 'fetch by SHA-1 without tag following' '
+	SERVER="$HTTPD_DOCUMENT_ROOT_PATH/server" &&
+	rm -rf "$SERVER" client &&
+
+	git init "$SERVER" &&
+	test_commit -C "$SERVER" foo &&
+
+	git clone $HTTPD_URL/smart/server client &&
+
+	test_commit -C "$SERVER" bar &&
+	git -C "$SERVER" rev-parse bar >bar_hash &&
+	git -C client -c protocol.version=0 fetch \
+		--no-tags origin $(cat bar_hash)
+'
+
 test_expect_success 'GIT_REDACT_COOKIES redacts cookies' '
 	rm -rf clone &&
 	echo "Set-Cookie: Foo=1" >cookies &&
diff --git a/t/t5702-protocol-v2.sh b/t/t5702-protocol-v2.sh
index a316bb9bf4..1a97331648 100755
--- a/t/t5702-protocol-v2.sh
+++ b/t/t5702-protocol-v2.sh
@@ -79,6 +79,19 @@  test_expect_success 'fetch with git:// using protocol v2' '
 	grep "fetch< version 2" log
 '
 
+test_expect_success 'fetch by hash without tag following with protocol v2 does not list refs' '
+	test_when_finished "rm -f log" &&
+
+	test_commit -C "$daemon_parent" two_a &&
+	git -C "$daemon_parent" rev-parse two_a >two_a_hash &&
+
+	GIT_TRACE_PACKET="$(pwd)/log" git -C daemon_child -c protocol.version=2 \
+		fetch --no-tags origin $(cat two_a_hash) &&
+
+	grep "fetch< version 2" log &&
+	! grep "fetch> command=ls-refs" log
+'
+
 test_expect_success 'pull with git:// using protocol v2' '
 	test_when_finished "rm -f log" &&