From patchwork Thu Sep 27 19:24:04 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Tan X-Patchwork-Id: 10618499 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 60BA8112B for ; Thu, 27 Sep 2018 19:24:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4D67A285A6 for ; Thu, 27 Sep 2018 19:24:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 418A128796; Thu, 27 Sep 2018 19:24:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DCD5E285A6 for ; Thu, 27 Sep 2018 19:24:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728672AbeI1BoN (ORCPT ); Thu, 27 Sep 2018 21:44:13 -0400 Received: from mail-qt1-f202.google.com ([209.85.160.202]:45704 "EHLO mail-qt1-f202.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728101AbeI1BoM (ORCPT ); Thu, 27 Sep 2018 21:44:12 -0400 Received: by mail-qt1-f202.google.com with SMTP id t17-v6so3251906qtq.12 for ; Thu, 27 Sep 2018 12:24:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=7L6GRv8XGzkdEJ4/sYfyFR8klFyTmaI0c6m1lnrIAh4=; b=n5fnWXdvhBHo8lIDMfG7z0ucInqovicwPzOhUiEEmi1Y9ZAE6lxtkk5yQ9nQvOYPjo UsxMyNXUlF5uDFEdVjiZ0hPt8gppk4tUZ9nCZdfPSJVmHbAmw4dR7rm1ex3Xyh+IQBy4 Fiq290rY3EIOb0b1Vb6J9t0QdRxt9udeMSUWAZSeToHCHw8ZpsCkueDqMNVpJ6Llzquj pgfKF5Xt4Iv3zEFUxLJZzVqoir6gNcsW+b5BImmLuOMRNerpwDgfczynkdhT3KaLdWo7 9LeVwkRY9xDRHTYsJ4SRJSk5VEwipqt4qY10BWNSgqyiIVaErDWfSoJ70zTqWzTVdfMP ic6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=7L6GRv8XGzkdEJ4/sYfyFR8klFyTmaI0c6m1lnrIAh4=; b=eBFy4JCbwAokF02mXVRMKsw3z0lrxBsPlXEuF+Q7qAx6Sx+Cr6UpiDpG3BxD8R3iaJ GVvDF6gYMUqvRn/2W1bK+s+WvW5Eoh+1whKs9mkkhDbBlsdloAXqy83VbdFPA22TyNWi 1zIojsdrNe0P4/DfSjauUNmY9B0Nsg33FEZxDKjNr8L3LXvbhmHaz8jJdyGA2EtRDbQZ X/0QrNUJ6Zh4laOvEjB/zBMmcmIEZcuusKKX7GrZ31CMIPnUAnwOsgMjXJ4jNSJGyQ81 VqEciIODOo4BW5sJtpigS0l5CSqHcrzFaUI8c4gWDXvCpBvzYyU9Meytfl9Lt7l29GTg lZDw== X-Gm-Message-State: ABuFfogCNWY9vVLcnVRDdhIGBhR5VldcLOyjT5UMM7pAcoqwrtjoK+RP 4FIDiD6tawm38IYCJFYHup62yPHLOjBWKZvZF7McK/sxG/mKftBAEapdkOWJto1T8wzmJ+0fDAl 4WJvaQAC5Mzi9fRB1tt0WOsg9Mq0Bvc//aVVHkXb3v+De+EwlBCQZNPpunYXicZVCRsKciC0P7Q rg X-Google-Smtp-Source: ACcGV606snQ1EELSYBGCBwkSPVjzFGeqtPhzqpOzHOrM8ZJf2ATt8c/NOCClIeiluKXADv5FgG2Zqg6PeAU3BG//U2zI X-Received: by 2002:a37:9ed0:: with SMTP id h199-v6mr4518748qke.11.1538076265067; Thu, 27 Sep 2018 12:24:25 -0700 (PDT) Date: Thu, 27 Sep 2018 12:24:04 -0700 In-Reply-To: Message-Id: <1baf4254c928865ca83f35a1b6ed2d2a9ffcc941.1538075680.git.jonathantanmy@google.com> Mime-Version: 1.0 References: <20180925225355.74237-1-jonathantanmy@google.com> X-Mailer: git-send-email 2.19.0.605.g01d371f741-goog Subject: [RFC PATCH v2 1/4] transport: allow skipping of ref listing From: Jonathan Tan To: git@vger.kernel.org Cc: Jonathan Tan , gitster@pobox.com Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The get_refs_via_connect() function both performs the handshake (including determining the protocol version) and obtaining the list of remote refs. However, the fetch protocol v2 supports fetching objects without the listing of refs, so make it possible for the user to skip the listing by creating a new handshake() function. This will be used in a subsequent commit. --- transport.c | 29 +++++++++++++++++++++++++---- 1 file changed, 25 insertions(+), 4 deletions(-) diff --git a/transport.c b/transport.c index 1c76d64aba..5fb9ff6b56 100644 --- a/transport.c +++ b/transport.c @@ -252,8 +252,18 @@ static int connect_setup(struct transport *transport, int for_push) return 0; } -static struct ref *get_refs_via_connect(struct transport *transport, int for_push, - const struct argv_array *ref_prefixes) +/* + * Obtains the protocol version from the transport and writes it to + * transport->data->version, first connecting if not already connected. + * + * If the protocol version is one that allows skipping the listing of remote + * refs, and must_list_refs is 0, the listing of remote refs is skipped and + * this function returns NULL. Otherwise, this function returns the list of + * remote refs. + */ +static struct ref *handshake(struct transport *transport, int for_push, + const struct argv_array *ref_prefixes, + int must_list_refs) { struct git_transport_data *data = transport->data; struct ref *refs = NULL; @@ -268,8 +278,10 @@ static struct ref *get_refs_via_connect(struct transport *transport, int for_pus data->version = discover_version(&reader); switch (data->version) { case protocol_v2: - get_remote_refs(data->fd[1], &reader, &refs, for_push, - ref_prefixes, transport->server_options); + if (must_list_refs) + get_remote_refs(data->fd[1], &reader, &refs, for_push, + ref_prefixes, + transport->server_options); break; case protocol_v1: case protocol_v0: @@ -283,9 +295,18 @@ static struct ref *get_refs_via_connect(struct transport *transport, int for_pus } data->got_remote_heads = 1; + if (reader.line_peeked) + BUG("buffer must be empty at the end of handshake()"); + return refs; } +static struct ref *get_refs_via_connect(struct transport *transport, int for_push, + const struct argv_array *ref_prefixes) +{ + return handshake(transport, for_push, ref_prefixes, 1); +} + static int fetch_refs_via_pack(struct transport *transport, int nr_heads, struct ref **to_fetch) { From patchwork Thu Sep 27 19:24:05 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Tan X-Patchwork-Id: 10618501 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F40EF913 for ; Thu, 27 Sep 2018 19:24:35 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E1BE7285A6 for ; Thu, 27 Sep 2018 19:24:35 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D5FC42886E; Thu, 27 Sep 2018 19:24:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CB491285A6 for ; Thu, 27 Sep 2018 19:24:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728586AbeI1BoU (ORCPT ); Thu, 27 Sep 2018 21:44:20 -0400 Received: from mail-qk1-f201.google.com ([209.85.222.201]:56841 "EHLO mail-qk1-f201.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728101AbeI1BoU (ORCPT ); Thu, 27 Sep 2018 21:44:20 -0400 Received: by mail-qk1-f201.google.com with SMTP id v65-v6so3584120qka.23 for ; Thu, 27 Sep 2018 12:24:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=4/aVI0SPMVCxN+uLZb4o+JMD//324UsaUVHgk9AeVQI=; b=cuQR47NXoLa2NN/xPvfoUxbUpXoru64tXGJNsw4oYuFJ3lg88r0j8DuqhoTG0O4mpI 4BB11aN1rDuJd6nLQ3mU/c6ohLnsGlsepnJ/r1c5xhNZ3TorsLRZ7fWhuxTpQqU3a/mr i7nN8b41bU9nBA7iof+d+GYLyJSqzqWGzWOwBaq8DULcst/Rmbi9RrGzrFzJCQY9d3Li t5rcdtNAmJMOr5MiQ4gkGu4YG56VCdu3szV7JJ/gqzbrbgKw1CnDvxU0kGz126ZCyXF/ WHHIzXCqFDo8q8NtS57nGutHqeEh5xsgY6+TUZelTUmwEWUV/x2CWqsJcqiZmXarfA7f trrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=4/aVI0SPMVCxN+uLZb4o+JMD//324UsaUVHgk9AeVQI=; b=FfK64qo3/PPl3oXyekNkq6MCni3GhQryAfxwj24E2xpTpmhLGhIEh/+KdyV45sPLoE EiBhu1avWMKWxd8eounq1qW2kqb7MqkxelkLM2akjIE8lyCv8UySUWmlyxdjQPrHA/Sc SvLkE7LlMloYjHydUD64dXJgjKGt0e3G8jJivfn31bNJQHCiviAWXVCqslBAciXzofgg IqtMd0Ef8H4E6OnFKuccZhjmeVX5Ubc6Zzg6xUKqkRQQbu028ef90EvkclIRfaA6w6s6 a5qZuNSoDaw+ko4VfvMt0K809dw6IqPHeCUpdL9OfX3ciWDDeBPasytyzjnBW4RrO43k eLEg== X-Gm-Message-State: ABuFfoijdcX0JzE4vsO3301VJWtwncMLu5tmJzXWtivHhNnJnUwcMJfb A6nX2eeUsz7Twitn1Ji8ABFSZhNh1TUmX9wsAbine9sAi/Ysfq28pSG7gdDm3QIzT5l/GTXAOMQ VvoExYt5TGCx4RcSRST8TJxxcQJKUgiuRBRS0AktrgVYiChKcP4nMVjYU2DfTjeBV67Jh3EBFZA Rp X-Google-Smtp-Source: ACcGV62uiLlhiRLrTruNefd6qffC67Hx4UY5p47QsRGW10FE29xl/+wzlcZg0AeGDRuXi2Ppc7x5CYkQ/p5jJwzHtQsI X-Received: by 2002:a0c:a8cd:: with SMTP id h13-v6mr3329873qvc.39.1538076272057; Thu, 27 Sep 2018 12:24:32 -0700 (PDT) Date: Thu, 27 Sep 2018 12:24:05 -0700 In-Reply-To: Message-Id: <61ad64245470e51cb97988e0f2f5ea76c9e2276c.1538075680.git.jonathantanmy@google.com> Mime-Version: 1.0 References: <20180925225355.74237-1-jonathantanmy@google.com> X-Mailer: git-send-email 2.19.0.605.g01d371f741-goog Subject: [RFC PATCH v2 2/4] transport: do not list refs if possible From: Jonathan Tan To: git@vger.kernel.org Cc: Jonathan Tan , gitster@pobox.com Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP When all refs to be fetched are exact OIDs, it is possible to perform a fetch without requiring the remote to list refs if protocol v2 is used. Teach Git to do this. This currently has an effect only for lazy fetches done from partial clones. The change necessary to likewise optimize "git fetch " will be done in a subsequent patch. Signed-off-by: Jonathan Tan --- fetch-pack.c | 2 +- t/t5702-protocol-v2.sh | 5 +++++ transport.c | 13 +++++++++++-- 3 files changed, 17 insertions(+), 3 deletions(-) diff --git a/fetch-pack.c b/fetch-pack.c index 75047a4b2a..15652b4776 100644 --- a/fetch-pack.c +++ b/fetch-pack.c @@ -1598,7 +1598,7 @@ struct ref *fetch_pack(struct fetch_pack_args *args, if (nr_sought) nr_sought = remove_duplicates_in_refs(sought, nr_sought); - if (!ref) { + if (version != protocol_v2 && !ref) { packet_flush(fd[1]); die(_("no matching remote head")); } diff --git a/t/t5702-protocol-v2.sh b/t/t5702-protocol-v2.sh index 3beeed4546..a316bb9bf4 100755 --- a/t/t5702-protocol-v2.sh +++ b/t/t5702-protocol-v2.sh @@ -286,6 +286,11 @@ test_expect_success 'dynamically fetch missing object' ' grep "version 2" trace ' +test_expect_success 'when dynamically fetching missing object, do not list refs' ' + cat trace && + ! grep "git> command=ls-refs" trace +' + test_expect_success 'partial fetch' ' rm -rf client "$(pwd)/trace" && git init client && diff --git a/transport.c b/transport.c index 5fb9ff6b56..4329cca8e5 100644 --- a/transport.c +++ b/transport.c @@ -341,8 +341,17 @@ static int fetch_refs_via_pack(struct transport *transport, args.server_options = transport->server_options; args.negotiation_tips = data->options.negotiation_tips; - if (!data->got_remote_heads) - refs_tmp = get_refs_via_connect(transport, 0, NULL); + if (!data->got_remote_heads) { + int i; + int must_list_refs = 0; + for (i = 0; i < nr_heads; i++) { + if (!to_fetch[i]->exact_oid) { + must_list_refs = 1; + break; + } + } + refs_tmp = handshake(transport, 0, NULL, must_list_refs); + } switch (data->version) { case protocol_v2: From patchwork Thu Sep 27 19:24:06 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Tan X-Patchwork-Id: 10618503 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D1DF8913 for ; Thu, 27 Sep 2018 19:24:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BFE9A285B0 for ; Thu, 27 Sep 2018 19:24:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B448C28AC1; Thu, 27 Sep 2018 19:24:37 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 30A3D285B0 for ; Thu, 27 Sep 2018 19:24:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728658AbeI1BoX (ORCPT ); Thu, 27 Sep 2018 21:44:23 -0400 Received: from mail-qk1-f202.google.com ([209.85.222.202]:42183 "EHLO mail-qk1-f202.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728101AbeI1BoW (ORCPT ); Thu, 27 Sep 2018 21:44:22 -0400 Received: by mail-qk1-f202.google.com with SMTP id z17-v6so3689515qka.9 for ; Thu, 27 Sep 2018 12:24:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=yW07xfrG9F09zjKIND8N4BMCzKFoSNM01OoNXDTquLc=; b=SccTr4aMktpsVWJoFdU8wfjE5KM3koyDwLFweGQM7Li9Vu+XhzHHJzNhpiKA7kAW0z 9pVV433AWg1BjNmBFVii7cJAadylEo0GXxvzqjdWatH4yo/5U5UULEkHSXTcJgvmxhyZ L1+fv6zQ/jeIPPIXpTVluoYMTFfDhwrUZPw5dSZJbIFOTIXSHHyOZIfCXxSXNQs3CezB IcO2BcBryTUdPXbSfGHIpDIZ7O9DBy83l0OVjgCQGdLJYZ+jLjorAbpX4AO2n2/dBGpR W9DmIU7Ly2IXpe4xE8tK45/PHFEQEhLvgK/pMqFS088tA3VYVC0mRLgSVXs30kqmuTwm hUuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=yW07xfrG9F09zjKIND8N4BMCzKFoSNM01OoNXDTquLc=; b=IAmP8AVfVzAP/Ypfuu93sOzyraNm1nxpZvZIViQt/HAdGTm9AcG95unq2EzE+cdX+b Aw6lEMdSEjaa9FtF6B/E1nRNMCKOrceDGucBMHfQSDs/iSxlLRI7xZDpaKAIHlpsNVS+ QgkyUgdsxEdwflLCEAc7PybpAgpykWYa659olWDQFSmp7HW4sbnizp8BvmuQ4c3h0g4N pXl3c02E432h5dQQZgJsu9upLMt+RB+swux0I9Fw9AQM5eaJ5lmAD9aWIH39QeGlaQIc CQYV6RLBfLnu/i9D1EzYsszCXyig1ZQWgW3VvTfDbMHPBAQBNspDY+t3bbm1Zj1cCmj5 hxwQ== X-Gm-Message-State: ABuFfohUYOT2PC1xRvRsCzF12jXHW95MAYzPG2gX8PKHiknoYDe5Zolf VM05YQ7EJiMkseP2CEZQPxyrBawo4cRYfKhS0d4ebYxC3T1BEUjLzHnOLDzsIH89VlHnOGWFrl+ /WLddq3uF4OK/zWEglvXaWpSgL2vcwZtUy6fcT+l48z5YnzZyBvlBLHYoFLvrPzJmrzKdS2/y/Z +p X-Google-Smtp-Source: ACcGV63vqdsT5sWGUTx3OySINNSlDbA+g38lzJTspovYMMYlkONWNQ8qLdN2y2tGBdxNO8vojuFY7N18opkNJmAkY7ko X-Received: by 2002:a0c:d2b5:: with SMTP id q50-v6mr5565697qvh.1.1538076274678; Thu, 27 Sep 2018 12:24:34 -0700 (PDT) Date: Thu, 27 Sep 2018 12:24:06 -0700 In-Reply-To: Message-Id: <33b22cf96ee27053a6de22fdfb48f58cc8012644.1538075680.git.jonathantanmy@google.com> Mime-Version: 1.0 References: <20180925225355.74237-1-jonathantanmy@google.com> X-Mailer: git-send-email 2.19.0.605.g01d371f741-goog Subject: [RFC PATCH v2 3/4] transport: list refs before fetch if necessary From: Jonathan Tan To: git@vger.kernel.org Cc: Jonathan Tan , gitster@pobox.com Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The built-in bundle transport and the transport helper interface do not work when transport_fetch_refs() is called immediately after transport creation. This will be needed in a subsequent patch, so fix this. Evidence: fetch_refs_from_bundle() relies on data->header being initialized in get_refs_from_bundle(), and fetch() in transport-helper.c relies on either data->fetch or data->import being set by get_helper(), but neither transport_helper_init() nor fetch() calls get_helper(). Up until the introduction of the partial clone feature, this has not been a problem, because transport_fetch_refs() is always called after transport_get_remote_refs(). With the introduction of the partial clone feature, which involves calling transport_fetch_refs() (to fetch objects by their OIDs) without transport_get_remote_refs(), this is still not a problem, but only coincidentally - we do not support partially cloning a bundle, and as for cloning using a transport-helper-using protocol, it so happens that before transport_fetch_refs() is called, fetch_refs() in fetch-object.c calls transport_set_option(), which means that the aforementioned get_helper() is invoked through set_helper_option() in transport-helper.c. This could be fixed by fixing the transports themselves, but it doesn't seem like a good idea to me to open up previously untested code paths; also, there may be transport helpers in the wild that assume that "list" is always called before "fetch". Instead, fix this by having transport_fetch_refs() call transport_get_remote_refs() to ensure that the latter is always called at least once, unless the transport explicitly states that it supports fetching without listing refs. Signed-off-by: Jonathan Tan --- transport-helper.c | 1 + transport-internal.h | 6 ++++++ transport.c | 12 ++++++++++++ 3 files changed, 19 insertions(+) diff --git a/transport-helper.c b/transport-helper.c index 143ca008c8..7213fa0d32 100644 --- a/transport-helper.c +++ b/transport-helper.c @@ -1105,6 +1105,7 @@ static struct ref *get_refs_list(struct transport *transport, int for_push, } static struct transport_vtable vtable = { + 0, set_helper_option, get_refs_list, fetch, diff --git a/transport-internal.h b/transport-internal.h index 1cde6258a7..004bee5e36 100644 --- a/transport-internal.h +++ b/transport-internal.h @@ -6,6 +6,12 @@ struct transport; struct argv_array; struct transport_vtable { + /** + * This transport supports the fetch() function being called + * without get_refs_list() first being called. + */ + unsigned fetch_without_list : 1; + /** * Returns 0 if successful, positive if the option is not * recognized or is inapplicable, and negative if the option diff --git a/transport.c b/transport.c index 4329cca8e5..ea72fff6a6 100644 --- a/transport.c +++ b/transport.c @@ -733,6 +733,7 @@ static int disconnect_git(struct transport *transport) } static struct transport_vtable taken_over_vtable = { + 1, NULL, get_refs_via_connect, fetch_refs_via_pack, @@ -882,6 +883,7 @@ void transport_check_allowed(const char *type) } static struct transport_vtable bundle_vtable = { + 0, NULL, get_refs_from_bundle, fetch_refs_from_bundle, @@ -891,6 +893,7 @@ static struct transport_vtable bundle_vtable = { }; static struct transport_vtable builtin_smart_vtable = { + 1, NULL, get_refs_via_connect, fetch_refs_via_pack, @@ -1254,6 +1257,15 @@ int transport_fetch_refs(struct transport *transport, struct ref *refs) struct ref **heads = NULL; struct ref *rm; + if (!transport->vtable->fetch_without_list) + /* + * Some transports (e.g. the built-in bundle transport and the + * transport helper interface) do not work when fetching is + * done immediately after transport creation. List the remote + * refs anyway (if not already listed) as a workaround. + */ + transport_get_remote_refs(transport, NULL); + for (rm = refs; rm; rm = rm->next) { nr_refs++; if (rm->peer_ref && From patchwork Thu Sep 27 19:24:07 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Tan X-Patchwork-Id: 10618505 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DDD1C112B for ; Thu, 27 Sep 2018 19:24:40 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CBD30285A6 for ; Thu, 27 Sep 2018 19:24:40 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C042028796; Thu, 27 Sep 2018 19:24:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9AC2D285B0 for ; Thu, 27 Sep 2018 19:24:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728680AbeI1BoZ (ORCPT ); Thu, 27 Sep 2018 21:44:25 -0400 Received: from mail-io1-f73.google.com ([209.85.166.73]:39832 "EHLO mail-io1-f73.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728101AbeI1BoZ (ORCPT ); Thu, 27 Sep 2018 21:44:25 -0400 Received: by mail-io1-f73.google.com with SMTP id x5-v6so4069231ioa.6 for ; Thu, 27 Sep 2018 12:24:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=VIcC7tx9B4oUDsvz6GmX4ErQ3TOv3BnJOy8jvlzqb14=; b=brPbUatG9pYoLS/qNOiRpkexsxqyxrNyvki+E4RAfzM8Ocr+317dZZV0qDTPKZJdOF yiK1o09sH0FbTu+Ye5csJ6GeRpLGRsubo+TDEyMTUEPcV7suseY1Tv7NXvVg0b/7DUOh iEh8pM8j89g2rBZQAL8AB3HEIbPw75HfBDJn1vd/7ZpSiWxmAOkPCP+haprkybPWIoox DMBtpV2piy/UF0n1t8aXjrM/ogxtEf0ga/6n6emqnBWUaSpoZwAiEV42AbuUGdeRdocg gijKtzzaRl8ivfjn8KtY9I9TpaDL6Iipo/Cu2hJtu1sMhu5bE4JSXGKC++LY5wZ8wMby AXGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=VIcC7tx9B4oUDsvz6GmX4ErQ3TOv3BnJOy8jvlzqb14=; b=WqKjL8yo+QBtsbZ5l32mP80n2dbXz7/Cc5b5lEXeu8f0Fmztfa4TSQ8m6SbbK3/LJo qzIzUJLdnxx0kw0eBprcla+YZ+GKvZfVOOzPfbDFEZ0HKPUdamLwByKjWCoKe+zcWBO2 y3zzqKMVn7vwGnFue2tq7mYBlsusosuvPNAvZwp3SeixQOtHjOKD3pLArzLzj2w8jSb0 KzhjpGjgPCn3Fk39Vix7uDosthtBp9X5ERqXAsVVnfmR0TaTTcXIORcgwJK0K0uM05UE AfsrPx3XcRsJNdoo7d+pnjN5aIh9vT1srzp5b5gHZAlYSH7WB4g0WaCVoKo8eOo2IUKP CUKw== X-Gm-Message-State: ABuFfohIWrJKnpV4EkasxBDUtRYH5PPFYMsb7QwVroMOPydfYEeObIb7 SDPrJiaf1MwGZP4MFdtalFumIX7MRD/KsvX57xUKEo2RVX+u8EhaWPl+n5XB0n8dEWwYDlFc28X vnt3JfTt6aDmEaIXiJXoXwArh2HPY5JWSrrIAdvJVHLBVwh/ruyW05VmVYB98QsL+4xvjSXP/nj HF X-Google-Smtp-Source: ACcGV6087iMtpkHyRQk71EqPY3CHj599q7ShpdMvBPb98/K0zqeU2cObGEruWOsoCW/0OZxcHfkAipfynxRzcfhljgcK X-Received: by 2002:a24:f585:: with SMTP id k127-v6mr7773148ith.1.1538076276901; Thu, 27 Sep 2018 12:24:36 -0700 (PDT) Date: Thu, 27 Sep 2018 12:24:07 -0700 In-Reply-To: Message-Id: <1ae00ea1fdd1118b92ac90d67f27a988750b60f2.1538075680.git.jonathantanmy@google.com> Mime-Version: 1.0 References: <20180925225355.74237-1-jonathantanmy@google.com> X-Mailer: git-send-email 2.19.0.605.g01d371f741-goog Subject: [RFC PATCH v2 4/4] fetch: do not list refs if fetching only hashes From: Jonathan Tan To: git@vger.kernel.org Cc: Jonathan Tan , gitster@pobox.com Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP If only hash literals are given on a "git fetch" command-line, tag following is not requested, and the fetch is done using protocol v2, a list of refs is not required from the remote. Therefore, optimize by invoking transport_get_remote_refs() only if we need the refs. Signed-off-by: Jonathan Tan --- builtin/fetch.c | 32 ++++++++++++++++++++++++++------ t/t5551-http-fetch-smart.sh | 15 +++++++++++++++ t/t5702-protocol-v2.sh | 13 +++++++++++++ 3 files changed, 54 insertions(+), 6 deletions(-) diff --git a/builtin/fetch.c b/builtin/fetch.c index 0696abfc2a..4c4f8fa194 100644 --- a/builtin/fetch.c +++ b/builtin/fetch.c @@ -1175,6 +1175,7 @@ static int do_fetch(struct transport *transport, int retcode = 0; const struct ref *remote_refs; struct argv_array ref_prefixes = ARGV_ARRAY_INIT; + int must_list_refs = 1; if (tags == TAGS_DEFAULT) { if (transport->remote->fetch_tags == 2) @@ -1190,17 +1191,36 @@ static int do_fetch(struct transport *transport, goto cleanup; } - if (rs->nr) + if (rs->nr) { + int i; + refspec_ref_prefixes(rs, &ref_prefixes); - else if (transport->remote && transport->remote->fetch.nr) + + /* + * We can avoid listing refs if all of them are exact + * OIDs + */ + must_list_refs = 0; + for (i = 0; i < rs->nr; i++) { + if (!rs->items[i].exact_sha1) { + must_list_refs = 1; + break; + } + } + } else if (transport->remote && transport->remote->fetch.nr) refspec_ref_prefixes(&transport->remote->fetch, &ref_prefixes); - if (ref_prefixes.argc && - (tags == TAGS_SET || (tags == TAGS_DEFAULT))) { - argv_array_push(&ref_prefixes, "refs/tags/"); + if (tags == TAGS_SET || tags == TAGS_DEFAULT) { + must_list_refs = 1; + if (ref_prefixes.argc) + argv_array_push(&ref_prefixes, "refs/tags/"); } - remote_refs = transport_get_remote_refs(transport, &ref_prefixes); + if (must_list_refs) + remote_refs = transport_get_remote_refs(transport, &ref_prefixes); + else + remote_refs = NULL; + argv_array_clear(&ref_prefixes); ref_map = get_ref_map(transport->remote, remote_refs, rs, diff --git a/t/t5551-http-fetch-smart.sh b/t/t5551-http-fetch-smart.sh index 771f36f9ff..12b339d239 100755 --- a/t/t5551-http-fetch-smart.sh +++ b/t/t5551-http-fetch-smart.sh @@ -381,6 +381,21 @@ test_expect_success 'using fetch command in remote-curl updates refs' ' test_cmp expect actual ' +test_expect_success 'fetch by SHA-1 without tag following' ' + SERVER="$HTTPD_DOCUMENT_ROOT_PATH/server" && + rm -rf "$SERVER" client && + + git init "$SERVER" && + test_commit -C "$SERVER" foo && + + git clone $HTTPD_URL/smart/server client && + + test_commit -C "$SERVER" bar && + git -C "$SERVER" rev-parse bar >bar_hash && + git -C client -c protocol.version=0 fetch \ + --no-tags origin $(cat bar_hash) +' + test_expect_success 'GIT_REDACT_COOKIES redacts cookies' ' rm -rf clone && echo "Set-Cookie: Foo=1" >cookies && diff --git a/t/t5702-protocol-v2.sh b/t/t5702-protocol-v2.sh index a316bb9bf4..1a97331648 100755 --- a/t/t5702-protocol-v2.sh +++ b/t/t5702-protocol-v2.sh @@ -79,6 +79,19 @@ test_expect_success 'fetch with git:// using protocol v2' ' grep "fetch< version 2" log ' +test_expect_success 'fetch by hash without tag following with protocol v2 does not list refs' ' + test_when_finished "rm -f log" && + + test_commit -C "$daemon_parent" two_a && + git -C "$daemon_parent" rev-parse two_a >two_a_hash && + + GIT_TRACE_PACKET="$(pwd)/log" git -C daemon_child -c protocol.version=2 \ + fetch --no-tags origin $(cat two_a_hash) && + + grep "fetch< version 2" log && + ! grep "fetch> command=ls-refs" log +' + test_expect_success 'pull with git:// using protocol v2' ' test_when_finished "rm -f log" &&