From patchwork Wed Aug 5 01:20:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Tan X-Patchwork-Id: 11701131 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BFC4B6C1 for ; Wed, 5 Aug 2020 01:21:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B62E8208C7 for ; Wed, 5 Aug 2020 01:21:11 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="EJAIoUTW" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727800AbgHEBVI (ORCPT ); Tue, 4 Aug 2020 21:21:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44518 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726981AbgHEBU6 (ORCPT ); Tue, 4 Aug 2020 21:20:58 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3BACDC06174A for ; Tue, 4 Aug 2020 18:20:58 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id a5so38104998ybh.3 for ; Tue, 04 Aug 2020 18:20:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=NGOwp5Ax/Dj/W31vyP2rRIH9UgY9nuf4W9Gq8qSK26Y=; b=EJAIoUTW9EVPZ3QgbGtxzAYNKR7lvALo17R6amFNM8eeXBGZ2zhw+HtZa/JYT0+cA4 8iP8OmkarwIJ7Bf+0MA5AgF6NOgYNKSoHJfLvm+Y2A+klw06HGqGV9wCNn7ync2JYEer jH2MJnLlnzap9pTtIWSHnsAxKJ/HJHaIa8WFV9oyGftvyfAPedusAynNHMiXe9Ze1Idv 3V/pR2BeZ/5UrLwxbaJR5kAfgSY3+HsgSvntXevjTo8NbnIePjB4KcjTM1w6d/mOK0Bm KB16d4omE+VhaWL9Q3VQO/RQORaP/oMTmMhCUC2+UQsA/SX6wy62vQZI0AGcBb6470VK Czig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=NGOwp5Ax/Dj/W31vyP2rRIH9UgY9nuf4W9Gq8qSK26Y=; b=VPrcRdE8z8VjAV5WHX1cFUiioJjrR5YTtoPm2kj9oyFQuFYdYPhCNzOF3JwAXES3eK OUUBOKCvg2aQ/6RXNtbf3CVLLaz1aSAOvrBBLKpk2JIR+sv06uahY7E8V6BmTTxt4Cj7 0bsMsqQd7HH0AcziTq9QHIbkqseYePXKeedgiCPgG5vGR9lFmNDjrGjah8qBawy6pXZQ 9wBdeUvODCwb2n2KR5vLPelIlB9qPE0Ps8QXPxngERJfuz/UDj6hvj1OiMrR1nEc9TMc rDemt2l5+OcaonxJH77kcQOs+9GfbpOk5qYq0SDf87K6XicXUo9d6/cqY17K0Muu4/8v qgfg== X-Gm-Message-State: AOAM532Z6prfZ14WavvAUxaTmG5KKQFZ4Ft5S+5ipNTlj5t2rpZnJHKF 1+8bjvoeLFridK8Vgz55IEOOMx5mXlJVsYgrutBIlti4L4S9iduEuzoAlsvku/8QpL+0l3VE3WU d5YqiKpcFEMjFKrYa/gAFoZG3E/SrNobxpllVroS9iNDHVvi8cQ64zGkII4UOlMcXJdwbO0NcG1 b6 X-Google-Smtp-Source: ABdhPJy5kn1PUCyTyggXJt6+Ee8Y+rZPi/WDTzcgg6R2MopNIsae6/n5hHicEZLuZg/nEdgzvtM8sYRzjEqQEmRkFplc X-Received: by 2002:a25:3446:: with SMTP id b67mr1161420yba.460.1596590457393; Tue, 04 Aug 2020 18:20:57 -0700 (PDT) Date: Tue, 4 Aug 2020 18:20:44 -0700 In-Reply-To: Message-Id: <044a782863dd690ed4e430b3793538f701a850c6.1596590295.git.jonathantanmy@google.com> Mime-Version: 1.0 References: <20200724223844.2723397-1-jonathantanmy@google.com> X-Mailer: git-send-email 2.28.0.163.g6104cc2f0b6-goog Subject: [RFC PATCH 1/7] fetch-pack: allow NULL negotiator->add_tip From: Jonathan Tan To: git@vger.kernel.org Cc: Jonathan Tan , peff@peff.net, gitster@pobox.com Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In a subsequent patch, a null fetch negotiator will be introduced. This negotiator, among other things, will not need any tip information and will have a NULL add_tip. Teach fetch-pack to allow this, and to take advantage of this by avoiding a ref and alternate traversal when it can. To do this, this patch combines all invocations of functions that invoke add_tip into one function, and that function first checks whether add_tip is NULL. Signed-off-by: Jonathan Tan --- fetch-pack.c | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/fetch-pack.c b/fetch-pack.c index 80fb3bd899..6c786f5970 100644 --- a/fetch-pack.c +++ b/fetch-pack.c @@ -240,6 +240,16 @@ static void mark_tips(struct fetch_negotiator *negotiator, return; } +static void add_tips_and_alternates(struct fetch_negotiator *negotiator, + const struct oid_array *negotiation_tips) +{ + if (!negotiator->add_tip) + return; + + mark_tips(negotiator, negotiation_tips); + for_each_cached_alternate(negotiator, insert_one_alternate_object); +} + static int find_common(struct fetch_negotiator *negotiator, struct fetch_pack_args *args, int fd[2], struct object_id *result_oid, @@ -262,10 +272,8 @@ static int find_common(struct fetch_negotiator *negotiator, PACKET_READ_CHOMP_NEWLINE | PACKET_READ_DIE_ON_ERR_PACKET); - if (!args->no_dependents) { - mark_tips(negotiator, args->negotiation_tips); - for_each_cached_alternate(negotiator, insert_one_alternate_object); - } + if (!args->no_dependents) + add_tips_and_alternates(negotiator, args->negotiation_tips); fetching = 0; for ( ; refs ; refs = refs->next) { @@ -1575,9 +1583,7 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args, else state = FETCH_SEND_REQUEST; - mark_tips(negotiator, args->negotiation_tips); - for_each_cached_alternate(negotiator, - insert_one_alternate_object); + add_tips_and_alternates(negotiator, args->negotiation_tips); } else { filter_refs(args, &ref, sought, nr_sought); state = FETCH_SEND_REQUEST; From patchwork Wed Aug 5 01:20:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Tan X-Patchwork-Id: 11701135 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 404B913B1 for ; Wed, 5 Aug 2020 01:21:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 358B62086A for ; Wed, 5 Aug 2020 01:21:16 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="gL9tjknU" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727817AbgHEBVM (ORCPT ); Tue, 4 Aug 2020 21:21:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44528 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727023AbgHEBVA (ORCPT ); Tue, 4 Aug 2020 21:21:00 -0400 Received: from mail-pl1-x649.google.com (mail-pl1-x649.google.com [IPv6:2607:f8b0:4864:20::649]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4B51BC061756 for ; Tue, 4 Aug 2020 18:21:00 -0700 (PDT) Received: by mail-pl1-x649.google.com with SMTP id ba2so31259777plb.0 for ; Tue, 04 Aug 2020 18:21:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=y3TJIPgSlOhpPXMQgeJ+p+hOSwozEm0cC28n1jZmGVc=; b=gL9tjknUCigwoKaOLFhouvIRhJBrT1ARbuqe86JGMI/hgA3041j/dfiW2nKk4mR/Eh sGUv0ezAgd7FkP+awRSvoxNYoh7vkRUMu4+7L9pJKX+zlWlCShrOMWm/uuwH+DFdKibv 5dO3DR+Oh8I4QmOwuNsbN9LplCNMkuhlv2DCCcPNAs42IKQ4TuHhY2NX40RgpT9SBRPE KrOPYB7UN5h2f09oAGi7qEoC6AhDfU57jwgYocah2TIfdKXp/8ekjYqppVQLWtOLYz2/ F5NIKuRut7yMXks+tHlBOfFry1zKIRO7OTuz5hIBKw4DOu4lOzOyUjx7ccIxMYK1S2I+ u3qw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=y3TJIPgSlOhpPXMQgeJ+p+hOSwozEm0cC28n1jZmGVc=; b=DtEdlrSN7Je85WRr0mVdpcoboVpD+aIFB+AKidmre8+a8MtEn4j7gG1tjJUHykFcre WMbFs00/OrGRhDo2bScxLxr0iHEk09Y0yJl3fx7mSH5wCuqArogNjuGJAjwz6lTi4hb4 /KImBvMefr8uEsL2abmzMZPQW5dSnKIjB5LlnmEnmpJhwB/Y1Laq9C5tKIr5R5shyQLe me1TFFpPwqll82K5aojeZoX9G1q58DnHA08LvH9VymHIqe4dYyVa5sEKD6JEnBeKgZV6 StkOjOJELXTNWTWPSWqWuZkpyq+T2VOlg1dc9wTR0ZO6AREoLkwKPBdUPlLx1OHkBqfJ pdyg== X-Gm-Message-State: AOAM530DxdYx5OkgGqfbkUNbPEc2WaJPYFVbyiX0XuNB4luXL1N/fdlq ajQuRqO+6SnT4sR8RrWSzKVL5DyXwwV5UoqfbtFqaZFSL4coNp6VUtZPlyXBYQ2bMmuUGsbyjsp xLPImvm01gnDIdgVNAgU/lQxl+LuYJcEGrMWoWRLhQQt2lgNQa8eH+EsfUEuK/6ZiuEAjEn8s5/ 8F X-Google-Smtp-Source: ABdhPJzobJ/qUqqj45o/JLArtGwu2pnjPWfe0z1DxY0AQOs8Z01v6mE6Ks4NZ3cp5Kc39QTVaEHfRC4zbiBW68tXJ74P X-Received: by 2002:a17:90a:f014:: with SMTP id bt20mr746912pjb.0.1596590459026; Tue, 04 Aug 2020 18:20:59 -0700 (PDT) Date: Tue, 4 Aug 2020 18:20:45 -0700 In-Reply-To: Message-Id: <401227c2220b6b45d80e21b52e29b6821ca139f9.1596590295.git.jonathantanmy@google.com> Mime-Version: 1.0 References: <20200724223844.2723397-1-jonathantanmy@google.com> X-Mailer: git-send-email 2.28.0.163.g6104cc2f0b6-goog Subject: [RFC PATCH 2/7] fetch-pack: allow NULL negotiator->known_common From: Jonathan Tan To: git@vger.kernel.org Cc: Jonathan Tan , peff@peff.net, gitster@pobox.com Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In a subsequent patch, a null fetch negotiator will be introduced. This negotiator, among other things, will not need any information about common objects and will have a NULL known_common. Teach fetch-pack to allow this. [NEEDSWORK] Optimizing out the ref iteration also affects the execution of everything_local(), which relies on COMPLETE being set. (Having said that, the typical use case - lazy fetching - would be fine with everything_local() always returning that not everything is local.) This optimization is needed so that in the future, fetch_pack() can be used to lazily fetch in a partial clone (without the no_dependents flag). This means that fetch_pack() needs a way to execute without relying on any targets of refs being present, and thus it cannot use the ref iterator (because it checks and lazy-fetches any missing targets). (Git currently does not have this problem because we use the no_dependents flag, but lazy-fetching will in a subsequent patch be changed to use the user-facing fetch command, which does not use this flag.) [/NEEDSWORK] Signed-off-by: Jonathan Tan --- fetch-pack.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/fetch-pack.c b/fetch-pack.c index 6c786f5970..5f5474dbed 100644 --- a/fetch-pack.c +++ b/fetch-pack.c @@ -677,6 +677,9 @@ static void mark_complete_and_common_ref(struct fetch_negotiator *negotiator, int old_save_commit_buffer = save_commit_buffer; timestamp_t cutoff = 0; + if (!negotiator->known_common) + return; + save_commit_buffer = 0; trace2_region_enter("fetch-pack", "parse_remote_refs_and_find_cutoff", NULL); From patchwork Wed Aug 5 01:20:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Tan X-Patchwork-Id: 11701133 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7B3A213B1 for ; Wed, 5 Aug 2020 01:21:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 69A432086A for ; Wed, 5 Aug 2020 01:21:15 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="X8u2N3B1" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727811AbgHEBVN (ORCPT ); Tue, 4 Aug 2020 21:21:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44532 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726891AbgHEBVB (ORCPT ); Tue, 4 Aug 2020 21:21:01 -0400 Received: from mail-pf1-x449.google.com (mail-pf1-x449.google.com [IPv6:2607:f8b0:4864:20::449]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5D9ADC061757 for ; Tue, 4 Aug 2020 18:21:01 -0700 (PDT) Received: by mail-pf1-x449.google.com with SMTP id x142so6319239pfc.14 for ; Tue, 04 Aug 2020 18:21:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=jQRk6XnMqgw0IvuWmUklA/LbL76Yl6tFeoZ4vnMo7tY=; b=X8u2N3B1mzBiiwOsvzP5DRguQkmXbGSI2CJx+FgoacVeRqTyIs6zsZ/yilNL3e3LYc PT2RehNDACrJb0j4E2cpABlNdtH8NEFF00Ug6D0zSzwtWiuQDYAsshX4jxKlBoCTwgs+ 8OlIjaRG6mxfqek7utbSX8mA6eiUzebkoackood2Gk9Fq/TR5N4Yx+8FO1Y/NxrZoSPZ fH1I7iRy7KPi7eUXUROVy2pCd/ArJDZ56qd4CdJ9F50cUsnUg6r1LW3lEZssp5gDaOQ+ wP6tbjxCXndBB8DDlCtPmm4XbBTDhlRG1Ja62lfjZ8sksupMvDpey3ApZhKR+tFEYoSH U3JA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=jQRk6XnMqgw0IvuWmUklA/LbL76Yl6tFeoZ4vnMo7tY=; b=o3zXPRkiJoVQOzs1zO9yJYWU5FdTvVMTziQmye61cmbe7HJ4sFAuIqzAoQgdwOBsYo XW2Eeq5TfmmwONjydWCtilL+Uf7461oLYVZ9M8Vh8+PFu84cjEl280CEed9ghhDvW9S3 weh2vuojmp9uwFLWW+naC/H5BgWUUyXEQ/O/rKrsuVvmTEs0vDp9R3avwYOlLsZVUYVY 0G9dYEnBxnZtSBRcKeJ4RedgUtmyITDkn2E72iF6EzMVqp8tqZg8bVKyTTlJ4ioncgi7 L6Im4/bRBN6uZzkybtTI0Dx4R7X40ewPutTdQ0Wvu8M+A7mC8pzDFENHKGxdM7aKElqI /E5Q== X-Gm-Message-State: AOAM532Mi4zxy8X7guWTZ/9/Q9Kpz5IyeO7SphxN2ET8s2W59jLIwJvF IIhknPnpIda3SVPJuoLj7NgoGLG5legwYAWqNMTjq1XliWysyQg8XvOMhCF4uNaJRZ/D31FLuqZ hEjTf1qMNad69DF1TuIkbEF8gq28OOgPAF2yTccZYS5l30b7JeGK1XIkFxEmHT6s0VaXQpSAxHH 6c X-Google-Smtp-Source: ABdhPJzHlXQ2KyocOpoJdDCBXJsfXHoW2cJNcdQqJ2AQuNMS7MwHG8HR5SPgILi4WCE9ek5lLRhl9n8LyRhOD27azszI X-Received: by 2002:aa7:947b:: with SMTP id t27mr985175pfq.117.1596590460751; Tue, 04 Aug 2020 18:21:00 -0700 (PDT) Date: Tue, 4 Aug 2020 18:20:46 -0700 In-Reply-To: Message-Id: <78d47204044dbe650cc782001de793a7cdf85692.1596590295.git.jonathantanmy@google.com> Mime-Version: 1.0 References: <20200724223844.2723397-1-jonathantanmy@google.com> X-Mailer: git-send-email 2.28.0.163.g6104cc2f0b6-goog Subject: [RFC PATCH 3/7] negotiator/null: add null fetch negotiator From: Jonathan Tan To: git@vger.kernel.org Cc: Jonathan Tan , peff@peff.net, gitster@pobox.com Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Add a null fetch negotiator. This is introduced to allow partial clones to perform lazy fetches (which do not need negotiation at all) by spawning a subprocess (to be done in a subsequent patch), but can be useful for end users, e.g. as a blunt fix for object corruption. Signed-off-by: Jonathan Tan --- Documentation/config/fetch.txt | 5 ++++- Makefile | 1 + fetch-negotiator.c | 5 +++++ negotiator/null.c | 34 ++++++++++++++++++++++++++++++++ negotiator/null.h | 8 ++++++++ repo-settings.c | 2 ++ repository.h | 1 + t/t5554-null-fetch-negotiator.sh | 22 +++++++++++++++++++++ 8 files changed, 77 insertions(+), 1 deletion(-) create mode 100644 negotiator/null.c create mode 100644 negotiator/null.h create mode 100755 t/t5554-null-fetch-negotiator.sh diff --git a/Documentation/config/fetch.txt b/Documentation/config/fetch.txt index 0aaa05e8c0..09aff404be 100644 --- a/Documentation/config/fetch.txt +++ b/Documentation/config/fetch.txt @@ -60,7 +60,10 @@ fetch.negotiationAlgorithm:: sent when negotiating the contents of the packfile to be sent by the server. Set to "skipping" to use an algorithm that skips commits in an effort to converge faster, but may result in a larger-than-necessary - packfile; The default is "default" which instructs Git to use the default algorithm + packfile; or set to "null" to not send any information at all, which + will almost certainly result in a larger-than-necessary packfile, but + will skip the negotiation step. + The default is "default" which instructs Git to use the default algorithm that never skips commits (unless the server has acknowledged it or one of its descendants). If `feature.experimental` is enabled, then this setting defaults to "skipping". diff --git a/Makefile b/Makefile index 372139f1f2..297ea5e517 100644 --- a/Makefile +++ b/Makefile @@ -917,6 +917,7 @@ LIB_OBJS += mergesort.o LIB_OBJS += midx.o LIB_OBJS += name-hash.o LIB_OBJS += negotiator/default.o +LIB_OBJS += negotiator/null.o LIB_OBJS += negotiator/skipping.o LIB_OBJS += notes-cache.o LIB_OBJS += notes-merge.o diff --git a/fetch-negotiator.c b/fetch-negotiator.c index 0a1357dc9d..e2ecbe4367 100644 --- a/fetch-negotiator.c +++ b/fetch-negotiator.c @@ -2,6 +2,7 @@ #include "fetch-negotiator.h" #include "negotiator/default.h" #include "negotiator/skipping.h" +#include "negotiator/null.h" #include "repository.h" void fetch_negotiator_init(struct repository *r, @@ -13,6 +14,10 @@ void fetch_negotiator_init(struct repository *r, skipping_negotiator_init(negotiator); return; + case FETCH_NEGOTIATION_NULL: + null_negotiator_init(negotiator); + return; + case FETCH_NEGOTIATION_DEFAULT: default: default_negotiator_init(negotiator); diff --git a/negotiator/null.c b/negotiator/null.c new file mode 100644 index 0000000000..db63ce66c2 --- /dev/null +++ b/negotiator/null.c @@ -0,0 +1,34 @@ +#include "cache.h" +#include "null.h" +#include "../commit.h" +#include "../fetch-negotiator.h" + +static const struct object_id *next(struct fetch_negotiator *n) +{ + return NULL; +} + +static int ack(struct fetch_negotiator *n, struct commit *c) +{ + /* + * This negotiator does not emit any commits, so there is no commit to + * be acknowledged. If there is any ack, there is a bug. + */ + BUG("ack with null negotiator, which does not emit any commits"); + return 0; +} + +static void release(struct fetch_negotiator *n) +{ + /* nothing to release */ +} + +void null_negotiator_init(struct fetch_negotiator *negotiator) +{ + negotiator->known_common = NULL; + negotiator->add_tip = NULL; + negotiator->next = next; + negotiator->ack = ack; + negotiator->release = release; + negotiator->data = NULL; +} diff --git a/negotiator/null.h b/negotiator/null.h new file mode 100644 index 0000000000..96713f280e --- /dev/null +++ b/negotiator/null.h @@ -0,0 +1,8 @@ +#ifndef NEGOTIATOR_NULL_H +#define NEGOTIATOR_NULL_H + +struct fetch_negotiator; + +void null_negotiator_init(struct fetch_negotiator *negotiator); + +#endif diff --git a/repo-settings.c b/repo-settings.c index 0918408b34..a8c7e1edd7 100644 --- a/repo-settings.c +++ b/repo-settings.c @@ -39,6 +39,8 @@ void prepare_repo_settings(struct repository *r) if (!repo_config_get_string(r, "fetch.negotiationalgorithm", &strval)) { if (!strcasecmp(strval, "skipping")) r->settings.fetch_negotiation_algorithm = FETCH_NEGOTIATION_SKIPPING; + else if (!strcasecmp(strval, "null")) + r->settings.fetch_negotiation_algorithm = FETCH_NEGOTIATION_NULL; else r->settings.fetch_negotiation_algorithm = FETCH_NEGOTIATION_DEFAULT; } diff --git a/repository.h b/repository.h index 3c1f7d54bd..f72911a185 100644 --- a/repository.h +++ b/repository.h @@ -23,6 +23,7 @@ enum fetch_negotiation_setting { FETCH_NEGOTIATION_NONE = 0, FETCH_NEGOTIATION_DEFAULT = 1, FETCH_NEGOTIATION_SKIPPING = 2, + FETCH_NEGOTIATION_NULL = 3, }; struct repo_settings { diff --git a/t/t5554-null-fetch-negotiator.sh b/t/t5554-null-fetch-negotiator.sh new file mode 100755 index 0000000000..09a8f0d608 --- /dev/null +++ b/t/t5554-null-fetch-negotiator.sh @@ -0,0 +1,22 @@ +#!/bin/sh + +test_description='test null fetch negotiator' +. ./test-lib.sh + +test_expect_success 'null negotiator does not emit any "have"' ' + rm -f trace && + + test_create_repo server && + test_commit -C server to_fetch && + + test_create_repo client && + test_commit -C client we_have && + + test_config -C client fetch.negotiationalgorithm null && + GIT_TRACE_PACKET="$(pwd)/trace" git -C client fetch "$(pwd)/server" && + + ! grep "fetch> have" trace && + grep "fetch> done" trace +' + +test_done From patchwork Wed Aug 5 01:20:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Tan X-Patchwork-Id: 11701137 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CBEC66C1 for ; Wed, 5 Aug 2020 01:21:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C34AE208C7 for ; Wed, 5 Aug 2020 01:21:16 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="IQG4ZqR9" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727823AbgHEBVP (ORCPT ); Tue, 4 Aug 2020 21:21:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44538 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727035AbgHEBVD (ORCPT ); Tue, 4 Aug 2020 21:21:03 -0400 Received: from mail-pg1-x549.google.com (mail-pg1-x549.google.com [IPv6:2607:f8b0:4864:20::549]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F3D61C06174A for ; Tue, 4 Aug 2020 18:21:02 -0700 (PDT) Received: by mail-pg1-x549.google.com with SMTP id h5so30023033pgq.4 for ; Tue, 04 Aug 2020 18:21:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=V6HJ8a80+gSNHl9U79xv02mGKNkVnr97q8RY6TaYSrA=; b=IQG4ZqR9Rt8tgf4UqK/YRDzMlbyzQEqGGsfy1KyZcU6lbusShRFy4GTQEWbAn8kWZA e6d6HkY2Jhyq7rtb6jb6UHUK6wdDap4TFKIBqU4sTHiuWvf81nf5vRSyG73ZgIqaVQtE MZ75x+AJFj1WsfeWIp+P1M5y+UbZnkfgx33XXTlo+3vKyktwbECL4aoLQ5UejOunb9LH a7Dr4wd3/aqZCtxj/C5THcu7Hir1i6Mm7GZagTcw1zGZLRsoSead2l4FV31IfhneHYbR PNgD+9s5/Mix75uFc+DtxSVgzQhECxojHHZLlf6UvVK6SlWyLYZ1rKrAyf5Cr1fTJvEu +5dg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=V6HJ8a80+gSNHl9U79xv02mGKNkVnr97q8RY6TaYSrA=; b=bGzqu8VtRvSE7B4xyh7tt3e8LhTZ4CRRMtN361gWuD4AJE3iEEPWMZupSTwtPpJIUl G63SdXEDbEUSykRcgADyVMwjrWkLOMi7nQJwerNYemF53uaohH3QwPvuFfIbdaoevPgS ueJkQDPxN6EnqKL/+qgEfNlYdNEBoEYNt57I21thydnm7aROHeQsPQjUEO9/pAVCK5f3 9min0RIwQa5xdtMXYKccwzunQGBBr2W2kJa906oP15QCl+akk+Jx6Afg1SxzW3LTcFNY ysupSDHW6Pe7IY0u0xOSg7/kWYJ3/bJVlWDLrf1RLYzhbAunzkNILCLodV1NE+iOCHB3 1cTw== X-Gm-Message-State: AOAM532/pIzfYCNA/wjFuzMiCfYBFKdkBxlbVA/yotjCUHmmbC4H8i6K elNvJYQOH4qhfS3637EMJLK/LiGiStb5sL9SZOCxY2yuITVpv5I0hyGmwsehpYPwa+bjZkmwbJT q4/5I1rq/1myTVYnzsGdQqbfMyUV9jsV1O/xheMYzGi8FYrsX8PVWifzkU8D4lq6E9/x5dIv8RX ji X-Google-Smtp-Source: ABdhPJz5OC321/K8bBFtFeWKEK/gPOE50jNkbuPi0PHbQoVt/RoQwx9kyvPZXSBJBJCxNHrp6v6U0NmUeAQ8qePHEvb+ X-Received: by 2002:a17:902:7fcb:: with SMTP id t11mr908084plb.266.1596590462285; Tue, 04 Aug 2020 18:21:02 -0700 (PDT) Date: Tue, 4 Aug 2020 18:20:47 -0700 In-Reply-To: Message-Id: <9ebcc5808a21e1f87370b6c18ea28168e63d4e3a.1596590295.git.jonathantanmy@google.com> Mime-Version: 1.0 References: <20200724223844.2723397-1-jonathantanmy@google.com> X-Mailer: git-send-email 2.28.0.163.g6104cc2f0b6-goog Subject: [RFC PATCH 4/7] fetch: --stdin From: Jonathan Tan To: git@vger.kernel.org Cc: Jonathan Tan , peff@peff.net, gitster@pobox.com Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Signed-off-by: Jonathan Tan --- builtin/fetch.c | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) diff --git a/builtin/fetch.c b/builtin/fetch.c index 3ccf69753f..a5498646bf 100644 --- a/builtin/fetch.c +++ b/builtin/fetch.c @@ -80,6 +80,7 @@ static struct list_objects_filter_options filter_options; static struct string_list server_options = STRING_LIST_INIT_DUP; static struct string_list negotiation_tip = STRING_LIST_INIT_NODUP; static int fetch_write_commit_graph = -1; +static int stdin_refspecs = 0; static int git_fetch_config(const char *k, const char *v, void *cb) { @@ -209,6 +210,8 @@ static struct option builtin_fetch_options[] = { N_("check for forced-updates on all updated branches")), OPT_BOOL(0, "write-commit-graph", &fetch_write_commit_graph, N_("write the commit-graph after fetching")), + OPT_BOOL(0, "stdin", &stdin_refspecs, + N_("accept refspecs from stdin")), OPT_END() }; @@ -1684,7 +1687,8 @@ static inline void fetch_one_setup_partial(struct remote *remote) return; } -static int fetch_one(struct remote *remote, int argc, const char **argv, int prune_tags_ok) +static int fetch_one(struct remote *remote, int argc, const char **argv, + int prune_tags_ok, int use_stdin_refspecs) { struct refspec rs = REFSPEC_INIT_FETCH; int i; @@ -1741,6 +1745,13 @@ static int fetch_one(struct remote *remote, int argc, const char **argv, int pru } } + if (use_stdin_refspecs) { + struct strbuf line = STRBUF_INIT; + while (strbuf_getline_lf(&line, stdin) != EOF) + refspec_append(&rs, line.buf); + strbuf_release(&line); + } + if (server_options.nr) gtransport->server_options = &server_options; @@ -1841,7 +1852,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix) if (remote) { if (filter_options.choice || has_promisor_remote()) fetch_one_setup_partial(remote); - result = fetch_one(remote, argc, argv, prune_tags_ok); + result = fetch_one(remote, argc, argv, prune_tags_ok, stdin_refspecs); } else { int max_children = max_jobs; @@ -1849,6 +1860,10 @@ int cmd_fetch(int argc, const char **argv, const char *prefix) die(_("--filter can only be used with the remote " "configured in extensions.partialclone")); + if (stdin_refspecs) + die(_("--stdin can only be used when fetching " + "from one remote")); + if (max_children < 0) max_children = fetch_parallel_config; From patchwork Wed Aug 5 01:20:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Tan X-Patchwork-Id: 11701139 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 81C4113B1 for ; Wed, 5 Aug 2020 01:21:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 790812086A for ; Wed, 5 Aug 2020 01:21:24 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="p3IMRZPk" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727824AbgHEBVR (ORCPT ); Tue, 4 Aug 2020 21:21:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44544 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727779AbgHEBVE (ORCPT ); Tue, 4 Aug 2020 21:21:04 -0400 Received: from mail-pg1-x549.google.com (mail-pg1-x549.google.com [IPv6:2607:f8b0:4864:20::549]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DCD03C061756 for ; Tue, 4 Aug 2020 18:21:04 -0700 (PDT) Received: by mail-pg1-x549.google.com with SMTP id j37so6469217pgi.16 for ; Tue, 04 Aug 2020 18:21:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=n4nIRHuMkjqaAXrUTuDya0OQZtOswcfUy1h/GasPJvE=; b=p3IMRZPkyTkwmQ9TrVx+rG5k3QiEpbg78b1d7diKVJqr5WWwd1M3WKNFhve5w7U8MN BO5se129/0oiyirziJWoks0MkZAKRO6qmVLdIc6WljCrZZOND9jLprfvmizqDxtcnazu 8Z7AiqhdhC41AiXZ9H1af4utX807SYS7+PBI4XVjkGOaEXk8rlTNW9w0n5T7qkBeCkfd I3m3Z18J23oLlWf9cCjkv8LfHj0yZpGMj+YS17Cc3RVpZVQoyYwQF1QaAF0lDKPGeoTM ECPJgmpC4kT99tEZlCm9yct6P1DpQmZuPGwQu991HhzvZqaj8/0rMQVZg/hdtZ8KLtH0 ajhw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=n4nIRHuMkjqaAXrUTuDya0OQZtOswcfUy1h/GasPJvE=; b=GHXzPsE4/wPe4eLXeC+zc0lYRWuq4HwkaOa/uQVzryuzI1qkQvwDHnTHIDxz++8DH1 sif128jpkayb6MwSSB/+uY/hPJy0CCWvTA84kMTb5tjPRKkLjLPcDeLVWPGkXM+7P2+/ /oHP3zWCmIgpGIZwlCzYWkt/NpXSsiZYNLuGWYCnoPS1pfqh00ZWMmA3DVND3KBQORPK wxTOd5M1/ljTXpKBroBwEdZJXz+7mQmDOhj9dyewq0FTy1BCYR3ce6lUQg+xtB3/T+me /aoSN7oU4D2adTRqccF+77XEpvt+Gg1s1oCVWn9eNfDOVdm/6rbY18TFcDr6r0iX/nrb 1IIQ== X-Gm-Message-State: AOAM533gyVc31Hxy5Be3GP28evDXkv4Qc/laT90watYKX/blkRjpEkQ3 DhVxCa4WDCRk2MWSVqrRCVuypVZup5KLTnIGML0RSRZ/ymkwkliftNVeLqmQkKKEdVBGH36zuBO YH9/Goh6W6cziXac9c8onk/I3eln5aE+eCrwjT3GtuRCUda7d2AByIuQDl/W5ybGrku49ysLdUz gq X-Google-Smtp-Source: ABdhPJxsXQS12egxDRbTlMzKeFC76yl3SxHxA4kOxjqMqGeSiZTINEvM3E/CPq05qu9/MSyuPd90ceZOYltQFEAt+UlA X-Received: by 2002:a63:b18:: with SMTP id 24mr996160pgl.406.1596590464200; Tue, 04 Aug 2020 18:21:04 -0700 (PDT) Date: Tue, 4 Aug 2020 18:20:48 -0700 In-Reply-To: Message-Id: Mime-Version: 1.0 References: <20200724223844.2723397-1-jonathantanmy@google.com> X-Mailer: git-send-email 2.28.0.163.g6104cc2f0b6-goog Subject: [RFC PATCH 5/7] fetch: submodule config From: Jonathan Tan To: git@vger.kernel.org Cc: Jonathan Tan , peff@peff.net, gitster@pobox.com Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Signed-off-by: Jonathan Tan --- builtin/fetch.c | 10 ++++++++-- submodule-config.c | 5 +++-- 2 files changed, 11 insertions(+), 4 deletions(-) diff --git a/builtin/fetch.c b/builtin/fetch.c index a5498646bf..29db219c68 100644 --- a/builtin/fetch.c +++ b/builtin/fetch.c @@ -1786,12 +1786,18 @@ int cmd_fetch(int argc, const char **argv, const char *prefix) free(anon); } - fetch_config_from_gitmodules(&submodule_fetch_jobs_config, - &recurse_submodules); git_config(git_fetch_config, NULL); argc = parse_options(argc, argv, prefix, builtin_fetch_options, builtin_fetch_usage, 0); + if (recurse_submodules != RECURSE_SUBMODULES_OFF) { + int *sfjc = submodule_fetch_jobs_config == -1 + ? &submodule_fetch_jobs_config : NULL; + int *rs = recurse_submodules == RECURSE_SUBMODULES_DEFAULT + ? &recurse_submodules : NULL; + + fetch_config_from_gitmodules(sfjc, rs); + } if (deepen_relative) { if (deepen_relative < 0) diff --git a/submodule-config.c b/submodule-config.c index e175dfbc38..8d65273ed2 100644 --- a/submodule-config.c +++ b/submodule-config.c @@ -776,10 +776,11 @@ struct fetch_config { static int gitmodules_fetch_config(const char *var, const char *value, void *cb) { struct fetch_config *config = cb; - if (!strcmp(var, "submodule.fetchjobs")) { + if (!strcmp(var, "submodule.fetchjobs") && config->max_children) { *(config->max_children) = parse_submodule_fetchjobs(var, value); return 0; - } else if (!strcmp(var, "fetch.recursesubmodules")) { + } else if (!strcmp(var, "fetch.recursesubmodules") && + config->recurse_submodules) { *(config->recurse_submodules) = parse_fetch_recurse_submodules_arg(var, value); return 0; } From patchwork Wed Aug 5 01:20:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Tan X-Patchwork-Id: 11701143 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4C26B6C1 for ; Wed, 5 Aug 2020 01:21:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 41E4C208C7 for ; Wed, 5 Aug 2020 01:21:33 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Y16DgxH6" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727826AbgHEBV2 (ORCPT ); Tue, 4 Aug 2020 21:21:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44552 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727782AbgHEBVG (ORCPT ); Tue, 4 Aug 2020 21:21:06 -0400 Received: from mail-pf1-x449.google.com (mail-pf1-x449.google.com [IPv6:2607:f8b0:4864:20::449]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5FF95C06174A for ; Tue, 4 Aug 2020 18:21:06 -0700 (PDT) Received: by mail-pf1-x449.google.com with SMTP id 19so12342119pfu.20 for ; Tue, 04 Aug 2020 18:21:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=rBtjqe68U0wwmHsT0FRWRNllwmP9aO0zsilqUwBM7xU=; b=Y16DgxH6Ag2b+5Ia811Dwi2sPerAD2a5K3ZyJ6bczvfzouCtYTk9AVmGfm5pcs4PXA +gV6x+8qYPqzTBrsccxsfNw6KximAPPaTDu7hU9BQ/kELYFb3b364j+H+a8XP2j2XVAE wnSTyM37rQQlm8FXHYn5GwAmZQ69/e0fzDNPivT3LCP8gpOQ0UfLWQVzyy+jbNaaCIea /MrSSnEiXS02oAN3/irzONFTMUDGmqEK/FSbRQlIeV/lNXXfsJf5PcW6uJqTyiU4dm0c 7+NkqWugv+0lCwVMbqY6hS+uHeQ+vuJGkT+fy/Wq0//fYAJUhp0XhU0gc5T3xDh0EpzV ev7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=rBtjqe68U0wwmHsT0FRWRNllwmP9aO0zsilqUwBM7xU=; b=QR3O69FDCR+eJ+3uUFIzvGFNsni3eD0R2m3NrwX6uxXV7Le7kAYM72lZvKBkS4q5ZP VssAL7UBmq8FnTpHooUhthKlUNBja620ZPCQMmXIPq3IRYjs5HtHOvXtQtfaflUSzUqO JzdN4QsQSNJMaDn8D8IRlIGESKu43uRy33P6CmwhPRlI7qv5FPt4UMqnpbNyl6BE0Pl/ jbPXEDqPeeQZxSXnkEkPKkr09RYRjrXH3G42/BiQtpDQlp2DQJZc5QByt5KbtX47AS3B 4Thjx4ZA66QyXFVEFYP4B88W9xlW7QLgoi57JHD/Lc5PR16zTftJKKMFvVACrTHOQbWc IBHA== X-Gm-Message-State: AOAM532uYKhk7WShbF49lHvue9w1ezJrqaGQXEQO2H8opAvhBW3ziKn6 xxQSE69ohjaNVF2U0VRiKB3f4I9+cRQVDZRbBe9e7+tzyeTs6C52lfsitcz5Y95Qzq8Ndg50OA9 AWUiBlCnIZfkixAlGiTiFrxAfhht2oeBoKibRiE7ANMYimgHrbDfQLjMy68Bi0PuoppwnfYy+OM 3k X-Google-Smtp-Source: ABdhPJycVXYIgstHXjnisiakA/saTQ0ifZyL/2pmX6AqXQ7DInZXjx0U8SQv+iMMO4y/IAgBLuBpmAJb6/PChhMhHHLg X-Received: by 2002:a17:902:a981:: with SMTP id bh1mr892658plb.157.1596590465776; Tue, 04 Aug 2020 18:21:05 -0700 (PDT) Date: Tue, 4 Aug 2020 18:20:49 -0700 In-Reply-To: Message-Id: Mime-Version: 1.0 References: <20200724223844.2723397-1-jonathantanmy@google.com> X-Mailer: git-send-email 2.28.0.163.g6104cc2f0b6-goog Subject: [RFC PATCH 6/7] fetch: only populate existing_refs if needed From: Jonathan Tan To: git@vger.kernel.org Cc: Jonathan Tan , peff@peff.net, gitster@pobox.com Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Signed-off-by: Jonathan Tan --- builtin/fetch.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/builtin/fetch.c b/builtin/fetch.c index 29db219c68..6460ce3f4e 100644 --- a/builtin/fetch.c +++ b/builtin/fetch.c @@ -449,6 +449,7 @@ static struct ref *get_ref_map(struct remote *remote, struct ref *orefs = NULL, **oref_tail = &orefs; struct hashmap existing_refs; + int existing_refs_populated = 0; if (rs->nr) { struct refspec *fetch_refspec; @@ -542,15 +543,18 @@ static struct ref *get_ref_map(struct remote *remote, ref_map = ref_remove_duplicates(ref_map); - refname_hash_init(&existing_refs); - for_each_ref(add_one_refname, &existing_refs); - for (rm = ref_map; rm; rm = rm->next) { if (rm->peer_ref) { const char *refname = rm->peer_ref->name; struct refname_hash_entry *peer_item; unsigned int hash = strhash(refname); + if (!existing_refs_populated) { + refname_hash_init(&existing_refs); + for_each_ref(add_one_refname, &existing_refs); + existing_refs_populated = 1; + } + peer_item = hashmap_get_entry_from_hash(&existing_refs, hash, refname, struct refname_hash_entry, ent); @@ -560,7 +564,8 @@ static struct ref *get_ref_map(struct remote *remote, } } } - hashmap_free_entries(&existing_refs, struct refname_hash_entry, ent); + if (existing_refs_populated) + hashmap_free_entries(&existing_refs, struct refname_hash_entry, ent); return ref_map; } From patchwork Wed Aug 5 01:20:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Tan X-Patchwork-Id: 11701141 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E36D96C1 for ; Wed, 5 Aug 2020 01:21:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D24032086A for ; Wed, 5 Aug 2020 01:21:31 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="rVVVbMgT" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727829AbgHEBV2 (ORCPT ); Tue, 4 Aug 2020 21:21:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44558 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727808AbgHEBVI (ORCPT ); Tue, 4 Aug 2020 21:21:08 -0400 Received: from mail-qt1-x84a.google.com (mail-qt1-x84a.google.com [IPv6:2607:f8b0:4864:20::84a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 150D9C061757 for ; Tue, 4 Aug 2020 18:21:08 -0700 (PDT) Received: by mail-qt1-x84a.google.com with SMTP id s29so10813988qtc.12 for ; Tue, 04 Aug 2020 18:21:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=V/JMEPevwFwEsza3SZ1pzsNtyh/xUJw94pjt6KT4hlE=; b=rVVVbMgTFMSwzP5JQ+nbIYvkScgnZE+GVyoK3cMAaBVRcOk28YuD8ds0Kw/W31Jfmu xLxIb5GHtELPTwD1PfDtO23Ueui2a8ZRxAUOrtkfImuFr6Kge8GyzRkKXT2/qmjfU/qn m3j5gGMSapG6W8pXfhfZ0JNmSS/lR4XzGU16GyYjAHYznToY6lB1tg7aGB8AKRbiroGH bQe7ynRtG/sZjqJdVx8+HB2QRaGDOUAERfRG1YKUTwJpOC9xbafsCWhLeOMcg4iTc/n7 1/tfZ36pO9Pc2cK8NdoYlgdU7iRQL8P66cZjtnr6XkIxMvTRbSMUumj3fiIMw1PaJwGY HAwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=V/JMEPevwFwEsza3SZ1pzsNtyh/xUJw94pjt6KT4hlE=; b=V0yLC+TkFCxiV6E9tZQXOx8idjY2qWND8l42iYEb4kKxIXW/8UBzDWGMRFWUkDvF+i 0x8s6/SVA6H2UHH3yNvRu6JyAsIULPaTJM3n7tNT+lBTQ6McNUHnuR305A3CpBEi8v5y ZgqL6/NyVKlLbVw3ax0bpKde7Kn+SzqskbF+Ccu0dnz2BBR4yy8Y5XhDID5T91l5W3FU SfOg7266WmnCcGcTqmteGZfz5+kvTGcIg6Yrt2nVFBIOc16qkyj97R2PkZssYB7pfcvO gBQfg5s0AEBRP1OIMF1fyy+bnWG9vcCC95qtA/T1SWoEaILi5iMORAOitukhFtqtdicE HajQ== X-Gm-Message-State: AOAM530fT8uM6JGH1B2gklWnOojddSkgJe43kQ3c8sHliPYYujSZ+K8P OPaIY6uddVgTuC7HVZXEaBO6cpmo2RxRrF7gRHTrUPCjX4BSznwJ7nNpIxFS35GvRvpD7ymwGF4 PxR2yr1TQJRmurngM0BkRiuQ5KQ+nTrvxs+/LY1YTm/5bNun+sSEhqo5olrkIh2gAJBwRfi+w3g Rq X-Google-Smtp-Source: ABdhPJwEzOb0bdha0mTmtB0UGO4Fa3uF/Ft3qRwJXv4iN1x9U1ragBzTOwgth0mpm1zfrr/2Rv8FPjlBFLL04Vuj0ETq X-Received: by 2002:a05:6214:140d:: with SMTP id n13mr1224699qvx.69.1596590467219; Tue, 04 Aug 2020 18:21:07 -0700 (PDT) Date: Tue, 4 Aug 2020 18:20:50 -0700 In-Reply-To: Message-Id: <38bbad333e3eda1322943e29efabad9d9c6a996a.1596590295.git.jonathantanmy@google.com> Mime-Version: 1.0 References: <20200724223844.2723397-1-jonathantanmy@google.com> X-Mailer: git-send-email 2.28.0.163.g6104cc2f0b6-goog Subject: [RFC PATCH 7/7] promisor-remote: use subprocess to fetch From: Jonathan Tan To: git@vger.kernel.org Cc: Jonathan Tan , peff@peff.net, gitster@pobox.com Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org --- promisor-remote.c | 46 ++++++++++++++++------------------- t/t0410-partial-clone.sh | 2 +- t/t4067-diff-partial-clone.sh | 8 +++--- t/t5601-clone.sh | 2 +- 4 files changed, 27 insertions(+), 31 deletions(-) diff --git a/promisor-remote.c b/promisor-remote.c index baaea12fd6..6e647610e9 100644 --- a/promisor-remote.c +++ b/promisor-remote.c @@ -3,6 +3,7 @@ #include "promisor-remote.h" #include "config.h" #include "transport.h" +#include "argv-array.h" static char *repository_format_partial_clone; static const char *core_partial_clone_filter_default; @@ -12,39 +13,34 @@ void set_repository_format_partial_clone(char *partial_clone) repository_format_partial_clone = xstrdup_or_null(partial_clone); } -static int fetch_refs(const char *remote_name, struct ref *ref) -{ - struct remote *remote; - struct transport *transport; - int res; - - remote = remote_get(remote_name); - if (!remote->url[0]) - die(_("Remote with no URL")); - transport = transport_get(remote, remote->url[0]); - - transport_set_option(transport, TRANS_OPT_FROM_PROMISOR, "1"); - transport_set_option(transport, TRANS_OPT_NO_DEPENDENTS, "1"); - res = transport_fetch_refs(transport, ref); - - return res; -} - static int fetch_objects(const char *remote_name, const struct object_id *oids, int oid_nr) { - struct ref *ref = NULL; + struct child_process child = CHILD_PROCESS_INIT; int i; + FILE *child_in; + + child.git_cmd = 1; + child.in = -1; + argv_array_pushl(&child.args, "-c", "fetch.negotiationAlgorithm=null", + "fetch", remote_name, "--no-tags", + "--no-write-fetch-head", "--recurse-submodules=no", + "--filter=blob:none", "--stdin", NULL); + if (start_command(&child)) + die(_("promisor-remote: unable to fork off fetch subprocess")); + child_in = xfdopen(child.in, "w"); for (i = 0; i < oid_nr; i++) { - struct ref *new_ref = alloc_ref(oid_to_hex(&oids[i])); - oidcpy(&new_ref->old_oid, &oids[i]); - new_ref->exact_oid = 1; - new_ref->next = ref; - ref = new_ref; + if (fputs(oid_to_hex(&oids[i]), child_in) < 0) + die_errno(_("promisor-remote: could not write to fetch subprocess")); + if (fputc('\n', child_in) < 0) + die_errno(_("promisor-remote: could not write to fetch subprocess")); } - return fetch_refs(remote_name, ref); + + if (fclose(child_in) < 0) + die_errno(_("promisor-remote: could not close stdin to fetch subprocess")); + return finish_command(&child) ? -1 : 0; } static struct promisor_remote *promisors; diff --git a/t/t0410-partial-clone.sh b/t/t0410-partial-clone.sh index 463dc3a8be..3e454f934e 100755 --- a/t/t0410-partial-clone.sh +++ b/t/t0410-partial-clone.sh @@ -203,7 +203,7 @@ test_expect_success 'fetching of missing objects works with ref-in-want enabled' rm -rf repo/.git/objects/* && rm -f trace && GIT_TRACE_PACKET="$(pwd)/trace" git -C repo cat-file -p "$HASH" && - grep "git< fetch=.*ref-in-want" trace + grep "fetch< fetch=.*ref-in-want" trace ' test_expect_success 'fetching of missing objects from another promisor remote' ' diff --git a/t/t4067-diff-partial-clone.sh b/t/t4067-diff-partial-clone.sh index ef8e0e9cb0..804f2a82e8 100755 --- a/t/t4067-diff-partial-clone.sh +++ b/t/t4067-diff-partial-clone.sh @@ -20,7 +20,7 @@ test_expect_success 'git show batches blobs' ' # Ensure that there is exactly 1 negotiation by checking that there is # only 1 "done" line sent. ("done" marks the end of negotiation.) GIT_TRACE_PACKET="$(pwd)/trace" git -C client show HEAD && - grep "git> done" trace >done_lines && + grep "fetch> done" trace >done_lines && test_line_count = 1 done_lines ' @@ -44,7 +44,7 @@ test_expect_success 'diff batches blobs' ' # Ensure that there is exactly 1 negotiation by checking that there is # only 1 "done" line sent. ("done" marks the end of negotiation.) GIT_TRACE_PACKET="$(pwd)/trace" git -C client diff HEAD^ HEAD && - grep "git> done" trace >done_lines && + grep "fetch> done" trace >done_lines && test_line_count = 1 done_lines ' @@ -127,7 +127,7 @@ test_expect_success 'diff with rename detection batches blobs' ' # only 1 "done" line sent. ("done" marks the end of negotiation.) GIT_TRACE_PACKET="$(pwd)/trace" git -C client diff --raw -M HEAD^ HEAD >out && grep ":100644 100644.*R[0-9][0-9][0-9].*b.*c" out && - grep "git> done" trace >done_lines && + grep "fetch> done" trace >done_lines && test_line_count = 1 done_lines ' @@ -175,7 +175,7 @@ test_expect_success 'diff --break-rewrites fetches only if necessary, and batche # by checking that there is only 1 "done" line sent. ("done" marks the # end of negotiation.) GIT_TRACE_PACKET="$(pwd)/trace" git -C client diff --break-rewrites --raw -M HEAD^ HEAD && - grep "git> done" trace >done_lines && + grep "fetch> done" trace >done_lines && test_line_count = 1 done_lines ' diff --git a/t/t5601-clone.sh b/t/t5601-clone.sh index 84ea2a3eb7..f82b0dbb5a 100755 --- a/t/t5601-clone.sh +++ b/t/t5601-clone.sh @@ -702,7 +702,7 @@ test_expect_success 'batch missing blob request during checkout' ' # Ensure that there is only one negotiation by checking that there is # only "done" line sent. ("done" marks the end of negotiation.) GIT_TRACE_PACKET="$(pwd)/trace" git -C client checkout HEAD^ && - grep "git> done" trace >done_lines && + grep "fetch> done" trace >done_lines && test_line_count = 1 done_lines '