From patchwork Thu Oct 25 23:32:22 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Beller X-Patchwork-Id: 10656791 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BDE5A13A4 for ; Thu, 25 Oct 2018 23:32:46 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B262F2C6EC for ; Thu, 25 Oct 2018 23:32:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A61882C6F8; Thu, 25 Oct 2018 23:32:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 559F72C6EC for ; Thu, 25 Oct 2018 23:32:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727609AbeJZIHR (ORCPT ); Fri, 26 Oct 2018 04:07:17 -0400 Received: from mail-io1-f73.google.com ([209.85.166.73]:56279 "EHLO mail-io1-f73.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727487AbeJZIHR (ORCPT ); Fri, 26 Oct 2018 04:07:17 -0400 Received: by mail-io1-f73.google.com with SMTP id o7-v6so8377463ioh.22 for ; Thu, 25 Oct 2018 16:32:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=+rOFzW5bJHQCIBM022kwARqDlluGnq5Nr5y183LRgpg=; b=JW+2/Rik53yBG5VNMaxrUZtm5saq3w0k5IaGdMDXHaEAj+i6ywFU487GiJbUDbqr7I bQMfvioS2LK569ivVDWcmNvGlm3yWksmMoLaP1KU98V1ibOu+MakPAPnDpMdA31QgAAs V5XJVQXAzEz/n/ANNopRRl8H1KuNZ3iKlw7KSLc5WsrqGV+OmMWaWHe6wQpyxkG2RyGJ yK8G8O3DGM2pcbR1HJv7GAQHWEQFTooutW3vQNin9MvI0t7CtsAU8KPgfN0bPMaIHHZH E2A3NzvotJ2d5ZtWK4wqDKzsQrUYCPoicBO4pjlmv5O3LSGL/yfow6tk9p4+soetO/RI felQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=+rOFzW5bJHQCIBM022kwARqDlluGnq5Nr5y183LRgpg=; b=Jev2fmSunK/T5d5VPamfxh5r0aZPFxcUkUsvWJAcnQTaFNYx+w3r8XfPyBYs2M5uSQ gB3a2lC/wrlVuoygERD8N3mWuXcaEyaegbawh2jC1ANgD253PkvrevwPAzC6xyfHv3bT ObAOSdBpSdT5kxbXF72WhlxWEqrFG5LMpEwgcAW4jnFLQtBcUmyZRtbQdPphqLbv/WdY ZvKMIpbETIUcmry0vzjNwIOi0qlRXrYNPxcLeMzOzHQKSRJz34VQnontCPisHZ+CzN6I 1yxKEUegjYNhycOpzN7MSVKoBgjOxwZfj5zXUW7Jvz4h/KWy28AaZSEO71zAX6bB0PsZ A+Dw== X-Gm-Message-State: AGRZ1gJodU18B8yJdO8Yo9S4Toj3Gpi/EsVdOZGYbxMzKjrmTDTKVOA5 1iYM5uOmUJdx/YqV0rovWMsWAySmx3jJ X-Google-Smtp-Source: AJdET5cLlnTjR/FDW2TnMfbvgaLyl9cB7buaZbOoDQrcC0NDKLpetQTUaLS8MFHHvo2gc6tMgZZ79/ePcgqq X-Received: by 2002:a24:6955:: with SMTP id e82-v6mr1016165itc.23.1540510358934; Thu, 25 Oct 2018 16:32:38 -0700 (PDT) Date: Thu, 25 Oct 2018 16:32:22 -0700 In-Reply-To: <20181025233231.102245-1-sbeller@google.com> Message-Id: <20181025233231.102245-2-sbeller@google.com> Mime-Version: 1.0 References: <20181025233231.102245-1-sbeller@google.com> X-Mailer: git-send-email 2.19.0 Subject: [PATCH 01/10] sha1-array: provide oid_array_filter From: Stefan Beller To: jonathantanmy@google.com Cc: git@vger.kernel.org, Stefan Beller , Junio C Hamano Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Helped-by: Junio C Hamano Signed-off-by: Stefan Beller Signed-off-by: Junio C Hamano --- Documentation/technical/api-oid-array.txt | 5 +++++ sha1-array.c | 17 +++++++++++++++++ sha1-array.h | 3 +++ 3 files changed, 25 insertions(+) diff --git a/Documentation/technical/api-oid-array.txt b/Documentation/technical/api-oid-array.txt index 9febfb1d52..c97428c2c3 100644 --- a/Documentation/technical/api-oid-array.txt +++ b/Documentation/technical/api-oid-array.txt @@ -48,6 +48,11 @@ Functions is not sorted, this function has the side effect of sorting it. +`oid_array_filter`:: + Apply the callback function `want` to each entry in the array, + retaining only the entries for which the function returns true. + Preserve the order of the entries that are retained. + Examples -------- diff --git a/sha1-array.c b/sha1-array.c index 265941fbf4..d505a004bb 100644 --- a/sha1-array.c +++ b/sha1-array.c @@ -77,3 +77,20 @@ int oid_array_for_each_unique(struct oid_array *array, } return 0; } + +void oid_array_filter(struct oid_array *array, + for_each_oid_fn want, + void *cb_data) +{ + unsigned nr = array->nr, src, dst; + struct object_id *oids = array->oid; + + for (src = dst = 0; src < nr; src++) { + if (want(&oids[src], cb_data)) { + if (src != dst) + oidcpy(&oids[dst], &oids[src]); + dst++; + } + } + array->nr = dst; +} diff --git a/sha1-array.h b/sha1-array.h index 232bf95017..55d016c4bf 100644 --- a/sha1-array.h +++ b/sha1-array.h @@ -22,5 +22,8 @@ int oid_array_for_each(struct oid_array *array, int oid_array_for_each_unique(struct oid_array *array, for_each_oid_fn fn, void *data); +void oid_array_filter(struct oid_array *array, + for_each_oid_fn want, + void *cbdata); #endif /* SHA1_ARRAY_H */ From patchwork Thu Oct 25 23:32:23 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Beller X-Patchwork-Id: 10656793 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0358F15A7 for ; Thu, 25 Oct 2018 23:32:47 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EEE5A2C6EC for ; Thu, 25 Oct 2018 23:32:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E37902C6F1; Thu, 25 Oct 2018 23:32:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 75ECE2C6F0 for ; Thu, 25 Oct 2018 23:32:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727654AbeJZIHV (ORCPT ); Fri, 26 Oct 2018 04:07:21 -0400 Received: from mail-vk1-f201.google.com ([209.85.221.201]:45303 "EHLO mail-vk1-f201.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727650AbeJZIHU (ORCPT ); Fri, 26 Oct 2018 04:07:20 -0400 Received: by mail-vk1-f201.google.com with SMTP id 80so3011324vki.12 for ; Thu, 25 Oct 2018 16:32:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=B00n941xueuBWwdXzRKPIBl5kroqyU94J3aafKflOrg=; b=qnv2Glrt03GXRTysa4i17kviPQk/kpbpbWu5RrcXr3d22nX/YeqOMaFMD0dcJDHjwM bsRPAZStNOapG35JZpyiH75ZLyFyiXEM2dTYnALNVyqE651pDBOd+JVDib3XrWbCJlhj jCvGP6jlq9UVRHmaTwBydf/Rnv5dlQIQWcEYevB2f7A0Yti87z0WHt/jmtQO4B4nNfk4 34+AgO1FkYkP8LciRMqK7hpXBJkm7ZzCeyW+p8xCcIvwzc+l0NOakzhW8FjDc6qcbFgN YfURKJnTcZYTbYuO+Lnb1WivqfkCoDsRpn2a1RNhdyEDjyEeI8uL4ukF+oM3Y9okXkzc PnRQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=B00n941xueuBWwdXzRKPIBl5kroqyU94J3aafKflOrg=; b=mEMEzsALh+MPVFGC/i02VtpvY2dTTiAavGaT68pStgPVTGQRvA3RvA/uuxqHe+zn5z rYNk3pgnnFMrl1SwaWBYbnKAwxu4jj96i2MqP/91yRY5Cex6Bbs2v9/oSaPh6fhUNBgy vxiFdyOdTO/Q0hfGeJMzfPfF/94zdpzuChthXOFggr1tF+GTviU7O/Wwppxa60eIQR1Q KnH9tKb3TheQuyqjzxOiLA2o4BtQQj9qghqUAYDHuNgMvp5kHtgpv7RIgOHxXXJVbPVe wlgqPQE8JokFipd1V3jgBAYskfFDrcGMZnV/GfnwShsAH2SZNvD4oRYD1tux4C3rqZvI iwUg== X-Gm-Message-State: AGRZ1gLUwLBYNcksKnMwSoFmo42ZKzOiu2N3jKP/HLM8v+6Gjgf9kyEo 4oG3z/Jr6YlPIVWU5RJo3m19DLo4/yzJ X-Google-Smtp-Source: AJdET5cL5ML8L6UpZ5lQary25WOQf/f3qdVf4OEPCTDSDulouNb3fncR3vTWdzIKnMk8koo5DpB7xwdrBA+p X-Received: by 2002:ab0:185f:: with SMTP id j31mr1269480uag.9.1540510361600; Thu, 25 Oct 2018 16:32:41 -0700 (PDT) Date: Thu, 25 Oct 2018 16:32:23 -0700 In-Reply-To: <20181025233231.102245-1-sbeller@google.com> Message-Id: <20181025233231.102245-3-sbeller@google.com> Mime-Version: 1.0 References: <20181025233231.102245-1-sbeller@google.com> X-Mailer: git-send-email 2.19.0 Subject: [PATCH 02/10] submodule.c: fix indentation From: Stefan Beller To: jonathantanmy@google.com Cc: git@vger.kernel.org, Stefan Beller , Junio C Hamano Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The submodule subsystem is really bad at staying within 80 characters. Fix it while we are here. Signed-off-by: Stefan Beller Signed-off-by: Junio C Hamano --- submodule.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/submodule.c b/submodule.c index 2b7082b2db..e145ebbb16 100644 --- a/submodule.c +++ b/submodule.c @@ -1258,7 +1258,8 @@ static int get_next_submodule(struct child_process *cp, if (!submodule) { const char *name = default_name_or_path(ce->name); if (name) { - default_submodule.path = default_submodule.name = name; + default_submodule.path = name; + default_submodule.name = name; submodule = &default_submodule; } } @@ -1268,8 +1269,10 @@ static int get_next_submodule(struct child_process *cp, default: case RECURSE_SUBMODULES_DEFAULT: case RECURSE_SUBMODULES_ON_DEMAND: - if (!submodule || !unsorted_string_list_lookup(&changed_submodule_names, - submodule->name)) + if (!submodule || + !unsorted_string_list_lookup( + &changed_submodule_names, + submodule->name)) continue; default_argv = "on-demand"; break; From patchwork Thu Oct 25 23:32:24 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Beller X-Patchwork-Id: 10656797 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B2D4513A4 for ; Thu, 25 Oct 2018 23:32:52 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A6A852C6E9 for ; Thu, 25 Oct 2018 23:32:52 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9861D2C6F8; Thu, 25 Oct 2018 23:32:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 457D32C6E9 for ; Thu, 25 Oct 2018 23:32:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727658AbeJZIHX (ORCPT ); Fri, 26 Oct 2018 04:07:23 -0400 Received: from mail-qt1-f202.google.com ([209.85.160.202]:38136 "EHLO mail-qt1-f202.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727650AbeJZIHW (ORCPT ); Fri, 26 Oct 2018 04:07:22 -0400 Received: by mail-qt1-f202.google.com with SMTP id b55-v6so11001910qtb.5 for ; Thu, 25 Oct 2018 16:32:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=dKkiRNzxRqU0laEc6xb/t6nsn4iaEPm7vic1aGSiMvA=; b=nApvsjj2CIcdu2pjJtV0C/oQod1qenJqzDfKuJWbDZc9+f8RY+47enLb2MzgsRNWno dgLzuun4I0MtV7NlzA/CCH414MmUkWjsk7VUtdoW8RPgmy4LYIOU64yqmVGzPw5dqb/w tvxZuAShqr7rfHRmImI4dhWC34/ecHHo9sAS4IBa0bK5yOgu3AzGkiaRiWzYZVCsUxOU C8QxmIOInKEgrSuJ9Wz0Y+ISewN/qp5+R0xYQslRFj8mBPKaL7aofUVzDAUDdnsFMQdU 2p9y+yMfTOYKdojnGCVky0A0ja9UXDJLIRzDKjxrZbTL5ADYCR4GD33oT96f7151aThI 6jMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=dKkiRNzxRqU0laEc6xb/t6nsn4iaEPm7vic1aGSiMvA=; b=uHxCVmhIzLAmNiJqfqjU1N89e8PkqLGA1YTAZdZuxGERbk8ApLTWjDI9P4qTede+v0 ZykzTOVZRl8OH8QgGQdXzX+NjV5+vhDWE8/FvyV4ROcOpIhIa1mVc2aZiyAD8tg4E2R/ Eq+1l8jkLPXrpmjbKrMnVpkwbFiNWCpj/IDHZjeAR55wTbTlOHrFqgNLC0jCMF2/zhKt 4u2NC1MgQGYUaozK1LzXDbrluAhqrXletghVVrMsbRoQtBwqi6MshcFnJxGR6oBU6oF6 hGlpdQcUEOI1mTHRkd0GxaHflf1fqW7x2SBTqnO++Za1CkA2DqygXatWqCl0OQnVl0vm IqBA== X-Gm-Message-State: AGRZ1gJr40sSbyfn6Fdxu5w5gtIwyv7i3Nc09i20KlRiEsH4pNDF+nWj lXzY92TjFp6oko0yIrbc5O9SgaaQFEZk X-Google-Smtp-Source: AJdET5d3o5Y5bgqxISd64wz8NcW+HeH1nIhFFx+NgdAlV0cHoL35gcJVOjSdVWQAnRZKTpUoc6DRFZJbcGYF X-Received: by 2002:a37:6082:: with SMTP id u124-v6mr1197501qkb.14.1540510364140; Thu, 25 Oct 2018 16:32:44 -0700 (PDT) Date: Thu, 25 Oct 2018 16:32:24 -0700 In-Reply-To: <20181025233231.102245-1-sbeller@google.com> Message-Id: <20181025233231.102245-4-sbeller@google.com> Mime-Version: 1.0 References: <20181025233231.102245-1-sbeller@google.com> X-Mailer: git-send-email 2.19.0 Subject: [PATCH 03/10] submodule.c: sort changed_submodule_names before searching it From: Stefan Beller To: jonathantanmy@google.com Cc: git@vger.kernel.org, Stefan Beller , Junio C Hamano Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We can string_list_insert() to maintain sorted-ness of the list as we find new items, or we can string_list_append() to build an unsorted list and sort it at the end just once. As we do not rely on the sortedness while building the list, we pick the "append and sort at the end" as it has better worst case execution times. Signed-off-by: Stefan Beller Signed-off-by: Junio C Hamano --- submodule.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/submodule.c b/submodule.c index e145ebbb16..9fbfcfcfe1 100644 --- a/submodule.c +++ b/submodule.c @@ -1270,7 +1270,7 @@ static int get_next_submodule(struct child_process *cp, case RECURSE_SUBMODULES_DEFAULT: case RECURSE_SUBMODULES_ON_DEMAND: if (!submodule || - !unsorted_string_list_lookup( + !string_list_lookup( &changed_submodule_names, submodule->name)) continue; @@ -1364,6 +1364,7 @@ int fetch_populated_submodules(struct repository *r, /* default value, "--submodule-prefix" and its value are added later */ calculate_changed_submodule_paths(); + string_list_sort(&changed_submodule_names); run_processes_parallel(max_parallel_jobs, get_next_submodule, fetch_start_failure, From patchwork Thu Oct 25 23:32:25 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Beller X-Patchwork-Id: 10656799 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D8D1015A7 for ; Thu, 25 Oct 2018 23:32:52 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CFEF12C6E9 for ; Thu, 25 Oct 2018 23:32:52 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C4D772C6F0; Thu, 25 Oct 2018 23:32:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 673912C6EC for ; Thu, 25 Oct 2018 23:32:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727662AbeJZIH0 (ORCPT ); Fri, 26 Oct 2018 04:07:26 -0400 Received: from mail-qk1-f201.google.com ([209.85.222.201]:37975 "EHLO mail-qk1-f201.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727659AbeJZIHZ (ORCPT ); Fri, 26 Oct 2018 04:07:25 -0400 Received: by mail-qk1-f201.google.com with SMTP id l7-v6so2046147qkd.5 for ; Thu, 25 Oct 2018 16:32:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=opfxhbaPBWglrWsOqD4QxeriUWyogeJbwgs6SADZz3M=; b=g7WM0Y5gr88DP9BeOPSkeKYmbwP3TyHkaMkS59cNzH1tIgPdmfk18wZwzkkZwvWUNE XsIrtlrtOOAmNmJTra9OrA3U8TyK8ADV8xoPzp5rraEF4mKNK5E4wiNSZ0R89H+0FI8a hUcZeALnsRcse+KPwWIKFgsbkMiuJRoEn7PCtHC9zYDmaILdPUHT/Bygln6YAAUrxCSj Ki2BDCQLEQ6cjLke/XBlmz8mJgS1rbcBI2KbSC6RY2nKLPPbZ1aHF2pYN+EPQ9QJDSX3 9ZKnwEiCSLWZobdXoeVpppR4XSsc/H+KAsYUypeVgbkNsuZ2dl8cCrvl/1xpAoDrc0oP htwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=opfxhbaPBWglrWsOqD4QxeriUWyogeJbwgs6SADZz3M=; b=FIswfIi/oIIo+FJK3Xb2IR6j9tMWwdiBcM/yG3BDSofcuu4d8dVZLxboMzOcf2bQ/D OQEuUH5jWxPN11FPVtacMgMnra/xZkbY8fUct1TSvtbaC3TdpbJ8pjKT359b2+xC4t+v TvZIa/dT464Rzyy8dvklD6/+syn6+mRhj5KiQZl9JrODrqLQAIMh/FUdGx5e0SiL2G51 /ZwIykHNx72zpqL3hmRR9t9Sp820uZKwGl7aKrpKlnMO1qA29JwFMGQwahnZQpYtkZM9 i7htyx1C/QsTUk1OudQTYSOWk7bomXefmEhfCGcKq6lmbYuLL2pL8pjqGfF04xsKl6m/ sfjQ== X-Gm-Message-State: AGRZ1gLTZHCk6zfPuFuSX8TOwnLOHjRPbmwdbxpjeESXTJhiAx6y81s/ 8vZzCezz5aEidTb3hHhDoDjs9hjOMDFM X-Google-Smtp-Source: AJdET5eb/7IqpPZ1WUhtJ+ng34I5aAClHs9AlZkE/U4t8wOcBzDt8U70/zTDEYBwVG39Ue21eU45X03m6sHv X-Received: by 2002:a0c:b9a9:: with SMTP id v41mr1092571qvf.43.1540510366723; Thu, 25 Oct 2018 16:32:46 -0700 (PDT) Date: Thu, 25 Oct 2018 16:32:25 -0700 In-Reply-To: <20181025233231.102245-1-sbeller@google.com> Message-Id: <20181025233231.102245-5-sbeller@google.com> Mime-Version: 1.0 References: <20181025233231.102245-1-sbeller@google.com> X-Mailer: git-send-email 2.19.0 Subject: [PATCH 04/10] submodule.c: tighten scope of changed_submodule_names struct From: Stefan Beller To: jonathantanmy@google.com Cc: git@vger.kernel.org, Stefan Beller Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The `changed_submodule_names` are only used for fetching, so let's make it part of the struct that is passed around for fetching submodules. Signed-off-by: Stefan Beller --- submodule.c | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/submodule.c b/submodule.c index 9fbfcfcfe1..6fb0b9d783 100644 --- a/submodule.c +++ b/submodule.c @@ -24,7 +24,6 @@ #include "object-store.h" static int config_update_recurse_submodules = RECURSE_SUBMODULES_OFF; -static struct string_list changed_submodule_names = STRING_LIST_INIT_DUP; static int initialized_fetch_ref_tips; static struct oid_array ref_tips_before_fetch; static struct oid_array ref_tips_after_fetch; @@ -1124,7 +1123,8 @@ void check_for_new_submodule_commits(struct object_id *oid) oid_array_append(&ref_tips_after_fetch, oid); } -static void calculate_changed_submodule_paths(void) +static void calculate_changed_submodule_paths( + struct string_list *changed_submodule_names) { struct argv_array argv = ARGV_ARRAY_INIT; struct string_list changed_submodules = STRING_LIST_INIT_DUP; @@ -1162,7 +1162,8 @@ static void calculate_changed_submodule_paths(void) continue; if (!submodule_has_commits(path, commits)) - string_list_append(&changed_submodule_names, name->string); + string_list_append(changed_submodule_names, + name->string); } free_submodules_oids(&changed_submodules); @@ -1208,8 +1209,10 @@ struct submodule_parallel_fetch { int default_option; int quiet; int result; + + struct string_list changed_submodule_names; }; -#define SPF_INIT {0, ARGV_ARRAY_INIT, NULL, NULL, 0, 0, 0, 0} +#define SPF_INIT {0, ARGV_ARRAY_INIT, NULL, NULL, 0, 0, 0, 0, STRING_LIST_INIT_DUP } static int get_fetch_recurse_config(const struct submodule *submodule, struct submodule_parallel_fetch *spf) @@ -1271,7 +1274,7 @@ static int get_next_submodule(struct child_process *cp, case RECURSE_SUBMODULES_ON_DEMAND: if (!submodule || !string_list_lookup( - &changed_submodule_names, + &spf->changed_submodule_names, submodule->name)) continue; default_argv = "on-demand"; @@ -1363,8 +1366,8 @@ int fetch_populated_submodules(struct repository *r, argv_array_push(&spf.args, "--recurse-submodules-default"); /* default value, "--submodule-prefix" and its value are added later */ - calculate_changed_submodule_paths(); - string_list_sort(&changed_submodule_names); + calculate_changed_submodule_paths(&spf.changed_submodule_names); + string_list_sort(&spf.changed_submodule_names); run_processes_parallel(max_parallel_jobs, get_next_submodule, fetch_start_failure, @@ -1373,7 +1376,7 @@ int fetch_populated_submodules(struct repository *r, argv_array_clear(&spf.args); out: - string_list_clear(&changed_submodule_names, 1); + string_list_clear(&spf.changed_submodule_names, 1); return spf.result; } From patchwork Thu Oct 25 23:32:26 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Beller X-Patchwork-Id: 10656801 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 046EF17F3 for ; Thu, 25 Oct 2018 23:32:53 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EEC162C6EC for ; Thu, 25 Oct 2018 23:32:52 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E375C2C6E9; Thu, 25 Oct 2018 23:32:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 87F5F2C6F1 for ; Thu, 25 Oct 2018 23:32:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727666AbeJZIH2 (ORCPT ); Fri, 26 Oct 2018 04:07:28 -0400 Received: from mail-io1-f74.google.com ([209.85.166.74]:42128 "EHLO mail-io1-f74.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727650AbeJZIH1 (ORCPT ); Fri, 26 Oct 2018 04:07:27 -0400 Received: by mail-io1-f74.google.com with SMTP id d4-v6so5334785ioh.9 for ; Thu, 25 Oct 2018 16:32:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=b8vO50YBsmLkL0nMY7OxbmpOtYnJRHSmdT3806trdv8=; b=RWIfHZ7/Z097fqCmpKKs8ujSyctNKcZSSCAMph7KX6RlUi8fnXoU+KlAXSsV0XjNn0 k6VZjEIaLWkAKcfw4BS6jEfQ0dN6LE0NwH5BK9U2QSyeN9qS7VKshdNousyOA8YJoLKh VYpPqvFc5OyCbKo6G9XTuWL19o11vkiZm/aTDC7y5UzPRjxuwNsNyf732mX0ZYarTgLO JM1s/OJQMzJNQ2A8lpeYJAFcvw6AnBHRFG3uafBmj4vehC14/A/vlE6pTrp7Y4OC0NZh +KcTzzoTJzxuZZFdVLknujAy+WuzhQfEbANFsf0BhQHcAcU/5eA+hWrzzCJPbkw3S3TV zuXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=b8vO50YBsmLkL0nMY7OxbmpOtYnJRHSmdT3806trdv8=; b=sjaPnSN8mBF3dp30IQKMO5ATK0aTTiby3im9KdOi3dZiPlZTQmran2sY/fL990vM2K bfykFye7KHD1HT0XzCVU4kuU5Bkr9Ir+pqryUOXXBITMSjqhU541nQ6SPk6IHJpIU7Xd RszuyPUe8Nz1dlVq2q4xm5Gms16HDXjxt7NtBNK7I9rIWoDpA5ScUTKeEcwC55S84pWX oWsZnoPb80mLCilEqxCk5mAo9zo77he0evTwQ3yfqa7HNewZ/qPEXmF85Ul4nufnushh 4nOjw0AU7vxtCw5j8MM9kbnEMT+9e+oRM1xyiP8SbkSpHSsHF+vQTkWIZ1EtwInO534h Amuw== X-Gm-Message-State: AGRZ1gKoTyGOM9+VzqKFQVuLzm5h+zozT0rQrthw+4ps3fuVFoBKUaWb TKaauWpiSKxWp/gLMcLBCrqYVhbeyHIM X-Google-Smtp-Source: AJdET5dvjlTzynanHztxoSHPTY+NFzDeGQxJDyLFenQGlvxtPKwpFL55M6vNxwSTd0MpxLS2HSi8rZRWu/cx X-Received: by 2002:a24:648e:: with SMTP id t136-v6mr996224itc.25.1540510369312; Thu, 25 Oct 2018 16:32:49 -0700 (PDT) Date: Thu, 25 Oct 2018 16:32:26 -0700 In-Reply-To: <20181025233231.102245-1-sbeller@google.com> Message-Id: <20181025233231.102245-6-sbeller@google.com> Mime-Version: 1.0 References: <20181025233231.102245-1-sbeller@google.com> X-Mailer: git-send-email 2.19.0 Subject: [PATCH 05/10] submodule: store OIDs in changed_submodule_names From: Stefan Beller To: jonathantanmy@google.com Cc: git@vger.kernel.org, Stefan Beller Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP 'calculate_changed_submodule_paths' uses a local list to compute the changed submodules, and then produces the result by copying appropriate items into the result list. Instead use the result list directly and prune items afterwards using string_list_remove_empty_items. By doing so we'll have access to the util pointer for longer that contains the commits that we need to fetch, which will be useful in a later patch. Signed-off-by: Stefan Beller Reviewed-by: Jonathan Tan --- submodule.c | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/submodule.c b/submodule.c index 6fb0b9d783..e4b494af7b 100644 --- a/submodule.c +++ b/submodule.c @@ -1127,8 +1127,7 @@ static void calculate_changed_submodule_paths( struct string_list *changed_submodule_names) { struct argv_array argv = ARGV_ARRAY_INIT; - struct string_list changed_submodules = STRING_LIST_INIT_DUP; - const struct string_list_item *name; + struct string_list_item *name; /* No need to check if there are no submodules configured */ if (!submodule_from_path(the_repository, NULL, NULL)) @@ -1145,9 +1144,9 @@ static void calculate_changed_submodule_paths( * Collect all submodules (whether checked out or not) for which new * commits have been recorded upstream in "changed_submodule_names". */ - collect_changed_submodules(&changed_submodules, &argv); + collect_changed_submodules(changed_submodule_names, &argv); - for_each_string_list_item(name, &changed_submodules) { + for_each_string_list_item(name, changed_submodule_names) { struct oid_array *commits = name->util; const struct submodule *submodule; const char *path = NULL; @@ -1161,12 +1160,14 @@ static void calculate_changed_submodule_paths( if (!path) continue; - if (!submodule_has_commits(path, commits)) - string_list_append(changed_submodule_names, - name->string); + if (submodule_has_commits(path, commits)) { + oid_array_clear(commits); + *name->string = '\0'; + } } - free_submodules_oids(&changed_submodules); + string_list_remove_empty_items(changed_submodule_names, 1); + argv_array_clear(&argv); oid_array_clear(&ref_tips_before_fetch); oid_array_clear(&ref_tips_after_fetch); @@ -1376,7 +1377,7 @@ int fetch_populated_submodules(struct repository *r, argv_array_clear(&spf.args); out: - string_list_clear(&spf.changed_submodule_names, 1); + free_submodules_oids(&spf.changed_submodule_names); return spf.result; } From patchwork Thu Oct 25 23:32:27 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Beller X-Patchwork-Id: 10656803 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8B35613A4 for ; Thu, 25 Oct 2018 23:32:54 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8035F2C6E9 for ; Thu, 25 Oct 2018 23:32:54 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 74B402C6F0; Thu, 25 Oct 2018 23:32:54 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C941A2C6E9 for ; Thu, 25 Oct 2018 23:32:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727669AbeJZIHa (ORCPT ); Fri, 26 Oct 2018 04:07:30 -0400 Received: from mail-io1-f73.google.com ([209.85.166.73]:45300 "EHLO mail-io1-f73.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727650AbeJZIHa (ORCPT ); Fri, 26 Oct 2018 04:07:30 -0400 Received: by mail-io1-f73.google.com with SMTP id t7-v6so3407328ioc.12 for ; Thu, 25 Oct 2018 16:32:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=/gJclo0m+Xp84lGzWv1S8UsmRx7fTgws2wwlzW+JDo0=; b=dvpVIECRWFzs8S6qnGl/1mroSwdlvtjCeFbZAzuaYjiCWkagox8i9hPhY65qBw3KPv OzEeAg7//BPDFSZBTXTFIk66wSjeTriADdL5nVlvXmFsuU3ThacWcsu4B0HEE8odLvPp V+FPbLrIHwxvfISRe35aKoMjIRbp6QHEwKDDWICh8u45bxz70Z7kuzANLfAV1JXVUbNp UC3Oe2TS8f6JBedf1K2Kfj+qIbHjdqripU91n24unGeQnf9EgAe0kdbq5ypGTVdIa62S gNQK2/oH8Y/MQokIYFUidvGXgLOCbRWh5n/CxWl7j8PvTDq5l2ziYaRiJo8xNJMvtPl5 df4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=/gJclo0m+Xp84lGzWv1S8UsmRx7fTgws2wwlzW+JDo0=; b=NcLoUy7utTyr83ymrb/kcdBdrWfreRVdG0HFfQfNsbKH05F13s3JzAVy0jkmkd4bsx CdwClv9kt902TbWmwVpSUnb25MXQiGkwdj3it/vHByGkAkGngIUIXbYmXQOQNZbXMDcs mmYN8WFmILjBk+0pKvGDsUdXStK7BXhyucBO6gVI0U8gwOf4PQLSRe5FfnuMm4T3vPdG kOVq13bJVj+1Sl30K2Dh2PQc+nh+Op7PzaqlHNPdx95gckM4kGCu0i1R2574faWyHdQV cJUm2i3AWv+85dx865xyVqtMvoXohrbWU1igK+nbHzpFCqZCy6bffpcn4lfsCGo3W+7i /0vg== X-Gm-Message-State: AGRZ1gKak4OZrxskPFMSYvD6v7/DTgFYlbLScSURpTZYCaBjsDPvG8Yp BR4bsgBifYqqVq743IqHjKRmZceORWbx X-Google-Smtp-Source: AJdET5esAFvqya12iIuYlZwI91McXMtQALCmXONNyJPOksAl5Drwg4eZxPLnGpfQUHetGmWSfQ/W1NIGaqLt X-Received: by 2002:a24:a048:: with SMTP id o69-v6mr972802ite.39.1540510371702; Thu, 25 Oct 2018 16:32:51 -0700 (PDT) Date: Thu, 25 Oct 2018 16:32:27 -0700 In-Reply-To: <20181025233231.102245-1-sbeller@google.com> Message-Id: <20181025233231.102245-7-sbeller@google.com> Mime-Version: 1.0 References: <20181025233231.102245-1-sbeller@google.com> X-Mailer: git-send-email 2.19.0 Subject: [PATCH 06/10] repository: repo_submodule_init to take a submodule struct From: Stefan Beller To: jonathantanmy@google.com Cc: git@vger.kernel.org, Stefan Beller Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP When constructing a struct repository for a submodule for some revision of the superproject where the submodule is not contained in the index, it may not be present in the working tree currently either. In that situation giving a 'path' argument is not useful. Upgrade the repo_submodule_init function to take a struct submodule instead. The submodule struct can be obtained via submodule_from_{path, name} or an artificial submodule struct can be passed in. While we are at it, rename the repository struct in the repo_submodule_init function, which is to be initialized, to a name that is not confused with the struct submodule as easily. Perform such renames in similar functions as well. Also move its documentation into the header file. Signed-off-by: Stefan Beller --- builtin/grep.c | 17 +++++++----- builtin/ls-files.c | 12 +++++---- builtin/submodule--helper.c | 2 +- repository.c | 27 ++++++++------------ repository.h | 12 +++++++-- t/helper/test-submodule-nested-repo-config.c | 8 +++--- 6 files changed, 43 insertions(+), 35 deletions(-) diff --git a/builtin/grep.c b/builtin/grep.c index 7da8fef31a..ba7634258a 100644 --- a/builtin/grep.c +++ b/builtin/grep.c @@ -418,7 +418,10 @@ static int grep_submodule(struct grep_opt *opt, struct repository *superproject, const struct object_id *oid, const char *filename, const char *path) { - struct repository submodule; + struct repository subrepo; + const struct submodule *sub = submodule_from_path(superproject, + &null_oid, path); + int hit; /* @@ -434,12 +437,12 @@ static int grep_submodule(struct grep_opt *opt, struct repository *superproject, return 0; } - if (repo_submodule_init(&submodule, superproject, path)) { + if (repo_submodule_init(&subrepo, superproject, sub)) { grep_read_unlock(); return 0; } - repo_read_gitmodules(&submodule); + repo_read_gitmodules(&subrepo); /* * NEEDSWORK: This adds the submodule's object directory to the list of @@ -451,7 +454,7 @@ static int grep_submodule(struct grep_opt *opt, struct repository *superproject, * store is no longer global and instead is a member of the repository * object. */ - add_to_alternates_memory(submodule.objects->objectdir); + add_to_alternates_memory(subrepo.objects->objectdir); grep_read_unlock(); if (oid) { @@ -476,14 +479,14 @@ static int grep_submodule(struct grep_opt *opt, struct repository *superproject, init_tree_desc(&tree, data, size); hit = grep_tree(opt, pathspec, &tree, &base, base.len, - object->type == OBJ_COMMIT, &submodule); + object->type == OBJ_COMMIT, &subrepo); strbuf_release(&base); free(data); } else { - hit = grep_cache(opt, &submodule, pathspec, 1); + hit = grep_cache(opt, &subrepo, pathspec, 1); } - repo_clear(&submodule); + repo_clear(&subrepo); return hit; } diff --git a/builtin/ls-files.c b/builtin/ls-files.c index 7f9919a362..4d1649c1b3 100644 --- a/builtin/ls-files.c +++ b/builtin/ls-files.c @@ -206,17 +206,19 @@ static void show_files(struct repository *repo, struct dir_struct *dir); static void show_submodule(struct repository *superproject, struct dir_struct *dir, const char *path) { - struct repository submodule; + struct repository subrepo; + const struct submodule *sub = submodule_from_path(superproject, + &null_oid, path); - if (repo_submodule_init(&submodule, superproject, path)) + if (repo_submodule_init(&subrepo, superproject, sub)) return; - if (repo_read_index(&submodule) < 0) + if (repo_read_index(&subrepo) < 0) die("index file corrupt"); - show_files(&submodule, dir); + show_files(&subrepo, dir); - repo_clear(&submodule); + repo_clear(&subrepo); } static void show_ce(struct repository *repo, struct dir_struct *dir, diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c index 5f8a804a6e..015aa1471f 100644 --- a/builtin/submodule--helper.c +++ b/builtin/submodule--helper.c @@ -2037,7 +2037,7 @@ static int ensure_core_worktree(int argc, const char **argv, const char *prefix) if (!sub) BUG("We could get the submodule handle before?"); - if (repo_submodule_init(&subrepo, the_repository, path)) + if (repo_submodule_init(&subrepo, the_repository, sub)) die(_("could not get a repository handle for submodule '%s'"), path); if (!repo_config_get_string(&subrepo, "core.worktree", &cw)) { diff --git a/repository.c b/repository.c index 5dd1486718..aabe64ee5d 100644 --- a/repository.c +++ b/repository.c @@ -166,30 +166,23 @@ int repo_init(struct repository *repo, return -1; } -/* - * Initialize 'submodule' as the submodule given by 'path' in parent repository - * 'superproject'. - * Return 0 upon success and a non-zero value upon failure. - */ -int repo_submodule_init(struct repository *submodule, +int repo_submodule_init(struct repository *subrepo, struct repository *superproject, - const char *path) + const struct submodule *sub) { - const struct submodule *sub; struct strbuf gitdir = STRBUF_INIT; struct strbuf worktree = STRBUF_INIT; int ret = 0; - sub = submodule_from_path(superproject, &null_oid, path); if (!sub) { ret = -1; goto out; } - strbuf_repo_worktree_path(&gitdir, superproject, "%s/.git", path); - strbuf_repo_worktree_path(&worktree, superproject, "%s", path); + strbuf_repo_worktree_path(&gitdir, superproject, "%s/.git", sub->path); + strbuf_repo_worktree_path(&worktree, superproject, "%s", sub->path); - if (repo_init(submodule, gitdir.buf, worktree.buf)) { + if (repo_init(subrepo, gitdir.buf, worktree.buf)) { /* * If initilization fails then it may be due to the submodule * not being populated in the superproject's worktree. Instead @@ -201,16 +194,16 @@ int repo_submodule_init(struct repository *submodule, strbuf_repo_git_path(&gitdir, superproject, "modules/%s", sub->name); - if (repo_init(submodule, gitdir.buf, NULL)) { + if (repo_init(subrepo, gitdir.buf, NULL)) { ret = -1; goto out; } } - submodule->submodule_prefix = xstrfmt("%s%s/", - superproject->submodule_prefix ? - superproject->submodule_prefix : - "", path); + subrepo->submodule_prefix = xstrfmt("%s%s/", + superproject->submodule_prefix ? + superproject->submodule_prefix : + "", sub->path); out: strbuf_release(&gitdir); diff --git a/repository.h b/repository.h index 9f16c42c1e..0e482b7d49 100644 --- a/repository.h +++ b/repository.h @@ -116,9 +116,17 @@ void repo_set_worktree(struct repository *repo, const char *path); void repo_set_hash_algo(struct repository *repo, int algo); void initialize_the_repository(void); int repo_init(struct repository *r, const char *gitdir, const char *worktree); -int repo_submodule_init(struct repository *submodule, + +/* + * Initialize the repository 'subrepo' as the submodule given by the + * struct submodule 'sub' in parent repository 'superproject'. + * Return 0 upon success and a non-zero value upon failure, which may happen + * if the submodule is not found, or 'sub' is NULL. + */ +struct submodule; +int repo_submodule_init(struct repository *subrepo, struct repository *superproject, - const char *path); + const struct submodule *sub); void repo_clear(struct repository *repo); /* diff --git a/t/helper/test-submodule-nested-repo-config.c b/t/helper/test-submodule-nested-repo-config.c index a31e2a9bea..bc97929bbc 100644 --- a/t/helper/test-submodule-nested-repo-config.c +++ b/t/helper/test-submodule-nested-repo-config.c @@ -10,19 +10,21 @@ static void die_usage(int argc, const char **argv, const char *msg) int cmd__submodule_nested_repo_config(int argc, const char **argv) { - struct repository submodule; + struct repository subrepo; + const struct submodule *sub; if (argc < 3) die_usage(argc, argv, "Wrong number of arguments."); setup_git_directory(); - if (repo_submodule_init(&submodule, the_repository, argv[1])) { + sub = submodule_from_path(the_repository, &null_oid, argv[1]); + if (repo_submodule_init(&subrepo, the_repository, sub)) { die_usage(argc, argv, "Submodule not found."); } /* Read the config of _child_ submodules. */ - print_config_from_gitmodules(&submodule, argv[2]); + print_config_from_gitmodules(&subrepo, argv[2]); submodule_free(the_repository); From patchwork Thu Oct 25 23:32:28 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Beller X-Patchwork-Id: 10656805 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B665413A4 for ; Thu, 25 Oct 2018 23:32:56 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id ABACD2C6E9 for ; Thu, 25 Oct 2018 23:32:56 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A05DC2C6F0; Thu, 25 Oct 2018 23:32:56 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3A37C2C6E9 for ; Thu, 25 Oct 2018 23:32:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727674AbeJZIHc (ORCPT ); Fri, 26 Oct 2018 04:07:32 -0400 Received: from mail-yb1-f201.google.com ([209.85.219.201]:53667 "EHLO mail-yb1-f201.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727650AbeJZIHc (ORCPT ); Fri, 26 Oct 2018 04:07:32 -0400 Received: by mail-yb1-f201.google.com with SMTP id s13-v6so2887633ybj.20 for ; Thu, 25 Oct 2018 16:32:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=vtum7eNCa3t2AxSIn/heEkjb9w63WQZBcA4OB+byAB0=; b=Ne/l46P4zoF8oT0ZZJ0fHQuPJJnC6t1ci3PqwWg5is0W4n1Hfu7cctD1jnQKZHuEcm zgX4oYXnlkM53i8ZrNupsNazjiPxtPbFF9QEqpfS8I+Cs0iAiUmQxybPE/15XvSdf8MS 5mJhQHKKxv9Hy/pYX9IUHIDssezpukV0UVSWSjpAZtdeOQJ8camOVR3rk97xd9QzfVWF xXTGLMzxgU9OZaPJPeIaON4R6B7oJtIOaBrxrCejhkMZDlHZ52GVfeYZ4hoAqD/0o/7o nHOYKFPYwgXwY/AMquRxp2MPWF4wHWOh0E51CFa6x3rWiREmVKCoWcRooQPPtlWPJHBi 8RDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=vtum7eNCa3t2AxSIn/heEkjb9w63WQZBcA4OB+byAB0=; b=jZUHe6ih48SH0ZmLRFaBB496auqq88ag4HXZlMfyyR1G7XadZzz46PUYIO+XkNx3wB 6fXkS4KuD3bYpaNghRWydaL1eVa0/aW3PsARPRRxATaIhv8BApWBldtyhlNkjBi2MrNO MjVjhzbPPw/NcBZ3COJfkVC3NYn4lv+Ic3vgkWwLs8OHb9t25h+eBhTufuE5Y16CABjF J0KuxhZZMERnjN4+qcn6BlK4R73xXUiVJvIC0QKMnHRuaSLCFH4/lKU506gB264kPj1s BRJLdnRtrACOKxPwKBo8duw9KxJohxNKypvTJTDD1rETPogO8i3VQWmmTx+K049xrhZb Iv2Q== X-Gm-Message-State: AGRZ1gKXWFwR7kXcQ3eNGVlRGuHnDTKL0hCd1nyxMiU+0bOZcC4z/Lad qPhW0wM3DBVW8XS/uVF+ph9WjZxaIAAP X-Google-Smtp-Source: AJdET5clqywOyTopZgKf+SrqPwZHisHgGUUUzOuXYMYtY9ssDnD0gP5v6Xe1U9ANnfL8NWQqE34cF1flKrWJ X-Received: by 2002:a5b:18f:: with SMTP id r15-v6mr866027ybl.25.1540510374154; Thu, 25 Oct 2018 16:32:54 -0700 (PDT) Date: Thu, 25 Oct 2018 16:32:28 -0700 In-Reply-To: <20181025233231.102245-1-sbeller@google.com> Message-Id: <20181025233231.102245-8-sbeller@google.com> Mime-Version: 1.0 References: <20181025233231.102245-1-sbeller@google.com> X-Mailer: git-send-email 2.19.0 Subject: [PATCH 07/10] submodule: migrate get_next_submodule to use repository structs From: Stefan Beller To: jonathantanmy@google.com Cc: git@vger.kernel.org, Stefan Beller Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We used to recurse into submodules, even if they were broken having only an objects directory. The child process executed in the submodule would fail though if the submodule was broken. This patch tightens the check upfront, such that we do not need to spawn a child process to find out if the submodule is broken. Signed-off-by: Stefan Beller --- submodule.c | 51 +++++++++++++++++++++++++++++++++++++++------------ 1 file changed, 39 insertions(+), 12 deletions(-) diff --git a/submodule.c b/submodule.c index e4b494af7b..a1a32cab7d 100644 --- a/submodule.c +++ b/submodule.c @@ -1240,6 +1240,29 @@ static int get_fetch_recurse_config(const struct submodule *submodule, return spf->default_option; } +static struct repository *get_submodule_repo_for(struct repository *r, + const struct submodule *sub) +{ + struct repository *ret = xmalloc(sizeof(*ret)); + + if (repo_submodule_init(ret, r, sub)) { + /* + * No entry in .gitmodules? Technically not a submodule, + * but historically we supported repositories that happen to be + * in-place where a gitlink is. Keep supporting them. + */ + struct strbuf gitdir = STRBUF_INIT; + strbuf_repo_worktree_path(&gitdir, r, "%s/.git", sub->path); + if (repo_init(ret, gitdir.buf, NULL)) { + strbuf_release(&gitdir); + return NULL; + } + strbuf_release(&gitdir); + } + + return ret; +} + static int get_next_submodule(struct child_process *cp, struct strbuf *err, void *data, void **task_cb) { @@ -1247,12 +1270,11 @@ static int get_next_submodule(struct child_process *cp, struct submodule_parallel_fetch *spf = data; for (; spf->count < spf->r->index->cache_nr; spf->count++) { - struct strbuf submodule_path = STRBUF_INIT; - struct strbuf submodule_git_dir = STRBUF_INIT; struct strbuf submodule_prefix = STRBUF_INIT; const struct cache_entry *ce = spf->r->index->cache[spf->count]; - const char *git_dir, *default_argv; + const char *default_argv; const struct submodule *submodule; + struct repository *repo; struct submodule default_submodule = SUBMODULE_INIT; if (!S_ISGITLINK(ce->ce_mode)) @@ -1287,16 +1309,12 @@ static int get_next_submodule(struct child_process *cp, continue; } - strbuf_repo_worktree_path(&submodule_path, spf->r, "%s", ce->name); - strbuf_addf(&submodule_git_dir, "%s/.git", submodule_path.buf); strbuf_addf(&submodule_prefix, "%s%s/", spf->prefix, ce->name); - git_dir = read_gitfile(submodule_git_dir.buf); - if (!git_dir) - git_dir = submodule_git_dir.buf; - if (is_directory(git_dir)) { + repo = get_submodule_repo_for(spf->r, submodule); + if (repo) { child_process_init(cp); - cp->dir = strbuf_detach(&submodule_path, NULL); prepare_submodule_repo_env(&cp->env_array); + cp->dir = xstrdup(repo->worktree); cp->git_cmd = 1; if (!spf->quiet) strbuf_addf(err, "Fetching submodule %s%s\n", @@ -1306,10 +1324,19 @@ static int get_next_submodule(struct child_process *cp, argv_array_push(&cp->args, default_argv); argv_array_push(&cp->args, "--submodule-prefix"); argv_array_push(&cp->args, submodule_prefix.buf); + + repo_clear(repo); + free(repo); ret = 1; + } else { + /* + * An empty directory is normal, + * the submodule is not initialized + */ + if (S_ISGITLINK(ce->ce_mode) && + !is_empty_dir(ce->name)) + die(_("Could not access submodule '%s'"), ce->name); } - strbuf_release(&submodule_path); - strbuf_release(&submodule_git_dir); strbuf_release(&submodule_prefix); if (ret) { spf->count++; From patchwork Thu Oct 25 23:32:29 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Beller X-Patchwork-Id: 10656807 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AE46613A4 for ; Thu, 25 Oct 2018 23:32:59 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A41362C6E9 for ; Thu, 25 Oct 2018 23:32:59 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 989072C6F0; Thu, 25 Oct 2018 23:32:59 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 488B12C6E9 for ; Thu, 25 Oct 2018 23:32:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727680AbeJZIHf (ORCPT ); Fri, 26 Oct 2018 04:07:35 -0400 Received: from mail-qt1-f202.google.com ([209.85.160.202]:39839 "EHLO mail-qt1-f202.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727650AbeJZIHf (ORCPT ); Fri, 26 Oct 2018 04:07:35 -0400 Received: by mail-qt1-f202.google.com with SMTP id f3-v6so2738304qtq.6 for ; Thu, 25 Oct 2018 16:32:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=dWhsO3RuVHErbUQo8a8xEXnIEU+fJGorsgZzYVqZ9F0=; b=YP2V7QipIU1wETMqFVyxF/10idI4mV+27U/6Z0HDGAyNoeypBAX8z7s8P48b6OsYMc 2jc2bw6wiV793+HBl5FtQZqNrIPLl3TgeNpIMIgfakgTOOIg8EsMO7/9prHiNOVzV/2P G6g9LF6LO50dy9evJ51HDKHD1lNYf6v2U/5anhQJurVJ/bslXg2WjD2mUFVya4WtQQuV IZSBx59ZYvXX6UrKKy2jSu5tJ6nROg2haLeEYdHfMDFPBHtUww82PwaLL5l1BaPCQTY6 hlq3kcdcSmIE2Xm/LoFf8UKMsbUx8+70aghQyyPuhKKZwjUvfrK33P9g4I8/X0Ax2XL6 y2lw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=dWhsO3RuVHErbUQo8a8xEXnIEU+fJGorsgZzYVqZ9F0=; b=CBJN29yAriTAI1CNyM0NMPoso1xaGeamFKh8ajJd+QsgbBowSCJxQUs/TsfYyJ7a80 ITm2j5ZjKOHgumZ8rSD2PC3CqNG1zg5+Vp0SlmQMLEqK7OopHOjsoHAoSazPfHCjXwgp cL36cJKUber/uogxD6+PYlq1sgNKHwaswqrWc5sUK9kQUSe5pbTlrWB5/HfuAUxFL5U2 OKffpn1wAOKDjp6aqmM5LYrdavpE3Rf/QXHBJbsUSGMLC9mcNymHx4bsYXredxal6+OB d1gCKYfzOIgawgo/StTU3Bv9S12LuRHjYngr916OpwZogn2pRj4xQvCwzB1jpBt6KSRn rBiQ== X-Gm-Message-State: AGRZ1gIi/d22MQNWqd8+C59XJ0THfFKSGFC/rY2Q2fcpcjnLvTTXeSoa Y45sF41e8Qku8X/8OhYA+l5mrQ0JcEAc X-Google-Smtp-Source: AJdET5dgFq3F0112ZGMXm9yIVNTugXzPgk2jLcYRi+k+v+/cFib7IQPPdGYg9GFb3HZ+G9Yu0GdGZb+h9bVJ X-Received: by 2002:a37:4bd2:: with SMTP id y201mr1142596qka.53.1540510377031; Thu, 25 Oct 2018 16:32:57 -0700 (PDT) Date: Thu, 25 Oct 2018 16:32:29 -0700 In-Reply-To: <20181025233231.102245-1-sbeller@google.com> Message-Id: <20181025233231.102245-9-sbeller@google.com> Mime-Version: 1.0 References: <20181025233231.102245-1-sbeller@google.com> X-Mailer: git-send-email 2.19.0 Subject: [PATCH 08/10] submodule.c: fetch in submodules git directory instead of in worktree From: Stefan Beller To: jonathantanmy@google.com Cc: git@vger.kernel.org, Stefan Beller Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Keep the properties introduced in 10f5c52656 (submodule: avoid auto-discovery in prepare_submodule_repo_env(), 2016-09-01), by fixating the git directory of the submodule. Signed-off-by: Stefan Beller --- submodule.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/submodule.c b/submodule.c index a1a32cab7d..67813fbe78 100644 --- a/submodule.c +++ b/submodule.c @@ -494,6 +494,12 @@ void prepare_submodule_repo_env(struct argv_array *out) DEFAULT_GIT_DIR_ENVIRONMENT); } +static void prepare_submodule_repo_env_in_gitdir(struct argv_array *out) +{ + prepare_submodule_repo_env_no_git_dir(out); + argv_array_pushf(out, "%s=.", GIT_DIR_ENVIRONMENT); +} + /* Helper function to display the submodule header line prior to the full * summary output. If it can locate the submodule objects directory it will * attempt to lookup both the left and right commits and put them into the @@ -1313,8 +1319,8 @@ static int get_next_submodule(struct child_process *cp, repo = get_submodule_repo_for(spf->r, submodule); if (repo) { child_process_init(cp); - prepare_submodule_repo_env(&cp->env_array); - cp->dir = xstrdup(repo->worktree); + prepare_submodule_repo_env_in_gitdir(&cp->env_array); + cp->dir = xstrdup(repo->gitdir); cp->git_cmd = 1; if (!spf->quiet) strbuf_addf(err, "Fetching submodule %s%s\n", From patchwork Thu Oct 25 23:32:30 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Beller X-Patchwork-Id: 10656809 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 83B9F13BF for ; Thu, 25 Oct 2018 23:33:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 787F52C6E9 for ; Thu, 25 Oct 2018 23:33:04 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6D0C72C6F1; Thu, 25 Oct 2018 23:33:04 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 81E352C6E9 for ; Thu, 25 Oct 2018 23:33:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727685AbeJZIHj (ORCPT ); Fri, 26 Oct 2018 04:07:39 -0400 Received: from mail-io1-f74.google.com ([209.85.166.74]:54771 "EHLO mail-io1-f74.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727650AbeJZIHj (ORCPT ); Fri, 26 Oct 2018 04:07:39 -0400 Received: by mail-io1-f74.google.com with SMTP id q26-v6so6766341ioi.21 for ; Thu, 25 Oct 2018 16:33:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=xnFFld3YqPX5nlx/Ef4tpPqOz7dO5K9t9tQOaql7OTE=; b=BoZBFQurDp/GgnGyNG9gGGdCOScOVM/RYgCND9sP9yPh4o6FPjYKDm+Tv7u0upY3dn YL9UdQ44SBldYcir8KZC1lDDMfQySty9p3gtGRHdqlrRJ7NPb561qaoBFO49PKJX71Fs 5l+CjN+o6lO/eDVVvMAweLtYe0MyvYDhr2Kt/u8xVPq5M6oySQR0Vnd+Fam5YxUZzQ71 x3zAOZ/1+B5Qp2iW8GD1Wdic+HhFGrDn5YBng+eb8zUXynsOcgri3HYAeazx8/x+CrtQ QcXtOIcB+7xjV2kfnK0IpCd4NvaXqz+pL+CsMBC5mPFYuiw0woRLjqAiTvzj8Jf6qNxa ziPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=xnFFld3YqPX5nlx/Ef4tpPqOz7dO5K9t9tQOaql7OTE=; b=F6E1m4usb4mZ7rTiv/4v6RDpOPYpFDhryGeVrkUMx7t64Z5s9AvJdzOis5b3xOypzS cyIwa8iEGm91QWumFLqx92G9y7nNiWH5UDwuSrqWPcL7Ou2irDTxJxv1unyHOWu04OBF fUIiN40VKvoA20Si7NbDe8Se6M8qRjyC/xUlpgPwaGBHJS52jhAvGRQsF/yKvHR4PmIw I/Pzeg93gZ94mTsC7xfvuMfb1WaTX3jZtxJmjVNyfj6AuvCbAOnokwMpyxvcr8HV62J/ AdCpYYLz2cMC9GUzOPk1MPXyrHPD34XHy128D9ktgnNux5wtGZSpbRZ7rH1I1DgLNRg7 V6kg== X-Gm-Message-State: AGRZ1gLgPi8EgYabkutcHk7+fhnOzaiYNUqqnVHG0/Ug4y8njNVSVuV0 S7svlMPVsytZbMM74R2EckJF1uccrEub X-Google-Smtp-Source: AJdET5fOxpWbCMn+m3NUTOd5Em1mXFG+Ls9jCn70DLaHikEJ8MaVvbexGh/362ukBw4w+0g9551JPZk9M7ct X-Received: by 2002:a24:8082:: with SMTP id g124-v6mr1087917itd.12.1540510379790; Thu, 25 Oct 2018 16:32:59 -0700 (PDT) Date: Thu, 25 Oct 2018 16:32:30 -0700 In-Reply-To: <20181025233231.102245-1-sbeller@google.com> Message-Id: <20181025233231.102245-10-sbeller@google.com> Mime-Version: 1.0 References: <20181025233231.102245-1-sbeller@google.com> X-Mailer: git-send-email 2.19.0 Subject: [PATCH 09/10] fetch: try fetching submodules if needed objects were not fetched From: Stefan Beller To: jonathantanmy@google.com Cc: git@vger.kernel.org, Stefan Beller Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Currently when git-fetch is asked to recurse into submodules, it dispatches a plain "git-fetch -C " (with some submodule related options such as prefix and recusing strategy, but) without any information of the remote or the tip that should be fetched. But this default fetch is not sufficient, as a newly fetched commit in the superproject could point to a commit in the submodule that is not in the default refspec. This is common in workflows like Gerrit's. When fetching a Gerrit change under review (from refs/changes/??), the commits in that change likely point to submodule commits that have not been merged to a branch yet. Try fetching a submodule by object id if the object id that the superproject points to, cannot be found. The try does not happen when the "git fetch" done at the superproject is not storing the fetched results in remote tracking branches (i.e. instead just recording them to FETCH_HEAD) in this step. A later patch will fix this. builtin/fetch used to only inspect submodules when they were fetched "on-demand", as in either on/off case it was clear whether the submodule needs to be fetched. However to know whether we need to try fetching the object ids, we need to identify the object names, which is done in this function check_for_new_submodule_commits(), so we'll also run that code in case the submodule recursion is set to "on". Signed-off-by: Stefan Beller --- builtin/fetch.c | 9 +- submodule.c | 192 ++++++++++++++++++++++++++++++------ t/t5526-fetch-submodules.sh | 31 ++++++ 3 files changed, 198 insertions(+), 34 deletions(-) diff --git a/builtin/fetch.c b/builtin/fetch.c index 61bec5d213..95c44bf6ff 100644 --- a/builtin/fetch.c +++ b/builtin/fetch.c @@ -700,8 +700,7 @@ static int update_local_ref(struct ref *ref, what = _("[new ref]"); } - if ((recurse_submodules != RECURSE_SUBMODULES_OFF) && - (recurse_submodules != RECURSE_SUBMODULES_ON)) + if (recurse_submodules != RECURSE_SUBMODULES_OFF) check_for_new_submodule_commits(&ref->new_oid); r = s_update_ref(msg, ref, 0); format_display(display, r ? '!' : '*', what, @@ -716,8 +715,7 @@ static int update_local_ref(struct ref *ref, strbuf_add_unique_abbrev(&quickref, ¤t->object.oid, DEFAULT_ABBREV); strbuf_addstr(&quickref, ".."); strbuf_add_unique_abbrev(&quickref, &ref->new_oid, DEFAULT_ABBREV); - if ((recurse_submodules != RECURSE_SUBMODULES_OFF) && - (recurse_submodules != RECURSE_SUBMODULES_ON)) + if (recurse_submodules != RECURSE_SUBMODULES_OFF) check_for_new_submodule_commits(&ref->new_oid); r = s_update_ref("fast-forward", ref, 1); format_display(display, r ? '!' : ' ', quickref.buf, @@ -731,8 +729,7 @@ static int update_local_ref(struct ref *ref, strbuf_add_unique_abbrev(&quickref, ¤t->object.oid, DEFAULT_ABBREV); strbuf_addstr(&quickref, "..."); strbuf_add_unique_abbrev(&quickref, &ref->new_oid, DEFAULT_ABBREV); - if ((recurse_submodules != RECURSE_SUBMODULES_OFF) && - (recurse_submodules != RECURSE_SUBMODULES_ON)) + if (recurse_submodules != RECURSE_SUBMODULES_OFF) check_for_new_submodule_commits(&ref->new_oid); r = s_update_ref("forced-update", ref, 1); format_display(display, r ? '!' : '+', quickref.buf, diff --git a/submodule.c b/submodule.c index 67813fbe78..c978a38c81 100644 --- a/submodule.c +++ b/submodule.c @@ -1218,8 +1218,12 @@ struct submodule_parallel_fetch { int result; struct string_list changed_submodule_names; + struct get_next_submodule_task **fetch_specific_oids; + int fetch_specific_oids_nr, fetch_specific_oids_alloc; }; -#define SPF_INIT {0, ARGV_ARRAY_INIT, NULL, NULL, 0, 0, 0, 0, STRING_LIST_INIT_DUP } +#define SPF_INIT {0, ARGV_ARRAY_INIT, NULL, NULL, 0, 0, 0, 0, \ + STRING_LIST_INIT_DUP, \ + NULL, 0, 0} static int get_fetch_recurse_config(const struct submodule *submodule, struct submodule_parallel_fetch *spf) @@ -1246,6 +1250,58 @@ static int get_fetch_recurse_config(const struct submodule *submodule, return spf->default_option; } +struct get_next_submodule_task { + struct repository *repo; + const struct submodule *sub; + unsigned free_sub : 1; /* Do we need to free the submodule? */ + + /* fetch specific oids if set, otherwise fetch default refspec */ + struct oid_array *commits; +}; + +static const struct submodule *get_default_submodule(const char *path) +{ + struct submodule *ret = NULL; + const char *name = default_name_or_path(path); + + if (!name) + return NULL; + + ret = xmalloc(sizeof(*ret)); + memset(ret, 0, sizeof(*ret)); + ret->path = name; + ret->name = name; + + return (const struct submodule *) ret; +} + +static struct get_next_submodule_task *get_next_submodule_task_create( + struct repository *r, const char *path) +{ + struct get_next_submodule_task *task = xmalloc(sizeof(*task)); + memset(task, 0, sizeof(*task)); + + task->sub = submodule_from_path(r, &null_oid, path); + if (!task->sub) { + task->sub = get_default_submodule(path); + task->free_sub = 1; + } + + return task; +} + +static void get_next_submodule_task_release(struct get_next_submodule_task *p) +{ + if (p->free_sub) + free((void*)p->sub); + p->free_sub = 0; + p->sub = NULL; + + if (p->repo) + repo_clear(p->repo); + FREE_AND_NULL(p->repo); +} + static struct repository *get_submodule_repo_for(struct repository *r, const struct submodule *sub) { @@ -1272,39 +1328,35 @@ static struct repository *get_submodule_repo_for(struct repository *r, static int get_next_submodule(struct child_process *cp, struct strbuf *err, void *data, void **task_cb) { - int ret = 0; struct submodule_parallel_fetch *spf = data; for (; spf->count < spf->r->index->cache_nr; spf->count++) { - struct strbuf submodule_prefix = STRBUF_INIT; + int recurse_config; const struct cache_entry *ce = spf->r->index->cache[spf->count]; const char *default_argv; - const struct submodule *submodule; - struct repository *repo; - struct submodule default_submodule = SUBMODULE_INIT; + struct get_next_submodule_task *task; if (!S_ISGITLINK(ce->ce_mode)) continue; - submodule = submodule_from_path(spf->r, &null_oid, ce->name); - if (!submodule) { - const char *name = default_name_or_path(ce->name); - if (name) { - default_submodule.path = name; - default_submodule.name = name; - submodule = &default_submodule; - } + task = get_next_submodule_task_create(spf->r, ce->name); + + if (!task->sub) { + free(task); + continue; } - switch (get_fetch_recurse_config(submodule, spf)) + recurse_config = get_fetch_recurse_config(task->sub, spf); + + switch (recurse_config) { default: case RECURSE_SUBMODULES_DEFAULT: case RECURSE_SUBMODULES_ON_DEMAND: - if (!submodule || + if (!task->sub || !string_list_lookup( &spf->changed_submodule_names, - submodule->name)) + task->sub->name)) continue; default_argv = "on-demand"; break; @@ -1315,12 +1367,12 @@ static int get_next_submodule(struct child_process *cp, continue; } - strbuf_addf(&submodule_prefix, "%s%s/", spf->prefix, ce->name); - repo = get_submodule_repo_for(spf->r, submodule); - if (repo) { + task->repo = get_submodule_repo_for(spf->r, task->sub); + if (task->repo) { + struct strbuf submodule_prefix = STRBUF_INIT; child_process_init(cp); prepare_submodule_repo_env_in_gitdir(&cp->env_array); - cp->dir = xstrdup(repo->gitdir); + cp->dir = task->repo->gitdir; cp->git_cmd = 1; if (!spf->quiet) strbuf_addf(err, "Fetching submodule %s%s\n", @@ -1329,12 +1381,22 @@ static int get_next_submodule(struct child_process *cp, argv_array_pushv(&cp->args, spf->args.argv); argv_array_push(&cp->args, default_argv); argv_array_push(&cp->args, "--submodule-prefix"); + + strbuf_addf(&submodule_prefix, "%s%s/", + spf->prefix, + task->sub->path); argv_array_push(&cp->args, submodule_prefix.buf); - repo_clear(repo); - free(repo); - ret = 1; + spf->count++; + *task_cb = task; + + strbuf_release(&submodule_prefix); + return 1; } else { + + get_next_submodule_task_release(task); + free(task); + /* * An empty directory is normal, * the submodule is not initialized @@ -1343,12 +1405,36 @@ static int get_next_submodule(struct child_process *cp, !is_empty_dir(ce->name)) die(_("Could not access submodule '%s'"), ce->name); } + } + + if (spf->fetch_specific_oids_nr) { + struct get_next_submodule_task *task = spf->fetch_specific_oids[spf->fetch_specific_oids_nr - 1]; + struct strbuf submodule_prefix = STRBUF_INIT; + spf->fetch_specific_oids_nr--; + + strbuf_addf(&submodule_prefix, "%s%s/", spf->prefix, task->sub->path); + + child_process_init(cp); + prepare_submodule_repo_env_in_gitdir(&cp->env_array); + cp->git_cmd = 1; + cp->dir = task->repo->gitdir; + + argv_array_init(&cp->args); + argv_array_pushv(&cp->args, spf->args.argv); + argv_array_push(&cp->args, "on-demand"); + argv_array_push(&cp->args, "--submodule-prefix"); + argv_array_push(&cp->args, submodule_prefix.buf); + + /* NEEDSWORK: have get_default_remote from s--h */ + argv_array_push(&cp->args, "origin"); + oid_array_for_each_unique(task->commits, + append_oid_to_argv, &cp->args); + + *task_cb = task; strbuf_release(&submodule_prefix); - if (ret) { - spf->count++; - return 1; - } + return 1; } + return 0; } @@ -1356,20 +1442,70 @@ static int fetch_start_failure(struct strbuf *err, void *cb, void *task_cb) { struct submodule_parallel_fetch *spf = cb; + struct get_next_submodule_task *task = task_cb; spf->result = 1; + get_next_submodule_task_release(task); return 0; } +static int commit_exists_in_sub(const struct object_id *oid, void *data) +{ + struct repository *subrepo = data; + + enum object_type type = oid_object_info(subrepo, oid, NULL); + + return type != OBJ_COMMIT; +} + static int fetch_finish(int retvalue, struct strbuf *err, void *cb, void *task_cb) { struct submodule_parallel_fetch *spf = cb; + struct get_next_submodule_task *task = task_cb; + const struct submodule *sub; + + struct string_list_item *it; + struct oid_array *commits; if (retvalue) spf->result = 1; + if (!task) + return 0; + + sub = task->sub; + if (!sub) + goto out; + + it = string_list_lookup(&spf->changed_submodule_names, sub->name); + if (!it) + goto out; + + commits = it->util; + oid_array_filter(commits, + commit_exists_in_sub, + task->repo); + + /* Are there commits that do not exist? */ + if (commits->nr) { + /* We already tried fetching them, do not try again. */ + if (task->commits) + return 0; + + task->commits = commits; + ALLOC_GROW(spf->fetch_specific_oids, + spf->fetch_specific_oids_nr + 1, + spf->fetch_specific_oids_alloc); + spf->fetch_specific_oids[spf->fetch_specific_oids_nr] = task; + spf->fetch_specific_oids_nr++; + return 0; + } + +out: + get_next_submodule_task_release(task); + return 0; } diff --git a/t/t5526-fetch-submodules.sh b/t/t5526-fetch-submodules.sh index 6c2f9b2ba2..5a75b57852 100755 --- a/t/t5526-fetch-submodules.sh +++ b/t/t5526-fetch-submodules.sh @@ -600,4 +600,35 @@ test_expect_success "fetch new commits when submodule got renamed" ' test_cmp expect actual ' +test_expect_success "fetch new submodule commits on-demand outside standard refspec" ' + # add a second submodule and ensure it is around in downstream first + git clone submodule sub1 && + git submodule add ./sub1 && + git commit -m "adding a second submodule" && + git -C downstream pull && + git -C downstream submodule update --init --recursive && + + git checkout --detach && + + C=$(git -C submodule commit-tree -m "new change outside refs/heads" HEAD^{tree}) && + git -C submodule update-ref refs/changes/1 $C && + git update-index --cacheinfo 160000 $C submodule && + test_tick && + + D=$(git -C sub1 commit-tree -m "new change outside refs/heads" HEAD^{tree}) && + git -C sub1 update-ref refs/changes/2 $D && + git update-index --cacheinfo 160000 $D sub1 && + + git commit -m "updated submodules outside of refs/heads" && + E=$(git rev-parse HEAD) && + git update-ref refs/changes/2 $E && + ( + cd downstream && + git fetch --recurse-submodules origin refs/changes/2:refs/heads/my_branch && + git -C submodule cat-file -t $C && + git -C sub1 cat-file -t $D && + git checkout --recurse-submodules FETCH_HEAD + ) +' + test_done From patchwork Thu Oct 25 23:32:31 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Beller X-Patchwork-Id: 10656811 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AF13A15A7 for ; Thu, 25 Oct 2018 23:33:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A45392C6E9 for ; Thu, 25 Oct 2018 23:33:04 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 98A512C6F0; Thu, 25 Oct 2018 23:33:04 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 26ABF2C6EC for ; Thu, 25 Oct 2018 23:33:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727686AbeJZIHl (ORCPT ); Fri, 26 Oct 2018 04:07:41 -0400 Received: from mail-yb1-f201.google.com ([209.85.219.201]:43355 "EHLO mail-yb1-f201.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727582AbeJZIHk (ORCPT ); Fri, 26 Oct 2018 04:07:40 -0400 Received: by mail-yb1-f201.google.com with SMTP id c6-v6so6876569ybm.10 for ; Thu, 25 Oct 2018 16:33:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=AVaboUMQrDQEcJvQHGNvju2g1A+oI4XM1VDQkjDem4o=; b=AVR5cXr1FHVfhri8aevqDwKX6k7T9zQUkmPVjRJrwZWJ1g574THHOnlpNk7PVvF7Rm IPl6TKWz8L41iWtQxIbQm1Kla9sVQpAtpFwBUC8x8GRsMJPV622p7JlcOw7Oj9Y1vj1I XcQtV0drsFBb5vTUepgl25T5idflAZhq+sf7+W6xeoSkjTip+SOjp9kz0OCVDr3tm35S 0oi8Cf5BELY2kB6QjD7/5l5oFkjNbucK+tppOFCb6w5izXBZiXn0A768+kCiCaqcsF2y 5Cz3fzjy3OHaCRGQMIvJr9LdRoI3Pgu894Zpc3eCgMRH9DYt8BbqgUAAVEC9GzNLudHo qAoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=AVaboUMQrDQEcJvQHGNvju2g1A+oI4XM1VDQkjDem4o=; b=aula5nP4bIEebiWuRBsjHkehnCRBmBAGCMRP2dJ0abzWD9uvVaQ/4+vCbGQudQweiE ugv2oJ2cjICxmSzkwtb/MRRjMDIgr+i45F4bVrXtOvTgZjxEeJZahs9DYAPUNi4v+l0G hjOxBMCRGgO2zmByAsEsUqxcfaCj+wniqEvFY9GUpjKa2hQ7fkT3DAI/pvBy2XkYo7pD TvLIffKUBWk87Ko5ntvltTcoI/UA0jDCJXvsmn++jNUcZrx4prQvPGWgoxhayyXXgcdG RVJRboPeTU8mymTgUVsEx4wWP0P3KDW58Ry7kal9cu8gNlESNUCHs9hrd+l8oTFULchb Qj3w== X-Gm-Message-State: AGRZ1gIWp06f86YbKaU9gAlMjmv7sycBkBMn40lsfD9opdAUfIn3SU5F pcKsk+PQQkCjbLL56IDcYGY7X+RkbMjQ X-Google-Smtp-Source: AJdET5dxhmLZINYxdL5sExDWQLx3vqbgVySJcjb1Oc2Z5UqClt8YGCUufy/rU09rd99xo1fSfQst+iVJLS6Q X-Received: by 2002:a25:bb13:: with SMTP id z19-v6mr903918ybg.23.1540510382472; Thu, 25 Oct 2018 16:33:02 -0700 (PDT) Date: Thu, 25 Oct 2018 16:32:31 -0700 In-Reply-To: <20181025233231.102245-1-sbeller@google.com> Message-Id: <20181025233231.102245-11-sbeller@google.com> Mime-Version: 1.0 References: <20181025233231.102245-1-sbeller@google.com> X-Mailer: git-send-email 2.19.0 Subject: [PATCH 10/10] builtin/fetch: check for submodule updates in any ref update From: Stefan Beller To: jonathantanmy@google.com Cc: git@vger.kernel.org, Stefan Beller Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Gerrit, the code review tool, has a different workflow than our mailing list based approach. Usually users upload changes to a Gerrit server and continuous integration and testing happens by bots. Sometimes however a user wants to checkout a change locally and look at it locally. For this use case, Gerrit offers a command line snippet to copy and paste to your terminal, which looks like git fetch https:///gerrit refs/changes/ && git checkout FETCH_HEAD For Gerrit changes that contain changing submodule gitlinks, it would be easy to extend both the fetch and checkout with the '--recurse-submodules' flag, such that this command line snippet would produce the state of a change locally. However the functionality added in the previous patch, which would ensure that we fetch the objects in the submodule that the gitlink pointed at, only works for remote tracking branches so far, not for FETCH_HEAD. Make sure that fetching a superproject to its FETCH_HEAD, also respects the existence checks for objects in the submodule recursion. Signed-off-by: Stefan Beller --- builtin/fetch.c | 8 ++------ t/t5526-fetch-submodules.sh | 24 ++++++++++++++++++++++++ 2 files changed, 26 insertions(+), 6 deletions(-) diff --git a/builtin/fetch.c b/builtin/fetch.c index 95c44bf6ff..f39012d7c2 100644 --- a/builtin/fetch.c +++ b/builtin/fetch.c @@ -700,8 +700,6 @@ static int update_local_ref(struct ref *ref, what = _("[new ref]"); } - if (recurse_submodules != RECURSE_SUBMODULES_OFF) - check_for_new_submodule_commits(&ref->new_oid); r = s_update_ref(msg, ref, 0); format_display(display, r ? '!' : '*', what, r ? _("unable to update local ref") : NULL, @@ -715,8 +713,6 @@ static int update_local_ref(struct ref *ref, strbuf_add_unique_abbrev(&quickref, ¤t->object.oid, DEFAULT_ABBREV); strbuf_addstr(&quickref, ".."); strbuf_add_unique_abbrev(&quickref, &ref->new_oid, DEFAULT_ABBREV); - if (recurse_submodules != RECURSE_SUBMODULES_OFF) - check_for_new_submodule_commits(&ref->new_oid); r = s_update_ref("fast-forward", ref, 1); format_display(display, r ? '!' : ' ', quickref.buf, r ? _("unable to update local ref") : NULL, @@ -729,8 +725,6 @@ static int update_local_ref(struct ref *ref, strbuf_add_unique_abbrev(&quickref, ¤t->object.oid, DEFAULT_ABBREV); strbuf_addstr(&quickref, "..."); strbuf_add_unique_abbrev(&quickref, &ref->new_oid, DEFAULT_ABBREV); - if (recurse_submodules != RECURSE_SUBMODULES_OFF) - check_for_new_submodule_commits(&ref->new_oid); r = s_update_ref("forced-update", ref, 1); format_display(display, r ? '!' : '+', quickref.buf, r ? _("unable to update local ref") : _("forced update"), @@ -826,6 +820,8 @@ static int store_updated_refs(const char *raw_url, const char *remote_name, ref->force = rm->peer_ref->force; } + if (recurse_submodules != RECURSE_SUBMODULES_OFF) + check_for_new_submodule_commits(&rm->old_oid); if (!strcmp(rm->name, "HEAD")) { kind = ""; diff --git a/t/t5526-fetch-submodules.sh b/t/t5526-fetch-submodules.sh index 5a75b57852..799785783f 100755 --- a/t/t5526-fetch-submodules.sh +++ b/t/t5526-fetch-submodules.sh @@ -631,4 +631,28 @@ test_expect_success "fetch new submodule commits on-demand outside standard refs ) ' +test_expect_success 'fetch new submodule commits on-demand in FETCH_HEAD' ' + # depends on the previous test for setup + + C=$(git -C submodule commit-tree -m "another change outside refs/heads" HEAD^{tree}) && + git -C submodule update-ref refs/changes/1 $C && + git update-index --cacheinfo 160000 $C submodule && + test_tick && + + D=$(git -C sub1 commit-tree -m "another change outside refs/heads" HEAD^{tree}) && + git -C sub1 update-ref refs/changes/2 $D && + git update-index --cacheinfo 160000 $D sub1 && + + git commit -m "updated submodules outside of refs/heads" && + E=$(git rev-parse HEAD) && + git update-ref refs/changes/2 $E && + ( + cd downstream && + git fetch --recurse-submodules origin refs/changes/2 && + git -C submodule cat-file -t $C && + git -C sub1 cat-file -t $D && + git checkout --recurse-submodules FETCH_HEAD + ) +' + test_done