From patchwork Wed May 6 00:07:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 11530073 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BABE881 for ; Wed, 6 May 2020 00:07:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A352120752 for ; Wed, 6 May 2020 00:07:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20150623.gappssmtp.com header.i=@ttaylorr-com.20150623.gappssmtp.com header.b="igQ0fOFr" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729170AbgEFAHe (ORCPT ); Tue, 5 May 2020 20:07:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57136 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728717AbgEFAHe (ORCPT ); Tue, 5 May 2020 20:07:34 -0400 Received: from mail-pl1-x641.google.com (mail-pl1-x641.google.com [IPv6:2607:f8b0:4864:20::641]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3DDFFC061A0F for ; Tue, 5 May 2020 17:07:34 -0700 (PDT) Received: by mail-pl1-x641.google.com with SMTP id u22so1512467plq.12 for ; Tue, 05 May 2020 17:07:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=yn/llej/jTolC/XlId/Yu+ZvA9EsPYY1FRqM7XVwbno=; b=igQ0fOFrO7nqGjNseTdOtLWPvGQJ9jM+HD2BCo28TMKiy/J4RS9VQ6pm7q2FURL3yc 3KICppwk8CEgDYeYqrUD3QWh0t07V5yullYX5aBoqW/3LRRlIpwuVOAVMK1KgsZo+YS4 lrJqg4Uni48cTUvfo06EQ5N1pE7RGxnQs5N7unbQiVruA3DCacHRuLd14ZOAGohiFvcs BA5ys5/jm58vUYk/zFBcnBsbw1+xh7XafHtgtOOxCthn35S890Z2QxlF0i7WLvrpwXqH VX7dDiS9syDuBZcSIOaRvszW+njo7/IZiv2bFcMTRFZgK5/W0mhK0wuHBnrz/Gkk2Dn5 mkPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=yn/llej/jTolC/XlId/Yu+ZvA9EsPYY1FRqM7XVwbno=; b=f90EZyzEaF/qVt11iW31g6Cyr7svdOxLCRHoTqB8yE77qa/SNG6lIGip13P5KQiT+q kThDnzGzQBXGUu0Orx9Ip2RBRt/cZhm+MjpHeNmJRcgLfVvz1kTp0hdYUv6B/sg5ndLT f+o8RPoqE2hXa9vxU9LuLX8X1t3Lu84FY73K8NWXe+bj7WAwGluU5OL87EDWfrG1lyjG BUnEwuSEl/KFMZ8Q33beKKhP6Kt03yKFjtbHtR9N7yjyEcTFFBLR8kWuXD6F8JfxJyRt uG9cVOfzZ492rQFkD/P1GAwLZoOTzL391jY2UcRkfmuGquhIjt8Zs7/BsJcMIJFw/jx3 bx1Q== X-Gm-Message-State: AGi0PuaM0YPyDcvlRjH4V1oJamqJhMYJQybdsTd5if3vruf+eK66/R7y yUeuPvRe3JG8nY/FGe8vxaiaLm0ARV0= X-Google-Smtp-Source: APiQypL5ZDHlUqrRW39FlUBd+rPlQ24z3hmxHRv8sx49FZF1+RqvfRljXC+4JG+wwGIiAbZlyVNjHg== X-Received: by 2002:a17:90a:2149:: with SMTP id a67mr6251768pje.43.1588723653400; Tue, 05 May 2020 17:07:33 -0700 (PDT) Received: from localhost ([8.44.146.30]) by smtp.gmail.com with ESMTPSA id d12sm2957264pjx.1.2020.05.05.17.07.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 May 2020 17:07:32 -0700 (PDT) Date: Tue, 5 May 2020 18:07:31 -0600 From: Taylor Blau To: git@vger.kernel.org Cc: dstolee@microsoft.com, gitster@pobox.com, peff@peff.net, szeder.dev@gmail.com Subject: [PATCH v2 1/8] commit-graph.c: extract 'refs_cb_data' Message-ID: <43286c3c45b2dcef8d9d39b4998c17f364eb1060.1588723544.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In subsequent patches, we are going to update a progress meter when 'add_ref_to_set()' is called, and need a convenient way to pass a 'struct progress *' in from the caller. Introduce 'refs_cb_data' as a catch-all for parameters that 'add_ref_to_set' may need, and wrap the existing single parameter in that struct. Signed-off-by: Taylor Blau --- commit-graph.c | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/commit-graph.c b/commit-graph.c index 6dc777e2f3..00da281f39 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -1318,13 +1318,17 @@ static void compute_bloom_filters(struct write_commit_graph_context *ctx) stop_progress(&progress); } +struct refs_cb_data { + struct oidset *commits; +}; + static int add_ref_to_set(const char *refname, const struct object_id *oid, int flags, void *cb_data) { - struct oidset *commits = (struct oidset *)cb_data; + struct refs_cb_data *data = (struct refs_cb_data *)cb_data; - oidset_insert(commits, oid); + oidset_insert(data->commits, oid); return 0; } @@ -1333,9 +1337,13 @@ int write_commit_graph_reachable(struct object_directory *odb, const struct split_commit_graph_opts *split_opts) { struct oidset commits = OIDSET_INIT; + struct refs_cb_data data; int result; - for_each_ref(add_ref_to_set, &commits); + memset(&data, 0, sizeof(data)); + data.commits = &commits; + + for_each_ref(add_ref_to_set, &data); result = write_commit_graph(odb, NULL, &commits, flags, split_opts); From patchwork Wed May 6 00:07:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 11530075 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 48A0181 for ; Wed, 6 May 2020 00:07:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3116120721 for ; Wed, 6 May 2020 00:07:43 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20150623.gappssmtp.com header.i=@ttaylorr-com.20150623.gappssmtp.com header.b="SWmHSjEA" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729398AbgEFAHm (ORCPT ); Tue, 5 May 2020 20:07:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57144 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728717AbgEFAHj (ORCPT ); Tue, 5 May 2020 20:07:39 -0400 Received: from mail-pj1-x1043.google.com (mail-pj1-x1043.google.com [IPv6:2607:f8b0:4864:20::1043]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 04DC8C061A0F for ; Tue, 5 May 2020 17:07:38 -0700 (PDT) Received: by mail-pj1-x1043.google.com with SMTP id mq3so63474pjb.1 for ; Tue, 05 May 2020 17:07:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=iu3L/AEcjj8ZCUAc3p5nfa00WagHa7V7ljbKMe2JdAM=; b=SWmHSjEAak/eN2aH4hE54+PIMvJofmPzs7NiCjLuO+Oii3O0j2JxHcvEcBOmHLmlQz 5k3DniyrzJ6pR0F+K6n1xoSa4k8k8Ld1jLRDfBY4N3fHdStsuw3BC8EJ2fB6LkGwQKwd 3s+0FmR252hdr5qv884r6lQjMhZ7IS6Zwn8DqTfsklL9LOABEAYxWYqPGCH0p1K4rMQk WH7VZwptAu8XWYF+/tQhkl4TbHTC0ugS5i1GlplmEE78250iGnf9fCYgaHZII313eYSG l3C5NvHN5pxhIuo9W7gzfp7Ems+/1IH3nIDcaHW1CvV4AplEqCjP3+mnlF9P+BG1RHIl 6Hdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=iu3L/AEcjj8ZCUAc3p5nfa00WagHa7V7ljbKMe2JdAM=; b=ClM4uLXrV7QyJge78CuyP+MS7wTwuZI+2cQN5vBhLzY5sAM4ZekW7J1nUg1nBunzM3 Jn6K5musBeuRHI0hZFBnweq7B50eI4fTKgXuFc2HjYijTPwjY86e30kdaiJhEoQpDjAU jKCm9dW1trER3jq9vM5qsLbvfpDGLkW8JD/TD7EX1ij4d8JKia9r19Q/GLld9I/RY/bh xh5u6l62vzTrLVVe3hVkrBHUTicWXHJ2IbQ9TmoG5qPFlkxe7RjvbnS/bsSnh32ZOvZ+ QlkldIpv0wCb335LKb1WzsJbGubuuq/xsHvZmhulAv8cdaoz2290k023u9uHOirVE3Y7 7Gkg== X-Gm-Message-State: AGi0PubsjSaoXS8FTEA0JkZ2IQuk5ouhbtBA8ngQ0h46WflSuCavm412 Zed5l4kbnBY3P9k/+iiX3pi0rqL8s7c= X-Google-Smtp-Source: APiQypJnNe0fGFgkzsndTRj/cwLxWkTXthBcoijxJe6PhMWieHGWk9VJ0/yPUhOCR3/yAjYZxTkuiQ== X-Received: by 2002:a17:90a:368d:: with SMTP id t13mr5728608pjb.175.1588723657065; Tue, 05 May 2020 17:07:37 -0700 (PDT) Received: from localhost ([8.44.146.30]) by smtp.gmail.com with ESMTPSA id g184sm124376pfb.80.2020.05.05.17.07.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 May 2020 17:07:36 -0700 (PDT) Date: Tue, 5 May 2020 18:07:35 -0600 From: Taylor Blau To: git@vger.kernel.org Cc: dstolee@microsoft.com, gitster@pobox.com, peff@peff.net, szeder.dev@gmail.com Subject: [PATCH v2 2/8] commit-graph.c: show progress of finding reachable commits Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When 'git commit-graph write --reachable' is invoked, the commit-graph machinery calls 'for_each_ref()' to discover the set of reachable commits. Right now the 'add_ref_to_set' callback is not doing anything other than adding an OID to the set of known-reachable OIDs. In a subsequent commit, 'add_ref_to_set' will presumptively peel references. This operation should be fast for repositories with an up-to-date '$GIT_DIR/packed-refs', but may be slow in the general case. So that it doesn't appear that 'git commit-graph write' is idling with '--reachable' in the slow case, add a progress meter to provide some output in the meantime. In general, we don't expect a progress meter to appear at all, since peeling references with a 'packed-refs' file is quick. If it's slow and we do show a progress meter, the subsequent 'fill_oids_from_commits()' will be fast, since all of the calls to 'lookup_commit_reference_gently()' will be no-ops. Both progress meters are delayed, so it is unlikely that more than one will appear. In either case, this intermediate state will go away in a handful of patches, at which point there will be at most one progress meter. Signed-off-by: Taylor Blau --- commit-graph.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/commit-graph.c b/commit-graph.c index 00da281f39..d0397f2a23 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -1320,6 +1320,7 @@ static void compute_bloom_filters(struct write_commit_graph_context *ctx) struct refs_cb_data { struct oidset *commits; + struct progress *progress; }; static int add_ref_to_set(const char *refname, @@ -1328,6 +1329,8 @@ static int add_ref_to_set(const char *refname, { struct refs_cb_data *data = (struct refs_cb_data *)cb_data; + display_progress(data->progress, oidset_size(data->commits) + 1); + oidset_insert(data->commits, oid); return 0; } @@ -1342,12 +1345,17 @@ int write_commit_graph_reachable(struct object_directory *odb, memset(&data, 0, sizeof(data)); data.commits = &commits; + if (flags & COMMIT_GRAPH_WRITE_PROGRESS) + data.progress = start_delayed_progress( + _("Collecting referenced commits"), 0); for_each_ref(add_ref_to_set, &data); result = write_commit_graph(odb, NULL, &commits, flags, split_opts); oidset_clear(&commits); + if (data.progress) + stop_progress(&data.progress); return result; } From patchwork Wed May 6 00:07:39 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 11530077 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 19A9481 for ; Wed, 6 May 2020 00:07:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 00C1420721 for ; Wed, 6 May 2020 00:07:46 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20150623.gappssmtp.com header.i=@ttaylorr-com.20150623.gappssmtp.com header.b="VYntB3yJ" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729448AbgEFAHp (ORCPT ); Tue, 5 May 2020 20:07:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57156 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728717AbgEFAHn (ORCPT ); Tue, 5 May 2020 20:07:43 -0400 Received: from mail-pj1-x1042.google.com (mail-pj1-x1042.google.com [IPv6:2607:f8b0:4864:20::1042]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F1E9EC061A0F for ; Tue, 5 May 2020 17:07:41 -0700 (PDT) Received: by mail-pj1-x1042.google.com with SMTP id t9so14601pjw.0 for ; Tue, 05 May 2020 17:07:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=S+j2Q8wZMBdb2cFeNRFmMRJNgzicKJJKA/eqQ+F3t3w=; b=VYntB3yJi3J3ljNSz/11uQItIs3Yuf+xOcIk3vGYiDH3hRyLQaSb3dNGuOFYfCfBB+ T1kZJ8HY7QI5f39k0zJnE7ZQ5fddaP+C1EDuxexV6Ufz+MPPWgjopjp9YZCSBtnizmB2 IMMKlMGS05cxHgZfBM09u5JSU4SnSeSt1vggsJ5S5ANnsZd0cOR6r23ml95cBK0lYgjU pmjSo8ZBJuym+P+htX9l2eBwowxoUZZkND7cgqcXc/K6tFeAjnQ+xN6V5jqA11s0wtwS FTdU60y1N+CyF8VwlzPA5fuK4B62Npj6uh/NTJNzSvdFB3YyoSL9lo7x4aDUXRzmnpr+ Axiw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=S+j2Q8wZMBdb2cFeNRFmMRJNgzicKJJKA/eqQ+F3t3w=; b=HV6woV4Z51n95LhLnDXPhNNE8n2D/X5fJeIrH+xJYHEnUjGXv72T67tTiDS51u2Rbw bngbLdPTc6IOITZoeik0lb3bRJDR++leUI8Z/zQdPSVUwMFzOu7N78zMZXp9YsBgmAGx vR3N0MI0jmqfPqXsKST7N594wmixJpr+eyq+tcHN7eoQdGkksxfkkeOfY1HNSwQKvp1Q iymYMO+KuCiGWSYpmBksTLvT2nRF0tlOb0M82oKe5fqmKsly76BiAw/sbFki2Tys8WJY 8JC1OqNb1OtZgxD7Nn+yCUFWmfz8VTSLt9WHvfuoZxeOY0r1GOGJjSBzrMKWGj6mlnF2 KvsQ== X-Gm-Message-State: AGi0PuYn5+KHrE9RGXo5z6fHZGlY+Tu5eaLMW2NydO2UwCLOcQQARQ5A O0TQ+mwALq+XDFv++wu3gnsEKeCwm8Y= X-Google-Smtp-Source: APiQypJ4W16I1OZE9wEdp7IZwN0xyGoRTsMUaYt4Mur6F7yCYhmo5foU+GDQj139JNcoGnWWCy5bmg== X-Received: by 2002:a17:902:7241:: with SMTP id c1mr5318212pll.113.1588723661079; Tue, 05 May 2020 17:07:41 -0700 (PDT) Received: from localhost ([8.44.146.30]) by smtp.gmail.com with ESMTPSA id a196sm142081pfd.184.2020.05.05.17.07.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 May 2020 17:07:40 -0700 (PDT) Date: Tue, 5 May 2020 18:07:39 -0600 From: Taylor Blau To: git@vger.kernel.org Cc: dstolee@microsoft.com, gitster@pobox.com, peff@peff.net, szeder.dev@gmail.com Subject: [PATCH v2 3/8] commit-graph.c: peel refs in 'add_ref_to_set' Message-ID: <85c388a077ed9b43ad4876e6b3efaa0358c78715.1588723544.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org While iterating references (to discover the set of commits to write to the commit-graph with 'git commit-graph write --reachable'), 'add_ref_to_set' can save 'fill_oids_from_commits()' some time by peeling the references beforehand. Move peeling out of 'fill_oids_from_commits()' and into 'add_ref_to_set()' to use 'peel_ref()' instead of 'deref_tag()'. Doing so allows the commit-graph machinery to use the peeled value from '$GIT_DIR/packed-refs' instead of having to load and parse tags. While we're at it, discard non-commit objects reachable from ref tips. This would be done automatically by 'fill_oids_from_commits()', but such functionality will be removed in a subsequent patch after the call to 'lookup_commit_reference_gently' is dropped (at which point a non-commit object in the commits oidset will become an error). Suggested-by: Jeff King Signed-off-by: Taylor Blau --- commit-graph.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/commit-graph.c b/commit-graph.c index d0397f2a23..2c3b5fd09d 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -1327,11 +1327,15 @@ static int add_ref_to_set(const char *refname, const struct object_id *oid, int flags, void *cb_data) { + struct object_id peeled; struct refs_cb_data *data = (struct refs_cb_data *)cb_data; display_progress(data->progress, oidset_size(data->commits) + 1); - oidset_insert(data->commits, oid); + if (peel_ref(refname, &peeled)) + peeled = *oid; + if (oid_object_info(the_repository, &peeled, NULL) == OBJ_COMMIT) + oidset_insert(data->commits, &peeled); return 0; } From patchwork Wed May 6 00:07:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 11530079 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6CB1481 for ; Wed, 6 May 2020 00:07:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5378720721 for ; Wed, 6 May 2020 00:07:48 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20150623.gappssmtp.com header.i=@ttaylorr-com.20150623.gappssmtp.com header.b="MA0crFjU" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729464AbgEFAHr (ORCPT ); Tue, 5 May 2020 20:07:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57168 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728717AbgEFAHr (ORCPT ); Tue, 5 May 2020 20:07:47 -0400 Received: from mail-pf1-x444.google.com (mail-pf1-x444.google.com [IPv6:2607:f8b0:4864:20::444]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D7B39C061A0F for ; Tue, 5 May 2020 17:07:45 -0700 (PDT) Received: by mail-pf1-x444.google.com with SMTP id d184so105486pfd.4 for ; Tue, 05 May 2020 17:07:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=vHRIiWx89t6b6i0iG2vGJSmwe30s4IFWz7nTGrcVFZU=; b=MA0crFjUk7t25+qTAjQCBQ1FG5BMpM3wbuOsf0EFWZOIeBUTxjkl4t0SHnQhya1aq4 rr8OPN1zlZp8XU8KHvOlHAmiJtMF5brWR6loWNxf7KU6ma4YuS3O7Y23x3YgGe+LHCXz qANz9Rd3IDpGEJEK3h4er9YNAJYYsEmbdV4y/Zb4Ha8RBV0Yv+9CIfmNeVB1uiJH4hVk F4sMi6+D0J4E6IK4292khKNatq2G/9LM7kdJLsCiTZESnbBQt9R67921jyaEMtGCNWi1 DOoV2ZgueI/mw5Stxspio7WIuDrGzHhsLP3ypbNwk6k1OrSoawXiMGK4X3kmCqnTJvUz xy0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=vHRIiWx89t6b6i0iG2vGJSmwe30s4IFWz7nTGrcVFZU=; b=i5e00XaD+yIeuw0B878h7spOtPxMmHL+Lp/IiHl8hqdXcOkVDK3nqFz2Xj4OZPS1eS C32AUFAYcedzb0Xhb/nNqsVmfPZf3MWJuPJb1U23aHZp3gwzCG32KN1u/nJ3kybT0ZSD 87GO6bYwMPzx47pTC6v1+kAPts+/7gf6vjpS5ciu2S+YuZJXeh6HDYyUoa1+hbPHMaZb DBbx3sZ+35iU+jRIXN5WMdvRHgqWLtoFaKp4BeNwgfplxGYUgEhlXcnaM4BA8nnEgWwD yJah8QmMuRpjpBRHU/9qQwKXqrs/2FRsFdLhvKC6g4kJA7NKhi8xz1qMYE4KVOCALIhd PC+w== X-Gm-Message-State: AGi0PuYDj86E6hS8/ZRclNJQ4se2TndPaXUulobnFDKva5VfqiRSU7sq FzE+SyjWXT2xqZN8IKkAWLFhSds5xrs= X-Google-Smtp-Source: APiQypJpI7Gj8QdN0vnitLhd0qC7XucN+/XQgTddsck3+H3/t64nsAcU6Ajami3vVaohCWDymwKC4A== X-Received: by 2002:a65:62c7:: with SMTP id m7mr4723527pgv.16.1588723665014; Tue, 05 May 2020 17:07:45 -0700 (PDT) Received: from localhost ([8.44.146.30]) by smtp.gmail.com with ESMTPSA id i8sm133712pfq.126.2020.05.05.17.07.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 May 2020 17:07:44 -0700 (PDT) Date: Tue, 5 May 2020 18:07:43 -0600 From: Taylor Blau To: git@vger.kernel.org Cc: dstolee@microsoft.com, gitster@pobox.com, peff@peff.net, szeder.dev@gmail.com Subject: [PATCH v2 4/8] builtin/commit-graph.c: extract 'read_one_commit()' Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org With either '--stdin-commits' or '--stdin-packs', the commit-graph builtin will read line-delimited input, and interpret it either as a series of commit OIDs, or pack names. In a subsequent commit, we will begin handling '--stdin-commits' differently by processing each line as it comes in, instead of in one shot at the end. To make adequate room for this additional logic, split the '--stdin-commits' case from '--stdin-packs' by only storing the input when '--stdin-packs' is given. In the case of '--stdin-commits', feed each line to a new 'read_one_commit' helper, which (for now) will merely call 'parse_oid_hex'. Signed-off-by: Taylor Blau --- builtin/commit-graph.c | 54 +++++++++++++++++++++++------------------- 1 file changed, 29 insertions(+), 25 deletions(-) diff --git a/builtin/commit-graph.c b/builtin/commit-graph.c index 15fe60317c..f550d8489a 100644 --- a/builtin/commit-graph.c +++ b/builtin/commit-graph.c @@ -138,12 +138,25 @@ static int write_option_parse_split(const struct option *opt, const char *arg, return 0; } +static int read_one_commit(struct oidset *commits, char *hash) +{ + struct object_id oid; + const char *end; + + if (parse_oid_hex(hash, &oid, &end)) { + error(_("unexpected non-hex object ID: %s"), hash); + return 1; + } + + oidset_insert(commits, &oid); + return 0; +} + static int graph_write(int argc, const char **argv) { - struct string_list *pack_indexes = NULL; + struct string_list pack_indexes; struct oidset commits = OIDSET_INIT; struct object_directory *odb = NULL; - struct string_list lines; int result = 0; enum commit_graph_write_flags flags = 0; @@ -209,44 +222,35 @@ static int graph_write(int argc, const char **argv) return 0; } - string_list_init(&lines, 0); + string_list_init(&pack_indexes, 0); if (opts.stdin_packs || opts.stdin_commits) { struct strbuf buf = STRBUF_INIT; - - while (strbuf_getline(&buf, stdin) != EOF) - string_list_append(&lines, strbuf_detach(&buf, NULL)); - - if (opts.stdin_packs) - pack_indexes = &lines; if (opts.stdin_commits) { - struct string_list_item *item; - oidset_init(&commits, lines.nr); - for_each_string_list_item(item, &lines) { - struct object_id oid; - const char *end; - - if (parse_oid_hex(item->string, &oid, &end)) { - error(_("unexpected non-hex object ID: " - "%s"), item->string); - return 1; - } - - oidset_insert(&commits, &oid); - } + oidset_init(&commits, 0); flags |= COMMIT_GRAPH_WRITE_CHECK_OIDS; } + while (strbuf_getline(&buf, stdin) != EOF) { + char *line = strbuf_detach(&buf, NULL); + if (opts.stdin_commits) { + int result = read_one_commit(&commits, line); + if (result) + return result; + } else + string_list_append(&pack_indexes, line); + } + UNLEAK(buf); } if (write_commit_graph(odb, - pack_indexes, + opts.stdin_packs ? &pack_indexes : NULL, opts.stdin_commits ? &commits : NULL, flags, &split_opts)) result = 1; - UNLEAK(lines); + UNLEAK(pack_indexes); return result; } From patchwork Wed May 6 00:07:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 11530081 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0988F81 for ; Wed, 6 May 2020 00:07:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E697920721 for ; Wed, 6 May 2020 00:07:52 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20150623.gappssmtp.com header.i=@ttaylorr-com.20150623.gappssmtp.com header.b="nH6jEIeY" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729505AbgEFAHw (ORCPT ); Tue, 5 May 2020 20:07:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57190 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728717AbgEFAHv (ORCPT ); Tue, 5 May 2020 20:07:51 -0400 Received: from mail-pj1-x1042.google.com (mail-pj1-x1042.google.com [IPv6:2607:f8b0:4864:20::1042]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5E303C061A0F for ; Tue, 5 May 2020 17:07:50 -0700 (PDT) Received: by mail-pj1-x1042.google.com with SMTP id a5so65942pjh.2 for ; Tue, 05 May 2020 17:07:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=RgpVrKtDEr5YGNcOHEq41/OIKzHF7MuIh5xDh9Eusw0=; b=nH6jEIeYf4yEnmJJpL9NB9wA1r76zGd69yy5BjApThJzxiCUI2S1iPeMtICulQXpjr VKCrrcPbRtyi3QRukh2m0o9B1tRqcwjax4WQS34uHdaHxGaPinQR+l/Hi2eY8Svvdn37 FZzxvFyegQErjap18cbBadHe2fA4mlt8Wb9lBv1Jbpo4F5HuDbag7MhIBBObz2yG05AN T5lEO1aVSyD2RpqRBpFxH5gfFCa1boLwEgTnnRxXuUHLCE2Iqb0ZG/aI3rHIIVXUkEO2 DMFZbGWeaT0VQhBz3nWhCm5BAR5PMtF/fv2dIJtUBJpaXlhh7ZlWZo8OgdR6lGS21BO7 7Ppg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=RgpVrKtDEr5YGNcOHEq41/OIKzHF7MuIh5xDh9Eusw0=; b=gGDLLmiaiQdII4fwWV+x7D+fbfGotnwWFfLYer7HOsG0VPVtHcTmX3XDT/IZLOdTRV Pcr+drTuD6ivUJizBmLKZ4GSW2n1d73lMgTH5H8S3aVC8xoP7PZKQlSBhks5CGf97+0P xgTRn6Vp263nY+MAzu6QqiJOO5CT8rwpGDP1Hr14Bn1t6TotTSdGDJqFWNvoUg8sxTqw 5EaX6IQKPD0CjW2nF7BYDa6zLDLunXVNQIbTbdXwPx3ADNCpoK+UBRd2zePOMaB66fmC mXi9z6TO4IajApTf4SJo0dB231hqWBy4N3mroOYfIRWuGBKnqHLiIk9GbRZNYWQ4sy/j GFdg== X-Gm-Message-State: AGi0PuZdri2J2GIgTZt4Nvwwy9zY0zdl1BHMJ02qiwYb4k0Vu9CVR7xW iAsQOBrQmm1LFAeKHz2G/1fahjfya1k= X-Google-Smtp-Source: APiQypL23PvnkAl/Kg6+Yr94oiTaXqqgYfAGOfQoQigR35Mlkva+runVn8H+6NU793vSxVk83xEtWw== X-Received: by 2002:a17:902:b40b:: with SMTP id x11mr5549306plr.174.1588723669476; Tue, 05 May 2020 17:07:49 -0700 (PDT) Received: from localhost ([8.44.146.30]) by smtp.gmail.com with ESMTPSA id gz14sm3029122pjb.42.2020.05.05.17.07.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 May 2020 17:07:48 -0700 (PDT) Date: Tue, 5 May 2020 18:07:47 -0600 From: Taylor Blau To: git@vger.kernel.org Cc: dstolee@microsoft.com, gitster@pobox.com, peff@peff.net, szeder.dev@gmail.com Subject: [PATCH v2 5/8] builtin/commit-graph.c: dereference tags in builtin Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When given a list of commits, the commit-graph machinery calls 'lookup_commit_reference_gently()' on each element in the set and treats the resulting set of OIDs as the base over which to close for reachability. In an earlier collection of commits, the 'git commit-graph write --reachable' case made the inner-most call to 'lookup_commit_reference_gently()' by peeling references before they were passed over to the commit-graph internals. Do the analog for 'git commit-graph write --stdin-commits' by calling 'lookup_commit_reference_gently()' outside of the commit-graph machinery, making the inner-most call a noop. Since this may incur additional processing time, surround 'read_one_commit' with a progress meter to provide output to the caller. Signed-off-by: Taylor Blau --- builtin/commit-graph.c | 25 ++++++++++++++++++++++--- 1 file changed, 22 insertions(+), 3 deletions(-) diff --git a/builtin/commit-graph.c b/builtin/commit-graph.c index f550d8489a..6537e9acef 100644 --- a/builtin/commit-graph.c +++ b/builtin/commit-graph.c @@ -6,6 +6,7 @@ #include "repository.h" #include "commit-graph.h" #include "object-store.h" +#include "progress.h" static char const * const builtin_commit_graph_usage[] = { N_("git commit-graph verify [--object-dir ] [--shallow] [--[no-]progress]"), @@ -138,8 +139,10 @@ static int write_option_parse_split(const struct option *opt, const char *arg, return 0; } -static int read_one_commit(struct oidset *commits, char *hash) +static int read_one_commit(struct oidset *commits, struct progress *progress, + char *hash) { + struct commit *result; struct object_id oid; const char *end; @@ -148,7 +151,15 @@ static int read_one_commit(struct oidset *commits, char *hash) return 1; } - oidset_insert(commits, &oid); + display_progress(progress, oidset_size(commits) + 1); + + result = lookup_commit_reference_gently(the_repository, &oid, 1); + if (result) + oidset_insert(commits, &result->object.oid); + else { + error(_("invalid commit object id: %s"), hash); + return 1; + } return 0; } @@ -159,6 +170,7 @@ static int graph_write(int argc, const char **argv) struct object_directory *odb = NULL; int result = 0; enum commit_graph_write_flags flags = 0; + struct progress *progress = NULL; static struct option builtin_commit_graph_write_options[] = { OPT_STRING(0, "object-dir", &opts.obj_dir, @@ -228,18 +240,25 @@ static int graph_write(int argc, const char **argv) if (opts.stdin_commits) { oidset_init(&commits, 0); flags |= COMMIT_GRAPH_WRITE_CHECK_OIDS; + if (opts.progress) + progress = start_delayed_progress( + _("Collecting commits from input"), 0); } while (strbuf_getline(&buf, stdin) != EOF) { char *line = strbuf_detach(&buf, NULL); if (opts.stdin_commits) { - int result = read_one_commit(&commits, line); + int result = read_one_commit(&commits, progress, + line); if (result) return result; } else string_list_append(&pack_indexes, line); } + if (progress) + stop_progress(&progress); + UNLEAK(buf); } From patchwork Wed May 6 00:07:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 11530083 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7F18E1392 for ; Wed, 6 May 2020 00:07:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6694520721 for ; Wed, 6 May 2020 00:07:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20150623.gappssmtp.com header.i=@ttaylorr-com.20150623.gappssmtp.com header.b="MDfg/nzF" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729516AbgEFAHy (ORCPT ); Tue, 5 May 2020 20:07:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57198 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728717AbgEFAHy (ORCPT ); Tue, 5 May 2020 20:07:54 -0400 Received: from mail-pl1-x642.google.com (mail-pl1-x642.google.com [IPv6:2607:f8b0:4864:20::642]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 04F5CC061A0F for ; Tue, 5 May 2020 17:07:54 -0700 (PDT) Received: by mail-pl1-x642.google.com with SMTP id x10so247059plr.4 for ; Tue, 05 May 2020 17:07:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=ZLtBzuI9rDePR+08aciWXpR6Rg+IfC1H1o29cmmEwO8=; b=MDfg/nzFLJ15TuFiOpAuxYlYzN3S4eN2pUMcRBjWSC4RCMx15F8vxJAcfiYgBHfo7Z 44XJLnD4Sj+dHsTzQEA+P64MD4++9SZ5GxzQPuBLsqNt9A6egWMqUXxNQaGpdIlConfj voJqdguEB+9F3FkgxlfpbTZhLHZLrwJ7LDRuJhTW4XrC9bH1KZVbsKc8uf/c+XKkUYmz mLa8flopGyizYykcgYCT/W3N8mL9NHSrOyOkdg1Rf9vQl4Qu/C1jL/ZQ9/bMaXpIIo62 T4HH/oAKqmXCaSMk8NUTHg+wdQFMtjySS+fUekcejEZ7p5CFzQmsXsqbviL3W2zvDjns iIyA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=ZLtBzuI9rDePR+08aciWXpR6Rg+IfC1H1o29cmmEwO8=; b=QFW5o04tlTy84tsUhAGhp9pmglcs/kLTrBArsvCeJaNoM1g/0aglNXlxJMlNghbkiP gVhIUrx4pVpKd2tx7zN3SaotQoW9XLY1sFREktdsCPJlyncTJ4xve6D+1NmuhFtzLb2G bhrIwU3wKM8UjzAYBkMSZcl09F13D5QujX9lwhQIrmsxO1FiovMu0uK+BDWr5zGk8s45 gt9wD6LKGTh0yItUuOYUEm/PRjD8qtpd9Q944VBDItx5PmvOz20Pzxsby5T1QOyvnaqy LD9icF9H0p5Qg+7BnMz/FaXiEaYpKlR3cX0yRv7XzQLbhzY9m6hRnfFU+0DuNnKaBUl3 Adbg== X-Gm-Message-State: AGi0PuZzKJ87acruTBk6yBeaOTXKoHb80rL148krXt7sqyRESYCi/V6Z hp0dND6yuKbKWKiH520thoEUKDrcHbo= X-Google-Smtp-Source: APiQypLsEzgo4cvCp1eo4PWE3GKeLvsy2Mdb182nwXtfxWPzzXnSVsxUZfUIsbMXaCSrDWSsKfPlDQ== X-Received: by 2002:a17:90a:5584:: with SMTP id c4mr6102428pji.51.1588723673161; Tue, 05 May 2020 17:07:53 -0700 (PDT) Received: from localhost ([8.44.146.30]) by smtp.gmail.com with ESMTPSA id 28sm3064188pjh.43.2020.05.05.17.07.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 May 2020 17:07:52 -0700 (PDT) Date: Tue, 5 May 2020 18:07:51 -0600 From: Taylor Blau To: git@vger.kernel.org Cc: dstolee@microsoft.com, gitster@pobox.com, peff@peff.net, szeder.dev@gmail.com Subject: [PATCH v2 6/8] commit-graph.c: simplify 'fill_oids_from_commits' Message-ID: <61887870c722d0934b02d7e2a94972a147dc4d3b.1588723544.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In the previous handful of commits, both 'git commit-graph write --reachable' and '--stdin-commits' learned to peel tags down to the commits which they refer to before passing them into the commit-graph internals. This makes the call to 'lookup_commit_reference_gently()' inside of 'fill_oids_from_commits()' a noop, since all OIDs are commits by that point. As such, remove the call entirely, as well as the progress meter, which has been split and moved out to the callers in the aforementioned earlier commits. Signed-off-by: Taylor Blau --- commit-graph.c | 33 +++------------------------------ 1 file changed, 3 insertions(+), 30 deletions(-) diff --git a/commit-graph.c b/commit-graph.c index 2c3b5fd09d..56a4a43b30 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -1411,46 +1411,19 @@ static int fill_oids_from_packs(struct write_commit_graph_context *ctx, static int fill_oids_from_commits(struct write_commit_graph_context *ctx, struct oidset *commits) { - uint32_t i = 0; - struct strbuf progress_title = STRBUF_INIT; struct oidset_iter iter; struct object_id *oid; if (!oidset_size(commits)) return 0; - if (ctx->report_progress) { - strbuf_addf(&progress_title, - Q_("Finding commits for commit graph from %d ref", - "Finding commits for commit graph from %d refs", - oidset_size(commits)), - oidset_size(commits)); - ctx->progress = start_delayed_progress( - progress_title.buf, - oidset_size(commits)); - } - oidset_iter_init(commits, &iter); while ((oid = oidset_iter_next(&iter))) { - struct commit *result; - - display_progress(ctx->progress, ++i); - - result = lookup_commit_reference_gently(ctx->r, oid, 1); - if (result) { - ALLOC_GROW(ctx->oids.list, ctx->oids.nr + 1, ctx->oids.alloc); - oidcpy(&ctx->oids.list[ctx->oids.nr], &(result->object.oid)); - ctx->oids.nr++; - } else if (ctx->check_oids) { - error(_("invalid commit object id: %s"), - oid_to_hex(oid)); - return -1; - } + ALLOC_GROW(ctx->oids.list, ctx->oids.nr + 1, ctx->oids.alloc); + oidcpy(&ctx->oids.list[ctx->oids.nr], oid); + ctx->oids.nr++; } - stop_progress(&ctx->progress); - strbuf_release(&progress_title); - return 0; } From patchwork Wed May 6 00:07:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 11530085 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 606861392 for ; Wed, 6 May 2020 00:07:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 465C520721 for ; Wed, 6 May 2020 00:07:59 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20150623.gappssmtp.com header.i=@ttaylorr-com.20150623.gappssmtp.com header.b="iFC4IeAV" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729549AbgEFAH6 (ORCPT ); Tue, 5 May 2020 20:07:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57212 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729539AbgEFAH5 (ORCPT ); Tue, 5 May 2020 20:07:57 -0400 Received: from mail-pj1-x1042.google.com (mail-pj1-x1042.google.com [IPv6:2607:f8b0:4864:20::1042]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 88E75C061A10 for ; Tue, 5 May 2020 17:07:57 -0700 (PDT) Received: by mail-pj1-x1042.google.com with SMTP id y6so70333pjc.4 for ; Tue, 05 May 2020 17:07:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=iHhSGMGzCi6ysuU18iYcv3qCsk++2Ddth7iq5fAYkqc=; b=iFC4IeAVco4OPh7JDyQeWBAZ0EHZlHzJfYHeOaVQIHaqSsUlfVOyc2efllKnUm6nd2 XBdDuL8ZrfMDFSF4KKEjHrqIL5DaiXkmkqmrGZaL5H6ZUbC+++dYL+VxXZ4DIo6jC8ew f27hCwIsSk9Q8q6LM6dGeWp03ojXOepnykYDszm1sPwqhBqS4VvSjVcDeHqiVQ2WcGx4 Au8t5zG75aZbzcxy+BY7DK6Hyqj90DHl9X5SU83LyILMSz0/d4aAnIfT7Bv1J8TGWksF gpbCjSzCn02kBy/Kwv0r+7ZPcsxynXO7x7pJDndlawoBsKYJfNlgHOMwr2AqU9k8Oc1C tKgQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=iHhSGMGzCi6ysuU18iYcv3qCsk++2Ddth7iq5fAYkqc=; b=HmUnUbZjOFEg+x/MKnqxG2xAB7Lo+Q3U51d0B9ZdfXe+ADyuekR4FVg0tb4Lf90nFc 9lkGStixCJNW6cmfyj7wNEFH0HSedNQhKj0h2tkjIoaWm0NjsaRLAqHJ4+IXbyAjJ/lg lDpgTf3EMnZfXvNzlzuj8x4BUifu0xb76uR7Kp5L7cBHy1CFsGbQBiT5ljntma56ka/n elodEPXHkni7FM0ZTdBkukFFl1bHJ99/x6qMF0EvW+l7Klz+NsW/xDHudrTSHffnqZpk 94bQ0SCF4oOjlk2qOyzYK05GHcvJxCUDQf8ZoLZZvlpcBlKf/NpZ+pJpLuMRKQGeXMf0 4fxg== X-Gm-Message-State: AGi0PuZyXmSvznAGeec0Vl7UnWZdj4u4Rk3T/WisdnzIwG3HuUUWSXTi YdtfW2stPaZ0XZs2fca4lJFYEiNuJGY= X-Google-Smtp-Source: APiQypJs661ROWPLBEjvnCtBAOzvCjjMtArMSZfUjGQEMdjAYpXYa4pBUi6IxMp6cPBULDJlrfJ0og== X-Received: by 2002:a17:902:421:: with SMTP id 30mr5059950ple.271.1588723676662; Tue, 05 May 2020 17:07:56 -0700 (PDT) Received: from localhost ([8.44.146.30]) by smtp.gmail.com with ESMTPSA id o21sm2900251pjr.37.2020.05.05.17.07.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 May 2020 17:07:56 -0700 (PDT) Date: Tue, 5 May 2020 18:07:54 -0600 From: Taylor Blau To: git@vger.kernel.org Cc: dstolee@microsoft.com, gitster@pobox.com, peff@peff.net, szeder.dev@gmail.com Subject: [PATCH v2 7/8] t5318: reorder test below 'graph_read_expect' Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In the subsequent commit, we will introduce a dependency on 'graph_read_expect' from t5318.7. Preemptively move it below 'graph_read_expect()'s definition so that the test can call it. Signed-off-by: Taylor Blau --- t/t5318-commit-graph.sh | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/t/t5318-commit-graph.sh b/t/t5318-commit-graph.sh index 39e2918a32..89020d3d44 100755 --- a/t/t5318-commit-graph.sh +++ b/t/t5318-commit-graph.sh @@ -42,15 +42,6 @@ test_expect_success 'create commits and repack' ' git repack ' -test_expect_success 'exit with correct error on bad input to --stdin-commits' ' - cd "$TRASH_DIRECTORY/full" && - echo HEAD | test_expect_code 1 git commit-graph write --stdin-commits 2>stderr && - test_i18ngrep "unexpected non-hex object ID: HEAD" stderr && - # valid tree OID, but not a commit OID - git rev-parse HEAD^{tree} | test_expect_code 1 git commit-graph write --stdin-commits 2>stderr && - test_i18ngrep "invalid commit object id" stderr -' - graph_git_two_modes() { git -c core.commitGraph=true $1 >output git -c core.commitGraph=false $1 >expect @@ -91,6 +82,15 @@ graph_read_expect() { test_cmp expect output } +test_expect_success 'exit with correct error on bad input to --stdin-commits' ' + cd "$TRASH_DIRECTORY/full" && + echo HEAD | test_expect_code 1 git commit-graph write --stdin-commits 2>stderr && + test_i18ngrep "unexpected non-hex object ID: HEAD" stderr && + # valid tree OID, but not a commit OID + git rev-parse HEAD^{tree} | test_expect_code 1 git commit-graph write --stdin-commits 2>stderr && + test_i18ngrep "invalid commit object id" stderr +' + test_expect_success 'write graph' ' cd "$TRASH_DIRECTORY/full" && git commit-graph write && From patchwork Wed May 6 00:07:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 11530087 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 08FDC81 for ; Wed, 6 May 2020 00:08:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E0E7E20721 for ; Wed, 6 May 2020 00:08:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20150623.gappssmtp.com header.i=@ttaylorr-com.20150623.gappssmtp.com header.b="F59IBE8K" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729617AbgEFAID (ORCPT ); Tue, 5 May 2020 20:08:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57226 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729603AbgEFAIC (ORCPT ); Tue, 5 May 2020 20:08:02 -0400 Received: from mail-pf1-x442.google.com (mail-pf1-x442.google.com [IPv6:2607:f8b0:4864:20::442]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 761D2C061A0F for ; Tue, 5 May 2020 17:08:01 -0700 (PDT) Received: by mail-pf1-x442.google.com with SMTP id p25so88507pfn.11 for ; Tue, 05 May 2020 17:08:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=QTfW4oWalXFbQpuks+wOsWzh2o2ZyNdcHhc5TjQ6BVs=; b=F59IBE8KFWd76/Qs5F3aSNEm1GLt2CbJ7Ddb4b9H/Y9IN/0Gx7mYyNrdFMGRJXARAW UExwrVFMCfIgeiXVdZGsCfP83xvRlJivz2OkHqZlyWImxoP3SB/tjOrwy4+EPmJ20mcZ T5UjNHTqvKs6/qDdt+INrSztjmCjW7jf/LXxQcHVezs6PcMLrtsllLhG8ryKfBh0q3sx VxXZ4DinDub1xIa/8eubP5rusJ8BVDW/8vYLnPRuhjSL8mmAqDBWCePoxZ9GlXKoclIW 4gVKiBATOeqgGhhzRmqM09vxYUcT+Cv+ECbzxSeT4xhxJrly61aqcFyVOaFHv+fQPTsR y6yw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=QTfW4oWalXFbQpuks+wOsWzh2o2ZyNdcHhc5TjQ6BVs=; b=D1hSvffa+gtU81HxcAg6j40fpzwgfBNGcm2yyLuqVA19fO1E+01odMB+OaxONhp1it nD1hIAhkJoNRUGlNTYHNilGMBeZljomXkF3MHStIcqfby9KmkMv+2z3FSzszYip5jgmv FC53yT+dGjktTEXljlB9bfJ01hZvcdwMcM83zXK2MVZVI/Rx5zj28U/v8OVxCRdYXfT/ qPi11q3Wrlae7h9VvOgrJ1y+vHYp9SrdINzRxZ4xo23PFacKBG0Z47jUOAdd4c/Otu0V BVcwR/6rRHk35CtuPNLU4ykIIvEcg1/lf6tjDSHG5C3NnkIrTjtuw12O9hRV3w0jBx0P rfyw== X-Gm-Message-State: AGi0PubVMsfWOwxjKcLixz9egvyL2nnk5FQbUQ3r+JCqOolofygWU0qQ st6pkg5RrvZz/eRwMNLYVtjF1K61Urc= X-Google-Smtp-Source: APiQypKXZ3Jyk6i3AIHnZ9lMVNpdmJcV1bgbKCakLOlqLdpcR/6K9zj3uMC0KWMmfPz/eZ9/FV/eHA== X-Received: by 2002:a63:7ce:: with SMTP id 197mr5058453pgh.113.1588723680400; Tue, 05 May 2020 17:08:00 -0700 (PDT) Received: from localhost ([8.44.146.30]) by smtp.gmail.com with ESMTPSA id u5sm138052pfu.198.2020.05.05.17.07.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 May 2020 17:07:59 -0700 (PDT) Date: Tue, 5 May 2020 18:07:58 -0600 From: Taylor Blau To: git@vger.kernel.org Cc: dstolee@microsoft.com, gitster@pobox.com, peff@peff.net, szeder.dev@gmail.com Subject: [PATCH v2 8/8] commit-graph: drop COMMIT_GRAPH_WRITE_CHECK_OIDS flag Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Since 7c5c9b9c57 (commit-graph: error out on invalid commit oids in 'write --stdin-commits', 2019-08-05), the commit-graph builtin dies on receiving non-commit OIDs as input to '--stdin-commits'. This behavior can be cumbersome to work around in, say, the case of piping 'git for-each-ref' to 'git commit-graph write --stdin-commits' if the caller does not want to cull out non-commits themselves. In this situation, it would be ideal if 'git commit-graph write' wrote the graph containing the inputs that did pertain to commits, and silently ignored the remainder of the input. Some options have been proposed to the effect of '--[no-]check-oids' which would allow callers to have the commit-graph builtin do just that. After some discussion, it is difficult to imagine a caller who wouldn't want to pass '--no-check-oids', suggesting that we should get rid of the behavior of complaining about non-commit inputs altogether. If callers do wish to retain this behavior, they can easily work around this change by doing the following: git for-each-ref --format='%(objectname) %(objecttype) %(*objecttype)' | awk '/commit/ { print $1 }' | git commit-graph write --stdin-commits To make it so that valid OIDs that refer to non-existent objects are indeed an error after loosening the error handling, perform an extra lookup to make sure that object indeed exists before sending it to the commit-graph internals. Signed-off-by: Taylor Blau --- Documentation/git-commit-graph.txt | 6 ++++-- builtin/commit-graph.c | 10 +++++----- commit-graph.c | 2 -- commit-graph.h | 10 ++++------ t/t5318-commit-graph.sh | 15 +++++++++++---- 5 files changed, 24 insertions(+), 19 deletions(-) diff --git a/Documentation/git-commit-graph.txt b/Documentation/git-commit-graph.txt index 53a650225a..fcac7d12e1 100644 --- a/Documentation/git-commit-graph.txt +++ b/Documentation/git-commit-graph.txt @@ -47,8 +47,10 @@ with `--stdin-commits` or `--reachable`.) + With the `--stdin-commits` option, generate the new commit graph by walking commits starting at the commits specified in stdin as a list -of OIDs in hex, one OID per line. (Cannot be combined with -`--stdin-packs` or `--reachable`.) +of OIDs in hex, one OID per line. OIDs that resolve to non-commits +(either directly, or by peeling tags) are silently ignored. OIDs that +are malformed, or do not exist generate an error. (Cannot be combined +with `--stdin-packs` or `--reachable`.) + With the `--reachable` option, generate the new commit graph by walking commits starting at all refs. (Cannot be combined with `--stdin-commits` diff --git a/builtin/commit-graph.c b/builtin/commit-graph.c index 6537e9acef..a4685a1553 100644 --- a/builtin/commit-graph.c +++ b/builtin/commit-graph.c @@ -153,13 +153,14 @@ static int read_one_commit(struct oidset *commits, struct progress *progress, display_progress(progress, oidset_size(commits) + 1); + if (oid_object_info(the_repository, &oid, NULL) < 0) { + error(_("object %s does not exist"), hash); + return 1; + } + result = lookup_commit_reference_gently(the_repository, &oid, 1); if (result) oidset_insert(commits, &result->object.oid); - else { - error(_("invalid commit object id: %s"), hash); - return 1; - } return 0; } @@ -239,7 +240,6 @@ static int graph_write(int argc, const char **argv) struct strbuf buf = STRBUF_INIT; if (opts.stdin_commits) { oidset_init(&commits, 0); - flags |= COMMIT_GRAPH_WRITE_CHECK_OIDS; if (opts.progress) progress = start_delayed_progress( _("Collecting commits from input"), 0); diff --git a/commit-graph.c b/commit-graph.c index 56a4a43b30..3c3ada29b1 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -880,7 +880,6 @@ struct write_commit_graph_context { unsigned append:1, report_progress:1, split:1, - check_oids:1, changed_paths:1, order_by_pack:1; @@ -2001,7 +2000,6 @@ int write_commit_graph(struct object_directory *odb, ctx->append = flags & COMMIT_GRAPH_WRITE_APPEND ? 1 : 0; ctx->report_progress = flags & COMMIT_GRAPH_WRITE_PROGRESS ? 1 : 0; ctx->split = flags & COMMIT_GRAPH_WRITE_SPLIT ? 1 : 0; - ctx->check_oids = flags & COMMIT_GRAPH_WRITE_CHECK_OIDS ? 1 : 0; ctx->split_opts = split_opts; ctx->changed_paths = flags & COMMIT_GRAPH_WRITE_BLOOM_FILTERS ? 1 : 0; ctx->total_bloom_filter_data_size = 0; diff --git a/commit-graph.h b/commit-graph.h index 4212766a4f..659393e07f 100644 --- a/commit-graph.h +++ b/commit-graph.h @@ -88,12 +88,10 @@ struct commit_graph *parse_commit_graph(void *graph_map, size_t graph_size); int generation_numbers_enabled(struct repository *r); enum commit_graph_write_flags { - COMMIT_GRAPH_WRITE_APPEND = (1 << 0), - COMMIT_GRAPH_WRITE_PROGRESS = (1 << 1), - COMMIT_GRAPH_WRITE_SPLIT = (1 << 2), - /* Make sure that each OID in the input is a valid commit OID. */ - COMMIT_GRAPH_WRITE_CHECK_OIDS = (1 << 3), - COMMIT_GRAPH_WRITE_BLOOM_FILTERS = (1 << 4), + COMMIT_GRAPH_WRITE_APPEND = (1 << 0), + COMMIT_GRAPH_WRITE_PROGRESS = (1 << 1), + COMMIT_GRAPH_WRITE_SPLIT = (1 << 2), + COMMIT_GRAPH_WRITE_BLOOM_FILTERS = (1 << 3) }; enum commit_graph_split_flags { diff --git a/t/t5318-commit-graph.sh b/t/t5318-commit-graph.sh index 89020d3d44..74f93f0a17 100755 --- a/t/t5318-commit-graph.sh +++ b/t/t5318-commit-graph.sh @@ -84,11 +84,18 @@ graph_read_expect() { test_expect_success 'exit with correct error on bad input to --stdin-commits' ' cd "$TRASH_DIRECTORY/full" && - echo HEAD | test_expect_code 1 git commit-graph write --stdin-commits 2>stderr && + # invalid, non-hex OID + echo HEAD >in && + test_expect_code 1 git commit-graph write --stdin-commits stderr && test_i18ngrep "unexpected non-hex object ID: HEAD" stderr && - # valid tree OID, but not a commit OID - git rev-parse HEAD^{tree} | test_expect_code 1 git commit-graph write --stdin-commits 2>stderr && - test_i18ngrep "invalid commit object id" stderr + # non-existent OID + echo $ZERO_OID >in && + test_expect_code 1 git commit-graph write --stdin-commits stderr && + test_i18ngrep "does not exist" stderr && + # valid commit and tree OID + git rev-parse HEAD HEAD^{tree} >in && + git commit-graph write --stdin-commits