From patchwork Tue May 5 01:13:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 11527973 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 13A461862 for ; Tue, 5 May 2020 01:13:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id F10E220658 for ; Tue, 5 May 2020 01:13:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20150623.gappssmtp.com header.i=@ttaylorr-com.20150623.gappssmtp.com header.b="Vh8f6X2M" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727892AbgEEBNi (ORCPT ); Mon, 4 May 2020 21:13:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40018 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1727820AbgEEBNi (ORCPT ); Mon, 4 May 2020 21:13:38 -0400 Received: from mail-pj1-x1044.google.com (mail-pj1-x1044.google.com [IPv6:2607:f8b0:4864:20::1044]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 385A1C061A0F for ; Mon, 4 May 2020 18:13:38 -0700 (PDT) Received: by mail-pj1-x1044.google.com with SMTP id a31so277275pje.1 for ; Mon, 04 May 2020 18:13:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=yn/llej/jTolC/XlId/Yu+ZvA9EsPYY1FRqM7XVwbno=; b=Vh8f6X2MYJxyrY7y3AC6eU3iIHEBfPnkSI+arqU+pU3uitz6coEUwkycrMgm/80sf5 o7nhr5gbs+dAajqAW/9FxN8NgGvc1xmynHAKUWjaxwcDtmpmgTMNrlWB2MnlBYNRDmmK 7H++EqereZaOR7a1fkwmN9x2zVApYbS/xEE5WhC9efpVX5nmbHyqHYZXQMoXiFDQWJm/ 6G2Qh1OSRujZq9CnUHHTrE9ndouz5HVjj7YoFCUiq3a3XJl6hxX/ApqgQQlkiLOPL9wh bvGhiYIGX1KYUmWHG+SFasic0NVHY7A+xC/6AljkJyw51RXER3qsWS7lkteO8CYbX0bg Jezw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=yn/llej/jTolC/XlId/Yu+ZvA9EsPYY1FRqM7XVwbno=; b=FkXJdhRlV3BpgdKkF+8E+QLAN5XiyKZFpIe1vc1iwzRRAZ4odSV5ajzMfdfOAtgHRY AQhBJ+7r3wzwDNLo2A47wj7sMNSOZ2eHiNJruaQj6uMarvcMgycxmPa8jE3gzaDE7Ekv Z4b8kBVWMyDulpjqs855ph/RTm7Xz+Qyh2IoBBkYZ20WWyJoWAcDDcsxKzw81omIdw9a ulgd2TylBfYyiRP3rnRRuquMGFtgkMvjvm4qRfoAzgDOr5ybMm3dyNoBfy43uqqpsgiG 1WtLe8nX5NbERfESBpVUQOOI2Rs4n4Na1cGOPyfMAuslAWmYSNrgxUUfvOAyMMN44rF2 DpOg== X-Gm-Message-State: AGi0PuY2MEHP0YzaXwUwmdcTzOmbh9o9390cKh4m/A09zBVEu3NmrcIf AEi0TtJZObZAonvzvbPRkzV6ai+NqafqEA== X-Google-Smtp-Source: APiQypJ8tJgSlUHn558+s7c0C0m+6F01it30fQtDCUAgUa1/HhGc8KAmyTQSANDLZ2rl4yHbnbjOcQ== X-Received: by 2002:a17:90a:7e4:: with SMTP id m91mr101771pjm.155.1588641217375; Mon, 04 May 2020 18:13:37 -0700 (PDT) Received: from localhost ([8.44.146.30]) by smtp.gmail.com with ESMTPSA id y16sm318989pfp.45.2020.05.04.18.13.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 May 2020 18:13:36 -0700 (PDT) Date: Mon, 4 May 2020 19:13:35 -0600 From: Taylor Blau To: git@vger.kernel.org Cc: dstolee@microsoft.com, gitster@pobox.com, peff@peff.net, szeder.dev@gmail.com Subject: [PATCH 1/8] commit-graph.c: extract 'refs_cb_data' Message-ID: <43286c3c45b2dcef8d9d39b4998c17f364eb1060.1588641176.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In subsequent patches, we are going to update a progress meter when 'add_ref_to_set()' is called, and need a convenient way to pass a 'struct progress *' in from the caller. Introduce 'refs_cb_data' as a catch-all for parameters that 'add_ref_to_set' may need, and wrap the existing single parameter in that struct. Signed-off-by: Taylor Blau --- commit-graph.c | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/commit-graph.c b/commit-graph.c index 6dc777e2f3..00da281f39 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -1318,13 +1318,17 @@ static void compute_bloom_filters(struct write_commit_graph_context *ctx) stop_progress(&progress); } +struct refs_cb_data { + struct oidset *commits; +}; + static int add_ref_to_set(const char *refname, const struct object_id *oid, int flags, void *cb_data) { - struct oidset *commits = (struct oidset *)cb_data; + struct refs_cb_data *data = (struct refs_cb_data *)cb_data; - oidset_insert(commits, oid); + oidset_insert(data->commits, oid); return 0; } @@ -1333,9 +1337,13 @@ int write_commit_graph_reachable(struct object_directory *odb, const struct split_commit_graph_opts *split_opts) { struct oidset commits = OIDSET_INIT; + struct refs_cb_data data; int result; - for_each_ref(add_ref_to_set, &commits); + memset(&data, 0, sizeof(data)); + data.commits = &commits; + + for_each_ref(add_ref_to_set, &data); result = write_commit_graph(odb, NULL, &commits, flags, split_opts); From patchwork Tue May 5 01:13:39 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 11527975 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2A20915E6 for ; Tue, 5 May 2020 01:13:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 13114206C0 for ; Tue, 5 May 2020 01:13:46 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20150623.gappssmtp.com header.i=@ttaylorr-com.20150623.gappssmtp.com header.b="Hmjv8yKt" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727932AbgEEBNn (ORCPT ); Mon, 4 May 2020 21:13:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40030 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1727899AbgEEBNn (ORCPT ); Mon, 4 May 2020 21:13:43 -0400 Received: from mail-pj1-x1042.google.com (mail-pj1-x1042.google.com [IPv6:2607:f8b0:4864:20::1042]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EF574C061A0F for ; Mon, 4 May 2020 18:13:41 -0700 (PDT) Received: by mail-pj1-x1042.google.com with SMTP id y6so273082pjc.4 for ; Mon, 04 May 2020 18:13:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=8nHD7/l+FWqHP8kpZBKCwMK0CWsUxzQbGEIs9CiQc7Q=; b=Hmjv8yKtkGWYT+rcHnmP89PBFAbBWq3xhythq0B4Ra5eGQhzC0Q6XaqDXaus9Fq53W FFOWFLM+4BM7Tuh1KHzsNJqdUenbNzL22lH7z1VQmzZ0qrC6FY3GFBBrD151PNYiHJE2 6of7ajJnkDPoNkXI7f/NPUDAZfm3uoSSJ/Q+YZL45BGNJb6eW4DK/c4aFLd3tBqtaPhL yuCrYXGwiIXCQ7QzFK9cKunis28Eeeai9TnE5I6Cut64KvowrFDnPPZtCqQIxkCyhvIC TeeneeC/8NbSKUe6DvbEsjAvaQ8cuDr8axPUvfyBWLNNBLLCLh507/P/Vp+FKkG2FxtK oPWQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=8nHD7/l+FWqHP8kpZBKCwMK0CWsUxzQbGEIs9CiQc7Q=; b=bkt6ZgWEoxq9VohhXFiyvbq/TZTwHNSKWH0ct1fuLxEMzCUZ4vvRo2T401Zqe8WtwJ bsEraTaIUXpZvgTFKQAV037y/drmRnn0qTYfW+njk3lzGl097i4IT22ojOg/kyylIRye v8BD7vSSMI0cw+ltOm5uV3D0Gfz7nz2OSFOdS6o5CLQKI12YN3ILN4LUmSH4WumRyCVA lZuuyxtmKnNQcoDb6/M/e/LuxALfIiU6dtyB0N+EhNviE+EOyGtB1SEKicUOxPEhN5jR AqSSx3T2ogA0yJG53IsqnT+WjumVWQ6JRQDRgBodiXz1mkd+j6XlA6Sp8IzeUIk3y3LT ymGw== X-Gm-Message-State: AGi0PuYaoIvMb0hyqvrRcPs6GyyRe7+VB+Xfr2QZlkL2yF93yf5OqDaB o+VFOd7l1g/nvx9XAtGbTndcbBHmYJ5scA== X-Google-Smtp-Source: APiQypL41YHCiC1SK0mbw6pX9Vp3Yaf+AowLqGlNXdGa/J2ErkUnbJ8EmEWO4gKXEwwr7gJEX796gw== X-Received: by 2002:a17:90a:d504:: with SMTP id t4mr37215pju.123.1588641221074; Mon, 04 May 2020 18:13:41 -0700 (PDT) Received: from localhost ([8.44.146.30]) by smtp.gmail.com with ESMTPSA id ie17sm198478pjb.19.2020.05.04.18.13.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 May 2020 18:13:40 -0700 (PDT) Date: Mon, 4 May 2020 19:13:39 -0600 From: Taylor Blau To: git@vger.kernel.org Cc: dstolee@microsoft.com, gitster@pobox.com, peff@peff.net, szeder.dev@gmail.com Subject: [PATCH 2/8] commit-graph.c: show progress of finding reachable commits Message-ID: <5bdbeaf374b6050670f800fcdd3b54ddd0750754.1588641176.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When 'git commit-graph write --reachable' is invoked, the commit-graph machinery calls 'for_each_ref()' to discover the set of reachable commits. Right now the 'add_ref_to_set' callback is not doing anything other than adding an OID to the set of known-reachable OIDs. In a subsequent commit, 'add_ref_to_set' will presumptively peel references. This operation should be fast for repositories with an up-to-date '$GIT_DIR/packed-refs', but may be slow in the general case. So that it doesn't appear that 'git commit-graph write' is idling with '--reachable' in the slow case, add a progress meter to provide some output in the meantime. In general, we don't expect a progress meter to appear at all, since peeling references with a 'packed-refs' file is quick. If it's slow and we do show a progress meter, the subsequent 'fill_oids_from_commits()' will be fast, since all of the calls to 'lookup_commit_reference_gently()' will be no-ops. Both progress meters are delayed, so it is unlikely that more than one will appear. In either case, this intermediate state will go away in a handful of patches, at which point there will be at most one progress meter. Signed-off-by: Taylor Blau --- commit-graph.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/commit-graph.c b/commit-graph.c index 00da281f39..8f61256b0a 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -1320,6 +1320,7 @@ static void compute_bloom_filters(struct write_commit_graph_context *ctx) struct refs_cb_data { struct oidset *commits; + struct progress *progress; }; static int add_ref_to_set(const char *refname, @@ -1328,6 +1329,8 @@ static int add_ref_to_set(const char *refname, { struct refs_cb_data *data = (struct refs_cb_data *)cb_data; + display_progress(data->progress, oidset_size(data->commits) + 1); + oidset_insert(data->commits, oid); return 0; } @@ -1342,12 +1345,17 @@ int write_commit_graph_reachable(struct object_directory *odb, memset(&data, 0, sizeof(data)); data.commits = &commits; + if (flags & COMMIT_GRAPH_WRITE_PROGRESS) + data.progress = start_delayed_progress( + _("Finding reachable commits"), 0); for_each_ref(add_ref_to_set, &data); result = write_commit_graph(odb, NULL, &commits, flags, split_opts); oidset_clear(&commits); + if (data.progress) + stop_progress(&data.progress); return result; } From patchwork Tue May 5 01:13:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 11527979 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9208815E6 for ; Tue, 5 May 2020 01:13:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7A66F20658 for ; Tue, 5 May 2020 01:13:49 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20150623.gappssmtp.com header.i=@ttaylorr-com.20150623.gappssmtp.com header.b="PGDvIRcR" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727820AbgEEBNr (ORCPT ); Mon, 4 May 2020 21:13:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40042 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1726449AbgEEBNq (ORCPT ); Mon, 4 May 2020 21:13:46 -0400 Received: from mail-pj1-x1043.google.com (mail-pj1-x1043.google.com [IPv6:2607:f8b0:4864:20::1043]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9F727C061A0F for ; Mon, 4 May 2020 18:13:45 -0700 (PDT) Received: by mail-pj1-x1043.google.com with SMTP id 7so519935pjo.0 for ; Mon, 04 May 2020 18:13:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=jK+Rwt+nS7OjuO/flIwEQvarAHH/qY14AqXmcGd1LX4=; b=PGDvIRcRN33oW4g5LxEWunGv/Ooy466u0jR4JpB98FVDzYQDDNfZ+3EoFDlP7K+5l0 /Hst/OyqML9Yk2q3dk3HFQfgHpB8tPzmIR74+h4PMlKAs3ZXnIWUUbtX/AS+4DS68anF iLhH7//AGd9H6f/WdPm4g13J4mSiFbpKTK/sM6q0pB2jg1VEMC7BmRmMWJXEC25MxICu eG6EqyKE1r9hAUMEdIknNVX2eMHMbMhEpjZafGPHL1K2wnLvFILtD7/oX3MB0YhSUB1r Rk0L9/P4fjCi3yvmLHVDEe5ZlwDYzKiTzV9e3j6u5KpTuVSeDDBSUx/AuSnhhumIOxYn ij6A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=jK+Rwt+nS7OjuO/flIwEQvarAHH/qY14AqXmcGd1LX4=; b=c/kynbKJc18/J7sfoxrH04t7wKI5Aqyu77vclc1Hb9cjsBZ1pDjVGrA7obsBTPwK+J t/QdDA6IrhnFmRXO8NCh7NCp1/chZsQPH6bl5mSJEjqfdBhx3bTt9OlxtOKIIykFxcgk aehTJEUPROmVzVhtm3+LfUwzP9FJ1FLm66vv5OfiRLwkWYpbRWXpgHKyMikEJHGqO/Kz GPZujZmk2FGluoc5tfuVBWzTb4aQiBYZKtZYOmc/N83kRanKtmLn1+TpdJNHv1Umd8Hz aJ7mZ+C/Y7E59qUc7YRk7YDH/ChuhtGFlnyZ12TWQx+xB+VRe7MCnPHMkV09SZsqhJT1 hg+g== X-Gm-Message-State: AGi0PuaecuY/4Fy2k0eJBrcOhRONjC5dwL2rGLJZT5AQO1ht5wZx3nbs aXJ2hmhcOqdBvYCsuS/Ha+ZhufNR8+s5IA== X-Google-Smtp-Source: APiQypIk2j0V7wyQd6Rsn60XDNIEF1l3oBF/Z4gpSVdsPIj3Z0+kElK0MgYK8AHAo79kV06EJeKTrQ== X-Received: by 2002:a17:90a:e28c:: with SMTP id d12mr59250pjz.19.1588641224731; Mon, 04 May 2020 18:13:44 -0700 (PDT) Received: from localhost ([8.44.146.30]) by smtp.gmail.com with ESMTPSA id b15sm297483pfd.139.2020.05.04.18.13.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 May 2020 18:13:44 -0700 (PDT) Date: Mon, 4 May 2020 19:13:43 -0600 From: Taylor Blau To: git@vger.kernel.org Cc: dstolee@microsoft.com, gitster@pobox.com, peff@peff.net, szeder.dev@gmail.com Subject: [PATCH 3/8] commit-graph.c: peel refs in 'add_ref_to_set' Message-ID: <5ff56feab55b005b4a4d9559909ce7a08e5fa81e.1588641176.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org While iterating references (to discover the set of commits to write to the commit-graph with 'git commit-graph write --reachable'), 'add_ref_to_set' can save 'fill_oids_from_commits()' some time by peeling the references beforehand. Move peeling out of 'fill_oids_from_commits()' and into 'add_ref_to_set()' to use 'peel_ref()' instead of 'deref_tag()'. Doing so allows the commit-graph machinery to use the peeled value from '$GIT_DIR/packed-refs' instead of having to load and parse tags. While we're at it, discard non-commit objects reachable from ref tips. This would be done automatically by 'fill_oids_from_commits()', but such functionality will be removed in a subsequent patch after the call to 'lookup_commit_reference_gently' is dropped (at which point a non-commit object in the commits oidset will become an error). Suggested-by: Jeff King Signed-off-by: Taylor Blau --- commit-graph.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/commit-graph.c b/commit-graph.c index 8f61256b0a..5c3fad0dd7 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -1327,11 +1327,15 @@ static int add_ref_to_set(const char *refname, const struct object_id *oid, int flags, void *cb_data) { + struct object_id peeled; struct refs_cb_data *data = (struct refs_cb_data *)cb_data; display_progress(data->progress, oidset_size(data->commits) + 1); - oidset_insert(data->commits, oid); + if (peel_ref(refname, &peeled)) + peeled = *oid; + if (oid_object_info(the_repository, &peeled, NULL) == OBJ_COMMIT) + oidset_insert(data->commits, &peeled); return 0; } From patchwork Tue May 5 01:13:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 11527981 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 29FC015E6 for ; Tue, 5 May 2020 01:13:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 132D6206D7 for ; Tue, 5 May 2020 01:13:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20150623.gappssmtp.com header.i=@ttaylorr-com.20150623.gappssmtp.com header.b="XbUxE3P8" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727982AbgEEBNu (ORCPT ); Mon, 4 May 2020 21:13:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40056 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1726449AbgEEBNt (ORCPT ); Mon, 4 May 2020 21:13:49 -0400 Received: from mail-pg1-x543.google.com (mail-pg1-x543.google.com [IPv6:2607:f8b0:4864:20::543]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1E557C061A0F for ; Mon, 4 May 2020 18:13:49 -0700 (PDT) Received: by mail-pg1-x543.google.com with SMTP id q124so250649pgq.13 for ; Mon, 04 May 2020 18:13:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=vHRIiWx89t6b6i0iG2vGJSmwe30s4IFWz7nTGrcVFZU=; b=XbUxE3P8hnaem+sfnKy+gQSgEGQzaffJU61kv72BLziGNCeGcQrrcQ3KIOQ/DBaA/E P9/blkBfruBenMETxVbjZFid0DCxX7lLY9EILq0HVvgns0Ovry9t65KPRwh1ggn9IdcU 9Fz2W0DhY69JKE7OvdaMWAMbRXkSiCkSSOBHOUBSZ1ed+R4w4++CT1P2r3Vw/cxptH89 eSyi6Nq/kO/ee3pJ9ciJAQEtaUaUOnZYcG/Mk0ln13gQBwuTLuojmKG7RxeLTSFic5eO kXQD6NK0Db61P9UPJmOCPAdf480B2qsrj4bMQnTTZH5ceScZP/CRfEt0/mqMWcEsGjD2 ELlQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=vHRIiWx89t6b6i0iG2vGJSmwe30s4IFWz7nTGrcVFZU=; b=gW1+5twwewEGshhM1pGNdlnHDgmmk4pFc3qrQ32Usq9DvIK41gr4Mf7inYV/29jOYS l9cWDUEV09PbQWnBZesJY2vrj6ETYLm0SCTpMMJ8r4c8b2o7BM2xMoXRlN/HO3e+AG4W /Spn1oxToN0Dc7oiB2y9TOYrcJ1hWs/iDvkTaox8q6dzLzD6+dAgTgDb3BM0LmCV4k5p gxp5URekEZO/tuzsZBYUIxbbbMm2gvJIzOyt0X1qCE1IxS7o2VDIiQJ6KeJPSQLBNlUt wLvLFkt5j7ZPjctqV5fJSRGjhHLZXT9PAO8WZSwaxsAQZQc8oab9eRZTgxbDZ4p3WqPM amZQ== X-Gm-Message-State: AGi0PuZla6r0n1hrkYh8p/eOljUoqx2JPQKFMUgTeNbY13DTlYBfo9WG 2s0Fdhz7r8ZWcguKN0jdOFe4DWBgNRb7+w== X-Google-Smtp-Source: APiQypJF+HW56OtBnD81SuyhZ0oh4wjRxfS7TwXwprIviYiky5uYER6Y5vaegYDuONgdQAxgzsgGqA== X-Received: by 2002:aa7:9ac9:: with SMTP id x9mr705851pfp.304.1588641228188; Mon, 04 May 2020 18:13:48 -0700 (PDT) Received: from localhost ([8.44.146.30]) by smtp.gmail.com with ESMTPSA id t103sm183437pjb.46.2020.05.04.18.13.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 May 2020 18:13:47 -0700 (PDT) Date: Mon, 4 May 2020 19:13:46 -0600 From: Taylor Blau To: git@vger.kernel.org Cc: dstolee@microsoft.com, gitster@pobox.com, peff@peff.net, szeder.dev@gmail.com Subject: [PATCH 4/8] builtin/commit-graph.c: extract 'read_one_commit()' Message-ID: <9ae8745dc090de37af0475ab12b79d541a52713d.1588641176.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org With either '--stdin-commits' or '--stdin-packs', the commit-graph builtin will read line-delimited input, and interpret it either as a series of commit OIDs, or pack names. In a subsequent commit, we will begin handling '--stdin-commits' differently by processing each line as it comes in, instead of in one shot at the end. To make adequate room for this additional logic, split the '--stdin-commits' case from '--stdin-packs' by only storing the input when '--stdin-packs' is given. In the case of '--stdin-commits', feed each line to a new 'read_one_commit' helper, which (for now) will merely call 'parse_oid_hex'. Signed-off-by: Taylor Blau --- builtin/commit-graph.c | 54 +++++++++++++++++++++++------------------- 1 file changed, 29 insertions(+), 25 deletions(-) diff --git a/builtin/commit-graph.c b/builtin/commit-graph.c index 15fe60317c..f550d8489a 100644 --- a/builtin/commit-graph.c +++ b/builtin/commit-graph.c @@ -138,12 +138,25 @@ static int write_option_parse_split(const struct option *opt, const char *arg, return 0; } +static int read_one_commit(struct oidset *commits, char *hash) +{ + struct object_id oid; + const char *end; + + if (parse_oid_hex(hash, &oid, &end)) { + error(_("unexpected non-hex object ID: %s"), hash); + return 1; + } + + oidset_insert(commits, &oid); + return 0; +} + static int graph_write(int argc, const char **argv) { - struct string_list *pack_indexes = NULL; + struct string_list pack_indexes; struct oidset commits = OIDSET_INIT; struct object_directory *odb = NULL; - struct string_list lines; int result = 0; enum commit_graph_write_flags flags = 0; @@ -209,44 +222,35 @@ static int graph_write(int argc, const char **argv) return 0; } - string_list_init(&lines, 0); + string_list_init(&pack_indexes, 0); if (opts.stdin_packs || opts.stdin_commits) { struct strbuf buf = STRBUF_INIT; - - while (strbuf_getline(&buf, stdin) != EOF) - string_list_append(&lines, strbuf_detach(&buf, NULL)); - - if (opts.stdin_packs) - pack_indexes = &lines; if (opts.stdin_commits) { - struct string_list_item *item; - oidset_init(&commits, lines.nr); - for_each_string_list_item(item, &lines) { - struct object_id oid; - const char *end; - - if (parse_oid_hex(item->string, &oid, &end)) { - error(_("unexpected non-hex object ID: " - "%s"), item->string); - return 1; - } - - oidset_insert(&commits, &oid); - } + oidset_init(&commits, 0); flags |= COMMIT_GRAPH_WRITE_CHECK_OIDS; } + while (strbuf_getline(&buf, stdin) != EOF) { + char *line = strbuf_detach(&buf, NULL); + if (opts.stdin_commits) { + int result = read_one_commit(&commits, line); + if (result) + return result; + } else + string_list_append(&pack_indexes, line); + } + UNLEAK(buf); } if (write_commit_graph(odb, - pack_indexes, + opts.stdin_packs ? &pack_indexes : NULL, opts.stdin_commits ? &commits : NULL, flags, &split_opts)) result = 1; - UNLEAK(lines); + UNLEAK(pack_indexes); return result; } From patchwork Tue May 5 01:13:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 11527985 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DDBBD15E6 for ; Tue, 5 May 2020 01:13:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C6208206D7 for ; Tue, 5 May 2020 01:13:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20150623.gappssmtp.com header.i=@ttaylorr-com.20150623.gappssmtp.com header.b="ia3T67Rz" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728015AbgEEBNy (ORCPT ); Mon, 4 May 2020 21:13:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40070 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1727995AbgEEBNx (ORCPT ); Mon, 4 May 2020 21:13:53 -0400 Received: from mail-pj1-x1043.google.com (mail-pj1-x1043.google.com [IPv6:2607:f8b0:4864:20::1043]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 39AE3C061A10 for ; Mon, 4 May 2020 18:13:53 -0700 (PDT) Received: by mail-pj1-x1043.google.com with SMTP id h12so518018pjz.1 for ; Mon, 04 May 2020 18:13:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=5S/9u2/3DE3w+vJ+ZYwegIJwhPJg8PsBxFjVeDOImrI=; b=ia3T67Rz72TpFSgb4vnElRtFji3JdoAQfeVTs4Z0h0Zx/NueDItkz2LTEP41DEnJpK C9EIitRdfK2+FkCmLIJJ4OV6mjX3xTMSVoLsMCLPqPiU3zpPm0y5MVY2IPK8A4O5x4B9 ap3MCh6g6J5ABHh4vm/3xBwQpCPDsauwLvDJiLGzSvi4OrFf88NsDrao+53uHMUiQw9K J/9iBjMJ9s8GrPDrF/WoHbos3BBVetO17gT/afnvCpZXq5VKVDmbVHjkE0KCo3dSgw+F U7GVY04QPhuCl+4e5UjMALW2UR590Znay6gkGJL5xVzBmb7vCacWY2lbbllVoaTQDfrK y4kQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=5S/9u2/3DE3w+vJ+ZYwegIJwhPJg8PsBxFjVeDOImrI=; b=SCa4rH40aXlQPk+3bTkqWYMPhU5TU5IR+HJCnQoVbc2k+vicQ2M1ZI6fbL9ogsOp2e inKAcUIEXNnEe3zRtSNPMhhJNaDyasdljEBkweV2tyHuiUS6iTJgyEmPePrzQCRF1K0Y HXQ6nDnbtVMT0Jzknsi/Yk/nx57UKL1l/8Vmg3FzkYrbqalplNw2rlGwCW8Od+sxofAG pBIu50vBXxPdXDv9iBpobkN3dRT+Ar5yYvxn6iP77N9KSKm02e4MBeOg68Bi3rDeEH0E q9l1EUy1FaLycLcHdUwYmAb8vNRS5Mx39zAe5uU3S+1gMGCrMA/lm8vmh/nZ73Eeq9Y0 NNXQ== X-Gm-Message-State: AGi0PuY8/CT0LvVw7aNyqVYaAJ2G19XjDVAnAL+mYBAmbZOHPnvwkFyd +OxnC+iEG4/uHo0CuMY9HpAkXE+Ys/NZmA== X-Google-Smtp-Source: APiQypJuVlTRXk+b1sJ7yG/HvddNrPdiMoG+eBZlObMSmddJy5SiJVEfaYC/WR/dE3QYYzfv95Bnuw== X-Received: by 2002:a17:90a:3781:: with SMTP id v1mr86287pjb.83.1588641231537; Mon, 04 May 2020 18:13:51 -0700 (PDT) Received: from localhost ([8.44.146.30]) by smtp.gmail.com with ESMTPSA id o11sm236041pgd.58.2020.05.04.18.13.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 May 2020 18:13:50 -0700 (PDT) Date: Mon, 4 May 2020 19:13:49 -0600 From: Taylor Blau To: git@vger.kernel.org Cc: dstolee@microsoft.com, gitster@pobox.com, peff@peff.net, szeder.dev@gmail.com Subject: [PATCH 5/8] builtin/commit-graph.c: dereference tags in builtin Message-ID: <513a634f14e90ec0c2e80a6aaf8cb66bbedf8966.1588641176.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When given a list of commits, the commit-graph machinery calls 'lookup_commit_reference_gently()' on each element in the set and treats the resulting set of OIDs as the base over which to close for reachability. In an earlier collection of commits, the 'git commit-graph write --reachable' case made the inner-most call to 'lookup_commit_reference_gently()' by peeling references before they were passed over to the commit-graph internals. Do the analog for 'git commit-graph write --stdin-commits' by calling 'lookup_commit_reference_gently()' outside of the commit-graph machinery, making the inner-most call a noop. Since this may incur additional processing time, surround 'read_one_commit' with a progress meter to provide output to the caller. Signed-off-by: Taylor Blau --- builtin/commit-graph.c | 25 ++++++++++++++++++++++--- 1 file changed, 22 insertions(+), 3 deletions(-) diff --git a/builtin/commit-graph.c b/builtin/commit-graph.c index f550d8489a..9eec68572f 100644 --- a/builtin/commit-graph.c +++ b/builtin/commit-graph.c @@ -6,6 +6,7 @@ #include "repository.h" #include "commit-graph.h" #include "object-store.h" +#include "progress.h" static char const * const builtin_commit_graph_usage[] = { N_("git commit-graph verify [--object-dir ] [--shallow] [--[no-]progress]"), @@ -138,8 +139,10 @@ static int write_option_parse_split(const struct option *opt, const char *arg, return 0; } -static int read_one_commit(struct oidset *commits, char *hash) +static int read_one_commit(struct oidset *commits, struct progress *progress, + char *hash) { + struct commit *result; struct object_id oid; const char *end; @@ -148,7 +151,15 @@ static int read_one_commit(struct oidset *commits, char *hash) return 1; } - oidset_insert(commits, &oid); + display_progress(progress, oidset_size(commits) + 1); + + result = lookup_commit_reference_gently(the_repository, &oid, 1); + if (result) + oidset_insert(commits, &result->object.oid); + else { + error(_("invalid commit object id: %s"), hash); + return 1; + } return 0; } @@ -159,6 +170,7 @@ static int graph_write(int argc, const char **argv) struct object_directory *odb = NULL; int result = 0; enum commit_graph_write_flags flags = 0; + struct progress *progress = NULL; static struct option builtin_commit_graph_write_options[] = { OPT_STRING(0, "object-dir", &opts.obj_dir, @@ -228,18 +240,25 @@ static int graph_write(int argc, const char **argv) if (opts.stdin_commits) { oidset_init(&commits, 0); flags |= COMMIT_GRAPH_WRITE_CHECK_OIDS; + if (opts.progress) + progress = start_delayed_progress( + _("Analyzing commits from stdin"), 0); } while (strbuf_getline(&buf, stdin) != EOF) { char *line = strbuf_detach(&buf, NULL); if (opts.stdin_commits) { - int result = read_one_commit(&commits, line); + int result = read_one_commit(&commits, progress, + line); if (result) return result; } else string_list_append(&pack_indexes, line); } + if (progress) + stop_progress(&progress); + UNLEAK(buf); } From patchwork Tue May 5 01:13:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 11527987 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8D7C415E6 for ; Tue, 5 May 2020 01:13:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7685E206D7 for ; Tue, 5 May 2020 01:13:58 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20150623.gappssmtp.com header.i=@ttaylorr-com.20150623.gappssmtp.com header.b="TLdVOFKN" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728050AbgEEBN5 (ORCPT ); Mon, 4 May 2020 21:13:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40084 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1728025AbgEEBN4 (ORCPT ); Mon, 4 May 2020 21:13:56 -0400 Received: from mail-pf1-x444.google.com (mail-pf1-x444.google.com [IPv6:2607:f8b0:4864:20::444]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CFAC0C061A0F for ; Mon, 4 May 2020 18:13:56 -0700 (PDT) Received: by mail-pf1-x444.google.com with SMTP id x15so81693pfa.1 for ; Mon, 04 May 2020 18:13:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=hMHhlnIpEEVGwdI31E60N9MAO0ZW52wwNAwbjWLYaws=; b=TLdVOFKNmbbPjtGuvJu7WKcA4ebFdMjLIkZq9Qe1qud0nIezJLjCtP2KNx4Yj5BfQt pzQphNw1M634YQN16BT24A/6rXdSXfXbERaI4ZrY2r9dUvg9QmId9M1dVOW4GKn6MaRg QxBvWFWIlAClR9TW5Gn4Od/Q8g8g1F0/9z1cKnh3/WM7DCz/ladfBsNzKovsu28fYN3E AQmE1afQI/BCO/hVhJknrZLbYYHEqPwqy1toC0aDBB54zXkBpKn9MKQNYg2rtXVhFEHv 7DVWwN2RWfnk6oyN+j7xWOx8oPGO+MHyQRcqFKvzR3IImvFM9V1nmB+W5R1Q9DlM9dvi uzBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=hMHhlnIpEEVGwdI31E60N9MAO0ZW52wwNAwbjWLYaws=; b=TxRSZ4HR5XxgT9y8uZn8gzSr+Ho9uFAD/6VXlVbD2kO8vrWqxnrW/r2kgWD5w60UPr naRyqSRIdoPig3UruPeSNSf/08Ax/2wQAcQTnysDnWf1SoqE6lOUgzsTwlQ106KOi1wN n8z8MZ0fkXtcRExTxjvOQM9Ab+s9z/pfg32HVLCvga1JAh7N1giqmYDwuVl5saB1wASe 2/inpneS+ZM2bnl2roK7CfiPCGwlegcbGsbb/GIPN+bQiid4UAn8US3bPiVAyFgzN2mG VDd0yoUtoIkpcwNoxr56dTN//sh+7tV2TpWlGZ4BSxtT0dOSbhQNB+2+8wZH9trmeHrf pzcg== X-Gm-Message-State: AGi0PuZs8tTs7xgsD0/2D56Y9n5KLjw9AF28i9v8l8rjI8VbBrfkqMyk rOpF9ICd16es1tY7o23rcXCJ+t+4JaCe2Q== X-Google-Smtp-Source: APiQypJ0Z2yvp5HNzSi6IiQJlc7WSmwvzTa9T1iL5FnKcmsMHsWSiW6lew3lgAB7KSUCVuZFhOTk3g== X-Received: by 2002:a62:3784:: with SMTP id e126mr713192pfa.303.1588641235964; Mon, 04 May 2020 18:13:55 -0700 (PDT) Received: from localhost ([8.44.146.30]) by smtp.gmail.com with ESMTPSA id y6sm223377pjw.15.2020.05.04.18.13.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 May 2020 18:13:55 -0700 (PDT) Date: Mon, 4 May 2020 19:13:54 -0600 From: Taylor Blau To: git@vger.kernel.org Cc: dstolee@microsoft.com, gitster@pobox.com, peff@peff.net, szeder.dev@gmail.com Subject: [PATCH 6/8] commit-graph.c: simplify 'fill_oids_from_commits' Message-ID: <7e9d8c1f1a124171ebb5b4d874718053d1c9064a.1588641176.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In the previous handful of commits, both 'git commit-graph write --reachable' and '--stdin-commits' learned to peel tags down to the commits which they refer to before passing them into the commit-graph internals. This makes the call to 'lookup_commit_reference_gently()' inside of 'fill_oids_from_commits()' a noop, since all OIDs are commits by that point. As such, remove the call entirely, as well as the progress meter, which has been split and moved out to the callers in the aforementioned earlier commits. Signed-off-by: Taylor Blau --- commit-graph.c | 33 +++------------------------------ 1 file changed, 3 insertions(+), 30 deletions(-) diff --git a/commit-graph.c b/commit-graph.c index 5c3fad0dd7..24c2d80935 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -1411,46 +1411,19 @@ static int fill_oids_from_packs(struct write_commit_graph_context *ctx, static int fill_oids_from_commits(struct write_commit_graph_context *ctx, struct oidset *commits) { - uint32_t i = 0; - struct strbuf progress_title = STRBUF_INIT; struct oidset_iter iter; struct object_id *oid; if (!oidset_size(commits)) return 0; - if (ctx->report_progress) { - strbuf_addf(&progress_title, - Q_("Finding commits for commit graph from %d ref", - "Finding commits for commit graph from %d refs", - oidset_size(commits)), - oidset_size(commits)); - ctx->progress = start_delayed_progress( - progress_title.buf, - oidset_size(commits)); - } - oidset_iter_init(commits, &iter); while ((oid = oidset_iter_next(&iter))) { - struct commit *result; - - display_progress(ctx->progress, ++i); - - result = lookup_commit_reference_gently(ctx->r, oid, 1); - if (result) { - ALLOC_GROW(ctx->oids.list, ctx->oids.nr + 1, ctx->oids.alloc); - oidcpy(&ctx->oids.list[ctx->oids.nr], &(result->object.oid)); - ctx->oids.nr++; - } else if (ctx->check_oids) { - error(_("invalid commit object id: %s"), - oid_to_hex(oid)); - return -1; - } + ALLOC_GROW(ctx->oids.list, ctx->oids.nr + 1, ctx->oids.alloc); + oidcpy(&ctx->oids.list[ctx->oids.nr], oid); + ctx->oids.nr++; } - stop_progress(&ctx->progress); - strbuf_release(&progress_title); - return 0; } From patchwork Tue May 5 01:14:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 11527991 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6C52F15E6 for ; Tue, 5 May 2020 01:14:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5559520705 for ; Tue, 5 May 2020 01:14:04 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20150623.gappssmtp.com header.i=@ttaylorr-com.20150623.gappssmtp.com header.b="LtzcHu2M" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728083AbgEEBOD (ORCPT ); Mon, 4 May 2020 21:14:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40106 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1727995AbgEEBOC (ORCPT ); Mon, 4 May 2020 21:14:02 -0400 Received: from mail-pg1-x543.google.com (mail-pg1-x543.google.com [IPv6:2607:f8b0:4864:20::543]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AD962C061A0F for ; Mon, 4 May 2020 18:14:02 -0700 (PDT) Received: by mail-pg1-x543.google.com with SMTP id d22so277842pgk.3 for ; Mon, 04 May 2020 18:14:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=iHhSGMGzCi6ysuU18iYcv3qCsk++2Ddth7iq5fAYkqc=; b=LtzcHu2MkpkKfL5q/q9kIMb0bUiLy9JrrhuktLaZOpnG5TzdwrgK9LeccMi8swj7r1 2Mes3kziuXhxrCRmqIb65GDR0czTqkLAG7PO32SPq7ygtEVI/QX+ETi4q2Yyml7mArvj 51J17HBHZywSHh7EMKw0eWPvmiRxX+y4RJAK8otGC4FmO2iLI0tMkyl7j87V1lVVX7f4 itFuUBFnkQlxM84fMO8TIlQF5GdWl3QXRb5NF1UzjcbslIKwIHSTjK9rJOQPVheWKryC 5xRx2mIQ2lrOEOiUZzN66GLqTPIovW0tKS5GqCqbtC6owlDR4nPzkUg4IhwjyOweGyry aYZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=iHhSGMGzCi6ysuU18iYcv3qCsk++2Ddth7iq5fAYkqc=; b=HfSRfyWQ0TJfFzyTma2KI2CXpK/zaDHDP0roaBrsspr15qb53FupOKmKxOcQl/ozyC kPF2yTJ2lDv2Ddxo06JhKSkm4AWeBS2u46DEiLz0JagzGvoUcM+AoUKRLAUc0ayWG1Nz CS/03iKUO6z0prRAauvXuSPC511HKtAaG7xDewIHaPXKjYq78ExS7mpPhGDqZO0GOfZl Urh1J1Osl82tXOMtGSd4F8yIKjHNWtS1aiJfO3FxAsO5/oCZWJcBwfEAzP6ctIEGh2xD TIGhcBj3DzSgvqwyEXP+9mxwRJqIEPhlNOTNTCjeX90DhZBvqv/S3mMr2PmcNJLpiyNO ULzA== X-Gm-Message-State: AGi0PuaFQ8d3oJ+Zf+nFQPmmysfaQ+jI4zrZoLxTec/KsXpBQLfeYXJQ rwEF8tLtbCYZPPIxtA0cRLVQ/Agc60s7Cg== X-Google-Smtp-Source: APiQypJ7XFy8bDPS5SNrp9NFtjJfdenbg74Ke80LizNhsdfkDsw/Mv8PXvVX20tB/VTveuiWlgtYCQ== X-Received: by 2002:a63:6847:: with SMTP id d68mr910358pgc.56.1588641241884; Mon, 04 May 2020 18:14:01 -0700 (PDT) Received: from localhost ([8.44.146.30]) by smtp.gmail.com with ESMTPSA id q7sm252979pgs.13.2020.05.04.18.14.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 May 2020 18:14:01 -0700 (PDT) Date: Mon, 4 May 2020 19:14:00 -0600 From: Taylor Blau To: git@vger.kernel.org Cc: dstolee@microsoft.com, gitster@pobox.com, peff@peff.net, szeder.dev@gmail.com Subject: [PATCH 7/8] t5318: reorder test below 'graph_read_expect' Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In the subsequent commit, we will introduce a dependency on 'graph_read_expect' from t5318.7. Preemptively move it below 'graph_read_expect()'s definition so that the test can call it. Signed-off-by: Taylor Blau --- t/t5318-commit-graph.sh | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/t/t5318-commit-graph.sh b/t/t5318-commit-graph.sh index 39e2918a32..89020d3d44 100755 --- a/t/t5318-commit-graph.sh +++ b/t/t5318-commit-graph.sh @@ -42,15 +42,6 @@ test_expect_success 'create commits and repack' ' git repack ' -test_expect_success 'exit with correct error on bad input to --stdin-commits' ' - cd "$TRASH_DIRECTORY/full" && - echo HEAD | test_expect_code 1 git commit-graph write --stdin-commits 2>stderr && - test_i18ngrep "unexpected non-hex object ID: HEAD" stderr && - # valid tree OID, but not a commit OID - git rev-parse HEAD^{tree} | test_expect_code 1 git commit-graph write --stdin-commits 2>stderr && - test_i18ngrep "invalid commit object id" stderr -' - graph_git_two_modes() { git -c core.commitGraph=true $1 >output git -c core.commitGraph=false $1 >expect @@ -91,6 +82,15 @@ graph_read_expect() { test_cmp expect output } +test_expect_success 'exit with correct error on bad input to --stdin-commits' ' + cd "$TRASH_DIRECTORY/full" && + echo HEAD | test_expect_code 1 git commit-graph write --stdin-commits 2>stderr && + test_i18ngrep "unexpected non-hex object ID: HEAD" stderr && + # valid tree OID, but not a commit OID + git rev-parse HEAD^{tree} | test_expect_code 1 git commit-graph write --stdin-commits 2>stderr && + test_i18ngrep "invalid commit object id" stderr +' + test_expect_success 'write graph' ' cd "$TRASH_DIRECTORY/full" && git commit-graph write && From patchwork Tue May 5 01:14:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 11527993 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D93CB15E6 for ; Tue, 5 May 2020 01:14:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BC284206D7 for ; Tue, 5 May 2020 01:14:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20150623.gappssmtp.com header.i=@ttaylorr-com.20150623.gappssmtp.com header.b="mqot0c6b" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728092AbgEEBOI (ORCPT ); Mon, 4 May 2020 21:14:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40116 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1727995AbgEEBOH (ORCPT ); Mon, 4 May 2020 21:14:07 -0400 Received: from mail-pg1-x541.google.com (mail-pg1-x541.google.com [IPv6:2607:f8b0:4864:20::541]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 46877C061A10 for ; Mon, 4 May 2020 18:14:06 -0700 (PDT) Received: by mail-pg1-x541.google.com with SMTP id d22so277915pgk.3 for ; Mon, 04 May 2020 18:14:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=Qxp+RYUQvLl8UoxEVVi+qxYVgTs+8lkxtE2/JD3YErA=; b=mqot0c6b77QVc9ajKlBwR2jqSITxMjx6pL7L9USISyADFofHZYJ2NepYMVl+zBvfM9 GkXmntqzHJ6kDzzcH6If39EYxB8XxwAPjyzCa/gTav83+MIpXr8i73+OXA7B1QI9Rk+t ovgFCGa36mLbLALUmREJ86WHkLqqArEEAdtO1SwhKpQG9ca0LCv2IuCRZKijEWItvK5P FBy1pUjc153NceBkop1Ngq5oQjmGm6JjF3A0oxvMmMJiyx19AnOR7IHNCW5iaePhll+M IGhN4nspBIdsE4KIq+dCeVKFDIioibKwZ69FZxitIHCG98i8O8efVQmyWAZCuUy0A4jf Kn0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=Qxp+RYUQvLl8UoxEVVi+qxYVgTs+8lkxtE2/JD3YErA=; b=kKoEHKKGvlSvghBVAyvCFxfuI0cJ4e0RCZHAm1/toI7RUczPfxpfn/M9K+uKBjPgBa wrnX71HYtxhk1KbrTuVz2Ub5oxc1jxGr1NrsQgXcltFLil8/ywUNPaQKXX7Y9IxnKhFI cxVfkrRqRgVlLqVp4xaFQ347OeNp0FovbT02aJHqDWHVO+MyhuF8ZSrrMNJUEEPTuwKk 956rLJOncZlmz0MFj5prKaJ5zjhWKq5uPOobhWVmeVlOW+5/Eqodx9ZUQcme6RSUJMKF rvID9JMND7eISByGvGiV3MsL/fjP8fd2Sve8VUYRRtDyN5I1UKmvjz6BbuQVfI42v5c0 OTXw== X-Gm-Message-State: AGi0PuaCORJwbJ2vpltEfzZl8ah8wxBmPT15nEoJkmTY3egQhD/FmTCo 925bJiDF9hRfnCQpZEsHPxMFYyMAT4Eb4A== X-Google-Smtp-Source: APiQypLNyim69jroftsmRms/XIYp4MiMBfro4BNRdZY777IeehjZAGa9a4MN0F6v13WBV0EhVMVPWQ== X-Received: by 2002:a63:1447:: with SMTP id 7mr868601pgu.253.1588641245223; Mon, 04 May 2020 18:14:05 -0700 (PDT) Received: from localhost ([8.44.146.30]) by smtp.gmail.com with ESMTPSA id j7sm234210pjy.9.2020.05.04.18.14.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 May 2020 18:14:04 -0700 (PDT) Date: Mon, 4 May 2020 19:14:03 -0600 From: Taylor Blau To: git@vger.kernel.org Cc: dstolee@microsoft.com, gitster@pobox.com, peff@peff.net, szeder.dev@gmail.com Subject: [PATCH 8/8] commit-graph: drop COMMIT_GRAPH_WRITE_CHECK_OIDS flag Message-ID: <6c2d130b0cd4b6a8a541b362ae7dd44d4c282e3f.1588641176.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Since 7c5c9b9c57 (commit-graph: error out on invalid commit oids in 'write --stdin-commits', 2019-08-05), the commit-graph builtin dies on receiving non-commit OIDs as input to '--stdin-commits'. This behavior can be cumbersome to work around in, say, the case of piping 'git for-each-ref' to 'git commit-graph write --stdin-commits' if the caller does not want to cull out non-commits themselves. In this situation, it would be ideal if 'git commit-graph write' wrote the graph containing the inputs that did pertain to commits, and silently ignored the remainder of the input. Some options have been proposed to the effect of '--[no-]check-oids' which would allow callers to have the commit-graph builtin do just that. After some discussion, it is difficult to imagine a caller who wouldn't want to pass '--no-check-oids', suggesting that we should get rid of the behavior of complaining about non-commit inputs altogether. If callers do wish to retain this behavior, they can easily work around this change by doing the following: git for-each-ref --format='%(objectname) %(objecttype) %(*objecttype)' | awk '/commit/ { print $1 }' | git commit-graph write --stdin-commits To make it so that valid OIDs that refer to non-existent objects are indeed an error after loosening the error handling, perform an extra lookup to make sure that object indeed exists before sending it to the commit-graph internals. Signed-off-by: Taylor Blau Reviewed-by: Derrick Stolee --- Documentation/git-commit-graph.txt | 6 ++++-- builtin/commit-graph.c | 10 +++++----- commit-graph.c | 2 -- commit-graph.h | 10 ++++------ t/t5318-commit-graph.sh | 15 +++++++++++---- 5 files changed, 24 insertions(+), 19 deletions(-) diff --git a/Documentation/git-commit-graph.txt b/Documentation/git-commit-graph.txt index 53a650225a..fcac7d12e1 100644 --- a/Documentation/git-commit-graph.txt +++ b/Documentation/git-commit-graph.txt @@ -47,8 +47,10 @@ with `--stdin-commits` or `--reachable`.) + With the `--stdin-commits` option, generate the new commit graph by walking commits starting at the commits specified in stdin as a list -of OIDs in hex, one OID per line. (Cannot be combined with -`--stdin-packs` or `--reachable`.) +of OIDs in hex, one OID per line. OIDs that resolve to non-commits +(either directly, or by peeling tags) are silently ignored. OIDs that +are malformed, or do not exist generate an error. (Cannot be combined +with `--stdin-packs` or `--reachable`.) + With the `--reachable` option, generate the new commit graph by walking commits starting at all refs. (Cannot be combined with `--stdin-commits` diff --git a/builtin/commit-graph.c b/builtin/commit-graph.c index 9eec68572f..3637d079fb 100644 --- a/builtin/commit-graph.c +++ b/builtin/commit-graph.c @@ -153,13 +153,14 @@ static int read_one_commit(struct oidset *commits, struct progress *progress, display_progress(progress, oidset_size(commits) + 1); + if (oid_object_info(the_repository, &oid, NULL) < 0) { + error(_("object %s does not exist"), hash); + return 1; + } + result = lookup_commit_reference_gently(the_repository, &oid, 1); if (result) oidset_insert(commits, &result->object.oid); - else { - error(_("invalid commit object id: %s"), hash); - return 1; - } return 0; } @@ -239,7 +240,6 @@ static int graph_write(int argc, const char **argv) struct strbuf buf = STRBUF_INIT; if (opts.stdin_commits) { oidset_init(&commits, 0); - flags |= COMMIT_GRAPH_WRITE_CHECK_OIDS; if (opts.progress) progress = start_delayed_progress( _("Analyzing commits from stdin"), 0); diff --git a/commit-graph.c b/commit-graph.c index 24c2d80935..4e5539bac5 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -880,7 +880,6 @@ struct write_commit_graph_context { unsigned append:1, report_progress:1, split:1, - check_oids:1, changed_paths:1, order_by_pack:1; @@ -2001,7 +2000,6 @@ int write_commit_graph(struct object_directory *odb, ctx->append = flags & COMMIT_GRAPH_WRITE_APPEND ? 1 : 0; ctx->report_progress = flags & COMMIT_GRAPH_WRITE_PROGRESS ? 1 : 0; ctx->split = flags & COMMIT_GRAPH_WRITE_SPLIT ? 1 : 0; - ctx->check_oids = flags & COMMIT_GRAPH_WRITE_CHECK_OIDS ? 1 : 0; ctx->split_opts = split_opts; ctx->changed_paths = flags & COMMIT_GRAPH_WRITE_BLOOM_FILTERS ? 1 : 0; ctx->total_bloom_filter_data_size = 0; diff --git a/commit-graph.h b/commit-graph.h index 4212766a4f..659393e07f 100644 --- a/commit-graph.h +++ b/commit-graph.h @@ -88,12 +88,10 @@ struct commit_graph *parse_commit_graph(void *graph_map, size_t graph_size); int generation_numbers_enabled(struct repository *r); enum commit_graph_write_flags { - COMMIT_GRAPH_WRITE_APPEND = (1 << 0), - COMMIT_GRAPH_WRITE_PROGRESS = (1 << 1), - COMMIT_GRAPH_WRITE_SPLIT = (1 << 2), - /* Make sure that each OID in the input is a valid commit OID. */ - COMMIT_GRAPH_WRITE_CHECK_OIDS = (1 << 3), - COMMIT_GRAPH_WRITE_BLOOM_FILTERS = (1 << 4), + COMMIT_GRAPH_WRITE_APPEND = (1 << 0), + COMMIT_GRAPH_WRITE_PROGRESS = (1 << 1), + COMMIT_GRAPH_WRITE_SPLIT = (1 << 2), + COMMIT_GRAPH_WRITE_BLOOM_FILTERS = (1 << 3) }; enum commit_graph_split_flags { diff --git a/t/t5318-commit-graph.sh b/t/t5318-commit-graph.sh index 89020d3d44..74f93f0a17 100755 --- a/t/t5318-commit-graph.sh +++ b/t/t5318-commit-graph.sh @@ -84,11 +84,18 @@ graph_read_expect() { test_expect_success 'exit with correct error on bad input to --stdin-commits' ' cd "$TRASH_DIRECTORY/full" && - echo HEAD | test_expect_code 1 git commit-graph write --stdin-commits 2>stderr && + # invalid, non-hex OID + echo HEAD >in && + test_expect_code 1 git commit-graph write --stdin-commits stderr && test_i18ngrep "unexpected non-hex object ID: HEAD" stderr && - # valid tree OID, but not a commit OID - git rev-parse HEAD^{tree} | test_expect_code 1 git commit-graph write --stdin-commits 2>stderr && - test_i18ngrep "invalid commit object id" stderr + # non-existent OID + echo $ZERO_OID >in && + test_expect_code 1 git commit-graph write --stdin-commits stderr && + test_i18ngrep "does not exist" stderr && + # valid commit and tree OID + git rev-parse HEAD HEAD^{tree} >in && + git commit-graph write --stdin-commits