From patchwork Fri Sep 14 00:55:21 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew DeVore X-Patchwork-Id: 10600223 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6A5E314BD for ; Fri, 14 Sep 2018 00:56:05 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D0D862B307 for ; Fri, 14 Sep 2018 00:56:04 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C300B2B34E; Fri, 14 Sep 2018 00:56:04 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C07CC2B307 for ; Fri, 14 Sep 2018 00:56:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728186AbeINGHu (ORCPT ); Fri, 14 Sep 2018 02:07:50 -0400 Received: from mail-yw1-f73.google.com ([209.85.161.73]:38060 "EHLO mail-yw1-f73.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726558AbeINGHu (ORCPT ); Fri, 14 Sep 2018 02:07:50 -0400 Received: by mail-yw1-f73.google.com with SMTP id q141-v6so2933384ywg.5 for ; Thu, 13 Sep 2018 17:55:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=Yfcas47zeDu4DD5jwqOAeKK4EXmcqgYS+tbPtnA61Mg=; b=XACFX4JTh82WgUJO5w25mA4TSBxIp+IIfRzsLLM3JOqUpZFC01ymtWBivj5McyVQai hyInecZ8tPQNnD8n+6XWaugQtFS3Lo0WRtfJ/N5Hqr/YuOWrhhKNVHOwvGHx9dLde/t2 VvYW+m3Hm1g6GK3ondMD+dDDLFr3F33JhshQeJufIq6Y5K9dUeHyq5PXA0FxjutGX33F +f4gP4Fv0uqiDAizgq0/SvRpl3BPo45dDiry5rUm2dpsFZBgdThe0hISd1FkuRZFPBME ZW7mpBApfwAV50oWvsOhEGmasyu2B9kqNINexb+rII7g1FddrIoFsJZpiIhkDHkmI475 CF+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=Yfcas47zeDu4DD5jwqOAeKK4EXmcqgYS+tbPtnA61Mg=; b=Y4d5o/Ig+n4p1pJuWi7THO4+0ioKm3C12lpou3dVDru1WTjCurR7TqqwGFV4EC6oEd H3NRjMB3aLTW0eel25duvNX7Nswjz6+CBMsqmxZzdhqeMbYlkhcrLc85NMPoz2RQ03Vq pZY7U+fdtzHZBS/BIEDyMSpR4Y9qM2PhFxx2KvNbk4Tp1R9zW2uVHLrKDJ9+Wvrwpm98 zfaQWaGw/tXLH0WFRvA8XlG0qLSHPQUZdokIM52XfTraTqeO482w+QRumo5GvQI6MSyp TGF1etYiZAyizX5oCMKhpvGG9o8rkSUtS3FV0AeOkb6K2ggU9BU1KSOBuLdfStvLpLXl FC3A== X-Gm-Message-State: APzg51DGd6d5I00TgOVMf6abSKTNW9VLPsJ/+KEYx+h0iaclOTimuiTF xuHa3L3f8c2rP7ZsJRddvGsQX6xd96Q= X-Google-Smtp-Source: ANB0VdYQR5XIWob3ntXS+k14MN/lXEzs+tNKeJYih6eYoeWrzmXvgBNaXr3du4jlTbKb977UoLpj1rJzrxdT X-Received: by 2002:a5b:444:: with SMTP id s4-v6mr1674675ybp.58.1536886554157; Thu, 13 Sep 2018 17:55:54 -0700 (PDT) Date: Thu, 13 Sep 2018 17:55:21 -0700 In-Reply-To: Message-Id: <3de57eb7f9e662d93fd166b92c35719af9ea6c31.1536885967.git.matvore@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.19.0.397.gdd90340f6a-goog Subject: [PATCH v8 1/7] list-objects: store common func args in struct From: Matthew DeVore To: sbeller@google.com, git@vger.kernel.org Cc: Matthew DeVore , git@jeffhostetler.com, jeffhost@microsoft.com, peff@peff.net, stefanbeller@gmail.com, jonathantanmy@google.com, gitster@pobox.com, pclouds@gmail.com Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This will make utility functions easier to create, as done by the next patch. Signed-off-by: Matthew DeVore --- list-objects.c | 158 +++++++++++++++++++++++-------------------------- 1 file changed, 74 insertions(+), 84 deletions(-) diff --git a/list-objects.c b/list-objects.c index c99c47ac1..584518a3f 100644 --- a/list-objects.c +++ b/list-objects.c @@ -12,20 +12,25 @@ #include "packfile.h" #include "object-store.h" -static void process_blob(struct rev_info *revs, +struct traversal_context { + struct rev_info *revs; + show_object_fn show_object; + show_commit_fn show_commit; + void *show_data; + filter_object_fn filter_fn; + void *filter_data; +}; + +static void process_blob(struct traversal_context *ctx, struct blob *blob, - show_object_fn show, struct strbuf *path, - const char *name, - void *cb_data, - filter_object_fn filter_fn, - void *filter_data) + const char *name) { struct object *obj = &blob->object; size_t pathlen; enum list_objects_filter_result r = LOFR_MARK_SEEN | LOFR_DO_SHOW; - if (!revs->blob_objects) + if (!ctx->revs->blob_objects) return; if (!obj) die("bad blob object"); @@ -41,21 +46,21 @@ static void process_blob(struct rev_info *revs, * may cause the actual filter to report an incomplete list * of missing objects. */ - if (revs->exclude_promisor_objects && + if (ctx->revs->exclude_promisor_objects && !has_object_file(&obj->oid) && is_promisor_object(&obj->oid)) return; pathlen = path->len; strbuf_addstr(path, name); - if (!(obj->flags & USER_GIVEN) && filter_fn) - r = filter_fn(LOFS_BLOB, obj, - path->buf, &path->buf[pathlen], - filter_data); + if (!(obj->flags & USER_GIVEN) && ctx->filter_fn) + r = ctx->filter_fn(LOFS_BLOB, obj, + path->buf, &path->buf[pathlen], + ctx->filter_data); if (r & LOFR_MARK_SEEN) obj->flags |= SEEN; if (r & LOFR_DO_SHOW) - show(obj, path->buf, cb_data); + ctx->show_object(obj, path->buf, ctx->show_data); strbuf_setlen(path, pathlen); } @@ -81,26 +86,21 @@ static void process_blob(struct rev_info *revs, * the link, and how to do it. Whether it necessarily makes * any sense what-so-ever to ever do that is another issue. */ -static void process_gitlink(struct rev_info *revs, +static void process_gitlink(struct traversal_context *ctx, const unsigned char *sha1, - show_object_fn show, struct strbuf *path, - const char *name, - void *cb_data) + const char *name) { /* Nothing to do */ } -static void process_tree(struct rev_info *revs, +static void process_tree(struct traversal_context *ctx, struct tree *tree, - show_object_fn show, struct strbuf *base, - const char *name, - void *cb_data, - filter_object_fn filter_fn, - void *filter_data) + const char *name) { struct object *obj = &tree->object; + struct rev_info *revs = ctx->revs; struct tree_desc desc; struct name_entry entry; enum interesting match = revs->diffopt.pathspec.nr == 0 ? @@ -133,14 +133,14 @@ static void process_tree(struct rev_info *revs, } strbuf_addstr(base, name); - if (!(obj->flags & USER_GIVEN) && filter_fn) - r = filter_fn(LOFS_BEGIN_TREE, obj, - base->buf, &base->buf[baselen], - filter_data); + if (!(obj->flags & USER_GIVEN) && ctx->filter_fn) + r = ctx->filter_fn(LOFS_BEGIN_TREE, obj, + base->buf, &base->buf[baselen], + ctx->filter_data); if (r & LOFR_MARK_SEEN) obj->flags |= SEEN; if (r & LOFR_DO_SHOW) - show(obj, base->buf, cb_data); + ctx->show_object(obj, base->buf, ctx->show_data); if (base->len) strbuf_addch(base, '/'); @@ -157,29 +157,25 @@ static void process_tree(struct rev_info *revs, } if (S_ISDIR(entry.mode)) - process_tree(revs, + process_tree(ctx, lookup_tree(the_repository, entry.oid), - show, base, entry.path, - cb_data, filter_fn, filter_data); + base, entry.path); else if (S_ISGITLINK(entry.mode)) - process_gitlink(revs, entry.oid->hash, - show, base, entry.path, - cb_data); + process_gitlink(ctx, entry.oid->hash, base, entry.path); else - process_blob(revs, + process_blob(ctx, lookup_blob(the_repository, entry.oid), - show, base, entry.path, - cb_data, filter_fn, filter_data); + base, entry.path); } - if (!(obj->flags & USER_GIVEN) && filter_fn) { - r = filter_fn(LOFS_END_TREE, obj, - base->buf, &base->buf[baselen], - filter_data); + if (!(obj->flags & USER_GIVEN) && ctx->filter_fn) { + r = ctx->filter_fn(LOFS_END_TREE, obj, + base->buf, &base->buf[baselen], + ctx->filter_data); if (r & LOFR_MARK_SEEN) obj->flags |= SEEN; if (r & LOFR_DO_SHOW) - show(obj, base->buf, cb_data); + ctx->show_object(obj, base->buf, ctx->show_data); } strbuf_setlen(base, baselen); @@ -242,19 +238,15 @@ static void add_pending_tree(struct rev_info *revs, struct tree *tree) add_pending_object(revs, &tree->object, ""); } -static void traverse_trees_and_blobs(struct rev_info *revs, - struct strbuf *base, - show_object_fn show_object, - void *show_data, - filter_object_fn filter_fn, - void *filter_data) +static void traverse_trees_and_blobs(struct traversal_context *ctx, + struct strbuf *base) { int i; assert(base->len == 0); - for (i = 0; i < revs->pending.nr; i++) { - struct object_array_entry *pending = revs->pending.objects + i; + for (i = 0; i < ctx->revs->pending.nr; i++) { + struct object_array_entry *pending = ctx->revs->pending.objects + i; struct object *obj = pending->item; const char *name = pending->name; const char *path = pending->path; @@ -262,62 +254,49 @@ static void traverse_trees_and_blobs(struct rev_info *revs, continue; if (obj->type == OBJ_TAG) { obj->flags |= SEEN; - show_object(obj, name, show_data); + ctx->show_object(obj, name, ctx->show_data); continue; } if (!path) path = ""; if (obj->type == OBJ_TREE) { - process_tree(revs, (struct tree *)obj, show_object, - base, path, show_data, - filter_fn, filter_data); + process_tree(ctx, (struct tree *)obj, base, path); continue; } if (obj->type == OBJ_BLOB) { - process_blob(revs, (struct blob *)obj, show_object, - base, path, show_data, - filter_fn, filter_data); + process_blob(ctx, (struct blob *)obj, base, path); continue; } die("unknown pending object %s (%s)", oid_to_hex(&obj->oid), name); } - object_array_clear(&revs->pending); + object_array_clear(&ctx->revs->pending); } -static void do_traverse(struct rev_info *revs, - show_commit_fn show_commit, - show_object_fn show_object, - void *show_data, - filter_object_fn filter_fn, - void *filter_data) +static void do_traverse(struct traversal_context *ctx) { struct commit *commit; struct strbuf csp; /* callee's scratch pad */ strbuf_init(&csp, PATH_MAX); - while ((commit = get_revision(revs)) != NULL) { + while ((commit = get_revision(ctx->revs)) != NULL) { /* * an uninteresting boundary commit may not have its tree * parsed yet, but we are not going to show them anyway */ if (get_commit_tree(commit)) - add_pending_tree(revs, get_commit_tree(commit)); - show_commit(commit, show_data); + add_pending_tree(ctx->revs, get_commit_tree(commit)); + ctx->show_commit(commit, ctx->show_data); - if (revs->tree_blobs_in_commit_order) + if (ctx->revs->tree_blobs_in_commit_order) /* * NEEDSWORK: Adding the tree and then flushing it here * needs a reallocation for each commit. Can we pass the * tree directory without allocation churn? */ - traverse_trees_and_blobs(revs, &csp, - show_object, show_data, - filter_fn, filter_data); + traverse_trees_and_blobs(ctx, &csp); } - traverse_trees_and_blobs(revs, &csp, - show_object, show_data, - filter_fn, filter_data); + traverse_trees_and_blobs(ctx, &csp); strbuf_release(&csp); } @@ -326,7 +305,14 @@ void traverse_commit_list(struct rev_info *revs, show_object_fn show_object, void *show_data) { - do_traverse(revs, show_commit, show_object, show_data, NULL, NULL); + struct traversal_context ctx; + ctx.revs = revs; + ctx.show_commit = show_commit; + ctx.show_object = show_object; + ctx.show_data = show_data; + ctx.filter_fn = NULL; + ctx.filter_data = NULL; + do_traverse(&ctx); } void traverse_commit_list_filtered( @@ -337,14 +323,18 @@ void traverse_commit_list_filtered( void *show_data, struct oidset *omitted) { - filter_object_fn filter_fn = NULL; + struct traversal_context ctx; filter_free_fn filter_free_fn = NULL; - void *filter_data = NULL; - - filter_data = list_objects_filter__init(omitted, filter_options, - &filter_fn, &filter_free_fn); - do_traverse(revs, show_commit, show_object, show_data, - filter_fn, filter_data); - if (filter_data && filter_free_fn) - filter_free_fn(filter_data); + + ctx.revs = revs; + ctx.show_object = show_object; + ctx.show_commit = show_commit; + ctx.show_data = show_data; + ctx.filter_fn = NULL; + + ctx.filter_data = list_objects_filter__init(omitted, filter_options, + &ctx.filter_fn, &filter_free_fn); + do_traverse(&ctx); + if (ctx.filter_data && filter_free_fn) + filter_free_fn(ctx.filter_data); } From patchwork Fri Sep 14 00:55:22 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew DeVore X-Patchwork-Id: 10600225 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F321514BD for ; Fri, 14 Sep 2018 00:56:07 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 651A32B307 for ; Fri, 14 Sep 2018 00:56:07 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 597162B34C; Fri, 14 Sep 2018 00:56:07 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EC0F62B34C for ; Fri, 14 Sep 2018 00:56:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727383AbeINGH6 (ORCPT ); Fri, 14 Sep 2018 02:07:58 -0400 Received: from mail-io1-f74.google.com ([209.85.166.74]:37202 "EHLO mail-io1-f74.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726558AbeINGH6 (ORCPT ); Fri, 14 Sep 2018 02:07:58 -0400 Received: by mail-io1-f74.google.com with SMTP id l6-v6so6656452iog.4 for ; Thu, 13 Sep 2018 17:56:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=IWMtfmkF75GsybnM11356Qak2u2LcOOZFz9T3sTmvEw=; b=GighBETpxnQq1oo0A4XCHSOvXRt9TF6NpnGIe4WcvpnTWBc6EbLfVBB4ZMfEak6KQQ BVg1DP9WXZqX8mjT/DE+K/gRRLuFp0Q/jZ9ll3hSOLFdoFJZlRMlSBfZO2ff4a0jrvKq MLyQND/YbTcjvAk2nDGpZxREEPcCJSiC7HBkulfZumSErwDHnh5EZOk60jHwrKR5i6Xn 13qWv8cdNzAxTwikmt2jVz8udcnwTesGLfW0OFQMgjt/oWe1KncibSAljq4lpdBqIbj+ GBVLxyz7D3FpQndjibsZS8sq7PWFfv4AGiy/ZHTYwCe2Uo+fk9Emj8tS1/0ux19kr/XS 1A3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=IWMtfmkF75GsybnM11356Qak2u2LcOOZFz9T3sTmvEw=; b=ZSyr0TGNMNT5CzGQl7zOYkeE1zdC0TYD83GHJyfhMqAxPHJfbXoV+ESM32OlAESlu/ CHU+cPKAGRJxt5ZaJB94WYppZB1qJnLntkPmGD8SKL3CVSg7UiyaQKei+mNng3I6JqS5 Icb/AR9cGmqHGDsPPwMfADGa6O3gJw6+9yKm4z0htAa/EGUbkcGwgcAWuAVvZFEVcDfL 4vWDwlqECyQIfQEoWv6LGKomVKDiboNR2ObmmLnS5K6zY0lDsdpB2Gi3hWpnJb6e1Kfu kdhE8eBcmyzspiZJpglWP61+SdMS9Oe1h+aqYSX3GhE9blMSeyLTJNGWNx5vkGivLF6w iz3Q== X-Gm-Message-State: APzg51DAdubx/nVTJqfTDWzMCpOzWhUwG0x/4V03bBradCH4ew/ItXM1 P7C0KB7q94wS/W8hVfc96YLYEW8bDAs= X-Google-Smtp-Source: ANB0VdYWle6z0JiPed/xVxGBjOsTzN79QvbmqbE+h7NEfCdvra6eflKpscBl/rTCXAMM6U3x9jz3wXQBqTNt X-Received: by 2002:a6b:ac45:: with SMTP id v66-v6mr7307127ioe.101.1536886562565; Thu, 13 Sep 2018 17:56:02 -0700 (PDT) Date: Thu, 13 Sep 2018 17:55:22 -0700 In-Reply-To: Message-Id: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.19.0.397.gdd90340f6a-goog Subject: [PATCH v8 2/7] list-objects: refactor to process_tree_contents From: Matthew DeVore To: sbeller@google.com, git@vger.kernel.org Cc: Matthew DeVore , git@jeffhostetler.com, jeffhost@microsoft.com, peff@peff.net, stefanbeller@gmail.com, jonathantanmy@google.com, gitster@pobox.com, pclouds@gmail.com Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This will be used in a follow-up patch to reduce indentation needed when invoking the logic conditionally. i.e. rather than: if (foo) { while (...) { /* this is very indented */ } } we will have: if (foo) process_tree_contents(...); Signed-off-by: Matthew DeVore --- list-objects.c | 68 ++++++++++++++++++++++++++++++-------------------- 1 file changed, 41 insertions(+), 27 deletions(-) diff --git a/list-objects.c b/list-objects.c index 584518a3f..ccc529e5e 100644 --- a/list-objects.c +++ b/list-objects.c @@ -94,6 +94,46 @@ static void process_gitlink(struct traversal_context *ctx, /* Nothing to do */ } +static void process_tree(struct traversal_context *ctx, + struct tree *tree, + struct strbuf *base, + const char *name); + +static void process_tree_contents(struct traversal_context *ctx, + struct tree *tree, + struct strbuf *base) +{ + struct tree_desc desc; + struct name_entry entry; + enum interesting match = ctx->revs->diffopt.pathspec.nr == 0 ? + all_entries_interesting : entry_not_interesting; + + init_tree_desc(&desc, tree->buffer, tree->size); + + while (tree_entry(&desc, &entry)) { + if (match != all_entries_interesting) { + match = tree_entry_interesting(&entry, base, 0, + &ctx->revs->diffopt.pathspec); + if (match == all_entries_not_interesting) + break; + if (match == entry_not_interesting) + continue; + } + + if (S_ISDIR(entry.mode)) + process_tree(ctx, + lookup_tree(the_repository, entry.oid), + base, entry.path); + else if (S_ISGITLINK(entry.mode)) + process_gitlink(ctx, entry.oid->hash, + base, entry.path); + else + process_blob(ctx, + lookup_blob(the_repository, entry.oid), + base, entry.path); + } +} + static void process_tree(struct traversal_context *ctx, struct tree *tree, struct strbuf *base, @@ -101,10 +141,6 @@ static void process_tree(struct traversal_context *ctx, { struct object *obj = &tree->object; struct rev_info *revs = ctx->revs; - struct tree_desc desc; - struct name_entry entry; - enum interesting match = revs->diffopt.pathspec.nr == 0 ? - all_entries_interesting: entry_not_interesting; int baselen = base->len; enum list_objects_filter_result r = LOFR_MARK_SEEN | LOFR_DO_SHOW; int gently = revs->ignore_missing_links || @@ -144,29 +180,7 @@ static void process_tree(struct traversal_context *ctx, if (base->len) strbuf_addch(base, '/'); - init_tree_desc(&desc, tree->buffer, tree->size); - - while (tree_entry(&desc, &entry)) { - if (match != all_entries_interesting) { - match = tree_entry_interesting(&entry, base, 0, - &revs->diffopt.pathspec); - if (match == all_entries_not_interesting) - break; - if (match == entry_not_interesting) - continue; - } - - if (S_ISDIR(entry.mode)) - process_tree(ctx, - lookup_tree(the_repository, entry.oid), - base, entry.path); - else if (S_ISGITLINK(entry.mode)) - process_gitlink(ctx, entry.oid->hash, base, entry.path); - else - process_blob(ctx, - lookup_blob(the_repository, entry.oid), - base, entry.path); - } + process_tree_contents(ctx, tree, base); if (!(obj->flags & USER_GIVEN) && ctx->filter_fn) { r = ctx->filter_fn(LOFS_END_TREE, obj, From patchwork Fri Sep 14 00:55:23 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew DeVore X-Patchwork-Id: 10600227 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CF62914DA for ; Fri, 14 Sep 2018 00:56:15 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3E4402B307 for ; Fri, 14 Sep 2018 00:56:15 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 308E92B34E; Fri, 14 Sep 2018 00:56:15 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CDF4E2B307 for ; Fri, 14 Sep 2018 00:56:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728197AbeINGIH (ORCPT ); Fri, 14 Sep 2018 02:08:07 -0400 Received: from mail-ua1-f73.google.com ([209.85.222.73]:33421 "EHLO mail-ua1-f73.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726558AbeINGIH (ORCPT ); Fri, 14 Sep 2018 02:08:07 -0400 Received: by mail-ua1-f73.google.com with SMTP id z12-v6so2601595uao.0 for ; Thu, 13 Sep 2018 17:56:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=tFOkoJrX1T+lw4BO0Jcp4q4CDKJpXdSkWCIbJ2xMgZA=; b=nYH8iCRXGznd5PCw37cpUMjccdVdhvBRzG7va5YoZnZcANlTreum8/3R/0KYhXNu/y DDaQhWzm0gJYl0s1qxnIy3nS5+vIOrKTdi8mGMig7bEfsUfhRoynoC8yfzLC1wqAd3HC VzQ0HHwskh32QXHFJSEo88Jf7QawryAa4BS431Mdy/JSfJYsJk8tlJPDrC54FIiqpoZF H4Xc5bR5cfrWxF8GitiJWDLastujHs6WEKrp7wG/fSVT3HorORkrkHSMkEN+1n3Bglhl t+g5wOATdmrfYMfza4p4nYzQ2myjZIUIjnDBd++a1hV/gzugL98vacJzhPQLJLrqG0Fs S10A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=tFOkoJrX1T+lw4BO0Jcp4q4CDKJpXdSkWCIbJ2xMgZA=; b=UP9DUJsDu2bbi/1b2HpouV9VZBGHc0JfmKTGqqKk7p+iaEptladYer1vgCiUlsek9L uh6AkQXn3jZ2HDq5vr5YNXJBnLBYo9TWs9AhrrFGAnXS/OCtzeyRHLoKOecBYBqUU8/C jOP1vrf4OglGcCTSnrJmq+K86Q7MqnKUovV5meM9SH1Rgz5xbk0b5QmCyarQ/c5ZKImP uuagZCjMryoHGuInE8pw09sIF08flWIVSo10EP50WLLCWgnr8ioZCwBvhlzGMelBIeCL ibhdhDafwwiWMO6OjbmWOkIfXfK0hJkgoY4dV9INPi8q/2BJBixSXINESTbrGtJJnNPx McrA== X-Gm-Message-State: APzg51Bwl/tQkuNIb0i1QdL2za4rvv4amiFB48UXym4UGKX1nZaycGlg SJbYW6k3uLQDEY8SxKj1q4EPFJVZPKI= X-Google-Smtp-Source: ANB0VdY9ixqVRpdzB3KZOI2ynxNNd5x85Ix3v80mSQx+SW6YDBZFR5G8HHNHs6MccvlfoKHRS2Z6v7NKpuCy X-Received: by 2002:a1f:d5:: with SMTP id 204-v6mr1656725vka.3.1536886571311; Thu, 13 Sep 2018 17:56:11 -0700 (PDT) Date: Thu, 13 Sep 2018 17:55:23 -0700 In-Reply-To: Message-Id: <28f5ce6d48a69fd7326ce6a6995e5567a6f32d62.1536885967.git.matvore@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.19.0.397.gdd90340f6a-goog Subject: [PATCH v8 3/7] list-objects: always parse trees gently From: Matthew DeVore To: sbeller@google.com, git@vger.kernel.org Cc: Matthew DeVore , git@jeffhostetler.com, jeffhost@microsoft.com, peff@peff.net, stefanbeller@gmail.com, jonathantanmy@google.com, gitster@pobox.com, pclouds@gmail.com Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP If parsing fails when revs->ignore_missing_links and revs->exclude_promisor_objects are both false, we print the OID anyway in the die("bad tree object...") call, so any message printed by parse_tree_gently() is superfluous. Signed-off-by: Matthew DeVore --- list-objects.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/list-objects.c b/list-objects.c index ccc529e5e..f9b51db7a 100644 --- a/list-objects.c +++ b/list-objects.c @@ -143,8 +143,6 @@ static void process_tree(struct traversal_context *ctx, struct rev_info *revs = ctx->revs; int baselen = base->len; enum list_objects_filter_result r = LOFR_MARK_SEEN | LOFR_DO_SHOW; - int gently = revs->ignore_missing_links || - revs->exclude_promisor_objects; if (!revs->tree_objects) return; @@ -152,7 +150,7 @@ static void process_tree(struct traversal_context *ctx, die("bad tree object"); if (obj->flags & (UNINTERESTING | SEEN)) return; - if (parse_tree_gently(tree, gently) < 0) { + if (parse_tree_gently(tree, 1) < 0) { if (revs->ignore_missing_links) return; From patchwork Fri Sep 14 00:55:24 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew DeVore X-Patchwork-Id: 10600229 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 50E3D14DA for ; Fri, 14 Sep 2018 00:56:24 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B62F12B307 for ; Fri, 14 Sep 2018 00:56:23 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AA8D42B388; Fri, 14 Sep 2018 00:56:23 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BEE412B307 for ; Fri, 14 Sep 2018 00:56:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728200AbeINGIQ (ORCPT ); Fri, 14 Sep 2018 02:08:16 -0400 Received: from mail-it0-f73.google.com ([209.85.214.73]:58986 "EHLO mail-it0-f73.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726558AbeINGIP (ORCPT ); Fri, 14 Sep 2018 02:08:15 -0400 Received: by mail-it0-f73.google.com with SMTP id x15-v6so555364ite.8 for ; Thu, 13 Sep 2018 17:56:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=/AV/kjzwfehD0Rd2bVjDVcpNNejzFvvcj3OUfJdFQvk=; b=UDNMTJ5GqLq2mym8ibXiC1Xz6zGVztLVGWbsOKBxzdC3iy6oH5uTRQGl9MIHFlTEQk 42wcxTuoL5zqVo1o0f++IPvf8BcRHV6QuwV/0irWcVpn0/YLM7676ASjQHjDtsHsgobL d9QL6ddPOj02S9HoUKVupBLeWuNvdhvl4StHnQjo+alFLPHc2vgX8b6S4bDxiF/0fP1T 36IVrtLoxRJsb9MRtkKC+IiBV9Q6N4A7Ba+ZVknRqzYTNMs+1WNOKobbCuvN5TjqrT1c pRa5r/qh5YbJBd8zcW0bHTOyUwy59u4f1QHInQVTDaTRgczEssHOR31xOOmwz1mgcOEo eV4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=/AV/kjzwfehD0Rd2bVjDVcpNNejzFvvcj3OUfJdFQvk=; b=Pl6pc7iaKDNEPOsnMxRKI2mtzXzOzUGjuho/Z9sfJBpudA4b5fC1udBaV04jeRR0L8 PFjHj3X2tZlTPRPFAKeYXs1CFGXkO+hCvvKs8QXwJKkUCN2+jTC/0tDl8rBKgND2PefF zi6uPvcRWC5aQ/E9Yde3TYLu4Q3ib+8dkhvoZ3YCW667iofQ0eG+Vj0nnZYb3A50aEP9 jQN21XZo/0MbFWjIqUQ+CyjC5Wst43pfthnYvRp4uvHOJpuW4ASniZIsFpWi4fGzFoNp nNv+S4DxlFNo5lDT+nVLYHHvwmuYz/UpKaEs4wUYyNKifJf3H7AO0BJSBQNcRRneZaGM ADww== X-Gm-Message-State: APzg51DeD89uGa1lGIDyM+0DRpCxE0/Lai0V5rh9wxWicSPDB1zYie3r AFw1409BF4qCwFSqMRoPgK9iHX1/UX4= X-Google-Smtp-Source: ANB0Vda65awoOctS5WGsuF6dq++tI9TxJ2gfy1URouR7bevDcLgNzNfv1ElTCyIQ2kkjT+gFEXkEeygmTe+v X-Received: by 2002:a24:68c6:: with SMTP id v189-v6mr437966itb.11.1536886579399; Thu, 13 Sep 2018 17:56:19 -0700 (PDT) Date: Thu, 13 Sep 2018 17:55:24 -0700 In-Reply-To: Message-Id: <9216afb6305ff2efd62ca056973d7548db9e00a0.1536885967.git.matvore@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.19.0.397.gdd90340f6a-goog Subject: [PATCH v8 4/7] rev-list: handle missing tree objects properly From: Matthew DeVore To: sbeller@google.com, git@vger.kernel.org Cc: Matthew DeVore , git@jeffhostetler.com, jeffhost@microsoft.com, peff@peff.net, stefanbeller@gmail.com, jonathantanmy@google.com, gitster@pobox.com, pclouds@gmail.com Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Previously, we assumed only blob objects could be missing. This patch makes rev-list handle missing trees like missing blobs. The --missing=* and --exclude-promisor-objects flags now work for trees as they already do for blobs. This is demonstrated in t6112. Signed-off-by: Matthew DeVore --- builtin/rev-list.c | 11 ++++--- list-objects.c | 11 +++++-- revision.h | 15 +++++++++ t/t0410-partial-clone.sh | 45 ++++++++++++++++++++++++++ t/t5317-pack-objects-filter-objects.sh | 13 ++++++++ t/t6112-rev-list-filters-objects.sh | 17 ++++++++++ 6 files changed, 105 insertions(+), 7 deletions(-) diff --git a/builtin/rev-list.c b/builtin/rev-list.c index 5b07f3f4a..49d6deed7 100644 --- a/builtin/rev-list.c +++ b/builtin/rev-list.c @@ -6,6 +6,7 @@ #include "list-objects.h" #include "list-objects-filter.h" #include "list-objects-filter-options.h" +#include "object.h" #include "object-store.h" #include "pack.h" #include "pack-bitmap.h" @@ -209,7 +210,8 @@ static inline void finish_object__ma(struct object *obj) */ switch (arg_missing_action) { case MA_ERROR: - die("missing blob object '%s'", oid_to_hex(&obj->oid)); + die("missing %s object '%s'", + type_name(obj->type), oid_to_hex(&obj->oid)); return; case MA_ALLOW_ANY: @@ -222,8 +224,8 @@ static inline void finish_object__ma(struct object *obj) case MA_ALLOW_PROMISOR: if (is_promisor_object(&obj->oid)) return; - die("unexpected missing blob object '%s'", - oid_to_hex(&obj->oid)); + die("unexpected missing %s object '%s'", + type_name(obj->type), oid_to_hex(&obj->oid)); return; default: @@ -235,7 +237,7 @@ static inline void finish_object__ma(struct object *obj) static int finish_object(struct object *obj, const char *name, void *cb_data) { struct rev_list_info *info = cb_data; - if (obj->type == OBJ_BLOB && !has_object_file(&obj->oid)) { + if (!has_object_file(&obj->oid)) { finish_object__ma(obj); return 1; } @@ -373,6 +375,7 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix) init_revisions(&revs, prefix); revs.abbrev = DEFAULT_ABBREV; revs.commit_format = CMIT_FMT_UNSPECIFIED; + revs.do_not_die_on_missing_tree = 1; /* * Scan the argument list before invoking setup_revisions(), so that we diff --git a/list-objects.c b/list-objects.c index f9b51db7a..243192af5 100644 --- a/list-objects.c +++ b/list-objects.c @@ -143,6 +143,7 @@ static void process_tree(struct traversal_context *ctx, struct rev_info *revs = ctx->revs; int baselen = base->len; enum list_objects_filter_result r = LOFR_MARK_SEEN | LOFR_DO_SHOW; + int failed_parse; if (!revs->tree_objects) return; @@ -150,7 +151,9 @@ static void process_tree(struct traversal_context *ctx, die("bad tree object"); if (obj->flags & (UNINTERESTING | SEEN)) return; - if (parse_tree_gently(tree, 1) < 0) { + + failed_parse = parse_tree_gently(tree, 1); + if (failed_parse) { if (revs->ignore_missing_links) return; @@ -163,7 +166,8 @@ static void process_tree(struct traversal_context *ctx, is_promisor_object(&obj->oid)) return; - die("bad tree object %s", oid_to_hex(&obj->oid)); + if (!revs->do_not_die_on_missing_tree) + die("bad tree object %s", oid_to_hex(&obj->oid)); } strbuf_addstr(base, name); @@ -178,7 +182,8 @@ static void process_tree(struct traversal_context *ctx, if (base->len) strbuf_addch(base, '/'); - process_tree_contents(ctx, tree, base); + if (!failed_parse) + process_tree_contents(ctx, tree, base); if (!(obj->flags & USER_GIVEN) && ctx->filter_fn) { r = ctx->filter_fn(LOFS_END_TREE, obj, diff --git a/revision.h b/revision.h index 007278cc1..5910613cb 100644 --- a/revision.h +++ b/revision.h @@ -126,6 +126,21 @@ struct rev_info { line_level_traverse:1, tree_blobs_in_commit_order:1, + /* + * Blobs are shown without regard for their existence. + * But not so for trees: unless exclude_promisor_objects + * is set and the tree in question is a promisor object; + * OR ignore_missing_links is set, the revision walker + * dies with a "bad tree object HASH" message when + * encountering a missing tree. For callers that can + * handle missing trees and want them to be filterable + * and showable, set this to true. The revision walker + * will filter and show such a missing tree as usual, + * but will not attempt to recurse into this tree + * object. + */ + do_not_die_on_missing_tree:1, + /* for internal use only */ exclude_promisor_objects:1; diff --git a/t/t0410-partial-clone.sh b/t/t0410-partial-clone.sh index 128130066..f02b9ae37 100755 --- a/t/t0410-partial-clone.sh +++ b/t/t0410-partial-clone.sh @@ -186,6 +186,51 @@ test_expect_success 'rev-list stops traversal at missing and promised commit' ' ! grep $FOO out ' +test_expect_success 'missing tree objects with --missing=allow-promisor and --exclude-promisor-objects' ' + rm -rf repo && + test_create_repo repo && + test_commit -C repo foo && + test_commit -C repo bar && + test_commit -C repo baz && + + promise_and_delete $(git -C repo rev-parse bar^{tree}) && + promise_and_delete $(git -C repo rev-parse foo^{tree}) && + + git -C repo config core.repositoryformatversion 1 && + git -C repo config extensions.partialclone "arbitrary string" && + + git -C repo rev-list --missing=allow-promisor --objects HEAD >objs 2>rev_list_err && + test_must_be_empty rev_list_err && + # 3 commits, 3 blobs, and 1 tree + test_line_count = 7 objs && + + # Do the same for --exclude-promisor-objects, but with all trees gone. + promise_and_delete $(git -C repo rev-parse baz^{tree}) && + git -C repo rev-list --exclude-promisor-objects --objects HEAD >objs 2>rev_list_err && + test_must_be_empty rev_list_err && + # 3 commits, no blobs or trees + test_line_count = 3 objs +' + +test_expect_success 'missing non-root tree object and rev-list' ' + rm -rf repo && + test_create_repo repo && + mkdir repo/dir && + echo foo > repo/dir/foo && + git -C repo add dir/foo && + git -C repo commit -m "commit dir/foo" && + + promise_and_delete $(git -C repo rev-parse HEAD:dir) && + + git -C repo config core.repositoryformatversion 1 && + git -C repo config extensions.partialclone "arbitrary string" && + + git -C repo rev-list --missing=allow-any --objects HEAD >objs 2>rev_list_err && + test_must_be_empty rev_list_err && + # 1 commit and 1 tree + test_line_count = 2 objs +' + test_expect_success 'rev-list stops traversal at missing and promised tree' ' rm -rf repo && test_create_repo repo && diff --git a/t/t5317-pack-objects-filter-objects.sh b/t/t5317-pack-objects-filter-objects.sh index 6710c8bc8..5e35f33bf 100755 --- a/t/t5317-pack-objects-filter-objects.sh +++ b/t/t5317-pack-objects-filter-objects.sh @@ -59,6 +59,19 @@ test_expect_success 'verify normal and blob:none packfiles have same commits/tre test_cmp observed expected ' +test_expect_success 'get an error for missing tree object' ' + git init r5 && + echo foo > r5/foo && + git -C r5 add foo && + git -C r5 commit -m "foo" && + del=$(git -C r5 rev-parse HEAD^{tree} | sed "s|..|&/|") && + rm r5/.git/objects/$del && + test_must_fail git -C r5 pack-objects --rev --stdout 2>bad_tree <<-EOF && + HEAD + EOF + grep -q "bad tree object" bad_tree +' + # Test blob:limit=[kmg] filter. # We boundary test around the size parameter. The filter is strictly less than # the value, so size 500 and 1000 should have the same results, but 1001 should diff --git a/t/t6112-rev-list-filters-objects.sh b/t/t6112-rev-list-filters-objects.sh index d4ff0b3be..c662c97db 100755 --- a/t/t6112-rev-list-filters-objects.sh +++ b/t/t6112-rev-list-filters-objects.sh @@ -195,6 +195,23 @@ test_expect_success 'verify sparse:oid=oid-ish omits top-level files' ' test_cmp observed expected ' +test_expect_success 'rev-list W/ --missing=print and --missing=allow-any for trees' ' + TREE=$(git -C r3 rev-parse HEAD:dir1) && + + rm r3/.git/objects/$(echo $TREE | sed "s|^..|&/|") && + + git -C r3 rev-list --quiet --missing=print --objects HEAD >missing_objs 2>rev_list_err && + echo "?$TREE" >expected && + test_cmp expected missing_objs && + + # do not complain when a missing tree cannot be parsed + test_must_be_empty rev_list_err && + + git -C r3 rev-list --missing=allow-any --objects HEAD >objs 2>rev_list_err && + ! grep $TREE objs && + test_must_be_empty rev_list_err +' + # Delete some loose objects and use rev-list, but WITHOUT any filtering. # This models previously omitted objects that we did not receive. From patchwork Fri Sep 14 00:55:25 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew DeVore X-Patchwork-Id: 10600231 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 01AA814BD for ; Fri, 14 Sep 2018 00:56:32 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 665452B34C for ; Fri, 14 Sep 2018 00:56:31 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 591772B388; Fri, 14 Sep 2018 00:56:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BCE2E2B34C for ; Fri, 14 Sep 2018 00:56:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728270AbeINGIY (ORCPT ); Fri, 14 Sep 2018 02:08:24 -0400 Received: from mail-it0-f73.google.com ([209.85.214.73]:54561 "EHLO mail-it0-f73.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726558AbeINGIY (ORCPT ); Fri, 14 Sep 2018 02:08:24 -0400 Received: by mail-it0-f73.google.com with SMTP id r19-v6so578420itc.4 for ; Thu, 13 Sep 2018 17:56:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=ZwklA75rQ7BrudwMDD+19SdRV7q4xsjNfmcq1CPWApo=; b=rV9xKBz8MwADDOmaCwd6krMkv602iMfyxNK9iDRjAY/kGoh4F97myIO5XU19WtWZE1 SzeAnG9zz8+6+B7nxO857PMFplAdU5IXKaye4pm+TXrcAUrqV9WVm/zVQ3czF0o/n2oQ UVxzQKpZtC+Yi2qd1UyudLwIx4iQSnU98dss+d4+ALaMpvwBD2Xd5JXrgWQCiVYRx9oe Fe6YsCB7dZv4EoZAMUxzbsUjvRc6ukwlRdHjCdcHoF2qWAG8oA5U0+3ggIlkX4l4D3Df 76AqkOVgoxoHlrr0wXp8okXuYGv+6Bo0KIFVWd/7CFWPVrCYvC8NrBt2r3rSDXHEu7Hk vdhQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=ZwklA75rQ7BrudwMDD+19SdRV7q4xsjNfmcq1CPWApo=; b=puqoRfVt/g7ltbaE9uA32LvWzAQQYbhDQDuszY+zpRL7LaDeSdH0p87Arqbspq6LJd 2okpchIsHJdGf/yfzvY3/Y6PNsdda1O1C6jVD54N05y6S2dDsLjqcGo6HvNYAjI4iUCF HRfHqsmtETzt+l0FQNMxcQK/fGaf7XbIwvDJrUu2Y2r3GnIKJy7AWWr3cK08CY8V/hrZ N6MRZLbG7RaUluCxbibbnTss/iL85VKpsJW3YaGuvrO806UcpaZ0c+cpTxHn9qmVCKgU 9PNS9LxrzJn2tVCN+pacajDmZERhG1P5gNA8qjLN9WdcwnQ+5PrCIXISa9vIPyThTTrH PBGQ== X-Gm-Message-State: APzg51DHuvOIolvOOpgDRFCe3EWpt5nYOIyZ67qFXcTzk7SE8gFuUGmz vb/SmRA59dvpldk+yDSD1UEnDhgj6rE= X-Google-Smtp-Source: ANB0VdZ1RvlglYPckR7minf004tU2WsZu1qmMpe24Eire+Jy6AOlCODaq6F0jF1gjkqez68mUwgnIp1DkAgu X-Received: by 2002:a24:eec8:: with SMTP id b191-v6mr409347iti.42.1536886587725; Thu, 13 Sep 2018 17:56:27 -0700 (PDT) Date: Thu, 13 Sep 2018 17:55:25 -0700 In-Reply-To: Message-Id: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.19.0.397.gdd90340f6a-goog Subject: [PATCH v8 5/7] revision: mark non-user-given objects instead From: Matthew DeVore To: sbeller@google.com, git@vger.kernel.org Cc: Matthew DeVore , git@jeffhostetler.com, jeffhost@microsoft.com, peff@peff.net, stefanbeller@gmail.com, jonathantanmy@google.com, gitster@pobox.com, pclouds@gmail.com Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Currently, list-objects.c incorrectly treats all root trees of commits as USER_GIVEN. Also, it would be easier to mark objects that are non-user-given instead of user-given, since the places in the code where we access an object through a reference are more obvious than the places where we access an object that was given by the user. Resolve these two problems by introducing a flag NOT_USER_GIVEN that marks blobs and trees that are non-user-given, replacing USER_GIVEN. (Only blobs and trees are marked because this mark is only used when filtering objects, and filtering of other types of objects is not supported yet.) This fixes a bug in that git rev-list behaved differently from git pack-objects. pack-objects would *not* filter objects given explicitly on the command line and rev-list would filter. This was because the two commands used a different function to add objects to the rev_info struct. This seems to have been an oversight, and pack-objects has the correct behavior, so I added a test to make sure that rev-list now behaves properly. Signed-off-by: Matthew DeVore fixup of 6defd70de --- list-objects.c | 31 +++++++++++++++++------------ revision.c | 1 - revision.h | 11 ++++++++-- t/t6112-rev-list-filters-objects.sh | 10 ++++++++++ 4 files changed, 37 insertions(+), 16 deletions(-) diff --git a/list-objects.c b/list-objects.c index 243192af5..7a1a0929d 100644 --- a/list-objects.c +++ b/list-objects.c @@ -53,7 +53,7 @@ static void process_blob(struct traversal_context *ctx, pathlen = path->len; strbuf_addstr(path, name); - if (!(obj->flags & USER_GIVEN) && ctx->filter_fn) + if ((obj->flags & NOT_USER_GIVEN) && ctx->filter_fn) r = ctx->filter_fn(LOFS_BLOB, obj, path->buf, &path->buf[pathlen], ctx->filter_data); @@ -120,17 +120,19 @@ static void process_tree_contents(struct traversal_context *ctx, continue; } - if (S_ISDIR(entry.mode)) - process_tree(ctx, - lookup_tree(the_repository, entry.oid), - base, entry.path); + if (S_ISDIR(entry.mode)) { + struct tree *t = lookup_tree(the_repository, entry.oid); + t->object.flags |= NOT_USER_GIVEN; + process_tree(ctx, t, base, entry.path); + } else if (S_ISGITLINK(entry.mode)) process_gitlink(ctx, entry.oid->hash, base, entry.path); - else - process_blob(ctx, - lookup_blob(the_repository, entry.oid), - base, entry.path); + else { + struct blob *b = lookup_blob(the_repository, entry.oid); + b->object.flags |= NOT_USER_GIVEN; + process_blob(ctx, b, base, entry.path); + } } } @@ -171,7 +173,7 @@ static void process_tree(struct traversal_context *ctx, } strbuf_addstr(base, name); - if (!(obj->flags & USER_GIVEN) && ctx->filter_fn) + if ((obj->flags & NOT_USER_GIVEN) && ctx->filter_fn) r = ctx->filter_fn(LOFS_BEGIN_TREE, obj, base->buf, &base->buf[baselen], ctx->filter_data); @@ -185,7 +187,7 @@ static void process_tree(struct traversal_context *ctx, if (!failed_parse) process_tree_contents(ctx, tree, base); - if (!(obj->flags & USER_GIVEN) && ctx->filter_fn) { + if ((obj->flags & NOT_USER_GIVEN) && ctx->filter_fn) { r = ctx->filter_fn(LOFS_END_TREE, obj, base->buf, &base->buf[baselen], ctx->filter_data); @@ -301,8 +303,11 @@ static void do_traverse(struct traversal_context *ctx) * an uninteresting boundary commit may not have its tree * parsed yet, but we are not going to show them anyway */ - if (get_commit_tree(commit)) - add_pending_tree(ctx->revs, get_commit_tree(commit)); + if (get_commit_tree(commit)) { + struct tree *tree = get_commit_tree(commit); + tree->object.flags |= NOT_USER_GIVEN; + add_pending_tree(ctx->revs, tree); + } ctx->show_commit(commit, ctx->show_data); if (ctx->revs->tree_blobs_in_commit_order) diff --git a/revision.c b/revision.c index de4dce600..72d48a17f 100644 --- a/revision.c +++ b/revision.c @@ -175,7 +175,6 @@ static void add_pending_object_with_path(struct rev_info *revs, strbuf_release(&buf); return; /* do not add the commit itself */ } - obj->flags |= USER_GIVEN; add_object_array_with_path(obj, name, &revs->pending, mode, path); } diff --git a/revision.h b/revision.h index 5910613cb..83e164039 100644 --- a/revision.h +++ b/revision.h @@ -21,9 +21,16 @@ #define SYMMETRIC_LEFT (1u<<8) #define PATCHSAME (1u<<9) #define BOTTOM (1u<<10) -#define USER_GIVEN (1u<<25) /* given directly by the user */ +/* + * Indicates object was reached by traversal. i.e. not given by user on + * command-line or stdin. + * NEEDSWORK: NOT_USER_GIVEN doesn't apply to commits because we only support + * filtering trees and blobs, but it may be useful to support filtering commits + * in the future. + */ +#define NOT_USER_GIVEN (1u<<25) #define TRACK_LINEAR (1u<<26) -#define ALL_REV_FLAGS (((1u<<11)-1) | USER_GIVEN | TRACK_LINEAR) +#define ALL_REV_FLAGS (((1u<<11)-1) | NOT_USER_GIVEN | TRACK_LINEAR) #define DECORATE_SHORT_REFS 1 #define DECORATE_FULL_REFS 2 diff --git a/t/t6112-rev-list-filters-objects.sh b/t/t6112-rev-list-filters-objects.sh index c662c97db..2e07dadf0 100755 --- a/t/t6112-rev-list-filters-objects.sh +++ b/t/t6112-rev-list-filters-objects.sh @@ -30,6 +30,16 @@ test_expect_success 'verify blob:none omits all 5 blobs' ' test_cmp observed expected ' +test_expect_success 'specify blob explicitly prevents filtering' ' + file_3=$(git -C r1 ls-files -s file.3 \ + | awk -f print_2.awk) && + file_4=$(git -C r1 ls-files -s file.4 \ + | awk -f print_2.awk) && + git -C r1 rev-list HEAD --objects --filter=blob:none HEAD $file_3 >observed && + grep -q "$file_3" observed && + test_must_fail grep -q "$file_4" observed +' + test_expect_success 'verify emitted+omitted == all' ' git -C r1 rev-list HEAD --objects \ | awk -f print_1.awk \ From patchwork Fri Sep 14 00:55:26 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew DeVore X-Patchwork-Id: 10600233 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 56C8C14DA for ; Fri, 14 Sep 2018 00:56:39 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BDF062B34C for ; Fri, 14 Sep 2018 00:56:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B1FEE2B388; Fri, 14 Sep 2018 00:56:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5FE612B34C for ; Fri, 14 Sep 2018 00:56:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728287AbeINGIc (ORCPT ); Fri, 14 Sep 2018 02:08:32 -0400 Received: from mail-ua1-f74.google.com ([209.85.222.74]:54901 "EHLO mail-ua1-f74.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726558AbeINGIc (ORCPT ); Fri, 14 Sep 2018 02:08:32 -0400 Received: by mail-ua1-f74.google.com with SMTP id l14-v6so2579398uaf.21 for ; Thu, 13 Sep 2018 17:56:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=or8aEAWBIZHq8J0TD2TlhHIWogfIMG08lPBrb5j91Cw=; b=loTBpBHc4D1GTvtpl+lO7yy+7HTN2T5BXn31m+AtC6ce/zRg5YoNuQ/r7Is8ON6joV jxr9osv+WxvegtuJDgOdKNtwa6t8jTOne4qEDE3utEWGT48UCGu+YVJcTvBdY35YTd+h na0E9SZwsMORAz+GvXObgfqWXmvqmlzx3wmjkYSXHUtm0xpt8Mm5+vHzmSQqJqWnbh0P qWxuWpYQ6Lrw3LD65sYBsX4+DakkihpcSgw1Iv3x94ay8ZC+jqWkiQoFdkAsmPHgOpB8 OFA1WtpfxEjsxg0pR50/RLeWiJcA0FlYrySfkFeZ+rVZ98c9PyfV+Agw+vhbxvtLIVJa kQiA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=or8aEAWBIZHq8J0TD2TlhHIWogfIMG08lPBrb5j91Cw=; b=rbg/2Fif4c00wHSJTSiAgPPBiz8WDxnOYnweOvgGc0i1GjL1MUMXMr+F5mSJ2XB+f7 FMrZsMlDff5SDfgZJEleKQza1PFUJM59NVpo3OP8nBLCQfJs6D+Quh2kKyfp04fdoVcT nx3TBwq0tapv1503hlMnxnN/m+2y90AtKIN2rPe8JZ4alyNbukWpFudJKxZC3+xp4ecn X1s0H+b3HDKZIpMEK7dBuitxbexZbcU7wpVVXuqbvdtTGEMLBjtJ5IYR2CM3b5R6IW8m 7uutZL+jGzAJZIi2AXBlphtu2PS6sjedajjeEkt3rs58OLKuOlvGVf9evX/NlZmdq2Mf P5xQ== X-Gm-Message-State: APzg51BAw58qownHX2J7JWGqF4UiIxwx1HQ2do90bxZ9b/yEZQvnjvs8 x8xUX+xNOD0SySW1Kq1mxpYxyfpGhhM= X-Google-Smtp-Source: ANB0VdbxFCiIY35REUUSjGkLM+5EwlwY7xxZO5iErykU7JnLYroCRIDC7hbyWpN093905qaE0aNUaY0vtgDO X-Received: by 2002:a67:4689:: with SMTP id a9-v6mr1759203vsg.31.1536886595976; Thu, 13 Sep 2018 17:56:35 -0700 (PDT) Date: Thu, 13 Sep 2018 17:55:26 -0700 In-Reply-To: Message-Id: <0e0e7306625e6e88b7779e900fd04acb365e2f78.1536885967.git.matvore@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.19.0.397.gdd90340f6a-goog Subject: [PATCH v8 6/7] list-objects-filter: use BUG rather than die From: Matthew DeVore To: sbeller@google.com, git@vger.kernel.org Cc: Matthew DeVore , git@jeffhostetler.com, jeffhost@microsoft.com, peff@peff.net, stefanbeller@gmail.com, jonathantanmy@google.com, gitster@pobox.com, pclouds@gmail.com Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP In some cases in this file, BUG makes more sense than die. In such cases, a we get there from a coding error rather than a user error. 'return' has been removed following some instances of BUG since BUG does not return. Signed-off-by: Matthew DeVore --- list-objects-filter.c | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-) diff --git a/list-objects-filter.c b/list-objects-filter.c index a0ba78b20..5f8b1a002 100644 --- a/list-objects-filter.c +++ b/list-objects-filter.c @@ -44,8 +44,7 @@ static enum list_objects_filter_result filter_blobs_none( switch (filter_situation) { default: - die("unknown filter_situation"); - return LOFR_ZERO; + BUG("unknown filter_situation: %d", filter_situation); case LOFS_BEGIN_TREE: assert(obj->type == OBJ_TREE); @@ -102,8 +101,7 @@ static enum list_objects_filter_result filter_blobs_limit( switch (filter_situation) { default: - die("unknown filter_situation"); - return LOFR_ZERO; + BUG("unknown filter_situation: %d", filter_situation); case LOFS_BEGIN_TREE: assert(obj->type == OBJ_TREE); @@ -208,8 +206,7 @@ static enum list_objects_filter_result filter_sparse( switch (filter_situation) { default: - die("unknown filter_situation"); - return LOFR_ZERO; + BUG("unknown filter_situation: %d", filter_situation); case LOFS_BEGIN_TREE: assert(obj->type == OBJ_TREE); @@ -389,7 +386,7 @@ void *list_objects_filter__init( assert((sizeof(s_filters) / sizeof(s_filters[0])) == LOFC__COUNT); if (filter_options->choice >= LOFC__COUNT) - die("invalid list-objects filter choice: %d", + BUG("invalid list-objects filter choice: %d", filter_options->choice); init_fn = s_filters[filter_options->choice]; From patchwork Fri Sep 14 00:55:27 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew DeVore X-Patchwork-Id: 10600235 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 703D114DA for ; Fri, 14 Sep 2018 00:56:48 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D66642B388 for ; Fri, 14 Sep 2018 00:56:47 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C7B412B415; Fri, 14 Sep 2018 00:56:47 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id ECCA52B388 for ; Fri, 14 Sep 2018 00:56:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728291AbeINGIl (ORCPT ); Fri, 14 Sep 2018 02:08:41 -0400 Received: from mail-it0-f74.google.com ([209.85.214.74]:44148 "EHLO mail-it0-f74.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726558AbeINGIk (ORCPT ); Fri, 14 Sep 2018 02:08:40 -0400 Received: by mail-it0-f74.google.com with SMTP id b124-v6so589944itb.9 for ; Thu, 13 Sep 2018 17:56:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=ahTpd4ys5SFH+zIolvyLiyE/8KmQs84kbHJyUAgeQB0=; b=KOcW5GfkVwaeUWphwl0WkRj4+vjRyf1E2/II7y09pzxg2zwJxQXqT4KR2j1Cjyp1x3 nGsCM0c5ar39FzHOSaHEjfjqNGpjb6jQ/3H0M9J8QP4K8k3zxnqkwF2MirKqMzntZ0CG NeKc+R/C5oSUb49yRVV+sNjdVIYVioiTIqkKIxQzmglhAp0uft5Ugq9f4zdlepmn9by2 iO71+BQPMIZF6fnEK2uqyhgX1ielxUtdhQEw0BSXNDqNhkxKYveK+3LhSNIUp08hn4H+ 6XWZ9nq6+FqYOCVknWVWwNp541/RWKyoRKX6vGFkNk+pXXzXX6nYZ1HX4w7s8C+CiGF1 EyGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=ahTpd4ys5SFH+zIolvyLiyE/8KmQs84kbHJyUAgeQB0=; b=PgytpyDJEpifs9EPlYX9smr7zVqh4dl/+Jm9/8g0zVrT/cnZcIHjUwWXpcTrUIAt3F HgN5Tl3SqAlpZ7imdsDD9DNPl/ItVjNyo0KHkRwv94mfShIT5CK20jYz7kFz7i16OpYi dw/CagGP9D7YhDK1VzwJAy0AemqFEMaRHrOAKDQ6CYlXA8hUzoZQihW2yvLyFTH3UGmW Oq5n6JOuf8twvnHyzG0P3okMJe6lCYqNk+IjcYTZxwLBOYv9+bA5io5MR2VEgkuNSVWQ YhCJ5ZVqIrfqKNctCQ3PCgreMwTKlQVhc2OXkfQKH6+969MB9dBB1tFLXqH3xmpKEYfF o+cA== X-Gm-Message-State: APzg51BKfYj8f+6N+V0Bucjpfalo0Cr7cRcEVwBOrf64vblAEGwyG+m/ sQ1Zm5LUym3h+jLU9mu6E+lShkQrPKk= X-Google-Smtp-Source: ANB0VdadwBY52UE3pscBGFc0Eh1FeiGSrdzw0xN0XDyxMvXpDSEM7nWnbtSyJJTVcAI9X4zUC/HuCihekivx X-Received: by 2002:a24:7a51:: with SMTP id a78-v6mr458570itc.8.1536886604457; Thu, 13 Sep 2018 17:56:44 -0700 (PDT) Date: Thu, 13 Sep 2018 17:55:27 -0700 In-Reply-To: Message-Id: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.19.0.397.gdd90340f6a-goog Subject: [PATCH v8 7/7] list-objects-filter: implement filter tree:0 From: Matthew DeVore To: sbeller@google.com, git@vger.kernel.org Cc: Matthew DeVore , git@jeffhostetler.com, jeffhost@microsoft.com, peff@peff.net, stefanbeller@gmail.com, jonathantanmy@google.com, gitster@pobox.com, pclouds@gmail.com Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Teach list-objects the "tree:0" filter which allows for filtering out all tree and blob objects (unless other objects are explicitly specified by the user). The purpose of this patch is to allow smaller partial clones. The name of this filter - tree:0 - does not explicitly specify that it also filters out all blobs, but this should not cause much confusion because blobs are not at all useful without the trees that refer to them. I also considered only:commits as a name, but this is inaccurate because it suggests that annotated tags are omitted, but actually they are included. The name "tree:0" allows later filtering based on depth, i.e. "tree:1" would filter out all but the root tree and blobs. In order to avoid confusion between 0 and capital O, the documentation was worded in a somewhat round-about way that also hints at this future improvement to the feature. Signed-off-by: Matthew DeVore --- Documentation/rev-list-options.txt | 5 +++ list-objects-filter-options.c | 14 ++++++++ list-objects-filter-options.h | 1 + list-objects-filter.c | 49 ++++++++++++++++++++++++++ t/t5317-pack-objects-filter-objects.sh | 28 +++++++++++++++ t/t5616-partial-clone.sh | 38 ++++++++++++++++++++ t/t6112-rev-list-filters-objects.sh | 12 +++++++ 7 files changed, 147 insertions(+) diff --git a/Documentation/rev-list-options.txt b/Documentation/rev-list-options.txt index 7b273635d..5f1672913 100644 --- a/Documentation/rev-list-options.txt +++ b/Documentation/rev-list-options.txt @@ -731,6 +731,11 @@ the requested refs. + The form '--filter=sparse:path=' similarly uses a sparse-checkout specification contained in . ++ +The form '--filter=tree:' omits all blobs and trees whose depth +from the root tree is >= (minimum depth if an object is located +at multiple depths in the commits traversed). Currently, only =0 +is supported, which omits all blobs and trees. --no-filter:: Turn off any previous `--filter=` argument. diff --git a/list-objects-filter-options.c b/list-objects-filter-options.c index c0e2bd6a0..14f251de4 100644 --- a/list-objects-filter-options.c +++ b/list-objects-filter-options.c @@ -50,6 +50,20 @@ static int gently_parse_list_objects_filter( return 0; } + } else if (skip_prefix(arg, "tree:", &v0)) { + unsigned long depth; + if (!git_parse_ulong(v0, &depth) || depth != 0) { + if (errbuf) { + strbuf_init(errbuf, 0); + strbuf_addstr( + errbuf, + _("only 'tree:0' is supported")); + } + return 1; + } + filter_options->choice = LOFC_TREE_NONE; + return 0; + } else if (skip_prefix(arg, "sparse:oid=", &v0)) { struct object_context oc; struct object_id sparse_oid; diff --git a/list-objects-filter-options.h b/list-objects-filter-options.h index 0000a61f8..af64e5c66 100644 --- a/list-objects-filter-options.h +++ b/list-objects-filter-options.h @@ -10,6 +10,7 @@ enum list_objects_filter_choice { LOFC_DISABLED = 0, LOFC_BLOB_NONE, LOFC_BLOB_LIMIT, + LOFC_TREE_NONE, LOFC_SPARSE_OID, LOFC_SPARSE_PATH, LOFC__COUNT /* must be last */ diff --git a/list-objects-filter.c b/list-objects-filter.c index 5f8b1a002..09b2b05d5 100644 --- a/list-objects-filter.c +++ b/list-objects-filter.c @@ -79,6 +79,54 @@ static void *filter_blobs_none__init( return d; } +/* + * A filter for list-objects to omit ALL trees and blobs from the traversal. + * Can OPTIONALLY collect a list of the omitted OIDs. + */ +struct filter_trees_none_data { + struct oidset *omits; +}; + +static enum list_objects_filter_result filter_trees_none( + enum list_objects_filter_situation filter_situation, + struct object *obj, + const char *pathname, + const char *filename, + void *filter_data_) +{ + struct filter_trees_none_data *filter_data = filter_data_; + + switch (filter_situation) { + default: + BUG("unknown filter_situation: %d", filter_situation); + + case LOFS_BEGIN_TREE: + case LOFS_BLOB: + if (filter_data->omits) + oidset_insert(filter_data->omits, &obj->oid); + return LOFR_MARK_SEEN; /* but not LOFR_DO_SHOW (hard omit) */ + + case LOFS_END_TREE: + assert(obj->type == OBJ_TREE); + return LOFR_ZERO; + + } +} + +static void* filter_trees_none__init( + struct oidset *omitted, + struct list_objects_filter_options *filter_options, + filter_object_fn *filter_fn, + filter_free_fn *filter_free_fn) +{ + struct filter_trees_none_data *d = xcalloc(1, sizeof(*d)); + d->omits = omitted; + + *filter_fn = filter_trees_none; + *filter_free_fn = free; + return d; +} + /* * A filter for list-objects to omit large blobs. * And to OPTIONALLY collect a list of the omitted OIDs. @@ -371,6 +419,7 @@ static filter_init_fn s_filters[] = { NULL, filter_blobs_none__init, filter_blobs_limit__init, + filter_trees_none__init, filter_sparse_oid__init, filter_sparse_path__init, }; diff --git a/t/t5317-pack-objects-filter-objects.sh b/t/t5317-pack-objects-filter-objects.sh index 5e35f33bf..7a4d49ea1 100755 --- a/t/t5317-pack-objects-filter-objects.sh +++ b/t/t5317-pack-objects-filter-objects.sh @@ -72,6 +72,34 @@ test_expect_success 'get an error for missing tree object' ' grep -q "bad tree object" bad_tree ' +test_expect_success 'setup for tests of tree:0' ' + mkdir r1/subtree && + echo "This is a file in a subtree" >r1/subtree/file && + git -C r1 add subtree/file && + git -C r1 commit -m subtree +' + +test_expect_success 'verify tree:0 packfile has no blobs or trees' ' + git -C r1 pack-objects --rev --stdout --filter=tree:0 >commitsonly.pack <<-EOF && + HEAD + EOF + git -C r1 index-pack ../commitsonly.pack && + git -C r1 verify-pack -v ../commitsonly.pack >objs && + ! grep -E "tree|blob" objs +' + +test_expect_success 'grab tree directly when using tree:0' ' + # We should get the tree specified directly but not its blobs or subtrees. + git -C r1 pack-objects --rev --stdout --filter=tree:0 >commitsonly.pack <<-EOF && + HEAD: + EOF + git -C r1 index-pack ../commitsonly.pack && + git -C r1 verify-pack -v ../commitsonly.pack >objs && + awk "/tree|blob/{print \$1}" objs >trees_and_blobs && + git -C r1 rev-parse HEAD: >expected && + test_cmp trees_and_blobs expected +' + # Test blob:limit=[kmg] filter. # We boundary test around the size parameter. The filter is strictly less than # the value, so size 500 and 1000 should have the same results, but 1001 should diff --git a/t/t5616-partial-clone.sh b/t/t5616-partial-clone.sh index bbbe7537d..8eeb85fbc 100755 --- a/t/t5616-partial-clone.sh +++ b/t/t5616-partial-clone.sh @@ -154,6 +154,44 @@ test_expect_success 'partial clone with transfer.fsckobjects=1 uses index-pack - grep "git index-pack.*--fsck-objects" trace ' +test_expect_success 'use fsck before and after manually fetching a missing subtree' ' + # push new commit so server has a subtree + mkdir src/dir && + echo "in dir" >src/dir/file.txt && + git -C src add dir/file.txt && + git -C src commit -m "file in dir" && + git -C src push -u srv master && + SUBTREE=$(git -C src rev-parse HEAD:dir) && + + rm -rf dst && + git clone --no-checkout --filter=tree:0 "file://$(pwd)/srv.bare" dst && + git -C dst fsck && + + # Make sure we only have commits, and all trees and blobs are missing. + git -C dst rev-list master --missing=allow-any --objects >fetched_objects && + awk -f print_1.awk fetched_objects \ + | xargs -n1 git -C dst cat-file -t >fetched_types && + sort fetched_types -u >unique_types.observed && + echo commit >unique_types.expected && + test_cmp unique_types.observed unique_types.expected && + + # Auto-fetch a tree with cat-file. + git -C dst cat-file -p $SUBTREE >tree_contents && + grep file.txt tree_contents && + + # fsck still works after an auto-fetch of a tree. + git -C dst fsck && + + # Auto-fetch all remaining trees and blobs with --missing=error + git -C dst rev-list master --missing=error --objects >fetched_objects && + test_line_count = 70 fetched_objects && + awk -f print_1.awk fetched_objects \ + | xargs -n1 git -C dst cat-file -t >fetched_types && + sort fetched_types -u >unique_types.observed && + printf "blob\ncommit\ntree\n" >unique_types.expected && + test_cmp unique_types.observed unique_types.expected +' + test_expect_success 'partial clone fetches blobs pointed to by refs even if normally filtered out' ' rm -rf src dst && git init src && diff --git a/t/t6112-rev-list-filters-objects.sh b/t/t6112-rev-list-filters-objects.sh index 2e07dadf0..a989a7082 100755 --- a/t/t6112-rev-list-filters-objects.sh +++ b/t/t6112-rev-list-filters-objects.sh @@ -222,6 +222,18 @@ test_expect_success 'rev-list W/ --missing=print and --missing=allow-any for tre test_must_be_empty rev_list_err ' +# Test tree:0 filter. + +test_expect_success 'verify tree:0 includes trees in "filtered" output' ' + git -C r3 rev-list HEAD --quiet --objects --filter-print-omitted --filter=tree:0 \ + | awk -f print_1.awk \ + | sed s/~// \ + | xargs -n1 git -C r3 cat-file -t \ + | sort -u >filtered_types && + printf "blob\ntree\n" > expected && + test_cmp filtered_types expected +' + # Delete some loose objects and use rev-list, but WITHOUT any filtering. # This models previously omitted objects that we did not receive.