From patchwork Mon Mar 15 13:14:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 12139215 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D2C1BC43381 for ; Mon, 15 Mar 2021 13:15:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8FF2264EF2 for ; Mon, 15 Mar 2021 13:15:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229948AbhCONOx (ORCPT ); Mon, 15 Mar 2021 09:14:53 -0400 Received: from out4-smtp.messagingengine.com ([66.111.4.28]:52337 "EHLO out4-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229933AbhCONOe (ORCPT ); Mon, 15 Mar 2021 09:14:34 -0400 Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id E66B85C0181; Mon, 15 Mar 2021 09:14:33 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute1.internal (MEProxy); Mon, 15 Mar 2021 09:14:33 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=date :from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=fm3; bh=/+jiUPbs495isgCRZn2Y3WR6/bP Ft/z94SXdT6KzY1w=; b=Jb2BbjI+vn+2ujrwtJV8J3KTlSeRBUGTItIrmjZ/sbK 6F0bToPLyc/xCLXg8zUOJHwmAbKHGVx9LGWOdOBMqtuT03oU33LdI+M0bzHv7oQF 4XDtcOa2rvTiEXxhE3naaZVzZTTmE/oxCqcX+U0yQMGYvpUqMB4ieWlJCSLHD0jc /gpvnLolcnIhYDdQobcGSABSdlH+czgdWxnpRziRSMzLAgHuBhaZj8DQ/e+FtbX/ YPZtyx05HSoTe7f3aIXrz8KgDIeM3LKVSbCUoSKYZiQKgsj/7D+EDxYeowCXY+EF zm9M+madT8Z9XHQXF5kC5Ze+5IIVLa1Ifqd3M8cq0QQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=/+jiUP bs495isgCRZn2Y3WR6/bPFt/z94SXdT6KzY1w=; b=lHg/8TAMFTNlv3Hp0M3ftI feK9FgtrRD1c1l0okwsdDuHghhffJAgX20icxfVlBp2V2YFEXzYgOymrr3czda7z 6jzZavBfYD8Wy2YJ1doMDpgk/Ucm6E21hZfbcYox9kWn7pcCgcJD8RERdebVclsF CS/5GyanhLD27HVJhuUniu0IJLpwaHiv/skyAMASeXsRf8ZmYiirq9nGl4pPNFak 1FEdNAIgyfxoKytgHWaU1G9lsC23tbCLYUmX2V97pMG9Wzms2tQPnXmECAJjetke Ss0itAQ/URmNDQ5Ml0F56GcjeezVcG/Bk/YO//hmcDzTu3cR/LgOv77BdqPYcvUg == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledruddvledggeelucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvffukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefrrghtrhhi tghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtthgvrh hnpeehgefhtdefueffheekgfffudelffejtdfhvdejkedthfehvdelgfetgfdvtedthfen ucfkphepkeelrddugedrleeirddvgedunecuvehluhhsthgvrhfuihiivgeptdenucfrrg hrrghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh X-ME-Proxy: Received: from vm-mail.pks.im (dynamic-089-014-096-241.89.14.pool.telefonica.de [89.14.96.241]) by mail.messagingengine.com (Postfix) with ESMTPA id 42F981080067; Mon, 15 Mar 2021 09:14:33 -0400 (EDT) Received: from localhost (ncase [10.192.0.11]) by vm-mail.pks.im (OpenSMTPD) with ESMTPSA id 2e00912c (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 15 Mar 2021 13:14:32 +0000 (UTC) Date: Mon, 15 Mar 2021 14:14:31 +0100 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Christian Couder , Taylor Blau , Jeff King Subject: [PATCH v2 1/8] uploadpack.txt: document implication of `uploadpackfilter.allow` Message-ID: <270ff80dacff96f441e12954b059a68300157f2d.1615813673.git.ps@pks.im> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When `uploadpackfilter.allow` is set to `true`, it means that filters are enabled by default except in the case where a filter is explicitly disabled via `uploadpackilter..allow`. This option will not only enable the currently supported set of filters, but also any filters which get added in the future. As such, an admin which wants to have tight control over which filters are allowed and which aren't probably shouldn't ever set `uploadpackfilter.allow=true`. Amend the documentation to make the ramifications more explicit so that admins are aware of this. Signed-off-by: Patrick Steinhardt --- Documentation/config/uploadpack.txt | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/Documentation/config/uploadpack.txt b/Documentation/config/uploadpack.txt index b0d761282c..6729a072ea 100644 --- a/Documentation/config/uploadpack.txt +++ b/Documentation/config/uploadpack.txt @@ -59,7 +59,8 @@ uploadpack.allowFilter:: uploadpackfilter.allow:: Provides a default value for unspecified object filters (see: the - below configuration variable). + below configuration variable). If set to `true`, this will also + enable all filters which get added in the future. Defaults to `true`. uploadpackfilter..allow:: From patchwork Mon Mar 15 13:14:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 12139217 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D9A54C4332B for ; Mon, 15 Mar 2021 13:15:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B1E5864EED for ; Mon, 15 Mar 2021 13:15:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230016AbhCONOz (ORCPT ); Mon, 15 Mar 2021 09:14:55 -0400 Received: from out4-smtp.messagingengine.com ([66.111.4.28]:54417 "EHLO out4-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229928AbhCONOj (ORCPT ); Mon, 15 Mar 2021 09:14:39 -0400 Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id C9CAB5C012C; Mon, 15 Mar 2021 09:14:38 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute1.internal (MEProxy); Mon, 15 Mar 2021 09:14:38 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=date :from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=fm3; bh=RZFw8wKBrgJe4hFnO8el6flbQJx MCGMGQRH4bKbMLkI=; b=ej5HVMsjaLUEpVp8zipFnB6fy8+wgkCok9RNWFJ3vES RQuGWnmoHyFCuWrHxa+RV9U8JAC0FEzEW20wxKwiTUpODgE/GvS/8xoKWJDPPT61 kHod+d5BU0/quQhgx1GtIZ+tYE285TMQ/b3EteMCP33E0Aie/4PSHMsgiQzqyOUu RKJulT94WbAYPmDn8mlecWRRdSsUdGDjDld8XnyKCVlvTo/fk4Zr25NR9KNMZo0P sQL8VaxbkQSl5+75swv27WxXhGGHeR4QL5U18AM71We5i/46xIwm/RrP784NKsnz rcYMCLgTem9mK+nX02fN40X5ShvM9DKquHD/zUZ/XpA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=RZFw8w KBrgJe4hFnO8el6flbQJxMCGMGQRH4bKbMLkI=; b=HxBVgR/LNvHzLhCzMiO96h 20ynzSzPozIIrUyKvRHfGriKbnNNdE41cczk4jKbtL6h/qLtbxVGdkIs8ytSWDM+ cKTFPT0Ea5CBaxtKS44jeQNp8aGwDNej+HMOelmaCl0hgjcg922xK2VyEy9Ak5oM D70eCZ66FpxDXbLw9HNVeUAp9XRPbndCw9aKbd41UQpBx7hwQbL9fn/XyJ0zQk8n vjoPRSs3RTYA2fglVvU0cWA2OiyKPaqLYa9POAAW4XDKGYVKmFY/EqZ/2E/aOq/W LBHFkLWfXDf/6nbxeGUhBK/ulXc3StFctXjQN/C4J6reCDWiEGy1tzoykXSTFgjg == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledruddvledggeelucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvffukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefrrghtrhhi tghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtthgvrh hnpeehgefhtdefueffheekgfffudelffejtdfhvdejkedthfehvdelgfetgfdvtedthfen ucfkphepkeelrddugedrleeirddvgedunecuvehluhhsthgvrhfuihiivgepvdenucfrrg hrrghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh X-ME-Proxy: Received: from vm-mail.pks.im (dynamic-089-014-096-241.89.14.pool.telefonica.de [89.14.96.241]) by mail.messagingengine.com (Postfix) with ESMTPA id 1642F1080054; Mon, 15 Mar 2021 09:14:37 -0400 (EDT) Received: from localhost (ncase [10.192.0.11]) by vm-mail.pks.im (OpenSMTPD) with ESMTPSA id 3a1e248a (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 15 Mar 2021 13:14:37 +0000 (UTC) Date: Mon, 15 Mar 2021 14:14:36 +0100 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Christian Couder , Taylor Blau , Jeff King Subject: [PATCH v2 2/8] revision: mark commit parents as NOT_USER_GIVEN Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org The NOT_USER_GIVEN flag of an object marks whether a flag was explicitly provided by the user or not. The most important use case for this is when filtering objects: only objects that were not explicitly requested will get filtered. The flag is currently only set for blobs and trees, which has been fine given that there are no filters for tags or commits currently. We're about to extend filtering capabilities to add object type filter though, which requires us to set up the NOT_USER_GIVEN flag correctly -- if it's not set, the object wouldn't get filtered at all. Mark unseen commit parents as NOT_USER_GIVEN when processing parents. Like this, explicitly provided parents stay user-given and thus unfiltered, while parents which get loaded as part of the graph walk can be filtered. This commit shouldn't have any user-visible impact yet as there is no logic to filter commits yet. Signed-off-by: Patrick Steinhardt --- revision.c | 4 ++-- revision.h | 3 --- 2 files changed, 2 insertions(+), 5 deletions(-) diff --git a/revision.c b/revision.c index b78733f508..26f422f50d 100644 --- a/revision.c +++ b/revision.c @@ -1123,7 +1123,7 @@ static int process_parents(struct rev_info *revs, struct commit *commit, mark_parents_uninteresting(p); if (p->object.flags & SEEN) continue; - p->object.flags |= SEEN; + p->object.flags |= (SEEN | NOT_USER_GIVEN); if (list) commit_list_insert_by_date(p, list); if (queue) @@ -1165,7 +1165,7 @@ static int process_parents(struct rev_info *revs, struct commit *commit, } p->object.flags |= left_flag; if (!(p->object.flags & SEEN)) { - p->object.flags |= SEEN; + p->object.flags |= (SEEN | NOT_USER_GIVEN); if (list) commit_list_insert_by_date(p, list); if (queue) diff --git a/revision.h b/revision.h index e6be3c845e..f1f324a19b 100644 --- a/revision.h +++ b/revision.h @@ -44,9 +44,6 @@ /* * Indicates object was reached by traversal. i.e. not given by user on * command-line or stdin. - * NEEDSWORK: NOT_USER_GIVEN doesn't apply to commits because we only support - * filtering trees and blobs, but it may be useful to support filtering commits - * in the future. */ #define NOT_USER_GIVEN (1u<<25) #define TRACK_LINEAR (1u<<26) From patchwork Mon Mar 15 13:14:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 12139221 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BD122C43332 for ; Mon, 15 Mar 2021 13:15:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 785D764F0B for ; Mon, 15 Mar 2021 13:15:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230057AbhCONO4 (ORCPT ); Mon, 15 Mar 2021 09:14:56 -0400 Received: from out4-smtp.messagingengine.com ([66.111.4.28]:39355 "EHLO out4-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229734AbhCONOo (ORCPT ); Mon, 15 Mar 2021 09:14:44 -0400 Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id A195A5C012C; Mon, 15 Mar 2021 09:14:43 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute1.internal (MEProxy); Mon, 15 Mar 2021 09:14:43 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=date :from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=fm3; bh=utpoRoEnTPrLeARRCbgslW7vCl1 TYcgixior6fr3THU=; b=rELpKV5yKQwp7gXhijzWjZsiWZOj4PMdHP9y5ttVSRd dCipsYOYvVx7l0Ewem3rHQEniHhgmJClGB5y1aQGJAbU7N+OJl4dKLq1dGkiEFi0 DzWxpIuVk+7cKNNSvQabuK9RfgMIxcYjl0M8oHWi3gATLiff3SeVg8f+6QpX/aPb hP9xscyeA4QMjkT+9SychA+f3i5dRj2SkbibcfYjvBezoidnlUO2JzeXldPU7sak zMXCLsgc/mgC44kpBOIpPgxNGV5LP5hz0R/Ah/SJhDSWs95+PA49zaHa+wUSX+km ZBuimE7ncrNLUls0bKjCy53rCfafT8UdqWEe8yDEDpw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=utpoRo EnTPrLeARRCbgslW7vCl1TYcgixior6fr3THU=; b=GOS0zSRzXjuzSdhcZg348b Q9cZlzu2GCBcjrr0L96rlMxQyEDi5hBjLc/+rWM6YvnzqdTlicUAJXFP3nQ+dkjU z462k/DSgNEvVAYt6s4OsdUMYQf8KKG+3eFmirDIAfEMltPb0BUyMeS7y/MMy1J8 imwSHq3Qcqo6KfjUT+gEZ1wTRtHDgvudC+D3Zn/jaczPhjJeYfL1ZxIBl37znoP6 EYUGwByxmO5cHf9OXrrhZ320w1t4Dzv4YZNcd9lqZ7PWNN2u1ZLsxdtE+cgjBHbm Pa2tbmRSe4YlElQS5zUIlfkJAoSceuI2ilkHZfqo8hNqdZqDHVGGse373DwBeEcA == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledruddvledggeelucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvffukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefrrghtrhhi tghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtthgvrh hnpeehgefhtdefueffheekgfffudelffejtdfhvdejkedthfehvdelgfetgfdvtedthfen ucfkphepkeelrddugedrleeirddvgedunecuvehluhhsthgvrhfuihiivgepfeenucfrrg hrrghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh X-ME-Proxy: Received: from vm-mail.pks.im (dynamic-089-014-096-241.89.14.pool.telefonica.de [89.14.96.241]) by mail.messagingengine.com (Postfix) with ESMTPA id DE96A1080067; Mon, 15 Mar 2021 09:14:42 -0400 (EDT) Received: from localhost (ncase [10.192.0.11]) by vm-mail.pks.im (OpenSMTPD) with ESMTPSA id ca555be9 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 15 Mar 2021 13:14:42 +0000 (UTC) Date: Mon, 15 Mar 2021 14:14:40 +0100 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Christian Couder , Taylor Blau , Jeff King Subject: [PATCH v2 3/8] list-objects: move tag processing into its own function Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Move processing of tags into its own function to make the logic easier to extend when we're going to implement filtering for tags. No change in behaviour is expected from this commit. Signed-off-by: Patrick Steinhardt --- list-objects.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/list-objects.c b/list-objects.c index e19589baa0..093adf85b1 100644 --- a/list-objects.c +++ b/list-objects.c @@ -213,6 +213,15 @@ static void process_tree(struct traversal_context *ctx, free_tree_buffer(tree); } +static void process_tag(struct traversal_context *ctx, + struct tag *tag, + struct strbuf *base, + const char *name) +{ + tag->object.flags |= SEEN; + ctx->show_object(&tag->object, name, ctx->show_data); +} + static void mark_edge_parents_uninteresting(struct commit *commit, struct rev_info *revs, show_edge_fn show_edge) @@ -334,8 +343,7 @@ static void traverse_trees_and_blobs(struct traversal_context *ctx, if (obj->flags & (UNINTERESTING | SEEN)) continue; if (obj->type == OBJ_TAG) { - obj->flags |= SEEN; - ctx->show_object(obj, name, ctx->show_data); + process_tag(ctx, (struct tag *)obj, base, name); continue; } if (!path) From patchwork Mon Mar 15 13:14:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 12139219 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 28E9EC4332E for ; Mon, 15 Mar 2021 13:15:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id F3F2364EEB for ; Mon, 15 Mar 2021 13:15:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230097AbhCONO5 (ORCPT ); Mon, 15 Mar 2021 09:14:57 -0400 Received: from out4-smtp.messagingengine.com ([66.111.4.28]:37161 "EHLO out4-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229746AbhCONOs (ORCPT ); Mon, 15 Mar 2021 09:14:48 -0400 Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 68AB15C0099; Mon, 15 Mar 2021 09:14:48 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute1.internal (MEProxy); Mon, 15 Mar 2021 09:14:48 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=date :from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=fm3; bh=KGl2MQyrZ179/4MwwiHddwZ3eDl k4JtYeNlXls/gb7w=; b=il4c0AviapILfMyACW2QJ2eoEU7dsHIfRRzFYPk1UCJ LyrieOSoSD7daH5jVaFe92c9NyvAathgxT7j+MyYoxM3J9Abi5ndhtcB6d+KIUIx HwG50trAf2jMpNUtMsxzD2MLUWWr1YGtVQhaRzYcNhgRLXOgnPlKjnToZkPUvjk8 EtxHIr0MmMEh5fJvxsL5x8kjH5z6N9aUMIFaWH2C7uCcbJF3YdCc88CqlGOV2rOE TooIOk1mSLOAgyTOOXNAO3837qG2VtJMKgsuuDWDijRLEvZNnYTZQNCDkWWhnhR0 ZBSCyey8B4qTcxbxOHSv6hxcjZ5iZpz5VHfzV5PQ1ew== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=KGl2MQ yrZ179/4MwwiHddwZ3eDlk4JtYeNlXls/gb7w=; b=nSXFhSJ9jJ0KX6PzJ+XL81 Ri7XPaeyV6O0aGOIy3SySlZ7echVoxUZLKdvYKwXlNTKTyaaEP2XHbnJaPojllue ekZHfXhUYTJULxBwvK5RU7TQV6nFPvMSq+e4MEtRXDDY2dHFOlA7w0gpBQ+SplZk 6Wh3Df80HUemBXTsa3YIXyqxX2lTfmYI7f4q9zgHJCccqlP4HiwbjlgxD7R/tivc TOVgwUeoKivKpdJpSW0U/Kf1AKoanstjd1CSyHaFJL0J3tvt1/WojkSHh8kE7S+I hw9NdBlme71DKvIQbp3iidLogROMCfef37sUAgtRaqhVuyKr/4Hahn1TtaHS0lVg == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledruddvledggeelucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvffukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefrrghtrhhi tghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtthgvrh hnpeehgefhtdefueffheekgfffudelffejtdfhvdejkedthfehvdelgfetgfdvtedthfen ucfkphepkeelrddugedrleeirddvgedunecuvehluhhsthgvrhfuihiivgepgeenucfrrg hrrghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh X-ME-Proxy: Received: from vm-mail.pks.im (dynamic-089-014-096-241.89.14.pool.telefonica.de [89.14.96.241]) by mail.messagingengine.com (Postfix) with ESMTPA id B4E921080066; Mon, 15 Mar 2021 09:14:47 -0400 (EDT) Received: from localhost (ncase [10.192.0.11]) by vm-mail.pks.im (OpenSMTPD) with ESMTPSA id 782db972 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 15 Mar 2021 13:14:47 +0000 (UTC) Date: Mon, 15 Mar 2021 14:14:45 +0100 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Christian Couder , Taylor Blau , Jeff King Subject: [PATCH v2 4/8] list-objects: support filtering by tag and commit Message-ID: <5545c189c56cb44d0621fb24a531420c29de55d0.1615813673.git.ps@pks.im> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Object filters currently only support filtering blobs or trees based on some criteria. This commit lays the foundation to also allow filtering of tags and commits. No change in behaviour is expected from this commit given that there are no filters yet for those object types. Signed-off-by: Patrick Steinhardt --- list-objects-filter.c | 40 ++++++++++++++++++++++++++++++++++++++++ list-objects-filter.h | 2 ++ list-objects.c | 24 +++++++++++++++++++++--- 3 files changed, 63 insertions(+), 3 deletions(-) diff --git a/list-objects-filter.c b/list-objects-filter.c index 4ec0041cfb..7def039435 100644 --- a/list-objects-filter.c +++ b/list-objects-filter.c @@ -82,6 +82,16 @@ static enum list_objects_filter_result filter_blobs_none( default: BUG("unknown filter_situation: %d", filter_situation); + case LOFS_TAG: + assert(obj->type == OBJ_TAG); + /* always include all tag objects */ + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + + case LOFS_COMMIT: + assert(obj->type == OBJ_COMMIT); + /* always include all commit objects */ + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + case LOFS_BEGIN_TREE: assert(obj->type == OBJ_TREE); /* always include all tree objects */ @@ -173,6 +183,16 @@ static enum list_objects_filter_result filter_trees_depth( default: BUG("unknown filter_situation: %d", filter_situation); + case LOFS_TAG: + assert(obj->type == OBJ_TAG); + /* always include all tag objects */ + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + + case LOFS_COMMIT: + assert(obj->type == OBJ_COMMIT); + /* always include all commit objects */ + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + case LOFS_END_TREE: assert(obj->type == OBJ_TREE); filter_data->current_depth--; @@ -267,6 +287,16 @@ static enum list_objects_filter_result filter_blobs_limit( default: BUG("unknown filter_situation: %d", filter_situation); + case LOFS_TAG: + assert(obj->type == OBJ_TAG); + /* always include all tag objects */ + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + + case LOFS_COMMIT: + assert(obj->type == OBJ_COMMIT); + /* always include all commit objects */ + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + case LOFS_BEGIN_TREE: assert(obj->type == OBJ_TREE); /* always include all tree objects */ @@ -371,6 +401,16 @@ static enum list_objects_filter_result filter_sparse( default: BUG("unknown filter_situation: %d", filter_situation); + case LOFS_TAG: + assert(obj->type == OBJ_TAG); + /* always include all tag objects */ + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + + case LOFS_COMMIT: + assert(obj->type == OBJ_COMMIT); + /* always include all commit objects */ + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + case LOFS_BEGIN_TREE: assert(obj->type == OBJ_TREE); dtype = DT_DIR; diff --git a/list-objects-filter.h b/list-objects-filter.h index cfd784e203..9e98814111 100644 --- a/list-objects-filter.h +++ b/list-objects-filter.h @@ -55,6 +55,8 @@ enum list_objects_filter_result { }; enum list_objects_filter_situation { + LOFS_COMMIT, + LOFS_TAG, LOFS_BEGIN_TREE, LOFS_END_TREE, LOFS_BLOB diff --git a/list-objects.c b/list-objects.c index 093adf85b1..3b63dfd4f2 100644 --- a/list-objects.c +++ b/list-objects.c @@ -218,8 +218,16 @@ static void process_tag(struct traversal_context *ctx, struct strbuf *base, const char *name) { - tag->object.flags |= SEEN; - ctx->show_object(&tag->object, name, ctx->show_data); + enum list_objects_filter_result r; + + r = list_objects_filter__filter_object(ctx->revs->repo, LOFS_TAG, + &tag->object, base->buf, + &base->buf[base->len], + ctx->filter); + if (r & LOFR_MARK_SEEN) + tag->object.flags |= SEEN; + if (r & LOFR_DO_SHOW) + ctx->show_object(&tag->object, name, ctx->show_data); } static void mark_edge_parents_uninteresting(struct commit *commit, @@ -369,6 +377,12 @@ static void do_traverse(struct traversal_context *ctx) strbuf_init(&csp, PATH_MAX); while ((commit = get_revision(ctx->revs)) != NULL) { + enum list_objects_filter_result r; + + r = list_objects_filter__filter_object(ctx->revs->repo, + LOFS_COMMIT, &commit->object, + NULL, NULL, ctx->filter); + /* * an uninteresting boundary commit may not have its tree * parsed yet, but we are not going to show them anyway @@ -383,7 +397,11 @@ static void do_traverse(struct traversal_context *ctx) die(_("unable to load root tree for commit %s"), oid_to_hex(&commit->object.oid)); } - ctx->show_commit(commit, ctx->show_data); + + if (r & LOFR_MARK_SEEN) + commit->object.flags |= SEEN; + if (r & LOFR_DO_SHOW) + ctx->show_commit(commit, ctx->show_data); if (ctx->revs->tree_blobs_in_commit_order) /* From patchwork Mon Mar 15 13:14:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 12139223 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C6591C433E0 for ; Mon, 15 Mar 2021 13:15:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 82FF764EF2 for ; Mon, 15 Mar 2021 13:15:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230187AbhCONP0 (ORCPT ); Mon, 15 Mar 2021 09:15:26 -0400 Received: from out4-smtp.messagingengine.com ([66.111.4.28]:47861 "EHLO out4-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229826AbhCONOx (ORCPT ); Mon, 15 Mar 2021 09:14:53 -0400 Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 59B0A5C00B9; Mon, 15 Mar 2021 09:14:53 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute1.internal (MEProxy); Mon, 15 Mar 2021 09:14:53 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=date :from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=fm3; bh=8LCftP1bzNoxKFN4pJhiwGdsV8T EsIwE7lYWp8D3AHI=; b=SvIDyNMrJdsJasz2+WflLIS1LRI5XHrYmJiX4K0QEco Qr+mnVy8B1+qahttwhnPgXpUsrEo0xrpgPN/ZXRcxrcQw/pQpxWI8OpE6GJM2t48 n2W+0dMoKqN3Qio0kBharRnzNX5kwtMnJvUdeFQyhRvZFm5NsHhni+lIs2GsDQ2J jPuWO9bJJTUox/ZX4yuzbPu+dVB6Fcf8xH8g6JFkKPrhhrAmxo6cBkNYKTT9tRVg s5+dKf9y6l6A1f+NS2iKVJVdxnZS6KGQ/SlVtCbQEnRRBVYM3MwGaLnYYmpGLhRi fWdxdDAhyv/z4JrJ5oX7L4YLTwZol+usPH6dgrHOaFQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=8LCftP 1bzNoxKFN4pJhiwGdsV8TEsIwE7lYWp8D3AHI=; b=v3stL864Ffz49Lc9DQnKLT iDPohaSYyNM5AxHvvZvyhRBbQyn7ExdTYR+//nfY0cpXVMrHn21rpNOhZ8Deq8Hi /8J5EmFt1R+8aQo0x8NF8w4HaDa59nERRU/ime5MGbEZEzf4UPmLsT9oXg9SUEvF jdkcAu9FoA8XlFdMISfMmf39HHOMFUQ8J2jNI7AxoosS+saCu7vT3wxcm5iQS/Dv VO6WZygfuK5u4CSvTFpBaHrMHSTniuQz1lOJyz2qPUzsHaXSB4eyvOyFm9GN6wEJ cDxKcnyqNN/CcIKYVIy3gF+hGBGSy3WC8JDsAARUgEBkRlPreXuSXtI7t3fWkkow == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledruddvledggeelucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvffukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefrrghtrhhi tghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtthgvrh hnpeehgefhtdefueffheekgfffudelffejtdfhvdejkedthfehvdelgfetgfdvtedthfen ucfkphepkeelrddugedrleeirddvgedunecuvehluhhsthgvrhfuihiivgepheenucfrrg hrrghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh X-ME-Proxy: Received: from vm-mail.pks.im (dynamic-089-014-096-241.89.14.pool.telefonica.de [89.14.96.241]) by mail.messagingengine.com (Postfix) with ESMTPA id 974B41080067; Mon, 15 Mar 2021 09:14:52 -0400 (EDT) Received: from localhost (ncase [10.192.0.11]) by vm-mail.pks.im (OpenSMTPD) with ESMTPSA id 4c30ea0f (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 15 Mar 2021 13:14:52 +0000 (UTC) Date: Mon, 15 Mar 2021 14:14:50 +0100 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Christian Couder , Taylor Blau , Jeff King Subject: [PATCH v2 5/8] list-objects: implement object type filter Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org While it already is possible to filter objects by some criteria in git-rev-list(1), it is not yet possible to filter out only a specific type of objects. This makes some filters less useful. The `blob:limit` filter for example filters blobs such that only those which are smaller than the given limit are returned. But it is unfit to ask only for these smallish blobs, given that git-rev-list(1) will continue to print tags, commits and trees. Now that we have the infrastructure in place to also filter tags and commits, we can improve this situation by implementing a new filter which selects objects based on their type. Above query can thus trivially be implemented with the following command: $ git rev-list --objects --filter=object:type=blob \ --filter=blob:limit=200 Furthermore, this filter allows to optimize for certain other cases: if for example only tags or commits have been selected, there is no need to walk down trees. The new filter is not yet supported in bitmaps. This is going to be implemented in a subsequent commit. Signed-off-by: Patrick Steinhardt --- Documentation/config/uploadpack.txt | 6 +-- Documentation/rev-list-options.txt | 3 ++ list-objects-filter-options.c | 14 ++++++ list-objects-filter-options.h | 2 + list-objects-filter.c | 76 +++++++++++++++++++++++++++++ t/t6112-rev-list-filters-objects.sh | 48 ++++++++++++++++++ 6 files changed, 146 insertions(+), 3 deletions(-) diff --git a/Documentation/config/uploadpack.txt b/Documentation/config/uploadpack.txt index 6729a072ea..32fad5bbe8 100644 --- a/Documentation/config/uploadpack.txt +++ b/Documentation/config/uploadpack.txt @@ -66,9 +66,9 @@ uploadpackfilter.allow:: uploadpackfilter..allow:: Explicitly allow or ban the object filter corresponding to ``, where `` may be one of: `blob:none`, - `blob:limit`, `tree`, `sparse:oid`, or `combine`. If using - combined filters, both `combine` and all of the nested filter - kinds must be allowed. Defaults to `uploadpackfilter.allow`. + `blob:limit`, `object:type`, `tree`, `sparse:oid`, or `combine`. + If using combined filters, both `combine` and all of the nested + filter kinds must be allowed. Defaults to `uploadpackfilter.allow`. uploadpackfilter.tree.maxDepth:: Only allow `--filter=tree:` when `` is no more than the value of diff --git a/Documentation/rev-list-options.txt b/Documentation/rev-list-options.txt index b1c8f86c6e..3afa8fffbd 100644 --- a/Documentation/rev-list-options.txt +++ b/Documentation/rev-list-options.txt @@ -892,6 +892,9 @@ or units. n may be zero. The suffixes k, m, and g can be used to name units in KiB, MiB, or GiB. For example, 'blob:limit=1k' is the same as 'blob:limit=1024'. + +The form '--filter=object:type=(tag|commit|tree|blob)' omits all objects +which are not of the requested type. ++ The form '--filter=sparse:oid=' uses a sparse-checkout specification contained in the blob (or blob-expression) '' to omit blobs that would not be not required for a sparse checkout on diff --git a/list-objects-filter-options.c b/list-objects-filter-options.c index d2d1c81caf..bb6f6577d5 100644 --- a/list-objects-filter-options.c +++ b/list-objects-filter-options.c @@ -29,6 +29,8 @@ const char *list_object_filter_config_name(enum list_objects_filter_choice c) return "tree"; case LOFC_SPARSE_OID: return "sparse:oid"; + case LOFC_OBJECT_TYPE: + return "object:type"; case LOFC_COMBINE: return "combine"; case LOFC__COUNT: @@ -97,6 +99,18 @@ static int gently_parse_list_objects_filter( } return 1; + } else if (skip_prefix(arg, "object:type=", &v0)) { + int type = type_from_string_gently(v0, -1, 1); + if (type < 0) { + strbuf_addstr(errbuf, _("expected 'object:type='")); + return 1; + } + + filter_options->object_type = type; + filter_options->choice = LOFC_OBJECT_TYPE; + + return 0; + } else if (skip_prefix(arg, "combine:", &v0)) { return parse_combine_filter(filter_options, v0, errbuf); diff --git a/list-objects-filter-options.h b/list-objects-filter-options.h index 01767c3c96..4d0d0588cc 100644 --- a/list-objects-filter-options.h +++ b/list-objects-filter-options.h @@ -13,6 +13,7 @@ enum list_objects_filter_choice { LOFC_BLOB_LIMIT, LOFC_TREE_DEPTH, LOFC_SPARSE_OID, + LOFC_OBJECT_TYPE, LOFC_COMBINE, LOFC__COUNT /* must be last */ }; @@ -54,6 +55,7 @@ struct list_objects_filter_options { char *sparse_oid_name; unsigned long blob_limit_value; unsigned long tree_exclude_depth; + enum object_type object_type; /* LOFC_COMBINE values */ diff --git a/list-objects-filter.c b/list-objects-filter.c index 7def039435..650a7c2c80 100644 --- a/list-objects-filter.c +++ b/list-objects-filter.c @@ -545,6 +545,81 @@ static void filter_sparse_oid__init( filter->free_fn = filter_sparse_free; } +/* + * A filter for list-objects to omit large blobs. + * And to OPTIONALLY collect a list of the omitted OIDs. + */ +struct filter_object_type_data { + enum object_type object_type; +}; + +static enum list_objects_filter_result filter_object_type( + struct repository *r, + enum list_objects_filter_situation filter_situation, + struct object *obj, + const char *pathname, + const char *filename, + struct oidset *omits, + void *filter_data_) +{ + struct filter_object_type_data *filter_data = filter_data_; + + switch (filter_situation) { + default: + BUG("unknown filter_situation: %d", filter_situation); + + case LOFS_TAG: + assert(obj->type == OBJ_TAG); + if (filter_data->object_type == OBJ_TAG) + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + return LOFR_MARK_SEEN; + + case LOFS_COMMIT: + assert(obj->type == OBJ_COMMIT); + if (filter_data->object_type == OBJ_COMMIT) + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + return LOFR_MARK_SEEN; + + case LOFS_BEGIN_TREE: + assert(obj->type == OBJ_TREE); + + /* + * If we only want to show commits or tags, then there is no + * need to walk down trees. + */ + if (filter_data->object_type == OBJ_COMMIT || + filter_data->object_type == OBJ_TAG) + return LOFR_SKIP_TREE; + + if (filter_data->object_type == OBJ_TREE) + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + + return LOFR_MARK_SEEN; + + case LOFS_BLOB: + assert(obj->type == OBJ_BLOB); + + if (filter_data->object_type == OBJ_BLOB) + return LOFR_MARK_SEEN | LOFR_DO_SHOW; + return LOFR_MARK_SEEN; + + case LOFS_END_TREE: + return LOFR_ZERO; + } +} + +static void filter_object_type__init( + struct list_objects_filter_options *filter_options, + struct filter *filter) +{ + struct filter_object_type_data *d = xcalloc(1, sizeof(*d)); + d->object_type = filter_options->object_type; + + filter->filter_data = d; + filter->filter_object_fn = filter_object_type; + filter->free_fn = free; +} + /* A filter which only shows objects shown by all sub-filters. */ struct combine_filter_data { struct subfilter *sub; @@ -691,6 +766,7 @@ static filter_init_fn s_filters[] = { filter_blobs_limit__init, filter_trees_depth__init, filter_sparse_oid__init, + filter_object_type__init, filter_combine__init, }; diff --git a/t/t6112-rev-list-filters-objects.sh b/t/t6112-rev-list-filters-objects.sh index 31457d13b9..c79ec04060 100755 --- a/t/t6112-rev-list-filters-objects.sh +++ b/t/t6112-rev-list-filters-objects.sh @@ -159,6 +159,54 @@ test_expect_success 'verify blob:limit=1m' ' test_must_be_empty observed ' +# Test object:type= filter. + +test_expect_success 'setup object-type' ' + git init object-type && + echo contents >object-type/blob && + git -C object-type add blob && + git -C object-type commit -m commit-message && + git -C object-type tag tag -m tag-message +' + +test_expect_success 'verify object:type= fails with invalid type' ' + test_must_fail git -C object-type rev-list --objects --filter=object:type= HEAD && + test_must_fail git -C object-type rev-list --objects --filter=object:type=invalid HEAD +' + +test_expect_success 'verify object:type=blob prints blob and commit' ' + ( + git -C object-type rev-parse HEAD && + printf "%s blob\n" $(git -C object-type rev-parse HEAD:blob) + ) >expected && + git -C object-type rev-list --objects --filter=object:type=blob HEAD >actual && + test_cmp expected actual +' + +test_expect_success 'verify object:type=tree prints tree and commit' ' + ( + git -C object-type rev-parse HEAD && + printf "%s \n" $(git -C object-type rev-parse HEAD^{tree}) + ) >expected && + git -C object-type rev-list --objects --filter=object:type=tree HEAD >actual && + test_cmp expected actual +' + +test_expect_success 'verify object:type=commit prints commit' ' + git -C object-type rev-parse HEAD >expected && + git -C object-type rev-list --objects --filter=object:type=commit HEAD >actual && + test_cmp expected actual +' + +test_expect_success 'verify object:type=tag prints tag' ' + ( + git -C object-type rev-parse HEAD && + printf "%s tag\n" $(git -C object-type rev-parse tag) + ) >expected && + git -C object-type rev-list --objects --filter=object:type=tag tag >actual && + test_cmp expected actual +' + # Test sparse:path= filter. # !!!! # NOTE: sparse:path filter support has been dropped for security reasons, From patchwork Mon Mar 15 13:14:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 12139229 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 415DEC2BA1A for ; Mon, 15 Mar 2021 13:16:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 218C764F23 for ; Mon, 15 Mar 2021 13:16:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230205AbhCONP1 (ORCPT ); Mon, 15 Mar 2021 09:15:27 -0400 Received: from out4-smtp.messagingengine.com ([66.111.4.28]:55021 "EHLO out4-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229735AbhCONO6 (ORCPT ); Mon, 15 Mar 2021 09:14:58 -0400 Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 3D7365C0181; Mon, 15 Mar 2021 09:14:58 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute1.internal (MEProxy); Mon, 15 Mar 2021 09:14:58 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=date :from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=fm3; bh=CO8sHZpWCOvOkCZLg12P0W2i+uC a8Yy9gmypQy7z53I=; b=dj0No030ZcjKlIjoznC/CNf79z5MJeXMkWVTlZCeEtX KNzh8Zil1I2JwsY1lltuJKapB+I2Z0k8a3La4nuyXcK1IqGPYXQ3CYJSxLmlvzkV AIAilX3iRx/LT6KpUKVh7NXrZmKkXPzfFhmTrEGyrgY7hgbUW4P6F09PxjZ3VOn4 aCy9wkwdrDCGPtKiaA9pt3fjfA8zAvFszLpghYV88VywaS8T+y6zMbHTq+lfst2r K/xAmoRTwRjWl4BUHGISKtKYmeuQ/u2aHPOVLE/u8GUhQzL03X8QZhbqKYG+1RgC cXdXPvvnfQRWYijqyd3R9TD6c7DK6gNQPy6JohPdsAg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=CO8sHZ pWCOvOkCZLg12P0W2i+uCa8Yy9gmypQy7z53I=; b=lTzUW0VUjmm3pbuk5jPKAC m9qrLOZtbrbH0L37d8+/fglIHNiQIYYXPOYLGt4VaJCUMDxIzQmngVlNuXPVOnje kitMI0PewT+fbF06cqS9tDNeI6kLcUMetLKNZcC6bCNuo/vWpNAlMSBLh788Iv9m C9Ooc8K3b2Vh3OQTeIScMt5D4hNL9JMMnYc7Qa29w/tL2F4uDmr7wwVcz3PLUx66 2aHsGc1xT2OisqFCZcBOjzjg3nxtlJByRyQn+64A9/gjY4FdnUrGjIgWen80D0nj 2hqN8dSBIcaTfhayUHuT3gA+rwj7IUcUEAsI2iZSJeQghMy5L3sVX5Cw3gWZmvVA == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledruddvledggeelucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvffukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefrrghtrhhi tghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtthgvrh hnpeehgefhtdefueffheekgfffudelffejtdfhvdejkedthfehvdelgfetgfdvtedthfen ucfkphepkeelrddugedrleeirddvgedunecuvehluhhsthgvrhfuihiivgepheenucfrrg hrrghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh X-ME-Proxy: Received: from vm-mail.pks.im (dynamic-089-014-096-241.89.14.pool.telefonica.de [89.14.96.241]) by mail.messagingengine.com (Postfix) with ESMTPA id 705A61080067; Mon, 15 Mar 2021 09:14:57 -0400 (EDT) Received: from localhost (ncase [10.192.0.11]) by vm-mail.pks.im (OpenSMTPD) with ESMTPSA id 8e93f94a (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 15 Mar 2021 13:14:56 +0000 (UTC) Date: Mon, 15 Mar 2021 14:14:55 +0100 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Christian Couder , Taylor Blau , Jeff King Subject: [PATCH v2 6/8] pack-bitmap: implement object type filter Message-ID: <8073ab665b07cf653478482f801a06e072233230.1615813673.git.ps@pks.im> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org The preceding commit has added a new object filter for git-rev-list(1) which allows to filter objects by type. Implement the equivalent filter for packfile bitmaps so that we can answer these queries fast. Signed-off-by: Patrick Steinhardt --- pack-bitmap.c | 28 +++++++++++++++++++++++++--- t/t6113-rev-list-bitmap-filters.sh | 25 ++++++++++++++++++++++++- 2 files changed, 49 insertions(+), 4 deletions(-) diff --git a/pack-bitmap.c b/pack-bitmap.c index 1f69b5fa85..196d38c91d 100644 --- a/pack-bitmap.c +++ b/pack-bitmap.c @@ -779,9 +779,6 @@ static void filter_bitmap_exclude_type(struct bitmap_index *bitmap_git, eword_t mask; uint32_t i; - if (type != OBJ_BLOB && type != OBJ_TREE) - BUG("filter_bitmap_exclude_type: unsupported type '%d'", type); - /* * The non-bitmap version of this filter never removes * objects which the other side specifically asked for, @@ -911,6 +908,23 @@ static void filter_bitmap_tree_depth(struct bitmap_index *bitmap_git, OBJ_BLOB); } +static void filter_bitmap_object_type(struct bitmap_index *bitmap_git, + struct object_list *tip_objects, + struct bitmap *to_filter, + enum object_type object_type) +{ + enum object_type t; + + if (object_type < OBJ_COMMIT || object_type > OBJ_TAG) + BUG("filter_bitmap_object_type given invalid object"); + + for (t = OBJ_COMMIT; t <= OBJ_TAG; t++) { + if (t == object_type) + continue; + filter_bitmap_exclude_type(bitmap_git, tip_objects, to_filter, t); + } +} + static int filter_bitmap(struct bitmap_index *bitmap_git, struct object_list *tip_objects, struct bitmap *to_filter, @@ -943,6 +957,14 @@ static int filter_bitmap(struct bitmap_index *bitmap_git, return 0; } + if (filter->choice == LOFC_OBJECT_TYPE) { + if (bitmap_git) + filter_bitmap_object_type(bitmap_git, tip_objects, + to_filter, + filter->object_type); + return 0; + } + /* filter choice not handled */ return -1; } diff --git a/t/t6113-rev-list-bitmap-filters.sh b/t/t6113-rev-list-bitmap-filters.sh index 3f889949ca..fb66735ac8 100755 --- a/t/t6113-rev-list-bitmap-filters.sh +++ b/t/t6113-rev-list-bitmap-filters.sh @@ -10,7 +10,8 @@ test_expect_success 'set up bitmapped repo' ' test_commit much-larger-blob-one && git repack -adb && test_commit two && - test_commit much-larger-blob-two + test_commit much-larger-blob-two && + git tag tag ' test_expect_success 'filters fallback to non-bitmap traversal' ' @@ -75,4 +76,26 @@ test_expect_success 'tree:1 filter' ' test_cmp expect actual ' +test_expect_success 'object:type filter' ' + git rev-list --objects --filter=object:type=tag tag >expect && + git rev-list --use-bitmap-index \ + --objects --filter=object:type=tag tag >actual && + test_cmp expect actual && + + git rev-list --objects --filter=object:type=commit tag >expect && + git rev-list --use-bitmap-index \ + --objects --filter=object:type=commit tag >actual && + test_bitmap_traversal expect actual && + + git rev-list --objects --filter=object:type=tree tag >expect && + git rev-list --use-bitmap-index \ + --objects --filter=object:type=tree tag >actual && + test_bitmap_traversal expect actual && + + git rev-list --objects --filter=object:type=blob tag >expect && + git rev-list --use-bitmap-index \ + --objects --filter=object:type=blob tag >actual && + test_bitmap_traversal expect actual +' + test_done From patchwork Mon Mar 15 13:14:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 12139225 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C1089C4332D for ; Mon, 15 Mar 2021 13:15:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A9FE564F05 for ; Mon, 15 Mar 2021 13:15:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230247AbhCONPa (ORCPT ); Mon, 15 Mar 2021 09:15:30 -0400 Received: from out4-smtp.messagingengine.com ([66.111.4.28]:41131 "EHLO out4-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230129AbhCONPC (ORCPT ); Mon, 15 Mar 2021 09:15:02 -0400 Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 4F3C85C012C; Mon, 15 Mar 2021 09:15:02 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute1.internal (MEProxy); Mon, 15 Mar 2021 09:15:02 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=date :from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=fm3; bh=CYpq74jijYigs+yUJvwYq2iXL4i R0X0K3wAHbtNjmOg=; b=WFkZ+ZCBjX3YWoOMRYr/3edaj6AKif+QTuOwfdLNCrN rLdqC9JXaPoUHeFgenu0syn3JXVlbjvl+6ros2XDiulO9q3SGhhbEhVVNU0gepu2 rkRWjydeock8R0MlxD/3alzl2QtXMVr61M4MaZI0RtcMoBZH4c/k9TQyWnUWl1fu RQHeeHPbWdbQ+Y9ZKjDbRrSzS8+hhI4DtE94axvYrvdTYTFyww1e22v12R19TIMw +D0EMdKv3C/uc2Kk33dMKjpj/ZcZh+Y12S4EIWo2u1epvIFQ6ftUr9JiTTpffFlc hgAtJqPohM0w7+ln7O6R2iiTv5JG1oSXN+WNPgPy9iA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=CYpq74 jijYigs+yUJvwYq2iXL4iR0X0K3wAHbtNjmOg=; b=EQZCQ+Hlk/uJ+PJLFRP7wE qIxmAGtHdvzvA/z4qzcP9b6k79iiIsuxolm8aPTmSOTLQLlgBD1Oejr89mnxYSQL jaOsIQRZxBiPCuLj+U6xlaCiccvSKBCo4U6AWlSYBYZat092aYupuwxYBHHjlFiC xvZloNwKDdgwwjodlqMFcW6Zs9ouXB+nVRL3hYkLFXnZ7ryxGlCaVSRrNIjK/ksj CArQlFpnLN2rS4G77dSWcFyMFsoD+iCSyvIK+1PXYu3H6c5bE7oPv4tGCePl/HMp wbaQNm3+X1VdPhAd6sZPxAbF4FmkgIer1YlP9NXFr7mcw8kSTPT2cC1YvSaE+yfg == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledruddvledggeelucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvffukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefrrghtrhhi tghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtthgvrh hnpeehgefhtdefueffheekgfffudelffejtdfhvdejkedthfehvdelgfetgfdvtedthfen ucfkphepkeelrddugedrleeirddvgedunecuvehluhhsthgvrhfuihiivgepjeenucfrrg hrrghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh X-ME-Proxy: Received: from vm-mail.pks.im (dynamic-089-014-096-241.89.14.pool.telefonica.de [89.14.96.241]) by mail.messagingengine.com (Postfix) with ESMTPA id 4DA391080068; Mon, 15 Mar 2021 09:15:01 -0400 (EDT) Received: from localhost (ncase [10.192.0.11]) by vm-mail.pks.im (OpenSMTPD) with ESMTPSA id 37188f0c (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 15 Mar 2021 13:15:01 +0000 (UTC) Date: Mon, 15 Mar 2021 14:14:59 +0100 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Christian Couder , Taylor Blau , Jeff King Subject: [PATCH v2 7/8] pack-bitmap: implement combined filter Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When the user has multiple objects filters specified, then this is internally represented by having a "combined" filter. These combined filters aren't yet supported by bitmap indices and can thus not be accelerated. Fix this by implementing support for these combined filters. The implementation is quite trivial: when there's a combined filter, we simply recurse into `filter_bitmap()` for all of the sub-filters. Signed-off-by: Patrick Steinhardt --- pack-bitmap.c | 40 +++++++++++++++++++++++++++--- t/t6113-rev-list-bitmap-filters.sh | 7 ++++++ 2 files changed, 43 insertions(+), 4 deletions(-) diff --git a/pack-bitmap.c b/pack-bitmap.c index 196d38c91d..e33805e076 100644 --- a/pack-bitmap.c +++ b/pack-bitmap.c @@ -925,6 +925,29 @@ static void filter_bitmap_object_type(struct bitmap_index *bitmap_git, } } +static int filter_supported(struct list_objects_filter_options *filter) +{ + int i; + + switch (filter->choice) { + case LOFC_BLOB_NONE: + case LOFC_BLOB_LIMIT: + case LOFC_OBJECT_TYPE: + return 1; + case LOFC_TREE_DEPTH: + if (filter->tree_exclude_depth == 0) + return 1; + return 0; + case LOFC_COMBINE: + for (i = 0; i < filter->sub_nr; i++) + if (!filter_supported(&filter->sub[i])) + return 0; + return 1; + default: + return 0; + } +} + static int filter_bitmap(struct bitmap_index *bitmap_git, struct object_list *tip_objects, struct bitmap *to_filter, @@ -932,6 +955,8 @@ static int filter_bitmap(struct bitmap_index *bitmap_git, { if (!filter || filter->choice == LOFC_DISABLED) return 0; + if (!filter_supported(filter)) + return -1; if (filter->choice == LOFC_BLOB_NONE) { if (bitmap_git) @@ -948,8 +973,7 @@ static int filter_bitmap(struct bitmap_index *bitmap_git, return 0; } - if (filter->choice == LOFC_TREE_DEPTH && - filter->tree_exclude_depth == 0) { + if (filter->choice == LOFC_TREE_DEPTH) { if (bitmap_git) filter_bitmap_tree_depth(bitmap_git, tip_objects, to_filter, @@ -965,8 +989,16 @@ static int filter_bitmap(struct bitmap_index *bitmap_git, return 0; } - /* filter choice not handled */ - return -1; + if (filter->choice == LOFC_COMBINE) { + int i; + for (i = 0; i < filter->sub_nr; i++) { + filter_bitmap(bitmap_git, tip_objects, to_filter, + &filter->sub[i]); + } + return 0; + } + + BUG("unsupported filter choice"); } static int can_filter_bitmap(struct list_objects_filter_options *filter) diff --git a/t/t6113-rev-list-bitmap-filters.sh b/t/t6113-rev-list-bitmap-filters.sh index fb66735ac8..cb9db7df6f 100755 --- a/t/t6113-rev-list-bitmap-filters.sh +++ b/t/t6113-rev-list-bitmap-filters.sh @@ -98,4 +98,11 @@ test_expect_success 'object:type filter' ' test_bitmap_traversal expect actual ' +test_expect_success 'combine filter' ' + git rev-list --objects --filter=blob:limit=1000 --filter=object:type=blob tag >expect && + git rev-list --use-bitmap-index \ + --objects --filter=blob:limit=1000 --filter=object:type=blob tag >actual && + test_bitmap_traversal expect actual +' + test_done From patchwork Mon Mar 15 13:15:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 12139227 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9806BC43619 for ; Mon, 15 Mar 2021 13:16:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5159A64F0C for ; Mon, 15 Mar 2021 13:16:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230332AbhCONPc (ORCPT ); Mon, 15 Mar 2021 09:15:32 -0400 Received: from out4-smtp.messagingengine.com ([66.111.4.28]:50723 "EHLO out4-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230142AbhCONPH (ORCPT ); Mon, 15 Mar 2021 09:15:07 -0400 Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id 7BB335C00AD; Mon, 15 Mar 2021 09:15:07 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Mon, 15 Mar 2021 09:15:07 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=date :from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=fm3; bh=DR61WtGk8bsm/kzRdxcNYB6QvQI qcUutzgQmPh1sYbA=; b=Gzo8S7RYB0xIAAt0knrezIxVCHEqAenH67mNipy3KLC McPoycCnvEIgrlgJzPqFJdtHptD6Cb06k9pVckS+oruPKUY6ZfJQ3G5B+otxk6Hx rWjG4iGTkNLuBQ8ztz/iZ1poPGYneUj/LujOeOG15sYGg/eQ788HTbWQvll6z35j NCF6Ua8a5BA2iue8whl/adm3TzNm9TNWCgGiBXbIX8hl4UOnwYJ7rZ9IJg+Zqf2U RKmef2v9NEUREVLU4ZRLu7VXxe1K8WrgznN+IrNQsqk5wWCrrIk5hUXJ9XcPH4KB uI9tDMFrp+nZwZFoKXke1y5plqqI9vSsbBJtNgGzwdw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=DR61Wt Gk8bsm/kzRdxcNYB6QvQIqcUutzgQmPh1sYbA=; b=mvIjtUeEejIydV6DY5Yrb8 fzDIyNDpK9Zq7VPPmLrbbgFecByjOT9y/+4j/hgMQdAVPB4w3wILTpsKMAiAEZt7 sE2IAsZmU98S2ZeTAYuapdIaXqG8cF2ZS9Ssza236ZHGJdKzR6e0O80emBtMDet3 O1m72G+5ypzVkX6tNhQOiJ+kP6QIBRKHyrGxTngreGlZM/hrDZomY18BfKnqfWex q/0vTIFOErXz0SMgDweclCMQWWClcmjBUZOKRlPFfAysG7wgqhsE0hc0lwNXSNcr RoOxcWIySH+AQKaJt38nyuCcx89EDZsApK17bbyZ5/tOpd7B4qMB+J683nPW7ibg == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledruddvledggeekucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvffukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefrrghtrhhi tghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtthgvrh hnpefgfeejuefgheegueeljeffteefuedthfdvvedugfevvdffleduueeiiefgvedtheen ucffohhmrghinhepphgvnhguihhnghdrnhhrnecukfhppeekledrudegrdeliedrvdegud enucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehpshes phhkshdrihhm X-ME-Proxy: Received: from vm-mail.pks.im (dynamic-089-014-096-241.89.14.pool.telefonica.de [89.14.96.241]) by mail.messagingengine.com (Postfix) with ESMTPA id 76CE7108006C; Mon, 15 Mar 2021 09:15:06 -0400 (EDT) Received: from localhost (ncase [10.192.0.11]) by vm-mail.pks.im (OpenSMTPD) with ESMTPSA id 4c037072 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 15 Mar 2021 13:15:06 +0000 (UTC) Date: Mon, 15 Mar 2021 14:15:05 +0100 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Christian Couder , Taylor Blau , Jeff King Subject: [PATCH v2 8/8] rev-list: allow filtering of provided items Message-ID: <0e26fee8b31e46e87fb9fa1ac599506502a9d622.1615813673.git.ps@pks.im> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When providing an object filter, it is currently impossible to also filter provided items. E.g. when executing `git rev-list HEAD` , the commit this reference points to will be treated as user-provided and is thus excluded from the filtering mechanism. This makes it harder than necessary to properly use the new `--filter=object:type` filter given that even if the user wants to only see blobs, he'll still see commits of provided references. Improve this by introducing a new `--filter-provided` option to the git-rev-parse(1) command. If given, then all user-provided references will be subject to filtering. Signed-off-by: Patrick Steinhardt --- builtin/rev-list.c | 14 +++++++++++ list-objects-filter-options.c | 4 ++++ list-objects-filter-options.h | 6 +++++ pack-bitmap.c | 3 ++- t/t6112-rev-list-filters-objects.sh | 28 ++++++++++++++++++++++ t/t6113-rev-list-bitmap-filters.sh | 36 +++++++++++++++++++++++++++++ 6 files changed, 90 insertions(+), 1 deletion(-) diff --git a/builtin/rev-list.c b/builtin/rev-list.c index b4d8ea0a35..0f959b266d 100644 --- a/builtin/rev-list.c +++ b/builtin/rev-list.c @@ -599,6 +599,10 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix) list_objects_filter_set_no_filter(&filter_options); continue; } + if (!strcmp(arg, "--filter-provided")) { + filter_options.filter_wants = 1; + continue; + } if (!strcmp(arg, "--filter-print-omitted")) { arg_print_omitted = 1; continue; @@ -694,6 +698,16 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix) return show_bisect_vars(&info, reaches, all); } + if (filter_options.filter_wants) { + struct commit_list *c; + for (i = 0; i < revs.pending.nr; i++) { + struct object_array_entry *pending = revs.pending.objects + i; + pending->item->flags |= NOT_USER_GIVEN; + } + for (c = revs.commits; c; c = c->next) + c->item->object.flags |= NOT_USER_GIVEN; + } + if (arg_print_omitted) oidset_init(&omitted_objects, DEFAULT_OIDSET_SIZE); if (arg_missing_action == MA_PRINT) diff --git a/list-objects-filter-options.c b/list-objects-filter-options.c index bb6f6577d5..2877aa9e96 100644 --- a/list-objects-filter-options.c +++ b/list-objects-filter-options.c @@ -242,6 +242,7 @@ static void transform_to_combine_type( memset(filter_options, 0, sizeof(*filter_options)); filter_options->sub = sub_array; filter_options->sub_alloc = initial_sub_alloc; + filter_options->filter_wants = sub_array[0].filter_wants; } filter_options->sub_nr = 1; filter_options->choice = LOFC_COMBINE; @@ -290,6 +291,9 @@ void parse_list_objects_filter( parse_error = gently_parse_list_objects_filter( &filter_options->sub[filter_options->sub_nr - 1], arg, &errbuf); + if (!parse_error) + filter_options->sub[filter_options->sub_nr - 1].filter_wants = + filter_options->filter_wants; } if (parse_error) die("%s", errbuf.buf); diff --git a/list-objects-filter-options.h b/list-objects-filter-options.h index 4d0d0588cc..5e609e307a 100644 --- a/list-objects-filter-options.h +++ b/list-objects-filter-options.h @@ -42,6 +42,12 @@ struct list_objects_filter_options { */ enum list_objects_filter_choice choice; + /* + * "--filter-provided" was given by the user, instructing us to also + * filter all explicitly provided objects. + */ + unsigned int filter_wants : 1; + /* * Choice is LOFC_DISABLED because "--no-filter" was requested. */ diff --git a/pack-bitmap.c b/pack-bitmap.c index e33805e076..5ff800316b 100644 --- a/pack-bitmap.c +++ b/pack-bitmap.c @@ -1101,7 +1101,8 @@ struct bitmap_index *prepare_bitmap_walk(struct rev_info *revs, if (haves_bitmap) bitmap_and_not(wants_bitmap, haves_bitmap); - filter_bitmap(bitmap_git, wants, wants_bitmap, filter); + filter_bitmap(bitmap_git, (filter && filter->filter_wants) ? NULL : wants, + wants_bitmap, filter); bitmap_git->result = wants_bitmap; bitmap_git->haves = haves_bitmap; diff --git a/t/t6112-rev-list-filters-objects.sh b/t/t6112-rev-list-filters-objects.sh index c79ec04060..47c558ab0e 100755 --- a/t/t6112-rev-list-filters-objects.sh +++ b/t/t6112-rev-list-filters-objects.sh @@ -207,6 +207,34 @@ test_expect_success 'verify object:type=tag prints tag' ' test_cmp expected actual ' +test_expect_success 'verify object:type=blob prints only blob with --filter-provided' ' + printf "%s blob\n" $(git -C object-type rev-parse HEAD:blob) >expected && + git -C object-type rev-list --objects \ + --filter=object:type=blob --filter-provided HEAD >actual && + test_cmp expected actual +' + +test_expect_success 'verify object:type=tree prints only tree with --filter-provided' ' + printf "%s \n" $(git -C object-type rev-parse HEAD^{tree}) >expected && + git -C object-type rev-list --objects \ + --filter=object:type=tree HEAD --filter-provided >actual && + test_cmp expected actual +' + +test_expect_success 'verify object:type=commit prints only commit with --filter-provided' ' + git -C object-type rev-parse HEAD >expected && + git -C object-type rev-list --objects \ + --filter=object:type=commit --filter-provided HEAD >actual && + test_cmp expected actual +' + +test_expect_success 'verify object:type=tag prints only tag with --filter-provided' ' + printf "%s tag\n" $(git -C object-type rev-parse tag) >expected && + git -C object-type rev-list --objects \ + --filter=object:type=tag --filter-provided tag >actual && + test_cmp expected actual +' + # Test sparse:path= filter. # !!!! # NOTE: sparse:path filter support has been dropped for security reasons, diff --git a/t/t6113-rev-list-bitmap-filters.sh b/t/t6113-rev-list-bitmap-filters.sh index cb9db7df6f..9053ac5059 100755 --- a/t/t6113-rev-list-bitmap-filters.sh +++ b/t/t6113-rev-list-bitmap-filters.sh @@ -98,6 +98,28 @@ test_expect_success 'object:type filter' ' test_bitmap_traversal expect actual ' +test_expect_success 'object:type filter with --filter-provided' ' + git rev-list --objects --filter-provided --filter=object:type=tag tag >expect && + git rev-list --use-bitmap-index \ + --objects --filter-provided --filter=object:type=tag tag >actual && + test_cmp expect actual && + + git rev-list --objects --filter-provided --filter=object:type=commit tag >expect && + git rev-list --use-bitmap-index \ + --objects --filter-provided --filter=object:type=commit tag >actual && + test_bitmap_traversal expect actual && + + git rev-list --objects --filter-provided --filter=object:type=tree tag >expect && + git rev-list --use-bitmap-index \ + --objects --filter-provided --filter=object:type=tree tag >actual && + test_bitmap_traversal expect actual && + + git rev-list --objects --filter-provided --filter=object:type=blob tag >expect && + git rev-list --use-bitmap-index \ + --objects --filter-provided --filter=object:type=blob tag >actual && + test_bitmap_traversal expect actual +' + test_expect_success 'combine filter' ' git rev-list --objects --filter=blob:limit=1000 --filter=object:type=blob tag >expect && git rev-list --use-bitmap-index \ @@ -105,4 +127,18 @@ test_expect_success 'combine filter' ' test_bitmap_traversal expect actual ' +test_expect_success 'combine filter with --filter-provided' ' + git rev-list --objects --filter-provided --filter=blob:limit=1000 --filter=object:type=blob tag >expect && + git rev-list --use-bitmap-index \ + --objects --filter-provided --filter=blob:limit=1000 --filter=object:type=blob tag >actual && + test_bitmap_traversal expect actual && + + git cat-file --batch-check="%(objecttype) %(objectsize)" objects && + while read objecttype objectsize + do + test "$objecttype" = blob || return 1 + test "$objectsize" -le 1000 || return 1 + done