From patchwork Mon May 2 17:08:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Calvin Wan X-Patchwork-Id: 12834523 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0624FC433EF for ; Mon, 2 May 2022 17:09:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1386555AbiEBRNW (ORCPT ); Mon, 2 May 2022 13:13:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52394 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1386544AbiEBRNV (ORCPT ); Mon, 2 May 2022 13:13:21 -0400 Received: from mail-pf1-x449.google.com (mail-pf1-x449.google.com [IPv6:2607:f8b0:4864:20::449]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6088D5F55 for ; Mon, 2 May 2022 10:09:52 -0700 (PDT) Received: by mail-pf1-x449.google.com with SMTP id z34-20020a056a001da200b0050e057fdd7eso338112pfw.12 for ; Mon, 02 May 2022 10:09:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=b7Or1K2HTHUnCI3ox1uTSfm7WCDiVHJa+NiADeIjCnk=; b=qFXNIlsmpOmEmCEM85qdV7AXwqiB/UVsaYQT6Rf3t4Y3hzcNzK/yW1jDyV/c4EDY2X lhu0hj0ESmqeYZkErdo2zEdUYeSqAPF13nsjQi0KfbjAXMY0MQvJunrZ78otqGlGVpGz /aGUT4RNG04/4F1FpvQKK0nKQHWqWyBMCJrIoffwl9Ij4RNONcdv7WmlznjuiiMDiuQ4 Tc/rhavuRLfilVaDHFwHJv1VN56aq2CU2Yxsw5epeQW0VDcAdl3FADkkC4WuxNR0WtsL FmBKV55kD8YhDQIS/KuDT0mpzJn6hZckmoejDT36gfSTap7JIVICDVWP1TDHWxn2RBK3 PFow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=b7Or1K2HTHUnCI3ox1uTSfm7WCDiVHJa+NiADeIjCnk=; b=3Bbe30UR1/95U8PYAo4CmzwEH/iBd/i1+6MxJhWHaB33Kx8hL6xWIi0+TV3+M0gpUC n2SlRjo3JEFFpH1YqmmURGaq1aZ7Ch9OFo05xRfWPxVosUFq/hMbMRei/RWw+/7aB8FV mQvytqBAeEs+yOLhxNkPWEy7Jr/vPntDeA5ckP0MK6qYROLH2kVDMA4OLvaY/PkQapVg OLbxw1J6uncU/v4Pyb1daJvnHWc31OpwFp2MO8jX4JVl/m57dels6KH/Ha2halI2gR5Y aiURh4nWC42vkW/ePpXT2Le4oms9CbBrkV9hRzXdmMMUybNTgcaUZljLWmb0x5xmDAZG PWXA== X-Gm-Message-State: AOAM531HTrsL89ekfYHlzPlcerI05Le6j2Tot9hqhkPfe3eSWnkDUwpq mEyGuSuSxNXkF6bJbF0wpLYsySO0amD3WW8HMJGwvb03C3fSi79TiJNvnmqA6o6iwWeqpYmmpio H+d7hICGL/bXHsc7bP2XQKHcWyM2znATI1YH+Yw5bFLeXpP/xwNWI8Nz0+l/umq47Xg== X-Google-Smtp-Source: ABdhPJw8JU3ATo3DpKzQhCadXTpRrqpO4i5wESjvbiY0qcLZropsSJHPcxUMNR5hJGWw6U36Cb79YbbKmLdEJcI= X-Received: from barleywine.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:3bd4]) (user=calvinwan job=sendgmr) by 2002:a05:6a00:890:b0:4f6:686e:a8a9 with SMTP id q16-20020a056a00089000b004f6686ea8a9mr12007737pfj.83.1651511391636; Mon, 02 May 2022 10:09:51 -0700 (PDT) Date: Mon, 2 May 2022 17:08:57 +0000 In-Reply-To: <20220502170904.2770649-1-calvinwan@google.com> Message-Id: <20220502170904.2770649-2-calvinwan@google.com> Mime-Version: 1.0 References: <20220328191112.3092139-1-calvinwan@google.com> <20220502170904.2770649-1-calvinwan@google.com> X-Mailer: git-send-email 2.36.0.rc2.10170.gb555eefa6f Subject: [PATCH v4 1/8] fetch-pack: refactor packet writing From: Calvin Wan To: git@vger.kernel.org Cc: gitster@pobox.com, jonathantanmy@google.com, philipoakley@iee.email, johncai86@gmail.com, calvinwan@google.com, me@ttaylorr.com Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org A subsequent patch will need to write capabilities for another command, so refactor write_fetch_command_and_capabilities() to be used by both fetch and future command. --- fetch-pack.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/fetch-pack.c b/fetch-pack.c index 4e1e88eea0..e06125c90a 100644 --- a/fetch-pack.c +++ b/fetch-pack.c @@ -1246,13 +1246,13 @@ static int add_haves(struct fetch_negotiator *negotiator, return haves_added; } -static void write_fetch_command_and_capabilities(struct strbuf *req_buf, - const struct string_list *server_options) +static void write_command_and_capabilities(struct strbuf *req_buf, + const struct string_list *server_options, const char* command) { const char *hash_name; - if (server_supports_v2("fetch", 1)) - packet_buf_write(req_buf, "command=fetch"); + if (server_supports_v2(command, 1)) + packet_buf_write(req_buf, "command=%s", command); if (server_supports_v2("agent", 0)) packet_buf_write(req_buf, "agent=%s", git_user_agent_sanitized()); if (advertise_sid && server_supports_v2("session-id", 0)) @@ -1288,7 +1288,7 @@ static int send_fetch_request(struct fetch_negotiator *negotiator, int fd_out, int done_sent = 0; struct strbuf req_buf = STRBUF_INIT; - write_fetch_command_and_capabilities(&req_buf, args->server_options); + write_command_and_capabilities(&req_buf, args->server_options, "fetch"); if (args->use_thin_pack) packet_buf_write(&req_buf, "thin-pack"); @@ -2072,7 +2072,7 @@ void negotiate_using_fetch(const struct oid_array *negotiation_tips, int received_ready = 0; strbuf_reset(&req_buf); - write_fetch_command_and_capabilities(&req_buf, server_options); + write_command_and_capabilities(&req_buf, server_options, "fetch"); packet_buf_write(&req_buf, "wait-for-done"); From patchwork Mon May 2 17:08:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Calvin Wan X-Patchwork-Id: 12834526 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 796F8C433EF for ; Mon, 2 May 2022 17:10:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1386569AbiEBRNi (ORCPT ); Mon, 2 May 2022 13:13:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52636 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1381800AbiEBRNd (ORCPT ); Mon, 2 May 2022 13:13:33 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3C84A5F76 for ; Mon, 2 May 2022 10:09:59 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id b12-20020a056902030c00b0061d720e274aso13608680ybs.20 for ; Mon, 02 May 2022 10:09:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=FRPEd1jdsVWBIW/HZyF5sMLEXA3CuDpuEK/14caxkEs=; b=j9VEGXp8W8KUzC4fagOoBaldQUWj7Z7LjbgaCvj5HnzEfbdkRAO0sxpTE09xXKsNkM DcEKVe6donu4131ag0ERNWUVxdbY8jKTxY6n3N4ANnUskn3sC+cqC0uNtAsrCXjva5bK udr6q6vle/vO/VjCMWx7PB5kTuMfBkimMWqwXweuABnX2wvUl9z90ko7ZWjTcHxp13lK hg0aU8NvZH5tqaazyY8py1gSNPVs4/m3DaP6ruA8oogQPof5o+3/8yPEQ02F2WIFKXFn Kr95BzY88+ezR3OzvVVZoKJPgQszmr6hMF8oI+IAztW+PisAKTBeF8VRp3HY7M5WkgmN f6bA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=FRPEd1jdsVWBIW/HZyF5sMLEXA3CuDpuEK/14caxkEs=; b=F/e5mRNhhcDmxET+dbeUS8B9uZM/Yvy0dkm0JbmRzPTvQtbxVwDzKGZDB3RLOzvTNT FoYqyktavMHmkSvRGH7dHVyzEEg0rZDFga9cYqbjjatiu52ZJNeiTZ3l4d/EgCaMjZKR g19mFEdky9HldAVgER5xlrgNWxV4QpLtwzwbK8nRJ0PPrNaQqcDaoicdO7XWRF9m46c9 QS6wHaExhch+qBELpZN7a3tnlW33qHnnFfIkB78PDRLzLggP+GNMjnzLUPNqANMAw2VU CxBYyx7qPlnt+3QWpnoAfdxvi+ZTtmBe5cGXi6Nr/8DmuDMQsUhVWFkC8P0AqnjwrWhy oVWA== X-Gm-Message-State: AOAM533Lbb78GZRv7gzIwLQOgW3f46BBeqf/irAe6ysBTcfrdVW+zL/Z 3gffDhMj5jbQhp/cg6wn7rBEfdN+MuDH/u/DDBzntYD9I1nU8uKquwe4h6YrXaj/OeumZHsLDOo w/lhe3K5mN6+lHflvu4UK5ckB0PwafNhBhxrZL5fy8UPrHqbksWwlqWEKvZ70c+wyXg== X-Google-Smtp-Source: ABdhPJz1m6WMB1esuo53zcHOAO1Pkv221pE43FHaoJ9MmP9Mmstt6dUFKmswCmvp4K5vhaem0j13XZrBYICG7nM= X-Received: from barleywine.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:3bd4]) (user=calvinwan job=sendgmr) by 2002:a25:841:0:b0:641:960f:c47f with SMTP id 62-20020a250841000000b00641960fc47fmr10742725ybi.607.1651511398341; Mon, 02 May 2022 10:09:58 -0700 (PDT) Date: Mon, 2 May 2022 17:08:58 +0000 In-Reply-To: <20220502170904.2770649-1-calvinwan@google.com> Message-Id: <20220502170904.2770649-3-calvinwan@google.com> Mime-Version: 1.0 References: <20220328191112.3092139-1-calvinwan@google.com> <20220502170904.2770649-1-calvinwan@google.com> X-Mailer: git-send-email 2.36.0.rc2.10170.gb555eefa6f Subject: [PATCH v4 2/8] fetch-pack: move fetch default settings From: Calvin Wan To: git@vger.kernel.org Cc: gitster@pobox.com, jonathantanmy@google.com, philipoakley@iee.email, johncai86@gmail.com, calvinwan@google.com, me@ttaylorr.com Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When the state machine in do_fetch_pack_v2() is in FETCH_CHECK_LOCAL, we set a few v2-specific defaults. It will be helpful for a future patch to have these defaults set if the initial state is not FETCH_CHECK_LOCAL. This is a safe change since the initial state is currently always FETCH_CHECK_LOCAL, so we're guaranteed to hit that code whether it's in the state machine or not. --- fetch-pack.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/fetch-pack.c b/fetch-pack.c index e06125c90a..45473b4602 100644 --- a/fetch-pack.c +++ b/fetch-pack.c @@ -1610,18 +1610,18 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args, reader.me = "fetch-pack"; } + /* v2 supports these by default */ + allow_unadvertised_object_request |= ALLOW_REACHABLE_SHA1; + use_sideband = 2; + if (args->depth > 0 || args->deepen_since || args->deepen_not) + args->deepen = 1; + while (state != FETCH_DONE) { switch (state) { case FETCH_CHECK_LOCAL: sort_ref_list(&ref, ref_compare_name); QSORT(sought, nr_sought, cmp_ref_by_name); - /* v2 supports these by default */ - allow_unadvertised_object_request |= ALLOW_REACHABLE_SHA1; - use_sideband = 2; - if (args->depth > 0 || args->deepen_since || args->deepen_not) - args->deepen = 1; - /* Filter 'ref' by 'sought' and those that aren't local */ mark_complete_and_common_ref(negotiator, args, &ref); filter_refs(args, &ref, sought, nr_sought); From patchwork Mon May 2 17:08:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Calvin Wan X-Patchwork-Id: 12834524 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 40CA3C433F5 for ; Mon, 2 May 2022 17:10:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1353028AbiEBRNc (ORCPT ); Mon, 2 May 2022 13:13:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52612 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1386544AbiEBRNb (ORCPT ); Mon, 2 May 2022 13:13:31 -0400 Received: from mail-yw1-x1149.google.com (mail-yw1-x1149.google.com [IPv6:2607:f8b0:4864:20::1149]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B7AB95FE6 for ; Mon, 2 May 2022 10:10:01 -0700 (PDT) Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-2f7c322f770so140074037b3.20 for ; Mon, 02 May 2022 10:10:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=0Qd7+3fsF3+7ZN+LU+unprZr4AWRLqqP/7TZGEeVq78=; b=YDjFGM1iFZ2Kuup9myfFjPFs5TQ5RRF0QXyEXx067Q5Xh8vrTle9pbSZA2avSjpbZ3 fFdqiNmwPSQImIZRV8DNaSw5KNlzo4zW/WbwgAjzueY5KjU8M2CHqhsQBHJMmMQtvWER 2ArrNG2Ys6jk1Uwbt8dZM+Bv4GpXnJ+OrbMdjYOiZ70RdZx+P3nEEuxzbPdDcM/ZAO0k IWJVK+xA0f9cm9RAGhkxU/JtllKhGcdLhYvptFihklbdgXuMMSOz64TmVelsBjuzkHC0 L7UnRRxwDr9LY97rhbmtrri6xdsdgZYrZSa3IXfOT8xQLEOlBJ2hGjGYFTXbk2aauY3b Z0Xg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=0Qd7+3fsF3+7ZN+LU+unprZr4AWRLqqP/7TZGEeVq78=; b=dobpTZsbcoSIs0qULbjDrDytOd2sM77iGDnw6GbxxUyuH4TrsfoBMyRo4iuFE5OLbs crm1asBd35kg2QVpOHZ+X2943wT9xVM26ZwBbudhgjFZc7SBeyf4gjzlns8W8btKAG5l 34iY90VNJjv4B3fRvcpPPlb1DGAi2PStnzSRtt6/zANuf27yr+RBju5wvAvx3uCDmvtK qCd3xnVWpmE58o48IJ0gs4Vg9ZNZhD1zxXHijr9YeYTrX3GwNX/u1OrS9lf8fzomzsiB zf1R9ezVtiapEfZTiuZHFZ6DTM0p0qZJVtIMW0Kuz6F0xiG4sGGJsljwxQPquRS0Tfw+ 5adw== X-Gm-Message-State: AOAM531ehaXwmZxp/Damwqh8fx4wvBDmItnfsJyCK2v1W0h2XrITF2eQ Ew6XdWwuQ2xqCIHIBKbm6IrPWOMm9OSHrDzhJhzXm38B4PdzsvzEIrnKIu7gwAbj/DckAdIuRzM EH79jdtB4ZeLJQWter2YAPKJ3R+Z0hYFao3F5CPl3nJ4Wpwx1ycu4omg+zgpdcRKAbw== X-Google-Smtp-Source: ABdhPJxzAEsFJmegEEMiLeQ0VVBsVvUhQ7wJESa2KRFsMOy2uElp4FQE5ThZHQCIR68uNIFJBT2nzEmvKYnEbJU= X-Received: from barleywine.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:3bd4]) (user=calvinwan job=sendgmr) by 2002:a81:78ce:0:b0:2f8:fbad:c446 with SMTP id t197-20020a8178ce000000b002f8fbadc446mr682538ywc.498.1651511400904; Mon, 02 May 2022 10:10:00 -0700 (PDT) Date: Mon, 2 May 2022 17:08:59 +0000 In-Reply-To: <20220502170904.2770649-1-calvinwan@google.com> Message-Id: <20220502170904.2770649-4-calvinwan@google.com> Mime-Version: 1.0 References: <20220328191112.3092139-1-calvinwan@google.com> <20220502170904.2770649-1-calvinwan@google.com> X-Mailer: git-send-email 2.36.0.rc2.10170.gb555eefa6f Subject: [PATCH v4 3/8] object-store: add function to free object_info contents From: Calvin Wan To: git@vger.kernel.org Cc: gitster@pobox.com, jonathantanmy@google.com, philipoakley@iee.email, johncai86@gmail.com, calvinwan@google.com, me@ttaylorr.com Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Introduce free_object_info_contents. --- object-file.c | 16 ++++++++++++++++ object-store.h | 3 +++ 2 files changed, 19 insertions(+) diff --git a/object-file.c b/object-file.c index 5ffbf3d4fd..34a6e34adf 100644 --- a/object-file.c +++ b/object-file.c @@ -2659,3 +2659,19 @@ int read_loose_object(const char *path, munmap(map, mapsize); return ret; } + +void free_object_info_contents(struct object_info *object_info) +{ + if (!object_info) + return; + if (object_info->typep) + free(object_info->typep); + if (object_info->sizep) + free(object_info->sizep); + if (object_info->disk_sizep) + free(object_info->disk_sizep); + if (object_info->delta_base_oid) + free(object_info->delta_base_oid); + if (object_info->type_name) + free(object_info->type_name); +} \ No newline at end of file diff --git a/object-store.h b/object-store.h index bd2322ed8c..840e04b56f 100644 --- a/object-store.h +++ b/object-store.h @@ -533,4 +533,7 @@ int for_each_object_in_pack(struct packed_git *p, int for_each_packed_object(each_packed_object_fn, void *, enum for_each_object_flags flags); +/* Free pointers inside of object_info, but not object_info itself */ +void free_object_info_contents(struct object_info *object_info); + #endif /* OBJECT_STORE_H */ From patchwork Mon May 2 17:09:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Calvin Wan X-Patchwork-Id: 12834525 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2BA3EC433F5 for ; Mon, 2 May 2022 17:10:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1386562AbiEBRNg (ORCPT ); Mon, 2 May 2022 13:13:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52624 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1386556AbiEBRNc (ORCPT ); Mon, 2 May 2022 13:13:32 -0400 Received: from mail-pf1-x44a.google.com (mail-pf1-x44a.google.com [IPv6:2607:f8b0:4864:20::44a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8E7615F55 for ; Mon, 2 May 2022 10:10:03 -0700 (PDT) Received: by mail-pf1-x44a.google.com with SMTP id 15-20020aa7920f000000b0050cf449957fso8335730pfo.9 for ; Mon, 02 May 2022 10:10:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=E8Ra9hXify+H+FfVI+GyGahKHJm8UfQuVYMQkEJVd3s=; b=IVRbrDefF53GfBWEEmcpBVkbnn3JGxt+8mEXAAWgSut7BYOTb0iT3putgApDHV8VQm 7dVozJM0WXFfol1BtiMOY7V/rriCo+y4NllE9iJKDCHB5fCKRCjkXMC33HpkoSeLw6h9 uEMaWDfNtdTuTpppBat7z94IzfCZ4+DCEiuyvqmXZFKQepRQgJkxm1WJCmQoCgTk1dEQ 7Oo5f+OEuL9S/OrM5R27UHVpaaY0tZDgp/7MZxxJnU+odHx68TBTfLBBvgNB25WqFPVg +VtcnNnaRZ9CRj23AVNCtmK15gx2MStR658WF/YlR5fHgkcmhPoGbCkyPkIt8EvrVmJ9 dcRQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=E8Ra9hXify+H+FfVI+GyGahKHJm8UfQuVYMQkEJVd3s=; b=jp627i/laQnvpZWTtxuk78ARnZpPNdp3IKW/w50PkmNDzUvfUBCbCRqOF9Lo/+psEa 7E6FWsv5HJdqIOMvs/XYl86vKcpbFM/qTLl7tfXANmQ6PE1bLiiQbK87F498jDT6Ost5 gfVJMRU0T16ZkM7E4LDdhRTZjFJ2nEg7eySsvuYAgTa+gUBa5FCqq7FQcfo19SBASZtb oy7GuT20BA9z96aOcpUZAsel45AfSO4WFi+T6q9wDr/hq5ZtygMJDUCXgpgZaZxHtrHS bq6/+qMh/pLOBVZQCzHodz37xgKvX6jkX49tL30ACc1QFEQF0A0+26YbmH+P6nUqW4jz Gn2g== X-Gm-Message-State: AOAM5311t3/lObxQL9QCGP4ffYrwkKmgf9R44VAaw28z1ltKfzKnnWqC /dkk7gb/pK+aM4q9y94qS7nPi7Makerjx4vzJlIb+YM0jJ+K44xkCJ/6Q80DLm16QwliogtSg6c 1C/r6Hi9KtsgzMg+vckhl6vrhUadYJjF/B26FRoKO9mW0CWPbarzuqha8cjIKLZqpTg== X-Google-Smtp-Source: ABdhPJyJCNfF1ldn3c1z7rR8I+KIdjB3M9YkWSJOnkAwuj32Ce9U5RYuOfFPS+F1ASsKSm0w0yonPfUSd7w5plk= X-Received: from barleywine.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:3bd4]) (user=calvinwan job=sendgmr) by 2002:a05:6a00:e8e:b0:4fa:a52f:59cf with SMTP id bo14-20020a056a000e8e00b004faa52f59cfmr12380294pfb.84.1651511402943; Mon, 02 May 2022 10:10:02 -0700 (PDT) Date: Mon, 2 May 2022 17:09:00 +0000 In-Reply-To: <20220502170904.2770649-1-calvinwan@google.com> Message-Id: <20220502170904.2770649-5-calvinwan@google.com> Mime-Version: 1.0 References: <20220328191112.3092139-1-calvinwan@google.com> <20220502170904.2770649-1-calvinwan@google.com> X-Mailer: git-send-email 2.36.0.rc2.10170.gb555eefa6f Subject: [PATCH v4 4/8] object-info: send attribute packet regardless of object ids From: Calvin Wan To: git@vger.kernel.org Cc: gitster@pobox.com, jonathantanmy@google.com, philipoakley@iee.email, johncai86@gmail.com, calvinwan@google.com, me@ttaylorr.com Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Currently on the server side of object-info, if the client does not send any object ids in the request or if the client requests an attribute the server does not support, the server will either not send any packets back or error out. Consider the scenario where the client git version is ahead of the server git version, and the client can request additional object-info besides 'size'. There needs to be a way to tell whether the server can honor all of the client attribute requests before the client sends a request with all of the object ids. In a future patch, if the client were to make an initial command request with only attributes, the server would be able to confirm which attributes it could return. --- protocol-caps.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/protocol-caps.c b/protocol-caps.c index bbde91810a..bc7def0727 100644 --- a/protocol-caps.c +++ b/protocol-caps.c @@ -11,6 +11,7 @@ struct requested_info { unsigned size : 1; + unsigned unknown : 1; }; /* @@ -40,12 +41,12 @@ static void send_info(struct repository *r, struct packet_writer *writer, struct string_list_item *item; struct strbuf send_buffer = STRBUF_INIT; - if (!oid_str_list->nr) - return; - if (info->size) packet_writer_write(writer, "size"); + if (info->unknown || !oid_str_list->nr) + goto release; + for_each_string_list_item (item, oid_str_list) { const char *oid_str = item->string; struct object_id oid; @@ -72,12 +73,13 @@ static void send_info(struct repository *r, struct packet_writer *writer, packet_writer_write(writer, "%s", send_buffer.buf); strbuf_reset(&send_buffer); } +release: strbuf_release(&send_buffer); } int cap_object_info(struct repository *r, struct packet_reader *request) { - struct requested_info info; + struct requested_info info = { 0 }; struct packet_writer writer; struct string_list oid_str_list = STRING_LIST_INIT_DUP; @@ -92,9 +94,7 @@ int cap_object_info(struct repository *r, struct packet_reader *request) if (parse_oid(request->line, &oid_str_list)) continue; - packet_writer_error(&writer, - "object-info: unexpected line: '%s'", - request->line); + info.unknown = 1; } if (request->status != PACKET_READ_FLUSH) { From patchwork Mon May 2 17:09:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Calvin Wan X-Patchwork-Id: 12834527 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B821CC433F5 for ; Mon, 2 May 2022 17:10:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1386572AbiEBRNk (ORCPT ); Mon, 2 May 2022 13:13:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52694 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1386558AbiEBRNf (ORCPT ); Mon, 2 May 2022 13:13:35 -0400 Received: from mail-pf1-x44a.google.com (mail-pf1-x44a.google.com [IPv6:2607:f8b0:4864:20::44a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9E0B65F76 for ; Mon, 2 May 2022 10:10:05 -0700 (PDT) Received: by mail-pf1-x44a.google.com with SMTP id b184-20020a62cfc1000000b0050d209cb8dcso8346964pfg.3 for ; Mon, 02 May 2022 10:10:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc:content-transfer-encoding; bh=gIGDHPziLWSVgKkyF2vUur9VylGh6NVOnERCjmA4MhA=; b=Mi0zT+HE/noM7lTy07tbBYupqPRehpPflyJ0rVStliz073aLEIABvcI2SgvvV4WLGq LAM1nNOeiZ/NN0X0GTqB3ygtiQ0M7aewHF1hoZ1cMAL8HpZokiqXkuVz0VGzFlKqHY+1 viLPaCRK7CgKpDStQdVmjnZyd851mKX9Pls6G9ZtHdEx/4OrnIvgtdxgijfZgT9xgjAR miNR9tDZB15AAoxt9WvQ6dhdFH6oCUPu3wdNAum7O/afzrDlpGJFB6OsXShP+WjjeuiD r/rvbqqSVY0lrfpr47O8viOT4ErvGK6ISZgxwGMbxsExEPigCAhPG5GxPlZQiS5ywnZH lqoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc:content-transfer-encoding; bh=gIGDHPziLWSVgKkyF2vUur9VylGh6NVOnERCjmA4MhA=; b=ODtdIgjLeCkgIxFTX7kputPJxPR5d/6VcxbvHfiu+0LNsfwyMOg3jLbr5FWXci5UZi S91grx2KezCInTM9D48eL4+3WUqtszaTE/yrTvismgM6DIgXUMWxj4LvSh7NXMFE1FEB 3ZGuhLdGIvts5cyndJfH06EjBGezuIVO+CO+PSLGV9jd0aNQ1tKalXaFLozlFaxhyL7s 9Z5pJ2+GinFxFyQvUDxKxoIXCG796JsEsI/aU/h4czuLMlW6ejMepQ9C90t8YwHwrZmt 8FBY0lSX58PoyJwh4KEvxay6LKPvg0opiFqNOtXD4EWBVt6WTHOpJUplnU+LEp/5b03+ jMrA== X-Gm-Message-State: AOAM532JJ85BQLdTsXhMCLh91pdmoYl7yROOCiRjEfMZvwONE+Hf6lvQ STHoTceOpqEbPIznvRJr73JQ1H7uHZMVMcVV54aUlXos+WGKiDDcFWdKH2wDTt2de+0kDKOzTH4 yoB/ZolGFraFROqmaXssNs0DnoXHn51UNEjd21tQ/+yEdN3LMKPBUlF7kGnvUdOyiJQ== X-Google-Smtp-Source: ABdhPJzGX0txm4lXKPpVRcrIZpIm4cyZPU1X2oJ7jqPjVobmleNTq1/9ieX4ZbTPPjMggInjL7ArgwlOYenCgR0= X-Received: from barleywine.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:3bd4]) (user=calvinwan job=sendgmr) by 2002:a17:902:8c8f:b0:15e:ab1c:591b with SMTP id t15-20020a1709028c8f00b0015eab1c591bmr3877301plo.171.1651511404766; Mon, 02 May 2022 10:10:04 -0700 (PDT) Date: Mon, 2 May 2022 17:09:01 +0000 In-Reply-To: <20220502170904.2770649-1-calvinwan@google.com> Message-Id: <20220502170904.2770649-6-calvinwan@google.com> Mime-Version: 1.0 References: <20220328191112.3092139-1-calvinwan@google.com> <20220502170904.2770649-1-calvinwan@google.com> X-Mailer: git-send-email 2.36.0.rc2.10170.gb555eefa6f Subject: [PATCH v4 5/8] transport: add client side capability to request object-info From: Calvin Wan To: git@vger.kernel.org Cc: gitster@pobox.com, jonathantanmy@google.com, philipoakley@iee.email, johncai86@gmail.com, calvinwan@google.com, me@ttaylorr.com Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Sometimes it is useful to get information about an object without having to download it completely. The server logic has already been implemented as “a2ba162cda (object-info: support for retrieving object info, 2021-04-20)”. This patch implements the client option for it. Currently, only 'size' is supported and the server must be v2, however future patches can implement additional options. The question of version mismatch often comes up with client/server relationships. There are two cases to consider here (assuming either client or server functionality for object-info changes between the different versions): 1. client version > server version - client can request additional attributes from server 2. server version > client version - server can return additional info to the client The second case is a non-issue since the client would never be able to request additional info from the server. In order to solve the first case, recall an earlier patch where the server sends back the attributes even if no object ids are sent by the client. This allows the client to first check whether the server can accept the requested attributes before sending the entire request. --- fetch-pack.c | 23 +++++++++++++ fetch-pack.h | 8 +++++ transport-helper.c | 7 ++-- transport.c | 81 +++++++++++++++++++++++++++++++++++++++++++++- transport.h | 11 +++++++ 5 files changed, 127 insertions(+), 3 deletions(-) diff --git a/fetch-pack.c b/fetch-pack.c index 45473b4602..506ca28e05 100644 --- a/fetch-pack.c +++ b/fetch-pack.c @@ -1278,6 +1278,29 @@ static void write_command_and_capabilities(struct strbuf *req_buf, packet_buf_delim(req_buf); } +void send_object_info_request(int fd_out, struct object_info_args *args) +{ + struct strbuf req_buf = STRBUF_INIT; + struct string_list unsorted_object_info_options = *args->object_info_options; + size_t i; + + write_command_and_capabilities(&req_buf, args->server_options, "object-info"); + + if (unsorted_string_list_has_string(&unsorted_object_info_options, "size")) + packet_buf_write(&req_buf, "size"); + + if (args->oids) { + for (i = 0; i < args->oids->nr; i++) + packet_buf_write(&req_buf, "oid %s", oid_to_hex(&args->oids->oid[i])); + } + + packet_buf_flush(&req_buf); + if (write_in_full(fd_out, req_buf.buf, req_buf.len) < 0) + die_errno(_("unable to write request to remote")); + + strbuf_release(&req_buf); +} + static int send_fetch_request(struct fetch_negotiator *negotiator, int fd_out, struct fetch_pack_args *args, const struct ref *wants, struct oidset *common, diff --git a/fetch-pack.h b/fetch-pack.h index 8c7752fc82..c27018d48c 100644 --- a/fetch-pack.h +++ b/fetch-pack.h @@ -69,6 +69,12 @@ struct fetch_pack_args { unsigned connectivity_checked:1; }; +struct object_info_args { + const struct string_list *object_info_options; + const struct string_list *server_options; + const struct oid_array *oids; +}; + /* * sought represents remote references that should be updated from. * On return, the names that were found on the remote will have been @@ -102,4 +108,6 @@ void negotiate_using_fetch(const struct oid_array *negotiation_tips, */ int report_unmatched_refs(struct ref **sought, int nr_sought); +void send_object_info_request(int fd_out, struct object_info_args *args); + #endif diff --git a/transport-helper.c b/transport-helper.c index b4dbbabb0c..093946f7fd 100644 --- a/transport-helper.c +++ b/transport-helper.c @@ -686,13 +686,16 @@ static int fetch_refs(struct transport *transport, /* * If we reach here, then the server, the client, and/or the transport - * helper does not support protocol v2. --negotiate-only requires - * protocol v2. + * helper does not support protocol v2. --negotiate-only and --object-info + * require protocol v2. */ if (data->transport_options.acked_commits) { warning(_("--negotiate-only requires protocol v2")); return -1; } + if (transport->smart_options->object_info) { + die(_("--object-info requires protocol v2")); + } if (!data->get_refs_list_called) get_refs_list_using_list(transport, 0); diff --git a/transport.c b/transport.c index 3d64a43ab3..08c505e1d0 100644 --- a/transport.c +++ b/transport.c @@ -353,6 +353,79 @@ static struct ref *handshake(struct transport *transport, int for_push, return refs; } +static int fetch_object_info(struct transport *transport, struct object_info **object_info_data) +{ + size_t i; + int size_index = -1; + struct git_transport_data *data = transport->data; + struct object_info_args args; + struct packet_reader reader; + struct string_list server_attributes = STRING_LIST_INIT_DUP; + + memset(&args, 0, sizeof(args)); + args.server_options = transport->server_options; + args.object_info_options = transport->smart_options->object_info_options; + + connect_setup(transport, 0); + packet_reader_init(&reader, data->fd[0], NULL, 0, + PACKET_READ_CHOMP_NEWLINE | + PACKET_READ_DIE_ON_ERR_PACKET); + data->version = discover_version(&reader); + + transport->hash_algo = reader.hash_algo; + + switch (data->version) { + case protocol_v2: + if (!server_supports_v2("object-info", 0)) + return -1; + /** + * Send a request with only attributes first. If server can return all + * of the requested attributes, then send request with object ids + */ + send_object_info_request(data->fd[1], &args); + if (packet_reader_read(&reader) != PACKET_READ_NORMAL) { + check_stateless_delimiter(transport->stateless_rpc, &reader, "stateless delimiter expected"); + return -1; + } + string_list_split(&server_attributes, reader.line, ' ', -1); + packet_reader_read(&reader); + check_stateless_delimiter(transport->stateless_rpc, &reader, "stateless delimiter expected"); + if (server_attributes.nr != args.object_info_options->nr) + return -1; + for (i = 0; i < server_attributes.nr; i++) { + if (!strcmp(server_attributes.items[i].string, "size")) + size_index = i + 1; + } + args.oids = transport->smart_options->object_info_oids; + send_object_info_request(data->fd[1], &args); + break; + case protocol_v1: + case protocol_v0: + die(_("wrong protocol version. expected v2")); + case protocol_unknown_version: + BUG("unknown protocol version"); + } + + if (packet_reader_read(&reader) != PACKET_READ_NORMAL) + die(_("error reading object info header")); + i = 0; + while (packet_reader_read(&reader) == PACKET_READ_NORMAL) { + struct string_list object_info_values = STRING_LIST_INIT_DUP; + + string_list_split(&object_info_values, reader.line, ' ', -1); + if (size_index > 0) { + if (!strcmp(object_info_values.items[size_index].string, "")) + die("object-info: not our ref %s", + object_info_values.items[0].string); + *(*object_info_data)[i].sizep = strtoul(object_info_values.items[size_index].string, NULL, 10); + } + i++; + } + check_stateless_delimiter(transport->stateless_rpc, &reader, "stateless delimiter expected"); + + return 0; +} + static struct ref *get_refs_via_connect(struct transport *transport, int for_push, struct transport_ls_refs_options *options) { @@ -392,8 +465,14 @@ static int fetch_refs_via_pack(struct transport *transport, args.server_options = transport->server_options; args.negotiation_tips = data->options.negotiation_tips; args.reject_shallow_remote = transport->smart_options->reject_shallow; + args.object_info = transport->smart_options->object_info; - if (!data->got_remote_heads) { + if (transport->smart_options && transport->smart_options->object_info) { + if (!fetch_object_info(transport, data->options.object_info_data)) + goto cleanup; + ret = -1; + goto cleanup; + } else if (!data->got_remote_heads) { int i; int must_list_refs = 0; for (i = 0; i < nr_heads; i++) { diff --git a/transport.h b/transport.h index 12bc08fc33..dbf60bb710 100644 --- a/transport.h +++ b/transport.h @@ -6,6 +6,7 @@ #include "remote.h" #include "list-objects-filter-options.h" #include "string-list.h" +#include "object-store.h" struct git_transport_options { unsigned thin : 1; @@ -31,6 +32,12 @@ struct git_transport_options { */ unsigned connectivity_checked:1; + /* + * Transport will attempt to pull only object-info. Fallbacks + * to pulling entire object if object-info is not supported + */ + unsigned object_info : 1; + int depth; const char *deepen_since; const struct string_list *deepen_not; @@ -54,6 +61,10 @@ struct git_transport_options { * common commits to this oidset instead of fetching any packfiles. */ struct oidset *acked_commits; + + struct oid_array *object_info_oids; + struct object_info **object_info_data; + struct string_list *object_info_options; }; enum transport_family { From patchwork Mon May 2 17:09:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Calvin Wan X-Patchwork-Id: 12834529 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6EE8FC433EF for ; Mon, 2 May 2022 17:10:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1386586AbiEBRNr (ORCPT ); Mon, 2 May 2022 13:13:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52748 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1386568AbiEBRNh (ORCPT ); Mon, 2 May 2022 13:13:37 -0400 Received: from mail-pg1-x54a.google.com (mail-pg1-x54a.google.com [IPv6:2607:f8b0:4864:20::54a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6CD5860EB for ; Mon, 2 May 2022 10:10:07 -0700 (PDT) Received: by mail-pg1-x54a.google.com with SMTP id s68-20020a637747000000b003aaff19b95bso7271086pgc.1 for ; Mon, 02 May 2022 10:10:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=akw57T62qzLu5XOJTDKT0BdT9BIQ15RZPIpdPddN9QM=; b=elJI615I4aBweTgPpWNQhercIxGX6Nbx2SjQmVWb224EntEio3tc8CjOHCiYY/t2P3 wTKrlI0Nmz0qgjzrk/HoHQ9BEed0TJT8j1GL4CYFgts2QfBlTaNsmPWq673HkbMrR/Jf zDm2Fy69MtWYLxPfkBAYuQEVv4xj7lb1ERCmEmzir0dMjCwNJrb2yMzgAOiVc57pNtla vord8HbP/01REK057sdxh2oQXZey/DuRa87ZYipgBqEqyPaFp7LGaDidAyK8kahTbyZr pDsTcLUqtVcQuQ1f5CfQijhv43HvevrYeuLfXL0EokRm9B9Ibkop07wyp5KqNQPcCBSa dqbQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=akw57T62qzLu5XOJTDKT0BdT9BIQ15RZPIpdPddN9QM=; b=bK5k0YL5KdSS177oXiXgqF2cXukjfE4+vGA+ihTwgL3B7UW7aAk4LKeZGWR8X4B2SO /gELl9kr1ROxuOP+c/e2MUtjQ+kp3zDw8PCER2nQOE6t8/ahA0jFSPWnbg5iR9oEriVy IX2kkJ3sJxKye4KDAEKxaN+n4/La10h0TNbpx+FKbK4EO9/hueLIYjukFvicI9KmZU1F e8im9nQT/5wqsNZ8/EhXoiiuBPNfNU5hqaiu7aRgjPIdxwY3WwYPbwxTfKtuNPLE4a/e nIkWt9NpSWTbEs3MGk8cnyBosTqm2Mp4+1F1XxfJ/+LKk4+ITb+bFFaE3p3hhfhq4UaZ n6jQ== X-Gm-Message-State: AOAM533Z+eE8iKWB3zRIZLcCJPdDSflrxXzm0VfLjx3vIkqeDK0Mritc GQaIU0wVEw2TW6/yeOSmceusR3WmiceGWAGGyaSVYwupCqW9v+otaoX514VlkCu6/C0VXnBxIh3 39ZmuRkC+NtpdrOBMsZ7XqpR4RbmpWqZzPPz5DSIxsqDQ033nTPtf2eYu7lz+HdifXw== X-Google-Smtp-Source: ABdhPJy6MUXqJbLzphsUMHhLFd+3zRY0yyIydxnLPChv+7vb/ZP9j798yoQzdxFR+Qu41mpfo4RcdbDLzvNIvPc= X-Received: from barleywine.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:3bd4]) (user=calvinwan job=sendgmr) by 2002:a17:902:d2d2:b0:15e:b28d:8acc with SMTP id n18-20020a170902d2d200b0015eb28d8accmr1576958plc.4.1651511406748; Mon, 02 May 2022 10:10:06 -0700 (PDT) Date: Mon, 2 May 2022 17:09:02 +0000 In-Reply-To: <20220502170904.2770649-1-calvinwan@google.com> Message-Id: <20220502170904.2770649-7-calvinwan@google.com> Mime-Version: 1.0 References: <20220328191112.3092139-1-calvinwan@google.com> <20220502170904.2770649-1-calvinwan@google.com> X-Mailer: git-send-email 2.36.0.rc2.10170.gb555eefa6f Subject: [PATCH v4 6/8] transport: add object-info fallback to fetch From: Calvin Wan To: git@vger.kernel.org Cc: gitster@pobox.com, jonathantanmy@google.com, philipoakley@iee.email, johncai86@gmail.com, calvinwan@google.com, me@ttaylorr.com Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org If a v2 server does not support object-info or if the client requests information that isn't supported by object-info, fetch the objects as if it were a standard v2 fetch (however without changing any refs). --- fetch-pack.c | 14 +++++++++++++- fetch-pack.h | 2 ++ transport.c | 18 +++++++++++++++--- 3 files changed, 30 insertions(+), 4 deletions(-) diff --git a/fetch-pack.c b/fetch-pack.c index 506ca28e05..938d082b80 100644 --- a/fetch-pack.c +++ b/fetch-pack.c @@ -1639,6 +1639,9 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args, if (args->depth > 0 || args->deepen_since || args->deepen_not) args->deepen = 1; + if (args->object_info) + state = FETCH_SEND_REQUEST; + while (state != FETCH_DONE) { switch (state) { case FETCH_CHECK_LOCAL: @@ -1648,7 +1651,7 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args, /* Filter 'ref' by 'sought' and those that aren't local */ mark_complete_and_common_ref(negotiator, args, &ref); filter_refs(args, &ref, sought, nr_sought); - if (!args->refetch && everything_local(args, &ref)) + if (!args->refetch && !args->object_info && everything_local(args, &ref)) state = FETCH_DONE; else state = FETCH_SEND_REQUEST; @@ -2035,6 +2038,15 @@ struct ref *fetch_pack(struct fetch_pack_args *args, args->connectivity_checked = 1; } + if (args->object_info) { + struct ref *ref_cpy_reader = ref_cpy; + int i = 0; + while (ref_cpy_reader) { + oid_object_info_extended(the_repository, &ref_cpy_reader->old_oid, &(*args->object_info_data)[i], OBJECT_INFO_LOOKUP_REPLACE); + ref_cpy_reader = ref_cpy_reader->next; + i++; + } + } update_shallow(args, sought, nr_sought, &si); cleanup: clear_shallow_info(&si); diff --git a/fetch-pack.h b/fetch-pack.h index c27018d48c..552fd7bde0 100644 --- a/fetch-pack.h +++ b/fetch-pack.h @@ -17,6 +17,7 @@ struct fetch_pack_args { const struct string_list *deepen_not; struct list_objects_filter_options filter_options; const struct string_list *server_options; + struct object_info **object_info_data; /* * If not NULL, during packfile negotiation, fetch-pack will send "have" @@ -43,6 +44,7 @@ struct fetch_pack_args { unsigned reject_shallow_remote:1; unsigned deepen:1; unsigned refetch:1; + unsigned object_info:1; /* * Indicate that the remote of this request is a promisor remote. The diff --git a/transport.c b/transport.c index 08c505e1d0..b85e52d9a8 100644 --- a/transport.c +++ b/transport.c @@ -436,10 +436,12 @@ static int fetch_refs_via_pack(struct transport *transport, int nr_heads, struct ref **to_fetch) { int ret = 0; + size_t i; struct git_transport_data *data = transport->data; struct ref *refs = NULL; struct fetch_pack_args args; struct ref *refs_tmp = NULL; + struct ref *wanted_refs = xcalloc(1, sizeof (struct ref)); memset(&args, 0, sizeof(args)); args.uploadpack = data->options.uploadpack; @@ -468,10 +470,19 @@ static int fetch_refs_via_pack(struct transport *transport, args.object_info = transport->smart_options->object_info; if (transport->smart_options && transport->smart_options->object_info) { + struct ref *ref = wanted_refs; + if (!fetch_object_info(transport, data->options.object_info_data)) goto cleanup; - ret = -1; - goto cleanup; + args.object_info_data = data->options.object_info_data; + for (i = 0; i < transport->smart_options->object_info_oids->nr; i++) { + struct ref *temp_ref = xcalloc(1, sizeof (struct ref)); + temp_ref->old_oid = *(transport->smart_options->object_info_oids->oid + i); + temp_ref->exact_oid = 1; + ref->next = temp_ref; + ref = ref->next; + } + transport->remote_refs = wanted_refs->next; } else if (!data->got_remote_heads) { int i; int must_list_refs = 0; @@ -489,7 +500,7 @@ static int fetch_refs_via_pack(struct transport *transport, else if (data->version <= protocol_v1) die_if_server_options(transport); - if (data->options.acked_commits) { + if (data->options.acked_commits && !transport->smart_options->object_info) { if (data->version < protocol_v2) { warning(_("--negotiate-only requires protocol v2")); ret = -1; @@ -532,6 +543,7 @@ static int fetch_refs_via_pack(struct transport *transport, free_refs(refs_tmp); free_refs(refs); + free_refs(wanted_refs); return ret; } From patchwork Mon May 2 17:09:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Calvin Wan X-Patchwork-Id: 12834528 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 968DFC433FE for ; Mon, 2 May 2022 17:10:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1386591AbiEBRNt (ORCPT ); Mon, 2 May 2022 13:13:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52776 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1382485AbiEBRNi (ORCPT ); Mon, 2 May 2022 13:13:38 -0400 Received: from mail-pl1-x649.google.com (mail-pl1-x649.google.com [IPv6:2607:f8b0:4864:20::649]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 47A3655AC for ; Mon, 2 May 2022 10:10:09 -0700 (PDT) Received: by mail-pl1-x649.google.com with SMTP id u8-20020a170903124800b0015195a5826cso6811652plh.4 for ; Mon, 02 May 2022 10:10:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=MG17PURULN3OASD5uWrKve8WRbCujnWrVG6t7FyK5kQ=; b=W3qepcNfGdhTl7z+s5sfvFqJMwoulRvQ9FtWNSlTa+g+4Jtc7nPJxjxrl0x1PfCJ41 Z3/hgeFoWCOulX5dOTsrb1IS3iToEmrU9dyPtlEE9GPzrcrJqiR9h/Ful2q7lhhA0R8/ PIPrqIMDXZpauEnwh+HgP9rS5PRrI4TEANFEFsX6kXuHhEBHigZlAGLFNQwKPCNIK4sh ZM9M5bqQWPg6/Nc7AGDRgkKFvPWCUnOss7MITaxrVygOxxup3i7LyLzaroNOB//wjIws gFM1kdsI9mAY6GwY4is6y/0GY+yw4PSHK5ACUVFP1nvBkrLgv6qbr1cZFs7QpPMfKFDv b0Rg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=MG17PURULN3OASD5uWrKve8WRbCujnWrVG6t7FyK5kQ=; b=J2cg4qViZnKkQtPSgVa7b8A/NoNyS6IRvabeeQWnSGi67UzT6SpvkPkOiRaRdnKjfj rG9S1813MwMLD/Jzz/vYsA/NsY/BDJ/kz9dENyqIyEAGZW0jy70GFcGXACywgqExr1ek onOC0uoX+CZ5dbr38oFnB10L43wi+4JIsker2ty3FV3jUexn0N/RmZq2KRek5WvtIiSQ DoqR/RfY7qqBhPhoCxtPvmLjAHsC8Ddaz1ACdXPl3kbAeKqMzgS1OireEb8G6KNNWS3P yww9ILGa/eac7Ld7I/hjARhr3rhJc+U9X2iPSa0WWKVOjU90d87Wm+4WSJ27VMle4pjo s69w== X-Gm-Message-State: AOAM533ScbQlAt4HkSVbt98E0vi9rXOHtwgkBnkb6p8TWuT5lHlcRPlO OlkpweQmc5uKFnxsJOkxhiCyYl01DKRWyG+TFnaIncT8ZYBsF5fLBTk9lrZBapnWhxxVTqMR1Q6 LDHYE73DpweO53g3WOQnUG+ZbVguXcyD/Qwi3ly9qioB9FITgmWFzrBtqyITLV+ay6w== X-Google-Smtp-Source: ABdhPJz8nn7UMES9r57zY45u2o3tECGPHszZJuJt5S492PSHdyfe9K1nGOIkdNu4cavDY3RdRNyhCF5nmuJD++w= X-Received: from barleywine.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:3bd4]) (user=calvinwan job=sendgmr) by 2002:a05:6a00:1acb:b0:50d:fa52:ced with SMTP id f11-20020a056a001acb00b0050dfa520cedmr2730801pfv.76.1651511408653; Mon, 02 May 2022 10:10:08 -0700 (PDT) Date: Mon, 2 May 2022 17:09:03 +0000 In-Reply-To: <20220502170904.2770649-1-calvinwan@google.com> Message-Id: <20220502170904.2770649-8-calvinwan@google.com> Mime-Version: 1.0 References: <20220328191112.3092139-1-calvinwan@google.com> <20220502170904.2770649-1-calvinwan@google.com> X-Mailer: git-send-email 2.36.0.rc2.10170.gb555eefa6f Subject: [PATCH v4 7/8] cat-file: move parse_cmd and DEFAULT_FORMAT up From: Calvin Wan To: git@vger.kernel.org Cc: gitster@pobox.com, jonathantanmy@google.com, philipoakley@iee.email, johncai86@gmail.com, calvinwan@google.com, me@ttaylorr.com Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org It would be useful for a future patch to use these definitions so moving them higher. --- builtin/cat-file.c | 25 ++++++++++++------------- 1 file changed, 12 insertions(+), 13 deletions(-) diff --git a/builtin/cat-file.c b/builtin/cat-file.c index 50cf38999d..cfe74c36eb 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -34,6 +34,8 @@ struct batch_options { const char *format; }; +#define DEFAULT_FORMAT "%(objectname) %(objecttype) %(objectsize)" + static const char *force_path; static int filter_object(const char *path, unsigned mode, @@ -556,6 +558,16 @@ static void parse_cmd_info(struct batch_options *opt, batch_one_object(line, output, opt, data); } +static const struct parse_cmd { + const char *name; + parse_cmd_fn_t fn; + unsigned takes_args; +} commands[] = { + { "contents", parse_cmd_contents, 1}, + { "info", parse_cmd_info, 1}, + { "flush", NULL, 0}, +}; + static void dispatch_calls(struct batch_options *opt, struct strbuf *output, struct expand_data *data, @@ -583,17 +595,6 @@ static void free_cmds(struct queued_cmd *cmd, size_t *nr) *nr = 0; } - -static const struct parse_cmd { - const char *name; - parse_cmd_fn_t fn; - unsigned takes_args; -} commands[] = { - { "contents", parse_cmd_contents, 1}, - { "info", parse_cmd_info, 1}, - { "flush", NULL, 0}, -}; - static void batch_objects_command(struct batch_options *opt, struct strbuf *output, struct expand_data *data) @@ -659,8 +660,6 @@ static void batch_objects_command(struct batch_options *opt, strbuf_release(&input); } -#define DEFAULT_FORMAT "%(objectname) %(objecttype) %(objectsize)" - static int batch_objects(struct batch_options *opt) { struct strbuf input = STRBUF_INIT; From patchwork Mon May 2 17:09:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Calvin Wan X-Patchwork-Id: 12834530 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A5A8DC433F5 for ; Mon, 2 May 2022 17:10:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1386625AbiEBRNw (ORCPT ); Mon, 2 May 2022 13:13:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53130 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1386576AbiEBRNn (ORCPT ); Mon, 2 May 2022 13:13:43 -0400 Received: from mail-pg1-x54a.google.com (mail-pg1-x54a.google.com [IPv6:2607:f8b0:4864:20::54a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2E1AB1F6 for ; Mon, 2 May 2022 10:10:10 -0700 (PDT) Received: by mail-pg1-x54a.google.com with SMTP id bj12-20020a056a02018c00b003a9eebaad34so7249326pgb.10 for ; Mon, 02 May 2022 10:10:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=KIcdDExwiOWtYbZVdamkiwlgjINW/QWj55/Di02BHm0=; b=aQbTnGzeZDsznn/QOzGgYr5NAkhqwt3KNSUttzrfH38UqUwSfeXot+dJLC0oQb25Kf HZY9c2JfiWS8jGiynCxB0MGKqcRlt5XYkYJ9VQbq8qQ+fp1y8wfmQ4IbMfD7aBGknfl4 UL587XrhnP/GVJrQf0kmni8H9u2rTtAXxpSymfDnVEbsFi+w4dkZGk9PcQoWfJZpIOpE /qpz7J+JzRTjD3SAfXcoXpxwz+ckYpzjauRFJiVpA1Nhwpb40uBMfYaCoiFlUod5l5vm GGSOTFJz+EIKogBwiU7RRFJEDL/HVlTTJ4EIRyvSxji/QCpuGi1KSN9BdWcBpLOI6P8w 2SZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=KIcdDExwiOWtYbZVdamkiwlgjINW/QWj55/Di02BHm0=; b=tmisa3yQWprwG/0430aP7cztObcOH96jK/ozQm8M1drREGy4AXztbk3VC2BwIJ0obF xkS5InRgLJMkgEITNrtH4O0kWIkWR7WInGzGJH2n8Mmqwt3aUutniQ/3ep8UmO3OLMEq wFYzk1ALGu1oHpA/ahwBTrPy6q0+P5nCyzPnAjglaOy7yWd+NkLPOl1yuWN0k+V/ld0N FjRa+ZfwIewN0xqOWjltATY1ZaRQEi998DVS2VQNNMl2HbvBJ3q013ySu2dTN4yvQf4f jk57uLxS64X2y1xkU7eYtDkBt3xAxXojq+Ww7NmM6145x3qnWDUmDPWVIOZlzl0p/g1k yc6w== X-Gm-Message-State: AOAM530mQFgo2D6sDeCXduh4fQHXTRVZThwMBDibA66+isSiy0rfYfT6 ueyv9v5h4eY0bglRpyOXXJlBs4YGW6VJdpnIFHFPqBot5NS+aL/mm0GJmB+/Gmu2UjkLz8/uhXU sQUOWN7LCxfNVHuUZY5pkCdMgNf+bCexT4kSfFmgIPxjKrUhqwYdHojmaSVPEGMZO3Q== X-Google-Smtp-Source: ABdhPJwIN8EF6c3qMfxiQNwyo1XcYI195ognyBhAEBhEaB5ZS2HeiiR7FRpgeET10o9XLXHspqzkSAsoXJUld0M= X-Received: from barleywine.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:3bd4]) (user=calvinwan job=sendgmr) by 2002:a17:903:1105:b0:156:3e9d:bb7b with SMTP id n5-20020a170903110500b001563e9dbb7bmr12313348plh.136.1651511410280; Mon, 02 May 2022 10:10:10 -0700 (PDT) Date: Mon, 2 May 2022 17:09:04 +0000 In-Reply-To: <20220502170904.2770649-1-calvinwan@google.com> Message-Id: <20220502170904.2770649-9-calvinwan@google.com> Mime-Version: 1.0 References: <20220328191112.3092139-1-calvinwan@google.com> <20220502170904.2770649-1-calvinwan@google.com> X-Mailer: git-send-email 2.36.0.rc2.10170.gb555eefa6f Subject: [PATCH v4 8/8] cat-file: add --batch-command remote-object-info command From: Calvin Wan To: git@vger.kernel.org Cc: gitster@pobox.com, jonathantanmy@google.com, philipoakley@iee.email, johncai86@gmail.com, calvinwan@google.com, me@ttaylorr.com Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Since the `info` command in cat-file --batch-command prints object info for a given object, it is natural to add another command in cat-file --batch-command to print object info for a given object from a remote. Add `remote-object-info` to cat-file --batch-command. While `info` takes object ids one at a time, this creates overhead when making requests to a server so `remote-object-info` instead can take multiple object ids at once. cat-file --batch-command is generally implemented in the following manner: - Receive and parse input from user - Call respective function attached to command - Set batch mode state, get object info, print object info In --buffer mode, this changes to: - Receive and parse input from user - Store respective function attached to command in a queue - After flush, loop through commands in queue - Call respective function attached to command - Set batch mode state, get object info, print object info Notice how the getting and printing of object info is accomplished one at a time. As described above, this creates a problem for making requests to a server. Therefore, `remote-object-info` is implemented in the following manner: - Receive and parse input from user If command is `remote-object-info`: - Get object info from remote - Loop through object info - Call respective function attached to `info` - Set batch mode state, use passed in object info, print object info Else: - Call respective function attached to command - Parse input, get object info, print object info And finally for --buffer mode `remote-object-info`: - Receive and parse input from user - Store respective function attached to command in a queue - After flush, loop through commands in queue: If command is `remote-object-info`: - Get object info from remote - Loop through object info - Call respective function attached to `info` - Set batch mode state, use passed in object info, print object info Else: - Call respective function attached to command - Set batch mode state, get object info, print object info To summarize, `remote-object-info` gets object info from the remote and then generates multiples `info` commands with the object info passed in. It cannot be implemented similarly to `info` and `content` because of the overhead with remote commenication. --- Documentation/git-cat-file.txt | 16 +- builtin/cat-file.c | 200 +++++++++++++++---- t/t1006-cat-file.sh | 347 +++++++++++++++++++++++++++++++++ 3 files changed, 522 insertions(+), 41 deletions(-) diff --git a/Documentation/git-cat-file.txt b/Documentation/git-cat-file.txt index 24a811f0ef..0d9e8e6214 100644 --- a/Documentation/git-cat-file.txt +++ b/Documentation/git-cat-file.txt @@ -115,6 +115,10 @@ info :: Print object info for object reference ``. This corresponds to the output of `--batch-check`. +remote-object-info []:: + Print object info for object references `` at specified . + This command may only be combined with `--buffer`. + flush:: Used with `--buffer` to execute all preceding commands that were issued since the beginning or since the last flush was issued. When `--buffer` @@ -245,7 +249,8 @@ newline. The available atoms are: The full hex representation of the object name. `objecttype`:: - The type of the object (the same as `cat-file -t` reports). + The type of the object (the same as `cat-file -t` reports). See + `CAVEATS` below. `objectsize`:: The size, in bytes, of the object (the same as `cat-file -s` @@ -253,13 +258,14 @@ newline. The available atoms are: `objectsize:disk`:: The size, in bytes, that the object takes up on disk. See the - note about on-disk sizes in the `CAVEATS` section below. + note about on-disk sizes in the `CAVEATS` section below. Not + supported by `remote-object-info`. `deltabase`:: If the object is stored as a delta on-disk, this expands to the full hex representation of the delta base object name. Otherwise, expands to the null OID (all zeroes). See `CAVEATS` - below. + below. Not supported by `remote-object-info`. `rest`:: If this atom is used in the output string, input lines are split @@ -346,6 +352,10 @@ directory name. CAVEATS ------- +Note that since object type is currently not supported by the +object-info command request, git fetches the entire object instead to +get the object info. + Note that the sizes of objects on disk are reported accurately, but care should be taken in drawing conclusions about which refs or objects are responsible for disk usage. The size of a packed non-delta object may be diff --git a/builtin/cat-file.c b/builtin/cat-file.c index cfe74c36eb..7baa9bed61 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -16,6 +16,9 @@ #include "packfile.h" #include "object-store.h" #include "promisor-remote.h" +#include "alias.h" +#include "remote.h" +#include "transport.h" enum batch_mode { BATCH_MODE_CONTENTS, @@ -32,11 +35,14 @@ struct batch_options { int unordered; int transform_mode; /* may be 'w' or 'c' for --filters or --textconv */ const char *format; + int use_remote_info; }; #define DEFAULT_FORMAT "%(objectname) %(objecttype) %(objectsize)" static const char *force_path; +static struct object_info *remote_object_info; +static struct oid_array object_info_oids = OID_ARRAY_INIT; static int filter_object(const char *path, unsigned mode, const struct object_id *oid, @@ -425,48 +431,115 @@ static void batch_one_object(const char *obj_name, int flags = opt->follow_symlinks ? GET_OID_FOLLOW_SYMLINKS : 0; enum get_oid_result result; - result = get_oid_with_context(the_repository, obj_name, - flags, &data->oid, &ctx); - if (result != FOUND) { - switch (result) { - case MISSING_OBJECT: - printf("%s missing\n", obj_name); - break; - case SHORT_NAME_AMBIGUOUS: - printf("%s ambiguous\n", obj_name); - break; - case DANGLING_SYMLINK: - printf("dangling %"PRIuMAX"\n%s\n", - (uintmax_t)strlen(obj_name), obj_name); - break; - case SYMLINK_LOOP: - printf("loop %"PRIuMAX"\n%s\n", - (uintmax_t)strlen(obj_name), obj_name); - break; - case NOT_DIR: - printf("notdir %"PRIuMAX"\n%s\n", - (uintmax_t)strlen(obj_name), obj_name); - break; - default: - BUG("unknown get_sha1_with_context result %d\n", - result); - break; + if (!opt->use_remote_info) { + result = get_oid_with_context(the_repository, obj_name, + flags, &data->oid, &ctx); + if (result != FOUND) { + switch (result) { + case MISSING_OBJECT: + printf("%s missing\n", obj_name); + break; + case SHORT_NAME_AMBIGUOUS: + printf("%s ambiguous\n", obj_name); + break; + case DANGLING_SYMLINK: + printf("dangling %"PRIuMAX"\n%s\n", + (uintmax_t)strlen(obj_name), obj_name); + break; + case SYMLINK_LOOP: + printf("loop %"PRIuMAX"\n%s\n", + (uintmax_t)strlen(obj_name), obj_name); + break; + case NOT_DIR: + printf("notdir %"PRIuMAX"\n%s\n", + (uintmax_t)strlen(obj_name), obj_name); + break; + default: + BUG("unknown get_sha1_with_context result %d\n", + result); + break; + } + fflush(stdout); + return; } - fflush(stdout); - return; - } - if (ctx.mode == 0) { - printf("symlink %"PRIuMAX"\n%s\n", - (uintmax_t)ctx.symlink_path.len, - ctx.symlink_path.buf); - fflush(stdout); - return; + if (ctx.mode == 0) { + printf("symlink %"PRIuMAX"\n%s\n", + (uintmax_t)ctx.symlink_path.len, + ctx.symlink_path.buf); + fflush(stdout); + return; + } } batch_object_write(obj_name, scratch, opt, data, NULL, 0); } +static int get_remote_info(struct batch_options *opt, int argc, const char **argv) +{ + int retval = 0; + size_t i; + struct remote *remote = NULL; + struct object_id oid; + struct string_list object_info_options = STRING_LIST_INIT_NODUP; + static struct transport *gtransport; + char *format = DEFAULT_FORMAT; + + if (opt->format) + format = xstrdup(opt->format); + + remote = remote_get(argv[0]); + if (!remote) + die(_("must supply valid remote when using --object-info")); + oid_array_clear(&object_info_oids); + for (i = 1; i < argc; i++) { + if (get_oid(argv[i], &oid)) + die(_("malformed object id '%s'"), argv[i]); + oid_array_append(&object_info_oids, &oid); + } + + gtransport = transport_get(remote, NULL); + if (gtransport->smart_options) { + size_t j; + int include_size = 0, include_type = 0; + + remote_object_info = xcalloc(object_info_oids.nr, sizeof(struct object_info)); + gtransport->smart_options->object_info = 1; + gtransport->smart_options->object_info_oids = &object_info_oids; + /** + * 'size' is the only option currently supported. + * Other options that are passed in the format will default to a + * standard fetch request rather than object-info. + */ + if (strstr(format, "%(objectsize)")) { + string_list_append(&object_info_options, "size"); + include_size = 1; + } + if (strstr(format, "%(objecttype)")) { + string_list_append(&object_info_options, "type"); + include_type = 1; + } + if (strstr(format, "%(objectsize:disk)")) + die(_("objectsize:disk is currently not supported with remote-object-info")); + if (strstr(format, "%(deltabase)")) + die(_("deltabase is currently not supported with remote-object-info")); + if (object_info_options.nr > 0) { + gtransport->smart_options->object_info_options = &object_info_options; + for (j = 0; j < object_info_oids.nr; j++) { + if (include_size) + remote_object_info[j].sizep = xcalloc(1, sizeof(long)); + if (include_type) + remote_object_info[j].typep = xcalloc(1, sizeof(enum object_type)); + } + gtransport->smart_options->object_info_data = &remote_object_info; + retval = transport_fetch_refs(gtransport, NULL); + } + } else { + retval = -1; + } + return retval; +} + struct object_cb_data { struct batch_options *opt; struct expand_data *expand; @@ -538,6 +611,7 @@ typedef void (*parse_cmd_fn_t)(struct batch_options *, const char *, struct queued_cmd { parse_cmd_fn_t fn; char *line; + const char *name; }; static void parse_cmd_contents(struct batch_options *opt, @@ -565,9 +639,49 @@ static const struct parse_cmd { } commands[] = { { "contents", parse_cmd_contents, 1}, { "info", parse_cmd_info, 1}, + { "remote-object-info", parse_cmd_info, 1}, { "flush", NULL, 0}, }; +static void parse_remote_info(struct batch_options *opt, + char *line, + struct strbuf *output, + struct expand_data *data, + const struct parse_cmd *p_cmd, + struct queued_cmd *q_cmd) +{ + int count; + size_t i; + const char **argv; + + count = split_cmdline(line, &argv); + if (get_remote_info(opt, count, argv)) + goto cleanup; + opt->use_remote_info = 1; + data->skip_object_info = 1; + data->mark_query = 0; + for (i = 0; i < object_info_oids.nr; i++) { + if (remote_object_info[i].sizep) + data->size = *remote_object_info[i].sizep; + if (remote_object_info[i].typep) + data->type = *remote_object_info[i].typep; + + data->oid = object_info_oids.oid[i]; + if (p_cmd) + p_cmd->fn(opt, argv[i+1], output, data); + else + q_cmd->fn(opt, argv[i+1], output, data); + } + opt->use_remote_info = 0; + data->skip_object_info = 0; + data->mark_query = 1; + +cleanup: + for (i = 0; i < object_info_oids.nr; i++) + free_object_info_contents(&remote_object_info[i]); + free(remote_object_info); +} + static void dispatch_calls(struct batch_options *opt, struct strbuf *output, struct expand_data *data, @@ -579,8 +693,12 @@ static void dispatch_calls(struct batch_options *opt, if (!opt->buffer_output) die(_("flush is only for --buffer mode")); - for (i = 0; i < nr; i++) - cmd[i].fn(opt, cmd[i].line, output, data); + for (i = 0; i < nr; i++) { + if (!strcmp(cmd[i].name, "remote-object-info")) + parse_remote_info(opt, cmd[i].line, output, data, NULL, &cmd[i]); + else + cmd[i].fn(opt, cmd[i].line, output, data); + } fflush(stdout); } @@ -640,11 +758,17 @@ static void batch_objects_command(struct batch_options *opt, dispatch_calls(opt, output, data, queued_cmd, nr); free_cmds(queued_cmd, &nr); } else if (!opt->buffer_output) { - cmd->fn(opt, p, output, data); + if (!strcmp(cmd->name, "remote-object-info")) { + char *line = xstrdup_or_null(p); + parse_remote_info(opt, line, output, data, cmd, NULL); + } else { + cmd->fn(opt, p, output, data); + } } else { ALLOC_GROW(queued_cmd, nr + 1, alloc); call.fn = cmd->fn; call.line = xstrdup_or_null(p); + call.name = cmd->name; queued_cmd[nr++] = call; } } diff --git a/t/t1006-cat-file.sh b/t/t1006-cat-file.sh index 1b85207694..69ee3f8330 100755 --- a/t/t1006-cat-file.sh +++ b/t/t1006-cat-file.sh @@ -2,6 +2,9 @@ test_description='git cat-file' +GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main +export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME + . ./test-lib.sh test_cmdmode_usage () { @@ -1087,4 +1090,348 @@ test_expect_success 'batch-command flush without --buffer' ' grep "^fatal:.*flush is only for --buffer mode.*" err ' +# This section tests --batch-command with remote-object-info command +# If a filter is not set, the filter defaults to "%(objectname) %(objectsize) %(objecttype)" +# Since "%(objecttype)" is currently not supported by the command request, object-info, +# the filters are set to "%(objectname) %(objectsize)". + +set_transport_variables () { + tree_sha1=$(git -C "$1" write-tree) + commit_sha1=$(echo_without_newline "$commit_message" | git -C "$1" commit-tree $tree_sha1) + tag_sha1=$(echo_without_newline "$tag_content" | git -C "$1" hash-object -t tag --stdin -w) + tag_size=$(strlen "$tag_content") +} + +# Test --batch-command remote-object-info with 'git://' transport + +. "$TEST_DIRECTORY"/lib-git-daemon.sh +start_git_daemon --export-all --enable=receive-pack +daemon_parent=$GIT_DAEMON_DOCUMENT_ROOT_PATH/parent + +test_expect_success 'create repo to be served by git-daemon' ' + git init "$daemon_parent" && + echo_without_newline "$hello_content" > $daemon_parent/hello && + git -C "$daemon_parent" update-index --add hello +' + +set_transport_variables "$daemon_parent" + +test_expect_success 'batch-command remote-object-info git://' ' + ( + cd "$daemon_parent" && + + echo "$hello_sha1 $hello_size" >expect && + echo "$tree_sha1 $tree_size" >>expect && + echo "$commit_sha1 $commit_size" >>expect && + echo "$tag_sha1 $tag_size" >>expect && + git cat-file --batch-command="%(objectname) %(objectsize)" >actual <<-EOF && + remote-object-info "$GIT_DAEMON_URL/parent" $hello_sha1 + remote-object-info "$GIT_DAEMON_URL/parent" $tree_sha1 + remote-object-info "$GIT_DAEMON_URL/parent" $commit_sha1 + remote-object-info "$GIT_DAEMON_URL/parent" $tag_sha1 + EOF + test_cmp expect actual + ) +' + +test_expect_success 'batch-command remote-object-info git:// multiple sha1 per line' ' + ( + cd "$daemon_parent" && + + echo "$hello_sha1 $hello_size" >expect && + echo "$tree_sha1 $tree_size" >>expect && + echo "$commit_sha1 $commit_size" >>expect && + echo "$tag_sha1 $tag_size" >>expect && + git cat-file --batch-command="%(objectname) %(objectsize)" >actual <<-EOF && + remote-object-info "$GIT_DAEMON_URL/parent" $hello_sha1 $tree_sha1 $commit_sha1 $tag_sha1 + EOF + test_cmp expect actual + ) +' + +test_expect_success 'batch-command remote-object-info http:// default filter' ' + ( + cd "$daemon_parent" && + + echo "$hello_sha1 blob $hello_size" >expect && + echo "$tree_sha1 tree $tree_size" >>expect && + echo "$commit_sha1 commit $commit_size" >>expect && + echo "$tag_sha1 tag $tag_size" >>expect && + git cat-file --batch-command >actual <<-EOF && + remote-object-info "$GIT_DAEMON_URL/parent" $hello_sha1 $tree_sha1 + remote-object-info "$GIT_DAEMON_URL/parent" $commit_sha1 $tag_sha1 + EOF + test_cmp expect actual + ) +' + +test_expect_success 'batch-command --buffer remote-object-info git://' ' + ( + cd "$daemon_parent" && + + echo "$hello_sha1 $hello_size" >expect && + echo "$tree_sha1 $tree_size" >>expect && + echo "$commit_sha1 $commit_size" >>expect && + echo "$tag_sha1 $tag_size" >>expect && + git cat-file --batch-command="%(objectname) %(objectsize)" --buffer >actual <<-EOF && + remote-object-info "$GIT_DAEMON_URL/parent" $hello_sha1 $tree_sha1 + remote-object-info "$GIT_DAEMON_URL/parent" $commit_sha1 $tag_sha1 + flush + EOF + test_cmp expect actual + ) +' + +stop_git_daemon + +# Test --batch-command remote-object-info with 'http://' transport + +. "$TEST_DIRECTORY"/lib-httpd.sh +start_httpd + +test_expect_success 'create repo to be served by http:// transport' ' + git init "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + git -C "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" config http.receivepack true && + echo_without_newline "$hello_content" > $HTTPD_DOCUMENT_ROOT_PATH/http_parent/hello && + git -C "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" update-index --add hello +' +set_transport_variables "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" + +test_expect_success 'batch-command remote-object-info http://' ' + ( + cd "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + + echo "$hello_sha1 $hello_size" >expect && + echo "$tree_sha1 $tree_size" >>expect && + echo "$commit_sha1 $commit_size" >>expect && + echo "$tag_sha1 $tag_size" >>expect && + git cat-file --batch-command="%(objectname) %(objectsize)" >actual <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" $hello_sha1 + remote-object-info "$HTTPD_URL/smart/http_parent" $tree_sha1 + remote-object-info "$HTTPD_URL/smart/http_parent" $commit_sha1 + remote-object-info "$HTTPD_URL/smart/http_parent" $tag_sha1 + EOF + test_cmp expect actual + ) +' + +test_expect_success 'batch-command remote-object-info http:// one line' ' + ( + cd "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + + echo "$hello_sha1 $hello_size" >expect && + echo "$tree_sha1 $tree_size" >>expect && + echo "$commit_sha1 $commit_size" >>expect && + echo "$tag_sha1 $tag_size" >>expect && + git cat-file --batch-command="%(objectname) %(objectsize)" >actual <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" $hello_sha1 $tree_sha1 $commit_sha1 $tag_sha1 + EOF + test_cmp expect actual + ) +' + +test_expect_success 'batch-command --buffer remote-object-info http://' ' + ( + cd "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + + echo "$hello_sha1 $hello_size" >expect && + echo "$tree_sha1 $tree_size" >>expect && + echo "$commit_sha1 $commit_size" >>expect && + echo "$tag_sha1 $tag_size" >>expect && + + git cat-file --batch-command="%(objectname) %(objectsize)" --buffer >actual <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" $hello_sha1 $tree_sha1 + remote-object-info "$HTTPD_URL/smart/http_parent" $commit_sha1 $tag_sha1 + flush + EOF + test_cmp expect actual + ) +' + +test_expect_success 'batch-command remote-object-info http:// default filter' ' + ( + cd "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + + echo "$hello_sha1 blob $hello_size" >expect && + echo "$tree_sha1 tree $tree_size" >>expect && + echo "$commit_sha1 commit $commit_size" >>expect && + echo "$tag_sha1 tag $tag_size" >>expect && + + git cat-file --batch-command >actual <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" $hello_sha1 $tree_sha1 + remote-object-info "$HTTPD_URL/smart/http_parent" $commit_sha1 $tag_sha1 + EOF + test_cmp expect actual + ) +' + +test_expect_success 'remote-object-info fails on unspported filter option (objectsize:disk)' ' + ( + cd "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + + test_must_fail git cat-file --batch-command="%(objectsize:disk)" 2>err <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" $hello_sha1 + EOF + test_i18ngrep "objectsize:disk is currently not supported with remote-object-info" err + ) +' + +test_expect_success 'remote-object-info fails on unspported filter option (deltabase)' ' + ( + cd "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + + test_must_fail git cat-file --batch-command="%(deltabase)" 2>err <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" $hello_sha1 + EOF + test_i18ngrep "deltabase is currently not supported with remote-object-info" err + ) +' + +test_expect_success 'remote-object-info fails on server with legacy protocol' ' + ( + cd "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + + test_must_fail git -c protocol.version=0 cat-file --batch-command="%(objectname) %(objectsize)" 2>err <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" $hello_sha1 + EOF + test_i18ngrep "object-info requires protocol v2" err + ) +' + +test_expect_success 'remote-object-info fails on server with legacy protocol fallback' ' + ( + cd "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + + test_must_fail git -c protocol.version=0 cat-file --batch-command 2>err <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" $hello_sha1 + EOF + test_i18ngrep "object-info requires protocol v2" err + ) +' + +test_expect_success 'remote-object-info fails on malformed OID' ' + ( + cd "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + malformed_object_id="this_id_is_not_valid" && + + test_must_fail git cat-file --batch-command="%(objectname) %(objectsize)" 2>err <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" $malformed_object_id + EOF + test_i18ngrep "malformed object id '$malformed_object_id'" err + ) +' + +test_expect_success 'remote-object-info fails on malformed OID fallback' ' + ( + cd "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + malformed_object_id="this_id_is_not_valid" && + + test_must_fail git cat-file --batch-command 2>err <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" $malformed_object_id + EOF + test_i18ngrep "malformed object id '$malformed_object_id'" err + ) +' + +test_expect_success 'remote-object-info fails on missing OID' ' + git clone "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" missing_oid_repo && + test_commit -C missing_oid_repo message1 c.txt && + ( + cd missing_oid_repo && + object_id=$(git rev-parse message1:c.txt) && + test_must_fail git cat-file --batch-command="%(objectname) %(objectsize)" 2>err <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" $object_id + EOF + test_i18ngrep "object-info: not our ref $object_id" err + ) +' + +test_expect_success 'remote-object-info fails on missing OID fallback' ' + ( + cd missing_oid_repo && + object_id=$(git rev-parse message1:c.txt) && + test_must_fail git cat-file --batch-command 2>err <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" $object_id + EOF + test_i18ngrep "fatal: remote error: upload-pack: not our ref $object_id" err + ) +' + +# Test --batch-command remote-object-info with 'file://' transport + +test_expect_success 'create repo to be served by file:// transport' ' + git init server && + git -C server config protocol.version 2 && + echo_without_newline "$hello_content" > server/hello && + git -C server update-index --add hello +' + +set_transport_variables "server" + +test_expect_success 'batch-command remote-object-info file://' ' + ( + cd server && + + echo "$hello_sha1 $hello_size" >expect && + echo "$tree_sha1 $tree_size" >>expect && + echo "$commit_sha1 $commit_size" >>expect && + echo "$tag_sha1 $tag_size" >>expect && + git cat-file --batch-command="%(objectname) %(objectsize)" >actual <<-EOF && + remote-object-info "file://$(pwd)" $hello_sha1 + remote-object-info "file://$(pwd)" $tree_sha1 + remote-object-info "file://$(pwd)" $commit_sha1 + remote-object-info "file://$(pwd)" $tag_sha1 + EOF + test_cmp expect actual + ) +' + +test_expect_success 'batch-command remote-object-info file:// multiple sha1 per line' ' + ( + cd server && + + echo "$hello_sha1 $hello_size" >expect && + echo "$tree_sha1 $tree_size" >>expect && + echo "$commit_sha1 $commit_size" >>expect && + echo "$tag_sha1 $tag_size" >>expect && + git cat-file --batch-command="%(objectname) %(objectsize)" >actual <<-EOF && + remote-object-info "file://$(pwd)" $hello_sha1 $tree_sha1 $commit_sha1 $tag_sha1 + EOF + test_cmp expect actual + ) +' + +test_expect_success 'batch-command --buffer remote-object-info file://' ' + ( + cd server && + + echo "$hello_sha1 $hello_size" >expect && + echo "$tree_sha1 $tree_size" >>expect && + echo "$commit_sha1 $commit_size" >>expect && + echo "$tag_sha1 $tag_size" >>expect && + git cat-file --batch-command="%(objectname) %(objectsize)" --buffer >actual <<-EOF && + remote-object-info "file://$(pwd)" $hello_sha1 $tree_sha1 + remote-object-info "file://$(pwd)" $commit_sha1 $tag_sha1 + flush + EOF + test_cmp expect actual + ) +' + +test_expect_success 'batch-command remote-object-info file:// default filter' ' + ( + cd server && + + echo "$hello_sha1 blob $hello_size" >expect && + echo "$tree_sha1 tree $tree_size" >>expect && + echo "$commit_sha1 commit $commit_size" >>expect && + echo "$tag_sha1 tag $tag_size" >>expect && + git cat-file --batch-command >actual <<-EOF && + remote-object-info "file://$(pwd)" $hello_sha1 $tree_sha1 + remote-object-info "file://$(pwd)" $commit_sha1 $tag_sha1 + EOF + test_cmp expect actual + ) +' + test_done