From patchwork Wed Oct 4 15:21:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiang Xin X-Patchwork-Id: 13408899 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2323FE7C4CE for ; Wed, 4 Oct 2023 15:21:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243114AbjJDPVx (ORCPT ); Wed, 4 Oct 2023 11:21:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45272 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243104AbjJDPVv (ORCPT ); Wed, 4 Oct 2023 11:21:51 -0400 Received: from mail-pf1-x435.google.com (mail-pf1-x435.google.com [IPv6:2607:f8b0:4864:20::435]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 598C0C4 for ; Wed, 4 Oct 2023 08:21:48 -0700 (PDT) Received: by mail-pf1-x435.google.com with SMTP id d2e1a72fcca58-690d2e13074so1794076b3a.1 for ; Wed, 04 Oct 2023 08:21:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696432907; x=1697037707; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=oVLallzDkOOn/QVapkE5m3n1syzHG35ySXXdojkDgfM=; b=DIKqKbWEyTiQ32m5DcG9+OycWB4z2bFoGSEGSBk19pTwptLzogQB9q9Dng7xtviENu BOi44Qs7EcicGRcpOv5TajP78rZdaobz296Fg96etuZjMOuYQ1SHuUSW8sZh0NiELG8K oVrT/OUbJgKvR79hSV4jTuYFNXaiXkPYRenV5TqN/8kErj2bSeSyZc59Mam+Ap5PPr66 pfZzZTh1vSPEyIedlaTJvl2Gyd6YxxMIo9JCPj0vPsq95/NWqq13kszDVRzUKt4WIXV9 z8WViDxIcqJaxcVZkczVdVTRJWm6N10FRt8a+cNmrNFvhvcKODHwUghGj+JrMHNenIDQ TGIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696432907; x=1697037707; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=oVLallzDkOOn/QVapkE5m3n1syzHG35ySXXdojkDgfM=; b=oR7p/44rD/7q4CMP4AxPcp1DGYwy89ASSBo5eW2zrP0ZC9LPHrGR8cnG4mUn9XRFww BkfnRExerH6A4bCxyp6lQ5RLEL4g8/tDjUxDBRzlDcbD7Rwy7clqKGR/vege3NVwd/GQ ybM7clhvnrPDSSpv/YImB8pDzML1GV2/Ek33W/XnDYkZ+CiT4YgdoTj1bYzq+VPHekBn IHTtKYmncns9WHH1hwqTEU2OfQ5dOmHuEyMOiKMaHVWx2HStyRsp3SfEHpERexLIkfi1 b006yBeOncoWmUfE6pyivtF5hhm3raC7m8T4+OKIBPZJ/Nwky+GCc2iWhMczcn1CxB0n bVkw== X-Gm-Message-State: AOJu0YzRo5CtmmonSLQvMypqio/PSZYuMhbN9jGdq1XZ1Q6q+K/M0O5m sOXBnuwL1WidKlt7fD5u/iJkPVI3j7s= X-Google-Smtp-Source: AGHT+IHpUmcRv0QR2GBdgFUaIaVYs5GZaBlerhQe2kbDECAcP+r8ZhDjTPMgFSKWsUPNWq+jVg/OKA== X-Received: by 2002:a05:6a20:3c8b:b0:12c:b10d:693d with SMTP id b11-20020a056a203c8b00b0012cb10d693dmr3097357pzj.6.1696432907582; Wed, 04 Oct 2023 08:21:47 -0700 (PDT) Received: from tigtog-proxy.localdomain.localdomain (144.34.163.219.16clouds.com. [144.34.163.219]) by smtp.gmail.com with ESMTPSA id c11-20020a170903234b00b001c5f0fe64c2sm3838199plh.56.2023.10.04.08.21.46 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 04 Oct 2023 08:21:47 -0700 (PDT) From: Jiang Xin To: Git List , Junio C Hamano , Eric Sunshine Cc: Jiang Xin Subject: [PATCH v3 1/4] transport-helper: no connection restriction in connect_helper Date: Wed, 4 Oct 2023 23:21:40 +0800 Message-Id: X-Mailer: git-send-email 2.32.0.rc3 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Jiang Xin When commit b236752a (Support remote archive from all smart transports, 2009-12-09) added "remote archive" support for "smart transports", it was for transport that supports the ".connect" method. The "connect_helper()" function protected itself from getting called for a transport without the method before calling process_connect_service(), which did not work with such a transport. Later, commit edc9caf7 (transport-helper: introduce stateless-connect, 2018-03-15) added a way for a transport without the ".connect" method to establish a "stateless" connection in protocol-v2, which process_connect_service() was taught to handle the "stateless" connection, making the old safety valve in its caller that insisted that ".connect" method must be defined too strict, and forgot to loosen it. Remove the restriction in the "connect_helper()" function and give the function "process_connect_service()" the opportunity to establish a connection using ".connect" or ".stateless_connect" for protocol v2. So we can connect with a stateless-rpc and do something useful. E.g., in a later commit, implements remote archive for a repository over HTTP protocol. Helped-by: Junio C Hamano Signed-off-by: Jiang Xin --- transport-helper.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/transport-helper.c b/transport-helper.c index 49811ef176..2e127d24a5 100644 --- a/transport-helper.c +++ b/transport-helper.c @@ -662,8 +662,6 @@ static int connect_helper(struct transport *transport, const char *name, /* Get_helper so connect is inited. */ get_helper(transport); - if (!data->connect) - die(_("operation not supported by protocol")); if (!process_connect_service(transport, name, exec)) die(_("can't connect to subservice %s"), name); From patchwork Wed Oct 4 15:21:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiang Xin X-Patchwork-Id: 13408902 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6043BE7C4D0 for ; Wed, 4 Oct 2023 15:22:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243213AbjJDPWC (ORCPT ); Wed, 4 Oct 2023 11:22:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45310 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243163AbjJDPVy (ORCPT ); Wed, 4 Oct 2023 11:21:54 -0400 Received: from mail-pl1-x630.google.com (mail-pl1-x630.google.com [IPv6:2607:f8b0:4864:20::630]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2073FD8 for ; Wed, 4 Oct 2023 08:21:49 -0700 (PDT) Received: by mail-pl1-x630.google.com with SMTP id d9443c01a7336-1c3f97f2239so18020065ad.0 for ; Wed, 04 Oct 2023 08:21:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696432908; x=1697037708; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ROQ9UozEZPA2+mJSw4EWiQmxaUtdl4B3V+VnwxyVx5A=; b=nL4jg7+KfSjSJ5j4aFKJ6eIt5YBTRhDhEUPWzJP/2FK6bEdXT3CZweUWddB7fepS0T tzRB0psYbs0Yccyh+tEs9RTcC45yj+hHcxCLLu2ur78Exti9oowvkCppU/rCsY6JT7N8 AT23f5XuwiTfJQkBM11HNzB3s+5N46VJMK1vsvFeTEwffYIeGsdEAHjKNmh+bL7pJO9H 684+4RF9Qx+lcdWaGfvzDrtro3ilc3neIS2pNlbmcKyG6afO7PpTK9RFiaiTbd/DYtg9 WG9Zf+rtz1CFvdQffRRNycisoeNzbFpd22K3SpLpSr8bHivxIlvfGO4qbnj3SaGcJAPm R7Aw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696432908; x=1697037708; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ROQ9UozEZPA2+mJSw4EWiQmxaUtdl4B3V+VnwxyVx5A=; b=sBuLAXgTlh1YkLoXwGCd5TQHg6T2exUC1cSCPB6Yof1i2K2qRzHe3YkC8Ew5gEIEcw lA0C16vqZAkWcbHM/ylZNU2HtafQjBgi5QJVeqcSzO5M7MhiRJ8P2Q75Nays7QbdK+m/ TN8oaR24ZWZr1VlRLRXU97AlYC+5wtKZAWWD1BNIE4fewj9eR2ZLFtTutsG2a4gFGxsH 8e9zjdBNHnMTclfEo193nuFhBtGNtwa+zOmx7bLF211JcaHWA7pMUNyb8pDxcXyKacwx xYhCYNCghweYC9Ah05tIsgAzPhOpug44u7H9o9FvhgG05E2dpW0m31lAClydYPS1pxLj Z40g== X-Gm-Message-State: AOJu0Yzt899PbYIcZOD4vXp/8r0NQReV6iJGlEa34l1/wpmDrXF/uQSg OwAAcInQ4goB7a+9z75qQXmnAuA8O1Y= X-Google-Smtp-Source: AGHT+IGdsgaz87+GogLPKSjrIaBHK8e7q1VGuTdnNHylRnilZGVzYLhjpDNNHkBPjQY5P2ScxgVMAw== X-Received: by 2002:a17:902:e811:b0:1b8:8af0:416f with SMTP id u17-20020a170902e81100b001b88af0416fmr3162723plg.1.1696432908411; Wed, 04 Oct 2023 08:21:48 -0700 (PDT) Received: from tigtog-proxy.localdomain.localdomain (144.34.163.219.16clouds.com. [144.34.163.219]) by smtp.gmail.com with ESMTPSA id c11-20020a170903234b00b001c5f0fe64c2sm3838199plh.56.2023.10.04.08.21.47 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 04 Oct 2023 08:21:48 -0700 (PDT) From: Jiang Xin To: Git List , Junio C Hamano , Eric Sunshine Cc: Jiang Xin Subject: [PATCH v3 2/4] transport-helper: call do_take_over() in process_connect Date: Wed, 4 Oct 2023 23:21:41 +0800 Message-Id: X-Mailer: git-send-email 2.32.0.rc3 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Jiang Xin The existing pattern among all callers of process_connect() seems to be if (process_connect(...)) { do_take_over(); ... dispatch to the underlying method ... } ... otherwise implement the fallback ... where the return value from process_connect() is the return value of the call it makes to process_connect_service(). It is safe to make a refactor by moving the call of do_take_over() into the function process_connect(). Suggested-by: Junio C Hamano Signed-off-by: Jiang Xin --- transport-helper.c | 22 +++++++++------------- 1 file changed, 9 insertions(+), 13 deletions(-) diff --git a/transport-helper.c b/transport-helper.c index 2e127d24a5..51088cc03a 100644 --- a/transport-helper.c +++ b/transport-helper.c @@ -645,6 +645,7 @@ static int process_connect(struct transport *transport, struct helper_data *data = transport->data; const char *name; const char *exec; + int ret; name = for_push ? "git-receive-pack" : "git-upload-pack"; if (for_push) @@ -652,7 +653,10 @@ static int process_connect(struct transport *transport, else exec = data->transport_options.uploadpack; - return process_connect_service(transport, name, exec); + ret = process_connect_service(transport, name, exec); + if (ret) + do_take_over(transport); + return ret; } static int connect_helper(struct transport *transport, const char *name, @@ -682,10 +686,8 @@ static int fetch_refs(struct transport *transport, get_helper(transport); - if (process_connect(transport, 0)) { - do_take_over(transport); + if (process_connect(transport, 0)) return transport->vtable->fetch_refs(transport, nr_heads, to_fetch); - } /* * If we reach here, then the server, the client, and/or the transport @@ -1142,10 +1144,8 @@ static int push_refs(struct transport *transport, { struct helper_data *data = transport->data; - if (process_connect(transport, 1)) { - do_take_over(transport); + if (process_connect(transport, 1)) return transport->vtable->push_refs(transport, remote_refs, flags); - } if (!remote_refs) { fprintf(stderr, @@ -1186,11 +1186,9 @@ static struct ref *get_refs_list(struct transport *transport, int for_push, { get_helper(transport); - if (process_connect(transport, for_push)) { - do_take_over(transport); + if (process_connect(transport, for_push)) return transport->vtable->get_refs_list(transport, for_push, transport_options); - } return get_refs_list_using_list(transport, for_push); } @@ -1274,10 +1272,8 @@ static int get_bundle_uri(struct transport *transport) { get_helper(transport); - if (process_connect(transport, 0)) { - do_take_over(transport); + if (process_connect(transport, 0)) return transport->vtable->get_bundle_uri(transport); - } return -1; } From patchwork Wed Oct 4 15:21:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiang Xin X-Patchwork-Id: 13408900 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 37C7AE7C4D5 for ; Wed, 4 Oct 2023 15:21:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243199AbjJDPVz (ORCPT ); Wed, 4 Oct 2023 11:21:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45296 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233394AbjJDPVw (ORCPT ); Wed, 4 Oct 2023 11:21:52 -0400 Received: from mail-pl1-x62c.google.com (mail-pl1-x62c.google.com [IPv6:2607:f8b0:4864:20::62c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DA7C193 for ; Wed, 4 Oct 2023 08:21:49 -0700 (PDT) Received: by mail-pl1-x62c.google.com with SMTP id d9443c01a7336-1c737d61a00so17135025ad.3 for ; Wed, 04 Oct 2023 08:21:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696432909; x=1697037709; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=k7Dnp/yNHi6Twk4js+gl4yySR843/EtBt3LDNrZEsq0=; b=Psvalo71y5ty4tK0cAvcLXa6OIntY/+DyRIn7itr1wnA81hLP7ZzDepoG6EfORpm7l Dqx6IW/hcrYWEUSI4oqFvRSQcGFdFy1tpVzm1+6l8YQ9schZiXqGccXlgy7VUOu44ilU Ovv7NAIYS6Sy60gce8E8kwLTG8G9g/cf8adW5AyeMukTt8csls9I7gzXbWuuvx9DoA8H ucNmNcR62lby/lKdT65wssKy3OKOxYZr4j1zLaKYiEfuhxcVgEni+AzDLuHodW595lqi 2uR2ULdhOweMwsocpf2wqZ9B8hglVp+L79iokdt6wxpO0OF/Mf+/PFbGRxXVyaY/fssI Bc6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696432909; x=1697037709; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=k7Dnp/yNHi6Twk4js+gl4yySR843/EtBt3LDNrZEsq0=; b=IynDVrJUZrY8vSbjvsd3fnj/+TETpJdDIMLJljakcjXu4pEo8fuszl2KWcZa4B8WvK +d1Sd3S0S46R1D0vigOtZ0iRIAWXGs5dzPyYDAP2Z5s8gS3AQv6DeWB5vV2fCn/gpC6f DTC3YkreceLm70kg34pgVdjOKrdXAN2L0vQin5b8DdBWMLxZ6LTsUP1O0eYSpDCMtbMY G2sIUhq9qTzTfD4HXB5iA2p6sFrLNNDtm0MfONbTL9f2XX1FoNJJypHWXFVwwSbF0M84 OSfUhXLJ0kKBNY2tF2uvPbbiOnZe+fnGbRAUqWrOaUoEuCzQrOX7pXjAIxQxp7SbIczF wl0g== X-Gm-Message-State: AOJu0YyaCETKggV5DYJfyaTv8qQIofyhfiLg/WcIzFU1GkBClcNc5SOz fptjdZAZjicmgGwa3CiHy+X3ejXH310= X-Google-Smtp-Source: AGHT+IEaTLrWe6iYXaOz1Hy9+DZFOFuXc91ZMJMX9EdyiJs2R7OT1qG5v9q5Foud9s2oDUB1yZIl8w== X-Received: by 2002:a17:902:e749:b0:1b8:76ce:9d91 with SMTP id p9-20020a170902e74900b001b876ce9d91mr3413979plf.1.1696432909205; Wed, 04 Oct 2023 08:21:49 -0700 (PDT) Received: from tigtog-proxy.localdomain.localdomain (144.34.163.219.16clouds.com. [144.34.163.219]) by smtp.gmail.com with ESMTPSA id c11-20020a170903234b00b001c5f0fe64c2sm3838199plh.56.2023.10.04.08.21.48 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 04 Oct 2023 08:21:48 -0700 (PDT) From: Jiang Xin To: Git List , Junio C Hamano , Eric Sunshine Cc: Jiang Xin Subject: [PATCH v3 3/4] transport-helper: call do_take_over() in connect_helper Date: Wed, 4 Oct 2023 23:21:42 +0800 Message-Id: <01699822c3f8ecda2c25dc2e2922c1f993c8beb2.1696432594.git.zhiyou.jx@alibaba-inc.com> X-Mailer: git-send-email 2.32.0.rc3 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Jiang Xin After successfully connecting to the smart transport by calling process_connect_service() in connect_helper(), run do_take_over() to replace the old vtable with a new one which has methods ready for the smart transport connection. The connect_helper() function is used as the connect method of the vtable in "transport-helper.c", and it is called by transport_connect() in "transport.c" to setup a connection. The only place that we call transport_connect() so far is in "builtin/archive.c". Without running do_take_over(), it may fail to call transport_disconnect() in run_remote_archiver() of "builtin/archive.c". This is because for a stateless connection or a service like "git-upload-pack-archive", the remote helper may receive a SIGPIPE signal and exit early. To have a graceful disconnect method by calling do_take_over() will solve this issue. The subsequent commit will introduce remote archive over a stateless-rpc connection. Signed-off-by: Jiang Xin --- transport-helper.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/transport-helper.c b/transport-helper.c index 51088cc03a..3b036ae1ca 100644 --- a/transport-helper.c +++ b/transport-helper.c @@ -672,6 +672,8 @@ static int connect_helper(struct transport *transport, const char *name, fd[0] = data->helper->out; fd[1] = data->helper->in; + + do_take_over(transport); return 0; } From patchwork Wed Oct 4 15:21:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiang Xin X-Patchwork-Id: 13408901 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 28E17E7C4CE for ; Wed, 4 Oct 2023 15:22:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233394AbjJDPWB (ORCPT ); Wed, 4 Oct 2023 11:22:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45316 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243165AbjJDPVy (ORCPT ); Wed, 4 Oct 2023 11:21:54 -0400 Received: from mail-pl1-x62e.google.com (mail-pl1-x62e.google.com [IPv6:2607:f8b0:4864:20::62e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C4C56BF for ; Wed, 4 Oct 2023 08:21:50 -0700 (PDT) Received: by mail-pl1-x62e.google.com with SMTP id d9443c01a7336-1c364fb8a4cso19124295ad.1 for ; Wed, 04 Oct 2023 08:21:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696432910; x=1697037710; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=haSykPK1Q0SBrB0XQrzMn/b6dVSkUGq2BgAaA/DLOcs=; b=MTN8TKvkfS8+K2umN8T+linIOipFlOT01eulG2Ew+zBsCyQ5IqAIxYzxaEN3vRXD77 ygoUy3YUY265sI+0m8TXnm+FvtzuRrF0vxSlHt7H7r/t5GJQOK348BaQJWhRfJ8OCkUo Vf8VbkCwAzI3e3ol6yM2ldFJLtBnwFffOithX07K5kNpDtfGcq/OAVMdCljSlg5DkjEc 1GOUwmxnjRWaVQ4+SGyOKXvxM4PrAftJ8xvKQiGlxabDd3/GVWYUluzOzOIoKxN90ZgG meFBZA5+gAn3QvmD273JzlXp0jvkypRYa67RBlDlAKnrN/d7mw69KgP5vz9tRizguo1Z ib1g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696432910; x=1697037710; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=haSykPK1Q0SBrB0XQrzMn/b6dVSkUGq2BgAaA/DLOcs=; b=Av27O+wwhc0gi3LgvZ6epPS+ne22leffNi/ajE+pgFed7vA8Ds25/q6Sx6cIjXF2AH eq9+sVqivd7Gip0p/J32X7ylTHzHyhMZF9Wq0gxHtf31oexBwrpLetWod5HYeaePU/wb AtIz0TaK03upOb3lKfgNDCVjRb3maBHgzD+zi0saGv6FDozXbctEgM4EVzj+iHRdZqxD SyIJ9j3KSk4GZIA1+IP6z2JKAoCUCuWXeRuFyxwFEu82QX2hcuhlcn/O65+D7NJMGuuR eYBwaMV9ttjxJAXuSyRqLxTTR/364notMOWn5dd254z3MskgnFNNbeQVFxKmBG5Vkzhe ve7g== X-Gm-Message-State: AOJu0YyDD4eeDaWCS1vaR6jzhHOcuTlJ/3WUAecBWPrOeVt8lhOogZVO rteOJxEOYr1EvhZHj6CKEvea1xsTEgQ= X-Google-Smtp-Source: AGHT+IFz5L5PqYIPNxIJJD+fSJMJcH+MonkmwwUFFItVoQFth8bEVMtEb7BBKD87zf6nqjRoiu9KLw== X-Received: by 2002:a17:903:182:b0:1bd:d6b5:a54f with SMTP id z2-20020a170903018200b001bdd6b5a54fmr2659676plg.55.1696432910037; Wed, 04 Oct 2023 08:21:50 -0700 (PDT) Received: from tigtog-proxy.localdomain.localdomain (144.34.163.219.16clouds.com. [144.34.163.219]) by smtp.gmail.com with ESMTPSA id c11-20020a170903234b00b001c5f0fe64c2sm3838199plh.56.2023.10.04.08.21.49 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 04 Oct 2023 08:21:49 -0700 (PDT) From: Jiang Xin To: Git List , Junio C Hamano , Eric Sunshine Cc: Jiang Xin Subject: [PATCH v3 4/4] archive: support remote archive from stateless transport Date: Wed, 4 Oct 2023 23:21:43 +0800 Message-Id: X-Mailer: git-send-email 2.32.0.rc3 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Jiang Xin Even though we can establish a stateless connection, we still cannot archive the remote repository using a stateless HTTP protocol. Try the following steps to make it work. 1. Add support for "git-upload-archive" service in "http-backend". 2. Use the URL ".../info/refs?service=git-upload-pack" to detect the protocol version, instead of use the "git-upload-archive" service. 3. "git-archive" does not expect to see protocol version and capabilities when connecting to remote-helper, so do not send them in "remote-curl.c" for the "git-upload-archive" service. Helped-by: Eric Sunshine Signed-off-by: Jiang Xin --- http-backend.c | 15 +++++++++++---- remote-curl.c | 14 +++++++++++--- t/t5003-archive-zip.sh | 30 ++++++++++++++++++++++++++++++ transport-helper.c | 3 ++- 4 files changed, 54 insertions(+), 8 deletions(-) diff --git a/http-backend.c b/http-backend.c index ff07b87e64..6a2c919839 100644 --- a/http-backend.c +++ b/http-backend.c @@ -38,6 +38,7 @@ struct rpc_service { static struct rpc_service rpc_service[] = { { "upload-pack", "uploadpack", 1, 1 }, { "receive-pack", "receivepack", 0, -1 }, + { "upload-archive", "uploadarchive", 0, -1 }, }; static struct string_list *get_parameters(void) @@ -639,10 +640,15 @@ static void check_content_type(struct strbuf *hdr, const char *accepted_type) static void service_rpc(struct strbuf *hdr, char *service_name) { - const char *argv[] = {NULL, "--stateless-rpc", ".", NULL}; + struct strvec argv = STRVEC_INIT; struct rpc_service *svc = select_service(hdr, service_name); struct strbuf buf = STRBUF_INIT; + strvec_push(&argv, svc->name); + if (strcmp(service_name, "git-upload-archive")) + strvec_push(&argv, "--stateless-rpc"); + strvec_push(&argv, "."); + strbuf_reset(&buf); strbuf_addf(&buf, "application/x-git-%s-request", svc->name); check_content_type(hdr, buf.buf); @@ -655,9 +661,9 @@ static void service_rpc(struct strbuf *hdr, char *service_name) end_headers(hdr); - argv[0] = svc->name; - run_service(argv, svc->buffer_input); + run_service(argv.v, svc->buffer_input); strbuf_release(&buf); + strvec_clear(&argv); } static int dead; @@ -723,7 +729,8 @@ static struct service_cmd { {"GET", "/objects/pack/pack-[0-9a-f]{64}\\.idx$", get_idx_file}, {"POST", "/git-upload-pack$", service_rpc}, - {"POST", "/git-receive-pack$", service_rpc} + {"POST", "/git-receive-pack$", service_rpc}, + {"POST", "/git-upload-archive$", service_rpc} }; static int bad_request(struct strbuf *hdr, const struct service_cmd *c) diff --git a/remote-curl.c b/remote-curl.c index ef05752ca5..ce6cb8ac05 100644 --- a/remote-curl.c +++ b/remote-curl.c @@ -1447,8 +1447,14 @@ static int stateless_connect(const char *service_name) * establish a stateless connection, otherwise we need to tell the * client to fallback to using other transport helper functions to * complete their request. + * + * The "git-upload-archive" service is a read-only operation. Fallback + * to use "git-upload-pack" service to discover protocol version. */ - discover = discover_refs(service_name, 0); + if (!strcmp(service_name, "git-upload-archive")) + discover = discover_refs("git-upload-pack", 0); + else + discover = discover_refs(service_name, 0); if (discover->version != protocol_v2) { printf("fallback\n"); fflush(stdout); @@ -1486,9 +1492,11 @@ static int stateless_connect(const char *service_name) /* * Dump the capability listing that we got from the server earlier - * during the info/refs request. + * during the info/refs request. This does not work with the + * "git-upload-archive" service. */ - write_or_die(rpc.in, discover->buf, discover->len); + if (strcmp(service_name, "git-upload-archive")) + write_or_die(rpc.in, discover->buf, discover->len); /* Until we see EOF keep sending POSTs */ while (1) { diff --git a/t/t5003-archive-zip.sh b/t/t5003-archive-zip.sh index fc499cdff0..80123c1e06 100755 --- a/t/t5003-archive-zip.sh +++ b/t/t5003-archive-zip.sh @@ -239,4 +239,34 @@ check_zip with_untracked2 check_added with_untracked2 untracked one/untracked check_added with_untracked2 untracked two/untracked +. "$TEST_DIRECTORY"/lib-httpd.sh +start_httpd + +test_expect_success "setup for HTTP protocol" ' + cp -R bare.git "$HTTPD_DOCUMENT_ROOT_PATH/bare.git" && + git -C "$HTTPD_DOCUMENT_ROOT_PATH/bare.git" \ + config http.uploadpack true && + set_askpass user@host pass@host +' + +setup_askpass_helper + +test_expect_success 'remote archive does not work with protocol v1' ' + test_when_finished "rm -f d5.zip" && + test_must_fail git -c protocol.version=1 archive \ + --remote="$HTTPD_URL/auth/smart/bare.git" \ + --output=d5.zip HEAD >actual 2>&1 && + cat >expect <<-EOF && + fatal: can${SQ}t connect to subservice git-upload-archive + EOF + test_cmp expect actual +' + +test_expect_success 'archive remote http repository' ' + test_when_finished "rm -f d5.zip" && + git archive --remote="$HTTPD_URL/auth/smart/bare.git" \ + --output=d5.zip HEAD && + test_cmp_bin d.zip d5.zip +' + test_done diff --git a/transport-helper.c b/transport-helper.c index 3b036ae1ca..566f7473df 100644 --- a/transport-helper.c +++ b/transport-helper.c @@ -628,7 +628,8 @@ static int process_connect_service(struct transport *transport, ret = run_connect(transport, &cmdbuf); } else if (data->stateless_connect && (get_protocol_version_config() == protocol_v2) && - !strcmp("git-upload-pack", name)) { + (!strcmp("git-upload-pack", name) || + !strcmp("git-upload-archive", name))) { strbuf_addf(&cmdbuf, "stateless-connect %s\n", name); ret = run_connect(transport, &cmdbuf); if (ret)