From patchwork Tue Jan 16 13:39:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiang Xin X-Patchwork-Id: 13520818 Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 58BA71BC5A for ; Tue, 16 Jan 2024 13:39:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="FDEISxyk" Received: by mail-pl1-f169.google.com with SMTP id d9443c01a7336-1d480c6342dso74772985ad.2 for ; Tue, 16 Jan 2024 05:39:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1705412374; x=1706017174; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7X3uz/zjGS6q2PRVI/WvWaSPUaq9er1s6Ew94X99/y8=; b=FDEISxykXlDVEjRBzwYwHFmy0JLMCDIMBH9X5fj95RLXUANGociOxPQ8L40n3xKBFM +zSz8fzAmGt2qVPia3sHYW82ummuAwwTxnV+PE1Esgmg3LOBi/l31AGsUj9xiH+q23l/ Ly5pAW8u2Pv/+VX2Fzoat/BW2roC4CVHOwkmREB/Eafg0m+GzRL6hHhr5WCBxEZIYKdy SqQoaAScxY/+QQQjGln4S6o5GMhdl0nYPx1mDMmASfAw2+kMJJzD1NyC+rEqD9K8KcHW OYbPhFaXgZPhlUVRFe2BycKtEzmCptdfn9IGvVnFMpTq4wv2MLwigOOxwcKIi68knXqG bblw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705412374; x=1706017174; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7X3uz/zjGS6q2PRVI/WvWaSPUaq9er1s6Ew94X99/y8=; b=ix/CKM+1IIuKnYfT2RrCSB4EDgK5gTrUV3jMADA9Fis9oeN7hbWm/oU4l6n4WmM1mn YotzJIbrxSaAySwE8vguUFfZV2HSJahiPN2sn7ZSGV/2BtkUCAF8EJvPgf3XJ/Juh3y2 wgxW2uQOlT7a5SM79hR0DffZQizH9RknQHv/UBStKC3Ri1jCP9NP8cO9kxH1yAUMBd6a o3I84ML3qw/zfyQC/ZB6+swtHloWuXSrQwM3rET6gTViSCOZ1/TU7KnhpyIwk8Mu8FDi PlPkgK/2pAgi9xEElJyHCdZ1WZkcm3APzn7aAK5LkxXWGAPzJ9/DDeCZ1V70uPed3djg 4dlQ== X-Gm-Message-State: AOJu0YzhHT3BYGjPyCN7BynYTwQ4re5omccVWlLFN0bvc3ab12duTCga 2AFZv1MRQ/6VNUdxn3rZ08TCiMC9Gb+5jw== X-Google-Smtp-Source: AGHT+IHq+So+kkstmufV/5yQ9BB/SK1ZA2SMcp7ZJTQcIvlzGZr+CSDNWPHfZEhj4PCjYS+gj7dLsQ== X-Received: by 2002:a17:902:76c2:b0:1d5:167:48d0 with SMTP id j2-20020a17090276c200b001d5016748d0mr7817899plt.47.1705412374424; Tue, 16 Jan 2024 05:39:34 -0800 (PST) Received: from tigtog-proxy.localdomain.localdomain (144.34.163.219.16clouds.com. [144.34.163.219]) by smtp.gmail.com with ESMTPSA id l8-20020a170902f68800b001d5c38bfa33sm4303351plg.307.2024.01.16.05.39.33 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 16 Jan 2024 05:39:33 -0800 (PST) From: Jiang Xin To: Git List , Junio C Hamano , Linus Arver Cc: Jiang Xin Subject: [PATCH v5 1/6] transport-helper: no connection restriction in connect_helper Date: Tue, 16 Jan 2024 21:39:25 +0800 Message-Id: X-Mailer: git-send-email 2.32.0.rc3 In-Reply-To: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Jiang Xin When commit b236752a (Support remote archive from all smart transports, 2009-12-09) added "remote archive" support for "smart transports", it was for transport that supports the ".connect" method. The "connect_helper()" function protected itself from getting called for a transport without the method before calling process_connect_service(), which only worked with the ".connect" method. Later, commit edc9caf7 (transport-helper: introduce stateless-connect, 2018-03-15) added a way for a transport without the ".connect" method to establish a "stateless" connection in protocol-v2, where process_connect_service() was taught to handle the ".stateless_connect" method, making the old protection too strict. But commit edc9caf7 forgot to adjust this protection accordingly. Remove the restriction in the "connect_helper()" function and give the function "process_connect_service()" the opportunity to establish a connection using ".connect" or ".stateless_connect" for protocol v2. So we can connect with a stateless-rpc and do something useful. E.g., in a later commit, implements remote archive for a repository over HTTP protocol. Helped-by: Junio C Hamano Helped-by: Linus Arver Signed-off-by: Jiang Xin --- transport-helper.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/transport-helper.c b/transport-helper.c index 49811ef176..2e127d24a5 100644 --- a/transport-helper.c +++ b/transport-helper.c @@ -662,8 +662,6 @@ static int connect_helper(struct transport *transport, const char *name, /* Get_helper so connect is inited. */ get_helper(transport); - if (!data->connect) - die(_("operation not supported by protocol")); if (!process_connect_service(transport, name, exec)) die(_("can't connect to subservice %s"), name); From patchwork Tue Jan 16 13:39:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiang Xin X-Patchwork-Id: 13520819 Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E4F501BC5C for ; Tue, 16 Jan 2024 13:39:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="WxIsoNMN" Received: by mail-pl1-f177.google.com with SMTP id d9443c01a7336-1d5efddb33dso2925515ad.1 for ; Tue, 16 Jan 2024 05:39:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1705412375; x=1706017175; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=cAUGC2z588o0P2pXwul4EzAT8d2kTMFHkZMpn6Ua8qs=; b=WxIsoNMN3q3neu4fhlTzKxYNMnPP+eeJKQdGKxn2y3AOefwSumscrgf27Zn58ixcPT DkduOn57eY8nXWJrbx+HTStBYaSN8P9iW1un4LIrHxx1+xevI9KXEESfSfzGNHHLidcg eGxAQ4AR4SKC95PWDXOOINbx8mHmkhpp7rmJLHtNzX2tOw+A/2PwkBQUixL6OVxOc7pf IM677sc27as1GJcfjW+9dCAVbeVobJJVEskNCIUdPtLnxxlCEL5nwfhTLNIQMFmfKAdm InuapjCPAHeOWIYBolTXwSSEV2ESoKUl4QJI1xvYFnHIewXcHqE/u7ehY8+Zqr0tG0uY uVZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705412375; x=1706017175; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cAUGC2z588o0P2pXwul4EzAT8d2kTMFHkZMpn6Ua8qs=; b=sChVZCKUqX5LyL1BqlYNNqB73ib6fh7lRiZhCxAiLRM1tvQecgUyJGcBynUoLlWBTm W4vx2FloVssWuUwG/q2aXV6eGHGETLcVCqLqb4eIw5HIR0psHlhJL867R0K7Lv7JbjwL +4HG/z5dI+0Lcw8pNnJVCBrlQYsXDhKjICYnaYta12lWAxgbn8Zz35CgczpUbcZHM4Si Cb0ey4NrxCc7MdCbOeUVlhOvIRXGhRlgoyg5Xeq4tDJSuExY7KmgXTYfXqqWihQGztd4 o78PFJF5i3qr71y0xyWqreFMR8O1tkvaBFp4K+nOdtm+XhzND0uyDICE9662Lc5Y6llA GaYQ== X-Gm-Message-State: AOJu0Yxr8gU75THkOhyRLVVtQ9H1Sc4FEOgZLYvaQ67d+hl0lr7HOf2P 4hf5HT8hr/D0WUs/X6A0JvEUqQo6I3Q6zg== X-Google-Smtp-Source: AGHT+IFYgKtMdmiDFLIt4v8MHor8bgHgHuF4FGqvS/qlfJveDobZweGViWxU3950ZEtS05YDgtNAIA== X-Received: by 2002:a17:902:d485:b0:1d4:d5bb:5d7e with SMTP id c5-20020a170902d48500b001d4d5bb5d7emr4569322plg.58.1705412375099; Tue, 16 Jan 2024 05:39:35 -0800 (PST) Received: from tigtog-proxy.localdomain.localdomain (144.34.163.219.16clouds.com. [144.34.163.219]) by smtp.gmail.com with ESMTPSA id l8-20020a170902f68800b001d5c38bfa33sm4303351plg.307.2024.01.16.05.39.34 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 16 Jan 2024 05:39:34 -0800 (PST) From: Jiang Xin To: Git List , Junio C Hamano , Linus Arver Cc: Jiang Xin Subject: [PATCH v5 2/6] remote-curl: supports git-upload-archive service Date: Tue, 16 Jan 2024 21:39:26 +0800 Message-Id: <6be331b22d51e1f6f96cb0035d99db5b8cede676.1705411391.git.zhiyou.jx@alibaba-inc.com> X-Mailer: git-send-email 2.32.0.rc3 In-Reply-To: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Jiang Xin Add new service (git-upload-archive) support in remote-curl, so we can support remote archive over HTTP/HTTPS protocols. Differences between git-upload-archive and other serices: 1. The git-archive command does not expect to see protocol version and capabilities when connecting to remote-helper, so do not send them in remote-curl for the git-upload-archive service. 2. We need to detect protocol version by calling discover_refs(), Fallback to use the git-upload-pack service (which, like git-upload-archive, is a read-only operation) to discover protocol version. Signed-off-by: Jiang Xin --- remote-curl.c | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/remote-curl.c b/remote-curl.c index ef05752ca5..ce6cb8ac05 100644 --- a/remote-curl.c +++ b/remote-curl.c @@ -1447,8 +1447,14 @@ static int stateless_connect(const char *service_name) * establish a stateless connection, otherwise we need to tell the * client to fallback to using other transport helper functions to * complete their request. + * + * The "git-upload-archive" service is a read-only operation. Fallback + * to use "git-upload-pack" service to discover protocol version. */ - discover = discover_refs(service_name, 0); + if (!strcmp(service_name, "git-upload-archive")) + discover = discover_refs("git-upload-pack", 0); + else + discover = discover_refs(service_name, 0); if (discover->version != protocol_v2) { printf("fallback\n"); fflush(stdout); @@ -1486,9 +1492,11 @@ static int stateless_connect(const char *service_name) /* * Dump the capability listing that we got from the server earlier - * during the info/refs request. + * during the info/refs request. This does not work with the + * "git-upload-archive" service. */ - write_or_die(rpc.in, discover->buf, discover->len); + if (strcmp(service_name, "git-upload-archive")) + write_or_die(rpc.in, discover->buf, discover->len); /* Until we see EOF keep sending POSTs */ while (1) { From patchwork Tue Jan 16 13:39:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiang Xin X-Patchwork-Id: 13520820 Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D075C1BDCB for ; Tue, 16 Jan 2024 13:39:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Ob41tJL6" Received: by mail-pl1-f182.google.com with SMTP id d9443c01a7336-1d5f5a2e828so657405ad.3 for ; Tue, 16 Jan 2024 05:39:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1705412376; x=1706017176; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=AhTtqH48/C86XL/TwWluZBelSMih1M4GmOBqLA2IEew=; b=Ob41tJL6DJ+dnf/IqUAv1RC0DYJzIgsb3BB2wDQ7QJgywEFIwPqyYmeVKHZbwLi4yG X8cTdlZTWOj3PNUSCP1uTyyEDwioaT+WiP3WVlHmzfqX7yWmn8E89CYuxQoeCnu4l9Jt g5B+hNHHP7ZSj4R0kC3gUBd0kLMmPeFMglXkxwgOaefCYaziSKrDRf48yAk6K2kGzS03 Q/pO0chwMEeDikBSoQhWV4pYyNQUZ8f7I7FW8ZHPjdd0LeWyR6wr76q8DK2cZHftBHC8 WgEgF5ph7E4zTif3jxti4+keyIHck7Ux42eeEPU/2By5CnQ1Sw0syhpWWnBq2XZ4xXV5 5wDw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705412376; x=1706017176; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AhTtqH48/C86XL/TwWluZBelSMih1M4GmOBqLA2IEew=; b=K1pi/q5icWWaA2kHIPIvsh1e5gAQgKL3W6iwo7661EaDhfvksG1bSKTt9InkfraDvf TGBQMtdaaqLCd5nRSme+m/fZvvnYUulaIzZLzD9NwrvKmtLN6ercVOPN51lZ4W7cI409 f+LO44BNXEbffbyvZk+f/58d+/6BvURj/7pDb7PrZey1uS6sv6IN7aFXOYDPUGe8kKqB nPiWOmdKywHATXSpGzSFjMljFnzRGORFqQY+/vX1UzR3hYh6IuKY83tglrm361GpswpD 9SBteqrH9g1zV2DLNCEGadrWUqOFqZF4G019U615XHw2rXtB4YWtgYm286SSLcJa0fu4 IO/w== X-Gm-Message-State: AOJu0YyHpS3Z5bMpe49MdCy2n11MPW85k+MFXLYfOAW7R8ymk4MuiQGf yTpZLdgh9nCa+IiJMkB3acPN7QXtTQSnFw== X-Google-Smtp-Source: AGHT+IGmos3/7eBWVW5F7ZQ2WX7bFSaGe5HUeeIrUfTTxU4074E9N/aW2LzBUORWs5GgGpRDgK2VWQ== X-Received: by 2002:a17:902:d4d2:b0:1d3:77c4:59db with SMTP id o18-20020a170902d4d200b001d377c459dbmr4601365plg.139.1705412375841; Tue, 16 Jan 2024 05:39:35 -0800 (PST) Received: from tigtog-proxy.localdomain.localdomain (144.34.163.219.16clouds.com. [144.34.163.219]) by smtp.gmail.com with ESMTPSA id l8-20020a170902f68800b001d5c38bfa33sm4303351plg.307.2024.01.16.05.39.35 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 16 Jan 2024 05:39:35 -0800 (PST) From: Jiang Xin To: Git List , Junio C Hamano , Linus Arver Cc: Jiang Xin Subject: [PATCH v5 3/6] transport-helper: protocol-v2 supports upload-archive Date: Tue, 16 Jan 2024 21:39:27 +0800 Message-Id: X-Mailer: git-send-email 2.32.0.rc3 In-Reply-To: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Jiang Xin We used to support only git-upload-pack service for protocol-v2. In order to support remote archive over HTTP/HTTPS protocols, add new service support for git-upload-archive in protocol-v2. Signed-off-by: Jiang Xin --- transport-helper.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/transport-helper.c b/transport-helper.c index 2e127d24a5..6fe9f4f208 100644 --- a/transport-helper.c +++ b/transport-helper.c @@ -628,7 +628,8 @@ static int process_connect_service(struct transport *transport, ret = run_connect(transport, &cmdbuf); } else if (data->stateless_connect && (get_protocol_version_config() == protocol_v2) && - !strcmp("git-upload-pack", name)) { + (!strcmp("git-upload-pack", name) || + !strcmp("git-upload-archive", name))) { strbuf_addf(&cmdbuf, "stateless-connect %s\n", name); ret = run_connect(transport, &cmdbuf); if (ret) From patchwork Tue Jan 16 13:39:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiang Xin X-Patchwork-Id: 13520821 Received: from mail-pf1-f180.google.com (mail-pf1-f180.google.com [209.85.210.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6A8BD1BDDC for ; Tue, 16 Jan 2024 13:39:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="FxxgswzW" Received: by mail-pf1-f180.google.com with SMTP id d2e1a72fcca58-6d9b267007fso5628046b3a.3 for ; Tue, 16 Jan 2024 05:39:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1705412376; x=1706017176; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=AoNFgjU2uuAwx1uE3fJGx4ANiWcUafSa0KGE2hPV8Zg=; b=FxxgswzW7l7RtcUDhyZrRDWg/isGDkrzcgiEud6tDQZhixRQf82q6zMH+msGYtX0kR dxaQvzknLCkmiUEZ+CJNx38e7E1gE/AI6eXVmQOxLN3ShTUZ8tTGiiYec42TUAgKsYGZ xU+0zSLEzALYcjcJx4Eoyg6pJKej/jafKcDZrEgPciLlVrBEl+8a9CmF/Sh8IjzPK5P4 hdfmlTnhz0KkLzlS8l4GMeZmF7+EdZCxkK9SkIYfeAPfyFKcxJ/CwSIH1dPrOgr0VSzE DP9gPOcTRPgLpH7gGxudN3+tW7z1dRUYCgz3uyZHCpatKt/JpTmc0Jm1WJTwsKP9qq5S XUzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705412376; x=1706017176; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AoNFgjU2uuAwx1uE3fJGx4ANiWcUafSa0KGE2hPV8Zg=; b=uFcN3jfY6q97bUYH4X+0p2VLFcQgpMZ0hc0mr8ZEoLp9KdQxPZOzZZQ6LFVqnMB6Gp RrHOJOVH4ZJI9zqQ8zaNzeFCY5aAzGQykMeEqvbQhqU0PY/EluSo6LkNIilzfIU0tayJ c3cIlnxQRkHnWtP4vFYX43IizO4ZpdPGLKogJxPRL2zwUzwwXRt++CW2YjtgvUrIy5op wACCNeyzUY3gP4kWyMNVYYtSN0jXMVknbIBpyaQuU7O4M/GNKpNjbemovLRzevJUrgM9 XZ3AEnDnhFLFZycJnDJHoizhAktrKXlpCNsquJ3nvZ5QX+fubkwKNxk3fPdf3xASRZ41 dIIA== X-Gm-Message-State: AOJu0Yxl78nPRb+Yo7Whrs1Y1zNRh1/DSrBBcPB6ARNuSUnJJrdqHCdY aW0S2kd5LBEoDyXKmNHkVDVajFBAi2aTTQ== X-Google-Smtp-Source: AGHT+IFigsRzZovtE+FsGjFOpfR/K5MWWBVj+Oetu/jmudK16AvR+bc5PXDsaw+9DwWbIfQUqg391A== X-Received: by 2002:a05:6a20:9f86:b0:19a:fad2:5051 with SMTP id mm6-20020a056a209f8600b0019afad25051mr2240543pzb.43.1705412376613; Tue, 16 Jan 2024 05:39:36 -0800 (PST) Received: from tigtog-proxy.localdomain.localdomain (144.34.163.219.16clouds.com. [144.34.163.219]) by smtp.gmail.com with ESMTPSA id l8-20020a170902f68800b001d5c38bfa33sm4303351plg.307.2024.01.16.05.39.35 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 16 Jan 2024 05:39:36 -0800 (PST) From: Jiang Xin To: Git List , Junio C Hamano , Linus Arver Cc: Jiang Xin , Eric Sunshine Subject: [PATCH v5 4/6] http-backend: new rpc-service for git-upload-archive Date: Tue, 16 Jan 2024 21:39:28 +0800 Message-Id: X-Mailer: git-send-email 2.32.0.rc3 In-Reply-To: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Jiang Xin Add new rpc-service "upload-archive" in http-backend to add server side support for remote archive over HTTP/HTTPS protocols. Also add new test cases in t5003. In the test case "archive remote http repository", git-archive exits with a non-0 exit code even though we create the archive correctly. It will be fixed in a later commit. Helped-by: Eric Sunshine Signed-off-by: Jiang Xin --- http-backend.c | 13 ++++++++++--- t/t5003-archive-zip.sh | 34 ++++++++++++++++++++++++++++++++++ 2 files changed, 44 insertions(+), 3 deletions(-) diff --git a/http-backend.c b/http-backend.c index ff07b87e64..1ed1e29d07 100644 --- a/http-backend.c +++ b/http-backend.c @@ -38,6 +38,7 @@ struct rpc_service { static struct rpc_service rpc_service[] = { { "upload-pack", "uploadpack", 1, 1 }, { "receive-pack", "receivepack", 0, -1 }, + { "upload-archive", "uploadarchive", 0, -1 }, }; static struct string_list *get_parameters(void) @@ -639,10 +640,15 @@ static void check_content_type(struct strbuf *hdr, const char *accepted_type) static void service_rpc(struct strbuf *hdr, char *service_name) { - const char *argv[] = {NULL, "--stateless-rpc", ".", NULL}; + struct strvec argv = STRVEC_INIT; struct rpc_service *svc = select_service(hdr, service_name); struct strbuf buf = STRBUF_INIT; + strvec_push(&argv, svc->name); + if (strcmp(service_name, "git-upload-archive")) + strvec_push(&argv, "--stateless-rpc"); + strvec_push(&argv, "."); + strbuf_reset(&buf); strbuf_addf(&buf, "application/x-git-%s-request", svc->name); check_content_type(hdr, buf.buf); @@ -655,9 +661,9 @@ static void service_rpc(struct strbuf *hdr, char *service_name) end_headers(hdr); - argv[0] = svc->name; - run_service(argv, svc->buffer_input); + run_service(argv.v, svc->buffer_input); strbuf_release(&buf); + strvec_clear(&argv); } static int dead; @@ -723,6 +729,7 @@ static struct service_cmd { {"GET", "/objects/pack/pack-[0-9a-f]{64}\\.idx$", get_idx_file}, {"POST", "/git-upload-pack$", service_rpc}, + {"POST", "/git-upload-archive$", service_rpc}, {"POST", "/git-receive-pack$", service_rpc} }; diff --git a/t/t5003-archive-zip.sh b/t/t5003-archive-zip.sh index fc499cdff0..6f85bd3463 100755 --- a/t/t5003-archive-zip.sh +++ b/t/t5003-archive-zip.sh @@ -239,4 +239,38 @@ check_zip with_untracked2 check_added with_untracked2 untracked one/untracked check_added with_untracked2 untracked two/untracked +# Test remote archive over HTTP protocol. +# +# Note: this should be the last part of this test suite, because +# by including lib-httpd.sh, the test may end early if httpd tests +# should not be run. +# +. "$TEST_DIRECTORY"/lib-httpd.sh +start_httpd + +test_expect_success "setup for HTTP protocol" ' + cp -R bare.git "$HTTPD_DOCUMENT_ROOT_PATH/bare.git" && + git -C "$HTTPD_DOCUMENT_ROOT_PATH/bare.git" \ + config http.uploadpack true && + set_askpass user@host pass@host +' + +setup_askpass_helper + +test_expect_success 'remote archive does not work with protocol v1' ' + test_must_fail git -c protocol.version=1 archive \ + --remote="$HTTPD_URL/auth/smart/bare.git" \ + --output=remote-http.zip HEAD >actual 2>&1 && + cat >expect <<-EOF && + fatal: can${SQ}t connect to subservice git-upload-archive + EOF + test_cmp expect actual +' + +test_expect_success 'archive remote http repository' ' + test_must_fail git archive --remote="$HTTPD_URL/auth/smart/bare.git" \ + --output=remote-http.zip HEAD && + test_cmp_bin d.zip remote-http.zip +' + test_done From patchwork Tue Jan 16 13:39:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiang Xin X-Patchwork-Id: 13520822 Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4B6FA1BDED for ; Tue, 16 Jan 2024 13:39:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="POA2rJt3" Received: by mail-pl1-f176.google.com with SMTP id d9443c01a7336-1d51ba18e1bso82908285ad.0 for ; Tue, 16 Jan 2024 05:39:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1705412377; x=1706017177; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=nX+1DBLKTnyVMt/iObLdOwJH8HehaYVHlLjcOyCJsKA=; b=POA2rJt3qww4HyTKMav3H2yMRStKBMdrsHK7rXwyY0nEx6Sd6D0gRm/BrJN9avhwIg s8SwF3tAZwcPTrQ0h3JV2O/xzykc9EiCUBt3pqVjHfjHBVMKJqp3RRU5FDAq9+uMBbuq Xf0/b+nbOwDH68XUDozfPQP5h22VrQOrT1x1JpLIY9VhhhnmwcnFUfp14Ragnn3nQtlo w6qCwWths8K4XimQn/va6uHG/Z67NUqVPHVyurBJTUXcu+1F1LMLl4kfzIuGnzjtvIx1 8ejbktMTxxWmWOAZ2PEsNVxSkHOORE+T1sBHLFBi7g5GWxveBHGby5ZyFO70Zv4lSv2u wj6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705412377; x=1706017177; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nX+1DBLKTnyVMt/iObLdOwJH8HehaYVHlLjcOyCJsKA=; b=r7Klr0VoXaL70zn9/LiA9D5Y8/6kUETSUBRGtueJqmKgWjMnTlUNd7gFmA5JUYMGIx 5m92/Tnm6FVgf+uZkSCbNFWiM6Tc8+HB9Ntl/Nurezdg8dGENakZap6ZI4wip6nm/NBa 2O+23nFx+EtIWOGocwBlqKeX/Ol8UIhwf4yKkiudmvIiUGlj3k0l1JnzYUdFT9Azwczs 1I/4Q+QEDBTi4B6YnJMbBQeIogrjdYkCRG4h51/N+cwVwgz0emhoDJbAbjuFt8fEachq j5nKxV5OO3yAMg4eE7/Akv9Jwcd9SB49k93IfM3pFfcztlj9n+vRJGbhRW+WrF4wfdTo OFbg== X-Gm-Message-State: AOJu0Yx3Bxd8z8agPmivHtfp+/6l6opf+Ne/T24eCLP9+HY/Hcffd/a1 59851cziiqmL7B74OjomwR1oMYvc+7OTvg== X-Google-Smtp-Source: AGHT+IHrc7/sjCSPbGthBeTzbNQ/qn1AEQWjZlflAU86HENfJ4skABwnDEztRNKKZVdjYPeBYUpLyg== X-Received: by 2002:a17:903:32cf:b0:1d5:aed4:b4c2 with SMTP id i15-20020a17090332cf00b001d5aed4b4c2mr9516404plr.35.1705412377335; Tue, 16 Jan 2024 05:39:37 -0800 (PST) Received: from tigtog-proxy.localdomain.localdomain (144.34.163.219.16clouds.com. [144.34.163.219]) by smtp.gmail.com with ESMTPSA id l8-20020a170902f68800b001d5c38bfa33sm4303351plg.307.2024.01.16.05.39.36 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 16 Jan 2024 05:39:37 -0800 (PST) From: Jiang Xin To: Git List , Junio C Hamano , Linus Arver Cc: Jiang Xin Subject: [PATCH v5 5/6] transport-helper: call do_take_over() in connect_helper Date: Tue, 16 Jan 2024 21:39:29 +0800 Message-Id: <6ac0c8e105febe526dc64182845832297656a8a5.1705411391.git.zhiyou.jx@alibaba-inc.com> X-Mailer: git-send-email 2.32.0.rc3 In-Reply-To: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Jiang Xin After successfully connecting to the smart transport by calling process_connect_service() in connect_helper(), run do_take_over() to replace the old vtable with a new one which has methods ready for the smart transport connection. This will fix the exit code of git-archive in test case "archive remote http repository" of t5003. The connect_helper() function is used as the connect method of the vtable in "transport-helper.c", and it is called by transport_connect() in "transport.c" to setup a connection. The only place that we call transport_connect() so far is in "builtin/archive.c". Without running do_take_over(), it may fail to call transport_disconnect() in run_remote_archiver() of "builtin/archive.c". This is because for a stateless connection and a service like "git-upload-archive", the remote helper may receive a SIGPIPE signal and exit early. To have a graceful disconnect method by calling do_take_over() will solve this issue. Helped-by: Linus Arver Signed-off-by: Jiang Xin --- t/t5003-archive-zip.sh | 2 +- transport-helper.c | 2 ++ 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/t/t5003-archive-zip.sh b/t/t5003-archive-zip.sh index 6f85bd3463..961c6aac25 100755 --- a/t/t5003-archive-zip.sh +++ b/t/t5003-archive-zip.sh @@ -268,7 +268,7 @@ test_expect_success 'remote archive does not work with protocol v1' ' ' test_expect_success 'archive remote http repository' ' - test_must_fail git archive --remote="$HTTPD_URL/auth/smart/bare.git" \ + git archive --remote="$HTTPD_URL/auth/smart/bare.git" \ --output=remote-http.zip HEAD && test_cmp_bin d.zip remote-http.zip ' diff --git a/transport-helper.c b/transport-helper.c index 6fe9f4f208..91381be622 100644 --- a/transport-helper.c +++ b/transport-helper.c @@ -669,6 +669,8 @@ static int connect_helper(struct transport *transport, const char *name, fd[0] = data->helper->out; fd[1] = data->helper->in; + + do_take_over(transport); return 0; } From patchwork Tue Jan 16 13:39:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiang Xin X-Patchwork-Id: 13520823 Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0C5311BDFD for ; Tue, 16 Jan 2024 13:39:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ALr+pH9e" Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-1d427518d52so71476035ad.0 for ; Tue, 16 Jan 2024 05:39:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1705412378; x=1706017178; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=CozsLbOmwkgZRdoKGjxxiV76kRVAqRlsUpx1KPOtBJg=; b=ALr+pH9e2XZ9NPW6HhO24x911codgs8kffhNMBNA/wpzxtdIHER1nGMIJBgX6N+edS imdWj92pe+Gx6aAPDYGhzwrPlM1qut/xUqfa2Re1sqLp1Ll6NVXfsN3N6nRg2AbB1VsW J1rA2Dx1CpO37VLVpepsqM6twIlpSyCSFwdPAy6O9fpDHAnCqLOakyNg0ap6GgSCYrgr bDtJhsJ8JWbPWbB3jNwGPRy/QiAzObV2Apc/M5yEG18d9VfrXU8I0QxZ7MUrd1N/qkWB a9PA9g7us+NSCci56Q8RKUo28Sxlm1tVraDwsMqKpTTtd8ondIsE64/mJrzWph7VMR4x ebzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705412378; x=1706017178; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=CozsLbOmwkgZRdoKGjxxiV76kRVAqRlsUpx1KPOtBJg=; b=Qvky0mFpXz4B7BdUpbfWBdKQCgBFt7EdZRgCqqEyGSAC1oOABs6wIm3jbiw+wnKZxr A8aExlvKq18KdzuNXU48PaDiMuZcv7QaKwQ+edPq+Jfy2Y4hP+I57G4f8XbXC+urWmmQ TxcjIgP/G+OUHsRwytc4kcONbgtWuDgEC4BzAedyN/HsmCrszdm7Kual0BWuDiqMJks+ bAZ3tQKPZ+Da+ptWhWPqnqLPnYzzE9N0LbOntS5UW6r6qG5H3EO+2PPsH+6wjYlLlKKX Q7yvQ+CIPcDNbcYkCrSdI2yq4RecWC38V9DQzdzoE6HhrPLzbKK/rCPTISHnch51/XAg t/pQ== X-Gm-Message-State: AOJu0YwIbkHXkyK5TYPyyGgvIDjVI/SzFX5RQBIVG3Q4CKm7X7RiRLPC z+eCRxICA2eg081YERxCV1ADdkNaxqALmQ== X-Google-Smtp-Source: AGHT+IH/hiN2plMLZpOHAQoqW3J+w2Cl0TqdVgsi+qPyeJyV9AG+wq3UXZtKYE1oQNyaazjoqFGFlw== X-Received: by 2002:a17:902:ed04:b0:1d5:4df5:df9b with SMTP id b4-20020a170902ed0400b001d54df5df9bmr8710108pld.4.1705412378106; Tue, 16 Jan 2024 05:39:38 -0800 (PST) Received: from tigtog-proxy.localdomain.localdomain (144.34.163.219.16clouds.com. [144.34.163.219]) by smtp.gmail.com with ESMTPSA id l8-20020a170902f68800b001d5c38bfa33sm4303351plg.307.2024.01.16.05.39.37 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 16 Jan 2024 05:39:37 -0800 (PST) From: Jiang Xin To: Git List , Junio C Hamano , Linus Arver Cc: Jiang Xin Subject: [PATCH v5 6/6] transport-helper: call do_take_over() in process_connect Date: Tue, 16 Jan 2024 21:39:30 +0800 Message-Id: <423a89c59306e9c33851b7d36c685ddfce45736c.1705411391.git.zhiyou.jx@alibaba-inc.com> X-Mailer: git-send-email 2.32.0.rc3 In-Reply-To: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Jiang Xin The existing pattern among all callers of process_connect() seems to be if (process_connect(...)) { do_take_over(); ... dispatch to the underlying method ... } ... otherwise implement the fallback ... where the return value from process_connect() is the return value of the call it makes to process_connect_service(). Move the call of do_take_over() inside process_connect(), so that calling the process_connect() function is more concise and will not miss do_take_over(). Suggested-by: Junio C Hamano Signed-off-by: Jiang Xin --- transport-helper.c | 22 +++++++++------------- 1 file changed, 9 insertions(+), 13 deletions(-) diff --git a/transport-helper.c b/transport-helper.c index 91381be622..566f7473df 100644 --- a/transport-helper.c +++ b/transport-helper.c @@ -646,6 +646,7 @@ static int process_connect(struct transport *transport, struct helper_data *data = transport->data; const char *name; const char *exec; + int ret; name = for_push ? "git-receive-pack" : "git-upload-pack"; if (for_push) @@ -653,7 +654,10 @@ static int process_connect(struct transport *transport, else exec = data->transport_options.uploadpack; - return process_connect_service(transport, name, exec); + ret = process_connect_service(transport, name, exec); + if (ret) + do_take_over(transport); + return ret; } static int connect_helper(struct transport *transport, const char *name, @@ -685,10 +689,8 @@ static int fetch_refs(struct transport *transport, get_helper(transport); - if (process_connect(transport, 0)) { - do_take_over(transport); + if (process_connect(transport, 0)) return transport->vtable->fetch_refs(transport, nr_heads, to_fetch); - } /* * If we reach here, then the server, the client, and/or the transport @@ -1145,10 +1147,8 @@ static int push_refs(struct transport *transport, { struct helper_data *data = transport->data; - if (process_connect(transport, 1)) { - do_take_over(transport); + if (process_connect(transport, 1)) return transport->vtable->push_refs(transport, remote_refs, flags); - } if (!remote_refs) { fprintf(stderr, @@ -1189,11 +1189,9 @@ static struct ref *get_refs_list(struct transport *transport, int for_push, { get_helper(transport); - if (process_connect(transport, for_push)) { - do_take_over(transport); + if (process_connect(transport, for_push)) return transport->vtable->get_refs_list(transport, for_push, transport_options); - } return get_refs_list_using_list(transport, for_push); } @@ -1277,10 +1275,8 @@ static int get_bundle_uri(struct transport *transport) { get_helper(transport); - if (process_connect(transport, 0)) { - do_take_over(transport); + if (process_connect(transport, 0)) return transport->vtable->get_bundle_uri(transport); - } return -1; }