From patchwork Thu Feb 21 20:24:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Tan X-Patchwork-Id: 10824731 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C1F9F6C2 for ; Thu, 21 Feb 2019 20:24:50 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B2BD031C18 for ; Thu, 21 Feb 2019 20:24:50 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A6BC931C5F; Thu, 21 Feb 2019 20:24:50 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4589831C18 for ; Thu, 21 Feb 2019 20:24:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726383AbfBUUYt (ORCPT ); Thu, 21 Feb 2019 15:24:49 -0500 Received: from mail-io1-f73.google.com ([209.85.166.73]:35028 "EHLO mail-io1-f73.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726074AbfBUUYs (ORCPT ); Thu, 21 Feb 2019 15:24:48 -0500 Received: by mail-io1-f73.google.com with SMTP id s9so2746889ioj.2 for ; Thu, 21 Feb 2019 12:24:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=M/vmNLYOylHU5sLtsUVnH3J1gmi7Ht+utm7HKHC32rM=; b=CL21NoTQbjoJ1qvmNezkIvOeMwV1qCFBGvqZRs4+X0VPu3VUyqqchgbuGAl0u/iW0Y i95SoGbp8E0gJP/EU/wd0WuOGwC8EsUjzapAeCW7gvOYNzefgFtphTYH8aGSUnV9FZBB boukW5x9fD5HX2pYlmZf0oVSq7qy6HV7vYzydJxmifQwRsLOl1+FEcddnoiduCppIogg HNozr5no0sfFCBndCBbHoPQpjtqx6NiiuFrWBA6r8cBRJVYhpAe3nolxQmD93M+m+xuG PxQ3Mjl+DH8ghhVwzQ/pY+fLfTZA56gZcGKa5GP3GGIEBNUBR29GMUFgKXPDBEcUEZEG uSRw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=M/vmNLYOylHU5sLtsUVnH3J1gmi7Ht+utm7HKHC32rM=; b=tlp7KypUjLjHkRhrzA/9IWvMRv89Os2+nXhC2mhHuSYeuh+BSnLyKdhoZ9aTDpz3VA vlNUxOWfCYYQ5SyU0Yu/SYdnqtvla3QFdgYwpiSa9kvjQQKZw2OEiVwzLVO4KBFMccoG 8kiUDwX57p1PEfqbdYdp9tJp91FGlvI1aHoW/u3Q1aDU6saZ2ThAKgSa1IuPmmjB3/Eg qbQQHNtSocC5euYTJTgd3c0xJGvtDRjdALkijSOxaaIxLTG+uoQS0q2204tBOHHc0BZO n4ks68beOwbU2g8GZbJtb44WelYUHfeHbUe1pIWAK+M2fbmZVQKSXEZTh2+2peG7JL6G /fuA== X-Gm-Message-State: AHQUAuYqPOIUAFsBgAU478zslehYz5S3ClRSAJRCLMoVKlK6l3o58Dor TyKDkoRDTYMIB81Fz1JcBsK0mzHFy1axxkh3xw2ZXkizYiR3JixEL+10SonlqNxoQishwe+H+RI Tq8Zw8z3vL+AlSAIOalGw8SfB3w3OZdApSTwQlz1cJJREPowYcVvxMYcuhAF2IjYmjsns9VmzbG Y2 X-Google-Smtp-Source: AHgI3IbSybUHpIFX4lRoRO+/XdNIXkMez5butSBdtK+8wjYkcBlfQuup0HgBgBnuAMcvIbssDdgDlCNJlcuqOGGLPK5v X-Received: by 2002:a24:2c53:: with SMTP id i80mr266954iti.2.1550780687187; Thu, 21 Feb 2019 12:24:47 -0800 (PST) Date: Thu, 21 Feb 2019 12:24:37 -0800 In-Reply-To: Message-Id: <8062cf9ce3d35bab974e745ef85ee4149619773d.1550780213.git.jonathantanmy@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.19.0.271.gfe8321ec05.dirty Subject: [PATCH v2 1/5] remote-curl: reduce scope of rpc_state.argv From: Jonathan Tan To: git@vger.kernel.org Cc: Jonathan Tan , peff@peff.net Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The argv field in struct rpc_state is only used in rpc_service(), and not in any functions it directly or indirectly calls. Refactor it to become an argument of rpc_service() instead. Signed-off-by: Jonathan Tan --- remote-curl.c | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/remote-curl.c b/remote-curl.c index bb7421023b..8b7baf6298 100644 --- a/remote-curl.c +++ b/remote-curl.c @@ -505,7 +505,6 @@ static void output_refs(struct ref *refs) struct rpc_state { const char *service_name; - const char **argv; struct strbuf *stdin_preamble; char *service_url; char *hdr_content_type; @@ -829,7 +828,8 @@ static int post_rpc(struct rpc_state *rpc) return err; } -static int rpc_service(struct rpc_state *rpc, struct discovery *heads) +static int rpc_service(struct rpc_state *rpc, struct discovery *heads, + const char **client_argv) { const char *svc = rpc->service_name; struct strbuf buf = STRBUF_INIT; @@ -840,7 +840,7 @@ static int rpc_service(struct rpc_state *rpc, struct discovery *heads) client.in = -1; client.out = -1; client.git_cmd = 1; - client.argv = rpc->argv; + client.argv = client_argv; if (start_command(&client)) exit(1); if (preamble) @@ -978,11 +978,10 @@ static int fetch_git(struct discovery *heads, memset(&rpc, 0, sizeof(rpc)); rpc.service_name = "git-upload-pack", - rpc.argv = args.argv; rpc.stdin_preamble = &preamble; rpc.gzip_request = 1; - err = rpc_service(&rpc, heads); + err = rpc_service(&rpc, heads, args.argv); if (rpc.result.len) write_or_die(1, rpc.result.buf, rpc.result.len); strbuf_release(&rpc.result); @@ -1112,10 +1111,9 @@ static int push_git(struct discovery *heads, int nr_spec, char **specs) memset(&rpc, 0, sizeof(rpc)); rpc.service_name = "git-receive-pack", - rpc.argv = args.argv; rpc.stdin_preamble = &preamble; - err = rpc_service(&rpc, heads); + err = rpc_service(&rpc, heads, args.argv); if (rpc.result.len) write_or_die(1, rpc.result.buf, rpc.result.len); strbuf_release(&rpc.result); From patchwork Thu Feb 21 20:24:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Tan X-Patchwork-Id: 10824733 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0810E6C2 for ; Thu, 21 Feb 2019 20:24:53 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E78CF31C18 for ; Thu, 21 Feb 2019 20:24:52 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DB95431C5F; Thu, 21 Feb 2019 20:24:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8076531C18 for ; Thu, 21 Feb 2019 20:24:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726429AbfBUUYv (ORCPT ); Thu, 21 Feb 2019 15:24:51 -0500 Received: from mail-qt1-f201.google.com ([209.85.160.201]:53777 "EHLO mail-qt1-f201.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726074AbfBUUYu (ORCPT ); Thu, 21 Feb 2019 15:24:50 -0500 Received: by mail-qt1-f201.google.com with SMTP id k37so27779113qtb.20 for ; Thu, 21 Feb 2019 12:24:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=g2OctTVip+vhQfZMynnuLwKCHVgBzQWnQ+tuo7kQZa4=; b=Ykqp0WLJfYs+qab22KaaG9gwuOas2iFuyaTQuT+VJkI18hZljsyVE12CQIXz2+/deF EvaR9nvYRprj6m0Uk1+sO/wlQxw1QCgkrBAZcQ+FLNTxEqLmy/nA+01OWus8cNp3YAr9 VE8GeUwvrgh1z77XDmzVWcu1c4E9IvXQ4r45YRbItmRMNYpjo50nugTJkxgGJukF1lrb zpcnBHbtgvXRJNo3lT4/rG9O+WoD4fmSfBrhukGiGR537yH4AcLlzSrfQdWUBxxX5Dm9 Av1p22SOBA4TyWbumUzugY1D43V8v2JwvZWrEU/EThe6Kuu2QxHaivw0VjwiJUeAzUJ3 qsfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=g2OctTVip+vhQfZMynnuLwKCHVgBzQWnQ+tuo7kQZa4=; b=beCds89GkUJhfUjCFUyv2d2aEVN50Ee7tvl+uilNkn0vC/jfWbqcqrSOybj7amxTK8 xx6NhapFnqZh8as5/T1sW70jIS4afifSoOgQfw4OAyqwVuDeh9Xao7XE2aXNYpCFIrmy Usv+QPTVxS0JoOOk6YdwxxcsykTO6zr4BK5QtiDe91/ytAkkLHPlaNED4Sc2n2xMlYy4 BgKJ8Qh2NTx2L8nIO23129010YCIcJ7ECBkPEA5N5tfHOdj675MGuQklPZTbM+8cc36a NfAxg+tXCFf0/EIB52FwIAvAQMizh/aZBdrCCmWnq03OzJDRa4wCjtfpV9mVyKNzZ2oh OgPA== X-Gm-Message-State: AHQUAuYLi0OACbaEesQN89IvwsyHwhF6rJ6Ry2IIDbJP6UpljLJ+J7A7 JsSSlhhJL3i1qh3/2K2ljizUqPzQr1ogN4f6l+Me7HPYf69r6kwD66C8EgrbP3uoDSN1lL3Bw4B NCU63TXUxKsuVZ7iNg6fzGNI+CnYgmD3Y9byBI6+tiQJ6tPXc9lwETEWsfFqNAXXlhY4KhawU8x bi X-Google-Smtp-Source: AHgI3IZZukR3tGWRy0IXU8SrzzsYudRINXRdBy+uWdkZof+vUUjrozV7LcVTKT47dU4QZLNnyKN7AFT7X8axL0FYRCSk X-Received: by 2002:a0c:95cd:: with SMTP id t13mr280024qvt.14.1550780689687; Thu, 21 Feb 2019 12:24:49 -0800 (PST) Date: Thu, 21 Feb 2019 12:24:38 -0800 In-Reply-To: Message-Id: <23d2414994e8ca2559940ccd43df0e2b2ceb712b.1550780213.git.jonathantanmy@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.19.0.271.gfe8321ec05.dirty Subject: [PATCH v2 2/5] remote-curl: reduce scope of rpc_state.stdin_preamble From: Jonathan Tan To: git@vger.kernel.org Cc: Jonathan Tan , peff@peff.net Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The stdin_preamble field in struct rpc_state is only used in rpc_service(), and not in any functions it directly or indirectly calls. Refactor it to become an argument of rpc_service() instead. An observation of all callers of rpc_service() shows that the preamble is always set, so we no longer need the "if (preamble)" check. Signed-off-by: Jonathan Tan --- remote-curl.c | 13 ++++--------- 1 file changed, 4 insertions(+), 9 deletions(-) diff --git a/remote-curl.c b/remote-curl.c index 8b7baf6298..d33d5bbfa8 100644 --- a/remote-curl.c +++ b/remote-curl.c @@ -505,7 +505,6 @@ static void output_refs(struct ref *refs) struct rpc_state { const char *service_name; - struct strbuf *stdin_preamble; char *service_url; char *hdr_content_type; char *hdr_accept; @@ -829,11 +828,10 @@ static int post_rpc(struct rpc_state *rpc) } static int rpc_service(struct rpc_state *rpc, struct discovery *heads, - const char **client_argv) + const char **client_argv, const struct strbuf *preamble) { const char *svc = rpc->service_name; struct strbuf buf = STRBUF_INIT; - struct strbuf *preamble = rpc->stdin_preamble; struct child_process client = CHILD_PROCESS_INIT; int err = 0; @@ -843,8 +841,7 @@ static int rpc_service(struct rpc_state *rpc, struct discovery *heads, client.argv = client_argv; if (start_command(&client)) exit(1); - if (preamble) - write_or_die(client.in, preamble->buf, preamble->len); + write_or_die(client.in, preamble->buf, preamble->len); if (heads) write_or_die(client.in, heads->buf, heads->len); @@ -978,10 +975,9 @@ static int fetch_git(struct discovery *heads, memset(&rpc, 0, sizeof(rpc)); rpc.service_name = "git-upload-pack", - rpc.stdin_preamble = &preamble; rpc.gzip_request = 1; - err = rpc_service(&rpc, heads, args.argv); + err = rpc_service(&rpc, heads, args.argv, &preamble); if (rpc.result.len) write_or_die(1, rpc.result.buf, rpc.result.len); strbuf_release(&rpc.result); @@ -1111,9 +1107,8 @@ static int push_git(struct discovery *heads, int nr_spec, char **specs) memset(&rpc, 0, sizeof(rpc)); rpc.service_name = "git-receive-pack", - rpc.stdin_preamble = &preamble; - err = rpc_service(&rpc, heads, args.argv); + err = rpc_service(&rpc, heads, args.argv, &preamble); if (rpc.result.len) write_or_die(1, rpc.result.buf, rpc.result.len); strbuf_release(&rpc.result); From patchwork Thu Feb 21 20:24:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Tan X-Patchwork-Id: 10824735 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7F6B11399 for ; Thu, 21 Feb 2019 20:24:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7017331C18 for ; Thu, 21 Feb 2019 20:24:55 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 64B5C31C5F; Thu, 21 Feb 2019 20:24:55 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F342331C18 for ; Thu, 21 Feb 2019 20:24:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726470AbfBUUYx (ORCPT ); Thu, 21 Feb 2019 15:24:53 -0500 Received: from mail-qt1-f201.google.com ([209.85.160.201]:48645 "EHLO mail-qt1-f201.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726460AbfBUUYx (ORCPT ); Thu, 21 Feb 2019 15:24:53 -0500 Received: by mail-qt1-f201.google.com with SMTP id d49so2052050qtd.15 for ; Thu, 21 Feb 2019 12:24:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=gzKXV/Ezcy/d680BCNoPVAbD431nng+K/jc23V8OOig=; b=aaIySczIhxKCTw2UMkhs6pwRy/e+o+Ts/WS29hjsSOUDo8UJGWuRk6P8V1Zc2OVNSk dv6k81/8zyrEx5lGph9ftk3WISrxPjkpHc9izAk5zQjYyyz4eizk3ooaweoODwgokQg0 YKVyYAMaWlNdZ9qIxlD5xIAUsvEDSk9KW0EQpGKXbOOcM7kpiJYu6zU99HPtFtmq1OXH l6MY9k/I7+5NANA6T3GSUgFEwaYl3rTwW2SPt061uh+hQOMLlUgd1cQpl2vl0HOJ0Jcr uxyGbunpJppw6IBGCr35vF1Z+s/Wi5836JXp39QXnqtB8HT8SopQJGXbwfyVKgvZA994 OuGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=gzKXV/Ezcy/d680BCNoPVAbD431nng+K/jc23V8OOig=; b=fSSfYcksI5/zc3e+5Zow3gA1nzc0SQVOzAEKGbNI+u7u8axGoXP7CZWHGSwbL7Hb0d FwO+ZCAjQoC5x25RMjDhGdOKy/tLHQkQVZADV7/0qjCMZeATN2fbXlz36qhItfa3214h NM8qbGqAOFaMWYL9nQUcSgGtazx5Y3RukruTL7nziRAQ2ebKaiJzQj9wPcV3qs3hbmi6 QEhoT0FeeoofZ2S9R7x74sgY9jjWfxoZ+xWzepQtL2/gXyZ/qWQXL0Irs7M303ziRZod +7gm0jNA7rVv39qXb7aQjqTE24qMalXyh7PgdPzi4lM92LHfVzP0DHorCnYVsN7ar2R0 v91w== X-Gm-Message-State: AHQUAua1Fg1NKl+hosSNjUWVjGKniz56uTGmkoxpicuuu0f0Gcy3OjEV b+nxljyae4q5u2mDhnCCg5Gt6RqObhs5Yr8TsE+ys8DWUE3k6PD2+LKwoSStHn65hU2TFuhFtuO PtGxOkz3rrvVUE/XqnlT6OPXIF6SIRp4GZJmp0jPNQi9gRDMF0vTaHOOKQqa3EblaoXoxD8a8XV Tc X-Google-Smtp-Source: AHgI3IZEBmoijaqrEjee63H5jh3XLy7f1zLl48tBSq5HcaEbgSGs089E/PQs7AMSv7gt+eanDnte2EhsqH5VGE5UT+OS X-Received: by 2002:ac8:32aa:: with SMTP id z39mr229995qta.35.1550780692243; Thu, 21 Feb 2019 12:24:52 -0800 (PST) Date: Thu, 21 Feb 2019 12:24:39 -0800 In-Reply-To: Message-Id: <19ccc0324803f07f3ceee2f47a998779fe656c0c.1550780213.git.jonathantanmy@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.19.0.271.gfe8321ec05.dirty Subject: [PATCH v2 3/5] remote-curl: reduce scope of rpc_state.result From: Jonathan Tan To: git@vger.kernel.org Cc: Jonathan Tan , peff@peff.net Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The result field in struct rpc_state is only used in rpc_service(), and not in any functions it directly or indirectly calls. Refactor it to become an argument of rpc_service() instead. Signed-off-by: Jonathan Tan --- remote-curl.c | 25 +++++++++++++------------ 1 file changed, 13 insertions(+), 12 deletions(-) diff --git a/remote-curl.c b/remote-curl.c index d33d5bbfa8..8e0e37ed3d 100644 --- a/remote-curl.c +++ b/remote-curl.c @@ -516,7 +516,6 @@ struct rpc_state { int in; int out; int any_written; - struct strbuf result; unsigned gzip_request : 1; unsigned initial_buffer : 1; }; @@ -828,7 +827,8 @@ static int post_rpc(struct rpc_state *rpc) } static int rpc_service(struct rpc_state *rpc, struct discovery *heads, - const char **client_argv, const struct strbuf *preamble) + const char **client_argv, const struct strbuf *preamble, + struct strbuf *rpc_result) { const char *svc = rpc->service_name; struct strbuf buf = STRBUF_INIT; @@ -849,7 +849,6 @@ static int rpc_service(struct rpc_state *rpc, struct discovery *heads, rpc->buf = xmalloc(rpc->alloc); rpc->in = client.in; rpc->out = client.out; - strbuf_init(&rpc->result, 0); strbuf_addf(&buf, "%s%s", url.buf, svc); rpc->service_url = strbuf_detach(&buf, NULL); @@ -877,7 +876,7 @@ static int rpc_service(struct rpc_state *rpc, struct discovery *heads, close(client.in); client.in = -1; if (!err) { - strbuf_read(&rpc->result, client.out, 0); + strbuf_read(rpc_result, client.out, 0); } else { char buf[4096]; for (;;) @@ -930,6 +929,7 @@ static int fetch_git(struct discovery *heads, struct strbuf preamble = STRBUF_INIT; int i, err; struct argv_array args = ARGV_ARRAY_INIT; + struct strbuf rpc_result = STRBUF_INIT; argv_array_pushl(&args, "fetch-pack", "--stateless-rpc", "--stdin", "--lock-pack", NULL); @@ -977,10 +977,10 @@ static int fetch_git(struct discovery *heads, rpc.service_name = "git-upload-pack", rpc.gzip_request = 1; - err = rpc_service(&rpc, heads, args.argv, &preamble); - if (rpc.result.len) - write_or_die(1, rpc.result.buf, rpc.result.len); - strbuf_release(&rpc.result); + err = rpc_service(&rpc, heads, args.argv, &preamble, &rpc_result); + if (rpc_result.len) + write_or_die(1, rpc_result.buf, rpc_result.len); + strbuf_release(&rpc_result); strbuf_release(&preamble); argv_array_clear(&args); return err; @@ -1075,6 +1075,7 @@ static int push_git(struct discovery *heads, int nr_spec, char **specs) struct argv_array args; struct string_list_item *cas_option; struct strbuf preamble = STRBUF_INIT; + struct strbuf rpc_result = STRBUF_INIT; argv_array_init(&args); argv_array_pushl(&args, "send-pack", "--stateless-rpc", "--helper-status", @@ -1108,10 +1109,10 @@ static int push_git(struct discovery *heads, int nr_spec, char **specs) memset(&rpc, 0, sizeof(rpc)); rpc.service_name = "git-receive-pack", - err = rpc_service(&rpc, heads, args.argv, &preamble); - if (rpc.result.len) - write_or_die(1, rpc.result.buf, rpc.result.len); - strbuf_release(&rpc.result); + err = rpc_service(&rpc, heads, args.argv, &preamble, &rpc_result); + if (rpc_result.len) + write_or_die(1, rpc_result.buf, rpc_result.len); + strbuf_release(&rpc_result); strbuf_release(&preamble); argv_array_clear(&args); return err; From patchwork Thu Feb 21 20:24:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Tan X-Patchwork-Id: 10824737 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3424C1399 for ; Thu, 21 Feb 2019 20:24:58 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 255CE31C18 for ; Thu, 21 Feb 2019 20:24:58 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1986431C5F; Thu, 21 Feb 2019 20:24:58 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B5F2031C18 for ; Thu, 21 Feb 2019 20:24:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726481AbfBUUY4 (ORCPT ); Thu, 21 Feb 2019 15:24:56 -0500 Received: from mail-qk1-f202.google.com ([209.85.222.202]:38746 "EHLO mail-qk1-f202.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726074AbfBUUY4 (ORCPT ); Thu, 21 Feb 2019 15:24:56 -0500 Received: by mail-qk1-f202.google.com with SMTP id x63so6270080qka.5 for ; Thu, 21 Feb 2019 12:24:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=iwufZRvJNcjOpGS1qOmJlSkS9PhHEcvAOHrAj4YqHX4=; b=cvtpzhUl7mdSDMN5gRqq5IhXIsV6rhdPRiVeznhpSMVTxKdsTl9aMUh4X9qtEjXUkJ T5Gu4xF2d/i2XtsHCnKuIdV/wpRqYPDx6H7RZYjcXsM1TfSwsgWoFQ4tyxZelkYdJq+M iyWtpiM4jz49DY/qmCt1ecAwxH1W1dDbpBBvfLfmw49snYu6HnJU+LFxfCQhlnOBECL1 pHCBEpCsc0otMDoGnqV7eyhbsJbw9BP6Ouws4HYSEedQzobWkkSzkOkN1iFinG1hGtYW /G5UDAe+3ue+8OnQsysJ+X3U/MnvXS9uKPBarUYj6BfO2BUs2A0nUEivoiBhIye9dnZd wjYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=iwufZRvJNcjOpGS1qOmJlSkS9PhHEcvAOHrAj4YqHX4=; b=a8OltT2IJ+9hlPkonektffH4L3pSGUspDdPLE/7IVWJZOI08ls3SKg5Th5OwlUUe7B s8viEvFn2KWje6KcVUK1RzmXJMEllv8HEQXutKu+bpFBzwKFJ+8YDttWPUo+jfqvdzaY gm8LIOwzmNz3FagMmkRMByHXm/Snb4xoUPWInqCMJ4l2c7wBnxX7jrGkNZXPpNRWyFMp ogw7AdiMa8Ou+zG7/Ga6LWT18b/rEYn//hhiJQMoPCbOzjCpVP6c6orCkAHfBpLq9zZ/ JK8lzhjyOUAncaxTYmbrRDlhUYuosoJA0cv0JRQvlOavrbwqYX+U0eY2+/UvrqJSwCEb nrQw== X-Gm-Message-State: AHQUAuZ5f7dJGHSzwrQTF7LIAHNZuwlOE1VzgdPuVU/MNmrLmlaOUvEB b5MG9V02Psw31LX12CsR2HTc6cTOlMbthYQSK7OpueNl6yj/s0/oXD5mgZC6Hy69QVgV4t7AWYB GmpvRaU966ccS11WaSg121KajiRl395CLml2CEXpC1Zj0ykNBL7M8JgaigJWNFqycjVfDAIs6DC qE X-Google-Smtp-Source: AHgI3IbHHSUt+Tv32nep0IrkiUeUdO7kvqUtieYV2vhQM2QZVoADL1iHGP4/FTty0NCQf8EariF1MrUm5bXsIDklLM0N X-Received: by 2002:ae9:f217:: with SMTP id m23mr237823qkg.5.1550780694895; Thu, 21 Feb 2019 12:24:54 -0800 (PST) Date: Thu, 21 Feb 2019 12:24:40 -0800 In-Reply-To: Message-Id: <2ecf63458740c631f0a1eb2acdfc877d7d06c278.1550780213.git.jonathantanmy@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.19.0.271.gfe8321ec05.dirty Subject: [PATCH v2 4/5] remote-curl: refactor reading into rpc_state's buf From: Jonathan Tan To: git@vger.kernel.org Cc: Jonathan Tan , peff@peff.net Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Currently, whenever remote-curl reads pkt-lines from its response file descriptor, only the payload is written to its buf, not the 4 characters denoting the length. A future patch will require the ability to also write those 4 characters, so in preparation for that, refactor this read into its own function. Signed-off-by: Jonathan Tan --- remote-curl.c | 33 ++++++++++++++++++++++++--------- 1 file changed, 24 insertions(+), 9 deletions(-) diff --git a/remote-curl.c b/remote-curl.c index 8e0e37ed3d..1f0161475d 100644 --- a/remote-curl.c +++ b/remote-curl.c @@ -520,6 +520,25 @@ struct rpc_state { unsigned initial_buffer : 1; }; +/* + * Appends the result of reading from rpc->out to the string represented by + * rpc->buf and rpc->len if there is enough space. Returns 1 if there was + * enough space, 0 otherwise. + * + * Writes the number of bytes appended into appended. + */ +static int rpc_read_from_out(struct rpc_state *rpc, size_t *appended) { + size_t left = rpc->alloc - rpc->len; + char *buf = rpc->buf + rpc->len; + + if (left < LARGE_PACKET_MAX) + return 0; + + *appended = packet_read(rpc->out, NULL, NULL, buf, left, 0); + rpc->len += *appended; + return 1; +} + static size_t rpc_out(void *ptr, size_t eltsize, size_t nmemb, void *buffer_) { @@ -529,11 +548,12 @@ static size_t rpc_out(void *ptr, size_t eltsize, if (!avail) { rpc->initial_buffer = 0; - avail = packet_read(rpc->out, NULL, NULL, rpc->buf, rpc->alloc, 0); + rpc->len = 0; + if (!rpc_read_from_out(rpc, &avail)) + BUG("The entire rpc->buf should be larger than LARGE_PACKET_DATA_MAX"); if (!avail) return 0; rpc->pos = 0; - rpc->len = avail; } if (max < avail) @@ -677,20 +697,15 @@ static int post_rpc(struct rpc_state *rpc) * chunked encoding mess. */ while (1) { - size_t left = rpc->alloc - rpc->len; - char *buf = rpc->buf + rpc->len; - int n; + size_t n; - if (left < LARGE_PACKET_MAX) { + if (!rpc_read_from_out(rpc, &n)) { large_request = 1; use_gzip = 0; break; } - - n = packet_read(rpc->out, NULL, NULL, buf, left, 0); if (!n) break; - rpc->len += n; } if (large_request) { From patchwork Thu Feb 21 20:24:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Tan X-Patchwork-Id: 10824739 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 40C586C2 for ; Thu, 21 Feb 2019 20:25:01 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3062031C8C for ; Thu, 21 Feb 2019 20:25:01 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2419331C5F; Thu, 21 Feb 2019 20:25:01 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0CC5931C18 for ; Thu, 21 Feb 2019 20:25:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726498AbfBUUY6 (ORCPT ); Thu, 21 Feb 2019 15:24:58 -0500 Received: from mail-ot1-f74.google.com ([209.85.210.74]:43162 "EHLO mail-ot1-f74.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726074AbfBUUY6 (ORCPT ); Thu, 21 Feb 2019 15:24:58 -0500 Received: by mail-ot1-f74.google.com with SMTP id r13so23723775otn.10 for ; Thu, 21 Feb 2019 12:24:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=KAFwh+uYqj0XK7UzkOCw/0x76Tdwuc2KG1k1F1snq0o=; b=DS794IG4KyO73hmha/UB5tQGd6PdBHweLc6C6zLaZ+OgKilwrW3CRPKypRNjuXjkeF BSoGv3VvYkftQ+FhDuumjQaXdctgZyjBVSFj8d25F8aG1MHsl9pUaav0w+UKeuXMq0aT KMnL/zelxoAUn3BGUH3bdKjbrKMpASaSd2DXfdRZ/kUcB6uRhJFdkoSaeoKGQA3WOV9T gwVqi2ojtMgtJERA94PxXKstEv2+P4XK94+sMl1ErXukOCef5kGrRADPSh4ywUECXXdo TfZjDozIGrKO1dRQmfkGBp3RLfouMaMX0oqXOG/LVYRGVeYAwgJ5oxjXb54ziaeC6EOq wNWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=KAFwh+uYqj0XK7UzkOCw/0x76Tdwuc2KG1k1F1snq0o=; b=L0wqiwqpNxD5x/HgPOceSJ8GpkH8XaLVwZxLTSJaeq5eQwsQinhch7p8jA9eaiMwDh oIMQffwxKp+3YyBHqGOLDKMHzNibweoTKsuxE036qzub/e9ltBkRjqLBfOFOboMEtC0F cvptDqE/b/6qqJ4jJhvKn8iZ4s2Iw1l8p1kR4NdO3TpZx8eUWcf36clJwF/Y6CwdjG7m 0m9YQ6RyHy2d4pjJLxnG4axiJr52vDPziTVdCTRrGMtTOV9a2GdeVw0yY07lJSOiEqeg x47wXMRhmL1CNwbUPr4qsl51BKmxg29c8xEXNOK6lQiA/7gWem9k0G+5kTyQAW1GyDRb 7w4w== X-Gm-Message-State: AHQUAuY00dt/vW7NoVG1JY6pSZNQldjUUT5lfisp5orNy5vW4XxRMJrz mIt8Vb6rcDBE03yGFveSos+41yF2W6P5pzompaXtSYeCOLptEj4L5nAHvQxY2cSUbR1RwOMpwG+ Qig6TQSgKjO90pXTeq+Z4LqiLgXx1sLrj8E242RGkmY55NsQdLqSvSLZnN4QprR6tyNalVqnau0 3D X-Google-Smtp-Source: AHgI3IaNHixjDWtTWSmTp9+e95QAQcbkQYAXpY+Sezn0R9y93fYMkuxAg4XXjWPWlloL419mjhHgpZ++KIH1sc1j8ikw X-Received: by 2002:a9d:5f17:: with SMTP id f23mr229921oti.31.1550780697347; Thu, 21 Feb 2019 12:24:57 -0800 (PST) Date: Thu, 21 Feb 2019 12:24:41 -0800 In-Reply-To: Message-Id: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.19.0.271.gfe8321ec05.dirty Subject: [PATCH v2 5/5] remote-curl: use post_rpc() for protocol v2 also From: Jonathan Tan To: git@vger.kernel.org Cc: Jonathan Tan , peff@peff.net Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP When transmitting and receiving POSTs for protocol v0 and v1, remote-curl uses post_rpc() (and associated functions), but when doing the same for protocol v2, it uses a separate set of functions (proxy_rpc() and others). Besides duplication of code, this has caused at least one bug: the auth retry mechanism that was implemented in v0/v1 was not implemented in v2. To fix this issue and avoid it in the future, make remote-curl also use post_rpc() when handling protocol v2. Because line lengths are written to the HTTP request in protocol v2 (unlike in protocol v0/v1), this necessitates changes in post_rpc() and some of the functions it uses; perform these changes too. A test has been included to ensure that the code for both the unchunked and chunked variants of the HTTP request is exercised. Note: stateless_connect() has been updated to use the lower-level packet reading functions instead of struct packet_reader. The low-level control is necessary here because we cannot change the destination buffer of struct packet_reader while it is being used; struct packet_buffer has a peeking mechanism which relies on the destination buffer being present in between a peek and a read. Signed-off-by: Jonathan Tan --- pkt-line.c | 2 +- pkt-line.h | 1 + remote-curl.c | 309 ++++++++++++++++++----------------------- t/t5702-protocol-v2.sh | 33 ++++- 4 files changed, 168 insertions(+), 177 deletions(-) diff --git a/pkt-line.c b/pkt-line.c index d4b71d3e82..60329b301b 100644 --- a/pkt-line.c +++ b/pkt-line.c @@ -117,7 +117,7 @@ void packet_buf_delim(struct strbuf *buf) strbuf_add(buf, "0001", 4); } -static void set_packet_header(char *buf, const int size) +void set_packet_header(char *buf, const int size) { static char hexchar[] = "0123456789abcdef"; diff --git a/pkt-line.h b/pkt-line.h index ad9a4a2cd7..c36cb788ed 100644 --- a/pkt-line.h +++ b/pkt-line.h @@ -25,6 +25,7 @@ void packet_delim(int fd); void packet_write_fmt(int fd, const char *fmt, ...) __attribute__((format (printf, 2, 3))); void packet_buf_flush(struct strbuf *buf); void packet_buf_delim(struct strbuf *buf); +void set_packet_header(char *buf, int size); void packet_write(int fd_out, const char *buf, size_t size); void packet_buf_write(struct strbuf *buf, const char *fmt, ...) __attribute__((format (printf, 2, 3))); void packet_buf_write_len(struct strbuf *buf, const char *data, size_t len); diff --git a/remote-curl.c b/remote-curl.c index 1f0161475d..8c03c78fc6 100644 --- a/remote-curl.c +++ b/remote-curl.c @@ -518,6 +518,21 @@ struct rpc_state { int any_written; unsigned gzip_request : 1; unsigned initial_buffer : 1; + + /* + * Whenever a pkt-line is read into buf, append the 4 characters + * denoting its length before appending the payload. + */ + unsigned write_line_lengths : 1; + + /* + * Used by rpc_out; initialize to 0. This is true if a flush has been + * read, but the corresponding line length (if write_line_lengths is + * true) and EOF have not been sent to libcurl. Since each flush marks + * the end of a request, each flush must be completely sent before any + * further reading occurs. + */ + unsigned flush_read_but_not_sent : 1; }; /* @@ -525,17 +540,54 @@ struct rpc_state { * rpc->buf and rpc->len if there is enough space. Returns 1 if there was * enough space, 0 otherwise. * - * Writes the number of bytes appended into appended. + * If rpc->write_line_lengths is true, appends the line length as a 4-byte + * hexadecimal string before appending the result described above. + * + * Writes the total number of bytes appended into appended. */ -static int rpc_read_from_out(struct rpc_state *rpc, size_t *appended) { - size_t left = rpc->alloc - rpc->len; - char *buf = rpc->buf + rpc->len; +static int rpc_read_from_out(struct rpc_state *rpc, int options, + size_t *appended, + enum packet_read_status *status) { + size_t left; + char *buf; + int pktlen_raw; + + if (rpc->write_line_lengths) { + left = rpc->alloc - rpc->len - 4; + buf = rpc->buf + rpc->len + 4; + } else { + left = rpc->alloc - rpc->len; + buf = rpc->buf + rpc->len; + } if (left < LARGE_PACKET_MAX) return 0; - *appended = packet_read(rpc->out, NULL, NULL, buf, left, 0); - rpc->len += *appended; + *status = packet_read_with_status(rpc->out, NULL, NULL, buf, + left, &pktlen_raw, options); + if (*status != PACKET_READ_EOF) { + *appended = pktlen_raw + (rpc->write_line_lengths ? 4 : 0); + rpc->len += *appended; + } + + if (rpc->write_line_lengths) { + switch (*status) { + case PACKET_READ_EOF: + if (!(options & PACKET_READ_GENTLE_ON_EOF)) + die("shouldn't have EOF when not gentle on EOF"); + break; + case PACKET_READ_NORMAL: + set_packet_header(buf - 4, *appended); + break; + case PACKET_READ_DELIM: + memcpy(buf - 4, "0001", 4); + break; + case PACKET_READ_FLUSH: + memcpy(buf - 4, "0000", 4); + break; + } + } + return 1; } @@ -545,15 +597,40 @@ static size_t rpc_out(void *ptr, size_t eltsize, size_t max = eltsize * nmemb; struct rpc_state *rpc = buffer_; size_t avail = rpc->len - rpc->pos; + enum packet_read_status status; if (!avail) { rpc->initial_buffer = 0; rpc->len = 0; - if (!rpc_read_from_out(rpc, &avail)) - BUG("The entire rpc->buf should be larger than LARGE_PACKET_DATA_MAX"); - if (!avail) - return 0; rpc->pos = 0; + if (!rpc->flush_read_but_not_sent) { + if (!rpc_read_from_out(rpc, 0, &avail, &status)) + BUG("The entire rpc->buf should be larger than LARGE_PACKET_MAX"); + if (status == PACKET_READ_FLUSH) + rpc->flush_read_but_not_sent = 1; + } + /* + * If flush_read_but_not_sent is true, we have already read one + * full request but have not fully sent it + EOF, which is why + * we need to refrain from reading. + */ + } + if (rpc->flush_read_but_not_sent) { + if (!avail) { + /* + * The line length either does not need to be sent at + * all or has already been completely sent. Now we can + * return 0, indicating EOF, meaning that the flush has + * been fully sent. + */ + rpc->flush_read_but_not_sent = 0; + return 0; + } + /* + * If avail is non-zerp, the line length for the flush still + * hasn't been fully sent. Proceed with sending the line + * length. + */ } if (max < avail) @@ -698,13 +775,14 @@ static int post_rpc(struct rpc_state *rpc) */ while (1) { size_t n; + enum packet_read_status status; - if (!rpc_read_from_out(rpc, &n)) { + if (!rpc_read_from_out(rpc, 0, &n, &status)) { large_request = 1; use_gzip = 0; break; } - if (!n) + if (status == PACKET_READ_FLUSH) break; } @@ -1179,165 +1257,11 @@ static void parse_push(struct strbuf *buf) free(specs); } -/* - * Used to represent the state of a connection to an HTTP server when - * communicating using git's wire-protocol version 2. - */ -struct proxy_state { - char *service_name; - char *service_url; - struct curl_slist *headers; - struct strbuf request_buffer; - int in; - int out; - struct packet_reader reader; - size_t pos; - int seen_flush; -}; - -static void proxy_state_init(struct proxy_state *p, const char *service_name, - enum protocol_version version) -{ - struct strbuf buf = STRBUF_INIT; - - memset(p, 0, sizeof(*p)); - p->service_name = xstrdup(service_name); - - p->in = 0; - p->out = 1; - strbuf_init(&p->request_buffer, 0); - - strbuf_addf(&buf, "%s%s", url.buf, p->service_name); - p->service_url = strbuf_detach(&buf, NULL); - - p->headers = http_copy_default_headers(); - - strbuf_addf(&buf, "Content-Type: application/x-%s-request", p->service_name); - p->headers = curl_slist_append(p->headers, buf.buf); - strbuf_reset(&buf); - - strbuf_addf(&buf, "Accept: application/x-%s-result", p->service_name); - p->headers = curl_slist_append(p->headers, buf.buf); - strbuf_reset(&buf); - - p->headers = curl_slist_append(p->headers, "Transfer-Encoding: chunked"); - - /* Add the Git-Protocol header */ - if (get_protocol_http_header(version, &buf)) - p->headers = curl_slist_append(p->headers, buf.buf); - - packet_reader_init(&p->reader, p->in, NULL, 0, - PACKET_READ_GENTLE_ON_EOF | - PACKET_READ_DIE_ON_ERR_PACKET); - - strbuf_release(&buf); -} - -static void proxy_state_clear(struct proxy_state *p) -{ - free(p->service_name); - free(p->service_url); - curl_slist_free_all(p->headers); - strbuf_release(&p->request_buffer); -} - -/* - * CURLOPT_READFUNCTION callback function. - * Attempts to copy over a single packet-line at a time into the - * curl provided buffer. - */ -static size_t proxy_in(char *buffer, size_t eltsize, - size_t nmemb, void *userdata) -{ - size_t max; - struct proxy_state *p = userdata; - size_t avail = p->request_buffer.len - p->pos; - - - if (eltsize != 1) - BUG("curl read callback called with size = %"PRIuMAX" != 1", - (uintmax_t)eltsize); - max = nmemb; - - if (!avail) { - if (p->seen_flush) { - p->seen_flush = 0; - return 0; - } - - strbuf_reset(&p->request_buffer); - switch (packet_reader_read(&p->reader)) { - case PACKET_READ_EOF: - die("unexpected EOF when reading from parent process"); - case PACKET_READ_NORMAL: - packet_buf_write_len(&p->request_buffer, p->reader.line, - p->reader.pktlen); - break; - case PACKET_READ_DELIM: - packet_buf_delim(&p->request_buffer); - break; - case PACKET_READ_FLUSH: - packet_buf_flush(&p->request_buffer); - p->seen_flush = 1; - break; - } - p->pos = 0; - avail = p->request_buffer.len; - } - - if (max < avail) - avail = max; - memcpy(buffer, p->request_buffer.buf + p->pos, avail); - p->pos += avail; - return avail; -} - -static size_t proxy_out(char *buffer, size_t eltsize, - size_t nmemb, void *userdata) -{ - size_t size; - struct proxy_state *p = userdata; - - if (eltsize != 1) - BUG("curl read callback called with size = %"PRIuMAX" != 1", - (uintmax_t)eltsize); - size = nmemb; - - write_or_die(p->out, buffer, size); - return size; -} - -/* Issues a request to the HTTP server configured in `p` */ -static int proxy_request(struct proxy_state *p) -{ - struct active_request_slot *slot; - - slot = get_active_slot(); - - curl_easy_setopt(slot->curl, CURLOPT_ENCODING, ""); - curl_easy_setopt(slot->curl, CURLOPT_NOBODY, 0); - curl_easy_setopt(slot->curl, CURLOPT_POST, 1); - curl_easy_setopt(slot->curl, CURLOPT_URL, p->service_url); - curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, p->headers); - - /* Setup function to read request from client */ - curl_easy_setopt(slot->curl, CURLOPT_READFUNCTION, proxy_in); - curl_easy_setopt(slot->curl, CURLOPT_READDATA, p); - - /* Setup function to write server response to client */ - curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, proxy_out); - curl_easy_setopt(slot->curl, CURLOPT_WRITEDATA, p); - - if (run_slot(slot, NULL) != HTTP_OK) - return -1; - - return 0; -} - static int stateless_connect(const char *service_name) { struct discovery *discover; - struct proxy_state p; + struct rpc_state rpc; + struct strbuf buf = STRBUF_INIT; /* * Run the info/refs request and see if the server supports protocol @@ -1357,23 +1281,58 @@ static int stateless_connect(const char *service_name) fflush(stdout); } - proxy_state_init(&p, service_name, discover->version); + rpc.service_name = service_name; + rpc.service_url = xstrfmt("%s%s", url.buf, rpc.service_name); + rpc.hdr_content_type = xstrfmt("Content-Type: application/x-%s-request", rpc.service_name); + rpc.hdr_accept = xstrfmt("Accept: application/x-%s-result", rpc.service_name); + if (get_protocol_http_header(discover->version, &buf)) { + rpc.protocol_header = strbuf_detach(&buf, NULL); + } else { + rpc.protocol_header = NULL; + strbuf_release(&buf); + } + rpc.buf = xmalloc(http_post_buffer); + rpc.alloc = http_post_buffer; + rpc.len = 0; + rpc.pos = 0; + rpc.in = 1; + rpc.out = 0; + rpc.any_written = 0; + rpc.gzip_request = 1; + rpc.initial_buffer = 0; + rpc.write_line_lengths = 1; + rpc.flush_read_but_not_sent = 0; /* * Dump the capability listing that we got from the server earlier * during the info/refs request. */ - write_or_die(p.out, discover->buf, discover->len); + write_or_die(rpc.in, discover->buf, discover->len); + + /* Until we see EOF keep sending POSTs */ + while (1) { + size_t avail; + enum packet_read_status status; - /* Peek the next packet line. Until we see EOF keep sending POSTs */ - while (packet_reader_peek(&p.reader) != PACKET_READ_EOF) { - if (proxy_request(&p)) { + if (!rpc_read_from_out(&rpc, PACKET_READ_GENTLE_ON_EOF, &avail, + &status)) + BUG("The entire rpc->buf should be larger than LARGE_PACKET_MAX"); + if (status == PACKET_READ_EOF) + break; + if (post_rpc(&rpc)) /* We would have an err here */ break; - } + /* Reset the buffer for next request */ + rpc.len = 0; } - proxy_state_clear(&p); + free(rpc.service_url); + free(rpc.hdr_content_type); + free(rpc.hdr_accept); + free(rpc.protocol_header); + free(rpc.buf); + strbuf_release(&buf); + return 0; } diff --git a/t/t5702-protocol-v2.sh b/t/t5702-protocol-v2.sh index db4ae09f2f..e112b6086c 100755 --- a/t/t5702-protocol-v2.sh +++ b/t/t5702-protocol-v2.sh @@ -542,7 +542,38 @@ test_expect_success 'clone with http:// using protocol v2' ' # Client requested to use protocol v2 grep "Git-Protocol: version=2" log && # Server responded using protocol v2 - grep "git< version 2" log + grep "git< version 2" log && + # Verify that the chunked encoding sending codepath is NOT exercised + ! grep "Send header: Transfer-Encoding: chunked" log +' + +test_expect_success 'clone big repository with http:// using protocol v2' ' + test_when_finished "rm -f log" && + + git init "$HTTPD_DOCUMENT_ROOT_PATH/big" && + # Ensure that the list of wants is greater than http.postbuffer below + for i in $(test_seq 1 1500) + do + # do not use here-doc, because it requires a process + # per loop iteration + echo "commit refs/heads/too-many-refs-$i" && + echo "committer git $i +0000" && + echo "data 0" && + echo "M 644 inline bla.txt" && + echo "data 4" && + echo "bla" + done | git -C "$HTTPD_DOCUMENT_ROOT_PATH/big" fast-import && + + GIT_TRACE_PACKET="$(pwd)/log" GIT_TRACE_CURL="$(pwd)/log" git \ + -c protocol.version=2 -c http.postbuffer=65536 \ + clone "$HTTPD_URL/smart/big" big_child && + + # Client requested to use protocol v2 + grep "Git-Protocol: version=2" log && + # Server responded using protocol v2 + grep "git< version 2" log && + # Verify that the chunked encoding sending codepath is exercised + grep "Send header: Transfer-Encoding: chunked" log ' test_expect_success 'fetch with http:// using protocol v2' '