From patchwork Thu Sep 27 01:24:52 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Steadmon X-Patchwork-Id: 10617103 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8C9CC913 for ; Thu, 27 Sep 2018 01:25:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7CC152B4EC for ; Thu, 27 Sep 2018 01:25:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 714082B4F1; Thu, 27 Sep 2018 01:25:37 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1105F2B4EC for ; Thu, 27 Sep 2018 01:25:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726817AbeI0HlS (ORCPT ); Thu, 27 Sep 2018 03:41:18 -0400 Received: from mail-vs1-f74.google.com ([209.85.217.74]:44348 "EHLO mail-vs1-f74.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726588AbeI0HlS (ORCPT ); Thu, 27 Sep 2018 03:41:18 -0400 Received: by mail-vs1-f74.google.com with SMTP id i25-v6so366008vso.11 for ; Wed, 26 Sep 2018 18:25:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=4M2AXJvgSC3+E19DesQWuQGaLX7D02c3LUmaQenKhy8=; b=LrHW1CX0RyVYVQbvwXT7sh5J76ebTChyRS3EKijFoWFRGYPg/D8Bcg60B/bj0dlb99 wQock1HN1LIDbqaj9OvipoaIn0A9gHyaXvPTisOPFWlHTOquy/4tQ3jIhFP1x6zPKIRi ptqpP4fawWN78eNKdqYreJu4VT67cHYMHWPW8qOZn+UqSskSOTf6ElI5tSi8fjrfq5Qr 7RJbQzJ/kjKVzbNpwchR7PTxYJHJTRgBDcYaspJjfjrgLHl3hnZg4umufciGmlBtpe+k /x9urfB09mTwMdYVG2b5fVgFRbVbr2z2OMxrhfXm9O0ym/rBqwEVfwBnu4xGxIhl1Dn5 wq+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=4M2AXJvgSC3+E19DesQWuQGaLX7D02c3LUmaQenKhy8=; b=QocVyoW3HatqN/uf8JdaBTRNFL0ogFOJKV5b7MzrnDF9kmR7n/7BHCzmeDahxiy0vj +WkYaKBO98CnN1is7vGkL6x6cueiVOWiTcoSS7VOcs+cCMGsgm0RkxJAnJgqc5MBqYnp BtFxfa5QOMq8qkdHHzBvfjeMQRXwcIFnyC9RSCbNUejab3m3uhkpIdxr74YV8uj52hkK ZwPcz+BzYH1joW08oTFscaesLER3BIE1TeDn9D36t91Hdgb13SeUM4mGzJEgXdPjK2br XL1hdLJmQ21zuNOjFttHNIvuoaxL6JlejZbln6HnnkGQ6UI/y4g2h09Y9h3pxH6TlvCF Gv6A== X-Gm-Message-State: ABuFfogMp+rFm5unuuZtV0Q56OuhOivQMmL7cFGh+C9QYd1LLvWI0aV4 VijUC9ub1WJcMeIAjhHJ/MfVKYlC6G5VOwlXPr7I/sczNYHRwGugoBBdFT7ErfhuVxxJXGkxCUD s/KOMyynE7dJwbCTt/t9wJl15WYDboSBCGvg3kKhQns+5HPdvSs9uo7ASSUZ0HVM= X-Google-Smtp-Source: ACcGV60mp5CxztkGDhBS3/2eWYSe2NRITW6QQdOxuAqPelmdXem3tCI2NQD92G7k2cIKPerndoSXGjivkv0H0A== X-Received: by 2002:a1f:29d7:: with SMTP id p206-v6mr3493853vkp.22.1538011534735; Wed, 26 Sep 2018 18:25:34 -0700 (PDT) Date: Wed, 26 Sep 2018 18:24:52 -0700 In-Reply-To: <20180927012455.234876-1-steadmon@google.com> Message-Id: <20180927012455.234876-2-steadmon@google.com> Mime-Version: 1.0 References: <20180912053519.31085-1-steadmon@google.com> <20180927012455.234876-1-steadmon@google.com> X-Mailer: git-send-email 2.19.0.605.g01d371f741-goog Subject: [PATCH v2 1/4] archive: follow test standards around assertions From: Josh Steadmon To: git@vger.kernel.org Cc: Josh Steadmon Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Move assertions outside of the check_tar function so that all top-level code is wrapped in a test_expect_* assertion. Signed-off-by: Josh Steadmon --- t/t5000-tar-tree.sh | 26 ++++++++++---------------- 1 file changed, 10 insertions(+), 16 deletions(-) diff --git a/t/t5000-tar-tree.sh b/t/t5000-tar-tree.sh index 2a97b27b0a..c408e3a23d 100755 --- a/t/t5000-tar-tree.sh +++ b/t/t5000-tar-tree.sh @@ -62,11 +62,9 @@ check_tar() { dir=$1 dir_with_prefix=$dir/$2 - test_expect_success ' extract tar archive' ' - (mkdir $dir && cd $dir && "$TAR" xf -) <$tarfile - ' + (mkdir $dir && cd $dir && "$TAR" xf -) <$tarfile && - test_expect_success TAR_NEEDS_PAX_FALLBACK ' interpret pax headers' ' + if test_have_prereq TAR_NEEDS_PAX_FALLBACK ; then ( cd $dir && for header in *.paxheader @@ -82,16 +80,11 @@ check_tar() { fi done ) - ' + fi && - test_expect_success ' validate filenames' ' - (cd ${dir_with_prefix}a && find .) | sort >$listfile && - test_cmp a.lst $listfile - ' - - test_expect_success ' validate file contents' ' - diff -r a ${dir_with_prefix}a - ' + (cd ${dir_with_prefix}a && find .) | sort >$listfile && + test_cmp a.lst $listfile && + diff -r a ${dir_with_prefix}a } test_expect_success \ @@ -143,19 +136,20 @@ test_expect_success \ 'git archive' \ 'git archive HEAD >b.tar' -check_tar b +test_expect_success 'extract archive' 'check_tar b' test_expect_success 'git archive --prefix=prefix/' ' git archive --prefix=prefix/ HEAD >with_prefix.tar ' -check_tar with_prefix prefix/ +test_expect_success 'extract with prefix' 'check_tar with_prefix prefix/' test_expect_success 'git-archive --prefix=olde-' ' git archive --prefix=olde- HEAD >with_olde-prefix.tar ' -check_tar with_olde-prefix olde- +test_expect_success 'extract with olde- prefix' \ + 'check_tar with_olde-prefix olde-' test_expect_success 'git archive on large files' ' test_config core.bigfilethreshold 1 && From patchwork Thu Sep 27 01:24:53 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Steadmon X-Patchwork-Id: 10617105 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 015CD913 for ; Thu, 27 Sep 2018 01:25:40 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E2D372B4EB for ; Thu, 27 Sep 2018 01:25:39 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D47362B4F1; Thu, 27 Sep 2018 01:25:39 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 79B912B4EC for ; Thu, 27 Sep 2018 01:25:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726924AbeI0HlU (ORCPT ); Thu, 27 Sep 2018 03:41:20 -0400 Received: from mail-ot1-f73.google.com ([209.85.210.73]:45573 "EHLO mail-ot1-f73.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726588AbeI0HlU (ORCPT ); Thu, 27 Sep 2018 03:41:20 -0400 Received: by mail-ot1-f73.google.com with SMTP id c46-v6so1278351otd.12 for ; Wed, 26 Sep 2018 18:25:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=G5qgRp129EspzskcFzeY0PL7JE8nJck0Ql5xq8kxcbU=; b=SDOL5JQ/TO/YbPlXHMG8cYAZ/iXnEoVTFkmcp8Pr+7qHZ+s3y5E0An8jVFxW4dzKLD 9iJlkBMsQXku76lZy6U0ScVEVwdZ/Q1YpxQEOGP8i8m9p7h26BAU7bQouNZK085+LS2B ohFRx8e9PFmSIfCXanJ8t40hIxwpZKYqC+2soKCow4TYI7VCuGkdCQgNFqJdOzrYPKrr cb4wFhqcuNPSEYq3GI35hLEgee33LntRKZOZLI7QDhheLhStiGf3UhaAkkU+ZOEU1Ise NfIu2DV7yNSqCSmCQ1eRm5TEmOppaAks/q1MLl5fHc4MJvfw0iSrHbVZRJBGVo8JaB3e ugvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=G5qgRp129EspzskcFzeY0PL7JE8nJck0Ql5xq8kxcbU=; b=OK3Za3fYgj9uRiHhvY3HGgPKh0oDvsd12K2GJUG0APmbxNCcuGosBprX0+UL/Yen6U wKZ+8F7GpGKXk2VQu7VdnWjvfNWiKcU3zy/yDeGA0nusYPaah/GFRpZ9PPAqNgl8bYXa bPWZpG0Zi9nmz9jBkMuctj5EYLUJyUsyu97Q71GbSlfMp3I1WaCck85oFOs7BvD5ZaSR EV6PimV6gxf9AL9D3LnJPhE6Smxi9u1owoZdTENXuXB3WTUD2MuagXmBINMRftOpeUpZ j65+0gJZNxn44KdVxTV1EbUPB1p7IbDpDqoc5/wjA1OR29Readw/8QMrbKI7Lj/6GAa/ WAfQ== X-Gm-Message-State: ABuFfogN2fjJLq3QlFu8qAcwXvTyVZTNoYLMcG8wIjHN954+T+y3VaGe lVMkxyBL46LvPrbJ1RwVjpujPfshm5jXe+j1oU6BrbtXCYIYsO44zbuoYtG3rbpfSRanXLPFmiV ymZ5cOI5uCn4xTlC+AW2KHcJVGnm6hWau2zeridyXeFHPAUp7uzlNqjQmLzN3EZk= X-Google-Smtp-Source: ACcGV61wfQs/f/gtqIz0XMyCADtRkaSNdhlt0KosC8HJvdbXWED9GrSgwpu4vjstUWcXFZSq17y++rPQs9NHEg== X-Received: by 2002:a9d:4b17:: with SMTP id q23-v6mr1875368otf.82.1538011537608; Wed, 26 Sep 2018 18:25:37 -0700 (PDT) Date: Wed, 26 Sep 2018 18:24:53 -0700 In-Reply-To: <20180927012455.234876-1-steadmon@google.com> Message-Id: <20180927012455.234876-3-steadmon@google.com> Mime-Version: 1.0 References: <20180912053519.31085-1-steadmon@google.com> <20180927012455.234876-1-steadmon@google.com> X-Mailer: git-send-email 2.19.0.605.g01d371f741-goog Subject: [PATCH v2 2/4] archive: use packet_reader for communications From: Josh Steadmon To: git@vger.kernel.org Cc: Josh Steadmon Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Using packet_reader will simplify version detection and capability handling, which will make implementation of protocol v2 support in git-archive easier. This refactoring does not change the behavior of "git archive". Signed-off-by: Josh Steadmon Reviewed-by: Stefan Beller --- builtin/archive.c | 23 ++++++++++++++--------- 1 file changed, 14 insertions(+), 9 deletions(-) diff --git a/builtin/archive.c b/builtin/archive.c index e74f675390..4eb547c5b7 100644 --- a/builtin/archive.c +++ b/builtin/archive.c @@ -27,10 +27,11 @@ static int run_remote_archiver(int argc, const char **argv, const char *remote, const char *exec, const char *name_hint) { - char *buf; int fd[2], i, rv; struct transport *transport; struct remote *_remote; + struct packet_reader reader; + enum packet_read_status status; _remote = remote_get(remote); if (!_remote->url[0]) @@ -38,6 +39,8 @@ static int run_remote_archiver(int argc, const char **argv, transport = transport_get(_remote, _remote->url[0]); transport_connect(transport, "git-upload-archive", exec, fd); + packet_reader_init(&reader, fd[0], NULL, 0, PACKET_READ_CHOMP_NEWLINE); + /* * Inject a fake --format field at the beginning of the * arguments, with the format inferred from our output @@ -53,18 +56,20 @@ static int run_remote_archiver(int argc, const char **argv, packet_write_fmt(fd[1], "argument %s\n", argv[i]); packet_flush(fd[1]); - buf = packet_read_line(fd[0], NULL); - if (!buf) + status = packet_reader_read(&reader); + + if (status != PACKET_READ_NORMAL || reader.pktlen <= 0) die(_("git archive: expected ACK/NAK, got a flush packet")); - if (strcmp(buf, "ACK")) { - if (starts_with(buf, "NACK ")) - die(_("git archive: NACK %s"), buf + 5); - if (starts_with(buf, "ERR ")) - die(_("remote error: %s"), buf + 4); + if (strcmp(reader.line, "ACK")) { + if (starts_with(reader.line, "NACK ")) + die(_("git archive: NACK %s"), reader.line + 5); + if (starts_with(reader.line, "ERR ")) + die(_("remote error: %s"), reader.line + 4); die(_("git archive: protocol error")); } - if (packet_read_line(fd[0], NULL)) + status = packet_reader_read(&reader); + if (status == PACKET_READ_NORMAL && reader.pktlen > 0) die(_("git archive: expected a flush")); /* Now, start reading from fd[0] and spit it out to stdout */ From patchwork Thu Sep 27 01:24:54 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Steadmon X-Patchwork-Id: 10617107 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2204D13A4 for ; Thu, 27 Sep 2018 01:25:44 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 115762B4EB for ; Thu, 27 Sep 2018 01:25:44 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 05FD22B4ED; Thu, 27 Sep 2018 01:25:44 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 327962B4EB for ; Thu, 27 Sep 2018 01:25:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726944AbeI0HlY (ORCPT ); Thu, 27 Sep 2018 03:41:24 -0400 Received: from mail-it1-f201.google.com ([209.85.166.201]:51695 "EHLO mail-it1-f201.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726588AbeI0HlY (ORCPT ); Thu, 27 Sep 2018 03:41:24 -0400 Received: by mail-it1-f201.google.com with SMTP id 204-v6so6129008itf.1 for ; Wed, 26 Sep 2018 18:25:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=7BMbW1ZLeHMHpsj/3KNM+xvA8r87bDDmqKwRvA27CGE=; b=cYaXwmeils33kkvnSQzzKLXwTRB07UCazig3Qg3BZwz/38vpXCuH+oLmX6g9OQBacP dmlaGAfYHUsXFxF8zKdeYjBeNXOV8EVTKD+XlQ/svcJkMPWbWaPfHSiTP8mWkj4U5FKm g9DlVctBBjm3Gx2Cxm8djKRAgICFnOTbm8ZRxvhr8LY6U0wkO8Sw/wvmSUE7Hm/WRtEY 10iHA0EPynKR8xWpZxnmhD+xsbmR8h4r9+AsOlAcfT/9us5Ow1hrdSt0HzImpENw5dOm ySNJdHqE7Z3rc0Rfu1YYksNrFV/Gif3OaOO0embfdSeZxvUJIcaG6Wz6VqWENEKnKL4F ebQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=7BMbW1ZLeHMHpsj/3KNM+xvA8r87bDDmqKwRvA27CGE=; b=WgflSyYvTNZPcazmYQP/wq/0sJz0ea3jyH+V4sxH4I843xkgSiO4nxwTrOzDV2o2cF 3mMH5v1n5CuRNJBqyJSL8w4wfvgAHGHy9ryf956hOG36bU4zwJTW4+kKqLW2Xpx6TPYy NttkbOg/OMj0MXS4wKOPBlYhweb3jgwqADAjg+iGBAy/vGsBLfwSDScpj8GRHa6Y8qFV kvNwLIZVYub6jWT/gghzHCOKNlQ4ez0SJMzpsn0fB8FmwMIqaDDYZIJDInmh+9B6wq7B DNCJnkHRAzzTYcB537WPn+jIX5E0czf+7rlvFmDdiWW6qwzJG9rViGwKUNMdz6u+3BAn QpOA== X-Gm-Message-State: ABuFfoi7jzIXa5PXZKrRmBVtlqlxOUOyTGiYzE3zAttFsJaUs2W3y4zC ewGEAy0TcWDa8cf6qRS0W+bYb1QJJqM4i8fkIRg6qGGErH/X6Ek3g8kCHxTQOP3HkotPH2LIDyK gLBLBBrg5erMECBNf67d35Zb6favgmblysst0SRuVsE5VDB8YpdDmlUvb+qLLii4= X-Google-Smtp-Source: ACcGV63xUvLNZFS45G3MHO+pivJ+nOrGFgDNr9ltmCmrPH2iGwiyXuQc3wYpJPpNF8/2Sdx3WU3ktJA6mRH9TA== X-Received: by 2002:a6b:ca84:: with SMTP id a126-v6mr5335329iog.45.1538011540588; Wed, 26 Sep 2018 18:25:40 -0700 (PDT) Date: Wed, 26 Sep 2018 18:24:54 -0700 In-Reply-To: <20180927012455.234876-1-steadmon@google.com> Message-Id: <20180927012455.234876-4-steadmon@google.com> Mime-Version: 1.0 References: <20180912053519.31085-1-steadmon@google.com> <20180927012455.234876-1-steadmon@google.com> X-Mailer: git-send-email 2.19.0.605.g01d371f741-goog Subject: [PATCH v2 3/4] archive: implement protocol v2 archive command From: Josh Steadmon To: git@vger.kernel.org Cc: Josh Steadmon Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This adds a new archive command for protocol v2. The command expects arguments in the form "argument X" which are passed unmodified to git-upload-archive--writer. This command works over the file://, Git, and SSH transports. HTTP support will be added in a separate patch. NEEDSWORK: this is not backwards-compatible with older clients that set GIT_PROTOCOL=version=2 or configure protocol.version=2. Signed-off-by: Josh Steadmon --- Documentation/technical/protocol-v2.txt | 21 +++++++++++- builtin/archive.c | 45 +++++++++++++++++-------- builtin/upload-archive.c | 27 +++++++++++++-- serve.c | 7 ++++ t/t5000-tar-tree.sh | 7 ++++ t/t5701-git-serve.sh | 1 + 6 files changed, 90 insertions(+), 18 deletions(-) diff --git a/Documentation/technical/protocol-v2.txt b/Documentation/technical/protocol-v2.txt index 09e4e0273f..6126fc5738 100644 --- a/Documentation/technical/protocol-v2.txt +++ b/Documentation/technical/protocol-v2.txt @@ -77,7 +77,8 @@ A v2 server would reply: S: Subsequent requests are then made directly to the service -`$GIT_URL/git-upload-pack`. (This works the same for git-receive-pack). +`$GIT_URL/git-upload-pack`. (This works the same for git-receive-pack or +git-upload-archive). Capability Advertisement -------------------------- @@ -168,6 +169,24 @@ printable ASCII characters except space (i.e., the byte range 32 < x < and debugging purposes, and MUST NOT be used to programmatically assume the presence or absence of particular features. + archive +~~~~~~~~~ + +`archive` is the command to request an archive (such as a tarball) from a server +over v2. + +archive takes the following optional arguments: + + argument + This specifies that should be passed as an argument to the + underlying archive writer. This option can be repeated to pass + additional arguments to the archive writer. + +See linkgit:git-archive[1] for allowed values of ``. + +The output of archive is a packet-line stream of the resulting archive, with +error messages transferred over the sideband channel. + ls-refs ~~~~~~~~~ diff --git a/builtin/archive.c b/builtin/archive.c index 4eb547c5b7..f91d222677 100644 --- a/builtin/archive.c +++ b/builtin/archive.c @@ -5,9 +5,11 @@ #include "cache.h" #include "builtin.h" #include "archive.h" +#include "connect.h" #include "transport.h" #include "parse-options.h" #include "pkt-line.h" +#include "protocol.h" #include "sideband.h" static void create_output_file(const char *output_file) @@ -23,6 +25,13 @@ static void create_output_file(const char *output_file) } } +static void do_v2_command_and_cap(int out) +{ + packet_write_fmt(out, "command=archive\n"); + /* Capability list would go here, if we wanted to request any. */ + packet_delim(out); +} + static int run_remote_archiver(int argc, const char **argv, const char *remote, const char *exec, const char *name_hint) @@ -32,6 +41,7 @@ static int run_remote_archiver(int argc, const char **argv, struct remote *_remote; struct packet_reader reader; enum packet_read_status status; + enum protocol_version version; _remote = remote_get(remote); if (!_remote->url[0]) @@ -41,6 +51,11 @@ static int run_remote_archiver(int argc, const char **argv, packet_reader_init(&reader, fd[0], NULL, 0, PACKET_READ_CHOMP_NEWLINE); + version = discover_version(&reader); + + if (version == protocol_v2 && server_supports_v2("archive", 0)) + do_v2_command_and_cap(fd[1]); + /* * Inject a fake --format field at the beginning of the * arguments, with the format inferred from our output @@ -56,22 +71,24 @@ static int run_remote_archiver(int argc, const char **argv, packet_write_fmt(fd[1], "argument %s\n", argv[i]); packet_flush(fd[1]); - status = packet_reader_read(&reader); - - if (status != PACKET_READ_NORMAL || reader.pktlen <= 0) - die(_("git archive: expected ACK/NAK, got a flush packet")); - if (strcmp(reader.line, "ACK")) { - if (starts_with(reader.line, "NACK ")) - die(_("git archive: NACK %s"), reader.line + 5); - if (starts_with(reader.line, "ERR ")) - die(_("remote error: %s"), reader.line + 4); - die(_("git archive: protocol error")); + if (version == protocol_v0) { + status = packet_reader_read(&reader); + + if (status != PACKET_READ_NORMAL || reader.pktlen <= 0) + die(_("git archive: expected ACK/NAK, got a flush packet")); + if (strcmp(reader.line, "ACK")) { + if (starts_with(reader.line, "NACK ")) + die(_("git archive: NACK %s"), reader.line + 5); + if (starts_with(reader.line, "ERR ")) + die(_("remote error: %s"), reader.line + 4); + die(_("git archive: protocol error")); + } + + status = packet_reader_read(&reader); + if (status == PACKET_READ_NORMAL && reader.pktlen > 0) + die(_("git archive: expected a flush")); } - status = packet_reader_read(&reader); - if (status == PACKET_READ_NORMAL && reader.pktlen > 0) - die(_("git archive: expected a flush")); - /* Now, start reading from fd[0] and spit it out to stdout */ rv = recv_sideband("archive", fd[0], 1); rv |= transport_disconnect(transport); diff --git a/builtin/upload-archive.c b/builtin/upload-archive.c index 25d9116356..b8a8fb256f 100644 --- a/builtin/upload-archive.c +++ b/builtin/upload-archive.c @@ -5,6 +5,8 @@ #include "builtin.h" #include "archive.h" #include "pkt-line.h" +#include "protocol.h" +#include "serve.h" #include "sideband.h" #include "run-command.h" #include "argv-array.h" @@ -76,10 +78,24 @@ static ssize_t process_input(int child_fd, int band) int cmd_upload_archive(int argc, const char **argv, const char *prefix) { struct child_process writer = { argv }; + enum protocol_version version = determine_protocol_version_server(); if (argc == 2 && !strcmp(argv[1], "-h")) usage(upload_archive_usage); + if (version == protocol_v2) { + struct serve_options options = SERVE_OPTIONS_INIT; + + /* Send "version 2" message and capabilities. */ + options.advertise_capabilities = 1; + serve(&options); + + /* Process command request and validate client capabilities. */ + options.advertise_capabilities = 0; + options.stateless_rpc = 1; + serve(&options); + } + /* * Set up sideband subprocess. * @@ -92,12 +108,17 @@ int cmd_upload_archive(int argc, const char **argv, const char *prefix) writer.git_cmd = 1; if (start_command(&writer)) { int err = errno; - packet_write_fmt(1, "NACK unable to spawn subprocess\n"); + if (version == protocol_v0 || version == protocol_v1) + packet_write_fmt(1, "NACK unable to spawn subprocess\n"); + else if (version == protocol_v2) + error_clnt("unable to spawn subprocess\n"); die("upload-archive: %s", strerror(err)); } - packet_write_fmt(1, "ACK\n"); - packet_flush(1); + if (version == protocol_v0 || version == protocol_v1) { + packet_write_fmt(1, "ACK\n"); + packet_flush(1); + } while (1) { struct pollfd pfd[2]; diff --git a/serve.c b/serve.c index bda085f09c..cef575086f 100644 --- a/serve.c +++ b/serve.c @@ -52,8 +52,15 @@ struct protocol_capability { struct packet_reader *request); }; +static int noop(struct repository *r, struct argv_array *keys, + struct packet_reader *request) +{ + return 0; +} + static struct protocol_capability capabilities[] = { { "agent", agent_advertise, NULL }, + { "archive", always_advertise, noop }, { "ls-refs", always_advertise, ls_refs }, { "fetch", upload_pack_advertise, upload_pack_v2 }, { "server-option", always_advertise, NULL }, diff --git a/t/t5000-tar-tree.sh b/t/t5000-tar-tree.sh index c408e3a23d..4a76354021 100755 --- a/t/t5000-tar-tree.sh +++ b/t/t5000-tar-tree.sh @@ -138,6 +138,13 @@ test_expect_success \ test_expect_success 'extract archive' 'check_tar b' +test_expect_success 'protocol v2 for remote' ' + GIT_PROTOCOL="version=2" GIT_TRACE_PACKET="$(pwd)/log" git archive \ + --remote=. HEAD >v2_remote.tar && + grep "version 2" log +' +test_expect_success 'extract v2 archive' 'check_tar v2_remote' + test_expect_success 'git archive --prefix=prefix/' ' git archive --prefix=prefix/ HEAD >with_prefix.tar ' diff --git a/t/t5701-git-serve.sh b/t/t5701-git-serve.sh index 75ec79e6cb..f47a14660b 100755 --- a/t/t5701-git-serve.sh +++ b/t/t5701-git-serve.sh @@ -8,6 +8,7 @@ test_expect_success 'test capability advertisement' ' cat >expect <<-EOF && version 2 agent=git/$(git version | cut -d" " -f3) + archive ls-refs fetch=shallow server-option From patchwork Thu Sep 27 01:24:55 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Steadmon X-Patchwork-Id: 10617109 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E161613A4 for ; Thu, 27 Sep 2018 01:25:46 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CC1DE2B4EB for ; Thu, 27 Sep 2018 01:25:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BDC222B4ED; Thu, 27 Sep 2018 01:25:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4CFC62B4EB for ; Thu, 27 Sep 2018 01:25:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726948AbeI0Hl1 (ORCPT ); Thu, 27 Sep 2018 03:41:27 -0400 Received: from mail-vk1-f202.google.com ([209.85.221.202]:49079 "EHLO mail-vk1-f202.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726588AbeI0Hl1 (ORCPT ); Thu, 27 Sep 2018 03:41:27 -0400 Received: by mail-vk1-f202.google.com with SMTP id g71-v6so322172vke.15 for ; Wed, 26 Sep 2018 18:25:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=IJjbq8DDjp/TM3ofFJtHDEPDb7EyViyLhbWH6raeeEo=; b=vaXUPCMh+tb5A2xps1QTGNU7K27OmQT4uAyDFoikBDaFfLCI4jCJhLuvqheE1NahOU 4KFdr5P4qkbnOvVpj/ssudvmzq26Od0PxhQAt4x/l74vDmHDgcFYQOfSHY126E1Ri3wL 6E49kUo0Ou3/3+MN/yOMJInlqeL22kc0RuUkKsUBZP+j2yoOE8DOmrll9WZyoFXAT6Zl Jwc/oDa9ZbUmUo0Ox3O7oKJOwJ//His/e/BOveK07EOQzA0INtWpZbwyKH4CBEYJ1S5K HMhJXFw1aL4lckKM7Suum22eqdO72KcunzToyEviTStQ9rlu3fjpwrRtqQ1hbfHQeJ5f 7t3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=IJjbq8DDjp/TM3ofFJtHDEPDb7EyViyLhbWH6raeeEo=; b=aRyxImT6xA4DtR+iCG1ZbUzmgltsHYw+RAkAuKm/AeFzGPwenVHbJrYlXS2RZpqIlN 3hjFUQdxUihiSYOishTEb5GvHq351VWuCtPqzHOi4jD6WD99IuM6Fx8bEYxWFyb+mjDH ZbTPNnnHzb/WVj/63tROAI+PFrPJq9nsGIrPGSaT0RlGwv0W0LPUmerq49/zH3NSU9pI leJDsBK6o0yYmkClOT4WqaN3oqP/WHZRiaVODXnLSg/YbshVTOIAqUx+88OY+eaNCAQB XOs8R2b/+ofTZRr6Xy1R8VAN3EIy61c/V8GcQ3+Plrz4mrylDdab1IBQcqewsnlGCEy7 IjHw== X-Gm-Message-State: ABuFfoj6ZpJuxWmoihMDeMkg+ru3h8gES3b+xOqTSVXvJPRwewKaTTRY e4eLpERkiS79fip6O09GvZgyE6H2x10LqSIUewq1MIF6hDdRW4YbPx6tb+s/PJod63UxUYzksUp n39P5KOBqHOR5tznq8MYLmGViRVRsLsKdjvq/vdLpU68kQBkiIfrJmc6vSMavjew= X-Google-Smtp-Source: ACcGV60Oei/DcTxXRdKH3dcPziS6gzS11UBxwHqzucQK+6QHCysuH3oo2+1zcE4Z1uM8lBtC8XI7KzYyWGmbSA== X-Received: by 2002:a1f:b915:: with SMTP id j21-v6mr3597538vkf.13.1538011543767; Wed, 26 Sep 2018 18:25:43 -0700 (PDT) Date: Wed, 26 Sep 2018 18:24:55 -0700 In-Reply-To: <20180927012455.234876-1-steadmon@google.com> Message-Id: <20180927012455.234876-5-steadmon@google.com> Mime-Version: 1.0 References: <20180912053519.31085-1-steadmon@google.com> <20180927012455.234876-1-steadmon@google.com> X-Mailer: git-send-email 2.19.0.605.g01d371f741-goog Subject: [PATCH v2 4/4] archive: allow archive over HTTP(S) with proto v2 From: Josh Steadmon To: git@vger.kernel.org Cc: Josh Steadmon Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Signed-off-by: Josh Steadmon --- builtin/archive.c | 12 +++++++++++- http-backend.c | 13 ++++++++++++- transport-helper.c | 7 ++++--- 3 files changed, 27 insertions(+), 5 deletions(-) diff --git a/builtin/archive.c b/builtin/archive.c index f91d222677..78a259518d 100644 --- a/builtin/archive.c +++ b/builtin/archive.c @@ -87,7 +87,17 @@ static int run_remote_archiver(int argc, const char **argv, status = packet_reader_read(&reader); if (status == PACKET_READ_NORMAL && reader.pktlen > 0) die(_("git archive: expected a flush")); - } + } else if (version == protocol_v2 && + (starts_with(transport->url, "http://") || + starts_with(transport->url, "https://"))) + /* + * Commands over HTTP require two requests, so there's an + * additional server response to parse. We do only basic sanity + * checking here that the versions presented match across + * requests. + */ + if (version != discover_version(&reader)) + die(_("git archive: received different protocol versions in subsequent requests")); /* Now, start reading from fd[0] and spit it out to stdout */ rv = recv_sideband("archive", fd[0], 1); diff --git a/http-backend.c b/http-backend.c index 9e894f197f..8e262d50e0 100644 --- a/http-backend.c +++ b/http-backend.c @@ -32,6 +32,7 @@ struct rpc_service { static struct rpc_service rpc_service[] = { { "upload-pack", "uploadpack", 1, 1 }, { "receive-pack", "receivepack", 0, -1 }, + { "upload-archive", "uploadarchive", 1, 1 }, }; static struct string_list *get_parameters(void) @@ -637,6 +638,15 @@ static void service_rpc(struct strbuf *hdr, char *service_name) struct rpc_service *svc = select_service(hdr, service_name); struct strbuf buf = STRBUF_INIT; + if (!strcmp(service_name, "git-upload-archive")) { + /* + * git-upload-archive doesn't need --stateless-rpc, because it + * always handles only a single request. + */ + argv[1] = "."; + argv[2] = NULL; + } + strbuf_reset(&buf); strbuf_addf(&buf, "application/x-git-%s-request", svc->name); check_content_type(hdr, buf.buf); @@ -713,7 +723,8 @@ static struct service_cmd { {"GET", "/objects/pack/pack-[0-9a-f]{40}\\.idx$", get_idx_file}, {"POST", "/git-upload-pack$", service_rpc}, - {"POST", "/git-receive-pack$", service_rpc} + {"POST", "/git-receive-pack$", service_rpc}, + {"POST", "/git-upload-archive$", service_rpc}, }; static int bad_request(struct strbuf *hdr, const struct service_cmd *c) diff --git a/transport-helper.c b/transport-helper.c index 143ca008c8..c41c3dfcff 100644 --- a/transport-helper.c +++ b/transport-helper.c @@ -604,8 +604,9 @@ static int process_connect_service(struct transport *transport, strbuf_addf(&cmdbuf, "connect %s\n", name); ret = run_connect(transport, &cmdbuf); } else if (data->stateless_connect && - (get_protocol_version_config() == protocol_v2) && - !strcmp("git-upload-pack", name)) { + get_protocol_version_config() == protocol_v2 && + (!strcmp("git-upload-pack", name) || + !strcmp("git-upload-archive", name))) { strbuf_addf(&cmdbuf, "stateless-connect %s\n", name); ret = run_connect(transport, &cmdbuf); if (ret) @@ -639,7 +640,7 @@ static int connect_helper(struct transport *transport, const char *name, /* Get_helper so connect is inited. */ get_helper(transport); - if (!data->connect) + if (!data->connect && !data->stateless_connect) die(_("operation not supported by protocol")); if (!process_connect_service(transport, name, exec))