From patchwork Wed Jun 15 16:58:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ren=C3=A9_Scharfe?= X-Patchwork-Id: 12882773 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7B2F1C433EF for ; Wed, 15 Jun 2022 16:59:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1349950AbiFOQ7U (ORCPT ); Wed, 15 Jun 2022 12:59:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33902 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1349729AbiFOQ7T (ORCPT ); Wed, 15 Jun 2022 12:59:19 -0400 Received: from mout.web.de (mout.web.de [212.227.15.3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4F0342DD7C for ; Wed, 15 Jun 2022 09:59:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=web.de; s=dbaedf251592; t=1655312321; bh=xVyn0C+GKzuVSsXqWWA3P9r89IIV36g9KYMrsH2j2bc=; h=X-UI-Sender-Class:Date:Subject:From:To:Cc:References:In-Reply-To; b=PriwAmP84S14yzh5MpnjH9JlTWY11gtBQIvh84eaSp9S4aOmhZLQdrYYJBP/IrCoL opSaXMu4RkFnxF3Kr/oC26Z6T50wjI8ORQ3z2mYzUHYcofmPLr1Mo82f71QNqaCjyi kHcMkjn5XYFS1HpYmZWke4HZsaXB8oW9jQ7DrvQY= X-UI-Sender-Class: c548c8c5-30a9-4db5-a2e7-cb6cb037b8f9 Received: from [192.168.178.29] ([79.203.31.99]) by smtp.web.de (mrweb006 [213.165.67.108]) with ESMTPSA (Nemesis) id 1N5UkQ-1nd1ER224A-0173oY; Wed, 15 Jun 2022 18:58:41 +0200 Message-ID: <56205881-ccb1-165b-889e-3aa641117981@web.de> Date: Wed, 15 Jun 2022 18:58:38 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:91.0) Gecko/20100101 Thunderbird/91.10.0 Subject: [PATCH v4 1/6] archive: update format documentation Content-Language: en-US From: =?utf-8?q?Ren=C3=A9_Scharfe?= To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin , Rohit Ashiwal , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= , Jeff King , "brian m . carlson" References: <9df761c3-355a-ede9-7971-b32687fe9abb@web.de> In-Reply-To: <9df761c3-355a-ede9-7971-b32687fe9abb@web.de> X-Provags-ID: V03:K1:mNrqFMzXn/7JH/BpKxTN9KCqTsI7jlIDNPoaAKjexnffhAMAHem PAsi9XXLlQ15zu6z9FXVsJB6gi7TjGHS2BYKOW6D416uL9M4lbgpE8xOk1m+9svIVS/4u3Y j8331ClMzYtHSmqJYPFpcVChAwd4hgGgeEywgu4Ug5uTN59iH4vV98h0kYrXixLoNQpiidm LJp6eBFJiAJSykPjcheeA== X-UI-Out-Filterresults: notjunk:1;V03:K0:CVLKGiOPzjc=:rvwSk4p7p37GWSKfHIdlIT e9gsrl0sL5fQ5hLhCNCduh4wvECRD7J09w9i3TEEFRoDTGt9ecMunDRPFB//jysPA2H+inbSl RxXhhfY0ixxJokJMskrSQsNNtxLhjER+t3hL9Mkurwb9uupyT6zRTCDlhnR75leWwmcoA8uSO fzn2l9y0u5d6uQCZ5md8XEoGmf1sZT09ToANJR+3StdYyWESAune4f41T/vMGzxeYPy0sWuZY jg6nEHW6jRkwQkGyV0lZSUtJI8zoqXQLnL8heeHCxZ6qNZDE5YECUDH1h/c7USlm4tq/W8cji LxzhBYB1iNk8G1WN+BuaWQZwFCeNFF4SUj3H3XcmHeUl4RQPdsXwa9prXbOwK/qHO/VlVPjzi M/SvV94uFIgjgpr7dUdh1XmqO4VI/iZynYi+4nN9p+D1RyOf1gtk0HlX+aTF8QN4JBVxqwdG4 ZxZTusOHZJ9Qx8Tt1MUvyCWMvuRHIGfHwdNwPCSddFvMzHxd0C1lFD9nlsX7G0bB/JPVeGlTl Ys/9Sg5o02gJ2tdqVgfa/Syvpp07RDI+cVpCVxVSvJrISgnsCBdkslQLKt+8WH51XORx2JIui Z1BI7R1jMj13RN7NVK509jD04QypSWC5Zlp7BTcbyXm4yWbKuoLKOljSaTwp1529yqHdrFcsb w8qYeNO0wgQ9rOEszJXrD+L17BKNNUE7OdyPQKJhD91w7mbuz5gSG8VK4fWa7awjkNQ1iPsMf UxmOmierQJ5MDXp1Ir7fleFpB/UffAJLx5Wzc4vS10SQzjKUCE6wq76kp0FD0Ru5Gdxx/pJrG rqdOpOL8aJa/x4Bb0KuHSxefsAFiUFuc1U7q1JKmE8Q4OnZLQV/HXBwGukEvpwaWEo057jSyc Mn6AvB5ArUFlZzcV6x9XaUjm6v5K93rNuvyGbvkiyiryLHOsahPJC5ZpRnJQgpIfdhwGN/XEd WEw8PYcIoT55OPsBHyFVU3uHWSq/HgLX+s3jMrOKYBI5fjhe307mGckD/9CkFyQCSpLyGL25k PONkIgJjWfo+4/2o6W8dcVacynMjufgUDBYsBTkt5lDKwxvx93HHEaCSnfHcZtpC8LAJi1edK AVb/AijcsgYaq7omvXpL35px1DCPwjagJnH Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Mention all formats in the --format section, use backtick quoting for literal values throughout, clarify the description of the configuration option. Helped-by: Junio C Hamano Signed-off-by: René Scharfe --- Documentation/git-archive.txt | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) -- 2.36.1 diff --git a/Documentation/git-archive.txt b/Documentation/git-archive.txt index 56989a2f34..ff3f7b0344 100644 --- a/Documentation/git-archive.txt +++ b/Documentation/git-archive.txt @@ -34,10 +34,12 @@ OPTIONS ------- --format=:: - Format of the resulting archive: 'tar' or 'zip'. If this option + Format of the resulting archive. Possible values are `tar`, + `zip`, `tar.gz`, `tgz`, and any format defined using the + configuration option `tar..command`. If `--format` is not given, and the output file is specified, the format is - inferred from the filename if possible (e.g. writing to "foo.zip" - makes the output to be in the zip format). Otherwise the output + inferred from the filename if possible (e.g. writing to `foo.zip` + makes the output to be in the `zip` format). Otherwise the output format is `tar`. -l:: @@ -143,17 +145,15 @@ tar..command:: is executed using the shell with the generated tar file on its standard input, and should produce the final output on its standard output. Any compression-level options will be passed - to the command (e.g., "-9"). An output file with the same - extension as `` will be use this format if no other - format is given. + to the command (e.g., `-9`). + -The "tar.gz" and "tgz" formats are defined automatically and default to -`gzip -cn`. You may override them with custom commands. +The `tar.gz` and `tgz` formats are defined automatically and use the +command `gzip -cn` by default. tar..remote:: - If true, enable `` for use by remote clients via + If true, enable the format for use by remote clients via linkgit:git-upload-archive[1]. Defaults to false for - user-defined formats, but true for the "tar.gz" and "tgz" + user-defined formats, but true for the `tar.gz` and `tgz` formats. [[ATTRIBUTES]] From patchwork Wed Jun 15 16:59:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ren=C3=A9_Scharfe?= X-Patchwork-Id: 12882774 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E8E7C433EF for ; Wed, 15 Jun 2022 17:00:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1356664AbiFORAz (ORCPT ); Wed, 15 Jun 2022 13:00:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34814 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1357189AbiFORAh (ORCPT ); Wed, 15 Jun 2022 13:00:37 -0400 Received: from mout.web.de (mout.web.de [212.227.15.3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B9C7A51E54 for ; Wed, 15 Jun 2022 10:00:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=web.de; s=dbaedf251592; t=1655312398; bh=l25bGK4F8qm1yqKuZ/f4WA2c4yj52roMCXYwFXn623Q=; h=X-UI-Sender-Class:Date:Subject:From:To:Cc:References:In-Reply-To; b=beLPnSaIO8plJGvjyQyfCKl6uX2Gi6IYIeLQolYQghUOhVren8hcL9F8jWrdfcI/G b0HncmATfrvDzf8N2HBggzWllmREq0CsCVE+vSErGGZrCZw0cFl84m0GLM/Xa3TJkx 29acX6aHU9sxTEVXSZi/XpJ6oUXIKGm5bPUMo+tI= X-UI-Sender-Class: c548c8c5-30a9-4db5-a2e7-cb6cb037b8f9 Received: from [192.168.178.29] ([79.203.31.99]) by smtp.web.de (mrweb005 [213.165.67.108]) with ESMTPSA (Nemesis) id 1MXXRL-1oDRpJ17xX-00Z4Gf; Wed, 15 Jun 2022 18:59:58 +0200 Message-ID: <00e5da9f-a3fb-8a5b-ad29-4016873b0e90@web.de> Date: Wed, 15 Jun 2022 18:59:57 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:91.0) Gecko/20100101 Thunderbird/91.10.0 Subject: [PATCH v4 2/6] archive: rename archiver data field to filter_command Content-Language: en-US From: =?utf-8?q?Ren=C3=A9_Scharfe?= To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin , Rohit Ashiwal , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= , Jeff King , "brian m . carlson" References: <9df761c3-355a-ede9-7971-b32687fe9abb@web.de> In-Reply-To: <9df761c3-355a-ede9-7971-b32687fe9abb@web.de> X-Provags-ID: V03:K1:gpvrPSwwkPwo5rcJKqESwROALrKP5s+hq+AXciaSZRaANunzLUh uN6NmXoldTL5NaImOZTLZUNTxSccGfQJXFTGNzk4IIgUhBY2wGqOxvAsHJls/JAjyOxvM2p ng5MnGdtB0Voeuw5Hy9f8fhwTU6KZ5smxyidSuAjXFKkNksSR0iYoYUgveK8Tjcew5NukT9 xAzAhdG/z1NnOlayEvA/Q== X-UI-Out-Filterresults: notjunk:1;V03:K0:6afyA4lyfJk=:rTy/YG5Cuy1fcJQ1mwDuxq TtyLIdZQLivVDn3MQ5zvAOp53fYlxyuHUrqISFSfwdeYdsPV0Ta8S+h6PewAPFFPvbWRkF1o0 nfr6MKUHMkxXL0Kll9HhvAEVZMMRIg34vRZmeGLwNePvROSkpV2WLIlDdSgOatkxXUKDqnG1y WHOc+YPHD7zPaDmTLCRyrgyNwTjikkgFn1P5/aglIk3+Y3xjThK2CSJtzp7E29Jv+KJLDb50Z Cy9FlrgBVixOOTdSYXTDV7EskYTghaQit/cmYlY6GsxIM4ZZe1mx3ba6IXxx9ZJZSPbGKxdXt zVtIdocnyuUtm8IyXkb8tgcNhy2aN02AqeHxm5bmraZ7My7HNR8gmOkHlyLPAFbRbNLe20/s3 zfFjBbXX9ue/BVoL39PoW3x8fR7Y0DwTrHc+JW+76MN8do4hDZ8HJZhomjM3pERSY3P7Vh34Q eIT3vUg96iZ6UYcbrceeGOTC2WvNRkc0ThhYN/8dHvp7arALN62omafl7ohHJK7FmfGgKvyYF mqNtHf+W/DZPNDPTCK6K7+osNbsWdizuwFvqSx9gihl0Q4oseK12ZakL4NSPhkvjCvLIkgfZ4 cQdlAiUv8Yeqm7cPgP4tY+/oQ3gOVdz1vcH1LpmsAy3fnsIfNWoFKqSWuHNmz+qUWMGiIRALM vp/o16K/bD5xbPw83D7GhEYcV8BqIJmG90iLazUg9AID1sg/FV6mWIyqNsrPptDIP+sqmsmoj n+EHBKF2o4GAomg3KKOHGVerMvNIB1Gcu73h0ZRBTCPQLQ1GS+8LlPkNfonp2mze2QhIgrzFn AWTYNJPDOEfIj1p0Qy6OXLWlqlQDvPfnnsfTgG79dasIpWzQBYoijHfIvvb92Dq7oLslU12q+ 5D90ycj6dHkD1+BWU+xTDMDOrVCLWo95gx/jDOerRQwat5sYNKjwtKATqKl0msy7BcWQFxn5Q G7rBNJxt6oCOTTmyLZJoc99SgJZ5eLvXqbK9ks72zKvzhfsctxe8gY8YHz7zv1XQbNWfnk4U3 zWMUpPH8obOPrjIBw9qSE+a+VHQPK396ngsUBygpKAipXisupaXma/K2ty+Ydg/GYkBUc08nR sbIbpqDBRbv7/zp7vZ+98TJ26nDqfdwewLd Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org The void pointer "data" in struct archiver is only used to store filter commands to pass tar archives to, like gzip. Rename it accordingly and also turn it into a char pointer to document the fact that it's a string reference. Signed-off-by: René Scharfe --- archive-tar.c | 10 +++++----- archive.h | 2 +- 2 files changed, 6 insertions(+), 6 deletions(-) -- 2.36.1 diff --git a/archive-tar.c b/archive-tar.c index 042feb66d2..2717e34a1d 100644 --- a/archive-tar.c +++ b/archive-tar.c @@ -383,8 +383,8 @@ static int tar_filter_config(const char *var, const char *value, void *data) if (!strcmp(type, "command")) { if (!value) return config_error_nonbool(var); - free(ar->data); - ar->data = xstrdup(value); + free(ar->filter_command); + ar->filter_command = xstrdup(value); return 0; } if (!strcmp(type, "remote")) { @@ -432,10 +432,10 @@ static int write_tar_filter_archive(const struct archiver *ar, struct child_process filter = CHILD_PROCESS_INIT; int r; - if (!ar->data) + if (!ar->filter_command) BUG("tar-filter archiver called with no filter defined"); - strbuf_addstr(&cmd, ar->data); + strbuf_addstr(&cmd, ar->filter_command); if (args->compression_level >= 0) strbuf_addf(&cmd, " -%d", args->compression_level); @@ -478,7 +478,7 @@ void init_tar_archiver(void) git_config(git_tar_config, NULL); for (i = 0; i < nr_tar_filters; i++) { /* omit any filters that never had a command configured */ - if (tar_filters[i]->data) + if (tar_filters[i]->filter_command) register_archiver(tar_filters[i]); } } diff --git a/archive.h b/archive.h index 49fab71aaf..08bed3ed3a 100644 --- a/archive.h +++ b/archive.h @@ -43,7 +43,7 @@ struct archiver { const char *name; int (*write_archive)(const struct archiver *, struct archiver_args *); unsigned flags; - void *data; + char *filter_command; }; void register_archiver(struct archiver *); From patchwork Wed Jun 15 17:01:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ren=C3=A9_Scharfe?= X-Patchwork-Id: 12882775 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6F315C43334 for ; Wed, 15 Jun 2022 17:01:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1357457AbiFORBz (ORCPT ); Wed, 15 Jun 2022 13:01:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34790 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1356915AbiFORBb (ORCPT ); Wed, 15 Jun 2022 13:01:31 -0400 Received: from mout.web.de (mout.web.de [212.227.15.4]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 97EA13E0C1 for ; Wed, 15 Jun 2022 10:01:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=web.de; s=dbaedf251592; t=1655312475; bh=qsdawq715O/V/nmmBycYUKaJbChEBTxQP9xdGa7cxHw=; h=X-UI-Sender-Class:Date:Subject:From:To:Cc:References:In-Reply-To; b=Ah5vpoSRdViGyVkK57rtyyto6LulQkKfHwYQoGSO3RdltpgvMJaj6NUo+TEmmG+yc 6jcY+BHhX5BQ08rL0s8XDocP/FYeydF/he2dJBZAXqGxzoWTHf0zIsByLJ9zRSl8fc 4gKwoTCdpWXnZPG/kz444j/ZQIYtyDA7c8Y11xOk= X-UI-Sender-Class: c548c8c5-30a9-4db5-a2e7-cb6cb037b8f9 Received: from [192.168.178.29] ([79.203.31.99]) by smtp.web.de (mrweb006 [213.165.67.108]) with ESMTPSA (Nemesis) id 1MnG2C-1nKnj52B6K-00jV7e; Wed, 15 Jun 2022 19:01:15 +0200 Message-ID: <2415cea0-47c4-b63f-7ef8-1d8f3e08d523@web.de> Date: Wed, 15 Jun 2022 19:01:14 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:91.0) Gecko/20100101 Thunderbird/91.10.0 Subject: [PATCH v4 3/6] archive-tar: factor out write_block() Content-Language: en-US From: =?utf-8?q?Ren=C3=A9_Scharfe?= To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin , Rohit Ashiwal , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= , Jeff King , "brian m . carlson" References: <9df761c3-355a-ede9-7971-b32687fe9abb@web.de> In-Reply-To: <9df761c3-355a-ede9-7971-b32687fe9abb@web.de> X-Provags-ID: V03:K1:lEUf/Pcvhh1U4rCfHA4RtfuJL8doxSNUguagOYNInY6gRLvNulM eM5l+IPeFwctnWgB4ntJ00uJ/arsD1Kwn5pU6yyKaSPh0r3jIIXCHcdA5U/yoVZchNgwwgX scJNQQ4lov+sRNXhc6Jz4QsttkBI+xLUDggMZkuNHtUCj0yNQOD2gRPjPlUgDR3KrsuvA4V weSgNJjudOs2AeEm5uG1g== X-UI-Out-Filterresults: notjunk:1;V03:K0:K0b2rW2P5J4=:0M6QdM1SjuaKxwBNESYhYQ uyNEcJNhcXq6YWocZzuBfB2cYvZOKiYdrR6AvUxvvpWpgOfmN/RB+bEOSK4dcK7cvh7p8+nfb mrxrvvGNbcH8ZtB7YN78xdwJxBpjcaTJ6a2lwF8cPREf6GJF+6dYXYIBv7aIVvzlDoZuFpAHI YNuxRDMuuwVlXKZ5lHMbNy+OcBukK0GeGfnR6/i9l8730pzFd3uRDz/NRf5eMoSYYoeolo2pH hAqunZjxnm4johPeK2KQrKupgHC9DOR53cU60D0IyyfU5ByiuuuSHmOqv0YPIIAGudfQI5Utj KKs3L8wvPHk67jkRewhLGfucZVHeSnmKD3qBq2IFWOiKywOd7e+CgfAfUSS6WpHYnIkNlpvRl KGuqHkqCFpolT+bqmFA0vGH5XpSMVYrMCT6msv9VJ/pGddhu4W2yIuvQF7L9WwFVDNXAT9Gp3 L0PGI9TdCH8vLyH0E95uwvnmfeqgulXH3WpayPa/Yp4aUld5+6XoEaOiCQYxACen7NFNxSiWf BA28Qx/5qg1BA0dZJcAzy2pIL112cI2K8At+WiLEVCF2xU6LQqODLjeb69srIqnnI2CUGGbrj G3/ZXHl23OJc9Rlfs2DGQ78icWEw73Dx7rSM8s9LWV2FdqaYnd0QHyZ5gRhS/2gUfGRh3FYBr /hK6l/nNkIfvvT8a5to+XMXcyJDwNwpCVmjalFVeugXCNmBidEagp0Bb4vb51pgt7YA09jsA2 ksvycLShO81aGe3o34Y9FaHVT9Wk41kr/wGxSV4cYBxLjOKa2m19FMyPbSeYEcyylvcTL62uS ehmx7FZk+c+V/boW3kJgmY2aOeNR3fs1vPfbB0tErKnENXYKz8DVA/Jz7ybvSc/4S6vRqpBE7 muRPVWA5Lwws/JwF7zoOkmDwV19hN8WfwzdGF9EazxFFCz/4HkYLhu88kg9DDyHgo42KI1bfc 0wepIBUgLNwTckd04556cid8R+u6fURvedqdvia1Cz8CKvIJdO4xFnVQFrF+Zz7Rv+hvGv4z6 1VLPfwMr8MbptakigSFb6hJG/jfSAHDu4yXiaCRO0QiIUWr+xnF7O3bnEKepRvTvKhfV08wkN CHS4UCrzUShqjQ1nIvclbcilf10JdkpUWaL Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org All tar archive writes have the same size and are done to the same file descriptor. Move them to a common function, write_block(), to reduce code duplication and make it easy to change the destination. Original-patch-by: Rohit Ashiwal Signed-off-by: René Scharfe --- archive-tar.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) -- 2.36.1 diff --git a/archive-tar.c b/archive-tar.c index 2717e34a1d..4e6a3deb80 100644 --- a/archive-tar.c +++ b/archive-tar.c @@ -38,11 +38,16 @@ static int write_tar_filter_archive(const struct archiver *ar, #define USTAR_MAX_MTIME 077777777777ULL #endif +static void write_block(const void *buf) +{ + write_or_die(1, buf, BLOCKSIZE); +} + /* writes out the whole block, but only if it is full */ static void write_if_needed(void) { if (offset == BLOCKSIZE) { - write_or_die(1, block, BLOCKSIZE); + write_block(block); offset = 0; } } @@ -66,7 +71,7 @@ static void do_write_blocked(const void *data, unsigned long size) write_if_needed(); } while (size >= BLOCKSIZE) { - write_or_die(1, buf, BLOCKSIZE); + write_block(buf); size -= BLOCKSIZE; buf += BLOCKSIZE; } @@ -101,10 +106,10 @@ static void write_trailer(void) { int tail = BLOCKSIZE - offset; memset(block + offset, 0, tail); - write_or_die(1, block, BLOCKSIZE); + write_block(block); if (tail < 2 * RECORDSIZE) { memset(block, 0, offset); - write_or_die(1, block, BLOCKSIZE); + write_block(block); } } From patchwork Wed Jun 15 17:02:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ren=C3=A9_Scharfe?= X-Patchwork-Id: 12882776 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 34E31C43334 for ; Wed, 15 Jun 2022 17:02:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1356915AbiFORCu (ORCPT ); Wed, 15 Jun 2022 13:02:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38498 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1357037AbiFORCs (ORCPT ); Wed, 15 Jun 2022 13:02:48 -0400 Received: from mout.web.de (mout.web.de [212.227.15.3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 14ACDFFC for ; Wed, 15 Jun 2022 10:02:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=web.de; s=dbaedf251592; t=1655312554; bh=ZrrhL/w710jhy2SG3C8tbBlUZiFDmG1XCiqrO1Tyg5k=; h=X-UI-Sender-Class:Date:Subject:From:To:Cc:References:In-Reply-To; b=GY06bNL8rFQdx8L7qPrUStBM33/sB9aGC+BX8pVn9tZnAFtjse/0cWvCvoVtT3H5y UdsWx6RMgWq9ytIXnmoapfZ3n0OVqhNGQokCwpliPFOYsXI5vcmGcz0WUF1mmXKL3C V+L0EgpAxqegTZmrDwrblwT9R7ZNOUkxeiMDYWNw= X-UI-Sender-Class: c548c8c5-30a9-4db5-a2e7-cb6cb037b8f9 Received: from [192.168.178.29] ([79.203.31.99]) by smtp.web.de (mrweb006 [213.165.67.108]) with ESMTPSA (Nemesis) id 1MyO0u-1nnQNK1TA4-00ymKR; Wed, 15 Jun 2022 19:02:34 +0200 Message-ID: <1328fe72-1a27-b214-c226-d239099be673@web.de> Date: Wed, 15 Jun 2022 19:02:33 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:91.0) Gecko/20100101 Thunderbird/91.10.0 Subject: [PATCH v4 4/6] archive-tar: add internal gzip implementation Content-Language: en-US From: =?utf-8?q?Ren=C3=A9_Scharfe?= To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin , Rohit Ashiwal , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= , Jeff King , "brian m . carlson" References: <9df761c3-355a-ede9-7971-b32687fe9abb@web.de> In-Reply-To: <9df761c3-355a-ede9-7971-b32687fe9abb@web.de> X-Provags-ID: V03:K1:RtljyghAEK2xia6rSMdhkJ+ZYqgZIInU+EXC/d0DdVQef4KUptW nXlaZhFOC2zeqaKeaufanSXN1lto3uW4CIvbmqo9VLJTSGDVfCN20Q0CGTrn11fYZnJvX5e oxqf+evlPMYco8DP0fgOlvwNzSMqplV2elwVid3bSO3OlZIeD+mUtd/Zfu8q9bE3q3VzlMe HAZobQk2cT6wnyNAKPBKQ== X-UI-Out-Filterresults: notjunk:1;V03:K0:m/t3iB4l0ak=:EgRAS6d/fx5LVYUUFkCVUe gdjyhlakz2L06yEH3z0fvWYqAKB3LFR2QkZjCdVytgIxBuR/5ynKPxiwo6aWKXZ4RTvQ+br11 00uk8UsG/6hUsLimv+2Hr/QTucrTKoyXV8nwwFH6Zg8tHjO/xqoKw4S4BzI9HUH5e4RaodsFc sU4zFljTeiNHXTFnxvL5UG0l7Jf5L321PcP3fGSK4nDAarwZ1dB5Emu3VhdNFqiOP27qQ15jd V+I+vqtZkDMf8mb0CGq0Ut62WtF/Rje5uKNu+FbrY4BSXH/AJjyPsi87GJKsPfBTCUCEkjybJ ZX/G0+x9MfLEfJHrxbxQtZzMKEtD7iViB+hECeAcHcTse+STSEcu+eVwYsVHV+UDJKFETSKgk BhfMBi39MOKd5g0notPMhg6zDgA/B6sRaYAxxK1R65vpjHVs1srpoVenS42oZPEQyW/ZuMNBw ZXDsB+P8ZSGAZhUSIO0woEpWHc61Mk3/A9WKZntxOAeMWrybETiX0zW2Tht91Ca6r+UMSlSUq A7vaotMDeu/MKMlLqi7VGminCjQmm8CXO76CuAB3HKmjE+uMP6VYO0spaIvemxd+TPb89ZGLT F+i3PCGKOD8EJa03dYxQl7Ofl6mtV2Z3CYFSrM9uxSh+P1SWABnCNOx2KUHW5E6iZlzeLkdIo RjY4QpUl5veaqcGVb+DHrrHjqz67MgBjdVXIjiefRJWFoFVUp7iX6kS/WnNzV0FbJzymNr+y8 97IEhLoJ73C6Z9wScR7SHc1OPJtGppH+mr5n9rsILRuJHL+JyDDmPTxiKlMV+VDvE1C1rCKJf pZ3kQnzCsNShZ17z+dMpcI0pnlAZWXeyInAUmfYo7/1zOX1u5Wwn81J1IUyi++OArBlYJfUmJ WaT3kZJMDZPsu8r7nBUnyGGejbQFlYsiywYsqbBlWO0SKQEf359f6DmQsZPvJ8Qb0qrSk+k0X 2ffNa2mAkkUaJr3cWvSxqhk7DMvH/8mJu8VtkkpCkaUlW1Pc9BcbLEKlLkoPyKJRsAa/++Tww 4fOtoqpQip5Hwtjqb1me5tuMMpnOfPNcU2U93LupPcR6AxDHjC4heC44W0d64aDswoNRAzZdY S5ntNj2bTX8a3Hu7IPPmkP+LehOFiFe8Vmq Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Git uses zlib for its own object store, but calls gzip when creating tgz archives. Add an option to perform the gzip compression for the latter using zlib, without depending on the external gzip binary. Plug it in by making write_block a function pointer and switching to a compressing variant if the filter command has the magic value "git archive gzip". Does that indirection slow down tar creation? Not really, at least not in this test: $ hyperfine -w3 -L rev HEAD,origin/main -p 'git checkout {rev} && make' \ './git -C ../linux archive --format=tar HEAD # {rev}' Benchmark #1: ./git -C ../linux archive --format=tar HEAD # HEAD Time (mean ± σ): 4.044 s ± 0.007 s [User: 3.901 s, System: 0.137 s] Range (min … max): 4.038 s … 4.059 s 10 runs Benchmark #2: ./git -C ../linux archive --format=tar HEAD # origin/main Time (mean ± σ): 4.047 s ± 0.009 s [User: 3.903 s, System: 0.138 s] Range (min … max): 4.038 s … 4.066 s 10 runs How does tgz creation perform? $ hyperfine -w3 -L command 'gzip -cn','git archive gzip' \ './git -c tar.tgz.command="{command}" -C ../linux archive --format=tgz HEAD' Benchmark #1: ./git -c tar.tgz.command="gzip -cn" -C ../linux archive --format=tgz HEAD Time (mean ± σ): 20.404 s ± 0.006 s [User: 23.943 s, System: 0.401 s] Range (min … max): 20.395 s … 20.414 s 10 runs Benchmark #2: ./git -c tar.tgz.command="git archive gzip" -C ../linux archive --format=tgz HEAD Time (mean ± σ): 23.807 s ± 0.023 s [User: 23.655 s, System: 0.145 s] Range (min … max): 23.782 s … 23.857 s 10 runs Summary './git -c tar.tgz.command="gzip -cn" -C ../linux archive --format=tgz HEAD' ran 1.17 ± 0.00 times faster than './git -c tar.tgz.command="git archive gzip" -C ../linux archive --format=tgz HEAD' So the internal implementation takes 17% longer on the Linux repo, but uses 2% less CPU time. That's because the external gzip can run in parallel on its own processor, while the internal one works sequentially and avoids the inter-process communication overhead. What are the benefits? Only an internal sequential implementation can offer this eco mode, and it allows avoiding the gzip(1) requirement. This implementation uses the helper functions from our zlib.c instead of the convenient gz* functions from zlib, because the latter doesn't give the control over the generated gzip header that the next patch requires. Original-patch-by: Rohit Ashiwal Signed-off-by: René Scharfe --- Documentation/git-archive.txt | 3 ++- archive-tar.c | 45 ++++++++++++++++++++++++++++++++++- t/t5000-tar-tree.sh | 16 +++++++++++++ 3 files changed, 62 insertions(+), 2 deletions(-) -- 2.36.1 diff --git a/Documentation/git-archive.txt b/Documentation/git-archive.txt index ff3f7b0344..b2d1b63d31 100644 --- a/Documentation/git-archive.txt +++ b/Documentation/git-archive.txt @@ -148,7 +148,8 @@ tar..command:: to the command (e.g., `-9`). + The `tar.gz` and `tgz` formats are defined automatically and use the -command `gzip -cn` by default. +command `gzip -cn` by default. An internal gzip implementation can be +used by specifying the value `git archive gzip`. tar..remote:: If true, enable the format for use by remote clients via diff --git a/archive-tar.c b/archive-tar.c index 4e6a3deb80..53d0ef685c 100644 --- a/archive-tar.c +++ b/archive-tar.c @@ -38,11 +38,13 @@ static int write_tar_filter_archive(const struct archiver *ar, #define USTAR_MAX_MTIME 077777777777ULL #endif -static void write_block(const void *buf) +static void tar_write_block(const void *buf) { write_or_die(1, buf, BLOCKSIZE); } +static void (*write_block)(const void *) = tar_write_block; + /* writes out the whole block, but only if it is full */ static void write_if_needed(void) { @@ -430,6 +432,34 @@ static int write_tar_archive(const struct archiver *ar, return err; } +static git_zstream gzstream; +static unsigned char outbuf[16384]; + +static void tgz_deflate(int flush) +{ + while (gzstream.avail_in || flush == Z_FINISH) { + int status = git_deflate(&gzstream, flush); + if (!gzstream.avail_out || status == Z_STREAM_END) { + write_or_die(1, outbuf, gzstream.next_out - outbuf); + gzstream.next_out = outbuf; + gzstream.avail_out = sizeof(outbuf); + if (status == Z_STREAM_END) + break; + } + if (status != Z_OK && status != Z_BUF_ERROR) + die(_("deflate error (%d)"), status); + } +} + +static void tgz_write_block(const void *data) +{ + gzstream.next_in = (void *)data; + gzstream.avail_in = BLOCKSIZE; + tgz_deflate(Z_NO_FLUSH); +} + +static const char internal_gzip_command[] = "git archive gzip"; + static int write_tar_filter_archive(const struct archiver *ar, struct archiver_args *args) { @@ -440,6 +470,19 @@ static int write_tar_filter_archive(const struct archiver *ar, if (!ar->filter_command) BUG("tar-filter archiver called with no filter defined"); + if (!strcmp(ar->filter_command, internal_gzip_command)) { + write_block = tgz_write_block; + git_deflate_init_gzip(&gzstream, args->compression_level); + gzstream.next_out = outbuf; + gzstream.avail_out = sizeof(outbuf); + + r = write_tar_archive(ar, args); + + tgz_deflate(Z_FINISH); + git_deflate_end(&gzstream); + return r; + } + strbuf_addstr(&cmd, ar->filter_command); if (args->compression_level >= 0) strbuf_addf(&cmd, " -%d", args->compression_level); diff --git a/t/t5000-tar-tree.sh b/t/t5000-tar-tree.sh index 7f8d2ab0a7..9ac0ec67fe 100755 --- a/t/t5000-tar-tree.sh +++ b/t/t5000-tar-tree.sh @@ -374,6 +374,22 @@ test_expect_success GZIP 'remote tar.gz can be disabled' ' >remote.tar.gz ' +test_expect_success 'git archive --format=tgz (internal gzip)' ' + test_config tar.tgz.command "git archive gzip" && + git archive --format=tgz HEAD >internal_gzip.tgz +' + +test_expect_success 'git archive --format=tar.gz (internal gzip)' ' + test_config tar.tar.gz.command "git archive gzip" && + git archive --format=tar.gz HEAD >internal_gzip.tar.gz && + test_cmp_bin internal_gzip.tgz internal_gzip.tar.gz +' + +test_expect_success GZIP 'extract tgz file (internal gzip)' ' + gzip -d -c internal_gzip.tar && + test_cmp_bin b.tar internal_gzip.tar +' + test_expect_success 'archive and :(glob)' ' git archive -v HEAD -- ":(glob)**/sh" >/dev/null 2>actual && cat >expect < X-Patchwork-Id: 12882783 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D4E61C43334 for ; Wed, 15 Jun 2022 17:04:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234573AbiFOREg (ORCPT ); Wed, 15 Jun 2022 13:04:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40840 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229974AbiFOREe (ORCPT ); Wed, 15 Jun 2022 13:04:34 -0400 Received: from mout.web.de (mout.web.de [212.227.15.14]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F3EBC4BFCB for ; Wed, 15 Jun 2022 10:04:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=web.de; s=dbaedf251592; t=1655312652; bh=mXVTWZ9xjwXYTJa31Owk0xKwD3jj0x/tDsBDzIbgStE=; h=X-UI-Sender-Class:Date:Subject:From:To:Cc:References:In-Reply-To; b=RZDnnvqww5/wt1gG0NhKWlURuxWVoxfgI2bL4WerbRco/rPP0wHGHdEFHZsGRYqPD 5rbGE8RbuwOKizmob7WnDHtAsrHqRTIl9xsEhKYgJtk4+J0FVMs9uXw9oYip9EcKB6 tgpEfNioAWaH48vJ0q6AQWt0oUqFLhNopkULweC0= X-UI-Sender-Class: c548c8c5-30a9-4db5-a2e7-cb6cb037b8f9 Received: from [192.168.178.29] ([79.203.31.99]) by smtp.web.de (mrweb005 [213.165.67.108]) with ESMTPSA (Nemesis) id 1Meler-1nTqBC3b4H-00ahAE; Wed, 15 Jun 2022 19:04:11 +0200 Message-ID: <71a9fb1b-55af-ecff-d672-197e6e557c81@web.de> Date: Wed, 15 Jun 2022 19:04:09 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:91.0) Gecko/20100101 Thunderbird/91.10.0 Subject: [PATCH v4 5/6] archive-tar: use OS_CODE 3 (Unix) for internal gzip Content-Language: en-US From: =?utf-8?q?Ren=C3=A9_Scharfe?= To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin , Rohit Ashiwal , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= , Jeff King , "brian m . carlson" References: <9df761c3-355a-ede9-7971-b32687fe9abb@web.de> In-Reply-To: <9df761c3-355a-ede9-7971-b32687fe9abb@web.de> X-Provags-ID: V03:K1:TWKe6DAHWwBkPppwYgIKab+EqqSpVdFMvXeve3lh3D9Y2/jSxho TzqyEAw4QGGkkqdL7d+HwcUKFjQi/pw/PDIA1ByUYeKEMrxPjKmy1S46HMiNA9R5HxW59Z+ sZjrD1kASaCOsJOvMP5tuf2u1qMtDu2tWsG+yTDBkq+OhEfuSam8bE5MBqapb5cHomudMCa B+IRPu4ykT7tEhOOG2LHg== X-UI-Out-Filterresults: notjunk:1;V03:K0:Vrb7N07gf8M=:BT/Xpzw3oKdbbXfqqpd1nu YK7p/A6z8XJ+rksgruIPxkTj3GiwPrRji0of4FYDs+5Q1pmsC3x2GNq59O81Y52sLRNU7MEZB gSAoTU+PXqdY5ALgB/PBAxyxOT2eLDbPIv6DzlyoEwh2b/SH6WKWa0WeDNtSAONZEcoGrBmZK e0nABtBzyYPYBmlDqBvwaT95nqvwZALrAwOP15SAVSz+DYZfJn7X1G3nmddu+bFCSGKPQpQsk 6j+lZWSeDd2o1b/qlV0nDkyQb3GznWjQg4MfKMEMozOr533YbKj6QMXZE3WEhQf9a1oT0XQIe ZyCGrHpKUMJaqyTeIlBilTiQQVEkUe9TTZy4fxfn+tCwQlAUVNktkhezVaVQx7YEOyiQj5xCk rPBNecLUEAj1qWLUGjfkJgwvJmE3+S/Hv/ldUSqJj+eXhOBiMgcESZNmtQRZy3y8zMl2Ja/K7 z0QloBB+HsmT24Ff+MCh0Wae3PlFPryMQTP165AxSV6xpGaup/7MW4LGwnSFtnSNfAgpfPNkk 5IvA7Wv1Xn43BlspK2aA8Tw33StGV2Et3UtqLyHK8p8ijtZcZqs2kwuhFZ9dXSMaIuxWu2dH3 NdU8o3oe/hYgVaZ6jR0CS3YvMYiEiF3PniX1cMEh1a0/P4Wv15S2w5zayhjea07fkZyV53hM3 WetFbz0mBgfT9dHGIpRBUnU0birg/m+jifX+B6LNEoUcJDoA7cOZK7vTkXn6C2x50c8l7Alg+ 6QlpLHrgXr6zvhmtjClLx0WhWNsdHbL6npPU4uoBRpLPdu+x15p73DP5GCpJMgZk/1duNbSaL Rb2UOIAlTBXTLRi245SbzsHfFDV/L2Z5LHEsMn/VQ+4GYRnUMginU44cMHJ1cyyvPIQPSchqt kMobh1gcmtC7fl7iFSAmBZUZYNM8g/JMnWR0tXQkmpNF0w0rPrFSTGj889P8Tl8eBJp0mIkob 1fJi8Qd3mBLS7zjSk//zzyIckoAouhdIF3/X4nkaJZiWWhdIqLLIgn4HEJjG0jwBYXPyWo6O4 wo/gr6xz2f2u5pIhvrH/VdS7hs0Lwt3qE2i+DGpzuVZ6r4gJ8bDvl3aIjeRisHZ/dmFiCIN9a Vs/JdovjGutEIT7rKx9q7Qv8oVMKJn5OfuQ Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org gzip(1) encodes the OS it runs on in the 10th byte of its output. It uses the following OS_CODE values according to its tailor.h [1]: 0 - MS-DOS 3 - UNIX 5 - Atari ST 6 - OS/2 10 - TOPS-20 11 - Windows NT The gzip.exe that comes with Git for Windows uses OS_CODE 3 for some reason, so this value is used on practically all supported platforms when generating tgz archives using gzip(1). Zlib uses a bigger set of values according to its zutil.h [2], aligned with section 4.4.2 of the ZIP specification, APPNOTE.txt [3]: 0 - MS-DOS 1 - Amiga 3 - UNIX 4 - VM/CMS 5 - Atari ST 6 - OS/2 7 - Macintosh 8 - Z-System 10 - Windows NT 11 - MVS (OS/390 - Z/OS) 13 - Acorn Risc 16 - BeOS 18 - OS/400 19 - OS X (Darwin) Thus the internal gzip implementation in archive-tar.c sets different OS_CODE header values on major platforms Windows and macOS. Git for Windows uses its own zlib-based variant since v2.20.1 by default and thus embeds OS_CODE 10 in tgz archives. The tar archive for a commit is generated consistently on all systems (by the same Git version). The OS_CODE in the gzip header does not influence extraction. Avoid leaking OS information and make tgz archives constistent and reproducable (with the same Git and libz versions) by using OS_CODE 3 everywhere. At least on macOS 12.4 this produces the same output as gzip(1) for the examples I tried: # before $ git -c tar.tgz.command='git archive gzip' archive --format=tgz v2.36.0 | shasum 3abbffb40b7c63cf9b7d91afc682f11682f80759 - # with this patch $ git -c tar.tgz.command='git archive gzip' archive --format=tgz v2.36.0 | shasum dc6dc6ba9636d522799085d0d77ab6a110bcc141 - $ git archive --format=tar v2.36.0 | gzip -cn | shasum dc6dc6ba9636d522799085d0d77ab6a110bcc141 - [1] https://git.savannah.gnu.org/cgit/gzip.git/tree/tailor.h [2] https://github.com/madler/zlib/blob/master/zutil.h [3] https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT Helped-by: Johannes Schindelin Signed-off-by: René Scharfe --- archive-tar.c | 7 +++++++ 1 file changed, 7 insertions(+) -- 2.36.1 diff --git a/archive-tar.c b/archive-tar.c index 53d0ef685c..efba78118b 100644 --- a/archive-tar.c +++ b/archive-tar.c @@ -463,6 +463,9 @@ static const char internal_gzip_command[] = "git archive gzip"; static int write_tar_filter_archive(const struct archiver *ar, struct archiver_args *args) { +#if ZLIB_VERNUM >= 0x1221 + struct gz_header_s gzhead = { .os = 3 }; /* Unix, for reproducibility */ +#endif struct strbuf cmd = STRBUF_INIT; struct child_process filter = CHILD_PROCESS_INIT; int r; @@ -473,6 +476,10 @@ static int write_tar_filter_archive(const struct archiver *ar, if (!strcmp(ar->filter_command, internal_gzip_command)) { write_block = tgz_write_block; git_deflate_init_gzip(&gzstream, args->compression_level); +#if ZLIB_VERNUM >= 0x1221 + if (deflateSetHeader(&gzstream.z, &gzhead) != Z_OK) + BUG("deflateSetHeader() called too late"); +#endif gzstream.next_out = outbuf; gzstream.avail_out = sizeof(outbuf); From patchwork Wed Jun 15 17:05:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ren=C3=A9_Scharfe?= X-Patchwork-Id: 12882784 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DA738C43334 for ; Wed, 15 Jun 2022 17:05:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244057AbiFORFW (ORCPT ); Wed, 15 Jun 2022 13:05:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41294 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240589AbiFORFV (ORCPT ); Wed, 15 Jun 2022 13:05:21 -0400 Received: from mout.web.de (mout.web.de [212.227.15.14]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 915EE4F443 for ; Wed, 15 Jun 2022 10:05:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=web.de; s=dbaedf251592; t=1655312704; bh=Ixt2ZeG3Cp3Z24g+Sya1sLQRzm37+0Ax3Iori4Ynbzs=; h=X-UI-Sender-Class:Date:Subject:From:To:Cc:References:In-Reply-To; b=LzuQzqsSJflcFs53gxhGcmzQN//7eCGzo9Rj8S67jM8Dgv7uKow78+Edh/yxlXkB4 KKMTb2q4YL2SfHSvGEHOZyOd7fWPt4VDsORbGAnmTVjS9CBsTcTESqJEEfTkuSM+fy n5t3GDctcip33ebeNii6ZRN1byblqxzQ0PN5oXKU= X-UI-Sender-Class: c548c8c5-30a9-4db5-a2e7-cb6cb037b8f9 Received: from [192.168.178.29] ([79.203.31.99]) by smtp.web.de (mrweb005 [213.165.67.108]) with ESMTPSA (Nemesis) id 1MOm0x-1oMDAl07Lj-00QIwA; Wed, 15 Jun 2022 19:05:04 +0200 Message-ID: Date: Wed, 15 Jun 2022 19:05:03 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:91.0) Gecko/20100101 Thunderbird/91.10.0 Subject: [PATCH v4 6/6] archive-tar: use internal gzip by default Content-Language: en-US From: =?utf-8?q?Ren=C3=A9_Scharfe?= To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin , Rohit Ashiwal , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= , Jeff King , "brian m . carlson" References: <9df761c3-355a-ede9-7971-b32687fe9abb@web.de> In-Reply-To: <9df761c3-355a-ede9-7971-b32687fe9abb@web.de> X-Provags-ID: V03:K1:bik/D80M1Nkm2hameqZ/xCOWl59mrBR2flDi8W2x1GXPoBshkVD EIFB7PBsi5rfJe0rVZkQ32Nc4Vlxz3V5w6ottT9j9uIx+r/G72sjIfcGSHqm37FVT7o150u EItV3/+gr2UBkjpJyJaG45JgcAA2GGQdb+V3IYwaA67mjMqC/GzipdMQ5G7835uTQpTkHkn YhBYU+pMTYAP8tYNYiVKw== X-UI-Out-Filterresults: notjunk:1;V03:K0:TLsFjttL2F0=:boDO4TZpEj8spBS9OBqZNt zyj3Ih0fzPRuAcsCS9HXCZ5QN0r49Li1RXwKBa+nbGiTYwjvj63GXyKXvFGnJmSM5JpZ0N6w1 NEU8JoAcGmAp+s6ldszf/i9W4oXlMMfFS6U6/1SCv3ov5xOEKHv1Flw9n4rEVlOsv/BpF6isv /2fgquE/tkVOAskoEBqtYBhNC5cLRGpnCL/nkaqduBFPSpKnW0xjwlZqbhikHgn47GgpPC2KP XxOVjw3vTw7ufojmFrYyqK/x6Ti/ksesYiRktBiG3pOY7tYCBTjw39iBuVA/eu2rZSlkxUOXl +5Y1gmKZkl83DnSvPuxjzy4JfbrAFhvDQE9e5DvykWZ+dreDMg5Q1w+xbpNMMldweruZNClUl FVwuD/dXj0+VrC/eGn/07ycy6nNh9wJsnB8YpyuNZN9WaDZUeEqM9oq9s+oQrf98yodioIe5+ +yx3foCznBkbRTlyNW9UZIYqBFUMYBD/JZCCJKiG/5HXaDIWRaOoCg+ilO0B5N4FyvLmCHOar p6SCqWdOhxwIih8tNZyWZSTAiaMT50vPeM+4reyBE+poNuzBLFbbsLo7QWcr+RNOEv8S+333v tZMpUJTZfvNZ1KKevVWKJSa6FRxIlPtZttCFfShTsKtkcBVF2C3eO2CREsxIDYU4wegncv+4d qIK6l7dJFzR0u+l36tB2m+SfwCrHU+dXHnCw2XbxWWVB+Q3mp0JX0c4Jevx522WM/ol1lEoOV 1iMTotS08kG+IxdbPshJ02+m1qi0efScq0+dqjzMPHuOJzE1OmlcWqKkyU53MPl8VgOQGNcg0 0SFavn2zKdVsNhLyrOii5Oa6xj19u+IMg9N+785hd/6C43exT1ub6JVoWPVxwHci/HCeA5c/0 2So+KoHdNPepA/YG/EeTiMd7SChKMLMie4ypuZkNjBOkeTQQF/eitlvKzgjlducnMdN4Qb2ME fCPD7PoGvrmpak3gh8/by6ig0mHaLYl2V8+npiFXm4QNCFcI1MRiN3iLqCmRdR24NCDAh0g/G oe0OX3KzZO/QtVD/rUpUJAeWQ8Ac20dNjsAiJB10nLXl4FOLZ/uHLoONytJSzGXYABDbLQDeT TNkLhd/HY4mVCMoprk2tSpBbM/EeQXJiyqa Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Drop the dependency on gzip(1) and use our internal implementation to create tar.gz and tgz files. Signed-off-by: René Scharfe --- Documentation/git-archive.txt | 4 ++-- archive-tar.c | 4 ++-- t/t5000-tar-tree.sh | 32 ++++++++++++++++---------------- 3 files changed, 20 insertions(+), 20 deletions(-) -- 2.36.1 diff --git a/Documentation/git-archive.txt b/Documentation/git-archive.txt index b2d1b63d31..60c040988b 100644 --- a/Documentation/git-archive.txt +++ b/Documentation/git-archive.txt @@ -148,8 +148,8 @@ tar..command:: to the command (e.g., `-9`). + The `tar.gz` and `tgz` formats are defined automatically and use the -command `gzip -cn` by default. An internal gzip implementation can be -used by specifying the value `git archive gzip`. +magic command `git archive gzip` by default, which invokes an internal +implementation of gzip. tar..remote:: If true, enable the format for use by remote clients via diff --git a/archive-tar.c b/archive-tar.c index efba78118b..3d77e0f750 100644 --- a/archive-tar.c +++ b/archive-tar.c @@ -526,9 +526,9 @@ void init_tar_archiver(void) int i; register_archiver(&tar_archiver); - tar_filter_config("tar.tgz.command", "gzip -cn", NULL); + tar_filter_config("tar.tgz.command", internal_gzip_command, NULL); tar_filter_config("tar.tgz.remote", "true", NULL); - tar_filter_config("tar.tar.gz.command", "gzip -cn", NULL); + tar_filter_config("tar.tar.gz.command", internal_gzip_command, NULL); tar_filter_config("tar.tar.gz.remote", "true", NULL); git_config(git_tar_config, NULL); for (i = 0; i < nr_tar_filters; i++) { diff --git a/t/t5000-tar-tree.sh b/t/t5000-tar-tree.sh index 9ac0ec67fe..1a68e89a55 100755 --- a/t/t5000-tar-tree.sh +++ b/t/t5000-tar-tree.sh @@ -339,21 +339,21 @@ test_expect_success 'only enabled filters are available remotely' ' test_cmp_bin remote.bar config.bar ' -test_expect_success GZIP 'git archive --format=tgz' ' +test_expect_success 'git archive --format=tgz' ' git archive --format=tgz HEAD >j.tgz ' -test_expect_success GZIP 'git archive --format=tar.gz' ' +test_expect_success 'git archive --format=tar.gz' ' git archive --format=tar.gz HEAD >j1.tar.gz && test_cmp_bin j.tgz j1.tar.gz ' -test_expect_success GZIP 'infer tgz from .tgz filename' ' +test_expect_success 'infer tgz from .tgz filename' ' git archive --output=j2.tgz HEAD && test_cmp_bin j.tgz j2.tgz ' -test_expect_success GZIP 'infer tgz from .tar.gz filename' ' +test_expect_success 'infer tgz from .tar.gz filename' ' git archive --output=j3.tar.gz HEAD && test_cmp_bin j.tgz j3.tar.gz ' @@ -363,31 +363,31 @@ test_expect_success GZIP 'extract tgz file' ' test_cmp_bin b.tar j.tar ' -test_expect_success GZIP 'remote tar.gz is allowed by default' ' +test_expect_success 'remote tar.gz is allowed by default' ' git archive --remote=. --format=tar.gz HEAD >remote.tar.gz && test_cmp_bin j.tgz remote.tar.gz ' -test_expect_success GZIP 'remote tar.gz can be disabled' ' +test_expect_success 'remote tar.gz can be disabled' ' git config tar.tar.gz.remote false && test_must_fail git archive --remote=. --format=tar.gz HEAD \ >remote.tar.gz ' -test_expect_success 'git archive --format=tgz (internal gzip)' ' - test_config tar.tgz.command "git archive gzip" && - git archive --format=tgz HEAD >internal_gzip.tgz +test_expect_success GZIP 'git archive --format=tgz (external gzip)' ' + test_config tar.tgz.command "gzip -cn" && + git archive --format=tgz HEAD >external_gzip.tgz ' -test_expect_success 'git archive --format=tar.gz (internal gzip)' ' - test_config tar.tar.gz.command "git archive gzip" && - git archive --format=tar.gz HEAD >internal_gzip.tar.gz && - test_cmp_bin internal_gzip.tgz internal_gzip.tar.gz +test_expect_success GZIP 'git archive --format=tar.gz (external gzip)' ' + test_config tar.tar.gz.command "gzip -cn" && + git archive --format=tar.gz HEAD >external_gzip.tar.gz && + test_cmp_bin external_gzip.tgz external_gzip.tar.gz ' -test_expect_success GZIP 'extract tgz file (internal gzip)' ' - gzip -d -c internal_gzip.tar && - test_cmp_bin b.tar internal_gzip.tar +test_expect_success GZIP 'extract tgz file (external gzip)' ' + gzip -d -c external_gzip.tar && + test_cmp_bin b.tar external_gzip.tar ' test_expect_success 'archive and :(glob)' '