From patchwork Fri Dec 10 15:13:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12669745 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AEFF3C433EF for ; Fri, 10 Dec 2021 15:13:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242508AbhLJPQw (ORCPT ); Fri, 10 Dec 2021 10:16:52 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37910 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242452AbhLJPQv (ORCPT ); Fri, 10 Dec 2021 10:16:51 -0500 Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 24DB4C061746 for ; Fri, 10 Dec 2021 07:13:16 -0800 (PST) Received: by mail-wr1-x432.google.com with SMTP id t18so15407269wrg.11 for ; Fri, 10 Dec 2021 07:13:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=P83ymSPA3zYRQA1Q95p87N+uvnV0pSZrI9/9u64n4Y8=; b=c3b47Zz1RRipRvS05xDYo1M08uPqoZ1TjWkz5+TvKkUwdOGjwkPWEtXuuWkcM0nvLm DvwUCQ7FkSKpZ12FHa0qr6XXhIcGWg3zrYQ0fSIt7NupUt3/QqdntGb/LsesLQCwBHz6 k+UEvzGgdZKFWHPIP+XTpWl0J0tqdDhTMkzw0KkO0WMNuWURTmtfjgotohhwDqp4+LuA xlJ50z2Mw2P7enRxh7ZXow4DlzGqZ89WGDdM8AAeK1h0E/yrUAJEOq0zJURJjPy0XCRj Umj5eoFYg9avYgqeS+jTl7lspACBS8oAyPCwXF3hHLWYMn+b64VgBgVH/UDC/LIamJn2 z7hQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=P83ymSPA3zYRQA1Q95p87N+uvnV0pSZrI9/9u64n4Y8=; b=gT/GW2n+d5WX7Oi7JPiX7VXgsef1JJDWNfePwTSf+nWfwkgqFXSl1YSrgS89AolMtP i+cQBa6EGGBp/Ch249prQCzTN3qZnwiD0wRl1f1qelESEz8Ljv5KuysDd5myp5XmDh4b 5xGQslPpUB3z9VKx9ZXzZIot75XiUPQEde8mGmKGt8opXeOjWphL9Vk+Vo7sL1l+oDs2 RZGSPxQ9CEu2EVKm/rkxZ/QkenOV+Jw2ib7zM8rJ6IIpNQzO+jm3BNKG3g/GXERbO+M5 COgO0cZtEwAnbfFHeFyqmmu4MdrYJzmbLCuQalTyFsdngYQx0NYsQGmREuMVs+nfG9xC LllQ== X-Gm-Message-State: AOAM531SgzEG336J7zPhQUpkd3e9qqWxUlqtI4SGbJ9erUbjo9lKiLhC ntFLnMRC5mWClV5Eo5zM60UomBN9xik= X-Google-Smtp-Source: ABdhPJyc1xAj20O1o75psZLXi8mD+cre1QFa87BNp5MAT87id5lHJEPULnzce0ZmBSKS0lJZn8f1DQ== X-Received: by 2002:a05:6000:1548:: with SMTP id 8mr14341906wry.279.1639149194477; Fri, 10 Dec 2021 07:13:14 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id u13sm14022300wmq.14.2021.12.10.07.13.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Dec 2021 07:13:14 -0800 (PST) Message-Id: In-Reply-To: References: Date: Fri, 10 Dec 2021 15:13:08 +0000 Subject: [PATCH v3 1/5] fetch/pull: use the sparse index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, newren@gmail.com, vdye@github.com, Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee The 'git fetch' and 'git pull' commands parse the index in order to determine if submodules exist. Without command_requires_full_index=0, this will expand a sparse index, causing slow performance even when there is no new data to fetch. The .gitmodules file will never be inside a sparse directory entry, and even if it was, the index_name_pos() method would expand the sparse index if needed as we search for the path by name. These commands do not iterate over the index, which is the typical thing we are careful about when integrating with the sparse index. Signed-off-by: Derrick Stolee --- builtin/fetch.c | 2 ++ builtin/pull.c | 2 ++ t/t1092-sparse-checkout-compatibility.sh | 10 ++++++++++ 3 files changed, 14 insertions(+) diff --git a/builtin/fetch.c b/builtin/fetch.c index f7abbc31ff1..1696b7791d4 100644 --- a/builtin/fetch.c +++ b/builtin/fetch.c @@ -1993,6 +1993,8 @@ int cmd_fetch(int argc, const char **argv, const char *prefix) } git_config(git_fetch_config, NULL); + prepare_repo_settings(the_repository); + the_repository->settings.command_requires_full_index = 0; argc = parse_options(argc, argv, prefix, builtin_fetch_options, builtin_fetch_usage, 0); diff --git a/builtin/pull.c b/builtin/pull.c index 1cfaf9f3436..7bce3bd80f9 100644 --- a/builtin/pull.c +++ b/builtin/pull.c @@ -994,6 +994,8 @@ int cmd_pull(int argc, const char **argv, const char *prefix) set_reflog_message(argc, argv); git_config(git_pull_config, NULL); + prepare_repo_settings(the_repository); + the_repository->settings.command_requires_full_index = 0; argc = parse_options(argc, argv, prefix, pull_options, pull_usage, 0); diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index 83971174ac3..3ba19ba1c14 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -964,6 +964,16 @@ test_expect_success 'sparse index is not expanded: blame' ' done ' +test_expect_success 'sparse index is not expanded: fetch/pull' ' + init_repos && + + git -C sparse-index remote add full "file://$(pwd)/full-checkout" && + ensure_not_expanded fetch full && + git -C full-checkout commit --allow-empty -m "for pull merge" && + git -C sparse-index commit --allow-empty -m "for pull merge" && + ensure_not_expanded pull full base +' + # NEEDSWORK: a sparse-checkout behaves differently from a full checkout # in this scenario, but it shouldn't. test_expect_success 'reset mixed and checkout orphan' ' From patchwork Fri Dec 10 15:13:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12669747 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D55AEC433F5 for ; Fri, 10 Dec 2021 15:13:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242584AbhLJPQx (ORCPT ); Fri, 10 Dec 2021 10:16:53 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37912 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242452AbhLJPQw (ORCPT ); Fri, 10 Dec 2021 10:16:52 -0500 Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 15796C061746 for ; Fri, 10 Dec 2021 07:13:17 -0800 (PST) Received: by mail-wm1-x32b.google.com with SMTP id i8-20020a7bc948000000b0030db7b70b6bso9261851wml.1 for ; Fri, 10 Dec 2021 07:13:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=e7joETFF8NerTgaM+WI6cDtVJOVrVnPxu1yMnUGHR3w=; b=lo5ZqXe8C1nkLHqPTpHEuqgVntsnbNioSOQBgSa7MptDBCSNqhUG6ZxJfQH4RH1h9n sfusrrmE6uT0xDD6yaBYJzlkB4rWWJuW5jSLe/w0hzEFHjHbHQZcQ1A7X1whC+54Fwxp 6ATTO0aaC07Y5Mf+QhVLJtcySibqezz+mpLBsSBo6V2YJaCibI+uyD/ymJQNWG5tl3nd zgMwmrUpDmf22olllXgg+UtXLc03ttKlldWAHmRw/fTcocPO0MNhT/9N7ARa8kRNKRtK 2a3ZEb3QE8RqrBpZESopuVIrAesgGr6VDSyqEDhH+GEL0sClI2Wa6M7doQ2WF/MJGTZi /2fQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=e7joETFF8NerTgaM+WI6cDtVJOVrVnPxu1yMnUGHR3w=; b=MxQviMvB3ieeuP80O+yDxVxDYGlVbJcJ8V4/6FsslR2p1NVA6IeqPI7sJSR3kumAdD d2SiEYmdGn+azCITvfgkdqvc+ZOOVNb6ofkehijfTyJFXdnXemGZCsVJ+K02dgMkU3bJ Sq7hZG5qWjxW2MSc5ayYt3BPpUnJn/jvngANI+bFG5THNkkt9wlyqAshi0zlLIo+d8Ct c13HX3JVKS0rdwKxinClXaGHKsTfzRmxFN/T+LBx7odC58sPJuVmBdRUds4TLIsaseOO k/jXSkfcg27EMR1plu9p5qPAXnHNg3jpbVtFXp94YOlWPjmbUAm29SpHz/n8Dfmu5ulr M5Sw== X-Gm-Message-State: AOAM533ZqdKrcdfQEQvHq2H0iJQUsd8p3mWwCcALn8RWxXpRXOwrCkom OjRKGjGrijmRqOTLQ59yK4j/xi2/2/M= X-Google-Smtp-Source: ABdhPJwdKrGL2f/lUV/tKA1CPqNYxYEcDk/vYBekm7N/l0gNQ2gaWL8nV+aIwNZx/x4ZzXnMpXe8Fg== X-Received: by 2002:a05:600c:6022:: with SMTP id az34mr17321806wmb.22.1639149195340; Fri, 10 Dec 2021 07:13:15 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id d1sm2614510wrz.92.2021.12.10.07.13.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Dec 2021 07:13:14 -0800 (PST) Message-Id: In-Reply-To: References: Date: Fri, 10 Dec 2021 15:13:09 +0000 Subject: [PATCH v3 2/5] ls-files: add --sparse option Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, newren@gmail.com, vdye@github.com, Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Existing callers to 'git ls-files' are expecting file names, not directories. It is best to expand a sparse index to show all of the contained files in this case. However, expert users may want to inspect the contents of the index itself including which directories are sparse. Add a --sparse option to allow users to request this information. During testing, I noticed that options such as --modified did not affect the output when the files in question were outside the sparse-checkout definition. Tests are added to document this preexisting behavior and how it remains unchanged with the sparse index and the --sparse option. Signed-off-by: Derrick Stolee --- Documentation/git-ls-files.txt | 4 + builtin/ls-files.c | 12 ++- t/t1092-sparse-checkout-compatibility.sh | 96 ++++++++++++++++++++++++ 3 files changed, 110 insertions(+), 2 deletions(-) diff --git a/Documentation/git-ls-files.txt b/Documentation/git-ls-files.txt index 6d11ab506b7..1c5d5f85ec5 100644 --- a/Documentation/git-ls-files.txt +++ b/Documentation/git-ls-files.txt @@ -187,6 +187,10 @@ Both the in the index ("i/") and in the working tree ("w/") are shown for regular files, followed by the ("attr/"). +--sparse:: + If the index is sparse, show the sparse directories without expanding + to the contained files. + \--:: Do not interpret any more arguments as options. diff --git a/builtin/ls-files.c b/builtin/ls-files.c index 031fef1bcaa..c151eb1fb77 100644 --- a/builtin/ls-files.c +++ b/builtin/ls-files.c @@ -37,6 +37,7 @@ static int debug_mode; static int show_eol; static int recurse_submodules; static int skipping_duplicates; +static int show_sparse_dirs; static const char *prefix; static int max_prefix_len; @@ -315,8 +316,10 @@ static void show_files(struct repository *repo, struct dir_struct *dir) if (!(show_cached || show_stage || show_deleted || show_modified)) return; - /* TODO: audit for interaction with sparse-index. */ - ensure_full_index(repo->index); + + if (!show_sparse_dirs) + ensure_full_index(repo->index); + for (i = 0; i < repo->index->cache_nr; i++) { const struct cache_entry *ce = repo->index->cache[i]; struct stat st; @@ -670,6 +673,8 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix) OPT_BOOL(0, "debug", &debug_mode, N_("show debugging data")), OPT_BOOL(0, "deduplicate", &skipping_duplicates, N_("suppress duplicate entries")), + OPT_BOOL(0, "sparse", &show_sparse_dirs, + N_("show sparse directories in the presence of a sparse index")), OPT_END() }; int ret = 0; @@ -677,6 +682,9 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix) if (argc == 2 && !strcmp(argv[1], "-h")) usage_with_options(ls_files_usage, builtin_ls_files_options); + prepare_repo_settings(the_repository); + the_repository->settings.command_requires_full_index = 0; + prefix = cmd_prefix; if (prefix) prefix_len = strlen(prefix); diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index 3ba19ba1c14..29b97667378 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -802,6 +802,12 @@ test_expect_success 'sparse-index is expanded and converted back' ' GIT_TRACE2_EVENT="$(pwd)/trace2.txt" GIT_TRACE2_EVENT_NESTING=10 \ git -C sparse-index reset -- folder1/a && test_region index convert_to_sparse trace2.txt && + test_region index ensure_full_index trace2.txt && + + # ls-files expands on read, but does not write. + rm trace2.txt && + GIT_TRACE2_EVENT="$(pwd)/trace2.txt" GIT_TRACE2_EVENT_NESTING=10 \ + git -C sparse-index ls-files && test_region index ensure_full_index trace2.txt ' @@ -826,6 +832,7 @@ test_expect_success 'sparse-index is not expanded' ' init_repos && ensure_not_expanded status && + ensure_not_expanded ls-files --sparse && ensure_not_expanded commit --allow-empty -m empty && echo >>sparse-index/a && ensure_not_expanded commit -a -m a && @@ -974,6 +981,95 @@ test_expect_success 'sparse index is not expanded: fetch/pull' ' ensure_not_expanded pull full base ' +test_expect_success 'ls-files' ' + init_repos && + + # Behavior agrees by default. Sparse index is expanded. + test_all_match git ls-files && + + # With --sparse, the sparse index data changes behavior. + git -C sparse-index ls-files >dense && + git -C sparse-index ls-files --sparse >sparse && + + cat >expect <<-\EOF && + @@ -13,13 +13,9 @@ + e + folder1- + folder1.x + -folder1/0/0/0 + -folder1/0/1 + -folder1/a + +folder1/ + folder10 + -folder2/0/0/0 + -folder2/0/1 + -folder2/a + +folder2/ + g + -x/a + +x/ + z + EOF + + diff -u dense sparse | tail -n +3 >actual && + test_cmp expect actual && + + # With --sparse and no sparse index, nothing changes. + git -C sparse-checkout ls-files >dense && + git -C sparse-checkout ls-files --sparse >sparse && + test_cmp dense sparse && + + # Set up a strange condition of having a file edit + # outside of the sparse-checkout cone. This is just + # to verify that sparse-checkout and sparse-index + # behave the same in this case. + write_script edit-content <<-\EOF && + mkdir folder1 && + echo content >>folder1/a + EOF + run_on_sparse ../edit-content && + + # ls-files does not currently notice modified files whose + # cache entries are marked SKIP_WORKTREE. This may change + # in the future, but here we test that sparse index does + # not accidentally create a change of behavior. + test_sparse_match git ls-files --modified && + test_must_be_empty sparse-checkout-out && + test_must_be_empty sparse-index-out && + + git -C sparse-index ls-files --sparse --modified >sparse-index-out && + test_must_be_empty sparse-index-out && + + # Add folder1 to the sparse-checkout cone and + # check that ls-files shows the expanded files. + test_sparse_match git sparse-checkout add folder1 && + test_sparse_match git ls-files --modified && + + git -C sparse-index ls-files >dense && + git -C sparse-index ls-files --sparse >sparse && + + cat >expect <<-\EOF && + @@ -17,9 +17,7 @@ + folder1/0/1 + folder1/a + folder10 + -folder2/0/0/0 + -folder2/0/1 + -folder2/a + +folder2/ + g + -x/a + +x/ + z + EOF + + diff -u dense sparse | tail -n +3 >actual && + test_cmp expect actual && + + # Double-check index expansion is avoided + ensure_not_expanded ls-files --sparse +' + # NEEDSWORK: a sparse-checkout behaves differently from a full checkout # in this scenario, but it shouldn't. test_expect_success 'reset mixed and checkout orphan' ' From patchwork Fri Dec 10 15:13:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12669749 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A547EC433EF for ; Fri, 10 Dec 2021 15:13:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242635AbhLJPQz (ORCPT ); Fri, 10 Dec 2021 10:16:55 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37920 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242628AbhLJPQy (ORCPT ); Fri, 10 Dec 2021 10:16:54 -0500 Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com [IPv6:2a00:1450:4864:20::332]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C19EEC061746 for ; Fri, 10 Dec 2021 07:13:18 -0800 (PST) Received: by mail-wm1-x332.google.com with SMTP id 137so7064340wma.1 for ; Fri, 10 Dec 2021 07:13:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=XfoLAbru6OVVuU2WQMn25q33O/C0bEdZXzyg2YKjQQc=; b=W7+fcQifMeEdb+8n9mRRFRs16iD2BkBPvwfivP19eERlv4WNTbuHBCbVE1PqHqg6z8 Ph2Wkomo2x6WV6QuduMLnPkcpC6wQbf1yKORQRNdEnsEx+WD8gqJ5rSEr855Dj8NbPOm CjrSZKpWyEFBPg0V3M14QbxiwOxb57fBCm9aSeWXth5J7caTH1Xvg79GhYXAEvLTd5yP /q4RyL2L6TufwrWnjV/KEb6vjWWhE3BJdIDbPKF3IW7OlrgTwNN71k9RWIrnC8/vjS9Z uLV2s29ouy6Dvp9ldOuCprgMZUJwDTr6292RN2gsVzPgWBTMt1BOF+iYtRrBW8YgkF6I coBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=XfoLAbru6OVVuU2WQMn25q33O/C0bEdZXzyg2YKjQQc=; b=rCY8oo36DQn+nZPxPAaTlE3F7r4KqrU4wTVS8lCzMN43fa0QXaSQHMRi2jNbG7yWC/ XQl1G0qDDtp2MaMFZa0cqPmTi9dEn7xxmkf2WuGpRZvfx/PkX1B4ppumk96bx5FblZ7R FJdOWO3wSQvLekwlftRvhddtPVVDM6xKKpRLjfVr4/Y0D2E53A90LeDe9PM2ab5biXFj 2dq4Cc+SVkOYkjIS9sl0iSnLFrLQGK8lawFNWk0B1xUwYw5RO4w4e+LJ4z6F20g6M1AA XprAA1Cu74yAAEGrQo6ZJzzOOnNr2FU4qH3Ze3UDU7Id7l0W6LArzuhVrHUD/r1l3Dy+ Zmgg== X-Gm-Message-State: AOAM532lTw1Oe8TxhWLuYvelsk6sCOOH+hNjwNtoENyDRV07/3Ugfowa M8pnKWBr/LNi9IZmE/aGAjYRqXSUQ/I= X-Google-Smtp-Source: ABdhPJzhM5LV0Y09OXtG3SiAONRkFfpknqc0lrx7IG1e55z/OPpoZrszGXbxDUyqZfR2gstoIgu35A== X-Received: by 2002:a7b:cc95:: with SMTP id p21mr17397600wma.45.1639149196206; Fri, 10 Dec 2021 07:13:16 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id c4sm2721254wrr.37.2021.12.10.07.13.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Dec 2021 07:13:15 -0800 (PST) Message-Id: <2a6a1c5a39c9f07f4714d0d9916c3e1775836372.1639149192.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Fri, 10 Dec 2021 15:13:10 +0000 Subject: [PATCH v3 3/5] t1092: replace 'read-cache --table' with 'ls-files --sparse' Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, newren@gmail.com, vdye@github.com, Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Now that 'git ls-files --sparse' exists, we can use it to verify the state of a sparse index instead of 'test-tool read-cache table'. Replace these usages within t1092-sparse-checkout-compatibility.sh. The important changes are due to the different output format. We need to use the '--stage' output to get a file mode and OID, but it also includes a stage value and drops the object type. This leads to some differences in how we handle looking for specific entries. Some places where we previously looked for no 'tree' entries, we can instead directly compare the output across the repository with a sparse index and the one without. Signed-off-by: Derrick Stolee --- t/t1092-sparse-checkout-compatibility.sh | 31 +++++++++++------------- 1 file changed, 14 insertions(+), 17 deletions(-) diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index 29b97667378..d4a0d8ce825 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -200,45 +200,42 @@ test_sparse_unstaged () { test_expect_success 'sparse-index contents' ' init_repos && - test-tool -C sparse-index read-cache --table >cache && + git -C sparse-index ls-files --sparse --stage >cache && for dir in folder1 folder2 x do TREE=$(git -C sparse-index rev-parse HEAD:$dir) && - grep "040000 tree $TREE $dir/" cache \ + grep "040000 $TREE 0 $dir/" cache \ || return 1 done && git -C sparse-index sparse-checkout set folder1 && - test-tool -C sparse-index read-cache --table >cache && + git -C sparse-index ls-files --sparse --stage >cache && for dir in deep folder2 x do TREE=$(git -C sparse-index rev-parse HEAD:$dir) && - grep "040000 tree $TREE $dir/" cache \ + grep "040000 $TREE 0 $dir/" cache \ || return 1 done && git -C sparse-index sparse-checkout set deep/deeper1 && - test-tool -C sparse-index read-cache --table >cache && + git -C sparse-index ls-files --sparse --stage >cache && for dir in deep/deeper2 folder1 folder2 x do TREE=$(git -C sparse-index rev-parse HEAD:$dir) && - grep "040000 tree $TREE $dir/" cache \ + grep "040000 $TREE 0 $dir/" cache \ || return 1 done && - # Disabling the sparse-index removes tree entries with full ones + # Disabling the sparse-index replaces tree entries with full ones git -C sparse-index sparse-checkout init --no-sparse-index && - - test-tool -C sparse-index read-cache --table >cache && - ! grep "040000 tree" cache && - test_sparse_match test-tool read-cache --table + test_sparse_match git ls-files --stage --sparse ' test_expect_success 'expanded in-memory index matches full index' ' init_repos && - test_sparse_match test-tool read-cache --expand --table + test_sparse_match git ls-files --stage ' test_expect_success 'status with options' ' @@ -787,9 +784,9 @@ test_expect_success 'submodule handling' ' # having a submodule prevents "modules" from collapse test_sparse_match git sparse-checkout set deep/deeper1 && - test-tool -C sparse-index read-cache --table >cache && - grep "100644 blob .* modules/a" cache && - grep "160000 commit $(git -C initial-repo rev-parse HEAD) modules/sub" cache + git -C sparse-index ls-files --sparse --stage >cache && + grep "100644 .* modules/a" cache && + grep "160000 $(git -C initial-repo rev-parse HEAD) 0 modules/sub" cache ' # When working with a sparse index, some commands will need to expand the @@ -1085,13 +1082,13 @@ test_expect_success 'reset mixed and checkout orphan' ' # the sparse checkouts skip "adding" the other side of # the conflict. test_sparse_match git reset --mixed HEAD~1 && - test_sparse_match test-tool read-cache --table --expand && + test_sparse_match git ls-files --stage && test_sparse_match git status --porcelain=v2 && # At this point, sparse-checkouts behave differently # from the full-checkout. test_sparse_match git checkout --orphan new-branch && - test_sparse_match test-tool read-cache --table --expand && + test_sparse_match git ls-files --stage && test_sparse_match git status --porcelain=v2 ' From patchwork Fri Dec 10 15:13:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12669751 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AAF80C433FE for ; Fri, 10 Dec 2021 15:13:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242696AbhLJPQz (ORCPT ); Fri, 10 Dec 2021 10:16:55 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37930 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242644AbhLJPQy (ORCPT ); Fri, 10 Dec 2021 10:16:54 -0500 Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 75D76C0617A1 for ; Fri, 10 Dec 2021 07:13:19 -0800 (PST) Received: by mail-wr1-x435.google.com with SMTP id c4so15420876wrd.9 for ; Fri, 10 Dec 2021 07:13:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=z1lsY1IELUoUSjAMa97erruqhiY9BJgxhgMt/g/gEqo=; b=i82NCnCctIu2HuTb5DtM4V1n4y3N57wfiEcj5fpX2oSTzm/gbFO01QhwLJU452Fpdi 4hiBxSnVwjzo0CszCwsJIpg5gEBb1QCkXF0JEnyfUVxLEdH0+/mWGbf7IhTMooMOihEi FCcDtkJrZL3q5DF+s06i7RsXitAyY6KfS8PJh/+YIV9CZGOStUwsyJyKmdNmFt/IfxuV gl428bLF4FgW6beEXpeYE75Y+4XocSdd6PCgOXtHxx5UcDJxSSornr33mWnPQq/Gpe9K 75HwO7HMd4MgJnizk8ZlCEFNhqFAObKoW0BdLvLhdXIThTs19siZa9LlMWLWyIfOSPt+ wtzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=z1lsY1IELUoUSjAMa97erruqhiY9BJgxhgMt/g/gEqo=; b=pS4SDIavx2r/+ZX0k1gyM+r2v4+rACQQu0myJ7jQr30Z0V9pyVhKmRE2ZjWxka4tg1 4RpzObjlusOYjQibHaCIW8FiaHpNTC3AEtiVEyvNJqELozPNpQwgOr1P89Zs4RYqq8Yt aBjmANFjmZk2NDv+/rrhZAD0r+GeAyjrwUKaTo4m1snTv4fMpUt7ztvWrtXxfk6nSSnJ Khyzh949NCYsZz09KEDcT8X60TCurR2NHJE2KIGk/MShE/xuvIacX/QxXVeT75th1E8m 1GA4XfqWrlhOOFF6tAbudZmYfmj5gRTpv7IhgRr5Cf9oudntvbozs6qAjnMhKag7+4Nm E8bw== X-Gm-Message-State: AOAM532zN8cXq9tHx/GYgS0mVjlWAg0uTSnYZaPmdq7qG9A1neb8HuQm 1V9PnQKXZNsRTJnUo28d3szpxY9du/E= X-Google-Smtp-Source: ABdhPJyDz62EeQCNvU1XxpMo1S7eAoQ9kbQopnDT/oDSNG3D67+wbYF1Z4tbHbyn9CGME41K9jLHSQ== X-Received: by 2002:adf:f001:: with SMTP id j1mr14539018wro.351.1639149197792; Fri, 10 Dec 2021 07:13:17 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id n2sm11801525wmi.36.2021.12.10.07.13.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Dec 2021 07:13:17 -0800 (PST) Message-Id: In-Reply-To: References: Date: Fri, 10 Dec 2021 15:13:11 +0000 Subject: [PATCH v3 4/5] t1091/t3705: remove 'test-tool read-cache --table' Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, newren@gmail.com, vdye@github.com, Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Now that 'git ls-files --sparse' exists, we can use it to verify the state of a sparse index instead of 'test-tool read-cache table'. Replace these usages within t1091-sparse-checkout-builtin.sh and t3705-add-sparse-checkout.sh. The important changes are due to the different output format. In t3705, we need to use the '--stage' output to get a file mode and OID, but it also includes a stage value and drops the object type. This leads to some differences in how we handle looking for specific entries. In t1091, the test focuses on enabling the sparse index, so we do not need the --stage flag to demonstrate how the index changes, and instead can use a diff. Signed-off-by: Derrick Stolee --- t/t1091-sparse-checkout-builtin.sh | 25 ++++++++++++++++++++----- t/t3705-add-sparse-checkout.sh | 8 ++++---- 2 files changed, 24 insertions(+), 9 deletions(-) diff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh index 272ba1b566b..680e0043c36 100755 --- a/t/t1091-sparse-checkout-builtin.sh +++ b/t/t1091-sparse-checkout-builtin.sh @@ -212,12 +212,27 @@ test_expect_success 'sparse-index enabled and disabled' ' git -C repo sparse-checkout init --cone --sparse-index && test_cmp_config -C repo true index.sparse && - test-tool -C repo read-cache --table >cache && - grep " tree " cache && - + git -C repo ls-files --sparse >sparse && git -C repo sparse-checkout disable && - test-tool -C repo read-cache --table >cache && - ! grep " tree " cache && + git -C repo ls-files --sparse >full && + + cat >expect <<-\EOF && + @@ -1,4 +1,7 @@ + a + -deep/ + -folder1/ + -folder2/ + +deep/a + +deep/deeper1/a + +deep/deeper1/deepest/a + +deep/deeper2/a + +folder1/a + +folder2/a + EOF + + diff -u sparse full | tail -n +3 >actual && + test_cmp expect actual && + git -C repo config --list >config && ! grep index.sparse config ) diff --git a/t/t3705-add-sparse-checkout.sh b/t/t3705-add-sparse-checkout.sh index f3143c92908..81f3384eeed 100755 --- a/t/t3705-add-sparse-checkout.sh +++ b/t/t3705-add-sparse-checkout.sh @@ -181,13 +181,13 @@ test_expect_success 'git add fails outside of sparse-checkout definition' ' # Avoid munging CRLFs to avoid an error message git -c core.autocrlf=input add --sparse sparse_entry 2>stderr && test_must_be_empty stderr && - test-tool read-cache --table >actual && - grep "^100644 blob.*sparse_entry\$" actual && + git ls-files --stage >actual && + grep "^100644 .*sparse_entry\$" actual && git add --sparse --chmod=+x sparse_entry 2>stderr && test_must_be_empty stderr && - test-tool read-cache --table >actual && - grep "^100755 blob.*sparse_entry\$" actual && + git ls-files --stage >actual && + grep "^100755 .*sparse_entry\$" actual && git reset && From patchwork Fri Dec 10 15:13:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12669753 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3E6BDC4321E for ; Fri, 10 Dec 2021 15:13:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242702AbhLJPQ4 (ORCPT ); Fri, 10 Dec 2021 10:16:56 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37932 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242687AbhLJPQz (ORCPT ); Fri, 10 Dec 2021 10:16:55 -0500 Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [IPv6:2a00:1450:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 41B26C061746 for ; Fri, 10 Dec 2021 07:13:20 -0800 (PST) Received: by mail-wr1-x42d.google.com with SMTP id q3so15443588wru.5 for ; Fri, 10 Dec 2021 07:13:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=a6pLplMAAQeTlcrzPDl3fCgT4av+qlYwa5dP/iJQKp0=; b=J6+UZa7RVAIp4nuZ1/fjdw3WmHkLpIDlANYASDPzrybfG/iwEJFyOZUYvJ00Hmgsef IqUuTq0ARo3uhHPiRw2r0q+Td67lA6V7wgsmKNos9HwrgxMVpTADCtyukWn+e5klD1iE 5hsUREovOu2ynB5iJWeu8x/CsGRWM2AFmB6//XgXpazDvj+70EpgN72upYUEHTm0Xw7r h06doYkgNMLvbXhYsNrLZPdfjzhlOtuxDNb0pdSK1PMMsj8EYNo0hvezTXEBqf4UmO6L pFwi2RLaBAV8XFbmf8qARNVdXTpghJmYzE7l7kJceaLttFlmqzFmpHxg9KZVDhKhxteW N+Ag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=a6pLplMAAQeTlcrzPDl3fCgT4av+qlYwa5dP/iJQKp0=; b=7kAF9wfKr8RED8Ed0HEtJ40F/JWcP76S589QK6ZjkiCHTrFlteyAljZsna6dHedvAn TptEAfitfpmapmEAs2kyfupw2VtbYjVsY1MSJz8JdWN4iSLcj7jEc5YRCsbT6AFIvyHB fKkZt+OhVOe6zIIWnMM+4dZ3B143OuLyLUDP5bkXo17ir2U6pEtAaI97ePEaUgenzg5X C+dQTZ82ihGjHwgy3LB72m5T7x2ofAB+cqW0EYvbY62WGP9YUuRQy1qmjIoHDk7JHCjT O6gA0FFa01/6caOrnwacvhwJoT+ukrYGCqRd99cj8Y6SHzA2VYJP4xm4UmgtT1q5l/2Z +5vw== X-Gm-Message-State: AOAM532ecmrfooJ4cATSoy86l/Nly20NLeoSXRchKA7Apxa7XZn7NTL7 TLR9SydcPc4tQB/yV89O3i5B8ZNP9DI= X-Google-Smtp-Source: ABdhPJwTpXb9/gGR5++UKQ8WxXuc6ttUui2BZN2hMOpWhN4lREm4qoUtyR488cS/eFm7+Y4fiCS0YA== X-Received: by 2002:a05:6000:1548:: with SMTP id 8mr14342359wry.279.1639149198620; Fri, 10 Dec 2021 07:13:18 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id z6sm2758139wrm.93.2021.12.10.07.13.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Dec 2021 07:13:18 -0800 (PST) Message-Id: <9227dc54165e4d452d886905996e7164b3c5e525.1639149192.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Fri, 10 Dec 2021 15:13:12 +0000 Subject: [PATCH v3 5/5] test-read-cache: remove --table, --expand options Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, newren@gmail.com, vdye@github.com, Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee This commit effectively reverts 2782db3 (test-tool: don't force full index, 2021-03-30) and e2df6c3 (test-read-cache: print cache entries with --table, 2021-03-30) to remove the --table and --expand options from 'test-tool read-cache'. The previous changes already removed these options from the test suite in favor of 'git ls-files --sparse'. The initial thought of creating these options was to allow for tests to see additional information with every cache entry. In particular, the object type is still not mirrored in 'git ls-files'. Since sparse directory entries always end with a slash, the object type is not critical to verify the sparse index is enabled. It was thought that it would be helpful to have additional information, such as flags, but that was not needed for the FS Monitor integration and hasn't been needed since. Signed-off-by: Derrick Stolee --- t/helper/test-read-cache.c | 64 ++++++-------------------------------- 1 file changed, 10 insertions(+), 54 deletions(-) diff --git a/t/helper/test-read-cache.c b/t/helper/test-read-cache.c index 0d9f08931a1..b736ef16421 100644 --- a/t/helper/test-read-cache.c +++ b/t/helper/test-read-cache.c @@ -1,83 +1,39 @@ #include "test-tool.h" #include "cache.h" #include "config.h" -#include "blob.h" -#include "commit.h" -#include "tree.h" -#include "sparse-index.h" - -static void print_cache_entry(struct cache_entry *ce) -{ - const char *type; - printf("%06o ", ce->ce_mode & 0177777); - - if (S_ISSPARSEDIR(ce->ce_mode)) - type = tree_type; - else if (S_ISGITLINK(ce->ce_mode)) - type = commit_type; - else - type = blob_type; - - printf("%s %s\t%s\n", - type, - oid_to_hex(&ce->oid), - ce->name); -} - -static void print_cache(struct index_state *istate) -{ - int i; - for (i = 0; i < istate->cache_nr; i++) - print_cache_entry(istate->cache[i]); -} int cmd__read_cache(int argc, const char **argv) { - struct repository *r = the_repository; int i, cnt = 1; const char *name = NULL; - int table = 0, expand = 0; initialize_the_repository(); - for (++argv, --argc; *argv && starts_with(*argv, "--"); ++argv, --argc) { - if (skip_prefix(*argv, "--print-and-refresh=", &name)) - continue; - if (!strcmp(*argv, "--table")) - table = 1; - else if (!strcmp(*argv, "--expand")) - expand = 1; + if (argc > 1 && skip_prefix(argv[1], "--print-and-refresh=", &name)) { + argc--; + argv++; } - if (argc == 1) - cnt = strtol(argv[0], NULL, 0); + if (argc == 2) + cnt = strtol(argv[1], NULL, 0); setup_git_directory(); git_config(git_default_config, NULL); - prepare_repo_settings(r); - r->settings.command_requires_full_index = 0; - for (i = 0; i < cnt; i++) { - repo_read_index(r); - - if (expand) - ensure_full_index(r->index); - + read_cache(); if (name) { int pos; - refresh_index(r->index, REFRESH_QUIET, + refresh_index(&the_index, REFRESH_QUIET, NULL, NULL, NULL); - pos = index_name_pos(r->index, name, strlen(name)); + pos = index_name_pos(&the_index, name, strlen(name)); if (pos < 0) die("%s not in index", name); printf("%s is%s up to date\n", name, - ce_uptodate(r->index->cache[pos]) ? "" : " not"); + ce_uptodate(the_index.cache[pos]) ? "" : " not"); write_file(name, "%d\n", i); } - if (table) - print_cache(r->index); - discard_index(r->index); + discard_cache(); } return 0; }