From patchwork Tue Aug 17 13:23:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12441447 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 060FAC4320A for ; Tue, 17 Aug 2021 13:23:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D72FB60FD9 for ; Tue, 17 Aug 2021 13:23:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239942AbhHQNYR (ORCPT ); Tue, 17 Aug 2021 09:24:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41006 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240028AbhHQNYL (ORCPT ); Tue, 17 Aug 2021 09:24:11 -0400 Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com [IPv6:2a00:1450:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 07B68C0612A6 for ; Tue, 17 Aug 2021 06:23:27 -0700 (PDT) Received: by mail-wr1-x433.google.com with SMTP id h13so28673855wrp.1 for ; Tue, 17 Aug 2021 06:23:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=h0jP/WQTQwK/dNNHt3JH4FuVCZwq3HAW8lG1yL7NHbc=; b=Ia03hKAOnsJFd7zwknfj6Rdx5lulrzUSt4EowWqK8J7YmbBhfnl7D+e0PbBEpmLP4A o5SiL+rfXBYJJ9qRkf0yicKiQlSE3T2i8oFnDR6a9IvspsubjzoaUacEYrGskSdaxq+o Kfme+GOD3o4GPWP+Bbhyq3N+6O08TkyHhT0SHG1WWc84al2bI7aSnZ6jueok5+TXG+Sq iqM1EP9V+rtHgAKGlKzvXfbvo9jx28x99UcXuRBh8DEUqo7Vd8gjHfaV9fDVyqHtEdhi Xsd2ABi7/FNLfBhGmLeMl8YYxkClYsdlAW62m7mNtE9OnrLfTxPdN3qIbLp7R3imvA4v H09A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=h0jP/WQTQwK/dNNHt3JH4FuVCZwq3HAW8lG1yL7NHbc=; b=GGafUibTXBi9JHktRVejVgVLVCri0QaZCOZn+bf3KJWdNJqh+M4OCh0DtLc7/2l4Yu 2SFdK/XfX+ZlJGIbAqvNOKUqiZqjiLJio40Id9oAsyBx9HC91KW2GPCLxNWOWsRBG27t UQOv4mpUpfkVuwc/toBuaeFlWYao92ufPlHsxbUqbL+b/jnbX3XuQ7n63EjE4H1roUuY DQDTe4EwQd8VLRRIAWk1KwWNzfPMTEXF3SCNNbERTJGx6JKO9z0vPxUOfzPCayPIHH2N ZCrxJgm6XTwVR6yYwbbDMbj/rdFwTGFCpdRUKIGY9R/waCOYVpm+YQEBCwKeIWtPpUxF wkGA== X-Gm-Message-State: AOAM532ByGznuTrFLOSbuzLB9kZB3d3CK225k85lylXg8FqPJ6aMRQ10 gQhGMD1XiX1liva8m6okOyHjp+rAue8= X-Google-Smtp-Source: ABdhPJyYAtrcra51mpP4boE2RFpKKaqbu3WjJGZleTQuyzf6moejGKoocdHVlxCHFnm8d+3ZfPJnng== X-Received: by 2002:adf:f5d2:: with SMTP id k18mr4063370wrp.362.1629206605682; Tue, 17 Aug 2021 06:23:25 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id y3sm2422228wma.32.2021.08.17.06.23.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Aug 2021 06:23:25 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Tue, 17 Aug 2021 13:23:15 +0000 Subject: [PATCH v3 1/8] t7519: rewrite sparse index test Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, newren@gmail.com, matheus.bernardino@usp.br, stolee@gmail.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee The sparse index is tested with the FS Monitor hook and extension since f8fe49e (fsmonitor: integrate with sparse index, 2021-07-14). This test was very fragile because it shared an index across sparse and non-sparse behavior. Since that expansion and contraction could cause the index to lose its FS Monitor bitmap and token, behavior is fragile to changes in 'git sparse-checkout set'. Rewrite the test to use two clones of the original repo: full and sparse. This allows us to also keep the test files (actual, expect, trace2.txt) out of the repos we are testing with 'git status'. Signed-off-by: Derrick Stolee --- t/t7519-status-fsmonitor.sh | 38 ++++++++++++++++++++----------------- 1 file changed, 21 insertions(+), 17 deletions(-) diff --git a/t/t7519-status-fsmonitor.sh b/t/t7519-status-fsmonitor.sh index deea88d4431..6f2cf306f66 100755 --- a/t/t7519-status-fsmonitor.sh +++ b/t/t7519-status-fsmonitor.sh @@ -389,43 +389,47 @@ test_expect_success 'status succeeds after staging/unstaging' ' # If "!" is supplied, then we verify that we do not call ensure_full_index # during a call to 'git status'. Otherwise, we verify that we _do_ call it. check_sparse_index_behavior () { - git status --porcelain=v2 >expect && - git sparse-checkout init --cone --sparse-index && - git sparse-checkout set dir1 dir2 && + git -C full status --porcelain=v2 >expect && GIT_TRACE2_EVENT="$(pwd)/trace2.txt" GIT_TRACE2_EVENT_NESTING=10 \ - git status --porcelain=v2 >actual && + git -C sparse status --porcelain=v2 >actual && test_region $1 index ensure_full_index trace2.txt && test_region fsm_hook query trace2.txt && test_cmp expect actual && - rm trace2.txt && - git sparse-checkout disable + rm trace2.txt } test_expect_success 'status succeeds with sparse index' ' - git reset --hard && + git clone . full && + git clone . sparse && + git -C sparse sparse-checkout init --cone --sparse-index && + git -C sparse sparse-checkout set dir1 dir2 && - test_config core.fsmonitor "$TEST_DIRECTORY/t7519/fsmonitor-all" && - check_sparse_index_behavior ! && - - write_script .git/hooks/fsmonitor-test<<-\EOF && + write_script .git/hooks/fsmonitor-test <<-\EOF && printf "last_update_token\0" EOF - git config core.fsmonitor .git/hooks/fsmonitor-test && + git -C full config core.fsmonitor ../.git/hooks/fsmonitor-test && + git -C sparse config core.fsmonitor ../.git/hooks/fsmonitor-test && check_sparse_index_behavior ! && - write_script .git/hooks/fsmonitor-test<<-\EOF && + write_script .git/hooks/fsmonitor-test <<-\EOF && printf "last_update_token\0" printf "dir1/modified\0" EOF check_sparse_index_behavior ! && - cp -r dir1 dir1a && - git add dir1a && - git commit -m "add dir1a" && + git -C sparse sparse-checkout add dir1a && + + for repo in full sparse + do + cp -r $repo/dir1 $repo/dir1a && + git -C $repo add dir1a && + git -C $repo commit -m "add dir1a" || return 1 + done && + git -C sparse sparse-checkout set dir1 dir2 && # This one modifies outside the sparse-checkout definition # and hence we expect to expand the sparse-index. - write_script .git/hooks/fsmonitor-test<<-\EOF && + write_script .git/hooks/fsmonitor-test <<-\EOF && printf "last_update_token\0" printf "dir1a/modified\0" EOF From patchwork Tue Aug 17 13:23:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12441445 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9B86CC4338F for ; Tue, 17 Aug 2021 13:23:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8006560F35 for ; Tue, 17 Aug 2021 13:23:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237615AbhHQNYQ (ORCPT ); Tue, 17 Aug 2021 09:24:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40980 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240037AbhHQNYM (ORCPT ); Tue, 17 Aug 2021 09:24:12 -0400 Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com [IPv6:2a00:1450:4864:20::32a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F13B5C0612A7 for ; Tue, 17 Aug 2021 06:23:27 -0700 (PDT) Received: by mail-wm1-x32a.google.com with SMTP id u15so13714855wmj.1 for ; Tue, 17 Aug 2021 06:23:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=xF4Ml5ovwsHr66snlzRz3TrGLFSkIqW9omB3/KckS/A=; b=jYs+B8Xe1Jxbqdg3XrUT5zU/FPyyCtYH1lZzQgFmsgpf2as34Hta6UrbwrqvigoQoM iWWte59tAxBIQIpbEBJx2BQoOak/QYU7VY1j4d3l+NTrjS6QXczwZT1Itxs6Kp3i+Wlm ggMlWovPwNZZIh4JGZRfFEZStvnPeql6FnDkPE758qwGfXPDn7YFAJTtq/ePyarmP5Mt 4IhvPyAm+WUDaAlQqqrRErwFA4mHl9F+S+yc29+vXrlbH2IcaUC3yC3sGHqA+fK5gZTo nHnZnIiLjdCZj9C2tLFUUWlE6ASS4JsDXe0ov7umXdeID/RZhOeKth1P3Tj9MwqN/sxm F/cw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=xF4Ml5ovwsHr66snlzRz3TrGLFSkIqW9omB3/KckS/A=; b=ghYYj4y6UgOGHFIRjXqbCxFbj7EqGWWxd+Z0il+Ubexr7+MUSndJ7zhDuxEfGsB8gg J7KOGsKVOSDQfzqCKhgm/eFFaJFmrZJSgdMEQ3zXrSC4bnPMNaoAqHvgbxUoX/pjObHT TOWbS6RMEIhoR/g1CZMpYbHlTOGV/SE/KVcM+BKmMabmn7bR2omr4uZIJR0UjyUU88jU 4kvK3/Gt5W5Ym6svtYVl1MNzfMryT9B7z7XY1frmE0+zJgKNb3SqqMXQWcTwwigBB5Yq H9zS/kzugJO4l8noPAGgcSd+7gHLSoE94Hz/Icr3TwA+2KIObCgydELVaK8zY90jZKwY ohGw== X-Gm-Message-State: AOAM530hf4YxZevG922qiFGwlbcbIt2d1RVhrFowCn3j5Qs3ve1jUY+L tLbZeS9p4Sf7aTyW4557r3tdz/G8Kpg= X-Google-Smtp-Source: ABdhPJyapc0PAj/l6eHgdWbwKypms3GfUJYO0mA9onwFJ8F1Jzio3s1/LeQg9/5Z9sXfToX54Q1DxQ== X-Received: by 2002:a1c:ed10:: with SMTP id l16mr3293424wmh.8.1629206606649; Tue, 17 Aug 2021 06:23:26 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id k17sm2454998wmj.0.2021.08.17.06.23.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Aug 2021 06:23:26 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Tue, 17 Aug 2021 13:23:16 +0000 Subject: [PATCH v3 2/8] sparse-index: silently return when not using cone-mode patterns Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, newren@gmail.com, matheus.bernardino@usp.br, stolee@gmail.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee While the sparse-index is only enabled when core.sparseCheckoutCone is also enabled, it is possible for the user to modify the sparse-checkout file manually in a way that does not match cone-mode patterns. In this case, we should refuse to convert an index into a sparse index, since the sparse_checkout_patterns will not be initialized with recursive and parent path hashsets. Also silently return if there are no cache entries, which is a simple case: there are no paths to make sparse! Signed-off-by: Derrick Stolee --- sparse-index.c | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/sparse-index.c b/sparse-index.c index c6b4feec413..bc5900eae35 100644 --- a/sparse-index.c +++ b/sparse-index.c @@ -130,7 +130,8 @@ static int index_has_unmerged_entries(struct index_state *istate) int convert_to_sparse(struct index_state *istate) { int test_env; - if (istate->split_index || istate->sparse_index || + + if (istate->split_index || istate->sparse_index || !istate->cache_nr || !core_apply_sparse_checkout || !core_sparse_checkout_cone) return 0; @@ -158,10 +159,16 @@ int convert_to_sparse(struct index_state *istate) return 0; } - if (!istate->sparse_checkout_patterns->use_cone_patterns) { - warning(_("attempting to use sparse-index without cone mode")); - return -1; - } + /* + * We need cone-mode patterns to use sparse-index. If a user edits + * their sparse-checkout file manually, then we can detect during + * parsing that they are not actually using cone-mode patterns and + * hence we need to abort this conversion _without error_. Warnings + * already exist in the pattern parsing to inform the user of their + * bad patterns. + */ + if (!istate->sparse_checkout_patterns->use_cone_patterns) + return 0; /* * NEEDSWORK: If we have unmerged entries, then stay full. From patchwork Tue Aug 17 13:23:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12441451 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C658BC4338F for ; Tue, 17 Aug 2021 13:23:50 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AB5C660F39 for ; Tue, 17 Aug 2021 13:23:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239969AbhHQNYT (ORCPT ); Tue, 17 Aug 2021 09:24:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41008 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240039AbhHQNYM (ORCPT ); Tue, 17 Aug 2021 09:24:12 -0400 Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com [IPv6:2a00:1450:4864:20::32d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A6844C0612A8 for ; Tue, 17 Aug 2021 06:23:28 -0700 (PDT) Received: by mail-wm1-x32d.google.com with SMTP id c129-20020a1c35870000b02902e6b6135279so2163304wma.0 for ; Tue, 17 Aug 2021 06:23:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=1hHIMQenL0YY07ojegpV/bWS0DueNnK3juz9ucy/bVk=; b=P+abpnWKku679HQ5SfgX4RpqPkw1GE+reMiOw6BcdsOWF0OBx+xiufwJi0OaqxSlY8 d7rnGYZ03ONXW13/6jIgaWbK/gPZOGqCzkn4SRpnFsrzwCVoDe/amrGV2Pf5Dzm75YG+ dEjrFMMSRBtZzceDobChwtOioLZOYWE32cp6HIFZl+4N86yLbFtT7SFjr2SeewcA36S7 7K1aqit5hs8mHd6yea+b3TxW1oDxjv076H996+n4vrslr6b/ntgywZVyyN4SFwz2qlko KdRrvAFdWtnGN0ulGE52gl5+bTIrQsVKNKjhzAIlmxBTUlSjkAjbkY/m/Hk580ZNf1UF Wvwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=1hHIMQenL0YY07ojegpV/bWS0DueNnK3juz9ucy/bVk=; b=oueHFDtHpbA6zsigu6oaBdjsXEh/uuJYzS/NfVmdJhCxkZYTDfix0WgWlKZ19Vu/cJ iV3EG4aMs8tfOJmf72mHcOLlh4teVA/HtPPm0IMGjv/t2TSvV7m2/IShVqK2AIcT8B0e mJKd/q2o0WCae9r+Fnmbgrhd75pENFxWz25FnHBadnzl1YivgF1lpfOHM+Kduu+Kc4qg NPpWml4JHEElEOYlXt+dIPLW/kuPF/ynT9eDOl/euKe6iZSJUw9RNK+9qb3n8nXymvSn pLt/3gDOhMMDoKUMbBaotIIE6FxkzErEHedRk1uJxLOVkdLOqoH6K+i0ppaRk9HNhwGM 0J9Q== X-Gm-Message-State: AOAM530MUgBi08RLwFQD0/8uo4q97kgQrosCtJ4VC+iRJ5cHWQt0VMxd nb0i9O41kUh8KcGNOIhmyTLqYjM6Sjc= X-Google-Smtp-Source: ABdhPJzqRDKsrlYSwG+cePCYy838jXSpj5O72mQ0oH1dMfqDVWP/9fH+LOGyUX2Wu5YZwttJTiA+Vw== X-Received: by 2002:a1c:9dd0:: with SMTP id g199mr3377146wme.12.1629206607553; Tue, 17 Aug 2021 06:23:27 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id k14sm2461875wri.46.2021.08.17.06.23.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Aug 2021 06:23:26 -0700 (PDT) Message-Id: <371985352680a767dfacb5477aa77e92e04008ee.1629206603.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 17 Aug 2021 13:23:17 +0000 Subject: [PATCH v3 3/8] sparse-index: silently return when cache tree fails Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, newren@gmail.com, matheus.bernardino@usp.br, stolee@gmail.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee If cache_tree_update() returns a non-zero value, then it could not create the cache tree. This is likely due to a path having a merge conflict. Since we are already returning early, let's return silently to avoid making it seem like we failed to write the index at all. If we remove our dependence on the cache tree within convert_to_sparse(), then we could still recover from this scenario and have a sparse index. When constructing the cache-tree extension in convert_to_sparse(), it is possible that we construct a tree object that is new to the object database. Without the WRITE_TREE_MISSING_OK flag, this results in an error that halts our conversion to a sparse index. Add this flag to remove this limitation. Signed-off-by: Derrick Stolee --- sparse-index.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/sparse-index.c b/sparse-index.c index bc5900eae35..b6e90417556 100644 --- a/sparse-index.c +++ b/sparse-index.c @@ -179,10 +179,15 @@ int convert_to_sparse(struct index_state *istate) /* Clear and recompute the cache-tree */ cache_tree_free(&istate->cache_tree); - if (cache_tree_update(istate, 0)) { - warning(_("unable to update cache-tree, staying full")); - return -1; - } + /* + * Silently return if there is a problem with the cache tree update, + * which might just be due to a conflict state in some entry. + * + * This might create new tree objects, so be sure to use + * WRITE_TREE_MISSING_OK. + */ + if (cache_tree_update(istate, WRITE_TREE_MISSING_OK)) + return 0; remove_fsmonitor(istate); From patchwork Tue Aug 17 13:23:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12441455 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 97AF4C43214 for ; Tue, 17 Aug 2021 13:23:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7E44D60F39 for ; Tue, 17 Aug 2021 13:23:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239957AbhHQNYY (ORCPT ); Tue, 17 Aug 2021 09:24:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41000 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240044AbhHQNYM (ORCPT ); Tue, 17 Aug 2021 09:24:12 -0400 Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2FB93C0612AC for ; Tue, 17 Aug 2021 06:23:29 -0700 (PDT) Received: by mail-wr1-x432.google.com with SMTP id r6so28645434wrt.4 for ; Tue, 17 Aug 2021 06:23:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=oc2wvXngXb26lsP7r8c0mk+JZ8u847riiXPKJWHBYNk=; b=oZXViYDK2DFnA/jP4QNO5GkF1hM6PSxqlkplr/LF9F6v2WcXcO+/7q5AhakCRVO4Gf kQNCFLBG9/zDBCV3dtBEYBd0Pax9nSmTwmB/xqs4zpPsLAfdsW83Ebj5sOJs3ygmzftJ xiGJil02Me7xONDQgdxAAwieLgE080NpW37J+fp3sk+WFiYc1FR4peH+HJ4URP3AFm2v T8gdl1YzLq/mPEyTdsKu0BI3u8e3t5Rjnr7Bo/1SexGO+LBzxpVgwC64DfKxswAEoQSg nFtWMv0gD3iB8oJKQoiMJUou0gYGDx99awMHusFj3W+XJ+2PjPuTfoZYxEsoxoAyr+oo iPRQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=oc2wvXngXb26lsP7r8c0mk+JZ8u847riiXPKJWHBYNk=; b=gQArBxBMDkG3hL1HOlLBqsXMImgrYlO7WFWYiFSUunRWLacNT4gXCr3WDPAmqvRtrh btoQcK1TcWi9K5n+/7jaSixzHnnMH8QdjWHWu9QH0bh682HdCWrz+ZdsDONEgoThReny Ku70JjHIsjPZGZPXA4c0T0362mLq/8oTDaRIi+sLAeRUvE9Aq3+09Q1CND1fLQHegZS7 6lK+/IC7z04z2u9Dj6RkfGWyWoBOOSLIhLPzOaZfWuE4U45F84886YwusIpw4FVQxzCr DMHcuDDGxCPS7lCESfaUXFkNWFIK2bHu+2xBNo0txV7s00xJQJP3ZTMDR9hdoVI2ho7X 3Psg== X-Gm-Message-State: AOAM533Q9iUnr1eg2hS80KCdtnGXMHW9UqbwNMU9iYlvJT7tkPkt/pKx qsPsiM/hBFxP+PuHLL6jjWK0tHd1nng= X-Google-Smtp-Source: ABdhPJxwrtWrWSHdS46B0uZvZ5wp3Y8d6bBLFkN0cZhHdaFHJtfY89gnl7ugc8U/PubZozBUrjb3ew== X-Received: by 2002:a5d:5144:: with SMTP id u4mr4189585wrt.30.1629206608355; Tue, 17 Aug 2021 06:23:28 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id y15sm2417392wrw.58.2021.08.17.06.23.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Aug 2021 06:23:27 -0700 (PDT) Message-Id: <10bcadb284e49419f9b4baf75e05f719ec395d98.1629206603.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 17 Aug 2021 13:23:18 +0000 Subject: [PATCH v3 4/8] unpack-trees: fix nested sparse-dir search Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, newren@gmail.com, matheus.bernardino@usp.br, stolee@gmail.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee The iterated search in find_cache_entry() was recently modified to include a loop that searches backwards for a sparse directory entry that matches the given traverse_info and name_entry. However, the string comparison failed to actually concatenate those two strings, so this failed to find a sparse directory when it was not a top-level directory. This caused some errors in rare cases where a 'git checkout' spanned a diff that modified files within the sparse directory entry, but we could not correctly find the entry. Signed-off-by: Derrick Stolee --- unpack-trees.c | 18 +++++++++++++----- 1 file changed, 13 insertions(+), 5 deletions(-) diff --git a/unpack-trees.c b/unpack-trees.c index 5786645f315..df1f4437723 100644 --- a/unpack-trees.c +++ b/unpack-trees.c @@ -1255,9 +1255,10 @@ static int sparse_dir_matches_path(const struct cache_entry *ce, static struct cache_entry *find_cache_entry(struct traverse_info *info, const struct name_entry *p) { - struct cache_entry *ce; + struct cache_entry *ce = NULL; int pos = find_cache_pos(info, p->path, p->pathlen); struct unpack_trees_options *o = info->data; + struct strbuf full_path = STRBUF_INIT; if (0 <= pos) return o->src_index->cache[pos]; @@ -1273,6 +1274,10 @@ static struct cache_entry *find_cache_entry(struct traverse_info *info, if (pos < 0 || pos >= o->src_index->cache_nr) return NULL; + strbuf_addstr(&full_path, info->traverse_path); + strbuf_add(&full_path, p->path, p->pathlen); + strbuf_addch(&full_path, '/'); + /* * Due to lexicographic sorting and sparse directory * entries ending with a trailing slash, our path as a @@ -1283,17 +1288,20 @@ static struct cache_entry *find_cache_entry(struct traverse_info *info, while (pos >= 0) { ce = o->src_index->cache[pos]; - if (strncmp(ce->name, p->path, p->pathlen)) - return NULL; + if (strncmp(ce->name, full_path.buf, full_path.len)) { + ce = NULL; + break; + } if (S_ISSPARSEDIR(ce->ce_mode) && sparse_dir_matches_path(ce, info, p)) - return ce; + break; pos--; } - return NULL; + strbuf_release(&full_path); + return ce; } static void debug_path(struct traverse_info *info) From patchwork Tue Aug 17 13:23:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12441453 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3CAA6C4320A for ; Tue, 17 Aug 2021 13:23:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 23D7A60F39 for ; Tue, 17 Aug 2021 13:23:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239988AbhHQNYX (ORCPT ); Tue, 17 Aug 2021 09:24:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40996 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240042AbhHQNYM (ORCPT ); Tue, 17 Aug 2021 09:24:12 -0400 Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [IPv6:2a00:1450:4864:20::32f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2FE2AC0612AD for ; Tue, 17 Aug 2021 06:23:31 -0700 (PDT) Received: by mail-wm1-x32f.google.com with SMTP id g138so13697419wmg.4 for ; Tue, 17 Aug 2021 06:23:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=5SS0RogwT3BDZ9sDPz8oKtXXDz/pUkaYBLo7L26BzEM=; b=lnggUh389xYW2n+lvMY/O9dnH4SDhL+YvQD96IQGhwKixKkGB4ZvRdhBVJ+DiQTu58 akk5umu01w4f4Db58XswLCqt2GjK2CPlLJinyK+qqpislPdzIBd3WU0UlEkdNQWRvFqH 4iboFpgRPkdOCSAY8Mfj08aadOjDuOuIhzkZ1QpZuFmvxHgyQOj7Gh3bYpqFgsvRDm6K Rq9FIGiIvrcOQVh5M9HZddl5kB/HPYZhuj7cRmza3huVROEhZ4vegzcbIF4f3FmP4wUD 7gqG7h2oWgBh8wEO/awIa3HfnSkaL1JP2nuRjUSW3mHbXouGvt8BYPxoZ24QyXgrs/gn UgJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=5SS0RogwT3BDZ9sDPz8oKtXXDz/pUkaYBLo7L26BzEM=; b=uefJ/y5z0CYYTYkeleCDolERiUjLk9/5/z3m0qIF4osn637eqhsWZOY4LZal3XQWhC jcUL+dBPfcxo1djTBAcxYRCP40wXeOkwPXiS3U/iXzYDFkB0en/jbTmSV7ZMayDtHXZi l97zIliHDlekl2nXPh8pNfNeq63M7chs2X2fH0GOyotKk5jt5dGa6rdNP4xHUYKUfBEG 7sW9d6F10xWB0VwamAi6j4CxqYRHYkZsPOHMVGmVvwmQDMQtsj1krfB3nY1dhYXyy4oc kjeYx58mf30jY+Lz2yDNRhWG4/qFCHTpRSu7p1/0xPOg3+hfgwti1S1Gvq9pxdNth16+ +tCQ== X-Gm-Message-State: AOAM5326sfbLOUcXVDfdCsEKMgHqBbsPtXl67wymbZ59IY2P1xMd9w6m 1ZKmzoPjImuRI2Bygc7LCxNibnE5haw= X-Google-Smtp-Source: ABdhPJzbgVhaOmSnJpXRVrVROHvLD49iVjKYYM3LXpzMgbJc/PkRvLp6S5SqTYxgFv/bCOyfff37Mw== X-Received: by 2002:a1c:a543:: with SMTP id o64mr3246992wme.13.1629206609983; Tue, 17 Aug 2021 06:23:29 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id 19sm2514892wmj.23.2021.08.17.06.23.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Aug 2021 06:23:28 -0700 (PDT) Message-Id: <5d28570c82af19b4bda4253e72ace3760dfe2606.1629206603.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 17 Aug 2021 13:23:19 +0000 Subject: [PATCH v3 5/8] sparse-checkout: create helper methods Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, newren@gmail.com, matheus.bernardino@usp.br, stolee@gmail.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee As we integrate the sparse index into more builtins, we occasionally need to check the sparse-checkout patterns to see if a path is within the sparse-checkout cone. Create some helper methods that help initialize the patterns and check for pattern matching to make this easier. The existing callers of commands like get_sparse_checkout_patterns() use a custom 'struct pattern_list' that is not necessarily the one in the 'struct index_state', so there are not many previous uses that could adopt these helpers. There are just two in builtin/add.c and sparse-index.c that can use path_in_sparse_checkout(). Signed-off-by: Derrick Stolee --- builtin/add.c | 8 ++------ dir.c | 33 +++++++++++++++++++++++++++++++++ dir.h | 6 ++++++ sparse-index.c | 12 +++--------- 4 files changed, 44 insertions(+), 15 deletions(-) diff --git a/builtin/add.c b/builtin/add.c index 17528e8f922..f675bdeae4a 100644 --- a/builtin/add.c +++ b/builtin/add.c @@ -190,8 +190,6 @@ static int refresh(int verbose, const struct pathspec *pathspec) struct string_list only_match_skip_worktree = STRING_LIST_INIT_NODUP; int flags = REFRESH_IGNORE_SKIP_WORKTREE | (verbose ? REFRESH_IN_PORCELAIN : REFRESH_QUIET); - struct pattern_list pl = { 0 }; - int sparse_checkout_enabled = !get_sparse_checkout_patterns(&pl); seen = xcalloc(pathspec->nr, 1); refresh_index(&the_index, flags, pathspec, seen, @@ -199,12 +197,10 @@ static int refresh(int verbose, const struct pathspec *pathspec) for (i = 0; i < pathspec->nr; i++) { if (!seen[i]) { const char *path = pathspec->items[i].original; - int dtype = DT_REG; if (matches_skip_worktree(pathspec, i, &skip_worktree_seen) || - (sparse_checkout_enabled && - !path_matches_pattern_list(path, strlen(path), NULL, - &dtype, &pl, &the_index))) { + (core_apply_sparse_checkout && + path_in_sparse_checkout(path, &the_index) == NOT_MATCHED)) { string_list_append(&only_match_skip_worktree, pathspec->items[i].original); } else { diff --git a/dir.c b/dir.c index 03c4d212672..6fd4f2e0f27 100644 --- a/dir.c +++ b/dir.c @@ -1439,6 +1439,39 @@ done: return result; } +int init_sparse_checkout_patterns(struct index_state *istate) +{ + if (!core_apply_sparse_checkout || + istate->sparse_checkout_patterns) + return 0; + + CALLOC_ARRAY(istate->sparse_checkout_patterns, 1); + + if (get_sparse_checkout_patterns(istate->sparse_checkout_patterns) < 0) { + FREE_AND_NULL(istate->sparse_checkout_patterns); + return -1; + } + + return 0; +} + +int path_in_sparse_checkout(const char *path, + struct index_state *istate) +{ + const char *base; + int dtype = DT_REG; + init_sparse_checkout_patterns(istate); + + if (!istate->sparse_checkout_patterns) + return MATCHED; + + base = strrchr(path, '/'); + return path_matches_pattern_list(path, strlen(path), base ? base + 1 : path, + &dtype, + istate->sparse_checkout_patterns, + istate) > 0; +} + static struct path_pattern *last_matching_pattern_from_lists( struct dir_struct *dir, struct index_state *istate, const char *pathname, int pathlen, diff --git a/dir.h b/dir.h index b3e1a54a971..b899ee43d81 100644 --- a/dir.h +++ b/dir.h @@ -394,6 +394,12 @@ enum pattern_match_result path_matches_pattern_list(const char *pathname, const char *basename, int *dtype, struct pattern_list *pl, struct index_state *istate); + +int init_sparse_checkout_patterns(struct index_state *state); + +int path_in_sparse_checkout(const char *path, + struct index_state *istate); + struct dir_entry *dir_add_ignored(struct dir_struct *dir, struct index_state *istate, const char *pathname, int len); diff --git a/sparse-index.c b/sparse-index.c index b6e90417556..2efc9fd4910 100644 --- a/sparse-index.c +++ b/sparse-index.c @@ -34,17 +34,14 @@ static int convert_to_sparse_rec(struct index_state *istate, int i, can_convert = 1; int start_converted = num_converted; enum pattern_match_result match; - int dtype = DT_UNKNOWN; struct strbuf child_path = STRBUF_INIT; - struct pattern_list *pl = istate->sparse_checkout_patterns; /* * Is the current path outside of the sparse cone? * Then check if the region can be replaced by a sparse * directory entry (everything is sparse and merged). */ - match = path_matches_pattern_list(ct_path, ct_pathlen, - NULL, &dtype, pl, istate); + match = path_in_sparse_checkout(ct_path, istate); if (match != NOT_MATCHED) can_convert = 0; @@ -153,11 +150,8 @@ int convert_to_sparse(struct index_state *istate) if (!istate->repo->settings.sparse_index) return 0; - if (!istate->sparse_checkout_patterns) { - istate->sparse_checkout_patterns = xcalloc(1, sizeof(struct pattern_list)); - if (get_sparse_checkout_patterns(istate->sparse_checkout_patterns) < 0) - return 0; - } + if (init_sparse_checkout_patterns(istate) < 0) + return 0; /* * We need cone-mode patterns to use sparse-index. If a user edits From patchwork Tue Aug 17 13:23:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12441461 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BB0B4C4338F for ; Tue, 17 Aug 2021 13:23:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A2C4D60F35 for ; Tue, 17 Aug 2021 13:23:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240061AbhHQNY1 (ORCPT ); Tue, 17 Aug 2021 09:24:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40984 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240047AbhHQNYM (ORCPT ); Tue, 17 Aug 2021 09:24:12 -0400 Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com [IPv6:2a00:1450:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A7977C0612AF for ; Tue, 17 Aug 2021 06:23:32 -0700 (PDT) Received: by mail-wr1-x433.google.com with SMTP id q10so28643294wro.2 for ; Tue, 17 Aug 2021 06:23:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=dgfH8FHC7lRtH5aYzMRWtNR/tzYMsPRc7peitaicWXk=; b=CiuygpsjeFiEr1T2F6uKjJnFZ1Gx0zfcGgdRRq2xxg4/AVh5kxbUaev7TmDI2Z2Ind jUW3Bv09LoaoKMHEhWN/XMAmHFxKr/1yL3uaG3h48USJreSN19cAkroaYxr/ltDOPTho LOomfzq9ozfZMcmFKIHaNSzEc1AjlaFHPg/oF6PNAi2oURmmTScIai16R9vxSwqV9xBL Tc+eKrGjeMxTYO2tPiyU+h/K+PeEK8rWZhmpRYO2qCVBM05GacsQmoBSMxf1Y3s3EWKU vjpQYoejLDR0W7EvyJQbw+j0CGUDqbypzEmIvniXbWyqvxWu5oVZOOkHG7kk4yBVdtQa kUtA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=dgfH8FHC7lRtH5aYzMRWtNR/tzYMsPRc7peitaicWXk=; b=VsHrWg0m1sTd1KLJnAWqCCa4436jaiNXfeilF5v+l7VSlXjNC+QKx70oNtAf50k44x sdXnyaNBPNzk9iQXLxtWHKwLa96ccErtaoG9Tn9PppZYc24zQBlPte+jVwhR0zrNkaM/ gneIDbJtwQ4QnKcCpYWQDIdfziXHXLE+oQUDtbjxe7S3rzEL8nfEErsPTpztv8oCLxDA wJ/1GmKZJAwusIlZoxCg79iV8XI6QWTtLBr/apsHDyrDXgLaJdXuqzisLyJo7PZZEknR gRyHGWzuDkoGZVj4Uqq5uBJe0H3Ik+ANDIAzbEyYfDXPA2JfdqU8I0TchBIL9N8kiMre iWgw== X-Gm-Message-State: AOAM533/0+hyzK01q75j1EVWpOl0Jj8sPzekv9ItKxdsIdybU01eHEc1 nMi7kcPjVuxLahnosolgV4b50uEJ0L0= X-Google-Smtp-Source: ABdhPJwtuMTCiUPSLbesZ8ww/luIg+JGA/+yelhkUjzbwyTFCcqFhLd7yiNCl75ntg5T+w1ypritxw== X-Received: by 2002:a5d:574d:: with SMTP id q13mr4149550wrw.425.1629206611275; Tue, 17 Aug 2021 06:23:31 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id q75sm2322685wme.40.2021.08.17.06.23.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Aug 2021 06:23:30 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Tue, 17 Aug 2021 13:23:20 +0000 Subject: [PATCH v3 6/8] attr: be careful about sparse directories Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, newren@gmail.com, matheus.bernardino@usp.br, stolee@gmail.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Signed-off-by: Derrick Stolee --- attr.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/attr.c b/attr.c index d029e681f28..a1009f78029 100644 --- a/attr.c +++ b/attr.c @@ -14,6 +14,7 @@ #include "utf8.h" #include "quote.h" #include "thread-utils.h" +#include "dir.h" const char git_attr__true[] = "(builtin)true"; const char git_attr__false[] = "\0(builtin)false"; @@ -744,6 +745,19 @@ static struct attr_stack *read_attr_from_index(struct index_state *istate, if (!istate) return NULL; + /* + * In the case of cone-mode sparse-checkout, getting the + * .gitattributes file from a directory is meaningless: all + * contained paths will be sparse if the .gitattributes is also + * sparse. In the case of a sparse index, it is critical that we + * don't go looking for one as it will expand the index. + */ + init_sparse_checkout_patterns(istate); + if (istate->sparse_checkout_patterns && + istate->sparse_checkout_patterns->use_cone_patterns && + path_in_sparse_checkout(path, istate) == NOT_MATCHED) + return NULL; + buf = read_blob_data_from_index(istate, path, NULL); if (!buf) return NULL; From patchwork Tue Aug 17 13:23:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12441459 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 54E39C4338F for ; Tue, 17 Aug 2021 13:23:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3B85A60FBF for ; Tue, 17 Aug 2021 13:23:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240046AbhHQNYZ (ORCPT ); Tue, 17 Aug 2021 09:24:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40998 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240045AbhHQNYM (ORCPT ); Tue, 17 Aug 2021 09:24:12 -0400 Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [IPv6:2a00:1450:4864:20::335]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B5F0FC06124A for ; Tue, 17 Aug 2021 06:23:33 -0700 (PDT) Received: by mail-wm1-x335.google.com with SMTP id u1so13753598wmm.0 for ; Tue, 17 Aug 2021 06:23:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=siV5lRTXPuRg8PtQx00TLBS2bGCepL9AAAoIBksPutM=; b=r7qG2+jCE6fi5NrT28r21iQx9BO623i5vg4QEGlbY/SJlOGIp+kA44ld8SHf2sq0x8 UpehA5MOBxqrxuPPaCfAV67msIU46culUkx25Mb+MS5z3bnIQba+Xr2M/wwTMQQVzszp 2aO8/y0A33VVGlKd0cpLDKB9T7sjO5Avqv+rI5AjlP3cVmuafHoAHq2UElacrKjFsrW6 J/fiQBljZXSBVgxUPoDTinSPy6d4qUPWAh55bQQWrcG4UxsgkibvVjpzHwkhsllsUyp4 hMw+pbGZbdCgp6fjU6lKtUd2v1Zo/fPkrv/+ba4lw/CnDqSqdDkvSD/kE0gdMoaOhqBr JYLQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=siV5lRTXPuRg8PtQx00TLBS2bGCepL9AAAoIBksPutM=; b=gc9JYnfQIrJ3lQBIvZSYA0q4ZCtSKNKKT5oOatNEr+l0qfP5IswLgTw1ZnQmPXM1pC NTJr6sqh0JtGTXY9RYT714AUUAUJp/ZW0x5S5A07pNWo/QcIvgm7VK85QGBQJjmienuv FGT8QqKD/PsyNNvSNUoR/Cosj8rPtDxt4SLn8xIluy6XWNv7ruLmd2rts70WKoGMX1A/ Lw1aSNR4xHjazZCl+t+4W+qGgIT7hyczjwWt9D4EcRaLpco5B/PTWSkVuoK1HkngySi8 5h2Cwv+54oma1X3Ge27/trJXTWmfg+T6lYalMwq0a8mIswrk4EdYyOf/PoNK7uXyXBoK 7+Gg== X-Gm-Message-State: AOAM531XqRpd+2e5n4ptCXykweyKht767lfOt1BwZ0f/CC26VG9c+q1O ull5neu6FNqUmAdIVD5jCPyv1G8e3+A= X-Google-Smtp-Source: ABdhPJw97CTy86xu6BKDNOUf4uhgK9Ef8fXjRXn49nU73stKiHvYYnromqvcyofWiojxuAHqehMyYQ== X-Received: by 2002:a1c:4c05:: with SMTP id z5mr380985wmf.145.1629206612344; Tue, 17 Aug 2021 06:23:32 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id e3sm2591408wrv.65.2021.08.17.06.23.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Aug 2021 06:23:31 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Tue, 17 Aug 2021 13:23:21 +0000 Subject: [PATCH v3 7/8] sparse-index: add SPARSE_INDEX_IGNORE_CONFIG flag Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, newren@gmail.com, matheus.bernardino@usp.br, stolee@gmail.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee The convert_to_sparse() method checks for the GIT_TEST_SPARSE_INDEX environment variable or the "index.sparse" config setting before converting the index to a sparse one. This is for ease of use since all current consumers are preparing to compress the index before writing it to disk. If these settings are not enabled, then convert_to_sparse() silently returns without doing anything. We will add a consumer in the next change that wants to use the sparse index as an in-memory data structure, regardless of whether the on-disk format should be sparse. To that end, create the SPARSE_INDEX_IGNORE_CONFIG flag that will skip these config checks when enabled. All current consumers are modified to pass '0' in the new 'flags' parameter. Signed-off-by: Derrick Stolee --- read-cache.c | 4 ++-- sparse-index.c | 30 ++++++++++++++++-------------- sparse-index.h | 3 ++- 3 files changed, 20 insertions(+), 17 deletions(-) diff --git a/read-cache.c b/read-cache.c index 9048ef9e905..f5d4385c408 100644 --- a/read-cache.c +++ b/read-cache.c @@ -3069,7 +3069,7 @@ static int do_write_locked_index(struct index_state *istate, struct lock_file *l int ret; int was_full = !istate->sparse_index; - ret = convert_to_sparse(istate); + ret = convert_to_sparse(istate, 0); if (ret) { warning(_("failed to convert to a sparse-index")); @@ -3182,7 +3182,7 @@ static int write_shared_index(struct index_state *istate, int ret, was_full = !istate->sparse_index; move_cache_to_base_index(istate); - convert_to_sparse(istate); + convert_to_sparse(istate, 0); trace2_region_enter_printf("index", "shared/do_write_index", the_repository, "%s", get_tempfile_path(*temp)); diff --git a/sparse-index.c b/sparse-index.c index 2efc9fd4910..532fd11787e 100644 --- a/sparse-index.c +++ b/sparse-index.c @@ -124,7 +124,7 @@ static int index_has_unmerged_entries(struct index_state *istate) return 0; } -int convert_to_sparse(struct index_state *istate) +int convert_to_sparse(struct index_state *istate, int flags) { int test_env; @@ -135,20 +135,22 @@ int convert_to_sparse(struct index_state *istate) if (!istate->repo) istate->repo = the_repository; - /* - * The GIT_TEST_SPARSE_INDEX environment variable triggers the - * index.sparse config variable to be on. - */ - test_env = git_env_bool("GIT_TEST_SPARSE_INDEX", -1); - if (test_env >= 0) - set_sparse_index_config(istate->repo, test_env); + if (!(flags & SPARSE_INDEX_IGNORE_CONFIG)) { + /* + * The GIT_TEST_SPARSE_INDEX environment variable triggers the + * index.sparse config variable to be on. + */ + test_env = git_env_bool("GIT_TEST_SPARSE_INDEX", -1); + if (test_env >= 0) + set_sparse_index_config(istate->repo, test_env); - /* - * Only convert to sparse if index.sparse is set. - */ - prepare_repo_settings(istate->repo); - if (!istate->repo->settings.sparse_index) - return 0; + /* + * Only convert to sparse if index.sparse is set. + */ + prepare_repo_settings(istate->repo); + if (!istate->repo->settings.sparse_index) + return 0; + } if (init_sparse_checkout_patterns(istate) < 0) return 0; diff --git a/sparse-index.h b/sparse-index.h index 1115a0d7dd9..475f4f0f8da 100644 --- a/sparse-index.h +++ b/sparse-index.h @@ -2,7 +2,8 @@ #define SPARSE_INDEX_H__ struct index_state; -int convert_to_sparse(struct index_state *istate); +#define SPARSE_INDEX_IGNORE_CONFIG (1 << 0) +int convert_to_sparse(struct index_state *istate, int flags); /* * Some places in the codebase expect to search for a specific path. From patchwork Tue Aug 17 13:23:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12441457 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 29FBCC432BE for ; Tue, 17 Aug 2021 13:23:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 07EDE60F39 for ; Tue, 17 Aug 2021 13:23:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240039AbhHQNYY (ORCPT ); Tue, 17 Aug 2021 09:24:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40994 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240046AbhHQNYM (ORCPT ); Tue, 17 Aug 2021 09:24:12 -0400 Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com [IPv6:2a00:1450:4864:20::336]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B18F6C06124C for ; Tue, 17 Aug 2021 06:23:34 -0700 (PDT) Received: by mail-wm1-x336.google.com with SMTP id 79-20020a1c0452000000b002e6cf79e572so1920162wme.1 for ; Tue, 17 Aug 2021 06:23:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=pcwjy+V2SEhtxyn0vJUrmLOj7GCOWJeEG7NdwuPhcdA=; b=iVh1SOJyBR27b7lBNwDLsLucGIa9mgt+0EBCei0KLQOWX49gVeOmseAVuc4TxPt5Fl LYNLf2UpsFrIpH996tpXPu3DIvqTZIiW0fm31tMt0DFw7hMsGV/6r2trI0NWHAyfhBU0 rDPtrdWicSLUzq441QXw7JYOAFlVrHmAPclB1syv+ywUTiQUyWCIYMyJdDYYoJMP6S9j jcXCfb50vr80hE6rrS+a6crOs/C8o/Ol1UyC2B1zvH65hAkxk+WX476bUwig8Pd1Gm0i 6xV0oCzotcM/IyxSOao3nKQ0KEe5O8Ny16C574K6rWyXZ0tTKu57Z7qm2asoRFCup/p5 1CWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=pcwjy+V2SEhtxyn0vJUrmLOj7GCOWJeEG7NdwuPhcdA=; b=b35odvXHvQLi0SgSZKgN/6vlERvvraYs1Ff08VQYbw2svgZvH7ZX+99rKzN2ussY5t xv1uIiVNgHdzYwtisa5j6T7+hEJN30abtsih3DloIIU70tWaS55ODwWHJ2HpRQK8rqNm gj/Mw0myQLk6EOZK7tXsF1V2ONB6DDqLXd2mYZ/c0kplu2KRRQk9x7tAt7DtdeHeWmsM ZGzUIeGl2CO9xUGmE/ZG9Jf4IX7NteZqOYBK0lHQbK9Tr8RuUpnix3O03HmnXHLq+yTh 2D0qBE43RApmsvt/fSbBRhEymCMTSp3k32UCP3JG1AbEpPwGskJpGN75oYJ1H6NG6QvD gykg== X-Gm-Message-State: AOAM5325uLHRk6i6V3tBWd8GjQvZOS/SiyZpfGE82K9DEMTqn9rdBODh z+03hmkck9uLrL1gsuEVETfsEtsC0Oo= X-Google-Smtp-Source: ABdhPJw5ybeWrQ7x0q0meX5UM5bM8o0OgNHCmaax3mWoB2EbWwcz7oiSUR4hC+NI84lcMeKP327CLg== X-Received: by 2002:a05:600c:1ca7:: with SMTP id k39mr3476317wms.115.1629206613291; Tue, 17 Aug 2021 06:23:33 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id t1sm2186203wma.25.2021.08.17.06.23.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Aug 2021 06:23:32 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Tue, 17 Aug 2021 13:23:22 +0000 Subject: [PATCH v3 8/8] sparse-checkout: clear tracked sparse dirs Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, newren@gmail.com, matheus.bernardino@usp.br, stolee@gmail.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee When changing the scope of a sparse-checkout using cone mode, we might have some tracked directories go out of scope. The current logic removes the tracked files from within those directories, but leaves the ignored files within those directories. This is a bit unexpected to users who have given input to Git saying they don't need those directories anymore. This is something that is new to the cone mode pattern type: the user has explicitly said "I want these directories and _not_ those directories." The typical sparse-checkout patterns more generally apply to "I want files with with these patterns" so it is natural to leave ignored files as they are. This focus on directories in cone mode provides us an opportunity to change the behavior. Leaving these ignored files in the sparse directories makes it impossible to gain performance benefits in the sparse index. When we track into these directories, we need to know if the files are ignored or not, which might depend on the _tracked_ .gitignore file(s) within the sparse directory. This depends on the indexed version of the file, so the sparse directory must be expanded. By deleting the sparse directories when changing scope (or running 'git sparse-checkout reapply') we regain these performance benefits as if the repository was in a clean state. Since these ignored files are frequently build output or helper files from IDEs, the users should not need the files now that the tracked files are removed. If the tracked files reappear, then they will have newer timestamps than the build artifacts, so the artifacts will need to be regenerated anyway. Use the sparse-index as a data structure in order to find the sparse directories that can be safely deleted. Re-expand the index to a full one if it was full before. Signed-off-by: Derrick Stolee --- Documentation/git-sparse-checkout.txt | 8 +++ builtin/sparse-checkout.c | 95 +++++++++++++++++++++++++++ t/t1091-sparse-checkout-builtin.sh | 59 +++++++++++++++++ 3 files changed, 162 insertions(+) diff --git a/Documentation/git-sparse-checkout.txt b/Documentation/git-sparse-checkout.txt index fdcf43f87cb..f9022b9d555 100644 --- a/Documentation/git-sparse-checkout.txt +++ b/Documentation/git-sparse-checkout.txt @@ -210,6 +210,14 @@ case-insensitive check. This corrects for case mismatched filenames in the 'git sparse-checkout set' command to reflect the expected cone in the working directory. +The cone mode sparse-checkout patterns will also remove ignored files that +are not within the sparse-checkout definition. This is important behavior +to preserve the performance of the sparse index, but also matches that +cone mode patterns care about directories, not files. If there exist files +that are untracked and not ignored, then Git will not delete files within +that directory other than the tracked files that are now out of scope. +These files should be removed manually to ensure Git can behave optimally. + SUBMODULES ---------- diff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c index 8ba9f13787b..b06c8f885ac 100644 --- a/builtin/sparse-checkout.c +++ b/builtin/sparse-checkout.c @@ -100,6 +100,99 @@ static int sparse_checkout_list(int argc, const char **argv) return 0; } +static void clean_tracked_sparse_directories(struct repository *r) +{ + int i, was_full = 0; + struct strbuf path = STRBUF_INIT; + size_t pathlen; + struct string_list_item *item; + struct string_list sparse_dirs = STRING_LIST_INIT_DUP; + + /* + * If we are not using cone mode patterns, then we cannot + * delete directories outside of the sparse cone. + */ + if (!r || !r->index || !r->worktree) + return; + init_sparse_checkout_patterns(r->index); + if (!r->index->sparse_checkout_patterns || + !r->index->sparse_checkout_patterns->use_cone_patterns) + return; + + /* + * Use the sparse index as a data structure to assist finding + * directories that are safe to delete. This conversion to a + * sparse index will not delete directories that contain + * conflicted entries or submodules. + */ + if (!r->index->sparse_index) { + /* + * If something, such as a merge conflict or other concern, + * prevents us from converting to a sparse index, then do + * not try deleting files. + */ + if (convert_to_sparse(r->index, SPARSE_INDEX_IGNORE_CONFIG)) + return; + was_full = 1; + } + + strbuf_addstr(&path, r->worktree); + strbuf_complete(&path, '/'); + pathlen = path.len; + + /* + * Collect directories that have gone out of scope but also + * exist on disk, so there is some work to be done. We need to + * store the entries in a list before exploring, since that might + * expand the sparse-index again. + */ + for (i = 0; i < r->index->cache_nr; i++) { + struct cache_entry *ce = r->index->cache[i]; + + if (S_ISSPARSEDIR(ce->ce_mode) && + repo_file_exists(r, ce->name)) + string_list_append(&sparse_dirs, ce->name); + } + + for_each_string_list_item(item, &sparse_dirs) { + struct dir_struct dir = DIR_INIT; + struct pathspec p = { 0 }; + struct strvec s = STRVEC_INIT; + + strbuf_setlen(&path, pathlen); + strbuf_addstr(&path, item->string); + + dir.flags |= DIR_SHOW_IGNORED_TOO; + + setup_standard_excludes(&dir); + strvec_push(&s, path.buf); + + parse_pathspec(&p, PATHSPEC_GLOB, 0, NULL, s.v); + fill_directory(&dir, r->index, &p); + + if (dir.nr) { + warning(_("directory '%s' contains untracked files," + " but is not in the sparse-checkout cone"), + item->string); + } else if (remove_dir_recursively(&path, 0)) { + /* + * Removal is "best effort". If something blocks + * the deletion, then continue with a warning. + */ + warning(_("failed to remove directory '%s'"), + item->string); + } + + dir_clear(&dir); + } + + string_list_clear(&sparse_dirs, 0); + strbuf_release(&path); + + if (was_full) + ensure_full_index(r->index); +} + static int update_working_directory(struct pattern_list *pl) { enum update_sparsity_result result; @@ -141,6 +234,8 @@ static int update_working_directory(struct pattern_list *pl) else rollback_lock_file(&lock_file); + clean_tracked_sparse_directories(r); + r->index->sparse_checkout_patterns = NULL; return result; } diff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh index 38fc8340f5c..71236981e64 100755 --- a/t/t1091-sparse-checkout-builtin.sh +++ b/t/t1091-sparse-checkout-builtin.sh @@ -642,4 +642,63 @@ test_expect_success MINGW 'cone mode replaces backslashes with slashes' ' check_files repo/deep a deeper1 ' +test_expect_success 'cone mode clears ignored subdirectories' ' + rm repo/.git/info/sparse-checkout && + + git -C repo sparse-checkout init --cone && + git -C repo sparse-checkout set deep/deeper1 && + + cat >repo/.gitignore <<-\EOF && + obj/ + *.o + EOF + + git -C repo add .gitignore && + git -C repo commit -m ".gitignore" && + + mkdir -p repo/obj repo/folder1/obj repo/deep/deeper2/obj && + for file in folder1/obj/a obj/a folder1/file.o folder1.o \ + deep/deeper2/obj/a deep/deeper2/file.o file.o + do + echo ignored >repo/$file || return 1 + done && + + git -C repo status --porcelain=v2 >out && + test_must_be_empty out && + + git -C repo sparse-checkout reapply && + test_path_is_missing repo/folder1 && + test_path_is_missing repo/deep/deeper2 && + test_path_is_dir repo/obj && + test_path_is_file repo/file.o && + + git -C repo status --porcelain=v2 >out && + test_must_be_empty out && + + git -C repo sparse-checkout set deep/deeper2 && + test_path_is_missing repo/deep/deeper1 && + test_path_is_dir repo/deep/deeper2 && + test_path_is_dir repo/obj && + test_path_is_file repo/file.o && + + >repo/deep/deeper2/ignored.o && + >repo/deep/deeper2/untracked && + + # When an untracked file is in the way, all untracked files + # (even ignored files) are preserved. + git -C repo sparse-checkout set folder1 2>err && + grep "contains untracked files" err && + test_path_is_file repo/deep/deeper2/ignored.o && + test_path_is_file repo/deep/deeper2/untracked && + + # The rest of the cone matches expectation + test_path_is_missing repo/deep/deeper1 && + test_path_is_dir repo/obj && + test_path_is_file repo/file.o && + + git -C repo status --porcelain=v2 >out && + echo "? deep/deeper2/untracked" >expect && + test_cmp expect out +' + test_done