From patchwork Tue Aug 10 19:50:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12429395 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6B805C4320A for ; Tue, 10 Aug 2021 19:50:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3A81861051 for ; Tue, 10 Aug 2021 19:50:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232135AbhHJTuk (ORCPT ); Tue, 10 Aug 2021 15:50:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54456 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231654AbhHJTuj (ORCPT ); Tue, 10 Aug 2021 15:50:39 -0400 Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com [IPv6:2a00:1450:4864:20::332]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3DD34C0613D3 for ; Tue, 10 Aug 2021 12:50:17 -0700 (PDT) Received: by mail-wm1-x332.google.com with SMTP id w21-20020a7bc1150000b02902e69ba66ce6so321331wmi.1 for ; Tue, 10 Aug 2021 12:50:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=h0jP/WQTQwK/dNNHt3JH4FuVCZwq3HAW8lG1yL7NHbc=; b=S/ezSGj7Y932A600ls7huY7zxKf7D+F4Exx386Y+v8ueqtuLY16Q9okuBjGrW0BCSH A5ijDCQ2SiTsVbIAw82hS+PpF6Oqeanxzz8f57Hxwv4sYJ+Dw6LmWdORA0sAS8mLZU8E qggmXAfGnm85kFO10zFBSg9mDpBqNg4yNFW3BkHEEBd215/d1ThyEZuvTSNS8JKBV/sD xjfhyZqsTtpBrvL6vyqJMCW9J2mg4vlTHyQPnP/zSoV4iP0CPJUm/t88gNXrqMkZj3Bp XAJaWTol+E+MRxrpKt02N299OEOxvwyCNTA5qI2FFNHKHVN2UxiAwvsCJoz4WgUOGTLa TH/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=h0jP/WQTQwK/dNNHt3JH4FuVCZwq3HAW8lG1yL7NHbc=; b=avRdIs9r6XZLCwwHOo32/x1a8gBoyrBnIP/jdd91fiPPINWgbUeKZIeLY/lovg/dJp tQxjPXr0km3YM8vChvDBFqbbufthBxoaWgfH8fEDDkCFUMZqfuQhQCG7AC7nWumTOzyJ gk2ZppHgK4PvdiAlt+wCvOys4+pe+c+vc2DgLdkYXcGcpxJIjaYeLR6o6QzFrc0Eu/Cq KCcDTfLauIst2cNIux7CrJbZ3dtwBaSWkTVxaG+l/4gbOZana7wFaMktkWGoy884FtdZ V9tTgBkz7sdfFIlaA8b1vGVWh+3b8wJm1MKd5LI+VnubA+9uFwD0ptjsnwVuSfGjzNoa C8LA== X-Gm-Message-State: AOAM53365CLEgUQNXi9lmwRk/reyOD9ubSYA5ZGXXb82gX1i2xdzSJWB IWYlYGrpBnx8keBEBJ34Z+/18FYoXUo= X-Google-Smtp-Source: ABdhPJy+ZwjLCjrkiSP45lPy1Uftgyhrt5ssaORMmqmhcpQMb9eYpb1Sq5oVfkNqcpXQD34YfwoC8A== X-Received: by 2002:a7b:c3d6:: with SMTP id t22mr4130920wmj.117.1628625015872; Tue, 10 Aug 2021 12:50:15 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id l38sm3404491wmp.15.2021.08.10.12.50.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Aug 2021 12:50:15 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Tue, 10 Aug 2021 19:50:06 +0000 Subject: [PATCH v2 1/8] t7519: rewrite sparse index test Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, newren@gmail.com, matheus.bernardino@usp.br, stolee@gmail.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee The sparse index is tested with the FS Monitor hook and extension since f8fe49e (fsmonitor: integrate with sparse index, 2021-07-14). This test was very fragile because it shared an index across sparse and non-sparse behavior. Since that expansion and contraction could cause the index to lose its FS Monitor bitmap and token, behavior is fragile to changes in 'git sparse-checkout set'. Rewrite the test to use two clones of the original repo: full and sparse. This allows us to also keep the test files (actual, expect, trace2.txt) out of the repos we are testing with 'git status'. Signed-off-by: Derrick Stolee --- t/t7519-status-fsmonitor.sh | 38 ++++++++++++++++++++----------------- 1 file changed, 21 insertions(+), 17 deletions(-) diff --git a/t/t7519-status-fsmonitor.sh b/t/t7519-status-fsmonitor.sh index deea88d4431..6f2cf306f66 100755 --- a/t/t7519-status-fsmonitor.sh +++ b/t/t7519-status-fsmonitor.sh @@ -389,43 +389,47 @@ test_expect_success 'status succeeds after staging/unstaging' ' # If "!" is supplied, then we verify that we do not call ensure_full_index # during a call to 'git status'. Otherwise, we verify that we _do_ call it. check_sparse_index_behavior () { - git status --porcelain=v2 >expect && - git sparse-checkout init --cone --sparse-index && - git sparse-checkout set dir1 dir2 && + git -C full status --porcelain=v2 >expect && GIT_TRACE2_EVENT="$(pwd)/trace2.txt" GIT_TRACE2_EVENT_NESTING=10 \ - git status --porcelain=v2 >actual && + git -C sparse status --porcelain=v2 >actual && test_region $1 index ensure_full_index trace2.txt && test_region fsm_hook query trace2.txt && test_cmp expect actual && - rm trace2.txt && - git sparse-checkout disable + rm trace2.txt } test_expect_success 'status succeeds with sparse index' ' - git reset --hard && + git clone . full && + git clone . sparse && + git -C sparse sparse-checkout init --cone --sparse-index && + git -C sparse sparse-checkout set dir1 dir2 && - test_config core.fsmonitor "$TEST_DIRECTORY/t7519/fsmonitor-all" && - check_sparse_index_behavior ! && - - write_script .git/hooks/fsmonitor-test<<-\EOF && + write_script .git/hooks/fsmonitor-test <<-\EOF && printf "last_update_token\0" EOF - git config core.fsmonitor .git/hooks/fsmonitor-test && + git -C full config core.fsmonitor ../.git/hooks/fsmonitor-test && + git -C sparse config core.fsmonitor ../.git/hooks/fsmonitor-test && check_sparse_index_behavior ! && - write_script .git/hooks/fsmonitor-test<<-\EOF && + write_script .git/hooks/fsmonitor-test <<-\EOF && printf "last_update_token\0" printf "dir1/modified\0" EOF check_sparse_index_behavior ! && - cp -r dir1 dir1a && - git add dir1a && - git commit -m "add dir1a" && + git -C sparse sparse-checkout add dir1a && + + for repo in full sparse + do + cp -r $repo/dir1 $repo/dir1a && + git -C $repo add dir1a && + git -C $repo commit -m "add dir1a" || return 1 + done && + git -C sparse sparse-checkout set dir1 dir2 && # This one modifies outside the sparse-checkout definition # and hence we expect to expand the sparse-index. - write_script .git/hooks/fsmonitor-test<<-\EOF && + write_script .git/hooks/fsmonitor-test <<-\EOF && printf "last_update_token\0" printf "dir1a/modified\0" EOF From patchwork Tue Aug 10 19:50:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12429397 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4E425C432BE for ; Tue, 10 Aug 2021 19:50:20 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3086B60F94 for ; Tue, 10 Aug 2021 19:50:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232412AbhHJTul (ORCPT ); Tue, 10 Aug 2021 15:50:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54460 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232085AbhHJTuk (ORCPT ); Tue, 10 Aug 2021 15:50:40 -0400 Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BD8CDC0613C1 for ; Tue, 10 Aug 2021 12:50:17 -0700 (PDT) Received: by mail-wr1-x429.google.com with SMTP id b11so11705523wrx.6 for ; Tue, 10 Aug 2021 12:50:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=xF4Ml5ovwsHr66snlzRz3TrGLFSkIqW9omB3/KckS/A=; b=rM+Vwq34lpV37sHsn8XMqoVifrSvdmoPy738QC9nJn7nhMRjM7zD3jFqxR+zBmrHZK JeAfmTRt2S5/bN6RcFUz53u0itMqV6n8kjmpkztB98aItvRi+QguSNGjvKutyqMBZ7yK jHMFxgpuzk7YGH7jooP0l+5qcoC3F1OB0skxxy9J0AepI1Grksm1HKqeZytENOrXVzYO vlmDGxwFiUWmz+hXhzCRAsBDafIBJsFnV3XSbbySN3ofI/aHJZMuj7UyQfN0/Sd2Kbq1 0TRJLAQwKQZjrrQ7dsHrzg0t7TjAckd0uZQcjp/hbObCgiCh4DDIW4a2AUrgJccoEc3y Zhrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=xF4Ml5ovwsHr66snlzRz3TrGLFSkIqW9omB3/KckS/A=; b=njb9sk5BM1tBiiVljbDvAaK9GavJ8WKbvIHrO+fguj26SYnlrViaYFT1vDf1Xe8pxC KYczxI8h1cE6Agzs83TsQbLv2lRI9L+Qt/6NBdNS6VU4ccVRoAoMzdXyzlENg4oTvFdk aHhypqVQciqskyPSd6opWDk9K/2oxBLFrMnPqvczySEIERPJojq2s6yTsf72vgEeab1b onF/rFfudRd0gZZxIL6WszlFFG7XdKvZvlEjqrO0mWBzLmZcShbPZAS0DezXDU461Ab3 l/yvi+8OW48fnbOVkxIRHF2ydiP54cK8qG9AR1h4x26wws7ljyqqAnhb/nT0MJzPYHSr IcVQ== X-Gm-Message-State: AOAM532rio4M+C+WCQ3eOrL8el7AI9KpwKXYt4TCHajBOiNDRUrIik7f xh4Lee2+YnteWRBLAL3jUj+9juT/CYU= X-Google-Smtp-Source: ABdhPJy9Cr3lwEIt6ED1Nq3C5Qo0WicpCpZnNi/YiiqLHEv8gKRHrPntrCvQxzJ8s1EJBVxIfrUMjw== X-Received: by 2002:a5d:4d8e:: with SMTP id b14mr2605655wru.422.1628625016450; Tue, 10 Aug 2021 12:50:16 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id i14sm19093561wmq.40.2021.08.10.12.50.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Aug 2021 12:50:16 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Tue, 10 Aug 2021 19:50:07 +0000 Subject: [PATCH v2 2/8] sparse-index: silently return when not using cone-mode patterns Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, newren@gmail.com, matheus.bernardino@usp.br, stolee@gmail.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee While the sparse-index is only enabled when core.sparseCheckoutCone is also enabled, it is possible for the user to modify the sparse-checkout file manually in a way that does not match cone-mode patterns. In this case, we should refuse to convert an index into a sparse index, since the sparse_checkout_patterns will not be initialized with recursive and parent path hashsets. Also silently return if there are no cache entries, which is a simple case: there are no paths to make sparse! Signed-off-by: Derrick Stolee --- sparse-index.c | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/sparse-index.c b/sparse-index.c index c6b4feec413..bc5900eae35 100644 --- a/sparse-index.c +++ b/sparse-index.c @@ -130,7 +130,8 @@ static int index_has_unmerged_entries(struct index_state *istate) int convert_to_sparse(struct index_state *istate) { int test_env; - if (istate->split_index || istate->sparse_index || + + if (istate->split_index || istate->sparse_index || !istate->cache_nr || !core_apply_sparse_checkout || !core_sparse_checkout_cone) return 0; @@ -158,10 +159,16 @@ int convert_to_sparse(struct index_state *istate) return 0; } - if (!istate->sparse_checkout_patterns->use_cone_patterns) { - warning(_("attempting to use sparse-index without cone mode")); - return -1; - } + /* + * We need cone-mode patterns to use sparse-index. If a user edits + * their sparse-checkout file manually, then we can detect during + * parsing that they are not actually using cone-mode patterns and + * hence we need to abort this conversion _without error_. Warnings + * already exist in the pattern parsing to inform the user of their + * bad patterns. + */ + if (!istate->sparse_checkout_patterns->use_cone_patterns) + return 0; /* * NEEDSWORK: If we have unmerged entries, then stay full. From patchwork Tue Aug 10 19:50:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12429399 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EBABCC4338F for ; Tue, 10 Aug 2021 19:50:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D16E860ED8 for ; Tue, 10 Aug 2021 19:50:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232558AbhHJTun (ORCPT ); Tue, 10 Aug 2021 15:50:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54462 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231654AbhHJTul (ORCPT ); Tue, 10 Aug 2021 15:50:41 -0400 Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6AE29C0613C1 for ; Tue, 10 Aug 2021 12:50:18 -0700 (PDT) Received: by mail-wm1-x334.google.com with SMTP id g138so245039wmg.4 for ; Tue, 10 Aug 2021 12:50:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=1hHIMQenL0YY07ojegpV/bWS0DueNnK3juz9ucy/bVk=; b=QVgdGHPCvzMmM/zg2hPEhtfQ1fLRTM1tqjOeOaC6coIFmIvEzTIJzjvb12cEjAyORd OchNEF09fyiDNNMPNTOL1Ul8A41OmABk4LaTyNXeKmqLG8lyZrPg1nN8XPz5f8cv4pwH hPEC822CyuxPFM0qIDDftgw7hJx7vXi5f3xtlFWb9Bf1j5s+uCs7+sN2v6bdBp9kkWZW t8WQW3wY4zlw131AVv4HhPG7b0adRWiS95iZvu61yYMEK0AywatN2j8jUvE0/mUSZMtW PNyt9QJlk5iW8lriEx+t9IJ37sNn9VKtY84ULjnNG9XPX6vveqmAV6GT66j4vIjqojf6 g70Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=1hHIMQenL0YY07ojegpV/bWS0DueNnK3juz9ucy/bVk=; b=W+pGQVb242DG/mySa8Z8C10Dd6kvBfhtKjT1R0Nus4ySB0VsQVzbnmwGZteCIXKcbV 94s8J8Ns/YcBgW/e8xoIgY8tTjtKdlTVemPioAEggiTJzYrZaRZhrEA7dT0H5c7TOusB h6P06sQdGYIyBxtwuF+dMXWrix2sPEerUSBJEfzuU3XlX8EC9NxDItaz4OPk8T7yi46h KfSzdojhe+qY3pl+27DYfm7dQDZqY2Q8tctglGSBil6VZfQF8MT5+Wjwicf72GD9xqBU I9mLgu2Mn5e6I8wV3vog4j90siTelYy3gSjs2OVeKkQj+uBtX60zwFfMEn09md6WP0OZ 1mWw== X-Gm-Message-State: AOAM531NjibWMNXA+ocjmXCeZb8r2Mhpar8Ew/+A4DooQ+KGd9ZkRmZM 8VUbwEy9qaJ7TMkpHIpVMM6CaIBcvKo= X-Google-Smtp-Source: ABdhPJzzbHq8IsMBJMxLTbY0mmDOUPgi2n1rqX+rMOm9ZpRM4tU0c+RHjRN/zNflqN8tNybuyOA2/g== X-Received: by 2002:a7b:cd86:: with SMTP id y6mr6222821wmj.181.1628625017093; Tue, 10 Aug 2021 12:50:17 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id c9sm10261897wrm.43.2021.08.10.12.50.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Aug 2021 12:50:16 -0700 (PDT) Message-Id: <371985352680a767dfacb5477aa77e92e04008ee.1628625013.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 10 Aug 2021 19:50:08 +0000 Subject: [PATCH v2 3/8] sparse-index: silently return when cache tree fails Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, newren@gmail.com, matheus.bernardino@usp.br, stolee@gmail.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee If cache_tree_update() returns a non-zero value, then it could not create the cache tree. This is likely due to a path having a merge conflict. Since we are already returning early, let's return silently to avoid making it seem like we failed to write the index at all. If we remove our dependence on the cache tree within convert_to_sparse(), then we could still recover from this scenario and have a sparse index. When constructing the cache-tree extension in convert_to_sparse(), it is possible that we construct a tree object that is new to the object database. Without the WRITE_TREE_MISSING_OK flag, this results in an error that halts our conversion to a sparse index. Add this flag to remove this limitation. Signed-off-by: Derrick Stolee --- sparse-index.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/sparse-index.c b/sparse-index.c index bc5900eae35..b6e90417556 100644 --- a/sparse-index.c +++ b/sparse-index.c @@ -179,10 +179,15 @@ int convert_to_sparse(struct index_state *istate) /* Clear and recompute the cache-tree */ cache_tree_free(&istate->cache_tree); - if (cache_tree_update(istate, 0)) { - warning(_("unable to update cache-tree, staying full")); - return -1; - } + /* + * Silently return if there is a problem with the cache tree update, + * which might just be due to a conflict state in some entry. + * + * This might create new tree objects, so be sure to use + * WRITE_TREE_MISSING_OK. + */ + if (cache_tree_update(istate, WRITE_TREE_MISSING_OK)) + return 0; remove_fsmonitor(istate); From patchwork Tue Aug 10 19:50:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12429401 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BD205C432BE for ; Tue, 10 Aug 2021 19:50:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 99BF161051 for ; Tue, 10 Aug 2021 19:50:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232465AbhHJTup (ORCPT ); Tue, 10 Aug 2021 15:50:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54468 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232424AbhHJTul (ORCPT ); Tue, 10 Aug 2021 15:50:41 -0400 Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E451BC0613D3 for ; Tue, 10 Aug 2021 12:50:18 -0700 (PDT) Received: by mail-wr1-x429.google.com with SMTP id h13so27837444wrp.1 for ; Tue, 10 Aug 2021 12:50:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=oc2wvXngXb26lsP7r8c0mk+JZ8u847riiXPKJWHBYNk=; b=B4Lb46ahWBFeNrq2OngRNM8QHMngLZ1ijZlbyPMemBEJOah3m/QHcmGN0tLrRfvQqC I0/kI0bHp0njGtEPW7k9VcS26MeIE0QnSBvzEm55Mu84BkMYFiM8CZgqGqX90+1GOUaQ 0hfbSujkdDb0ysk9ZtmD1hWQiC1+KUril93lEQjb94fL5DHH6rQcWqEmGOU/wSP9rXEt GE02hxfOqhBAmDNvcMQ6T1TbS8lr3tugtPiEakafeHz30gGb+cVEi2PzqPS3vNouL1DA iZrZxbRiCOv2vzKkDb6nhK8lR1YmGSgRnET4G3T5qjA6f1nk/rP+/aXXzMADk23Gn9Hv NiyA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=oc2wvXngXb26lsP7r8c0mk+JZ8u847riiXPKJWHBYNk=; b=h6f/lGJ5YFaS6UDuy+hydSXskR8phKExEsXf5ZOcx4cpnTazEU5sRhSaLHNRb3jrSc oYrTxUtwAR79ZnwKG7lFCWCOy6z93T2w0i2mfvgnN4+kX5/mtPZj1w8z7OcywvAT/OpG SuNw/ncTs3NOWuvXWCF0RLRr9u1zFw67hvDC2U5+AMiH6pVT6srEgBuZ0/o9475N/34i vn5PYxiXd8C7qhFz0KoNc4NtZNGBjFy01EH0R/aBQcKAqGuY0751pA7kQF2T1zlK+jFG brvI1GKPJ+OISKOcqk2X7xPBVQPhjmy3BA4zl/hZ/1EhSSiwyBYfE3tEOuXqoDX34l6K n0QQ== X-Gm-Message-State: AOAM530402RkPWZ59x+ADngz5sbFz/4S1hE0TBUsDd3rot0FzFfY/4Qo OorFPhWCDIjTxULmN3Be2LGESNfK18s= X-Google-Smtp-Source: ABdhPJywFUwpEg6Dj6ETEi8DRowCpF4yvX53pT7Mrxyq7wfpll+lNPGgPTDrUOtuEGH5v6flx9yANg== X-Received: by 2002:a5d:5106:: with SMTP id s6mr32735894wrt.359.1628625017644; Tue, 10 Aug 2021 12:50:17 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id z17sm24887110wrt.47.2021.08.10.12.50.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Aug 2021 12:50:17 -0700 (PDT) Message-Id: <10bcadb284e49419f9b4baf75e05f719ec395d98.1628625013.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 10 Aug 2021 19:50:09 +0000 Subject: [PATCH v2 4/8] unpack-trees: fix nested sparse-dir search Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, newren@gmail.com, matheus.bernardino@usp.br, stolee@gmail.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee The iterated search in find_cache_entry() was recently modified to include a loop that searches backwards for a sparse directory entry that matches the given traverse_info and name_entry. However, the string comparison failed to actually concatenate those two strings, so this failed to find a sparse directory when it was not a top-level directory. This caused some errors in rare cases where a 'git checkout' spanned a diff that modified files within the sparse directory entry, but we could not correctly find the entry. Signed-off-by: Derrick Stolee --- unpack-trees.c | 18 +++++++++++++----- 1 file changed, 13 insertions(+), 5 deletions(-) diff --git a/unpack-trees.c b/unpack-trees.c index 5786645f315..df1f4437723 100644 --- a/unpack-trees.c +++ b/unpack-trees.c @@ -1255,9 +1255,10 @@ static int sparse_dir_matches_path(const struct cache_entry *ce, static struct cache_entry *find_cache_entry(struct traverse_info *info, const struct name_entry *p) { - struct cache_entry *ce; + struct cache_entry *ce = NULL; int pos = find_cache_pos(info, p->path, p->pathlen); struct unpack_trees_options *o = info->data; + struct strbuf full_path = STRBUF_INIT; if (0 <= pos) return o->src_index->cache[pos]; @@ -1273,6 +1274,10 @@ static struct cache_entry *find_cache_entry(struct traverse_info *info, if (pos < 0 || pos >= o->src_index->cache_nr) return NULL; + strbuf_addstr(&full_path, info->traverse_path); + strbuf_add(&full_path, p->path, p->pathlen); + strbuf_addch(&full_path, '/'); + /* * Due to lexicographic sorting and sparse directory * entries ending with a trailing slash, our path as a @@ -1283,17 +1288,20 @@ static struct cache_entry *find_cache_entry(struct traverse_info *info, while (pos >= 0) { ce = o->src_index->cache[pos]; - if (strncmp(ce->name, p->path, p->pathlen)) - return NULL; + if (strncmp(ce->name, full_path.buf, full_path.len)) { + ce = NULL; + break; + } if (S_ISSPARSEDIR(ce->ce_mode) && sparse_dir_matches_path(ce, info, p)) - return ce; + break; pos--; } - return NULL; + strbuf_release(&full_path); + return ce; } static void debug_path(struct traverse_info *info) From patchwork Tue Aug 10 19:50:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12429403 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BC2DEC4338F for ; Tue, 10 Aug 2021 19:50:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9F4D161075 for ; Tue, 10 Aug 2021 19:50:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232627AbhHJTuq (ORCPT ); Tue, 10 Aug 2021 15:50:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54470 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232457AbhHJTum (ORCPT ); Tue, 10 Aug 2021 15:50:42 -0400 Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com [IPv6:2a00:1450:4864:20::32d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 80C87C0613C1 for ; Tue, 10 Aug 2021 12:50:19 -0700 (PDT) Received: by mail-wm1-x32d.google.com with SMTP id d131-20020a1c1d890000b02902516717f562so306426wmd.3 for ; Tue, 10 Aug 2021 12:50:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=AkDIkGh+DheEaNgO3OtM/BD/mizdLYg7oJenasMDoPY=; b=GbjlOSZGntIE02DFVDU05jz5XUog3GphYLWWJ3+wkHH0jOtwHnV4HAFGYHd51YaJ35 +ADJsyNOY7s5X3Uy4a/Gh//UqL5PKgGzxmAGE8j95AMQZ1QqrT0sPos7+x/xzcRciEhS obbcs8OHX4eqoO+Ywd+kCv6+ywNruu5amGH/k5PBgCaDtwqzcvWG0DM4SbPxi1mHEGLQ LiUQhYrSEk+Ap6GSmj/m700pqyNG3rhBdveTzzch12OS9fUVUf4R8zaGeRAXMrZK9Pq7 LzPrM3xTLx/QkquyUzOm8vxbF5Q5KxlvhBeLl5NV1etOuteegokHBBLFBP+aF8yG+SP2 Dv1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=AkDIkGh+DheEaNgO3OtM/BD/mizdLYg7oJenasMDoPY=; b=Z4Ys61P59FeUGZ1C1RmZvWuA2IeW3NoPkUpfZkkC7Zag41ixVw4KZ/9FJ0v+IOuwzm kABP1bOapJyGXfvHQzbbARdMxV3DETeprVzl82He/viKygHuq3La/uvb7uzG+1O2UAwJ D04fcEzoEZPBRmNP5sy6jMw083n3u5X/JOE+WrOkIaevF1L4Ns87w8pjvK+ntdEvkykS kEJdRL4czJ3dPZZVhJDsRL7UuAs2JoDZr6jfWFB7qakjPW1o6Z5JQI45N1py9arowKDo GkjjG6YOk9F65ehTDOPwaGsYySp78KtsVEYF6dNi1HLtFQfe1Z3G0AwA+G+0TlVmLAU7 R24Q== X-Gm-Message-State: AOAM532AvD3vHb//vFD6efS7IViQ0OMWcJtmnatH9T3aObQTrsGSZ08e dB8CbdZn0v6YQyPhp60eXyLetc+XZFc= X-Google-Smtp-Source: ABdhPJytixIN/ifI6TWu1QpctwYixFRSK/MNtZRZNjiT/3qE6/BDIwzY8UDxNO6amawvpTw8y2S5EA== X-Received: by 2002:a7b:cc16:: with SMTP id f22mr6245062wmh.99.1628625018181; Tue, 10 Aug 2021 12:50:18 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id y14sm12775785wrs.29.2021.08.10.12.50.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Aug 2021 12:50:17 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Tue, 10 Aug 2021 19:50:10 +0000 Subject: [PATCH v2 5/8] sparse-checkout: create helper methods Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, newren@gmail.com, matheus.bernardino@usp.br, stolee@gmail.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee As we integrate the sparse index into more builtins, we occasionally need to check the sparse-checkout patterns to see if a path is within the sparse-checkout cone. Create some helper methods that help initialize the patterns and check for pattern matching to make this easier. The existing callers of commands like get_sparse_checkout_patterns() use a custom 'struct pattern_list' that is not necessarily the one in the 'struct index_state', so there are not many previous uses that could adopt these helpers. There are just two in builtin/add.c and sparse-index.c that can use path_in_sparse_checkout(). Signed-off-by: Derrick Stolee --- builtin/add.c | 8 ++------ dir.c | 29 +++++++++++++++++++++++++++++ dir.h | 6 ++++++ sparse-index.c | 12 +++--------- 4 files changed, 40 insertions(+), 15 deletions(-) diff --git a/builtin/add.c b/builtin/add.c index 17528e8f922..f675bdeae4a 100644 --- a/builtin/add.c +++ b/builtin/add.c @@ -190,8 +190,6 @@ static int refresh(int verbose, const struct pathspec *pathspec) struct string_list only_match_skip_worktree = STRING_LIST_INIT_NODUP; int flags = REFRESH_IGNORE_SKIP_WORKTREE | (verbose ? REFRESH_IN_PORCELAIN : REFRESH_QUIET); - struct pattern_list pl = { 0 }; - int sparse_checkout_enabled = !get_sparse_checkout_patterns(&pl); seen = xcalloc(pathspec->nr, 1); refresh_index(&the_index, flags, pathspec, seen, @@ -199,12 +197,10 @@ static int refresh(int verbose, const struct pathspec *pathspec) for (i = 0; i < pathspec->nr; i++) { if (!seen[i]) { const char *path = pathspec->items[i].original; - int dtype = DT_REG; if (matches_skip_worktree(pathspec, i, &skip_worktree_seen) || - (sparse_checkout_enabled && - !path_matches_pattern_list(path, strlen(path), NULL, - &dtype, &pl, &the_index))) { + (core_apply_sparse_checkout && + path_in_sparse_checkout(path, &the_index) == NOT_MATCHED)) { string_list_append(&only_match_skip_worktree, pathspec->items[i].original); } else { diff --git a/dir.c b/dir.c index 03c4d212672..7027bdfa068 100644 --- a/dir.c +++ b/dir.c @@ -1439,6 +1439,35 @@ done: return result; } +int init_sparse_checkout_patterns(struct index_state *istate) +{ + if (istate->sparse_checkout_patterns) + return 0; + + CALLOC_ARRAY(istate->sparse_checkout_patterns, 1); + + if (get_sparse_checkout_patterns(istate->sparse_checkout_patterns) < 0) { + FREE_AND_NULL(istate->sparse_checkout_patterns); + return -1; + } + + return 0; +} + +enum pattern_match_result path_in_sparse_checkout(const char *path, + struct index_state *istate) +{ + int dtype = DT_REG; + init_sparse_checkout_patterns(istate); + + if (!istate->sparse_checkout_patterns) + return MATCHED; + + return path_matches_pattern_list(path, strlen(path), NULL, &dtype, + istate->sparse_checkout_patterns, + istate); +} + static struct path_pattern *last_matching_pattern_from_lists( struct dir_struct *dir, struct index_state *istate, const char *pathname, int pathlen, diff --git a/dir.h b/dir.h index b3e1a54a971..9af2e8c4ba4 100644 --- a/dir.h +++ b/dir.h @@ -394,6 +394,12 @@ enum pattern_match_result path_matches_pattern_list(const char *pathname, const char *basename, int *dtype, struct pattern_list *pl, struct index_state *istate); + +int init_sparse_checkout_patterns(struct index_state *state); + +enum pattern_match_result path_in_sparse_checkout(const char *path, + struct index_state *istate); + struct dir_entry *dir_add_ignored(struct dir_struct *dir, struct index_state *istate, const char *pathname, int len); diff --git a/sparse-index.c b/sparse-index.c index b6e90417556..2efc9fd4910 100644 --- a/sparse-index.c +++ b/sparse-index.c @@ -34,17 +34,14 @@ static int convert_to_sparse_rec(struct index_state *istate, int i, can_convert = 1; int start_converted = num_converted; enum pattern_match_result match; - int dtype = DT_UNKNOWN; struct strbuf child_path = STRBUF_INIT; - struct pattern_list *pl = istate->sparse_checkout_patterns; /* * Is the current path outside of the sparse cone? * Then check if the region can be replaced by a sparse * directory entry (everything is sparse and merged). */ - match = path_matches_pattern_list(ct_path, ct_pathlen, - NULL, &dtype, pl, istate); + match = path_in_sparse_checkout(ct_path, istate); if (match != NOT_MATCHED) can_convert = 0; @@ -153,11 +150,8 @@ int convert_to_sparse(struct index_state *istate) if (!istate->repo->settings.sparse_index) return 0; - if (!istate->sparse_checkout_patterns) { - istate->sparse_checkout_patterns = xcalloc(1, sizeof(struct pattern_list)); - if (get_sparse_checkout_patterns(istate->sparse_checkout_patterns) < 0) - return 0; - } + if (init_sparse_checkout_patterns(istate) < 0) + return 0; /* * We need cone-mode patterns to use sparse-index. If a user edits From patchwork Tue Aug 10 19:50:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12429405 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 87013C4320A for ; Tue, 10 Aug 2021 19:50:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6A8B661078 for ; Tue, 10 Aug 2021 19:50:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232559AbhHJTuq (ORCPT ); Tue, 10 Aug 2021 15:50:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54476 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232521AbhHJTum (ORCPT ); Tue, 10 Aug 2021 15:50:42 -0400 Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2430BC0613C1 for ; Tue, 10 Aug 2021 12:50:20 -0700 (PDT) Received: by mail-wr1-x42b.google.com with SMTP id f5so12056574wrm.13 for ; Tue, 10 Aug 2021 12:50:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=dgfH8FHC7lRtH5aYzMRWtNR/tzYMsPRc7peitaicWXk=; b=CB/XLEk9idwkqsql2iAFQr/B5Gkz4DD+PIGhRyei/FqZHwyLZ9Apj/4zB8mx+yFumF HpysIEygYESRNe6ldLxhyLlURYJye1khjN9uIsVozZzqLrqHWkWQR8svtD/S2xoiLIdd 3zPhTTJyxHXu98zFQesfc719yfa1rbDLeGIeb6lxjuXsC9hSfAhKLVbXOBdKVUyVITfL B8MPrllBB/WvV4eXjhjcN1dH4IAj5ypxotIaiWpu2jWEGc2ghk82qqIYv5GVCchaYSMo jicdysfkdaIfMdyuY4renrN2j85qmYib9P2cpVBHk0IHlY6aN1LXwbMNlNx7/v0SSSkv GN6A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=dgfH8FHC7lRtH5aYzMRWtNR/tzYMsPRc7peitaicWXk=; b=nbHKBnOAml3j0m10OegRkmKG/JS6dWCTJPXGwVZygGaiIqygkxNUp6DrbLZmKtbT1/ 4nmq5+J8PTL1yvuxEYblHv67cqzCd43l8ffDa/X6QNt693LVRwz1iCF8lPAO14P0hZHu Qoug+deeZmPnRNxM7f8Adhlo4Ks8+ZNp2r+ps0v2LKzE2pN2Gp1lPmR6smx5UzsL19lu AtCn75H9/eXoeXbPCVrknUyHzGIuxoGzC+RVWI5KRFAartQEQjvHOUsgdtGM5DZCeMEW 6k+/Mo4v/Zo1bTAqjRaEEpCj/ul/Amiv4GluXFSDycBmFgDuyBUuqadE0bI16Zx9ztVR juQw== X-Gm-Message-State: AOAM532dU35E+TXInIehqYYfpHT0hzmgnSLzfNSRC1HwOFaXBv9d8YmG 4yc248lxO6Fg1dSfgyuztI+ldn06/PE= X-Google-Smtp-Source: ABdhPJwsqIf/QLQCJbET2zEIMJeJRlDu601KyYfqjivBqHwcY2LIPcr/+pJOXSpnofzq9cQ0Lt4gJQ== X-Received: by 2002:adf:e551:: with SMTP id z17mr15736086wrm.40.1628625018849; Tue, 10 Aug 2021 12:50:18 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id n8sm23873294wrx.46.2021.08.10.12.50.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Aug 2021 12:50:18 -0700 (PDT) Message-Id: <5680df62e1cd56714d5b6a61081aed381749ac54.1628625014.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 10 Aug 2021 19:50:11 +0000 Subject: [PATCH v2 6/8] attr: be careful about sparse directories Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, newren@gmail.com, matheus.bernardino@usp.br, stolee@gmail.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Signed-off-by: Derrick Stolee --- attr.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/attr.c b/attr.c index d029e681f28..a1009f78029 100644 --- a/attr.c +++ b/attr.c @@ -14,6 +14,7 @@ #include "utf8.h" #include "quote.h" #include "thread-utils.h" +#include "dir.h" const char git_attr__true[] = "(builtin)true"; const char git_attr__false[] = "\0(builtin)false"; @@ -744,6 +745,19 @@ static struct attr_stack *read_attr_from_index(struct index_state *istate, if (!istate) return NULL; + /* + * In the case of cone-mode sparse-checkout, getting the + * .gitattributes file from a directory is meaningless: all + * contained paths will be sparse if the .gitattributes is also + * sparse. In the case of a sparse index, it is critical that we + * don't go looking for one as it will expand the index. + */ + init_sparse_checkout_patterns(istate); + if (istate->sparse_checkout_patterns && + istate->sparse_checkout_patterns->use_cone_patterns && + path_in_sparse_checkout(path, istate) == NOT_MATCHED) + return NULL; + buf = read_blob_data_from_index(istate, path, NULL); if (!buf) return NULL; From patchwork Tue Aug 10 19:50:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12429407 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 58D00C432BE for ; Tue, 10 Aug 2021 19:50:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3EC2A60ED8 for ; Tue, 10 Aug 2021 19:50:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232585AbhHJTur (ORCPT ); Tue, 10 Aug 2021 15:50:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54482 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232549AbhHJTun (ORCPT ); Tue, 10 Aug 2021 15:50:43 -0400 Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D7DE8C0613D3 for ; Tue, 10 Aug 2021 12:50:20 -0700 (PDT) Received: by mail-wr1-x42a.google.com with SMTP id q10so4585750wro.2 for ; Tue, 10 Aug 2021 12:50:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=siV5lRTXPuRg8PtQx00TLBS2bGCepL9AAAoIBksPutM=; b=Am07ueTagsuq2yEXcTQYjhPaWv0NT+SKDCW8kXmXV9T6klU00pibqA5dGD/6nuf7rI JefOGxvwS2aWeQ2LRFjM6frVX59eC//hjcGwldlV9IQZGLcohZXN3JSOnsoiXKLR3apl 09PxTvxnwMpPzffHV3X4ibT/e4ktRYMvqZ6uTaRMyR2xdxxmJkpuKcZwFtnV7tWS2qCi L0cseT7Jrd0br21o6rce5KuC7qN4bUw18IxMhVtyuUpwpyXJ2oUOLgS83U4hZ+49ZTdk qdXFKkF+RJgB/Kr6a692+L3qfgi/pqdvfthBo45vyabjJIsl67EKT8tDQo6QhLkmbHx2 hzww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=siV5lRTXPuRg8PtQx00TLBS2bGCepL9AAAoIBksPutM=; b=a5+64oXqFFOWwnM74OL8srfi55R8nCVkpEUsAKaX3sIlPq7Rg+OK3UTemASwxO1/jT UGOR/J71oPOL5Pv8ww0YNzkw3fiTeTA4nBHsYK6oH/VI7KaqlwhHX0IbPUyKPCPpYKbl HDdJmXUCCKgyEW4e88sB/iaxWby+PI3fpP9avifI6diNQqmsyrgrtvXmkLbV+9EXiRQP mXC06ShCOsVVCmfTYUPfhNDQfvi5TG13kaWxNmDjBl8sbTj9CyuO5nvdbZG4mAsi/U2K hoqOf4iMZGHkcpdW3i7xpuzU8SzDfcWlV/ffZFO2DG9P4v6OVX0Eg4dvJfYRWwpjj6Rm ZjZA== X-Gm-Message-State: AOAM53122cu1ezx12LN8Sn5WzosfDxqR40vvOz6HfpCaEk2BF4LfjQeq tEALUZzgNLg0yoXFqgTYCHrXXtv2gFc= X-Google-Smtp-Source: ABdhPJyJeB4QRYAAVGqSsJyea3UD+m4C4+jhJ+VwOgAOqwc3KQ9hclk8Z/7jXTDijotKQKuiK0p6CA== X-Received: by 2002:adf:dc05:: with SMTP id t5mr2354572wri.326.1628625019555; Tue, 10 Aug 2021 12:50:19 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id a8sm13921974wrh.88.2021.08.10.12.50.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Aug 2021 12:50:19 -0700 (PDT) Message-Id: <1dd73b36eb493fbd3f7a7c964e57179a4ae26523.1628625014.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 10 Aug 2021 19:50:12 +0000 Subject: [PATCH v2 7/8] sparse-index: add SPARSE_INDEX_IGNORE_CONFIG flag Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, newren@gmail.com, matheus.bernardino@usp.br, stolee@gmail.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee The convert_to_sparse() method checks for the GIT_TEST_SPARSE_INDEX environment variable or the "index.sparse" config setting before converting the index to a sparse one. This is for ease of use since all current consumers are preparing to compress the index before writing it to disk. If these settings are not enabled, then convert_to_sparse() silently returns without doing anything. We will add a consumer in the next change that wants to use the sparse index as an in-memory data structure, regardless of whether the on-disk format should be sparse. To that end, create the SPARSE_INDEX_IGNORE_CONFIG flag that will skip these config checks when enabled. All current consumers are modified to pass '0' in the new 'flags' parameter. Signed-off-by: Derrick Stolee --- read-cache.c | 4 ++-- sparse-index.c | 30 ++++++++++++++++-------------- sparse-index.h | 3 ++- 3 files changed, 20 insertions(+), 17 deletions(-) diff --git a/read-cache.c b/read-cache.c index 9048ef9e905..f5d4385c408 100644 --- a/read-cache.c +++ b/read-cache.c @@ -3069,7 +3069,7 @@ static int do_write_locked_index(struct index_state *istate, struct lock_file *l int ret; int was_full = !istate->sparse_index; - ret = convert_to_sparse(istate); + ret = convert_to_sparse(istate, 0); if (ret) { warning(_("failed to convert to a sparse-index")); @@ -3182,7 +3182,7 @@ static int write_shared_index(struct index_state *istate, int ret, was_full = !istate->sparse_index; move_cache_to_base_index(istate); - convert_to_sparse(istate); + convert_to_sparse(istate, 0); trace2_region_enter_printf("index", "shared/do_write_index", the_repository, "%s", get_tempfile_path(*temp)); diff --git a/sparse-index.c b/sparse-index.c index 2efc9fd4910..532fd11787e 100644 --- a/sparse-index.c +++ b/sparse-index.c @@ -124,7 +124,7 @@ static int index_has_unmerged_entries(struct index_state *istate) return 0; } -int convert_to_sparse(struct index_state *istate) +int convert_to_sparse(struct index_state *istate, int flags) { int test_env; @@ -135,20 +135,22 @@ int convert_to_sparse(struct index_state *istate) if (!istate->repo) istate->repo = the_repository; - /* - * The GIT_TEST_SPARSE_INDEX environment variable triggers the - * index.sparse config variable to be on. - */ - test_env = git_env_bool("GIT_TEST_SPARSE_INDEX", -1); - if (test_env >= 0) - set_sparse_index_config(istate->repo, test_env); + if (!(flags & SPARSE_INDEX_IGNORE_CONFIG)) { + /* + * The GIT_TEST_SPARSE_INDEX environment variable triggers the + * index.sparse config variable to be on. + */ + test_env = git_env_bool("GIT_TEST_SPARSE_INDEX", -1); + if (test_env >= 0) + set_sparse_index_config(istate->repo, test_env); - /* - * Only convert to sparse if index.sparse is set. - */ - prepare_repo_settings(istate->repo); - if (!istate->repo->settings.sparse_index) - return 0; + /* + * Only convert to sparse if index.sparse is set. + */ + prepare_repo_settings(istate->repo); + if (!istate->repo->settings.sparse_index) + return 0; + } if (init_sparse_checkout_patterns(istate) < 0) return 0; diff --git a/sparse-index.h b/sparse-index.h index 1115a0d7dd9..475f4f0f8da 100644 --- a/sparse-index.h +++ b/sparse-index.h @@ -2,7 +2,8 @@ #define SPARSE_INDEX_H__ struct index_state; -int convert_to_sparse(struct index_state *istate); +#define SPARSE_INDEX_IGNORE_CONFIG (1 << 0) +int convert_to_sparse(struct index_state *istate, int flags); /* * Some places in the codebase expect to search for a specific path. From patchwork Tue Aug 10 19:50:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12429409 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5A96C4338F for ; Tue, 10 Aug 2021 19:50:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C522C60ED8 for ; Tue, 10 Aug 2021 19:50:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232665AbhHJTus (ORCPT ); Tue, 10 Aug 2021 15:50:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54494 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232085AbhHJTuo (ORCPT ); Tue, 10 Aug 2021 15:50:44 -0400 Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com [IPv6:2a00:1450:4864:20::32a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A6609C06179B for ; Tue, 10 Aug 2021 12:50:21 -0700 (PDT) Received: by mail-wm1-x32a.google.com with SMTP id q11-20020a7bce8b0000b02902e6880d0accso2775139wmj.0 for ; Tue, 10 Aug 2021 12:50:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=pcwjy+V2SEhtxyn0vJUrmLOj7GCOWJeEG7NdwuPhcdA=; b=roEHwXZP1AsQ2wj7YYf3O85F9kD87kkxcK0U8HVJ9gkyZdlklrgIvPduJCJieVE6E8 rbNfzl9FrSol91iYZxNB6s6nPDaAz6+86YcAz3AbKKdItKkAGpvMAp4eXJ7Z1SxySlfl bpn2+O7EDUVmMZsH7fCkuV457iyZu9BVeXubLjCbh8NhP/ySM7XkZZpeB2XVCspxgE5/ TG/socyRh49m6mTA4L3Vnac9OTCnEp7hv42wceOX67zB/Oa2GsTN9lDHOXxL1uvZv5mK L82LfPiKT7nPs4a7NvclcUeTx35wMupgMGcQzbWr1xjpKhN2BfrZDqEGw2yZS4/4jVp3 OQiA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=pcwjy+V2SEhtxyn0vJUrmLOj7GCOWJeEG7NdwuPhcdA=; b=HJ0M9Wb0nXXFwF5m9HOJXxpdKFm6msLUw9vKjqhWUXjeq0v1q6XQ5jK4jFSvJVg7o3 5SktQcy+6yKGjqYEq+iEkEYuu9+7aMdLSqILivuJ58aDmEWwNansiWsmvEyMLxTL8xlm 01BkwVF3OM1uaOwFx19nkDeInSWOHftd9H6Ro3hgWuX6c3MJJIp473GsMWki4Rg3A65i qSQ0XJ+EiVc4nPYYBf2aM31Hd/EDFzuD3/HQcD5FO9uQI2dZQ5+ubqxrmPUfqXKu08t1 35DOzwtLznWsFfwEgs7tKYBx4zM5DOIcXyllAfQsqYBBQmFXuSRThZlVM4mMzrf0w20U fSpg== X-Gm-Message-State: AOAM533XJ6u2FzGIqEYpvDfMw+StE0rvmZQ7Wrw8la7rffRLo/e55LgQ lVzdsI76va5992HN5yXET/4ke+IJiKI= X-Google-Smtp-Source: ABdhPJzBr9hdSPeX4h5SDQS+BCVGahv8rUsUivUnGaMOfMXQmZqFBjECL6fjbnMFZq1en6Y8B7rQVw== X-Received: by 2002:a05:600c:cc:: with SMTP id u12mr23629956wmm.63.1628625020207; Tue, 10 Aug 2021 12:50:20 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id p2sm24651324wrr.21.2021.08.10.12.50.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Aug 2021 12:50:19 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Tue, 10 Aug 2021 19:50:13 +0000 Subject: [PATCH v2 8/8] sparse-checkout: clear tracked sparse dirs Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, newren@gmail.com, matheus.bernardino@usp.br, stolee@gmail.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee When changing the scope of a sparse-checkout using cone mode, we might have some tracked directories go out of scope. The current logic removes the tracked files from within those directories, but leaves the ignored files within those directories. This is a bit unexpected to users who have given input to Git saying they don't need those directories anymore. This is something that is new to the cone mode pattern type: the user has explicitly said "I want these directories and _not_ those directories." The typical sparse-checkout patterns more generally apply to "I want files with with these patterns" so it is natural to leave ignored files as they are. This focus on directories in cone mode provides us an opportunity to change the behavior. Leaving these ignored files in the sparse directories makes it impossible to gain performance benefits in the sparse index. When we track into these directories, we need to know if the files are ignored or not, which might depend on the _tracked_ .gitignore file(s) within the sparse directory. This depends on the indexed version of the file, so the sparse directory must be expanded. By deleting the sparse directories when changing scope (or running 'git sparse-checkout reapply') we regain these performance benefits as if the repository was in a clean state. Since these ignored files are frequently build output or helper files from IDEs, the users should not need the files now that the tracked files are removed. If the tracked files reappear, then they will have newer timestamps than the build artifacts, so the artifacts will need to be regenerated anyway. Use the sparse-index as a data structure in order to find the sparse directories that can be safely deleted. Re-expand the index to a full one if it was full before. Signed-off-by: Derrick Stolee --- Documentation/git-sparse-checkout.txt | 8 +++ builtin/sparse-checkout.c | 95 +++++++++++++++++++++++++++ t/t1091-sparse-checkout-builtin.sh | 59 +++++++++++++++++ 3 files changed, 162 insertions(+) diff --git a/Documentation/git-sparse-checkout.txt b/Documentation/git-sparse-checkout.txt index fdcf43f87cb..f9022b9d555 100644 --- a/Documentation/git-sparse-checkout.txt +++ b/Documentation/git-sparse-checkout.txt @@ -210,6 +210,14 @@ case-insensitive check. This corrects for case mismatched filenames in the 'git sparse-checkout set' command to reflect the expected cone in the working directory. +The cone mode sparse-checkout patterns will also remove ignored files that +are not within the sparse-checkout definition. This is important behavior +to preserve the performance of the sparse index, but also matches that +cone mode patterns care about directories, not files. If there exist files +that are untracked and not ignored, then Git will not delete files within +that directory other than the tracked files that are now out of scope. +These files should be removed manually to ensure Git can behave optimally. + SUBMODULES ---------- diff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c index 8ba9f13787b..b06c8f885ac 100644 --- a/builtin/sparse-checkout.c +++ b/builtin/sparse-checkout.c @@ -100,6 +100,99 @@ static int sparse_checkout_list(int argc, const char **argv) return 0; } +static void clean_tracked_sparse_directories(struct repository *r) +{ + int i, was_full = 0; + struct strbuf path = STRBUF_INIT; + size_t pathlen; + struct string_list_item *item; + struct string_list sparse_dirs = STRING_LIST_INIT_DUP; + + /* + * If we are not using cone mode patterns, then we cannot + * delete directories outside of the sparse cone. + */ + if (!r || !r->index || !r->worktree) + return; + init_sparse_checkout_patterns(r->index); + if (!r->index->sparse_checkout_patterns || + !r->index->sparse_checkout_patterns->use_cone_patterns) + return; + + /* + * Use the sparse index as a data structure to assist finding + * directories that are safe to delete. This conversion to a + * sparse index will not delete directories that contain + * conflicted entries or submodules. + */ + if (!r->index->sparse_index) { + /* + * If something, such as a merge conflict or other concern, + * prevents us from converting to a sparse index, then do + * not try deleting files. + */ + if (convert_to_sparse(r->index, SPARSE_INDEX_IGNORE_CONFIG)) + return; + was_full = 1; + } + + strbuf_addstr(&path, r->worktree); + strbuf_complete(&path, '/'); + pathlen = path.len; + + /* + * Collect directories that have gone out of scope but also + * exist on disk, so there is some work to be done. We need to + * store the entries in a list before exploring, since that might + * expand the sparse-index again. + */ + for (i = 0; i < r->index->cache_nr; i++) { + struct cache_entry *ce = r->index->cache[i]; + + if (S_ISSPARSEDIR(ce->ce_mode) && + repo_file_exists(r, ce->name)) + string_list_append(&sparse_dirs, ce->name); + } + + for_each_string_list_item(item, &sparse_dirs) { + struct dir_struct dir = DIR_INIT; + struct pathspec p = { 0 }; + struct strvec s = STRVEC_INIT; + + strbuf_setlen(&path, pathlen); + strbuf_addstr(&path, item->string); + + dir.flags |= DIR_SHOW_IGNORED_TOO; + + setup_standard_excludes(&dir); + strvec_push(&s, path.buf); + + parse_pathspec(&p, PATHSPEC_GLOB, 0, NULL, s.v); + fill_directory(&dir, r->index, &p); + + if (dir.nr) { + warning(_("directory '%s' contains untracked files," + " but is not in the sparse-checkout cone"), + item->string); + } else if (remove_dir_recursively(&path, 0)) { + /* + * Removal is "best effort". If something blocks + * the deletion, then continue with a warning. + */ + warning(_("failed to remove directory '%s'"), + item->string); + } + + dir_clear(&dir); + } + + string_list_clear(&sparse_dirs, 0); + strbuf_release(&path); + + if (was_full) + ensure_full_index(r->index); +} + static int update_working_directory(struct pattern_list *pl) { enum update_sparsity_result result; @@ -141,6 +234,8 @@ static int update_working_directory(struct pattern_list *pl) else rollback_lock_file(&lock_file); + clean_tracked_sparse_directories(r); + r->index->sparse_checkout_patterns = NULL; return result; } diff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh index 38fc8340f5c..71236981e64 100755 --- a/t/t1091-sparse-checkout-builtin.sh +++ b/t/t1091-sparse-checkout-builtin.sh @@ -642,4 +642,63 @@ test_expect_success MINGW 'cone mode replaces backslashes with slashes' ' check_files repo/deep a deeper1 ' +test_expect_success 'cone mode clears ignored subdirectories' ' + rm repo/.git/info/sparse-checkout && + + git -C repo sparse-checkout init --cone && + git -C repo sparse-checkout set deep/deeper1 && + + cat >repo/.gitignore <<-\EOF && + obj/ + *.o + EOF + + git -C repo add .gitignore && + git -C repo commit -m ".gitignore" && + + mkdir -p repo/obj repo/folder1/obj repo/deep/deeper2/obj && + for file in folder1/obj/a obj/a folder1/file.o folder1.o \ + deep/deeper2/obj/a deep/deeper2/file.o file.o + do + echo ignored >repo/$file || return 1 + done && + + git -C repo status --porcelain=v2 >out && + test_must_be_empty out && + + git -C repo sparse-checkout reapply && + test_path_is_missing repo/folder1 && + test_path_is_missing repo/deep/deeper2 && + test_path_is_dir repo/obj && + test_path_is_file repo/file.o && + + git -C repo status --porcelain=v2 >out && + test_must_be_empty out && + + git -C repo sparse-checkout set deep/deeper2 && + test_path_is_missing repo/deep/deeper1 && + test_path_is_dir repo/deep/deeper2 && + test_path_is_dir repo/obj && + test_path_is_file repo/file.o && + + >repo/deep/deeper2/ignored.o && + >repo/deep/deeper2/untracked && + + # When an untracked file is in the way, all untracked files + # (even ignored files) are preserved. + git -C repo sparse-checkout set folder1 2>err && + grep "contains untracked files" err && + test_path_is_file repo/deep/deeper2/ignored.o && + test_path_is_file repo/deep/deeper2/untracked && + + # The rest of the cone matches expectation + test_path_is_missing repo/deep/deeper1 && + test_path_is_dir repo/obj && + test_path_is_file repo/file.o && + + git -C repo status --porcelain=v2 >out && + echo "? deep/deeper2/untracked" >expect && + test_cmp expect out +' + test_done