From patchwork Wed Aug 3 04:51:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shaoxuan Yuan X-Patchwork-Id: 12935192 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 10CC7C19F29 for ; Wed, 3 Aug 2022 04:51:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235568AbiHCEvt (ORCPT ); Wed, 3 Aug 2022 00:51:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39936 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229457AbiHCEvq (ORCPT ); Wed, 3 Aug 2022 00:51:46 -0400 Received: from mail-pf1-x42d.google.com (mail-pf1-x42d.google.com [IPv6:2607:f8b0:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A722AA444 for ; Tue, 2 Aug 2022 21:51:42 -0700 (PDT) Received: by mail-pf1-x42d.google.com with SMTP id g12so15473963pfb.3 for ; Tue, 02 Aug 2022 21:51:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=8Giyu19Nl2bViNaKwVJz8wtqPCydX94E30pkaHtiIhc=; b=Ivja0BKIfSfBoE01goytG3YOfIy5cjKPteCDr/QeybM0myrHK53ohq988+1MoUgzes wtiS/FD3c96szXMMXBNlZBOKWyJnHK1QYPNFJqA4LsroLXnech2eLeIVrG3lvPFqFpMm zyXdh/mnPttzpym6pNLBiNgBLsw8aZhIGE0egFcU4/VAPmmac9r9nO9KlBi8OW3JKpWF MVHCsxW8/pRMFEnsu7Z08JeZVOjNGqScnqFEeTsT32EyYWAJswMjkG3pLW1/YPUuE/eo VfEVEbnAbrB5uCbrSqG8187QKFJvCpEfIWpkU4JAOl5tLbguM5onBPF/Zsptf3oIQD+N 7BTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8Giyu19Nl2bViNaKwVJz8wtqPCydX94E30pkaHtiIhc=; b=jSTWBtf2B8aYFXHep+sFG4lAsPjReg+N8pFcgAJPLpcdJgXs5F4cufTdX9ZFNOYhPb JDDiVnkunVu9Px6sh0ofa9Dr6c7E8OVlk4CAVQcGLSUkSkHwRAMXem3FFAJqQtuOSAzq EQnbmW/17WuHa4uR3OolKQ0Z2UXcttvhIFsNWDVcvpwnwLDJg8F3t1qkMaIi5CWYsdHU hsMwHRQHNtMI+rY9CnZ11D25v964sUTwbWyyblb8EF632VwlvyKklE6e46qAPAMRbj/b ggW336KFHKUNlW6Hbaxt7I9C6wsrRy44U/93FFspwbnNuk4R40iSgpV6F2fd3pnIanWr 0xgw== X-Gm-Message-State: AJIora/DIebsXDhh01aZwLxbtYL2RaMnLFPqPDMCmDJsP+kPpuOTVmq7 AX+YfP+Os4NYwfbMRqt/6ztqvm2BikOrm8SP X-Google-Smtp-Source: AGRyM1seXgrVd3+Ej1oPZ34yuXyEjUMheT2w2gC3Bx5twsyI4gZav8Ru1lE8jNSO3BzOcKKFvCfkqA== X-Received: by 2002:a63:293:0:b0:412:4f02:950f with SMTP id 141-20020a630293000000b004124f02950fmr19548044pgc.187.1659502301929; Tue, 02 Aug 2022 21:51:41 -0700 (PDT) Received: from ffyuanda.localdomain ([113.65.228.71]) by smtp.gmail.com with ESMTPSA id f131-20020a623889000000b0052e2a1edab8sm470211pfa.24.2022.08.02.21.51.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Aug 2022 21:51:41 -0700 (PDT) From: Shaoxuan Yuan To: git@vger.kernel.org Cc: vdye@github.com, derrickstolee@github.com, Shaoxuan Yuan Subject: [PATCH v1 1/4] t1092: add tests for `git-rm` Date: Wed, 3 Aug 2022 12:51:15 +0800 Message-Id: <20220803045118.1243087-2-shaoxuan.yuan02@gmail.com> X-Mailer: git-send-email 2.37.0 In-Reply-To: <20220803045118.1243087-1-shaoxuan.yuan02@gmail.com> References: <20220803045118.1243087-1-shaoxuan.yuan02@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Add tests for `git-rm`, make sure it behaves as expected when is both inside or outside of sparse-checkout definition. Also add ensure_not_expanded test to make sure `git-rm` does not accidentally expand the index when is within the sparse-checkout definition. Signed-off-by: Shaoxuan Yuan --- t/t1092-sparse-checkout-compatibility.sh | 71 ++++++++++++++++++++++++ 1 file changed, 71 insertions(+) diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index 763c6cc684..75649e3265 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -1853,4 +1853,75 @@ test_expect_success 'mv directory from out-of-cone to in-cone' ' grep -e "H deep/0/1" actual ' +test_expect_success 'rm pathspec inside sparse definition' ' + init_repos && + + test_all_match git rm deep/a && + test_all_match git status --porcelain=v2 && + + # test wildcard + run_on_all git reset --hard && + test_all_match git rm deep/* && + test_all_match git status --porcelain=v2 && + + # test recursive rm + run_on_all git reset --hard && + test_all_match git rm -r deep && + test_all_match git status --porcelain=v2 +' + +test_expect_failure 'rm pathspec outside sparse definition' ' + init_repos && + + for file in folder1/a folder1/0/1 + do + test_sparse_match test_must_fail git rm $file && + test_sparse_match test_must_fail git rm --cached $file && + test_sparse_match git rm --sparse $file && + test_sparse_match git status --porcelain=v2 + done && + + cat >folder1-full <<-EOF && + rm ${SQ}folder1/0/0/0${SQ} + rm ${SQ}folder1/0/1${SQ} + rm ${SQ}folder1/a${SQ} + EOF + + cat >folder1-sparse <<-EOF && + rm ${SQ}folder1/${SQ} + EOF + + # test wildcard + run_on_sparse git reset --hard && + run_on_sparse git sparse-checkout reapply && + test_sparse_match test_must_fail git rm folder1/* && + run_on_sparse git rm --sparse folder1/* && + test_cmp folder1-full sparse-checkout-out && + test_cmp folder1-sparse sparse-index-out && + test_sparse_match git status --porcelain=v2 && + + # test recursive rm + run_on_sparse git reset --hard && + run_on_sparse git sparse-checkout reapply && + test_sparse_match test_must_fail git rm --sparse folder1 && + run_on_sparse git rm --sparse -r folder1 && + test_cmp folder1-full sparse-checkout-out && + test_cmp folder1-sparse sparse-index-out && + test_sparse_match git status --porcelain=v2 +' + +test_expect_failure 'sparse index is not expanded: rm' ' + init_repos && + + ensure_not_expanded rm deep/a && + + # test in-cone wildcard + git -C sparse-index reset --hard && + ensure_not_expanded rm deep/* && + + # test recursive rm + git -C sparse-index reset --hard && + ensure_not_expanded rm -r deep +' + test_done From patchwork Wed Aug 3 04:51:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shaoxuan Yuan X-Patchwork-Id: 12935190 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9DF73C19F2A for ; Wed, 3 Aug 2022 04:51:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235504AbiHCEvr (ORCPT ); Wed, 3 Aug 2022 00:51:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39930 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234270AbiHCEvq (ORCPT ); Wed, 3 Aug 2022 00:51:46 -0400 Received: from mail-pf1-x431.google.com (mail-pf1-x431.google.com [IPv6:2607:f8b0:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5F774A46D for ; Tue, 2 Aug 2022 21:51:44 -0700 (PDT) Received: by mail-pf1-x431.google.com with SMTP id f28so7218894pfk.1 for ; Tue, 02 Aug 2022 21:51:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=HLZKNiQZYFGGfDNhVw+EdBECChnMLUM6J/sEvCwyLBQ=; b=nzXD06BjryX6PQtJjq96PhPNl2VSAVfz/K74NGMODmZcXWlvHjv+RN9CcPAK2c6lDm imk55nNMM4QYp95Z6+bK9COfXqN46uAX1hLOysYf7oDsb5eu8WKFECmfSeJgY3expjTI h8lvg5dbN6HPUuhod61L5symSHGkfdl0YyKLq7AgFhB5a/0c6dh/Q+8QhWf4MY4H0fxa zpr6vsfK6QfSPaTAohg7Jsi/7l9vCH5cCWLzZi2RW/m6nH4t6qk73jJdfvpqpaaE7wX1 Lnln014C7+ZrWfHa+JtS9AkCroRjifekojxf3cK+kZj7ATsB6kIUWZDlGrXeoZC9DRxL WOtw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HLZKNiQZYFGGfDNhVw+EdBECChnMLUM6J/sEvCwyLBQ=; b=tPgfa0Pq2D4I766/C95yEPLsEfMZuV9bM2JJ4Gs4Tp1iHn7e5AJ1KnJBQq6AATgJq5 x/iDg46iWhjhwXObYkJzCaiZEIm4q+nBikns+OfVCCUrLhoSNb2xmq8/vOkpVltlMbxw s3Kj7PmVcShh27mbEfPs5h4v/MEwJN4WN9K6OgOLq+rwtQHw5v8uQbhs/Ik+3RHeE/zp x3k911g4veYaQ+CdrdSadoSW5GzM4zLfuLdsT1KU/ZFBtnRWXw/t9DhXrdVrCNL62lF5 z+jdgVm0G81Dfu1FWYOYaDrkwbItm9j011E+rKC4Gvrp5AbJuDb2j7zLa5lEvTvMrLM5 Y0Iw== X-Gm-Message-State: AJIora+e9zj6fv5XtwyYGX5hIodvXYp5CP0QtN9UoQk/zx17s7pmChad og51V0msr2i32H1v7i6yEFWbDEf8vWOxPo16 X-Google-Smtp-Source: AGRyM1tHZA6NKaXFHEyFZYf1xiyIfbLMo1VTyiB3K56mKIGkix/dnySrmHyILychRDA/UUWRv5a6IQ== X-Received: by 2002:a65:6d19:0:b0:41a:625e:7d7a with SMTP id bf25-20020a656d19000000b0041a625e7d7amr19159661pgb.506.1659502303657; Tue, 02 Aug 2022 21:51:43 -0700 (PDT) Received: from ffyuanda.localdomain ([113.65.228.71]) by smtp.gmail.com with ESMTPSA id f131-20020a623889000000b0052e2a1edab8sm470211pfa.24.2022.08.02.21.51.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Aug 2022 21:51:43 -0700 (PDT) From: Shaoxuan Yuan To: git@vger.kernel.org Cc: vdye@github.com, derrickstolee@github.com, Shaoxuan Yuan Subject: [PATCH v1 2/4] pathspec.h: move pathspec_needs_expanded_index() from reset.c to here Date: Wed, 3 Aug 2022 12:51:16 +0800 Message-Id: <20220803045118.1243087-3-shaoxuan.yuan02@gmail.com> X-Mailer: git-send-email 2.37.0 In-Reply-To: <20220803045118.1243087-1-shaoxuan.yuan02@gmail.com> References: <20220803045118.1243087-1-shaoxuan.yuan02@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Method pathspec_needs_expanded_index() in reset.c from 4d1cfc1351 (reset: make --mixed sparse-aware, 2021-11-29) is reusable when we need to verify if the index needs to be expanded when the command is utilizing a pathspec rather than a literal path. Move it to pathspec.h for reusability. Add a few items to the function so it can better serve its purpose as a standalone public function: * Add a check in front so if the index is not sparse, return early since no expansion is needed. * Add documentation to the function. Signed-off-by: Shaoxuan Yuan --- builtin/reset.c | 84 +--------------------------------------------- pathspec.c | 89 +++++++++++++++++++++++++++++++++++++++++++++++++ pathspec.h | 12 +++++++ 3 files changed, 102 insertions(+), 83 deletions(-) diff --git a/builtin/reset.c b/builtin/reset.c index 344fff8f3a..fdce6f8c85 100644 --- a/builtin/reset.c +++ b/builtin/reset.c @@ -174,88 +174,6 @@ static void update_index_from_diff(struct diff_queue_struct *q, } } -static int pathspec_needs_expanded_index(const struct pathspec *pathspec) -{ - unsigned int i, pos; - int res = 0; - char *skip_worktree_seen = NULL; - - /* - * When using a magic pathspec, assume for the sake of simplicity that - * the index needs to be expanded to match all matchable files. - */ - if (pathspec->magic) - return 1; - - for (i = 0; i < pathspec->nr; i++) { - struct pathspec_item item = pathspec->items[i]; - - /* - * If the pathspec item has a wildcard, the index should be expanded - * if the pathspec has the possibility of matching a subset of entries inside - * of a sparse directory (but not the entire directory). - * - * If the pathspec item is a literal path, the index only needs to be expanded - * if a) the pathspec isn't in the sparse checkout cone (to make sure we don't - * expand for in-cone files) and b) it doesn't match any sparse directories - * (since we can reset whole sparse directories without expanding them). - */ - if (item.nowildcard_len < item.len) { - /* - * Special case: if the pattern is a path inside the cone - * followed by only wildcards, the pattern cannot match - * partial sparse directories, so we know we don't need to - * expand the index. - * - * Examples: - * - in-cone/foo***: doesn't need expanded index - * - not-in-cone/bar*: may need expanded index - * - **.c: may need expanded index - */ - if (strspn(item.original + item.nowildcard_len, "*") == item.len - item.nowildcard_len && - path_in_cone_mode_sparse_checkout(item.original, &the_index)) - continue; - - for (pos = 0; pos < active_nr; pos++) { - struct cache_entry *ce = active_cache[pos]; - - if (!S_ISSPARSEDIR(ce->ce_mode)) - continue; - - /* - * If the pre-wildcard length is longer than the sparse - * directory name and the sparse directory is the first - * component of the pathspec, need to expand the index. - */ - if (item.nowildcard_len > ce_namelen(ce) && - !strncmp(item.original, ce->name, ce_namelen(ce))) { - res = 1; - break; - } - - /* - * If the pre-wildcard length is shorter than the sparse - * directory and the pathspec does not match the whole - * directory, need to expand the index. - */ - if (!strncmp(item.original, ce->name, item.nowildcard_len) && - wildmatch(item.original, ce->name, 0)) { - res = 1; - break; - } - } - } else if (!path_in_cone_mode_sparse_checkout(item.original, &the_index) && - !matches_skip_worktree(pathspec, i, &skip_worktree_seen)) - res = 1; - - if (res > 0) - break; - } - - free(skip_worktree_seen); - return res; -} - static int read_from_tree(const struct pathspec *pathspec, struct object_id *tree_oid, int intent_to_add) @@ -273,7 +191,7 @@ static int read_from_tree(const struct pathspec *pathspec, opt.change = diff_change; opt.add_remove = diff_addremove; - if (pathspec->nr && the_index.sparse_index && pathspec_needs_expanded_index(pathspec)) + if (pathspec->nr && pathspec_needs_expanded_index(&the_index, pathspec)) ensure_full_index(&the_index); if (do_diff_cache(tree_oid, &opt)) diff --git a/pathspec.c b/pathspec.c index 84ad9c73cf..46e77a85fe 100644 --- a/pathspec.c +++ b/pathspec.c @@ -759,3 +759,92 @@ int match_pathspec_attrs(struct index_state *istate, return 1; } + +int pathspec_needs_expanded_index(struct index_state *istate, + const struct pathspec *pathspec) +{ + unsigned int i, pos; + int res = 0; + char *skip_worktree_seen = NULL; + + /* + * If index is not sparse, no index expansion is needed. + */ + if (!istate->sparse_index) + return 0; + + /* + * When using a magic pathspec, assume for the sake of simplicity that + * the index needs to be expanded to match all matchable files. + */ + if (pathspec->magic) + return 1; + + for (i = 0; i < pathspec->nr; i++) { + struct pathspec_item item = pathspec->items[i]; + + /* + * If the pathspec item has a wildcard, the index should be expanded + * if the pathspec has the possibility of matching a subset of entries inside + * of a sparse directory (but not the entire directory). + * + * If the pathspec item is a literal path, the index only needs to be expanded + * if a) the pathspec isn't in the sparse checkout cone (to make sure we don't + * expand for in-cone files) and b) it doesn't match any sparse directories + * (since we can reset whole sparse directories without expanding them). + */ + if (item.nowildcard_len < item.len) { + /* + * Special case: if the pattern is a path inside the cone + * followed by only wildcards, the pattern cannot match + * partial sparse directories, so we know we don't need to + * expand the index. + * + * Examples: + * - in-cone/foo***: doesn't need expanded index + * - not-in-cone/bar*: may need expanded index + * - **.c: may need expanded index + */ + if (strspn(item.original + item.nowildcard_len, "*") == item.len - item.nowildcard_len && + path_in_cone_mode_sparse_checkout(item.original, istate)) + continue; + + for (pos = 0; pos < istate->cache_nr; pos++) { + struct cache_entry *ce = istate->cache[pos]; + + if (!S_ISSPARSEDIR(ce->ce_mode)) + continue; + + /* + * If the pre-wildcard length is longer than the sparse + * directory name and the sparse directory is the first + * component of the pathspec, need to expand the index. + */ + if (item.nowildcard_len > ce_namelen(ce) && + !strncmp(item.original, ce->name, ce_namelen(ce))) { + res = 1; + break; + } + + /* + * If the pre-wildcard length is shorter than the sparse + * directory and the pathspec does not match the whole + * directory, need to expand the index. + */ + if (!strncmp(item.original, ce->name, item.nowildcard_len) && + wildmatch(item.original, ce->name, 0)) { + res = 1; + break; + } + } + } else if (!path_in_cone_mode_sparse_checkout(item.original, istate) && + !matches_skip_worktree(pathspec, i, &skip_worktree_seen)) + res = 1; + + if (res > 0) + break; + } + + free(skip_worktree_seen); + return res; +} diff --git a/pathspec.h b/pathspec.h index 402ebb8080..41f6adfbb4 100644 --- a/pathspec.h +++ b/pathspec.h @@ -171,4 +171,16 @@ int match_pathspec_attrs(struct index_state *istate, const char *name, int namelen, const struct pathspec_item *item); +/* + * Determine whether a pathspec will match only entire index entries (non-sparse + * files and/or entire sparse directories). If the pathspec has the potential to + * match partial contents of a sparse directory, return 1 to indicate the index + * should be expanded to match the appropriate index entries. + * + * For the sake of simplicity, always return 1 if using a more complex "magic" + * pathspec. + */ +int pathspec_needs_expanded_index(struct index_state *istate, + const struct pathspec *pathspec); + #endif /* PATHSPEC_H */ From patchwork Wed Aug 3 04:51:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shaoxuan Yuan X-Patchwork-Id: 12935191 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 20B65C19F28 for ; Wed, 3 Aug 2022 04:51:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235567AbiHCEvs (ORCPT ); Wed, 3 Aug 2022 00:51:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39938 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235103AbiHCEvq (ORCPT ); Wed, 3 Aug 2022 00:51:46 -0400 Received: from mail-pf1-x434.google.com (mail-pf1-x434.google.com [IPv6:2607:f8b0:4864:20::434]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 50C59E26 for ; Tue, 2 Aug 2022 21:51:46 -0700 (PDT) Received: by mail-pf1-x434.google.com with SMTP id f28so7218924pfk.1 for ; Tue, 02 Aug 2022 21:51:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ZYKysQHj+mvXX6SXRY5wmLCSLUeL9vSxQerANzUuAeA=; b=W+fBEFTr9cMaMxd3J6l7U0dS0ALWLZm2QKxfGV7AMAK1G7Z90IRTFrvvGNjLjswiE2 morWhwVSnGzA4Qev6lK17hbfAYS1ys7DgdUfuYejFVPOjm6htQiIV//dF2OHqhQAef4L b7KUGbUXkQ5bLRer46eL6B8aUHtS4LeWmt1C2g95lBxkwJhT/5etEYx4ZnwzzWBcLfg1 UeVVsoQLuJD5RyvuopmBgrSCS2wnE0yYdFdCOYe2qHuGC/4CEg34BVvjDhY6HsgJ2XpH dwHfzEMJ5fGdBZ/pwoiRWotqDde5MMyUSgk9uagGDtbYQ3vXWe5oJp77tv+dOJx9miUH al6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ZYKysQHj+mvXX6SXRY5wmLCSLUeL9vSxQerANzUuAeA=; b=SeE2SM6/gEVCMy8HbrMT7s4nmgut8arlf0NScPkva2lx+nSguBKFP/C/tCrLyyafBI SXaybgragyMt79Idey0kJM8SbmmlzgoImnfVP4XOGe0hf4I1XgwYaC9EidUo2/63L4Gu xDUuj8PEWORH37V9pnTS1dNbDaCxwDZTX7MMkGvCJ+JCSS8l/isaxgVe3pEg7gSRYiTH qiAxMnoltHJejnhbtoURH/nGnTMNCW3LtHEYFfvIXZnqV0xlBN1ak6U9QPluvBATYxuV Y1Yfw98NauK4QfhbBDvv1kfl3KMzUsx8vfrtO5/x73G95EYvyTB9KN06WvRweo8g9TXA Z+Yw== X-Gm-Message-State: ACgBeo1NWg7iO8R6L4oiCJlEt+PJeuub6g5qBtmlGmSSXKPRwlBx6MbM 0Xkg6WIskxj4YPTxECf2HDYD7mIZbuPVPZBd X-Google-Smtp-Source: AA6agR4bXdrzurTy+8PEdriO7C/wPED0ojue/Za+Ttwb05TjpnXPsDbgFjZt+UbHcGzMIsblZlHITQ== X-Received: by 2002:a63:3143:0:b0:41b:b5dc:e6b6 with SMTP id x64-20020a633143000000b0041bb5dce6b6mr16171973pgx.422.1659502305645; Tue, 02 Aug 2022 21:51:45 -0700 (PDT) Received: from ffyuanda.localdomain ([113.65.228.71]) by smtp.gmail.com with ESMTPSA id f131-20020a623889000000b0052e2a1edab8sm470211pfa.24.2022.08.02.21.51.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Aug 2022 21:51:45 -0700 (PDT) From: Shaoxuan Yuan To: git@vger.kernel.org Cc: vdye@github.com, derrickstolee@github.com, Shaoxuan Yuan Subject: [PATCH v1 3/4] rm: expand the index only when necessary Date: Wed, 3 Aug 2022 12:51:17 +0800 Message-Id: <20220803045118.1243087-4-shaoxuan.yuan02@gmail.com> X-Mailer: git-send-email 2.37.0 In-Reply-To: <20220803045118.1243087-1-shaoxuan.yuan02@gmail.com> References: <20220803045118.1243087-1-shaoxuan.yuan02@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Originally, rm a pathspec that is out-of-cone in a sparse-index environment, Git dies with "pathspec '' did not match any files", mainly because it does not expand the index so nothing is matched. Remove the `ensure_full_index()` method so `git-rm` does not always expand the index when the expansion is unnecessary, i.e. when does not have any possibilities to match anything outside of sparse-checkout definition. Expand the index when the needs an expanded index, i.e. the contains wildcard that may need a full-index or the is simply outside of sparse-checkout definition. Signed-off-by: Shaoxuan Yuan --- builtin/rm.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/builtin/rm.c b/builtin/rm.c index 84a935a16e..58ed924f0d 100644 --- a/builtin/rm.c +++ b/builtin/rm.c @@ -296,8 +296,9 @@ int cmd_rm(int argc, const char **argv, const char *prefix) seen = xcalloc(pathspec.nr, 1); - /* TODO: audit for interaction with sparse-index. */ - ensure_full_index(&the_index); + if (pathspec_needs_expanded_index(&the_index, &pathspec)) + ensure_full_index(&the_index); + for (i = 0; i < active_nr; i++) { const struct cache_entry *ce = active_cache[i]; From patchwork Wed Aug 3 04:51:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shaoxuan Yuan X-Patchwork-Id: 12935193 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 33617C19F28 for ; Wed, 3 Aug 2022 04:51:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235574AbiHCEv4 (ORCPT ); Wed, 3 Aug 2022 00:51:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40008 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229457AbiHCEvw (ORCPT ); Wed, 3 Aug 2022 00:51:52 -0400 Received: from mail-pf1-x434.google.com (mail-pf1-x434.google.com [IPv6:2607:f8b0:4864:20::434]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2B653DFD2 for ; Tue, 2 Aug 2022 21:51:48 -0700 (PDT) Received: by mail-pf1-x434.google.com with SMTP id u133so8794073pfc.10 for ; Tue, 02 Aug 2022 21:51:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=GbMtIN7CsnhBPR+4zS7TVgnlzrUAp+IhVUnvo90SnK8=; b=iqU6OZXyFPnSkRNCYSm31miszyyW2zYnkT1Zi7DQH2FWgLBimKi/9dv/cU70rYnlMK HPvhGC8s7rs6GmHoOLoT3M6/ys4rsY0HPMyycK+Eph2rP4W5J27afpo66OhSPkiVtvxg C9EI9c3BopkEYpJLTQjRU3B+2WyV6cB/5VZADs8BGycD99SOpVKIE8Qmw/tLVpMv52hH PqaHIRT3QGmUBHOA0Zoiylf8zc7DWdZC4hokO9zFg7XVyWITE2WZ29Nnatx+uR+Bblk+ zfZQIe5jMIXbTuJG+stukqd7DRDMJBg0/D1dk9QJVsoUiLqc9bEBuL0HdSOcj7NOXGvc s58g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=GbMtIN7CsnhBPR+4zS7TVgnlzrUAp+IhVUnvo90SnK8=; b=ZIGRGvZ++sRrvOb9uV2MNPlj6ht2o3LU1BlFRN7rUn7IX3mAs1rA0cBkVg3rG5UPWm QQFMM2CH9F2MnPMrNl1nM2EgE/9BZ6zjJ2h8VYXkzrRm+DHUOK/mlG/pIuHhk6PxQuKE Rfco284718JEt9cdodGtXyQeMQxphnDn5IlhPwqurfetnfeHBE4YLKARHifSmGwfZMgC ylU92lWR2BYnvCr0OYZ5NOwTCbWUDFRdZhHwVp39M7x1w9kVfXWswBpPSGjAQfuGGGLf HfZNEMUuUBLkQ4rIZymcNY1FOsZd73IOn98iBsmpyL4zILHSm4TLaZa1jB5wQwe9xH/U x5cA== X-Gm-Message-State: ACgBeo0iIauDOfJE1H4wlx7f+L/Qkx/roYA6ftpr3qhNjB24EkVjj/4k oxmSWPMie5rhYcPP9fXKD9IrXSbB9/4u0Sdh X-Google-Smtp-Source: AA6agR5tdn4NdiYOx1R23P5VO+cUjNGWGMttzclZDDpptbnutrNiFeoqXLpwRDU8EWYu6GdHlk30Og== X-Received: by 2002:a62:cd8c:0:b0:52d:a632:a8e0 with SMTP id o134-20020a62cd8c000000b0052da632a8e0mr10050412pfg.52.1659502307500; Tue, 02 Aug 2022 21:51:47 -0700 (PDT) Received: from ffyuanda.localdomain ([113.65.228.71]) by smtp.gmail.com with ESMTPSA id f131-20020a623889000000b0052e2a1edab8sm470211pfa.24.2022.08.02.21.51.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Aug 2022 21:51:47 -0700 (PDT) From: Shaoxuan Yuan To: git@vger.kernel.org Cc: vdye@github.com, derrickstolee@github.com, Shaoxuan Yuan Subject: [PATCH v1 4/4] rm: integrate with sparse-index Date: Wed, 3 Aug 2022 12:51:18 +0800 Message-Id: <20220803045118.1243087-5-shaoxuan.yuan02@gmail.com> X-Mailer: git-send-email 2.37.0 In-Reply-To: <20220803045118.1243087-1-shaoxuan.yuan02@gmail.com> References: <20220803045118.1243087-1-shaoxuan.yuan02@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Enable the sparse index within the `git-rm` command. The `p2000` tests demonstrate a ~96% execution time reduction for 'git rm' using a sparse index. Test before after ------------------------------------------------------------- 2000.74: git rm -f f2/f4/a (full-v3) 0.66 0.88 +33.0% 2000.75: git rm -f f2/f4/a (full-v4) 0.67 0.75 +12.0% 2000.76: git rm -f f2/f4/a (sparse-v3) 1.99 0.08 -96.0% 2000.77: git rm -f f2/f4/a (sparse-v4) 2.06 0.07 -96.6% Also, normalize a behavioral difference of `git-rm` under sparse-index. See related discussion [1]. `git-rm` a sparse-directory entry within a sparse-index enabled repo behaves differently from a sparse directory within a sparse-checkout enabled repo. For example, in a sparse-index repo, where 'folder1' is a sparse-directory entry, `git rm -r --sparse folder1` provides this: rm 'folder1/' Whereas in a sparse-checkout repo *without* sparse-index, doing so provides this: rm 'folder1/0/0/0' rm 'folder1/0/1' rm 'folder1/a' Because `git rm` a sparse-directory entry does not need to expand the index, therefore we should accept the current behavior, which is faster than "expand the sparse-directory entry to match the sparse-checkout situation". Modify a previous test so such difference is not considered as an error. [1] https://github.com/ffyuanda/git/pull/6#discussion_r934861398 Signed-off-by: Shaoxuan Yuan --- builtin/rm.c | 2 ++ t/perf/p2000-sparse-operations.sh | 1 + t/t1092-sparse-checkout-compatibility.sh | 6 +++--- 3 files changed, 6 insertions(+), 3 deletions(-) diff --git a/builtin/rm.c b/builtin/rm.c index 58ed924f0d..b6ba859fe4 100644 --- a/builtin/rm.c +++ b/builtin/rm.c @@ -287,6 +287,8 @@ int cmd_rm(int argc, const char **argv, const char *prefix) if (!index_only) setup_work_tree(); + prepare_repo_settings(the_repository); + the_repository->settings.command_requires_full_index = 0; hold_locked_index(&lock_file, LOCK_DIE_ON_ERROR); if (read_cache() < 0) diff --git a/t/perf/p2000-sparse-operations.sh b/t/perf/p2000-sparse-operations.sh index c181110a43..853513eb9b 100755 --- a/t/perf/p2000-sparse-operations.sh +++ b/t/perf/p2000-sparse-operations.sh @@ -123,5 +123,6 @@ test_perf_on_all git blame $SPARSE_CONE/f3/a test_perf_on_all git read-tree -mu HEAD test_perf_on_all git checkout-index -f --all test_perf_on_all git update-index --add --remove $SPARSE_CONE/a +test_perf_on_all git rm -f $SPARSE_CONE/a test_done diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index 75649e3265..58632fe483 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -912,7 +912,7 @@ test_expect_success 'read-tree --prefix' ' test_all_match git read-tree --prefix=deep/deeper1/deepest -u deepest && test_all_match git status --porcelain=v2 && - test_all_match git rm -rf --sparse folder1/ && + run_on_all git rm -rf --sparse folder1/ && test_all_match git read-tree --prefix=folder1/ -u update-folder1 && test_all_match git status --porcelain=v2 && @@ -1870,7 +1870,7 @@ test_expect_success 'rm pathspec inside sparse definition' ' test_all_match git status --porcelain=v2 ' -test_expect_failure 'rm pathspec outside sparse definition' ' +test_expect_success 'rm pathspec outside sparse definition' ' init_repos && for file in folder1/a folder1/0/1 @@ -1910,7 +1910,7 @@ test_expect_failure 'rm pathspec outside sparse definition' ' test_sparse_match git status --porcelain=v2 ' -test_expect_failure 'sparse index is not expanded: rm' ' +test_expect_success 'sparse index is not expanded: rm' ' init_repos && ensure_not_expanded rm deep/a &&