From patchwork Mon Apr 12 21:07:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12198805 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A3ACDC433B4 for ; Mon, 12 Apr 2021 21:08:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7DC6260238 for ; Mon, 12 Apr 2021 21:08:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239314AbhDLVIl (ORCPT ); Mon, 12 Apr 2021 17:08:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57942 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238045AbhDLVIj (ORCPT ); Mon, 12 Apr 2021 17:08:39 -0400 Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5729DC061574 for ; Mon, 12 Apr 2021 14:08:21 -0700 (PDT) Received: by mail-wm1-x334.google.com with SMTP id u20so3045791wmj.0 for ; Mon, 12 Apr 2021 14:08:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=EEiLnZcxBnalSrtS0KgvDP7CGUo9RluvCjn0alofKlM=; b=DEv4MpHDAXi4Z9lSP0rZV50luh+cl1axctVF2OEN/bdmJbNB+Ouiu1VYD3t7mfg7CZ cSW3KNVRm1H01IjN9od+weO1HPWgwMc1U52nyRUWEcAyqG0biDkySb5kzoIoWdjVL/Ea snqJ2Te1Zv4I5DHlFHZ80x2k+nn6HX4PJC892eFrfwCrMycJNpfbQHEtvyU/6gIYzaXf Kfr6jWIeKcxp9MhewdcjcWD9o+PgiqP5xQqXMZt+CIBi7ByT5BG0c4fz/1njI8AjHs/q 6AwKATxSHNj00DR4rK2roiwOxkfzZtZXhwmw+EUe8MzuLiQXPPjcmyt51F21yZzZtF2M j7pg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=EEiLnZcxBnalSrtS0KgvDP7CGUo9RluvCjn0alofKlM=; b=dNQgahukUKQiap9ceQ/mjBqra2OplZkub2cfAmz8Bw+hw/66/3Zr2J9KFnph9exBEz 7AL0qTPI7FrHsFPL212cvmkpPvnvwKipwheYOxZi0Zt9FwPS9o/2Kc349E/ke+NWnTf9 rggAkuS9Ia3NkNcieCt5kwshyC7Zp73OZTNny5oWp4el8Yb80Bai3u3vA9bEMWl4uGwQ Vdlx0J2Xx258BT40AFFI0ra15smUYBo+Js2bMwepstyFTowReQmvEsMtB0NsaKHrdyO7 KTOrGs6Axefs5GwzJB1lgfmOXDH/WudKKFrYiS2ZaAnSVgSHAmtsgqNiJCA1hg5cPG2B qZzA== X-Gm-Message-State: AOAM530r3yKA2yLVMzdUbjjMwpaGuTHeiqMELMP1kW/yW96f29e2h5Ba QM7no+yPj6ySj7VTJGb3koQ8zI6+gsA= X-Google-Smtp-Source: ABdhPJz8xWQSCYbD39XPvS3coQOrqSwdQK/RUsMJSCIsd26k2MEnL6JkpX2PUu11s628/ayC8F+0Ew== X-Received: by 2002:a1c:4e0b:: with SMTP id g11mr866031wmh.149.1618261699983; Mon, 12 Apr 2021 14:08:19 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id n6sm18642133wrt.22.2021.04.12.14.08.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Apr 2021 14:08:19 -0700 (PDT) Message-Id: <4731f610ba6e70a271126e513f8df46d4e0cc5c6.1618261697.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 12 Apr 2021 21:07:52 +0000 Subject: [PATCH v3 01/26] sparse-index: API protection strategy Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Edit and expand the sparse-index design document with the plan for guarding index operations with ensure_full_index(). Notably, the plan has changed to not have an expand_to_path() method in favor of checking for a sparse-directory hit inside of the index_path_pos() API. The changes that follow this one will incrementally add ensure_full_index() guards to iterations over all cache entries. Some iterations over the cache entries are not protected due to a few categories listed in the document. Since these are not being modified, here is a short list of the files and methods that will not receive these guards: Looking for non-zero stage: * builtin/add.c:chmod_pathspec() * builtin/merge.c:count_unmerged_entries() * merge-ort.c:record_conflicted_index_entries() * read-cache.c:unmerged_index() * rerere.c:check_one_conflict(), find_conflict(), rerere_remaining() * revision.c:prepare_show_merge() * sequencer.c:append_conflicts_hint() * wt-status.c:wt_status_collect_changes_initial() Looking for submodules: * builtin/submodule--helper.c:module_list_compute() * submodule.c: several methods * worktree.c:validate_no_submodules() Part of the index API: * name-hash.c: lazy init methods * preload-index.c:preload_thread(), preload_index() * read-cache.c: file format methods Checking for correct order of cache entries: * read-cache.c:check_ce_order() Ignores SKIP_WORKTREE entries or already aware: * unpack-trees.c:mark_new_skip_worktree() * wt-status.c:wt_status_check_sparse_checkout() Signed-off-by: Derrick Stolee --- Documentation/technical/sparse-index.txt | 37 ++++++++++++++++++++++-- 1 file changed, 35 insertions(+), 2 deletions(-) diff --git a/Documentation/technical/sparse-index.txt b/Documentation/technical/sparse-index.txt index 8d3d80804604..3b24c1a219f8 100644 --- a/Documentation/technical/sparse-index.txt +++ b/Documentation/technical/sparse-index.txt @@ -85,8 +85,41 @@ index, as well. Next, consumers of the index will be guarded against operating on a sparse-index by inserting calls to `ensure_full_index()` or -`expand_index_to_path()`. After these guards are in place, we can begin -leaving sparse-directory entries in the in-memory index structure. +`expand_index_to_path()`. If a specific path is requested, then those will +be protected from within the `index_file_exists()` and `index_name_pos()` +API calls: they will call `ensure_full_index()` if necessary. The +intention here is to preserve existing behavior when interacting with a +sparse-checkout. We don't want a change to happen by accident, without +tests. Many of these locations may not need any change before removing the +guards, but we should not do so without tests to ensure the expected +behavior happens. + +It may be desirable to _change_ the behavior of some commands in the +presence of a sparse index or more generally in any sparse-checkout +scenario. In such cases, these should be carefully communicated and +tested. No such behavior changes are intended during this phase. + +During a scan of the codebase, not every iteration of the cache entries +needs an `ensure_full_index()` check. The basic reasons include: + +1. The loop is scanning for entries with non-zero stage. These entries + are not collapsed into a sparse-directory entry. + +2. The loop is scanning for submodules. These entries are not collapsed + into a sparse-directory entry. + +3. The loop is part of the index API, especially around reading or + writing the format. + +4. The loop is checking for correct order of cache entries and that is + correct if and only if the sparse-directory entries are in the correct + location. + +5. The loop ignores entries with the `SKIP_WORKTREE` bit set, or is + otherwise already aware of sparse directory entries. + +6. The sparse-index is disabled at this point when using the split-index + feature, so no effort is made to protect the split-index API. Even after inserting these guards, we will keep expanding sparse-indexes for most Git commands using the `command_requires_full_index` repository From patchwork Mon Apr 12 21:07:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12198811 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E2435C433ED for ; Mon, 12 Apr 2021 21:08:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AAF60610CE for ; Mon, 12 Apr 2021 21:08:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240456AbhDLVIt (ORCPT ); Mon, 12 Apr 2021 17:08:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57948 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238828AbhDLVIl (ORCPT ); Mon, 12 Apr 2021 17:08:41 -0400 Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 010FCC061574 for ; Mon, 12 Apr 2021 14:08:22 -0700 (PDT) Received: by mail-wr1-x434.google.com with SMTP id m9so1588085wrx.3 for ; Mon, 12 Apr 2021 14:08:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=5UGhbG5cF/mdIR/eQy+KY1VVNkBNL/pS5IaECgqGc0E=; b=tzgNY12inDsaXMXlTAvclnLJMQoXiuSK5JkQ0gm4DL5tpJF2kn7f0FcAxqQeEWbKKR earfx2RHD+Cv9GqyAxVNE2Nra2c8KeSNBD9ERStZeIk4zQ9wwBuTF3j2qQLrB3MAhvin /e34r4NEHTeJqQpGMSAWL3EQbAKPuti4wxkqx/4Fh2QJZTo5e2ssh6CDZQbWu99wAUOK dc79P97NbgwtvcWirtVoac/3GJXixY25un6nwsTZhSX4uUnW5LTZUEp2fwbQF1huUIAz nvyYwR1wMhtnYPcSpbbNM/xbJOAcH125QEbGFpIz1VfwVA5LvuLN5Mp6vsmCtvbuIlZu wZ5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=5UGhbG5cF/mdIR/eQy+KY1VVNkBNL/pS5IaECgqGc0E=; b=taVHbwDtOhWMkzh0koIvwDdltGd+H19LZvD6l4pFd/f8nPCYBmbLQePLfGQqInMvGO S/Y36hMSsiyyBvtCFC5gDB4As5PtD4VqTqQaiNd6ds9mp+EB0jhBLJ+hE6VpCr9VvQ9k PI7yV/9CPYXC4yeqWpwuk0/4KwHCkcmuhP42tRVnBS4vJyB2BBcTTb3A7o2WcTGRJfbO ER8LpmO1byCSBpWbGs3QCcNIgnFzD6pswakdcszlpaDVDkZpwajF9wWj89n/Ci/dtEz5 gxmPPALxWLXAyz4abipE6bpL/kYPhXqgjWbsGHalj4/VeTQqJIMFdojnlsCfxDxyBHYk rl8w== X-Gm-Message-State: AOAM530RzE8iWYa/fDvyl7urfXwum/9CpSfyi2EL8OBQzxF+322UK+gM kJFz6gXvbiKQfPKd2bK599U0/kZqyvA= X-Google-Smtp-Source: ABdhPJwKKJXoA7g8Akz0Y2Qy6nVup3T2Du4G3DXklAzggyce6Wq7lFebNyKkbcVThxUiQ1wC6Jg1RA== X-Received: by 2002:adf:f38e:: with SMTP id m14mr12662662wro.34.1618261700643; Mon, 12 Apr 2021 14:08:20 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id f12sm19088691wrr.61.2021.04.12.14.08.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Apr 2021 14:08:20 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Mon, 12 Apr 2021 21:07:53 +0000 Subject: [PATCH v3 02/26] *: remove 'const' qualifier for struct index_state Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Several methods specify that they take a 'struct index_state' pointer with the 'const' qualifier because they intend to only query the data, not change it. However, we will be introducing a step very low in the method stack that might modify a sparse-index to become a full index in the case that our queries venture inside a sparse-directory entry. This change only removes the 'const' qualifiers that are necessary for the following change which will actually modify the implementation of index_name_stage_pos(). Signed-off-by: Derrick Stolee --- attr.c | 14 +++++++------- attr.h | 4 ++-- builtin/ls-files.c | 10 +++++----- cache.h | 6 +++--- convert.c | 26 +++++++++++++------------- convert.h | 20 ++++++++++---------- dir.c | 12 ++++++------ dir.h | 8 ++++---- merge-recursive.c | 2 +- pathspec.c | 6 +++--- pathspec.h | 6 +++--- read-cache.c | 10 +++++----- submodule.c | 6 +++--- submodule.h | 6 +++--- 14 files changed, 68 insertions(+), 68 deletions(-) diff --git a/attr.c b/attr.c index 4ef85d668b54..8de553293ee4 100644 --- a/attr.c +++ b/attr.c @@ -718,7 +718,7 @@ static struct attr_stack *read_attr_from_file(const char *path, int macro_ok) return res; } -static struct attr_stack *read_attr_from_index(const struct index_state *istate, +static struct attr_stack *read_attr_from_index(struct index_state *istate, const char *path, int macro_ok) { @@ -748,7 +748,7 @@ static struct attr_stack *read_attr_from_index(const struct index_state *istate, return res; } -static struct attr_stack *read_attr(const struct index_state *istate, +static struct attr_stack *read_attr(struct index_state *istate, const char *path, int macro_ok) { struct attr_stack *res = NULL; @@ -840,7 +840,7 @@ static void push_stack(struct attr_stack **attr_stack_p, } } -static void bootstrap_attr_stack(const struct index_state *istate, +static void bootstrap_attr_stack(struct index_state *istate, struct attr_stack **stack) { struct attr_stack *e; @@ -878,7 +878,7 @@ static void bootstrap_attr_stack(const struct index_state *istate, push_stack(stack, e, NULL, 0); } -static void prepare_attr_stack(const struct index_state *istate, +static void prepare_attr_stack(struct index_state *istate, const char *path, int dirlen, struct attr_stack **stack) { @@ -1078,7 +1078,7 @@ static void determine_macros(struct all_attrs_item *all_attrs, * If check->check_nr is non-zero, only attributes in check[] are collected. * Otherwise all attributes are collected. */ -static void collect_some_attrs(const struct index_state *istate, +static void collect_some_attrs(struct index_state *istate, const char *path, struct attr_check *check) { @@ -1107,7 +1107,7 @@ static void collect_some_attrs(const struct index_state *istate, fill(path, pathlen, basename_offset, check->stack, check->all_attrs, rem); } -void git_check_attr(const struct index_state *istate, +void git_check_attr(struct index_state *istate, const char *path, struct attr_check *check) { @@ -1124,7 +1124,7 @@ void git_check_attr(const struct index_state *istate, } } -void git_all_attrs(const struct index_state *istate, +void git_all_attrs(struct index_state *istate, const char *path, struct attr_check *check) { int i; diff --git a/attr.h b/attr.h index 404548f028a8..3732505edae8 100644 --- a/attr.h +++ b/attr.h @@ -190,14 +190,14 @@ void attr_check_free(struct attr_check *check); */ const char *git_attr_name(const struct git_attr *); -void git_check_attr(const struct index_state *istate, +void git_check_attr(struct index_state *istate, const char *path, struct attr_check *check); /* * Retrieve all attributes that apply to the specified path. * check holds the attributes and their values. */ -void git_all_attrs(const struct index_state *istate, +void git_all_attrs(struct index_state *istate, const char *path, struct attr_check *check); enum git_attr_direction { diff --git a/builtin/ls-files.c b/builtin/ls-files.c index 60a2913a01e9..4f9ed1fb29b7 100644 --- a/builtin/ls-files.c +++ b/builtin/ls-files.c @@ -57,7 +57,7 @@ static const char *tag_modified = ""; static const char *tag_skip_worktree = ""; static const char *tag_resolve_undo = ""; -static void write_eolinfo(const struct index_state *istate, +static void write_eolinfo(struct index_state *istate, const struct cache_entry *ce, const char *path) { if (show_eol) { @@ -122,7 +122,7 @@ static void print_debug(const struct cache_entry *ce) } } -static void show_dir_entry(const struct index_state *istate, +static void show_dir_entry(struct index_state *istate, const char *tag, struct dir_entry *ent) { int len = max_prefix_len; @@ -139,7 +139,7 @@ static void show_dir_entry(const struct index_state *istate, write_name(ent->name); } -static void show_other_files(const struct index_state *istate, +static void show_other_files(struct index_state *istate, const struct dir_struct *dir) { int i; @@ -152,7 +152,7 @@ static void show_other_files(const struct index_state *istate, } } -static void show_killed_files(const struct index_state *istate, +static void show_killed_files(struct index_state *istate, const struct dir_struct *dir) { int i; @@ -254,7 +254,7 @@ static void show_ce(struct repository *repo, struct dir_struct *dir, } } -static void show_ru_info(const struct index_state *istate) +static void show_ru_info(struct index_state *istate) { struct string_list_item *item; diff --git a/cache.h b/cache.h index 8aede373aeb3..5006278c13ca 100644 --- a/cache.h +++ b/cache.h @@ -800,7 +800,7 @@ struct cache_entry *index_file_exists(struct index_state *istate, const char *na * index_name_pos(&index, "f", 1) -> -3 * index_name_pos(&index, "g", 1) -> -5 */ -int index_name_pos(const struct index_state *, const char *name, int namelen); +int index_name_pos(struct index_state *, const char *name, int namelen); /* * Some functions return the negative complement of an insert position when a @@ -850,8 +850,8 @@ int add_file_to_index(struct index_state *, const char *path, int flags); int chmod_index_entry(struct index_state *, struct cache_entry *ce, char flip); int ce_same_name(const struct cache_entry *a, const struct cache_entry *b); void set_object_name_for_intent_to_add_entry(struct cache_entry *ce); -int index_name_is_other(const struct index_state *, const char *, int); -void *read_blob_data_from_index(const struct index_state *, const char *, unsigned long *); +int index_name_is_other(struct index_state *, const char *, int); +void *read_blob_data_from_index(struct index_state *, const char *, unsigned long *); /* do stat comparison even if CE_VALID is true */ #define CE_MATCH_IGNORE_VALID 01 diff --git a/convert.c b/convert.c index ee360c2f07ce..20b19abffa36 100644 --- a/convert.c +++ b/convert.c @@ -138,7 +138,7 @@ static const char *gather_convert_stats_ascii(const char *data, unsigned long si } } -const char *get_cached_convert_stats_ascii(const struct index_state *istate, +const char *get_cached_convert_stats_ascii(struct index_state *istate, const char *path) { const char *ret; @@ -222,7 +222,7 @@ static void check_global_conv_flags_eol(const char *path, } } -static int has_crlf_in_index(const struct index_state *istate, const char *path) +static int has_crlf_in_index(struct index_state *istate, const char *path) { unsigned long sz; void *data; @@ -496,7 +496,7 @@ static int encode_to_worktree(const char *path, const char *src, size_t src_len, return 1; } -static int crlf_to_git(const struct index_state *istate, +static int crlf_to_git(struct index_state *istate, const char *path, const char *src, size_t len, struct strbuf *buf, enum crlf_action crlf_action, int conv_flags) @@ -1307,7 +1307,7 @@ struct conv_attrs { static struct attr_check *check; -static void convert_attrs(const struct index_state *istate, +static void convert_attrs(struct index_state *istate, struct conv_attrs *ca, const char *path) { struct attr_check_item *ccheck = NULL; @@ -1369,7 +1369,7 @@ void reset_parsed_attributes(void) user_convert_tail = NULL; } -int would_convert_to_git_filter_fd(const struct index_state *istate, const char *path) +int would_convert_to_git_filter_fd(struct index_state *istate, const char *path) { struct conv_attrs ca; @@ -1388,7 +1388,7 @@ int would_convert_to_git_filter_fd(const struct index_state *istate, const char return apply_filter(path, NULL, 0, -1, NULL, ca.drv, CAP_CLEAN, NULL, NULL); } -const char *get_convert_attr_ascii(const struct index_state *istate, const char *path) +const char *get_convert_attr_ascii(struct index_state *istate, const char *path) { struct conv_attrs ca; @@ -1414,7 +1414,7 @@ const char *get_convert_attr_ascii(const struct index_state *istate, const char return ""; } -int convert_to_git(const struct index_state *istate, +int convert_to_git(struct index_state *istate, const char *path, const char *src, size_t len, struct strbuf *dst, int conv_flags) { @@ -1448,7 +1448,7 @@ int convert_to_git(const struct index_state *istate, return ret | ident_to_git(src, len, dst, ca.ident); } -void convert_to_git_filter_fd(const struct index_state *istate, +void convert_to_git_filter_fd(struct index_state *istate, const char *path, int fd, struct strbuf *dst, int conv_flags) { @@ -1466,7 +1466,7 @@ void convert_to_git_filter_fd(const struct index_state *istate, ident_to_git(dst->buf, dst->len, dst, ca.ident); } -static int convert_to_working_tree_internal(const struct index_state *istate, +static int convert_to_working_tree_internal(struct index_state *istate, const char *path, const char *src, size_t len, struct strbuf *dst, int normalizing, @@ -1510,7 +1510,7 @@ static int convert_to_working_tree_internal(const struct index_state *istate, return ret | ret_filter; } -int async_convert_to_working_tree(const struct index_state *istate, +int async_convert_to_working_tree(struct index_state *istate, const char *path, const char *src, size_t len, struct strbuf *dst, const struct checkout_metadata *meta, @@ -1519,7 +1519,7 @@ int async_convert_to_working_tree(const struct index_state *istate, return convert_to_working_tree_internal(istate, path, src, len, dst, 0, meta, dco); } -int convert_to_working_tree(const struct index_state *istate, +int convert_to_working_tree(struct index_state *istate, const char *path, const char *src, size_t len, struct strbuf *dst, const struct checkout_metadata *meta) @@ -1527,7 +1527,7 @@ int convert_to_working_tree(const struct index_state *istate, return convert_to_working_tree_internal(istate, path, src, len, dst, 0, meta, NULL); } -int renormalize_buffer(const struct index_state *istate, const char *path, +int renormalize_buffer(struct index_state *istate, const char *path, const char *src, size_t len, struct strbuf *dst) { int ret = convert_to_working_tree_internal(istate, path, src, len, dst, 1, NULL, NULL); @@ -1964,7 +1964,7 @@ static struct stream_filter *ident_filter(const struct object_id *oid) * Note that you would be crazy to set CRLF, smudge/clean or ident to a * large binary blob you would want us not to slurp into the memory! */ -struct stream_filter *get_stream_filter(const struct index_state *istate, +struct stream_filter *get_stream_filter(struct index_state *istate, const char *path, const struct object_id *oid) { diff --git a/convert.h b/convert.h index e29d1026a686..0f7c8a1f0446 100644 --- a/convert.h +++ b/convert.h @@ -65,41 +65,41 @@ struct checkout_metadata { extern enum eol core_eol; extern char *check_roundtrip_encoding; -const char *get_cached_convert_stats_ascii(const struct index_state *istate, +const char *get_cached_convert_stats_ascii(struct index_state *istate, const char *path); const char *get_wt_convert_stats_ascii(const char *path); -const char *get_convert_attr_ascii(const struct index_state *istate, +const char *get_convert_attr_ascii(struct index_state *istate, const char *path); /* returns 1 if *dst was used */ -int convert_to_git(const struct index_state *istate, +int convert_to_git(struct index_state *istate, const char *path, const char *src, size_t len, struct strbuf *dst, int conv_flags); -int convert_to_working_tree(const struct index_state *istate, +int convert_to_working_tree(struct index_state *istate, const char *path, const char *src, size_t len, struct strbuf *dst, const struct checkout_metadata *meta); -int async_convert_to_working_tree(const struct index_state *istate, +int async_convert_to_working_tree(struct index_state *istate, const char *path, const char *src, size_t len, struct strbuf *dst, const struct checkout_metadata *meta, void *dco); int async_query_available_blobs(const char *cmd, struct string_list *available_paths); -int renormalize_buffer(const struct index_state *istate, +int renormalize_buffer(struct index_state *istate, const char *path, const char *src, size_t len, struct strbuf *dst); -static inline int would_convert_to_git(const struct index_state *istate, +static inline int would_convert_to_git(struct index_state *istate, const char *path) { return convert_to_git(istate, path, NULL, 0, NULL, 0); } /* Precondition: would_convert_to_git_filter_fd(path) == true */ -void convert_to_git_filter_fd(const struct index_state *istate, +void convert_to_git_filter_fd(struct index_state *istate, const char *path, int fd, struct strbuf *dst, int conv_flags); -int would_convert_to_git_filter_fd(const struct index_state *istate, +int would_convert_to_git_filter_fd(struct index_state *istate, const char *path); /* @@ -133,7 +133,7 @@ void reset_parsed_attributes(void); struct stream_filter; /* opaque */ -struct stream_filter *get_stream_filter(const struct index_state *istate, +struct stream_filter *get_stream_filter(struct index_state *istate, const char *path, const struct object_id *); void free_stream_filter(struct stream_filter *); diff --git a/dir.c b/dir.c index fd8aa7c40faa..5b00dfb5b144 100644 --- a/dir.c +++ b/dir.c @@ -306,7 +306,7 @@ static int do_read_blob(const struct object_id *oid, struct oid_stat *oid_stat, * [1] Only if DO_MATCH_DIRECTORY is passed; otherwise, this is NOT a match. * [2] Only if DO_MATCH_LEADING_PATHSPEC is passed; otherwise, not a match. */ -static int match_pathspec_item(const struct index_state *istate, +static int match_pathspec_item(struct index_state *istate, const struct pathspec_item *item, int prefix, const char *name, int namelen, unsigned flags) { @@ -429,7 +429,7 @@ static int match_pathspec_item(const struct index_state *istate, * pathspec did not match any names, which could indicate that the * user mistyped the nth pathspec. */ -static int do_match_pathspec(const struct index_state *istate, +static int do_match_pathspec(struct index_state *istate, const struct pathspec *ps, const char *name, int namelen, int prefix, char *seen, @@ -500,7 +500,7 @@ static int do_match_pathspec(const struct index_state *istate, return retval; } -static int match_pathspec_with_flags(const struct index_state *istate, +static int match_pathspec_with_flags(struct index_state *istate, const struct pathspec *ps, const char *name, int namelen, int prefix, char *seen, unsigned flags) @@ -516,7 +516,7 @@ static int match_pathspec_with_flags(const struct index_state *istate, return negative ? 0 : positive; } -int match_pathspec(const struct index_state *istate, +int match_pathspec(struct index_state *istate, const struct pathspec *ps, const char *name, int namelen, int prefix, char *seen, int is_dir) @@ -529,7 +529,7 @@ int match_pathspec(const struct index_state *istate, /** * Check if a submodule is a superset of the pathspec */ -int submodule_path_match(const struct index_state *istate, +int submodule_path_match(struct index_state *istate, const struct pathspec *ps, const char *submodule_name, char *seen) @@ -892,7 +892,7 @@ void add_pattern(const char *string, const char *base, add_pattern_to_hashsets(pl, pattern); } -static int read_skip_worktree_file_from_index(const struct index_state *istate, +static int read_skip_worktree_file_from_index(struct index_state *istate, const char *path, size_t *size_out, char **data_out, struct oid_stat *oid_stat) diff --git a/dir.h b/dir.h index facfae47402a..51cb0e217247 100644 --- a/dir.h +++ b/dir.h @@ -354,7 +354,7 @@ int count_slashes(const char *s); int simple_length(const char *match); int no_wildcard(const char *string); char *common_prefix(const struct pathspec *pathspec); -int match_pathspec(const struct index_state *istate, +int match_pathspec(struct index_state *istate, const struct pathspec *pathspec, const char *name, int namelen, int prefix, char *seen, int is_dir); @@ -492,12 +492,12 @@ int git_fnmatch(const struct pathspec_item *item, const char *pattern, const char *string, int prefix); -int submodule_path_match(const struct index_state *istate, +int submodule_path_match(struct index_state *istate, const struct pathspec *ps, const char *submodule_name, char *seen); -static inline int ce_path_match(const struct index_state *istate, +static inline int ce_path_match(struct index_state *istate, const struct cache_entry *ce, const struct pathspec *pathspec, char *seen) @@ -506,7 +506,7 @@ static inline int ce_path_match(const struct index_state *istate, S_ISDIR(ce->ce_mode) || S_ISGITLINK(ce->ce_mode)); } -static inline int dir_path_match(const struct index_state *istate, +static inline int dir_path_match(struct index_state *istate, const struct dir_entry *ent, const struct pathspec *pathspec, int prefix, char *seen) diff --git a/merge-recursive.c b/merge-recursive.c index 3d9207455b3a..b8de7a704eae 100644 --- a/merge-recursive.c +++ b/merge-recursive.c @@ -2987,7 +2987,7 @@ static int blob_unchanged(struct merge_options *opt, struct strbuf obuf = STRBUF_INIT; struct strbuf abuf = STRBUF_INIT; int ret = 0; /* assume changed for safety */ - const struct index_state *idx = opt->repo->index; + struct index_state *idx = opt->repo->index; if (a->mode != o->mode) return 0; diff --git a/pathspec.c b/pathspec.c index 7a229d8d22f2..b6e333965cb4 100644 --- a/pathspec.c +++ b/pathspec.c @@ -20,7 +20,7 @@ * to use find_pathspecs_matching_against_index() instead. */ void add_pathspec_matches_against_index(const struct pathspec *pathspec, - const struct index_state *istate, + struct index_state *istate, char *seen) { int num_unmatched = 0, i; @@ -51,7 +51,7 @@ void add_pathspec_matches_against_index(const struct pathspec *pathspec, * given pathspecs achieves against all items in the index. */ char *find_pathspecs_matching_against_index(const struct pathspec *pathspec, - const struct index_state *istate) + struct index_state *istate) { char *seen = xcalloc(pathspec->nr, 1); add_pathspec_matches_against_index(pathspec, istate, seen); @@ -702,7 +702,7 @@ void clear_pathspec(struct pathspec *pathspec) pathspec->nr = 0; } -int match_pathspec_attrs(const struct index_state *istate, +int match_pathspec_attrs(struct index_state *istate, const char *name, int namelen, const struct pathspec_item *item) { diff --git a/pathspec.h b/pathspec.h index 454ce364fac7..2ccc8080b6cc 100644 --- a/pathspec.h +++ b/pathspec.h @@ -150,11 +150,11 @@ static inline int ps_strcmp(const struct pathspec_item *item, } void add_pathspec_matches_against_index(const struct pathspec *pathspec, - const struct index_state *istate, + struct index_state *istate, char *seen); char *find_pathspecs_matching_against_index(const struct pathspec *pathspec, - const struct index_state *istate); -int match_pathspec_attrs(const struct index_state *istate, + struct index_state *istate); +int match_pathspec_attrs(struct index_state *istate, const char *name, int namelen, const struct pathspec_item *item); diff --git a/read-cache.c b/read-cache.c index 2410e6e0df13..3ad94578095e 100644 --- a/read-cache.c +++ b/read-cache.c @@ -549,7 +549,7 @@ int cache_name_stage_compare(const char *name1, int len1, int stage1, const char return 0; } -static int index_name_stage_pos(const struct index_state *istate, const char *name, int namelen, int stage) +static int index_name_stage_pos(struct index_state *istate, const char *name, int namelen, int stage) { int first, last; @@ -570,7 +570,7 @@ static int index_name_stage_pos(const struct index_state *istate, const char *na return -first-1; } -int index_name_pos(const struct index_state *istate, const char *name, int namelen) +int index_name_pos(struct index_state *istate, const char *name, int namelen) { return index_name_stage_pos(istate, name, namelen, 0); } @@ -3389,8 +3389,8 @@ int repo_read_index_unmerged(struct repository *repo) * We helpfully remove a trailing "/" from directories so that * the output of read_directory can be used as-is. */ -int index_name_is_other(const struct index_state *istate, const char *name, - int namelen) +int index_name_is_other(struct index_state *istate, const char *name, + int namelen) { int pos; if (namelen && name[namelen - 1] == '/') @@ -3408,7 +3408,7 @@ int index_name_is_other(const struct index_state *istate, const char *name, return 1; } -void *read_blob_data_from_index(const struct index_state *istate, +void *read_blob_data_from_index(struct index_state *istate, const char *path, unsigned long *size) { int pos, len; diff --git a/submodule.c b/submodule.c index 9767ba9893cc..83809a4f7bd2 100644 --- a/submodule.c +++ b/submodule.c @@ -33,7 +33,7 @@ static struct oid_array ref_tips_after_fetch; * will be disabled because we can't guess what might be configured in * .gitmodules unless the user resolves the conflict. */ -int is_gitmodules_unmerged(const struct index_state *istate) +int is_gitmodules_unmerged(struct index_state *istate) { int pos = index_name_pos(istate, GITMODULES_FILE, strlen(GITMODULES_FILE)); if (pos < 0) { /* .gitmodules not found or isn't merged */ @@ -301,7 +301,7 @@ int is_submodule_populated_gently(const char *path, int *return_error_code) /* * Dies if the provided 'prefix' corresponds to an unpopulated submodule */ -void die_in_unpopulated_submodule(const struct index_state *istate, +void die_in_unpopulated_submodule(struct index_state *istate, const char *prefix) { int i, prefixlen; @@ -331,7 +331,7 @@ void die_in_unpopulated_submodule(const struct index_state *istate, /* * Dies if any paths in the provided pathspec descends into a submodule */ -void die_path_inside_submodule(const struct index_state *istate, +void die_path_inside_submodule(struct index_state *istate, const struct pathspec *ps) { int i, j; diff --git a/submodule.h b/submodule.h index 4ac6e31cf1f7..84640c49c114 100644 --- a/submodule.h +++ b/submodule.h @@ -39,7 +39,7 @@ struct submodule_update_strategy { }; #define SUBMODULE_UPDATE_STRATEGY_INIT {SM_UPDATE_UNSPECIFIED, NULL} -int is_gitmodules_unmerged(const struct index_state *istate); +int is_gitmodules_unmerged(struct index_state *istate); int is_writing_gitmodules_ok(void); int is_staging_gitmodules_ok(struct index_state *istate); int update_path_in_gitmodules(const char *oldpath, const char *newpath); @@ -60,9 +60,9 @@ int is_submodule_active(struct repository *repo, const char *path); * Otherwise the return error code is the same as of resolve_gitdir_gently. */ int is_submodule_populated_gently(const char *path, int *return_error_code); -void die_in_unpopulated_submodule(const struct index_state *istate, +void die_in_unpopulated_submodule(struct index_state *istate, const char *prefix); -void die_path_inside_submodule(const struct index_state *istate, +void die_path_inside_submodule(struct index_state *istate, const struct pathspec *ps); enum submodule_update_type parse_submodule_update_type(const char *value); int parse_submodule_update_strategy(const char *value, From patchwork Mon Apr 12 21:07:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12198807 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C5E75C433B4 for ; Mon, 12 Apr 2021 21:08:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A1D8D60238 for ; Mon, 12 Apr 2021 21:08:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242476AbhDLVIr (ORCPT ); Mon, 12 Apr 2021 17:08:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57950 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238778AbhDLVIl (ORCPT ); Mon, 12 Apr 2021 17:08:41 -0400 Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com [IPv6:2a00:1450:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 984EEC061756 for ; Mon, 12 Apr 2021 14:08:22 -0700 (PDT) Received: by mail-wr1-x433.google.com with SMTP id a4so14339415wrr.2 for ; Mon, 12 Apr 2021 14:08:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=gJLBcZut5BdoZ62MD0BUa+X2VnGfeSA3JBLk1DbzHGQ=; b=EegA4+XDZS3rRX+DsUqkOPoNiJFnX8CrhKbSu5C8jblUDZf3q2OSItLR8fMUdt3a3E ClUd9IPMNAQMjfnRu1em12EZ+3dItYXW0YJ6o/mfXHyOR6OjIE6NU6A48vLTtkiAuYNZ sxi6BeZA2i8atAQ8r7n0Ef6U2lxsdE+sNCtYBJ1v3OAVKdvs6xRemEZ27Av7NqveTUWo kvPeVPYGB8RhqxqNFuzYJOvrMLPp86+s/V+LGQemH27Y7xf/ru9E/v7qUI7wb6MsyQ9D z2n+nIaxEU6ayn6SzVALqcLX52+VJ7XDlYNWkc/JSITJwv776Kl6+U+Z6r8ZCH3moIxG AutQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=gJLBcZut5BdoZ62MD0BUa+X2VnGfeSA3JBLk1DbzHGQ=; b=HkpQpgR+ZBHy3U4cnimPvKgMNJrl0H9Fi05xQO8t3wei4HUDjTA8IN6urVQaZoiGhP GUcpf4+xW/rxUPH7ftYjSh+mp0nPzJfgDdEJBbAVTJAxeU3dqCIIyLsmp8gDOLN7QsSf I8TvmnXujqGrTT5XuV54Kd5Liatsg2OHdkh+l52J6k97E9s7OwWezeeWUjXUTBb2s2ya Zeb+mmjHSEhaD5FogWq8ZbTFouIRyhWHHPj4iExj8Iw+0HaDgtwAam7XZ8pB17f5PYjN 6Fqc5E616JnU2+ZgARY1NKG9nS33dekSyy3cfMYvl039mTlz0ck30dABqJ+tL6IrArAQ WOMw== X-Gm-Message-State: AOAM531Ih/2evoDmESCJC/YxAiZp30Fntb4Mn5ziO3sNSL80QMsD/p2s fHLb+hD+dqUSXSUwHA4DGLSaZY/q1TM= X-Google-Smtp-Source: ABdhPJy2AxgY7U4L9Zkqg8ziUZeyA3FVSEuaF/H8eGSmged0oZzxAdRMuBpCfWeQjT0VgI66XvvWsg== X-Received: by 2002:adf:e50d:: with SMTP id j13mr6346222wrm.80.1618261701361; Mon, 12 Apr 2021 14:08:21 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id q19sm453125wmc.44.2021.04.12.14.08.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Apr 2021 14:08:21 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Mon, 12 Apr 2021 21:07:54 +0000 Subject: [PATCH v3 03/26] read-cache: expand on query into sparse-directory entry Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Callers to index_name_pos() or index_name_stage_pos() have a specific path in mind. If that happens to be a path with an ancestor being a sparse-directory entry, it can lead to unexpected results. In the case that we did not find the requested path, check to see if the position _before_ the inserted position is a sparse directory entry that matches the initial segment of the input path (including the directory separator at the end of the directory name). If so, then expand the index to be a full index and search again. This expansion will only happen once per index read. Future enhancements could be more careful to expand only the necessary sparse directory entry, but then we would have a special "not fully sparse, but also not fully expanded" mode that could affect writing the index to file. Since this only occurs if a specific file is requested outside of the sparse checkout definition, this is unlikely to be a common situation. Signed-off-by: Derrick Stolee --- read-cache.c | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/read-cache.c b/read-cache.c index 3ad94578095e..3698bc7bf77d 100644 --- a/read-cache.c +++ b/read-cache.c @@ -567,6 +567,27 @@ static int index_name_stage_pos(struct index_state *istate, const char *name, in } first = next+1; } + + if (istate->sparse_index && + first > 0) { + /* Note: first <= istate->cache_nr */ + struct cache_entry *ce = istate->cache[first - 1]; + + /* + * If we are in a sparse-index _and_ the entry before the + * insertion position is a sparse-directory entry that is + * an ancestor of 'name', then we need to expand the index + * and search again. This will only trigger once, because + * thereafter the index is fully expanded. + */ + if (S_ISSPARSEDIR(ce->ce_mode) && + ce_namelen(ce) < namelen && + !strncmp(name, ce->name, ce_namelen(ce))) { + ensure_full_index(istate); + return index_name_stage_pos(istate, name, namelen, stage); + } + } + return -first-1; } From patchwork Mon Apr 12 21:07:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12198819 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EACA3C43461 for ; Mon, 12 Apr 2021 21:08:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CC8746135D for ; Mon, 12 Apr 2021 21:08:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243823AbhDLVIy (ORCPT ); Mon, 12 Apr 2021 17:08:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57954 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238045AbhDLVIl (ORCPT ); Mon, 12 Apr 2021 17:08:41 -0400 Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com [IPv6:2a00:1450:4864:20::32d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 23462C06138C for ; Mon, 12 Apr 2021 14:08:23 -0700 (PDT) Received: by mail-wm1-x32d.google.com with SMTP id o20-20020a05600c4fd4b0290114265518afso7640668wmq.4 for ; Mon, 12 Apr 2021 14:08:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=icvbxp/b5Ftybmb6oGJlzeU1p7Pw4+4yErHEotpNNtc=; b=qw3zWiFse6MZ9xeDr19C1W3pMVLkrIPLhDiyvCDngIA+2uZ85scGYYWL5NQ2TR0nt/ Rl1Jjnopjds9/Hj+qwylk7sLw8BMCUC7rb3IHudLj/xL8lZJZXsyLm5BszXnaRqxpNbd 1qTqaFdtwiZiEPeDsjMcFhEnBKYZIPO+81njqLdoJ196pVQMP6d09onPjU2JTxhCJ40Q Z5NOWzny6F5jG6/gyOPFB0LpAUMDCraxQquRF3Gh2uhIFlPTOJ9bAG/nIDW2UhTUhklg B97E1Xru5vAfogMYwl/uGQA2q2S/w8DXx+b8Q2NzKGGJIz3hzt92BOPIvAbwhrAfpYTm dQ6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=icvbxp/b5Ftybmb6oGJlzeU1p7Pw4+4yErHEotpNNtc=; b=FdLf8K3GHuv/bYMccDUuyd+7A8xO5sK9i/KgARYcu5PJtyIQYp+Xl92JAZEAt6FJgF nvIkLydKmR/A0xAbo2yfWHnQfDtT549hnnGwObwRTmTBDjxsx02XyUqiyL098oVajcs5 3wlXLATdIpHrYQ6QvkkDAE8eCbW9HEJqoscy/rXNVKxaCJT9bDgemFdlVROIkW2+UstI iwNSwZRUvwNPmyNW+5wrGm6Dyn1AiUA071K+EqJi4/Uz/lhJzMIkFHE5xCMgNXgtNAFO SmEmZ2GPZ6aCW4gpovl12vc7OUYaRX7BBY3tlCKHDflKdzK156t6jXB2mWJYqVnuZvdx Gfhg== X-Gm-Message-State: AOAM533wZcpNnBOXjUL09PZ+BTKIeav+bIsuvRded911BZP3C7us07Jj oduKnU5QSezPPal9NfKf6vpzc3vAGck= X-Google-Smtp-Source: ABdhPJyBM/gyA66GV4gzU47gsLGuPtbiw5nJC70hbBd/WM+7ayyQyj0J+2/Vw76bz0Nse0t52ZxELw== X-Received: by 2002:a7b:c20c:: with SMTP id x12mr870811wmi.51.1618261701970; Mon, 12 Apr 2021 14:08:21 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id u8sm18803987wrr.42.2021.04.12.14.08.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Apr 2021 14:08:21 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Mon, 12 Apr 2021 21:07:55 +0000 Subject: [PATCH v3 04/26] cache: move ensure_full_index() to cache.h Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Soon we will insert ensure_full_index() calls across the codebase. Instead of also adding include statements for sparse-index.h, let's just use the fact that anything that cares about the index already has cache.h in its includes. Signed-off-by: Derrick Stolee --- cache.h | 1 + sparse-index.h | 1 - 2 files changed, 1 insertion(+), 1 deletion(-) diff --git a/cache.h b/cache.h index 5006278c13ca..b7e20e9778db 100644 --- a/cache.h +++ b/cache.h @@ -350,6 +350,7 @@ void add_name_hash(struct index_state *istate, struct cache_entry *ce); void remove_name_hash(struct index_state *istate, struct cache_entry *ce); void free_name_hash(struct index_state *istate); +void ensure_full_index(struct index_state *istate); /* Cache entry creation and cleanup */ diff --git a/sparse-index.h b/sparse-index.h index 39dcc859735e..0268f38753c0 100644 --- a/sparse-index.h +++ b/sparse-index.h @@ -2,7 +2,6 @@ #define SPARSE_INDEX_H__ struct index_state; -void ensure_full_index(struct index_state *istate); int convert_to_sparse(struct index_state *istate); struct repository; From patchwork Mon Apr 12 21:07:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12198817 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E8447C43460 for ; Mon, 12 Apr 2021 21:08:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BF50F60238 for ; Mon, 12 Apr 2021 21:08:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243730AbhDLVIy (ORCPT ); Mon, 12 Apr 2021 17:08:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57956 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240417AbhDLVIn (ORCPT ); Mon, 12 Apr 2021 17:08:43 -0400 Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CD4F4C06138D for ; Mon, 12 Apr 2021 14:08:23 -0700 (PDT) Received: by mail-wr1-x436.google.com with SMTP id a4so14339462wrr.2 for ; Mon, 12 Apr 2021 14:08:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=dTUCQB3WXCizXGIVy+rJNkDOcat4/a7nb8eHNj32aL4=; b=htx0UHiDbUIVFDR1ZNEbu+/FN2fIEzzvnlpkt93rlVZH6fJ9A18CmgT7e8gjjhk1oe 4xej07ZWgn3YFU6jNPkDrG7+5YMmiDSSvo2ZfnUbhnY7HxCsFjc0noK10MFENPZGJbco 6b8Jp+O4CWuLYU+Z1ZSQkEbEqHa475woU2Y1fiwGRitNjWFFLDVyliuzHoRav2tpnfZi QcVxAo74hB81J9Ryu2aRCJjXziFWp4k4qxT1pYihKrbgWxqx2c/yNgfw60qr2eOa1KwS HPCVCcZknTUbDKdx2YinxqmK9j3gJHh54FlReO09v+9uZeEcrjuSTT71kpE5M3dvDQRi NudQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=dTUCQB3WXCizXGIVy+rJNkDOcat4/a7nb8eHNj32aL4=; b=RcDK8Sio3mdCabjpkxgCnB3l3kvQ2ZQOUho+2+Wf3qpFnlLi16UOVp4mXUgjDMusKA 4rxAe1emY1k7NEGqInq9IPcgcBmbBGyGOl7/s08QA0pPuITsynzRuKDXXZQFiV5OfUrJ 0aaj2rQMgZb2i8GZObOKZ1cR+bV0YPzBz9oUibJ9kbKNiHbvTV6mVbftwyxka0iPF5x6 dk/ruVDzbvlciguIBS4Fjo3dWUJVM2jJlcVMkgMAoPPtQ9bv4ZcYFAXu2MZgAWNCbfaX aJAQaXUzmbzWD9uR5XAzLbtznynHBHoKwwu1XO0Sf/zkhjv2W0zeHjIBxymB8iAg3F/c vbyA== X-Gm-Message-State: AOAM531A6bcIi9gBJr6+OerIqzQqnm84P1ZA9CqLsg+duy1FKu1y5d5B nMltQhhKfE/S8XcxDwbzSRAgylDVCS0= X-Google-Smtp-Source: ABdhPJyZ3vsLayjcqyu1dtElpV4KOPJQtzEBop9tgMx7bt0Mzt6a6kXOLWSiedoEaasjj3LA8RcIHA== X-Received: by 2002:adf:fa11:: with SMTP id m17mr33363294wrr.287.1618261702589; Mon, 12 Apr 2021 14:08:22 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id s9sm494277wmh.31.2021.04.12.14.08.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Apr 2021 14:08:22 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Mon, 12 Apr 2021 21:07:56 +0000 Subject: [PATCH v3 05/26] add: ensure full index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Before iterating over all cache entries, ensure that a sparse index is expanded to a full index to avoid unexpected behavior. Signed-off-by: Derrick Stolee --- builtin/add.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/builtin/add.c b/builtin/add.c index ea762a41e3a2..afccf2fd5543 100644 --- a/builtin/add.c +++ b/builtin/add.c @@ -141,6 +141,8 @@ static int renormalize_tracked_files(const struct pathspec *pathspec, int flags) { int i, retval = 0; + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(&the_index); for (i = 0; i < active_nr; i++) { struct cache_entry *ce = active_cache[i]; From patchwork Mon Apr 12 21:07:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12198813 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B3FA3C433ED for ; Mon, 12 Apr 2021 21:08:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 903CC6135C for ; Mon, 12 Apr 2021 21:08:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243337AbhDLVIu (ORCPT ); Mon, 12 Apr 2021 17:08:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57964 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240719AbhDLVIn (ORCPT ); Mon, 12 Apr 2021 17:08:43 -0400 Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 68854C06138E for ; Mon, 12 Apr 2021 14:08:24 -0700 (PDT) Received: by mail-wr1-x42a.google.com with SMTP id h4so5292130wrt.12 for ; Mon, 12 Apr 2021 14:08:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=YIfeJaOwJ2PF4AvCIcf7mIqwTKwspa0fWugQUcTQFgU=; b=Tpi/Wu9DoY2lKhArje/+PMkhtrbO9ugMdIW6eSDV24Vz88Tuck00LAC2c1rh9gTw/U jUt2Avhqb9Fde26xAnd24Cl5skFIthywti2/xRM5t6+czuX2+bu74euQeaUSM11Tu5zB tyhzbVbAS5FQYriQ+XFmucocNmhi5mwAXk6RVOlfUvwr4fmFz/hQ/oxtnK/aVxDcYbvo EgQNiUgZdawo3mELlxkdCOP1tE+SarcMcYIFpgRtxOjkFOeNbN11iUuUlbR5VC4G8E96 1ImpTXgL2Yvq0b0SVRgRg8Yw9ESTEJg9x0zQlzqaPKmRUoTqeUC+JoMwdWJllPBWV/fJ IDYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=YIfeJaOwJ2PF4AvCIcf7mIqwTKwspa0fWugQUcTQFgU=; b=k/UsNzGIiRb7joWCZPJFiBUpnC3fzujK78cLrnWGogr+PHoilB0rW16DK6Mzw7DiQd /rp0i7qiSNWcWTWyuqSjxPppkwVlIlTFnDaBE0LxQZ4kgZqNsBXSwh8qymq4RPcIFRFD G8TXu9zvXHyST4LaKMyIdG6MwfkvGqPAExaxMJAsBIWJ+t35j2MVyvzo+VLDiPlROPJd UpiU59YlccC6mvDrb2p9BdIAkwXQlbPyAYh/p4EWqPxK9xRQg8C9xWAAjaQ1K97IFco4 4sBxEhgCUz+09jARAGVAYAEHu9gwtqsSZQNkGFU5IzRN6B48xhlQaCdtMN+lkyve+h3u Z/sA== X-Gm-Message-State: AOAM532gYhD3clEZD7OyxrSNE9wIMiSX1Kh3RuLwX0WlXiUCCNby5qdb l5GfT3h4duL8Eze/9FqYDX9yZQvpdHw= X-Google-Smtp-Source: ABdhPJyyyuDpjmZujFp9tZmBbPx9AAsfdR8w8+rv4ciIQIN34Gnfn52LsnYIjHxFx+S2k9j6lhViIQ== X-Received: by 2002:a05:6000:190:: with SMTP id p16mr34880715wrx.70.1618261703273; Mon, 12 Apr 2021 14:08:23 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id k22sm18113993wrh.5.2021.04.12.14.08.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Apr 2021 14:08:22 -0700 (PDT) Message-Id: <0082855b59617f6f04f36024d79e99c4d8f303b7.1618261698.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 12 Apr 2021 21:07:57 +0000 Subject: [PATCH v3 06/26] checkout-index: ensure full index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Before we iterate over all cache entries, ensure that the index is not sparse. This loop in checkout_all() might be safe to iterate over a sparse index, but let's put this protection here until it can be carefully tested. Signed-off-by: Derrick Stolee --- builtin/checkout-index.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/builtin/checkout-index.c b/builtin/checkout-index.c index 023e49e271c2..2c2936a9dae0 100644 --- a/builtin/checkout-index.c +++ b/builtin/checkout-index.c @@ -119,6 +119,8 @@ static void checkout_all(const char *prefix, int prefix_length) int i, errs = 0; struct cache_entry *last_ce = NULL; + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(&the_index); for (i = 0; i < active_nr ; i++) { struct cache_entry *ce = active_cache[i]; if (ce_stage(ce) != checkout_stage From patchwork Mon Apr 12 21:07:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12198815 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 66467C433B4 for ; Mon, 12 Apr 2021 21:08:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 470306135C for ; Mon, 12 Apr 2021 21:08:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243618AbhDLVIy (ORCPT ); Mon, 12 Apr 2021 17:08:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57974 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240409AbhDLVIp (ORCPT ); Mon, 12 Apr 2021 17:08:45 -0400 Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 10AD8C06138F for ; Mon, 12 Apr 2021 14:08:25 -0700 (PDT) Received: by mail-wm1-x32b.google.com with SMTP id p19so7601044wmq.1 for ; Mon, 12 Apr 2021 14:08:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=rzO06S1ief0qGjiR3z1dlgur8sjPW9z+JkmBA4FJEXI=; b=lrzu4D+jSHCaoUXJxcGmeaCdiAlT8ud+tuFP+GOJ6Bu0QsqkDbOGyT4RKn367/Tx2g lNEtrGolgDVZnRkteAK5LdFrNv0yvXx5jQfsv6VdQJouhUBUc2XbJqHBxu5P2SLCQn/P vOnfe+XNRN6ph8t3TS1AAzqYl6U4ANmQQ2BhkcrhWrz3QpxYY8WbEjs7VzXw97s0dVyS vihpyHZgaiLpLiQtbLKmSTR3ZCBbfzbGW+aoqE3JdiFfniuFfuprDUiR3Hl2R+T0Iv1e dpF9qe9v3CptMI+1Hd/fL+QQqe8q00p2/nwZXwroPiIdy99Hk4nazkCZml49lmM54Tak RKWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=rzO06S1ief0qGjiR3z1dlgur8sjPW9z+JkmBA4FJEXI=; b=Um53Mdr1zksSVBiRlSbWEUXn4fXe4qVktvdZDKv5y1y1ozLXUVh56YcCawoiLTdU3N njZWG0+LS7lFNg2dZx3KjvvFbtbT3ExPcOlIn4DnlpTD1sY+vD80ndfYPx2v1mSW/6ia 5lYukbvKXmj0a5FExdVBlTg/lhLuqm+TNzyXu+AhO7I4zZwtJnPA2XTjjfU7w2bdyqM9 l0SW2zR3GCtIwdXqJhai53HajRWND3k7wj1jUnwiUretO3NxzBAK8AdH7jO1vtZ/lC3c Dn+ozjMTgQsiMihjtj5CbL+oqzYPSalcsthILDKyx+U1WPYICAw2PXMbOXPD127fIYt8 gchQ== X-Gm-Message-State: AOAM533FIcYJIynSVk8hwSlJvSsraHf4PuCTUM2FKe7tZLU8TRCiSzHz 2CkvBtNAp/iR8rY/HXxdmPIwkRJUpbo= X-Google-Smtp-Source: ABdhPJygAzhy/7HYImYnOllwwRla6jajxPLDyBRFZqT0Fgyx74eLF0QM3yTVj4QdHP5jEadaMZmekw== X-Received: by 2002:a7b:cd07:: with SMTP id f7mr897128wmj.119.1618261703841; Mon, 12 Apr 2021 14:08:23 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id m25sm473026wmi.33.2021.04.12.14.08.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Apr 2021 14:08:23 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Mon, 12 Apr 2021 21:07:58 +0000 Subject: [PATCH v3 07/26] checkout: ensure full index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Before iterating over all cache entries in the checkout builtin, ensure that we have a full index to avoid any unexpected behavior. Signed-off-by: Derrick Stolee --- builtin/checkout.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/builtin/checkout.c b/builtin/checkout.c index 0e6639052001..d0dbe63ea119 100644 --- a/builtin/checkout.c +++ b/builtin/checkout.c @@ -368,6 +368,9 @@ static int checkout_worktree(const struct checkout_opts *opts, NULL); enable_delayed_checkout(&state); + + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(&the_index); for (pos = 0; pos < active_nr; pos++) { struct cache_entry *ce = active_cache[pos]; if (ce->ce_flags & CE_MATCHED) { @@ -512,6 +515,8 @@ static int checkout_paths(const struct checkout_opts *opts, * Make sure all pathspecs participated in locating the paths * to be checked out. */ + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(&the_index); for (pos = 0; pos < active_nr; pos++) if (opts->overlay_mode) mark_ce_for_checkout_overlay(active_cache[pos], From patchwork Mon Apr 12 21:07:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12198809 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6D63EC43461 for ; Mon, 12 Apr 2021 21:08:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4ADDB6135D for ; Mon, 12 Apr 2021 21:08:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242939AbhDLVIu (ORCPT ); Mon, 12 Apr 2021 17:08:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57976 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241290AbhDLVIp (ORCPT ); Mon, 12 Apr 2021 17:08:45 -0400 Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9FE80C061342 for ; Mon, 12 Apr 2021 14:08:25 -0700 (PDT) Received: by mail-wr1-x434.google.com with SMTP id r7so2331913wrm.1 for ; Mon, 12 Apr 2021 14:08:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=zERwEV3I5iAUQNSYAsx1jxOMzyYF1N8NzKg79Kbu40c=; b=DYfb3gigRPxudgkbEs5CqlLeTVoCfjvYRqw8FOiLC0Ek+jFzN8aiM4ZmiGb50mv/F0 29HdQ9RRbSuzZtGpwTr/BlMY7rc2oeGXJGO++NplgICgpDBUpuQQD1DGhsid6i75qUWQ 3LCO98s50bepUqEddR6wwZ/0BaIJ2knFOKrT0w7nVbR4TIU38Z1xdhAJx0OlpRTIJTWL O9VojX6qqYTxwlK+eTMrlZtox+gKumIQ+L7Iqb4k9FykHqcGU3ekWeDWL3ijjeWItK90 Jg+COJX/XlfnIUEM18souzVJbqOYq3LD05vfHF7KOgVvsaGm8PYQ/kN8oV8G2fCJHkHH xnAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=zERwEV3I5iAUQNSYAsx1jxOMzyYF1N8NzKg79Kbu40c=; b=nM5vNyxMAjxOEHH1/JWlJi2Qf8XTKXbNjXoK9VPBOixUZxO/gcJ0dDm5RVnm6b8o/n UzkW4qhb/RjDv+3+4IWf7RPLXEsqcm8SKbp6m9qEXe9blkLcZDb6DEHjwLJ7ebe0ITnY r0Cog9evPX4gIyfgkHxnbsxcQPZ5FVfX1c2S11Db0hC6vch4uDTox8ycGMspZAf3PB/U kW7UV7tDempiOtoOIVDjlkvNEZbkQD3eQhMRpncNPODLJxKxfcPaEx/bFy/Vs1jkmEw9 1TUfdxh/XxPcSla3MvY9K/yH4OpkhUOyIwMO9WeGCRPfky+v9TD/iMo2tRhzTDDAqkx/ t5xg== X-Gm-Message-State: AOAM533rFiATknxXjpKnX2emY5SglHXPEdakbUL4F0OZRlJOtvjhI4Gl X4ec3Bvv6JAOuy0JwEBeX3MLGabAjXI= X-Google-Smtp-Source: ABdhPJyGgLggZTXQme5fcWF74K0TZMmE0WafZyrghyo2YwMsKyGTESKnsrG7F30PxiQCP4jgSNFSUA== X-Received: by 2002:a5d:5642:: with SMTP id j2mr33063647wrw.172.1618261704495; Mon, 12 Apr 2021 14:08:24 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id y15sm1266851wrh.8.2021.04.12.14.08.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Apr 2021 14:08:24 -0700 (PDT) Message-Id: <1a3b51fd3c4bdfc50fa390d04c49b49acc47b90a.1618261698.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 12 Apr 2021 21:07:59 +0000 Subject: [PATCH v3 08/26] commit: ensure full index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee These two loops iterate over all cache entries, so ensure that a sparse index is expanded to a full index before we do so. Signed-off-by: Derrick Stolee --- builtin/commit.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/builtin/commit.c b/builtin/commit.c index 739110c5a7f6..cf0c36d1dcb2 100644 --- a/builtin/commit.c +++ b/builtin/commit.c @@ -251,6 +251,8 @@ static int list_paths(struct string_list *list, const char *with_tree, free(max_prefix); } + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(&the_index); for (i = 0; i < active_nr; i++) { const struct cache_entry *ce = active_cache[i]; struct string_list_item *item; @@ -931,6 +933,8 @@ static int prepare_to_commit(const char *index_file, const char *prefix, if (get_oid(parent, &oid)) { int i, ita_nr = 0; + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(&the_index); for (i = 0; i < active_nr; i++) if (ce_intent_to_add(active_cache[i])) ita_nr++; From patchwork Mon Apr 12 21:08:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12198821 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 789B8C433B4 for ; Mon, 12 Apr 2021 21:08:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 567D66135C for ; Mon, 12 Apr 2021 21:08:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244048AbhDLVI4 (ORCPT ); Mon, 12 Apr 2021 17:08:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57978 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241474AbhDLVIq (ORCPT ); Mon, 12 Apr 2021 17:08:46 -0400 Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6F0B1C061343 for ; Mon, 12 Apr 2021 14:08:26 -0700 (PDT) Received: by mail-wr1-x436.google.com with SMTP id c15so5372246wro.13 for ; Mon, 12 Apr 2021 14:08:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=/kQ/3c8UbEqQOrilneslkKpts4OYd95dOalBYXNQd/M=; b=g1m+/DOXLYBJhOsOf1E0Z+tJVJYHZ/pCimKCynVRS+3+EFcOPqoG9YHP2t8fbcOfXA 5/w4lWPMz3mtX5L1s9gdUqLba3QIVu/7vFGQBuxQxZsUgExaJfadObOeyJzrtcpul08U FEZC3naeJ4AG9hi2zJOCKNuqdCuUmJOqoaU31gEMyBN/brtdsZ7cTqcZRokyWX6xEOcH u6ARJHh1kafqIXV6VpM0ThkPZiliKVELo8EpcVEc55Ds8IB1YROvIhVXVKqhQ/h1avvd SWs0n7PbiqulELCJmGzvZMEzXMfWdZqzsdMvucauTJtxxpT1qEXEE4n2qW8LJMVsHKGX v7dA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=/kQ/3c8UbEqQOrilneslkKpts4OYd95dOalBYXNQd/M=; b=sgRBRqaGxCnj+GNkDiYx87Az3Ha9kVJmtBqOW4Kn6Ztt2GL9dvGEN4Q6U9AbzKEoE5 MALlmdb4l4eW4iYZt0zyEze7ZZ4/aSe7EajA1RL6TKbPLoqfVaLrnFlZ9CCxpaPWCYRK yW0Xo1px453GEPQseo97OeHbiOjxFNQumiYq/UIkjaRbOyhL+X6yR+Zp8YmaXBZ8lxZr witqfR/7R9QCYtVe4llivxnQ8dasJxjoB3bB5rLKKWz6lEFv2HMZUGmTrooLrrtCRyDz gYkdLXGUrCmj5g92XicyrtxT07gblrZFNSkDo+DNpni5YAHm3WM+kkuw941gHjnIvHMQ JUYQ== X-Gm-Message-State: AOAM531HzJ98ccVgSo6eCWADfgB3PLwF01M0d/ryh3lhjK/mxFX0oza5 ys8ekOzodcUDLxWnY1ZyYFvTH8dPexo= X-Google-Smtp-Source: ABdhPJx2GrgL/iUSliuD904wCbGSSFCVBc7LyeNrhHxO6fDmbqNfTEeLdXePg4L1Hlqy0W6KWq+ZhQ== X-Received: by 2002:adf:a119:: with SMTP id o25mr2195974wro.36.1618261705072; Mon, 12 Apr 2021 14:08:25 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id l4sm501287wmh.8.2021.04.12.14.08.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Apr 2021 14:08:24 -0700 (PDT) Message-Id: <8c61d40dfe01d89d248ff9208e955801c1a18c18.1618261698.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 12 Apr 2021 21:08:00 +0000 Subject: [PATCH v3 09/26] difftool: ensure full index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Before iterating over all cache entries, ensure that a sparse index has been expanded to a full one to avoid unexpected behavior. Signed-off-by: Derrick Stolee --- builtin/difftool.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/builtin/difftool.c b/builtin/difftool.c index 6e18e623fddf..32c914dde6a0 100644 --- a/builtin/difftool.c +++ b/builtin/difftool.c @@ -584,6 +584,9 @@ static int run_dir_diff(const char *extcmd, int symlinks, const char *prefix, setenv("GIT_DIFFTOOL_DIRDIFF", "true", 1); rc = run_command_v_opt(helper_argv, flags); + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(&wtindex); + /* * If the diff includes working copy files and those * files were modified during the diff, then the changes From patchwork Mon Apr 12 21:08:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12198853 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3F161C433B4 for ; Mon, 12 Apr 2021 21:10:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 14C9E610CE for ; Mon, 12 Apr 2021 21:10:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231854AbhDLVKs (ORCPT ); Mon, 12 Apr 2021 17:10:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57980 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241910AbhDLVIq (ORCPT ); Mon, 12 Apr 2021 17:08:46 -0400 Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C94EFC061344 for ; Mon, 12 Apr 2021 14:08:26 -0700 (PDT) Received: by mail-wr1-x432.google.com with SMTP id 12so14339263wrz.7 for ; Mon, 12 Apr 2021 14:08:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=63cgv+d6hF7bJxk9XJMcdXLWzyM+y2aIDW1QZ5VUN1A=; b=CaJlVQ8XsDfLToS1PTrzgyp6IP9SWAFmvFjEfmBye8imCRiG3ZEo7S0mFY5CbBxRxS 04HMHB0DsEzqls8pswJe6C0CkhVFkyrpfwdiLZRo3UgomQHUbkizhP4P26ta6pDCEy81 if0yXvYBRAhZ5YmNzuiuSCQAqexyX22YSLvjSICEHVED/S3KgebB/cY3fcBqMRmMAz/o aYO/rUEK2gcpnHqYy1aEvvjl8tOMlZO4Ds/7QKK16Pt4Pj4v8dQYgcqyDlkpMl47Cuhn RdUW99cMiW7ZmCJXkJI38p1WygdHbrgmPbHr2rN1a9hWVl8nzBQA9hG5/YNwHE4Uj3w5 NFGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=63cgv+d6hF7bJxk9XJMcdXLWzyM+y2aIDW1QZ5VUN1A=; b=DJNKDWcOmUCRYoPCPOfzC0vPuwLM/Fx5YI3GYhuVrSvm1klagLLGYFOutghSHJpJKV +wue/FZDOe/g0tWQvXtvHLh/77My9t4qB85/dYozZVX+O9bv8PfqVIvr2Wvcp64Pfibs eZii6FCumdWn1d6a9ujXzNPts7deqSht5a4M4c5zpdL4eoQWEIlsQjoXjpvq1WY+xqrR 84hHsK7aN7YYt7K7ku7GKO0z4km7nJUK2orVOWCtnvHE56VWZhOyeIE1dBS4g6GL9UAu DceWXf11gAPLjjZMqKTVuwOGUPolzpZlBvJCoukmo25O1yWmGwY/gBT7GGGX8tmNtrln GnyQ== X-Gm-Message-State: AOAM532UkmRdhOgwLXyqqhD+AknnvF8p1dCpyLmVhVYVIzrXcT0wBBy1 aQuwNBOMY3c5Z/OtacnVfJ5P8nJXtNw= X-Google-Smtp-Source: ABdhPJxUP0iT2/MyJew00b/JItiQW5ecF6/h2rnYFaykMxZFw47JDmq7eHvs6UcYJMxeJKk3L40VLQ== X-Received: by 2002:a05:6000:18cd:: with SMTP id w13mr32916850wrq.20.1618261705679; Mon, 12 Apr 2021 14:08:25 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id b206sm491352wmc.15.2021.04.12.14.08.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Apr 2021 14:08:25 -0700 (PDT) Message-Id: <45b603379422bcf7382ff905a97921e3ca5114a3.1618261698.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 12 Apr 2021 21:08:01 +0000 Subject: [PATCH v3 10/26] fsck: ensure full index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee When verifying all blobs reachable from the index, ensure that a sparse index has been expanded to a full one to avoid missing some blobs. Signed-off-by: Derrick Stolee --- builtin/fsck.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/builtin/fsck.c b/builtin/fsck.c index 821e7798c706..4d7f5c63ce0d 100644 --- a/builtin/fsck.c +++ b/builtin/fsck.c @@ -883,6 +883,8 @@ int cmd_fsck(int argc, const char **argv, const char *prefix) verify_index_checksum = 1; verify_ce_order = 1; read_cache(); + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(&the_index); for (i = 0; i < active_nr; i++) { unsigned int mode; struct blob *blob; From patchwork Mon Apr 12 21:08:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12198855 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1B7F1C43460 for ; Mon, 12 Apr 2021 21:10:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EAF0F6135C for ; Mon, 12 Apr 2021 21:10:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241474AbhDLVKt (ORCPT ); Mon, 12 Apr 2021 17:10:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57984 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242228AbhDLVIq (ORCPT ); Mon, 12 Apr 2021 17:08:46 -0400 Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 74DC4C061346 for ; Mon, 12 Apr 2021 14:08:27 -0700 (PDT) Received: by mail-wr1-x42f.google.com with SMTP id a6so14336932wrw.8 for ; Mon, 12 Apr 2021 14:08:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=AqzgF3ydQhx2SGKGBO9a/JEpTg8x4sKJCQWGxp4c3hY=; b=GmBA1Ndq9J8MEUwpHEB1mqKTVdB2vhcQLZQdsvjXVdeZ9HQCoC7KNRA2kSdVqtaux5 xpKBX+tCdPw9oGERbPmSAUPRBAQwIByW42qRXNmeuyU8RfbZWDxppcYsngWdpXhUw0jQ Z6HxLR5rdhOMMfuT0bH0acugvVHsuOq73GN8jM/m6UhW6weZGreZFmfZ5DkglnfvTJZ7 Fsx05nYjPqXlKVe/wjoNkBj+q3a7VCGrOaywb4MUAdJOt9TYPE+XbeDxf0SCQ03ukZgi +m0njWU3LYPodSUhMVNdOvEynyF7edmmxuf/RZzzG2Jmv8L+6lWFIh4FExCQ8DYhRaWU 0zeA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=AqzgF3ydQhx2SGKGBO9a/JEpTg8x4sKJCQWGxp4c3hY=; b=XWLvnjrUMZHz8q6F0qeISfVmbLqFgv8KatE2t3hi2wRf+8ri36zYNLbTvGPY5NDazt Vdbe31e+zq3ISS5SvXrR3M9IiYaVstxGuF5XF7WmArNi09NauRX/rgmCrUvZkIJeOwNo NL4dL28vDA6SuewcsW01XaB4ezM3ECGuTDq83boYRzxSiNdhq/1PUJQ4VDT5OuyywSVr EjkT3hRKBNNorWhyqmMZCL3majRt4JdT0YoMdRNrXly1o8JjbssvLyVBCUsO3HoHNn3r aCLbsyip+g3REJhOhrh5huRt4ECIi0RKJlgNY76uil/nYfJqB8eAcEKOPNwVfVruIJQH h/kA== X-Gm-Message-State: AOAM530yOgGwNcfIs5jegiTDI8xlWCiKTTZr7oI662fxy/Nh8OcGAoMs 0E8OyLMWdtU9ZAIU+YUrzyS8jKktJJA= X-Google-Smtp-Source: ABdhPJwh76+KDpVvwnSgMouNceeB4CBvOtLL0rv50nabKCObCW6wbrZ/lqRFswZgy4Mi1ahOumYElQ== X-Received: by 2002:a5d:47c4:: with SMTP id o4mr32759024wrc.138.1618261706275; Mon, 12 Apr 2021 14:08:26 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id s10sm46659wrt.23.2021.04.12.14.08.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Apr 2021 14:08:25 -0700 (PDT) Message-Id: <97124e9fdc7f21d1f82162b8269551c023392fa3.1618261698.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 12 Apr 2021 21:08:02 +0000 Subject: [PATCH v3 11/26] grep: ensure full index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Before iterating over all cache entries, ensure that a sparse index is expanded to a full one so we do not miss blobs to scan. Later, this can integrate more carefully with sparse indexes with proper testing. Signed-off-by: Derrick Stolee --- builtin/grep.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/builtin/grep.c b/builtin/grep.c index 4e91a253ac3b..c2d40414e975 100644 --- a/builtin/grep.c +++ b/builtin/grep.c @@ -504,6 +504,8 @@ static int grep_cache(struct grep_opt *opt, if (repo_read_index(repo) < 0) die(_("index file corrupt")); + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(repo->index); for (nr = 0; nr < repo->index->cache_nr; nr++) { const struct cache_entry *ce = repo->index->cache[nr]; From patchwork Mon Apr 12 21:08:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12198823 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4A521C433ED for ; Mon, 12 Apr 2021 21:08:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2C24160C3E for ; Mon, 12 Apr 2021 21:08:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242786AbhDLVI6 (ORCPT ); Mon, 12 Apr 2021 17:08:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57994 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242396AbhDLVIr (ORCPT ); Mon, 12 Apr 2021 17:08:47 -0400 Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 211D6C061574 for ; Mon, 12 Apr 2021 14:08:28 -0700 (PDT) Received: by mail-wr1-x42b.google.com with SMTP id r7so2331997wrm.1 for ; Mon, 12 Apr 2021 14:08:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=DOjJ2jNd1O2RVKGUUk1xSCeZyh/PdDSjmF8DqHmFsvk=; b=m4GXazeo15J61SFZm3gvgzOFnqs26iH/pEKNhercknlx72nBN6Dfni71KHvwHQtAkE okjiR9RCBlBTfbd6WuwhtpKSIAeo7TLxGCdkR2ipYt/976MRkPjqdmHGx5AkfI2y03AL YlQ/Nbkj4pMhOeaCiTpMB3tipcV3RWNw+88KG8GXk6c5835P6rIDPhqZPFJo9j4otTqr E1agUS+sq6+zIuw1363+oK0/IbxHZY9kE5qeQRIP3ZChTsoiJY7RYYgBfXVMhoTCakcF 1yFo0kkAw7hOt+BfePcZSmZUD/Ss+dgspbbHwQ/sTlQoa1c289qqaavslDpnDwS7Uvco IPNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=DOjJ2jNd1O2RVKGUUk1xSCeZyh/PdDSjmF8DqHmFsvk=; b=qzisimKcq5Br5HkfvZqHcrmiOIUzKuutx09D+K0BdEG1l6H9u6RA8ovV0tYofJj1uz TCahVOWN+hRCIlIynkxwo6QTetToSF/brcjFjCi6fnYocZYZ2H7Cz1aLxv6V4fiTMTVA tDKveF1KYjbVXYrmx84u8EqhbthVWtMtDfxfPhWE7ovyIR0ixqrlAYjTUuG7RYifZeX3 +ggtqD7E/b09IiV8Q4H/9W5XKe4+XrF14+3wB1Yojb7ifInMbMmXita3Hnf4iNyUu8Rq MMuqmMOOKg27KsB81oPhZIFT6LZBFPtTBSzlcCpMzoA6pEOw3NW89D0+akbtXJi9a2vs qJAA== X-Gm-Message-State: AOAM530Nn9HBvd2O5Oru3SDSELpHoWL1aqr1kvFOTfBHj09kISKYKrn7 0HMF7jwCP0NkaY4aZEiOXOoFn5y3Dso= X-Google-Smtp-Source: ABdhPJze1dMbzF2niAdR2+o64tnjishDe3z32+s2Q1f7j4AJ7RaeU5LoqoXQZzSvQC42dswaFm0GfQ== X-Received: by 2002:a05:6000:128f:: with SMTP id f15mr23274132wrx.19.1618261706913; Mon, 12 Apr 2021 14:08:26 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id y31sm477809wmp.46.2021.04.12.14.08.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Apr 2021 14:08:26 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Mon, 12 Apr 2021 21:08:03 +0000 Subject: [PATCH v3 12/26] ls-files: ensure full index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Before iterating over all cache entries, ensure that a sparse index is expanded to a full one to avoid missing files. Signed-off-by: Derrick Stolee --- builtin/ls-files.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/builtin/ls-files.c b/builtin/ls-files.c index 4f9ed1fb29b7..a0b4e54d1149 100644 --- a/builtin/ls-files.c +++ b/builtin/ls-files.c @@ -317,6 +317,8 @@ static void show_files(struct repository *repo, struct dir_struct *dir) if (!(show_cached || show_stage || show_deleted || show_modified)) return; + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(repo->index); for (i = 0; i < repo->index->cache_nr; i++) { const struct cache_entry *ce = repo->index->cache[i]; struct stat st; @@ -494,6 +496,8 @@ void overlay_tree_on_index(struct index_state *istate, die("bad tree-ish %s", tree_name); /* Hoist the unmerged entries up to stage #3 to make room */ + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(istate); for (i = 0; i < istate->cache_nr; i++) { struct cache_entry *ce = istate->cache[i]; if (!ce_stage(ce)) From patchwork Mon Apr 12 21:08:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12198825 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 932F8C433ED for ; Mon, 12 Apr 2021 21:08:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6A285610CE for ; Mon, 12 Apr 2021 21:08:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242574AbhDLVJC (ORCPT ); Mon, 12 Apr 2021 17:09:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57996 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242431AbhDLVIr (ORCPT ); Mon, 12 Apr 2021 17:08:47 -0400 Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B9DB4C061756 for ; Mon, 12 Apr 2021 14:08:28 -0700 (PDT) Received: by mail-wr1-x434.google.com with SMTP id a4so14339638wrr.2 for ; Mon, 12 Apr 2021 14:08:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=CJjXReyOnuxiS+rkr7b3ZvpKZ1VCS6UDhvoKhleGnNw=; b=dxNq3lwVlgdq3Ht0PFsUkasSEB+DjcdORysycppwN4UKBjUzgv4DpdH/E0tSiW1rLR NZqH3ba0yOyGbtiSG/RwlleHFbnt9bPoJWNSWCkXAxc+8+OWDtolOKfM8/gUHpu/6X+E wZbT3a8T+8aXGLSqy4cf1Cz/dvjb9l7v8Z5z3iZAxRSW7sPc49toriRsGW9Sy9nJQyVI jgQUyMuYPvYzLSs2ORi5yotyzEjxE/QIAXbI6dSmnRtqBNu8eQwHEUzXEezb3vq3ljEw hYt0bOa+700DMa9L6JiLDpq28m/zdHi8fPBDo50o5vXZfX58MMS3Y0q7FTophaqSX0ox UrsA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=CJjXReyOnuxiS+rkr7b3ZvpKZ1VCS6UDhvoKhleGnNw=; b=HpYJPOXVDBjc8uXjKz2DyiOTe7bnZqX+wMuJNFPaksFdBqOT2C862w7kmRAIura33v MrefWsEp1fu/4Ltji40KBUsmJCd0slyyGnYWYz6xjN9bWqzUwOTAqtsLPOfxB1wESN5T QIryvdWw894i5Kt8OURDs/kC1PoBMzCnGZhhv2BdTmXozMTg/n34ui5MNjOGg3phhgLQ IO4XTAJ4dvMPnc1keSA08S/B6VW2bSekPjjUW4zFIoR491/4cCctfv/zKLPajUVbTcel 6/gcKtSztt+TfHcGQdrMQMmQdW6SB+CPlVhNnPIeKvO+teuvzyEHt6Y3clisnzHbJK01 yTMQ== X-Gm-Message-State: AOAM533FXW8Qcp2ozUVOgr7wm/pqLASUmzFZs8KqiIgX7HL9jHhtH5Eq 2BkWAOXAh6e5YsaTkkHMn4r354u+9Eg= X-Google-Smtp-Source: ABdhPJy1sbCoRdMQBEZnQ1UvstLZ5cdt2TICPGxx+bpt31s2prHleIurTekMpBn1A1QSMhQ8lnY9iw== X-Received: by 2002:a5d:6e06:: with SMTP id h6mr3589678wrz.201.1618261707545; Mon, 12 Apr 2021 14:08:27 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id v3sm475523wmj.25.2021.04.12.14.08.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Apr 2021 14:08:27 -0700 (PDT) Message-Id: <6497f2ce225bba4f3706be3e5970f815b767b72d.1618261698.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 12 Apr 2021 21:08:04 +0000 Subject: [PATCH v3 13/26] merge-index: ensure full index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Before iterating over all cache entries, ensure that a sparse index is expanded to a full one to avoid unexpected behavior. Signed-off-by: Derrick Stolee --- builtin/merge-index.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/builtin/merge-index.c b/builtin/merge-index.c index 38ea6ad6ca25..c0383fe9df9a 100644 --- a/builtin/merge-index.c +++ b/builtin/merge-index.c @@ -58,6 +58,8 @@ static void merge_one_path(const char *path) static void merge_all(void) { int i; + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(&the_index); for (i = 0; i < active_nr; i++) { const struct cache_entry *ce = active_cache[i]; if (!ce_stage(ce)) @@ -80,6 +82,9 @@ int cmd_merge_index(int argc, const char **argv, const char *prefix) read_cache(); + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(&the_index); + i = 1; if (!strcmp(argv[i], "-o")) { one_shot = 1; From patchwork Mon Apr 12 21:08:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12198827 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9D281C433B4 for ; Mon, 12 Apr 2021 21:08:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 74E8660C3E for ; Mon, 12 Apr 2021 21:08:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244563AbhDLVJD (ORCPT ); Mon, 12 Apr 2021 17:09:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57974 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231750AbhDLVIs (ORCPT ); Mon, 12 Apr 2021 17:08:48 -0400 Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 78ABEC06138C for ; Mon, 12 Apr 2021 14:08:29 -0700 (PDT) Received: by mail-wr1-x42c.google.com with SMTP id 12so14339390wrz.7 for ; Mon, 12 Apr 2021 14:08:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=bdB5os8ysADiA9ihl7DyMq1Tj0074xySPTMD9uYQI04=; b=JZPM0thnn7fpC0l9tBUA2eDMks7vPtOVvsguX+ffijobfw22vEqDMJ1qPmaKd/h6gl ZsgMLDgYRuVKKXNiBFoDcOn2HKldwgR4n7DBiU0Df2XM+CkpwJBmw6HEEVPd4Y4B5NFY rTVISP5TqSDTBcCrIEcdLiBhFbmE+3fGbVFZUnajpM3Y8GuuTr7bqyR5ycaH0nAWEZo7 ZKXtn0/lgj3aeiOLi/Xjx5PxNUwv2Ob+Mcoa7yAlyjgfhMOy9NQTd0LqBArsBekilQaa jDkFWaBOu/9Ln16NiJMGrN91pCH3xATopka+TBs/v1hMvXioYH0CDbokO0IPWqXtVfzh Q6AQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=bdB5os8ysADiA9ihl7DyMq1Tj0074xySPTMD9uYQI04=; b=LDWdt0OWfhVUEp3Sy6fssIlN+Ebatot6Hhf84VWMTLjgayDKuNTgSvjCR89V48YP+Z uN/eZOQxIqSGo2S+psHeQNZTyAf5DPpcxfZao82HE/kvDYNTwidflfNQs43nGKHrk55y 8OWTW34jXIpeLjWCrt1bI1EmDRCpJwrUWj/tRKAZIryVjust97ox6/ynoUpqHaV/RUH9 sbQPw/wHawOc2rSyG/r3P7a33f8zhJrJWf5hZhEqU1T7RYYFhtmuXScip+I5YWwxZgLo ja3d+xOnw3WO06GhMhvmt4sv7Z+nw5pFVx51Wnp3ugWkuN8yXf4W/Sp4bioWNxi3SgDH kGsg== X-Gm-Message-State: AOAM5320fTRwwgq/JimUjTeQ6KtDy/LRHurwsObp3SLVFdlHxnEQC1u9 gCfEoVttgRpvT1ZLHf7nytAg/V6nfZU= X-Google-Smtp-Source: ABdhPJynbXdsCBVV65ml7OWnx5QZtstTl8K7PSihVxGiuFf6ynzxNTxGZVpRNHefapnbB5gJjgF1QQ== X-Received: by 2002:a5d:50c5:: with SMTP id f5mr1127445wrt.136.1618261708330; Mon, 12 Apr 2021 14:08:28 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id o38sm452999wmp.37.2021.04.12.14.08.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Apr 2021 14:08:27 -0700 (PDT) Message-Id: <175f3bc6b336539700d50b1906fd2a33c0bed906.1618261698.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 12 Apr 2021 21:08:05 +0000 Subject: [PATCH v3 14/26] rm: ensure full index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Before iterating over all cache entries, ensure that a sparse index is expanded to a full index to avoid unexpected behavior. Signed-off-by: Derrick Stolee --- builtin/rm.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/builtin/rm.c b/builtin/rm.c index 4858631e0f02..5559a0b453a3 100644 --- a/builtin/rm.c +++ b/builtin/rm.c @@ -293,6 +293,8 @@ int cmd_rm(int argc, const char **argv, const char *prefix) seen = xcalloc(pathspec.nr, 1); + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(&the_index); for (i = 0; i < active_nr; i++) { const struct cache_entry *ce = active_cache[i]; if (!ce_path_match(&the_index, ce, &pathspec, seen)) From patchwork Mon Apr 12 21:08:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12198829 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0AA80C43461 for ; Mon, 12 Apr 2021 21:08:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E37C46135C for ; Mon, 12 Apr 2021 21:08:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244609AbhDLVJF (ORCPT ); Mon, 12 Apr 2021 17:09:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57976 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231477AbhDLVIs (ORCPT ); Mon, 12 Apr 2021 17:08:48 -0400 Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 13169C06138D for ; Mon, 12 Apr 2021 14:08:30 -0700 (PDT) Received: by mail-wr1-x42b.google.com with SMTP id a4so14339685wrr.2 for ; Mon, 12 Apr 2021 14:08:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=y3kBuuE1kqkzlxpCDekvhONE9z9UoTHKBwBc33s5cD4=; b=O5MO2C3AYS+X5iehYFLyxk8+Nq9ObLhm5RrcSglwAngzvF4UdvogbeEtnkqpam5xHL bG6fEsZXVt2dlziZx7gPpiS0QOVbEB5iORGh7mOSjgEBIIZAItSCFhtnzzLMzjbEotm7 n0Jh22tyY2jpmHmC6X3x1nruRpW0GEC8abvIZID0PndTdq+642p/MDoYvmSvXqFtPzVt Ln2CDUXPq1tTFgD6JQlZ62m3NcVBJX9iT7+P3K7pmqrdstfUAlTvV5bXTHecXDVHIevg gJ6rsyC2PC5hwc/BUvbSjtbvdWCJ8jLE4UalhcTvT4zeJzk9ZumJFcpPl+dnm5S5cw17 5uMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=y3kBuuE1kqkzlxpCDekvhONE9z9UoTHKBwBc33s5cD4=; b=mJDiEaS76s0yV889B2XmE4BMLV8rF7VqESm3D2TAmtwER4GGc4Du77ZCiRNbjlZb3t lJgR/3A23B2IpRQn5NZqX8pEQrBrTBGjxOIUXOLC6BdiD9xTPGmC0Hr3ZJpn7vqyGGG1 wFOFJi39QAcs8iAylyRrjZDhpQREqZ2gz24Z79u73Hnqo9NYy1v1wrKBN+EaZHOFu9WY 5bQha+5vLEVnnS39EvBO88uZwG+CssPBawZhXNfOH3p0fTV3w9z7mBaxYpwWxIw5VT3B d1biA0b3nreRQtZSQIIezDvIA3cgOxcJN+F3L5eCsdP/1gvtdhI4g6N+C5gajxoKSEkr MzKw== X-Gm-Message-State: AOAM531oieIy/Yg6jb5zsojJI6OjddY1Pdt0Hacr1W3jLFNWwXNq3HdY giIXwOsJzbwD9PpFq3/3e3Ip8ZVyqN0= X-Google-Smtp-Source: ABdhPJznyGBd0xjM9bxJeIeog+ddV2qiNqzCaFdzNAOAaUCsL+ZR5yvfv+pWGrHIAPlD/d0rzYEpwA== X-Received: by 2002:a5d:6145:: with SMTP id y5mr25431940wrt.27.1618261708926; Mon, 12 Apr 2021 14:08:28 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id m15sm17068522wrp.96.2021.04.12.14.08.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Apr 2021 14:08:28 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Mon, 12 Apr 2021 21:08:06 +0000 Subject: [PATCH v3 15/26] stash: ensure full index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Before iterating over all cache entries, ensure that a sparse index is expanded to a full index to avoid unexpected behavior. Signed-off-by: Derrick Stolee --- builtin/stash.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/builtin/stash.c b/builtin/stash.c index ba774cce674f..6fb7178ef2fa 100644 --- a/builtin/stash.c +++ b/builtin/stash.c @@ -1350,6 +1350,8 @@ static int do_push_stash(const struct pathspec *ps, const char *stash_msg, int q int i; char *ps_matched = xcalloc(ps->nr, 1); + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(&the_index); for (i = 0; i < active_nr; i++) ce_path_match(&the_index, active_cache[i], ps, ps_matched); From patchwork Mon Apr 12 21:08:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12198831 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C1C8DC433ED for ; Mon, 12 Apr 2021 21:08:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A42C460C3E for ; Mon, 12 Apr 2021 21:08:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241290AbhDLVJG (ORCPT ); Mon, 12 Apr 2021 17:09:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57984 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242708AbhDLVIt (ORCPT ); Mon, 12 Apr 2021 17:08:49 -0400 Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A9E26C061574 for ; Mon, 12 Apr 2021 14:08:30 -0700 (PDT) Received: by mail-wm1-x32b.google.com with SMTP id y204so6192355wmg.2 for ; Mon, 12 Apr 2021 14:08:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=P9rJd368hm+Jl6xbpIBIuWVC2ZyfP3m2Ah742l9tbUc=; b=iQICstAfDohYBFWyDNnd7BcmbtZEElz14OcFO9G5UvIjszV9IHsV8+vAnb1wPXVLlE U209cjHaJaHgZOfImXluy9dSho43w61QW3iUkMysdNCaImhRL+AOdjocksmTLFaP8uj9 RG79o8DmDt5QdQawUTVZjMPNDny05JiopU4VIYHdQ/uuQfD24diObf9lmliy31fbh9E/ qobULB+5Tco1bDor8RR/fdoTJ/5PdYsr4LciYIZZ526Xs34NySLTD12ks+Lr2c8UG7dd Ry3cPcAijc/RZcedTK6LrKEaG8o3G6EGZl0m4pVvDDd17R4TnP4scRmrNKA2Rc32lc5N zk4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=P9rJd368hm+Jl6xbpIBIuWVC2ZyfP3m2Ah742l9tbUc=; b=HZ+A6UgzuxNn1dzZ779hCjULNPioQULqT+hFexpCWfFbkjtvEprKxQz1SW7g9QYIHS JTf3YTR+bAN3hV/Pvpc5BVQYA23jOASvPzJIf2IMfP9RUx/FJWFXBWlkp1ElzojZSDh9 ydsTnGGS/Hgi9c37Lu0vzpfil2MW9oCIExX2I4ojMXqln0bRCWM+DRr0xMdx/So67XZq NxXZonOg15X8i5JhZ6dQ90ABw03n70H0SSIMazc3BAA7bbeaOgWqcBxT1IC0LSzwJSKc e7+oCLHNWUrxDSWBhP13CFgV9+S4nCaziJowT9C9hUpG0oVbaB5et5IVCmPlezF7k6YW E/Hg== X-Gm-Message-State: AOAM532j9rlaO/SDjHGzEB7C8sh+9Tg9zd0m2yqIZ/Ub+4sqXrxjm3IJ jVGZhNVwEuLjssLTTuriFJjxu7yol24= X-Google-Smtp-Source: ABdhPJwWdKcrV0wfLVBAKSM6+ClDW4jNeOVEiEyeeB/RhchSKTPbnopspB8R5CI778aSRTxDxW11Sg== X-Received: by 2002:a1c:a145:: with SMTP id k66mr898164wme.54.1618261709506; Mon, 12 Apr 2021 14:08:29 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id m26sm482280wmg.17.2021.04.12.14.08.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Apr 2021 14:08:29 -0700 (PDT) Message-Id: <8c5336964d9b513825a9f7ac983ff06cc5757d2f.1618261698.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 12 Apr 2021 21:08:07 +0000 Subject: [PATCH v3 16/26] update-index: ensure full index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Before iterating over all cache entries, ensure that a sparse index is expanded to a full index to avoid unexpected behavior. Signed-off-by: Derrick Stolee --- builtin/update-index.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/builtin/update-index.c b/builtin/update-index.c index 79087bccea4b..f1f16f2de526 100644 --- a/builtin/update-index.c +++ b/builtin/update-index.c @@ -745,6 +745,8 @@ static int do_reupdate(int ac, const char **av, */ has_head = 0; redo: + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(&the_index); for (pos = 0; pos < active_nr; pos++) { const struct cache_entry *ce = active_cache[pos]; struct cache_entry *old = NULL; From patchwork Mon Apr 12 21:08:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12198833 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E12D3C433B4 for ; Mon, 12 Apr 2021 21:08:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C52B161369 for ; Mon, 12 Apr 2021 21:08:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244767AbhDLVJJ (ORCPT ); Mon, 12 Apr 2021 17:09:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57992 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242802AbhDLVIt (ORCPT ); Mon, 12 Apr 2021 17:08:49 -0400 Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com [IPv6:2a00:1450:4864:20::42e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 43F29C061756 for ; Mon, 12 Apr 2021 14:08:31 -0700 (PDT) Received: by mail-wr1-x42e.google.com with SMTP id s7so14202785wru.6 for ; Mon, 12 Apr 2021 14:08:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=7qKugyqjTL3Qic3WcAPItv5DSYivY5GEtPPJAkAOQqc=; b=KBpP6PA0nQJAOrRyKPergIr8Czl6FrT7L+fmPBQy3NFJYRbYrq1ZVRfm+3+JvQdvcR kxXJ3S8cT8fGbjx4RmMRwlJCQgscpUnrJgP6iSxYhYrIbhfeNMERvbg/iIbq4CKT0jk4 qfWYuyMJwmBLVBV56HKzFC1z2m9DE0PbeTbLc0MQ2jA6ifpAw5x3YqqPGyXbZrdBdQM3 lDZ4M14W6t2iHlJZnXzSzgZ1I+oHRF6vpb7sPbyuLAZLgFSsBdl4UzvE8RWYD2EQ1jY/ 2tGMnoJaFgQUz3JKSnnuEtQXaWAu8vY29DyczjKfpL8wIBRRk9AAqPIcTzg5hTqhaqGR GANw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=7qKugyqjTL3Qic3WcAPItv5DSYivY5GEtPPJAkAOQqc=; b=cgqFYlm4jSSN5XMr+deyKJNsG47JSnWqCPeSCDBkUG8QeAmP5VKyo1CD0z2WdiPWcs 1XpFztvmebR0ExjwMW4tWqK5m3knh//Uef5L70NjgpMxyjidBjVX/+4z1N3UuF7OXEKc dWWhfVbjIeDnJq1uYe74KJ2B+JFg8Ib+yRxBRkGpRucUX6d90fZeZ0U/HNWx8kxa/YKn DwPiygW/nVfPHgmCrWvbdor9NY/QoeBFRYrb1NA9+vxbH8xrNG/x0CXIoGCRFWAnfpv8 nlZ0LapzAIgrvElHuSAIskDUE5Sdl1WQefI3IuRHZpbrg4lvx9z8Pcjoo4i8JTt7BajN gDqw== X-Gm-Message-State: AOAM532AlKvca3UkSQgbXgnl+zMZwHEBL12FGdkWI0QKE77cTErAJ4H6 i2UrvyztGpvqfgbLZ7tADz6x4m5J3vE= X-Google-Smtp-Source: ABdhPJzwS1C8Lp3ESJqkZoMe8kS/XvrPvULdoJgKT41gxBepyprFovfgWHF6jPHz7eLRZ5bZ5h3StQ== X-Received: by 2002:adf:de08:: with SMTP id b8mr6709755wrm.279.1618261710120; Mon, 12 Apr 2021 14:08:30 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id g12sm8940316wru.47.2021.04.12.14.08.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Apr 2021 14:08:29 -0700 (PDT) Message-Id: <08a62c23c8f77c9ea033bf4f7b8e164bdff8b7de.1618261698.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 12 Apr 2021 21:08:08 +0000 Subject: [PATCH v3 17/26] dir: ensure full index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Before iterating over all cache entries, ensure that a sparse index is expanded to a full index to avoid unexpected behavior. Signed-off-by: Derrick Stolee --- dir.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/dir.c b/dir.c index 5b00dfb5b144..166238e79f52 100644 --- a/dir.c +++ b/dir.c @@ -3533,6 +3533,8 @@ static void connect_wt_gitdir_in_nested(const char *sub_worktree, if (repo_read_index(&subrepo) < 0) die(_("index file corrupt in repo %s"), subrepo.gitdir); + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(subrepo.index); for (i = 0; i < subrepo.index->cache_nr; i++) { const struct cache_entry *ce = subrepo.index->cache[i]; From patchwork Mon Apr 12 21:08:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12198835 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AF269C43461 for ; Mon, 12 Apr 2021 21:09:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9665E60C3E for ; Mon, 12 Apr 2021 21:09:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242802AbhDLVJS (ORCPT ); Mon, 12 Apr 2021 17:09:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57976 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243309AbhDLVIu (ORCPT ); Mon, 12 Apr 2021 17:08:50 -0400 Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D793FC06138D for ; Mon, 12 Apr 2021 14:08:31 -0700 (PDT) Received: by mail-wr1-x435.google.com with SMTP id m9so1588402wrx.3 for ; Mon, 12 Apr 2021 14:08:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=DN0bJRJEdqZFG9ILPnNiBkEoj0NZzXfi9k5Yav8/FLY=; b=piDnghmgtJmQgFm3sh1ePGzKYMAxeknLFfb2VIs9Sii7OaDOZNxG1POa+D1b7tgm00 Fg/gX1/MQbpBO4KpZCFXl0nElFwazJ8z3UFQv5Nl/epdsSm7MJ+oWCHp1vvXSLbrenEK 9SrUOnbIb1eWo2irCzJ7nd1+u6jV8cRbfHQBTJ5BY5jh4jIPN+N8QaGLoBcvwwaN953t I5FYKHu9eXEcKUs5pxvU2WdGAUzTWoRBMvWh68jsTWqzAAsbW6whubDZ09yDHlFJjoY/ IVgSy+qNSkOcuF288DxtnrIxKZIbkvA6Fe6wa+/+ynz18IsYttPCxLXsKuPrcDgovKJj BVfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=DN0bJRJEdqZFG9ILPnNiBkEoj0NZzXfi9k5Yav8/FLY=; b=WpEuTXaQOk0XyeE8KhlIEREa/BS80rlUeNyXEg40vX1gz9D5SUdnDB+hOFjwqYwRVa HSIngcwu3HweUzUK0GaUzyVa+gZpzaUy3EqEom24ogLJQISOJejlX0JqgCIdrhK2HVfc CMYn+sLBQiZKuJEY2MfPUunnnW6gR47Ts6pQT+PUz8YdtpmT+xBWtKo6lgMKq9+pPd4Z IynHuWqZVkGvdDEAcB70o0QYfYg/RF/Y0SQbIPZbvGNGgUJci8mZGOEU2IR/iLIuOR5G 3zj5q9cETHeHop0cwxVS7blQ8RgtHFkS2nriP5vK8rQfplxD47PIdgHK9OS7Zrqu/+RE hJhg== X-Gm-Message-State: AOAM531HDf4Qjx8ltWmfAB8eYoyOEzzUNJHBlFmZnN8FvAOV/ccsYRV5 hdAt+8GUi3866rBfMQne6TrF88o5t3U= X-Google-Smtp-Source: ABdhPJzjBxGTN36dTmTmBTZTiFEo5dgMDbyr/7wemsw4zvmP3DmnP4Qlx7IJAewvkjQhyKZB83CoEQ== X-Received: by 2002:a5d:5644:: with SMTP id j4mr33952195wrw.284.1618261710723; Mon, 12 Apr 2021 14:08:30 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id n2sm506381wmb.32.2021.04.12.14.08.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Apr 2021 14:08:30 -0700 (PDT) Message-Id: <825ebceee508bd5a00e5113bd43ccf82421ec5e2.1618261698.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 12 Apr 2021 21:08:09 +0000 Subject: [PATCH v3 18/26] entry: ensure full index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Before iterating over all cache entries, ensure that a sparse index is expanded to a full index to avoid unexpected behavior. Signed-off-by: Derrick Stolee --- entry.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/entry.c b/entry.c index 7b9f43716f76..891e4ba2b45a 100644 --- a/entry.c +++ b/entry.c @@ -412,6 +412,8 @@ static void mark_colliding_entries(const struct checkout *state, ce->ce_flags |= CE_MATCHED; + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(state->istate); for (i = 0; i < state->istate->cache_nr; i++) { struct cache_entry *dup = state->istate->cache[i]; From patchwork Mon Apr 12 21:08:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12198837 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9BA8AC43462 for ; Mon, 12 Apr 2021 21:09:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 825CD6135C for ; Mon, 12 Apr 2021 21:09:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243872AbhDLVJT (ORCPT ); Mon, 12 Apr 2021 17:09:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57984 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243376AbhDLVIv (ORCPT ); Mon, 12 Apr 2021 17:08:51 -0400 Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8F4C2C061574 for ; Mon, 12 Apr 2021 14:08:32 -0700 (PDT) Received: by mail-wm1-x32c.google.com with SMTP id n11-20020a05600c4f8bb029010e5cf86347so161377wmq.1 for ; Mon, 12 Apr 2021 14:08:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=Cwy0/DFLfwxmnrNn1jvtNHYmLs4yYZjayHJKnh4xlfU=; b=rUlJ0gxXlwjIvOdvZ/jffbzyFvsjzgKORI7B77n7I0Yh4aqhZVwdy4rDAISUJYZ7t2 L0KD9YCy+YhoeqDHabfQcaLrigcIw7Z6JHVPkIMQM1f4eXhlJOz+G79RBUSqSWgbnVk2 DWbc9C8e1wFnHTN31JeyW/V/pfDBkAyBiD2Q5Hr7Lab1iJczg7Q5yW3P1cpoJ2LIU/71 rnvPlCAh5LGveYX8SUzfstOxH3gEsjDP3soS6gmOBs0NHydqTturWRRECoxYEnvIvobw OrK2o0W9xvyDnj1e8efghi1NicKLNRZwB5Q+Um0X0kcYSt3yrziMOeLNTFWA24k0Z+VU YYdA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=Cwy0/DFLfwxmnrNn1jvtNHYmLs4yYZjayHJKnh4xlfU=; b=RjJVXeBUl1Y/y90EM/MhGoDS5qWZQ/Z5AdOOfAP6bL0OK/yTUrBczx40ySgstemIfv MUTtz0gjy7XZaOd8+02jkGUhy8m36d4oVEk70eIB1b6nSIhTgoFZzW46g2zuF0+mYduL Yj7MksfJ6+RxiN+QMQXX64MLiQZip9sZHxiDyOSVbnoAeCsa/m1ASBgkP0a1+JKUHgvp 1bKH1Y5OH9kH6ogU68LMmPjiNEurA2d1JlgMrmK8ZRzCKuvXZrG8JcV14kAnwlDGihwQ qQnamEBGjkjKIpyMnOrbQ5dDxpjHRf7s6oOa/0YeWJKFYpzXW/wWK0ZsL+2qQDOZF+7C Rkbg== X-Gm-Message-State: AOAM533D3Ck+QDvM8nu9Zfb04v4wohJTFruM3/NjpIHjHAHiTvEmRurh ENkqYlhfcCJ0IsQYOIuvgkpnz5sGA2M= X-Google-Smtp-Source: ABdhPJy05qVgQayUBvQ/snjURD2xF4P6DKIBpiu6AsY+S7no5xz5M/DS5/+OBKE/QbwwLr9Xf/ATQw== X-Received: by 2002:a1c:9dd1:: with SMTP id g200mr922483wme.60.1618261711357; Mon, 12 Apr 2021 14:08:31 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id l14sm18289883wrm.77.2021.04.12.14.08.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Apr 2021 14:08:31 -0700 (PDT) Message-Id: <3673db5172352152fcaa29feafae432530f36b36.1618261698.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 12 Apr 2021 21:08:10 +0000 Subject: [PATCH v3 19/26] merge-recursive: ensure full index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Before iterating over all cache entries, ensure that a sparse index is expanded to a full index to avoid unexpected behavior. Signed-off-by: Derrick Stolee --- merge-recursive.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/merge-recursive.c b/merge-recursive.c index b8de7a704eae..91d8597728c1 100644 --- a/merge-recursive.c +++ b/merge-recursive.c @@ -522,6 +522,8 @@ static struct string_list *get_unmerged(struct index_state *istate) unmerged->strdup_strings = 1; + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(istate); for (i = 0; i < istate->cache_nr; i++) { struct string_list_item *item; struct stage_data *e; From patchwork Mon Apr 12 21:08:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12198847 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 974CDC433ED for ; Mon, 12 Apr 2021 21:09:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7BEB66135C for ; Mon, 12 Apr 2021 21:09:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244029AbhDLVJY (ORCPT ); Mon, 12 Apr 2021 17:09:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58012 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240719AbhDLVIv (ORCPT ); Mon, 12 Apr 2021 17:08:51 -0400 Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [IPv6:2a00:1450:4864:20::32e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2631BC06138E for ; Mon, 12 Apr 2021 14:08:33 -0700 (PDT) Received: by mail-wm1-x32e.google.com with SMTP id y204so6192408wmg.2 for ; Mon, 12 Apr 2021 14:08:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=xOPUNFGXkgQBSC36MohDBVpe4Y/fE1xOn5Zg727kSss=; b=IyKjaH/sw3WVd3l2I4fKZBsWh2RMJEzGGxaeAXj5BaJPrvDGj96CDuWHkZRe8led2q xhhQ007fnwLd7ZB1hGIQPIOy3APIqblWIcnCF2O+NbgoT8yjfgNgEgtqOBFPnOPNyk7F 3egB1MTmUcNT+Jviims8m70sF3wzK0xmFpdbrPJC6hHgLS6t2HdjOGdzYj8LFXypjI/+ c6XrGUmwz/YOLUqr871SD8AH+Qx/e6aOo0fk+KRLLLPK6Y/GI4zijk/MnB6qRTOJ7c+H QffX59UHdAv8w0LyEpfx2+3wfpyb/f1ZiPIVqq2G2SXPtuO2yXhAUvXkyIKng3tWc6Dt snIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=xOPUNFGXkgQBSC36MohDBVpe4Y/fE1xOn5Zg727kSss=; b=cIbZX+buOU7gIKHykdaEPSOXQmYq/+06e2yOixCOnSDtoxB/0v/zWLzOOVj3Nlxd7a L/0wzh7MMc+hXMDb9XB5fqcUznQGDG9NpEeGeVC6K7TNoNC9j+cXRTZ5QXSNOdKFZCFN i0FEdAhOigIPvMmGY7gHdLbWMI1tWg40pyBZku1XU15XbmprSjC9EszGLrwK37dJh+Pe eT1/mBaC5wv7TJHbVYF8MqLTJRNV7lKTLURaOtXR8HHSZ/ExSfKnppS1hvfnsUpv7CVK tj9OOBjK5UuR+QeJb9ApmA9hoC8bJNNZtdBVEjelh2CmVzy8UtI464/iFnYpEC9NGFNB 97ig== X-Gm-Message-State: AOAM5315eaXg+cnD6LOEHEKEhYRpYg2qI9WjYSeU1uIVEc5jhzIhIiHl 7zlGtYeK83qOQpXpTndYnhC3H1IkNtA= X-Google-Smtp-Source: ABdhPJxB/ALf2MqPhrsQXLGvWrju7Gs4W6oUBlGt3U3IiVQt0Ra9SE+r/encs77iZjZINDKUHhCvkg== X-Received: by 2002:a1c:658a:: with SMTP id z132mr846118wmb.39.1618261711989; Mon, 12 Apr 2021 14:08:31 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id x14sm17574120wrw.13.2021.04.12.14.08.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Apr 2021 14:08:31 -0700 (PDT) Message-Id: <4d3f6de29a6393e46923bbaece69ed7e21a03101.1618261698.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 12 Apr 2021 21:08:11 +0000 Subject: [PATCH v3 20/26] pathspec: ensure full index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Before iterating over all cache entries, ensure that a sparse index is expanded to a full index to avoid unexpected behavior. Signed-off-by: Derrick Stolee --- pathspec.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/pathspec.c b/pathspec.c index b6e333965cb4..d67688bab74b 100644 --- a/pathspec.c +++ b/pathspec.c @@ -36,6 +36,8 @@ void add_pathspec_matches_against_index(const struct pathspec *pathspec, num_unmatched++; if (!num_unmatched) return; + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(istate); for (i = 0; i < istate->cache_nr; i++) { const struct cache_entry *ce = istate->cache[i]; ce_path_match(istate, ce, pathspec, seen); From patchwork Mon Apr 12 21:08:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12198845 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1F8C5C433B4 for ; Mon, 12 Apr 2021 21:09:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0660F60238 for ; Mon, 12 Apr 2021 21:09:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245518AbhDLVJX (ORCPT ); Mon, 12 Apr 2021 17:09:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58016 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243433AbhDLVIw (ORCPT ); Mon, 12 Apr 2021 17:08:52 -0400 Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [IPv6:2a00:1450:4864:20::32f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CB340C06138F for ; Mon, 12 Apr 2021 14:08:33 -0700 (PDT) Received: by mail-wm1-x32f.google.com with SMTP id y204so6192421wmg.2 for ; Mon, 12 Apr 2021 14:08:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=gXHTyk/0rr+wbe8rzhDt1uZ1A0ElBNbmaZcI4hkvDxk=; b=n92g8NPj336nQRolmUvTur7HdaM2qpUSWromvq6dqRFgCn7auvt0K1jSbUvEPcO7fE wui03EiGOkVvmcJqQdlgZ9NvyxxUNqG/frYwzmWpYXoZHAAc5W2VzbfBoBoG9Pd/AC9j fkEtcnEXYNAaBC31AHZ0cBKDhiLi6Jp4z4NXbNeHlR68rdNU6FWaRf5c2xuO3I9mdRq3 VHicq4uwP8EryOR6QfgN5UlLS1VeS8SFISVI7BUocUbz9S8/9XevPqK0/ANcagpTzweu JYwm8U1qgLPk3QG8eRyQXXUaH+8kMtnc1CIRz9jRgB2TvP2wI3b9GDxYdOTy2FgZ8BKi XkAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=gXHTyk/0rr+wbe8rzhDt1uZ1A0ElBNbmaZcI4hkvDxk=; b=Tv6nBZfrnJKqciDUMzdTpuWRRjEyMztWPvQfunUmonB2xxFKr+/Ts+lzqDrVcdshIL dd+b+FtzPnst0cyqP/akiD3ATcuxakQwIL0OYWEc8zfYAS5X9E+ZN1rPbVtBbERpiLcv I366o5/3OJG7V6ZhpdEwK4WIxq8FfJ/exrrQOPEyWFz5WaT/KxJuRBlpo/B1UVnfYbdR HiCVQwoq8rB4tNTWqM4clS3lAAvk5mqjJG82+30+yGk9VCOKlQc8fB+xuElL2OGnbd41 Det1VLiXF6sABumHTPU51FyRvA5PvO+ZPjoCAd+uOt2quYjtWcwmZI9oUO0B1JPwwcVQ n9+A== X-Gm-Message-State: AOAM532tjatYlJsX01okFxN49EpkgBLB2BcJtwj3HEa9N6FJ6y7g5ejH V6vDaRfPteErUV/rfPoKp2bm4okmK7g= X-Google-Smtp-Source: ABdhPJzwVir/NrkxpQkGW5HIhOm6YJkV39/jBSYZuHlircGdziATVZ9/5e4E5CzrRAFubwPqnN5+qA== X-Received: by 2002:a1c:c2c2:: with SMTP id s185mr866459wmf.33.1618261712650; Mon, 12 Apr 2021 14:08:32 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id t63sm478292wma.20.2021.04.12.14.08.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Apr 2021 14:08:32 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Mon, 12 Apr 2021 21:08:12 +0000 Subject: [PATCH v3 21/26] read-cache: ensure full index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Before iterating over all cache entries, ensure that a sparse index is expanded to a full index to avoid unexpected behavior. Signed-off-by: Derrick Stolee --- read-cache.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/read-cache.c b/read-cache.c index 3698bc7bf77d..a9dcf0ab4f78 100644 --- a/read-cache.c +++ b/read-cache.c @@ -1577,6 +1577,8 @@ int refresh_index(struct index_state *istate, unsigned int flags, */ preload_index(istate, pathspec, 0); trace2_region_enter("index", "refresh", NULL); + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(istate); for (i = 0; i < istate->cache_nr; i++) { struct cache_entry *ce, *new_entry; int cache_errno = 0; @@ -2498,6 +2500,8 @@ int repo_index_has_changes(struct repository *repo, diff_flush(&opt); return opt.flags.has_changes != 0; } else { + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(istate); for (i = 0; sb && i < istate->cache_nr; i++) { if (i) strbuf_addch(sb, ' '); From patchwork Mon Apr 12 21:08:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12198843 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D2146C433ED for ; Mon, 12 Apr 2021 21:09:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B8A9B60238 for ; Mon, 12 Apr 2021 21:09:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244301AbhDLVJU (ORCPT ); Mon, 12 Apr 2021 17:09:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58022 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243496AbhDLVIx (ORCPT ); Mon, 12 Apr 2021 17:08:53 -0400 Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [IPv6:2a00:1450:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5B4CDC061342 for ; Mon, 12 Apr 2021 14:08:34 -0700 (PDT) Received: by mail-wr1-x42d.google.com with SMTP id h4so5292449wrt.12 for ; Mon, 12 Apr 2021 14:08:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=5ujY+w6SXlDeI30iu//cxteaQB/3nSsRybSiz0GS0fM=; b=ThkE5qKrUX7062q6R4WCLsurlCmrqjRl/At9Ni+lrV/+N7U/hIW7eFu+jE6T+kms0j rMSpyOaTj3PMp6Pyslg+xptB7uvteHhpNhDxOi8PotvUJxxh+Ep0Xlk4LliuIco8HHvP R2B49akYRhf6t8R7G6I1qHExY8/sbIxONY5EsKwMMPq1Zx4lNGEit5z62jdzSgS3Yuzn IVTrcj/7ozXsOXDaEA6UQKyWfs40NOzsyVEEnJa0B31j40Fw43PEY/HhAYiT16zpbwrl h/G6vaKHeb6390Y4yey987qs4H8RQa/qSwKRVDGmiqXMttfWdNp7lDkoxU9vTERcHgax z7VQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=5ujY+w6SXlDeI30iu//cxteaQB/3nSsRybSiz0GS0fM=; b=LG3eHESQA2Kh0XIbZ/Ke6oxJLtw9WVGaXdTH9Ah8DV1o6D6BU8kQR87YOksSog65Gi +nNJ9QlUPHyCRvjpj0rFRdP3PBGQrRZBrwkuBiK1eLk/K6MMZOqxxk/zcqfzrIirS4Ds /eGvkV9v9ya/nxgucIjake48S2R68VlrgehBwCAI5/oE0kYzxNiL2RDj1Qv0bbUmU98F PPWlUyXYHJAMYGvmLFTcqXgndjcVOLSDhBc3t0dHN5hlYv/8ND1z1UyFXE3rUJF2U2Bd 9kP3WT4OT0LSVocy9Si90LXO7Drpq2/OwlaJsDpcRlBRfBx17VtwipkjvMc0OmvzQ50E lJ6w== X-Gm-Message-State: AOAM533D9bMM4UFnUHm8lBXLopH3zFA9M6Lu2ylZIO9JRYR+p5SataR6 2aWld69bScBdjsg2s9HxAsgoiXSOJB8= X-Google-Smtp-Source: ABdhPJzSvo1KU9CQbyJFUxIL6YWY/Jw4Xxe7dcoqSryIqcNuDAmOW/vFwF8DTm0F2tEh/gaUeEfmxA== X-Received: by 2002:a5d:6a11:: with SMTP id m17mr34404667wru.361.1618261713194; Mon, 12 Apr 2021 14:08:33 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id z17sm5988031wro.1.2021.04.12.14.08.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Apr 2021 14:08:32 -0700 (PDT) Message-Id: <38f295a41ec1bcbf0a67b0b1e12bd2590877f576.1618261698.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 12 Apr 2021 21:08:13 +0000 Subject: [PATCH v3 22/26] resolve-undo: ensure full index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Before iterating over all cache entries, ensure that a sparse index is expanded to a full index to avoid unexpected behavior. Signed-off-by: Derrick Stolee --- resolve-undo.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/resolve-undo.c b/resolve-undo.c index 236320f179cb..ee36ca58b135 100644 --- a/resolve-undo.c +++ b/resolve-undo.c @@ -172,6 +172,8 @@ void unmerge_marked_index(struct index_state *istate) if (!istate->resolve_undo) return; + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(istate); for (i = 0; i < istate->cache_nr; i++) { const struct cache_entry *ce = istate->cache[i]; if (ce->ce_flags & CE_MATCHED) @@ -186,6 +188,8 @@ void unmerge_index(struct index_state *istate, const struct pathspec *pathspec) if (!istate->resolve_undo) return; + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(istate); for (i = 0; i < istate->cache_nr; i++) { const struct cache_entry *ce = istate->cache[i]; if (!ce_path_match(istate, ce, pathspec, NULL)) From patchwork Mon Apr 12 21:08:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12198841 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CBE3CC433B4 for ; Mon, 12 Apr 2021 21:09:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AD96760238 for ; Mon, 12 Apr 2021 21:09:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245050AbhDLVJU (ORCPT ); Mon, 12 Apr 2021 17:09:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58024 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243522AbhDLVIx (ORCPT ); Mon, 12 Apr 2021 17:08:53 -0400 Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DFA56C061343 for ; Mon, 12 Apr 2021 14:08:34 -0700 (PDT) Received: by mail-wr1-x436.google.com with SMTP id f12so14365236wro.0 for ; Mon, 12 Apr 2021 14:08:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=hhLYYq4GDu3XvoSbcr7sJoPjWfM0oLCRJFTPz+pdmsw=; b=RJFbRS7rnZmVppqDNjHs9yWRY/YOSGq/GoyS3S0VkNReGRDIzRrAZ3p1Qs6mDmnDuE MsrEK86g+RlM3jYCNhulMvMHBgjV6U+5roCOYyLOpKmolr6LYL+KMkbCwhDKBiStzjbk Afgl3HipzM11LX+kOm8CkI/7HxJh2TnLzATYslcgQG7DNOLlvW38nGJ8t8bQ3yGnVtWq 31gawdS2ijQPXpCSazElELJH7hyyRwFJyXqqgIqzQsNa/3CyBRttU3CgRuX04piCeGQ4 fwHpiBdtE8OxqkHIjNUVCNH2aJoaTnwKLT8eJLExXei6N/FW/Mn8z+A2LVw1Eln1TBM7 FlCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=hhLYYq4GDu3XvoSbcr7sJoPjWfM0oLCRJFTPz+pdmsw=; b=sAV2zVwL+dyw2yWmc+wq+dDGHdmb7br3+cq04yknWxKjrizh1oQVcYJylo5k5Bt7NS FUWtVM+d7NPtjMjvGr9S0vq/Kc2k3NRALEXX7sp4ST2aEYqKL4rMYMyr6O7wbvUL280v gMFmflOWtxh9s+U6kRGzDd/2pUN5kMp5WzvdNQD8JBb7THcmC6rfQIAfq+rp5DKX2GfM dL7hoo7QdmeRjm9ue6FonZjMFRtpu84r1F+UDMfCHKoAjkH4JNjxqE8cdIHL6l9DmuAN K5q3YMq23Wow08Jp7at9kGRbUtBjFqP5CLyfFcdHnQO9kk8T8C+pqD17zGNxvv4Dt7j1 pMAQ== X-Gm-Message-State: AOAM532dy1OHZFofusQMhztXisoXwVEEK8SBIOarZa9JrXCQr+o+aTtc MldzF6NK3xul9lH2On69yDDjporHg+k= X-Google-Smtp-Source: ABdhPJziJ0ScQmt+bK08n7mHKv64MTse2AchkLCg4hVumLeSgpInujNlIAEv19SniKzA8EWpwEXyWg== X-Received: by 2002:a5d:62cd:: with SMTP id o13mr4162671wrv.77.1618261713755; Mon, 12 Apr 2021 14:08:33 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id z66sm505213wmc.4.2021.04.12.14.08.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Apr 2021 14:08:33 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Mon, 12 Apr 2021 21:08:14 +0000 Subject: [PATCH v3 23/26] revision: ensure full index Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Before iterating over all index entries, ensure that a sparse index is expanded to a full index to avoid unexpected behavior. This case could be integrated later by ensuring that we walk the tree in the sparse-directory entry, but the current behavior is only expecting blobs. Save this integration for later when it can be properly tested. Signed-off-by: Derrick Stolee --- revision.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/revision.c b/revision.c index b78733f5089b..b72e0ac1bdca 100644 --- a/revision.c +++ b/revision.c @@ -1680,6 +1680,8 @@ static void do_add_index_objects_to_pending(struct rev_info *revs, { int i; + /* TODO: audit for interaction with sparse-index. */ + ensure_full_index(istate); for (i = 0; i < istate->cache_nr; i++) { struct cache_entry *ce = istate->cache[i]; struct blob *blob; From patchwork Mon Apr 12 21:08:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12198839 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46222C43460 for ; Mon, 12 Apr 2021 21:09:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 297E961369 for ; Mon, 12 Apr 2021 21:09:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244188AbhDLVJT (ORCPT ); Mon, 12 Apr 2021 17:09:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58028 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240409AbhDLVIy (ORCPT ); Mon, 12 Apr 2021 17:08:54 -0400 Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6C026C061344 for ; Mon, 12 Apr 2021 14:08:35 -0700 (PDT) Received: by mail-wr1-x429.google.com with SMTP id s7so14202920wru.6 for ; Mon, 12 Apr 2021 14:08:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=ikt/CHZ3vNbn9OBWK+EZ28ZSXcACALejndYwRFSQIwY=; b=IM9TN1pZF2wykmVwxFpRRaPLNIqais2Ys9ZQbf/oKFA831IpgxZX8aWYEbVw3hBi5h bLzt67NULIOq/5yzW+iQb6Y86qyCUed1EL+PYdi31pzxjKWmpRDMW2Tq2J3/b2ttj7S/ WEOf5F8AJVKQfd2prne/kx5xLqXhnojRzB6VVf7brVXy6boRLUDUuOPHXKUbglHb8BCN 74YFDQCQRp6Xj5xSUz8Brd+dSYpPBK1kfnaiHtG8v1VJKYrBRXgjpBKPtKUtyERafjUU 9dDZi2/54hCM0ugJehcQ2vTbdhvc9Z49AzUxVOs/LHlTszwrlmbaMz2/bpMa5r7kUW8W KkxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=ikt/CHZ3vNbn9OBWK+EZ28ZSXcACALejndYwRFSQIwY=; b=fZOHNeSFYip3niwulzFOGZ2DiyJhE5bYZpO+jNTBGZ9W6WLclOiSMBMB+g6X159zEO FB6pk0Dtg65DzE7+MOyHSWn6wyuHBknFt7lYU2DrzJFvikIfDDta4VjLQuBUcjCQcnQZ QVbog3xFwJi3KDZQWbQSBbbEDKR+yQUfOLQv0R8C1I5JgWWmn+Dvsb7BKkVZXYJQWQDr +ImI6oigYcO1dFIrneVbmR9Q5gAdpM705dLLxs21cxA6qcjEN/4KGeX0KyDMQi3Tfx78 FhgjFDdJ+qs+H5/3HGvT6zbue6AJTJhR/mSQliEKOed7X2o5AZAPCXbJtASQgzjtF4R2 d1dA== X-Gm-Message-State: AOAM530buU9x10pQico8Sptk/1hs51R8y7eyO5QX/B3yvDmP342XeW3X TXdUFfmaDe3YIqId1XDK96QEpdIz5LU= X-Google-Smtp-Source: ABdhPJxUpXkvO6Uf+ohpuvnAvJjjhjtScYlB+2rJoNlRp1KsmyisUaLvUOrGSAqZBUfWpvh5Lk1rqQ== X-Received: by 2002:adf:f04d:: with SMTP id t13mr8944837wro.52.1618261714281; Mon, 12 Apr 2021 14:08:34 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id w7sm18194089wrt.15.2021.04.12.14.08.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Apr 2021 14:08:34 -0700 (PDT) Message-Id: <5fd83dcf2747dddc04a1386b7e57a48d3d9aa49b.1618261698.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 12 Apr 2021 21:08:15 +0000 Subject: [PATCH v3 24/26] name-hash: don't add directories to name_hash Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Sparse directory entries represent a directory that is outside the sparse-checkout definition. These are not paths to blobs, so should not be added to the name_hash table. Instead, they should be added to the directory hashtable when 'ignore_case' is true. Add a condition to avoid placing sparse directories into the name_hash hashtable. This avoids filling the table with extra entries that will never be queried. Signed-off-by: Derrick Stolee --- name-hash.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/name-hash.c b/name-hash.c index 4e03fac9bb12..d08deaa2c9e7 100644 --- a/name-hash.c +++ b/name-hash.c @@ -109,8 +109,11 @@ static void hash_index_entry(struct index_state *istate, struct cache_entry *ce) if (ce->ce_flags & CE_HASHED) return; ce->ce_flags |= CE_HASHED; - hashmap_entry_init(&ce->ent, memihash(ce->name, ce_namelen(ce))); - hashmap_add(&istate->name_hash, &ce->ent); + + if (!S_ISSPARSEDIR(ce->ce_mode)) { + hashmap_entry_init(&ce->ent, memihash(ce->name, ce_namelen(ce))); + hashmap_add(&istate->name_hash, &ce->ent); + } if (ignore_case) add_dir_entry(istate, ce); From patchwork Mon Apr 12 21:08:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12198851 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8CFD5C433B4 for ; Mon, 12 Apr 2021 21:09:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7261C610CE for ; Mon, 12 Apr 2021 21:09:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244509AbhDLVJ1 (ORCPT ); Mon, 12 Apr 2021 17:09:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58030 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240417AbhDLVIy (ORCPT ); Mon, 12 Apr 2021 17:08:54 -0400 Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com [IPv6:2a00:1450:4864:20::32d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0ACDDC061345 for ; Mon, 12 Apr 2021 14:08:36 -0700 (PDT) Received: by mail-wm1-x32d.google.com with SMTP id y124-20020a1c32820000b029010c93864955so9455859wmy.5 for ; Mon, 12 Apr 2021 14:08:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=UTgS6fWlaPrB1tG5T4dMOidBrl95/NqnCBhlxrzN4js=; b=mLnJ9KO6o9Q/sSA2iK8D4fO4GBya9rJ6zqPseCkB81XwHUjNfKvJ4RgHSVyl1/Ceh3 8n1uonMA3LjFu0P1oaSKbw9wIVUCXsccvdJ0ffiyV7nD/Zz3P37kwm+CQndfoNkysKKA h5Nd2BXTx8zAcCt6HE12+NkOU5uHcjFObX6s0jWdthTOcMYiW/GqKnLFXxeBZ77ur4bX szlC8K3kqDu3krqzxttqVldni7I/YJ0dcUZcQ46aeT9N8ldufySZYLHnZPNpJc+Qi00p FmjLaoYrziL+fewlttKocAK9fpGLQg67ErYCM3W6j8xOZdAuAv8tk6ug53DLGPmRL00W 3Tbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=UTgS6fWlaPrB1tG5T4dMOidBrl95/NqnCBhlxrzN4js=; b=OjjAGun0JWKItUZgqX1A1ifQo1mPooKmtMyuy1QRgizm3HtCFJMyn3/X+8Eh33Gwjn qmfyHoCFYQK+zz2cSveTM4l9HyV8vJbHSv0LH1JWawKQ/fPI/jMYF+gEkuYiUfZjuVLX Nev+W4PbU8rqb4KzcA6+XpyVdrfMcczeDHfz2sKr4CAb35Uy+EyL/V+ngjL8DI06ajVv GwVVhcTyVD0aanfe/zLoYPMH+n271DOMjI/WGZ+LC4BPz7lKuX/cS2VotsGzpjU2tc8D p00RuPfIcwQrRvRHUQQ2UTU1OyTbly+cufCzciqwO+JEHGPvP4EAPLg2LmO8bQkU9ZhP Uucg== X-Gm-Message-State: AOAM533C1xH9lea7wMJPMnx3edv6rfcYidWDfT6ptz87uVzXauPBc7Ox YsEWX/vmCC5KLFTuPqS9dm+FnAU1H+4= X-Google-Smtp-Source: ABdhPJzR3JPNqAxzIneBlUj1eNyN+qG3ZQVgVAfsOD0P3I7G2ZW752yF45ju+6ok9VYmh6+d6RL2jQ== X-Received: by 2002:a7b:cf12:: with SMTP id l18mr867510wmg.37.1618261714802; Mon, 12 Apr 2021 14:08:34 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id s13sm18191996wrv.80.2021.04.12.14.08.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Apr 2021 14:08:34 -0700 (PDT) Message-Id: <335fec3676a0f66dc68582027970b0bad886bde7.1618261698.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 12 Apr 2021 21:08:16 +0000 Subject: [PATCH v3 25/26] sparse-index: expand_to_path() Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Some users of the index API have a specific path they are looking for, but choose to use index_file_exists() to rely on the name-hash hashtable instead of doing binary search with index_name_pos(). These users only need to know a yes/no answer, not a position within the cache array. When the index is sparse, the name-hash hash table does not contain the full list of paths within sparse directories. It _does_ contain the directory names for the sparse-directory entries. Create a helper function, expand_to_path(), for intended use with the name-hash hashtable functions. The integration with name-hash.c will follow in a later change. The solution here is to use ensure_full_index() when we determine that the requested path is within a sparse directory entry. This will populate the name-hash hashtable as the index is recomputed from scratch. There may be cases where the caller is trying to find an untracked path that is not in the index but also is not within a sparse directory entry. We want to minimize the overhead for these requests. If we used index_name_pos() to find the insertion order of the path, then we could determine from that position if a sparse-directory exists. (In fact, just calling index_name_pos() in that case would lead to expanding the index to a full index.) However, this takes O(log N) time where N is the number of cache entries. To keep the performance of this call based mostly on the input string, use index_file_exists() to look for the ancestors of the path. Using the heuristic that a sparse directory is likely to have a small number of parent directories, we start from the bottom and build up. Use a string buffer to allow mutating the path name to terminate after each slash for each hashset test. Signed-off-by: Derrick Stolee --- sparse-index.c | 73 ++++++++++++++++++++++++++++++++++++++++++++++++++ sparse-index.h | 13 +++++++++ 2 files changed, 86 insertions(+) diff --git a/sparse-index.c b/sparse-index.c index 95ea17174da3..6f21397e2ee0 100644 --- a/sparse-index.c +++ b/sparse-index.c @@ -283,3 +283,76 @@ void ensure_full_index(struct index_state *istate) trace2_region_leave("index", "ensure_full_index", istate->repo); } + +/* + * This static global helps avoid infinite recursion between + * expand_to_path() and index_file_exists(). + */ +static int in_expand_to_path = 0; + +void expand_to_path(struct index_state *istate, + const char *path, size_t pathlen, int icase) +{ + struct strbuf path_mutable = STRBUF_INIT; + size_t substr_len; + + /* prevent extra recursion */ + if (in_expand_to_path) + return; + + if (!istate || !istate->sparse_index) + return; + + if (!istate->repo) + istate->repo = the_repository; + + in_expand_to_path = 1; + + /* + * We only need to actually expand a region if the + * following are both true: + * + * 1. 'path' is not already in the index. + * 2. Some parent directory of 'path' is a sparse directory. + */ + + if (index_file_exists(istate, path, pathlen, icase)) + goto cleanup; + + strbuf_add(&path_mutable, path, pathlen); + strbuf_addch(&path_mutable, '/'); + + /* Check the name hash for all parent directories */ + substr_len = 0; + while (substr_len < pathlen) { + char temp; + char *replace = strchr(path_mutable.buf + substr_len, '/'); + + if (!replace) + break; + + /* replace the character _after_ the slash */ + replace++; + temp = *replace; + *replace = '\0'; + if (index_file_exists(istate, path_mutable.buf, + path_mutable.len, icase)) { + /* + * We found a parent directory in the name-hash + * hashtable, because only sparse directory entries + * have a trailing '/' character. Since "path" wasn't + * in the index, perhaps it exists within this + * sparse-directory. Expand accordingly. + */ + ensure_full_index(istate); + break; + } + + *replace = temp; + substr_len = replace - path_mutable.buf; + } + +cleanup: + strbuf_release(&path_mutable); + in_expand_to_path = 0; +} diff --git a/sparse-index.h b/sparse-index.h index 0268f38753c0..1115a0d7dd98 100644 --- a/sparse-index.h +++ b/sparse-index.h @@ -4,6 +4,19 @@ struct index_state; int convert_to_sparse(struct index_state *istate); +/* + * Some places in the codebase expect to search for a specific path. + * This path might be outside of the sparse-checkout definition, in + * which case a sparse-index may not contain a path for that index. + * + * Given an index and a path, check to see if a leading directory for + * 'path' exists in the index as a sparse directory. In that case, + * expand that sparse directory to a full range of cache entries and + * populate the index accordingly. + */ +void expand_to_path(struct index_state *istate, + const char *path, size_t pathlen, int icase); + struct repository; int set_sparse_index_config(struct repository *repo, int enable); From patchwork Mon Apr 12 21:08:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12198849 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CECEFC433B4 for ; Mon, 12 Apr 2021 21:09:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B48876135C for ; Mon, 12 Apr 2021 21:09:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244303AbhDLVJZ (ORCPT ); Mon, 12 Apr 2021 17:09:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58040 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243998AbhDLVIz (ORCPT ); Mon, 12 Apr 2021 17:08:55 -0400 Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 96190C061346 for ; Mon, 12 Apr 2021 14:08:36 -0700 (PDT) Received: by mail-wm1-x334.google.com with SMTP id p19so7601218wmq.1 for ; Mon, 12 Apr 2021 14:08:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=Ry30Ub54zs5fS2rT/tk3DbrPmbbOYHZgw4kzgYnFvxE=; b=dFpHbp7eDAWCFg6WOZ/2EGai7Jxhg01GOe8dXaye7iGIz2JW8uWg3Ip1BcPYj0o1Y1 tCyG0HhCj1j7dEH7j7ltdv3p3q2QlYieSu3w1PhainBpFg0HJFYteI05kjl8OJv6Awpt arnLphIxN2tx4hh+eGhh3FGLZEihnqGifsgFmCJTN8n+zgCOfjomvxK4SWo2oMDpqj9f IpyfElXUMaKqYu/gJ6Nn6+n1z60Y4jqhiOq3IS+semmMZ49GCxokJNdKbHCEcRnMzusb u8sEWSEvUlnJ2HTaBOsdaa2PQvpC9j0LoMUUad6bTW0IxSu+832HUtx8WAFEck4MttsG vRLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=Ry30Ub54zs5fS2rT/tk3DbrPmbbOYHZgw4kzgYnFvxE=; b=jkOLk700Yqsj5VedL/lGKMspPTP+N95ErdwVeuB4Vmkrs15nS7/dmdbbbNRi2r4SM9 rhppH8wDiPpxMSA79bXxaYcErB/Mw7LW3nMhVqXR7O5wxbgwNCN8HxlZyEUFFhdoCIwl JoZvLqbHtu0cntyxJBU8S18lXK8qp8xXCheIDOYMVAl6YWPa5TwiK1zEyF8q8vxqHWwJ YoLeRA8+fV1A1y7fv+Nv9FNd/2eWAo4WdTQhWTEcVkvSH5R0WPelVes2y2Z96Bhs6jdi 4wkGetarbny2mdci+FhgBMQZG+uxvu3XCu3rA/Lc/u7nAfWacjGhJ0if/3a4a8qLnUpa VMeg== X-Gm-Message-State: AOAM5307Edx2146ZFYxIM1VQ3SmxHGwzRH//0uufJUIW1doGTtcDMgSz uH2eYomgKw8Nk/7oqIJRgXSQHAqHXx0= X-Google-Smtp-Source: ABdhPJwpp4ce1erqhc+LH9wQVIVq+8MKLOakJYLuSK6UtGkjEoIybHyfbng9Xep+7C4dM38aH6x2fA== X-Received: by 2002:a1c:b002:: with SMTP id z2mr866502wme.121.1618261715369; Mon, 12 Apr 2021 14:08:35 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id s10sm46889wrt.23.2021.04.12.14.08.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Apr 2021 14:08:35 -0700 (PDT) Message-Id: <1f3af8a886e5015a70a880dabd71f791d63c88df.1618261698.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 12 Apr 2021 21:08:17 +0000 Subject: [PATCH v3 26/26] name-hash: use expand_to_path() Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: newren@gmail.com, gitster@pobox.com, Derrick Stolee , Matheus Tavares Bernardino , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee A sparse-index loads the name-hash data for its entries, including the sparse-directory entries. If a caller asks for a path that is contained within a sparse-directory entry, we need to expand to a full index and recalculate the name hash table before returning the result. Insert calls to expand_to_path() to protect against this case. Signed-off-by: Derrick Stolee --- name-hash.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/name-hash.c b/name-hash.c index d08deaa2c9e7..383cf589969c 100644 --- a/name-hash.c +++ b/name-hash.c @@ -8,6 +8,7 @@ #include "cache.h" #include "thread-utils.h" #include "trace2.h" +#include "sparse-index.h" struct dir_entry { struct hashmap_entry ent; @@ -683,6 +684,7 @@ int index_dir_exists(struct index_state *istate, const char *name, int namelen) struct dir_entry *dir; lazy_init_name_hash(istate); + expand_to_path(istate, name, namelen, 0); dir = find_dir_entry(istate, name, namelen); return dir && dir->nr; } @@ -693,6 +695,7 @@ void adjust_dirname_case(struct index_state *istate, char *name) const char *ptr = startPtr; lazy_init_name_hash(istate); + expand_to_path(istate, name, strlen(name), 0); while (*ptr) { while (*ptr && *ptr != '/') ptr++; @@ -716,6 +719,7 @@ struct cache_entry *index_file_exists(struct index_state *istate, const char *na unsigned int hash = memihash(name, namelen); lazy_init_name_hash(istate); + expand_to_path(istate, name, namelen, icase); ce = hashmap_get_entry_from_hash(&istate->name_hash, hash, NULL, struct cache_entry, ent);