From patchwork Tue Feb 2 03:01:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12060485 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3959EC433E6 for ; Tue, 2 Feb 2021 03:02:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id ECFB664E9A for ; Tue, 2 Feb 2021 03:02:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230281AbhBBDC1 (ORCPT ); Mon, 1 Feb 2021 22:02:27 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49376 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230178AbhBBDCY (ORCPT ); Mon, 1 Feb 2021 22:02:24 -0500 Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [IPv6:2a00:1450:4864:20::32e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B816DC06178A for ; Mon, 1 Feb 2021 19:01:27 -0800 (PST) Received: by mail-wm1-x32e.google.com with SMTP id y187so1007716wmd.3 for ; Mon, 01 Feb 2021 19:01:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=juCAe5atgoTFygs1wNmd/FGDKpeIjGfc51PxxQjB/ho=; b=TQ8lES/4YqltBEiemAyM3zSyLPdbQ5yIBjU2P+kjZtFeIzOzEVxrYKWIEEA35lUlSN 4p3UbB1t5ovYjkvgpoFMpklQZ+kmxLjRregsLGrDtC+OmECtAB0jhLC9kkvGTOtYiBt5 icizNNTwLtAJDWtvQxUYWMSdkdruBZiyB7W9hg6lLu7RDRUSwqXtooKJaBgaNBvCvTKF zcPuElXgF6cKkWyzZcO9238/LRh4q9T13DObVXzxG+RUlUkRnSXqCi7z/y8fe3HfKhQn uiSxkg6DBTG6ZDXuWDPdsKWJX3r/livzAzZzzoI9YByCdisoCylw4TwCrjoSeBNasbMe +QWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=juCAe5atgoTFygs1wNmd/FGDKpeIjGfc51PxxQjB/ho=; b=nRf25o77J0ug82raICsCm18BPv7g41zXWrthkbm7vRnVFXLqTluY4ygH+njXmPJ6aJ bwU9RA+ccFuNqdTCBTVLfGeII5IcbvrMCqVqCJSBEuH2XagLSDVdj/EBcX6+EWNGO0Ad 15p4vTkwhmBFqfSQkfO88mfWOgOOmXoZ4BhkGezydMeneerDgZcE4y11EKxbruC4n5pW brStJRckryb4D/0KSc+5o1SRkPJ3oruoI61ZpN6u3TirLaCVVFyWTcxU9JzhsBHR/lsg neEuB73llVkhBAHIfjY+LQ5UftV8WP77ltHDVv0f9O+wMJqrg/lm3DXlnLFVUfo9baFN aqzA== X-Gm-Message-State: AOAM531lWft7dA3r7J/XOxXXVPH4LmTB/sZLceAJIRoASslWAlMFNYzZ S0y0BFKG3rhoVeJ7DnzbHKout8xJ6ZI= X-Google-Smtp-Source: ABdhPJyabaQDYmGSww9efXrXawOwuIZtB50D/ikojO/N/o+bRREx+/aeJ+xfNNmcSFXuu3w2jHoW1A== X-Received: by 2002:a1c:8109:: with SMTP id c9mr1468092wmd.137.1612234885875; Mon, 01 Feb 2021 19:01:25 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id m24sm68478wmi.24.2021.02.01.19.01.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Feb 2021 19:01:25 -0800 (PST) Message-Id: <9c605c99f66cae3c66bcdd4cbeefbfa1ec2be192.1612234883.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 02 Feb 2021 03:01:18 +0000 Subject: [PATCH v2 1/6] commit-graph: use repo_parse_commit Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: me@ttaylorr.com, peff@peff.net, gitster@pobox.com, abhishekkumar8222@gmail.com, Derrick Stolee , Taylor Blau , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee The write_commit_graph_context has a repository pointer, so use it. Signed-off-by: Derrick Stolee --- commit-graph.c | 10 +++++----- commit.h | 5 +++-- 2 files changed, 8 insertions(+), 7 deletions(-) diff --git a/commit-graph.c b/commit-graph.c index f3bde2ad95a..03e5a987968 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -1098,7 +1098,7 @@ static int write_graph_chunk_data(struct hashfile *f, uint32_t packedDate[2]; display_progress(ctx->progress, ++ctx->progress_cnt); - if (parse_commit_no_graph(*list)) + if (repo_parse_commit_no_graph(ctx->r, *list)) die(_("unable to parse commit %s"), oid_to_hex(&(*list)->object.oid)); tree = get_commit_tree_oid(*list); @@ -1411,11 +1411,11 @@ static void close_reachable(struct write_commit_graph_context *ctx) if (!commit) continue; if (ctx->split) { - if ((!parse_commit(commit) && + if ((!repo_parse_commit(ctx->r, commit) && commit_graph_position(commit) == COMMIT_NOT_FROM_GRAPH) || flags == COMMIT_GRAPH_SPLIT_REPLACE) add_missing_parents(ctx, commit); - } else if (!parse_commit_no_graph(commit)) + } else if (!repo_parse_commit_no_graph(ctx->r, commit)) add_missing_parents(ctx, commit); } stop_progress(&ctx->progress); @@ -1710,9 +1710,9 @@ static void copy_oids_to_commits(struct write_commit_graph_context *ctx) continue; if (ctx->split && flags == COMMIT_GRAPH_SPLIT_REPLACE) - parse_commit(ctx->commits.list[ctx->commits.nr]); + repo_parse_commit(ctx->r, ctx->commits.list[ctx->commits.nr]); else - parse_commit_no_graph(ctx->commits.list[ctx->commits.nr]); + repo_parse_commit_no_graph(ctx->r, ctx->commits.list[ctx->commits.nr]); num_parents = commit_list_count(ctx->commits.list[ctx->commits.nr]->parents); if (num_parents > 2) diff --git a/commit.h b/commit.h index 251d877fcf6..b05ab558ce2 100644 --- a/commit.h +++ b/commit.h @@ -89,9 +89,10 @@ static inline int repo_parse_commit(struct repository *r, struct commit *item) return repo_parse_commit_gently(r, item, 0); } -static inline int parse_commit_no_graph(struct commit *commit) +static inline int repo_parse_commit_no_graph(struct repository *r, + struct commit *commit) { - return repo_parse_commit_internal(the_repository, commit, 0, 0); + return repo_parse_commit_internal(r, commit, 0, 0); } #ifndef NO_THE_REPOSITORY_COMPATIBILITY_MACROS From patchwork Tue Feb 2 03:01:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12060491 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D2566C433DB for ; Tue, 2 Feb 2021 03:02:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A4D1B64E9C for ; Tue, 2 Feb 2021 03:02:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231209AbhBBDCp (ORCPT ); Mon, 1 Feb 2021 22:02:45 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49426 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230470AbhBBDCh (ORCPT ); Mon, 1 Feb 2021 22:02:37 -0500 Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3178FC06178C for ; Mon, 1 Feb 2021 19:01:28 -0800 (PST) Received: by mail-wr1-x431.google.com with SMTP id 7so18841371wrz.0 for ; Mon, 01 Feb 2021 19:01:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=iNawx3rMk7XaAthMregzHDm9XD6OPZ1/mU6+xcSKWlk=; b=qjpR05p5r0XTldJHiex7wGBWWEoTsrU5bzLwLuG9t3G2UgjiefhhHUliy0mzjxZ2QO gympNTaJVHRlNRM1jPIPlXwTkJqkK9vRve73FO89Lls+xczENl5W1xPOL/qoYnGggNZE ns9dKRyVwsW9ygicXTPwOJzB8Ez4NqbL1M8+9eNvUZVqsbP9W8aigtj7VlDeMKj9OU28 yyqLPWae7DXA3olEKGIjnMcxWZ9pVnrlJyzdPAlHfAeXTgvXK7/zP35uPBGT2xPB4HTP yAKUJjHJMLZeNWr66guVm2VlXWyFJ+4eAg30cbeqmzm7enzWqIYKN/44GXW03+WB8Pzz NB9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=iNawx3rMk7XaAthMregzHDm9XD6OPZ1/mU6+xcSKWlk=; b=KY4KgSIIjD7Aa9fdsADbyq4cZLVqLFk8rxaPqHDGgST9i4ZfK1DTUr4Eu+stjI4BRG FewZhYl9k/QyHW9tPS/x1uYEegqE7TWHq5KV7AoLizSEbApMQflfF20S33rOhkcuxa9X swOU248qVkNmH9naKEpcOPwtAn7GtdEii1WRMJ26puc9qMIaQ/bmWxNLr5FT08sM2Q6i uMszxc6H2uoE/3xryNBUGD4tfzzG1G8S4rqfTlU6+d+iV871j10SCJanJx0eiswfZogW jG2fPb09Z/N6hK964THrRWxePlsepdfosRtrmtPB/aUK0wE5Bd4ZvwOyStCpkjmm9ktH guPA== X-Gm-Message-State: AOAM530bfEYsh6z74/cgsbYmNJCWzPYXgHMKYLHelPNqhcBn0JEPaNqY I7etpiz6A3c6eu5hVmx/Xir480KrvfU= X-Google-Smtp-Source: ABdhPJyCe6Z+st1VratSXjjh4vzNGNjmf9vCipRpbR/g2/HIJwsyEzS/xD6dELAkeG6EI/xoGbbQpA== X-Received: by 2002:adf:ba49:: with SMTP id t9mr21324963wrg.183.1612234886643; Mon, 01 Feb 2021 19:01:26 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id q9sm1226124wme.18.2021.02.01.19.01.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Feb 2021 19:01:26 -0800 (PST) Message-Id: <454b183b9ba502da7f40dc36aaa95cc3d12b5c2f.1612234883.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 02 Feb 2021 03:01:19 +0000 Subject: [PATCH v2 2/6] commit-graph: always parse before commit_graph_data_at() Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: me@ttaylorr.com, peff@peff.net, gitster@pobox.com, abhishekkumar8222@gmail.com, Derrick Stolee , Taylor Blau , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee There is a subtle failure happening when computing corrected commit dates with --split enabled. It requires a base layer needing the generation_data_overflow chunk. Then, the next layer on top erroneously thinks it needs an overflow chunk due to a bug leading to recalculating all reachable generation numbers. The output of the failure is BUG: commit-graph.c:1912: expected to write 8 bytes to chunk 47444f56, but wrote 0 instead These "expected" 8 bytes are due to re-computing the corrected commit date for the lower layer but the new layer does not need any overflow. Add a test to t5318-commit-graph.sh that demonstrates this bug. However, it does not trigger consistently with the existing code. The generation number data is stored in a slab and accessed by commit_graph_data_at(). This data is initialized when parsing a commit, but is otherwise used assuming it has been populated. The loop in compute_generation_numbers() did not enforce that all reachable commits were parsed and had correct values. This could lead to some problems when writing a commit-graph with corrected commit dates based on a commit-graph without them. It has been difficult to identify the issue here because it was so hard to reproduce. It relies on this uninitialized data having a non-zero value, but also on specifically in a way that overwrites the existing data. This patch adds the extra parse to ensure the data is filled before we compute the generation number of a commit. This triggers the new test to fail because the generation number overflow count does not match between this computation and the write for that chunk. The actual fix will follow as the next few changes. Signed-off-by: Derrick Stolee --- commit-graph.c | 16 ++++++++++++---- t/t5318-commit-graph.sh | 21 +++++++++++++++++++++ 2 files changed, 33 insertions(+), 4 deletions(-) diff --git a/commit-graph.c b/commit-graph.c index 03e5a987968..edbb3a0f2cc 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -1193,7 +1193,9 @@ static int write_graph_chunk_generation_data(struct hashfile *f, for (i = 0; i < ctx->commits.nr; i++) { struct commit *c = ctx->commits.list[i]; - timestamp_t offset = commit_graph_data_at(c)->generation - c->date; + timestamp_t offset; + repo_parse_commit(ctx->r, c); + offset = commit_graph_data_at(c)->generation - c->date; display_progress(ctx->progress, ++ctx->progress_cnt); if (offset > GENERATION_NUMBER_V2_OFFSET_MAX) { @@ -1444,15 +1446,20 @@ static void compute_generation_numbers(struct write_commit_graph_context *ctx) _("Computing commit graph generation numbers"), ctx->commits.nr); for (i = 0; i < ctx->commits.nr; i++) { - uint32_t level = *topo_level_slab_at(ctx->topo_levels, ctx->commits.list[i]); - timestamp_t corrected_commit_date = commit_graph_data_at(ctx->commits.list[i])->generation; + struct commit *c = ctx->commits.list[i]; + uint32_t level; + timestamp_t corrected_commit_date; + + repo_parse_commit(ctx->r, c); + level = *topo_level_slab_at(ctx->topo_levels, c); + corrected_commit_date = commit_graph_data_at(c)->generation; display_progress(ctx->progress, i + 1); if (level != GENERATION_NUMBER_ZERO && corrected_commit_date != GENERATION_NUMBER_ZERO) continue; - commit_list_insert(ctx->commits.list[i], &list); + commit_list_insert(c, &list); while (list) { struct commit *current = list->item; struct commit_list *parent; @@ -1461,6 +1468,7 @@ static void compute_generation_numbers(struct write_commit_graph_context *ctx) timestamp_t max_corrected_commit_date = 0; for (parent = current->parents; parent; parent = parent->next) { + repo_parse_commit(ctx->r, parent->item); level = *topo_level_slab_at(ctx->topo_levels, parent->item); corrected_commit_date = commit_graph_data_at(parent->item)->generation; diff --git a/t/t5318-commit-graph.sh b/t/t5318-commit-graph.sh index fa27df579a5..2cf29f425a0 100755 --- a/t/t5318-commit-graph.sh +++ b/t/t5318-commit-graph.sh @@ -446,6 +446,27 @@ test_expect_success 'warn on improper hash version' ' ) ' +test_expect_failure 'lower layers have overflow chunk' ' + cd "$TRASH_DIRECTORY/full" && + UNIX_EPOCH_ZERO="@0 +0000" && + FUTURE_DATE="@2147483646 +0000" && + rm -f .git/objects/info/commit-graph && + test_commit --date "$FUTURE_DATE" future-1 && + test_commit --date "$UNIX_EPOCH_ZERO" old-1 && + git commit-graph write --reachable && + test_commit --date "$FUTURE_DATE" future-2 && + test_commit --date "$UNIX_EPOCH_ZERO" old-2 && + git commit-graph write --reachable --split=no-merge && + test_commit extra && + git commit-graph write --reachable --split=no-merge && + git commit-graph write --reachable && + graph_read_expect 16 "generation_data generation_data_overflow extra_edges" && + mv .git/objects/info/commit-graph commit-graph-upgraded && + git commit-graph write --reachable && + graph_read_expect 16 "generation_data generation_data_overflow extra_edges" && + test_cmp .git/objects/info/commit-graph commit-graph-upgraded +' + # the verify tests below expect the commit-graph to contain # exactly the commits reachable from the commits/8 branch. # If the file changes the set of commits in the list, then the From patchwork Tue Feb 2 03:01:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12060489 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 34430C433E6 for ; Tue, 2 Feb 2021 03:02:50 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id F012664DA5 for ; Tue, 2 Feb 2021 03:02:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231167AbhBBDCl (ORCPT ); Mon, 1 Feb 2021 22:02:41 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49428 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230430AbhBBDCh (ORCPT ); Mon, 1 Feb 2021 22:02:37 -0500 Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com [IPv6:2a00:1450:4864:20::336]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0E698C061793 for ; Mon, 1 Feb 2021 19:01:29 -0800 (PST) Received: by mail-wm1-x336.google.com with SMTP id 190so1072297wmz.0 for ; Mon, 01 Feb 2021 19:01:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=oBHpbe8SMwzLDpj1QYTBv7xfKLQQS7mGrfV1ZkEAEgA=; b=EY5pdShRod4U+lvvt5yXBhxvj98MDDf+IWJOe9pxvskiZzSlJu0uh4IZX8LkLIYkP0 71ttB5VfT1w5i9ByCQMLchNAV20KDk0dE3tfL5BGiLLmBYBKwmGN8Uj824eeSZm8Onzo 1Quuq3dpygAJ/kt6nJAfpblRfH2/QgzqRUeh2sn6WZPScBZKekd1nsQmclLtOC7tmAxa KMc62o93v6pAW6w0OZ3Osg2HSYxpnMY0IWYavyj+ZyOJCWJKJOGMGe2KbnRqqRhHjf0o IxMBwXZYsc53MCeJ7Tg2ocAbasUKLNrWIFCKSiZH7480reJWwmqyDs0a1/fnSK3OMSs1 6G4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=oBHpbe8SMwzLDpj1QYTBv7xfKLQQS7mGrfV1ZkEAEgA=; b=EFoxoTRHPfrOgNtgXsN0QC4SeyXnZzaVr785wy5lWJZ92b99ZUx8ooFgsdwOZcDj09 I92phwBsyy+PL2dizbwh0r83G7R6/fw0sYKZagrWhwTggiwsvcWX8R/ca3vaeBfwoyay 1qybZRv5yJxN8q0YG6wheZ6WQxhegIHC4DFY5RUYZ1RgN46EGNAbAT+5/iSxQ9UJPBnL 9QyxOrhc25nFEEbp9EhPwFLoV83D+b9JqJb2ANyLQ0uiNiF3h7REZ6U0csvlZWjUCjl+ XOjqnVfqCCFEJ0Egl47+/puWIVSEXHckGbQ98q/3hcw6Voutuf5eEzC5l5H3Arx5MdAk a1ew== X-Gm-Message-State: AOAM533jVKus/b5yhFOMPiAOUEzlMUVIC2ynEJHGEoM1Y9QnYQ1CG2Mn WFrt3sd3wva0OTeSdldSpohNSjHVLdU= X-Google-Smtp-Source: ABdhPJxRPgYN3D2pLVbRbCkwv3DwJrZ05vnPpmT9fRXDs4JCFhGWnMEt7ouyCls+R1bBP9A9hC6iEw== X-Received: by 2002:a1c:545d:: with SMTP id p29mr474603wmi.77.1612234887651; Mon, 01 Feb 2021 19:01:27 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id o18sm1119942wmh.20.2021.02.01.19.01.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Feb 2021 19:01:27 -0800 (PST) Message-Id: <3d223fa2156530f67997be8fa7e254e599f3b13a.1612234883.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 02 Feb 2021 03:01:20 +0000 Subject: [PATCH v2 3/6] commit-graph: validate layers for generation data Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: me@ttaylorr.com, peff@peff.net, gitster@pobox.com, abhishekkumar8222@gmail.com, Derrick Stolee , Taylor Blau , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee We need to be extra careful that we don't use corrected commit dates from any layer of a commit-graph chain if there is a single commit-graph file that is missing the generation_data chunk. Update validate_mixed_generation_chain() to correctly update each layer to ignore the generation_data chunk in this case. It now also returns 1 if all layers have a generation_data chunk. This return value will be used in the next change. Signed-off-by: Derrick Stolee --- commit-graph.c | 22 ++++++++++++++++------ 1 file changed, 16 insertions(+), 6 deletions(-) diff --git a/commit-graph.c b/commit-graph.c index edbb3a0f2cc..b3f7c3bbcb3 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -614,19 +614,29 @@ static struct commit_graph *load_commit_graph_chain(struct repository *r, return graph_chain; } -static void validate_mixed_generation_chain(struct commit_graph *g) +/* + * returns 1 if and only if all graphs in the chain have + * corrected commit dates stored in the generation_data chunk. + */ +static int validate_mixed_generation_chain(struct commit_graph *g) { - int read_generation_data; + int read_generation_data = 1; + struct commit_graph *p = g; - if (!g) - return; + while (read_generation_data && p) { + read_generation_data = p->read_generation_data; + p = p->base_graph; + } - read_generation_data = !!g->chunk_generation_data; + if (read_generation_data) + return 1; while (g) { - g->read_generation_data = read_generation_data; + g->read_generation_data = 0; g = g->base_graph; } + + return 0; } struct commit_graph *read_commit_graph_one(struct repository *r, From patchwork Tue Feb 2 03:01:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12060493 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C2EA7C433E0 for ; Tue, 2 Feb 2021 03:02:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8F58064E9C for ; Tue, 2 Feb 2021 03:02:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231211AbhBBDC4 (ORCPT ); Mon, 1 Feb 2021 22:02:56 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49434 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230380AbhBBDCh (ORCPT ); Mon, 1 Feb 2021 22:02:37 -0500 Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0EA55C061794 for ; Mon, 1 Feb 2021 19:01:30 -0800 (PST) Received: by mail-wr1-x435.google.com with SMTP id l12so18834936wry.2 for ; Mon, 01 Feb 2021 19:01:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=0bnFoX5kshYd6yQsZTa9+a7wKtLUiSkJzedXbC9O7CM=; b=sdHrd9Q8iUntZ05IZmaKNbF3vllAKlUBcW/gUsXz4AT09OVWGDp4HCN2Oy++ZZNF0+ e5eLzm3xv2kqvNHx6fRylsZkkc9nsAvlLtl2QT5htEe7Zy14fPQ5ZHkDkAvRytxHDxJg Kif2AQBhHBA/xehqZ2kCUvPBRFeKkuhT+wRrRwpPH2jLb1ZfQtASm0I0dEPGthWQWxk2 90jJ0sgnlQaFXBmpk+JB4IlTBHwFvCswtanYLwIHpBgOZR905uBt+AujpZb4Q3ZKuFKA yXuudY9XPir1nbJWkVJ2vfPonLoQmsGg64UmL/o7sbCBejI4PO8ZtTZLdZfgDhahW9SB Oqdw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=0bnFoX5kshYd6yQsZTa9+a7wKtLUiSkJzedXbC9O7CM=; b=jLWj6FBDqJO/iRXIaasIs159HeoV18pGv0JQxlA2hfmNQUHW1xmBmdsk2qhw4nT9mT zmIvUDeviyDVHbxvtgscLvsHmHQMO/Go+vdtOkCoG/83HgBqLhvT0s0R+u0zmzED38Ls pXBK3OXgDHfUIRVpUY6kH7HKIF437Ge7+Kn/WvYG0G1sAv98sYMyk1XVBLOOf0yiVHSj XQLBbwMLb1e69bFoILZFGG25TH1BilNEflwRxoPRBr6Qyatht9aHu4RGoGFtB9cb40Wk DPYKsx9oAVUK2qjgNHMqZub8LM2gUzG2lKc4iExCto7RbcNz1HjS1KPm4o2FEnVkOYsY gWKw== X-Gm-Message-State: AOAM532vOcnAkrzlW5C5PNjZQmEI6dR6iCs9Bw7xV9naO29KymFPzYfa n33gTUP8Hs7Ve/ghrKrVduVHMyK51H0= X-Google-Smtp-Source: ABdhPJxKs2Jrq5tEcyl4DaiEaILzIDkb1AufxwSGdDpInZKbGyTR7l96aC1Nb5+UaK4JZbPD+rX7Ag== X-Received: by 2002:a5d:690e:: with SMTP id t14mr20956762wru.64.1612234888567; Mon, 01 Feb 2021 19:01:28 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id y14sm28668401wro.58.2021.02.01.19.01.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Feb 2021 19:01:28 -0800 (PST) Message-Id: <05248ff222fe7f2a7976e7a7cf2ee272eec6edbb.1612234883.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 02 Feb 2021 03:01:21 +0000 Subject: [PATCH v2 4/6] commit-graph: compute generations separately Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: me@ttaylorr.com, peff@peff.net, gitster@pobox.com, abhishekkumar8222@gmail.com, Derrick Stolee , Taylor Blau , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee The compute_generation_numbers() method was introduced by 3258c663 (commit-graph: compute generation numbers, 2018-05-01) to compute what is now known as "topological levels". These are still stored in the commit-graph file for compatibility sake while c1a09119 (commit-graph: implement corrected commit date, 2021-01-16) updated the method to also compute the new version of generation numbers: corrected commit date. It makes sense why these are grouped. They perform very similar walks of the necessary commits and compute similar maximums over each parent. However, having these two together conflates them in subtle ways that is hard to separate. In particular, the topo_level slab is used to store the topological levels in all cases, but the commit_graph_data_at(c)->generation member stores different values depending on the state of the existing commit-graph file. * If the existing commit-graph file has a "GDAT" chunk, then these values represent corrected commit dates. * If the existing commit-graph file doesn't have a "GDAT" chunk, then these values are actually the topological levels. This issue only occurs only when upgrading an existing commit-graph file into one that has the "GDAT" chunk. The current change does not resolve this upgrade problem, but splitting the implementation into two pieces here helps with that process, which will follow in the next change. The important thing this helps with is the case where the num_generation_data_overflows was being incremented incorrectly, triggering a write of the overflow chunk. Signed-off-by: Derrick Stolee --- commit-graph.c | 70 ++++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 56 insertions(+), 14 deletions(-) diff --git a/commit-graph.c b/commit-graph.c index b3f7c3bbcb3..2790f70d113 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -1446,27 +1446,24 @@ static void close_reachable(struct write_commit_graph_context *ctx) stop_progress(&ctx->progress); } -static void compute_generation_numbers(struct write_commit_graph_context *ctx) +static void compute_topological_levels(struct write_commit_graph_context *ctx) { int i; struct commit_list *list = NULL; if (ctx->report_progress) ctx->progress = start_delayed_progress( - _("Computing commit graph generation numbers"), + _("Computing commit graph topological levels"), ctx->commits.nr); for (i = 0; i < ctx->commits.nr; i++) { struct commit *c = ctx->commits.list[i]; uint32_t level; - timestamp_t corrected_commit_date; repo_parse_commit(ctx->r, c); level = *topo_level_slab_at(ctx->topo_levels, c); - corrected_commit_date = commit_graph_data_at(c)->generation; display_progress(ctx->progress, i + 1); - if (level != GENERATION_NUMBER_ZERO && - corrected_commit_date != GENERATION_NUMBER_ZERO) + if (level != GENERATION_NUMBER_ZERO) continue; commit_list_insert(c, &list); @@ -1475,15 +1472,12 @@ static void compute_generation_numbers(struct write_commit_graph_context *ctx) struct commit_list *parent; int all_parents_computed = 1; uint32_t max_level = 0; - timestamp_t max_corrected_commit_date = 0; for (parent = current->parents; parent; parent = parent->next) { repo_parse_commit(ctx->r, parent->item); level = *topo_level_slab_at(ctx->topo_levels, parent->item); - corrected_commit_date = commit_graph_data_at(parent->item)->generation; - if (level == GENERATION_NUMBER_ZERO || - corrected_commit_date == GENERATION_NUMBER_ZERO) { + if (level == GENERATION_NUMBER_ZERO) { all_parents_computed = 0; commit_list_insert(parent->item, &list); break; @@ -1491,9 +1485,6 @@ static void compute_generation_numbers(struct write_commit_graph_context *ctx) if (level > max_level) max_level = level; - - if (corrected_commit_date > max_corrected_commit_date) - max_corrected_commit_date = corrected_commit_date; } if (all_parents_computed) { @@ -1502,6 +1493,55 @@ static void compute_generation_numbers(struct write_commit_graph_context *ctx) if (max_level > GENERATION_NUMBER_V1_MAX - 1) max_level = GENERATION_NUMBER_V1_MAX - 1; *topo_level_slab_at(ctx->topo_levels, current) = max_level + 1; + } + } + } + stop_progress(&ctx->progress); +} + +static void compute_generation_numbers(struct write_commit_graph_context *ctx) +{ + int i; + struct commit_list *list = NULL; + + if (ctx->report_progress) + ctx->progress = start_delayed_progress( + _("Computing commit graph generation numbers"), + ctx->commits.nr); + for (i = 0; i < ctx->commits.nr; i++) { + struct commit *c = ctx->commits.list[i]; + timestamp_t corrected_commit_date; + + repo_parse_commit(ctx->r, c); + corrected_commit_date = commit_graph_data_at(c)->generation; + + display_progress(ctx->progress, i + 1); + if (corrected_commit_date != GENERATION_NUMBER_ZERO) + continue; + + commit_list_insert(c, &list); + while (list) { + struct commit *current = list->item; + struct commit_list *parent; + int all_parents_computed = 1; + timestamp_t max_corrected_commit_date = 0; + + for (parent = current->parents; parent; parent = parent->next) { + repo_parse_commit(ctx->r, parent->item); + corrected_commit_date = commit_graph_data_at(parent->item)->generation; + + if (corrected_commit_date == GENERATION_NUMBER_ZERO) { + all_parents_computed = 0; + commit_list_insert(parent->item, &list); + break; + } + + if (corrected_commit_date > max_corrected_commit_date) + max_corrected_commit_date = corrected_commit_date; + } + + if (all_parents_computed) { + pop_commit(&list); if (current->date && current->date > max_corrected_commit_date) max_corrected_commit_date = current->date - 1; @@ -2401,7 +2441,9 @@ int write_commit_graph(struct object_directory *odb, validate_mixed_generation_chain(ctx->r->objects->commit_graph); - compute_generation_numbers(ctx); + compute_topological_levels(ctx); + if (ctx->write_generation_data) + compute_generation_numbers(ctx); if (ctx->changed_paths) compute_bloom_filters(ctx); From patchwork Tue Feb 2 03:01:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12060497 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4BA0CC433DB for ; Tue, 2 Feb 2021 03:03:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1B6AD64E9C for ; Tue, 2 Feb 2021 03:03:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231201AbhBBDDG (ORCPT ); Mon, 1 Feb 2021 22:03:06 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49436 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231132AbhBBDCh (ORCPT ); Mon, 1 Feb 2021 22:02:37 -0500 Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com [IPv6:2a00:1450:4864:20::32a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1F4ADC061797 for ; Mon, 1 Feb 2021 19:01:31 -0800 (PST) Received: by mail-wm1-x32a.google.com with SMTP id u14so1047635wml.4 for ; Mon, 01 Feb 2021 19:01:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=YdGHpYEjlWjy/M9oqm/ijQbzattP/S90vUHrg7e3h0M=; b=B/CAeGEEUWIxX5apsRiXJY4006qT51ZyXgWJuierWVGmAeFR6xF73IhfgIKZdqBcT/ zaGa0jcOviA1UiJUz3OCdnSKkRubpo27Mw813Am7Hbbr2zB1DUKqLwxbrsBdIDWWnnTm HYIfnw5+cqyR5wsyO3MV9a5GlUe+xC8TXJtv6ih3XHln+Y1Z1KcwHwSGtTe0yvI2sijp 8wYmfwl9oKRRc9us1UMHC5DtyvcvUjB1DnusuhVoTGK2v4lTpFaFY6YaxGG4F0snYa5x b51eO8z4tw2Ior6Fne4gNwp/dmx/r03sB1xaghUFAr8XmgSa+v8DWxIqKuNOfMg9BE2D AYCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=YdGHpYEjlWjy/M9oqm/ijQbzattP/S90vUHrg7e3h0M=; b=toSh31cf1H2cVsxaPgyTfcPn8x1zYh9ow+KQX02vDIrO1l2MatnitwdY4yQ9GPKscB vWVghlmLGXWSvPSBh4WmrablibpSEfC9hEs4Jw1rrBEIgtqR7iMec3oSKv5rmRXLPpZC YUvBVo+NtGT3ZZJoTszFeeRgXOW6XKzk1x2VDB3xze30WDW1X9vUjQvjFif9oLBvhRGE +V5SqZCnOwTN9ZNkAs+C2s/sgX9NzZwFHYsLPrVQYAvbD1OxQO557z1PETKUQQcEW7GZ 5z9foTgFZDMVptFye6SZ+JL6vtxAmIqo8NS47lcQgEqxIj88MkmN4wOIthe9iMn/yWn4 bQvg== X-Gm-Message-State: AOAM530OlXuGIN2onEwH6RZOffMxDwba+sKKmzH2kMNTar2uFuqNP5um O9mtC1nee1UgZId8yiMkcaskoWOU4LI= X-Google-Smtp-Source: ABdhPJxURW8gb77bPlSKAU5eG5zB3LE6y4Ow2sww4KUqiPJphMtZfo+lgXWvlZMx1ctVqp7mQMeszg== X-Received: by 2002:a1c:a90f:: with SMTP id s15mr1535271wme.154.1612234889625; Mon, 01 Feb 2021 19:01:29 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id m18sm28609884wrx.17.2021.02.01.19.01.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Feb 2021 19:01:29 -0800 (PST) Message-Id: <9bccee8fb6381102d543d6b28e90959307ed08eb.1612234883.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 02 Feb 2021 03:01:22 +0000 Subject: [PATCH v2 5/6] commit-graph: be extra careful about mixed generations Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: me@ttaylorr.com, peff@peff.net, gitster@pobox.com, abhishekkumar8222@gmail.com, Derrick Stolee , Taylor Blau , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee When upgrading to a commit-graph with corrected commit dates from one without, there are a few things that need to be considered. When computing generation numbers for the new commit-graph file that expects to add the generation_data chunk with corrected commit dates, we need to ensure that the 'generation' member of the commit_graph_data struct is set to zero for these commits. Unfortunately, the fallback to use topological level for generation number when corrected commit dates are not available are causing us harm here: parsing commits notices that read_generation_data is false and populates 'generation' with the topological level. The solution is to iterate through the commits, parse the commits to populate initial values, then reset the generation values to zero to trigger recalculation. This loop only occurs when the existing commit-graph data has no corrected commit dates. While this improves our situation somewhat, we have not completely solved the issue for correctly computing generation numbers for mixed layers. That follows in the next change. Signed-off-by: Derrick Stolee --- commit-graph.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/commit-graph.c b/commit-graph.c index 2790f70d113..ee8d5a0cdb4 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -1036,7 +1036,8 @@ struct write_commit_graph_context { split:1, changed_paths:1, order_by_pack:1, - write_generation_data:1; + write_generation_data:1, + trust_generation_numbers:1; struct topo_level_slab *topo_levels; const struct commit_graph_opts *opts; @@ -1508,6 +1509,15 @@ static void compute_generation_numbers(struct write_commit_graph_context *ctx) ctx->progress = start_delayed_progress( _("Computing commit graph generation numbers"), ctx->commits.nr); + + if (!ctx->trust_generation_numbers) { + for (i = 0; i < ctx->commits.nr; i++) { + struct commit *c = ctx->commits.list[i]; + repo_parse_commit(ctx->r, c); + commit_graph_data_at(c)->generation = GENERATION_NUMBER_ZERO; + } + } + for (i = 0; i < ctx->commits.nr; i++) { struct commit *c = ctx->commits.list[i]; timestamp_t corrected_commit_date; @@ -2439,7 +2449,7 @@ int write_commit_graph(struct object_directory *odb, } else ctx->num_commit_graphs_after = 1; - validate_mixed_generation_chain(ctx->r->objects->commit_graph); + ctx->trust_generation_numbers = validate_mixed_generation_chain(ctx->r->objects->commit_graph); compute_topological_levels(ctx); if (ctx->write_generation_data) From patchwork Tue Feb 2 03:01:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12060495 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 18001C433E6 for ; Tue, 2 Feb 2021 03:03:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CA58164E9C for ; Tue, 2 Feb 2021 03:03:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231225AbhBBDDB (ORCPT ); Mon, 1 Feb 2021 22:03:01 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49438 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231144AbhBBDCi (ORCPT ); Mon, 1 Feb 2021 22:02:38 -0500 Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [IPv6:2a00:1450:4864:20::32e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2CB2AC0617A7 for ; Mon, 1 Feb 2021 19:01:32 -0800 (PST) Received: by mail-wm1-x32e.google.com with SMTP id 190so1072372wmz.0 for ; Mon, 01 Feb 2021 19:01:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=9X82xTzD0DtAhscl1U1xIpgUzeNPjHgAUzvkhzk+efk=; b=PtTWF+fTGo1685zTiTvZPzoV8Vqb6HaANil+fjUn9khSA1sITtfCneeGfXkrjiQ5ag FpIjBAQWpna2XS/RA49m9SO0akM5u9EZjAgCPZ+wbNwJAn/gzj36R6jLhmEJiXe0qZgp BRR8DsIz63EcxJ0PqpIzO6dGyNFq3auh2UwgTqAvdJsvrCEz92iBP7HRJU2TmfrZiL6F X7OKfuDPjlYZvPXGmdB2xHKA+iHtIgheR3j6i83hRKcYSprt/Zgm1G8WdnQqPFsrEl2q 0Bn53pl2V4Gr5szJVWWa9mqTa3y4n73bkIiDmBv4w/GuN8jM8fc4nBvLSXnMjRBJfd3w SHmQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=9X82xTzD0DtAhscl1U1xIpgUzeNPjHgAUzvkhzk+efk=; b=P7uLp5js8L4kKz3+Q+TL3dv+eN735eImMZEgnWvvL3tnIhilgpspoLoMasSfDmFprR PVFE8+PlJ21qT4OX1Qc8ZBQC/0xgJsxiQj5hjAUAnhgisLQBCxT4JudVlx3OjCmflGS8 c8evj/pnimr04BDmVLLfeWeaZA2j5XkrLIe8Vzf1E2JhBBWIrtHs/KgXLkMyHzQbnOqm gHSZWZp4ICdWsbPOGO76NLcDfAe6c5mqaa8OKnYDVE/AilxihxiO/9eh9nS2EHUV4bIl 6N1wo4nzPrVOGsGxMO1EERT7gLcbyz+ARWzhuYjYzbGT2joWH648UJZZyNoEjHtxu50D QP3A== X-Gm-Message-State: AOAM530IB7p8dEETHyskRSLDoxPXButZxr1B5R1KMLALdqbbdHY+fjQH gV7jgv9EtljHEZU+uJLtJOSYH5U6wIM= X-Google-Smtp-Source: ABdhPJxZ/MJ4FBtqCsIZTtvOPh209mjDLZinGwHPbg/eIaLd0FHaBcsdDqNuVcI29D1vKVeoEcNc/A== X-Received: by 2002:a1c:2d8a:: with SMTP id t132mr1444153wmt.119.1612234890669; Mon, 01 Feb 2021 19:01:30 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id q18sm29252036wrr.55.2021.02.01.19.01.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Feb 2021 19:01:30 -0800 (PST) Message-Id: <38086c85b529f033d243746984227faf87e363d9.1612234883.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 02 Feb 2021 03:01:23 +0000 Subject: [PATCH v2 6/6] commit-graph: prepare commit graph Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: me@ttaylorr.com, peff@peff.net, gitster@pobox.com, abhishekkumar8222@gmail.com, Derrick Stolee , Taylor Blau , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Before checking if the repository has a commit-graph loaded, be sure to run prepare_commit_graph(). This is necessary because otherwise the topo_levels slab is not initialized. As we compute topo_levels for the new commits, we iterate further into the lower layers since the first visit to each commit looks as though the topo_level is not populated. By properly initializing the topo_slab, we fix the previously broken case of a split commit graph where a base layer has the generation_data_overflow chunk. Signed-off-by: Derrick Stolee --- commit-graph.c | 10 ++-------- t/t5318-commit-graph.sh | 2 +- 2 files changed, 3 insertions(+), 9 deletions(-) diff --git a/commit-graph.c b/commit-graph.c index ee8d5a0cdb4..94d3ab4a04f 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -2348,6 +2348,7 @@ int write_commit_graph(struct object_directory *odb, init_topo_level_slab(&topo_levels); ctx->topo_levels = &topo_levels; + prepare_commit_graph(ctx->r); if (ctx->r->objects->commit_graph) { struct commit_graph *g = ctx->r->objects->commit_graph; @@ -2361,7 +2362,6 @@ int write_commit_graph(struct object_directory *odb, ctx->changed_paths = 1; if (!(flags & COMMIT_GRAPH_NO_WRITE_BLOOM_FILTERS)) { struct commit_graph *g; - prepare_commit_graph_one(ctx->r, ctx->odb); g = ctx->r->objects->commit_graph; @@ -2373,10 +2373,7 @@ int write_commit_graph(struct object_directory *odb, } if (ctx->split) { - struct commit_graph *g; - prepare_commit_graph(ctx->r); - - g = ctx->r->objects->commit_graph; + struct commit_graph *g = ctx->r->objects->commit_graph; while (g) { ctx->num_commit_graphs_before++; @@ -2400,9 +2397,6 @@ int write_commit_graph(struct object_directory *odb, ctx->approx_nr_objects = approximate_object_count(); - if (ctx->append) - prepare_commit_graph_one(ctx->r, ctx->odb); - if (ctx->append && ctx->r->objects->commit_graph) { struct commit_graph *g = ctx->r->objects->commit_graph; for (i = 0; i < g->num_commits; i++) { diff --git a/t/t5318-commit-graph.sh b/t/t5318-commit-graph.sh index 2cf29f425a0..5b15f1aa0f6 100755 --- a/t/t5318-commit-graph.sh +++ b/t/t5318-commit-graph.sh @@ -446,7 +446,7 @@ test_expect_success 'warn on improper hash version' ' ) ' -test_expect_failure 'lower layers have overflow chunk' ' +test_expect_success 'lower layers have overflow chunk' ' cd "$TRASH_DIRECTORY/full" && UNIX_EPOCH_ZERO="@0 +0000" && FUTURE_DATE="@2147483646 +0000" &&