From patchwork Wed Jun 17 09:14:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Abhishek Kumar X-Patchwork-Id: 11609479 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4ECAA912 for ; Wed, 17 Jun 2020 09:16:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 31E742082F for ; Wed, 17 Jun 2020 09:16:25 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Ul9PF3ve" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726331AbgFQJQY (ORCPT ); Wed, 17 Jun 2020 05:16:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38778 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725554AbgFQJQW (ORCPT ); Wed, 17 Jun 2020 05:16:22 -0400 Received: from mail-pg1-x533.google.com (mail-pg1-x533.google.com [IPv6:2607:f8b0:4864:20::533]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AF088C061573 for ; Wed, 17 Jun 2020 02:16:20 -0700 (PDT) Received: by mail-pg1-x533.google.com with SMTP id e9so953234pgo.9 for ; Wed, 17 Jun 2020 02:16:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Ij+Yib+f3PlB+fwRGnASqTjNUKP9NOz2qNcqC+UQMJs=; b=Ul9PF3veV969I+63Q0K/SFrL3FpMbSiLUgO5XntaBVBxMPI6Qf7PF8cT8aBOErNxeX vulPex/dDRPnaaPrr77LzRWykfiS5TZ2dEyYrm2VifEksr81Ugupux7nzjboI9x6YYRH rEeBEiSnTHJFLIxGSu62+9qG/3yN1t2pnT3NkBGevNwfDLpPfwS35X2rod5Dy4cLnd8R BC4T6PhBeKC8vjPWw00u2JzIjHOLxrKLvtgXK20m38lobHoGpRnpl5QfqY13g/ECJK2o 3ZVoUmM5oAQOmGdYVOABMHK+cZEZmNcA3uSYkZkStGv0l6GUhdqn2lKzZRBOLeETUKV/ fj/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Ij+Yib+f3PlB+fwRGnASqTjNUKP9NOz2qNcqC+UQMJs=; b=eD5BmnUfKiibi4LiJpZk3YpF3AW3ig7kVqzXLPYpsu4Rw8xZnftAlgplh6ulWIB7S3 0l2KB6BAoucHNAdXhfVMPrzuvgNq7DrQcr1Zk5sqO6Th+RVA/t29HM6gakxkCndCjeAo W5XfLaDG3gXjCD9em20FiLhyvNaQearWwQu7hi+wb1L92+zCWx1BW3NuAahvUNAcuOfT uxZlHNRFr2s8AYXDr1VMuuIjnV8bCEgwp6vcbOliF//USUnJ7OhqyNLrv48d1HKJnZc+ X2ci7em4CE6RgoK+3sfMtsJn84fY4ZTfnNkYUVDUUTL97i4e1+zKox+REHIldqs/KD/5 B+Pw== X-Gm-Message-State: AOAM533u7a6cfxi8v7XLjoJmPQudTW00WMxUOqvNuLdHjU7yqINhc+CO eSiBEnOI/cSzxsyJrHBY9paDupNC X-Google-Smtp-Source: ABdhPJx6K+W8ODcLnFaxf6wymvJeG1718+BVT5s3DANaVB7WOADh0+RhefUc7GXg8IE0H03CvSoSXw== X-Received: by 2002:a63:4407:: with SMTP id r7mr5863252pga.274.1592385379546; Wed, 17 Jun 2020 02:16:19 -0700 (PDT) Received: from localhost.localdomain ([2409:4064:40b:ccca:8c2b:e6b:e670:4483]) by smtp.gmail.com with ESMTPSA id r8sm17162110pgn.19.2020.06.17.02.16.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 Jun 2020 02:16:18 -0700 (PDT) From: Abhishek Kumar To: git@vger.kernel.org Cc: jnareb@gmail.com, stolee@gmail.com Subject: [GSOC Patch v4 1/4] object: drop parsed_object_pool->commit_count Date: Wed, 17 Jun 2020 14:44:08 +0530 Message-Id: <20200617091411.14650-2-abhishekkumar8222@gmail.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200617091411.14650-1-abhishekkumar8222@gmail.com> References: <20200604072759.19142-1-abhishekkumar8222@gmail.com> <20200617091411.14650-1-abhishekkumar8222@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org 14ba97f8 (alloc: allow arbitrary repositories for alloc functions, 2018-05-15) introduced parsed_object_pool->commit_count to keep count of commits per repository and was used to assign commit->index. However, commit-slab code requires commit->index values to be unique and a global count would be correct, rather than a per-repo count. Let's introduce a static counter variable, `parsed_commits_count` to keep track of parsed commits so far. As commit_count has no use anymore, let's also drop it from the struct. Signed-off-by: Abhishek Kumar --- alloc.c | 16 +++++++++++----- alloc.h | 2 +- blob.c | 2 +- builtin/commit-graph.c | 2 +- builtin/fsck.c | 2 +- commit.c | 4 ++-- object.c | 4 ++-- object.h | 3 +-- refs.c | 2 +- t/helper/test-reach.c | 2 +- tag.c | 2 +- tree.c | 2 +- 12 files changed, 24 insertions(+), 19 deletions(-) diff --git a/alloc.c b/alloc.c index 1c64c4dd16..99fa934b32 100644 --- a/alloc.c +++ b/alloc.c @@ -99,15 +99,21 @@ void *alloc_object_node(struct repository *r) return obj; } -static unsigned int alloc_commit_index(struct repository *r) +/* + * The returned count is to be used as an index into commit slabs, + * that are *NOT* maintained per repository, and that is why a single + * global counter is used. + */ +static unsigned int alloc_commit_index(void) { - return r->parsed_objects->commit_count++; + static unsigned int parsed_commits_count; + return parsed_commits_count++; } -void init_commit_node(struct repository *r, struct commit *c) +void init_commit_node(struct commit *c) { c->object.type = OBJ_COMMIT; - c->index = alloc_commit_index(r); + c->index = alloc_commit_index(); c->graph_pos = COMMIT_NOT_FROM_GRAPH; c->generation = GENERATION_NUMBER_INFINITY; } @@ -115,7 +121,7 @@ void init_commit_node(struct repository *r, struct commit *c) void *alloc_commit_node(struct repository *r) { struct commit *c = alloc_node(r->parsed_objects->commit_state, sizeof(struct commit)); - init_commit_node(r, c); + init_commit_node(c); return c; } diff --git a/alloc.h b/alloc.h index ed1071c11e..371d388b55 100644 --- a/alloc.h +++ b/alloc.h @@ -9,7 +9,7 @@ struct repository; void *alloc_blob_node(struct repository *r); void *alloc_tree_node(struct repository *r); -void init_commit_node(struct repository *r, struct commit *c); +void init_commit_node(struct commit *c); void *alloc_commit_node(struct repository *r); void *alloc_tag_node(struct repository *r); void *alloc_object_node(struct repository *r); diff --git a/blob.c b/blob.c index 36f9abda19..182718aba9 100644 --- a/blob.c +++ b/blob.c @@ -10,7 +10,7 @@ struct blob *lookup_blob(struct repository *r, const struct object_id *oid) struct object *obj = lookup_object(r, oid); if (!obj) return create_object(r, oid, alloc_blob_node(r)); - return object_as_type(r, obj, OBJ_BLOB, 0); + return object_as_type(obj, OBJ_BLOB, 0); } int parse_blob_buffer(struct blob *item, void *buffer, unsigned long size) diff --git a/builtin/commit-graph.c b/builtin/commit-graph.c index 75455da138..f6797e2a9f 100644 --- a/builtin/commit-graph.c +++ b/builtin/commit-graph.c @@ -154,7 +154,7 @@ static int read_one_commit(struct oidset *commits, struct progress *progress, NULL, 0); if (!result) return error(_("invalid object: %s"), hash); - else if (object_as_type(the_repository, result, OBJ_COMMIT, 1)) + else if (object_as_type(result, OBJ_COMMIT, 1)) oidset_insert(commits, &result->oid); display_progress(progress, oidset_size(commits)); diff --git a/builtin/fsck.c b/builtin/fsck.c index f02cbdb439..b2cef01389 100644 --- a/builtin/fsck.c +++ b/builtin/fsck.c @@ -241,7 +241,7 @@ static void mark_unreachable_referents(const struct object_id *oid) enum object_type type = oid_object_info(the_repository, &obj->oid, NULL); if (type > 0) - object_as_type(the_repository, obj, type, 0); + object_as_type(obj, type, 0); } options.walk = mark_used; diff --git a/commit.c b/commit.c index 87686a7055..b30875e66b 100644 --- a/commit.c +++ b/commit.c @@ -37,7 +37,7 @@ struct commit *lookup_commit_reference_gently(struct repository *r, if (!obj) return NULL; - return object_as_type(r, obj, OBJ_COMMIT, quiet); + return object_as_type(obj, OBJ_COMMIT, quiet); } struct commit *lookup_commit_reference(struct repository *r, const struct object_id *oid) @@ -62,7 +62,7 @@ struct commit *lookup_commit(struct repository *r, const struct object_id *oid) struct object *obj = lookup_object(r, oid); if (!obj) return create_object(r, oid, alloc_commit_node(r)); - return object_as_type(r, obj, OBJ_COMMIT, 0); + return object_as_type(obj, OBJ_COMMIT, 0); } struct commit *lookup_commit_reference_by_name(const char *name) diff --git a/object.c b/object.c index 794c86650e..3257518656 100644 --- a/object.c +++ b/object.c @@ -157,13 +157,13 @@ void *create_object(struct repository *r, const struct object_id *oid, void *o) return obj; } -void *object_as_type(struct repository *r, struct object *obj, enum object_type type, int quiet) +void *object_as_type(struct object *obj, enum object_type type, int quiet) { if (obj->type == type) return obj; else if (obj->type == OBJ_NONE) { if (type == OBJ_COMMIT) - init_commit_node(r, (struct commit *) obj); + init_commit_node((struct commit *) obj); else obj->type = type; return obj; diff --git a/object.h b/object.h index b22328b838..532d7d7f28 100644 --- a/object.h +++ b/object.h @@ -15,7 +15,6 @@ struct parsed_object_pool { struct alloc_state *commit_state; struct alloc_state *tag_state; struct alloc_state *object_state; - unsigned commit_count; /* parent substitutions from .git/info/grafts and .git/shallow */ struct commit_graft **grafts; @@ -121,7 +120,7 @@ struct object *lookup_object(struct repository *r, const struct object_id *oid); void *create_object(struct repository *r, const struct object_id *oid, void *obj); -void *object_as_type(struct repository *r, struct object *obj, enum object_type type, int quiet); +void *object_as_type(struct object *obj, enum object_type type, int quiet); /* * Returns the object, having parsed it to find out what it is. diff --git a/refs.c b/refs.c index 224ff66c7b..1f551dd279 100644 --- a/refs.c +++ b/refs.c @@ -339,7 +339,7 @@ enum peel_status peel_object(const struct object_id *name, struct object_id *oid if (o->type == OBJ_NONE) { int type = oid_object_info(the_repository, name, NULL); - if (type < 0 || !object_as_type(the_repository, o, type, 0)) + if (type < 0 || !object_as_type(o, type, 0)) return PEEL_INVALID; } diff --git a/t/helper/test-reach.c b/t/helper/test-reach.c index a0272178b7..ccf837cb33 100644 --- a/t/helper/test-reach.c +++ b/t/helper/test-reach.c @@ -67,7 +67,7 @@ int cmd__reach(int ac, const char **av) die("failed to load commit for input %s resulting in oid %s\n", buf.buf, oid_to_hex(&oid)); - c = object_as_type(r, peeled, OBJ_COMMIT, 0); + c = object_as_type(peeled, OBJ_COMMIT, 0); if (!c) die("failed to load commit for input %s resulting in oid %s\n", diff --git a/tag.c b/tag.c index 71b544467e..1ed2684e45 100644 --- a/tag.c +++ b/tag.c @@ -103,7 +103,7 @@ struct tag *lookup_tag(struct repository *r, const struct object_id *oid) struct object *obj = lookup_object(r, oid); if (!obj) return create_object(r, oid, alloc_tag_node(r)); - return object_as_type(r, obj, OBJ_TAG, 0); + return object_as_type(obj, OBJ_TAG, 0); } static timestamp_t parse_tag_date(const char *buf, const char *tail) diff --git a/tree.c b/tree.c index 1466bcc6a8..e76517f6b1 100644 --- a/tree.c +++ b/tree.c @@ -200,7 +200,7 @@ struct tree *lookup_tree(struct repository *r, const struct object_id *oid) struct object *obj = lookup_object(r, oid); if (!obj) return create_object(r, oid, alloc_tree_node(r)); - return object_as_type(r, obj, OBJ_TREE, 0); + return object_as_type(obj, OBJ_TREE, 0); } int parse_tree_buffer(struct tree *item, void *buffer, unsigned long size) From patchwork Wed Jun 17 09:14:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Abhishek Kumar X-Patchwork-Id: 11609481 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 89710912 for ; Wed, 17 Jun 2020 09:16:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6D96B2100A for ; Wed, 17 Jun 2020 09:16:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="WQOKIsLL" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726510AbgFQJQZ (ORCPT ); Wed, 17 Jun 2020 05:16:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38790 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725554AbgFQJQY (ORCPT ); Wed, 17 Jun 2020 05:16:24 -0400 Received: from mail-pj1-x1044.google.com (mail-pj1-x1044.google.com [IPv6:2607:f8b0:4864:20::1044]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 96D75C061573 for ; Wed, 17 Jun 2020 02:16:24 -0700 (PDT) Received: by mail-pj1-x1044.google.com with SMTP id m2so729492pjv.2 for ; Wed, 17 Jun 2020 02:16:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=WSarGbZhPLgwcPVKPXtGQ3pDmuwQ7O06TtHEYG7Keus=; b=WQOKIsLLdZU5HnSofd9CxhWdUDNIo8BZhW4bK37vR//rpTCinb15UEXidS1f/5A4FE HVpw4A6UvyPXKA+F9brSFJyAa7jjRur8+9UMXW+j5RckeLTpRS0o84ndRhzrslhlb9xo N0Xa+aYhJr9R2/WxQvJQuaWGG9OdLx4Kmea94rvh2VtrgcX1LH9zzLaI43TvUx9E+BUX joqtmtzt42al5kMU82zBLZJQNP/9o65WdfIBbNTuGS9wN5o+wtHqpe8iX7xAJQYm0dwG oNNBoQ9MjdRPiicPzPSGcAI+1g5hJBBNg83sZKgR73b3yUpvbk++6AHmUo0xmRK6TjWK Jb5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=WSarGbZhPLgwcPVKPXtGQ3pDmuwQ7O06TtHEYG7Keus=; b=Vmb7GWYR+CVn6igsRGAKAouNhriRIMwr9KNWx3rezJz8KYCeFbXdfhrWpeI0t6M8JN iuyi4Y2kJ7kXkN0hFTfgSclFkteD8mSBqJ9WhMC1wzDGz+cCvEgXSIoI4LondKZxwsYC btF0FqOGiU8sN5wE8404wLe7UdjkyUyYN7Q0iV4+ktA/yjq1CXDIC0NnkyUUnCzIfpDM 7oCavcOrZNrQ3AfpZa0OY3SgYQdjxYJIGzsohwOIQ3m2Vs9+3Fimzhz23+f0o5rf22OL Bce3k3taUVcjP7m7q36YsA3Czrh3vgNsZ9ze7+6/urJck0q24BUlduCsfHXIjl4GmYWE Xn/w== X-Gm-Message-State: AOAM530fUmritK1Zk7tGDfMPXdj3GHV3VX2YsUkBX6PwphPjwadJcCEP OSuZjXZe9gYhbpJ2xVqHJIcF3nAT X-Google-Smtp-Source: ABdhPJzI1MoqpN63n7z2hfYuQHbPArlqS+SvycvXFWjIfh0obGRLk96LHW8fc6vKqzbkiBrdNbwYZg== X-Received: by 2002:a17:902:ab8e:: with SMTP id f14mr6035873plr.80.1592385383565; Wed, 17 Jun 2020 02:16:23 -0700 (PDT) Received: from localhost.localdomain ([2409:4064:40b:ccca:8c2b:e6b:e670:4483]) by smtp.gmail.com with ESMTPSA id r8sm17162110pgn.19.2020.06.17.02.16.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 Jun 2020 02:16:22 -0700 (PDT) From: Abhishek Kumar To: git@vger.kernel.org Cc: jnareb@gmail.com, stolee@gmail.com Subject: [GSOC Patch v4 2/4] commit-graph: introduce commit_graph_data_slab Date: Wed, 17 Jun 2020 14:44:09 +0530 Message-Id: <20200617091411.14650-3-abhishekkumar8222@gmail.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200617091411.14650-1-abhishekkumar8222@gmail.com> References: <20200604072759.19142-1-abhishekkumar8222@gmail.com> <20200617091411.14650-1-abhishekkumar8222@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org The struct commit is used in many contexts. However, members `generation` and `graph_pos` are only used for commit-graph related operations and otherwise waste memory. This wastage would have been more pronounced as we transition to generation number v2, which uses 64-bit generation number instead of current 32-bits. As they are often accessed together, let's introduce struct commit_graph_data and move them to a commit_graph_data slab. While the overall test suite runs just as fast as master, (series: 26m48s, master: 27m34s, faster by 2.87%), certain commands like `git merge-base --is-ancestor` were slowed by 40% as discovered by Szeder Gábor [1]. After minimizing commit-slab access, the slow down persists but is closer to 20%. Derrick Stolee believes the slow down is attributable to the underlying algorithm rather than the slowness of commit-slab access [2] and we will follow-up in a later series. [1]: https://lore.kernel.org/git/20200607195347.GA8232@szeder.dev/ [2]: https://lore.kernel.org/git/13db757a-9412-7f1e-805c-8a028c4ab2b1@gmail.com/ Signed-off-by: Abhishek Kumar --- On linux.git with HEAD at 08bf1a27 (Merge tag 'powerpc-5.8-2' of git://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux, 2020-06-13): `git merge-base --is-ancestor HEAD~50000 HEAD` Time (master): 0.787s Time (series): 0.927s Change: 17.79% (slower) Max RSS (master): 177694kb Max RSS (series): 177707kb Change: 0.01% (more) `git gc` Time (master): 3m55s Time (series): 3m38s Change: 7.23% (faster) Max RSS (master): 4889868kb Max RSS (series): 4911960kb Change: 0.45% (more) Earlier implementation of commit_graph_data_at() was incorrect, as we used to iterate from old slab count to new slab count - assuming all intermediate slabs are allocated. This is incorrect as the slabs are allocated only when there's a corresponding commit. It now makes *two slab accesses* in the worst case, but it's okay since the worst case occurs once nearly every (512kb / 8b) commits. commit-graph.c | 78 +++++++++++++++++++++++++++++++++++++++++++------- commit-graph.h | 10 +++++++ 2 files changed, 78 insertions(+), 10 deletions(-) diff --git a/commit-graph.c b/commit-graph.c index 2ff042fbf4..8ad7d202b2 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -87,6 +87,58 @@ static int commit_pos_cmp(const void *va, const void *vb) commit_pos_at(&commit_pos, b); } +define_commit_slab(commit_graph_data_slab, struct commit_graph_data); +static struct commit_graph_data_slab commit_graph_data_slab = + COMMIT_SLAB_INIT(1, commit_graph_data_slab); + +uint32_t commit_graph_position(const struct commit *c) +{ + struct commit_graph_data *data = + commit_graph_data_slab_peek(&commit_graph_data_slab, c); + + return data ? data->graph_pos : COMMIT_NOT_FROM_GRAPH; +} + +uint32_t commit_graph_generation(const struct commit *c) +{ + struct commit_graph_data *data = + commit_graph_data_slab_peek(&commit_graph_data_slab, c); + + if (!data) + return GENERATION_NUMBER_INFINITY; + else if (data->graph_pos == COMMIT_NOT_FROM_GRAPH) + return GENERATION_NUMBER_INFINITY; + + return data->generation; +} + +static struct commit_graph_data *commit_graph_data_at(const struct commit *c) +{ + unsigned int i, nth_slab; + struct commit_graph_data *data = + commit_graph_data_slab_peek(&commit_graph_data_slab, c); + + if (data) + return data; + + nth_slab = c->index / commit_graph_data_slab.slab_size; + data = commit_graph_data_slab_at(&commit_graph_data_slab, c); + + /* + * commit-slab initializes elements with zero, overwrite this with + * COMMIT_NOT_FROM_GRAPH for graph_pos. + * + * We avoid initializing generation with checking if graph position + * is not COMMIT_NOT_FROM_GRAPH. + */ + for (i = 0; i < commit_graph_data_slab.slab_size; i++) { + commit_graph_data_slab.slab[nth_slab][i].graph_pos = + COMMIT_NOT_FROM_GRAPH; + } + + return data; +} + static int commit_gen_cmp(const void *va, const void *vb) { const struct commit *a = *(const struct commit **)va; @@ -1020,7 +1072,7 @@ static void write_graph_chunk_data(struct hashfile *f, int hash_len, else packedDate[0] = 0; - packedDate[0] |= htonl((*list)->generation << 2); + packedDate[0] |= htonl(commit_graph_data_at(*list)->generation << 2); packedDate[1] = htonl((*list)->date); hashwrite(f, packedDate, 8); @@ -1251,9 +1303,11 @@ static void compute_generation_numbers(struct write_commit_graph_context *ctx) _("Computing commit graph generation numbers"), ctx->commits.nr); for (i = 0; i < ctx->commits.nr; i++) { + uint32_t generation = commit_graph_data_at(ctx->commits.list[i])->generation; + display_progress(ctx->progress, i + 1); - if (ctx->commits.list[i]->generation != GENERATION_NUMBER_INFINITY && - ctx->commits.list[i]->generation != GENERATION_NUMBER_ZERO) + if (generation != GENERATION_NUMBER_INFINITY && + generation != GENERATION_NUMBER_ZERO) continue; commit_list_insert(ctx->commits.list[i], &list); @@ -1264,22 +1318,26 @@ static void compute_generation_numbers(struct write_commit_graph_context *ctx) uint32_t max_generation = 0; for (parent = current->parents; parent; parent = parent->next) { - if (parent->item->generation == GENERATION_NUMBER_INFINITY || - parent->item->generation == GENERATION_NUMBER_ZERO) { + generation = commit_graph_data_at(parent->item)->generation; + + if (generation == GENERATION_NUMBER_INFINITY || + generation == GENERATION_NUMBER_ZERO) { all_parents_computed = 0; commit_list_insert(parent->item, &list); break; - } else if (parent->item->generation > max_generation) { - max_generation = parent->item->generation; + } else if (generation > max_generation) { + max_generation = generation; } } if (all_parents_computed) { - current->generation = max_generation + 1; + struct commit_graph_data *data = commit_graph_data_at(current); + + data->generation = max_generation + 1; pop_commit(&list); - if (current->generation > GENERATION_NUMBER_MAX) - current->generation = GENERATION_NUMBER_MAX; + if (data->generation > GENERATION_NUMBER_MAX) + data->generation = GENERATION_NUMBER_MAX; } } } diff --git a/commit-graph.h b/commit-graph.h index 3ba0da1e5f..28f89cdf3e 100644 --- a/commit-graph.h +++ b/commit-graph.h @@ -135,4 +135,14 @@ void free_commit_graph(struct commit_graph *); */ void disable_commit_graph(struct repository *r); +struct commit_graph_data { + uint32_t graph_pos; + uint32_t generation; +}; + +/* + * Commits should be parsed before accessing generation, graph positions. + */ +uint32_t commit_graph_generation(const struct commit *); +uint32_t commit_graph_position(const struct commit *); #endif From patchwork Wed Jun 17 09:14:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Abhishek Kumar X-Patchwork-Id: 11609483 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BE32E13B1 for ; Wed, 17 Jun 2020 09:16:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8D5D72100A for ; Wed, 17 Jun 2020 09:16:33 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="pVNAtpQs" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726523AbgFQJQc (ORCPT ); Wed, 17 Jun 2020 05:16:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38808 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725554AbgFQJQc (ORCPT ); Wed, 17 Jun 2020 05:16:32 -0400 Received: from mail-pj1-x1042.google.com (mail-pj1-x1042.google.com [IPv6:2607:f8b0:4864:20::1042]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5C450C061573 for ; Wed, 17 Jun 2020 02:16:30 -0700 (PDT) Received: by mail-pj1-x1042.google.com with SMTP id k2so691510pjs.2 for ; Wed, 17 Jun 2020 02:16:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=rolcCkdc5FmEMe6RUtdTl4C+35KIMxSgNCc2A+YE0sM=; b=pVNAtpQszoJx6d5ceHU2af/hleOFhg+QeOeXli8IU97FRvO2EKnfUVgs6ksotVpmwG OtDUcSxnCbh5gZywe+rUzChFuh+1APb+LLRh8m74DWGJ37UkQMPPGzQjF5PN2leU87jw 4l8ww13JpmF4lRPIHQf6kQibETXqjVMYj0bubiPYHLAQFWsZgHvqSuG09cOJqxtMo1B9 m+/nnmPL20wXeaBgsgOz8+wsX7EzMTz5DSBtZds7n9hksQIdo8uyb2eihUBVcyEKpHDR AEndR2z883NZcsF/bxL85nr2uY8LdMdKsAsXOCLxf3gQMuy3SK7lITR1XFRugGC9BBre Jn5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=rolcCkdc5FmEMe6RUtdTl4C+35KIMxSgNCc2A+YE0sM=; b=RpohB/HveAH8lHNO2oNbS46bs5a83DeG/LTq1WT6iowm4eiLPFtWqH0t5e4SVKziui SASy0XRCTzUF69QxgFNfVFgovWnLRU3mHpBpNwUPQ0cVXqxVIvBdLYRew/qF4YgTLI/e HWuIw/ADqDNvmuESH2vC6VRh1G3mv4W9AUnf0T1eAsusBBOw8gkDzSGCgWpaEDHJuD46 rnEQj1s/FUUe9rQVNS9bJ9zt3fbbB0DYN6gOnlZE4zsHPXBsL3zIpAUii5Ce5arE8jGK Yxpu4YijKL70fU0HlqMSosPl0oGgp5z5rqA9VwQHQ8XAQTEOMZrBC+jPhRPtBqTnlmqh dN3A== X-Gm-Message-State: AOAM530hlFhVOhwmOt1P8OMLPoFqtaPGdlYOQZfAutGZT7ymxW0q3Ex2 HEQzsZ7HSa/X3wdL3h7hYH7tuY/r X-Google-Smtp-Source: ABdhPJzhYaolPo+kXTMJvua70yBV94VcXO+CXrIRXseLX1jZVwHWmqvxHOf2wOBmbsPLsCzGd4BTpg== X-Received: by 2002:a17:90a:7645:: with SMTP id s5mr7245923pjl.50.1592385388944; Wed, 17 Jun 2020 02:16:28 -0700 (PDT) Received: from localhost.localdomain ([2409:4064:40b:ccca:8c2b:e6b:e670:4483]) by smtp.gmail.com with ESMTPSA id r8sm17162110pgn.19.2020.06.17.02.16.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 Jun 2020 02:16:28 -0700 (PDT) From: Abhishek Kumar To: git@vger.kernel.org Cc: jnareb@gmail.com, stolee@gmail.com Subject: [GSOC Patch v4 3/4] commit: move members graph_pos, generation to a slab Date: Wed, 17 Jun 2020 14:44:10 +0530 Message-Id: <20200617091411.14650-4-abhishekkumar8222@gmail.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200617091411.14650-1-abhishekkumar8222@gmail.com> References: <20200604072759.19142-1-abhishekkumar8222@gmail.com> <20200617091411.14650-1-abhishekkumar8222@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org We remove members `graph_pos` and `generation` from the struct commit. The default assignments in init_commit_node() are no longer valid, which is fine as the slab helpers return appropriate default values and the assignments are removed. We will replace existing use of commit->generation and commit->graph_pos by commit_graph_data_slab helpers using `contrib/coccinelle/commit.cocci'. Signed-off-by: Abhishek Kumar --- alloc.c | 2 -- blame.c | 2 +- bloom.c | 6 ++-- commit-graph.c | 40 +++++++++++++------------- commit-reach.c | 50 ++++++++++++++++----------------- commit.c | 6 ++-- commit.h | 2 -- contrib/coccinelle/commit.cocci | 18 ++++++++++++ revision.c | 16 +++++------ 9 files changed, 78 insertions(+), 64 deletions(-) diff --git a/alloc.c b/alloc.c index 99fa934b32..957a0af362 100644 --- a/alloc.c +++ b/alloc.c @@ -114,8 +114,6 @@ void init_commit_node(struct commit *c) { c->object.type = OBJ_COMMIT; c->index = alloc_commit_index(); - c->graph_pos = COMMIT_NOT_FROM_GRAPH; - c->generation = GENERATION_NUMBER_INFINITY; } void *alloc_commit_node(struct repository *r) diff --git a/blame.c b/blame.c index da7e28800e..82fa16d658 100644 --- a/blame.c +++ b/blame.c @@ -1272,7 +1272,7 @@ static int maybe_changed_path(struct repository *r, if (!bd) return 1; - if (origin->commit->generation == GENERATION_NUMBER_INFINITY) + if (commit_graph_generation(origin->commit) == GENERATION_NUMBER_INFINITY) return 1; filter = get_bloom_filter(r, origin->commit, 0); diff --git a/bloom.c b/bloom.c index 6c7611847a..3062aafaba 100644 --- a/bloom.c +++ b/bloom.c @@ -34,14 +34,14 @@ static int load_bloom_filter_from_graph(struct commit_graph *g, { uint32_t lex_pos, start_index, end_index; - while (c->graph_pos < g->num_commits_in_base) + while (commit_graph_position(c) < g->num_commits_in_base) g = g->base_graph; /* The commit graph commit 'c' lives in doesn't carry bloom filters. */ if (!g->chunk_bloom_indexes) return 0; - lex_pos = c->graph_pos - g->num_commits_in_base; + lex_pos = commit_graph_position(c) - g->num_commits_in_base; end_index = get_be32(g->chunk_bloom_indexes + 4 * lex_pos); @@ -193,7 +193,7 @@ struct bloom_filter *get_bloom_filter(struct repository *r, if (!filter->data) { load_commit_graph_info(r, c); - if (c->graph_pos != COMMIT_NOT_FROM_GRAPH && + if (commit_graph_position(c) != COMMIT_NOT_FROM_GRAPH && r->objects->commit_graph->chunk_bloom_indexes) { if (load_bloom_filter_from_graph(r->objects->commit_graph, filter, c)) return filter; diff --git a/commit-graph.c b/commit-graph.c index 8ad7d202b2..14cc7e931c 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -145,9 +145,9 @@ static int commit_gen_cmp(const void *va, const void *vb) const struct commit *b = *(const struct commit **)vb; /* lower generation commits first */ - if (a->generation < b->generation) + if (commit_graph_generation(a) < commit_graph_generation(b)) return -1; - else if (a->generation > b->generation) + else if (commit_graph_generation(a) > commit_graph_generation(b)) return 1; /* use date as a heuristic when generations are equal */ @@ -722,7 +722,7 @@ static struct commit_list **insert_parent_or_die(struct repository *r, c = lookup_commit(r, &oid); if (!c) die(_("could not find commit %s"), oid_to_hex(&oid)); - c->graph_pos = pos; + commit_graph_data_at(c)->graph_pos = pos; return &commit_list_insert(c, pptr)->next; } @@ -736,8 +736,8 @@ static void fill_commit_graph_info(struct commit *item, struct commit_graph *g, lex_index = pos - g->num_commits_in_base; commit_data = g->chunk_commit_data + GRAPH_DATA_WIDTH * lex_index; - item->graph_pos = pos; - item->generation = get_be32(commit_data + g->hash_len + 8) >> 2; + commit_graph_data_at(item)->graph_pos = pos; + commit_graph_data_at(item)->generation = get_be32(commit_data + g->hash_len + 8) >> 2; } static inline void set_commit_tree(struct commit *c, struct tree *t) @@ -766,7 +766,7 @@ static int fill_commit_in_graph(struct repository *r, * Store the "full" position, but then use the * "local" position for the rest of the calculation. */ - item->graph_pos = pos; + commit_graph_data_at(item)->graph_pos = pos; lex_index = pos - g->num_commits_in_base; commit_data = g->chunk_commit_data + (g->hash_len + 16) * lex_index; @@ -779,7 +779,7 @@ static int fill_commit_in_graph(struct repository *r, date_low = get_be32(commit_data + g->hash_len + 12); item->date = (timestamp_t)((date_high << 32) | date_low); - item->generation = get_be32(commit_data + g->hash_len + 8) >> 2; + commit_graph_data_at(item)->generation = get_be32(commit_data + g->hash_len + 8) >> 2; pptr = &item->parents; @@ -811,8 +811,8 @@ static int fill_commit_in_graph(struct repository *r, static int find_commit_in_graph(struct commit *item, struct commit_graph *g, uint32_t *pos) { - if (item->graph_pos != COMMIT_NOT_FROM_GRAPH) { - *pos = item->graph_pos; + if (commit_graph_position(item) != COMMIT_NOT_FROM_GRAPH) { + *pos = commit_graph_position(item); return 1; } else { struct commit_graph *cur_g = g; @@ -868,11 +868,11 @@ static struct tree *load_tree_for_commit(struct repository *r, struct object_id oid; const unsigned char *commit_data; - while (c->graph_pos < g->num_commits_in_base) + while (commit_graph_position(c) < g->num_commits_in_base) g = g->base_graph; commit_data = g->chunk_commit_data + - GRAPH_DATA_WIDTH * (c->graph_pos - g->num_commits_in_base); + GRAPH_DATA_WIDTH * (commit_graph_position(c) - g->num_commits_in_base); hashcpy(oid.hash, commit_data); set_commit_tree(c, lookup_tree(r, &oid)); @@ -886,7 +886,7 @@ static struct tree *get_commit_tree_in_graph_one(struct repository *r, { if (c->maybe_tree) return c->maybe_tree; - if (c->graph_pos == COMMIT_NOT_FROM_GRAPH) + if (commit_graph_position(c) == COMMIT_NOT_FROM_GRAPH) BUG("get_commit_tree_in_graph_one called from non-commit-graph commit"); return load_tree_for_commit(r, g, (struct commit *)c); @@ -1271,7 +1271,7 @@ static void close_reachable(struct write_commit_graph_context *ctx) continue; if (ctx->split) { if ((!parse_commit(commit) && - commit->graph_pos == COMMIT_NOT_FROM_GRAPH) || + commit_graph_position(commit) == COMMIT_NOT_FROM_GRAPH) || flags == COMMIT_GRAPH_SPLIT_REPLACE) add_missing_parents(ctx, commit); } else if (!parse_commit_no_graph(commit)) @@ -1516,7 +1516,7 @@ static uint32_t count_distinct_commits(struct write_commit_graph_context *ctx) if (ctx->split) { struct commit *c = lookup_commit(ctx->r, &ctx->oids.list[i]); - if (!c || c->graph_pos != COMMIT_NOT_FROM_GRAPH) + if (!c || commit_graph_position(c) != COMMIT_NOT_FROM_GRAPH) continue; } @@ -1550,7 +1550,7 @@ static void copy_oids_to_commits(struct write_commit_graph_context *ctx) ctx->commits.list[ctx->commits.nr] = lookup_commit(ctx->r, &ctx->oids.list[i]); if (ctx->split && flags != COMMIT_GRAPH_SPLIT_REPLACE && - ctx->commits.list[ctx->commits.nr]->graph_pos != COMMIT_NOT_FROM_GRAPH) + commit_graph_position(ctx->commits.list[ctx->commits.nr]) != COMMIT_NOT_FROM_GRAPH) continue; if (ctx->split && flags == COMMIT_GRAPH_SPLIT_REPLACE) @@ -2337,8 +2337,8 @@ int verify_commit_graph(struct repository *r, struct commit_graph *g, int flags) oid_to_hex(&graph_parents->item->object.oid), oid_to_hex(&odb_parents->item->object.oid)); - if (graph_parents->item->generation > max_generation) - max_generation = graph_parents->item->generation; + if (commit_graph_generation(graph_parents->item) > max_generation) + max_generation = commit_graph_generation(graph_parents->item); graph_parents = graph_parents->next; odb_parents = odb_parents->next; @@ -2348,7 +2348,7 @@ int verify_commit_graph(struct repository *r, struct commit_graph *g, int flags) graph_report(_("commit-graph parent list for commit %s terminates early"), oid_to_hex(&cur_oid)); - if (!graph_commit->generation) { + if (!commit_graph_generation(graph_commit)) { if (generation_zero == GENERATION_NUMBER_EXISTS) graph_report(_("commit-graph has generation number zero for commit %s, but non-zero elsewhere"), oid_to_hex(&cur_oid)); @@ -2368,10 +2368,10 @@ int verify_commit_graph(struct repository *r, struct commit_graph *g, int flags) if (max_generation == GENERATION_NUMBER_MAX) max_generation--; - if (graph_commit->generation != max_generation + 1) + if (commit_graph_generation(graph_commit) != max_generation + 1) graph_report(_("commit-graph generation for commit %s is %u != %u"), oid_to_hex(&cur_oid), - graph_commit->generation, + commit_graph_generation(graph_commit), max_generation + 1); if (graph_commit->date != odb_commit->date) diff --git a/commit-reach.c b/commit-reach.c index 4ca7e706a1..3b2f863f5f 100644 --- a/commit-reach.c +++ b/commit-reach.c @@ -59,13 +59,13 @@ static struct commit_list *paint_down_to_common(struct repository *r, struct commit_list *parents; int flags; - if (min_generation && commit->generation > last_gen) + if (min_generation && commit_graph_generation(commit) > last_gen) BUG("bad generation skip %8x > %8x at %s", - commit->generation, last_gen, + commit_graph_generation(commit), last_gen, oid_to_hex(&commit->object.oid)); - last_gen = commit->generation; + last_gen = commit_graph_generation(commit); - if (commit->generation < min_generation) + if (commit_graph_generation(commit) < min_generation) break; flags = commit->object.flags & (PARENT1 | PARENT2 | STALE); @@ -176,7 +176,7 @@ static int remove_redundant(struct repository *r, struct commit **array, int cnt repo_parse_commit(r, array[i]); for (i = 0; i < cnt; i++) { struct commit_list *common; - uint32_t min_generation = array[i]->generation; + uint32_t min_generation = commit_graph_generation(array[i]); if (redundant[i]) continue; @@ -186,8 +186,8 @@ static int remove_redundant(struct repository *r, struct commit **array, int cnt filled_index[filled] = j; work[filled++] = array[j]; - if (array[j]->generation < min_generation) - min_generation = array[j]->generation; + if (commit_graph_generation(array[j]) < min_generation) + min_generation = commit_graph_generation(array[j]); } common = paint_down_to_common(r, array[i], filled, work, min_generation); @@ -323,16 +323,16 @@ int repo_in_merge_bases_many(struct repository *r, struct commit *commit, for (i = 0; i < nr_reference; i++) { if (repo_parse_commit(r, reference[i])) return ret; - if (reference[i]->generation < min_generation) - min_generation = reference[i]->generation; + if (commit_graph_generation(reference[i]) < min_generation) + min_generation = commit_graph_generation(reference[i]); } - if (commit->generation > min_generation) + if (commit_graph_generation(commit) > min_generation) return ret; bases = paint_down_to_common(r, commit, nr_reference, reference, - commit->generation); + commit_graph_generation(commit)); if (commit->object.flags & PARENT2) ret = 1; clear_commit_marks(commit, all_flags); @@ -467,7 +467,7 @@ static enum contains_result contains_test(struct commit *candidate, /* Otherwise, we don't know; prepare to recurse */ parse_commit_or_die(candidate); - if (candidate->generation < cutoff) + if (commit_graph_generation(candidate) < cutoff) return CONTAINS_NO; return CONTAINS_UNKNOWN; @@ -492,8 +492,8 @@ static enum contains_result contains_tag_algo(struct commit *candidate, for (p = want; p; p = p->next) { struct commit *c = p->item; load_commit_graph_info(the_repository, c); - if (c->generation < cutoff) - cutoff = c->generation; + if (commit_graph_generation(c) < cutoff) + cutoff = commit_graph_generation(c); } result = contains_test(candidate, want, cache, cutoff); @@ -544,9 +544,9 @@ static int compare_commits_by_gen(const void *_a, const void *_b) const struct commit *a = *(const struct commit * const *)_a; const struct commit *b = *(const struct commit * const *)_b; - if (a->generation < b->generation) + if (commit_graph_generation(a) < commit_graph_generation(b)) return -1; - if (a->generation > b->generation) + if (commit_graph_generation(a) > commit_graph_generation(b)) return 1; return 0; } @@ -585,7 +585,7 @@ int can_all_from_reach_with_flag(struct object_array *from, list[nr_commits] = (struct commit *)from_one; if (parse_commit(list[nr_commits]) || - list[nr_commits]->generation < min_generation) { + commit_graph_generation(list[nr_commits]) < min_generation) { result = 0; goto cleanup; } @@ -621,7 +621,7 @@ int can_all_from_reach_with_flag(struct object_array *from, if (parse_commit(parent->item) || parent->item->date < min_commit_date || - parent->item->generation < min_generation) + commit_graph_generation(parent->item) < min_generation) continue; commit_list_insert(parent->item, &stack); @@ -665,8 +665,8 @@ int can_all_from_reach(struct commit_list *from, struct commit_list *to, if (from_iter->item->date < min_commit_date) min_commit_date = from_iter->item->date; - if (from_iter->item->generation < min_generation) - min_generation = from_iter->item->generation; + if (commit_graph_generation(from_iter->item) < min_generation) + min_generation = commit_graph_generation(from_iter->item); } from_iter = from_iter->next; @@ -677,8 +677,8 @@ int can_all_from_reach(struct commit_list *from, struct commit_list *to, if (to_iter->item->date < min_commit_date) min_commit_date = to_iter->item->date; - if (to_iter->item->generation < min_generation) - min_generation = to_iter->item->generation; + if (commit_graph_generation(to_iter->item) < min_generation) + min_generation = commit_graph_generation(to_iter->item); } to_iter->item->object.flags |= PARENT2; @@ -721,8 +721,8 @@ struct commit_list *get_reachable_subset(struct commit **from, int nr_from, struct commit *c = *item; parse_commit(c); - if (c->generation < min_generation) - min_generation = c->generation; + if (commit_graph_generation(c) < min_generation) + min_generation = commit_graph_generation(c); if (!(c->object.flags & PARENT1)) { c->object.flags |= PARENT1; @@ -755,7 +755,7 @@ struct commit_list *get_reachable_subset(struct commit **from, int nr_from, parse_commit(p); - if (p->generation < min_generation) + if (commit_graph_generation(p) < min_generation) continue; if (p->object.flags & PARENT2) diff --git a/commit.c b/commit.c index b30875e66b..ed0917a2c7 100644 --- a/commit.c +++ b/commit.c @@ -339,7 +339,7 @@ struct tree *repo_get_commit_tree(struct repository *r, if (commit->maybe_tree || !commit->object.parsed) return commit->maybe_tree; - if (commit->graph_pos != COMMIT_NOT_FROM_GRAPH) + if (commit_graph_position(commit) != COMMIT_NOT_FROM_GRAPH) return get_commit_tree_in_graph(r, commit); return NULL; @@ -731,9 +731,9 @@ int compare_commits_by_gen_then_commit_date(const void *a_, const void *b_, void const struct commit *a = a_, *b = b_; /* newer commits first */ - if (a->generation < b->generation) + if (commit_graph_generation(a) < commit_graph_generation(b)) return 1; - else if (a->generation > b->generation) + else if (commit_graph_generation(a) > commit_graph_generation(b)) return -1; /* use date as a heuristic when generations are equal */ diff --git a/commit.h b/commit.h index 1b2dea5d85..e901538909 100644 --- a/commit.h +++ b/commit.h @@ -36,8 +36,6 @@ struct commit { * or get_commit_tree_oid(). */ struct tree *maybe_tree; - uint32_t graph_pos; - uint32_t generation; unsigned int index; }; diff --git a/contrib/coccinelle/commit.cocci b/contrib/coccinelle/commit.cocci index 778e4704f6..af6dd4c20c 100644 --- a/contrib/coccinelle/commit.cocci +++ b/contrib/coccinelle/commit.cocci @@ -32,3 +32,21 @@ expression c; - c->maybe_tree + repo_get_commit_tree(specify_the_right_repo_here, c) ...>} + +@@ +struct commit *c; +expression E; +@@ +( +- c->generation = E; ++ commit_graph_data_at(c)->generation = E; +| +- c->graph_pos = E; ++ commit_graph_data_at(c)->graph_pos = E; +| +- c->generation ++ commit_graph_generation(c) +| +- c->graph_pos ++ commit_graph_position(c) +) diff --git a/revision.c b/revision.c index ebb4d2a0f2..8648d7c43c 100644 --- a/revision.c +++ b/revision.c @@ -725,7 +725,7 @@ static int check_maybe_different_in_bloom_filter(struct rev_info *revs, if (!revs->repo->objects->commit_graph) return -1; - if (commit->generation == GENERATION_NUMBER_INFINITY) + if (commit_graph_generation(commit) == GENERATION_NUMBER_INFINITY) return -1; filter = get_bloom_filter(revs->repo, commit, 0); @@ -3320,7 +3320,7 @@ static void explore_to_depth(struct rev_info *revs, struct topo_walk_info *info = revs->topo_walk_info; struct commit *c; while ((c = prio_queue_peek(&info->explore_queue)) && - c->generation >= gen_cutoff) + commit_graph_generation(c) >= gen_cutoff) explore_walk_step(revs); } @@ -3336,7 +3336,7 @@ static void indegree_walk_step(struct rev_info *revs) if (parse_commit_gently(c, 1) < 0) return; - explore_to_depth(revs, c->generation); + explore_to_depth(revs, commit_graph_generation(c)); for (p = c->parents; p; p = p->next) { struct commit *parent = p->item; @@ -3360,7 +3360,7 @@ static void compute_indegrees_to_depth(struct rev_info *revs, struct topo_walk_info *info = revs->topo_walk_info; struct commit *c; while ((c = prio_queue_peek(&info->indegree_queue)) && - c->generation >= gen_cutoff) + commit_graph_generation(c) >= gen_cutoff) indegree_walk_step(revs); } @@ -3420,8 +3420,8 @@ static void init_topo_walk(struct rev_info *revs) test_flag_and_insert(&info->explore_queue, c, TOPO_WALK_EXPLORED); test_flag_and_insert(&info->indegree_queue, c, TOPO_WALK_INDEGREE); - if (c->generation < info->min_generation) - info->min_generation = c->generation; + if (commit_graph_generation(c) < info->min_generation) + info->min_generation = commit_graph_generation(c); *(indegree_slab_at(&info->indegree, c)) = 1; @@ -3479,8 +3479,8 @@ static void expand_topo_walk(struct rev_info *revs, struct commit *commit) if (parse_commit_gently(parent, 1) < 0) continue; - if (parent->generation < info->min_generation) { - info->min_generation = parent->generation; + if (commit_graph_generation(parent) < info->min_generation) { + info->min_generation = commit_graph_generation(parent); compute_indegrees_to_depth(revs, info->min_generation); } From patchwork Wed Jun 17 09:14:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Abhishek Kumar X-Patchwork-Id: 11609485 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 50B9513B1 for ; Wed, 17 Jun 2020 09:16:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2F062208B3 for ; Wed, 17 Jun 2020 09:16:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="fE2D8xN6" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726540AbgFQJQf (ORCPT ); Wed, 17 Jun 2020 05:16:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38820 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725554AbgFQJQe (ORCPT ); Wed, 17 Jun 2020 05:16:34 -0400 Received: from mail-pj1-x1044.google.com (mail-pj1-x1044.google.com [IPv6:2607:f8b0:4864:20::1044]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5443BC061573 for ; Wed, 17 Jun 2020 02:16:34 -0700 (PDT) Received: by mail-pj1-x1044.google.com with SMTP id a45so2139657pje.1 for ; Wed, 17 Jun 2020 02:16:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=r6DQq0kpT8OmkRGrs5JJITy/igOnhgeKSaIO2OwPGKA=; b=fE2D8xN6Zcec2ypkJLN8HGatxAbYUpwH2z6X33egXYD0jQk9hr7yKKsBg9SOrS9d9S Cz7fjp9IRh7JLBGBrqlcB0RU/CyMC21GzqdzyN4eZRsn8h6cB90d/kOmRlPYozPW3+St Vvqbi2XLSCpZKi64dQUN5ybMwtjJFQUuEVvadn/FD2E4IZ4ODa0kW1Aq5aV3PfqTqujB VfixU8iXHad0qTV0Wrle8KFyPTOBFgK1MQq0z/WezlLqjsQDjAMLxVywg03/CQ9Df6KF ZsibOlWx3j4GKe5NayDm/xLal1EDN9sOFJ70DV/smVZSP27Q/b0TghQEnggla3V43p0y qNog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=r6DQq0kpT8OmkRGrs5JJITy/igOnhgeKSaIO2OwPGKA=; b=M7JYYFWO31fOjuuDHQI32kk7/KLcj6vEYp5Ht6byCpBi1syvsu13VLL77dWVrUyqIX EWJ/em1HV1JByhUOLBgCJAPjxrmIx5rYaKTF4O9CGGARrLCYqlx6QLeO1aurGmImVhBV jI/PEL8tsfWY/nwYa6Zlr4rE7/JsD04fBKrRRFC0bYmzXB++aRZWf3JNAU5pZ7hRJWV4 IHFzOv7TzTjVPC2q8DZPD06fRs1ZavOlZi03X2sMaRZN+Tz+JK7oBzynCZR+32vAo5TN VeXJRy1GAc2mF1o4iOX07/Oq23RLR1LUA6bWl+sgKu15uqC/yYilZ+TOcRt4pw0pENLv TIPg== X-Gm-Message-State: AOAM533myGHVp5SjDDuvhRejcy9VI/sysJPiXr5Td3fMf6+jbndnf6Oa yvbNIpFCg4Jzeke2qlSPpUfWyz+x X-Google-Smtp-Source: ABdhPJyGG0ZEtH80YkPhIRr3yEyk4MsarWB6d3Hf26gouR2SaujlADf/+pLw951Wc2fnMuZCHUczXw== X-Received: by 2002:a17:902:b78a:: with SMTP id e10mr5829650pls.201.1592385393153; Wed, 17 Jun 2020 02:16:33 -0700 (PDT) Received: from localhost.localdomain ([2409:4064:40b:ccca:8c2b:e6b:e670:4483]) by smtp.gmail.com with ESMTPSA id r8sm17162110pgn.19.2020.06.17.02.16.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 Jun 2020 02:16:32 -0700 (PDT) From: Abhishek Kumar To: git@vger.kernel.org Cc: jnareb@gmail.com, stolee@gmail.com Subject: [GSOC Patch v4 4/4] commit-graph: minimize commit_graph_data_slab access Date: Wed, 17 Jun 2020 14:44:11 +0530 Message-Id: <20200617091411.14650-5-abhishekkumar8222@gmail.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200617091411.14650-1-abhishekkumar8222@gmail.com> References: <20200604072759.19142-1-abhishekkumar8222@gmail.com> <20200617091411.14650-1-abhishekkumar8222@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In an earlier patch, multiple struct acccesses to `graph_pos` and `generation` were auto-converted to multiple method calls. Since the values are fixed and commit-slab access costly, we would be better off with storing the values as a local variable and reusing it. Signed-off-by: Abhishek Kumar --- bloom.c | 5 +++-- commit-graph.c | 40 ++++++++++++++++++++++------------ commit-reach.c | 59 ++++++++++++++++++++++++++++++++------------------ commit.c | 6 +++-- revision.c | 12 ++++++---- 5 files changed, 79 insertions(+), 43 deletions(-) diff --git a/bloom.c b/bloom.c index 3062aafaba..6a7f2f2bdc 100644 --- a/bloom.c +++ b/bloom.c @@ -33,15 +33,16 @@ static int load_bloom_filter_from_graph(struct commit_graph *g, struct commit *c) { uint32_t lex_pos, start_index, end_index; + uint32_t graph_pos = commit_graph_position(c); - while (commit_graph_position(c) < g->num_commits_in_base) + while (graph_pos < g->num_commits_in_base) g = g->base_graph; /* The commit graph commit 'c' lives in doesn't carry bloom filters. */ if (!g->chunk_bloom_indexes) return 0; - lex_pos = commit_graph_position(c) - g->num_commits_in_base; + lex_pos = graph_pos - g->num_commits_in_base; end_index = get_be32(g->chunk_bloom_indexes + 4 * lex_pos); diff --git a/commit-graph.c b/commit-graph.c index 14cc7e931c..fdd1c4fa7c 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -144,10 +144,12 @@ static int commit_gen_cmp(const void *va, const void *vb) const struct commit *a = *(const struct commit **)va; const struct commit *b = *(const struct commit **)vb; + uint32_t generation_a = commit_graph_generation(a); + uint32_t generation_b = commit_graph_generation(b); /* lower generation commits first */ - if (commit_graph_generation(a) < commit_graph_generation(b)) + if (generation_a < generation_b) return -1; - else if (commit_graph_generation(a) > commit_graph_generation(b)) + else if (generation_a > generation_b) return 1; /* use date as a heuristic when generations are equal */ @@ -729,6 +731,7 @@ static struct commit_list **insert_parent_or_die(struct repository *r, static void fill_commit_graph_info(struct commit *item, struct commit_graph *g, uint32_t pos) { const unsigned char *commit_data; + struct commit_graph_data *graph_data; uint32_t lex_index; while (pos < g->num_commits_in_base) @@ -736,8 +739,10 @@ static void fill_commit_graph_info(struct commit *item, struct commit_graph *g, lex_index = pos - g->num_commits_in_base; commit_data = g->chunk_commit_data + GRAPH_DATA_WIDTH * lex_index; - commit_graph_data_at(item)->graph_pos = pos; - commit_graph_data_at(item)->generation = get_be32(commit_data + g->hash_len + 8) >> 2; + + graph_data = commit_graph_data_at(item); + graph_data->graph_pos = pos; + graph_data->generation = get_be32(commit_data + g->hash_len + 8) >> 2; } static inline void set_commit_tree(struct commit *c, struct tree *t) @@ -753,6 +758,7 @@ static int fill_commit_in_graph(struct repository *r, uint32_t *parent_data_ptr; uint64_t date_low, date_high; struct commit_list **pptr; + struct commit_graph_data *graph_data; const unsigned char *commit_data; uint32_t lex_index; @@ -766,7 +772,8 @@ static int fill_commit_in_graph(struct repository *r, * Store the "full" position, but then use the * "local" position for the rest of the calculation. */ - commit_graph_data_at(item)->graph_pos = pos; + graph_data = commit_graph_data_at(item); + graph_data->graph_pos = pos; lex_index = pos - g->num_commits_in_base; commit_data = g->chunk_commit_data + (g->hash_len + 16) * lex_index; @@ -779,7 +786,7 @@ static int fill_commit_in_graph(struct repository *r, date_low = get_be32(commit_data + g->hash_len + 12); item->date = (timestamp_t)((date_high << 32) | date_low); - commit_graph_data_at(item)->generation = get_be32(commit_data + g->hash_len + 8) >> 2; + graph_data->generation = get_be32(commit_data + g->hash_len + 8) >> 2; pptr = &item->parents; @@ -811,8 +818,9 @@ static int fill_commit_in_graph(struct repository *r, static int find_commit_in_graph(struct commit *item, struct commit_graph *g, uint32_t *pos) { - if (commit_graph_position(item) != COMMIT_NOT_FROM_GRAPH) { - *pos = commit_graph_position(item); + uint32_t graph_pos = commit_graph_position(item); + if (graph_pos != COMMIT_NOT_FROM_GRAPH) { + *pos = graph_pos; return 1; } else { struct commit_graph *cur_g = g; @@ -867,12 +875,13 @@ static struct tree *load_tree_for_commit(struct repository *r, { struct object_id oid; const unsigned char *commit_data; + uint32_t graph_pos = commit_graph_position(c); - while (commit_graph_position(c) < g->num_commits_in_base) + while (graph_pos < g->num_commits_in_base) g = g->base_graph; commit_data = g->chunk_commit_data + - GRAPH_DATA_WIDTH * (commit_graph_position(c) - g->num_commits_in_base); + GRAPH_DATA_WIDTH * (graph_pos - g->num_commits_in_base); hashcpy(oid.hash, commit_data); set_commit_tree(c, lookup_tree(r, &oid)); @@ -2299,6 +2308,7 @@ int verify_commit_graph(struct repository *r, struct commit_graph *g, int flags) struct commit *graph_commit, *odb_commit; struct commit_list *graph_parents, *odb_parents; uint32_t max_generation = 0; + uint32_t generation; display_progress(progress, i + 1); hashcpy(cur_oid.hash, g->chunk_oid_lookup + g->hash_len * i); @@ -2337,8 +2347,9 @@ int verify_commit_graph(struct repository *r, struct commit_graph *g, int flags) oid_to_hex(&graph_parents->item->object.oid), oid_to_hex(&odb_parents->item->object.oid)); - if (commit_graph_generation(graph_parents->item) > max_generation) - max_generation = commit_graph_generation(graph_parents->item); + generation = commit_graph_generation(graph_parents->item); + if (generation > max_generation) + max_generation = generation; graph_parents = graph_parents->next; odb_parents = odb_parents->next; @@ -2368,10 +2379,11 @@ int verify_commit_graph(struct repository *r, struct commit_graph *g, int flags) if (max_generation == GENERATION_NUMBER_MAX) max_generation--; - if (commit_graph_generation(graph_commit) != max_generation + 1) + generation = commit_graph_generation(graph_commit); + if (generation != max_generation + 1) graph_report(_("commit-graph generation for commit %s is %u != %u"), oid_to_hex(&cur_oid), - commit_graph_generation(graph_commit), + generation, max_generation + 1); if (graph_commit->date != odb_commit->date) diff --git a/commit-reach.c b/commit-reach.c index 3b2f863f5f..f5e5c0a32b 100644 --- a/commit-reach.c +++ b/commit-reach.c @@ -58,14 +58,15 @@ static struct commit_list *paint_down_to_common(struct repository *r, struct commit *commit = prio_queue_get(&queue); struct commit_list *parents; int flags; + uint32_t generation = commit_graph_generation(commit); - if (min_generation && commit_graph_generation(commit) > last_gen) + if (min_generation && generation > last_gen) BUG("bad generation skip %8x > %8x at %s", - commit_graph_generation(commit), last_gen, + generation, last_gen, oid_to_hex(&commit->object.oid)); - last_gen = commit_graph_generation(commit); + last_gen = generation; - if (commit_graph_generation(commit) < min_generation) + if (generation < min_generation) break; flags = commit->object.flags & (PARENT1 | PARENT2 | STALE); @@ -181,13 +182,15 @@ static int remove_redundant(struct repository *r, struct commit **array, int cnt if (redundant[i]) continue; for (j = filled = 0; j < cnt; j++) { + uint32_t curr_generation; if (i == j || redundant[j]) continue; filled_index[filled] = j; work[filled++] = array[j]; - if (commit_graph_generation(array[j]) < min_generation) - min_generation = commit_graph_generation(array[j]); + curr_generation = commit_graph_generation(array[j]); + if (curr_generation < min_generation) + min_generation = curr_generation; } common = paint_down_to_common(r, array[i], filled, work, min_generation); @@ -316,23 +319,26 @@ int repo_in_merge_bases_many(struct repository *r, struct commit *commit, { struct commit_list *bases; int ret = 0, i; - uint32_t min_generation = GENERATION_NUMBER_INFINITY; + uint32_t generation, min_generation = GENERATION_NUMBER_INFINITY; if (repo_parse_commit(r, commit)) return ret; for (i = 0; i < nr_reference; i++) { if (repo_parse_commit(r, reference[i])) return ret; - if (commit_graph_generation(reference[i]) < min_generation) - min_generation = commit_graph_generation(reference[i]); + + generation = commit_graph_generation(reference[i]); + if (generation < min_generation) + min_generation = generation; } - if (commit_graph_generation(commit) > min_generation) + generation = commit_graph_generation(commit); + if (generation > min_generation) return ret; bases = paint_down_to_common(r, commit, nr_reference, reference, - commit_graph_generation(commit)); + generation); if (commit->object.flags & PARENT2) ret = 1; clear_commit_marks(commit, all_flags); @@ -490,10 +496,12 @@ static enum contains_result contains_tag_algo(struct commit *candidate, const struct commit_list *p; for (p = want; p; p = p->next) { + uint32_t generation; struct commit *c = p->item; load_commit_graph_info(the_repository, c); - if (commit_graph_generation(c) < cutoff) - cutoff = commit_graph_generation(c); + generation = commit_graph_generation(c); + if (generation < cutoff) + cutoff = generation; } result = contains_test(candidate, want, cache, cutoff); @@ -544,9 +552,12 @@ static int compare_commits_by_gen(const void *_a, const void *_b) const struct commit *a = *(const struct commit * const *)_a; const struct commit *b = *(const struct commit * const *)_b; - if (commit_graph_generation(a) < commit_graph_generation(b)) + uint32_t generation_a = commit_graph_generation(a); + uint32_t generation_b = commit_graph_generation(b); + + if (generation_a < generation_b) return -1; - if (commit_graph_generation(a) > commit_graph_generation(b)) + if (generation_a > generation_b) return 1; return 0; } @@ -662,11 +673,13 @@ int can_all_from_reach(struct commit_list *from, struct commit_list *to, add_object_array(&from_iter->item->object, NULL, &from_objs); if (!parse_commit(from_iter->item)) { + uint32_t generation; if (from_iter->item->date < min_commit_date) min_commit_date = from_iter->item->date; - if (commit_graph_generation(from_iter->item) < min_generation) - min_generation = commit_graph_generation(from_iter->item); + generation = commit_graph_generation(from_iter->item); + if (generation < min_generation) + min_generation = generation; } from_iter = from_iter->next; @@ -674,11 +687,13 @@ int can_all_from_reach(struct commit_list *from, struct commit_list *to, while (to_iter) { if (!parse_commit(to_iter->item)) { + uint32_t generation; if (to_iter->item->date < min_commit_date) min_commit_date = to_iter->item->date; - if (commit_graph_generation(to_iter->item) < min_generation) - min_generation = commit_graph_generation(to_iter->item); + generation = commit_graph_generation(to_iter->item); + if (generation < min_generation) + min_generation = generation; } to_iter->item->object.flags |= PARENT2; @@ -718,11 +733,13 @@ struct commit_list *get_reachable_subset(struct commit **from, int nr_from, struct prio_queue queue = { compare_commits_by_gen_then_commit_date }; for (item = to; item < to_last; item++) { + uint32_t generation; struct commit *c = *item; parse_commit(c); - if (commit_graph_generation(c) < min_generation) - min_generation = commit_graph_generation(c); + generation = commit_graph_generation(c); + if (generation < min_generation) + min_generation = generation; if (!(c->object.flags & PARENT1)) { c->object.flags |= PARENT1; diff --git a/commit.c b/commit.c index ed0917a2c7..43d29a800d 100644 --- a/commit.c +++ b/commit.c @@ -729,11 +729,13 @@ int compare_commits_by_author_date(const void *a_, const void *b_, int compare_commits_by_gen_then_commit_date(const void *a_, const void *b_, void *unused) { const struct commit *a = a_, *b = b_; + const uint32_t generation_a = commit_graph_generation(a), + generation_b = commit_graph_generation(b); /* newer commits first */ - if (commit_graph_generation(a) < commit_graph_generation(b)) + if (generation_a < generation_b) return 1; - else if (commit_graph_generation(a) > commit_graph_generation(b)) + else if (generation_a > generation_b) return -1; /* use date as a heuristic when generations are equal */ diff --git a/revision.c b/revision.c index 8648d7c43c..32be93f404 100644 --- a/revision.c +++ b/revision.c @@ -3413,6 +3413,7 @@ static void init_topo_walk(struct rev_info *revs) info->min_generation = GENERATION_NUMBER_INFINITY; for (list = revs->commits; list; list = list->next) { struct commit *c = list->item; + uint32_t generation; if (parse_commit_gently(c, 1)) continue; @@ -3420,8 +3421,9 @@ static void init_topo_walk(struct rev_info *revs) test_flag_and_insert(&info->explore_queue, c, TOPO_WALK_EXPLORED); test_flag_and_insert(&info->indegree_queue, c, TOPO_WALK_INDEGREE); - if (commit_graph_generation(c) < info->min_generation) - info->min_generation = commit_graph_generation(c); + generation = commit_graph_generation(c); + if (generation < info->min_generation) + info->min_generation = generation; *(indegree_slab_at(&info->indegree, c)) = 1; @@ -3472,6 +3474,7 @@ static void expand_topo_walk(struct rev_info *revs, struct commit *commit) for (p = commit->parents; p; p = p->next) { struct commit *parent = p->item; int *pi; + uint32_t generation; if (parent->object.flags & UNINTERESTING) continue; @@ -3479,8 +3482,9 @@ static void expand_topo_walk(struct rev_info *revs, struct commit *commit) if (parse_commit_gently(parent, 1) < 0) continue; - if (commit_graph_generation(parent) < info->min_generation) { - info->min_generation = commit_graph_generation(parent); + generation = commit_graph_generation(parent); + if (generation < info->min_generation) { + info->min_generation = generation; compute_indegrees_to_depth(revs, info->min_generation); }