From patchwork Fri Aug 20 10:08:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 12448823 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 076DAC432BE for ; Fri, 20 Aug 2021 10:08:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E59E16024A for ; Fri, 20 Aug 2021 10:08:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238333AbhHTKJY (ORCPT ); Fri, 20 Aug 2021 06:09:24 -0400 Received: from out3-smtp.messagingengine.com ([66.111.4.27]:46859 "EHLO out3-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238748AbhHTKJW (ORCPT ); Fri, 20 Aug 2021 06:09:22 -0400 Received: from compute6.internal (compute6.nyi.internal [10.202.2.46]) by mailout.nyi.internal (Postfix) with ESMTP id 538045C00D0; Fri, 20 Aug 2021 06:08:44 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute6.internal (MEProxy); Fri, 20 Aug 2021 06:08:44 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=date :from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=fm1; bh=SpJ+4ony9KZCPhpIoQmxliNKfev uC4OHmprV+nP9Q6Y=; b=q0QUdeVANSEJhy2aA289ms4ZjhtbJVIBEsQgtEtgp5g 7vjp3V+LwfK4q/aVp+r8Brw9n7Kvu7+9H+5xYMeJqlbZQk92L4v3IvUiWV7riWsq zi5BEWgCgGPh2q255dqR2ytteLPn04Nlj65kzHjVYOcBjpU/lObJlPoX6Cq3WEl0 a0I1MmmM3/FJHlEmsQKq8T0J66lWnOwPBb+zVEhnHJR/KMqPOjFL/J1e/8jx/TAH Fm4nnh83p/7v+6Fyv0S9zo6lHnItufnV+FN+YKZfO2lSP56iJJJaw2FfdlHryf3c c6m2nGUyBx1Zt8UeRdsvYLje9Iqzp5np68nDr/oSPTw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=SpJ+4o ny9KZCPhpIoQmxliNKfevuC4OHmprV+nP9Q6Y=; b=nWdIpjamcTPCbtbpz7C6Ze /SuNGRfm9ZCI1gj2yMIWl6aFAFQ61RdPr+FHNKK8kChDvWB6xp/I36T+xRbZV/d3 0k83WVqS83QmYkRlRsCYQ0mnCLu0wVXAJFYkcI2mmufiEJvQMr2tA+p/Oe+z0L3T nGJ/95zF4vLzJjDRtFzOso1wUEG1SAaBmV5WIgdLYuX8eJ+PRv7nhaOhX+/AtTuN 4+vxw/gejxTcQCS29vg2T9nYIwurlCHs4bk2pmhwPxy+pTi9NQwVhNQx6e5UUpei A8HEiKgRv6XIE6w8M6S1Vft4O9AkU+gG8TUMSGv1PtJwv0WKW1yezpByNchoHUWA == X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvtddrleelgddvfecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhepfffhvffukfhfgggtuggjsehgtderre dttdejnecuhfhrohhmpefrrghtrhhitghkucfuthgvihhnhhgrrhguthcuoehpshesphhk shdrihhmqeenucggtffrrghtthgvrhhnpeehgfejueevjeetudehgffffeffvdejfeejie dvkeffgfekuefgheevteeufeelkeenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgr mhepmhgrihhlfhhrohhmpehpshesphhkshdrihhm X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 20 Aug 2021 06:08:43 -0400 (EDT) Received: from localhost (ncase [10.192.0.11]) by vm-mail.pks.im (OpenSMTPD) with ESMTPSA id a8ef8c86 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Fri, 20 Aug 2021 10:08:42 +0000 (UTC) Date: Fri, 20 Aug 2021 12:08:41 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Jeff King , =?iso-8859-1?q?=C6var_Arnfj=F6r=F0?= Bjarmason , Junio C Hamano Subject: [PATCH 4/6] fetch-pack: optimize loading of refs via commit graph Message-ID: <67917af7ceeefe41ae0f6edf69cd61e2ee8c0ea3.1629452412.git.ps@pks.im> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In order to negotiate a packfile, we need to dereference refs to see which commits we have in common with the remote. To do so, we first look up the object's type -- if it's a tag, we peel until we hit a non-tag object. If we hit a commit eventually, then we return that commit. In case the object ID points to a commit directly, we can avoid the initial lookup of the object type by opportunistically looking up the commit via the commit-graph, if available, which gives us a slight speed bump of about 2% in a huge repository with about 2.3M refs: Benchmark #1: HEAD~: git-fetch Time (mean ± σ): 31.634 s ± 0.258 s [User: 28.400 s, System: 5.090 s] Range (min … max): 31.280 s … 31.896 s 5 runs Benchmark #2: HEAD: git-fetch Time (mean ± σ): 31.129 s ± 0.543 s [User: 27.976 s, System: 5.056 s] Range (min … max): 30.172 s … 31.479 s 5 runs Summary 'HEAD: git-fetch' ran 1.02 ± 0.02 times faster than 'HEAD~: git-fetch' In case this fails, we fall back to the old code which peels the objects to a commit. --- fetch-pack.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/fetch-pack.c b/fetch-pack.c index 1a6242cd71..c57faf278f 100644 --- a/fetch-pack.c +++ b/fetch-pack.c @@ -119,6 +119,11 @@ static struct commit *deref_without_lazy_fetch(const struct object_id *oid, { enum object_type type; struct object_info info = { .typep = &type }; + struct commit *commit; + + commit = lookup_commit_in_graph(the_repository, oid); + if (commit) + return commit; while (1) { if (oid_object_info_extended(the_repository, oid, &info, @@ -139,7 +144,7 @@ static struct commit *deref_without_lazy_fetch(const struct object_id *oid, } if (type == OBJ_COMMIT) { - struct commit *commit = lookup_commit(the_repository, oid); + commit = lookup_commit(the_repository, oid); if (!commit || repo_parse_commit(the_repository, commit)) return NULL; return commit;