From patchwork Tue Aug 24 10:37:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 12454603 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8BFEAC4338F for ; Tue, 24 Aug 2021 10:37:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6E36D61265 for ; Tue, 24 Aug 2021 10:37:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236243AbhHXKiB (ORCPT ); Tue, 24 Aug 2021 06:38:01 -0400 Received: from out4-smtp.messagingengine.com ([66.111.4.28]:42105 "EHLO out4-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236232AbhHXKh4 (ORCPT ); Tue, 24 Aug 2021 06:37:56 -0400 Received: from compute6.internal (compute6.nyi.internal [10.202.2.46]) by mailout.nyi.internal (Postfix) with ESMTP id 3826C5C0079; Tue, 24 Aug 2021 06:37:12 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute6.internal (MEProxy); Tue, 24 Aug 2021 06:37:12 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=date :from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=fm1; bh=UWWOdz6xMcmICbFqoU/c4SXkFdY tjN6tbnvPKpberVs=; b=Bsn3zcUryceQ/H9vqpmZVR5FZHuZyImL7DpXvEPG0B5 4sX+/a37bttd/2zBYu51/1IKCb4XUh7HBKS/xD73dXwLHEfcBIPciJ74+hGeZS2x btYxmZVz+ZfK5QwrzOWHTjgYKDwSjW3rRHwd7wKxroZW6XdgYN+PPbpGzt4AYek/ RFI91To5IlvSewXSuhPcwsYjURcL7OjWhEGoFPKz22dgONawT7UrgV77iOPv7b1g +Z6mvJIQbWJ1U6+ZP6WU1SOnyJhaJoPwBGcqVfQsObSluieM6hJgAyEjg0XgB6ux 36hMlm9S8O8XG0nVcb5WB5UttSXtgToIEhI0792p10A== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=UWWOdz 6xMcmICbFqoU/c4SXkFdYtjN6tbnvPKpberVs=; b=La0KxQ/pkVWrkHLcL+8/kk L1B0kvMVs085AHkUg5GFnzlgFKrnBDge/YEdJuMw2WlxmERFwqfSB8CPvcZcwNPm OG6GUEU3X/zZpnQo/l1rBBzzkXb6TwviP3vAB0BOCfhwYHuFwZQlaOX46kv1ncvd NuNzdiaMrOAdk6RmScf0E6xCOq1A5v5CvPb5DpWeBgrq7L9Ir7KOmRjnb6yb4n8w Uc43n8aIrOv6Tg4OWmDC62paPb3iAVInoPlrkfaxdiOsMl26gc2+ta/Qi92Y0PJG kOU58MiqKS5fFHWhcG/LTsFYysH4IQ1xvteRhomvCxoWu/955NdOoCPMxv7Qm5oA == X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvtddruddtjedgfedtucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvffukfhfgggtuggjsehgtderredttdejnecuhfhrohhmpefrrghtrhhi tghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtthgvrh hnpeehgfejueevjeetudehgffffeffvdejfeejiedvkeffgfekuefgheevteeufeelkeen ucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehpshesph hkshdrihhm X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 24 Aug 2021 06:37:10 -0400 (EDT) Received: from localhost (ncase [10.192.0.11]) by vm-mail.pks.im (OpenSMTPD) with ESMTPSA id 96b2a443 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Tue, 24 Aug 2021 10:37:10 +0000 (UTC) Date: Tue, 24 Aug 2021 12:37:09 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Jeff King , =?iso-8859-1?q?=C6var_Arnfj=F6r=F0?= Bjarmason , Junio C Hamano , Derrick Stolee , =?iso-8859-1?q?Ren=E9?= Scharfe Subject: [PATCH v2 4/7] fetch-pack: optimize loading of refs via commit graph Message-ID: <6311203f08219a8fa3884af5d288340a3fa5f47a.1629800774.git.ps@pks.im> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In order to negotiate a packfile, we need to dereference refs to see which commits we have in common with the remote. To do so, we first look up the object's type -- if it's a tag, we peel until we hit a non-tag object. If we hit a commit eventually, then we return that commit. In case the object ID points to a commit directly, we can avoid the initial lookup of the object type by opportunistically looking up the commit via the commit-graph, if available, which gives us a slight speed bump of about 2% in a huge repository with about 2.3M refs: Benchmark #1: HEAD~: git-fetch Time (mean ± σ): 31.634 s ± 0.258 s [User: 28.400 s, System: 5.090 s] Range (min … max): 31.280 s … 31.896 s 5 runs Benchmark #2: HEAD: git-fetch Time (mean ± σ): 31.129 s ± 0.543 s [User: 27.976 s, System: 5.056 s] Range (min … max): 30.172 s … 31.479 s 5 runs Summary 'HEAD: git-fetch' ran 1.02 ± 0.02 times faster than 'HEAD~: git-fetch' In case this fails, we fall back to the old code which peels the objects to a commit. Signed-off-by: Patrick Steinhardt --- fetch-pack.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/fetch-pack.c b/fetch-pack.c index e6ec79f81a..dc800879cb 100644 --- a/fetch-pack.c +++ b/fetch-pack.c @@ -119,6 +119,11 @@ static struct commit *deref_without_lazy_fetch(const struct object_id *oid, { enum object_type type; struct object_info info = { .typep = &type }; + struct commit *commit; + + commit = lookup_commit_in_graph(the_repository, oid); + if (commit) + return commit; while (1) { if (oid_object_info_extended(the_repository, oid, &info, @@ -139,7 +144,7 @@ static struct commit *deref_without_lazy_fetch(const struct object_id *oid, } if (type == OBJ_COMMIT) { - struct commit *commit = lookup_commit(the_repository, oid); + commit = lookup_commit(the_repository, oid); if (!commit || repo_parse_commit(the_repository, commit)) return NULL; return commit;