From patchwork Fri Aug 23 12:43:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Han Young X-Patchwork-Id: 13775062 Received: from mail-ot1-f46.google.com (mail-ot1-f46.google.com [209.85.210.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AA8D11865E6 for ; Fri, 23 Aug 2024 12:44:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.46 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724417048; cv=none; b=LxvqigVAvOaj+REOlMSKkl6Z2HdSTQpyyqCVsvqCDTcuY3b117E7m9qvmLURjtGDaJRhQgf6P3P5W1XUPYNRxfqa0tDtrZs0zEH69JI8Jpj0L1LCQn+Yc/ZyDuzFdyoHL5/QhMFL65DeBpn5DjjPEkw6Pky/UimXNeHA6tprcj8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724417048; c=relaxed/simple; bh=5X2gGPNSjcvh5L6n2+4Tq6OHPKuZZLW3d1bN6RRzH2I=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Q4Ab5ZRBExUyykXGYbzSKWoHkTKiznixzBjqd710glQPUOrzJXnH1GLi5tNlun/Br+CCiBUTZK6QSVd+Z/krOvXe6LEyMn+P/0MZpSA3XRTxDIduuySZP7oCL27HIMMCe/jNKNB38s7pMTGBj3n6145mIUhh6dQE25go0bCiisQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=ZTCrWl67; arc=none smtp.client-ip=209.85.210.46 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="ZTCrWl67" Received: by mail-ot1-f46.google.com with SMTP id 46e09a7af769-7093472356dso1427789a34.0 for ; Fri, 23 Aug 2024 05:44:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1724417045; x=1725021845; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=uKDvxLwxBSWoHgEM9OE35/1hK70atfgbWuJ7BPwNOEU=; b=ZTCrWl67K6pSzFSYlXEZRK3jVOPkoCaJhGo8solhZVlvNDcWNDdWtPAxy/Jpv4XSAg I5T1gUvOeIvEjvFlBkIRRdfksSa1dvlSv0oiEtwDtysjK86NMkcFfpc049pKQu2anUjH Lz3Xcdb6ILpB+tplqtWFmPAcoTrLLVDhq1Ba6OJhkRS1ztMH00XZr05X8aaKSZVnX8Kt PtJ223XHHe/AYP+L5Thy392CP/8oCQJ6KYqjdyrtyAzqNa57XyqHk8FHtE6y88JkPY10 viBIWKjvM0AvMGY+o6zVfs1J2c7cww7EL70JhPy8+NekEh+k9E1bNyWOG1cnsvXkNlq3 iIxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724417045; x=1725021845; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=uKDvxLwxBSWoHgEM9OE35/1hK70atfgbWuJ7BPwNOEU=; b=WDc5Ehol0nV8dZ+/QELnjO9ThiQF0b6gUsHJl7Jlju5fZ49h62q7p/GLsYlcDW1HXy DJbt43pVcvA8Qbdtu1HsVLQWNEGzDymN4KkEUPRfzQtrn1VN6mgo9X2+NQ4dP1+3gvp1 L4aob8rKrkkl40pD+49F07MpVpPvRVO3s+/b6Gx5lNub93XljBrx2VwRxW+B+6gkZtsW GniCUKT/CWBvCz1M5wu7LNIVZa8eW4hCUy3x0SkGsr/wJFY8yF4SFbVfpPzScibzO4s+ /8H+QylqvILOPqvBdyweLIOIsbQMJQRcztiGMfD5bFDf7o8tB5/WHRwYn/1FNXIABFDf CgXA== X-Gm-Message-State: AOJu0Yxo7MnNnqG7GEOWiXNlJASZonpwvPskCd0uJCpUsuOezk17Vqh1 KifGpsxsl4M51MhIEpzXFcmKJ1uvA5SZWsqeLTrI255lLNG1i2rC/lmxqIyXSypSx/TSmrcUN7C 9 X-Google-Smtp-Source: AGHT+IH/IK0gt4voxFO7nYh+Z3wdXp2/FenUKr9X3GzdO5xFLNhfr9BsaILTfTGLWP7+Bx1X2WgyHw== X-Received: by 2002:a05:6830:661a:b0:703:61bc:e3e4 with SMTP id 46e09a7af769-70e0ec1c44dmr2357030a34.33.1724417045357; Fri, 23 Aug 2024 05:44:05 -0700 (PDT) Received: from localhost.localdomain ([139.177.225.236]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7cd9ad56c5esm3068023a12.75.2024.08.23.05.44.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Aug 2024 05:44:04 -0700 (PDT) From: Han Young To: git@vger.kernel.org Cc: Han Young Subject: [WIP v2 1/4] packfile: split promisor objects oidset into two Date: Fri, 23 Aug 2024 20:43:51 +0800 Message-ID: <20240823124354.12982-2-hanyang.tony@bytedance.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240823124354.12982-1-hanyang.tony@bytedance.com> References: <20240802073143.56731-1-hanyang.tony@bytedance.com> <20240823124354.12982-1-hanyang.tony@bytedance.com> Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 split promisor objects oidset into two, one is objects in promisor packfile, and other set is objects referenced in promisor packfile. This enable us to check if an object is in promisor packfile. --- packfile.c | 25 ++++++++++++++++--------- packfile.h | 7 ++++++- 2 files changed, 22 insertions(+), 10 deletions(-) diff --git a/packfile.c b/packfile.c index cf12a539ea..1cf69a17be 100644 --- a/packfile.c +++ b/packfile.c @@ -2234,12 +2234,17 @@ int for_each_packed_object(each_packed_object_fn cb, void *data, return r ? r : pack_errors; } +struct promisor_objects { + struct oidset promisor_pack_objects; + struct oidset promisor_pack_referenced_objects; +}; + static int add_promisor_object(const struct object_id *oid, struct packed_git *pack UNUSED, uint32_t pos UNUSED, void *set_) { - struct oidset *set = set_; + struct promisor_objects *set = set_; struct object *obj; int we_parsed_object; @@ -2254,7 +2259,7 @@ static int add_promisor_object(const struct object_id *oid, if (!obj) return 1; - oidset_insert(set, oid); + oidset_insert(&set->promisor_pack_objects, oid); /* * If this is a tree, commit, or tag, the objects it refers @@ -2272,26 +2277,26 @@ static int add_promisor_object(const struct object_id *oid, */ return 0; while (tree_entry_gently(&desc, &entry)) - oidset_insert(set, &entry.oid); + oidset_insert(&set->promisor_pack_referenced_objects, &entry.oid); if (we_parsed_object) free_tree_buffer(tree); } else if (obj->type == OBJ_COMMIT) { struct commit *commit = (struct commit *) obj; struct commit_list *parents = commit->parents; - oidset_insert(set, get_commit_tree_oid(commit)); + oidset_insert(&set->promisor_pack_referenced_objects, get_commit_tree_oid(commit)); for (; parents; parents = parents->next) - oidset_insert(set, &parents->item->object.oid); + oidset_insert(&set->promisor_pack_referenced_objects, &parents->item->object.oid); } else if (obj->type == OBJ_TAG) { struct tag *tag = (struct tag *) obj; - oidset_insert(set, get_tagged_oid(tag)); + oidset_insert(&set->promisor_pack_referenced_objects, get_tagged_oid(tag)); } return 0; } -int is_promisor_object(const struct object_id *oid) +int is_in_promisor_pack(const struct object_id *oid, int referenced) { - static struct oidset promisor_objects; + static struct promisor_objects promisor_objects; static int promisor_objects_prepared; if (!promisor_objects_prepared) { @@ -2303,5 +2308,7 @@ int is_promisor_object(const struct object_id *oid) } promisor_objects_prepared = 1; } - return oidset_contains(&promisor_objects, oid); + + return oidset_contains(&promisor_objects.promisor_pack_objects, oid) || + (referenced && oidset_contains(&promisor_objects.promisor_pack_referenced_objects, oid)); } diff --git a/packfile.h b/packfile.h index 0f78658229..13a349e223 100644 --- a/packfile.h +++ b/packfile.h @@ -195,11 +195,16 @@ int has_object_kept_pack(const struct object_id *oid, unsigned flags); int has_pack_index(const unsigned char *sha1); +int is_in_promisor_pack(const struct object_id *oid, int referenced); + /* * Return 1 if an object in a promisor packfile is or refers to the given * object, 0 otherwise. */ -int is_promisor_object(const struct object_id *oid); +static inline int is_promisor_object(const struct object_id *oid) +{ + return is_in_promisor_pack(oid, 1); +} /* * Expose a function for fuzz testing. From patchwork Fri Aug 23 12:43:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Han Young X-Patchwork-Id: 13775063 Received: from mail-ot1-f52.google.com (mail-ot1-f52.google.com [209.85.210.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7E0041865FF for ; Fri, 23 Aug 2024 12:44:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724417051; cv=none; b=I2sW97YOK7tTVoGQMQ2lcRt9za7P2ifbn/0dxMQa6+tfsWEmLHmVS88oMBFtoB2eQb6TJ9iJSo1JJD2efuEvguDslW9DTGyWBbfQ62YPrb73aQk9MbFCBUSYOQdSCHxjmHZcWHuMa7gF5atJk2S49uQ/h1JotM2FBFukrkuZ3sY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724417051; c=relaxed/simple; bh=FCzEY//xf+BuvkOHSIO2PRce52V9VJzjIAVmO9+r4/A=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=rWjApxp1M9fgpI07XP+yY6xXTIXdA0hEOdCW3pRh5ddlX6w+TAoBMBdYMxHUXVrIdve37Idg6uqJwMOy+RW2Jp1/xYQOwTAB1ohWR7aRvY8Skx2CxH3MphjM+ONew3h93gZC8tjXPwZAx47uOqR5RGI73MI3mFjFdSSZMhSKsUE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=fY950a1h; arc=none smtp.client-ip=209.85.210.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="fY950a1h" Received: by mail-ot1-f52.google.com with SMTP id 46e09a7af769-70949118d26so1715850a34.0 for ; Fri, 23 Aug 2024 05:44:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1724417048; x=1725021848; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=xluoJfrGQuCVGNWsP4HfqZmlgMsgKyVuqdL9urqyK5U=; b=fY950a1hKg4pCte9CH2LQFXv5pwn8UGDBahlcuik0ZNqvoO4eAU6m1SEJg81Nkr6Mr mB6fdcECk+ZLzKHm/TdOw35pmUp+VXvjIdL7AHN6hgoAf5nna/pa9LHuZwVNMnlQGYNu NlK6iMO0TJpkwQzwkLuBRiZrZLB2Zo886tBXZY2uAFHi7EveOzGYIi4Fb4VTws5hC0t5 Wl32o3CrE+Vgv2NahJkra8taSs12FdNF8a9u90KKDT3/3bsIrTFg4NdweOrDTaMF0/71 z8MKEYp+asuqosr36ot4g1XjE4gZas/DtPQ5zF9i1zhs7ztpqs+PSAYrxZpwRVn9c8ub RS+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724417048; x=1725021848; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xluoJfrGQuCVGNWsP4HfqZmlgMsgKyVuqdL9urqyK5U=; b=b8LDnTFfsTEJOTngvNf5v2PLMfB4M4g10ye6TCWZyXI07LfRW4OcDR6Zw7jrslwWb8 3Q5RbtpfUVo/C9RhXA7a+srzvZVFmtvbITIFXRG5bOltFWUiuRPcPhrs7gSgMQSWFR+U Q3nXHdjJY3TTipceXUbr8+QwMmgOAE4FL5zA8T3SkDXq8WLz1i7jBbd+c4jEhYI2QfJF fMl1XdJxFHNEdqPfV1ZBi1FRGzxD/2vlRYoF7OJ//TpkMRjBwtUaSLjbCnpSCWkvwxXy XNw9JugoN3WNQB6kBMaiVma+zOZyOpirYlmHzayb0zZbjFOZO/ulXBlFKzEYcP718JZW qVnA== X-Gm-Message-State: AOJu0YxS2xZmmvg1hT40dgVDIFH75X/eHI8gBAmCm2xYh1ymWRJfIRf6 C7A2N94dS5e1BVy2F3sknf/y+/3JL26a/E/XWn0IPOFGEc39c8ccnU7pnyW7WMGAZ5TeoZ8aYJi / X-Google-Smtp-Source: AGHT+IFD3doH1CNCudkkoxff2HgItgP9Tkbe7Ne8KsL8SrFfxcwE9mwjNEhprXR2rM2p2CENIhiMXg== X-Received: by 2002:a05:6808:3988:b0:3d5:64be:890c with SMTP id 5614622812f47-3de2a872691mr2493624b6e.17.1724417048125; Fri, 23 Aug 2024 05:44:08 -0700 (PDT) Received: from localhost.localdomain ([139.177.225.236]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7cd9ad56c5esm3068023a12.75.2024.08.23.05.44.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Aug 2024 05:44:07 -0700 (PDT) From: Han Young To: git@vger.kernel.org Cc: Han Young Subject: [WIP v2 2/4] revision: add exclude-promisor-pack-objects option Date: Fri, 23 Aug 2024 20:43:52 +0800 Message-ID: <20240823124354.12982-3-hanyang.tony@bytedance.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240823124354.12982-1-hanyang.tony@bytedance.com> References: <20240802073143.56731-1-hanyang.tony@bytedance.com> <20240823124354.12982-1-hanyang.tony@bytedance.com> Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 add --exclude-promisor-pack-objects option to revision walk, this option will be used by git repack in following commits. Unlike --exclude-promisor-objects, which exclude promisor objects, --exclude-promisor-pack-objects only excludes objects in promisor packfile, objects referenced by objects in promisor pack are not excluded. --- revision.c | 13 ++++++++++++- revision.h | 3 ++- 2 files changed, 14 insertions(+), 2 deletions(-) diff --git a/revision.c b/revision.c index 6b33bd814f..7bb03a84c2 100644 --- a/revision.c +++ b/revision.c @@ -2701,6 +2701,11 @@ static int handle_revision_opt(struct rev_info *revs, int argc, const char **arg if (fetch_if_missing) BUG("exclude_promisor_objects can only be used when fetch_if_missing is 0"); revs->exclude_promisor_objects = 1; + } else if (opt && opt->allow_exclude_promisor_objects && + !strcmp(arg, "--exclude-promisor-pack-objects")) { + if (fetch_if_missing) + BUG("exclude_promisor_pack_objects can only be used when fetch_if_missing is 0"); + revs->exclude_promisor_pack_objects = 1; } else { int opts = diff_opt_parse(&revs->diffopt, argv, argc, revs->prefix); if (!opts) @@ -3908,7 +3913,7 @@ int prepare_revision_walk(struct rev_info *revs) (revs->limited && limiting_can_increase_treesame(revs))) revs->treesame.name = "treesame"; - if (revs->exclude_promisor_objects) { + if (revs->exclude_promisor_objects || revs->exclude_promisor_pack_objects) { for_each_packed_object(mark_uninteresting, revs, FOR_EACH_OBJECT_PROMISOR_ONLY); } @@ -4275,6 +4280,12 @@ static struct commit *get_revision_1(struct rev_info *revs) if (!commit) return NULL; + if (revs->exclude_promisor_objects && is_promisor_object(&commit->object.oid)) + continue; + + if (revs->exclude_promisor_pack_objects && is_in_promisor_pack(&commit->object.oid, 0)) + continue; + if (revs->reflog_info) commit->object.flags &= ~(ADDED | SEEN | SHOWN); diff --git a/revision.h b/revision.h index 0e470d1df1..6219c35c45 100644 --- a/revision.h +++ b/revision.h @@ -229,7 +229,8 @@ struct rev_info { do_not_die_on_missing_objects:1, /* for internal use only */ - exclude_promisor_objects:1; + exclude_promisor_objects:1, + exclude_promisor_pack_objects:1; /* Diff flags */ unsigned int diff:1, From patchwork Fri Aug 23 12:43:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Han Young X-Patchwork-Id: 13775064 Received: from mail-ot1-f54.google.com (mail-ot1-f54.google.com [209.85.210.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0FF961865EE for ; Fri, 23 Aug 2024 12:44:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.54 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724417053; cv=none; b=S3p35nEyHmyt1tT0BxTEC+stEyrCOfGqmcMUqe3JAkQG0jeyMPMSyT7j8yhRX290h6mDwqyK5m+TeHz25xL3U3qY9VzQgyZI71ODmrFVHlAy0NPwQ5bNO6V7kY+E7Ij2xK+t9gNMfQxihz0ILUrqSVOWHJ1iDB5CW4zZrKi+M+g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724417053; c=relaxed/simple; bh=pShdfL5DyaDkyb2qj+zt00vbZTuiAEvYjUqDw6kFjXc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TE7VDnMZLC5/Qnm517F4WRIM/jZDwsEATcAJ52fNf5bMeb+PfMzzwo8gxCQHICBoSU+BIKP2KQr4cxWGcmX7iwpz7JhT+/ouoCp2njEr4STlsh+OM/JvqPzSlazhLD0jtzN1hUCqry/A34IrdhWxu3aaGrSH19BSjz50LiIt4ic= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=GBXPSw4v; arc=none smtp.client-ip=209.85.210.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="GBXPSw4v" Received: by mail-ot1-f54.google.com with SMTP id 46e09a7af769-709485aca4bso1335601a34.1 for ; Fri, 23 Aug 2024 05:44:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1724417051; x=1725021851; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7Zbu7mH6e7/7IFLivkTHCmTicxEdXAGGnDBzecr0APs=; b=GBXPSw4vanDepzg5Jj7KoXf9IcUZ8diwZSpdSSDZm2Tvvg/f9I/2kCHCZ5C9jKFGNE Wp/Vf3CheOecOY28rnCmGFds0yKTND1ewlGPRrOFjyZqjF8WF26JwIzmwr+XjyXUIOr2 NcTxDO4vesfQq6aO6eRECirK/cEQ8cNnv1hCqfxH6ldSgi7hhFOrkEWxBiTTUA/YAWup AayfDLgmaJZEzkBLevjEKq/4B48hzRhPIVrzIJz8SaCvt5d3lAjFJlGe5OC8ouOxjdIj OsLvKBrJx/hY7gcRVDFCccZEND/fWzvABG3xm+n0te36lRseQbhx2dyz7HdXEG2oEyII x7cw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724417051; x=1725021851; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7Zbu7mH6e7/7IFLivkTHCmTicxEdXAGGnDBzecr0APs=; b=JjyHGCpi45sSwbh+ukkbMefc53FxXM9nuAUQeqA+cixR9KWSeqIwFs6weLWFaizcTQ iZ47Mw6ih4h/UvKi11TWnx4vYIfkQsLPhGk+r9YHsBOIntDLBe9BQqAnzj9276fzC55a 582KWunqTrT0zWAFqheKsH3+85FNWACQPk2WTyLlcDvv0UcSklVUYmPjnGzOeed2ZfLi ohf3T2MG/mw2Wyc24vTQAjn/ZGzm/wI6RfAxI3Ei0kmb9bXpYmFWzJR9O6BPLVdl/kKL m1cxgPiln0DHtzxEhzvwB8dp+DYaTp8hwB68M3rtnb2tArJAhkXcN9DpWzH8x+qfjdCh VBJQ== X-Gm-Message-State: AOJu0Yyc5ilO1/a7N5t65HzF7kmM4uBFOn01rdsqfHu63bO98Bjfz38G 8W3Gc8p38hHT/H7EF5c5jY0Zrx9WtNFyvJAH5dvDREAW5XkIsOvFU7iFVID0GlEGJJSm/w/6zYQ a X-Google-Smtp-Source: AGHT+IEo6AblqlNvZ9wYxrbGF7/c+p/ePxw39l70p/KU0BQuUkKoE+GJED339ze0CYQmL478XqV87A== X-Received: by 2002:a05:6808:1508:b0:3de:13bf:3092 with SMTP id 5614622812f47-3de2a86db99mr2228260b6e.24.1724417050755; Fri, 23 Aug 2024 05:44:10 -0700 (PDT) Received: from localhost.localdomain ([139.177.225.236]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7cd9ad56c5esm3068023a12.75.2024.08.23.05.44.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Aug 2024 05:44:10 -0700 (PDT) From: Han Young To: git@vger.kernel.org Cc: Han Young Subject: [WIP v2 3/4] revision: don't mark commit as UNINTERESTING if --exclude-promisor-objects is set Date: Fri, 23 Aug 2024 20:43:53 +0800 Message-ID: <20240823124354.12982-4-hanyang.tony@bytedance.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240823124354.12982-1-hanyang.tony@bytedance.com> References: <20240802073143.56731-1-hanyang.tony@bytedance.com> <20240823124354.12982-1-hanyang.tony@bytedance.com> Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 if commit is marked as UNINTERESTING, the bit will propagate down to its parents. This is undesirable in --exclude-promisor-objects, since a promisor objects' parents can be a normal object. --- revision.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/revision.c b/revision.c index 7bb03a84c2..02227e6a0a 100644 --- a/revision.c +++ b/revision.c @@ -3609,7 +3609,9 @@ static int mark_uninteresting(const struct object_id *oid, { struct rev_info *revs = cb; struct object *o = lookup_unknown_object(revs->repo, oid); - o->flags |= UNINTERESTING | SEEN; + if (o->type != OBJ_COMMIT) + o->flags |= UNINTERESTING | SEEN; + return 0; } From patchwork Fri Aug 23 12:43:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Han Young X-Patchwork-Id: 13775065 Received: from mail-oo1-f50.google.com (mail-oo1-f50.google.com [209.85.161.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C05C8186601 for ; Fri, 23 Aug 2024 12:44:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.161.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724417056; cv=none; b=VlM7jF1Uk+yyCxJz98u9hV2/Wz8UMGGQklFI6qkjDcFxiz/TgfwYfepbq338nk+TY8acnyI32AuCEiBsIOD0Wd4FtcU1UCpTuotQQ013ddl5qd4TLePJzOrJpgDTNCfjEYYnbH62RiFInlWyDgEJKh6j9qpSkuizoEiXgM39fls= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724417056; c=relaxed/simple; bh=ui4ilQR0SI9ItjcLqBtDUpRYAT/gFkKj+aD9xFqnA7M=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=lxDMSQqwXdq9j+Ao4iFLxG95vdBeZsPEOVswKhMnrR51naJOLJfkilyX6zk3ASw/cfPBa0/9lfQl+9XQ2pMcf4p94HTQ+8jB6YEaKEzGdUaMOLT9pumZZ2UPgX9oQhYa8zyjwOx+9SX5uB2n5DUKtpBM4Rnj8otgsnk38k4lxpc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=AdMex7jS; arc=none smtp.client-ip=209.85.161.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="AdMex7jS" Received: by mail-oo1-f50.google.com with SMTP id 006d021491bc7-5dca997d29eso1397871eaf.0 for ; Fri, 23 Aug 2024 05:44:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1724417053; x=1725021853; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=I90/UP/q6icXCkBilSfBdZt6nPg8mq0ZvwJtHoU+Nnw=; b=AdMex7jS/KDLaNcZxjQCP0KLZWZETYmauKsC/KsvJ1Xn/EQKQm+qxHpe9Sq1L0WVHW D4Ag+GI2EHnOvr4oL/w2xnLIJrCi22dSxlQcAL3bj3S75QAy6/EMNnkcjGqEvou20dkg se96lMFvx7A8tnPK1YxopJe4ub5ufEvVswROASb0/wyvFjuGezkID1cI/ghJ6+e64aJy OLqeQ5/J7Wh0E+c/X8KsUpjQ/gbtEgF2Nhj93IOsmnOkfn4BcMjB4B/DZJvbdzgg3xLi 3VxayVi5T6h/S3pSEIfChw0DMJ2M9Vvwo0/dKbB0lo/Mf4TwDF02rjxyR3rLBGEuX0d7 OqKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724417053; x=1725021853; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=I90/UP/q6icXCkBilSfBdZt6nPg8mq0ZvwJtHoU+Nnw=; b=ldCIRXJ/xRWQLn31HtRcCzc7CFWTecHZVyTBEqX2nIO8FU1gHB5Qggk1C+/4ig8hkT 0ROdMgVT5y/3MYTWgxVdmPINq8KS9d+Vuse9Yuev+ECpSeLqakSFmbq9Dq9irnvKaSky K8V6ZfDzjijvBDTsmc4xJD9tSdKpZ7YjAdSxPkqpuB4KmlXTiU2TBEDkUbhpjcBT6Hfk PGyWAWPj29WOmEHfLEJOTqpMuqz8TfF1rvh+kv9GRY4/SyfhI8Tnr2JP90b3Jb9BXjzF 1uSg4ErXVCfAlc05Q4ZilumoY/uCYBaQtG2Uey6XE2A9nyUb5odhsxWQOSUHQYc6XujP /QOA== X-Gm-Message-State: AOJu0Yz1Qfse4TsMaLti+6QH8bJyK+NQRFTV/bhOuSOYdTj0duaNUe+v LM+ZfeEx+3YYVqVPArqab+knII6izJAKPZF4+53g79IuE+30Ui80T9wCphNunygCZyUkDXoC5K/ K X-Google-Smtp-Source: AGHT+IHzLtiWZ0QXKU//n0pQtxi9nQFhv6cfoQrpLYjhMHnErJCKGx0a5/KXkRP3+VnxPfsHpyHT7g== X-Received: by 2002:a05:6808:d4e:b0:3d5:5e18:cf32 with SMTP id 5614622812f47-3de2a916b5cmr2132339b6e.48.1724417053439; Fri, 23 Aug 2024 05:44:13 -0700 (PDT) Received: from localhost.localdomain ([139.177.225.236]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7cd9ad56c5esm3068023a12.75.2024.08.23.05.44.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Aug 2024 05:44:12 -0700 (PDT) From: Han Young To: git@vger.kernel.org Cc: Han Young Subject: [WIP v2 4/4] repack: use new exclude promisor pack objects option Date: Fri, 23 Aug 2024 20:43:54 +0800 Message-ID: <20240823124354.12982-5-hanyang.tony@bytedance.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240823124354.12982-1-hanyang.tony@bytedance.com> References: <20240802073143.56731-1-hanyang.tony@bytedance.com> <20240823124354.12982-1-hanyang.tony@bytedance.com> Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 use --exclude-promisor-pack-objects to pack objects in partial clone repo. git repack will call git pack-objects twice on a partially cloned repo. The first call to pack-objects combines all the objects in promisor packfiles, and the second pack-objects command packs all reachable non-promisor objects into a normal packfile. However 'objects in promisor packfiles' plus 'non-promisor objects' does not equal 'all the reachable objects in repo', Since promisor objects also include objects referenced in promisor packfile. --exclude-promisor-pack-objects only excludes objects in promisor packfiles, this way we don't discard any reachable objects in git repack. --- builtin/pack-objects.c | 8 ++++---- builtin/repack.c | 2 +- list-objects.c | 3 ++- 3 files changed, 7 insertions(+), 6 deletions(-) diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c index c481feadbf..a2b1aaa2e0 100644 --- a/builtin/pack-objects.c +++ b/builtin/pack-objects.c @@ -238,7 +238,7 @@ static enum { } write_bitmap_index; static uint16_t write_bitmap_options = BITMAP_OPT_HASH_CACHE; -static int exclude_promisor_objects; +static int exclude_promisor_pack_objects; static int use_delta_islands; @@ -4391,7 +4391,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix) OPT_CALLBACK_F(0, "missing", NULL, N_("action"), N_("handling for missing objects"), PARSE_OPT_NONEG, option_parse_missing_action), - OPT_BOOL(0, "exclude-promisor-objects", &exclude_promisor_objects, + OPT_BOOL(0, "exclude-promisor-pack-objects", &exclude_promisor_pack_objects, N_("do not pack objects in promisor packfiles")), OPT_BOOL(0, "delta-islands", &use_delta_islands, N_("respect islands during delta compression")), @@ -4473,10 +4473,10 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix) strvec_push(&rp, "--unpacked"); } - if (exclude_promisor_objects) { + if (exclude_promisor_pack_objects) { use_internal_rev_list = 1; fetch_if_missing = 0; - strvec_push(&rp, "--exclude-promisor-objects"); + strvec_push(&rp, "--exclude-promisor-pack-objects"); } if (unpack_unreachable || keep_unreachable || pack_loose_unreachable) use_internal_rev_list = 1; diff --git a/builtin/repack.c b/builtin/repack.c index 62cfa50c50..aafe7d30ce 100644 --- a/builtin/repack.c +++ b/builtin/repack.c @@ -1289,7 +1289,7 @@ int cmd_repack(int argc, const char **argv, const char *prefix) strvec_push(&cmd.args, "--indexed-objects"); } if (repo_has_promisor_remote(the_repository)) - strvec_push(&cmd.args, "--exclude-promisor-objects"); + strvec_push(&cmd.args, "--exclude-promisor-pack-objects"); if (!write_midx) { if (write_bitmaps > 0) strvec_push(&cmd.args, "--write-bitmap-index"); diff --git a/list-objects.c b/list-objects.c index 985d008799..9b3ff0fe1d 100644 --- a/list-objects.c +++ b/list-objects.c @@ -178,7 +178,8 @@ static void process_tree(struct traversal_context *ctx, * requested. This may cause the actual filter to report * an incomplete list of missing objects. */ - if (revs->exclude_promisor_objects && + if ((revs->exclude_promisor_objects || + revs->exclude_promisor_pack_objects) && is_promisor_object(&obj->oid)) return;