From patchwork Thu Jun 6 05:29:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13687827 Received: from wfout6-smtp.messagingengine.com (wfout6-smtp.messagingengine.com [64.147.123.149]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AE6DD38F9C for ; Thu, 6 Jun 2024 05:29:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=64.147.123.149 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717651786; cv=none; b=AhdQTr5otke4GMl6SDqo7VZlR6BW1w1tYYKd1buOPZ7mdUlhKncdMkX6deiIfwRjQMmqlKAsoDmfXydh7C5egK6mjfumd1zbiYcgstuWW/4OfD0r3MdsVNchHDX4urV4nlQTEWhARAAp5GTz9hivYjPmDfJGDFOM35fcqTV+WFw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717651786; c=relaxed/simple; bh=GWsPpmc/e8WlZsLw60fwdL0NJea4HGKtxXdmiTChUmM=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=IiKRudbfxeAMkQdTFBRD3xOONDvEFlVqAEAqdbUVHpBkVisg4azk0eia/1RW61yYolu1tGDrcAiz+J/NNjXWYGSorms7RWwGHpqcUPzbZJrst6luF8QLY2RfWUA3jFBuzLk8f3n5GsRWnv0YGCdlQvzqaftaFkviBgdTfhtdSA8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=fHzk787e; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=CLyoRi5M; arc=none smtp.client-ip=64.147.123.149 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="fHzk787e"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="CLyoRi5M" Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailfout.west.internal (Postfix) with ESMTP id 99B3E1C001CF; Thu, 6 Jun 2024 01:29:43 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Thu, 06 Jun 2024 01:29:44 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm1; t=1717651783; x=1717738183; bh=YBjFRcmq/9 dGqKp+ruDeWDGb9J0m4MXTI9lpZeQ36hM=; b=fHzk787eNwxUL9dl2qP+lhsiO6 N17FYf15P5sW531cyOjdTUZL6Swe4ZDHaoZK4BuWIMVeUbjiB9C4KnscXvgaXKK6 z9lbRRr2ddboJnTYw3V1an43ESGfWM6mNiJzrY12YaAyOAeuS/wr6AfIldkkMmYZ Wo+NM6xrhqzLpkq4+itnbaMyjClJAHaCYpe1PLiT8kWyPm8pBKMHm+q2CEJY3VUY T9zvVh3VKT1isVrsF+n7JVkahNmM9iykUNbEBBDqkfGIWJsVfrNI7r1dy7SECQsU +7Vz8mPzU5+y8k2Dzrr7rpa5vmNMTXVI3p/rz7JIwZKivv/Z6ohhv20VeWgQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1717651783; x=1717738183; bh=YBjFRcmq/9dGqKp+ruDeWDGb9J0m 4MXTI9lpZeQ36hM=; b=CLyoRi5MYYUCPOVGuI/ncw0DcGHh+sPfVJkvzOs5CQVe 1ybre0EWfHJgd47JYkLFN0w0rQ3oq7qHSrFo3nsEs7a+6BHF605Rbq6XUYTdYrBq oOVBSvvtUlAn7XXZ44sgBduIskEbgMNnRBJbjCp55XlVftNilQMGelWYUY2ANJvD TazS41eFuYAesl3Sv20d4+hp60sjq9/MruqNAsqGkyLs+d4MxAkym1/scSrrY74A sKgXgmRl3P7jZTJhqyRKaAvJ/SJ/fxCtP68G2tbLbtEHxzq+z7nIQcsypMhpj/VG s11dwDnR2TtGZixWjc63nzXG7TtCawholnNq2RaLvQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdeljedgleeiucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvdenucfhrhhomheprfgrthhr ihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvg hrnhepueektdevtdffveeljeetgfehheeigeekleduvdeffeeghefgledttdehjeelffet necuvehluhhsthgvrhfuihiivgepgeenucfrrghrrghmpehmrghilhhfrhhomhepphhsse hpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 6 Jun 2024 01:29:41 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id 82111a79 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Thu, 6 Jun 2024 05:29:11 +0000 (UTC) Date: Thu, 6 Jun 2024 07:29:39 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Eric Sunshine , Junio C Hamano , Ramsay Jones , Justin Tobler , Karthik Nayak , Jeff King Subject: [PATCH v5 10/12] refs: implement removal of ref storages Message-ID: <7b5fee21856d970bdf6cd423d2b354e610e78df6.1717649802.git.ps@pks.im> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: We're about to introduce logic to migrate ref storages. One part of the migration will be to delete the files that are part of the old ref storage format. We don't yet have a way to delete such data generically across ref backends though. Implement a new `delete` callback and expose it via a new `ref_storage_delete()` function. Signed-off-by: Patrick Steinhardt --- refs.c | 5 ++++ refs.h | 5 ++++ refs/files-backend.c | 62 +++++++++++++++++++++++++++++++++++++++++ refs/packed-backend.c | 15 ++++++++++ refs/refs-internal.h | 7 +++++ refs/reftable-backend.c | 52 ++++++++++++++++++++++++++++++++++ 6 files changed, 146 insertions(+) diff --git a/refs.c b/refs.c index fa3b0a82d4..31fd391214 100644 --- a/refs.c +++ b/refs.c @@ -1861,6 +1861,11 @@ int ref_store_create_on_disk(struct ref_store *refs, int flags, struct strbuf *e return refs->be->create_on_disk(refs, flags, err); } +int ref_store_remove_on_disk(struct ref_store *refs, struct strbuf *err) +{ + return refs->be->remove_on_disk(refs, err); +} + int repo_resolve_gitlink_ref(struct repository *r, const char *submodule, const char *refname, struct object_id *oid) diff --git a/refs.h b/refs.h index 50a2b3ab09..61ee7b7a15 100644 --- a/refs.h +++ b/refs.h @@ -129,6 +129,11 @@ int ref_store_create_on_disk(struct ref_store *refs, int flags, struct strbuf *e */ void ref_store_release(struct ref_store *ref_store); +/* + * Remove the ref store from disk. This deletes all associated data. + */ +int ref_store_remove_on_disk(struct ref_store *refs, struct strbuf *err); + /* * Return the peeled value of the oid currently being iterated via * for_each_ref(), etc. This is equivalent to calling: diff --git a/refs/files-backend.c b/refs/files-backend.c index de8cc83174..e663781199 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -3342,11 +3342,73 @@ static int files_ref_store_create_on_disk(struct ref_store *ref_store, return 0; } +struct remove_one_root_ref_data { + const char *gitdir; + struct strbuf *err; +}; + +static int remove_one_root_ref(const char *refname, + void *cb_data) +{ + struct remove_one_root_ref_data *data = cb_data; + struct strbuf buf = STRBUF_INIT; + int ret = 0; + + strbuf_addf(&buf, "%s/%s", data->gitdir, refname); + + ret = unlink(buf.buf); + if (ret < 0) + strbuf_addf(data->err, "could not delete %s: %s\n", + refname, strerror(errno)); + + strbuf_release(&buf); + return ret; +} + +static int files_ref_store_remove_on_disk(struct ref_store *ref_store, + struct strbuf *err) +{ + struct files_ref_store *refs = + files_downcast(ref_store, REF_STORE_WRITE, "remove"); + struct remove_one_root_ref_data data = { + .gitdir = refs->base.gitdir, + .err = err, + }; + struct strbuf sb = STRBUF_INIT; + int ret = 0; + + strbuf_addf(&sb, "%s/refs", refs->base.gitdir); + if (remove_dir_recursively(&sb, 0) < 0) { + strbuf_addf(err, "could not delete refs: %s", + strerror(errno)); + ret = -1; + } + strbuf_reset(&sb); + + strbuf_addf(&sb, "%s/logs", refs->base.gitdir); + if (remove_dir_recursively(&sb, 0) < 0) { + strbuf_addf(err, "could not delete logs: %s", + strerror(errno)); + ret = -1; + } + strbuf_reset(&sb); + + if (for_each_root_ref(refs, remove_one_root_ref, &data) < 0) + ret = -1; + + if (ref_store_remove_on_disk(refs->packed_ref_store, err) < 0) + ret = -1; + + strbuf_release(&sb); + return ret; +} + struct ref_storage_be refs_be_files = { .name = "files", .init = files_ref_store_init, .release = files_ref_store_release, .create_on_disk = files_ref_store_create_on_disk, + .remove_on_disk = files_ref_store_remove_on_disk, .transaction_prepare = files_transaction_prepare, .transaction_finish = files_transaction_finish, diff --git a/refs/packed-backend.c b/refs/packed-backend.c index 2789fd92f5..c4c1e36aa2 100644 --- a/refs/packed-backend.c +++ b/refs/packed-backend.c @@ -1,5 +1,6 @@ #include "../git-compat-util.h" #include "../config.h" +#include "../dir.h" #include "../gettext.h" #include "../hash.h" #include "../hex.h" @@ -1266,6 +1267,19 @@ static int packed_ref_store_create_on_disk(struct ref_store *ref_store UNUSED, return 0; } +static int packed_ref_store_remove_on_disk(struct ref_store *ref_store, + struct strbuf *err) +{ + struct packed_ref_store *refs = packed_downcast(ref_store, 0, "remove"); + + if (remove_path(refs->path) < 0) { + strbuf_addstr(err, "could not delete packed-refs"); + return -1; + } + + return 0; +} + /* * Write the packed refs from the current snapshot to the packed-refs * tempfile, incorporating any changes from `updates`. `updates` must @@ -1724,6 +1738,7 @@ struct ref_storage_be refs_be_packed = { .init = packed_ref_store_init, .release = packed_ref_store_release, .create_on_disk = packed_ref_store_create_on_disk, + .remove_on_disk = packed_ref_store_remove_on_disk, .transaction_prepare = packed_transaction_prepare, .transaction_finish = packed_transaction_finish, diff --git a/refs/refs-internal.h b/refs/refs-internal.h index 33749fbd83..cbcb6f9c36 100644 --- a/refs/refs-internal.h +++ b/refs/refs-internal.h @@ -517,6 +517,12 @@ typedef int ref_store_create_on_disk_fn(struct ref_store *refs, int flags, struct strbuf *err); +/* + * Remove the reference store from disk. + */ +typedef int ref_store_remove_on_disk_fn(struct ref_store *refs, + struct strbuf *err); + typedef int ref_transaction_prepare_fn(struct ref_store *refs, struct ref_transaction *transaction, struct strbuf *err); @@ -649,6 +655,7 @@ struct ref_storage_be { ref_store_init_fn *init; ref_store_release_fn *release; ref_store_create_on_disk_fn *create_on_disk; + ref_store_remove_on_disk_fn *remove_on_disk; ref_transaction_prepare_fn *transaction_prepare; ref_transaction_finish_fn *transaction_finish; diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c index bffed9257f..da6b3162f3 100644 --- a/refs/reftable-backend.c +++ b/refs/reftable-backend.c @@ -1,6 +1,7 @@ #include "../git-compat-util.h" #include "../abspath.h" #include "../chdir-notify.h" +#include "../dir.h" #include "../environment.h" #include "../gettext.h" #include "../hash.h" @@ -343,6 +344,56 @@ static int reftable_be_create_on_disk(struct ref_store *ref_store, return 0; } +static int reftable_be_remove_on_disk(struct ref_store *ref_store, + struct strbuf *err) +{ + struct reftable_ref_store *refs = + reftable_be_downcast(ref_store, REF_STORE_WRITE, "remove"); + struct strbuf sb = STRBUF_INIT; + int ret = 0; + + /* + * Release the ref store such that all stacks are closed. This is + * required so that the "tables.list" file is not open anymore, which + * would otherwise make it impossible to remove the file on Windows. + */ + reftable_be_release(ref_store); + + strbuf_addf(&sb, "%s/reftable", refs->base.gitdir); + if (remove_dir_recursively(&sb, 0) < 0) { + strbuf_addf(err, "could not delete reftables: %s", + strerror(errno)); + ret = -1; + } + strbuf_reset(&sb); + + strbuf_addf(&sb, "%s/HEAD", refs->base.gitdir); + if (unlink(sb.buf) < 0) { + strbuf_addf(err, "could not delete stub HEAD: %s", + strerror(errno)); + ret = -1; + } + strbuf_reset(&sb); + + strbuf_addf(&sb, "%s/refs/heads", refs->base.gitdir); + if (unlink(sb.buf) < 0) { + strbuf_addf(err, "could not delete stub heads: %s", + strerror(errno)); + ret = -1; + } + strbuf_reset(&sb); + + strbuf_addf(&sb, "%s/refs", refs->base.gitdir); + if (rmdir(sb.buf) < 0) { + strbuf_addf(err, "could not delete refs directory: %s", + strerror(errno)); + ret = -1; + } + + strbuf_release(&sb); + return ret; +} + struct reftable_ref_iterator { struct ref_iterator base; struct reftable_ref_store *refs; @@ -2196,6 +2247,7 @@ struct ref_storage_be refs_be_reftable = { .init = reftable_be_init, .release = reftable_be_release, .create_on_disk = reftable_be_create_on_disk, + .remove_on_disk = reftable_be_remove_on_disk, .transaction_prepare = reftable_be_transaction_prepare, .transaction_finish = reftable_be_transaction_finish,