From patchwork Mon Dec 16 16:44:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: karthik nayak X-Patchwork-Id: 13910066 Received: from mail-ej1-f50.google.com (mail-ej1-f50.google.com [209.85.218.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3606F1C9B9B for ; Mon, 16 Dec 2024 16:44:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734367480; cv=none; b=Kx9Cb3Ntl5VlX1VCUp+/cAD2ba4pFeEIjpqawmQp74p4ccMZD75d/HZvhzoO7AhISCVJqRlgfdDNvtpz3mQzjMBTol6y+exOUDIvc0pCGXwYtXW74cptWMQmEDy7hNFHa89R+A84khRNGUlrgzw/er0kqRDNSUkpUl3Ynh6wFzk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734367480; c=relaxed/simple; bh=1zQAW1gf6s8EkfhWmsjwMbxZIi5DU5Z49Kos2YDhOaQ=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=vE/5AgtDwZtDP7kJ5MT8BMnOi4poy9NbNUlWvr3sQy3IBrZQUM1p6o5FZCCusSJRTy5vrnyJnkdWtkPr1cKS4vdliNxQ1+zh93UiyR8eaF33dytG7xTN6g46O5ESeFRvJ83MNuqV24BIJqCdowpIDvCqF1lZkmjwlzgL0nGLo2w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=ex0PFIWT; arc=none smtp.client-ip=209.85.218.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ex0PFIWT" Received: by mail-ej1-f50.google.com with SMTP id a640c23a62f3a-aab925654d9so530452166b.2 for ; Mon, 16 Dec 2024 08:44:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734367476; x=1734972276; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=2yP66MpHoRJMrRT8Yqjo1czS94YPdDgttlZIZ/ZdMEA=; b=ex0PFIWTlPxXsDyleEJN/igNI2hntKubPdnOiv0v5LkKtvXjo2dBpHiMMkNdOV8dtR TZCj6xFZEx77KbXjk4OWcJ0rS7jj1jkdYkfc8TIYo/e5Rq77lmPhFt5TlBPW9FUzS4qC iva2KOXO9sP04yAGe+FkJxRiqYta6PFtVUH2mUgmHSkU6eso660nmLMYUuiTZOR2SHIo Zr9Z76vM5zxsr/ZHgpPpkli0Mc0JpSRqcR2H3k/UEuavDWzIZHLsUIqIXKrJIBl6OVnP 6+3F2ng1h+LgGaiLVVaPncDMPZ328qisgJjMTfc1EPpwl35GcTtyJvRGXEtAM9oIl0XX XHeA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734367476; x=1734972276; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2yP66MpHoRJMrRT8Yqjo1czS94YPdDgttlZIZ/ZdMEA=; b=WYTQTfM+0KzpLloIVbWDx7fGs5AWSwZ+z/BVpGoNpXO2EM0oPNzsp6qPl7InluTEfo f1x5O0/bArqQ8eVts3OUkphnNTPepaFb5yVbDcakSFBwIVODydq9qvVoPFeXEgGBjy3n f9Ui4Bn1X6GTk13ys8mEKMjTWeplXMBULNk/wlVAhQznSbZ8BdyjSIa4F+Be59FerF2c GPqnC2zUtTcsiG9zhtQ1IFPhlp4+CzHhEgDzCexF3XhOiwIxj0UwFj5cHqhLKuIv3IEy mp5whRLDG8y0jvzKJ/bR/fPKyVpKmDDdXN6t18LHLr9Kd4zHJz/aMsrS6hP4r7SJeR56 vTUQ== X-Gm-Message-State: AOJu0YwFm/J5kOuy3pzGjWvu2+8JSDYVUnl9aTGjeG0io5dPccG2Ji4o 4Es0r9D7/R7DIh9gv81uhyWQqZbPZ+hDrwqzy96lOe9lCwqLaHdnOyKLwbpM X-Gm-Gg: ASbGnctVY1rTDjZfYCFv36ZKIuoqonPrUYX3kPRGmeC2bjL52got+8ubFhZL9WjD0BW /6LK8neNKGhgrnKprY+NjuHsY64oaiYlw+YqL+NW7nykSnq4hBghOrv4aTAe30zFubFsJbJ7xO4 v8Ft33juOqibdsNDhUz9FdwaYvQnTdPSQG4GjN1mDIbIul6sE5SEhNJscGLwpIOdvP44WoaYmSv M1pmS2vYTs53T1PLf27myzPaBxjeooWt1x/b2dKDxraNUJJMpH9WEVGIqc= X-Google-Smtp-Source: AGHT+IFIku0Ar/4NE+lk0Z+DUnnqDxOEj8bsduBsQXhLIw4RLajlFQIGsm/r/O5WyAiDNsyZaMg3ow== X-Received: by 2002:a17:906:1313:b0:aab:dc3e:1c84 with SMTP id a640c23a62f3a-aabdc3e1f33mr8908166b.17.1734367475408; Mon, 16 Dec 2024 08:44:35 -0800 (PST) Received: from [10.82.4.202] ([217.110.80.4]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aab9635957fsm354834366b.109.2024.12.16.08.44.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 16 Dec 2024 08:44:35 -0800 (PST) From: Karthik Nayak Date: Mon, 16 Dec 2024 17:44:26 +0100 Subject: [PATCH v4 1/8] refs: include committer info in `ref_update` struct Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241216-320-git-refs-migrate-reflogs-v4-1-d7cd3f197453@gmail.com> References: <20241216-320-git-refs-migrate-reflogs-v4-0-d7cd3f197453@gmail.com> In-Reply-To: <20241216-320-git-refs-migrate-reflogs-v4-0-d7cd3f197453@gmail.com> To: git@vger.kernel.org Cc: Karthik Nayak , ps@pks.im, Christian Couder X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=5688; i=karthik.188@gmail.com; h=from:subject:message-id; bh=1zQAW1gf6s8EkfhWmsjwMbxZIi5DU5Z49Kos2YDhOaQ=; b=owEB7QES/pANAwAIAT7VnySORox/AcsmYgBnYFjwekNcbQ9vXgeJRSCHbamOq5/HU8q6IoxN/ 4a2DqPGcHOJAbMEAAEIAB0WIQRXzkx/Y3VxD8tlxgY+1Z8kjkaMfwUCZ2BY8AAKCRA+1Z8kjkaM f25RC/wMmo4rqDhYO7/zup3fS9QV7s7sneRsZpm/UcEo43J36SKpSNR3T7jocxYcoHogy3mI9/8 +D7TIanyODSel0cS9O7GzMYR7or+r6Vz7NpFW9Kb95/agDE1r/bOI61YBgMrIwzforUrailDIfh gIMR5xO3FtnN+t0NTeMCGmSwuDeI+pLxoQfUA/BH5JlC9onpaQC4QP5wrKrS0eVw3sPt8QTnrCT uag81hmBdd7ekA4y2QG3kk31/hNBGzUKHQ+rX93C9TUhZDCnP2VMNmDEqkw5ms51RDf9W7YmLK1 o6T3KtwME38NrmhdB8Q+TkCBe/9z0a1kag+DyLGcsJRAEq6tz/ZpibF0UwToRL6hrnJMlQYsk/8 LxrRIGlrFV+u3aGTiydNqPKowoWLapgq/Kbja5IPgsDeRqgBh8+aHMJQL+JbBBKPFt3/OhN5gcs U6/9B9TULYELia7D8qk6ezKz71+L3YXwTPP1L8bzKC9uCjV+cIgDya6ReVKZFcOPWAQAM= X-Developer-Key: i=karthik.188@gmail.com; a=openpgp; fpr=57CE4C7F6375710FCB65C6063ED59F248E468C7F The reference backends obtain the committer information from `git_committer_info(0)` when adding a reflog. The upcoming patches introduce support for migrating reflogs between the reference backends. This requires an interface to creating reflogs, including custom committer information. Add a new field `committer_info` to the `ref_update` struct, which is then used by the reference backends. If there is no `committer_info` provided, the reference backends default to using `git_committer_info(0)`. The field itself cannot be set to `git_committer_info(0)` since the values are dynamic and must be obtained right when the reflog is being committed. Signed-off-by: Karthik Nayak --- refs.c | 1 + refs/files-backend.c | 24 ++++++++++++++---------- refs/refs-internal.h | 1 + refs/reftable-backend.c | 12 +++++++++++- 4 files changed, 27 insertions(+), 11 deletions(-) diff --git a/refs.c b/refs.c index 762f3e324d59c60cd4f05c2f257e54de8deb00e5..f003e51c6bf5229bfbce8ce61ffad7cdba0572e0 100644 --- a/refs.c +++ b/refs.c @@ -1151,6 +1151,7 @@ void ref_transaction_free(struct ref_transaction *transaction) for (i = 0; i < transaction->nr; i++) { free(transaction->updates[i]->msg); + free(transaction->updates[i]->committer_info); free((char *)transaction->updates[i]->new_target); free((char *)transaction->updates[i]->old_target); free(transaction->updates[i]); diff --git a/refs/files-backend.c b/refs/files-backend.c index 64f51f0da905a9a8a1ac4109c6b0a9a85a355db7..6078668c99ee254e794e3ba49689aa34e6022efd 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -1858,6 +1858,9 @@ static int log_ref_write_fd(int fd, const struct object_id *old_oid, struct strbuf sb = STRBUF_INIT; int ret = 0; + if (!committer) + committer = git_committer_info(0); + strbuf_addf(&sb, "%s %s %s", oid_to_hex(old_oid), oid_to_hex(new_oid), committer); if (msg && *msg) { strbuf_addch(&sb, '\t'); @@ -1871,8 +1874,10 @@ static int log_ref_write_fd(int fd, const struct object_id *old_oid, } static int files_log_ref_write(struct files_ref_store *refs, - const char *refname, const struct object_id *old_oid, - const struct object_id *new_oid, const char *msg, + const char *refname, + const struct object_id *old_oid, + const struct object_id *new_oid, + const char *committer_info, const char *msg, int flags, struct strbuf *err) { int logfd, result; @@ -1889,8 +1894,7 @@ static int files_log_ref_write(struct files_ref_store *refs, if (logfd < 0) return 0; - result = log_ref_write_fd(logfd, old_oid, new_oid, - git_committer_info(0), msg); + result = log_ref_write_fd(logfd, old_oid, new_oid, committer_info, msg); if (result) { struct strbuf sb = STRBUF_INIT; int save_errno = errno; @@ -1974,8 +1978,7 @@ static int commit_ref_update(struct files_ref_store *refs, files_assert_main_repository(refs, "commit_ref_update"); clear_loose_ref_cache(refs); - if (files_log_ref_write(refs, lock->ref_name, - &lock->old_oid, oid, + if (files_log_ref_write(refs, lock->ref_name, &lock->old_oid, oid, NULL, logmsg, flags, err)) { char *old_msg = strbuf_detach(err, NULL); strbuf_addf(err, "cannot update the ref '%s': %s", @@ -2007,9 +2010,9 @@ static int commit_ref_update(struct files_ref_store *refs, if (head_ref && (head_flag & REF_ISSYMREF) && !strcmp(head_ref, lock->ref_name)) { struct strbuf log_err = STRBUF_INIT; - if (files_log_ref_write(refs, "HEAD", - &lock->old_oid, oid, - logmsg, flags, &log_err)) { + if (files_log_ref_write(refs, "HEAD", &lock->old_oid, + oid, NULL, logmsg, flags, + &log_err)) { error("%s", log_err.buf); strbuf_release(&log_err); } @@ -2969,7 +2972,8 @@ static int parse_and_write_reflog(struct files_ref_store *refs, } if (files_log_ref_write(refs, lock->ref_name, &lock->old_oid, - &update->new_oid, update->msg, update->flags, err)) { + &update->new_oid, update->committer_info, + update->msg, update->flags, err)) { char *old_msg = strbuf_detach(err, NULL); strbuf_addf(err, "cannot update the ref '%s': %s", diff --git a/refs/refs-internal.h b/refs/refs-internal.h index 58aa56d1b27c85d606ed7c8c0d908e4b87d1066b..0fd95cdacd99e4a728c22f5286f6b3f0f360c110 100644 --- a/refs/refs-internal.h +++ b/refs/refs-internal.h @@ -113,6 +113,7 @@ struct ref_update { void *backend_data; unsigned int type; char *msg; + char *committer_info; /* * If this ref_update was split off of a symref update via diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c index 647ef9b05b1dc9a376ed054330b487f7595c5caa..e882602487c66261d586a94101bb1b4e9a2ed60e 100644 --- a/refs/reftable-backend.c +++ b/refs/reftable-backend.c @@ -1379,11 +1379,21 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data } if (create_reflog) { + struct ident_split c; + ALLOC_GROW(logs, logs_nr + 1, logs_alloc); log = &logs[logs_nr++]; memset(log, 0, sizeof(*log)); - fill_reftable_log_record(log, &committer_ident); + if (u->committer_info) { + if (split_ident_line(&c, u->committer_info, + strlen(u->committer_info))) + BUG("failed splitting committer info"); + } else { + c = committer_ident; + } + + fill_reftable_log_record(log, &c); log->update_index = ts; log->refname = xstrdup(u->refname); memcpy(log->value.update.new_hash, From patchwork Mon Dec 16 16:44:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: karthik nayak X-Patchwork-Id: 13910067 Received: from mail-ed1-f49.google.com (mail-ed1-f49.google.com [209.85.208.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 48E15205AC6 for ; Mon, 16 Dec 2024 16:44:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.49 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734367480; cv=none; b=qGcV0JheZbri2hSdFy54GX1wl09f4mI72ulBw1Hg3zPlP6gNDoYSBWJpXhqoTO+2jCwA+Fs1rfwXHRMPg9E6SvS1CE5qPqhWw7iuaaefrRMdGx1Fxijx3HPGeS85nYSXegM96mqqTLDJcQtiaWdE6wLrqhBLUHs6Dcd+IJWprPk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734367480; c=relaxed/simple; bh=IB9h00v4zfs7q+FeZZk33xlgGFW2QmXAcKQGj56BkwY=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=aeUieSbsIKBCMQggNCzXi3smLnrlOEKMZ711R8AiudWmLoMNieuriOo3rwgIvjUohVxBUF8NiMhDLMMXyDnY37lzlgwZ1kHAD3OoJlsmuLT+8jvz1lNKtePiuB18u/k9F6WaOTGb7BJm0JPvzEOebn76KqsuYpCnWW4dhOF10Jk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Wn9k65Ow; arc=none smtp.client-ip=209.85.208.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Wn9k65Ow" Received: by mail-ed1-f49.google.com with SMTP id 4fb4d7f45d1cf-5d3f65844deso7263757a12.0 for ; Mon, 16 Dec 2024 08:44:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734367476; x=1734972276; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=lUvb5dIPp19cc7hdfTanzPMDqohb3VapJTcypa4CDlg=; b=Wn9k65OwMRrFkSigBtHR22QQ26YeucvrPZ2YafpvlIfFLW8S592xPF+1+f9eMf6kw1 9dLpXNt1drUDzKfsMQLcJIAWnOO9iRGf7nTHkGEc/Q5rb9s7e7C4NrdhqUj6c9Wn1l81 P2XbND1m/HX0zPaPoYjiwNs2Rzfr4Wbczo0cvU3m9sr1oV1kV7ncN6e3gAnNu+zy9QX1 HrOLl8k3BvAA71+cMzszjY8/p4xNvD/arzLA/4eITHkrysoTSr9NUZrHU2BT1EkvrQpw cT2tTswSmhNr30Xun4G3b684RiVbs+6Tf9n++ms3r+P/x+ARnsdSqRAUZy2hMnWNhvwx Q8zQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734367476; x=1734972276; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=lUvb5dIPp19cc7hdfTanzPMDqohb3VapJTcypa4CDlg=; b=DTPbCb0MVSGEOoj2S6Jmv3cJ4AcG2BwanQuFcvmKnZQXJz9GFAh75OU+CD7gcGHcsb Eguk3MUIOvcjjfT4sA86H0x7aE3SvccFMuTQNhCR05Q0PWS8httwF0gaZ4+9zQy293Et sOrMXKvjWMeTXl8Q/dJLfXnjD5z+hIgPUTfP93/cohEsj5KEewTCuDAX7D7sB5O3VLOb cVw5j3cqq83cKB47bEwAwxMcWumqGi69MwK2MOBlShUX1RLR+snQxD7ZCs1m+tfnNEHN PMlbkA9geeB+PEI9ANgE8Vk+keLyZ4F0DO5TryxKmBhRQlO5Ve11tqiQbMCQcfQiIsLf vCyw== X-Gm-Message-State: AOJu0Yxl7bXFyOK9RCtSm6ZX5NzwiOlpeJbM1L+TgwNNGHVFI7byOaSt Jl/8SxlCRacU1NdMs6NJCQwdIyH+9afyvm4yo+gsa4t9Wam6kATjRQ3Pieka X-Gm-Gg: ASbGncvFEKWk9I0hsE79/Zp3UVib4CIgpTxGR0A5G2N4CAJb2YaSsBhkOHqasNHIQZy f3qz1LlTdiqw6zr5oKRquMs+IBSAUq+wjS9cSUr0wbwLVpRG6XGb0bB48bVs/GZL9/QJCFsBE/6 fMJdEDcUa++7zS7T+V3MmlCV1sU1zyeJaNIbhCLy21xFo67UjZWwzN2xLnWa3vPRhBM424iz/3h LLkY146FQDuh0ANpWS0HQhq9QZYk/aYHqfs/vnzfhCwzLWFRf001CI+JuY= X-Google-Smtp-Source: AGHT+IGN9fflX82RNxXubu41UsdcYCiJ8+RmCwW0GhyLSkTP83PlOuhjdhNAJTDao//ZEbXYsFEn4A== X-Received: by 2002:a17:907:3e12:b0:aa6:ac19:b579 with SMTP id a640c23a62f3a-aab779af3bfmr1105046366b.26.1734367476191; Mon, 16 Dec 2024 08:44:36 -0800 (PST) Received: from [10.82.4.202] ([217.110.80.4]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aab9635957fsm354834366b.109.2024.12.16.08.44.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 16 Dec 2024 08:44:35 -0800 (PST) From: Karthik Nayak Date: Mon, 16 Dec 2024 17:44:27 +0100 Subject: [PATCH v4 2/8] refs: add `index` field to `struct ref_udpate` Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241216-320-git-refs-migrate-reflogs-v4-2-d7cd3f197453@gmail.com> References: <20241216-320-git-refs-migrate-reflogs-v4-0-d7cd3f197453@gmail.com> In-Reply-To: <20241216-320-git-refs-migrate-reflogs-v4-0-d7cd3f197453@gmail.com> To: git@vger.kernel.org Cc: Karthik Nayak , ps@pks.im, Christian Couder X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=2464; i=karthik.188@gmail.com; h=from:subject:message-id; bh=IB9h00v4zfs7q+FeZZk33xlgGFW2QmXAcKQGj56BkwY=; b=owEB7QES/pANAwAIAT7VnySORox/AcsmYgBnYFjwkuWgX4RyG+aajJ1c18One0gFB1bAdPnfk 8m/trihzBGJAbMEAAEIAB0WIQRXzkx/Y3VxD8tlxgY+1Z8kjkaMfwUCZ2BY8AAKCRA+1Z8kjkaM f0CGC/4xkK3ctoLg19Dr+R/qWVPkqKZP48H15BTlLv6dpBDk5ixtrsa6M2L/30B51up0pa+QV5J b3m/SeMg1Jl3glPXslQ4g0kd8rvO0O8gIXvKxwlGh+ULav1G8alPe1/7gXt6Ew6len4WLN2Zb0t WsRFttvv8BBjCj5wF0e/vfB+0L9eC2KLwcuG1Dv/SopgyYH/brd+W7NsWttdsy1OpNqqEOlWQaP LnwOl19u+Xl4H5U2FGl/rhB2IpOqsxekdReeVazWR//qJka6JOnarcWK6wbujqSjEJ9HOAy1rqJ W/hdZWBpx3Fiswrsd498MtEsV5CPhgvP8VFOAWhKQ/DYv1WSNrEv7krzVft2JdSXFEIhtLt/3n4 GKks/vdfB0vciqpnaNd+d0CvYnyqobmQuajalwmbJBn122A4A3zvqs/6JJ1zmfAFOEi045HS10g u92l+y4M7rPatQs+6OOuItzLYlobGBSWaP+fe1nCdtwiLPM+hc8rZBda4FA0MhMSszrpc= X-Developer-Key: i=karthik.188@gmail.com; a=openpgp; fpr=57CE4C7F6375710FCB65C6063ED59F248E468C7F The reftable backend, sorts its updates by refname before applying them, this ensures that the references are stored sorted. When migrating reflogs from one backend to another, the order of the reflogs must be maintained. Add a new `index` field to the `ref_update` struct to facilitate this. This field is used in the reftable backend's sort comparison function `transaction_update_cmp`, to ensure that indexed fields maintain their order. Signed-off-by: Karthik Nayak --- refs/refs-internal.h | 7 +++++++ refs/reftable-backend.c | 13 +++++++++++-- 2 files changed, 18 insertions(+), 2 deletions(-) diff --git a/refs/refs-internal.h b/refs/refs-internal.h index 0fd95cdacd99e4a728c22f5286f6b3f0f360c110..f5c733d099f0c6f1076a25f4f77d9d5eb345ec87 100644 --- a/refs/refs-internal.h +++ b/refs/refs-internal.h @@ -115,6 +115,13 @@ struct ref_update { char *msg; char *committer_info; + /* + * The index overrides the default sort algorithm. This is needed + * when migrating reflogs and we want to ensure we carry over the + * same order. + */ + unsigned int index; + /* * If this ref_update was split off of a symref update via * split_symref_update(), then this member points at that diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c index e882602487c66261d586a94101bb1b4e9a2ed60e..c008f20be719fec3af6a8f81c821cb9c263764d7 100644 --- a/refs/reftable-backend.c +++ b/refs/reftable-backend.c @@ -1279,8 +1279,17 @@ static int reftable_be_transaction_abort(struct ref_store *ref_store UNUSED, static int transaction_update_cmp(const void *a, const void *b) { - return strcmp(((struct reftable_transaction_update *)a)->update->refname, - ((struct reftable_transaction_update *)b)->update->refname); + struct reftable_transaction_update *update_a = (struct reftable_transaction_update *)a; + struct reftable_transaction_update *update_b = (struct reftable_transaction_update *)b; + + /* + * If there is an index set, it should take preference (default is 0). + * This ensures that updates with indexes are sorted amongst themselves. + */ + if (update_a->update->index || update_b->update->index) + return update_a->update->index - update_b->update->index; + + return strcmp(update_a->update->refname, update_b->update->refname); } static int write_transaction_table(struct reftable_writer *writer, void *cb_data) From patchwork Mon Dec 16 16:44:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: karthik nayak X-Patchwork-Id: 13910068 Received: from mail-ej1-f49.google.com (mail-ej1-f49.google.com [209.85.218.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A727A207DF1 for ; Mon, 16 Dec 2024 16:44:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.49 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734367481; cv=none; b=VZZ9D3yTMFYA9OYNEuTbd/Y9gjZDO1A/TbMpXRbyYpEHfpdsMi3lneo7Uc4g5/S0S47XPie+U5KjkJxNvRFi2F3NBSZodVOHB9TfLw8oBkNf7TdP+v21fqdKO0VPVNiMGqPA1A2OClPGj8ohTWc7E+9aeSvT69o6pEOudNz4dS8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734367481; c=relaxed/simple; bh=ERhO5bUagvCcVWd1ajCqhTTv/qHXvpyDSomL3moOTTM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=LlfW4hc7nRY0nizM2kqWek2jki4K+dB3XMpHiP/S5U+o0fSvMG3qoW1Jgsaj4Bn9BWrZPJvfAfolfH7EkJDooRKL0Tuvcgy48OooPhaPnt91kgIlUxOz2IwBqtdpPrjNm+//Eu9pr/qQ6+ct5abg/o2ykHzBVunJWIc3f4vOSbc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=XQ5N1k0k; arc=none smtp.client-ip=209.85.218.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="XQ5N1k0k" Received: by mail-ej1-f49.google.com with SMTP id a640c23a62f3a-a9e8522445dso881754966b.1 for ; Mon, 16 Dec 2024 08:44:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734367478; x=1734972278; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=5gNfOmPXL16KqA+CPu0YSmndFRynLUkndelIwKCAq4k=; b=XQ5N1k0kAM4v3Syq4hsqah1jDAcOidl+Yv5RqBwaVhwXgdTfMSTsxTe3AJh8Tik60k i33sP2HyxhD+wpb2yJuMAXi07k6pKr1Ukfo3c5WnTqi1QKxo1tmje9aDt+lHCP9o/l06 pGsLDUjTWLk1/OfG9prhqMS3e169vb8xc2ce5MZLJnUQrbFBrFFSkmzCFP1Ph84sDUl5 6Xc7fWmmp5uEEufbj4ctVu+Tp1hwG6jMkyy9VVEORoEAvG27hNR8Kno7mGhpQ9NIZ6k6 MqgiaPagdukT3ia7u2HLN/0Z8UQiJzFXjR1LJfRlA8UapsZQubt5gF6RzLE4teFYMA4P gurA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734367478; x=1734972278; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5gNfOmPXL16KqA+CPu0YSmndFRynLUkndelIwKCAq4k=; b=AYivDwyCaeJ56SHsqbGjNHOavfYWZVmK7zAd/2N0Sb29zx74xB+jdufogqlqfIDwnL fevjtrl/TmROaat7so+gGkAdrbUIMJ+ddtu3juYKCw+nx9W4mXWfRBiYRDsJFmn/sUe+ Hj8nQqpEgiZSr11wvTlUfN3EGPa9Gt/okuCiIL62tl3GF2yVWrjSV614B8BM/X1iTnhm KvEMAc0U7mH49reBVBZLYOg3tc1p+P/71pyqTdeHHdTzWclMC/dmJdeEvivquw1cppRi WpH6BN+11PhQJu4kYXJBPKG97//dwAyowtGEftxdPxzSv5CyIFwou1WOTIor7E0mi6sX 2KWg== X-Gm-Message-State: AOJu0Yz8UZQGEPp+BMWsLS/cgeyyvk3M2MOezTOPykJuLZIhKoAdV08q ba/cZV0NMN1e3JcaPkbxX/dG09xzl6QPm9U25Y3Zx0bxAFCjmZ0wcOX9JvL6 X-Gm-Gg: ASbGnctnWOrL9LTUiYWnfreV7UowEmasiEMQ61STk3AGnDQIwXswG7zAJq8ULCCsmjD +PEJV6sU4h/KCmPgteF+odH00ZpyOGwPwQkR9IuHHG6IKVAkZg+tj8KmxsH6GAxDsF7FS3Tjnj+ xcir1GLaSPlaWLJNyZCVidGijvE1p1gBc5rTv6DeGKNgUiZUpztFBxMUp0GNhqbEM50DHPk6D0t CsZJbT8M14Gj2qcieE0qXbHZG1KNsyYl20MgaGuwrMAsXA1wANo/sTK7to= X-Google-Smtp-Source: AGHT+IFA9qXAzllvbSArLN8cs6ZnI6rZIBWJfQDxk29LLtPv4xtlt/y1g1sjhIADmvTuBCXdyhaX4w== X-Received: by 2002:a17:907:6d16:b0:aa6:8f71:1645 with SMTP id a640c23a62f3a-aab77ed35abmr1328044866b.55.1734367477373; Mon, 16 Dec 2024 08:44:37 -0800 (PST) Received: from [10.82.4.202] ([217.110.80.4]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aab9635957fsm354834366b.109.2024.12.16.08.44.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 16 Dec 2024 08:44:36 -0800 (PST) From: Karthik Nayak Date: Mon, 16 Dec 2024 17:44:28 +0100 Subject: [PATCH v4 3/8] refs/files: add count field to ref_lock Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241216-320-git-refs-migrate-reflogs-v4-3-d7cd3f197453@gmail.com> References: <20241216-320-git-refs-migrate-reflogs-v4-0-d7cd3f197453@gmail.com> In-Reply-To: <20241216-320-git-refs-migrate-reflogs-v4-0-d7cd3f197453@gmail.com> To: git@vger.kernel.org Cc: Karthik Nayak , ps@pks.im, Christian Couder X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=5670; i=karthik.188@gmail.com; h=from:subject:message-id; bh=ERhO5bUagvCcVWd1ajCqhTTv/qHXvpyDSomL3moOTTM=; b=owEB7QES/pANAwAIAT7VnySORox/AcsmYgBnYFjwyVryaUgb+HLymh3pJc+6LHCjlUZ54L06Z EePezgdKy6JAbMEAAEIAB0WIQRXzkx/Y3VxD8tlxgY+1Z8kjkaMfwUCZ2BY8AAKCRA+1Z8kjkaM f2znC/0bk2EBNO6tvIKJxG5exJFEIZOzOVC0VndJSiLk5K5oYOdXxw/DkZ4SxkhDO0y8aB7nBrU 3H1lqIoOusdI5BM0nOWD1YaAYbBoq0i33lutPiP8t10vwJ6OdWZ//s/dHbVmUl1VF57IF3lWBnr iqtJQ1jInjo3oDJRfjTNhNFECPNApfF//2FD8zJ94jy9iDvNFbPVlS5WvpDPO7992Pgob0ydsH+ 0KjGXcl6TFtBKvtdGmYusO9+bdHHxLGJkeiur8TU02sW+NtbosrYZH/TYQpSHfmGw1/w8ARZVuf eOO8ta7SGfvb8JFcbhzjXf303kf3DUWsKVtRs5DncXetVWn07mr8Rf3cgrRIXo4zyRZFT9DBi3S PO96J+KKVRqR3wEHaPOSzyDRHDWoNzkm/7PccjL8Rctu8G1c9IA0qKxlggjPvQ1AgfFqH1kmOOP iWVD2HXpfLXxi8DPNSzCILDvO8T+dKejHwzqureW8E6Fc8Zpicu3N9iYnxIHtwSFj+NPo= X-Developer-Key: i=karthik.188@gmail.com; a=openpgp; fpr=57CE4C7F6375710FCB65C6063ED59F248E468C7F When refs are updated in the files-backend, a lock is obtained for the corresponding file path. This is the case even for reflogs, i.e. a lock is obtained on the reference path instead of the reflog path. This works, since generally, reflogs are updated alongside the ref. The upcoming patches will add support for reflog updates in ref transaction. This means, in a particular transaction we want to have ref updates and reflog updates. For a given ref in a given transaction there can be at most one update. But we can theoretically have multiple reflog updates for a given ref in a given transaction. A great example of this would be when migrating reflogs from one backend to another. There we would batch all the reflog updates for a given reference in a single transaction. The current flow does not support this, because currently refs & reflogs are treated as a single entity and capture the lock together. To separate this, add a count field to ref_lock. With this, multiple updates can hold onto a single ref_lock and the lock will only be released when all of them release the lock. This patch only adds the `count` field to `ref_lock` and adds the logic to increment and decrement the lock. In a follow up commit, we'll separate the reflog update logic from ref updates and utilize this functionality. Signed-off-by: Karthik Nayak --- refs/files-backend.c | 58 +++++++++++++++++++++++++++++++++++----------------- 1 file changed, 39 insertions(+), 19 deletions(-) diff --git a/refs/files-backend.c b/refs/files-backend.c index 6078668c99ee254e794e3ba49689aa34e6022efd..02cb4907d8659e87a227fed4f60a5f6606be8764 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -71,6 +71,7 @@ struct ref_lock { char *ref_name; struct lock_file lk; struct object_id old_oid; + unsigned int count; /* track users of the lock (ref update + reflog updates) */ }; struct files_ref_store { @@ -638,9 +639,12 @@ int parse_loose_ref_contents(const struct git_hash_algo *algop, static void unlock_ref(struct ref_lock *lock) { - rollback_lock_file(&lock->lk); - free(lock->ref_name); - free(lock); + lock->count--; + if (!lock->count) { + rollback_lock_file(&lock->lk); + free(lock->ref_name); + free(lock); + } } /* @@ -696,6 +700,7 @@ static int lock_raw_ref(struct files_ref_store *refs, *lock_p = CALLOC_ARRAY(lock, 1); lock->ref_name = xstrdup(refname); + lock->count = 1; files_ref_path(refs, &ref_file, refname); retry: @@ -1169,6 +1174,7 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs, goto error_return; lock->ref_name = xstrdup(refname); + lock->count = 1; if (raceproof_create_file(ref_file.buf, create_reflock, &lock->lk)) { unable_to_lock_message(ref_file.buf, errno, err); @@ -2535,6 +2541,12 @@ static int check_old_oid(struct ref_update *update, struct object_id *oid, return -1; } +struct files_transaction_backend_data { + struct ref_transaction *packed_transaction; + int packed_refs_locked; + struct strmap ref_locks; +}; + /* * Prepare for carrying out update: * - Lock the reference referred to by update. @@ -2557,11 +2569,14 @@ static int lock_ref_for_update(struct files_ref_store *refs, { struct strbuf referent = STRBUF_INIT; int mustexist = ref_update_expects_existing_old_ref(update); + struct files_transaction_backend_data *backend_data; int ret = 0; struct ref_lock *lock; files_assert_main_repository(refs, "lock_ref_for_update"); + backend_data = transaction->backend_data; + if ((update->flags & REF_HAVE_NEW) && ref_update_has_null_new_value(update)) update->flags |= REF_DELETING; @@ -2572,18 +2587,25 @@ static int lock_ref_for_update(struct files_ref_store *refs, goto out; } - ret = lock_raw_ref(refs, update->refname, mustexist, - affected_refnames, - &lock, &referent, - &update->type, err); - if (ret) { - char *reason; + lock = strmap_get(&backend_data->ref_locks, update->refname); + if (lock) { + lock->count++; + } else { + ret = lock_raw_ref(refs, update->refname, mustexist, + affected_refnames, + &lock, &referent, + &update->type, err); + if (ret) { + char *reason; + + reason = strbuf_detach(err, NULL); + strbuf_addf(err, "cannot lock ref '%s': %s", + ref_update_original_update_refname(update), reason); + free(reason); + goto out; + } - reason = strbuf_detach(err, NULL); - strbuf_addf(err, "cannot lock ref '%s': %s", - ref_update_original_update_refname(update), reason); - free(reason); - goto out; + strmap_put(&backend_data->ref_locks, update->refname, lock); } update->backend_data = lock; @@ -2730,11 +2752,6 @@ static int lock_ref_for_update(struct files_ref_store *refs, return ret; } -struct files_transaction_backend_data { - struct ref_transaction *packed_transaction; - int packed_refs_locked; -}; - /* * Unlock any references in `transaction` that are still locked, and * mark the transaction closed. @@ -2767,6 +2784,8 @@ static void files_transaction_cleanup(struct files_ref_store *refs, if (backend_data->packed_refs_locked) packed_refs_unlock(refs->packed_ref_store); + strmap_clear(&backend_data->ref_locks, 0); + free(backend_data); } @@ -2796,6 +2815,7 @@ static int files_transaction_prepare(struct ref_store *ref_store, goto cleanup; CALLOC_ARRAY(backend_data, 1); + strmap_init(&backend_data->ref_locks); transaction->backend_data = backend_data; /* From patchwork Mon Dec 16 16:44:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: karthik nayak X-Patchwork-Id: 13910069 Received: from mail-ed1-f50.google.com (mail-ed1-f50.google.com [209.85.208.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D4C99207E10 for ; Mon, 16 Dec 2024 16:44:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734367482; cv=none; b=Bdd/FAXC0N5uXnd5U7yZNNDzoNFr7jqWJxvc0ICFcsQu1+PYYSd1XnWzVvJsrExtjh4E/xpUl89V+Mpo9NGN6uhZycNceE0iMyiJlrXXT8I91xXQo4EY/KlquAnF6yscVIBJVnAvH1NQqokLdpgzXB49qXbRHOXPzi7NSd8eUjo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734367482; c=relaxed/simple; bh=NoEckIGxQa5jQrZ0dQ7BHqCovFckF/qXPjbGFqkkuA0=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=j2tCBssErGpU7ve+qJ9oZDy0wgZFUdTntSp5q0UYkw8Eei+GOswv1jXKo7YBbX5HYuVVmGeOZBC5fBzlrLjG0M38wYRDVfwcydEX0ToupG9FtiRhUGJ3jF8Cy5mPv0bUi6wNh7nGs8R+ZYlG7GQ8yFw8aBkeRwd+n9uYvrvtobw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=OHmB17yl; arc=none smtp.client-ip=209.85.208.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="OHmB17yl" Received: by mail-ed1-f50.google.com with SMTP id 4fb4d7f45d1cf-5d3cf094768so6789151a12.0 for ; Mon, 16 Dec 2024 08:44:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734367479; x=1734972279; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=Io5XE6RXiBUov+SaLOGtphgg3UciL24hpxxR053Wce4=; b=OHmB17ylqWMRL6cebjsvpiiei4Cot4AK3BGPzk+aCLqBVhaOaNDQ84dSlcf+PsvDeh JSpRCAQ/yI+QzEPukXXiYY4pHZlPuaC3HFOMKnfyw8pr46Fo57K86k43X4SrxM8RIOGg 8XSrdbcBaWBrtcqA4puR66A7YeuOjNJY5prz+MOEdDzdt2m65w9Wj+jCUEZ+92WYnliW OhPtTa8nr03qa+dNKMQShhkmi7Y0OxLt1+ILbhyU8LQXKsScemYQDKcGZpJzycEZ/9S2 JJI9lKOmYSvt92PPR93Cm9938Is0zOnsGJM7KavAwRp7AnkZ/owPPOZRrhrz4cI42eyd lzIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734367479; x=1734972279; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Io5XE6RXiBUov+SaLOGtphgg3UciL24hpxxR053Wce4=; b=Ze/0V4tsMDkf3o3jxWNpA5lcKfBXFwEmpWNdaHJjDtep8+PDTHKBrnh/EoAfbF7GYP gBnn3uh/C6ObU6dOnVDcLYZZyHf67blJn+i77UEfkadG/BW9iOGSm1IQtAbxySDSOA3P qzv10unY1d0K+QFN96W6d36R2coI56+C6HcSSUk0gK1oArCWRCnOTq+bwgPdPzK2rVBw jwszvU/RWl6GaOBU43VNO6NNYFowgrpIeRueRzFIKbjOQF5MX/XDke78dM//I044wtNx rX2mx20AHIOkQjfRo2HZK6Cego9tjgtSC0doHtF/k0wjv1w4vmUnOdxwidve8L28jVTY xPYw== X-Gm-Message-State: AOJu0YyLtF+L+EftmjQvNgovJ4WhZEefZI8yoYv/sAsNK0Ia+1d0IWkg kkO2laoYeyZjCz4Tageg03/8jPfEInGMnwPpMJlKnqLobcFQNbNYT6+q7eP0 X-Gm-Gg: ASbGncs4KaOy1dj05c4eqYZYgicDIysAuoBUASl/IxY2RPnbkdeoosDzLnsOOG/39c3 AABVLaPsZ/ZweqG9DbZa8RU7qopyH5MtVpoiXQjPNKnpsnkvC7i7q12iYXfEFIhXn6x1wI4FC5Z O43+Dl114/QjlYf1CDnf6TmpzhLKRm3Nm587ER/tXixzNqZ4H6dl5Nq7JmLzrHEP6E1CivDF6PO +76QluadbiOcIxU9jIQRX1JGiSwTZPNV8d2bTP7DN/E5tW3Jm0BRdJrFYE= X-Google-Smtp-Source: AGHT+IGh441fcBJLOZfFrBFkLq4gFExfBfGvlvC2+poduJ13VjRaldXRpFKKcicperdFwwvot2TlJQ== X-Received: by 2002:a17:907:96a5:b0:aa6:9c08:cd3 with SMTP id a640c23a62f3a-aab77ec88d2mr1399568366b.48.1734367478515; Mon, 16 Dec 2024 08:44:38 -0800 (PST) Received: from [10.82.4.202] ([217.110.80.4]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aab9635957fsm354834366b.109.2024.12.16.08.44.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 16 Dec 2024 08:44:38 -0800 (PST) From: Karthik Nayak Date: Mon, 16 Dec 2024 17:44:29 +0100 Subject: [PATCH v4 4/8] refs: extract out refname verification in transactions Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241216-320-git-refs-migrate-reflogs-v4-4-d7cd3f197453@gmail.com> References: <20241216-320-git-refs-migrate-reflogs-v4-0-d7cd3f197453@gmail.com> In-Reply-To: <20241216-320-git-refs-migrate-reflogs-v4-0-d7cd3f197453@gmail.com> To: git@vger.kernel.org Cc: Karthik Nayak , ps@pks.im, Christian Couder X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=2366; i=karthik.188@gmail.com; h=from:subject:message-id; bh=NoEckIGxQa5jQrZ0dQ7BHqCovFckF/qXPjbGFqkkuA0=; b=owEB7QES/pANAwAIAT7VnySORox/AcsmYgBnYFjw+SGjp1K4d5N+wtGwzw9/efOnPIR38CPVg EJqyO3E0lGJAbMEAAEIAB0WIQRXzkx/Y3VxD8tlxgY+1Z8kjkaMfwUCZ2BY8AAKCRA+1Z8kjkaM fx/tC/46joNxf+uCLIaOD42QrR/kFu4frBNf1Ok81GWavb0gxAuuCNDNmYCeh1Ld0N6xbuVPQ6q BDWXvjfVTsKKZBVZLEhIwDfuaOdVezoXO7PRCwSWgXapK7ZKf8mBAZn8hSzsv0P6kUyxSnI3rpN CrCCumR3mXp8ubI7/xpP2eKBcTMcLtODsMisGIz+UQ8nj3uCgUnDLQjSCczud0mailjzMIy0uK2 IGerDwQIk864at1jF7QPj7ZBg9Hkqusyy7thInfqRYYCEHACFAc/7x0rLjhLvWUuvgPDtn1nY9v /mmAZqmHZEM81utMSnkPVeglB7QdFwBO/jQBXjup7lRGVv3BtCMlssC10bj/q/munA4Sv3xgLKZ hnz2BtljIiBedTxw65OvDg5F3d4+XnxbTrp277E6dlSC8FBMbL9jMGITZhMcB7oUr7a5sbHdcx3 nrbWIGb2XwgErYRDiusCRE9V+Wpx/guYKlteUqdJKvTmR/feXfUNMTyOwm7UyBU/1DeEY= X-Developer-Key: i=karthik.188@gmail.com; a=openpgp; fpr=57CE4C7F6375710FCB65C6063ED59F248E468C7F Unless the `REF_SKIP_REFNAME_VERIFICATION` flag is set for an update, the refname of the update is verified for: - Ensuring it is not a pseudoref. - Checking the refname format. These checks will also be needed in a following commit where the function to add reflog updates to the transaction is introduced. Extract the code out into a new static function. Signed-off-by: Karthik Nayak --- refs.c | 37 +++++++++++++++++++++++-------------- 1 file changed, 23 insertions(+), 14 deletions(-) diff --git a/refs.c b/refs.c index f003e51c6bf5229bfbce8ce61ffad7cdba0572e0..9c9f4940c60d3cdd34ce8f1e668d17b9da3cd801 100644 --- a/refs.c +++ b/refs.c @@ -1196,6 +1196,28 @@ struct ref_update *ref_transaction_add_update( return update; } +static int transaction_refname_valid(const char *refname, + const struct object_id *new_oid, + unsigned int flags, struct strbuf *err) +{ + if (flags & REF_SKIP_REFNAME_VERIFICATION) + return 1; + + if (is_pseudo_ref(refname)) { + strbuf_addf(err, _("refusing to update pseudoref '%s'"), + refname); + return 0; + } else if ((new_oid && !is_null_oid(new_oid)) ? + check_refname_format(refname, REFNAME_ALLOW_ONELEVEL) : + !refname_is_safe(refname)) { + strbuf_addf(err, _("refusing to update ref with bad name '%s'"), + refname); + return 0; + } + + return 1; +} + int ref_transaction_update(struct ref_transaction *transaction, const char *refname, const struct object_id *new_oid, @@ -1213,21 +1235,8 @@ int ref_transaction_update(struct ref_transaction *transaction, return -1; } - if (!(flags & REF_SKIP_REFNAME_VERIFICATION) && - ((new_oid && !is_null_oid(new_oid)) ? - check_refname_format(refname, REFNAME_ALLOW_ONELEVEL) : - !refname_is_safe(refname))) { - strbuf_addf(err, _("refusing to update ref with bad name '%s'"), - refname); + if (!transaction_refname_valid(refname, new_oid, flags, err)) return -1; - } - - if (!(flags & REF_SKIP_REFNAME_VERIFICATION) && - is_pseudo_ref(refname)) { - strbuf_addf(err, _("refusing to update pseudoref '%s'"), - refname); - return -1; - } if (flags & ~REF_TRANSACTION_UPDATE_ALLOWED_FLAGS) BUG("illegal flags 0x%x passed to ref_transaction_update()", flags); From patchwork Mon Dec 16 16:44:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: karthik nayak X-Patchwork-Id: 13910070 Received: from mail-ej1-f53.google.com (mail-ej1-f53.google.com [209.85.218.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CA8E4207E1F for ; Mon, 16 Dec 2024 16:44:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.53 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734367483; cv=none; b=Ilh4Ic45qQEB9YTm/Nz0R9E72rN/ZAE+0Ct8Tmge02SRksI9YBeWl5sM5wFW9HsPss4PqnVTOYclp2EIc64A6S3QBIEV/pnksvVL8TMHM4l6FjpIFDvbV+cbMT8zTqboXR783WgK3+C7FwV21p3Lebf/lKdo7MqUkgnjwpIjVkQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734367483; c=relaxed/simple; bh=ATy4ssKxG+1kuX/UpGVdpNwF/duWxcYMcv9ZJ2nZT2Y=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=L1vBbkxLCQiWz92+05TdPmHPMbTniwXTED4+hoSPTh1YyYCTp2YuyEDwXfVGkUqkkO+o+EYj0fW3d2kUi4gxOYGW6eC3Pl/QXNmR9P6ilPHyHnohozWxqenw/+aqZIZCszeAdr0gDIxjwiwdXFKL3fgS36ou12kNIrxiTYFA8Ts= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=MEkb8Tig; arc=none smtp.client-ip=209.85.218.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="MEkb8Tig" Received: by mail-ej1-f53.google.com with SMTP id a640c23a62f3a-aa6aee68a57so632296266b.3 for ; Mon, 16 Dec 2024 08:44:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734367480; x=1734972280; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=W1ch3s3JnDSoPZxdCqvaf+8QiC2QUWbiyEX7XSSiSX0=; b=MEkb8TigiHePgHoXAvpyVYNM4LkXG5/gKVItIKdUJO1lVuDIl+w93et3xVSCeRAJyb 2GpYiLoV7tBR5ptivwuGv3iWvyx6Qd9bd43fQycZ1ZQvP8Rw72uIoxjFO/GCSt3I9I8p 2z5ZR9/y6FxCd81LGLEXAi05BDT1I9NnPBv/7yf1m2MmFNwEULvvrtYDbwp+Q4WmemsJ HzJwgiYtqtI+LNfb2CChLIEWQMEzV3LL1Z8uvvO8ZcrmuCFCcpam52OiZLGxdynAMKYB ijN3S29OWUbizZI4Vnc7WqAvGPR9GEJ0y5+fA0o67OcQLnAoAh6HmpU1xR9CC87k4zFK pfiw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734367480; x=1734972280; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=W1ch3s3JnDSoPZxdCqvaf+8QiC2QUWbiyEX7XSSiSX0=; b=jbcuzWt6Jb4hXIlw7ps4W8En7MCi/37X133gnHZJzivSFifawtwYQvOpNRfo4GixXl ouviXuabKx/mhW7pnObkIiE4g4EUbCV+xlv7jaZ/Q2HVKNIQRvlQw9Golc+kz4mllbtc CE+w+7KhzG5qCJTKURqQEB/gvcFr/9bzbrHMQqpGklALSUnNSy17cU99gMsbgAt+ZcFF W3kwrdEXANkgG3N9uB3ipFrdQ+4K5VFsQ8BGu8MHoend0VpfQMExGVGpw1dxxWoknHc3 5UIwPvI/uGdEfpKgHhQbUtSsuKh0ei36/84AGNoTloSXVVtXBMtnNO65OwPjbytpZdWK GWQA== X-Gm-Message-State: AOJu0Yzm7pTonGMRUSttKZvSlzz2vALuahhYvVe9c5Oll/JgN1OsU4cD zaRxFPorSjf8v5ZIvNCR/FjauI+hYzU/0ofiW0JBqyB92qjUXOmZjb8JW806 X-Gm-Gg: ASbGnctSsWWuF4Vw/hQR8AcXS3EFs0qarohgvEygxM0rknQI5kCatIMj2qaVftulwaj rJCmfF4S9DMwOSsJ/IxYlKa0DvtuQlts0O0Q0YoxIpW6SV25Q5fHhlmNivWjeezgfmBNFRjgE+r AWBkcZvU8t9P/G/pVQMkn2acTPkrBHo0gi5wUOZjREqZeta9qFsnWTebDr9IKsKaK5xRNoV+hfR 9Nf7lSHqXC8vdRHTKFfKwZKE8BZJmH4ay8W7yjUUuR3rKGi3T5TO62g1Ew= X-Google-Smtp-Source: AGHT+IEXCHsJLwts4V3/zQ/C1pBd9ND/h09bD10674HqSD0VJuM6BMH0XZ9PXSXzy+NHCuoEBbGSaA== X-Received: by 2002:a17:907:7fa5:b0:aa6:8a1b:8b78 with SMTP id a640c23a62f3a-aab778c1e9dmr1541444566b.6.1734367479715; Mon, 16 Dec 2024 08:44:39 -0800 (PST) Received: from [10.82.4.202] ([217.110.80.4]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aab9635957fsm354834366b.109.2024.12.16.08.44.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 16 Dec 2024 08:44:39 -0800 (PST) From: Karthik Nayak Date: Mon, 16 Dec 2024 17:44:30 +0100 Subject: [PATCH v4 5/8] refs: add `committer_info` to `ref_transaction_add_update()` Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241216-320-git-refs-migrate-reflogs-v4-5-d7cd3f197453@gmail.com> References: <20241216-320-git-refs-migrate-reflogs-v4-0-d7cd3f197453@gmail.com> In-Reply-To: <20241216-320-git-refs-migrate-reflogs-v4-0-d7cd3f197453@gmail.com> To: git@vger.kernel.org Cc: Karthik Nayak , ps@pks.im, Christian Couder X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=5487; i=karthik.188@gmail.com; h=from:subject:message-id; bh=ATy4ssKxG+1kuX/UpGVdpNwF/duWxcYMcv9ZJ2nZT2Y=; b=owEB7QES/pANAwAIAT7VnySORox/AcsmYgBnYFjwq/0DsKHogDxkdsf4BUxQvkyxazMigcb7j SCS4qW9TSaJAbMEAAEIAB0WIQRXzkx/Y3VxD8tlxgY+1Z8kjkaMfwUCZ2BY8AAKCRA+1Z8kjkaM fyRjC/4wh39x5fku5opaRU2udnKSZlOQBT2exudjhR9H6wMCpOOEEfgpoLQGvWMLoHIsMs6kalQ EgjXSRf3ddg7/nMeeFuY4VQ5YmjrZZfVYp/jtzrbMAHNAILUXZNStMgw3rD5Q2RjSb0X0rdJ7y4 cnKZyALhgX7J9RJXWs0TWTquXQxoL5kzYbar7bgJnNf1aZns4Itr4uk32J/x1FaWKcDjFu85SaG SX2MewvIdY78GewfFor+lH7kLiHt/kUrrAXgoQei9/cXzF1MFWdNsucjXWajddacn2EDUDJXzUy aBDWe8EXPa+OLgATPTwFOzRTDqs50PtDQcKbl/INZvTGonu9VHeqAZVqO0UZ6G1OkJZ4oVYEUK8 xvIVM30ZVlEdEQdBwUTaEaCdMfrej53IlPVf67oj/FCkqiSUSyvwwW34MfCZDTo4DdHjTwLpAFP c7BGEfuKGhDUuV2QYZCmktP5bvPxUhBH85VdvfCk4+qvbPTkmW/46poWYSYbzD5jCwWks= X-Developer-Key: i=karthik.188@gmail.com; a=openpgp; fpr=57CE4C7F6375710FCB65C6063ED59F248E468C7F The `ref_transaction_add_update()` creates the `ref_update` struct. To facilitate addition of reflogs in the next commit, the function needs to accommodate setting the `committer_info` field in the struct. So modify the function to also take `committer_info` as an argument and set it accordingly. Signed-off-by: Karthik Nayak --- refs.c | 7 +++++-- refs/files-backend.c | 14 ++++++++------ refs/refs-internal.h | 1 + refs/reftable-backend.c | 6 ++++-- 4 files changed, 18 insertions(+), 10 deletions(-) diff --git a/refs.c b/refs.c index 9c9f4940c60d3cdd34ce8f1e668d17b9da3cd801..782bf1090af65196263a3c35ed18d878bb4f2967 100644 --- a/refs.c +++ b/refs.c @@ -1166,6 +1166,7 @@ struct ref_update *ref_transaction_add_update( const struct object_id *new_oid, const struct object_id *old_oid, const char *new_target, const char *old_target, + const char *committer_info, const char *msg) { struct ref_update *update; @@ -1190,8 +1191,10 @@ struct ref_update *ref_transaction_add_update( oidcpy(&update->new_oid, new_oid); if ((flags & REF_HAVE_OLD) && old_oid) oidcpy(&update->old_oid, old_oid); - if (!(flags & REF_SKIP_CREATE_REFLOG)) + if (!(flags & REF_SKIP_CREATE_REFLOG)) { + update->committer_info = xstrdup_or_null(committer_info); update->msg = normalize_reflog_message(msg); + } return update; } @@ -1253,7 +1256,7 @@ int ref_transaction_update(struct ref_transaction *transaction, ref_transaction_add_update(transaction, refname, flags, new_oid, old_oid, new_target, - old_target, msg); + old_target, NULL, msg); return 0; } diff --git a/refs/files-backend.c b/refs/files-backend.c index 02cb4907d8659e87a227fed4f60a5f6606be8764..255fed8354cae982f785b1b85340e2a1eeecf2a6 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -1270,7 +1270,7 @@ static void prune_ref(struct files_ref_store *refs, struct ref_to_prune *r) ref_transaction_add_update( transaction, r->name, REF_NO_DEREF | REF_HAVE_NEW | REF_HAVE_OLD | REF_IS_PRUNING, - null_oid(), &r->oid, NULL, NULL, NULL); + null_oid(), &r->oid, NULL, NULL, NULL, NULL); if (ref_transaction_commit(transaction, &err)) goto cleanup; @@ -2417,7 +2417,7 @@ static int split_head_update(struct ref_update *update, transaction, "HEAD", update->flags | REF_LOG_ONLY | REF_NO_DEREF, &update->new_oid, &update->old_oid, - NULL, NULL, update->msg); + NULL, NULL, update->committer_info, update->msg); /* * Add "HEAD". This insertion is O(N) in the transaction @@ -2481,7 +2481,8 @@ static int split_symref_update(struct ref_update *update, transaction, referent, new_flags, update->new_target ? NULL : &update->new_oid, update->old_target ? NULL : &update->old_oid, - update->new_target, update->old_target, update->msg); + update->new_target, update->old_target, NULL, + update->msg); new_update->parent_update = update; @@ -2914,7 +2915,7 @@ static int files_transaction_prepare(struct ref_store *ref_store, packed_transaction, update->refname, REF_HAVE_NEW | REF_NO_DEREF, &update->new_oid, NULL, - NULL, NULL, NULL); + NULL, NULL, NULL, NULL); } } @@ -3094,12 +3095,13 @@ static int files_transaction_finish_initial(struct files_ref_store *refs, ref_transaction_add_update(loose_transaction, update->refname, update->flags & ~REF_HAVE_OLD, update->new_target ? NULL : &update->new_oid, NULL, - update->new_target, NULL, NULL); + update->new_target, NULL, update->committer_info, + NULL); } else { ref_transaction_add_update(packed_transaction, update->refname, update->flags & ~REF_HAVE_OLD, &update->new_oid, &update->old_oid, - NULL, NULL, NULL); + NULL, NULL, update->committer_info, NULL); } } diff --git a/refs/refs-internal.h b/refs/refs-internal.h index f5c733d099f0c6f1076a25f4f77d9d5eb345ec87..79b287c5ec5c7d8f759869cf93cda405640186dc 100644 --- a/refs/refs-internal.h +++ b/refs/refs-internal.h @@ -162,6 +162,7 @@ struct ref_update *ref_transaction_add_update( const struct object_id *new_oid, const struct object_id *old_oid, const char *new_target, const char *old_target, + const char *committer_info, const char *msg); /* diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c index c008f20be719fec3af6a8f81c821cb9c263764d7..b2e3ba877de9e59fea5a4d066eb13e60ef22a32b 100644 --- a/refs/reftable-backend.c +++ b/refs/reftable-backend.c @@ -1078,7 +1078,8 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store, new_update = ref_transaction_add_update( transaction, "HEAD", u->flags | REF_LOG_ONLY | REF_NO_DEREF, - &u->new_oid, &u->old_oid, NULL, NULL, u->msg); + &u->new_oid, &u->old_oid, NULL, NULL, NULL, + u->msg); string_list_insert(&affected_refnames, new_update->refname); } @@ -1161,7 +1162,8 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store, transaction, referent.buf, new_flags, u->new_target ? NULL : &u->new_oid, u->old_target ? NULL : &u->old_oid, - u->new_target, u->old_target, u->msg); + u->new_target, u->old_target, + u->committer_info, u->msg); new_update->parent_update = u; From patchwork Mon Dec 16 16:44:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: karthik nayak X-Patchwork-Id: 13910072 Received: from mail-ej1-f46.google.com (mail-ej1-f46.google.com [209.85.218.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 77ED120E02B for ; Mon, 16 Dec 2024 16:44:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.46 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734367485; cv=none; b=jXmjhiPoHfQfCBpNhWDNZal1XhUaFtNx0ZpRgJxldMB+tdLPw/T6esnHMzPbIA0R3nyxtcnGtrLNAOif75vWZk0ZGAsg9TvjcHtXfOiVw4aAkYS2yd7qYLb3chys1Lawzm7lbBu24gmJsvGpjN3/q+ki4InJFafhcv1PnbjXOyA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734367485; c=relaxed/simple; bh=KzZfc3eupGdaCgu8IhgU0wTKwFN0zZpW2VhtYjGjgj0=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=YzFnIQ/HixBflgLtCZhEOlWeGshsoMxiBt4gU8N7NB1FhK9oORomdSQpVGon3lTjx8MV2dW1TxT6Vh6P3QcMLVNr6V7285GGN6oqyCNKnPGP1fqnzCah3FDDdyWhTKkxy4VxyWY+tpJOl7fzZy3vkgx6h0xPJKuM6wwk+K741Mc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=IoIyVcKZ; arc=none smtp.client-ip=209.85.218.46 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="IoIyVcKZ" Received: by mail-ej1-f46.google.com with SMTP id a640c23a62f3a-aa670ffe302so812163566b.2 for ; Mon, 16 Dec 2024 08:44:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734367480; x=1734972280; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=JbLgtM6fZRugS/23dnDGKiBeriksEAzT/0GxzT9Q5yo=; b=IoIyVcKZBtAw8DxLkGLdjezgwSxReItCnr7gowiS5JPn8VUQ1EMws79gvM0370vTkK yx8eCQgrrf4UJXLToPZNk5IRz5iQuDZ6yoH3CYu6VJIcgcAijeSXC6uO5TQj/t+RxcRY XD49edR6Gmtdh0PlAKP/4aJqzVWgQO7x7ih+pjhv30+K8VLZgBAJBox3UOoOk6ejTR9c z4C3GOVnD7LUNA53QvZBb38XQeXoUAxU2csrIY5858UsX9VMUKg2Y9HXPRJ93qL8G1EZ k0OgRPJQk1laTfnxGzGweAYYkMl5TtQy4KV/3bObdBxnWMOEVUsxyR+ZtUEONj5MPhyR GeQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734367480; x=1734972280; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JbLgtM6fZRugS/23dnDGKiBeriksEAzT/0GxzT9Q5yo=; b=QLSVREpyqY6qOOfU+1WB0Jn52yJQr84dfSL+hhCK7FGywzKwCzgj1mSJvjT6VZQG5k p4DM6Pkss5RFzR0e++x8lFnJmeLVscalVTy/XZ+p6ZojcimA6CoYkTNJUtN6lk4H256p g3lViQ1BMXz0+9kyc1mxzJVYaSXPnoEr7oot+bDtPXW6Sl+Ky11ti833nx8hja5JSbdM CDdpanZXEkiEnZXLJj30ty/uDpajXNCAY5yq4p+/6jXQMK1TIbazyvKH6zl7d1zz0pIx b/j9XXrVdEaM1hkFTZrfV0i/07lrnMP9L0Z1N+gi96gJjSZ7Jr/D/Qwavx7SReByVdou 2Hbg== X-Gm-Message-State: AOJu0YyTLvkLEzoQf3HSzC9cM9qcourJVgdg0A5bYi9LS1xLoB9eSb1I IClkR1dQZwJ0Xv62reas1U9hvdP0LJZt4zG9e7WrWwCd/fCitX1BqAoZXJr/ X-Gm-Gg: ASbGncsZ6CrCguQQpW4pbZoKg0zQclIEm/WA034gnFzIv/7aXONa/JX8jjoFIOuLlzo iMVxHgXlbCVzm5fUCSOev8GX7uVqWSD+SNhywBauA2POXVIuLGZ1z1tqAh608R+WD7fkJx0vJX0 jbSWKz1/EqZ0uAPbO2fQCs6dQKL08p78YyvCCJ5cshovcHaV186az1CWnnXcdBbHfGOxtqjzeft zqhDbEyfp53gEW8zObmpbaI2s6ZVgO5s7kvTuKCMFkEOG9rIiNbIdK9t8o= X-Google-Smtp-Source: AGHT+IH2DZW4mrv/K5qe88/5zyluS6nQ06m0KxBo3qO6R55zOel8qjg4gKnGpaxwwlELK9dfl26Rjw== X-Received: by 2002:a17:906:c10b:b0:aa6:841a:dff0 with SMTP id a640c23a62f3a-aab779ba4f8mr1547977666b.32.1734367480415; Mon, 16 Dec 2024 08:44:40 -0800 (PST) Received: from [10.82.4.202] ([217.110.80.4]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aab9635957fsm354834366b.109.2024.12.16.08.44.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 16 Dec 2024 08:44:40 -0800 (PST) From: Karthik Nayak Date: Mon, 16 Dec 2024 17:44:31 +0100 Subject: [PATCH v4 6/8] refs: introduce the `ref_transaction_update_reflog` function Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241216-320-git-refs-migrate-reflogs-v4-6-d7cd3f197453@gmail.com> References: <20241216-320-git-refs-migrate-reflogs-v4-0-d7cd3f197453@gmail.com> In-Reply-To: <20241216-320-git-refs-migrate-reflogs-v4-0-d7cd3f197453@gmail.com> To: git@vger.kernel.org Cc: Karthik Nayak , ps@pks.im, Christian Couder X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=6145; i=karthik.188@gmail.com; h=from:subject:message-id; bh=KzZfc3eupGdaCgu8IhgU0wTKwFN0zZpW2VhtYjGjgj0=; b=owEB7QES/pANAwAIAT7VnySORox/AcsmYgBnYFjwO7CJOsljWtADPbh2n5psDqeFZ/y2akPxg DUIRlb2wd+JAbMEAAEIAB0WIQRXzkx/Y3VxD8tlxgY+1Z8kjkaMfwUCZ2BY8AAKCRA+1Z8kjkaM f1ESC/oDqppgxXdB4BOr1RuwWmCqylhn95hrliiv4PecN4Id/A2c1Pz4xSIA8GObNv4XeiP6VE4 DGeeM+JlyCk6L79z8xV7WH+U+pLkP4EptMntZXXnDslb0Uan/8iJlxBvV3aw5wNtIrzwgI9yqHS hHT7EVrWzTK/ic3g/mknNUSSk/LJ+Mn55y4hkDZ3CjcF1Eg/cjMWQ775t0eGe1fkZXnotXIv2WW kDhNYrEcm1DNSdSdOfai0tbc02T65PXisUG5hZLnBl/haH/gLYTMtP1ip6LrGIEgJ3zRTWONDHo IdYxU1UHAuZ7cdmJ9quCvqh6KhK8b5xgho2Mqvqjr1ccviSRqImYTbndrL1yhbpQ9DVV4y99XxK HhJNMWtvcU9ZYQ6eNJHOx6HP4qRvLuz7yGiZ9TU0x5vuqeiVjTqHFCKeXlHEHscE0nGQOZaQYgB BsQSZ81NXewdaIZgadKu6LEMxZRpYeDwlHv0bCvIWs/OotilPjE6ky2k7f/SbKISzUYcI= X-Developer-Key: i=karthik.188@gmail.com; a=openpgp; fpr=57CE4C7F6375710FCB65C6063ED59F248E468C7F Introduce a new function `ref_transaction_update_reflog`, for clients to add a reflog update to a transaction. While the existing function `ref_transaction_update` also allows clients to add a reflog entry, this function does a few things more, It: - Enforces that only a reflog entry is added and does not update the ref itself. - Allows the users to also provide the committer information. This means clients can add reflog entries with custom committer information. The `transaction_refname_valid()` function also modifies the error message selectively based on the type of the update. This change also affects reflog updates which go through `ref_transaction_update()`. A follow up commit will utilize this function to add reflog support to `git refs migrate`. Signed-off-by: Karthik Nayak --- refs.c | 39 +++++++++++++++++++++++++++++++++++---- refs.h | 14 ++++++++++++++ refs/files-backend.c | 24 ++++++++++++++++-------- 3 files changed, 65 insertions(+), 12 deletions(-) diff --git a/refs.c b/refs.c index 782bf1090af65196263a3c35ed18d878bb4f2967..8b3882cff17e5e3b0376f75654e32f81a23e5cb2 100644 --- a/refs.c +++ b/refs.c @@ -1207,14 +1207,14 @@ static int transaction_refname_valid(const char *refname, return 1; if (is_pseudo_ref(refname)) { - strbuf_addf(err, _("refusing to update pseudoref '%s'"), - refname); + const char *what = flags & REF_LOG_ONLY ? "reflog for pseudoref" : "pseudoref"; + strbuf_addf(err, _("refusing to update %s '%s'"), what, refname); return 0; } else if ((new_oid && !is_null_oid(new_oid)) ? check_refname_format(refname, REFNAME_ALLOW_ONELEVEL) : !refname_is_safe(refname)) { - strbuf_addf(err, _("refusing to update ref with bad name '%s'"), - refname); + const char *what = flags & REF_LOG_ONLY ? "reflog with bad name" : "ref with bad name"; + strbuf_addf(err, _("refusing to update %s '%s'"), what, refname); return 0; } @@ -1257,6 +1257,37 @@ int ref_transaction_update(struct ref_transaction *transaction, ref_transaction_add_update(transaction, refname, flags, new_oid, old_oid, new_target, old_target, NULL, msg); + + return 0; +} + +int ref_transaction_update_reflog(struct ref_transaction *transaction, + const char *refname, + const struct object_id *new_oid, + const struct object_id *old_oid, + const char *committer_info, unsigned int flags, + const char *msg, unsigned int index, + struct strbuf *err) +{ + struct ref_update *update; + + assert(err); + + flags |= REF_LOG_ONLY | REF_NO_DEREF; + + if (!transaction_refname_valid(refname, new_oid, flags, err)) + return -1; + + update = ref_transaction_add_update(transaction, refname, flags, + new_oid, old_oid, NULL, NULL, + committer_info, msg); + /* + * While we do set the old_oid value, we unset the flag to skip + * old_oid verification which only makes sense for refs. + */ + update->flags &= ~REF_HAVE_OLD; + update->index = index; + return 0; } diff --git a/refs.h b/refs.h index a5bedf48cf6de91005a7e8d0bf58ca98350397a6..b0dfc65ed2e59c4b66967840339f81e7746a96d3 100644 --- a/refs.h +++ b/refs.h @@ -727,6 +727,20 @@ int ref_transaction_update(struct ref_transaction *transaction, unsigned int flags, const char *msg, struct strbuf *err); +/* + * Similar to`ref_transaction_update`, but this function is only for adding + * a reflog update. Supports providing custom committer information. The index + * field can be utiltized to order updates as desired. When not used, the + * updates default to being ordered by refname. + */ +int ref_transaction_update_reflog(struct ref_transaction *transaction, + const char *refname, + const struct object_id *new_oid, + const struct object_id *old_oid, + const char *committer_info, unsigned int flags, + const char *msg, unsigned int index, + struct strbuf *err); + /* * Add a reference creation to transaction. new_oid is the value that * the reference should have after the update; it must not be diff --git a/refs/files-backend.c b/refs/files-backend.c index 255fed8354cae982f785b1b85340e2a1eeecf2a6..c11213f52065bcf2fa7612df8f9500692ee2d02c 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -3080,10 +3080,12 @@ static int files_transaction_finish_initial(struct files_ref_store *refs, } /* - * packed-refs don't support symbolic refs and root refs, so we - * have to queue these references via the loose transaction. + * packed-refs don't support symbolic refs, root refs and reflogs, + * so we have to queue these references via the loose transaction. */ - if (update->new_target || is_root_ref(update->refname)) { + if (update->new_target || + is_root_ref(update->refname) || + (update->flags & REF_LOG_ONLY)) { if (!loose_transaction) { loose_transaction = ref_store_transaction_begin(&refs->base, 0, err); if (!loose_transaction) { @@ -3092,11 +3094,17 @@ static int files_transaction_finish_initial(struct files_ref_store *refs, } } - ref_transaction_add_update(loose_transaction, update->refname, - update->flags & ~REF_HAVE_OLD, - update->new_target ? NULL : &update->new_oid, NULL, - update->new_target, NULL, update->committer_info, - NULL); + if (update->flags & REF_LOG_ONLY) + ref_transaction_add_update(loose_transaction, update->refname, + update->flags, &update->new_oid, + &update->old_oid, NULL, NULL, + update->committer_info, update->msg); + else + ref_transaction_add_update(loose_transaction, update->refname, + update->flags & ~REF_HAVE_OLD, + update->new_target ? NULL : &update->new_oid, NULL, + update->new_target, NULL, update->committer_info, + NULL); } else { ref_transaction_add_update(packed_transaction, update->refname, update->flags & ~REF_HAVE_OLD, From patchwork Mon Dec 16 16:44:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: karthik nayak X-Patchwork-Id: 13910071 Received: from mail-ej1-f51.google.com (mail-ej1-f51.google.com [209.85.218.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 36EED2080F4 for ; Mon, 16 Dec 2024 16:44:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734367485; cv=none; b=geDjcuIlBKJWsaJuqXw7bGBG5HnO2cDvwssMEmX+OcL0lIEvHLreFB8Lm51/0DA9ul8kKs7e1WryY+Wvn1gCFOCakAXKZIZRewsI3aSt0JABZ/LDrvTGyAZTWntpw6YssUidH8Zba90iHOU6jIssiWSnHaMN8c/sWVuek9AWJsM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734367485; c=relaxed/simple; bh=jDPMqKatsQhg+EBjNPw/kC4fYprM2j3L9khrIu5BB0Q=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=FQIhygA2+CCc7Go8hzHTVcVqfFTY5hWNTpZC2P+yOjXx9Qw/ej8qXIJShxkU/lCq45tOij0lxaiBWgttFx1wVAC2dAXuDXR0sRcwFqAwg1Io3wg3zl3SX1UvVi5OUsmFru9jCfSuc6t19mvcEVVqzRBloiDyRZ6oMD7A3NuC2+k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Cc8Dc/TF; arc=none smtp.client-ip=209.85.218.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Cc8Dc/TF" Received: by mail-ej1-f51.google.com with SMTP id a640c23a62f3a-aa684b6d9c7so712914466b.2 for ; Mon, 16 Dec 2024 08:44:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734367481; x=1734972281; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=hFja4Dce2bjeu8KBc7aHCgLjbi7WVN8kJntGzJ+xVtg=; b=Cc8Dc/TFsyduJmzcx8OxqJ7M2nJDFOni1HQouGbrLkX4PRkcaSLSJLOLCHBvnqbJn4 GANz3FtcUCpz1xjhHRPYQtaXl/akrqVTOQ0xfGo9mdy2V7kb6q5rEKHO94YCUSLRbHCb mae2/L1+Qcg+eOQsUA4MAysZiKmqwepiWN0rr7Vydm20alMcGFKIlZpO2ghS4uPrR64o 2x7Zko9xYf6cqSG+OPSp3unuagbF+b47gASy9/OzqJL3S0CfUEx9OASUy/yz8uXXZw6B 2RoZENoP0Gedm8IDzF3ghvy2z50d0WkL0RiW7bvJBiLcq1HEtGJlE0mrYFqIK5rAw6mO pbyA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734367481; x=1734972281; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hFja4Dce2bjeu8KBc7aHCgLjbi7WVN8kJntGzJ+xVtg=; b=O1pGbr/rwh9XtsJC4AC8muSAeUIcDXdymmL4mTXWH6Am7SAAnzJF7GOleacQUMoyKb uJtMLsAQO2KH/wtrYlvG+eENG5ZdbZO9RNT1WKMmIuxkeEH9SxE0VfY/iTynyCzQaJMq 3yUoV+T1as/iTOfdaQ+wm5noA9w8lY9IwrEma/OaCaS815TkZ1g2Zplm7hGMtUPdVXl3 +3j+/IQ71MdsGlMn6PgnQS6ZJszCVTbuuWa8AEcD+lv38yAWyR84DpxBrpuIUJ2R07a9 MTtv2kz/B98LuiO4rlTEQFBh7Q7F7fleBO0e/UgBGhBCj6a/ga/YtcME8J3kqL45fpX5 +vcQ== X-Gm-Message-State: AOJu0YyyXMn5EpDhRWzDUYSjDrUV+shsZlLiXAeq8dDmJwEeMTAwKJh5 h+ponLW2YR5wRM6Q1xwA1BtOTiFLae/iEAbHuCZT92KVIvIxufA381P3E9Be X-Gm-Gg: ASbGncvnUVmTt0BSkIbS0KaJFUipF+8o/MAIxYHuu18QcnKkB43qcaT6m+8xOJd3SBx C1NQfO7QmZckfAH7QpXnzy7K58RUBhHd/RjVf/eqLCrIee3Brzk3WOF27piOBWkuAExHGeqxl3l wbRUOu9wDvrmlGq36wtLwjVpTU6WeTGMswofoHKsTr1cr5hk7hTchutNOely8VkSzwei7dCHQ+R y9VC/mPg4WGHu+7AiC6j4j4ztRs6Ehdu1iPV/DTWuKNpgB840mZjVpoeAw= X-Google-Smtp-Source: AGHT+IG0m9MdR8R5euzWpztVUxgFnEhI3CBZx+6N524QhYyxrXk+lu1PllEUoUrg3Ctl4VGiLR0ieg== X-Received: by 2002:a17:907:780b:b0:aa6:8fa5:f3f5 with SMTP id a640c23a62f3a-aab77909c78mr1214061866b.15.1734367481061; Mon, 16 Dec 2024 08:44:41 -0800 (PST) Received: from [10.82.4.202] ([217.110.80.4]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aab9635957fsm354834366b.109.2024.12.16.08.44.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 16 Dec 2024 08:44:40 -0800 (PST) From: Karthik Nayak Date: Mon, 16 Dec 2024 17:44:32 +0100 Subject: [PATCH v4 7/8] refs: allow multiple reflog entries for the same refname Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241216-320-git-refs-migrate-reflogs-v4-7-d7cd3f197453@gmail.com> References: <20241216-320-git-refs-migrate-reflogs-v4-0-d7cd3f197453@gmail.com> In-Reply-To: <20241216-320-git-refs-migrate-reflogs-v4-0-d7cd3f197453@gmail.com> To: git@vger.kernel.org Cc: Karthik Nayak , ps@pks.im, Christian Couder X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=4945; i=karthik.188@gmail.com; h=from:subject:message-id; bh=jDPMqKatsQhg+EBjNPw/kC4fYprM2j3L9khrIu5BB0Q=; b=owEB7QES/pANAwAIAT7VnySORox/AcsmYgBnYFjwPyYaq0W+TAdPm0kqK+Cfotz/LtZJzdVM8 Vieyzu0B5iJAbMEAAEIAB0WIQRXzkx/Y3VxD8tlxgY+1Z8kjkaMfwUCZ2BY8AAKCRA+1Z8kjkaM f+auDAChVzlTSvUHxW6oN8cokduDxWNNcvvP2lSz8q+a79U6xVQGczyaXxirwK305WqRZzyGcHf p7ph9k8SZl0VQJUquTNNwyL2p8TG5WMGwVxFW6hrAPx4TpC2JebUOY/Y+ZwlPp1KT5ZOLCC980z e0F8/ymOwWCvgM0YOERndSTfcE3IUtWPzsPL+7dIJ45kGpikJIi+M/bbgDiq+r+nRfUDX0p2+3u Vk7/YyjeU1m0Ix1aDyIbMN0xUHtZLM1mkt52dxRQWOmi+X6vgT93pXef1r/vg5TqCgbwqbpEYjT 23lMbI0R4UJnmldwZNu/eEoOkHpZkjZIZNcZSoofvYD+RauFSNe1lkhxZNDkBxwP0ynKhM1wpkq WmxvF1g6gRPWS2UNsQhUkhTG4ZCRMCJgqDMmE1uLLW0TKnSn4I8USj08oIrTEHHDE93jFq+w20k 2ysEbChtSOHD8LDrPvP4bpvD9rziZEE14bV9RAUBFKNjfyZ9QrJy1Nuaa/wbwbJWnF+lo= X-Developer-Key: i=karthik.188@gmail.com; a=openpgp; fpr=57CE4C7F6375710FCB65C6063ED59F248E468C7F The reference transaction only allows a single update for a given reference to avoid conflicts. This, however, isn't an issue for reflogs. There are no conflicts to be resolved in reflogs and when migrating reflogs between backends we'd have multiple reflog entries for the same refname. So allow multiple reflog updates within a single transaction. Also the reflog creation logic isn't exposed to the end user. While this might change in the future, currently, this reduces the scope of issues to think about. In the reftable backend, the writer sorts all updates based on the update_index before writing to the block. When there are multiple reflogs for a given refname, it is essential that the order of the reflogs is maintained. So add the `index` value to the `update_index`. The `index` field is only set when multiple reflog entries for a given refname are added and as such in most scenarios the old behavior remains. This is required to add reflog migration support to `git refs migrate`. Signed-off-by: Karthik Nayak --- refs/files-backend.c | 15 +++++++++++---- refs/reftable-backend.c | 22 +++++++++++++++++++--- 2 files changed, 30 insertions(+), 7 deletions(-) diff --git a/refs/files-backend.c b/refs/files-backend.c index c11213f52065bcf2fa7612df8f9500692ee2d02c..8953d1c6d37b13b0db701888b3db92fd87a68aaa 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -2611,6 +2611,9 @@ static int lock_ref_for_update(struct files_ref_store *refs, update->backend_data = lock; + if (update->flags & REF_LOG_ONLY) + goto out; + if (update->type & REF_ISSYMREF) { if (update->flags & REF_NO_DEREF) { /* @@ -2829,13 +2832,16 @@ static int files_transaction_prepare(struct ref_store *ref_store, */ for (i = 0; i < transaction->nr; i++) { struct ref_update *update = transaction->updates[i]; - struct string_list_item *item = - string_list_append(&affected_refnames, update->refname); + struct string_list_item *item; if ((update->flags & REF_IS_PRUNING) && !(update->flags & REF_NO_DEREF)) BUG("REF_IS_PRUNING set without REF_NO_DEREF"); + if (update->flags & REF_LOG_ONLY) + continue; + + item = string_list_append(&affected_refnames, update->refname); /* * We store a pointer to update in item->util, but at * the moment we never use the value of this field @@ -3035,8 +3041,9 @@ static int files_transaction_finish_initial(struct files_ref_store *refs, /* Fail if a refname appears more than once in the transaction: */ for (i = 0; i < transaction->nr; i++) - string_list_append(&affected_refnames, - transaction->updates[i]->refname); + if (!(transaction->updates[i]->flags & REF_LOG_ONLY)) + string_list_append(&affected_refnames, + transaction->updates[i]->refname); string_list_sort(&affected_refnames); if (ref_update_reject_duplicates(&affected_refnames, err)) { ret = TRANSACTION_GENERIC_ERROR; diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c index b2e3ba877de9e59fea5a4d066eb13e60ef22a32b..bec5962debea7b62572d08f6fa8fd38ab4cd8af6 100644 --- a/refs/reftable-backend.c +++ b/refs/reftable-backend.c @@ -990,8 +990,9 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store, if (ret) goto done; - string_list_append(&affected_refnames, - transaction->updates[i]->refname); + if (!(transaction->updates[i]->flags & REF_LOG_ONLY)) + string_list_append(&affected_refnames, + transaction->updates[i]->refname); } /* @@ -1301,6 +1302,7 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data struct reftable_log_record *logs = NULL; struct ident_split committer_ident = {0}; size_t logs_nr = 0, logs_alloc = 0, i; + uint64_t max_update_index = ts; const char *committer_info; int ret = 0; @@ -1405,7 +1407,19 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data } fill_reftable_log_record(log, &c); - log->update_index = ts; + + /* + * Updates are sorted by the writer. So updates for the same + * refname need to contain different update indices. + */ + log->update_index = ts + u->index; + + /* + * Note the max update_index so the limit can be set later on. + */ + if (log->update_index > max_update_index) + max_update_index = log->update_index; + log->refname = xstrdup(u->refname); memcpy(log->value.update.new_hash, u->new_oid.hash, GIT_MAX_RAWSZ); @@ -1469,6 +1483,8 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data * and log blocks. */ if (logs) { + reftable_writer_set_limits(writer, ts, max_update_index); + ret = reftable_writer_add_logs(writer, logs, logs_nr); if (ret < 0) goto done; From patchwork Mon Dec 16 16:44:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: karthik nayak X-Patchwork-Id: 13910073 Received: from mail-ej1-f47.google.com (mail-ej1-f47.google.com [209.85.218.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0395920E038 for ; Mon, 16 Dec 2024 16:44:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.47 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734367486; cv=none; b=Hkkxe/0OoXdIHyqqIPJ5S+2gbpCVZbgiyYlrzYn9/6LxEihQtG6makH7OnKdF4g13fVipUxL3lyNDe8wC+HNUVCQs81W1ttC84cOLpJhsT0dbMazHJt4si6aY8n94Vaf1C0UWamrD5ujUozFItpJWaPo2bDlhhv3cri7e4lmvIg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734367486; c=relaxed/simple; bh=pQgBGaG3lRxPgmTt/jE9VJu9X4iPfd9VMFBjrbINC70=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=NnGHnz8/BD2PObJg2FjFJJSEH8dQKXM7hxD0B+GJBRyRQitzMHiBnWwvrca+CKYWEYDyqDVW2kzLMBFjK4uQEFeQCenN4mYwRzG3r2KrD/FDk8XriuBueyhel88r6lXwBoeR4v0RnTJAgVaNXYU9t+4FFwWqo6K+3EggLxvktBQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=MSzpE/hc; arc=none smtp.client-ip=209.85.218.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="MSzpE/hc" Received: by mail-ej1-f47.google.com with SMTP id a640c23a62f3a-aa6c0d1833eso753216966b.1 for ; Mon, 16 Dec 2024 08:44:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734367482; x=1734972282; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=9rty06NC7hnmumKMwmmrJr/mk37YTFLOxIExFq/Z5Ok=; b=MSzpE/hcYrWM+scVPKWr1apouB57MWELEm6nNLN2YbDgm4EpoF3nMwVBdA/eOlyjt5 MFBvTxkNiN5TcHUxtlWVm+hdXuSK1D65L9x/c5OR70r7+yPNR4SgdQtC6z/pOUBZx5x3 K5afVlHnwahcOqMFj+M6CqJ6sNHAzgR/hAKTcx9rb1Xay7Azvgx06rnM/5/e+t1eSfhM ePTzpSJ8l3/GT9qQFcu13MX7bWM4Nsy7vzJUaS+Gxgex0Zxq9d0Vdho2P3Y6nEugM+8O TJwii4/5kUSkC/C5/6wUkxS0a2gjFEEoegm4KV+EOEKRfLXsOIo+p5OMVmink6jBKf0v k7mA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734367482; x=1734972282; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9rty06NC7hnmumKMwmmrJr/mk37YTFLOxIExFq/Z5Ok=; b=jACtL1p3lKDF68lGUqX56iXxPpXQPTlT0fV15rJhJZp7U/qSlr/saJVTi+Slp3ncgJ cu+ZgQJyM/VcgI+NoOXa2rs2EqIeqCJ9J5m/b9g9I6d3khhL15rYcxh4vuzUyqdh3h8F 85OzP+gmQx6jJAMUDgl6f4MlNNDVpAUNAj+YhV31ilDieu9+pKVwSO1tq02EPyX+EQ/i e1sb9N08D/DmXVneSTDK7uKt1cZJLmnym6o43t47vzJ5qVyry+fwarrBysGvLmnfL1HZ XCf5mqmzADxYUDZw6GyJ1zSxlCz00MacwxJDDzbvbQwzgA02Mm7t0W8PIQ1C/cl2rfzV pRXA== X-Gm-Message-State: AOJu0YyN2ZjUwe8Kz94CmM7eP5yA8apw4ORwfMx4IpUPWzpqw/D58U7i 3SIN+1cz/JQzqZvtp7QSPN4MzBU4KXYgGjNyS56uDjfx6xEuISG5gui9AK1d X-Gm-Gg: ASbGncsdIwnLtJJKlzB0z9OlJSruGO+75uAIlX1XRxl5+ff0gPs+KrBgayVjFO2KWMQ 35f0md6H3mpcAr/AjJOkp2oU3PMe3VdbQ10uawkmqoVC/8rm6ctxuASQVrB4ZEvxgIs/nWF1yY8 HbAxOPdXa8OTPL3I9udAzpgQoTc9qasfsU/dswfHKUMx7ujISiGigLjRwU7zK22ZWknqa8lPt3m zrAd18S71G7RkXXDNHI1HMUX4D7Npr9O0He8iQlLc1zjFTqsnmaxYNt4JA= X-Google-Smtp-Source: AGHT+IHrqlytzxMTC4eCe1Kke4nwqRppMhkaTBZAn1oBxhxlg4PN1gAKkdRpIt6rOBfrnok1FPL6sQ== X-Received: by 2002:a17:906:309a:b0:aa6:ab70:4a78 with SMTP id a640c23a62f3a-aabdb885361mr28553766b.37.1734367481861; Mon, 16 Dec 2024 08:44:41 -0800 (PST) Received: from [10.82.4.202] ([217.110.80.4]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aab9635957fsm354834366b.109.2024.12.16.08.44.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 16 Dec 2024 08:44:41 -0800 (PST) From: Karthik Nayak Date: Mon, 16 Dec 2024 17:44:33 +0100 Subject: [PATCH v4 8/8] refs: add support for migrating reflogs Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241216-320-git-refs-migrate-reflogs-v4-8-d7cd3f197453@gmail.com> References: <20241216-320-git-refs-migrate-reflogs-v4-0-d7cd3f197453@gmail.com> In-Reply-To: <20241216-320-git-refs-migrate-reflogs-v4-0-d7cd3f197453@gmail.com> To: git@vger.kernel.org Cc: Karthik Nayak , ps@pks.im, Christian Couder X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=11375; i=karthik.188@gmail.com; h=from:subject:message-id; bh=pQgBGaG3lRxPgmTt/jE9VJu9X4iPfd9VMFBjrbINC70=; b=owEB7QES/pANAwAIAT7VnySORox/AcsmYgBnYFjwyxepYeYkXtZqfMNhkEjnMi19WRGTDjUi0 fEOywu7NEaJAbMEAAEIAB0WIQRXzkx/Y3VxD8tlxgY+1Z8kjkaMfwUCZ2BY8AAKCRA+1Z8kjkaM fwhZC/0R8l28qRhh25bIeyawY7LynGm1f8NcQ7yKLNs4tJvEADFqbIlmcwzFTuosRCksXkc8uBW +j98EJGHh63F3xPmf1j41/rz1/ts0WXQbgbKR1CRqDIW8QcjaYlvPjD2yXEJJuBcRapv3K95G1o 2eCYOGBNTg90NmMaiG7uIEOfq/EZSEDtypr07AOBwiqbfxLxZBQWSBrYEM3suFiu5A5vNaBOKeR nC8vWq+F5++6FX3YYeR8j9UrUPVGClunOTk93yOt/vHZNJpR64vfJSPI09gdyV3HpPguikNy5Sw VWzf15mJZCPNMCOKsPnE/IIHkKxt6LHUWGffmDbjGDPbTrtzJvBQT4UveoTzMvZPPQ8L0DAmy5t p9CbHIQo6XHcZW/IFyrEYbAWulWh5p60h5MEtJHM/eL8kCjfkG+Zk81KAgo6Gop5okasJtcqZrh ALJdTiGFzcqtRxmpJVwwu41W6k/hF3rLdYKpzweKeuvGSqNIy0oRSfQ6FKYiivSTKhb1c= X-Developer-Key: i=karthik.188@gmail.com; a=openpgp; fpr=57CE4C7F6375710FCB65C6063ED59F248E468C7F The `git refs migrate` command was introduced in 25a0023f28 (builtin/refs: new command to migrate ref storage formats, 2024-06-06) to support migrating from one reference backend to another. One limitation of the command was that it didn't support migrating repositories which contained reflogs. A previous commit, added support for adding reflog updates in ref transactions. Using the added functionality bake in reflog support for `git refs migrate`. To ensure that the order of the reflogs is maintained during the migration, we add the index for each reflog update as we iterate over the reflogs from the old reference backend. This is to ensure that the order is maintained in the new backend. Helped-by: Patrick Steinhardt Signed-off-by: Karthik Nayak --- Documentation/git-refs.txt | 2 - refs.c | 92 ++++++++++++++++++++++++++++++++-------------- t/t1460-refs-migrate.sh | 73 +++++++++++++++++++++++++----------- 3 files changed, 115 insertions(+), 52 deletions(-) diff --git a/Documentation/git-refs.txt b/Documentation/git-refs.txt index ce31f93061db5e5d16aca516dd3d15f6527db870..9829984b0a4c4f54ec7f9b6c6c7072f62b1d198d 100644 --- a/Documentation/git-refs.txt +++ b/Documentation/git-refs.txt @@ -57,8 +57,6 @@ KNOWN LIMITATIONS The ref format migration has several known limitations in its current form: -* It is not possible to migrate repositories that have reflogs. - * It is not possible to migrate repositories that have worktrees. * There is no way to block concurrent writes to the repository during an diff --git a/refs.c b/refs.c index 8b3882cff17e5e3b0376f75654e32f81a23e5cb2..5d541ddc41aa84905e688c92565aa18fbf55323b 100644 --- a/refs.c +++ b/refs.c @@ -30,6 +30,7 @@ #include "date.h" #include "commit.h" #include "wildmatch.h" +#include "ident.h" /* * List of all available backends @@ -2673,6 +2674,7 @@ struct migration_data { struct ref_store *old_refs; struct ref_transaction *transaction; struct strbuf *errbuf; + struct strbuf sb; }; static int migrate_one_ref(const char *refname, const char *referent UNUSED, const struct object_id *oid, @@ -2705,6 +2707,52 @@ static int migrate_one_ref(const char *refname, const char *referent UNUSED, con return ret; } +struct reflog_migration_data { + unsigned int index; + const char *refname; + struct ref_store *old_refs; + struct ref_transaction *transaction; + struct strbuf *errbuf; + struct strbuf *sb; +}; + +static int migrate_one_reflog_entry(struct object_id *old_oid, + struct object_id *new_oid, + const char *committer, + timestamp_t timestamp, int tz, + const char *msg, void *cb_data) +{ + struct reflog_migration_data *data = cb_data; + const char *date; + int ret; + + date = show_date(timestamp, tz, DATE_MODE(NORMAL)); + strbuf_reset(data->sb); + /* committer contains name and email */ + strbuf_addstr(data->sb, fmt_ident("", committer, WANT_BLANK_IDENT, date, 0)); + + ret = ref_transaction_update_reflog(data->transaction, data->refname, + new_oid, old_oid, data->sb->buf, + REF_HAVE_NEW | REF_HAVE_OLD, msg, + data->index++, data->errbuf); + return ret; +} + +static int migrate_one_reflog(const char *refname, void *cb_data) +{ + struct migration_data *migration_data = cb_data; + struct reflog_migration_data data = { + .refname = refname, + .old_refs = migration_data->old_refs, + .transaction = migration_data->transaction, + .errbuf = migration_data->errbuf, + .sb = &migration_data->sb, + }; + + return refs_for_each_reflog_ent(migration_data->old_refs, refname, + migrate_one_reflog_entry, &data); +} + static int move_files(const char *from_path, const char *to_path, struct strbuf *errbuf) { struct strbuf from_buf = STRBUF_INIT, to_buf = STRBUF_INIT; @@ -2771,13 +2819,6 @@ static int move_files(const char *from_path, const char *to_path, struct strbuf return ret; } -static int count_reflogs(const char *reflog UNUSED, void *payload) -{ - size_t *reflog_count = payload; - (*reflog_count)++; - return 0; -} - static int has_worktrees(void) { struct worktree **worktrees = get_worktrees(); @@ -2802,8 +2843,9 @@ int repo_migrate_ref_storage_format(struct repository *repo, struct ref_store *old_refs = NULL, *new_refs = NULL; struct ref_transaction *transaction = NULL; struct strbuf new_gitdir = STRBUF_INIT; - struct migration_data data; - size_t reflog_count = 0; + struct migration_data data = { + .sb = STRBUF_INIT, + }; int did_migrate_refs = 0; int ret; @@ -2815,21 +2857,6 @@ int repo_migrate_ref_storage_format(struct repository *repo, old_refs = get_main_ref_store(repo); - /* - * We do not have any interfaces that would allow us to write many - * reflog entries. Once we have them we can remove this restriction. - */ - if (refs_for_each_reflog(old_refs, count_reflogs, &reflog_count) < 0) { - strbuf_addstr(errbuf, "cannot count reflogs"); - ret = -1; - goto done; - } - if (reflog_count) { - strbuf_addstr(errbuf, "migrating reflogs is not supported yet"); - ret = -1; - goto done; - } - /* * Worktrees complicate the migration because every worktree has a * separate ref storage. While it should be feasible to implement, this @@ -2855,17 +2882,21 @@ int repo_migrate_ref_storage_format(struct repository *repo, * This operation is safe as we do not yet modify the main * repository. * - * 3. If we're in dry-run mode then we are done and can hand over the + * 3. Enumerate all reflogs and write them into the new ref storage. + * This operation is safe as we do not yet modify the main + * repository. + * + * 4. If we're in dry-run mode then we are done and can hand over the * directory to the caller for inspection. If not, we now start * with the destructive part. * - * 4. Delete the old ref storage from disk. As we have a copy of refs + * 5. Delete the old ref storage from disk. As we have a copy of refs * in the new ref storage it's okay(ish) if we now get interrupted * as there is an equivalent copy of all refs available. * - * 5. Move the new ref storage files into place. + * 6. Move the new ref storage files into place. * - * 6. Change the repository format to the new ref format. + * 7. Change the repository format to the new ref format. */ strbuf_addf(&new_gitdir, "%s/%s", old_refs->gitdir, "ref_migration.XXXXXX"); if (!mkdtemp(new_gitdir.buf)) { @@ -2907,6 +2938,10 @@ int repo_migrate_ref_storage_format(struct repository *repo, if (ret < 0) goto done; + ret = refs_for_each_reflog(old_refs, migrate_one_reflog, &data); + if (ret < 0) + goto done; + ret = ref_transaction_commit(transaction, errbuf); if (ret < 0) goto done; @@ -2982,6 +3017,7 @@ int repo_migrate_ref_storage_format(struct repository *repo, } ref_transaction_free(transaction); strbuf_release(&new_gitdir); + strbuf_release(&data.sb); return ret; } diff --git a/t/t1460-refs-migrate.sh b/t/t1460-refs-migrate.sh index 1bfff3a7afd5acc470424dfe7ec3e97d45f5c481..f59bc4860f19c4af82dc6f2984bdb69d61fe3ec2 100755 --- a/t/t1460-refs-migrate.sh +++ b/t/t1460-refs-migrate.sh @@ -7,23 +7,44 @@ export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME . ./test-lib.sh +# Migrate the provided repository from one format to the other and +# verify that the references and logs are migrated over correctly. +# Usage: test_migration +# is the relative path to the repo to be migrated. +# is the ref format to be migrated to. +# (true or false) whether to skip reflog verification. test_migration () { - git -C "$1" for-each-ref --include-root-refs \ + repo=$1 && + format=$2 && + skip_reflog_verify=${3:-false} && + git -C "$repo" for-each-ref --include-root-refs \ --format='%(refname) %(objectname) %(symref)' >expect && - git -C "$1" refs migrate --ref-format="$2" && - git -C "$1" for-each-ref --include-root-refs \ + if ! $skip_reflog_verify + then + git -C "$repo" reflog --all >expect_logs && + git -C "$repo" reflog list >expect_log_list + fi && + + git -C "$repo" refs migrate --ref-format="$2" && + + git -C "$repo" for-each-ref --include-root-refs \ --format='%(refname) %(objectname) %(symref)' >actual && test_cmp expect actual && + if ! $skip_reflog_verify + then + git -C "$repo" reflog --all >actual_logs && + git -C "$repo" reflog list >actual_log_list && + test_cmp expect_logs actual_logs && + test_cmp expect_log_list actual_log_list + fi && - git -C "$1" rev-parse --show-ref-format >actual && - echo "$2" >expect && + git -C "$repo" rev-parse --show-ref-format >actual && + echo "$format" >expect && test_cmp expect actual } test_expect_success 'setup' ' - rm -rf .git && - # The migration does not yet support reflogs. - git config --global core.logAllRefUpdates false + rm -rf .git ' test_expect_success "superfluous arguments" ' @@ -78,19 +99,6 @@ do test_cmp expect err ' - test_expect_success "$from_format -> $to_format: migration with reflog fails" ' - test_when_finished "rm -rf repo" && - git init --ref-format=$from_format repo && - test_config -C repo core.logAllRefUpdates true && - test_commit -C repo logged && - test_must_fail git -C repo refs migrate \ - --ref-format=$to_format 2>err && - cat >expect <<-EOF && - error: migrating reflogs is not supported yet - EOF - test_cmp expect err - ' - test_expect_success "$from_format -> $to_format: migration with worktree fails" ' test_when_finished "rm -rf repo" && git init --ref-format=$from_format repo && @@ -141,7 +149,7 @@ do test_commit -C repo initial && test-tool -C repo ref-store main update-ref "" refs/heads/broken \ "$(test_oid 001)" "$ZERO_OID" REF_SKIP_CREATE_REFLOG,REF_SKIP_OID_VERIFICATION && - test_migration repo "$to_format" && + test_migration repo "$to_format" true && test_oid 001 >expect && git -C repo rev-parse refs/heads/broken >actual && test_cmp expect actual @@ -195,6 +203,27 @@ do git -C repo rev-parse --show-ref-format >actual && test_cmp expect actual ' + + test_expect_success "$from_format -> $to_format: reflogs of symrefs with target deleted" ' + test_when_finished "rm -rf repo" && + git init --ref-format=$from_format repo && + test_commit -C repo initial && + git -C repo branch branch-1 HEAD && + git -C repo symbolic-ref refs/heads/symref refs/heads/branch-1 && + cat >input <<-EOF && + delete refs/heads/branch-1 + EOF + git -C repo update-ref --stdin $to_format: reflogs order is retained" ' + test_when_finished "rm -rf repo" && + git init --ref-format=$from_format repo && + test_commit --date "100005000 +0700" --no-tag -C repo initial && + test_commit --date "100003000 +0700" --no-tag -C repo second && + test_migration repo "$to_format" + ' done done