[v6,2/9] ref: check the full refname instead of basename

Message ID	ZxZYXpuCD2I_3bNh@ArchLinux (mailing list archive)
State	New
Headers	show Received: from mail-pg1-f177.google.com (mail-pg1-f177.google.com [209.85.215.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A90951F8F04 for <git@vger.kernel.org>; Mon, 21 Oct 2024 13:34:20 +0000 (UTC) Date: Mon, 21 Oct 2024 21:34:22 +0800 From: shejialuo <shejialuo@gmail.com> To: git@vger.kernel.org Cc: Patrick Steinhardt <ps@pks.im>, Karthik Nayak <karthik.188@gmail.com>, Junio C Hamano <gitster@pobox.com> Subject: [PATCH v6 2/9] ref: check the full refname instead of basename Message-ID: <ZxZYXpuCD2I_3bNh@ArchLinux> References: <ZxZX5HDdq_R0C77b@ArchLinux> Precedence: bulk MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <ZxZX5HDdq_R0C77b@ArchLinux>
Series	[v6,1/9] ref: initialize "fsck_ref_report" with zero \| expand [v6,1/9] ref: initialize "fsck_ref_report" with zero [v6,2/9] ref: check the full refname instead of basename [v6,3/9] ref: initialize target name outside of check functions [v6,4/9] ref: support multiple worktrees check for refs [v6,5/9] ref: port git-fsck(1) regular refs check for files backend [v6,6/9] ref: add more strict checks for regular refs [v6,7/9] ref: add basic symref content check for files backend [v6,8/9] ref: check whether the target of the symref is a ref [v6,9/9] ref: add symlink ref content check for files backend

Message ID

ZxZYXpuCD2I_3bNh@ArchLinux (mailing list archive)

State

New

Headers

Date: Mon, 21 Oct 2024 21:34:22 +0800
From: shejialuo <shejialuo@gmail.com>
To: git@vger.kernel.org
Cc: Patrick Steinhardt <ps@pks.im>, Karthik Nayak <karthik.188@gmail.com>,
	Junio C Hamano <gitster@pobox.com>
Subject: [PATCH v6 2/9] ref: check the full refname instead of basename
Message-ID: <ZxZYXpuCD2I_3bNh@ArchLinux>
References: <ZxZX5HDdq_R0C77b@ArchLinux>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <ZxZX5HDdq_R0C77b@ArchLinux>

Series

[v6,1/9] ref: initialize "fsck_ref_report" with zero | expand

Commit Message

shejialuo Oct. 21, 2024, 1:34 p.m. UTC

In "files-backend.c::files_fsck_refs_name", we validate the refname
format by using "check_refname_format" to check the basename of the
iterator with "REFNAME_ALLOW_ONELEVEL" flag.

However, this is a bad implementation. Although we doesn't allow a
single "@" in ".git" directory, we do allow "refs/heads/@". So, we will
report an error wrongly when there is a "refs/heads/@" ref by using one
level refname "@".

Because we just check one level refname, we either cannot check the
other parts of the full refname. And we will ignore the following
errors:

  "refs/heads/ new-feature/test"
  "refs/heads/~new-feature/test"

In order to fix the above problem, enhance "files_fsck_refs_name" to use
the full name for "check_refname_format". Then, replace the tests which
are related to "@" and add tests to exercise the above situations.

Mentored-by: Patrick Steinhardt <ps@pks.im>
Mentored-by: Karthik Nayak <karthik.188@gmail.com>
Signed-off-by: shejialuo <shejialuo@gmail.com>
---
 refs/files-backend.c     |  4 ++--
 t/t0602-reffiles-fsck.sh | 30 +++++++++++++++++++++++-------
 2 files changed, 25 insertions(+), 9 deletions(-)

Comments

karthik nayak Oct. 21, 2024, 3:38 p.m. UTC | #1

shejialuo <shejialuo@gmail.com> writes:

[snip]

> diff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh
> index 71a4d1a5ae..0aee377439 100755
> --- a/t/t0602-reffiles-fsck.sh
> +++ b/t/t0602-reffiles-fsck.sh
> @@ -25,6 +25,13 @@ test_expect_success 'ref name should be checked' '
>  	git tag tag-2 &&
>  	git tag multi_hierarchy/tag-2 &&
>
> +	cp $branch_dir_prefix/branch-1 $branch_dir_prefix/@ &&
> +	git refs verify 2>err &&
> +	cat >expect <<-EOF &&
> +	EOF
> +	test_must_be_empty err &&
> +	rm $branch_dir_prefix/@ &&
> +
>  	cp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&
>  	test_must_fail git refs verify 2>err &&
>  	cat >expect <<-EOF &&
> @@ -33,20 +40,20 @@ test_expect_success 'ref name should be checked' '
>  	rm $branch_dir_prefix/.branch-1 &&
>  	test_cmp expect err &&
>
> -	cp $branch_dir_prefix/branch-1 $branch_dir_prefix/@ &&
> +	cp $branch_dir_prefix/branch-1 $branch_dir_prefix/'\'' branch-1'\'' &&

Nit: Here and below we could use ${SQ} instead.

[snip]

shejialuo Oct. 22, 2024, 11:42 a.m. UTC | #2

On Mon, Oct 21, 2024 at 10:38:02AM -0500, karthik nayak wrote:
> shejialuo <shejialuo@gmail.com> writes:
> 
> [snip]
> 
> > diff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh
> > index 71a4d1a5ae..0aee377439 100755
> > --- a/t/t0602-reffiles-fsck.sh
> > +++ b/t/t0602-reffiles-fsck.sh
> > @@ -25,6 +25,13 @@ test_expect_success 'ref name should be checked' '
> >  	git tag tag-2 &&
> >  	git tag multi_hierarchy/tag-2 &&
> >
> > +	cp $branch_dir_prefix/branch-1 $branch_dir_prefix/@ &&
> > +	git refs verify 2>err &&
> > +	cat >expect <<-EOF &&
> > +	EOF
> > +	test_must_be_empty err &&
> > +	rm $branch_dir_prefix/@ &&
> > +
> >  	cp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&
> >  	test_must_fail git refs verify 2>err &&
> >  	cat >expect <<-EOF &&
> > @@ -33,20 +40,20 @@ test_expect_success 'ref name should be checked' '
> >  	rm $branch_dir_prefix/.branch-1 &&
> >  	test_cmp expect err &&
> >
> > -	cp $branch_dir_prefix/branch-1 $branch_dir_prefix/@ &&
> > +	cp $branch_dir_prefix/branch-1 $branch_dir_prefix/'\'' branch-1'\'' &&
> 
> Nit: Here and below we could use ${SQ} instead.
> 

I agree.

> [snip]

Patrick Steinhardt Nov. 5, 2024, 7:11 a.m. UTC | #3

On Mon, Oct 21, 2024 at 09:34:22PM +0800, shejialuo wrote:
> In "files-backend.c::files_fsck_refs_name", we validate the refname
> format by using "check_refname_format" to check the basename of the
> iterator with "REFNAME_ALLOW_ONELEVEL" flag.
> 
> However, this is a bad implementation. Although we doesn't allow a
> single "@" in ".git" directory, we do allow "refs/heads/@". So, we will
> report an error wrongly when there is a "refs/heads/@" ref by using one
> level refname "@".
> 
> Because we just check one level refname, we either cannot check the
> other parts of the full refname. And we will ignore the following
> errors:
> 
>   "refs/heads/ new-feature/test"
>   "refs/heads/~new-feature/test"
> 
> In order to fix the above problem, enhance "files_fsck_refs_name" to use
> the full name for "check_refname_format". Then, replace the tests which
> are related to "@" and add tests to exercise the above situations.

Okay, makes sense.

> diff --git a/refs/files-backend.c b/refs/files-backend.c
> index 03d2503276..f246c92684 100644
> --- a/refs/files-backend.c
> +++ b/refs/files-backend.c
> @@ -3519,10 +3519,10 @@ static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,
>  	if (iter->basename[0] != '.' && ends_with(iter->basename, ".lock"))
>  		goto cleanup;
>  
> -	if (check_refname_format(iter->basename, REFNAME_ALLOW_ONELEVEL)) {
> +	strbuf_addf(&sb, "%s/%s", refs_check_dir, iter->relative_path);
> +	if (check_refname_format(sb.buf, 0)) {
>  		struct fsck_ref_report report = { 0 };
>  
> -		strbuf_addf(&sb, "%s/%s", refs_check_dir, iter->relative_path);
>  		report.path = sb.buf;
>  		ret = fsck_report_ref(o, &report,
>  				      FSCK_MSG_BAD_REF_NAME,

So this only works right now because we never check root refs in the
first place? Maybe that is worth a comment.

> diff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh
> index 71a4d1a5ae..0aee377439 100755
> --- a/t/t0602-reffiles-fsck.sh
> +++ b/t/t0602-reffiles-fsck.sh
> @@ -25,6 +25,13 @@ test_expect_success 'ref name should be checked' '
>  	git tag tag-2 &&
>  	git tag multi_hierarchy/tag-2 &&
>  
> +	cp $branch_dir_prefix/branch-1 $branch_dir_prefix/@ &&
> +	git refs verify 2>err &&
> +	cat >expect <<-EOF &&
> +	EOF
> +	test_must_be_empty err &&
> +	rm $branch_dir_prefix/@ &&

`expect` isn't used here as you use `test_must_be_empty`.

>  	cp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&
>  	test_must_fail git refs verify 2>err &&
>  	cat >expect <<-EOF &&
> @@ -33,20 +40,20 @@ test_expect_success 'ref name should be checked' '
>  	rm $branch_dir_prefix/.branch-1 &&
>  	test_cmp expect err &&
>  
> -	cp $branch_dir_prefix/branch-1 $branch_dir_prefix/@ &&
> +	cp $branch_dir_prefix/branch-1 $branch_dir_prefix/'\'' branch-1'\'' &&
>  	test_must_fail git refs verify 2>err &&
>  	cat >expect <<-EOF &&
> -	error: refs/heads/@: badRefName: invalid refname format
> +	error: refs/heads/ branch-1: badRefName: invalid refname format
>  	EOF
> -	rm $branch_dir_prefix/@ &&
> +	rm $branch_dir_prefix/'\'' branch-1'\'' &&
>  	test_cmp expect err &&

Okay, we now allow `refs/heads/@`, but still don't allow other bad
formatting like spaces in the refname.

Patrick

shejialuo Nov. 6, 2024, 12:37 p.m. UTC | #4

On Tue, Nov 05, 2024 at 08:11:42AM +0100, Patrick Steinhardt wrote:

[snip]

> > diff --git a/refs/files-backend.c b/refs/files-backend.c
> > index 03d2503276..f246c92684 100644
> > --- a/refs/files-backend.c
> > +++ b/refs/files-backend.c
> > @@ -3519,10 +3519,10 @@ static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,
> >  	if (iter->basename[0] != '.' && ends_with(iter->basename, ".lock"))
> >  		goto cleanup;
> >  
> > -	if (check_refname_format(iter->basename, REFNAME_ALLOW_ONELEVEL)) {
> > +	strbuf_addf(&sb, "%s/%s", refs_check_dir, iter->relative_path);
> > +	if (check_refname_format(sb.buf, 0)) {
> >  		struct fsck_ref_report report = { 0 };
> >  
> > -		strbuf_addf(&sb, "%s/%s", refs_check_dir, iter->relative_path);
> >  		report.path = sb.buf;
> >  		ret = fsck_report_ref(o, &report,
> >  				      FSCK_MSG_BAD_REF_NAME,
> 
> So this only works right now because we never check root refs in the
> first place? Maybe that is worth a comment.
> 

Yes, I agree. I will improve this in the next version.

> > diff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh
> > index 71a4d1a5ae..0aee377439 100755
> > --- a/t/t0602-reffiles-fsck.sh
> > +++ b/t/t0602-reffiles-fsck.sh
> > @@ -25,6 +25,13 @@ test_expect_success 'ref name should be checked' '
> >  	git tag tag-2 &&
> >  	git tag multi_hierarchy/tag-2 &&
> >  
> > +	cp $branch_dir_prefix/branch-1 $branch_dir_prefix/@ &&
> > +	git refs verify 2>err &&
> > +	cat >expect <<-EOF &&
> > +	EOF
> > +	test_must_be_empty err &&
> > +	rm $branch_dir_prefix/@ &&
> 
> `expect` isn't used here as you use `test_must_be_empty`.
> 

Thanks, I will improve this in the next version.

> >  	cp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&
> >  	test_must_fail git refs verify 2>err &&
> >  	cat >expect <<-EOF &&
> > @@ -33,20 +40,20 @@ test_expect_success 'ref name should be checked' '
> >  	rm $branch_dir_prefix/.branch-1 &&
> >  	test_cmp expect err &&
> >  
> > -	cp $branch_dir_prefix/branch-1 $branch_dir_prefix/@ &&
> > +	cp $branch_dir_prefix/branch-1 $branch_dir_prefix/'\'' branch-1'\'' &&
> >  	test_must_fail git refs verify 2>err &&
> >  	cat >expect <<-EOF &&
> > -	error: refs/heads/@: badRefName: invalid refname format
> > +	error: refs/heads/ branch-1: badRefName: invalid refname format
> >  	EOF
> > -	rm $branch_dir_prefix/@ &&
> > +	rm $branch_dir_prefix/'\'' branch-1'\'' &&
> >  	test_cmp expect err &&
> 
> Okay, we now allow `refs/heads/@`, but still don't allow other bad
> formatting like spaces in the refname.
> 

Yes, this is a mistake. Junio have told me in this patch and I have
realized this.

  https://lore.kernel.org/git/xmqqjzei1mtb.fsf@gitster.g/

> Patrick

Thanks,
Jialuo

diff --git a/refs/files-backend.c b/refs/files-backend.c
index 03d2503276..f246c92684 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -3519,10 +3519,10 @@  static int files_fsck_refs_name(struct ref_store *ref_store UNUSED,
 	if (iter->basename[0] != '.' && ends_with(iter->basename, ".lock"))
 		goto cleanup;
 
-	if (check_refname_format(iter->basename, REFNAME_ALLOW_ONELEVEL)) {
+	strbuf_addf(&sb, "%s/%s", refs_check_dir, iter->relative_path);
+	if (check_refname_format(sb.buf, 0)) {
 		struct fsck_ref_report report = { 0 };
 
-		strbuf_addf(&sb, "%s/%s", refs_check_dir, iter->relative_path);
 		report.path = sb.buf;
 		ret = fsck_report_ref(o, &report,
 				      FSCK_MSG_BAD_REF_NAME,
diff --git a/t/t0602-reffiles-fsck.sh b/t/t0602-reffiles-fsck.sh
index 71a4d1a5ae..0aee377439 100755
--- a/t/t0602-reffiles-fsck.sh
+++ b/t/t0602-reffiles-fsck.sh
@@ -25,6 +25,13 @@  test_expect_success 'ref name should be checked' '
 	git tag tag-2 &&
 	git tag multi_hierarchy/tag-2 &&
 
+	cp $branch_dir_prefix/branch-1 $branch_dir_prefix/@ &&
+	git refs verify 2>err &&
+	cat >expect <<-EOF &&
+	EOF
+	test_must_be_empty err &&
+	rm $branch_dir_prefix/@ &&
+
 	cp $branch_dir_prefix/branch-1 $branch_dir_prefix/.branch-1 &&
 	test_must_fail git refs verify 2>err &&
 	cat >expect <<-EOF &&
@@ -33,20 +40,20 @@  test_expect_success 'ref name should be checked' '
 	rm $branch_dir_prefix/.branch-1 &&
 	test_cmp expect err &&
 
-	cp $branch_dir_prefix/branch-1 $branch_dir_prefix/@ &&
+	cp $branch_dir_prefix/branch-1 $branch_dir_prefix/'\'' branch-1'\'' &&
 	test_must_fail git refs verify 2>err &&
 	cat >expect <<-EOF &&
-	error: refs/heads/@: badRefName: invalid refname format
+	error: refs/heads/ branch-1: badRefName: invalid refname format
 	EOF
-	rm $branch_dir_prefix/@ &&
+	rm $branch_dir_prefix/'\'' branch-1'\'' &&
 	test_cmp expect err &&
 
-	cp $tag_dir_prefix/multi_hierarchy/tag-2 $tag_dir_prefix/multi_hierarchy/@ &&
+	cp $tag_dir_prefix/multi_hierarchy/tag-2 $tag_dir_prefix/multi_hierarchy/'\''~tag-2'\'' &&
 	test_must_fail git refs verify 2>err &&
 	cat >expect <<-EOF &&
-	error: refs/tags/multi_hierarchy/@: badRefName: invalid refname format
+	error: refs/tags/multi_hierarchy/~tag-2: badRefName: invalid refname format
 	EOF
-	rm $tag_dir_prefix/multi_hierarchy/@ &&
+	rm $tag_dir_prefix/multi_hierarchy/'\''~tag-2'\'' &&
 	test_cmp expect err &&
 
 	cp $tag_dir_prefix/tag-1 $tag_dir_prefix/tag-1.lock &&
@@ -60,6 +67,15 @@  test_expect_success 'ref name should be checked' '
 	error: refs/tags/.lock: badRefName: invalid refname format
 	EOF
 	rm $tag_dir_prefix/.lock &&
+	test_cmp expect err &&
+
+	mkdir $tag_dir_prefix/'\''~new-feature'\'' &&
+	cp $tag_dir_prefix/tag-1 $tag_dir_prefix/'\''~new-feature'\''/tag-1 &&
+	test_must_fail git refs verify 2>err &&
+	cat >expect <<-EOF &&
+	error: refs/tags/~new-feature/tag-1: badRefName: invalid refname format
+	EOF
+	rm -rf $tag_dir_prefix/'\''~new-feature'\'' &&
 	test_cmp expect err
 '
 
@@ -84,7 +100,7 @@  test_expect_success 'ref name check should be adapted into fsck messages' '
 	rm $branch_dir_prefix/.branch-1 &&
 	test_cmp expect err &&
 
-	cp $branch_dir_prefix/branch-1 $branch_dir_prefix/@ &&
+	cp $branch_dir_prefix/branch-1 $branch_dir_prefix/'\''~branch-1'\'' &&
 	git -c fsck.badRefName=ignore refs verify 2>err &&
 	test_must_be_empty err
 '

[v6,2/9] ref: check the full refname instead of basename

Commit Message

Comments

Patch