diff mbox series

[4/4] lib-submodule-update: pass OVERWRITING_FAIL

Message ID 22eacd20a8213c3281974a6c103fd2d00c95efaa.1588162842.git.liu.denton@gmail.com (mailing list archive)
State New, archived
Headers show
Series t: replace incorrect test_must_fail usage (part 5) | expand

Commit Message

Denton Liu April 29, 2020, 12:22 p.m. UTC
We are using `test_must_fail $command`. However, $command is not
necessarily a git command; it could be a test helper function.

In an effort to stop using test_must_fail with non-git commands, instead
of invoking `test_must_fail $command`, run
`OVERWRITING_FAIL=test_must_fail $command` instead. Increase the
granularity of the test helper functions by specifically choosing the
individual git invocation which is designed to fail.

While we're at it, some helper functions have git commands piping into
another git command. Break these pipes up into two separate invocations
with a file buffer so that the return code of the first command is not
lost.

This patch can be better viewed with `--ignore-all-space`.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
---
 t/lib-submodule-update.sh   | 25 +++++++++++++++++--------
 t/t3426-rebase-submodule.sh |  4 ++--
 t/t3513-revert-submodule.sh | 20 ++++++++++++--------
 t/t3906-stash-submodule.sh  | 15 +++++++++------
 t/t4137-apply-submodule.sh  |  6 ++++--
 t/t4255-am-submodule.sh     |  6 ++++--
 t/t5572-pull-submodule.sh   |  8 ++++----
 t/t6041-bisect-submodule.sh | 33 ++++++++++++++++++---------------
 8 files changed, 70 insertions(+), 47 deletions(-)

Comments

Junio C Hamano April 29, 2020, 7:24 p.m. UTC | #1
Denton Liu <liu.denton@gmail.com> writes:

> We are using `test_must_fail $command`. However, $command is not
> necessarily a git command; it could be a test helper function.
>
> In an effort to stop using test_must_fail with non-git commands, instead
> of invoking `test_must_fail $command`, run
> `OVERWRITING_FAIL=test_must_fail $command` instead. Increase the
> granularity of the test helper functions by specifically choosing the
> individual git invocation which is designed to fail.

Sorry, but I do not know why this is a good idea.

>  test_submodule_switch_common () {
> +	OVERWRITING_FAIL=
>  	command="$1"
>  	######################### Appearing submodule #########################
>  	# Switching to a commit letting a submodule appear creates empty dir ...
> @@ -443,7 +446,7 @@ test_submodule_switch_common () {
>  		(
>  			cd submodule_update &&
>  			git branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&
> -			test_must_fail $command replace_sub1_with_directory &&
> +			OVERWRITING_FAIL=test_must_fail $command replace_sub1_with_directory &&

Here, $command may or may not be a git command and more importantly,
it could be a shell function, right?  Then we need to take it into
account that 

	VAR=VAL shell_function args...

will not work, no?

Some shells do not make this a single-shot environment variable
assignment that will not persist once the single function invocation
returns.
Denton Liu April 30, 2020, 1:10 a.m. UTC | #2
Hi Junio,

On Wed, Apr 29, 2020 at 12:24:48PM -0700, Junio C Hamano wrote:
> Denton Liu <liu.denton@gmail.com> writes:
> 
> > We are using `test_must_fail $command`. However, $command is not
> > necessarily a git command; it could be a test helper function.
> >
> > In an effort to stop using test_must_fail with non-git commands, instead
> > of invoking `test_must_fail $command`, run
> > `OVERWRITING_FAIL=test_must_fail $command` instead. Increase the
> > granularity of the test helper functions by specifically choosing the
> > individual git invocation which is designed to fail.
> 
> Sorry, but I do not know why this is a good idea.

This is useful because currently, when we run a test helper function, we
just mark the whole thing as `test_must_fail`. However, it's possible
that the helper function might fail earlier or later than expected due
to some side effect. If this happens, then the test case will still
be reported as passing but I think that it should be marked as failing
instead since it didn't actually display the desired behaviour.

> >  test_submodule_switch_common () {
> > +	OVERWRITING_FAIL=
> >  	command="$1"
> >  	######################### Appearing submodule #########################
> >  	# Switching to a commit letting a submodule appear creates empty dir ...
> > @@ -443,7 +446,7 @@ test_submodule_switch_common () {
> >  		(
> >  			cd submodule_update &&
> >  			git branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&
> > -			test_must_fail $command replace_sub1_with_directory &&
> > +			OVERWRITING_FAIL=test_must_fail $command replace_sub1_with_directory &&
> 
> Here, $command may or may not be a git command and more importantly,
> it could be a shell function, right?  Then we need to take it into
> account that 
> 
> 	VAR=VAL shell_function args...
> 
> will not work, no?
> 
> Some shells do not make this a single-shot environment variable
> assignment that will not persist once the single function invocation
> returns.

I looked through POSIX specification and it says under 2.9.1 Simple
Comamnds,

	If no command name results, variable assignments shall affect
	the current execution environment. Otherwise, the variable
	assignments shall be exported for the execution environment of
	the command and shall not affect the current execution
	environment (except for special built-ins).

which makes me suspect that these shells are not POSIX-compliant. What
are some examples of shells that behave this way?

That being said, I know that we live in the real world and POSIX
standards don't justify breaking test cases. I'll reroll this to add an
`OVERWRITING_FAIL=` after the command to ensure that the variable is
cleared.

Thanks,

Denton
Junio C Hamano April 30, 2020, 3:41 a.m. UTC | #3
Denton Liu <liu.denton@gmail.com> writes:

>> Here, $command may or may not be a git command and more importantly,
>> it could be a shell function, right?  Then we need to take it into
>> account that 
>> 
>> 	VAR=VAL shell_function args...
>> 
>> will not work, no?
>> 
>> Some shells do not make this a single-shot environment variable
>> assignment that will not persist once the single function invocation
>> returns.
>
> ...
> which makes me suspect that these shells are not POSIX-compliant. What
> are some examples of shells that behave this way?

I think the most relevant thread is the one that contains this
message:

  https://public-inbox.org/git/7vljfzz0yd.fsf@alter.siamese.dyndns.org/

FWIW, shells that do not retain the assignment after a function
returns are not POSIX compliant.
Denton Liu April 30, 2020, 9:22 a.m. UTC | #4
On Wed, Apr 29, 2020 at 08:41:23PM -0700, Junio C Hamano wrote:
> Denton Liu <liu.denton@gmail.com> writes:
> 
> >> Here, $command may or may not be a git command and more importantly,
> >> it could be a shell function, right?  Then we need to take it into
> >> account that 
> >> 
> >> 	VAR=VAL shell_function args...
> >> 
> >> will not work, no?
> >> 
> >> Some shells do not make this a single-shot environment variable
> >> assignment that will not persist once the single function invocation
> >> returns.
> >
> > ...
> > which makes me suspect that these shells are not POSIX-compliant. What
> > are some examples of shells that behave this way?
> 
> I think the most relevant thread is the one that contains this
> message:
> 
>   https://public-inbox.org/git/7vljfzz0yd.fsf@alter.siamese.dyndns.org/
> 
> FWIW, shells that do not retain the assignment after a function
> returns are not POSIX compliant.

Hmm, interesting. Running an experiment:

	$ cat test.sh
	f () {
		echo $var
	}

	var=test f
	echo $var
	$ bash test.sh
	test

	$ bash --posix test.sh
	test
	test
	$ dash test.sh
	test


So it seems like there's a bug in dash. Guess it's time to file a bug
report...
diff mbox series

Patch

diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
index cd80d77707..c8d1b6f528 100644
--- a/t/lib-submodule-update.sh
+++ b/t/lib-submodule-update.sh
@@ -304,12 +304,15 @@  test_submodule_content () {
 # a removed submodule.
 #
 # Removing a submodule containing a .git directory must fail even when forced
-# to protect the history!
+# to protect the history! If we are testing this case,
+# OVERWRITING_FAIL=test_must_fail, otherwise OVERWRITING_FAIL will be the empty
+# string.
 #
 
 # Internal function; use test_submodule_switch_func(), test_submodule_switch_func(),
 # or test_submodule_forced_switch() instead.
 test_submodule_switch_common () {
+	OVERWRITING_FAIL=
 	command="$1"
 	######################### Appearing submodule #########################
 	# Switching to a commit letting a submodule appear creates empty dir ...
@@ -443,7 +446,7 @@  test_submodule_switch_common () {
 		(
 			cd submodule_update &&
 			git branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&
-			test_must_fail $command replace_sub1_with_directory &&
+			OVERWRITING_FAIL=test_must_fail $command replace_sub1_with_directory &&
 			test_superproject_content origin/add_sub1 &&
 			test_submodule_content sub1 origin/add_sub1
 		)
@@ -456,7 +459,7 @@  test_submodule_switch_common () {
 			cd submodule_update &&
 			git branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&
 			replace_gitfile_with_git_dir sub1 &&
-			test_must_fail $command replace_sub1_with_directory &&
+			OVERWRITING_FAIL=test_must_fail $command replace_sub1_with_directory &&
 			test_superproject_content origin/add_sub1 &&
 			test_git_directory_is_unchanged sub1 &&
 			test_submodule_content sub1 origin/add_sub1
@@ -470,7 +473,7 @@  test_submodule_switch_common () {
 		(
 			cd submodule_update &&
 			git branch -t replace_sub1_with_file origin/replace_sub1_with_file &&
-			test_must_fail $command replace_sub1_with_file &&
+			OVERWRITING_FAIL=test_must_fail $command replace_sub1_with_file &&
 			test_superproject_content origin/add_sub1 &&
 			test_submodule_content sub1 origin/add_sub1
 		)
@@ -484,7 +487,7 @@  test_submodule_switch_common () {
 			cd submodule_update &&
 			git branch -t replace_sub1_with_file origin/replace_sub1_with_file &&
 			replace_gitfile_with_git_dir sub1 &&
-			test_must_fail $command replace_sub1_with_file &&
+			OVERWRITING_FAIL=test_must_fail $command replace_sub1_with_file &&
 			test_superproject_content origin/add_sub1 &&
 			test_git_directory_is_unchanged sub1 &&
 			test_submodule_content sub1 origin/add_sub1
@@ -559,6 +562,11 @@  test_submodule_switch_common () {
 # conditions, set the appropriate KNOWN_FAILURE_* variable used in the tests
 # below to 1.
 #
+# Removing a submodule containing a .git directory must fail even when forced
+# to protect the history! If we are testing this case,
+# OVERWRITING_FAIL=test_must_fail, otherwise OVERWRITING_FAIL will be the empty
+# string.
+#
 # Use as follows:
 #
 # my_func () {
@@ -568,6 +576,7 @@  test_submodule_switch_common () {
 # }
 # test_submodule_switch_func "my_func"
 test_submodule_switch_func () {
+	OVERWRITING_FAIL=
 	command="$1"
 	test_submodule_switch_common "$command"
 
@@ -580,7 +589,7 @@  test_submodule_switch_func () {
 			cd submodule_update &&
 			git branch -t add_sub1 origin/add_sub1 &&
 			>sub1 &&
-			test_must_fail $command add_sub1 &&
+			OVERWRITING_FAIL=test_must_fail $command add_sub1 &&
 			test_superproject_content origin/no_submodule &&
 			test_must_be_empty sub1
 		)
@@ -588,7 +597,7 @@  test_submodule_switch_func () {
 }
 
 test_submodule_switch () {
-	test_submodule_switch_func "git $1"
+	test_submodule_switch_func "eval \$OVERWRITING_FAIL git $1"
 }
 
 # Same as test_submodule_switch(), except that throwing away local changes in
@@ -596,7 +605,7 @@  test_submodule_switch () {
 test_submodule_forced_switch () {
 	command="$1"
 	KNOWN_FAILURE_FORCED_SWITCH_TESTS=1
-	test_submodule_switch_common "git $command"
+	test_submodule_switch_common "eval \$OVERWRITING_FAIL git $command"
 
 	# When forced, a file in the superproject does not prevent creating a
 	# submodule of the same name.
diff --git a/t/t3426-rebase-submodule.sh b/t/t3426-rebase-submodule.sh
index 788605ccc0..c6a7f584ed 100755
--- a/t/t3426-rebase-submodule.sh
+++ b/t/t3426-rebase-submodule.sh
@@ -17,7 +17,7 @@  git_rebase () {
 	git status -su >actual &&
 	ls -1pR * >>actual &&
 	test_cmp expect actual &&
-	git rebase "$1"
+	$OVERWRITING_FAIL git rebase "$1"
 }
 
 test_submodule_switch_func "git_rebase"
@@ -35,7 +35,7 @@  git_rebase_interactive () {
 	test_cmp expect actual &&
 	set_fake_editor &&
 	echo "fake-editor.sh" >.git/info/exclude &&
-	git rebase -i "$1"
+	$OVERWRITING_FAIL git rebase -i "$1"
 }
 
 test_submodule_switch_func "git_rebase_interactive"
diff --git a/t/t3513-revert-submodule.sh b/t/t3513-revert-submodule.sh
index 95a7f64471..6c899db7e1 100755
--- a/t/t3513-revert-submodule.sh
+++ b/t/t3513-revert-submodule.sh
@@ -15,14 +15,18 @@  git_revert () {
 	git status -su >expect &&
 	ls -1pR * >>expect &&
 	tar cf "$TRASH_DIRECTORY/tmp.tar" * &&
-	git checkout "$1" &&
-	git revert HEAD &&
-	rm -rf * &&
-	tar xf "$TRASH_DIRECTORY/tmp.tar" &&
-	git status -su >actual &&
-	ls -1pR * >>actual &&
-	test_cmp expect actual &&
-	git revert HEAD
+	$OVERWRITING_FAIL git checkout "$1" &&
+	if test -z "$OVERWRITING_FAIL"
+	then
+		git checkout "$1" &&
+		git revert HEAD &&
+		rm -rf * &&
+		tar xf "$TRASH_DIRECTORY/tmp.tar" &&
+		git status -su >actual &&
+		ls -1pR * >>actual &&
+		test_cmp expect actual &&
+		git revert HEAD
+	fi
 }
 
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
diff --git a/t/t3906-stash-submodule.sh b/t/t3906-stash-submodule.sh
index 6a7e801ca0..860940072d 100755
--- a/t/t3906-stash-submodule.sh
+++ b/t/t3906-stash-submodule.sh
@@ -8,12 +8,15 @@  test_description='stash can handle submodules'
 git_stash () {
 	git status -su >expect &&
 	ls -1pR * >>expect &&
-	git read-tree -u -m "$1" &&
-	git stash &&
-	git status -su >actual &&
-	ls -1pR * >>actual &&
-	test_cmp expect actual &&
-	git stash apply
+	$OVERWRITING_FAIL git read-tree -u -m "$1" &&
+	if test -z "$OVERWRITING_FAIL"
+	then
+		git stash &&
+		git status -su >actual &&
+		ls -1pR * >>actual &&
+		test_cmp expect actual &&
+		git stash apply
+	fi
 }
 
 KNOWN_FAILURE_STASH_DOES_IGNORE_SUBMODULE_CHANGES=1
diff --git a/t/t4137-apply-submodule.sh b/t/t4137-apply-submodule.sh
index b645e303a0..dd08d9e1a4 100755
--- a/t/t4137-apply-submodule.sh
+++ b/t/t4137-apply-submodule.sh
@@ -6,13 +6,15 @@  test_description='git apply handling submodules'
 . "$TEST_DIRECTORY"/lib-submodule-update.sh
 
 apply_index () {
-	git diff --ignore-submodules=dirty "..$1" | git apply --index -
+	git diff --ignore-submodules=dirty "..$1" >diff &&
+	$OVERWRITING_FAIL git apply --index - <diff
 }
 
 test_submodule_switch_func "apply_index"
 
 apply_3way () {
-	git diff --ignore-submodules=dirty "..$1" | git apply --3way -
+	git diff --ignore-submodules=dirty "..$1" >diff &&
+	$OVERWRITING_FAIL git apply --3way - <diff
 }
 
 test_submodule_switch_func "apply_3way"
diff --git a/t/t4255-am-submodule.sh b/t/t4255-am-submodule.sh
index 1b179d5f45..b0fe78a956 100755
--- a/t/t4255-am-submodule.sh
+++ b/t/t4255-am-submodule.sh
@@ -6,13 +6,15 @@  test_description='git am handling submodules'
 . "$TEST_DIRECTORY"/lib-submodule-update.sh
 
 am () {
-	git format-patch --stdout --ignore-submodules=dirty "..$1" | git am -
+	git format-patch --stdout --ignore-submodules=dirty "..$1" >patch &&
+	$OVERWRITING_FAIL git am - <patch
 }
 
 test_submodule_switch_func "am"
 
 am_3way () {
-	git format-patch --stdout --ignore-submodules=dirty "..$1" | git am --3way -
+	git format-patch --stdout --ignore-submodules=dirty "..$1" >patch &&
+	$OVERWRITING_FAIL git am --3way - <patch
 }
 
 KNOWN_FAILURE_NOFF_MERGE_ATTEMPTS_TO_MERGE_REMOVED_SUBMODULE_FILES=1
diff --git a/t/t5572-pull-submodule.sh b/t/t5572-pull-submodule.sh
index f911bf631e..4dd9913b6b 100755
--- a/t/t5572-pull-submodule.sh
+++ b/t/t5572-pull-submodule.sh
@@ -13,7 +13,7 @@  reset_branch_to_HEAD () {
 
 git_pull () {
 	reset_branch_to_HEAD "$1" &&
-	git pull
+	$OVERWRITING_FAIL git pull
 }
 
 # pulls without conflicts
@@ -21,21 +21,21 @@  test_submodule_switch_func "git_pull"
 
 git_pull_ff () {
 	reset_branch_to_HEAD "$1" &&
-	git pull --ff
+	$OVERWRITING_FAIL git pull --ff
 }
 
 test_submodule_switch_func "git_pull_ff"
 
 git_pull_ff_only () {
 	reset_branch_to_HEAD "$1" &&
-	git pull --ff-only
+	$OVERWRITING_FAIL git pull --ff-only
 }
 
 test_submodule_switch_func "git_pull_ff_only"
 
 git_pull_noff () {
 	reset_branch_to_HEAD "$1" &&
-	git pull --no-ff
+	$OVERWRITING_FAIL git pull --no-ff
 }
 
 KNOWN_FAILURE_NOFF_MERGE_DOESNT_CREATE_EMPTY_SUBMODULE_DIR=1
diff --git a/t/t6041-bisect-submodule.sh b/t/t6041-bisect-submodule.sh
index 0e0cdf638d..714d393899 100755
--- a/t/t6041-bisect-submodule.sh
+++ b/t/t6041-bisect-submodule.sh
@@ -10,21 +10,24 @@  git_bisect () {
 	ls -1pR * >>expect &&
 	tar cf "$TRASH_DIRECTORY/tmp.tar" * &&
 	GOOD=$(git rev-parse --verify HEAD) &&
-	git checkout "$1" &&
-	echo "foo" >bar &&
-	git add bar &&
-	git commit -m "bisect bad" &&
-	BAD=$(git rev-parse --verify HEAD) &&
-	git reset --hard HEAD^^ &&
-	git submodule update &&
-	git bisect start &&
-	git bisect good $GOOD &&
-	rm -rf * &&
-	tar xf "$TRASH_DIRECTORY/tmp.tar" &&
-	git status -su >actual &&
-	ls -1pR * >>actual &&
-	test_cmp expect actual &&
-	git bisect bad $BAD
+	$OVERWRITING_FAIL git checkout "$1" &&
+	if test -z "$OVERWRITING_FAIL"
+	then
+		echo "foo" >bar &&
+		git add bar &&
+		git commit -m "bisect bad" &&
+		BAD=$(git rev-parse --verify HEAD) &&
+		git reset --hard HEAD^^ &&
+		git submodule update &&
+		git bisect start &&
+		git bisect good $GOOD &&
+		rm -rf * &&
+		tar xf "$TRASH_DIRECTORY/tmp.tar" &&
+		git status -su >actual &&
+		ls -1pR * >>actual &&
+		test_cmp expect actual &&
+		git bisect bad $BAD
+	fi
 }
 
 test_submodule_switch_func "git_bisect"