[v2,0/4] Documentation: improve linting for manpage existence

Message

Patrick Steinhardt June 6, 2024, 8 a.m. UTC

Hi,

this is the second version of my patch series that improves linting for
the existence and/or absence of manpages.

Changes compared to v1:

  - Converted the extracted script to be compatible with POSIX shells.

  - Fix an empty line being printed when there are no findings.

  - Silence the patched-in Makefile recipe, which otherwise leads to one
    value existing twice in the converted output.

  - Fix `check_missing_docs()` to use `ALL_COMMANDS`, not `BUILT_INS`.

Thanks!

Patrick

Patrick Steinhardt (4):
  Makefile: extract script to lint missing/extraneous manpages
  Documentation/lint-manpages: bubble up errors
  gitlab-ci: add job to run `make check-docs`
  ci/test-documentation: work around SyntaxWarning in Python 3.12

 .gitlab-ci.yml                 |   9 +++
 Documentation/Makefile         |   4 ++
 Documentation/lint-manpages.sh | 108 +++++++++++++++++++++++++++++++++
 Makefile                       |  36 -----------
 ci/test-documentation.sh       |   1 +
 5 files changed, 122 insertions(+), 36 deletions(-)
 create mode 100755 Documentation/lint-manpages.sh

Range-diff against v1:
1:  b06088a2ff ! 1:  489a6eaf2d Makefile: extract script to lint missing/extraneous manpages
    @@ Documentation/Makefile: $(LINT_DOCS_FSCK_MSGIDS): ../fsck.h fsck-msgids.txt
     
      ## Documentation/lint-manpages.sh (new) ##
     @@
    -+#!/usr/bin/env bash
    -+
    -+cd "$(dirname "${BASH_SOURCE[0]}")"/..
    ++#!/bin/sh
     +
     +extract_variable () {
     +	(
    -+		cat Makefile
    ++		cat ../Makefile
     +		cat <<EOF
     +print_variable:
    -+	\$(foreach b,\$($1),echo XXX \$(b:\$X=) YYY;)
    ++	@\$(foreach b,\$($1),echo XXX \$(b:\$X=) YYY;)
     +EOF
     +	) |
    -+	make -f - print_variable 2>/dev/null |
    ++	make -C .. -f - print_variable 2>/dev/null |
     +	sed -n -e 's/.*XXX \(.*\) YYY.*/\1/p'
     +}
     +
     +check_missing_docs () {
    -+	for v in $BUILT_INS
    ++	for v in $ALL_COMMANDS
     +	do
     +		case "$v" in
     +		git-merge-octopus) continue;;
    @@ Documentation/lint-manpages.sh (new)
     +		git-?*--?* ) continue ;;
     +		esac
     +
    -+		if ! test -f "Documentation/$v.txt"
    ++		if ! test -f "$v.txt"
     +		then
     +			echo "no doc: $v"
     +		fi
     +
    -+		if ! sed -e '1,/^### command list/d' -e '/^#/d' command-list.txt |
    ++		if ! sed -e '1,/^### command list/d' -e '/^#/d' ../command-list.txt |
     +			grep -q "^$v[ 	]"
     +		then
     +			case "$v" in
    @@ Documentation/lint-manpages.sh (new)
     +}
     +
     +check_extraneous_docs () {
    -+	local commands="$(printf "%s\n" "$ALL_COMMANDS" "$BUILT_INS" "$EXCLUDED_PROGRAMS")"
    -+
    -+	while read how cmd
    -+	do
    -+		if ! [[ $commands = *"$cmd"* ]]
    -+		then
    -+			echo "removed but $how: $cmd"
    -+		fi
    -+	done < <(
    ++	(
     +		sed -e '1,/^### command list/d' \
     +		    -e '/^#/d' \
     +		    -e '/guide$/d' \
     +		    -e '/interfaces$/d' \
     +		    -e 's/[ 	].*//' \
    -+		    -e 's/^/listed /' command-list.txt
    -+		make -C Documentation print-man1 |
    ++		    -e 's/^/listed /' ../command-list.txt
    ++		make print-man1 |
     +		grep '\.txt$' |
     +		sed -e 's|^|documented |' \
     +		    -e 's/\.txt//'
    ++	) | (
    ++		all_commands="$(printf "%s " "$ALL_COMMANDS" "$BUILT_INS" "$EXCLUDED_PROGRAMS" | tr '\n' ' ')"
    ++
    ++		while read how cmd
    ++		do
    ++			case " $all_commands " in
    ++			*" $cmd "*) ;;
    ++			*)
    ++				echo "removed but $how: $cmd";;
    ++			esac
    ++		done
     +	)
     +}
     +
2:  b39a780d33 ! 2:  9f21c305b9 Documentation/lint-manpages: bubble up errors
    @@ Commit message
     
      ## Documentation/lint-manpages.sh ##
     @@ Documentation/lint-manpages.sh: EOF
    + 	sed -n -e 's/.*XXX \(.*\) YYY.*/\1/p'
      }
      
    - check_missing_docs () {
    -+	local ret=0
    +-check_missing_docs () {
    ++check_missing_docs () (
    ++	ret=0
     +
    - 	for v in $BUILT_INS
    + 	for v in $ALL_COMMANDS
      	do
      		case "$v" in
     @@ Documentation/lint-manpages.sh: check_missing_docs () {
    - 		if ! test -f "Documentation/$v.txt"
    + 		if ! test -f "$v.txt"
      		then
      			echo "no doc: $v"
     +			ret=1
      		fi
      
    - 		if ! sed -e '1,/^### command list/d' -e '/^#/d' command-list.txt |
    + 		if ! sed -e '1,/^### command list/d' -e '/^#/d' ../command-list.txt |
     @@ Documentation/lint-manpages.sh: check_missing_docs () {
      			git)
      				;;
    @@ Documentation/lint-manpages.sh: check_missing_docs () {
      			esac
      		fi
      	done
    +-}
     +
    -+	return $ret
    - }
    ++	exit $ret
    ++)
      
      check_extraneous_docs () {
    - 	local commands="$(printf "%s\n" "$ALL_COMMANDS" "$BUILT_INS" "$EXCLUDED_PROGRAMS")"
    -+	local ret=0
    - 
    - 	while read how cmd
    - 	do
    - 		if ! [[ $commands = *"$cmd"* ]]
    - 		then
    - 			echo "removed but $how: $cmd"
    -+			ret=1
    - 		fi
    - 	done < <(
    - 		sed -e '1,/^### command list/d' \
    + 	(
     @@ Documentation/lint-manpages.sh: check_extraneous_docs () {
    - 		sed -e 's|^|documented |' \
      		    -e 's/\.txt//'
    - 	)
    + 	) | (
    + 		all_commands="$(printf "%s " "$ALL_COMMANDS" "$BUILT_INS" "$EXCLUDED_PROGRAMS" | tr '\n' ' ')"
    ++		ret=0
    + 
    + 		while read how cmd
    + 		do
    + 			case " $all_commands " in
    + 			*" $cmd "*) ;;
    + 			*)
    +-				echo "removed but $how: $cmd";;
    ++				echo "removed but $how: $cmd"
    ++				ret=1;;
    + 			esac
    + 		done
     +
    -+	return $ret
    ++		exit $ret
    + 	)
      }
      
    - BUILT_INS="$(extract_variable BUILT_INS)"
    +@@ Documentation/lint-manpages.sh: BUILT_INS="$(extract_variable BUILT_INS)"
      ALL_COMMANDS="$(extract_variable ALL_COMMANDS)"
      EXCLUDED_PROGRAMS="$(extract_variable EXCLUDED_PROGRAMS)"
      
    @@ Documentation/lint-manpages.sh: check_extraneous_docs () {
     +)
     +ret=$?
     +
    -+echo "$findings" | sort
    ++printf "%s" "$findings" | sort
     +
     +exit $ret
3:  a44d57a732 = 3:  d6d3628797 gitlab-ci: add job to run `make check-docs`
4:  c758b45282 = 4:  c13fed9ebf ci/test-documentation: work around SyntaxWarning in Python 3.12

Comments

James Liu June 7, 2024, 12:59 a.m. UTC | #1

On Thu Jun 6, 2024 at 6:00 PM AEST, Patrick Steinhardt wrote:
> Hi,
>
> this is the second version of my patch series that improves linting for
> the existence and/or absence of manpages.
>
> Changes compared to v1:
>
>   - Converted the extracted script to be compatible with POSIX shells.
>
>   - Fix an empty line being printed when there are no findings.
>
>   - Silence the patched-in Makefile recipe, which otherwise leads to one
>     value existing twice in the converted output.
>
>   - Fix `check_missing_docs()` to use `ALL_COMMANDS`, not `BUILT_INS`.
>
> Thanks!
>
> Patrick

Thanks Patrick, this looks good to me! Learnt a few shell tricks too.

Cheers,
James