| Message ID | 20221102224843.2104-2-philipoakley@iee.email (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | Provide example docbundles | expand |
On Wed, Nov 02 2022, Philip Oakley wrote: > [...] > diff --git a/Documentation/technical/docbundle.txt b/Documentation/technical/docbundle.txt > new file mode 100644 > index 0000000000..8c17a7847f > --- /dev/null > +++ b/Documentation/technical/docbundle.txt > @@ -0,0 +1,97 @@ > +docbundle Design Document > +========================= > + > +The Git man pages contain many example setups to demonstrate aspects > +of the related Git commands, such as history simplification in the `log` > +and `show commands. However, while illustrative, these are not > +accessible, as example repositories, to regular, potentially confused, > +users who may need to understand the set up of the examples. > + > +The docbundle concept is that for each of these examples there is a > +matching `bundle` file, provided with the Documentation, which can be > +simply cloned to provide an example repository containing the example. > + > +There are also many example repositories in the Git `/t` test system > +that could be exposed to users which already demonstrate some important > +testable aspect of Git within a carefully constructed repository. > + > + > +This proposal is to provide DocBundles to users via the Git project. > + > + > +Firstly, it should be reasonably self-evident that the man page examples > +should, in themselves, be tested for accuracy, especially if provided > +to users. > + > +Secondly, the existing test system is the proper place to setup and > +test any such docbundle repositories. > + > +Third, that the bundle is an obvious pre-existing mechanism for > +transporting repositories (see also the Bundle-URI proposals). > + > +Fourthly, the preparation and distribution of the docbundle should > +require minimal maintainer effort, preferably being integrated into > +the existing automation for Documentation preparation and distribution. > + > +Fifthly, the docbudles themselves should not to be part of the git.git > +repo. They are prepared via the test suite and should be stored (locally) > +in a designated Git_DocBundle_Dir, and be .gitignored. The DocBundles > +are transferred to the maintainer's Doc repo via the automation. > + > +Sixthly, the docbundles to be named based on their `tnnnn` test number > +and a (short) descriptive name. > + > +Seven, (dev) start with an existing test, rather than add a test (most > +doc/man examples aren't tested!). Add tests for missing examples later. > + > +Eight, (option) add a `git docbundle` command to assist in locating and > +listing the docbundle examples. It also provides place to explain their > +purpose as "Tutorial Examples" (Note, many existing examples are to > +illustrate complex/difficult scenarios where the wording was insufficient; > +the docbudles are minimal examples). > + > + > +Status > +====== > + > +This is an RFC/PoC (Request for Comment / Proof of Concept). > + > +The attached commits establish: > + > +Set out this design in the Documentation/technical directory. > + > +A default location for the docbundles and an env variable to allow > +flexibility by extending test-lib. > + > +The docbundles are ignored, as is the default directory. > + > +Two docbundles are generated from a 'random' test that I happened to > +have open in my editor (t6102 rev-list-simplify). > + > +The docbundle creation is protected by `test_expect_success` wrappers. > + > +ToDo > +==== > + > +DocBundles - Is this the best name, or just a convenient way of > +indicating the delivery mechanism? > + > +Add Prerequisite to control if docbundles are generated (don't waste > +maintainers PC time when not needed). > + > +CI integration (i.e. whether to test the bundling aspect - should be > +reliable). > + > +How to update the man pages, perhaps by inclusion of the test comment > +(exported?), and distinguish between man page descriptions (being > +packaged by the test suite) and tests which offer users insight into > +Git operations, usually with 'interesting' commit-graph set-ups. > + > +Claim t999x as the man doc bundle test sequence ('999' is the UK > +emergency ("Help!") number; 911, 101 and 102 are taken test areas. > + > +Other stuff implicit in the eight point list I've skimmed this, and provided some rough comments inline. I think the core idea sounds nice, if I'm getting it right. Let me see if I can rephrase it, and you can point out where I'm wrong/didn't get it: We have stuff in our docs like e.g. this, at the top of git-rebase(1); Assume the following history exists and the current branch is "topic": A---B---C topic / D---E---F---G master It would be nice if for that and other things we could add a blurb like: To play around with this history, do: git clone git-doc://git-rebase/description-1.bndl && cd description-1 Or whatever, this POC doesn't fleshen that out, but the idea is that through <some mechanism> we would bridge the gap such that the user could be looking at and playing with the history we're describing in our docs. The above assumes we'd ship those with installed git, but these could also be hosted online, e.g.: git clone https://git-scm.com/test-bundles/git-rebase/description-1.bdl So as an end-goal you could both run these locally, and we could imagine that e.g. git-scm.com could eventually have UX similar to Redis's website: https://redis.io/commands/hgetall/ I.e. where it drops you into a shell to play with the command. In reply to 3/5 I said I had some "local patches", but I forgot the relevant part was on master already (the whole subsequent leak analysis is local): 366bd129dc3 (test-lib: add a SANITIZE=leak logging mode, 2022-07-28) So I think you're making the whole "have the test suite make these" part of this too complex, i.e. with a minor variation of that you can do this in test-lib.sh: TEST_RESULTS_BDL_DIR="$TEST_RESULTS_DIR/$TEST_NAME.test-bdl" Then have a helper like: do_my_test_bundle_stuff () { local name="$1" && shift && git bundle create "$TEST_RESULTS_BDL_DIR/$name.bdl" "$@" } Then all you need to make these is: make test GIT_TEST_OPTS=--debug And then: ls -l t/test-results/*.bdl/* Will give you all the bundles you want. We're then missing the "connect it with the docs" part, but that part would be e.g.: - Have a validation target for it that depends on such a "make test" run - Sanity check that we did create the ones the docs expect Anyway. I like the end-goals, but I can't help think that bundles are the wrong direction to go in, as opposed to: 1. We could have "make install" ship test-lib{,-functions}.sh along with git itself 2. Have e.g. t/doc-setup/*.sh with the snippets to set up these test demos, which we'd also ship. And you'd then do e.g.: git test-case rebase/description-1 Which would just be a thin wrapper around (pseudocode): dir=$(git test--helper get-tempdir) && ( cd "$dir" && # And whatever else we need to set up to say "use this local installed git" GIT_TEST_INSTALLED=[...] && . "$(git --test-dir)"/test-lib.sh . "$(git --test-dir)"/doc-setup/rebase-description-1.sh" # This would run test_commit, and any other arbitrary setup code setup_rebase_description_1() ) && echo "Go play with '$dir', and have fun" The big advantage of that is: 1. It ships with git, so it works offline, and we won't get into the inevitable confusion of the user reading v2.40 docs locally, but the online url tracking a v2.50 example or whatever. 2. We can represent whatever arbitrary repo state, e.g. drop the user into a merge conflict that's already happening. 3. We can also represent "repo-extra" state, e.g. imagine the "git-gc/git-repack" docs discussing loose objects, and wanting to get you into some arbitrary state with regards to your loose objects etc.
Hi Ævar, Thanks for the review, On 03/11/2022 09:19, Ævar Arnfjörð Bjarmason wrote: > On Wed, Nov 02 2022, Philip Oakley wrote: > >> [...] >> diff --git a/Documentation/technical/docbundle.txt b/Documentation/technical/docbundle.txt >> new file mode 100644 >> index 0000000000..8c17a7847f >> --- /dev/null >> +++ b/Documentation/technical/docbundle.txt >> @@ -0,0 +1,97 @@ >> +docbundle Design Document >> +========================= >> + >> +The Git man pages contain many example setups to demonstrate aspects >> +of the related Git commands, such as history simplification in the `log` >> +and `show commands. However, while illustrative, these are not >> +accessible, as example repositories, to regular, potentially confused, >> +users who may need to understand the set up of the examples. >> + >> +The docbundle concept is that for each of these examples there is a >> +matching `bundle` file, provided with the Documentation, which can be >> +simply cloned to provide an example repository containing the example. >> + >> +There are also many example repositories in the Git `/t` test system >> +that could be exposed to users which already demonstrate some important >> +testable aspect of Git within a carefully constructed repository. >> + >> + >> +This proposal is to provide DocBundles to users via the Git project. >> + >> + >> +Firstly, it should be reasonably self-evident that the man page examples >> +should, in themselves, be tested for accuracy, especially if provided >> +to users. >> + >> +Secondly, the existing test system is the proper place to setup and >> +test any such docbundle repositories. >> + >> +Third, that the bundle is an obvious pre-existing mechanism for >> +transporting repositories (see also the Bundle-URI proposals). >> + >> +Fourthly, the preparation and distribution of the docbundle should >> +require minimal maintainer effort, preferably being integrated into >> +the existing automation for Documentation preparation and distribution. >> + >> +Fifthly, the docbudles themselves should not to be part of the git.git >> +repo. They are prepared via the test suite and should be stored (locally) >> +in a designated Git_DocBundle_Dir, and be .gitignored. The DocBundles >> +are transferred to the maintainer's Doc repo via the automation. >> + >> +Sixthly, the docbundles to be named based on their `tnnnn` test number >> +and a (short) descriptive name. >> + >> +Seven, (dev) start with an existing test, rather than add a test (most >> +doc/man examples aren't tested!). Add tests for missing examples later. >> + >> +Eight, (option) add a `git docbundle` command to assist in locating and >> +listing the docbundle examples. It also provides place to explain their >> +purpose as "Tutorial Examples" (Note, many existing examples are to >> +illustrate complex/difficult scenarios where the wording was insufficient; >> +the docbudles are minimal examples). >> + >> + >> +Status >> +====== >> + >> +This is an RFC/PoC (Request for Comment / Proof of Concept). >> + >> +The attached commits establish: >> + >> +Set out this design in the Documentation/technical directory. >> + >> +A default location for the docbundles and an env variable to allow >> +flexibility by extending test-lib. >> + >> +The docbundles are ignored, as is the default directory. >> + >> +Two docbundles are generated from a 'random' test that I happened to >> +have open in my editor (t6102 rev-list-simplify). >> + >> +The docbundle creation is protected by `test_expect_success` wrappers. >> + >> +ToDo >> +==== >> + >> +DocBundles - Is this the best name, or just a convenient way of >> +indicating the delivery mechanism? >> + >> +Add Prerequisite to control if docbundles are generated (don't waste >> +maintainers PC time when not needed). >> + >> +CI integration (i.e. whether to test the bundling aspect - should be >> +reliable). >> + >> +How to update the man pages, perhaps by inclusion of the test comment >> +(exported?), and distinguish between man page descriptions (being >> +packaged by the test suite) and tests which offer users insight into >> +Git operations, usually with 'interesting' commit-graph set-ups. >> + >> +Claim t999x as the man doc bundle test sequence ('999' is the UK >> +emergency ("Help!") number; 911, 101 and 102 are taken test areas. >> + >> +Other stuff implicit in the eight point list > I've skimmed this, and provided some rough comments inline. > > I think the core idea sounds nice, if I'm getting it right. Let me see > if I can rephrase it, and you can point out where I'm wrong/didn't get > it: > > We have stuff in our docs like e.g. this, at the top of git-rebase(1); > > Assume the following history exists and the current branch is "topic": > > A---B---C topic > / > D---E---F---G master > > It would be nice if for that and other things we could add a blurb like: > > To play around with this history, do: > > git clone git-doc://git-rebase/description-1.bndl && cd description-1 That is generically correct. The motivating example was History Simplification (see 5/5), where the example needed to cover a range of setups which then confused new users in a catch 22 manner (link in the 'PoC patch). I had been aware of the issue previously, when looking at the support I'd seen in some other tools which have example setups. > > Or whatever, this POC doesn't fleshen that out, but the idea is that > through <some mechanism> we would bridge the gap such that the user > could be looking at and playing with the history we're describing in our > docs. In terms of fleshing the PoC out, I didn't want to try stepping on too many toes too soon, rather wanted to pick out a few pinch points with suggestions. > > The above assumes we'd ship those with installed git, but these could > also be hosted online, e.g.: > > git clone https://git-scm.com/test-bundles/git-rebase/description-1.bdl > > So as an end-goal you could both run these locally, and we could imagine > that e.g. git-scm.com I was assuming that the bundles would be static documentation artefacts, available locally and potentially on-line. They could even be hosted on the forges.. > could eventually have UX similar to Redis's > website: https://redis.io/commands/hgetall/ > > I.e. where it drops you into a shell to play with the command. I wasn't expecting to have a virtualised on-line git experience from this, but never say never ;-) > In reply to 3/5 I said I had some "local patches", but I forgot the > relevant part was on master already (the whole subsequent leak analysis > is local): 366bd129dc3 (test-lib: add a SANITIZE=leak logging mode, > 2022-07-28) > > So I think you're making the whole "have the test suite make these" part > of this too complex, My main point was that the examples need to be covered by the test suite, and that bundles were a simple way of capturing the tested setup. The GIT_BUNDLE_DIR was simply a way of collecting them in one place for ease of distribution with the documentation (as you mentioned that's a gap in the automation step) > i.e. with a minor variation of that you can do this > in test-lib.sh: > > TEST_RESULTS_BDL_DIR="$TEST_RESULTS_DIR/$TEST_NAME.test-bdl" > > Then have a helper like: > > do_my_test_bundle_stuff () { > local name="$1" && > shift && > git bundle create "$TEST_RESULTS_BDL_DIR/$name.bdl" "$@" > } That would be a simplification > > Then all you need to make these is: > > make test GIT_TEST_OPTS=--debug I didn't quite get the link in that step between the `--debug` and the bndl prep. > > And then: > > ls -l t/test-results/*.bdl/* > > Will give you all the bundles you want. > > We're then missing the "connect it with the docs" part, but that part > would be e.g.: > > - Have a validation target for it that depends on such a "make test" run > - Sanity check that we did create the ones the docs expect > > Anyway. > > I like the end-goals, but I can't help think that bundles are the wrong > direction to go in, as opposed to: > > 1. We could have "make install" ship test-lib{,-functions}.sh along > with git itself > 2. Have e.g. t/doc-setup/*.sh with the snippets to set up these test > demos, which we'd also ship. I'm not into the idea of shipping a load of extra `test` infrastructure with Git itself. We don't ship any in the Git for Windows install. I'd viewed the bundles as an adjunct to the also independent manuals and documentation. > > And you'd then do e.g.: > > git test-case rebase/description-1 > > Which would just be a thin wrapper around (pseudocode): > > dir=$(git test--helper get-tempdir) && > ( > cd "$dir" && > # And whatever else we need to set up to say "use this local installed git" > GIT_TEST_INSTALLED=[...] && > . "$(git --test-dir)"/test-lib.sh > . "$(git --test-dir)"/doc-setup/rebase-description-1.sh" > > # This would run test_commit, and any other arbitrary setup code > setup_rebase_description_1() > ) && > echo "Go play with '$dir', and have fun" > > The big advantage of that is: > > 1. It ships with git, so it works offline, and we won't get into the > inevitable confusion of the user reading v2.40 docs locally, but the > online url tracking a v2.50 example or whatever. Isn't this always a problem (lack of synchronisation between multiple sources). If the user has local documentation, it should have matching bundles locally, No? > 2. We can represent whatever arbitrary repo state, e.g. drop the user > into a merge conflict that's already happening. Isn't the manual's example already meant to describe that? And the user then (hopefully) follows along in their throw-away copy of the repo bundle. > 3. We can also represent "repo-extra" state, e.g. imagine the > "git-gc/git-repack" docs discussing loose objects, and wanting to > get you into some arbitrary state with regards to your loose objects > etc. That's a future option, but way past the initial concept for user on-boarding. Thanks Philip
On Thu, Nov 03 2022, Philip Oakley wrote: > Hi Ævar, > Thanks for the review, > > On 03/11/2022 09:19, Ævar Arnfjörð Bjarmason wrote: > My main point was that the examples need to be covered by the test > suite, and that bundles were a simple way of capturing the tested setup. > The GIT_BUNDLE_DIR was simply a way of collecting them in one place for > ease of distribution with the documentation (as you mentioned that's a > gap in the automation step) >> i.e. with a minor variation of that you can do this >> in test-lib.sh: >> >> TEST_RESULTS_BDL_DIR="$TEST_RESULTS_DIR/$TEST_NAME.test-bdl" >> >> Then have a helper like: >> >> do_my_test_bundle_stuff () { >> local name="$1" && >> shift && >> git bundle create "$TEST_RESULTS_BDL_DIR/$name.bdl" "$@" >> } > That would be a simplification I forgot that there needs to be a "mkdir" of the "$TEST_RESULTS_BDL_DIR" in there somewhere (see the *.leak dir setup), but other than that that's pretty much all there's to it. >> Then all you need to make these is: >> >> make test GIT_TEST_OPTS=--debug > > I didn't quite get the link in that step between the `--debug` and the > bndl prep. Sorry, I was misrecalling that we wiped away "test-results" except without "--debug", which when I spend more than 2 seconds thinking about it makes no sense, that's just for the trash directories. Nevermind. >> And then: >> >> ls -l t/test-results/*.bdl/* >> >> Will give you all the bundles you want. >> >> We're then missing the "connect it with the docs" part, but that part >> would be e.g.: >> >> - Have a validation target for it that depends on such a "make test" run >> - Sanity check that we did create the ones the docs expect >> >> Anyway. >> >> I like the end-goals, but I can't help think that bundles are the wrong >> direction to go in, as opposed to: >> >> 1. We could have "make install" ship test-lib{,-functions}.sh along >> with git itself >> 2. Have e.g. t/doc-setup/*.sh with the snippets to set up these test >> demos, which we'd also ship. > > I'm not into the idea of shipping a load of extra `test` infrastructure > with Git itself. We don't ship any in the Git for Windows install. We don't ship in anywhere now, but presumably GfW has the required parts to use it, since it has git-submodule.sh and other *.sh now? Anyway.... > I'd viewed the bundles as an adjunct to the also independent manuals and > documentation. ...yeah me too, I'm just wondering if we won't back ourselves into a corner with some examples, where we'll want to hand the user more complex state. But maybe simpler is better, and it can just be a bundle plus "run these commands manually". The bundle can also contain a script in-tree that performs the finishing touches, if required (unless there's no 'sh' at runtime in GfW?). Anyway, *if* that's a good idea (and I've got no idea, just throwing it out there) it doesn't need to be hindered by lack of 'sh', we could just ship compiled C code that does the required run_command() sequences to set up the desired state. Then in the test suite we'd call that as a test helper, or a new built-in helper,. >> >> And you'd then do e.g.: >> >> git test-case rebase/description-1 >> >> Which would just be a thin wrapper around (pseudocode): >> >> dir=$(git test--helper get-tempdir) && >> ( >> cd "$dir" && >> # And whatever else we need to set up to say "use this local installed git" >> GIT_TEST_INSTALLED=[...] && >> . "$(git --test-dir)"/test-lib.sh >> . "$(git --test-dir)"/doc-setup/rebase-description-1.sh" >> >> # This would run test_commit, and any other arbitrary setup code >> setup_rebase_description_1() >> ) && >> echo "Go play with '$dir', and have fun" >> >> The big advantage of that is: >> >> 1. It ships with git, so it works offline, and we won't get into the >> inevitable confusion of the user reading v2.40 docs locally, but the >> online url tracking a v2.50 example or whatever. > > Isn't this always a problem (lack of synchronisation between multiple > sources). If the user has local documentation, it should have matching > bundles locally, No? Yes, I just didn't know if that was your plan, so if that's how it works then no problem. It's just a depressingly common pattern these days that manual pages are "just read about it at this link", which inevitably links to some out-of-date documentation, even some of GCC's docs are doing that these days. It's good that that's not the plan here. >> 2. We can represent whatever arbitrary repo state, e.g. drop the user >> into a merge conflict that's already happening. > Isn't the manual's example already meant to describe that? And the user > then (hopefully) follows along in their throw-away copy of the repo bundle. Yeah, maybe that's fine. >> 3. We can also represent "repo-extra" state, e.g. imagine the >> "git-gc/git-repack" docs discussing loose objects, and wanting to >> get you into some arbitrary state with regards to your loose objects >> etc. > That's a future option, but way past the initial concept for user > on-boarding.
diff --git a/Documentation/technical/docbundle.txt b/Documentation/technical/docbundle.txt new file mode 100644 index 0000000000..8c17a7847f --- /dev/null +++ b/Documentation/technical/docbundle.txt @@ -0,0 +1,97 @@ +docbundle Design Document +========================= + +The Git man pages contain many example setups to demonstrate aspects +of the related Git commands, such as history simplification in the `log` +and `show commands. However, while illustrative, these are not +accessible, as example repositories, to regular, potentially confused, +users who may need to understand the set up of the examples. + +The docbundle concept is that for each of these examples there is a +matching `bundle` file, provided with the Documentation, which can be +simply cloned to provide an example repository containing the example. + +There are also many example repositories in the Git `/t` test system +that could be exposed to users which already demonstrate some important +testable aspect of Git within a carefully constructed repository. + + +This proposal is to provide DocBundles to users via the Git project. + + +Firstly, it should be reasonably self-evident that the man page examples +should, in themselves, be tested for accuracy, especially if provided +to users. + +Secondly, the existing test system is the proper place to setup and +test any such docbundle repositories. + +Third, that the bundle is an obvious pre-existing mechanism for +transporting repositories (see also the Bundle-URI proposals). + +Fourthly, the preparation and distribution of the docbundle should +require minimal maintainer effort, preferably being integrated into +the existing automation for Documentation preparation and distribution. + +Fifthly, the docbudles themselves should not to be part of the git.git +repo. They are prepared via the test suite and should be stored (locally) +in a designated Git_DocBundle_Dir, and be .gitignored. The DocBundles +are transferred to the maintainer's Doc repo via the automation. + +Sixthly, the docbundles to be named based on their `tnnnn` test number +and a (short) descriptive name. + +Seven, (dev) start with an existing test, rather than add a test (most +doc/man examples aren't tested!). Add tests for missing examples later. + +Eight, (option) add a `git docbundle` command to assist in locating and +listing the docbundle examples. It also provides place to explain their +purpose as "Tutorial Examples" (Note, many existing examples are to +illustrate complex/difficult scenarios where the wording was insufficient; +the docbudles are minimal examples). + + +Status +====== + +This is an RFC/PoC (Request for Comment / Proof of Concept). + +The attached commits establish: + +Set out this design in the Documentation/technical directory. + +A default location for the docbundles and an env variable to allow +flexibility by extending test-lib. + +The docbundles are ignored, as is the default directory. + +Two docbundles are generated from a 'random' test that I happened to +have open in my editor (t6102 rev-list-simplify). + +The docbundle creation is protected by `test_expect_success` wrappers. + + + +ToDo +==== + +DocBundles - Is this the best name, or just a convenient way of +indicating the delivery mechanism? + +Add Prerequisite to control if docbundles are generated (don't waste +maintainers PC time when not needed). + +CI integration (i.e. whether to test the bundling aspect - should be +reliable). + +How to update the man pages, perhaps by inclusion of the test comment +(exported?), and distinguish between man page descriptions (being +packaged by the test suite) and tests which offer users insight into +Git operations, usually with 'interesting' commit-graph set-ups. + +Claim t999x as the man doc bundle test sequence ('999' is the UK +emergency ("Help!") number; 911, 101 and 102 are taken test areas. + +Other stuff implicit in the eight point list + +
DocBundles are exemplar repositories, provided as bundles, which contain either 'interesting' test setups for exploration, or the described examples from within the man pages. Signed-off-by: Philip Oakley <philipoakley@iee.email> --- Documentation/technical/docbundle.txt | 97 +++++++++++++++++++++++++++ 1 file changed, 97 insertions(+) create mode 100644 Documentation/technical/docbundle.txt