| Message ID | pull.1452.v4.git.git.1676927082.gitgitgadget@gmail.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | Teach diff to honor diff algorithms set through git attributes | expand |
"John Cai via GitGitGadget" <gitgitgadget@gmail.com> writes: > When a repository contains different kinds of files, it may be desirable to > use different algorithms based on file type. This is currently not feasible > through the command line or using git configs. However, we can leverage the > fact that gitattributes are path aware. > ... > To address some of the performance concerns in the previous series, a > benchmark shows that now only a minor performance penalty is incurred, now > that we are no longer adding an additional attributes parsing call: > > $ echo "*.[ch] diff=other" >> .gitattributes $ hyperfine -r 10 -L a > git-bin-wrapper,git '{a} -c diff.other.algorithm=myers diff v2.0.0 v2.28.0' > Benchmark 1: git-bin-wrapper -c diff.other.algorithm=myers diff v2.0.0 > v2.28.0 Time (mean ± σ): 716.3 ms ± 3.8 ms [User: 660.2 ms, System: 50.8 ms] > Range (min … max): 709.8 ms … 720.6 ms 10 runs > > Benchmark 2: git -c diff.other.algorithm=myers diff v2.0.0 v2.28.0 Time > (mean ± σ): 704.3 ms ± 2.9 ms [User: 656.6 ms, System: 44.3 ms] Range (min … > max): 700.1 ms … 708.6 ms 10 runs > > Summary 'git -c diff.other.algorithm=myers diff v2.0.0 v2.28.0' ran 1.02 ± > 0.01 times faster than 'git-bin-wrapper -c diff.other.algorithm=myers diff > v2.0.0 v2.28.0' Hopefully this round can immediately be merged down to 'next'? Thanks.
On Tue, Feb 21, 2023 at 9:34 AM Junio C Hamano <gitster@pobox.com> wrote: > > "John Cai via GitGitGadget" <gitgitgadget@gmail.com> writes: > > > When a repository contains different kinds of files, it may be desirable to > > use different algorithms based on file type. This is currently not feasible > > through the command line or using git configs. However, we can leverage the > > fact that gitattributes are path aware. > > ... > > To address some of the performance concerns in the previous series, a > > benchmark shows that now only a minor performance penalty is incurred, now > > that we are no longer adding an additional attributes parsing call: > > > > $ echo "*.[ch] diff=other" >> .gitattributes $ hyperfine -r 10 -L a > > git-bin-wrapper,git '{a} -c diff.other.algorithm=myers diff v2.0.0 v2.28.0' > > Benchmark 1: git-bin-wrapper -c diff.other.algorithm=myers diff v2.0.0 > > v2.28.0 Time (mean ± σ): 716.3 ms ± 3.8 ms [User: 660.2 ms, System: 50.8 ms] > > Range (min … max): 709.8 ms … 720.6 ms 10 runs > > > > Benchmark 2: git -c diff.other.algorithm=myers diff v2.0.0 v2.28.0 Time > > (mean ± σ): 704.3 ms ± 2.9 ms [User: 656.6 ms, System: 44.3 ms] Range (min … > > max): 700.1 ms … 708.6 ms 10 runs > > > > Summary 'git -c diff.other.algorithm=myers diff v2.0.0 v2.28.0' ran 1.02 ± > > 0.01 times faster than 'git-bin-wrapper -c diff.other.algorithm=myers diff > > v2.0.0 v2.28.0' > > Hopefully this round can immediately be merged down to 'next'? > Thanks. I'll leave that up to you and John, but are we risking merging code that could go unused or that we need to fundamentally change? I don't see how to handle the issues over at https://lore.kernel.org/git/647D3D49-B85B-4B66-A857-695CFF9685EE@gmail.com/ (not that I've spent much thought on it besides asking how things are going to work), and would be worried that we'd end up needing to change the mechanism somehow to cover the things John ultimately wants.
Elijah Newren <newren@gmail.com> writes: > I'll leave that up to you and John, but are we risking merging code > that could go unused or that we need to fundamentally change? I don't > see how to handle the issues over at > https://lore.kernel.org/git/647D3D49-B85B-4B66-A857-695CFF9685EE@gmail.com/ If this is useful enough for desktop users already, then that is a good enough reason to take it, I would say. GitLab can easily add WebUI that says "You can define what diff algorithm is used for files with which suffix" to allow you to configure a table like this: json | histogram c | patience * | myers and populate the server-side equivalent of .git/info/attributes and .git/config based on that, and without anything further than the posted patches, the result should just work, no?
On 21 Feb 2023, at 13:51, Junio C Hamano wrote: > Elijah Newren <newren@gmail.com> writes: > >> I'll leave that up to you and John, but are we risking merging code >> that could go unused or that we need to fundamentally change? I don't >> see how to handle the issues over at >> https://lore.kernel.org/git/647D3D49-B85B-4B66-A857-695CFF9685EE@gmail.com/ > > If this is useful enough for desktop users already, then that is a > good enough reason to take it, I would say. > > GitLab can easily add WebUI that says "You can define what diff > algorithm is used for files with which suffix" to allow you to > configure a table like this: > > json | histogram > c | patience > * | myers > > and populate the server-side equivalent of .git/info/attributes and > .git/config based on that, and without anything further than the > posted patches, the result should just work, no? Yes, that is my thinking too. The goal was to make this user friendly for everyone on the Git command line, and then GitLab can hook into this accordingly to make it ussable with the GitLab UI. thanks John
On Tue, Feb 21, 2023 at 11:36 AM John Cai <johncai86@gmail.com> wrote: > > On 21 Feb 2023, at 13:51, Junio C Hamano wrote: > > > Elijah Newren <newren@gmail.com> writes: > > > >> I'll leave that up to you and John, but are we risking merging code > >> that could go unused or that we need to fundamentally change? I don't > >> see how to handle the issues over at > >> https://lore.kernel.org/git/647D3D49-B85B-4B66-A857-695CFF9685EE@gmail.com/ > > > > If this is useful enough for desktop users already, then that is a > > good enough reason to take it, I would say. > > > > GitLab can easily add WebUI that says "You can define what diff > > algorithm is used for files with which suffix" to allow you to > > configure a table like this: > > > > json | histogram > > c | patience > > * | myers > > > > and populate the server-side equivalent of .git/info/attributes and > > .git/config based on that, and without anything further than the > > posted patches, the result should just work, no? > > Yes, that is my thinking too. The goal was to make this user friendly for > everyone on the Git command line, and then GitLab can hook into this accordingly > to make it ussable with the GitLab UI. Ok, sounds good. Merge away. :-)
When a repository contains different kinds of files, it may be desirable to use different algorithms based on file type. This is currently not feasible through the command line or using git configs. However, we can leverage the fact that gitattributes are path aware. Teach the diff machinery to check gitattributes when diffing files by using the existing diff. scheme, and add an "algorithm" type to the external driver config. Change since V3: * cleaned up documentation, typos * minor cleanup such as if statement ordering, and overly long lines Changes since V2: * minor clean up and variable renaming * avoid parsing attribute files for the driver if the diff algorithm is set through the command line Changes since V1: * utilize the existing diff.<driver>.* scheme where the driver is defined in gitattributes, but the algorithm is defined in the gitconfig. To address some of the performance concerns in the previous series, a benchmark shows that now only a minor performance penalty is incurred, now that we are no longer adding an additional attributes parsing call: $ echo "*.[ch] diff=other" >> .gitattributes $ hyperfine -r 10 -L a git-bin-wrapper,git '{a} -c diff.other.algorithm=myers diff v2.0.0 v2.28.0' Benchmark 1: git-bin-wrapper -c diff.other.algorithm=myers diff v2.0.0 v2.28.0 Time (mean ± σ): 716.3 ms ± 3.8 ms [User: 660.2 ms, System: 50.8 ms] Range (min … max): 709.8 ms … 720.6 ms 10 runs Benchmark 2: git -c diff.other.algorithm=myers diff v2.0.0 v2.28.0 Time (mean ± σ): 704.3 ms ± 2.9 ms [User: 656.6 ms, System: 44.3 ms] Range (min … max): 700.1 ms … 708.6 ms 10 runs Summary 'git -c diff.other.algorithm=myers diff v2.0.0 v2.28.0' ran 1.02 ± 0.01 times faster than 'git-bin-wrapper -c diff.other.algorithm=myers diff v2.0.0 v2.28.0' John Cai (2): diff: consolidate diff algorithm option parsing diff: teach diff to read algorithm from diff driver Documentation/gitattributes.txt | 31 ++++++++++++ diff.c | 90 ++++++++++++++++++++++++--------- diff.h | 1 + t/lib-diff-alternative.sh | 38 +++++++++++++- userdiff.c | 4 +- userdiff.h | 1 + 6 files changed, 140 insertions(+), 25 deletions(-) base-commit: c867e4fa180bec4750e9b54eb10f459030dbebfd Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1452%2Fjohn-cai%2Fjc%2Fattr-diff-algo-v4 Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1452/john-cai/jc/attr-diff-algo-v4 Pull-Request: https://github.com/git/git/pull/1452 Range-diff vs v3: 1: 816c47aa414 = 1: 816c47aa414 diff: consolidate diff algorithm option parsing 2: b330222ce83 ! 2: 77e66ab98fc diff: teach diff to read algorithm from diff driver @@ Commit message finally the diff.algorithm config. To enforce precedence order, use a new `ignore_driver_algorithm` member - during options pasing to indicate the diff algorithm was set via command + during options parsing to indicate the diff algorithm was set via command line args. Signed-off-by: John Cai <johncai86@gmail.com> @@ Documentation/gitattributes.txt: with the above configuration, i.e. `j-c-diff`, +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The diff algorithm can be set through the `diff.algorithm` config key, but -+sometimes it may be helpful to set the diff algorithm by path. For example, one -+might wish to set a diff algorithm automatically for all `.json` files such that -+the user would not need to pass in a separate command line `--diff-algorithm` -+flag each time. ++sometimes it may be helpful to set the diff algorithm per path. For example, ++one may want to use the `minimal` diff algorithm for .json files, and the ++`histogram` for .c files, and so on without having to pass in the algorithm ++through the command line each time. + +First, in `.gitattributes`, assign the `diff` attribute for paths. + @@ Documentation/gitattributes.txt: with the above configuration, i.e. `j-c-diff`, +------------------------ + +Then, define a "diff.<name>.algorithm" configuration to specify the diff -+algorithm, choosing from `meyers`, `patience`, `minimal`, or `histogram`. ++algorithm, choosing from `myers`, `patience`, `minimal`, or `histogram`. + +---------------------------------------------------------------- +[diff "<name>"] @@ Documentation/gitattributes.txt: with the above configuration, i.e. `j-c-diff`, +git-show(1) and is used for the `--stat` output as well. The merge machinery +will not use the diff algorithm set through this method. + -+NOTE: If the `command` key also exists, then Git will treat this as an external -+diff and attempt to use the value set for `command` as an external program. For -+instance, the following config, combined with the above `.gitattributes` file, -+will result in `command` favored over `algorithm`. -+ -+---------------------------------------------------------------- -+[diff "<name>"] -+ command = j-c-diff -+ algorithm = histogram -+---------------------------------------------------------------- ++NOTE: If `diff.<name>.command` is defined for path with the ++`diff=<name>` attribute, it is executed as an external diff driver ++(see above), and adding `diff.<name>.algorithm` has no effect, as the ++algorithm is not passed to the external diff driver. Defining a custom hunk-header ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ diff.c: static void run_diff_cmd(const char *pgm, } - if (one && two) + if (one && two) { -+ if (drv && !o->ignore_driver_algorithm && drv->algorithm) ++ if (!o->ignore_driver_algorithm && drv && drv->algorithm) + set_diff_algorithm(o, drv->algorithm); + builtin_diff(name, other ? other : name, @@ diff.c: static void run_diffstat(struct diff_filepair *p, struct diff_options *o const char *other; + if (!o->ignore_driver_algorithm) { -+ struct userdiff_driver *drv = userdiff_find_by_path(o->repo->index, p->one->path); ++ struct userdiff_driver *drv = userdiff_find_by_path(o->repo->index, ++ p->one->path); + -+ if (drv && drv->algorithm) { ++ if (drv && drv->algorithm) + set_diff_algorithm(o, drv->algorithm); -+ } + } + if (DIFF_PAIR_UNMERGED(p)) { @@ t/lib-diff-alternative.sh: index $file1..$file2 100644 + + test_expect_success "$STRATEGY diff command line precedence before attributes" ' + echo "file* diff=driver" >.gitattributes && -+ git config diff.driver.algorithm meyers && ++ git config diff.driver.algorithm myers && + test_must_fail git diff --no-index "--diff-algorithm=$STRATEGY" file1 file2 > output && + test_cmp expect output + '