mbox series

[00/12] fsmonitor: Implement fsmonitor for Linux

Message ID pull.1352.git.git.1665326258.gitgitgadget@gmail.com (mailing list archive)
Headers show
Series fsmonitor: Implement fsmonitor for Linux | expand

Message

Philippe Blain via GitGitGadget Oct. 9, 2022, 2:37 p.m. UTC
Goal is to deliver fsmonitor for Linux that is on par with fsmonitor for
Windows and Mac OS.

This patch set builds upon previous work for done for Windows and Mac OS
(first 6 patches) to implement a fsmonitor back-end for Linux based on the
Linux inotify API. inotify differs significantly from the equivalent Windows
and Mac OS APIs in that a watch must be registered for every directory of
interest (rather than a singular watch at the root of the directory tree)
and special care must be taken to handle directory renames correctly.

More information about inotify:
https://man7.org/linux/man-pages/man7/inotify.7.html

Eric DeCosta (12):
  fsmonitor: refactor filesystem checks to common interface
  fsmonitor: relocate socket file if .git directory is remote
  fsmonitor: avoid socket location check if using hook
  fsmonitor: deal with synthetic firmlinks on macOS
  fsmonitor: check for compatability before communicating with fsmonitor
  fsmonitor: add documentation for allowRemote and socketDir options
  fsmonitor: prepare to share code between Mac OS and Linux
  fsmonitor: determine if filesystem is local or remote
  fsmonitor: implement filesystem change listener for Linux
  fsmonitor: enable fsmonitor for Linux
  fsmonitor: test updates
  fsmonitor: update doc for Linux

 Documentation/config.txt                      |   2 +
 Documentation/config/fsmonitor--daemon.txt    |  11 +
 Documentation/git-fsmonitor--daemon.txt       |  45 +-
 Makefile                                      |   6 +-
 builtin/fsmonitor--daemon.c                   |  11 +-
 ...{fsm-health-darwin.c => fsm-health-unix.c} |   0
 compat/fsmonitor/fsm-ipc-unix.c               |  52 ++
 compat/fsmonitor/fsm-ipc-win32.c              |   9 +
 compat/fsmonitor/fsm-listen-darwin.c          |  14 +-
 compat/fsmonitor/fsm-listen-linux.c           | 664 ++++++++++++++++++
 compat/fsmonitor/fsm-path-utils-darwin.c      | 135 ++++
 compat/fsmonitor/fsm-path-utils-linux.c       | 163 +++++
 compat/fsmonitor/fsm-path-utils-win32.c       | 145 ++++
 compat/fsmonitor/fsm-settings-darwin.c        |  89 ---
 compat/fsmonitor/fsm-settings-unix.c          |  62 ++
 compat/fsmonitor/fsm-settings-win32.c         | 174 +----
 config.mak.uname                              |  13 +
 contrib/buildsystems/CMakeLists.txt           |  19 +-
 fsmonitor--daemon.h                           |   3 +
 fsmonitor-ipc.c                               |  18 +-
 fsmonitor-ipc.h                               |   4 +-
 fsmonitor-path-utils.h                        |  60 ++
 fsmonitor-settings.c                          |  68 +-
 fsmonitor-settings.h                          |   4 +-
 fsmonitor.c                                   |   7 +
 t/t7527-builtin-fsmonitor.sh                  |  58 +-
 26 files changed, 1531 insertions(+), 305 deletions(-)
 create mode 100644 Documentation/config/fsmonitor--daemon.txt
 rename compat/fsmonitor/{fsm-health-darwin.c => fsm-health-unix.c} (100%)
 create mode 100644 compat/fsmonitor/fsm-ipc-unix.c
 create mode 100644 compat/fsmonitor/fsm-ipc-win32.c
 create mode 100644 compat/fsmonitor/fsm-listen-linux.c
 create mode 100644 compat/fsmonitor/fsm-path-utils-darwin.c
 create mode 100644 compat/fsmonitor/fsm-path-utils-linux.c
 create mode 100644 compat/fsmonitor/fsm-path-utils-win32.c
 delete mode 100644 compat/fsmonitor/fsm-settings-darwin.c
 create mode 100644 compat/fsmonitor/fsm-settings-unix.c
 create mode 100644 fsmonitor-path-utils.h


base-commit: 3dcec76d9df911ed8321007b1d197c1a206dc164
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1352%2Fedecosta-mw%2Ffsmonitor_linux-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1352/edecosta-mw/fsmonitor_linux-v1
Pull-Request: https://github.com/git/git/pull/1352

Comments

Junio C Hamano Oct. 9, 2022, 10:24 p.m. UTC | #1
"Eric DeCosta via GitGitGadget" <gitgitgadget@gmail.com> writes:

> Goal is to deliver fsmonitor for Linux that is on par with fsmonitor for
> Windows and Mac OS.
>
> This patch set builds upon previous work for done for Windows and Mac OS
> (first 6 patches) ...

For those who are watching from sidelines...

The Windows part is already in Git 2.38; the changes needed for
macOS are already in 'next' and the first 6 patches in this 12-patch
series are identical to them.  The patches 7-12 are new.

> to implement a fsmonitor back-end for Linux based on the
> Linux inotify API. inotify differs significantly from the equivalent Windows
> and Mac OS APIs in that a watch must be registered for every directory of
> interest (rather than a singular watch at the root of the directory tree)
> and special care must be taken to handle directory renames correctly.
Eric Sunshine Oct. 10, 2022, 12:19 a.m. UTC | #2
On Sun, Oct 9, 2022 at 7:55 PM Junio C Hamano <gitster@pobox.com> wrote:
> "Eric DeCosta via GitGitGadget" <gitgitgadget@gmail.com> writes:
> > Goal is to deliver fsmonitor for Linux that is on par with fsmonitor for
> > Windows and Mac OS.
> >
> > This patch set builds upon previous work for done for Windows and Mac OS
> > (first 6 patches) ...
>
> For those who are watching from sidelines...
>
> The Windows part is already in Git 2.38; the changes needed for
> macOS are already in 'next' and the first 6 patches in this 12-patch
> series are identical to them.  The patches 7-12 are new.

Thanks for clarifying. I found it confusing that there were a number
of patches in this series which I had already seen despite the cover
letter's claim that this series builds upon "previous work". Thus, it
wasn't clear whether this series was a reroll (refining some
already-seen patches) with additional patches for Linux, or if it was
purely new work with the original patches included by accident[1].

[1]: In the few times I've used GitGitGadget, I've had to jump through
hoops to make it send just the "new" patches when I've built a series
atop some other series only in 'next' or 'seen', so I can understand
the inclusion of the first six patches being accidental. (Regarding
the hoop-jumping, it may be that I just don't understand how to "work"
GitGitGadget or GitHub pull-requests.)
Junio C Hamano Oct. 10, 2022, 9:08 p.m. UTC | #3
"Eric DeCosta via GitGitGadget" <gitgitgadget@gmail.com> writes:

> Goal is to deliver fsmonitor for Linux that is on par with fsmonitor for
> Windows and Mac OS.
>
> This patch set builds upon previous work for done for Windows and Mac OS
> (first 6 patches) to implement a fsmonitor back-end for Linux based on the
> Linux inotify API. inotify differs significantly from the equivalent Windows
> and Mac OS APIs in that a watch must be registered for every directory of
> interest (rather than a singular watch at the root of the directory tree)
> and special care must be taken to handle directory renames correctly.

I am seeing occasional breakages of t7527.16 that does not seem to
reproduce reliably.
Ævar Arnfjörð Bjarmason Oct. 18, 2022, 12:47 p.m. UTC | #4
On Sun, Oct 09 2022, Eric DeCosta via GitGitGadget wrote:

> Goal is to deliver fsmonitor for Linux that is on par with fsmonitor for
> Windows and Mac OS.

I applaud the effort, I gave this some light reviewing, I hope it helps.

> This patch set builds upon previous work for done for Windows and Mac OS
> (first 6 patches) to implement a fsmonitor back-end for Linux based on the
> Linux inotify API. inotify differs significantly from the equivalent Windows
> and Mac OS APIs in that a watch must be registered for every directory of
> interest (rather than a singular watch at the root of the directory tree)
> and special care must be taken to handle directory renames correctly.
>
> More information about inotify:
> https://man7.org/linux/man-pages/man7/inotify.7.html

You haven't said why/if you considered using fanotify() instead of
inotify(), which seems like a more natural target to me. It's unlikely
that anyone who cares to use this isn't also using a new enough kernel.

See previous on-list discussions at:
https://lore.kernel.org/git/?q=fanotify

I think it would address some of the rac econditions you're mentioning
here.