| Message ID | 3cece716fc09724793aa832e755abfc9d70a8bb3.1684892404.git-series.apopple@nvidia.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [1/2] mmu_notifiers: Restore documentation for .invalidate_range() | expand |
On 5/23/23 18:47, Alistair Popple wrote: > The .invalidate_range() callback is called by > mmu_notifier_invalidate_range() which is often called while holding > the ptl spin-lock. Therefore any implementations of this callback must > not sleep. This was originally documented when the call back was added > in commit 0f0a327fa12c ("mmu_notifier: add the callback for > mmu_notifier_invalidate_range()") but appears to have been Thanks for digging into this. I expect that you're on the right track, I'm just wondering about something still: > inadvertently removed by commit 5ff7091f5a2c ("mm, mmu_notifier: > annotate mmu notifiers with blockable invalidate callbacks"). Was it really inadvertent, though? The initial patch proposed said this: "Also remove a bogus comment about invalidate_range() always being called under the ptl spinlock." [1] I've added David Rientjes to CC. I almost think we should rename the callback to something with "non blocking" or similar in the name. It not great to have to do this much research to figure out the intent. And it still feels backwards. [1] https://lore.kernel.org/all/alpine.DEB.2.10.1801091339570.240101@chino.kir.corp.google.com/T/#u > > Restore the comment to make it clear that .invalidate_range() > callbacks may not sleep. > > Signed-off-by: Alistair Popple <apopple@nvidia.com> > --- > include/linux/mmu_notifier.h | 3 +++ > 1 file changed, 3 insertions(+) > > diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h > index 64a3e05..447d757 100644 > --- a/include/linux/mmu_notifier.h > +++ b/include/linux/mmu_notifier.h > @@ -200,6 +200,9 @@ struct mmu_notifier_ops { > * external TLB range needs to be flushed. For more in depth > * discussion on this see Documentation/mm/mmu_notifier.rst > * > + * The invalidate_range() function is called under the ptl > + * spin-lock and not allowed to sleep. > + * > * Note that this function might be called with just a sub-range > * of what was passed to invalidate_range_start()/end(), if > * called between those functions. > > base-commit: 44c026a73be8038f03dbdeef028b642880cf1511 Hooray for --base! :) thanks,
On Wed, 24 May 2023 11:47:28 +1000
Alistair Popple <apopple@nvidia.com> wrote:
It is actually commit 4e15a073a168 ("Revert "mm, mmu_notifier:
annotate mmu notifiers with blockable invalidate callbacks"")'s problem.
It tries to revert commit 5ff7091f5a2c but forgets to get back this
piece of comment.
It would be better to have a Fixes tag in the commit message.
> The .invalidate_range() callback is called by
> mmu_notifier_invalidate_range() which is often called while holding
> the ptl spin-lock. Therefore any implementations of this callback must
> not sleep. This was originally documented when the call back was added
> in commit 0f0a327fa12c ("mmu_notifier: add the callback for
> mmu_notifier_invalidate_range()") but appears to have been
> inadvertently removed by commit 5ff7091f5a2c ("mm, mmu_notifier:
> annotate mmu notifiers with blockable invalidate callbacks").
>
> Restore the comment to make it clear that .invalidate_range()
> callbacks may not sleep.
>
> Signed-off-by: Alistair Popple <apopple@nvidia.com>
> ---
> include/linux/mmu_notifier.h | 3 +++
> 1 file changed, 3 insertions(+)
>
> diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h
> index 64a3e05..447d757 100644
> --- a/include/linux/mmu_notifier.h
> +++ b/include/linux/mmu_notifier.h
> @@ -200,6 +200,9 @@ struct mmu_notifier_ops {
> * external TLB range needs to be flushed. For more in depth
> * discussion on this see Documentation/mm/mmu_notifier.rst
> *
> + * The invalidate_range() function is called under the ptl
> + * spin-lock and not allowed to sleep.
> + *
> * Note that this function might be called with just a sub-range
> * of what was passed to invalidate_range_start()/end(), if
> * called between those functions.
>
> base-commit: 44c026a73be8038f03dbdeef028b642880cf1511
John Hubbard <jhubbard@nvidia.com> writes: > On 5/23/23 18:47, Alistair Popple wrote: >> The .invalidate_range() callback is called by >> mmu_notifier_invalidate_range() which is often called while holding >> the ptl spin-lock. Therefore any implementations of this callback must >> not sleep. This was originally documented when the call back was added >> in commit 0f0a327fa12c ("mmu_notifier: add the callback for >> mmu_notifier_invalidate_range()") but appears to have been > > Thanks for digging into this. I expect that you're on the right > track, I'm just wondering about something still: > >> inadvertently removed by commit 5ff7091f5a2c ("mm, mmu_notifier: >> annotate mmu notifiers with blockable invalidate callbacks"). > > Was it really inadvertent, though? The initial patch proposed said this: > > "Also remove a bogus comment about invalidate_range() always being called > under the ptl spinlock." [1] Perhaps it was imprecise, inaccurate or incorrect. I'm not entirely sure but wanted to give benefit of doubt. The point is the commit log doesn't describe the patch correctly. It claims there was "a bogus comment about invalidate_range() *always* being called under the ptl spinlock". That's not correct - the comment which it actually removed (and which is added back here) simply says it is called under the ptl and not allow to sleep. Nothing in that series changed that. I suppose if you wanted to be really explicit the comment could read: * The invalidate_range() function may be called under the ptl * spin-lock and is therefore not allowed to sleep. But I don't mind either way, both clearly communicate the important point which is it can't sleep and AFICT that has always been the case. > I've added David Rientjes to CC. Thanks, I had meant to add them to cc myself. > I almost think we should rename the callback to something with > "non blocking" or similar in the name. It not great to have to > do this much research to figure out the intent. And it still feels > backwards. Hopefully the comments reduce the amount of research required and nothing can replace reading the documentation/comment for a subsystem prior to implementing the required callbacks, etc anyway. I don't think adding _non_blocking to the end of function names is particularly helpful though unless it's to differentiate between different versions of the same thing. In this case there's no _blocking version so it would just add confusion IMHO. > [1] https://lore.kernel.org/all/alpine.DEB.2.10.1801091339570.240101@chino.kir.corp.google.com/T/#u > > >> Restore the comment to make it clear that .invalidate_range() >> callbacks may not sleep. >> Signed-off-by: Alistair Popple <apopple@nvidia.com> >> --- >> include/linux/mmu_notifier.h | 3 +++ >> 1 file changed, 3 insertions(+) >> diff --git a/include/linux/mmu_notifier.h >> b/include/linux/mmu_notifier.h >> index 64a3e05..447d757 100644 >> --- a/include/linux/mmu_notifier.h >> +++ b/include/linux/mmu_notifier.h >> @@ -200,6 +200,9 @@ struct mmu_notifier_ops { >> * external TLB range needs to be flushed. For more in depth >> * discussion on this see Documentation/mm/mmu_notifier.rst >> * >> + * The invalidate_range() function is called under the ptl >> + * spin-lock and not allowed to sleep. >> + * >> * Note that this function might be called with just a sub-range >> * of what was passed to invalidate_range_start()/end(), if >> * called between those functions. >> base-commit: 44c026a73be8038f03dbdeef028b642880cf1511 > > Hooray for --base! :) Heh. I usually use git-series to generate the patches which adds --base automatically but I occasionally don't when sending a single patch fix and end up forgetting it. > thanks,
Zhi Wang <zhi.wang.linux@gmail.com> writes: > On Wed, 24 May 2023 11:47:28 +1000 > Alistair Popple <apopple@nvidia.com> wrote: > > It is actually commit 4e15a073a168 ("Revert "mm, mmu_notifier: > annotate mmu notifiers with blockable invalidate callbacks"")'s problem. > It tries to revert commit 5ff7091f5a2c but forgets to get back this > piece of comment. See my response to John [1]. I don't think it was entirely correct to remove the comment in the first place but I haven't reviewed that whole series given it got mostly reverted so I'm probably missing something. The important thing is the comment is put back and that it was and still is true. There is enough history in the commit message for people to dig through how it happened if it's of interest, but commit 4e15a073a168 seems like a reasonble enough thing to put as a Fixes tag so will add that. [1] - https://lore.kernel.org/linux-mm/87lehe72v4.fsf@nvidia.com/ > It would be better to have a Fixes tag in the commit message. > >> The .invalidate_range() callback is called by >> mmu_notifier_invalidate_range() which is often called while holding >> the ptl spin-lock. Therefore any implementations of this callback must >> not sleep. This was originally documented when the call back was added >> in commit 0f0a327fa12c ("mmu_notifier: add the callback for >> mmu_notifier_invalidate_range()") but appears to have been >> inadvertently removed by commit 5ff7091f5a2c ("mm, mmu_notifier: >> annotate mmu notifiers with blockable invalidate callbacks"). >> >> Restore the comment to make it clear that .invalidate_range() >> callbacks may not sleep. >> >> Signed-off-by: Alistair Popple <apopple@nvidia.com> >> --- >> include/linux/mmu_notifier.h | 3 +++ >> 1 file changed, 3 insertions(+) >> >> diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h >> index 64a3e05..447d757 100644 >> --- a/include/linux/mmu_notifier.h >> +++ b/include/linux/mmu_notifier.h >> @@ -200,6 +200,9 @@ struct mmu_notifier_ops { >> * external TLB range needs to be flushed. For more in depth >> * discussion on this see Documentation/mm/mmu_notifier.rst >> * >> + * The invalidate_range() function is called under the ptl >> + * spin-lock and not allowed to sleep. >> + * >> * Note that this function might be called with just a sub-range >> * of what was passed to invalidate_range_start()/end(), if >> * called between those functions. >> >> base-commit: 44c026a73be8038f03dbdeef028b642880cf1511
On Tue, May 23, 2023 at 07:20:31PM -0700, John Hubbard wrote: > On 5/23/23 18:47, Alistair Popple wrote: > > The .invalidate_range() callback is called by > > mmu_notifier_invalidate_range() which is often called while holding > > the ptl spin-lock. Therefore any implementations of this callback must > > not sleep. This was originally documented when the call back was added > > in commit 0f0a327fa12c ("mmu_notifier: add the callback for > > mmu_notifier_invalidate_range()") but appears to have been > > Thanks for digging into this. I expect that you're on the right > track, I'm just wondering about something still: > > > inadvertently removed by commit 5ff7091f5a2c ("mm, mmu_notifier: > > annotate mmu notifiers with blockable invalidate callbacks"). > > Was it really inadvertent, though? The initial patch proposed said this: > > "Also remove a bogus comment about invalidate_range() always being called > under the ptl spinlock." [1] Right, it is not always called under PTL spinlocks and the implementation cannot assume it, but that doesn't mean the implementation is allowed to block. That was one of the main motivating reasons to have both the invalidate_start/end and invalidate_range variations, start/end are allowed to block and range is not. invalidate_range really only exists for the iommu drivers to use it for SVA designs, there are a few other weird users, but iommu was the motivation to create it in the first place. So the comment should just clarify that it is not allowed to sleep and can't assume anything about its locking environment. Jason
diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h index 64a3e05..447d757 100644 --- a/include/linux/mmu_notifier.h +++ b/include/linux/mmu_notifier.h @@ -200,6 +200,9 @@ struct mmu_notifier_ops { * external TLB range needs to be flushed. For more in depth * discussion on this see Documentation/mm/mmu_notifier.rst * + * The invalidate_range() function is called under the ptl + * spin-lock and not allowed to sleep. + * * Note that this function might be called with just a sub-range * of what was passed to invalidate_range_start()/end(), if * called between those functions.
The .invalidate_range() callback is called by mmu_notifier_invalidate_range() which is often called while holding the ptl spin-lock. Therefore any implementations of this callback must not sleep. This was originally documented when the call back was added in commit 0f0a327fa12c ("mmu_notifier: add the callback for mmu_notifier_invalidate_range()") but appears to have been inadvertently removed by commit 5ff7091f5a2c ("mm, mmu_notifier: annotate mmu notifiers with blockable invalidate callbacks"). Restore the comment to make it clear that .invalidate_range() callbacks may not sleep. Signed-off-by: Alistair Popple <apopple@nvidia.com> --- include/linux/mmu_notifier.h | 3 +++ 1 file changed, 3 insertions(+) base-commit: 44c026a73be8038f03dbdeef028b642880cf1511