| Message ID | 20220526180950.13916-1-mike.kravetz@oracle.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | [v2] madvise.2: Clarify addr/length and update hugetlb support | expand |
On Thu, May 26, 2022 at 11:09:50AM -0700, Mike Kravetz wrote: > Clarify that madvise only works on full pages, and remove references > to 'bytes'. > > Update MADV_DONTNEED and MADV_REMOVE sections to remove notes that > HugeTLB mappings are not supported. Indicate the releases when they > were first supported as well as alignment restrictions. > > Signed-off-by: Mike Kravetz <mike.kravetz@oracle.com> Acked-by: Peter Xu <peterx@redhat.com>
On 26.05.22 20:09, Mike Kravetz wrote: > Clarify that madvise only works on full pages, and remove references > to 'bytes'. > > Update MADV_DONTNEED and MADV_REMOVE sections to remove notes that > HugeTLB mappings are not supported. Indicate the releases when they > were first supported as well as alignment restrictions. > > Signed-off-by: Mike Kravetz <mike.kravetz@oracle.com> > --- BTW, the git repository [1] was last updated 2021-09-02. One of my patches [2] was apparently picked up (reply via mail) but doesn't appear in that repository. Is the project currently dead/unmaintained or am I just staring at the wrong repository? [1] https://git.kernel.org/pub/scm/docs/man-pages/man-pages.git [2] https://lkml.kernel.org/r/20210823120645.8223-1-david@redhat.com > v1 -> v2 Added releases when Huge TLB support was added and moved > alignment requirements to corresponding section. (Peter) > > man2/madvise.2 | 31 +++++++++++++++++++++---------- > 1 file changed, 21 insertions(+), 10 deletions(-) > > diff --git a/man2/madvise.2 b/man2/madvise.2 > index f1f384c0c..50686e7e3 100644 > --- a/man2/madvise.2 > +++ b/man2/madvise.2 > @@ -61,9 +61,13 @@ system call is used to give advice or directions to the kernel > about the address range beginning at address > .I addr > and with size > +.IR length . > +.BR madvise () > +only operates on whole pages, therefore > +.I addr > +must be page-aligned. The value of > .I length > -bytes > -In most cases, > +is rounded up to a multiple of page size. In most cases, > the goal of such advice is to improve system or application performance. > .PP > Initially, the system call supported a set of "conventional" > @@ -143,7 +147,7 @@ The resident set size (RSS) of the calling process will be immediately > reduced however. > .IP > .B MADV_DONTNEED > -cannot be applied to locked pages, Huge TLB pages, or > +cannot be applied to locked pages, or > .BR VM_PFNMAP > pages. > (Pages marked with the kernel-internal > @@ -153,6 +157,11 @@ flag are special memory areas that are not managed > by the virtual memory subsystem. > Such pages are typically created by device drivers that > map the pages into user space.) > +.IP > +Support for Huge TLB pages was added in Linux v5.18. Addresses within a > +mapping backed by Huge TLB pages must be aligned to the underlying Huge TLB > +page size, and the range length is rounded up to a multiple of the underlying > +Huge TLB page size. > .\" > .\" ====================================================================== > .\" > @@ -170,24 +179,24 @@ Note that some of these operations change the semantics of memory accesses. > .\" commit f6b3ec238d12c8cc6cc71490c6e3127988460349 > Free up a given range of pages > and its associated backing store. > -This is equivalent to punching a hole in the corresponding byte > +This is equivalent to punching a hole in the corresponding > range of the backing store (see > .BR fallocate (2)). > Subsequent accesses in the specified address range will see > -bytes containing zero. > +pages containing zero. > .\" Databases want to use this feature to drop a section of their > .\" bufferpool (shared memory segments) - without writing back to > .\" disk/swap space. This feature is also useful for supporting > .\" hot-plug memory on UML. > .IP > The specified address range must be mapped shared and writable. > -This flag cannot be applied to locked pages, Huge TLB pages, or > +This flag cannot be applied to locked pages, or > .BR VM_PFNMAP > pages. > .IP > In the initial implementation, only > .BR tmpfs (5) > -was supported > +supported > .BR MADV_REMOVE ; > but since Linux 3.5, > .\" commit 3f31d07571eeea18a7d34db9af21d2285b807a17 > @@ -196,10 +205,12 @@ any filesystem which supports the > .BR FALLOC_FL_PUNCH_HOLE > mode also supports > .BR MADV_REMOVE . > -Hugetlbfs fails with the error > -.BR EINVAL > -and other filesystems fail with the error > +Filesystems which do not support > +.BR MADV_REMOVE > +fail with the error > .BR EOPNOTSUPP . > +.IP > +Support for Huge TLB filesystem was added in Linux v4.3. > .TP > .BR MADV_DONTFORK " (since Linux 2.6.16)" > .\" commit f822566165dd46ff5de9bf895cfa6c51f53bb0c4
Hi David! On 5/31/22 11:10, David Hildenbrand wrote: > BTW, the git repository [1] was last updated 2021-09-02. One of my > patches [2] was apparently picked up (reply via mail) but doesn't appear > in that repository. > > Is the project currently dead/unmaintained or am I just staring at the > wrong repository? > > [1] https://git.kernel.org/pub/scm/docs/man-pages/man-pages.git > [2] https://lkml.kernel.org/r/20210823120645.8223-1-david@redhat.com > Your patch has been applied here: <http://www.alejandro-colomar.es/src/alx/linux/man-pages/man-pages.git/commit/man2/madvise.2?id=9f307c06efc8c40a8aff30cf13ef3a8528be9222> The project is maintained, but the git repository is unmaintained. I keep reviewing and applying patches in my personal git repository, but I don't have access to kernel.org yet, so the official sites are unmaintained. And of course, that also means no releases since 5.13. I'm not very concerned about this, since in essence, a fork of the manual pages is still very well maintained on my server, and free for anyone interested in reading up-to-date pages. And since I do this on a hobby basis (my company doesn't pay me to do this at all), I don't care at all about not having released in almost a year now. That's more of a problem for distros and companies, which need releases. For individuals, I recommend getting a copy of my manual pages' git repo and `make install`. I applied recently for a kernel.org account, so maybe the official repo can be maintained again soon. Still, I don't expect releasing soon even if I get access to kernel.org and can update the git repo. I need to learn how to do a release properly before I'll release a new version, and that will take a lot more effort and time from my side than updating the repo. Thanks for your patience! Cheers, Alex
On 6/3/22 19:02, Alejandro Colomar wrote: > Hi David! > > On 5/31/22 11:10, David Hildenbrand wrote: > > BTW, the git repository [1] was last updated 2021-09-02. One of my > > patches [2] was apparently picked up (reply via mail) but doesn't appear > > in that repository. > > > > Is the project currently dead/unmaintained or am I just staring at the > > wrong repository? > > > > [1] https://git.kernel.org/pub/scm/docs/man-pages/man-pages.git > > [2] https://lkml.kernel.org/r/20210823120645.8223-1-david@redhat.com > > > > Your patch has been applied here: > <http://www.alejandro-colomar.es/src/alx/linux/man-pages/man-pages.git/commit/man2/madvise.2?id=9f307c06efc8c40a8aff30cf13ef3a8528be9222> > > > The project is maintained, but the git repository is unmaintained. > I keep reviewing and applying patches in my personal git repository, but > I don't have access to kernel.org yet, so the official sites are > unmaintained. And of course, that also means no releases since 5.13. > > I'm not very concerned about this, since in essence, a fork of the > manual pages is still very well maintained on my server, and free for > anyone interested in reading up-to-date pages. And since I do this on a > hobby basis (my company doesn't pay me to do this at all), I don't care > at all about not having released in almost a year now. That's more of a > problem for distros and companies, which need releases. For > individuals, I recommend getting a copy of my manual pages' git repo and > `make install`. > > I applied recently for a kernel.org account, so maybe the official repo > can be maintained again soon. Still, I don't expect releasing soon even > if I get access to kernel.org and can update the git repo. I need to > learn how to do a release properly before I'll release a new version, > and that will take a lot more effort and time from my side than updating > the repo. > > Thanks for your patience! BTW, I just released a signed git tag in my repo, to make it a bit more secure to know that you're getting my manual pages, and not some MITM fake stuff. <http://www.alejandro-colomar.es/src/alx/linux/man-pages/man-pages.git/tag/?h=man-pages-5.19-rc1> My PGP key is signed by mtk. Only trust it if it has his signature. I'm attaching it in this email, so that it can be found in the list (I'm having some issue with the keyserver). Cheers, Alex
Hi Mike, On 5/26/22 20:09, Mike Kravetz wrote: > Clarify that madvise only works on full pages, and remove references > to 'bytes'. > > Update MADV_DONTNEED and MADV_REMOVE sections to remove notes that > HugeTLB mappings are not supported. Indicate the releases when they > were first supported as well as alignment restrictions. > > Signed-off-by: Mike Kravetz <mike.kravetz@oracle.com> Could you please rebase against man-pages-5.19-rc1 in the kernel.org repo? There have been important changes, especially in the formatting. Thanks, Alex > --- > v1 -> v2 Added releases when Huge TLB support was added and moved > alignment requirements to corresponding section. (Peter) > > man2/madvise.2 | 31 +++++++++++++++++++++---------- > 1 file changed, 21 insertions(+), 10 deletions(-) > > diff --git a/man2/madvise.2 b/man2/madvise.2 > index f1f384c0c..50686e7e3 100644 > --- a/man2/madvise.2 > +++ b/man2/madvise.2 > @@ -61,9 +61,13 @@ system call is used to give advice or directions to the kernel > about the address range beginning at address > .I addr > and with size > +.IR length . > +.BR madvise () > +only operates on whole pages, therefore > +.I addr > +must be page-aligned. The value of Please break input lines at end-of-phrase/clause/sentence puntuation. See man-pages(7): Use semantic newlines In the source of a manual page, new sentences should be started on new lines, and long sentences should be split into lines at clause breaks (commas, semicolons, colons, and so on). This convention, sometimes known as "seman‐ tic newlines", makes it easier to see the effect of patches, which often operate at the level of individual sentences or sentence clauses. > .I length > -bytes > -In most cases, > +is rounded up to a multiple of page size. In most cases, Same here. > the goal of such advice is to improve system or application performance. > .PP > Initially, the system call supported a set of "conventional" > @@ -143,7 +147,7 @@ The resident set size (RSS) of the calling process will be immediately > reduced however. > .IP > .B MADV_DONTNEED > -cannot be applied to locked pages, Huge TLB pages, or > +cannot be applied to locked pages, or > .BR VM_PFNMAP > pages. > (Pages marked with the kernel-internal > @@ -153,6 +157,11 @@ flag are special memory areas that are not managed > by the virtual memory subsystem. > Such pages are typically created by device drivers that > map the pages into user space.) > +.IP > +Support for Huge TLB pages was added in Linux v5.18. Addresses within a Same here. And also break sentences at semantically-relevant points, instead of just before the 80 column. > +mapping backed by Huge TLB pages must be aligned to the underlying Huge TLB > +page size, and the range length is rounded up to a multiple of the underlying > +Huge TLB page size. > .\" > .\" ====================================================================== > .\" > @@ -170,24 +179,24 @@ Note that some of these operations change the semantics of memory accesses. > .\" commit f6b3ec238d12c8cc6cc71490c6e3127988460349 > Free up a given range of pages > and its associated backing store. > -This is equivalent to punching a hole in the corresponding byte > +This is equivalent to punching a hole in the corresponding > range of the backing store (see > .BR fallocate (2)). > Subsequent accesses in the specified address range will see > -bytes containing zero. > +pages containing zero. > .\" Databases want to use this feature to drop a section of their > .\" bufferpool (shared memory segments) - without writing back to > .\" disk/swap space. This feature is also useful for supporting > .\" hot-plug memory on UML. > .IP > The specified address range must be mapped shared and writable. > -This flag cannot be applied to locked pages, Huge TLB pages, or > +This flag cannot be applied to locked pages, or > .BR VM_PFNMAP > pages. > .IP > In the initial implementation, only > .BR tmpfs (5) > -was supported > +supported > .BR MADV_REMOVE ; > but since Linux 3.5, > .\" commit 3f31d07571eeea18a7d34db9af21d2285b807a17 > @@ -196,10 +205,12 @@ any filesystem which supports the > .BR FALLOC_FL_PUNCH_HOLE > mode also supports > .BR MADV_REMOVE . > -Hugetlbfs fails with the error > -.BR EINVAL > -and other filesystems fail with the error > +Filesystems which do not support > +.BR MADV_REMOVE s/BR/B/ See man(7): .B Bold .BR Bold alternating with Roman (especially useful for referring to other manual pages) > +fail with the error > .BR EOPNOTSUPP . > +.IP > +Support for Huge TLB filesystem was added in Linux v4.3. > .TP > .BR MADV_DONTFORK " (since Linux 2.6.16)" > .\" commit f822566165dd46ff5de9bf895cfa6c51f53bb0c4
On 03.06.22 19:42, Alejandro Colomar wrote: > On 6/3/22 19:02, Alejandro Colomar wrote: >> Hi David! >> >> On 5/31/22 11:10, David Hildenbrand wrote: >> > BTW, the git repository [1] was last updated 2021-09-02. One of my >> > patches [2] was apparently picked up (reply via mail) but doesn't appear >> > in that repository. >> > >> > Is the project currently dead/unmaintained or am I just staring at the >> > wrong repository? >> > >> > [1] https://git.kernel.org/pub/scm/docs/man-pages/man-pages.git >> > [2] https://lkml.kernel.org/r/20210823120645.8223-1-david@redhat.com >> > >> >> Your patch has been applied here: >> <http://www.alejandro-colomar.es/src/alx/linux/man-pages/man-pages.git/commit/man2/madvise.2?id=9f307c06efc8c40a8aff30cf13ef3a8528be9222> >> >> >> The project is maintained, but the git repository is unmaintained. >> I keep reviewing and applying patches in my personal git repository, but >> I don't have access to kernel.org yet, so the official sites are >> unmaintained. And of course, that also means no releases since 5.13. Ah, that explains things. The man-page info page [1] was/is still pointing at that git tree, that's why I wondered what happened. >> >> I'm not very concerned about this, since in essence, a fork of the >> manual pages is still very well maintained on my server, and free for >> anyone interested in reading up-to-date pages. And since I do this on a >> hobby basis (my company doesn't pay me to do this at all), I don't care >> at all about not having released in almost a year now. That's more of a >> problem for distros and companies, which need releases. For Heh, including me ;) >> individuals, I recommend getting a copy of my manual pages' git repo and >> `make install`. The point I was missing is that there is a fork(). I haven't seen notice of that, that's why I asked. It might have been helpful to reply with something like "Patch applied to my man-pages git fork"., including an URL. But you seem to have access to the official man-pages project now, which is good to know. >> >> I applied recently for a kernel.org account, so maybe the official repo >> can be maintained again soon. Still, I don't expect releasing soon even >> if I get access to kernel.org and can update the git repo. I need to >> learn how to do a release properly before I'll release a new version, >> and that will take a lot more effort and time from my side than updating >> the repo. >> >> Thanks for your patience! Thanks for your hard, voluntary work! > > BTW, I just released a signed git tag in my repo, to make it a bit more > secure to know that you're getting my manual pages, and not some MITM > fake stuff. > > <http://www.alejandro-colomar.es/src/alx/linux/man-pages/man-pages.git/tag/?h=man-pages-5.19-rc1> > > My PGP key is signed by mtk. Only trust it if it has his signature. > I'm attaching it in this email, so that it can be found in the list (I'm > having some issue with the keyserver). I can spot it on the official git tree [2] as well, essentially via alx/main. Nice. [1] https://www.kernel.org/doc/man-pages/ [2] https://git.kernel.org/pub/scm/docs/man-pages/man-pages.git/tag/?h=man-pages-5.19-rc1
Hi David, On 6/7/22 09:44, David Hildenbrand wrote: > Ah, that explains things. The man-page info page [1] was/is still > pointing at that git tree, that's why I wondered what happened. Yup. BTW, that reminds me I need to ask how to update that website. Now that everything goes back to normal, I don't need to; but still should know how to. >>> I'm not very concerned about this, since in essence, a fork of the >>> manual pages is still very well maintained on my server, and free for >>> anyone interested in reading up-to-date pages. And since I do this on a >>> hobby basis (my company doesn't pay me to do this at all), I don't care >>> at all about not having released in almost a year now. That's more of a >>> problem for distros and companies, which need releases. For > > Heh, including me ;) Which part of that text includes you? "interested in reading up-to-date pages"? Or "need releases"? Or both? :-) > >>> individuals, I recommend getting a copy of my manual pages' git repo and >>> `make install`. > > The point I was missing is that there is a fork(). I haven't seen notice > of that, that's why I asked. > > It might have been helpful to reply with something like "Patch applied > to my man-pages git fork"., including an URL. But you seem to have > access to the official man-pages project now, which is good to know. Yup. Sorry, I was a bit lazy :) > >>> >>> I applied recently for a kernel.org account, so maybe the official repo >>> can be maintained again soon. Still, I don't expect releasing soon even >>> if I get access to kernel.org and can update the git repo. I need to >>> learn how to do a release properly before I'll release a new version, >>> and that will take a lot more effort and time from my side than updating >>> the repo. >>> >>> Thanks for your patience! > > Thanks for your hard, voluntary work! =) > >> >> BTW, I just released a signed git tag in my repo, to make it a bit more >> secure to know that you're getting my manual pages, and not some MITM >> fake stuff. >> >> <http://www.alejandro-colomar.es/src/alx/linux/man-pages/man-pages.git/tag/?h=man-pages-5.19-rc1> >> >> My PGP key is signed by mtk. Only trust it if it has his signature. >> I'm attaching it in this email, so that it can be found in the list (I'm >> having some issue with the keyserver). > > I can spot it on the official git tree [2] as well, essentially via > alx/main. Nice. Yes, and soon in master. :) Still far from a release, however; I need help with that. Cheers, Alex
On 07.06.22 11:37, Alejandro Colomar wrote: > Hi David, > > On 6/7/22 09:44, David Hildenbrand wrote: >> Ah, that explains things. The man-page info page [1] was/is still >> pointing at that git tree, that's why I wondered what happened. > > Yup. BTW, that reminds me I need to ask how to update that website. > Now that everything goes back to normal, I don't need to; but still > should know how to. > >>>> I'm not very concerned about this, since in essence, a fork of the >>>> manual pages is still very well maintained on my server, and free for >>>> anyone interested in reading up-to-date pages. And since I do this on a >>>> hobby basis (my company doesn't pay me to do this at all), I don't care >>>> at all about not having released in almost a year now. That's more of a >>>> problem for distros and companies, which need releases. For >> >> Heh, including me ;) > > Which part of that text includes you? "interested in reading up-to-date > pages"? Or "need releases"? Or both? :-) A little bit of everything. Obviously, I'm interested in up-to-date man pages for Red Hat Linux users :) >> >>> >>> BTW, I just released a signed git tag in my repo, to make it a bit more >>> secure to know that you're getting my manual pages, and not some MITM >>> fake stuff. >>> >>> <http://www.alejandro-colomar.es/src/alx/linux/man-pages/man-pages.git/tag/?h=man-pages-5.19-rc1> >>> >>> My PGP key is signed by mtk. Only trust it if it has his signature. >>> I'm attaching it in this email, so that it can be found in the list (I'm >>> having some issue with the keyserver). >> >> I can spot it on the official git tree [2] as well, essentially via >> alx/main. Nice. > > Yes, and soon in master. :) > Still far from a release, however; I need help with that. What kind of help would you be needing?
Hi David, On 6/8/22 12:00, David Hildenbrand wrote: [...] >>> Heh, including me ;) >> >> Which part of that text includes you? "interested in reading up-to-date >> pages"? Or "need releases"? Or both? :-) > > A little bit of everything. Obviously, I'm interested in up-to-date man > pages for Red Hat Linux users :) :) [...] >> Still far from a release, however; I need help with that. > > What kind of help would you be needing? > Okay, so normally, Michael generates a changelog from the git log. That's done with a script and then some manual editing. I've never ever written a changelog before, so I'd like to do it as close to what was being done before as possible. Having Michael's script would be nice. See commit a7846609db2751694c98faa975657ccf990a747a for an example of what I need to produce. I'll also need a script for the timestamps, but I can write that one; it should be just a few sh(1) lines. I'll write that script this week. Thanks, Alex
diff --git a/man2/madvise.2 b/man2/madvise.2 index f1f384c0c..50686e7e3 100644 --- a/man2/madvise.2 +++ b/man2/madvise.2 @@ -61,9 +61,13 @@ system call is used to give advice or directions to the kernel about the address range beginning at address .I addr and with size +.IR length . +.BR madvise () +only operates on whole pages, therefore +.I addr +must be page-aligned. The value of .I length -bytes -In most cases, +is rounded up to a multiple of page size. In most cases, the goal of such advice is to improve system or application performance. .PP Initially, the system call supported a set of "conventional" @@ -143,7 +147,7 @@ The resident set size (RSS) of the calling process will be immediately reduced however. .IP .B MADV_DONTNEED -cannot be applied to locked pages, Huge TLB pages, or +cannot be applied to locked pages, or .BR VM_PFNMAP pages. (Pages marked with the kernel-internal @@ -153,6 +157,11 @@ flag are special memory areas that are not managed by the virtual memory subsystem. Such pages are typically created by device drivers that map the pages into user space.) +.IP +Support for Huge TLB pages was added in Linux v5.18. Addresses within a +mapping backed by Huge TLB pages must be aligned to the underlying Huge TLB +page size, and the range length is rounded up to a multiple of the underlying +Huge TLB page size. .\" .\" ====================================================================== .\" @@ -170,24 +179,24 @@ Note that some of these operations change the semantics of memory accesses. .\" commit f6b3ec238d12c8cc6cc71490c6e3127988460349 Free up a given range of pages and its associated backing store. -This is equivalent to punching a hole in the corresponding byte +This is equivalent to punching a hole in the corresponding range of the backing store (see .BR fallocate (2)). Subsequent accesses in the specified address range will see -bytes containing zero. +pages containing zero. .\" Databases want to use this feature to drop a section of their .\" bufferpool (shared memory segments) - without writing back to .\" disk/swap space. This feature is also useful for supporting .\" hot-plug memory on UML. .IP The specified address range must be mapped shared and writable. -This flag cannot be applied to locked pages, Huge TLB pages, or +This flag cannot be applied to locked pages, or .BR VM_PFNMAP pages. .IP In the initial implementation, only .BR tmpfs (5) -was supported +supported .BR MADV_REMOVE ; but since Linux 3.5, .\" commit 3f31d07571eeea18a7d34db9af21d2285b807a17 @@ -196,10 +205,12 @@ any filesystem which supports the .BR FALLOC_FL_PUNCH_HOLE mode also supports .BR MADV_REMOVE . -Hugetlbfs fails with the error -.BR EINVAL -and other filesystems fail with the error +Filesystems which do not support +.BR MADV_REMOVE +fail with the error .BR EOPNOTSUPP . +.IP +Support for Huge TLB filesystem was added in Linux v4.3. .TP .BR MADV_DONTFORK " (since Linux 2.6.16)" .\" commit f822566165dd46ff5de9bf895cfa6c51f53bb0c4
Clarify that madvise only works on full pages, and remove references to 'bytes'. Update MADV_DONTNEED and MADV_REMOVE sections to remove notes that HugeTLB mappings are not supported. Indicate the releases when they were first supported as well as alignment restrictions. Signed-off-by: Mike Kravetz <mike.kravetz@oracle.com> --- v1 -> v2 Added releases when Huge TLB support was added and moved alignment requirements to corresponding section. (Peter) man2/madvise.2 | 31 +++++++++++++++++++++---------- 1 file changed, 21 insertions(+), 10 deletions(-)