| Message ID | 20220326010003.3155137-1-yuzhao@google.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [GIT,PULL] Multi-gen LRU for 5.18-rc1 | expand |
On Fri, Mar 25, 2022 at 6:00 PM Yu Zhao <yuzhao@google.com> wrote: > > This is more of an option than a request for 5.18. I'm sending it to > you directly because, in my judgement, it's now as ready as it'll ever > be. So I do expect to merge this, but I don't think it has been in linux-next, has it? I would really like to see it get that linux-next workout with various automation bots going after it. I'd also like to see explicit acks from the usual suspects, or at least make sure we don't have any explicit NAK's coming in. Andrew & co? Linus
On Fri, Mar 25, 2022 at 7:07 PM Linus Torvalds <torvalds@linux-foundation.org> wrote: > > On Fri, Mar 25, 2022 at 6:00 PM Yu Zhao <yuzhao@google.com> wrote: > > > > This is more of an option than a request for 5.18. I'm sending it to > > you directly because, in my judgement, it's now as ready as it'll ever > > be. > > So I do expect to merge this, but I don't think it has been in > linux-next, has it? No. I could ask Stephen to see if he is willing to take this series. I was hoping to go through Andrew since his tree is what most MM developers test. I haven't heard from Andrew, so I assume he has no strong opinion and I don't want to put him in a different position.
On Fri, Mar 25, 2022 at 6:16 PM Yu Zhao <yuzhao@google.com> wrote: > > No. I could ask Stephen to see if he is willing to take this series. I > was hoping to go through Andrew since his tree is what most MM > developers test. I haven't heard from Andrew, so I assume he has no > strong opinion and I don't want to put him in a different position. So I'd definitely want to see this in linux-next for a while, which implies it's not ready for this merge window. I'm not convinced linux-next sees a lot of runtime testing (but it gets *some* of that too, certainly), but it does get a fair amount of at least build verification with a lot of odd configurations and a lot of different architectures. Linus
On Fri, 25 Mar 2022 19:16:15 -0600 Yu Zhao <yuzhao@google.com> wrote: > On Fri, Mar 25, 2022 at 7:07 PM Linus Torvalds > <torvalds@linux-foundation.org> wrote: > > > > On Fri, Mar 25, 2022 at 6:00 PM Yu Zhao <yuzhao@google.com> wrote: > > > > > > This is more of an option than a request for 5.18. I'm sending it to > > > you directly because, in my judgement, it's now as ready as it'll ever > > > be. > > > > So I do expect to merge this, but I don't think it has been in > > linux-next, has it? > > No. I could ask Stephen to see if he is willing to take this series. I > was hoping to go through Andrew since his tree is what most MM > developers test. I haven't heard from Andrew, so I assume he has no > strong opinion and I don't want to put him in a different position. hm, sorry, something in the headers here fooled my (elaborate) procmail rules :( Please yes, let's have a cycle in -next. I thought we decided to do that in discussion with Matthew Wilcox? Also, sorry, but I'm not seeing many commonly-seen names amongst the reviewers. I'd be more comfortable if people who have done most work on page reclaim up to this time have had time to review and comment.
On Sat, Mar 26, 2022 at 2:28 PM Linus Torvalds <torvalds@linux-foundation.org> wrote: > > On Fri, Mar 25, 2022 at 6:16 PM Yu Zhao <yuzhao@google.com> wrote: > > > > No. I could ask Stephen to see if he is willing to take this series. I > > was hoping to go through Andrew since his tree is what most MM > > developers test. I haven't heard from Andrew, so I assume he has no > > strong opinion and I don't want to put him in a different position. > > So I'd definitely want to see this in linux-next for a while, which > implies it's not ready for this merge window. Thanks for the clear instructions. Will follow them.
On Sat, Mar 26, 2022 at 2:49 PM Andrew Morton <akpm@linux-foundation.org> wrote: > > On Fri, 25 Mar 2022 19:16:15 -0600 Yu Zhao <yuzhao@google.com> wrote: > > > On Fri, Mar 25, 2022 at 7:07 PM Linus Torvalds > > <torvalds@linux-foundation.org> wrote: > > > > > > On Fri, Mar 25, 2022 at 6:00 PM Yu Zhao <yuzhao@google.com> wrote: > > > > > > > > This is more of an option than a request for 5.18. I'm sending it to > > > > you directly because, in my judgement, it's now as ready as it'll ever > > > > be. > > > > > > So I do expect to merge this, but I don't think it has been in > > > linux-next, has it? > > > > No. I could ask Stephen to see if he is willing to take this series. I > > was hoping to go through Andrew since his tree is what most MM > > developers test. I haven't heard from Andrew, so I assume he has no > > strong opinion and I don't want to put him in a different position. > > hm, sorry, something in the headers here fooled my (elaborate) procmail > rules :( > > Please yes, let's have a cycle in -next. I thought we decided to do > that in discussion with Matthew Wilcox? Will do. > Also, sorry, but I'm not seeing many commonly-seen names amongst the > reviewers. I'd be more comfortable if people who have done most work on > page reclaim up to this time have had time to review and comment. Me too :) Unfortunately I can't dictate others' priorities or methodologies. So I have reset my expectations from getting their reviewed-by's to not getting their NAKs.
On 26.03.22 21:49, Andrew Morton wrote: > On Fri, 25 Mar 2022 19:16:15 -0600 Yu Zhao <yuzhao@google.com> wrote: > >> On Fri, Mar 25, 2022 at 7:07 PM Linus Torvalds >> <torvalds@linux-foundation.org> wrote: >>> >>> On Fri, Mar 25, 2022 at 6:00 PM Yu Zhao <yuzhao@google.com> wrote: >>>> >>>> This is more of an option than a request for 5.18. I'm sending it to >>>> you directly because, in my judgement, it's now as ready as it'll ever >>>> be. >>> >>> So I do expect to merge this, but I don't think it has been in >>> linux-next, has it? >> >> No. I could ask Stephen to see if he is willing to take this series. I >> was hoping to go through Andrew since his tree is what most MM >> developers test. I haven't heard from Andrew, so I assume he has no >> strong opinion and I don't want to put him in a different position. > > hm, sorry, something in the headers here fooled my (elaborate) procmail > rules :( > > Please yes, let's have a cycle in -next. I thought we decided to do > that in discussion with Matthew Wilcox? > I'd appreciate if we could merge most MM-related stuff through the -MM tree; it exists for a reason IMHO. Andrew, you usually have a very good feeling when something's ready to be merged upstream (sufficient review from relevant folks, sufficient exposure via -mm and -next, ...).
On Mon, Mar 28, 2022 at 02:29:20PM +0200, David Hildenbrand wrote: > I'd appreciate if we could merge most MM-related stuff through the -MM > tree; it exists for a reason IMHO. Andrew, you usually have a very good > feeling when something's ready to be merged upstream (sufficient review > from relevant folks, sufficient exposure via -mm and -next, ...). The problem is that the MM tree is completely unusable when patches going in through other trees need to be based on it. The MM workflow clearly works well for Andrew, but it doesn't work well for us as a community. Fortunately folios is past that point now, but I fear that maple tree will get to that point if it doesn't go in through Andrew's tree this cycle, as it may have other users.
On 28.03.22 15:43, Matthew Wilcox wrote: > On Mon, Mar 28, 2022 at 02:29:20PM +0200, David Hildenbrand wrote: >> I'd appreciate if we could merge most MM-related stuff through the -MM >> tree; it exists for a reason IMHO. Andrew, you usually have a very good >> feeling when something's ready to be merged upstream (sufficient review >> from relevant folks, sufficient exposure via -mm and -next, ...). > > The problem is that the MM tree is completely unusable when patches going > in through other trees need to be based on it. The MM workflow clearly > works well for Andrew, but it doesn't work well for us as a community. > > Fortunately folios is past that point now, but I fear that maple tree > will get to that point if it doesn't go in through Andrew's tree this > cycle, as it may have other users. Yes, there are most certainly special cases, like folios and eventually like the maple tree rework. But I consider these exceptions to the general rule. And the exceptions should follow the same ACKing/review process as other -MM stuff, as Linus nicely identified here.
On Mon 28-03-22 14:43:52, Matthew Wilcox wrote: > On Mon, Mar 28, 2022 at 02:29:20PM +0200, David Hildenbrand wrote: > > I'd appreciate if we could merge most MM-related stuff through the -MM > > tree; it exists for a reason IMHO. Andrew, you usually have a very good > > feeling when something's ready to be merged upstream (sufficient review > > from relevant folks, sufficient exposure via -mm and -next, ...). > > The problem is that the MM tree is completely unusable when patches going > in through other trees need to be based on it. The MM workflow clearly > works well for Andrew, but it doesn't work well for us as a community. This is not the first time this has been brought up over years. We have discussed the existing workflow at the LSFMM two years ago without a larger consensus. I believe we want/need to resurrect those discussions again. IMHO the existing workflow is more and more hitting its limits so we should really think about a long term solution.
Hi Linus, This is more of an option than a request for 5.18. I'm sending it to you directly because, in my judgement, it's now as ready as it'll ever be. Thanks! The following changes since commit 46f538bf2404ee9c32648deafb886f49144bfd5e: Merge tag 'backlight-next-5.18' of git://git.kernel.org/pub/scm/linux/kernel/git/lee/backlight (2022-03-25 14:01:23 -0700) are available in the Git repository at: https://linux-mm.googlesource.com/mglru/ tags/mglru-for-5.18-rc1 for you to fetch changes up to bcf5f2a801b85f09ad691d07460fdf3fbef8404d: mm: multi-gen LRU: design doc (2022-03-25 15:30:50 -0600) ---------------------------------------------------------------- Multi-gen LRU for 5.18-rc1 This series was retested on top of 5.18-ed4643521e6a. All previous comments have been addressed. Changes since v8 [1] * Removed two user-hostile config options (suggested by Linus Torvalds). Changes since v9 [2] * Resolved the conflicts with the latest folio changes. * Switched to spin_trylock() to move onto the next page table rather than spin on the current one when trying to clear the accessed bit on many page tables. * Added introductory paragraphs to the admin guide and the design doc (suggested by Mike Rapoport). * Expanded comments in get_nr_evictable() (suggested by Barry Song and Aneesh Kumar). * Expanded comments in inc_max_seq() (suggested by Barry Song). [1] https://lkml.kernel.org/r/20220308234723.3834941-1-yuzhao@google.com [2] https://lkml.kernel.org/r/20220309021230.721028-1-yuzhao@google.com Signed-off-by: Yu Zhao <yuzhao@google.com> ---------------------------------------------------------------- Yu Zhao (14): mm: x86, arm64: add arch_has_hw_pte_young() mm: x86: add CONFIG_ARCH_HAS_NONLEAF_PMD_YOUNG mm/vmscan.c: refactor shrink_node() Revert "include/linux/mm_inline.h: fold __update_lru_size() into its sole caller" mm: multi-gen LRU: groundwork mm: multi-gen LRU: minimal implementation mm: multi-gen LRU: exploit locality in rmap mm: multi-gen LRU: support page table walks mm: multi-gen LRU: optimize multiple memcgs mm: multi-gen LRU: kill switch mm: multi-gen LRU: thrashing prevention mm: multi-gen LRU: debugfs interface mm: multi-gen LRU: admin guide mm: multi-gen LRU: design doc Documentation/admin-guide/mm/index.rst | 1 + Documentation/admin-guide/mm/multigen_lru.rst | 152 ++ Documentation/vm/index.rst | 1 + Documentation/vm/multigen_lru.rst | 160 ++ arch/Kconfig | 9 + arch/arm64/include/asm/pgtable.h | 14 +- arch/x86/Kconfig | 1 + arch/x86/include/asm/pgtable.h | 9 +- arch/x86/mm/pgtable.c | 5 +- fs/exec.c | 2 + fs/fuse/dev.c | 3 +- include/linux/cgroup.h | 15 +- include/linux/memcontrol.h | 36 + include/linux/mm.h | 7 + include/linux/mm_inline.h | 217 +- include/linux/mm_types.h | 78 + include/linux/mmzone.h | 211 ++ include/linux/nodemask.h | 1 + include/linux/page-flags-layout.h | 11 +- include/linux/page-flags.h | 4 +- include/linux/pgtable.h | 17 +- include/linux/sched.h | 4 + include/linux/swap.h | 4 + kernel/bounds.c | 7 + kernel/cgroup/cgroup-internal.h | 1 - kernel/exit.c | 1 + kernel/fork.c | 9 + kernel/sched/core.c | 1 + mm/Kconfig | 26 + mm/huge_memory.c | 3 +- mm/internal.h | 1 + mm/memcontrol.c | 27 + mm/memory.c | 39 +- mm/mm_init.c | 6 +- mm/mmzone.c | 2 + mm/rmap.c | 7 + mm/swap.c | 55 +- mm/vmscan.c | 3102 +++++++++++++++++++++++-- mm/workingset.c | 119 +- 39 files changed, 4097 insertions(+), 271 deletions(-) create mode 100644 Documentation/admin-guide/mm/multigen_lru.rst create mode 100644 Documentation/vm/multigen_lru.rst