| Message ID | 20200219014433.88424-1-minchan@kernel.org (mailing list archive) |
|---|---|
| Headers | show |
| Series | introduce memory hinting API for external process | expand |
On Tue, 18 Feb 2020 17:44:26 -0800 Minchan Kim <minchan@kernel.org> wrote: > Now, we have MADV_PAGEOUT and MADV_COLD as madvise hinting API. With that, > application could give hints to kernel what memory range are preferred to be > reclaimed. However, in some platform(e.g., Android), the information > required to make the hinting decision is not known to the app. > Instead, it is known to a centralized userspace daemon(e.g., ActivityManagerService), > and that daemon must be able to initiate reclaim on its own without any app > involvement. > This patchset doesn't seem to be getting a lot of interest from other potential users? It seems very specialized. Are there or will there ever be any users of this apart from one Android daemon? Also, it doesn't terribly hard for ActivityManagerService to tell another process "now run madvise with these arguments". Please explain why this is not practical in ActivityManagerService and also within other potential users of this syscall.
On Wed, Feb 19, 2020 at 12:01 PM Andrew Morton <akpm@linux-foundation.org> wrote: > > On Tue, 18 Feb 2020 17:44:26 -0800 Minchan Kim <minchan@kernel.org> wrote: > > > Now, we have MADV_PAGEOUT and MADV_COLD as madvise hinting API. With that, > > application could give hints to kernel what memory range are preferred to be > > reclaimed. However, in some platform(e.g., Android), the information > > required to make the hinting decision is not known to the app. > > Instead, it is known to a centralized userspace daemon(e.g., ActivityManagerService), > > and that daemon must be able to initiate reclaim on its own without any app > > involvement. > > > > This patchset doesn't seem to be getting a lot of interest from other > potential users? It seems very specialized. Are there or will there > ever be any users of this apart from one Android daemon? Don't know if this can be considered another user since it's still in the Android realm of things. I'm interested in extending process_madvise() to support MADV_DONTNEED to expedite memory ripping of a process killed by Android Low Memory Killer. But for that I need process_madvise() to be accepted first. IIRC Crome team was interested in these madv hints as well at some point... > > Also, it doesn't terribly hard for ActivityManagerService to tell > another process "now run madvise with these arguments". Please explain > why this is not practical in ActivityManagerService and also within > other potential users of this syscall. > >
Hi Andrew, On Wed, Feb 19, 2020 at 12:01:23PM -0800, Andrew Morton wrote: > On Tue, 18 Feb 2020 17:44:26 -0800 Minchan Kim <minchan@kernel.org> wrote: > > > Now, we have MADV_PAGEOUT and MADV_COLD as madvise hinting API. With that, > > application could give hints to kernel what memory range are preferred to be > > reclaimed. However, in some platform(e.g., Android), the information > > required to make the hinting decision is not known to the app. > > Instead, it is known to a centralized userspace daemon(e.g., ActivityManagerService), > > and that daemon must be able to initiate reclaim on its own without any app > > involvement. > > > > This patchset doesn't seem to be getting a lot of interest from other > potential users? It seems very specialized. Are there or will there > ever be any users of this apart from one Android daemon? Quote from http://lkml.kernel.org/r/20190531064313.193437-1-minchan@kernel.org " Brian Geffon in ChromeOS team had an experiment with process_madvise(2) Quote form him: "What I found is that by using process_madvise after a tab has been back grounded for more than 45 seconds reduced the average tab switch times by 25%! This is a huge result and very obvious validation that process_madvise hints works well for the ChromeOS use case." " > > Also, it doesn't terribly hard for ActivityManagerService to tell > another process "now run madvise with these arguments". Please explain > why this is not practical in ActivityManagerService and also within > other potential users of this syscall. I think that's the almost a same question why ptrace doesn't work so I summarizes the part in [2/7]: * makes target task runnable creates memory layout change window so hiniting a wrong vma * target task(e.g., background task) could live in little core with cpuset/group limited environment so we couldn't react quick enough, which causes more killing. Thanks.
To expand on how ChromeOS benefits from this, we've advanced far beyond the experimentation phase we've deployed an older version of this idea that was procfs based on several ChromeOS kernels. These are now rolled out to ChromeOS stable channel where we've been testing and the results have been amazing. To elaborate on the setup, Chrome is a multi process architecture where each tab is a separate process and sometimes a single tab can even represent multiple processes. The primary Chrome process has a lot of visibility into the amount of time a user has been spending interacting with a tab (process) and using this knowledge these hints provided to the kernel allow it to make much better swap decisions and amortize the cost of swap over different memory pressure levels meaning that we were better able to reclaim memory which allow us to avoid having to discard tabs or even worse oom. I'd be happy to expand even more if anyone is interested. Brian On Wed, Feb 19, 2020 at 2:32 PM Minchan Kim <minchan@kernel.org> wrote: > > Hi Andrew, > > On Wed, Feb 19, 2020 at 12:01:23PM -0800, Andrew Morton wrote: > > On Tue, 18 Feb 2020 17:44:26 -0800 Minchan Kim <minchan@kernel.org> wrote: > > > > > Now, we have MADV_PAGEOUT and MADV_COLD as madvise hinting API. With that, > > > application could give hints to kernel what memory range are preferred to be > > > reclaimed. However, in some platform(e.g., Android), the information > > > required to make the hinting decision is not known to the app. > > > Instead, it is known to a centralized userspace daemon(e.g., ActivityManagerService), > > > and that daemon must be able to initiate reclaim on its own without any app > > > involvement. > > > > > > > This patchset doesn't seem to be getting a lot of interest from other > > potential users? It seems very specialized. Are there or will there > > ever be any users of this apart from one Android daemon? > > > Quote from http://lkml.kernel.org/r/20190531064313.193437-1-minchan@kernel.org > > " > Brian Geffon in ChromeOS team had an experiment with process_madvise(2) > Quote form him: > "What I found is that by using process_madvise after a tab has been back > grounded for more than 45 seconds reduced the average tab switch times by > 25%! This is a huge result and very obvious validation that process_madvise > hints works well for the ChromeOS use case." > " > > > > > Also, it doesn't terribly hard for ActivityManagerService to tell > > another process "now run madvise with these arguments". Please explain > > why this is not practical in ActivityManagerService and also within > > other potential users of this syscall. > > I think that's the almost a same question why ptrace doesn't work so > I summarizes the part in [2/7]: > > * makes target task runnable creates memory layout change window so > hiniting a wrong vma > > * target task(e.g., background task) could live in little core with > cpuset/group limited environment so we couldn't react quick enough, > which causes more killing. > > > Thanks.
Now, we have MADV_PAGEOUT and MADV_COLD as madvise hinting API. With that, application could give hints to kernel what memory range are preferred to be reclaimed. However, in some platform(e.g., Android), the information required to make the hinting decision is not known to the app. Instead, it is known to a centralized userspace daemon(e.g., ActivityManagerService), and that daemon must be able to initiate reclaim on its own without any app involvement. To solve the concern, this patch introduces new syscall - process_madvise(2). Bascially, it's same with madvise(2) syscall but it has some differences. 1. It needs pidfd of target process to provide the hint 2. It supports only MADV_{COLD|PAGEOUT|MERGEABLE|UNMEREABLE} at this moment. Other hints in madvise will be opened when there are explicit requests from community to prevent unexpected bugs we couldn't support. 3. Only privileged processes can do something for other process's address space. For more detail of the new API, please see "mm: introduce external memory hinting API" description in this patchset. * from v5 - https://lore.kernel.org/linux-mm/20200214170520.160271-1-minchan@kernel.org/ * use null task and requestor's mm for io_madvise - Jann and Jens * use right commit description for moving pidfd_get_pid - Christoph * from v4 - https://lore.kernel.org/linux-mm/20200212233946.246210-1-minchan@kernel.org/ * pass mm down to functions, not accessing task->mm - Jann * clean up - Alexander * add Reviewed-by - Alexander, SeongJae * patch reordering * from v3 - https://lore.kernel.org/linux-mm/20200128001641.5086-1-minchan@kernel.org/ * verify task->mm aftere access_mm - Oleg * split some patches for easy review - Alexander * clean up fatal signal checking - Suren * from v2 - https://lore.kernel.org/linux-mm/20200116235953.163318-1-minchan@kernel.org/ * check signal callee and caller to bail out - Kirill Tkhai * put more clarification for justification of new API * from v1 - https://lore.kernel.org/linux-mm/20200110213433.94739-1-minchan@kernel.org/ * fix syscall number - SeongJae * use get_pid_task - Kirill Tkhai * extend API to support pid as well as pidfd - Kirill Tkhai Minchan Kim (5): mm: pass task and mm to do_madvise mm: introduce external memory hinting API mm: check fatal signal pending of target process pid: move pidfd_get_pid function to pid.c mm: support both pid and pidfd for process_madvise Oleksandr Natalenko (2): mm/madvise: employ mmget_still_valid for write lock mm/madvise: allow KSM hints for remote API arch/alpha/kernel/syscalls/syscall.tbl | 1 + arch/arm/tools/syscall.tbl | 1 + arch/arm64/include/asm/unistd.h | 2 +- arch/arm64/include/asm/unistd32.h | 2 + arch/ia64/kernel/syscalls/syscall.tbl | 1 + arch/m68k/kernel/syscalls/syscall.tbl | 1 + arch/microblaze/kernel/syscalls/syscall.tbl | 1 + arch/mips/kernel/syscalls/syscall_n32.tbl | 1 + arch/mips/kernel/syscalls/syscall_n64.tbl | 1 + arch/parisc/kernel/syscalls/syscall.tbl | 1 + arch/powerpc/kernel/syscalls/syscall.tbl | 1 + arch/s390/kernel/syscalls/syscall.tbl | 1 + arch/sh/kernel/syscalls/syscall.tbl | 1 + arch/sparc/kernel/syscalls/syscall.tbl | 1 + arch/x86/entry/syscalls/syscall_32.tbl | 1 + arch/x86/entry/syscalls/syscall_64.tbl | 1 + arch/xtensa/kernel/syscalls/syscall.tbl | 1 + fs/io_uring.c | 2 +- include/linux/mm.h | 3 +- include/linux/pid.h | 1 + include/linux/syscalls.h | 3 + include/uapi/asm-generic/unistd.h | 4 +- kernel/exit.c | 17 --- kernel/pid.c | 17 +++ kernel/sys_ni.c | 1 + mm/madvise.c | 144 ++++++++++++++++---- 26 files changed, 167 insertions(+), 44 deletions(-)