| Message ID | 20220421234837.3629927-1-kent.overstreet@gmail.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | Printbufs & shrinker OOM reporting | expand |
v1 of this patch series here:
https://lore.kernel.org/linux-mm/20220419203202.2670193-1-kent.overstreet@gmail.com/T/
Changes since v1:
- Converted some, not all, seq_buf code to printbufs (thanks Christoph for
pointing out seq_buf).
The seq_buf code I didn't convert is arch specific code where the memory
allocation context is unclear, and they're using seq_buf to do everything on
the stack. I'm considering adding a mode to printbufs where we point it at an
external buffer instead - it wouldn't be much code, but OTOH seq_buf isn't
much code either and it seems to be somewhat tied to tracing infrastructure.
Deferring a decision on what to do for now.
- pr_human_readable_u64() now uses string_get_size() (thanks Matthew for
pointing this one out)
- added new helpers printbuf_str() for getting a guaranteed-null-terminated
string, and printbuf_atomic_inc() and printbuf_atomic_dec() for marking
sections where allocations must be GFP_ATOMIC.
- Broke out shrinker_to_text(): this new helper could be used by new sysfs or
debugfs code, for displaying information about a single shrinker (as Roman is
working on)
- Added new tracking, per shrinker, for # of objects requested to be freed and
# actually freed. Shrinkers won't necessarily free all objects requested for
perfectly legitimate reasons, but if the two numbers are wildly off then
that's going to lead to memory reclaim issues - these are both also included
in shrinker_to_text().
Kent Overstreet (8):
lib/printbuf: New data structure for heap-allocated strings
Input/joystick/analog: Convert from seq_buf -> printbuf
mm/memcontrol.c: Convert to printbuf
clk: tegra: bpmp: Convert to printbuf
mm: Add a .to_text() method for shrinkers
mm: Count requests to free & nr freed per shrinker
mm: Move lib/show_mem.c to mm/
mm: Centralize & improve oom reporting in show_mem.c
drivers/clk/tegra/clk-bpmp.c | 21 ++-
drivers/input/joystick/analog.c | 37 +++--
include/linux/printbuf.h | 164 +++++++++++++++++++
include/linux/shrinker.h | 8 +
lib/Makefile | 4 +-
lib/printbuf.c | 274 ++++++++++++++++++++++++++++++++
mm/Makefile | 2 +-
mm/memcontrol.c | 68 ++++----
mm/oom_kill.c | 23 ---
{lib => mm}/show_mem.c | 14 ++
mm/slab.h | 6 +-
mm/slab_common.c | 53 +++++-
mm/vmscan.c | 88 ++++++++++
13 files changed, 666 insertions(+), 96 deletions(-)
create mode 100644 include/linux/printbuf.h
create mode 100644 lib/printbuf.c
rename {lib => mm}/show_mem.c (77%)
Hi Kent, On Fri, 22 Apr 2022 at 07:56, Kent Overstreet <kent.overstreet@gmail.com> wrote: > > Debugging OOMs has been one of my sources of frustration, so this patch series > is an attempt to do something about it. > > The first patch in the series is something I've been slowly evolving in bcachefs > for years: simple heap allocated strings meant for appending to and building up > structured log/error messages. They make it easy and straightforward to write > pretty-printers for everything, which in turn makes good logging and error > messages something that just happens naturally. > > We want it here because that means the reporting I'm adding to shrinkers can be > used by both OOM reporting, and for the sysfs (or is it debugfs now) interface > that Roman is adding. > I added the kexec list in cc. It seems like a nice enhancement to oom reporting. I suspect kdump tooling need changes to retrieve the kmsg log from vmcore, could you confirm it? For example makedumpfile, crash, and kexec-tools (its vmcore-dmesg tool). > This patch series also: > - adds OOM reporting on shrinkers, reporting on top 10 shrinkers (in sorted > order!) > - changes slab reporting to be always-on, also reporting top 10 slabs in sorted > order > - starts centralizing OOM reporting in mm/show_mem.c > > The last patch in the series is only a demonstration of how to implement the > shrinker .to_text() method, since bcachefs isn't upstream yet. > > Kent Overstreet (4): > lib/printbuf: New data structure for heap-allocated strings > mm: Add a .to_text() method for shrinkers > mm: Centralize & improve oom reporting in show_mem.c > bcachefs: shrinker.to_text() methods > > fs/bcachefs/btree_cache.c | 18 ++- > fs/bcachefs/btree_key_cache.c | 18 ++- > include/linux/printbuf.h | 140 ++++++++++++++++++ > include/linux/shrinker.h | 5 + > lib/Makefile | 4 +- > lib/printbuf.c | 271 ++++++++++++++++++++++++++++++++++ > mm/Makefile | 2 +- > mm/oom_kill.c | 23 --- > {lib => mm}/show_mem.c | 14 ++ > mm/slab.h | 6 +- > mm/slab_common.c | 53 ++++++- > mm/vmscan.c | 75 ++++++++++ > 12 files changed, 587 insertions(+), 42 deletions(-) > create mode 100644 include/linux/printbuf.h > create mode 100644 lib/printbuf.c > rename {lib => mm}/show_mem.c (78%) > > -- > 2.35.2 > Thanks Dave
Debugging OOMs has been one of my sources of frustration, so this patch series is an attempt to do something about it. The first patch in the series is something I've been slowly evolving in bcachefs for years: simple heap allocated strings meant for appending to and building up structured log/error messages. They make it easy and straightforward to write pretty-printers for everything, which in turn makes good logging and error messages something that just happens naturally. We want it here because that means the reporting I'm adding to shrinkers can be used by both OOM reporting, and for the sysfs (or is it debugfs now) interface that Roman is adding. This patch series also: - adds OOM reporting on shrinkers, reporting on top 10 shrinkers (in sorted order!) - changes slab reporting to be always-on, also reporting top 10 slabs in sorted order - starts centralizing OOM reporting in mm/show_mem.c The last patch in the series is only a demonstration of how to implement the shrinker .to_text() method, since bcachefs isn't upstream yet. Kent Overstreet (4): lib/printbuf: New data structure for heap-allocated strings mm: Add a .to_text() method for shrinkers mm: Centralize & improve oom reporting in show_mem.c bcachefs: shrinker.to_text() methods fs/bcachefs/btree_cache.c | 18 ++- fs/bcachefs/btree_key_cache.c | 18 ++- include/linux/printbuf.h | 140 ++++++++++++++++++ include/linux/shrinker.h | 5 + lib/Makefile | 4 +- lib/printbuf.c | 271 ++++++++++++++++++++++++++++++++++ mm/Makefile | 2 +- mm/oom_kill.c | 23 --- {lib => mm}/show_mem.c | 14 ++ mm/slab.h | 6 +- mm/slab_common.c | 53 ++++++- mm/vmscan.c | 75 ++++++++++ 12 files changed, 587 insertions(+), 42 deletions(-) create mode 100644 include/linux/printbuf.h create mode 100644 lib/printbuf.c rename {lib => mm}/show_mem.c (78%)