From patchwork Fri Apr 21 10:14:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13219767 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4BF44C7618E for ; Fri, 21 Apr 2023 10:14:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D95B66B0072; Fri, 21 Apr 2023 06:14:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D45946B0074; Fri, 21 Apr 2023 06:14:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BE68B6B0075; Fri, 21 Apr 2023 06:14:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id AE2B16B0072 for ; Fri, 21 Apr 2023 06:14:39 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 765AD40742 for ; Fri, 21 Apr 2023 10:14:39 +0000 (UTC) X-FDA: 80704989078.19.9C10C1C Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by imf20.hostedemail.com (Postfix) with ESMTP id 84E381C0013 for ; Fri, 21 Apr 2023 10:14:37 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=E5cBcBJ0; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=CWbSShbg; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf20.hostedemail.com: domain of osalvador@suse.de designates 195.135.220.29 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1682072077; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=NhQ8pvDEDYdK4MH7I174Hpey5DF3W+He8Wm+uYrTkgw=; b=EsYU1tJN5vMe98X0Snav3Rvsc7wqgOgfnPkR0p9jPPxVXQZ0Do2V7ZGnjyiXhLmHddYIUh 4jNaawwOMREITNozriwypqmjUStFh90JW0BHAYgZN49gJ7KAPngprcKjD3YuZvhNKFPeB/ LwxZdkUyCiG3vgdiNnbnBck2s01J6nw= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=E5cBcBJ0; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=CWbSShbg; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf20.hostedemail.com: domain of osalvador@suse.de designates 195.135.220.29 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1682072077; a=rsa-sha256; cv=none; b=Z2F8fZkDttxpndSSd8jOe8A+Pq2a073zLmiFm9yYJfws5ZgbWrMrr4L5xa/Xy4bQzam+h/ kuTrGMOHKvbvXbd0xOk5I/6kP/pQEUrR+LfaNMlfjQ3N61PJGqA/i0elJtJrbs5JlYwdq/ IPMifpBWSmg9UXw324XwL5d2G+xHpgc= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 53B631FDDC; Fri, 21 Apr 2023 10:14:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1682072076; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NhQ8pvDEDYdK4MH7I174Hpey5DF3W+He8Wm+uYrTkgw=; b=E5cBcBJ0NS1SCMaDeyEpm1IRLZaQHjgK1FAE4SYuJ40GAfoaJ6W70GfGcSsWIk3Xe1hfRe 3PR8N9XM2u1eZB8mD+cvlzlVt4WpBWIskHlztPK4Wi5JFJku1IHWSGEtjMYGHobusXWq96 vn0t7I0G5Y3PGrtKt1u1P76dOI/WrAM= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1682072076; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NhQ8pvDEDYdK4MH7I174Hpey5DF3W+He8Wm+uYrTkgw=; b=CWbSShbgBZJ0E617Q6QJQE0QaIJAUC+0i7cKB+O8UWP4cWLAVoqZkpEIGA61IUeWgdD4+D lQ5hqFH6RMS1NrDQ== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 88D7A1390E; Fri, 21 Apr 2023 10:14:35 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 6KUrHgtiQmRNaAAAMHmgww (envelope-from ); Fri, 21 Apr 2023 10:14:35 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Michal Hocko , Vlastimil Babka , Eric Dumazet , Waiman Long , Suren Baghdasaryan , Marco Elver , Andrey Konovalov , Alexander Potapenko , Oscar Salvador Subject: [PATCH v4 1/3] lib/stackdepot: Add a refcount field in stack_record Date: Fri, 21 Apr 2023 12:14:13 +0200 Message-Id: <20230421101415.5734-2-osalvador@suse.de> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230421101415.5734-1-osalvador@suse.de> References: <20230421101415.5734-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 84E381C0013 X-Stat-Signature: 633y7m6p9dtss91uqap8fc9tp4usji1r X-HE-Tag: 1682072077-185258 X-HE-Meta: U2FsdGVkX185CKMJqSWDnVNChbEW6C2lmxC5+HpyQQYBTawksA6HR1XK31+UFHYrB/KfZoTgQRthCMNihcMP8mEalzl/lgxhEYUsVpJ/D0flPqMELUFy0v2x1ejNxR6Ogy593TQy8OQZW0azrL6n/vmQoxG6BCYlZdv0zWeokaSbVDdoWYGiKdP4WReXV6jWdV+YZx1xL6orVHCZrWWO3IUGzkoDTpvSM6BCKcjeioDsuioJupP2h0F5PkGymqJ5xA7cumbHdr+freNYjfiKXMF770JX8Q/HNOj1jqL8Dg9yyltqKUsUcfavuPz+tbG1prXanm/EqQ3wxUoWM4dWSx0z1k9KdOyIhiRm9nCQDrndEto1Dj0yUTxrp3bm2pEWHMykrUEWyoGSdIHGxlJTB12x9B18BbkSud2NUyx1G0x7p4L09eIrmZE/Mpi7N5pf9Oa0K9crVtnasUjKDmX65ttdvDQxZ6kxKpsZf+lFkotRaUgbRUF+q7iylclatdRRv0oJV4A2lmU73uEOJXVjqaNjbr6yA8K3ajkymlod2g4JzadDkOg/pWJ9hIoCC4MA2u993CfrxzaTPPqRs7beu6hDVj7oF5qnAAkJcR7EuqZ4YRVMARs+IMqECfJdUmCml9oRahn3quoSKJC6DNxaCUnCBMqVnSj/R9PAHA8FAsUv6FnkcLg+r+jhVI2sm8JgqgFchl+ZcEA4nG9hXU1f2Uf5htqGSM/IoU9Xbu1LdJctikuv5nDA39e3/klwDGdmlaRnVkTaXNuiqbqOvFPY47YlF1sIHf8d7s/ooLN2UeXsBQ/rmIUnI9vBndf5mG+AVEhYmv/AGUnwcqveFy5EbFmBBL1a4TZse9SZo4dOVnKjzpEdKplffen9P7cZUNAAJ00x5/BwgaKpYJD4RZr3B2GAD1BQgukDCxoiYsSpwvOoCc8lH2042bWyGFrdedwBqhnKyfRQwtSeDWn/Iu7 09Ru3/h+ 0PffRQYqCc6Gh4NWpwqO2sfbYHBv/9P/Pby2oMQRtbVxyvtjvBqjc/TswygcuqwO1uPz8tX/AiAE7uhkXbZVoOc+RL6LG+ejojT0oTrT/7Qc1LyIn0zO04M9LKNu/m1iWkhRlrCcyHPOfs7uC9Wy30F/QEh/qeM+8TFuA6P6RVy7zG/dJsbDroR0xcqaABYVs6sVHMXTwSwEi7iaOtidSzLFWjFaAvpSmxsR9saA2ZiEV4+bl81MFZz17zK7lmBW0fkPf X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: We want to filter out page_owner output and print only those stacks that have been repeated beyond a certain threshold. This gives us the chance to get rid of a lot of noise. In order to do that, we need to keep track of how many repeated stacks (for allocation) do we have, so we add a new refcount_t field in the stack_record struct. Note that this might increase the size of the struct for some architectures. E.g: x86_64 is not affected due to alignment, but x86 32bits might. The alternative would be to have some kind of struct like this: struct track_stacks { struct stack_record *stack; struct track_stacks *next; refcount_t stack_count; But ithat would imply to perform more allocations and glue everything together, which would make the code more complex, so I think that going with a new field in the struct stack_record is good enough. Note that on __set_page_owner_handle(), page_owner->handle is set, and on __reset_page_owner(), page_owner->free_handle is set. We are interested in page_owner->handle, so when __set_page_owner() gets called, we derive the stack_record struct from page_owner->handle, and we increment its refcount_t field; and when __reset_page_owner() gets called, we derive its stack_record from page_owner->handle() and we decrement its refcount_t field. Signed-off-by: Oscar Salvador --- include/linux/stackdepot.h | 8 ++++- lib/stackdepot.c | 72 ++++++++++++++++++++++++++++++++------ mm/kasan/common.c | 3 +- mm/page_owner.c | 13 ++++--- 4 files changed, 79 insertions(+), 17 deletions(-) diff --git a/include/linux/stackdepot.h b/include/linux/stackdepot.h index e58306783d8e..b94d33312839 100644 --- a/include/linux/stackdepot.h +++ b/include/linux/stackdepot.h @@ -93,7 +93,9 @@ static inline int stack_depot_early_init(void) { return 0; } */ depot_stack_handle_t __stack_depot_save(unsigned long *entries, unsigned int nr_entries, - gfp_t gfp_flags, bool can_alloc); + gfp_t gfp_flags, bool can_alloc, + bool counter); +void stack_depot_dec_count(depot_stack_handle_t handle); /** * stack_depot_save - Save a stack trace to stack depot @@ -109,6 +111,10 @@ depot_stack_handle_t __stack_depot_save(unsigned long *entries, */ depot_stack_handle_t stack_depot_save(unsigned long *entries, unsigned int nr_entries, gfp_t gfp_flags); +depot_stack_handle_t stack_depot_save_action(unsigned long *entries, + unsigned int nr_entries, + gfp_t gfp_flags, + bool counter); /** * stack_depot_fetch - Fetch a stack trace from stack depot diff --git a/lib/stackdepot.c b/lib/stackdepot.c index 036da8e295d1..e99f4ef218ef 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -59,6 +59,7 @@ struct stack_record { u32 hash; /* Hash in the hash table */ u32 size; /* Number of stored frames */ union handle_parts handle; + refcount_t count; /* Number of the same repeated stacks */ unsigned long entries[]; /* Variable-sized array of frames */ }; @@ -304,6 +305,7 @@ depot_alloc_stack(unsigned long *entries, int size, u32 hash, void **prealloc) stack->handle.offset = pool_offset >> DEPOT_STACK_ALIGN; stack->handle.valid = 1; stack->handle.extra = 0; + refcount_set(&stack->count, 1); memcpy(stack->entries, entries, flex_array_size(stack, entries, size)); pool_offset += required_size; @@ -349,9 +351,15 @@ static inline struct stack_record *find_stack(struct stack_record *bucket, return NULL; } +static void stack_depot_inc_count(struct stack_record *stack) +{ + refcount_inc(&stack->count); +} + depot_stack_handle_t __stack_depot_save(unsigned long *entries, unsigned int nr_entries, - gfp_t alloc_flags, bool can_alloc) + gfp_t alloc_flags, bool can_alloc, + bool counter) { struct stack_record *found = NULL, **bucket; union handle_parts retval = { .handle = 0 }; @@ -436,8 +444,11 @@ depot_stack_handle_t __stack_depot_save(unsigned long *entries, /* Stack depot didn't use this memory, free it. */ free_pages((unsigned long)prealloc, DEPOT_POOL_ORDER); } - if (found) + if (found) { retval.handle = found->handle.handle; + if (counter) + stack_depot_inc_count(found); + } fast_exit: return retval.handle; } @@ -447,12 +458,20 @@ depot_stack_handle_t stack_depot_save(unsigned long *entries, unsigned int nr_entries, gfp_t alloc_flags) { - return __stack_depot_save(entries, nr_entries, alloc_flags, true); + return __stack_depot_save(entries, nr_entries, alloc_flags, true, false); } EXPORT_SYMBOL_GPL(stack_depot_save); -unsigned int stack_depot_fetch(depot_stack_handle_t handle, - unsigned long **entries) +depot_stack_handle_t stack_depot_save_action(unsigned long *entries, + unsigned int nr_entries, + gfp_t alloc_flags, + bool counter) +{ + return __stack_depot_save(entries, nr_entries, alloc_flags, true, counter); +} +EXPORT_SYMBOL_GPL(stack_depot_save_action); + +static struct stack_record *stack_depot_getstack(depot_stack_handle_t handle) { union handle_parts parts = { .handle = handle }; /* @@ -464,25 +483,56 @@ unsigned int stack_depot_fetch(depot_stack_handle_t handle, size_t offset = parts.offset << DEPOT_STACK_ALIGN; struct stack_record *stack; - *entries = NULL; - if (!handle) - return 0; + if(!handle) + return NULL; if (parts.pool_index > pool_index_cached) { WARN(1, "pool index %d out of bounds (%d) for stack id %08x\n", - parts.pool_index, pool_index_cached, handle); - return 0; + parts.pool_index, pool_index_cached, handle); + return NULL; } pool = stack_pools[parts.pool_index]; if (!pool) - return 0; + return NULL; + stack = pool + offset; + return stack; +} + +unsigned int stack_depot_fetch(depot_stack_handle_t handle, + unsigned long **entries) +{ + struct stack_record *stack; + + *entries = NULL; + if (!handle) + return 0; + + stack = stack_depot_getstack(handle); + if (!stack) + return 0; *entries = stack->entries; return stack->size; } EXPORT_SYMBOL_GPL(stack_depot_fetch); +void stack_depot_dec_count(depot_stack_handle_t handle) +{ + struct stack_record *stack = NULL; + + stack = stack_depot_getstack(handle); + if (stack) { + /* + * page_owner creates some stacks via create_dummy_stack(). + * We are not interested in those, so make sure we only decrement + * "valid" stacks. + */ + if (refcount_read(&stack->count) > 1) + refcount_dec(&stack->count); + } +} + void stack_depot_print(depot_stack_handle_t stack) { unsigned long *entries; diff --git a/mm/kasan/common.c b/mm/kasan/common.c index b376a5d055e5..ea0061ea8ae9 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -43,7 +43,8 @@ depot_stack_handle_t kasan_save_stack(gfp_t flags, bool can_alloc) unsigned int nr_entries; nr_entries = stack_trace_save(entries, ARRAY_SIZE(entries), 0); - return __stack_depot_save(entries, nr_entries, flags, can_alloc); + return __stack_depot_save(entries, nr_entries, flags, can_alloc, + false); } void kasan_set_track(struct kasan_track *track, gfp_t flags) diff --git a/mm/page_owner.c b/mm/page_owner.c index 220cdeddc295..b6637524e442 100644 --- a/mm/page_owner.c +++ b/mm/page_owner.c @@ -107,7 +107,7 @@ static inline struct page_owner *get_page_owner(struct page_ext *page_ext) return (void *)page_ext + page_owner_ops.offset; } -static noinline depot_stack_handle_t save_stack(gfp_t flags) +static noinline depot_stack_handle_t save_stack(gfp_t flags, bool counter) { unsigned long entries[PAGE_OWNER_STACK_DEPTH]; depot_stack_handle_t handle; @@ -126,7 +126,7 @@ static noinline depot_stack_handle_t save_stack(gfp_t flags) current->in_page_owner = 1; nr_entries = stack_trace_save(entries, ARRAY_SIZE(entries), 2); - handle = stack_depot_save(entries, nr_entries, flags); + handle = stack_depot_save_action(entries, nr_entries, flags, counter); if (!handle) handle = failure_handle; @@ -139,6 +139,7 @@ void __reset_page_owner(struct page *page, unsigned short order) int i; struct page_ext *page_ext; depot_stack_handle_t handle; + depot_stack_handle_t alloc_handle; struct page_owner *page_owner; u64 free_ts_nsec = local_clock(); @@ -146,7 +147,10 @@ void __reset_page_owner(struct page *page, unsigned short order) if (unlikely(!page_ext)) return; - handle = save_stack(GFP_NOWAIT | __GFP_NOWARN); + page_owner = get_page_owner(page_ext); + alloc_handle = page_owner->handle; + + handle = save_stack(GFP_NOWAIT | __GFP_NOWARN, false); for (i = 0; i < (1 << order); i++) { __clear_bit(PAGE_EXT_OWNER_ALLOCATED, &page_ext->flags); page_owner = get_page_owner(page_ext); @@ -155,6 +159,7 @@ void __reset_page_owner(struct page *page, unsigned short order) page_ext = page_ext_next(page_ext); } page_ext_put(page_ext); + stack_depot_dec_count(alloc_handle); } static inline void __set_page_owner_handle(struct page_ext *page_ext, @@ -189,7 +194,7 @@ noinline void __set_page_owner(struct page *page, unsigned short order, struct page_ext *page_ext; depot_stack_handle_t handle; - handle = save_stack(gfp_mask); + handle = save_stack(gfp_mask, true); page_ext = page_ext_get(page); if (unlikely(!page_ext))