From patchwork Thu Apr 4 07:07:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13617329 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B8E05CD1292 for ; Thu, 4 Apr 2024 07:05:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0E5666B0088; Thu, 4 Apr 2024 03:05:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 094EA6B0089; Thu, 4 Apr 2024 03:05:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E298E6B008A; Thu, 4 Apr 2024 03:05:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id C1B116B0088 for ; Thu, 4 Apr 2024 03:05:53 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 7BD07140152 for ; Thu, 4 Apr 2024 07:05:53 +0000 (UTC) X-FDA: 81970964586.07.7F4181E Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf03.hostedemail.com (Postfix) with ESMTP id 6730820010 for ; Thu, 4 Apr 2024 07:05:51 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=BHMMgYkU; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=DMflKcu2; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf03.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712214351; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ghkXx40nWEPSM9bclOWwsS092PohOPXWVqRF6XEBYQE=; b=liWmXOhXwaog9ZkllomsOgy3rsM/lYDGTuA14lPVan5gaizjNhuIigntLKTEt5gBKibQbq cmi0exznF1A3P3L2Ofq4ELCnnsmVYskh+Uh7GAFWk5V+3kDhy8Vm2OBS+Kw/3anqpAkBvt RZlLRMN1bPHFTIrvYNp6RNGUMP0QhEI= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=BHMMgYkU; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=DMflKcu2; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf03.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712214351; a=rsa-sha256; cv=none; b=4LjoyoQUVnbir7WskOqASuLib877bM9pNIJ79aXhEosnvwmQGvuJQALDXkAtrxT7PuhWnX 4rmLJW+WZXTMZQzWz4h+X8LyAP8JRVGnv2hO7siD1MvDtkTe5xszhXMSYoZRi96mjR/2Lj lasSJLzPXtJd9vS6SOT2u6KTnL6VXpU= Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org [10.150.64.98]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id CF8805D673; Thu, 4 Apr 2024 07:05:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1712214349; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ghkXx40nWEPSM9bclOWwsS092PohOPXWVqRF6XEBYQE=; b=BHMMgYkUiju095r33Rm4aFYQVJAwhFlIDN4uo666GZf1EZGTkXYRxMz5A3BQTzfRhJ8oNb CtTBgFODKCu7/cFXbTZUQheOsQIhLtf4bHs5qlDS6ulVH5UdWyu5SiRrmOxqe0qk6e8ihl +wYgMi7rurqELUnAhNYcjr67THsJgts= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1712214349; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ghkXx40nWEPSM9bclOWwsS092PohOPXWVqRF6XEBYQE=; b=DMflKcu20nVPELyns9/pBRYiLwB8A3fIMLkoYrZfn85iYZ6nYiNRUcVI529KzKH7zGBig0 jJO3FfEYbmXXKABg== Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id 370B413A91; Thu, 4 Apr 2024 07:05:49 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap2.dmz-prg2.suse.org with ESMTPSA id wAuiCk1RDmY6cgAAn2gu4w (envelope-from ); Thu, 04 Apr 2024 07:05:49 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Michal Hocko , Vlastimil Babka , Marco Elver , Andrey Konovalov , Alexander Potapenko , Alexandre Ghiti , Oscar Salvador , syzbot+41bbfdb8d41003d12c0f@syzkaller.appspotmail.com Subject: [PATCH v4 2/4] mm,page_owner: Fix refcount imbalance Date: Thu, 4 Apr 2024 09:07:00 +0200 Message-ID: <20240404070702.2744-3-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240404070702.2744-1-osalvador@suse.de> References: <20240404070702.2744-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 6730820010 X-Stat-Signature: fenn8zhcj3rim7udy9psgk45d9xhrruq X-HE-Tag: 1712214351-906268 X-HE-Meta: U2FsdGVkX18UVF2nZlQXIRvnGaHEo8Z0+wK7n+yrjAmAIqTu/OUCTQt3nHT9a0ZJJa8AnbNq3JEYHUBzPOa9eOcbzm7qFvlZMDHmjm3nJlI7zakkCBW8RPSgSKEaBw86aReir2qCD2TrhwpVsIJTPnA5ss8RvSI1oSRxOO31DbKv51g5hJ3meyz2FG2sHdhFVDcugR6EXqGAfOu1kbncKZka6ZtV4rLY3Gz6KtWMRekJJ76+L2C1zNUbKo00P5nf3CmyKPB0CKe7nYmfBJbmSyq/f/5XVDD6iNhP8zePhJQJOEXFrykIjA9ODGjIJPLsp4DqDmo0gtWzOPa0Z0O23oLJy8ET3nt4CBMbd1jXLWAVEVL1iQGd7kalxfnTFy/wYgELUp8eK2TlN3tckJ7Zo30FCfrEvBbXcIBJbhQrJp+gnNDiNfe2FyyiUevnJRGtiWPNcJ9cM9QqQiESogTov/fhSgvox1E8HWER8sRxdwe6xgTz5brGoRhDtNAlVvfyfDGnBFuKlNpJEeK3YVzNqtpdR1XYT2rAOgih7WViVDKgJk+v2qEHyNuF08DKpQ5hKxf5UTRJ5C0R8L3orJMbFjlH1gqLx/jABZGLTfmfEe/vXxV2eSOX3APddCMrm1tMp4wd0F8vs6BUArA0AISj3FACqmmbOtbr3mnfJU/g3lty+XOlOZPF/J2Wfxe4RtI/9C2NAnVnrA6Gxqdt00uovdgrZy5loElYRZWjXWwtcFHXLNlqiG52/H+bdqNGD7hBwGrKRoomvZlR0Dk1ASi7TGrAfiaSVYBrdKVeZwRyjBsvjvut7VYAiFboM7WHWSbKam4WFCzyWUfcXFV5Odog+maZcSCTnOLuZXUHSQs/EqILHOzdix/drz2pX0oBYh/ymHntK7JvbP6Nc4hP2Dnli5AHxkePeJNoC9mlUcb5K9rCoC5JvEO4Ld0c8EbIN5c1MVF74vFHSNSnPymkPOE 0DzDVsMX X7uOge468nt3xg7ZmRuyl1wVDFX/HUa0Vq3ai2zi4du8w3tpFR6kuUDxjg2ZZy8ZNPvrruVWX4ewRhThSdKJQiMO1cvVV9vm1SpMlCfkIUqcKF9hQ4HcFC8MXFxxCeEHMSX8tClXlAByRLV436vmB1h4RLMg7gVCRPALoYz5GeVPuCq/PiWxAzx5Ow/ymH62bw6Y694Zr99Kah9JTz87QAUc/+Ow0NE43D+ha0sz1gsjQRS4F6lLMCcs/6u9CWMwjZPwWv0JEzTezzdddk8mPpfCPyiU8q7GMixbdIcIcUfPZpiiVI+h4JtwOdwI277p4ss1qG9epIEnTFXpFiTwOhlL1fa0Ecez3or8a99FImxBYZ/w+CjKKDxXu81V5rG6crOD56aRrfINwe304AOnk9G+2mPraLHwN/JKyR45DPdB/J9eGmoZ5/8Uor3X5y4ovJSk5AWL06ye3Hl/DfJsLlnm3AIjbxgWAt3aY8jSI41i6AAe4Jz69dnJcjQDE+VECKhuFYXJhWU/zpCL1k2Xeu+l6Jdqh1B4bWSsJ91NaNHDygRyMXuMLVcOLpQ9Ua89f69HB3oDR0kE25qc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Current code does not contemplate scenarios were an allocation and free operation on the same pages do not handle it in the same amount at once. To give an example, page_alloc_exact(), where we will allocate a page of enough order to stafisfy the size request, but we will free the remainings right away. In the above example, we will increment the stack_record refcount only once, but we will decrease it the same number of times as number of unused pages we have to free. This will lead to a warning because of refcount imbalance. Fix this by recording the number of base pages in the refcount field. Reported-by: syzbot+41bbfdb8d41003d12c0f@syzkaller.appspotmail.com Closes: https://lore.kernel.org/linux-mm/00000000000090e8ff0613eda0e5@google.com Fixes: 217b2119b9e2 ("mm,page_owner: implement the tracking of the stacks count") Signed-off-by: Oscar Salvador Reviewed-by: Vlastimil Babka Tested-by: Alexandre Ghiti --- Documentation/mm/page_owner.rst | 73 +++++++++++++++++---------------- mm/page_owner.c | 34 ++++++++------- 2 files changed, 58 insertions(+), 49 deletions(-) diff --git a/Documentation/mm/page_owner.rst b/Documentation/mm/page_owner.rst index 0d0334cd5179..3a45a20fc05a 100644 --- a/Documentation/mm/page_owner.rst +++ b/Documentation/mm/page_owner.rst @@ -24,10 +24,10 @@ fragmentation statistics can be obtained through gfp flag information of each page. It is already implemented and activated if page owner is enabled. Other usages are more than welcome. -It can also be used to show all the stacks and their outstanding -allocations, which gives us a quick overview of where the memory is going -without the need to screen through all the pages and match the allocation -and free operation. +It can also be used to show all the stacks and their current number of +allocated base pages, which gives us a quick overview of where the memory +is going without the need to screen through all the pages and match the +allocation and free operation. page owner is disabled by default. So, if you'd like to use it, you need to add "page_owner=on" to your boot cmdline. If the kernel is built @@ -75,42 +75,45 @@ Usage cat /sys/kernel/debug/page_owner_stacks/show_stacks > stacks.txt cat stacks.txt - prep_new_page+0xa9/0x120 - get_page_from_freelist+0x7e6/0x2140 - __alloc_pages+0x18a/0x370 - new_slab+0xc8/0x580 - ___slab_alloc+0x1f2/0xaf0 - __slab_alloc.isra.86+0x22/0x40 - kmem_cache_alloc+0x31b/0x350 - __khugepaged_enter+0x39/0x100 - dup_mmap+0x1c7/0x5ce - copy_process+0x1afe/0x1c90 - kernel_clone+0x9a/0x3c0 - __do_sys_clone+0x66/0x90 - do_syscall_64+0x7f/0x160 - entry_SYSCALL_64_after_hwframe+0x6c/0x74 - stack_count: 234 + post_alloc_hook+0x177/0x1a0 + get_page_from_freelist+0xd01/0xd80 + __alloc_pages+0x39e/0x7e0 + allocate_slab+0xbc/0x3f0 + ___slab_alloc+0x528/0x8a0 + kmem_cache_alloc+0x224/0x3b0 + sk_prot_alloc+0x58/0x1a0 + sk_alloc+0x32/0x4f0 + inet_create+0x427/0xb50 + __sock_create+0x2e4/0x650 + inet_ctl_sock_create+0x30/0x180 + igmp_net_init+0xc1/0x130 + ops_init+0x167/0x410 + setup_net+0x304/0xa60 + copy_net_ns+0x29b/0x4a0 + create_new_namespaces+0x4a1/0x820 + nr_base_pages: 16 ... ... echo 7000 > /sys/kernel/debug/page_owner_stacks/count_threshold cat /sys/kernel/debug/page_owner_stacks/show_stacks> stacks_7000.txt cat stacks_7000.txt - prep_new_page+0xa9/0x120 - get_page_from_freelist+0x7e6/0x2140 - __alloc_pages+0x18a/0x370 - alloc_pages_mpol+0xdf/0x1e0 - folio_alloc+0x14/0x50 - filemap_alloc_folio+0xb0/0x100 - page_cache_ra_unbounded+0x97/0x180 - filemap_fault+0x4b4/0x1200 - __do_fault+0x2d/0x110 - do_pte_missing+0x4b0/0xa30 - __handle_mm_fault+0x7fa/0xb70 - handle_mm_fault+0x125/0x300 - do_user_addr_fault+0x3c9/0x840 - exc_page_fault+0x68/0x150 - asm_exc_page_fault+0x22/0x30 - stack_count: 8248 + post_alloc_hook+0x177/0x1a0 + get_page_from_freelist+0xd01/0xd80 + __alloc_pages+0x39e/0x7e0 + alloc_pages_mpol+0x22e/0x490 + folio_alloc+0xd5/0x110 + filemap_alloc_folio+0x78/0x230 + page_cache_ra_order+0x287/0x6f0 + filemap_get_pages+0x517/0x1160 + filemap_read+0x304/0x9f0 + xfs_file_buffered_read+0xe6/0x1d0 [xfs] + xfs_file_read_iter+0x1f0/0x380 [xfs] + __kernel_read+0x3b9/0x730 + kernel_read_file+0x309/0x4d0 + __do_sys_finit_module+0x381/0x730 + do_syscall_64+0x8d/0x150 + entry_SYSCALL_64_after_hwframe+0x62/0x6a + nr_base_pages: 20824 ... cat /sys/kernel/debug/page_owner > page_owner_full.txt diff --git a/mm/page_owner.c b/mm/page_owner.c index 52d1ced0b57f..5df0d6892bdc 100644 --- a/mm/page_owner.c +++ b/mm/page_owner.c @@ -196,7 +196,8 @@ static void add_stack_record_to_list(struct stack_record *stack_record, spin_unlock_irqrestore(&stack_list_lock, flags); } -static void inc_stack_record_count(depot_stack_handle_t handle, gfp_t gfp_mask) +static void inc_stack_record_count(depot_stack_handle_t handle, gfp_t gfp_mask, + int nr_base_pages) { struct stack_record *stack_record = __stack_depot_get_stack_record(handle); @@ -217,15 +218,20 @@ static void inc_stack_record_count(depot_stack_handle_t handle, gfp_t gfp_mask) /* Add the new stack_record to our list */ add_stack_record_to_list(stack_record, gfp_mask); } - refcount_inc(&stack_record->count); + refcount_add(nr_base_pages, &stack_record->count); } -static void dec_stack_record_count(depot_stack_handle_t handle) +static void dec_stack_record_count(depot_stack_handle_t handle, + int nr_base_pages) { struct stack_record *stack_record = __stack_depot_get_stack_record(handle); - if (stack_record) - refcount_dec(&stack_record->count); + if (!stack_record) + return; + + if (refcount_sub_and_test(nr_base_pages, &stack_record->count)) + pr_warn("%s: refcount went to 0 for %u handle\n", __func__, + handle); } static inline void __update_page_owner_handle(struct page_ext *page_ext, @@ -306,7 +312,7 @@ void __reset_page_owner(struct page *page, unsigned short order) * the machinery is not ready yet, we cannot decrement * their refcount either. */ - dec_stack_record_count(alloc_handle); + dec_stack_record_count(alloc_handle, 1 << order); } noinline void __set_page_owner(struct page *page, unsigned short order, @@ -325,7 +331,7 @@ noinline void __set_page_owner(struct page *page, unsigned short order, current->pid, current->tgid, ts_nsec, current->comm); page_ext_put(page_ext); - inc_stack_record_count(handle, gfp_mask); + inc_stack_record_count(handle, gfp_mask, 1 << order); } void __set_page_owner_migrate_reason(struct page *page, int reason) @@ -872,11 +878,11 @@ static void *stack_next(struct seq_file *m, void *v, loff_t *ppos) return stack; } -static unsigned long page_owner_stack_threshold; +static unsigned long page_owner_pages_threshold; static int stack_print(struct seq_file *m, void *v) { - int i, stack_count; + int i, nr_base_pages; struct stack *stack = v; unsigned long *entries; unsigned long nr_entries; @@ -887,14 +893,14 @@ static int stack_print(struct seq_file *m, void *v) nr_entries = stack_record->size; entries = stack_record->entries; - stack_count = refcount_read(&stack_record->count) - 1; + nr_base_pages = refcount_read(&stack_record->count) - 1; - if (stack_count < 1 || stack_count < page_owner_stack_threshold) + if (nr_base_pages < 1 || nr_base_pages < page_owner_pages_threshold) return 0; for (i = 0; i < nr_entries; i++) seq_printf(m, " %pS\n", (void *)entries[i]); - seq_printf(m, "stack_count: %d\n\n", stack_count); + seq_printf(m, "nr_base_pages: %d\n\n", nr_base_pages); return 0; } @@ -924,13 +930,13 @@ static const struct file_operations page_owner_stack_operations = { static int page_owner_threshold_get(void *data, u64 *val) { - *val = READ_ONCE(page_owner_stack_threshold); + *val = READ_ONCE(page_owner_pages_threshold); return 0; } static int page_owner_threshold_set(void *data, u64 val) { - WRITE_ONCE(page_owner_stack_threshold, val); + WRITE_ONCE(page_owner_pages_threshold, val); return 0; }