From patchwork Tue Mar 19 18:32:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13597012 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5B1FDC54E68 for ; Tue, 19 Mar 2024 18:31:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 990F06B008A; Tue, 19 Mar 2024 14:31:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 925A76B0093; Tue, 19 Mar 2024 14:31:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 60C246B008A; Tue, 19 Mar 2024 14:31:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 494D96B008A for ; Tue, 19 Mar 2024 14:31:04 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 27CAEA05FC for ; Tue, 19 Mar 2024 18:31:04 +0000 (UTC) X-FDA: 81914630448.03.C217FE8 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf15.hostedemail.com (Postfix) with ESMTP id 909D6A000E for ; Tue, 19 Mar 2024 18:31:01 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=oepN7adL; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=BOn84qAt; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=XkAouCbD; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b="UUwS1H/y"; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf15.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710873061; a=rsa-sha256; cv=none; b=STDC7ymWNdg2XDsONyhL+PvsyXRx6hgY23zzVvEgypP8zW+8SxoCgJLmHD4ehvMNOpOM18 v0NAHxz0yaLZlDbX9YZiA2JWAA1Fcdi4hPbZQDwRx/QRVoHepBab86FS9/MLy7HvXnj+up KGKPWg5IiulQlV/Z1BZVZt9/1nf0qEg= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=oepN7adL; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=BOn84qAt; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=XkAouCbD; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b="UUwS1H/y"; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf15.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710873061; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=yVHpNVT84Nmx+HHMzsTFanj+M20EJroCOHm/l9Ol/3k=; b=Vy/ZxyAL6yA6+tqmEDvpcxiiTf7MvWYpoMJ2phXdrQ+5vp1O6CbulhTiJKvGybVUg262pG 9mAxKlKPyaM/AWBzcZh58OXQTJjfFKuqAgyjZ9hoKz2g0i4aS2g/Avvp1FRhlubVrqMwPt aFNxkzY+IGg8RJdpmb//QFS50EWTTvM= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id B184B1F793; Tue, 19 Mar 2024 18:30:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1710873059; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yVHpNVT84Nmx+HHMzsTFanj+M20EJroCOHm/l9Ol/3k=; b=oepN7adLYEkqu0/FqXSG1vbkvf7JYO/8bJpl62R6cYwRQAJfju7cJm/GwpJoln7kZRGlBe P7MZVOWnizt6kfNa0Egbt8sNADRRntQlzdlWkM+oyFjThUPrEPjYfWyVITLxZ+TWSXKqs9 zwzrmBX9BvefVeLtoQBrxSqk344SDvQ= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1710873059; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yVHpNVT84Nmx+HHMzsTFanj+M20EJroCOHm/l9Ol/3k=; b=BOn84qAtIgqBmZA9fhiijuT60AJbt2zmzMSMdd9JYDPqFHTd9VrQxnR+y6sXhYj0V26ef/ YeaGEzIshnUfWkCg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1710873057; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yVHpNVT84Nmx+HHMzsTFanj+M20EJroCOHm/l9Ol/3k=; b=XkAouCbDn1pAn9GVmuLui7lev/VVjBHkqeNzeTBZY42dLeDmZn603+qm86mbvyO6CNPLOP 71Wsk1HBfYWVMk9SdPKzPGELk0n5rtjEIi47GZAElyLg0LKtYNrkVwNyFXi3Ae/qaAKaHI D5FqIM83HJ4M5gvw2X+iZr8Yemm4ZqE= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1710873057; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yVHpNVT84Nmx+HHMzsTFanj+M20EJroCOHm/l9Ol/3k=; b=UUwS1H/yX5oTcR1MKprv2M2b/mhcT50lDbVvyu50VRUBQ/rQa+DFhyZO2XSaCpHqUs4L5t 8iXfY2RkvNXP20DA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 14F37136D6; Tue, 19 Mar 2024 18:30:57 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id AEF7AuHZ+WUcTwAAD6G6ig (envelope-from ); Tue, 19 Mar 2024 18:30:57 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Michal Hocko , Vlastimil Babka , Marco Elver , Andrey Konovalov , Alexander Potapenko , Tetsuo Handa , Oscar Salvador , syzbot+41bbfdb8d41003d12c0f@syzkaller.appspotmail.com Subject: [PATCH v2 1/2] mm,page_owner: Fix refcount imbalance Date: Tue, 19 Mar 2024 19:32:11 +0100 Message-ID: <20240319183212.17156-2-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240319183212.17156-1-osalvador@suse.de> References: <20240319183212.17156-1-osalvador@suse.de> MIME-Version: 1.0 X-Spamd-Bar: + X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 909D6A000E X-Stat-Signature: j8a9i9n4idwmbg88oydue1id9jgm9wfo X-HE-Tag: 1710873061-735191 X-HE-Meta: U2FsdGVkX19k6fq/HwkHcO613WG59pDdX4gZTSHcV7EOswJxAQE/2DDzEACTZ5JDvBbFDjXYRhNEu2eXcYaE9jXqeew8I5BMK1I1lOrN9cfN8xX6+QAJax/e7BhXLlXqAE9huBmwCvVDDS6xkUku6/CRywbQ4inIaQyvtqJkG5949SdNSsavnSrKeC1vwbAykfGF9DG86UiuNznfStwvO6paXKc095sA3wtFrS0nbg1uuJ4X6uCgcyhhFyUas7zgZpOdjuAVPdULC+hu1i+kj0gqKa0+6IDIxHYAZBYKw+slvpojTV5pOTrKnxgM2goePWK53cwqoNUuUXVgAssvgHgs69QlBrItY9gAzjqAY0h+ir16aEew5pw7tOnqYAPO9yGyzVVKJL33/eDmcFNy1GgBA1+gkAwOCEeKIoMA8sW1cittI6KxfiP1S45DP/lXmoMw7uyYjEwMQkzjg1j4ABkY5IKvmIauQfmBrq5Dl3ywIbDheQ80HhtubOlsj2qU3sNJfIYaGRpyT6+vscBq2rbhkuhASvTGbeq+nDPadpZ71rxBlXyKJNI1MGPTDhfNtvUYha/yYorh/cKmicA6FxqMVQ9qZMwcGw76d+s5EPKAslBFzKRJDzJBH7v4etz1WWnS1AQb7rz+0EXGJlrMQxJWPunFsUYqNeEg5TFvOmUCMsVvdKtnsxeldPAq/N7GIATtpXkRs1OLaTZ+AaeRhM3oh3WKkhqzY1FnW9MMH4vX30saUXHmYieaaYsV5hUh7qJVn8IXoYPKIvEm14wJdCwhFniuSoeDm1agDl0iTZ/onO+FiETK7hliov3uIhDOA63zapDhg1zIQ/cyYz3H1S+gLz5y5hI6W+X4s2L5a9ys0QqCGmC57CzIk6enOGZn2ilFhh52yQs3eBvLWKgNF5ptuA39GC3r4wFYBq36MH0Qk8xPVlBqaiCcvycIoWIe64j/GwSol4AYM2EF9rS NFBzsksy 7MlkNmeViLSSb35xm7CaDSYRYEw3f4Wv0tJB9k/Z2jm5v0oEByV7DP56q78UbYA5F+hPnDL3IjJxnFBCOsdhPcUMggkJN6RfZtPG3W6yAZKkfSYy9abuxTIy8jb1wl985bjRjto8Z84JRrcHyQE9sYNR1i0oWK8p2oXOyQvF8MQVTlNVomFYdpVtocf1v1xUj2aDuHq77t2yNf7hIXciu/ZEZffIDs2fjF79ilcU3aaYEpI3Wa4x3HIVc1kUFelBVmsEo4+qycCUijxkhJV9pAlITJilTM1ExX3curVGxhsI6hy8mgIa//sWcpHf5WL98DEH6eNihMdtLJjPsyWik1o0ROeUd+qGznXmkVUc0qLBIqx2Agge3uG8D/jip0HWdPW7GGCB+s4RoRNsQziC3HMMJB1Pt6bpwMxt1Uk+xdBu0z5E5fpNtJamCYtk5MIQ5PvpJJDzAxfy4f8YEJa6IyTnBQqSYzPwYtJtSRVotsx7IxbSgWsO2IU16HD6AMNa0Gs1YaObBxp7ZLoI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Current code does not contemplate scenarios were an allocation and free operation on the same pages do not handle it in the same amount at once. To give an example, page_alloc_exact(), where we will allocate a page of enough order to stafisfy the size request, but we will free the remainings right away. In the above example, we will increment the stack_record refcount only once, but we will decrease it the same number of times as number of unused pages we have to free. This will lead to a warning because of refcount imbalance. Fix this by recording the number of base pages in the refcount field. Reported-by: syzbot+41bbfdb8d41003d12c0f@syzkaller.appspotmail.com Closes: https://lore.kernel.org/linux-mm/00000000000090e8ff0613eda0e5@google.com Fixes: 217b2119b9e2 ("mm,page_owner: implement the tracking of the stacks count") Signed-off-by: Oscar Salvador Reviewed-by: Vlastimil Babka --- Documentation/mm/page_owner.rst | 73 +++++++++++++++++---------------- mm/page_owner.c | 38 ++++++++--------- 2 files changed, 56 insertions(+), 55 deletions(-) diff --git a/Documentation/mm/page_owner.rst b/Documentation/mm/page_owner.rst index 0d0334cd5179..3a45a20fc05a 100644 --- a/Documentation/mm/page_owner.rst +++ b/Documentation/mm/page_owner.rst @@ -24,10 +24,10 @@ fragmentation statistics can be obtained through gfp flag information of each page. It is already implemented and activated if page owner is enabled. Other usages are more than welcome. -It can also be used to show all the stacks and their outstanding -allocations, which gives us a quick overview of where the memory is going -without the need to screen through all the pages and match the allocation -and free operation. +It can also be used to show all the stacks and their current number of +allocated base pages, which gives us a quick overview of where the memory +is going without the need to screen through all the pages and match the +allocation and free operation. page owner is disabled by default. So, if you'd like to use it, you need to add "page_owner=on" to your boot cmdline. If the kernel is built @@ -75,42 +75,45 @@ Usage cat /sys/kernel/debug/page_owner_stacks/show_stacks > stacks.txt cat stacks.txt - prep_new_page+0xa9/0x120 - get_page_from_freelist+0x7e6/0x2140 - __alloc_pages+0x18a/0x370 - new_slab+0xc8/0x580 - ___slab_alloc+0x1f2/0xaf0 - __slab_alloc.isra.86+0x22/0x40 - kmem_cache_alloc+0x31b/0x350 - __khugepaged_enter+0x39/0x100 - dup_mmap+0x1c7/0x5ce - copy_process+0x1afe/0x1c90 - kernel_clone+0x9a/0x3c0 - __do_sys_clone+0x66/0x90 - do_syscall_64+0x7f/0x160 - entry_SYSCALL_64_after_hwframe+0x6c/0x74 - stack_count: 234 + post_alloc_hook+0x177/0x1a0 + get_page_from_freelist+0xd01/0xd80 + __alloc_pages+0x39e/0x7e0 + allocate_slab+0xbc/0x3f0 + ___slab_alloc+0x528/0x8a0 + kmem_cache_alloc+0x224/0x3b0 + sk_prot_alloc+0x58/0x1a0 + sk_alloc+0x32/0x4f0 + inet_create+0x427/0xb50 + __sock_create+0x2e4/0x650 + inet_ctl_sock_create+0x30/0x180 + igmp_net_init+0xc1/0x130 + ops_init+0x167/0x410 + setup_net+0x304/0xa60 + copy_net_ns+0x29b/0x4a0 + create_new_namespaces+0x4a1/0x820 + nr_base_pages: 16 ... ... echo 7000 > /sys/kernel/debug/page_owner_stacks/count_threshold cat /sys/kernel/debug/page_owner_stacks/show_stacks> stacks_7000.txt cat stacks_7000.txt - prep_new_page+0xa9/0x120 - get_page_from_freelist+0x7e6/0x2140 - __alloc_pages+0x18a/0x370 - alloc_pages_mpol+0xdf/0x1e0 - folio_alloc+0x14/0x50 - filemap_alloc_folio+0xb0/0x100 - page_cache_ra_unbounded+0x97/0x180 - filemap_fault+0x4b4/0x1200 - __do_fault+0x2d/0x110 - do_pte_missing+0x4b0/0xa30 - __handle_mm_fault+0x7fa/0xb70 - handle_mm_fault+0x125/0x300 - do_user_addr_fault+0x3c9/0x840 - exc_page_fault+0x68/0x150 - asm_exc_page_fault+0x22/0x30 - stack_count: 8248 + post_alloc_hook+0x177/0x1a0 + get_page_from_freelist+0xd01/0xd80 + __alloc_pages+0x39e/0x7e0 + alloc_pages_mpol+0x22e/0x490 + folio_alloc+0xd5/0x110 + filemap_alloc_folio+0x78/0x230 + page_cache_ra_order+0x287/0x6f0 + filemap_get_pages+0x517/0x1160 + filemap_read+0x304/0x9f0 + xfs_file_buffered_read+0xe6/0x1d0 [xfs] + xfs_file_read_iter+0x1f0/0x380 [xfs] + __kernel_read+0x3b9/0x730 + kernel_read_file+0x309/0x4d0 + __do_sys_finit_module+0x381/0x730 + do_syscall_64+0x8d/0x150 + entry_SYSCALL_64_after_hwframe+0x62/0x6a + nr_base_pages: 20824 ... cat /sys/kernel/debug/page_owner > page_owner_full.txt diff --git a/mm/page_owner.c b/mm/page_owner.c index d17d1351ec84..2613805cb665 100644 --- a/mm/page_owner.c +++ b/mm/page_owner.c @@ -196,9 +196,11 @@ static void add_stack_record_to_list(struct stack_record *stack_record, spin_unlock_irqrestore(&stack_list_lock, flags); } -static void inc_stack_record_count(depot_stack_handle_t handle, gfp_t gfp_mask) +static void inc_stack_record_count(depot_stack_handle_t handle, gfp_t gfp_mask, + int nr_base_pages) { struct stack_record *stack_record = __stack_depot_get_stack_record(handle); + int old = REFCOUNT_SATURATED; if (!stack_record) return; @@ -210,22 +212,18 @@ static void inc_stack_record_count(depot_stack_handle_t handle, gfp_t gfp_mask) * Since we do not use STACK_DEPOT_FLAG_GET API, let us * set a refcount of 1 ourselves. */ - if (refcount_read(&stack_record->count) == REFCOUNT_SATURATED) { - int old = REFCOUNT_SATURATED; - - if (atomic_try_cmpxchg_relaxed(&stack_record->count.refs, &old, 1)) - /* Add the new stack_record to our list */ - add_stack_record_to_list(stack_record, gfp_mask); - } - refcount_inc(&stack_record->count); + if (atomic_try_cmpxchg_relaxed(&stack_record->count.refs, &old, 1)) + add_stack_record_to_list(stack_record, gfp_mask); + refcount_add(nr_base_pages, &stack_record->count); } -static void dec_stack_record_count(depot_stack_handle_t handle) +static void dec_stack_record_count(depot_stack_handle_t handle, + int nr_base_pages) { struct stack_record *stack_record = __stack_depot_get_stack_record(handle); if (stack_record) - refcount_dec(&stack_record->count); + refcount_sub_and_test(nr_base_pages, &stack_record->count); } void __reset_page_owner(struct page *page, unsigned short order) @@ -263,7 +261,7 @@ void __reset_page_owner(struct page *page, unsigned short order) * the machinery is not ready yet, we cannot decrement * their refcount either. */ - dec_stack_record_count(alloc_handle); + dec_stack_record_count(alloc_handle, 1 << order); } static inline void __set_page_owner_handle(struct page_ext *page_ext, @@ -305,7 +303,7 @@ noinline void __set_page_owner(struct page *page, unsigned short order, return; __set_page_owner_handle(page_ext, handle, order, gfp_mask); page_ext_put(page_ext); - inc_stack_record_count(handle, gfp_mask); + inc_stack_record_count(handle, gfp_mask, 1 << order); } void __set_page_owner_migrate_reason(struct page *page, int reason) @@ -861,11 +859,11 @@ static void *stack_next(struct seq_file *m, void *v, loff_t *ppos) return stack; } -static unsigned long page_owner_stack_threshold; +static unsigned long page_owner_pages_threshold; static int stack_print(struct seq_file *m, void *v) { - int i, stack_count; + int i, nr_base_pages; struct stack *stack = v; unsigned long *entries; unsigned long nr_entries; @@ -876,14 +874,14 @@ static int stack_print(struct seq_file *m, void *v) nr_entries = stack_record->size; entries = stack_record->entries; - stack_count = refcount_read(&stack_record->count) - 1; + nr_base_pages = refcount_read(&stack_record->count) - 1; - if (stack_count < 1 || stack_count < page_owner_stack_threshold) + if (nr_base_pages < 1 || nr_base_pages < page_owner_pages_threshold) return 0; for (i = 0; i < nr_entries; i++) seq_printf(m, " %pS\n", (void *)entries[i]); - seq_printf(m, "stack_count: %d\n\n", stack_count); + seq_printf(m, "nr_base_pages: %d\n\n", nr_base_pages); return 0; } @@ -913,13 +911,13 @@ static const struct file_operations page_owner_stack_operations = { static int page_owner_threshold_get(void *data, u64 *val) { - *val = READ_ONCE(page_owner_stack_threshold); + *val = READ_ONCE(page_owner_pages_threshold); return 0; } static int page_owner_threshold_set(void *data, u64 val) { - WRITE_ONCE(page_owner_stack_threshold, val); + WRITE_ONCE(page_owner_pages_threshold, val); return 0; } From patchwork Tue Mar 19 18:32:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13597011 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D95C4C54E71 for ; Tue, 19 Mar 2024 18:31:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 612886B0092; Tue, 19 Mar 2024 14:31:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5BCDD6B0095; Tue, 19 Mar 2024 14:31:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 437C56B0092; Tue, 19 Mar 2024 14:31:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 2DE6D6B008A for ; Tue, 19 Mar 2024 14:31:04 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 01F62A08E2 for ; Tue, 19 Mar 2024 18:31:03 +0000 (UTC) X-FDA: 81914630448.05.745335A Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf03.hostedemail.com (Postfix) with ESMTP id C79A42000A for ; Tue, 19 Mar 2024 18:31:01 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=pfjDOKf1; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=BoyQKkry; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=pfjDOKf1; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=BoyQKkry; spf=pass (imf03.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710873062; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=qBBDN69VSXz+frM1TA/lUSR1pRpjSQYoj51jkyNzGIg=; b=5ceyaqdua0/Ea4ocg3rCr5WBWtmjr5hfnQKptXi5l3uxRv/XcbXbWCyk1cPSZRIL/t/W+E GK74HFJ+yin+/guaXc+L+Zey4i3yHlLGWjrld8NPohaRN8iymVD/L5Lo2lxdccJAc9R4sP h/S9cZ4aMc+pDPQJeVBxPHxU4+p0NqA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710873062; a=rsa-sha256; cv=none; b=WbtRS/QDLcAJuJPvHmBmQBK5cQnP7GGi9dX8P49heBfpxMU6R593f6DDLQf3BrlCKFZohQ HKSsEYmrElt+WdS+xQxVRO7gx1NDOJmDK2pL8Cff2T8fJqiRzHnT8hHt1owahcOCT4gIWZ eoMbyQYGRJxK2gP6utMBM3NDOhoPMEg= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=pfjDOKf1; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=BoyQKkry; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=pfjDOKf1; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=BoyQKkry; spf=pass (imf03.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 45A201F796; Tue, 19 Mar 2024 18:31:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1710873060; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qBBDN69VSXz+frM1TA/lUSR1pRpjSQYoj51jkyNzGIg=; b=pfjDOKf1uxzUNJ613z2gjhyKrzyrDuE8LKyTAT5UQm6rAtbuXqpqdP8UBZJz6l1MH5xLTy smF3WyK1+vBwfEdlPS6dnAELJcWwm1rgYNOA4mM8HwzcOocBjHNKE06257PIzdqNEpzNqF LRSAAF5lzHrFo/Hxb6eW4NNICqTXaJU= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1710873060; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qBBDN69VSXz+frM1TA/lUSR1pRpjSQYoj51jkyNzGIg=; b=BoyQKkryctPqM+pUgEPoLlD4178P6d2phPz0yy86fhQEV6syMeGv4jT7b9hWlsYTd8Nmyf SCesS3h7e8TtulCA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1710873060; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qBBDN69VSXz+frM1TA/lUSR1pRpjSQYoj51jkyNzGIg=; b=pfjDOKf1uxzUNJ613z2gjhyKrzyrDuE8LKyTAT5UQm6rAtbuXqpqdP8UBZJz6l1MH5xLTy smF3WyK1+vBwfEdlPS6dnAELJcWwm1rgYNOA4mM8HwzcOocBjHNKE06257PIzdqNEpzNqF LRSAAF5lzHrFo/Hxb6eW4NNICqTXaJU= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1710873060; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qBBDN69VSXz+frM1TA/lUSR1pRpjSQYoj51jkyNzGIg=; b=BoyQKkryctPqM+pUgEPoLlD4178P6d2phPz0yy86fhQEV6syMeGv4jT7b9hWlsYTd8Nmyf SCesS3h7e8TtulCA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id A91941376B; Tue, 19 Mar 2024 18:30:59 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id wBWfJuPZ+WUcTwAAD6G6ig (envelope-from ); Tue, 19 Mar 2024 18:30:59 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Michal Hocko , Vlastimil Babka , Marco Elver , Andrey Konovalov , Alexander Potapenko , Tetsuo Handa , Oscar Salvador Subject: [PATCH v2 2/2] mm,page_owner: Fix accounting of pages when migrating Date: Tue, 19 Mar 2024 19:32:12 +0100 Message-ID: <20240319183212.17156-3-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240319183212.17156-1-osalvador@suse.de> References: <20240319183212.17156-1-osalvador@suse.de> MIME-Version: 1.0 X-Spamd-Bar: +++ X-Rspamd-Queue-Id: C79A42000A X-Rspam-User: X-Stat-Signature: wyajthypp9zywxawfd9hgo4n8p4kfcgy X-Rspamd-Server: rspam03 X-HE-Tag: 1710873061-568951 X-HE-Meta: U2FsdGVkX1+afQL/JIkbaBQkZ/Wg9hfJ9Ecepl8GNVpiKC1uCVv8C6PYIiF5MgDkAWdaDCmZvmY4ZbU7iPutZUTlnCxZ3oNJ8G7z9VjBDIz83DVI7yVz38kqbiRqLCXnihXS4ei1+40n3SRw2yeWhAfrodsHrazDlT5rYp6FttkKL61NtiKLQJQ7GIVE2rBOomhcXdklNHqUvOej4y6MwJ/zM1DKTCNqfrJjS7GbQqwx6ry/36OFWOGnEnJv3+eruE8csq1sW/8OyxOGvH81ndt2rf6j80+rDo0IqS32RtuGYBdcJg2vhUHjOC2FWctDF/fuTpUIO5kVfrp+IUl503RkPA73pdHTg1bv7JGRUh1McnKQq3QF22FdNBI8rAVOEiZc9yw7KgBFZnotEVwjfwtdxhezE32Sq6+en/19lP8+7Yb25RyamLwqYvs+fmCu+Wnya+Fpa/7iCXr46ja8liYRejw0qcykoK3B+Axqt+iLeaFTXzDnz+nlDILvdnRZb7ScKxkIGmEH6BsTcSGIeaWsIV8+6Ocpi3uGCuTxQwNHCkKrdXBmgJumyECr7Q3J7LmVHJQYsediM4nZaa2XXRfST4XuBmEDGGnM1hQZlkhBWw33wmJHKt+AKng8Ka34uNCpeD2PTMAy3eqP3TQiFVooq6jIp/oYGXJJrav8zCEkx7z3oq+SJoDDc9zFZtlMF1cn/58Km2E0iZeaAewAfmP/u1dXGvSI/8euloxXKlI1UF1tHo0hdqUnKB6eS97FeB36C9AWoms9eayG58l3M9ebWCHC8aL/6FOcxKdCAmqfUBFki6+QjDeQd8B5SGguXNWi5Xxeh1vtPc//bST16BxMpR6XC7HXMEl1U6dfuojAQUTcnRDIi4mREPM8b4sMYspQHkCnLwQPf7M5Qq+Byn7bZvO6kjTma18xEizAGYFe4eIv6LbcoV3AvYssH/zw2ixJZjYJLNldBsucVhN lfni5Gs7 syTy4j6DixZJuupgkqwMZJ7xqKgsP1rWq7dWi X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Upon migration, new allocated pages are being given the handle of the old pages. This is problematic because it means that for the stack which allocated the old page, we will be substracting the old page + the new one when that page is freed, creating an accounting imbalance. Fix this by adding a new migrate_handle in the page_owner struct, and record the handle that allocated the new page in __folio_copy_owner(). Upon freeing, we check whether we have a migrate_handle, and if we do, we use migrate_handle for dec_stack_record_count(), which will subtract those pages from its right handle. Fixes: 217b2119b9e2 ("mm,page_owner: implement the tracking of the stacks count") Signed-off-by: Oscar Salvador --- mm/page_owner.c | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/mm/page_owner.c b/mm/page_owner.c index 2613805cb665..1a7d0d1dc640 100644 --- a/mm/page_owner.c +++ b/mm/page_owner.c @@ -27,6 +27,7 @@ struct page_owner { gfp_t gfp_mask; depot_stack_handle_t handle; depot_stack_handle_t free_handle; + depot_stack_handle_t migrate_handle; u64 ts_nsec; u64 free_ts_nsec; char comm[TASK_COMM_LEN]; @@ -240,7 +241,15 @@ void __reset_page_owner(struct page *page, unsigned short order) return; page_owner = get_page_owner(page_ext); - alloc_handle = page_owner->handle; + /* + * If this page was allocated for migration purposes, its handle doesn't + * reference the stack it was allocated from, so make sure to use the + * migrate_handle in order to subtract it from the right stack. + */ + if (!page_owner->migrate_handle) + alloc_handle = page_owner->handle; + else + alloc_handle = page_owner->migrate_handle; handle = save_stack(GFP_NOWAIT | __GFP_NOWARN); for (i = 0; i < (1 << order); i++) { @@ -277,6 +286,7 @@ static inline void __set_page_owner_handle(struct page_ext *page_ext, page_owner->handle = handle; page_owner->order = order; page_owner->gfp_mask = gfp_mask; + page_owner->migrate_handle = 0; page_owner->last_migrate_reason = -1; page_owner->pid = current->pid; page_owner->tgid = current->tgid; @@ -358,6 +368,7 @@ void __folio_copy_owner(struct folio *newfolio, struct folio *old) new_page_owner->gfp_mask = old_page_owner->gfp_mask; new_page_owner->last_migrate_reason = old_page_owner->last_migrate_reason; + new_page_owner->migrate_handle = new_page_owner->handle; new_page_owner->handle = old_page_owner->handle; new_page_owner->pid = old_page_owner->pid; new_page_owner->tgid = old_page_owner->tgid;