From patchwork Wed Feb 21 19:40:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suren Baghdasaryan X-Patchwork-Id: 13566180 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 20148C48BEB for ; Wed, 21 Feb 2024 19:41:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A958F6B006E; Wed, 21 Feb 2024 14:40:59 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A1F146B0083; Wed, 21 Feb 2024 14:40:59 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 871B06B0085; Wed, 21 Feb 2024 14:40:59 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 6E3756B006E for ; Wed, 21 Feb 2024 14:40:59 -0500 (EST) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 07554A0BBC for ; Wed, 21 Feb 2024 19:40:59 +0000 (UTC) X-FDA: 81816829038.04.4A4E5DC Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) by imf19.hostedemail.com (Postfix) with ESMTP id 456821A0006 for ; Wed, 21 Feb 2024 19:40:57 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=XgqaUJAJ; spf=pass (imf19.hostedemail.com: domain of 3yFHWZQYKCPktvsfochpphmf.dpnmjovy-nnlwbdl.psh@flex--surenb.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3yFHWZQYKCPktvsfochpphmf.dpnmjovy-nnlwbdl.psh@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1708544457; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=h6iKERH6ntSMYXEOBWW++ghbM5EttVzR7SwASxK2UGU=; b=KdaOh55etFWzQc6R1huTG/YLHDPYCKwJLGm46pqM/FcEXXU4NxUUJl68uP0lwMu1OF7QZA tcAfpxeoiniak6MLxCRkt4mwHeVwhmMX4OZ0MIKybreonTXiRQkKBDI0DoSEHric8iGumY YI18jr8YWfdAMHkwingC5bnHHLIrGYY= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=XgqaUJAJ; spf=pass (imf19.hostedemail.com: domain of 3yFHWZQYKCPktvsfochpphmf.dpnmjovy-nnlwbdl.psh@flex--surenb.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3yFHWZQYKCPktvsfochpphmf.dpnmjovy-nnlwbdl.psh@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1708544457; a=rsa-sha256; cv=none; b=tBFmr2poOiCwtWXJvpTN08Vbsq2fljHYYYcqRN3+0C65VTrcLPLOg7q2XNYo10TM/ZPWDz 5etGpe+fDmqGRVmwhGICbCLST3XN0qJf0bIPM2tWHn8+mZvgKGmDxIsiELtEOBFm71OF7V lKU5QpkfqPzwKz3TC/XKRfS+4dhBh4o= Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6087e575573so22015707b3.1 for ; Wed, 21 Feb 2024 11:40:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1708544456; x=1709149256; darn=kvack.org; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=h6iKERH6ntSMYXEOBWW++ghbM5EttVzR7SwASxK2UGU=; b=XgqaUJAJ+VgFbbV7TfE2+WtU3Ko+moXM/UgdoRjM8Bmpqe/wBhEicBcmFa4+zyug5W 0qJxT0DbZcctcI1iAYZoTojcMCpNjhpNzfSio3yAmssVv3Pz1J1Syzx/GRSmfGPwPyhD 4XO7mSr9qmSA7DvRFU4tX7GmsIaEWCbhYSkAqj2GjVZ5Dl2h7XTtDG2qLmbS4TjOHNUk D7QPhN3GCIUrGquJy5K2Xscy8UNdv3RXhtSvmRJr0YnFba42zT3nrdWtaWj8pxv7G8Hf /5DBgY0aqjY7TI8lC+aWyM2FsQycpr8HBhsefGWp/ZpJ89kLdliID/tXGO3YKByBIKPN bpbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708544456; x=1709149256; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=h6iKERH6ntSMYXEOBWW++ghbM5EttVzR7SwASxK2UGU=; b=kaAbtrYzPGzuCQkcxyKqcC539zJM3FDIYrloz2pIEmFMzazu+nSU0i9g0mkHiwaoXm siKVmV2R3EqiJ2AiyNALU8pqh2lfbayDsWdXV3Ng16g3WYkakwm4qgHv8ODbECZt6oSL NEsgRvneV3UtzbRKpAuofIV+YCv+AEUlz0xHNOKtsz8iVYQ5WNBXHj6r9xj3n7ds+vJv fOnfODD+x/w+4eXerL5TLoH08BNIRIPhACi0PVlxpxWA0Cg02PYVpEL8hMESJlMqum2y 64AaxnR0ouNqt6cyf+GhWRE/65X6woz2sBsWgenGLln5IMO5NhD9WTf6ZEbAM8QjIQWy fBdw== X-Forwarded-Encrypted: i=1; AJvYcCVtg0OH7tNEHl2pd5tXWAMEGf+YVG0wbJJVGrwmdwWaJct8s5HixDlPG9/oF1nwR9DUVUskyQFgMX+yINMKcuuEL58= X-Gm-Message-State: AOJu0Yza7+dwVgqt53GBcjhFDBHCz9aPLWZ1DPnYTr0+UHdKe0XEykwD C7ByzxcNjJ3Oc6lQ/ojKfWVmDCacMVID2ioVSyeaGTpxSlgl9F/s9RNSDAOUUHGYeo8cmFKvbit ukw== X-Google-Smtp-Source: AGHT+IERtCiqJmpfE+kOOdfj1tQRATSlaGrV4xShe/vfh3pjWvq23/f72b3b+6kHY2joSvf7k2m+y0djO/k= X-Received: from surenb-desktop.mtv.corp.google.com ([2620:15c:211:201:953b:9a4e:1e10:3f07]) (user=surenb job=sendgmr) by 2002:a81:92ce:0:b0:607:74b2:579c with SMTP id j197-20020a8192ce000000b0060774b2579cmr4971839ywg.1.1708544456179; Wed, 21 Feb 2024 11:40:56 -0800 (PST) Date: Wed, 21 Feb 2024 11:40:13 -0800 Mime-Version: 1.0 X-Mailer: git-send-email 2.44.0.rc0.258.g7320e95886-goog Message-ID: <20240221194052.927623-1-surenb@google.com> Subject: [PATCH v4 00/36] Memory allocation profiling From: Suren Baghdasaryan To: akpm@linux-foundation.org Cc: kent.overstreet@linux.dev, mhocko@suse.com, vbabka@suse.cz, hannes@cmpxchg.org, roman.gushchin@linux.dev, mgorman@suse.de, dave@stgolabs.net, willy@infradead.org, liam.howlett@oracle.com, penguin-kernel@i-love.sakura.ne.jp, corbet@lwn.net, void@manifault.com, peterz@infradead.org, juri.lelli@redhat.com, catalin.marinas@arm.com, will@kernel.org, arnd@arndb.de, tglx@linutronix.de, mingo@redhat.com, dave.hansen@linux.intel.com, x86@kernel.org, peterx@redhat.com, david@redhat.com, axboe@kernel.dk, mcgrof@kernel.org, masahiroy@kernel.org, nathan@kernel.org, dennis@kernel.org, tj@kernel.org, muchun.song@linux.dev, rppt@kernel.org, paulmck@kernel.org, pasha.tatashin@soleen.com, yosryahmed@google.com, yuzhao@google.com, dhowells@redhat.com, hughd@google.com, andreyknvl@gmail.com, keescook@chromium.org, ndesaulniers@google.com, vvvvvv@google.com, gregkh@linuxfoundation.org, ebiggers@google.com, ytcoode@gmail.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, bristot@redhat.com, vschneid@redhat.com, cl@linux.com, penberg@kernel.org, iamjoonsoo.kim@lge.com, 42.hyeyoo@gmail.com, glider@google.com, elver@google.com, dvyukov@google.com, shakeelb@google.com, songmuchun@bytedance.com, jbaron@akamai.com, rientjes@google.com, minchan@google.com, kaleshsingh@google.com, surenb@google.com, kernel-team@android.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev, linux-arch@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-modules@vger.kernel.org, kasan-dev@googlegroups.com, cgroups@vger.kernel.org X-Rspamd-Queue-Id: 456821A0006 X-Rspam-User: X-Stat-Signature: 88obdpro11t1qru6wx3fpuu5w4qud8s1 X-Rspamd-Server: rspam01 X-HE-Tag: 1708544457-732754 X-HE-Meta: U2FsdGVkX18V/bFywxGnbdRgB5eOemabQ0Ha7grTSlYrqpbYotpWbsSG8gTuZZyxMPlGeQBJzi3XXNYIKvoIIjJ0BeoxZEDM1OaGzAgw7vfsjUPGu9gnnaRV6Hb1pvdQPbE6ZlVrTXNE91ysHaAvuDFmYDyIaVqD/S9wxs9CdQD2FO69IVV87ryD0sh8kXy6zxtmJrKKJh/kykDnF/r8epKt3aSqM9SbMeT5KDuBvDpFRxOl0xkoFcGISkx3YPUn2kwqln+9xJqTZXBw6PdiO7VV5MiQQJPXTDor7k20wbC4mUU5u3i476oFrUGtbb6JeEGG2YFQU7osjuYDABBZr4n9vgg23DC+1nFGIPqQM7/frHAD70Ty8lZZ3HfD4xgUStY6meCHyCFJk90ghvF37ayOa6vhwkevYfo6GPpEhSAJsMkx3f2hzwQ9q93vdUr6CQjLc6mVA4iXrRar7rvXd7b7Tcv6AeiisuSCumtUdMfv6Rq/qZYNyECjgHMiWex/X1+5Jbh6G08ksVIzTo6/3Zq5fh/93yr0tzQamVSqqYjxOd/xY7/85YD7VM5XkfCFa/ykyp8wZyX2201RA7I0/FUfLrO3LWtReB3TpHlfVjsxJ2nFIOVen9DFlNU65hPA5EITYbfQHNQMukrKzXl9dNRYUN3Pq5DgMg4kCW67n8ZJ8d/jt8SzcQHawWyFmoWrmoXCL9ETAiwWZthptPlemluGCCKWf5q+xE/ssSUsrsFyRewhp1hFKPnex2z5OUoQURo5PPn64a/SEEc3pN8dSrQWLlhhwZdmqXUh4MEo5K3EvgTRkc0KsmmoD5QGGQZ4QsgB8qE3Gn56+fegD2/HtUYnRtfsjPezAJHxkMGD6ek1Ffm7lAwBTxy5YeqmcYz7OO6i0QieZd56LHx0Xq6otEv0kXgX3O2IaQX77Qhf3jlc2iZip5rUdJ/Ojc1t95hZ9KE2xcFIFmN7fRLa4xl V+496XZo 9TFmhgN01DzQzynDaITwzm+n0sEtaWVLlSRi0IbSNu2DkqXrJtGF7lGkMM8/UbI9IvLPTJgadBThOtZ+m+/Y8FRcrMR/4m1YNfCQOAav/FnhM9HQMKPfMEzdjZw6JFsgr+hL9bZzxyHTL46mwEXBKq3kB4aAAUhWe8/Cv1weVhYUyEKd3EZ+54aMl2uxrIMWP8CNkk1GXVCdh/kMa/r1ev7ywLcV0hHwJu675qtUOTiXmv87Ss/YYQi23/BmQFj1F4FQ2spsKdh4irPDTMU7kaL1e2M9+SG4ndopOfuBePGu9espiBmFDr44UB5AYpmU1JgqknbnfrXSRLJLnSQNBkY37Jaxn4kVW+OOi9A1Lj+5NVQfNBDHoez+p4E1bpJfN9xyoL/Bb/T4WPBLDgKf/iNzSTV65WKkBPUf1DGG5OTdm/NJQPlQjif/Ku64yvraILmL9BI+/1NWZS38/ZROZdX/HfLlWRxZxf4PEb/3FBqvducA+r8cj4QBWszJzmjPUi38y6IPpB2TdQ592QOzNpyz1i0S8odZ1Ik4xystRL+DSF6Z74Uv2RYxp3O4FP8agGw2gLd4Hjz9fsx0LAHqxZeQCqxJgEYEt3xYnN1W01EhTL7IS2JdHULMDbnJw4GZd+Vaj5XCvw3Ec0KCWe/V8WhCdeAkNsqpFZ8D1ptuKV5dcRxuaImIH3HrGqJVmzDo453PGry5AqVZTtStFs+Lf8DYXAb8syrM4EJ7pIM6+ojsfxKHJnbrmPmb/O+g9v870LS75eoCCunEmwy18TA4KczcJiRYVIWU3RlO+ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Overview: Low overhead [1] per-callsite memory allocation profiling. Not just for debug kernels, overhead low enough to be deployed in production. Example output: root@moria-kvm:~# sort -rn /proc/allocinfo 127664128 31168 mm/page_ext.c:270 func:alloc_page_ext 56373248 4737 mm/slub.c:2259 func:alloc_slab_page 14880768 3633 mm/readahead.c:247 func:page_cache_ra_unbounded 14417920 3520 mm/mm_init.c:2530 func:alloc_large_system_hash 13377536 234 block/blk-mq.c:3421 func:blk_mq_alloc_rqs 11718656 2861 mm/filemap.c:1919 func:__filemap_get_folio 9192960 2800 kernel/fork.c:307 func:alloc_thread_stack_node 4206592 4 net/netfilter/nf_conntrack_core.c:2567 func:nf_ct_alloc_hashtable 4136960 1010 drivers/staging/ctagmod/ctagmod.c:20 [ctagmod] func:ctagmod_start 3940352 962 mm/memory.c:4214 func:alloc_anon_folio 2894464 22613 fs/kernfs/dir.c:615 func:__kernfs_new_node ... Since v3: - Dropped patch changing string_get_size() [2] as not needed - Dropped patch modifying xfs allocators [3] as non needed, per Dave Chinner - Added Reviewed-by, per Kees Cook - Moved prepare_slab_obj_exts_hook() and alloc_slab_obj_exts() where they are used, per Vlastimil Babka - Fixed SLAB_NO_OBJ_EXT definition to use unused bit, per Vlastimil Babka - Refactored patch [4] into other patches, per Vlastimil Babka - Replaced snprintf() with seq_buf_printf(), per Kees Cook - Changed output to report bytes, per Andrew Morton and Pasha Tatashin - Changed output to report [module] only for loadable modules, per Vlastimil Babka - Moved mem_alloc_profiling_enabled() check earlier, per Vlastimil Babka - Changed the code to handle page splitting to be more understandable, per Vlastimil Babka - Moved alloc_tagging_slab_free_hook(), mark_objexts_empty(), mark_failed_objexts_alloc() and handle_failed_objexts_alloc(), per Vlastimil Babka - Fixed loss of __alloc_size(1, 2) in kvmalloc functions, per Vlastimil Babka - Refactored the code in show_mem() to avoid memory allocations, per Michal Hocko - Changed to trylock in show_mem() to avoid blocking in atomic context, per Tetsuo Handa - Added mm mailing list into MAINTAINERS, per Kees Cook - Added base commit SHA, per Andy Shevchenko - Added a patch with documentation, per Jani Nikula - Fixed 0day bugs - Added benchmark results [5], per Steven Rostedt - Rebased over Linux 6.8-rc5 Items not yet addressed: - An early_boot option to prevent pageext overhead. We are looking into ways for using the same sysctr instead of adding additional early boot parameter. Usage: kconfig options: - CONFIG_MEM_ALLOC_PROFILING - CONFIG_MEM_ALLOC_PROFILING_ENABLED_BY_DEFAULT - CONFIG_MEM_ALLOC_PROFILING_DEBUG adds warnings for allocations that weren't accounted because of a missing annotation sysctl: /proc/sys/vm/mem_profiling Runtime info: /proc/allocinfo Notes: [1]: Overhead To measure the overhead we are comparing the following configurations: (1) Baseline with CONFIG_MEMCG_KMEM=n (2) Disabled by default (CONFIG_MEM_ALLOC_PROFILING=y && CONFIG_MEM_ALLOC_PROFILING_BY_DEFAULT=n) (3) Enabled by default (CONFIG_MEM_ALLOC_PROFILING=y && CONFIG_MEM_ALLOC_PROFILING_BY_DEFAULT=y) (4) Enabled at runtime (CONFIG_MEM_ALLOC_PROFILING=y && CONFIG_MEM_ALLOC_PROFILING_BY_DEFAULT=n && /proc/sys/vm/mem_profiling=1) (5) Baseline with CONFIG_MEMCG_KMEM=y && allocating with __GFP_ACCOUNT (6) Disabled by default (CONFIG_MEM_ALLOC_PROFILING=y && CONFIG_MEM_ALLOC_PROFILING_BY_DEFAULT=n) && CONFIG_MEMCG_KMEM=y (7) Enabled by default (CONFIG_MEM_ALLOC_PROFILING=y && CONFIG_MEM_ALLOC_PROFILING_BY_DEFAULT=y) && CONFIG_MEMCG_KMEM=y Performance overhead: To evaluate performance we implemented an in-kernel test executing multiple get_free_page/free_page and kmalloc/kfree calls with allocation sizes growing from 8 to 240 bytes with CPU frequency set to max and CPU affinity set to a specific CPU to minimize the noise. Below are results from running the test on Ubuntu 22.04.2 LTS with 6.8.0-rc1 kernel on 56 core Intel Xeon: kmalloc pgalloc (1 baseline) 6.764s 16.902s (2 default disabled) 6.793s (+0.43%) 17.007s (+0.62%) (3 default enabled) 7.197s (+6.40%) 23.666s (+40.02%) (4 runtime enabled) 7.405s (+9.48%) 23.901s (+41.41%) (5 memcg) 13.388s (+97.94%) 48.460s (+186.71%) (6 def disabled+memcg) 13.332s (+97.10%) 48.105s (+184.61%) (7 def enabled+memcg) 13.446s (+98.78%) 54.963s (+225.18%) Memory overhead: Kernel size: text data bss dec diff (1) 26515311 18890222 17018880 62424413 (2) 26524728 19423818 16740352 62688898 264485 (3) 26524724 19423818 16740352 62688894 264481 (4) 26524728 19423818 16740352 62688898 264485 (5) 26541782 18964374 16957440 62463596 39183 Memory consumption on a 56 core Intel CPU with 125GB of memory: Code tags: 192 kB PageExts: 262144 kB (256MB) SlabExts: 9876 kB (9.6MB) PcpuExts: 512 kB (0.5MB) Total overhead is 0.2% of total memory. [2] https://lore.kernel.org/all/20240212213922.783301-2-surenb@google.com/ [3] https://lore.kernel.org/all/20240212213922.783301-26-surenb@google.com/ [4] https://lore.kernel.org/all/20240212213922.783301-9-surenb@google.com/ [5] Benchmarks: Hackbench tests run 100 times: hackbench -s 512 -l 200 -g 15 -f 25 -P baseline disabled profiling enabled profiling avg 0.3543 0.3559 (+0.0016) 0.3566 (+0.0023) stdev 0.0137 0.0188 0.0077 hackbench -l 10000 baseline disabled profiling enabled profiling avg 6.4218 6.4306 (+0.0088) 6.5077 (+0.0859) stdev 0.0933 0.0286 0.0489 stress-ng tests: stress-ng --class memory --seq 4 -t 60 stress-ng --class cpu --seq 4 -t 60 Results posted at: https://evilpiepirate.org/~kent/memalloc_prof_v4_stress-ng/ Kent Overstreet (13): fix missing vmalloc.h includes asm-generic/io.h: Kill vmalloc.h dependency mm/slub: Mark slab_free_freelist_hook() __always_inline scripts/kallysms: Always include __start and __stop symbols fs: Convert alloc_inode_sb() to a macro rust: Add a rust helper for krealloc() mempool: Hook up to memory allocation profiling mm: percpu: Introduce pcpuobj_ext mm: percpu: Add codetag reference into pcpuobj_ext mm: vmalloc: Enable memory allocation profiling rhashtable: Plumb through alloc tag MAINTAINERS: Add entries for code tagging and memory allocation profiling memprofiling: Documentation Suren Baghdasaryan (23): mm: enumerate all gfp flags mm: introduce slabobj_ext to support slab object extensions mm: introduce __GFP_NO_OBJ_EXT flag to selectively prevent slabobj_ext creation mm/slab: introduce SLAB_NO_OBJ_EXT to avoid obj_ext creation slab: objext: introduce objext_flags as extension to page_memcg_data_flags lib: code tagging framework lib: code tagging module support lib: prevent module unloading if memory is not freed lib: add allocation tagging support for memory allocation profiling lib: introduce support for page allocation tagging mm: percpu: increase PERCPU_MODULE_RESERVE to accommodate allocation tags change alloc_pages name in dma_map_ops to avoid name conflicts mm: enable page allocation tagging mm: create new codetag references during page splitting mm/page_ext: enable early_page_ext when CONFIG_MEM_ALLOC_PROFILING_DEBUG=y lib: add codetag reference into slabobj_ext mm/slab: add allocation accounting into slab allocation and free paths mm/slab: enable slab allocation tagging for kmalloc and friends mm: percpu: enable per-cpu allocation tagging lib: add memory allocations report in show_mem() codetag: debug: skip objext checking when it's for objext itself codetag: debug: mark codetags for reserved pages as empty codetag: debug: introduce OBJEXTS_ALLOC_FAIL to mark failed slab_ext allocations Documentation/admin-guide/sysctl/vm.rst | 16 + Documentation/filesystems/proc.rst | 29 ++ Documentation/mm/allocation-profiling.rst | 86 ++++++ MAINTAINERS | 17 ++ arch/alpha/kernel/pci_iommu.c | 2 +- arch/alpha/lib/checksum.c | 1 + arch/alpha/lib/fpreg.c | 1 + arch/alpha/lib/memcpy.c | 1 + arch/arm/kernel/irq.c | 1 + arch/arm/kernel/traps.c | 1 + arch/arm64/kernel/efi.c | 1 + arch/loongarch/include/asm/kfence.h | 1 + arch/mips/jazz/jazzdma.c | 2 +- arch/powerpc/kernel/dma-iommu.c | 2 +- arch/powerpc/kernel/iommu.c | 1 + arch/powerpc/mm/mem.c | 1 + arch/powerpc/platforms/ps3/system-bus.c | 4 +- arch/powerpc/platforms/pseries/vio.c | 2 +- arch/riscv/kernel/elf_kexec.c | 1 + arch/riscv/kernel/probes/kprobes.c | 1 + arch/s390/kernel/cert_store.c | 1 + arch/s390/kernel/ipl.c | 1 + arch/x86/include/asm/io.h | 1 + arch/x86/kernel/amd_gart_64.c | 2 +- arch/x86/kernel/cpu/sgx/main.c | 1 + arch/x86/kernel/irq_64.c | 1 + arch/x86/mm/fault.c | 1 + drivers/accel/ivpu/ivpu_mmu_context.c | 1 + drivers/gpu/drm/gma500/mmu.c | 1 + drivers/gpu/drm/i915/gem/i915_gem_pages.c | 1 + .../gpu/drm/i915/gem/selftests/mock_dmabuf.c | 1 + drivers/gpu/drm/i915/gt/shmem_utils.c | 1 + drivers/gpu/drm/i915/gvt/firmware.c | 1 + drivers/gpu/drm/i915/gvt/gtt.c | 1 + drivers/gpu/drm/i915/gvt/handlers.c | 1 + drivers/gpu/drm/i915/gvt/mmio.c | 1 + drivers/gpu/drm/i915/gvt/vgpu.c | 1 + drivers/gpu/drm/i915/intel_gvt.c | 1 + drivers/gpu/drm/imagination/pvr_vm_mips.c | 1 + drivers/gpu/drm/mediatek/mtk_drm_gem.c | 1 + drivers/gpu/drm/omapdrm/omap_gem.c | 1 + drivers/gpu/drm/v3d/v3d_bo.c | 1 + drivers/gpu/drm/vmwgfx/vmwgfx_binding.c | 1 + drivers/gpu/drm/vmwgfx/vmwgfx_cmd.c | 1 + drivers/gpu/drm/vmwgfx/vmwgfx_devcaps.c | 1 + drivers/gpu/drm/vmwgfx/vmwgfx_drv.c | 1 + drivers/gpu/drm/vmwgfx/vmwgfx_execbuf.c | 1 + drivers/gpu/drm/vmwgfx/vmwgfx_ioctl.c | 1 + drivers/gpu/drm/xen/xen_drm_front_gem.c | 1 + drivers/hwtracing/coresight/coresight-trbe.c | 1 + drivers/iommu/dma-iommu.c | 2 +- .../marvell/octeon_ep/octep_pfvf_mbox.c | 1 + .../net/ethernet/microsoft/mana/hw_channel.c | 1 + drivers/parisc/ccio-dma.c | 2 +- drivers/parisc/sba_iommu.c | 2 +- drivers/platform/x86/uv_sysfs.c | 1 + drivers/scsi/mpi3mr/mpi3mr_transport.c | 2 + drivers/staging/media/atomisp/pci/hmm/hmm.c | 2 +- drivers/vfio/pci/pds/dirty.c | 1 + drivers/virt/acrn/mm.c | 1 + drivers/virtio/virtio_mem.c | 1 + drivers/xen/grant-dma-ops.c | 2 +- drivers/xen/swiotlb-xen.c | 2 +- include/asm-generic/codetag.lds.h | 14 + include/asm-generic/io.h | 1 - include/asm-generic/vmlinux.lds.h | 3 + include/linux/alloc_tag.h | 195 ++++++++++++ include/linux/codetag.h | 81 +++++ include/linux/dma-map-ops.h | 2 +- include/linux/fortify-string.h | 5 +- include/linux/fs.h | 6 +- include/linux/gfp.h | 126 +++++--- include/linux/gfp_types.h | 101 +++++-- include/linux/memcontrol.h | 56 +++- include/linux/mempool.h | 73 +++-- include/linux/mm.h | 9 + include/linux/mm_types.h | 4 +- include/linux/page_ext.h | 1 - include/linux/pagemap.h | 9 +- include/linux/pds/pds_common.h | 2 + include/linux/percpu.h | 27 +- include/linux/pgalloc_tag.h | 110 +++++++ include/linux/rhashtable-types.h | 11 +- include/linux/sched.h | 24 ++ include/linux/slab.h | 175 +++++------ include/linux/string.h | 4 +- include/linux/vmalloc.h | 60 +++- include/rdma/rdmavt_qp.h | 1 + init/Kconfig | 4 + kernel/dma/mapping.c | 4 +- kernel/kallsyms_selftest.c | 2 +- kernel/module/main.c | 25 +- lib/Kconfig.debug | 31 ++ lib/Makefile | 3 + lib/alloc_tag.c | 204 +++++++++++++ lib/codetag.c | 283 ++++++++++++++++++ lib/rhashtable.c | 28 +- mm/compaction.c | 7 +- mm/debug_vm_pgtable.c | 1 + mm/filemap.c | 6 +- mm/huge_memory.c | 2 + mm/kfence/core.c | 14 +- mm/kfence/kfence.h | 4 +- mm/memcontrol.c | 56 +--- mm/mempolicy.c | 52 ++-- mm/mempool.c | 36 +-- mm/mm_init.c | 13 +- mm/nommu.c | 64 ++-- mm/page_alloc.c | 66 ++-- mm/page_ext.c | 13 + mm/page_owner.c | 2 +- mm/percpu-internal.h | 26 +- mm/percpu.c | 120 +++----- mm/show_mem.c | 26 ++ mm/slab.h | 126 ++++++-- mm/slab_common.c | 6 +- mm/slub.c | 244 +++++++++++---- mm/util.c | 44 +-- mm/vmalloc.c | 88 +++--- rust/helpers.c | 8 + scripts/kallsyms.c | 13 + scripts/module.lds.S | 7 + sound/pci/hda/cs35l41_hda.c | 1 + 123 files changed, 2269 insertions(+), 682 deletions(-) create mode 100644 Documentation/mm/allocation-profiling.rst create mode 100644 include/asm-generic/codetag.lds.h create mode 100644 include/linux/alloc_tag.h create mode 100644 include/linux/codetag.h create mode 100644 include/linux/pgalloc_tag.h create mode 100644 lib/alloc_tag.c create mode 100644 lib/codetag.c base-commit: 39133352cbed6626956d38ed72012f49b0421e7b