From patchwork Mon May 1 16:54:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suren Baghdasaryan X-Patchwork-Id: 13227646 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BA6FFC77B7C for ; Mon, 1 May 2023 16:55:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 30BE090000D; Mon, 1 May 2023 12:55:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2958E900002; Mon, 1 May 2023 12:55:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1351A90000D; Mon, 1 May 2023 12:55:34 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id F299D900002 for ; Mon, 1 May 2023 12:55:33 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id D3192407A1 for ; Mon, 1 May 2023 16:55:33 +0000 (UTC) X-FDA: 80742287346.05.65E2146 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) by imf10.hostedemail.com (Postfix) with ESMTP id 0FE26C0022 for ; Mon, 1 May 2023 16:55:31 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=MVmPTJvn; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf10.hostedemail.com: domain of 3A-9PZAYKCEo463qzns00sxq.o0yxuz69-yyw7mow.03s@flex--surenb.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=3A-9PZAYKCEo463qzns00sxq.o0yxuz69-yyw7mow.03s@flex--surenb.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1682960132; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=gDL+lQo0P3G6fblFMKJonaRbqsrkB9ljveMgEFO8c80=; b=oDqCHryuebUkFs1Jey+RoFDW+l1ZL4+zN0oHawf34XlaFzUBEIFm+6dxG6La5Us3o2X+NT oBHDvgeDmma05/83VM/yti33zfNYbm/MNaCC0szj6d2029hsv7jUXk/2w4vEfjSRwffLFi iXFeMI+TsNQxAJgoTHXbAKSGCcsaM6U= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=MVmPTJvn; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf10.hostedemail.com: domain of 3A-9PZAYKCEo463qzns00sxq.o0yxuz69-yyw7mow.03s@flex--surenb.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=3A-9PZAYKCEo463qzns00sxq.o0yxuz69-yyw7mow.03s@flex--surenb.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1682960132; a=rsa-sha256; cv=none; b=y5Rm7IF5CWgBVmDG5/SHzOMGnf/h/XsydMoECjcj96t68OSjvY6PEOuMaa3vIhSpw5l5tV w0UGI//VGFXvORkWe4OFpWzWf/2Zofir0q+FMAU6QaEsWYXKhUNoCN7xmcuux9tcDJjWlB eGWvAbGq1q3lOFifn/KOvBB/ALpF9uo= Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-b9a8023ccf1so5355605276.2 for ; Mon, 01 May 2023 09:55:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1682960131; x=1685552131; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=gDL+lQo0P3G6fblFMKJonaRbqsrkB9ljveMgEFO8c80=; b=MVmPTJvnt/d4OHnObDpLo5Yx/m0JxigNMHefGJUQ6YHCHiKWwXI+TPB1lVnlbOYOEt 5jgACI5oKYMCB7Pq7cz3X81+bpZcuEUKDUynk+/ZyYCZwjUk5TCfCarGFU4MMvcurlND oln4td3qiCopzaVDD/Q4rMLFlXyQJx4wDKAU0Wzgmky/qZcwJ14zhRRSwMOBlU9Kez/h x9gFamkwY5h9zdy2v/nUepQsOsBIbNt92VQg8iIvlxGC803m7Adm/vEjtGzf8sBaILXc DjUHk8NLlqyBasCJKVPDW64fh5SwGVhMkreauSlsOSNHTVu6NR6T/SSX4RnGRR6l1W62 EEqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682960131; x=1685552131; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=gDL+lQo0P3G6fblFMKJonaRbqsrkB9ljveMgEFO8c80=; b=Jy7D0St90Gz30qAq4+ahEYoeKBedelL7lGoqCj5GjISWv9Rg55+OTfmxdLO9JrnKt7 p6KREhtNu94wR1SclWD5OciBTOjKdcRolXLPJ1vJ4KpH8hRgKRKqC1BQxe52VIGrdzVe 0vCpv9KWoqBCrbcKJW1j5eUtVHouhxQ1QM7Y2IrC0cp8yNCp/Rs/72/0KMw5qY0FhAeb LnN11t3ik7lsTfPI4Ra0+33SoJcAlTKNngUfM/tHFbQM7VvxfgG1NT9GCI96d5DgaspW 69jENx9o/HyCqmja7uxs9ZBnH3WUpZRzJF75qs8T4DzJ69X/O4Iyg+AUoa3Jtu4vV/KU QsQg== X-Gm-Message-State: AC+VfDysKKlwvpsrxhGxD4AqZb9J0b4ldYLWfgaVx1iK1l57wauTL/rz XzNRLieAIJT1X9rYbTjvVZrOQC0Tp4g= X-Google-Smtp-Source: ACHHUZ6enP7kBREmhiQ0i0LYDuH3RuGEJuyGpBzYXGRy3FeK34ct4OWkfjl1R+bRIQUodsczhvp1FyVsBhk= X-Received: from surenb-desktop.mtv.corp.google.com ([2620:15c:211:201:6d24:3efd:facc:7ac4]) (user=surenb job=sendgmr) by 2002:a25:6b05:0:b0:b8b:f5fb:5986 with SMTP id g5-20020a256b05000000b00b8bf5fb5986mr8475612ybc.10.1682960131180; Mon, 01 May 2023 09:55:31 -0700 (PDT) Date: Mon, 1 May 2023 09:54:20 -0700 In-Reply-To: <20230501165450.15352-1-surenb@google.com> Mime-Version: 1.0 References: <20230501165450.15352-1-surenb@google.com> X-Mailer: git-send-email 2.40.1.495.gc816e09b53d-goog Message-ID: <20230501165450.15352-11-surenb@google.com> Subject: [PATCH 10/40] mm/slab: introduce SLAB_NO_OBJ_EXT to avoid obj_ext creation From: Suren Baghdasaryan To: akpm@linux-foundation.org Cc: kent.overstreet@linux.dev, mhocko@suse.com, vbabka@suse.cz, hannes@cmpxchg.org, roman.gushchin@linux.dev, mgorman@suse.de, dave@stgolabs.net, willy@infradead.org, liam.howlett@oracle.com, corbet@lwn.net, void@manifault.com, peterz@infradead.org, juri.lelli@redhat.com, ldufour@linux.ibm.com, catalin.marinas@arm.com, will@kernel.org, arnd@arndb.de, tglx@linutronix.de, mingo@redhat.com, dave.hansen@linux.intel.com, x86@kernel.org, peterx@redhat.com, david@redhat.com, axboe@kernel.dk, mcgrof@kernel.org, masahiroy@kernel.org, nathan@kernel.org, dennis@kernel.org, tj@kernel.org, muchun.song@linux.dev, rppt@kernel.org, paulmck@kernel.org, pasha.tatashin@soleen.com, yosryahmed@google.com, yuzhao@google.com, dhowells@redhat.com, hughd@google.com, andreyknvl@gmail.com, keescook@chromium.org, ndesaulniers@google.com, gregkh@linuxfoundation.org, ebiggers@google.com, ytcoode@gmail.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, bristot@redhat.com, vschneid@redhat.com, cl@linux.com, penberg@kernel.org, iamjoonsoo.kim@lge.com, 42.hyeyoo@gmail.com, glider@google.com, elver@google.com, dvyukov@google.com, shakeelb@google.com, songmuchun@bytedance.com, jbaron@akamai.com, rientjes@google.com, minchan@google.com, kaleshsingh@google.com, surenb@google.com, kernel-team@android.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev, linux-arch@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-modules@vger.kernel.org, kasan-dev@googlegroups.com, cgroups@vger.kernel.org X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 0FE26C0022 X-Stat-Signature: m7g88oq4cbyuwbs7mp1oai1yeokrtthi X-HE-Tag: 1682960131-314032 X-HE-Meta: U2FsdGVkX1/Wq7GrxCiHtHwhqk11KerTLoJxxD6liOv4BznchZmovtnZQL+Fd+TsOlHlkmZOLwcVWyK7FVz+dTPP3Bg3LzD60BEfyjZPk6bLRQNRv2YBLDURH/arYkseH+6zK4Oz2lD0pZsz8g837b6v0oogj7gMk65AIBU7yojqq28DEVrM+sBnHS51cVfkxD8+6kQy0dc9keXPL7U1CFsB3hhYQU10o/lFnCI2RuNhMUWCns5YFr0g6uD7ip7klIVzAz1n/RFBRYUpKiYJ+fPQbf/mEL1N1dhjNLapz1h5A5Vy4Xu7v5Fn8ICgHwQEDswHv0QPrTU3Y6ThTGgb8GzeoFTzf8uexpo6VFpwYvTGcKVTr9xoZQUXXyUdgIzXOaY4iVPq9LvyB8jDvjIFpWvJXnGnKUM33NTwc0h4JPhANZ44w0NDxUGWz3ialnmQrDPca096kbNa7d1F2lfHHl3KOeIZy5FsfIrvLp/5622MBr0DGrGrXVBzWWOKkOJF1UF68XMZJbV2n4wOnrMx+0X3ly5Ic9cbz6pUIAvcnzrZhNoEeGLyMh6VtVSrgIhyJjSasaZ48F9RjfX+Zvz2gRAMPGSSFxoijTutL0kqK8jA96kTUClC6SoO5GrzLV+vHxjQ1r0Rt7eXp+iP7NBkFGEgEuh/q4yjyRY9KbkVS74v7f80jbeAW6bPPMTVDfo5Z7lwsN4ryYnGsCTkx6gt3iz28wxI3Ga0Gnzq2Ztb0DKnrmFlaGvunLML7Z1klucbetFN3YN9AvKwmtaQ9mzlmYzozskKBqMNMPVgRf9mQsdI12J/gm6HhmsxeWzOv7DdMeuoCz7tB128O4Kq39T5dLkjDkyKfXsKqge4zvWFIZHkOS2PWotRdVkB435maExCcFHGLjObjmihfqyfQzQnCMd+ItMxbVhelY172URo25AxqVk+w22a/zNyP4rrVNcJ1lc99upsxW8SfLaJSzn 5mprrZxz 1sAnubUj3PDmed4dox8en6XIo6DPegaqIXnJmEtu5qCvUOiTyb1x9N+XfpwHIU+Ky4ZiZH5CyDI9MqMg2XVOahQhZuR96of+bWXFbVElIbEhDiDCSOcFcI/ti6dMECbFhgEy/lcCQmOClLb68aNPtRvVWr5/+4rHdTcgu+pVSBMPrNE7Y26jF6l+xtWBp7YUdfU1d4GmVzQlx3ph60AOMMbmrnbZnxXxINtw9Mjam+UlU1ulZ0ideXxLtC6lok8vbxUpH3REG/vKRPs5qZ2tl6WAzNexQx2OaUDCibVqwbzQrPe9suZ/KiFBAANot9Otayxq3uJbNc9Bm6syoHieqcca+6RjUGZHQuB3AMIfIsV3nSKhyllHvmEZDxaRu+YAfpaNadmPou5/sWgcnMkqS2DD9DLkoT0fCxcP/XqXbeqRkVPaxxExb652k9zzfkmPanj34aS8UjiouCtBRtf2pEppaTWkKc9xFbzYmCbNS/VlB8YGWUmei8bmnu6EI7kf7PPJRDVKbpu3Wrf845ZACakBx1OFotlID4Ii1Le3xOzVYb6xxJiXJuYS8cqrwUArC15BcGKyeed3chsTlxrsK7gVDjuuSS/kzY88h+4PqnDqB4ZAlI5JXbsxQaPkzTS6GhoajMN7uDkXqssotm+MquPJjeUVrltUEbNZaPbg+qRqAIbXwgyJGR9tm5IcDhpgjMEJbkTk7jdbuWxfyfOrEjZwgRUALcKPpgavG X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Slab extension objects can't be allocated before slab infrastructure is initialized. Some caches, like kmem_cache and kmem_cache_node, are created before slab infrastructure is initialized. Objects from these caches can't have extension objects. Introduce SLAB_NO_OBJ_EXT slab flag to mark these caches and avoid creating extensions for objects allocated from these slabs. Signed-off-by: Suren Baghdasaryan --- include/linux/slab.h | 7 +++++++ mm/slab.c | 2 +- mm/slub.c | 5 +++-- 3 files changed, 11 insertions(+), 3 deletions(-) diff --git a/include/linux/slab.h b/include/linux/slab.h index 6b3e155b70bf..99a146f3cedf 100644 --- a/include/linux/slab.h +++ b/include/linux/slab.h @@ -147,6 +147,13 @@ #endif #define SLAB_TEMPORARY SLAB_RECLAIM_ACCOUNT /* Objects are short-lived */ +#ifdef CONFIG_SLAB_OBJ_EXT +/* Slab created using create_boot_cache */ +#define SLAB_NO_OBJ_EXT ((slab_flags_t __force)0x20000000U) +#else +#define SLAB_NO_OBJ_EXT 0 +#endif + /* * ZERO_SIZE_PTR will be returned for zero sized kmalloc requests. * diff --git a/mm/slab.c b/mm/slab.c index bb57f7fdbae1..ccc76f7455e9 100644 --- a/mm/slab.c +++ b/mm/slab.c @@ -1232,7 +1232,7 @@ void __init kmem_cache_init(void) create_boot_cache(kmem_cache, "kmem_cache", offsetof(struct kmem_cache, node) + nr_node_ids * sizeof(struct kmem_cache_node *), - SLAB_HWCACHE_ALIGN, 0, 0); + SLAB_HWCACHE_ALIGN | SLAB_NO_OBJ_EXT, 0, 0); list_add(&kmem_cache->list, &slab_caches); slab_state = PARTIAL; diff --git a/mm/slub.c b/mm/slub.c index c87628cd8a9a..507b71372ee4 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -5020,7 +5020,8 @@ void __init kmem_cache_init(void) node_set(node, slab_nodes); create_boot_cache(kmem_cache_node, "kmem_cache_node", - sizeof(struct kmem_cache_node), SLAB_HWCACHE_ALIGN, 0, 0); + sizeof(struct kmem_cache_node), + SLAB_HWCACHE_ALIGN | SLAB_NO_OBJ_EXT, 0, 0); hotplug_memory_notifier(slab_memory_callback, SLAB_CALLBACK_PRI); @@ -5030,7 +5031,7 @@ void __init kmem_cache_init(void) create_boot_cache(kmem_cache, "kmem_cache", offsetof(struct kmem_cache, node) + nr_node_ids * sizeof(struct kmem_cache_node *), - SLAB_HWCACHE_ALIGN, 0, 0); + SLAB_HWCACHE_ALIGN | SLAB_NO_OBJ_EXT, 0, 0); kmem_cache = bootstrap(&boot_kmem_cache); kmem_cache_node = bootstrap(&boot_kmem_cache_node);