From patchwork Mon Feb 12 21:38:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suren Baghdasaryan X-Patchwork-Id: 13554045 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BACD5C4829D for ; Mon, 12 Feb 2024 21:39:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3F4946B0085; Mon, 12 Feb 2024 16:39:51 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3A3426B0087; Mon, 12 Feb 2024 16:39:51 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1F6056B0088; Mon, 12 Feb 2024 16:39:51 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 0309B6B0085 for ; Mon, 12 Feb 2024 16:39:51 -0500 (EST) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 8A91FA20A8 for ; Mon, 12 Feb 2024 21:39:50 +0000 (UTC) X-FDA: 81784469340.22.D5CEA20 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) by imf14.hostedemail.com (Postfix) with ESMTP id A5DD3100013 for ; Mon, 12 Feb 2024 21:39:48 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=vjqeCmmn; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf14.hostedemail.com: domain of 3I5DKZQYKCKIUWTGPDIQQING.EQONKPWZ-OOMXCEM.QTI@flex--surenb.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=3I5DKZQYKCKIUWTGPDIQQING.EQONKPWZ-OOMXCEM.QTI@flex--surenb.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1707773988; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=QPRNF15NS/yHs+nflQ9JCJcKpeGLPrq2C4Bm4JvXiOY=; b=WWXigp21vMkGMcYnhax+UgZJ7CblVsNpRHuZe+m+lQIl3T7fKqpg8S6Xnmn6s5RSbZsAOz RLLyPXZ4vROUGn3uEcdWSV4cNoLVU7CeE6y/ViNi3UQVuSrlfs7iQqEfYeOZDXEMbYWW5E 7oy1cdJPQmxt3KszlSuI8XWmgtHQo5s= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=vjqeCmmn; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf14.hostedemail.com: domain of 3I5DKZQYKCKIUWTGPDIQQING.EQONKPWZ-OOMXCEM.QTI@flex--surenb.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=3I5DKZQYKCKIUWTGPDIQQING.EQONKPWZ-OOMXCEM.QTI@flex--surenb.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1707773988; a=rsa-sha256; cv=none; b=1Cu2jGfaXxzDllMoTMdTUPX9y7Jl1yHHo2usLJDyx2SR8YJbKwgf3yPZsICmJlMs71LuEQ 9DoRp51ouEj5P5oNts8ZpYE3IWvilBDzTsRgstigxsf8vrmdHBALIi8prbV2JV7fJI+Fja 6CTutunOR7zJAmUHW7Rcp6nJoLMNH8k= Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-604ac3c75ebso68374597b3.0 for ; Mon, 12 Feb 2024 13:39:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1707773988; x=1708378788; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=QPRNF15NS/yHs+nflQ9JCJcKpeGLPrq2C4Bm4JvXiOY=; b=vjqeCmmn7AGhsMkhYehoZGKIwMwOBY1bfDV+6bIbzIuXJi1Vuy1/ZCQmvam5y1ZBEm 8dtLnDwo6m8y2UScswE9NGJpHCFQ2+kHK6Wxsjy8C1hJvBdruxD8MVYCBXbBtJpcS39w jm/6NrxhVpDdJxr6mNWD+yhNxuOF/iOE55NLUgDBvDZdcs4G0EAT5P6Ti59F4b46FDPO CMXGuerwm0CLokWf43PJu6dugzofqhhsufmShwXozu8IkS04x6fBSF6SjnW4cva89uio JMY+yqH5vasacP9Vmk5p37EB8ffONvBhAhY5tCyfj4Zj7XCTO9iScPdUd0xBxxNbzxsC WGEw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707773988; x=1708378788; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=QPRNF15NS/yHs+nflQ9JCJcKpeGLPrq2C4Bm4JvXiOY=; b=Bd6zHMLoxLViKCLjdSrxR0DvrhaJB7o712AVcULFQ6sN51yBBx708Kl4yuBatYVw3U wja5ZgkVg0zLCCImC0QBWACgYZgi5ludUiaD+QUqM93AI88u47yTsW1rekktD/Vrlzk8 +rwTXxVrfkl6jMjTAAx4pHezKjDnjj9NAb80Ew6QQe1u1Q08HH5mevCmJk3xUIuE19mv NkpwM05m3c9ZbKYe92fEOjKHFAHEnogi0OdSizKsbWxbkB+g/Xpl4OvIXS431ErIoI/F 7qNYe+B5ycM63f9G01fa3TWd+Rd1jZgR/M+9445y1hiXM/DfEoasBfyowo+cHD0RKJlG 68Xw== X-Forwarded-Encrypted: i=1; AJvYcCXtHd/XvHnlqy9BJR0/cul/7REQhrMfPUSEqXix5URD+HF5vtpFxqwZ+LzqiC11Zsh4jV9yzmj/P1kltU0Yjv3r1Sc= X-Gm-Message-State: AOJu0Ywn556+c1kk4iwwKZd4JAz7dnNDQuWDSmPHMEuP90wHJgfdXHMx c8mwlXGNboLu1hTNMeC0BK4CCQZJDxWf6L6AwklK1Ofzqc0IGB4KeIQBhHDr1+KZE5vm6uY+GID 55A== X-Google-Smtp-Source: AGHT+IFIMBcV3x/W8fhRczbwea2w1q1gcT3ETfM1GWq+zDEkeIxrI9ytOpFdbEX8EDgRr+EtsMkcZCh6eRk= X-Received: from surenb-desktop.mtv.corp.google.com ([2620:15c:211:201:b848:2b3f:be49:9cbc]) (user=surenb job=sendgmr) by 2002:a25:53c3:0:b0:dcb:c4d3:6e07 with SMTP id h186-20020a2553c3000000b00dcbc4d36e07mr382686ybb.5.1707773987608; Mon, 12 Feb 2024 13:39:47 -0800 (PST) Date: Mon, 12 Feb 2024 13:38:53 -0800 In-Reply-To: <20240212213922.783301-1-surenb@google.com> Mime-Version: 1.0 References: <20240212213922.783301-1-surenb@google.com> X-Mailer: git-send-email 2.43.0.687.g38aa6559b0-goog Message-ID: <20240212213922.783301-8-surenb@google.com> Subject: [PATCH v3 07/35] mm/slab: introduce SLAB_NO_OBJ_EXT to avoid obj_ext creation From: Suren Baghdasaryan To: akpm@linux-foundation.org Cc: kent.overstreet@linux.dev, mhocko@suse.com, vbabka@suse.cz, hannes@cmpxchg.org, roman.gushchin@linux.dev, mgorman@suse.de, dave@stgolabs.net, willy@infradead.org, liam.howlett@oracle.com, corbet@lwn.net, void@manifault.com, peterz@infradead.org, juri.lelli@redhat.com, catalin.marinas@arm.com, will@kernel.org, arnd@arndb.de, tglx@linutronix.de, mingo@redhat.com, dave.hansen@linux.intel.com, x86@kernel.org, peterx@redhat.com, david@redhat.com, axboe@kernel.dk, mcgrof@kernel.org, masahiroy@kernel.org, nathan@kernel.org, dennis@kernel.org, tj@kernel.org, muchun.song@linux.dev, rppt@kernel.org, paulmck@kernel.org, pasha.tatashin@soleen.com, yosryahmed@google.com, yuzhao@google.com, dhowells@redhat.com, hughd@google.com, andreyknvl@gmail.com, keescook@chromium.org, ndesaulniers@google.com, vvvvvv@google.com, gregkh@linuxfoundation.org, ebiggers@google.com, ytcoode@gmail.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, bristot@redhat.com, vschneid@redhat.com, cl@linux.com, penberg@kernel.org, iamjoonsoo.kim@lge.com, 42.hyeyoo@gmail.com, glider@google.com, elver@google.com, dvyukov@google.com, shakeelb@google.com, songmuchun@bytedance.com, jbaron@akamai.com, rientjes@google.com, minchan@google.com, kaleshsingh@google.com, surenb@google.com, kernel-team@android.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev, linux-arch@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-modules@vger.kernel.org, kasan-dev@googlegroups.com, cgroups@vger.kernel.org X-Rspamd-Queue-Id: A5DD3100013 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: gweq1y1it9oz55kyo6hg9t68hi5ni7o4 X-HE-Tag: 1707773988-389844 X-HE-Meta: U2FsdGVkX1/aH6IAJJ0PP45iLsQ512N/SfU/cZoMAROIRZEvk050xWLe7jzNCOWxxHAiADrAd1HvPAMc+603WoHuNmtTJbQDPw6lSlNq8rl267mcyEBmYDLb1VdvT8y1BvkCb0RSC2cdiuCguOKH5IQqV98gNQqc5SHHoFI1ym3tTHsvieQbBhei7QL3kKNvkQbOZ6VPUJ8YOR87ruCPXSslmrTbrjcb/NcBHvtFjnzUHHV/kJwCj89b/+l4uh++jcCNBa3iEtySv74jKdaXPtOfWD0omrwXGgC5YSHPWDTssJckd9GNWo/Yjk4kf4OpMQuktbOgSIgzcwmGGq2Buy8qSHNVkKpBWh7ESWqI3lbIQ8iMK9tGiMOA2crZQVpovyZqqXKGFmpe7O78/JwkKSd1Nf4LJUmBohROrJAe+u19mxOqtM0jPpbvBSOH69UkqIZYMHwV6+xMmOdvzAXaSax/cFmmBOtz+zZvP52QVfDCSS6I3+yy3kFwm9llsc6L+1Vy/priE3EBvr6FBd/cNH7vhtbjk5trPLA4y9fTS03xeKZf6/tJCFlrEvePmOggVYU0u3CEMfqA/Y/hIXHyHDol8nZ+95C94xs4VlC2cZSKV7nFudf17RpTJbTtMWROCl7c9RTTA1B3Y9Js1+K6TT+3v5n7fS/VzMjtQ/JMntwuTVaoySYgF9Tzt4q8mceZLmD9GNQaOCcJZ2ER7vjCDnz/HZ9f2ZcH0cyv3HW5ZfK28M6JWMkHVn0A0CYOSaPi1GIbzTE8eWhwd7gad3xNZttytCqwE1ovDM7SENLlgz9+unXCrPNj7SMtOJiIF8P85fYt+WmG3AU44Xzfy/KnVgxG0OfO9SBiMdLp3UnwozN4tT4QyqBSNJdlMa4H42HWRuSlWZpP1/Y9uu4M8dGCHZImpddqiRtnv084n737TLheSAEawthPZWZzAZYbcH7lNAWGNqTwuBKG+QP3fde Mdvk9SKu J0h1jlzzrAgafQ4bOwopAIGGZ2ZHtBzk98rX9NHx3BeEhsgD+Bk87t7haDsHqhu7RGTxxaZjHO3byaQDDTFEVDheGmpsxa8Y3HJTrB7ZwCe6+jNGcpgri2I/nNMAjdNbHQW7LMOqVUmHu8plUVzs0r9f/2u4LVhwtKzfROFit+7jhfD2tnvKQE1gc7Wmx6xtMbgYXIlv7lBMdu8fT1BPhl4xszA08cqn11Jv5oWzuzoRaHknlmxxmzT3UGKNhr8343a0HpWAlOd4vt6lyNADj4MA2HNDMGnkYOrVG+LuDH7AUH8rIPgSWCNU79bHmYEDVJDnNZ8APvwYPndLIPNLOR4fxgZQ/kkxNLf5Gq2wjYLBwmYHTPzL1b+VH7iZauE6zABwP0TcU/EtG+jfFZbJ9eeC8Ch5XhbzYTis6D/fbu+23q+sxJJBQ3ro+cH1Hv95zXZy1vz0qL4WEuykql3uF/E2NJyxscT3RrglDH0/StAxmwPH4OQ6MQBj9CxEJSNopKkJE4OZ53CK8SJgV/Uo0lAO+viM17c6tjeY9pwKjgyrqNqi/57aWRSoHZgyWzgooK7/AFenV17L+/taKsriqchubjFu5bOG8Y3bn6MT9g63vbDp2h6CU+/7R2jwNT+SfLeNLYIO2ILo8/6oMdDhqpwNYcQ3s1SsTL5kqhCvyjHGQFH3Ly373o/NUlm55CgkQhiPf/4d6G8F+8vEDf2Bpr6sRUg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Slab extension objects can't be allocated before slab infrastructure is initialized. Some caches, like kmem_cache and kmem_cache_node, are created before slab infrastructure is initialized. Objects from these caches can't have extension objects. Introduce SLAB_NO_OBJ_EXT slab flag to mark these caches and avoid creating extensions for objects allocated from these slabs. Signed-off-by: Suren Baghdasaryan Reviewed-by: Kees Cook --- include/linux/slab.h | 7 +++++++ mm/slub.c | 5 +++-- 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/include/linux/slab.h b/include/linux/slab.h index b5f5ee8308d0..3ac2fc830f0f 100644 --- a/include/linux/slab.h +++ b/include/linux/slab.h @@ -164,6 +164,13 @@ #endif #define SLAB_TEMPORARY SLAB_RECLAIM_ACCOUNT /* Objects are short-lived */ +#ifdef CONFIG_SLAB_OBJ_EXT +/* Slab created using create_boot_cache */ +#define SLAB_NO_OBJ_EXT ((slab_flags_t __force)0x20000000U) +#else +#define SLAB_NO_OBJ_EXT 0 +#endif + /* * ZERO_SIZE_PTR will be returned for zero sized kmalloc requests. * diff --git a/mm/slub.c b/mm/slub.c index 1eb1050814aa..9fd96238ed39 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -5650,7 +5650,8 @@ void __init kmem_cache_init(void) node_set(node, slab_nodes); create_boot_cache(kmem_cache_node, "kmem_cache_node", - sizeof(struct kmem_cache_node), SLAB_HWCACHE_ALIGN, 0, 0); + sizeof(struct kmem_cache_node), + SLAB_HWCACHE_ALIGN | SLAB_NO_OBJ_EXT, 0, 0); hotplug_memory_notifier(slab_memory_callback, SLAB_CALLBACK_PRI); @@ -5660,7 +5661,7 @@ void __init kmem_cache_init(void) create_boot_cache(kmem_cache, "kmem_cache", offsetof(struct kmem_cache, node) + nr_node_ids * sizeof(struct kmem_cache_node *), - SLAB_HWCACHE_ALIGN, 0, 0); + SLAB_HWCACHE_ALIGN | SLAB_NO_OBJ_EXT, 0, 0); kmem_cache = bootstrap(&boot_kmem_cache); kmem_cache_node = bootstrap(&boot_kmem_cache_node);