From patchwork Wed Apr 24 21:40:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kees Cook X-Patchwork-Id: 13642527 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 16A6FC10F15 for ; Wed, 24 Apr 2024 21:41:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D081C6B0314; Wed, 24 Apr 2024 17:41:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CB7F76B0316; Wed, 24 Apr 2024 17:41:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B30356B0317; Wed, 24 Apr 2024 17:41:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 974146B0314 for ; Wed, 24 Apr 2024 17:41:10 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 41CDC1A1123 for ; Wed, 24 Apr 2024 21:41:10 +0000 (UTC) X-FDA: 82045746300.30.C9B7D4E Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) by imf22.hostedemail.com (Postfix) with ESMTP id 72FFDC000A for ; Wed, 24 Apr 2024 21:41:08 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=TfmLQe64; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf22.hostedemail.com: domain of keescook@chromium.org designates 209.85.214.177 as permitted sender) smtp.mailfrom=keescook@chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1713994868; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=ZN4X6k1taEixm9wIK5apcc1Zl8K4ael4BkfXJ2q5v0k=; b=XA7vmCHl+vIXLXijH+AF0Ka+R6hEvW2Sh4TctvlfRRhf4k5YRa5pkc3djLLJtqCNPrjOPW Fh7OhyZUqsTcWuGAK7oQxigwCupcbybChbxvJWKkUYZP3Av2ZqBw31o043PCcV3BuGxinq GK51TphBDPLF+yvJG3O3dLipvPtdeY8= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=TfmLQe64; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf22.hostedemail.com: domain of keescook@chromium.org designates 209.85.214.177 as permitted sender) smtp.mailfrom=keescook@chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1713994868; a=rsa-sha256; cv=none; b=KhtQcwvM8mjiS2pH2sk/HOEtW2EgBnbzG65wehPh5s1C2+aFC2qjX5NIRbJUl1OMGTjCdw RmyKCHEso2fOEt77z2YAsgjvUPROyF6/ULonQ0UoCgrO9Ur/l2QmS25XH0tfhA02P5htYl ycZkJjVLOo3dSIUbsj8OSP7EfrAzoM0= Received: by mail-pl1-f177.google.com with SMTP id d9443c01a7336-1e86d56b3bcso2919995ad.1 for ; Wed, 24 Apr 2024 14:41:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1713994867; x=1714599667; darn=kvack.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=ZN4X6k1taEixm9wIK5apcc1Zl8K4ael4BkfXJ2q5v0k=; b=TfmLQe64k2gjvknguj2Ely6KtysG5yTBkKcz2JODH3xglL9ZUPXyeXyb14fuM0iRxi cxIIoZvjJQbllH0GjC0X7JJIqg4sxjEL5k0tyoPklJTsBoJ2mp3fBaDoW2nwkutP/Lg3 tzwrDgxQ9ikAlbiST+odgLksMcJjfhO8N14m8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713994867; x=1714599667; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=ZN4X6k1taEixm9wIK5apcc1Zl8K4ael4BkfXJ2q5v0k=; b=wCKIV9FPgH/GIzEFjl9g3E2Ch37ZQ/NPdwY5xd3YOUb77SHvThovIJEiSKAEIqbXBj iPYavZeTIcqr3ngvIV+iv2Wm9EgmYfN2FwPzgHnWN+tNc5Bb9w2sw80qn1L4BXGMwQkN Vaeuk0WSvT7G+7KoKawHZgEfZ6PHA0URQ3gJOl+HEcQqRrYEPiWhOAeosmPpD6+NK1L0 8QkXsQktzSi+k8SKDFWiE7LxnOE6Jh8S7OzuBZdcPhC7Xqu9QMJM0prVGjZsAWbb59/P NJB3eUBoixDm4OyfVA9pp0ooGkGxJUzZJVN11yLGYiBzeILwCICpGRmGRfZBe/Kekvik BgGw== X-Forwarded-Encrypted: i=1; AJvYcCU1ajjYAkokpadaE4CibwaXJzdc6k76W8PiYBJmWuNcc1tiMISSqxK207xB8WRZG9YBj+yIZYPwarbxhR7FeKVb4/c= X-Gm-Message-State: AOJu0YxnPHBxf8hsSXKJOoPGSGl612RZ0y4umfaJz3SStytgSYk/Zrc9 zgv0sPVvbi0EN1t5KKs0q/5rIyHdyxIM6sgTRvl9/QayhV86X13hXr7j6bit+w== X-Google-Smtp-Source: AGHT+IGsKG0KETtSrCcshPv9+RfF012Av+jDYjVSzUmXh3bqJ3o6ya/7ATIavZVy0HYtqqTaTGzC9Q== X-Received: by 2002:a17:902:6b42:b0:1e2:aa62:2fbf with SMTP id g2-20020a1709026b4200b001e2aa622fbfmr3421800plt.45.1713994866961; Wed, 24 Apr 2024 14:41:06 -0700 (PDT) Received: from www.outflux.net ([198.0.35.241]) by smtp.gmail.com with ESMTPSA id b5-20020a170902d60500b001e421f98ebdsm12397269plp.280.2024.04.24.14.41.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Apr 2024 14:41:05 -0700 (PDT) From: Kees Cook To: Vlastimil Babka Cc: Kees Cook , Andrew Morton , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "GONG, Ruiqi" , Xiu Jianfeng , Suren Baghdasaryan , Kent Overstreet , Jann Horn , Matteo Rizzo , Thomas Graf , Herbert Xu , julien.voisin@dustri.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-hardening@vger.kernel.org Subject: [PATCH v3 0/6] slab: Introduce dedicated bucket allocator Date: Wed, 24 Apr 2024 14:40:57 -0700 Message-Id: <20240424213019.make.366-kees@kernel.org> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=5704; i=keescook@chromium.org; h=from:subject:message-id; bh=CSAxST9l1ML3bK9B48Kc1gJu93/PFzNoh1S6a1nXHI8=; b=owEBbQKS/ZANAwAKAYly9N/cbcAmAcsmYgBmKXxuxwETH4SRJIHb8kWj0m1zUTHql5HJhw+fz mdszYRAjJiJAjMEAAEKAB0WIQSlw/aPIp3WD3I+bhOJcvTf3G3AJgUCZil8bgAKCRCJcvTf3G3A JvZTD/0emHHf4xvjdKdDiFBuShtObaN1uKCbv7/oH5gtys3bEISPdgj4zMk/cxYH553top2uDPd RkE8sWeyyj1YywWB8kMcsG+gipbi/nmPicqIUT+Oflrdz77pmLG8OtcPued74mQbOa4CgpKaBqx DBcWmnImRUGzQr/1wOzAdyj86L/C4nsmiyZWdPKRGVvR/hATQaKsDr1zsQe7LrkFtssFI/oUSA4 Y0oo8fPFxtaZOlA2R0vL25oC32E0CbbaRdGeVHPZXHT9GH1pUJeBp1EOPOsAhAcQ7yk2xL0exr5 3Fw+qGigeSVEfglcqdrY7V38sCokZvMOtQV3QLd8BCD5sn4AR8/7qdWwN5MnGRPk/SWa9Efch+w q/rAdkA3p6xGbNOFFmafGh4kL4tRjk/cSgG583WJFy5z7oBOsFmloPh0P22kLrQROzYRonqzEna 8sjKMVDrcp8ROy+2y3nS1PHJSKVyXp4M8AS838x1i5ISBH2xbDPg2ohhIHnQ4rTiojcuEAov/Rn gUsbW2iNWBvPn6Hz6iyZQbsSua1vsEDNzODC9VQXJ46DqX7VqJuzGjstmX6hP9RHDdUDco9Fu+N Z8O9yOHxrdch8FXScDdJvFD/XRCFPExGKymM9b/Kpor8sO1mMgpyanlizX6I08BYM/pyvSGQFSw /QIjFVz lv0QVvPA== X-Developer-Key: i=keescook@chromium.org; a=openpgp; fpr=A5C3F68F229DD60F723E6E138972F4DFDC6DC026 X-Rspam-User: X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 72FFDC000A X-Stat-Signature: coiygwt9b96ngn89gsrpyoy71cf4use1 X-HE-Tag: 1713994868-425509 X-HE-Meta: U2FsdGVkX19ZE3AmlV9TbAhsBlMM8miwVQlzM3bys4YIkhjZ+QeM7/SXwe+vnL28ArfDX7JVQ1wEmjVFx/knme1s9EfpujFtbY1K5/Av9ClEYqAVMNNaNZQlsTP3RY7LJDBZ6ul7h5p3/5jGLF/bmK8fyFK39K3Evff4UI45FJBFVCsAqsgF177VVxpb5k9Ebo6kZZ2sKJvqONXw4i4nJxO2v6CYfKnvwi0WyiuEa75EXFoor1WWsauE2V8yWI/t83eu43vqtUaD0k1kn6C2W/Ozqpv6zY/jB8NUvus2MNp+okyHgQC7/4GUTKasLP3bcgieKLt6BIbj9JJXSW28PfHkCoMmYneLGcn0ZroYcmaN632uR6YUvsjY1bFP1ktFVq1GsVtv7kdLgAdsaoOvu/JkABEpDPpTZRw5xtHQ3vG96zdvSue+LNaFfYwHqLw1dJUbwXhurBccfmstEmU0ns71VyWWHa7lo5DjUgoOJ1PrmFiFrAzeVTY1Is8plwLFBGNG3CIY7XetGSlqWKz+OkUNQG3SdZUCHl8fHeIB6kfJogYBmHVBm+8YJoxvGWuwi7dzVLnXqcPRNUNe8MDAXKV2waQfKNsuXj2toKAONWEW89gyhYyEIjP51IN4ObBi5R2lVvbB8vz4BeOCUesCcIbuvzFE4AIkmFznUBdT0ex0/IUcXu4hsre3bSeClLTeLubEW5md3CyahpWMlEvCQ9Q1p6vDWhnk8R8F9wxjm4TMGmcS23jWCkRvvbUfC6TITWNC4GcEYTST548iyHBZgkYQT+3zXe6A7zHcc2RRCFfJI/WoObUPzHs4OBfNKmNQQuteou0tHtjUf4bkN3BVA7HlZUSguW8cni+dvo6CeANI19f6IyeJvRxjEY0UE7Q/MgOEsKBV0mx2W9+8ap8zMKTV2VyG7AdHP2sXY0tqEFM/I+xWWJZXanVfraVG7iVxQEhrZo4L+zk8GIc7jEA xI6ywqPq z8dkXdEU/QOKmCo4e+r78UT1njMgC0oef/zePX3S6tOy5RPIKAoevbPf0YhZrbWzQzRiSNrLtLaOurAEA1QypMOlNn0KLZP5ds/XuAK3LOg44Z/oMh5twGWaYX/v7PGH6wOBXDZwSpplAt0ZJmuW+gOG/kvIh0h5cKqN67W9qVhFbRlsEsJA0dqrvD7LoYTXd8y0RdNZeMAwSNff3On4v86XwIwgcoF6uURyTwj3ko0u0pUUyIXeW86txvJjcklwjbLmdQP0PSCJpr/5s8Qn6U+PagptiXuYSNQnlfbqAWTqXKHLvO8mrBMCIfIBcx9jYRhGCsN/sP31XSoMLFrEk4AMxWUdc3O4034Y3Qary2odmQtYYiwNWLuyt/Lo5P1eAWBgL39YJnY7H9DM80C4D0pObSJwrCmvQD+6PZME9NHshwckXnLyWdYjBnbKCGZKFEgqWDe1qRvjy5wnQA12YO+2IeQTmfitbYPiq5EcwffS6MaOQumzVITtVpbRpuHxfbKW5BFFibG07ti408uQvd2no2J4rXCTfqyTUzpwz43jzRPRh3chUW1M78/yBcV8jxWEMyn26FmKyl32FXIuy+pl3aEPKefr41zk1pbFzxncppP8MDJzRQVR+JLiJuic/mKVQeAd6O2Ke7d340wVfNPevQoNkqmpFt11lzR/0e4hzY0YebXz+k28Sp8lcHkhLPCHLjkdyQFQghASuhBV4Yl5mcmpzjeR6kKRC28O1t15hsWw8/JvZU3IIFVB/VgjFy5OjQG78yDT3zGGX5ecr6fGKpx4q5kBxvirSupBAveuKIh3LVGYz7TV17Y/MzA2srOBmKqMKIE0VwtJNzvfpdWJXREd/IE1lZuLfxvDpHFHzqhA9kXYEH828weUPTXrTEFaDFqHMZL+B8ePitte4HR/al17vLVJJcUcVkxMkO9SkBX4eo5MncqkKd44noDFHqbHxjLCst0BSmqJ4/lGTXdD3R1cd mDdOoMnS WdqvVzjMCUubFaOvtUOnBhNQQBgE9SWtuhcvpgQ/k31PB8RUtmngpbKz2RjH84CdGfv4IIsrFfaciWj9mRrUwbltyPgdw6EeJaPcFiZ6dpHXjo6FYvQNdBoEU6eXrSpYVofWEtIbufQTOxStAOYkN3BEOKKW5ZR7JMg/+nkYqSXIf+rctW2on2hDLYANXiLzqXZz5MnRd9rTMKouOqDBLoSxvRhleH0EB3Cid+uqCewD78BLPwJBfsa85b9Uk6/qwWzPBsbXEfFIxlRpF90lxfJSkj4ChbdzXAn+aBuhIv0XXV27Qs8YIXOLoOH0PapV8I6KYW2klCgO9qWpk8O/3Av82NuhASTI4j1bV4dZ1KL4u64kqgOPBW7fjtfQPB0C X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hi, Series change history: v3: - clarify rationale and purpose in commit log - rebase to -next (CONFIG_CODE_TAGGING) - simplify calling styles and split out bucket plumbing more cleanly - consolidate kmem_buckets_*() family introduction patches v2: https://lore.kernel.org/lkml/20240305100933.it.923-kees@kernel.org/ v1: https://lore.kernel.org/lkml/20240304184252.work.496-kees@kernel.org/ For the cover letter, I'm repeating commit log for patch 4 here, which has additional clarifications and rationale since v2: Dedicated caches are available for fixed size allocations via kmem_cache_alloc(), but for dynamically sized allocations there is only the global kmalloc API's set of buckets available. This means it isn't possible to separate specific sets of dynamically sized allocations into a separate collection of caches. This leads to a use-after-free exploitation weakness in the Linux kernel since many heap memory spraying/grooming attacks depend on using userspace-controllable dynamically sized allocations to collide with fixed size allocations that end up in same cache. While CONFIG_RANDOM_KMALLOC_CACHES provides a probabilistic defense against these kinds of "type confusion" attacks, including for fixed same-size heap objects, we can create a complementary deterministic defense for dynamically sized allocations that are directly user controlled. Addressing these cases is limited in scope, so isolation these kinds of interfaces will not become an unbounded game of whack-a-mole. For example, pass through memdup_user(), making isolation there very effective. In order to isolate user-controllable sized allocations from system allocations, introduce kmem_buckets_create(), which behaves like kmem_cache_create(). Introduce kmem_buckets_alloc(), which behaves like kmem_cache_alloc(). Introduce kmem_buckets_alloc_track_caller() for where caller tracking is needed. Introduce kmem_buckets_valloc() for cases where vmalloc callback is needed. Allows for confining allocations to a dedicated set of sized caches (which have the same layout as the kmalloc caches). This can also be used in the future to extend codetag allocation annotations to implement per-caller allocation cache isolation[1] even for dynamic allocations. Memory allocation pinning[2] is still needed to plug the Use-After-Free cross-allocator weakness, but that is an existing and separate issue which is complementary to this improvement. Development continues for that feature via the SLAB_VIRTUAL[3] series (which could also provide guard pages -- another complementary improvement). Link: https://lore.kernel.org/lkml/202402211449.401382D2AF@keescook [1] Link: https://googleprojectzero.blogspot.com/2021/10/how-simple-linux-kernel-memory.html [2] Link: https://lore.kernel.org/lkml/20230915105933.495735-1-matteorizzo@google.com/ [3] After the core implementation are 2 patches that cover the most heavily abused "repeat offenders" used in exploits. Repeating those details here: The msg subsystem is a common target for exploiting[1][2][3][4][5][6] use-after-free type confusion flaws in the kernel for both read and write primitives. Avoid having a user-controlled size cache share the global kmalloc allocator by using a separate set of kmalloc buckets. Link: https://blog.hacktivesecurity.com/index.php/2022/06/13/linux-kernel-exploit-development-1day-case-study/ [1] Link: https://hardenedvault.net/blog/2022-11-13-msg_msg-recon-mitigation-ved/ [2] Link: https://www.willsroot.io/2021/08/corctf-2021-fire-of-salvation-writeup.html [3] Link: https://a13xp0p0v.github.io/2021/02/09/CVE-2021-26708.html [4] Link: https://google.github.io/security-research/pocs/linux/cve-2021-22555/writeup.html [5] Link: https://zplin.me/papers/ELOISE.pdf [6] Link: https://syst3mfailure.io/wall-of-perdition/ [7] Both memdup_user() and vmemdup_user() handle allocations that are regularly used for exploiting use-after-free type confusion flaws in the kernel (e.g. prctl() PR_SET_VMA_ANON_NAME[1] and setxattr[2][3][4] respectively). Since both are designed for contents coming from userspace, it allows for userspace-controlled allocation sizes. Use a dedicated set of kmalloc buckets so these allocations do not share caches with the global kmalloc buckets. Link: https://starlabs.sg/blog/2023/07-prctl-anon_vma_name-an-amusing-heap-spray/ [1] Link: https://duasynt.com/blog/linux-kernel-heap-spray [2] Link: https://etenal.me/archives/1336 [3] Link: https://github.com/a13xp0p0v/kernel-hack-drill/blob/master/drill_exploit_uaf.c [4] Thanks! -Kees Kees Cook (6): mm/slab: Introduce kmem_buckets typedef mm/slab: Plumb kmem_buckets into __do_kmalloc_node() mm/slab: Introduce __kvmalloc_node() that can take kmem_buckets argument mm/slab: Introduce kmem_buckets_create() and family ipc, msg: Use dedicated slab buckets for alloc_msg() mm/util: Use dedicated slab buckets for memdup_user() include/linux/slab.h | 44 ++++++++++++++++-------- ipc/msgutil.c | 13 +++++++- lib/fortify_kunit.c | 2 +- lib/rhashtable.c | 2 +- mm/slab.h | 6 ++-- mm/slab_common.c | 79 +++++++++++++++++++++++++++++++++++++++++--- mm/slub.c | 14 ++++---- mm/util.c | 21 +++++++++--- 8 files changed, 146 insertions(+), 35 deletions(-)