From patchwork Wed Apr 24 21:40:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kees Cook X-Patchwork-Id: 13642534 Received: from mail-pl1-f178.google.com (mail-pl1-f178.google.com [209.85.214.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DAC77156F25 for ; Wed, 24 Apr 2024 21:41:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713994869; cv=none; b=lKRqtAN2xaybCow8ku0yzpf6si2ciP0QhNjWbKI6jNlKqF8tSMpzlSYvxPy0VU/8kaOO5k8hLUryCBYtyuTlVCsvs0WOOvm3dGD+gNR0HqBKCParl3wBhZe4jvAojQDSL+W1fzOaAyzbp/Cu9Wow5hmht637fK/ecQf5S7v6D+k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713994869; c=relaxed/simple; bh=CSAxST9l1ML3bK9B48Kc1gJu93/PFzNoh1S6a1nXHI8=; h=From:To:Cc:Subject:Date:Message-Id:MIME-Version; b=VRlEh3v1pK7RgPOVliriJwm1q9hKob1JefqfWL+KcKtmdDH54zWnvEqNaqZSa2+XE5gVGGOqhjQVEQZ/UHBRnO5nCsL0hMGht1zAcOyW9MxtU/1EMg3Yutl4witPOMPKbW/KT7wxaJ1JrUuW3eE40qg8MouD+wz/oa3x3jjp5Qw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org; spf=pass smtp.mailfrom=chromium.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b=Xy+gnUVI; arc=none smtp.client-ip=209.85.214.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=chromium.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b="Xy+gnUVI" Received: by mail-pl1-f178.google.com with SMTP id d9443c01a7336-1e8fce77bb2so2528425ad.0 for ; Wed, 24 Apr 2024 14:41:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1713994867; x=1714599667; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=ZN4X6k1taEixm9wIK5apcc1Zl8K4ael4BkfXJ2q5v0k=; b=Xy+gnUVIU94bJtUxLPVkLT9QDtDzY9yMC+6TWlXjEKMNojNcZDckA//lPc231R9Eh5 dQsg5NWXpVivDB1MhcvdX3Y7MvQl2xtNhS3xkgRYXL+7tGiK7gOEcVSmyAqnKCtLGVhi 80I2EjezI7+MtAHjTjjnCmbIKjl0BKBO+4omk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713994867; x=1714599667; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=ZN4X6k1taEixm9wIK5apcc1Zl8K4ael4BkfXJ2q5v0k=; b=qewP+i70VeWbYRgaKMDBdIqcFsllixY+DElsYtBrEiVVBHpmf4PZ3+m47HrGDsLUAb diZBwcl9NOqcTGkEPq7fRCdFbN9ebM6kWOBumeAK0Mi/bR8N8RR0VNf7bewhUFjcC/8V yhCJl19mhtwgwC4c96g40tc5KSbpFoVDeO05haGgshcPgtbGYv3M74cTevTbQs9I8NEL RmqvcnsYAJySW8EhIkQCc0n+EKBZRudwekoxvKQlYDeMem/N4UHMGDmBqowUmvdZY66C hpYV5qgxD6w6oUJSeEOBJKPH3MYeh/tZD0QpfJHkTSSefIQfnTlMSfHgU/J9SQq5lOJ1 J5Hw== X-Forwarded-Encrypted: i=1; AJvYcCXSLr8JCwAH+M63eWQxkevvdy24IbZT69OdFXhgDF43xxaxUB0NT1qvCCrYWWToU925f58vdvxk5pn/F4SKwyKNpaeV+8Vc9ks5wpRQaVs4 X-Gm-Message-State: AOJu0YyfNWPrrRd+AjERjGK6HsDrCvgohkpGiYP3zaLcTFRp6gUhoJi9 sBCScsKFAC1nw2/eUyXdOpggfKC2X4u0AE5kCGdLNnSuh/qJgTpgHJz5faABVw== X-Google-Smtp-Source: AGHT+IGsKG0KETtSrCcshPv9+RfF012Av+jDYjVSzUmXh3bqJ3o6ya/7ATIavZVy0HYtqqTaTGzC9Q== X-Received: by 2002:a17:902:6b42:b0:1e2:aa62:2fbf with SMTP id g2-20020a1709026b4200b001e2aa622fbfmr3421800plt.45.1713994866961; Wed, 24 Apr 2024 14:41:06 -0700 (PDT) Received: from www.outflux.net ([198.0.35.241]) by smtp.gmail.com with ESMTPSA id b5-20020a170902d60500b001e421f98ebdsm12397269plp.280.2024.04.24.14.41.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Apr 2024 14:41:05 -0700 (PDT) From: Kees Cook To: Vlastimil Babka Cc: Kees Cook , Andrew Morton , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "GONG, Ruiqi" , Xiu Jianfeng , Suren Baghdasaryan , Kent Overstreet , Jann Horn , Matteo Rizzo , Thomas Graf , Herbert Xu , julien.voisin@dustri.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-hardening@vger.kernel.org Subject: [PATCH v3 0/6] slab: Introduce dedicated bucket allocator Date: Wed, 24 Apr 2024 14:40:57 -0700 Message-Id: <20240424213019.make.366-kees@kernel.org> X-Mailer: git-send-email 2.34.1 Precedence: bulk X-Mailing-List: linux-hardening@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=5704; i=keescook@chromium.org; h=from:subject:message-id; bh=CSAxST9l1ML3bK9B48Kc1gJu93/PFzNoh1S6a1nXHI8=; b=owEBbQKS/ZANAwAKAYly9N/cbcAmAcsmYgBmKXxuxwETH4SRJIHb8kWj0m1zUTHql5HJhw+fz mdszYRAjJiJAjMEAAEKAB0WIQSlw/aPIp3WD3I+bhOJcvTf3G3AJgUCZil8bgAKCRCJcvTf3G3A JvZTD/0emHHf4xvjdKdDiFBuShtObaN1uKCbv7/oH5gtys3bEISPdgj4zMk/cxYH553top2uDPd RkE8sWeyyj1YywWB8kMcsG+gipbi/nmPicqIUT+Oflrdz77pmLG8OtcPued74mQbOa4CgpKaBqx DBcWmnImRUGzQr/1wOzAdyj86L/C4nsmiyZWdPKRGVvR/hATQaKsDr1zsQe7LrkFtssFI/oUSA4 Y0oo8fPFxtaZOlA2R0vL25oC32E0CbbaRdGeVHPZXHT9GH1pUJeBp1EOPOsAhAcQ7yk2xL0exr5 3Fw+qGigeSVEfglcqdrY7V38sCokZvMOtQV3QLd8BCD5sn4AR8/7qdWwN5MnGRPk/SWa9Efch+w q/rAdkA3p6xGbNOFFmafGh4kL4tRjk/cSgG583WJFy5z7oBOsFmloPh0P22kLrQROzYRonqzEna 8sjKMVDrcp8ROy+2y3nS1PHJSKVyXp4M8AS838x1i5ISBH2xbDPg2ohhIHnQ4rTiojcuEAov/Rn gUsbW2iNWBvPn6Hz6iyZQbsSua1vsEDNzODC9VQXJ46DqX7VqJuzGjstmX6hP9RHDdUDco9Fu+N Z8O9yOHxrdch8FXScDdJvFD/XRCFPExGKymM9b/Kpor8sO1mMgpyanlizX6I08BYM/pyvSGQFSw /QIjFVz lv0QVvPA== X-Developer-Key: i=keescook@chromium.org; a=openpgp; fpr=A5C3F68F229DD60F723E6E138972F4DFDC6DC026 Hi, Series change history: v3: - clarify rationale and purpose in commit log - rebase to -next (CONFIG_CODE_TAGGING) - simplify calling styles and split out bucket plumbing more cleanly - consolidate kmem_buckets_*() family introduction patches v2: https://lore.kernel.org/lkml/20240305100933.it.923-kees@kernel.org/ v1: https://lore.kernel.org/lkml/20240304184252.work.496-kees@kernel.org/ For the cover letter, I'm repeating commit log for patch 4 here, which has additional clarifications and rationale since v2: Dedicated caches are available for fixed size allocations via kmem_cache_alloc(), but for dynamically sized allocations there is only the global kmalloc API's set of buckets available. This means it isn't possible to separate specific sets of dynamically sized allocations into a separate collection of caches. This leads to a use-after-free exploitation weakness in the Linux kernel since many heap memory spraying/grooming attacks depend on using userspace-controllable dynamically sized allocations to collide with fixed size allocations that end up in same cache. While CONFIG_RANDOM_KMALLOC_CACHES provides a probabilistic defense against these kinds of "type confusion" attacks, including for fixed same-size heap objects, we can create a complementary deterministic defense for dynamically sized allocations that are directly user controlled. Addressing these cases is limited in scope, so isolation these kinds of interfaces will not become an unbounded game of whack-a-mole. For example, pass through memdup_user(), making isolation there very effective. In order to isolate user-controllable sized allocations from system allocations, introduce kmem_buckets_create(), which behaves like kmem_cache_create(). Introduce kmem_buckets_alloc(), which behaves like kmem_cache_alloc(). Introduce kmem_buckets_alloc_track_caller() for where caller tracking is needed. Introduce kmem_buckets_valloc() for cases where vmalloc callback is needed. Allows for confining allocations to a dedicated set of sized caches (which have the same layout as the kmalloc caches). This can also be used in the future to extend codetag allocation annotations to implement per-caller allocation cache isolation[1] even for dynamic allocations. Memory allocation pinning[2] is still needed to plug the Use-After-Free cross-allocator weakness, but that is an existing and separate issue which is complementary to this improvement. Development continues for that feature via the SLAB_VIRTUAL[3] series (which could also provide guard pages -- another complementary improvement). Link: https://lore.kernel.org/lkml/202402211449.401382D2AF@keescook [1] Link: https://googleprojectzero.blogspot.com/2021/10/how-simple-linux-kernel-memory.html [2] Link: https://lore.kernel.org/lkml/20230915105933.495735-1-matteorizzo@google.com/ [3] After the core implementation are 2 patches that cover the most heavily abused "repeat offenders" used in exploits. Repeating those details here: The msg subsystem is a common target for exploiting[1][2][3][4][5][6] use-after-free type confusion flaws in the kernel for both read and write primitives. Avoid having a user-controlled size cache share the global kmalloc allocator by using a separate set of kmalloc buckets. Link: https://blog.hacktivesecurity.com/index.php/2022/06/13/linux-kernel-exploit-development-1day-case-study/ [1] Link: https://hardenedvault.net/blog/2022-11-13-msg_msg-recon-mitigation-ved/ [2] Link: https://www.willsroot.io/2021/08/corctf-2021-fire-of-salvation-writeup.html [3] Link: https://a13xp0p0v.github.io/2021/02/09/CVE-2021-26708.html [4] Link: https://google.github.io/security-research/pocs/linux/cve-2021-22555/writeup.html [5] Link: https://zplin.me/papers/ELOISE.pdf [6] Link: https://syst3mfailure.io/wall-of-perdition/ [7] Both memdup_user() and vmemdup_user() handle allocations that are regularly used for exploiting use-after-free type confusion flaws in the kernel (e.g. prctl() PR_SET_VMA_ANON_NAME[1] and setxattr[2][3][4] respectively). Since both are designed for contents coming from userspace, it allows for userspace-controlled allocation sizes. Use a dedicated set of kmalloc buckets so these allocations do not share caches with the global kmalloc buckets. Link: https://starlabs.sg/blog/2023/07-prctl-anon_vma_name-an-amusing-heap-spray/ [1] Link: https://duasynt.com/blog/linux-kernel-heap-spray [2] Link: https://etenal.me/archives/1336 [3] Link: https://github.com/a13xp0p0v/kernel-hack-drill/blob/master/drill_exploit_uaf.c [4] Thanks! -Kees Kees Cook (6): mm/slab: Introduce kmem_buckets typedef mm/slab: Plumb kmem_buckets into __do_kmalloc_node() mm/slab: Introduce __kvmalloc_node() that can take kmem_buckets argument mm/slab: Introduce kmem_buckets_create() and family ipc, msg: Use dedicated slab buckets for alloc_msg() mm/util: Use dedicated slab buckets for memdup_user() include/linux/slab.h | 44 ++++++++++++++++-------- ipc/msgutil.c | 13 +++++++- lib/fortify_kunit.c | 2 +- lib/rhashtable.c | 2 +- mm/slab.h | 6 ++-- mm/slab_common.c | 79 +++++++++++++++++++++++++++++++++++++++++--- mm/slub.c | 14 ++++---- mm/util.c | 21 +++++++++--- 8 files changed, 146 insertions(+), 35 deletions(-)