From patchwork Fri Apr 11 09:16:24 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 14047909 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9F67BC36010 for ; Fri, 11 Apr 2025 09:17:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 34F582801A0; Fri, 11 Apr 2025 05:17:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2FEE528019B; Fri, 11 Apr 2025 05:17:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1C6AA2801A0; Fri, 11 Apr 2025 05:17:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id F291928019B for ; Fri, 11 Apr 2025 05:17:43 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id A868FBD03F for ; Fri, 11 Apr 2025 09:17:44 +0000 (UTC) X-FDA: 83321210448.09.787BF55 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf16.hostedemail.com (Postfix) with ESMTP id 16FC3180005 for ; Fri, 11 Apr 2025 09:17:42 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf16.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744363063; a=rsa-sha256; cv=none; b=He36x22V4PRBhT5GGxBIW29giFNmk7kd78oeZx07ZeDWAYAdSCQybkywGz4bWkQGRpjzH5 yvEawI7zq4nLNYZ2vGFJsubWDk+FXc6xcoK8ewRKOTIm6yfIRcKD7Jli0hYfIw6dtMaEN1 JEQa1Cf4++H7IXZ1nGdNiQMORWoNi4A= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf16.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744363063; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Vu1673XrmeD4DkkrL7shuinMre40KndBvHpM+lv2ifE=; b=DOToJzxbjg2xn4iV/99csavnwV2M9N1tHjv8rFOa0LFM5pUiTXTyswO5Kd3RKHQqQEHXIm gIpuxFIeptyjNbT4SFlvxjgjW50fDft7COurkpH5OWbF1DX1N6MNNkXwWtzmX0obITby34 owIUuwrafl970L/qK4W3MshWtFm0K6Y= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1DC4D106F; Fri, 11 Apr 2025 02:17:42 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 8C2003F6A8; Fri, 11 Apr 2025 02:17:38 -0700 (PDT) From: Kevin Brodsky To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Mark Brown , Catalin Marinas , Dave Hansen , David Hildenbrand , Ira Weiny , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Andy Lutomirski , Marc Zyngier , Peter Zijlstra , Pierre Langlois , Quentin Perret , Rick Edgecombe , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-arm-kernel@lists.infradead.org, x86@kernel.org Subject: [RFC PATCH v4 11/18] mm: Introduce kpkeys_hardened_pgtables Date: Fri, 11 Apr 2025 10:16:24 +0100 Message-ID: <20250411091631.954228-12-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250411091631.954228-1-kevin.brodsky@arm.com> References: <20250411091631.954228-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 16FC3180005 X-Stat-Signature: amsed351j4sqbkfrnpjwkyfinnc5xjfm X-Rspam-User: X-Rspamd-Server: rspam06 X-HE-Tag: 1744363062-589821 X-HE-Meta: U2FsdGVkX1+AkiFRgcxprb+QnyZ3cKYPxSTpmHshVtmotn079cMgaWCeSa6o5C/sjZlnEJIOLGY358yx1dUO+WaVpi8BDnMuizO34hCLWAtibTyCGfvv9mIirpdOdXHBMnvZQH8tEGKMwPIyuiZUfSSa2q9fJW0W9q/xCyI86J0u+N2rfls5kzea+iW/oB9mFdrgNV9k1AjSXJQoSnB84DSx0H7TvEe766w1YJyZwix21m3BEwtDbBLJevpFDbIxmSICrVUayMRfcyhg5nWmhJ5dVLicvIx3ijbIL5IoeRFuu2NcP6rzNCmEdwn5//bKzVLq9TorpdCElBDsJLjIKkGWq4ezPvzmEMytNnnexLzTvWwk6Gt+598ILlTXnupUnoa8917rGOBO1CBJbC2P8XS8VhNxE8Gf4Bml/hQNCh1AkbDRZFHHnBCBqDzk/XRD4Ut/TZVuItby8d6ViBH3Z0KLAjR8x573E3x8esaJFdMNqqaBWeas4oIPhYKNA9TAQVsbY5kdVEYaAR5Yogu0GXvzuqW15BJOtRhKfNGe5qiBzAZOhhTuKRKq8DajnGeu8+T1wclHiaujKY9zQzxUvYY1qAETdmnKL4hIgKGdPuyRw2wTQWguIKsfXbgoXVGd0Phnv8DYmocpBP2YUUAcaLqV5QhiachNiWMEakrLjNNVi41pcsJ47QdfAW55tYw7+j3w4ll0FhMGlGBr+igfhATRMgBjDlOHKtRCBNuurfQIYBWhUcMnaw7Y2Pgukq+NK/CZiptHj345S9V216AUrk4WkWxFTLecT/Ert8mhOeS/28jVa1V8kAQmm/PmMrXhLtecr75ZoYwu1H4u3HuuMVXAMckVFVVL8mAEjXQfKJI52A3KpYgPy328Ti9MyTapYA/rpo07mAkRfDIu8oQOfbkOUtWYoResvM1/rIpiarRIuuS9wQkL0Zbrf4+2dhFBpBuRrZh8PADWwSnoHjn kZAN/hcr UenN6o16zVnboq6MNEfy8SIGa1HkKYrPkX5aIednZMTemrqif6R+cLeh5AG3I5YQJHqsrarF8xJNZm+vHY1KQG6T20DGUm6oLlnssfV+tyhz/6lXu8gda1OSEtHFt91ApwO/D/jCqfUMpCPGMc11K73oby+F/QxwoElrLLfQUSP59cccXP5MS/OnKyWqEUQNlG7pCYuwzV986SXg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: kpkeys_hardened_pgtables is a hardening feature based on kpkeys. It aims to prevent the corruption of page tables by: 1. mapping all page table pages, both kernel and user, with a privileged pkey (KPKEYS_PKEY_PGTABLES), and 2. granting write access to that pkey only when running at a higher kpkeys level (KPKEYS_LVL_PGTABLES). The feature is exposed as CONFIG_KPKEYS_HARDENED_PGTABLES; it requires explicit architecture opt-in by selecting ARCH_HAS_KPKEYS_HARDENED_PGTABLES, since much of the page table handling is arch-specific. This patch introduces an API to modify the PTPs' pkey. Because this API is going to be called from low-level pgtable helpers, it must be inactive on boot and explicitly switched on if and when kpkeys become available. A static key is used for that purpose; it is the responsibility of each architecture supporting kpkeys_hardened_pgtables to call kpkeys_hardened_pgtables_enable() as early as possible to switch on that static key. The initial kernel page tables are also walked to set their pkey, since they have already been allocated at that point. Signed-off-by: Kevin Brodsky --- include/asm-generic/kpkeys.h | 4 +++ include/linux/kpkeys.h | 46 ++++++++++++++++++++++++++++++++++- mm/Kconfig | 3 +++ mm/Makefile | 1 + mm/kpkeys_hardened_pgtables.c | 44 +++++++++++++++++++++++++++++++++ security/Kconfig.hardening | 12 +++++++++ 6 files changed, 109 insertions(+), 1 deletion(-) create mode 100644 mm/kpkeys_hardened_pgtables.c diff --git a/include/asm-generic/kpkeys.h b/include/asm-generic/kpkeys.h index ab819f157d6a..cec92334a9f3 100644 --- a/include/asm-generic/kpkeys.h +++ b/include/asm-generic/kpkeys.h @@ -2,6 +2,10 @@ #ifndef __ASM_GENERIC_KPKEYS_H #define __ASM_GENERIC_KPKEYS_H +#ifndef KPKEYS_PKEY_PGTABLES +#define KPKEYS_PKEY_PGTABLES 1 +#endif + #ifndef KPKEYS_PKEY_DEFAULT #define KPKEYS_PKEY_DEFAULT 0 #endif diff --git a/include/linux/kpkeys.h b/include/linux/kpkeys.h index faa6e2615798..5f4b096374ba 100644 --- a/include/linux/kpkeys.h +++ b/include/linux/kpkeys.h @@ -4,11 +4,15 @@ #include #include +#include + +struct folio; #define KPKEYS_LVL_DEFAULT 0 +#define KPKEYS_LVL_PGTABLES 1 #define KPKEYS_LVL_MIN KPKEYS_LVL_DEFAULT -#define KPKEYS_LVL_MAX KPKEYS_LVL_DEFAULT +#define KPKEYS_LVL_MAX KPKEYS_LVL_PGTABLES #define __KPKEYS_GUARD(name, set_level, restore_pkey_reg, set_arg, ...) \ __DEFINE_CLASS_IS_CONDITIONAL(name, false); \ @@ -110,4 +114,44 @@ static inline bool arch_kpkeys_enabled(void) #endif /* CONFIG_ARCH_HAS_KPKEYS */ +#ifdef CONFIG_KPKEYS_HARDENED_PGTABLES + +DECLARE_STATIC_KEY_FALSE(kpkeys_hardened_pgtables_key); + +static inline bool kpkeys_hardened_pgtables_enabled(void) +{ + return static_branch_unlikely(&kpkeys_hardened_pgtables_key); +} + +int kpkeys_protect_pgtable_memory(struct folio *folio); +int kpkeys_unprotect_pgtable_memory(struct folio *folio); + +/* + * Enables kpkeys_hardened_pgtables and switches existing kernel page tables to + * a privileged pkey (KPKEYS_PKEY_PGTABLES). + * + * Should be called as early as possible by architecture code, after (k)pkeys + * are initialised and before any user task is spawned. + */ +void kpkeys_hardened_pgtables_enable(void); + +#else /* CONFIG_KPKEYS_HARDENED_PGTABLES */ + +static inline bool kpkeys_hardened_pgtables_enabled(void) +{ + return false; +} + +static inline int kpkeys_protect_pgtable_memory(struct folio *folio) +{ + return 0; +} +static inline int kpkeys_unprotect_pgtable_memory(struct folio *folio) +{ + return 0; +} +static inline void kpkeys_hardened_pgtables_enable(void) {} + +#endif /* CONFIG_KPKEYS_HARDENED_PGTABLES */ + #endif /* _LINUX_KPKEYS_H */ diff --git a/mm/Kconfig b/mm/Kconfig index 819ef5b70695..ae1cb209a1ac 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -1133,6 +1133,9 @@ config ARCH_HAS_PKEYS bool config ARCH_HAS_KPKEYS bool +# ARCH_HAS_KPKEYS must be selected when selecting this option +config ARCH_HAS_KPKEYS_HARDENED_PGTABLES + bool config ARCH_USES_PG_ARCH_2 bool diff --git a/mm/Makefile b/mm/Makefile index e7f6bbf8ae5f..0f30cc85c6f1 100644 --- a/mm/Makefile +++ b/mm/Makefile @@ -148,3 +148,4 @@ obj-$(CONFIG_SHRINKER_DEBUG) += shrinker_debug.o obj-$(CONFIG_EXECMEM) += execmem.o obj-$(CONFIG_TMPFS_QUOTA) += shmem_quota.o obj-$(CONFIG_PT_RECLAIM) += pt_reclaim.o +obj-$(CONFIG_KPKEYS_HARDENED_PGTABLES) += kpkeys_hardened_pgtables.o diff --git a/mm/kpkeys_hardened_pgtables.c b/mm/kpkeys_hardened_pgtables.c new file mode 100644 index 000000000000..931fa97bc8a7 --- /dev/null +++ b/mm/kpkeys_hardened_pgtables.c @@ -0,0 +1,44 @@ +// SPDX-License-Identifier: GPL-2.0-only +#include +#include +#include + +DEFINE_STATIC_KEY_FALSE(kpkeys_hardened_pgtables_key); + +int kpkeys_protect_pgtable_memory(struct folio *folio) +{ + unsigned long addr = (unsigned long)folio_address(folio); + unsigned int order = folio_order(folio); + int ret = 0; + + if (kpkeys_hardened_pgtables_enabled()) + ret = set_memory_pkey(addr, 1 << order, KPKEYS_PKEY_PGTABLES); + + WARN_ON(ret); + return ret; +} + +int kpkeys_unprotect_pgtable_memory(struct folio *folio) +{ + unsigned long addr = (unsigned long)folio_address(folio); + unsigned int order = folio_order(folio); + int ret = 0; + + if (kpkeys_hardened_pgtables_enabled()) + ret = set_memory_pkey(addr, 1 << order, KPKEYS_PKEY_DEFAULT); + + WARN_ON(ret); + return ret; +} + +void __init kpkeys_hardened_pgtables_enable(void) +{ + int ret; + + if (!arch_kpkeys_enabled()) + return; + + static_branch_enable(&kpkeys_hardened_pgtables_key); + ret = kernel_pgtables_set_pkey(KPKEYS_PKEY_PGTABLES); + WARN_ON(ret); +} diff --git a/security/Kconfig.hardening b/security/Kconfig.hardening index c17366ce8224..c2b0987768ca 100644 --- a/security/Kconfig.hardening +++ b/security/Kconfig.hardening @@ -334,6 +334,18 @@ config BUG_ON_DATA_CORRUPTION If unsure, say N. +config KPKEYS_HARDENED_PGTABLES + bool "Harden page tables using kernel pkeys" + depends on ARCH_HAS_KPKEYS_HARDENED_PGTABLES + help + This option makes all page tables mostly read-only by + allocating them with a non-default protection key (pkey) and + only enabling write access to that pkey in routines that are + expected to write to page table entries. + + This option has no effect if the system does not support + kernel pkeys. + endmenu config CC_HAS_RANDSTRUCT