From patchwork Wed Feb 14 12:29:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 13556425 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4A5D7C48BC1 for ; Wed, 14 Feb 2024 12:32:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=4JuopDHcRwnlISQH5prfVdc1YSTZ8UJ9DBnM0cDtVxM=; b=gHVNiEvxj5osHYxH8MaWsVc+6c yXu7UiAopWUgBoQBvFN/gWTxNOXsnmtzNeO/nxNDjnxPAtKMQr51xChpmHnKO5MZegXfu+Tb5RwMO mBsIYot5tIJUSZ6I9kZo+fQjEnlhIVrreQ8Ge1b0S7VlVC45s7c+lotTYRl7AXNBc8MvThb+Z9wPE /8q/VVxwpMdpr0KPhcfD5pvxWQe2oze/UA40RZR1LBLF96Nrctbgg3L7gXDp/jaCuUWB6wLOszN+H Q9t2P486MGBJZmR5h1z3WsUZfMvG4fZwM9NIw2HHhM7YsYc9AC3V5Gim78ysDjMK8M2fhVGCCqxYk qN7vrN2g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1raER1-0000000CpWw-2GnK; Wed, 14 Feb 2024 12:32:19 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1raEPb-0000000Cow7-41nz for linux-arm-kernel@bombadil.infradead.org; Wed, 14 Feb 2024 12:30:52 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Type:Cc:To:From:Subject: Message-ID:References:Mime-Version:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=HyvjalD/6fvbAX20kRuPTKXh7jyw9GTdF8mbD1bkAPI=; b=lU9Mb3IFcCVUGGmrARVZbvpgvn sEx5GBCGBkUTvX3h9pqZ8zUyjwrXLIqk9SiacAGALjnL6dVngf68jT61WE7F72owfYwivpQUCMM3R g3MBHRhF+4bnvmlDlFdPi8YVigHuA7oWDAXx+cEBFHVlrIJlQWHBpdRN/9GWfCODzJAp06YdQSPpy jOG+nd70qyR9ZKDr/t5jfWyuBNNCFoI+J+aywI1vw+ggUNKJe6olLUYJOaU3esGBevYUICwSLXwub 0bgvI0Tcg6yqKhFSofxuNqR7i8Tnh0NVVeQJ/8D3GV0D6coyLqKr6+A3eceFAJ+0wGLN5mVD37yFA 0m3QE+hg==; Received: from mail-wr1-x44a.google.com ([2a00:1450:4864:20::44a]) by desiato.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1raEPW-0000000GE7j-2KR5 for linux-arm-kernel@lists.infradead.org; Wed, 14 Feb 2024 12:30:50 +0000 Received: by mail-wr1-x44a.google.com with SMTP id ffacd0b85a97d-33cf46a8932so18309f8f.1 for ; Wed, 14 Feb 2024 04:30:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1707913845; x=1708518645; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=HyvjalD/6fvbAX20kRuPTKXh7jyw9GTdF8mbD1bkAPI=; b=Itw9P5w5WK99CidYdIWYD4kQL3HnqzEOVMh0JUQQB0OtgN6Qtmit+lBUFrq+wwWDum B27eit1tdoU+ks2JsnTr30IJ6R/EvhU1nJassIXIiBo56ncIcB0itDlLCPeIKUXZCFut rlGwe+1n3ooOH5SfLUk5O39/nh6IgZJ/NmXobW74yVxyWUY0+p66kAPJSiFZ2uc5584I LggwaMe4EJJqxdrJ4CDW3juQ1FTWpZ1d5JAI4rd37zPcUrzpKI0RIW2xbIcUsbdCEN44 Dsv8YsFiJ8F0crZ6XrRQsXFX5uWQC4nsdbsKvlF187TmmQcoHiZTrxbJC0xQtNv48/w8 VKtw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707913845; x=1708518645; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=HyvjalD/6fvbAX20kRuPTKXh7jyw9GTdF8mbD1bkAPI=; b=ARYi0o42H0YTrtdafSO0AsIUx6uy07RWedMBbAduA0uOVyzBP4azSnP0Pq1LaKR7J0 Un6UOtcAYx68SCfBRtZ6RmqJuaMtuxUh6CK1/KgCDgOT2CJ6URBaVPjhRwQeEtN/uzOG 4U2y+W+2sgo2e6Du3wUjWZU291+u/VlgItuHOpwfYeG+4v7s8Xs6thglz8KPNGvmUXLF +XZ6Hzr6dl1rkB/+vLDdfF/PUy8/Y8BcqHMLeIzotnI6QCpZJrOVLFAaA9XAyk5zZNIV SCf0WxuKFcaSg03DiKXklg3YyLum+7l19qz/mkkP5voVqQjecg+yuSYm2mzhY7tfB+8C kHpA== X-Gm-Message-State: AOJu0YzoU+PbkZwSMVkhBhzAwMF2VhEneRNF6+rvCDIcYT54TxroDBnT D2S28akMywRvenxhR4hszOo8MK3Axp3KZXAWLFZHo9AWQNKLyktt9O/tyedHBQdf25bbj3wexlQ wIhRl/UzXTm/4sbYNtVSA+UWIeVHADpRgg3HaAtmFgzkzmW/fXFtgvDD5ZaBPbz6csyQe9q8H5B 7RBw1EZ5WCLbCSeZuCDqMxlMSpPuBSjjpT+GHfIYw8 X-Google-Smtp-Source: AGHT+IEr4dzBT9iN6w4yezP7LsLZXoX626Bo3jX9QHhZlShcAr1vhCxGQ1PSv1QeE9OYYUVU9D3JyI+n X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:118a]) (user=ardb job=sendgmr) by 2002:a05:6000:1d89:b0:33c:e2f7:b157 with SMTP id bk9-20020a0560001d8900b0033ce2f7b157mr4361wrb.4.1707913844526; Wed, 14 Feb 2024 04:30:44 -0800 (PST) Date: Wed, 14 Feb 2024 13:29:15 +0100 In-Reply-To: <20240214122845.2033971-45-ardb+git@google.com> Mime-Version: 1.0 References: <20240214122845.2033971-45-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=3974; i=ardb@kernel.org; h=from:subject; bh=k4s2tDBZMYe2OfnHcnfaPU4Fz87tpYu4VFMcOiHTEr8=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIfXMJln726IbOB/K151v6jf5/vloxFeZvUF2S69vVPrwy vtLnkh4RykLgxgHg6yYIovA7L/vdp6eKFXrPEsWZg4rE8gQBi5OAZjI3hhGhkVTfkRktetMPXG5 z22ra7qzbPRErU8Hk1f800+VmbYoewbDP5UvWUzFrrUmpi+OzT1p119T7f/Xym6OdHvQ9YU9GzW EuAA= X-Mailer: git-send-email 2.43.0.687.g38aa6559b0-goog Message-ID: <20240214122845.2033971-74-ardb+git@google.com> Subject: [PATCH v8 29/43] arm64: mm: Wire up TCR.DS bit to PTE shareability fields From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Catalin Marinas , Will Deacon , Marc Zyngier , Mark Rutland , Ryan Roberts , Anshuman Khandual , Kees Cook X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240214_123046_933919_4704EEF6 X-CRM114-Status: GOOD ( 15.84 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Ard Biesheuvel When LPA2 is enabled, bits 8 and 9 of page and block descriptors become part of the output address instead of carrying shareability attributes for the region in question. So avoid setting these bits if TCR.DS == 1, which means LPA2 is enabled. Signed-off-by: Ard Biesheuvel --- arch/arm64/Kconfig | 4 ++++ arch/arm64/include/asm/pgtable-hwdef.h | 1 + arch/arm64/include/asm/pgtable-prot.h | 16 ++++++++++++++-- arch/arm64/mm/mmap.c | 4 ++++ arch/arm64/mm/proc.S | 2 ++ 5 files changed, 25 insertions(+), 2 deletions(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index aa7c1d435139..8c2c36fffcf5 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -1377,6 +1377,10 @@ config ARM64_PA_BITS default 48 if ARM64_PA_BITS_48 default 52 if ARM64_PA_BITS_52 +config ARM64_LPA2 + def_bool y + depends on ARM64_PA_BITS_52 && !ARM64_64K_PAGES + choice prompt "Endianness" default CPU_LITTLE_ENDIAN diff --git a/arch/arm64/include/asm/pgtable-hwdef.h b/arch/arm64/include/asm/pgtable-hwdef.h index e4944d517c99..b770f98fc0b5 100644 --- a/arch/arm64/include/asm/pgtable-hwdef.h +++ b/arch/arm64/include/asm/pgtable-hwdef.h @@ -284,6 +284,7 @@ #define TCR_E0PD1 (UL(1) << 56) #define TCR_TCMA0 (UL(1) << 57) #define TCR_TCMA1 (UL(1) << 58) +#define TCR_DS (UL(1) << 59) /* * TTBR. diff --git a/arch/arm64/include/asm/pgtable-prot.h b/arch/arm64/include/asm/pgtable-prot.h index 63ced9ccec21..dd9ee67d1d87 100644 --- a/arch/arm64/include/asm/pgtable-prot.h +++ b/arch/arm64/include/asm/pgtable-prot.h @@ -30,8 +30,8 @@ #define _PROT_DEFAULT (PTE_TYPE_PAGE | PTE_AF | PTE_SHARED) #define _PROT_SECT_DEFAULT (PMD_TYPE_SECT | PMD_SECT_AF | PMD_SECT_S) -#define PROT_DEFAULT (_PROT_DEFAULT | PTE_MAYBE_NG) -#define PROT_SECT_DEFAULT (_PROT_SECT_DEFAULT | PMD_MAYBE_NG) +#define PROT_DEFAULT (PTE_TYPE_PAGE | PTE_MAYBE_NG | PTE_MAYBE_SHARED | PTE_AF) +#define PROT_SECT_DEFAULT (PMD_TYPE_SECT | PMD_MAYBE_NG | PMD_MAYBE_SHARED | PMD_SECT_AF) #define PROT_DEVICE_nGnRnE (PROT_DEFAULT | PTE_PXN | PTE_UXN | PTE_WRITE | PTE_ATTRINDX(MT_DEVICE_nGnRnE)) #define PROT_DEVICE_nGnRE (PROT_DEFAULT | PTE_PXN | PTE_UXN | PTE_WRITE | PTE_ATTRINDX(MT_DEVICE_nGnRE)) @@ -67,7 +67,19 @@ extern bool arm64_use_ng_mappings; #define PTE_MAYBE_NG (arm64_use_ng_mappings ? PTE_NG : 0) #define PMD_MAYBE_NG (arm64_use_ng_mappings ? PMD_SECT_NG : 0) +#ifndef CONFIG_ARM64_LPA2 #define lpa2_is_enabled() false +#define PTE_MAYBE_SHARED PTE_SHARED +#define PMD_MAYBE_SHARED PMD_SECT_S +#else +static inline bool __pure lpa2_is_enabled(void) +{ + return read_tcr() & TCR_DS; +} + +#define PTE_MAYBE_SHARED (lpa2_is_enabled() ? 0 : PTE_SHARED) +#define PMD_MAYBE_SHARED (lpa2_is_enabled() ? 0 : PMD_SECT_S) +#endif /* * If we have userspace only BTI we don't want to mark kernel pages diff --git a/arch/arm64/mm/mmap.c b/arch/arm64/mm/mmap.c index 645fe60d000f..642bdf908b22 100644 --- a/arch/arm64/mm/mmap.c +++ b/arch/arm64/mm/mmap.c @@ -73,6 +73,10 @@ static int __init adjust_protection_map(void) protection_map[VM_EXEC | VM_SHARED] = PAGE_EXECONLY; } + if (lpa2_is_enabled()) + for (int i = 0; i < ARRAY_SIZE(protection_map); i++) + pgprot_val(protection_map[i]) &= ~PTE_SHARED; + return 0; } arch_initcall(adjust_protection_map); diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index 6e1b2bc41a9f..7c46f8cfd6ae 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -466,6 +466,7 @@ alternative_else_nop_endif */ #define PTE_MAYBE_NG 0 +#define PTE_MAYBE_SHARED 0 mov_q x0, PIE_E0 msr REG_PIRE0_EL1, x0 @@ -473,6 +474,7 @@ alternative_else_nop_endif msr REG_PIR_EL1, x0 #undef PTE_MAYBE_NG +#undef PTE_MAYBE_SHARED mov x0, TCR2_EL1x_PIE msr REG_TCR2_EL1, x0