From patchwork Tue Jan 23 21:16:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13528143 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B2C98C47DDB for ; Tue, 23 Jan 2024 21:16:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References:Message-Id :MIME-Version:Subject:Date:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=LYn1vaKJCyWDb+eW847i+rcG1CKg7eWTNWqILubo8ro=; b=AZFd8WFy5EiJPJ m9d0koSKMhSjRgz55aI/JgtBnBCcEsrR+kBiZuNXCuGD0eOIJXE8otg2P0yblXSYTLHJzSEQUYQsI 5aIGmm7rBi/HzSqOS1vx1R0CWATBG9cWKJIZxXquPlVj7y6S0b2pUpCQDn48V4O3t0MCAhw1k2OBm VfvFMhSF2TuYKkPt/FA9rnEsEydQrfD3lBis7fZYU5amy5RkAYXYF8UOSKPK+9ybpUu3NrASdYiWo +PV2Mc6eXWD9cP40WHyu1O/XO8wrmRc0IJi8MYOxBrQjvrzaozrtO6ilCqUxRlx9X47R2ZrABJCVu osSWyULfuZRZ1wy4T3Qg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1rSO88-000Uv9-0R; Tue, 23 Jan 2024 21:16:24 +0000 Received: from mail-lf1-x133.google.com ([2a00:1450:4864:20::133]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1rSO84-000UsB-0p for linux-arm-kernel@lists.infradead.org; Tue, 23 Jan 2024 21:16:21 +0000 Received: by mail-lf1-x133.google.com with SMTP id 2adb3069b0e04-50ed808db11so5274276e87.2 for ; Tue, 23 Jan 2024 13:16:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1706044576; x=1706649376; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=ZIS+IHRcd0ERLxt100cpcLsozLhRREys7xiPQsR20Dk=; b=cSdS1QHWZsX2VCxZZG4QeeqKqeIXLY5Jw5bFSZG3qkvolyYmTRFIxhK0n51UQNeGFg 0uLfQtB2KHi/XtBV6srbgSfFMxtvEhuuQ27dz9f2e54iUmoaKsGmrZY0bH+iCo2F+Fxt Qn+4AGXEIjeBQbdMTnjTyGCemdP/W8jO5eMBWOxsiC0LVIS5xqe5paY2KOotbGdvX4fo btQQ6TJ3lAJrYB1PuaCcU/UJKW/r61sA5jpzhgliraTKHh22lvFJgZm5hRPrcWyd2hw0 75FEpOQLaBWS7ai2ejXFm6PGNyfPdPRI/1HzY/MyOHKeIYKdTrqzyBTi9c7vi6TtQGjG LJLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706044576; x=1706649376; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZIS+IHRcd0ERLxt100cpcLsozLhRREys7xiPQsR20Dk=; b=ZP3nhJI1jDeY0tmeUTeJ0o6RvKqBcn3xlnCa81o8xKx1phJWs+Zf1DDqwpP3ql5e9i jemJrDMGA1fJbNn9nHx8jKO66nltMNZrsAUWV7u0RFA5kTt0DwbCk7GsPAoZLFVnMojK 797AKG7mmced15/UQgNG9LEww9DP9T9QpLcmrR57Y2qkt/ZEcZJwaAFAkoQk86d6M2II rINmApzY7aYLNhDwXZu190IJdqjwGJM42Y1RB+MRQeE8biTLVqOJi2L4to4a7l+1EkX7 OplLJdh9LKflXlLGiODjI13AR8b7wiFYSbMSEKJIn1zj/J2RJ1rI3yZV9+zArSc4YBkc bz1Q== X-Gm-Message-State: AOJu0YxKsy6iJoYEmTFyihOhhfnEjCyNDs4BCs60bU9NEzwrC6oZhk+9 CcuhFh/i+QJuE2cllXN4/uqpf54EvdpyTOxf0OQDZOf0EtgKu3PLkWf8Mo+rhEU= X-Google-Smtp-Source: AGHT+IET/n6ywAQzEiobrpHmqsX1BJBCCffmlb8CRfmv4f7ReSApZJHE/vFVkDEQUu12tQC5dkhmSA== X-Received: by 2002:a19:654e:0:b0:510:676:7966 with SMTP id c14-20020a19654e000000b0051006767966mr814968lfj.59.1706044576472; Tue, 23 Jan 2024 13:16:16 -0800 (PST) Received: from [127.0.1.1] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id o3-20020a056512230300b0050ee557f1dcsm2385427lfu.115.2024.01.23.13.16.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 Jan 2024 13:16:16 -0800 (PST) From: Linus Walleij Date: Tue, 23 Jan 2024 22:16:14 +0100 Subject: [PATCH 1/4] ARM: Add TTBCR_* definitions to pgtable-3level-hwdef.h MIME-Version: 1.0 Message-Id: <20240123-arm32-lpae-pan-v1-1-7ea98a20514c@linaro.org> References: <20240123-arm32-lpae-pan-v1-0-7ea98a20514c@linaro.org> In-Reply-To: <20240123-arm32-lpae-pan-v1-0-7ea98a20514c@linaro.org> To: Russell King , Ard Biesheuvel , Arnd Bergmann , Stefan Wahren , Kees Cook , Geert Uytterhoeven Cc: linux-arm-kernel@lists.infradead.org, Linus Walleij , Catalin Marinas X-Mailer: b4 0.12.4 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240123_131620_291631_B03BB311 X-CRM114-Status: GOOD ( 12.40 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Catalin Marinas These macros will be used in a subsequent patch. At one point these were part of the ARM32 KVM but that is no longer the case. Since these macros are only relevant to LPAE kernel builds, they are added to pgtable-3level-hwdef.h Signed-off-by: Catalin Marinas Reviewed-by: Kees Cook Signed-off-by: Linus Walleij --- arch/arm/include/asm/pgtable-3level-hwdef.h | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/arch/arm/include/asm/pgtable-3level-hwdef.h b/arch/arm/include/asm/pgtable-3level-hwdef.h index 2f35b4eddaa8..19da7753a0b8 100644 --- a/arch/arm/include/asm/pgtable-3level-hwdef.h +++ b/arch/arm/include/asm/pgtable-3level-hwdef.h @@ -94,4 +94,21 @@ #define TTBR1_SIZE (((PAGE_OFFSET >> 30) - 1) << 16) +/* + * TTBCR register bits. + */ +#define TTBCR_EAE (1 << 31) +#define TTBCR_IMP (1 << 30) +#define TTBCR_SH1_MASK (3 << 28) +#define TTBCR_ORGN1_MASK (3 << 26) +#define TTBCR_IRGN1_MASK (3 << 24) +#define TTBCR_EPD1 (1 << 23) +#define TTBCR_A1 (1 << 22) +#define TTBCR_T1SZ_MASK (7 << 16) +#define TTBCR_SH0_MASK (3 << 12) +#define TTBCR_ORGN0_MASK (3 << 10) +#define TTBCR_IRGN0_MASK (3 << 8) +#define TTBCR_EPD0 (1 << 7) +#define TTBCR_T0SZ_MASK (7 << 0) + #endif From patchwork Tue Jan 23 21:16:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13528142 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5A42DC47258 for ; Tue, 23 Jan 2024 21:16:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References:Message-Id :MIME-Version:Subject:Date:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=z1azzH9nLTudzAFHeEdfivGL9a4LPg3lUWA0U94a8t8=; b=nv39sZssAN3GRn kIH32povoSh5nNInB94s+F1MvbNsjgPh4IWa0NpLCdyBKyaQQ+YhsDvIAJrhxy0gaDw2aTM2TvSVz n6g3tptIGtlYMatl+lm3jSHp6Lyd6+UA3kgoJRdUAs6vUYZbQlqU147DKD65SEYruvY7ih0LOtZbq dcfc+ssmjvWWWLBBZ+Vrlzf2w54bKfqZS+3k1bg6G7X1DiKTcPDb89liqs5L6Pe5X9vy03J9DNE5Z taKovIpgpIsidiKXbj4Glc/l394HQDABRLPDtBU/FkhBh8IWvsUcdCgRICA9dVSR6AeASbfgbcddz 5R06GVAQMtvPIrCbWqyw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1rSO8H-000UwH-32; Tue, 23 Jan 2024 21:16:33 +0000 Received: from mail-lf1-x12a.google.com ([2a00:1450:4864:20::12a]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1rSO85-000UsT-1O for linux-arm-kernel@lists.infradead.org; Tue, 23 Jan 2024 21:16:22 +0000 Received: by mail-lf1-x12a.google.com with SMTP id 2adb3069b0e04-50eac018059so6654528e87.0 for ; Tue, 23 Jan 2024 13:16:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1706044577; x=1706649377; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=Tfvmlb2pBeVG0ZpgnBJ6vzKJTvgDXiBmio0NoiXFRrY=; b=mV6mdOvXMutI7EOaxJeCWzwlP+nEIy5LSV7fIGeQInTihj/JGN8ZcYTHF9DjMeWTiX x/ODOm4Un9ooaLNxoKQxJwNX5OUGS3D6JorJWeonEgrOtLgzg/zswALox6hhk2S3Ww0r I7HiO3JnJq0Yg7SbGEJcGESUlMaLnc1IdW11r+Kt20vHbV4ZqD/3h5px7s0nUk9YmJbR Lz4246mkOmK6idutwbLBVFsKoSpWt+GunvmIc506uUkjFGn3EpSuoNTJVrH76de7L2iT 6qaTdkdYDAbtxNiokdwQ8pi5Mu+TDLGY4ZZJTaaj8EvhKfDEIn/us+Z1C+NfPJyyhOuQ 6gFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706044577; x=1706649377; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Tfvmlb2pBeVG0ZpgnBJ6vzKJTvgDXiBmio0NoiXFRrY=; b=SIhCBEwLbFHHkTD4pRw+y6uaYnPRIA4D51e6hDmdIFURE1uZUfxoyumIRMTa5wu6bQ QCq+i+x0jkbBA9OXQaonE0LUvDajj1/gqyBsev5VOUCnnt5xcujLTtAtX56lf7OOP0hN NzVSbLkj7i+wkXWXupWo2s3AfEtFCLaE74Bv+6KOzq7VhGkPT5LBOHx/WcXBEaDIaMbT CFOjn3OU9bADZ6wbwa8Zs7JPXp/adu8WOhL5D/ISaiYMp8pG71KFMZAezlcoIw3PTM5d 6LpqTOVxG9ORLosyh78Qexr+/m1i1/XFCt1DYG31LtzkzuxgDU2PnKjnC68O2eyTABAv /ziQ== X-Gm-Message-State: AOJu0YzUrJgUSni4u1J48vwMI6k4Ryldd6pAOddlKT/RXndcg6tCZqJ7 uFZ0nTPS42i0wH+OTbL8Vbc5tdsoqnOcpd1JRl9za0UZU3tqiT97u1TWKa7eMQQ= X-Google-Smtp-Source: AGHT+IHkxXBmx1CjiTx9zzTfFegYsN0fFRX1UJi1psE+7kIxbHKYpH19QhOgkvtW7CVPfuFHo0oNFw== X-Received: by 2002:a05:6512:6d2:b0:50b:e713:574d with SMTP id u18-20020a05651206d200b0050be713574dmr3924429lff.75.1706044577498; Tue, 23 Jan 2024 13:16:17 -0800 (PST) Received: from [127.0.1.1] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id o3-20020a056512230300b0050ee557f1dcsm2385427lfu.115.2024.01.23.13.16.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 Jan 2024 13:16:16 -0800 (PST) From: Linus Walleij Date: Tue, 23 Jan 2024 22:16:15 +0100 Subject: [PATCH 2/4] ARM: Move asm statements accessing TTBCR into C functions MIME-Version: 1.0 Message-Id: <20240123-arm32-lpae-pan-v1-2-7ea98a20514c@linaro.org> References: <20240123-arm32-lpae-pan-v1-0-7ea98a20514c@linaro.org> In-Reply-To: <20240123-arm32-lpae-pan-v1-0-7ea98a20514c@linaro.org> To: Russell King , Ard Biesheuvel , Arnd Bergmann , Stefan Wahren , Kees Cook , Geert Uytterhoeven Cc: linux-arm-kernel@lists.infradead.org, Linus Walleij , Catalin Marinas X-Mailer: b4 0.12.4 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240123_131621_489421_EA9F600F X-CRM114-Status: GOOD ( 13.56 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Catalin Marinas This patch implements cpu_get_ttbcr() and cpu_set_ttbcr() and replaces the corresponding asm statements. Signed-off-by: Catalin Marinas Reviewed-by: Kees Cook Signed-off-by: Linus Walleij --- arch/arm/include/asm/proc-fns.h | 12 ++++++++++++ arch/arm/mm/mmu.c | 7 +++---- 2 files changed, 15 insertions(+), 4 deletions(-) diff --git a/arch/arm/include/asm/proc-fns.h b/arch/arm/include/asm/proc-fns.h index 280396483f5d..a13e5cf3d1ff 100644 --- a/arch/arm/include/asm/proc-fns.h +++ b/arch/arm/include/asm/proc-fns.h @@ -178,6 +178,18 @@ extern void cpu_resume(void); }) #endif +static inline unsigned int cpu_get_ttbcr(void) +{ + unsigned int ttbcr; + asm("mrc p15, 0, %0, c2, c0, 2" : "=r" (ttbcr)); + return ttbcr; +} + +static inline void cpu_set_ttbcr(unsigned int ttbcr) +{ + asm volatile("mcr p15, 0, %0, c2, c0, 2" : : "r" (ttbcr)); +} + #else /*!CONFIG_MMU */ #define cpu_switch_mm(pgd,mm) { } diff --git a/arch/arm/mm/mmu.c b/arch/arm/mm/mmu.c index 674ed71573a8..9a780da6a4e1 100644 --- a/arch/arm/mm/mmu.c +++ b/arch/arm/mm/mmu.c @@ -1687,9 +1687,8 @@ static void __init early_paging_init(const struct machine_desc *mdesc) */ cr = get_cr(); set_cr(cr & ~(CR_I | CR_C)); - asm("mrc p15, 0, %0, c2, c0, 2" : "=r" (ttbcr)); - asm volatile("mcr p15, 0, %0, c2, c0, 2" - : : "r" (ttbcr & ~(3 << 8 | 3 << 10))); + ttbcr = cpu_get_ttbcr(); + cpu_set_ttbcr(ttbcr & ~(3 << 8 | 3 << 10)); flush_cache_all(); /* @@ -1701,7 +1700,7 @@ static void __init early_paging_init(const struct machine_desc *mdesc) lpae_pgtables_remap(offset, pa_pgd); /* Re-enable the caches and cacheable TLB walks */ - asm volatile("mcr p15, 0, %0, c2, c0, 2" : : "r" (ttbcr)); + cpu_set_ttbcr(ttbcr); set_cr(cr); } From patchwork Tue Jan 23 21:16:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13528145 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 75B6AC47DDC for ; Tue, 23 Jan 2024 21:17:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References:Message-Id :MIME-Version:Subject:Date:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=j/40Al3w8cdRaHdCYdvkGqQ6OXDOcjKPN5zVYwMIAUE=; b=ek52QRWNBkJ2vz 76UP/3XXRViU0DfwQ+DJpmkPm6HfmQWVimT+2W75FXecAo4iq8S/iPjv80XFbEmeQqDwlm7bYJm2Q NVDrK3Yvp3FR0VqIyz79kN4iNteAjwTLHZZ6HH+ehCh0t/2uuYaS8setQRLkIYiJgQ/0WXSESoZvb r2tjF7bIONA3TWfEGBLKIRPZ5oo5NBSoLNL8AwPlMQ/ncN8gVm2gRb2b8Dw/vpApEz5ZvYir/u266 /j0OQU3SUJAaHCyyRq4GRHR+mzbBfRPzF7giYhVJrKXRV5C5tct8CfxT57zi7LnMydbsjKL56PRIF 8m8jWs8iATCjyv0Ys92w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1rSO8I-000UwZ-1b; Tue, 23 Jan 2024 21:16:34 +0000 Received: from mail-lf1-x12a.google.com ([2a00:1450:4864:20::12a]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1rSO85-000Usv-1O for linux-arm-kernel@lists.infradead.org; Tue, 23 Jan 2024 21:16:22 +0000 Received: by mail-lf1-x12a.google.com with SMTP id 2adb3069b0e04-50e78f1f41fso4982044e87.2 for ; Tue, 23 Jan 2024 13:16:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1706044578; x=1706649378; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=ZFpflVTrSqZq+OIvSg55/4vxK9r3hNXp8u8Whz8PYyI=; b=LcEyo0rf3iEN+r693SU2YKOt+YNyVtXBCOwW6qePysh3KzYHB0OGkYMimmQxnOd/mv OZVT/bL97hvnvFfub+ReWrE+M1FrSjAB19rfVRgE0+cm8YvgXCC5yvHTHnGmresqu4cx u39LH01Zjv3X/8dY3uKKnDmVgR+SUliOUGKR0v01/DVXeT1YniRDKsyRJrEckDqvgLhl ROEcKqxaor5OOuSxve3a3SGFFa7YGInoTInuI9VjieokjsaIUaTVNMxH2T0RCMGKE86I T6BRXqK+WqscPND2mZfOANmk5t9/QOTpf685Z0Qqkvj3s4AVy1RahCNUC/eQzH0/fy23 Jk2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706044578; x=1706649378; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZFpflVTrSqZq+OIvSg55/4vxK9r3hNXp8u8Whz8PYyI=; b=veSXNX64wBIPcJ7ufUNZgMkjQaeTuurDCf5P/PpaSCUEUVOYchrTdBYUIo0uX2oG+h swjJodBQe2dlkMVjV+BXAYEyLplgn0ExPYxvQdpYJ4IiJKK+xw4lwFAhs+bIXMuKgj4r DNxZXh2V8YTZAWSnrka88Fi170czH0o5M7UGiqSVX+J1xB+7JJXF2qHvDSR8JiOhlqGZ CWt1LPaRXqFJepjf4OK2xlAYrX6rI3S+s52ewJv8VJFo3vFWpQGHEASOqE1aRAyFKYUp FQcQfY0GVVpzCgoMnJKQhq2rJMg+sHWxHlxtBLMyrhQJElhlycJHgbLP220FdDvmFH2R QRaw== X-Gm-Message-State: AOJu0YydPValyAB9kMG8ie0hqvUTaJ4e1ssO3kHxkb3QfFZUKzfCeGjq CqEuoPVVWx7GnYfv0fTHm/iqiev3HV9yVFRebSyaZh2jNz26O2i/9VzT+p2l4iE= X-Google-Smtp-Source: AGHT+IHUM+yRzGSe7jvZ4KD5VGbUBI0KsEq96+OcT27I68DjmypSiqYTQwfGQDw26dh74B9govTXTw== X-Received: by 2002:a05:6512:3150:b0:50e:e173:b02f with SMTP id s16-20020a056512315000b0050ee173b02fmr2533201lfi.63.1706044578297; Tue, 23 Jan 2024 13:16:18 -0800 (PST) Received: from [127.0.1.1] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id o3-20020a056512230300b0050ee557f1dcsm2385427lfu.115.2024.01.23.13.16.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 Jan 2024 13:16:18 -0800 (PST) From: Linus Walleij Date: Tue, 23 Jan 2024 22:16:16 +0100 Subject: [PATCH 3/4] ARM: Reduce the number of #ifdef CONFIG_CPU_SW_DOMAIN_PAN MIME-Version: 1.0 Message-Id: <20240123-arm32-lpae-pan-v1-3-7ea98a20514c@linaro.org> References: <20240123-arm32-lpae-pan-v1-0-7ea98a20514c@linaro.org> In-Reply-To: <20240123-arm32-lpae-pan-v1-0-7ea98a20514c@linaro.org> To: Russell King , Ard Biesheuvel , Arnd Bergmann , Stefan Wahren , Kees Cook , Geert Uytterhoeven Cc: linux-arm-kernel@lists.infradead.org, Linus Walleij , Catalin Marinas X-Mailer: b4 0.12.4 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240123_131621_489160_FEFA247E X-CRM114-Status: GOOD ( 15.84 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Catalin Marinas This is a clean-up patch aimed at reducing the number of checks on CONFIG_CPU_SW_DOMAIN_PAN, together with some empty lines for better clarity once the CONFIG_CPU_TTBR0_PAN is introduced. Signed-off-by: Catalin Marinas Reviewed-by: Kees Cook Signed-off-by: Linus Walleij --- arch/arm/include/asm/uaccess-asm.h | 16 ++++++++++++---- arch/arm/include/asm/uaccess.h | 21 +++++++++++++++------ arch/arm/lib/csumpartialcopyuser.S | 6 +++++- 3 files changed, 32 insertions(+), 11 deletions(-) diff --git a/arch/arm/include/asm/uaccess-asm.h b/arch/arm/include/asm/uaccess-asm.h index 65da32e1f1c1..ea42ba25920f 100644 --- a/arch/arm/include/asm/uaccess-asm.h +++ b/arch/arm/include/asm/uaccess-asm.h @@ -39,8 +39,9 @@ #endif .endm - .macro uaccess_disable, tmp, isb=1 #ifdef CONFIG_CPU_SW_DOMAIN_PAN + + .macro uaccess_disable, tmp, isb=1 /* * Whenever we re-enter userspace, the domains should always be * set appropriately. @@ -50,11 +51,9 @@ .if \isb instr_sync .endif -#endif .endm .macro uaccess_enable, tmp, isb=1 -#ifdef CONFIG_CPU_SW_DOMAIN_PAN /* * Whenever we re-enter userspace, the domains should always be * set appropriately. @@ -64,9 +63,18 @@ .if \isb instr_sync .endif -#endif .endm +#else + + .macro uaccess_disable, tmp, isb=1 + .endm + + .macro uaccess_enable, tmp, isb=1 + .endm + +#endif + #if defined(CONFIG_CPU_SW_DOMAIN_PAN) || defined(CONFIG_CPU_USE_DOMAINS) #define DACR(x...) x #else diff --git a/arch/arm/include/asm/uaccess.h b/arch/arm/include/asm/uaccess.h index 9556d04387f7..9b9234d1bb6a 100644 --- a/arch/arm/include/asm/uaccess.h +++ b/arch/arm/include/asm/uaccess.h @@ -24,9 +24,10 @@ * perform such accesses (eg, via list poison values) which could then * be exploited for priviledge escalation. */ +#if defined(CONFIG_CPU_SW_DOMAIN_PAN) + static __always_inline unsigned int uaccess_save_and_enable(void) { -#ifdef CONFIG_CPU_SW_DOMAIN_PAN unsigned int old_domain = get_domain(); /* Set the current domain access to permit user accesses */ @@ -34,19 +35,27 @@ static __always_inline unsigned int uaccess_save_and_enable(void) domain_val(DOMAIN_USER, DOMAIN_CLIENT)); return old_domain; -#else - return 0; -#endif } static __always_inline void uaccess_restore(unsigned int flags) { -#ifdef CONFIG_CPU_SW_DOMAIN_PAN /* Restore the user access mask */ set_domain(flags); -#endif } +#else + +static inline unsigned int uaccess_save_and_enable(void) +{ + return 0; +} + +static inline void uaccess_restore(unsigned int flags) +{ +} + +#endif + /* * These two are intentionally not defined anywhere - if the kernel * code generates any references to them, that's a bug. diff --git a/arch/arm/lib/csumpartialcopyuser.S b/arch/arm/lib/csumpartialcopyuser.S index 6928781e6bee..04d8d9d741c7 100644 --- a/arch/arm/lib/csumpartialcopyuser.S +++ b/arch/arm/lib/csumpartialcopyuser.S @@ -13,7 +13,8 @@ .text -#ifdef CONFIG_CPU_SW_DOMAIN_PAN +#if defined(CONFIG_CPU_SW_DOMAIN_PAN) + .macro save_regs mrc p15, 0, ip, c3, c0, 0 stmfd sp!, {r1, r2, r4 - r8, ip, lr} @@ -25,7 +26,9 @@ mcr p15, 0, ip, c3, c0, 0 ret lr .endm + #else + .macro save_regs stmfd sp!, {r1, r2, r4 - r8, lr} .endm @@ -33,6 +36,7 @@ .macro load_regs ldmfd sp!, {r1, r2, r4 - r8, pc} .endm + #endif .macro load1b, reg1 From patchwork Tue Jan 23 21:16:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13528144 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 97460C47258 for ; Tue, 23 Jan 2024 21:17:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References:Message-Id :MIME-Version:Subject:Date:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=jksVBOlQXJQMl3iwgOZPm8wk+SCXDG5lq7CAMzVq9wk=; b=fL84rBRAl7lbMU uFQQxd1y4ze+TwUUotGUuabhMN5uIRIPUyigZKMJkJH90dFGQRkHpYPWUooAtgZo/m4/J+fVPjWoA /k1xzCyoeMApY7n2wd8XbHAsDI1526CyQgb1kgUn5Xwz4Lm8Q+qlrpOa3SbnGHKjX7prWyWQQ4ZOy 9p30c3wg+qtJVafvPYAA/Pn+vzrIZRVzeCsGPxB5eJPYiIzMx6MVyCbSy1DlUwvwXu3rVmOZCzNT3 NCnjPtFLI3JNQLK4dov7P9BRnrfEnRkGRXCwGVHOjfWbnvifQqJrJEznuggPdQGjXGGkL+YHlSL5p ffehxnA6hwpGtoJlrTBw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1rSO8J-000Uwu-0L; Tue, 23 Jan 2024 21:16:35 +0000 Received: from mail-lf1-x131.google.com ([2a00:1450:4864:20::131]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1rSO86-000Usx-2r for linux-arm-kernel@lists.infradead.org; Tue, 23 Jan 2024 21:16:24 +0000 Received: by mail-lf1-x131.google.com with SMTP id 2adb3069b0e04-5100c4aa08cso915010e87.2 for ; Tue, 23 Jan 2024 13:16:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1706044579; x=1706649379; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=S4HuAed7r8NaCT3Mg24cZLRyp9JCwTaXTMOG4V4IzqM=; b=k7uudRbopjAzhF/eqgpgXU+dOUqQ/AGFuaVpNTtt/GT/myxiz7r/Xs2PRhaGxd4z57 ySNiT6tljpaAnsEcLA79mI7JB8TRW2pWcph/5IFVoDbKAk7Y28DRlMmInnbjcbjriOmk 4S9PtnQkxBuauzIsID3uhnbrYq/4HQsiwf+MlyuotYvypPiByMlxnyhtZnUibVl2lhsZ oU7B4OXRumNKw6d1c1g30+njXWX6NnmYWET9ZZzYAdl5L1azHR+vnz4Tmv+oSdZjYTH+ 72G4No91ZzXjwP+c4/AJ7j2LeNyQjcggq2+y8dLauVBHJ6eMXKPkti7xTVMhLN7S6TPU ro4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706044579; x=1706649379; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=S4HuAed7r8NaCT3Mg24cZLRyp9JCwTaXTMOG4V4IzqM=; b=EcaEnAglFCGAu+GPC0pOuxc8MYLmWDYznqq9VMi/iCi7qxbrJJ7OHZFxNhkGizDsBL xBZiq0vnM9qm+JEfOPQkrM7mbDFiuJwhx/bFZLHQ47efCoBuIyPT7IFRyf6nf/Gt4AcF sGFAtId4AIiKoGvl5yAkmcBKzepn6uj6fG5vZLqkAaADkzg+NilZgWfXxgcEJZvsQuDR OE8ul1pTtZs79c6sqMrfpwqyC9nBLHwgu4zX0Slf6ZPKkfYGjZjB68W9B2bmuUpJHbd9 DgoUWXhUUR6NhFc2fzc9luYZj3WFfUJ1jCkFJbNrpViMkv9RDhSNqSy20uZMk9fF3HDo DlXw== X-Gm-Message-State: AOJu0YzyaKzIP0ERDrcuaK6634zGyk0V/f8rPvWQSk9l5YXlbzEBoXDu NGoX7ysiF1uDKEfgryUxBcS2LMTlfo7WcSdHNb/V40eWHlEb1R1rj4yo//SjWBE= X-Google-Smtp-Source: AGHT+IGG1AwmnWQjvYCQv0WxbgAhHpPq2iY7e6YlEgIMJE2nLWDV3iShgcLgP57zfMcwdo3nBvxBJQ== X-Received: by 2002:a05:6512:2088:b0:510:ecc:18b1 with SMTP id t8-20020a056512208800b005100ecc18b1mr92660lfr.138.1706044579046; Tue, 23 Jan 2024 13:16:19 -0800 (PST) Received: from [127.0.1.1] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id o3-20020a056512230300b0050ee557f1dcsm2385427lfu.115.2024.01.23.13.16.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 Jan 2024 13:16:18 -0800 (PST) From: Linus Walleij Date: Tue, 23 Jan 2024 22:16:17 +0100 Subject: [PATCH 4/4] ARM: Implement privileged no-access using TTBR0 page table walks disabling MIME-Version: 1.0 Message-Id: <20240123-arm32-lpae-pan-v1-4-7ea98a20514c@linaro.org> References: <20240123-arm32-lpae-pan-v1-0-7ea98a20514c@linaro.org> In-Reply-To: <20240123-arm32-lpae-pan-v1-0-7ea98a20514c@linaro.org> To: Russell King , Ard Biesheuvel , Arnd Bergmann , Stefan Wahren , Kees Cook , Geert Uytterhoeven Cc: linux-arm-kernel@lists.infradead.org, Linus Walleij , Catalin Marinas X-Mailer: b4 0.12.4 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240123_131622_929208_F7B55BC0 X-CRM114-Status: GOOD ( 29.36 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Catalin Marinas With LPAE enabled, privileged no-access cannot be enforced using CPU domains as such feature is not available. This patch implements PAN by disabling TTBR0 page table walks while in kernel mode. The ARM architecture allows page table walks to be split between TTBR0 and TTBR1. With LPAE enabled, the split is defined by a combination of TTBCR T0SZ and T1SZ bits. Currently, an LPAE-enabled kernel uses TTBR0 for user addresses and TTBR1 for kernel addresses with the VMSPLIT_2G and VMSPLIT_3G configurations. The main advantage for the 3:1 split is that TTBR1 is reduced to 2 levels, so potentially faster TLB refill (though usually the first level entries are already cached in the TLB). The PAN support on LPAE-enabled kernels uses TTBR0 when running in user space or in kernel space during user access routines (TTBCR T0SZ and T1SZ are both 0). When running user accesses are disabled in kernel mode, TTBR0 page table walks are disabled by setting TTBCR.EPD0. TTBR1 is used for kernel accesses (including loadable modules; anything covered by swapper_pg_dir) by reducing the TTBCR.T0SZ to the minimum (2^(32-7) = 32MB). To avoid user accesses potentially hitting stale TLB entries, the ASID is switched to 0 (reserved) by setting TTBCR.A1 and using the ASID value in TTBR1. The difference from a non-PAN kernel is that with the 3:1 memory split, TTBR1 always uses 3 levels of page tables. Signed-off-by: Catalin Marinas Reviewed-by: Kees Cook Signed-off-by: Linus Walleij --- arch/arm/Kconfig | 22 ++++++++++++-- arch/arm/include/asm/assembler.h | 1 + arch/arm/include/asm/pgtable-3level-hwdef.h | 9 ++++++ arch/arm/include/asm/uaccess-asm.h | 42 ++++++++++++++++++++++++++ arch/arm/include/asm/uaccess.h | 47 +++++++++++++++++++++++++++++ arch/arm/kernel/suspend.c | 8 +++++ arch/arm/lib/csumpartialcopyuser.S | 14 +++++++++ arch/arm/mm/fault.c | 8 +++++ 8 files changed, 148 insertions(+), 3 deletions(-) diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig index 0af6709570d1..3d97a15a3e2d 100644 --- a/arch/arm/Kconfig +++ b/arch/arm/Kconfig @@ -1231,9 +1231,9 @@ config HIGHPTE consumed by page tables. Setting this option will allow user-space 2nd level page tables to reside in high memory. -config CPU_SW_DOMAIN_PAN - bool "Enable use of CPU domains to implement privileged no-access" - depends on MMU && !ARM_LPAE +config ARM_PAN + bool "Enable privileged no-access" + depends on MMU default y help Increase kernel security by ensuring that normal kernel accesses @@ -1242,10 +1242,26 @@ config CPU_SW_DOMAIN_PAN by ensuring that magic values (such as LIST_POISON) will always fault when dereferenced. + The implementation uses CPU domains when !CONFIG_ARM_LPAE and + disabling of TTBR0 page table walks with CONFIG_ARM_LPAE. + +config CPU_SW_DOMAIN_PAN + def_bool y + depends on ARM_PAN && !ARM_LPAE + help + Enable use of CPU domains to implement privileged no-access. + CPUs with low-vector mappings use a best-efforts implementation. Their lower 1MB needs to remain accessible for the vectors, but the remainder of userspace will become appropriately inaccessible. +config CPU_TTBR0_PAN + def_bool y + depends on ARM_PAN && ARM_LPAE + help + Enable privileged no-access by disabling TTBR0 page table walks when + running in kernel mode. + config HW_PERF_EVENTS def_bool y depends on ARM_PMU diff --git a/arch/arm/include/asm/assembler.h b/arch/arm/include/asm/assembler.h index aebe2c8f6a68..d33c1e24e00b 100644 --- a/arch/arm/include/asm/assembler.h +++ b/arch/arm/include/asm/assembler.h @@ -21,6 +21,7 @@ #include #include #include +#include #include #include diff --git a/arch/arm/include/asm/pgtable-3level-hwdef.h b/arch/arm/include/asm/pgtable-3level-hwdef.h index 19da7753a0b8..323ad811732e 100644 --- a/arch/arm/include/asm/pgtable-3level-hwdef.h +++ b/arch/arm/include/asm/pgtable-3level-hwdef.h @@ -74,6 +74,7 @@ #define PHYS_MASK_SHIFT (40) #define PHYS_MASK ((1ULL << PHYS_MASK_SHIFT) - 1) +#ifndef CONFIG_CPU_TTBR0_PAN /* * TTBR0/TTBR1 split (PAGE_OFFSET): * 0x40000000: T0SZ = 2, T1SZ = 0 (not used) @@ -93,6 +94,14 @@ #endif #define TTBR1_SIZE (((PAGE_OFFSET >> 30) - 1) << 16) +#else +/* + * With CONFIG_CPU_TTBR0_PAN enabled, TTBR1 is only used during uaccess + * disabled regions when TTBR0 is disabled. + */ +#define TTBR1_OFFSET 0 /* pointing to swapper_pg_dir */ +#define TTBR1_SIZE 0 /* TTBR1 size controlled via TTBCR.T0SZ */ +#endif /* * TTBCR register bits. diff --git a/arch/arm/include/asm/uaccess-asm.h b/arch/arm/include/asm/uaccess-asm.h index ea42ba25920f..f7acf4cabbdc 100644 --- a/arch/arm/include/asm/uaccess-asm.h +++ b/arch/arm/include/asm/uaccess-asm.h @@ -65,6 +65,37 @@ .endif .endm +#elif defined(CONFIG_CPU_TTBR0_PAN) + + .macro uaccess_disable, tmp, isb=1 + /* + * Disable TTBR0 page table walks (EDP0 = 1), use the reserved ASID + * from TTBR1 (A1 = 1) and enable TTBR1 page table walks for kernel + * addresses by reducing TTBR0 range to 32MB (T0SZ = 7). + */ + mrc p15, 0, \tmp, c2, c0, 2 @ read TTBCR + orr \tmp, \tmp, #TTBCR_EPD0 | TTBCR_T0SZ_MASK + orr \tmp, \tmp, #TTBCR_A1 + mcr p15, 0, \tmp, c2, c0, 2 @ write TTBCR + .if \isb + instr_sync + .endif + .endm + + .macro uaccess_enable, tmp, isb=1 + /* + * Enable TTBR0 page table walks (T0SZ = 0, EDP0 = 0) and ASID from + * TTBR0 (A1 = 0). + */ + mrc p15, 0, \tmp, c2, c0, 2 @ read TTBCR + bic \tmp, \tmp, #TTBCR_EPD0 | TTBCR_T0SZ_MASK + bic \tmp, \tmp, #TTBCR_A1 + mcr p15, 0, \tmp, c2, c0, 2 @ write TTBCR + .if \isb + instr_sync + .endif + .endm + #else .macro uaccess_disable, tmp, isb=1 @@ -79,6 +110,12 @@ #define DACR(x...) x #else #define DACR(x...) +#endif + +#ifdef CONFIG_CPU_TTBR0_PAN +#define PAN(x...) x +#else +#define PAN(x...) #endif /* @@ -94,6 +131,8 @@ .macro uaccess_entry, tsk, tmp0, tmp1, tmp2, disable DACR( mrc p15, 0, \tmp0, c3, c0, 0) DACR( str \tmp0, [sp, #SVC_DACR]) + PAN( mrc p15, 0, \tmp0, c2, c0, 2) + PAN( str \tmp0, [sp, #SVC_DACR]) .if \disable && IS_ENABLED(CONFIG_CPU_SW_DOMAIN_PAN) /* kernel=client, user=no access */ mov \tmp2, #DACR_UACCESS_DISABLE @@ -112,8 +151,11 @@ .macro uaccess_exit, tsk, tmp0, tmp1 DACR( ldr \tmp0, [sp, #SVC_DACR]) DACR( mcr p15, 0, \tmp0, c3, c0, 0) + PAN( ldr \tmp0, [sp, #SVC_DACR]) + PAN( mcr p15, 0, \tmp0, c2, c0, 2) .endm #undef DACR +#undef PAN #endif /* __ASM_UACCESS_ASM_H__ */ diff --git a/arch/arm/include/asm/uaccess.h b/arch/arm/include/asm/uaccess.h index 9b9234d1bb6a..5b542eab009f 100644 --- a/arch/arm/include/asm/uaccess.h +++ b/arch/arm/include/asm/uaccess.h @@ -14,6 +14,8 @@ #include #include #include +#include +#include #include #include @@ -43,6 +45,45 @@ static __always_inline void uaccess_restore(unsigned int flags) set_domain(flags); } +static inline bool uaccess_disabled(struct pt_regs *regs) +{ + /* + * This is handled by hardware domain checks but included for + * completeness. + */ + return !(get_domain() & domain_mask(DOMAIN_USER)); +} + +#elif defined(CONFIG_CPU_TTBR0_PAN) + +static inline unsigned int uaccess_save_and_enable(void) +{ + unsigned int old_ttbcr = cpu_get_ttbcr(); + + /* + * Enable TTBR0 page table walks (T0SZ = 0, EDP0 = 0) and ASID from + * TTBR0 (A1 = 0). + */ + cpu_set_ttbcr(old_ttbcr & ~(TTBCR_A1 | TTBCR_EPD0 | TTBCR_T0SZ_MASK)); + isb(); + + return old_ttbcr; +} + +static inline void uaccess_restore(unsigned int flags) +{ + cpu_set_ttbcr(flags); + isb(); +} + +static inline bool uaccess_disabled(struct pt_regs *regs) +{ + /* uaccess state saved above pt_regs on SVC exception entry */ + unsigned int ttbcr = *(unsigned int *)(regs + 1); + + return ttbcr & TTBCR_EPD0; +} + #else static inline unsigned int uaccess_save_and_enable(void) @@ -54,6 +95,12 @@ static inline void uaccess_restore(unsigned int flags) { } +static inline bool uaccess_disabled(struct pt_regs *regs) +{ + /* Without PAN userspace is always available */ + return false; +} + #endif /* diff --git a/arch/arm/kernel/suspend.c b/arch/arm/kernel/suspend.c index c3ec3861dd07..58a6441b58c4 100644 --- a/arch/arm/kernel/suspend.c +++ b/arch/arm/kernel/suspend.c @@ -12,6 +12,7 @@ #include #include #include +#include extern int __cpu_suspend(unsigned long, int (*)(unsigned long), u32 cpuid); extern void cpu_resume_mmu(void); @@ -26,6 +27,13 @@ int cpu_suspend(unsigned long arg, int (*fn)(unsigned long)) if (!idmap_pgd) return -EINVAL; + /* + * Needed for the MMU disabling/enabing code to be able to run from + * TTBR0 addresses. + */ + if (IS_ENABLED(CONFIG_CPU_TTBR0_PAN)) + uaccess_save_and_enable(); + /* * Function graph tracer state gets incosistent when the kernel * calls functions that never return (aka suspend finishers) hence diff --git a/arch/arm/lib/csumpartialcopyuser.S b/arch/arm/lib/csumpartialcopyuser.S index 04d8d9d741c7..c289bde04743 100644 --- a/arch/arm/lib/csumpartialcopyuser.S +++ b/arch/arm/lib/csumpartialcopyuser.S @@ -27,6 +27,20 @@ ret lr .endm +#elif defined(CONFIG_CPU_TTBR0_PAN) + + .macro save_regs + mrc p15, 0, ip, c2, c0, 2 @ read TTBCR + stmfd sp!, {r1, r2, r4 - r8, ip, lr} + uaccess_enable ip + .endm + + .macro load_regs + ldmfd sp!, {r1, r2, r4 - r8, ip, lr} + mcr p15, 0, ip, c2, c0, 2 @ restore TTBCR + ret lr + .endm + #else .macro save_regs diff --git a/arch/arm/mm/fault.c b/arch/arm/mm/fault.c index e96fb40b9cc3..de4abf9dfd6a 100644 --- a/arch/arm/mm/fault.c +++ b/arch/arm/mm/fault.c @@ -278,6 +278,14 @@ do_page_fault(unsigned long addr, unsigned int fsr, struct pt_regs *regs) perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS, 1, regs, addr); + /* + * Privileged access aborts with CONFIG_CPU_TTBR0_PAN enabled are + * routed via the translation fault mechanism. Check whether uaccess + * is disabled while in kernel mode. + */ + if (IS_ENABLED(CONFIG_CPU_TTBR0_PAN) && !user_mode(regs) && uaccess_disabled(regs)) + goto no_context; + if (!(flags & FAULT_FLAG_USER)) goto lock_mmap;