From patchwork Mon Oct 14 10:59:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834723 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DF800D1A443 for ; Mon, 14 Oct 2024 11:02:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 97B076B00EB; Mon, 14 Oct 2024 07:02:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 903CA6B00ED; Mon, 14 Oct 2024 07:02:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7A2B16B00EE; Mon, 14 Oct 2024 07:02:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 5A6DC6B00EB for ; Mon, 14 Oct 2024 07:02:16 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id EBB96ABCCA for ; Mon, 14 Oct 2024 11:02:00 +0000 (UTC) X-FDA: 82671918504.21.CD867FF Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf04.hostedemail.com (Postfix) with ESMTP id 4BAF240014 for ; Mon, 14 Oct 2024 11:02:06 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf04.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903619; a=rsa-sha256; cv=none; b=TUWZxyovyFQ34bEznMN+Hsgev1lsiAAf98q5pQ6esY8bxBEt+w6217eVwG/6P+oD84MFuM p1io+r0moDVcTRZB6SuFD0u5UE22/gnwmbwzsq9Q6gYkUkGSQX6GsE2u8B5BsJHc2Ww8j5 2rXMk+zJwuwG9j7OGy5ivaPXWbOj1Is= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf04.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903619; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BkgcTQ3nJpwwleh27xwrJdMVBeha9+gRlJOOjRb7rV4=; b=xKD3B3/xGJVAwQlG0nalqNs/1M88gbwRuXXOsKNv+uKck+01dUrWG4S2xHrA+WN+Vqvphs DgC84iwWDC7+CxmfHiqIPOZAIFSqt5jDRCss2QFv+wGywQWPwTAEbYYXxWQjaa8UeKKlZE ahJmQtBUNFjtLshjpmQfLpwUNOHh+qM= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5E32F1756; Mon, 14 Oct 2024 04:02:43 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 5E9263F51B; Mon, 14 Oct 2024 04:02:11 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Greg Marsden , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Will Deacon Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v1 55/57] arm64: TRAMP_VALIAS is no longer compile-time constant Date: Mon, 14 Oct 2024 11:59:02 +0100 Message-ID: <20241014105912.3207374-55-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 4BAF240014 X-Stat-Signature: 38k7zu73khbband9uod66p77th4s5bwt X-Rspam-User: X-HE-Tag: 1728903726-328534 X-HE-Meta: U2FsdGVkX19N8OycGUm1/+1Z4qsIJefY3N7yGym7HKBiNuBTXyQ5UJk+C/v2GscuFTx4egGe5dc88BJobT5nKOWW2uDbQy6Uo7frWm3oiOi1rEjttW0ghFck84dpaDoWHFEMttQK01D1cSoMmLKnqBGV7mPh1OPzTjCM42JG3H3YsqqRD6P119LQxv2DfhmrveCm2HAruWcfZhjSGrZi9LjUnCBBSbQTm6JyygNugKwGmL2wuVMua6uBhUSX9RUwP9dw/gGvS+3QAaQ5dE7hyhTdtLmCSYFubU5ovMTLTiX2T5dnrqjhipOswVAfPNBqY8TyCiIWulQrn7YFTTeahf7nfbCfLNqijOtBo4lMmqJKZq2rKFaLIthzhLHIc+xx0Z7M3UG4Eo0ELQ10GkjC/5BxAS7MqDxnG5eX7vW16pkrusZu73+UbRp963ZfcthQks4ocKKiviN9dSHsBzMHoDvncxz5gClpfdyfYvuZCAFvb4z0HrcuUkbPkYY4dDNRznyQhTq+igbGQfB7dFYbnooilHW7m7ckzwySjUoQVm3JvDfrCrF6xb65iiCByERB40YdTBGDs6iq2HlDIztmbtDYJQ4tWdilYtK/NYvoxl78gPQDXa2ll5q1TcMiAgSDnTzUVQw2b9qOuUS0y77R0Vlkoo3um1udcdf5cLCJ2epRaaNAD6k4fodwdE2PhJwtdOsIn3P/17SZsNmeC1iST3VpXGg56xh+bF1hxufSxxfQ1u4iXjWr1xVGTeC4qYwGPQgHbd55afDO5AO21zPoOGZd+yMXZThUaG5MPq+nlLZDx8g58l4W9LJyY5Y3rJ3Zl3jEVj3eHp1vhm42AlviKnzwcvQzwMKe58FFs/sih4WdCiSSbgU3yZiDlKzREVmypXQ1P2m+E0nHOerd3ke2ju1hkasjQiDj/K5e8muPw6ssKRQFq9opv5rXb/94b6/UwNPPOtlaAeb/6C0GrYO 4QFHwS1D 79NGLGg3pj5cuTSfTQGn/i/s+4e4CXU3//jpOATLkn0LWW25Q1U1GNdGY78TlJYUBJ5AlRqqqeo62mJrs9bstcBRcmLm48RJCOIHYE/5ZS6KrqCRdjaf3fQZfKaveZKAKI10EpBRtHsTb0w6HLbQ9SoETK/Ht0ssbIxPTVvQoB8znV7L56dZkSlme/yWIZseZr+za13dDlPRxU240+XooaK0p29pLVjbFLU9X X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When boot-time page size is in operation, TRAMP_VALIAS is no longer a compile-time constant, because the VA of a fixmap slot depends upon PAGE_SIZE. Let's handle this by instead exporting the slot index, FIX_ENTRY_TRAMP_BEGIN,to assembly, then do the TRAMP_VALIAS calculation per page size and use alternatives to decide which variant to activate. Note that for the tramp_map_kernel case, we are one instruction short of space in the vector to have NOPs for all 3 page size variants. So we do if/else for 16K/64K and branch around it for the 4K case. This saves 2 instructions. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ arch/arm64/kernel/asm-offsets.c | 2 +- arch/arm64/kernel/entry.S | 50 ++++++++++++++++++++++++++------- 2 files changed, 41 insertions(+), 11 deletions(-) diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index f32b8d7f00b2a..c45fa3e281884 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -172,7 +172,7 @@ int main(void) DEFINE(ARM64_FTR_SYSVAL, offsetof(struct arm64_ftr_reg, sys_val)); BLANK(); #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 - DEFINE(TRAMP_VALIAS, TRAMP_VALIAS); + DEFINE(FIX_ENTRY_TRAMP_BEGIN, FIX_ENTRY_TRAMP_BEGIN); #endif #ifdef CONFIG_ARM_SDE_INTERFACE DEFINE(SDEI_EVENT_INTREGS, offsetof(struct sdei_registered_event, interrupted_regs)); diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index 7ef0e127b149f..ba47dc8672c04 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -101,11 +101,27 @@ .org .Lventry_start\@ + 128 // Did we overflow the ventry slot? .endm +#define TRAMP_VALIAS(page_shift) (FIXADDR_TOP - (FIX_ENTRY_TRAMP_BEGIN << (page_shift))) + .macro tramp_alias, dst, sym - .set .Lalias\@, TRAMP_VALIAS + \sym - .entry.tramp.text - movz \dst, :abs_g2_s:.Lalias\@ - movk \dst, :abs_g1_nc:.Lalias\@ - movk \dst, :abs_g0_nc:.Lalias\@ +alternative_if ARM64_USE_PAGE_SIZE_4K + .set .Lalias4k\@, TRAMP_VALIAS(ARM64_PAGE_SHIFT_4K) + \sym - .entry.tramp.text + movz \dst, :abs_g2_s:.Lalias4k\@ + movk \dst, :abs_g1_nc:.Lalias4k\@ + movk \dst, :abs_g0_nc:.Lalias4k\@ +alternative_else_nop_endif +alternative_if ARM64_USE_PAGE_SIZE_16K + .set .Lalias16k\@, TRAMP_VALIAS(ARM64_PAGE_SHIFT_16K) + \sym - .entry.tramp.text + movz \dst, :abs_g2_s:.Lalias16k\@ + movk \dst, :abs_g1_nc:.Lalias16k\@ + movk \dst, :abs_g0_nc:.Lalias16k\@ +alternative_else_nop_endif +alternative_if ARM64_USE_PAGE_SIZE_64K + .set .Lalias64k\@, TRAMP_VALIAS(ARM64_PAGE_SHIFT_64K) + \sym - .entry.tramp.text + movz \dst, :abs_g2_s:.Lalias64k\@ + movk \dst, :abs_g1_nc:.Lalias64k\@ + movk \dst, :abs_g0_nc:.Lalias64k\@ +alternative_else_nop_endif .endm /* @@ -627,16 +643,30 @@ SYM_CODE_END(ret_to_user) bic \tmp, \tmp, #USER_ASID_FLAG msr ttbr1_el1, \tmp #ifdef CONFIG_QCOM_FALKOR_ERRATUM_1003 -alternative_if ARM64_WORKAROUND_QCOM_FALKOR_E1003 +alternative_if_not ARM64_WORKAROUND_QCOM_FALKOR_E1003 + b .Lskip_falkor_e1003\@ +alternative_else_nop_endif /* ASID already in \tmp[63:48] */ - movk \tmp, #:abs_g2_nc:(TRAMP_VALIAS >> 12) - movk \tmp, #:abs_g1_nc:(TRAMP_VALIAS >> 12) - /* 2MB boundary containing the vectors, so we nobble the walk cache */ - movk \tmp, #:abs_g0_nc:((TRAMP_VALIAS & ~(SZ_2M - 1)) >> 12) +alternative_if ARM64_USE_PAGE_SIZE_4K + movk \tmp, #:abs_g2_nc:(TRAMP_VALIAS(ARM64_PAGE_SHIFT_4K) >> 12) + movk \tmp, #:abs_g1_nc:(TRAMP_VALIAS(ARM64_PAGE_SHIFT_4K) >> 12) + movk \tmp, #:abs_g0_nc:((TRAMP_VALIAS(ARM64_PAGE_SHIFT_4K) & ~(SZ_2M - 1)) >> 12) + b .Lfinish_falkor_e1003\@ +alternative_else_nop_endif +alternative_if ARM64_USE_PAGE_SIZE_16K + movk \tmp, #:abs_g2_nc:(TRAMP_VALIAS(ARM64_PAGE_SHIFT_16K) >> 12) + movk \tmp, #:abs_g1_nc:(TRAMP_VALIAS(ARM64_PAGE_SHIFT_16K) >> 12) + movk \tmp, #:abs_g0_nc:((TRAMP_VALIAS(ARM64_PAGE_SHIFT_16K) & ~(SZ_2M - 1)) >> 12) +alternative_else /* ARM64_USE_PAGE_SIZE_64K */ + movk \tmp, #:abs_g2_nc:(TRAMP_VALIAS(ARM64_PAGE_SHIFT_64K) >> 12) + movk \tmp, #:abs_g1_nc:(TRAMP_VALIAS(ARM64_PAGE_SHIFT_64K) >> 12) + movk \tmp, #:abs_g0_nc:((TRAMP_VALIAS(ARM64_PAGE_SHIFT_64K) & ~(SZ_2M - 1)) >> 12) +alternative_endif +.Lfinish_falkor_e1003\@: isb tlbi vae1, \tmp dsb nsh -alternative_else_nop_endif +.Lskip_falkor_e1003\@: #endif /* CONFIG_QCOM_FALKOR_ERRATUM_1003 */ .endm