From patchwork Mon Apr 15 07:54:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 13629566 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CE5D4C4345F for ; Mon, 15 Apr 2024 07:55:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=yRhyLXadvMjhfE/QNu4HxtM++eqLGKuAgjSuejUKNWI=; b=ZepngiswivkuolLpBmRNLl4q7j xNjO9aMWrhS9XEu3FfZUNfhUIlrsZcAzUp9gdbLXImKDwrz2gUDMuZMqRwIVKzx1sPI3C0OUSknmB W/+AoYe7UtB37/o/heCp6z2gHcEeofea/KFXjOCvklamTqgKIwcZXyVf2FbONkJN9NbG7sFHrp8Mn AVy8SDiYNwynHShinovpq2aWMbiBwGFKc5UUmG0lgrR327AyNLojkdJu7FBdRInWqWumKspdkt7Uw qPdqwVmz+oGOBxWJdIyx26SotGb2Q0z/VvSwNYYRM3p1W/3Y5vyBYTllYWc3utAMb5cSwodHSSUVy RBvRWxvQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rwHBd-00000007Q0W-1OVF; Mon, 15 Apr 2024 07:55:33 +0000 Received: from mail-wr1-x44a.google.com ([2a00:1450:4864:20::44a]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rwHAW-00000007PDf-3V11 for linux-arm-kernel@lists.infradead.org; Mon, 15 Apr 2024 07:54:27 +0000 Received: by mail-wr1-x44a.google.com with SMTP id ffacd0b85a97d-343e00c8979so2264298f8f.1 for ; Mon, 15 Apr 2024 00:54:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1713167661; x=1713772461; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=fHqlbZDv0EFTOe6sB1Tpm64dcQ4J1hYC/eVPeIjZT+w=; b=18dBmq1dKlOQsYErcl9qax08sdyDOOzgLwYwOgJbpMxiTUdJNzPdk0XVDG0c1OdCx3 0T/ZNyQ87KAe1K7lQ49Fvcrg1Wn97fseZ4apVqvR1V5o6nMWXeft1hczq0aOgocMVOaA xpaATnxRVOprUm5zpoSp4jo9ssGzDOYgLXla1R3FOsC+5getZVXRyUS/5/ZkvqU4Nl9G KBxWMX3s+/cF8TWaJVzT60M+h3hcK9HaBQWMAPqGa3dX4IoOLHU+CjB/VCKQpaTrrU5X j/dWI9UDUMCWcmNHjCNJfB0SYE/GDS5UQkFvUyQqy5doH96zVbm8BPsEXdsoauWhSmmN iqYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713167661; x=1713772461; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=fHqlbZDv0EFTOe6sB1Tpm64dcQ4J1hYC/eVPeIjZT+w=; b=PjUpr4WJRdhqOsdCR70T8PBMU5WSrfHqKLsmC5gQKVUauD8wh7eTDqYFBQ9SkaZFrn nus3JFoS4lBjda71mE6lTM4MfP5WvqigL0FQqnXDAK8h3eK0DWCD3XBfYw7kEkQDEYQ3 ZNc+FS4B0zhmqcISYFO8iTgcdzPmFnUkdNVEHI2nt+FHwtQHrWau4BhmW04eElH4P5yo KSkZziEJHdidVSPvZNJ4j6m/OsZ8REotD8gr41XOiIzn8v1utJO1rhSKReNWuyplcJai VvnitiFkmMRKAqS42W+8p5RbIFz4cwPP0JOaxcxSVls+ikm9y2P14Z3XJjSGSIVNl0S6 pH8g== X-Gm-Message-State: AOJu0YybKE2KQxpekPZzGlBBSL45WuQMFF+/jhcYxTLoOCfjYCXkb9/+ bR86fmkqZK1nKdjO2/SfBKTnxM+KTCtvDNq1Es5M/gVwQzrptgQx8FiMK7Zc9/DNJTpityqCnj1 Zhha2mACSauWe0Oe2DMbZqgZ/H9UPEaqVV9zqp9ukrJmwkwMw+uWRNidXeKrT1nBXveA3mVx8J+ 5kHSAuOziuHCo8tMv6tOc1Dn/UbABJ4RtPwPLaUWmp X-Google-Smtp-Source: AGHT+IHDhQFFntwk99BPAbJhoRELexbL2bcolPYV7G8ehL/63jSHsr54Ty9yc9ul/Woguo1DaBjA4XUc X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:118a]) (user=ardb job=sendgmr) by 2002:adf:e343:0:b0:345:7d20:27c5 with SMTP id n3-20020adfe343000000b003457d2027c5mr16096wrj.1.1713167660803; Mon, 15 Apr 2024 00:54:20 -0700 (PDT) Date: Mon, 15 Apr 2024 09:54:14 +0200 In-Reply-To: <20240415075412.2347624-4-ardb+git@google.com> Mime-Version: 1.0 References: <20240415075412.2347624-4-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=858; i=ardb@kernel.org; h=from:subject; bh=ODo4vOYSSTd3CKdVF9qBZxLUUnytPQ0BotDws5LU5Tk=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIU3mrlrC1+JD0mk/vDmPJcvumWnr2vBl2+1kU74HjQem1 u+y/Leoo5SFQYyDQVZMkUVg9t93O09PlKp1niULM4eVCWQIAxenAExEzYGR4WrSUc56D5X3e+Ns GX0jF5+5Oml1kuPFxqj8fXzeJqax8xn+qQtWKfCut17rf25L0pYH98UMyv4kNE2w9Yrb7c11N3Q NAwA= X-Mailer: git-send-email 2.44.0.683.g7961c838ac-goog Message-ID: <20240415075412.2347624-5-ardb+git@google.com> Subject: [PATCH 1/2] arm64/head: Drop unnecessary pre-disable-MMU workaround From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Catalin Marinas , Will Deacon , Marc Zyngier , Mark Rutland X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240415_005424_953801_E91CFA0E X-CRM114-Status: GOOD ( 10.55 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Ard Biesheuvel The Falkor erratum that results in the need for an ISB before clearing the M bit in SCTLR_ELx only applies to execution at exception level x, and so the workaround is not needed when disabling the EL1 MMU while running at EL2. Signed-off-by: Ard Biesheuvel Acked-by: Marc Zyngier Acked-by: Mark Rutland --- arch/arm64/kernel/head.S | 2 -- 1 file changed, 2 deletions(-) diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index 06234c3a15f3..b8bbd72cb194 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -323,13 +323,11 @@ SYM_INNER_LABEL(init_el2, SYM_L_LOCAL) cbz x0, 2f /* Set a sane SCTLR_EL1, the VHE way */ - pre_disable_mmu_workaround msr_s SYS_SCTLR_EL12, x1 mov x2, #BOOT_CPU_FLAG_E2H b 3f 2: - pre_disable_mmu_workaround msr sctlr_el1, x1 mov x2, xzr 3: From patchwork Mon Apr 15 07:54:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 13629584 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EAD65C4345F for ; Mon, 15 Apr 2024 07:56:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=C4FQOhSOtiH/1iQuL0Di2LBmxfbnFBRkTSG/ADhG97A=; b=kztSwQTzgAO9jgmJvxFe3UxPUI dWmXwJbBUDCRQ5667fyw3VDRG1a2R0XVCX06SqrK8j7VWhjz/kCK75/gmKgSCTo0hC+XCneioPfX3 Vjeqaws13/lb9bvAkWhfv5Osk4TuAZx+IFDAuGmGwrS41ytmul+plm7BGzCKkJlte+8VrKTemUclL axX4DNz983sCYA2gZXMtFnnbuEGGrJbdQ/2VbfHNh8/m3Q4uOat1pSLK3nDpFjDNPylMLpu98a9Gx SzY1IdU4pVhcWRuQnHTU6PHZP5qd+LQdmaxOTTJoycFvYIzdNIwTKcS/HOCGGYKmncUZFLo8nLsvV hno4Gkhg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rwHC4-00000007QHA-1fFe; Mon, 15 Apr 2024 07:56:00 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rwHAW-00000007PFg-36PK for linux-arm-kernel@lists.infradead.org; Mon, 15 Apr 2024 07:54:27 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-60cd041665bso50595507b3.0 for ; Mon, 15 Apr 2024 00:54:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1713167663; x=1713772463; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=ImFCkKmyz1fR3hS1Fkp5BI5BSGmDzkSF7yRLivElJRg=; b=b9I5QlpwjyUua4kHVbmtosynDta1gzN5sVrwH0Vqz00FUPrbDoq+/1YD58T4EujicA 9yr1t74RKrGnp7yR+bUJeHH5M3xdEXEDPftPUm7TsVrUrKyyjF1TlXD/v6OyQ8vwuHQL 91oNndJN6nhQvkooz+WuFpIyXKJBWI9moCqJOS7rY9SQlZdl/NfLPsxa4dInpqwm9Zun 7zVCFkE+jngc1udgjOXvI4bPqdix2HUeDM23sKnJ/NDdUAJcGOlOupbWtBUuAQJj6gz/ OCxQZD9FhRh4/2ZPxdHPJGpM8KKAfluFwvGVY07QghmtBZsSabEJ9Wm0fC0n3ev18AJL qbwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713167663; x=1713772463; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=ImFCkKmyz1fR3hS1Fkp5BI5BSGmDzkSF7yRLivElJRg=; b=l141TMNmBiNGiFzlGCv7gCS9n4qh26FXM94IERke7ZfaROXOlG7XeW+JwSy44SzVcY vXyvlLq0oyqMuinYRS3Rup33hSswcWqAJLRde5iDvExpDHdniqmeYlCrZqpPDc73RPcN SwqCcagH1AG1LEAUymQ/mTjNv0o3plYpbdJLUPpsm+RdHtmlOSibu2tdnEU8JgL6fzdc 6mb5a/rxKsDCziRJxPWi0yhrwkdFUL9EqkW21t/KdxotCBkxxrK/xLYTn0ZmrEGQeAw9 gpjM2he4mFIrUrxM0R5NJICIm0Sg3+f2ZLbZMWb6H67uRuu0D2BjLlYvXbu42hI+cayS Dozg== X-Gm-Message-State: AOJu0Yy/xxMCvcXhlFvtGh3lgIGZw3nIoAUtV1aE6qPARh5hreKhtyQq R1KkpedquIkZsUh1BS2+MbLMAysu7cliYE77tOUL02fE+0LO7HUAMzKyruil+erYVQ0WJDs3Jsn rdAVkblNj/JNGvyM9UvEujQxpO7Ph9UeUHyjt6Pc9/U9JB+B9PhCpoVVH09k07TjZAFnOqBNyEM XjJHa/mhe5qJAL3IfeSurman5yJi2ovrw0HcaBliFm X-Google-Smtp-Source: AGHT+IFByPk3tmnKNPSljqSZcYe3IdvM8Vi9brzHb9/UbmyOlQjWVI789nu0ax2Q8ORJgLHixzTlU6wH X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:118a]) (user=ardb job=sendgmr) by 2002:a81:e947:0:b0:615:7f59:9dbb with SMTP id e7-20020a81e947000000b006157f599dbbmr3183866ywm.2.1713167663616; Mon, 15 Apr 2024 00:54:23 -0700 (PDT) Date: Mon, 15 Apr 2024 09:54:15 +0200 In-Reply-To: <20240415075412.2347624-4-ardb+git@google.com> Mime-Version: 1.0 References: <20240415075412.2347624-4-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=1754; i=ardb@kernel.org; h=from:subject; bh=BXh8QRy7FzifpH7etDyz1TLzG5d09q0JkZxc6A1d074=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIU3mrlq/4drzqY8um2U0fb/FHH5A47v385fCd31POczzT +G8oFHSUcrCIMbBICumyCIw+++7nacnStU6z5KFmcPKBDKEgYtTACayy4uR4eqj6Su3aU/eWGUv vzdBoVO8pZDz/f6Ci+xZi+9qGE65qMPwV+5U1/c/ElIzTk1OEbgrZLc13900sPXw1mMXH+zaecc 6hAsA X-Mailer: git-send-email 2.44.0.683.g7961c838ac-goog Message-ID: <20240415075412.2347624-6-ardb+git@google.com> Subject: [PATCH 2/2] arm64/head: Disable MMU at EL2 before clearing HCR_EL2.E2H From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Catalin Marinas , Will Deacon , Marc Zyngier , Mark Rutland X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240415_005424_907305_F5402A3D X-CRM114-Status: GOOD ( 14.79 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Ard Biesheuvel Even though the boot protocol stipulates otherwise, an exception has been made for the EFI stub, and entering the core kernel with the MMU enabled is permitted. This allows a substantial amount of cache maintenance to be elided, wich is significant when fast boot times are critical (e.g., for booting micro-VMs) Once the initial ID map has been populated, the MMU is disabled as part of the logic sequence that puts all system registers into a known state. Any code that needs to execute within the window where the MMU is off is cleaned to the PoC explicitly, which includes all of HYP text when entering at EL2. However, the current sequence of initializing the EL2 system registers is not safe: HCR_EL2 is set to its nVHE initial state before SCTLR_EL2 is reprogrammed, and this means that a VHE-to-nVHE switch may occur while the MMU is enabled. This switch causes some system registers as well as page table descriptors to be interpreted in a different way, potentially resulting in spurious exceptions relating to MMU translation. So disable the MMU explicitly first when entering in EL2 with the MMU and caches enabled. Signed-off-by: Ard Biesheuvel Acked-by: Marc Zyngier Acked-by: Mark Rutland --- arch/arm64/kernel/head.S | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index b8bbd72cb194..cb68adcabe07 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -289,6 +289,11 @@ SYM_INNER_LABEL(init_el2, SYM_L_LOCAL) adr_l x1, __hyp_text_end adr_l x2, dcache_clean_poc blr x2 + + mov_q x0, INIT_SCTLR_EL2_MMU_OFF + pre_disable_mmu_workaround + msr sctlr_el2, x0 + isb 0: mov_q x0, HCR_HOST_NVHE_FLAGS