From patchwork Mon Apr 15 07:54:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 13629584 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EAD65C4345F for ; Mon, 15 Apr 2024 07:56:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=C4FQOhSOtiH/1iQuL0Di2LBmxfbnFBRkTSG/ADhG97A=; b=kztSwQTzgAO9jgmJvxFe3UxPUI dWmXwJbBUDCRQ5667fyw3VDRG1a2R0XVCX06SqrK8j7VWhjz/kCK75/gmKgSCTo0hC+XCneioPfX3 Vjeqaws13/lb9bvAkWhfv5Osk4TuAZx+IFDAuGmGwrS41ytmul+plm7BGzCKkJlte+8VrKTemUclL axX4DNz983sCYA2gZXMtFnnbuEGGrJbdQ/2VbfHNh8/m3Q4uOat1pSLK3nDpFjDNPylMLpu98a9Gx SzY1IdU4pVhcWRuQnHTU6PHZP5qd+LQdmaxOTTJoycFvYIzdNIwTKcS/HOCGGYKmncUZFLo8nLsvV hno4Gkhg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rwHC4-00000007QHA-1fFe; Mon, 15 Apr 2024 07:56:00 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rwHAW-00000007PFg-36PK for linux-arm-kernel@lists.infradead.org; Mon, 15 Apr 2024 07:54:27 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-60cd041665bso50595507b3.0 for ; Mon, 15 Apr 2024 00:54:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1713167663; x=1713772463; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=ImFCkKmyz1fR3hS1Fkp5BI5BSGmDzkSF7yRLivElJRg=; b=b9I5QlpwjyUua4kHVbmtosynDta1gzN5sVrwH0Vqz00FUPrbDoq+/1YD58T4EujicA 9yr1t74RKrGnp7yR+bUJeHH5M3xdEXEDPftPUm7TsVrUrKyyjF1TlXD/v6OyQ8vwuHQL 91oNndJN6nhQvkooz+WuFpIyXKJBWI9moCqJOS7rY9SQlZdl/NfLPsxa4dInpqwm9Zun 7zVCFkE+jngc1udgjOXvI4bPqdix2HUeDM23sKnJ/NDdUAJcGOlOupbWtBUuAQJj6gz/ OCxQZD9FhRh4/2ZPxdHPJGpM8KKAfluFwvGVY07QghmtBZsSabEJ9Wm0fC0n3ev18AJL qbwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713167663; x=1713772463; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=ImFCkKmyz1fR3hS1Fkp5BI5BSGmDzkSF7yRLivElJRg=; b=l141TMNmBiNGiFzlGCv7gCS9n4qh26FXM94IERke7ZfaROXOlG7XeW+JwSy44SzVcY vXyvlLq0oyqMuinYRS3Rup33hSswcWqAJLRde5iDvExpDHdniqmeYlCrZqpPDc73RPcN SwqCcagH1AG1LEAUymQ/mTjNv0o3plYpbdJLUPpsm+RdHtmlOSibu2tdnEU8JgL6fzdc 6mb5a/rxKsDCziRJxPWi0yhrwkdFUL9EqkW21t/KdxotCBkxxrK/xLYTn0ZmrEGQeAw9 gpjM2he4mFIrUrxM0R5NJICIm0Sg3+f2ZLbZMWb6H67uRuu0D2BjLlYvXbu42hI+cayS Dozg== X-Gm-Message-State: AOJu0Yy/xxMCvcXhlFvtGh3lgIGZw3nIoAUtV1aE6qPARh5hreKhtyQq R1KkpedquIkZsUh1BS2+MbLMAysu7cliYE77tOUL02fE+0LO7HUAMzKyruil+erYVQ0WJDs3Jsn rdAVkblNj/JNGvyM9UvEujQxpO7Ph9UeUHyjt6Pc9/U9JB+B9PhCpoVVH09k07TjZAFnOqBNyEM XjJHa/mhe5qJAL3IfeSurman5yJi2ovrw0HcaBliFm X-Google-Smtp-Source: AGHT+IFByPk3tmnKNPSljqSZcYe3IdvM8Vi9brzHb9/UbmyOlQjWVI789nu0ax2Q8ORJgLHixzTlU6wH X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:118a]) (user=ardb job=sendgmr) by 2002:a81:e947:0:b0:615:7f59:9dbb with SMTP id e7-20020a81e947000000b006157f599dbbmr3183866ywm.2.1713167663616; Mon, 15 Apr 2024 00:54:23 -0700 (PDT) Date: Mon, 15 Apr 2024 09:54:15 +0200 In-Reply-To: <20240415075412.2347624-4-ardb+git@google.com> Mime-Version: 1.0 References: <20240415075412.2347624-4-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=1754; i=ardb@kernel.org; h=from:subject; bh=BXh8QRy7FzifpH7etDyz1TLzG5d09q0JkZxc6A1d074=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIU3mrlq/4drzqY8um2U0fb/FHH5A47v385fCd31POczzT +G8oFHSUcrCIMbBICumyCIw+++7nacnStU6z5KFmcPKBDKEgYtTACayy4uR4eqj6Su3aU/eWGUv vzdBoVO8pZDz/f6Ci+xZi+9qGE65qMPwV+5U1/c/ElIzTk1OEbgrZLc13900sPXw1mMXH+zaecc 6hAsA X-Mailer: git-send-email 2.44.0.683.g7961c838ac-goog Message-ID: <20240415075412.2347624-6-ardb+git@google.com> Subject: [PATCH 2/2] arm64/head: Disable MMU at EL2 before clearing HCR_EL2.E2H From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Catalin Marinas , Will Deacon , Marc Zyngier , Mark Rutland X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240415_005424_907305_F5402A3D X-CRM114-Status: GOOD ( 14.79 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Ard Biesheuvel Even though the boot protocol stipulates otherwise, an exception has been made for the EFI stub, and entering the core kernel with the MMU enabled is permitted. This allows a substantial amount of cache maintenance to be elided, wich is significant when fast boot times are critical (e.g., for booting micro-VMs) Once the initial ID map has been populated, the MMU is disabled as part of the logic sequence that puts all system registers into a known state. Any code that needs to execute within the window where the MMU is off is cleaned to the PoC explicitly, which includes all of HYP text when entering at EL2. However, the current sequence of initializing the EL2 system registers is not safe: HCR_EL2 is set to its nVHE initial state before SCTLR_EL2 is reprogrammed, and this means that a VHE-to-nVHE switch may occur while the MMU is enabled. This switch causes some system registers as well as page table descriptors to be interpreted in a different way, potentially resulting in spurious exceptions relating to MMU translation. So disable the MMU explicitly first when entering in EL2 with the MMU and caches enabled. Signed-off-by: Ard Biesheuvel Acked-by: Marc Zyngier Acked-by: Mark Rutland --- arch/arm64/kernel/head.S | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index b8bbd72cb194..cb68adcabe07 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -289,6 +289,11 @@ SYM_INNER_LABEL(init_el2, SYM_L_LOCAL) adr_l x1, __hyp_text_end adr_l x2, dcache_clean_poc blr x2 + + mov_q x0, INIT_SCTLR_EL2_MMU_OFF + pre_disable_mmu_workaround + msr sctlr_el2, x0 + isb 0: mov_q x0, HCR_HOST_NVHE_FLAGS