From patchwork Wed Dec 9 13:17:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11961365 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E71FC4361B for ; Wed, 9 Dec 2020 13:19:35 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CD041207C8 for ; Wed, 9 Dec 2020 13:19:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CD041207C8 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=3eRvYOiR9pOAiDGGum33QHDe6k1sT1XDyum2T41Z68M=; b=iTD2/y2mYjDpuPS1INWTlHZ5L p8xp1IxE3aKTCBC7Bvn5MHPlPSdhVtcIGZip5e32onY8r1PPtiwWYX6wWzEd6d3x8owZrCfWThRKQ bmhKCj80FY5os6tg4mDiuQcpjntQdOoRjQVPgfv17XE72H3r6Cl7faraKEBLX8OdDl2ym6GXLE9EV dofcdeyKoq6H5Z0PV2XSZGMyBiqewbp+w6gtKKlghP3oRvjNp72l9g7tRWTKA/UpDEWimMMANOsSw adPKwoMS0Lg4P3vFhS91bwyNZbCQ9wNQTZ/wMs0W7wv3iois2xuYmtfMuVCQe2qYANN3y1b53o6gZ rg92K/Oyw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kmzMM-0003JD-Cm; Wed, 09 Dec 2020 13:18:22 +0000 Received: from mail-wr1-x442.google.com ([2a00:1450:4864:20::442]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kmzM4-0003Dt-0W for linux-arm-kernel@lists.infradead.org; Wed, 09 Dec 2020 13:18:07 +0000 Received: by mail-wr1-x442.google.com with SMTP id x6so1698612wro.11 for ; Wed, 09 Dec 2020 05:18:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=AnSc1Sgxbdzkn7VrpHIozks65PIKCh74xA+FbntzNXY=; b=fV0IqsCKJt3ivo+aspsTCn4tySosMb+82n5RnA2sAYTUUnT7GT1fshweSpCBAReyj/ 2SoMtQDPP3FnvvJAKFaqDKH5dcWNHZqK/CZL7MGg9dFP7zD8KCf5PZ2a6BBcicj977Ou 0zgxnB/Skz+GAbr34KDFIGm1SWSVJpJ4mJqz8BgANsF6kW6dH0nSYakL6ykDEUf64EjV KnZWuetkZ8TvwYngLljRR9HKZHMH6bIxtjytbs5/EALMbZ/N/nU5gn7BWB3n4tc72taH T4YeQt7WHrQgQmEdBWJY/V/9Ypebzu9I3VWoWfG9dWXZ4KSomVBQkPfeyuxeCJJ1U/Cq 1mEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=AnSc1Sgxbdzkn7VrpHIozks65PIKCh74xA+FbntzNXY=; b=YVak5ERI40mB9lV5kVJnl1YT5Blk/MJOX0mu7VqEzBIBpI66LQuk+g5MeQs59jOszb s5RRzOUVJLgF+n66qTxPhxTVFJVoC7Lo02qzsDt2D4E1rHh8YNs4psUtKlw5wMm5vNLq 9Sfne5VgqYiqjycT7Y6R1Gtnf+FEPRTqrOxyHipqGfF/vu6j2bZUnsXN7XRWhdy0M+zh SLrYfnZoRPk5DZNVovYozS6dv0HdulkDMaMHYJzkqUZl/6JMxxwW0x+iyhCCEoCdFexh hXVSD5ZGvxP/4Zuy7db0hwlc0mjyvIoKOw4owwJLWXUhJs+YNyjo8230RXe3XLnzQ40z yCfw== X-Gm-Message-State: AOAM530zzW0arOFrAao+oaBInEBTYDgdcv6X6reP6A4nGkGlH0VBnE+b AKiDu7MEtdal9poFBKBHYERxFQ== X-Google-Smtp-Source: ABdhPJyFNsrtiECMYekEKiA3Vo/iG+G5NK6FdhcQmRaR2bdsDNr4G07R9LSXaBa+5xVT0OnDyzSXRw== X-Received: by 2002:adf:97ce:: with SMTP id t14mr2716349wrb.368.1607519882606; Wed, 09 Dec 2020 05:18:02 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:9d1b:d0eb:db43:6cd2]) by smtp.gmail.com with ESMTPSA id k11sm3202279wmj.42.2020.12.09.05.18.01 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 09 Dec 2020 05:18:01 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH 6/9] KVM: arm64: Apply hyp relocations at runtime Date: Wed, 9 Dec 2020 13:17:43 +0000 Message-Id: <20201209131746.85622-7-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201209131746.85622-1-dbrazdil@google.com> References: <20201209131746.85622-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201209_081804_171259_48CF321B X-CRM114-Status: GOOD ( 20.90 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kernel-team@android.com, Suzuki K Poulose , Catalin Marinas , linux-kernel@vger.kernel.org, James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , David Brazdil , Will Deacon , Ard Biesheuvel , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org KVM nVHE code runs under a different VA mapping than the kernel, hence so far it avoided using absolute addressing because the VA in a constant pool is relocated by the linker to a kernel VA (see hyp_symbol_addr). Now the kernel has access to a list of positions that contain a kimg VA but will be accessed only in hyp execution context. These are generated by the gen-hyprel build-time tool and stored in .hyp.reloc. Add early boot pass over the entries and convert the kimg VAs to hyp VAs. Note that this requires for .hyp* ELF sections to be mapped read-write at that point. Signed-off-by: David Brazdil --- arch/arm64/configs/defconfig | 1 + arch/arm64/include/asm/kvm_mmu.h | 1 + arch/arm64/include/asm/sections.h | 1 + arch/arm64/kernel/smp.c | 4 +++- arch/arm64/kvm/va_layout.c | 28 ++++++++++++++++++++++++++++ 5 files changed, 34 insertions(+), 1 deletion(-) diff --git a/arch/arm64/configs/defconfig b/arch/arm64/configs/defconfig index 5cfe3cf6f2ac..73fc9f2f2661 100644 --- a/arch/arm64/configs/defconfig +++ b/arch/arm64/configs/defconfig @@ -1092,3 +1092,4 @@ CONFIG_DEBUG_KERNEL=y # CONFIG_DEBUG_PREEMPT is not set # CONFIG_FTRACE is not set CONFIG_MEMTEST=y +# CONFIG_ARM64_VHE is not set diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h index e52d82aeadca..6bbb44011c84 100644 --- a/arch/arm64/include/asm/kvm_mmu.h +++ b/arch/arm64/include/asm/kvm_mmu.h @@ -129,6 +129,7 @@ alternative_cb_end void kvm_update_va_mask(struct alt_instr *alt, __le32 *origptr, __le32 *updptr, int nr_inst); void kvm_compute_layout(void); +void kvm_apply_hyp_relocations(void); static __always_inline unsigned long __kern_hyp_va(unsigned long v) { diff --git a/arch/arm64/include/asm/sections.h b/arch/arm64/include/asm/sections.h index a6f3557d1ab2..2f36b16a5b5d 100644 --- a/arch/arm64/include/asm/sections.h +++ b/arch/arm64/include/asm/sections.h @@ -12,6 +12,7 @@ extern char __hibernate_exit_text_start[], __hibernate_exit_text_end[]; extern char __hyp_idmap_text_start[], __hyp_idmap_text_end[]; extern char __hyp_text_start[], __hyp_text_end[]; extern char __hyp_rodata_start[], __hyp_rodata_end[]; +extern char __hyp_reloc_begin[], __hyp_reloc_end[]; extern char __idmap_text_start[], __idmap_text_end[]; extern char __initdata_begin[], __initdata_end[]; extern char __inittext_begin[], __inittext_end[]; diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index 18e9727d3f64..47142395bc91 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -434,8 +434,10 @@ static void __init hyp_mode_check(void) "CPU: CPUs started in inconsistent modes"); else pr_info("CPU: All CPU(s) started at EL1\n"); - if (IS_ENABLED(CONFIG_KVM)) + if (IS_ENABLED(CONFIG_KVM)) { kvm_compute_layout(); + kvm_apply_hyp_relocations(); + } } void __init smp_cpus_done(unsigned int max_cpus) diff --git a/arch/arm64/kvm/va_layout.c b/arch/arm64/kvm/va_layout.c index d8cc51bd60bf..fb2ca02b7270 100644 --- a/arch/arm64/kvm/va_layout.c +++ b/arch/arm64/kvm/va_layout.c @@ -82,6 +82,34 @@ __init void kvm_compute_layout(void) init_hyp_physvirt_offset(); } +/* + * The .hyp.reloc ELF section contains a list of kimg positions that + * contains kimg VAs but will be accessed only in hyp execution context. + * Convert them to hyp VAs. See gen-hyprel.c for more details. + */ +__init void kvm_apply_hyp_relocations(void) +{ + int32_t *rel; + int32_t *begin = (int32_t*)__hyp_reloc_begin; + int32_t *end = (int32_t*)__hyp_reloc_end; + + for (rel = begin; rel < end; ++rel) { + uintptr_t *ptr, kimg_va; + + /* + * Each entry contains a 32-bit relative offset from itself + * to a kimg VA position. + */ + ptr = (uintptr_t*)lm_alias((char*)rel + *rel); + + /* Read the kimg VA value at the relocation address. */ + kimg_va = *ptr; + + /* Convert to hyp VA and store back to the relocation address. */ + *ptr = __early_kern_hyp_va((uintptr_t)lm_alias(kimg_va)); + } +} + static u32 compute_instruction(int n, u32 rd, u32 rn) { u32 insn = AARCH64_BREAK_FAULT;