From patchwork Wed Dec 9 13:17:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11961361 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DC0FFC433FE for ; Wed, 9 Dec 2020 13:19:23 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9E916207C8 for ; Wed, 9 Dec 2020 13:19:23 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9E916207C8 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=ZNYRskgzxQvuLs3XLzV/7uuxkYmG1Fzv/SwutKOJstk=; b=scmDuTYRjzrBiFKB29wJiOHL7 59wWcQsggtNs2Bm+v+7ocxbUm9yWXm9ZAav+cfc9eC63rXBudPiLgUhMLhZmPN5oc11W31On7NzaB merS8FwgNBEPzRVWcgApUq9rpfV2OAHKfVGs6jXv7Vxn+jIEpDXYKY8XCgtR2gagzLFWQQtURNgTD Uab4cpDsrlIcrQTAW1+6JfQzjoc4mvQdBCBn5Qwq2XjLHZ/L1vLalaAT5Rf6vDjT4gNfYq/vE744V 6BbGyO0Q+Dao1rcnqYCiYDgHGUJv4TgBw0ZJwPh5L8Rxh0WUbB5Ujm8lzKjsJi9lYBKcqlR77GSG9 2umD9lDEg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kmzLz-0003CR-1f; Wed, 09 Dec 2020 13:17:59 +0000 Received: from mail-wr1-x441.google.com ([2a00:1450:4864:20::441]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kmzLt-0003Ar-Fu for linux-arm-kernel@lists.infradead.org; Wed, 09 Dec 2020 13:17:54 +0000 Received: by mail-wr1-x441.google.com with SMTP id x6so1698028wro.11 for ; Wed, 09 Dec 2020 05:17:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ui2AyYakjARNpvT610MmpwVISaykxIjU1EziL4bq/8g=; b=TtPn5gbE5cUnZn/XoZAH5Q1SPuX8zUYSCs/O66h6C9BXK3fpK/Gxba6BH70xbSUMQI rysLsdyUKSLybXMLAMuifFTOgm/GYv7CzvMxgqTkJEp1Fzljb4N/quSYihWIbdHGfOMS YewS6M/7CjC7YsdpjGCrrDdk6y4EyOkTNlena25qIj6QWnxi/8p/lfEfwkgi6UyCbeb6 sjUB6nqC+Uxvz6IBGuj5pn76G9Dt2QO8PEqw5mKADfHfuFpx49k489ssqc8jcB5yU5iE ahnxDbrtLwfx8ViQriceqefi2FsulHCF3546sTznByZRVN57V5UzPws2FuRb/mBmH1Xt 6yQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ui2AyYakjARNpvT610MmpwVISaykxIjU1EziL4bq/8g=; b=AJyn6Fl9GR93DrrGXAEEY67N6gnWvOIR2v36E/r757gtxzky3IwAm/oDYqkalt7cac Xa1niMtv72ULF82rmE0LJH2dQWNm7SuPW6IuXM+fP9NLgbjrbeZU6rb40cvlJ9FJjM9y Fr3HUyrecgVlT02JPyKGbJKo5uouR8cW4G69ysYvjHskbs4W+hiSv8mHuT7bJ/D98oQd ZUZLWdorwtzc0OwAs9NWfnTDnYn0c3i10KjRcGKShAbHaFi4QhKG0ISNDIWYq9ES8VB3 05SOQswBMDPLXBbmCxN6OjKqEW5qltMSRXXYVF0w4eHMReOdFsvXeVokRS24hWjesklz BKNw== X-Gm-Message-State: AOAM532D01OKuQyeSoeuUXaP1TAuP6l8fjmDWcVlkrZtEUfMZqTSCCGB BZ9IZscDZSP5/14StZunndybpA== X-Google-Smtp-Source: ABdhPJzlT1ZAQI8+iYY8fpTcJpj5l6KBwxxqgFiRZRdWFROI/wunQbz/voXHfXAhUBU4AIC7QtIReQ== X-Received: by 2002:a5d:5146:: with SMTP id u6mr2832541wrt.66.1607519872306; Wed, 09 Dec 2020 05:17:52 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:9d1b:d0eb:db43:6cd2]) by smtp.gmail.com with ESMTPSA id g192sm3294480wme.48.2020.12.09.05.17.50 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 09 Dec 2020 05:17:51 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH 1/9] KVM: arm64: Correctly align nVHE percpu data Date: Wed, 9 Dec 2020 13:17:38 +0000 Message-Id: <20201209131746.85622-2-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201209131746.85622-1-dbrazdil@google.com> References: <20201209131746.85622-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201209_081753_551516_721C2D81 X-CRM114-Status: GOOD ( 16.44 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kernel-team@android.com, Suzuki K Poulose , Catalin Marinas , Jamie Iles , linux-kernel@vger.kernel.org, James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , David Brazdil , Will Deacon , Ard Biesheuvel , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Jamie Iles The nVHE percpu data is partially linked but the nVHE linker script did not align the percpu section. The PERCPU_INPUT macro would then align the data to a page boundary: #define PERCPU_INPUT(cacheline) \ __per_cpu_start = .; \ *(.data..percpu..first) \ . = ALIGN(PAGE_SIZE); \ *(.data..percpu..page_aligned) \ . = ALIGN(cacheline); \ *(.data..percpu..read_mostly) \ . = ALIGN(cacheline); \ *(.data..percpu) \ *(.data..percpu..shared_aligned) \ PERCPU_DECRYPTED_SECTION \ __per_cpu_end = .; but then when the final vmlinux linking happens the hypervisor percpu data is included after page alignment and so the offsets potentially don't match. On my build I saw that the .hyp.data..percpu section was at address 0x20 and then the percpu data would begin at 0x1000 (because of the page alignment in PERCPU_INPUT), but when linked into vmlinux, everything would be shifted down by 0x20 bytes. This manifests as one of the CPUs getting lost when running kvm-unit-tests or starting any VM and subsequent soft lockup on a Cortex A72 device. Fixes: 30c953911c43 ("kvm: arm64: Set up hyp percpu data for nVHE") Signed-off-by: Jamie Iles Signed-off-by: Marc Zyngier Acked-by: David Brazdil Cc: David Brazdil Cc: Marc Zyngier Cc: Will Deacon Link: https://lore.kernel.org/r/20201113150406.14314-1-jamie@nuviainc.com --- arch/arm64/kvm/hyp/nvhe/hyp.lds.S | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/arch/arm64/kvm/hyp/nvhe/hyp.lds.S b/arch/arm64/kvm/hyp/nvhe/hyp.lds.S index 5d76ff2ba63e..1206d0d754d5 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp.lds.S +++ b/arch/arm64/kvm/hyp/nvhe/hyp.lds.S @@ -13,6 +13,11 @@ SECTIONS { HYP_SECTION(.text) + /* + * .hyp..data..percpu needs to be page aligned to maintain the same + * alignment for when linking into vmlinux. + */ + . = ALIGN(PAGE_SIZE); HYP_SECTION_NAME(.data..percpu) : { PERCPU_INPUT(L1_CACHE_BYTES) } From patchwork Wed Dec 9 13:17:39 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11961355 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A6CDFC4361B for ; Wed, 9 Dec 2020 13:19:20 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 63EF7207C8 for ; Wed, 9 Dec 2020 13:19:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 63EF7207C8 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=WCuXConvyMlMsIK5JHeVOc+N+hDOEmV+leeijpoc3jA=; b=BQ7+pzyT9+ixfq8+bfwbDzpEl XBLetZakUjWsx2b+Q9H3rX2DnHj1dnojNRmVZrLbYqqW9RzTgDDCwpGWLzpm98Kv2jSi/f2FeHc7u qzczaZW4iAoKlqpVOekoMDmV2J/3zk8bHiNk+Q7jlkayIqmV25nuFQjw8Pgv76ILSHcnxdMkgfST4 00hOOIWaHJZZqcsOSWmmo3qMQbBFovWsXbDAT0bpPZQZ652XLEsaEFU8FE6OMvbZW/sSHogYPaXdJ pYlh3MjoGYNDr3ibkqFBgVjg5LIIQJXqSZ5iA9FTwD7IKARRYo/oVCN9c1PeuLgVxxtxt4xutCJN6 o10bOmPEw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kmzM2-0003D4-4Y; Wed, 09 Dec 2020 13:18:02 +0000 Received: from mail-wm1-x342.google.com ([2a00:1450:4864:20::342]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kmzLv-0003BL-H0 for linux-arm-kernel@lists.infradead.org; Wed, 09 Dec 2020 13:17:56 +0000 Received: by mail-wm1-x342.google.com with SMTP id c198so1454748wmd.0 for ; Wed, 09 Dec 2020 05:17:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ODR//cwk6hwl4QVv9WxKniuODKXuHa0De3QF5nrHIF8=; b=D9l4XlNV7e+nqwFqjwgoM8i0tZtjPXnV/1t9K4p0LHGAHrDPuK6Uh4KI0K1Ccf4rYR RyzWt8rtG2bUkWDMfweQh4sRU//DySGqA6cTqBEqWrjCga2vPp0k8VLsnHuhaRz8WW0p PIh6vQZhhs+WknKoOizMB6X9fo2KSgguJmaznyyhXFHP3tg8lgDhJP4ZEDwdn+hc3nwc CJe6Say+NGiW2MOE3ooxvI8ONCt2j+yYmQP9X7HEsWluFyovewhRwyv7SS9kVpnRoZi9 Xsgl/EBsv8L0ZK5fNxVuQD0bOcT1j24yA9hR6pEk1+ESD99B5RZPMzRIQimFBNqgTeTp wicg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ODR//cwk6hwl4QVv9WxKniuODKXuHa0De3QF5nrHIF8=; b=GKuCYPh+Zkl13vmm2QGLzbEv/8swCU7LnbexcsLGorXBBoTitr6B5NXrm3xEqZQmJP QhmaXIdxurC3lIIy+SX3xHLgsEDmghiZ14EEjRdkDkcreFBp0IaV0zFeiiO5kXU7xJIM WTT3i8hfMIBAUY7Pzdz+f3ZJKmBgFK4aTM15PuJskNloqLgLk57ZhIpH4sBJqNvg+Uqq TAHqa5hHf8t35A3y5/cgYxPJRdLItkX19O1kq1G0dON7jp55gxECXWJqKhvcuT2siQvF R8vCQLoLZ+fbcZZxitGK4RucFIGGdVSpgO5TU9y8WAB9vBP/tIe1f++cV63dVKKqQFDJ wFDg== X-Gm-Message-State: AOAM532JEVnE2QPrv6qZf5vfMzGiOgAFD01YNLhQYrvJfXPMCmLSzvO6 lzTUvqQUytgtxiZPMhwOrc/GdQ== X-Google-Smtp-Source: ABdhPJxiJo216uaUR1rOvqNVgnsgrbK2oTOt+NyDqS8ioUu6zr1yo3ZpJ7RnHh81iFNCSDXZK4u0DQ== X-Received: by 2002:a7b:c843:: with SMTP id c3mr2777094wml.100.1607519874381; Wed, 09 Dec 2020 05:17:54 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:9d1b:d0eb:db43:6cd2]) by smtp.gmail.com with ESMTPSA id q25sm3750194wmq.37.2020.12.09.05.17.53 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 09 Dec 2020 05:17:53 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH 2/9] KVM: arm64: Rename .idmap.text in hyp linker script Date: Wed, 9 Dec 2020 13:17:39 +0000 Message-Id: <20201209131746.85622-3-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201209131746.85622-1-dbrazdil@google.com> References: <20201209131746.85622-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201209_081755_658906_CD559B94 X-CRM114-Status: GOOD ( 15.73 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kernel-team@android.com, Suzuki K Poulose , Catalin Marinas , linux-kernel@vger.kernel.org, James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , David Brazdil , Will Deacon , Ard Biesheuvel , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org So far hyp-init.S created a .hyp.idmap.text section directly, without relying on the hyp linker script to prefix its name. Change it to create .idmap.text and add a HYP_SECTION entry to hyp.lds.S. This way all .hyp* sections go through the linker script and can be instrumented there. Signed-off-by: David Brazdil --- arch/arm64/kvm/hyp/nvhe/hyp-init.S | 2 +- arch/arm64/kvm/hyp/nvhe/hyp.lds.S | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-init.S b/arch/arm64/kvm/hyp/nvhe/hyp-init.S index 31b060a44045..68fd64f2313e 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-init.S +++ b/arch/arm64/kvm/hyp/nvhe/hyp-init.S @@ -18,7 +18,7 @@ #include .text - .pushsection .hyp.idmap.text, "ax" + .pushsection .idmap.text, "ax" .align 11 diff --git a/arch/arm64/kvm/hyp/nvhe/hyp.lds.S b/arch/arm64/kvm/hyp/nvhe/hyp.lds.S index 1206d0d754d5..70ac48ccede7 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp.lds.S +++ b/arch/arm64/kvm/hyp/nvhe/hyp.lds.S @@ -12,6 +12,7 @@ #include SECTIONS { + HYP_SECTION(.idmap.text) HYP_SECTION(.text) /* * .hyp..data..percpu needs to be page aligned to maintain the same From patchwork Wed Dec 9 13:17:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11961359 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8F1A0C1B0E3 for ; Wed, 9 Dec 2020 13:19:23 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 513E3207C8 for ; Wed, 9 Dec 2020 13:19:23 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 513E3207C8 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=zuLX2UuMAR+pkVm9lRClK05eLVKiPRv/sVJjtvCvP8I=; b=vu3CYSSIIKK+byYmsLpGnt8U8 +2nBgth1w8Cpo9+dWb46+CXAiYcWZyrLJL7XWYyiOtTUV3KsEC/W7DWBW0uCsWpy/ewD+BwwZXiee c1EJPjl/9x6DZ3LJ17ru+sqbeqofdMSaRwJniUZEMj+Kf3vPQBr73hEuH1ySTxADJ4La12aRgy+TI pDCPssnYV0EeR5phVQftooE6oqCxqf06uuRB+itM0Kj4fyPAMZG/PlBqlriXVOS1OdZrQHGDb6eqc be2PlaYsBQq6ieAErP99759dCFyp2FgRHy/a4j0dF4W3yz9eSwR9M5bXt7FLc0JO6WU6aHhMdTOOb +mqY5RhFw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kmzM8-0003Fh-J1; Wed, 09 Dec 2020 13:18:08 +0000 Received: from mail-wm1-x343.google.com ([2a00:1450:4864:20::343]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kmzLx-0003By-PE for linux-arm-kernel@lists.infradead.org; Wed, 09 Dec 2020 13:17:58 +0000 Received: by mail-wm1-x343.google.com with SMTP id a3so1636226wmb.5 for ; Wed, 09 Dec 2020 05:17:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=eC3dfrduk9N3bItne4Bt2OJ3xFC82tVD26IiWD/PAzc=; b=ZhdHHWTrxybOS3ckjtAmTjfzvZosyvpJGZcwcPadU5OeCIPFNwt1A3F29cIGChqQBi 5+cQAEs0QKUeKAH+f6I3IbNBTY+dZ6M6X2DC4cxNXVKRlCLMUyoGV/iIJJQ++TGYt7cN HWQ+ttLyL29Ckl0l8dhFUEMOYLBhF9H7F62yggNGMq9+pE6vz7cxq3JDXBa2owu7dv4q Fek1uPRomzQ3VbhTcMdLcgYbIayHaX/zgPY0yjxJGjH8qB7QIFZfIrOdceHWn3ldY3Zg lbhHMVOb92NEJeoEnF/6sQXRszqVs+xGV8veUIdFoDvVQ0ts2N+ory1Vz2SwyYpZ+x2n csmg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=eC3dfrduk9N3bItne4Bt2OJ3xFC82tVD26IiWD/PAzc=; b=bodHlaBwVQKImwV4vbyZmZCUhK/WVtjm2e5vmaIUyHVxr2tKrq8m52T5VrjOsnb0yT VdxeFdLLjrQ5A0hyj0RQGW8OFkmtaLQlxJR5h4cJ25AC/GJ3H5kMWblDra4gHjebqbGn TNBYGkywI+3dTmeQNyER4JQR2Oao9dO3UEp5m/oDX+Lh0DyTtoLopwuNcVC36Bu0HXmg reJKYGCVDm6GzopdsGBWOnNF11wcPvY9LLwXCNIQwzE2DUXuCuRFy7FTpFNNGfuLNebS nVzVLRzFfINFGiR5ouTl8tvm/5yO1qtsLfKahIQ9TFUUh2t7b3PuecQq9f4kGEIvIRB1 zFSw== X-Gm-Message-State: AOAM530TecHkcbNqF64EohA5xuNidXUZkQDEpgxknrHVS2Fja6eyAZga SzaXicdjnflHu6aLEK9xjCVA4g== X-Google-Smtp-Source: ABdhPJyJlAdG9Q2cJj2wx1wE7lWFVUjh7hiYEXBuQsKX5NFis+JOymoWzsrCIbt9tGIBYhRtb6nAqw== X-Received: by 2002:a1c:2c4:: with SMTP id 187mr2810150wmc.187.1607519876443; Wed, 09 Dec 2020 05:17:56 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:9d1b:d0eb:db43:6cd2]) by smtp.gmail.com with ESMTPSA id d8sm3211263wrp.44.2020.12.09.05.17.55 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 09 Dec 2020 05:17:55 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH 3/9] KVM: arm64: Set up .hyp.rodata ELF section Date: Wed, 9 Dec 2020 13:17:40 +0000 Message-Id: <20201209131746.85622-4-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201209131746.85622-1-dbrazdil@google.com> References: <20201209131746.85622-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201209_081757_883466_A1CD8FD5 X-CRM114-Status: GOOD ( 15.45 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kernel-team@android.com, Suzuki K Poulose , Catalin Marinas , linux-kernel@vger.kernel.org, James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , David Brazdil , Will Deacon , Ard Biesheuvel , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org We will need to recognize pointers in .rodata specific to hyp, so establish a .hyp.rodata ELF section. Merge it with the existing .hyp.data..ro_after_init as they are treated the same at runtime. Signed-off-by: David Brazdil --- arch/arm64/include/asm/sections.h | 2 +- arch/arm64/kernel/vmlinux.lds.S | 7 ++++--- arch/arm64/kvm/arm.c | 7 +++---- arch/arm64/kvm/hyp/nvhe/hyp.lds.S | 4 +++- 4 files changed, 11 insertions(+), 9 deletions(-) diff --git a/arch/arm64/include/asm/sections.h b/arch/arm64/include/asm/sections.h index 8ff579361731..a6f3557d1ab2 100644 --- a/arch/arm64/include/asm/sections.h +++ b/arch/arm64/include/asm/sections.h @@ -11,7 +11,7 @@ extern char __alt_instructions[], __alt_instructions_end[]; extern char __hibernate_exit_text_start[], __hibernate_exit_text_end[]; extern char __hyp_idmap_text_start[], __hyp_idmap_text_end[]; extern char __hyp_text_start[], __hyp_text_end[]; -extern char __hyp_data_ro_after_init_start[], __hyp_data_ro_after_init_end[]; +extern char __hyp_rodata_start[], __hyp_rodata_end[]; extern char __idmap_text_start[], __idmap_text_end[]; extern char __initdata_begin[], __initdata_end[]; extern char __inittext_begin[], __inittext_end[]; diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index 43af13968dfd..f294f2048955 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -31,10 +31,11 @@ jiffies = jiffies_64; __stop___kvm_ex_table = .; #define HYPERVISOR_DATA_SECTIONS \ - HYP_SECTION_NAME(.data..ro_after_init) : { \ - __hyp_data_ro_after_init_start = .; \ + HYP_SECTION_NAME(.rodata) : { \ + __hyp_rodata_start = .; \ *(HYP_SECTION_NAME(.data..ro_after_init)) \ - __hyp_data_ro_after_init_end = .; \ + *(HYP_SECTION_NAME(.rodata)) \ + __hyp_rodata_end = .; \ } #define HYPERVISOR_PERCPU_SECTION \ diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 6e637d2b4cfb..c244e57f9cd9 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1745,11 +1745,10 @@ static int init_hyp_mode(void) goto out_err; } - err = create_hyp_mappings(kvm_ksym_ref(__hyp_data_ro_after_init_start), - kvm_ksym_ref(__hyp_data_ro_after_init_end), - PAGE_HYP_RO); + err = create_hyp_mappings(kvm_ksym_ref(__hyp_rodata_start), + kvm_ksym_ref(__hyp_rodata_end), PAGE_HYP_RO); if (err) { - kvm_err("Cannot map .hyp.data..ro_after_init section\n"); + kvm_err("Cannot map .hyp.rodata section\n"); goto out_err; } diff --git a/arch/arm64/kvm/hyp/nvhe/hyp.lds.S b/arch/arm64/kvm/hyp/nvhe/hyp.lds.S index 70ac48ccede7..cfdc59b4329b 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp.lds.S +++ b/arch/arm64/kvm/hyp/nvhe/hyp.lds.S @@ -14,6 +14,9 @@ SECTIONS { HYP_SECTION(.idmap.text) HYP_SECTION(.text) + HYP_SECTION(.data..ro_after_init) + HYP_SECTION(.rodata) + /* * .hyp..data..percpu needs to be page aligned to maintain the same * alignment for when linking into vmlinux. @@ -22,5 +25,4 @@ SECTIONS { HYP_SECTION_NAME(.data..percpu) : { PERCPU_INPUT(L1_CACHE_BYTES) } - HYP_SECTION(.data..ro_after_init) } From patchwork Wed Dec 9 13:17:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11961363 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2731DC433FE for ; Wed, 9 Dec 2020 13:19:30 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C5EAF207C8 for ; Wed, 9 Dec 2020 13:19:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C5EAF207C8 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=OxmsF4sYafeLyp/xDHxYyoIte3xpyn84emwdMMYAkns=; b=rjOI4afjy4aQCAygepJWF9X+T 2EQ/NlP8wINIm3CJdnoZ6vscn/cZWWLPTGoL0eTOsgNKPKcEwpwdHlUIKLCddsJm8/ObQ5zxZ2WzE +JkaVdlUofJ8uvoxMx7+0UQRt7shuDW6hxbbeXptDx+SZm+cRVLDjTR/vqNsv+CFWwZo8LtZIyq6+ oB1pQoTe1+tYrgFb0ozpQkLtZAq99k2C3M7wJQgPLsmqk7lfjWLLVUkwowayQVggTTqSekurJsFgY njgxfs5bCm/sJCxunV5CZsZfEjzf6SL5/QX5Q4rO5paKHcqSiJj6K5EqMdUNd3RdjJGlz/65eT41q GSZF5mM3Q==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kmzMC-0003Gw-NZ; Wed, 09 Dec 2020 13:18:12 +0000 Received: from mail-wm1-x341.google.com ([2a00:1450:4864:20::341]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kmzLz-0003Cd-Nz for linux-arm-kernel@lists.infradead.org; Wed, 09 Dec 2020 13:18:01 +0000 Received: by mail-wm1-x341.google.com with SMTP id e25so1650792wme.0 for ; Wed, 09 Dec 2020 05:17:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=1ZeZvh+Y/5GhqSEze9JV4PDzqUs6TgMApcSFco5AbC8=; b=FSv0XsPfA6t0nU+P8LEUQP0tv0hSw7PcN78XgG+fOiyTmrIZsXkOpEb6lrmGaeXIac A7LoODo6KlWtNVNCtHJMeRB5YtnDXbwXILbLgoPcp9yrkSGDrWJr6zQJ1Zz/9k6w/o2/ m7MPV/g+eUqmVdskv7+Bj5LRzZt1V1cqdJlLlZ9TypxfYszydTO37FmVk7alAZSWCILM Pvzt0D0eeNv/KhxW1CbNgh0s4SACLOu+kP+NBHG8gsiDH/2tA8ezrIpogJ1e38XC3wYr +/mEZJ2kxdrxADV+SiyFvqLXlubb0CDfu1J35+gPzN4lowUTKo2vnwK3b7PkeDyroIf/ vr9A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=1ZeZvh+Y/5GhqSEze9JV4PDzqUs6TgMApcSFco5AbC8=; b=UzSxaOaZ20bvfaE9exKPoSSu572syxQ5zGItxwOlOetXCwRUfxWp+0sQu7YrG79LJn Oy4W0fxvHZpdZhU766jAqE1c2ZjbVZ9qJ/nNYiwEEDxLqtXBXJYpIlYWj+NQ4BsTjyUN tOzkwxQ10gaHcfsv/I9vtXbtc0ZoxvtAL12MhiiuEuAlbh9H//DsoKqgeNeDnmQ4XOj7 gY5qn7K1zLE0GnrFBBYqJVsMo/lcXZ9AJZqh5OHB9mc4TribLo5XkHnbAop58i7QQBAn nbxwddzUPc6AAdHQqPEXhFqEzBQFNpatmB50kST2UXKf9zfqw+VUzmnIDalogv73qUIu 8qYw== X-Gm-Message-State: AOAM533GVBgBTHzKlC70xqIcllyf8OeYNBRxVyhlrRTiK0g6EfsWwTU1 S81dAqOT6w+brDiEvWZYmAuU1A== X-Google-Smtp-Source: ABdhPJwBw6fUgCRvWYXkUr8RF5AcTIkAYAGAsL4FDkvYjYFMccPRebZNbR1xN6emEzWHsqOjDqACIA== X-Received: by 2002:a7b:c406:: with SMTP id k6mr2792910wmi.90.1607519878533; Wed, 09 Dec 2020 05:17:58 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:9d1b:d0eb:db43:6cd2]) by smtp.gmail.com with ESMTPSA id j7sm3159281wmb.40.2020.12.09.05.17.57 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 09 Dec 2020 05:17:57 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH 4/9] KVM: arm64: Add symbol at the beginning of each hyp section Date: Wed, 9 Dec 2020 13:17:41 +0000 Message-Id: <20201209131746.85622-5-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201209131746.85622-1-dbrazdil@google.com> References: <20201209131746.85622-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201209_081759_881689_4D982FDB X-CRM114-Status: GOOD ( 18.00 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kernel-team@android.com, Suzuki K Poulose , Catalin Marinas , linux-kernel@vger.kernel.org, James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , David Brazdil , Will Deacon , Ard Biesheuvel , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Generating hyp relocations will require referencing positions at a given offset from the beginning of hyp sections. Since the final layout will not be determined until the linking of `vmlinux`, modify the hyp linker script to insert a symbol at the first byte of each hyp section to use as an anchor. The linker of `vmlinux` will place the symbols together with the sections. Signed-off-by: David Brazdil --- arch/arm64/include/asm/hyp_image.h | 29 +++++++++++++++++++++++++++-- arch/arm64/kvm/hyp/nvhe/hyp.lds.S | 4 ++-- 2 files changed, 29 insertions(+), 4 deletions(-) diff --git a/arch/arm64/include/asm/hyp_image.h b/arch/arm64/include/asm/hyp_image.h index daa1a1da539e..65e8008da932 100644 --- a/arch/arm64/include/asm/hyp_image.h +++ b/arch/arm64/include/asm/hyp_image.h @@ -7,6 +7,9 @@ #ifndef __ARM64_HYP_IMAGE_H__ #define __ARM64_HYP_IMAGE_H__ +#define HYP_CONCAT(a, b) __HYP_CONCAT(a, b) +#define __HYP_CONCAT(a, b) a ## b + /* * KVM nVHE code has its own symbol namespace prefixed with __kvm_nvhe_, * to separate it from the kernel proper. @@ -21,9 +24,31 @@ */ #define HYP_SECTION_NAME(NAME) .hyp##NAME +/* Symbol defined at the beginning of each hyp section. */ +#define HYP_SECTION_SYMBOL_NAME(NAME) \ + HYP_CONCAT(__hyp_section_, HYP_SECTION_NAME(NAME)) + +/* + * Helper to generate linker script statements starting a hyp section. + * + * A symbol with a well-known name is defined at the first byte. This + * is used as a base for hyp relocations (see gen-hyprel.c). It must + * be defined inside the section so the linker of `vmlinux` cannot + * separate it from the section data. + */ +#define BEGIN_HYP_SECTION(NAME) \ + HYP_SECTION_NAME(NAME) : { \ + HYP_SECTION_SYMBOL_NAME(NAME) = .; + +/* Helper to generate linker script statements ending a hyp section. */ +#define END_HYP_SECTION \ + } + /* Defines an ELF hyp section from input section @NAME and its subsections. */ -#define HYP_SECTION(NAME) \ - HYP_SECTION_NAME(NAME) : { *(NAME NAME##.*) } +#define HYP_SECTION(NAME) \ + BEGIN_HYP_SECTION(NAME) \ + *(NAME NAME##.*) \ + END_HYP_SECTION /* * Defines a linker script alias of a kernel-proper symbol referenced by diff --git a/arch/arm64/kvm/hyp/nvhe/hyp.lds.S b/arch/arm64/kvm/hyp/nvhe/hyp.lds.S index cfdc59b4329b..cd119d82d8e3 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp.lds.S +++ b/arch/arm64/kvm/hyp/nvhe/hyp.lds.S @@ -22,7 +22,7 @@ SECTIONS { * alignment for when linking into vmlinux. */ . = ALIGN(PAGE_SIZE); - HYP_SECTION_NAME(.data..percpu) : { + BEGIN_HYP_SECTION(.data..percpu) PERCPU_INPUT(L1_CACHE_BYTES) - } + END_HYP_SECTION } From patchwork Wed Dec 9 13:17:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11961367 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6EF09C4167B for ; Wed, 9 Dec 2020 13:19:36 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 16D39207C8 for ; Wed, 9 Dec 2020 13:19:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 16D39207C8 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=wWy+8ebBIH8sf/qYpKmwmvs2MLAg76h4hnC3w1idiTc=; b=ACS4I5or+53ald/wGo1dNn5CE qHOHT2c6oaHgvx7GkwTeFvk9MUVACLmxr7mOiKr50z+o2JegtV+D41m5KTOWUYivtgO4GL/fqYMMv 69S/kuwzjM1yOUpH3fxG7uqoN4ionVqvZgnqgY9YABGnqoM7wSGOqfFnj1duFoDLm/Nmt0IREtTLp Wy3akySlG1eAT0/AYdOviS9HnAR2k0inaV2TCv8LEWf2SrgsuExsTBc8kshkPRHslv+wQH4mGgtRD REkevMSEFz7UTbGQA6hOEmcllr8PuL67xlXAKoeoA1qiDdVmEAfBOsvzi+KTizWEw9zw9qvLa8rM/ TM+1FpzrQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kmzMG-0003Hv-L7; Wed, 09 Dec 2020 13:18:16 +0000 Received: from mail-wm1-x342.google.com ([2a00:1450:4864:20::342]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kmzM1-0003D7-Mc for linux-arm-kernel@lists.infradead.org; Wed, 09 Dec 2020 13:18:05 +0000 Received: by mail-wm1-x342.google.com with SMTP id q75so1639458wme.2 for ; Wed, 09 Dec 2020 05:18:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=P5bAFePvbaR1pJmLDHOGdjiNclSu0w7s2YN6/fQb2pw=; b=XiVTBmmJ8R6sZr/zHHE8/NBpFkElPZUpWhtIcEiFRGajkowNmv9tSUqPrWoZKpIOSe J/NcrcFa23sMD8oF+TBryfyf6/Y/taTtxuxWSrV9avklCVul/62WSraRxIPp9yipJmFK Ky7S6WubyNLa9CG01gMP+UjRccNlBy/zHlNWqPKElzWFy/VZSrt/7O0S8SJIiRaWX+EV W/nFrfl3wjAfmJuA91nEMGZL7W5cABawopi0vaxGo9G9QCsf5WrXosvvdj0yA8Vz8foW Z9WGSjTcRH+vjMUpV1p5u+Dyi6BpR4cUVDPJl+tuzk7Deg936uojTIcujuvPDsRjLk2f MFYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=P5bAFePvbaR1pJmLDHOGdjiNclSu0w7s2YN6/fQb2pw=; b=T8xXZfDUAmUhJ9tUPKerwN+AjjLSDH3LX6In1aFpYTZsRjjZ2j78ODaaZXn2tO5n3J HzIcRP84EC+y3QLJtebbw5uLvJNGpsJh7nCJKhDcP1A4zsoGHGmp390fGPxrQlG9JHRQ hfTgdPEJ3m95P/hUAYNIa1Q+Ga8oyxrxSXsHND27NWbCmzCuFfI3TW2MMJ0m4ZshhoQi b+9dyiVjjgY7k8qOuGD7PNX9NyXzNtx5D9Fk8jumvDfYRf+oAesOvwV3Cb6SXrnrGz7G fRpe2+5QHtoIGQb/aLE17UJ3Lc2T4Hh7RolVbt7giMI6cBcqDP2cNEkj41JSsynCZCOB p/3w== X-Gm-Message-State: AOAM533bT83EmJT2zC9AYRPBQOMMLW4k3tC/KqAw82xnpTymXFWp6zjO C+QE68p4o3UqMGNPGSRAHZPUfA== X-Google-Smtp-Source: ABdhPJwCMCoVIoN6y4hOUH0wQLQUvTcMBRQR2JCOCmuc4i/YVi7QBkuEnmM9EChPOfFRnEyY86u6xA== X-Received: by 2002:a1c:2c4:: with SMTP id 187mr2810453wmc.187.1607519880566; Wed, 09 Dec 2020 05:18:00 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:9d1b:d0eb:db43:6cd2]) by smtp.gmail.com with ESMTPSA id f9sm3620503wrw.81.2020.12.09.05.17.59 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 09 Dec 2020 05:17:59 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH 5/9] KVM: arm64: Generate hyp relocation data Date: Wed, 9 Dec 2020 13:17:42 +0000 Message-Id: <20201209131746.85622-6-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201209131746.85622-1-dbrazdil@google.com> References: <20201209131746.85622-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201209_081801_930268_29558B80 X-CRM114-Status: GOOD ( 32.37 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kernel-team@android.com, Suzuki K Poulose , Catalin Marinas , linux-kernel@vger.kernel.org, James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , David Brazdil , Will Deacon , Ard Biesheuvel , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add a post-processing step to compilation of KVM nVHE hyp code which calls a custom host tool (gen-hyprel) on the partially linked object file (hyp sections' names prefixed). The tool lists all R_AARCH64_ABS64 data relocations targeting hyp sections and generates an assembly file that will form a new section .hyp.reloc in the kernel binary. The new section contains an array of 32-bit offsets to the positions targeted by these relocations. Since these addresses of those positions will not be determined until linking of `vmlinux`, each 32-bit entry carries a R_AARCH64_PREL32 relocation with addend + . The linker of `vmlinux` will therefore fill the slot accordingly. This relocation data will be used at runtime to convert the kernel VAs at those positions to hyp VAs. Signed-off-by: David Brazdil --- arch/arm64/kernel/vmlinux.lds.S | 11 + arch/arm64/kvm/hyp/nvhe/Makefile | 28 ++- arch/arm64/kvm/hyp/nvhe/gen-hyprel.c | 326 +++++++++++++++++++++++++++ 3 files changed, 362 insertions(+), 3 deletions(-) create mode 100644 arch/arm64/kvm/hyp/nvhe/gen-hyprel.c diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index f294f2048955..93cef9607c0e 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -43,10 +43,19 @@ jiffies = jiffies_64; HYP_SECTION_NAME(.data..percpu) : { \ *(HYP_SECTION_NAME(.data..percpu)) \ } + +#define HYPERVISOR_RELOC_SECTION \ + .hyp.reloc : ALIGN(4) { \ + __hyp_reloc_begin = .; \ + *(.hyp.reloc) \ + __hyp_reloc_end = .; \ + } + #else /* CONFIG_KVM */ #define HYPERVISOR_EXTABLE #define HYPERVISOR_DATA_SECTIONS #define HYPERVISOR_PERCPU_SECTION +#define HYPERVISOR_RELOC_SECTION #endif #define HYPERVISOR_TEXT \ @@ -219,6 +228,8 @@ SECTIONS PERCPU_SECTION(L1_CACHE_BYTES) HYPERVISOR_PERCPU_SECTION + HYPERVISOR_RELOC_SECTION + .rela.dyn : ALIGN(8) { *(.rela .rela*) } diff --git a/arch/arm64/kvm/hyp/nvhe/Makefile b/arch/arm64/kvm/hyp/nvhe/Makefile index 1f1e351c5fe2..268be1376f74 100644 --- a/arch/arm64/kvm/hyp/nvhe/Makefile +++ b/arch/arm64/kvm/hyp/nvhe/Makefile @@ -6,6 +6,8 @@ asflags-y := -D__KVM_NVHE_HYPERVISOR__ ccflags-y := -D__KVM_NVHE_HYPERVISOR__ +hostprogs := gen-hyprel + obj-y := timer-sr.o sysreg-sr.o debug-sr.o switch.o tlb.o hyp-init.o host.o \ hyp-main.o hyp-smp.o psci-relay.o obj-y += ../vgic-v3-sr.o ../aarch32.o ../vgic-v2-cpuif-proxy.o ../entry.o \ @@ -19,7 +21,7 @@ obj-y += ../vgic-v3-sr.o ../aarch32.o ../vgic-v2-cpuif-proxy.o ../entry.o \ hyp-obj := $(patsubst %.o,%.nvhe.o,$(obj-y)) obj-y := kvm_nvhe.o -extra-y := $(hyp-obj) kvm_nvhe.tmp.o hyp.lds +extra-y := $(hyp-obj) kvm_nvhe.tmp.o kvm_nvhe.rel.o hyp.lds hyp-reloc.S hyp-reloc.o # 1) Compile all source files to `.nvhe.o` object files. The file extension # avoids file name clashes for files shared with VHE. @@ -42,11 +44,31 @@ LDFLAGS_kvm_nvhe.tmp.o := -r -T $(obj)/kvm_nvhe.tmp.o: $(obj)/hyp.lds $(addprefix $(obj)/,$(hyp-obj)) FORCE $(call if_changed,ld) -# 4) Produce the final 'kvm_nvhe.o', ready to be linked into 'vmlinux'. +# 4) Generate list of hyp code/data positions that need to be relocated at +# runtime. Because the hypervisor is part of the kernel binary, relocations +# produce a kernel VA. We enumerate relocations targeting hyp at build time +# and convert the kernel VAs at those positions to hyp VAs. +$(obj)/hyp-reloc.S: $(obj)/kvm_nvhe.tmp.o $(obj)/gen-hyprel + $(call if_changed,hyprel) + +# 5) Compile hyp-reloc.S and link it into the existing partially linked object. +# The object file now contains a section with pointers to hyp positions that +# will contain kernel VAs at runtime. These pointers have relocations on them +# so that they get updated as the hyp object is linked into `vmlinux`. +LDFLAGS_kvm_nvhe.rel.o := -r +$(obj)/kvm_nvhe.rel.o: $(obj)/kvm_nvhe.tmp.o $(obj)/hyp-reloc.o FORCE + $(call if_changed,ld) + +# 6) Produce the final 'kvm_nvhe.o', ready to be linked into 'vmlinux'. # Prefixes names of ELF symbols with '__kvm_nvhe_'. -$(obj)/kvm_nvhe.o: $(obj)/kvm_nvhe.tmp.o FORCE +$(obj)/kvm_nvhe.o: $(obj)/kvm_nvhe.rel.o FORCE $(call if_changed,hypcopy) +# The HYPREL command calls `gen-hyprel` to generate an assembly file with +# a list of relocations targeting hyp code/data. +quiet_cmd_hyprel = HYPREL $@ + cmd_hyprel = $(obj)/gen-hyprel $< > $@ + # The HYPCOPY command uses `objcopy` to prefix all ELF symbol names # to avoid clashes with VHE code/data. quiet_cmd_hypcopy = HYPCOPY $@ diff --git a/arch/arm64/kvm/hyp/nvhe/gen-hyprel.c b/arch/arm64/kvm/hyp/nvhe/gen-hyprel.c new file mode 100644 index 000000000000..c8423eea73e4 --- /dev/null +++ b/arch/arm64/kvm/hyp/nvhe/gen-hyprel.c @@ -0,0 +1,326 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2020 - Google LLC + * Author: David Brazdil + * + * Generates relocation information used by the kernel to convert + * absolute addresses in hyp data from kernel VAs to hyp VAs. + * + * This is necessary because hyp code is linked into the same binary + * as the kernel but executes under different memory mappings. + * If the compiler used absolute addressing, those addresses need to + * be converted before they are used by hyp code. + * + * The input of this program is the relocatable ELF object containing + * all hyp code/data, not yet linked into vmlinux. Hyp section names + * should have been prefixed with `.hyp` at this point. + * + * The output (printed to stdout) is an assembly file containing + * an array of 32-bit integers and static relocations that instruct + * the linker of `vmlinux` to populate the array entries with offsets + * to positions in the kernel binary containing VAs used by hyp code. + * + * Note that dynamic relocations could be used for the same purpose. + * However, those are only generated if CONFIG_RELOCATABLE=y. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#define HYP_SECTION_PREFIX ".hyp" +#define HYP_RELOC_SECTION ".hyp.reloc" +#define HYP_SECTION_SYMBOL_PREFIX "__hyp_section_" + +static struct { + const char *path; + char *begin; + size_t size; + Elf64_Ehdr *ehdr; + Elf64_Shdr *sh_table; + const char *sh_string; +} elf; + +static size_t asm_reloc_offset; + +#define fatal_error(fmt, ...) \ + ({ \ + fprintf(stderr, "error: %s: " fmt "\n", \ + elf.path, ## __VA_ARGS__); \ + exit(EXIT_FAILURE); \ + __builtin_unreachable(); \ + }) + +#define fatal_perror(msg) \ + ({ \ + fprintf(stderr, "error: %s: " msg ": %s\n", \ + elf.path, strerror(errno)); \ + exit(EXIT_FAILURE); \ + __builtin_unreachable(); \ + }) + +#define assert_op(lhs, rhs, fmt, op) \ + ({ \ + typeof(lhs) _lhs = (lhs); \ + typeof(rhs) _rhs = (rhs); \ + \ + if (!(_lhs op _rhs)) { \ + fatal_error("assertion " #lhs " " #op " " #rhs \ + " failed (lhs=" fmt ", rhs=" fmt \ + ", line=%d)", _lhs, _rhs, __LINE__); \ + } \ + }) + +#define assert_eq(lhs, rhs, fmt) assert_op(lhs, rhs, fmt, ==) +#define assert_ne(lhs, rhs, fmt) assert_op(lhs, rhs, fmt, !=) +#define assert_lt(lhs, rhs, fmt) assert_op(lhs, rhs, fmt, <) +#define assert_ge(lhs, rhs, fmt) assert_op(lhs, rhs, fmt, >=) + +/* + * Return a pointer of a given type at a given offset from + * the beginning of the ELF file. + */ +#define elf_ptr(type, off) ((type *)(elf.begin + (off))) + +/* Iterate over all sections in the ELF. */ +#define for_each_section(var) \ + for (var = elf.sh_table; var < elf.sh_table + elf.ehdr->e_shnum; ++var) + +/* Iterate over all Elf64_Rela relocations in a given section. */ +#define for_each_rela(shdr, var) \ + for (var = elf_ptr(Elf64_Rela, shdr->sh_offset); \ + var < elf_ptr(Elf64_Rela, shdr->sh_offset + shdr->sh_size); var++) + +/* True if a string starts with a given prefix. */ +static inline bool starts_with(const char *str, const char *prefix) +{ + return memcmp(str, prefix, strlen(prefix)) == 0; +} + +/* Returns a string containing the name of a given section. */ +static inline const char *section_name(Elf64_Shdr *shdr) +{ + return elf.sh_string + shdr->sh_name; +} + +/* Returns a pointer to the first byte of section data. */ +static inline const char *section_begin(Elf64_Shdr *shdr) +{ + return elf_ptr(char, shdr->sh_offset); +} + +/* Find a section by its offset from the beginning of the file. */ +static inline Elf64_Shdr *section_by_off(Elf64_Off off) +{ + assert_ne(off, 0UL, "%lu"); + return elf_ptr(Elf64_Shdr, off); +} + +/* Find a section by its index. */ +static inline Elf64_Shdr *section_by_idx(uint16_t idx) +{ + assert_ne(idx, SHN_UNDEF, "%u"); + return &elf.sh_table[idx]; +} + +/* + * Memory-map the given ELF file, perform sanity checks, and + * populate global state. + */ +static void init_elf(const char *path) +{ + int fd, ret; + struct stat stat; + + /* Store path in the global struct for error printing. */ + elf.path = path; + + /* Open the ELF file. */ + fd = open(path, O_RDONLY); + if (fd < 0) + fatal_perror("Could not open ELF file"); + + /* Get status of ELF file to obtain its size. */ + ret = fstat(fd, &stat); + if (ret < 0) { + close(fd); + fatal_perror("Could not get status of ELF file"); + } + + /* mmap() the entire ELF file read-only at an arbitrary address. */ + elf.begin = mmap(0, stat.st_size, PROT_READ, MAP_PRIVATE, fd, 0); + if (elf.begin == MAP_FAILED) { + close(fd); + fatal_perror("Could not mmap ELF file"); + } + + /* mmap() was successful, close the FD. */ + close(fd); + + /* Get pointer to the ELF header. */ + assert_ge(stat.st_size, sizeof(*elf.ehdr), "%lu"); + elf.ehdr = elf_ptr(Elf64_Ehdr, 0); + + /* Check the ELF magic. */ + assert_eq(elf.ehdr->e_ident[EI_MAG0], ELFMAG0, "0x%x"); + assert_eq(elf.ehdr->e_ident[EI_MAG1], ELFMAG1, "0x%x"); + assert_eq(elf.ehdr->e_ident[EI_MAG2], ELFMAG2, "0x%x"); + assert_eq(elf.ehdr->e_ident[EI_MAG3], ELFMAG3, "0x%x"); + + /* Sanity check that this is an ELF64 relocatable object for Aarch64. */ + assert_eq(elf.ehdr->e_ident[EI_CLASS], ELFCLASS64, "%u"); + assert_eq(elf.ehdr->e_ident[EI_DATA], ELFDATA2LSB, "%u"); + assert_eq(elf.ehdr->e_type, ET_REL, "%u"); + assert_eq(elf.ehdr->e_machine, EM_AARCH64, "%u"); + + /* Populate fields of the global struct. */ + elf.sh_table = section_by_off(elf.ehdr->e_shoff); + elf.sh_string = section_begin(section_by_idx(elf.ehdr->e_shstrndx)); +} + +/* Print the prologue of the output ASM file. */ +static void emit_prologue(void) +{ + printf(".data\n" + ".pushsection " HYP_RELOC_SECTION ", \"a\"\n"); +} + +/* + * Print ASM statements to create a hyp relocation entry for a given + * R_AARCH64_ABS64 relocation. + * + * The linker of vmlinux will populate the position given by `rela` with + * an absolute 64-bit kernel VA. If the kernel is relocatable, it will + * also generate a dynamic relocation entry so that the kernel can shift + * the address at runtime for KASLR. + * + * Emit a 32-bit offset from the current address to the position given + * by `rela`. This way the kernel can iterate over all kernel VAs used + * by hyp at runtime and convert them to hyp VAs. However, that offset + * will not be known until linking of `vmlinux`, so emit a PREL32 + * relocation referencing a symbol that the hyp linker script put at + * the beginning of the relocated section + the offset from `rela`. + */ +static void emit_rela_abs64(Elf64_Rela *rela, const char *sh_orig_name) +{ + /* Create storage for the 32-bit offset. */ + printf(".word 0\n"); + + /* + * Create a PREL32 relocation which instructs the linker of `vmlinux` + * to insert offset to position + , where is + * a symbol at the beginning of the relocated section, and + * is `rela->r_offset`. + */ + printf(".reloc %lu, R_AARCH64_PREL32, %s%s + 0x%lx\n", + asm_reloc_offset, HYP_SECTION_SYMBOL_PREFIX, sh_orig_name, + rela->r_offset); + asm_reloc_offset += 4; +} + +/* Print the epilogue of the output ASM file. */ +static void emit_epilogue(void) +{ + printf(".popsection\n"); +} + +/* + * Iterate over all RELA relocations in a given section and emit + * hyp relocation data for all absolute addresses in hyp code/data. + * + * Static relocations that generate PC-relative-addressing are ignored. + */ +static void emit_rela_section(Elf64_Shdr *sh_rela) +{ + Elf64_Shdr *sh_orig = &elf.sh_table[sh_rela->sh_info]; + const char *sh_orig_name = section_name(sh_orig); + Elf64_Rela *rela; + + /* Skip all non-hyp sections. */ + if (!starts_with(sh_orig_name, HYP_SECTION_PREFIX)) + return; + + for_each_rela(sh_rela, rela) { + uint32_t type = (uint32_t)rela->r_info; + + /* Check that rela points inside the relocated section. */ + assert_lt(rela->r_offset, sh_orig->sh_size, "0x%lx"); + + switch (type) { + /* Relocations to generate PC-relative addressing. */ + case R_AARCH64_LD_PREL_LO19: + case R_AARCH64_ADR_PREL_LO21: + case R_AARCH64_ADR_PREL_PG_HI21: + case R_AARCH64_ADR_PREL_PG_HI21_NC: + case R_AARCH64_ADD_ABS_LO12_NC: + case R_AARCH64_LDST8_ABS_LO12_NC: + case R_AARCH64_LDST16_ABS_LO12_NC: + case R_AARCH64_LDST32_ABS_LO12_NC: + case R_AARCH64_LDST64_ABS_LO12_NC: + case R_AARCH64_LDST128_ABS_LO12_NC: + break; + /* Relocations for control-flow instructions. */ + case R_AARCH64_TSTBR14: + case R_AARCH64_CONDBR19: + case R_AARCH64_JUMP26: + case R_AARCH64_CALL26: + break; + /* Group relocations to create PC-relative offset inline. */ + case R_AARCH64_MOVW_PREL_G0: + case R_AARCH64_MOVW_PREL_G0_NC: + case R_AARCH64_MOVW_PREL_G1: + case R_AARCH64_MOVW_PREL_G1_NC: + case R_AARCH64_MOVW_PREL_G2: + case R_AARCH64_MOVW_PREL_G2_NC: + case R_AARCH64_MOVW_PREL_G3: + break; + /* Data relocations to generate absolute addressing. */ + case R_AARCH64_ABS64: + emit_rela_abs64(rela, sh_orig_name); + break; + default: + fatal_error("Unexpected RELA type %u", type); + } + } +} + +/* Iterate over all sections and emit hyp relocation data for RELA sections. */ +static void emit_all_relocs(void) +{ + Elf64_Shdr *shdr; + + for_each_section(shdr) { + switch (shdr->sh_type) { + case SHT_REL: + fatal_error("Unexpected SHT_REL section \"%s\"", + section_name(shdr)); + case SHT_RELA: + emit_rela_section(shdr); + break; + } + } +} + +int main(int argc, const char **argv) +{ + if(argc != 2) { + fprintf(stderr, "Usage: %s \n", argv[0]); + return EXIT_FAILURE; + } + + init_elf(argv[1]); + + emit_prologue(); + emit_all_relocs(); + emit_epilogue(); + + return EXIT_SUCCESS; +} From patchwork Wed Dec 9 13:17:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11961365 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E71FC4361B for ; Wed, 9 Dec 2020 13:19:35 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CD041207C8 for ; Wed, 9 Dec 2020 13:19:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CD041207C8 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=3eRvYOiR9pOAiDGGum33QHDe6k1sT1XDyum2T41Z68M=; b=iTD2/y2mYjDpuPS1INWTlHZ5L p8xp1IxE3aKTCBC7Bvn5MHPlPSdhVtcIGZip5e32onY8r1PPtiwWYX6wWzEd6d3x8owZrCfWThRKQ bmhKCj80FY5os6tg4mDiuQcpjntQdOoRjQVPgfv17XE72H3r6Cl7faraKEBLX8OdDl2ym6GXLE9EV dofcdeyKoq6H5Z0PV2XSZGMyBiqewbp+w6gtKKlghP3oRvjNp72l9g7tRWTKA/UpDEWimMMANOsSw adPKwoMS0Lg4P3vFhS91bwyNZbCQ9wNQTZ/wMs0W7wv3iois2xuYmtfMuVCQe2qYANN3y1b53o6gZ rg92K/Oyw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kmzMM-0003JD-Cm; Wed, 09 Dec 2020 13:18:22 +0000 Received: from mail-wr1-x442.google.com ([2a00:1450:4864:20::442]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kmzM4-0003Dt-0W for linux-arm-kernel@lists.infradead.org; Wed, 09 Dec 2020 13:18:07 +0000 Received: by mail-wr1-x442.google.com with SMTP id x6so1698612wro.11 for ; Wed, 09 Dec 2020 05:18:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=AnSc1Sgxbdzkn7VrpHIozks65PIKCh74xA+FbntzNXY=; b=fV0IqsCKJt3ivo+aspsTCn4tySosMb+82n5RnA2sAYTUUnT7GT1fshweSpCBAReyj/ 2SoMtQDPP3FnvvJAKFaqDKH5dcWNHZqK/CZL7MGg9dFP7zD8KCf5PZ2a6BBcicj977Ou 0zgxnB/Skz+GAbr34KDFIGm1SWSVJpJ4mJqz8BgANsF6kW6dH0nSYakL6ykDEUf64EjV KnZWuetkZ8TvwYngLljRR9HKZHMH6bIxtjytbs5/EALMbZ/N/nU5gn7BWB3n4tc72taH T4YeQt7WHrQgQmEdBWJY/V/9Ypebzu9I3VWoWfG9dWXZ4KSomVBQkPfeyuxeCJJ1U/Cq 1mEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=AnSc1Sgxbdzkn7VrpHIozks65PIKCh74xA+FbntzNXY=; b=YVak5ERI40mB9lV5kVJnl1YT5Blk/MJOX0mu7VqEzBIBpI66LQuk+g5MeQs59jOszb s5RRzOUVJLgF+n66qTxPhxTVFJVoC7Lo02qzsDt2D4E1rHh8YNs4psUtKlw5wMm5vNLq 9Sfne5VgqYiqjycT7Y6R1Gtnf+FEPRTqrOxyHipqGfF/vu6j2bZUnsXN7XRWhdy0M+zh SLrYfnZoRPk5DZNVovYozS6dv0HdulkDMaMHYJzkqUZl/6JMxxwW0x+iyhCCEoCdFexh hXVSD5ZGvxP/4Zuy7db0hwlc0mjyvIoKOw4owwJLWXUhJs+YNyjo8230RXe3XLnzQ40z yCfw== X-Gm-Message-State: AOAM530zzW0arOFrAao+oaBInEBTYDgdcv6X6reP6A4nGkGlH0VBnE+b AKiDu7MEtdal9poFBKBHYERxFQ== X-Google-Smtp-Source: ABdhPJyFNsrtiECMYekEKiA3Vo/iG+G5NK6FdhcQmRaR2bdsDNr4G07R9LSXaBa+5xVT0OnDyzSXRw== X-Received: by 2002:adf:97ce:: with SMTP id t14mr2716349wrb.368.1607519882606; Wed, 09 Dec 2020 05:18:02 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:9d1b:d0eb:db43:6cd2]) by smtp.gmail.com with ESMTPSA id k11sm3202279wmj.42.2020.12.09.05.18.01 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 09 Dec 2020 05:18:01 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH 6/9] KVM: arm64: Apply hyp relocations at runtime Date: Wed, 9 Dec 2020 13:17:43 +0000 Message-Id: <20201209131746.85622-7-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201209131746.85622-1-dbrazdil@google.com> References: <20201209131746.85622-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201209_081804_171259_48CF321B X-CRM114-Status: GOOD ( 20.90 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kernel-team@android.com, Suzuki K Poulose , Catalin Marinas , linux-kernel@vger.kernel.org, James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , David Brazdil , Will Deacon , Ard Biesheuvel , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org KVM nVHE code runs under a different VA mapping than the kernel, hence so far it avoided using absolute addressing because the VA in a constant pool is relocated by the linker to a kernel VA (see hyp_symbol_addr). Now the kernel has access to a list of positions that contain a kimg VA but will be accessed only in hyp execution context. These are generated by the gen-hyprel build-time tool and stored in .hyp.reloc. Add early boot pass over the entries and convert the kimg VAs to hyp VAs. Note that this requires for .hyp* ELF sections to be mapped read-write at that point. Signed-off-by: David Brazdil --- arch/arm64/configs/defconfig | 1 + arch/arm64/include/asm/kvm_mmu.h | 1 + arch/arm64/include/asm/sections.h | 1 + arch/arm64/kernel/smp.c | 4 +++- arch/arm64/kvm/va_layout.c | 28 ++++++++++++++++++++++++++++ 5 files changed, 34 insertions(+), 1 deletion(-) diff --git a/arch/arm64/configs/defconfig b/arch/arm64/configs/defconfig index 5cfe3cf6f2ac..73fc9f2f2661 100644 --- a/arch/arm64/configs/defconfig +++ b/arch/arm64/configs/defconfig @@ -1092,3 +1092,4 @@ CONFIG_DEBUG_KERNEL=y # CONFIG_DEBUG_PREEMPT is not set # CONFIG_FTRACE is not set CONFIG_MEMTEST=y +# CONFIG_ARM64_VHE is not set diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h index e52d82aeadca..6bbb44011c84 100644 --- a/arch/arm64/include/asm/kvm_mmu.h +++ b/arch/arm64/include/asm/kvm_mmu.h @@ -129,6 +129,7 @@ alternative_cb_end void kvm_update_va_mask(struct alt_instr *alt, __le32 *origptr, __le32 *updptr, int nr_inst); void kvm_compute_layout(void); +void kvm_apply_hyp_relocations(void); static __always_inline unsigned long __kern_hyp_va(unsigned long v) { diff --git a/arch/arm64/include/asm/sections.h b/arch/arm64/include/asm/sections.h index a6f3557d1ab2..2f36b16a5b5d 100644 --- a/arch/arm64/include/asm/sections.h +++ b/arch/arm64/include/asm/sections.h @@ -12,6 +12,7 @@ extern char __hibernate_exit_text_start[], __hibernate_exit_text_end[]; extern char __hyp_idmap_text_start[], __hyp_idmap_text_end[]; extern char __hyp_text_start[], __hyp_text_end[]; extern char __hyp_rodata_start[], __hyp_rodata_end[]; +extern char __hyp_reloc_begin[], __hyp_reloc_end[]; extern char __idmap_text_start[], __idmap_text_end[]; extern char __initdata_begin[], __initdata_end[]; extern char __inittext_begin[], __inittext_end[]; diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index 18e9727d3f64..47142395bc91 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -434,8 +434,10 @@ static void __init hyp_mode_check(void) "CPU: CPUs started in inconsistent modes"); else pr_info("CPU: All CPU(s) started at EL1\n"); - if (IS_ENABLED(CONFIG_KVM)) + if (IS_ENABLED(CONFIG_KVM)) { kvm_compute_layout(); + kvm_apply_hyp_relocations(); + } } void __init smp_cpus_done(unsigned int max_cpus) diff --git a/arch/arm64/kvm/va_layout.c b/arch/arm64/kvm/va_layout.c index d8cc51bd60bf..fb2ca02b7270 100644 --- a/arch/arm64/kvm/va_layout.c +++ b/arch/arm64/kvm/va_layout.c @@ -82,6 +82,34 @@ __init void kvm_compute_layout(void) init_hyp_physvirt_offset(); } +/* + * The .hyp.reloc ELF section contains a list of kimg positions that + * contains kimg VAs but will be accessed only in hyp execution context. + * Convert them to hyp VAs. See gen-hyprel.c for more details. + */ +__init void kvm_apply_hyp_relocations(void) +{ + int32_t *rel; + int32_t *begin = (int32_t*)__hyp_reloc_begin; + int32_t *end = (int32_t*)__hyp_reloc_end; + + for (rel = begin; rel < end; ++rel) { + uintptr_t *ptr, kimg_va; + + /* + * Each entry contains a 32-bit relative offset from itself + * to a kimg VA position. + */ + ptr = (uintptr_t*)lm_alias((char*)rel + *rel); + + /* Read the kimg VA value at the relocation address. */ + kimg_va = *ptr; + + /* Convert to hyp VA and store back to the relocation address. */ + *ptr = __early_kern_hyp_va((uintptr_t)lm_alias(kimg_va)); + } +} + static u32 compute_instruction(int n, u32 rd, u32 rn) { u32 insn = AARCH64_BREAK_FAULT; From patchwork Wed Dec 9 13:17:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11961369 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 66B54C4361B for ; Wed, 9 Dec 2020 13:19:42 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 22A5E207C8 for ; Wed, 9 Dec 2020 13:19:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 22A5E207C8 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=jOkCEMa8kaMqSzolkFzTfeBiio4tbszOKZmu5mDCvQs=; b=phFnryZBex8huZ9EYcCf5iSS9 r3kjBBE/25sIgWQvR9CiEQpS9K7/SFlPwcNry0G9l0ZZyhb2B2U5xCrpLC79VNPBxYrXnDo0J0QMd 9/Q2QQ8IJo9VGsW+fcT+vymiahkCcSVvF8nqCsIPwiCjLkdJ9606DqJReRLI8pCyXdL4oM73ypHb8 yIeqUEXh4+9DL9x67Fyru+OeWs9h0Heuapb02UhN/pz78TJ4KPyXIUhojQEkHjypVQ6Y3c+EuAFIk DbmGc552ukl/HOxcGHw+K5D1B12btVJWNIOdxkmBnrUmR5uKoC4ym6WLPWMrFhaKR0w7hHaPQb4/O FutfCihyQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kmzMT-0003La-6e; Wed, 09 Dec 2020 13:18:29 +0000 Received: from mail-wm1-x342.google.com ([2a00:1450:4864:20::342]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kmzM5-0003EV-Rk for linux-arm-kernel@lists.infradead.org; Wed, 09 Dec 2020 13:18:07 +0000 Received: by mail-wm1-x342.google.com with SMTP id x22so1443683wmc.5 for ; Wed, 09 Dec 2020 05:18:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Ka29+EvLzBVuz3fav1ANCIbzhqqYwi6Q4TnNct9LPkw=; b=GPfaJhmZTf789tZ6o4bBL5H00oWtl28s03PNuYmXtEC0DzS7vZ6fCi2db4iIw/YC+D r1dM6gMfsFzU6WDlWzCNMeBoWmQ6xEjz4kIgTDVNATNmHG75fJQfbXu4jdZRTiYfpq2H Dk7CPz33RuE6JEDrNdDlpREIanGgLVvYaQOeaOn7XWPvQhziehz3AmfI5oRiyyTn9dq3 fp9/AM3vLnDDsyE4sfNiz56C2yhMp2bG0rme7rq1sxMIRb+AKEbJR/j4/FDFGAYqX6/s Ea/eQzy86guK9WGUt9pl73Ymd5MjC7fLdWmrNJ6fYcgqmjk/jIFL+8NHWrD6BljBKy/S kH8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Ka29+EvLzBVuz3fav1ANCIbzhqqYwi6Q4TnNct9LPkw=; b=Q/rpz033XpD+Ga50HP7S651MduNHQ3kBd6KdwyV+6ggIcTwXoGmiPExAKq5tgNq7Tx oQlWh6JfH+cEukQWf4WguXZHVWu+X0cxh3S5I3nHxbqWqwE3Hlxlp+oGnvBqCs0oJ8cP nCDi/ZvdnJGWLuZKkFeMubEsi+cv2+ydsjqfrGXhLd0vusO7+6BjSbMKJLA0YnO4ahaC Ux0cquXaG3h5FB/A+i++E/uEz3lN0Fnbpjz2sRsrHzQL6HB5zvB0tE0NsFJFgx5nCybH 5QP23/qcMDQZ4V7hj8nQhFeRrrDin8A1+eJVHtMRAyxJyfDtJ1Zy2u3crMUdaBQXfkK1 fUOg== X-Gm-Message-State: AOAM531vL2VB9k8Cnwl3NHkv5wjvvFDtKJybVgXfOO3NxVEBTbvd4k9q S3fyu/bvJ+g32/sate1U3IqMgA== X-Google-Smtp-Source: ABdhPJxUH+7Om8LR7fdHJ1zEhtXybe8x1ZcwYnq6Mf5JFSkMDCTGF7hQ7t2De1RstIBLw7B9PuXGCg== X-Received: by 2002:a1c:e1c6:: with SMTP id y189mr2689110wmg.172.1607519884763; Wed, 09 Dec 2020 05:18:04 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:9d1b:d0eb:db43:6cd2]) by smtp.gmail.com with ESMTPSA id z64sm3240255wme.10.2020.12.09.05.18.03 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 09 Dec 2020 05:18:03 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH 7/9] KVM: arm64: Fix constant-pool users in hyp Date: Wed, 9 Dec 2020 13:17:44 +0000 Message-Id: <20201209131746.85622-8-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201209131746.85622-1-dbrazdil@google.com> References: <20201209131746.85622-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201209_081805_978796_71FE5E27 X-CRM114-Status: GOOD ( 18.36 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kernel-team@android.com, Suzuki K Poulose , Catalin Marinas , linux-kernel@vger.kernel.org, James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , David Brazdil , Will Deacon , Ard Biesheuvel , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hyp code uses absolute addressing to obtain a kimg VA of a small number of kernel symbols. Since the kernel now converts constant pool addresses to hyp VAs, this trick does not work anymore. Change the helpers to convert from hyp VA back to kimg VA or PA, as needed and rework the callers accordingly. Signed-off-by: David Brazdil --- arch/arm64/include/asm/kvm_mmu.h | 42 ++++++++++++------------------ arch/arm64/kvm/hyp/nvhe/host.S | 29 +++++++++++---------- arch/arm64/kvm/hyp/nvhe/hyp-init.S | 2 -- 3 files changed, 31 insertions(+), 42 deletions(-) diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h index 6bbb44011c84..adadc468cc71 100644 --- a/arch/arm64/include/asm/kvm_mmu.h +++ b/arch/arm64/include/asm/kvm_mmu.h @@ -73,49 +73,39 @@ alternative_cb_end .endm /* - * Convert a kernel image address to a PA - * reg: kernel address to be converted in place + * Convert a hypervisor VA to a PA + * reg: hypervisor address to be converted in place * tmp: temporary register - * - * The actual code generation takes place in kvm_get_kimage_voffset, and - * the instructions below are only there to reserve the space and - * perform the register allocation (kvm_get_kimage_voffset uses the - * specific registers encoded in the instructions). */ -.macro kimg_pa reg, tmp -alternative_cb kvm_get_kimage_voffset - movz \tmp, #0 - movk \tmp, #0, lsl #16 - movk \tmp, #0, lsl #32 - movk \tmp, #0, lsl #48 -alternative_cb_end - - /* reg = __pa(reg) */ - sub \reg, \reg, \tmp +.macro hyp_pa reg, tmp + ldr_l \tmp, hyp_physvirt_offset + add \reg, \reg, \tmp .endm /* - * Convert a kernel image address to a hyp VA - * reg: kernel address to be converted in place + * Convert a hypervisor VA to a kernel image address + * reg: hypervisor address to be converted in place * tmp: temporary register * * The actual code generation takes place in kvm_get_kimage_voffset, and * the instructions below are only there to reserve the space and - * perform the register allocation (kvm_update_kimg_phys_offset uses the + * perform the register allocation (kvm_get_kimage_voffset uses the * specific registers encoded in the instructions). */ -.macro kimg_hyp_va reg, tmp -alternative_cb kvm_update_kimg_phys_offset +.macro hyp_kimg_va reg, tmp + /* Convert hyp VA -> PA. */ + hyp_pa \reg, \tmp + + /* Load kimage_voffset. */ +alternative_cb kvm_get_kimage_voffset movz \tmp, #0 movk \tmp, #0, lsl #16 movk \tmp, #0, lsl #32 movk \tmp, #0, lsl #48 alternative_cb_end - sub \reg, \reg, \tmp - mov_q \tmp, PAGE_OFFSET - orr \reg, \reg, \tmp - kern_hyp_va \reg + /* Convert PA -> kimg VA. */ + add \reg, \reg, \tmp .endm #else diff --git a/arch/arm64/kvm/hyp/nvhe/host.S b/arch/arm64/kvm/hyp/nvhe/host.S index a820dfdc9c25..6585a7cbbc56 100644 --- a/arch/arm64/kvm/hyp/nvhe/host.S +++ b/arch/arm64/kvm/hyp/nvhe/host.S @@ -74,27 +74,28 @@ SYM_FUNC_END(__host_enter) * void __noreturn __hyp_do_panic(bool restore_host, u64 spsr, u64 elr, u64 par); */ SYM_FUNC_START(__hyp_do_panic) - /* Load the format arguments into x1-7 */ - mov x6, x3 - get_vcpu_ptr x7, x3 - - mrs x3, esr_el2 - mrs x4, far_el2 - mrs x5, hpfar_el2 - /* Prepare and exit to the host's panic funciton. */ mov lr, #(PSR_F_BIT | PSR_I_BIT | PSR_A_BIT | PSR_D_BIT |\ PSR_MODE_EL1h) msr spsr_el2, lr ldr lr, =panic + hyp_kimg_va lr, x6 msr elr_el2, lr - /* - * Set the panic format string and enter the host, conditionally - * restoring the host context. - */ + /* Set the panic format string. Use the, now free, LR as scratch. */ + ldr lr, =__hyp_panic_string + hyp_kimg_va lr, x6 + + /* Load the format arguments into x1-7. */ + mov x6, x3 + get_vcpu_ptr x7, x3 + mrs x3, esr_el2 + mrs x4, far_el2 + mrs x5, hpfar_el2 + + /* Enter the host, conditionally restoring the host context. */ cmp x0, xzr - ldr x0, =__hyp_panic_string + mov x0, lr b.eq __host_enter_without_restoring b __host_enter_for_panic SYM_FUNC_END(__hyp_do_panic) @@ -124,7 +125,7 @@ SYM_FUNC_END(__hyp_do_panic) * Preserve x0-x4, which may contain stub parameters. */ ldr x5, =__kvm_handle_stub_hvc - kimg_pa x5, x6 + hyp_pa x5, x6 br x5 .L__vect_end\@: .if ((.L__vect_end\@ - .L__vect_start\@) > 0x80) diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-init.S b/arch/arm64/kvm/hyp/nvhe/hyp-init.S index 68fd64f2313e..99b408fe09ee 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-init.S +++ b/arch/arm64/kvm/hyp/nvhe/hyp-init.S @@ -139,7 +139,6 @@ alternative_else_nop_endif /* Set the host vector */ ldr x0, =__kvm_hyp_host_vector - kimg_hyp_va x0, x1 msr vbar_el2, x0 ret @@ -198,7 +197,6 @@ SYM_CODE_START_LOCAL(__kvm_hyp_init_cpu) /* Leave idmap. */ mov x0, x29 ldr x1, =kvm_host_psci_cpu_entry - kimg_hyp_va x1, x2 br x1 SYM_CODE_END(__kvm_hyp_init_cpu) From patchwork Wed Dec 9 13:17:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11961371 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 39410C4361B for ; Wed, 9 Dec 2020 13:19:47 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E8324207C8 for ; Wed, 9 Dec 2020 13:19:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E8324207C8 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=c2Qg6XLZT2Dx/C+lGPhyBmQjSpYmMt7uLDJ+ZO1fIHo=; b=Mn5xSZ5rE8G8kcZ/TT+9ulKs2 6oxcjnE9JxdvLqy814uaJcJ89UP4h5TrXQ+KrxeMCvEd/ngYw89xGmm6hDLGbgx2+hqYjij/OKzKp n2xsajInmSCvfob5NkEN0FnnOv4l3ngR+OTyeaWy6oRFialRnRnUCvuU7GFiQfBIfWxc7NC1+E1lz UlchWS9Ehf52eyavo2iMql57IQVcXg1HRqG0UX8gFwIywUyy3cHRXdBfXVmtAuG+LJU7peIhpWWD9 Bj6CyvQhuwvpuXxncCcbmM1j0JvGF8qvXBLzTOoZZA0dHAllZCl3N+NyT4oQGbUegWFFX6WxVIzzL J+Xd9EdaA==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kmzMY-0003Nx-5u; Wed, 09 Dec 2020 13:18:34 +0000 Received: from mail-wr1-x442.google.com ([2a00:1450:4864:20::442]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kmzM8-0003FR-1c for linux-arm-kernel@lists.infradead.org; Wed, 09 Dec 2020 13:18:09 +0000 Received: by mail-wr1-x442.google.com with SMTP id r3so1724096wrt.2 for ; Wed, 09 Dec 2020 05:18:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=epDnqaqaZseNBctE5f6pG2jQivwHl0Mc5pVZrVSTknc=; b=vMZTrkYojozaS4K0jVCmBqRNU23vVXRN02oEMfrHJs+eTcblucPsYLIWc3Vynrw+mn mznHBDBHtGjXdR8iqBA6OGh8SbXDKjYnPobJKkjxEf50OjacfCEa2rGjfuQQ2H4H9sQJ 5huM2YXJ2VUEtYLttFKHif/j0m1y/kX0dp9/ySWEbQUDoNDnOQoOQi9Wk67kT9gYy+hO nu9q8KpDQW49sVES3PmG8NmnSn+TFu/aqzjglEBXHVBANm2olkYlmLHucLxVnRFxPQAj LHy2Ipd8nHoWiHjEDUHhwskIoTDUqmBEm8BYh+e/w3HriGC821g+MSWWqpEJBJSufmNz fHEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=epDnqaqaZseNBctE5f6pG2jQivwHl0Mc5pVZrVSTknc=; b=q5SuSmt6r1Y+guy2Sq3EDeJouQgYrdwy0INkCKu8rhD3Te7Sh7PJTEwifM7MRd5N4H 6H7o8OYGGzGMt3sl9Gaa6dp3C4b3HRFk3XR2UMYeYB4GZ5YK+puvVP6DlIUfWKwIUSsh R2ks/Ym7rOIagNuq4Cootc77HP8PDLPKoLn1o0h0yFmM2qo8IeMZosg/YRMglAP1hlQa Nl8QnVDsnh1FZZAG1OWVifCGpBv0k0frGw/eEmWxDWxpjzCAb72vuM7UqXkW6CMVMZsd brMmKiPazp76+sq002cWy2HCjg1i+ftrnQuiuQLMNaE/xmEoAHGc7zxr/AKt+7iJU0Ta 54Wg== X-Gm-Message-State: AOAM532Ps+GQaKLde2lMCw/Rs1EcIM5dBWwzvNbD6ZgUr8e1OqLKDPaD MidbQFdpcgdlxW6Ipg1gmtyL1A== X-Google-Smtp-Source: ABdhPJxW/XmoEaKVme8L2Gku0Ug7u+SX6ybe+SCrh0vpKo7yHlnDZJnDPITz8Rh8FhHshHxxfEtGIg== X-Received: by 2002:adf:f84b:: with SMTP id d11mr2693844wrq.216.1607519887099; Wed, 09 Dec 2020 05:18:07 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:9d1b:d0eb:db43:6cd2]) by smtp.gmail.com with ESMTPSA id f16sm3285407wmh.7.2020.12.09.05.18.05 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 09 Dec 2020 05:18:06 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH 8/9] KVM: arm64: Remove patching of fn pointers in hyp Date: Wed, 9 Dec 2020 13:17:45 +0000 Message-Id: <20201209131746.85622-9-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201209131746.85622-1-dbrazdil@google.com> References: <20201209131746.85622-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201209_081808_135373_E77DFA3D X-CRM114-Status: GOOD ( 14.91 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kernel-team@android.com, Suzuki K Poulose , Catalin Marinas , linux-kernel@vger.kernel.org, James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , David Brazdil , Will Deacon , Ard Biesheuvel , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Storing a function pointer in hyp now generates relocation information used at early boot to convert the address to hyp VA. The existing alternative-based conversion mechanism is therefore obsolete. Remove it and simplify its users. Acked-by: Ard Biesheuvel Signed-off-by: David Brazdil --- arch/arm64/include/asm/kvm_mmu.h | 18 ------------------ arch/arm64/kernel/image-vars.h | 1 - arch/arm64/kvm/hyp/nvhe/hyp-main.c | 11 ++++------- arch/arm64/kvm/va_layout.c | 6 ------ 4 files changed, 4 insertions(+), 32 deletions(-) diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h index adadc468cc71..90873851f677 100644 --- a/arch/arm64/include/asm/kvm_mmu.h +++ b/arch/arm64/include/asm/kvm_mmu.h @@ -135,24 +135,6 @@ static __always_inline unsigned long __kern_hyp_va(unsigned long v) #define kern_hyp_va(v) ((typeof(v))(__kern_hyp_va((unsigned long)(v)))) -static __always_inline unsigned long __kimg_hyp_va(unsigned long v) -{ - unsigned long offset; - - asm volatile(ALTERNATIVE_CB("movz %0, #0\n" - "movk %0, #0, lsl #16\n" - "movk %0, #0, lsl #32\n" - "movk %0, #0, lsl #48\n", - kvm_update_kimg_phys_offset) - : "=r" (offset)); - - return __kern_hyp_va((v - offset) | PAGE_OFFSET); -} - -#define kimg_fn_hyp_va(v) ((typeof(*v))(__kimg_hyp_va((unsigned long)(v)))) - -#define kimg_fn_ptr(x) (typeof(x) **)(x) - /* * We currently support using a VM-specified IPA size. For backward * compatibility, the default IPA size is fixed to 40bits. diff --git a/arch/arm64/kernel/image-vars.h b/arch/arm64/kernel/image-vars.h index 39289d75118d..3242502f45fa 100644 --- a/arch/arm64/kernel/image-vars.h +++ b/arch/arm64/kernel/image-vars.h @@ -64,7 +64,6 @@ __efistub__ctype = _ctype; /* Alternative callbacks for init-time patching of nVHE hyp code. */ KVM_NVHE_ALIAS(kvm_patch_vector_branch); KVM_NVHE_ALIAS(kvm_update_va_mask); -KVM_NVHE_ALIAS(kvm_update_kimg_phys_offset); KVM_NVHE_ALIAS(kvm_get_kimage_voffset); /* Global kernel state accessed by nVHE hyp code. */ diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c index bde658d51404..0cf4b750a090 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -108,9 +108,9 @@ static void handle___vgic_v3_restore_aprs(struct kvm_cpu_context *host_ctxt) typedef void (*hcall_t)(struct kvm_cpu_context *); -#define HANDLE_FUNC(x) [__KVM_HOST_SMCCC_FUNC_##x] = kimg_fn_ptr(handle_##x) +#define HANDLE_FUNC(x) [__KVM_HOST_SMCCC_FUNC_##x] = (hcall_t)handle_##x -static const hcall_t *host_hcall[] = { +static const hcall_t host_hcall[] = { HANDLE_FUNC(__kvm_vcpu_run), HANDLE_FUNC(__kvm_flush_vm_context), HANDLE_FUNC(__kvm_tlb_flush_vmid_ipa), @@ -130,7 +130,6 @@ static const hcall_t *host_hcall[] = { static void handle_host_hcall(struct kvm_cpu_context *host_ctxt) { DECLARE_REG(unsigned long, id, host_ctxt, 0); - const hcall_t *kfn; hcall_t hfn; id -= KVM_HOST_SMCCC_ID(0); @@ -138,13 +137,11 @@ static void handle_host_hcall(struct kvm_cpu_context *host_ctxt) if (unlikely(id >= ARRAY_SIZE(host_hcall))) goto inval; - kfn = host_hcall[id]; - if (unlikely(!kfn)) + hfn = host_hcall[id]; + if (unlikely(!hfn)) goto inval; cpu_reg(host_ctxt, 0) = SMCCC_RET_SUCCESS; - - hfn = kimg_fn_hyp_va(kfn); hfn(host_ctxt); return; diff --git a/arch/arm64/kvm/va_layout.c b/arch/arm64/kvm/va_layout.c index fb2ca02b7270..e0021ba960b5 100644 --- a/arch/arm64/kvm/va_layout.c +++ b/arch/arm64/kvm/va_layout.c @@ -284,12 +284,6 @@ static void generate_mov_q(u64 val, __le32 *origptr, __le32 *updptr, int nr_inst *updptr++ = cpu_to_le32(insn); } -void kvm_update_kimg_phys_offset(struct alt_instr *alt, - __le32 *origptr, __le32 *updptr, int nr_inst) -{ - generate_mov_q(kimage_voffset + PHYS_OFFSET, origptr, updptr, nr_inst); -} - void kvm_get_kimage_voffset(struct alt_instr *alt, __le32 *origptr, __le32 *updptr, int nr_inst) { From patchwork Wed Dec 9 13:17:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11961373 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2D25EC433FE for ; Wed, 9 Dec 2020 13:19:52 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E0BB3207C8 for ; Wed, 9 Dec 2020 13:19:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E0BB3207C8 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=cLvW28LLyTakttwLTJjD0371BrYH6kQrXiq4shGidAA=; b=1c6TAe0lzqiEmZsagCRuvtXLh vcXqq7lkcdNg4agqXGMlmPUaeBB50mx3fREBA2nAjanLVi4kjtfp/GFd+twwhdkO98UCKC3eBu1C9 CHgMqc9ZJshgO18khsMtIYxZhvsgCrtxTqhoY4gaX91or8+j9fELtTTlGW2uyEYrWFgthZSj6tx1L CV5ChPgslVcVijhyWL2Ti9ILmPaqsLKLUM5ebq3pTkf1vunK7SGV4COGNlRZhEIqklhdpDNlJZNH5 8hl4GPbrscSuzsIXatgLaPpnSBMiCxiCMBEY1tS7CHcIIWXG2OznY21iLsBf3CDOaKlVJ6c+mLWR1 NSzl5HyYA==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kmzMc-0003Pn-Lc; Wed, 09 Dec 2020 13:18:38 +0000 Received: from mail-wr1-x443.google.com ([2a00:1450:4864:20::443]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kmzMA-0003GK-Hs for linux-arm-kernel@lists.infradead.org; Wed, 09 Dec 2020 13:18:12 +0000 Received: by mail-wr1-x443.google.com with SMTP id r3so1724237wrt.2 for ; Wed, 09 Dec 2020 05:18:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=w6vSHAYoQKu8XGlm7CRNOjlMO18mzd4d/VT7UzJjsWc=; b=M2EjPjYaPTZQdOf4ksEbBwAQW94UxN9wS929V4aj5l6pAeXJokz3aZyp7xjnRcNiMG NKGLB/sX55udo5pfl2U52FthoDQzEdWWwqRiQEPxOHwWZ/rX2iVZmt2LOmX6nHfh8uHh R3EKiTz+TOxLmuhtEYHFPgPSsvsO0OLJcj97R8WlvFlor4NaGA2wiuAsTZ3Y40dthg1h 2AzE3Yhh1mJGmwVlXAdzxpUDJxBMzLsGjTMzkWDhUpYOogFH5yQ3IY/qJhIi08uEDH3q bhyZ2WoLFJLdPkrVQNtrLITs1ByMKmo2BAqzibbJ+hhvH8EuvBDBeMf1VNIiEmCv0k+z ADuA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=w6vSHAYoQKu8XGlm7CRNOjlMO18mzd4d/VT7UzJjsWc=; b=bxycpGMcxSRsWZ1QUtf+/XmCJZ2M7dFw4t0kUCm4v7qQLs+/FBg6ZuAXDpb07l+hxz Dst7H9wegP51m1BOqICGXDn+Dd/vbbZPI5HjIBVczlM76pTpD+rhs2y5gXY1XO1Ycqwu MZcmsVCb8rNO8zeEcaNEAewKUzXxhCyDyBB9SuoyPUsRJfQDfwy2HQX+u0tdJZ4DxJfb VC5pXzYpWUqHykp3kQleog052M4zurcDTHGF7beaw9ChXmTFNhCHndNz6gWBaPeYL60h ii7d8IXU2Djyr4Z0odvMISjEMQZveY3PU5cAlQPbqBFGISe3VeXKLN/fvw35Q8v97+6E dY6Q== X-Gm-Message-State: AOAM533FrUXJSDGFFmZ8+IrNKxyZC1OIHrygJplXyAtNQAogE6S7QPpd wDb9aAvLUcAXXJ13VfSS4whp/g== X-Google-Smtp-Source: ABdhPJyvB9EL367TzDMyOMYd5F+ScAh1vR3kafMvhosXFATpK1QBNQ3dG7m4JoGBD31iEAkyLXYwaw== X-Received: by 2002:a5d:4d88:: with SMTP id b8mr2668962wru.134.1607519889197; Wed, 09 Dec 2020 05:18:09 -0800 (PST) Received: from localhost ([2a01:4b00:8523:2d03:9d1b:d0eb:db43:6cd2]) by smtp.gmail.com with ESMTPSA id o83sm3205019wme.21.2020.12.09.05.18.07 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 09 Dec 2020 05:18:08 -0800 (PST) From: David Brazdil To: kvmarm@lists.cs.columbia.edu Subject: [PATCH 9/9] KVM: arm64: Remove hyp_symbol_addr Date: Wed, 9 Dec 2020 13:17:46 +0000 Message-Id: <20201209131746.85622-10-dbrazdil@google.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201209131746.85622-1-dbrazdil@google.com> References: <20201209131746.85622-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201209_081810_751240_46731D21 X-CRM114-Status: GOOD ( 20.24 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kernel-team@android.com, Suzuki K Poulose , Catalin Marinas , linux-kernel@vger.kernel.org, James Morse , linux-arm-kernel@lists.infradead.org, Marc Zyngier , David Brazdil , Will Deacon , Ard Biesheuvel , Julien Thierry Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hyp code used the hyp_symbol_addr helper to force PC-relative addressing because absolute addressing results in kernel VAs due to the way hyp code is linked. This is not true anymore, so remove the helper and update all of its users. Acked-by: Ard Biesheuvel Signed-off-by: David Brazdil --- arch/arm64/include/asm/kvm_asm.h | 20 -------------------- arch/arm64/kvm/hyp/include/hyp/switch.h | 4 ++-- arch/arm64/kvm/hyp/nvhe/hyp-smp.c | 4 ++-- arch/arm64/kvm/hyp/nvhe/psci-relay.c | 24 ++++++++++++------------ arch/arm64/kvm/hyp/vgic-v2-cpuif-proxy.c | 2 +- 5 files changed, 17 insertions(+), 37 deletions(-) diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h index 7ccf770c53d9..22d933e9b59e 100644 --- a/arch/arm64/include/asm/kvm_asm.h +++ b/arch/arm64/include/asm/kvm_asm.h @@ -199,26 +199,6 @@ extern void __vgic_v3_init_lrs(void); extern u32 __kvm_get_mdcr_el2(void); -/* - * Obtain the PC-relative address of a kernel symbol - * s: symbol - * - * The goal of this macro is to return a symbol's address based on a - * PC-relative computation, as opposed to a loading the VA from a - * constant pool or something similar. This works well for HYP, as an - * absolute VA is guaranteed to be wrong. Only use this if trying to - * obtain the address of a symbol (i.e. not something you obtained by - * following a pointer). - */ -#define hyp_symbol_addr(s) \ - ({ \ - typeof(s) *addr; \ - asm("adrp %0, %1\n" \ - "add %0, %0, :lo12:%1\n" \ - : "=r" (addr) : "S" (&s)); \ - addr; \ - }) - #define __KVM_EXTABLE(from, to) \ " .pushsection __kvm_ex_table, \"a\"\n" \ " .align 3\n" \ diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h index 84473574c2e7..54f4860cd87c 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -505,8 +505,8 @@ static inline void __kvm_unexpected_el2_exception(void) struct exception_table_entry *entry, *end; unsigned long elr_el2 = read_sysreg(elr_el2); - entry = hyp_symbol_addr(__start___kvm_ex_table); - end = hyp_symbol_addr(__stop___kvm_ex_table); + entry = &__start___kvm_ex_table; + end = &__stop___kvm_ex_table; while (entry < end) { addr = (unsigned long)&entry->insn + entry->insn; diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-smp.c b/arch/arm64/kvm/hyp/nvhe/hyp-smp.c index cbab0c6246e2..2048725517f8 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-smp.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-smp.c @@ -33,8 +33,8 @@ unsigned long __hyp_per_cpu_offset(unsigned int cpu) if (cpu >= ARRAY_SIZE(kvm_arm_hyp_percpu_base)) hyp_panic(); - cpu_base_array = (unsigned long *)hyp_symbol_addr(kvm_arm_hyp_percpu_base); + cpu_base_array = (unsigned long *)&kvm_arm_hyp_percpu_base; this_cpu_base = kern_hyp_va(cpu_base_array[cpu]); - elf_base = (unsigned long)hyp_symbol_addr(__per_cpu_start); + elf_base = (unsigned long)&__per_cpu_start; return this_cpu_base - elf_base; } diff --git a/arch/arm64/kvm/hyp/nvhe/psci-relay.c b/arch/arm64/kvm/hyp/nvhe/psci-relay.c index 08dc9de69314..746fb7079581 100644 --- a/arch/arm64/kvm/hyp/nvhe/psci-relay.c +++ b/arch/arm64/kvm/hyp/nvhe/psci-relay.c @@ -151,8 +151,8 @@ static int psci_cpu_on(u64 func_id, struct kvm_cpu_context *host_ctxt) if (cpu_id == INVALID_CPU_ID) return PSCI_RET_INVALID_PARAMS; - boot_args = per_cpu_ptr(hyp_symbol_addr(cpu_on_args), cpu_id); - init_params = per_cpu_ptr(hyp_symbol_addr(kvm_init_params), cpu_id); + boot_args = per_cpu_ptr(&cpu_on_args, cpu_id); + init_params = per_cpu_ptr(&kvm_init_params, cpu_id); /* Check if the target CPU is already being booted. */ if (!try_acquire_boot_args(boot_args)) @@ -163,7 +163,7 @@ static int psci_cpu_on(u64 func_id, struct kvm_cpu_context *host_ctxt) wmb(); ret = psci_call(func_id, mpidr, - __hyp_pa(hyp_symbol_addr(kvm_hyp_cpu_entry)), + __hyp_pa(&kvm_hyp_cpu_entry), __hyp_pa(init_params)); /* If successful, the lock will be released by the target CPU. */ @@ -182,8 +182,8 @@ static int psci_cpu_suspend(u64 func_id, struct kvm_cpu_context *host_ctxt) struct psci_boot_args *boot_args; struct kvm_nvhe_init_params *init_params; - boot_args = this_cpu_ptr(hyp_symbol_addr(suspend_args)); - init_params = this_cpu_ptr(hyp_symbol_addr(kvm_init_params)); + boot_args = this_cpu_ptr(&suspend_args); + init_params = this_cpu_ptr(&kvm_init_params); /* * No need to acquire a lock before writing to boot_args because a core @@ -197,7 +197,7 @@ static int psci_cpu_suspend(u64 func_id, struct kvm_cpu_context *host_ctxt) * point if it is a deep sleep state. */ return psci_call(func_id, power_state, - __hyp_pa(hyp_symbol_addr(kvm_hyp_cpu_resume)), + __hyp_pa(&kvm_hyp_cpu_resume), __hyp_pa(init_params)); } @@ -209,8 +209,8 @@ static int psci_system_suspend(u64 func_id, struct kvm_cpu_context *host_ctxt) struct psci_boot_args *boot_args; struct kvm_nvhe_init_params *init_params; - boot_args = this_cpu_ptr(hyp_symbol_addr(suspend_args)); - init_params = this_cpu_ptr(hyp_symbol_addr(kvm_init_params)); + boot_args = this_cpu_ptr(&suspend_args); + init_params = this_cpu_ptr(&kvm_init_params); /* * No need to acquire a lock before writing to boot_args because a core @@ -221,7 +221,7 @@ static int psci_system_suspend(u64 func_id, struct kvm_cpu_context *host_ctxt) /* Will only return on error. */ return psci_call(func_id, - __hyp_pa(hyp_symbol_addr(kvm_hyp_cpu_resume)), + __hyp_pa(&kvm_hyp_cpu_resume), __hyp_pa(init_params), 0); } @@ -230,12 +230,12 @@ asmlinkage void __noreturn kvm_host_psci_cpu_entry(bool is_cpu_on) struct psci_boot_args *boot_args; struct kvm_cpu_context *host_ctxt; - host_ctxt = &this_cpu_ptr(hyp_symbol_addr(kvm_host_data))->host_ctxt; + host_ctxt = &this_cpu_ptr(&kvm_host_data)->host_ctxt; if (is_cpu_on) - boot_args = this_cpu_ptr(hyp_symbol_addr(cpu_on_args)); + boot_args = this_cpu_ptr(&cpu_on_args); else - boot_args = this_cpu_ptr(hyp_symbol_addr(suspend_args)); + boot_args = this_cpu_ptr(&suspend_args); cpu_reg(host_ctxt, 0) = boot_args->r0; write_sysreg_el2(boot_args->pc, SYS_ELR); diff --git a/arch/arm64/kvm/hyp/vgic-v2-cpuif-proxy.c b/arch/arm64/kvm/hyp/vgic-v2-cpuif-proxy.c index 8f0585640241..87a54375bd6e 100644 --- a/arch/arm64/kvm/hyp/vgic-v2-cpuif-proxy.c +++ b/arch/arm64/kvm/hyp/vgic-v2-cpuif-proxy.c @@ -64,7 +64,7 @@ int __vgic_v2_perform_cpuif_access(struct kvm_vcpu *vcpu) } rd = kvm_vcpu_dabt_get_rd(vcpu); - addr = hyp_symbol_addr(kvm_vgic_global_state)->vcpu_hyp_va; + addr = kvm_vgic_global_state.vcpu_hyp_va; addr += fault_ipa - vgic->vgic_cpu_base; if (kvm_vcpu_dabt_iswrite(vcpu)) {