From patchwork Mon Jul 26 09:28:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12398927 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 23E02C432BE for ; Mon, 26 Jul 2021 09:31:41 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E7E8460238 for ; Mon, 26 Jul 2021 09:31:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org E7E8460238 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=J9yLLqnAZ3DXDtXSOobwdtuXGHkQCNNcKeF4soEIxEk=; b=siwplf6E0ZbpNznweyljRmUbyW qlbaptfGsXRPW7aWreQG+kWOweX7FN0UTcQg5O7bnsujSjqVe0F0ya8TMzmt5q00j/nfbOj0YFUdt yz8VpOemAJB9gi71lUbrY0InQorMrXHLGk3EX8cwkXgXtrqImiwlBA6j2ZCXK5+38TDlPkgAekIyy TtBi+jf/vC2YFrDb7794H1n8TuRoV59sUbDqrLCdy+jRGlXnfjPayWjQ+ygJsPEiDEO/OtB8xjuxy 8uwajTdJKa0wVFBp21v5DRIvpqC+J+dWLzbJ2xz1ZrRI6KRY20fVPWd7hcgTBcYsnlFvS4iexv+wO 0wVwxcBg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m7wvP-00ARYb-VR; Mon, 26 Jul 2021 09:29:28 +0000 Received: from mail-qt1-x84a.google.com ([2607:f8b0:4864:20::84a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m7wvD-00ARVP-N7 for linux-arm-kernel@lists.infradead.org; Mon, 26 Jul 2021 09:29:17 +0000 Received: by mail-qt1-x84a.google.com with SMTP id e19-20020ac84b530000b0290251f0b91196so4017780qts.14 for ; Mon, 26 Jul 2021 02:29:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=Gjf1jLwbr60oU/lDUo56+HNoAVcvWGbxZryQe2MxgVI=; b=DUXlt4CIx1Jj1CbDy7cdGv8bTdi62I2RYHsrT/s+KJOvTfJ/fwZsR6eGvk7It8rJcN bBoWcqIo5xKqnQFtfIFlQBaEBPm+SOq7Xuyw31L0WnzSfdn2t6kbbMQIZzXB0MUUPX8Q wxW+8/zX7bGrdM87rM65vxKXgcwv/bXY77icyCnicQZ+mAEy26wCuFtcvS96nYsEpGkg 0z8vJbr77sREiu6d0IOmXfqa+VIbetUENBqdqiS0fCh4TnmOc6+k33nLzzQDzu0GsDiL pmQlhJkSyvFpULMaSDUptBv+CZ5WsBH34jLy6q57bE4W63AjjHdbQf7QAxbCShWBSI8V MdwQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=Gjf1jLwbr60oU/lDUo56+HNoAVcvWGbxZryQe2MxgVI=; b=MzEcmOFVcpow1OczlBAhQeHWiVbGPF5hgE6kwxc/Z7+9TuR00NbIlBTHOPRnACIk3M noVSJjeZ85Bg6zPxtS+qbKaPYUCIt8WjUNL8vf9AVpV/+Lmkf5ZOrFFtRP/uOV2a/YH9 reQ9Qpd0N8MA4cEefSo8C5rQMi5lpH7TnpMx/gtJfpM/ZphS3h7TKmmcCUU/+qWaqYCq AVazp8n1w77MeQF7GfE2BZ3OXIM47eQW97G3PFHiaaf0yD/sTlhlPgEKGpTek2THBfrF 7PaWJRmRykT+rR+JIQFs/iHTvVWTljeVaPv1aGCUIxo2qQQrVHtUhv1R7IWqduQ+Gp99 ZWuA== X-Gm-Message-State: AOAM530w5WOX4BD0EXODmxiQwyerofo76fUb0k+hCAfs1e063Zw54iLF +WxZBNewE8zEySJLwovE6v+2VSpHSX+R X-Google-Smtp-Source: ABdhPJwsdWxD5Vpv+7zklcHUJYq1XgMklZukeoIEPoWdSvMcON5nO372Kxs1TISOnicfY2UxNWOoh5v1fVJj X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:23a0:2f14:433:e6cb]) (user=qperret job=sendgmr) by 2002:a05:6214:dce:: with SMTP id 14mr16478055qvt.40.1627291753422; Mon, 26 Jul 2021 02:29:13 -0700 (PDT) Date: Mon, 26 Jul 2021 10:28:50 +0100 In-Reply-To: <20210726092905.2198501-1-qperret@google.com> Message-Id: <20210726092905.2198501-2-qperret@google.com> Mime-Version: 1.0 References: <20210726092905.2198501-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.432.gabb21c7263-goog Subject: [PATCH v2 01/16] KVM: arm64: Introduce helper to retrieve a PTE and its level From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, Quentin Perret X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210726_022915_798482_4187791D X-CRM114-Status: GOOD ( 13.60 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Marc Zyngier It is becoming a common need to fetch the PTE for a given address together with its level. Add such a helper. Signed-off-by: Marc Zyngier Signed-off-by: Quentin Perret --- arch/arm64/include/asm/kvm_pgtable.h | 19 ++++++++++++++ arch/arm64/kvm/hyp/pgtable.c | 39 ++++++++++++++++++++++++++++ 2 files changed, 58 insertions(+) diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h index f004c0115d89..082b9d65f40b 100644 --- a/arch/arm64/include/asm/kvm_pgtable.h +++ b/arch/arm64/include/asm/kvm_pgtable.h @@ -432,6 +432,25 @@ int kvm_pgtable_stage2_flush(struct kvm_pgtable *pgt, u64 addr, u64 size); int kvm_pgtable_walk(struct kvm_pgtable *pgt, u64 addr, u64 size, struct kvm_pgtable_walker *walker); +/** + * kvm_pgtable_get_leaf() - Walk a page-table and retrieve the leaf entry + * with its level. + * @pgt: Page-table structure initialised by kvm_pgtable_*_init(). + * @addr: Input address for the start of the walk. + * @ptep: Pointer to storage for the retrieved PTE. + * @level: Pointer to storage for the level of the retrieved PTE. + * + * The offset of @addr within a page is ignored. + * + * The walker will walk the page-table entries corresponding to the input + * address specified, retrieving the leaf corresponding to this address. + * Invalid entries are treated as leaf entries. + * + * Return: 0 on success, negative error code on failure. + */ +int kvm_pgtable_get_leaf(struct kvm_pgtable *pgt, u64 addr, + kvm_pte_t *ptep, u32 *level); + /** * kvm_pgtable_stage2_find_range() - Find a range of Intermediate Physical * Addresses with compatible permission diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index 05321f4165e3..78f36bd5df6c 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -326,6 +326,45 @@ int kvm_pgtable_walk(struct kvm_pgtable *pgt, u64 addr, u64 size, return _kvm_pgtable_walk(&walk_data); } +struct leaf_walk_data { + kvm_pte_t pte; + u32 level; +}; + +static int leaf_walker(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, + enum kvm_pgtable_walk_flags flag, void * const arg) +{ + struct leaf_walk_data *data = arg; + + data->pte = *ptep; + data->level = level; + + return 0; +} + +int kvm_pgtable_get_leaf(struct kvm_pgtable *pgt, u64 addr, + kvm_pte_t *ptep, u32 *level) +{ + struct leaf_walk_data data; + struct kvm_pgtable_walker walker = { + .cb = leaf_walker, + .flags = KVM_PGTABLE_WALK_LEAF, + .arg = &data, + }; + int ret; + + ret = kvm_pgtable_walk(pgt, ALIGN_DOWN(addr, PAGE_SIZE), + PAGE_SIZE, &walker); + if (!ret) { + if (ptep) + *ptep = data.pte; + if (level) + *level = data.level; + } + + return ret; +} + struct hyp_map_data { u64 phys; kvm_pte_t attr; From patchwork Mon Jul 26 09:28:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12398931 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1F33DC4338F for ; Mon, 26 Jul 2021 09:32:01 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DB6A460238 for ; Mon, 26 Jul 2021 09:32:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org DB6A460238 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=7PG83Z9ARrSqfzhvUZnd8cKdCjHZoOwdtLzdvo02PYk=; b=yiexu0+zP8+XZ5dqpx2QXdCL+F XNP4LBzWjqLI6raCDE/uSzboHpXSO6sfkjAZ2lUAcKKsvW+Amx0quiH0YLF3FpGF/RoZM2HBMC9um fz3g7bxbMawsIOT9cVrkOD1JU4wOhmErcOHbmbCSItRZzBRFYomUjEy2pm0w1vcCzbwpqxwU1tY4Z HFHnDEj6ux7ChDrh5ZzQHMnTO6qXGCMQz5S9SQGw/3MQIus3FSHPmPrK3JoQqckKpUfwLhqGdEtzc Ld1YFQyKaxM2wXOc+PmXEuewx/I0mJhmzEmRU3aXwPkbdW6gPYq7sDaZa7AJSyzBTqWW/mvS1eXw3 mSe4VSTQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m7wva-00ARal-MM; Mon, 26 Jul 2021 09:29:38 +0000 Received: from mail-qk1-x749.google.com ([2607:f8b0:4864:20::749]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m7wvH-00ARVc-OW for linux-arm-kernel@lists.infradead.org; Mon, 26 Jul 2021 09:29:21 +0000 Received: by mail-qk1-x749.google.com with SMTP id b9-20020a05620a1269b02903b8bd5c7d95so8470169qkl.12 for ; Mon, 26 Jul 2021 02:29:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=SklhMqwq/2h4UO2xk/TmhlS4/BSmhG4vZDkK4FOF1kk=; b=LcLE/Jxwj2rOGbqgdQiy58nAwXiFuJnR0+9TqiIE8hs+Iyxf9iUvWeng8nwkqOFgWB BuOc+xFO+Gucu5aszLn/tMe2JBUDM0tc2WMztZaRwSiTi0Z4dI7WRe4nF1DGw2Ada5mQ EJsy1NU9tzqDQEMIt0kQYVRgSEktTdB91LYl0+Z47z30mnDwuEWgGlbsuqbVEr+rmgwo WUV8b9MRyYMu/8EWYGeJZ2scufoqME6mhLaF+TJYVQIj6VnxRKpweuJnPiv8LSh/Uy7l v+NIhechSjHyaVP7EIEbLZpQNJ+rJ7CFadROzDvvnI3yU16amLsOWHV/8yMYGz9H3HUz fFZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=SklhMqwq/2h4UO2xk/TmhlS4/BSmhG4vZDkK4FOF1kk=; b=JQtfhwX5t9MMk7zJM4Q/84uz2r7ulZHpLKgrzNXpGnRzT4UXK0VUwU4m0AP3vEMqob Vn1mDqIch6WchBXurVk3LIVqjxiO4AZlO8NFKyGvUj4dcfci+3ZWu53ZKYTXpUfFAILu 5m14D/FvUaCRUpm9x3N+c50JBDdbJJeS1Vj2Dw03FR/SSymzDoYLm3DMg6M3r+7H9uYm TjuR8owqI9RBF4sot+emzPq7KaL7OP799QuQ8X34wk6oGO6/0shLQTzGrDma8zeU5aZy bOM8gHTPYxOkYL0GPs+0mKZLfoulwcYumv/RlHvVSW/Vla6UPS2I49ocqoiAmE4Mwq/I JEoA== X-Gm-Message-State: AOAM532Uh0HicgV/skOJKhGIHQFhvqanFHgqNM5P5vxeUv8kzc3z4N3b FMVPe+fBdMIiz3jrlvCzlh9Za7FU+9Dg X-Google-Smtp-Source: ABdhPJyogngKNGamlr33/F+qdMicL+YqDoeMIdDoBqR2/1/6VaABd58xqeW5MV1R2hLXiW7ahS2yx7ibByaj X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:23a0:2f14:433:e6cb]) (user=qperret job=sendgmr) by 2002:a05:6214:ca5:: with SMTP id s5mr16960121qvs.58.1627291755454; Mon, 26 Jul 2021 02:29:15 -0700 (PDT) Date: Mon, 26 Jul 2021 10:28:51 +0100 In-Reply-To: <20210726092905.2198501-1-qperret@google.com> Message-Id: <20210726092905.2198501-3-qperret@google.com> Mime-Version: 1.0 References: <20210726092905.2198501-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.432.gabb21c7263-goog Subject: [PATCH v2 02/16] KVM: arm64: Provide the host_stage2_try() helper macro From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, Quentin Perret X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210726_022919_845222_E4F62652 X-CRM114-Status: GOOD ( 16.28 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org We currently unmap all MMIO mappings from the host stage-2 to recycle the pages whenever we run out. In order to make this pattern easy to re-use from other places, factor the logic out into a dedicated macro. While at it, apply the macro for the kvm_pgtable_stage2_set_owner() calls. They're currently only called early on and are guaranteed to succeed, but making them robust to the -ENOMEM case doesn't hurt and will avoid painful debugging sessions later on. Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 40 +++++++++++++++------------ 1 file changed, 22 insertions(+), 18 deletions(-) diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c index d938ce95d3bd..871149246f5f 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -208,6 +208,25 @@ static inline int __host_stage2_idmap(u64 start, u64 end, prot, &host_s2_pool); } +/* + * The pool has been provided with enough pages to cover all of memory with + * page granularity, but it is difficult to know how much of the MMIO range + * we will need to cover upfront, so we may need to 'recycle' the pages if we + * run out. + * + * Must be called with host_kvm.lock held. + */ +#define host_stage2_try(fn, ...) \ + ({ \ + int __ret = fn(__VA_ARGS__); \ + if (__ret == -ENOMEM) { \ + __ret = host_stage2_unmap_dev_all(); \ + if (!__ret) \ + __ret = fn(__VA_ARGS__); \ + } \ + __ret; \ + }) + static int host_stage2_idmap(u64 addr) { enum kvm_pgtable_prot prot = KVM_PGTABLE_PROT_R | KVM_PGTABLE_PROT_W; @@ -223,22 +242,7 @@ static int host_stage2_idmap(u64 addr) if (ret) goto unlock; - ret = __host_stage2_idmap(range.start, range.end, prot); - if (ret != -ENOMEM) - goto unlock; - - /* - * The pool has been provided with enough pages to cover all of memory - * with page granularity, but it is difficult to know how much of the - * MMIO range we will need to cover upfront, so we may need to 'recycle' - * the pages if we run out. - */ - ret = host_stage2_unmap_dev_all(); - if (ret) - goto unlock; - - ret = __host_stage2_idmap(range.start, range.end, prot); - + ret = host_stage2_try(__host_stage2_idmap, range.start, range.end, prot); unlock: hyp_spin_unlock(&host_kvm.lock); @@ -257,8 +261,8 @@ int __pkvm_mark_hyp(phys_addr_t start, phys_addr_t end) return -EINVAL; hyp_spin_lock(&host_kvm.lock); - ret = kvm_pgtable_stage2_set_owner(&host_kvm.pgt, start, end - start, - &host_s2_pool, pkvm_hyp_id); + ret = host_stage2_try(kvm_pgtable_stage2_set_owner, &host_kvm.pgt, + start, end - start, &host_s2_pool, pkvm_hyp_id); hyp_spin_unlock(&host_kvm.lock); return ret != -EAGAIN ? ret : 0; From patchwork Mon Jul 26 09:28:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12398925 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B27B1C4338F for ; Mon, 26 Jul 2021 09:31:40 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7798F60E09 for ; Mon, 26 Jul 2021 09:31:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 7798F60E09 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=mQDnT40fed/Cxl54orrJ8Kuh3NIC0JMi62u1Xtp4xaM=; b=3/3Usr0o3A6Dssc/XHy0wGVLdJ wneKKz6CiXYTDo4ISgs1m6Zgfe3M2m9sbJDNLkLqm0mpDwm/AWMPDKjzZLDVv7NpL9qEAEeI4dlzW BSrEtpT/hm+FzWhqA5MTvjSc0xKjzLGiSUPRODbe7tSpYLzBCP7qCtm5384AFAobwoP3+GDByCB53 rpZIuNP0nZB7dnwyn73dxEXj4mNBTEC+vGNYe8aBkuy6Yph+ygfYipSeApmBB709EgZjTlGZBd0dE Mb6z2xW8HGMsZn8I7EQsOVpvNEsikFoE+RoTK7oczscNh0hq0WYaNVbmc82chYqfY2PuH7IerfAD9 4VyW7SmQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m7wvm-00AReB-2k; Mon, 26 Jul 2021 09:29:50 +0000 Received: from mail-wm1-x349.google.com ([2a00:1450:4864:20::349]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m7wvI-00ARW5-64 for linux-arm-kernel@lists.infradead.org; Mon, 26 Jul 2021 09:29:21 +0000 Received: by mail-wm1-x349.google.com with SMTP id r125-20020a1c2b830000b0290197a4be97b7so2917128wmr.9 for ; Mon, 26 Jul 2021 02:29:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=q1zqD73fJkFVmAAq4dO3uZVcGUuPFE+HczOzBsWKqIA=; b=MGESZEF7diylBA6Kx1cQU7HsTAr2Dx3Sx2H/5ckbKa1+lNzUwqXNq3+HyK7pQ7t+79 GdSkBeHZk2on8PnJvvmF7eeNr3gYvaA4hb2R/vEdxFXLjdIYQMs6dXGIAgcJKAFuJuvk 9YIEBUcgIAThdYkh3hLGPnW+J4a2ufP8TaR5xvj7I7smpbQd6k81PXyf0wIvDOjPHSnO GsUi68byOHXQ1RiabG7/pQ9xq3UZP89oLuQwZsRgH0Szp0DpmeIOIEA518pldBB0RMru b5fxHNLKCpJtoYgrOEFncCNUgrmp+YvtHf1zA5RkIxmDYe9WMiYoGQg0yzNbKzMtuKKm B4xw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=q1zqD73fJkFVmAAq4dO3uZVcGUuPFE+HczOzBsWKqIA=; b=D3av3wSpSwkMYxxDoK01RJGexkj1WW6swi+8sFfX/Zta+Ktnae1jfXmIsrvO35b9fY zLA7Ngl17RxdiPanVfRLdwDMkKkr+3oYfWnkVta3mXULGLgGzFHakj3Tc7grScutGUW/ jzm1ufDhjvqCsmNebomOKKGtnxZ5Lk9v9fd+DS+KKyMW+JQXsWTE6F766n4sQwdCKPsK vW8XeLDpPbUsSleKqj/oLmJgsSf3DUeJ/cbqb8XmQQN0WtIfC+hyMf0flvHr+ONOzy8l I6YxQJ8NY7CNZqHvEahsObr3k/b6ZJFvnGjFdLWIbGj6ZNfvkqDNuwdmNhMrKdWlM79d ogSA== X-Gm-Message-State: AOAM530kNPvEK4/U2gzJOajzHr+6EFc7GhMPOVMMUMMbauVEEr5bIOZ7 +J+3O9Egl1Ql7XwN02Nm+mvshebSg4BZ X-Google-Smtp-Source: ABdhPJx1wCuqsWbjKEjDSr1O9R264wGcMnHVqFdGsLki/Uzi81IzUAaHKQtIg2C1kDCAaUPziAz6p3UcRgZ0 X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:23a0:2f14:433:e6cb]) (user=qperret job=sendgmr) by 2002:a05:6000:120f:: with SMTP id e15mr17783536wrx.399.1627291757591; Mon, 26 Jul 2021 02:29:17 -0700 (PDT) Date: Mon, 26 Jul 2021 10:28:52 +0100 In-Reply-To: <20210726092905.2198501-1-qperret@google.com> Message-Id: <20210726092905.2198501-4-qperret@google.com> Mime-Version: 1.0 References: <20210726092905.2198501-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.432.gabb21c7263-goog Subject: [PATCH v2 03/16] KVM: arm64: Expose page-table helpers From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, Quentin Perret X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210726_022920_289221_DFEA968B X-CRM114-Status: GOOD ( 14.76 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The KVM pgtable API exposes the kvm_pgtable_walk() function to allow the definition of walkers outside of pgtable.c. However, it is not easy to implement any of those walkers without some of the low-level helpers. Move some of them to the header file to allow re-use from other places. Signed-off-by: Quentin Perret --- arch/arm64/include/asm/kvm_pgtable.h | 27 +++++++++++++++++++++++++++ arch/arm64/kvm/hyp/pgtable.c | 26 -------------------------- 2 files changed, 27 insertions(+), 26 deletions(-) diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h index 082b9d65f40b..5a7a13bbd4a1 100644 --- a/arch/arm64/include/asm/kvm_pgtable.h +++ b/arch/arm64/include/asm/kvm_pgtable.h @@ -25,6 +25,33 @@ static inline u64 kvm_get_parange(u64 mmfr0) typedef u64 kvm_pte_t; +#define KVM_PTE_VALID BIT(0) + +static inline bool kvm_pte_valid(kvm_pte_t pte) +{ + return pte & KVM_PTE_VALID; +} + +static inline u64 kvm_granule_shift(u32 level) +{ + /* Assumes KVM_PGTABLE_MAX_LEVELS is 4 */ + return ARM64_HW_PGTABLE_LEVEL_SHIFT(level); +} + +static inline u64 kvm_granule_size(u32 level) +{ + return BIT(kvm_granule_shift(level)); +} + +static inline bool kvm_level_supports_block_mapping(u32 level) +{ + /* + * Reject invalid block mappings and don't bother with 4TB mappings for + * 52-bit PAs. + */ + return !(level == 0 || (PAGE_SIZE != SZ_4K && level == 1)); +} + /** * struct kvm_pgtable_mm_ops - Memory management callbacks. * @zalloc_page: Allocate a single zeroed memory page. diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index 78f36bd5df6c..2c5d4d3e31cc 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -11,7 +11,6 @@ #include #include -#define KVM_PTE_VALID BIT(0) #define KVM_PTE_TYPE BIT(1) #define KVM_PTE_TYPE_BLOCK 0 @@ -61,17 +60,6 @@ struct kvm_pgtable_walk_data { u64 end; }; -static u64 kvm_granule_shift(u32 level) -{ - /* Assumes KVM_PGTABLE_MAX_LEVELS is 4 */ - return ARM64_HW_PGTABLE_LEVEL_SHIFT(level); -} - -static u64 kvm_granule_size(u32 level) -{ - return BIT(kvm_granule_shift(level)); -} - #define KVM_PHYS_INVALID (-1ULL) static bool kvm_phys_is_valid(u64 phys) @@ -79,15 +67,6 @@ static bool kvm_phys_is_valid(u64 phys) return phys < BIT(id_aa64mmfr0_parange_to_phys_shift(ID_AA64MMFR0_PARANGE_MAX)); } -static bool kvm_level_supports_block_mapping(u32 level) -{ - /* - * Reject invalid block mappings and don't bother with 4TB mappings for - * 52-bit PAs. - */ - return !(level == 0 || (PAGE_SIZE != SZ_4K && level == 1)); -} - static bool kvm_block_mapping_supported(u64 addr, u64 end, u64 phys, u32 level) { u64 granule = kvm_granule_size(level); @@ -135,11 +114,6 @@ static u32 kvm_pgd_pages(u32 ia_bits, u32 start_level) return __kvm_pgd_page_idx(&pgt, -1ULL) + 1; } -static bool kvm_pte_valid(kvm_pte_t pte) -{ - return pte & KVM_PTE_VALID; -} - static bool kvm_pte_table(kvm_pte_t pte, u32 level) { if (level == KVM_PGTABLE_MAX_LEVELS - 1) From patchwork Mon Jul 26 09:28:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12398929 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0D41FC4338F for ; Mon, 26 Jul 2021 09:31:49 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D15FB60E09 for ; Mon, 26 Jul 2021 09:31:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org D15FB60E09 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=/FiBSOi0PcKIABitz39ALo2Ie12ZNpPB1wXE3/nOA0M=; b=x1ARFSlJ5V7S5lGq+rijb860Pd X9MOLjboaIEsWvm7JHaxiZ4iTWQpC+6UD/7fJb+3HKxWaBZrKTOZsGG+NKoTeZfq8ZARe9A9mg3Fe vV2Kpl7v9csqBnrjtHbeXWWKNrBwJXanjue4PN+8nyZE0FUizuLgwDngVQ5PvynpAer3B+oCWkAeI 5bs2WuxmZUNnBKxmFwvRaDeLp5We6oHOlQc2oBpoveRnYlmKjic7XMhBnwJ0zc2i2w4PytKaRc/oC iJEyBuKSwNvwABXhvf8CTKyB/bDiiOFdpVDSBV0Gmp3roMHAvoBQXfLAs9iBD4GO1f79S01+SdIVp JYCENnew==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m7ww1-00ARif-Jn; Mon, 26 Jul 2021 09:30:05 +0000 Received: from mail-qk1-x74a.google.com ([2607:f8b0:4864:20::74a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m7wvK-00ARX1-AF for linux-arm-kernel@lists.infradead.org; Mon, 26 Jul 2021 09:29:24 +0000 Received: by mail-qk1-x74a.google.com with SMTP id x2-20020a05620a0ec2b02903b8bd8b612eso7620386qkm.19 for ; Mon, 26 Jul 2021 02:29:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=84HOeXnU0pbF9G+lxWpLi5LmB45NE4KeQQD7BXdPsl8=; b=cMHNjgNR6iYTCg9SnlXd5JprFEBC42R2JR6JBPVmKO5+6SCNE5EZLYh4v7UTkvUfN6 t6wkyF9CKlEI5cXxhmXYMDFSOZYDO0AAckEXNLRWd6H6gEiNQPN6wMMeyDir/HKshs1+ FCPT5va21Vb13pSxl8fS6/iXkR+tjTYdUEjbxVBdLTbShgstSgqjPL86AcHc+Ye+ue49 VjuwZWfQB//83w9P76K/E4i6WCjZD9lZY9GImVnLwIRXaLmUJVjuwN/AlDId+x29ZZUg tCi2TqTjQ4JURKMRHBqh3A98VQyq/2IqxxOJQwm1erdvNictsIMzafbusfyAQSrOMvBm NPzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=84HOeXnU0pbF9G+lxWpLi5LmB45NE4KeQQD7BXdPsl8=; b=hOP4VITc/pvusKC+9hu0+PTmqK5PLCWPHB9xvQu8rFNN7Ydg0j5XGbTzuWUnRyFpGy HhVoEQ6cbxllo/IFjnpryRGt311AyEeZpR8sl5cpTxPV1Wm187mc0keaoAupXH9CZtj2 iIQG/2HyfkENtaBNeGY4x244c2QIju9ANjpU8MbsxQrUPWgvvyHRivJsEnya6W2CGGU5 B55EjakqvP5mw+a2wOdhnjtj7YrOpJZ7W5iDPHj5CmTGBzzKIdAk/Ej+0+D+z610u4Ta HV36an6sf9+LFSFNBoIGPp6HryS6INO3+XQBLBDKjyDlfLg7Zp5OyXJt64YrhNLxSFqx s2Ag== X-Gm-Message-State: AOAM532sbQxs1Zs/H2LzQaKYMyWg2Vsy8Xi4S2OXAibcotsNHns6CYZi LfCnLyuAzOzMa5h6Wbm9HYU5E0TPNG5h X-Google-Smtp-Source: ABdhPJyfe1AhRkOctDL7NrTZYRv4fXMomxCCFPgiSVK1+oOT3FvNCUPlQ7/f+TViZ8GyJREQgbWtnan1Sdjn X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:23a0:2f14:433:e6cb]) (user=qperret job=sendgmr) by 2002:a05:6214:a62:: with SMTP id ef2mr868179qvb.56.1627291760042; Mon, 26 Jul 2021 02:29:20 -0700 (PDT) Date: Mon, 26 Jul 2021 10:28:53 +0100 In-Reply-To: <20210726092905.2198501-1-qperret@google.com> Message-Id: <20210726092905.2198501-5-qperret@google.com> Mime-Version: 1.0 References: <20210726092905.2198501-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.432.gabb21c7263-goog Subject: [PATCH v2 04/16] KVM: arm64: Optimize host memory aborts From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, Quentin Perret X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210726_022922_405432_C83C298F X-CRM114-Status: GOOD ( 25.18 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The kvm_pgtable_stage2_find_range() function is used in the host memory abort path to try and look for the largest block mapping that can be used to map the faulting address. In order to do so, the function currently walks the stage-2 page-table and looks for existing incompatible mappings within the range of the largest possible block. If incompatible mappings are found, it tries the same procedure again, but using a smaller block range, and repeats until a matching range is found (potentially up to page granularity). While this approach has benefits (mostly in the fact that it proactively coalesces host stage-2 mappings), it can be slow if the ranges are fragmented, and it isn't optimized to deal with CPUs faulting on the same IPA as all of them will do all the work every time. To avoid these issues, remove kvm_pgtable_stage2_find_range(), and walk the page-table only once in the host_mem_abort() path to find the closest leaf to the input address. With this, use the corresponding range if it is invalid and not owned by another entity. If a valid leaf is found, return -EAGAIN similar to what is done in the kvm_pgtable_stage2_map() path to optimize concurrent faults. Signed-off-by: Quentin Perret --- arch/arm64/include/asm/kvm_pgtable.h | 30 ----------- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 39 +++++++++++++- arch/arm64/kvm/hyp/pgtable.c | 74 --------------------------- 3 files changed, 38 insertions(+), 105 deletions(-) diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h index 5a7a13bbd4a1..cec76a49f521 100644 --- a/arch/arm64/include/asm/kvm_pgtable.h +++ b/arch/arm64/include/asm/kvm_pgtable.h @@ -141,16 +141,6 @@ enum kvm_pgtable_prot { #define PAGE_HYP_RO (KVM_PGTABLE_PROT_R) #define PAGE_HYP_DEVICE (PAGE_HYP | KVM_PGTABLE_PROT_DEVICE) -/** - * struct kvm_mem_range - Range of Intermediate Physical Addresses - * @start: Start of the range. - * @end: End of the range. - */ -struct kvm_mem_range { - u64 start; - u64 end; -}; - /** * enum kvm_pgtable_walk_flags - Flags to control a depth-first page-table walk. * @KVM_PGTABLE_WALK_LEAF: Visit leaf entries, including invalid @@ -477,24 +467,4 @@ int kvm_pgtable_walk(struct kvm_pgtable *pgt, u64 addr, u64 size, */ int kvm_pgtable_get_leaf(struct kvm_pgtable *pgt, u64 addr, kvm_pte_t *ptep, u32 *level); - -/** - * kvm_pgtable_stage2_find_range() - Find a range of Intermediate Physical - * Addresses with compatible permission - * attributes. - * @pgt: Page-table structure initialised by kvm_pgtable_stage2_init*(). - * @addr: Address that must be covered by the range. - * @prot: Protection attributes that the range must be compatible with. - * @range: Range structure used to limit the search space at call time and - * that will hold the result. - * - * The offset of @addr within a page is ignored. An IPA is compatible with @prot - * iff its corresponding stage-2 page-table entry has default ownership and, if - * valid, is mapped with protection attributes identical to @prot. - * - * Return: 0 on success, negative error code on failure. - */ -int kvm_pgtable_stage2_find_range(struct kvm_pgtable *pgt, u64 addr, - enum kvm_pgtable_prot prot, - struct kvm_mem_range *range); #endif /* __ARM64_KVM_PGTABLE_H__ */ diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c index 871149246f5f..01700a908bb7 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -159,6 +159,11 @@ static int host_stage2_unmap_dev_all(void) return kvm_pgtable_stage2_unmap(pgt, addr, BIT(pgt->ia_bits) - addr); } +struct kvm_mem_range { + u64 start; + u64 end; +}; + static bool find_mem_range(phys_addr_t addr, struct kvm_mem_range *range) { int cur, left = 0, right = hyp_memblock_nr; @@ -227,6 +232,38 @@ static inline int __host_stage2_idmap(u64 start, u64 end, __ret; \ }) +static int host_stage2_find_range(u64 addr, struct kvm_mem_range *range) +{ + u64 granule, start, end; + kvm_pte_t pte; + u32 level; + int ret; + + ret = kvm_pgtable_get_leaf(&host_kvm.pgt, addr, &pte, &level); + if (ret) + return ret; + + if (kvm_pte_valid(pte)) + return -EAGAIN; + + if (pte) + return -EPERM; + + do { + granule = kvm_granule_size(level); + start = ALIGN_DOWN(addr, granule); + end = start + granule; + level++; + } while ((level < KVM_PGTABLE_MAX_LEVELS) && + (!kvm_level_supports_block_mapping(level) || + start < range->start || range->end < end)); + + range->start = start; + range->end = end; + + return 0; +} + static int host_stage2_idmap(u64 addr) { enum kvm_pgtable_prot prot = KVM_PGTABLE_PROT_R | KVM_PGTABLE_PROT_W; @@ -238,7 +275,7 @@ static int host_stage2_idmap(u64 addr) prot |= KVM_PGTABLE_PROT_X; hyp_spin_lock(&host_kvm.lock); - ret = kvm_pgtable_stage2_find_range(&host_kvm.pgt, addr, prot, &range); + ret = host_stage2_find_range(addr, &range); if (ret) goto unlock; diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index 2c5d4d3e31cc..55199e579863 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -1115,77 +1115,3 @@ void kvm_pgtable_stage2_destroy(struct kvm_pgtable *pgt) pgt->mm_ops->free_pages_exact(pgt->pgd, pgd_sz); pgt->pgd = NULL; } - -#define KVM_PTE_LEAF_S2_COMPAT_MASK (KVM_PTE_LEAF_ATTR_S2_PERMS | \ - KVM_PTE_LEAF_ATTR_LO_S2_MEMATTR | \ - KVM_PTE_LEAF_ATTR_S2_IGNORED) - -static int stage2_check_permission_walker(u64 addr, u64 end, u32 level, - kvm_pte_t *ptep, - enum kvm_pgtable_walk_flags flag, - void * const arg) -{ - kvm_pte_t old_attr, pte = *ptep, *new_attr = arg; - - /* - * Compatible mappings are either invalid and owned by the page-table - * owner (whose id is 0), or valid with matching permission attributes. - */ - if (kvm_pte_valid(pte)) { - old_attr = pte & KVM_PTE_LEAF_S2_COMPAT_MASK; - if (old_attr != *new_attr) - return -EEXIST; - } else if (pte) { - return -EEXIST; - } - - return 0; -} - -int kvm_pgtable_stage2_find_range(struct kvm_pgtable *pgt, u64 addr, - enum kvm_pgtable_prot prot, - struct kvm_mem_range *range) -{ - kvm_pte_t attr; - struct kvm_pgtable_walker check_perm_walker = { - .cb = stage2_check_permission_walker, - .flags = KVM_PGTABLE_WALK_LEAF, - .arg = &attr, - }; - u64 granule, start, end; - u32 level; - int ret; - - ret = stage2_set_prot_attr(pgt, prot, &attr); - if (ret) - return ret; - attr &= KVM_PTE_LEAF_S2_COMPAT_MASK; - - for (level = pgt->start_level; level < KVM_PGTABLE_MAX_LEVELS; level++) { - granule = kvm_granule_size(level); - start = ALIGN_DOWN(addr, granule); - end = start + granule; - - if (!kvm_level_supports_block_mapping(level)) - continue; - - if (start < range->start || range->end < end) - continue; - - /* - * Check the presence of existing mappings with incompatible - * permissions within the current block range, and try one level - * deeper if one is found. - */ - ret = kvm_pgtable_walk(pgt, start, granule, &check_perm_walker); - if (ret != -EEXIST) - break; - } - - if (!ret) { - range->start = start; - range->end = end; - } - - return ret; -} From patchwork Mon Jul 26 09:28:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12398933 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D341C4338F for ; Mon, 26 Jul 2021 09:32:07 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 42C1060E09 for ; Mon, 26 Jul 2021 09:32:07 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 42C1060E09 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=v1RUxfJ/UdMBzi8oboiJaaYjfElGRUniwCWToirrcHc=; b=XEghqX15qmlFCM/vfJl2zJWA9m a0ZCPfPtNBhYNzOXlgz6QeiZjAG+QTGYTksjejoJrvKCyIzSNslc3wR2k1dxX9/xLev/R93QZfmSa BK3m+eX7KLQ/unwQbqQ1ty6ApmjUhYdCFVbL+57jQt9q0l2CZM7gNNTTBEYN6JFzUBLD/trvSIKQr OaqT3mjCp+xrwsiaVlTEHqBtNNWbu6UCQYxNhsJQJ8X9oYBA3iRkLcCLYfYTrxZh1yH0Sm/xmOOOi T6BpTMzVWRGSlAonvOgcJhsaY2WSPk8xdXM9NQPhB72o+VuTfzLYiDoJaukCNJLORT2HNjsnytVFx RDbkWDmg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m7wwL-00ARpr-Nu; Mon, 26 Jul 2021 09:30:25 +0000 Received: from mail-wm1-x34a.google.com ([2a00:1450:4864:20::34a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m7wvM-00ARXc-VL for linux-arm-kernel@lists.infradead.org; Mon, 26 Jul 2021 09:29:26 +0000 Received: by mail-wm1-x34a.google.com with SMTP id j11-20020a05600c190bb02902190142995dso2491541wmq.4 for ; Mon, 26 Jul 2021 02:29:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=xLHFWhuaosj6mgx+BXw5wN11Ic3FuxfIig998pTjIIc=; b=L9NlMkGjnx/9vx5Zya428co80U3Z2zbDCQLn8xzTXUsTLyQSWWGmcSo3KOQsDM9tun gkbXLLdNGv/H8zmepyiFOjmTJqSPwxzvWiQDZNMPxesdKwZv63J00lJ050+AyLJd7prc LCNMi7oofPMTRj12y9X8R1JJKUERjyj7lY2e1qtZZoW7qVzrFyBgCRtzYMz46yZKycjG HDlP7DcejUKaphuW7w1QCXKWochGfRtFuNGEH3poHzXLjIKnBbNe3p1nQ5KItcuF8gub YwRPeAdiLvNHz/owaoGbrv/kYiCdNGXorjPhq74Rczi8BB7POlnwIJyUdI33/SvPcmUF 8ihA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=xLHFWhuaosj6mgx+BXw5wN11Ic3FuxfIig998pTjIIc=; b=WkMhQtGjNc8tyd9tgiSxg6pxi1IIdep5qG5wHLWE37UEj+Q/hvgbXep6aiohJMgfeQ rLIa4V4kQ3d44UBH+Yfiua4r4t9bbrXT+fyf4H4efCKfuIONB9f3gulOyi0tr7uSoHDT RU5iqn3sh4EZBDhKm4GaSbB3kiWqeux31TtHlqXTcpMfGEt0evZ4ZxBe8IoR+oed1GST 58a4ZKIySWGuotr8kpFfwEOEpFLdzofp49Y8bnRi2tNsatugR0mUqO3hvpnyKK3+ihUK MEuLxLOopJ8WtaO2a+0lp0PXUNDw29x7XpkIgfbmHg7YGf9K95ifhPp5oekeuPFCPN1j E9SA== X-Gm-Message-State: AOAM53032z3aRBj9aoXCZD3HQaDT1V0Jc79LL46LoP+pxY4paR/J1g0s 5inIxoKJsYWf7w7twVILvwfs0GqPxvBB X-Google-Smtp-Source: ABdhPJxgHTVLPDvaEjvcrmQ9zRhiX1DsEcG3LQTts/yU9YAi0Km3deA725PoMri/xgk5EjXL/D8h91KlNlZc X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:23a0:2f14:433:e6cb]) (user=qperret job=sendgmr) by 2002:a05:6000:2a1:: with SMTP id l1mr17703329wry.128.1627291762209; Mon, 26 Jul 2021 02:29:22 -0700 (PDT) Date: Mon, 26 Jul 2021 10:28:54 +0100 In-Reply-To: <20210726092905.2198501-1-qperret@google.com> Message-Id: <20210726092905.2198501-6-qperret@google.com> Mime-Version: 1.0 References: <20210726092905.2198501-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.432.gabb21c7263-goog Subject: [PATCH v2 05/16] KVM: arm64: Rename KVM_PTE_LEAF_ATTR_S2_IGNORED From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, Quentin Perret X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210726_022925_059433_C67828B8 X-CRM114-Status: GOOD ( 11.41 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The ignored bits for both stage-1 and stage-2 page and block descriptors are in [55:58], so rename KVM_PTE_LEAF_ATTR_S2_IGNORED to make it applicable to both. And while at it, since these bits are more commonly known as 'software' bits, rename accordingly. Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/pgtable.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index 55199e579863..516c1b8ce6b8 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -39,6 +39,8 @@ #define KVM_PTE_LEAF_ATTR_HI GENMASK(63, 51) +#define KVM_PTE_LEAF_ATTR_HI_SW GENMASK(58, 55) + #define KVM_PTE_LEAF_ATTR_HI_S1_XN BIT(54) #define KVM_PTE_LEAF_ATTR_HI_S2_XN BIT(54) @@ -47,8 +49,6 @@ KVM_PTE_LEAF_ATTR_LO_S2_S2AP_W | \ KVM_PTE_LEAF_ATTR_HI_S2_XN) -#define KVM_PTE_LEAF_ATTR_S2_IGNORED GENMASK(58, 55) - #define KVM_INVALID_PTE_OWNER_MASK GENMASK(63, 56) #define KVM_MAX_OWNER_ID 1 From patchwork Mon Jul 26 09:28:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12398935 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E8E13C4338F for ; Mon, 26 Jul 2021 09:33:19 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B0F8C60F02 for ; Mon, 26 Jul 2021 09:33:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org B0F8C60F02 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=gYdvdGh+FCZER1mSLpEvMG9aRAbpUyWByGtgLf/caRE=; b=qIWMbhf1Bqsb6QoCf3A6rDnC2b u90B1i6CccQGc2ebhQ+srp8yGb92uA/1g2u0rmKygu4y8MYdo2TsIyp8ENdMWBTSNb7zf0TKwmKFa KDIt6P2VARMAeK+ukDsf6X0rELNLRb6bSol4a1hTQNBdxg5vqyLyHVjVSyXIC8hQryFopGh3fXVL0 QYaMP7OWoSh/MudAk6JGHroxhk6Qy64nh1K35PcrCgB8Blx+olgenu/Bh834IZ+imuQ4bPhvzcfD0 rPoLr6bQT7KY0bY6Xbe7Q7azrW+RfPyvID1hQziKdDdS9SKFj/GWqhyDzbPtrJC4m/uDAtDE7J4F7 yf8s+KPg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m7wxJ-00ASHM-K7; Mon, 26 Jul 2021 09:31:26 +0000 Received: from mail-qv1-xf49.google.com ([2607:f8b0:4864:20::f49]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m7wvO-00ARXw-NG for linux-arm-kernel@lists.infradead.org; Mon, 26 Jul 2021 09:29:28 +0000 Received: by mail-qv1-xf49.google.com with SMTP id t4-20020a0cef040000b02902e2f9404330so7876020qvr.9 for ; Mon, 26 Jul 2021 02:29:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=xhqFHmC/x7AnH8ZRcl4l1dWbyNuPeqmFIi1qz6ztr7w=; b=ky9L1rdM2E3/pnvC1byWpnnDrketwgcUM2bCx8+9oPsj/Yt617M9WBkhR3Ympi/l4L VTeD1nruEtGe4LWOp3GLcU+7LstB63zkiS4nWY8QanDvEdLTpsDsMTSnXOla07D8Y4LE CKd5U8AUGdUjttxk0FKMZofS9v9BiNVjLm3R2YTEuFvxA2hECbp42flWzlsfaVnzYCaW ff1fKfwwzoN8osSFvI3OV/+2ArYR9Z0WC5vtT3Q8fMLNmb0YI3GsjsikhLyr2K4YGhdV OtFyNK5p9iKQ8eOXfzV9kQ58g82+/f5r/bb15bcMbUTlfDrVHghBIyXO41eZuDmkB84i dKbw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=xhqFHmC/x7AnH8ZRcl4l1dWbyNuPeqmFIi1qz6ztr7w=; b=mh8b+Hs/KvNtFxuv4OGXQ39BvEe6Mxojeb2dkKzM02EHbBlmw9q3XRQeH/3UiU/v1C 7LopjHvWdy1oajnOLsf7Dsgob/tDBx6leLNAsGYIETIoblEcNiqr1/ebtQYLto1v7Yf5 B0aRj6t2VZvLOtIocsPZ9LT/JtKevOLrHHgYh+/N/s5VgD1OlahXCH4/FPcahQDixftT sLdQX5l40ZcSHgCzqTxJfeX/bqu8TiLn5jrv8YaqkWHi/Pv1hUK/UCC7ITvmgUD4vTne xV8folcc8FjB6HLbNxdrcLj69KQmM3Sq0Nu951UG1EDT8GN2GYfOVVukG7Et0gdZuk4B p29w== X-Gm-Message-State: AOAM532QFaTjQA/IbmsIM3SC48sQ09wIM5Q9Nal6Y1Z5/SmDSut/yUKu t1KmnI84ksJdZtBzJ8a68vHo0n+s804M X-Google-Smtp-Source: ABdhPJw4miv6yKMoP660KkuAQKoQs6QJrbWUOze6sSCFx/jyqDsnRSbtfSCLwuMOdKieMRJBqFuxCCiKdHzU X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:23a0:2f14:433:e6cb]) (user=qperret job=sendgmr) by 2002:a0c:fbd1:: with SMTP id n17mr17016888qvp.19.1627291764500; Mon, 26 Jul 2021 02:29:24 -0700 (PDT) Date: Mon, 26 Jul 2021 10:28:55 +0100 In-Reply-To: <20210726092905.2198501-1-qperret@google.com> Message-Id: <20210726092905.2198501-7-qperret@google.com> Mime-Version: 1.0 References: <20210726092905.2198501-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.432.gabb21c7263-goog Subject: [PATCH v2 06/16] KVM: arm64: Don't overwrite software bits with owner id From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, Quentin Perret X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210726_022926_816139_869DE9F8 X-CRM114-Status: GOOD ( 14.06 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org We will soon start annotating page-tables with new flags to track shared pages and such, and we will do so in valid mappings using software bits in the PTEs, as provided by the architecture. However, it is possible that we will need to use those flags to annotate invalid mappings as well in the future, similar to what we do to track page ownership in the host stage-2. In order to facilitate the annotation of invalid mappings with such flags, it would be preferable to re-use the same bits as for valid mappings (bits [58-55]), but these are currently used for ownership encoding. Since we have plenty of bits left to use in invalid mappings, move the ownership bits further down the PTE to avoid the conflict. Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/pgtable.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index 516c1b8ce6b8..b5ca21b44b6a 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -49,7 +49,7 @@ KVM_PTE_LEAF_ATTR_LO_S2_S2AP_W | \ KVM_PTE_LEAF_ATTR_HI_S2_XN) -#define KVM_INVALID_PTE_OWNER_MASK GENMASK(63, 56) +#define KVM_INVALID_PTE_OWNER_MASK GENMASK(9, 2) #define KVM_MAX_OWNER_ID 1 struct kvm_pgtable_walk_data { From patchwork Mon Jul 26 09:28:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12398937 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 16072C4338F for ; Mon, 26 Jul 2021 09:33:58 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D0F9E60F02 for ; Mon, 26 Jul 2021 09:33:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org D0F9E60F02 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=0BpDw4TZoRjOvCshN2+iKfbvsvqVU5NpxJXPWuuZH5w=; b=iO1WJeejCLVxVjBHp9W8TmtmT3 +IKVAPGUpuYPSG+KFxnz3rryRpp5qEmGwkMWPNNAp3nb8pLSgqUXymJEU3hPvUDT4vq859g3LuReD HCw+OUGnJkFCwcZ72fTtOhfzzLNKBjfaCZeY8tVI9pvxLPDkiQ8xDczK22dnBVuOCvyAZrA1cJLJJ KuhoA3WGB6iFpGjiZeye04B3BqD952YWySQv3zV698xfOeGSREkmhrL4m0/H2ODGisZQUGGuIl+zZ 3qd9Q+UZsWuW+648cFzGt5+Q3aqi2F/tTgQssF+dOaPPkqANhMBrPMM0IP+pRHiMerH1r9bzDKi7o g04phnMA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m7wxw-00ASeN-6J; Mon, 26 Jul 2021 09:32:04 +0000 Received: from mail-qv1-xf4a.google.com ([2607:f8b0:4864:20::f4a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m7wvR-00ARYo-41 for linux-arm-kernel@lists.infradead.org; Mon, 26 Jul 2021 09:29:30 +0000 Received: by mail-qv1-xf4a.google.com with SMTP id js9-20020a0562142aa9b0290310fecb5f78so7949867qvb.0 for ; Mon, 26 Jul 2021 02:29:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=HFQLFqYnlTo1lpqJ7D2ypgfrC23LiNE+ZmEwXkcYw0A=; b=emynXi8zGe9T2RndgYYOenyBOAtMls9A7ba9xIszaaQomvI/LjgYmTFavbkZgJJOTH jtvzzyteKVqko+G94XCfuANCCNBvhNP6T8mqrsBlRt2dIVhBPJQgOhI/dnyTrfGziGRE GcgWV2cqwrrCNSKHbUGrT0Hl1O/yQGv0tUrhtwxijYozBbsYTGsDQ6n+LTmNJwhn3cg8 WcykCVqR8O4yeSvhqiW3Qloa6Dgf7LRz4x2f1knnPA+hmnUz6ZdGf46FMq0Ziiv4b5kW v56uzZXpFd2basc8actTQOWCM+4oWAr9MsHdmrPOv0tj9Oqw6eVBoJowa+i4YfgCPyWT R30w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=HFQLFqYnlTo1lpqJ7D2ypgfrC23LiNE+ZmEwXkcYw0A=; b=ttQtTdCkA84OP/aFBz+g3nCv7PiHIwvS0GdXRy288L9FdvNMsa2XK3D7O5XMv3hbCc 11TVH9JgT/3NcpP87Q9DSemTNOHv8hmZs9KR61+PHB78UBh/bOd9VCNF5mvGHUW8jEkz t3qcTn12lN4EDVmtNyu1661Nu/zBzewsEzH4X03kAQ2JSJnNLZ+TMQmhwtxPm/yN5V2J KCL8pQrt3e4OfIgw7hNjRgyihJ3RAh46m/JRZEocS0ar7+DAYmckVVUQCoYeheiy7meU SF69pWsFi5x97YIa4ozDBE64CWlqKkb8L3DP0i0p2x2i5y+cfjoFG2WkNrmBOZ/vxASY vvJQ== X-Gm-Message-State: AOAM531Kv4tqaVDgcmCKYwhR/G3eRqyt1pZqxJh4y4BdOymv8xXC4szk 28WYAUTIYHg+R7wx/9mMjs2XHBl+WCkT X-Google-Smtp-Source: ABdhPJypoUwsNuwCxy8ymHAKoR1zqHYLDOPbF+TzYxseet5jGrMrBQsRY36QeoNA7Ahvpckl3UlgU5nhuRXS X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:23a0:2f14:433:e6cb]) (user=qperret job=sendgmr) by 2002:a05:6214:1021:: with SMTP id k1mr17176909qvr.4.1627291766710; Mon, 26 Jul 2021 02:29:26 -0700 (PDT) Date: Mon, 26 Jul 2021 10:28:56 +0100 In-Reply-To: <20210726092905.2198501-1-qperret@google.com> Message-Id: <20210726092905.2198501-8-qperret@google.com> Mime-Version: 1.0 References: <20210726092905.2198501-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.432.gabb21c7263-goog Subject: [PATCH v2 07/16] KVM: arm64: Tolerate re-creating hyp mappings to set software bits From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, Quentin Perret X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210726_022929_209111_32381655 X-CRM114-Status: GOOD ( 12.61 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The current hypervisor stage-1 mapping code doesn't allow changing an existing valid mapping. Relax this condition by allowing changes that only target software bits, as that will soon be needed to annotate shared pages. Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/pgtable.c | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index b5ca21b44b6a..93cc9de4d46c 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -375,6 +375,21 @@ static int hyp_set_prot_attr(enum kvm_pgtable_prot prot, kvm_pte_t *ptep) return 0; } +static bool hyp_pte_needs_update(kvm_pte_t old, kvm_pte_t new) +{ + /* + * Tolerate KVM recreating the exact same mapping, or changing software + * bits if the existing mapping was valid. + */ + if (old == new) + return false; + + if (!kvm_pte_valid(old)) + return true; + + return !WARN_ON((old ^ new) & ~KVM_PTE_LEAF_ATTR_HI_SW); +} + static bool hyp_map_walker_try_leaf(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, struct hyp_map_data *data) { @@ -384,9 +399,8 @@ static bool hyp_map_walker_try_leaf(u64 addr, u64 end, u32 level, if (!kvm_block_mapping_supported(addr, end, phys, level)) return false; - /* Tolerate KVM recreating the exact same mapping */ new = kvm_init_valid_leaf_pte(phys, data->attr, level); - if (old != new && !WARN_ON(kvm_pte_valid(old))) + if (hyp_pte_needs_update(old, new)) smp_store_release(ptep, new); data->phys += granule; From patchwork Mon Jul 26 09:28:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12398939 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9FF03C4338F for ; Mon, 26 Jul 2021 09:34:10 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6E1CB60238 for ; Mon, 26 Jul 2021 09:34:10 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 6E1CB60238 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=twowt0c8bKo6oerwOxhrduGznKED7NVXnsYB3M5xtv4=; b=ItOGRXKChSZDrdeNPCV6YA9vrd NlGGYMV60L6uAx+5K5Dw8usZHK1RlqyZO+3GrySCbptM2dnMa1bnD5emG+6VIERj1ZAtLpCVDhR7y o4E+ZYaHfcYRn2W2a45GJVQOS3KPsmdbPT4YzJDyxK8hQ7pNedMo05N+L6GuPoHV+9zD/kwxATtSD o28aR1yEpNSMhmK9dk+yZzvD6aiuy5L7iZtu+Tjfp770jFokBis4BtI5/nEIkHYvD13Z6IgnmKbyU nkIY98z1hCsrC4Ju53bEU/BuNy+6yx/4puDrz9hTlyiqOKCZoIEQiuux3Cn4OflI6BK1EyqBoF0jM ugOIRojw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m7wyV-00ASw2-Mb; Mon, 26 Jul 2021 09:32:40 +0000 Received: from mail-qt1-x84a.google.com ([2607:f8b0:4864:20::84a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m7wvS-00ARZS-NU for linux-arm-kernel@lists.infradead.org; Mon, 26 Jul 2021 09:29:32 +0000 Received: by mail-qt1-x84a.google.com with SMTP id l24-20020ac872580000b029024e988e8277so3979178qtp.23 for ; Mon, 26 Jul 2021 02:29:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=SB9+MK0azv5hy9fv+M4EiAat00cfiG99F8cam3Yhacw=; b=oKVyDC7MAciAcBBknu0tes3RbiFfGdIQ18yplm1bFyzcM+XCCalXYxUaAASoPva5Sp z41cgsIzwAt35jZCCJd3thUZeYOic3sVfQhn1yAO2DBQA4FsMPsUFZqaUDtSnr8R6h3/ 54p7by8yBiRsnUmfCtlDrV3wO/YJ7+H/lUfolANOf3mmN28sgrN3qDAYKYfSGJPbgYvZ /SddlODRWQ1yG5nE+CUu6+Y98fq+9kiLKZ0LYeT8Bq4dLoQEWVl3X+lxxUov+pp5uCC/ 3nm8BR7O+24QQDAQek6tJpJ3DxSD5BYQo0FXvLPuP7ngy9PIcuw2t+A+bXefoscaOfG5 mlnA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=SB9+MK0azv5hy9fv+M4EiAat00cfiG99F8cam3Yhacw=; b=rLNEofG0nIyFyR9bY9xxOH/7PotGWh12WR6oe/NPrtiAWyms4tPhhBqWi3U8Ed1RMW vomJXQEqgpcf3cwSbNpNOOdE5/krjwr8U15GSXR5KLQLHQyElDuyz4dL8KYjIMSQTSol /KdAyt2qerudaL+TA3VUZYqHECWR8CDhKYlA8iMGP5fdq/B3UB9X7Yl5ZuGnVVpVJALV IZzHC6B6uKZTGw7tHzB5FaaYA2+E/AK73JrN6c2WoQbE3i9uRzXuvC7KT/YBNVhOjjYX NsssO29/lwmfhqZzillnEMHDR3JhzjtwiShcyncJPDTA5M2WOV/HF4bZmDKAUUfgJsgk EMJw== X-Gm-Message-State: AOAM5334vCtzTRhddJF0gGiOZrfJjXtNOUqL6IeiPiK3B3xLAdaJvw09 vhA5h7sMc7GyR06hi/bytFPavxnc2amO X-Google-Smtp-Source: ABdhPJzf/M1RYcOXENfiih3D5Wtal+/bRaegMDyg/wKSiHnRPB5ywQ09j9wCKxzpCm3N4ZjMrgdqCwa7R4VC X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:23a0:2f14:433:e6cb]) (user=qperret job=sendgmr) by 2002:a0c:c245:: with SMTP id w5mr13907822qvh.21.1627291768922; Mon, 26 Jul 2021 02:29:28 -0700 (PDT) Date: Mon, 26 Jul 2021 10:28:57 +0100 In-Reply-To: <20210726092905.2198501-1-qperret@google.com> Message-Id: <20210726092905.2198501-9-qperret@google.com> Mime-Version: 1.0 References: <20210726092905.2198501-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.432.gabb21c7263-goog Subject: [PATCH v2 08/16] KVM: arm64: Enable forcing page-level stage-2 mappings From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, Quentin Perret X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210726_022930_840126_FBD03182 X-CRM114-Status: GOOD ( 25.88 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Much of the stage-2 manipulation logic relies on being able to destroy block mappings if e.g. installing a smaller mapping in the range. The rationale for this behaviour is that stage-2 mappings can always be re-created lazily. However, this gets more complicated when the stage-2 page-table is used to store metadata about the underlying pages. In such cases, destroying a block mapping may lead to losing part of the state, and confuse the user of those metadata (such as the hypervisor in nVHE protected mode). To avoid this, introduce a callback function in the pgtable struct which is called during all map operations to determine whether the mappings can use blocks, or should be forced to page granularity. This is used by the hypervisor when creating the host stage-2 to force page-level mappings when using non-default protection attributes. Signed-off-by: Quentin Perret --- arch/arm64/include/asm/kvm_pgtable.h | 65 ++++++++++++++++----------- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 30 +++++++++++-- arch/arm64/kvm/hyp/pgtable.c | 29 +++++++++--- 3 files changed, 91 insertions(+), 33 deletions(-) diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h index cec76a49f521..9246a27e2839 100644 --- a/arch/arm64/include/asm/kvm_pgtable.h +++ b/arch/arm64/include/asm/kvm_pgtable.h @@ -102,25 +102,6 @@ enum kvm_pgtable_stage2_flags { KVM_PGTABLE_S2_IDMAP = BIT(1), }; -/** - * struct kvm_pgtable - KVM page-table. - * @ia_bits: Maximum input address size, in bits. - * @start_level: Level at which the page-table walk starts. - * @pgd: Pointer to the first top-level entry of the page-table. - * @mm_ops: Memory management callbacks. - * @mmu: Stage-2 KVM MMU struct. Unused for stage-1 page-tables. - */ -struct kvm_pgtable { - u32 ia_bits; - u32 start_level; - kvm_pte_t *pgd; - struct kvm_pgtable_mm_ops *mm_ops; - - /* Stage-2 only */ - struct kvm_s2_mmu *mmu; - enum kvm_pgtable_stage2_flags flags; -}; - /** * enum kvm_pgtable_prot - Page-table permissions and attributes. * @KVM_PGTABLE_PROT_X: Execute permission. @@ -136,11 +117,41 @@ enum kvm_pgtable_prot { KVM_PGTABLE_PROT_DEVICE = BIT(3), }; -#define PAGE_HYP (KVM_PGTABLE_PROT_R | KVM_PGTABLE_PROT_W) +#define KVM_PGTABLE_PROT_RW (KVM_PGTABLE_PROT_R | KVM_PGTABLE_PROT_W) +#define KVM_PGTABLE_PROT_RWX (KVM_PGTABLE_PROT_RW | KVM_PGTABLE_PROT_X) + +#define PAGE_HYP KVM_PGTABLE_PROT_RW #define PAGE_HYP_EXEC (KVM_PGTABLE_PROT_R | KVM_PGTABLE_PROT_X) #define PAGE_HYP_RO (KVM_PGTABLE_PROT_R) #define PAGE_HYP_DEVICE (PAGE_HYP | KVM_PGTABLE_PROT_DEVICE) +typedef bool (*kvm_pgtable_force_pte_cb_t)(u64 addr, u64 end, + enum kvm_pgtable_prot prot); + +/** + * struct kvm_pgtable - KVM page-table. + * @ia_bits: Maximum input address size, in bits. + * @start_level: Level at which the page-table walk starts. + * @pgd: Pointer to the first top-level entry of the page-table. + * @mm_ops: Memory management callbacks. + * @mmu: Stage-2 KVM MMU struct. Unused for stage-1 page-tables. + * @flags: Stage-2 page-table flags. + * @force_pte_cb: Callback function used during map operations to decide + * whether block mappings can be used to map the given IPA + * range. + */ +struct kvm_pgtable { + u32 ia_bits; + u32 start_level; + kvm_pte_t *pgd; + struct kvm_pgtable_mm_ops *mm_ops; + + /* Stage-2 only */ + struct kvm_s2_mmu *mmu; + enum kvm_pgtable_stage2_flags flags; + kvm_pgtable_force_pte_cb_t force_pte_cb; +}; + /** * enum kvm_pgtable_walk_flags - Flags to control a depth-first page-table walk. * @KVM_PGTABLE_WALK_LEAF: Visit leaf entries, including invalid @@ -233,21 +244,25 @@ int kvm_pgtable_hyp_map(struct kvm_pgtable *pgt, u64 addr, u64 size, u64 phys, u64 kvm_get_vtcr(u64 mmfr0, u64 mmfr1, u32 phys_shift); /** - * kvm_pgtable_stage2_init_flags() - Initialise a guest stage-2 page-table. + * __kvm_pgtable_stage2_init() - Initialise a guest stage-2 page-table. * @pgt: Uninitialised page-table structure to initialise. * @arch: Arch-specific KVM structure representing the guest virtual * machine. * @mm_ops: Memory management callbacks. * @flags: Stage-2 configuration flags. + * @force_pte_cb: Callback function used during map operations to decide + * whether block mappings can be used to map the given IPA + * range. * * Return: 0 on success, negative error code on failure. */ -int kvm_pgtable_stage2_init_flags(struct kvm_pgtable *pgt, struct kvm_arch *arch, - struct kvm_pgtable_mm_ops *mm_ops, - enum kvm_pgtable_stage2_flags flags); +int __kvm_pgtable_stage2_init(struct kvm_pgtable *pgt, struct kvm_arch *arch, + struct kvm_pgtable_mm_ops *mm_ops, + enum kvm_pgtable_stage2_flags flags, + kvm_pgtable_force_pte_cb_t force_pte_cb); #define kvm_pgtable_stage2_init(pgt, arch, mm_ops) \ - kvm_pgtable_stage2_init_flags(pgt, arch, mm_ops, 0) + __kvm_pgtable_stage2_init(pgt, arch, mm_ops, 0, NULL) /** * kvm_pgtable_stage2_destroy() - Destroy an unused guest stage-2 page-table. diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c index 01700a908bb7..2d41d4fa8901 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -89,6 +89,7 @@ static void prepare_host_vtcr(void) id_aa64mmfr1_el1_sys_val, phys_shift); } +static bool host_stage2_force_pte_cb(u64 addr, u64 end, enum kvm_pgtable_prot prot); int kvm_host_prepare_stage2(void *pgt_pool_base) { struct kvm_s2_mmu *mmu = &host_kvm.arch.mmu; @@ -101,8 +102,9 @@ int kvm_host_prepare_stage2(void *pgt_pool_base) if (ret) return ret; - ret = kvm_pgtable_stage2_init_flags(&host_kvm.pgt, &host_kvm.arch, - &host_kvm.mm_ops, KVM_HOST_S2_FLAGS); + ret = __kvm_pgtable_stage2_init(&host_kvm.pgt, &host_kvm.arch, + &host_kvm.mm_ops, KVM_HOST_S2_FLAGS, + host_stage2_force_pte_cb); if (ret) return ret; @@ -264,9 +266,31 @@ static int host_stage2_find_range(u64 addr, struct kvm_mem_range *range) return 0; } +static bool host_stage2_force_pte_cb(u64 addr, u64 end, enum kvm_pgtable_prot prot) +{ + /* + * Block mappings must be used with care in the host stage-2 as a + * kvm_pgtable_stage2_map() operation targeting a page in the range of + * an existing block will delete the block under the assumption that + * mappings in the rest of the block range can always be rebuilt lazily. + * That assumption is correct for the host stage-2 with RWX mappings + * targeting memory or RW mappings targeting MMIO ranges (see + * host_stage2_idmap() below which implements some of the host memory + * abort logic). However, this is not safe for any other mappings where + * the host stage-2 page-table is in fact the only place where this + * state is stored. In all those cases, it is safer to use page-level + * mappings, hence avoiding to lose the state because of side-effects in + * kvm_pgtable_stage2_map(). + */ + if (range_is_memory(addr, end)) + return prot != KVM_PGTABLE_PROT_RWX; + else + return prot != KVM_PGTABLE_PROT_RW; +} + static int host_stage2_idmap(u64 addr) { - enum kvm_pgtable_prot prot = KVM_PGTABLE_PROT_R | KVM_PGTABLE_PROT_W; + enum kvm_pgtable_prot prot = KVM_PGTABLE_PROT_RW; struct kvm_mem_range range; bool is_memory = find_mem_range(addr, &range); int ret; diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index 93cc9de4d46c..e0cd748e4af6 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -465,6 +465,8 @@ int kvm_pgtable_hyp_init(struct kvm_pgtable *pgt, u32 va_bits, pgt->start_level = KVM_PGTABLE_MAX_LEVELS - levels; pgt->mm_ops = mm_ops; pgt->mmu = NULL; + pgt->force_pte_cb = NULL; + return 0; } @@ -502,6 +504,9 @@ struct stage2_map_data { void *memcache; struct kvm_pgtable_mm_ops *mm_ops; + + /* Force mappings to page granularity */ + bool force_pte; }; u64 kvm_get_vtcr(u64 mmfr0, u64 mmfr1, u32 phys_shift) @@ -615,6 +620,15 @@ static bool stage2_pte_executable(kvm_pte_t pte) return !(pte & KVM_PTE_LEAF_ATTR_HI_S2_XN); } +static bool stage2_block_mapping_allowed(u64 addr, u64 end, u32 level, + struct stage2_map_data *data) +{ + if (data->force_pte && (level < (KVM_PGTABLE_MAX_LEVELS - 1))) + return false; + + return kvm_block_mapping_supported(addr, end, data->phys, level); +} + static int stage2_map_walker_try_leaf(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, struct stage2_map_data *data) @@ -624,7 +638,7 @@ static int stage2_map_walker_try_leaf(u64 addr, u64 end, u32 level, struct kvm_pgtable *pgt = data->mmu->pgt; struct kvm_pgtable_mm_ops *mm_ops = data->mm_ops; - if (!kvm_block_mapping_supported(addr, end, phys, level)) + if (!stage2_block_mapping_allowed(addr, end, level, data)) return -E2BIG; if (kvm_phys_is_valid(phys)) @@ -668,7 +682,7 @@ static int stage2_map_walk_table_pre(u64 addr, u64 end, u32 level, if (data->anchor) return 0; - if (!kvm_block_mapping_supported(addr, end, data->phys, level)) + if (!stage2_block_mapping_allowed(addr, end, level, data)) return 0; data->childp = kvm_pte_follow(*ptep, data->mm_ops); @@ -798,6 +812,7 @@ int kvm_pgtable_stage2_map(struct kvm_pgtable *pgt, u64 addr, u64 size, .mmu = pgt->mmu, .memcache = mc, .mm_ops = pgt->mm_ops, + .force_pte = pgt->force_pte_cb && pgt->force_pte_cb(addr, addr + size, prot), }; struct kvm_pgtable_walker walker = { .cb = stage2_map_walker, @@ -829,6 +844,7 @@ int kvm_pgtable_stage2_set_owner(struct kvm_pgtable *pgt, u64 addr, u64 size, .memcache = mc, .mm_ops = pgt->mm_ops, .owner_id = owner_id, + .force_pte = true, }; struct kvm_pgtable_walker walker = { .cb = stage2_map_walker, @@ -1070,9 +1086,11 @@ int kvm_pgtable_stage2_flush(struct kvm_pgtable *pgt, u64 addr, u64 size) return kvm_pgtable_walk(pgt, addr, size, &walker); } -int kvm_pgtable_stage2_init_flags(struct kvm_pgtable *pgt, struct kvm_arch *arch, - struct kvm_pgtable_mm_ops *mm_ops, - enum kvm_pgtable_stage2_flags flags) + +int __kvm_pgtable_stage2_init(struct kvm_pgtable *pgt, struct kvm_arch *arch, + struct kvm_pgtable_mm_ops *mm_ops, + enum kvm_pgtable_stage2_flags flags, + kvm_pgtable_force_pte_cb_t force_pte_cb) { size_t pgd_sz; u64 vtcr = arch->vtcr; @@ -1090,6 +1108,7 @@ int kvm_pgtable_stage2_init_flags(struct kvm_pgtable *pgt, struct kvm_arch *arch pgt->mm_ops = mm_ops; pgt->mmu = &arch->mmu; pgt->flags = flags; + pgt->force_pte_cb = force_pte_cb; /* Ensure zeroed PGD pages are visible to the hardware walker */ dsb(ishst); From patchwork Mon Jul 26 09:28:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12398941 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 89F62C4338F for ; Mon, 26 Jul 2021 09:35:09 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5193C60238 for ; Mon, 26 Jul 2021 09:35:09 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 5193C60238 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=bjYgdbmAPL4MRSHjmn+85k6sv3L+Ko+g9xk9t3bc/U8=; b=vJIUcdu65FcQ11QIuRRLHkElOi damMFcETP+SdG+8JafXVEBuOCdaBA0a5AA5A463LYlOq66pjwqfV9OxUD4QZE8q59gEetheXLVaCa CqziJpgUHP0HGn7Nc3n037oNBQAitn4PbHgJivrRNe9zfHmHfI4lTv7NB0p1lP9Mkm6cogDdrAYbU fbCOhTFxXsjmJGtsobxSRSus15/26cVKuXqlM/3ypoQsE5FyyD8FeFwJ3UpPywvEJjpwvbVZRV8b4 FG7nI2ZjPIa/Nl2KrP3ofJvMuiNJ5zdOfZxLsfAJwR4dZqy3VjySUCf9lcJyxPFTt8H555bgh1+jj E+VE5CfQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m7wz0-00AT9i-Rc; Mon, 26 Jul 2021 09:33:11 +0000 Received: from mail-qk1-x749.google.com ([2607:f8b0:4864:20::749]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m7wvU-00ARZv-Ht for linux-arm-kernel@lists.infradead.org; Mon, 26 Jul 2021 09:29:34 +0000 Received: by mail-qk1-x749.google.com with SMTP id h5-20020a05620a0525b02903b861bec838so8483263qkh.7 for ; Mon, 26 Jul 2021 02:29:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=hjv3q9oqO8KJkTui97+bkuRVMGWdHhFVwbAOefAmuts=; b=K4Np44DCnp9BZz24AKPA6E28uJ8LOJp91dy0Wm/EHbQJI7pAQDa2a8mi+EA/DZXSlk GoG2gmduJ7YZYfatlUidQ9d0VAniN3/JmTbPm71T6b4P77OPCFz9BNUKp8ONmEowp3w/ +EHlEkCKgDkbLFk9y1zDY6RfRxq2WiIl0Og9u8g5HrFr4JTxKUxlrLo4+zEdUifNs6W8 JmhIv0lqvT/uAvDr5GgydXVCEY8FGiCe07r7MYrwAi0db/RDEYrNT00/DpYdaeNPmkQ8 ZLKO/aUnD0MVomB9CV6RjJbp1KN1Bzw9DuQJdFYBlMJBVi2xfvWSfOA8vbkSqq6lKfjK aBFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=hjv3q9oqO8KJkTui97+bkuRVMGWdHhFVwbAOefAmuts=; b=p3YsyM2nOzcMGLaz8TnywjbIBmUw31CbhRfCq9aZOFaOEwrZj1OEcZrml0GPTGH/9Y u/K3xq+TIwfXn4PYf5TrvjnGnC+KY5bs3UWFh6uZA2BW/KedYX6d4XsZvO/hAKeZF9ce 4q/9lZv8KoQdbNskX2teQsZUElxncHdtty+gIoAm4THbKMzm11WyXWIPjbixJNlmhid6 TcQ7Eaeue2/Eps7l6/CHhx1VZ+0UBqI/K9RG+/UrW8+HnfVI7LVtt/tB6JNRS3xQKUqU eRvteqiyQrZtTleESaUgceQWAJ6blv3YLasF5vBoOC7HFTCOc17ZjuGXbVAIjnRHoSMF U4JQ== X-Gm-Message-State: AOAM530cD4id28BhaLoJ1YgkoX9ZJSShNuAsDCi8kfdVOuFLvOV2VZsp B+wSJw4x5mfp9bKgf3F9IMctXOgyH1Vb X-Google-Smtp-Source: ABdhPJzI3c0tpPwx43KTm4Da4gOx40eS0i8088IhwtbH4MD7mLJsbFdIcU3LsaNGXrE1RIGFD4BUqlkCiWJN X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:23a0:2f14:433:e6cb]) (user=qperret job=sendgmr) by 2002:a05:6214:ca5:: with SMTP id s5mr16960791qvs.58.1627291771064; Mon, 26 Jul 2021 02:29:31 -0700 (PDT) Date: Mon, 26 Jul 2021 10:28:58 +0100 In-Reply-To: <20210726092905.2198501-1-qperret@google.com> Message-Id: <20210726092905.2198501-10-qperret@google.com> Mime-Version: 1.0 References: <20210726092905.2198501-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.432.gabb21c7263-goog Subject: [PATCH v2 09/16] KVM: arm64: Allow populating software bits From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, Quentin Perret X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210726_022932_638504_DFC5F23C X-CRM114-Status: GOOD ( 16.26 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Introduce infrastructure allowing to manipulate software bits in stage-1 and stage-2 page-tables using additional entries in the kvm_pgtable_prot enum. This is heavily inspired by Marc's implementation of a similar feature in the NV patch series, but adapted to allow stage-1 changes as well: https://lore.kernel.org/kvmarm/20210510165920.1913477-56-maz@kernel.org/ Suggested-by: Marc Zyngier Signed-off-by: Quentin Perret --- arch/arm64/include/asm/kvm_pgtable.h | 12 +++++++++++- arch/arm64/kvm/hyp/pgtable.c | 5 +++++ 2 files changed, 16 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h index 9246a27e2839..0be9f83974ad 100644 --- a/arch/arm64/include/asm/kvm_pgtable.h +++ b/arch/arm64/include/asm/kvm_pgtable.h @@ -108,6 +108,10 @@ enum kvm_pgtable_stage2_flags { * @KVM_PGTABLE_PROT_W: Write permission. * @KVM_PGTABLE_PROT_R: Read permission. * @KVM_PGTABLE_PROT_DEVICE: Device attributes. + * @KVM_PGTABLE_PROT_SW0: Software bit 0. + * @KVM_PGTABLE_PROT_SW1: Software bit 1. + * @KVM_PGTABLE_PROT_SW2: Software bit 2. + * @KVM_PGTABLE_PROT_SW3: Software bit 3. */ enum kvm_pgtable_prot { KVM_PGTABLE_PROT_X = BIT(0), @@ -115,6 +119,11 @@ enum kvm_pgtable_prot { KVM_PGTABLE_PROT_R = BIT(2), KVM_PGTABLE_PROT_DEVICE = BIT(3), + + KVM_PGTABLE_PROT_SW0 = BIT(55), + KVM_PGTABLE_PROT_SW1 = BIT(56), + KVM_PGTABLE_PROT_SW2 = BIT(57), + KVM_PGTABLE_PROT_SW3 = BIT(58), }; #define KVM_PGTABLE_PROT_RW (KVM_PGTABLE_PROT_R | KVM_PGTABLE_PROT_W) @@ -406,7 +415,8 @@ kvm_pte_t kvm_pgtable_stage2_mkold(struct kvm_pgtable *pgt, u64 addr); * If there is a valid, leaf page-table entry used to translate @addr, then * relax the permissions in that entry according to the read, write and * execute permissions specified by @prot. No permissions are removed, and - * TLB invalidation is performed after updating the entry. + * TLB invalidation is performed after updating the entry. Software bits cannot + * be set or cleared using kvm_pgtable_stage2_relax_perms(). * * Return: 0 on success, negative error code on failure. */ diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index e0cd748e4af6..bd409d524dea 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -370,6 +370,7 @@ static int hyp_set_prot_attr(enum kvm_pgtable_prot prot, kvm_pte_t *ptep) attr |= FIELD_PREP(KVM_PTE_LEAF_ATTR_LO_S1_AP, ap); attr |= FIELD_PREP(KVM_PTE_LEAF_ATTR_LO_S1_SH, sh); attr |= KVM_PTE_LEAF_ATTR_LO_S1_AF; + attr |= prot & KVM_PTE_LEAF_ATTR_HI_SW; *ptep = attr; return 0; @@ -571,6 +572,7 @@ static int stage2_set_prot_attr(struct kvm_pgtable *pgt, enum kvm_pgtable_prot p attr |= FIELD_PREP(KVM_PTE_LEAF_ATTR_LO_S2_SH, sh); attr |= KVM_PTE_LEAF_ATTR_LO_S2_AF; + attr |= prot & KVM_PTE_LEAF_ATTR_HI_SW; *ptep = attr; return 0; @@ -1038,6 +1040,9 @@ int kvm_pgtable_stage2_relax_perms(struct kvm_pgtable *pgt, u64 addr, u32 level; kvm_pte_t set = 0, clr = 0; + if (prot & KVM_PTE_LEAF_ATTR_HI_SW) + return -EINVAL; + if (prot & KVM_PGTABLE_PROT_R) set |= KVM_PTE_LEAF_ATTR_LO_S2_S2AP_R; From patchwork Mon Jul 26 09:28:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12398943 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 40A31C4338F for ; Mon, 26 Jul 2021 09:35:50 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0B3A360E09 for ; Mon, 26 Jul 2021 09:35:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 0B3A360E09 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=j39KdAxmArPZAmhY7kGlaOo7EWdX2DPzIjH+ekOsCPs=; b=bpPrWN3ay7BGzz5BSmfqzcRFxv xfCVsSDUYJZH9eZCMX+4pk5SQHMOKhv5bksXt93vsEfh71gTLgCe3/0twAQ4NWL6hhvhLqr85HPpC DSeMwY6Jbc8BFre8Kq+U+rgNU96pgBm3FNLlndUIdQbf2x7+dxfLfE9YatqU3tbiQ1vuYXNl7X+q2 zvWh9X+YSSno8sskuu1LF/PNfTnV/s+WT1KPXWQDNPU+XaqHYzJSCJSvOj3vvuFFgIDBy9DreQwtm 23jVkQO8u5zBCvqO2tqcH+J8u+SCQkc4eTEHuwCPQbXojeWLv/gq1dD3DXV0BiRNkKR08olSxNi7V axGl/Kyw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m7wzX-00ATMt-I6; Mon, 26 Jul 2021 09:33:43 +0000 Received: from mail-qt1-x849.google.com ([2607:f8b0:4864:20::849]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m7wvX-00ARaH-Fh for linux-arm-kernel@lists.infradead.org; Mon, 26 Jul 2021 09:29:36 +0000 Received: by mail-qt1-x849.google.com with SMTP id e19-20020ac84b530000b0290251f0b91196so4018590qts.14 for ; Mon, 26 Jul 2021 02:29:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=oI1lsqd2Q7dxLS0xam0vuO1J8Xxzsd51ZvtsfH3kSqA=; b=bXUXb/i9hSD5qN+TC+k+TwbawB1Ioga3ua3ibJeKI8gHVrRfQF1zVfzZ9qUKUJWIZb D3xLOECrzB08csaTg6TGm2x6MiN2M0qkKt7aUpIzWlvypa7VgYw7HoaS8o9nwDLUc3Nt H/I3JOAjQfDHzXOk2iBvFyJl4Ofl/yegYHjGNrZWBdXPhk1D876uuHch99124AS6qus7 EeemZ0cg6jHhbWX/57lWGRwafDUed/QpP+m0HQ1U2ANbhgXhsZwGerOpdJs9Df1RiqXo uEtAXLQXFJZUQjutSGSIyRCOHeo6U82U5bsQSPmEcXs8Wfx1gjaZbyNisbC3A+TAs1WN NoOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=oI1lsqd2Q7dxLS0xam0vuO1J8Xxzsd51ZvtsfH3kSqA=; b=WGDGIFpWVuxLPMNylewAMdcy2UiD2/5EsxRMV0VINbL9UfUwFkKgA5XlS+VGPR5k7D /d99Rl/dYQaIlEdS6Jdhf6Otd4lrGeGaC8pXASTSct/XB1CHY3eDNWO16MnjrUEgI5FR KM6D4+Sr2qc0WmXQT017XbZYqVaWQVIpWeDlESd9XKeChM2U6Q5lzHHK10skXnUWWERS d4abeLS4xUB7EZKbtbv7GPC1ghK6TOoux/JYKVMmvr4qyKCWVtCNPr+uoOaM6XNKGEYC ZdEvBz9jKkBbLPXuBrqLQTD1SwMUadnPWWhrbiXWqG+79Wpwc3arhAQGsEaWbqQckd1/ F3Og== X-Gm-Message-State: AOAM532MF2fJqGqh+uh7b5QufFmN1ocDDeyaAp5BFGVy75IO9SR3Wm0T ffIIImm22mhzfSW4RB9tJftNdFuxrgVB X-Google-Smtp-Source: ABdhPJyCiJBcW6GLm8E7sXXL2J+ym67KXeRU47WQn21gBL8HZOPCdK1XRTVEI/TyzJ/m4gVul31p0IuTxjtT X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:23a0:2f14:433:e6cb]) (user=qperret job=sendgmr) by 2002:a0c:aacd:: with SMTP id g13mr2247331qvb.20.1627291773337; Mon, 26 Jul 2021 02:29:33 -0700 (PDT) Date: Mon, 26 Jul 2021 10:28:59 +0100 In-Reply-To: <20210726092905.2198501-1-qperret@google.com> Message-Id: <20210726092905.2198501-11-qperret@google.com> Mime-Version: 1.0 References: <20210726092905.2198501-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.432.gabb21c7263-goog Subject: [PATCH v2 10/16] KVM: arm64: Add helpers to tag shared pages in SW bits From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, Quentin Perret X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210726_022935_563934_A22B8AAE X-CRM114-Status: GOOD ( 12.72 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org We will soon start annotating shared pages in page-tables in nVHE protected mode. Define all the states in which a page can be (owned, shared and owned, shared and borrowed), and provide helpers allowing to convert this into SW bits annotations using the matching prot attributes. Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/include/nvhe/mem_protect.h | 27 +++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h index 9c227d87c36d..facbd9a7ab99 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h +++ b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h @@ -12,6 +12,33 @@ #include #include +/* + * SW bits 0-1 are reserved to track the memory ownership state of each page: + * 00: The page is owned solely by the page-table owner. + * 01: The page is owned by the page-table owner, but is shared + * with another entity. + * 10: The page is shared with, but not owned by the page-table owner. + * 11: Reserved for future use (lending). + */ +enum pkvm_page_state +{ + PKVM_PAGE_OWNED = 0ULL, + PKVM_PAGE_SHARED_OWNED = KVM_PGTABLE_PROT_SW0, + PKVM_PAGE_SHARED_BORROWED = KVM_PGTABLE_PROT_SW1, +}; + +#define PKVM_PAGE_STATE_PROT_MASK (KVM_PGTABLE_PROT_SW0 | KVM_PGTABLE_PROT_SW1) +static inline enum kvm_pgtable_prot pkvm_mkstate(enum kvm_pgtable_prot prot, + enum pkvm_page_state state) +{ + return (prot & ~PKVM_PAGE_STATE_PROT_MASK) | state; +} + +static inline enum pkvm_page_state pkvm_getstate(enum kvm_pgtable_prot prot) +{ + return prot & PKVM_PAGE_STATE_PROT_MASK; +} + struct host_kvm { struct kvm_arch arch; struct kvm_pgtable pgt; From patchwork Mon Jul 26 09:29:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12398947 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 18E18C4338F for ; Mon, 26 Jul 2021 09:37:57 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D110060E09 for ; Mon, 26 Jul 2021 09:37:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org D110060E09 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=2cEDZsns+vkqff2TKQJzfgsESppunvZlQEkCs99Evvc=; b=hsxj/1MREbwnuB1JIrKJbs1cwX fb/siKmYQ3RXuo6bDRUl1VB3tJPJt0MELgzT18jRvEQ+h2ocJJJLV+9Eb8yPoB2QcdDKXA2lVIm1a UB4uwplCO6TKQvo2yKWQLtwrO765QElBv7L5JI3fieccSipMIDkIeQKCX9cUHGBgu/aauhaIW1mb8 XZwGQC5RNAeO/XnP4hs5N6C3Gf5sNla6PMbJmwP9mpuv5ev5PB3Az5agynqtzJDSwD0sNxxDQ39li aVLxQTkWOpnbUVZWNAp+6H8k/BdNCK4R3ex9m48C7InuXMJO4IacKEJSeI1vC+4gWhAZECVYmbgvP FohQF5+g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m7x1a-00AUFr-FY; Mon, 26 Jul 2021 09:35:51 +0000 Received: from mail-wr1-x44a.google.com ([2a00:1450:4864:20::44a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m7wvc-00ARb7-Mv for linux-arm-kernel@lists.infradead.org; Mon, 26 Jul 2021 09:29:42 +0000 Received: by mail-wr1-x44a.google.com with SMTP id o11-20020a5d474b0000b02901533f8ed22cso3019222wrs.22 for ; Mon, 26 Jul 2021 02:29:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=fyCHOW6maJF9L7guSoQxIXFDKYajZ5tEfL0M1nzcu3g=; b=b6cEmvJ0vdEy2FGe8oxQq0HlmnKC08AOEIcGg2mj6fy+SSGskiuESRrYptCgJBbUZu mt0BDG+nEREOBOG2gON7kzVvLlgj83N/3hY4W0c6StPuMfpFjuKVeuGnNBubwob1lg/y noakRpnUfRpGOAidTPg/rki9g0KqzEwYypxs4M7HTnvPXf7iFxu/B6GlMaKalYBVBH14 YkQii/9BBIPj6M2rqhx9TIywxni7ift1Cnpcz6Om6njwYOV6G8R2RL+mltGn89rqQd3x lPyubLT/kp4B80c1Rt5WC3DkG7lsQlG7ga2RipuW01KExo0V3PSpKTM1DAMcXKZlw0ft 7ibg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=fyCHOW6maJF9L7guSoQxIXFDKYajZ5tEfL0M1nzcu3g=; b=smDbPjiVih5E8di/UEVSyPAGQQVdEmzHtLVZxQ889BPVSzmv9cNWN4k8B2JqQvtKxK 4pDmR+jcQpxXE8ZASWi/K56Z58iPwSaMGlhdmMIpG/pxerclQbp9v3FBlI5cyJv369Un zFb/yasj7xYHCk10nakcS10tncVmRSd0y/HwmzcggTrN31kH5bQtY/ZG0GvFDzjGTPbW EsW5xW/LvBs0zgrm5ZJPB2ZKrOBeZCIkukap0AXugvwzawvX/n3sTjJZL5ieYByheTvL +Ztu31G9mxiuh9RWBA3GCxY5PRDUjzOmsED3QtyVSAiSL8dj1XwSdndKAlg9SbrNwCzL mCdA== X-Gm-Message-State: AOAM531Tv1RKVHKK5gU2FAk18YiP0ss6STmNgmSLRWHfXqKkDwUHPg8H 3MJnfQuXzoGbaK8mKRIX/C7LcdIZdzP7 X-Google-Smtp-Source: ABdhPJwpwF4hCeEcD8WP2F52FmGpoCMVmSoE2R/VRCPmMc5vvMGS4yzbAZ1WyJM7j6+dONr435cj4ZE0ccVT X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:23a0:2f14:433:e6cb]) (user=qperret job=sendgmr) by 2002:a7b:cb4d:: with SMTP id v13mr14985934wmj.68.1627291775721; Mon, 26 Jul 2021 02:29:35 -0700 (PDT) Date: Mon, 26 Jul 2021 10:29:00 +0100 In-Reply-To: <20210726092905.2198501-1-qperret@google.com> Message-Id: <20210726092905.2198501-12-qperret@google.com> Mime-Version: 1.0 References: <20210726092905.2198501-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.432.gabb21c7263-goog Subject: [PATCH v2 11/16] KVM: arm64: Introduce and export host_stage2_idmap_locked() From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, Quentin Perret X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210726_022940_776383_AD4E712F X-CRM114-Status: GOOD ( 12.93 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org We will need to manipulate the host stage-2 page-table from outside mem_protect.c soon. Introduce a function wrapping the host_stage2_try() call and make it usable to users of mem_protect.h. Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/include/nvhe/mem_protect.h | 1 + arch/arm64/kvm/hyp/nvhe/mem_protect.c | 7 ++++++- 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h index facbd9a7ab99..8e5725d032b2 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h +++ b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h @@ -50,6 +50,7 @@ extern struct host_kvm host_kvm; int __pkvm_prot_finalize(void); int __pkvm_mark_hyp(phys_addr_t start, phys_addr_t end); +int host_stage2_idmap_locked(u64 start, u64 end, enum kvm_pgtable_prot prot); int kvm_host_prepare_stage2(void *pgt_pool_base); void handle_host_mem_abort(struct kvm_cpu_context *host_ctxt); diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c index 2d41d4fa8901..223c541a7051 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -266,6 +266,11 @@ static int host_stage2_find_range(u64 addr, struct kvm_mem_range *range) return 0; } +int host_stage2_idmap_locked(u64 start, u64 end, enum kvm_pgtable_prot prot) +{ + return host_stage2_try(__host_stage2_idmap, start, end, prot); +} + static bool host_stage2_force_pte_cb(u64 addr, u64 end, enum kvm_pgtable_prot prot) { /* @@ -303,7 +308,7 @@ static int host_stage2_idmap(u64 addr) if (ret) goto unlock; - ret = host_stage2_try(__host_stage2_idmap, range.start, range.end, prot); + ret = host_stage2_idmap_locked(range.start, range.end, prot); unlock: hyp_spin_unlock(&host_kvm.lock); From patchwork Mon Jul 26 09:29:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12398945 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C06B1C4338F for ; Mon, 26 Jul 2021 09:36:49 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 916A360F02 for ; Mon, 26 Jul 2021 09:36:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 916A360F02 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=JpXq8ncGoKBZTDViNKexFXTvhUSs1SAB+Z08wh9TA+4=; b=eObCemuuJlfOi06UPsLB/6C3NV j3kv28t0g1XN8KJ/GcfKI3Pr+YwHR/+yvlyWIUj24rlDLeIgqarJNSno/rnlnZqHtLO+W3jK9Gtdk CNb0zH46Dj9v/llrlrjCdOC31RqXtVrIJOoCzmkZCAtoN5ojRLK19wHEdOoh1Jz5dNyMSWQrmBejX KVHD6lbR4rjTUSnbiaX768dJ6THwoW8cbmjE1voiwPxXqH/NFR/WYqLSbUAp/FSBC5QKeZDcc2RHV I5GDccEjw6RE0p6/x+0X2Vy9e9y6BeHyrejEVuG06bowfa2/Df7zYhiXEnPf//dme1UyxKoiZOpvM S7CACjiQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m7x0d-00ATod-C5; Mon, 26 Jul 2021 09:34:51 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m7wvc-00ARb9-CA for linux-arm-kernel@lists.infradead.org; Mon, 26 Jul 2021 09:29:42 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id s123-20020a2577810000b02904f84a5c5297so12928397ybc.16 for ; Mon, 26 Jul 2021 02:29:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=MJIro2WTfm4GzLtRoDylovuZ45jqLgFOexzn+Zy4wjs=; b=FSUlFamdvZmCkq1K/PPVAVpzWQ2IdtDwEy8DAyitecCP9UoJnIxQmUsQfqZUzVuPEk Hn6Y0ZdCypMyVxkddka1PFuUUmL82O98WDLQQxOW9w5/1xNezpgSU3+VjjTAM3QiiFYb lc+Oyf1eR0aufgHOQtp6oaewCjRopIvo2NIIYIzU038nPMU79T0GnRIAM620SA+Uowrp PsNp374AxcYuwdiWR5/zympTvFOzT+i4A8R/bc8CeqCmKLxZ6wYASldpJ9oV5r2GkoYK WtkZCG4iM7dS0bKA4FDvw5WqKlxqUe23jdLdajTqkFZbRzI4KaYW5x8/tmwqWByjNRsS KBAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=MJIro2WTfm4GzLtRoDylovuZ45jqLgFOexzn+Zy4wjs=; b=KxR5hhYXVfd3Lx7Q87jrM1SZq7v6m54NFNAVb+4pIn3rw2Mm1SDqqskMvOQRFreShB CU4+p0FYhGn1K+NqT56JMk3zt7XfKS+rq7Gx+DNi/ihJlBbAZjcqVimvkm4Wick2CUk3 iao1dy9tUM/BirpQaebLex0k9jSq8+BqZBre93JsivuOaf7bw/+TkqWAya/CuwfDnVT8 eXLpGw9t5flfjBSdS6bRIbGKtxvnL6GvezUB8S3UiFHc4OMOLfPeNTFwUGp1u3RaUrCD lIFZc2ckzP7rbSSPgnvPvj9d+iwQoxm6XhFW03PqHzYu6hVv2Xd/iaVJwGIgUv+ihKD5 8O3A== X-Gm-Message-State: AOAM532H79wMDgKMlFTEmz+Th/1HyR+hNTF0PkaBJAUpggoosK4dDfdw 03q+OdvPU15sU8z19gVueiFolHmsPyq+ X-Google-Smtp-Source: ABdhPJzFtCNc3wRUZQkuMVZikxDARzQyAhA2u+pAgC1/vs0Dkse5PCCu9I9np2lmsIsb+H72joir6vU8V2M2 X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:23a0:2f14:433:e6cb]) (user=qperret job=sendgmr) by 2002:a25:6088:: with SMTP id u130mr23486646ybb.257.1627291778107; Mon, 26 Jul 2021 02:29:38 -0700 (PDT) Date: Mon, 26 Jul 2021 10:29:01 +0100 In-Reply-To: <20210726092905.2198501-1-qperret@google.com> Message-Id: <20210726092905.2198501-13-qperret@google.com> Mime-Version: 1.0 References: <20210726092905.2198501-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.432.gabb21c7263-goog Subject: [PATCH v2 12/16] KVM: arm64: Mark host bss and rodata section as shared From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, Quentin Perret X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210726_022940_515908_E0B5D02B X-CRM114-Status: GOOD ( 20.11 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org As the hypervisor maps the host's .bss and .rodata sections in its stage-1, make sure to tag them as shared in hyp and host page-tables. But since the hypervisor relies on the presence of these mappings, we cannot let the host in complete control of the memory regions -- it must not unshare or donate them to another entity for example. To prevent this, let's transfer the ownership of those ranges to the hypervisor itself, and share the pages back with the host. Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/nvhe/setup.c | 52 ++++++++++++++++++++++++++++----- 1 file changed, 44 insertions(+), 8 deletions(-) diff --git a/arch/arm64/kvm/hyp/nvhe/setup.c b/arch/arm64/kvm/hyp/nvhe/setup.c index 0b574d106519..285c8aea5065 100644 --- a/arch/arm64/kvm/hyp/nvhe/setup.c +++ b/arch/arm64/kvm/hyp/nvhe/setup.c @@ -58,6 +58,7 @@ static int recreate_hyp_mappings(phys_addr_t phys, unsigned long size, { void *start, *end, *virt = hyp_phys_to_virt(phys); unsigned long pgt_size = hyp_s1_pgtable_pages() << PAGE_SHIFT; + enum kvm_pgtable_prot prot; int ret, i; /* Recreate the hyp page-table using the early page allocator */ @@ -83,10 +84,6 @@ static int recreate_hyp_mappings(phys_addr_t phys, unsigned long size, if (ret) return ret; - ret = pkvm_create_mappings(__start_rodata, __end_rodata, PAGE_HYP_RO); - if (ret) - return ret; - ret = pkvm_create_mappings(__hyp_rodata_start, __hyp_rodata_end, PAGE_HYP_RO); if (ret) return ret; @@ -95,10 +92,6 @@ static int recreate_hyp_mappings(phys_addr_t phys, unsigned long size, if (ret) return ret; - ret = pkvm_create_mappings(__hyp_bss_end, __bss_stop, PAGE_HYP_RO); - if (ret) - return ret; - ret = pkvm_create_mappings(virt, virt + size, PAGE_HYP); if (ret) return ret; @@ -117,6 +110,24 @@ static int recreate_hyp_mappings(phys_addr_t phys, unsigned long size, return ret; } + /* + * Map the host's .bss and .rodata sections RO in the hypervisor, but + * transfer the ownerhsip from the host to the hypervisor itself to + * make sure it can't be donated or shared with another entity. + * + * The ownership transtion requires matching changes in the host + * stage-2. This will done later (see finalize_mappings()) once the + * hyp_vmemmap is addressable. + */ + prot = pkvm_mkstate(PAGE_HYP_RO, PKVM_PAGE_SHARED_OWNED); + ret = pkvm_create_mappings(__start_rodata, __end_rodata, prot); + if (ret) + return ret; + + ret = pkvm_create_mappings(__hyp_bss_end, __bss_stop, prot); + if (ret) + return ret; + return 0; } @@ -148,6 +159,27 @@ static void hpool_put_page(void *addr) hyp_put_page(&hpool, addr); } +static int finalize_mappings(void) +{ + enum kvm_pgtable_prot prot; + int ret; + + /* + * The host's .bss and .rodata sections are now conceptually owned by + * the hypervisor, so mark them as 'borrowed' in the host stage-2. We + * can safely use host_stage2_idmap_locked() at this point since the + * host stage-2 has not been enabled yet. + */ + prot = pkvm_mkstate(KVM_PGTABLE_PROT_RWX, PKVM_PAGE_SHARED_BORROWED); + ret = host_stage2_idmap_locked(__hyp_pa(__start_rodata), + __hyp_pa(__end_rodata), prot); + if (ret) + return ret; + + return host_stage2_idmap_locked(__hyp_pa(__hyp_bss_end), + __hyp_pa(__bss_stop), prot); +} + void __noreturn __pkvm_init_finalise(void) { struct kvm_host_data *host_data = this_cpu_ptr(&kvm_host_data); @@ -167,6 +199,10 @@ void __noreturn __pkvm_init_finalise(void) if (ret) goto out; + ret = finalize_mappings(); + if (ret) + goto out; + pkvm_pgtable_mm_ops = (struct kvm_pgtable_mm_ops) { .zalloc_page = hyp_zalloc_hyp_page, .phys_to_virt = hyp_phys_to_virt, From patchwork Mon Jul 26 09:29:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12398949 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 263FFC4338F for ; Mon, 26 Jul 2021 09:38:55 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E876A60238 for ; Mon, 26 Jul 2021 09:38:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org E876A60238 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=2dlWOadZeFS3OtUuQbRKBxXJZagUdWxpJ+Sc15V9AXI=; b=LEPYD9oppubmuzBNbJHF4Jza8X 9zdDLJ/96vSHB0TCe+zXrUgbm3N4rw6fmp3bKO9nI2tMdgtInTBcsGn329N9dcwwVzn8Q1wn5mGCf ECzRiijmf9XILBufuJjraLVmfB15yYQ1evN5OsW5251I70Uj32wpqfymZEk87ODLl1vXuEqn9nh+I lJUv4OcfQZYR31N18cA2AV+KOIJpEOflGbfjB8b1Lfw5qvfDj3VeOTxMe2PBiN1fsK3Zig8biUuuB Qxst4a9VKJOX7/+vT0zYJeWRirId/mZqmDHJxBPlRt3xmj+eHpyVM6xIKkLYF0SLxQkx15M0EEqGt Txf7tTTA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m7x2b-00AUi9-6u; Mon, 26 Jul 2021 09:36:54 +0000 Received: from mail-qk1-x74a.google.com ([2607:f8b0:4864:20::74a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m7wvd-00ARbt-P8 for linux-arm-kernel@lists.infradead.org; Mon, 26 Jul 2021 09:29:43 +0000 Received: by mail-qk1-x74a.google.com with SMTP id k5-20020a05620a4145b02903b8eff05707so8510054qko.5 for ; Mon, 26 Jul 2021 02:29:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=tkKuELagekwMv5lJ5KC16O2ECB7SNyEmk6kFSF/fHxM=; b=nk3MJc1TvlrOLzzIkiMf4rqf5hlIGKplrBy0rbLmw27uz1W6VBHL3ebA0vXPU5hqe/ QoJ1dpPFJWYNA881TkoFyPstgtGjtstMfCXCBrv86MDx6OrlwzOGtjIAWJB+y8Ts/OF/ uzElj+Fi5baAI3+Gd7/gAYFnufyRr/kcef4dno05tPLzPDrZFvZW5Zw4E6oA8mNnAsqz kmK+rxP55y6Q7IRGVH0DsjQTrpuBlDwMkGE8Q0sdJ2BOb5c9+MDQuNfFTrhqhDtnWWJT 8Dkhb+jqs5SFv774P+xFvG/36uFaxoBLVLa+7H3VSPMzcE0DjbMAeNFWU0F9g6VrRrc8 SSJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=tkKuELagekwMv5lJ5KC16O2ECB7SNyEmk6kFSF/fHxM=; b=bV/GFMPM/hs+KPBDOo+kSi8pv59WiBz6jmzoo/VnN4AU0tMGD8RPFX8ypsbbm1sUiU kkw4R+3G0HZU1yKwq+jvwnOjHT5hOayTfKjZs6EvGuzKIf6mSw6TI7TGOzl+VxhXDs/0 DQE2YO702CED9ul7osogpmYdIy34zhch0JgLrWUeMT/PKtYpZHrdD19v5DJMYC6A1o6n VSCi/nlcVe8UzGXp85EGpF84XZJVG2f0mvpyHxAkclKmuYq1YKFp/ghW3DkHO5Or4NB0 FMyFUYKCRu0HVvMz44XtM66aDuenAG+GnK/awYQSPTV6azdTDTEwgDIyMRrx5Z4eKCb/ putg== X-Gm-Message-State: AOAM5339zNC2Es3Pa9FW36cOq0fCb2+e3OYMzj3k/7ei15uFZR+yIHQ4 vArXmOTdkUZUEiBx9MtMJj5TvbzVJis8 X-Google-Smtp-Source: ABdhPJw/Ly763xE6/tFkoek1tqopFddG61YOXzM273sCNQ1M4TAjeeg9umQGIef8D0FMW2bam9q3NSXPHFid X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:23a0:2f14:433:e6cb]) (user=qperret job=sendgmr) by 2002:a05:6214:188c:: with SMTP id cx12mr16903787qvb.7.1627291780379; Mon, 26 Jul 2021 02:29:40 -0700 (PDT) Date: Mon, 26 Jul 2021 10:29:02 +0100 In-Reply-To: <20210726092905.2198501-1-qperret@google.com> Message-Id: <20210726092905.2198501-14-qperret@google.com> Mime-Version: 1.0 References: <20210726092905.2198501-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.432.gabb21c7263-goog Subject: [PATCH v2 13/16] KVM: arm64: Enable retrieving protections attributes of PTEs From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, Quentin Perret X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210726_022941_866468_259E0999 X-CRM114-Status: GOOD ( 13.10 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Introduce helper functions in the KVM stage-2 and stage-1 page-table manipulation library allowing to retrieve the enum kvm_pgtable_prot of a PTE. This will be useful to implement custom walkers outside of pgtable.c. Signed-off-by: Quentin Perret --- arch/arm64/include/asm/kvm_pgtable.h | 20 +++++++++++++++ arch/arm64/kvm/hyp/pgtable.c | 37 ++++++++++++++++++++++++++++ 2 files changed, 57 insertions(+) diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h index 0be9f83974ad..f253fb5bcee2 100644 --- a/arch/arm64/include/asm/kvm_pgtable.h +++ b/arch/arm64/include/asm/kvm_pgtable.h @@ -492,4 +492,24 @@ int kvm_pgtable_walk(struct kvm_pgtable *pgt, u64 addr, u64 size, */ int kvm_pgtable_get_leaf(struct kvm_pgtable *pgt, u64 addr, kvm_pte_t *ptep, u32 *level); + +/** + * kvm_pgtable_stage2_pte_prot() - Retrieve the protection attributes of a + * stage-2 Page-Table Entry. + * @pte: Page-table entry + * + * Return: protection attributes of the page-table entry in the enum + * kvm_pgtable_prot format. + */ +enum kvm_pgtable_prot kvm_pgtable_stage2_pte_prot(kvm_pte_t pte); + +/** + * kvm_pgtable_hyp_pte_prot() - Retrieve the protection attributes of a stage-1 + * Page-Table Entry. + * @pte: Page-table entry + * + * Return: protection attributes of the page-table entry in the enum + * kvm_pgtable_prot format. + */ +enum kvm_pgtable_prot kvm_pgtable_hyp_pte_prot(kvm_pte_t pte); #endif /* __ARM64_KVM_PGTABLE_H__ */ diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index bd409d524dea..fb80d3019cff 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -376,6 +376,26 @@ static int hyp_set_prot_attr(enum kvm_pgtable_prot prot, kvm_pte_t *ptep) return 0; } +enum kvm_pgtable_prot kvm_pgtable_hyp_pte_prot(kvm_pte_t pte) +{ + enum kvm_pgtable_prot prot = pte & KVM_PTE_LEAF_ATTR_HI_SW; + u32 ap; + + if (!kvm_pte_valid(pte)) + return prot; + + if (!(pte & KVM_PTE_LEAF_ATTR_HI_S1_XN)) + prot |= KVM_PGTABLE_PROT_X; + + ap = FIELD_GET(KVM_PTE_LEAF_ATTR_LO_S1_AP, pte); + if (ap == KVM_PTE_LEAF_ATTR_LO_S1_AP_RO) + prot |= KVM_PGTABLE_PROT_R; + else if (ap == KVM_PTE_LEAF_ATTR_LO_S1_AP_RW) + prot |= KVM_PGTABLE_PROT_R | KVM_PGTABLE_PROT_W; + + return prot; +} + static bool hyp_pte_needs_update(kvm_pte_t old, kvm_pte_t new) { /* @@ -578,6 +598,23 @@ static int stage2_set_prot_attr(struct kvm_pgtable *pgt, enum kvm_pgtable_prot p return 0; } +enum kvm_pgtable_prot kvm_pgtable_stage2_pte_prot(kvm_pte_t pte) +{ + enum kvm_pgtable_prot prot = pte & KVM_PTE_LEAF_ATTR_HI_SW; + + if (!kvm_pte_valid(pte)) + return prot; + + if (pte & KVM_PTE_LEAF_ATTR_LO_S2_S2AP_R) + prot |= KVM_PGTABLE_PROT_R; + if (pte & KVM_PTE_LEAF_ATTR_LO_S2_S2AP_W) + prot |= KVM_PGTABLE_PROT_W; + if (!(pte & KVM_PTE_LEAF_ATTR_HI_S2_XN)) + prot |= KVM_PGTABLE_PROT_X; + + return prot; +} + static bool stage2_pte_needs_update(kvm_pte_t old, kvm_pte_t new) { if (!kvm_pte_valid(old) || !kvm_pte_valid(new)) From patchwork Mon Jul 26 09:29:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12398951 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7227AC4338F for ; Mon, 26 Jul 2021 09:40:06 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3743160238 for ; Mon, 26 Jul 2021 09:40:06 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 3743160238 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=bXn1q57cBLE8kASNc6rf7iHAmQdD19rITUpRG7TqzIs=; b=3sTEDyIzSOr4UH+S4TF6h/CSVz kRw4qQXm2dLPqC2KQY3JKEeK9IK1Ut/I2MSeYb9zoRcrQKIX1yBTGJm4qfcQrSEun7Adnm/pvD2Ct b9kgNcUMA6jjtXvq1D/2yG3gjqDp5ccI7QfQoU+FLV2G/UZcObBoEynVNGSf0EQJg54HkZLspbDIx Qp1THVjaru5HWcbuRqFXa2AEB4H64CCZpGHRysVMPvADQm+kap+elSFWzcB49dnYqrwENgijrDx/g L6pQpBVbGc9C0Trk3L8DWngu27ipf7UXLJcHOj1QGkSXPRK0r9ER7/2BwmIZq1CBm04C1idVxYT+g SbI+T3kg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m7x3f-00AV3T-Lm; Mon, 26 Jul 2021 09:38:01 +0000 Received: from mail-qk1-x74a.google.com ([2607:f8b0:4864:20::74a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m7wvg-00ARce-0T for linux-arm-kernel@lists.infradead.org; Mon, 26 Jul 2021 09:29:45 +0000 Received: by mail-qk1-x74a.google.com with SMTP id bm25-20020a05620a1999b02903a9c3f8b89fso8521453qkb.2 for ; Mon, 26 Jul 2021 02:29:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=6dY3W9X/AGxD9r1RRTSS9yD4t5RoHsE1QSE3BCqo+aE=; b=Ww9dZYatKjMjIjIW2ylBqljvFXMjm0aQtmZZd32Hr/KhiQqtTftNNfU69HEOr0lDbm zKee/K1VB/L+wtlEDRN5rVkSG/delJAR7PejN/gdK8Jfu8lQAw3AbAugd/GC3GoKPdRM kLKaIWP/+D4QBdboQQ3q/eARC/T+BaNDzFr77y3qvaXSJzgq2aV2CNkWnUuT5eK7E1vI iPbtNmIcBPgzr/nU52DfK1/CvEFwtQ0pEtbAe48HgEUDcRebZvoACysb+/jnYB2DPFmN wGmU9dDogUx6Isrlkg+YCz3jySICBEXb7IY5wYtN7RAnqe7ILDgtZZJ2mLrbRgZcpEVn WuNA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=6dY3W9X/AGxD9r1RRTSS9yD4t5RoHsE1QSE3BCqo+aE=; b=MyfH8ckzIZwjYXiedXuJdUbAE2cng2qxDG6fAcQTcIGERJs84dvFWPm8vxLc2HpQrz bFPYSmQngnEVM8gxsGVTEB0qC4f2e+ngCaTcdZDJ6bN/JmoKef01eHgeyzGirLHc3aW4 9Eh+ocBef/REk435YvrrKiuckH7Bt0dlIKo6g1Om9N3oaAna7Ry+xljq5uBfKPjk2V0q DTZZsyzEzX3fvFaOOzMu9Ao1xZF6unZX4s02j8t0xOsGXNhfBXl4MP1m0iz21nh3SoNc bk3okxM2hmcn56uONp5xFK8vd7+nVW3IcGpVGcDu2haozBNiIoWc+cg4DeH7rRHwKh26 Zy3g== X-Gm-Message-State: AOAM532SlWUGsszo82qPy5eD1fiqjZjFJJPqYaSG7aiAR0auTw+IYoVm 0vYtQWhWGorDhCLtqDIvoxVuKDbYMCiU X-Google-Smtp-Source: ABdhPJxja1z3cl1S4eHZsbLWfOgzNw9V3DmMjl7MkdDjMfOUrn37i6l0Vcan0CxmYEphVI3A4P6qPR4TuQL7 X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:23a0:2f14:433:e6cb]) (user=qperret job=sendgmr) by 2002:a05:6214:e83:: with SMTP id hf3mr17024207qvb.19.1627291782528; Mon, 26 Jul 2021 02:29:42 -0700 (PDT) Date: Mon, 26 Jul 2021 10:29:03 +0100 In-Reply-To: <20210726092905.2198501-1-qperret@google.com> Message-Id: <20210726092905.2198501-15-qperret@google.com> Mime-Version: 1.0 References: <20210726092905.2198501-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.432.gabb21c7263-goog Subject: [PATCH v2 14/16] KVM: arm64: Refactor protected nVHE stage-1 locking From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, Quentin Perret X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210726_022944_106438_81B04237 X-CRM114-Status: GOOD ( 13.45 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Refactor the hypervisor stage-1 locking in nVHE protected mode to expose a new pkvm_create_mappings_locked() function. This will be used in later patches to allow walking and changing the hypervisor stage-1 without releasing the lock. Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/include/nvhe/mm.h | 1 + arch/arm64/kvm/hyp/nvhe/mm.c | 17 +++++++++++++++-- 2 files changed, 16 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kvm/hyp/include/nvhe/mm.h b/arch/arm64/kvm/hyp/include/nvhe/mm.h index 8ec3a5a7744b..c76d7136ed9b 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/mm.h +++ b/arch/arm64/kvm/hyp/include/nvhe/mm.h @@ -23,6 +23,7 @@ int hyp_map_vectors(void); int hyp_back_vmemmap(phys_addr_t phys, unsigned long size, phys_addr_t back); int pkvm_cpu_set_vector(enum arm64_hyp_spectre_vector slot); int pkvm_create_mappings(void *from, void *to, enum kvm_pgtable_prot prot); +int pkvm_create_mappings_locked(void *from, void *to, enum kvm_pgtable_prot prot); int __pkvm_create_mappings(unsigned long start, unsigned long size, unsigned long phys, enum kvm_pgtable_prot prot); unsigned long __pkvm_create_private_mapping(phys_addr_t phys, size_t size, diff --git a/arch/arm64/kvm/hyp/nvhe/mm.c b/arch/arm64/kvm/hyp/nvhe/mm.c index a8efdf0f9003..256cbe5c0dca 100644 --- a/arch/arm64/kvm/hyp/nvhe/mm.c +++ b/arch/arm64/kvm/hyp/nvhe/mm.c @@ -67,7 +67,8 @@ unsigned long __pkvm_create_private_mapping(phys_addr_t phys, size_t size, return addr; } -int pkvm_create_mappings(void *from, void *to, enum kvm_pgtable_prot prot) +/* Must be called with the pkvm_pgd_lock held */ +int pkvm_create_mappings_locked(void *from, void *to, enum kvm_pgtable_prot prot) { unsigned long start = (unsigned long)from; unsigned long end = (unsigned long)to; @@ -81,7 +82,8 @@ int pkvm_create_mappings(void *from, void *to, enum kvm_pgtable_prot prot) int err; phys = hyp_virt_to_phys((void *)virt_addr); - err = __pkvm_create_mappings(virt_addr, PAGE_SIZE, phys, prot); + err = kvm_pgtable_hyp_map(&pkvm_pgtable, virt_addr, PAGE_SIZE, + phys, prot); if (err) return err; } @@ -89,6 +91,17 @@ int pkvm_create_mappings(void *from, void *to, enum kvm_pgtable_prot prot) return 0; } +int pkvm_create_mappings(void *from, void *to, enum kvm_pgtable_prot prot) +{ + int ret; + + hyp_spin_lock(&pkvm_pgd_lock); + ret = pkvm_create_mappings_locked(from, to, prot); + hyp_spin_unlock(&pkvm_pgd_lock); + + return ret; +} + int hyp_back_vmemmap(phys_addr_t phys, unsigned long size, phys_addr_t back) { unsigned long start, end; From patchwork Mon Jul 26 09:29:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12398953 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 93614C4338F for ; Mon, 26 Jul 2021 09:41:35 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 57D1060F11 for ; Mon, 26 Jul 2021 09:41:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 57D1060F11 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=6H5rIGEQ3+9LiSRrIun/Beo6cmBbjvxLEQ9TM6YMugg=; b=JkGkQi5ye427IURzFDUgrx04vB Nf3ckStgwkLO/DVM/eOvOxWeG5ONIC6dfY4+oNniITN99l79avRuPPM0ZyFk75YQAbs+SWqsBznC9 Yj+rXiOlO9bSTCQB6buBbz9Ja1V87CShGy438i1ApHpK3QHvoKlpKm8DuU7aVovU6HUL2Ac6F53vv ezEbqa8CRm5eutXKqPem0cTKo9IF6acP5oWfcYJCtfOZ9GG3DZIAnaDS7Ijdg8Bfv9fJA7sfiZcCZ erh47POnUBaF21zTDMuQfXMEK0jy4KgJsBHMvikXNhCBkEYcan0XLJQUK84DIHARLfQfFwDygHG0i nFgVDm2Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m7x4m-00AVSz-Ad; Mon, 26 Jul 2021 09:39:09 +0000 Received: from mail-qt1-x849.google.com ([2607:f8b0:4864:20::849]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m7wvi-00ARdD-4B for linux-arm-kernel@lists.infradead.org; Mon, 26 Jul 2021 09:29:47 +0000 Received: by mail-qt1-x849.google.com with SMTP id f9-20020a05622a1a09b02902615523e725so3981860qtb.21 for ; Mon, 26 Jul 2021 02:29:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=aIX6xnE8zrFaLzY6zKSYesqv/ybRjMtzHpsGgsZjePU=; b=AKqVOtdbPJZfuI9SGEOBzNkUPBgGuAKlIF2CC0rYzNIewRZE3+fMM1rZZFHKBcvAWU 91wTXuuRYF9wXou5LxhrIRnVc8hpBI3hQe28Su92hOEdfubiwLrD/1t41N2BOfpz9TtU TPDu7xDyRgtzFVVw5OQFvXEB31WuwZe9PcPofSBr85aUX4B1W+E2LmYRHq5Nf1oOaYA1 JhL7+gD7tyclI2KfyX22t20aetWQz/MfhGckwTVXUr2POFp9WgjtTx1WdR+/NNYwGMsi Q0OkNTxymKpgV4U++xw1gbR0kc5DN3blpr6Wd3DgJ98iwhlf94bZqyGujI8XDtMkw02w w12A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=aIX6xnE8zrFaLzY6zKSYesqv/ybRjMtzHpsGgsZjePU=; b=SKjfJGUsWlzOipHblaeAXA87n49w573dqyYhjgw3iGWmJg9HhLQLNJl943f7h8MJMd /+oQHcEwAcs0TTVRcvdsLOeO2C6fISvr7wPXaBAv/0vrZEsFb5sLaNp856igQPQDZqXy I122aScOJ9btVZZR2BRVmvetPhsHVw7+dB80CgzQvWA1FRSymHZORN9iYsqCGCqflD3I QuOxU9dTrV7Z0rd7rMqPFGCMV+AlQSz5Gkw45oUBAmlyKY84Nq7ouzwRWc/mboHjg/Fy boLT7XZZwOa2Qt441ThhxqXD4IYyVDb/IvmVQFVShktH+bVC6NTG69cMtPDUkdt6Ab3p e0+A== X-Gm-Message-State: AOAM533f5/LEDvvESg4JLAlR50x0M/u/7KSy5pR6HrzavLNhZmclL4R/ 7EzNMBO14Y5ovNpdguGAjtM588ZZrP0u X-Google-Smtp-Source: ABdhPJz+6y5CnoN87oobsVqCl0gHo4b+5rQT7Y9mI+YcG/ciA9YQ647DFc4SlAzgitr8BhoEDkXPRXpOOssh X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:23a0:2f14:433:e6cb]) (user=qperret job=sendgmr) by 2002:ad4:4bae:: with SMTP id i14mr17035153qvw.24.1627291784628; Mon, 26 Jul 2021 02:29:44 -0700 (PDT) Date: Mon, 26 Jul 2021 10:29:04 +0100 In-Reply-To: <20210726092905.2198501-1-qperret@google.com> Message-Id: <20210726092905.2198501-16-qperret@google.com> Mime-Version: 1.0 References: <20210726092905.2198501-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.432.gabb21c7263-goog Subject: [PATCH v2 15/16] KVM: arm64: Restrict EL2 stage-1 changes in protected mode From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, Quentin Perret X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210726_022946_222160_09B9CE0A X-CRM114-Status: GOOD ( 22.33 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The host kernel is currently able to change EL2 stage-1 mappings without restrictions thanks to the __pkvm_create_mappings() hypercall. But in a world where the host is no longer part of the TCB, this clearly poses a problem. To fix this, introduce a new hypercall to allow the host to share a physical memory page with the hypervisor, and remove the __pkvm_create_mappings() variant. The new hypercall implements ownership and permission checks before allowing the sharing operation, and it annotates the shared page in the hypervisor stage-1 and host stage-2 page-tables. Signed-off-by: Quentin Perret --- arch/arm64/include/asm/kvm_asm.h | 2 +- arch/arm64/kvm/hyp/include/nvhe/mem_protect.h | 1 + arch/arm64/kvm/hyp/nvhe/hyp-main.c | 11 +-- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 90 +++++++++++++++++++ arch/arm64/kvm/mmu.c | 28 +++++- 5 files changed, 120 insertions(+), 12 deletions(-) diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h index 9f0bf2109be7..78db818ae2c9 100644 --- a/arch/arm64/include/asm/kvm_asm.h +++ b/arch/arm64/include/asm/kvm_asm.h @@ -59,7 +59,7 @@ #define __KVM_HOST_SMCCC_FUNC___vgic_v3_save_aprs 13 #define __KVM_HOST_SMCCC_FUNC___vgic_v3_restore_aprs 14 #define __KVM_HOST_SMCCC_FUNC___pkvm_init 15 -#define __KVM_HOST_SMCCC_FUNC___pkvm_create_mappings 16 +#define __KVM_HOST_SMCCC_FUNC___pkvm_host_share_hyp 16 #define __KVM_HOST_SMCCC_FUNC___pkvm_create_private_mapping 17 #define __KVM_HOST_SMCCC_FUNC___pkvm_cpu_set_vector 18 #define __KVM_HOST_SMCCC_FUNC___pkvm_prot_finalize 19 diff --git a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h index 8e5725d032b2..1b8e59a9d065 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h +++ b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h @@ -49,6 +49,7 @@ extern struct host_kvm host_kvm; int __pkvm_prot_finalize(void); int __pkvm_mark_hyp(phys_addr_t start, phys_addr_t end); +int __pkvm_host_share_hyp(u64 pfn); int host_stage2_idmap_locked(u64 start, u64 end, enum kvm_pgtable_prot prot); int kvm_host_prepare_stage2(void *pgt_pool_base); diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c index 1632f001f4ed..dd155646ec86 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -140,14 +140,11 @@ static void handle___pkvm_cpu_set_vector(struct kvm_cpu_context *host_ctxt) cpu_reg(host_ctxt, 1) = pkvm_cpu_set_vector(slot); } -static void handle___pkvm_create_mappings(struct kvm_cpu_context *host_ctxt) +static void handle___pkvm_host_share_hyp(struct kvm_cpu_context *host_ctxt) { - DECLARE_REG(unsigned long, start, host_ctxt, 1); - DECLARE_REG(unsigned long, size, host_ctxt, 2); - DECLARE_REG(unsigned long, phys, host_ctxt, 3); - DECLARE_REG(enum kvm_pgtable_prot, prot, host_ctxt, 4); + DECLARE_REG(u64, pfn, host_ctxt, 1); - cpu_reg(host_ctxt, 1) = __pkvm_create_mappings(start, size, phys, prot); + cpu_reg(host_ctxt, 1) = __pkvm_host_share_hyp(pfn); } static void handle___pkvm_create_private_mapping(struct kvm_cpu_context *host_ctxt) @@ -193,7 +190,7 @@ static const hcall_t host_hcall[] = { HANDLE_FUNC(__vgic_v3_restore_aprs), HANDLE_FUNC(__pkvm_init), HANDLE_FUNC(__pkvm_cpu_set_vector), - HANDLE_FUNC(__pkvm_create_mappings), + HANDLE_FUNC(__pkvm_host_share_hyp), HANDLE_FUNC(__pkvm_create_private_mapping), HANDLE_FUNC(__pkvm_prot_finalize), HANDLE_FUNC(__pkvm_mark_hyp), diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c index 223c541a7051..75273166d2c5 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -315,6 +315,96 @@ static int host_stage2_idmap(u64 addr) return ret; } +static inline bool check_prot(enum kvm_pgtable_prot prot, + enum kvm_pgtable_prot required, + enum kvm_pgtable_prot denied) +{ + return (prot & (required | denied)) == required; +} + +int __pkvm_host_share_hyp(u64 pfn) +{ + phys_addr_t addr = hyp_pfn_to_phys(pfn); + enum kvm_pgtable_prot prot, cur; + void * virt = __hyp_va(addr); + enum pkvm_page_state state; + kvm_pte_t pte; + u32 level; + int ret; + + if (!range_is_memory(addr, addr + PAGE_SIZE)) + return -EINVAL; + + hyp_spin_lock(&host_kvm.lock); + hyp_spin_lock(&pkvm_pgd_lock); + + ret = kvm_pgtable_get_leaf(&host_kvm.pgt, addr, &pte, &level); + if (ret) + goto unlock; + if (!pte) + goto map_shared; + + /* + * Check attributes in the host stage-2 PTE. We need the page to be: + * - mapped RWX as we're sharing memory; + * - not borrowed, as that implies absence of ownership. + * Otherwise, we can't let it got through + */ + cur = kvm_pgtable_stage2_pte_prot(pte); + prot = pkvm_mkstate(0, PKVM_PAGE_SHARED_BORROWED); + if (!check_prot(cur, KVM_PGTABLE_PROT_RWX, prot)) { + ret = -EPERM; + goto unlock; + } + + state = pkvm_getstate(cur); + if (state == PKVM_PAGE_OWNED) + goto map_shared; + + /* + * Tolerate double-sharing the same page, but this requires + * cross-checking the hypervisor stage-1. + */ + if (state != PKVM_PAGE_SHARED_OWNED) { + ret = -EPERM; + goto unlock; + } + + ret = kvm_pgtable_get_leaf(&pkvm_pgtable, (u64)virt, &pte, &level); + if (ret) + goto unlock; + + /* + * If the page has been shared with the hypervisor, it must be + * SHARED_BORROWED already. + */ + cur = kvm_pgtable_hyp_pte_prot(pte); + prot = pkvm_mkstate(PAGE_HYP, PKVM_PAGE_SHARED_BORROWED); + if (!check_prot(cur, prot, ~prot)) + ret = EPERM; + goto unlock; + +map_shared: + /* + * If the page is not yet shared, adjust mappings in both page-tables + * while both locks are held. + */ + prot = pkvm_mkstate(PAGE_HYP, PKVM_PAGE_SHARED_BORROWED); + ret = pkvm_create_mappings_locked(virt, virt + PAGE_SIZE, prot); + if (ret) + goto unlock; + + prot = pkvm_mkstate(KVM_PGTABLE_PROT_RWX, PKVM_PAGE_SHARED_OWNED); + ret = host_stage2_idmap_locked(addr, addr + PAGE_SIZE, prot); + BUG_ON(ret); + +unlock: + hyp_spin_unlock(&pkvm_pgd_lock); + hyp_spin_unlock(&host_kvm.lock); + + return ret; +} + int __pkvm_mark_hyp(phys_addr_t start, phys_addr_t end) { int ret; diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index 0625bf2353c2..cbab146cda6a 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -259,10 +259,8 @@ static int __create_hyp_mappings(unsigned long start, unsigned long size, { int err; - if (!kvm_host_owns_hyp_mappings()) { - return kvm_call_hyp_nvhe(__pkvm_create_mappings, - start, size, phys, prot); - } + if (WARN_ON(!kvm_host_owns_hyp_mappings())) + return -EINVAL; mutex_lock(&kvm_hyp_pgd_mutex); err = kvm_pgtable_hyp_map(hyp_pgtable, start, size, phys, prot); @@ -282,6 +280,21 @@ static phys_addr_t kvm_kaddr_to_phys(void *kaddr) } } +static int pkvm_share_hyp(phys_addr_t start, phys_addr_t end) +{ + phys_addr_t addr; + int ret; + + for (addr = ALIGN_DOWN(start, PAGE_SIZE); addr < end; addr += PAGE_SIZE) { + ret = kvm_call_hyp_nvhe(__pkvm_host_share_hyp, + __phys_to_pfn(addr)); + if (ret) + return ret; + } + + return 0; +} + /** * create_hyp_mappings - duplicate a kernel virtual address range in Hyp mode * @from: The virtual kernel start address of the range @@ -302,6 +315,13 @@ int create_hyp_mappings(void *from, void *to, enum kvm_pgtable_prot prot) if (is_kernel_in_hyp_mode()) return 0; + if (!kvm_host_owns_hyp_mappings()) { + if (WARN_ON(prot != PAGE_HYP)) + return -EPERM; + return pkvm_share_hyp(kvm_kaddr_to_phys(from), + kvm_kaddr_to_phys(to)); + } + start = start & PAGE_MASK; end = PAGE_ALIGN(end); From patchwork Mon Jul 26 09:29:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12398955 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9C880C4338F for ; Mon, 26 Jul 2021 09:42:44 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5BCB860F42 for ; Mon, 26 Jul 2021 09:42:44 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 5BCB860F42 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=hb+WJ96igbGXgu8Eqz+TxN0n3m5Ergwiav1Bxbf+Cvo=; b=iA8GgB5EH/8nRZctNGsBshgcyG etMgRyGjuR3Sj2912W4PGRd8N1BUKRFukzf9H17yMmVlRorF9F3zw4wv72zMpLnTe0kqJqajie1UU C2H2nd+zmvwNDhIErItYg4g1dFgDIqyg3RFujccHSndI7qtmN93FiDDjw5EhxZGjpZr1BXhzptjIJ pHQJf1Vh9rbWtAXL+ZCBuyk6K3zpp+c2ZoElQB7ymWDmqwQny0i4mRYdv1zqOCtv1yE4y1nwfpMl7 qy6fwoiMd4Lw9XY8S8ro1fzFECtWGBXUaBJ4t/J/k79u92asYz7ZQWFa2VsckYGbHZr98molwSXlh Kg19Po+Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m7x5n-00AVsS-Nl; Mon, 26 Jul 2021 09:40:12 +0000 Received: from mail-qk1-x749.google.com ([2607:f8b0:4864:20::749]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m7wvk-00ARdi-BS for linux-arm-kernel@lists.infradead.org; Mon, 26 Jul 2021 09:29:49 +0000 Received: by mail-qk1-x749.google.com with SMTP id p14-20020a05620a22eeb02903b94aaa0909so8450718qki.15 for ; Mon, 26 Jul 2021 02:29:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=4KB8zz71n+ckn7q+A5npd/SGGm4PNZq4hk1c6+NwEJY=; b=kkbETL0dMsBo3aqN8t9V/kIx0Ztkf3eaIwZl0PzH5SGS9l2R07gZCo3SrYhlVi0Aq8 DYE6R8DSj9iT04q9VHzh6K+4SpsFIWhZ21iXDGINKGvc1prCFhcttOusRwxU37nyzoBJ cGXebYm+WF+eQ3xmw+KAK1J3BjSZ3bPpSTpuIEjXY099wkk5S/7jGm6iLAJW4Ee1idbp 5Skr4vyXufnVr58Rb40+iWEREgGtIjuPt7Gzvmz14VSeW3GBhdK8I9sPFbIysDe099cl VJTQxXrn2ZOjquQC+U5SzPZy1y6uov5YuZ6+JVs5ya29jcZkLE3TflfMn+pwghu4p+ZT uC9A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=4KB8zz71n+ckn7q+A5npd/SGGm4PNZq4hk1c6+NwEJY=; b=RNXcdCwuXtp93XdYVbXGG/hUjA1lTHyS6h/wNkQjdZimaUvZyEt0kS3R5MCqdBmuV7 WPJoFcmYJWZvBhlKw+IjdeprpGI1/XyWEtSEvqYkfsicjy4/CQyP3PoojUGCsYOvAOPo +7BL+G0NaZr52VyDjTbJPXpa7Jg5gsgmDXA7uQ9ZyUypQ1uHVy/hV63eD9va1xjuCBrI ffm/LAPvfUEHSfwzwh/VwejeK97c9uprI/QUH45trFi8/mQd26EDpe/ve32LChBMig1R L48v9F4TBaStYBkAtSrh98KP9n64g5iIwAiDfQQ6NDdZHfsDT2JomYbWf0TY/Hp36mC7 qSMg== X-Gm-Message-State: AOAM53151Qe6kpPhhFrXVZPDEXqly3HvI4sTmQTCj9NI5JPVmzUdIFkV tQXwpOkXZ0mj0DsQ2NyXodvz4MScY3Za X-Google-Smtp-Source: ABdhPJw4rV2DFKIOrC4bodeI8pRwP94WRLnn2cUKstVXAH0e5UMS4sxvzAPdNEu8x+a/0hc9RNwJfIDJlSr6 X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:23a0:2f14:433:e6cb]) (user=qperret job=sendgmr) by 2002:a05:6214:2482:: with SMTP id gi2mr13928371qvb.51.1627291786850; Mon, 26 Jul 2021 02:29:46 -0700 (PDT) Date: Mon, 26 Jul 2021 10:29:05 +0100 In-Reply-To: <20210726092905.2198501-1-qperret@google.com> Message-Id: <20210726092905.2198501-17-qperret@google.com> Mime-Version: 1.0 References: <20210726092905.2198501-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.432.gabb21c7263-goog Subject: [PATCH v2 16/16] KVM: arm64: Make __pkvm_create_mappings static From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, Quentin Perret X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210726_022948_435552_C38353B3 X-CRM114-Status: GOOD ( 11.06 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The __pkvm_create_mappings() function is no longer used outside of nvhe/mm.c, make it static. Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/include/nvhe/mm.h | 2 -- arch/arm64/kvm/hyp/nvhe/mm.c | 4 ++-- 2 files changed, 2 insertions(+), 4 deletions(-) diff --git a/arch/arm64/kvm/hyp/include/nvhe/mm.h b/arch/arm64/kvm/hyp/include/nvhe/mm.h index c76d7136ed9b..c9a8f535212e 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/mm.h +++ b/arch/arm64/kvm/hyp/include/nvhe/mm.h @@ -24,8 +24,6 @@ int hyp_back_vmemmap(phys_addr_t phys, unsigned long size, phys_addr_t back); int pkvm_cpu_set_vector(enum arm64_hyp_spectre_vector slot); int pkvm_create_mappings(void *from, void *to, enum kvm_pgtable_prot prot); int pkvm_create_mappings_locked(void *from, void *to, enum kvm_pgtable_prot prot); -int __pkvm_create_mappings(unsigned long start, unsigned long size, - unsigned long phys, enum kvm_pgtable_prot prot); unsigned long __pkvm_create_private_mapping(phys_addr_t phys, size_t size, enum kvm_pgtable_prot prot); diff --git a/arch/arm64/kvm/hyp/nvhe/mm.c b/arch/arm64/kvm/hyp/nvhe/mm.c index 256cbe5c0dca..80303041f72f 100644 --- a/arch/arm64/kvm/hyp/nvhe/mm.c +++ b/arch/arm64/kvm/hyp/nvhe/mm.c @@ -23,8 +23,8 @@ u64 __io_map_base; struct memblock_region hyp_memory[HYP_MEMBLOCK_REGIONS]; unsigned int hyp_memblock_nr; -int __pkvm_create_mappings(unsigned long start, unsigned long size, - unsigned long phys, enum kvm_pgtable_prot prot) +static int __pkvm_create_mappings(unsigned long start, unsigned long size, + unsigned long phys, enum kvm_pgtable_prot prot) { int err;