From patchwork Wed Mar 27 12:48:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 13606538 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1CF9BC54E67 for ; Wed, 27 Mar 2024 12:49:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=1BwgTrQF6bFFhClPLJ2eSjlKOYfgQcj5InSYr8qf1BY=; b=SYHdtFaDxLqYBz 0oOXCAtVam9z9G6NxC8TyyhPid2ZPb3gBIIrqGr1mK2P/peqXIvzXmMU2hbctsUz2gaTfhm/yg18h Le+7iet/J8+otjgfpSCuLYSuU1udMHXCFBN+ArSSovChsr3dYlcMIsNflQvT3X+Pb8mkUpYAZyxx/ roVT4ANh/XchgxqvF9LuOg+2oQGrWeOIW5gI3PHE1MuJp4qqkpDjrlHPSEykxQ9bqS0VWQLDZhvD0 c5DDVKJI/xa6RUYW8eGAT8w9p9E02FQuQmc6UfcXHdB634MXjsqV+JzHJuxAp1H1yHnGOkGvBGqKc 9/xPFaT226eKREW5ILmQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rpSiO-00000008znV-0shw; Wed, 27 Mar 2024 12:49:12 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rpSiG-00000008zjS-0e5A for linux-arm-kernel@lists.infradead.org; Wed, 27 Mar 2024 12:49:05 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 996DB61503; Wed, 27 Mar 2024 12:49:03 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8F41FC43399; Wed, 27 Mar 2024 12:48:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1711543741; bh=gzOFJmJsseLiXdBw91aKhD2nQfl7xlgBvKD45uJaxL0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=kzhvURJn5vap1L6jRO7DUBAGBkuFGggxrSmz3yjm9A90s6oZOUsPM9lOnMZKa4t+k 7tzdI1dFq1jzmoYZEH5sau+s7YT+dns/jZYnWV8SShpl38VfdUVxF/jbMXei7cZwUz D7SKC8QKMCsd98isWoyT3hnBtJ8mQSCOl1Uh56Exz8Ufls2pI1JtJ4o8PiDBRwbeZV +vvRWBaTaqIdCHbsMPXDvTVQBxfsm1zmPDgCDo7rB4cJtolYl9v5puT1/DvDcmr9X6 /X7hm3pyFPyyu2/Tufj9QbhH+pJLUkhFEk3i949UmO/LlpZaLy6Rr0Q0glwMgfd1yx VLGyVlbGjGUUw== From: Will Deacon To: kvmarm@lists.linux.dev Cc: linux-arm-kernel@lists.infradead.org, Will Deacon , Catalin Marinas , Gavin Shan , Marc Zyngier , Mostafa Saleh , Oliver Upton , Quentin Perret , Raghavendra Rao Ananta , Ryan Roberts , Shaoqin Huang Subject: [PATCH v2 1/4] KVM: arm64: Don't defer TLB invalidation when zapping table entries Date: Wed, 27 Mar 2024 12:48:50 +0000 Message-Id: <20240327124853.11206-2-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20240327124853.11206-1-will@kernel.org> References: <20240327124853.11206-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240327_054904_426757_028B57FC X-CRM114-Status: GOOD ( 13.60 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Commit 7657ea920c54 ("KVM: arm64: Use TLBI range-based instructions for unmap") introduced deferred TLB invalidation for the stage-2 page-table so that range-based invalidation can be used for the accumulated addresses. This works fine if the structure of the page-tables remains unchanged, but if entire tables are zapped and subsequently freed then we transiently leave the hardware page-table walker with a reference to freed memory thanks to the translation walk caches. For example, stage2_unmap_walker() will free page-table pages: if (childp) mm_ops->put_page(childp); and issue the TLB invalidation later in kvm_pgtable_stage2_unmap(): if (stage2_unmap_defer_tlb_flush(pgt)) /* Perform the deferred TLB invalidations */ kvm_tlb_flush_vmid_range(pgt->mmu, addr, size); For now, take the conservative approach and invalidate the TLB eagerly when we clear a table entry. Note, however, that the existing level hint passed to __kvm_tlb_flush_vmid_ipa() is incorrect and will be fixed in a subsequent patch. Cc: Raghavendra Rao Ananta Cc: Shaoqin Huang Cc: Marc Zyngier Cc: Oliver Upton Signed-off-by: Will Deacon --- arch/arm64/kvm/hyp/pgtable.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index 3fae5830f8d2..de0b667ba296 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -896,9 +896,11 @@ static void stage2_unmap_put_pte(const struct kvm_pgtable_visit_ctx *ctx, if (kvm_pte_valid(ctx->old)) { kvm_clear_pte(ctx->ptep); - if (!stage2_unmap_defer_tlb_flush(pgt)) + if (!stage2_unmap_defer_tlb_flush(pgt) || + kvm_pte_table(ctx->old, ctx->level)) { kvm_call_hyp(__kvm_tlb_flush_vmid_ipa, mmu, ctx->addr, ctx->level); + } } mm_ops->put_page(ctx->ptep); From patchwork Wed Mar 27 12:48:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 13606539 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 14C99C54E67 for ; Wed, 27 Mar 2024 12:49:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=guw71TxnbIcXcQL2p0cguWE4AhRVUl2m10pxaZtjFd8=; b=PH+pCG3E5ipH94 Ccr4A+MQu/rQ9dTATZYT3OrbwAGfLOgHrTiWg3r4SBWzWWcQoSrdEo8OH81G/V8Z+5NcGq6PnbvYS f/77jGScawn89EFxJ0KK2mLa0Igo4n3P2XhiRfMljD3P4llxMVP/craOhQcn4WeFjJKCdhfMkZc4p IUuCm/AJHgLvpftHhZ7sR2EVddPtt5ZPfg5LaLIH75XtBdplUT0e1RanNu+D3RLy9lXUd7tQQbfCf LdrXO6tvZilKfTCuXkDrDib8kLYi4VGT7WoPjnxF2urgvJ7oLs1knksAZMVARIj7UlrkmMQEp4ltj ej1HrpFYXL1Fo6JvYu9A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rpSiO-00000008zo7-47iQ; Wed, 27 Mar 2024 12:49:12 +0000 Received: from sin.source.kernel.org ([2604:1380:40e1:4800::1]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rpSiJ-00000008zkM-17pG for linux-arm-kernel@lists.infradead.org; Wed, 27 Mar 2024 12:49:08 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 7FE2FCE19A2; Wed, 27 Mar 2024 12:49:05 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 680F5C433F1; Wed, 27 Mar 2024 12:49:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1711543744; bh=bXN/HevD7CPAdyg/mpq/H+7HLHjel9WBtmGPHdrhUQY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QLO7/kcqYjCcdII7Jo9ynbKhRdeNj36WLWFGPCWwUpP+sibe4I6YThh/jHQltBHT0 GnTaMZJq6eeRUS8Bd9Kp1JjRCtNqLNUTxUaz1DHDd+tp3inpurlhbSTrTuK8CHdVGB sP9URBFsDQdpArFFjH3Pgq0dS99hz6FLnBsSbiwTbqUB89YCdprP0olff/jwyzA0Tr AB8xnL1QUBcMRRnRbAcFcvGf6sXzfWPkdMd/Q4zB2Wouy+TF9zxcM1N411wpGqDzsi IpW6U4B6t0rys61rI6iqfzS5JaSP5WYkU6FWZzGK5/gPCQwIZAPMdphFw0Bjh2NOEc tW6w/1ev5Q0sQ== From: Will Deacon To: kvmarm@lists.linux.dev Cc: linux-arm-kernel@lists.infradead.org, Will Deacon , Catalin Marinas , Gavin Shan , Marc Zyngier , Mostafa Saleh , Oliver Upton , Quentin Perret , Raghavendra Rao Ananta , Ryan Roberts , Shaoqin Huang Subject: [PATCH v2 2/4] KVM: arm64: Don't pass a TLBI level hint when zapping table entries Date: Wed, 27 Mar 2024 12:48:51 +0000 Message-Id: <20240327124853.11206-3-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20240327124853.11206-1-will@kernel.org> References: <20240327124853.11206-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240327_054907_539585_B169FEF8 X-CRM114-Status: GOOD ( 10.61 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The TLBI level hints are for leaf entries only, so take care not to pass them incorrectly after clearing a table entry. Cc: Gavin Shan Cc: Marc Zyngier Cc: Quentin Perret Fixes: 82bb02445de5 ("KVM: arm64: Implement kvm_pgtable_hyp_unmap() at EL2") Fixes: 6d9d2115c480 ("KVM: arm64: Add support for stage-2 map()/unmap() in generic page-table") Signed-off-by: Will Deacon --- arch/arm64/kvm/hyp/pgtable.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index de0b667ba296..a40dafc43bb6 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -528,7 +528,7 @@ static int hyp_unmap_walker(const struct kvm_pgtable_visit_ctx *ctx, kvm_clear_pte(ctx->ptep); dsb(ishst); - __tlbi_level(vae2is, __TLBI_VADDR(ctx->addr, 0), ctx->level); + __tlbi_level(vae2is, __TLBI_VADDR(ctx->addr, 0), TLBI_TTL_UNKNOWN); } else { if (ctx->end - ctx->addr < granule) return -EINVAL; @@ -896,10 +896,12 @@ static void stage2_unmap_put_pte(const struct kvm_pgtable_visit_ctx *ctx, if (kvm_pte_valid(ctx->old)) { kvm_clear_pte(ctx->ptep); - if (!stage2_unmap_defer_tlb_flush(pgt) || - kvm_pte_table(ctx->old, ctx->level)) { - kvm_call_hyp(__kvm_tlb_flush_vmid_ipa, mmu, - ctx->addr, ctx->level); + if (kvm_pte_table(ctx->old, ctx->level)) { + kvm_call_hyp(__kvm_tlb_flush_vmid_ipa, mmu, ctx->addr, + TLBI_TTL_UNKNOWN); + } else if (!stage2_unmap_defer_tlb_flush(pgt)) { + kvm_call_hyp(__kvm_tlb_flush_vmid_ipa, mmu, ctx->addr, + ctx->level); } } From patchwork Wed Mar 27 12:48:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 13606540 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 100FFC47DD9 for ; Wed, 27 Mar 2024 12:49:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=gl/0CqLx7EqsPhmlR0Rhk0XF5QDo28aZEg/jEhvSYFs=; b=WtSJSeX/4CK/aP MfjKUz2wdlfFwIliwMBlSDoLUgAl/hPrpdQNDYfPNuoVNYbqYH0HPbM2uh5ss4kRjs+a82PQQJUVK iFjHLnEahaR1bhAdHWe7+YexeQIEffLGvklHClO02xdnMkyva5E7PK+G7BXJkRLbOC+JvbIuR0Dte k4nxyxj9/L+AVbxzMSIwSJa+YsrWSmhM7hpxHcl3Rq4TqKmjn8SBna8rviHQn/mkw9gmcDeoJcWFA TVQCbXCzoXNig8nxaWXGkrtr6lcksKXR8/EIXnPMblTOahQYlmBsyYLgdzbqPjY8pnJt9gKTWHvap d2sHyAhjn+dSlJlIq34Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rpSie-00000008zwS-3yTU; Wed, 27 Mar 2024 12:49:28 +0000 Received: from sin.source.kernel.org ([2604:1380:40e1:4800::1]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rpSiM-00000008zlc-0xtO for linux-arm-kernel@lists.infradead.org; Wed, 27 Mar 2024 12:49:12 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 57D1CCE262F; Wed, 27 Mar 2024 12:49:08 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 41156C43399; Wed, 27 Mar 2024 12:49:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1711543747; bh=RUbUwiDDCmKdVzP1MptQx2+25gra+2nVr0exWg9VYxo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dFfLlfW+8VqDLHrIgG3G7EIhdB4e2uSOo0qBiKw1dYwO+LgbN7VCySLQNEgrYHLDE qtPeM3nlUN7OqAqHJwwOKHs/to/tsOO3eq1h4YdZJjYratX1gWhxL7/yOp5OiKvWX5 SyfTQ8l6SJ6E81JEC1nwBJJmIEzUOQd1GhaL/rOEDHYUcngzOMPFNMEu3YwBepgEcb CKBtKmyjwh6V5flzlGiGR1S139Ih7jJd2uSrm7Q9nw04tPgsIPZ/d3YPH28NjDvW7F QHmGoxatjwmntE4kVq08ln57ZYbiCNwaTBckyI+RAAJ2Kvf9xCZ5Gc0ZjDMBkrMgsk vb+Ze52bwGebA== From: Will Deacon To: kvmarm@lists.linux.dev Cc: linux-arm-kernel@lists.infradead.org, Will Deacon , Catalin Marinas , Gavin Shan , Marc Zyngier , Mostafa Saleh , Oliver Upton , Quentin Perret , Raghavendra Rao Ananta , Ryan Roberts , Shaoqin Huang Subject: [PATCH v2 3/4] KVM: arm64: Use TLBI_TTL_UNKNOWN in __kvm_tlb_flush_vmid_range() Date: Wed, 27 Mar 2024 12:48:52 +0000 Message-Id: <20240327124853.11206-4-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20240327124853.11206-1-will@kernel.org> References: <20240327124853.11206-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240327_054910_576589_EF5C98D7 X-CRM114-Status: GOOD ( 11.92 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Commit c910f2b65518 ("arm64/mm: Update tlb invalidation routines for FEAT_LPA2") updated the __tlbi_level() macro to take the target level as an argument, with TLBI_TTL_UNKNOWN (rather than 0) indicating that the caller cannot provide level information. Unfortunately, the two implementations of __kvm_tlb_flush_vmid_range() were not updated and so now ask for an level 0 invalidation if FEAT_LPA2 is implemented. Fix the problem by passing TLBI_TTL_UNKNOWN instead of 0 as the level argument to __flush_s2_tlb_range_op() in __kvm_tlb_flush_vmid_range(). Cc: Catalin Marinas Cc: Oliver Upton Cc: Marc Zyngier Reviewed-by: Ryan Roberts Fixes: c910f2b65518 ("arm64/mm: Update tlb invalidation routines for FEAT_LPA2") Signed-off-by: Will Deacon --- arch/arm64/kvm/hyp/nvhe/tlb.c | 3 ++- arch/arm64/kvm/hyp/vhe/tlb.c | 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kvm/hyp/nvhe/tlb.c b/arch/arm64/kvm/hyp/nvhe/tlb.c index a60fb13e2192..2fc68da4036d 100644 --- a/arch/arm64/kvm/hyp/nvhe/tlb.c +++ b/arch/arm64/kvm/hyp/nvhe/tlb.c @@ -154,7 +154,8 @@ void __kvm_tlb_flush_vmid_range(struct kvm_s2_mmu *mmu, /* Switch to requested VMID */ __tlb_switch_to_guest(mmu, &cxt, false); - __flush_s2_tlb_range_op(ipas2e1is, start, pages, stride, 0); + __flush_s2_tlb_range_op(ipas2e1is, start, pages, stride, + TLBI_TTL_UNKNOWN); dsb(ish); __tlbi(vmalle1is); diff --git a/arch/arm64/kvm/hyp/vhe/tlb.c b/arch/arm64/kvm/hyp/vhe/tlb.c index b32e2940df7d..1a60b95381e8 100644 --- a/arch/arm64/kvm/hyp/vhe/tlb.c +++ b/arch/arm64/kvm/hyp/vhe/tlb.c @@ -171,7 +171,8 @@ void __kvm_tlb_flush_vmid_range(struct kvm_s2_mmu *mmu, /* Switch to requested VMID */ __tlb_switch_to_guest(mmu, &cxt); - __flush_s2_tlb_range_op(ipas2e1is, start, pages, stride, 0); + __flush_s2_tlb_range_op(ipas2e1is, start, pages, stride, + TLBI_TTL_UNKNOWN); dsb(ish); __tlbi(vmalle1is); From patchwork Wed Mar 27 12:48:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 13606541 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 29238C54E67 for ; Wed, 27 Mar 2024 12:49:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Fuz4Kmqear+mRE0/l7xYasSszo/BFEeq/CQbMgh6bHk=; b=xYLH/BsGUOef0m +/y3LL3hh6rZBV1pDgKDHcLn+RUKNOhAocga118mowBWzDWoBpYxCahiEiHrTxz7tvtoPdAQ8vRv0 hdgFAEKPaeN7+4BxYTYEY7LAY4xJWPJvzHVWQStpRqCQnVnOSVUSLLu9TgyrhdPYWqR1zp2Ov8UGF NTI8l1YBNz6rlkqmh5224cKGhBIFWkMO1r0jzoqfzxyArOfAWKjCknR7mapsSHEltA3RmyWJUgEyX 04vq2FV3v8BtcqyvLRnkl4A91XlzUI358igdBwyMgxRdrxZ0wmAeVWLzK7/OTSn7SagNURywaP+RN yVJRLtjhCuGQ0e7mDoXg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rpSif-00000008zxA-3FCf; Wed, 27 Mar 2024 12:49:29 +0000 Received: from sin.source.kernel.org ([2604:1380:40e1:4800::1]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rpSiO-00000008znL-3HnH for linux-arm-kernel@lists.infradead.org; Wed, 27 Mar 2024 12:49:15 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 3104BCE2615; Wed, 27 Mar 2024 12:49:11 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 18E6DC433C7; Wed, 27 Mar 2024 12:49:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1711543750; bh=objC47luTJxKbsIzJPJvLhwVeuYX+ge06C/74jvV0DM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nhGBex4AXUVbMACcE7T10JlD9cggBVLBRuupnanYXtoijba0MW5kUykotADa9ADk+ +/xqNVuiybdW7rh9txbbaIU0Ma3TWOQ68BnU00hmNGh0nQlcVA6I8AYChZxkhzyeWK dhUGah+FJS1PK/92Z7KfJxLkOEmwT3rbfpJdidVqJXj+xNviZAvjkWKJTkRn4mAbyu 31qaQowZ5WXU3zyfwlZ8yIsqCG3x4IXtzrgjKcfoLEpk99+ZCIiK421t2R1gNJJlbF MV7PBuNQjLe+pLI12s/jDI2UK2+4i0zorJPux7YCl5OkL4yR6YHHlG9hHXEBgep9AX CKGVrc9nwrkVA== From: Will Deacon To: kvmarm@lists.linux.dev Cc: linux-arm-kernel@lists.infradead.org, Will Deacon , Catalin Marinas , Gavin Shan , Marc Zyngier , Mostafa Saleh , Oliver Upton , Quentin Perret , Raghavendra Rao Ananta , Ryan Roberts , Shaoqin Huang Subject: [PATCH v2 4/4] KVM: arm64: Ensure target address is granule-aligned for range TLBI Date: Wed, 27 Mar 2024 12:48:53 +0000 Message-Id: <20240327124853.11206-5-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20240327124853.11206-1-will@kernel.org> References: <20240327124853.11206-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240327_054913_462590_CE6523B5 X-CRM114-Status: GOOD ( 11.60 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When zapping a table entry in stage2_try_break_pte(), we issue range TLB invalidation for the region that was mapped by the table. However, we neglect to align the base address down to the granule size and so if we ended up reaching the table entry via a misaligned address then we will accidentally skip invalidation for some prefix of the affected address range. Align 'ctx->addr' down to the granule size when performing TLB invalidation for an unmapped table in stage2_try_break_pte(). Cc: Raghavendra Rao Ananta Cc: Gavin Shan Cc: Shaoqin Huang Cc: Quentin Perret Fixes: defc8cc7abf0 ("KVM: arm64: Invalidate the table entries upon a range") Signed-off-by: Will Deacon --- arch/arm64/kvm/hyp/pgtable.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index a40dafc43bb6..5a59ef88b646 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -843,12 +843,15 @@ static bool stage2_try_break_pte(const struct kvm_pgtable_visit_ctx *ctx, * Perform the appropriate TLB invalidation based on the * evicted pte value (if any). */ - if (kvm_pte_table(ctx->old, ctx->level)) - kvm_tlb_flush_vmid_range(mmu, ctx->addr, - kvm_granule_size(ctx->level)); - else if (kvm_pte_valid(ctx->old)) + if (kvm_pte_table(ctx->old, ctx->level)) { + u64 size = kvm_granule_size(ctx->level); + u64 addr = ALIGN_DOWN(ctx->addr, size); + + kvm_tlb_flush_vmid_range(mmu, addr, size); + } else if (kvm_pte_valid(ctx->old)) { kvm_call_hyp(__kvm_tlb_flush_vmid_ipa, mmu, ctx->addr, ctx->level); + } } if (stage2_pte_is_counted(ctx->old))