From patchwork Fri Apr 15 21:58:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 12815476 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 85A53C433EF for ; Fri, 15 Apr 2022 22:07:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=ZCG3anlkamIUlIpgRUWDwSZAXnikRgoQpcpEXOcTPlo=; b=Q/G+qjUpro+ISl+yMRt/N7PCDE LkQWcpOq7dW4mrb2WV/kori6CTu9LFM927B/oCRHpDcpASL2774sLg7SQ6umuPCCz0EUzAOUxmT1L ai8Fa6ezDpnsBmSrUGYijUQ5XgWdx2zEmdP8XmvYMXb8ijPdrsa8rERfS5BNcw+PrmWtcENcIoBbi bomjiBmHVcZPfBffMGSTGymmuPe5do9kIMchH3ENcSrp2H9YVToZiPOBDpztqy1dwCcDc6jxJU85c S4H43H9PF+ZQSfRWg0paIeq+PRCW3Z6Zs+6oOIPn9XMfV6lNAIJ0RKVMjU/cTNftzboDq59UlN9br 6IocL9pA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nfU58-00BV8w-QH; Fri, 15 Apr 2022 22:06:23 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nfTyO-00BS7L-1R for linux-arm-kernel@lists.infradead.org; Fri, 15 Apr 2022 21:59:26 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-2ec0490dc1bso75812657b3.5 for ; Fri, 15 Apr 2022 14:59:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=+g21tjl52haZEGL9PeLa8ZsXfugpwy0IPvVVK6CWGsg=; b=ZLsICKC3SmkNR+5tKzOjUbnQi/MrOWshrvj8ba9gD9ErKmF7bM9Ls3/73nj7AN223f 2SqgDq0mAv2Q+0NseF2Q6yJ9vAST2pHisfGt1aJ03ZR+yeXY4WCtFCLFnkoU0PMNx7bv GtmK0jCaxnoFBzIF482k+LXDT3QNYJ5NswQc3tSxLDzrxHjhk7kCcz/EML91XtB0TNBK Xm9rTMKrDR8cvHZo0rs4YCcBNImFSd/Zp6dYdWGcwa6fTHERGWIJByQqZFmWaHjjCdY8 3CDcSDTVe1yeF5TkHlWZQLhTDhaOUlu2ycQ37ChtdjS0d0bGJEyFflrGc5HuZQxrxKqT /V6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=+g21tjl52haZEGL9PeLa8ZsXfugpwy0IPvVVK6CWGsg=; b=lboEB/m0aa3yvmcS0tU2vXzuM3fyTRiztcM9mrHOGjfsDggUpTfLu/wWtFis895/8w SDa2PXurufgnIBhFmDitoxVh1LdZejiJwoJf7vn4UpgmBvTOKHOIgIVWLb6fyNWgzc0h Lvmffho0ph2LGC25MzMj+eQX3gZ3cgZwZYF64vtn+oS03/+L9kRK5qPUaT7EyzJDAxSS mQe3ewGWQ/VKBob6ulr6giRynA3ZTonXaCBzBySa/xyPqh/uGzWT+p2/WChxgOfxo8Yp jHYn+jzVzxZpgcd40IOp2pyx0IViz5kBI0nbHj8K0vhJG+EFOGxS31j1KLtnZUKr9ggk dZrw== X-Gm-Message-State: AOAM53084NrK30MRtRJaPSpXywRWrw0k6klLMZXa5Prcbpldeqtei0Ae 6Azdqk0OxCF6ljUlTdfq0t9h+0PSq6E= X-Google-Smtp-Source: ABdhPJxd3FM6U0CPN7En0pTvibRCFHGaVeXWp61fOLSmlPBx3PvTF9e7RZlcLR5fnOduP4TGiTN80GlSguU= X-Received: from oupton.c.googlers.com ([fda3:e722:ac3:cc00:2b:ff92:c0a8:404]) (user=oupton job=sendgmr) by 2002:a0d:e743:0:b0:2eb:3106:9b32 with SMTP id q64-20020a0de743000000b002eb31069b32mr940693ywe.512.1650059962733; Fri, 15 Apr 2022 14:59:22 -0700 (PDT) Date: Fri, 15 Apr 2022 21:58:59 +0000 In-Reply-To: <20220415215901.1737897-1-oupton@google.com> Message-Id: <20220415215901.1737897-16-oupton@google.com> Mime-Version: 1.0 References: <20220415215901.1737897-1-oupton@google.com> X-Mailer: git-send-email 2.36.0.rc0.470.gd361397f0d-goog Subject: [RFC PATCH 15/17] KVM: arm64: Allow parallel calls to kvm_pgtable_stage2_map() From: Oliver Upton To: kvmarm@lists.cs.columbia.edu Cc: kvm@vger.kernel.org, Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , linux-arm-kernel@lists.infradead.org, Peter Shier , Ricardo Koller , Reiji Watanabe , Paolo Bonzini , Sean Christopherson , Ben Gardon , David Matlack , Oliver Upton X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220415_145924_150826_51D58EDA X-CRM114-Status: GOOD ( 14.40 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The map walker is now appraised of how to walk the tables in parallel with another table walker. Take a parameter indicating whether or not a walk is done in parallel so as to relax the atomicity/locking requirements on ptes. Defer actually using parallel walks to a later change. Signed-off-by: Oliver Upton --- arch/arm64/include/asm/kvm_pgtable.h | 4 +++- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 2 +- arch/arm64/kvm/hyp/pgtable.c | 4 ++-- arch/arm64/kvm/mmu.c | 6 +++--- 4 files changed, 9 insertions(+), 7 deletions(-) diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h index 52e55e00f0ca..9830eea19de4 100644 --- a/arch/arm64/include/asm/kvm_pgtable.h +++ b/arch/arm64/include/asm/kvm_pgtable.h @@ -328,6 +328,8 @@ void kvm_pgtable_stage2_destroy(struct kvm_pgtable *pgt); * @prot: Permissions and attributes for the mapping. * @mc: Cache of pre-allocated and zeroed memory from which to allocate * page-table pages. + * @shared: true if multiple software walkers could be traversing the tables + * in parallel * * The offset of @addr within a page is ignored, @size is rounded-up to * the next page boundary and @phys is rounded-down to the previous page @@ -349,7 +351,7 @@ void kvm_pgtable_stage2_destroy(struct kvm_pgtable *pgt); */ int kvm_pgtable_stage2_map(struct kvm_pgtable *pgt, u64 addr, u64 size, u64 phys, enum kvm_pgtable_prot prot, - void *mc); + void *mc, bool shared); /** * kvm_pgtable_stage2_set_owner() - Unmap and annotate pages in the IPA space to diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c index 42a5f35cd819..53b172036c2a 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -251,7 +251,7 @@ static inline int __host_stage2_idmap(u64 start, u64 end, enum kvm_pgtable_prot prot) { return kvm_pgtable_stage2_map(&host_kvm.pgt, start, end - start, start, - prot, &host_s2_pool); + prot, &host_s2_pool, false); } /* diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index a9a48edba63b..20ff198ebef7 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -1119,7 +1119,7 @@ static int stage2_map_walker(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, kvm_ int kvm_pgtable_stage2_map(struct kvm_pgtable *pgt, u64 addr, u64 size, u64 phys, enum kvm_pgtable_prot prot, - void *mc) + void *mc, bool shared) { int ret; struct stage2_map_data map_data = { @@ -1144,7 +1144,7 @@ int kvm_pgtable_stage2_map(struct kvm_pgtable *pgt, u64 addr, u64 size, if (ret) return ret; - ret = kvm_pgtable_walk(pgt, addr, size, &walker, false); + ret = kvm_pgtable_walk(pgt, addr, size, &walker, shared); dsb(ishst); return ret; } diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index 6ecf37009c21..63cf18cdb978 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -832,7 +832,7 @@ int kvm_phys_addr_ioremap(struct kvm *kvm, phys_addr_t guest_ipa, write_lock(&kvm->mmu_lock); ret = kvm_pgtable_stage2_map(pgt, addr, PAGE_SIZE, pa, prot, - &cache); + &cache, false); write_unlock(&kvm->mmu_lock); if (ret) break; @@ -1326,7 +1326,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, ret = kvm_pgtable_stage2_map(pgt, fault_ipa, vma_pagesize, __pfn_to_phys(pfn), prot, - mmu_caches); + mmu_caches, true); } /* Mark the page dirty only if the fault is handled successfully */ @@ -1526,7 +1526,7 @@ bool kvm_set_spte_gfn(struct kvm *kvm, struct kvm_gfn_range *range) */ kvm_pgtable_stage2_map(kvm->arch.mmu.pgt, range->start << PAGE_SHIFT, PAGE_SIZE, __pfn_to_phys(pfn), - KVM_PGTABLE_PROT_R, NULL); + KVM_PGTABLE_PROT_R, NULL, false); return false; }