From patchwork Tue Oct 19 12:12:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12569747 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 52E25C433F5 for ; Tue, 19 Oct 2021 12:18:16 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 225D061260 for ; Tue, 19 Oct 2021 12:18:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 225D061260 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=05vMcv582X3cxTD/ljdspUW1NV9h5PXPW5ZAgRS1Ixs=; b=S+ogqfFz75YIX0UMIjtHuMoRgy Etbg0u84yF3kx9by69gXSd7gcoyWPTiIasOwEZi8SPzWCH29r9KqIJk2zgLgqHFDaEEf+wJS4abnN 5tLrMpWgOjsul3pxJy40Zfi5/KdZ04nWc7yF9B15nB990PX7DGEBcsufAyD74CLV8Cr55K0nDtF0M oIagWlmv9zROwPPIaat3utiPw4/MRdvnysOkt12nzxQYq3ksUfDm3mQAtNOIimKz/OhglYxPhyGfE 4ZXFlbhtt66vxIoiRKGp04xYljq7CtxDQY7E1Bpqet4rDZUp++51amLnkFrsrgRurrQ5JVWpff+gG 3g35OOxA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mco2v-0016r7-7Q; Tue, 19 Oct 2021 12:16:45 +0000 Received: from mail-wm1-x34a.google.com ([2a00:1450:4864:20::34a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mcnzj-0015H1-G1 for linux-arm-kernel@lists.infradead.org; Tue, 19 Oct 2021 12:13:28 +0000 Received: by mail-wm1-x34a.google.com with SMTP id v18-20020a7bcb52000000b00322fea1d5b7so1065151wmj.9 for ; Tue, 19 Oct 2021 05:13:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=FltoTPFX6jcmVGDt6TNVLWqV+gbyvD26Qibq15L3bkE=; b=jlG1oEJu0XALf3UMY9dDzZLKa57GGILI3CeQZbwtRWFXxRmxyJ4hrufsS2cJNR8YIr JNvbgekimwjNmNstdhzrPkwKZqgPJPeSo0J71SXvKrgiUXiuiHLmruUHs8rcc/indAiX 26CLj+txDLSji5YnWMA6yzm7AtzfIVGTatEphI+y5c26xa90hXLAlmYFL//1dTQQOyPw HMCbzOhB4xv2VomYIlo14xWYxcD4vbtMZhrxk2yL7qWC5KwsBpwln8lsNwynvPrxIQHB Xiss9bsk7TSpL1SH0P53UFrzTXqsorbicgZEDet4/UyMoolHEu3NjoWnHFqaYu3ykP1r VGRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=FltoTPFX6jcmVGDt6TNVLWqV+gbyvD26Qibq15L3bkE=; b=srg/3SayQJerVn0Ds/wNHtMHCLSOD9gOxQqAPBhIVmFnlC+fyUVwIq5hKlMYB+kqzn doZdAACSNn1WzYPFtUvCEM+SYSUlSEc/tZMnRaS94NA2Te+Y7Nf6TrKsOs9HFls4UIPT 150BGvjTpS1lkMs3m5klwTQjkf/Aa6hCg2eV6Em9VpIY3Jr9b1qtFV02vv5dOu2L7SO1 68wTJ0MtsBpNTR9uK6Sd6zHBHy76MpDtImm1YLYVsTvpPq3dVd+fRuTrD1WOtMP9r5Fh FhvDNjELhImd3GzzMWbqvYiQKZ2Uoksnrbnhpg1elb0WN7mZrmNrUEFwDXpyyWETGDOf 8hfA== X-Gm-Message-State: AOAM533PKSEYboNJJvc7AtaPkQ6CLbFZoVmM9Ii4fIGP/dneuq/apXa9 toJ4rmfHHAVeQctZASRj9oUcM+beCoa9 X-Google-Smtp-Source: ABdhPJzHTfoE95BxQfr07fXGEheE3lVwrZ1OysKwSSC2rt9N5CO+2igZJDqU/WMlnTkRLHPLM8olUAyj+sVU X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:59ca:401f:83a8:de6d]) (user=qperret job=sendgmr) by 2002:a1c:2309:: with SMTP id j9mr5432829wmj.189.1634645605661; Tue, 19 Oct 2021 05:13:25 -0700 (PDT) Date: Tue, 19 Oct 2021 13:12:57 +0100 In-Reply-To: <20211019121304.2732332-1-qperret@google.com> Message-Id: <20211019121304.2732332-9-qperret@google.com> Mime-Version: 1.0 References: <20211019121304.2732332-1-qperret@google.com> X-Mailer: git-send-email 2.33.0.1079.g6e70778dc9-goog Subject: [PATCH v2 08/15] KVM: arm64: pkvm: Refcount the pages shared with EL2 From: Quentin Perret To: Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Catalin Marinas , Will Deacon , Fuad Tabba , David Brazdil , Andrew Walbran Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, kernel-team@android.com, qperret@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211019_051327_580620_C2F19D92 X-CRM114-Status: GOOD ( 16.25 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In order to simplify the page tracking infrastructure at EL2 in nVHE protected mode, move the responsibility of refcounting pages that are shared multiple times on the host. In order to do so, let's create a red-black tree tracking all the PFNs that have been shared, along with a refcount. Signed-off-by: Quentin Perret --- arch/arm64/kvm/mmu.c | 78 ++++++++++++++++++++++++++++++++++++++------ 1 file changed, 68 insertions(+), 10 deletions(-) diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index 0cc4b295e525..f0c16ed95974 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -284,30 +284,88 @@ static phys_addr_t kvm_kaddr_to_phys(void *kaddr) } } -static int pkvm_share_hyp(phys_addr_t start, phys_addr_t end) +struct hyp_shared_pfn { + u64 pfn; + int count; + struct rb_node node; +}; + +static DEFINE_MUTEX(hyp_shared_pfns_lock); +static struct rb_root hyp_shared_pfns = RB_ROOT; + +static struct hyp_shared_pfn *find_shared_pfn(u64 pfn, struct rb_node ***node, + struct rb_node **parent) { - phys_addr_t addr; - int ret; + struct hyp_shared_pfn *this; + + *node = &hyp_shared_pfns.rb_node; + *parent = NULL; + while (**node) { + this = container_of(**node, struct hyp_shared_pfn, node); + *parent = **node; + if (this->pfn < pfn) + *node = &((**node)->rb_left); + else if (this->pfn > pfn) + *node = &((**node)->rb_right); + else + return this; + } - for (addr = ALIGN_DOWN(start, PAGE_SIZE); addr < end; addr += PAGE_SIZE) { - ret = kvm_call_hyp_nvhe(__pkvm_host_share_hyp, - __phys_to_pfn(addr)); - if (ret) - return ret; + return NULL; +} + +static int share_pfn_hyp(u64 pfn) +{ + struct rb_node **node, *parent; + struct hyp_shared_pfn *this; + int ret = 0; + + mutex_lock(&hyp_shared_pfns_lock); + this = find_shared_pfn(pfn, &node, &parent); + if (this) { + this->count++; + goto unlock; } - return 0; + this = kzalloc(sizeof(*this), GFP_KERNEL); + if (!this) { + ret = -ENOMEM; + goto unlock; + } + + this->pfn = pfn; + this->count = 1; + rb_link_node(&this->node, parent, node); + rb_insert_color(&this->node, &hyp_shared_pfns); + ret = kvm_call_hyp_nvhe(__pkvm_host_share_hyp, pfn, 1); +unlock: + mutex_unlock(&hyp_shared_pfns_lock); + + return ret; } int kvm_share_hyp(void *from, void *to) { + phys_addr_t start, end, cur; + u64 pfn; + int ret; + if (is_kernel_in_hyp_mode()) return 0; if (kvm_host_owns_hyp_mappings()) return create_hyp_mappings(from, to, PAGE_HYP); - return pkvm_share_hyp(kvm_kaddr_to_phys(from), kvm_kaddr_to_phys(to)); + start = ALIGN_DOWN(kvm_kaddr_to_phys(from), PAGE_SIZE); + end = PAGE_ALIGN(kvm_kaddr_to_phys(to)); + for (cur = start; cur < end; cur += PAGE_SIZE) { + pfn = __phys_to_pfn(cur); + ret = share_pfn_hyp(pfn); + if (ret) + return ret; + } + + return 0; } /**