From patchwork Wed Dec 15 16:12:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12696299 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 828E1C433F5 for ; Wed, 15 Dec 2021 16:24:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=31uFeb0/regaCyz/V4oyzqc3gtYYo+VxOVzQ7i2Osvc=; b=dp1lo1F6UJCx+2aVtp1iZqxnEG e0FfIMUuLDwo1CB9lflNWX9HLqAqf+FjResveCBQvtCEpuzKU361cW2QecqUkmHRqDfhuA1CMSkRj WhqNw1QFAcpe6bhVTRnLlPe2+Yl6hSlFrBGNn4s4cGJu0yIBtn4dRq31vdPKw94jeCkIgOgCqwCUd szFCeizJHbxJB2KyIZtvKyuosSSKaKqkSlpPetn/Gpz4paK9FuTGoF3wpt23PJeCYYsu/Px4bjg2f TC7ziNcn6olNEZCEIaUv7VUhQjoY+Eq0bus4qJFz3fcp35zUbbc3O+cScmk/vJz+iMMcgk/eth/HQ EaF0X/Eg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mxX3P-001gGU-7S; Wed, 15 Dec 2021 16:22:56 +0000 Received: from mail-ed1-x549.google.com ([2a00:1450:4864:20::549]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mxWta-001byC-I7 for linux-arm-kernel@lists.infradead.org; Wed, 15 Dec 2021 16:12:48 +0000 Received: by mail-ed1-x549.google.com with SMTP id z14-20020a05640235ce00b003f7e90c32b6so2127993edc.4 for ; Wed, 15 Dec 2021 08:12:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=9UncJz09b52+rulxMEJfzwQAU0lESQ6s/aE8Lup77NM=; b=IoTXNcUh80VkspkGKEhVZPGmIgfwuEX2hfWC85I6z4BP91X7qiSyJSelcdBwSw1+z/ ENHIv3PJ0avwndy5ipE//qEnaQSgTyXUbFnvdFYPJ9WV487qISa/CYlxv4wvG7jxgNnE uklhvdl0bjLKzKwujXWYoEg5w88I7O1+ks77TzECjDlQ/E56agN6hwB7e7EumJx7le6e IkHpzTlXUq/uxD56jvtOXsS+joBN2L8q4BTBmur++c6puzfe6OlwiQeG1wD2kPS6Abtc 277ohHjDPdJuIij7EWGEkECn+V/zDZqYSe3KMBnaE+mIXvhyAxJf4mZl27WNdzb0jUEC 03xQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=9UncJz09b52+rulxMEJfzwQAU0lESQ6s/aE8Lup77NM=; b=dv07boaaK6/tchn3zi4K6xtGpeh5ggiN3wxkmDXSf38lWU8+oM0qg3fR1wG/GEYmXH n2EIRyTQKDENt7OdLkuCRLanf+YUPblK9bRCcYDxpa3QR4gfdq0JZ9gG+dTBXNGr1zuS d1Xz0seDktdNjgLqjZi3KAYjdyTAzD91xnYEwIs2evPCYbzMlD/B8PFwkh4nEFwbF3gp qB+GIdj0k557o5uo1ymJYwe2DBNxNXgLUdx1KpT0j5WZvvLP5TCHhrPD7f2zhAGj+uGz J3V22ypCdUzso0v/nqRmENhQWsH4bYkoyZAPXFrJ9HliyUYJa9MIs7zdAhKhrmaNjsvs IZQA== X-Gm-Message-State: AOAM532tkwTD0QbenEoNVVMDqvQV0o4NYB5qO2O+2EX9edH2C4Qs4yzQ j2T+b9N0wVAbwfrZfwGTOOTiiqSKiHvi X-Google-Smtp-Source: ABdhPJwCHc7fxmz5DGHLcuofkXghjPhU8hYkQXG2pB5/De5FAlnDqDB9RnWoWN1bvIdS4I1Eas5U47PGeAb+ X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:fc03:4f5b:4e9b:3ec1]) (user=qperret job=sendgmr) by 2002:a17:906:4fc4:: with SMTP id i4mr11224234ejw.81.1639584764044; Wed, 15 Dec 2021 08:12:44 -0800 (PST) Date: Wed, 15 Dec 2021 16:12:19 +0000 In-Reply-To: <20211215161232.1480836-1-qperret@google.com> Message-Id: <20211215161232.1480836-3-qperret@google.com> Mime-Version: 1.0 References: <20211215161232.1480836-1-qperret@google.com> X-Mailer: git-send-email 2.34.1.173.g76aa8bc2d0-goog Subject: [PATCH v4 02/14] KVM: arm64: Refcount hyp stage-1 pgtable pages From: Quentin Perret To: Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Catalin Marinas , Will Deacon Cc: qperret@google.com, qwandor@google.com, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, kernel-team@android.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211215_081246_674017_22BB5AB4 X-CRM114-Status: GOOD ( 13.53 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org To prepare the ground for allowing hyp stage-1 mappings to be removed at run-time, update the KVM page-table code to maintain a correct refcount using the ->{get,put}_page() function callbacks. Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/pgtable.c | 39 ++++++++++++++++++------------------ 1 file changed, 19 insertions(+), 20 deletions(-) diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index f8ceebe4982e..e50e9158fc56 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -383,21 +383,6 @@ enum kvm_pgtable_prot kvm_pgtable_hyp_pte_prot(kvm_pte_t pte) return prot; } -static bool hyp_pte_needs_update(kvm_pte_t old, kvm_pte_t new) -{ - /* - * Tolerate KVM recreating the exact same mapping, or changing software - * bits if the existing mapping was valid. - */ - if (old == new) - return false; - - if (!kvm_pte_valid(old)) - return true; - - return !WARN_ON((old ^ new) & ~KVM_PTE_LEAF_ATTR_HI_SW); -} - static bool hyp_map_walker_try_leaf(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, struct hyp_map_data *data) { @@ -407,11 +392,16 @@ static bool hyp_map_walker_try_leaf(u64 addr, u64 end, u32 level, if (!kvm_block_mapping_supported(addr, end, phys, level)) return false; + data->phys += granule; new = kvm_init_valid_leaf_pte(phys, data->attr, level); - if (hyp_pte_needs_update(old, new)) - smp_store_release(ptep, new); + if (old == new) + return true; + if (!kvm_pte_valid(old)) + data->mm_ops->get_page(ptep); + else if (WARN_ON((old ^ new) & ~KVM_PTE_LEAF_ATTR_HI_SW)) + return false; - data->phys += granule; + smp_store_release(ptep, new); return true; } @@ -433,6 +423,7 @@ static int hyp_map_walker(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, return -ENOMEM; kvm_set_table_pte(ptep, childp, mm_ops); + mm_ops->get_page(ptep); return 0; } @@ -482,8 +473,16 @@ static int hyp_free_walker(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, enum kvm_pgtable_walk_flags flag, void * const arg) { struct kvm_pgtable_mm_ops *mm_ops = arg; + kvm_pte_t pte = *ptep; + + if (!kvm_pte_valid(pte)) + return 0; + + mm_ops->put_page(ptep); + + if (kvm_pte_table(pte, level)) + mm_ops->put_page(kvm_pte_follow(pte, mm_ops)); - mm_ops->put_page((void *)kvm_pte_follow(*ptep, mm_ops)); return 0; } @@ -491,7 +490,7 @@ void kvm_pgtable_hyp_destroy(struct kvm_pgtable *pgt) { struct kvm_pgtable_walker walker = { .cb = hyp_free_walker, - .flags = KVM_PGTABLE_WALK_TABLE_POST, + .flags = KVM_PGTABLE_WALK_LEAF | KVM_PGTABLE_WALK_TABLE_POST, .arg = pgt->mm_ops, };