From patchwork Tue Oct 5 09:01:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12536005 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0020CC433EF for ; Tue, 5 Oct 2021 09:05:52 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BD35F61371 for ; Tue, 5 Oct 2021 09:05:52 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org BD35F61371 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=cL7rq+okM3OFSIF6QiutKzIks1uwcClj/6g419hIw4Q=; b=PveELolIgJvs2J7PBmfJpcuD7p Y2KZch92OTuuszIR9Bidfbpdwy/fQ1MKrnImbgArqNSK2PIFITRQ//c8NElwHb7HXwcdmWFCwxqlh dSinrT8UJb8YvVV1A2X1DlUrEHBjQzaPqLsFxkfBflGOxdbKuxSjB7jxE4W4c+6XWZcrOFvkbY4qE fcZBHUOsh9/cICf1uTAm55+JCgi/OhEZ2+Lk7hTySfTN42xzODvOfvzIJoPuISAYyU7McXIzQI82r VVzR+KPYaz48g/84T20nZLyBEJjjVooAfkXO5wssrSNpa6sR8FPX3O+S2XCDpoDhlhpWC6kcw7xu3 HqaMWilw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mXgMD-009dk9-UD; Tue, 05 Oct 2021 09:03:30 +0000 Received: from mail-wr1-x449.google.com ([2a00:1450:4864:20::449]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mXgLr-009dab-Ul for linux-arm-kernel@lists.infradead.org; Tue, 05 Oct 2021 09:03:09 +0000 Received: by mail-wr1-x449.google.com with SMTP id j19-20020adfb313000000b00160a9de13b3so1859587wrd.8 for ; Tue, 05 Oct 2021 02:03:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=Zg+gUiu3ox2wHV7TW+7r/xAJBwKqffZp8cWRWiJTXrY=; b=ethTiy3lTzcSneIbLwawi+5olik16z23jgqCFT9E6S+igJ69OtJ1TPMCM3LUeMuDWh kwcRJn93q32+YoPQuqYwcmM2DdY9Fe9IYJKgK+GXLbcXpWzMDJRW3LbNqRmKiJUsc6DC CebD0o+l0nqeRevVQD1bAdXZQHBHCWfG8Rcoy58tJcFsFAxUPA0SJI81RMitxFI9xF+C YxiAvhisoF3VIur01ICFsq3MrvN2eqrXvL8fZNMl/vG0AqdzcjPWDTaTqFZDJuk+h9ab QCMcSvBsSTabymd8vvOKuV6YemDU3oBdSpPn6xfZo8O6vSq3YJh46dvt7fuELrY2dzVh x/ig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=Zg+gUiu3ox2wHV7TW+7r/xAJBwKqffZp8cWRWiJTXrY=; b=tfaEWicNhzDmDZciXf6XZOqq8KNThg8KcrMqbfguTrO7rdnVcxrY7ofkfLcUq0f3ve 3b2iZQWksqm5fWESrHOOPH9X6wAKWsivVHsgyz7MZznqgakFcQZRnwonJHlq9y+6j1sw iQ8bqcS1l6WCjnbXGD82Xw8cOKHtA91bYUKGa7bcRmlwqY9ll7SQfF7UGsSMgRb4n3E8 6fL8zC19vtCEzJ5yddbbHJRbq2XasYsZpK69tlmePoENs7pghYyDwq9hIs45Nh+jEpam DNtU/CyBBf4xUeiHUlFhKygJKLZ6Sf8SY4eS+j1i4aL0hIsBksOYMkBa+FYwR6zdkFPD sd2A== X-Gm-Message-State: AOAM5306RjF0d2BQcS6cxrXKPrwOfZ5af5AIIZouAXNjegU+lrbEz74e ciRY6HgwIEa5Sn/xuFdXDyNJ1in4oPfG X-Google-Smtp-Source: ABdhPJxUSKr1FTBsupYcr01qanV2kSzxpxGMspE323G2ZVQjyutuN9ik2CBJcXODTdTUv+X78VCL5yV8AtM/ X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:5700:9128:3106:b389]) (user=qperret job=sendgmr) by 2002:adf:a45e:: with SMTP id e30mr18268872wra.269.1633424585068; Tue, 05 Oct 2021 02:03:05 -0700 (PDT) Date: Tue, 5 Oct 2021 10:01:41 +0100 In-Reply-To: <20211005090155.734578-1-qperret@google.com> Message-Id: <20211005090155.734578-5-qperret@google.com> Mime-Version: 1.0 References: <20211005090155.734578-1-qperret@google.com> X-Mailer: git-send-email 2.33.0.800.g4c38ced690-goog Subject: [PATCH v2 1/2] KVM: arm64: Fix host stage-2 PGD refcount From: Quentin Perret To: Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Catalin Marinas , Will Deacon , Quentin Perret , Fuad Tabba , David Brazdil , linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org Cc: kernel-team@android.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211005_020308_050698_50CDA92C X-CRM114-Status: GOOD ( 16.74 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The KVM page-table library refcounts the pages of concatenated stage-2 PGDs individually. However, when running KVM in protected mode, the host's stage-2 PGD is currently managed by EL2 as a single high-order compound page, which can cause the refcount of the tail pages to reach 0 when they shouldn't, hence corrupting the page-table. Fix this by introducing a new hyp_split_page() helper in the EL2 page allocator (matching the kernel's split_page() function), and make use of it from host_s2_zalloc_pages_exact(). Fixes: 1025c8c0c6ac ("KVM: arm64: Wrap the host with a stage 2") Acked-by: Will Deacon Suggested-by: Will Deacon Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/include/nvhe/gfp.h | 1 + arch/arm64/kvm/hyp/nvhe/mem_protect.c | 13 ++++++++++++- arch/arm64/kvm/hyp/nvhe/page_alloc.c | 14 ++++++++++++++ 3 files changed, 27 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kvm/hyp/include/nvhe/gfp.h b/arch/arm64/kvm/hyp/include/nvhe/gfp.h index fb0f523d1492..0a048dc06a7d 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/gfp.h +++ b/arch/arm64/kvm/hyp/include/nvhe/gfp.h @@ -24,6 +24,7 @@ struct hyp_pool { /* Allocation */ void *hyp_alloc_pages(struct hyp_pool *pool, unsigned short order); +void hyp_split_page(struct hyp_page *page); void hyp_get_page(struct hyp_pool *pool, void *addr); void hyp_put_page(struct hyp_pool *pool, void *addr); diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c index bacd493a4eac..34eeb524b686 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -35,7 +35,18 @@ const u8 pkvm_hyp_id = 1; static void *host_s2_zalloc_pages_exact(size_t size) { - return hyp_alloc_pages(&host_s2_pool, get_order(size)); + void *addr = hyp_alloc_pages(&host_s2_pool, get_order(size)); + + hyp_split_page(hyp_virt_to_page(addr)); + + /* + * The size of concatenated PGDs is always a power of two of PAGE_SIZE, + * so there should be no need to free any of the tail pages to make the + * allocation exact. + */ + WARN_ON(size != (PAGE_SIZE << get_order(size))); + + return addr; } static void *host_s2_zalloc_page(void *pool) diff --git a/arch/arm64/kvm/hyp/nvhe/page_alloc.c b/arch/arm64/kvm/hyp/nvhe/page_alloc.c index 41fc25bdfb34..a6e874e61a40 100644 --- a/arch/arm64/kvm/hyp/nvhe/page_alloc.c +++ b/arch/arm64/kvm/hyp/nvhe/page_alloc.c @@ -193,6 +193,20 @@ void hyp_get_page(struct hyp_pool *pool, void *addr) hyp_spin_unlock(&pool->lock); } +void hyp_split_page(struct hyp_page *p) +{ + unsigned short order = p->order; + unsigned int i; + + p->order = 0; + for (i = 1; i < (1 << order); i++) { + struct hyp_page *tail = p + i; + + tail->order = 0; + hyp_set_page_refcounted(tail); + } +} + void *hyp_alloc_pages(struct hyp_pool *pool, unsigned short order) { unsigned short i = order;