From patchwork Mon Oct 4 09:03:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12533527 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id ED23AC433F5 for ; Mon, 4 Oct 2021 09:06:15 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BC55F613A2 for ; Mon, 4 Oct 2021 09:06:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org BC55F613A2 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=xADWCAVTsrsVhA3Eviaqdsx/+cC9VmqxPo+PKmPEnJw=; b=JJx7d3qSlSobykb5pP3AD3AgYP GM2CnZ+4Fih39/EzasdRW7Ikifm2VkJvHa8QG/yyS96/pmk24Z+orZKc5yhJAOsQ1jwOeC6uZItvs KGaehbva9AU/GCYRsJff/RS/Ao84lpb3nHRG70r4PXOBKH3phnlL1d/GlFbbjlF9UueEMyv45a76E lDE4eeyWnq73Nn05gYwVnfslhMliaeHttwHK9M/Hlp5Ok/iSz3Ci21Efk25M5Q8CAybvIsGpOLpZR ko9pti2uZVaOoV3tqmKsAFRjVFWt7SHbikSQkPHK/HQ2uhVqMxkbtsxgPoqR7jeTdytIC84KdmeAu y8JIyrRw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mXJtH-005iGb-L1; Mon, 04 Oct 2021 09:04:07 +0000 Received: from mail-qk1-x749.google.com ([2607:f8b0:4864:20::749]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mXJtB-005iEi-7a for linux-arm-kernel@lists.infradead.org; Mon, 04 Oct 2021 09:04:02 +0000 Received: by mail-qk1-x749.google.com with SMTP id bk9-20020a05620a1a0900b0045df00f93a9so23345816qkb.1 for ; Mon, 04 Oct 2021 02:03:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=tHk4fNmSvcmkrUgevYD4U41Hr/4Newl1f8mzt2RPUWc=; b=myW/CpAigjPZ2o/T9HjLhLXbZVJxyS9JodGVVoxwHVcl/oDBQWttTnf0/LE1PwwT89 gKHppZ/CIgxO5msodZmnUfQBvJdqFUxdYc2bpazuwiq6sHG2QRGBrRuLvZr9ama7hWIy ZMJVyyU+o0d18Ju2reVMrpwo7YeU6Ga9AixILWDG8AnRm/y8T+cOnFt3lc+f0kCaWJdY pnU5kjI5gSCdVuPAbJjWYsFQ5TkKHM5RnmPP8Cax+hFd8/qzWgC9yJEvhQaq+CaYCveL mPuIgKFgnPJ9kaD32mXvPB1nryO78HAjInlK0UW7Z7MGbp2HyWIXaRHkD94LZXB0c6Xr UHRg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=tHk4fNmSvcmkrUgevYD4U41Hr/4Newl1f8mzt2RPUWc=; b=0GTW2JNSLHfj45JbKwV+RmHNVOhrr+96lFi0ELlPEM0BhVBMKI+4Wkv2dafQSt7Lmq c50R46wm9PjVqWK9uJ/iDqy/vrRBRatHzmFVtAwkNY76Y4i6YCxquOksXiycCsW5xa+R JfFsx98MJuCR9ucDi2qU7khkRN1Fbd3D/zZYtDKSKXFY9ks51risRCssVGIArQHSWdR3 fl3LRuuBbjbMLvot/+CTzq5rzj2a5GczrFvHUyua6TK/qZwbxVbrhBY4HQo8R/j/abtt /Ie22PYu9vBu3tmgi9BlA/3tmGNJFmjGfc1p6IahIx5AqtcXdud33k+Wp8TwsRLtTB77 +sfQ== X-Gm-Message-State: AOAM533bGNP6nlKR5MU8x/3oP2cGFZ8GI9DD8uL065exQlvtaR+LyimQ dw4+DxPQ/f/qTWobdr0SRESumpMxH8vI X-Google-Smtp-Source: ABdhPJycEaMXqOovXmHOEABw2fHm7gyvxCA3r77eIjHFoIIvQP3KmydvfcyBUYygJRYLqZ5CcnPGaqUsZunc X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:669b:5b16:60b7:a3d4]) (user=qperret job=sendgmr) by 2002:a05:6214:c47:: with SMTP id r7mr4515752qvj.12.1633338238007; Mon, 04 Oct 2021 02:03:58 -0700 (PDT) Date: Mon, 4 Oct 2021 10:03:13 +0100 In-Reply-To: <20211004090328.540941-1-qperret@google.com> Message-Id: <20211004090328.540941-2-qperret@google.com> Mime-Version: 1.0 References: <20211004090328.540941-1-qperret@google.com> X-Mailer: git-send-email 2.33.0.800.g4c38ced690-goog Subject: [PATCH 1/2] KVM: arm64: Fix host stage-2 PGD refcount From: Quentin Perret To: Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Catalin Marinas , Will Deacon , Quentin Perret , Fuad Tabba , David Brazdil , linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org Cc: kernel-team@android.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211004_020401_297436_5B10DC16 X-CRM114-Status: GOOD ( 14.84 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The KVM page-table library refcounts the pages of concatenated stage-2 PGDs individually. However, the host's stage-2 PGD is currently managed by EL2 as a single high-order compound page, which can cause the refcount of the tail pages to reach 0 when they really shouldn't, hence corrupting the page-table. Fix this by introducing a new hyp_split_page() helper in the EL2 page allocator (matching EL1's split_page() function), and make use of it from host_s2_zalloc_page(). Fixes: 1025c8c0c6ac ("KVM: arm64: Wrap the host with a stage 2") Suggested-by: Will Deacon Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/include/nvhe/gfp.h | 1 + arch/arm64/kvm/hyp/nvhe/mem_protect.c | 6 +++++- arch/arm64/kvm/hyp/nvhe/page_alloc.c | 14 ++++++++++++++ 3 files changed, 20 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kvm/hyp/include/nvhe/gfp.h b/arch/arm64/kvm/hyp/include/nvhe/gfp.h index fb0f523d1492..0a048dc06a7d 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/gfp.h +++ b/arch/arm64/kvm/hyp/include/nvhe/gfp.h @@ -24,6 +24,7 @@ struct hyp_pool { /* Allocation */ void *hyp_alloc_pages(struct hyp_pool *pool, unsigned short order); +void hyp_split_page(struct hyp_page *page); void hyp_get_page(struct hyp_pool *pool, void *addr); void hyp_put_page(struct hyp_pool *pool, void *addr); diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c index bacd493a4eac..93a79736c283 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -35,7 +35,11 @@ const u8 pkvm_hyp_id = 1; static void *host_s2_zalloc_pages_exact(size_t size) { - return hyp_alloc_pages(&host_s2_pool, get_order(size)); + void *addr = hyp_alloc_pages(&host_s2_pool, get_order(size)); + + hyp_split_page(hyp_virt_to_page(addr)); + + return addr; } static void *host_s2_zalloc_page(void *pool) diff --git a/arch/arm64/kvm/hyp/nvhe/page_alloc.c b/arch/arm64/kvm/hyp/nvhe/page_alloc.c index 41fc25bdfb34..a6e874e61a40 100644 --- a/arch/arm64/kvm/hyp/nvhe/page_alloc.c +++ b/arch/arm64/kvm/hyp/nvhe/page_alloc.c @@ -193,6 +193,20 @@ void hyp_get_page(struct hyp_pool *pool, void *addr) hyp_spin_unlock(&pool->lock); } +void hyp_split_page(struct hyp_page *p) +{ + unsigned short order = p->order; + unsigned int i; + + p->order = 0; + for (i = 1; i < (1 << order); i++) { + struct hyp_page *tail = p + i; + + tail->order = 0; + hyp_set_page_refcounted(tail); + } +} + void *hyp_alloc_pages(struct hyp_pool *pool, unsigned short order) { unsigned short i = order; From patchwork Mon Oct 4 09:03:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12533529 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 515ADC433EF for ; Mon, 4 Oct 2021 09:06:17 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 23DF96139F for ; Mon, 4 Oct 2021 09:06:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 23DF96139F Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=v/SppXpXz3LZCJ4QluIJrWxMVNLV81BnuFiNDNYuwbU=; b=blbaaVcYAUs8xN1uj8niN+DzC3 AKS7yqzpHOMwHqRxZzHYFZ1t6xIf4h+ys42Pw5U/R2bYz6MBGxN5MItg0ilBcQzKgK9tGgboFacgA g+Pci35UnJzt4zsrR8hGKPYdNpIl3qvKGuOHsAmZrbGmrJmkIbZoo3gNLpMPQYJFJ/GIk5E8PJJJ2 xMwYpnPKX/dU3Ghx+/ZkGgoDC5e6XOsUW33CeUi+qmAC1bVctTXlYU0YUegtihsjDzC5huVT1m1PE vmyfG2GMPikZm7yBcPrRhHvd/jMcHoRyPlBDWXv0NKuT/H4+LXnm/b7Ha7vP3iejdjsojfFSiREJ6 r9FwUu8w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mXJtT-005iK9-RD; Mon, 04 Oct 2021 09:04:20 +0000 Received: from mail-qk1-x74a.google.com ([2607:f8b0:4864:20::74a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mXJtP-005iIN-GO for linux-arm-kernel@lists.infradead.org; Mon, 04 Oct 2021 09:04:16 +0000 Received: by mail-qk1-x74a.google.com with SMTP id v14-20020a05620a0f0e00b0043355ed67d1so23239715qkl.7 for ; Mon, 04 Oct 2021 02:04:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=wkyuiZINwNBLMyCf2gjjqTHN+Go0IESaSFToIb6F4Fw=; b=TMjOfQHcr/Buzl+oIVWhFVyiArxE1cZ3vpKMWyLANeP2U9TxY8SYN5pke9uBTQYz+U RIB1ZRrnPZHnfENtF9YmVzL5b2dsm+8nezApYrkcELyYsTgIxCXqXNO1hgtl+kxWOHqt YYimqCXssd0CsMUdzG3aiDaPHuW0jO1FRRSQN9CmPVoQKROHZ0soqL+iSp+ePqyxkxJ4 aBBfUHuRZy/2zMpYKVQ68s/iuanxLmWb85P3CroP0EVlbrCmCoq91Okjk9cJ5jEAWbgt MbAZGlaksbCy17axlm5ayqNk85VH9Kb4lIdjYPdiad/U6IoxI4yhM1WcNa0iTDKHwSHP B1ow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=wkyuiZINwNBLMyCf2gjjqTHN+Go0IESaSFToIb6F4Fw=; b=RrZPRgGLOtgt/0T2Lj/+5BkWLiIgmPwdVVDtorcg757rr8xFrzzabV4WoCqq2m8bSY lbz3x7MPK23ZHjvUk0emUWI7WuyVqftCtBJNbRPjtJqsKpYO3EcQn0UwajQJ1XZoq7Q4 je90QltuAXMSYDsFzTM78ZL5ZJHXERu9wybzAO8K6XIJSMctJWJgpeX00LtR5CPGcnoz qQXVdZpV7xOfaWLcGJaJTQyAIgqdtwQ4FghXhBZy55CyZY7hP5azoGT4R2+VPgBDO7aK 5hTF3zEUOgjC3IsrpwUxHpv+q+IYWmwl4Qy/jXztHqHjdgBJvrg5N7cePWinDuYZblx/ yNOA== X-Gm-Message-State: AOAM5308hzMSlda0IJNuJE6AJEpRexo8DWJcOOJ82xXC11myTS7UG+1i lz2Jy9U9HfFwLT6Y+sGLA3xZtU/yQegL X-Google-Smtp-Source: ABdhPJz17Z7ue1W7hq3hrv29ed/PiZ+fLhUfwKMW5antWu0eaJWy3bZ4lxRPpDIEYNfIk8cbjwcuEEkEbV4g X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:669b:5b16:60b7:a3d4]) (user=qperret job=sendgmr) by 2002:a0c:85e6:: with SMTP id o93mr21326866qva.16.1633338253264; Mon, 04 Oct 2021 02:04:13 -0700 (PDT) Date: Mon, 4 Oct 2021 10:03:14 +0100 In-Reply-To: <20211004090328.540941-1-qperret@google.com> Message-Id: <20211004090328.540941-3-qperret@google.com> Mime-Version: 1.0 References: <20211004090328.540941-1-qperret@google.com> X-Mailer: git-send-email 2.33.0.800.g4c38ced690-goog Subject: [PATCH 2/2] KVM: arm64: Report corrupted refcount at EL2 From: Quentin Perret To: Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Catalin Marinas , Will Deacon , Quentin Perret , Fuad Tabba , David Brazdil , linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org Cc: kernel-team@android.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211004_020415_573504_13688516 X-CRM114-Status: UNSURE ( 9.18 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Some of the refcount manipulation helpers used at EL2 are instrumented to catch a corrupt state, but not all of them are treated equally. Let's make things more consistent by instrumenting hyp_page_ref_dec_and_test() as well. Suggested-by: Will Deacon Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/nvhe/page_alloc.c | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/arm64/kvm/hyp/nvhe/page_alloc.c b/arch/arm64/kvm/hyp/nvhe/page_alloc.c index a6e874e61a40..0bd7701ad1df 100644 --- a/arch/arm64/kvm/hyp/nvhe/page_alloc.c +++ b/arch/arm64/kvm/hyp/nvhe/page_alloc.c @@ -152,6 +152,7 @@ static inline void hyp_page_ref_inc(struct hyp_page *p) static inline int hyp_page_ref_dec_and_test(struct hyp_page *p) { + BUG_ON(!p->refcount); p->refcount--; return (p->refcount == 0); }